Win a copy of Head First Agile this week in the Agile forum!
  • Post Reply Bookmark Topic Watch Topic
  • New Topic

How to fetch data from a website to my j2ee application?  RSS feed

 
SunilK Chauhan
Ranch Hand
Posts: 60
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hello sir,

I have a task to make a page which will displays me data which were fetched from a Web page. There is a authentication required to go for a Web page first (I. E. When we open the website we need to enter user name and password and then there an option to go for a page which contains small table with data which I have to fetch to my application).

This data changes at every 1 hour of interval. So, I think I have to make the program which will work with timer. I didn't get any proper solution for that. Please help me here to make this possible.
 
Bear Bibeault
Author and ninkuma
Marshal
Posts: 66208
151
IntelliJ IDE Java jQuery Mac Mac OS X
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Do you have the site's permission to screen-scrape their data? If the site wishes to export their data are you sure there isn't a published API for that?
 
SunilK Chauhan
Ranch Hand
Posts: 60
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Ya I am sure that I have permission to screen scrap data from website. But i am not sure about the published api used over it. Because it's a task to make users effort easy to get Web data to the local j2ee application directly by a single click. We do not gonna talk with the website authority anymore for that.

Here I have one major issue to solve is authentication page. Because the web page what I want is just after the authentication only.

Please guide me.
 
William Brogden
Author and all-around good cowpoke
Rancher
Posts: 13078
6
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
A popular toolkit for performing HTTP - browser like functions is open source.

Thats what I would use if I had your problem.

Bill
 
SunilK Chauhan
Ranch Hand
Posts: 60
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
In that page we need to give input. And than after we can get data from that web page.

Let me describe it into details. I have one URL for that web-page, when we open the link, We need to select date criteria and than after we can see the data which i want to fetch.

So, i am asking that how to give that input indirectly using my java application, because i want that data to store into my database from that web-page.

Thanks and regards,
Sunil
 
William Brogden
Author and all-around good cowpoke
Rancher
Posts: 13078
6
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I suggest you use a tool (or tools) that let you capture the full text of the requests and responses of a normal interaction with this web site.

That information will help you understand what your Java application will have to generate and read.

Popular tools include TCPMON and SOAPui - your browser "view source" will also help.

Bill
 
Don't get me started about those stupid light bulbs.
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
Boost this thread!