Getting web pages from an app.

[Bob Burnquist]

My question is, what is the best way to call a number of web pages from a java class.

I put this in the servlets section but then realisied my code does not need to be a servlet as this application will be called by a cron job.

It needs to go through a list and make requests to a lot of websites so that the urls can be cached server-side. Im not interested in rendering any of these websites, as long as the pages are sent from the server.

This easy to do through sockets?

Thanks loads.
[ March 10, 2006: Message edited by: Sebastian Patten ]

[Vishnu Prakash]

Looks more appropriate to in servlet forum.

[Stan James]

If you just want a Java client that can pull pages from web sites and write them to disk it's pretty simple to do. Look into the URL class which can get you a URLConnection. From there you get the stream and read whatever is at that address. For HTML you can probably use a buffered reader with no problem. For images or other binary files stick with binary streams.

Take a look at the doc and see what you can put together. If you get stuck, show us some code and we'll try to get you going again.

Keep it fun!