• Post Reply Bookmark Topic Watch Topic
  • New Topic

Java code for automated scraping  RSS feed

 
Eugene Pio Murphy
Greenhorn
Posts: 1
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Greetings all,

I have a favour to ask of the altruistic java enthusiasts on here.
I have a project demonstration due soon, and to convert my project from a proof of concept, to a working version i need to create a class that will scrape, parse and import to a MySql table, free to access data from a table on an airline site.

The code needs to be automated to perform the function every 45 minutes.

Please assist me with any relevant information if you can, unfortunately i would consider myself a novice java enthusiast.

Yours
[ March 05, 2007: Message edited by: Bear Bibeault ]
 
Bear Bibeault
Author and ninkuma
Marshal
Posts: 65833
134
IntelliJ IDE Java jQuery Mac Mac OS X
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Welcome to the Ranch. Please take note of the forums that are available, and carefully choose the appropriate forum for your posts. This forum is dedicated to JSP.

Your post has been moved to a mrore appropriate forum.
 
Ulf Dittmer
Rancher
Posts: 42970
73
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Two basic ways to go about this would be to either download the URL using the HttpURLConnection class, and then using regular expressions to extract the content you're looking for, or use a library like HtmlUnit/HttpUnit/jWebUnit to download it and give you what's basically a DOM representation of the page. I'd advise to go with the latter, but both ways are probably a bit tricky if you're just starting out with Java.
 
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
Boost this thread!