Win a copy of Cross-Platform Desktop Applications: Using Node, Electron, and NW.js this week in the JavaScript forum!
  • Post Reply Bookmark Topic Watch Topic
  • New Topic

How can I parse a webpage for information?  RSS feed

 
Chris Stewart
Ranch Hand
Posts: 184
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I'm trying to parse a webpage ( http://setiathome.ssl.berkeley.edu/fcgi-bin/fcgi?email=CompiledMonkey@attbi.com&cmd=user_stats_new ) in my servlet to return how many units I've completed. I'd also like to do this for other websites (weather and such). How should I go about doing this? Thanks for any advice.
[ October 20, 2002: Message edited by: James Davis ]
 
Mike Curwen
Ranch Hand
Posts: 3695
IntelliJ IDE Java Ubuntu
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I was going to reply "try URLConnection", but then I decided to try it myself (for the first time ever), and it is a bit harder than I thought. Finally found the code on java.sun.com here:

http://java.sun.com/docs/books/tutorial/networking/urls/readingURL.html

I've modified that code slightly because of firewalls and I've used a StringBuffer so that it can all be parsed later. You might modify your code to only readLine() until you find the data you're looking for, and then exit that loop and close the connection (no need to read in the entire page if you only want something appearing in the first 20% of the page).


[ October 21, 2002: Message edited by: Mike Curwen ]
 
Chris Stewart
Ranch Hand
Posts: 184
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Thanks! It works great.
 
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
Boost this thread!