• Post Reply Bookmark Topic Watch Topic
  • New Topic

Reading multiple URLs in loop, but it goes slow after some time  RSS feed

 
Zidan Kahn
Greenhorn
Posts: 3
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hi,

I using following code to read multiple URLs in loop. I have scheduled my program to run at every 30 Minutes, for whole day.




But some time is gone very slow in java, but these Urls load in browser fast. Now there are only 10 URLs in loop.
But I have to implement 50-100 URLs read in loop.

Kindly help me out to improve the speed read Urls in loop. What else I can do.

Please give me solutions ASAP.

Thanks & Regards
Zidan
 
Carey Brown
Saloon Keeper
Posts: 3323
46
Eclipse IDE Firefox Browser Java MySQL Database VI Editor Windows
  • Likes 1
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
One way you could improve the speed is by making your reader and parser into a thread, that way you can read several sites in parallel. Note that you may want a way to limit the number of threads active at any given point in time.
 
Tony Docherty
Bartender
Posts: 3271
82
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Carey Brown wrote:One way you could improve the speed is by making your reader and parser into a thread, that way you can read several sites in parallel. Note that you may want a way to limit the number of threads active at any given point in time.

Good idea, the way to do this is use java.util.concurrent.Executers to create a suitable ExecutorService.
 
Campbell Ritchie
Marshal
Posts: 56562
172
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
And welcome to the Ranch
 
Carey Brown
Saloon Keeper
Posts: 3323
46
Eclipse IDE Firefox Browser Java MySQL Database VI Editor Windows
  • Likes 2
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Also, this can be very inefficient for long pages

Use a StringBuilder to concatenate the lines.
 
Stefan Evans
Bartender
Posts: 1837
10
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
At a glance, it appears you are doing string operations on XML.


I might suggest you try parsing the XML with an XML parser rather than hacking it with a string.
Or running a simple transformation via XSLT.

There are better and more efficient tools than what you are using here. And why re-invent the square wheel if you don't have to?

 
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
Boost this thread!