• Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other Pie Elite all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Jeanne Boyarsky
  • Ron McLeod
  • Paul Clapham
  • Liutauras Vilda
Sheriffs:
  • paul wheaton
  • Rob Spoor
  • Devaka Cooray
Saloon Keepers:
  • Stephan van Hulst
  • Tim Holloway
  • Carey Brown
  • Frits Walraven
  • Tim Moores
Bartenders:
  • Mikalai Zaikin

Reading multiple URLs in loop, but it goes slow after some time

 
Greenhorn
Posts: 3
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Hi,

I using following code to read multiple URLs in loop. I have scheduled my program to run at every 30 Minutes, for whole day.




But some time is gone very slow in java, but these Urls load in browser fast. Now there are only 10 URLs in loop.
But I have to implement 50-100 URLs read in loop.

Kindly help me out to improve the speed read Urls in loop. What else I can do.

Please give me solutions ASAP.

Thanks & Regards
Zidan
 
Saloon Keeper
Posts: 10705
86
Eclipse IDE Firefox Browser MySQL Database VI Editor Java Windows ChatGPT
  • Likes 1
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
One way you could improve the speed is by making your reader and parser into a thread, that way you can read several sites in parallel. Note that you may want a way to limit the number of threads active at any given point in time.
 
Bartender
Posts: 3323
86
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Carey Brown wrote:One way you could improve the speed is by making your reader and parser into a thread, that way you can read several sites in parallel. Note that you may want a way to limit the number of threads active at any given point in time.


Good idea, the way to do this is use java.util.concurrent.Executers to create a suitable ExecutorService.
 
Marshal
Posts: 79178
377
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
And welcome to the Ranch
 
Carey Brown
Saloon Keeper
Posts: 10705
86
Eclipse IDE Firefox Browser MySQL Database VI Editor Java Windows ChatGPT
  • Likes 2
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Also, this can be very inefficient for long pages

Use a StringBuilder to concatenate the lines.
 
Bartender
Posts: 1845
10
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
At a glance, it appears you are doing string operations on XML.


I might suggest you try parsing the XML with an XML parser rather than hacking it with a string.
Or running a simple transformation via XSLT.

There are better and more efficient tools than what you are using here. And why re-invent the square wheel if you don't have to?

 
With a little knowledge, a cast iron skillet is non-stick and lasts a lifetime.
reply
    Bookmark Topic Watch Topic
  • New Topic