• Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other Pie Elite all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Ron McLeod
  • Rob Spoor
  • Tim Cooke
  • Junilu Lacar
Sheriffs:
  • Henry Wong
  • Liutauras Vilda
  • Jeanne Boyarsky
Saloon Keepers:
  • Jesse Silverman
  • Tim Holloway
  • Stephan van Hulst
  • Tim Moores
  • Carey Brown
Bartenders:
  • Al Hobbs
  • Mikalai Zaikin
  • Piet Souris

extract entire content from url till the end of that url

 
Ranch Hand
Posts: 146
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

I want to get entire content from particular url..i mean all the content till the bottom of that..i used this code to read the url content...but it takes only limited before loading....i want to get the content even after loading (I.e get all content from url without any buffering)
 
Master Rancher
Posts: 4465
38
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Also posted at: http://www.dreamincode.net/forums/topic/398259-get-entire-content-from-a-url-using-java/page__pid__2286897__st__0&#entry2286897
 
surya preethaaa
Ranch Hand
Posts: 146
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
i dint get solution there....thats y....i m posted here
 
Saloon Keeper
Posts: 7093
165
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
The point is, you shouldn't post on multiple sites without mentioning it: BeForthrightWhenCrossPostingToOtherSites
 
Rancher
Posts: 4801
50
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Unless you can figure out a way for JSoup to represent scrolling to the end to force the Javascript to fire off whatever the request is to get more data then, as with the Twitter restriction on how much data is returned via their API, you are likely out of luck.

They do this for a reason, and that is to prevent people just trawling through months of data and hammering their servers.

There may be a paid for service that would let you do what you want, but I wouldn't bet on it.
 
surya preethaaa
Ranch Hand
Posts: 146
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
other than twitter...generally speaking...is there any methods to get whole content without loading or buffering
 
Dave Tolls
Rancher
Posts: 4801
50
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
If you are talking about how Twitter does its partial loading, then it requires that the tool you are using allows you to trigger the event that causes the Javascript call, and that is likely different for each of these sorts of pages.
 
surya preethaaa
Ranch Hand
Posts: 146
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
thanks for your response....i try to do something
 
You showed up just in time for the waffles! And this tiny ad:
Building a Better World in your Backyard by Paul Wheaton and Shawn Klassen-Koop
https://coderanch.com/wiki/718759/books/Building-World-Backyard-Paul-Wheaton
reply
    Bookmark Topic Watch Topic
  • New Topic