• Post Reply
  • Bookmark Topic Watch Topic
  • New Topic

how to make program log in to get access to subscriber only content?

 
Michael Tan
Greenhorn
Posts: 6
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
i am a paid subscriber of a website, and i want to write a program that automatically downloads the subscriber-only content and make sense of it automatically (the scale is too large to do it manually).

however, im stuck at the login. how do i make the java program log in to the site so that the program will be able to download the subscriber only content to make sense of it all automatically?



thanks.
 
Paul Clapham
Sheriff
Posts: 21416
33
Eclipse IDE Firefox Browser MySQL Database
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
First step: go to the site's Terms of Use section. Read through it to make sure that the site doesn't prohibit this kind of access. (Many of them do.) If the site does prohibit access by programs, then your problem is solved, i.e. you aren't allowed to do it.
 
Michael Tan
Greenhorn
Posts: 6
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
why do people reply without actually replying to the topic?

ego?
 
Paul Clapham
Sheriff
Posts: 21416
33
Eclipse IDE Firefox Browser MySQL Database
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
That was a reply to the topic.

Do you think it's unimportant whether the site owner permits you to write a bot to download the contents of the site? We here at the Ranch don't think it's unimportant. In fact we don't support this kind of activity at all. And if you're okay with breaking contracts, then perhaps the "ego" problem is yours.

Now it might be that the owner of that site doesn't mind you downloading the site's contents. But it's your responsibility to find that out first.
 
Steve Luke
Bartender
Posts: 4181
22
IntelliJ IDE Java Python
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
One other benefit of reading the ToS might be you find an API is available for the purpose you wish to use it for. For example if the content is truly too large for manual processing, the site owner may understand this and provide a programmatic means for you to get at it - without the need to screen-scrape.

So do some research (i.e. contact the web site) and see if there is something available to make your job easier.
 
Darryl Burke
Bartender
Posts: 5148
11
Java Netbeans IDE Opera
 
  • Post Reply
  • Bookmark Topic Watch Topic
  • New Topic