Forum-Reader-Package

[Sascha Henning]

Hi.

I try to find a java package, that helps me to automatically read a forum in the net (like this). I'd like to iterate over the postings and write them into files. Is there any (rudimental) solution for this?

Thanks,

Sascha

[Valentin Crettaz]

Moving this thread to the General Computing forum. Please continue this discussion there. Thank you

[Ernest Friedman-Hill]

Note that doing something like this is generally considered a hostile act by most online forums, and you'd risk having your IP address banned. I know Slashdot, for example, can and has banned people for this. You'd be consuming more than your fair share of bandwidth, and inconveniencing other users. Please don't pursue this project.

[Sascha Henning]

@Ernest Friedman-Hill

Hi.

Do you mean my first initial posting? If it is so, then I don't understand what you mean. If posted my question to the category i thougth it is the right one. What is my fault?

[Jeroen Wenting]

leeching websites is generally considered poor taste.
You're eating up bandwidth fast, costing the site operators money and resources and inconveniencing other users.

When discovered you may be permanently banned.

[Adrian Yan]

I think what Ernest meant was that the application you are asking for is often consider harmful to alot of board on the net. Your application will essentail crawl thru the site/board and check for new posts. Some sites admins out there consider this as hacking, since your application if not well written, would takes up alot of bandwidth from the site. Causing inconvenience to other users as well.
Slashdot do ban that pratice. However, alot of sites do provide RSS feed that allows your to connect your application to and update you everytime when some new is posted.

[Ernest Friedman-Hill]

Originally posted by Sascha Henning:

Do you mean my first initial posting?

No, I meant what Jeroen said. Attempting to vacuum all the content off a forum site will likely get your IP address banned from the forum.

[Sascha Henning]

Ok, i understand your demur. Its not my intention to cause any trouble.

I need many articles to do information retrieval on them. Alternatively i can use newsgrouparticles. When i'm not wrong ones more, it is common to load down newsgrouparticles to view them.
So my question is now: Are there any libaries (best when open source)in java, which can be used to load down such articles, to save them and use them, to do ir on them?

Thanks,

Sascha

[Ernest Friedman-Hill]

OK, now we're getting somewhere. The protocol that news server use is called NNTP. Searching for "Java NNTP" at Google produces a lot of hits, including what look like some open-source libraries. Anybody have any specific recommendations?

[Jeroen Wenting]

I've once started work on an NNTP client which archived downloaded topics in a database (or was supposed to), but that was not done in Java...

I used Delphi which has built-in libraries for connecting to NNTP services (as well as most other common internet protocols).

[Sascha Henning]

Now i've found the right thing! The apache jakarta project offers a libary that contains very usefull classes. It is the commons project (http://jakarta.apache.org/commons/) that contains a net component. And in this, there are the basics for handling all that belongs to nntp.

Thanks for the valuable hints, to come to right direction!

with kind regards,

Sascha