login and downloading files from website using java code

[Karan Jain]

Hi,

I have access to the client website where i can login and can download some file by selecting my specific criteria. I can then download 1 file at a time by selecting the file.

What I want is write a java program which will login me on the website and then enable me to download the files 1 after the other. I have login URL and the file URLs (as I can modify the IDs in the URL to point to next file).

I tried to use Commons HTTPClient but I am not successful in doing so. I am unable to set the cookies and then point to file URLs.

Client's website is in .Net.

Do I need to ask the client how they are setting the cookie so that i can do the same in my java program?

Please guide me to achieve this.

Thanks in advance...

Regards,
Karan

[Norm Radder]

I'm not familiar with the Commons HTTPClient code so I can't help you with that. The class must have methods to set the various fields in the HTTP headers.
Cookies are set by the server and passed to the client to save and return the next time the client connects to that server. Cookies are passed as part of the HTTPHeader.

The code that the server is written in should NOT have any effect on the HTTP protocol.

I understand your project to be a program to run on a client that is to connect to a server at a URL. Not sure what you mean by "log me in on the website". Are there user-ids and passwords involved? If so, the server will have to have a way to indicate that you have logged in successfully, perhaps via a cookie. After that you would issue HTTP GETs for the desired files, passing the cookie each call.
For projects like I just described, I use a proxy server I wrote that allows me to see the HTTP files going and coming between a browser and the server.
That allows me to write a program that does what the browser does and hopefully to satisfy the server.

[Karan Jain]

Thanks Norm.

Yes the website is setting the cookies.

I know the login URL and I am able to get the cookies in the response which I am able to print.

But how to set the cookies in the system so that next time when I am trying to access the website it should recognise me and should allow me to download the required files.

Thanks,
Karan

[Norm Radder]

how to set the cookies in the system

Where/what is 'the system'?

the cookie is initially set by the server. It passes a SetCookie field to a browser that connects to that server. The browser saves the cookie.
The next time the browser connects to that server, it passes a Cookie field in the request header with the value of the cookie it received from the server.

For your program to get a cookie:
1)have your program connect to the server and be assigned a cookie that it saves, just like a browser does
2)somehow get the current cookie value that is in your browser and code that into your program.
[ June 20, 2008: Message edited by: Norm Radder ]

[Karan Jain]

Thanks Norm,

I was able to make it work using Commons HttpClient.

I did the following:
1. Send the request using the logon URL with user name and password as NV pairs.
2. The URL send the response with cookies.
3. Send the next request setting the cookies in the header and it gives the response as the file.

Thanks a lot for pointing me to right direction.
reagrds,
Karan

[t horo]

hi
can you put a code snippet?
I'm trying to do the same with no luck
thanks
[ July 21, 2008: Message edited by: Ulf Dittmer ]