Simulating browser in a application

[san jan]

Hi,
may i know how to simulate a browser in an application.
i wanna send details of a form to a url on internet & wanna catch the response & give status whether the submit was perfect or not.
i wanna find whether this can be done by URLConnection? can we submit a form through URLConnection?
its simulating a browser functionality in the application.
-santu

[James Swan]

Here is the almost complete code for POSTING data to a HTTP resource, you can fill in the exception handling to suit.

try {
    URL url = new URL("http://myserver.myhost.com");
    ...
    HttpURLConnection httpConnection = null;
    PrintWriter stdOut = null;
        
    try {
        httpConnection = (HttpURLConnection)url.openConnection();
        // we will be "posting" data to the HTTP resource
        httpConnection.setDoOutput(true);
        httpConnection.setRequestMethod("POST"); // can set this to "GET" also
        httpConnection.connect();
        // retrieve the output stream of the connection (stdout)
        stdOut = new PrintWriter(httpConnection.getOutputStream());
        // put your form data here
        stdOut.print("&myText1=myValue1");
        stdOut.print("&myCheckbox1=myValue2");
        ...
        stdOut.flush();
        // check the response code
        if (httpConnection.getResponseCode() != HttpURLConnection.HTTP_OK) {
            // do something to handle this
            ...
        }
        // retrieve the input stream of the connection
        BufferedReader in = new BufferedReader(new InputStreamReader(httpConnection.getInputStream()));
        for (String l = null; (l = in.readLine()) != null; ) {
            System.out.println(l);
        }
        in.close();
    } catch (IOException ioe) {
        // do something to handle this
        ...
    } finally {
        if (stdOut != null) {
            stdOut.close();
        }
    }
} catch (MalformedURLException mue) {
    // do something to handle this
    ...
}

James.

[san jan]

hi james,
thanks a lot, but am behind a firewall/proxy. i am getting following exception. how to set for a proxy address,port, and also provide username and password, so that application makes the connection.
the following exception i am getting at "httpConnection.connect()" line.
I have tried it but in vain.
can i have ur e-mail id?
-santu

C:\WINDOWS\Desktop>java TestURLConnect
entered program
opening http connection......
connecting........
IO ExceptionOperation timed out: connect
java.net.NoRouteToHostException: Operation timed out: connect
        at java.net.PlainSocketImpl.socketConnect(Native Method)
        at java.net.PlainSocketImpl.doConnect(Unknown Source)
        at java.net.PlainSocketImpl.connectToAddress(Unknown Source)
        at java.net.PlainSocketImpl.connect(Unknown Source)
        at java.net.Socket.<init>(Unknown Source)
        at java.net.Socket.<init>(Unknown Source)
        at sun.net.NetworkClient.doConnect(Unknown Source)
        at sun.net.www.http.HttpClient.openServer(Unknown Source)
        at sun.net.www.http.HttpClient.openServer(Unknown Source)
        at sun.net.www.http.HttpClient.<init>(Unknown Source)
        at sun.net.www.http.HttpClient.<init>(Unknown Source)
        at sun.net.www.http.HttpClient.New(Unknown Source)
        at sun.net.www.protocol.http.HttpURLConnection.connect(Unknown Source)
        at TestURLConnect.main(TestURLConnect.java:26)
leaving program

[This message has been edited by santosh kumar janmanchi (edited August 04, 2001).]

[Phil Hanna]

If you are behind a firewall, you need to define some system properties that point to your proxy:
System.setProperty("http.proxyHost", "myproxy.myserver.com");
System.setProperty("http.proxyPort", "80");
System.setProperty("http.proxySet", "true");
Note that this works for URLConnection access, but not plain sockets.
------------------
Phil Hanna
Sun Certified Programmer for the Java 2 Platform
Author of :
JSP: The Complete Reference
Instant Java Servlets

[san jan]

Hi Phil,
thank u. now i am able to connect to internet sites. i am able to read sites info.
<B>BUT AM NOT ABLE TO POST DATA TO ANY SITE</B>
am giving proxy settings, am giving authentication info thru subclassing Authenticator & then using its static method setDefault().
site i tested is : http://www.123greetings.com/
from this site's source i took the form that searches the site for greeting cards particular to some given words(can see under "Cool Services").
The code snippet i used for posting is

........
here setting the proxy, registering authenticator for proxy-authorization
........
URL url = new URL("http://www.123greetings.com/cgi-bin/search/search.pl");
HttpURLConnection connection = (HttpURLConnection) url.openConnection();
connection.setDoOutput(true);
connection.setRequestMethod("GET");
PrintWriter out = new PrintWriter(connection.getOutputStream());
out.println("?words="+URLEncoder.encode(args[0]));
// out.println("&words="+URLEncoder.encode(args[0]));
// out.println("words="+URLEncoder.encode(args[0]));
out.flush();
out.close();
.........
here reading the response & the page
.........

<B>I have tried all the above 3 options to post to the site, the response i get is '200', that is ok.
But the response page should contain 'searched word:birthday' as i have given birthday as the word to search, but instead gives in response page as 'searched word:' i.e., null is taken.</B>
not only with this, but with many other sites also am not getting the page am supposed to get, infact the post is not happening in a proper manner.
MAY I KNOW WHAT COULD BE THE REASON?
-santu

[san jan]

hi,
infact the code works perfectly fine with the following site http://java.sun.com/cgi-bin/backwards
which is given in sun's tutorials for testing of posting data to a url using url-connection objects.
but the same program is not posting data to any other sites, its not working perfectly.
-santu