Login and Download file over https

[Mike Boota]

Hi,

I am trying to automate (i.e. write a program to replace) the manual process of having a human use a web browser to log into a company's HTTPS website and download a file.

I have tried to hunt for code examples and forum issues about doing the same, but no luck. I hope someone will point me at how to get started.

Thanks

[Scott Dunbar]

I would look at using Apache Commons HttpClient to do this. It has support for SSL along with different login methods.

[Mike Boota]

I looked at HttpClient and tried the following example. But somehow I can't connect and it keeps giving error. Here is the code I am trying:

import org.apache.commons.httpclient.*;
import org.apache.commons.httpclient.kookie.kookiePolicy;
import org.apache.commons.httpclient.kookie.kookieSpec;
import org.apache.commons.httpclient.methods.*;
 
public class FormLoginDemo
{
    static final String LOGON_SITE = "https://my.domain.com/bin/logon.asp";//default logon page
    static final int    LOGON_PORT = 80;
 
    public FormLoginDemo() {
        super();
    }
 
    public static void main(String[] args) throws Exception {
 
        HttpClient client = new HttpClient();
                        
        PostMethod authpost = new PostMethod("https://my.domain.com/bin/forward.asp");//controller page to check the uid/pwd validity
        // Prepare login parameters
        NameValuePair action   = new NameValuePair("action", "login");
        NameValuePair url      = new NameValuePair("url", "https://my.domain.com/bin/default.asp"); //if uid/pwd valid user is routed to this page
        NameValuePair userid   = new NameValuePair("UserId", "userid");
        NameValuePair password = new NameValuePair("Password", "password");
        authpost.setRequestBody( 
          new NameValuePair[] {action, url, userid, password});
        
        client.executeMethod(authpost);
        System.out.println("Login form post: " + authpost.getStatusLine().toString()); 
        // release any connection resources used by the method
        authpost.releaseConnection();
        // See if we got any kookies
        // The only way of telling whether logon succeeded is 
        // by finding a session kookie
        kookie[] logonkookies = kookiespec.match(
            LOGON_SITE, LOGON_PORT, "/", false, client.getState().getkookies());
        System.out.println("Logon kookies:");    
        if (logonkookies.length == 0) {
            System.out.println("None");    
        } else {
            for (int i = 0; i < logonkookies.length; i++) {
                System.out.println("- " + logonkookies[i].toString());    
            }
        }
        // Usually a successful form-based login results in a redicrect to 
        // another url
        int statuscode = authpost.getStatusCode();
        if ((statuscode == HttpStatus.SC_MOVED_TEMPORARILY) ||
            (statuscode == HttpStatus.SC_MOVED_PERMANENTLY) ||
            (statuscode == HttpStatus.SC_SEE_OTHER) ||
            (statuscode == HttpStatus.SC_TEMPORARY_REDIRECT)) {
            Header header = authpost.getResponseHeader("location");
            if (header != null) {
                String newuri = header.getValue();
                if ((newuri == null) || (newuri.equals(""))) {
                    newuri = "/";
                }
                System.out.println("Redirect target: " + newuri); 
                GetMethod redirect = new GetMethod(newuri);
 
                client.executeMethod(redirect);
                System.out.println("Redirect: " + redirect.getStatusLine().toString()); 
                // release any connection resources used by the method
                redirect.releaseConnection();
            } else {
                System.out.println("Invalid redirect");
                System.exit(1);
            }
        }
    }
}

but I keep getting the following error. Can someone please help me out here:

Exception in thread main
javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
at com.sun.net.ssl.internal.ssl.Alerts.getSSLException(Alerts.java:150)
at com.sun.net.ssl.internal.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1476)
at com.sun.net.ssl.internal.ssl.Handshaker.fatalSE(Handshaker.java:174)
at com.sun.net.ssl.internal.ssl.Handshaker.fatalSE(Handshaker.java:169)
at com.sun.net.ssl.internal.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:847)
at com.sun.net.ssl.internal.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:106)
at com.sun.net.ssl.internal.ssl.Handshaker.processLoop(Handshaker.java:495)
at com.sun.net.ssl.internal.ssl.Handshaker.process_record(Handshaker.java:433)
at com.sun.net.ssl.internal.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:815)
at com.sun.net.ssl.internal.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1025)
at com.sun.net.ssl.internal.ssl.SSLSocketImpl.writeRecord(SSLSocketImpl.java:619)
at com.sun.net.ssl.internal.ssl.AppOutputStream.write(AppOutputStream.java:59)
at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:65)
at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:123)
at org.apache.commons.httpclient.methods.EntityEnclosingMethod.writeRequestBody(EntityEnclosingMethod.java:502)
at org.apache.commons.httpclient.HttpMethodBase.writeRequest(HttpMethodBase.java:1973)
at org.apache.commons.httpclient.HttpMethodBase.execute(HttpMethodBase.java:993)
at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:397)
at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:170)
at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:396)
at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:322)
at com.acdo.login.common.FormLoginDemo.main(FormLoginDemo.java:47)
Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:221)
at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:145)
at sun.security.validator.Validator.validate(Validator.java:203)
at com.sun.net.ssl.internal.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:172)
at com.sun.net.ssl.internal.ssl.JsseX509TrustManager.checkServerTrusted(SSLContextImpl.java:320)
at com.sun.net.ssl.internal.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:840)
... 17 more
Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
at sun.security.provider.certpath.SunCertPathBuilderException.init(SunCertPathBuilderException.java:87)
at sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:236)
at java.security.cert.CertPathBuilder.build(CertPathBuilder.java:194)
at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:216)
... 22 more
 

If I open a browser and copy the following in URL it directly logs me in

https://my.domain.com/bin/default.asp&UserId=userid?Password=password

How can I log-in throug my script. Thanks

[S Hayes]

Is this when you're compiling or running it?

It looks like your call to the getStatusCode() method on line 47 throws a javax.net.ssl.SSLHandshakeException. I think you just need to catch it.

[Mike Boota]

It's throwing exception here:

client.executeMethod(authpost);

Can someone please tell me what might be wrong.

[S Hayes]

That line is probably where you need to catch the exception.

Could you provide a link to the Javadocs for the Apache files? I could probably be of more help to you then.