Java automation to Login to website

[Nikhil Velayudham]

All,

I am in need of writing java application which logs into website.

The website which i want to log in.

i have attached the login screenshot , login html , login javascript and the javacode which i am using.

when i execute the java code , it gives me 200 http response but does not log into the website.

important thing to note is the 'password' in the login page is as part of the javascript and not part of the login form. I have hardcoded to generate this attribute to make it available as part of the post request.

i have also tried loggin using chrome poster ....When i try to post the request via chrome poster to the action url specified in the login form of login.html , i am not able to login to the site.

can anyone guide me as to what am i missing ?
how to set password filed which is available in javascript , is my hard coding in login.java correct?
Is it feasible to automate this login process using java.

any suggestions are greatly welcomed....hoping to have some guidance.

Please let me know if i need to provide further details.




login.java code snippet
------------------------------

import java.io.BufferedReader;
import java.io.DataOutputStream;
import java.io.InputStreamReader;
import java.io.UnsupportedEncodingException;
import java.net.CookieHandler;
import java.net.CookieManager;
import java.net.URL;
import java.net.URLEncoder;
import java.util.ArrayList;
import java.util.List;
import javax.net.ssl.HttpsURLConnection;
import org.jsoup.Jsoup;
import org.jsoup.nodes.Document;
import org.jsoup.nodes.Element;
import org.jsoup.select.Elements;
 
public class VendorPortalLogin {
 
  private List<String> cookies;
  private HttpsURLConnection conn;
 
  private final String USER_AGENT = "Mozilla/5.0";
 
  public static void main(String[] args) throws Exception {
 
	String url = "https://dvp-connect.abcd.com/vpn/index.html";
	String url1 =
        "https://dvp-connect.abcd.com/cvpn/aHR0cHM6Ly9kc3VpYXBwcHJkLndzZ2MuY29tOjQ0Mw/direct/authenticate.html";
	
 
	VendorPortalLogin http = new VendorPortalLogin();
 
	// make sure cookies is turn on
	CookieHandler.setDefault(new CookieManager());
 
	// 1. Send a "GET" request, so that you can extract the form's data.
	String page = http.GetPageContent(url);
	String postParams = http.getFormParams(page, "xyz", "xyz");
 
	// 2. Construct above post's content and then send a POST request for
	// authentication
	http.sendPost(url1, postParams);
 
	// getting the DSPortal login form details	
	
	// 3. success then go to login HomePage.
		String result = http.GetPageContent(url1);
		System.out.println(result);
	
  }
 
  private void sendPost(String url, String postParams) throws Exception {
 
	URL obj = new URL(url);
	conn = (HttpsURLConnection) obj.openConnection();
 
	// Acts like a browser
	conn.setUseCaches(false);
	conn.setRequestMethod("POST");
	conn.setRequestProperty("Host", "dvp-connect.abcd.com");
	conn.setRequestProperty("User-Agent", USER_AGENT);
	conn.setRequestProperty("Accept",
		"text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8");
	conn.setRequestProperty("Accept-Language", "en-US,en;q=0.5");
	for (String cookie : this.cookies) {
		conn.addRequestProperty("Cookie", cookie.split(";", 1)[0]);
	}
	conn.setRequestProperty("Connection", "keep-alive");
	conn.setRequestProperty("Referer", "https://dvp-connect.abcd.com/cvpn");
	conn.setRequestProperty("Content-Type", "application/x-www-form-urlencoded");
	conn.setRequestProperty("Content-Length", Integer.toString(postParams.length()));
 
	conn.setDoOutput(true);
	conn.setDoInput(true);
 
	// Send post request
	DataOutputStream wr = new DataOutputStream(conn.getOutputStream());
	wr.writeBytes(postParams);
	wr.flush();
	wr.close();
 
	int responseCode = conn.getResponseCode();
	System.out.println("\nSending 'POST' request to URL : " + url);
	System.out.println("Post parameters : " + postParams);
	System.out.println("Response Code : " + responseCode);
 
	BufferedReader in = 
             new BufferedReader(new InputStreamReader(conn.getInputStream()));
	String inputLine;
	StringBuffer response = new StringBuffer();
 
	while ((inputLine = in.readLine()) != null) {
		response.append(inputLine);
	}
	in.close();
	// System.out.println(response.toString());
 
  }
  
 
  private String GetPageContent(String url) throws Exception {
 
	URL obj = new URL(url);
	conn = (HttpsURLConnection) obj.openConnection();
 
	// default is GET
	conn.setRequestMethod("GET");
 
	conn.setUseCaches(false);
 
	// act like a browser
	conn.setRequestProperty("User-Agent", USER_AGENT);
	conn.setRequestProperty("Accept",
		"text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8");
	conn.setRequestProperty("Accept-Language", "en-US,en;q=0.5");
	if (cookies != null) {
		for (String cookie : this.cookies) {
			conn.addRequestProperty("Cookie", cookie.split(";", 1)[0]);
		}
	}
	int responseCode = conn.getResponseCode();
	System.out.println("\nSending 'GET' request to URL : " + url);
	System.out.println("Response Code : " + responseCode);
 
	BufferedReader in = 
            new BufferedReader(new InputStreamReader(conn.getInputStream()));
	String inputLine;
	StringBuffer response = new StringBuffer();
 
	while ((inputLine = in.readLine()) != null) {
		response.append(inputLine);
	}
	in.close();
 
	// Get the response cookies
	setCookies(conn.getHeaderFields().get("Set-Cookie"));
 
	return response.toString();
 
  }
 
  public String getFormParams(String html, String username, String password)
		throws UnsupportedEncodingException {
 
	System.out.println("Extracting form's data...");
 
	Document doc = Jsoup.parse(html);
 
	// Google form id
	Element loginform = doc.getElementById("login-form");
	//Elements inputElements = loginform.getElementsByTag("input");
	Elements inputElements = doc.select("input");
	List<String> paramList = new ArrayList<String>();
	for (Element inputElement : inputElements) {
		String key = inputElement.attr("name");
		String value = inputElement.attr("value");
 
		if (key.equals("login"))
			value = username;
		else if (key.equals("LogonButton"))
			value = "Log On";
		else if (key.equals("chk1_button"))
			value = "on";
		paramList.add(key + "=" + URLEncoder.encode(value, "UTF-8"));
	}
 String key1="passwd";
 String value1="xyz";
 paramList.add(key1 + "=" + URLEncoder.encode(value1, "UTF-8"));
	// build parameters list
	StringBuilder result = new StringBuilder();
	for (String param : paramList) {
		if (result.length() == 0) {
			result.append(param);
		} else {
			result.append("&" + param);
		}
	}
	return result.toString();
  }
 
  
  
  public List<String> getCookies() {
	return cookies;
  }
 
  public void setCookies(List<String> cookies) {
	this.cookies = cookies;
  }
 
}

Javascript snippet
-------------------

var loginPrefilled = false; function SetCookie(cookieName,cookieValue) { var today = new Date(); var expire = new Date(); expire.setTime(today.getTime() + 3600000*24); document.cookie = cookieName+"="+encodeURIComponent(cookieValue)+";path=/;expires="+expire.toGMTString(); } function UnsetCookie(cookieName) { document.cookie=cookieName+"=nothing;expires=Thursday, 1 Jan 1970 00:00:00 GMT; path=/"; } function ns_check() { var login = document.forms['vpnForm'].login.value; var passwd = document.forms['vpnForm'].passwd.value; if(login == "") { window.alert(_("You need to enter login name")); return false; } if (passwd == "") { window.alert(_("You need to enter passwd")); return false; } return true; } function ns_disperrmsg() { var vpnerrCookieVal = ns_getcookie("NSC_VPNERR"); if(vpnerrCookieVal == "4001" || vpnerrCookieVal == "4002") { document.writeln('<div id="feedbackArea">'); document.writeln('<div id="feedbackStyle" class="feedbackStyleError">'); document.writeln('<span id="errorMessageLabel" class="messageStyle">'); if(vpnerrCookieVal == "4001") { document.writeln(_("errorMessageLabel1")); } else { document.writeln(_("errorMessageLabel2")); } document.writeln('</span>'); document.writeln('</div>'); document.writeln('</div>'); UnsetCookie("NSC_VPNERR"); return; } document.writeln(" "); } UnsetCookie("NSC_USER"); function ns_getcookie(name) { var cookie=document.cookie; if(cookie.length > 0) { begin=cookie.indexOf(name+"="); if(begin!=-1) { begin+=name.length+1; end=cookie.indexOf(";", begin); if(end==-1) end=cookie.length; return decodeURIComponent(cookie.substring(begin, end)); } } return null; } function ns_fillName() { var value=ns_getcookie("NSC_NAME"); document.vpnForm.login.focus(); if(value!=null) { document.vpnForm.login.value = value; loginPrefilled = true; document.vpnForm.login.style.color = "#777"; document.vpnForm.passwd.focus(); } } function loginFieldCheck() { if(loginPrefilled == true) { document.vpnForm.login.blur(); } } function clean_name_cookie() { UnsetCookie("NSC_NAME"); } function ns_showpwd() { var pwc = ns_getcookie("pwcount"); document.write( '<TR><TD align=right style="padding-right:10px;white-space:nowrap;"><SPAN class=CTXMSAM_LogonFont>' + _("Password")); if ( pwc == 2 ) { document.write(''); } document.write(':</SPAN></TD>'); document.write( '<TD colspan=2 style="padding-right:8px;"><input class=CTXMSAM_ContentFont type="Password" title="' + _("Enter password") + '" name="passwd" size="30" maxlength="32" style="width:100%;"></TD></TR>'); if ( pwc == 2 ) { document.write( '<TR><TD align=right style="padding-right:10px;white-space:nowrap;"><SPAN class=CTXMSAM_LogonFont>' + 'RSA Passcode:' + '</SPAN></TD> <TD colspan=2 style="padding-right:8px;"><input class=CTXMSAM_ContentFont type="Password" title="' + _("Enter password") + '" name="passwd1" size="30" maxlength="32" style="width:100%;"></TD></TR>'); } UnsetCookie("pwcount"); } function enableLogonButton(obj) { var loginForm = document.vpnForm; if(obj.checked){ loginForm.elements["LogonButton"].disabled=false; } else{ loginForm.elements["LogonButton"].disabled=true; } }




login or index html
---------------------

<HTML><HEAD><TITLE>Citrix Access Gateway</TITLE>
<link rel="SHORTCUT ICON" href="/vpn/images/AccessGateway.ico" type="image/vnd.microsoft.icon">
<META http-equiv="Content-Type" content="text/html; charset=UTF-8">
<META content=noindex,nofollow,noarchive name=robots>
<LINK href="/vpn/images/caxtonstyle.css" type=text/css rel=STYLESHEET>
<script type="text/javascript" src="/vpn/resources.js"></script>
<script type="text/javascript" language="javascript">
var Resources = new ResourceManager("resources/{lang}", "logon");
</script>
<style type="text/css">
body
{
visibility: hidden;
}
</style>
<script language="javascript" type="text/javascript" src="/vpn/nsshare.js"></script>
<script language="JavaScript" type="text/javascript" src="login.js"></script>
<SCRIPT language=JavaScript>
// frame-busting script
setTimeout ("changePage()", 0);
function changePage()
{
if (self.parent != self)
top.location=document.location;
}
function setFocus(obj)
{
if (obj != null) {
obj.focus();
}
}
//-->
</SCRIPT>

<META content="MSHTML 6.00.2900.2802" name=GENERATOR></HEAD>
<BODY id=bodyTag onload="ns_fillName();">
<!-- Copyright 2001-2005 Citrix Systems, Inc. All Rights Reserved -->

<NOSCRIPT>
<BR><BR>
<table style="WIDTH: 100%">
<tr>
<td align="center">
<table class="CTXMSAM_LogonFont">
<tr id="errorMessageRow">
<td class="glowBoxLeft"> </td>
<td class="loginTableMidWidth">
<div id="feedbackArea">
<div id="feedbackStyle" class="feedbackStyleError">
<span id="errorMessageLabel" class="messageStyle">
JavaScript is either disabled in or not supported by the Web browser.
To continue logon, use a Web browser that supports JavaScript or enable JavaScript in your current browser.
</span>
</div>
</div>
</td>
<td class="glowBoxRight"> </td>
</tr>
</table>
</td>
</tr>
</table>

<table style="display:none">
</NOSCRIPT>

<!-- Header -->
<script language="javascript" type="text/javascript">
CreateMainTable();
AddHeaderAndBar();
</script>

<!-- Logon box -->
<tr class="mainPane" >
<td class="carbonBoxBottom" valign="bottom">
<script language="javascript" type="text/javascript">
documentWriteGlowBoxUpper();
</script>
<FORM method="post" action="/cgi/login" name="vpnForm" autocomplete="off" style="margin:0"
onsubmit="clean_name_cookie();">
<table cellspacing="4" cellpadding="0" border="0" width=100%>
<tr>
<td class="CTX_ContentTitleHeader">
<div id="ctl08_loginAgentCdaHeaderText" style="padding-left:15px"><span id="Welcome"></span></div>
</td>
</tr>
<tr>
<td class="CTXMSAM_LogonFont">
<div id="ctl08_loginAgentCdaHeaderText2" style="padding-left:15px"></div>
</td>
</tr>
<tr border=10>
<td class="CTXMSAM_LogonFont" valign="top">
<script language="javascript" type="text/javascript">
documentWriteActionPane();
</script>
<table width=70% id="ctl08_inputTable" class="CTXMSAM_LogonFont" cellspacing="0" cellpadding="4" border="0" style="border-width:0px;border-collapse:collapse;">
<tr><td colspan=3></td></tr>
<tr>
<td align="right" style="padding-right:10px;white-space:nowrap;"><span id="User_name" class="CTXMSAM_LogonFont"></span></td>
<td colspan=2 style="padding-right:8px;"><input id="Enter user name" class="CTXMSAM_ContentFont" style="font-size: 8pt" type="text" title="" name="login" size="30" maxlength="127" onFocus="loginFieldCheck()"style="width:100%;" /></td>
</tr>
<SCRIPT language=JavaScript>ns_showpwd();</SCRIPT>
<tr>
<!-- Disclaimer customization - Add a name to the LogonButton in order to be referenced later -->
<td></td><td></td><td align="right"><span style="display:none"><img src="/vpn/images/LoginButtonRolloverGlow.gif"/></span><input type="submit" id="Log_On" value="" class="CTX_CaxtonButton" onclick="ns_check();" onmouseover="this.className='CTX_CaxtonButton_Hover';" onmouseout="this.className='CTX_CaxtonButton';" name="LogonButton" disabled="true"/></td>
</tr>
<!-- Disclaimer customization -->
<tr><td colspan=3 align="center" style="font-size: 12pt" type="text" ><input type="checkbox" name="chk1_button" onClick="enableLogonButton(this);"/>  I Accept</td></tr>
<tr><td colspan=3 align="left" style="font-size: 8pt" type="text" >  By checking <b>"I Accept"</b> above you agree to the following terms and conditions:</td></tr>
<tr><td colspan=3 align="left" style="font-size: 8pt" type="text" >  Use of this system evidences an express consent to such monitoring and agreement that if such monitoring reveals evidence of possible abuse or criminal activity, system personnel may provide the results of such monitoring to appropriate officials.</td></tr>
</table>
<!-- End of Disclaimer customization-->

</div>
</td>
</tr>
</table>
</FORM>
<script language="javascript" type="text/javascript">
documentWriteGlowBoxLower();
</script>
</td>
</tr>

<!-- Error message box -->
<tr>
<td align="center">
<table class="CTXMSAM_LogonFont">
<tr id="errorMessageRow">
<td class="glowBoxLeft"> </td>

<td class="loginTableMidWidth">

<script language='javascript'>ns_disperrmsg();</script>

</td>

<td class="glowBoxRight"> </td>
</tr>
</table>
</td>
</tr>

<!-- Footer -->
<script language="javascript" type="text/javascript">
AddFooter();
</script>
</table>

<DIV id=scriptArea></DIV>
<script type="text/javascript" language="javascript">
Resources.Load();
</script>
</BODY>
</HTML>

[Ulf Dittmer]

If this was my problem I'd use the HtmlUnit library for this. Much easier than coding it all yourself.

[Nikhil Velayudham]

Thanks Marshall , I will look up some examples to see how to get it done.

If you happen to have any code samples please share.

Thanks

[Nikhil Velayudham]

Hi Folks,

Looking for more suggestions , if you have any please share.

thanks!

[Bear Bibeault]

My suggestion is to follow Ulf's suggestion.

[Nikhil Velayudham]

thanks Bear...i am in the process of trying it out.

[Campbell Ritchie]

Welcome to the Ranch

I added code tags to your post. Always use them (you can even change them to format JS). Doesn't it look better I had to break some of the long lines. I am not sure how to format the HTML, however.

[Nikhil Velayudham]

Aim;

Login to the citrix portal shown in the above screenshot successfully.

Problem:

Password field is not available in the login form [index.html].

Javascript function[ns_showpwd()] is being used in the index.html at the place of password field.

This javascript function has the html code for generating the password field [code is attached in the begining of the thread].

When i look at the view source of index.html , the html content of ns_showpwd() is not loaded in the page, page is designed that way .

Error:

With my current htmlunit program i am getting below error as mycode is not finding the passwd field as it is not available in [index.html]

Exception in thread "main" com.gargoylesoftware.htmlunit.ElementNotFoundException: elementName=[input] attributeName=[name] attributeValue=[passwd]
at com.gargoylesoftware.htmlunit.html.HtmlForm.getInputByName(HtmlForm.java:412)
at com.auto.login.JiraLogin.logIn(JiraLogin.java:74)
at com.auto.login.JiraLogin.main(JiraLogin.java:52)

Question

How do i read the password field which is in the javascript using htmlunit , so that i can click and log in to the portal.

Attaching my current htmlunit code for reference.


Please help guide me as to how we can retrieve passwd field .

[Nikhil Velayudham]

htmlunit java code

package com.auto.login;

import java.io.IOException;
import java.net.URL;
import java.util.List;

import com.gargoylesoftware.htmlunit.BrowserVersion;
import com.gargoylesoftware.htmlunit.Page;
import com.gargoylesoftware.htmlunit.RefreshHandler;
import com.gargoylesoftware.htmlunit.WebClient;
import com.gargoylesoftware.htmlunit.WebClient.*;
import com.gargoylesoftware.htmlunit.html.HtmlAnchor;
import com.gargoylesoftware.htmlunit.html.HtmlForm;
import com.gargoylesoftware.htmlunit.html.HtmlPage;
import com.gargoylesoftware.htmlunit.html.HtmlTable;
import com.gargoylesoftware.htmlunit.html.HtmlTableRow;
import com.gargoylesoftware.htmlunit.html.HtmlSubmitInput;
import com.gargoylesoftware.htmlunit.html.HtmlTextInput;
import com.gargoylesoftware.htmlunit.html.HtmlPasswordInput;
import com.gargoylesoftware.htmlunit.html.HtmlDivision;
import com.gargoylesoftware.htmlunit.html.HtmlAnchor;
import com.gargoylesoftware.htmlunit.html.HtmlCheckBoxInput;
import com.gargoylesoftware.htmlunit.*;
import java.net.CookieHandler;
import java.net.CookieManager;

public class JiraLogin {

public static void main(String[] args) throws Exception {

    // Create and initialize WebClient object
    	WebClient webClient = new WebClient(/*BrowserVersion.CHROME_16*/);
    	webClient.setThrowExceptionOnScriptError(false);
    	webClient.setJavaScriptEnabled(true);
    //	webClient.setCssEnabled(false);       
    // 	webClient.getCookieManager().setCookiesEnabled(true);
    // 	setCredentials(webClient);

    
   // webClient.Headers.Add ("User-Agent:Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/37.0.2062.120 Safari/537.36");
    
    HtmlPage subj_page = null;
    //visit login page and get it
    String url = "https://dvp-connect.abc.com/vpn/index.html";
    //webClient.wait(1 * 10);
    HtmlPage page = (HtmlPage) webClient.getPage(url);
    
    String theContent1 = webClient.getPage(url).getWebResponse().getContentAsString();
    System.out.println("page string"+theContent1);
    
    HtmlAnchor anchor = null;
    page = logIn(page);

    // search for content
   // page = (HtmlPage) searchPage(page, " A Jira 5 Cheat Sheet can be");     

    // click on the paper link      
    anchor = (HtmlAnchor) page.getAnchorByHref("https://dvp-connect.abc.com/cvpn/aHR0cHM6Ly9kc3VpYXBwcHJkLndzZ2MuY29tOjQ0Mw/directshipment/login");
    page = (HtmlPage) anchor.click();

    // loop through found articles
    //{{{page   

}


public static HtmlPage logIn(HtmlPage page) {
    HtmlPage nextpage = null;
    final HtmlForm form = page.getFormByName("vpnForm");
    //final HtmlForm form = page.get;
    final HtmlTextInput username =(HtmlTextInput) form.getInputByName("login");
    final HtmlPasswordInput password = (HtmlPasswordInput)form.getInputByName("passwd");
    //final HtmlPasswordInput password1=form.document.forms['vpnForm'].passwd.name;
    final HtmlCheckBoxInput check = (HtmlCheckBoxInput)form.getInputByValue("Log In");
    final HtmlSubmitInput button = (HtmlSubmitInput)form.getInputByValue("LogonButton");
    username.setValueAttribute("xyz");
    password.setValueAttribute("abc");
    check.setValueAttribute("on"); 
    button.setValueAttribute("Log On"); 

    // hit submit button and return the requested page
    try {
        nextpage =(HtmlPage) button.click();
    } catch (IOException e) {
        e.printStackTrace();
    }
        return nextpage;
}

}

[Nikhil Velayudham]

All,

Any pointers or help on this topic........need inputs to be able to complete this task .


Regards,
Nikhil

[Ulf Dittmer]

Oh, I thought the code you posted yesterday was the solution, since there wasn't any question on it. What does or does it not do? What's still missing?

[Nikhil Velayudham]

here are the questions..


Aim;

Login to the citrix portal shown in the above screenshot successfully.

Problem:

Password field is not available in the login form [index.html].

Javascript function[ns_showpwd()] is being used in the index.html at the place of password field.

This javascript function has the html code for generating the password field [code is attached in the begining of the thread].

When i look at the view source of index.html , the html content of ns_showpwd() is not loaded in the page, page is designed that way .

Error:

With my current htmlunit program i am getting below error as mycode is not finding the passwd field as it is not available in [index.html]

Exception in thread "main" com.gargoylesoftware.htmlunit.ElementNotFoundException: elementName=[input] attributeName=[name] attributeValue=[passwd]
at com.gargoylesoftware.htmlunit.html.HtmlForm.getInputByName(HtmlForm.java:412)
at com.auto.login.JiraLogin.logIn(JiraLogin.java:74)
at com.auto.login.JiraLogin.main(JiraLogin.java:52)

Question

How do i read the password field which is in the javascript using htmlunit , so that i can click and log in to the portal.

Attaching my current htmlunit code for reference.


Please help guide me as to how we can retrieve passwd field .