Communicate with the server without a browser.

[Hemal Mahagederawatte]

Hi everyone,

I need to create a java class which can do the following requirement.

- When the openid and the password is given it should create and HttpRequest and send to the https://www.myopenid.com/signin_password page and get the authResponse which has the openid token.

- I need to do accomplish the above task without using a browser.

How can i accomplish my task. A sample code would be much appreciated.

Thanks in advanced.

[Tom Reilly]

Take a look at the java.net.URLConnection documentation.

[Hemal Mahagederawatte]

Thanks for your information.

But a small problem.
When i write URLConnection uRLConnection = new URLConnection(url) it gives and error saying that all abstract methods are not implemented.
When i click the suggestion button and click implement all abstract methods its shows something like this.

URLConnection uRLConnection = new URLConnection(url) {

@Override
public void connect() throws IOException {
throw new UnsupportedOperationException("Not supported yet.");
}
};


What should be written inside the connect method?

Thank you.

[Hemal Mahagederawatte]

Sorry i figured out the solution from some other forum.

instead of creating a URLConnection by new URLConnection()
need to write the code as follow

URL url = new URL("http://www.abc.com");
URLConnection uRLConnection = url.OpenConnection();

[Ulf Dittmer]

You don't use the URLConnection constructor. You get an URLConnection object via new URL(...).openConnection().

Edit: Too late :-)

[Rob Spoor]

Apache's HttpClient may be a bit easier to work with.

[Hemal Mahagederawatte]

How do i give the username and the password

I tried the following code, but it didn't succeed.

URL url = new URL("https://www.myopenid.com/signin_password");
URLConnection uRLConnection = url.openConnection();
uRLConnection.setAllowUserInteraction(true);
uRLConnection.setDoInput(true);
uRLConnection.setDoOutput(true);
uRLConnection.setRequestProperty("Username", "hemaljoes");
uRLConnection.setRequestProperty("password", "password");
uRLConnection.connect();

OutputStream outputStream = uRLConnection.getOutputStream();
outputStream.flush();

InputStream in = uRLConnection.getInputStream();
InputStreamReader inReader = new InputStreamReader(in);
BufferedReader reader = new BufferedReader(inReader);

String newLine = null;
while ((newLine = reader.readLine()) != null)
System.out.println(newLine);
reader.close();

String headerName=null;
for (int i=1; (headerName = uRLConnection.getHeaderFieldKey(i))!=null; i++) {
System.out.println(uRLConnection.getHeaderFieldKey(i));
System.out.println(uRLConnection.getHeaderField(i));
}

Can anyone help me please.

[Ulf Dittmer]

Where did you get the idea about Username and Password headers?
See https://coderanch.com/how-to/java/AppletsFaq#authentication (it doesn't matter that the page talks about applets - the code is the same).

If you're doing more involved things over this connection, then using HttpClient will likely be easier, as Rob said.

[Hemal Mahagederawatte]

I changed the following code to code below. But i don't get authenticated

URL url = new URL("https://www.myopenid.com/signin_password");
URLConnection uRLConnection = url.openConnection();
uRLConnection.setAllowUserInteraction(true);
uRLConnection.setDoInput(true);
uRLConnection.setDoOutput(true);
uRLConnection.setRequestProperty("Username", "hemaljoes");
uRLConnection.setRequestProperty("password", "password");
uRLConnection.connect();

URL url = new URL("https://www.myopenid.com/signin_password");
URLConnection uRLConnection = url.openConnection();
uRLConnection.setAllowUserInteraction(true);
uRLConnection.setDoInput(true);
uRLConnection.setDoOutput(true);

sun.misc.BASE64Encoder encoder = new BASE64Encoder();
String authorization ="hemaljoes:password";
byte[] bt =authorization.getBytes();
String encodedData = encoder.encode(bt);

uRLConnection.setRequestProperty("Authorization", "Basic " + encodedData);
uRLConnection.connect();

[Ulf Dittmer]

But i don't get authenticated

What does that mean? What's the server response? Post the HTTP status code and the body of the response.

You can also use a tool like tcpmon to observe the HTTP that gets sent to the server, and compare it to what gets sent if you access that site from within a browser.

[Hemal Mahagederawatte]

I said i don't get authenticated because after I run my code and went to the https://www.myopenid.com/signin_password it asks for my username and password. If I was successfully authenticated when i run my code it shouldn't be asking like that. Isn't it?

Well when i run the above code i get the reply as this . which is the html of the https://www.myopenid.com/signin_password

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">


<html>
  <head >
    <base href="https://www.myopenid.com/" />
    <title>Sign In
</title>
<script type="text/javascript"
   src="https://www.myopenid.com/static/MochiKit/MochiKit.js"></script>
<script type="text/javascript" src="https://www.myopenid.com/static/entangle.js"></script>

<script src="https://www.myopenid.com/static/InformationCards/InformationCards.js"
        type="text/javascript"></script>


<link rel="shortcut icon" href="https://www.myopenid.com/favicon.ico?version=1" />
<link rel="icon" href="https://www.myopenid.com/favicon.ico?version=1" />

<link rel="seatbelt.config" type="application/xml" href="https://www.myopenid.com/verisign_plugin_config" />

<link rel="stylesheet" href="/stylesheets/screen.css?version=5" type="text/css" media="screen" />
<link rel="stylesheet" href="/stylesheets/print.css?version=1" type="text/css" media="print" />
<!--[if IE]><link rel="Stylesheet" href="/stylesheets/screen_ie.css?version=2" type="text/css" media="screen" /><![endif]-->
<!--[if lt IE 6]><link rel="Stylesheet" href="/stylesheets/screen_ie5.css?version=1" type="text/css" media="screen" /><![endif]-->
  </head>
  <body>
    <script type="text/javascript">
      if (top != self) top.location.href = 'https://www.myopenid.com/framed';
    </script>
<p class="skip"><a >Skip to Content</a></p>
<p class="skip"><a >Skip to Navigation</a></p>
<div id="container">

  <div id="header">
    <h6><a href="https://www.myopenid.com/" rel="nofollow">myOpenID - The free, secure OpenID server</a></h6>

  </div>
  <div id="columns" class="ca">
    <div id="content">
    <div id="personal-icon">
      <p id="personal-icon-container">
        <a href="https://www.myopenid.com/set_cookie_image" rel="nofollow">Your Personal Icon</a>
      </p>
    </div>
      <div style="position: absolute; margin-bottom: 1em;"><h1><span>Sign In
</span></h1></div><h1 style="margin-bottom: 2em;"> </h1>
      






<form method="post" action="https://www.myopenid.com/signin_submit" id="password-signin-form">
<table class="form_layout">
   <tbody>
     <tr >
       <td><label for="identity">Username</label></td>
       <td>
         <input type="text" name="user_name" id="identity"
                tabindex="1" value="" />
       </td>
     </tr>
     <tr>
       <td><label for="password">Password</label></td>
       <td>
         <input id="password" type="password" name="password" tabindex="2" />
       </td>
     </tr>
     <tr class="radiochecks">
       <td>
         <input
          title="Checking this box will keep you signed in to
                    myOpenID even after you exit your browser.  You
                    should NOT check this box on a shared computer!."
         id="stay_signed_in" type="checkbox" name="stay_signed_in"
         ) tabindex="3" />
       </td>
       <td>
         <label for="stay_signed_in">Stay signed in</label>
       </td>
     </tr>
   </tbody>
 </table>
<script>
  document.getElementById('identity').focus()
</script>
<div class="buttons">


<span id="spinner_signin_button" style="display: none;" >
  <img src="https://www.myopenid.com/static/images/spinner.gif" />
  <span> Authenticating... </span>
</span>

<input id="signin_button" type="submit" tabindex="4" value="Sign In"
    onclick="spinner_click_handler_signin_button();"
/>
<input type="submit" tabindex="5" value=Cancel name="cancelled" onclick="document.location.href='https://www.myopenid.com/'; return false;" />
<input type="hidden" name="tid" value="e85702fb" />

<script type="text/javascript">
  spinner_click_handler_signin_button = function() {
    var buttonEl = document.getElementById("signin_button");
    document.getElementById('spinner_signin_button').style.display='inline';
    buttonEl.disabled = true;
    if (buttonEl.getAttribute('type') == 'submit') {
      buttonEl.form.submit();
    }
  }
  document.getElementById("signin_button").disabled = false;
</script>

</div>
<input type="hidden" name="token" value="f8c5257f858605113c207837e8bedf160000000000062802" /><input type="hidden" name="_" value="*1475-a7a2-7e9b" /></form>

<ul>
  <li id="infocard-li">
      <form method="post" action="https://www.myopenid.com/signin_infocard" id="infocard-form" class='noline'>
  <input type="hidden" name="tid" value="e85702fb" />
  <input type="hidden" name="resume_action" value="signin_password" />
  <input type="image" id="infocard-logo-button"
         value="Choose an Information Card"
         src="https://www.myopenid.com/static/InformationCards/images/infocard_30x21.png">
  <label for='infocard-logo-button'><a id="infocard-label"  rel="nofollow">Sign in with an Information Card</a></label>
  <div style="display:none;" id="infocard-holder"></div>
  <noscript>
  
<OBJECT type="application/x-informationCard"
        name="xmlToken"
        class="skip">
  <PARAM Name="tokenType"
         Value="urn:oasis:names:tc:SAML:1.0:assertion">
  <PARAM Name="issuer"
         Value="http://schemas.xmlsoap.org/ws/2005/05/identity/issuer/self">
  <PARAM Name="requiredClaims"
         Value="http://schemas.xmlsoap.org/ws/2005/05/identity/claims/privatepersonalidentifier">
  <PARAM Name="optionalClaims"
         Value="http://schemas.xmlsoap.org/ws/2005/05/identity/claims/emailaddress http://schemas.xmlsoap.org/ws/2005/05/identity/claims/givenname http://schemas.xmlsoap.org/ws/2005/05/identity/claims/surname">
</OBJECT>


  </noscript>
  <script defer="defer">
    (function() {
        var hide = function () {
            // hide widgets if infocard is not supported
            var infocardFormHolder = document.getElementById('infocard-li');
            infocardFormHolder.style.display = 'none';
        };
        if (InformationCard.AreCardsSupported()) {
            var enable = function () {
                try {
                    document.getElementById('infocard-label').setAttribute('href', '#');
                    document.getElementById('infocard-holder').innerHTML = '\x0a<OBJECT type=\x22application/x-informationCard\x22\x0a        name=\x22xmlToken\x22\x0a        class=\x22skip\x22>\x0a  <PARAM Name=\x22tokenType\x22\x0a         Value=\x22urn:oasis:names:tc:SAML:1.0:assertion\x22>\x0a  <PARAM Name=\x22issuer\x22\x0a         Value=\x22http://schemas.xmlsoap.org/ws/2005/05/identity/issuer/self\x22>\x0a  <PARAM Name=\x22requiredClaims\x22\x0a         Value=\x22http://schemas.xmlsoap.org/ws/2005/05/identity/claims/privatepersonalidentifier\x22>\x0a  <PARAM Name=\x22optionalClaims\x22\x0a         Value=\x22http://schemas.xmlsoap.org/ws/2005/05/identity/claims/emailaddress http://schemas.xmlsoap.org/ws/2005/05/identity/claims/givenname http://schemas.xmlsoap.org/ws/2005/05/identity/claims/surname\x22>\x0a</OBJECT>\x0a';
                } catch (e) {
                    hide();
                }
            };
            if (window.ActiveXObject) {
                window.attachEvent("onload", enable);
            } else {
                window.addEventListener("load", enable, false);
            }
        } else {
            hide();
        }
    })();
  </script>
  <input type="hidden" name="token" value="31a9101a3a3e91b2e845c8f7aad7b0e50000000000091f01" /><input type="hidden" name="_" value="*1475-a7a2-7e9b" /></form>

  </li>
  <li><a href="https://www.myopenid.com/signin_certificate?_=%2A1475-a7a2-7e9b&tid=e85702fb&token=fc1bb60f1f59cdc152bda2646c657b5600000000000d723f" rel="nofollow">
    Sign in with an SSL certificate</a></li>
  <li><a href="https://www.myopenid.com/account_recover_initiate" rel="nofollow">
    I cannot access my account</a></li>
</ul>



    </div>
    <div id="navigation">
          <h2>
      Options
    </h2>
 
    <ul>
        
<li id="navigation-home"><a href="https://www.myopenid.com/" rel="nofollow">Home</a></li>

        
<li><a href="https://www.myopenid.com/signup" rel="nofollow">Sign Up</a></li>

        
<li><a href="https://www.myopenid.com/account_recover_initiate" rel="nofollow">Recover Account</a></li>

        
<li><a href="https://www.myopenid.com/directory" rel="nofollow">OpenID Site Directory</a></li>

    </ul>

    </div>
  </div>
  <div id="footer">
    <p class="fl"> 
  <a href="https://www.myopenid.com/help" rel="nofollow">Help</a> |
  <a href="https://www.myopenid.com/privacy" rel="nofollow">Privacy</a> |

          Language: <a href="https://www.myopenid.com/set_language" rel="nofollow">Not set</a>
</p>

<p class="fr"> 
  <a href="http://blog.janrain.com/" rel="nofollow">Blog</a> |

  <a href="http://janrain.com/background/" rel="nofollow">About Us</a> |

  © 2008 <a href="http://janrain.com/" rel="nofollow">Janrain, Inc.</a>

  <br />

  myOpenID™ and the myOpenID™ website are
     trademarks of Janrain, Inc.
</p>

  </div>
</div>

<script type="text/javascript">
var gaJsHost = (("https:" == document.location.protocol) ? "https://ssl." : "http://www.");
document.write(unescape("%3Cscript src='" + gaJsHost + "google-analytics.com/ga.js' type='text/javascript'%3E%3C/script%3E"));
</script>
<script type="text/javascript">
if (window._gat) {
  var pageTracker = _gat._getTracker('UA-3697105-3');
  pageTracker._initData();
  pageTracker._trackPageview();
}
</script>
  </body>
</html>

Date
Sat, 29 Jan 2011 19:26:30 GMT
Server
Apache/2.2
Content-Length
9227
Content-Type
text/html; charset=UTF-8
Set-Cookie
ephemeral_session_id=fe90fa60478e0107616ac9fe48a803b0784c44e4319cc61a6af3b06f8c4bc6e9; domain=myopenid.com; path=/
Set-Cookie
browser_id=14758d0f3fd05ad24d40ff26c7286df5f161fbc939cf425f50e3d77dca1a9fcf; domain=myopenid.com; path=/; expires=Sun, 29-Jan-2012 19:26:30 GMT
Set-Cookie
secure_session_id=45821a0e8563d543e50bd894b558f818832a04d700569ca7652cf1f33647109c; domain=myopenid.com; path=/; secure; expires=Sun, 29-Jan-2012 19:26:30 GMT
Set-Cookie
session_id=82b46b818e3309a1ecf7e05d5adc0908227f6f0ac77d27924b54c21a8fc7db09; domain=myopenid.com; path=/; expires=Sun, 29-Jan-2012 19:26:30 GMT
P3P
CP="IDC DSP COR ADM DEVi TAIi PSA PSD IVAi IVDi CONi HIS OUR IND CNT"
Connection
close
BUILD SUCCESSFUL (total time: 2 seconds)

[Ulf Dittmer]

The form on that page submits to https://www.myopenid.com/signin_submit, not the URL you're using. And it uses form parameters, not basic authentication.

Where did you get the idea that using basic authentication against https://www.myopenid.com/signin_password should work?

[Hemal Mahagederawatte]

I'm really sorry, I'm new to java.

I though it's not possible to directly send request to the https://www.myopenid.com/signin_submit. Because if you do it in the browser it will give the following message

Error
You have followed a bad link. Please inform the owners of the site from which you came.

Any way can you give me any suggestion on how to authenticate.

Thanks in advanced.

[Campbell Ritchie]

This question is too difficult for "beginning". Not sure where to move it to, but I'll try JiG

[Ulf Dittmer]

I though it's not possible to directly send request to the https://www.myopenid.com/signin_submit. Because if you do it in the browser...

A browser sends GET request, whereas the form is submitted via POST; you can't generally substitute one for the other.

It looks as if you need to send a POST request to https://www.myopenid.com/signin_submit that has parameters "user_name and "password". This example should help you get going: http://www.exampledepot.com/egs/java.net/Post.html

[Hemal Mahagederawatte]

I tried the following code

String data = URLEncoder.encode("user_name", "UTF-8") + "=" + URLEncoder.encode(userName, "UTF-8");
        data += "&" + URLEncoder.encode("password", "UTF-8") + "=" + URLEncoder.encode(password, "UTF-8");

        URL url = new URL("https://www.myopenid.com/signin_submit");
        URLConnection conn = url.openConnection();
        //conn.setAllowUserInteraction(true);
        //conn.setDoInput(true);
        conn.setDoOutput(true);

        OutputStreamWriter writer = new OutputStreamWriter(conn.getOutputStream());
        writer.write(data);
        writer.flush();

        //Printing the Response
        BufferedReader reader = new BufferedReader(new InputStreamReader(conn.getInputStream()));
        String line="";
        while((line = reader.readLine()) != null){
            System.out.println(line);
        }

But it returns the index page not the page that should be get if it is successfully authenticated.

In the example that is in the http://www.exampledepot.com/egs/java.net/Post.html it uses http. In my scenario it is a https connection. Does some additional thing to be done when you are posting to a https connection.