[HELP] How to login to a website and get XML data of that site

[Gynnad Paullussen]

Hello,

I'm trying to login to a website, named Maxithlon.com, to get xml data from that site. So I can make a program for the users. The only problem is, I can't get the connection.

This is the login string query: http://www.maxithlon.com/maxi-xml/login.php?user=Gynnad&scode=1q2w3e4r
Where Gynnad is the username and 1q2w3e4r is the securitycode.

First I thought it was a POST-method but it isn't, so this code isn't right, but is what i had:

/*
 * To change this template, choose Tools | Templates
 * and open the template in the editor.
 */

package Connection;

/**
 *
 * @author Danny
 */
import java.net.*;
import java.io.*;

    public class ConnectToURL {

        // Variables to hold the URL object and its connection to that URL.
        private static URL URLObj;
        private static URLConnection connect;

    public static void main(String[] args) {
        try {
            // Establish a URL and open a connection to it. Set it to output mode.
            //login link: http://www.maxithlon.com/maxi-xml/login.php?user=Gynnad&scode=1q2w3e4r
            URLObj = new URL("http://www.maxithlon.com/maxi-xml/login.php");
            connect = URLObj.openConnection();
            connect.setDoOutput(true);
       }
        catch (MalformedURLException ex) {
            System.out.println("The URL specified was unable to be parsed or uses an invalid protocol. Please try again.");
            System.exit(1);
        }
        catch (Exception ex) {
            System.out.println("An exception occurred. " + ex.getMessage());
            System.exit(1);
        }


      try {
            // Create a buffered writer to the URLConnection's output stream and write our forms parameters.
            BufferedWriter writer = new BufferedWriter(new OutputStreamWriter(connect.getOutputStream()));
            writer.write("user=Gynnad&scode=1q2w3e4r");
            //username=MyUsername&pass=MyPassword&submit=Login (Origineel van de tutorial)
            //user=Gynnad&scode=1q2w3e4r (dit is wat de weburl zelf gebruikt)
            writer.close();

            // Now establish a buffered reader to read the URLConnection's input stream.
            BufferedReader reader = new BufferedReader(new InputStreamReader(connect.getInputStream()));

            String lineRead = "";

            // Read all available lines of data from the URL and print them to screen.
            while ((lineRead = reader.readLine()) != null) {
                System.out.println(lineRead);
            }

            reader.close();
        }
            catch (Exception ex) {
                System.out.println("There was an error reading or writing to the URL: " + ex.getMessage());
            }
        }
    }

and this was the result of that code:

<?xml version="1.0" encoding="utf-8"?>
<maxi-xml>
  <error>Login failed</error>
</maxi-xml>

Can someone help me with a explanation/tutorial or something?

Thanks!

[Ulf Dittmer]

If the login works via GET, why are you trying to use POST? Do you have any indication that that should work?

[Rob Spoor]

The page apparently doesn't accept POST requests; it does accept GET requests. Just put the parameters in the URL:

new URL("http://www.maxithlon.com/maxi-xml/login.php?user=Gynnad&scode=1q2w3e4r");

Usually I'd suggest to not post read passwords, but in this case we wouldn't be able to test any solutions without it. I do suggest you to change it as soon as possible after finding a proper solution though.

[Gynnad Paullussen]

Rob Spoor wrote:The page apparently doesn't accept POST requests; it does accept GET requests. Just put the parameters in the URL:

new URL("http://www.maxithlon.com/maxi-xml/login.php?user=Gynnad&scode=1q2w3e4r");

Usually I'd suggest to not post read passwords, but in this case we wouldn't be able to test any solutions without it. I do suggest you to change it as soon as possible after finding a proper solution though.

/*
 * To change this template, choose Tools | Templates
 * and open the template in the editor.
 */

package Connection;

/**
 *
 * @author Danny
 */
import java.net.*;
import java.io.*;

	public class ConnectToURL {

	    // Variables to hold the URL object and its connection to that URL.
	    private static URL URLObj;
	    private static URLConnection connect;

    public static void main(String[] args) {
        try {
            // Establish a URL and open a connection to it. Set it to output mode.
            //login link: http://www.maxithlon.com/maxi-xml/login.php?user=Gynnad&scode=1q2w3e4r
            URLObj = new URL("http://www.maxithlon.com/maxi-xml/login.php?user=Gynnad&scode=1q2w3e4r");
            connect = URLObj.openConnection();
            connect.setDoOutput(true);
       }
        catch (MalformedURLException ex) {
            System.out.println("The URL specified was unable to be parsed or uses an invalid protocol. Please try again.");
            System.exit(1);
        }
        catch (Exception ex) {
            System.out.println("An exception occurred. " + ex.getMessage());
            System.exit(1);
        }


      try {
//            // Create a buffered writer to the URLConnection's output stream and write our forms parameters.
//            BufferedWriter writer = new BufferedWriter(new OutputStreamWriter(connect.getOutputStream()));
//            writer.write("user=Gynnad&scode=1q2w3e4r");
//            //username=MyUsername&pass=MyPassword&submit=Login (Origineel van de tutorial)
//            //user=Gynnad&scode=1q2w3e4r (dit is wat de weburl zelf gebruikt)
//            writer.close();

            // Now establish a buffered reader to read the URLConnection's input stream.
            BufferedReader reader = new BufferedReader(new InputStreamReader(connect.getInputStream()));

            String lineRead = "";

            // Read all available lines of data from the URL and print them to screen.
            while ((lineRead = reader.readLine()) != null) {
                System.out.println(lineRead);
            }

            reader.close();
        }
	        catch (Exception ex) {
	            System.out.println("There was an error reading or writing to the URL: " + ex.getMessage());
	        }
	    }
	}

Now it works to login, now i'm going to set some cookies so I can go to another website and get the content of that page. I will post that also here. If you have some comments please give them so I can learn about it!

The password is not the real password of the page, it is a securitycode and you can only access the pages with the xml, not the game.

[Campbell Ritchie]

Welcome to the Ranch

That question looks too difficult for us here on "beginning"; shall move it. Not sure whether to "servlets" in view of POST and GET, maybe I'll try "in general" and let Rob move it again if he thinks appropriate.

[Rob Spoor]

Nah, it's good here. But you could also have considered Sockets and Internet Protocols

[Gynnad Paullussen]

    public static void signIn() throws MalformedURLException {
        try {
            URL myUrl = new URL("http://www.maxithlon.com/maxi-xml/login.php?user=Gynnad&scode=1q2w3e4r");
            URLConnection urlConn = myUrl.openConnection();
            urlConn.connect();

            String headerName = null;
            for (int i = 1; (headerName = urlConn.getHeaderFieldKey(i)) != null; i++) {
                if (headerName.equals("Set-Cookie")) {
                    String cookie = urlConn.getHeaderField(i);
                    cookie = cookie.substring(0, cookie.indexOf(";"));
                    System.out.println("cookie=" +cookie);
                    String cookieName = cookie.substring(0, cookie.indexOf("="));
                    String cookieValue = cookie.substring(cookie.indexOf("=") + 1, cookie.length());
                    System.out.println("cookieName=" +cookieName);
                    System.out.println("cookieValue=" +cookieValue);                    
                }

            }


        } catch (Exception e) {
            e.printStackTrace();
        }

    }

Now I have this, with this output:

cookie=xml=ucahtb0gg1ue2vj7159bhrchv5
cookieName=xml
cookieValue=ucahtb0gg1ue2vj7159bhrchv5
cookie=xml=onnt9km0us21inoof5t6q7g091
cookieName=xml
cookieValue=onnt9km0us21inoof5t6q7g091

So I understand this page has 2 cookies, but witch one should I use and how can I go now to another webpage?

[Gynnad Paullussen]

public static void signIn() throws MalformedURLException {
        try {
            URL myUrl = new URL("http://www.maxithlon.com/maxi-xml/login.php?user=Gynnad&scode=1q2w3e4r");
            URLConnection urlConn = myUrl.openConnection();
            urlConn.connect();

            String headerName = null;
            for (int i = 1; (headerName = urlConn.getHeaderFieldKey(i)) != null; i++) {
                if (headerName.equals("Set-Cookie")) {
                    String cookie = urlConn.getHeaderField(i);
                    cookie = cookie.substring(0, cookie.indexOf(";"));
                    System.out.println("cookie=" + cookie);
                    String cookieName = cookie.substring(0, cookie.indexOf("="));
                    String cookieValue = cookie.substring(cookie.indexOf("=") + 1, cookie.length());
                    System.out.println("cookieName=" + cookieName);
                    System.out.println("cookieValue=" + cookieValue);

                    URL anotherUrl = new URL("http://www.maxithlon.com/maxi-xml/world.php");
                    URLConnection connection = anotherUrl.openConnection();
                    connection.setRequestProperty("Cookie", cookie);
                    BufferedReader reader = new BufferedReader(new InputStreamReader(connection.getInputStream()));

                    String lineRead = "";

                    // Read all available lines of data from the URL and print them to screen.
                    while ((lineRead = reader.readLine()) != null) {
                        System.out.println(lineRead);
                    }

                    reader.close();
                }


            }




        } catch (Exception e) {
            e.printStackTrace();
        }

    }

Ok, with the second cookie the connection works! So now I will make the code a bit clearer and only use the 2th cookie. Someone has any tips for me?

[Gynnad Paullussen]

How can I now see if the user is sign in? Because the website returns always a Cookie. So I can't check if the cookie is null.