Reg Manual Creation of new Topics -

[Migrated From Jforum.net]

Hi,
I am using jforum with tomcat 6.0 and mysql 4.1. We have a requirement wherein there is a need to create certain topics manually by inserting records into the appropriate tables in JForum. I have analyzed the jforum table structure in the database and I have written a small UNIX shell script which will create a new topic by inserting records into the jforum database. Now I have two problems-

1) The newly created topic is not visible in the web pages, unless I restart tomcat. I have tried all the three connection types - SimpleConnection, PooledConnection and DataSource Connection. My MYSQL transaction isolation level is READ-Committed.
2)The topic created via shell script is not indexed and hence is not searchable.

I am sure that I am creating a topic properly, because it shows as a regular topic after I restart tomcat. I mean, I am inserting records into the jforum database in the right places.

I need to make sure that the topic created via shell script is visible in the web pages, immediately and it is also indexed.

The indexing does not happen even if I manually use the command line tool to re-index.

Will changing the storage engine of jforum database from InnoDB to MYISAM help here? If yes, will it have any other detrimental effects?

Any other ideas?


[originally posted on jforum.net by bbharath]

[Migrated From Jforum.net]

For performance reasons, jForum cache's a lot of topic information in memory. So, just updating the database will not update the memory cache. That's why you don't see them until you restart Tomcat.

As you noticed, doing this also skips a lot of other normal actions that occur if you go thru the software, e.g. indexing, sending out e-mail notices, and the like.

The best way to automatically post things is to use "browser" code, like apache's httpClient java extension to use the standard jForum posting mechanism. This will update the memory cache, send out e-mails to people watching the forum, and the like.
[originally posted on jforum.net by monroe]

[Migrated From Jforum.net]

Thanks Monroe.

If I understood you correctly, the "standard jforum posting" mechanism, consists of accessing a URL (with appropriate parameters) , which creates a new post/topic.

Correct me if I am wrong.

Further. can you please elaborate on the url that needs to be accessed particularly "to create a new topic". My understanding is that we will need
1) user name, fro creating the topic
2) topic title
3) Topic subject

Assuming I have installed jforum on www.abc.com/jforum.

Specifically, I need information on how to go about setting these parameters in a HTTP request. I have questions like -
1) what is the URL that needs to accessed?
2)What are the names of the parameters to be set?
3)Should they be set as cookies, or sent as a GET or POST request?
4)If any cookies need to be set, what are their names?

Any online documentation regarding the above will be really useful.


Just out of curiosity - is there any way in which I can disable jforum from caching results and force it go to the database each time? I understand that this will have considerable amount of overhead, but I just need to know if it can be done?
[originally posted on jforum.net by bbharath]

[Migrated From Jforum.net]

It's simply a matter of looking at the HTML Forms and actions. Then dealing with the cookies returned.

To make a new post manually, you need to:

1) Login
2)Fill out and submit the new topic form.

Let's look at 1). If you view at the HTML source for the login screen, you will see that that there is a form with an action URL, hidden fields, and visible fields for user and password.

To programatically log in, you need to use something like the HTTPClient PostMethod to create the same input that a user would. Something like the code below but done so it matches the jForum login form:

        PostMethod post = new PostMethod("http://jakarata.apache.org/");
        NameValuePair[] data = {
          new NameValuePair("user", "joe"),
          new NameValuePair("password", "bloggs")
        };
        post.setRequestBody(data);
        // execute method and handle any error responses.
        ...
        InputStream in = post.getResponseBodyAsStream();
        // handle response.

Then you need to deal with the information that has been returned. In particular, you need to get the session id associated with that login. Generally this is done by your Appserver returning a session cookie with this information. The HTTP client package as ways to get this from the response.

To do 2), just look at the new topic form's action and fields. Then generate a new post request with your new topic information, including the Cookies received from 1).

If you need to do more than one topic, just keep using the session cookie and doing more posts.

If you want, you can pro grammatically log out the user, but since the session will time out, this is not needed.
[originally posted on jforum.net by monroe]

[Migrated From Jforum.net]

Hi Monroe,

I've been trying your piece of code above, but it doesn't seem to work for me.

This is what I have for the moment:

public void doPost(HttpServletRequest request, HttpServletResponse response)
			throws ServletException, IOException {
		...

		String loginUrl = "http://localhost:8080/jforum/jforum.page?module=user&action=login&returnPath="
				+ "jforum/jforum.page";
		
		String url = "http://localhost:8080/jforum/jforum.page?"
				+ "module=adminCategories" + "&action=insertSave"
				+ "&category_name=" + categoryName + "&moderate=0"
				+ "&groups=1" + "&groups=2" + "&groups=3" + "&groups=4";

		...

		// Create an instance of HttpClient.
		HttpClient client = new HttpClient();

		// Create a method instance.
		PostMethod method = new PostMethod(loginUrl);
		NameValuePair[] data = { new NameValuePair("user", "admin"),
				new NameValuePair("password", "adminpw") };
		method.setRequestBody(data);

		try {
			int statusCode = client.executeMethod(method);

                        if (statusCode != HttpStatus.SC_OK) {  
                             out.println("Method failed: " + method.getStatusLine());  
                        }		

//			byte[] responseBody = method.getResponseBody();
			InputStream in = method.getResponseBodyAsStream(); 
			// Deal with the response.
			// Use caution: ensure correct character encoding and is not binary
			// data
			out.println(in.toString());

		} catch (HttpException e) {
			out.println("Fatal protocol violation: " + e.getMessage());
			e.printStackTrace();
		} catch (IOException e) {
			out.println("Fatal transport error: " + e.getMessage());
			e.printStackTrace();
		} finally {
			method.releaseConnection();
		}
		out.println("  </BODY>");
		out.println("</HTML>");
		out.flush();
		out.close();

	}

When I would execute this code with the loginUrl I'd just be shown the loginpage, but not be logged in myself.

Am I missing something? Am I using the incorrect url to login?

How exactly do you need to login in JForum programmatically? I've already tried several other options. You can see that in another topic I responded in.

Any help would be much appreciated. Thx!
[originally posted on jforum.net by jeetn]

[Migrated From Jforum.net]

monroe wrote:It's simply a matter of looking at the HTML Forms and actions. Then dealing with the cookies returned.

To make a new post manually, you need to:

1) Login
2)Fill out and submit the new topic form.

Let's look at 1). If you view at the HTML source for the login screen, you will see that that there is a form with an action URL, hidden fields, and visible fields for user and password.

To programatically log in, you need to use something like the HTTPClient PostMethod to create the same input that a user would. Something like the code below but done so it matches the jForum login form:

        PostMethod post = new PostMethod("http://jakarata.apache.org/");
        NameValuePair[] data = {
          new NameValuePair("user", "joe"),
          new NameValuePair("password", "bloggs")
        };
        post.setRequestBody(data);
        // execute method and handle any error responses.
        ...
        InputStream in = post.getResponseBodyAsStream();
        // handle response.

Then you need to deal with the information that has been returned. In particular, you need to get the session id associated with that login. Generally this is done by your Appserver returning a session cookie with this information. The HTTP client package as ways to get this from the response.

To do 2), just look at the new topic form's action and fields. Then generate a new post request with your new topic information, including the Cookies received from 1).

If you need to do more than one topic, just keep using the session cookie and doing more posts.

If you want, you can pro grammatically log out the user, but since the session will time out, this is not needed.

How do you retrieve the cookie with the session status?

When I use client, I can't find anything to retrieve a cookie or session. The same with response. In response I can only add a cookie.

I have noticed something though. When I execute my code and use the loginUrl, I'm shown the login page of JForum but the url is the one of my Servlet. When I hover over the login button and look at the link (left bottom) I see a sessionId has been added to the link.

Really could use some help here :s Tried several things and I always keep getting the Login page :s
[originally posted on jforum.net by jeetn]

[Migrated From Jforum.net]

If I remember right, the HTTPClient object should maintain cookies from request to request as long as you are using the same client object. This is stored in the HTTPState object. So, you can get the Cookies passed back by doing a client.getState().getCookies() call.

If the state cookies aren't there, it might be because the default cookie parser doesn't match the one your server is sending back. See the HTTPClient docs and sample programs about setting a different parser.
[originally posted on jforum.net by monroe]

[Migrated From Jforum.net]

Hi Monroe,


Thanks for your reply. I did find the cookies now. And tested it, but still doesn't work.

Went looking on google then to find some more info and came across some examples. But that didn't help me either.

This is what I have now:

public void doPost(HttpServletRequest request, HttpServletResponse response)
			throws ServletException, IOException {

		response.setContentType("text/html");
		String categoryName = request.getParameter("category");
		PrintWriter out = response.getWriter();
		String contents = "";

		String loginUrl = "http://localhost:8080/jforum/user/login.page";

		String url = "http://localhost:8080/jforum/jforum.page?"
				+ "module=adminCategories" + "&action=insertSave"
				+ "&category_name=" + categoryName + "&moderate=0"
				+ "&groups=1" + "&groups=2" + "&groups=3" + "&groups=4";

		out
				.println("<!DOCTYPE HTML PUBLIC \"-//W3C//DTD HTML 4.01 Transitional//EN\">");
		out.println("<HTML>");
		out.println("  <HEAD><TITLE>A Servlet</TITLE></HEAD>");
		out.println("  <BODY>");
		// Create an instance of HttpClient.
		HttpState initialState = new HttpState();

		HttpClient client = new HttpClient();
		client.getParams().setCookiePolicy(CookiePolicy.BROWSER_COMPATIBILITY);
		client.setState(initialState);

		// Create a method instance.
		GetMethod get = new GetMethod(loginUrl);

		int result = client.executeMethod(get);

		Cookie[] cookies = client.getState().getCookies();

		System.out.println("Present cookies: ");
		for (int i = 0; i < cookies.length; i++) {
			System.out.println(" - " + cookies[i].toExternalForm());
		}

		get.releaseConnection();

		PostMethod method = new PostMethod(loginUrl);

		NameValuePair[] data = { new NameValuePair("username", "admin"),
				new NameValuePair("password", "adminpw") };
		method.setRequestBody(data);
		for (int i = 0; i < cookies.length; i++) {
			method.setRequestHeader("Cookie:", cookies[i].toExternalForm());
		}

		// method.setDoAuthentication(true);
		try {
			// Execute the method.
			int statusCode = client.executeMethod(method);

			method = new PostMethod(url);
			NameValuePair[] data2 = { new NameValuePair("username", "admin"),
					new NameValuePair("password", "adminpw") };
			method.setRequestBody(data2);
			for (int i = 0; i < cookies.length; i++) {
				method.setRequestHeader("Cookie:", cookies[i].toExternalForm());
			}
			statusCode = client.executeMethod(method);

			if (statusCode != HttpStatus.SC_OK) {
				if (statusCode == HttpStatus.SC_MOVED_TEMPORARILY) {
					response.sendRedirect(url);
				}
				out.println("Method failed: " + method.getStatusLine());
			}

			contents = method.getResponseBodyAsString();

			method.releaseConnection();
			out.println(contents);
		} catch (HttpException e) {
			out.println("Fatal protocol violation: " + e.getMessage());
			e.printStackTrace();
		} catch (IOException e) {
			out.println("Fatal transport error: " + e.getMessage());
			e.printStackTrace();
		} finally {

		}

		out.println("  </BODY>");
		out.println("</HTML>");
		out.flush();
		out.close();

	}

[originally posted on jforum.net by jeetn]

[Migrated From Jforum.net]

Hi,

Just to inform everyone that it worked for me

This is my final code:

public void doPost(HttpServletRequest request, HttpServletResponse response)
			throws ServletException, IOException {

		response.setContentType("text/html");
		String categoryName = request.getParameter("category");
		PrintWriter out = response.getWriter();
		String contents = "";

		String loginUrl = "http://localhost:8080/jforum/user/login.page";

		String url = "http://localhost:8080/jforum/jforum.page?"
				+ "module=adminCategories" + "&action=insertSave"
				+ "&category_name=" + categoryName + "&moderate=0"
				+ "&groups=1" + "&groups=2" + "&groups=3" + "&groups=4";

		out
				.println("<!DOCTYPE HTML PUBLIC \"-//W3C//DTD HTML 4.01 Transitional//EN\">");
		out.println("<HTML>");
		out.println("  <HEAD><TITLE>A Servlet</TITLE></HEAD>");
		out.println("  <BODY>");
		// Create an instance of HttpClient.
		HttpState initialState = new HttpState();

		HttpClient client = new HttpClient();
		client.getParams().setCookiePolicy(CookiePolicy.BROWSER_COMPATIBILITY);
		client.setState(initialState);

		// Create a method instance.
		GetMethod get = new GetMethod(loginUrl);

		int result = client.executeMethod(get);

		Cookie[] cookies = client.getState().getCookies();

		System.out.println("Present cookies: ");
		for (int i = 0; i < cookies.length; i++) {
			System.out.println(" - " + cookies[i].toExternalForm());
		}

		get.releaseConnection();

		PostMethod method = new PostMethod(loginUrl);
		method.addParameter("username", "admin");
		method.addParameter("password", "adminpw");
		method.addParameter("module", "user");
		method.addParameter("action", "validateLogin");
		method.addParameter("returnPath", "http://localhost:8080/jforum/forums/list.page");
		method.addParameter("login", "Login");

		for (int i = 0; i < cookies.length; i++) {
			method.setRequestHeader("Cookie:", cookies[i].toExternalForm());
		}
		
		try {
			// Execute the method.
			int statusCode = client.executeMethod(method);
			String s = request.getParameter("returnPath");
			
			if(statusCode == HttpStatus.SC_MOVED_TEMPORARILY){
				response.sendRedirect("http://localhost:8080/jforum/forums/list.page");
			}

			method = new PostMethod(url);

			for (int i = 0; i < cookies.length; i++) {
				method.setRequestHeader("Cookie:", cookies[i].toExternalForm());
			}
			statusCode = client.executeMethod(method);
			
			if(statusCode == HttpStatus.SC_MOVED_TEMPORARILY){
				response.sendRedirect("http://localhost:8080/jforum/admBase/login.page");
			}

			contents = method.getResponseBodyAsString();

			method.releaseConnection();
			out.println(contents);
		} catch (HttpException e) {
			out.println("Fatal protocol violation: " + e.getMessage());
			e.printStackTrace();
		} catch (IOException e) {
			out.println("Fatal transport error: " + e.getMessage());
			e.printStackTrace();
		} finally {

		}

		out.println("  </BODY>");
		out.println("</HTML>");
		out.flush();
		out.close();

	}

What you need to do is add every parameter you find on the page that has a value. So every input field with a value needs to be added to the method. This only if you want to login!

After that just keep using the cookie to do whatever you want
[originally posted on jforum.net by jeetn]

[Migrated From Jforum.net]

Not sure I understand exactly what you are trying to do. Is this what the subject line says, e.g. Automatically create a topic?

Or are you trying to do some kind of automatic login from a different web app?

If it's the latter, you'll need to pass the cookies from the internal request back to the browser making the original request. And this has to be done BEFORE the request object is committed (see the specs for why).
[originally posted on jforum.net by monroe]

[Migrated From Jforum.net]

Hi,

The problem is already solved ;)

What I was trying to do was create a category from a different web app.

I had a jsp in which I had to fill in the category name. When I pressed the submit button, I needed to be logged in to JForum and then execute the url which add's the category.

With above code, this works ;)

Thanks for all your help
[originally posted on jforum.net by jeetn]