RMI + Thread: perform searching on different worker PCs

[fadzizo abdullah]

hi all

im developing a client/server application to perform data searching on some workers' database (some PCs)...on PC A i use RMI to call remote objects on PC B and PC C... PC A actually contains a Servlet program...i managed to implement this on one-to-one basis... now, im trying to use Thread on PC A so that it can make concurrent call to PC B and PC C....i don't want my program on PC A to call remote objects from the other two PCs sequentially....im intend to develop something like multiprocessing application...

So guys, do you think im doing the right thing...?? any ideas or opinions about my above intentions...?? Below are my programs...it worked fine JUST FOR the first running time....when i tried to call the Servlet again while it's still running, it's not working....

here is my program on PC A:

public class StoreServlet extends HttpServlet implements Runnable
{
	Thread searcher;
	NameInterface db1, db2, db3;
 
  	public void init() throws ServletException
  	{
		searcher = new Thread(this);
		searcher.setPriority(Thread.MIN_PRIORITY);
  	}
 
  	public void run()
  	{
		try
		{
			db1.printRecord("Thread 1",1,3);
			db2.printRecord("Thread 2",2,3);
			db3.printRecord("Thread 3",3,3);
		}
		catch(Exception e)
		{
			e.printStackTrace();
		}
	}
 
	public void destroy()
	{
		searcher = null;
	}
 
	public void doGet(HttpServletRequest req, HttpServletResponse res)throws ServletException, IOException
	{
		doPost(req,res);
	}
 
	public void doPost(HttpServletRequest req, HttpServletResponse res)throws ServletException, IOException
	{
		res.setContentType("text/plain");
		PrintWriter out = null;
		String msg = null;
 
		try
		{
			DataInputStream in = new DataInputStream((InputStream)req.getInputStream());
			out = res.getWriter();
 
			db1 = (NameInterface)Naming.lookup("rmi://localhost:1313/Server1");
			db2 = (NameInterface)Naming.lookup("rmi://localhost:1313/Server2");
			db3 = (NameInterface)Naming.lookup("rmi://localhost:1313/Server3");
			searcher.start();
			searcher = null;
 
 
                        String message = "Success!";
			res.setContentLength(message.length());
			out.println(message);
 
			out.flush();
			out.close();
		}
		catch(Exception e)
		{
			e.printStackTrace();
		}
	} // doPost
} // class

This is my program on PC B:

import java.net.MalformedURLException;
import java.rmi.*;
import java.rmi.registry.LocateRegistry;
import java.rmi.registry.Registry;
import java.rmi.server.*;
import java.sql.*;
import java.io.*;
 
// server class implementation
public class NameServer extends UnicastRemoteObject implements NameInterface
{
    private static Connection con = null;
 
    // constructor
    public NameServer() throws RemoteException, SQLException
    {
    	// connection to the database;
    	System.out.println("Server is up....");
	}
 
    public void printRecord(String name, int bil, int totalw) throws RemoteException, SQLException, IOException
    {
		System.out.println( "Starting thread: " + name );
 
    	Statement stt = null;
    	ResultSet rst = null;
 
      	stt = con.createStatement();
      	rst = stt.executeQuery("SELECT * FROM gambar");
 
      	rst.last();
      	int numrow = rst.getRow();
 
      	rst.first();
      	int start = 0;
      	int stop = 0;
      	if(bil == 1)
      		stop = numrow / totalw;
      	else if(bil == 2)
      	{
      		start = numrow / totalw;
      		stop = (numrow / totalw) * 2;
      		for(int i=0; i<start; i++)
      			rst.next();
		}
      	else
      	{
      		start = (numrow / totalw) * 2;
      		stop = numrow;
      		for(int i=0; i><start; i++)
      			rst.next();
		}
 
      	int count = 1;
      	for(int i = start; i><stop; i++)
      	{
        	int gamid = rst.getInt(1);
           	byte[] gampic = rst.getBytes(2);
 
			System.out.print("\n"+name+"\n");
			System.out.println("\t"+count+".");
        	System.out.println("\tPic ID	: "+gamid);
        	System.out.println("\tPic	: "+gampic);
 
        	rst.next();
 
        	count++;
      	}
 
      	System.out.println( "\nThreads started, main ends\n" );
	}
 
	// main method
    public static void main(String[]args)
    {
        try {
            NameServer objServer = new NameServer();
 
        	Class.forName("com.mysql.jdbc.Driver").newInstance();
 
        	con = DriverManager.getConnection("jdbc:mysql:///photo","root", "aku113");
 
            Registry reg = LocateRegistry.createRegistry(1313);
            reg.rebind("Server1", objServer);
 
			System.out.println("Server Ready");
        }
		catch (IllegalAccessException ex)
        {   ex.printStackTrace();	}
        catch (InstantiationException ex)
        {   ex.printStackTrace();	}
        catch (SQLException ex)
        {   ex.printStackTrace();	}
        catch (ClassNotFoundException ex)
		{   ex.printStackTrace();	}
        catch(RemoteException ex)
        {   ex.printStackTrace();	}
    } // main
} // class

>

[Ajay Saxena]

I see several issues in your program.

1> Why do you want to make the Servlet a runnable object. Spawning a thread in the method of an object passing a reference to the same object as a runnable is an extremely convoluted way of programming which is easily avoidable.

2> Secondly, your RMI client (the Servlet's doPost() method) doesn't invoke the three RMI servers in different threads. It calls all of them sequentially in a separate thread ,the searcher (Line no 50).

3> It appears from your code that you intend to use the servlet to actually invoke multiple RMI servers concurrently,and then cook up a response message once all the threads are done with their work. The last point is important.This would mean that you would want the doPost() method of the servlet to block till all the threads are done.


My approach would be the following...

- Write a Class say RMICaller,that implements runnable,and in the run() method of the same invoke the RMI server. Of course this class would have member variables that capture the RMI server location info.

public class RMICaller implements Runnable{

    NameInterface db;
    String threadNo;
    int bil;
    int totalw;
    
    public RMICaller(NameInterface db,String threadNo,int bil,int totalw){
        this.db = db;
        this.threadNo = threadNo;
        this.bil = bil;
        this.totalw=totalw;
    }
 
    public void run(){

      try  
       {  
            db.printRecord(threadNo,bil,totalw);  
        }  
        catch(Exception e)  
        {  
            e.printStackTrace();  
        }  
        
        
    }
}

- Keep the servlet clean ( Don't make it implement Runnable or maintain any member variable searcher)

In the doPost() method, construct the RMICaller objects and hand each of them off to a new thread.

- Now ,the most important part. You need to make the doPost() method block till the RMIcaller threads are done. Figuring out how to do this would be a good exercise

[William Brogden]

If this was my problem I would solve it with a JavaSpaces approach.

Javaspaces conceals all the details of RMI etc. and lets you work with more generalized tools.

On the other hand, you will learn a lot by doing all the detail work yourself.

Bill

[fadzizo abdullah]

thanks guys...

Ajay...

do you mean:
1. i need to write the class RMICaller in the servlet and then invoke the class within the servlet..? or
2. no need to use the servlet, instead just use the class RMICaller...?

actually i still need to use the servlet....it is a compulsory part of my system...

[Ajay Saxena]

I didn't suggest avoiding servlets altogether. RMICaller is of course a different class that implements runnable.

And,as I said,you may want to construct the RMICaller objects and hand each of them to a new thread in the doPost() method of the servlets. Take a look at my last post again.

[fadzizo abdullah]

hi Ajay.... thanks for your quick reply....

ok here are the codes that i have modified just now...is this what you mean?

public class StoreServlet extends HttpServlet implements Runnable
{
	public void doGet(HttpServletRequest req, HttpServletResponse res)throws ServletException, IOException
	{
		doPost(req,res);
	}

	public void doPost(HttpServletRequest req, HttpServletResponse res)throws ServletException, IOException
	{
		res.setContentType("text/plain");
		PrintWriter out = null;
		String msg = null;

		try
		{
			DataInputStream in = new DataInputStream((InputStream)req.getInputStream());
			out = res.getWriter();

			RMICaller rmi1 = new RMICaller(Naming.lookup("rmi://localhost:1313/Server1"), 1,3);
			RMICaller rmi2 = new RMICaller(Naming.lookup("rmi://localhost:1313/Server3"), 2,3);
			RMICaller rmi3 = new RMICaller(Naming.lookup("rmi://localhost:1313/Server3"), 3,3);
			
			rmi1.start();
			rmi2.start();
			rmi3.start();

			String message = "Success!";
			res.setContentLength(message.length());
			out.println(message);

			out.flush();
			out.close();
		}
		catch(Exception e)
		{
			e.printStackTrace();
		}
	} // doPost
} // class

class RMICaller implements Runnable{  
    NameInterface db;  
    String threadNo;  
    int bil;  
    int totalw;  
      
    public RMICaller(NameInterface db,String threadNo,int bil,int totalw){  
        this.db = db;  
        this.threadNo = threadNo;  
        this.bil = bil;  
        this.totalw=totalw;  
    }  
   
    public void run(){  
  
      try    
       {    
            db.printRecord(threadNo,bil,totalw);    
        }    
        catch(Exception e)    
        {    
            e.printStackTrace();    
        }                        
    }  
}  

[Ajay Saxena]

You got it right.

However, I guess you would want the parent thread (the thread spawning the RMI caller threads) to remain suspended till all the RMI callers are done with their tasks,and then send the response message back.

You would have to figure this part out.

[fadzizo abdullah]

thanks Ajay

emmm i think i need to use wait() in order to suspend and wait till all the threads are done...ok, i'll find it first and once i found it i'll post it here for you to give guidance ok...

but before that, i got two things to ask you that keep me wondering about:
1. are the people out there use the same way im using now (servlet + rmi + multithread)?
2. is my method of combining the servlet + RMI + multithreading good for my system as mentioned in the first post above (multiprocessed-based searching application using servlet)? any other better alternative?

thanks Ajay, may God bless you...

[Ajay Saxena]

1. are the people out there use the same way im using now (servlet + rmi + multithread)?

It depends on the architecture of the existing system you want to build up on. Your system seems to mandate a web interface which explains why you have a servlet in the first place. However I'm still wondering why you chose to have an RMI server as an intermediary (between the servlet container and the database) to access the DB.

You could have a simple DataAccess class similar to the RMICaller class that could directly read the DB,instead of routing the access through an RMI server.No remote JVMs required at all.

2. is my method of combining the servlet + RMI + multithreading good for my system

You need tell us more about the existing system architecture,and what you are trying to achieve through the new design.

[fadzizo abdullah]

hi Ajay

thanks for your previous reply....ok, it's true that i can just use a servlet alone without the rmi part...but the answers why im using RMI are:

1. my database will be located in several servers (workers) or in another word i have replicated databases (same records in all databases in all workers).

2. im using servlet as a communication service between my client and rmi workers.

3. im using multithreading to enhance searching performance. Searching will be using horizontal partitioning. Client will request something to be searched, received by the servlet, broadcast the request to all workers (to me this is the multithreading part of what we have discussed yesterday). The first worker returns with the data will be accepted (don't know how to do this part yet ), the data will be sent to the client, and the searching processes will be stopped for the rest of the wokers (don't know how to do this part yet).

4. so, my architecture would be Client <-> multithreaded Servlet <-> RMI <-> Database

Ajay, i would really appreciate if you could give some comments and guidance of what im trying to do...thanks in advance...

[Ajay Saxena]

The fact that you have replicated DBs distributed across a host of machines doesn't mandate use of RMI. What you are achieving through RMI is plain DB access.This could be done directly from the servlet itself. Just spawn multiple threads handing off a DataAccess object to each.

And in the run() method of each DataAccess object establish DB connectivity through JDBC directly.That is, just put the code of NameInterface.printRecord() directly in DataAccess.run().

[fadzizo abdullah]

hi Ajay....thanks for your concern....

is this what you mean...?

 
public class StoreServlet extends HttpServlet
{

	public void doGet(HttpServletRequest req, HttpServletResponse res)throws ServletException, IOException
    {
    	doPost(req,res);
    }

    public void doPost(HttpServletRequest req, HttpServletResponse res)throws ServletException, IOException
    {
    	res.setContentType("text/plain");
        PrintWriter out = null;
        String msg = null;

        try
        {
    	    DataInputStream in = new DataInputStream((InputStream)req.getInputStream());
            out = res.getWriter();

            // instantiate DataAccess objects
            DataAccess da1 = new DataAccess("worker1","Thread 1",1,3);
            DataAccess da2 = new DataAccess("worker2","Thread 2",2,3);
            DataAccess da3 = new DataAccess("worker3","Thread 3",3,3);

			// instantiate Thread objects
			Thread thda1 = new Thread(da1);
			Thread thda2 = new Thread(da2);
			Thread thda3 = new Thread(da3);

			// start the threads
			thda1.start();
			thda2.start();
			thda3.start();

            String message = "Success!";
            res.setContentLength(message.length());
            out.println(message);

            out.flush();
            out.close();
        }
        catch(Exception e)
        {
            e.printStackTrace();
        }
    } // doPost
} // class

class DataAccess implements Runnable
{
	String workerName, threadNo;
    int bil;
    int totalw;

    public DataAccess(String workerName, String threadNo,int bil,int totalw)
    {
    	this.workerName = workerName;
    	this.threadNo = threadNo;
        this.bil = bil;
        this.totalw=totalw;
    }

    public void run()
    {
    	try
        {
        	Class.forName("com.mysql.jdbc.Driver").newInstance();
        	
        	// create connection base on jdbc connectivity of worker's name
        	Connection con = DriverManager.getConnection("jdbc:mysql:///"+workerName,"root", "aku113");

			Statement stt = null;
			ResultSet rst = null;

			stt = con.createStatement();
			rst = stt.executeQuery("SELECT * FROM gambar");

			// database processing here!!
        }
        catch(Exception e)
        {
            e.printStackTrace();
        }
    }
}

[Ajay Saxena]

Yes..This is exactly what I described in my last post.

However,the solution is far from complete as you want the response to be generated when one of the threads is done with its job.
In the current implementation the three threads are being spawned by the parent thread and it's not blocking on the child threads.

Designing a solution where you would like the parent thread to block only till ONE of the threads is done,and then probably interrupt the remaining threads would be tricky.

[fadzizo abdullah]

hi Ajay

emmm ok, so up to this point i've done it correctly right....

alright, i'll work on the next tricky parts...if i got something to ask, i hope you'll be here to guide me again....thanks a million to you Ajay....till then, God bless you always....

[Ajay Saxena]

Would be happy to be able to guide you! All the best

[fadzizo abdullah]

hi Ajay....

i want to ask for your opinion....if i just use servlet without rmi means that i leave the processing burdens all to the servlet only...what if there are so many requests...??

in my thought, if i use rmi i will divide the processing into some parts, lets say i have 4 servers with the same database in each, and users request to do something, so 1st server do the 1st quarter, 2nd server for 2nd quarter, and so on....now, one problem divided into four worker instead of using on worker only (the servlet only).....

thanks in advance...

[Ajay Saxena]

Consider deploying the web application in a cluster of web servers with a Load balancer to distribute the web requests. A common example would be an apache Load balancer in front of a cluster of Tomcat web containers.

[fadzizo abdullah]

hi Ajay...

a bit late reply from me since i got flu (not that H1N1 ) these four days...now im recovering....but stil quite dizzy...

so you mean that it is enough with that one server (servlet app) to service all requests....?? and yet i can still have a distributed architecture where there can be some databases resided in several computers serviced by that single server (servlet app)..??

im sorry if i got it wrong.....thanks in advance

[Ajay Saxena]

What I mean is that you can configure a cluster (group) of web containers(servers) ,each hosting the same web (servlet based) application. And you can distribute the client requests across these servers using a Load balancer,may be the Apache LB. Note this architecture serves to distribute only the client request load across the server nodes. Each request will again perform the same action,that of spawning multiple DataAccess objects to hit multiple DB instances.

Read up Load balancing and Clustering in the context of web applications. Do some research on Tomcat clustering.

[fadzizo abdullah]

hi Ajay...

thanks for your reply and information...

back to our previous discussion....

Ajay Saxena wrote:
However,the solution is far from complete as you want the response to be generated when one of the threads is done with its job.
In the current implementation the three threads are being spawned by the parent thread and it's not blocking on the child threads.

Designing a solution where you would like the parent thread to block only till ONE of the threads is done,and then probably interrupt the remaining threads would be tricky.

i think i have found a solution, but i need your comments...i put it this way....i used a while loop to check whether or not the thread has done...if done, i called a getPwd() which returns a data of the process done in the thread:

   
public void doPost(HttpServletRequest req, HttpServletResponse res)throws ServletException, IOException
    {
    	:
        :

        try
        {
    	    DataInputStream in = new DataInputStream((InputStream)req.getInputStream());
            out = res.getWriter();

            String logusn = in.readUTF();

            // instantiate DataAccess objects
            DataAccess da1 = new DataAccess("10.62.255.13/photo2",logusn,1,2);
            DataAccess da2 = new DataAccess("localhost/photo",logusn,2,2);

            
			// instantiate Thread objects
			Thread thda1 = new Thread(da1);
			Thread thda2 = new Thread(da2);

			// start the threads
			thda1.start();
			thda2.start();

			// check whether the thread has done processing....then get the result
			while(!da1.isDone() || !da2.isDone())
			{}

			String message = "None";
			if(da1.isDone())
				message = da1.getPwd();
			if(da2.isDone())
				message = da2.getPwd();

			res.setContentLength(message.length());
            out.println(message);

            in.close();
            out.flush();
            out.close();
        }

and, in the class DataAccess i added another variable "done" to indicate whether or not the process has finished:

   
class DataAccess implements Runnable
{
	String workerName, data, logpwd;
    int bil;
    int totalw;
   boolean done = false;

    public DataAccess(String workerName, String data, int bil,int totalw)
    {
    	this.workerName = workerName;
    	this.bil = bil;
        this.totalw = totalw;
        this.data = data;
    }

    public void run()
    {
    	try
        {
			// database processing here!!
			.........

			// update variable done when process is finished
			done = true;
        }
        catch(Exception e)
        {
            e.printStackTrace();
        }
    }// run

    public boolean isDone()
    {
		return done;
	}

    public String getPwd()
    {
		return logpwd;
	}
}// DataAccess

ok, so am i on the right track...?? thanks in advance....

[Ajay Saxena]

You are very close to the solution!
However, I see few issues.

while(!da1.isDone() || !da2.isDone()){}  

Here you loop till both the DA (DataAccess) threads are complete,whereas you said in an earlier post that the parent thread should block till one of the threads completes.

  if(da1.isDone())  
                 message = da1.getPwd();  
             if(da2.isDone())  
                 message = da2.getPwd();  

Note that the parent thread reaches the first 'if' statement only once both the DA threads are done. So both of the 'if' conditions would evaluate to true,and hence your message would be overwritten.

And most importantly, the while loop

while(!da1.isDone() || !da2.isDone()) {}

puts the parent thread in a busy wait wasting CPU resources.Whenever this thread is scheduled by the OS(assuming the JVM uses native threads),the former does nothing but polls the completion status of the two DA threads.

This could be easily eliminated by taking a different approach (use wait-notify)
So net net there are two major issues.

1. Your parent thread loops till BOTH the threads are done.You probably don't want this.

2. Your parent thread does a busy wait which should be avoided.

-thanks
Ajay

[fadzizo abdullah]

Ajay Saxena wrote:You are very close to the solution!

happy to hear that....

ok, i will try my best push up my effort more and more.....i will post again my new progress soon...thanks a million Ajay....

[fadzizo abdullah]

hi Ajay...

emmm im quite confused here...actually i did read about wait() and notify(), but they are all used in synchronizing some threads so that they can work together when accessing the same object....in my case request from a client will be spawned to some servers....means that one thread for one server.... the threads didn't share the same object since they target different servers...

as these lines mean:

     // instantiate DataAccess objects  
             DataAccess da1 = new DataAccess("10.62.255.13/photo2",logusn,1,2);   // SERVER 1
             DataAccess da2 = new DataAccess("localhost/photo",logusn,2,2);           // SERVER 2
   
               
             // instantiate Thread objects  
             Thread thda1 = new Thread(da1);  
             Thread thda2 = new Thread(da2);  
   
             // start the threads  
             thda1.start();  
             thda2.start();  

and, then i need to wait till ONLY ONE of the threads finished WITH A RESULT...as these lines mean (i think the || operator should be changed to &&):

    
			// loop until one of the thread has done processing....then get the result
			while(!da1.isDone() || !da2.isDone())
			{}

once any ONE of the threads finished and the loop stops, i will extract the data from the finished thread by doing this:

			String message = "None";
			if(da1.isDone())
				message = da1.getPwd();
			if(da2.isDone())
				message = da2.getPwd();

please guide me and correct me....thanks in advance...

[Ajay Saxena]

(i think the || operator should be changed to &&):

Correct ,this would do.

However it still leaves the problem of busy wait unsolved.

To answer your question of access to some shared data in a wait-notify situation,I'd say that you can have a member variable in the servlet ,let's say 'complete' ,protect it by some monitor and have the parent thread wait on the same monitor till the condition (complete==true) is true.

The child threads should each have access to this variable and as soon as they are done they would update it to true and also notify the parent thread. To implement this ,you would have to modify the definition of the DataAccess class little bit so that you can pass the monitor that protects the member variable 'complete' in the servlet.

-ATB
Ajay

[fadzizo abdullah]

hi Ajay...good morning and i hope you are fine

it has been quite a long time not messaging each other right...huhhhh so many things happened...

alright, back to our previous discussion....emmm i'm quite confused...did you mean that i should not use the while looping structure that puts my parent thread in a busy state? and, i must change it to another better way as what you have suggested to me?

if that is so, im still not getting the answer of your last post....to terminate/ stop threads other than the finished one...could you please guide me more??

thanks in advance