Thread in clustered environment

[Venkat Sadasivam]

I am writing a scheduler with thread implementation; how do I ensure that only one thread runs in clustered environment?

[Jimmy Clark]

How many JVMs are in the clustered environment? Does the environment contain a Java Application Server?

[Venkat Sadasivam]

The environment contains three Tomcat server with Oracle backend.

[Chris Hurst]

Terracota is open source works with Tomcat and you can achieve that (you can achieve singletons etc, cross clustered locking) ... other solutions do exist ;-) (google away :-) )...

[Venkat Sadasivam]

I did googling before posting the question here.

I am looking for a solution from database perspective, I know quartz scheduler has the implementation I am looking for. But I am not sure how they implemented.

[Jimmy Clark]

Thanks. Based on your reply, there are three machines hosting an instance of Apache Tomcat web server and a fourth machine hosting an Oracle RDBMS. Is this correct?

I am looking for a solution from database perspective

What type of relationship does the RDBMS have with the Java threads?

Are you looking for a way to control the Java threads from logic in the RDBMS?
[ October 19, 2008: Message edited by: James Clark ]

[Venkat Sadasivam]

<<Thanks. Based on your reply, there are three machines hosting an instance of Apache Tomcat web server and a fourth machine hosting an Oracle RDBMS. Is this correct?>>
[Venkat] Yes.

<<What type of relationship does the RDBMS have with the Java threads?>>
[venkat] I am writing a job scheduler the job details are stored in the RDBMS database. Three thread (i.e. scheduler) will be running in three tomcat server. I want to ensure at the same time only one thread does the job scheduling, other thread should wait or should go to sleep mode.

<<Are you looking for a way to control the Java threads from logic in the RDBMS?>>
[venkat] No. More from synchronization perspective. Please see the above answer.

[Jimmy Clark]

I am writing a job scheduler the job details are stored in the RDBMS database.

What role does the Java-based web server play in regards to the Oracle-based scheduler? In other words, how is the scheduler related to the web server?

I think your design will be cleaner if you implement you own POJO-based server. You certainly will have greater control of the threads.

Apache Tomact is a Java web server, designed for handling HTTP requests received from a web browser. It has a web container which manages threads related to deployed web applications. Your Oracle scheduler and whatever Java programs that will be running in the threads is not a "web" application.
[ October 19, 2008: Message edited by: James Clark ]

[Nitesh Kant]

Venkat:
I want to ensure at the same time only one thread does the job scheduling, other thread should wait or should go to sleep mode.

I think you can do away with a lock table in the database.
This table just has a column to indicate whether some thread is working on the job.
After a thread is done with the job, it will set the flag back to "free" so that any other thread can take it up in the next iteration.

A wait-notify paradigm will probably be an overkill in this distributed environment if what you are concerned about is only a periodic job processing and not "as soon as it is available" processing.

James: I think your design will be cleaner if you implement you own POJO-based server. You certainly will have greater control of the threads.

Agree with you James. But probably, being an application vendor, one would be hesitant in creating a new standalone process on the server machine. I feel the predominant reason being the deployment lifecycle of the process in the data center and IT problems around the same.

[Jimmy Clark]

What are we talking about here, using just the JRE that comes shipped with Apache Tomcat to execute batch jobs called from Oracle RDBMS scripts?

As I mentioned earlier, it is unclear where the "web server" comes into the picture? Executing standalone Java programs is much better than twisting this logic into clandestine "servlet" threads, I think.

[Venkat Sadasivam]

Thanks Nitesh Kant. Atlast you got what I was looking for.

I think you can do away with a lock table in the database.
This table just has a column to indicate whether some thread is working on the job.
After a thread is done with the job, it will set the flag back to "free" so that any other thread can take it up in the next iteration.

I am using this approach now. My concerns are what if the job fails without updating the flag as free?

[Nitesh Kant]

Venkat:
My concerns are what if the job fails without updating the flag as free?

I feel there are two ways of handling it, viz.,

Have a lock timeout. Store also the lock acquire time. If the second thread sees the lock has been acquired for more than the stipulated time then goes ahead with the task. (The timeout period will be critical here)

Store the owner thread details. You need to provide a way to do a health check of the owner thread. It can be an HTTP request or an RMI call(assuming that a single process does not spawn two worker threads) to find out whether the thread is alive. If not then you can acquire the lock else wait. This will be in addition to the timeout period as you would not really want to do it everytime a thread finds out that a lock is acquired by another thread.

[Venkat Sadasivam]

Excellent!. Your answers make sense.

[Mike Stach]

I agree with what Nitesh is saying. However, I presume the scheduler is trying to poll the database for some state change. Instead of writing a scheduler, you might as well could consider using JMS topics that would send a message to one of the app server instances to carry out the task.

[arulk pillai]

I am confused as James is

scheduler with thread implementation

It is not a good idea to start your own threads inside a Web Container.Why can't you run it as a stand alone batch process? Are you using a Scheduler like Quartz or TimerTask? Alsso, need to think if your Web Application will be impacted when the scheduled job is run? is it a long running scheduled job? Quartz do offer clustering capabilities. You also may need a NTS to synchronize your time on all three servers.

how do I ensure that only one thread runs in clustered environment

Do you you mean that a job needs to be run only once?
[ November 03, 2008: Message edited by: arulk pillai ]