Struggling to run a script at regular intervals - any ideas?

[Michael Cropper]

Hi,

I have been having a play around with scheduling a java class file with a few different technologies and I am struggling to get them working in an acceptable way.

First attempt

Was to set up a CRON job (via cPanel on my web host) to run my .class file - although that didn't work as it was not a binary file which could be executed. (I'm still working with my web host to see if this is possible at all, but not getting very far at the moment)


Second attempt

Was to set up scheduling within my application using the TimerTask and Timer API's using the following code

TimerTask task = new RunMeTask();
Timer timer = new Timer();
timer.schedule(task, 1000, 5000);

public class RunMeTask extends TimerTask {
@Override
public void run() {
System.out.println("Run Me ~");
}
}

While this did achieve the required result, it had a draw back. Being that if the URL was accessed (which was mapped to the Servlet) twice then two instances were created and the scheduled task which used to run every 10 minutes (for example) would now run twice within that 10 minute period.

The only way I could think of to get around this was to place the 'start timer' call on a URL which would have to be accessed each time the web application is restarted (quite frequently!). So back to the drawing board.


Third attempt

Was to use the Quartz Scheduler, and when finally getting this to work (they have made significant changes in version 2.1 which have not been documented very well! grrr) I found that this has inherently a similar problem as option 2.

Meaning that when I place the following code into a Servlet

try{
            QuartzTest trigger = new QuartzTest();
            trigger.myFunction();
        }catch(Exception e){
            System.out.println(e);
        }

Which calls the following script (POJO)

public class QuartzTest {
    public static void main(String[] args) throws Exception{
        System.out.println("in main method");
    }

    public static void myFunction() throws Exception{
        System.out.println("in QuartzTest.java");
        SchedulerFactory sf = new StdSchedulerFactory();
        Scheduler sched = sf.getScheduler();

        //Job #1 is scheduled to run every 20 seconds
        JobDetail job = newJob(RunMeJob.class).withIdentity("job1", "group1").build();

        CronTrigger trigger = newTrigger().withIdentity("trigger1", "group1").withSchedule(cronSchedule("0/20 * * * * ?")).build();

        sched.scheduleJob(job, trigger);
        //end job 1

        sched.start();

        Thread.sleep(300L * 1000L);

        sched.shutdown(true);

    }
}

Which is calling the simple RunMeJob file

public class RunMeJob implements Job {

    public void execute(JobExecutionContext context) throws JobExecutionException {
        System.out.println("in RunMeJob.java");
        JobKey jobKey = context.getJobDetail().getKey();
        System.out.println("SimpleJob says: " + jobKey + " executing at " + new Date());
    }
}

The problem being that when I access the original Servlet URL which called these scripts, then the page hangs until the Quartz Timer script has finished. Which brings me back round to the same problem as in solution 2 whereby I would need to create a single URL which would only be accessed once and every time after the web app is restarted - which isn't feasible.



So I am a little lost on where to go from here. From what I have read about Quartz I believe I have implemented the basic timer options correctly. Does anyone know of a way to only call the script once so that the Quartz timer will take over from this point on?

One thought I have just had is to possible set up something in web.xml which automatically runs the script when the server starts up (although I am not sure if that is even possible, I may have just made that functionality up in my head..).

Any information / guidance on this topic is much appreciated.


Thanks

Michael

[fred rosenberger]

a cron job doesn't have to be a binary. I have scheduled many, many perl scripts (all of which are plain text) that run all the time.

What i would guess you need to do is not run the .class file but the full java command..something like

00,15,30,45 * * * * java yourClassName

[Jeff Verdegan]

Michael Cropper wrote:Hi,

I have been having a play around with scheduling a java class file with a few different technologies and I am struggling to get them working in an acceptable way.

First attempt

Was to set up a CRON job (via cPanel on my web host) to run my .class file - although that didn't work as it was not a binary file which could be executed.

When you run your program from the command line, do you type MyClass.class? No, you type java -cp whatever MyClass. Similarly, your cron entry needs to be java -cp whatever MyClass. Note that you'll probably have to provide the full path to the java executable as it may not be in the PATH variable in the cron environment. Alternatively, you could have cron invoke a shell script that does whatever env setup is needed and then launches java to run your class.

Was to set up scheduling within my application using the TimerTask and Timer API's using the following code

Which is fine as long as your Java program is always running. Normally this is done by providing an appropriate /etc/init.d and rc directory entries.

TimerTask task = new RunMeTask();
Timer timer = new Timer();
timer.schedule(task, 1000, 5000);

public class RunMeTask extends TimerTask {
@Override
public void run() {
System.out.println("Run Me ~");
}
}

While this did achieve the required result, it had a draw back. Being that if the URL was accessed (which was mapped to the Servlet) twice then two instances were created and the scheduled task which used to run every 10 minutes (for example) would now run twice within that 10 minute period.

Eh???

The only way I could think of to get around this was to place the 'start timer' call on a URL which would have to be accessed each time the web application is restarted (quite frequently!). So back to the drawing board.

Again, Eh???

So I am a little lost on where to go from here.

If your Java app isn't a service that needs to be constantly running, but just needs to execute periodically, cron is the way to go.

[Michael Cropper]

Thanks for the quick reply.

Following on from the first point about CRON jobs, if I were to use the CRON job "java -cp /my/file/directory/structure MyClass" would that work? I am not too familiar with compiling / running files from the command line as I use an IDE.

Then would the "MyClass" file be something as simple as the following?

public class MyClass{

    public static void main(String args[]) throws Exception {

            System.out.println("it worked!");

    }

}

Would that work?


Onto the second point you mentioned "Normally this is done by providing an appropriate /etc/init.d and rc directory entries. " - I will go and read up on that as it is not something I have come across before.


With the third and fourth point's / Eh's, this is related to how I have understood what I have read so far. Based on the 'Ehs' I can only assume that I haven't quite implemented things correctly.

The way I have currently implemented either the TimerTask or the Quartz Timer is by mapping a URL to a Servlet so that I can call the script manually (i.e. by visiting the URL in a browser) when I restart the server. I am guessing there is a way to call a POJO file via Web.xml (or another config file) when the server starts up - although I have been unable to find a way to do this so far - is this possible? So the point I was making with this way is that if I accessed the URL twice then the process would occur twice as fast due to two instances running for the script.


Based on your information at the start and end of your reply, CRON seems like the way forward so best if we just focus on that as a starting point (if that doesn't work then can explore the other options at a later stage).


If feedback / guidance / recommended reading could be provided regarding the CRON setup with a Java file that would be much appreciated since I already have all the logic created in my source files, I just can't get them to run.


Thanks
Michael

[Michael Cropper]

fred rosenberger wrote:a cron job doesn't have to be a binary. I have scheduled many, many perl scripts (all of which are plain text) that run all the time.

What i would guess you need to do is not run the .class file but the full java command..something like

00,15,30,45 * * * * java yourClassName

Thanks for the info Fred, (I already replied before I saw your reply).

The script I run will have to be a Java file since that is the extent of my knowledge, plus the logic contains DB access and other Java API's to do some funky stuff. I have tried a huge variety of combinations with the CRON job at the moment and I believe it is an issue with either a) how I am calling the script or b) the actual script being called.

As per the example I have provided above, I hope this will provide information to help direct me on what I am doing wrong and how to solve the issue.


Thanks
Michael

[Jeff Verdegan]

Michael Cropper wrote:Thanks for the quick reply.

Following on from the first point about CRON jobs, if I were to use the CRON job "java -cp /my/file/directory/structure MyClass" would that work? I am not too familiar with compiling / running files from the command line as I use an IDE.

Unless this is your one and only Java program ever, I strongly suggest you get familiar. Obviously there are plenty of examples on the web, but, roughly, if you have com.mycompany.MyClass (that is, class MyClass in package com.mycompany), and if the .class file is /my/file/dir/struct/com/mycompany/MyClass.class then you need to run java -cp /my/file/dir/struct com.mycompany.MyClass . That is, the classpath needs to include the parent directory of the root of the package, and you need to name the class by its fully qualified name, including the package.

Note that you can't "shift" elements from one cmd line arg to the other. That is, the above command is NOT equivalent to. java -cp /my/fil/dir/struct/com/mycompany MyClass .

If you have other jars that are needed, then you need to include them in the -cp, separated by the appropriate delimiter ( colon ( : ) on Linux, semicolon ( ; ) on Windows).

[Jeff Verdegan]

Michael Cropper wrote:
The script I run will have to be a Java file since that is the extent of my knowledge, plus the logic contains DB access and other Java API's to do some funky stuff. I have tried a huge variety of combinations with the CRON job at the moment and I believe it is an issue with either a) how I am calling the script or b) the actual script being called.

Note that Java programs are not "scripts". If you use that term, especially in this context, itcan lead to confusion.

It sound like the best approach for you will be to first get it working from the command line, then put that command line (and any additional env setup you need to do prior to executing that command line) into a shell script, and then having cron run the shell script. That way you have one simple, turnkey means to execute this program, and you can call it at will from the command line or cron or whatever.

[Michael Cropper]

Thanks for the information. I have to say that what you have said is beyond my level of expertise. Are there any recommended reading / resources that you can point me in to get up to speed.

I have read a lot of Java / programming books over the past 12 months and would like to think I have a reasonable amount of knowledge, but I am struggling to understand what you have just said.

Note that Java programs are not "scripts". If you use that term, especially in this context, itcan lead to confusion.

What would be the correct term? Personally I see the terms as inter-changable based on how I use them.


Thanks
Michael

[Pat Farrell]

Michael Cropper wrote:What would be the correct term? Personally I see the terms as inter-changable based on how I use them.

Some languages, such as python, javascript or php are script languages, you essentially execute the source code.

Java programs are programs, built from source and processed though a compiler generating output. You run the Java program, or execute the
program within a Java 'jar file"

Of course, you can write a three line bash script that executes a java program, so the difference is more important for human understanding than it is
as a technical challenge.

Note well: javascript and java have nothing to do with each other. The fact that the first four letters are the same is pure marketing BS that causes much confusion.

[Jeff Verdegan]

Michael Cropper wrote:Thanks for the information. I have to say that what you have said is beyond my level of expertise. Are there any recommended reading / resources that you can point me in to get up to speed.l

I don't know what in particular you're confused about.

[Michael Cropper]

If you have other jars that are needed, then you need to include them in the -cp, separated by the appropriate delimiter ( colon ( : ) on Linux, semicolon ( ; ) on Windows).

Looking through my source code there would be two .jar files which would need to be included in the CRON job which are;

- mail.jar (for JavaMail API)
- mysql-connector-java-5.1.17-bin.jar (for DB connections)

So would that translate to having a CRON command similar to the following

java -cp /my/dir/struct/ MyPackage.MyClass mail.jar : mysql-connector-java-5.1.17-bin.jar

Is that along the right lines?

And is the MyClass correct to contain all the logic within the main method so that it runs automatically? Do all of the other imports within a class work, since the program is calling other class files to accomplish certain tasks.


Thanks
Michael

[Jeff Verdegan]

Michael Cropper wrote:

If you have other jars that are needed, then you need to include them in the -cp, separated by the appropriate delimiter ( colon ( : ) on Linux, semicolon ( ; ) on Windows).

Looking through my source code there would be two .jar files which would need to be included in the CRON job which are;

- mail.jar (for JavaMail API)
- mysql-connector-java-5.1.17-bin.jar (for DB connections)

So would that translate to having a CRON command similar to the following

java -cp /my/dir/struct/ MyPackage.MyClass mail.jar : mysql-connector-java-5.1.17-bin.jar

Is that along the right lines?

No.


java -cp /my/dir/struct:/path/to/mail.jar:/path/to/mysql-whatever.jvar MyPackage.MyClass

And is the MyClass correct to contain all the logic within the main method so that it runs automatically?

Yes, MyClass would be the one that contains the public static void main(String[] args) method that is the entry point for your program. Normally you don't put "all the logic" in the main() method. It's just there to do a small amount of initial setup, and then kick off the meat of your app. How you handled division of responsibility in your code is a totally separate issue from what you're asking about here though.

Do all of the other imports within a class work, since the program is calling other class files to accomplish certain tasks.

As long as you have all the same elements on your classpath as you did when running in the IDE, and as long as your code doesn't make any assumptions about things like pwd that might not be the same as they were in the IDE, then, yes, it will work. Of course, the only way to know is to try it.

[Randall Twede]

this has been an informative post. about the options from the command line(-cp etc.)
i always knew there were more options but i never cared

[fred rosenberger]

Michael Cropper wrote: I am not too familiar with compiling / running files from the command line as I use an IDE. ...
Personally I see the terms as inter-changable based on how I use them.

As to point 1: you now know why everyone here tells beginners NOT to use an IDE. You have missed out on some key pieces of understanding.

As to point 2: YOU may use them interchangeably, but the rest of the programming world does not. They mean certain things, and when you use them in a different way, it confuses folks. Granted, the difference is rather subtle, but traditions and customs are slow to change.

MOST of the time, once you know the command line syntax to run what you need to run, you can just drop it into the crontab entry. There are sometimes environment variable issues, but that is for another day. A single crontab entry can be quite long. Here is an example from my TEST system at work


15 * * * * HOST=mars; CL_INSTALL_DIR=/opt/quovadx/qdx5.7; PATH=/hci/bin:/opt/quovadx/qdx5.7/integrator/bin:/usr/bin:usr/local/bin:opt/quovadx/qdx5.7/integrator/sbin; export HOST;export PATH;export CL_INSTALL_DIR; cd /hci/bin/monitor;/hci/bin/monitor/san.pl

(note: the above is all one line)

You can see this sets several variables, exports them, changes to a new directory, and runs a perl script.

your cronjob can look similar. you may not need all the environment variables, but you can certainly pass arguments into your java program.

[Michael Cropper]

Great, thanks for all the help / information / examples that has been really useful.

I believe that should give me enough to go off now and give things another go. I will no doubt be back either asking a few more questions or letting you know that it worked :-)


Thanks again.

Michael