servlet filter to hain System.out to jsp out.println() -Challenge !-

[Yahya Elyasse]

Hi guys,
this is a Challenging servlet problem and i would appreciate some smart guys to help me solve it.
what i'm trying to do is dynamically load a java .class .this involves using ClassLoader and reflection to invoke the main() method of the java program. (i can do that).
loading and invoking main() will mainely execute the java program and will send output to System.out.
What i'm asking to do is find a way to get or redirect output from System.out ,System.err to the outputStream of servlets/jsp pages i.e direct System.out output to the web browser.(probably using a filter or chaining)
for example :
test.java
---------
public class Test{
public static void main(String[] arg){
System.out.println("hello");
}
}
----------------
now from a servlet /jsp page we should run test.java which will display "hello" ...i.e "hello" should be displayed to the web-browser in the jsp page. (should be done for any java program dynamically).

i would appreciate a lot any help on this issue.

thanking you.

[Paul Clapham]

Can't be done reliably. You can redirect System.out (stdout) to somewhere else by calling System.setOut(somewhereElse), so that would appear to work. However there is only one stdout for the entire JVM. So if other tasks in the app server are writing to stdout, you will divert their output as well. Those other tasks could be other instances of your servlet, they could be other threads that are logging to stdout.

But that's a pretty weak design. By putting some thoughtful limitations on the classes you're going to load, you could produce something workable.

[Ernest Friedman-Hill]

Our friend hasn't shared with us what exactly he's working on, but I've imagined it's some kind of "homework server." Students submit assignments that include a main() and write to System.out like most student programs; then his server loads the class up and runs it, displaying the output.

As Paul outlined, doing this reliably is nigh impossible unless you get jiggy with classloaders. I imagine that with enough care, you could have each student class loaded by an individual classloader that provided a local copy of java.lang.System, such that that student's program had its own private System.out, which you could then redirect with impunity. It'd be tricky, and involve plenty of dark magic, but it should be possible.

[David O'Meara]

Rather than trying to load the external class in the same VM, I'd try invoking the class via the command line and use the process to distinguish different outputs. I can see how EFH's solution would work, but it is not a job for the wary. Personally I'd only touch an application with that sort code if EFH had written it

[Ernest Friedman-Hill]

Originally posted by David O'Meara:
Personally I'd only touch an application with that sort code if EFH had written it

Another alternative -- if I've guessed the intent correctly -- is not to use servlets at all, but to write your own server from scratch which is single-threaded, so that it's safe to set and reset System.out as you wish. A crazy idea, you say? Not so much.. The link is to "Mojasef", a server framework which does exactly that, written by our own irascible Frank Carver. It's available under the Creative Commons license. Have fun!
[ March 09, 2006: Message edited by: Ernest Friedman-Hill ]

[Yahya Elyasse]

Originally posted by Ernest Friedman-Hill:
Our friend hasn't shared with us what exactly he's working on, but I've imagined it's some kind of "homework server." Students submit assignments that include a main() and write to System.out like most student programs; then his server loads the class up and runs it, displaying the output.

No its not a homework at all : any way i'm aged 34 and i've graduated since a long time ago But i'm learning java with my own efforts
this is a real project i'm working on. you r right on some parts it's a university computer system .

for david suggestion :
is it possible to call java from external command to load class ? does this require that a jdk be installed on machine ?
does tomcat contain the java.exe program to execute classes ? where and how should i do that ?

thanks for u all.

[Ernest Friedman-Hill]

Originally posted by othman El Moulat:

for david suggestion :
is it possible to call java from external command to load class ? does this require that a jdk be installed on machine ?
does tomcat contain the java.exe program to execute classes ? where and how should i do that ?

Tomcat is a Java program running in java(.exe). If you've got Tomcat running, you've got a JVM runner of some kind.

David is suggesting you use Runtime.exec() to launch a second JVM -- i.e.,

Process p = Runtime.getRuntime().exec("java -classpath wherever MyClass");

then read the output from p.getInputStream(). This will work fine but won't scale to large numbers of simultaneous users.

[Yahya Elyasse]

I think i'll go for this last solution using Process and Runtime.exec() then trying to get output and direct it to web-page.
but i'm curious to know why it won't scale well for large number of users ?

[Ernest Friedman-Hill]

Originally posted by othman El Moulat:
i'm curious to know why it won't scale well for large number of users ?

How many java.exe's can your server run simultaneously before it runs out of memory? 50? 100? Probably not 1,000? How about 10,000?

[Yahya Elyasse]

Originally posted by Ernest Friedman-Hill:


How many java.exe's can your server run simultaneously before it runs out of memory? 50? 100? Probably not 1,000? How about 10,000?

But if i'm not wrong tomcat server was designed to handel such scaling issues isn't it ?
any way that shouldn't be a very serious matter for me at the moment.

I'm trying to find a solution to my problem. david suggestion was one i have understood and appreciated.( i will try to implement it)
If some one is willing to give me other Ideas (with some java code)it would be nice (if some one provide a complete solution which is merely a nice dream for me till now , i think this would make me a happy guy ).

back to Process.getRuntime.exec():
we suppose here that java.exe is in the system path. i.e any java class could be run by simply the command : java -classpath cc Myclass. right ?
now what if this isn't the case? i.e we need to find first where the jdk java.exe is installed.
should we use for example System.getProperty("java.home") ?
will this work from a servlet ?

thanks.

[Ishan Jayawardene]

Ernest Friedman-Hill wrote:I imagine that with enough care, you could have each student class loaded by an individual classloader that provided a local copy of java.lang.System, such that that student's program had its own private System.out, which you could then redirect with impunity. It'd be tricky, and involve plenty of dark magic, but it should be possible.

Hi Ernest, allow me to revive this old thread. I've run into the same problem as the OP.
There's a java program I need to run, to get the status (running, stopped etc.) of several components. Rather than just "running it" for each component with the full overhead of the JVM startup, I thought of importing the classes to servlet and drive it via http requests, hoping to get better performance. But this program prints its output to System.out and System.err.

Can you give some examples of how a classloader can be used to load classes with private System.out/err, or links to some resources? I did find an old version of the Echidna library that could do this, but there's no documentation.

I'm looking into Mojasef now (to which you linked in a later post but since we have J2EE app servers a dime a dozen in our environment, a simple servlet would be easier to maintain.

[Ernest Friedman-Hill]

Well, I'm a few years older and wiser now, so I would try to design something less complicated

Here's the first thing that comes to mind, although maybe someone else has a simpler idea:

1) Create a custom class loader MyClassLoader. Give MyClassLoader setOut(), getOut(), setErr(), getErr() methods, so it can keep track of its own set of output streams. You'd pretty much have to do this anyway -- if you're compiling and loading student programs, you're going to do it with a classloader, one instance per program, so this hardest part of the problem is something you're already going to do. You must use a separate instance of MyClassLoader to load each application, but of course, you were going to anyway.

2) Create a subclass of OutputStream "MyOutputStream" and override the write() methods to do the following peculiar dance when called:

Look up the call stack, one frame at a time. At each frame, ask the class for the ClassLoader that loaded it. If you find an instance of MyClassLoader, call getOut() or getErr() (which one could be controlled by a constructor parameter) and forward the call to that object. If you get to the top of the stack without finding a MyClassLoader, forward the call to the original System.out (or System.err), which you've saved as a member variable in MyOutputStream.

3) Use System.setOut()/setErr() to replace the original streams -- after saving them, of course -- with instances of MyOutputStream wrapped in PrintStreams.

4) When you want to load an application, create a MyClassLoader, and create whatever OutputStreams you want to store the output -- i.e., ByteArrayOutputStreams -- and put then into the ClassLoader.

I think that's about it. Now whenever application code uses System.out.println(), the MyOutputStream instance figures out where the output should go, and the designated streams fill up.

[Ishan Jayawardene]

Ernest Friedman-Hill wrote:
1) Create a custom class loader MyClassLoader. Give MyClassLoader setOut(), getOut(), setErr(), getErr() methods, so it can keep track of its own set of output streams. You'd pretty much have to do this anyway -- if you're compiling and loading student programs, you're going to do it with a classloader, one instance per program, so this hardest part of the problem is something you're already going to do. You must use a separate instance of MyClassLoader to load each application, but of course, you were going to anyway.

2) Create a subclass of OutputStream "MyOutputStream" and override the write() methods to do the following peculiar dance when called:

Look up the call stack, one frame at a time. At each frame, ask the class for the ClassLoader that loaded it. If you find an instance of MyClassLoader, call getOut() or getErr() (which one could be controlled by a constructor parameter) and forward the call to that object. If you get to the top of the stack without finding a MyClassLoader, forward the call to the original System.out (or System.err), which you've saved as a member variable in MyOutputStream.

3) Use System.setOut()/setErr() to replace the original streams -- after saving them, of course -- with instances of MyOutputStream wrapped in PrintStreams.

4) When you want to load an application, create a MyClassLoader, and create whatever OutputStreams you want to store the output -- i.e., ByteArrayOutputStreams -- and put then into the ClassLoader.

Hi Ernest,

Thanks for the quick reply! Let's see if I understood this correctly... After step 3, all standard IO of this JVM will go through MyOutputStream?

[Ernest Friedman-Hill]

Ishan Jayawardene wrote:

Hi Ernest,

Thanks for the quick reply! Let's see if I understood this correctly... After step 3, all standard IO of this JVM will go through MyOutputStream?

Yes. But MyOutputStream will forward to whatever System.out was before MyOutputStream was installed for all code not invoked by code loaded by a MyClassLoader; i.e., if there are other web apps in this same container, they should continue to operate as normal.

[Ishan Jayawardene]

Ernest Friedman-Hill wrote:

MyOutputStream will forward to whatever System.out was before MyOutputStream was installed for all code not invoked by code loaded by a MyClassLoader; i.e., if there are other web apps in this same container, they should continue to operate as normal.

Yes, that's how I understood it

I've got this forwarding part working... But not sure how to check the call stack for each class's classloader. Can I use

Thread.currentThread().getStackTrace()

for this? Still, don't see how to get a reference the class (and later the classloader of that class) in each stack trace element...