Servlet communication

[Neo Wills]

Hi,
If the webserver want to communicate through non-blocking with some process for each http request it gets, the webserver will need to store some information to handle the reply part of it. But the current mechanisms I see dont really provide a nice method to store and retrieve that info. Is there any known methods for that?

Neo

[William Brogden]

I think we need some clarification on your problem.

The servlet API is loaded with methods for sharing information at different levels - request, web application, entire server, networked servers.

Give us a "use case" - start with a new request from a client - what happens next?

Bill

[Neo Wills]

Pardon me for not being clear.

Each time a new request from a client is obtained, the webserver needs to send out a non blocking msg on the socket out to a process X. Later it receives a msg over socket from X and has to correlate it to which client request it got that reply for and respond to the client.

[Mark E Hansen]

When you say non blocking, who is it you need not to block? The servlet?

If the Servlet is not blocked by the external communication, that means the external communication will go on its merry way, while the Servlet goes on its merry way. When the Servlet is finished, it will send its response to the client.

How do you plan to integrate the response from your external communications with the Servlet response, if the Servlet has already responded by the time your external communication response comes in?

Perhaps you could just explain a little more about what you're trying to do.

[Neo Wills]

Should the response for the external client be done only from the same thread on which it got invoked?

Say this is the code piece:

public class temp extends HttpServlet{
   public  void doGet(HttpServletRequest request, HttpServletResponse  response) {
           PrintWriter pw;
           pw = response.getWriter();

          // Send the data to X through non blocking sockets
  
          // Do not respond to the external client (as in dont write anything on pw)
    }
}

Later when X sends back the response on a socket to the webserver in a separate dedicated thread, can we respond to the external client request?

[Mark E Hansen]

Of course you can do that, but if you wait for the external communication to complete before your Servlet is allowed to send its response, will it take too long? Will your browser user be willing to wait that long? Will the browser session time out while waiting for this to happen (some browsers will not wait all that long for their requests to receive a response)?

Those are questions you need to answer before you can decide. In broad terms, you really don't want any application on which the user is waiting for a response, to take a long time doing any unnecessary work. On the other hand, if the work needs to be done before the response is sent, then it needs to be done. If it takes too long to do it, there's an architecture/design problem.

How long can this external communication take? Note I said "how long can it take" not "how long might it take". These are different. If it's possible that this call could take as much as (but never more than) 5 minutes, then you need to account for this in the design. Even if it might normally only take 10 seconds, if it can take 5 minutes will the user/browser be willing to wait that long? What happens to your application if they are not willing?

I don't really have a good idea (yet) of what you're trying to do, but it sounds like you may have a problem brewing here.

Give some thought to the above and let's see where we go from here.

[Neo Wills]

Hi Mark,

Thanks for the input. But the response to the servlet should be back in less than a second. Still I dont want this thread to be blocked for that 1 second just for the response from the external process.

Say I get client requests A1, A2 and then the servlet sends the non blocking msg for these to X. When the response comes for A1 on the dedicated thread, how can the servlet respond for A1 to the client?

Neo

[Mark E Hansen]

Once you make the call, how do you know when the response is ready?
Do you have to keep checking a queue somewhere?

For example, when you call System.out.println(...), you know the call is done when the method call returns (basic stuff). However, when you make an asynchronous call, the method call returns immediately (or there abouts). There needs to be some way you can tell when/how to get the response.

What does the API for that external communication have to say about this?

Do you have an example provided by the communication vendor which shows how the client that makes the call can get the response?

I expect you're over thinking this because of the Servlet environment, but it doesn't sounds like it really matters here. How would you do this in a standard client application?

[Neo Wills]

Ok. This is how it would have been done on a standard client app

Events are shown in timeline here:
T1 C1 ------> Web Server
T2 Web Server ---(nb socket send Request1)------> X
T3 C2 ------> Web Server
T4 Web Server ---(nb socket send Request 2)-----> X
T5 Web Server <---(nb socket resp Response 1)----X
T6 Web Server <---(nb socket resp Response 2)----X

The webserver would receive responses on a dedicated thread from X. Now how would it respond to C1 and C2?

[Mark E Hansen]

How would it work in your stand-alone application?

[Neo Wills]

In a standard application, C1 sends some info to the webserver. I can have the connection info of C1 stored in some data structure and send the non blocking request to X.

When the response for X comes back on a different thread, I can look up the connection info of C1 from the data structure and reply on that.

[Mark E Hansen]

Yes, but how does the response come back on a different thread? How does your client know this thread even exists? The threads you're application are using need to be created by your application. Threads created elsewhere are just "attached" to your application.

How is your client notified that the response is available? Do you call a method, like "areYouThere()"?
Do you wait on a semaphore?

By the way, when I say Thread, I'm referring to process Threads provided for example through the java.lang.Thread class.

[Neo Wills]

For example, I create a thread in the init() routine in the servlet, which binds to some port opened by the external client.

Thread Sender in webserver:

        Socket sock = new Socket(IP,PORT);
        DataOutputStream out = new DataOutputStream(sock.getOutputStream());
        while (TRUE) {
            // Wait on a condvar
            out.write(some_data);
         }

Thread Receiver in webserver:

       BufferedInputStream in = new BufferedInputStream(sock.getInputStream());
        while(TRUE) {
            in.read(some_data);
            // Send the response to the client which sent request to the webserver.
        }

Thread which gets the request from the client in webserver:

public  void doGet(HttpServletRequest request, HttpServletResponse  response) {  
            PrintWriter pw;  
            pw = response.getWriter();  
    
           // Signal the condvar for the Thread sender to send some data.

           // Do not respond to the external client (as in dont write anything on pw)  
     }  

[Mark E Hansen]

When the message comes back in to the Servlet, does the message have information in it that will tell you which client it belongs to? If so, they can't you create a condvar in the client and pass it to the "Thread Sender in webserver" to save off. You client can then wait on this condvar. When the Servlet gets the response which belongs to your client, it can signal the condvar, so the specific client will know the response is ready.

But again, this should be no different than how you would do it in a standard client application. I doesn't look to me like the Servlet environment is adding any complexity to it. I would ask your question in a forum more appropriate for Threads and IPCS. You're more likely to get the right kind of help there.

Good luck.

[Schandha Ravi]

Is this not something related to asynchronous communication ? If so, can't we use JMS messaging model. Is there any constraint that it needs to be a web application. I'm having questions on the reliability of using a web application here. Once the servlet sends the request to an external process opening a socket (which may be external to the web application context) how do we maintain the session.

This seem to be a very interesting usecase and I would be glad to learn the solution if anyone has.

[Neo Wills]

Hi Mark,
There is no doubt in how the IPC / Thread communication needs to work here. There are only 3 clarifications I need from the servlet environment:

The external client (X) is not going to give me the info of which web client I need to respond to. This needs to be stored in some global data structure in the webserver and when the response comes from the external client I need to figure which webclient to respond to.
1. How do I store the global data structure in servlets (across all sessions) with each web client's connection info?
2. Does my servlet thread have to be blocked in condvar till the response comes from the external client? Or can I respond from a different thread?
3. If I can respond, what is the web client's connection info we need to hold? In case of normal ipc, its just fd's. Would storing "HttpServletResponse response" be good enough for responding later for that client?

Neo

[William Brogden]

This external process you want to talk to by a socket - what provision does it have for handling multiple requests at the same time? What provision does it have for identifying a particular client?

1. any object used by all sessions can have a reference in the ServletContext
2. blocking the servelet request/response Thread until you finish the conversation with the external process is a recipe for disaster. Believe me - this happened to me - mysterious lock up of entire server because all Threads were waiting.
3. NO - never ever store a HttpeServletResponse or request reference - these objects are managed by the container. The sessionID is the conventional method of identifying a particular user/client.

Bill

[Neo Wills]

Thanks William.

The external process creates a socket on a port and keeps waiting on it. All the webserver has to do is to bind to it.

1. When you say we can store the reference in servlet context, you mean through setattribute? I can look more into that.
2. Since blocking on the request/response thread is not good, can I reply for a session from a different thread?
3. I can get the session id from the http request. But how do I convert it back to http request and use to send response back?

Neo

[Bear Bibeault]

Neo Wills wrote:3. I can get the session id from the http request. But how do I convert it back to http request and use to send response back

You can't. You can only send a response when you get a request.

A web app is not like a desktop app that's "always running". A web app only "runs" when a request comes in.

So you have two choices when dealing with long-running processes:

Block and send the response when it's done -- as pointed out, not a good idea.

Fire off a thread and return the response immediately.

When the thread completes, store the information in the DB (or wherever else is appropriate). At this point, you do not have the option of returning a response as there is no request. Besides, by this time, the user is probably off looking at goofy videos on youtube.

What you provide is a means for the user to come back -- at their own good time -- and check the status and results of the operation.

[Neo Wills]

Thanks Folks.

Did not know that we cant send a response without getting a request. That means we really cant do much on long running processes.

If webservers interact with application servers, does the response always happen in the same thread where it gets the request?

Like if the webserver wanted to get some info from the application server, it does it then and there and replies inline?

[Bear Bibeault]

What are you thinking is the difference between a web server and an app server?

[William Brogden]

Long running processes are usually handled by having the first response include a link by which the user can get the current status of the process. This gets discussed a LOT on the servlet and JSP forums so search for "long running process."

Typically the first request starts a new Thread which carries out the work in an object designed to start the external process and provide for recording all possible results. This means you need a mechanism to keep track of all of the pending working objects and associate them with the originating user. To keep this simple, you can build and test this outside the servlet environment.

Bill