pushing real-time data to screen

[nikki lorenzo]

Hi,

I am faced with a design issue. I am supposed to display real-time data to the users of the application we're building. The data is coming from an MQ server.

I want to avoid polling the data source (i.e. running a thread say every 5secs to check for new data). Thus, we decide to consume the messages subscribers using Message Driven Bean(MDB) as it has a callback method onMessage() that can be invoked by the publisher when a new message is on queue. Inside this callback method, we update the database/session.

But how can we implement pushing of data towards the client-side? I can't think of any approach besides a timed AJAX script checking on the database/session for new records every 5secs. Any suggestions?

Thanks.

[Peter Chase]

You don't state that this is a Web application, but you hint at it by suggesting AJAX.

If it is a Web application, then basically there is no way to push data from a server at a client; the client has to start every request.

Polling using background requests (AJAX) is a common way of obtaining the illusion of server-initiated updates. If you have a good network between clients and server, it shouldn't have too much overhead. Make sure that, when there are no changes, the returned data says so concisely.

I believe some people have had success with long-lived connections to achieve a closer simulation of server-initiated updates. I am not expert on this, but I believe that the idea is that the client makes a request, which the server responds to bit-by-bit, without closing the connection. The client has to process each bit immediately. If the connection gets closed, the client needs to start another one. All quite hairy and also consumptive of resources. Some browsers don't allow lots of connections to same server.

A final alternative might be an applet that talks to the server via a different protocol, which does support two-way comms better.

[Jesper de Jong]

Displaying real-time data in a web application? That's going to be difficult to impossible, depending on how strict the timing criteria are.

As Peter says, if it's a web application with only HTML and JavaScript on the client (whether it's AJAX or not), then it's not really possible to make the server send data to the client spontaneously - such web applications are locked into the HTTP communication model, which means the client always initiates the communication.

You could make an applet that opens a connection back to the server. You can use any protocol you want over that connection (you're not bound to the HTTP request-response model), so you can make the applet listen on the connection and have the server send data.

*edit*: Oh, I didn't see that Peter also already mentioned an applet...
[ February 05, 2007: Message edited by: Jesper Young ]

[nikki lorenzo]

Thanks Peter and Jesper for your input. Now it's clear to me that there's no other way besides client polling to display real-time data for web-based applications.

[William Brogden]

Note that a polling operation can be quite fast if you use a HEAD request that essentially asks if a dataset has changed, rather than a GET or POST which require the dataset being sent even if it has not changed.

Note that a "REST" approach to addressing the data might be helpful. (Here is a nice article on REST)

Bill

[Stan James]

We use an applet that opens a ServerSocket. At logon it registers its address and port with the server. When the server wants to send a message it looks up the address, does the usual socket client open-send-close. The applet parses the inbound message for a JavaScript method name and arguments and calls out to the script on the window.

Open-send-close seems to be taking about 2 seconds, so if you want to broadcast to thousands of users you may have some issues. Theoretically you could go JMS right to an Applet which might be faster ... with a pretty big jar download.

[Atul Mishra]

Hi all,

I was reading "Pushlets" which does this, like pushing real time data. Check www.pushlets.com, Since I also have a similar requirement I was reading resources.

Anyone any comments on pushlets ?

PS: I was reading this article:
http://www.javaworld.com/javaworld/jw-03-2000/jw-03-pushlet.html
Thanks
[ February 06, 2007: Message edited by: Atul Mishra ]

[Atul Mishra]

Nikki, Could you please post what you ended up using ? I too have a similar requirement and will be useful rather than doing another post.

Thanks.

[Stan James]

The pushlet paper is interesting. I've heard this approach suggested before, but without the Javascript framework to go around it. My work app has about 2000 concurrent users and sends messages like this very rarely, so it didn't seem like a good trade off to eat up all those connection, memory and thread resources on the server. It might be a good fit for a few users with frequent activity.

[Atul Mishra]

Stan,

Thanks for your response. Our app will have 2000-3000 concurrent users. I was thinking pushlets wont be a good idea in that case.

Whats your recommendation for the same purpose for this many users ?

Thanks

[Stan James]

As mentioned above, we have an applet with an open ServerSocket. The 2 seconds per client turned out to be an abberation. Clients who are registered but don't really exist - eg closed the browser - cost some time for the connect to fail, but the others ran at several clients per second, as high as 9 per second, in the latest test.