Better Response Times on WebSphere Cluster

[Sudharshan Govindan]

Hi,
We have our web application deployed on a Cluster of 2 IBM WebSphere 4.0.4 Application Servers on
Sun Solaris boxes. Clustering was achieved using "Horizontal Cloning" of the Application Servers.
Our application does not have EJBs and the objects in session are persisted on a DataBase. The
problem we are facing is POOR RESPONSE TIMES. We have tuned the
1. Connection Pooling parameters for both Application and Session DataSources.
2. Web Container threads on both the machines.
When Load/Performance Testing was done using Rational Robot, the response times were not
satisfactory as compared when the application was deployed on a Single Server. This defeats the
very purpose of going for a Custer - Scalability with minimal Response Times.
Any help/pointers/suggestions with respect to WebSphere Cluster will be greatly appreciated.
Thanks and Regards,
Sudharshan Govindan

[Rob Misek]

Hi Sudharshan,
Unfortunately, that is often what happens in a cluster. When moving from a single server environment to a clustered environment there is typically a performance hit from the disabling of caches. This is due to the added complexity of a clustered environment.
One of the main reasons that people suffer from performance degredation when moving from a single server environment to a clustered environment in WebSphere is that in a clustered WebSphere environment persists the HTTPSessions in the DataBase. That means after every modification to a Session object WebSphere (behind the scenes) makes a DB call to update the DB in synch with the modification to the Session object.
If you want to avoid such a bottleneck I would point you to Tangosol Coherence which has an HTTPSession Replication Module for WebSphere which replicates the Session objects in-memory across the cluster. This avoids the costly DB calls and provides fail-over of the Session data.
Further, Coherence provides the ability to synchronously cache/store data/objects in-memory in a clustered environment. It implements cluster-wide locking and event notification. Replicated and Distributed caching strategies are both supported through a java.util.Map interface.
Later,
Rob Misek

[Simon Song]

Another tip is, if you can tolerate certain fail over. You can remove persistent HTTP session, and use session affinity in the dispather.
This will guarantee your user will be rerouted to the same webcontainer in each request.
Don't forget tune the persistent session caches setting, check out redbooks for performance tuning.

[Sudharshan Govindan]

Hi Rob,
Thanks for the suggestions. We will explore the option of an external product (Coherence) and see the results.

Originally posted by Rob Misek:
Hi Sudharshan,
Unfortunately, that is often what happens in a cluster. When moving from a single server environment to a clustered environment there is typically a performance hit from the disabling of caches. This is due to the added complexity of a clustered environment.
One of the main reasons that people suffer from performance degredation when moving from a single server environment to a clustered environment in WebSphere is that in a clustered WebSphere environment persists the HTTPSessions in the DataBase. That means after every modification to a Session object WebSphere (behind the scenes) makes a DB call to update the DB in synch with the modification to the Session object.
If you want to avoid such a bottleneck I would point you to Tangosol Coherence which has an HTTPSession Replication Module for WebSphere which replicates the Session objects in-memory across the cluster. This avoids the costly DB calls and provides fail-over of the Session data.
Further, Coherence provides the ability to synchronously cache/store data/objects in-memory in a clustered environment. It implements cluster-wide locking and event notification. Replicated and Distributed caching strategies are both supported through a java.util.Map interface.
Later,
Rob Misek

[Sudharshan Govindan]

Hi Simon,
Thanks for the suggestions. If we remove session persistence the results are better. I was trying if I could get good results with persistence also.

Originally posted by Simon Song:
Another tip is, if you can tolerate certain fail over. You can remove persistent HTTP session, and use session affinity in the dispather.
This will guarantee your user will be rerouted to the same webcontainer in each request.
Don't forget tune the persistent session caches setting, check out redbooks for performance tuning.

[Patrick Finnegan]

The other potential solution is don't clone. Set up two independent WebSphere installations and point them both at the same DB2 session database. IHS will still load balance between the two boxes,detect server failures and re-route. One of the major advantages of cloning is work load management for EJBs which you don't have.
I have session persistance enabled and there is no performance degradation under load. See the new tuning guide in the infocentre.

[Sudharshan Govindan]

Hi Patrick,
Thanks for the pointers. We are not using any hardware load balancer like IBM Network Dispatcher. If the Application Servers are not part of a Server Group, then how will the HTTP-Plugin route the requests to the 2 servers ? If possible, can you send me a sample plugin-cfg.xml file (HTTP Plugin) ? Also, will the 2 AppServers be part of a single domain (sharing the same Admin Repository) or multiple domains ?
Regards,
Sudharshan Govindan

Originally posted by Patrick Finnegan:
The other potential solution is don't clone. Set up two independent WebSphere installations and point them both at the same DB2 session database. IHS will still load balance between the two boxes,detect server failures and re-route. One of the major advantages of cloning is work load management for EJBs which you don't have.
I have session persistance enabled and there is no performance degradation under load. See the new tuning guide in the infocentre.

[Patrick Finnegan]

The appservers would be in the same domain i.e the two WebSphere instances share the same admin database and you can see two nodes in the admin client. The plugin would show two server groups representing each node.
You then copy the plugin to the remote IHS server which load balances the requests across the nodes and preserves session affinity.
Let's say you have two WebSphere boxes earth and mars each running one JVM called "default server".

The plugin looks like:
<ServerGroup Name="earth/Default Server">
<Transport Hostname="earth" Port="9085" Protocol="http"/>
</ServerGroup>
<ServerGroup Name="mars/Default Server">
<Transport Hostname="mars" Port="9084" Protocol="http"/>
</ServerGroup>
Get rid of clone id to improve performance.

[Patrick Finnegan]

Let's say IHS is on Pluto and the WebSphere boxes are on Earth and Mars.
I start my browser which connects to Pluto. IHS redirects to Mars and establishes session affinity with Mars so that all subsequent requests with the same session ID are routed to Mars. Suddenly Mars goes down. IHS detects that Mars is no longer available and re-routes to Earth. WebSphere on Earth checks the session id in the request against it's session id cache. It doesn't exist. It then check's it's session database which it shares with Mars and finds the session object previously stored by Mars and regenerates the session details. If you have more than one WebSphere box then you must have session affinity and persisted sessions.

[Sudharshan Govindan]

Hi Patrick,
Thank you very much for the details on plugin-cfg.xml. Our team is testing the options and I will keep posted our results.
Regards,
Sudharshan Govindan

Originally posted by Patrick Finnegan:
Let's say IHS is on Pluto and the WebSphere boxes are on Earth and Mars.
I start my browser which connects to Pluto. IHS redirects to Mars and establishes session affinity with Mars so that all subsequent requests with the same session ID are routed to Mars. Suddenly Mars goes down. IHS detects that Mars is no longer available and re-routes to Earth. WebSphere on Earth checks the session id in the request against it's session id cache. It doesn't exist. It then check's it's session database which it shares with Mars and finds the session object previously stored by Mars and regenerates the session details. If you have more than one WebSphere box then you must have session affinity and persisted sessions.

[Simon Song]

Just reading through WAS5.0 document, in 5.0 there is DRS(Distributed Replication Service) for replicating the HTTPSession cross appservers in the cluster. This is very cool, and you don't have to integrate 3rd party caching product to have this feature.
Time to migrate directly to WAS5.0?