Tomcat & Memory

[Roel De Nijs]

I wrote a web-application and it really needs a lot of memory, i am taking about 200 MB to 250 MB for each client that runs the application via a browser.

So my question: is there a limitation on the amount of memory tomcat can work with (if my server has no limitation of physical memory)? Or there other alternatives of configuration that can be tought of (e.g. 3 instances of tomcat on the same server and in front a load-balancer that controls the load of each instance and redirect the request to the appropriate tomcat-instance.

Any tips, hints and tricks are welcome.
Thanks!

PS. I'm using Tomcat Apache 5.0.28

[Scott Dunbar]

You don't tell us your JVM or O/S but the Sun JVM seems to be limited to about 1.5GB under Windows, 2GB under Solaris x86 and Linux, and 4GB under Solaris SPARC. These are all 32 bit JVM's. Under Solaris it seems likely that if you used some O/S specific settings (ISM) you should be able increase it. I have not seen what the max is under Sun's 64bit JVM's.

Based on what you're saying it sounds like you may need to run multiple instances depending on your load and your platform.

[William Brogden]

Instead of multiple instances of Tomcat consider running the application in a separate JVMs, communicating with Tomcat by RMI, JMS, JavaSpace or sockets. That way you can add processing power elsewhere on your network as needed.
Bill

[Roel De Nijs]

Originally posted by William Brogden:
Instead of multiple instances of Tomcat consider running the application in a separate JVMs, communicating with Tomcat by RMI, JMS, JavaSpace or sockets. That way you can add processing power elsewhere on your network as needed.
Bill

All ideas will be taken in consideration, because the memory is really important key factor. But i don't quite understand what you mean, so is there any docs about this, or can you be a little bit more detailed in what you mean.

I'll give a little bit more info:
our application contains a struts-part and also a lot of classes containing the business logic. There is also a second servlet that's not written by us, but we bought that application and it's rendering images. So our application and the bought one must run in the same JVM because else some things are done twice (can't change this) and this is less performant.

[Roel De Nijs]

Originally posted by Scott Dunbar:
You don't tell us your JVM or O/S but the Sun JVM seems to be limited to about 1.5GB under Windows, 2GB under Solaris x86 and Linux, and 4GB under Solaris SPARC. These are all 32 bit JVM's. Under Solaris it seems likely that if you used some O/S specific settings (ISM) you should be able increase it. I have not seen what the max is under Sun's 64bit JVM's.

Based on what you're saying it sounds like you may need to run multiple instances depending on your load and your platform.

Our OS would be something like Linux or Windows i guess (don't now for sure, because it's not my working area).

Don't know what's meant with the JVM, but i use the JDK1.4.2_05 and also the JRE1.4.2_05 for running and building the app. Has this anything to do with the JVM or is that something else? and if so, what specifically is it. Always willing to learn something more

[Roel De Nijs]

And are there any other application servers besides tomcat that doesn't have these memory limitations, both free ones and ones where you have to pay?

[Scott Dunbar]

The JVM is the Java Virtual Machine. In your case it looks like you're using the one from Sun and are using 1.4.2_05.

As far as the memory goes, this is a limitation of the JVM, not of Tomcat. Because it is a 32 bit JVM the best it could hope for is 4GB but much of what I've seen is half of that (2^31). Tomcat runs within the JVM, just as WebPhere or WebLogic would.

With those large of memory requirements I'd encourage you to carefully consider your design and deployment options. I do understand that rendering can take a boat load of memory but what you're describing sounds pretty high. Certainly you don't want to do the work more than once and you don't want to copy that much data over the network.

If you really can't split up the processing you're really going to need to consider a 64 bit JVM. After I responded to you before I did some digging. Under Solaris/SPARC with the 64 bit JVM people have reported allocating 32GB with no problem. I don't know the actual maximum but you may have to rethink your deployment platform.

[jhon Reader]

i heard for jdk 1.5 ,
sun has tested it for 500 gb of ram
check it out if it is correct

[Roel De Nijs]

Originally posted by Scott Dunbar:

With those large of memory requirements I'd encourage you to carefully consider your design and deployment options. I do understand that rendering can take a boat load of memory but what you're describing sounds pretty high. Certainly you don't want to do the work more than once and you don't want to copy that much data over the network.

Thanks for your reply and the digging

Our application renders an image (gif file of a couple of KBs, max around 30 KBs), so only this image is transferred over the network. But we keep in memory a lot of data (comes out of a database) and is kept in memory because of performance issues. WE could skip that part of holding it in memory, but then we have to do each time the user does a request go to database and retrieve all records needed and re-arrange them so an image could be generated. So the time to do this with the initialisation is around 1,5 minutes and after then it's approx 10secs. But if we decide to go get it from database, then user has to wait again for 1,5 minutes and for a web-app that is much too slow. So everything's kept in memory to get an acceptable performance and it has to stay that way. And I'm completely aware that memory is a big issue in this story. I also did quiet some profiling and with the results did some re-engineering, because initially it was 350 MB for each client that runs the application via a browser, so i saved with the re-engineering 100 MB for each client.
At the beginning the initial request of a client took 30-35min and used 350 MB, now it's 1,5min with 250 MB used for 1 client. So now we are at a point where performance is acceptable (and really high) with as less as possible memory used. Any change is diminuishing the performance and that isn't acceptable.

[Roel De Nijs]

Originally posted by Scott Dunbar:

If you really can't split up the processing you're really going to need to consider a 64 bit JVM. After I responded to you before I did some digging. Under Solaris/SPARC with the 64 bit JVM people have reported allocating 32GB with no problem. I don't know the actual maximum but you may have to rethink your deployment platform.

And which is an example of a 64 bit JVM ?

[Roel De Nijs]

Originally posted by jhon Reader:
i heard for jdk 1.5 ,
sun has tested it for 500 gb of ram
check it out if it is correct

Problem we encounter: we also use another (bought) application and when i try to run it with JDK1.5 it gives an error, so we can't use the 1.5 JDK until the vendors of our bought application release a JDK1.5 compliant application.

So i think JDK1.5 isn't an option for the moment, but thanks for the suggestion. I 'll check it out and ask vendor when they are releasing 1,5 compliant code.

[William Brogden]

All ideas will be taken in consideration, because the memory is really important key factor. But i don't quite understand what you mean, so is there any docs about this, or can you be a little bit more detailed in what you mean.

If I understand your application problem correctly, you need a bunch of data in memory so that you can create a small GIF. This sounds ideal for distributed processing using a JavaSpace. The idea being that an instance of your application, running in a JVM separate from the server, would get rendering jobs from the JavaSpace and return the result through the space.

A recent web seminar at jini.org titled Distributed Image Reformation Using JavaSpaces Technology may be a good introduction.

I actually have a micro example of distributed processing using a JavaSpace running on my web site. This page talks a bit about JavaSpaces and has a link to the demo. Instances of a phonetic lookup application are running on various places in my network, each one has a different dictionary in memory for speed in lookup. Each instance has registered with a JavaSpace as being able to take certain classes of phonetic lookup job. I would be glad to provide more details.
Bill

[Roel De Nijs]

Originally posted by William Brogden:

A recent web seminar at jini.org titled Distributed Image Reformation Using JavaSpaces Technology may be a good introduction.

I actually have a micro example of distributed processing using a JavaSpace running on my web site. This page talks a bit about JavaSpaces and has a link to the demo. Instances of a phonetic lookup application are running on various places in my network, each one has a different dictionary in memory for speed in lookup. Each instance has registered with a JavaSpace as being able to take certain classes of phonetic lookup job. I would be glad to provide more details.
Bill

I took a look at the distributed image reformation and also to your example, and i think it's worth taking JavaSpaces in consideration. but it's still really obvious to me what have to be done to use this. But the main idea is something like this: you have one large pc where your data is stored and then a lot of small ones that take some of the stored data and do the processing and store the result on the 1 large pc. Is this a little bit in the neighbourhood of the idea behind JavaSpaces ???

So is there any documentation about this or an example (with some dummy classes) where step by step is explained what you have to do?

[Mike Curwen]

my mind is boggling.

... now it's 1,5min with 250 MB used for 1 client. So now we are at a point where performance is acceptable (and really high) with as less as possible memory used.

1.5 minutes per request, and 250MB per user is both acceptable and the least memory possible?

Recognizing that not all apps are built alike, and admiting I've never worked on big-iron enterprise apps... I find it hard to understand how it's possible to have 250MB of data that is completely non-shareable and critical to retain 'in memory'.

Two questions I'd ask myself:
1) Is there absolutely no data in that 250MB that can be shared between all users?
2) Have I investigated the usage of a data abstraction/querying/caching layer enough?

[Roel De Nijs]

Originally posted by Mike Curwen:
my mind is boggling.

... now it's 1,5min with 250 MB used for 1 client. So now we are at a point where performance is acceptable (and really high) with as less as possible memory used.

1.5 minutes per request, and 250MB per user is both acceptable and the least memory possible?

Recognizing that not all apps are built alike, and admiting I've never worked on big-iron enterprise apps... I find it hard to understand how it's possible to have 250MB of data that is completely non-shareable and critical to retain 'in memory'.

Two questions I'd ask myself:
1) Is there absolutely no data in that 250MB that can be shared between all users?
2) Have I investigated the usage of a data abstraction/querying/caching layer enough?

Oke, i go a little bit more in detail:
- our app is built to visualize the sewer system. so a user can select a village and then he gets an image (gif-file) with the villages sewer system. and then he can zoom in/out, recenter, print, ... The gif-file is generated by a bought application (mapXtreme)
- so when i talk about the 1.5min & 250MB, this is the largest village. it contains about 26000 objects (nodes & pipe lines). A node-record has 120 database-fields, a pipe line-record has about 60 database-fields. all these 26000 records are read from database and only those who aren't NULL in db, are kept in memory (according to the manner of working described by mapXtreme). It's done like this, like I said it many times: to get an acceptable performance.
- the 1.5min is the initial request from the user (read all objects from db, keep it in memory & generate the image). The image generation take about 20 secs.
- in the 250 MB there is some data that's shared by all users and is kept once in memory, but that's max 15-20%.
- Of course not everyone is interested in the largest village, thus the less objects there are in a db, the less memory is used. But we must provide that a certain amount of concurrent users can load the largest village. In a next phase user must be able to update values of db-fields, so i think it's not an option to keep once all the objects from a certain village in memory and share this by all the users that want to see this village.

[Roel De Nijs]

I used the method Runtime.getRuntime().totalMemory() and it gives as output 195911680 (186 - 187 MB) and in Task Manager is displayed that it takes 201 MB, so a difference of 15 MB. What causes this difference ?

[William Brogden]

The way a JavaSpace would be applied here is that requests for image creation from the data for a certain village would be written to the space. Any PC in the network that already had that data in memory could take that request, do the calculation and return the GIF. The PCs doing the calculation have to have large memory, while the PC handing out requests can have smaller memory. I gather that a typical user is only going to be interested in one village but with repeated views at various scales so this would work rather well. You would have a "rendering farm" of PCs having large memory, fast ethernet but slow/cheap processors.
Naturally you would have to have a mechanism by which the system can decide that no "worker" machine has the data in memory and request that one be assigned to gather the data.

What an interesting problem!
Bill
[ November 02, 2004: Message edited by: William Brogden ]

[Roel De Nijs]

Yeah, it's very intresting to search for a solution that's for everyone satisfying, and a year ago i was still studying so i learned in these months (from august till now) more then in those 3 years.

and how do i have to picture such a JavaSpace? is it a pc or a kind of database or ...?

So user selects from combo a village, the unique number of the village is written to this (spooky) JavaSpace. There a mechanism looks around which worker-pc has loaded this village and the image is rendered on that worker-pc. the result is given back to the javaspace.

Couple remarks:
- so if i got three worker-pcs i will also need 3 licenses of the rendering-software , and 1 license costs about 15000 dollar
- in a second phase user must be able to update the database-fields, so if 1 worker has data in its memory that's shared by 2 users and they both do an update, i think you'll have concurrency-problems with inconsistent data as consequence

Any other suggestions on this remarks...

[William Brogden]

Picture a JavaSpace as sort of a contractor handing out jobs to workers based on what each worker says it can do. It takes a bit of a shift in viewpoint to understand this

- so if i got three worker-pcs i will also need 3 licenses of the rendering-software , and 1 license costs about 15000 dollar

AH! NOW you mention the real controlling factor in your design project - that throws all previous calculations out the window. I bet that is one of those annoying "per CPU" licenses, isn't it.
Consult a local gaming fanatic for advice on overclocking and watercooling your CPU (just kidding.)
You can in fact run all your JavaSpace workers inside the same CPU - thus minimizing the license fee.
My little demo with phonetic coding starts three worker Threads, each one reads a particular dictionary of words, then tells the space manager (which can live on the same or different machine) that it is ready to look up words from a particular dictionary.
Bill

[Roel De Nijs]

Originally posted by William Brogden:
I bet that is one of those annoying "per CPU" licenses, isn't it.

you've got it

[Roel De Nijs]

Originally posted by William Brogden:

My little demo with phonetic coding starts three worker Threads, each one reads a particular dictionary of words, then tells the space manager (which can live on the same or different machine) that it is ready to look up words from a particular dictionary.
Bill

What's then the difference between those JavaSpaces and 1 pc with 3 or more instances of Tomcat with a load balancer in front of them ???

[William Brogden]

What's then the difference between those JavaSpaces and 1 pc with 3 or more instances of Tomcat with a load balancer in front of them ???

I don't understand why you are even considering multiple instances of Tomcat, just hold all of your data for a given village in an object that lasts longer than a single request/response cycle. Your servlet can keep a hashmap of these data objects and direct a request to the correct one. Since it is likely that a given user is doing repeated requests to the same village, most requests will find the data already in memory. Some sort of least-recently-used mechanism would let you discard data objects when memory gets low.
If you went the JavaSpaces route, there are three possible advantages:
1. The Tomcat server could live elsewhere on the network without using memory in your rendering machine - you could then cache images on that system so that repeated requests would not need to be re-rendered.
2. Each JavaSpace worker can live in a separate JVM - if it crashes, the system as a whole stays up.
3. If you had to double the capacity you would just plug in a new rendering machine.
Bill

[Roel De Nijs]

At the moment i keep a hashmap in memory with the session-id of the browser (key) and a class containin all info to render the image (value) and it's kept in memory until the browser is closed and then this class is collected for garbage. So data-retrieval from database happens 1 time at initial request.

Because we have a per cpu license, we have to implement all workers on 1 pc and then make several threads, so when one crashes the whole system will crash too or not?