How to leverage a server with 32 go RAM

[Eric Malalel]

I currently host customers applications on a single Tomcat instance on a server with 4 Go RAM.
I will upgrade and rent a new server with 32 Go RAM.
How can I leverage this RAM to serve more customers with the same server?
I was thinking of installing 10 Tomcat instances, each running its own JVM with up to 2 Go RAM.
is this a good idea?
If yes, how to do it? Especially, do I need specific configuration to have each Tomcat running in its own JVM ?

[Karthik Shiraly]

I was thinking of installing 10 Tomcat instances, each running its own JVM with up to 2 Go RAM.

This is a good idea from the point of view of process isolation.
But it brings with it one constraint - that each tomcat instance has to listen on a different port. They can't all listen on the usual port 8080.
That doesn't look so bad at first, but turns out to be a major downside both for you and for your customers who wish to deploy their webapps in your server, as described below.

A typical use case is, customer deploys their webapp "cust1". Let's say you allocate them the tomcat instance on port 8082.
Their URL is now "http://your.domain.com:8082/cust1/".
But your customer wants that webapp to be served via their own domain "http://theirshinynewdomain.com" (implicitly port 80).
It's now upto you to setup the translation of "http://theirshinynewdomain.com" to "http://your.domain.com:8082/cust1/".
This is typically done by installing a reverse proxy like apache httpd with virtual hosts configuration, but it is you, not your customer, who has to configure this.
I suspect you must already be doing this for your current tomcat instance, but anyway just something to remember.
If they want to change their domain, they have to contact you and you'll have to reconfigure the virtual hosting correctly.

The downside for your customers is that their developers have to be more disciplined about URLs. If they use a URL like "/images/photo.png" in their HTML or javascript or JSON ajax responses because it works in their local developer tomcat, those url's when deployed will go to client as "http://your.domain.com:8082/images/photo.png". Not at all what is expected.
Even the solution is not straightforward. You may end up in a situation where you'll have to use mod_substitute or mod_proxy_html to replace all proxied URLs with the correct URLs, or ask them to change their code.
All this is quite difficult to get correct in practice, and details are likely to get missed in communication gaps between your customer, their developers, and you.

If this is a physical server you've rented, I think I'd look at virtualization (like ESXi / KVM / Xen) + a self management control panel like cPanel. Basically, create a dedicated virtual private server for each of your customers and run 1 tomcat instance in each virtual server. Your customers can use the control panel themselves to configure their tomcat instance and deploy their own webapps. I'm sure there are other details which may turn out to be complicated - for example, how to give each of your virtual servers a publicly reachable IP.

If this rented server is itself a virtual private server, then virtualization is probably not recommended, even if it works at all. Then your options are reverse proxying as mentioned above, or possibly containers like LXC (but I'm not very familiar with them and I might be completely wrong, just one more thing to investigate).

[Eric Malalel]

Thanks for taking time to put a detailed answer.
This is a Windows 2012 server and I use IIS 8.5 as front end of Tomcat 8, and it is very easy to define rewrite rules on IIS so domain of customer 1 is redirected to one tomcat instance on port 8080, and domain of customer N is redirected to another tomcat instance on another port.
Change in domain names are not so frequent so I can handle that too.
I do not deploy customer applications, I only deploy a solution that we have developed and rent to our customers, so taking care of URL is something we can handle too.
So it sounds it is worth doing this way.
On my current server, I have almost every two weeks a java error on permgen space. I hope I will get rid of it by setting java maxpermsize to 2 Go for each Tomcat instance, while I currently use 1 Go on my current server. Is this correct or do I need to set other parameters which may be useful to solve this issue?

[Karthik Shiraly]

Eric Malalel wrote:On my current server, I have almost every two weeks a java error on permgen space. I hope I will get rid of it by setting java maxpermsize to 2 Go for each Tomcat instance, while I currently use 1 Go on my current server. Is this correct or do I need to set other parameters which may be useful to solve this issue?

You have not mentioned the JVM being used, but I'm guessing it's the popular Oracle/Sun Hotspot VM.
For that VM, the
total tomcat memory usage = Heap size + permgen size + C heap for JVM's internal data structures

So if you set maxpermgen size to 2Gb, maxheap size to say 1Gb and let's say JVM internal usage is some 128MB,
that's a total of some ~3.1 GB reserved memory for 1 tomcat instance.

10 instances brings it to ~31GB just for the tomcat instances, leaving you with just 1 GB for the Windows OS, IIS, etc.

I don't know if 1GB for Windows OS + IIS + whatever else you are running on that server (like AV, firewall) are enough.
It seems too close to the physical RAM limit, increasing the chances of heavy swapping to disk.

You must already be having a good idea of your server's non-tomcat memory usage.
So set the sizes accordingly - probably 2GB permgen is too high.

As for getting rid of the problem, setting permgen only delays the problem.
Increasing the permgen will certainly reduce the frequency of this error
(ie, it may happen only once in 6-8 weeks, for example) but not solve the root cause.

The root cause is that everytime a web app is deployed or redeployed on tomcat,
the class metadata of the previous deployment is not garbage collected and accumulates in the permgen space.

One solution is to configure your JVM to switch to a garbage collector that is capable of cleaning up permgen.
Both the G1 collector and CMS collector handle this permgen problem in different ways.
See https://blogs.oracle.com/poonam/entry/about_g1_garbage_collector_permanent for more info.

G1 is only available from some release of JRE7 as far as I know. You can set "-XX:+UseG1GC" and see if it's enabled.
CMS is available in JRE6 and 7. Enable it with "-XX:+UseConcMarkSweepGC -XX:+CMSClassUnloadingEnabled",
This is in addition to setting the "-xx:+MaxPermSize"

A more drastic solution is to switch to JRE8, which has got rid of permgen altogether.
But switching JRE may cause unexpected problems for your customers.

With either approach, I think it's good to advise your customers about the solution so that they don't run into unexpected problems.

You can also monitor your tomcat instance(s) using jconsole or jvisualvm tools which come with JDK (see the screenshot of jconsole below).
They can tell you if your permgen problem is solved to some extent.
Constantly increasing steps in the permgen pool indicates that the problem is not solved.
With JRE8, you should not see a permgen at all and ideally see the heap size line drop everytime an application is redeployed.

[Tim Holloway]

If you have had reliable operation with multiple clients in one JVM, I'd keep doing that. The downside of running multiple JVM's is that they each are going to be using a certain minimum amount of system RAM and if the JVM's are over-committed (to avoid peak-memory failures), then you'll have a lot of wasted RAM, which in turn reduces your overall system capacity. Within a JVM you have extensive application isolation, but a shared memory pool, so memory can be used more efficiently.

If your apps weren't that reliable, I wouldn't recommend that, since you don't want one unstable app potentially bringing down everyone else, but if you trust things to behave well, it's a different matter. There are also some (mostly experimental) JVMs that further isolate apps within themselves, IBM and WebLogic/Oracle have done work in that area. You might want to investigate Oracle's jRocket. Presumably the level of isolation for that solution is high enough that you'd be running multiple Tomcat instances, even though they all ran under one JVM, unlike the standard deal of one JVM/one Tomcat.

You could also split the difference and run multiple Tomcats, but several clients per Tomcat.

If you're really creative and own the hardware, there's probably some options in VM and/or container technology that could be used to run multiple Tomcats while sharing the RAM that is identical in all instances. Since Tomcat itself isn't very big, I don't know if that would help much, though, unless you're talking dozens of them.

Definitely, it's a challenge to keep multiple Tomcats configured and mapped to public URLs. I'd be using something like Puppet or Ansible on my Linux servers. It Puppet I could keep the specs in a database. Not sure what the best approach for a Windows machine might be.

[Karthik Shiraly]

Tim Holloway wrote:Oracle's jRocket. Presumably the level of isolation for that solution is high enough that you'd be running multiple Tomcat instances, even though they all ran under one JVM, unlike the standard deal of one JVM/one Tomcat.

That sounds very interesting. I'll have to investigate this JRocket ( .... though oracle's crappy site seems to be doing its best to discourage anybody from downloading it and the only available version is Java 6 )

Edit: Ok, a little more searching and I came across https://blogs.oracle.com/henrik/entry/java_7_questions_answers:

When will JRockit be available for Java 7?
A: It won't. As we explained last year we are merging JRockit and HotSpot into one single JVM. JDK 7 contains the first release of this converged JVM, where one of the first steps was to start removing the PermGen concept. Future JDK 7 updates will complete the PermGen removal, as well as add more visible features from JRockit.

and coincidentally, it seems our buddy permgen was a big enough issue to warrant merging two different JVMs!

[Tim Holloway]

PermGen has been a major pain for Tomcat ever since Tomcat 5. That was when I first started running out of PermGen space when I attempted to redeploy apps as I updated them. The PermGen space allocated for the previous instance of the app remained allocated, but the new instance also allocated PermGen spaces (a lot of it, if the app was using something like Hibernate JPA), and before you knew it, the entire Tomcat server had crashed.

PermGen will not be missed. Except in the way you'd miss a painful infection.