• Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other Pie Elite all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Jeanne Boyarsky
  • Ron McLeod
  • Paul Clapham
  • Liutauras Vilda
Sheriffs:
  • paul wheaton
  • Rob Spoor
  • Devaka Cooray
Saloon Keepers:
  • Stephan van Hulst
  • Tim Holloway
  • Carey Brown
  • Frits Walraven
  • Tim Moores
Bartenders:
  • Mikalai Zaikin

Tomcat crashes with Out of Memory Error!

 
Greenhorn
Posts: 15
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Dear Experts,

I seek your advise/solution for one of my ongoing production problem.
My tomcat in production machine crashes abruptly with logs in "catalina.out" as "Out of Memory".
Here is some useful information for you.

OS:Red Hat Enterprise Linux Server release 5 (Tikanga)
Kernel \r on an \m

bash# uname -a
Linux xxxx 2.6.18-8.el5 #1 SMP Fri Jan 26 14:15:21 EST 2007 i686 i686 i386 GNU/Linux

Tomcat version:
Server version: Apache Tomcat/6.0.29
Server built: July 19 2010 1458
Server number: 6.0.0.29
OS Name: Linux
OS Version: 2.6.18-8.el5
Architecture: i386
JVM Version: 1.5.0_01-b08
JVM Vendor: Sun Microsystems Inc.

Tomcat CATALINA.SH settings:JAVA_OPTS="-Djava.awt.headless=true -Dfile.encoding=UTF8 -server -Xms2048m -Xmx2048m
-XX:NewSize=512m -XX:MaxNewSize=512m -XX:PermSize=512m
-XX:MaxPermSize=512m -XX:+DisableExplicitGC -XX:+UseConcMarkSweepGC -XX:+UseParNewGC"



Logs in "catalina.out"


Exception in thread "CompilerThread0" java.lang.OutOfMemoryError: requested 1356912 bytes for Chunk::new. Out of swap space?

1. Is there any thing to do with application?
2. Does it seems OS/Hardware problem? (By the way server has 3Gig RAM and its IBM Blade server)

Regards,
Rashmin Gadhavi




 
Author and all-around good cowpoke
Posts: 13078
6
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
How long does it run before crashing? -or- How many requests processed before crashing?

If this was my problem I would be using the Tomcat Management App to frequently check on available memory, number of request threads running, number of sessions, etc.

Bill
 
RashMin Gadhavi
Greenhorn
Posts: 15
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

First of all, thanks a lot William for your response.

1. How long does it run before crashing? - Its random
2. How many requests processed before crashing - I have not measured yet (How to measure I do not know)

Also I logged in to Tomcat manager section and got following information as you suggested.

jk-8409
***************
Max threads: 200 Current thread count: 20 Current thread busy: 11
Max processing time: 4398274 ms Processing time: 5746.116 s Request count: 3223 Error count: 3 Bytes received: 2.24 MB Bytes sent: 58.79 MB

More detailed status of my application which is hosted in "webapps" directory of tomcat.
***************************************************************************
Start time: Thu Dec 15 06:17:30 EST 2011 Startup time: 43 ms TLD scan time: 19 ms
Active sessions: 44 Session count: 489 Max active sessions: 49 Rejected session creations: 0 Expired sessions: 445 Longest session alive time: 13358 s Average session alive time: 3212 s Processing time: -4397991 ms
JSPs loaded: 49 JSPs reloaded: 49


Do I have to run JProfiler? Since its a production machine..its difficult for me to run JProfiler though I can do that, it at all its required.


Regards,
Rashmin Gadhavi


 
Sheriff
Posts: 22781
131
Eclipse IDE Spring VI Editor Chrome Java Windows
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
There are two reasons you get out of memory errors:
1) You have a memory leak somewhere. You keep references longer than you should, possibly even indefinitely. JProfile could definitely help you find the causes.
2) Your server simply cannot handle the load.

I suspect the first, but you really would need to profile your application. Perhaps you can create a copy in a test environment. You probably won't see the errors (or at least not directly), but you should still be able to find memory leaks.
 
William Brogden
Author and all-around good cowpoke
Posts: 13078
6
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
The Manager app should also show current memory use - for example, mine just reported:



How much Free memory does your system show?

Bill
 
RashMin Gadhavi
Greenhorn
Posts: 15
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
William,

Here is the free memory information I got from my tomcat's manager application.

Free memory: 1701.19 MB Total memory: 2047.56 MB Max memory: 2047.56 MB

We have a very good IBM Blade server, but my application runs on a VM machine with given 3gig memory to VM software. I think it should be fine with tomcat.
Only 2 applications are loaded in tomcat's web apps.

Regards,
Rashmin Gadhavi
 
William Brogden
Author and all-around good cowpoke
Posts: 13078
6
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Good first step.

Now, keep monitoring - frequently check the Management app for number of requests, long running Threads and memory use.

The management app also reports the number of requests and number of HTTP errors - all important information for tracking down your problem.

If free memory gradually decreases as the number of requests goes up, that indicates a programmatic memory leak like Rob said.

Bill

 
New rule: no elephants at the chess tournament. Tiny ads are still okay.
a bit of art, as a gift, the permaculture playing cards
https://gardener-gift.com
reply
    Bookmark Topic Watch Topic
  • New Topic