Win a copy of Reactive Design Patterns this week in the Functional Programming forum
or Learn Linux in a Month of Lunches in the Linux forum!
  • Post Reply Bookmark Topic Watch Topic
  • New Topic

My production server (tomcat-7) get shutdown automatically

 
saravanan sambandam
Ranch Hand
Posts: 56
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hi,

We have created mobile application for that java is an service provider (Presentation Service), which (PS) get request from apps and sent it to WCS(Web Commerce Server 7).  Getting xml based response then we are formatting as json and sending it back to mobile apps.  This was doing the PS layer. 

We have created web project using spring framework, also we have created another one web project which is get the request from PS and get data from endeca and send it back to PS.  This is small application deployed on the same server (tomcat-7).

My application deployed on two instance server#1 and server#2 (tomcat 7).  These two instance are installed two Linux system.  Those two instance are tested with load balance (concurrent user 1000) very beginning.  It was OK.   Now, we are facing issue like Servers are getting shutting down automatically.    We suspected there was some memory leak.  We have executed with jMap tool on two PRODUCTION server, then generated with heapDump.   It shows like



We have provided heap size is min:512m & Maxsize:1024 on tomcat setenv.sh itself.

Do you have any idea, why this occurring.   What i have done the mistake.

The report is below:


Top Consumers : Retained Set :  Possible Memory Waste :  Duplicate Strings

Found 48 occurrences of char[] with at least 10 instances having identical content. Total size is 4,309,320 bytes.

Top elements include:
•16,009 × product.review.avg_rating_range (80 bytes)
•15,844 × Current_Offers (48 bytes)
•15,307 × Brand (32 bytes)
•7,243 × Price (32 bytes)
•9,025 × Type (24 bytes)

Details »

In our spring-config.xml file, we have provided the scope is empty.  So that it act as singleton.  Then how the instance is having same content.   Do you have any idea or any suggestion.  Please let me know

Thanks
saravanan s


[Rob] split a long line
heapdump.JPG
[Thumbnail for heapdump.JPG]
heapdump
duplicate_string.JPG
[Thumbnail for duplicate_string.JPG]
duplicate_string on 10 instance
 
Tim Holloway
Saloon Keeper
Posts: 18388
58
Android Eclipse IDE Linux
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Tomcat does not "shut down automatically". It has to be told to shut down. Or crash, which seems to be what you suspect.

If the JVM abnormally terminates, there should be a JVM dumpfile in the Tomcat directory which can be analyzed. If Tomcat shuts itself down because of an OutOfMemory Exception, then there should be something in the Tomcat log that tells what was being attempted when the condition occurred.

You do seem to have 3 classes that have a fairly large and almost identical number of instances. Very possibly because they cross-reference each other. So the first step would be to determine what causes these instances to be created and whether they are being disposed of properly when no longer needed.If there is some reason why you want to have as many of these available at all times as possible, you might be in need of some sort of caching system.
 
saravanan sambandam
Ranch Hand
Posts: 56
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Thanks for your answers..


Could you able to identify the classes from the screen shot.   Here, those string are getting it from endeca (dB).
 
Tim Holloway
Saloon Keeper
Posts: 18388
58
Android Eclipse IDE Linux
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Take a look at an object dump. That many strings would normally have a correspondingly large number of objects containing them.
 
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
Boost this thread!