I'm running a small website with Tomcat 5.5 serving up jsp and the (relatively few) http requests.
Recently, I've begun seeing problems in which the site will suddenly stop responding. I don't see any error messages written to the catalina.out file or any other file in the tomcat/logs directory. But attempts to access the page (even requests on the same server via localhost) eventually just time out.
I'm at my wits end. Nothing changed in the tomcat configuration files, and nothing else is running on the server. (CentOS 5.2).
I'm not a Tomcat expert by any means, so any help would be appreciated.
You might use the Tomcat Management application to monitor the server. You might see a trend - for example I caught a bug in which the request Threads ended up waiting indefinately for a service that had been removed. Eventually all available request Threads were consumed.
Hope this gives you some corners to peel at. Does this help?
Thanks for your help. The process is still alive and is consuming 0% of the CPU. I'm not sure if it's still listening on the server point at that time or not. I'll follow those instructions the next time it happens and will let you know.
William Brogden wrote:You might use the Tomcat Management application to monitor the server. You might see a trend - for example I caught a bug in which the request Threads ended up waiting indefinately for a service that had been removed. Eventually all available request Threads were consumed.
Thanks, Bill. Can you point me to any how-to guides for using the Tomcat Management application to monitor the server? I've never done that before. If all request threads were consumed, wouldn't an error be written to the log file? I'm not seeing any errors being written out.
The problems occurred again last night, right on schedule. I ran a netstat and a jstack each time when the site wasn't responding, and nothing looked out of the ordinary.
Netstat shows one line: "tcp 0 0 :::80 :::* LISTEN 20604/java" indicating that the process is still listening on port 80, and it shows 16 other entries talking to port 80, either in a "FIN_WAIT2", "FIN_WAIT1", "CLOSE_WAIT", or "ESTABLISHED" state.
The jstack dump shows 36 threads, 5 with a state of "IN_WAITING", the rest with a state of "BLOCKED".
The strange thing is that apart from last weekend, when we saw the problem all weekend long, on week days the problem only occurs between 10:00 pm and 1:00 am. Some nights the problems stop as early as 11:20 pm, and some nights as late as 1:10 am. There's nothing scheduled in the crontab to run between those hours.