• Post Reply Bookmark Topic Watch Topic
  • New Topic

How to debug a thread in production environment?  RSS feed

 
Rahul Toaikani
Ranch Hand
Posts: 36
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
if a thread has some performance issue,slows down the system for some reason or appears in deadlock situation, how do you debug? (other than looking at the logs).

How do you do this in production environment and in developer environment?

Appreciate your input.
 
Stan James
(instanceof Sidekick)
Ranch Hand
Posts: 8791
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
This probably varies by JVM and OS and server, but we were able to get thread dumps - snapshots of what's happening right now - which gave good information about what method every thread is in. We found a huge number of threads in a single method which helped us find something that ran too long. Then again, many threads were in the logger just because it is called so often.

Later we used Wily Introscope which is quite expensive but gives moutains of information about all aspects of performance. I think Mercury has something comparable.
 
Chris Hurst
Ranch Hand
Posts: 443
3
C++ Eclipse IDE Java
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
JMX is worth a look if your on a JVM 1.5 or above, for deadlock then thread dumps (as previously stated) will quickly give the game away how you get them depends on your OS i.e. unix signal, windows key combination etc etc i.e. if you give us an exact idea of what OS, JVM your intending to deploy with, we can help more :-)
 
Rahul Toaikani
Ranch Hand
Posts: 36
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Thanks Stan and James for the response.

OS: Windows 2000 server.
I found good artcile about stacktrace from SUn.
http://java.sun.com/developer/technicalArticles/Programming/Stacktrace/

But how about performance? Now I understand using dumps, you can see what is current thread and what method has lock etc., but are there any ways to figure which thread takes up more space and time etc., will the regular profiling tools help?
 
M Easter
Ranch Hand
Posts: 133
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Using formal profiling tools is undoubtedly the best practice, but using the log files to your advantage can be useful, if you have the opportunity/time to modify the source.

I have seen effective use of a timer/stopwatch at certain boundaries of a system (e.g. the DB layer, or the client layer to RMI/Corba calls). IMHO, these should be "baked-in" right from the start of development. The stopwatch should be configurable: on/off, a threshold for which "fast" events are not logged, etc. The idea here is not to determine which _thread_ is slow but rather to find out which _activity_ is slow.

FWIW, I've written a modest blog entry on this:

http://codetojoy.blogspot.com/2007/02/stopwatch-idiom.html
 
steve souza
Ranch Hand
Posts: 862
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I have seen effective use of a timer/stopwatch at certain boundaries of a system


JAMon does the things you request and more.

It works like a stopwatch that you pass a String to. This String represents what you want to time (a page name, method, sql, etc). Whenever you pass jamon the same String aggregate statistics are gathered. i.e. You will have collected the following statistics (in ms.): hits, avg, total, min, max, std dev, when last invoked, and whether or not the code is currently running in addition to some other useful stats. The data is viewable via the jamon.war or programmatically.

It is easy to use, fast, and open source.


You can also monitor your database inteteractions and web pages with NO code changes using JAMon. You can use the jamon jdbc driver to gather all statistics for queries and jdbc calls, and you can use the jamon servlet filter to gather statistics for all pages in your web app. Both only take a couple minutes to set up.

See the links below for more info including a live demo...
 
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
Boost this thread!