• Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other Pie Elite all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Jeanne Boyarsky
  • Ron McLeod
  • Paul Clapham
  • Liutauras Vilda
Sheriffs:
  • paul wheaton
  • Rob Spoor
  • Devaka Cooray
Saloon Keepers:
  • Stephan van Hulst
  • Tim Holloway
  • Carey Brown
  • Frits Walraven
  • Tim Moores
Bartenders:
  • Mikalai Zaikin

Thread stuck problems - StuckThreadMaxTime exceeded

 
Greenhorn
Posts: 13
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Hi,
I am having the problems running continuous 24 hrs test against Weblogic 8.1. After some time the server starts complaining about the threads stuck
<Warning> <WebLogicServer> <BEA-000337> <ExecuteThread: '7' for queue: 'weblogic.kernel.Default' has been busy for "630" seconds working on the request "weblogic.ejb20.internal.JMSMessagePoller@d64412", which is
more than the configured time (StuckThreadMaxTime) of "600" seconds.>
This happens for a number of threads and then the server goes into a continuous period of inactivity trying desperately to free memory and then eventually it start giving "java.lang.OutOfMemoryException"
My queries are
1) How to detect on what the thread is stuck on
2) In case of a stuck thread - how to roll back/stop the thread so that the rest of the processing can continue.
Thanks
Navjeet
 
Ranch Hand
Posts: 41
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
hi,
I am also facing similar problem.
This is problem reported due to JMS Server.
Cause the warning tells about JMS Server port only.
With me I am not facing OutOfMemoryError.
Navjeet, let me know were you able to solve ur problem or not.
Is there any tuning to be done on JMS server ?
 
Ranch Hand
Posts: 47
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Hi,
I am also getting the same problem.
How did you resolve this issue?
Thanks,
Purvesh
 
Greenhorn
Posts: 29
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Hi I am also getting the same problem while doing batch query on DELETE operation in a DAO class.

removeMetaDataDefinition-------------->DELETE FROM TDMS_METADATA_DEFN WHERE RESCLASSID ='05011132220000180' AND METADATADEFNID ='05011132190000179'
<Feb 5, 2005 12:09:20 PM IST> <Warning> <WebLogicServer> <BEA-000337> <ExecuteThread: '11' for queue: 'weblogic.kernel.Default' has been busy for "864" seconds working on the request "Http Request: /dmsweb/folder.do", which is more than the configured time (StuckThreadMaxTime) of "600" seconds.>
<Feb 5, 2005 12:09:21 PM IST> <Warning> <WebLogicServer> <BEA-000337> <ExecuteThread: '13' for queue: 'weblogic.kernel.Default' has been busy for "1,539" seconds working on the request "Http Request: /dmsweb/folder.do", which is more than the configured time (StuckThreadMaxTime) of "600" seconds.>
<Feb 5, 2005 12:19:21 PM IST> <Warning> <WebLogicServer> <BEA-000337> <ExecuteThread: '10' for queue: 'weblogic.kernel.Default' has been busy for "828" seconds working on the request "Http Request: /dmsweb/folder.do", which is more than the configured time (StuckThreadMaxTime) of "600" seconds.>
<Feb 5, 2005 12:19:21 PM IST> <Warning> <WebLogicServer> <BEA-000337> <ExecuteThread: '11' for queue: 'weblogic.kernel.Default' has been busy for "1,464" seconds working on the request "Http Request: /dmsweb/folder.do", which is more than the configured time (StuckThreadMaxTime) of "600" seconds.>


If you have solved the problem please let me know

thanks in advance
 
ranger
Posts: 17347
11
Mac IntelliJ IDE Spring
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
"sathish"-
Welcome to the JavaRanch! Please adjust your displayed name to meet the

JavaRanch Naming Policy.

You can change it

here.

Thanks! and welcome to the JavaRanch!

Mark
 
drifter
Posts: 1364
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
I check on stuck threads in the console. There are other ways. See BEA Documentation.

Of course the console looks a little different in each version, I'm looking at 8.1 right now.

Go to the managed server node of interest in the node tree (list on left -- mine shows a world icon for my domain; under that is the "servers" folder; within the servers folder are my three servers).

Fo direct access to threads and what they are doing -- right click on the server node -- select "View Execute Threads". Look to see what the Default threads are "doing" in the "current request" column.

Alternatively you can go the the server's "Monitoring Tab". Select monitor all active Queues. There you can see the number of threads idle for each type of thread. Your user's requests will go to the default queue unless you configure it differently. Click the link for "weblogic.kernel.Default" to see what those threads are doing.

Then what to do? I have a process that runs a few hours and I get a stuck thread warning (1 thread), but no harm to the server. I don't do anything about it.

Check Tuning WebLogic Server Applications, there's a section on "Using Execute Queues to Control Thread Usage". Maybe you just need more threads. Maybe you need some of your application to use a different queue. Maybe your application needs tuning.

Once you know what your threads are stuck on you could run a profiling tool like Optimizeit or Jprobe. You could have tune your db use.

Regarding stopping threads. That's a tricky business and I'm pretty sure WebLogic doesn't support killing threads.
[ February 09, 2005: Message edited by: Carol Enderlin ]
 
Ranch Hand
Posts: 977
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Originally posted by Navjeet Nehra:

1) How to detect on what the thread is stuck on
2) In case of a stuck thread - how to roll back/stop the thread so that the rest of the processing can continue.

Thanks
Navjeet




1) The best way to discover it is taking a thread dump, to do that you�ll have to start the server in foreground(so you can see the console messages) or redirect the standart output to a file, then all you need to do is hit CTR+BREAK on windows or use the command kill -3 on unix environments, it�ll provide you with a thread dump where you can see exactly where your thread�s are stucked.

2) You can�t roll back a stucked thread, usually as Carol mentioned you�ll have to tune the Default thread queue on weblogic and also tune your application, most of the problems like this i have faced were solved tunning the DB then the application and then, if still required weblogic where you can tune thread count, garbage collection, etc.
You can also consider programming a threshould so the node manager can reboot the managed server in case of too much thread count usage(this will actually kill all running threads of course).

best regards.
 
Greenhorn
Posts: 10
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Hi,

Found this thread to be interesting, I am facing similar problem, my application has 200 as thread queue and when this problem occurs , I was not even able to get into the console.

Could any one of you help me in solving this ?

If tuning in the database is required, what kind of tuning is to be made. I would appreciate your early response.

-- Vetrik
 
Greenhorn
Posts: 24
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Thread dump would help to nail down the issue, also check is there any DB access your app is doing,is it waiting to get connection.,is there any operation going on in DB at that point of time..
 
Greenhorn
Posts: 21
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Hi,
Our app also having same StuckThread problem.My senior said this is because of the weblogic is redirecting the request to a proxy server then to a external system.if the proxy is hanging the weblogic server is getting warning like "ExecuteThread: '17' for queue: 'default' has been busy for "816" seconds working on the request "de.dl.crl.facade.sb.CRLFacadeBeanEJB_r251p2_EOImpl",which is more than the configured time (StuckThreadMaxTime) of "600" seconds.>"

Now we want to track this problem and return back to the user with proper excpetion related information. Is there a way to get this proxy status from weblogic??
if anything is not clear please tell me.
Thanks in advance.
 
Greenhorn
Posts: 2
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Hi all,
Any solution for this?
 
Greenhorn
Posts: 1
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Originally posted by J Praveen Reddy:
Hi all,
Any solution for this?



Please have a look at

https://support.bea.com/application_content/product_portlets/support_patterns/wls/JDBCCausesServerHangPattern.html

Where it mentions:
This does not interrupt the thread, as this is just a notification for the administrator

Hope this is useful..
 
Greenhorn
Posts: 1
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
we still didn't find the solution for this?

I am having the same problem.

Its been revolving from long back.

Please find a solution for this.
 
Greenhorn
Posts: 10
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Check out this link
http://e-docs.bea.com/wls/docs81/perform/WLSTuning.html#1125714

to be precise, do the following
To configure WebLogic Server stuck thread detection behavior:
1. Start the Administration Server if it is not already running.
2. Access the Administration Console for the domain.
3. Expand the Servers node in the left pane to display the servers configured in your domain.
4. Click the name of the server instance that you want to modify for improved stuck thread detection.
Note: You configure stuck thread detection parameters on a per-server basis, rather than on a per-execute queue basis.
5. Select the Configuration —> Tuning tab in the right pane.
6. Modify the following attributes as necessary to tune thread detection behavior for the server:
* Stuck Thread Max Time: Enter the number of seconds, that a thread must be continually working before this server diagnoses the thread as being stuck. By default, WebLogic Server considers a thread to be "stuck" after 600 seconds of continuous use.
* Stuck Thread Timer Interval: Enter the number of seconds, after which WebLogic Server periodically scans threads to see if they have been continually working for the length of time specified by Stuck Thread Max Time. By default, WebLogic Server sets this interval to 600 seconds.
7. Click Apply to save your changes.
8. Reboot the server to use the new settings.
 
Ranch Hand
Posts: 110
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
I strongly agree with Marcos.

We need to take thread dumps at regular intervals and should analyse them.

You take the thread dumps at an interval of 1 or 2 minutes and take the help of samurai tool to analyse the thread dumps. Then you will be able to know what exactly happenning and possible reasons

1) DB locks
2) Native thread problems if any
3) Huge amount of data being pulled up
4) extended time in getting responses from interface systems

with all these try checking the Garbage collection also.
 
With a little knowledge, a cast iron skillet is non-stick and lasts a lifetime.
reply
    Bookmark Topic Watch Topic
  • New Topic