• Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other Pie Elite all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Tim Cooke
  • Campbell Ritchie
  • Paul Clapham
  • Ron McLeod
  • Liutauras Vilda
Sheriffs:
  • Jeanne Boyarsky
  • Rob Spoor
  • Bear Bibeault
Saloon Keepers:
  • Jesse Silverman
  • Tim Moores
  • Stephan van Hulst
  • Tim Holloway
  • Carey Brown
Bartenders:
  • Piet Souris
  • Al Hobbs
  • salvin francis

"stuck thread" issues in weblogic 9.2

 
Ranch Hand
Posts: 36
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Hello all,
We are having some trouble in production, where we have deployed our application on wl9.2. We have been encountering "stuck thread" issues and are hoping you can help us resolve this issue.

Basically we have configured the application server to use WL 8.1 thread pooling mechanism, where were can define user defined execute queues. Such thread pooling was replaced in WL9.2 to use work managers. We have set the default_ queue size at 50. During peak hours, all 50 threads on the Defualt execute queue are occupied. Processes running on these threads are taking a little while longer to execute causing the system response to slow down. The server logs are full of stuck threads.

I believe there is a fundamental configuration issue we are missing here or there is potentially a pacth for wl9.2. But i am just not able to place my finger on it. Could somebody please help me out with that.

Appreciate it.

Here are some snippets of the exceptions.

<blockquote>code:
<pre name="code" class="core"> <Jul 16, 2008 1:37:00 PM EDT> <Error> <WebLogicServer> <BEA-000337> <ExecuteThread: '34' for queue: 'default' has been busy for "671" seconds working on the request "weblogic.work.ExecuteRequestAdapter@50931a", which is more than the configured time (StuckThreadMaxTime) of "600" seconds. Stack trace:
java.lang.ClassLoader.loadClass(ClassLoader.java:295)
java.lang.ClassLoader.loadClass(ClassLoader.java:299)
java.lang.ClassLoader.loadClass(ClassLoader.java:251)
weblogic.utils.classloaders.GenericClassLoader.loadClass(GenericClassLoader.java:158)
weblogic.utils.classloaders.ChangeAwareClassLoader.loadClass(ChangeAwareClassLoader.java:35)
javax.xml.parsers.FactoryFinder.newInstance(FactoryFinder.java:88)
javax.xml.parsers.FactoryFinder.findJarServiceProvider(FactoryFinder.java:278)
</pre>
</blockquote>

<blockquote>code:
<pre name="code" class="core"> <Jul 16, 2008 1:40:24 PM EDT> <Error> <WebLogicServer> <BEA-000337> <ExecuteThread: '44' for queue: 'default' has been busy for "686" seconds working on the request "weblogic.work.ExecuteRequestAdapter@2e4333", which is more than the configured time (StuckThreadMaxTime) of "600" seconds. Stack trace:
weblogic.common.resourcepool.ResourcePoolImpl.reserveResourceInternal(ResourcePoolImpl.java:322)
weblogic.common.resourcepool.ResourcePoolImpl.reserveResource(ResourcePoolImpl.java:294)
weblogic.common.resourcepool.ResourcePoolImpl.reserveResource(ResourcePoolImpl.java:284)
weblogic.jdbc.common.internal.ConnectionPool.reserve(ConnectionPool.java:466)
weblogic.jdbc.common.internal.ConnectionPool.reserve(ConnectionPool.java:357)
weblogic.jdbc.common.internal.ConnectionPoolManager.reserve(ConnectionPoolManager.java:83)
</pre>
</blockquote>

<blockquote>code:
<pre name="code" class="core"> "ExecuteThread: '37' for queue: 'default'" daemon prio=10 tid=02f17860 nid=412 lwp_id=4622133 waiting for monitor entry [2d372000..2d36f578]
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:168)
at org.hibernate.util.ReflectHelper.classForName(ReflectHelper.java:108)
at org.hibernate.impl.SessionFactoryImpl.getImportedClassName(SessionFactoryImpl.java:722)
at org.hibernate.hql.classic.QueryTranslatorImpl.getEntityPersisterUsingImports(QueryTranslatorImpl.java:352)
at org.hibernate.hql.classic.WhereParser.doToken(WhereParser.java:389)
at org.hibernate.hql.classic.WhereParser.token(WhereParser.java:263)
at org.hibernate.hql.classic.ClauseParser.token(ClauseParser.java:86)
</pre>
</blockquote>
 
Ranch Hand
Posts: 977
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Hi,

you have 2 log messages where you have threads 34 and 44 doing their jobs for more than 600 secs, the default configuret time for weblogic. Then you post a thread dump from thread 37? What you need to do is take some thread dumps 4 or 5 in an interval of 10 to 15 seconds each. Than from the log message you take a look on a stuck thread, for example the 34 from your log message and take a look at it's thread dump. See if you have same method holding lock or waiting for something.. Find the cause of it. Usually some db access, or slow network resource or system.
 
Greenhorn
Posts: 19
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Take 5-6 thread dumps and analyze the same in samurai tool so that you will exactly see which thread is actually blocking all the other threads and analyze the thread for what it is doing.
 
You showed up just in time for the waffles! And this tiny ad:
Building a Better World in your Backyard by Paul Wheaton and Shawn Klassen-Koop
https://coderanch.com/wiki/718759/books/Building-World-Backyard-Paul-Wheaton
reply
    Bookmark Topic Watch Topic
  • New Topic