• Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other Pie Elite all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Jeanne Boyarsky
  • Ron McLeod
  • Paul Clapham
  • Liutauras Vilda
Sheriffs:
  • paul wheaton
  • Rob Spoor
  • Devaka Cooray
Saloon Keepers:
  • Stephan van Hulst
  • Tim Holloway
  • Carey Brown
  • Frits Walraven
  • Tim Moores
Bartenders:
  • Mikalai Zaikin

Weblogic nodes are failing down in production environment.

 
Greenhorn
Posts: 1
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Hi,

In production environment, the Weblogic nodes are failing down sometimes. But this is not happening frequently. I got this common exception whenever the Node fails. jcifs.http.NtlmHttpFilter.doFilter(NtlmHttpFilter.java:171)

I’m looking for your help. Please share your views.

Log:

####<01-Nov-2010 15:11:39 o'clock GMT> <Error> <WebLogicServer> <UKBLPKTLB01> <ManagedServer1> <[ACTIVE] ExecuteThread: '1' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <> <1288624299665> <BEA-000337> <[STUCK] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)' has been busy for "636" seconds working on the request "Http Request: /pkt_library/PKTPublishDocuments", which is more than the configured time (StuckThreadMaxTime) of "600" seconds. Stack trace:
Thread-14 "[STUCK] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)'" <alive, in native, suspended, priority=1, DAEMON> {
sun.nio.ch.SocketDispatcher.read0(SocketDispatcher.java:???)
sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:25)
sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:221)
sun.nio.ch.IOUtil.read0(IOUtil.java:197)
sun.nio.ch.IOUtil.read(IOUtil.java:???)
sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:135)
^-- Holding lock: java.lang.Object@583188[thin lock]
com.documentum.fc.impl.util.io.MessageChannel.readSocket(MessageChannel.java:121)
com.documentum.fc.impl.util.io.MessageChannel.readLength(MessageChannel.java:99)
com.documentum.fc.impl.util.io.MessageChannel.getIncomingMessageLength(MessageChannel.java:91)
com.documentum.fc.impl.util.io.MessageChannel.read(MessageChannel.java:77)
^-- Holding lock: java.lang.Object@583188[thin lock]
com.documentum.fc.client.impl.connection.netwise.AbstractNetwiseRpcClient.receiveMessage(AbstractNetwiseRpcClient.java:161)
com.documentum.fc.client.impl.connection.docbase.netwise.NetwiseDocbaseRpcClient.applyForCollection(NetwiseDocbaseRpcClient.java:396)
^-- Holding lock: java.lang.Object@583188[thin lock]
com.documentum.fc.client.impl.connection.docbase.DocbaseConnection$3.evaluate(DocbaseConnection.java:1129)
^-- Holding lock: java.lang.Object@583188[thin lock]
com.documentum.fc.client.impl.connection.docbase.DocbaseConnection.evaluateRpc(DocbaseConnection.java:976)
^-- Holding lock: java.lang.Object@583188[thin lock]
com.documentum.fc.client.impl.connection.docbase.DocbaseConnection.applyForCollection(DocbaseConnection.java:1123)
com.documentum.fc.client.impl.docbase.DocbaseApi.exec(DocbaseApi.java:64)
com.documentum.fc.client.impl.session.Session.query(Session.java:3242)
com.documentum.fc.client.impl.session.SessionHandle.query(SessionHandle.java:2216)
com.documentum.fc.client.impl.dql.DqlUpdate.processUpdate(DqlUpdate.java:439)
com.documentum.fc.client.DfQuery.runQuery(DfQuery.java:133)
com.documentum.fc.client.DfQuery.execute(DfQuery.java:200)
com.astrazeneca.documentum.pkt.servlets.PublishDocumentsServlet.runUpdateDQL(PublishDocumentsServlet.java:407)
com.astrazeneca.documentum.pkt.servlets.PublishDocumentsServlet.doGet(PublishDocumentsServlet.java:20)
javax.servlet.http.HttpServlet.service(HttpServlet.java:700)
javax.servlet.http.HttpServlet.service(HttpServlet.java:815)
weblogic.servlet.internal.StubSecurityHelper$ServletServiceAction.run(StubSecurityHelper.java:223)
weblogic.servlet.internal.StubSecurityHelper.invokeServlet(StubSecurityHelper.java:107)
weblogic.servlet.internal.ServletStubImpl.execute(ServletStubImpl.java:198)
weblogic.servlet.internal.TailFilter.doFilter(TailFilter.java:26)
weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:41)
jcifs.http.NtlmHttpFilter.doFilter(NtlmHttpFilter.java:171)
weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:41)
com.astrazeneca.pkt.servlet.filter.InterceptFilter.doFilter(InterceptFilter.java:42)
weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:41)
weblogic.servlet.internal.WebAppServletContext$ServletInvocationAction.run(WebAppServletContext.java:3365)
weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:308)
weblogic.security.service.SecurityManager.runAs(Unknown Source)
weblogic.servlet.internal.WebAppServletContext.securedExecute(WebAppServletContext.java:2076)
weblogic.servlet.internal.WebAppServletContext.execute(WebAppServletContext.java:1998)
weblogic.servlet.internal.ServletRequestImpl.run(ServletRequestImpl.java:1331)
weblogic.work.ExecuteThread.execute(ExecuteThread.java:197)
weblogic.work.ExecuteThread.run(ExecuteThread.java:164)
}

>

Thanks in advance,
Rajkumar
 
Bartender
Posts: 6663
5
MyEclipse IDE Firefox Browser Linux
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Welcome to JR rajkumar.

I assume you have some sort of documentum related class deployed on that server ? The thread that is handling the HTTP call is taking more than 10 minutes to return. Possible reasons are

1. Thread deadlock
2. Blocking calls
3. Infinite loops
etc

Can you post a thread dump ? WL 10 allows you to take a thread dump at any time when the server is running. Take the dump when you have some stuck threads
 
reply
    Bookmark Topic Watch Topic
  • New Topic