Resolving Rollback, Deadlock exception

[Deepan Ignatius]

I have a piece of servlet code which pulls data from DB and generates a excel file and this runs for more than a hour. During the process below exceptions are thrown and from there the execution also stops intermediately.

Error Message is: Deadlock detected, timing out call after 90 seconds wait for Thread[AJPRequestHandler-HTTPThreadGroup-5,5,HTTPThreadGroup]
Stack Trace is : com.evermind.server.DeadlockException: Deadlock detected, timing out call after 90 seconds wait for Thread[AJPRequestHandler-HTTPThreadGroup-5,5,HTTPThreadGroup] at com.evermind.server.ejb.EntityEJBObject.startCall(EntityEJBObject.java:1013) at Icoms_EntityBeanWrapper0.findDealersList(Icoms_EntityBeanWrapper0.java:39616) at com.tcs.icoms.web.java.ControlServlet.doPost(ControlServlet.java:663) at javax.servlet.http.HttpServlet.service(HttpServlet.java:763) at javax.servlet.http.HttpServlet.service(HttpServlet.java:856) at com.evermind.server.http.ResourceFilterChain.doFilter(ResourceFilterChain.java:64) at com.tcs.icoms.web.filter.StatusCheckerFilter.doFilter(StatusCheckerFilter.java:178) at com.evermind.server.http.ServletRequestDispatcher.invoke(ServletRequestDispatcher.java:621) at com.evermind.server.http.ServletRequestDispatcher.forwardInternal(ServletRequestDispatcher.java:368) at com.evermind.server.http.HttpRequestHandler.doProcessRequest(HttpRequestHandler.java:866) at com.evermind.server.http.HttpRequestHandler.processRequest(HttpRequestHandler.java:448) at com.evermind.server.http.AJPRequestHandler.run(AJPRequestHandler.java:302) at com.evermind.server.http.AJPRequestHandler.run(AJPRequestHandler.java:190) at oracle.oc4j.network.ServerSocketReadHandler$SafeRunnable.run(ServerSocketReadHandler.java:260) at oracle.oc4j.network.ServerSocketAcceptHandler.procClientSocket(ServerSocketAcceptHandler.java:239) at oracle.oc4j.network.ServerSocketAcceptHandler.access$700(ServerSocketAcceptHandler.java:34) at oracle.oc4j.network.ServerSocketAcceptHandler$AcceptHandlerHorse.run(ServerSocketAcceptHandler.java:880) at com.evermind.util.ReleasableResourcePooledExecutor$MyWorker.run(ReleasableResourcePooledExecutor.java:303) at java.lang.Thread.run(Thread.java:595)

Error Message is: An exception occurred during transaction completion: ; nested exception is: javax.transaction.RollbackException: Timed out
Stack Trace is : oracle.oc4j.rmi.OracleRemoteException: An exception occurred during transaction completion: at com.evermind.server.ejb.EJBTransactionManager.end(EJBTransactionManager.java:141) at com.evermind.server.ejb.AbstractEJBObject.__completeGlobalTransaction(AbstractEJBObject.java:231) at Icoms_EntityBeanWrapper0.generateCommissionExcelReport(Icoms_EntityBeanWrapper0.java:37436) at com.tcs.icoms.web.java.ControlServlet.doPost(ControlServlet.java:672) at javax.servlet.http.HttpServlet.service(HttpServlet.java:763) at javax.servlet.http.HttpServlet.service(HttpServlet.java:856) at com.evermind.server.http.ResourceFilterChain.doFilter(ResourceFilterChain.java:64) at com.tcs.icoms.web.filter.StatusCheckerFilter.doFilter(StatusCheckerFilter.java:178) at com.evermind.server.http.ServletRequestDispatcher.invoke(ServletRequestDispatcher.java:621) at com.evermind.server.http.ServletRequestDispatcher.forwardInternal(ServletRequestDispatcher.java:368) at com.evermind.server.http.HttpRequestHandler.doProcessRequest(HttpRequestHandler.java:866) at com.evermind.server.http.HttpRequestHandler.processRequest(HttpRequestHandler.java:448) at com.evermind.server.http.AJPRequestHandler.run(AJPRequestHandler.java:302) at com.evermind.server.http.AJPRequestHandler.run(AJPRequestHandler.java:190) at oracle.oc4j.network.ServerSocketReadHandler$SafeRunnable.run(ServerSocketReadHandler.java:260) at com.evermind.util.ReleasableResourcePooledExecutor$MyWorker.run(ReleasableResourcePooledExecutor.java:303) at java.lang.Thread.run(Thread.java:595) Nested exception is: javax.transaction.RollbackException: Timed out at com.evermind.server.ApplicationServerTransaction.checkForRollbackOnlyWhileInCommit(ApplicationServerTransaction.java:582) at com.evermind.server.ApplicationServerTransaction.doCommit(ApplicationServerTransaction.java:244) at com.evermind.server.ApplicationServerTransaction.commit(ApplicationServerTransaction.java:130) at com.evermind.server.ApplicationServerTransactionManager.commit(ApplicationServerTransactionManager.java:433) at com.evermind.server.ejb.EJBTransactionManager.end(EJBTransactionManager.java:132) at com.evermind.server.ejb.AbstractEJBObject.__completeGlobalTransaction(AbstractEJBObject.java:231) at Icoms_EntityBeanWrapper0.generateCommissionExcelReport(Icoms_EntityBeanWrapper0.java:37436) at com.tcs.icoms.web.java.ControlServlet.doPost(ControlServlet.java:672) at javax.servlet.http.HttpServlet.service(HttpServlet.java:763) at javax.servlet.http.HttpServlet.service(HttpServlet.java:856) at com.evermind.server.http.ResourceFilterChain.doFilter(ResourceFilterChain.java:64) at com.tcs.icoms.web.filter.StatusCheckerFilter.doFilter(StatusCheckerFilter.java:178) at com.evermind.server.http.ServletRequestDispatcher.invoke(ServletRequestDispatcher.java:621) at com.evermind.server.http.ServletRequestDispatcher.forwardInternal(ServletRequestDispatcher.java:368) at com.evermind.server.http.HttpRequestHandler.doProcessRequest(HttpRequestHandler.java:866) at com.evermind.server.http.HttpRequestHandler.processRequest(HttpRequestHandler.java:448) at com.evermind.server.http.AJPRequestHandler.run(AJPRequestHandler.java:302) at com.evermind.server.http.AJPRequestHandler.run(AJPRequestHandler.java:190) at oracle.oc4j.network.ServerSocketReadHandler$SafeRunnable.run(ServerSocketReadHandler.java:260) at com.evermind.util.ReleasableResourcePooledExecutor$MyWorker.run(ReleasableResourcePooledExecutor.java:303) at java.lang.Thread.run(Thread.java:595)

I am running it on oc4j 10.3.1.0 application server. The same piece of code when triggered as a standalone program works as expected without throwing these exceptions.

What could be the root cause for this problem? Should I make this syncronous transaction to be asynchronous?

Regards.

[Ashwin Sridhar]

Hi,

From logs, it seems to be transaction time out. Try to increase your transaction max time. you need to configure it on the application server.

[Deepan Ignatius]

I have increased the transaction time out to 7200 but Deadlock is thorwn.

<transaction-manager xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="http://xmlns.oracle.com/oracleas/schema/transaction-manager-10_0.xsd" transaction-timeout="7200" max-concurrent-transactions="-1" >

[Ashwin Sridhar]

Deadlock occurs when you try to update or insert or select a table simultaneously in separate Threads. Probably you have many Threads running and trying to lock the table.

[Kumar Raja]

Servlet thread running for more than an hr for generating the excel. Have you checked how much time does it usually take to generate Excel with out any web framework. How many requests does this servlet get concurrently? Does each of those threads take about 1 hr to generate the excel.

Is having a web framework an absolute requirement for the generation of an excel ? Can you make this asynchronous and send the results back, only if the task is completed. I heard servlet 3.0 has asynchronous feature. Even if you are not using 3.0, I think still you should be able to make this asynchronous background task by some other means, instead of keeping your servlet thread handling 1 request for an hr.

I think, first of all, before you employ servlets, try tuning your logic and see if the actual component generating excel is taking that long time and if yes, how can you minimize it.
secondly, as I said, you may have to look at running these process in background, so that you don't keep servlet thread waiting for 1 hr to send the response back.


Thirdly, you can also think about asynchronous webservice call.