• Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other Pie Elite all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Jeanne Boyarsky
  • Ron McLeod
  • Paul Clapham
  • Liutauras Vilda
Sheriffs:
  • paul wheaton
  • Rob Spoor
  • Devaka Cooray
Saloon Keepers:
  • Stephan van Hulst
  • Tim Holloway
  • Carey Brown
  • Frits Walraven
  • Tim Moores
Bartenders:
  • Mikalai Zaikin

Server stopped responding for almost 10 minutes and resumed to normal state.

 
Greenhorn
Posts: 3
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Hello,

We have faced an issue with production JBoss server. Server stopped responding for almost 10 minutes and returned back to normal state after it. During this interval clients were not able to connect and no error or warning messages were logged to log files. However we have an known and open bug in the deployed application. One connection is not closed and Resultset and Statement is also open on this connection, could this be cause of this problem ? Same application is deployed on other three JBoss server but none of these server had this issue, though waring "Closing a connection for you" is being logged frequently on all the four servers. Datasource is configured in mssql-ds.xml as non transactional DS. How JBoss Handles open resources or its Database Driver's call to handle the open resources? Here are the Environment and server details.

Environment Details :
Os : Linux 2.6.18-194.17.4.el5 (i386) CPU : 1

JVM Name : Java HotSpot(TM) Server VM

JVM version : 1.5.0_03-b07 (Sun Microsystems Inc.)

Max Memory : 512 MB

Min Memory : 128 MB

Database : SQL Server 2008

Driver being used : sqljdbc 2

JBoss Version: 4.0.4GA(build: CVSTag=JBoss_4_0_4_GA date=200605151000)

Version Name: Zion

I find this a very strange problem as we did not get any errors during the freeze.I hope there is someone who can help us.

Thanks in advance.


 
author
Posts: 5856
7
Android Eclipse IDE Ubuntu
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Sounds like a huge garbage collection pause. Are you by any chance running the CMS collector? There are ways of determining if GC is the issue, but not after the fact. For suggestions on how to gather GC data and analyze it, see these white papers:
Java Garbage Collection Statistical Analysis 101
Java Garbage Collection Performance Analysis 201
at http://www.cmg.org/cgi-bin/search.cgi?q=java+peter+johnson&x=30&y=10


 
Uttam Chauhan
Greenhorn
Posts: 3
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Hi Peter,

Thanks a lot for you reply. We got right direction to work.

Could you please also provide some help on these queries.

1 I monitored logs there was continuous logging in server.log during the interval for which client was not able to connect, so looks like JBoss was working in this duration, So only client was not able to connect for 10 minutes, In what situation this could happen ?

2 How are the resources left opened like Resultset, Statements and Connection are handled by JBoss if they are configured in *-ds.xml in JBoss.(What if developer forgot to close them). ?



 
Peter Johnson
author
Posts: 5856
7
Android Eclipse IDE Ubuntu
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Usually if a request doesn't close a datasource connection (or actually, release it back into the connection pool), the app server will print a warning in the log file and release the connection for you. This is, of course, if you are using a data source (have a *-ds.xml) and the code looks up the data source in JNDI.

One way that you could end up not connecting for many minutes is if all of the threads in the thread pool are active, in which case other request will have to wait until one of the threads finishes an becomes free.

What kind of requests are coming in? HTTP to servlets/JSPs/JSF? EJB requests? MDB requests?
 
Uttam Chauhan
Greenhorn
Posts: 3
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Hi Peter,

Yes exactly same warning we are getting for unclosed connection, But I am more interested to know about open Statements and open Resultsets on a pooled connection (configured in *-ds.xml and being retrieved from JNDI).

I think all the threads being active is most promising cause of server not responding as we have multiple long database conversations for a single request (each request spends more than 500 millis in database conversations). So If there are simultaneous long running requests than thread pool will be fully consumed. Am i right ?

Request coming in are HTTP request to a servlet, a stateless session bean works at middle layer.

Once again thank you.

 
reply
    Bookmark Topic Watch Topic
  • New Topic