Concurrent Request Processing in jBoss - Not working

[ishan dave]

Hi All,

I am facing a wired problem in jBoss AS. We deployed a RESTfull web service application developed using Java and Jersey and deployed the war in jBoss AS Version 6.0

I call this web service from different J2ME clients. But my jBoss serves one client at a time

for example if I call the same web service from 3 different mobiles then the jBoss serves one after another. We are writting the logs where we write the name of device and it shows that jBoss is not handling the requests concurrently instead it serves one after another.

I am very new to jBoss so I am sure that its my mistake and I am missing some configuration or something to achieve concurrency. Please help me to solve this.

My server.xml is as follows.

Server>

   <!-- Optional listener which ensures correct init and shutdown of APR,
        and provides information if it is not installed -->
   <Listener className="org.apache.catalina.core.AprLifecycleListener" SSLEngine="on" />
   <!--Initialize Jasper prior to webapps are loaded. Documentation at /docs/jasper-howto.html -->
   <Listener className="org.apache.catalina.core.JasperListener" />

   <Service name="jboss.web">

      <!-- A HTTP/1.1 Connector on port 8080 -->
      
      <Connector protocol="HTTP/1.1" port="${jboss.web.http.port}" address="${jboss.bind.address}" 
         redirectPort="${jboss.web.https.port}"
         maxThreads="250" maxHttpHeaderSize="8192"
     		 emptySessionPath="true" enableLookups="false" redirectPort="8443" acceptCount="100"
     		 connectionTimeout="20000" disableUploadTimeout="true"
          />
      <!--
      <Connector protocol="HTTP/1.1" port="8181" address="${jboss.bind.address}" 
         redirectPort="${jboss.web.https.port}" />
			-->
      <!-- Add this option to the connector to avoid problems with 
          .NET clients that don't implement HTTP/1.1 correctly 
         restrictedUserAgents="^.*MS Web Services Client Protocol 1.1.4322.*$"
      -->

      <!-- A AJP 1.3 Connector on port 8009 -->
      <Connector protocol="AJP/1.3" port="${jboss.web.ajp.port}" address="${jboss.bind.address}"
         redirectPort="${jboss.web.https.port}" maxThreads="200" emptySessionPath="true"/>
         

      <!-- SSL/TLS Connector configuration using the admin devl guide keystore
      <Connector protocol="HTTP/1.1" SSLEnabled="true" 
           port="${jboss.web.https.port}" address="${jboss.bind.address}"
           scheme="https" secure="true" clientAuth="false" 
           keystoreFile="${jboss.server.home.dir}/conf/chap8.keystore"
           keystorePass="rmi+ssl" sslProtocol = "TLS" />
      -->

      <Engine name="jboss.web" defaultHost="localhost">

         <!-- The JAAS based authentication and authorization realm implementation
         that is compatible with the jboss 3.2.x realm implementation.
         - certificatePrincipal : the class name of the
         org.jboss.security.auth.certs.CertificatePrincipal impl
         used for mapping X509[] cert chains to a Princpal.
         - allRolesMode : how to handle an auth-constraint with a role-name=*,
         one of strict, authOnly, strictAuthOnly
           + strict = Use the strict servlet spec interpretation which requires
           that the user have one of the web-app/security-role/role-name
           + authOnly = Allow any authenticated user
           + strictAuthOnly = Allow any authenticated user only if there are no
           web-app/security-roles
         -->
         <Realm className="org.jboss.web.tomcat.security.JBossWebRealm"
            certificatePrincipal="org.jboss.security.auth.certs.SubjectDNMapping"
            allRolesMode="authOnly"
            />
         <!-- A subclass of JBossSecurityMgrRealm that uses the authentication
         behavior of JBossSecurityMgrRealm, but overrides the authorization
         checks to use JACC permissions with the current java.security.Policy
         to determine authorized access.
         - allRolesMode : how to handle an auth-constraint with a role-name=*,
         one of strict, authOnly, strictAuthOnly
           + strict = Use the strict servlet spec interpretation which requires
           that the user have one of the web-app/security-role/role-name
           + authOnly = Allow any authenticated user
           + strictAuthOnly = Allow any authenticated user only if there are no
           web-app/security-roles
         <Realm className="org.jboss.web.tomcat.security.JaccAuthorizationRealm"
            certificatePrincipal="org.jboss.security.auth.certs.SubjectDNMapping"
            allRolesMode="authOnly"
            />
         -->

         <Host name="localhost"> 

            <!-- Uncomment to enable request dumper. This Valve "logs interesting 
                 contents from the specified Request (before processing) and the 
                 corresponding Response (after processing). It is especially useful 
                 in debugging problems related to headers and cookies."
            -->
            <!--
            <Valve className="org.apache.catalina.valves.RequestDumperValve" />
            -->
 
            <!-- Access logger -->
            <!--
            <Valve className="org.apache.catalina.valves.AccessLogValve"
                prefix="localhost_access_log." suffix=".log"
                pattern="common" directory="${jboss.server.log.dir}" 
                resolveHosts="false" />
            -->

            <!-- Uncomment to enable single sign-on across web apps
                deployed to this host. Does not provide SSO across a cluster.     
            
                If this valve is used, do not use the JBoss ClusteredSingleSignOn 
                valve shown below.
                
                A new configuration attribute is available beginning with
                release 4.0.4:
                
                cookieDomain  configures the domain to which the SSO cookie
                              will be scoped (i.e. the set of hosts to
                              which the cookie will be presented).  By default
                              the cookie is scoped to "/", meaning the host
                              that presented it.  Set cookieDomain to a
                              wider domain (e.g. "xyz.com") to allow an SSO
                              to span more than one hostname.
             -->
            <!--
            <Valve className="org.apache.catalina.authenticator.SingleSignOn" />
            -->

            <!-- Uncomment to enable single sign-on across web apps
               deployed to this host AND to all other hosts in the cluster.
            
               If this valve is used, do not use the standard Tomcat SingleSignOn
               valve shown above.
            
               Valve uses Infinispan to support SSO credential 
               caching and replication across the cluster.  
               Infinispan must be configured separately.  
               See the "/infinispan-configs.xml" file in the 
               server/all/deploy/cluster/infinispan-cache-registry.sar 
               directory for cache configuration details.
            
               Besides the attributes supported by the standard Tomcat
               SingleSignOn valve (see the Tomcat docs), this version also 
               supports the following attributes:
            
               cookieDomain   see non-clustered valve above
               
               cacheConfig    Name of the CacheManager service configuration
                              to use for the clustered SSO cache. See
                              deploy/cluster/jboss-cache-manager.sar/META-INF/jboss-cache-manager-jboss-beans.xml
                              Default is "clustered-sso".
               
               treeCacheName  Deprecated. Use "cacheConfig". 
                              JMX ObjectName of the JBoss Cache MBean used to 
                              support credential caching and replication across
                              the cluster. Only used if no cache can be located
                              from the CacheManager service using the "cacheConfig"
                              attribute (or its default value). If not set, the 
                              default is "jboss.cache:service=TomcatClusteringCache"
                              
               maxEmptyLife   The maximum number of seconds an SSO with no 
                              active sessions will be usable by a request
                              
               processExpiresInterval The minimum number of seconds between 
                              efforts by the valve to find and invalidate 
                              SSO's that have exceeded their 'maxEmptyLife'. 
                              Does not imply effort will be spent on such
                      			cleanup every 'processExpiresInterval'.
            -->
            <!--
            <Valve className="org.jboss.web.tomcat.service.sso.ClusteredSingleSignOn" />
            -->
         
            <!-- Check for unclosed connections and transaction terminated checks
                 in servlets/jsps.
                 
                 Important: The dependency on the CachedConnectionManager
                 in META-INF/jboss-service.xml must be uncommented, too
           -->

            <Valve className="org.jboss.web.tomcat.service.jca.CachedConnectionValve"
                cachedConnectionManagerObjectName="jboss.jca:service=CachedConnectionManager"
                transactionManagerObjectName="jboss:service=TransactionManager" />
                
         </Host>

      </Engine>

   </Service>

</Server>

and datasource.xml is

<?xml version="1.0" encoding="UTF-8"?>

<!-- ===================================================================== -->
<!--                                                                       -->
<!--  JBoss Server Configuration                                           -->
<!--                                                                       -->
<!-- ===================================================================== -->

<!-- $Id: jll-oracle-ds.xml,v 1.1 2011/05/05 12:58:58 vipulv Exp $ -->
<!-- ==================================================================== -->
<!--  Datasource config for Oracle originally from Steven Coy             -->
<!-- ==================================================================== -->


<datasources>
  <local-tx-datasource>
    <jndi-name>jdbc/myoracle_ds</jndi-name>
    <connection-url>jdbc:oracle:thin:@myip:1521:mf</connection-url>
	<!--
		Here are a couple of the possible OCI configurations.
		For more information, see http://otn.oracle.com/docs/products/oracle9i/doc_library/release2/java.920/a96654/toc.htm

	<connection-url>jdbc:oracle:oci:@youroracle-tns-name</connection-url>
		or
	<connection-url>jdbc:oracle:oci:@(description=(address=(host=youroraclehost)(protocol=tcp)(port=1521))(connect_data=(SERVICE_NAME=yourservicename)))</connection-url>

		Clearly, its better to have TNS set up properly.
	 -->
    <driver-class>oracle.jdbc.driver.OracleDriver</driver-class>
    <user-name>username</user-name>
    <password>password</password>
    <!-- Uses the pingDatabase method to check a connection is still valid before handing it out from the pool -->
    <!--valid-connection-checker-class-name>org.jboss.resource.adapter.jdbc.vendor.OracleValidConnectionChecker</valid-connection-checker-class-name-->
    <!-- Checks the Oracle error codes and messages for fatal errors -->
    <exception-sorter-class-name>org.jboss.resource.adapter.jdbc.vendor.OracleExceptionSorter</exception-sorter-class-name>
    <min-pool-size>75</min-pool-size>
  	<max-pool-size>100</max-pool-size>
        <!-- sql to call when connection is created
        <new-connection-sql>some arbitrary sql</new-connection-sql>
        -->

        <!-- sql to call on an existing pooled connection when it is obtained from pool - the OracleValidConnectionChecker is prefered
        <check-valid-connection-sql>some arbitrary sql</check-valid-connection-sql>
        -->

      <!-- corresponding type-mapping in the standardjbosscmp-jdbc.xml (optional) -->
      <metadata>
         <type-mapping>Oracle9i</type-mapping>
      </metadata>
  </local-tx-datasource>

</datasources>

Please guide me to solve this issue.

Thank you very much in advance.

With best regards,
Ishan

[Peter Johnson]

You have not limited the number of http connections, and 75 connections for the database is more than enough. So the problem has to be either in your code (locking, overuse of synchronized) or with Jersey.

[ishan dave]

Peter Johnson wrote:You have not limited the number of http connections, and 75 connections for the database is more than enough. So the problem has to be either in your code (locking, overuse of synchronized) or with Jersey.

Hey Peter Johnson,

Thank you very much for your valuable reply. I check with my code there is no synchronizing stuffs so will you please tell what should I check or look with Jersey ?

Thank you,

With best regards,
Ishan

[Peter Johnson]

Sorry, but I don't know Jersey, you are on your own there.

You might try taking a thread dump when there are two requests in process - you should see two threads processing the requests. Look at the code shown in the thread call stacks, that should give you a clue. (Depending on how quickly your code processes the requests you might have to take multiple thread dumps before you catch the culprit.)

[ishan dave]

Peter Johnson wrote:Sorry, but I don't know Jersey, you are on your own there.

You might try taking a thread dump when there are two requests in process - you should see two threads processing the requests. Look at the code shown in the thread call stacks, that should give you a clue. (Depending on how quickly your code processes the requests you might have to take multiple thread dumps before you catch the culprit.)

Thank you Peter for your reply.

In fact yesterday I downloaded jMeter and after generating concurrent requests from it everything seems right. Than checked my client code and the problem is there. In one thread I forgot to raise the exception in stead I catch it and this is the place where my application hangs.

Finally solved the issue.

In fact your reply opens a door for me to think on the lines other than jBoss and able to solve the issue.

Thank you very much for your valuable inputs.

With best regards,
Ishan