• Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other Pie Elite all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Jeanne Boyarsky
  • Ron McLeod
  • Paul Clapham
  • Liutauras Vilda
Sheriffs:
  • paul wheaton
  • Rob Spoor
  • Devaka Cooray
Saloon Keepers:
  • Stephan van Hulst
  • Tim Holloway
  • Carey Brown
  • Frits Walraven
  • Tim Moores
Bartenders:
  • Mikalai Zaikin

Tomcat 8 Migration, DB2 DataSource issue

 
Greenhorn
Posts: 4
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Hey, I am migrating an application from Tomcat7 to Tomcat8 and Java 1.8. However, on startup when it tries to initialize the datasource, I get an error about some pdq.jar. It seems to be something related to pureQuery, but that isn't something we've used with Tomcat7, or ever in the past.


Is there some different driver class name that should be used now, or some other class name I now need to specify to not use pureQuery?
Also, here is my context.xml snippet:

<?xml version="1.0" encoding="UTF-8"?>
<Context >
<!-- Default set of monitored resources -->
<WatchedResource>WEB-INF/web.xml</WatchedResource>
<Resource maxIdle="1" connectionProperties="enableSysplexWLB=true;maxRetriesForClientReroute=15;retryIntervalForClientReroute=10;maxTransportObjects=10;" validationQuery="SELECT COUNT(*) FROM SYSIBM.SYSDUMMY1" driverClassName="com.ibm.db2.jcc.DB2Driver" type="javax.sql.DataSource" password="*******" url="jdbc:db2://hostname:port/DB2" maxActive="2" username="userName" name="jndiName" minIdle="0" maxWait="5000" initialSize="1" auth="Container" />


15-Jun-2016 15:19:44.141 INFO [localhost-startStop-1] org.apache.catalina.startup.HostConfig.deployWAR Deploying web application archive /opt/tomcat-instances/appName/webapps/ROOT.war
15-Jun-2016 15:19:44.255 WARNING [localhost-startStop-1] org.apache.tomcat.util.scan.StandardJarScanner.scan Failed to scan [file:/opt/tomcat-instances/appName/lib/pdq.jar] from classloader hierarchy
java.io.FileNotFoundException: /opt/tomcat-instances/appName/lib/pdq.jar (No such file or directory)
at java.util.zip.ZipFile.open(Native Method)
at java.util.zip.ZipFile.<init>(ZipFile.java:219)
at java.util.zip.ZipFile.<init>(ZipFile.java:149)
at java.util.jar.JarFile.<init>(JarFile.java:166)
at java.util.jar.JarFile.<init>(JarFile.java:130)
at org.apache.tomcat.util.scan.JarFileUrlJar.<init>(JarFileUrlJar.java:60)
at org.apache.tomcat.util.scan.JarFactory.newInstance(JarFactory.java:43)
at org.apache.tomcat.util.scan.StandardJarScanner.process(StandardJarScanner.java:323)
at org.apache.tomcat.util.scan.StandardJarScanner.scan(StandardJarScanner.java:273)
at org.apache.catalina.startup.ContextConfig.processJarsForWebFragments(ContextConfig.java:1887)
at org.apache.catalina.startup.ContextConfig.webConfig(ContextConfig.java:1127)
at org.apache.catalina.startup.ContextConfig.configureStart(ContextConfig.java:779)
at org.apache.catalina.startup.ContextConfig.lifecycleEvent(ContextConfig.java:306)
at org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(LifecycleSupport.java:95)
at org.apache.catalina.util.LifecycleBase.fireLifecycleEvent(LifecycleBase.java:90)
at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5171)
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:147)
at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:725)
at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:701)
at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:717)
at org.apache.catalina.startup.HostConfig.deployWAR(HostConfig.java:940)
at org.apache.catalina.startup.HostConfig$DeployWar.run(HostConfig.java:1816)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)

 
Saloon Keeper
Posts: 27752
196
Android Eclipse IDE Tomcat Server Redhat Java Linux
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Oooh. That's a sticky one, I think.

Tomcat8 implements Servlet spec 3.0, which supports "web fragments". Which are basically snippets of web.xml that can be sectioned off and packaged in JARs in order to make component jars self-configuring.

Based on my reading of the stacktrace apparently one of the jars in the project has a web fragment that includes a "jar:" URL, something more or less like "jar://pdq.jar" as a dependency in it and it would appear that your WAR doesn't include a WEB-INF/lib/pdq.jar file.

So not really directly to the datasource, if at all.

Unfortunately, the trace doesn't indicate which jar in the webapp has the offending reference in it, so some dissection and debugging is in order.
 
Greenhorn
Posts: 6
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
I see this same error with db2 driver version 4.19.26 and Tomcat 8.0.36. In this case, my datasource is the same as what I used in Tomcat 7.
 
Tim Holloway
Saloon Keeper
Posts: 27752
196
Android Eclipse IDE Tomcat Server Redhat Java Linux
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
You aren't putting the DB2 driver in the webapp's WEB-INF/lib directory, I hope?

OK. I just realized that instead of one tomcat containing multiple webapps, your (Chris) setup apparently dedicates multiple Tomcat instances, this one containing a single webapp operating as the root context. Which made the Tomcat instance appear at first glance to be a webapp instance.

I just checked my own DB2 driver jar and confirmed that there are no web fragments in it.

The stack trace, however seems to indicate that something within the ROOT.war webapp is referencing /opt/tomcat-instances/appName/lib/pdq.jar. Which at best is bad practice, since the TOMCAT_HOME/lib directory is not a blessed location for a WAR's resources (WARs should be self-contained, not reaching out for external class resources).

I recommend doing a "grep -l --recursive pdq /opt/tomcat-instances/appName/*" and see if you can locate the file making this reference.
 
Chris Cerda
Greenhorn
Posts: 4
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
So it does seem unrelated to the datasource, I just thought it was since pdq.jar is used for PureQuery within DB2. The server starts and the application named ROOT.war works w/o issue, so at this point I guess it's just a cosmetic issue that it's throwing that stack trace, but it's definitely something I want to clean up and not be accepted as a "normal error".

It also does SEEM to be somehow related to ROOT.war, b/c I built a new tomcat instance with no applications in it, and it does not have the issue.

However, when doing a search for pdq in everything under ROOT.war, the only thing it matched on was an image and the binary coding of it.

It also matched on

find . -name "*" -type f -exec grep -il pdq {} \;
./ROOT/WEB-INF/lib/xml-apis-1.4.01.jar
./ROOT/WEB-INF/lib/xercesimpl-2.10.0.jar
./ROOT.war


However, when I extracted those two files, it didn't match on anything. So my thinking is it just matched on 'pdq' in the binary coding of the jar files.

our setup is that we have the tomcat binaries at /opt/apache-tomcat-8...., with a sym link that links to that directory of /opt/tomcat8. Same with java, i have /opt/java1.8.... with a sym link of /opt/java8 that points to java1.8.....

Then I have a directory /opt/tomcat-instances/<tomcat instance name>. Within there we have a config.properties which sets CATALINA_HOME, CATALINA_BASE, JAVA_HOME, that way we can easily move something between versions and we can create multiple tomcat instances from the same set of binaries and the same java.

I'm pretty stumped at this point. Is there any additional logging I could add to logging.properties, to maybe get more of a trace on this issue to see why it's happening?
 
Tim Holloway
Saloon Keeper
Posts: 27752
196
Android Eclipse IDE Tomcat Server Redhat Java Linux
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Hmmm. I prefer to keep my Tomcats and my webapps separate. I would typically have a /usr/local/apache-tomcat-8.0.35 as a CATALINA_HOME and my WARs would be files something like /opt/com/mousetech/mywebapp/mywebapp.war. The Tomcat instances (CATALINA_BASE) would be something like /var/lib/tomcat8_01, since those files need to be read/write and by convention /usr/local is supposed to be read-only. And if I'm in a really ambitious mood, I'd alias the logs and stuff to show up at /var/log.

Anyway, unzip the ROOT.war and grep through it. It may point you to a JAR, which you would then unzip and grep, etc. etc.

There is only limited logging in this part of Tomcat and in Tomcat 8, the logger names are abstracted, but you might try going against org.apache.tomcat.tomcat.util and its descendents.
 
Rich Mike
Greenhorn
Posts: 6
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
I removed all my webapps and started tomcat and I still see the WARNING. If I remove the driver jar from tomcat/lib, the error goes away. So it seems to be related to the driver.
 
Tim Holloway
Saloon Keeper
Posts: 27752
196
Android Eclipse IDE Tomcat Server Redhat Java Linux
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Don't count Tomcat as clean until you've purge the work, temp and logs directories.

I'd still look at your ROOT.war file, though. The DB2 driver I'm using doesn't come back with a hit when I grep it for pdq.
 
Rich Mike
Greenhorn
Posts: 6
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Thanks Tim, in this case I confirmed the error depends on the jdbc driver being the lib directory. If I remove the jdbc driver the error goes away. If I roll back to driver version 4.16.53 (fp1), the error does not occur, so it appears it is something that happens in later versions of the DB2 JDBC driver that Tomcat 8 does not like.
 
Tim Holloway
Saloon Keeper
Posts: 27752
196
Android Eclipse IDE Tomcat Server Redhat Java Linux
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
OK. Can you give me the version of the JDBC driver you're using? And maybe the download URL? I haven't updated in a while and I'd like to see what it's trying to do here.

As I said originally, I think that the offending mechanism depends on new servlet features that Tomcat 7 predates, which is why Tomcat 8 triggered them. They presumably are there to do something allegedly useful, assuming that IBM didn't outsource the job to some malware-infested place. Not that a big-name company would ever do something like that...
 
Rich Mike
Greenhorn
Posts: 6
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Hi,

If you go to http://www-01.ibm.com/support/docview.wss?uid=swg21363866 and download 4.19.26 you should see the error.
 
Tim Holloway
Saloon Keeper
Posts: 27752
196
Android Eclipse IDE Tomcat Server Redhat Java Linux
  • Likes 1
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Thanks.


Manifest-Version: 1.0
Ant-Version: Apache Ant 1.9.3
Created-By: 2.4 (IBM Corporation)
Version: 3.69.24
Class-Path: pdq.jar



"PDQ" means, I think "Pure Data Query" and here is some information that might provide some enlightenment: https://www.ibm.com/support/knowledgecenter/was_beta/com.ibm.websphere.base.doc/ae/tejb_configpdqjpase.html

Many of the classes in the DB2 driver jars use it and there's even a PDQCallback registry.

I'd check with IBM, because you may need to obtain the pdq jar to achieve proper functionality, or at least optimal functionality.
 
Rich Mike
Greenhorn
Posts: 6
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Thanks Tim. It turns out pdq.jar and pdqmgmt.jar come with the IBM Data Studio. I dropped them in tomcat/lib and the error message goes away.
 
Chris Cerda
Greenhorn
Posts: 4
  • Likes 1
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Rich Mike wrote:Thanks Tim. It turns out pdq.jar and pdqmgmt.jar come with the IBM Data Studio. I dropped them in tomcat/lib and the error message goes away.



Yeah this fixed mine as well. Just added pdq.jar and pdqmgmt.jar to the lib directory, and now the error goes away. Turns out in the MANIFEST.MF of db2jcc.jar, it is requiring pdq.jar. Very odd, but hey, at least we got it figured out :-D.

Thanks
 
reply
    Bookmark Topic Watch Topic
  • New Topic