Help coderanch get a
new server
by contributing to the fundraiser
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other Pie Elite all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Jeanne Boyarsky
  • Ron McLeod
  • Paul Clapham
  • Liutauras Vilda
Sheriffs:
  • paul wheaton
  • Rob Spoor
  • Devaka Cooray
Saloon Keepers:
  • Stephan van Hulst
  • Tim Holloway
  • Carey Brown
  • Frits Walraven
  • Tim Moores
Bartenders:
  • Mikalai Zaikin

does a Tomcat jar contain the org,apache.xpath package?

 
Ranch Hand
Posts: 195
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Hello. Tomcat apps running which parse xml. I am modifying one of the classes and trying to find the jar with the org.apache.xpath package. I've gone through a lot of the lib jars in Tomcat with jar -tf  and looking at contents but can't find org.apache.xpath. It's kind of getting frustrating.

I then dumped every jar from the Tomcat and web app libs into the compile classpath and still receive the org.apache.xpath does not exist.

I thought from working on xml stuff in the past that it was some xalan.jar or xerces.jar or something but I don't see those in the Tomcat nor app libs. So I'm not sure how these apps are functioning in runtime...
 
Saloon Keeper
Posts: 27930
198
Android Eclipse IDE Tomcat Server Redhat Java Linux
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
It doesn't matter. You canot steal stuff from Tomcat like that. If you need it in a webapp, you need to include it as part of your webapp.

It's in Xalan: https://mvnrepository.com/artifact/xalan/xalan/2.7.3
 
Thomas Griffith
Ranch Hand
Posts: 195
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
yeah, I don't see xerces in the web app jars. This couldn't run without it, right? but they've been running for years.

Could it be the class I'm looking at might not be instantiated in runtime? It's reading a config file somewhere so I'd think it needs an XML parser. Maybe I'm tracing the execution path wrong.
 
Marshal
Posts: 28271
95
Eclipse IDE Firefox Browser MySQL Database
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
But Java already has an XML parser built in. Sure, there are other XML parsers that you can acquire and plug in, but you don't have to do that.

Do you know for sure that the previous custodians of the web app had plugged in a particular XML parser?
 
Tim Holloway
Saloon Keeper
Posts: 27930
198
Android Eclipse IDE Tomcat Server Redhat Java Linux
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Paul Clapham wrote:But Java already has an XML parser built in. Sure, there are other XML parsers that you can acquire and plug in, but you don't have to do that.

Do you know for sure that the previous custodians of the web app had plugged in a particular XML parser?


I'm uncertain about that and what releases of Java that might be true for (remember JAXB?) I don't think Xalan would be putting in something unless either it couldn't depend on the JVM or the JVM version was somehow deficient.

At any rate, xpath isn't a parser, it's a tree-search mechanism.
 
Thomas Griffith
Ranch Hand
Posts: 195
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
ok, I think I figured it out. The xerces stuff might be an old class sitting in the web app but I think the xml is actually "parsed" by instantiating an org.apache.commons.digester Digester (which is in commons-digester.jar)...
 
Tim Holloway
Saloon Keeper
Posts: 27930
198
Android Eclipse IDE Tomcat Server Redhat Java Linux
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Parsing in XML/Java is done by a SAX parser, whether it's being invoked by DOM loading or stream processing. If memory serves, even the STaX parser has a SAX engine underneath it.

That's also true of the Apache Digester, which uses SAX to convert XML text from places like the TOMCAT__HOME/conf xml files, the Context definitions (both external and in WARs), the web.xml files from WARs being deployed, and essentially any place where XML text needs to be used to construct a network of JavaBeans.

However, an app cannot invoke Tomcat's Digester. If a webapp wants to digest, it must supply its own digester JAR.

SAX is an Interface, and there are indeed multiple implementations, but products that parse XML often include the Sun one and have an option for using an alternate parser if you don't like the builtin one.
 
Paul Clapham
Marshal
Posts: 28271
95
Eclipse IDE Firefox Browser MySQL Database
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Tim Holloway wrote:I'm uncertain about that and what releases of Java that might be true for (remember JAXB?) I don't think Xalan would be putting in something unless either it couldn't depend on the JVM or the JVM version was somehow deficient.

It was Java 1.4 when Sun added DocumentBuilder and SAXParser and so on to Java. From the beginning the design was that there was a system property which controlled which implementation of those classes would be used to do the parsing. There was a system default which was used if you didn't set that property.

And it was Java 5 when XSLT processing was added. This had a similar plug-in architecture and that's when Xalan was written to be a better XSLT processor than Sun's. It was Xerces which was supposed to be the better XML processor than Sun's. But at any rate you never have had to go searching for jar files if you want to process XML (or XSLT) in Java.

 
Thomas Griffith
Ranch Hand
Posts: 195
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Tim Holloway wrote:That's also true of the Apache Digester, which uses SAX to convert XML text from places like the TOMCAT__HOME/conf xml files, the Context definitions (both external and in WARs), the web.xml files from WARs being deployed, and essentially any place where XML text needs to be used to construct a network of JavaBeans.

However, an app cannot invoke Tomcat's Digester. If a webapp wants to digest, it must supply its own digester JAR.



oh, I see, the org.apache.common.digester.* stuff doesn't come with Tomcat...the commons-digester.jar is separate and has to be added...then the Digester constructed with the JavaBeans...then have to add a getter/setter class for each xml name/value pair.

I have a question on datasource pooling, related to this same datasource. This config file stores the username/password but I built a SSO filter on the front end of the app which uses a token so the individual users are already authenticated and authorized with the datasource.  However, I read datasource pooling needs one single db username/password to work, since it's pooling connections and pulling them as needed. So would it screw the pooling up if I connect with the user rather than the username/password indicated in this config file?

     
 
Tim Holloway
Saloon Keeper
Posts: 27930
198
Android Eclipse IDE Tomcat Server Redhat Java Linux
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
The Datasource is common to all users of the webapp. Well, actually there can be more than one datasource, but Datasources are never "per-user". A multiple datasource app might have one pool connected to the corporate general database and another to a separate Oracle Financials database, as an example.

Because of that, the Datasource credentials have to allow the greatest common denominator of all users and functions of the application. Which is why when you keep webapp user credentials in a database, that you use a separate, more secure webapp to maintain those credentials.

Bear in mind that a Datasource doesn't create Connections on-demand. It creates a pool of identical and interchangeable Connections, which is why individual webapp user IDs cannot be used to get a personalized custom connection. To do that, you'd have to connect for each and every request from each and every user, and that's a lot more overhead than using a Connection Pool. But Connection Pools are safe enough, as long as the webapp itself is secure.
 
Thomas Griffith
Ranch Hand
Posts: 195
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
ok, that's what i was thinking. The connection pool created in the web app context is dumped (by the garbage collector?) after some time if not used, and re-instantiated by the next user. Is that right?

do you obfuscate .class files in wars/jars? i was looking around at that and see all these utilities but not sure what is trustworthy.
 
Tim Holloway
Saloon Keeper
Posts: 27930
198
Android Eclipse IDE Tomcat Server Redhat Java Linux
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
The connection pool is created when the webapp is deployed. At that time, a number of Connections are pre-created, with that number being the pool's minimum size.

When the app requests a Connection, the pool pulls a Connection off of its list of Connections and wraps it in an object that serves as a fa├žade Connection. It differs from the underlying Connection only in that its close() method simply returns the Connection back to the Connection pool instead of disconnecting from the database. Thus, that Connection can be re-used.

Note that it is ABSOLUTELY ESSENTIAL that you close a Connection as soon as you possibly can. Holding it keeps other apps from being able to use it. And NEVER, EVER store it in a Session or Application object for continued use between requests!!!

If all of the Connections in a pool have been opened by requesting webapps, then the pool will create one or more new Connection objects to expand the pool. It will do that until the configured maximum pool sizee is reached, at which time, further Connection requests will fail (see paragraph above!)

The Connection pool lives for the lifetime of the webapp. It may shrink itself down to its nominal size over time, depending on how it is configured, but it will never drop below the minimum pool size.

One "gotcha" is that when the underlying Connections don't get used often enough, they may time out and be disconnected by the database server. In which case you may need to configure the Connection Pool to test and refresh stale Connections so that the webapps can use them safely. The mechanism that sets that up may vary depending on the particular pool implementation and/or the database server.

And, no, I don't obfuscate my code. It's pointless. Back in olden days I used to disassemble software all the time and the lack of meaningful names made no difference to me. Back then it was legal, incidentally. And a lot easier than persuading Microsoft to make a 1-line change to one of their major program products.
 
Thomas Griffith
Ranch Hand
Posts: 195
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
yeah, I put a coupe of System.out.printlns in the constructor for the DatabaseSessionFactory class which extends BasePoolableObjectFactory.

They print when I first run the db operation, then don't prints as I repeat the operation (I was assuming it's pulling the active connection from the pool for all these). However, if I let it sit idle for some time, like ten minutes, and run the same operation, the constructor System.out.printlns print again. Is that expected pooling behavior?
 
Tim Holloway
Saloon Keeper
Posts: 27930
198
Android Eclipse IDE Tomcat Server Redhat Java Linux
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
The Database Connection Pool is not only a "black box", it is actually plug-replaceable. The Connection pooler used by default in modern versions of Tomcat not the same object as older Tomcats used. The only thing that really matters is that it should dispense pool Connections on demand. How it manages that internally is strictly its own business.

Incidentally, based on what you found, your pool is based on the Apache Commons Pool 2 library, which is a general-purpose pooling facility that can be used for pools of anything; Connections, Threads, InkPens, what have you. Tomcat itself has quite a few pools in it which may or may not be using the Apache Commons 2 pool to manage them. But actually, as in many other Apache libraries, you can use the Commons Pool 2 library in your own apps if you like. Just don't use it instead of Tomcat's Connections pool. Tomcat has a better handle on the total server environment. Plus its pooling is already built in.
 
Thomas Griffith
Ranch Hand
Posts: 195
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
ok, thanks so much. I think i have a follow-up on that regarding the data source access again but I'm going to look at that a little more first. I'm kinda messing around with the obfuscation stuff (just in case something comes down from sec auditors). I know you don't use it but have you messed around with proguard for Tomcat wars? I get such a mixed bag via goggle and having to deal with the external libraries, blah, blah. It introduces it's own (disproportionate) risk but now I'm trying to figure out a set pf procedures just in case.
 
Tim Holloway
Saloon Keeper
Posts: 27930
198
Android Eclipse IDE Tomcat Server Redhat Java Linux
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
No, I have never wasted my time on "copy protection" or obfuscators.

Locks are for honest people. All of the public work I've done, and for that matter, the commercially-sold work-for-hire I've done has been clear code. On the other hand, as I said, I've deconstructed multiple binary-only packages and worked with people who for legitimate business reasons did likewise including circumventing "dongle" protection. None of the auditors I've worked with have ever demanded "protected" code.

As far as I'm concerned, effort made on "protecting" code is not only futile, but additionally, it sucks valuable resources out of the process of making the code better and more reliable. The best protection I've seen is to offer a quality product at a reasonable price and as evidence I can offer my fat retirement account courtesy of having invested in Red Hat, which not only has (pre-IBM) had an absolutely rabid policy against getting entangled in encumbered code, but actually condoned other people re-packaging and distributiny it (CentOS, for example). Despite which, they made money hand-over-first and still lead the Linux industry. At least until IBM ruins them.

Incidentally, I have several notebooks filled with hand-disassembled code from IBM's DOS/VS mainframe OS. It's strictly vanity now since I long ago actually acquired the official machine-readable source courtesy of the Hercules mainframe emulator. Up until about 1986, IBM made their OS source publicly available.
 
Thomas Griffith
Ranch Hand
Posts: 195
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
yeah, with defense in depth/mitigation comes a trail of bread crumbs you need to leave behind in order for a  good guy to find it all, decode, etc later. The trail can get longer and longer for the attacker but longer and tenuous for good guy and any one link gets lost... I already coded a custom password "scrambler" algorithm for the keystore and context (server and app) data sources in my dev and was thinking to try to obfuscate those class files to "hide" the logic. But this proguard is so confusing (especially for selectively trying to do stuff like this) and obfuscation seems prone to introducing errors.

I thought this would be the easiest thing to do of all with proguard or whatever...
input whatever.war
output  whatever_obfuscated.war
deploy

But it gets nuts with library dependencies, etc. Like i said, this is more contingency, if confronted with having to do it, just trying to nail down a procedure.
 
Tim Holloway
Saloon Keeper
Posts: 27930
198
Android Eclipse IDE Tomcat Server Redhat Java Linux
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Rather demonstrates my point. Development and maintenance gets more expensive if you add extra steps and given a fixed time and budget, that means you have to steal from the actual product.

Don't, by the way, try adding custom or double-encryption to your security code. It will more likely than not actually reduce security, especially if you are not a professionally-trained full-time security professional. It's hard enough even for the pros to stay ahead in that game. I cannot begin to count the websites I've seen that could be cracked by relatively non-technical people in under 10 minutes because the local shop genius thought that he could invent a better security system than the one that came with the webapp server. We're talking banking/finance apps and even a military app in one case.
 
Thomas Griffith
Ranch Hand
Posts: 195
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
yeah, for the password "scrambling" thing, it's just some algorithm stuff to take in the jumbled password stored in the config files and convert to regular password for keystore, connections, etc. I don't think I'd put this in regular production given what you've posted which is in line with what I'm finding but it's helping with grasping Tomcat operations and, even if finding out it's riskier and cumbersome, grasping the why.

I tried one obfuscation for kicks with proguard. I took the custom MyHttp11NioProtocol.jar, stored in Tomcat lib with the package/class referenced in server.xml...



When I obfuscated MyHttp11NioProtocol.jar with proguard, replace in Tomcat lib, Tomcat startup bombs with NullPointer at addSslHostConfig.

When i put back un-obfuscated MyHttp11NioProtocol.jar, it works again. I guess the package and/or class names gets all obfuscated as well and I need to set something to preserve package/class names in the proguard-rules.pro file. It's past onerous... I hate to walk away without knowing I could get at least one jar to work.
 
Saloon Keeper
Posts: 15656
367
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Why are you trying to hide stuff from security auditors? By definition, this means you're trying to hide vulnerabilities.

Faffing about with obfuscators is making your build more complex, is eating into your development time, and is making your application less maintainable.

As mentioned in your other thread, trying to encrypt your keystore passwords doesn't actually offer you any added security. Plain text is fine. Trust the OS to keep your files secure.

Isn't the point of a security audit that the security professionals will tell you what the right solution is when they find something wrong?
 
Tim Holloway
Saloon Keeper
Posts: 27930
198
Android Eclipse IDE Tomcat Server Redhat Java Linux
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Stephan van Hulst wrote:
Isn't the point of a security audit that the security professionals will tell you what the right solution is when they find something wrong?



Probably not. But at least with luck they'll tell you what is wrong.

I say "with luck", because I've seen what auditors have let pass.
 
Thomas Griffith
Ranch Hand
Posts: 195
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Tim Holloway wrote:

Stephan van Hulst wrote:
Isn't the point of a security audit that the security professionals will tell you what the right solution is when they find something wrong?



Probably not. But at least with luck they'll tell you what is wrong.

I say "with luck", because I've seen what auditors have let pass.



yeah, all I get is some spit-out of a drive(s) scan vs CVE and a "you got a lot of problems".
 
I've never won anything before. Not even a tiny ad:
We need your help - Coderanch server fundraiser
https://coderanch.com/t/782867/Coderanch-server-fundraiser
reply
    Bookmark Topic Watch Topic
  • New Topic