This week's book giveaway is in the Cloud forum.
We're giving away four copies of Terraform in Action and have Scott Winkler on-line!
See this thread for details.
Win a copy of Terraform in Action this week in the Cloud forum!

Reidar Gjerstad

Ranch Hand
+ Follow
since Dec 02, 2008
Cows and Likes
Cows
Total received
0
In last 30 days
0
Total given
0
Likes
Total received
0
Received in last 30 days
0
Total given
0
Given in last 30 days
0
Forums and Threads
Scavenger Hunt
expand Ranch Hand Scavenger Hunt
expand Greenhorn Scavenger Hunt

Recent posts by Reidar Gjerstad

Your last post brought me a little forward, but did not resolve the issue. I had of course forgotten to define the security Realm in server.xml (embarrassing) and it was in a different form:



I modified to use your suggested form



Unfortunately, it did not solve the problem. Exception still comes up.
2 months ago
Sorry for quoting too much. Will cut down.

I will try the things you suggest, one by one.

In addition to Tomcat's default test app, there is only one app on this server (local PC).

Is there a chance that Tomcat 9 simply won't work with the MySQL version that I have locally?



It appears that Tomcat 6 and Tomcat 9 have exactly the same instructions for which versions will work:

"Versions of MySQL and JDBC drivers that have been reported to work:

   MySQL 3.23.47, MySQL 3.23.47 using InnoDB,, MySQL 3.23.58, MySQL 4.0.1alpha
   Connector/J 3.0.11-stable (the official JDBC Driver)
   mm.mysql 2.0.14 (an old 3rd party JDBC Driver)"

BR
Reidar Gjerstad
2 months ago

Tim Holloway wrote:In XML, unsupported attributes are ignored, so don't expect that to help.

I cross-check the JRE source code. It's pretty definite. Something was attempting to parse a URL and an escape sequence was found in the URL. For example, the "%20" in "https://coderanch.com/messages/my%20message".

In your case, however, it looks like the URL contained "%$Z" and since "$Z" is obviously not a hexadecimal number code, it threw an internal NumberFormatException.

If you can't see anything like that anywhere in your failing Tomcat files, then you may have hardware problems or a corrupt VM.

One way to find out for sure would be to launch Tomcat as a remote debugging session and set a breakpoint on com.mysql.cj.exceptions.WrongArgumentException. I'd actually recommend catching the NumberFormatException itself, but unfortunately, there are jokers out there who deliberately toss around common exception types in their product code and you might get inundated with them.

Also, if you could post the Realm XML it would help, since that's where the Exception actually displays.



You mentioned that I may have a corrupt VM. I installed another complete JDK, but that did not make any difference:



BR
Reidar Gjerstad

2 months ago

Tim Holloway wrote:Uh-oh.

I think you took a wrong turning.

First, let me start by explaining deployment descriptors. Deployment descriptors are the meta-information used to properly deploy webapps. In JEE, each webapp has two: the server-dependent deployment descriptor and the server-independent deployment descriptor.

The server-independent deployment descriptor is the webapp's /WEB-INF/web.xml file, plus whatever dynamic equivalents might have been scooped up from annotations in the webapp's classes.

The server-dependent deployment descriptor is the Context.

You don't actually have to have an explicit Context. If you don't provide one, Tomcat will synthesize one internally. But an explicit Context allows you to define JDBC Connection Pools, JNDI Resources, security Realms, and the webapp URL context path itself. Also it permits you to direct the application's codeBase to an alternate location. Such as the target build directory in my Eclipse project or to a WAR file in /opt/com/mousetech/mywebaps/mywebapp.war

The Context can be located as a file named "META-INF/context.xml" in a WAR file. It can also be located as an XXXXX.xml file in /conf/Catalina/localhost, where the URL context path will be the same as the Context filename (minus the extension). This overrides the Context in the WAR, if there is one and that's handy for keeping a development context in the WAR and overriding it for production.

You can also put a Context as an XML element node in /conf/server.xml, but you should not. It's mostly a relic from old times.

And then there's conf/context.xml. You might notice that there's also a conf/web.xml file. These are the prototypes to which Tomcat will fall back when it cannot find what it needs in your actual deployment descriptors and in general they should not be modified (you'll note that context.xml does allow you to change it to disable the SER file option, though).

Also, the Context is a tomcat-specific (server-dependent) deployment descriptor. Any resemblance to deployment descriptors used by other JEE servers such as Wildfly and WebSphere are purely co-incidental. And in fact, typically different webapp servers use different filenames and/or locations so that they don't conflict. Finally, Context may be generated by the webapp server if you deploy a WAR via the webapp's administrative console GUI/webapp.


The web.xml file is more interesting, since when you submit a URL that doesn't map to a servlet, it's where Tomcat goes looking for guidance. The prototype web.xml takes unresolved URL paths that can map to WAR resource directories and produces a directory listing webpage. For paths that resolve to static resources such as javascript, css or image files, it locates and copies the static resource to the response stream. And if it can't find anything else to resolve with, it emits the "404 Not Found" page.

So, in short most people should only modify the conf/server.xml and leave the other files located in the conf directory alone (Exception noted below).



=====

Anyway on the grep[/search, don't forget that special symbols such as "%" and "$" are often magical unless properly escaped and thus might not match as expected. You might be better off scanning for "Realm", in fact.

Incidentally, the 2 Realm definitions in the default server.xml are the UserDatabaseRealm, which looks up userid/password and userid/role in the conf/tomcat-users.xml file (which incidentally, you can and often would edit), and the LockoutRealm that says that if the Realm(s) it's presiding over reject too many login attempts, that loginid will be locked out for a specified time.



Hi Tim.

Thank you for the thorough explanation. I passed the SCWCD exam some time around 2010. I remember some details, but in addition to being rusty my understanding does not go as deep as yours.

I ran the grep over again. It does find the "$Z" in the logs. Doesn't it mean that it would find it if it were present in any xml or jsp files...?



It also finds "$Z" if I paste it into a test html file:



In addition there are binary files (JARs, PDFs) that contain the "$Z".

This makes me puzzled:
-I have written this web app myself.
-I can not remember having put any URL into any file with that kind of "$Z" content
-All URLs that exist are in JSPs and refer to other JSPs or servlet mount patterns (.do links)
-There is only one exception to the above, a simple JSP test page that tests the database connection. It doesn't contain the "$Z" sequence, either.
-The URL for the database connection is specified in conf/context.xml. I can't remember having any other URL for the database.

Am I searching the wrong place for the wrong thing? The offending "$Z" must be in a text file, is it so?

BR
Reidar Gjerstad




2 months ago
Downloaded and installed another jdk, but it made no difference. The version was

2 months ago
My Java version:

2 months ago

Tim Holloway wrote:Oh, and by the way - why did you modify conf/context.xml?



It is for historical reasons. I got it working like that a looong time ago following instructions similar to this:

https://tomcat.apache.org/tomcat-9.0-doc/jndi-datasource-examples-howto.html

Then I never thought about experimenting and finding other ways to do it. Perhaps I should have :-)

BR
Reidar Gjerstad
2 months ago

Tim Holloway wrote:"Realm" is an XML node element that defines a security realm for webapps in Tomcat. It can be applied to a single webapp - in which case, it's normally part of conf/server.xml or - more commonly, it's defined in a Context for a single webapp. Your error message implies that there is a JDBC Realm defined somewhere in that Tomcat system and that it contains an invalid JDBC URL.

There's probably at least one sample Realm definition example (commented out) in server.xml. The Tomcat pre-installed management webapps are secured by a MemoryRealm or something like that.



Thanks again.

Recursive grep for '%$Z': It doesn't exist anywhere in tomcat-dir or my project's dir.

Recursive grep for '$Z': in tomcat-dir




Recursive grep for '$Z': in my project's dir:



There is a Realm defined in server.xml, but the invalid URL is nowhere to be seen:




Since there's no '%$Z' in my files, can I assume the problem is somewhere else?

BR
Reidar Gjerstad
2 months ago

Tim Holloway wrote:
Also, if you could post the Realm XML it would help, since that's where the Exception actually displays.



Tim, thank you for the reply. Apologies for the stupid question, but which "Realm XML" are you referring to? I made a recursive search for xml files, and I can't see any *.xml starting with "Realm" or similar.



The only additional xml is my web app's web.xml.
apache-tomcat-9.0.43/conf/context.xml is the only file I've modified.

Thanks
Reidar Gjerstad
2 months ago

Tim Holloway wrote:Welcome back!

Best I can tell is that you may have a non-printing or otherwise invalid character in your JDBC connection URL.

Since you blanked out details, I can't tell any more than that, however.



Thank you for the reply!

The blanked out details contain only printable characters.  And the same URL is working under Tomcat 6.0. I could try removing the

I don't remember when it was added.

BR
Reidar Gjerstad
2 months ago
Hello, this is my first post for ~10 years :-)

I would be thankful if somebody could give me some hints on how to solve following problem:

For a long while I've stuck to Apache Tomcat/6.0.53 on my local development computer because it just works. I've started to experiment with new features which require a higher Java version and a newer Tomcat.

Under Apache Tomcat/6.0.53 I've developed and become familiar with MySQL database connections. Everything works well under Tomcat 6.0.53:

-Using Java 1.6.0_45
-Placing mysql-connector-java-5.1.6-bin.jar under $CATALINA_HOME/lib and things have always worked well.

Then I tried to upgrade:
-Installed and switched to Java 1.8.0_281.
-Installed Tomcat 9.0.43 and verified that it starts, can serve JSP pages etc.
-Downloaded and copied "mysql-connector-java-8.0.11.jar" to $CATALINA_HOME/lib according to instructions valid at the time:

https://tomcat.apache.org/tomcat-9.0-doc/jndi-datasource-examples-howto.html

Added to my context.xml:



In localhost.YYYY-MM-DD.log I get the stack trace below. Obviously, there is no database connection available.

Does anybody have hints on how to solve this problem and which JDBC Driver actually works?

BR
Reidar Gjerstad

2 months ago

Ankit Garg wrote:Congrats Reidar

-Unless you are a real expert, expect to fail at least once. Don't give up.


I would disagree here, the exam is not that tough to clear in one go. If one has prepared properly, then the exam can be cleared on the first attempt...



Hi Ankit

With expert I mean a real software professional, someone doing software for a living, perhaps having studied computer science. I design electronics for a living. This is just a hobby I have. Thus I believe it should be easier to pass the exam for a real expert.

Cheers
11 years ago
Finally, on my second attempt I passed on 18 June 2010 with 78%. OK, it's not a good score, but PASS anyhow. Because I had to take care of my job and family it took nine months to gather the motivation to start preparing for my second attempt.

Here are my tips for passing:

-Unless you are a real expert, expect to fail at least once. Don't give up.

-If you fail, use the score report from Sun to find your weak areas. Go straight on to study those areas. It is very uncomfortable at first, but soon you will improve on the weak areas.

-Don't be intimidated if you fail and read about all people passing with 90-100%. Most people who fail keep quiet about it in this forum.

-I bought Enthuware exam simulator. It turned out to be very useful. The questions are quite similar in tone and level of difficulty as the real exam. Before my second attempt I used the simulator to focus on my weak areas, then taking the standard tests. Enthuware contains some errors, though.

-Make your own notes. I made a 30 page document in Notepad.

-Take the bus to work. When you get bored of looking out the window, review the notes.

-Write some real code. I spent almost one year writing a real application.

-Cut some wood! When you get tired of JSPs and cryptic custom tags there is nothing like swinging the axe and a chainsaw.

-Get enough sleep. Don't take too much stress out of it. Work hard, but not so hard that it starts keeping you awake at nights.

-Be nice to your wife/girlfriend/mistress. You will need her goodwill for spending all that time with your studies.

Cheers
11 years ago
OK. Thanks for clarifying.

I understand that one scope is enough since you can do everything necessary with it.

So I guess that the spec developers could equally well have decided to specify a Response scope in stead of the Request scope and giving Response scope all the setters and getters that Request has today, but that they decided that Response would be more natural to use.

It makes sense.

Cheers
Fellow ranchers

There are no stupid questions..., but perhaps this one is.

In the servlet world there are three scopes: Context, Request and Session.

-Why isn't there a scope called Response?

-Wouldn't it be useful to be able to set and remove attributes in the Response as well?

After all both the request and response objects are included when forwarding to another servlet or a JSP:



Cheers