• Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other Pie Elite all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Paul Clapham
  • Ron McLeod
  • Jeanne Boyarsky
  • Tim Cooke
Sheriffs:
  • Liutauras Vilda
  • paul wheaton
  • Henry Wong
Saloon Keepers:
  • Tim Moores
  • Tim Holloway
  • Stephan van Hulst
  • Carey Brown
  • Frits Walraven
Bartenders:
  • Piet Souris
  • Himai Minh

Tomcat JAAS Authentication works every three restarts

 
Greenhorn
Posts: 20
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
I am on a long journey, trying to get JAAS to do in a Tomcat, what I want it to do. Went through a lot of hoops and issues, because of this antiquated technology.

The current episode of this journey is a weird one.

When I am deploying the JAAS implementation to the Tomcat on Windows correctly, it only works 1 out of 3 Tomcat restarts. Reproducible and confirmed.

I restart the Tomcat once. I receive the following exception.



I restart the Tomcat again. Same exception.

I restart the Tomcat a third time. This time it works.



When debugging the Tomcat behaviour, I noticed that the configuration file never seems to be loaded. But it is loaded, else it would never work, not even the third time in each cycle.

https://github.com/apache/tomcat/blob/9.0.14/java/org/apache/catalina/realm/JAASRealm.java#L608-L611
This is the code part where the configuration is "loaded" the first time.

https://github.com/apache/tomcat/blob/9.0.14/java/org/apache/catalina/realm/JAASRealm.java#L604-L606
Then, on subsequent authentication requests, it just returns the `null` configuration.

So, I thought, this is the behaviour when it's not working the first two times. As, without the JAAS configuration, it wouldn't know which `LoginModule` implementation to load. Right? Wrong.

The third time, the same behaviour was observable. Yet, my `LoginModule` Implementation was loaded correctly, the code was executed correctly and authentication succeeded.
I have no idea how that is supposed to work. Why does it work the third time or why does it not work the first two times, if the config file never seems to be loaded?

To be absolutely clear: if the configuration file were never loaded (or would be incorrect), then my `LoginModule` implementation would never load.

Does anyone have any hints regarding this topic?

Some additional information:



  • My `LoginModule` implementation is confirmed to be loaded via the debugger. This is not a guess, but a fact. The JAAS configuration must therefore be correct and loaded.
  • The path to the JAAS configuration file is correct. See point 1.
  • If something were configured the wrong way or if it were a user error, then it wouldn't work, at all. See point 1.
  • The custom `LoginModule` implementation and deployment works locally in a very simple Tomcat setup on its own. But it does not work in a real world production scenario, as described in this post.
  • There is no Servlet involved. The `LoginModule` implementation is loaded globablly as another JAR for the Tomcat to use. I.e. the Tomcat itself calls the `login` method from `LoginContext.java`, etc. This is out of my hands. Tomcat has to handle that correctly.
  •  
    Saloon Keeper
    Posts: 25476
    180
    Android Eclipse IDE Tomcat Server Redhat Java Linux
    • Likes 1
    • Mark post as helpful
    • send pies
      Number of slices to send:
      Optional 'thank-you' note:
    • Quote
    • Report post to moderator
    Since your previous thread mentioned that you were debugging Tomcat itself, I need to make sure you're not doing things in an overly-complicated way.

    Tomcat implements JEE-standard container managed security using Realm security modules. Normally You would be using the JAASRealm, as documented here: https://tomcat.apache.org/tomcat-9.0-doc/realm-howto.html#JAASRealm

    Realms provide 2 primary services to Tomcat.

    1) They provide an authenticate() method where user credentials are passed in and the response is a binary pass/fail indicating that the login process may proceed.

    2) They construct a UserPrincipal object where the user's session security information is anchored.

    Additionally, they do things like check whether a logged-in user has been assigned a security role (called by isUserInRole())

    The authentication mechanism was originally only invoked when a request was made to a secured URL as defined by a pattern in the webapp's /WEBiINF/web.xml file. The authenticator (login) would be automatically invoked when the client made the request and no application logic is involved. Furthermore, you couldn't force a login by sending the client to the loginform, as the loginform only works properly when the container (Tomcat) makes an internal request. External (client) requests for the login page do not have proper context.

    In more recent times an explicit login() method was added to the JEE API so that one could also force a login attempt via application logic.

    In either event, the credentials gathered from the method call or login form or dialog are passed straight to the configured Realm's authenticate method. It's a simple and reliable process, so there's no need to trace through Tomcat itself. If you distrust the Realm, put a breakpoint on its authenticate method.

    The one thing I did notice that was tricky for JAASRealm was that it required the JAASLoginConfigFile to be set as a JVM parameter rather than as a Realm attribute like most Realms would. I'm presuming that this is because JAAS is operating at the JVM level rather than at the Tomcat or web application levels and that the JAASLoginConfileFile needs to be known before Tomcat constructs the JAASRealm itself.

    The Tomcat docs aren't clear, but one way to set environment options for Tomcat is to create a TOMCAT_HOME/bin/setenv.sh (or for Windows setenv.bat) file, which is an executable script that gets invoked by the catalina script that controls Tomcat. You can put the "JAVA_OPTS=$JAVA_OPTS -Djava.security.auth.login.config=$CATALINA_BASE/conf/jaas.config" environment variable assignment statement in that file. As shipped, Tomcat doesn't have a setenv, so you may need to create it.

    So this is what you need to know in order to do basic setup for Tomcat JAAS properly. If you've got all of this correct, then we'll probably have to look at the details of your Realm configuration and any log messages that Tomcat has produced. Note that not all log messages go to catalina.log so you may need to check the other Tomcat logs as well.
     
    Akito Kitsune
    Greenhorn
    Posts: 20
    • Mark post as helpful
    • send pies
      Number of slices to send:
      Optional 'thank-you' note:
    • Quote
    • Report post to moderator

    Tim Holloway wrote:

    The one thing I did notice that was tricky for JAASRealm was that it required the JAASLoginConfigFile to be set as a JVM parameter rather than as a Realm attribute like most Realms would. I'm presuming that this is because JAAS is operating at the JVM level rather than at the Tomcat or web application levels and that the JAASLoginConfileFile needs to be known before Tomcat constructs the JAASRealm itself.

    The Tomcat docs aren't clear, but one way to set environment options for Tomcat is to create a TOMCAT_HOME/bin/setenv.sh (or for Windows setenv.bat) file, which is an executable script that gets invoked by the catalina script that controls Tomcat. You can put the "JAVA_OPTS=$JAVA_OPTS -Djava.security.auth.login.config=$CATALINA_BASE/conf/jaas.config" environment variable assignment statement in that file. As shipped, Tomcat doesn't have a setenv, so you may need to create it.



    I indeed forgot to mention, that I provided the path to the JAAS configuration file as a JVM argument. If the path were wrong or the configuration incorrectly configured, then the authentication would never succeed, as my `LoginModule` implementation would never be called, which is the only way to authenticate the user with the provided username & password combination.

    Actually, the JAAS setup itself is nothing too special. The difference is that there is no servlet, which may invoke the `LoginModule` implementation manually, i.e. it's not in my hands to decide in what way the `login` method is invoked.

    Tim Holloway wrote:
    So this is what you need to know in order to do basic setup for Tomcat JAAS properly. If you've got all of this correct, then we'll probably have to look at the details of your Realm configuration and any log messages that Tomcat has produced. Note that not all log messages go to catalina.log so you may need to check the other Tomcat logs as well.



    I have seen a lot of logs and a lot of messages. It's a bit confusing. The only notable error I found was the exception that is thrown the first two runs, when the authentication does not succeed. Otherwise, the log is very empty aside from useless error messages like the one explained in the following link.

    https://stackoverflow.com/questions/46367851/what-causes-filenotfoundexception-pdq-jar-with-db2jcc4

    The biggest problem I have with the situation is that it does not work twice but then it works the third time. Every single cycle, this is the case. If it wouldn't work at all, I could find an issue and solve it. If it would work, I would be done. But if it only works sometimes, I felt like I had to check the sanity of Tomcat itself, leading to me indirectly debugging it, as well.
     
    Tim Holloway
    Saloon Keeper
    Posts: 25476
    180
    Android Eclipse IDE Tomcat Server Redhat Java Linux
    • Mark post as helpful
    • send pies
      Number of slices to send:
      Optional 'thank-you' note:
    • Quote
    • Report post to moderator
    No, you should never attempt to manually invoke JAAS login when using the JAASRealm. The Realm would do that as needed.

    The StackOverflow reference is for a completely different environment, so I don't think it's going to be much help.

    For diagnostics, I would set a breakpoint on the JAASRealm's authenticate() method and check the arguments being passed to the LoginContext constructor. If you can trace into the LoginContext constructor you might be able to confirm whether or not you have the correct config file path. Note that in Java, spaces and proper capitalization in file paths are critical regardless of whether the OS is.

    Also, in case you missed it, the security modules you define must setup Tomcat's security objects:

    Tomcat docs wrote:
    It is the responsibility of your login module to create and save User and Role objects representing Principals for the user (javax.security.auth.Subject). If your login module doesn't create a user object but also doesn't throw a login exception, then the Tomcat CMA will break and you will be left at the http://localhost:8080/myapp/j_security_check URI or at some other unspecified location.



    Finally, since you're adding code at the Tomcat application level, you do need to ensure that that code is in Tomcat's own CLASSPATH. Probably the best way to do that is to create a JAR and put the JAR with your code into the TOMCAT_HOME/lib directory. DON'T try to set it up as a CLASSPATH environment setting before running the Tomcat startup/shutdown script.
     
    Akito Kitsune
    Greenhorn
    Posts: 20
    • Mark post as helpful
    • send pies
      Number of slices to send:
      Optional 'thank-you' note:
    • Quote
    • Report post to moderator

    Tim Holloway wrote:No, you should never attempt to manually invoke JAAS login when using the JAASRealm. The Realm would do that as needed.

    The StackOverflow reference is for a completely different environment, so I don't think it's going to be much help.

    For diagnostics, I would set a breakpoint on the JAASRealm's authenticate() method and check the arguments being passed to the LoginContext constructor. If you can trace into the LoginContext constructor you might be able to confirm whether or not you have the correct config file path. Note that in Java, spaces and proper capitalization in file paths are critical regardless of whether the OS is.

    Also, in case you missed it, the security modules you define must setup Tomcat's security objects:

    Tomcat docs wrote:
    It is the responsibility of your login module to create and save User and Role objects representing Principals for the user (javax.security.auth.Subject). If your login module doesn't create a user object but also doesn't throw a login exception, then the Tomcat CMA will break and you will be left at the http://localhost:8080/myapp/j_security_check URI or at some other unspecified location.



    Finally, since you're adding code at the Tomcat application level, you do need to ensure that that code is in Tomcat's own CLASSPATH. Probably the best way to do that is to create a JAR and put the JAR with your code into the TOMCAT_HOME/lib directory. DON'T try to set it up as a CLASSPATH environment setting before running the Tomcat startup/shutdown script.



    The StackOverflow reference was just an example displaying that the other errors shown in the Catalina log are useless, as they are insignificant.

    The JAR is within the `lib` directory, which is why the custom `LoginModule` implementation is loaded, in the first place.

    The Users and Roles are created and applied, as needed.

    The JAAS authentication setup is correct and working. It's just not properly applied by the real world production Tomcat and this is what I am trying to wrap my head around.

    I will try to check what arguments are being passed the `LoginContext` constructor, to verify the correct path is provided.
     
    Akito Kitsune
    Greenhorn
    Posts: 20
    • Mark post as helpful
    • send pies
      Number of slices to send:
      Optional 'thank-you' note:
    • Quote
    • Report post to moderator

    Akito Kitsune wrote:

    I will try to check what arguments are being passed the `LoginContext` constructor, to verify the correct path is provided.



    I checked and noticed, this is what I initially had shown, at the beginning of this thread. The `config` variable is always `null`.
     
    Tim Holloway
    Saloon Keeper
    Posts: 25476
    180
    Android Eclipse IDE Tomcat Server Redhat Java Linux
    • Mark post as helpful
    • send pies
      Number of slices to send:
      Optional 'thank-you' note:
    • Quote
    • Report post to moderator
    OK. Confirm that your Realm config is correct. Your original stack trace says:

    I hate this kind of message. It doesn't make it clear that the expected appName value is "example".
     
    Akito Kitsune
    Greenhorn
    Posts: 20
    • Mark post as helpful
    • send pies
      Number of slices to send:
      Optional 'thank-you' note:
    • Quote
    • Report post to moderator

    Tim Holloway wrote:OK. Confirm that your Realm config is correct. Your original stack trace says:

    I hate this kind of message. It doesn't make it clear that the expected appName value is "example".



    If you are referring to the `appName` of the Realm definition inside the Tomcat's `context.xml` being the same as the configured entry's name in the JAAS configuration file, then yes, they match.
     
    Tim Holloway
    Saloon Keeper
    Posts: 25476
    180
    Android Eclipse IDE Tomcat Server Redhat Java Linux
    • Mark post as helpful
    • send pies
      Number of slices to send:
      Optional 'thank-you' note:
    • Quote
    • Report post to moderator
    Match or no match, JAAS expects that you have a definition for "example" in your JAAS configuration file and the error message says that it does not.

    That could be because your config file is improperly formatted. It should look something like this:

    Again, this is all case-sensitive!
     
    Akito Kitsune
    Greenhorn
    Posts: 20
    • Mark post as helpful
    • send pies
      Number of slices to send:
      Optional 'thank-you' note:
    • Quote
    • Report post to moderator

    Tim Holloway wrote:Match or no match, JAAS expects that you have a definition for "example" in your JAAS configuration file and the error message says that it does not.

    That could be because your config file is improperly formatted. It should look something like this:

    Again, this is all case-sensitive!



    I changed it from something like this:



    To something like this (your example):



    Still, no change in behaviour.

    I again tried to provide the path to the configuration file in multiple ways, using forward slashes, instead of backslashes, etc. None of that changed the behaviour, either.

    The `appName` is the same across configurations, including case sensitivity.

    The Tomcat code is supposed to work right, but even if the authentication worked each third time, the `configFile` variable was still set to `null`. This stinks.

    I don't even know, where I would be able to see the actual path to the configuration file in the debugger. I didn't see a single place, where the actual path is visible in the debugger through a variable of some sort.

    Through all these tests, I also noticed that sometimes access to the Tomcat instance becomes really slow. For example, sometimes it does not even load the Basic Authentication pop-up, unless I temporarily disconnect the debugger. Yes, running a debugger makes things slower in general, but why does it fluctuate so much? Sometimes it's quick enough, other times it's too slow to do anything, and I have to disconnect the debugger for the application to keep running.
     
    Akito Kitsune
    Greenhorn
    Posts: 20
    • Mark post as helpful
    • send pies
      Number of slices to send:
      Optional 'thank-you' note:
    • Quote
    • Report post to moderator
    I have deployed the same scenario into a Linux container and the debugger displays the same symptoms. `configFile` is `null`, etc.

    The difference is the resulting exception. On Linux, we are getting a `java.lang.SecurityException: java.io.IOException:  (No such file or directory)` exception. Again, strongly suggests the configuration file is not found.

    How is it possible that it does not find the configuration file? The JVM parameter is correctly provided. It points to an absolute path, where the configuration file is located. Permission issues cannot be the cause, either.

    I also checked the path via Linux' `ps` tool and it is absolutely correct. That's the correct path to the configuration file. Yet, it complains about not finding the file, which just does not make any sense.
     
    Tim Holloway
    Saloon Keeper
    Posts: 25476
    180
    Android Eclipse IDE Tomcat Server Redhat Java Linux
    • Mark post as helpful
    • send pies
      Number of slices to send:
      Optional 'thank-you' note:
    • Quote
    • Report post to moderator
    Rather than looking from the outside in, I recommend looking from the inside out (use System.getProperty() and look at what comes back). Just in case.

    If in fact, the path you see is letter-for-letter, space-for-space exact match including upper/lower case, then I'd venture that you may have a file access permission error. If the Tomcat userID cannot read (and possibly write) that file, then it may appear invisible. Also check the directory that it's in, although if it's Tomcat's own home, then I would hope that's not an issue.

    Don't forget to check the extended attributes (ls -lZh). The audit2allow/audit2why utilities can tell you if there's a problem.
     
    Akito Kitsune
    Greenhorn
    Posts: 20
    • Mark post as helpful
    • send pies
      Number of slices to send:
      Optional 'thank-you' note:
    • Quote
    • Report post to moderator

    Tim Holloway wrote:Rather than looking from the outside in, I recommend looking from the inside out (use System.getProperty() and look at what comes back). Just in case.

    If in fact, the path you see is letter-for-letter, space-for-space exact match including upper/lower case, then I'd venture that you may have a file access permission error. If the Tomcat userID cannot read (and possibly write) that file, then it may appear invisible. Also check the directory that it's in, although if it's Tomcat's own home, then I would hope that's not an issue.

    Don't forget to check the extended attributes (ls -lZh). The audit2allow/audit2why utilities can tell you if there's a problem.



    Excerpt from the arguments provided:



    When getting configuration values in the debugger:



    It clearly recognises the parameter, because it is an empty String. Not `null`!

    But how in the world it is an empty string, instead of at least `/conf/jaas.config`, I do not know.

    At least, I seem to be on the right track!
     
    Tim Holloway
    Saloon Keeper
    Posts: 25476
    180
    Android Eclipse IDE Tomcat Server Redhat Java Linux
    • Mark post as helpful
    • send pies
      Number of slices to send:
      Optional 'thank-you' note:
    • Quote
    • Report post to moderator
    OK. Looks like the JVM isn't being started with the "arguments provided". No, I don't expect that they'd default to "conf/anything" since this is raw JAAS and not everything has a "conf" directory.

    So how exactly are you starting Tomcat? What do you type into the command prompt? Are you running manually or via systemctl? Do you have a setenv.sh file, and if so, what does it look like?
     
    Akito Kitsune
    Greenhorn
    Posts: 20
    • Mark post as helpful
    • send pies
      Number of slices to send:
      Optional 'thank-you' note:
    • Quote
    • Report post to moderator

    Tim Holloway wrote:OK. Looks like the JVM isn't being started with the "arguments provided". No, I don't expect that they'd default to "conf/anything" since this is raw JAAS and not everything has a "conf" directory.

    So how exactly are you starting Tomcat? What do you type into the command prompt? Are you running manually or via systemctl? Do you have a setenv.sh file, and if so, what does it look like?



    Ah, I meant it should be at least `/conf/jaas.config`, in case the environment variable wouldn't have expanded properly. I did not expect any default.

    Inside the Linux container, the Tomcat application is managed via a `supervisord.conf` provided to the UNIX `supervisor` daemon.

    http://supervisord.org/



    The debugging and JAAS argument are new. The other arguments are there since years.

    That is almost the entire configuration. The missing parts of the configuration are just providing the necessary environment variables, like e.g. `CATALINA_BASE`.
    All these are guaranteed to be correct, because this is literally a production configuration, which is running since years. It must be correct enough for the application to fully work, as it should.
     
    Tim Holloway
    Saloon Keeper
    Posts: 25476
    180
    Android Eclipse IDE Tomcat Server Redhat Java Linux
    • Mark post as helpful
    • send pies
      Number of slices to send:
      Optional 'thank-you' note:
    • Quote
    • Report post to moderator
    You mentioned that you've been running in a container. Is it one of the public Tomcat containers whose Dockerfile I can look at?

    I have a suspicion...
     
    Akito Kitsune
    Greenhorn
    Posts: 20
    • Mark post as helpful
    • send pies
      Number of slices to send:
      Optional 'thank-you' note:
    • Quote
    • Report post to moderator

    Tim Holloway wrote:You mentioned that you've been running in a container. Is it one of the public Tomcat containers whose Dockerfile I can look at?

    I have a suspicion...



    Unfortunately, the Docker image is based on a lot of 3rd party Docker images, I have no control over. They are all basically the result of shoe horning a huge monolithic legacy application into Docker structures.

    The Tomcat part of the whole scenario is just a little component in this hugh mongous monolith. So, from a Docker image perspective, it's run inside an Ubuntu 18.04 image, however the whole monolith has customised everything Java related. Therefore, it's not clearly distinguishable and definable, how things are set up from a macro view, without significant amounts of research and reverse engineering.

    In short: the huge monolithic application has a Tomcat, which is just one of many components it needs to function correctly. This is the Tomcat this thread is about. It is run via the provided `supervisord.conf` I have shown an excerpt from.

    Perhaps you have a way of confirming/falsifying your suspicion through other means, beside looking at the Docker images and their Dockerfiles?
     
    Tim Holloway
    Saloon Keeper
    Posts: 25476
    180
    Android Eclipse IDE Tomcat Server Redhat Java Linux
    • Mark post as helpful
    • send pies
      Number of slices to send:
      Optional 'thank-you' note:
    • Quote
    • Report post to moderator
    I hope I didn't hear what it sounded like I heard.

    The point of containerization is to support microservices - do one thing and do it well. You can interconnect those services to make more complex systems, but if you start cramming multiple functions into a single container you might as well just create a full-blown VM.

    I'm familiar with supervisord. Containers aren't really very friendly to dæmons, and even less so to systemd, so supervisord fills the gap.

    OK, so here's my suspicion. One thing to bear in mind is that you have 2 different types of images in a Docker-style environment. One is the actual Docker image and the other is the container image. When you start a Docker container, the base image is used to construct a container image, with suitable run options configured. At that point, the two images diverge. When you restart the container, it is the container image that is used. So changes in the original image don't get seen. Or more precisely, since the container image is anchored to a particular base image, you can't simply update the base, your changes have to build a new image. And as far as I can recall, Docker can't (yet) rebase. So if you change a container, you need to destroy the current Docker container using the docker rmi command and construct a new container based on the new image build. You might was to take inventory of old images - Docker manages that stuff fairly efficiently, but who needs a lot of lint?

    If you don't update the container instance, then the old startup script still gets executed and your new VM parameters won't be properly set.
     
    Akito Kitsune
    Greenhorn
    Posts: 20
    • Mark post as helpful
    • send pies
      Number of slices to send:
      Optional 'thank-you' note:
    • Quote
    • Report post to moderator

    Tim Holloway wrote:I hope I didn't hear what it sounded like I heard.

    The point of containerization is to support microservices - do one thing and do it well. You can interconnect those services to make more complex systems, but if you start cramming multiple functions into a single container you might as well just create a full-blown VM.

    I'm familiar with supervisord. Containers aren't really very friendly to dæmons, and even less so to systemd, so supervisord fills the gap.



    Indeed, you are absolutely correct. But the last time I said exactly what you just said, I was told "it's good enough" and the discussion was over.

    Tim Holloway wrote:

    OK, so here's my suspicion. One thing to bear in mind is that you have 2 different types of images in a Docker-style environment. One is the actual Docker image and the other is the container image. When you start a Docker container, the base image is used to construct a container image, with suitable run options configured. At that point, the two images diverge. When you restart the container, it is the container image that is used. So changes in the original image don't get seen. Or more precisely, since the container image is anchored to a particular base image, you can't simply update the base, your changes have to build a new image. And as far as I can recall, Docker can't (yet) rebase. So if you change a container, you need to destroy the current Docker container using the docker rmi command and construct a new container based on the new image build. You might was to take inventory of old images - Docker manages that stuff fairly efficiently, but who needs a lot of lint?

    If you don't update the container instance, then the old startup script still gets executed and your new VM parameters won't be properly set.



    No worries, DevOps is one thing I have sufficient skills for.

    The Docker basics are not a problem.

    I just have a problem with decade old technology, which is the opposite of cloud native.

    And even then, it may be proven a 100% anyway, that the Docker basics are not a problem, because else I wouldn't have been able to connect via Remote Debugging, as this is a new change in the Java arguments, as well.

    Finally, I started the whole scenario on a Windows machine, where Docker wasn't used, at all. I just needed to run the Docker setup for the Linux side of things.
     
    Tim Holloway
    Saloon Keeper
    Posts: 25476
    180
    Android Eclipse IDE Tomcat Server Redhat Java Linux
    • Mark post as helpful
    • send pies
      Number of slices to send:
      Optional 'thank-you' note:
    • Quote
    • Report post to moderator
    So, put lipstick on a cow and pretend it's modern technology, eh? Oh well.

    Regardless, it appears that what you think is your JVM starting command line isn't your actual starting command line.

    As far as running this on a non-Docker Windows system, that's another matter. It has been a while since I've played with the Windows Service Manager, which is what I'd normally do and I don't know where the extra parameters would go.

    For testing purposes, you might want to bulld a docker image based on the official Apache Tomcat Docker image (https://hub.docker.com/_/tomcat). You should be able to set that as a base with a /usr/local/tomcat/lib/setenv.sh tacked on. The full repo base is here: https://github.com/docker-library/tomcat so you can select whichever version of Tomcat and Java you want, as well as base OS - they've Debian, Ubuntu, and Oracle Linix versions, I think.
     
    Akito Kitsune
    Greenhorn
    Posts: 20
    • Mark post as helpful
    • send pies
      Number of slices to send:
      Optional 'thank-you' note:
    • Quote
    • Report post to moderator

    Tim Holloway wrote:Regardless, it appears that what you think is your JVM starting command line isn't your actual starting command line.



    Excerpt from the output of running `ps auxf`:



    The argument is provided.

    When getting the system properties inside the debugger, it recognises both options. But it only has the correct value for `file.encoding`, whereas `java.security.auth.login.config` does have an empty String as its value.

    Therefore, the arguments are a 100%, without a doubt, provided. It just does not read the value of `java.security.auth.login.config`, as it should.
     
    Akito Kitsune
    Greenhorn
    Posts: 20
    • Mark post as helpful
    • send pies
      Number of slices to send:
      Optional 'thank-you' note:
    • Quote
    • Report post to moderator
    I have tested the same setup once again on the Windows side and the path to the configuration file is actually provided, i.e. the System Property is not an empty String, as it is the case on the Linux side.

    Still, even when it's not empty, the symptoms remain the same. `No LoginModules configured for example`, even though they are correctly configured and working.

    I'm starting to get a feeling that this is a bug with Tomcat. The local Tomcat version I used for initial testing successfully has a way higher patch version, than the production Tomcat. If I remember correctly, it was a difference of about 50 patches.
     
    Tim Holloway
    Saloon Keeper
    Posts: 25476
    180
    Android Eclipse IDE Tomcat Server Redhat Java Linux
    • Mark post as helpful
    • send pies
      Number of slices to send:
      Optional 'thank-you' note:
    • Quote
    • Report post to moderator
    If you have to choose between the "ps" command and an actual getProperty(), the one to believe is getProperty(). It's what JAAS itself will be using.

    I have no idea where the mismatch is, but generally the cause would be A) the ps info is for a different running instance than the one you're doing the debugging on, B) you haven't properly spelled/capitalized/punctuated the "-D" option name compared to what you're querying via getProperty() or C) something altered the property value after the VM launched. Highly unlikely and I'm not even sure that the System properties can be altered without checking.

    If I had to blame a bug in a vendor component, though, it wouldn't be Tomcat. The reason being that Tomcat itself isn't failing, the JAAS function is failing, and since the config file path is read from System properties directly by JAAS, Tomcat shouldn't be influencing it. That would mean that the bug, if any, was in the JRE, since JAAS is part of the JRE (or JRE component of a JDK).
     
    Akito Kitsune
    Greenhorn
    Posts: 20
    • Mark post as helpful
    • send pies
      Number of slices to send:
      Optional 'thank-you' note:
    • Quote
    • Report post to moderator

    Tim Holloway wrote:If you have to choose between the "ps" command and an actual getProperty(), the one to believe is getProperty(). It's what JAAS itself will be using.

    I have no idea where the mismatch is, but generally the cause would be A) the ps info is for a different running instance than the one you're doing the debugging on, B) you haven't properly spelled/capitalized/punctuated the "-D" option name compared to what you're querying via getProperty() or C) something altered the property value after the VM launched. Highly unlikely and I'm not even sure that the System properties can be altered without checking.

    If I had to blame a bug in a vendor component, though, it wouldn't be Tomcat. The reason being that Tomcat itself isn't failing, the JAAS function is failing, and since the config file path is read from System properties directly by JAAS, Tomcat shouldn't be influencing it. That would mean that the bug, if any, was in the JRE, since JAAS is part of the JRE (or JRE component of a JDK).



    Good points.

    I was already very paranoid about spelling. I have copied and pasted the same names and paths so many times now, I am sure I can exclude a misconfiguration by typing mistakes.

    This issue has become such pain, the next step will be running a debugger on `java.io.File` and `java.nio.File`, to see where the JAAS configuration is being loaded. Once I found that, I want to see what Java is reading and what kind of configuration it has. Then I want to see how it does not match the matching configuration...
     
    Akito Kitsune
    Greenhorn
    Posts: 20
    • Mark post as helpful
    • send pies
      Number of slices to send:
      Optional 'thank-you' note:
    • Quote
    • Report post to moderator
    After debugging `java.io.File` and `java.nio.File`, I found out that there is actually a second JAAS configuration file loaded somewhere in-between.

    It turns out the system property `java.security.auth.login.config` is overridden at some point in the whole Tomcat process. It does not happen in the beginning, else I would've noticed it immediately.

    So, I removed the explicit JVM parameter, specifying the path to my newly provided JAAS configuration file, so it can load the original one, only.

    I also added my additional LoginContext to the original JAAS configuration file I had discovered, so my LoginModule may be loaded, when defined in this single JAAS configuration file.

    The result is, that it indeed loads. Moast of the time.

    Sometimes, it still does not load. It still says, that no LoginModule can be found. However, when I restart the Tomcat it works most of the time. Not sure if this is a problem with the debugger making everything too slow.

    At least I found the main issue in the race conidition type of situation, described above.

    Now, I have to figure out why it sometimes works and why it sometimes does not. Another race condition? How?

    I also need to know, why it is set to an empty String, when deployed on the Linux target.
     
    Tim Holloway
    Saloon Keeper
    Posts: 25476
    180
    Android Eclipse IDE Tomcat Server Redhat Java Linux
    • Mark post as helpful
    • send pies
      Number of slices to send:
      Optional 'thank-you' note:
    • Quote
    • Report post to moderator
    This is indeed bizarre.

    Have you tried setting a conditional breakpoint on the java.lang.System#setProperty method? You'll want it conditional because otherwise you will probably get buried in useless breaks. Set the condition to match "key=java.security.auth.login.config".
     
    Tim Holloway
    Saloon Keeper
    Posts: 25476
    180
    Android Eclipse IDE Tomcat Server Redhat Java Linux
    • Mark post as helpful
    • send pies
      Number of slices to send:
      Optional 'thank-you' note:
    • Quote
    • Report post to moderator
    Just for info, I looked at  the source code for OpenJDK and for Tomcat.

    OpenJDK never sets the java.security.auth.login.config property, it only reads it.

    Tomcat WILL set the java.security.auth.login.config property, but its default value is "conf/jaas.conf" and the property is only set if spnego can't find a preset (command-line) value for java.security.auth.login.config. So in no wise should that property ever be set as an empty string in either the JVM or in Tomcat. Plus, I think spnego only applies if you're authenticating with Kerberos.

    So the mystery remains.
     
    It is no measure of health to be well adjusted to a profoundly sick society. -Krishnamurti Tiny ad:
    free, earth-friendly heat - a kickstarter for putting coin in your pocket while saving the earth
    https://coderanch.com/t/751654/free-earth-friendly-heat-kickstarter
    reply
      Bookmark Topic Watch Topic
    • New Topic