Richard Levy

Hi,

We solved the problem in the end - well, with help from the tomcat-users mailing list.

It came down to my lack of understanding about the AJP connector. I'd assumed (incorrectly) that it was pretty dumb and we needed to separate out the SSL traffic all the way to Tomcat (hence the 2 connectors). In reality, the AJP connector is pretty smart and retains information about the request. We simplified our configuration and now it works.

Full details can be found here : http://www.mail-archive.com/[email protected]/msg88208.html.

Thanks for your help.

Rich

Hi all,

First of all, sorry for the long post, but the reasoning and architecture requires explanation to get the full understanding of the situation.

We are currently migrating our website from Oracle 10g to Apache 2/Tomcat 6.0.29. The site itself is a set of pages where customers select what they want to purchase, then once they're happy, it switches to SSL to gather their customer details and take payments.

The current setup involves two intelligent load-balancers that terminates SSL then hands over to Oracle 10g. The 10g stack has custom Oracle versions of Apache which have configuration options not found in standard Apache and appear to have made configuration simpler. For instance, there is a "simulateHttps" directive that you can give to a virtual host in Oracle Apache which does not exist in the standard version. Using that option, we simply have 2 virtual hosts defined in Apache, one for http traffic and one for https traffic (with simulateHttps set to on). They both forward to OC4J and it works fine.

With the new architecture we have two load-balancer which round-robin to two Apache servers. These servers connect to two Tomcat servers which are configured with a mod_jk loadbalancer using session affinity. Through config we have separated HTTP & HTTPS traffic. We have done this because the application needs to identify when it is secure, which we do by simulating HTTP using settings in the connector in the Tomcat server.xml. This is detailed further down.

Obviously we want to keep the same functionality on the Apache/Tomcat stack, and also retain the termination of SSL at the load-balancer, but are having trouble when the application switches to HTTPS. It works fine if we have a single Tomcat instance running, but once the 2nd Tomcat instance is enabled, moving to HTTPS fails because a new session is created.

Our configuration is as follows:

Apache

We have 2 Apache servers running - they are served in a round-robin style by the load-balancer. They recieve 2 types of traffic, non-secure and secure, but the secure traffic is NOT encrypted (SSL terminated at the load-balancer).

httpd.conf:
JkWorkersFile conf/workers.properties Listen 5555 Listen 6666 NameVirtualHost *:5555 NameVirtualHost *:6666 <VirtualHost *:5555> ServerName http://www.mysite.com:80 JkMount /* loadbalancer </VirtualHost> <VirtualHost *:6666> ServerName https://www.mysite.com:443 JkMount /* loadbalancerSSL </VirtualHost>
We are using mod_jk to connect to Tomcat

workers.properties
worker.list = loadbalancer, loadbalancerSSL, status worker.tomcat1.type=ajp13 worker.tomcat1.host=192.168.1.36 worker.tomcat1.port=8009 worker.tomcat2.type=ajp13 worker.tomcat2.host=192.168.1.85 worker.tomcat2.port=8009 worker.tomcatSSL1.type=ajp13 worker.tomcatSSL1.host=192.168.1.36 worker.tomcatSSL1.port=8010 worker.tomcatSSL2.type=ajp13 worker.tomcatSSL2.host=192.168.1.85 worker.tomcatSSL2.port=8010 worker.loadbalancer.type = lb worker.loadbalancer.sticky_session = 1 worker.loadbalancer.balance_workers = tomcat1,tomcat2 worker.loadbalancer.session_cookie = JSESSIONID worker.loadbalancerSSL.type = lb worker.loadbalancerSSL.sticky_session = 1 worker.loadbalancerSSL.balance_workers = tomcatSSL1,tomcatSSL2 worker.loadbalancerSSL.session_cookie = JSESSIONID worker.status.type = status
As you can see, Apache can receive on either 5555 or 6666 dependant on the source of the traffic. If it's non-secure, it uses the loadbalancer called "loadbalancer" which uses two Tomcats listening on port 8009 - this is a connector in Tomcat that is not secure. For traffic on 6666, "loadbalancerSSL" is used. This is the same two Tomcats but on port 8010 - this is a connector that believes it is secure. Tomcat config is detailed below.

Tomcat

We have 2 Tomcat servers. They too recieve two types of traffic, non-secure and secure, as passed on from Apache.

server.xml Tomcat 1:
<!-- Define an AJP 1.3 Connector on port 8009 --> <Connector port="8009" protocol="AJP/1.3" redirectPort="8010" maxThreads="200"/> <!-- This is the secure connetor, although its not using SSL, it's just secure traffic --> <Connector port="8010" protocol="AJP/1.3" connectionTimeout="20000" scheme="https" SSLEnabled="true" secure="true" maxThreads="200" proxyPort="443"/> .... <Engine name="Catalina" defaultHost="localhost" jvmRoute="tomcat1">
server.xml Tomcat 2:
<!-- Define an AJP 1.3 Connector on port 8009 --> <Connector port="8009" protocol="AJP/1.3" redirectPort="8010" maxThreads="200"/> <!-- This is the secure connetor, although its not using SSL, it's just secure traffic --> <Connector port="8010" protocol="AJP/1.3" connectionTimeout="20000" scheme="https" SSLEnabled="true" secure="true" maxThreads="200" proxyPort="443"/> .... <Engine name="Catalina" defaultHost="localhost" jvmRoute="tomcat2">
As can be seen, the only difference between the two Tomcat configs is the jvmRoute parameter on the Engine config.
The connector on 8009 is a vanilla connector configuration. On 8010, SSL related parameters are set so that it believes it is secure.

What's happening

If I have a single Tomcat running, I can progress all the way through the site, go secure, retain the session and get to the end of the process. As soon as the 2nd Tomcat in introduced, when it goes secure, the site returns to the first page. I believe this is because sticky sessions fails and the requests ends up on a separate server, hence creating a new session. The application detects this and sends the user back to the first page in the process (it needs the data in the session at this point and as there is none, redirects to the start).

I think that sticky sessions fails because the process behind it uses the jvmRoute value from the Tomcat server.xml to append to the end of the JSESSIONID cookie. When the app goes secure and Apache receives traffic on 6666, it starts to look for tomcatSSL[1/2] but none of the tomcats use that jvmRoute and hence it will not be part of the JSESSIONID, so it will not stick to the same server. The reason that it works with one server is because it only has a single server to send it to, therefore will always find the session.

If this is the case, we may have approached this incorrectly. Separating non-SSL and SSL traffic all the way down to Tomcat may not be the answer and we may need to use something in Apache to modify request and response, in the same way that I assume "simulateHttps" worked.

Has anyone else setup a site using this sort of architecture? I can't imagine that this is uncommon in the enterprise?

Any help is greatly appreciated!

Many thanks
Rich

Hi,

I've written a much better description which explains the full architecture, has all the reasoning behind it, and contains the relevant config. I'll start a new topic with this as I haven't been very clear here.

This thread can be considered closed.

Thanks
Rich

Hi,

thanks for replying.

I have proved now that the the two connectors (as defined below) use the same session by changing the 2nd connector to an insecure one and moving through our site, manually changing port numbers to exercise both.

I'm a little confused - If the session ID changes, you're getting a different session object underneath. Why would the session ID need to change moving between HTTP & HTTPS?

We've now actually moved this forward a little. The problem we appear to be having is that when we move between HTTP & HTTPS, the sticky session routing in mod_jk in apache is not keeping us on the same Tomcat, so it creates a new session.

We do have two virtual hosts defined in Apache, but for the same site. One for non-ssl and one for ssl traffic. We need to do this because the SSL is terminated in the firewall in front of Apache and we need to separate the traffic down to Tomcat and use the appropriate connetor so it serves up pages with correctly rendered URLs etc (hence the reason for the 8010 connector which believes it's secure!).

My initial post was typed in a hurry so I hope this has cleared up a few things. I think a diagram would probably help!

Thanks
Rich

Hi all,

We have a web application that we're migrating to Tomcat 6 from OC4J. Half way thru the application flow it moves from HTTP to HTTPS. When it does this, instead of using the same session, it creates a new one and falls over. In OC4J there was a "shared=true" setting that meant the session was shared and the problem didn't occur. Tomcat is much more strict and does not allow sharing of the session between HTTP & HTTPS (its the law - well ,the servlet spec!).

When our application goes secure, SSL is terminated at the firewall, so actually Tomcat never performs any SSL processing, although it has to realise certain pages are secure. We've achieved this using 2 connectors in the server.xml. The firewall sends requests to either a secure or non-secure virtual host in Apache, which knows to use the appropriate connector in Tomcat. One is pretty much a vanilla AJP1.3 connector for non-secure and the other has the scheme property set to https, along with secure set to true and SSLEnabled set to false. This seems to ensure that pages are rendered appropriately. So, although neither connector is _actually_ secure, the 2nd one believes that it is and does not use the same session as the 1st one.

<!-- Define an AJP 1.3 Connector on port 8009 --> <Connector port="8009" protocol="AJP/1.3" redirectPort="8010" maxThreads="200"/> <!-- This is the secure connector, although its not using SSL, it's just secure traffic --> <Connector port="8010" protocol="AJP/1.3" connectionTimeout="20000" scheme="https" SSLEnabled="false" secure="true" maxThreads="200" proxyPort="443"/>
Is their a way to achieve session sharing through configuration in this environment, or do I need to write code to pass session data over?

Thanks for your help,
Rich

Hi all,

just got my results through, here's the breakdown:

General Con: 92
Documentation: 70
OOD: 30
GUI: 31
Locking: 80
Data Store: 40
Network Server: 40
Total: 383 (96%)

I'm a little surprised at the GUI - I felt I nailed the whole MVC and and dependancy issues, but there you go - do you find out "why" you were given a particular score?

My solution was RMI based. My locking was very simple. For my data store I did provide a fully featured DAO even though the GUI didnt use all of it. My documentation was HTML with screenshots (it looked quite professional). My choices document was quite detailed.

I've been doing the project on and off for 2 years. I scrapped my first attempt and did it in 3 large chunks of about 2 months each. It was a lot of work, but also enjoyable. My evenings are empty now! :-D

Thanks to everyone that read and replied to my many posts.

See you in the next forum :-)

Cheers
Rich

Anyone?

Hi,

I've decided with my assignment that the configuration will be presented on startup. This is consistent for all modes and a simple approach.

In my main GUI (after config) I will obviously adopt the appropriate threading model for handling Swing & work operations. However, for the configuration, which is pretty small (3 fields max!) I've decided that I can do everything in the Swing thread. This is for a number of reasons. 1) I beleive its overkill to do things like simple form validation in another thread. 2) The application hasnt actually started at this point so there is nothing else going on. 3) The dialog box approach I'm using is modal anyway.

These decisions will go in my notes, but do you think I'm cutting a corner and should just bite the bullet and do a "proper" implementation.

Thanks
Rich

Hi all,

Can someone just clarify my thinking on the GUI search requirements for B&S please.

In my instructions it says the following:

'It must allow the user to search the data for all records, or for records where the name and/or location fields exactly match values specified by the user.'

The 'name and/or location' is clear but does 'search the data for all records' simply mean 'display all records'.

Thanks
Rich

Hi all,

While developing my SCJD I've come up with all sorts of crazy ideas for extending it into the most amazing application ever! However, I realise that this turns the assignment into a never-ending development and the documentation for the assignment says that you don't get extra credit for functionality that is not requested.

With this in mind, I'm creating start points for this functionality, but no implementation. In the JavaDoc for these classes I'm describing what could be done here "in the future" but not actually adding any implementation.

Shutdown hooks are a prime example. Its not been asked for but its useful, so I've got a shutdown factory, which at launch returns me an empty shutdown implementation that is mode specific which subsequently gets added as a shutdown hook into the Runtime object. Like I said, the actual class contains an empty implementation and some JavaDoc about what could go here in the future.

I shall also be detailing this in my choices.txt file under a "future expansion" section.

Does this sound like a sensible approach? I don't want to toil over the app forever, but I want to indicate to the examiners that I've thought about it.

Thanks
Rich

Hi

I'm doing the 1.4 assignment so no java.util.concurrent for me.

Yes, my lock mechanism has a while loop exactly like yours.

Interesting about the timeout.

I was planning to have a watchdog process running for each lock that would remove it after a reasonable amount of time (either that or a single process that checks the length of time a lock has been active - this is probably better). That way the lock mechanism doesnt have to worry about timing out as the watchdog process will release excessive locks.

Thanks for the replies.

Rich

Hi all,

I'm going for the RMI approach with my network design.

I'm going to have a session manager, which is a single remote object that all clients will connect to and request data access. This will return a new remote adapter to the actual database. The database has a singleton for file access and locking.

This brings me on to locking. I've seen some quite complex locking mechanisms described here, but my assignment says "any attempt to lock a resource that is already locked should cause the current thread to give up the cpu....". To me this sounds like "if someone is using the record you want, wait". So my lock manager is simply tracking locked records (and associated cookies) using a hashmap and sync'd access to lock and unlock. It doesnt actually care WHO has the resource locked, only that it IS locked.

Have I missed the point?!

Thanks
Rich

Hi,

chipping in late, but I've gone with a facade.

I created a facade on top of the specified DB interface. This not only removed locking from the interface (do it in the implementation of the facade interface), but talks in records not string arrays and the method prototypes also throw RemoteExceptions so that the interface can be used locally and remotely.

Works fine locally, just doing the remote bit now! Hope my scribbled UML actually works :-)

Rich

Hi,

in my assignment it says "your data access class must be called Data.java, must be in a package called suncertify.db and must implement the following interface" <standard DB interface>.

I've taken this to mean that this class must be used to do access to the database from the remote client or the local/server side. So, my Data.java is an ellaborate facade (with factories and adapters etc) that either establishes a remote connection to a serverside database, or creates an object to read the database locally. So as far as data access goes, my Data.java is the highest level data access point and everything happens in it.

Has anyone else interpreted it like this, or do most people use Data.java for the direct file access to the DB then build on top of that?

Honestly, you'd think Sun were out to confuse people the way they write their assignments!

Thanks for your help,
Rich
[ September 06, 2006: Message edited by: Richard Levy ]

Hi

thanks for the reply. Its nice to know that I'm on the right track (or we're both way off the mark ;-) ).

Thanks
Rich