Connecting Tomcat to mod_jk on a cloud server running ISP manager

[Alex Lieb]

Hi! I'm trying to configure a cloud server so that it automatically forwards requests to ports 80 and 443 to Tomcat using an Apache http port-listener/worker service thing.

In a broader sense, I'm trying to set this server up to host a series of web applications through Tomcat 7 so we can have a test server at work that mirrors the production server. My boss asked me to set up the server we would use for the development environment, and yesterday a coworker asked me if I'd done the mod_jk stuff to it, because he wanted to set it up, but because this kind of thing is really the sort of thing I know the least about, I decided if it was ok with everyone I would really rather try to do it myself.

So, fast forward to today; I got my wish, and now I'm up a river without a paddle. It's not really that bad; this isn't the production site, so it's not incredibly time-sensitive, and my boss is fully aware that I hadn't even heard of this until yesterday, but I'm still getting stuck a lot here. Fortunately I had to do something like this about 4 months ago, and it was already set up on that server, so I've been doing a lot of comparing between the new server and that server, but I finally hit a roadblock that I can't really solve by doing that, and which is apparently too specific or technical for a google search.

I was having an issue where after I changed some things in the \etc\httpd\ configuration folders, httpd wouldn't start at all. Like; you go into ISP manager and go to "Services" and hit the green arrow, and... *nothing* happens. No log output, no "I tried to start but it didn't work ", nothing; but after combing through the log folder for a while and fiddling with some other settings I finally found a log file which was useful; the server boot log. It tries to start httpd on boot, but it fails, saying: Starting httpd: httpd: Syntax error on line 992 of <the config file>: Could not open configuration file /usr/local/mgr5/etc/ispmgr.inc: No such file or directory
[60G[[0;31mFAILED[0;39m]

Bingo. So I went back to the config file and found that line... I remember changing that line; in the old server it pointed to a different filepath because that one is running a slightly different version of ISP manager, so it was in a different place. I did change it though, so I know it's not just pointing at where it was in the old server. What's more interesting is when I first opened the config file on the new server and started looking through it, there was a line that said "Include /usr/local/mgr5/etc/ispmgr.inc", but that line was commented out. It was also in exactly the same location as the line "Include <path to ispmgr.inc on the other server>" in the old server's file, so I checked to see that the /usr/local/mgr5/etc/ directory existed and uncommented it. What I forgot to check though, was whether ispmgr.inc was *in* that folder.

Apparently this server has no file called ispmgr.inc. Since I'm not sure why it even wants to know where ispmgr.inc is in the first place though, I'm really not sure what I *should* have it look for instead, or if there should actually be an ispmgr.inc file there. I did a search for ispmgr.inc on the whole server,

So that's my specific problem, but I'd really just like to understand how these things work better as a generality. Like; for example, why are we even bothering with this mod_jk thing? I know there's a reason, but I don't know what that reason is. Instead of having a separate service listen on ports 80 and 443, then hand the request over to port 8009, which Tomcat then redirects to 8443, why don't we just have Tomcat listen directly to ports 80 and 443?

So... I know this whole process involves an Apache http server which receives requests, hands them to user/configuration-defined "workers", which; I'm not really sure how I should be conceptualizing these 'worker' things, are they a service? An object? A process that runs when called, then dies? What are these worker things? Also, I know mod_jk is involved in this, and you have to have it installed, but I'm not entirely sure how it works or how it's separate from the Apache http server or the workers. I know mod_jk is sort of the thing which actually *does* the connecting between Tomcat and the Apache server, but I'm not sure why they need to be connected; can't you just forward the request to the port and let it go from there? Also, there's a thing called AJP; it's some kind of protocol, but I'm definitely somewhat out of depth at this point; it looks like this is the thing that actually handles the proxying from whatever you put in to whatever port you're forwarded to... So you don't *actually* get redirected, you just put in blah.come/site, your browser requests https://blah.com:443/site and you're served up the content located at https://blah.com:8080/site... I think. I think that's how this works? I'm about 90% sure that there's numerous egregious errors in my understanding of this though... And if anyone knows what file I'm actually supposed to be including, that would be wonderful, but in a broader sense if anyone is super familiar with this stuff or knows where to find some good documentation that explains the relationship between all of these pieces, thus lending context to all of the explanations of the individual pieces, that would be great!

[Tim Holloway]

Welcome to the JavaRanch, Alex!

That's a lot to sort through, but here's some general wisdom while I try and make sense of what you said.

First of all, It's a really bad idea to run Tomcat on ports 80 and 443. The reason for that is that on most OS's, anything below port 4096 is considered "magic" and requires administrator privileges before you can listen on it. The magic had to do with supposedly making things more secure overall, but I'm not sure it ever did. But, in short, to give those ports to Tomcat means that the entire Tomcat server must be running as a privileged user, which means you've just opened up a great big opportunity for Bad Guys to exploit the entire machine.

Apache HTTP starts as a priviliged user, but once it's up and running, it downshifts itself to something more harmless - typically a user "apache", "apache2" or "httpd", depending on the OS and the moods of the sysadmin. So Apache makes a secure reverse proxy for Tomcat, in addition to being able to sponsor non-J2EE webapps at the same time.

Other servers can also do this. I use nginx these days. You can also use iptables rules on Linux. You can use a dedicated hardware proxy box.

And, in the cloud, the cloud itself may supply proxy services. Amazon does, for example.

Running a reverse proxy in Apache requires running either mod_proxy or mod_jk. The mod_proxy way is recommended, but if you want encryption between Apache and Tomcat (and not just SSL from the Internet to Apache), then you need the ajp protocol. mod_jk offers more load-balancing options (I think). And if you come across references to mod_jk2 or mod_jk3, ignore them. They were dead ends.

What these various Apache modules do is create proxy tunnels from Apache to the backend Tomcat server(s). The mod_proxy module can forwards almost the exact input stream that came in to Apache (less any TLS decryption and adding some proxy headers). The mod_jk module uses a custom binary data format, decoded at Tomcat's port 8009 or whatever you tweak Tomcat's server.xml file to use instead.

[Alex Lieb]

Alright; so this service doesn't really exist because "Connecting port A to port B is hard", it exists because "Letting people look at a port without giving them direct access to it is hard." The Apache server itself is just a listener, but it uses mod_jk or mod_proxy to do handle the connection to the other port, and AJP handles the encryption for the little leap between those two ports. That makes a lot more sense actually; thanks!

I just realized also; I stopped in mid-sentence above; I did a search for ispmgr.inc, and it isn't anywhere on the box. And I'm pretty sure it's not supposed to be, because based on that ispmgr.inc looks like in the other, working, server, it's a configuration file that looks like it's probably essential for ISP manager to run... And since ISP Manager is running on the other server, it must use some other kind of config file. Or it just isn't as necessary as I'm thinking...

[Alex Lieb]

So I finally just threw the old ispmgr.inc file into the new server just to see what would happen, and see if I could maybe get a more useful error message. It did give me something more useful; apparently it tries to run ispmgr.inc like a script; the boot log showed an error that said it failed on line 1 of ispmgr.inc because a file that it referenced did not exist... Which.. Of course it didn't, but when I searched for where that file was on the new box, I realized it didn't exist.. Which really begs the question; does the Apache server even *need* this information? If this much has changed in the ISPManager configuration, and this file is actually being run as a script, what are the odds that the function fulfilled by this script is being some somewhere else now, and I don't even need to be including ispmgr.inc in the first place?

So I just removed this line from my httpd.conf file entirely:

Include /usr/local/mgr5/etc/ispmgr.inc

And um... Well, it works now. It makes sense I guess; there's really no reason I can think of that httpd would need to be telling ISPManager to do anything, but I'm still perturbed because I don't really know why it was in there in the first place. What was it doing on the old server that doesn't need to be done now?

The good news, though, is that it's working!

Next step is telling it "Hey, if you get a request to the port, but there's nothing after that, send them to this context's welcome page so they can log in." I'm thinking that should be easier...