Robin Wilson

Greenhorn
+ Follow
since May 10, 2006
Cows and Likes
Cows
Total received
0
In last 30 days
0
Total given
0
Likes
Total received
0
Received in last 30 days
0
Total given
0
Given in last 30 days
0
Forums and Threads
Scavenger Hunt
expand Ranch Hand Scavenger Hunt
expand Greenhorn Scavenger Hunt

Recent posts by Robin Wilson

We are searching for a Java Web Developer in Austin, Texas. The position is a fairly junior position (Software Engineer I), and we're looking for someone with ~3 years of Java development experience, building web-applications. Our environment is Tomcat (6.0.20), Java 1.6, Apache 2.2, MySQL 5, Tapestry 5, Hibernate, etc.

If you are interested, please read the job posting at:

http://www.kingsisle.com/jobs#javaProgrammer

And reply there.

Thanks!
10 years ago
I'm not sure what that would do...

My goal is to be able to run our load generators at full speed indefinitely - and for our Tomcat server to handle the high load conditions without failure. Stopping the traffic would allow the Tomcat to recover, but I'm looking for a way to get Tomcat to properly clear the sessions that are already expired - so that they don't continue to eat up memory.

After I figure that out, then I will adjust the memory on the box to handle the max sustained load (plus some headroom) we expect to encounter on our production environment, and tune our session durations to work well for both our customers and our machine capacity.

Worst case scenario, we will implement something akin to your suggestion - where our external load balancers shift load away from each Tomcat in our cluster for some time-slice out of each hour, so that it has time to recover the session memory. But that would be a nasty "kludge" solution to something that Tomcat should handle correctly in the first place.
10 years ago
OK, it doesn't help to add the strict servlet stuff...

After reading the bug details, this doesn't look like the same problem I'm experiencing. The bug says that sessions just aren't expiring (ever). But my problem is that sessions don't expire while the machine is under load - as soon as I remove the load generators, the sessions start expiring properly (without any other action on my part).

So my problem seems to be that the session expiration runs at a lower priority than the servicing of incoming page (tapestry pages) requests.

The requests are coming over an AJP connection (if that makes a difference).

The ultimate question is, how do I increase the priority of the session expiration (even if it slows the machine down a little - it would be better than letting it build to a crash/hang).
10 years ago
Hmmm. I'll give that a try, but is it still supposed to be a bug in Tomcat 6.0.20? We're not running Tomcat 5.x, so if it was already fixed I'm skeptical that it's the issue.

Either way, give me about an hour and I'll have it tested out.
10 years ago
We've been running some load tests on our machines here, and we've noted a very baffling behavior. If we load up the machines so that the server is running near CPU capacity (80+ % CPU utilization for extended durations), Tomcat will not expire old sessions. We can run the load generators on the machine until it starts thrashing on Full Garbage Collection activity (i.e., a Full GC takes 20+ seconds, and it is doing a Full GC every 22-25 seconds, so we only get 2-5 seconds of 'run time' between GCs).

The thing we've noted is, our GC activity is caused by sessions filling up the memory space. If we pause the load generators for 2-3 minutes, the sessions will go away as expected and then we can restart the load gens and everything runs smoothly until we build up some more sessions.

Our environment is clustered, so the problem is even more significant - since the sessions are essentially copied from every server in the cluster (QA has 2 members in the cluster, production has 4 - so the problem is very significant when we get to prod!)

Even going into the "/manager/html" webapp and manually clicking the "Expire" button (for sessions with >= 1 min idle time) doesn't seem to affect the session expiration when the machine is under load. But as soon as we remove the load, the machine begins to recover and expire the sessions.

FYI, for testing purposes, I've set our session timeouts in the 'web.xml' file to 1 minute, but we actually control the session duration in code and have it set to 180 seconds in the code configuration. (When I examine the sessions - they seem to have the correct expiration times - 3 mins - and they even show that they have been idle for "20+" mins.)

We're wondering if there is a setting in Tomcat to tell it to alter the priority of session expiration, so that it will take precedence over the page requests.

In case it matters, we're using Tapestry 5.5 for our UI stuff, and we store nothing in the sessions if you aren't authenticated, but about 10K worth of data if you are authenticated. Our load generation is running 200 threads to get a plain page - unauthenticated, and 40 threads to perform registration and login actions, which stores about 10K per iteration... We're running with -Xmx6g, and -Xms2g and a 1g PermSize (although we've never really gone more than about 600KB in Perm space used).

Our site serves several million pages a day, and a substantial number of registrations each day (more than 10K). Which is why we're trying to evaluate loading conditions - to make sure we have our production equipment sizing correct...
10 years ago
I was having a similar problem, and in reading through posts (by the same user as the original note on this post), found that he fixed his problem by changing the '<Host name=...' on his config to be 'localhost')... Well, I had a similar problem, except my Host entry already had 'localhost', and my error was:

WARNING: Context manager doesn't exist:localhost#/><webapp_name>

After verifying that the web.xml in the webapp has the <distributable/> tag (it does), and we use the same '.war' file on each host in the cluster, I was stumped.

Also, a note about my setup - I was only getting the WARNING on _1_ host out of my 2-host cluster.

Anyway, I think I figured out the problem - and thought I would document here in case anyone else is searching for a solution.

1) Our setup uses multiple webapps in our cluster of tomcats. However, only 1 webapp is using session replication - so the rest don't setup to use the cluster manager.
2) If one of the hosts starts up first, it deploys the 'clusterable' webapp before the other host. But the other host starts it's cluster manager and gets a request for the session state of "localhost/<webapp>"... So it throws the WARNING message to alert the local tomcat to the fact that it's being asked for session state on a webapp that it doesn't have (yet). After the server completes startup, this problem goes away, because the webapp eventually gets deployed - and the session state is available to be replicated normally. (Doh!)

To eliminate the erroneous warning messages (for our production environment - no errors/warnings is "good" if they aren't worth worrying about), we added the following 2 lines to our 'logging.properties' (we're using the JULI logger) file:

org.apache.catalina.ha.session.ClusterSessionListener.level = SEVERE
org.apache.catalina.ha.session.ClusterSessionListener.handlers = 2localhost.org.apache.juli.FileHandler

This sets the ClusterSessionListener class to report only 'SEVERE' errors instead of a bunch of useless WARNING level messages.

FYI, we're using tomcat 6.0.18.
10 years ago
This is just a wild guess, since I don't have a lot of experience using Runtime.exec(), but Runtime.exec() is used to execute a "system" process on the server from the JVM. This means that any child of the JVM will inherit the identity of the JVM, not the JAAS login context. It is my understanding that the JAAS login context is related to the client, while the JVM identity is related to the server. This means that the two are separate and distinct entities, and you really can't define the JVM's identity to match the client's identity "directly". (You might be able to fake this by performing some sort of "set UID" functionality in the JVM, but I was unders the impression that there are security mechanisms to prevent this - since you are supposed to be operating in a sandbox that's isolated from the server's "system" environment.

A good question would be: what are you actually trying to accomplish? (Rather than asking everyone to debug your proposed solution, that way we might be able to provide an alternative that you haven't thought of...)
13 years ago
It is almost certainly related to whitespace... The information you are supposed to be hashing is explicitly the 4 lines specified. However, you are manually selecting 4 separate strings, without getting all the whitespace in the 4 specified lines. You are missing "\n" at the end of the first 3 lines, then " " at the beginning of lines 2 and 3... That does make a difference.

(Keep in mind that simply because you can't see a character doesn't mean it isn't there - and if it is there, it has a value that will significantly alter the resulting hash value. We see a file as a series of independent "lines" of data. The computer sees a file as a continuous stream of bit (bytes), including the bytes that represent line breaks and leading/trailing spaces.)
13 years ago
That may be all you need - go look at the PKCS#12 specifications at:

http://www.rsasecurity.com/rsalabs/node.asp?id=2138

But I suspect that you need the _issuer's_ certificate in your trust store. The issuer's certificate will be the one that verifies that the certificate you are trying to use is valid. If you don't have that, your SSL connection will still need to contact the issuer in order to validate the certificate.
13 years ago
Take a look at "JAAS" (Java Authentication and Authorization Service). You can find out _all_ there is to know about it from here:

http://java.sun.com/products/jaas/

If that doesn't cover all you need, take a look at Acegi (you might want to include Spring with that too).

http://www.acegisecurity.org/
http://www.springframework.org/

Hope that points you in the right direction...
13 years ago
Have you looked at "AppFuse"?

http://appfuse.org/

It seems to be a "pre-setup" project that includes Acegi (and Spring, etc.)... Perhaps it would provide a good example on how to do this for you?
13 years ago
Also, unless you have a special reason to use MD5 (like it's all you have), MD5 isn't a good choice. It has been shown to be fairly weak as a digest algorithm, and there are actually sites on the internet with dictionaries of MD5 values. It's better to use SHA (preferably at least SHA256 or better). Modern computing horsepower can brute-force MD5 pretty quickly, and there are a series of mathematical attacks that can further reduce the time it takes to derive the input values of an MD5 digest.

But as Ulf said, a message digest is a 1-way algorithm, it is not technically "encryption", since you can't get back the plaintext once it's been encoded with the algorithm (without using a brute-force or similar attack as mentioned above - and even then you can't be sure you got the _same_ plaintext as was used to create the digest, you can only be sure that the plaintext you get back generates the same digest).

A message digest is a number (very large number) that represents a value for a given string. The string has no size limit, but the number does. The number is calculated through an algorithm that changes the value of the number significantly even with only small changes in the value of the string. But because there are fixed number of values for the number (even though it's a very large number of them), and there are an infinite number of 'strings' that can be used (recall, they have no size limit), there will be more than one string that can generate the same number for the digest. So figuring out a string that generates the same number doesn't necessarily guarantee that the string you have is actually the one used to generate the digest. But this also guarantees that you don't have any way to reverse the algorithm to "decrypt" the number. The number could represent any number of actual plaintexts, so there would be no reasonable algorithm that could use the number to decipher the original plaintext.

Frequently a message digest is used to create a digital signature, or save a password that even the system admin can't know. For a password, the original plaintext is digested and stored (along with a seed to represent a date/time so that two people who use the same password wouldn't be able to figure it out). Then when the user logs in, the password they enter (in plaintext) is re-digested (by pulling the seed off of the stored password for the requested user) and compared to the originally digested password. If they match in digested form, they must have matched in plaintext, so the user is allowed in.

The signature system works similarly. You take a document (or chunk of data), and digest it. Then you encrypt the digest using a private key from a private key/public key pair. Then the document and the signature are transmitted. The encrypted digest is decrypted using the public key, restoring the originally calculated digest. Then the document that was recieved is digested locally (using the same digest algorithm), and compared to the value of the recieved digest from the signature. If they match you know several things to be true: 1) The document matches what the sender signed (since the digests match); and 2) The sender holds the private key (since the sender's public key worked to decrypt the digest).
13 years ago
Ulf makes a good point - if you don't trust your programmers, I suggest you get a set of programmers that you do trust. Now if you are just using "even the programmers" as a metaphor for "assume the most restrictive access controls", then the answer to your question is _yes_, you want to use PKI.

PKI works on the concept of 2 keys that share a relationship, but I can't derive the other one if I have one of the keys. One key is used to encrypt data, and only the other key may decrypt the data.

When you build your system - you will want the application code to immediately encrypt incoming data. If you have control over the client that is submitting the data (i.e., not a web browser), you might even want the data encrypted from the client side. (Actually, I suppose client side encryption would be _possible_ if you used a JavaScript implementation of some Public Key algorithm, but that would be very slow I'd bet.)

The issue is "how do I show the user what they've entered?" If only you have the private key, once the user submits the data - not even they can decrypt it. (If you share your "private" key with the user, it's not very private any more.)

A good way to do this is to actually sit down and evaluate the level of security your data needs. Rather than doing a blanket "encrypt it all and let God sort it out!" approach, take a more rational view of the data and try to encrypt only what actually needs to be encrypted.

Also, you need to carefully evaluate who should actually be able to see the data. If you only want a single individual to be able to see the data, then PKI will work. If you want more than one individual to see the data, it make require a more thought-out approach.
13 years ago
One other thing, you would want to encrypt the response - using the public key of the web application (server), so that the response can't be intercepted and modified along the way...
13 years ago
You could encrypt the e-mail using the user's public key (assuming they have a public key). Then _only_ the user who has the private key (which would be assumed to be the one you thought you were sending the message to) would be able to open the message and reply. Then you could use a one-time token embedded in the message to allow approval, and be completely secure.

Ultimately, if you are sending your message over e-mail, and it's not encrypted - it isn't secure. If your system has any requirement for "real" authentication, you can't do it without encryption. Keep in mind that you are getting an "approval" for a reason - you want to link the approver to the transaction, and guarantee that the approver can't repudiate the transaction (e.g. "I didn't approve that!"). If you send the approval in clear text via e-mail, you are guaranteeing that any approver _can_ repudiate the transaction, because any number of people could intercept the e-mail and approve the transaction. (And even prevent the "real" approver from seeing the e-mail.)
13 years ago