Multiplying by a power of two would have the effect of left-shifting the hash value, losing bits of info to the left and replacing them with zeros to the right. If you multiplied by 32, you'd use 5 bits of info with each multiplication, and for a long string, only the last 7 letters would end up affecting the hash at all. Which could be very bad if the strings have a common ending. By using an odd number, the final hash can still reflect contributions from all parts of a string.
I'm not sure what's special about 31, exactly, other than it being prime. Whatever the rationale, it's locked in now, since they (foolishly?) specified the exact hash formula for String as part of the API. There was no need to do this, and now it would prevent them from using a better formula if someone came up with one. Of course, they could just change the API too. But Sun has always been very reluctant to do this sort of thing, since in theory someone might have written code which depends on String behaving exactly as its API indicates. So they won't change the API without a very compelling reason.