but I want the letter on the button underlined as well, without pressing Alt)
Pressing the Alt is a well defined standard for invoking a button. MS and other companies have spend millions of dollars developing a common UI for Users to use and understand. I don't know why you would try to change this practice.
I would use a KeyListener
Using a KeyListener is not the proper way to attempt to do this. If you really want to go againt all conventions then instead you need to use Key Bindings. Read the section from the Swing tutorial on
How To Use Key Bindings. Then it would be a 2 step process:
1) You still have to do what you are doing, that is set the mnemonic so that the letter is underlined
2) Create the key binding using the same Action.
Now the user will have the choice of using either approach.