You wouldn't notice the efficiency difference. There might be a few milliseconds, but when you take maybe 0.3seconds to click a button, that is an imperceptible delay.
What I think you would notice, and I have fulminated about it on these fora before, is a maintenance overhead.
BTW: There is a link in the most recent Java Developers' Network newsletter to an interview with Brian Goetz. Goetz says you ought not to consider efficiency when coding at all; under "Write Dumb Code" he says we ought to write simple code and leave the efficiency and performance issues to the compiler.
If you get your outer class to implement several interfaces, you will end up with lots and lots of code like this:-which makes it very awkward to add (let alone remove) Components to the display.
Unless you have a Listener which listens to a Component as such (ComponentListener definitely, MouseMotionListener for positions of the mouse quite probably), in my opinion you are far better off with separate classes for the Listeners. Inner or outer, depending on whether you are planning to use them elsewhere.
All that thinking. Doesn't it hurt? What do you think about this tiny ad?
Building a Better World in your Backyard by Paul Wheaton and Shawn Klassen-Koop