The key
word here is "classpath".
Your client will have one classpath, as defined in the
applet tag (assuming it's an applet).
The EJB has its own internal classpath, which has it's container classpath appended to it.
The Utility will also have its own classpath.
Expect to need a log4j properties (or log4j.xml) in each one. It may be an identical copy of the other 2 log4j files, or it may be customized for the component in question, but each classpath must include a log4j properties file, located at the root level (for example, in the WEB-INF/classes directory of a WAR).
The secret of how to be miserable is to constantly expect things are going to happen the way that they are "supposed" to happen.
You can have faith, which carries the understanding that you may be disappointed. Then there's being a willfully-blind idiot, which virtually guarantees it.