WebSocket breaks after Tomcat 7 to 8 upgrade, using NIO Connector, while having ThreadLocal in code

[Marcin Gromadzki]

Websocket connection works on Spring’s WebSocket 4.1.6’s TextWebSocketHandler. For connection establishment we have HandshakeInterceptor, which in its beforeHandshake() method sets the user context:

AuthenticatedUser authUser = (AuthenticatedUser) httpServletRequest
    .getSession().getAttribute(WEB_SOCKET_USER);

UserContextHolder.setUserContext(new UserContext(authUser));

The UserContextHolder class is modeled like Spring’s org.springframework.context.i18n.LocaleContextHolder - a class to hold the current UserContext in thread local storage:

private static ThreadLocal<UserContext> userContextHolder = new InheritableThreadLocal<UserContext>(); 

public static void setUserContext(UserContext userContext) { 
    userContextHolder.set(userContext); 
    if (userContext != null) { 
        LocaleContextHolder.setLocaleContext(userContext); 
    } else { 
        LocaleContextHolder.setLocaleContext(new LocaleContext() { 
            public Locale getLocale() { 
                return UserContext.getDefaultLocale(); 
            } 
        }); 
    } 
} 

This UserContext holds the thread’s authenticated User information for entire app, not only Spring (we also use another software, like Quartz, coexisting on this codebase on JVM, so we need to communicate between them).

Everything works okay when we run on the previous Tomcat 7 with standard BIO Connector. The problem arises with the upgrade to Tomcat 8 with the new NIO Connector being enabled by default.

When the WebSocket’s messages arrive they are processed in a call to Service methods which is validated in @SecurityValidation annotation, the MethodInterceptor checks if given thread has the User Context set:

AuthenticatedUser user = UserContextHolder.getUser(); 

if (user == null) { 
   throw new Exception(/* ... */); 
} 

But it’s null, so Exception gets thrown.

We believe that the problem is in threading change after the switch from BIO to NIO Connector.

BIO Scenario – we have one thread per one WebSocket model, so one Handshake sets one UserContext and it works on this exact thread. It works okay, even when there are more sockets, because i.e., when we have 4 different WebSockets open, there are 4 different threads handling them, that’s why ThreadLocal usage is working well.

NIO Scenario – the Non-Blocking IO concept is for reducing the threads number (simplifying for our case), internally in 3rd party NIO Connector there is used NIO’s Selector to manage the workload on a single thread (with an Event Loop I guess, need to confirm). As we now have just one single thread to handle all the WebSockets (or at least, some part of them) the unexpected exception is thrown.

I’m not sure why once set UserContext gets nullified later, the code investigation brings us no clues, that’s why we think that might be a bug (or something).

The UserContext’s ThreadLocal being null in NIO seems to be the cause of the exception. Has anyone used ThreadLocal with NIO Connector? What is the best way to migrate Tomcat 7 BIO implementation to Tomcat 8 NIO implementation when using ThreadLocal in WebSocket communiction? Thanks!


Also available on: https://stackoverflow.com/questions/71066279/websocket-breaks-after-tomcat-7-to-8-upgrade-using-nio-connector-while-having