• Post Reply Bookmark Topic Watch Topic
  • New Topic

Thread acquires ReentrantLock which is already acquired by other thread.  RSS feed

 
Marco Antonio
Ranch Hand
Posts: 81
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hello, everybody.

Our application, running in a WebLogic 12c, is retrieving messages from a queuing system, where the queue we are retrieving messages from is configured as a FIFO. We are using Spring to configure the retrieval functionality and both the container (org.springframework.jms.listener.DefaultMessageListenerContainer) and the messages listener (org.springframework.jms.core.support.JmsGatewaySupport) are singletons. Furthermore, that container has configured a WorkManager by default as task executor. To guarantee that the messages are processed in the expected order (the order how they are sent to the queue), we use a ReentrantLock in the listener, and we were expecting that messages were retrieved and processed one by one. The listener code is the following one:


Even though two messages are placed on the queue in the correct order, and they are consumed in that order (recall that the queue is a FIFO one), somehow the two message are processed in parallel by the application as it is shown in the following log chunk:

Lock has been acquired by thread: 28
Backout count: 1
Message 1 / 1 received from XXX
Message ID1 received.
Lock has been acquired by thread: 54
Backout count: 1
Message 1 / 1 received from XXX
Message ID2 received.
***** ERROR *****
Lock is going to be released by thread: 54
Lock is going to be released by thread: 28


Why are we obtaining this behaviour? Any idea?


Thank you so much in advanced.
 
Campbell Ritchie
Sheriff
Posts: 53779
128
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
How do you know the release warnings occur after the apparently preceding lock notifications? Try adding numbers with System.nanoTime to the output and see what happens. There may be a delay in the logging which means the lock is unlocked before the unlocking is recorded by your logger.
 
Campbell Ritchie
Sheriff
Posts: 53779
128
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I presume you only have one Lock object?
 
Marco Antonio
Ranch Hand
Posts: 81
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
First of all, thank you for your answer.

The issue is that the second message has a dependency from the first one, that is why we need process them in the correct order. As you can note, there is an ERROR entry in the log, provoked by the second message because it has been processed before the first one has been successfully finished. That is why I'm sure the log entries with the threads ids are correct. And, yes, we have just defined one lock, the one which is shown in the code in my previous post.

How is possible that the second thread acquires the lock when it is actually acquired? Could it be some WebLogic misunderstood behaviour with threads pool?


Thanks!
 
Henry Wong
author
Sheriff
Posts: 22866
119
C++ Chrome Eclipse IDE Firefox Browser Java jQuery Linux VI Editor Windows
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Marco Antonio wrote:
The issue is that the second message has a dependency from the first one, that is why we need process them in the correct order. As you can note, there is an ERROR entry in the log, provoked by the second message because it has been processed before the first one has been successfully finished. That is why I'm sure the log entries with the threads ids are correct. And, yes, we have just defined one lock, the one which is shown in the code in my previous post.

How is possible that the second thread acquires the lock when it is actually acquired? Could it be some WebLogic misunderstood behaviour with threads pool?


Can you confirm that it is just one (and hence, the same) lock? The easiest way is to also print out the identity hash of the lock during the lock() and unlock() calls.

Henry
 
Marco Antonio
Ranch Hand
Posts: 81
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Thanks Henry for your answer.

No, I cannot confirm that the lock in both threads is the same. I assume that the lock is the same because it is defined in a singleton (instanced by Spring), but I haven't checked if it really is. I'm going to carry out what you have suggest to me, but I'm going to spend some time, because the described behaviour is taking place in the client environment, and I need to generate the patch and send it to be deployed. As soon as I'm informed the result of that test, I'll share it here.

By the way, why are you considering that the lock couldn't be the same? What can be provoking that behaviour?


Thank you so much!!!
 
Campbell Ritchie
Sheriff
Posts: 53779
128
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Marco Antonio wrote: . . . I assume that the lock is the same because it is defined in a singleton . . . .
Are you sure that is a singleton and there is only one objcet doing the connecting?
By the way, why are you considering that the lock couldn't be the same? What can be provoking that behaviour?


Thank you so much!!!
Because you would not expect thread 2 to be able to lock the lock until after thread 1 has unlocked it. Most likely cause of true simultaneous access is that there are two lock objects.

Henry: presumably getting different id hash codes from the two Lock objects proves they are different instances. Does getting the same id hash code prove they are the same instance?
 
Henry Wong
author
Sheriff
Posts: 22866
119
C++ Chrome Eclipse IDE Firefox Browser Java jQuery Linux VI Editor Windows
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Campbell Ritchie wrote:
Henry: presumably getting different id hash codes from the two Lock objects proves they are different instances. Does getting the same id hash code prove they are the same instance?


Guaranteed proof? No.

Confident proof? Willing to bet money on it? Yes...

For the paranoid, you can always run it a few times (to see if it stays equal). Unfortunately, for the really paranoid, you will need to confirm that it is only initialized once -- such as printing from an instance initializer.

Henry

 
Marco Antonio
Ranch Hand
Posts: 81
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Actually it is an assumption because the bean is instanced by Spring, and it is configured by default in this sense. Anyway, I'm going to test that both threads are running in the same instance logging both the bean and the thread identity hashes.
 
Marco Antonio
Ranch Hand
Posts: 81
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Related with what Henry has just mention, I have carried out more than 250 tests (sending/receiving pair of messages) with the explained situation and I have always obtained a successful result. This behaviour is only happening in the client environment

 
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
Boost this thread!