Multithreaded Code and Concurrency Issues

[Geoff McKay]

Congrats Kathy Sierra, Bert Bates & Trisha Gee on your new book: Head First Java.

I'm a huge fan of the visual style of the Head First book series by O'Reilly Media. The visual layout really works well for better learning and retention.

The Java Concurrency package (java.util.concurrent) makes it easier to build multithreaded code. However, Concurrency Issues can still arise when multiple threads are accessing the same object. For instance, when iterating through a process in one thread, another thread working at the same time could potentially change the logic of the original thread causing unexpected behaviours.

Are there suggested tips to avoid issues when writing Multithreaded Code?

Local Scope variables help, but there are times when multithreaded code must interact with non local variables.

How do you help ensure objects are safe for multiple threads to use at the same time?

[Himai Minh]

Hi Geoff,
You may consider using ReentrantLock to make a method thread safe.

[Geoff McKay]

Himai Minh wrote:Hi Geoff,
You may consider using ReentrantLock to make a method thread safe.

Thanks Himai.
I will look into the ReentrantLock class for methods accessing shared resources.
Appreciated!

[Stephan van Hulst]

Besides using the nifty tools in the java.util.concurrent package, these are the most important tips I can give:

Really understand Java's memory model. This means you need to know what the term happens-before means (it's a technical term), and how you can ensure that an action in one thread happens-before an action in another thread.

You need to understand that thread-safety involves two major concerns: Establishing clear happens-before relationships in concurrent execution paths, and guaranteeing that critical sections of your code are executed atomically. All tools provided to you by the language and standard library are only there to help you deal with these two concerns. You can't use the tools properly if you don't understand what these two concerns are, and how they differ from each other.

As a more practical tip, use your debugger's stepper. In many IDEs you can switch the currently active thread, and you can step through the code execution for just that one thread. That way, you can try to break your own code by stepping one thread into a critical section of your code when it's not supposed to be there.

[Trisha Gee]

Hi Geoff,

We tried to cover some of the common problems you'll run into when writing multi-threaded code in Chapter 18, and offer some suggestions on how to avoid them. Of course, one suggestion (taken very seriously by a company I used to work for, who wrote very high-speed applications) is to not use concurrency at all! One service = one thread = one CPU. Another suggestion, which we cover a few times in the book, is to limit which state, or data, can be written to by multiple threads (reading by multiple threads is usually not a problem), and make it clear which data might lead to contention so it's easier to reason about any potential issues. Using immutable objects where possible is one way to embrace this approach.

Deadlock is another common problem, a topic on which I'm sure many books and articles have been written! A simplistic approach to avoid or minimise this as a possibility is to control the order in which threads can access resources, and/or to use timeouts.

I got very excited writing some of the concurrency stuff in the book and could have written a whole book on just that topic! Fortunately, Brian Goetz has already done so, so I'll recommend his book instead.

[Geoff McKay]

Stephan van Hulst wrote:... As a more practical tip, use your debugger's stepper. In many IDEs you can switch the currently active thread, and you can step through the code execution for just that one thread. That way, you can try to break your own code by stepping one thread into a critical section of your code when it's not supposed to be there.

Thanks Stephan,
I think the debugger's stepper will be very useful. This will allow me to see exactly what is going on as the code executes.

[Geoff McKay]

Trisha Gee wrote:... Using immutable objects where possible is one way to embrace this approach....

Thanks Trisha. I will look into immutable objects. And thanks for the link to Brian Goetz's book on the topic of concurrency.

[Stephan van Hulst]

Geoff McKay wrote:This will allow me to see exactly what is going on as the code executes.

Not exactly.

The debugger hides optimizations that can trip up a multi-threaded application when it runs in production. You see, the processor is free to swap around instructions if that allows it to run faster, as long as it doesn't affect the final result according to the executing thread.

Warning: When I started writing this post, everything was still related to your original question, but I appear to have gone on a wild tangent. I can split this off to a different thread if you want.

Consider the following bit of code:

final class NotThreadSafe {

  private static final int PRINT_COUNT = 5;

  private int phase;

  void executeAllPhases() {
    phase = 1;
    phase = 2;
    phase = 3;
    phase = 4;
  }

  void printCurrentPhaseOften() {
    for (var i = 0; i < PRINT_COUNT; i++) {
      System.out.printf("We're now at phase %d%n", phase);
    }
  }

  public static void main(String... args) {
    var plan = new NotThreadSafe();

    var executor = Executors.newFixedThreadPool(2);

    try {
      executor.execute(plan::printCurrentPhaseOften);
      executor.execute(plan::executeAllPhases);
    }

    finally {
      executor.shutdown();
    }
  }
}

Now, it's unprobable this application will output anything other than:
We're now at phase 0 We're now at phase 4 We're now at phase 4 We're now at phase 4 We're now at phase 4
HOWEVER, theoretically it is possible that the print statements will perfectly interleave with the assignments to the phase field, and you can use your debugger to force this output:
We're now at phase 0 We're now at phase 1 We're now at phase 2 We're now at phase 3 We're now at phase 4
This seems obvious. What is less obvious is that the following is also a possible (if extremely unlikely) output of the application:
We're now at phase 0 We're now at phase 3 We're now at phase 1 We're now at phase 4 We're now at phase 2
You can't really force this output using your debugger, but it is still theoretically possible. "How?", you might ask.

The processor is free to swap the instructions in the executeAllPhases() method any way it wants, as long as it doesn't affect the apparent order of execution according to the thread that is performing executeAllPhases().

If we change the executeAllPhases() method like this:

  void executeAllPhases() {
    phase = 1;
    phase = 2;
    System.out.println(phase);
    phase = 3;
    phase = 4;
  }

That println() statement is *guaranteed* to print "2".

The important thing to keep in mind is that this guarantee is not made for other threads.

To the thread executing printCurrentPhaseOften(), it may appear that phase = 3 was executed before phase = 1 was executed.

In technical terms, to thread A there is a happens-before relationship between lines 9 and 10. To thread B, this happens-before relationship between lines 9 and 10 does not exist. You use so-called memory barriers to create happens-before relationships in threads other than the thread that is executing the critical section.

Object monitors are an easy way to use memory barriers:

  private final Object lock;

  private int phase;

  void executeAllPhases() {
    synchronized(lock) { phase = 1; }
    synchronized(lock) { phase = 2; }
    synchronized(lock) { phase = 3; }
    synchronized(lock) { phase = 4; }
  }

  void printCurrentPhaseOften() {
    for (var i = 0; i < PRINT_COUNT; i++) {
      synchronized(lock) {
        System.out.printf("We're now at phase %d%n", phase);
      }
    }
  }

While it's still unlikely that this application will print each phase exactly once, at least it's guaranteed that the phases will no longer be printed out of order. The happens-before relationship in thread A is synchronized with thread B every time they reach synchronized(lock).

A less verbose version can be achieve using volatile fields. The code above is functionally equivalent to:

  private volatile int phase;
 
  void executeAllPhases() {
    phase = 1;
    phase = 2;
    phase = 3;
    phase = 4;
  }
 
  void printCurrentPhaseOften() {
    for (var i = 0; i < PRINT_COUNT; i++) {
      System.out.printf("We're now at phase %d%n", phase);
    }
  }

Every time you access a volatile field in one thread, you kind of "publish" that thread's happens-before relationships to the global state. Every time another thread accesses that same volatile field, it reads the "published" relationships and then the correct execution order is also guaranteed for that thread.

The big difference between using volatile fields and the synchronized keyword (and concurrency utilities that are based upon the synchronized keyword, such as ReentrantLock) is that volatile fields do not provide atomicity. Code inside a synchronized block appears as if it is a single instruction to other threads that synchronize on the same lock.

[Campbell Ritchie]

Stephan van Hulst wrote:. . . the processor is free to swap around instructions . . . as long as it doesn't affect the final result . . . .

How do you decide what won't affect the final result? If you do that simply by having the instructions with disjoint frames, that can add to the trouble. The following three statements have disjoint frames and swapping them can cause problems.

x = new Foo(); // frame = x
x.setXYZ(123); // frame = x.xyz
y = new Bar(); // frame = y

[Geoff McKay]

Stephan van Hulst wrote:... While it's still unlikely that this application will print each phase exactly once, at least it's guaranteed that the phases will no longer be printed out of order. The happens-before relationship in thread A is synchronized with thread B every time they reach synchronized(lock)...

Thank-you very much for the detailed explanation. The code samples provided help very much. The happens-before relationship and synchronized(lock) approach would maintain the expected order and looks like a good way. Thanks!

[Himai Minh]

Hi, Stephen
I heard the happen-before relationship only happen in long or double primitive type. Can it happen to integer type like

int phase

in your previous example?

[Campbell Ritchie]

Himai Minh wrote:Hi, Stephen

Stephan.

I heard the happen-before relationship only happen in long or double primitive type. . . .

I think you have been told that incorrectly. Calling assignmnts to primitives in different threads does not establish any “happens‑before” relationship between any of the lines of code.
It is possible for half an action involving longs or doubles to be separated from its other half by another action. I don't know whether that still applies on 64‑bit hardware.

[Stephan van Hulst]

Campbell Ritchie wrote:How do you decide what won't affect the final result?

That's up to the compiler and processor, they must guarantee that any optimizations do not affect the final result.

Consider the following code:

var y = a - b + c;

Let's say that for some really silly reason, our processor hardware needs to give the ALU a short rest whenever it performs subtraction and also all operations occur directly in memory (there are no registers). A JIT compiler would be encouraged to move subtraction to the end of a sequence of arithmetic operations:
MOV a y ADD c y SUB b y
Even though Java guarantees that expressions are evaluated left-to-right, this is a valid compiler optimization because (a + c - b) gives the same result as (a - b + c). However, any other thread that prints the value of y before the expression has been completely evaluated might see the results of (a + c).

If you do that simply by having the instructions with disjoint frames, that can add to the trouble.

I'm not sure what you're referring to with "disjoint frames".

Geoff McKay wrote:The happens-before relationship and synchronized(lock) approach would maintain the expected order and looks like a good way. Thanks!

Be careful! The code I provided was just there to explain a theoretical concept. It is in no way meant to be used as an example of good production code.

Himai Minh wrote:I heard the happen-before relationship only happen in long or double primitive type.

I believe you are confusing happens-before with atomicity. As I already mentioned, these two concepts are completely separate.

If I'm not mistaken, Java guarantees that a write to a variable of type int is atomic, meaning that no thread will ever observe an int to be in a "half written" state. The same is not true for non-volatile fields of type double and long.

As Campbell pointed out, it may take two processor instructions to write a double or long, because computers with a 32-bit architecture are not able to move these around in one go. Even if your system does use a 64-bit architecture, a specific JVM implementation might still not use a single instruction to move double and long around (although this is unlikely).

Java guarantees that reads and writes of volatile fields are atomic, regardless of the underlying hardware.

In the discussion above, I used the term write and not assignment. Normal assignment to an int field counts as a single write, but compound assignment and increment and decrement might count as 3 instructions:

read variable

perform arithmetic

write variable

[Campbell Ritchie]

Stephan van Hulst wrote:. . . "disjoint frames". . .

The frame is the alphabet/“population” of variables acted on by a program; in the case of an assignment it means whatever is left of the = assignment operator. (That will all change if you use ++ or --.) By disjoint I meant that the frames of two programs have no overlap.

And have a cow for that post, Stephan