Unit Tests for Concurrent Java with VMLens

1. Introduction

While it is common to write unit tests for single-threaded Java, unit tests for concurrent Java are still rarely used.

By using VMLens, an open source tool to deterministically unit test concurrent Java, we can now change this.

In the following tutorial, we’ll learn how to use VMLens to write unit tests for concurrent Java.

2. Setup

As an example, we implement a BankAccount class. We want to update and get the current amount from multiple threads in parallel:

public class RegularFieldBankAccount { private int amount; public void update(int delta) { amount += delta; } // standard getter } 

First of all, we need to add Maven dependencies and plugins to our pom.xml:

<dependency> <groupId>com.vmlens</groupId> <artifactId>api</artifactId> <version>1.2.10</version> <scope>test</scope> </dependency> <plugin> <groupId>com.vmlens</groupId> <artifactId>vmlens-maven-plugin</artifactId> <version>1.2.10</version> <executions> <execution> <id>test</id> <goals> <goal>test</goal> </goals> </execution> </executions> </plugin>

The vmlens-maven-plugin extends the maven-surefire-plugin. So we can configure the VMLens Plugin the same way as the Maven Surefire Plugin. We can find the latest versions of com.vmlens:api and vmlens-maven-plugin:vmlens-maven-plugin in the Maven Central repository.

We can also use VMLens with Gradle or standalone as described here.

3. The Test

To test that we can indeed update the bank account from multiple threads, we let the main and a newly started thread call the update method in parallel. We surround this with a while loop, iterating over all thread interleavings:

@Test public void whenParallelUpdate_thenAmountSumOfBothUpdates() throws InterruptedException { try (AllInterleavings allInterleavings = new AllInterleavings("bankAccount.updateUpdate")) { while (allInterleavings.hasNext()) { RegularFieldBankAccount bankAccount = new RegularFieldBankAccount(); Thread first = new Thread() { @Override public void run() { bankAccount.update(5); } }; first.start(); bankAccount.update(10); first.join(); int amount = bankAccount.getAmount(); assertThat(amount, is(15)); } } } 

The problem with testing concurrent Java is that we need to test all possible execution orders of the threads.

By using the while loop, we instruct VMLens to test all thread interleavings at the specified location in the code.

VMLens runs as a byte-code agent. VMLens traces all synchronization actions and field accesses. Based on this information, VMLens calculates all thread interleavings.

4. Data Races

Running the test leads to the following error, a data race:

A data race happens when two threads access the same field simultaneously without proper synchronization. Synchronization actions include operations such as accessing a volatile field or using a synchronized block. We observe from the trace that there are no synchronization actions between the read and write operations to the amount field from different threads.

When a data race happens, there is no guarantee that a reading thread will see the last written value. This is because the compiler reorders instructions, and CPU cores cache field values. Only with synchronization actions in between can we ensure that the thread reads the most recent value.

To write concurrent classes, we need to eliminate data races.

5. Non-atomic Methods

To fix this error, we add a volatile modifier to the field declaration:

public class VolatileFieldBankAccount { private volatile int amount; // Methods same as above } 

Running the test, we get the following error:

Expected: is <15> but: was <10> 

The VMLens trace shows why the amount was not correctly updated:

The amount += delta operation is not one atomic operation but three independent ones:

1. reading the value from the field
2. updating the value
3. writing back the new value to the field

The trace shows that first the main thread and then Thread-8 read the field. And then first Thread-8 and then the main thread writes to the field. Therefore, the update to Thread-8 gets lost, resulting in the incorrect value.

6. Atomic Methods

The problem is that the update method is not atomic. The read of the amount and the write to the amount should be one indivisible operation.

Therefore, after eliminating data races, we need to make the methods atomic. We can do this by using a synchronized block in the update method:

public class AtomicBankAccount { private final Object LOCK = new Object(); private volatile int amount; public int getAmount() { return amount; } public void update(int delta) { synchronized (LOCK) { amount += delta; } } } 

This class now passes the test.

7. How Are Unit Tests for Concurrent Java Possible?

According to Unit Testing: Principles, Practices, and Patterns by Vladimir Khorikov,

A unit test is an automated test that

1. Verifies a small piece of code (also known as a unit),
2. Does it quickly,
3. And does it in an isolated manner.

The problem with testing concurrent Java is that we need to test all thread interleavings. And that the number of thread interleavings grows exponentially with the number of conflicting synchronization actions. This means that unit tests are a good fit for testing concurrent Java.

Unit tests are fast. This allows for repeated use multiple times. That the unit test verifies only a small piece of code makes it possible to treat the other part of the code as a black box. This reduces the number of thread interleavings that we need to test.

8. What to Test?

We need to test that the methods of our class are atomic. To test this, we need to execute all updating methods in parallel. And all updating and all reading methods in parallel.

The best way is to write a separate test for each combination of updating and reading methods.

So, for our example, we still need a test for the combination of reading and updating:

@Test public void whenParallelUpdateAndGet_thenResultEitherAmountBeforeOrAfterUpdate() throws InterruptedException { try (AllInterleavings allInterleavings = new AllInterleavings("bankAccount.updateGetAmount")) { while (allInterleavings.hasNext()) { RegularFieldBankAccount bankAccount = new RegularFieldBankAccount(); Thread first = new Thread() { @Override public void run() { bankAccount.update(5); } }; first.start(); int amount = bankAccount.getAmount(); assertThat(amount, anyOf(is(0), is(5))); first.join(); } } } 

As the method getAmount() is either executed before or after the update, the amount can be either 0, the value before the update, or 5, the value after the update.

9. Conclusion

In this article, we described the functionality of the VMLens.

To test a concurrent class, we need to test if the methods of the class are atomic and do not contain data races. We do this by writing a test for each combination of updating and reading methods. In the test, we call the methods in parallel and iterate over all thread interleavings using VMLens.