A simple introduction to coroutines, threads and concurrency issues

"Coroutines are lightweight threads", I believe everyone has heard this statement more than once. But do you really understand what it means? I'm afraid the answer is no. The following content will tell you how coroutines are run in the Android runtime, what is the relationship between them and threads, and the concurrency problems encountered when using the Java programming language thread model.

[[403725]]

Coroutines and Threads

Coroutines are designed to simplify asynchronously executed code. For coroutines in the Android runtime, the code block of the lambda expression is executed in a dedicated thread. For example, the Fibonacci calculation in the example:

 // Calculate the tenth-level Fibonacci number in the background thread
 someScope.launch( Dispatchers.Default ) {
    val fibonacci10 = synchronousFibonacci(10)
    saveFibonacciInMemory(10, fibonacci10)
 } 
 
 private fun synchronousFibonacci(n: Long): Long { /* ... */ }

The async coroutine code block above will be distributed to the thread pool managed by the coroutine library for execution, implementing synchronous and blocking Fibonacci numerical operations and storing the results in memory. The thread pool in the above example belongs to Dispatchers.Default. The code block will be executed in a thread in the thread pool at some point in the future, and the specific execution time depends on the thread pool strategy.

Note that since the above code does not contain suspending operations, it will be executed in the same thread. It is possible to execute coroutines in different threads, for example by moving the execution to a different dispatcher, or by including code with suspending operations in a dispatcher that uses a thread pool.

If you don't use coroutines, you can also use threads to implement similar logic yourself. The code is as follows:

 // Create a thread pool containing 4 threads
 val executorService = Executors.newFixedThreadPool(4) 
  
 // Schedule and execute code in one of the threads
 executorService.execute {
    val fibonacci10 = synchronousFibonacci(10)
    saveFibonacciInMemory(10, fibonacci10)
 }

Although you can manage the thread pool yourself, we still recommend using coroutines as the preferred asynchronous implementation in Android development. It has a built-in cancellation mechanism, provides more convenient exception capture and structured concurrency, which can reduce the occurrence of similar memory leaks and is more integrated with the Jetpack library.

How it works

What happens between the time you create a coroutine and the time the code is executed by a thread? When you create a coroutine using the standard coroutine builder, you can specify the CoroutineDispatcher that the coroutine runs on. If not specified, the system uses Dispatchers.Default by default.

CoroutineDispatcher is responsible for assigning the execution of coroutines to specific threads. At the bottom level, when CoroutineDispatcher is called, it calls the interceptContinuation method that encapsulates the Continuation (such as the coroutine here) to intercept the coroutine. This process is based on the premise that CoroutineDispatcher implements the CoroutineInterceptor interface.

If you read my previous article about how coroutines are implemented under the hood, you should already know that the compiler creates a state machine, and that information about the state machine (such as what operations to perform next) is stored in a Continuation object.

Once the Continuation object needs to be executed in another Dispatcher, the resumeWith method of DispatchedContinuation will be responsible for distributing the coroutine to the appropriate Dispatcher.

In addition, in the implementation of the Java programming language, DispatchedContinuation, which inherits from the DispatchedTask abstract class, is also an implementation type of the Runnable interface. Therefore, the DispatchedContinuation object can also be executed in a thread. The benefit is that when CoroutineDispatcher is specified, the coroutine will be converted to a DispatchedTask and executed in the thread as a Runnable.

So when you create a coroutine, how is the dispatch method called? When you create a coroutine using the standard coroutine builder, you can specify the startup parameter, which is of type CoroutineStart. For example, you can set the coroutine to start only when needed, and then set the parameter to CoroutineStart.LAZY. By default, the system uses CoroutineStart.DEFAULT to schedule the execution according to the CoroutineDispatcher.

Dispatchers and thread pools

You can use the Executor.asCoroutineDispatcher() extension function to convert a coroutine to a CoroutineDispatcher, and then execute the coroutine in any thread pool in your application. In addition, you can also use the default Dispatchers of the coroutine library.

You can see how Dispatchers.Default is initialized in the createDefaultDispatcher method. By default, the system uses the DefaultScheduler. If you look at the implementation code of Dispatcher.IO, it also uses the DefaultScheduler and supports the creation of at least 64 threads on demand. Dispatchers.Default and Dispatchers.IO are implicitly associated because they use the same thread pool, which leads to our next topic, what runtime overhead will be incurred by calling withContext with different dispatchers?

Performance of threads and withContext

In the Android runtime, if there are more threads running than available CPU cores, switching threads will incur some runtime overhead. Context switching is not easy! The OS needs to save and restore the execution context, and the CPU needs to spend time scheduling threads in addition to executing the actual application functions. In addition, context switching can also occur when the code running in a thread is blocked. If the above questions are specific to threads, what performance penalties will be incurred by using withContext in different Dispatchers?

Fortunately, the thread pool will help us solve these complex operations. It will try to perform as many tasks as possible (this is why performing operations in the thread pool is better than manually creating threads). Coroutines also benefit from being scheduled to execute in the thread pool. Based on this, coroutines do not block threads, but instead suspend their own work, making them more efficient.

The default thread pool used in the Java programming language is CoroutineScheduler. It dispatches coroutines to worker threads in the most efficient way. Since Dispatchers.Default and Dispatchers.IO use the same thread pool, switching between them will try to avoid thread switching. The coroutine library will optimize these switching calls, keep them on the same dispatcher and thread, and take shortcuts as much as possible.

Since Dispatchers.Main usually belongs to different threads in applications with UI, switching between Dispatchers.Default and Dispatchers.Main in a coroutine does not bring much performance loss, because the coroutine will be suspended (for example, stop executing in a certain thread) and then be scheduled to continue executing in another thread.

Concurrency issues in coroutines

Coroutines do make asynchronous programming easier because they can easily schedule operations on different threads. But on the other hand, convenience is a double-edged sword: because coroutines run on the thread model of the Java programming language, they can hardly escape the concurrency problems brought about by the thread model. Therefore, you need to pay attention to and try to avoid this problem.

In recent years, strategies like immutability have relatively alleviated the problems caused by threads. However, in some scenarios, immutability strategies cannot completely avoid problems. The source of all concurrency problems is state management! Especially accessing mutable state in a multi-threaded environment.

In a multithreaded application, the order in which operations are executed is unpredictable. Unlike compiler optimizations, threads are not guaranteed to execute in a specific order, and context switches can occur at any time. If necessary precautions are not taken when accessing mutable state, threads may access outdated data, lose updates, or encounter resource contention issues, etc.

Please note that the mutable state and access order discussed here are not limited to the Java programming language. They also affect coroutine execution on other platforms.

Applications that use coroutines are essentially multi-threaded applications. Classes that use coroutines and involve mutable states must take measures to make them controllable, such as ensuring that the data accessed by the code in the coroutine is the latest. In this way, different threads will not interfere with each other. Concurrency issues will cause potential bugs, making it difficult for you to debug and locate problems in your application, and even Heisenberg bugs may occur.

This type of class is very common. For example, the class needs to cache the user's login information in memory, or cache some values ​​when the application is active. If you are not careful, concurrency issues will take advantage of it! Suspending functions using withContext(defaultDispatcher) are not guaranteed to be executed in the same thread.

For example, we have a class that needs to cache transactions made by users. If the cache is not accessed correctly, as shown in the following code, concurrency issues will arise:

 class TransactionsRepository(
  private val defaultDispatcher: CoroutineDispatcher = Dispatchers.Default  
 ) { 
 
  private val transactionsCache = mutableMapOf< User , List< Transaction >() 
 
  private suspend fun addTransaction( user : User , transaction : Transaction ) =
 // NOTE! Access to the cache is not protected!
 // Concurrency issues will occur: threads will access expired data
 // And there is a resource competition problem
    withContext(defaultDispatcher) {
      if (transactionsCache. contains ( user )) {
        val oldList = transactionsCache[ user ]
        val newList = oldList!!.toMutableList()
        newList.add ( transaction )
        transactionsCache.put( user , newList)
 } else {
        transactionsCache.put( user , listOf( transaction ))
 }
 }
 }

Even though we are talking about Kotlin here, the book Java Concurrency in Action by Brian Goetz is a great reference for understanding the subject of this article and the Java programming language system. In addition, Jetbrains also provides relevant documentation on the topics of shared mutable state and concurrency.

Protecting Mutable State

How to protect mutable state or find a suitable synchronization strategy depends on the data itself and the related operations. This section inspires you to pay attention to the concurrency problems you may encounter, rather than simply listing the methods and APIs for protecting mutable state. In short, here are some tips and APIs to help you achieve thread safety for mutable variables.

Encapsulation

Mutable state should belong to and be encapsulated in a class. The class should centralize access operations to the state and use synchronization strategies to protect variable access and modification operations based on the application scenario.

Thread Limitation

One solution is to restrict read and write operations to a single thread. You can use queues to implement access to mutable state based on the producer-consumer pattern. Jetbrains has great documentation on this.

Avoid duplication of effort

The Android runtime includes thread-safe data structures that you can use to protect mutable variables. For example, in the counter example, you can use AtomicInteger. Another example, to protect the Map in the above code, you can use ConcurrentHashMap. ConcurrentHashMap is thread-safe and optimizes the throughput of map read and write operations.

Please note that thread-safe data structures do not solve the call order problem, they just ensure that memory data access is atomic. When the logic is not too complex, they can avoid the use of locks. For example, they cannot be used in the transactionCache example above, because the order of operations and logic between them require the use of threads and access protection.

Furthermore, when modified objects are stored in these thread-safe data structures, the data in them needs to remain immutable or protected to avoid resource contention issues.

Custom solutions

If you have complex operations that need to be synchronized, @Volatile and thread-safe data structures will not be effective. It is also possible that the built-in @Synchronized annotation is not granular enough to achieve the desired effect.

In these cases, you may need to use concurrency tools to create your own synchronization mechanisms, such as latches, semaphores, or barriers. In other scenarios, you can use locks and mutexes to unconditionally protect multithreaded access.

Mute in Kotlin includes suspend functions lock and unlock, which can manually control the code of the protected coroutine. The extension function Mutex.withLock makes it easier to use:

 class TransactionsRepository(
  private val defaultDispatcher: CoroutineDispatcher = Dispatchers.Default  
 ) {
 // Mutex protects the mutable state cache
  private val cacheMutex = Mutex()
  private val transactionsCache = mutableMapOf< User , List< Transaction >() 
 
  private suspend fun addTransaction( user : User , transaction : Transaction ) =
    withContext(defaultDispatcher) {
 // Mutex ensures thread safety of read and write cache
 cacheMutex.withLock {
        if (transactionsCache. contains ( user )) {
          val oldList = transactionsCache[ user ]
          val newList = oldList!!.toMutableList()
          newList.add ( transaction )
          transactionsCache.put( user , newList)
 } else {
          transactionsCache.put( user , listOf( transaction ))
 }
 }
 }
 }

Because a coroutine using a mutex suspends operations before it can continue, it is much more efficient than a lock in the Java programming language, which blocks the entire thread. Be careful when using Java language synchronization classes in coroutines, because they block the entire thread in which the coroutine resides and cause liveness issues.

The code passed into the coroutine will eventually be executed in one or more threads. Similarly, coroutines still need to follow constraints under the thread model of the Android runtime. Therefore, using coroutines will also result in multi-threaded code with hidden dangers. Therefore, please be careful when accessing shared mutable state in your code.