Two keys to effective multithreaded programs

In my last post, I went over a mostly lock-free producer/consumer queue that worked entirely off the ThreadPool. This covers the two most important aspects to effective multithreaded programming: avoid creating new Threads, and work lock-free where possible.

Avoiding the creation of new threads is easy- you just need to schedule tasks on the ThreadPool (though QueueUserWorkItem, or System.Threading.Tasks usage), instead of using Thread.Start or new Thread. The goal is, you want the ThreadPool to manage threads for optimal performance, you don’t want to create other Threads that are going to push the ThreadPool threads off the CPU, and are going to need to be created/destroyed. A long-lived thread is usually a waste because it is inactive for much of the time. A short-lived thread is a waste because allocating and destroying a thread is an expensive operation! So just avoid creating threads- you almost never have to if you design your systems properly.

Lock-free programming is much more difficult. The reason lock-free programming is important (in this case) is, you don’t want to block ThreadPool threads. If a ThreadPool thread is blocked for a while, the ThreadPool will create a new thread- which means we’re basically creating new Threads as above, which is bad! Worst case scenario: a Parallel.ForEach loop that involves the launching of a Process/WaitForExit that blocks the thread. You’ll end up with almost every iteration of the ForEach loop having launched its own process, so you’ll have n threads in your main program, with n processes running, with their own threads. You will have a huge mess and everything will be context switching like crazy and performance across the board will suffer!

So this all basically requires breaking sequential work into discrete chunks, and linking them together- and actually this is exactly what Task Based Parallelism is, and you can do it effectively with System.Threading.Tasks in .NET 4.0 (or 3.5 with Reactive Extensions’ System.Threading.dll), and with .NET 5.0’s async and await keywords.  However, these are necessarily high-level concepts and systems, and you cannot just throw code into these systems and expect them to work correctly or predictably.  Managing threads effectively, as described here, is the first (and easiest) component of writing effective asynchronous/multithreaded systems.  The more difficult part is writing systems that can function correctly and predictably in a asynchronous environment.  I’ve found, though, that thinking about the (easier) thread management aspect can help inform the (difficult) systems design component.