Y.A.T.L.
A Lock-Free C++ 23 threadpool, with support for Coroutines, and Functors. A pluggable scheduling strategy through template metaprogramming, and a custom domain-specific lock-free sized thread-caching allocator
For use in my game engine, Aurora, I wrote a fast and extensible C++23 lock free job system to make efficient use of the different cores of modern processors. The Engine makes use of both Coroutines and Functors, each for different situations. Coroutines are used for IO operations, such as asset loading/uploading, while functors are used for things such as fan-out operations, as they generally have less overhead in that use-case.
It has support for executing Coroutines, and Functors. Fibers can be easily added (an example of that can be seen in the blog), without requiring any changes to the threadpool. You can easily change which scheduling algorithm it uses -to best suit your usage- without any runtime dispatch. It by default makes use of a custom lock-free sized thread-caching allocator per worker, and a work stealing scheduler based on the "Efficient Task Dependency Graph Work Stealing" paper.
It has support for Fork/Join parallelism, both Functors and Coroutines, the Functor version of which allocates tasks on the stack. Using a divide-and-conquer approach, an allocation-free parallel for and parallel for-each implementation is available.
std::array values = {100, 200, 300, 400};
ytl::ThreadPool pool;
ytl::parallel::for_each(pool, values, [](int value) {});
// WorkStealing is the default value, so this is equivalent to omitting the template parameter.
ytl::ThreadPool<ytl::WorkStealing> pool;
If you want to add a custom scheduler, you can simply make a class/struct that satisfies the following concept, and enter it into the Scheduler template parameter in the ytl::ThreadPool. By default it uses the work stealing scheduler, which is generally the recommended one.
template <typename S>
concept Scheduler = requires(S &sched, task::TaskRef task) {
sched.init(size_t{});
{ sched.schedule(task) } -> std::same_as<bool>;
{ sched.next() } -> std::same_as<std::optional<task::TaskRef>>;
{ sched.try_next() } -> std::same_as<std::optional<task::TaskRef>>;
{ sched.has_ready() } -> std::same_as<bool>;
sched.shutdown();
{ sched.allocator() } -> std::same_as<IAllocator *>;
};The WorkStealing strategy has a custom lock-free allocator for each worker thread, where local operations don't require any synchronization. Multi-threaded frees -when a task was stolen- are appended to a lock-free linked list, which the worker thread can claim in a single cas operation- it does so when the local list has run out of allocations. There are multiple size tiers, based on powers of two, which does potentially have more fragmentation, but is very fast to calculate and doesn't require a header to know which size class the allocation belongs to, you can simply round it to the nearest power of two. Each worker has their own bounded task channel, when overflow occurs, it the excess will get sent to the global channel instead.