Is there a difference between ThreadPoolExecutor from concurrent.futures and ThreadPool from multiprocessing.dummy? This article suggests that ThreadPool queries the "threads" (task) to the different "threads" of the CPU. Does concurrent.futures do the same or will the "threads" (tasks) query to a single "thread" of a CPU?
1 Answer
The multiprocessing.dummy.ThreadPool is a copy of the multiprocessing.Pool API which uses threads rather than processes, which leads to some weirdness since thread and processes are very different, including returning a AsyncResult type which only it understands.
The concurrent.futures.ThreadPoolExecutor is a subclass concurrent.futures.Executor which is a newer simpler API, developped with both processes and threads in mind and so returns a common concurrent.futures.Future.
On a very broad and far look, both do the same but concurrent.futures.ThreadPoolExecutor does it better.
References:
From the multiprocessing.dummy documentation:
multiprocessing.dummyreplicates the API of multiprocessing but is no more than a wrapper around thethreadingmodule.In particular, the
Poolfunction provided bymultiprocessing.dummyreturns an instance ofThreadPool, which is a subclass ofPoolthat supports all the same method calls but uses a pool of worker threads rather than worker processes.
From the multiprocessing.dummy.ThreadPool documentation
A
ThreadPoolshares the same interface asPool, which is designed around a pool of processes and predates the introduction of theconcurrent.futuresmodule. As such, it inherits some operations that don’t make sense for a pool backed by threads, and it has its own type for representing the status of asynchronous jobs,AsyncResult, that is not understood by any other libraries.Users should generally prefer to use
concurrent.futures.ThreadPoolExecutor, which has a simpler interface that was designed around threads from the start, and which returnsconcurrent.futures.Futureinstances that are compatible with many other libraries, includingasyncio.