HOME

TheInfoList



OR:

In
computer programming Computer programming or coding is the composition of sequences of instructions, called computer program, programs, that computers can follow to perform tasks. It involves designing and implementing algorithms, step-by-step specifications of proc ...
, a thread pool is a
software design pattern In software engineering, a software design pattern or design pattern is a general, reusable solution to a commonly occurring problem in many contexts in software design. A design pattern is not a rigid structure to be transplanted directly into s ...
for achieving concurrency of execution in a computer program. Often also called a replicated workers or worker-crew model, a thread pool maintains multiple threads waiting for tasks to be allocated for concurrent execution by the supervising program. By maintaining a pool of threads, the model increases performance and avoids latency in execution due to frequent creation and destruction of threads for short-lived tasks. Another good property - the ability to limit system load, when we use fewer threads than available. The number of available threads is tuned to the computing resources available to the program, such as a parallel task queue after completion of execution.


Performance

The size of a thread pool is the number of threads kept in reserve for executing tasks. It is usually a tunable parameter of the application, adjusted to optimize program performance. Deciding the optimal thread pool size is crucial to optimize performance. One benefit of a thread pool over creating a new thread for each task is that thread creation and destruction overhead is restricted to the initial creation of the pool, which may result in better
performance A performance is an act or process of staging or presenting a play, concert, or other form of entertainment. It is also defined as the action or process of carrying out or accomplishing an action, task, or function. Performance has evolved glo ...
and better system
stability Stability may refer to: Mathematics *Stability theory, the study of the stability of solutions to differential equations and dynamical systems ** Asymptotic stability ** Exponential stability ** Linear stability **Lyapunov stability ** Marginal s ...
. Creating and destroying a thread and its associated resources can be an expensive process in terms of time. An excessive number of threads in reserve, however, wastes memory, and context-switching between the runnable threads invokes performance penalties. A socket connection to another network host, which might take many CPU cycles to drop and re-establish, can be maintained more efficiently by associating it with a thread that lives over the course of more than one network transaction. Using a thread pool may be useful even putting aside thread startup time. There are implementations of thread pools that make it trivial to queue up work, control concurrency and sync threads at a higher level than can be done easily when manually managing threads. In these cases the performance benefits of use may be secondary. Typically, a thread pool executes on a single computer. However, thread pools are conceptually related to
server farm A server farm or server cluster is a collection of Server (computing), computer servers, usually maintained by an organization to supply server functionality far beyond the capability of a single machine. They often consist of thousands of compu ...
s in which a master process, which might be a thread pool itself, distributes tasks to worker processes on different computers, in order to increase the overall throughput.
Embarrassingly parallel In parallel computing, an embarrassingly parallel workload or problem (also called embarrassingly parallelizable, perfectly parallel, delightfully parallel or pleasingly parallel) is one where little or no effort is needed to split the problem into ...
problems are highly amenable to this approach. The number of threads may be dynamically adjusted during the lifetime of an application based on the number of waiting tasks. For example, a
web server A web server is computer software and underlying Computer hardware, hardware that accepts requests via Hypertext Transfer Protocol, HTTP (the network protocol created to distribute web content) or its secure variant HTTPS. A user agent, co ...
can add threads if numerous
web page A web page (or webpage) is a World Wide Web, Web document that is accessed in a web browser. A website typically consists of many web pages hyperlink, linked together under a common domain name. The term "web page" is therefore a metaphor of pap ...
requests come in and can remove threads when those requests taper down. The cost of having a larger thread pool is increased resource usage. The algorithm used to determine when to create or destroy threads affects the overall performance: * Creating too many threads wastes resources and costs time creating the unused threads. * Destroying too many threads requires more time later when creating them again. * Creating threads too slowly might result in poor client performance (long wait times). * Destroying threads too slowly may starve other processes of resources.


In languages

In bash implemented by --max-procs / -P in xargs, for example: # Fetch 5 URLs in parallel urls=( "https://example.com/file1.txt" "https://example.com/file2.txt" "https://example.com/file3.txt" "https://example.com/file4.txt" "https://example.com/file5.txt" ) printf '%s\n' "$" , xargs -P 5 -I curl -sI , grep -i "content-length:" In Go, called worker pool: package main import ( "fmt" "time" ) func worker(id int, jobs <-chan int, results chan<- int) func main() It will print: $ time go run worker-pools.go worker 1 started job 1 worker 2 started job 2 worker 3 started job 3 worker 1 finished job 1 worker 1 started job 4 worker 2 finished job 2 worker 2 started job 5 worker 3 finished job 3 worker 1 finished job 4 worker 2 finished job 5 real 0m2.358s


See also

* Asynchrony (computer programming) * Object pool pattern *
Concurrency pattern In software engineering, concurrency patterns are those types of design patterns that deal with the multi-threaded programming paradigm. Examples of this class of patterns include: * Active object * Balking pattern * Barrier * Double-check ...
*
Grand Central Dispatch Grand Central Dispatch (GCD or libdispatch) is a technology developed by Apple Inc. to optimize application support for systems with multi-core processors and other symmetric multiprocessing systems. It is an implementation of task parallelism ...
* Parallel Extensions (.NET) *
Parallelization Parallel computing is a type of computation in which many calculations or processes are carried out simultaneously. Large problems can often be divided into smaller ones, which can then be solved at the same time. There are several different for ...
*
Server farm A server farm or server cluster is a collection of Server (computing), computer servers, usually maintained by an organization to supply server functionality far beyond the capability of a single machine. They often consist of thousands of compu ...
* Staged event-driven architecture


References


External links

*
Query by Slice, Parallel Execute, and Join: A Thread Pool Pattern in Java
by Binildas C. A. *
Thread pools and work queues
by Brian Goetz *
A Method of Worker Thread Pooling
by Pradeep Kumar Sahu *
Work Queue
by Uri Twig: C++ code demonstration of pooled threads executing a work queue. *
Windows Thread Pooling and Execution Chaining
*
Smart Thread Pool
by Ami Bar *
Programming the Thread Pool in the .NET Framework
by David Carmona *

by Amir Kirsh *
Practical Threaded Programming with Python: Thread Pools and Queues
by Noah Gift *
Optimizing Thread-Pool Strategies for Real-Time CORBA
by Irfan Pyarali, Marina Spivak, Douglas C. Schmidt and Ron Cytron *
Deferred cancellation. A behavioral pattern
by Philipp Bachmann *
A C++17 Thread Pool for High-Performance Scientific Computing
by Barak Shoshany {{Design Patterns patterns Threads (computing) Software design patterns Concurrent computing Parallel computing