Parallel slowdown is a phenomenon in
parallel computing
Parallel computing is a type of computation in which many calculations or processes are carried out simultaneously. Large problems can often be divided into smaller ones, which can then be solved at the same time. There are several different fo ...
where parallelization of a
parallel algorithm
In computer science, a parallel algorithm, as opposed to a traditional serial algorithm, is an algorithm which can do multiple operations in a given time. It has been a tradition of computer science to describe serial algorithms in abstract machin ...
beyond a certain point causes the program to run slower (take more time to run to completion).
Parallel slowdown is typically the result of a communications bottleneck. As more
processor node
This glossary of computer hardware terms is a list of definitions of terms and concepts related to computer hardware, i.e. the physical and structural components of computers, architectural issues, and peripheral devices.
A
...
s are added, each processing node spends progressively more time doing communication than useful processing. At some point, the communications overhead created by adding another processing node surpasses the increased processing power that node provides, and parallel slowdown occurs.
Parallel slowdown occurs when the algorithm requires significant communication, particularly of intermediate results. Some problems, known as
embarrassingly parallel problem
In parallel computing, an embarrassingly parallel workload or problem (also called embarrassingly parallelizable, perfectly parallel, delightfully parallel or pleasingly parallel) is one where little or no effort is needed to separate the problem ...
s, do not require such communication, and thus are not affected by slowdown.
References
{{reflist
See also
*
Mythical man month
''The Mythical Man-Month: Essays on Software Engineering'' is a book on software engineering and project management by Fred Brooks first published in 1975, with subsequent editions in 1982 and 1995. Its central theme is that adding manpower to a ...
, an analogous situation for a team programmers where productivity is affected by human communication.
Analysis of parallel algorithms