Multiprocessing is the use of two or more
central processing units
A central processing unit (CPU), also called a central processor, main processor or just processor, is the electronic circuitry that executes instructions comprising a computer program. The CPU performs basic arithmetic, logic, controlling, an ...
(CPUs) within a single
computer system.
The term also refers to the ability of a system to support more than one processor or the ability to allocate tasks between them. There are many variations on this basic theme, and the definition of multiprocessing can vary with context, mostly as a function of how CPUs are defined (
multiple cores on one
die
Die, as a verb, refers to death, the cessation of life.
Die may also refer to:
Games
* Die, singular of dice, small throwable objects used for producing random numbers
Manufacturing
* Die (integrated circuit), a rectangular piece of a semicondu ...
, multiple dies in one
package, multiple packages in one
system unit
A system is a group of interacting or interrelated elements that act according to a set of rules to form a unified whole. A system, surrounded and influenced by its environment, is described by its boundaries, structure and purpose and expresse ...
, etc.).
According to some on-line dictionaries, a multiprocessor is a computer system having two or more
processing units (multiple processors) each sharing
main memory
Computer data storage is a technology consisting of computer components and recording media that are used to retain digital data. It is a core function and fundamental component of computers.
The central processing unit (CPU) of a comput ...
and peripherals, in order to simultaneously process programs. A 2009 textbook defined multiprocessor system similarly, but noting that the processors may share "some or all of the system’s memory and I/O facilities"; it also gave tightly coupled system as a synonymous term.
At the
operating system
An operating system (OS) is system software that manages computer hardware, software resources, and provides common daemon (computing), services for computer programs.
Time-sharing operating systems scheduler (computing), schedule tasks for ef ...
level, ''multiprocessing'' is sometimes used to refer to the execution of multiple concurrent
processes
A process is a series or set of activities that interact to produce a result; it may occur once-only or be recurrent or periodic.
Things called a process include:
Business and management
*Business process, activities that produce a specific se ...
in a system, with each process running on a separate CPU or core, as opposed to a single process at any one instant.
When used with this definition, multiprocessing is sometimes contrasted with
multitasking, which may use just a single processor but switch it in time slices between tasks (i.e. a
time-sharing system). Multiprocessing however means true parallel execution of multiple processes using more than one processor.
Multiprocessing doesn't necessarily mean that a single process or task uses more than one processor simultaneously; the term
parallel processing is generally used to denote that scenario.
Other authors prefer to refer to the operating system techniques as
multiprogramming and reserve the term ''multiprocessing'' for the hardware aspect of having more than one processor.
The remainder of this article discusses multiprocessing only in this hardware sense.
In
Flynn's taxonomy, multiprocessors as defined above are
MIMD
In computing, multiple instruction, multiple data (MIMD) is a technique employed to achieve parallelism. Machines using MIMD have a number of processors that function asynchronously and independently. At any time, different processors may be exe ...
machines.
As the term "multiprocessor" normally refers to tightly coupled systems in which all processors share memory, multiprocessors are not the entire class of MIMD machines, which also contains
message passing
In computer science, message passing is a technique for invoking behavior (i.e., running a program) on a computer. The invoking program sends a message to a process (which may be an actor or object) and relies on that process and its supporting ...
multicomputer systems.
Pre-history
Possibly the first expression of the idea of multiprocessing was written by
Luigi Federico Menabrea in 1842, about
Charles Babbage
Charles Babbage (; 26 December 1791 – 18 October 1871) was an English polymath. A mathematician, philosopher, inventor and mechanical engineer, Babbage originated the concept of a digital programmable computer.
Babbage is considered ...
's
analytical engine
The Analytical Engine was a proposed mechanical general-purpose computer designed by English mathematician and computer pioneer Charles Babbage. It was first described in 1837 as the successor to Babbage's difference engine, which was a desig ...
(as translated by
Ada Lovelace): "the machine can be brought into play so as to give several results at the same time, which will greatly abridge the whole amount of the processes."
Key topics
Processor symmetry
In a multiprocessing system, all CPUs may be equal, or some may be reserved for special purposes. A combination of hardware and
operating system
An operating system (OS) is system software that manages computer hardware, software resources, and provides common daemon (computing), services for computer programs.
Time-sharing operating systems scheduler (computing), schedule tasks for ef ...
software design considerations determine the symmetry (or lack thereof) in a given system. For example, hardware or software considerations may require that only one particular CPU respond to all hardware interrupts, whereas all other work in the system may be distributed equally among CPUs; or execution of kernel-mode code may be restricted to only one particular CPU, whereas user-mode code may be executed in any combination of processors. Multiprocessing systems are often easier to design if such restrictions are imposed, but they tend to be less efficient than systems in which all CPUs are utilized.
Systems that treat all CPUs equally are called
symmetric multiprocessing
Symmetric multiprocessing or shared-memory multiprocessing (SMP) involves a multiprocessor computer hardware and software architecture where two or more identical processors are connected to a single, shared main memory, have full access to all ...
(SMP) systems. In systems where all CPUs are not equal, system resources may be divided in a number of ways, including
asymmetric multiprocessing (ASMP),
non-uniform memory access (NUMA) multiprocessing, and
clustered multiprocessing.
Master/slave multiprocessor system
In a master/slave multiprocessor system, the master CPU is in control of the computer and the slave CPU(s) performs assigned tasks. The CPUs can be completely different in terms of speed and architecture. Some (or all) of the CPUs can have share common bus, each can also have a private bus (for private resources), or they may be isolated except for a common communications pathway. Likewise, the CPUs can share common RAM and/or have private RAM that the other processor(s) cannot access. The roles of master and slave can change from one CPU to another.
An early example of a master/slave multiprocessor system is the Tandy/Radio Shack
TRS-80 Model 16
The TRS-80 Model II is a computer system launched by Tandy in October 1979, and targeted at the small-business market. It is not an upgrade of the original TRS-80 Model I, but a new system.
The Model II was succeeded by the compatible TRS-80 Mo ...
desktop computer which came out in February 1982 and ran the multi-user/multi-tasking
Xenix
Xenix is a discontinued version of the Unix operating system for various microcomputer platforms, licensed by Microsoft from AT&T Corporation in the late 1970s. The Santa Cruz Operation (SCO) later acquired exclusive rights to the software, ...
operating system, Microsoft's version of UNIX (called TRS-XENIX). The Model 16 has 3 microprocessors, an 8-bit
Zilog Z80 CPU running at 4MHz, a 16-bit
Motorola 68000
The Motorola 68000 (sometimes shortened to Motorola 68k or m68k and usually pronounced "sixty-eight-thousand") is a 16/32-bit complex instruction set computer (CISC) microprocessor, introduced in 1979 by Motorola Semiconductor Products Secto ...
CPU running at 6MHz and an
Intel 8021
The MCS-48 microcontroller series, Intel's first microcontroller, was originally released in 1976. Its first members were 8048, 8035 and 8748. The 8048 is probably the most prominent member of the family. Initially, this family was produced ...
in the keyboard. When the system was booted, the Z-80 was the master and the Xenix boot process initialized the slave 68000, and then transferred control to the 68000, whereupon the CPUs changed roles and the Z-80 became a slave processor that was responsible for all I/O operations including disk, communications, printer and network, as well as the keyboard and integrated monitor, while the operating system and applications ran on the 68000 CPU. The Z-80 could be used to do other tasks.
The earlier
TRS-80 Model II, which was released in 1979, could also be considered a multiprocessor system as it had both a Z-80 CPU and an Intel 8021
microprocessor in the keyboard. The 8021 made the Model II the first desktop computer system with a separate detachable lightweight keyboard connected with by a single thin flexible wire, and likely the first keyboard to use a dedicated microprocessor, both attributes that would later be copied years later by Apple and IBM.
Instruction and data streams
In multiprocessing, the processors can be used to execute a single sequence of instructions in multiple contexts (
single instruction, multiple data
Single instruction, multiple data (SIMD) is a type of parallel processing in Flynn's taxonomy. SIMD can be internal (part of the hardware design) and it can be directly accessible through an instruction set architecture (ISA), but it should ...
or SIMD, often used in
vector processing), multiple sequences of instructions in a single context (
multiple instruction, single data or MISD, used for
redundancy in fail-safe systems and sometimes applied to describe
pipelined processors or
hyper-threading), or multiple sequences of instructions in multiple contexts (
multiple instruction, multiple data or MIMD).
Processor coupling
Tightly coupled multiprocessor system
Tightly coupled multiprocessor systems contain multiple CPUs that are connected at the bus level. These CPUs may have access to a central shared memory (SMP or
UMA), or may participate in a memory hierarchy with both local and shared memory (SM)(
NUMA
Nuclear mitotic apparatus protein 1 is a protein that in humans is encoded by the ''NUMA1'' gene.
Interactions
Nuclear mitotic apparatus protein 1 has been shown to interact with PIM1, Band 4.1, GPSM2
G-protein-signaling modulator 2, also ca ...
). The
IBM p690 Regatta is an example of a high end SMP system.
Intel
Intel Corporation is an American multinational corporation and technology company headquartered in Santa Clara, California. It is the world's largest semiconductor chip manufacturer by revenue, and is one of the developers of the x86 ser ...
Xeon
Xeon ( ) is a brand of x86 microprocessors designed, manufactured, and marketed by Intel, targeted at the non-consumer workstation, server, and embedded system markets. It was introduced in June 1998. Xeon processors are based on the same ar ...
processors dominated the multiprocessor market for business PCs and were the only major x86 option until the release of
AMD's
Opteron range of processors in 2004. Both ranges of processors had their own onboard cache but provided access to shared memory; the Xeon processors via a common pipe and the Opteron processors via independent pathways to the system
RAM.
Chip multiprocessors, also known as
multi-core
A multi-core processor is a microprocessor on a single integrated circuit with two or more separate processing units, called cores, each of which reads and executes program instructions. The instructions are ordinary CPU instructions (such ...
computing, involves more than one processor placed on a single chip and can be thought of the most extreme form of tightly coupled multiprocessing. Mainframe systems with multiple processors are often tightly coupled.
Loosely coupled multiprocessor system
Loosely coupled multiprocessor systems (often referred to as
clusters) are based on multiple standalone relatively low processor count
commodity computers interconnected via a high speed communication system (
Gigabit Ethernet
In computer networking, Gigabit Ethernet (GbE or 1 GigE) is the term applied to transmitting Ethernet frames at a rate of a gigabit per second. The most popular variant, 1000BASE-T, is defined by the IEEE 802.3ab standard. It came into use ...
is common). A Linux
Beowulf cluster
A Beowulf cluster is a computer cluster of what are normally identical, commodity-grade computers networked into a small local area network with libraries and programs installed which allow processing to be shared among them. The result is a hig ...
is an example of a
loosely coupled system.
Tightly coupled systems perform better and are physically smaller than loosely coupled systems, but have historically required greater initial investments and may
depreciate rapidly; nodes in a loosely coupled system are usually inexpensive commodity computers and can be recycled as independent machines upon retirement from the cluster.
Power consumption is also a consideration. Tightly coupled systems tend to be much more energy-efficient than clusters. This is because a considerable reduction in power consumption can be realized by designing components to work together from the beginning in tightly coupled systems, whereas loosely coupled systems use components that were not necessarily intended specifically for use in such systems.
Loosely coupled systems have the ability to run different operating systems or OS versions on different systems.
See also
*
Multiprocessor system architecture
*
Symmetric multiprocessing
Symmetric multiprocessing or shared-memory multiprocessing (SMP) involves a multiprocessor computer hardware and software architecture where two or more identical processors are connected to a single, shared main memory, have full access to all ...
*
Asymmetric multiprocessing
*
Multi-core processor
A multi-core processor is a microprocessor on a single integrated circuit with two or more separate processing units, called cores, each of which reads and executes program instructions. The instructions are ordinary CPU instructions (such ...
*
BMDFM
Binary Modular Dataflow Machine (BMDFM) is a software package that enables running an application in parallel on shared memory symmetric multiprocessing (SMP) computers using the multiple processors to speed up the execution of single applicatio ...
– Binary Modular Dataflow Machine, a SMP MIMD runtime environment
*
Software lockout
*
OpenHMPP
OpenHMPP (HMPP for Hybrid Multicore Parallel Programming) - programming standard for heterogeneous computing. Based on a set of compiler directives, standard is a programming model designed to handle hardware accelerators without the complexity ...
References
{{Parallel computing
Parallel computing
Classes of computers
Computing terminology