HOME

TheInfoList



OR:

An AI accelerator is a class of specialized hardware accelerator or computer system designed to accelerate
artificial intelligence Artificial intelligence (AI) is intelligence—perceiving, synthesizing, and inferring information—demonstrated by machines, as opposed to intelligence displayed by animals and humans. Example tasks in which this is done include speech r ...
and
machine learning Machine learning (ML) is a field of inquiry devoted to understanding and building methods that 'learn', that is, methods that leverage data to improve performance on some set of tasks. It is seen as a part of artificial intelligence. Machine ...
applications, including
artificial neural network Artificial neural networks (ANNs), usually simply called neural networks (NNs) or neural nets, are computing systems inspired by the biological neural networks that constitute animal brains. An ANN is based on a collection of connected unit ...
s and
machine vision Machine vision (MV) is the technology and methods used to provide imaging-based automatic inspection and analysis for such applications as automatic inspection, process control, and robot guidance, usually in industry. Machine vision refers to ...
. Typical applications include algorithms for
robotics Robotics is an interdisciplinary branch of computer science and engineering. Robotics involves design, construction, operation, and use of robots. The goal of robotics is to design machines that can help and assist humans. Robotics integrate ...
, internet of things, and other
data In the pursuit of knowledge, data (; ) is a collection of discrete Value_(semiotics), values that convey information, describing quantity, qualitative property, quality, fact, statistics, other basic units of meaning, or simply sequences of sy ...
-intensive or sensor-driven tasks. They are often
manycore Manycore processors are special kinds of multi-core processors designed for a high degree of parallel processing, containing numerous simpler, independent processor cores (from a few tens of cores to thousands or more). Manycore processors are use ...
designs and generally focus on low-precision arithmetic, novel
dataflow architecture Dataflow architecture is a dataflow-based computer architecture that directly contrasts the traditional von Neumann architecture or control flow architecture. Dataflow architectures have no program counter, in concept: the executability and executi ...
s or
in-memory computing In computer science, in-memory processing is an emerging technology for processing of data stored in an in-memory database. In-memory processing is one method of addressing the performance and power bottlenecks caused by the movement of data be ...
capability. , a typical AI integrated circuit chip contains billions of MOSFET transistors. A number of vendor-specific terms exist for devices in this category, and it is an
emerging technology Emerging technologies are technologies whose development, practical applications, or both are still largely unrealized. These technologies are generally new but also include older technologies finding new applications. Emerging technologies ar ...
without a
dominant design Dominant design is a technology management concept introduced by James M. Utterback and William J. Abernathy in 1975, identifying key technological features that become a de facto standard. A dominant design is the one that wins the allegiance of t ...
.


History

Computer systems have frequently complemented the CPU with special-purpose accelerators for specialized tasks, known as coprocessors. Notable application-specific hardware units include
video card A graphics card (also called a video card, display card, graphics adapter, VGA card/VGA, video adapter, display adapter, or mistakenly GPU) is an expansion card which generates a feed of output images to a display device, such as a computer mo ...
s for
graphic Graphics () are visual images or designs on some surface, such as a wall, canvas, screen, paper, or stone, to inform, illustrate, or entertain. In contemporary usage, it includes a pictorial representation of data, as in design and manufacture, ...
s,
sound card A sound card (also known as an audio card) is an internal expansion card that provides input and output of audio signals to and from a computer under the control of computer programs. The term ''sound card'' is also applied to external audio ...
s,
graphics processing unit A graphics processing unit (GPU) is a specialized electronic circuit designed to manipulate and alter memory to accelerate the creation of images in a frame buffer intended for output to a display device. GPUs are used in embedded systems, m ...
s and digital signal processors. As deep learning and
artificial intelligence Artificial intelligence (AI) is intelligence—perceiving, synthesizing, and inferring information—demonstrated by machines, as opposed to intelligence displayed by animals and humans. Example tasks in which this is done include speech r ...
workloads rose in prominence in the 2010s, specialized hardware units were developed or adapted from existing products to
accelerate In mechanics, acceleration is the rate of change of the velocity of an object with respect to time. Accelerations are vector quantities (in that they have magnitude and direction). The orientation of an object's acceleration is given by t ...
these tasks.


Early attempts

First attempts like
Intel Intel Corporation is an American multinational corporation and technology company headquartered in Santa Clara, California. It is the world's largest semiconductor chip manufacturer by revenue, and is one of the developers of the x86 seri ...
's ETANN 80170NX incorporated analog circuits to compute neural functions. Later all-digital chips like the Nestor/Intel
Ni1000 The Ni1000 is an artificial neural network chip developed by Nestor Corporation and Intel. It is Intel's second-generation neural network chip but first all digital. The chip is aimed at image analysis applications, contains more than 3 million tra ...
followed. As early as 1993, digital signal processors were used as neural network accelerators to accelerate
optical character recognition Optical character recognition or optical character reader (OCR) is the electronic or mechanical conversion of images of typed, handwritten or printed text into machine-encoded text, whether from a scanned document, a photo of a document, a sc ...
software. In the 1990s, there were also attempts to create parallel high-throughput systems for workstations aimed at various applications, including neural network simulations.This presentation covers a past attempt at neural net accelerators, notes the similarity to the modern SLI GPGPU processor setup, and argues that general purpose vector accelerators are the way forward (in relation to RISC-V hwacha project. Argues that NN's are just dense and sparse matrices, one of several recurring algorithms) FPGA-based accelerators were also first explored in the 1990s for both inference and training.
Smartphone A smartphone is a portable computer device that combines mobile telephone and computing functions into one unit. They are distinguished from feature phones by their stronger hardware capabilities and extensive mobile operating systems, whic ...
s began incorporating AI accelerators starting with the Qualcomm Snapdragon 820 in 2015.


Heterogeneous computing

Heterogeneous computing Heterogeneous computing refers to systems that use more than one kind of processor or cores. These systems gain performance or energy efficiency not just by adding the same type of processors, but by adding dissimilar coprocessors, usually incor ...
refers to incorporating a number of specialized processors in a single system, or even a single chip, each optimized for a specific type of task. Architectures such as the
Cell microprocessor Cell is a multi-core microprocessor microarchitecture that combines a general-purpose PowerPC core of modest performance with streamlined coprocessing elements which greatly accelerate multimedia and vector processing applications, as well as m ...
have features significantly overlapping with AI accelerators including: support for packed low precision arithmetic,
dataflow architecture Dataflow architecture is a dataflow-based computer architecture that directly contrasts the traditional von Neumann architecture or control flow architecture. Dataflow architectures have no program counter, in concept: the executability and executi ...
, and prioritizing 'throughput' over latency. The Cell microprocessor was subsequently applied to a number of tasks including AI. In the 2000s, CPUs also gained increasingly wide
SIMD Single instruction, multiple data (SIMD) is a type of parallel processing in Flynn's taxonomy. SIMD can be internal (part of the hardware design) and it can be directly accessible through an instruction set architecture (ISA), but it shoul ...
units, driven by video and gaming workloads; as well as support for packed low-precision data types. Due to increasing performance of CPUs, they are also being used for running AI workloads. CPUs are superior for DNNs with small or medium-scale parallelism, for sparse DNNs and in low-batch-size scenarios.


Use of GPU

Graphics processing unit A graphics processing unit (GPU) is a specialized electronic circuit designed to manipulate and alter memory to accelerate the creation of images in a frame buffer intended for output to a display device. GPUs are used in embedded systems, m ...
s or GPUs are specialized hardware for the manipulation of images and calculation of local image properties. The mathematical basis of neural networks and
image manipulation Image editing encompasses the processes of altering images, whether they are digital photographs, traditional photo-chemical photographs, or illustrations. Traditional analog image editing is known as photo retouching, using tools such as ...
are similar,
embarrassingly parallel In parallel computing, an embarrassingly parallel workload or problem (also called embarrassingly parallelizable, perfectly parallel, delightfully parallel or pleasingly parallel) is one where little or no effort is needed to separate the problem ...
tasks involving matrices, leading GPUs to become increasingly used for machine learning tasks. , GPUs are popular for AI work, and they continue to evolve in a direction to facilitate deep learning, both for training and inference in devices such as
self-driving car A self-driving car, also known as an autonomous car, driver-less car, or robotic car (robo-car), is a car that is capable of traveling without human input.Xie, S.; Hu, J.; Bhowmick, P.; Ding, Z.; Arvin, F.,Distributed Motion Planning for S ...
s. GPU developers such as Nvidia
NVLink NVLink is a wire-based serial multi-lane near-range communications link developed by Nvidia. Unlike PCI Express, a device can consist of multiple NVLinks, and devices use mesh networking to communicate instead of a central hub. The protocol was f ...
are developing additional connective capability for the kind of dataflow workloads AI benefits from. As GPUs have been increasingly applied to AI acceleration, GPU manufacturers have incorporated neural network-
specific Specific may refer to: * Specificity (disambiguation) * Specific, a cure or therapy for a specific illness Law * Specific deterrence, focussed on an individual * Specific finding, intermediate verdict used by a jury in determining the fina ...
hardware to further accelerate these tasks. Tensor cores are intended to speed up the training of neural networks.


Use of FPGAs

Deep learning frameworks are still evolving, making it hard to design custom hardware. Reconfigurable devices such as field-programmable gate arrays (FPGA) make it easier to evolve hardware, frameworks, and software alongside each other. Microsoft has used FPGA chips to accelerate inference.


Emergence of dedicated AI accelerator ASICs

While GPUs and FPGAs perform far better than CPUs for AI-related tasks, a factor of up to 10 in efficiency may be gained with a more specific design, via an
application-specific integrated circuit An application-specific integrated circuit (ASIC ) is an integrated circuit (IC) chip customized for a particular use, rather than intended for general-purpose use, such as a chip designed to run in a digital voice recorder or a high-effici ...
(ASIC). These accelerators employ strategies such as optimized memory use and the use of lower precision arithmetic to accelerate calculation and increase
throughput Network throughput (or just throughput, when in context) refers to the rate of message delivery over a communication channel, such as Ethernet or packet radio, in a communication network. The data that these messages contain may be delivered ove ...
of computation. Some adopted low-precision floating-point formats used AI acceleration are
half-precision In computing, half precision (sometimes called FP16) is a binary floating-point computer number format that occupies 16 bits (two bytes in modern computers) in computer memory. It is intended for storage of floating-point values in applications w ...
and the bfloat16 floating-point format. Companies such as Google, Qualcomm, Amazon, Apple, Facebook, AMD and Samsung are all designing their own AI ASICs. Cerebras Systems has also built a dedicated AI accelerator based on the largest processor in the industry, the second-generation Wafer Scale Engine (WSE-2), to support deep learning workloads.


In-memory computing architectures

In June 2017, IBM researchers announced an architecture in contrast to the
Von Neumann architecture The von Neumann architecture — also known as the von Neumann model or Princeton architecture — is a computer architecture based on a 1945 description by John von Neumann, and by others, in the '' First Draft of a Report on the EDVAC''. T ...
based on
in-memory computing In computer science, in-memory processing is an emerging technology for processing of data stored in an in-memory database. In-memory processing is one method of addressing the performance and power bottlenecks caused by the movement of data be ...
and
phase-change memory Phase-change memory (also known as PCM, PCME, PRAM, PCRAM, OUM (ovonic unified memory) and C-RAM or CRAM (chalcogenide RAM)) is a type of non-volatile random-access memory. PRAMs exploit the unique behaviour of chalcogenide glass. In PCM, heat pr ...
arrays applied to temporal correlation detection, intending to generalize the approach to
heterogeneous computing Heterogeneous computing refers to systems that use more than one kind of processor or cores. These systems gain performance or energy efficiency not just by adding the same type of processors, but by adding dissimilar coprocessors, usually incor ...
and
massively parallel Massively parallel is the term for using a large number of computer processors (or separate computers) to simultaneously perform a set of coordinated computations in parallel. GPUs are massively parallel architecture with tens of thousands of t ...
systems. In October 2018, IBM researchers announced an architecture based on
in-memory processing In computer science, in-memory processing is an emerging technology for processing of data stored in an in-memory database. In-memory processing is one method of addressing the performance and power bottlenecks caused by the movement of data be ...
and modeled on the human brain's synaptic network to accelerate
deep neural network Deep learning (also known as deep structured learning) is part of a broader family of machine learning methods based on artificial neural networks with representation learning. Learning can be supervised, semi-supervised or unsupervised. D ...
s. The system is based on
phase-change memory Phase-change memory (also known as PCM, PCME, PRAM, PCRAM, OUM (ovonic unified memory) and C-RAM or CRAM (chalcogenide RAM)) is a type of non-volatile random-access memory. PRAMs exploit the unique behaviour of chalcogenide glass. In PCM, heat pr ...
arrays.


In-memory computing with analog resistive memories

In 2019, researchers from Politecnico di Milano found a way to solve systems of linear equations in a few tens of nanoseconds via a single operation. Their algorithm is based on
in-memory computing In computer science, in-memory processing is an emerging technology for processing of data stored in an in-memory database. In-memory processing is one method of addressing the performance and power bottlenecks caused by the movement of data be ...
with analog resistive memories which performs with high efficiencies of time and energy, via conducting
matrix–vector multiplication In mathematics, particularly in linear algebra, matrix multiplication is a binary operation that produces a matrix (mathematics), matrix from two matrices. For matrix multiplication, the number of columns in the first matrix must be equal to the n ...
in one step using Ohm's law and Kirchhoff's law. The researchers showed that a feedback circuit with cross-point resistive memories can solve algebraic problems such as systems of linear equations, matrix eigenvectors, and differential equations in just one step. Such an approach improves computational times drastically in comparison with digital algorithms.


Atomically thin semiconductors

In 2020, Marega et al. published experiments with a large-area active channel material for developing logic-in-memory devices and circuits based on
floating-gate The floating-gate MOSFET (FGMOS), also known as a floating-gate MOS transistor or floating-gate transistor, is a type of metal–oxide–semiconductor field-effect transistor (MOSFET) where the gate is electrically isolated, creating a floating no ...
field-effect transistor The field-effect transistor (FET) is a type of transistor that uses an electric field to control the flow of current in a semiconductor. FETs ( JFETs or MOSFETs) are devices with three terminals: ''source'', ''gate'', and ''drain''. FETs cont ...
s (FGFETs). Such atomically thin
semiconductor A semiconductor is a material which has an electrical conductivity value falling between that of a conductor, such as copper, and an insulator, such as glass. Its resistivity falls as its temperature rises; metals behave in the opposite way. ...
s are considered promising for energy-efficient
machine learning Machine learning (ML) is a field of inquiry devoted to understanding and building methods that 'learn', that is, methods that leverage data to improve performance on some set of tasks. It is seen as a part of artificial intelligence. Machine ...
applications, where the same basic device structure is used for both logic operations and data storage. The authors used two-dimensional materials such as semiconducting molybdenum disulfide.


Integrated photonic tensor core

In 2021, J. Feldmann et al. proposed an integrated
photonic Photonics is a branch of optics that involves the application of generation, detection, and manipulation of light in form of photons through emission, transmission, modulation, signal processing, switching, amplification, and sensing. Though ...
hardware accelerator for parallel convolutional processing. The authors identify two key advantages of integrated photonics over its electronic counterparts: (1) massively parallel data transfer through
wavelength In physics, the wavelength is the spatial period of a periodic wave—the distance over which the wave's shape repeats. It is the distance between consecutive corresponding points of the same phase on the wave, such as two adjacent crests, t ...
division multiplexing in conjunction with
frequency comb In optics, a frequency comb is a laser source whose spectrum consists of a series of discrete, equally spaced frequency lines. Frequency combs can be generated by a number of mechanisms, including periodic modulation (in amplitude and/or phase) of a ...
s, and (2) extremely high data modulation speeds. Their system can execute trillions of multiply-accumulate operations per second, indicating the potential of integrated
photonics Photonics is a branch of optics that involves the application of generation, detection, and manipulation of light in form of photons through emission, transmission, modulation, signal processing, switching, amplification, and sensing. Though ...
in data-heavy AI applications.


Nomenclature

As of 2016, the field is still in flux and vendors are pushing their own marketing term for what amounts to an "AI accelerator", in the hope that their designs and APIs will become the
dominant design Dominant design is a technology management concept introduced by James M. Utterback and William J. Abernathy in 1975, identifying key technological features that become a de facto standard. A dominant design is the one that wins the allegiance of t ...
. There is no consensus on the boundary between these devices, nor the exact form they will take; however several examples clearly aim to fill this new space, with a fair amount of overlap in capabilities. In the past when consumer
graphics accelerator A graphics processing unit (GPU) is a specialized electronic circuit designed to manipulate and alter memory to accelerate the creation of images in a frame buffer intended for output to a display device. GPUs are used in embedded systems, mob ...
s emerged, the industry eventually adopted
Nvidia Nvidia CorporationOfficially written as NVIDIA and stylized in its logo as VIDIA with the lowercase "n" the same height as the uppercase "VIDIA"; formerly stylized as VIDIA with a large italicized lowercase "n" on products from the mid 1990s to ...
's self-assigned term, "the GPU", as the collective noun for "graphics accelerators", which had taken many forms before settling on an overall
pipeline Pipeline may refer to: Electronics, computers and computing * Pipeline (computing), a chain of data-processing stages or a CPU optimization found on ** Instruction pipelining, a technique for implementing instruction-level parallelism within a s ...
implementing a model presented by Direct3D.


Potential applications

*
Agricultural robot An agricultural robot is a robot deployed for agricultural purposes. The main area of application of robots in agriculture today is at the harvesting stage. Emerging applications of robots or drones in agriculture include weed control, cloud s ...
s, for example herbicide-free weed control. *
Autonomous vehicles Vehicular automation involves the use of mechatronics, artificial intelligence, and multi-agent systems to assist the operator of a vehicle (car, aircraft, watercraft, or otherwise).Hu, J.; Bhowmick, P.; Lanzon, A.,Group Coordinated Control o ...
: Nvidia has targeted their
Drive PX-series Nvidia Drive is a computer platform by Nvidia, aimed at providing autonomous car and driver assistance functionality powered by deep learning. The platform was introduced at the Consumer Electronics Show (CES) in Las Vegas in January 2015. An enhan ...
boards at this application. *
Computer-aided diagnosis Computer-aided detection (CADe), also called computer-aided diagnosis (CADx), are systems that assist doctors in the interpretation of medical images. Imaging techniques in X-ray, MRI, Endoscopy, and ultrasound diagnostics yield a great deal o ...
*
Industrial robot An industrial robot is a robot system used for manufacturing. Industrial robots are automated, programmable and capable of movement on three or more axes. Typical applications of robots include welding, painting, assembly, disassembly, pick ...
s, increasing the range of tasks that can be automated, by adding adaptability to variable situations. *
Machine translation Machine translation, sometimes referred to by the abbreviation MT (not to be confused with computer-aided translation, machine-aided human translation or interactive translation), is a sub-field of computational linguistics that investigates t ...
*
Military robot Military robots are autonomous robots or remote-controlled mobile robots designed for military applications, from transport to search & rescue and attack. Some such systems are currently in use, and many are under development. History Broa ...
s * Natural language processing * Search engines, increasing the
energy efficiency Energy efficiency may refer to: * Energy efficiency (physics), the ratio between the useful output and input of an energy conversion process ** Electrical efficiency, useful power output per electrical power consumed ** Mechanical efficiency, a ra ...
of data centers and ability to use increasingly advanced queries. *
Unmanned aerial vehicle An unmanned aerial vehicle (UAV), commonly known as a drone, is an aircraft without any human pilot, crew, or passengers on board. UAVs are a component of an unmanned aircraft system (UAS), which includes adding a ground-based controll ...
s, e.g. navigation systems, e.g. the Movidius Myriad 2 has been demonstrated successfully guiding autonomous drones. *
Voice user interface A voice-user interface (VUI) makes spoken human interaction with computers possible, using speech recognition to understand spoken commands and answer questions, and typically text to speech to play a reply. A voice command device is a device con ...
, e.g. in mobile phones, a target for Qualcomm
Zeroth 0th or zeroth may refer to: Mathematics, science and technology * 0th or zeroth, an ordinal for the number zero * 0th dimension, a topological space * 0th element, of a data structure in computer science * Zeroth (software), deep learning softw ...
.


See also

* Cognitive computer * Deep learning processor *
Neuromorphic engineering Neuromorphic engineering, also known as neuromorphic computing, is the use of electronic circuits to mimic neuro-biological architectures present in the nervous system. A neuromorphic computer/chip is any device that uses physical artificial ...
*
Optical neural network An optical neural network is a physical implementation of an artificial neural network with photonics, optical components. Early optical neural networks used a photorefractive Volume hologram to interconnect arrays of input neurons to arrays of o ...
* Physical neural network


References


External links


Nvidia Puts The Accelerator To The Metal With Pascal.htm
The Next Platform
Eyeriss Project
MIT *https://alphaics.ai/ {{Hardware acceleration Application-specific integrated circuits Coprocessors Computer optimization Gate arrays