A vision processing unit (VPU) is (as of 2018) an emerging class of
microprocessor
A microprocessor is a computer processor where the data processing logic and control is included on a single integrated circuit, or a small number of integrated circuits. The microprocessor contains the arithmetic, logic, and control circ ...
; it is a specific type of
AI accelerator, designed to
accelerate
In mechanics, acceleration is the rate of change of the velocity of an object with respect to time. Accelerations are vector quantities (in that they have magnitude and direction). The orientation of an object's acceleration is given by t ...
machine vision
Machine vision (MV) is the technology and methods used to provide imaging-based automatic inspection and analysis for such applications as automatic inspection, process control, and robot guidance, usually in industry. Machine vision refers to ...
tasks.
Overview
Vision processing units are distinct from
video processing units (which are specialised for
video encoding and decoding) in their suitability for running
machine vision algorithms such as CNN (
convolutional neural network
In deep learning, a convolutional neural network (CNN, or ConvNet) is a class of artificial neural network (ANN), most commonly applied to analyze visual imagery. CNNs are also known as Shift Invariant or Space Invariant Artificial Neural Netwo ...
s), SIFT (
Scale-invariant feature transform
The scale-invariant feature transform (SIFT) is a computer vision algorithm to detect, describe, and match local ''features'' in images, invented by David Lowe in 1999.
Applications include object recognition, robotic mapping and navigation, ima ...
) and similar.
They may include
direct interfaces to take data from
cameras
A camera is an optical instrument that can capture an image. Most cameras can capture 2D images, with some more advanced models being able to capture 3D images. At a basic level, most cameras consist of sealed boxes (the camera body), with a ...
(bypassing any off chip buffers), and have a greater emphasis on on-chip
dataflow between many
parallel execution units with
scratchpad memory
Scratchpad memory (SPM), also known as scratchpad, scratchpad RAM or local store in computer terminology, is a high-speed internal memory used for temporary storage of calculations, data, and other work in progress. In reference to a microprocess ...
, like a
manycore DSP
DSP may refer to:
Computing
* Digital signal processing, the mathematical manipulation of an information signal
* Digital signal processor, a microprocessor designed for digital signal processing
* Yamaha DSP-1, a proprietary digital signal ...
. But, like video processing units, they may have a focus on
low precision fixed point arithmetic for
image processing.
Contrast with GPUs
They are distinct from
GPU
A graphics processing unit (GPU) is a specialized electronic circuit designed to manipulate and alter memory to accelerate the creation of images in a frame buffer intended for output to a display device. GPUs are used in embedded systems, mobi ...
s, which contain specialised hardware for
rasterization
In computer graphics, rasterisation (British English) or rasterization (American English) is the task of taking an image described in a vector graphics format (shapes) and converting it into a raster image (a series of pixels, dots or lines, whi ...
and
texture mapping
Texture mapping is a method for mapping a texture on a computer-generated graphic. Texture here can be high frequency detail, surface texture, or color.
History
The original technique was pioneered by Edwin Catmull in 1974.
Texture mappi ...
(for
3D graphics
3D computer graphics, or “3D graphics,” sometimes called CGI, 3D-CGI or three-dimensional computer graphics are graphics that use a three-dimensional representation of geometric data (often Cartesian) that is stored in the computer for th ...
), and whose
memory architecture
Memory architecture describes the methods used to implement electronic computer data storage in a manner that is a combination of the fastest, most reliable, most durable, and least expensive way to store and retrieve information. Depending on the ...
is optimised for manipulating
bitmap images in
off-chip memory (reading
textures, and modifying
frame buffers, with
random access patterns). VPUs are optimized for performance per watt, while GPUs mainly focus on absolute performance.
Target markets are
robotics
Robotics is an interdisciplinary branch of computer science and engineering. Robotics involves design, construction, operation, and use of robots. The goal of robotics is to design machines that can help and assist humans. Robotics integrate ...
, the
internet of things, new classes of
digital cameras
A digital camera is a camera that captures photographs in digital memory. Most cameras produced today are digital, largely replacing those that capture images on photographic film. Digital cameras are now widely incorporated into mobile devices ...
for
virtual reality
Virtual reality (VR) is a simulated experience that employs pose tracking and 3D near-eye displays to give the user an immersive feel of a virtual world. Applications of virtual reality include entertainment (particularly video games), e ...
and
augmented reality,
smart cameras, and integrating machine vision acceleration into
smartphone
A smartphone is a portable computer device that combines mobile telephone and computing functions into one unit. They are distinguished from feature phones by their stronger hardware capabilities and extensive mobile operating systems, whic ...
s and other
mobile devices
A mobile device (or handheld computer) is a computer small enough to hold and operate in the hand. Mobile devices typically have a flat LCD or OLED screen, a touchscreen interface, and digital or physical buttons. They may also have a physical ...
.
Examples
Movidius Myriad X which is the third-generation vision processing unit in the Myriad VPU line from
Intel Corporation.
*
Movidius Myriad 2, which finds use in
Google Project Tango,
Google Clips and DJI Drones
*
Pixel Visual Core (PVC), which is a fully programmable
Image, Vision and
AI processor for mobile devices
*
Microsoft HoloLens
Microsoft HoloLens is an augmented reality (AR)/ mixed reality (MR) headset developed and manufactured by Microsoft. HoloLens runs the Windows Mixed Reality platform under the Windows 10 operating system. Some of the positional tracking tech ...
, which includes an accelerator referred to as a ''Holographic Processing Unit'' (complementary to its CPU and GPU), aimed at interpreting camera inputs, to accelerate environment tracking & vision for augmented reality applications.
*
Eyeriss, a design from
MIT
The Massachusetts Institute of Technology (MIT) is a private land-grant research university in Cambridge, Massachusetts. Established in 1861, MIT has played a key role in the development of modern technology and science, and is one of the m ...
intended for running
convolutional neural network
In deep learning, a convolutional neural network (CNN, or ConvNet) is a class of artificial neural network (ANN), most commonly applied to analyze visual imagery. CNNs are also known as Shift Invariant or Space Invariant Artificial Neural Netwo ...
s.
*
NeuFlow, a design by
Yann LeCun
Yann André LeCun ( , ; originally spelled Le Cun; born 8 July 1960) is a French computer scientist working primarily in the fields of machine learning, computer vision, mobile robotics and computational neuroscience. He is the Silver Professo ...
(implemented in
FPGA) for accelerating
convolutions, using a dataflow architecture.
*
Mobileye EyeQ, by
Mobileye
Mobileye Global Inc. is a company developing autonomous driving technologies and advanced driver-assistance systems (ADAS) including cameras, computer chips and software. Mobileye was acquired by Intel in 2017 and went public again in 2022. Mobi ...
* Programmable Vision Accelerator (PVA), a
7-way VLIW Vision Processor designed by
Nvidia
Nvidia CorporationOfficially written as NVIDIA and stylized in its logo as VIDIA with the lowercase "n" the same height as the uppercase "VIDIA"; formerly stylized as VIDIA with a large italicized lowercase "n" on products from the mid 1990s to ...
.
Similar processors
Some processors are not described as VPUs, but are equally applicable to machine vision tasks. These may form a broader category of ''
AI accelerators'' (to which VPUs may also belong), however as of 2016 there is no consensus on the name:
*
IBM TrueNorth A cognitive computer is a computer that hardwires artificial intelligence and machine-learning algorithms into an integrated circuit (printed circuit board) that closely reproduces the behavior of the human brain. It generally adopts a neuromorphic ...
, a
neuromorphic processor aimed at similar sensor data
pattern recognition
Pattern recognition is the automated recognition of patterns and regularities in data. It has applications in statistical data analysis, signal processing, image analysis, information retrieval, bioinformatics, data compression, computer graphics ...
and intelligence tasks, including video/audio.
*
Qualcomm Zeroth Neural processing unit, another entry in the emerging class of sensor/AI oriented chips.
See also
*
Adapteva Epiphany, a
Manycore processor
Manycore processors are special kinds of multi-core processors designed for a high degree of parallel processing, containing numerous simpler, independent processor cores (from a few tens of cores to thousands or more). Manycore processors are us ...
with similar emphasis on on-chip dataflow, focussed on 32-bit
floating point
In computing, floating-point arithmetic (FP) is arithmetic that represents real numbers approximately, using an integer with a fixed precision, called the significand, scaled by an integer exponent of a fixed base. For example, 12.345 can b ...
performance.
*
CELL
Cell most often refers to:
* Cell (biology), the functional basic unit of life
Cell may also refer to:
Locations
* Monastic cell, a small room, hut, or cave in which a religious recluse lives, alternatively the small precursor of a monastery ...
, a multicore processor with features fairly consistent with vision processing units (
SIMD
Single instruction, multiple data (SIMD) is a type of parallel processing in Flynn's taxonomy. SIMD can be internal (part of the hardware design) and it can be directly accessible through an instruction set architecture (ISA), but it shoul ...
instructions & datatypes suitable for video, and on-chip
DMA between scratchpad memories).
*
Coprocessor
*
Graphics processing unit
A graphics processing unit (GPU) is a specialized electronic circuit designed to manipulate and alter memory to accelerate the creation of images in a frame buffer intended for output to a display device. GPUs are used in embedded systems, m ...
, also commonly used to run vision algorithms. NVidia's
Pascal architecture includes
FP16
In computing, half precision (sometimes called FP16) is a binary floating-point computer number format that occupies 16 bits (two bytes in modern computers) in computer memory. It is intended for storage of floating-point values in applications w ...
support, to provide a better precision/cost tradeoff for AI workloads.
*
MPSoC
*
OpenCL
OpenCL (Open Computing Language) is a framework for writing programs that execute across heterogeneous platforms consisting of central processing units (CPUs), graphics processing units (GPUs), digital signal processors (DSPs), field-progra ...
*
OpenVX
*
Physics processing unit, a past attempt to complement the
CPU and
GPU
A graphics processing unit (GPU) is a specialized electronic circuit designed to manipulate and alter memory to accelerate the creation of images in a frame buffer intended for output to a display device. GPUs are used in embedded systems, mobi ...
with a high throughput accelerator.
*
Tensor processing unit, a chip used internally by Google for accelerating AI calculations.
References
External links
Eyeriss architectureHolographic processing unitNeuFlow: A Runtime Reconfigurable Dataflow Processor for Vision
{{Differentiable computing
Microprocessors
AI accelerators
Machine vision