HOME

TheInfoList



OR:

A vision processing unit (VPU) is (as of 2018) an emerging class of
microprocessor A microprocessor is a computer processor where the data processing logic and control is included on a single integrated circuit, or a small number of integrated circuits. The microprocessor contains the arithmetic, logic, and control circ ...
; it is a specific type of AI accelerator, designed to
accelerate In mechanics, acceleration is the rate of change of the velocity of an object with respect to time. Accelerations are vector quantities (in that they have magnitude and direction). The orientation of an object's acceleration is given by t ...
machine vision Machine vision (MV) is the technology and methods used to provide imaging-based automatic inspection and analysis for such applications as automatic inspection, process control, and robot guidance, usually in industry. Machine vision refers to ...
tasks.


Overview

Vision processing units are distinct from video processing units (which are specialised for video encoding and decoding) in their suitability for running machine vision algorithms such as CNN (
convolutional neural network In deep learning, a convolutional neural network (CNN, or ConvNet) is a class of artificial neural network (ANN), most commonly applied to analyze visual imagery. CNNs are also known as Shift Invariant or Space Invariant Artificial Neural Netwo ...
s), SIFT (
Scale-invariant feature transform The scale-invariant feature transform (SIFT) is a computer vision algorithm to detect, describe, and match local ''features'' in images, invented by David Lowe in 1999. Applications include object recognition, robotic mapping and navigation, ima ...
) and similar. They may include direct interfaces to take data from
cameras A camera is an optical instrument that can capture an image. Most cameras can capture 2D images, with some more advanced models being able to capture 3D images. At a basic level, most cameras consist of sealed boxes (the camera body), with a ...
(bypassing any off chip buffers), and have a greater emphasis on on-chip dataflow between many parallel execution units with
scratchpad memory Scratchpad memory (SPM), also known as scratchpad, scratchpad RAM or local store in computer terminology, is a high-speed internal memory used for temporary storage of calculations, data, and other work in progress. In reference to a microprocess ...
, like a manycore
DSP DSP may refer to: Computing * Digital signal processing, the mathematical manipulation of an information signal * Digital signal processor, a microprocessor designed for digital signal processing * Yamaha DSP-1, a proprietary digital signal ...
. But, like video processing units, they may have a focus on low precision fixed point arithmetic for image processing.


Contrast with GPUs

They are distinct from
GPU A graphics processing unit (GPU) is a specialized electronic circuit designed to manipulate and alter memory to accelerate the creation of images in a frame buffer intended for output to a display device. GPUs are used in embedded systems, mobi ...
s, which contain specialised hardware for
rasterization In computer graphics, rasterisation (British English) or rasterization (American English) is the task of taking an image described in a vector graphics format (shapes) and converting it into a raster image (a series of pixels, dots or lines, whi ...
and
texture mapping Texture mapping is a method for mapping a texture on a computer-generated graphic. Texture here can be high frequency detail, surface texture, or color. History The original technique was pioneered by Edwin Catmull in 1974. Texture mappi ...
(for
3D graphics 3D computer graphics, or “3D graphics,” sometimes called CGI, 3D-CGI or three-dimensional computer graphics are graphics that use a three-dimensional representation of geometric data (often Cartesian) that is stored in the computer for th ...
), and whose
memory architecture Memory architecture describes the methods used to implement electronic computer data storage in a manner that is a combination of the fastest, most reliable, most durable, and least expensive way to store and retrieve information. Depending on the ...
is optimised for manipulating bitmap images in off-chip memory (reading textures, and modifying frame buffers, with random access patterns). VPUs are optimized for performance per watt, while GPUs mainly focus on absolute performance. Target markets are
robotics Robotics is an interdisciplinary branch of computer science and engineering. Robotics involves design, construction, operation, and use of robots. The goal of robotics is to design machines that can help and assist humans. Robotics integrate ...
, the internet of things, new classes of
digital cameras A digital camera is a camera that captures photographs in digital memory. Most cameras produced today are digital, largely replacing those that capture images on photographic film. Digital cameras are now widely incorporated into mobile devices ...
for
virtual reality Virtual reality (VR) is a simulated experience that employs pose tracking and 3D near-eye displays to give the user an immersive feel of a virtual world. Applications of virtual reality include entertainment (particularly video games), e ...
and augmented reality, smart cameras, and integrating machine vision acceleration into
smartphone A smartphone is a portable computer device that combines mobile telephone and computing functions into one unit. They are distinguished from feature phones by their stronger hardware capabilities and extensive mobile operating systems, whic ...
s and other
mobile devices A mobile device (or handheld computer) is a computer small enough to hold and operate in the hand. Mobile devices typically have a flat LCD or OLED screen, a touchscreen interface, and digital or physical buttons. They may also have a physical ...
.


Examples


Movidius Myriad X
which is the third-generation vision processing unit in the Myriad VPU line from Intel Corporation. * Movidius Myriad 2, which finds use in Google Project Tango, Google Clips and DJI Drones * Pixel Visual Core (PVC), which is a fully programmable Image, Vision and AI processor for mobile devices *
Microsoft HoloLens Microsoft HoloLens is an augmented reality (AR)/ mixed reality (MR) headset developed and manufactured by Microsoft. HoloLens runs the Windows Mixed Reality platform under the Windows 10 operating system. Some of the positional tracking tech ...
, which includes an accelerator referred to as a ''Holographic Processing Unit'' (complementary to its CPU and GPU), aimed at interpreting camera inputs, to accelerate environment tracking & vision for augmented reality applications. * Eyeriss, a design from
MIT The Massachusetts Institute of Technology (MIT) is a private land-grant research university in Cambridge, Massachusetts. Established in 1861, MIT has played a key role in the development of modern technology and science, and is one of the m ...
intended for running
convolutional neural network In deep learning, a convolutional neural network (CNN, or ConvNet) is a class of artificial neural network (ANN), most commonly applied to analyze visual imagery. CNNs are also known as Shift Invariant or Space Invariant Artificial Neural Netwo ...
s. * NeuFlow, a design by
Yann LeCun Yann André LeCun ( , ; originally spelled Le Cun; born 8 July 1960) is a French computer scientist working primarily in the fields of machine learning, computer vision, mobile robotics and computational neuroscience. He is the Silver Professo ...
(implemented in FPGA) for accelerating convolutions, using a dataflow architecture. * Mobileye EyeQ, by
Mobileye Mobileye Global Inc. is a company developing autonomous driving technologies and advanced driver-assistance systems (ADAS) including cameras, computer chips and software. Mobileye was acquired by Intel in 2017 and went public again in 2022. Mobi ...
* Programmable Vision Accelerator (PVA), a 7-way VLIW Vision Processor designed by
Nvidia Nvidia CorporationOfficially written as NVIDIA and stylized in its logo as VIDIA with the lowercase "n" the same height as the uppercase "VIDIA"; formerly stylized as VIDIA with a large italicized lowercase "n" on products from the mid 1990s to ...
.


Similar processors

Some processors are not described as VPUs, but are equally applicable to machine vision tasks. These may form a broader category of '' AI accelerators'' (to which VPUs may also belong), however as of 2016 there is no consensus on the name: * IBM
TrueNorth A cognitive computer is a computer that hardwires artificial intelligence and machine-learning algorithms into an integrated circuit (printed circuit board) that closely reproduces the behavior of the human brain. It generally adopts a neuromorphic ...
, a neuromorphic processor aimed at similar sensor data
pattern recognition Pattern recognition is the automated recognition of patterns and regularities in data. It has applications in statistical data analysis, signal processing, image analysis, information retrieval, bioinformatics, data compression, computer graphics ...
and intelligence tasks, including video/audio. * Qualcomm Zeroth Neural processing unit, another entry in the emerging class of sensor/AI oriented chips.


See also

* Adapteva Epiphany, a
Manycore processor Manycore processors are special kinds of multi-core processors designed for a high degree of parallel processing, containing numerous simpler, independent processor cores (from a few tens of cores to thousands or more). Manycore processors are us ...
with similar emphasis on on-chip dataflow, focussed on 32-bit
floating point In computing, floating-point arithmetic (FP) is arithmetic that represents real numbers approximately, using an integer with a fixed precision, called the significand, scaled by an integer exponent of a fixed base. For example, 12.345 can b ...
performance. *
CELL Cell most often refers to: * Cell (biology), the functional basic unit of life Cell may also refer to: Locations * Monastic cell, a small room, hut, or cave in which a religious recluse lives, alternatively the small precursor of a monastery ...
, a multicore processor with features fairly consistent with vision processing units (
SIMD Single instruction, multiple data (SIMD) is a type of parallel processing in Flynn's taxonomy. SIMD can be internal (part of the hardware design) and it can be directly accessible through an instruction set architecture (ISA), but it shoul ...
instructions & datatypes suitable for video, and on-chip DMA between scratchpad memories). * Coprocessor *
Graphics processing unit A graphics processing unit (GPU) is a specialized electronic circuit designed to manipulate and alter memory to accelerate the creation of images in a frame buffer intended for output to a display device. GPUs are used in embedded systems, m ...
, also commonly used to run vision algorithms. NVidia's Pascal architecture includes
FP16 In computing, half precision (sometimes called FP16) is a binary floating-point computer number format that occupies 16 bits (two bytes in modern computers) in computer memory. It is intended for storage of floating-point values in applications w ...
support, to provide a better precision/cost tradeoff for AI workloads. * MPSoC *
OpenCL OpenCL (Open Computing Language) is a framework for writing programs that execute across heterogeneous platforms consisting of central processing units (CPUs), graphics processing units (GPUs), digital signal processors (DSPs), field-progra ...
* OpenVX * Physics processing unit, a past attempt to complement the CPU and
GPU A graphics processing unit (GPU) is a specialized electronic circuit designed to manipulate and alter memory to accelerate the creation of images in a frame buffer intended for output to a display device. GPUs are used in embedded systems, mobi ...
with a high throughput accelerator. * Tensor processing unit, a chip used internally by Google for accelerating AI calculations.


References


External links


Eyeriss architecture

Holographic processing unit

NeuFlow: A Runtime Reconfigurable Dataflow Processor for Vision
{{Differentiable computing Microprocessors AI accelerators Machine vision