HOME

TheInfoList




An image processor, also known as an image processing engine, image processing unit (IPU), or image signal processor (ISP), is a type of
media processor A media processor, mostly used as an image processor, image/Video_scaler#Video_processor, video processor, is a microprocessor-based system-on-a-chip which is designed to deal with digital stream (computing), streaming data in real-time (e.g. displa ...
or specialized
digital signal processor The NeXTcube from 1990 had a Motorola 68040 (25 MHz) and a digital signal processor Motorola 56001">Motorola_68040.html" ;"title="NeXTcube from 1990 had a Motorola 68040">NeXTcube from 1990 had a Motorola 68040 (25 MHz) and a digital signal proce ...
(DSP) used for
image processing Digital image processing is the use of a digital computer A computer is a machine A machine is a man-made device that uses power to apply forces and control movement to perform an action. Machines can be driven by animals and people ...
, in
digital camera A digital camera is a camera that captures photographs in digital memory. Most cameras produced today are digital, largely replacing those that capture images on photographic film. Digital cameras are now widely incorporated into mobile devices ...

digital camera
s or other devices. Image processors often employ
parallel computing Parallel computing is a type of computation Computation is any type of calculation A calculation is a deliberate process that transforms one or more inputs into one or more results. The term is used in a variety of senses, from the very defi ...
even with
SIMD Single instruction, multiple data (SIMD) is a type of parallel processing in Flynn's taxonomy Flynn's taxonomy is a classification of computer architecture In computer engineering, computer architecture is a set of rules and methods that de ...

SIMD
or
MIMD In computing Computing is any goal-oriented activity requiring, benefiting from, or creating computing machinery. It includes the study and experimentation of algorithm of an algorithm (Euclid's algorithm) for calculating the greatest common ...

MIMD
technologies to increase speed and efficiency. The
digital image A digital image is an composed of s, also known as ''pixels'', each with ', ' of numeric representation for its or that is an output from its fed as input by its denoted with ''x'', ''y'' on the x-axis and y-axis, respectively. Depending on ...
processing engine can perform a range of tasks. To increase the system integration on embedded devices, often it is a
system on a chip A system on a chip (SoC; or This article uses the convention that SoC is pronounced . Therefore, it uses the convention "an" for the indefinite article Indefinite may refer to: * the opposite of definite in grammar ** indefinite article * ...
with
multi-core processor A multi-core processor is a computer processor on a single integrated circuit An integrated circuit or monolithic integrated circuit (also referred to as an IC, a chip, or a microchip) is a set of electronic circuit 200px, A circuit ...
architecture.


Function


Bayer transformation

The
photodiode A photodiode is a semiconductor p–n junction device that converts light Light or visible light is electromagnetic radiation within the portion of the electromagnetic spectrum that can be visual perception, perceived by the human eye. Visib ...

photodiode
s employed in an
image sensor An image sensor or imager is a that detects and conveys information used to make an . It does so by converting the variable of light s (as they or objects) into , small bursts of that convey the information. The waves can be light or other ...
are color-blind by nature: they can only record shades of grey. To get color into the picture, they are covered with different color filters: red, green and blue (
RGB s The RGB color model is an additive color, additive color model A color model is an abstract mathematical model describing the way colors can be represented as tuples of numbers, typically as three or four values or color components. When ...

RGB
) according to the pattern designated by the
Bayer filter A Bayer filter mosaic is a color filter array In digital imaging, a color filter array (CFA), or color filter mosaic (CFM), is a mosaic of tiny color filters placed over the pixel sensors of an image sensor to capture color Color ( ...
- named after its inventor. As each photodiode records the color information for exactly one
pixel In digital imaging Digital imaging or digital image acquisition is the creation of a representation of the visual characteristics of an object, such as a physical scene or the interior structure of an object. The term is often assumed to imp ...

pixel
of the image, without an image processor there would be a green pixel next to each red and blue pixel. (Actually, with most sensors there are two green for each blue and red diodes.) This process, however, is quite complex and involves a number of different operations. Its quality depends largely on the effectiveness of the algorithms applied to the raw data coming from the sensor. The mathematically manipulated data becomes the photo file recorded.


Demosaicing

As stated above, the image processor evaluates the color and brightness data of a given pixel, compares them with the data from neighboring pixels and then uses a
demosaicingA demosaicing (also de-mosaicing, demosaicking or debayering) algorithm of an algorithm (Euclid's algorithm) for calculating the greatest common divisor (g.c.d.) of two numbers ''a'' and ''b'' in locations named A and B. The algorithm proceeds by ...
algorithm to produce an appropriate colour and brightness value for the pixel. The image processor also assesses the whole picture to guess at the correct distribution of contrast. By adjusting the
gamma Gamma (uppercase , lowercase ; ''gámma'') is the third letter of the Greek alphabet. In the system of Greek numerals it has a value of 3. In Ancient Greek, the letter gamma represented a voiced velar stop . In Modern Greek, this letter rep ...
value (heightening or lowering the contrast range of an image's mid-tones) subtle tonal gradations, such as in human skin or the blue of the sky, become much more realistic.


Noise reduction

Noise Noise is unwanted sound In physics, sound is a vibration that propagates as an acoustic wave, through a transmission medium such as a gas, liquid or solid. In human physiology and psychology, sound is the ''reception'' of such waves and the ...
is a phenomenon found in any electronic circuitry. In digital photography its effect is often visible as random spots of obviously wrong colour in an otherwise smoothly-coloured area. Noise increases with temperature and exposure times. When higher
ISO The International Organization for Standardization (ISO ) is an international standard An international standard is a technical standard A technical standard is an established norm (social), norm or requirement for a repeatable technical task w ...
settings are chosen the electronic signal in the image sensor is amplified, which at the same time increases the noise level, leading to a lower
signal-to-noise ratio Signal-to-noise ratio (SNR or S/N) is a measure used in science and engineering Engineering is the use of scientific principles to design and build machines, structures, and other items, including bridges, tunnels, roads, vehicles, and ...
. The image processor attempts to separate the noise from the image information and to remove it. This can be quite a challenge, as the image may contain areas with fine textures which, if treated as noise, may lose some of their definition.


Image sharpening

As the color and brightness values for each pixel are
interpolated In the mathematics, mathematical field of numerical analysis, interpolation is a type of estimation, a method of constructing new data points within the range of a discrete set of known data points. In engineering and science, one often has a nu ...
some image softening is applied to even out any fuzziness that has occurred. To preserve the impression of depth, clarity and fine details, the image processor must sharpen edges and contours. It therefore must
detect edges
detect edges
correctly and reproduce them smoothly and without over-sharpening.


Models

Image processor users are using industry standard products, application-specific standard products (ASSP) or even
application-specific integrated circuit An application-specific integrated circuit (ASIC ) is an integrated circuit An integrated circuit or monolithic integrated circuit (also referred to as an IC, a chip, or a microchip) is a set of electronic circuit File:PExdcr01CJC.jpg, ...
s (ASIC) with trade names: Canon's is called
DIGIC Digital Imaging Integrated Circuit (often styled as "DiG!C") is 's name for a family of and s for s and s. DIGIC units are used as s by Canon in its own digital imaging products. Several generations of DIGICs exist, and are distinguished by a v ...
, Nikon's
Expeed The Nikon (, ; ), also known just as Nikon, is a Japanese multinational corporation headquartered in Tokyo, Japan, specializing in optics and Photography, imaging products. The companies held by Nikon form the Nikon Group. Nikon's products ...

Expeed
, Olympus' TruePic, Panasonic's
Venus Engine The Venus Engine is an image processor, image processing engine for digital cameras. It is developed by Panasonic, and almost all of their Lumix cameras use a version of it. It is based on the Panasonic MN103/MN103S. Image processors operate in ...
and Sony's Bionz. Some are known to be based on the
Fujitsu is a Japanese Multinational corporation, multinational information and communications technology equipment and services corporation, established in 1935 and headquartered in Tokyo. Fujitsu is the world's sixth-largest Information technology, I ...

Fujitsu
MilbeautThe Socionext Milbeaut image processor, image/Video processing#Video processor, video processors are media processors in Multiprocessor, multi-processor system on a chip architecture. Started by Fujitsu with the M-1 Series in 2000 each generation (2 ...
, the
Texas Instruments Texas Instruments Incorporated (TI) is an America The United States of America (U.S.A. or USA), commonly known as the United States (U.S. or US) or America, is a country primarily located in North America North America is a ...
OMAP The OMAP (Open Multimedia Applications Platform) family, developed by Texas Instruments Texas Instruments Incorporated (TI) is an America The United States of America (USA), commonly known as the United States (U.S. or US), or America ...
,
Panasonic formerly is a major Japanese multinational corporation, multinational Conglomerate (company), conglomerate company, headquartered in Kadoma, Osaka, Kadoma, Osaka Prefecture, Osaka. It was founded by Kōnosuke Matsushita in 1918 as a lightbulb ...

Panasonic
MN103, Zoran Coach, Altek Sunny or
Sanyo , stylized as SANYO, is a Japanese electronics company and formerly a member of the Fortune Global 500 The ''Fortune'' Global 500, also known as Global 500, is an annual ranking of the top 500 corporation A corporation is an organizat ...
image/video processors.
ARM architecture ARM (stylised in lowercase as arm, previously an acronym for Advanced RISC Machines and originally Acorn RISC Machine) is a family of reduced instruction set computing (RISC) instruction set, architectures for central processing unit, comp ...
processors with its NEON SIMD Media Processing Engines (MPE) are often used in
mobile phone A mobile phone, cellular phone, cell phone, cellphone, handphone, or hand phone, sometimes shortened to simply mobile, cell or just phone, is a portable telephone A telephone is a telecommunication Telecommunication is the tra ...

mobile phone
s.


Processor brand names

*ATI -
Imageon Imageon (previously ATI Imageon) was a series of media coprocessor A coprocessor is a computer processor used to supplement the functions of the primary processor (the CPU A central processing unit (CPU), also called a central processor, ma ...
(graphics co-processor used in many early mobile photos to offer camera image signal processing) *Canon -
DIGIC Digital Imaging Integrated Circuit (often styled as "DiG!C") is 's name for a family of and s for s and s. DIGIC units are used as s by Canon in its own digital imaging products. Several generations of DIGICs exist, and are distinguished by a v ...
(based on Texas Instruments
OMAP The OMAP (Open Multimedia Applications Platform) family, developed by Texas Instruments Texas Instruments Incorporated (TI) is an America The United States of America (USA), commonly known as the United States (U.S. or US), or America ...
) *Casio - EXILIM engine *Epson - EDiART *Fujifilm - EXR III or X Processor Pro *Google -
Pixel Visual CoreThe Pixel Visual Core (PVC) is a series of ARM architecture, ARM-based System in package, system in package (SiP) image processors designed by Google. The PVC is a fully programmable Image processor, image, Vision processing unit, vision and AI accel ...
*Minolta / Konica Minolta - SUPHEED with CxProcess *Leica - MAESTRO (based on Fujitsu
MilbeautThe Socionext Milbeaut image processor, image/Video processing#Video processor, video processors are media processors in Multiprocessor, multi-processor system on a chip architecture. Started by Fujitsu with the M-1 Series in 2000 each generation (2 ...
) *Nikon -
Expeed The Nikon (, ; ), also known just as Nikon, is a Japanese multinational corporation headquartered in Tokyo, Japan, specializing in optics and Photography, imaging products. The companies held by Nikon form the Nikon Group. Nikon's products ...

Expeed
(based on Fujitsu
MilbeautThe Socionext Milbeaut image processor, image/Video processing#Video processor, video processors are media processors in Multiprocessor, multi-processor system on a chip architecture. Started by Fujitsu with the M-1 Series in 2000 each generation (2 ...
)Milbeaut and EXPEED
byThom *Olympus - TruePic (based on Panasonic MN103/MN103S) *Panasonic -
Venus Engine The Venus Engine is an image processor, image processing engine for digital cameras. It is developed by Panasonic, and almost all of their Lumix cameras use a version of it. It is based on the Panasonic MN103/MN103S. Image processors operate in ...
(based on Panasonic MN103/MN103S) *Pentax - PRIME (Pentax Real IMage Engine) (newer variants based on Fujitsu
MilbeautThe Socionext Milbeaut image processor, image/Video processing#Video processor, video processors are media processors in Multiprocessor, multi-processor system on a chip architecture. Started by Fujitsu with the M-1 Series in 2000 each generation (2 ...
) *Qualcomm - Qualcomm Spectra *Ricoh - GR engine (GR digital), Smooth Imaging Engine *Samsung - DRIMe (based on
Samsung The Samsung Group (or simply Samsung) ( ko, 삼성) is a South Korean Multinational corporation, multinational manufacturing Conglomerate (company), conglomerate headquartered in Samsung Town, Seoul, South Korea. It comprises numerous affil ...
Exynos Exynos, formerly Hummingbird (), is a series of ARM In human anatomy, the arm is the part of the upper limb The upper Limb (anatomy), limbs or upper extremities are the forelimbs of an upright posture, upright-postured tetrapod vertebrate, ...
) *Sanyo - Platinum engine *Sigma - True *Sharp - ProPix *Socionext
Milbeaut
Family of ISPs - SC2000 (M-10V), SC2002 (M-11S) *Sony - Bionz *THine - THP serie

with compatible SDK Kit for developing firmwar

*HTC - ImageSense


Speed

With the ever-higher pixel count in image sensors, the image processor's speed becomes more critical: photographers don't want to wait for the camera's image processor to complete its job before they can carry on shooting - they don't even want to notice some processing is going on inside the camera. Therefore, image processors must be optimised to cope with more data in the same or even a shorter period of time.


See also

*
Color image pipelineAn image pipeline or video pipeline is the set of components commonly used between an image source (such as a camera, a scanner, or the rendering engine in a computer game), and an image renderer (such as a television set, a computer screen, a comput ...
*
Image processing Digital image processing is the use of a digital computer A computer is a machine A machine is a man-made device that uses power to apply forces and control movement to perform an action. Machines can be driven by animals and people ...
*
Digital image processing Digital image processing is the use of a digital computer A computer is a machine A machine is a man-made device that uses power to apply forces and control movement to perform an action. Machines can be driven by animals and people ...
*
Digital image editing Image editing encompasses the processes of altering image An SAR radar imaging, radar image acquired by the SIR-C/X-SAR radar on board the Space Shuttle Endeavour shows the Teide volcano. The city of Santa Cruz de Tenerife is visible a ...
*
DemosaicingA demosaicing (also de-mosaicing, demosaicking or debayering) algorithm of an algorithm (Euclid's algorithm) for calculating the greatest common divisor (g.c.d.) of two numbers ''a'' and ''b'' in locations named A and B. The algorithm proceeds by ...


References

{{System on a chip Digital signal processors Photography equipment