16:1 compression. Foveated image with fixation point at Stephen F. Austin statue.
Foveated imaging is a
digital image processing
Digital image processing is the use of a digital computer to process digital images through an algorithm. As a subcategory or field of digital signal processing, digital image processing has many advantages over analog image processing. It allo ...
technique in which the
image resolution
Image resolution is the detail an image holds. The term applies to digital images, film images, and other types of images. "Higher resolution" means more image detail.
Image resolution can be measured in various ways. Resolution quantifies how ...
, or amount of detail, varies across the
image
An image is a visual representation of something. It can be two-dimensional, three-dimensional, or somehow otherwise feed into the visual system to convey information. An image can be an artifact, such as a photograph or other two-dimensiona ...
according to one or more "fixation points". A fixation point indicates the highest resolution region of the image and corresponds to the center of the
eye
Eyes are organs of the visual system. They provide living organisms with vision, the ability to receive and process visual detail, as well as enabling several photo response functions that are independent of vision. Eyes detect light and conv ...
's
retina
The retina (from la, rete "net") is the innermost, light-sensitive layer of tissue of the eye of most vertebrates and some molluscs. The optics of the eye create a focused two-dimensional image of the visual world on the retina, which then ...
, the
fovea
Fovea () (Latin for "pit"; plural foveae ) is a term in anatomy. It refers to a pit or depression in a structure.
Human anatomy
*Fovea centralis of the retina
* Fovea buccalis or Dimple
* Fovea of the femoral head
* Trochlear fovea of the fr ...
.
The location of a fixation point may be specified in many ways.
For example, when viewing an image on a
computer monitor
A computer monitor is an output device that displays information in pictorial or textual form. A discrete monitor comprises a visual display, support electronics, power supply, housing, electrical connectors, and external user controls.
The di ...
, one may specify a fixation using a
pointing device
A pointing device is a human interface device that allows a user to input spatial (i.e., continuous and multi-dimensional) data to a computer. CAD systems and graphical user interfaces (GUI) allow the user to control and provide data to t ...
, like a computer mouse.
Eye tracker
Eye tracking is the process of measuring either the point of gaze (where one is looking) or the motion of an eye relative to the head. An eye tracker is a device for measuring eye positions and eye movement. Eye trackers are used in research ...
s which precisely measure the eye's position and movement are also commonly used to determine fixation points in perception experiments.
When the display is manipulated with the use of an eye tracker, this is known as a
gaze contingent display.
Fixations may also be determined automatically using
computer algorithm
In mathematics and computer science, an algorithm () is a finite sequence of rigorous instructions, typically used to solve a class of specific problems or to perform a computation. Algorithms are used as specifications for performing ca ...
s.
Some common applications of foveated imaging include imaging sensor hardware and image compression. For descriptions of these and other applications, see the list below.
Foveated imaging is also commonly referred to as space variant imaging or gaze contingent imaging.
Applications
Foveated imaging for progressive transmission
Compression
Contrast sensitivity
Contrast is the contradiction in luminance or colour that makes an object (or its representation in an image or display) distinguishable. In visual perception of the real world, contrast is determined by the difference in the colour and brightn ...
falls off dramatically as one moves from the center of the retina to the periphery.
In
lossy image compression, one may take advantage of this fact in order to compactly encode images.
If one knows the viewer's approximate point of gaze, one may reduce the amount of information contained in the image as the distance from the point of gaze increases. Because the fall-off in the eye's resolution is dramatic, the potential reduction in display information can be substantial. Also, foveation encoding may be applied to the image before other types of image compression are applied and therefore can result in a multiplicative reduction.
Foveated sensors
Foveated sensors are multiresolution hardware devices that allow image data to be collected with higher resolution concentrated at a fixation point. An advantage to using foveated sensor hardware is that the image collection and encoding can occur much faster than in a system that post-processes a high resolution image in software.
Simulation
Foveated imaging has been used to simulate visual fields with arbitrary spatial resolution. For example, one may present video containing a blurred region representing a
scotoma
A scotoma is an area of partial alteration in the field of vision consisting of a partially diminished or entirely degenerated visual acuity that is surrounded by a field of normal – or relatively well-preserved – vision.
Every normal mam ...
. By using an eye-tracker and holding the blurred region fixed relative to the viewer's gaze, the viewer will have a visual experience similar to that of a person with an actual scotoma. The figure on the right shows a frame from a simulation of a glaucoma patient with the eye fixated on the word "similar."
Video gaming
Foveated rendering
Foveated rendering is a rendering technique which uses an eye tracker integrated with a virtual reality headset to reduce the rendering workload by greatly reducing the image quality in the peripheral vision (outside of the zone gazed by the fovea ...
is an upcoming
video game
Video games, also known as computer games, are electronic games that involves interaction with a user interface or input device such as a joystick, controller, keyboard, or motion sensing device to generate visual feedback. This fee ...
technique which uses an
eye tracker
Eye tracking is the process of measuring either the point of gaze (where one is looking) or the motion of an eye relative to the head. An eye tracker is a device for measuring eye positions and eye movement. Eye trackers are used in research ...
integrated with a
virtual reality headset
A virtual reality headset (or VR headset) is a head-mounted device that provides virtual reality for the wearer. VR headsets are widely used with VR video games but they are also used in other applications, including simulators and trainers. VR ...
to reduce the rendering workload by greatly reducing the image quality in the
peripheral vision
Peripheral vision, or ''indirect vision'', is vision as it occurs outside the point of fixation, i.e. away from the center of gaze or, when viewed at large angles, in (or out of) the "corner of one's eye". The vast majority of the area in the ...
(outside of the zone gazed by the
fovea
Fovea () (Latin for "pit"; plural foveae ) is a term in anatomy. It refers to a pit or depression in a structure.
Human anatomy
*Fovea centralis of the retina
* Fovea buccalis or Dimple
* Fovea of the femoral head
* Trochlear fovea of the fr ...
).
At the
CES 2016
CES (; formerly an initialism for Consumer Electronics Show) is an annual trade show organized by the Consumer Technology Association (CTA). Held in January at the Las Vegas Convention Center in Winchester, Nevada, United States, the event typi ...
,
SensoMotoric Instruments (SMI) demoed a new 250 Hz eye tracking system and a working foveated rendering solution. It resulted from a partnership with camera sensor manufacturer
Omnivision
OmniVision Technologies Inc. is an American subsidiary of Chinese semiconductor device and mixed-signal integrated circuit design house Will Semiconductor. The company designs and develops digital imaging products for use in mobile phones, not ...
who provided the camera hardware for the new system.
Quality assessment
Foveated imaging may be useful in providing a subjective image quality measure.
[Z. Wang, A. C. Bovik, L. Lu and J. Kouloheris, "Foveated wavelet image quality index," SPIE's 46th Annual Meeting, Proc. SPIE, Application of digital image processing XXIV, vol. 4472, July-Aug. 2001.] Traditional image quality measures, such as
peak signal-to-noise ratio
Peak signal-to-noise ratio (PSNR) is an engineering term for the ratio between the maximum possible power of a signal and the power of corrupting noise that affects the fidelity of its representation. Because many signals have a very wide dynamic ...
, are typically performed on fixed resolution images and do not take into account some aspects of the human visual system, like the change in spatial resolution across the retina. A foveated quality index may therefore more accurately determine image quality as perceived by humans.
Image database retrieval
In databases that contain very high resolution images, such as a
satellite image
Satellite images (also Earth observation imagery, spaceborne photography, or simply satellite photo) are images of Earth collected by imaging satellites operated by governments and businesses around the world. Satellite imaging companies sell ima ...
database, it may be desirable to interactively retrieve images in order to reduce retrieval time. Foveated imaging allows one to scan low resolution images and retrieve only high resolution portions as they are needed. This is sometimes called progressive transmission.
Example images
Image:Foveated_tombstones.png, Image with foveated luminance channel
Image:FoveatedLandscape.png, Foveated image with fixation selected using an entropy minimization algorithm
image:Texas_state_cemetery_foveated2.png, 18:1 compression. Foveated image with fixation point on tombstone.
See also
*
Fovea
Fovea () (Latin for "pit"; plural foveae ) is a term in anatomy. It refers to a pit or depression in a structure.
Human anatomy
*Fovea centralis of the retina
* Fovea buccalis or Dimple
* Fovea of the femoral head
* Trochlear fovea of the fr ...
*
Gaze-contingency paradigm
Within computer technology, the gaze-contingency paradigm is a general term for techniques allowing a computer screen display to change in function depending on where the viewer is looking. Gaze-contingent techniques are part of the eye movement f ...
*
Human visual system model
A human visual system model (HVS model) is used by image processing, video processing and computer vision experts to deal with biological and psychological processes that are not yet fully understood. Such a model is used to simplify the behav ...
*
Low vision
Visual impairment, also known as vision impairment, is a medical definition primarily measured based on an individual's better eye visual acuity; in the absence of treatment such as correctable eyewear, assistive devices, and medical treatment†...
*
Image compression
Image compression is a type of data compression applied to digital images, to reduce their cost for storage or transmission. Algorithms may take advantage of visual perception and the statistical properties of image data to provide superior r ...
*
Digital image processing
Digital image processing is the use of a digital computer to process digital images through an algorithm. As a subcategory or field of digital signal processing, digital image processing has many advantages over analog image processing. It allo ...
*
Eye tracker
Eye tracking is the process of measuring either the point of gaze (where one is looking) or the motion of an eye relative to the head. An eye tracker is a device for measuring eye positions and eye movement. Eye trackers are used in research ...
*
Eye movement in language reading
Eye movement in reading involves the visual processing of written text. This was described by the French ophthalmologist Louis Émile Javal in the late 19th century. He reported that eyes do not move continuously along a line of text, but make shor ...
References
{{Reflist, 2
External links
* http://live.ece.utexas.edu
* http://svi.cps.utexas.edu
* http://webvision.med.utah.edu/
Computer vision, Machine Vision, Robotics