16:1 compression. Foveated image with fixation point at Stephen F. Austin statue. Foveated imaging is a

digital image processing Digital image processing is the use of a digital computer to process digital images through an algorithm. As a subcategory or field of digital signal processing, digital image processing has many advantages over analog image processing. It allo ...

technique in which the

image resolution Image resolution is the detail an image holds. The term applies to digital images, film images, and other types of images. "Higher resolution" means more image detail. Image resolution can be measured in various ways. Resolution quantifies how ...

, or amount of detail, varies across the

image An image is a visual representation of something. It can be two-dimensional, three-dimensional, or somehow otherwise feed into the visual system to convey information. An image can be an artifact, such as a photograph or other two-dimensiona ...

according to one or more "fixation points". A fixation point indicates the highest resolution region of the image and corresponds to the center of the

eye Eyes are organs of the visual system. They provide living organisms with vision, the ability to receive and process visual detail, as well as enabling several photo response functions that are independent of vision. Eyes detect light and conv ...

retina The retina (from la, rete "net") is the innermost, light-sensitive layer of tissue of the eye of most vertebrates and some molluscs. The optics of the eye create a focused two-dimensional image of the visual world on the retina, which then ...

, the

fovea Fovea () (Latin for "pit"; plural foveae ) is a term in anatomy. It refers to a pit or depression in a structure. Human anatomy *Fovea centralis of the retina * Fovea buccalis or Dimple * Fovea of the femoral head * Trochlear fovea of the fr ...

. The location of a fixation point may be specified in many ways. For example, when viewing an image on a

computer monitor A computer monitor is an output device that displays information in pictorial or textual form. A discrete monitor comprises a visual display, support electronics, power supply, housing, electrical connectors, and external user controls. The di ...

, one may specify a fixation using a

pointing device A pointing device is a human interface device that allows a user to input spatial (i.e., continuous and multi-dimensional) data to a computer. CAD systems and graphical user interfaces (GUI) allow the user to control and provide data to t ...

, like a computer mouse.

Eye tracker Eye tracking is the process of measuring either the point of gaze (where one is looking) or the motion of an eye relative to the head. An eye tracker is a device for measuring eye positions and eye movement. Eye trackers are used in research ...

s which precisely measure the eye's position and movement are also commonly used to determine fixation points in perception experiments. When the display is manipulated with the use of an eye tracker, this is known as a gaze contingent display. Fixations may also be determined automatically using

computer algorithm In mathematics and computer science, an algorithm () is a finite sequence of rigorous instructions, typically used to solve a class of specific problems or to perform a computation. Algorithms are used as specifications for performing ca ...

s. Some common applications of foveated imaging include imaging sensor hardware and image compression. For descriptions of these and other applications, see the list below. Foveated imaging is also commonly referred to as space variant imaging or gaze contingent imaging.

Applications

Foveated imaging for progressive transmission

Compression

Contrast sensitivity Contrast is the contradiction in luminance or colour that makes an object (or its representation in an image or display) distinguishable. In visual perception of the real world, contrast is determined by the difference in the colour and brightn ...

falls off dramatically as one moves from the center of the retina to the periphery. In lossy image compression, one may take advantage of this fact in order to compactly encode images. If one knows the viewer's approximate point of gaze, one may reduce the amount of information contained in the image as the distance from the point of gaze increases. Because the fall-off in the eye's resolution is dramatic, the potential reduction in display information can be substantial. Also, foveation encoding may be applied to the image before other types of image compression are applied and therefore can result in a multiplicative reduction.

Foveated sensors

Foveated sensors are multiresolution hardware devices that allow image data to be collected with higher resolution concentrated at a fixation point. An advantage to using foveated sensor hardware is that the image collection and encoding can occur much faster than in a system that post-processes a high resolution image in software.

Simulation

Foveated imaging has been used to simulate visual fields with arbitrary spatial resolution. For example, one may present video containing a blurred region representing a

scotoma A scotoma is an area of partial alteration in the field of vision consisting of a partially diminished or entirely degenerated visual acuity that is surrounded by a field of normal – or relatively well-preserved – vision. Every normal mam ...

. By using an eye-tracker and holding the blurred region fixed relative to the viewer's gaze, the viewer will have a visual experience similar to that of a person with an actual scotoma. The figure on the right shows a frame from a simulation of a glaucoma patient with the eye fixated on the word "similar."

Video gaming

Foveated rendering Foveated rendering is a rendering technique which uses an eye tracker integrated with a virtual reality headset to reduce the rendering workload by greatly reducing the image quality in the peripheral vision (outside of the zone gazed by the fovea ...

is an upcoming

video game Video games, also known as computer games, are electronic games that involves interaction with a user interface or input device such as a joystick, controller, keyboard, or motion sensing device to generate visual feedback. This fee ...

technique which uses an

eye tracker Eye tracking is the process of measuring either the point of gaze (where one is looking) or the motion of an eye relative to the head. An eye tracker is a device for measuring eye positions and eye movement. Eye trackers are used in research ...

integrated with a

virtual reality headset A virtual reality headset (or VR headset) is a head-mounted device that provides virtual reality for the wearer. VR headsets are widely used with VR video games but they are also used in other applications, including simulators and trainers. VR ...

to reduce the rendering workload by greatly reducing the image quality in the

peripheral vision Peripheral vision, or ''indirect vision'', is vision as it occurs outside the point of fixation, i.e. away from the center of gaze or, when viewed at large angles, in (or out of) the "corner of one's eye". The vast majority of the area in the ...

(outside of the zone gazed by the

). At the

CES 2016 CES (; formerly an initialism for Consumer Electronics Show) is an annual trade show organized by the Consumer Technology Association (CTA). Held in January at the Las Vegas Convention Center in Winchester, Nevada, United States, the event typi ...

, SensoMotoric Instruments (SMI) demoed a new 250 Hz eye tracking system and a working foveated rendering solution. It resulted from a partnership with camera sensor manufacturer

Omnivision OmniVision Technologies Inc. is an American subsidiary of Chinese semiconductor device and mixed-signal integrated circuit design house Will Semiconductor. The company designs and develops digital imaging products for use in mobile phones, not ...

who provided the camera hardware for the new system.

Quality assessment

Foveated imaging may be useful in providing a subjective image quality measure.Z. Wang, A. C. Bovik, L. Lu and J. Kouloheris, "Foveated wavelet image quality index," SPIE's 46th Annual Meeting, Proc. SPIE, Application of digital image processing XXIV, vol. 4472, July-Aug. 2001. Traditional image quality measures, such as

peak signal-to-noise ratio Peak signal-to-noise ratio (PSNR) is an engineering term for the ratio between the maximum possible power of a signal and the power of corrupting noise that affects the fidelity of its representation. Because many signals have a very wide dynamic ...

, are typically performed on fixed resolution images and do not take into account some aspects of the human visual system, like the change in spatial resolution across the retina. A foveated quality index may therefore more accurately determine image quality as perceived by humans.

Image database retrieval

In databases that contain very high resolution images, such as a

satellite image Satellite images (also Earth observation imagery, spaceborne photography, or simply satellite photo) are images of Earth collected by imaging satellites operated by governments and businesses around the world. Satellite imaging companies sell ima ...

database, it may be desirable to interactively retrieve images in order to reduce retrieval time. Foveated imaging allows one to scan low resolution images and retrieve only high resolution portions as they are needed. This is sometimes called progressive transmission.

Example images

Image:Foveated_tombstones.png, Image with foveated luminance channel Image:FoveatedLandscape.png, Foveated image with fixation selected using an entropy minimization algorithm image:Texas_state_cemetery_foveated2.png, 18:1 compression. Foveated image with fixation point on tombstone.

References

{{Reflist, 2

External links

* http://live.ece.utexas.edu * http://svi.cps.utexas.edu * http://webvision.med.utah.edu/ Computer vision, Machine Vision, Robotics