HOME

TheInfoList



OR:

Computational photography refers to digital image capture and processing techniques that use digital computation instead of optical processes. Computational photography can improve the capabilities of a camera, or introduce features that were not possible at all with film based photography, or reduce the cost or size of camera elements. Examples of computational photography include in-camera computation of digital
panoramas A panorama (formed from Greek πᾶν "all" + ὅραμα "view") is any wide-angle view or representation of a physical space, whether in painting, drawing, photography, film, seismic images, or 3D modeling. The word was originally coined in ...
, high-dynamic-range images, and light field cameras. Light field cameras use novel optical elements to capture three dimensional scene information which can then be used to produce 3D images, enhanced
depth-of-field The depth of field (DOF) is the distance between the nearest and the furthest objects that are in acceptably sharp focus in an image captured with a camera. Factors affecting depth of field For cameras that can only focus on one object dist ...
, and selective de-focusing (or "post focus"). Enhanced depth-of-field reduces the need for mechanical focusing systems. All of these features use computational imaging techniques. The definition of computational photography has evolved to cover a number of subject areas in
computer graphics Computer graphics deals with generating images with the aid of computers. Today, computer graphics is a core technology in digital photography, film, video games, cell phone and computer displays, and many specialized applications. A great de ...
,
computer vision Computer vision is an interdisciplinary scientific field that deals with how computers can gain high-level understanding from digital images or videos. From the perspective of engineering, it seeks to understand and automate tasks that the hum ...
, and applied
optics Optics is the branch of physics that studies the behaviour and properties of light, including its interactions with matter and the construction of instruments that use or detect it. Optics usually describes the behaviour of visible, ultraviole ...
. These areas are given below, organized according to a taxonomy proposed by Shree K. Nayar. Within each area is a list of techniques, and for each technique one or two representative papers or books are cited. Deliberately omitted from the taxonomy are
image processing An image is a visual representation of something. It can be two-dimensional, three-dimensional, or somehow otherwise feed into the visual system to convey information. An image can be an artifact, such as a photograph or other two-dimensiona ...
(see also
digital image processing Digital image processing is the use of a digital computer to process digital images through an algorithm. As a subcategory or field of digital signal processing, digital image processing has many advantages over analog image processing. It allo ...
) techniques applied to traditionally captured images in order to produce better images. Examples of such techniques are
image scaling In computer graphics and digital imaging, image scaling refers to the resizing of a digital image. In video technology, the magnification of digital material is known as upscaling or resolution enhancement. When scaling a vector graphic image, ...
, dynamic range compression (i.e.
tone mapping Tone mapping is a technique used in image processing and computer graphics to map one set of colors to another to approximate the appearance of high-dynamic-range images in a medium that has a more limited dynamic range. Print-outs, CRT or L ...
),
color management In digital imaging systems, color management (or colour management) is the controlled conversion between the color representations of various devices, such as image scanners, digital cameras, monitors, TV screens, film printers, computer printer ...
, image completion (a.k.a. inpainting or hole filling),
image compression Image compression is a type of data compression applied to digital images, to reduce their cost for storage or transmission. Algorithms may take advantage of visual perception and the statistical properties of image data to provide superior r ...
,
digital watermarking A digital watermark is a kind of marker covertly embedded in a noise-tolerant signal such as audio, video or image data. It is typically used to identify ownership of the copyright of such signal. "Watermarking" is the process of hiding digital inf ...
, and artistic image effects. Also omitted are techniques that produce range data, volume data,
3D model In 3D computer graphics, 3D modeling is the process of developing a mathematical coordinate-based representation of any surface of an object (inanimate or living) in three dimensions via specialized software by manipulating edges, vertices, an ...
s, 4D light fields, 4D, 6D, or 8D
BRDF The bidirectional reflectance distribution function (BRDF; f_(\omega_,\, \omega_) ) is a function of four real variables that defines how light is reflected at an opaque surface. It is employed in the optics of real-world light, in compute ...
s, or other high-dimensional image-based representations.
Epsilon photography Epsilon photography is a form of computational photography wherein multiple images are captured with slightly varying camera parameters (each image varying the parameter by a small amount ''ε'', hence the name) such as aperture, exposure, focus, ...
is a sub-field of computational photography.


Effect on photography

Photos taken using computational photography can allow amateurs to produce photographs rivalling the quality of professional photographers, but currently (2019) do not outperform the use of professional-level equipment.


Computational illumination

This is controlling photographic illumination in a structured fashion, then processing the captured images, to create new images. The applications include image-based relighting, image enhancement,
image deblurring Deblurring is the process of removing blurring artifacts from images. Deblurring recovers a sharp image ''S'' from a blurred image ''B'', where ''S'' is convolved with ''K'' (the blur kernel) to generate ''B''. Mathematically, this can be represen ...
, geometry/material recovery and so forth. High-dynamic-range imaging uses differently exposed pictures of the same scene to extend dynamic range. Other examples include processing and merging differently illuminated images of the same subject matter ("lightspace").


Computational optics

This is capture of optically coded images, followed by computational decoding to produce new images.
Coded aperture Coded apertures or coded-aperture masks are grids, gratings, or other patterns of materials opaque to various wavelengths of electromagnetic radiation. The wavelengths are usually high-energy radiation such as X-rays and gamma rays. By blocking ra ...
imaging was mainly applied in astronomy or X-ray imaging to boost the image quality. Instead of a single pin-hole, a pinhole pattern is applied in imaging, and
deconvolution In mathematics, deconvolution is the operation inverse to convolution. Both operations are used in signal processing and image processing. For example, it may be possible to recover the original signal after a filter (convolution) by using a deco ...
is performed to recover the image. In coded exposure imaging, the on/off state of the shutter is coded to modify the kernel of
motion blur Motion blur is the apparent streaking of moving objects in a photograph or a sequence of frames, such as a film or animation. It results when the image being recorded changes during the recording of a single exposure, due to rapid movement or lo ...
. In this way motion deblurring becomes a well-conditioned problem. Similarly, in a lens based coded aperture, the aperture can be modified by inserting a
broadband mask In telecommunications, broadband is wide bandwidth data transmission which transports multiple signals at a wide range of frequencies and Internet traffic types, that enables messages to be sent simultaneously, used in fast internet connections. ...
. Thus, out of focus deblurring becomes a well-conditioned problem. The coded aperture can also improve the quality in light field acquisition using Hadamard transform optics. Coded aperture patterns can also be designed using color filters, in order to apply different codes at different wavelengths. This allows to increase the amount of light that reaches the camera sensor, compared to binary masks.


Computational imaging

Computational imaging is a set of imaging techniques that combine data acquisition and data processing to create the image of an object through indirect means to yield enhanced resolution, additional information such as optical phase or
3D reconstruction In computer vision and computer graphics, 3D reconstruction is the process of capturing the shape and appearance of real objects. This process can be accomplished either by active or passive methods. If the model is allowed to change its shape i ...
. The information is often recorded without using a conventional optical microscope configuration or with limited datasets. Computational imaging allows to go beyond physical limitations of optical systems, such as
numerical aperture In optics, the numerical aperture (NA) of an optical system is a dimensionless number that characterizes the range of angles over which the system can accept or emit light. By incorporating index of refraction in its definition, NA has the proper ...
, or even obliterates the need for optical elements. For parts of the
optical spectrum The visible spectrum is the portion of the electromagnetic spectrum that is visible to the human eye. Electromagnetic radiation in this range of wavelengths is called ''visible light'' or simply light. A typical human eye will respond to wavele ...
where imaging elements such as objectives are difficult to manufacture or
image sensors An image sensor or imager is a sensor that detects and conveys information used to make an image. It does so by converting the variable attenuation of light waves (as they pass through or reflect off objects) into signals, small bursts of curr ...
cannot be miniaturized, computational imaging provides useful alternatives, in fields such as
X-ray An X-ray, or, much less commonly, X-radiation, is a penetrating form of high-energy electromagnetic radiation. Most X-rays have a wavelength ranging from 10  picometers to 10  nanometers, corresponding to frequencies in the range 30&nb ...
and THz radiations.


Common techniques

Among common computational imaging techniques are lensless imaging, computational speckle imaging,Katz et al.
"Non-invasive single-shot imaging through scattering layers and around corners via speckle correlations"
''Nature Photonics'' 8, 784–790 (2014)
ptychography Ptychography (/t(ʌ)ɪˈkogræfi/ t(a)i-KO-graf-ee) is a computational method of microscopic imaging. It generates images by processing many coherent interference patterns that have been scattered from an object of interest. Its defining cha ...
and
Fourier ptychography Fourier ptychography is a computational imaging technique based on optical microscopy that consists in the synthesis of a wider numerical aperture from a set of full-field images acquired at various coherent illumination angles, resulting in in ...
. Computational imaging technique often draws on
compressive sensing Compressed sensing (also known as compressive sensing, compressive sampling, or sparse sampling) is a signal processing technique for efficiently acquiring and reconstructing a Signal (electronics), signal, by finding solutions to Underdetermined ...
or
phase retrieval Phase retrieval is the process of algorithmically finding solutions to the phase problem. Given a complex signal F(k), of amplitude , F (k), , and phase \psi(k): ::F(k) = , F(k), e^ =\int_^ f(x)\ e^\,dx where ''x'' is an ''M''-dimensional spatia ...
techniques, where the angular spectrum of the object is being reconstructed. Other techniques are related to the field of computational imaging, such as
digital holography Digital holography refers to the acquisition and processing of holograms with a digital sensor array, typically a CCD camera or a similar device. Image rendering, or reconstruction of object ''data'' is performed numerically from digitized interfero ...
,
computer vision Computer vision is an interdisciplinary scientific field that deals with how computers can gain high-level understanding from digital images or videos. From the perspective of engineering, it seeks to understand and automate tasks that the hum ...
and inverse problems such as
tomography Tomography is imaging by sections or sectioning that uses any kind of penetrating wave. The method is used in radiology, archaeology, biology, atmospheric science, geophysics, oceanography, plasma physics, materials science, astrophysics, quantu ...
.


Computational processing

This is processing of non-optically-coded images to produce new images.


Computational sensors

These are detectors that combine sensing and processing, typically in hardware, like the
oversampled binary image sensor An oversampled binary image sensor is an image sensor with non-linear response capabilities reminiscent of traditional photographic film. Each pixel in the sensor has a binary response, giving only a one-bit quantized measurement of the local ligh ...
.


Early work in computer vision

Although computational photography is a currently popular buzzword in computer graphics, many of its techniques first appeared in the computer vision literature, either under other names or within papers aimed at 3D shape analysis.


Art history

Computational photography, as an art form, has been practiced by capture of differently exposed pictures of the same subject matter, and combining them together. This was the inspiration for the development of the
wearable computer A wearable computer, also known as a body-borne computer, is a computing device worn on the body. The definition of 'wearable computer' may be narrow or broad, extending to smartphones or even ordinary wristwatches. Wearables may be for general ...
in the 1970s and early 1980s. Computational photography was inspired by the work of Charles Wyckoff, and thus computational photography datasets (e.g. differently exposed pictures of the same subject matter that are taken in order to make a single composite image) are sometimes referred to as Wyckoff Sets, in his honor. Early work in this area (joint estimation of image projection and exposure value) was undertaken by Mann and Candoccia. Charles Wyckoff devoted much of his life to creating special kinds of 3-layer photographic films that captured different exposures of the same subject matter. A picture of a nuclear explosion, taken on Wyckoff's film, appeared on the cover of
Life Magazine ''Life'' was an American magazine published weekly from 1883 to 1972, as an intermittent "special" until 1978, and as a monthly from 1978 until 2000. During its golden age from 1936 to 1972, ''Life'' was a wide-ranging weekly general-interest ma ...
and showed the dynamic range from dark outer areas to inner core.


See also

*
Adaptive optics Adaptive optics (AO) is a technology used to improve the performance of optical systems by reducing the effect of incoming wavefront distortions by deforming a mirror in order to compensate for the distortion. It is used in astronomical tele ...
*
Multispectral imaging Multispectral imaging captures image data within specific wavelength ranges across the electromagnetic spectrum. The wavelengths may be separated by filters or detected with the use of instruments that are sensitive to particular wavelengths, ...
*
Simultaneous localization and mapping Simultaneous localization and mapping (SLAM) is the computational problem of constructing or updating a map of an unknown environment while simultaneously keeping track of an agent's location within it. While this initially appears to be a chick ...
*
Super-resolution microscopy Super-resolution microscopy is a series of techniques in optical microscopy that allow such images to have resolutions higher than those imposed by the diffraction limit, which is due to the diffraction of light. Super-resolution imaging techni ...
*
Time-of-flight camera A time-of-flight camera (ToF camera), also known as time-of-flight sensor (ToF sensor), is a range imaging camera system for measuring distances between the camera and the subject for each point of the image based on time-of-flight, the round tr ...


References


External links

* Nayar, Shree K. (2007)
"Computational Cameras"
''Conference on Machine Vision Applications''.
''Computational Photography'' (Raskar, R., Tumblin, J.,)
A.K. Peters. In press.
Special issue on Computational Photography
IEEE Computer, August 2006.
Camera Culture and Computational Journalism: Capturing and Sharing Visual Experiences
{{Webarchive, url=https://web.archive.org/web/20150906055242/http://www.computer.org/portal/web/computingnow/cgacfp1 , date=2015-09-06 , IEEE CG&A Special Issue, Feb 2011. * Rick Szeliski (2010),
Computer Vision: Algorithms and Applications
', Springer. * Computational Photography: Methods and Applications (Ed. Rastislav Lukac), CRC Press, 2010.

(John Wiley and Sons book information).


GJB-1: Increasing the dynamic range of a digital camera by using the Wyckoff principleExamples of wearable computational photography as an art formSiggraph Course in Computational Photography
Digital photography Computational fields of study Computer vision