Computational photography refers to digital image capture and processing techniques that use digital computation instead of optical processes. Computational photography can improve the capabilities of a camera, or introduce features that were not possible at all with film based photography, or reduce the cost or size of camera elements. Examples of computational photography include in-camera computation of digital
panoramas,
high-dynamic-range images, and
light field cameras. Light field cameras use novel optical elements to capture three dimensional scene information which can then be used to produce 3D images, enhanced
depth-of-field
The depth of field (DOF) is the distance between the nearest and the furthest objects that are in acceptably sharp focus in an image captured with a camera.
Factors affecting depth of field
For cameras that can only focus on one object dist ...
, and selective de-focusing (or "post focus"). Enhanced depth-of-field reduces the need for mechanical
focusing systems. All of these features use computational imaging techniques.
The definition of computational photography has evolved to cover a number of
subject areas in
computer graphics,
computer vision
Computer vision is an interdisciplinary scientific field that deals with how computers can gain high-level understanding from digital images or videos. From the perspective of engineering, it seeks to understand and automate tasks that the human ...
, and applied
optics. These areas are given below, organized according to a taxonomy
proposed by
Shree K. Nayar. Within each area is a list of techniques, and for
each technique one or two representative papers or books are cited.
Deliberately omitted from the
taxonomy are
image processing
An image is a visual representation of something. It can be two-dimensional, three-dimensional, or somehow otherwise feed into the visual system to convey information. An image can be an artifact, such as a photograph or other two-dimension ...
(see also
digital image processing)
techniques applied to traditionally captured
images in order to produce better images. Examples of such techniques are
image scaling, dynamic range compression (i.e.
tone mapping),
color management
In digital imaging systems, color management (or colour management) is the controlled conversion between the color representations of various devices, such as image scanners, digital cameras, monitors, TV screens, film printers, computer printe ...
, image completion (a.k.a. inpainting or hole filling),
image compression,
digital watermarking, and artistic image effects.
Also omitted are techniques that produce
range data,
volume data,
3D model
In 3D computer graphics, 3D modeling is the process of developing a mathematical coordinate-based representation of any surface of an object (inanimate or living) in three dimensions via specialized software by manipulating edges, vertices, an ...
s,
4D light fields,
4D, 6D, or 8D
BRDF
The bidirectional reflectance distribution function (BRDF; f_(\omega_,\, \omega_) ) is a function of four real variables that defines how light is reflected at an opaque surface. It is employed in the optics of real-world light, in computer ...
s, or other high-dimensional image-based representations.
Epsilon photography is a sub-field of computational photography.
Effect on photography
Photos taken using computational photography can allow amateurs to produce photographs rivalling the quality of professional photographers, but currently (2019) do not outperform the use of professional-level equipment.
Computational illumination
This is controlling photographic illumination in a structured fashion, then processing the captured images,
to create new images. The applications include image-based relighting, image enhancement,
image deblurring, geometry/material recovery and so forth.
High-dynamic-range imaging uses differently exposed pictures of the same scene to extend dynamic range. Other examples include processing and merging differently illuminated images of the same subject matter ("lightspace").
Computational optics
This is capture of optically coded images, followed by computational decoding to produce new images.
Coded aperture imaging was mainly applied in astronomy or X-ray imaging to boost the image quality. Instead of a single pin-hole, a pinhole pattern is applied in imaging, and
deconvolution is performed to recover the image. In
coded exposure imaging, the on/off state of the shutter is coded to modify the kernel of
motion blur. In this way motion deblurring becomes a
well-conditioned problem. Similarly, in a lens based coded aperture, the aperture can be modified by inserting a
broadband mask. Thus, out of focus deblurring becomes a well-conditioned problem. The coded aperture can also improve the quality in light field acquisition using Hadamard transform optics.
Coded aperture patterns can also be designed using color filters, in order to apply different codes at different wavelengths. This allows to increase the amount of light that reaches the camera sensor, compared to binary masks.
Computational imaging
Computational imaging is a set of imaging techniques that combine data acquisition and data processing to create the image of an object through indirect means to yield enhanced resolution, additional information such as optical phase or
3D reconstruction
In computer vision and computer graphics, 3D reconstruction is the process of capturing the shape and appearance of real objects.
This process can be accomplished either by active or passive methods. If the model is allowed to change its shape i ...
. The information is often recorded without using a
conventional optical microscope configuration or with limited datasets.
Computational imaging allows to go beyond physical limitations of optical systems, such as
numerical aperture, or even obliterates the need for
optical elements.
For parts of the
optical spectrum
The visible spectrum is the portion of the electromagnetic spectrum that is visible to the human eye. Electromagnetic radiation in this range of wavelengths is called ''visible light'' or simply light. A typical human eye will respond to wavele ...
where imaging elements such as objectives are difficult to manufacture or
image sensors cannot be miniaturized, computational imaging provides useful alternatives, in fields such as
X-ray
An X-ray, or, much less commonly, X-radiation, is a penetrating form of high-energy electromagnetic radiation. Most X-rays have a wavelength ranging from 10 picometers to 10 nanometers, corresponding to frequencies in the range 30  ...
and
THz radiations.
Common techniques
Among common computational imaging techniques are
lensless imaging, computational speckle imaging,
[Katz et al.]
"Non-invasive single-shot imaging through scattering layers and around corners via speckle correlations"
''Nature Photonics'' 8, 784–790 (2014) ptychography
Ptychography (/t(ʌ)ɪˈkogræfi/ t(a)i-KO-graf-ee) is a computational method of microscopic imaging. It generates images by processing many coherent interference patterns that have been scattered from an object of interest. Its defining ch ...
and
Fourier ptychography
Fourier ptychography is a computational imaging technique based on optical microscopy that consists in the synthesis of a wider numerical aperture from a set of full-field images acquired at various coherent illumination angles,
resulting in inc ...
.
Computational imaging technique often draws on
compressive sensing
Compressed sensing (also known as compressive sensing, compressive sampling, or sparse sampling) is a signal processing technique for efficiently acquiring and reconstructing a Signal (electronics), signal, by finding solutions to Underdetermined ...
or
phase retrieval techniques, where the angular spectrum of the object is being reconstructed. Other techniques are related to the field of computational imaging, such as
digital holography Digital holography refers to the acquisition and processing of holograms with a digital sensor array, typically a CCD camera or a similar device. Image rendering, or reconstruction of object ''data'' is performed numerically from digitized interfero ...
,
computer vision
Computer vision is an interdisciplinary scientific field that deals with how computers can gain high-level understanding from digital images or videos. From the perspective of engineering, it seeks to understand and automate tasks that the human ...
and inverse problems such as
tomography.
Computational processing
This is processing of non-optically-coded images to produce new images.
Computational sensors
These are detectors that combine sensing and processing, typically in hardware, like the
oversampled binary image sensor.
Early work in computer vision
Although computational photography is a currently popular buzzword in computer graphics, many of its
techniques first appeared in the computer vision literature,
either under other names or within papers aimed at 3D shape analysis.
Art history
Computational photography, as an art form, has been practiced by capture of differently exposed pictures of the same subject matter, and combining them together. This was the inspiration for the development of the
wearable computer in the 1970s and early 1980s. Computational photography was inspired by the work of
Charles Wyckoff, and thus computational photography datasets (e.g. differently exposed pictures of the same subject matter that are taken in order to make a single composite image) are sometimes referred to as Wyckoff Sets, in his honor.
Early work in this area (joint estimation of image projection and exposure value) was undertaken by Mann and Candoccia.
Charles Wyckoff devoted much of his life to creating special kinds of 3-layer photographic films that captured different exposures of the same subject matter. A picture of a nuclear explosion, taken on Wyckoff's film, appeared on the cover of
Life Magazine and showed the dynamic range from dark outer areas to inner core.
See also
*
Adaptive optics
Adaptive optics (AO) is a technology used to improve the performance of optical systems by reducing the effect of incoming wavefront distortions by deforming a mirror in order to compensate for the distortion. It is used in astronomical tele ...
*
Multispectral imaging
Multispectral imaging captures image data within specific wavelength ranges across the electromagnetic spectrum. The wavelengths may be separated by filters or detected with the use of instruments that are sensitive to particular wavelengths, ...
*
Simultaneous localization and mapping
*
Super-resolution microscopy
*
Time-of-flight camera
References
External links
* Nayar, Shree K. (2007)
"Computational Cameras" ''Conference on Machine Vision Applications''.
''Computational Photography'' (Raskar, R., Tumblin, J.,) A.K. Peters. In press.
Special issue on Computational Photography IEEE Computer, August 2006.
Camera Culture and Computational Journalism: Capturing and Sharing Visual Experiences{{Webarchive, url=https://web.archive.org/web/20150906055242/http://www.computer.org/portal/web/computingnow/cgacfp1 , date=2015-09-06 , IEEE CG&A Special Issue, Feb 2011.
* Rick Szeliski (2010),
Computer Vision: Algorithms and Applications', Springer.
* Computational Photography: Methods and Applications (Ed. Rastislav Lukac), CRC Press, 2010.
(John Wiley and Sons book information).
GJB-1: Increasing the dynamic range of a digital camera by using the Wyckoff principleExamples of wearable computational photography as an art formSiggraph Course in Computational Photography
Digital photography
Computational fields of study
Computer vision