![Photometric stereo](https://upload.wikimedia.org/wikipedia/commons/b/b5/Photometric_stereo.png)
Photometric stereo is a technique in
computer vision
Computer vision is an interdisciplinary scientific field that deals with how computers can gain high-level understanding from digital images or videos. From the perspective of engineering, it seeks to understand and automate tasks that the hum ...
for estimating the
surface normals
In geometry, a normal is an object such as a line, ray, or vector that is perpendicular to a given object. For example, the normal line to a plane curve at a given point is the (infinite) line perpendicular to the tangent line to the curve at ...
of objects by observing that object under different lighting conditions. It is based on the fact that the amount of light reflected by a surface is dependent on the orientation of the surface in relation to the light source and the observer. By measuring the amount of light reflected into a camera, the space of possible surface orientations is limited. Given enough light sources from different angles, the surface orientation may be constrained to a single orientation or even overconstrained.
The technique was originally introduced by Woodham in 1980.
[Woodham, R.J. 1980]
Photometric method for determining surface orientation from multiple images
Optical Engineerings 19, I, 139-144. The special case where the data is a single image is known as shape from shading, and was analyzed by B. K. P. Horn in 1989.
[B. K. P. Horn, 1989. Obtaining shape from shading information. In B. K. P. Horn and M. J. Brooks, eds., Shape from Shading, pages 121–171. MIT Press.] Photometric stereo has since been generalized to many other situations, including extended light sources and non-
Lambertian surface finishes. Current research aims to make the method work in the presence of projected shadows, highlights, and non-uniform lighting.
Basic Method
Under Woodham's original assumptions —
Lambertian reflectance
Lambertian reflectance is the property that defines an ideal "matte" or diffusely reflecting surface. The apparent brightness of a Lambertian surface to an observer is the same regardless of the observer's angle of view. More technically, the su ...
, known point-like distant light sources, and uniform
albedo
Albedo (; ) is the measure of the diffuse reflection of sunlight, solar radiation out of the total solar radiation and measured on a scale from 0, corresponding to a black body that absorbs all incident radiation, to 1, corresponding to a body ...
— the problem can be solved by inverting the linear equation
, where
is a (known) vector of
observed intensities,
is the (unknown) surface normal, and
is a (known)
matrix of normalized light directions.
This model can easily be extended to surfaces with non-uniform albedo, while keeping the problem linear.
[S. Barsky and Maria Petrou, 2003]
The 4-source photometric stereo technique for 3-dimensional surfaces in the presence of highlights and shadows
In IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 25, issue 10, pages 1239-1252. IEEE. Taking an albedo reflectivity of
, the formula for the reflected light intensity becomes:
:
If
is square (there are exactly 3 lights) and non-singular, it can be inverted, giving:
:
Since the normal vector is known to have length 1,
must be the length of the vector
, and
is the normalised direction of that vector.
If
is not square (there are more than 3 lights), a generalisation of the inverse can be obtained using the
Moore–Penrose pseudoinverse, by simply multiplying both sides with
giving:
:
:
After which the normal vector and albedo can be solved as described above.
Non-Lambertian surfaces
The classical photometric stereo problem concerns itself only with
Lambertian surfaces, with perfectly diffuse reflection. This is unrealistic for many types of materials, especially metals, glass and smooth plastics, and will lead to aberrations in the resulting normal vectors.
Many methods have been developed to lift this assumption. In this section, a few of these are listed.
Specular reflections
Historically, in
computer graphics
Computer graphics deals with generating images with the aid of computers. Today, computer graphics is a core technology in digital photography, film, video games, cell phone and computer displays, and many specialized applications. A great de ...
, the commonly used model to render surfaces started with Lambertian surfaces and progressed first to include simple
specular reflections. Computer vision followed a similar course with photometric stereo. Specular reflections were among the first deviations from the Lambertian model. These are a few adaptations that have been developed.
* Many techniques ultimately rely on modelling the reflectance function of the surface, that is, how much light is reflected in each direction. This reflectance function has to be
invertible
In mathematics, the concept of an inverse element generalises the concepts of opposite () and reciprocal () of numbers.
Given an operation denoted here , and an identity element denoted , if , one says that is a left inverse of , and that is ...
. The reflected light intensities towards the camera is measured, and the inverse reflectance function is fit onto the measured intensities, resulting in a unique solution for the normal vector.
General BRDFs and beyond
According to the
Bidirectional reflectance distribution function
The bidirectional reflectance distribution function (BRDF; f_(\omega_,\, \omega_) ) is a function of four real variables that defines how light is reflected at an opaque surface. It is employed in the optics of real-world light, in compute ...
(BRDF) model, a surface may distribute the amount of light it receives in any outward direction. This is the most general known model for
opaque
Opacity or opaque may refer to:
* Impediments to (especially, visible) light:
** Opacities, absorption coefficients
** Opacity (optics), property or degree of blocking the transmission of light
* Metaphors derived from literal optics:
** In lingu ...
surfaces. Some techniques have been developed to model (almost) general BRDFs. In practice, all of these require many light sources to obtain reliable data. These are methods in which surfaces with general BRDFs can be measured.
* Determine the explicit BRDF prior to scanning. To do this, a different surface is required that has the same or a very similar BRDF, of which the actual geometry (or at least the normal vectors for many points on the surface) is already known. The lights are then individually shone upon the known surface, and the amount of reflection into the camera is measured. Using this information, a look-up table can be created that maps reflected intensities for each light source to a list of possible normal vectors. This puts constraints on the possible normal vectors the surface may have, and reduces the photometric stereo problem to an interpolation between measurements. Typical known surfaces to calibrate the look-up table with are spheres for their wide variety of surface orientations.
* Restricting the BRDF to be symmetrical. If the BRDF is symmetrical, the direction of the light can be restricted to a cone about the direction to the camera. Which cone this is depends on the BRDF itself, the normal vector of the surface, and the measured intensity. Given enough measured intensities and the resulting light directions, these cones can be approximated and therefore the normal vectors of the surface.
Some progress has been made towards modelling an even more general surfaces, such as
Spatially Varying Bidirectional Distribution Functions (SVBRDF),
Bidirectional surface scattering reflectance distribution functions (BSSRDF), and accounting for
interreflections. However, such methods are still fairly restrictive in photometric stereo. Better results have been achieved with
structured light
A structured light pattern designed for surface inspection
An Automatix Seamtracker arc welding robot equipped with a camera and structured laser light source, enabling the robot to follow a welding seam automatically
Structured light is the p ...
.
[Tongbo Chen, Hendrik Lensch, Christian Fuchs and H.P. Seidel, 2007]
Polarization and Phase-shifting for 3D Scanning of Translucent Objects
In IEEE Conference on Computer Vision and Pattern Recognition, 2007, pages 1-8. IEEE.
See also
*
Photoclinometry Photoclinometry, or shape-from-shading, is the process by which a 2-dimensional image of a surface is transformed into a surface map that represents different levels of elevation. It uses the shadows and light direction as reference points. It is us ...
*
Photometry Photometry can refer to:
* Photometry (optics), the science of measurement of visible light in terms of its perceived brightness to human vision
* Photometry (astronomy), the measurement of the flux or intensity of an astronomical object's electro ...
*
Stereo vision
Stereopsis () is the component of depth perception retrieved through binocular vision.
Stereopsis is not the only contributor to depth perception, but it is a major one. Binocular vision happens because each eye receives a different image becaus ...
*
3D scanner
3D scanning is the process of analyzing a real-world object or environment to collect data on its shape and possibly its appearance (e.g. color). The collected data can then be used to construct digital 3D modelling, 3D models.
A 3D scanner can ...
References
{{Reflist
Computer graphics
Computer vision
3D imaging