Video stabilization

Image stabilization (IS) is a family of techniques that reduce blurring associated with the motion of a camera or other imaging device during exposure.

Generally, it compensates for pan and tilt (angular movement, equivalent to yaw and pitch) of the imaging device, though electronic image stabilization can also compensate for rotation. It is mainly used in high-end image-stabilized binoculars, still and video cameras, astronomical telescopes, and also smartphones. With still cameras, camera shake is a particular problem at slow shutter speeds or with long focal length lenses (telephoto or zoom). With video cameras, camera shake causes visible frame-to-frame jitter in the recorded video. In astronomy, the problem of lens shake is added to variation in the atmosphere, which changes the apparent positions of objects over time.

Application in still photography

In photography, image stabilization can facilitate shutter speeds 2 to 5.5 stops slower (exposures 4 to times longer), and even slower effective speeds have been reported.

A rule of thumb to determine the slowest shutter speed possible for hand-holding without noticeable blur due to camera shake is to take the reciprocal of the 35 mm equivalent focal length of the lens, also known as the "1/mm rule". For example, at a focal length of 125 mm on a 35 mm camera, vibration or camera shake could affect sharpness if the shutter speed is slower than second. As a result of the 2-to-4.5-stops slower shutter speeds allowed by IS, an image taken at second speed with an ordinary lens could be taken at or second with an IS-equipped lens and produce almost the same quality. The sharpness obtainable at a given speed can increase dramatically.
When calculating the effective focal length, it is important to take into account the image format a camera uses. For example, many digital SLR cameras use an image sensor that is , , or the size of a 35 mm film frame. This means that the 35 mm frame is 1.5, 1.6, or 2 times the size of the digital sensor. The latter values are referred to as the crop factor, field-of-view crop factor, focal-length multiplier, or format factor. On a 2× crop factor camera, for instance, a 50 mm lens produces the same field of view as a 100 mm lens used on a 35 mm film camera, and can typically be handheld at second.

However, image stabilization does ''not'' prevent motion blur caused by the movement of the subject or by extreme movements of the camera. Image stabilization is only designed for and capable of reducing blur that results from normal, minute shaking of a lens due to hand-held shooting. Some lenses and camera bodies include a secondary panning mode or a more aggressive 'active mode', both described in greater detail below under optical image stabilization.

Astrophotography makes much use of long-exposure photography, which requires the camera to be fixed in place. However, fastening it to the Earth is not enough, since the Earth rotates. The Pentax K-5 and K-r, when equipped with the O-GPS1 GPS accessory for position data, can use their sensor-shift capability to reduce the resulting star trails.

Stabilization can be applied in the lens, or in the camera body. Each method has distinctive advantages and disadvantages.

Techniques

Optical image stabilization

An optical image stabilizer (OIS, IS, or OS) is a mechanism used in still or video cameras that stabilizes the recorded image by varying the optical path to the sensor. This technology is implemented in the lens itself, as distinct from in-body image stabilization (''IBIS''), which operates by moving the sensor as the final element in the optical path. The key element of all optical stabilization systems is that they stabilize the image projected on the sensor before the sensor converts the image into digital information. IBIS can have up to 5 axis of movement: X, Y, Roll, Yaw, and Pitch. IBIS has the added advantage of working with all lenses.

Benefits of OIS

Optical image stabilization prolongs the shutter speed possible for handheld photography by reducing the likelihood of blurring the image from shake during the same exposure time.

For handheld video recording, regardless of lighting conditions, optical image stabilization compensates for minor shakes whose appearance magnifies when watched on a large display such as a television set or computer monitor.

Names by vendors

Different companies have different names for the OIS technology, for example:
* Vibration Reduction (VR) – Nikon (produced the first optical two-axis stabilized lens, a 38–105 mm ''f''/4–7.8 zoom built into the Nikon Zoom 700VR (US: Zoom-Touch 105 VR) camera in 1994)
* Image Stabilizer (IS) – Canon introduced the EF 75–300 mm ''f''/4–5.6 IS USM) in 1995. In 2009, they introduced their first lens (the EF 100mm F2.8 Macro L) to use a four-axis Hybrid IS.)
* Anti-Shake (AS) – Minolta and Konica Minolta (Minolta introduced the first sensor-based two-axis image stabilizer with the DiMAGE A1 in 2003)
* IBIS - In Body Image Stabilisation – Olympus and Fujifilm
* Optical SteadyShot (OSS) – Sony (for Cyber-shot and several α E-mount lenses)
* Optical Image Stabilization (OIS) – Fujifilm
* MegaOIS, PowerOIS – Panasonic and Leica
* SteadyShot (SS), Super SteadyShot (SSS), SteadyShot INSIDE (SSI) – Sony (based on Konica Minolta's Anti-Shake originally, Sony introduced a 2-axis full-frame variant for the DSLR-A900 in 2008 and a 5-axis stabilizer for the full-frame ILCE-7M2 in 2014)
* Optical Stabilization (OS) – Sigma
* Vibration Compensation (VC) – Tamron
* Shake Reduction (SR) – Pentax
* PureView – Nokia (produced the first cell phone optical stabilised sensor, built into the Lumia 920)
*UltraPixel – HTC (Image Stabilization is only available for the 2013 HTC One & 2016 HTC 10 with UltraPixel. It is not available for the HTC One (M8) or HTC Butterfly S, which also have UltraPixel)
Most high-end smartphones as of late 2014 use optical image stabilization for photos and videos.

Lens-based

In Nikon and Canon's implementation, it works by using a floating lens element that is moved orthogonally to the optical axis of the lens using electromagnets. Vibration is detected using two piezoelectric angular velocity sensors (often called gyroscopic sensors), one to detect horizontal movement and the other to detect vertical movement. As a result, this kind of image stabilizer corrects only for pitch and yaw axis rotations, and cannot correct for rotation around the optical axis. Some lenses have a secondary mode that counteracts vertical-only camera shake. This mode is useful when using a panning technique. Some such lenses activate it automatically; others use a switch on the lens.

To compensate for camera shake in shooting video while walking, Panasonic introduced Power Hybrid OIS+ with five-axis correction: axis rotation, horizontal rotation, vertical rotation, and horizontal and vertical motion.

Some Nikon VR-enabled lenses offer an "active" mode for shooting from a moving vehicle, such as a car or boat, which is supposed to correct for larger shakes than the "normal" mode. However, active mode used for normal shooting can produce poorer results than normal mode. This is because active mode is optimized for reducing higher angular velocity movements (typically when shooting from a heavily moving platform using faster shutter speeds), where normal mode tries to reduce lower angular velocity movements over a larger amplitude and timeframe (typically body and hand movement when standing on a stationary or slowly moving platform while using slower shutter speeds).

Most manufacturers suggest that the IS feature of a lens be turned off when the lens is mounted on a tripod as it can cause erratic results and is generally unnecessary. Many modern image stabilization lenses (notably Canon's more recent IS lenses) are able to auto-detect that they are tripod-mounted (as a result of extremely low vibration readings) and disable IS automatically to prevent this and any consequent image quality reduction. The system also draws battery power, so deactivating it when not needed extends the battery charge.

A disadvantage of lens-based image stabilization is cost. Each lens requires its own image stabilization system. Also, not every lens is available in an image-stabilized version. This is often the case for fast primes and wide-angle lenses. However, the fastest lens with image stabilisation is the Nocticron with a speed of ''f''/1.2. While the most obvious advantage for image stabilization lies with longer focal lengths, even normal and wide-angle lenses benefit from it in low-light applications.

Lens-based stabilization also has advantages over in-body stabilization. In low-light or low-contrast situations, the autofocus system (which has no stabilized sensors) is able to work more accurately when the image coming from the lens is already stabilized. In cameras with optical viewfinders, the image seen by the photographer through the stabilized lens (as opposed to in-body stabilization) reveals more detail because of its stability, and it also makes correct framing easier. This is especially the case with longer telephoto lenses. This advantage does not occur on compact system cameras, because the sensor output to the screen or electronic viewfinder would be stabilized.

Sensor-shift

The sensor capturing the image can be moved in such a way as to counteract the motion of the camera, a technology often referred to as mechanical image stabilization. When the camera rotates, causing angular error, gyroscopes encode information to the actuator that moves the sensor. The sensor is moved to maintain the projection of the image onto the image plane, which is a function of the focal length of the lens being used. Modern cameras can automatically acquire focal length information from modern lenses made for that camera. Minolta and Konica Minolta used a technique called Anti-Shake (AS) now marketed as SteadyShot (SS) in the Sony α line and Shake Reduction (SR) in the Pentax K-series and Q series cameras, which relies on a very precise angular rate sensor to detect camera motion. Olympus introduced image stabilization with their E-510 D-SLR body, employing a system built around their Supersonic Wave Drive. Other manufacturers use digital signal processors (DSP) to analyze the image on the fly and then move the sensor appropriately. Sensor shifting is also used in some cameras by Fujifilm, Samsung, Casio Exilim and Ricoh Caplio.

The advantage with moving the image sensor, instead of the lens, is that the image can be stabilized even on lenses made without stabilization. This may allow the stabilization to work with many otherwise-unstabilized lenses, and reduces the weight and complexity of the lenses. Further, when sensor-based image stabilization technology improves, it requires replacing only the camera to take advantage of the improvements, which is typically far less expensive than replacing all existing lenses if relying on lens-based image stabilization. Some sensor-based image stabilization implementations are capable of correcting camera roll rotation, a motion that is easily excited by pressing the shutter button. No lens-based system can address this potential source of image blur. A by-product of available "roll" compensation is that the camera can automatically correct for tilted horizons in the optical domain, provided it is equipped with an electronic spirit level, such as the Pentax K-7/K-5 cameras.

One of the primary disadvantages of moving the image sensor itself is that the image projected to the viewfinder is not stabilized. However, this is not an issue on cameras that use an electronic viewfinder (EVF), since the image projected on that viewfinder is taken from the image sensor itself. Similarly, the image projected to a phase-detection autofocus system that is not part of the image sensor, if used, is not stabilized.

Some, but not all, camera-bodies capable of in-body stabilization can be pre-set manually to a given focal length. Their stabilization system corrects as if that focal length lens is attached, so the camera can stabilize older lenses, and lenses from other makers. This isn't viable with zoom lenses, because their focal length is variable. Some adapters communicate focal length information from the maker of one lens to the body of another maker. Some lenses that do not report their focal length can be retrofitted with a chip which reports a pre-programmed focal-length to the camera body. Sometimes, none of these techniques work, and image-stabilization cannot be used with such lenses.

In-body image stabilization requires the lens to have a larger output image circle because the sensor is moved during exposure and thus uses a larger part of the image. Compared to lens movements in optical image stabilization systems the sensor movements are quite large, so the effectiveness is limited by the maximum range of sensor movement, where a typical modern optically-stabilized lens has greater freedom. Both the speed and range of the required sensor movement increase with the focal length of the lens being used, making sensor-shift technology less suited for very long telephoto lenses, especially when using slower shutter speeds, because the available motion range of the sensor quickly becomes insufficient to cope with the increasing image displacement.

Dual

Starting with the Panasonic Lumix DMC-GX8, announced in July 2015, and subsequently in the Panasonic Lumix DC-GH5, Panasonic, who formerly only equipped lens-based stabilization in its interchangeable lens camera system (of the Micro Four Thirds standard), introduced sensor-shift stabilization that works in concert with the existing lens-based system ("Dual IS").

In the meantime (2016), Olympus also offered two lenses with image stabilization that can be synchronized with the in-built image stabilization system of the image sensors of Olympus' Micro Four Thirds cameras ("Sync IS"). With this technology a gain of 6.5 ''f''-stops can be achieved without blurred images. This is limited by the rotational movement of the surface of the Earth, that fools the accelerometers of the camera. Therefore, depending on the angle of view, the maximum exposure time should not exceed second for long telephoto shots (with a 35 mm equivalent focal length of 800 millimeters) and a little more than ten seconds for wide angle shots (with a 35 mm equivalent focal length of 24 millimeters), if the movement of the Earth is not taken into consideration by the image stabilization process.

In 2015, the Sony E camera system also allowed combining image stabilization systems of lenses and camera bodies, but without synchronizing the same degrees of freedom. In this case, only the independent compensation degrees of the in-built image sensor stabilization are activated to support lens stabilisation.

Canon and Nikon now have full-frame mirrorless bodies that have IBIS and also support each company's lens-based stabilization. Canon's first two such bodies, the EOS R and RP, do not have IBIS, but the feature was added for the more recent R5 and R6. All of Nikon's full-frame Z-mount bodies—the Z 6, Z 7, and the Mark II versions of both—have IBIS. However, its APS-C Z 50 lacks IBIS.

Digital image stabilization

Real-time digital image stabilization, also called electronic image stabilization (EIS), is used in some video cameras. This technique shifts the electronic image from frame to frame of video, enough to counteract the motion. It uses pixels outside the border of the visible frame to provide a buffer for the motion. This technique reduces distracting vibrations from videos by smoothing the transition from one frame to another. This technique does not affect the noise level of the image, except in the extreme borders when the image is extrapolated. It cannot do anything about existing motion blur, which may result in an image seemingly losing focus as motion is compensated.

Some still camera manufacturers marketed their cameras as having digital image stabilization when they really only had a high-sensitivity mode that uses a short exposure time—producing pictures with less motion blur, but more noise. It reduces blur when photographing something that is moving, as well as from camera shake.

Others now also use digital signal processing (DSP) to reduce blur in stills, for example by sub-dividing the exposure into several shorter exposures in rapid succession, discarding blurred ones, re-aligning the sharpest sub-exposures and adding them together, and using the gyroscope to detect the best time to take each frame.

Stabilization filters

Many video non-linear editing systems use stabilization filters that can correct a non-stabilized image by tracking the movement of pixels in the image and correcting the image by moving the frame. The process is similar to digital image stabilization but since there is no ''larger'' image to work with the filter either crops the image down to hide the motion of the frame or attempts to recreate the lost image at the edge through spatial or temporal extrapolation.

Online services, including YouTube, are also beginning to provide video stabilization'' as a post-processing step after content is uploaded. This has the disadvantage of not having access to the realtime gyroscopic data, but the advantage of more computing power and the ability to analyze images both before and after a particular frame.

Orthogonal transfer CCD

Used in astronomy, an orthogonal transfer CCD (OTCCD) actually shifts the image within the CCD itself while the image is being captured, based on analysis of the apparent motion of bright stars. This is a rare example of digital stabilization for still pictures. An example of this is in the upcoming gigapixel telescope Pan-STARRS being constructed in Hawaii.

Stabilizing the camera body

A technique that requires no additional capabilities of any camera body–lens combination consists of stabilizing the entire camera body externally rather than using an internal method. This is achieved by attaching a gyroscope to the camera body, usually using the camera's built-in tripod mount. This lets the external gyro (gimbal) stabilize the camera, and is typically used in photography from a moving vehicle, when a lens or camera offering another type of image stabilization is not available.

A common way to stabilize moving cameras after approx. year 2015 is by using a camera stabilizer such as a stabilized remote camera head. The camera and lens are mounted in a remote controlled camera holder which is then mounted on anything that moves, such as rail systems, cables, cars or helicopters. An example of a remote stabilized head that is used to stabilize moving TV cameras that are broadcasting live is the Newton stabilized head.

Another technique for stabilizing a video or motion picture camera body is the Steadicam system, which isolates the camera from the operator's body using a harness and a camera boom with a counterweight.

Camera stabilizer

A camera stabilizer is any device or object that externally stabilizes the camera. This can refer to a Steadicam, a tripod, the camera operator's hand, or a combination of these.

In close-up photography, using rotation sensors to compensate for changes in pointing direction becomes insufficient. Moving, rather than tilting, the camera up/down or left/right by a fraction of a millimeter becomes noticeable if you are trying to resolve millimeter-size details on the object. Linear accelerometers in the camera, coupled with information such as the lens focal length and focused distance, can feed a secondary correction into the drive that moves the sensor or optics, to compensate for linear as well as rotational shake.

In biological eyes

In many animals, including human beings, the inner ear functions as the biological analogue of an accelerometer in camera image stabilization systems, to stabilize the image by moving the eyes. When a rotation of the head is detected, an inhibitory signal is sent to the extraocular muscles on one side and an excitatory signal to the muscles on the other side. The result is a compensatory movement of the eyes. Typically eye movements lag the head movements by less than 10 ms.

See also

* Adaptive optics
* Deblurring
* Heligimbal
* Hyperlapse
* Motion compensation
* Shaky camera

Notes

References

{{Reflist, 30em, refs =

{{cite web
, first=Steven
, last=Zimmerman
, url=http://www.xda-developers.com/sony-imx378-comprehensive-breakdown-of-the-google-pixels-sensor-and-its-features/
, title=Sony IMX378: Comprehensive Breakdown of the Google Pixel's Sensor and its Features
, publisher=XDA Developers
, date=12 October 2016
, access-date=17 October 2016

Articles containing video clips