Motion analysis is used in
computer vision,
image processing,
high-speed photography
High-speed photography is the science of taking pictures of very fast phenomena. In 1948, the Society of Motion Picture and Television Engineers (SMPTE) defined high-speed photography as any set of photographs captured by a camera capable of 69 ...
and
machine vision
Machine vision (MV) is the technology and methods used to provide imaging-based automatic inspection and analysis for such applications as automatic inspection, process control, and robot guidance, usually in industry. Machine vision refers to ...
that studies methods and applications in which two or more consecutive images from an image sequences, e.g., produced by a
video camera
A video camera is an optical instrument that captures videos (as opposed to a movie camera, which records images on film). Video cameras were initially developed for the television industry but have since become widely used for a variety of oth ...
or
high-speed camera
A high-speed camera is a device capable of capturing moving images with exposures of less than 1/1,000 second or frame rates in excess of 250 fps. It is used for recording fast-moving objects as photographic images onto a storage medium. After r ...
, are processed to produce information based on the apparent motion in the images. In some applications, the camera is fixed relative to the scene and objects are moving around in the scene, in some applications the scene is more or less fixed and the camera is moving, and in some cases both the camera and the scene are moving.
The motion analysis processing can in the simplest case be to detect motion, i.e., find the points in the image where something is moving. More complex types of processing can be to
track a specific object in the image over time, to group points that belong to the same rigid object that is moving in the scene, or to determine the magnitude and direction of the motion of every point in the image. The information that is produced is often related to a specific image in the sequence, corresponding to a specific time-point, but then depends also on the neighboring images. This means that motion analysis can produce time-dependent information about motion.
Applications of motion analysis can be found in rather diverse areas, such as surveillance, medicine, film industry, automotive crash safety, ballistic firearm studies, biological science, flame propagation, and navigation of autonomous vehicles to name a few examples.
Background
A video camera can be seen as an approximation of a
pinhole camera
A pinhole camera is a simple camera without a lens but with a tiny aperture (the so-called '' pinhole'')—effectively a light-proof box with a small hole in one side. Light from a scene passes through the aperture and projects an inverted image ...
, which means that each point in the image is illuminated by some (normally one) point in the scene in front of the camera, usually by means of light that the scene point reflects from a light source. Each visible point in the scene is projected along a straight line that passes through the camera aperture and intersects the image plane. This means that at a specific point in time, each point in the image refers to a specific point in the scene. This scene point has a position relative to the camera, and if this relative position changes, it corresponds to a ''relative motion in 3D''. It is a relative motion since it does not matter if it is the scene point, or the camera, or both, that are moving. It is only when there is a change in the relative position that the camera is able to detect that some motion has happened. By projecting the relative 3D motion of all visible points back into the image, the result is the ''
motion field In computer vision the motion field is an ideal representation of 3D motion as it is projected onto a camera image. Given a simplified camera model, each point (y_, y_) in the image is the projection of some point in the 3D scene but the positio ...
'', describing the apparent motion of each image point in terms of a magnitude and direction of velocity of that point in the image plane. A consequence of this observation is that if the relative 3D motion of some scene points are along their projection lines, the corresponding apparent motion is zero.
The camera measures the intensity of light at each image point, a light field. In practice, a digital camera measures this light field at discrete points,
pixels
In digital imaging, a pixel (abbreviated px), pel, or picture element is the smallest addressable element in a raster image, or the smallest point in an all points addressable display device.
In most digital display devices, pixels are the sm ...
, but given that the pixels are sufficiently dense, the pixel intensities can be used to represent most characteristics of the light field that falls onto the image plane. A common assumption of motion analysis is that the light reflected from the scene points does not vary over time. As a consequence, if an intensity ''I'' has been observed at some point in the image, at some point in time, the same intensity ''I'' will be observed at a position that is displaced relative to the first one as a consequence of the apparent motion. Another common assumption is that there is a fair amount of variation in the detected intensity over the pixels in an image. A consequence of this assumption is that if the scene point that corresponds to a certain pixel in the image has a relative 3D motion, then the pixel intensity is likely to change over time.
Methods
Motion detection
One of the simplest type of motion analysis is to detect image points that refer to moving points in the scene. The typical result of this processing is a binary image where all image points (pixels) that relate to moving points in the scene are set to 1 and all other points are set to 0. This binary image is then further processed, e.g., to remove noise, group neighboring pixels, and label objects.
Motion detection can be done using several methods; the two main groups are differential methods and methods based on background segmentation.
Applications
Human motion analysis
In the areas of
medicine
Medicine is the science and practice of caring for a patient, managing the diagnosis, prognosis, prevention, treatment, palliation of their injury or disease, and promoting their health. Medicine encompasses a variety of health care pr ...
, sports, video surveillance, physical therapy, and
kinesiology
Kinesiology () is the scientific study of human body movement. Kinesiology addresses physiological, anatomical, biomechanical, pathological, neuropsychological principles and mechanisms of movement. Applications of kinesiology to human healt ...
, human motion analysis has become an investigative and diagnostic tool. See the section on
motion capture for more detail on the technologies. Human motion analysis can be divided into three categories: human
activity recognition
Activity recognition aims to recognize the actions and goals of one or more agents from a series of observations on the agents' actions and the environmental conditions. Since the 1980s, this research field has captured the attention of several c ...
, human motion tracking, and analysis of body and body part movement.
Human activity recognition is most commonly used for
video surveillance
Closed-circuit television (CCTV), also known as video surveillance, is the use of video cameras to transmit a signal to a specific place, on a limited set of monitors. It differs from broadcast television in that the signal is not openly tr ...
, specifically automatic motion monitoring for security purposes. Most efforts in this area rely on state-space approaches, in which sequences of static postures are statistically analyzed and compared to modeled movements. Template-matching is an alternative method whereby static shape patterns are compared to pre-existing prototypes.
[Aggarwal, JK and Q Cai. "Human Motion Analysis: A Review." ''Computer Vision and Image Understanding'' 73, no. 3 (1999): 428-440.]
Human
motion tracking can be performed in two or three dimensions. Depending on the complexity of analysis, representations of the human body range from basic stick figures to volumetric models. Tracking relies on the
correspondence of image features between consecutive frames of video, taking into consideration information such as position, color, shape, and texture.
Edge detection
Edge detection includes a variety of mathematical methods that aim at identifying edges, curves in a digital image at which the image brightness changes sharply or, more formally, has discontinuities. The same problem of finding discontinuitie ...
can be performed by comparing the color and/or contrast of adjacent pixels, looking specifically for discontinuities or rapid changes.
[Fan, J, EA El-Kwae, M-S Hacid, and F Liang. "Novel tracking-based moving object extraction algorithm." ''J Electron Imaging'' 11, 393 (2002).] Three-dimensional tracking is fundamentally identical to two-dimensional tracking, with the added factor of spatial calibration.
Motion analysis of body parts is critical in the medical field. In postural and
gait analysis
Gait analysis is the systematic study of animal locomotion, more specifically the study of human motion, using the eye and the brain of observers, augmented by instrumentation for measuring body movements, body mechanics, and the activity of the ...
, joint angles are used to track the location and orientation of body parts. Gait analysis is also used in sports to optimize athletic performance or to identify motions that may cause injury or strain. Tracking software that does not require the use of optical markers is especially important in these fields, where the use of markers may impede natural movement.
[Green, RD, L Guan, and JA Burne. "Video analysis of gait for diagnosing movement disorders." ''J Electron Imaging'' 9, 16 (2000).]
Motion analysis in manufacturing
Motion analysis is also applicable in the
manufacturing
Manufacturing is the creation or production of goods with the help of equipment, labor, machines, tools, and chemical or biological processing or formulation. It is the essence of secondary sector of the economy. The term may refer to ...
process.
Using high speed video cameras and motion analysis software, one can monitor and analyze
assembly lines
An assembly line is a manufacturing process (often called a ''progressive assembly'') in which parts (usually interchangeable parts) are added as the semi-finished assembly moves from workstation to workstation where the parts are added in sequ ...
and production machines to detect inefficiencies or malfunctions. Manufacturers of sports equipment, such as baseball bats and hockey sticks, also use high speed video analysis to study the impact of projectiles. An experimental setup for this type of study typically uses a triggering device, external sensors (e.g.,
accelerometers
An accelerometer is a tool that measures proper acceleration. Proper acceleration is the acceleration (the rate of change of velocity) of a body in its own instantaneous rest frame; this is different from coordinate acceleration, which is acce ...
, strain gauges), data acquisition modules, a
high-speed camera
A high-speed camera is a device capable of capturing moving images with exposures of less than 1/1,000 second or frame rates in excess of 250 fps. It is used for recording fast-moving objects as photographic images onto a storage medium. After r ...
, and a computer for storing the synchronized video and data. Motion analysis software calculates parameters such as distance, velocity, acceleration, and deformation angles as functions of time. This data is then used to design equipment for optimal performance.
[Masi, CG.]
Vision improves bat performance.
''Vision Systems Design''. June 2006
Additional applications for motion analysis
The object and
feature detecting capabilities of motion analysis software can be applied to count and track particles, such as bacteria,
[Borrok, M. J., et al. (2009)]
Structure-based design of a periplasmic binding protein antagonist that prevents domain closure
''ACS Chemical Biology'', 4, 447-456.[Borrok, M. J., Kolonko, E. M., and Kiessling, L. L. (2008). Chemical probes of bacterial signal transduction reveal that repellents stabilize and attractants destabilize the chemoreceptor array. ''ACS Chemical Biology'', 3, 101-109.] viruses,
[Shopov, A. et al. "Improvements in image analysis and fluorescence microscopy to discriminate and enumerate bacteria and viruses in aquatic samples, or cells, and to analyze sprays and fragmenting debris." ''Aquatic Microbial Ecology'' 22 (2000): 103-110.] "ionic polymer-metal composites",
[Park, J. K., and Moore, R. B. (2009). Influence of ordered morphology on the anisotropic actuation in uniaxially oriented electroactive polymer systems. ''ACS Applied Materials & Interfaces'', 1, 697-702.][Phillips, A. K., and Moore, R. B. (2005). Ionic actuators based on novel sulfonated ethylene vinyl alcohol copolymer membranes. ''Polymer'', 46, 7788-7802.] micron-sized polystyrene beads,
[Nott, M. (2005). Teaching Brownian motion: demonstrations and role play. ''School Science Review'', 86, 18-28.] aphids,
[Kay, S., and Steinkraus, D. C. (2005). Effect of Neozygites fresenii infection on cotton aphid movement. ''AAES Research Series 543'', 245-248. Fayetteville, AR: Arkansas Agricultural Experiment Station. Available from http://arkansasagnews.uark.edu/543-43.pdf] and projectiles.
[Sparks, C. et al. "Comparison and Validation of Smooth Particle Hydrodynamics (SPH) and Coupled Euler Lagrange (CEL) Techniques for Modeling Hydrodynamic Ram." ''46th AIAA/ASME/ASCE/AHS/ASC Structures, Structural Dynamics and Materials Conference'', Austin, Texas, Apr. 18-21, 2005.]
See also
*
Mechanography
*
Structure from motion
Structure from motion (SfM) is a photogrammetric range imaging technique for estimating three-dimensional structures from two-dimensional image sequences that may be coupled with local motion signals. It is studied in the fields of computer visio ...
*
Video motion analysis
*
X-ray motion analysis
References
{{reflist
Research methods
Motion in computer vision