![AutovisionIIatRDT](https://upload.wikimedia.org/wikipedia/commons/d/d1/AutovisionIIatRDT.jpg)
Machine vision (MV) is the technology and methods used to provide
imaging
Imaging is the representation or reproduction of an object's form; especially a visual representation (i.e., the formation of an image).
Imaging technology is the application of materials and methods to create, preserve, or duplicate images.
...
-based
automatic inspection and analysis for such applications as automatic inspection,
process control
An industrial process control in continuous production processes is a discipline that uses industrial control systems to achieve a production level of consistency, economy and safety which could not be achieved purely by human manual control. I ...
, and robot guidance, usually in industry. Machine vision refers to many technologies, software and hardware products, integrated systems, actions, methods and expertise. Machine vision as a
systems engineering
Systems engineering is an interdisciplinary field of engineering and engineering management that focuses on how to design, integrate, and manage complex systems over their enterprise life cycle, life cycles. At its core, systems engineering util ...
discipline can be considered distinct from
computer vision
Computer vision is an interdisciplinary scientific field that deals with how computers can gain high-level understanding from digital images or videos. From the perspective of engineering, it seeks to understand and automate tasks that the hum ...
, a form of
computer science
Computer science is the study of computation, automation, and information. Computer science spans theoretical disciplines (such as algorithms, theory of computation, information theory, and automation) to Applied science, practical discipli ...
. It attempts to integrate existing technologies in new ways and apply them to solve real world problems. The term is the prevalent one for these functions in industrial automation environments but is also used for these functions in other environment vehicle guidance.
The overall machine vision process includes planning the details of the requirements and project, and then creating a solution. During run-time, the process starts with imaging, followed by automated
analysis
Analysis ( : analyses) is the process of breaking a complex topic or substance into smaller parts in order to gain a better understanding of it. The technique has been applied in the study of mathematics and logic since before Aristotle (38 ...
of the image and extraction of the required information.
Definition
Definitions of the term "Machine vision" vary, but all include the technology and methods used to extract information from an image on an automated basis, as opposed to
image processing
An image is a visual representation of something. It can be two-dimensional, three-dimensional, or somehow otherwise feed into the visual system to convey information. An image can be an artifact, such as a photograph or other two-dimensiona ...
, where the output is another image. The information extracted can be a simple good-part/bad-part signal, or more a complex set of data such as the identity, position and orientation of each object in an image. The information can be used for such applications as automatic inspection and robot and process guidance in industry, for security monitoring and vehicle guidance.
This field encompasses a large number of technologies, software and hardware products, integrated systems, actions, methods and expertise.
Machine vision is practically the only term used for these functions in industrial automation applications; the term is less universal for these functions in other environments such as security and vehicle guidance. Machine vision as a
systems engineering
Systems engineering is an interdisciplinary field of engineering and engineering management that focuses on how to design, integrate, and manage complex systems over their enterprise life cycle, life cycles. At its core, systems engineering util ...
discipline can be considered distinct from
computer vision
Computer vision is an interdisciplinary scientific field that deals with how computers can gain high-level understanding from digital images or videos. From the perspective of engineering, it seeks to understand and automate tasks that the hum ...
, a form of basic
computer science
Computer science is the study of computation, automation, and information. Computer science spans theoretical disciplines (such as algorithms, theory of computation, information theory, and automation) to Applied science, practical discipli ...
; machine vision attempts to integrate existing technologies in new ways and apply them to solve real world problems in a way that meets the requirements of industrial automation and similar application areas.
The term is also used in a broader sense by trade shows and trade groups such as the Automated Imaging Association and the European Machine Vision Association. This broader definition also encompasses products and applications most often associated with image processing.
[ The primary uses for machine vision are automatic inspection and ]industrial robot
An industrial robot is a robot system used for manufacturing. Industrial robots are automated, programmable and capable of movement on three or more axes.
Typical applications of robots include welding, painting, assembly, disassembly, pick a ...
/process guidance.[ See ]glossary of machine vision
The following are common definitions related to the machine vision field.
General related fields
* Machine vision
* Computer vision
* Image processing
* Signal processing
0-9
* 1394. FireWire is Apple Inc.'s brand name for the IEEE 1394 inte ...
.
Imaging based automatic inspection and sorting
The primary uses for machine vision are imaging-based automatic inspection and sorting and robot guidance.;[ in this section the former is abbreviated as "automatic inspection". The overall process includes planning the details of the requirements and project, and then creating a solution.] This section describes the technical process that occurs during the operation of the solution.
Methods and sequence of operation
The first step in the automatic inspection sequence of operation is acquisition of an image, typically using cameras, lenses, and lighting that has been designed to provide the differentiation required by subsequent processing. MV software
Software is a set of computer programs and associated documentation and data. This is in contrast to hardware, from which the system is built and which actually performs the work.
At the lowest programming level, executable code consists ...
packages and programs developed in them then employ various digital image processing
Digital image processing is the use of a digital computer to process digital images through an algorithm. As a subcategory or field of digital signal processing, digital image processing has many advantages over analog image processing. It allo ...
techniques to extract the required information, and often make decisions (such as pass/fail) based on the extracted information.
Equipment
The components of an automatic inspection system usually include lighting, a camera or other imager, a processor, software, and output devices.
Imaging
The imaging device (e.g. camera) can either be separate from the main image processing unit or combined with it in which case the combination is generally called a smart camera
A smart camera (sensor) or intelligent camera (sensor) or (smart) vision sensor or intelligent vision sensor or smart optical sensor or intelligent optical sensor or smart visual sensor or intelligent visual sensor is a machine vision system whic ...
or smart sensor. Inclusion of the full processing function into the same enclosure as the camera is often referred to as embedded processing.[''Critical Considerations for Embedded Vision Design'' by Dave Rice and Amber Thousand ''Photonics Spectra'' magazine published by Laurin Publishing Co. July 2019 issue Pages 60-64] When separated, the connection may be made to specialized intermediate hardware, a custom processing appliance, or a frame grabber
A frame grabber is an electronic device that captures (i.e., "grabs") individual, digital still frames from an analog video signal or a digital video stream. It is usually employed as a component of a computer vision system, in which video frames ...
within a computer using either an analog or standardized digital interface (Camera Link
Camera Link is a serial communication protocol standard''Specifications of the Camera Link Interface Standard for Digital Cameras and Frame Grabbers, Version 1.1'' Automated Imaging Association, Jan 2004 designed for camera interface applications b ...
, CoaXPress CoaXPress (CXP) is a digital interface standard developed for high speed image data transmission in machine vision applications.
The name is a portmanteau of 'express' (as in express train) and 'coaxial' to emphasize CoaXPress is faster than other ...
). MV implementations also use digital cameras capable of direct connections (without a framegrabber) to a computer via FireWire
IEEE 1394 is an interface standard for a serial bus for high-speed communications and isochronous real-time data transfer. It was developed in the late 1980s and early 1990s by Apple in cooperation with a number of companies, primarily Sony an ...
, USB
Universal Serial Bus (USB) is an industry standard that establishes specifications for cables, connectors and protocols for connection, communication and power supply (interfacing) between computers, peripherals and other computers. A broad v ...
or Gigabit Ethernet
In computer networking, Gigabit Ethernet (GbE or 1 GigE) is the term applied to transmitting Ethernet frames at a rate of a gigabit per second. The most popular variant, 1000BASE-T, is defined by the IEEE 802.3ab standard. It came into use i ...
interfaces.
While conventional (2D visible light) imaging is most commonly used in MV, alternatives include multispectral imaging
Multispectral imaging captures image data within specific wavelength ranges across the electromagnetic spectrum. The wavelengths may be separated by filters or detected with the use of instruments that are sensitive to particular wavelengths, ...
, hyperspectral imaging
Hyperspectral imaging collects and processes information from across the electromagnetic spectrum. The goal of hyperspectral imaging is to obtain the spectrum for each pixel in the image of a scene, with the purpose of finding objects, identifyi ...
, imaging various infrared bands, line scan imaging, 3D imaging
In computer vision and computer graphics, 3D reconstruction is the process of capturing the shape and appearance of real objects.
This process can be accomplished either by active or passive methods. If the model is allowed to change its shape i ...
of surfaces and X-ray imaging. Key differentiations within MV 2D visible light imaging are monochromatic vs. color, frame rate
Frame rate (expressed in or FPS) is the frequency (rate) at which consecutive images (frames) are captured or displayed. The term applies equally to film and video cameras, computer graphics, and motion capture systems. Frame rate may also be ca ...
, resolution, and whether or not the imaging process is simultaneous over the entire image, making it suitable for moving processes.[West, Perry ''High Speed, Real-Time Machine Vision '' CyberOptics, pages 1-38]
Though the vast majority of machine vision applications are solved using two-dimensional imaging, machine vision applications utilizing 3D imaging are a growing niche within the industry. The most commonly used method for 3D imaging is scanning based triangulation which utilizes motion of the product or image during the imaging process. A laser is projected onto the surfaces of an object. In machine vision this is accomplished with a scanning motion, either by moving the workpiece, or by moving the camera & laser imaging system. The line is viewed by a camera from a different angle; the deviation of the line represents shape variations. Lines from multiple scans are assembled into a depth map
In 3D computer graphics and computer vision, a depth map is an image or image channel that contains information relating to the distance of the surfaces of scene objects from a viewpoint. The term is related (and may be analogous) to ''depth ...
or point cloud.[ Stereoscopic vision is used in special cases involving unique features present in both views of a pair of cameras.][''3-D Imaging: A practical Overview for Machine Vision'' By Fred Turek & Kim Jackson Quality Magazine, March 2014 issue, Volume 53/Number 3 Pages 6-8] Other 3D methods used for machine vision are time of flight
Time of flight (ToF) is the measurement of the time taken by an object, particle or wave (be it acoustic, electromagnetic, etc.) to travel a distance through a medium. This information can then be used to measure velocity or path length, or as a w ...
and grid based.[ One method is grid array based systems using pseudorandom structured light system as employed by the Microsoft Kinect system circa 2012.][http://research.microsoft.com/en-us/people/fengwu/depth-icip-12.pdf HYBRID STRUCTURED LIGHT FOR SCALABLE DEPTH SENSING Yueyi Zhang, Zhiwei Xiong, Feng Wu University of Science and Technology of China, Hefei, China Microsoft Research Asia, Beijing, China][R.Morano, C.Ozturk, R.Conn, S.Dubin, S.Zietz, J.Nissano, "Structured light using pseudorandom codes", IEEE Transactions on Pattern Analysis and Machine Intelligence 20 (3)(1998)322–327]
Image processing
After an image is acquired, it is processed.[.] Central processing functions are generally done by a CPU, a GPU
A graphics processing unit (GPU) is a specialized electronic circuit designed to manipulate and alter memory to accelerate the creation of images in a frame buffer intended for output to a display device. GPUs are used in embedded systems, mobil ...
, a FPGA
A field-programmable gate array (FPGA) is an integrated circuit designed to be configured by a customer or a designer after manufacturinghence the term '' field-programmable''. The FPGA configuration is generally specified using a hardware de ...
or a combination of these.[ Deep learning training and inference impose higher processing performance requirements.][''Finding the optimal hardware for deep learining inference in machine vision'' by Mike Fussell Vision Systems Design magazine September 2019 issue pages 8-9] Multiple stages of processing are generally used in a sequence that ends up as a desired result. A typical sequence might start with tools such as filters which modify the image, followed by extraction of objects, then extraction (e.g. measurements, reading of codes) of data from those objects, followed by communicating that data, or comparing it against target values to create and communicate "pass/fail" results. Machine vision image processing methods include;
* Stitching/Registration
Register or registration may refer to:
Arts entertainment, and media Music
* Register (music), the relative "height" or range of a note, melody, part, instrument, etc.
* ''Register'', a 2017 album by Travis Miller
* Registration (organ), th ...
: Combining of adjacent 2D or 3D images.
* Filtering (e.g. morphological filtering)
* Thresholding: Thresholding starts with setting or determining a gray value that will be useful for the following steps. The value is then used to separate portions of the image, and sometimes to transform each portion of the image to simply black and white based on whether it is below or above that grayscale value.
* Pixel counting: counts the number of light or dark pixel
In digital imaging, a pixel (abbreviated px), pel, or picture element is the smallest addressable element in a raster image, or the smallest point in an all points addressable display device.
In most digital display devices, pixels are the smal ...
s
* Segmentation: Partitioning a digital image
A digital image is an image composed of picture elements, also known as ''pixels'', each with ''finite'', '' discrete quantities'' of numeric representation for its intensity or gray level that is an output from its two-dimensional functions ...
into multiple segments to simplify and/or change the representation of an image into something that is more meaningful and easier to analyze.[ Linda G. Shapiro and George C. Stockman (2001): “Computer Vision”, pp 279-325, New Jersey, Prentice-Hall, ][Lauren Barghout. Visual Taxometric approach Image Segmentation using Fuzzy-Spatial Taxon Cut Yields Contextually Relevant Regions. Information Processing and Management of Uncertainty in Knowledge-Based
Systems. CCIS Springer-Verlag. 2014]
* Edge detection
Edge detection includes a variety of mathematical methods that aim at identifying edges, curves in a digital image at which the image brightness changes sharply or, more formally, has discontinuities. The same problem of finding discontinuitie ...
: finding object edges
* Color Analysis: Identify parts, products and items using color, assess quality from color, and isolate features
Feature may refer to:
Computing
* Feature (CAD), could be a hole, pocket, or notch
* Feature (computer vision), could be an edge, corner or blob
* Feature (software design) is an intentional distinguishing characteristic of a software item ...
using color.[
* Blob detection and extraction: inspecting an image for discrete blobs of connected pixels (e.g. a black hole in a grey object) as image landmarks.]
* Neural net
Artificial neural networks (ANNs), usually simply called neural networks (NNs) or neural nets, are computing systems inspired by the biological neural networks that constitute animal brains.
An ANN is based on a collection of connected unit ...
/ deep learning
Deep learning (also known as deep structured learning) is part of a broader family of machine learning methods based on artificial neural networks with representation learning. Learning can be supervised, semi-supervised or unsupervised.
De ...
/ machine learning
Machine learning (ML) is a field of inquiry devoted to understanding and building methods that 'learn', that is, methods that leverage data to improve performance on some set of tasks. It is seen as a part of artificial intelligence.
Machine ...
processing: weighted and self-training multi-variable decision making Circa 2019 there is a large expansion of this, using deep learning and machine learning to significantly expand machine vision capabilities. The most common result of such processing is classification. Examples of classification are object identification,"pass fail" classification of identified objects and OCR.
* Pattern recognition
Pattern recognition is the automated recognition of patterns and regularities in data. It has applications in statistical data analysis, signal processing, image analysis, information retrieval, bioinformatics, data compression, computer graphi ...
including template matching
Template matching is a technique in digital image processing for finding small parts of an image which match a template image. It can be used in manufacturing as a part of quality control, a way to navigate a mobile robot, or as a way to detect ...
. Finding, matching, and/or counting specific patterns. This may include location of an object that may be rotated, partially hidden by another object, or varying in size.
* Barcode
A barcode or bar code is a method of representing data in a visual, machine-readable form. Initially, barcodes represented data by varying the widths, spacings and sizes of parallel lines. These barcodes, now commonly referred to as linear or o ...
, Data Matrix
A Data Matrix is a two-dimensional code consisting of black and white "cells" or dots arranged in either a square or rectangular pattern, also known as a matrix. The information to be encoded can be text or numeric data. Usual data size is fro ...
and "2D barcode
A barcode or bar code is a method of representing data in a visual, machine-readable form. Initially, barcodes represented data by varying the widths, spacings and sizes of parallel lines. These barcodes, now commonly referred to as linear or o ...
" reading
* Optical character recognition
Optical character recognition or optical character reader (OCR) is the electronic or mechanical conversion of images of typed, handwritten or printed text into machine-encoded text, whether from a scanned document, a photo of a document, a scen ...
: automated reading of text such as serial numbers
* Gauging/Metrology: measurement of object dimensions (e.g. in pixel
In digital imaging, a pixel (abbreviated px), pel, or picture element is the smallest addressable element in a raster image, or the smallest point in an all points addressable display device.
In most digital display devices, pixels are the smal ...
s, inch
Measuring tape with inches
The inch (symbol: in or ″) is a unit of length in the British imperial and the United States customary systems of measurement. It is equal to yard or of a foot. Derived from the Roman uncia ("twelfth") ...
es or millimeter
330px, Different lengths as in respect to the electromagnetic spectrum, measured by the metre and its derived scales. The microwave is between 1 meter to 1 millimeter.
The millimetre (American and British English spelling differences#-re, -er, ...
s)
*Comparison against target values to determine a "pass or fail" or "go/no go" result. For example, with code or bar code verification, the read value is compared to the stored target value. For gauging, a measurement is compared against the proper value and tolerances. For verification of alpha-numberic codes, the OCR'd value is compared to the proper or target value. For inspection for blemishes, the measured size of the blemishes may be compared to the maximums allowed by quality standards.
Outputs
A common output from automatic inspection systems is pass/fail decisions.[ These decisions may in turn trigger mechanisms that reject failed items or sound an alarm. Other common outputs include object position and orientation information for robot guidance systems.][ Additionally, output types include numerical measurement data, data read from codes and characters, counts and classification of objects, displays of the process or results, stored images, alarms from automated space monitoring MV systems, and ]process control
An industrial process control in continuous production processes is a discipline that uses industrial control systems to achieve a production level of consistency, economy and safety which could not be achieved purely by human manual control. I ...
signals.[West, Perry ''A Roadmap For Building A Machine Vision System'' Pages 1-35][ This also includes user interfaces, interfaces for the integration of multi-component systems and automated data interchange.]
Deep Learning
The term "Deep Learning" has variable meanings, most of which can be applied to techniques used in machine vision for over 20 years. However the usage of the term in Machine Vision began in the later 2010s with the advent of the capability to successfully apply such techniques to entire images in the industrial machine vision space.[''The Place for Deep Learning in Machine Vision'' Quality Magazine May 2022 issue, Volume 61, Number 5 Published by BNP Media II] Conventional machine vision usually requires the "physics" phase of a machine vision automatic inspection solution to create reliable ''simple'' differentiation of defects. An example of "simple" differentiation is that the defects are dark and the good parts of the product are light. A common reason why some applications were not doable was when it was impossible to achieve the "simple"; deep learning removes this requirement, in essence "seeing" the object more as a human does, making it now possible to accomplish those automatic applications.[ The system learns from a large amount of images during a training phase and then executes the inspection during run-time use which is called "inference".][
]
Imaging based robot guidance
Machine vision commonly provides location and orientation information to a robot to allow the robot to properly grasp the product. This capability is also used to guide motion that is simpler than robots, such as a 1 or 2 axis motion controller.[ The overall process includes planning the details of the requirements and project, and then creating a solution. This section describes the technical process that occurs during the operation of the solution. Many of the process steps are the same as with automatic inspection except with a focus on providing position and orientation information as the result.][
]
Market
As recently as 2006, one industry consultant reported that MV represented a $1.5 billion market in North America. However, the editor-in-chief of an MV trade magazine asserted that "machine vision is not an industry per se" but rather "the integration of technologies and products that provide services or applications that benefit true industries such as automotive or consumer goods manufacturing, agriculture, and defense."
See also
* Machine vision glossary
The following are common definitions related to the machine vision field.
General related fields
* Machine vision
* Computer vision
* Image processing
* Signal processing
0-9
* 1394. FireWire is Apple Inc.'s brand name for the IEEE 1394 inter ...
* Feature detection (computer vision)
In computer vision and image processing, a feature is a piece of information about the content of an image; typically about whether a certain region of the image has certain properties. Features may be specific structures in the image such as poi ...
* Foreground detection Foreground and background or background and foreground may refer to:
*Background, foreground, sideground and postground intellectual property, distinct forms of intellectual property assets
* Background subtraction, a technique in image processing ...
* Vision processing unit
A vision processing unit (VPU) is (as of 2018) an emerging class of microprocessor; it is a specific type of AI accelerator, designed to accelerate machine vision tasks.
Overview
Vision processing units are distinct from video processing units ...
* Optical sorting
Optical sorting (sometimes called digital sorting) is the automated process of sorting solid products using cameras and/or lasers.
Depending on the types of sensors used and the software-driven intelligence of the image processing system, optical ...
References
{{DEFAULTSORT:Machine Vision
Applications of computer vision
Computer vision