Small object detection is a particular case of
object detection
Object detection is a computer technology related to computer vision and image processing that deals with detecting instances of semantic objects of a certain class (such as humans, buildings, or cars) in digital images and videos. Well-researched ...
where various techniques are employed to detect small objects in digital images and videos. "Small objects" are objects having a small pixel footprint in the input image. In areas such as
aerial imagery
Aerial photography (or airborne imagery) is the taking of photographs from an aircraft or other airborne platforms. When taking motion pictures, it is also known as aerial videography.
Platforms for aerial photography include fixed-wing aircra ...
,
state-of-the-art
The state of the art (sometimes cutting edge or leading edge) refers to the highest level of general development, as of a device, technique, or scientific field achieved at a particular time. However, in some contexts it can also refer to a level ...
object detection techniques under performed because of small objects.
Uses
Small object detection has applications in various fields such as Video
surveillance
Surveillance is the monitoring of behavior, many activities, or information for the purpose of information gathering, influencing, managing or directing. This can include observation from a distance by means of electronic equipment, such as c ...
(Traffic video Surveillance,
Small object retrieval,
Anomaly detection
In data analysis, anomaly detection (also referred to as outlier detection and sometimes as novelty detection) is generally understood to be the identification of rare items, events or observations which deviate significantly from the majority ...
,
Maritime surveillance {{Unreferenced, date=March 2008
Maritime patrol is the task of monitoring areas of water. Generally conducted by military and law enforcement agencies, maritime patrol is usually aimed at identifying human activities.
Maritime patrol refers to ac ...
,
Drone surveying,
Traffic flow analysis, and
Object tracking.
Problems with small objects
* Modern-day object detection algorithms such as You Only Look Once(YOLO) heavily uses convolution layers to learn
features
Feature may refer to:
Computing
* Feature (CAD), could be a hole, pocket, or notch
* Feature (computer vision), could be an edge, corner or blob
* Feature (software design) is an intentional distinguishing characteristic of a software item ...
. As an object passes through convolution layers, its size gets reduced. Therefore, the small object disappears after several layers and becomes undetectable.
* Sometimes, the shadow of an object is detected as a part of object itself. So, the placement of the bounding box tends to centre around a shadow rather than an object. In the case of vehicle detection,
pedestrian
A pedestrian is a person traveling on foot, whether walking or running. In modern times, the term usually refers to someone walking on a road or pavement, but this was not the case historically.
The meaning of pedestrian is displayed with ...
and two-wheeler detection suffer because of this.
* At present,
drones are very widely used in aerial imagery. They are equipped with hardware (
sensor
A sensor is a device that produces an output signal for the purpose of sensing a physical phenomenon.
In the broadest definition, a sensor is a device, module, machine, or subsystem that detects events or changes in its environment and sends ...
s) and software (
algorithm
In mathematics and computer science, an algorithm () is a finite sequence of rigorous instructions, typically used to solve a class of specific Computational problem, problems or to perform a computation. Algorithms are used as specificat ...
s) that help maintain a particular stable position during their flight. In windy conditions, the drone automatically makes fine moves to maintain its position and that changes the view near the boundary. It may be possible that some new objects appear near the image boundary. Overall, these affect classification, detection, and eventually tracking accuracy.
Methods
Various methods are available to detect small objects, which fall under three categories:
Improvising existing techniques
There are various ways to detect small objects with existing techniques. Some of them are mentioned below,
Choosing a data set that has small objects
The
machine learning
Machine learning (ML) is a field of inquiry devoted to understanding and building methods that 'learn', that is, methods that leverage data to improve performance on some set of tasks. It is seen as a part of artificial intelligence.
Machine ...
model's output depends on "How well it is trained."
So, the data set must include small objects to detect such objects. Also, modern-day detectors, such as YOLO, rely on anchors. Latest versions of YOLO (starting from YOLOv5) uses an auto-anchor algorithm to find good anchors based on the nature of object sizes in the data set. Therefore, it is mandatory to have smaller objects in the data set.
Generating more data via augmentation, if required
Deep learning
Deep learning (also known as deep structured learning) is part of a broader family of machine learning methods based on artificial neural networks with representation learning. Learning can be supervised, semi-supervised or unsupervised.
De ...
models have billions of neurons that settle down to some weights after training. Therefore, it requires a good amount of quantitative and qualitative data for better training.
Data augmentation
Data augmentation in data analysis are techniques used to increase the amount of data by adding slightly modified copies of already existing data or newly created synthetic data from existing data. It acts as a regularizer and helps reduce over ...
is useful technique to generate more diverse data
from an existing data set.
Increasing image capture resolution and model’s input resolution
These help to get more features from objects and eventually learn the best from them. For example, a bike object in the 1280 X 1280
resolution
Resolution(s) may refer to:
Common meanings
* Resolution (debate), the statement which is debated in policy debate
* Resolution (law), a written motion adopted by a deliberative body
* New Year's resolution, a commitment that an individual mak ...
image has more features than the 640 X 640 resolution.
Auto learning anchors
Selecting anchor size plays a vital role in small object detection. Instead of hand picking it, use algorithms that identify it based on the data set. YOLOv5 uses a
K-means algorithm
''k''-means clustering is a method of vector quantization, originally from signal processing, that aims to partition ''n'' observations into ''k'' clusters in which each observation belongs to the cluster with the nearest mean (cluster centers or ...
to define anchor size.
Tiling approach during training and inference
State-of-the-art object detectors allow only the fixed size of image and change the input image size according to it. This change may deform the small objects in the image. The tiling approach helps when an image has a high resolution than the model's fixed input size; instead of scaling it down, the image is broken down into tiles and then used in training. The same approach is used during inference as well.
Feature Pyramid Network (FPN)
Use a feature
pyramid
A pyramid (from el, πυραμίς ') is a structure whose outer surfaces are triangular and converge to a single step at the top, making the shape roughly a pyramid in the geometric sense. The base of a pyramid can be trilateral, quadrilat ...
network to learn features at a multi-scale: e.g., Twin Feature Pyramid Networks (TFPN), Extended Feature Pyramid Network (EFPN). FPN helps to sustain features of small objects against convolution layers.
Add-on techniques
Instead of modifying existing methods, some add-on techniques are there, which can be directly placed on top of existing approaches to detect smaller objects. One such technique is Slicing Aided Hyper Inference(SAHI). The image is sliced into different-sized multiple overlapping patches.
Hyper-parameters define their dimensions. Then patches are resized, while maintaining the aspect ratio during fine-tuning. These patches are then provided for training the model.
Well-Optimised techniques for small object detection
Various deep learning techniques are available that focus on such object detection problems: e.g., Feature-Fused SSD, YOLO-Z. Such methods work on "How to sustain features of small objects while they pass through convolution networks."
Other applications
* Crowd counting
* Vehicle re-identification
* Animal detection
* Fish detection
[{{Cite journal , title=Fish Detection Using Deep Learning , journal=Applied Computational Intelligence and Soft Computing , year=2020 , language=en , doi=10.1155/2020/3738108, doi-access=free , last1=Cui , first1=Suxia , last2=Zhou , first2=Yu , last3=Wang , first3=Yonghui , last4=Zhai , first4=Lujun , volume=2020 , pages=1–13 ]
See also
*
Convolutional neural network
In deep learning, a convolutional neural network (CNN, or ConvNet) is a class of artificial neural network (ANN), most commonly applied to analyze visual imagery. CNNs are also known as Shift Invariant or Space Invariant Artificial Neural Netwo ...
*
Use of UAVs in law enforcement
Unmanned aerial vehicles (UAVs) have been used for domestic police work in various countries around the world since the mid-2000s. Their appeal comes from their small size, lack of crew, and lower cost compared to police helicopters. UAVs may be ...
*
Underwater computer vision
*
Intelligent transportation system
References
External links
VisDronedataset by AISKYEYE team at Lab of Machine Learning and Data Mining, Tianjin University, China.
Image sensors
Imaging