Automatic image annotation (also known as automatic image tagging or linguistic indexing) is the process by which a computer system automatically assigns

metadata Metadata is "data that provides information about other data", but not the content of the data, such as the text of a message or the image itself. There are many distinct types of metadata, including: * Descriptive metadata – the descriptive ...

in the form of

captioning Closed captioning (CC) and subtitling are both processes of displaying text on a television, video screen, or other visual display to provide additional or interpretive information. Both are typically used as a transcription of the audio po ...

keyword Keyword may refer to: Computing * Keyword (Internet search), a word or phrase typically used by bloggers or online content creator to rank a web page on a particular topic * Index term, a term used as a keyword to documents in an information syst ...

s to a digital image. This application of

computer vision Computer vision is an interdisciplinary scientific field that deals with how computers can gain high-level understanding from digital images or videos. From the perspective of engineering, it seeks to understand and automate tasks that the hum ...

techniques is used in

image retrieval An image retrieval system is a computer system used for browsing, searching and retrieving images from a large database of digital images. Most traditional and common methods of image retrieval utilize some method of adding metadata such as captio ...

systems to organize and locate images of interest from a

database In computing, a database is an organized collection of data stored and accessed electronically. Small databases can be stored on a file system, while large databases are hosted on computer clusters or cloud storage. The design of databases sp ...

. This method can be regarded as a type of multi-class

image classification Computer vision is an interdisciplinary scientific field that deals with how computers can gain high-level understanding from digital images or videos. From the perspective of engineering, it seeks to understand and automate tasks that the hum ...

with a very large number of classes - as large as the vocabulary size. Typically,

image analysis Image analysis or imagery analysis is the extraction of meaningful information from images; mainly from digital images by means of digital image processing techniques. Image analysis tasks can be as simple as reading bar coded tags or as sophi ...

in the form of extracted

feature vector In machine learning and pattern recognition, a feature is an individual measurable property or characteristic of a phenomenon. Choosing informative, discriminating and independent features is a crucial element of effective algorithms in pattern r ...

s and the training annotation words are used by

machine learning Machine learning (ML) is a field of inquiry devoted to understanding and building methods that 'learn', that is, methods that leverage data to improve performance on some set of tasks. It is seen as a part of artificial intelligence. Machine ...

techniques to attempt to automatically apply annotations to new images. The first methods learned the correlations between image features and training annotations, then techniques were developed using

machine translation Machine translation, sometimes referred to by the abbreviation MT (not to be confused with computer-aided translation, machine-aided human translation or interactive translation), is a sub-field of computational linguistics that investigates t ...

to try to translate the textual vocabulary with the 'visual vocabulary', or clustered regions known as ''blobs''. Work following these efforts have included classification approaches, relevance models and so on. The advantages of automatic image annotation versus

content-based image retrieval Content-based image retrieval, also known as query by image content ( QBIC) and content-based visual information retrieval (CBVIR), is the application of computer vision techniques to the image retrieval problem, that is, the problem of searching ...

(CBIR) are that queries can be more naturally specified by the user. CBIR generally (at present) requires users to search by image concepts such as color and

texture Texture may refer to: Science and technology * Surface texture, the texture means smoothness, roughness, or bumpiness of the surface of an object * Texture (roads), road surface characteristics with waves shorter than road roughness * Texture ...

, or finding example queries. Certain image features in example images may override the concept that the user is really focusing on. The traditional methods of image retrieval such as those used by libraries have relied on manually annotated images, which is expensive and time-consuming, especially given the large and constantly growing image databases in existence.

References

* * *

: * Texture similarity : * Support Vector Machines : * Ensemble of Decision Trees and Random Subwindows : * Maximum Entropy : * Relevance models : * Relevance models using continuous probability density functions : * Coherent Language Model : * Inference networks : * Multiple Bernoulli distribution : * Multiple design alternatives : * Image captioning : * Natural scene annotation : * Relevant low-level global filters : * Global image features and nonparametric density estimation : * Video semantics : : * Image Annotation Refinement : : : : : * Automatic Image Annotation by Ensemble of Visual Descriptors : * A New Baseline for Image Annotation : Simultaneous Image Classification and Annotation : * TagProp: Discriminative Metric Learning in Nearest Neighbor Models for Image Auto-Annotation : * Image Annotation Using Metric Learning in Semantic Neighbourhoods : * Automatic Image Annotation Using Deep Learning Representations : * Medical Image Annotation using bayesian networks and active learning :{{cite conference , author1 = N. B. Marvasti, author2= E. Yörük and B. Acar, name-list-style=amp, url=https://www.researchgate.net/publication/320935564, title = Computer-Aided Medical Image Annotation: Preliminary Results With Liver Lesions in CT, book-title= IEEE Journal of Biomedical and Health Informatics , year = 2018 Applications of artificial intelligence Applications of computer vision

See also

References

Further reading