![MnistExamples](https://upload.wikimedia.org/wikipedia/commons/2/27/MnistExamples.png)
The MNIST database (''Modified
National Institute of Standards and Technology
The National Institute of Standards and Technology (NIST) is an agency of the United States Department of Commerce whose mission is to promote American innovation and industrial competitiveness. NIST's activities are organized into physical sci ...
database'') is a large
database
In computing, a database is an organized collection of data stored and accessed electronically. Small databases can be stored on a file system, while large databases are hosted on computer clusters or cloud storage. The design of databases sp ...
of handwritten digits that is commonly used for
training
Training is teaching, or developing in oneself or others, any skills and knowledge or Physical fitness, fitness that relate to specific practicality, useful Competence (human resources), competencies. Training has specific goals of improving on ...
various
image processing
An image is a visual representation of something. It can be two-dimensional, three-dimensional, or somehow otherwise feed into the visual system to convey information. An image can be an artifact, such as a photograph or other two-dimensiona ...
systems. The database is also widely used for training and testing in the field of
machine learning
Machine learning (ML) is a field of inquiry devoted to understanding and building methods that 'learn', that is, methods that leverage data to improve performance on some set of tasks. It is seen as a part of artificial intelligence.
Machine ...
. It was created by "re-mixing" the samples from NIST's original datasets. The creators felt that since NIST's training dataset was taken from American
Census Bureau
The United States Census Bureau (USCB), officially the Bureau of the Census, is a principal agency of the U.S. Federal Statistical System, responsible for producing data about the American people and economy. The Census Bureau is part of the ...
employees, while the testing dataset was taken from
American
American(s) may refer to:
* American, something of, from, or related to the United States of America, commonly known as the "United States" or "America"
** Americans, citizens and nationals of the United States of America
** American ancestry, pe ...
high school
A secondary school describes an institution that provides secondary education and also usually includes the building where this takes place. Some secondary schools provide both '' lower secondary education'' (ages 11 to 14) and ''upper seconda ...
students, it was not well-suited for machine learning experiments.
Furthermore, the black and white images from NIST were
normalized to fit into a 28x28 pixel bounding box and
anti-aliased Anti-aliasing may refer to any of a number of techniques to combat the problems of aliasing in a sampled signal such as a digital image or digital audio recording.
Specific topics in anti-aliasing include:
* Anti-aliasing filter, a filter used be ...
, which introduced grayscale levels.
The MNIST database contains 60,000 training images and 10,000 testing images. Half of the training set and half of the test set were taken from NIST's training dataset, while the other half of the training set and the other half of the test set were taken from NIST's testing dataset. The original creators of the database keep a list of some of the methods tested on it.
In their original paper, they use a
support-vector machine
In machine learning, support vector machines (SVMs, also support vector networks) are supervised learning models with associated learning algorithms that analyze data for classification and regression analysis. Developed at AT&T Bell Laboratorie ...
to get an error rate of 0.8%.
Extended MNIST (EMNIST) is a newer dataset developed and released by NIST to be the (final) successor to MNIST. MNIST included images only of handwritten digits. EMNIST includes all the images from NIST Special Database 19, which is a large database of handwritten uppercase and lower case letters as well as digits. The images in EMNIST were converted into the same 28x28 pixel format, by the same process, as were the MNIST images. Accordingly, tools which work with the older, smaller, MNIST dataset will likely work unmodified with EMNIST.
History
The set of images in the MNIST database was created in 1998 as a combination of two of
NIST
The National Institute of Standards and Technology (NIST) is an agency of the United States Department of Commerce whose mission is to promote American innovation and industrial competitiveness. NIST's activities are organized into physical sci ...
's databases: Special Database 1 and Special Database 3. Special Database 1 and Special Database 3 consist of digits written by high school students and employees of the
United States Census Bureau
The United States Census Bureau (USCB), officially the Bureau of the Census, is a principal agency of the U.S. Federal Statistical System, responsible for producing data about the American people and economy. The Census Bureau is part of the ...
, respectively.
Performance
Some researchers have achieved "near-human performance" on the MNIST database, using a committee of neural networks; in the same paper, the authors achieve performance double that of humans on other recognition tasks.
The highest error rate listed
on the original website of the database is 12 percent, which is achieved using a simple linear classifier with no preprocessing.
In 2004, a best-case error rate of 0.42 percent was achieved on the database by researchers using a new classifier called the LIRA, which is a neural classifier with three neuron layers based on Rosenblatt's perceptron principles.
Some researchers have tested artificial intelligence systems using the database put under random distortions. The systems in these cases are usually neural networks and the distortions used tend to be either
affine distortions or
elastic distortions.
Sometimes, these systems can be very successful; one such system achieved an error rate on the database of 0.39 percent.
In 2011, an error rate of 0.27 percent, improving on the previous best result, was reported by researchers using a similar system of neural networks. In 2013, an approach based on regularization of neural networks using DropConnect has been claimed to achieve a 0.21 percent error rate. In 2016, the single convolutional neural network best performance was 0.25 percent error rate.
As of August 2018, the best performance of a single convolutional neural network trained on MNIST training data using no
data augmentation
Data augmentation in data analysis are techniques used to increase the amount of data by adding slightly modified copies of already existing data or newly created synthetic data from existing data. It acts as a regularizer and helps reduce over ...
is 0.25 percent error rate.
Also, the Parallel Computing Center (Khmelnytskyi, Ukraine) obtained an ensemble of only 5 convolutional neural networks which performs on MNIST at 0.21 percent error rate.
Some images in the testing dataset are barely readable and may prevent reaching test error rates of 0%.
In 2018, researchers from Department of System and Information Engineering, University of Virginia announced 0.18% error with simultaneous stacked three kind of neural networks (fully connected, recurrent and convolution neural networks).
Classifiers
This is a table of some of the
machine learning
Machine learning (ML) is a field of inquiry devoted to understanding and building methods that 'learn', that is, methods that leverage data to improve performance on some set of tasks. It is seen as a part of artificial intelligence.
Machine ...
methods used on the dataset and their error rates, by type of classifier:
See also
*
List of datasets for machine learning research
These datasets are applied for machine learning research and have been cited in peer-reviewed academic journals. Datasets are an integral part of the field of machine learning. Major advances in this field can result from advances in learning ...
*
Caltech 101
Caltech 101 is a data set of digital images created in September 2003 and compiled by Fei-Fei Li, Marco Andreetto, Marc 'Aurelio Ranzato and Pietro Perona at the California Institute of Technology. It is intended to facilitate Computer Vision re ...
*
LabelMe
LabelMe is a project created by the MIT Computer Science and Artificial Intelligence Laboratory (CSAIL) which provides a dataset of digital images with annotations. The dataset is dynamic, free to use, and open to public contribution. The most ap ...
*
OCR
References
Further reading
*
External links
*
Visualization of the MNIST databasegroups of images of MNIST handwritten digits on GitHub
{{Standard test item
Datasets in computer vision