Automated machine learning (AutoML) is the process of automating the tasks of applying machine learning to real-world problems. AutoML potentially includes every stage from beginning with a raw dataset to building a machine learning model ready for deployment. AutoML was proposed as an artificial intelligence-based solution to the growing challenge of applying machine learning. The high degree of automation in AutoML aims to allow non-experts to make use of machine learning models and techniques without requiring them to become experts in machine learning. Automating the process of applying machine learning end-to-end additionally offers the advantages of producing simpler solutions, faster creation of those solutions, and models that often outperform hand-designed models. Common techniques used in AutoML include hyperparameter optimization,

meta-learning Meta-learning is a branch of metacognition concerned with learning about one's own learning and learning processes. The term comes from the meta prefix's modern meaning of an abstract recursion, or "X about X", similar to its use in metaknowled ...

and neural architecture search.

Comparison to the standard approach

In a typical machine learning application, practitioners have a set of input data points to be used for training. The raw data may not be in a form that all algorithms can be applied to. To make the data amenable for machine learning, an expert may have to apply appropriate data pre-processing, feature engineering, feature extraction, and feature selection methods. After these steps, practitioners must then perform

algorithm selection Algorithm selection (sometimes also called per-instance algorithm selection or offline algorithm selection) is a meta-algorithmic technique to choose an algorithm from a portfolio on an instance-by-instance basis. It is motivated by the observati ...

and hyperparameter optimization to maximize the predictive performance of their model. If deep learning is used, the architecture of the neural network must also be chosen by the machine learning expert. Each of these steps may be challenging, resulting in significant hurdles to using machine learning. AutoML aims to simplify these steps for non-experts, and to make it easier for them to use machine learning techniques correctly and effectively. AutoML plays an important role within the broader approach of automating

data science Data science is an interdisciplinary field that uses scientific methods, processes, algorithms and systems to extract or extrapolate knowledge and insights from noisy, structured and unstructured data, and apply knowledge from data across a br ...

, which also includes challenging tasks such as data engineering, data exploration and model interpretation.

Targets of automation

Automated machine learning can target various stages of the machine learning process. Steps to automate are: * Data preparation and ingestion (from raw data and miscellaneous formats) ** Column

type Type may refer to: Science and technology Computing * Typing, producing text via a keyboard, typewriter, etc. * Data type, collection of values used for computations. * File type * TYPE (DOS command), a command to display contents of a file. * Ty ...

detection; e.g., boolean, discrete numerical, continuous numerical, or text ** Column intent detection; e.g., target/label, stratification field, numerical feature, categorical text feature, or free text feature ** Task detection; e.g., binary classification,

regression Regression or regressions may refer to: Science * Marine regression, coastal advance due to falling sea level, the opposite of marine transgression * Regression (medicine), a characteristic of diseases to express lighter symptoms or less extent ( ...

, clustering, or ranking * Feature engineering ** Feature selection ** Feature extraction **

Meta-learning Meta-learning is a branch of metacognition concerned with learning about one's own learning and learning processes. The term comes from the meta prefix's modern meaning of an abstract recursion, or "X about X", similar to its use in metaknowled ...

and transfer learning ** Detection and handling of skewed data and/or missing values * Model selection - choosing which machine learning algorithm to use, often including multiple competing software implementations * Ensembling - a form of consensus where using multiple models often gives better results than any single model * Hyperparameter optimization of the learning algorithm and featurization * Pipeline selection under time, memory, and complexity constraints * Selection of evaluation metrics and validation procedures * Problem checking ** Leakage detection ** Misconfiguration detection * Analysis of obtained results * Creating user interfaces and visualizations

Comparison to the standard approach

Targets of automation

See also

References

Further reading