In
statistics, the term linear model is used in different ways according to the context. The most common occurrence is in connection with regression models and the term is often taken as synonymous with
linear regression model. However, the term is also used in
time series analysis
In mathematics
Mathematics is an area of knowledge that includes the topics of numbers, formulas and related structures, shapes and the spaces in which they are contained, and quantities and their changes. These topics are represented in m ...
with a different meaning. In each case, the designation "linear" is used to identify a subclass of models for which substantial reduction in the complexity of the related
statistical theory
The theory of statistics provides a basis for the whole range of techniques, in both study design and data analysis, that are used within applications of statistics.
The theory covers approaches to statistical-decision problems and to statistica ...
is possible.
Linear regression models
For the regression case, the
statistical model is as follows. Given a (random) sample
the relation between the observations
and the
independent variables
Dependent and independent variables are variables in mathematical modeling, statistical modeling and experimental sciences. Dependent variables receive this name because, in an experiment, their values are studied under the supposition or deman ...
is formulated as
:
where
may be
nonlinear
In mathematics and science, a nonlinear system is a system in which the change of the output is not proportional to the change of the input. Nonlinear problems are of interest to engineers, biologists, physicists, mathematicians, and many othe ...
functions. In the above, the quantities
are
random variables representing errors in the relationship. The "linear" part of the designation relates to the appearance of the
regression coefficient
In statistics, linear regression is a linear approach for modelling the relationship between a scalar response and one or more explanatory variables (also known as dependent and independent variables). The case of one explanatory variable is cal ...
s,
in a linear way in the above relationship. Alternatively, one may say that the predicted values corresponding to the above model, namely
:
are linear functions of the
.
Given that estimation is undertaken on the basis of a
least squares analysis, estimates of the unknown parameters
are determined by minimising a sum of squares function
:
From this, it can readily be seen that the "linear" aspect of the model means the following:
:*the function to be minimised is a quadratic function of the
for which minimisation is a relatively simple problem;
:*the derivatives of the function are linear functions of the
making it easy to find the minimising values;
:*the minimising values
are linear functions of the observations
;
:*the minimising values
are linear functions of the random errors
which makes it relatively easy to determine the statistical properties of the estimated values of
.
Time series models
An example of a linear time series model is an
autoregressive moving average model
In statistics, econometrics and signal processing, an autoregressive (AR) model is a representation of a type of random process; as such, it is used to describe certain time-varying processes in nature, economics, etc. The autoregressive model spe ...
. Here the model for values in a time series can be written in the form
:
where again the quantities
are random variables representing
innovations
Innovation is the practical implementation of ideas that result in the introduction of new goods or services or improvement in offering goods or services. ISO TC 279 in the standard ISO 56000:2020 defines innovation as "a new or changed entit ...
which are new random effects that appear at a certain time but also affect values of
at later times. In this instance the use of the term "linear model" refers to the structure of the above relationship in representing
as a linear function of past values of the same time series and of current and past values of the innovations.
[Priestley, M.B. (1988) ''Non-linear and Non-stationary time series analysis'', Academic Press. ] This particular aspect of the structure means that it is relatively simple to derive relations for the mean and
covariance
In probability theory and statistics, covariance is a measure of the joint variability of two random variables. If the greater values of one variable mainly correspond with the greater values of the other variable, and the same holds for the ...
properties of the time series. Note that here the "linear" part of the term "linear model" is not referring to the coefficients
and
, as it would be in the case of a regression model, which looks structurally similar.
Other uses in statistics
There are some other instances where "nonlinear model" is used to contrast with a linearly structured model, although the term "linear model" is not usually applied. One example of this is
nonlinear dimensionality reduction.
See also
*
General linear model
The general linear model or general multivariate regression model is a compact way of simultaneously writing several multiple linear regression models. In that sense it is not a separate statistical linear model. The various multiple linear regr ...
*
Generalized linear model
*
Linear predictor function In statistics and in machine learning, a linear predictor function is a linear function ( linear combination) of a set of coefficients and explanatory variables (independent variables), whose value is used to predict the outcome of a dependent vari ...
*
Linear system
In systems theory, a linear system is a mathematical model of a system based on the use of a linear operator.
Linear systems typically exhibit features and properties that are much simpler than the nonlinear case.
As a mathematical abstractio ...
*
Linear regression
*
Statistical model
References
{{Authority control
Curve fitting
Regression models
ar:نموذج الانحدار الخطي
fr:Modèle linéaire