In constrained least squares one solves a
linear least squares
Linear least squares (LLS) is the least squares approximation of linear functions to data.
It is a set of formulations for solving statistical problems involved in linear regression, including variants for ordinary (unweighted), weighted, and ...
problem with an additional constraint on the solution.
I.e., the unconstrained equation
must be fit as closely as possible (in the least squares sense) while ensuring that some other property of
is maintained.
There are often special-purpose algorithms for solving such problems efficiently. Some examples of constraints are given below:
*
Equality constrained least squares: the elements of
must exactly satisfy
(see
Ordinary least squares
In statistics, ordinary least squares (OLS) is a type of linear least squares method for choosing the unknown parameters in a linear regression model (with fixed level-one effects of a linear function of a set of explanatory variables) by the ...
).
*
Regularized least squares: the elements of
must satisfy
(choosing
in proportion to the noise standard deviation of y prevents over-fitting).
*
Non-negative least squares (NNLS): The vector
must satisfy the
vector inequality defined componentwise—that is, each component must be either positive or zero.
* Box-constrained least squares: The vector
must satisfy the
vector inequalities , each of which is defined componentwise.
* Integer-constrained least squares: all elements of
must be
integer
An integer is the number zero (), a positive natural number (, , , etc.) or a negative integer with a minus sign (−1, −2, −3, etc.). The negative numbers are the additive inverses of the corresponding positive numbers. In the language ...
s (instead of
real number
In mathematics, a real number is a number that can be used to measure a ''continuous'' one-dimensional quantity such as a distance, duration or temperature. Here, ''continuous'' means that values can have arbitrarily small variations. Every real ...
s).
* Phase-constrained least squares: all elements of
must be real numbers, or multiplied by the same complex number of unit modulus.
If the constraint only applies to some of the variables, the mixed problem may be solved using separable least squares by letting