Polynomial chaos (PC), also called polynomial chaos expansion (PCE) and Wiener chaos expansion, is a method for
representing a
random variable in terms of a
polynomial function of other random variables. The polynomials are chosen to be
orthogonal with respect to the joint
probability distribution of these random variables. PCE can be used, e.g., to determine the evolution of
uncertainty
Uncertainty refers to epistemic situations involving imperfect or unknown information. It applies to predictions of future events, to physical measurements that are already made, or to the unknown. Uncertainty arises in partially observable ...
in a
dynamical system
In mathematics, a dynamical system is a system in which a function describes the time dependence of a point in an ambient space. Examples include the mathematical models that describe the swinging of a clock pendulum, the flow of water in ...
when there is probabilistic uncertainty in the system parameters. Note that despite its name, PCE has no immediate connections to
chaos theory.
PCE was first introduced in 1938 by
Norbert Wiener using
Hermite polynomials to model
stochastic processes with
Gaussian
Carl Friedrich Gauss (1777–1855) is the eponym of all of the topics listed below.
There are over 100 topics all named after this German mathematician and scientist, all in the fields of mathematics, physics, and astronomy. The English eponymo ...
random
In common usage, randomness is the apparent or actual lack of pattern or predictability in events. A random sequence of events, symbols or steps often has no order and does not follow an intelligible pattern or combination. Individual ra ...
variables.
It was introduced to the physics and engineering community by R. Ghanem and P. D. Spanos in 1991
and generalized to other orthogonal polynomial families by D. Xiu and
G. E. Karniadakis in 2002. Mathematically rigorous proofs of existence and convergence of generalized PCE were given by O. G. Ernst and coworkers in 2012.
PCE has found widespread use in engineering and the applied sciences because it makes it possible to efficiently deal with probabilistic uncertainty in the parameters of a system. It is widely used in
stochastic finite element analysis and as a
surrogate model to facilitate
uncertainty quantification analyses.
Main principles
Polynomial chaos expansion (PCE) provides a way to represent a
random variable with finite variance (i.e.,
) as a function of an
-dimensional
random vector
In probability, and statistics, a multivariate random variable or random vector is a list of mathematical variables each of whose value is unknown, either because the value has not yet occurred or because there is imperfect knowledge of its value ...
, using a polynomial basis that is orthogonal to the distribution of this random vector. The prototypical PCE can be written as:
:
In this expression,
is a coefficient and
denotes a polynomial basis function. Depending on the distribution of
, different PCE types are distinguished.
Hermite polynomial chaos
The original PCE formulation used by
Norbert Wiener was limited to the case where
is a random vector with a Gaussian distribution. Considering only the one-dimensional case (i.e.,
and
), the polynomial basis function orthogonal w.r.t. the Gaussian distribution are the set of
-th degree
Hermite polynomials . The PCE of
can then be written as:
:
.
Generalized polynomial chaos
Xiu (in his PhD under Karniadakis at Brown University) generalized the
result of Cameron–Martin to various continuous and discrete distributions using
orthogonal polynomials
In mathematics, an orthogonal polynomial sequence is a family of polynomials such that any two different polynomials in the sequence are orthogonal to each other under some inner product.
The most widely used orthogonal polynomials are the class ...
from the so-called
Askey-scheme and demonstrated
convergence in the corresponding Hilbert functional space. This is popularly known as the generalized polynomial chaos (gPC) framework. The gPC framework has been applied to applications including stochastic
fluid dynamics, stochastic finite elements, solid
mechanics
Mechanics (from Ancient Greek: μηχανική, ''mēkhanikḗ'', "of machines") is the area of mathematics and physics concerned with the relationships between force, matter, and motion among physical objects. Forces applied to object ...
, nonlinear estimation, the evaluation of finite word-length effects in non-linear fixed-point digital systems and
probabilistic robust control. It has been demonstrated that gPC based methods are computationally superior to
Monte-Carlo
Monte Carlo (; ; french: Monte-Carlo , or colloquially ''Monte-Carl'' ; lij, Munte Carlu ; ) is officially an administrative area of the Principality of Monaco, specifically the ward of Monte Carlo/Spélugues, where the Monte Carlo Casino is ...
based methods in a number of applications. However, the method has a notable limitation. For large numbers of random variables, polynomial chaos becomes very computationally expensive and Monte-Carlo methods are typically more feasible .
Arbitrary polynomial chaos
Recently chaos expansion received a generalization towards the arbitrary polynomial chaos expansion (aPC), which is a so-called data-driven generalization of the PC. Like all polynomial chaos expansion techniques, aPC approximates the dependence of simulation model output on model parameters by expansion in an orthogonal polynomial basis. The aPC generalizes chaos expansion techniques towards arbitrary distributions with arbitrary probability measures, which can be either discrete, continuous, or discretized continuous and can be specified either analytically (as probability density/cumulative distribution functions), numerically as histogram or as raw data sets. The aPC at finite expansion order only demands the existence of a finite number of moments and does not require the complete knowledge or even existence of a probability density function. This avoids the necessity to assign parametric probability distributions that are not sufficiently supported by limited available data. Alternatively, it allows modellers to choose freely of technical constraints the shapes of their statistical assumptions. Investigations indicate that the aPC shows an exponential convergence rate and converges faster than classical polynomial chaos expansion techniques. Yet these techniques are in progress but the impact of them on CFD models is quite impressionable.
Polynomial chaos & incomplete statistical information
In many practical situations, only incomplete and inaccurate statistical knowledge on uncertain input parameters are available. Fortunately, to construct a finite-order expansion, only some partial information on the probability measure is required that can be simply represented by a finite number of statistical moments. Any order of expansion is only justified if accompanied by reliable statistical information on input data. Thus, incomplete statistical information limits the utility of high-order polynomial chaos expansions.
Polynomial chaos & non-linear prediction
Polynomial chaos can be utilized in the prediction of non-linear
functionals of
Gaussian
Carl Friedrich Gauss (1777–1855) is the eponym of all of the topics listed below.
There are over 100 topics all named after this German mathematician and scientist, all in the fields of mathematics, physics, and astronomy. The English eponymo ...
stationary increment processes conditioned on their past realizations.
[Daniel Alpay and Alon Kipnis, Wiener Chaos Approach to Optimal Prediction, Numerical Functional Analysis and Optimization, 36:10, 1286-1306, 2015. DOI: 10.1080/01630563.2015.1065273] Specifically, such prediction is obtained by deriving the chaos expansion of the functional with respect to a special
basis
Basis may refer to:
Finance and accounting
* Adjusted basis, the net cost of an asset after adjusting for various tax-related items
*Basis point, 0.01%, often used in the context of interest rates
* Basis trading, a trading strategy consisting ...
for the Gaussian
Hilbert space generated by the process that with the property that each basis element is either measurable or independent with respect to the given samples. For example, this approach leads to an easy prediction formula for the
Fractional Brownian motion In probability theory, fractional Brownian motion (fBm), also called a fractal Brownian motion, is a generalization of Brownian motion. Unlike classical Brownian motion, the increments of fBm need not be independent. fBm is a continuous-time Gauss ...
.
Bayesian polynomial chaos
In a non-intrusive setting, the estimation of the expansion coefficients
for a given set of basis functions
can considered a
Bayesian regression
Bayesian linear regression is a type of conditional modeling in which the mean of one variable is described by a linear combination of other variables, with the goal of obtaining the posterior probability of the regression coefficients (as well ...
problem by constructing a
surrogate model. This approach has benefits in that analytical expressions for the data evidence (in the sense of
Bayesian inference) as well as the uncertainty of the expansion coefficients are available.
The evidence then can be used as a measure for the selection of expansion terms and pruning of the series (see also
Bayesian model comparison). The uncertainty of the expansion coefficients can be used to assess the quality and trustworthiness of the PCE, and furthermore the impact of this assessment on the actual quantity of interest
.
Let
be a set of
pairs of input-output data that is used to estimate the expansion coefficients
. Let
be the data matrix with elements
, let
be the set of
output data written in vector form, and let be
the set of expansion coefficients in vector form. Under the assumption that the uncertainty of the PCE is of
Gaussian type with unknown variance and a scale-invariant
prior, the
expectation value
In probability theory, the expected value (also called expectation, expectancy, mathematical expectation, mean, average, or first moment) is a generalization of the weighted average. Informally, the expected value is the arithmetic mean of a l ...
for the expansion coefficients is
With
, then the covariance of the coefficients is
where
is the minimal misfit and
is the identity matrix. The uncertainty of the estimate for the coefficient
is then given by
.Thus the uncertainty of the estimate for expansion coefficients can be obtained with simple vector-matrix multiplications. For a given input propability density function
, it was shown the second moment for the quantity of interest then simply is
This equation amounts the matrix-vector multiplications above plus the
marginalization
Social exclusion or social marginalisation is the social disadvantage and relegation to the fringe of society. It is a term that has been used widely in Europe and was first used in France in the late 20th century. It is used across discipline ...
with respect to
. The first term
determines the primary uncertainty of the quantity of interest
, as obtained based on the PCE used as a surrogate. The second term
constitutes an additional
inferential uncertainty (often of mixed aleatoric-epistemic type) in the quantity of interest
that is due to a finite uncertainty of the PCE.
If enough data is available, in terms of quality and quantity, it can be shown that
becomes negligibly small and becomes small
This can be judged by simply building the ratios of the two terms, e.g.
.This ratio quantifies the amount of the PCE's own uncertainty in the total uncertainty and is in the interval