HOME

TheInfoList



OR:

In
statistics Statistics (from German language, German: ''wikt:Statistik#German, Statistik'', "description of a State (polity), state, a country") is the discipline that concerns the collection, organization, analysis, interpretation, and presentation of ...
, the Pearson correlation coefficient (PCC, pronounced ) ― also known as Pearson's ''r'', the Pearson product-moment correlation coefficient (PPMCC), the bivariate correlation, or colloquially simply as the correlation coefficient ― is a measure of
linear Linearity is the property of a mathematical relationship (''function'') that can be graphically represented as a straight line. Linearity is closely related to '' proportionality''. Examples in physics include rectilinear motion, the linear r ...
correlation In statistics, correlation or dependence is any statistical relationship, whether causal or not, between two random variables or bivariate data. Although in the broadest sense, "correlation" may indicate any type of association, in statistics ...
between two sets of data. It is the ratio between the
covariance In probability theory and statistics, covariance is a measure of the joint variability of two random variables. If the greater values of one variable mainly correspond with the greater values of the other variable, and the same holds for the les ...
of two variables and the product of their
standard deviation In statistics, the standard deviation is a measure of the amount of variation or dispersion of a set of values. A low standard deviation indicates that the values tend to be close to the mean (also called the expected value) of the set, while ...
s; thus, it is essentially a normalized measurement of the covariance, such that the result always has a value between −1 and 1. As with covariance itself, the measure can only reflect a linear correlation of variables, and ignores many other types of relationships or correlations. As a simple example, one would expect the age and height of a sample of teenagers from a high school to have a Pearson correlation coefficient significantly greater than 0, but less than 1 (as 1 would represent an unrealistically perfect correlation).


Naming and history

It was developed by
Karl Pearson Karl Pearson (; born Carl Pearson; 27 March 1857 – 27 April 1936) was an English mathematician and biostatistician. He has been credited with establishing the discipline of mathematical statistics. He founded the world's first university st ...
from a related idea introduced by
Francis Galton Sir Francis Galton, FRS FRAI (; 16 February 1822 – 17 January 1911), was an English Victorian era polymath: a statistician, sociologist, psychologist, anthropologist, tropical explorer, geographer, inventor, meteorologist, proto- ...
in the 1880s, and for which the mathematical formula was derived and published by
Auguste Bravais Auguste Bravais (; 23 August 1811, Annonay, Ardèche – 30 March 1863, Le Chesnay, France) was a French physicist known for his work in crystallography, the conception of Bravais lattices, and the formulation of Bravais law. Bravais also studied ...
in 1844. The naming of the coefficient is thus an example of
Stigler's Law Stigler's law of eponymy, proposed by University of Chicago statistics professor Stephen Stigler in his 1980 publication ''Stigler’s law of eponymy'', states that no scientific discovery is named after its original discoverer. Examples include ...
.


Definition

Pearson's correlation coefficient is the covariance of the two variables divided by the product of their standard deviations. The form of the definition involves a "product moment", that is, the mean (the first moment about the origin) of the product of the mean-adjusted random variables; hence the modifier ''product-moment'' in the name.


For a population

Pearson's correlation coefficient, when applied to a
population Population typically refers to the number of people in a single area, whether it be a city or town, region, country, continent, or the world. Governments typically quantify the size of the resident population within their jurisdiction using a ...
, is commonly represented by the Greek letter ''ρ'' (rho) and may be referred to as the ''population correlation coefficient'' or the ''population Pearson correlation coefficient''. Given a pair of random variables (X,Y), the formula for ''ρ''Real Statistics Using Excel: Correlation: Basic Concepts
retrieved 22 February 2015
is: \rho_= \frac where: * \operatorname is the
covariance In probability theory and statistics, covariance is a measure of the joint variability of two random variables. If the greater values of one variable mainly correspond with the greater values of the other variable, and the same holds for the les ...
* \sigma_X is the
standard deviation In statistics, the standard deviation is a measure of the amount of variation or dispersion of a set of values. A low standard deviation indicates that the values tend to be close to the mean (also called the expected value) of the set, while ...
of X * \sigma_Y is the
standard deviation In statistics, the standard deviation is a measure of the amount of variation or dispersion of a set of values. A low standard deviation indicates that the values tend to be close to the mean (also called the expected value) of the set, while ...
of Y The formula for \rho can be expressed in terms of mean and expectation. Since :\operatorname(X,Y) = \operatorname\mathbb X-\mu_X)(Y-\mu_Y) the formula for \rho can also be written as \rho_=\frac where: * \sigma_Y and \sigma_X are defined as above * \mu_X is the
mean There are several kinds of mean in mathematics, especially in statistics. Each mean serves to summarize a given group of data, often to better understand the overall value (magnitude and sign) of a given data set. For a data set, the ''arithme ...
of X * \mu_Y is the mean of Y * \operatorname\mathbb is the expectation. The formula for \rho can be expressed in terms of uncentered moments. Since :\mu_X=\operatorname\mathbb ,X\,/math> :\mu_Y=\operatorname\mathbb ,Y\,/math> :\sigma_X^2=\operatorname\mathbb ,\left(X-\operatorname\mathbb[Xright)^2\,.html" ;"title=".html" ;"title=",\left(X-\operatorname\mathbb[X">,\left(X-\operatorname\mathbb[Xright)^2\,">.html" ;"title=",\left(X-\operatorname\mathbb[X">,\left(X-\operatorname\mathbb[Xright)^2\,= \operatorname\mathbb[\,X^2\,]-\left(\operatorname\mathbb[\,X\,]\right)^2 :\sigma_Y^2=\operatorname\mathbb[\,\left(Y-\operatorname\mathbb[Y]\right)^2\,] = \operatorname\mathbb[\,Y^2\,]-\left(\,\operatorname\mathbb[\,Y\,]\right)^2 :\operatorname\mathbb ,\left(X-\mu_X\right)\left(Y-\mu_Y\right)\,= \operatorname\mathbb ,\left(X-\operatorname\mathbb[\,X\,right)\left(Y-\operatorname\mathbb[\,Y\,.html" ;"title=",X\,.html" ;"title=",\left(X-\operatorname\mathbb[\,X\,">,\left(X-\operatorname\mathbb[\,X\,right)\left(Y-\operatorname\mathbb[\,Y\,">,X\,.html" ;"title=",\left(X-\operatorname\mathbb[\,X\,">,\left(X-\operatorname\mathbb[\,X\,right)\left(Y-\operatorname\mathbb[\,Y\,right)\,] = \operatorname\mathbb[\,X\,Y\,]-\operatorname\mathbb[\,X\,]\operatorname\mathbb[\,Y\,] \,, the formula for \rho can also be written as \rho_=\frac. Peason's correlation coefficient does not exist when either \sigma_X or \sigma_Y are zero, infinite or undefined.


For a sample

Pearson's correlation coefficient, when applied to a
sample Sample or samples may refer to: Base meaning * Sample (statistics), a subset of a population – complete data set * Sample (signal), a digital discrete sample of a continuous analog signal * Sample (material), a specimen or small quantity of s ...
, is commonly represented by r_ and may be referred to as the ''sample correlation coefficient'' or the ''sample Pearson correlation coefficient''. We can obtain a formula for r_ by substituting estimates of the covariances and variances based on a sample into the formula above. Given paired data \left\ consisting of n pairs, r_ is defined as: r_ =\frac where: *n is sample size *x_i, y_i are the individual sample points indexed with ''i'' *\bar = \frac \sum_^n x_i (the sample mean); and analogously for \bar Rearranging gives us this formula for r_: :r_ = \frac . where n, x_i, y_i are defined as above. This formula suggests a convenient single-pass algorithm for calculating sample correlations, though depending on the numbers involved, it can sometimes be numerically unstable. Rearranging again gives us this formula for r_: :r_ = \frac . where n, x_i, y_i, \bar, \bar are defined as above. An equivalent expression gives the formula for r_ as the mean of the products of the
standard score In statistics, the standard score is the number of standard deviations by which the value of a raw score (i.e., an observed value or data point) is above or below the mean value of what is being observed or measured. Raw scores above the mean ...
s as follows: :r_ = \frac \sum ^n _ \left( \frac \right) \left( \frac \right) where: *n, x_i, y_i, \bar, \bar are defined as above, and s_x, s_y are defined below *\left( \frac \right) is the standard score (and analogously for the standard score of y) Alternative formulae for r_ are also available. For example, one can use the following formula for r_: :r_ =\frac where: *n, x_i, y_i, \bar, \bar are defined as above and: *s_x = \sqrt (the sample standard deviation); and analogously for s_y


Practical issues

Under heavy noise conditions, extracting the correlation coefficient between two sets of stochastic variables is nontrivial, in particular where Canonical Correlation Analysis reports degraded correlation values due to the heavy noise contributions. A generalization of the approach is given elsewhere. In case of missing data, Garren derived the
maximum likelihood In statistics, maximum likelihood estimation (MLE) is a method of estimation theory, estimating the Statistical parameter, parameters of an assumed probability distribution, given some observed data. This is achieved by Mathematical optimization, ...
estimator. Some distributions (e.g.,
stable distribution In probability theory, a distribution is said to be stable if a linear combination of two independent random variables with this distribution has the same distribution, up to location and scale parameters. A random variable is said to be stab ...
s other than a
normal distribution In statistics, a normal distribution or Gaussian distribution is a type of continuous probability distribution for a real-valued random variable. The general form of its probability density function is : f(x) = \frac e^ The parameter \mu ...
) do not have a defined variance.


Mathematical properties

The values of both the sample and population Pearson correlation coefficients are on or between −1 and 1. Correlations equal to +1 or −1 correspond to data points lying exactly on a line (in the case of the sample correlation), or to a bivariate distribution entirely supported on a line (in the case of the population correlation). The Pearson correlation coefficient is symmetric: corr(''X'',''Y'') = corr(''Y'',''X''). A key mathematical property of the Pearson correlation coefficient is that it is
invariant Invariant and invariance may refer to: Computer science * Invariant (computer science), an expression whose value doesn't change during program execution ** Loop invariant, a property of a program loop that is true before (and after) each iteratio ...
under separate changes in location and scale in the two variables. That is, we may transform ''X'' to and transform ''Y'' to , where ''a'', ''b'', ''c'', and ''d'' are constants with , without changing the correlation coefficient. (This holds for both the population and sample Pearson correlation coefficients.) Note that more general linear transformations do change the correlation: see ' for an application of this.


Interpretation

The correlation coefficient ranges from −1 to 1. An absolute value of exactly 1 implies that a linear equation describes the relationship between ''X'' and ''Y'' perfectly, with all data points lying on a
line Line most often refers to: * Line (geometry), object with zero thickness and curvature that stretches to infinity * Telephone line, a single-user circuit on a telephone communication system Line, lines, The Line, or LINE may also refer to: Arts ...
. The correlation sign is determined by the
regression slope In statistics, linear regression is a linear approach for modelling the relationship between a scalar response and one or more explanatory variables (also known as dependent and independent variables). The case of one explanatory variable is cal ...
: a value of +1 implies that all data points lie on a line for which ''Y'' increases as ''X'' increases, and vice versa for −1. A value of 0 implies that there is no linear dependency between the variables. More generally, note that is positive if and only if ''X''''i'' and ''Y''''i'' lie on the same side of their respective means. Thus the correlation coefficient is positive if ''X''''i'' and ''Y''''i'' tend to be simultaneously greater than, or simultaneously less than, their respective means. The correlation coefficient is negative (
anti-correlation In statistics, there is a negative relationship or inverse relationship between two variables if higher values of one variable tend to be associated with lower values of the other. A negative relationship between two variables usually implies that ...
) if ''X''''i'' and ''Y''''i'' tend to lie on opposite sides of their respective means. Moreover, the stronger either tendency is, the larger is the
absolute value In mathematics, the absolute value or modulus of a real number x, is the non-negative value without regard to its sign. Namely, , x, =x if is a positive number, and , x, =-x if x is negative (in which case negating x makes -x positive), an ...
of the correlation coefficient. Rodgers and Nicewander cataloged thirteen ways of interpreting correlation or simple functions of it: * Function of raw scores and means * Standardized covariance * Standardized slope of the regression line * Geometric mean of the two regression slopes * Square root of the ratio of two variances * Mean cross-product of standardized variables * Function of the angle between two standardized regression lines * Function of the angle between two variable vectors * Rescaled variance of the difference between standardized scores * Estimated from the balloon rule * Related to the bivariate ellipses of isoconcentration * Function of test statistics from designed experiments * Ratio of two means


Geometric interpretation

] For uncentered data, there is a relation between the correlation coefficient and the angle ''φ'' between the two regression lines, and , obtained by regressing ''y'' on ''x'' and ''x'' on ''y'' respectively. (Here, ''φ'' is measured counterclockwise within the first quadrant formed around the lines' intersection point if , or counterclockwise from the fourth to the second quadrant if .) One can show that if the standard deviations are equal, then , where sec and tan are
trigonometric functions In mathematics, the trigonometric functions (also called circular functions, angle functions or goniometric functions) are real functions which relate an angle of a right-angled triangle to ratios of two side lengths. They are widely used in all ...
. For centered data (i.e., data which have been shifted by the sample means of their respective variables so as to have an average of zero for each variable), the correlation coefficient can also be viewed as the cosine of the
angle In Euclidean geometry, an angle is the figure formed by two Ray (geometry), rays, called the ''Side (plane geometry), sides'' of the angle, sharing a common endpoint, called the ''vertex (geometry), vertex'' of the angle. Angles formed by two ...
''θ'' between the two observed vectors in ''N''-dimensional space (for ''N'' observations of each variable) Both the uncentered (non-Pearson-compliant) and centered correlation coefficients can be determined for a dataset. As an example, suppose five countries are found to have gross national products of 1, 2, 3, 5, and 8 billion dollars, respectively. Suppose these same five countries (in the same order) are found to have 11%, 12%, 13%, 15%, and 18% poverty. Then let x and y be ordered 5-element vectors containing the above data: and . By the usual procedure for finding the angle ''θ'' between two vectors (see
dot product In mathematics, the dot product or scalar productThe term ''scalar product'' means literally "product with a scalar as a result". It is also used sometimes for other symmetric bilinear forms, for example in a pseudo-Euclidean space. is an algebra ...
), the ''uncentered'' correlation coefficient is: : \cos \theta = \frac = \frac = 0.920814711. This uncentered correlation coefficient is identical with the
cosine similarity In data analysis, cosine similarity is a measure of similarity between two sequences of numbers. For defining it, the sequences are viewed as vectors in an inner product space, and the cosine similarity is defined as the cosine of the angle betwe ...
. Note that the above data were deliberately chosen to be perfectly correlated: . The Pearson correlation coefficient must therefore be exactly one. Centering the data (shifting x by and y by ) yields and , from which : \cos \theta = \frac = \frac = 1 = \rho_, as expected.


Interpretation of the size of a correlation

Several authors have offered guidelines for the interpretation of a correlation coefficient. However, all such criteria are in some ways arbitrary. The interpretation of a correlation coefficient depends on the context and purposes. A correlation of 0.8 may be very low if one is verifying a physical law using high-quality instruments, but may be regarded as very high in the social sciences, where there may be a greater contribution from complicating factors.


Inference

Statistical inference based on Pearson's correlation coefficient often focuses on one of the following two aims: * One aim is to test the
null hypothesis In scientific research, the null hypothesis (often denoted ''H''0) is the claim that no difference or relationship exists between two sets of data or variables being analyzed. The null hypothesis is that any experimentally observed difference is d ...
that the true correlation coefficient ''ρ'' is equal to 0, based on the value of the sample correlation coefficient ''r''. * The other aim is to derive a
confidence interval In frequentist statistics, a confidence interval (CI) is a range of estimates for an unknown parameter. A confidence interval is computed at a designated ''confidence level''; the 95% confidence level is most common, but other levels, such as 9 ...
that, on repeated sampling, has a given probability of containing ''ρ''. We discuss methods of achieving one or both of these aims below.


Using a permutation test

Permutation tests provide a direct approach to performing hypothesis tests and constructing confidence intervals. A permutation test for Pearson's correlation coefficient involves the following two steps: # Using the original paired data (''x''''i'', ''y''''i''), randomly redefine the pairs to create a new data set (''x''''i'', ''y''''i′''), where the ''i′'' are a permutation of the set . The permutation ''i′'' is selected randomly, with equal probabilities placed on all ''n''! possible permutations. This is equivalent to drawing the ''i′'' randomly without replacement from the set . In
bootstrapping In general, bootstrapping usually refers to a self-starting process that is supposed to continue or grow without external input. Etymology Tall boots may have a tab, loop or handle at the top known as a bootstrap, allowing one to use fingers ...
, a closely related approach, the ''i'' and the ''i′'' are equal and drawn with replacement from ; # Construct a correlation coefficient ''r'' from the randomized data. To perform the permutation test, repeat steps (1) and (2) a large number of times. The
p-value In null-hypothesis significance testing, the ''p''-value is the probability of obtaining test results at least as extreme as the result actually observed, under the assumption that the null hypothesis is correct. A very small ''p''-value means ...
for the permutation test is the proportion of the ''r'' values generated in step (2) that are larger than the Pearson correlation coefficient that was calculated from the original data. Here "larger" can mean either that the value is larger in magnitude, or larger in signed value, depending on whether a
two-sided In mathematics, specifically in topology of manifolds, a compact codimension-one submanifold F of a manifold M is said to be 2-sided in M when there is an embedding ::h\colon F\times 1,1to M with h(x,0)=x for each x\in F and ::h(F\times 1,1\ ...
or one-sided test is desired.


Using a bootstrap

The bootstrap can be used to construct confidence intervals for Pearson's correlation coefficient. In the "non-parametric" bootstrap, ''n'' pairs (''x''''i'', ''y''''i'') are resampled "with replacement" from the observed set of ''n'' pairs, and the correlation coefficient ''r'' is calculated based on the resampled data. This process is repeated a large number of times, and the empirical distribution of the resampled ''r'' values are used to approximate the
sampling distribution In statistics, a sampling distribution or finite-sample distribution is the probability distribution of a given random-sample-based statistic. If an arbitrarily large number of samples, each involving multiple observations (data points), were s ...
of the statistic. A 95%
confidence interval In frequentist statistics, a confidence interval (CI) is a range of estimates for an unknown parameter. A confidence interval is computed at a designated ''confidence level''; the 95% confidence level is most common, but other levels, such as 9 ...
for ''ρ'' can be defined as the interval spanning from the 2.5th to the 97.5th
percentile In statistics, a ''k''-th percentile (percentile score or centile) is a score ''below which'' a given percentage ''k'' of scores in its frequency distribution falls (exclusive definition) or a score ''at or below which'' a given percentage falls ...
of the resampled ''r'' values.


Standard error

If x and y are random variables, a
standard error The standard error (SE) of a statistic (usually an estimate of a parameter) is the standard deviation of its sampling distribution or an estimate of that standard deviation. If the statistic is the sample mean, it is called the standard error ...
is associated to the correlation in the null case is: :\sigma_r = \sqrt where r is the correlation (assumed ''r''≈0) and n the sample size.


Testing using Student's ''t''-distribution

For pairs from an uncorrelated
bivariate normal distribution In probability theory and statistics, the multivariate normal distribution, multivariate Gaussian distribution, or joint normal distribution is a generalization of the one-dimensional (univariate) normal distribution to higher dimensions. One ...
, the
sampling distribution In statistics, a sampling distribution or finite-sample distribution is the probability distribution of a given random-sample-based statistic. If an arbitrarily large number of samples, each involving multiple observations (data points), were s ...
of the
studentized In statistics, Studentization, named after William Sealy Gosset, who wrote under the pseudonym ''Student'', is the adjustment consisting of division of a first-degree statistic derived from a sample, by a sample-based estimate of a population standa ...
Pearson's correlation coefficient follows Student's ''t''-distribution with degrees of freedom ''n'' − 2. Specifically, if the underlying variables have a bivariate normal distribution, the variable :t = \frac = r\sqrt has a student's ''t''-distribution in the null case (zero correlation). This holds approximately in case of non-normal observed values if sample sizes are large enough. For determining the critical values for ''r'' the inverse function is needed: :r = \frac. Alternatively, large sample, asymptotic approaches can be used. Another early paper provides graphs and tables for general values of ''ρ'', for small sample sizes, and discusses computational approaches. In the case where the underlying variables are not normal, the sampling distribution of Pearson's correlation coefficient follows a Student's ''t''-distribution, but the degrees of freedom are reduced.


Using the exact distribution

For data that follow a
bivariate normal distribution In probability theory and statistics, the multivariate normal distribution, multivariate Gaussian distribution, or joint normal distribution is a generalization of the one-dimensional (univariate) normal distribution to higher dimensions. One ...
, the exact density function ''f''(''r'') for the sample correlation coefficient ''r'' of a normal bivariate is :f(r) = \frac _\mathrm_\bigl(\tfrac, \tfrac; \tfrac(2n - 1); \tfrac(\rho r + 1)\bigr) where \Gamma is the
gamma function In mathematics, the gamma function (represented by , the capital letter gamma from the Greek alphabet) is one commonly used extension of the factorial function to complex numbers. The gamma function is defined for all complex numbers except ...
and _\mathrm_(a,b;c;z) is the
Gaussian hypergeometric function Carl Friedrich Gauss (1777–1855) is the eponym of all of the topics listed below. There are over 100 topics all named after this German mathematician and scientist, all in the fields of mathematics, physics, and astronomy. The English eponymo ...
. In the special case when \rho = 0 (zero population correlation), the exact density function ''f''(''r'') can be written as: :f(r) = \frac, where \Beta is the
beta function In mathematics, the beta function, also called the Euler integral of the first kind, is a special function that is closely related to the gamma function and to binomial coefficients. It is defined by the integral : \Beta(z_1,z_2) = \int_0^1 t^(1 ...
, which is one way of writing the density of a Student's t-distribution, as above.


Using the exact confidence distribution

Confidence intervals and tests can be calculated from a
confidence distribution In statistical inference, the concept of a confidence distribution (CD) has often been loosely referred to as a distribution function on the parameter space that can represent confidence intervals of all levels for a parameter of interest. Histor ...
. An exact confidence density for ''ρ'' is \pi (\rho , r) = \frac (1 - r^2)^ \cdot (1 - \rho^2)^ \cdot (1 - r \rho )^ F\left(\tfrac,-\tfrac; \nu + \tfrac; \tfrac\right) where F is the Gaussian hypergeometric function and \nu = n-1 > 1.


Using the Fisher transformation

In practice,
confidence intervals In frequentist statistics, a confidence interval (CI) is a range of estimates for an unknown parameter. A confidence interval is computed at a designated ''confidence level''; the 95% confidence level is most common, but other levels, such as 9 ...
and
hypothesis test A statistical hypothesis test is a method of statistical inference used to decide whether the data at hand sufficiently support a particular hypothesis. Hypothesis testing allows us to make probabilistic statements about population parameters. ...
s relating to ''ρ'' are usually carried out using the
Fisher transformation In statistics, the Fisher transformation (or Fisher ''z''-transformation) of a Pearson correlation coefficient is its inverse hyperbolic tangent (artanh). When the sample correlation coefficient ''r'' is near 1 or -1, its distribution is high ...
, F'': :F(r) \equiv \tfrac \, \ln \left(\frac\right) = \operatorname(r) ''F''(''r'') approximately follows a
normal distribution In statistics, a normal distribution or Gaussian distribution is a type of continuous probability distribution for a real-valued random variable. The general form of its probability density function is : f(x) = \frac e^ The parameter \mu ...
with :\text = F(\rho) = \operatorname(\rho)and
standard error The standard error (SE) of a statistic (usually an estimate of a parameter) is the standard deviation of its sampling distribution or an estimate of that standard deviation. If the statistic is the sample mean, it is called the standard error ...
=\text = \frac, where ''n'' is the sample size. The approximation error is lowest for a large sample size n and small r and \rho_0 and increases otherwise. Using the approximation, a
z-score In statistics, the standard score is the number of standard deviations by which the value of a raw score (i.e., an observed value or data point) is above or below the mean value of what is being observed or measured. Raw scores above the mean ...
is :z = \frac = (r) - F(\rho_0)sqrt under the
null hypothesis In scientific research, the null hypothesis (often denoted ''H''0) is the claim that no difference or relationship exists between two sets of data or variables being analyzed. The null hypothesis is that any experimentally observed difference is d ...
that \rho = \rho_0, given the assumption that the sample pairs are
independent and identically distributed In probability theory and statistics, a collection of random variables is independent and identically distributed if each random variable has the same probability distribution as the others and all are mutually independent. This property is usua ...
and follow a
bivariate normal distribution In probability theory and statistics, the multivariate normal distribution, multivariate Gaussian distribution, or joint normal distribution is a generalization of the one-dimensional (univariate) normal distribution to higher dimensions. One ...
. Thus an approximate
p-value In null-hypothesis significance testing, the ''p''-value is the probability of obtaining test results at least as extreme as the result actually observed, under the assumption that the null hypothesis is correct. A very small ''p''-value means ...
can be obtained from a normal probability table. For example, if ''z'' = 2.2 is observed and a two-sided p-value is desired to test the null hypothesis that \rho = 0, the p-value is , where Φ is the standard normal
cumulative distribution function In probability theory and statistics, the cumulative distribution function (CDF) of a real-valued random variable X, or just distribution function of X, evaluated at x, is the probability that X will take a value less than or equal to x. Ev ...
. To obtain a confidence interval for ρ, we first compute a confidence interval for ''F''(''\rho''): :100(1 - \alpha)\%\text: \operatorname(\rho) \in operatorname(r) \pm z_\text/math> The inverse Fisher transformation brings the interval back to the correlation scale. :100(1 - \alpha)\%\text: \rho \in tanh(\operatorname(r) - z_\text), \tanh(\operatorname(r) + z_\text)/math> For example, suppose we observe ''r'' = 0.3 with a sample size of ''n''=50, and we wish to obtain a 95% confidence interval for ρ. The transformed value is arctanh(''r'') = 0.30952, so the confidence interval on the transformed scale is 0.30952 ± 1.96/, or (0.023624, 0.595415). Converting back to the correlation scale yields (0.024, 0.534).


In least squares regression analysis

The square of the sample correlation coefficient is typically denoted ''r''2 and is a special case of the
coefficient of determination In statistics, the coefficient of determination, denoted ''R''2 or ''r''2 and pronounced "R squared", is the proportion of the variation in the dependent variable that is predictable from the independent variable(s). It is a statistic used i ...
. In this case, it estimates the fraction of the variance in ''Y'' that is explained by ''X'' in a
simple linear regression In statistics, simple linear regression is a linear regression model with a single explanatory variable. That is, it concerns two-dimensional sample points with one independent variable and one dependent variable (conventionally, the ''x'' and ...
. So if we have the observed dataset Y_1, \dots , Y_n and the fitted dataset \hat Y_1, \dots , \hat Y_n then as a starting point the total variation in the ''Y''''i'' around their average value can be decomposed as follows :\sum_i (Y_i - \bar)^2 = \sum_i (Y_i-\hat_i)^2 + \sum_i (\hat_i-\bar)^2, where the \hat_i are the fitted values from the regression analysis. This can be rearranged to give :1 = \frac + \frac. The two summands above are the fraction of variance in ''Y'' that is explained by ''X'' (right) and that is unexplained by ''X'' (left). Next, we apply a property of least square regression models, that the sample covariance between \hat_i and Y_i-\hat_i is zero. Thus, the sample correlation coefficient between the observed and fitted response values in the regression can be written (calculation is under expectation, assumes Gaussian statistics) : \begin r(Y,\hat) &= \frac\\ pt&= \frac\\ pt&= \frac\\ pt&= \frac\\ pt&= \sqrt. \end Thus :r(Y,\hat)^2 = \frac where r(Y,\hat)^2 is the proportion of variance in ''Y'' explained by a linear function of ''X''. In the derivation above, the fact that :\sum_i (Y_i-\hat_i)(\hat_i-\bar) = 0 can be proved by noticing that the partial derivatives of the
residual sum of squares In statistics, the residual sum of squares (RSS), also known as the sum of squared estimate of errors (SSE), is the sum of the squares of residuals (deviations predicted from actual empirical values of data). It is a measure of the discrepan ...
() over ''β''0 and ''β''1 are equal to 0 in the least squares model, where :\text = \sum_i (Y_i - \hat_i)^2. In the end, the equation can be written as: :r(Y,\hat)^2 = \frac where *\text_\text = \sum_i (\hat_i-\bar)^2 *\text_\text = \sum_i (Y_i-\bar)^2 The symbol \text_\text is called the regression sum of squares, also called the
explained sum of squares In statistics, the explained sum of squares (ESS), alternatively known as the model sum of squares or sum of squares due to regression (SSR – not to be confused with the residual sum of squares (RSS) or sum of squares of errors), is a quantity ...
, and \text_\text is the
total sum of squares In statistical data analysis the total sum of squares (TSS or SST) is a quantity that appears as part of a standard way of presenting results of such analyses. For a set of observations, y_i, i\leq n, it is defined as the sum over all squared dif ...
(proportional to the
variance In probability theory and statistics, variance is the expectation of the squared deviation of a random variable from its population mean or sample mean. Variance is a measure of dispersion, meaning it is a measure of how far a set of numbers ...
of the data).


Sensitivity to the data distribution


Existence

The population Pearson correlation coefficient is defined in terms of moments, and therefore exists for any bivariate
probability distribution In probability theory and statistics, a probability distribution is the mathematical function that gives the probabilities of occurrence of different possible outcomes for an experiment. It is a mathematical description of a random phenomenon i ...
for which the
population Population typically refers to the number of people in a single area, whether it be a city or town, region, country, continent, or the world. Governments typically quantify the size of the resident population within their jurisdiction using a ...
covariance In probability theory and statistics, covariance is a measure of the joint variability of two random variables. If the greater values of one variable mainly correspond with the greater values of the other variable, and the same holds for the les ...
is defined and the
marginal Marginal may refer to: * ''Marginal'' (album), the third album of the Belgian rock band Dead Man Ray, released in 2001 * ''Marginal'' (manga) * '' El Marginal'', Argentine TV series * Marginal seat or marginal constituency or marginal, in polit ...
population variance In probability theory and statistics, variance is the expectation of the squared deviation of a random variable from its population mean or sample mean. Variance is a measure of dispersion, meaning it is a measure of how far a set of number ...
s are defined and are non-zero. Some probability distributions, such as the
Cauchy distribution The Cauchy distribution, named after Augustin Cauchy, is a continuous probability distribution. It is also known, especially among physicists, as the Lorentz distribution (after Hendrik Lorentz), Cauchy–Lorentz distribution, Lorentz(ian) fun ...
, have undefined variance and hence ρ is not defined if ''X'' or ''Y'' follows such a distribution. In some practical applications, such as those involving data suspected to follow a
heavy-tailed distribution In probability theory, heavy-tailed distributions are probability distributions whose tails are not exponentially bounded: that is, they have heavier tails than the exponential distribution. In many applications it is the right tail of the distrib ...
, this is an important consideration. However, the existence of the correlation coefficient is usually not a concern; for instance, if the range of the distribution is bounded, ρ is always defined.


Sample size

*If the sample size is moderate or large and the population is normal, then, in the case of the bivariate
normal distribution In statistics, a normal distribution or Gaussian distribution is a type of continuous probability distribution for a real-valued random variable. The general form of its probability density function is : f(x) = \frac e^ The parameter \mu ...
, the sample correlation coefficient is the
maximum likelihood estimate In statistics, maximum likelihood estimation (MLE) is a method of estimating the parameters of an assumed probability distribution, given some observed data. This is achieved by maximizing a likelihood function so that, under the assumed statisti ...
of the population correlation coefficient, and is
asymptotically In analytic geometry, an asymptote () of a curve is a line such that the distance between the curve and the line approaches zero as one or both of the ''x'' or ''y'' coordinates tends to infinity. In projective geometry and related contexts, ...
unbiased Bias is a disproportionate weight ''in favor of'' or ''against'' an idea or thing, usually in a way that is closed-minded, prejudicial, or unfair. Biases can be innate or learned. People may develop biases for or against an individual, a group, ...
and efficient, which roughly means that it is impossible to construct a more accurate estimate than the sample correlation coefficient. *If the sample size is large and the population is not normal, then the sample correlation coefficient remains approximately unbiased, but may not be efficient. *If the sample size is large, then the sample correlation coefficient is a
consistent estimator In statistics, a consistent estimator or asymptotically consistent estimator is an estimator—a rule for computing estimates of a parameter ''θ''0—having the property that as the number of data points used increases indefinitely, the result ...
of the population correlation coefficient as long as the sample means, variances, and covariance are consistent (which is guaranteed when the
law of large numbers In probability theory, the law of large numbers (LLN) is a theorem that describes the result of performing the same experiment a large number of times. According to the law, the average of the results obtained from a large number of trials shou ...
can be applied). *If the sample size is small, then the sample correlation coefficient ''r'' is not an unbiased estimate of ''ρ''. The adjusted correlation coefficient must be used instead: see elsewhere in this article for the definition. *Correlations can be different for imbalanced
dichotomous A dichotomy is a partition of a whole (or a set) into two parts (subsets). In other words, this couple of parts must be * jointly exhaustive: everything must belong to one part or the other, and * mutually exclusive: nothing can belong simultan ...
data when there is variance error in sample.


Robustness

Like many commonly used statistics, the sample statistic ''r'' is not
robust Robustness is the property of being strong and healthy in constitution. When it is transposed into a system, it refers to the ability of tolerating perturbations that might affect the system’s functional body. In the same line ''robustness'' ca ...
, so its value can be misleading if
outlier In statistics, an outlier is a data point that differs significantly from other observations. An outlier may be due to a variability in the measurement, an indication of novel data, or it may be the result of experimental error; the latter are ...
s are present. Specifically, the PMCC is neither distributionally robust, nor outlier resistant (see '). Inspection of the
scatterplot A scatter plot (also called a scatterplot, scatter graph, scatter chart, scattergram, or scatter diagram) is a type of plot or mathematical diagram using Cartesian coordinates to display values for typically two variables for a set of data. ...
between ''X'' and ''Y'' will typically reveal a situation where lack of robustness might be an issue, and in such cases it may be advisable to use a robust measure of association. Note however that while most robust estimators of association measure
statistical dependence Independence is a fundamental notion in probability theory, as in statistics and the theory of stochastic processes. Two events are independent, statistically independent, or stochastically independent if, informally speaking, the occurrence of o ...
in some way, they are generally not interpretable on the same scale as the Pearson correlation coefficient. Statistical inference for Pearson's correlation coefficient is sensitive to the data distribution. Exact tests, and asymptotic tests based on the
Fisher transformation In statistics, the Fisher transformation (or Fisher ''z''-transformation) of a Pearson correlation coefficient is its inverse hyperbolic tangent (artanh). When the sample correlation coefficient ''r'' is near 1 or -1, its distribution is high ...
can be applied if the data are approximately normally distributed, but may be misleading otherwise. In some situations, the bootstrap can be applied to construct confidence intervals, and
permutation tests A permutation test (also called re-randomization test) is an exact statistical hypothesis test making use of the proof by contradiction. A permutation test involves two or more samples. The null hypothesis is that all samples come from the same d ...
can be applied to carry out hypothesis tests. These
non-parametric Nonparametric statistics is the branch of statistics that is not based solely on parametrized families of probability distributions (common examples of parameters are the mean and variance). Nonparametric statistics is based on either being distri ...
approaches may give more meaningful results in some situations where bivariate normality does not hold. However the standard versions of these approaches rely on
exchangeability In statistics, an exchangeable sequence of random variables (also sometimes interchangeable) is a sequence ''X''1, ''X''2, ''X''3, ... (which may be finitely or infinitely long) whose joint probability distribution does not change whe ...
of the data, meaning that there is no ordering or grouping of the data pairs being analyzed that might affect the behavior of the correlation estimate. A stratified analysis is one way to either accommodate a lack of bivariate normality, or to isolate the correlation resulting from one factor while controlling for another. If ''W'' represents cluster membership or another factor that it is desirable to control, we can stratify the data based on the value of ''W'', then calculate a correlation coefficient within each stratum. The stratum-level estimates can then be combined to estimate the overall correlation while controlling for ''W''.


Variants

Variations of the correlation coefficient can be calculated for different purposes. Here are some examples.


Adjusted correlation coefficient

The sample correlation coefficient is not an unbiased estimate of . For data that follows a
bivariate normal distribution In probability theory and statistics, the multivariate normal distribution, multivariate Gaussian distribution, or joint normal distribution is a generalization of the one-dimensional (univariate) normal distribution to higher dimensions. One ...
, the expectation for the sample correlation coefficient of a normal bivariate is :\operatorname\mathbb\left \right= \rho - \frac + \cdots, \quad therefore is a biased estimator of \rho. The unique minimum variance unbiased estimator is given by where: *r, n are defined as above, *\mathbf(a,b;c;z) is the
Gaussian hypergeometric function Carl Friedrich Gauss (1777–1855) is the eponym of all of the topics listed below. There are over 100 topics all named after this German mathematician and scientist, all in the fields of mathematics, physics, and astronomy. The English eponymo ...
. An approximately unbiased estimator can be obtained by truncating and solving this truncated equation: An approximate solution to equation () is: where in (): *r, n are defined as above, * is a suboptimal estimator, * can also be obtained by maximizing log(''f''(''r'')), * has minimum variance for large values of , * has a bias of order . Another proposed adjusted correlation coefficient is: :r_\text=\sqrt. Note that for large values of .


Weighted correlation coefficient

Suppose observations to be correlated have differing degrees of importance that can be expressed with a weight vector ''w''. To calculate the correlation between vectors ''x'' and ''y'' with the weight vector ''w'' (all of length ''n''), * Weighted mean: \operatorname(x; w) = \frac. * Weighted covariance \operatorname(x,y;w) = \frac. * Weighted correlation \operatorname(x,y;w) = \frac.


Reflective correlation coefficient

The reflective correlation is a variant of Pearson's correlation in which the data are not centered around their mean values. The population reflective correlation is :\operatorname_r(X,Y) = \frac. The reflective correlation is symmetric, but it is not invariant under translation: :\operatorname_r(X, Y) = \operatorname_r(Y, X) = \operatorname_r(X, bY) \neq \operatorname_r(X, a + b Y), \quad a \neq 0, b > 0. The sample reflective correlation is equivalent to
cosine similarity In data analysis, cosine similarity is a measure of similarity between two sequences of numbers. For defining it, the sequences are viewed as vectors in an inner product space, and the cosine similarity is defined as the cosine of the angle betwe ...
: :rr_ = \frac. The weighted version of the sample reflective correlation is :rr_ = \frac.


Scaled correlation coefficient

Scaled correlation is a variant of Pearson's correlation in which the range of the data is restricted intentionally and in a controlled manner to reveal correlations between fast components in time series. Scaled correlation is defined as average correlation across short segments of data. Let K be the number of segments that can fit into the total length of the signal T for a given scale s: :K = \operatorname\left(\frac\right). The scaled correlation across the entire signals \bar_s is then computed as :\bar_s = \frac \sum\limits_^K r_k, where r_k is Pearson's coefficient of correlation for segment k. By choosing the parameter s, the range of values is reduced and the correlations on long time scale are filtered out, only the correlations on short time scales being revealed. Thus, the contributions of slow components are removed and those of fast components are retained.


Pearson's distance

A distance metric for two variables ''X'' and ''Y'' known as ''Pearson's distance'' can be defined from their correlation coefficient as :d_=1-\rho_. Considering that the Pearson correlation coefficient falls between 1, +1 the Pearson distance lies in
, 2 The comma is a punctuation mark that appears in several variants in different languages. It has the same shape as an apostrophe or single closing quotation mark () in many typefaces, but it differs from them in being placed on the baseline ...
The Pearson distance has been used in
cluster analysis Cluster analysis or clustering is the task of grouping a set of objects in such a way that objects in the same group (called a cluster) are more similar (in some sense) to each other than to those in other groups (clusters). It is a main task of ...
and data detection for communications and storage with unknown gain and offset. The Pearson "distance" defined this way assigns distance greater than 1 to negative correlations. In reality, both strong positive correlation and negative correlations are meaningful, so care must be taken when Pearson "distance" is used for nearest neighbor algorithm as such algorithm will only include neighbors with positive correlation and exclude neighbors with negative correlation. Alternatively, an absolute valued distance: d_=1-, \rho_, can be applied, which will take both positive and negative correlations into consideration. The information on positive and negative association can be extracted separately, later.


Circular correlation coefficient

For variables X = and Y = that are defined on the unit circle , it is possible to define a circular analog of Pearson's coefficient. This is done by transforming data points in X and Y with a
sine In mathematics, sine and cosine are trigonometric functions of an angle. The sine and cosine of an acute angle are defined in the context of a right triangle: for the specified angle, its sine is the ratio of the length of the side that is oppo ...
function such that the correlation coefficient is given as: :r_\text = \frac where \bar and \bar are the circular means of ''X'' and ''Y''. This measure can be useful in fields like meteorology where the angular direction of data is important.


Partial correlation

If a population or data-set is characterized by more than two variables, a
partial correlation In probability theory and statistics, partial correlation measures the degree of association between two random variables, with the effect of a set of controlling random variables removed. When determining the numerical relationship between two va ...
coefficient measures the strength of dependence between a pair of variables that is not accounted for by the way in which they both change in response to variations in a selected subset of the other variables.


Decorrelation of ''n'' random variables

It is always possible to remove the correlations between all pairs of an arbitrary number of random variables by using a data transformation, even if the relationship between the variables is nonlinear. A presentation of this result for population distributions is given by Cox & Hinkley. A corresponding result exists for reducing the sample correlations to zero. Suppose a vector of ''n'' random variables is observed ''m'' times. Let ''X'' be a matrix where X_ is the ''j''th variable of observation ''i''. Let Z_ be an ''m'' by ''m'' square matrix with every element 1. Then ''D'' is the data transformed so every random variable has zero mean, and ''T'' is the data transformed so all variables have zero mean and zero correlation with all other variables – the sample
correlation matrix In statistics, correlation or dependence is any statistical relationship, whether causal or not, between two random variables or bivariate data. Although in the broadest sense, "correlation" may indicate any type of association, in statistic ...
of ''T'' will be the identity matrix. This has to be further divided by the standard deviation to get unit variance. The transformed variables will be uncorrelated, even though they may not be
independent Independent or Independents may refer to: Arts, entertainment, and media Artist groups * Independents (artist group), a group of modernist painters based in the New Hope, Pennsylvania, area of the United States during the early 1930s * Independ ...
. :D = X -\frac Z_ X :T = D (D^ D)^, where an exponent of represents the
matrix square root In mathematics, the square root of a matrix extends the notion of square root from numbers to matrices. A matrix is said to be a square root of if the matrix product is equal to . Some authors use the name ''square root'' or the notation on ...
of the inverse of a matrix. The correlation matrix of ''T'' will be the identity matrix. If a new data observation ''x'' is a row vector of ''n'' elements, then the same transform can be applied to ''x'' to get the transformed vectors ''d'' and ''t'': :d = x - \frac Z_ X, :t = d (D^ D)^. This decorrelation is related to
principal components analysis Principal component analysis (PCA) is a popular technique for analyzing large datasets containing a high number of dimensions/features per observation, increasing the interpretability of data while preserving the maximum amount of information, and ...
for multivariate data.


Software implementations

* R's statistics base-package implements the correlation coefficient with cor(x, y), or (with the P value also) wit
cor.test(x, y)
* The
SciPy SciPy (pronounced "sigh pie") is a free and open-source Python library used for scientific computing and technical computing. SciPy contains modules for optimization, linear algebra, integration, interpolation, special functions, FFT, signal ...
Python Python may refer to: Snakes * Pythonidae, a family of nonvenomous snakes found in Africa, Asia, and Australia ** ''Python'' (genus), a genus of Pythonidae found in Africa and Asia * Python (mythology), a mythical serpent Computing * Python (pro ...
library vi
pearsonr(x, y)
* The
Pandas Pediatric autoimmune neuropsychiatric disorders associated with streptococcal infections (PANDAS) is a controversial hypothetical diagnosis for a subset of children with rapid onset of obsessive-compulsive disorder (OCD) or tic disorders. Sy ...
Python library implements Pearson correlation coefficient calculation as the default option for the metho
pandas.DataFrame.corr
*
Wolfram Mathematica Wolfram Mathematica is a software system with built-in libraries for several areas of technical computing that allow machine learning, statistics, symbolic computation, data manipulation, network analysis, time series analysis, NLP, optimizat ...
via th
Correlation
function, or (with the P value) wit

* The Boost
C++ C++ (pronounced "C plus plus") is a high-level general-purpose programming language created by Danish computer scientist Bjarne Stroustrup as an extension of the C programming language, or "C with Classes". The language has expanded significan ...
library via th
correlation_coefficient
function. *
Excel ExCeL London (an abbreviation for Exhibition Centre London) is an exhibition centre, international convention centre and former hospital in the Custom House area of Newham, East London. It is situated on a site on the northern quay of the ...
has an in-buil
correl(array1, array2)
function for calculationg the pearson's correlation coefficient.


See also

* Anscombe's quartet *
Association (statistics) In statistics, correlation or dependence is any statistical relationship, whether causal or not, between two random variables or bivariate data. Although in the broadest sense, "correlation" may indicate any type of association, in statistics ...
*
Coefficient of colligation In statistics, Yule's ''Y'', also known as the coefficient of colligation, is a measure of association between two binary variables. The measure was developed by George Udny Yule in 1912,Michel G. Soete. A new theory on the measurement of associa ...
**
Yule's Q In statistics, Goodman and Kruskal's gamma is a measure of rank correlation, i.e., the similarity of the orderings of the data when ranked by each of the quantities. It measures the strength of association of the cross tabulated data when both va ...
**
Yule's Y In statistics, Yule's ''Y'', also known as the coefficient of colligation, is a measure of association between two binary variables. The measure was developed by George Udny Yule in 1912,Michel G. Soete. A new theory on the measurement of associat ...
*
Concordance correlation coefficient In statistics, the concordance correlation coefficient measures the agreement between two variables, e.g., to evaluate reproducibility or for inter-rater reliability. Definition The form of the concordance correlation coefficient \rho_c as :\rho_c ...
*
Correlation and dependence In statistics, correlation or dependence is any statistical relationship, whether causal or not, between two random variables or bivariate data. Although in the broadest sense, "correlation" may indicate any type of association, in statistics ...
*
Correlation ratio In statistics, the correlation ratio is a measure of the curvilinear relationship between the statistical dispersion within individual categories and the dispersion across the whole population or sample. The measure is defined as the ''ratio'' of t ...
* Disattenuation *
Distance correlation In statistics and in probability theory, distance correlation or distance covariance is a measure of dependence between two paired random vectors of arbitrary, not necessarily equal, dimension. The population distance correlation coefficient is ze ...
*
Maximal information coefficient In statistics, the maximal information coefficient (MIC) is a measure of the strength of the linear or non-linear association between two variables ''X'' and ''Y''. The MIC belongs to the maximal information-based nonparametric exploration ( ...
*
Multiple correlation In statistics, the coefficient of multiple correlation is a measure of how well a given variable can be predicted using a linear function of a set of other variables. It is the correlation between the variable's values and the best predictions th ...
*
Normally distributed and uncorrelated does not imply independent In probability theory, although simple examples illustrate that linear uncorrelatedness of two random variables does not in general imply their independence, it is sometimes mistakenly thought that it does imply that when the two random variables a ...
*
Odds ratio An odds ratio (OR) is a statistic that quantifies the strength of the association between two events, A and B. The odds ratio is defined as the ratio of the odds of A in the presence of B and the odds of A in the absence of B, or equivalently (due ...
*
Partial correlation In probability theory and statistics, partial correlation measures the degree of association between two random variables, with the effect of a set of controlling random variables removed. When determining the numerical relationship between two va ...
*
Polychoric correlation In statistics, polychoric correlation{{Cite web, url=https://support.sas.com/documentation/cdl/en/procstat/65543/HTML/default/viewer.htm#procstat_corr_details14.htm, title=Base SAS(R) 9.3 Procedures Guide: Statistical Procedures, Second Edition, we ...
* Quadrant count ratio *
RV coefficient In statistics, the RV coefficient is a multivariate generalization of the ''squared'' Pearson correlation coefficient (because the RV coefficient takes values between 0 and 1). It measures the closeness of two set of points that may each be repres ...
*
Spearman's rank correlation coefficient In statistics, Spearman's rank correlation coefficient or Spearman's ''ρ'', named after Charles Spearman and often denoted by the Greek letter \rho (rho) or as r_s, is a nonparametric measure of rank correlation ( statistical dependence between ...


Footnotes


References


External links

* – A free web interface and R package for the statistical comparison of two dependent or independent correlations with overlapping or non-overlapping variables. * – an interactive Flash simulation on the correlation of two normally distributed variables. * – * – large table. * – A game where players guess how correlated two variables in a scatter plot are, in order to gain a better understanding of the concept of correlation. {{DEFAULTSORT:Pearson product-moment correlation coefficient Correlation indicators Parametric statistics Statistical ratios