Cauchy distribution

The Cauchy distribution, named after Augustin Cauchy, is a continuous probability distribution. It is also known, especially among physicists, as the Lorentz distribution (after Hendrik Lorentz), Cauchy–Lorentz distribution, Lorentz(ian) function, or Breit–Wigner distribution. The Cauchy distribution $f(x; x_0,\gamma)$ is the distribution of the -intercept of a ray issuing from $(x_0,\gamma)$ with a uniformly distributed angle. It is also the distribution of the ratio of two independent normally distributed random variables with mean zero.

The Cauchy distribution is often used in statistics as the canonical example of a "pathological" distribution since both its expected value and its variance are undefined (but see below). The Cauchy distribution does not have finite moments of order greater than or equal to one; only fractional absolute moments exist., Chapter 16. The Cauchy distribution has no moment generating function.

In mathematics, it is closely related to the Poisson kernel, which is the fundamental solution for the Laplace equation in the upper half-plane.

It is one of the few distributions that is stable and has a probability density function that can be expressed analytically, the others being the normal distribution and the Lévy distribution.

History

A function with the form of the density function of the Cauchy distribution was studied geometrically by Fermat in 1659, and later was known as the witch of Agnesi, after Agnesi included it as an example in her 1748 calculus textbook. Despite its name, the first explicit analysis of the properties of the Cauchy distribution was published by the French mathematician Poisson in 1824, with Cauchy only becoming associated with it during an academic controversy in 1853. Poisson noted that if the mean of observations following such a distribution were taken, the mean error did not converge to any finite number. As such, Laplace's use of the central limit theorem with such distribution was inappropriate, as it assumed a finite mean and variance. Despite this, Poisson did not regard the issue as important, in contrast to Bienaymé, who was to engage Cauchy in a long dispute over the matter.

Characterization

Probability density function

The Cauchy distribution has the probability density function (PDF)
:f(x; x_0,\gamma) = \frac = \left \right

where $x_0$ is the location parameter, specifying the location of the peak of the distribution, and $\gamma$ is the scale parameter which specifies the half-width at half-maximum (HWHM), alternatively $2\gamma$ is full width at half maximum (FWHM). $\gamma$ is also equal to half the interquartile range and is sometimes called the probable error. Augustin-Louis Cauchy exploited such a density function in 1827 with an infinitesimal scale parameter, defining what would now be called a Dirac delta function.

The maximum value or amplitude of the Cauchy PDF is $\frac$, located at $x=x_0$.

It is sometimes convenient to express the PDF in terms of the complex parameter $\psi= x_0 + i\gamma$

:$f(x;\psi)=\frac\,\textrm\left(\frac\right)=\frac\,\textrm\left(\frac\right)$

The special case when $x_0 = 0$ and $\gamma = 1$ is called the standard Cauchy distribution with the probability density function
:$f(x; 0,1) = \frac. \!$

In physics, a three-parameter Lorentzian function is often used:
:f(x; x_0,\gamma,I) = \frac = I \left \right
where $I$ is the height of the peak. The three-parameter Lorentzian function indicated is not, in general, a probability density function, since it does not integrate to 1, except in the special case where $I = \frac.\!$

Cumulative distribution function

The cumulative distribution function of the Cauchy distribution is:
:$F(x; x_0,\gamma)=\frac \arctan\left(\frac\right)+\frac$

and the quantile function (inverse cdf) of the Cauchy distribution is
:Q(p; x_0,\gamma) = x_0 + \gamma\,\tan\left pi\left(p-\tfrac\right)\right
It follows that the first and third quartiles are $(x_0 - \gamma, x_0 + \gamma)$, and hence the interquartile range is $2\gamma$.

For the standard distribution, the cumulative distribution function simplifies to arctangent function $\arctan(x)$:
:$F(x; 0,1)=\frac \arctan\left(x\right)+\frac$

Entropy

The entropy of the Cauchy distribution is given by:

:
\begin
H(\gamma) & =-\int_^\infty f(x;x_0,\gamma) \log(f(x;x_0,\gamma)) \, dx \\ pt& =\log(4\pi\gamma)
\end

The derivative of the quantile function, the quantile density function, for the Cauchy distribution is:

:Q'(p; \gamma) = \gamma\,\pi\,^2\left pi\left(p-\tfrac 1 2 \right)\right\!

The differential entropy of a distribution can be defined in terms of its quantile density, specifically:

:$H(\gamma) = \int_0^1 \log\,(Q'(p; \gamma))\,\mathrm dp = \log(4\pi\gamma)$

The Cauchy distribution is the maximum entropy probability distribution for a random variate $X$ for which

:\operatorname log(1+(X-x_0)^2/\gamma^2)\log 4

or, alternatively, for a random variate $X$ for which

:\operatorname log(1+(X-x_0)^2)2\log(1+\gamma).

In its standard form, it is the maximum entropy probability distribution for a random variate $X$ for which

:\operatorname\!\left ln(1+X^2) \right\ln 4.

Kullback-Leibler divergence

The Kullback-Leibler divergence between two Cauchy distributions has the following symmetric closed-form formula:

:$\mathrm\left(p_: p_\right)=\log \frac.$

Any f-divergence between two Cauchy distributions is symmetric and can be expressed as a function of the chi-squared divergence.
Closed-form expression for the total variation, Jensen–Shannon divergence, Hellinger distance, etc are available.

Properties

The Cauchy distribution is an example of a distribution which has no mean, variance or higher moments defined. Its mode and median are well defined and are both equal to $x_0$.

When $U$ and $V$ are two independent normally distributed random variables with expected value 0 and variance 1, then the ratio $U/V$ has the standard Cauchy distribution.

If $\Sigma$ is a $p\times p$ positive-semidefinite covariance matrix with strictly positive diagonal entries, then for independent and identically distributed $X,Y\sim N(0,\Sigma)$ and any random $p$-vector $w$ independent of $X$ and $Y$ such that $w_1+\cdots+w_p=1$ and $w_i\geq 0, i=1,\ldots,p,$ (defining a categorical distribution) it holds that

:$\sum_^p w_j\frac\sim\mathrm(0,1).$

If $X_1, \ldots, X_n$ are independent and identically distributed random variables, each with a standard Cauchy distribution, then the sample mean $(X_1 + \cdots + X_n)/n$ has the same standard Cauchy distribution. To see that this is true, compute the characteristic function of the sample mean:
:\varphi_(t) = \mathrm\left ^\right/math>

where $\overline$ is the sample mean. This example serves to show that the condition of finite variance in the central limit theorem cannot be dropped. It is also an example of a more generalized version of the central limit theorem that is characteristic of all stable distributions, of which the Cauchy distribution is a special case.

The Cauchy distribution is an infinitely divisible probability distribution. It is also a strictly stable distribution.

The standard Cauchy distribution coincides with the Student's ''t''-distribution with one degree of freedom.

Like all stable distributions, the location-scale family to which the Cauchy distribution belongs is closed under linear transformations with real coefficients. In addition, the Cauchy distribution is closed under linear fractional transformations with real coefficients. In this connection, see also McCullagh's parametrization of the Cauchy distributions.

Characteristic function

Let $X$ denote a Cauchy distributed random variable. The characteristic function of the Cauchy distribution is given by

:\varphi_X(t) = \operatorname\left ^ \right =\int_^\infty f(x;x_0,\gamma)e^\,dx = e^.

which is just the Fourier transform of the probability density. The original probability density may be expressed in terms of the characteristic function, essentially by using the inverse Fourier transform:

:$f(x; x_0,\gamma) = \frac\int_^\infty \varphi_X(t;x_0,\gamma)e^ \, dt \!$

The ''n''th moment of a distribution is the ''n''th derivative of the characteristic function evaluated at $t=0$. Observe that the characteristic function is not differentiable at the origin: this corresponds to the fact that the Cauchy distribution does not have well-defined moments higher than the zeroth moment.

Comparison with the normal distribution

Compared to the normal distribution, the Cauchy density function has a higher peak and lower tails.
An example is shown in the two figures added here

The figure to the left shows the ''Cauchy probability density function'' fitted to an observed histogram. The peak of the function is higher than the peak of the histogram while the tails are lower than those of the histogram.

The figure to the right shows the ''normal probability density function'' fitted to ''the same'' observed histogram. The peak of the function is lower than the peak of the histogram.

This illustates the above statement.

Explanation of undefined moments

Mean

If a probability distribution has a density function $f(x)$, then the mean, if it exists, is given by

We may evaluate this two-sided improper integral by computing the sum of two one-sided improper integrals. That is,

for an arbitrary real number $a$.

For the integral to exist (even as an infinite value), at least one of the terms in this sum should be finite, or both should be infinite and have the same sign. But in the case of the Cauchy distribution, both the terms in this sum () are infinite and have opposite sign. Hence () is undefined, and thus so is the mean.

Note that the Cauchy principal value of the mean of the Cauchy distribution is
$\lim_\int_^a x f(x)\,dx$
which is zero. On the other hand, the related integral
$\lim_\int_^a x f(x)\,dx$
is ''not'' zero, as can be seen by computing the integral. This again shows that the mean () cannot exist.

Various results in probability theory about expected values, such as the strong law of large numbers, fail to hold for the Cauchy distribution.

Smaller moments

The absolute moments for $p\in(-1,1)$ are defined.
For $X\sim\mathrm(0,\gamma)$ we have
:\operatorname raw_moments_do_exist_and_have_a_value_of_infinity,_for_example,_the_raw_second_moment:

:_
\begin
\operatorname ^2&_\propto_\int_^\infty_\frac\,dx_=_\int_^\infty_1_-_\frac\,dx_\\ pt&_=_\int_^\infty_dx_-_\int_^\infty_\frac\,dx_=_\int_^\infty_dx-\pi_=_\infty.
\end

By_re-arranging_the_formula,_one_can_see_that_the_second_moment_is_essentially_the_infinite_integral_of_a_constant_(here_1).__Higher_even-powered_raw_moments_will_also_evaluate_to_infinity.__Odd-powered_raw_moments,_however,_are_undefined,_which_is_distinctly_different_from_existing_with_the_value_of_infinity._The_odd-powered_raw_moments_are_undefined_because_their_values_are_essentially_equivalent_to_$\infty_-_\infty$_since_the_two_halves_of_the_integral_both_diverge_and_have_opposite_signs.__The_first_raw_moment_is_the_mean,_which,_being_odd,_does_not_exist._(See_also_the_discussion_above_about_this.)_This_in_turn_means_that_all_of_the_ central_moments_and_ standardized_moments_are_undefined_since_they_are_all_based_on_the_mean.__The_variance—which_is_the_second_central_moment—is_likewise_non-existent_(despite_the_fact_that_the_raw_second_moment_exists_with_the_value_infinity).

The_results_for_higher_moments_follow_from_ Hölder's_inequality,_which_implies_that_higher_moments_(or_halves_of_moments)_diverge_if_lower_ones_do.

_Moments_of_truncated_distributions

Consider_the_ truncated_distribution_defined_by_restricting_the_standard_Cauchy_distribution_to_the_interval_._Such_a_truncated_distribution_has_all_moments_(and_the_central_limit_theorem_applies_for_ i.i.d._observations_from_it);_yet_for_almost_all_practical_purposes_it_behaves_like_a_Cauchy_distribution.

_Estimation_of_parameters_

Because_the_parameters_of_the_Cauchy_distribution_do_not_correspond_to_a_mean_and_variance,_attempting_to_estimate_the_parameters_of_the_Cauchy_distribution_by_using_a_sample_mean_and_a_sample_variance_will_not_succeed._For_example,_if_an_i.i.d._sample_of_size_''n''_is_taken_from_a_Cauchy_distribution,_one_may_calculate_the_sample_mean_as:

:$\bar=\frac_1_n_\sum_^n_x_i$

Although_the_sample_values_$x_i$_will_be_concentrated_about_the_central_value_$x_0$,_the_sample_mean_will_become_increasingly_variable_as_more_observations_are_taken,_because_of_the_increased_probability_of_encountering_sample_points_with_a_large_absolute_value._In_fact,_the_distribution_of_the_sample_mean_will_be_equal_to_the_distribution_of_the_observations_themselves;_i.e.,_the_sample_mean_of_a_large_sample_is_no_better_(or_worse)_an_estimator_of_$x_0$_than_any_single_observation_from_the_sample._Similarly,_calculating_the_sample_variance_will_result_in_values_that_grow_larger_as_more_observations_are_taken.

Therefore,_more_robust_means_of_estimating_the_central_value_$x_0$_and_the_scaling_parameter_$\gamma$_are_needed._One_simple_method_is_to_take_the_median_value_of_the_sample_as_an_estimator_of_$x_0$_and_half_the_sample_interquartile_range_

In__descriptive_statistics,_the_interquartile_range_(IQR)_is_a_measure_of__statistical_dispersion,_which_is_the_spread_of_the_data._The_IQR_may_also_be_called_the_midspread,_middle_50%,_fourth_spread,_or_H‑spread._It_is_defined_as_the_differen_...