In statistics, almost sure hypothesis testing or a.s. hypothesis testing utilizes
almost sure convergence
In probability theory, there exist several different notions of convergence of random variables. The convergence of sequences of random variables to some limit random variable is an important concept in probability theory, and its applications to ...
in order to determine the validity of a statistical hypothesis with probability one. This is to say that whenever the
null hypothesis
In scientific research, the null hypothesis (often denoted ''H''0) is the claim that no difference or relationship exists between two sets of data or variables being analyzed. The null hypothesis is that any experimentally observed difference is d ...
is true, then an a.s. hypothesis test will fail to reject the null hypothesis w.p. 1 for all sufficiently large samples. Similarly, whenever the
alternative hypothesis
In statistical hypothesis testing, the alternative hypothesis is one of the proposed proposition in the hypothesis test. In general the goal of hypothesis test is to demonstrate that in the given condition, there is sufficient evidence supporting ...
is true, then an a.s. hypothesis test will reject the null hypothesis with probability one, for all sufficiently large samples. Along similar lines, an a.s.
confidence interval
In frequentist statistics, a confidence interval (CI) is a range of estimates for an unknown parameter. A confidence interval is computed at a designated ''confidence level''; the 95% confidence level is most common, but other levels, such as 9 ...
eventually contains the parameter of interest with probability 1. Dembo and Peres (1994) proved the existence of almost sure hypothesis tests.
Description
For simplicity, assume we have a sequence of independent and identically distributed normal random variables,
, with mean
, and unit variance. Suppose that nature or simulation has chosen the true mean to be
, then the probability distribution function of the mean,
, is given by
:
where_an_Iverson_bracket.html" ;"title="mu_0,+\infty.html" ;"title="t\in[\mu_0,+\infty">t\in[\mu_0,+\infty
where an Iverson bracket">mu_0,+\infty.html" ;"title="t\in[\mu_0,+\infty">t\in[\mu_0,+\infty
where an Iverson bracket has been used. A naïve approach to estimating this distribution function would be to replace true mean on the right hand side with an estimate such as the sample mean,
, but
:
which means the approximation to the true distribution function will be off by 0.5 at the true mean. However,