A ''t''-test is any
statistical hypothesis test
A statistical hypothesis test is a method of statistical inference used to decide whether the data at hand sufficiently support a particular hypothesis.
Hypothesis testing allows us to make probabilistic statements about population parameters.
...
in which the
test statistic
A test statistic is a statistic (a quantity derived from the sample) used in statistical hypothesis testing.Berger, R. L.; Casella, G. (2001). ''Statistical Inference'', Duxbury Press, Second Edition (p.374) A hypothesis test is typically specif ...
follows a
Student's ''t''-distribution under the
null hypothesis
In scientific research, the null hypothesis (often denoted ''H''0) is the claim that no difference or relationship exists between two sets of data or variables being analyzed. The null hypothesis is that any experimentally observed difference is d ...
. It is most commonly applied when the test statistic would follow a
normal distribution
In statistics, a normal distribution or Gaussian distribution is a type of continuous probability distribution for a real-valued random variable. The general form of its probability density function is
:
f(x) = \frac e^
The parameter \mu ...
if the value of a
scaling term in the test statistic were known (typically, the scaling term is unknown and therefore a
nuisance parameter
Nuisance (from archaic ''nocence'', through Fr. ''noisance'', ''nuisance'', from Lat. ''nocere'', "to hurt") is a common law tort. It means that which causes offence, annoyance, trouble or injury. A nuisance can be either public (also "common") ...
). When the scaling term is estimated based on the
data
In the pursuit of knowledge, data (; ) is a collection of discrete values that convey information, describing quantity, quality, fact, statistics, other basic units of meaning, or simply sequences of symbols that may be further interpreted ...
, the test statistic—under certain conditions—follows a Student's ''t'' distribution. The ''t''-test's most common application is to test whether the means of two populations are different.
History
![William Sealy Gosset](https://upload.wikimedia.org/wikipedia/commons/4/42/William_Sealy_Gosset.jpg)
The term "''t''-statistic" is abbreviated from "hypothesis test statistic". In statistics, the t-distribution was first derived as a
posterior distribution
The posterior probability is a type of conditional probability that results from updating the prior probability with information summarized by the likelihood via an application of Bayes' rule. From an epistemological perspective, the posterior p ...
in 1876 by
Helmert
Friedrich Robert Helmert (31 July 1843 – 15 June 1917) was a German geodesist and statistician with important contributions to the theory of errors.
Career
Helmert was born in Freiberg, Kingdom of Saxony. After schooling in Freiberg and D ...
and
Lüroth. The t-distribution also appeared in a more general form as Pearson Type
IV distribution in
Karl Pearson
Karl Pearson (; born Carl Pearson; 27 March 1857 – 27 April 1936) was an English mathematician and biostatistician. He has been credited with establishing the discipline of mathematical statistics. He founded the world's first university st ...
's 1895 paper. However, the T-Distribution, also known as
Student's t-distribution
In probability and statistics, Student's ''t''-distribution (or simply the ''t''-distribution) is any member of a family of continuous probability distributions that arise when estimating the mean of a normally distributed population in sit ...
, gets its name from
William Sealy Gosset
William Sealy Gosset (13 June 1876 – 16 October 1937) was an English statistician, chemist and brewer who served as Head Brewer of Guinness and Head Experimental Brewer of Guinness and was a pioneer of modern statistics. He pioneered small sa ...
who first published it in English in 1908 in the scientific journal
Biometrika
''Biometrika'' is a peer-reviewed scientific journal published by Oxford University Press for thBiometrika Trust The editor-in-chief is Paul Fearnhead (Lancaster University). The principal focus of this journal is theoretical statistics. It was es ...
using the pseudonym "Student" because his employer preferred staff to use pen names when publishing scientific papers. Gosset worked at the
Guinness Brewery in
Dublin
Dublin (; , or ) is the capital and largest city of Republic of Ireland, Ireland. On a bay at the mouth of the River Liffey, it is in the Provinces of Ireland, province of Leinster, bordered on the south by the Dublin Mountains, a part of th ...
,
Ireland
Ireland ( ; ga, Éire ; Ulster Scots dialect, Ulster-Scots: ) is an island in the Atlantic Ocean, North Atlantic Ocean, in Northwestern Europe, north-western Europe. It is separated from Great Britain to its east by the North Channel (Grea ...
, and was interested in the problems of small samples – for example, the chemical properties of barley with small sample sizes. Hence a second version of the etymology of the term Student is that Guinness did not want their competitors to know that they were using the t-test to determine the quality of raw material (see
Student's ''t''-distribution for a detailed history of this pseudonym, which is not to be confused with the literal term ''
student
A student is a person enrolled in a school or other educational institution.
In the United Kingdom and most commonwealth countries, a "student" attends a secondary school or higher (e.g., college or university); those in primary or elementar ...
''). Although it was William Gosset after whom the term "Student" is penned, it was actually through the work of
Ronald Fisher
Sir Ronald Aylmer Fisher (17 February 1890 – 29 July 1962) was a British polymath who was active as a mathematician, statistician, biologist, geneticist, and academic. For his work in statistics, he has been described as "a genius who a ...
that the distribution became well known as "Student's distribution" and "Student's t-test".
Gosset had been hired owing to
Claude Guinness's policy of recruiting the best graduates from
Oxford
Oxford () is a city in England. It is the county town and only city of Oxfordshire. In 2020, its population was estimated at 151,584. It is north-west of London, south-east of Birmingham and north-east of Bristol. The city is home to the ...
and
Cambridge
Cambridge ( ) is a university city and the county town in Cambridgeshire, England. It is located on the River Cam approximately north of London. As of the 2021 United Kingdom census, the population of Cambridge was 145,700. Cambridge bec ...
to apply
biochemistry
Biochemistry or biological chemistry is the study of chemical processes within and relating to living organisms. A sub-discipline of both chemistry and biology, biochemistry may be divided into three fields: structural biology, enzymology and ...
and
statistics
Statistics (from German language, German: ''wikt:Statistik#German, Statistik'', "description of a State (polity), state, a country") is the discipline that concerns the collection, organization, analysis, interpretation, and presentation of ...
to Guinness's industrial processes.
Gosset devised the ''t''-test as an economical way to monitor the quality of
stout
Stout is a dark, top-fermented beer with a number of variations, including dry stout, oatmeal stout, milk stout, and imperial stout.
The first known use of the word ''stout'' for beer, in a document dated 1677 found in the Egerton Manuscript ...
. The ''t''-test work was submitted to and accepted in the journal ''
Biometrika
''Biometrika'' is a peer-reviewed scientific journal published by Oxford University Press for thBiometrika Trust The editor-in-chief is Paul Fearnhead (Lancaster University). The principal focus of this journal is theoretical statistics. It was es ...
'' and published in 1908.
Guinness had a policy of allowing technical staff leave for study (so-called "study leave"), which Gosset used during the first two terms of the 1906–1907 academic year in
Professor Karl Pearson's Biometric Laboratory at
University College London
, mottoeng = Let all come who by merit deserve the most reward
, established =
, type = Public research university
, endowment = £143 million (2020)
, budget = ...
. Gosset's identity was then known to fellow statisticians and to editor-in-chief Karl Pearson.
Uses
The most frequently used ''t''-tests are one-sample and two-sample tests:
* A one-sample
location test
A location test is a statistical hypothesis test that compares the location parameter of a statistical population to a given constant, or that compares the location parameters of two statistical populations to each other. Most commonly, the locati ...
of whether the mean of a population has a value specified in a
null hypothesis
In scientific research, the null hypothesis (often denoted ''H''0) is the claim that no difference or relationship exists between two sets of data or variables being analyzed. The null hypothesis is that any experimentally observed difference is d ...
.
* A two-sample location test of the null hypothesis such that the
mean
There are several kinds of mean in mathematics, especially in statistics. Each mean serves to summarize a given group of data, often to better understand the overall value (magnitude and sign) of a given data set.
For a data set, the ''arithme ...
s of two populations are equal. All such tests are usually called Student's ''t''-tests, though strictly speaking that name should only be used if the
variance
In probability theory and statistics, variance is the expectation of the squared deviation of a random variable from its population mean or sample mean. Variance is a measure of dispersion, meaning it is a measure of how far a set of numbers ...
s of the two populations are also assumed to be equal; the form of the test used when this assumption is dropped is sometimes called
Welch's ''t''-test. These tests are often referred to as unpaired or ''independent samples'' ''t''-tests, as they are typically applied when the
statistical units underlying the two samples being compared are non-overlapping.
Assumptions
Most test statistics have the form , where and are functions of the data.
may be sensitive to the alternative hypothesis (i.e., its magnitude tends to be larger when the alternative hypothesis is true), whereas is a
scaling parameter that allows the distribution of to be determined.
As an example, in the one-sample ''t''-test
:
where is the
sample mean
The sample mean (or "empirical mean") and the sample covariance are statistics computed from a Sample (statistics), sample of data on one or more random variables.
The sample mean is the average value (or mean, mean value) of a sample (statistic ...
from a sample , of size , is the
standard error of the mean
The standard error (SE) of a statistic (usually an estimate of a parameter) is the standard deviation of its sampling distribution or an estimate of that standard deviation. If the statistic is the sample mean, it is called the standard error of ...
,
is the estimate of the
standard deviation
In statistics, the standard deviation is a measure of the amount of variation or dispersion of a set of values. A low standard deviation indicates that the values tend to be close to the mean (also called the expected value) of the set, while ...
of the population, and is the
population mean
In statistics, a population is a set of similar items or events which is of interest for some question or experiment. A statistical population can be a group of existing objects (e.g. the set of all stars within the Milky Way galaxy) or a hypothe ...
.
The assumptions underlying a ''t''-test in the simplest form above are that:
* follows a normal distribution with mean and variance
* follows a
distribution with
degrees of freedom
Degrees of freedom (often abbreviated df or DOF) refers to the number of independent variables or parameters of a thermodynamic system. In various scientific fields, the word "freedom" is used to describe the limits to which physical movement or ...
. This assumption is met when the observations used for estimating come from a normal distribution (and i.i.d for each group).
* and are
independent
Independent or Independents may refer to:
Arts, entertainment, and media Artist groups
* Independents (artist group), a group of modernist painters based in the New Hope, Pennsylvania, area of the United States during the early 1930s
* Independ ...
.
In the ''t''-test comparing the means of two independent samples, the following assumptions should be met:
* The means of the two populations being compared should follow
normal distributions. Under weak assumptions, this follows in large samples from the
central limit theorem, even when the distribution of observations in each group is non-normal.
* If using Student's original definition of the ''t''-test, the two populations being compared should have the same variance (testable using
''F''-test,
Levene's test In statistics, Levene's test is an inferential statistic used to assess the equality of variances for a variable calculated for two or more groups. Some common statistical procedures assume that variances of the populations from which different samp ...
,
Bartlett's test, or the
Brown–Forsythe test; or assessable graphically using a
Q–Q plot
In statistics, a Q–Q plot (quantile-quantile plot) is a probability plot, a List of graphical methods, graphical method for comparing two probability distributions by plotting their ''quantiles'' against each other. A point on the plot co ...
). If the sample sizes in the two groups being compared are equal, Student's original ''t''-test is highly robust to the presence of unequal variances.
Welch's ''t''-test is insensitive to equality of the variances regardless of whether the sample sizes are similar.
* The data used to carry out the test should either be sampled independently from the two populations being compared or be fully paired. This is in general not testable from the data, but if the data are known to be dependent (e.g. paired by test design), a dependent test has to be applied. For partially paired data, the classical independent ''t''-tests may give invalid results as the test statistic might not follow a ''t'' distribution, while the dependent ''t''-test is sub-optimal as it discards the unpaired data.
Most two-sample ''t''-tests are robust to all but large deviations from the assumptions.
For
exactness, the ''t''-test and ''Z''-test require normality of the sample means, and the ''t''-test additionally requires that the sample variance follows a scaled
''χ'' distribution, and that the sample mean and sample variance be
statistically independent
Independence is a fundamental notion in probability theory, as in statistics and the theory of stochastic processes. Two events are independent, statistically independent, or stochastically independent if, informally speaking, the occurrence of o ...
. Normality of the individual data values is not required if these conditions are met. By the
central limit theorem, sample means of moderately large samples are often well-approximated by a normal distribution even if the data are not normally distributed. For non-normal data, the distribution of the sample variance may deviate substantially from a ''χ'' distribution.
However, if the sample size is large,
Slutsky's theorem In probability theory, Slutsky’s theorem extends some properties of algebraic operations on convergent sequences of real numbers to sequences of random variables.
The theorem was named after Eugen Slutsky. Slutsky's theorem is also attributed to ...
implies that the distribution of the sample variance has little effect on the distribution of the test statistic. That is as sample size
increases:
*
as per the
Central limit theorem.
*
as per the
Law of large numbers
In probability theory, the law of large numbers (LLN) is a theorem that describes the result of performing the same experiment a large number of times. According to the law, the average of the results obtained from a large number of trials shou ...
.
*
Unpaired and paired two-sample ''t''-tests
![Type 1 error](https://upload.wikimedia.org/wikipedia/commons/c/c8/Type_1_error.png)
Two-sample ''t''-tests for a difference in means involve independent samples (unpaired samples) or paired samples. Paired ''t''-tests are a form of
blocking, and have greater
power
Power most often refers to:
* Power (physics), meaning "rate of doing work"
** Engine power, the power put out by an engine
** Electric power
* Power (social and political), the ability to influence people or events
** Abusive power
Power may a ...
(probability of avoiding a type II error, also known as a false negative) than unpaired tests when the paired units are similar with respect to "noise factors" that are independent of membership in the two groups being compared. In a different context, paired ''t''-tests can be used to reduce the effects of
confounding factors
In statistics, a confounder (also confounding variable, confounding factor, extraneous determinant or lurking variable) is a variable that influences both the dependent variable and independent variable, causing a spurious association. Con ...
in an
observational study
In fields such as epidemiology, social sciences, psychology and statistics, an observational study draws inferences from a sample (statistics), sample to a statistical population, population where the dependent and independent variables, independ ...
.
Independent (unpaired) samples
The independent samples ''t''-test is used when two separate sets of
independent and identically distributed
In probability theory and statistics, a collection of random variables is independent and identically distributed if each random variable has the same probability distribution as the others and all are mutually independent. This property is usua ...
samples are obtained, and one variable from each of the two populations is compared. For example, suppose we are evaluating the effect of a medical treatment, and we enroll 100 subjects into our study, then randomly assign 50 subjects to the treatment group and 50 subjects to the control group. In this case, we have two independent samples and would use the unpaired form of the ''t''-test.
Paired samples
Paired samples ''t''-tests typically consist of a sample of matched pairs of similar
units
Unit may refer to:
Arts and entertainment
* UNIT, a fictional military organization in the science fiction television series ''Doctor Who''
* Unit of action, a discrete piece of action (or beat) in a theatrical presentation
Music
* ''Unit'' (alb ...
, or one group of units that has been tested twice (a "repeated measures" ''t''-test).
A typical example of the repeated measures ''t''-test would be where subjects are tested prior to a treatment, say for high blood pressure, and the same subjects are tested again after treatment with a blood-pressure-lowering medication. By comparing the same patient's numbers before and after treatment, we are effectively using each patient as their own control. That way the correct rejection of the null hypothesis (here: of no difference made by the treatment) can become much more likely, with statistical power increasing simply because the random interpatient variation has now been eliminated. However, an increase of statistical power comes at a price: more tests are required, each subject having to be tested twice. Because half of the sample now depends on the other half, the paired version of Student's ''t''-test has only degrees of freedom (with being the total number of observations). Pairs become individual test units, and the sample has to be doubled to achieve the same number of degrees of freedom. Normally, there are degrees of freedom (with being the total number of observations).
A paired samples ''t''-test based on a "matched-pairs sample" results from an unpaired sample that is subsequently used to form a paired sample, by using additional variables that were measured along with the variable of interest. The matching is carried out by identifying pairs of values consisting of one observation from each of the two samples, where the pair is similar in terms of other measured variables. This approach is sometimes used in observational studies to reduce or eliminate the effects of confounding factors.
Paired samples ''t''-tests are often referred to as "dependent samples ''t''-tests".
Calculations
Explicit expressions that can be used to carry out various ''t''-tests are given below. In each case, the formula for a test statistic that either exactly follows or closely approximates a ''t''-distribution under the null hypothesis is given. Also, the appropriate
degrees of freedom
Degrees of freedom (often abbreviated df or DOF) refers to the number of independent variables or parameters of a thermodynamic system. In various scientific fields, the word "freedom" is used to describe the limits to which physical movement or ...
are given in each case. Each of these statistics can be used to carry out either a
one-tailed or two-tailed test.
Once the ''t'' value and degrees of freedom are determined, a
''p''-value can be found using a
table of values from Student's ''t''-distribution. If the calculated ''p''-value is below the threshold chosen for
statistical significance
In statistical hypothesis testing, a result has statistical significance when it is very unlikely to have occurred given the null hypothesis (simply by chance alone). More precisely, a study's defined significance level, denoted by \alpha, is the p ...
(usually the 0.10, the 0.05, or 0.01 level), then the null hypothesis is rejected in favor of the alternative hypothesis.
One-sample ''t''-test
In testing the null hypothesis that the population mean is equal to a specified value , one uses the statistic
:
where
is the sample mean, is the
sample standard deviation
In statistics, the standard deviation is a measure of the amount of variation or dispersion of a set of values. A low standard deviation indicates that the values tend to be close to the mean (also called the expected value) of the set, while ...
and is the sample size. The degrees of freedom used in this test are .
Although the parent population does not need to be normally distributed, the distribution of the population of sample means
is assumed to be normal.
By the
central limit theorem, if the observations are independent and the second moment exists, then
will be approximately normal N(0;1).
Slope of a regression line
Suppose one is fitting the model
:
where is known, and are unknown, is a normally distributed random variable with mean 0 and unknown variance , and is the outcome of interest. We want to test the null hypothesis that the slope is equal to some specified value (often taken to be 0, in which case the null hypothesis is that and are uncorrelated).
Let
:
Then
:
has a ''t''-distribution with degrees of freedom if the null hypothesis is true. The
standard error of the slope coefficient:
:
can be written in terms of the residuals. Let
:
Then
score is given by:
:
Another way to determine the
score is:
:
where ''r'' is the
Pearson correlation coefficient
In statistics, the Pearson correlation coefficient (PCC, pronounced ) ― also known as Pearson's ''r'', the Pearson product-moment correlation coefficient (PPMCC), the bivariate correlation, or colloquially simply as the correlation coefficient ...
.
The
score, intercept can be determined from the
score, slope:
:
where is the sample variance.
Independent two-sample ''t''-test
Equal sample sizes and variance
Given two groups (1, 2), this test is only applicable when:
*the two sample sizes are equal;
*it can be assumed that the two distributions have the same variance;
Violations of these assumptions are discussed below.
The statistic to test whether the means are different can be calculated as follows:
:
where
:
Here is the
pooled standard deviation
In statistics, pooled variance (also known as combined variance, composite variance, or overall variance, and written \sigma^2) is a method for estimating variance of several different populations when the mean of each population may be different ...
for and and are the
unbiased estimator
In statistics, the bias of an estimator (or bias function) is the difference between this estimator's expected value and the true value of the parameter being estimated. An estimator or decision rule with zero bias is called ''unbiased''. In stat ...
s of the population variance. The denominator of is the
standard error
The standard error (SE) of a statistic (usually an estimate of a parameter) is the standard deviation of its sampling distribution or an estimate of that standard deviation. If the statistic is the sample mean, it is called the standard error ...
of the difference between two means.
For significance testing, the
degrees of freedom
Degrees of freedom (often abbreviated df or DOF) refers to the number of independent variables or parameters of a thermodynamic system. In various scientific fields, the word "freedom" is used to describe the limits to which physical movement or ...
for this test is where is sample size.
Equal or unequal sample sizes, similar variances ( < < 2)
This test is used only when it can be assumed that the two distributions have the same variance. (When this assumption is violated, see below.)
The previous formulae are a special case of the formulae below, one recovers them when both samples are equal in size: .
The statistic to test whether the means are different can be calculated as follows:
:
where
:
is the
pooled standard deviation
In statistics, pooled variance (also known as combined variance, composite variance, or overall variance, and written \sigma^2) is a method for estimating variance of several different populations when the mean of each population may be different ...
of the two samples: it is defined in this way so that its square is an
unbiased estimator
In statistics, the bias of an estimator (or bias function) is the difference between this estimator's expected value and the true value of the parameter being estimated. An estimator or decision rule with zero bias is called ''unbiased''. In stat ...
of the common variance whether or not the population means are the same. In these formulae, is the number of degrees of freedom for each group, and the total sample size minus two (that is, ) is the total number of degrees of freedom, which is used in significance testing.
Equal or unequal sample sizes, unequal variances (''s''''X''1 > 2''s''''X''2 or ''s''''X''2 > 2''s''''X''1)
This test, also known as Welch's ''t''-test, is used only when the two population variances are not assumed to be equal (the two sample sizes may or may not be equal) and hence must be estimated separately. The statistic to test whether the population means are different is calculated as:
:
where
:
Here is the
unbiased estimator
In statistics, the bias of an estimator (or bias function) is the difference between this estimator's expected value and the true value of the parameter being estimated. An estimator or decision rule with zero bias is called ''unbiased''. In stat ...
of the
variance
In probability theory and statistics, variance is the expectation of the squared deviation of a random variable from its population mean or sample mean. Variance is a measure of dispersion, meaning it is a measure of how far a set of numbers ...
of each of the two samples with = number of participants in group ( = 1 or 2). In this case
is not a pooled variance. For use in significance testing, the distribution of the test statistic is approximated as an ordinary Student's ''t''-distribution with the degrees of freedom calculated using
:
This is known as the
Welch–Satterthwaite equation
In statistics and uncertainty analysis, the Welch–Satterthwaite equation is used to calculate an approximation to the effective degrees of freedom of a linear combination of independent sample variances, also known as the pooled degrees of freed ...
. The true distribution of the test statistic actually depends (slightly) on the two unknown population variances (see
Behrens–Fisher problem
In statistics, the Behrens–Fisher problem, named after Walter Behrens and Ronald Fisher, is the problem of interval estimation and hypothesis testing concerning the difference between the means of two normally distributed populations when the ...
).
Exact method for unequal variances and sample sizes
The test deals with the famous
Behrens–Fisher problem
In statistics, the Behrens–Fisher problem, named after Walter Behrens and Ronald Fisher, is the problem of interval estimation and hypothesis testing concerning the difference between the means of two normally distributed populations when the ...
, i.e., comparing the difference between the means of two normally distributed populations when the variances of the two populations are not assumed to be equal, based on two independent samples.
The test is developed as an
exact test
In statistics, an exact (significance) test is a test such that if the null hypothesis is true, then all assumptions made during the derivation of the distribution of the test statistic are met. Using an exact test provides a significance test th ...
that allows for unequal sample sizes and unequal variances of two populations. The exact property still holds even with small extremely small and unbalanced sample sizes (e.g.
).
The Te statistic to test whether the means are different can be calculated as follows:
Let
and
be the i.i.d. sample vectors (
) from
and
separately.
Let
be an
orthogonal matrix whose elements of the first row are all
, similarly, let
be the first n rows of an
orthogonal matrix (whose elements of the first row are all
).
Then
is an n-dimensional normal random vector.
:
From the above distribution we see that
:
:
:
:
Dependent ''t''-test for paired samples
This test is used when the samples are dependent; that is, when there is only one sample that has been tested twice (repeated measures) or when there are two samples that have been matched or "paired". This is an example of a
paired difference test In statistics, a paired difference test is a type of location test that is used when comparing two sets of measurements to assess whether their expected value, population means differ. A paired difference test uses additional information about the ...
. The ''t'' statistic is calculated as
:
where
and
are the average and standard deviation of the differences between all pairs. The pairs are e.g. either one person's pre-test and post-test scores or between-pairs of persons matched into meaningful groups (for instance drawn from the same family or age group: see table). The constant is zero if we want to test whether the average of the difference is significantly different. The degree of freedom used is , where represents the number of pairs.
:
:
Worked examples
Let denote a set obtained by drawing a random sample of six measurements:
:
and let denote a second set obtained similarly:
:
These could be, for example, the weights of screws that were chosen out of a bucket.
We will carry out tests of the null hypothesis that the
mean
There are several kinds of mean in mathematics, especially in statistics. Each mean serves to summarize a given group of data, often to better understand the overall value (magnitude and sign) of a given data set.
For a data set, the ''arithme ...
s of the populations from which the two samples were taken are equal.
The difference between the two sample means, each denoted by , which appears in the numerator for all the two-sample testing approaches discussed above, is
:
The sample
standard deviations
In statistics, the standard deviation is a measure of the amount of variation or dispersion of a set of values. A low standard deviation indicates that the values tend to be close to the mean (also called the expected value) of the set, while ...
for the two samples are approximately 0.05 and 0.11, respectively. For such small samples, a test of equality between the two population variances would not be very powerful. Since the sample sizes are equal, the two forms of the two-sample ''t''-test will perform similarly in this example.
Unequal variances
If the approach for unequal variances (discussed above) is followed, the results are
:
and the degrees of freedom
:
The test statistic is approximately 1.959, which gives a two-tailed test ''p''-value of 0.09077.
Equal variances
If the approach for equal variances (discussed above) is followed, the results are
:
and the degrees of freedom
:
The test statistic is approximately equal to 1.959, which gives a two-tailed ''p''-value of 0.07857.
Related statistical tests
Alternatives to the ''t''-test for location problems
The ''t''-test provides an exact test for the equality of the means of two i.i.d. normal populations with unknown, but equal, variances. (
Welch's ''t''-test is a nearly exact test for the case where the data are normal but the variances may differ.) For moderately large samples and a one tailed test, the ''t''-test is relatively robust to moderate violations of the normality assumption.
In large enough samples, the t-test asymptotically approaches the
''z''-test, and becomes robust even to large deviations from normality.
If the data are substantially non-normal and the sample size is small, the ''t''-test can give misleading results. See
Location test for Gaussian scale mixture distributions for some theory related to one particular family of non-normal distributions.
When the normality assumption does not hold, a
non-parametric
Nonparametric statistics is the branch of statistics that is not based solely on parametrized families of probability distributions (common examples of parameters are the mean and variance). Nonparametric statistics is based on either being distri ...
alternative to the ''t''-test may have better
statistical power
In statistics, the power of a binary hypothesis test is the probability that the test correctly rejects the null hypothesis (H_0) when a specific alternative hypothesis (H_1) is true. It is commonly denoted by 1-\beta, and represents the chances ...
. However, when data are non-normal with differing variances between groups, a t-test may have better
type-1 error control than some non-parametric alternatives. Furthermore, non-parametric methods, such as the
Mann-Whitney U test discussed below, typically do not test for a difference of means, so should be used carefully if a difference of means is of primary scientific interest.
For example, Mann-Whitney U test will keep the type 1 error at the desired level alpha if both groups have the same distribution. It will also have power in detecting an alternative by which group B has the same distribution as A but after some shift by a constant (in which case there would indeed be a difference in the means of the two groups). However, there could be cases where group A and B will have different distributions but with the same means (such as two distributions, one with positive skewness and the other with a negative one, but shifted so to have the same means). In such cases, MW could have more than alpha level power in rejecting the Null hypothesis but attributing the interpretation of difference in means to such a result would be incorrect.
In the presence of an
outlier
In statistics, an outlier is a data point that differs significantly from other observations. An outlier may be due to a variability in the measurement, an indication of novel data, or it may be the result of experimental error; the latter are ...
, the t-test is not robust. For example, for two independent samples when the data distributions are asymmetric (that is, the distributions are
skewed
In probability theory and statistics, skewness is a measure of the asymmetry of the probability distribution of a real-valued random variable about its mean. The skewness value can be positive, zero, negative, or undefined.
For a unimoda ...
) or the distributions have large tails, then the Wilcoxon rank-sum test (also known as the
Mann–Whitney ''U'' test) can have three to four times higher power than the ''t''-test.
The nonparametric counterpart to the paired samples ''t''-test is the
Wilcoxon signed-rank test
The Wilcoxon signed-rank test is a non-parametric statistical hypothesis test used either to test the location of a population based on a sample of data, or to compare the locations of two populations using two matched samples., p. 350 The one-sa ...
for paired samples. For a discussion on choosing between the ''t''-test and nonparametric alternatives, see Lumley, et al. (2002).
One-way
analysis of variance
Analysis of variance (ANOVA) is a collection of statistical models and their associated estimation procedures (such as the "variation" among and between groups) used to analyze the differences among means. ANOVA was developed by the statisticia ...
(ANOVA) generalizes the two-sample ''t''-test when the data belong to more than two groups.
A design which includes both paired observations and independent observations
When both paired observations and independent observations are present in the two sample design, assuming data are missing completely at random (MCAR), the paired observations or independent observations may be discarded in order to proceed with the standard tests above. Alternatively making use of all of the available data, assuming normality and MCAR, the generalized partially overlapping samples t-test could be used.
Multivariate testing
A generalization of Student's ''t'' statistic, called
Hotelling's ''t''-squared statistic, allows for the testing of hypotheses on multiple (often correlated) measures within the same sample. For instance, a researcher might submit a number of subjects to a personality test consisting of multiple personality scales (e.g. the
Minnesota Multiphasic Personality Inventory
The Minnesota Multiphasic Personality Inventory (MMPI) is a standardized psychometric test of adult personality and psychopathology. Psychologists and other mental health professionals use various versions of the MMPI to help develop treatment ...
). Because measures of this type are usually positively correlated, it is not advisable to conduct separate univariate ''t''-tests to test hypotheses, as these would neglect the covariance among measures and inflate the chance of falsely rejecting at least one hypothesis (
Type I error
In statistical hypothesis testing, a type I error is the mistaken rejection of an actually true null hypothesis (also known as a "false positive" finding or conclusion; example: "an innocent person is convicted"), while a type II error is the fa ...
). In this case a single multivariate test is preferable for hypothesis testing.
Fisher's Method
In statistics, Fisher's method, also known as Fisher's combined probability test, is a technique for data fusion or "meta-analysis" (analysis of analyses). It was developed by and named for Ronald Fisher. In its basic form, it is used to combin ...
for combining multiple tests with ''
alpha
Alpha (uppercase , lowercase ; grc, ἄλφα, ''álpha'', or ell, άλφα, álfa) is the first letter of the Greek alphabet. In the system of Greek numerals, it has a value of one. Alpha is derived from the Phoenician letter aleph , whic ...
'' reduced for positive correlation among tests is one. Another is Hotelling's ''T'' statistic follows a ''T'' distribution. However, in practice the distribution is rarely used, since tabulated values for ''T'' are hard to find. Usually, ''T'' is converted instead to an ''F'' statistic.
For a one-sample multivariate test, the hypothesis is that the mean vector () is equal to a given vector (). The test statistic is
Hotelling's ''t'':
:
where is the sample size, is the vector of column means and is an
sample covariance matrix
The sample mean (or "empirical mean") and the sample covariance are statistics computed from a sample of data on one or more random variables.
The sample mean is the average value (or mean value) of a sample of numbers taken from a larger popula ...
.
For a two-sample multivariate test, the hypothesis is that the mean vectors () of two samples are equal. The test statistic is
Hotelling's two-sample ''t'':
:
Software implementations
Many
spreadsheet
A spreadsheet is a computer application for computation, organization, analysis and storage of data in tabular form. Spreadsheets were developed as computerized analogs of paper accounting worksheets. The program operates on data entered in cel ...
programs and statistics packages, such as
QtiPlot
QtiPlot is a cross-platform computer program for interactive Plot (graphics), scientific graphing and data analysis. It is similar to the proprietary programs Origin (data analysis software), Origin or SigmaPlot.
QtiPlot can be used to present 2 ...
,
LibreOffice Calc
LibreOffice Calc is the spreadsheet component of the LibreOffice software package.
After forking from OpenOffice.org in 2010, LibreOffice Calc underwent a massive re-work of external reference handling to fix many defects in formula calculation ...
,
Microsoft Excel
Microsoft Excel is a spreadsheet developed by Microsoft for Microsoft Windows, Windows, macOS, Android (operating system), Android and iOS. It features calculation or computation capabilities, graphing tools, pivot tables, and a macro (comp ...
,
SAS
SAS or Sas may refer to:
Arts, entertainment, and media
* ''SAS'' (novel series), a French book series by Gérard de Villiers
* ''Shimmer and Shine'', an American animated children's television series
* Southern All Stars, a Japanese rock ba ...
,
SPSS
SPSS Statistics is a statistical software suite developed by IBM for data management, advanced analytics, multivariate analysis, business intelligence, and criminal investigation. Long produced by SPSS Inc., it was acquired by IBM in 2009. C ...
,
Stata
Stata (, , alternatively , occasionally stylized as STATA) is a general-purpose statistical software package developed by StataCorp for data manipulation, visualization, statistics, and automated reporting. It is used by researchers in many fie ...
,
DAP
DAP or Dap may refer to:
Science
* DAP (gene), human gene that encodes death-associated proteins, which mediate programmed cell death
* Diamidophosphate, phosphorylating compound
* Diaminopimelic acid, amino acid derivative of lysine
* Diamin ...
,
gretl
gretl is an open-source statistical package, mainly for econometrics. The name is an acronym for ''G''nu ''R''egression, ''E''conometrics and ''T''ime-series ''L''ibrary.
It has both a graphical user interface (GUI) and a command-line interfa ...
,
R,
Python
Python may refer to:
Snakes
* Pythonidae, a family of nonvenomous snakes found in Africa, Asia, and Australia
** ''Python'' (genus), a genus of Pythonidae found in Africa and Asia
* Python (mythology), a mythical serpent
Computing
* Python (pro ...
,
PSPP
PSPP is a free software application for analysis of sampled data, intended as a free alternative for IBM SPSS Statistics. It has a graphical user interface and conventional command-line interface. It is written in C and uses GNU Scientific Lib ...
,
MATLAB
MATLAB (an abbreviation of "MATrix LABoratory") is a proprietary multi-paradigm programming language and numeric computing environment developed by MathWorks. MATLAB allows matrix manipulations, plotting of functions and data, implementation ...
and
Minitab
Minitab is a statistics package developed at the Pennsylvania State University by researchers Barbara F. Ryan, Thomas A. Ryan, Jr., and Brian L. Joiner in conjunction with Triola Statistics Company in 1972. It began as a light version of OMNITA ...
, include implementations of Student's ''t''-test.
See also
*
Conditional change model
*
''F''-test
*
Noncentral ''t''-distribution in power analysis
*
Student's ''t''-statistic
*
''Z''-test
*
Mann–Whitney ''U'' test
*
Šidák correction for ''t''-test
*
Welch's ''t''-test
*
Analysis of variance
Analysis of variance (ANOVA) is a collection of statistical models and their associated estimation procedures (such as the "variation" among and between groups) used to analyze the differences among means. ANOVA was developed by the statisticia ...
(ANOVA)
References
Citations
Sources
*
*
Further reading
*
*
External links
*
* Trochim, William M.K.
The T-Test, ''Research Methods Knowledge Base'', conjoint.ly
* by
Mark Thoma
Mark Allen Thoma (born December 15, 1956) is a macroeconomist and econometrician and a professor of economics at the Department of Economics of the University of Oregon. Thoma is best known as a regular columnist for ''The Fiscal Times'' through ...
{{DEFAULTSORT:Student's T-Test
Statistical tests
Parametric statistics