In a scientific study, post hoc analysis (from
Latin
Latin (, or , ) is a classical language belonging to the Italic branch of the Indo-European languages. Latin was originally a dialect spoken in the lower Tiber area (then known as Latium) around present-day Rome, but through the power of the ...
''
post hoc'', "after this") consists of
statistical analyses
Statistics (from German: ''Statistik'', "description of a state, a country") is the discipline that concerns the collection, organization, analysis, interpretation, and presentation of data. In applying statistics to a scientific, industria ...
that were specified after the data were seen. They are usually used to uncover specific differences between three or more group means when an
analysis of variance
Analysis of variance (ANOVA) is a collection of statistical models and their associated estimation procedures (such as the "variation" among and between groups) used to analyze the differences among means. ANOVA was developed by the statisticia ...
(ANOVA) test is significant. This typically creates a
multiple testing
In statistics, the multiple comparisons, multiplicity or multiple testing problem occurs when one considers a set of statistical inferences simultaneously or infers a subset of parameters selected based on the observed values.
The more inferences ...
problem because each potential analysis is effectively a
statistical test
A statistical hypothesis test is a method of statistical inference used to decide whether the data at hand sufficiently support a particular hypothesis.
Hypothesis testing allows us to make probabilistic statements about population parameters.
...
. Multiple testing procedures are sometimes used to compensate, but that is often difficult or impossible to do precisely. Post hoc analysis that is conducted and interpreted without adequate consideration of this problem is sometimes called ''
data dredging
Data dredging (also known as data snooping or ''p''-hacking) is the misuse of data analysis to find patterns in data that can be presented as statistically significant, thus dramatically increasing and understating the risk of false positives. ...
'' by critics because the statistical associations that it finds are often spurious.
Common post hoc tests
Some common post hoc tests include:
[{{Cite web , last=Pamplona , first=Fabricio , date=2022-07-28 , title=Post Hoc Analysis: Process and types of tests , url=https://mindthegraph.com/blog/post-hoc-analysis/ , access-date=2022-12-09 , website=Mind the Graph Blog , language=en-US]
*
Holm-Bonferroni Procedure
*
Newman-Keuls
*
Rodger’s Method
*
Scheffé’s Method
*
Tukey’s Test Tukey's test is either:
* Tukey's range test, also called Tukey method, Tukey's honest significance test, Tukey's HSD (Honestly Significant Difference) test
* Tukey's test of additivity In statistics, Tukey's test of additivity, named for John Tu ...
(see also: Studentized Range Distribution)
Causes
Sometimes the temptation to engage in post hoc analysis is motivated by a desire to produce positive results or see a project as successful. In the case of pharmaceutical research, there may be significant financial consequences to a failed trial.
See also
*
HARKing
HARKing (hypothesizing after the results are known) is an acronym coined by social psychologist Norbert Kerr that refers to the questionable research practice of “presenting a post hoc hypothesis in the introduction of a research report as if it ...
*
Testing hypotheses suggested by the data
In statistics, hypotheses suggested by a given dataset, when tested with the same dataset that suggested them, are likely to be accepted even when they are not true. This is because circular reasoning (double dipping) would be involved: somethi ...
*
Nemenyi test
In statistics, the Nemenyi test is a post-hoc test intended to find the groups of data that differ after a global statistical test (such as the Friedman test) has rejected the null hypothesis that the performance of the comparisons on the groups ...
References
Data analysis
Multiple comparisons
Clinical research
Medical statistics