P-hacking

In pharmaceutical research

The problem of p-hacking is worryingly common in pharmaceutical drug development and research. Drugs with no discernible effect can be shown to have a statistically significant effect when hundreds, if not thousands, of analyses are run – on the same data – using different target groups, primary outcome measures, age adjustments, and myriad other variables which can be dropped or selected to obtain a meaningful result. Even where a drug has no measurable effect whatsoever, if enough analyses are run, a strong, statistically meaningful effect will be found (e.g., "the drug significantly improved memory function in women over 40", where memory function was not the original primary outcome being assessed). This does not require any data falsification, nor for inconvenient results to be omitted. Instead, p-hacking relies simply on repeated analyses of the same data until the researcher finds the correlation with the "best" p-value. The analysis is fine-tuned to the data, rather than the data being manipulated in any way.