HOME

TheInfoList



OR:

In medicine and psychology, clinical significance is the practical importance of a treatment effect—whether it has a real genuine, palpable, noticeable effect on daily life.


Types of significance


Statistical significance

Statistical significance is used in
hypothesis testing A statistical hypothesis test is a method of statistical inference used to decide whether the data at hand sufficiently support a particular hypothesis. Hypothesis testing allows us to make probabilistic statements about population parameters. ...
, whereby the
null hypothesis In scientific research, the null hypothesis (often denoted ''H''0) is the claim that no difference or relationship exists between two sets of data or variables being analyzed. The null hypothesis is that any experimentally observed difference is d ...
(that there is no relationship between variables) is tested. A
level of significance In statistical hypothesis testing, a type I error is the mistaken rejection of an actually true null hypothesis (also known as a "false positive" finding or conclusion; example: "an innocent person is convicted"), while a type II error is the fa ...
is selected (most commonly ''α'' = 0.05 or 0.01), which signifies the probability of incorrectly rejecting a true null hypothesis. If there is a significant difference between two groups at ''α'' = 0.05, it means that there is only a 5% probability of obtaining the observed results under the assumption that the difference is entirely due to chance (i.e., the null hypothesis is true); it gives no indication of the magnitude or clinical importance of the difference. When statistically significant results are achieved, they favor rejection of the null hypothesis, but they do not prove that the null hypothesis is false. Likewise, non-significant results do not prove that the null hypothesis is true; they also give no evidence of the truth or falsity of the hypothesis the researcher has generated. Statistical significance relates only to the compatibility between observed data and what would be expected under the assumption that the null hypothesis is true.


Practical significance

In broad usage, the "practical clinical significance" answers the question, ''how effective'' is the intervention or treatment, or how much change does the treatment cause. In terms of testing clinical treatments, practical significance optimally yields quantified information about the importance of a finding, using metrics such as
effect size In statistics, an effect size is a value measuring the strength of the relationship between two variables in a population, or a sample-based estimate of that quantity. It can refer to the value of a statistic calculated from a sample of data, the ...
,
number needed to treat The number needed to treat (NNT) or number needed to treat for an additional beneficial outcome (NNTB) is an epidemiological measure used in communicating the effectiveness of a health-care intervention, typically a treatment with medication. The ...
(NNT), and
preventive fraction In epidemiology, preventable fraction among the unexposed (PFu), is the proportion of incidents in the unexposed group that could be prevented by exposure. It is calculated as PF_u = (I_u - I_e)/I_u = 1 - RR, where I_e is the incidence in the expose ...
. Practical significance may also convey semi-quantitative, comparative, or feasibility assessments of utility. Effect size is one type of practical significance. It quantifies the extent to which a sample diverges from expectations. Effect size can provide important information about the results of a study, and are recommended for inclusion in addition to statistical significance. Effect sizes have their own sources of bias, are subject to change based on population variability of the dependent variable, and tend to focus on group effects, not individual changes. Although clinical significance and practical significance are often used synonymously, a more technical restrictive usage denotes this as erroneous. This technical use within psychology and psychotherapy not only results from a carefully drawn precision and particularity of language, but it enables a shift in perspective from group effects to the specifics of change(s) within an individual.


Specific usage

In contrast, when used as a technical term within psychology and psychotherapy, clinical significance yields information on whether a treatment was effective enough to change a patient’s diagnostic label. In terms of clinical treatment studies, clinical significance answers the question "Is a treatment effective enough to cause the patient to be normal ith respect to the diagnostic criteria in question" For example, a treatment might significantly change depressive symptoms (statistical significance), the change could be a large decrease in depressive symptoms (practical significance- effect size), and 40% of the patients no longer met the diagnostic criteria for depression (clinical significance). It is very possible to have a treatment that yields a significant difference and medium or large effect sizes, but does not move a patient from dysfunctional to functional. Within psychology and psychotherapy, clinical significance was first proposed by Jacobson, Follette, and Revenstorf as a way to answer the question, is a therapy or treatment effective enough such that a client does not meet the criteria for a diagnosis? Jacobson and Truax later defined clinical significance as "the extent to which therapy moves someone outside the range of the dysfunctional population or within the range of the functional population." They proposed two components of this index of change: the status of a patient or client after therapy has been completed, and "how much change has occurred during the course of therapy." Clinical significance is also a consideration when interpreting the results of the
psychological assessment Psychological evaluation is a method to assess an individual's behavior, personality, cognitive abilities, and several other domains. A common reason for a psychological evaluation is to identify psychological factors that may be inhibiting a pers ...
of an individual. Frequently, there will be a difference of scores or subscores that is
statistically significant In statistical hypothesis testing, a result has statistical significance when it is very unlikely to have occurred given the null hypothesis (simply by chance alone). More precisely, a study's defined significance level, denoted by \alpha, is the p ...
, unlikely to have occurred purely by chance. However, not all of those statistically significant differences are clinically significant, in that they do not either explain existing information about the client, or provide useful direction for intervention. Differences that are small in magnitude typically lack practical relevance and are unlikely to be clinically significant. Differences that are common in the population are also unlikely to be clinically significant, because they may simply reflect a level of normal human variation. Additionally, clinicians look for information in the assessment data and the client's history that corroborates the relevance of the statistical difference, to establish the connection between performance on the specific test and the individual's more general functioning.


Calculation of clinical significance

Just as there are many ways to calculate statistical significance and practical significance, there are a variety of ways to calculate clinical significance. Five common methods are the Jacobson-Truax method, the Gulliksen-Lord-Novick method, the Edwards-Nunnally method, the Hageman-Arrindell method, and hierarchical linear modeling.


Jacobson-Truax

Jacobson-Truax is common method of calculating clinical significance. It involves calculating a Reliability Change Index (RCI). The RCI equals the difference between a participant’s pre-test and post-test scores, divided by the
standard error The standard error (SE) of a statistic (usually an estimate of a parameter) is the standard deviation of its sampling distribution or an estimate of that standard deviation. If the statistic is the sample mean, it is called the standard error ...
of the difference. Cutoff scores are established for placing participants into one of four categories: recovered, improved, unchanged, or deteriorated, depending on the directionality of the RCI and whether the cutoff score was met.


Gulliksen-Lord-Novick

The Gulliksen-Lord-Novick method is similar to Jacobson-Truax, except that it takes into account
regression to the mean In statistics, regression toward the mean (also called reversion to the mean, and reversion to mediocrity) is the fact that if one sample of a random variable is extreme, the next sampling of the same random variable is likely to be closer to it ...
. This is done by subtracting the pre-test and post-test scores from a population mean, and dividing by the standard deviation of the population.


Edwards-Nunnally

The Edwards-Nunnally method of calculating clinical significance is a more stringent alternative to the Jacobson-Truax method. Reliability scores are used to bring the pre-test scores closer to the mean, and then a confidence interval is developed for this adjusted pre-test score. Confidence intervals are used when calculating the change from pre-test to post-test, so greater actual change in scores is necessary to show clinical significance, compared to the Jacobson-Truax method.


Hageman-Arrindell

The Hageman-Arrindell calculation of clinical significance involves indices of group change and of individual change. The reliability of change indicates whether a patient has improved, stayed the same, or deteriorated. A second index, the clinical significance of change, indicates four categories similar to those used by Jacobson-Truax: deteriorated, not reliably changed, improved but not recovered, and recovered.


Hierarchical linear modeling (HLM)

HLM involves growth curve analysis instead of pre-test post-test comparisons, so three data points are needed from each patient, instead of only two data points (pre-test and post-test). A computer program, such as Hierarchical Linear and Nonlinear Modeling is used to calculate change estimates for each participant. HLM also allows for analysis of growth curve models of dyads and groups.


See also

* Cohen's ''h'' *
Medical statistics Medical statistics deals with applications of statistics to medicine and the health sciences, including epidemiology, public health, forensic medicine, and clinical research. Medical statistics has been a recognized branch of statistics in the U ...
*
Minimal clinically important difference The minimal important difference (MID) or minimal clinically important difference (MCID) is the smallest change in a treatment outcome that an individual patient would identify as important and which would indicate a change in the patient's managem ...


References

{{DEFAULTSORT:Clinical Significance Clinical research Clinical trials Biostatistics