Cronbach's alpha

Cronbach's alpha (Cronbach's $\alpha$), also known as tau-equivalent reliability ($\rho_T$) or coefficient alpha (coefficient $\alpha$), is a reliability coefficient that provides a method of measuring internal consistency of tests and measures. Numerous studies warn against using it unconditionally, and note that reliability coefficients based on structural equation modeling (SEM) are in many cases a suitable alternative.Sijtsma, K. (2009). On the use, the misuse, and the very limited usefulness of Cronbach's alpha. Psychometrika, 74(1), 107–120. Green, S. B., & Yang, Y. (2009). Commentary on coefficient alpha: A cautionary tale. Psychometrika, 74(1), 121–135. Revelle, W., & Zinbarg, R. E. (2009). Coefficients alpha, beta, omega, and the glb: Comments on Sijtsma. Psychometrika, 74(1), 145–154. Cho, E., & Kim, S. (2015). Cronbach's coefficient alpha: Well known but poorly understood. Organizational Research Methods, 18(2), 207–230. Raykov, T., & Marcoulides, G. A. (2017). Thanks coefficient alpha, we still need you! Educational and Psychological Measurement, 79(1), 200–210.

History

Cronbach (1951)

As with previous studies,Hoyt, C. (1941). Test reliability estimated by analysis of variance. Psychometrika, 6(3), 153–160. Guttman, L. (1945). A basis for analyzing test-retest reliability. Psychometrika, 10(4), 255–282. Jackson, R. W. B., & Ferguson, G. A. (1941). Studies on the reliability of tests. University of Toronto Department of Educational Research Bulletin, 12, 132.Gulliksen, H. (1950). Theory of mental tests. John Wiley & Sons. Cronbach (1951) published an additional method to derive Cronbach's alpha.Cronbach, L.J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16 (3), 297–334. His interpretation was more intuitively attractive than those of previous studies and became quite popular.

After 1951

Novick and Lewis (1967)Novick, M. R., & Lewis, C. (1967). Coefficient alpha and the reliability of composite measurements. Psychometrika, 32(1), 1–13. proved the necessary and sufficient condition for $\rho_$ to be equal to reliability, and named it the condition of being essentially tau-equivalent.

Cronbach (1978) mentioned that the reason Cronbach (1951) received a lot of citations was "mostly because eput a brand name on a common-place coefficient". He explained that he had originally planned to name other types of reliability coefficients (e.g., inter-rater reliability or test-retest reliability) in consecutive Greek letter (e.g., $\beta$, $\gamma$, $\ldots$), but later changed his mind.

Cronbach and Schavelson (2004)Cronbach, L. J., & Shavelson, R. J. (2004). My Current Thoughts on Coefficient Alpha and Successor Procedures. Educational and Psychological Measurement, 64(3), 391–418. encouraged readers to use generalizability theory rather than $\rho_$. He opposed the use of the name Cronbach's alpha. He explicitly denied the existence of studies that had published the general formula of KR-20 prior to Cronbach (1951).

Prerequisites for using Cronbach's alpha

In order to use Cronbach’s alpha as a reliability coefficient, the data from the measure must satisfy the following conditions.
# Normality distributed and linear
# Tau-equivalence (essential)
# Independence between errors

Formula and calculation

Cronbach’s alpha is calculated by taking the score from each scale item and correlating them with the total score for each observation and then comparing that with the variance for all individual item scores. Cronbach’s alpha is best understood as a function of the number of questions or items in a measure, the between pairs of items average covariance, and the overall variance of the total measured score.$\alpha = () \left(1 - \right)$$k$ the number of items in the measure

$\sigma_y^2$ variance associated with each

$\sigma_x^2$ variance associated of the total scores

Common misconceptions

The value of Cronbach's alpha ranges between zero and one

By definition, reliability cannot be less than zero and cannot be greater than one. Many textbooks mistakenly equate $\rho_$ with reliability and give an inaccurate explanation of its range. $\rho_$ can be less than reliability when applied to data that are not tau-equivalent. Suppose that $X_2$ copied the value of $X_1$ as it is, and $X_3$ copied by multiplying the value of $X_1$ by -1. The covariance matrix between items is as follows, $\rho_=-3$.

Negative $\rho_$ can occur for reasons such as negative discrimination or mistakes in processing reversely scored items.

Unlike $\rho_$, SEM-based reliability coefficients (e.g., $\rho_$) are always greater than or equal to zero.

This anomaly was first pointed out by Cronbach (1943)Cronbach, L. J. (1943). On estimates of test reliability. Journal of Educational Psychology, 34(8), 485–494. to criticize $\rho_$, but Cronbach (1951) did not comment on this problem in his article, which discussed all conceivable issues related $\rho_$ and he himself described as being "encyclopedic".

If there is no measurement error, the value of Cronbach's alpha is one

This anomaly also originates from the fact that $\rho_$ underestimates reliability. Suppose that $X_2$ copied the value of $X_1$ as it is, and $X_3$ copied by multiplying the value of $X_1$ by two. The covariance matrix between items is as follows, $\rho_=0.9375$.

For the above data, both $\rho_$ and $\rho_$ have a value of one.

The above example is presented by Cho and Kim (2015).

A high value of Cronbach's alpha indicates homogeneity between the items

Many textbooks refer to $\rho_$ as an indicator of homogeneity between items. This misconception stems from the inaccurate explanation of Cronbach (1951) that high $\rho_$ values show homogeneity between the items. Homogeneity is a term that is rarely used in the modern literature, and related studies interpret the term as referring to uni-dimensionality. Several studies have provided proofs or counterexamples that high $\rho_$ values do not indicate uni-dimensionality.Cortina, J. M. (1993). What is coefficient alpha? An examination of theory and applications. Journal of Applied Psychology, 78(1), 98–104. Green, S. B., Lissitz, R. W., & Mulaik, S. A. (1977). Limitations of coefficient alpha as an Index of test unidimensionality. Educational and Psychological Measurement, 37(4), 827–838. Ten Berge, J. M. F., & Sočan, G. (2004). The greatest lower bound to the reliability of a test and the hypothesis of unidimensionality. Psychometrika, 69(4), 613–625. See counterexamples below.

$\rho_=0.72$ in the unidimensional data above.

$\rho_=0.72$ in the multidimensional data above.

The above data have $\rho_=0.9692$, but are multidimensional.

The above data have $\rho_=0.4$, but are unidimensional.

Uni-dimensionality is a prerequisite for $\rho_$. You should check uni-dimensionality before calculating $\rho_$, rather than calculating $\rho_$ to check uni-dimensionality.

A high value of Cronbach's alpha indicates internal consistency

The term internal consistency is commonly used in the reliability literature, but its meaning is not clearly defined. The term is sometimes used to refer to a certain kind of reliability (e.g., internal consistency reliability), but it is unclear exactly which reliability coefficients are included here, in addition to $\rho_$. Cronbach (1951) used the term in several senses without an explicit definition. Cho and Kim (2015) showed that is $\rho_$ is not an indicator of any of these.

Removing items using "alpha if item deleted" always increases reliability

Removing an item using "alpha if item deleted" may result in 'alpha inflation,' where sample-level reliability is reported to be higher than population-level reliability.Kopalle, P. K., & Lehmann, D. R. (1997). Alpha inflation? The impact of eliminating scale items on Cronbach's alpha. Organizational Behavior and Human Decision Processes, 70(3), 189–197. It may also reduce population-level reliability.Raykov, T. (2007). Reliability if deleted, not 'alpha if deleted': Evaluation of scale reliability following component deletion. The British Journal of Mathematical and Statistical Psychology, 60(2), 201–216. The elimination of less-reliable items should be based not only on a statistical basis but also on a theoretical and logical basis. It is also recommended that the whole sample be divided into two and cross-validated.

Ideal reliability level and how to increase reliability

Nunnally's recommendations for the level of reliability

The most frequently cited source of how high reliability coefficients should be is Nunnally's book.Nunnally, J. C. (1967). Psychometric theory. New York, NY: McGraw-Hill.Nunnally, J. C. (1978). Psychometric theory (2nd ed.). New York, NY: McGraw-Hill.Nunnally, J. C., & Bernstein, I. H. (1994). Psychometric theory (3rd ed.). New York, NY: McGraw-Hill. However, his recommendations are cited contrary to his intentions. What he meant was to apply different criteria depending on the purpose or stage of the study. However, regardless of the nature of the research, such as exploratory research, applied research, and scale development research, a criterion of 0.7 is universally used.Lance, C. E., Butts, M. M., & Michels, L. C. (2006). What did they really say? Organizational Research Methods, 9(2), 202–220. 0.7 is the criterion he recommended for the early stages of a study, which most studies published in the journal are not. Rather than 0.7, the criterion of 0.8 referred to applied research by Nunnally is more appropriate for most empirical studies.

His recommendation level did not imply a cutoff point. If a criterion means a cutoff point, it is important whether or not it is met, but it is unimportant how much it is over or under. He did not mean that it should be strictly 0.8 when referring to the criteria of 0.8. If the reliability has a value near 0.8 (e.g., 0.78), it can be considered that his recommendation has been met.Cho, E. (2020). A comprehensive review of so-called Cronbach's alpha. Journal of Product Research, 38(1), 9–20.

Cost to obtain a high level of reliability

Nunnally's idea was that there is a cost to increasing reliability, so there is no need to try to obtain maximum reliability in every situation.

Trade-off with validity

Measurements with perfect reliability lack validity. For example, a person who take the test with the reliability of one will get a perfect score or a zero score, because the examinee who gives the correct answer or incorrect answer on one item will give the correct answer or incorrect answer on all other items. The phenomenon in which validity is sacrificed to increase reliability is called attenuation paradox.

A high value of reliability can be in conflict with content validity. For high content validity, each item should be constructed to be able to comprehensively represent the content to be measured. However, a strategy of repeatedly measuring essentially the same question in different ways is often used only for the purpose of increasing reliability.

Trade-off with efficiency

When the other conditions are equal, reliability increases as the number of items increases. However, the increase in the number of items hinders the efficiency of measurements.

Methods to increase reliability

Despite the costs associated with increasing reliability discussed above, a high level of reliability may be required. The following methods can be considered to increase reliability.

Before data collection:
* Eliminate the ambiguity of the measurement item.
* Do not measure what the respondents do not know.
* Increase the number of items. However, care should be taken not to excessively inhibit the efficiency of the measurement.
* Use a scale that is known to be highly reliable.
* Conduct a pretest - discover in advance the problem of reliability.
* Exclude or modify items that are different in content or form from other items (e.g., reverse-scored items).

After data collection:
* Remove the problematic items using "alpha if item deleted". However, this deletion should be accompanied by a theoretical rationale.
* Use a more accurate reliability coefficient than $\rho_$. For example, $\rho_$ is 0.02 larger than $\rho_$ on average.Peterson, R. A., & Kim, Y. (2013). On the relationship between coefficient alpha and composite reliability. Journal of Applied Psychology, 98(1), 194–198.

Which reliability coefficient to use

$\rho_T$ is used in an overwhelming proportion. A study estimates that approximately 97% of studies use $\rho_T$ as a reliability coefficient.

However, simulation studies comparing the accuracy of several reliability coefficients have led to the common result that $\rho_T$ is an inaccurate reliability coefficient.Kamata, A., Turhan, A., & Darandari, E. (2003). Estimating reliability for multidimensional composite scale scores. Annual Meeting of American Educational Research Association, Chicago, April 2003, April, 1–27.Osburn, H. G. (2000). Coefficient alpha and related internal consistency reliability coefficients. Psychological Methods, 5(3), 343–355. Tang, W., & Cui, Y. (2012). A simulation study for comparing three lower bounds to reliability. Paper Presented on April 17, 2012 at the AERA Division D: Measurement and Research Methodology, Section 1: Educational Measurement, Psychometrics, and Assessment, 1–25.van der Ark, L. A., van der Palm, D. W., & Sijtsma, K. (2011). A latent class approach to estimating test-score reliability. Applied Psychological Measurement, 35(5), 380–392.

Methodological studies are critical of the use of $\rho_T$. Simplifying and classifying the conclusions of existing studies are as follows.
# Conditional use: Use $\rho_T$ only when certain conditions are met.
# Opposition to use: $\rho_T$ is inferior and should not be used.Dunn, T. J., Baguley, T., & Brunsden, V. (2014). From alpha to omega: A practical solution to the pervasive problem of internal consistency estimation. British Journal of Psychology, 105(3), 399–412. Peters, G. Y. (2014). The alpha and the omega of scale reliability and validity comprehensive assessment of scale quality. The European Health Psychologist, 1(2), 56–69.Yang, Y., & Green, S. B. (2011). Coefficient alpha: A reliability coefficient for the 21st century? Journal of Psychoeducational Assessment, 29(4), 377–392.

Alternatives to Cronbach's alpha

Existing studies are practically unanimous in that they oppose the widespread practice of using $\rho_T$ unconditionally for all data. However, different opinions are given on which reliability coefficient should be used instead of $\rho_T$.

Different reliability coefficients ranked first in each simulation study comparing the accuracy of several reliability coefficients.

The majority opinion is to use SEM-based reliability coefficients as an alternative to $\rho_T$.

However, there is no consensus on which of the several SEM-based reliability coefficients (e.g., unidimensional or multidimensional models) is the best to use.

Some people suggest $\omega_H$ as an alternative, but $\omega_H$ shows information that is completely different from reliability. $\omega_H$ is a type of coefficient comparable to Revelle's $\beta$.Revelle, W. (1979). Hierarchical cluster analysis and the internal structure of tests. Multivariate Behavioral Research, 14(1), 57–74. They do not substitute, but complement reliability.

Among SEM-based reliability coefficients, multidimensional reliability coefficients are rarely used, and the most commonly used is $\rho_C$, also known as composite or congeneric reliability.

Software for SEM-based reliability coefficients

General-purpose statistical software such as SPSS and SAS include a function to calculate $\rho_T$. Users who don't know the formula of $\rho_T$ have no problem in obtaining the estimates with just a few mouse clicks.

SEM software such as AMOS, LISREL, and MPLUS does not have a function to calculate SEM-based reliability coefficients. Users need to calculate the result by inputting it to the formula. To avoid this inconvenience and possible error, even studies reporting the use of SEM rely on $\rho_T$ instead of SEM-based reliability coefficients. There are a few alternatives to automatically calculate SEM-based reliability coefficients.

# R (free): The psych package calculates various reliability coefficients.
# EQS (paid): This SEM software has a function to calculate reliability coefficients.
# RelCalc (free): Available with Microsoft Excel. $\rho_C$ can be obtained without the need for SEM software. Various multidimensional SEM reliability coefficients and various types of $\omega_H$ can be calculated based on the results of SEM software.

References

External links

Cronbach's alpha SPSS tutorial
* The free web interface and R packag
cocron
allows to statistically compare two or more dependent or independent Cronbach alpha coefficients.

{{DEFAULTSORT:Cronbach's Alpha
Comparison of assessments
Statistical reliability
Psychometrics