Kendall's ''W'' (also known as Kendall's coefficient of concordance) is a
non-parametric statistic
Nonparametric statistics is the branch of statistics that is not based solely on parametrized families of probability distributions (common examples of parameters are the mean and variance). Nonparametric statistics is based on either being dist ...
for
rank correlation
In statistics, a rank correlation is any of several statistics that measure an ordinal association—the relationship between rankings of different ordinal variables or different rankings of the same variable, where a "ranking" is the assignment o ...
. It is a normalization of the statistic of the
Friedman test
The Friedman test is a non-parametric statistical test developed by Milton Friedman. Similar to the parametric repeated measures ANOVA, it is used to detect differences in treatments across multiple test attempts. The procedure involves ranking ...
, and can be used for assessing agreement among raters and in particular
inter-rater reliability
In statistics, inter-rater reliability (also called by various similar names, such as inter-rater agreement, inter-rater concordance, inter-observer reliability, inter-coder reliability, and so on) is the degree of agreement among independent obse ...
. Kendall's ''W'' ranges from 0 (no agreement) to 1 (complete agreement).
Suppose, for instance, that a number of people have been asked to rank a list of political concerns, from the most important to the least important. Kendall's ''W'' can be calculated from these data. If the test statistic ''W'' is 1, then all the survey respondents have been unanimous, and each respondent has assigned the same order to the list of concerns. If ''W'' is 0, then there is no overall trend of agreement among the respondents, and their responses may be regarded as essentially random. Intermediate values of ''W'' indicate a greater or lesser degree of unanimity among the various responses.
While tests using the standard
Pearson correlation coefficient
In statistics, the Pearson correlation coefficient (PCC, pronounced ) ― also known as Pearson's ''r'', the Pearson product-moment correlation coefficient (PPMCC), the bivariate correlation, or colloquially simply as the correlation coefficient ...
assume
normally distributed values and compare two sequences of outcomes simultaneously, Kendall's ''W'' makes no assumptions regarding the nature of the
probability distribution
In probability theory and statistics, a probability distribution is the mathematical function that gives the probabilities of occurrence of different possible outcomes for an experiment. It is a mathematical description of a random phenomenon i ...
and can handle any number of distinct outcomes.
Steps of Kendall's W
Suppose that object ''i'' is given the rank ''r
i,j'' by judge number ''j'', where there are in total ''n'' objects and ''m'' judges. Then the total rank given to object ''i'' is
:
and the mean value of these total ranks is
:
The sum of squared deviations, ''S'', is defined as
:
and then Kendall's ''W'' is defined as
:
If the test statistic ''W'' is 1, then all the judges or survey respondents have been unanimous, and each judge or respondent has assigned the same order to the list of objects or concerns. If ''W'' is 0, then there is no overall trend of agreement among the respondents, and their responses may be regarded as essentially random. Intermediate values of ''W'' indicate a greater or lesser degree of unanimity among the various judges or respondents.
Kendall and Gibbons (1990) also show ''W'' is linearly related to the mean value of the
Spearman's rank correlation coefficient
In statistics, Spearman's rank correlation coefficient or Spearman's ''ρ'', named after Charles Spearman and often denoted by the Greek letter \rho (rho) or as r_s, is a nonparametric measure of rank correlation ( statistical dependence between ...
s between all
possible pairs of rankings between judges
:
Incomplete Blocks
When the judges evaluate only some subset of the ''n'' objects, and when the correspondent block design is a
(n, m, r, p, λ)-design (note the different notation). In other words, when
# each judge ranks the same number ''p'' of objects for some
,
# every object is ranked exactly the same total number ''r'' of times,
# and each pair of objects is presented together to some judge a total of exactly λ times,
, a constant for all pairs.
Then Kendall's ''W'' is defined as
:
If
and
so that each judge ranks all ''n'' objects, the formula above is equivalent to the original one.
Correction for Ties
When tied values occur, they are each given the average of the ranks that would have been given had no ties occurred. For example, the data set has values of 80 tied for 4th, 5th, and 6th place; since the mean of = 5, ranks would be assigned to the raw data values as follows: .
The effect of ties is to reduce the value of ''W''; however, this effect is small unless there are a large number of ties. To correct for ties, assign ranks to tied values as above and compute the correction factors
:
where ''t
i'' is the number of tied ranks in the ''i''th group of tied ranks, (where a group is a set of values having constant (tied) rank,) and ''g
j'' is the number of groups of ties in the set of ranks (ranging from 1 to ''n'') for judge ''j''. Thus, ''T
j'' is the correction factor required for the set of ranks for judge ''j'', i.e. the ''j''th set of ranks. Note that if there are no tied ranks for judge ''j'', ''T
j'' equals 0.
With the correction for ties, the formula for ''W'' becomes
:
where ''R
i'' is the sum of the ranks for object ''i'', and
is the sum of the values of ''T
j'' over all ''m'' sets of ranks.
Steps of Weighted Kendall's W
In some cases, the importance of the raters (experts) might not be the same as each other. In this case, the Weighted Kendall's W should be used.
Suppose that object
is given the rank
by judge number
, where there are in total
objects and
judges. Also, the weight of judge
is shown by
(in real-world situation, the importance of each rater can be different). Indeed, the weight of judges is
. Then, the total rank given to object
is
:
and the mean value of these total ranks is,
:
The sum of squared deviations,
, is defined as,
:
and then Weighted Kendall's ''W'' is defined as,
:
The above formula is suitable when we do not have any tie rank.
Correction for Ties
In case of tie rank, we need to consider it in the above formula. To correct for ties, we should compute the correction factors,
:
where
represents the number of tie ranks in judge
for object
.
shows the total number of ties in judge
.
With the correction for ties, the formula for Weighted Kendall's ''W'' becomes,
:
If the weights of the raters are equal (the distribution of the weights is uniform), the value of Weighted Kendall's ''W'' and Kendall's ''W'' are equal.
Significance Tests
In the case of complete ranks, a commonly used significance test for ''W'' against a null hypothesis of no agreement (i.e. random rankings) is given by Kendall and Gibbons (1990)
:
Where the test statistic takes a chi-squared distribution with
degrees of freedom.
In the case of incomplete rankings (see above), this becomes
:
Where again, there are
degrees of freedom.
Legendre compared via simulation the power of the chi-square and
permutation test
A permutation test (also called re-randomization test) is an exact statistical hypothesis test making use of the proof by contradiction.
A permutation test involves two or more samples. The null hypothesis is that all samples come from the same dis ...
ing approaches to determining significance for Kendall's ''W''. Results indicated the chi-square method was overly conservative compared to a permutation test when
. Marozzi extended this by also considering the ''F'' test, as proposed in the original publication introducing the ''W'' statistic by Kendall & Babington Smith (1939):
:
Where the test statistic follows an F distribution with
and
degrees of freedom. Marozzi found the ''F'' test performs approximately as well as the permutation test method, and may be preferred to when
is small, as it is computationally simpler.
Software
Kendall's W and Weighted Kendall's W are implemented in
MATLAB
MATLAB (an abbreviation of "MATrix LABoratory") is a proprietary multi-paradigm programming language and numeric computing environment developed by MathWorks. MATLAB allows matrix manipulations, plotting of functions and data, implementation ...
,
SPSS
SPSS Statistics is a statistical software suite developed by IBM for data management, advanced analytics, multivariate analysis, business intelligence, and criminal investigation. Long produced by SPSS Inc., it was acquired by IBM in 2009. C ...
,
R,
and other statistical software packages.
See also
*
Maurice Kendall
Sir Maurice George Kendall, FBA (6 September 1907 – 29 March 1983) was a prominent British statistician. The Kendall tau rank correlation is named after him.
Education and early life
Maurice Kendall was born in Kettering, Northampton ...
*
Kendall's tau
In statistics, the Kendall rank correlation coefficient, commonly referred to as Kendall's τ coefficient (after the Greek letter τ, tau), is a statistic used to measure the ordinal association between two measured quantities. A τ test is a non ...
*
Spearman's rank correlation coefficient
In statistics, Spearman's rank correlation coefficient or Spearman's ''ρ'', named after Charles Spearman and often denoted by the Greek letter \rho (rho) or as r_s, is a nonparametric measure of rank correlation ( statistical dependence between ...
*
Friedman test
The Friedman test is a non-parametric statistical test developed by Milton Friedman. Similar to the parametric repeated measures ANOVA, it is used to detect differences in treatments across multiple test attempts. The procedure involves ranking ...
Notes
References
*
*Kendall, M. G., & Gibbons, J. D. (1990). Rank correlation methods. New York, NY : Oxford University Press.
*Corder, G.W., Foreman, D.I. (2009). ''Nonparametric Statistics for Non-Statisticians: A Step-by-Step Approach'' Wiley,
*
Dodge, Y. (2003). ''The Oxford Dictionary of Statistical Terms'', OUP.
*Legendre, P (2005) Species Associations: The Kendall Coefficient of Concordance Revisited. ''Journal of Agricultural, Biological and Environmental Statistics'', 10(2), 226–245
*
*{{cite book , title= Nonparametric Statistical Inference , last1= Gibbons , first1= Jean Dickinson , last2= Chakraborti , first2= Subhabrata , year= 2003 , publisher= Marcel Dekker , location= New York , isbn= 978-0-8247-4052-8, pages= 476–482 , edition= 4th
Inter-rater reliability
Nonparametric statistics
Rankings