The likelihood of social choice violations in rank sum scoring: algorithms and evidence from NCAA cross country running

Recent contributions have used combinatorial algorithms to determine the likelihood of particular social choice violations in rank sum scoring. Given the broad importance of rank sum scoring (e.g., in non-parametric statistical testing, sporting competition, and mathematical competition), it is important to establish the level of ambiguity generated by this aggregation rule. Combinatorial likelihoods are naïve, however, in that they assume each possible outcome sequence for an event to be equally likely. We develop a computational algorithm to extend upon previous combinatorial results as to the likelihood of a violation of transitivity or independence in rank sum scoring. We use a similar computational scoring approach to analyze the empirically-observed likelihood of each such violation across fourteen NCAA Cross Country Championships. Within the data, rank sum scoring fails to specify a robust winning team (i.e., one that also rank sum wins against each possible subset of opponents) in 4 of 14 cases. Overall, we find that empirical likelihoods of social choice violations are consistently (significantly) overestimated by combinatorial expectations. In the NCAA data, we find correlated ability (quality) levels within team (group) and discuss this as a cause of lower empirical likelihoods. Combinatorial analysis proves reliable in predicting the order of empirical likelihoods across violation type and event setting.