This paper explores possible biases in open peer-review using data from the
English superior courts. Exploiting the random timing of on-the-job interaction
between reviewers and reviewees, we find evidence that reviewers are reluctant
to reverse the judgments of reviewees with whom they are about to interact, and
that this effect is stronger when reviewer and reviewee share the same rank. The
average bias is substantial: the proportion of reviewer affirmances is 30% points
higher in the group where reviewers know they will soon work with their reviewee,
relative to groups where such interaction is absent. Our results suggest reforms
for the judicial listing process, and caution against recent trends in performance
appraisal techniques and scientific publishing