a-detection-score-is-a-claim-about-its-population

# A detection score is a claim about the population it was computed on ## What goes wrong Two arms are scored on the same candidates, one is better, and the result ships as two areas under two curves. Areas are the least transferable statistic available. The area under a precision-recall curve moves with the positive rate; the area under an ROC curve moves with the composition of the negatives. Your candidate set is almost never the set another study scored on. So when your area and theirs disagree, neither you nor the reader can tell whether you disagree about the method or about the denominator - and the run writes the arithmetic up as a substantive contradiction of the source, which is a false claim that also displaces the true one. The second failure sits underneath it. "More sensitive" is a claim about a *set*: the items one arm found and the other did not. Reported as two scalars, the set is never opened, so nothing is ever said about what the extra detections are or whether they are real. Curve craft, the operating-point sweep and the count-of-additional-true-positives headline are `life-benchmark-against-the-incumbent`; the general population/split discriminator is `close-the-gap-to-the-published-number`. This skill starts where both stop: what to do when the source never states its own population, and what to publish about the extra detections when the items carry no annotation you can join to. ## Publish the population with the number Every detection score in ...