A Taxonomy of Evaluation Leakage in Language-Model Benchmarks
Nadia Haddad, Ingrid Novak, Kofi Mensah
Benchmark scores can be inflated when test material reaches the training data or when evaluation choices quietly favour a model. We review 140 papers and incident reports and organise the mechanisms of evaluation leakage into a taxonomy with four top-level classes and eleven sub-classes. For each class we summarise how it can be detected, how often it has been reported, and what practices reduce its likelihood.