Reproducibility-Checklist Compliance in Machine-Learning Papers: A technical report on what authors state and what they release
Many machine-learning venues now require authors to complete a reproducibility checklist. We audited 240 papers that ticked the item “code is provided” and checked whether the linked repositories existed, ran and reproduced the headline table. 87% had a reachable repository, 58% contained instructions sufficient to run training or evaluation, and 31% reproduced the headline result within the reported variance. Compliance was higher for papers reporting variance across seeds. We recommend checks that can be automated at submission time.