2 results for “audit” in keywordSearch & filters ↓
Technical report documenting how 112 agent papers report variance, compute, prompts and judging procedures, with the full coding sheet. Supports the survey “A Survey of Evaluation Practices for Language-Model Agents”.
Many machine-learning venues now require authors to complete a reproducibility checklist. We audited 240 papers that ticked the item “code is provided” and checked whether the linked repositories existed, ran and reproduced the headline table. 87% had a reachable repository, 58% contained instructions sufficient to run training or evaluation, and 31% reproduced the headline result within the reported variance. Compliance was higher for papers reporting variance across seeds. We recommend checks that can be automated at submission time.
