4 results for “software” in keywordSearch & filters ↓
We attempt to build and run 400 research repositories released between 2019 and 2021. Only 38% still run without modification; the most common causes are unpinned dependencies and removed external data. Archived releases with lock files survive at more than twice the rate.
Many machine-learning venues now require authors to complete a reproducibility checklist. We audited 240 papers that ticked the item “code is provided” and checked whether the linked repositories existed, ran and reproduced the headline table. 87% had a reachable repository, 58% contained instructions sufficient to run training or evaluation, and 31% reproduced the headline result within the reported variance. Compliance was higher for papers reporting variance across seeds. We recommend checks that can be automated at submission time.
Python library for measuring and improving calibration of generated text, including reliability diagrams, expected calibration error variants and post-hoc recalibration methods.
Source code for the two-stage tool selector and the evaluation harness used in “Retrieval-Augmented Tool Selection for Long-Horizon Language Agents”. Includes configuration files to reproduce every table.
