CTM-Bench: Task Set, Perturbations and Evaluation Harness
- Marcus Tan · Open Systems Lab
- Kofi Mensah · Lakeshore Centre for Machine Learning
- Lena Hartwell · Open Systems Lab
Published 12 June 2026
DOI 10.xxxx/ctm.ar.2026.0001 · ID CTM-AR-2026-0001
Sample record. Fictional demonstration content; the files listed are not attached.
Description
1,200 tool-use tasks, each with five perturbed variants, together with the evaluation harness, a datasheet and the annotation guidelines. Accompanies the article “CTM-Bench: A Benchmark for Tool-Use Reliability in Language Agents”.
Keywords benchmark · tool use · reliability · agents
Files (5)
| Name | Format | Size | Download |
|---|---|---|---|
| tasks.jsonl | JSON Lines | 48 MB | Download |
| perturbations.jsonl | JSON Lines | 190 MB | Download |
| harness-1.2.zip | ZIP | 3.1 MB | Download |
| datasheet.pdf | 410 KB | Download | |
| README.md | Markdown | 6 KB | Download |
Related works
Versions
Version 1.2 · 12 June 2026Latest
Fixed 14 ambiguous task statements.
10.xxxx/ctm.ar.2026.0001.v1.2
Version 1.1 · 24 April 2026
Added datasheet; replaced two source descriptions.
10.xxxx/ctm.ar.2026.0001.v1.1
Version 1.0 · 20 January 2026
Initial release with the preprint.
10.xxxx/ctm.ar.2026.0001.v1.0
How to cite
Tan, M., Mensah, K., & Hartwell, L. (2026). CTM-Bench: Task Set, Perturbations and Evaluation Harness (Version 1.2) [Dataset]. CONTINUUM Research Archive. doi:10.xxxx/ctm.ar.2026.0001