Skip to main content

Search AEX CONTINUUM

Titles, abstracts, authors, keywords and DOIs across all five collections.

4 results for “agents” in keywordSearch & filters ↓

PublishingPerspectivePeer reviewed

Governing Autonomous Software Agents: Accountability Before Capability

Sofia Lindqvist

Version 3 · 3 reviewer reports · 4 September 2026 · 10.xxxx/ctm.pb.2026.0005

Debate about autonomous AI agents often centres on what they can do. This perspective argues that governance should instead start from who is accountable when an agent acts. Drawing on analogies from aviation, finance and medical devices, we propose three design requirements for deployed agents — attributable identity, bounded authority and reconstructable action logs — and discuss how each could be audited. We argue that these requirements are technically modest, can be adopted before agents become more capable, and would make later regulation easier to apply.

Abstract PDF 1,893 6401 cited
PublishingReview ArticlePeer reviewed

A Survey of Evaluation Practices for Language-Model Agents: What is measured, how it is measured, and what is missing

Kofi Mensah, Ingrid Novak, Lena Hartwell

Version 2 · 4 reviewer reports · 22 May 2026 · 10.xxxx/ctm.pb.2026.0003

As language-model agents move from demonstrations to deployments, the way they are evaluated determines which claims can be trusted. We review how agents are evaluated in the literature, organising practice along four axes: the task environment, the success criterion, the treatment of variance, and the reporting of cost. We identify recurring weaknesses: success is often judged by another model without validation, runs are rarely repeated, cost and latency are reported in under a third of studies, and environment versions are seldom pinned. We close with a checklist for evaluation reports and open problems, including measurement of safety-relevant side effects.

Abstract PDF 6,210 2,87011 cited
PublishingResearch ArticlePeer reviewed

Retrieval-Augmented Tool Selection for Long-Horizon Language Agents: Choosing from thousands of tools without placing them all in the prompt

Lena Hartwell, Tomás Ibarra, Marcus Tan

Version 2 · 4 reviewer reports · 27 February 2026 · 10.xxxx/ctm.pb.2026.0001

Language agents that act through external tools are limited by the size of their prompt: listing hundreds of tool descriptions crowds out the task and degrades accuracy. We study tool selection as a retrieval problem and propose a two-stage selector that (i) retrieves a short list of candidate tools using an embedding of the evolving task state and (ii) lets the model re-rank candidates with a single constrained call. Across a synthetic suite of 4,000 tools and three long-horizon task families, the selector keeps task success within 2.1 points of an oracle that sees only the correct tools, while reducing prompt length by 87%. Ablations show that conditioning retrieval on the task state, rather than on the initial instruction alone, accounts for most of the gain. We discuss failure cases, including tools with overlapping descriptions, and release the evaluation harness.

Abstract PDF 4,821 1,9367 cited
PublishingDataset & BenchmarkPeer reviewed

CTM-Bench: A Benchmark for Tool-Use Reliability in Language Agents

Kofi Mensah, Marcus Tan, Ingrid Novak

Version 2 · 3 reviewer reports · 12 June 2026 · 10.xxxx/ctm.pb.2026.0004

Existing agent benchmarks measure whether a task is solved once, not whether it is solved reliably. CTM-Bench contains 1,200 tool-use tasks, each with five perturbed variants (rephrased instructions, reordered tool lists, injected API errors, latency spikes and truncated outputs) and reports pass^k, the probability that an agent succeeds on all k independent attempts. We document collection, annotation and licensing following datasheet guidelines, evaluate six open models, and find that pass^1 overstates reliability: the best model falls from 71% pass^1 to 38% pass^5. The benchmark, scoring code and the datasheet are released under CC BY 4.0.

Abstract PDF 5,330 2,4129 cited