Skip to main content
Submit

Search AEX CONTINUUM

Titles, abstracts, authors, keywords and DOIs across all five collections.

49 resultsSearch & filters ↓

PreprintsTechnical ReportNot peer reviewed

Position Bias in Long-Context Retrieval: A Replication Study

Elias Wagner, Tomás Ibarra

Version 1 · 19 August 2026 · 10.xxxx/ctm.pp.2026.0003

Several studies report that language models use information in the middle of a long context less reliably than information at the start or end. We replicate these findings across eleven models and four context lengths using a controlled retrieval task. The effect is reproducible but smaller than originally reported, and largely disappears when the task format is held fixed across lengths. We release the harness and the full per-position results.

Abstract PDF 2,210 7401 cited
PublishingResearch ArticlePeer reviewed

Statistical Power in Agent Benchmarks: How Many Runs Are Enough?

Hana Kovács, Kofi Mensah, Carlos Medina

Version 2 · 3 reviewer reports · 14 August 2026 · 10.xxxx/ctm.pb.2026.0008

Most agent benchmarks report one run per task, yet agents are stochastic. We analyse run-to-run variance on four public agent benchmarks and derive the number of repeated runs needed to detect a given difference between two systems at conventional power. For differences smaller than five points, single-run comparisons have less than 40% power on all four benchmarks. We provide a sequential testing procedure that stops early when the evidence is clear, and show that it halves the cost of repeated evaluation while keeping the false-positive rate below 5%.

Abstract PDF 2,740 1,1033 cited
PreprintsTechnical ReportNot peer reviewed

Reproducibility-Checklist Compliance in Machine-Learning Papers: A technical report on what authors state and what they release

Ingrid Novak, Kofi Mensah

Version 1 · 27 July 2026 · 10.xxxx/ctm.pp.2026.0002

Many machine-learning venues now require authors to complete a reproducibility checklist. We audited 240 papers that ticked the item “code is provided” and checked whether the linked repositories existed, ran and reproduced the headline table. 87% had a reachable repository, 58% contained instructions sufficient to run training or evaluation, and 31% reproduced the headline result within the reported variance. Compliance was higher for papers reporting variance across seeds. We recommend checks that can be automated at submission time.

Abstract PDF 2,455 9314 cited
PublishingResearch ArticlePeer reviewed

Refusal Calibration in Instruction-Tuned Language Models: When should a model decline, and how well do current models know?

Nadia Haddad, Hana Kovács

Version 2 · 3 reviewer reports · 9 July 2026 · 10.xxxx/ctm.pb.2026.0007

Instruction-tuned models decline requests for many reasons, but little is known about how well their refusals track the actual risk of a request. We build a labelled set of 6,400 requests spanning benign, borderline and harmful categories and measure calibration of refusal decisions across eight open models. Refusal rates correlate only weakly with annotated risk for borderline requests, and over-refusal is concentrated in a handful of phrasing patterns rather than topics. A light-weight calibration step that uses the model's own confidence reduces over-refusal by 31% at no measurable cost to harmful-request refusal. We release the labelled set and the annotation guidelines.

Abstract PDF 3,180 1,2604 cited
ProceedingsFull paperBest paper

Retry Budgets for Tool-Using Agents: When Is Trying Again Worth It?

Aisha Rahman, Lena Hartwell

RTUA 2026 · pp. 1–9 · 3 July 2026 · 10.xxxx/ctm.pr.2026.0001

Agents that retry failed tool calls recover from transient errors but can also compound mistakes and waste resources. We analyse 18,000 recorded agent runs and show that retries help for network and rate-limit errors but hurt for malformed arguments. We propose a retry budget that depends on the error class and show that it improves success by 6 points at equal cost.

Abstract PDF 1,420 5602 cited
ProceedingsFull paper

Sandboxing Agent Tool Calls with Capability Tokens

Marcus Tan, Jonas Berg

RTUA 2026 · pp. 10–18 · 3 July 2026 · 10.xxxx/ctm.pr.2026.0002

Tool-using agents are typically granted broad credentials, so a single injected instruction can cause large harm. We describe a capability-token scheme in which every tool call carries a narrowly scoped, short-lived token issued by a policy layer outside the model. A prototype blocks all 41 attack scripts in our test suite with under 4 ms of added latency.

Abstract PDF 1,190 4301 cited