Version 1 · 0 reviewer reports · 5 October 2026 · 10.xxxx/ctm.pb.2026.0009
Users frequently mix languages within a single query, but retrieval benchmarks rarely reflect this. We construct code-mixed query sets for two language pairs and evaluate dense retrievers under different tokenisation and fine-tuning strategies. Adapting the query encoder with synthetic code-mixed data improves recall@20 by 9 points over a multilingual baseline, while transliteration normalisation helps for romanised queries but hurts for native-script queries.
Version 1 · 19 August 2026 · 10.xxxx/ctm.pp.2026.0003
Several studies report that language models use information in the middle of a long context less reliably than information at the start or end. We replicate these findings across eleven models and four context lengths using a controlled retrieval task. The effect is reproducible but smaller than originally reported, and largely disappears when the task format is held fixed across lengths. We release the harness and the full per-position results.
Version 2 (updated) · 5 June 2026 · 10.xxxx/ctm.pp.2026.0001
Retrieval-augmented generation grounds answers in retrieved documents, yet models remain over-confident when the retrieved evidence is irrelevant or contradictory. We propose a lightweight confidence estimator that combines the generator's token probabilities with a retrieval-agreement score, and calibrate it with temperature scaling on a small validation set. On three question-answering datasets the estimator lowers expected calibration error from 0.19 to 0.06 and enables selective answering: abstaining on the 20% least confident questions raises accuracy from 71% to 84%. The method needs no model retraining.
Version 0.4.0 · Apache-2.0 · 5 June 2026 · 10.xxxx/ctm.ar.2026.0007
Python library for measuring and improving calibration of generated text, including reliability diagrams, expected calibration error variants and post-hoc recalibration methods.
PhD thesis · 224 pages · ISBN 978-0-00-100005-0 · 15 May 2026 · 10.xxxx/ctm.bk.2026.0005
Doctoral thesis on estimating and communicating uncertainty for text generation. It develops calibration methods for retrieval-augmented systems, studies how users interpret expressed confidence, and proposes evaluation protocols for uncertainty in open-ended outputs.
Version 2 · 4 reviewer reports · 27 February 2026 · 10.xxxx/ctm.pb.2026.0001
Language agents that act through external tools are limited by the size of their prompt: listing hundreds of tool descriptions crowds out the task and degrades accuracy. We study tool selection as a retrieval problem and propose a two-stage selector that (i) retrieves a short list of candidate tools using an embedding of the evolving task state and (ii) lets the model re-rank candidates with a single constrained call. Across a synthetic suite of 4,000 tools and three long-horizon task families, the selector keeps task success within 2.1 points of an oracle that sees only the correct tools, while reducing prompt length by 87%. Ablations show that conditioning retrieval on the task state, rather than on the initial instruction alone, accounts for most of the gain. We discuss failure cases, including tools with overlapping descriptions, and release the evaluation harness.
Version 0.9.1 · MIT · 27 February 2026 · 10.xxxx/ctm.ar.2026.0002
Source code for the two-stage tool selector and the evaluation harness used in “Retrieval-Augmented Tool Selection for Long-Horizon Language Agents”. Includes configuration files to reproduce every table.
