49 resultsSearch & filters ↓
We measure energy per generated token for eleven small language models on three mobile systems-on-chip. Memory traffic, not arithmetic, explains most of the variation, and the best configuration per device differs by up to 2.4×.
Priya Venkataraman, Anders Holm
Personalising a model on a user's device requires small, fast-to-train updates. We study rank, placement and quantisation of low-rank adapters on a phone-class model and find that rank 4 on attention projections gives most of the gain at under 2 MB per user.
A poster summarising latency, memory and quality for twenty sub-billion-parameter models on a laptop CPU, a single-board computer and a mobile phone.
A practical and rigorous guide to measuring what agents actually do. The book develops a measurement framework for tool-using language agents, covers reliability, cost and safety metrics, and shows how to design benchmarks, report variance and avoid the most common evaluation mistakes. Worked examples and code accompany every chapter.
Adapting a pretrained model to a low-resource language often begins with extending the tokenizer, yet the choice of extension method is rarely compared. We evaluate five vocabulary-extension strategies on six languages and find that fertility reduction alone does not predict downstream accuracy. Strategies that initialise new embeddings from semantically related tokens consistently outperform random initialisation, at the same training budget.
Checkpoints and model card for the 1.1B-parameter mixed-precision student model described in the SEODML 2026 paper, with an evaluation script and measured on-device latency.
Debate about autonomous AI agents often centres on what they can do. This perspective argues that governance should instead start from who is accountable when an agent acts. Drawing on analogies from aviation, finance and medical devices, we propose three design requirements for deployed agents — attributable identity, bounded authority and reconstructable action logs — and discuss how each could be audited. We argue that these requirements are technically modest, can be adopted before agents become more capable, and would make later regulation easier to apply.
Daniel Okafor, Yuki Saito, Priya Venkataraman
Running assistant-grade language models on phones requires both small models and low-precision arithmetic, yet distillation and quantisation are usually applied one after the other. We train a 1.3B-parameter student with distillation losses computed through simulated 4-bit quantisation, so that the student learns to be robust to its own rounding error. Compared with distilling first and quantising afterwards, the joint procedure recovers 41% of the accuracy lost to 4-bit quantisation, reaches 94% of the full-precision student on a suite of assistant tasks, and runs 3.1× faster on a mid-range mobile processor.
