Sample record. Fictional demonstration content.
Abstract
Long conversations on a phone are limited by the size of the key-value cache. We compare eviction, quantisation and merging strategies under hard memory budgets and find that a hybrid of merging and 8-bit storage keeps perplexity within 3% at one quarter of the memory.
Keywords KV cache · memory · long context · inference
Full text
CTM-PR-2026-0007.pdf
How to cite
Desai, R. & Holm, A. (2026). KV-Cache Compression Under Strict Memory Budgets. In Proceedings of the Symposium on Efficient and On-Device Machine Learning (SEODML 2026) (pp. 11–20). CONTINUUM Proceedings. doi:10.xxxx/ctm.pr.2026.0007