Reviewed by the event organisers
Permanent, citable homes for conference papers.
Open-access proceedings for conferences, workshops and symposia. Organisers deposit their accepted papers; every paper and every volume receives a DOI.
2 resultsSearch & filters ↓
We measure energy per generated token for eleven small language models on three mobile systems-on-chip. Memory traffic, not arithmetic, explains most of the variation, and the best configuration per device differs by up to 2.4×.
Priya Venkataraman, Anders Holm
Personalising a model on a user's device requires small, fast-to-train updates. We study rank, placement and quantisation of low-rank adapters on a phone-class model and find that rank 4 on attention projections gives most of the gain at under 2 MB per user.
