Sample record. Fictional demonstration content.
Abstract
Four-bit quantisation of small language models often hurts instruction following. We distil from a full-precision teacher while letting the student's layers use different bit widths chosen by a sensitivity analysis. The resulting 1.1B-parameter model fits in 700 MB and retains 96% of teacher accuracy on our instruction-following suite.
Keywords quantisation · distillation · small language models
Full text
CTM-PR-2026-0006.pdf
How to cite
Saito, Y. & Okafor, D. (2026). Mixed-Precision Distillation for 4-bit Phone-Class Models. In Proceedings of the Symposium on Efficient and On-Device Machine Learning (SEODML 2026) (pp. 1–10). CONTINUUM Proceedings. doi:10.xxxx/ctm.pr.2026.0006