Skip to main content
Submit

Screened, not peer reviewed

CONTINUUM Preprints

Share your work the day it is ready.

Rapid, free dissemination of research before or alongside peer review. Preprints are screened for scope and compliance, posted with a DOI, and can be versioned as the work develops.

  • ✓ Posted within ~2 working days
  • ✓ DOI and version history
  • ✓ Request peer review at any time
Post a preprintGuidelines
Preprints
9
Authors
16
Downloads
6,199

2 resultsSearch & filters ↓

Short CommunicationNot peer reviewed

Speculative Decoding Under Memory Constraints

Rahul Desai, Yuki Saito

Version 1 · 5 November 2026 · 10.xxxx/ctm.pp.2026.0009

Speculative decoding accelerates generation by letting a small draft model propose tokens that a larger model verifies, but the draft model competes with the target model for scarce memory on edge devices. We characterise the trade-off between draft-model size, speed-up and memory footprint, and propose a draft that shares the target model's embedding table and lower layers. The shared draft reduces extra memory from 11% to 3% of the target model and retains 88% of the speed-up of an independent draft.

Abstract PDF 437 1210 cited
Short CommunicationNot peer reviewed

Cheap Reward Models for Small-Scale RLHF Experiments

Jonas Berg, Anders Holm

Version 1 · 29 October 2026 · 10.xxxx/ctm.pp.2026.0008

Preference-based fine-tuning is expensive to study because reward models are large. We show that a reward model distilled to 1% of the size of the original recovers 94% of its ranking accuracy on held-out preference pairs, and that policies trained against the small model transfer to the large one with a modest loss. The result lowers the cost of reward-modelling experiments for groups with limited compute.

Abstract PDF 1,480 5120 cited