A source-linked abstract prepared once when this article was collected.
NEWS ABSTRACT10/22/2026 · one shared analysis
The paper proposes PACE, a quantizer for recurrent states in linear-attention-based language model layers. Because the state is materialized at chunk granularity rather than every token, rounding can be adapted to chunk size, allowing lower-precision (4-bit) states and aligning with KV buffering during decoding. Authors report that a 4-bit recurrent state with PACE achieves performance competitive with INT8 quantizers, with an 8-token buffer giving the headline results on a 4B model (average accuracy 79.1 and 79.6 against 80.8 full precision, per the author comment). Buffer length trades off accuracy against stored higher-precision bits.
Why it matters
Lower-precision recurrent states could reduce memory per sequence during decoding in linear-attention models, potentially enabling more concurrent sequences on accelerators.
Read with care
Source claims not independently checked; accuracy figures come from an author comment, and CPU/single-sequence speedups were not measured.
quantizationlinear-attentionrecurrent-statellm-inferencekv-cache
Discussion
Comments are reviewed before publication. Discuss the evidence, not the person.
No published comments yet.
Reader discussion opens with accounts. You can read the original source and published comments now.