Cola2: Co-Adaptive Hierarchical Language Modeling with Continuous Semantic Compression and Balanced Discrete Generation
Cola2 is a hierarchical language generation framework that jointly studies continuous semantic compression, the balance between reconstruction and generation, and continuous-discrete co-adaptation. It dynamically segments text into semantic spans, uses a block-causal masked diffusion decoder, and decouples Flow Matching prior learning from representation learning via stop-gradient. Trained on about 210 billion tokens for ablations and 346.03 billion source tokens for a checkpoint, the authors report an unconverged model averaging 60.10 across eight tasks.
One primary source; other links are context, not independent corroboration.
- Primary sourceHugging Face Daily Papers
- Discovery linkHugging Face
The paper page is linked. Peer review and findings have not been independently checked by AINewsIQ.
Open full article view →- Original paper+45
- Fetched source page+30
- No editor link review+0
- Original non-discussion link+5
This measures the stored source trail and editorial steps. It does not certify accuracy, peer review, or independent corroboration. Popularity adds no points.
View full scoring →Read the scoring rubric →What the source says
Cola2 is a hierarchical language generation framework that jointly studies continuous semantic compression, the balance between reconstruction and generation, and continuous-discrete co-adaptation. It dynamically segments text into semantic spans, uses a block-causal masked diffusion decoder, and decouples Flow Matching prior learning from representation learning via stop-gradient. Trained on about 210 billion tokens for ablations and 346.03 billion source tokens for a checkpoint, the authors report an unconverged model averaging 60.10 across eight tasks.
Key findings from the stored brief
- Cola2 dynamically segments text into semantic spans and controls compression rate via auxiliary-loss-free negative feedback.
- The unconverged checkpoint reaches a 60.10 average across eight tasks after 346.03 billion source tokens.
- The model surpasses Llama-3-8B on HellaSwag and StoryCloze, despite the reported pretraining budget being over 43 times larger.
- Flow Matching prior learning is decoupled from representation learning by stop-gradient.
Source claims not independently checked; the checkpoint is unconverged and the comparison to Llama-3-8B depends on the authors' stated pretraining budgets.
This is a source-based abstract, not independent validation of the source's claims.
Evidence & sources
These links show where the signal came from and what the source page linked to. A linked page is context, not independent corroboration.
Related archive coverage
These relationships record a citation or topic overlap, not confirmation of the finding.
The paper page is linked. Peer review and findings have not been independently checked by AINewsIQ.
Timeline & updates
- Signal dated by source or discovery feed
This timestamp places the signal in the radar. It may be the discovery post date rather than the primary page's first publication date.
- Shared brief stored
AINewsIQ prepared one abstract from fetched source page.
A publication revision history is not stored for this signal. New engagement snapshots do not mean the source itself changed.
Why this may matter
The work explores whether continuous latent representations can serve as an effective intermediate layer for language generation. If the reported scaling holds, it suggests an alternative to purely discrete autoregressive modeling.
Limits and open questions
Source claims not independently checked; the checkpoint is unconverged and the comparison to Llama-3-8B depends on the authors' stated pretraining budgets.
Organization-specific financial, security, and operational effects require the organization's own context and review.
Discussion
Ask about the source, add context, or point readers to a relevant follow-up.
Discussion
Comments are reviewed before publication. Discuss the evidence, not the person.
No published comments yet.
Reader discussion opens with accounts. You can read the original source and published comments now.