IQuest-Q1: open-weight 320B MoE report targeting agentic coding

The IQuest-Q1 technical report describes a sparse Mixture-of-Experts agentic foundation model with 320B total parameters, 15B activated. Development uses three stages: stable pre-training with Stable SplitNorm, Muon, and muP; fine-grained mid-training with progressive context extension; and multi-scaffold post-training combining SFT, domain-specialized RL, and MOPD. Model merging combines experts and checkpoints. The authors state the model shows capabilities in coding and general tasks and is released as open source.

Key takeaways

• The model has 320B total parameters with 15B activated per token via sparse Mixture-of-Experts.
• Pre-training combines Stable SplitNorm, Muon, and muP with a two-stage curriculum increasing coding data.
• Post-training combines SFT, domain-specialized RL, and MOPD across multiple agent scaffolds.
• Model merging combines experts and checkpoints from different stages while mitigating parameter update conflicts.

Why it matters

It documents a large open-weight MoE model aimed at agentic coding, with a detailed training recipe that may inform community work on stable training and multi-scaffold post-training.

Read with care

Source claims not independently checked; the abstract gives no benchmark numbers, so performance claims are unquantified here.