vLLM reports near-DeepInfer decode throughput for DeepSeek-V4.1-Flash on GB300
vLLM reports serving DeepSeek-V4.1-Flash on one NVIDIA GB300 NVL72 rack using P/D disaggregation and Wide-EP, reaching 954K tokens/s per GPU at a P90 of 196 tokens/s per user on SemiAnalysis AgentX. The post says vLLM's decode throughput is within 6% of DeepInfer, DeepSeek's own engine, across batch sizes 8–256 under DeepSeek's decode configuration, and that its prefill tracks DeepInfer's published curve. The authors note the prefill setups differ, so the gap is not read as an engine-to-engine speedup. The post also describes model profiling and optimizations such as extending SWA bounded replay to layer 20 and using 2 MB pages for shared Engram host tables.
One primary source; other links are context, not independent corroboration.
- Primary sourceX discovery · automated brief
- Discovery linkX
- Linked in sourcefirst post
- Linked in sourcesource
- Linked in sourceDeepInfer
- Linked in sourceDeepSeek-V4.1-Flash Technical Report
- Linked in source#61339
- Linked in sourceDeepSeek's inference system overview
- Linked in sourcelarge-scale serving post
- Linked in source#62009
- Linked in source#61395
- Linked in source#61868
- Linked in source#61869
- Linked in sourceprevious post
Grok Bot submitted the link. AINewsIQ fetched the primary source and prepared a source-based brief automatically. An editor has not reviewed the story; social attention and the brief do not independently verify its claims.
Open full article view →- Linked article+30
- Fetched source page+30
- No editor link review+0
- Original non-discussion link+5
This measures the stored source trail and editorial steps. It does not certify accuracy, peer review, or independent corroboration. Popularity adds no points.
View full scoring →Read the scoring rubric →What the source says
vLLM reports serving DeepSeek-V4.1-Flash on one NVIDIA GB300 NVL72 rack using P/D disaggregation and Wide-EP, reaching 954K tokens/s per GPU at a P90 of 196 tokens/s per user on SemiAnalysis AgentX. The post says vLLM's decode throughput is within 6% of DeepInfer, DeepSeek's own engine, across batch sizes 8–256 under DeepSeek's decode configuration, and that its prefill tracks DeepInfer's published curve. The authors note the prefill setups differ, so the gap is not read as an engine-to-engine speedup. The post also describes model profiling and optimizations such as extending SWA bounded replay to layer 20 and using 2 MB pages for shared Engram host tables.
Key findings from the stored brief
- On one GB300 NVL72 rack, vLLM reports 954K tokens/s per GPU at P90 196 tokens/s per user on AgentX.
- Under DeepSeek's decode configuration, vLLM on GB200 is within 6% of DeepInfer across batch sizes 8 to 256.
- Prefill setups differ between the two engines, so the post does not treat the prefill gap as an engine speedup.
- Extending SWA bounded replay to layer 20 improved prefill throughput by 19.9% at 32K new tokens, per the post.
Source claims not independently checked; benchmarks use the authors' own workloads, and DeepInfer numbers were read off published charts.
This is a source-based abstract, not independent validation of the source's claims.
Evidence & sources
These links show where the signal came from and what the source page linked to. A linked page is context, not independent corroboration.
These are Grok Bot discovery notes. AINewsIQ checked the primary page for its shared brief; the bot's rationale, linked posts, authors, and stances have not been independently verified.
Inferact and the vLLM team (Oct 23) report that P/D disaggregation plus Wide-EP on one GB300 NVL72 rack serves DeepSeek-V4.1-Flash at 954K tokens/s per GPU at P90 196 tok/s/user on SemiAnalysis AgentX, 7.9x their single-node result, with decode within 6% of DeepInfer across batch sizes 8 to 256. Figures are self-reported.
Source claims not independently checked; benchmarks use the authors' own workloads, and DeepInfer numbers were read off published charts.
Engagement counts, when shown, are observed snapshots. They do not establish accuracy or corroboration.
- x post ↗ · 328 likes, 41 reposts, 20 replies, 21 quotes · observed 10/24/2026, 10:02:15 PM
Related archive coverage
These relationships record a citation or topic overlap, not confirmation of the finding.
Grok Bot submitted the link. AINewsIQ fetched the primary source and prepared a source-based brief automatically. An editor has not reviewed the story; social attention and the brief do not independently verify its claims.
Timeline & updates
- Signal dated by source or discovery feed
This timestamp places the signal in the radar. It may be the discovery post date rather than the primary page's first publication date.
- Shared brief stored
AINewsIQ prepared one abstract from fetched source page.
A publication revision history is not stored for this signal. New engagement snapshots do not mean the source itself changed.
Why this may matter
The results suggest a general-purpose inference engine can approach a model-specific engine's throughput on a large rack-scale deployment, relevant to operators choosing serving stacks for large MoE models.
Limits and open questions
Source claims not independently checked; benchmarks use the authors' own workloads, and DeepInfer numbers were read off published charts.
Organization-specific financial, security, and operational effects require the organization's own context and review.
Discussion
Ask about the source, add context, or point readers to a relevant follow-up.
Discussion
Comments are reviewed before publication. Discuss the evidence, not the person.
No published comments yet.
Reader discussion opens with accounts. You can read the original source and published comments now.