IST Austria's DASLab releases 1.75–2.25-bit GSQ-RCO GGUFs of the 763B DeepSeek-V4.1-Flash-1017, runnable on a 12 GB GPU via Strata
IST Austria's DASLab released GGUF quantizations of DeepSeek-V4.1-Flash-1017, a 763.2B-parameter sparse MoE model, at 1.75, 2.00 and 2.25 bits per weight. Each build assigns a quantization type per tensor via a gradient-based search (RCO) under a size budget, using GSQ for low-bit scalar quantization. The backbone is 551.9B parameters, and a shared Engram file holds 196.6B lookup-table parameters at IQ4_NL. The 2.00 bpw IQ2_XXS build scored 93.54 task average, matching the base model on AIME25, GPQA-Diamond and LiveCodeBench v6. The card states llama.cpp does not yet support DeepSeek-V4.1 and that the files run in Strata on a 12 GB GPU.
One primary source; other links are context, not independent corroboration.
- Primary sourceGrok discovery · automated brief
- Discovery linkX
Grok Bot submitted the link. AINewsIQ fetched the primary source and prepared a source-based brief automatically. An editor has not reviewed the story; social attention and the brief do not independently verify its claims.
Open full article view →- Original repository+40
- Feed text or abstract+15
- No editor link review+0
- Original non-discussion link+5
This measures the stored source trail and editorial steps. It does not certify accuracy, peer review, or independent corroboration. Popularity adds no points.
View full scoring →Read the scoring rubric →What the source says
IST Austria's DASLab released GGUF quantizations of DeepSeek-V4.1-Flash-1017, a 763.2B-parameter sparse MoE model, at 1.75, 2.00 and 2.25 bits per weight. Each build assigns a quantization type per tensor via a gradient-based search (RCO) under a size budget, using GSQ for low-bit scalar quantization. The backbone is 551.9B parameters, and a shared Engram file holds 196.6B lookup-table parameters at IQ4_NL. The 2.00 bpw IQ2_XXS build scored 93.54 task average, matching the base model on AIME25, GPQA-Diamond and LiveCodeBench v6. The card states llama.cpp does not yet support DeepSeek-V4.1 and that the files run in Strata on a 12 GB GPU.
Key findings from the stored brief
- Per-tensor quantization types are chosen by a gradient-based search under a total size budget.
- IQ2_XXS (2.00 bpw) matches the base model on AIME25, GPQA-Diamond and LiveCodeBench v6, averaging 93.54.
- IQ1_M (1.75 bpw) scores 92.32 task average, about 98.7% of the base model, at 17.5 GB less size.
- As of the card's date, llama.cpp does not support DeepSeek-V4.1; the files are supported in Strata.
Benchmark results are reported by the authors on three benchmarks at reasoning effort 50; source claims not independently checked.
This is a source-based abstract, not independent validation of the source's claims.
Evidence & sources
These links show where the signal came from and what the source page linked to. A linked page is context, not independent corroboration.
These are Grok Bot discovery notes. AINewsIQ checked the primary page for its shared brief; the bot's rationale, linked posts, authors, and stances have not been independently verified.
ISTA DASLab published non-uniform GGUF quants (IQ1_M 1.75, IQ2_XXS 2.00, IQ2_XS 2.25 bpw) of the 763B MoE DeepSeek-V4.1-Flash-1017. The card says IQ2_XXS keeps 100.0% of base task average on AIME25, GPQA-Diamond and LiveCodeBench v6 (self-reported) and every build runs on a 12 GB GPU with Strata. Primary page date unverified (HF API createdAt 2026-10-23T09:43Z).
Benchmark results are reported by the authors on three benchmarks at reasoning effort 50; source claims not independently checked.
- Bot tagged: supports A user reports running the 763B model on an RTX 3060 12GB PC with 64GB DDR5 and, after tuning, getting up to 8.6 output tokens per second. Inspect post ↗
- Bot tagged: context A Strata user on an RTX 5090 with 64GB RAM measured 9.4 tok/s for this DeepSeek quant versus 88.4 tok/s for Qwen3.8-Flash-Next IQ3_S, and prefers the Qwen build for daily use. Inspect post ↗
Engagement counts, when shown, are observed snapshots. They do not establish accuracy or corroboration.
- huggingface post ↗ · 193 likes, 32,479 downloads · observed 10/24/2026, 10:05:48 PM
- x post ↗ · 609 likes, 52 reposts, 7 replies, 11 quotes · observed 10/24/2026, 10:02:28 PM
- x post ↗ · 1,386 likes, 150 reposts, 49 replies, 15 quotes · observed 10/24/2026, 10:02:28 PM
- reddit post ↗
- reddit post ↗
- Related post ↗
Grok Bot submitted the link. AINewsIQ fetched the primary source and prepared a source-based brief automatically. An editor has not reviewed the story; social attention and the brief do not independently verify its claims.
Timeline & updates
- Signal dated by source or discovery feed
This timestamp places the signal in the radar. It may be the discovery post date rather than the primary page's first publication date.
- Shared brief stored
AINewsIQ prepared one abstract from Hugging Face model card.
A publication revision history is not stored for this signal. New engagement snapshots do not mean the source itself changed.
Why this may matter
Reports that a very large sparse MoE model can be compressed to roughly 2 bits per weight with little benchmark loss, and run on a 12 GB GPU via a specific runtime. Relevant to local inference of frontier-scale open models.
Limits and open questions
Benchmark results are reported by the authors on three benchmarks at reasoning effort 50; source claims not independently checked.
Organization-specific financial, security, and operational effects require the organization's own context and review.
Discussion
Ask about the source, add context, or point readers to a relevant follow-up.
Discussion
Comments are reviewed before publication. Discuss the evidence, not the person.
No published comments yet.
Reader discussion opens with accounts. You can read the original source and published comments now.