AtomicChat Squeezes a 763B DeepSeek V4.1 Flash Onto a 12 GB GPU
AtomicChat strips refusals from DeepSeek V4.1 Flash and ships 1.75 to 2.25-bit quants that run a 763B MoE on a single 12 GB GPU.
Takeaways
- AtomicChat releases uncensored GGUF quants of DeepSeek V4.1 Flash 1017, a 763B MoE with 384 experts.
- Refusals drop from ~85% to 0% on JailbreakBench across IQ1_M, IQ2_XXS and IQ2_XS rungs.
- Reasoning preserved: GPQA-Diamond 83.3% unchanged, AIME25 76.7% to 80.0% versus stock quant.
- Runs on a single 12 GB GPU with 64 GB RAM via Strata, streaming a 103 GiB Engram table from SSD.
- Built on ISTA-DASLab's GSQ-RCO per-tensor non-uniform quantization at 1.75 to 2.25 bits per weight.
- MIT licensed; routed-expert tensors left byte-identical, only dense residual writers edited for refusal removal.
AtomicChat has released an edited build of DeepSeek V4.1 Flash that combines ISTA-DASLab’s extreme low-bit quantization with refusal-direction projection. The 763-billion-parameter mixture-of-experts model can run with a 12 GB Nvidia GPU, 46 to 64 GB of system memory, and fast NVMe storage through the Strata runtime.
The derivative is available from the AtomicChat model page and builds on ISTA-DASLab’s quants. Its model card lists an MIT license and reports that measured jailbreak refusals fell from roughly 85% to zero while selected reasoning scores remained within evaluation noise. Those results come from the release authors and have not been independently reproduced here.
A 763B model split across three storage tiers
The 763B figure describes the model’s total parameter count. Its mixture-of-experts architecture activates selected experts for each token, reducing computation, while the complete weight set still requires substantial storage. Strata distributes that load across GPU memory, system memory, and NVMe instead of keeping the full model resident on the graphics card.
- GPU: A 12 GB Nvidia card handles accelerated computation.
- System memory: The documented configuration uses 46 to 64 GB of RAM.
- Storage: The selected quantization shards and a shared 103 GiB Engram file reside on a fast NVMe drive.
- Runtime: Strata manages the tiered layout and streams the Engram lookup data from storage.
The Engram file, DeepSeek-V4.1-Flash-Engram-IQ4_NL.gguf, contains 110,595,290,816 bytes, or 103.00 GiB. It is downloaded once from ISTA-DASLab’s repository. An IQ2_XXS installation requires roughly 250 GB for its 138 GB quant shards and the Engram file before allowing for temporary downloads or other overhead.