Alibaba's Qwen3.6 Beats Closed Models on Coding With Only 3B Active Parameters
Alibaba's Qwen3.6-35B-A3B-FP8 ships a memory-efficient quantized checkpoint for a frontier-class coding agent that runs on a single GPU
Takeaways
- Qwen3.6-35B-A3B-FP8 is Alibaba's official FP8-quantized open-weight MoE model with 35B total / 3B active parameters, available on Hugging Face under Apache 2.0.
- FP8 quantization halves memory vs BF16, but Qwen deliberately keeps sensitive linear-attention (SSM) layers in BF16 to prevent context degradation at 15K+ tokens.
- Scores 73.4% on SWE-bench Verified and 51.5% on Terminal-Bench 2.0, outperforming Gemma4-31B (52%) despite having ~10x fewer active parameters.
- New Thinking Preservation feature retains chain-of-thought reasoning across agent turns, reducing redundant token generation in multi-step workflows.
- Runs on a single H100/H200 in FP8, or ~22GB RAM via GGUF; MTP speculative decoding delivers 1.4–2.2x faster inference with no accuracy loss.
- Community debate: NVIDIA's FP8 checkpoint quantizes all linear attention layers, while Qwen warns this causes instruction drift at long contexts — pick your checkpoint carefully.
Qwen3.6-35B-A3B-FP8 is Alibaba's official FP8-quantized release of their latest open-weight MoE model, and it's already racked up over 6 million downloads. The headline number is deceptively simple: 35 billion total parameters, but only 3 billion active per token. That gap is the whole story.
The architecture behind the efficiency
Qwen3.6-35B-A3B is a hybrid MoE model with 35B total parameters and only 3B active per token, built on an architecture that combines Gated DeltaNet linear attention with standard Gated Attention and sparse MoE (256 experts, 8 routed + 1 shared active). Understanding what "Gated DeltaNet" means is key to appreciating why this model is special.
Standard transformer attention is quadratic in sequence length , it gets dramatically more expensive as context grows. Gated DeltaNet is a form of linear attention: 3 out of every 4 layers use it instead of standard attention, with only every 4th layer using full Gated Attention. This makes the model much more memory-efficient for long contexts. The practical result is a model that can handle very long sequences without the usual memory explosion.
The model features 40 transformer layers, a hidden dimension of 2048, and native support for 262,144 tokens of context, extensible up to 1,010,000 tokens. The MoE routing means your hardware only needs to handle the active parameters, not the total count , if you can run a 7B model, you can run this. The 3B active parameter count is the number that matters for speed.
What the FP8 checkpoint actually does
The quantization method is fine-grained FP8 quantization with block size of 128, and its performance metrics are nearly identical to those of the original model. FP8 (8-bit floating point) stores weights in roughly half the memory of the standard BF16 format, with minimal accuracy loss on most tasks.
There's an important nuance here that's sparked real debate in the community.
The official Qwen FP8 quantized models deliberately leave specific linear-attention projection weights in native BF16 to prevent context degradation , specifically the in_proj_a and in_proj_b weights.
Because the SSM (state space model component) relies on recurrent mathematical states to maintain memory, minor rounding errors introduced by lower-precision quantization compound with every single token , and conventional quantization crushes these weights down to 8-bit.
Qwen's official FP8 checkpoint avoids this by keeping those sensitive layers in BF16, which is why the file is slightly larger than a naive FP8 conversion would be.