Alibaba's Qwen3.6 Beats GPT-Class Coding Scores Activating Only 3B Parameters
Alibaba's new open-weight MoE activates only 3B of 35B parameters per token yet hits 73.4% on SWE-bench Verified.
Takeaways
- Alibaba released Qwen3.6-35B-A3B, an open-weight MoE with 35B total and 3B active parameters, Apache 2.0 licensed.
- Scores 73.4% on SWE-bench Verified, beating dense Gemma 4-31B (52.0%) at a fraction of the compute cost.
- Hybrid architecture combines Gated DeltaNet linear attention with standard attention and 256 MoE experts.
- Native 262K context window, extensible to 1M+ tokens via YaRN scaling.
- New thinking preservation feature keeps reasoning traces from prior turns for agent workflows.
- Available via OpenRouter at $0.08/M input, $0.75/M output tokens.
Alibaba's Qwen team has released Qwen3.6-35B-A3B, an open-weight sparse Mixture of Experts model that punches well above its compute weight class. The multimodal release from Alibaba Cloud carries 35 billion total parameters and activates 3 billion per token, using a hybrid sparse MoE architecture that combines Gated DeltaNet linear attention with standard gated attention layers. Weights ship under Apache 2.0, so you can fine-tune, host, and build products on top of it without a licensing fee.
The headline number has developers paying attention: 73.4% on SWE-bench Verified while activating only 3 billion of its 35 billion parameters per token. For context, Gemma 4-31B, a dense model that activates all 31 billion of its parameters every inference step, scores 52.0% on the same benchmark.
What A3B actually means for your GPU budget
The naming convention matters. The 35B is the total parameter count sitting in memory, but the router only activates a small fraction for each forward pass. In practice you get the representational capacity of a 35B model at roughly the inference cost of a 3B model, which is why the benchmarks look the way they do.
The internal layout is unusual. Instead of a stack of identical transformer blocks, Qwen3.6 uses a repeating pattern of 10 groups, each containing three Gated DeltaNet layers followed by one Gated Attention layer, all feeding into MoE feed-forward networks. Gated DeltaNet is a linear-attention variant that scales with sequence length far better than softmax attention, which is how the model natively supports a context length of 262,144 tokens, extensible up to 1,010,000 tokens through YaRN scaling. The MoE layer holds 256 experts and routes each token through 8 of them plus 1 shared expert.