DeepSeek's R1-0528-Qwen3-8B Beats o3-mini on Math Running on a Gaming GPU
DeepSeek distills its upgraded R1-0528 reasoning into an 8B model that matches a 235B giant on math benchmarks, free and MIT-licensed
Takeaways
- DeepSeek released DeepSeek-R1-0528-Qwen3-8B, an 8B reasoning model distilled from the upgraded R1-0528 flagship.
- The 8B model scores 86.0 on AIME 2024, matching Qwen3-235B-thinking and beating o3-mini (medium) at 79.6.
- Training used chain-of-thought distillation: the smaller Qwen3-8B base learned R1-0528's full reasoning traces, not just final answers.
- New features include system prompt support, native function calling, JSON output, and a reduced hallucination rate.
- Runs on a single RTX 3090/4090 (16GB VRAM); available free under MIT license with commercial use and further distillation permitted.
- Weakness: GPQA Diamond score of 61.1 trails Gemini Flash and o3-mini; no image input support.
DeepSeek-R1-0528-Qwen3-8B is a compact, 8-billion-parameter reasoning model that punches well above its weight class. Built by taking the upgraded DeepSeek-R1-0528 and distilling its reasoning patterns into Alibaba's Qwen3-8B base, the result is an open-source model that matches a 235-billion-parameter giant on competitive math benchmarks while fitting on a single consumer GPU. It's available right now on Hugging Face under the MIT license, free for commercial use.
Two releases in one
The DeepSeek R1 model underwent a minor version upgrade to DeepSeek-R1-0528, significantly improving its depth of reasoning and inference capabilities by leveraging increased computational resources and introducing algorithmic optimization mechanisms during post-training. The flagship update is a big deal on its own. But the more immediately practical release for most people is the distilled 8B variant.
DeepSeek distilled the chain-of-thought from DeepSeek-R1-0528 to post-train Qwen3-8B Base, obtaining DeepSeek-R1-0528-Qwen3-8B. This model achieves state-of-the-art performance among open-source models on AIME 2024, surpassing Qwen3-8B by +10.0% and matching the performance of Qwen3-235B-thinking. That last comparison is the headline: a model 29x smaller matching one of the most capable open-weight reasoning models available.
How the distillation works
Chain-of-thought distillation (CoT distillation) is the key technique here. Instead of training from scratch with reinforcement learning, this model was created by continuing the post-training process on the Qwen3-8B Base model using chain-of-thought traces from DeepSeek-R1-0528. The student model learns not just the final answers, but the full step-by-step reasoning paths the larger model took to get there.
Through this training process, the student model learns not only the final answers but also the reasoning steps and decision pathways used by the larger model. As a result, the smaller architecture becomes remarkably capable, even though it contains far fewer parameters. This is fundamentally different from standard fine-tuning, which only teaches a model what to say, not how to think.