Qwen3.5-9B-DeepSeek-V4-Flash Reasons 2.6x Faster on a Single Consumer GPU
A 9B Qwen model distilled from DeepSeek-V4 that thinks faster, uses fewer tokens, and still runs on modest hardware.
Takeaways
- Qwen3.5-9B distilled from DeepSeek-V4 using only 8,000 curated long-CoT examples.
- Distill reaches answers 2.2x faster with 2.6x shorter reasoning chains vs base Qwen3.5-9B.
- Tool calling holds at 5/6 PASS, so agent workflows keep working after fine-tune.
- Ships as GGUF from Q3_K_M (4.62 GB) to Q8_0 (9.53 GB) for llama.cpp, Ollama, vLLM.
- Trained in Unsloth on NVIDIA DGX, Apache 2.0, six languages including EN/ZH/JA/KO/ES/RU.
- Full reproducible fine-tuning guide and Colab notebook available on GitHub.
A community developer released a distilled version of Qwen3.5-9B that inherits reasoning behavior from DeepSeek-V4, and the result runs comfortably on a single consumer GPU. Qwen3.5-9B-DeepSeek-V4-Flash ships as GGUF quantizations from 3-bit to 8-bit, so it slots into llama.cpp, Ollama, LM Studio, and vLLM with a one-line command.
DeepSeek-V4 is a heavyweight reasoning model with long chain-of-thought traces and strong agentic behavior. Rather than serving that trillion-parameter-class teacher, this project transfers its reasoning style into a 9B student using a curated dataset of only 8,000 examples.
Inside the distillation recipe
The teacher is DeepSeek-V4, which the author describes as a flagship open-source series engineered for extreme efficiency, million-token context, and advanced agentic workflows. The student is Qwen3.5-9B, fine-tuned in the Unsloth framework on an NVIDIA DGX using the DeepSeek-V4-Distill-8000x dataset.
The training philosophy leans on a recent arXiv paper about reasoning SFT generalization. The claim: high-quality long-CoT data enables cross-domain transfer, and a short, tightly curated set of 8,000 examples prevents the student from overfitting to the teacher's stylistic quirks while preserving the core reasoning engine. The goal is to activate latent knowledge already sitting inside the base weights instead of pumping in millions of noisy samples.