Alibaba's Qwen3-Coder-30B Now Runs on a Single GPU After 4-Bit Compression
A community contributor packaged Alibaba's 30B mixture-of-experts coder into a 4-bit AWQ build that fits on consumer GPUs and streams through vLLM.
Takeaways
- Community contributor cyankiwi published a 4-bit AWQ quantization of Qwen3-Coder-30B-A3B-Instruct.
- The repo has pulled 942K downloads in the last month against 58 likes.
- Uses llm-compressor with group size 32 and Nvidia's Llama-Nemotron calibration dataset.
- Fits on a single 24GB GPU and confirmed working on dual RTX 3090 setups.
- Serves directly through vLLM and SGLang via compressed-tensors, no conversion needed.
- Non-thinking mode only, and a separate quantizer warns 4-bit causes noticeable quality loss.
A Hugging Face user going by cyankiwi (Ton Cao) has published a 4-bit AWQ quantization of Alibaba's Qwen3-Coder-30B-A3B-Instruct, and it is quietly becoming one of the most downloaded coder builds on the Hub. The repo has pulled in 942,131 downloads in the last month against just 58 likes, which usually signals a model people are actually wiring into pipelines rather than just bookmarking.
The base model is Alibaba's Qwen3-Coder-30B-A3B-Instruct, a mixture-of-experts architecture with 30.5B total parameters and 3.3B activated, 48 layers, GQA attention with 32 query heads and 4 KV heads, 128 experts with 8 active per token, and a native context length of 262,144 tokens. Alibaba's original release targets agentic coding and browser-use tasks, long-context repository understanding extendable to 1M tokens using YaRN, and a function call format compatible with tools like Qwen Code and CLINE. The catch has always been VRAM. The full BF16 weights need roughly 60GB, which locks out most single-GPU setups.
Inside the quantization recipe
The recipe is documented on the model card. The quantization uses cyankiwi AWQ v1.0 at 4 bits with a group size of 32, calibrated on nvidia/Llama-Nemotron-Post-Training-Dataset, produced with the llm-compressor tool, and evaluated at a 16,384-token context. A few things worth unpacking there: