Skip to main content
Back

Unsloth Packs OpenAI's gpt-oss-20b Into a 16 GB Local Reasoning Model

Unsloth's GGUF pack of OpenAI's gpt-oss-20b runs on 16GB of memory across llama.cpp, Ollama, LM Studio, and vLLM with Apache 2.0 licensing.

Model, Full Access
  • Unsloth published GGUF quantizations of OpenAI's gpt-oss-20b, from Q2 to F16.
  • Runs within 16GB memory thanks to native MXFP4 precision in the MoE layers.
  • 21B total parameters, 3.6B active, trained on OpenAI's Harmony response format.
  • One-line install for llama.cpp, Ollama, LM Studio, Jan, vLLM, and SGLang.
  • Apache 2.0 license with configurable low, medium, or high reasoning effort.
  • Fine-tunable on consumer GPUs via Unsloth's free Colab notebook.

Unsloth has published a full set of GGUF quantizations of gpt-oss-20b, OpenAI's open-weight reasoning model, covering everything from a 2-bit build at 11.5 GB to a full F16 build at 13.8 GB, with over half a million downloads on Hugging Face. The pack plugs directly into llama.cpp, Ollama, LM Studio, Jan, vLLM, and SGLang, giving you an OpenAI-compatible local server with a single command.

A mixture-of-experts that fits in 16 GB

gpt-oss-20b is a 21B-parameter mixture-of-experts model with 3.6B active parameters, trained on the OpenAI Harmony response format. Its MoE layer uses native MXFP4 precision, which is what lets the larger gpt-oss-120b run on a single H100 and keeps the 20B variant within 16 GB of memory. That puts it within reach of a consumer GPU or a modern Mac without any heroic engineering.

Unsloth's contribution is packaging that model into GGUF across a wide precision ladder. The F16 quant upcasts every layer from MXFP4 to BF16 and leaves them unquantized, so you can stay faithful to OpenAI's original weights or trade accuracy for a smaller footprint.