Skip to main content
Back

Scantist Shrinks GLM-5.3 by 25% to Fit on a Single NVIDIA B200

A quantized GLM-5.3 variant trims 64 experts via neural architecture search, shrinking the model 24.6% so it fits on a single B200 GPU.

  • New GLM-5.3-SLIM-GGUF fits a GLM-5.3-class model on one NVIDIA B200 at 179.9 GB.
  • Neural architecture search removed 64 of 256 routed experts per MoE layer, no retraining required.
  • Scores 92.7% HumanEval pass@1 and 84.8% GPQA-Diamond with reasoning on.
  • Parameter count drops from 753B to 563B, roughly 2.56 bits per weight.
  • Reaches 203 tok/s total generation at 64 parallel sequences on one B200.
  • Built on Unsloth's UD-IQ2_M quantization of zai-org/GLM-5.3; runs via llama.cpp, Ollama, LM Studio.

GLM-5.3-SLIM-GGUF reduces a 753-billion-parameter mixture-of-experts model to 563 billion parameters, allowing the quantized weights to run on a single NVIDIA B200. A neural architecture search selected 192 of the 256 routed experts in each MoE layer, and the conversion removed the remaining 64 without retraining or recalibration.

The release combines Unsloth’s UD-IQ2_M quantization of GLM-5.3 with the expert selection from GLM-5.3-SLIM-E192. GGUF is a model format designed for runtimes such as llama.cpp; here, its tensor layout makes it possible to delete complete experts while preserving the selected quantized weights byte for byte.

A 58.7 GB cut

Size and architecture comparison
Model Experts per MoE layer Parameters GGUF size
GLM-5.3 UD-IQ2_M 256 753B 238.6 GB
GLM-5.3-SLIM-GGUF 192 563B 179.9 GB
Reduction 25% 25.2% 24.6%

The slim release averages about 2.56 bits per weight. Its 238.6 GB parent exceeds the memory available on one B200 and a 192 GB Mac, forcing multi-device inference. On the tested B200, the authors report that the pruned model remains entirely on the GPU while leaving enough memory for roughly 100,000 tokens of shared key-value cache across as many as 10 concurrent requests.

Scores without a matched baseline

Results reported by the release authors
Benchmark Result Evaluation setup
HumanEval pass@1 92.7% (152/164) Greedy decoding; thinking disabled
GPQA-Diamond 84.8% (168/198) reasoning_effort=high; 65,536-token generation budget
MMLU subset 71.4% Zero-shot; 570 questions

A matched run of the 256-expert parent is required to quantify the quality lost through pruning and quantization. The harness also differs from the official GLM-5.3 evaluation, so these figures should be interpreted within the published setup rather than compared directly with the model card.