Z.ai's ChatGLM2-6B Hits 571% Math Gains on a Single Consumer GPU
A bilingual 6B chat model with 32K context, Multi-Query Attention, and a 42 percent inference speedup that runs on 6GB of VRAM
Takeaways
- ChatGLM2-6B is a bilingual open chat model from Z.ai (formerly THUDM) with 2K+ likes on Hugging Face
- Pretrained on 1.4T Chinese-English tokens with human preference alignment on top of the GLM hybrid objective
- Benchmark gains over v1: MMLU +23%, CEval +33%, GSM8K +571%, BBH +60%
- Context extended from 2K to 32K using FlashAttention, with 8K used in dialogue training
- Multi-Query Attention gives 42% faster inference and fits 8K dialogue in 6GB VRAM at INT4
- Apache-2.0 code, free commercial use for weights after filling out a registration questionnaire
The ChatGLM2-6B checkpoint from Z.ai (formerly THUDM) is drawing renewed attention on Hugging Face, sitting at over 2,000 likes and roughly 445K downloads. It is a compact bilingual Chinese-English chat model rebuilt from the ground up over the original ChatGLM-6B, and the design choices behind it remain a useful reference for anyone shipping small local LLMs.
What is actually in the box
ChatGLM2-6B is the second-generation version of the open-source bilingual chat model ChatGLM-6B. It keeps the smooth conversation flow and low deployment threshold of the original, but the base was fully rebuilt around the hybrid objective function of GLM, with pre-training on 1.4T bilingual tokens followed by human preference alignment.
Three upgrades matter for practitioners:
- Benchmark jumps. Against the first generation, ChatGLM2-6B posts gains of +23% on MMLU, +33% on CEval, +571% on GSM8K, and +60% on BBH, making it competitive among models of the same size. The math delta is the eye-catcher, since 6B models are usually poor at arithmetic and multi-step reasoning.
- Longer context. Using FlashAttention, the team stretched the base model's context from 2K in ChatGLM-6B to 32K, with 8K used during dialogue alignment training.