Skip to main content
Back

Z Lab's ParoQuant Fixes 4-bit AI Math Collapse With 2.1x Speedup

A new INT4 quantization method uses cheap pairwise rotations to close the accuracy gap with FP16 on reasoning models, running at near-AWQ speed.

  • Z Lab released ParoQuant, an ICLR 2026 INT4 post-training quantization method for reasoning LLMs.
  • Uses learned scaled pairwise (Givens) rotations instead of full rotation matrices to suppress weight outliers.
  • Achieves 2.4% average accuracy gain over AWQ with less than 10% overhead and 2.1x speedup vs FP16.
  • On Qwen3-4B AIME24, ParoQuant scores 73.3 vs AWQ's 62.2 and FP16's 75.6.
  • Supports vLLM, Transformers, and MLX with pre-quantized Qwen3, Gemma 4, and Llama 3 checkpoints.
  • Open-sourced under MIT license with fused CUDA kernel and quantization pipeline for custom models.

Reasoning models are unusually cruel to quantization. They emit tens of thousands of chain-of-thought tokens, and every rounding error at step N gets fed back in as input at step N+1. What looks like a modest hit on a single-token benchmark turns into collapse on a multi-step math problem. A team from Z Lab has just open-sourced ParoQuant, an ICLR 2026 method that attacks this compounding problem with a new INT4 weight-only quantization scheme.

ParoQuant uses learned pairwise rotations to suppress weight outliers, closing the accuracy gap with FP16 while running at near-AWQ speed. It supports NVIDIA GPUs (vLLM, Transformers) and Apple Silicon (MLX). Under the hood, there is a nice piece of algorithm-and-kernel co-design worth unpacking.

Why reasoning breaks 4-bit models

The core problem is well known. LLM weights contain a small number of outlier channels with unusually large magnitudes, and those outliers consume almost all of the dynamic range of an INT4 grid. Everything else gets quantized coarsely. The presence of outliers in weights and activations often leads to large quantization errors and severe accuracy degradation, especially in recent reasoning LLMs where errors accumulate across long chains of thought.

The concrete damage on Qwen3-4B is stark. AWQ, the most popular INT4 baseline, holds up on MMLU but caves on math. On AIME-24 it drops to 62.2 while the FP16 baseline scores 75.6, and EfficientQAT collapses further to 45.6 despite fine-tuning the quantized weights. That last data point is the interesting one: it suggests outlier suppression is the real bottleneck, not weight fine-tuning.