Skip to main content
Back

Unsloth Makes Liquid AI's LFM2.5 Run at 253 tok/s on a Laptop

Unsloth's Dynamic 2.0 GGUF quantization brings Liquid AI's blazing-fast 8B MoE edge model to any laptop, CPU, or local runtime

Unsloth has released a GGUF-quantized version of Liquid AI's LFM2.5-8B-A1B, making one of the fastest small models available today trivially easy to run on a laptop, a Mac, or any llama.cpp-compatible setup. The base model itself is a significant leap forward for on-device AI, and now it's one ollama run command away.

Not your average 8B model

LFM2.5-8B-A1B holds 8.3B total parameters but activates only 1.5B per token -- that sparsity is what lets it run on consumer hardware. The mechanism behind this is Mixture-of-Experts (MoE): instead of running every weight on every token, the model routes each token through a small subset of specialized sub-networks called "experts." The config lists 32 experts with 4 experts active per token. The result is a model that thinks at 1.5B-parameter speed while carrying 8.3B parameters worth of knowledge capacity.

The architecture has 24 layers: 18 are double-gated short-convolution blocks based on Liquid AI's proprietary LIV design, and 6 are GQA (Grouped Query Attention) layers. The convolutional blocks handle local dependencies at low computational cost, while the few attention layers take care of long-range relationships in the text. The result is a model that maintains a huge context window without the quadratic memory consumption typical of pure Transformer architectures.

This hybrid design is the key insight. Pure transformers pay a quadratic cost in memory as context grows -- every token attends to every other token. By replacing most layers with short convolutions that only look at nearby tokens, and reserving full attention for just 6 of the 24 layers, LFM2.5 sidesteps that bottleneck entirely.