Skip to main content
Back

Google's Gemma 4 12B Runs Fully Offline on a 16GB Gaming Laptop

Google's encoder-free Gemma 4 12B drops vision and audio encoders, fitting multimodal reasoning into a 6.6GB quantized file that runs on a 16GB laptop.

Model, Full Access
  • Google's Gemma 4 12B Unified ships as an encoder-free multimodal model running on 16 GB laptops
  • Single decoder-only transformer replaces 550M vision and 300M audio encoders with a 35M projection layer
  • Reported scores: 78.8% GPQA Diamond, 77.5% AIME, 77.2% MMLU Pro, 72% LiveCodeBench v6
  • Native support for text, image, audio, and video input with 256K context window and Apache 2.0 license
  • Unsloth Q4_K_M GGUF is 6.6 GB, runs at ~21 tok/s on an RTX 4060 via llama.cpp
  • Includes built-in thinking mode, function calling, system prompts, and Unsloth Studio fine-tuning support

A 12-billion parameter multimodal model that fits on a mid-range gaming laptop and outscores last year's flagship Gemma 3 27B on several benchmarks just landed on Hugging Face. The Unsloth GGUF build of Gemma 4 12B packages Google DeepMind's new encoder-free multimodal architecture into quantized files small enough to run on consumer GPUs, with the Q4_K_M variant clocking in at just 6.6 GB and pulling roughly 21 tokens per second on an RTX 4060.

The headline isn't the size, though. It's what Google quietly removed to get there.

One transformer, every modality

Almost every multimodal model shipping today follows the same recipe: take a frozen vision encoder, bolt on a frozen audio encoder, glue both to an LLM backbone with projection layers, and call it a day. Gemma 4 12B collapses all of that into a single decoder-only transformer that handles text, images, audio, and video with a lower memory footprint and reduced inference latency.

The savings are dramatic. In earlier Gemma 4 variants, a 550M parameter vision encoder and a 300M parameter audio encoder sat in front of the LLM. The 12B Unified model replaces them with a 35M parameter vision embedder, essentially a single matmul that projects raw 48x48 pixel patches directly into the LLM input space. Audio is handled the same way: raw 16 kHz signal sliced into 40ms frames, linearly projected, fed straight to the transformer. It brings native audio and vision understanding directly to local environments without the need for separate encoders. This unified approach to multimodality makes the model encoder-free, offering a deployment size that is perfect for consumer devices and streamlined local execution.

What it can actually do

Across the benchmarks Google published with the release, the 12B punches well above its parameter count:

  • AIME 2026 (math olympiad): 77.5%
  • GPQA Diamond (graduate-level science): 78.8%