Skip to main content
Back

DeepSeek's Janus-Pro-7B Beats DALL-E 3 by Splitting Vision Into Two Encoders

DeepSeek's Janus-Pro-7B unifies image understanding and generation in one open-weight model, beating DALL-E 3 on text-to-image benchmarks with a novel decoupled visual encoder design.

  • What: DeepSeek released Janus-Pro-7B, an open-weight model that does both image understanding and text-to-image generation in one transformer.
  • Key innovation: Decoupled visual encoding — separate encoders for understanding vs. generation tasks, solving a fundamental conflict in unified multimodal models.
  • Benchmark wins: Scores 0.80 on GenEval vs. DALL-E 3's 0.67, and 84.19 on DPG-Bench, beating both DALL-E 3 and Stable Diffusion 3 Medium.
  • Understanding performance: 79.2 on MMBench, outperforming TokenFlow-XL (13B) despite being a 7B model.
  • Limitations: Output resolution capped at 384x384 pixels; struggles with generating realistic human faces.
  • Access: Free, MIT-licensed code on GitHub; live demo on Hugging Face Spaces.

Janus-Pro-7B is DeepSeek's open-weight multimodal model that handles both directions of the vision-language problem in a single autoregressive transformer: feed it an image and ask questions about it, or give it a text prompt and get an image back. What makes it stand out is not just the dual capability, but how it achieves it, through a fundamentally different architectural choice that most unified models have avoided.

The Problem Every Unified Model Runs Into

Most multimodal models that try to do both image understanding and image generation share a single visual encoder for both tasks. On the surface that seems efficient, but the two tasks actually want very different things from that encoder. Since the representations required for understanding and generation differ, this often results in suboptimal performance, and Janus addressed this by decoupling visual encoding, alleviating the conflict between the two tasks.

Janus-Pro takes that original idea and scales it up across three axes: a larger model (7B parameters, up from 1.5B), more training data, and an improved training strategy. Janus-Pro is an enhanced version of Janus that incorporates improvements across training strategies, data, and model size, with the series including both 1B and 7B variants.

Two Encoders, One Transformer

Janus-Pro addresses the limitations of previous approaches by decoupling visual encoding into separate pathways, while still utilizing a single, unified transformer architecture for processing. The decoupling alleviates the conflict between the visual encoder's roles in understanding and generation, and enhances the framework's flexibility.

Concretely, the architecture has three specialized components: