Skip to main content
Back

Predictive Closure Theory Rewrites How JEPA Encoders Choose What to Keep

A new theoretical lens explains why JEPA encoders keep seemingly unpredictable features, and how to steer which features survive in the latent space.

  • New paper proposes predictive closure, a set-level account of which features JEPA encoders retain.
  • Fixes the worry that unpredictable features like a traffic light would be discarded by JEPA.
  • Features survive if they belong to a set that jointly lowers predictor error, not just individually.
  • Controlled experiments confirm cross-subsidization, set switching, and group-size reversal effects.
  • Shows two steering levers: auxiliary reconstruction and predictor side information, both acting indirectly.
  • Theory paper on arXiv; no code or model released.

A predictive-closure paper offers a set-level account of which features self-supervised JEPA models retain. Its controlled experiments show that an individually unpredictable feature can remain in the latent representation, the model’s internal vector, when that feature helps predict other signals. The result expands the common per-feature interpretation and suggests practical ways to steer encoders with auxiliary losses or predictor side information.

Why the loss leaves room

A typical JEPA trains three components: a context encoder, a target encoder, and a predictor. Each encoder converts an input view into a latent vector. The predictor estimates the target vector from the context vector, and training penalizes the difference between that estimate and the target.

Because the target representation is also learned, the objective does not specify which semantic features either encoder must preserve. Many feature combinations can support low prediction error, leaving the learned representation difficult to interpret.

Earlier accounts often treated selection feature by feature: predictable signals remain, and unpredictable signals disappear. The paper’s traffic-light example exposes the gap. A light that changes randomly between red and green has low individual predictability, yet its current state may be essential to a downstream driving policy.