Joint-Embedding Predictive Architecture (JEPA)

A Joint-Embedding Predictive Architecture, or JEPA, learns by predicting representations instead of reconstructing every pixel. The model receives visible context and predicts the latent representation of hidden or future content.

Core idea

visible context → context representation
hidden target  → target representation
 
predictor(context representation) ≈ target representation

This encourages the representation to keep useful structure and semantics. It can ignore pixel details that do not help the prediction.

Main forms

• I-JEPA predicts representations of masked image regions from visible image context
• V-JEPA extends the idea to masked regions across video and learns temporal representations
• V-JEPA 2 scales video pretraining and adds an action-conditioned world model for planning
• GeoWorld maps the visual representations into a Hyperbolic Latent Space and adds geometry-aware rollout refinement

Why it helps planning

Pixel generation spends capacity on texture, lighting, and other low-level details. A predictive JEPA can compare higher-level latent states instead. This makes it suitable for an Energy-Based Model that scores whether a predicted future is compatible with a goal.

Important distinction

JEPA is not another name for every World Model. JEPA describes how representations are learned and predicted. A world model describes the role of predicting future consequences.

My way of understanding JEPA

The whole idea of JEPA is that it is a model that does not generate pixels like a generic diffusion model. Observations are encoded into a smaller dimensional space, which is then used to make predictions that minimize the energy between the encoded goal and the predicted observation.

The benefit of making predictions in latent space instead of purely on pixels is that the model does not rely only on low-level pattern matching or pixel-level similarities. Instead, it can learn higher-level representations that contain hidden structure and temporal dependencies.

My comparison is with word embeddings. Words with similar meanings are closer to one another in an embedding space. In the same way, visually or temporally related states can be closer to one another in the JEPA representation space.

I-JEPA made the idea clearer for me. Part of the image is masked while other context-rich parts stay visible. The model predicts the masked part in latent representation instead of reconstructing every pixel. V-JEPA then applies this idea to another modality, which is video.