JEPA 引导扩散:用于生成式交通预测的预测性视觉—语言条件控制
英文标题:JEPA Guided Diffusion: Predictive Vision-Language Conditioning for Generative Traffic Forecasting
该研究提出一种解耦式交通预测框架,将未来场景表征学习与视频生成分离。方法先使用冻结的 V-JEPA 编码器从已观测交通视频中提取预测性潜在表征,再通过轻量级潜在对齐模块将其映射至冻结的 Cosmos 扩散模块条件空间,以生成未来视频。该方案无需重新训练大型生成模型,仅训练对齐模块,从而降低优化复杂度;在 AI City Challenge 2026 Track 5 基准上获得 75.1297 分,并在竞赛中排名第三。研究表明,V-JEPA 学到的预测性世界表征能够有效引导下游视频生成,为端到端扩散式预测提供了更实用、高效的替代路径。
612 B
380 B
9.1 KB
32.7 KB
22.8 KB
24.2 KB
34.7 KB
中文
[提交于 2026 年 9 月 18 日]
标题:JEPA 引导扩散:用于生成式交通预测的预测性视觉—语言条件控制
查看由 Trinh Tra Giang Nguyen 及另外 2 位作者撰写的论文《JEPA Guided Diffusion: Predictive Vision-Language Conditioning for Generative Traffic Forecasting》的 PDF
摘要:准确的交通预测既需要理解场景动态,也需要合成逼真的未来观测结果。近期基于扩散的视频生成模型能够产生视觉上可信的预测,但通常需要昂贵的端到端训练,并且往往将场景理解与图像合成相互耦合。本文提出一种解耦式预测框架,将未来表征学习与视频生成分离。首先,冻结的 V-JEPA 编码器从已观测交通视频中提取预测性潜在表征,在语义潜在空间中捕捉底层场景动态。随后,一个轻量级潜在对齐模块将这些表征投影到冻结的 Cosmos 扩散模块的条件空间中,从而无需重新训练大型生成模型即可实现未来视频合成。通过冻结所有基础模型、仅训练轻量级对齐模块,所提出的框架在保持预测能力的同时,大幅降低了优化复杂度。AI City Challenge 2026 Track 5 基准上的实验结果表明,该方法取得了 75.1297 分,在竞赛中排名第三。这些结果表明,V-JEPA 学得的预测性世界表征能够有效引导下游视频生成,为端到端扩散式预测提供了一种实用且高效的替代方案。
提交历史
来自:Trinh Tra Giang Nguyen [
]
[v1]
2026 年 9 月 18 日,星期五,06:45:37 UTC(1,747 KB)
English / 原文
[Submitted on 18 Sep 2026]
Title:JEPA Guided Diffusion: Predictive Vision-Language Conditioning for Generative Traffic Forecasting
View a PDF of the paper titled JEPA Guided Diffusion: Predictive Vision-Language Conditioning for Generative Traffic Forecasting, by Trinh Tra Giang Nguyen and 2 other authors
Abstract:Accurate traffic forecasting requires both understanding scene dynamics and synthesizing realistic future observations. Recent diffusion-based video generation models produce visually plausible predictions but require expensive end-to-end training and often entangle scene understanding with image synthesis. In this work, we propose a decoupled forecasting framework that separates future representation learning from video generation. A frozen V-JEPA encoder first extracts predictive latent representations from the observed traffic videos, capturing the underlying scene dynamics in a semantic latent space. A lightweight latent alignment module then projects these representations into the conditioning space of a frozen Cosmos diffusion module, enabling future video synthesis without retraining the large generative model. By freezing all foundation models and training only the lightweight alignment module, the proposed framework substantially reduces optimization complexity while preserving forecasting capability. Experimental results on the AI City Challenge 2026 Track 5 benchmark demonstrate that the proposed method achieved a score of 75.1297, ranking third in the competition. These results suggest that predictive world representations learned by V-JEPA can effectively guide downstream video generation, providing a practical and efficient alternative to end-to-end diffusion-based forecasting.
Submission history
From: Trinh Tra Giang Nguyen [
]
[v1]
Fri, 18 Sep 2026 06:45:37 UTC (1,747 KB)