论文研究 arXiv cs.CV Mon, 26 Oct 2026 正文 ok 223 words

基于多物体交互视频解析视频编码器中的物理规律表征

英文标题:Interpreting Physics in Video Encoders using Multi-Object Interaction Videos

本研究探讨了视频编码器作为具身物理 AI(Physical AI)世界模型时其潜在表征中所蕴含的物理规律。研究人员在双体交互设定下探测了两个冻结的自监督视频编码器(V-JEPA2 与 VideoMAE)的关系物理属性,并评估其跨环境泛化迁移能力。结果表明,两款模型均能可靠区分弹性与非弹性碰撞,但动量违背检测仅在单一模型中有效,且两者均无法在物体几何形状变化后维持有效的摩擦系数表征。该研究揭示了现有自监督视频模型在深层物理关系表征上的边界与局限,对改进具身智能规划与世界模型设计具有重要启示。
打开原文相关 2.4 · 机会 1
media2.0 KB media483 B media612 B media380 B media24.2 KB media34.7 KB

中文

[提交于 2026 年 10 月 22 日]

标题:基于多物体交互视频解析视频编码器中的物理规律表征

查看论文《基于多物体交互视频解析视频编码器中的物理规律表征》(Interpreting Physics in Video Encoders using Multi-Object Interaction Videos)PDF 版本,作者:Ayushi Mehrotra 与另外 1 位作者

查看 PDF HTML (实验性)

摘要:视频编码器模型正日益被用作具身物理 AI(Physical AI)的世界模型。在此类架构中,智能体策略通过推演潜在空间(latent)来进行动作规划,因此潜在表征中所包含的物理信息直接决定了智能体能够规划的动作边界。此外,现有的可解释性研究大多只探测归属于单个物体的物理量;而诸如动量传递、恢复系数和摩擦力等构成关系物理(relational physics)的关键物理量,是定义在多物体之间的,此前尚未被深入检验。我们在双体碰撞设定下探测了两个冻结的自监督视频编码器(V-JEPA2 与 VideoMAE)中的关系物理属性,并测试了它们是否能在不同环境间迁移。我们的初步研究结果喜忧参半:两种编码器都能可靠地区分弹性碰撞与非弹性碰撞;相比于产生动能的碰撞,V-JEPA2 自身的预测器预测出这种非物理情况的概率明显更低;然而,动量守恒违背检测仅在其中一个编码器中有效;动能读出能力在环境发生改变后仅在 V-JEPA2 中保留,而在 VideoMAE 中失效;且两个模型均未能学到能够在物体几何形状改变时依然有效泛化的摩擦系数。

提交历史

来自:Sai Vidyaranya Nuthalapati [

查看邮箱

]

[v1]

2026 年 10 月 22 日 星期四 22:47:56 UTC (1,589 KB)

English / 原文

[Submitted on 22 Oct 2026]

Title:Interpreting Physics in Video Encoders using Multi-Object Interaction Videos

View a PDF of the paper titled Interpreting Physics in Video Encoders using Multi-Object Interaction Videos, by Ayushi Mehrotra and 1 other authors

View PDF HTML (experimental)

Abstract:Video encoder models are increasingly used as world models for physical AI, where a policy plans by rolling out the latent, so what physics the latent holds bounds what the agent can plan over. Moreover, existing interpretability work probes quantities belonging to a single object; the quantities that make physics relational, such as momentum transfer, restitution, and friction, are defined between bodies and have not been examined. We probe relational physics attributes in two frozen self-supervised video encoders (V-JEPA2 and VideoMAE) under two-body settings and test if they transfer across environments. Our early findings are mixed. Both encoders separate elastic from inelastic collisions reliably. V-JEPA2's own predictor is less likely to predict a collision that creates kinetic energy compared to one that loses it. Momentum-violation detection appears in only one of the two encoders, a kinetic-energy readout survives a change of environment for V-JEPA2 but not for VideoMAE, and neither model holds a friction coefficient that survives a change of geometry.

Submission history

From: Sai Vidyaranya Nuthalapati [

view email

]

[v1]

Thu, 22 Oct 2026 22:47:56 UTC (1,589 KB)