摘要：粗糙到逼真（Coarse-to-real，C2R）视频生成技术能够将粗糙动画、物理仿真或游戏渲染视频转换为摄影级逼真视频，同时保持场景布局、相机运动轨迹与动态物理特征不变。训练此类模型往往需要精确对齐的（粗糙，真实）成对视频，其制作成本极高，导致先前的系统被局限在狭窄的合成领域中。我们利用了一种简易的非对称性：成对视频难以获取，但利用现有的图像编辑和 3D 重建工具，构建成对的“真实到粗糙”图像却轻而易举。利用这些图像对，我们训练了 DINO-R2C 模型，在 DINO 特征空间中将真实图像转换为粗糙图像，并将其逐帧应用于普通真实视频以合成“伪粗糙视频”（pseudo-coarse videos）。随后，C2R# 模型学习如何从对应的伪粗糙视频中生成真实视频，从而彻底摆脱了对（粗糙，真实）成对视频的需求。在推理阶段，DINO-R2C 模型被舍弃，由实际的粗糙视频直接驱动 C2R# 模型。因此，我们的无配对训练方法可以随任意真实视频语料库进行扩展，而无需依赖配对的合成视频。我们将在该项目主页链接发布模型与项目主页。

Abstract:Coarse-to-real (C2R) video generation converts coarse animation, simulation, or game-rendered videos into photorealistic videos while preserving scene layout, camera motion, and dynamics. Training such models requires aligned (coarse, realistic) video pairs, which are expensive to create and confine prior systems to narrow synthetic domains. We exploit a simple asymmetry: paired videos are hard to obtain, but paired real-to-coarse images are easy to construct with existing editing and 3D reconstruction tools. Using these image pairs, we train the DINO-R2C model to transfer realistic images to coarse images in DINO feature space, and apply it frame-wise to ordinary real videos to synthesize pseudo-coarse videos. The C2R# model then learns to generate each real video from its pseudo-coarse counterpart, free from needing (coarse, realistic) video pairs. At inference, the DINO-R2C model is discarded and actual coarse videos drive the C2R# model directly. Thus, our pair-free training scales with any real-video corpora rather than with paired synthetic videos. We will release the model and project webpage at this https URL.