PreviewDiff:扩散潜变量上的多模态评审器引导搜索
英文标题:PreviewDiff: Multimodal Critic-Guided Search over Diffusion Latents
PreviewDiff 是一种无需训练的测试时搜索方法,旨在改善扩散模型在对象数量、属性绑定、空间关系和时间动作等组合细节上的提示词遵循能力。该方法会在去噪过程的特定检查点解码部分预览图像或视频,利用多模态评审器进行评分和自然语言批评,并据此扩展语义提示词编辑与局部重新加噪的潜变量续写分支。论文称,在图像和视频生成基准上,PreviewDiff 持续优于计算预算相同的 Best-of-N 选择方法及强大的标量搜索基线。消融实验表明,更早介入和更大的搜索宽度带来最大增益,而更深的搜索及更多语义变体可提供互补改进。
640 B
612 B
380 B
9.1 KB
32.7 KB
22.8 KB
24.2 KB
34.7 KB
中文
[提交于 2026 年 9 月 28 日]
标题:PreviewDiff:扩散潜变量上的多模态评审器引导搜索
查看由 Vighnesh Subramaniam 及其他 5 位作者撰写的论文《PreviewDiff:扩散潜变量上的多模态评审器引导搜索》PDF
摘要:扩散模型能够生成令人惊艳的图像和视频,但在使生成结果忠实于提示词的组合细节上仍存在困难,例如对象数量、属性绑定、空间关系以及具有时间锚定的动作。提升提示词满足度的一种常见方式是在测试时通过 Best-of-N 采样投入更多计算资源,但最终样本选择是固定的。Best-of-N 只能在已完成的输出中进行选择,无法在一条有潜力的生成轨迹失败前对其进行修复。我们提出 PreviewDiff,这是一种无需训练的测试时搜索方法,将扩散采样从标量搜索转变为在中间潜变量上进行的多模态评审器引导搜索。在选定的去噪检查点,PreviewDiff 会解码部分预览结果,请多模态判别器对其进行评分和批评,并利用产生的自然语言反馈,在语义提示词编辑和局部重新加噪的潜变量续写之间建立分支。随后,这些分支会被评分并选择性地继续推进,使验证器计算能够在样本仍可编辑时引导生成过程。在图像和视频生成基准上,PreviewDiff 持续优于计算预算相同的 Best-of-N 选择方法以及强大的标量搜索基线。消融实验显示,更早的干预和更大的搜索宽度带来最大收益,而更深的搜索和额外的语义变体则带来互补改进。PreviewDiff 表明,多模态反馈最有价值的作用不仅是作为最终验证器,更是作为去噪过程内部的主动控制器。
提交历史
来自:Vighnesh Subramaniam [
]
[v1]
2026 年 9 月 28 日,星期一,20:00:41 UTC(8,901 KB)
English / 原文
[Submitted on 28 Sep 2026]
Title:PreviewDiff: Multimodal Critic-Guided Search over Diffusion Latents
View a PDF of the paper titled PreviewDiff: Multimodal Critic-Guided Search over Diffusion Latents, by Vighnesh Subramaniam and 5 other authors
Abstract:Diffusion models can produce striking images and videos, but they still struggle with the compositional details that make a generation faithful to a prompt, such as object counts, attribute binding, spatial relations, and temporally grounded actions. A common way to improve prompt satisfaction is to spend more compute at test time through Best-of-N sampling, but final-sample selection is fixed. Best-of-N can only choose among completed outputs and cannot repair a promising trajectory before it fails. We introduce PreviewDiff, a training-free test-time search method that turns diffusion sampling from scalar search into a multimodal critic-guided search over intermediate latents. At selected denoising checkpoints, PreviewDiff decodes a partial preview, asks a multimodal judge to score and critique it, and uses the resulting natural-language feedback to branch over semantic prompt edits and locally re-noised latent continuations. These branches are then scored and selectively rolled forward, allowing verifier compute to guide generation while the sample is still editable. Across image and video generation benchmarks, PreviewDiff consistently improves over budget-matched Best-of-N selection and strong scalar-search baselines. Ablations show that earlier interventions and increased search width provide the largest gains, while deeper search and additional semantic variants offer complementary improvements. PreviewDiff demonstrates that multimodal feedback is most useful not only as a final verifier, but as an active controller inside the denoising process.
Submission history
From: Vighnesh Subramaniam [
]
[v1]
Mon, 28 Sep 2026 20:00:41 UTC (8,901 KB)