摘要：视觉语言模型近年来在目标检测等客观且可验证的领域取得了进步，但在主观性和创意性设计任务上的表现仍然不足。造成这一性能差距的一个主要原因，是缺少能够捕捉多样化偏好与决策的高质量人类工作流数据，而这些偏好与决策正是人类专家擅长设计任务的关键。在本研究中，我们首先定义了一套由专家整理的、独特的设计技能与最佳实践分类体系，并进一步将其扩展为包含 126 项开放式、主观且长程任务的集合。基于这一体系及专家解决方案，我们构建了 FigmaTrace 数据集：该数据集包含超过 200 小时的人类录制视频数据，并通过一种新颖的、基于设计阶段的方法，转换为 3,469 条设计轨迹。我们使用该数据集训练了四个模型，结果显示，在四个分布外的 Agent GUI 环境中，使用 FigmaTrace 训练可带来可与 \textsc{Claude-Opus-5} 和 \textsc{GPT-5.6-Sol} 等前沿闭源模型相媲美的性能提升。我们还进行了有价值的消融实验，将这些性能提升归因于基于设计阶段的视频到轨迹转换方法；该方法优于以往基于长度的转换方式。最后，我们对表现最佳的 \textsc{Qwen3.8-27B} 输出进行了定性分析，以更好地将性能提升与 FigmaTrace 的趋势关联起来。我们面向社区开源了数据集和最佳模型。

Abstract:Vision Language Models have recently shown improvements in several objective and verifiable domains such as object detection but continue to underperform on subjective and creative design tasks. A major contributor to this performance gap is the lack of high quality human workflow data that captures a diverse set of preferences and decisions that make human experts good at design tasks. In this work, we first define a unique, expert curated taxonomy of design skills and best practices which we further expand into a set of 126 open ended, subjective, long horizon tasks. Built on top of this and expert solutions, our dataset FigmaTrace contains over 200 hours of human captured video data converted into 3469 design trajectories using a novel design phase-based method. We use our dataset to train four models and show that training on FigmaTrace leads to a performance improvement comparable to frontier closed models such as \textsc{Claude-Opus-5} and \textsc{GPT-5.6-Sol} on four out of distribution agentic GUI environments. We further perform a useful ablation to attribute these performance improvements to a design phase-based video to trajectory conversion which outperforms prior length-based conversion approaches. Finally, we perform a qualitative analysis on the best performing \textsc{Qwen3.8-27B} outputs to better correlate performance improvements to FigmaTrace's trends. We open source our dataset and the best model for the community.