摘要：面向任务的对话智能体（Task-oriented conversational agents）在真实世界的对话场景中往往表现脆弱，因为真实交互极少遵循可预测的既定脚本，尤其是在用户表现出非协作行为时。现有的函数调用（function-calling）基准通常侧重于成功的、协作式的交互，对对抗性对话轨迹的覆盖明显不足，从而限制了开发鲁棒 Agent 所需的训练资源。我们提出了 ToolRACER，这是一个合成数据生成流水线，通过协同用户、助手和工具模拟模型，生成并验证用户与 Agent 之间的多轮交互。利用 ToolRACER，我们构建了鲁棒多轮对话基准 ToolRACERBench，横跨 6 个领域、包含 55 种不同的人设画像，生成了由 5600 条对话轨迹组成的验证语料库，其中约 66% 的对话包含易出错的对话场景。我们通过注入对抗行为，产出了捕捉现实、鲁棒场景的验证对话交互轨迹。我们在内部基准以及 $\tau^2$-bench、BFCLv3 和 ACEBench 等函数调用基准上对使用 ToolRACERBench 训练的模型进行了评估，以检验 Agent 的准确率与鲁棒性。在 ToolRACERBench 上训练的模型在 $\tau^2$-bench 和 ACEBench 上均提升了端到端 Agent 准确率，证明了在小语言模型（SLM）的 Agent 能力任务中将其与域内数据集混合训练所带来的显著增益。

Abstract:Task-oriented conversational agents remain fragile under real world conversation scenarios as they rarely follow a predictable script, especially when users exhibit non-cooperative behavior. Existing function-calling benchmarks often emphasize successful, cooperative interactions and underrepresent adversarial conversation trajectories, thereby limiting the training resources available for developing robust agents. We present ToolRACER, a synthetic data generation pipeline that coordinates user, assistant and tool emulation models to generate and validated multi-turn interactions between a user and an agent. Using \sysn, we construct ToolRACERBench a robust multi-turn conversation benchmark spanning six domains, ranging over 55 varied personas, generating a validated corpus of 5.6K conversation trajectories, with approximately 66% of conversations containing failure-prone conversation scenarios. We inject adversarial behaviors, producing validated conversational interaction trajectories that capture realistic, robust scenarios. We evaluate models trained on ToolRACERBench against internal benchmarks, as well as on function calling benchmarks such as $\tau^2$-bench, BFCLv3 and ACEBench to evaluate agentic accuracy and robustness. Models trained on ToolRACERBench improve end to end agentic accuracy across $\tau^2$-bench and ACEBench, demonstrating significant gains when mixed with in-domain dataset in small language models for agent capability tasks.