摘要：基于 LLM 的聊天机器人正在改变银行等强监管领域的客户服务，但如何以可扩展且具成本效益的方式对其进行验证，仍是安全部署的关键障碍。我们为大规模聊天机器人验证提出两项贡献。首先，我们提出了一种方法，用于创建作为数字孪生的高保真合成客户智能体（Synthetic Customer Agents，SCA）。该方法以真实交易和对话数据为基础，能够自动生成智能体并进行行为条件控制，从而模拟多样化的客户画像和交互风格。评估表明，SCA 与真实客户具有较高的语义一致性、较低的幻觉率，并能够在可控干预下成功复现人格特征。其次，我们开发了一套基于 SCA 的验证框架，结合自动化的 LLM-as-a-Judge 评估、人工专家测试和对抗性探测。针对情绪状态、人口统计群体和语言因素开展的基于场景验证确认了其稳健表现。我们的方法已被用于验证一家英国领先银行的面向客户聊天机器人，为金融机构走向监管合规提供了一条可扩展的路径。

Abstract:LLM-based chatbots are transforming customer service in regulated domains such as banking, but scalable and cost-effective validation remains a critical barrier to safe deployment. We present a two-part contribution for large-scale chatbot validation. First, we introduce a methodology for creating high-fidelity synthetic customer agents (SCAs) as digital twins, grounded in real transactional and conversational data, that enables automatic generation and behavioral conditioning to simulate diverse customer profiles and interaction styles. Evaluation demonstrates that SCAs achieve high semantic alignment with real customers, low hallucination rates, and successful personality trait reproduction with controllable interventions. Second, we develop an SCA-based validation framework combining automated LLM-as-a-Judge evaluation, human expert testing, and adversarial probing. Scenario-based validation across emotional states, demographic groups, and linguistic factors confirms robust performance. Our approach was used to validate a customer facing chatbot at a leading UK bank, providing financial institutions with a scalable pathway toward regulatory compliance.