摘要：评估可解释人工智能（XAI）方法所生成解释的质量仍然颇具挑战性，因为现有方法通常依赖主观的人类判断，限制了研究之间的可复现性、可扩展性和可比性。本文考察 LLM 能否作为一种可复现且可扩展的机制，对 XAI 解释质量进行比较评估。我们提出 XAI-Arena，这是一个以 LLM 作为评审者的框架，可对 XAI 解释质量进行可扩展、可复现、多维度且对利益相关者敏感的评估。XAI-Arena 支持从多个维度比较 XAI 解释，包括感知简洁性、清晰度、任务适配性、信任校准、可操作性、透明度、忠实性以及整体可解释性。随后，我们在不同数据集、机器学习模型和利益相关者角色下，对 XAI 解释方法进行了基准测试。人工验证表明，LLM 生成的评分与人工评分之间具有较强的正相关关系（Spearman's rho=.693，p<.001）。总体而言，基于 LLM 的评估能够捕捉 XAI 解释质量中的系统性差异，并为 XAI 解释的比较评估提供一个可扩展且可复现的框架。

Abstract:Evaluating the quality of explanations produced by explainable AI (XAI) methods remains challenging because existing approaches often rely on subjective human judgment, limiting reproducibility, scalability, and comparability between studies. We examine whether LLMs can serve as a reproducible and scalable mechanism to make comparative assessments of the quality of XAI explanations. We introduce XAI-Arena, an LLM-as-a-judge framework for scalable, reproducible, multidimensional, and stakeholder-sensitive evaluation of XAI explanation quality. XAI-Arena then allows us to compare XAI explanations along various dimensions, namely, perceived simplicity, clarity, task adequacy, trust calibration, actionability, transparency, faithfulness, and overall interpretability. We then benchmark XAI explanation methods across various datasets, machine learning models, and stakeholder personas. Human validation shows a strong positive association between LLM-generated and human ratings (Spearman's rho=.693, p<.001). Together, LLM-based evaluations can capture systematic differences in XAI explanation quality and provide a scalable and reproducible framework for comparative assessment of XAI explanations.