摘要：随着大语言模型（LLM）越来越多地被用于临床场景，在真实的临床文本变化条件下评估其可靠性至关重要。我们在临床分诊任务中研究这一问题，比较了 LLM 与执业医师在保持底层临床情境不变的文本扰动下的表现。我们提出了一个基准测试集，其中包含超过 6,000 个临床情景、7,000 条医生标注和 225,000 条模型回复。利用这一基准，我们得出两项关键观察结果。第一，在基线条件下，LLM 比医生更可能建议不必要的医疗服务，并且这种倾向会随着输入受到扰动而增强。此外，我们发现，与人类医生的建议相比，LLM 的建议对性别和语气扰动更为敏感。综合来看，这些结果表明，LLM 会因临床上无关的文本变化而产生差异，凸显了开展以专家医生行为为基础、面向实际部署的评估之必要性。

Abstract:As large language models (LLMs) are increasingly used in clinical settings, it is critical to evaluate their reliability under realistic variation in clinical text. We study this question in clinical triage, comparing LLMs to practicing physicians under text perturbations that preserve the underlying clinical setting. We introduce a benchmark of over 6,000 clinical scenarios, 7,000 physician annotations, and 225,000 model responses. Using this benchmark, we make two key observations. First, LLMs are more likely than physicians to recommend unnecessary care at baseline, and this tendency increases under perturbed inputs. Further, we find that LLM recommendations are more sensitive to gender and tone perturbations than human recommendations. Together, these results demonstrate that LLMs can vary under clinically irrelevant textual changes, highlighting the need for deployment-oriented evaluations grounded in expert physician behavior.