识别、模拟与拒绝：一项关于 LLM Agent 经典心理学效应的防污染研究
英文标题：Recognition, Simulation, and Refusal: A Contamination-Aware Study of Classic Psychological Effects in LLM Agents
该研究深入探讨了“LLM Agent 展现人类心理学效应是否等于其真正拥有认知偏差”这一核心问题，并提出了全新基准 PsyAgentBench。通过对 5 种经典心理学范式进行标签提示、数据去污染与角色设定的析因实验（共计 41,904 次评测），研究发现模型看似类人的偏差其实源于完全不同的机制：例如阿希从众效应在盲测时为 0%、仅在显式标注范式后才跃升至 83.3%，锚定效应仅在缺乏知识的虚构数据上显现，而沉没成本效应则完全缺席。这项成果揭示了直接套用标量化心理学偏差评分评估 AI 的缺陷，并系统归纳了心理学范式在 LLM Agent 迁移中的三大失效模式，为开发者构建可靠的 Agent 行为评估与仿真机制提供了关键指引。
2.1 KB
640 B
612 B
380 B
9.1 KB
32.7 KB
22.8 KB
24.2 KB
34.7 KB
中文
[提交于 2026 年 7 月 23 日]
标题：识别、模拟与拒绝：一项关于 LLM Agent 经典心理学效应的防污染研究
查看论文《识别、模拟与拒绝：一项关于 LLM Agent 经典心理学效应的防污染研究》PDF，作者：Joy Bose
摘要：大语言模型（LLM）表现出与人类心理学效应相关的反应模式，并不等同于该 LLM 具备这种偏差。我们提出了 PsyAgentBench，这是一个在析因设计下重新在 LLM Agent 上运行经典心理学实验的基准测试，旨在解耦这两种情况：每个范式要么在 Prompt 中显式标出范式名称（命名组，named），要么将其构建为常规任务（盲测组，blind）；任务内容要么采用教科书级别的原版任务（经典组，canonical），要么采用结构匹配、旨在减少与可能训练数据词汇和场景重叠的变体（反事实组，counterfactual），并与角色设定（Persona）干预进行交叉实验。在已完成的五个范式中，针对最多三个开源权重模型家族进行了评估并发布了 41,904 次试验数据，结果表明，看似类人的效应是通过截然不同的路径产生的，而非单一的偏差易感性：范式标签门控伴随显式覆盖（阿希从众效应在 gpt-oss-120B 上盲测为 0%，显式命名后达到 83.3%）、基于知识的信号依赖（锚定效应，在具有事实依据的问题上为 0%，而在虚构数值上几乎完全依赖锚定，这种模式完全符合理性利用唯一可用信号的特征）、标签下新内容的放大效应（框架效应）、稳健的缺席（沉没成本效应），以及以拒绝响应本身作为主要发现的安全对齐介导选择（最小群体分配）。仅仅一句话的角色设定改变（如宜人性，作为指令输入而非经过验证的特质干预）就能消除、削弱或逆转这些效应，具体取决于具体范式，这反驳了任何单一的响应偏差解释。我们进一步形式化并以实证记录了心理学范式迁移到 LLM Agent 时可能失败的三种途径：角色主导（persona dominance）、群体崩溃（population collapse）和安全选择（safety selection）。我们认为标量偏差易感性得分掩盖了这种结构，并主张采用“复现画像（replication profiles）”进行报告。
提交历史
来自：Joy Bose [
]
[v1] 2026 年 7 月 23 日 星期四 10:34:06 UTC (339 KB)
English / 原文
[Submitted on 23 Jul 2026]
Title:Recognition, Simulation, and Refusal: A Contamination-Aware Study of Classic Psychological Effects in LLM Agents
View a PDF of the paper titled Recognition, Simulation, and Refusal: A Contamination-Aware Study of Classic Psychological Effects in LLM Agents, by Joy Bose
Abstract:An LLM producing the response pattern associated with a human psychological effect is not the same claim as the LLM possessing that bias. We present PsyAgentBench, a benchmark that re-runs classic psychology experiments on LLM agents under a factorial design built to separate these: each paradigm is run with the paradigm explicitly labeled in the prompt (named) or framed as a routine task (blind), and on the literal textbook version of the task (canonical) or a structurally matched variant written to reduce lexical and scenario overlap with likely training data (counterfactual), crossed with a persona manipulation. Across five completed paradigms, evaluated on up to three open-weight model families with 41,904 trials released, apparently human-like effects arise through qualitatively different routes rather than one susceptibility: paradigm-label gating with explicit override (Asch conformity, 0 percent blind to 83.3 percent named on gpt-oss-120B), knowledge-dependent signal reliance (anchoring, exactly zero on grounded facts versus near total on invented quantities, a pattern equally consistent with rational use of the only available signal), amplification on novel content under labeling (framing), robust absence (sunk cost), and safety-mediated selection where refusal itself is the primary finding (minimal-group allocation). A one-sentence persona change (agreeableness, framed as an instruction rather than a verified trait manipulation) eliminates, dampens, or reverses these effects depending on which effect it is, arguing against any single response-bias account. We further formalize, and in two cases document empirically, three ways a psychology paradigm can fail to port to LLM agents: persona dominance, population collapse, and safety selection. We argue scalar bias-susceptibility scores obscure this structure and report replication profiles instead.
Submission history
From: Joy Bose [
]
[v1]
Thu, 23 Jul 2026 10:34:06 UTC (339 KB)