摘要：组织需要在预算约束下决定对哪些对象进行干预，并希望在部署前了解某个目标策略原本能够获得怎样的收益。离策略评估（Off-Policy Evaluation）承诺可从日志数据中推断这一结果，但可部署的规则是确定性的 Top-k 策略：它消除了对动作的所有平均，因此重叠性不足会直接影响估计结果。我们在五个数据集和两组已知效应扫描实验中对六种估计器进行了基准测试，并通过一个非模拟的配对参考验证其作用机制。第一，重叠性不足由日志策略与目标策略在动作上的对齐程度决定，而不只是由日志策略的确定性决定：决定支持集的是日志策略选择目标策略动作的概率。在测试范围内，基于目标策略自身评分构建的日志策略即使变得更尖锐，对重叠性的影响也很小；而动作层面的分歧会使其迅速崩塌。有效样本量能够对不同日志环境下的这一风险进行排序，但对于实践者手中单份日志内的候选策略排序效果较弱，且其阈值无法迁移。第二，优化器诅咒无法仅靠对结果模型的干扰项进行交叉拟合来解决。当规则在用于评估它的数据上拟合时，仅对干扰项进行交叉拟合会保留数据复用偏差，并使其进一步恶化。诚实的策略级数据拆分通过估计学习过程的价值来避免复用问题——这改变了估计目标（estimand），而不是对全样本策略进行去偏。第三，倾向概率估计误差是我们测得的最大性能退化来源：样本外折估计对 IPS 的损害超过我们施加的任何其他压力测试，对双重稳健估计几乎没有影响，并且甚至可能颠倒重叠性诊断本身。日志数据为合成生成，倾向概率下限设为 0.02，因此所有失效均发生在权重有界的条件下；这一下限还使两个经过调参的混合方法退化为各自未调参的基础方法，最终仅剩四种在实践中具有区别的估计器。所有精确价值曲面均为合成或半合成数据。我们发布了该基准；仅使用公共数据。

Abstract:Organizations decide whom to treat under a budget and want to know what a targeting rule would have earned before deploying it. Off-policy evaluation promises this from logged data, but the deployable rule is a deterministic top-k policy: it removes all averaging over actions, so weak overlap hits the estimate directly. We benchmark six estimators across five datasets and two known-effect sweeps, and validate the mechanisms against a non-simulated paired reference. First, weak overlap is governed by logger-target action alignment, not by logging sharpness alone: what governs support is the logger's probability of the target's actions. Sharpening a logger built from the target's own score barely moves overlap over the tested range; action-level disagreement collapses it. Effective sample size ranks this risk across logging environments, but is weak at ranking candidates within the single log a practitioner holds, and its cut point does not transfer. Second, the optimizer's curse is not fixed by cross-fitting the outcome nuisance. When the rule is fit on the data used to evaluate it, cross-fitting the nuisance alone leaves the reuse bias in place and makes it worse. Honest policy-level splitting avoids the reuse by targeting the learning procedure's value -- a change of estimand, not a de-biasing of the full-sample policy. Third, propensity-estimation error is the largest degradation we measure: an out-of-fold estimate hurts IPS more than any other stress we apply, leaves doubly-robust estimation almost unchanged, and can invert the overlap diagnostic itself. Logging is synthesized and propensities floored at 0.02, so every failure occurs with bounded weights; the floor also reduces the two tuned hybrids to their untuned parents, leaving four practically distinct estimators, and all exact-value surfaces are synthetic or semi-synthetic. We release the benchmark; public data only.