摘要：资深科学家擅长权衡来自多种来源的实证结果，这些结果在统计可靠性与认识论相关性（epistemic relevance）上各有不同。越来越广泛地驱动自主科学发现系统的语言模型，在多大程度上具备这种权衡证据的能力？为此，我们推出了 EDB-Bio——一个基于药物扰动转录组学（drug-perturbation transcriptomics）、CRISPR 筛选和人类全基因组关联分析（GWAS）三大生物学领域真实数据构建的科学推理基准。EDB-Bio 包含 9.6 万个测试序列（episodes），每个序列呈现来自已发表研究的一连串实验结果，并要求模型预测目标假设为真的概率。评测表明，无论是开源权重模型还是前沿模型，在开箱即用的情况下均表现出系统性的低自信度（underconfidence），其 Brier 得分仅相当于简单的基础概率预测器。而通过针对严格真评分规则（strictly proper scoring rules）进行后训练，可以显著提高模型的校准度与判别力。这种校准能力在下游任务中成效斐然：在连续实验选择任务中，我们的 Ibis0 模型效率达到了最佳前沿模型的 2.5 倍。纵观整个研究，将证据权重定量表示为贝叶斯因子（Bayes factors），揭示了模型信念如何随时间更新、它们与生物学专家在相关性判断上是否一致，以及它们在哪些方面存在不一致与偏见。我们的工作将校准训练和不确定性分析扩展到了现实的科学预测场景中，并应用于人类癌症生物学。我们开源了所有代码、数据和训练好的模型。

Abstract:Expert scientists excel at weighing empirical findings from multiple sources that vary in both statistical reliability and epistemic relevance. To what extent are language models, which are increasingly driving autonomous discovery systems, capable of such evidence deliberation? We introduce EDB-Bio, a scientific reasoning benchmark curated from real-world data in three biological domains: drug-perturbation transcriptomics, CRISPR screens, and human genome-wide association studies (GWAS). Each of EDB-Bio's 96k episodes presents a sequence of experimental results from published studies and asks the model to predict the probability that a target hypothesis is true. Out-of-the-box, open-weight and frontier models alike demonstrate systematic underconfidence and attain Brier scores comparable to a trivial base-rate forecaster. Post-training against strictly proper scoring rules significantly improves both calibration and discrimination. This calibration pays off downstream: in a sequential experiment selection task, our Ibis0 model is 2.5 times more efficient than the best frontier model. Throughout, quantifying evidence weights as Bayes factors reveals how models update their beliefs over time, whether they agree with expert biologists on relevance, and where they exhibit inconsistencies and biases. Our work extends calibration training and uncertainty analysis to a realistic scientific forecasting setting with applications to human cancer biology. We release all code, data, and trained models.