PluraMath：将数学推理评估扩展至高资源语言之外
英文标题：PluraMath: Extending Mathematical Reasoning Evaluation Beyond High-Resource Languages
数学推理已成为评估与调优推理型大语言模型（LLMs）的核心任务，但现有基准仍严重偏向英语和中文等高资源语言。新发布的 PluraMath 数据集将 PolyMath 拓展至 18 种额外代表性不足的语言，覆盖 6 个语系，并由母语者人工验证翻译质量。基于该数据集对 27 款推理 LLM 的跨规模基准测试显示，高资源语言与低资源语言之间在数学推理性能上仍持续存在明显差距，更好的指令遵循能力与更强表现密切相关。作者已完全开源数据集、构建流程与评估框架，旨在降低为弱势语言社区构建多语言基准的门槛。
2.6 KB
2.1 KB
672 B
612 B
380 B
9.1 KB
22.8 KB
24.2 KB
34.7 KB
中文
提交于 2026年7月7日
PluraMath：将数学推理评估扩展至高资源语言之外
作者： Daryna Dementieva, Nikolay Babakov, Kathy Hämmerl, Ilseyar Alimova, Jindřich Libovický, Shu Okabe, Miras Baisbay, Lukas Edman, Abrorkhon Inomkhujaev, Antonia Karamolegkou, Mateusz Lango, Volkan Özer, Nikola Selic, Subhankar Swain, Tsedeniya Kinfe Temesgen, Galit Bary Weisberg, Alexander Fraser
查看题为《PluraMath：将数学推理评估扩展至高资源语言之外》的论文 PDF，作者 Daryna Dementieva 等 16 人 查看 PDF
摘要： 数学推理已成为评估和调优推理型大语言模型（LLMs）的核心任务，然而现有基准测试仍然严重偏向高资源语言，英语和中文主导了预训练语料库和评估套件。最近发布的 PolyMath（Wang 等人，2025）数据集迈出了重要一步，但其覆盖范围仍局限于 18 种高资源语言。为了填补这一空白，我们推出了 PluraMath，它将 PolyMath 扩展至 18 种额外的代表性不足的语言，涵盖 6 个语系——从中等资源到极低资源场景。我们通过人工策划的流程构建数据集，由母语者全面验证预先计算的翻译。利用 PluraMath，我们对 27 个推理型 LLM 在四个模型规模（小型、中型、大型和闭源集成）上进行了基准测试，探究了最先进模型在不同语言条件下的多语言数学推理能力。我们的细粒度分析确认了高资源语言与代表性不足语言之间在数学推理性能上持续存在的差距，更强的结果很大程度上与更好的指令遵循能力相关。我们将数据集、数据获取流程和评估框架完全开源，目的是降低为代表性不足的社区开发多语言基准测试的门槛。
提交历史
来自：Daryna Dementieva 查看邮件
[v1] 2026年7月7日周二 08:25:29 UTC (6,751 KB)
English / 原文
[Submitted on 7 Jul 2026]
Title:PluraMath: Extending Mathematical Reasoning Evaluation Beyond High-Resource Languages
Authors:Daryna Dementieva
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
View a PDF of the paper titled PluraMath: Extending Mathematical Reasoning Evaluation Beyond High-Resource Languages, by Daryna Dementieva and 16 other authors
Abstract:Mathematical reasoning has become a central task for evaluating and tuning reasoning Large Language Models (LLMs), yet existing benchmarks remain heavily biased toward high-resource languages, with English and Chinese dominating both pre-training corpora and evaluation suites. The recently released PolyMath (Wang et al., 2025) dataset represents a significant step forward, yet its coverage is still limited to 18 only high-resource languages. To address this gap, we introduce PluraMath, an extension of PolyMath to 18 additional {underrepresented languages spanning 6 language families -- ranging from mid-resource to extreme low-resource settings. We constructed the dataset through a human-curated pipeline, where native speakers thoroughly validated pre-computed translations. Using PluraMath, we then benchmark 27 reasoning LLMs across four model scales -- small, mid-size, large, and closed-source ensembles -- probing the multilingual mathematical reasoning capabilities of state-of-the-art models under diverse linguistic conditions. Our fine-grained analysis confirms a persistent gap in mathematical reasoning performance between high-resource and underrepresented languages, with stronger results largely associated with better instruction-following ability. We fully open-source our dataset, data acquisition pipeline, and evaluation framework, with the goal of lowering the barrier to multilingual benchmark development for underrepresented communities.
Submission history
From: Daryna Dementieva [
]
[v1]
Tue, 7 Jul 2026 08:25:29 UTC (6,751 KB)