摘要：大语言模型（LLM）正越来越多地被应用于涉及制度和政治文本的任务，但现有基准通常在孤立文档或单一任务上对其进行评估。在现实政治中，谈判属于纵向数据：参与方可在多轮互动中达成一致或展开争论，而每一轮都是此前各轮互动的结果，因此，理解某一轮内容需要追踪此前发生的一切。我们提出 TradeVerse，这是一个基于世界贸易组织（WTO）特定贸易关注事项构建的基准；在这些事项中，成员国彼此提出质疑，并在多轮互动中交换论点，有时持续数年。我们在 TradeVerse 中重建了 $1170$ 场会议的记录，覆盖 5 个群组和 $89$ 个产品组，并定义了三项任务：第一，系统必须分析纵向会议记录，并预测特定会议中所讨论产品的协调制度代码（HS 章节）；第二，我们考察系统在分析匿名化会议内容后，能否猜出回应方国家的名称；第三，我们要求系统扮演回应方国家，并为最后一轮提供声明。所有标签均直接从会议程序中恢复，无需人工标注。我们的实验凸显了这些任务对当前 LLM 构成的挑战。据我们所知，TradeVerse 是首个研究 LLM 理解纵向政治贸易谈判潜力的基准。

Abstract:LLMs are increasingly being applied to tasks involving institutional and political texts, but existing benchmarks evaluate them on isolated documents or single tasks. In realpolitik, negotiations are longitudinal data, where participating parties can align or argue over multiple iterations and each turn is an outcome of the previous turns, hence, understanding one turn requires tracking everything before it. We introduce TradeVerse, a benchmark built from the World Trade Organisation (WTO) specific trade concerns, where member states challenge one another and exchange arguments over multiple rounds, sometimes for years. We, in TradeVerse, reconstruct minutes of $1170$ meetings, spanning across 5 groups and $89$ product groups and define three tasks: first, the system has to analyze the longitudinal meeting records and predict the harmonized system codes (HS chapters) of the products under discussion in the particular meeting, second, we examine whether the system, upon analyzing the anonymized content of the meeting, can guess the name of the responding country and third, we ask the system to play the role of the responding country and provide the statement for the very last round. All labels are recovered directly from the proceedings, requiring no manual annotation. Our experiments highlight the challenges these tasks pose for current LLMs. To the best of our knowledge, TradeVerseis the first benchmark to investigate potential of LLMs in understanding longitudinal political trade negotiations.