宝可梦 2v2 双打对战：探究 LLM 多 Agent 协同与沟通协商能力
英文标题：🔶 LLM agents play Pokemon showdown
研究者 Oliver Grabner 基于 Pokémon Showdown 的 2v2 双打规则构建了 Gym 风格的多 Agent 评测环境，用于测试大模型之间的即时沟通、纠错与协同决策能力。实验结果显示，支持自由交流的 Qwen3.8-27B 团队对抗基线 Bot 的胜率从无交流时的 42% 提升至 64%，甚至优于单一模型同时控制两只宝可梦的 54%，但在带来显著胜率增益的同时 Token 消耗也增长了近 3.8 倍。这项研究为非完全信息博弈场景下的多 Agent 协作评估与强化学习（RL）训练提供了全新基准。
159.7 KB
198.4 KB
中文
宝可梦 2v2 环境中的多 LLM Agent 协同
2026 年 10 月 10 日
https://github.com/OliverGrabner/multi-agent-poke-showdown
LLM 之间的协作是未来的发展方向，因此评估 Agent 在协同工作、沟通交流以及行动协商方面的能力至关重要。目前大多数多 Agent 研究主要集中在工作流领域，即 Agent 沿着既定流水线向下游传递任务（Hong et al., 2024；Qian et al., 2024）。为了对此进行深入测试，我基于 Pokémon Showdown 的 2v2 双打格式构建了一个多 Agent 环境，以评估这些能力。
Qwen3.8-27B - 玩家 1
"对方两只都是水属性……所以我的火属性招式对它们都会被减半。……我打算对吃吼霸（Dondozo）使用鬼火（Will-O-Wisp）……你能先手使用击掌奇袭（Fake Out）让其中一个在本回合畏缩吗？"
Qwen3.8-27B - 玩家 2
"对吃吼霸使用鬼火是个好主意。不过快速纠正一下——格斗属性实际上并不被水属性或地面属性抵抗，所以我的近身战（Close Combat）对两者都是 1 倍基础伤害。……锁定招式：击掌奇袭 → 鲶鱼王（Whiscash）。"
Qwen3.8-27B - 玩家 1 选择：鬼火（Will-O-Wisp）→ 对方吃吼霸（p4）
Qwen3.8-27B - 玩家 2 选择：击掌奇袭（Fake Out）→ 对方鲶鱼王（p2）
两位 Qwen3.8-27B 队友之间真实对战的第 1 回合（节选）。它们合理分配了打击目标，其中一位队友还在做出最终决定前指出了对方在属性克制方面的理解错误。观看带队伍聊天的完整对战。
为什么选择宝可梦？
宝可梦双打对战属于非完全信息的博弈论环境。对手的招式配置是隐藏的，且技能伤害存在随机浮动，因此取胜需要真正的策略：揣摩对手队伍意图、分配打击目标以及围绕队友配合制定计划。近期诸如 PokéAgent Challenge（Karten et al., 2026）等工作为宝可梦对战建立了基准测试，但对战双方均由单一 Agent 操控，且专用的强化学习（RL）Agent 在该场景下表现依然优于通用 LLM。这并没有对必须相互协同的队友进行测试。
本项目的目标是搭建一个可控环境，既能量化 LLM Agent 的协同能力，后续也能用于强化学习训练以促进此类协同行为的产生。
方法
环境
采用官方 Pokémon Showdown 模拟器，封装为 Gym 风格的环境。共 4 名玩家，每方 2 名，各自从 3 只宝可梦组成的队伍中操控 1 只。
Agent
每个回合，Agent 会阅读一段纯英文的对战局势摘要以及一个由合法可选招式组成的带编号列表。它可以向队友发送私信 say，或者选择执行一个招式 choose；一旦其做出选择，队友就会得知其所选内容。
对话模式
- 自由交流（Free talk）：双方持续沟通，直到均做出招式选择；
- 各发言一次（One message each）；
- 禁止交流（No talk）；
- 单模型同时控制两只宝可梦（One model controls both）。
评测设定
由 Qwen3.8-27B 分别扮演两位队友，迎战基于硬编码的最大伤害基线 Bot（Max-damage bot）。对战设定固定随机种子，且每组抽取的队伍阵容都会互换场地进行对局，确保每种对话模式面临完全相同的对战条件。
实验结果
不同对话条件下对抗最大伤害基线 Bot 的胜率：
| 条件 | 胜率（30% - 50% - 70%） | 每场对战消耗 Token 数量 |
|---|---|---|
| 自由交流（Free talk） | 64% | 978k |
| 单模型控制两只宝可梦 | 54% | 450k |
| 各发言一次（One message each） | 52% | 654k |
| 禁止交流（No talk） | 42% | 259k |
交流确实能带来提升。可以自由交流的队友面对最大伤害 Bot 取得了 64% 的胜率，而完全无法交流的队友胜率仅为 42%。即使每人只说一句话（52%），表现也优于完全不交流。单模型控制两只宝可梦（54%）的表现虽好于完全不交流，但不及两个 Agent 自由交流，这表明能力提升来自于对话交互本身，而非仅仅是将两只宝可梦的行动置于单一模型的统筹思考之下。
但沟通也有其成本。自由交流模式下平均每场对战消耗 97.8 万 Prompt Token，是禁止交流模式（25.9 万 Token）的 3.8 倍，因为每发送一条消息都意味着需要进行一次新的模型调用，并重新读取迄今为止的对战状态和聊天记录。
开启自由交流的 Qwen3.8-27B 团队对抗不同对手的表现：
| 对手 | 胜率 |
|---|---|
| Qwen3.5-2B 团队 | 96% |
| Qwen3.5-9B 团队 | 74% |
| 最大伤害基线 Bot（Max-damage bot） | 64% |
我还验证了该环境是否具备区分不同模型实力的能力。Qwen3.8-27B 团队在面对 2B 模型队伍时取得了 96% 的胜率，对抗 9B 模型队伍的胜率为 74%，对抗最大伤害 Bot 的胜率则为 64%。对手越强，比赛悬念越大，这也证明该环境足以作为一个评估比较不同模型能力的基准测试（Benchmark）。
下一步计划
我目前正在将「对话交流带来的增益」与「看见队友招式带来的增益」进行解耦分析。此外，我也在考虑将该环境打造成一个支持跨模型排行榜的 Benchmark，并将其用于强化学习（RL）训练以激发协同合作行为。
致谢
通过这个项目，我希望能更深入地理解 Agent 协同与 RL 环境，并非常乐意分享这些成果！本次实验均在德州农工大学（Texas A&M）的 Grace 集群上完成。感谢 Maria Teleki，以及我的导师 James Caverlee 教授 在整个研究过程中给予的指导与反馈。
English / 原文
Multi-agent LLM coordination in a Pokémon 2v2 environment
October 10, 2026
https://github.com/OliverGrabner/multi-agent-poke-showdown
Watch a battle with team chat↗
The future is in LLM collaborations, and it is important to be able to understand how good agents are at working together, communicating and negotiating actions. Most multi-agent research is about workflows, where agents pass tasks down a pipeline (Hong et al., 2024; Qian et al., 2024). To test this, I built a multi-agent environment on Pokémon Showdown's 2v2 format to evaluate these abilities.
Qwen3.8-27B - Player 1
"Both opps are Water types … so my Fire moves are resisted on both. … I plan to Wisp Dondozo … can you open with Fake Out to flinch one of them this turn?"
Qwen3.8-27B - Player 2
"Good call on Wisp-ing Dondozo. Quick correction though - Fighting isn't actually resisted by Water or Ground, so my Close Combat is neutral (1x) on both. … Locking in Fake Out → Whiscash."
Qwen3.8-27B - Player 1 chose Will-O-Wisp → opposing Dondozo (p4)
Qwen3.8-27B - Player 2 chose Fake Out → opposing Whiscash (p2)
Turn 1 of a real battle between two Qwen3.8-27B teammates (excerpts). They split targets, and one corrects the other's type-matchup mistake before committing. Watch the full battle with team chat.
Why Pokémon?
Pokémon doubles is a game-theory environment without full information. The opponents' sets are hidden and damage is random, so winning takes real strategy: reading the other team, splitting targets and planning around your partner. Recent work like the PokéAgent Challenge (Karten et al., 2026) benchmarks Pokémon battling, but each side is played by a single agent, and specialist RL agents still outplay general LLMs there. It doesn't test teammates who have to coordinate.
The goal is a controlled environment where I can measure how LLM agents coordinate, and that can later be used to train them with RL to encourage these collaborative behaviors.
Methodology
Environment
The official Pokémon Showdown simulator, wrapped as a Gym-style environment. Four players each control one Pokémon from a team of three, two per side.
Agents
Each turn, an agent reads a plain-English summary of the battle and a numbered list of its legal moves. It can say a private message to its partner or choose a move, and once it chooses, its partner is told what it picked.
Talk modes
Free talk (keep talking until both have chosen), one message each, no talk, and one model controlling both Pokémon.
Evaluation
Qwen3.8-27B plays both teammates against a scripted max-damage bot. Battles are seeded and every team draw is played from both sides, so each talk mode faces exactly the same battles.
Results
| Win rate by talk condition vs. max-damage bot baseline Condition | 30%50%70% | Tokens per battle | |
|---|---|---|---|
| Free talk | 64% | 978k | |
| One model controls both | 54% | 450k | |
| One message each | 52% | 654k | |
| No talk | 42% | 259k |
Talking helps. Teammates who could talk freely won 64% of their battles against the max-damage bot, while teammates who couldn't talk at all won 42%. Even a single message each (52%) beat no talk. One model controlling both Pokémon (54%) did better than no talk but worse than two agents talking freely, so I think the gain comes from the conversation itself, not just from one mind planning both moves.
Talking also costs. Free talk used 978k prompt tokens per battle, 3.8x the 259k of no talk, since every message means another model call that re-reads the battle and the conversation so far.
Qwen3.8-27B team with free talk, against Opponent| Win rate|
---|---|---
Qwen3.5-2B team| | 96%
Qwen3.5-9B team| | 74%
Max-damage bot| | 64%
I also checked that the environment can tell models apart. A Qwen3.8-27B team won 96% of its battles against a team of 2B models and 74% against a team of 9B models, compared with 64% against the max-damage bot. The stronger the opponent, the closer the games, which is what makes the environment usable as a benchmark for comparing models.
Next
I'm separating the effect of talking from the effect of seeing your partner's move. I'm also looking into turning the environment into a benchmark with a leaderboard across models, and using it for RL training to encourage collaborative behaviors.
Thanks
Through this project I wanted to learn more about agent collaboration and RL environments, and thought I'd share the results! I ran these experiments on Texas A&M's Grace cluster. Thank you to Maria Teleki and to my advisor, Prof. James Caverlee, for the guidance and feedback along the way.