OpenAI 希望咨询顶尖数学家，以避免再次搞砸

在将一系列惊人的数学成果变成一场声誉危机后，OpenAI 正在征询人类数学家的意见，以寻找一条不那么灾难性的前进道路。

周一，该公司宣布成立一个新的独立数学家小组，负责就 AI 公司与数学研究及更广泛数学界之间的互动提供建议，其中包括如何展示和发布新成果。该小组的突然出现令许多数学家感到意外。研究人员告诉 The Verge，这个小组是一个不错的第一步，但许多人仍有一些基本疑问：它究竟会做什么、有多大影响力、OpenAI 是否真的会听取其意见；他们也担心，这样一个由知名研究者组成的小型团体能否代表更广泛的数学界。

这个由九人组成的小组汇集了研究人员向 The Verge 所称的该领域“明星人物”，成员来自斯坦福大学、哈佛大学、牛津大学和剑桥大学等机构，其中多人曾获菲尔兹奖和麦克阿瑟“天才奖”。小组将由新泽西州普林斯顿的高等研究院（Institute for Advanced Study，IAS）托管；该机构是全球最受尊崇的数学研究中心之一，阿尔伯特·爱因斯坦、约翰·冯·诺伊曼和J·罗伯特·奥本海默等人都曾在此从事学术工作。

“另一个基本事实是，我们的总体目标，是尽可能充分地代表数学界的利益。”

OpenAI 表示，这个名为“数学与人工智能咨询小组”（Advisory Group on Mathematics and Artificial Intelligence，AGMAI）的团体将独立运作，有权提出公司未曾要求的建议、公开评论 AI 对数学的影响，并公开其建议，不过 OpenAI 没有说明这些安排具体会如何落实。为完成其任务，该小组似乎将获得对 OpenAI 研究的提前访问权限。OpenAI 称，成员不会从 OpenAI 获得报酬，小组也可自行决定调整成员构成。“其价值取决于成员能否行使自己的判断，并对我们的观点提出挑战，”该公司表示。“该小组将就新兴成果的审查与沟通提供建议：帮助 OpenAI 评估其重要性，就如何协调传播提供建议，并就数学研究的学术与专业标准提供建议。它还将就我们的工具如何支持数学研究与学习提供建议。”

尽管反复强调独立性，该小组实际上将拥有多大影响力、以及它最初为何成立，仍不清楚。瑞士苏黎世联邦理工学院（ETH Zurich）的研究员 Simon Machado 表示，整件事似乎“笼罩在神秘之中”，他还质疑九名成员是如何被选定的。该小组的简陋网站称，它“是在 OpenAI 接触部分成员、寻求建立外部咨询委员会之后组建起来的”；但在与公司讨论后，这些数学家决定成立一个独立团体，并邀请其他人加入。目前尚不清楚哪些成员最初曾被 OpenAI 接触，以及其余成员是如何选出的。

咨询小组成员 Martin Hairer 周二在一篇博客文章中进一步说明了该小组的起源及其背后的考量，以回应他所说围绕其成立而出现的“海量错误信息”。“显而易见，AI 已经对数学研究产生深远影响，并引发了关于思想正确归属、优先权、人类对思想的理解等诸多问题，”他写道，并补充称，数学界没有必要采取“把头埋进沙子里”的强硬做法，即仅仅出于原则而无视 AI 实验室、拒绝与它们对话。

Hairer 承认，该小组是为回应 OpenAI 的接触而成立的，但他强调，并非所有成员都曾受该公司联系，最终形成的组织“真正独立于 OpenAI 及其他所谓的‘前沿’AI 实验室”。除“显而易见的保密要求”外，他表示成员没有签署限制其公开发言内容的协议；他还称，成员不获得任何经济报酬，技术支持则依赖 IAS。

“另一个基本事实是，我们的总体目标，是尽可能充分地代表数学界的利益，”Hairer 写道。他是伦敦帝国理工学院以及洛桑联邦理工学院（EPFL）的数学教授。他指出，该小组显然并非数学界的代表性样本，但表示自己“绝对确信，我们九个人都极其严肃地对待这件事，并且是出于善意行事”。

无论成员是如何选出的，这个小组目前都由一批顶尖学术数学家构成。鉴于数学前沿研究需要高度专业化的技能与知识，小组规模之小也进一步引发了疑问：它是否具备足够广泛的专业能力，以评估未来提交给它的一切内容。

即将担任苏格兰赫瑞瓦特大学数学教授的 Francesco Fournier-Facio 表示，他对该委员会的态度相当“矛盾”，但认为与数学家进行更多互动“只会让事情变得更好”。不过，他质疑这个小组是否充分反映了这样一个群体：其工作和生计正因 OpenAI 争夺数学奖杯的行动而受到颠覆。“它感觉像一座象牙塔，”他告诉 The Verge。

Machado 同样质疑，该小组成员是否真正理解、或能够代表更广泛数学界对于 AI 的担忧。“他们是出色的数学家，但我不知道，在更具政治性的问题上，他们是否是我希望代表我的人，”他说。Machado 表示，对于处于他们那样、或类似精英位置的数学家而言，“我认为他们面对的现实，与目前大多数数学家面对的现实非常不同”。

该小组能否赢得数学家的信任，最终将决定其在遏制 OpenAI 最近一连串发布成果失误所造成后果方面的成效；更重要的是，它能否为该公司下一次计划公布的内容减轻冲击。Hairer 在博客中称，成员若相信 AI 实验室不会试图以有利于自身公关机器的方式包装他们的言论，那就“非常天真”；而这种公关机器的能力“远非我们可能提出的任何东西可比”。

不过，Hairer 认为，既然 OpenAI 看起来正在努力改进，如今拒绝参与将是虚伪的，尤其是在如此多人批评过该公司的行为和糟糕沟通之后。他表示，该小组尚未确定将提出哪些具体建议，但其所回应的问题已经很明确：“事实是，AI 公司近几个月来一直在产出一些备受瞩目的数学成果，而无论如何定义可接受的数学实践，这些成果的发表与传播方式都远未达到可接受标准。”

“我们目前正面临一项非常具体的挑战：就 OpenAI 如何协调发布大量重要数学成果向其提供建议；该公司称，这些成果由其内部模型产出。”

目前，Hairer 表示，该小组正通过与同事交流、公开的反馈表单以及Proofs and Prompts上的讨论收集意见。这个博客已成为数学家审视 AI 对该领域影响的重要论坛。

但该小组最大的挑战可能还在后面。在宣布咨询小组时，OpenAI 随意透露了一项消息：其尚未发布的新模型——也就是解决纳维-斯托克斯千禧年大奖问题的同一模型——如今已经“解决了数学大多数领域中超过 100 个长期未解问题”。该公司此前曾表示，自己正接近解决另一个千禧年大奖问题。该小组在其网站上表示，处理这一积压问题是其首要任务：“我们目前正面临一项非常具体的挑战：就 OpenAI 如何协调发布大量重要数学成果向其提供建议；该公司称，这些成果由其内部模型产出。”

从某种程度上说，这项与数学家接触的新举措，是对其先前尝试所造成重大公关灾难的承认。当时，该公司不得不应对抢先发表研究成果的指控，被批评未能对其模型所依赖的人类工作给予署名，甚至被指从使用其模型的人手中窃取成果。这些争议让许多数学家开始怀疑，这家公司是否理解——甚至是否尊重——它强行闯入的这门学科及其社群。

经历过去几个月、并已有如此多关于数学家诉求的讨论之后，伦敦帝国理工学院数学教授 Kevin Buzzard 想知道，OpenAI 为什么认为自己需要这样一个精英机构，才能了解数学界想要什么。“问题不在于证明艰深定理，而在于更好地理解我们的学科，”他说。“在我看来，并不完全清楚为什么需要一个由杰出人士组成的委员会，才能把这一点讲明白。”

After turning a string of spectacular mathematical results into a reputational crisis, OpenAI is consulting human mathematicians to help it figure out a less disastrous path forward.

On Monday, the company announced a new independent panel of mathematicians tasked with advising it and other AI companies on their interactions with mathematical research and the wider mathematics community, including how new results are presented and released. Its abrupt arrival caught many mathematicians by surprise. Researchers told The Verge the group is a good first step, but many said they were left with basic questions about what it will actually do, how much influence it will have, and whether OpenAI will actually listen to it, as well as concerns over whether such a small group of prominent researchers can represent the wider mathematical community.

The nine-member group is stacked with who researchers described to The Verge as celebrities in the field, drawn from institutions including Stanford, Harvard, Oxford, and Cambridge, with multiple Fields Medals and MacArthur “genius grants” among them. It will be hosted by Princeton, New Jersey’s Institute for Advanced Study (IAS), one of the world’s most revered centers of mathematical research and the former intellectual home of figures including Albert Einstein, John von Neumann, and J. Robert Oppenheimer.

“The other fundamental fact is that our overarching aim is to represent as well as we possibly can the interests of the mathematical community.”

OpenAI said the group, called the Advisory Group on Mathematics and Artificial Intelligence (AGMAI), will operate independently, with the freedom to offer advice the company has not requested, publicly comment on its impact on mathematics, and make its advice public, though it did not provide details on what this may entail. It appears that the group will be granted early access to OpenAI’s research in order to fulfill its task. Members will not be paid by OpenAI and the group can change its membership as it sees fit, OpenAI said. “Its value depends on its members being able to exercise their own judgement and challenge ours,” the company said. “The group will advise on the review and communication of emerging results: they will help OpenAI assess their significance, advise on how to coordinate their dissemination, and advise on academic and professional standards of mathematical research. It will also advise on how our tools can support mathematical research and learning.”

For all the emphasis on independence, it remains unclear how much influence the group will actually have or why it was formed in the first place. Simon Machado, a researcher at ETH Zurich in Switzerland, said the whole thing seemed “shrouded in mystery” and he questioned how the nine members were selected. The group’s bare-bones website says it “came together after OpenAI approached some of its members about establishing an external advisory board,” but that, following discussions with the company, the mathematicians decided to launch an independent group instead and invite others to join. It’s not clear which members were initially approached by OpenAI or how the remaining members were selected.

In a blog post published Tuesday, advisory group member Martin Hairer elaborated on the group’s origins and the thinking behind it, responding to what he called a “torrent of misinformation” surrounding its formation. “It is obvious that AI has already had a profound impact on mathematical research and raises numerous questions of correct attribution of ideas, priority, human understanding of ideas, etc.,” he wrote, adding that there is no sense in the community “taking a hard ‘ostrich’ approach of simply ignoring the AI labs and refusing to talk to them on principle.”

While acknowledging that the group was formed in response to an approach from OpenAI, Hairer stressed that not all of its members had been contacted by the company and that the resulting organization is “genuinely independent of OpenAI and any other of the so-called ‘frontier’ AI labs.” Beyond “obvious confidentiality requirements,” he said members have not signed any restrictive agreements governing what they can say publicly, adding that they receive no financial compensation and are relying on the IAS for technical support.

“The other fundamental fact is that our overarching aim is to represent as well as we possibly can the interests of the mathematical community,” wrote Hairer, a professor of mathematics at Imperial College London and the Swiss Federal Institute of Technology in Lausanne (EPFL). While noting that the group is hardly a representative sample of the community, he said he is “absolutely certain that all nine of us take this extremely seriously and are acting in good faith.”

However the members were selected, the group currently comprises an elite cadre of academic mathematicians. The small size raises further questions about whether the group has the breadth of expertise required to evaluate everything coming its way, given the highly specialized skills and knowledge required to operate at the frontiers of mathematics.

Francesco Fournier-Facio, an incoming mathematics professor at Heriot-Watt University in Scotland, described himself as pretty “ambivalent” about the panel, saying that greater engagement with mathematicians “can only make things better.” But he questioned whether this group fully reflects the community whose work and livelihoods are being upended by OpenAI’s quest to rack up mathematical trophies. “It feels like an ivory tower,” he told The Verge.

Machado similarly questioned whether the group’s members fully understand, or can speak to, the concerns of the broader mathematical community when it comes to AI. “They’re amazing mathematicians, but I don’t know if they are the people I want to represent me in more political questions,” he said. For mathematicians in their and similarly elite positions, Machado said, “I think that the reality they are facing is very different from the reality that most mathematicians are facing right now.”

Whether the group can earn mathematicians’ trust will ultimately determine how effective it will be at containing the fallout from OpenAI’s recent parade of missteps in announcing results and, more importantly, soften the blow for whatever the company is planning to announce next. In his blog, Hairer said members would “be very naïve to believe that the AI labs won’t try to spin whatever we say in a way that suits their PR machine, which dwarfs anything we could possibly come up with.”

Nevertheless, Hairer argued it would be hypocritical not to engage now that OpenAI appears to be making an effort, particularly after so many criticized the company’s conduct and poor communications. The group has yet to settle on the advice it intends to provide, he said, but there is a clear problem it is responding to: “It is a fact that AI companies have in recent months been producing some high profile mathematical results and that their publication and dissemination has been falling far short of acceptable mathematical practice, whichever way that is defined.”

“We are currently facing the very specific challenge of advising OpenAI on how to coordinate the release of a large number of significant results in mathematics that they report have been produced by their internal model.”

For now, Hairer said the group is gathering input through conversations with colleagues, a public feedback form, and discussion on Proofs and Prompts, a blog that has emerged as a key forum for mathematicians reckoning with AI’s impact on the field.

But the group’s biggest challenge likely lies ahead. When announcing the advisory group, OpenAI casually dropped the news that its new, unreleased model — the same that resolved the Navier-Stokes Millennium Prize problem — has now “resolved more than 100 long-standing open problems across most areas of mathematics.” It has previously said it is closing in on another Millennium Prize problem. On its website, the group said dealing with this logjam is its first priority: “We are currently facing the very specific challenge of advising OpenAI on how to coordinate the release of a large number of significant results in mathematics that they report have been produced by their internal model.”

In part, the new effort at engaging mathematicians is an acknowledgement of the monumental PR disaster its attempt was, in which the company found itself fending off accusations of scooping researchers, failing to credit the human work its models built upon, and even stealing from those using its models. Those controversies led many mathematicians to wonder if the company understood — or even respected — the discipline and community it had barreled into.

After the last few months and with so much having been written about what mathematicians want, Kevin Buzzard, a mathematics professor at Imperial College London, wondered why OpenAI felt like it needed such an elite body to understand what the community wants. “It wasn’t proofs of hard theorems, it was better understanding of our subject,” he said. “It’s not entirely clear to me that you need a committee of brilliant people to hammer the point home.”

Follow topics and authors from this story to see more like this in your personalized homepage feed and to receive email updates.

• Robert Hart

• AI

• OpenAI