数学家要求 OpenAI 证明未使用其研究成果

另一位研究人员正就推动 OpenAI 日益令人瞩目的数学发现的数据来源向该公司提出质疑。就在一场围绕该公司模型是否从未发表研究中获益的激烈争论爆发数日后，第二位数学家站出来，指控这家 AI 巨头存在不道德且“不诚实”的行为，并且未透明说明其训练数据的来源。

数学家 Andreas Thom 在 Mastodon 上发表了一系列帖子中提出担忧：在 OpenAI 高调宣布成果之前，他和同事与 ChatGPT 聊天机器人的互动，可能为该公司在这一领域的成功作出了贡献。OpenAI 上月高调发布的 10 项成果中，有一项涉及 Thom 的专业领域——所谓的非 sofic 群；OpenAI 也承认，该成果在很大程度上建立在 Thom 和另一位数学家 Gábor Kun 此前工作的基础上。

Thom 表示，在纽约大学数学教授 Tristan Buckmaster 公开质疑该公司的 AI 模型是否曾受益于他使用 OpenAI Codex 的经历后，他开始反思自己与 OpenAI 的互动。在 OpenAI 宣布非 sofic 群成果后，数学界广泛批评该公司未承认 Thom 和 Kun 近期作出的贡献，随后该公司悄然修改了说明文稿。粗略而言，非 sofic 群是无法由有限结构近似的无限数学结构。

Thom 表示，他也对“OpenAI 对我们技术方法的详细掌握”感到震惊；他称，在当时，这些方法既不是最显而易见的路径，也不是最有希望通向解答的路径。他说，自己曾向 OpenAI 研究员 Sébastien Bubeck 和 Mark Sellke（后者同时是哈佛大学统计学家）发送电子邮件，询问自己与 ChatGPT 的互动是否“属于训练数据，或可被推理过程访问”，因而可能对这一成果有所贡献。

但 Thom 对得到的回答并不满意。他表示，对方只回答了自己与聊天机器人的对话能否被直接访问，并未回答这些对话是否进入公司用于改进模型的庞大训练数据池。“没有给出任何这样的限定、解释或证据，”他写道，“至少可以说，我认为这属于不诚实。”

Thom 表示，研究人员并不具备逆向工程 OpenAI 训练流程的能力，无法弄清自己的研究是否被使用。“只有 OpenAI 拥有与此相关的数据。”他说，如果公司要否认存在这种做法，举证责任就应由其承担——公司应披露所有必要的数据集，并澄清规定其如何使用数据的各种设置和条款。

OpenAI 不愿明确排除任何用户数据被使用的可能性，这与其为近期千禧年奖难题突破辩护的方式如出一辙——无论是在公开表述中，还是在与 Buckmaster 的沟通中。Buckmaster 当时正以个人身份与 Anthropic 研究员 Levent Alpöge 共同研究这一问题。在宣布 Navier-Stokes 解法的博客文章中，OpenAI 直接否认使用了任何_特定_用户数据；该问题涉及流体运动。公司写道：“我们（研究人员和 Agent）在他们公开发布研究前，未通过任何方式看到他们的任何工作——尤其是，为解决这一问题，没有访问任何特定用户数据。”

但该公司并未明确排除间接影响的可能性：“尽管可能性不大，我们无法排除源自他们使用我们产品的、经去标识化处理的数据曾帮助改进我们的模型⁠。”Thom 表示，这与公司在同他沟通时作出的模糊区分完全一样。“去标识化可能会删除姓名；但它不会删除一个数学思想的智力内容，”他说。

鉴于近期事件，Thom 表示，“Sellke 的绝对化回答至少是不当宽泛且在实质上具有误导性的；现在回看，它显然是不诚实的。”

Thom 表示，如果用户提供的非公开研究帮助改进了模型，而公司随后又用这些模型与同一批用户竞逐发表优先权，且没有获得同意、进行适当披露或给予署名，“这将在伦理上无法辩护”。

OpenAI 没有立即回应 The Verge 的置评请求。

Thom 的评论进一步加剧了数学界对 OpenAI 日益增长的不安——而这本应是该公司的一个凯旋时刻。该公司宣布解决了数学中一个传奇性的千禧年奖难题，这是一项非凡成就；如果得到验证，几乎无人会否认其意义。但 OpenAI 所称其最初决定研究该问题的特殊背景，使事情变得复杂：公司在线上听闻其他研究人员取得重大进展的传言，因而认为自己也可以一试。

这起持续发酵的事件给数学家们留下了不好的感受。多位研究人员向 The Verge 表示，他们担心此类行为会将该领域推向更保密的状态：如果数学家知道，即使只是有关他们接近重大突破的传言，也可能引发一场竞赛——而对手是一家资金雄厚、渴望赢得声誉的科技巨头——他们就会更加谨慎地保守研究秘密。

Another researcher is challenging OpenAI about the data driving its increasingly impressive array of mathematical discoveries. Just days after a bitter row erupted over whether the company’s models benefited from unpublished work, a second mathematician has come forward accusing the AI giant of unethical and “dishonest” behavior and a lack of transparency about the origins of its training data.

In a series of posts on Mastodon, mathematician Andreas Thom raised concerns that interactions he and his colleagues had had with the ChatGPT chatbot before OpenAI’s triumphant announcement may have contributed to its success in the field. One of the 10 results OpenAI announced with great fanfare last month involved Thom’s area of expertise, so-called non-sofic groups, and OpenAI acknowledged that their result built heavily on previous work by Thom and fellow mathematician Gábor Kun.

Thom said he began reflecting on his own interactions with OpenAI after Tristan Buckmaster, a mathematics professor at New York University, publicly questioned whether the company’s AI models had benefited from his use of OpenAI’s Codex. After OpenAI announced its non-sofic groups result, it was widely criticized in mathematical circles for failing to acknowledge recent contributions from Thom and Kun and the company quietly amended its writeup. Non-sofic groups are, roughly speaking, infinite mathematical structures that cannot be approximated by finite ones.

Thom said he was also struck by “OpenAI’s detailed command of our techniques,” which he said were neither the most obvious nor the most promising routes to a solution at the time. He said he wrote emails to OpenAI researchers Sébastien Bubeck and Mark Sellke, also a statistician at Harvard, to ask whether his interactions with ChatGPT were “part of the training data or accessible to the reasoning process” and could therefore have contributed to the result.

But the answer did not satisfy Thom, who said it only addressed whether his conversations with the chatbot could be accessed directly, not whether they had entered into the vast pools of training data the company uses to improve its models. “No such qualification, explanation, or evidence was given,” he wrote. “I take this as dishonesty to say the least.”

Thom said researchers aren’t equipped to reverse-engineer OpenAI’s training pipeline to figure out whether their work has been used or not. “Only OpenAI has the relevant data for that.” If the company is going to deny doing this, he said the responsibility is on them to prove that by disclosing all necessary datasets and clarifying various settings and terms setting out how it uses data.

OpenAI’s reluctance to conclusively rule out any use of user data echoes the way it defended its recent Millennium Prize breakthrough, both in its public messaging and its communications with Buckmaster — who was working on the problems with Anthropic researcher Levent Alpöge in a personal capacity. In the blog post announcing the Navier-Stokes solution, which concerns the movement of fluids, OpenAI flatly denied using any specific user data: “We (the researchers and the agents) did not see any of their work through any means until they released it publicly — in particular, no specific user data was accessed in order to solve this problem.”

But it would not conclusively rule out an indirect influence: “While unlikely, we cannot rule out that de-identified data derived from their usage of our products helped improve our models⁠.” Thom said it is the same obfuscatory distinction the company drew in its communications with him. “De-identification may remove a name; it does not remove the intellectual content of a mathematical idea,” he said.

In light of recent events, Thom said “Sellke’s categorical answer was, at minimum, unjustifiably broad and materially misleading; looking back it was plainly dishonest.”

Thom said it “would be ethically indefensible” if nonpublic research supplied by users helped to improve models that the company then used to race those very same users to publication, without consent, proper disclosure, or credit.

OpenAI did not immediately respond to The Verge ’s request for comment.

His comments add to mounting unease over OpenAI in mathematical circles at what should be a moment of triumph for the company. Its announced solution to one of mathematics’ legendary Millennium Prize problems is an extraordinary achievement that, should it be verified, few would deny. But this was complicated by the unusual circumstances OpenAI said led it to pursue the problem in the first place: It heard rumors online that other researchers had made major progress and thought it would try too.

The ongoing incident has left a sour taste in mathematicians’ mouths. Numerous researchers told The Verge they worry behavior like this will push the field into a more secretive state if mathematicians know that even rumors they are close to a big breakthrough could ignite a race with a well-resourced tech giant eager for glory.

Follow topics and authors from this story to see more like this in your personalized homepage feed and to receive email updates.

• Robert Hart

• AI

• News

• OpenAI

• Science