“纯属疯狂”:数学家或需数年才能理解 OpenAI 最新发布成果
英文标题:‘Pure insanity’: Mathematicians will need years to make sense of OpenAI’s latest drop
OpenAI 本周一次性发布了近 400 项由 AI 生成的数学结果,分散在 719 篇论文中,覆盖组合数学、几何、数论、理论计算机科学、拓扑学和数学物理等多个领域。数学家普遍认为,其中存在足以冲击黎曼猜想、Hodge 猜想和四维 Kakeya 猜想等重大问题的高质量成果,但材料规模巨大、写作质量参差且形式化验证不足,使全面评估可能需要数年。OpenAI 表示,719 篇论文中仅有 300 项顶层结果完成形式化,约占 42%,其余将陆续补充。此次发布已使部分研究者原有的研究计划和经费申请受到冲击,也加剧了学界对于 AI 批量产出数学成果、归因质量及科研生态变化的担忧。
306.9 KB
769.8 KB
659.0 KB
3.3 KB
1.1 MB
619.2 KB
113.6 KB
98.5 KB
98.3 KB
822.8 KB
230.3 KB
972.7 KB
88.4 KB
90.1 KB
114.7 KB
572.3 KB
257.2 KB
中文
“惊人。” “压倒性。” “前所未有。” “超现实。” “纯属疯狂。”
本周,OpenAI 突然向数学界投下了一场数学结果洪流。30 多位数学家在接受 The Verge 采访、试图理解这些成果时,用到了上述描述。面对这场规模庞大的冲击,人们既感到惊叹、兴奋和不确定,也对其含义以及后续发展怀有深层焦虑。尽管反应各不相同,研究者一致认为,仅仅理解 OpenAI 发布的内容就可能需要数年,更不用说弄清数学家自身在这个正在变化的领域中将处于何种位置。许多人担心,OpenAI 不会等那么久才继续推进,或发布更多成果。
OpenAI 总共发布了近 400 项由 AI 生成的结果。这些成果分布在 700 多篇论文中,涵盖广泛的数学学科,包括组合数学、几何学的多个分支、数论、理论计算机科学、代数学、拓扑学、概率与统计力学,以及数学物理。整个集合规模之大,以至于 OpenAI 专门发布了指南,说明如何浏览这个庞大的 GitHub 仓库。
工作量之大,使得即便只是初步评估这家公司究竟发布了什么也颇为困难。在发布后的数小时和数天里,多数接受 The Verge 采访的数学家表示,他们仍在努力消化全部内容;几位数学家称,仅仅浏览约 40 页的目录与摘要,就花去了他们大半个小时。康涅狄格大学数学教授 Álvaro Lozano-Robledo 表示:“光是通读所有摘要的列表就让人应接不暇。”
数百篇论文中还散布着用 Lean 编写的形式化内容。Lean 是一种编程语言和证明助手,可让结果通过计算方式得到验证。这些形式化内容已被证明对评估 OpenAI 此前的一些数学主张至关重要:即便研究者并不完全理解背后的论证,也能确信某项主张在逻辑上是正确的。
“如果 AI 现在就消失了,就像外星人来到地球后又离开了一样,我们接下来的 10 年都会用来研究这些内容,努力理解其中的一切。”
但每项结果获得验证的程度差异极大。OpenAI 在 GitHub 上承认,这些结果“处在不同的验证阶段”,且“许多、但并非全部论文都已形式化”。截至发稿时,这个集合中似乎不到一半的论文已有形式化描述。OpenAI 表示,719 篇论文中只有 300 项顶层结果已完成形式化,约占 42%;公司还称,“随着获得更多形式化内容,将更新该仓库。”
多位数学家向 The Verge 抱怨形式化不足的问题,尤其考虑到结果数量如此庞大。他们强调,即使一项结果附带 Lean 代码,评估也并非即时完成。研究人员仍须检查形式化内容是否确实证明了该结果所声称的内容,这同样是耗时的过程。一些正在研读论文的人还表示,即便提供了可由计算机验证的证明,其质量也并不一致;而且它们验证的陈述,并不总能与配套论文中的主张清晰对应。
伦敦帝国理工学院数学教授 Kevin Buzzard 表示,他已在自己的研究领域——代数数论——中发现了许多定理,但只有大约六个立刻“脱颖而出”。其中即便有,也很少看起来经过 Lean 的形式化验证。Buzzard 说:“因此,我要么得阅读可能并不正确的垃圾内容,要么等其他人也这么做,要么等有人将其形式化;在此之前,我甚至无法确定这些结果是否正确。”许多其他研究者也表达了与 Buzzard 相同的担忧。
担忧 AI“垃圾内容”(slop)的并不只有 Buzzard。这个词是对低质量、且常常存在错误的 AI 生成材料的简称。这类内容正越来越多地出现在网络上,以及现实世界中,包括学术论文。数学家告诉 The Verge,与其他领域一样,近年来他们看到由 ChatGPT 和 Claude 等工具生成的此类材料大幅增加。其中不少令人困惑、难以阅读,并且几乎没有显示出对相关学科的理解;在对其他研究者进行署名和致谢方面,尤其糟糕。
OpenAI 先前的数学文章曾受到广泛批评,专家认为其内容粗糙,尤其是归属标注不佳或完全缺失。在此次发布前与 The Verge 的交流中,几位研究者已经开始将即将到来的论文洪流称为“slopocalypse”(“垃圾末日”),或使用这一主题的类似变体。
人们所担心的“垃圾末日”是否真的出现,很难判断,主要原因就在于发布材料的数量令人眼花缭乱。早期迹象显示,OpenAI 此次对论文投入了更多谨慎,或者至少对其中一部分如此。多位数学家告诉 The Verge,他们的第一印象远好于预期;不过他们自己也承认,鉴于该公司此前发布材料的粗糙程度,这本来就不是一道很高的门槛。
“这是一团大乱。它可能导致学术文化的巨大崩塌,甚至直接毁掉大部分院系。这是一个社会问题,而 AI 实验室似乎完全忽略了这个问题。”
但有所改善不等于质量优良,更遑论达到通常对提出如此重大主张的学术工作所期待的标准。由于 OpenAI 的许多主张缺乏形式化验证,配套论文的质量便尤为重要;它们是数学家确认、理解、审查和将这些结果置于适当背景中的主要途径。
即便在最理想的情况下,撰写严谨的数学论文也是困难的工作。以这种规模完成它,则是一项艰巨任务。OpenAI 的模型正以令人目眩的速度、横跨广泛专业领域地产出数学内容,远远超出其人类员工的处理能力。这家公司根本不具备足够广泛的专业知识或资源,来恰当地审查其在数学前沿上的发现。研究人员告诉 The Verge,这一点在材料中显而易见。
许多人描述了一些难以理解、甚至几乎无法跟进的论文。布朗大学数学教授 Brendan Hassett 告诉 The Verge:“我最熟悉的问题,其文字说明在快速阅读后几乎没有什么意义。如果这是一位人写的,我不会再花更多时间去试图理解它。当然,这还剩下另外 721 篇预印本!”(Hassett 说这番话时,OpenAI 尚未撤回三篇论文。)
一些研究者告诉 The Verge,他们或同事注意到的几篇论文,似乎涵盖了其他数学家已经探索过的领域;但在有机会恰当审阅材料前,他们不愿公开如此断言。另一些人则指出,这些工作异常简短:他们通常预计需要数百页展开的结果,被压缩进几十页甚至更少的篇幅。
澳大利亚悉尼大学数学教授 Nalini Joshi 表示,她快速检索这次发布内容后,发现与自己的研究重叠之处很少;但她指出,自己查看的一些论文“参考文献很短”。考虑到 OpenAI 此前在署名实践上受到的批评,她说,自己“警惕这些论文中的归属标注可能没有呈现完整情况”。
但抛开所有垃圾内容和不确定性,整体共识是:这次发布包含了一些真正令人印象深刻的工作。
尽管强调了评估如此大量材料的困难,以及恰当验证结果的必要性,多位接受 The Verge 采访的研究者表示,这些工作尽管呈现方式存在不足,但看起来水准极高。他们说,在 AI 出现前的世界中,OpenAI 的许多成果显然足以在顶尖期刊发表,并且可能足以为作者赢得一份学术职业生涯。其中少数成果被形容为能让一位数学家成为菲尔兹奖有力竞争者的工作;菲尔兹奖是该学科最高荣誉之一。
“我要么得阅读可能并不正确的垃圾内容,要么等其他人也这么做,要么等有人将其形式化;在此之前,我甚至无法确定这些结果是否正确。”
斯坦福大学数学家 Jared Duker Lichtman 表示,有“数十项”结果会被他归入这一类别,其中包括对黎曼猜想的推进、Hodge 猜想的一个特例,以及四维 Kakeya 猜想的解答。这些都是数学中的重大问题。黎曼猜想——可谓整个学科中最著名的未解问题——与 Hodge 猜想,均属于著名的七个千禧年难题。两者分别与素数分布,以及概略而言如何用更简单的构件理解复杂几何形状有关。与此同时,Kakeya 问题大致是在问:要在所有方向上旋转一根针或铅笔,最少需要多少空间。今年早些时候,纽约大学数学家 Hong Wang 因证明 Kakeya 问题的三维版本,而被授予菲尔兹奖。
数学家 Scott Armstrong 说:“这些并不只是没人听说过的无聊问题。许多都是非常知名的问题,很多人已经尝试了数十年。”
随着尘埃开始落定,数学家们发现自己正面对一片突然在身边改变的研究景观。他们中的许多人刚刚看到自己多年工作和精心制定的研究计划在一瞬间化为乌有,或者认识经历同样情况的同事。整个领域中,无论反应夹杂着兴奋、恐惧、绝望,还是介于其间的任何情绪,人们都深深感到迷失方向。
到了第二天早晨,这种情绪没有太大改变。Armstrong 在巴黎步行时接受电话采访,他设想,如果索邦大学没有因持续的学生抗议而关闭,且同事们正聚在平日常去的咖啡机旁,氛围可能会相当“肃穆”。
在苏格兰,圣安德鲁斯大学数学教授 Colva Roney-Dougal 描述了同事和学生中类似的低迷氛围。她说,这场洪流让人感觉有些“可怕”,但在数周不确定性之后,它的到来也带来了一些解脱。“我有一批朋友和同事,他们的经费申请刚刚被彻底摧毁了。”
“我有一批朋友和同事,他们的经费申请刚刚被彻底摧毁了。”
Armstrong 表示,他知道有一个研究团队的整个研究计划几乎被这次发布“抹掉”。此前处于 OpenAI 围绕 Navier-Stokes 问题争议中心的纽约大学数学家 Tristan Buckmaster 则表示,他已经听说“有三个人的整个研究计划被彻底摧毁”。
在 The Verge 与数学家的对话中,类似故事一再出现,尽管这种冲击在该领域的一些方向尤其集中。苏格兰赫瑞瓦特大学数学教授 Francesco Fournier-Facio 表示,概率、组合数学和理论计算机科学领域的研究者似乎“尤其震惊”;他补充说,自己所在领域群论的一些区域已经被“推土机夷平”。
Armstrong 和其他研究者表示,某些领域似乎遭到了近乎军事精度的瞄准。Armstrong 说:“显然存在一些目标。”他特别提到 Wang 今年凭借相关研究获得菲尔兹奖的领域,以及数个千禧年难题。他说,数学物理中的一组结果清楚表明,OpenAI 正在追求杨-米尔斯理论——这一数学框架支撑了现代粒子物理的大部分内容——及其尚未解决的“质量间隙”问题;后者是七个千禧年难题之一。
在其他地方,研究者则因自己的工作似乎很少受到波及而表达了一种意外的宽慰。Roney-Dougal 表示,她所在的研究角落——主要以群论为中心——似乎相对安然无恙。她开玩笑说,幸好自己从事的是“非常不时髦的领域”;但她后来发消息称,在发现自己的工作被其中一篇论文引用后,她感到“并不确定”。
Joshi 同样发现与自己研究的重叠很少;她的工作主要聚焦于可积系统,comple
English / 原文
“Staggering.” “Overwhelming.” “Unprecedented.” “Surreal.” “Pure insanity.”
Those were among the descriptions more than three dozen mathematicians reached for in conversations with The Verge as they tried to make sense of the flood of mathematical results OpenAI abruptly dropped on the field this week. Amid the awe, excitement, and uncertainty over the sheer scale of the deluge was a deep-seated anxiety over what it all means — and what comes next. For all their different reactions, researchers agreed that simply understanding what OpenAI had released could take years, let alone figuring out where the mathematicians themselves fit in the field now changing around them. Many feared OpenAI would not wait that long before moving on — or releasing even more.
In all, OpenAI released nearly 400 AI-generated results. These were spread across more than 700 manuscripts and covered a diverse array of mathematical disciplines, including combinatorics, several branches of geometry, number theory, theoretical computer science, algebra, topology, probability and statistical mechanics, and mathematical physics. The collection is so vast that OpenAI felt the need to publish guidance on how to navigate the sprawling GitHub repository.
The sheer volume of work makes even a preliminary assessment as to exactly what the company has released difficult. In the hours and days following the drop, most mathematicians The Verge spoke with said they were still struggling to digest everything; several said that simply working through the roughly 40-page table of contents and abstracts took them the better part of an hour. “Just going over the entire list of abstracts is overwhelming,” said Álvaro Lozano-Robledo, a professor of mathematics at the University of Connecticut.
Sprinkled among the hundreds of manuscripts are formalizations in Lean, a programming language and proof assistant that allows results to be verified computationally. These formalizations have proven instrumental in assessing some of OpenAI’s previous mathematical claims, giving researchers confidence that a claim is logically correct even if they don’t fully understand the argument behind it.
“If the AIs would disappear now, as though there were aliens that came to Earth and then just left, we would be studying this for the next 10 years, trying to understand everything.”
But the degree to which each result had been verified varied wildly. On GitHub, OpenAI acknowledged that the results are “at different stages of verification” and that “many, but not all, of the manuscripts have been formalized.” As of writing, fewer than half the manuscripts in the collection appear to have been described formally. OpenAI said only 300 top-line results out of 719 manuscripts had been formalized, around 42 percent, and that it “will update the repository with more formalizations as we obtain them.”
Several mathematicians complained to The Verge about the lack of formalization, particularly given the sheer number of results, and stressed that even when Lean code accompanies a result, evaluation isn’t instantaneous. Researchers must check that the formalization actually proves what the result claims, another time-consuming process, and several digging through the papers said that even where computer-verifiable proofs had been provided, the quality was inconsistent and the statements they verified did not always appear to map neatly onto the claims in accompanying manuscripts.
Kevin Buzzard, a mathematics professor at Imperial College London, said he had identified numerous theorems in his area of work — algebraic number theory — of which only around six immediately “stood out.” Few, if any, of those appeared to be formally verified in Lean. “Hence, I either have to read possibly-not-correct slop, or wait for others to do the same, or wait for someone to formalise them before I can say for sure that the results are even correct.” Buzzard’s concerns were echoed by numerous other researchers.
Buzzard was far from alone in worrying about AI “slop.” The term is a shorthand for low-quality, frequently erroneous AI-generated material that increasingly crops up online — and in the real world — including academic papers. As in other fields, mathematicians told The Verge they have seen a huge uptick in such material produced with tools like ChatGPT and Claude in recent years. Much of it is confusing, hard to read, and demonstrates little understanding of the subject; it is especially shoddy when it comes to crediting other researchers.
OpenAI’s previous mathematical write-ups were widely criticized by experts for their sloppy nature, particularly their poor or nonexistent attribution. In conversations with The Verge ahead of the release, several researchers had taken to calling the impending flood of papers the “slopocalypse,” or similar variations on the theme.
Whether the feared “slopocalypse” actually materialized is difficult to say, largely due to the bewildering volume of material released. Early indications suggest OpenAI took more care with papers this time around, or at least with some of them. Several mathematicians told The Verge that their first impressions were far better than they had expected, though by their own admission that was hardly a high bar given the company’s previous shoddy publications.
“It’s a big mess. It can cause a huge collapse in the academic culture and simply kill most of the faculties. It’s a social problem and it seems that the AI labs are completely ignoring this issue.”
But better does not necessarily mean good, let alone up to the standards usually expected of academic work making claims of this magnitude. With formal verification absent for many of OpenAI’s claims, the quality of the accompanying papers becomes particularly important; they are the primary means by which mathematicians can confirm, understand, scrutinize, and contextualize the results.
Producing rigorous mathematical papers is difficult work under even the best of circumstances. Doing so at this kind of scale is a formidable undertaking. OpenAI’s models are pumping out mathematics at a dizzying speed and across a broad range of specialities, far outstripping the capacity of its human staff. The company simply does not have the breadth of expertise or resources to properly scrutinize its findings at the cutting edge of mathematics. Researchers told The Verge that it shows.
Many described papers that were difficult, sometimes practically impossible, to follow. “The write-up of the problem I know best made little sense after a quick read,” Brendan Hassett, a mathematics professor at Brown, told The Verge. “If this had been written by a person, I wouldn’t spend any more time trying to understand it. Of course, this leaves 721 other preprints!” (Hassett said this before OpenAI retracted three papers).
Some researchers told The Verge several papers they or their colleagues had noticed appeared to cover ground already trod by other mathematicians, though were wary of saying so publicly before they had a chance to properly review the material. Others pointed to the unusual brevity of the work, with results they would ordinarily expect to be developed over hundreds of pages compressed into a few dozen or less.
Nalini Joshi, a mathematics professor at the University of Sydney in Australia, said a quick search through the release revealed little that overlapped with her own work, but noted that some of the papers she examined “have short bibliographies.” Given previous criticism of OpenAI’s crediting practices, she said she is “wary that attribution in the papers may be lacking the complete story.”
But for all the slop and uncertainty, the overarching consensus is that the release contains some genuinely impressive work.
While stressing the difficulties assessing the volume of material — and the need to properly verify the results — numerous researchers The Verge spoke to said the work appeared to be of a very high caliber, despite shortcomings in its presentation. In a pre-AI world, they said, many of OpenAI’s results would clearly have warranted publication in top-tier journals and could have been enough to secure an academic career for their authors. A handful were described as being the kind of work that could make a mathematician a serious contender for a Fields Medal, one of the discipline’s highest honors.
“I either have to read possibly-not-correct slop, or wait for others to do the same, or wait for someone to formalise them before I can say for sure that the results are even correct.”
Stanford mathematician Jared Duker Lichtman said there were “tens” of results he would put in this category, including progress toward the Riemann hypothesis, a special case of the Hodge conjecture, and a solution to the four-dimensional Kakeya conjecture. These are major problems in mathematics. Riemann — arguably the most notorious unsolved problem in the entire discipline — and Hodge are both among the seven famed Millennium Prize problems. Respectively, they are concerned with the distribution of prime numbers and, very roughly, how complex geometric shapes can be understood in terms of simpler building blocks. Kakeya, meanwhile, roughly asks how little space is needed to rotate a needle or pencil in every direction. A proof to the three-dimensional version of Kakeya was among the achievements NYU mathematician Hong Wang was awarded a Fields Medal for earlier this year.
“It’s not the case that these are just silly problems that no one’s ever heard of,” mathematician Scott Armstrong said. “Many of them are like very well-known problems that many people have tried for decades.”
As the dust began to settle, mathematicians were left confronting a landscape that had abruptly changed around them. Many of them had just watched years of work and carefully laid research plans evaporate in an instant, or knew colleagues who had. Across the field, whether reactions were laced with excitement, dread, despair, or something in between, there was a profound sense of disorientation.
By the following morning, that mood had not altered much. Speaking by phone as he walked through Paris, Armstrong imagined that, had the Sorbonne not been closed amid ongoing student protests and his colleagues were gathered around their usual coffee machine, the atmosphere would have been rather “solemn.”
In Scotland, Colva Roney-Dougal, a professor of mathematics at St Andrews University, described a similarly bleak atmosphere among colleagues and students. The deluge felt somewhat “horrific,” she said, but its arrival brought some relief after weeks of uncertainty. “I have a bunch of friends and colleagues whose grant proposals have just been wiped out,” she said.
“I have a bunch of friends and colleagues whose grant proposals have just been wiped out.”
Armstrong said he knew of one group whose entire research program was practically “wiped out” by the release. Tristan Buckmaster, an NYU mathematician who was at the center of OpenAI’s earlier dispute over the Navier-Stokes problem, meanwhile, said he had already heard of “three people who had entire research programs obliterated.”
Similar stories surfaced repeatedly in The Verge ’s conversations with mathematicians, though the disruption was concentrated heavily in some areas of the field. Francesco Fournier-Facio, a professor of mathematics at Heriot-Watt University in Scotland, said researchers in probability, combinatorics, and theoretical computer science appeared “particularly in shock,” adding that some regions of his field, group theory, had been “bulldozed.”
Armstrong and other researchers said some areas appeared to have been aimed at with almost military precision. “There were definitely some targets,” said Armstrong. He singled out work in the area Wang received the Fields Medal for this year, as well as several Millennium Prize problems. A cluster of results in mathematical physics, he said, makes it clear OpenAI is pursuing Yang-Mills theory — the mathematical framework underpinning much of modern particle physics — and its unresolved “mass gap” problem, one of the seven Prize problems.
Elsewhere, researchers expressed a kind of surprised relief at how little their own work appeared to have been touched. Roney-Dougal said her own corner of the field — largely centered on group theory — appeared to have escaped relatively unscathed. She joked that it is fortunate she works in a “very unfashionable area,” though later messaged to say she was feeling “unsure” after finding her work cited in one of the papers.
Joshi similarly found little overlap with her work, which is largely focused on integrable systems, complex systems that can be solved exactly. She suggested that may be because her field is less driven by long-standing, formally stated conjectures and definitions and more by questions that wax and wane with developments in physics.
As of October 8th, that record already listed numerous corrections, including revisions on more than a dozen manuscripts and the removal of three papers due to a “sign error” it says invalidates an argument.
Whether their own work had been directly affected or not, most mathematicians The Verge spoke to shared a sense that the field had crossed some kind of threshold and was no longer the same as it had been a day earlier. Constantin Kogler, a researcher at the Institute for Advanced Study, called it “the most important single moment in the history of mathematics,” while Yang-Hui He, a fellow at the London Institute for Mathematical Sciences, reached back millennia, comparing AI-driven developments this year to the publication of Euclid’s Elements , one of the discipline’s most influential works.
Few researchers were quite so grand in their assessments, but there was a general consensus that mathematics was changing fast — and that mathematicians would have to change with it.
OpenAI knew the release would be disruptive and had made some effort to soften the blow and engage with the mathematical community. After several bruising encounters with researchers earlier this year, the company turned to mathematicians themselves for help, working with the newly formed Advisory Group on Mathematics and Artificial Intelligence (AGMAI) on how to release the results responsibly.
AGMAI had already laid out what it believed responsible engagement would be. AI labs should ideally release papers “that a human understands” or otherwise provide the necessary “support for the additional mathematical activities that are needed for humans to be able to understand and assimilate their AI-generated mathematical output and identify possible applications of it.” They said that, where possible, proofs should be formalized, and that companies should make public what models and prompts they used to create them. The group also urged AI labs to stop treating mathematical releases as “marketing vehicles to promote their models” and should “stop testing advanced mathematical problems on proprietary models” that are inaccessible to the broader scientific community.
“It’s not the case that these are just silly problems that no one’s ever heard of. Many of them are like very well-known problems that many people have tried for decades.”
OpenAI followed some of the group’s advice. It said it would be funding a series of workshops and conferences around its work in mathematics to help the community process and understand its work, though it provided no details as to what these might look like or when they may occur. The company also disclosed significantly more information than it had in previous releases — again, researchers said this was a low bar — including that its model attempted more than 4,000 problems and that a typical result used around three hours of ChatGPT Pro thinking compute. It has also implemented “protocols for paper revisions and citations” and on GitHub said it “will preserve the public release history” of the collection. As of October 8th, that record already listed numerous corrections, including revisions on more than a dozen manuscripts and the removal of three papers due to a “sign error” it says invalidates an argument. OpenAI did not respond on the record to The Verge ’s request for more details.
That change addresses a key source of friction with mathematicians, some of whom speak of a kind of “paranoia” around the company’s published claims. The company has shown a habit of surreptitiously altering press releases and papers in response to criticism without clearly disclosing those changes.
But OpenAI did not follow all of the group’s recommendations — including some of its most consequential. It did not identify the model behind the release nor did it disclose the prompts used or the full set of problems the model attempted. OpenAI also appeared to acknowledge that its formalizations were lacking — it said it will add more as it obtains them — and that papers were subpar, saying that “for future releases, we are committed to further improving the quality of the papers via the citations, mathematical exposition, and presentation of the results for better understanding.”
“The write-up of the problem I know best made little sense after a quick read.”
Researchers The Verge spoke to questioned why OpenAI couldn’t have improved the quality of these papers and expressed disappointment that it seemingly couldn’t be bothered to formalize — or even check, in the case of the retracted papers — many of the results ahead of time. Information like the prompts used would have also been incredibly useful in lessening what many felt was the burden of assessing the flood of material the company suddenly dropped on them.
Most importantly, the company indicated it has no plans to stop testing its models on mathematics problems, and indeed suggested it sees doing so as an imperative. “We want to directly empower scientists with state-of-the-art capabilities and are working to responsibly release the model that produced these results,” it said when announcing its latest results. “This is why it is important to continue to evaluate our internal frontier models on mathematics and other sciences, so we can accelerate developing the tools to advance those fields.”
After OpenAI offloaded its results, AGMAI described the publication as “a first step,” and reiterated calls for equitable access to compute and research tools. More fundamentally, the group argued that “the future of mathematical research cannot consist only of understanding results produced by AI labs.”
For all the hundreds of long-standing problems OpenAI claims to have solved, mathematicians say there is an extraordinary amount of work to be done. Many estimated that making sense of everything the company just released could take the community years.
Armstrong likened the sudden arrival of so much new mathematics to the commotion that might follow a brief extraterrestrial visit. “If the AIs would disappear now, as though there were aliens that came to Earth and then just left, we would be studying this for the next 10 years, trying to understand everything.”
Much of that will be exciting in its own right. Researchers will have to fill in gaps, extract useful ideas and connections, and place solutions in a broader mathematical context, all things that typically go hand in hand with producing solutions for humans. “For many of these results, relevant experts care deeply about the solutions and I am confident that they will be able to digest and present them to the wider world,” Lichtman said. Work involving explaining, verifying, and contextualizing results will need to be valued more as AI changes the fields, he argued. “As a society, we should be rewarding these digestive efforts more.”
There may be plenty of mathematics left for humans to do, too. As far as he could tell, Lichtman said all of the results, “amazing” as they are, come “out of existing methods and techniques.” They fill in parts of the known mathematical landscape rather than creating entirely new ones. “It turns out there is a lot more room to fill in than experts previously knew!” Lozano-Robledo echoed the point: “They are all using existing techniques in very ingenious ways.”
And the existing map is hardly the limit. “Math research is essentially infinite,” Lozano-Robledo said. “Research will go on.” The London Institute’s He said he was particularly excited to see what emerges next, speculating that researchers may end up creating new ideas and “maybe even new fields of math.”
The companies seem ready to move on instantly, long before the community has had any reasonable chance to digest what they’ve produced.
Few mathematicians The Verge spoke to objected in principle to AI producing new mathematics. Indeed, many welcomed it and, to varying degrees, said they were enthusiastic users of AI tools themselves. Most of the unease was directed at how the AI companies building those tools were going about entering their field.
Looking at the mathematical releases of OpenAI and other AI labs, you’d be forgiven for thinking that research mathematics essentially amounts to checking problems off a list. The manner in which AI labs unveil their results reinforces this idea: a flashy announcement, a preliminary write-up, and the rest tossed to mathematicians to figure out and contextualize. Multiple researchers told The Verge that the companies seem ready to move on instantly, long before the community has had any reasonable chance to digest what they’ve produced.
Researchers described that as an impoverished view of how mathematical research actually functions. Solutions certainly matter, but so does everything that happens on the way to finding them: developing ideas, making connections, and finding new questions or opening up other avenues of research. When companies like OpenAI just find answers without doing any of the surrounding work, mathematicians say the burden of doing so falls back on the community.
“It’s nice to have new results, but this scale is a different level,” said Bartosz Naskręcki, a mathematician at the Adam Mickiewicz University in Poznań, Poland. “It’s a big mess,” he said. “It can cause a huge collapse in the academic culture and simply kill most of the faculties. It’s a social problem and it seems that the AI labs are completely ignoring this issue.”
It’s not just the mathematics that will take years to understand. Researchers are also struggling to comprehend what the upheaval means for them. Many described a bleak, almost existential mood hanging over the community as mathematicians decipher where they fit in the rapidly changing field. They may not have much time to figure it out.
Rumors are already circulating among mathematicians about further releases from OpenAI, which did not respond to The Verge ’s questions about whether more are planned.
Roney-Dougal said she dreaded another drop — or the prospect of Anthropic or another AI lab entering the fray.
The disruption is hitting PhD students, junior researchers, and others without permanent tenure particularly hard. Open problems like the ones OpenAI’s models are plowing through can underpin dissertations, grant proposals, job applications, and years of planned research, all important building blocks for academic careers. Several researchers described an increasingly pervasive anxiety that the work they are building their careers around could suddenly be next and that the AI models generate few avenues for future inquiry as they close off others. “For someone at the end of funding or looking for a job now this is incredibly disruptive,” said Simon Machado, a mathematician at ETH Zurich in Switzerland who is joining the French National Centre for Scientific Research (CNRS).
“Math research is essentially infinite. Research will go on.”
Morale may be particularly low because everything feels relentless. OpenAI’s findings have arrived in increasingly large waves, with little pause between them and frequent indications from the company that yet more are on the way. “You get a sense that they don’t really care about these individual results,” Roney-Dougal said. “It’s a really nasty feeling.”
Mathematicians barely have time to digest one set of results before the next, bigger set is dropped on them. “There’s going to be something in two more months that’s going to blow this away,” Armstrong said. “It’s just, like, repeated strikes by bigger and bigger bombs.”
Armstrong said he considers himself among the mathematicians most enthusiastic and optimistic about AI. He uses the technology in his own work and believes it has great potential. But even he is growing increasingly uneasy. “It is kind of hard to sleep at night,” he admitted.
Asked what he planned to do next, Armstrong was less certain. Still walking through the streets of Paris on his way to get lunch, he said his immediate priority was to finish some work he’d been doing — sometimes with the help of OpenAI’s Codex — “before OpenAI scoops us.”
Beyond that, even the self-described AI optimist was unsure and questioned what kind of future there was for him in mathematics. He worried particularly about the young researchers in the field. “It’s going to be a really weird few years in math,” he said.
Follow topics and authors from this story to see more like this in your personalized homepage feed and to receive email updates.
-
Robert Hart
-
AI
-
OpenAI
-
Tech