欢迎来到数学界的 AI 危机

今天的 Decoder ，我将与《The Verge》驻伦敦 AI 记者 Robert Hart 对谈，讨论 AI 正在对数学领域做些什么，以及许多顶尖数学家正因此经历的存在主义危机。

OpenAI 刚刚发布了一组针对长期数学难题的解答，在这一领域引发了爆炸性反响。这在数学界激起了巨大争论，而 Rob 花了一些时间与当代一些最有成就的数学家交流，了解他们的看法。

有趣的是，AI 系统仍然普遍不擅长小学算术，却越来越擅长极高端的抽象数学。这为高等数学领域提出了一些重大问题。

如果 AI 能完成如此级别的数学工作，是否意味着 AI 实验室可以将这些能力迁移到其他领域？当人类数学家正接受大学项目和学术资助的训练，试图在解决既有问题的过程中发现新问题时，如果前沿模型能直接回答所有悬而未决的问题，那么这些项目和资助又有什么意义？

又或者，围绕数学的这番关注，只是前沿 AI 实验室的一场大型营销活动——它们根本不在乎这门最古老、最基础的学术学科之一最终会怎样？

这里有很多值得讨论的内容，而 Robert 也与许多对这些问题持不同观点的人交谈过。

好了：《The Verge》AI 记者 Robert Hart 谈 AI 正在如何改变数学。开始吧。

本次访谈经过轻度编辑，以提升篇幅效率和清晰度。

Robert Hart，你是《The Verge》驻伦敦的 AI 记者。欢迎来到 Decoder。

我非常期待和你聊聊。特别是在 AI 与数学方面，最近有很多事情正在发生，而你也刚刚深入报道了这一话题。你采访了许多顶尖数学家，了解 AI 给数学带来的危机。

这既让人觉得信息量巨大，也让人觉得还有大量未知与待发现的内容，涉及这两件事之间的互动。一场彻底的存在主义危机，这简直是纯正的 Decoder 诱饵。请你大致说说，究竟发生了什么？

我认为，“信息量巨大”很好地概括了这件事。基本上，这在数学界引发了一场存在主义危机：数学是什么？数学家在做什么？未来数学家的角色又是什么？

其中很大一部分，是由 AI 能力在过去六个月到一年间发生的阶段性跃迁所推动的。AI 在很短时间内，从表现非常糟糕变成了似乎确实已经能达到专业水平。这相当于其他领域过去五年一直在努力应对的问题，在数学领域被压缩到极短时间内发生了。

我会把这和软件工程放在一起看。我们已经经历了一段时间的软件工程 AI 危机。但就在 2024 年，甚至去年，人们的普遍看法仍是 AI 模型特别不擅长数学。著名的例子是，这些模型无法数出英文单词 strawberry 中有几个字母 R。它们连计数都做不到。

是什么让它们变得更擅长数学？它们是否依然不擅长一般算术、却擅长高等数学，还是情况介于两者之间？

它们在数学的某些领域依然真的、真的非常糟糕。我确实检查过，现在它们能处理 strawberry 了。我想是有人做了调整。

我认为 strawberry 被硬编码了。我想说得非常明确：我的阴谋论是，所有模型都把 strawberry 这件事硬编码进去了。

我也这么认为。这是我愿意相信的阴谋论。

但没错，它们在这类事情上仍然很糟糕——数学、算术，甚至星期几。我男朋友前几天说：“它一直以为今天是星期三。今天不是星期三。”还有时间问题。几个月前，Elissa Welle 曾为我们报道，ChatGPT 不会看时间。它现在还是不会。这并不代表数学的全部。

所以这里存在一种脱节。要擅长数学，你得擅长计数、加法或乘法。但数学中很大一部分实际上是推理。如果你看学术数学论文，很多时候根本看不到数字，我想这就说明了问题。因此，它们仍然很差，但如今也同时非常擅长数学的另一部分。

至于原因，在某个节点，这些系统能做的事情达到了临界规模。我们已经在写作上见过这一点，也在编程上见过。它们非常善于在不同领域之间建立联系，将旧方法应用于新场景，做这类事情。看起来，新近训练的模型显然已经达到了那个能力突然贯通的水平，如今它就能做数学了。

还必须指出，我们尤其从外部视角谈论数学时，常常把它视为一个单一学科。但不妨想想生物学：它涵盖从观察动物、描述行为，一直到细胞机制和生物化学。数学也不是一个单一学科。AI 在其中某些部分非常擅长，而在计数等部分仍然非常糟糕。

即便在更抽象的层面，数学家也提出，拓扑学似乎是 AI 仍然不太擅长的一个领域。坦白说，我无法验证这一点，它超出了我的专业范围。不过，总体而言，情况仍然参差不齐。

所以你把数学描述为一个显然极其庞大的领域，包含大量学术研究方向。其中一些部分，模型已经变得相当擅长；另一些部分，也许是人们通常想到的数学基础部分，比如单纯的计数，它们仍在挣扎；而在两者之间还有广阔的范围。

存在主义危机是在中间这片广阔地带吗——人们不知道会发生什么？还是在模型真正特别擅长的那些部分？

两者都有，我感觉这会成为贯穿整场讨论的主题。没有人真的害怕它成为一个平庸的数学家，但这显然关系到该领域大量的工作。从研究层面或前沿层面来看，正如那些引发热潮的成果所展示的，它到底能做什么？现在有些领域里，它似乎已经能产出与优秀数学家不相上下的工作；与此同时，在另外一些部分，它又确实不会计数。

与之交织在一起的，是它会不会重塑就业结构或资金结构。你也提出了一个模糊的问题：什么是数学知识？以及这些从业者未来将承担什么角色。这些问题全都缠绕在一起。

我认为，这大体上符合 AI 在每个领域崛起的规律。当你能够单纯为一个问题增加算力或计算资源，并且存在某种可验证性时，模型似乎就会持续变得更好。那些处于中间地带、可能需要某种世界知识，或要求模型对现实世界本身具备真正理解的部分，它们似乎仍然很吃力。

至少从你的报道以及实验室的说法来看，数学中的这些部分似乎几乎完全是自洽的理论问题。模型可以在这些领域生成证明，或解决无人能够解决的问题；然后人们尝试验证证明是否成立，并且可以一遍又一遍地运行验证。

我想，这就让我们来到今年 5 月：一个我们其实并未见到的 OpenAI 内部模型，推翻了单位距离猜想——一个已有 80 年历史的问题。随后不久，我们又听说了 OpenAI 的 Astra。Astra 似乎就是那个让开关被拨动、让所有人都认定这是一场存在主义危机的模型。

Astra 取得了什么成果？为什么这很重要？

我也相当确定，Astra 很可能也参与了此前的那个成果。OpenAI 只把它称为未具名的内部模型，大概就是 Astra。我问他们时，他们没有回答。

几周前，OpenAI 发布了一篇博客文章，并附上了大量证明文件，我想大概有数百页。他们将其称为“数学与理论计算机科学的 10 项进展”。基本上，它涵盖了一系列学科，OpenAI 声称最新模型 Astra 在其中以某种方式完成了求解。我记得其中有一个是量子博弈论，我甚至不知道该从何解释。更难解释的是高维球体堆积，也就是三维以上空间中的球体堆积，此外还有许多其他不同学科。

这在社区中引起了很大震动，颇像一颗重磅炸弹。正如我们所说，此前也发生过一些单独的突破，但 OpenAI 一次性抛出了 10 项，而且都是相当重要的成果。有研究者告诉我，如果是人类解决了这些问题，我们会感到印象深刻；如果是一个人解决了全部 10 个，我们大概都不会相信。

这些也是数学家真正关心的问题，这一点很重要。此前很多突破曾被批评为发生在数学家其实并不在意的领域。而这些问题则是数学家、优秀的数学家投入大量时间尝试解决、却未能解决的问题。

我们来谈谈这 10 项成果。你对此进行了报道，OpenAI 也提供了一些文档。但它们并不是完全凭空靠算力硬推出来的，对吧？

它们建立在已有工作之上，其中涉及一些归因问题。大家的反应是“模型做到了这些”，还是我们在许多 AI 工作中见到的那种反应——归根结底就是：“你偷了这些成果，却没有署名任何人；你站在巨人的肩膀上，却没有提到巨人。”大家最终如何看待这件事？

总体而言，我接触的人普遍反应相当积极，认为这些成果令人印象深刻。正如我所说，这些是数学家关心的问题。其中有一个问题因其署名方式，以及他们在博客中发布所有这些消息的方式，而受到了特别关注。OpenAI 最初称，这 10 个问题在过去 10 年中没有任何进展。但如果你实际阅读论文，其中一篇很明确地写道：“我们建立在这两位研究者取得的进展之上。”

之后，这一说法被相当低调地修改了。但我采访的一些研究者对此并不满意，他们确实认为其中存在这样一种问题：“没错，你们没有为这里大量使用、而且按你们自己的说法也承认使用过的成果进行署名。”不过，我确实采访到的一位研究者正是被提及的研究者之一，而他对整件事的态度也有些矛盾，所以情况确实相当复杂。

总体印象是，这确实相当令人印象深刻。这些是真正让人关切的突破，确实以一种方式推动了领域前进。多位研究者实际上说过，如果是一位人类数学家完成这些成果中的任意一个问题，“那么这位研究者的学术生涯大概就稳了。”

很有意思，署名和归因在学术界几乎就是全部规则。而 AI 公司似乎能够以一种任何人类都无法容忍的方式，在这方面表现得很随意。

发现或工作的规模，是否弥补了这种不严谨？如果一个人类实现了同样的目标，却在署名上同样草率，大家的反应会一样吗？

我内心总有一部分想诉诸“这看起来像抄袭”这个角度。但如果你真的读了他们发布的论文——我采访的一位研究者说，全球大概只有 50 个人会费心深入阅读这些内容——你会发现它很清楚，并没有试图抄袭。我认为，说到底这只是一篇糟糕的新闻稿。

尽管我有时很喜欢就此批评它，但我已经报道科学十多年了。我发现新闻稿经常会过度包装实际发现了什么、其意义有多大，以及它到底有多新颖。我认为这只是又一个这样的案例。

这看起来是可重复的吗？他们提供了一些证明，表明自己解决了 10 个原本无法解决的问题。他们是否提供了任何证据，证明自己还能再解决另外 10 个？

这正是问题所在。因此，我为此采访了近十几人，最大的未知数就是：“他们为了得到这 10 个成果，尝试了多少个问题？”没人知道。他们知道，但不会说。

但这正是这里的核心问题。目前并不清楚，为获得这 10 项成果究竟进行了多少次尝试。如果他们第一次尝试的目标就是这些问题，随后便产出了这 10 个令人印象深刻的结果，那我会非常震惊。

也不清楚他们接下来会聚焦哪些领域，以及为什么选择这些领域。我想，他们选择公开哪些模型能力对应的问题，背后存在商业理由。所有 AI 实验室都在幕后招聘一批资深数学家，因此他们会不会再来一次，谁都说不准。

至于证明的问题，数学是一个有些特殊的学科，因为它的可重复性并不像实验科学那样。证明就是证明；如果它成立，那它就是成立的。这里的问题在于，人们能否跟着理解他们究竟做了什么？每个领域都高度专业化，因此会有相关领域的个别数学家逐一审查这些工作。我采访的、从事这些覆盖领域工作的研究者说，一切看起来都非常可靠。

在数学中，有一种名为 Lean 的编程与计算证明语言，你基本上可以将数学证明编码进去并运行，而它——嗯——会证明它。我这里不断说“证明”。但它会检验其中所有内容的严谨性与假设条件，而 OpenAI 也发布了这些内容。因此，这些结果看起来确实站得住脚。尽管他们可能会说新闻稿已经

Today on Decoder , I’m talking with Robert Hart, The Verge ’s London-based AI reporter, about what AI is doing to the field of mathematics and the existential crisis many lead mathematicians are having about it.

OpenAI just published a set of solutions to longstanding problems in math that went off like a bombshell in the field. It caused a huge debate in the math community, and Rob spent some time talking to some of the most accomplished mathematicians of our time about it.

It’s funny that AI systems are all still pretty bad at elementary school arithmetic, but getting increasingly good at very high-end abstract math. That raises some big questions for the field of advanced math.

If AI can do math of this caliber, does that mean AI labs can transfer those skills to other domains? What good are academic grants and university programs training new generations of human mathematicians to identify new problems as they try to solve existing ones, if frontier models simply answer all the outstanding questions?

What if all this attention around math is just a big marketing exercise for frontier AI labs, which couldn’t care less what happens to one of the oldest and most fundamental academic disciplines there is?

There’s a lot here, and Robert has talked to a lot of people with a lot of views on all of it.

Okay: Verge AI reporter Robert Hart on what AI is doing to math. Here we go.

This interview has been lightly edited for length and clarity.

Robert Hart, you’re our London-based AI reporter here at** The Verge. Welcome to Decoder****.**

I am very excited to talk to you. There’s a lot going on in particular with AI and math thatyou recently dove into. You spoke to a lot of leading mathematicians about the crisis in mathematics due to AI.

It feels like a lot, and also like there’s a lot yet to know and discover about the interaction of these two things. A full existential crisis, which is pure** Decoder**bait. Broadly tell us what’s going on.

I think “a lot” sums it up quite well. Basically a bit of an existential crisis within, “what is mathematics? What are mathematicians doing, and what is the role of mathematicians going forward?”

A lot of that has been spurred by a phrase transition in what AI is capable of that has exploded in the last six months to a year. AI went from being very terrible to seemingly genuinely quite good at a professional level in a very short space of time. It’s a lot of what these other fields have been struggling to deal with for the last five years in a very compressed period of time.

I would put that next to software engineering. We’ve been living through the AI crisis in software engineering for some amount of time. But as recently as 2024, even last year, the conventional wisdom was that AI models were particularly bad at math. The famous example is that these modelscould not count the number of R’s in the word strawberry. Even just counting eluded them.

What has happened to make them better at math? Are they still bad at general arithmetic and they’re good at advanced math, or is it something in between?

They are still truly, truly terrible at some areas of math. I did check, they can do strawberries now. I think someone’s tweaked it.

I think strawberry’s hard-coded. I want to be very clear, my conspiracy theory is that the strawberry thing is hard-coded into all the models.

I think so too. That is a conspiracy I’ll buy into.

But yeah, it’s still terrible at those kinds of things — math, arithmetic, even the days of the week. My boyfriend was saying the other day, “It keeps thinking it’s Wednesday. It’s not Wednesday.” Or time. Elissa Welle for us a few months ago wrote that ChatGPT can’t tell time. Still can’t. That’s not all of math.

So there’s this disconnect. To be good at math, you’ve got to be good at counting, or adding, or multiplying. A lot of it is actually reasoning. If you look at academic math papers, a lot of the time you won’t see numbers, which sums that one up, I think. So they’re still terrible, but they’re now also very good at this other part.

As to why, at some point you reach a critical mass of what these systems can do. We saw it with writing, we’ve seen it with programming. They’re very good at forging connections between different areas, applying old methods in new ways, those kinds of things. It appears that the newer models they’re training have apparently reached that level where it clicks, and now it can do math.

It’s important to say as well that we speak of math as a unitary discipline, especially from the outside. But imagine, say, biology. You’ve got something that would range from literally watching animals and describing behavior all the way through to cellular mechanisms and biochemistry. Math is not a unitary discipline either. AI is really good at some bits. Some bits like counting, it’s still really bad at.

Even on the more abstract levels, mathematicians have floated topology as one area that AI apparently still quite bad at. I can’t verify that, to be honest. It’s beyond my area of expertise. Still, it’s a bit of a mixed bag.

So you’ve described mathematics as a huge field, obviously, with many, many academic areas of interest. There are some parts where the models have gotten quite good. There are other parts, maybe the basic parts that people think of as math, which is simply counting, where they’re still struggling, and then there’s a wide range in the middle.

Is it the wide range in the middle where the existential crisis is, people don’t know what’s going to happen? Or is it at the parts where it’s really good?

A bit of both, which I feel is going to be a running theme through this. No one’s really afraid of it being a mediocre mathematician, but obviously there’s a huge element of what this field does. In terms of the research elements or the cutting edge, as we see with a lot of the results that generate hype, what can it do? There are areas now where it seems to be producing work that is on par with good mathematicians, alongside other parts where yeah, it can’t count.

All caught up in that is whether it’s going to rewrite employment structures or funding structures. You also raised the murky question of, what is mathematical knowledge? And the roles that these workers will be doing as well. It’s all of that wrapped into one.

I think that tracks broadly with the rise of AI in every field. Where you can just add horsepower or compute to a problem, and there’s some kind of verifiability, it seems like the models continue to get better. Everything in the middle where you might need some world knowledge or the models might need some actual intelligence about the world itself, they seem to struggle.

Those parts of math, at least reported out in your piece and what the labs are talking about, seem to be almost entirely self-contained theoretical problems. That’s where the models can generate a proof or solve a problem that no one’s been able to solve, and then try to verify that that has existed and they can just run it again and again and again.

That brings us, I think, to May of this year where an internal OpenAI model, which we have not really seen, disproved the unit distance conjecture, which is an 80-year-old problem. And then just recently we heard about Astra from OpenAI. Astra is the one where it seemed like the switch flipped, and everyone decided it was an existential crisis.

What did Astra achieve, and why is it a big deal?

I’m also pretty sure that Astra was probably behind the early one as well. OpenAI just listed it as an unnamed internal model. It’s probably Astra. They didn’t answer me when I asked.

OpenAI a few weeks ago dropped a blog along with a lot of paperwork proving it, I think several hundred pages. They called it “10 Advances in Mathematics and Theoretical Computer Science.” It was basically an array of disciplines that they claimed the newest model Astra had solved in some capacity. I think one was in quantum game theory, which I don’t know how to begin to explain. Even harder to explain is there was sphere packing in higher dimensions, so more than three dimensions, and there were a lot of other different disciplines as well.

It caused a lot of stir in the community. It was a bit of a bombshell. As we’d said, there’d been these individual breakthroughs that had happened, but OpenAI dropped 10 in one go, and they were quite big ones. Researchers had told me that if a human had solved these, we would be impressed. If a human had solved all 10, we probably wouldn’t believe it.

They’re problems mathematicians actually care about as well, which is an important point. A lot of previous breakthroughs have been accused of being in areas mathematicians didn’t really bother with. These are ones that mathematicians, good mathematicians, have spent a lot of time trying to solve and hadn’t.

Let’s talk about these 10. You reported them out. OpenAI did produce some documentation. But they’re not all entirely horsepowered out of nothing, right?

They’re based on previous work. There’s some question of attribution. What was the response? Is it, “Oh, the models did this?” Or was it the response we see to so much AI work, which basically boils down to, “Well you stole this and didn’t attribute anyone, and you’ve built on the shoulders of giants without mentioning it.” How did the response land?

By and large, the reaction was generally one of being quite impressed, from the people I spoke to. As I said, these are problems mathematicians care about. There was one that drew particular attention for how they credited it and also how they’d announced all of this in their blog post. OpenAI initially had said that these are 10 problems, and there had been no progress in the last 10 years. And then if you actually read the papers, one of them quite clearly says, “Oh, we build on progress from these two researchers.”

So that was later changed quite quietly. But a few of the researchers I spoke to were quite unimpressed, and they did feel it was an element of, “Well, yeah, you’ve not credited something that you’ve used heavily here, and by your own acknowledgement.” That said, one of the researchers I did speak to who was one of the ones named, and who was a bit ambivalent on the whole thing as well, so it was a real mixed bag.

The general impression was that it was quite impressive. These were actual breakthroughs that bothered people, and it did move the field forward in a way that, if a human mathematician had done these, several researchers actually said that, “Well, if a researcher had done any one of these problems, they’d probably be set for an academic career.”

It’s funny, credit and attribution in academia is the whole game. And it seems like the AI companies get away with being sloppy in a way that no human would be able to get away with being sloppy.

Did the scale of the discovery or the work overcome the sloppiness? If a human had accomplished the same goals and had been as sloppy, would the reaction be the same?

Part of me always wants to lean on the whole, “Oh, it looks like plagiarism,” element. But if you actually read the papers they produced — and one of the researchers I spoke to said there’s probably about 50 people in the world who are going to bother reading through this in depth — it is very clear. It doesn’t attempt to plagiarize. I think it was just a poor press release, to be honest.

And as much as I love to go in on it sometimes, having covered science for a decade-plus, I find the press releases are often overselling what discovery has actually been made and the import of it and the novelty of it. I think that’s just another case of what happened here.

Does this seem repeatable? There’s some proof that they provided that they solved 10 problems that were unsolvable. Do they provide any proof that they can solve another 10?

That’s the question. So the big unknown from the near-dozen people I spoke to for this was, “Well how many did they try to get these 10?” Who knows? They know, but they won’t say.

But that is the big question here. It’s unclear quite how many attempts it took to get these 10. I would be very impressed if it was the first thing they went after, and then out come these 10 impressive results.

It’s unclear what areas they would focus on next and why. There are, I imagine, business reasons behind which problems they are choosing to publicize that their models can do. All the AI labs have been hiring a cohort of senior mathematicians behind the scenes, so it’s anyone’s guess as to whether they do it again.

On the question of proof, math is a bit of an odd discipline in that repeatability is not the same as in the experimental sciences. A proof is a proof, and if it works, it works. The problem here is, well, can people follow through what they’ve done? Each field is quite highly specialized, so there’ll be individual mathematicians who are in those fields that go through the work. Those I spoke to that worked in some of the fields that were covered here say it all looks very legit.

In math, there’s a programming and computational proving language called Lean, where you can basically codify the mathematical proofs and run them through and it, well, proves it. I keep saying prove a lot here. But it will test the rigor and the assumptions of everything going on there, and they’ve published that as well. So it does appear to hold. Whilst they may say that the press release has a lot of hype or there’s a lot of hype around it, no one I’ve spoken to seems to be doubting the essential breakthroughs that they’re claiming here.

I want to stay on this subject for one more second. There’s the mathematical proof. We’ve generated a proof, and that is, as you say, just repeatable in a way that math is just logic. You can just go through the steps and say, “This proof worked,” and anybody listening to this who had to suffer through writing a proof in calculus in high school probably remembers that process. There’s something there that’s pure logic.

Then there’s a part of it that is software code, as you’re describing in Lean, where you can take the pure logic, you can express it in code, and you can run it to see if it works. I understand how AI is theoretically good at all of that. You’re just going to run the reasoning, and the reasoning is going to generate some code. You’re going to run the code, you’re going to get some verifiability. We’ve seen this play out in software engineering, where the code runs or not, it’s verifiable or not, and the models can just reason out about it.

Then there’s, to me, the big question that you alluded to. How many times do you have to run this? Can we verify that the models did this, and they weren’t directed by human mathematicians who’ve been hired at high rates by the labs in a way that suggests the field is going topsy-turvy?

You have a quote here from James Maynard, who has won the Fields Medal, the highest prize in mathematics, who said he’s been soul-searching. I keep looking at that quote. You’ve got similar quotes from all these other mathematicians in the piece, and it seems like they’re soul-searching against a thing that hilariously they cannot verify, which is, “How did the models do this, and is that thing scalable in a way that threatens mathematics?” What do we know about how the models did this?

I’d say as much as we normally do and do not know about this. There are a few issues there. One is the nature of the models. Well, this is an unreleased model, so good luck to anyone wanting to independently test it. The same goes with anything proprietary really. That said, I am inclined to almost give the benefit of the doubt that they’re not lying in some capacity about the models they’re using.

As for the other part, it’s perhaps more noteworthy to ask how did we prompt the model, or how is it being guided? Is that by a mathematician who knows what they’re doing? That’s probably a key factor here, according to a lot of mathematicians I’ve spoken with who’ve tried using these. This is often the consumer models, but still it peaks to a broader landscape.

They say that if you know what you’re doing and you can point things out, it’s good. Or you can use it as a tool in a way that you want, and in a way that you wouldn’t be able to if you didn’t really know how to fact-check it. I’ve had situations where I’ve had ChatGPT doing a basic sum, and I say, “That number is not right.” It responds, “Wait, so sorry. You’re right, it’s this,” and it’s still wrong. But that’s still needed at this level as well.

It does allude to a broader problem. As you said, this almost soul-searching of, “Well, what if we can automate that away, and what if it gets to a point where we don’t understand it?” And that really cuts to a deeper question of, “Well, what is mathematics? Why do we do it? Why do we value it as a field?”

Everyone will have different answers to that. But a fear of a lot of people I spoke to was that this might move beyond a realm of human interest, and in which case, well, maybe we just won’t engage with it. Or it’ll be something that interested people will go through, and then the rest will continue as normal.

You’ve got a quote here from a researcher in Zurich named Johannes Schmitt who says, “We might be headed toward a situation where the math problems get ‘mowed down’ by AI, but we don’t actually push the field forward because humans are taken out of the loop and they’re not either checking, or they don’t understand it, or they don’t know what the future breakthroughs might be.” How likely does that feel? Is that a big concern?

There is an element of the mowing down of the problems. Especially those that are used as a training field for younger mathematicians coming up and cutting their teeth, so to speak. But I also think that this idea, that math is problem-solving, is very much an outsider’s perspective of the mathematical endeavor.

So a lot of the mathematicians I spoke to found that the ticking boxes part is the least interesting and valuable part of the field. The areas that are valuable for them aren’t the, “Oh, you’ve solved something, or you’ve proven something.” It’s what happens from that.

I think it was James Maynard that said that the most interesting discoveries in the field aren’t that you’ve solved something, it’s what evolves from that. Sometimes those solutions open entire new fields of research that no one ever thought were possible, or, “Oh, this is a new tool that you can apply everywhere in fun and exciting ways.”

I think if we look at the popularization of math — even what I’m thinking of as those theorems that people have posed — it’s the questions that endure, not the solutions. It’s always Fermat’s Last Theorem, not like, “Well, here’s the solution to whatever the last guy proposed.”

I think the concern here is that well, they’re going to tick off all of these questions. Normally in the process of doing so, one would hope they would branch out into all of these new exciting areas or pose new questions. But AI won’t do that, and that’s the concern. And then that would leave the field quite sterile, and it will have all of these things that have been done, and maybe nothing left to pursue.

The general consensus was, well, the jury’s out. It’s too early to tell. Even with human mathematicians, it takes a lot of time to realize the impact of these kinds of things. As I said, it’s exploded in the last six months to a year, and math is not a fast-moving discipline at the best of times. But it’s too early to tell really whether that will be a concern. But it is a concern, and a big one.

You have another quote from Maynard here saying, “If the standard for a publishable paper in math is something that an AI cannot do, particularly when a PhD is typically four years, the challenge is you’re not trying to come up with a problem that AI can’t do now, it’s an AI in four years’ time.”

So this is really related to the rate of improvement of the models, which as you say, particularly in math, seems to be increasing, but not at an even rate across all of the domains of mathematics.

That appears to be what is causing the soul-searching. If you’re a student and you start today, and you pick some obscure domain that maybe the AI isn’t good at, sometime halfway through your PhD thesis or your PhD research, the AI will just solve it and you’ll be done. That is a real problem for you.

Has the field reacted to that yet, or are they just still in the shock of, “Oh, the models can start to do things that we didn’t think they were capable of”?

Yeah, I think it’s shock, really. I keep a little notebook to the side to just write down broad feelings whenever I do these interviews, and I’ve written “shell shock” in it. It’s far from universal, but it feels like it’s happened so quickly that it has just taken a lot of people by surprise. Even if they knew in theory that, well, this is coming.

They’ve seen all these AI math startups going. They’ve seen colleagues moving around to different labs or areas of work. But it just happened very, very quickly. So it’s given them very little time to figure it out. It’s not necessarily even the fields that it might be good at, it’s more just like, “What can it do?”

But I can’t imagine if something like this had come out when I was studying and literally in the space of half a year, it just upended what was possible. And over the summer as well. So students are possibly coming back to a completely different discipline after a break.

There’s some skepticism here in the world. Gary Marcus is a reliable skeptic of AI, and he pointed out that over and over again, what you see is that AI accomplishes something in one domain and then it’s used to generalize AI’s ability across every domain.

There’s agood quote from Marcus here: “As we learned a decade ago from AI’s shambolic and ultimately failed attempt to turn Jeopardy-winning Watson into a cancer-fighting machine, success in one domain does not guarantee success in all.”

I can read this two ways. One, “AI has solved math,” which is not true as you’ve pointed out in several ways, all the way down to how it’s still bad at counting. And then there’s, “AI has solved math, and that means necessarily it’s going to come for everything else.” It will come for physics. It will come for law. It will come for whatever you want in the world. You can see if there’s any verifiability, AI can solve it, because you can just run it in this way.

I understand both sides of that argument, that obviously success in one domain does not guarantee success in every domain. And then the arc of AI is, well, it keeps collecting domains. If there is any verifiability, it is more likely to connect those domains than not. How do you see it?

I’m not entirely sure that the people that Marcus is criticizing here have actually said quite what he says they are saying. There’s a lot to say about the hype. But yeah, success in one domain does not even equal success throughout that domain, let alone in other domains.

That said, there has been an undeniable trajectory in the last few years of a broadening capability increase. I don’t think you need to be on the whole AGI train to acknowledge that, and to acknowledge that that will have an impact.

I think it was Andras Juhasz, one of the professors at Oxford, that I quoted in the story. Something else he’d said to me was that he’s been toying around with ChatGPT a bit, and he’s like, “I don’t think it has any geometric intuition whatsoever. Which might explain why there has been a very limited amount of progress in fields like topology.” I am in no position to verify that claim in terms of the math of it, but I think it illustrates it quite well. They call it the jagged edge.

It felt like an easy argument for me: “Let’s criticize the whole ‘the singularity is near.’” That’s what Elon Musk said in response to Astra, which is a lot. I also think it’s perhaps the least generous interpretation of that argument you can take to argue against. If you take a more nuanced element that does acknowledge that there has been clear progress here, and quite quickly, and as you said, it is racking up domains, I feel there’s a trajectory there that is a reasonable one to consider, rather than just dismiss out of hand.

One of the bigger arguments about AI in general is that it democratizes access. I was not a great software developer in my days trying to write software code, and now I can vibe-code apps at will to do all kinds of dumb stuff in my house.

Is there a similar argument here where a bunch of people who have mathematical intuition, but did not have the formalized language or training of academic mathematics, can now access a model and push the field forward? Because that is usually the thing that undercuts the criticism from the professionals is, well, many, many more people now have access to this thing that only you had access to because of your money and your training.

Annoyingly, I am going to say it’s a two-pronged thing again. But yes, in a broad sense, yes, it is. A lot of the mathematicians I spoke to were almost quite weary of this, actually. They love the idea, in theory, of democratizing access. They’re also quite fed up with AI-generated or -assisted papers that are flooding every publication imaginable, as well as the pre-print servers that they use in these fields.

Some of those I spoke to said things like, “Oh, I got three emails this week alone with people being like, ‘Hey, is this legit?’” Because they thought they’d solved something with ChatGPT or with Claude, and they also don’t have the mathematical skills to check whether they’ve actually solved something.

On the flip side, there are parts where they said, “Well, we’ve got a talented undergrad who’s done something that a talented undergrad would probably have never managed, and here they are doing grad-level work and they’ve produced a paper that is legit.” And in the bigger scheme of things, a few I spoke to said, “Well yeah, a lot of these are in the ivory tower. Having access to this kind of thing globally could really boost access to the kind of things here.”

On the flip side, the cost. These things cost a lot to run. It’s always easy to forget when you use, say, a free version of ChatGPT or Claude or something. At the higher levels, these things cost money. They may not necessarily cost a lot of money — OpenAI claimed, I think it was $2,000 for these 10 results, but that doesn’t factor in literally anything else once they’ve got these, so it’s a very generous number. But even taking that figure, math is quite a poor discipline, even at very well-off institutions.

Colva Roney-Dougal at St. Andrews, who I spoke to, said, “Well, a lot of the time I don’t bother getting a research grant. I don’t need one. I just have a blackboard.” And so if you’re not even getting a research grant, $2,000 is a lot to put up. So it could lock out researchers that way, even at quite well-funded institutions. Not to mention the speed at which this is happening, that virtually no one would’ve been able to bake any of this into a grant proposal yet, was another theme that I came across a lot.

Actually, Roney-Dougal has another great quote in your piece about the nature of the AI labs and how they are talking about math. She said, “They’re treating our discipline as an advertising playground.” A bunch of mathematicians have signed something called theLeiden Declaration, which is an open letter to then pledge not to buy into hype around AI.

These things are running right at each other. The AI labs are not going to stop using every discipline as an advertising playground. And a bunch of mathematicians saying, “We refuse to buy the hype,” certainly does not seem to be stopping the hype.

There’s just a piece of this that is organized professional resistance to a thing that is upending a field that has, as you say, been pretty cheap to operate, and now might be getting cheaper or easier to access or easier to upend, day by day. Do mathematicians feel that that is going to be effective? Historically, mathematicians are not savvy political operators. There’s a part of me that says, “Oh, they’re just going to get run over.”

I don’t know. In the history of math, actually, I think a lot of them were quite savvy. Isaac Newton is the one that always comes to mind for that — though quite a petty political operator as well. But yeah, that is the fear. A few that I spoke to, and one really comes to mind, mentioned that there’s often this belief that math is the pinnacle of knowledge. But he was like, “Well, that’s bullshit.” And he wasn’t alone in illustrating that sentiment.

But it is good for showcasing, and it’s a lot neater as a discipline, and a lot cheaper. You mentioned Marcus referencing IBM’s Watson and the curing cancer ambition. Well, that involves lots of messy experiments, including on people. You don’t need that in math, so it’s a really easy discipline to come in, throw your weight around, and then move to somewhere more lucrative if that’s what you want.

I’m not saying that that’s what they’re doing. A lot of the people at these companies have been hired. I don’t doubt their credentials for sure, and I don’t doubt their motivations as well. It does raise a question long-term as to how viable this is. Because let’s be clear, as a field goes, I cannot imagine mathematicians being a very lucrative enterprise customer for these companies.

The thing that might be lucrative is pushing a field forward to turn it into something economically viable. We push mathematics forward as a field, that turns into some engineering or physics breakthrough based on that mathematics, and that turns into, I don’t know, yet another way to launch rockets.

Some circle happens there that I don’t quite understand, but that is the history of innovation, from research, to engineering, to products or services that make money. Is that on the minds of any of these mathematicians, that pushing the boundaries here is upstream of something radically economically lucrative?

The immediate counter that would come to mind here is that a lot are scared that it’s closing off the field. So by definition, those breakthroughs that lead to something surprising and new that you can say, “Oh, this works here,” may not be happening anymore. If anything, that lucrative endeavor of applying math to this entire new field that may have a lot of money in it remains an open question as to whether anything like that would be possible if we’re closing off avenues, rather than opening them up.

Right, if the economic incentive of solving the unsolved problem is reduced because you personally won’t get rich if a computer is just solving every unsolved problem, something very fundamental breaks there.

A lot of it comes down to the fact that it’s not just solving problems. With a lot of these things, as we said, it’s about what solving that problem tells you elsewhere. If these were very lucrative problems to be solving, I imagine that more people would be trying to solve them than have left them for decades.

This is the nature of a lot of pure science, and it’s a broader criticism of what is going on perhaps with the Trump administration’s approach to science policy at the moment, in that it’s very applications-focused. There is something to doing pure research that can yield potentially very big dividends that is, by definition, utterly unpredictable as well.You cannot plan for it.

The fear I think with math is that in solving all of these problems, and then also doing so without opening up new areas of research, what are you left with? Even if it’s from a more lucrative, “What are you going after,” point of view, if you’re not opening up new areas of research and you’re just ticking off old ones, it just leaves a big question mark as to what might be left in its wake.

Even from those I spoke to that were very excited about what’s happening, they said that even they don’t really know what’s happening. And they’re excited from a personal level because, “Oh, we might be able to do this, might be to do that.” But there was still this lingering uncertainty of like, well, where does this leave the field?

Especially for more pure disciplines like research mathematics, it’s tougher to say what comes next. Because in a lot of the other sciences you can say, “Well okay, well they’d shift onto more engineering problems, or applying that.” But if you solve all the problems at the ground and there’s nothing being built up from that, where do you go from there?

One great thing about** The Verge**is our commenters are vast, they’re very knowledgeable, and there was acommentfrom a mathematics researcher on your story that I just want to read to you, and see if you think this is the right framework.

Here it is: “I have no doubt these models will bring massive change in the field, but in their current state, they won’t yet drive us to obsolescence. Just occupy a particularly useful spot in our bag of tricks. My apprehension comes from not knowing where these things will peak, but overall I remain optimistic. I think AI will be a net boon for math when used properly.”

I feel like “AI will be a net boon for X when used properly” is just where you land in life in a lot of things, but that’s the most optimistic response that I’ve heard: “If we get it right, it’s going to be great.” Is that the vibe, or is it still more shell-shocked than that?

I’d say shell shock is still the overriding impression. I think perhaps the gut response to that is like, “Well, it will be a net positive for whom, and what is ‘properly’?” All of those are quite legitimate questions here.

There were some bleak responses from graduate students I saw in essays posted online.Where’s their place in this as future researchers? Do they have a place in this? Is it as glorified AI proof checkers? That will be quite an unsatisfying career, I imagine.

Or maybe not, I don’t know. We will see. I think anything used properly will be a net boon. But yeah, I think it all comes down to what “properly” means, and for whom we’re talking about.

I feel like, oddly, that is an excellent place to leave it. Because I don’t think either one of us knows, and I suspect over the next year or so, things will come into focus. Because at some point, OpenAI will have to show people how they did the things of the models. And perhaps more importantly, the other labs are going to want to either replicate these results or show that they can push farther, which will necessarily have to lead to a little bit more transparency and yet more mathematicians having a crisis with you.

Robert, thank you so much for being on the show. We’ll have you back very soon.

Questions or comments? Hit us up at decoder@theverge.com. We really do read every email!

Decoder with Nilay Patel

A podcast from The Verge about big ideas and other problems.

SUBSCRIBE NOW!

Follow topics and authors from this story to see more like this in your personalized homepage feed and to receive email updates.

• Nilay Patel

• AI

• Decoder

• OpenAI

• Podcasts

• Tech