OpenAI 最新科研突破引发学界强烈反弹：当 AI 接连攻克科学谜题，为何科学家并不买账？

---

访谈实录（Transcript）

凯文·罗斯（Kevin Roose）： 科学家和数学家这次为什么又对 OpenAI 发火了？

凯西·牛顿（Casey Newton）： 当你需要用 AI 来工作，而你家孩子却觉得 AI 是邪恶的，你该怎么对他们解释？

罗斯： 你要如何关停一个失控的 Agent 集群？

牛顿： 还有，我们俩算不算是《青蛙和蟾蜍》的现实翻版？

罗斯： 我是凯文·罗斯。

牛顿： 我是凯西·牛顿。

罗斯： 这里是旧金山，欢迎收听由我们与 NPR 联合制作的播客节目《Machine Gods》。

牛顿： 奇点时代的官方播客。

罗斯： 节目现在开始。好的，凯西，我们今天的内容非常丰富。收到了很多听众发来的绝佳提问，但首先，我们得聊聊科学。

牛顿： 我们非得聊这个吗，凯文？你知道我上学那会儿科学成绩可不怎么样。

罗斯： [笑] 嗯，不过我敢保证，今天你在我们节目里学到的科学和数学知识，绝对会比以往任何一期都要多。

牛顿： 那我可来劲了，因为看到这些公告发布时，我就在想：“要是凯文能给我讲讲这到底是怎么回事就好了。”

罗斯： [笑] 没问题，包在我身上。

牛顿： 多谢啦。

罗斯： 本周，OpenAI 宣布了三项由 AI 生成的新科学发现，分别涉及数学、计算机科学和物理学领域，我们稍后会逐一拆解这些成果。但在那之前，我认为有必要先聊聊我们为什么要在当下讨论这些，因为目前在数学和科学界发生的事情确实至关重要，同时也充满了争议。数学家、计算机科学家以及各领域的学者，对 OpenAI 取得这些成果的方式以及公开发布的形式感到极其不满。

牛顿： 确实，某种程度上我能理解他们。虽然我知道 OpenAI 内部确实有人是纯粹的科学信徒，觉得能取得这类突破、推动人类认知边界非常不可思议；但从根本层面上看，人们难免会觉得这只是 OpenAI 在 AI 形象跌入谷底之际搞的一场公关营销。他们就像是在柜子里四处翻找，搜刮出一些发现展示给全世界看，喊着：“嘿，快看这些新成果！”但如果你一辈子都在专研这些难题，而这家公司突然横插一杠子宣称给出了解决方案，这确实会让人觉得挺令人恼火的。

罗斯： 完全没错。这一切其实始于今年 9 月初，当时 OpenAI 发布了关于纳维-斯托克斯方程（Navier-Stokes problem）的解法。这是困扰了数学家数十年的“千禧年大奖难题”之一。这随即引发了猜测，有人怀疑他们一直在悄悄监控那些使用其模型尝试做类似研究的人类数学家。不久之后，由 28 位菲尔兹奖（数学界最高荣誉）得主组成的小组签署了一份题为《AI 在数学领域的严重对齐失调》（A Severe Misalignment of AI in Mathematics）的声明，发出警告——

牛顿： 他们用这个词，就是深知指责一家 AI 实验室或公司“对齐失调”（misaligned）最能刺痛他们的神经。

罗斯： 没错。

牛顿： 是这样。

罗斯： 于是 OpenAI 试图对此作出回应，基本上同意与一个独立的专家小组合作，共同研究未来如何发布这类成果。紧接着在 10 月初，OpenAI 在 GitHub 上一口气倾倒了 722 篇由 AI 撰写的数学论文草稿，这直接引爆了数学界更大规模的抵制浪潮。他们对此极为反感。

牛顿： 可不是嘛，正如计算机科学家赖安·威廉姆斯（Ryan Williams）所说，这简直就像是对整个数学行业发起了一场 DDoS 攻击，对吧？就好像在说：“哦，你们还在吭哧吭哧解一道数学题呢？我们已经一口气解了 700 道了。拜拜了您嘞。”

罗斯： 没错。[笑]

牛顿： 这种滋味可一点都不好受。

罗斯： 确实不好受。所有这些争端铺垫到了本周——OpenAI 正式公布了最新的三项科研成果。我们马上就来细聊这三项发现，但同时他们也谈到了自己为平息众怒而制定的新发布流程。

牛顿： 是的，他们显然听取了外界的意见。我觉得整个事件中最有意思的地方在于：他们主动走访了这几个截然不同的学科领域，实质上是在询问：“你们希望我们以怎样的方式与你们合作？”而不同学科给出的反馈大相径庭。

罗斯： 没错。那我们先来快速过一遍这三项成果，我尽量讲得精炼些，免得用你早就精通的深奥数理知识把你听烦了。

牛顿： 谢谢你……另外，我今天手头确实还有一堆事要做呢。

罗斯： [笑] 那我们加快节奏。首先，我认为最重磅的成果是证伪了所谓的“指数时间假说”（exponential time hypothesis）。这是计算机科学领域的一项重大突破，对密码学有着深远影响。简而言之，它提供了一种解决超高难度谜题的捷径，而在此之前，科学家普遍认为这类难题只能通过指数级堆叠算力才能破解。

牛顿： 明白了。

罗斯： 第二个难题被称为“挂谷猜想”（Kakeya conjecture）。这是一个困扰学界数十年的难题，而 OpenAI 声称已经在任意维度上解决了该猜想——不仅是二维、三维、四维，而是无限维度。这个猜想大致探讨的是一根针为了能够转向任意方向，所需旋转扫过的空间范围问题。

牛顿： 有人跟我打比方说，解决这个问题就如同在某种无限维度的空间里捞一根针，对吧？

罗斯： 没错，一个多维的大海捞针。

牛顿： 嗯哼。

罗斯： [笑] 是的，这是属于极度纯粹的专业数学与科学谜题，可能会对其他类型的问题产生连锁影响。虽然对普通人来说可能无关痛痒，但确实有学者为此奉献了大半生心血。

牛顿： 确实，对此我们得对他们的付出说声感谢，现在或许他们可以转向其他研究了。[笑]

罗斯： [笑] 是啊，改行做播客也行，谁知道呢。

牛顿： 开个播客去吧。

罗斯： 好的，凯西，第三项成果来自物理学界，它与所谓的 QCD 相关，也就是“量子色动力学”（quantum chromodynamics）。

牛顿： 哇哦。

罗斯： 我猜你肯定想问什么是 QCD。

牛顿： 愿闻其详。

罗斯： 凯西，它是关于强相互作用力的理论，正是这种“胶水”将质子和中子内部的夸克紧紧黏合在一起。这在物理学界可是件了不得的大事。

牛顿： 听起来……说实话我刚才听岔了，我还以为你说的是什么“同性动力学”（quantum homodynamics），也就是在一间小公寓里的骄傲派对上究竟能塞下多少个男同性恋。不过那个问题——那个难题我其实早解开了。快跟我说说这个物理学的吧。

罗斯： [笑] 哈哈，这完全是两码事。但总的来说，几十年来物理学家一直依赖这套理论来解释维系物质存在的某种基本相互作用力。

牛顿： 是爱吗？

罗斯： 他们天天用它，但那不是爱。

牛顿： 好吧。

罗斯： 是量子色动力学。

牛顿： [笑] 行吧。

罗斯： 但物理学家从未在严谨的数学层面上真正构建出完整的理论体系，而 OpenAI 声称他们已经找到了构建该理论的方法。

牛顿： 也就是说，他们构建出了一套理论，来解释到底是什么力量维系着……刚才说维系什么来着？夸克？

罗斯： 没错，夸克（quarks）。

牛顿： 夸克——不是怪癖（quirks）吧。听着像你在说怪癖。

罗斯： 不，是夸克。

牛顿： 行，夸克。

罗斯： 对。

牛顿： 好吧。

罗斯： 差得可远了。[笑]

牛顿： 嗯哼。

罗斯： 但如果你是物理学家，这绝对是重磅炸弹。因此，物理学家对这一项发现尤为兴奋、愤怒，且充满了怀疑。

牛顿： 明白了。那我们接着聊聊，OpenAI 是如何与这三个不同学科打交道的？各学科之间的应对态度有何不同？

罗斯： 在今年早些时候遭到数学家和科学家的一致声讨后，OpenAI 基本上妥协了，表示：“好吧，请各位告诉我们，你们希望我们如何发布内部模型产出的这些成果。”于是他们咨询了这三个领域的专家，相当于对他们说：“请你们制定一套策略，指导我们该如何向公众发布。”他们有时把这些专家称作“引导者”（shepherds），因为这些人就像是在引导 AI 生成的解决方案平稳落地人类世界的牧羊人。

牛顿： 懂了。

罗斯： 各个领域给出的方案各不相同。受访的数学家表示：“直接公布 AI 的输出结果，让学者们自己去消化整理。简单说，扔给学术界就行，我们自会研究应对。”

牛顿： 明白，可以说是典型的按部就班风格。

罗斯： 计算机科学团队则要求所有成果向公众全面公开，但同时要求 OpenAI 出资举办一场学术会议来系统解释这些发现。

牛顿： 我不得不说，计算机科学家在情商上真的碾压了数学家，要是能顺水推舟白蹭一场学术会议赞助，那绝对是稳赚不赔。数学家们，好好学学吧！

罗斯： 一点没错。而物理学界对整个事情的抵触情绪最为强烈。

牛顿： 毕竟物理学里到处都是“强作用力”嘛。

罗斯： 确实如此。

牛顿： 是啊。

罗斯： 他们[笑]还在牢牢束缚着那些夸克呢。

牛顿： [笑] 嗯哼。

罗斯： 物理学者团队建议 OpenAI 不要直接公开 AI 撰写的论文，而是应该交由他们这些人类科学家、人类物理学家亲自来执笔撰写。

牛顿： 没错，他们的态度是：“这篇论文我们接手了，谢谢配合。”

罗斯： 确实是这样。

牛顿： 嗯。

罗斯： 值得注意的是，OpenAI 特别声明，他们并不寻求为其模型所产出的科学发现争取学术署名或学术荣誉。

牛顿： 对啊，他们不在乎学术荣誉。他们只是想抢先把这一切发在自己的官方博客上昭告天下，再在所有社交媒体渠道大肆推广一圈。但千万记住：别给他们算学术荣誉哦！

罗斯： 是啊，仿佛在说“哎呀，区区在下怎么好意思邀功呢？”[笑] 这种自行指定各个学科“引导者”和专家的做法，遭到了不少学者的批评。OpenAI 自身也承认，他们希望探索出更好的合作机制，但现状基本上是：“我们该怎么做，才能在将这些成果公之于世的同时，把得罪的人数降到最低？”

牛顿： 好的，说到这里，我反倒想[笑]给 OpenAI 一点理解和宽容了。这确实是一个前所未有的棘手处境。

罗斯： 确实。

牛顿： 对吧？你的软件程序刚刚攻破了基础科学界悬而未决数十年的顶级难题，而你现在必须凭空摸索出一套交付方式，既要让领域内的所有专家感到富有成效，又不能摧毁整个学科生态。因为这正是数学家们内心深处的巨大恐惧，凯文——如果你一口气把 700 多个解法直接砸到整个领域脸上，实际上就是在毁掉这门学科，对吧？有太多学者原本计划在接下来的 20 年、30 年乃至整个职业生涯中去攻克这些课题，现在突然之间，他们可能全无用武之地了。因此，确实需要有一套妥善的流程。而我必须承认，OpenAI 这一次采取的折中方案在我看来还算合理。你是怎么看的？

罗斯： 是的，我也觉得……这还算合情合理。显然不存在十全十美的解决方案，这种冲击注定是——

牛顿： 至少挂谷猜想有了个十全十美的解法嘛。

罗斯： [笑] 这倒是真的。但我认为他们已经竭尽所能，试图在彻底否定诸多学者及其学生多年心血之前，先去争取这些领域顶尖专家的认可与背书。你看，之前也有些人狂妄地宣称：“咱们推出汽车的时候，难道还得先去征求马车夫的意见吗？直接把汽车造出来上路就完事了。”对于这种论调，我只能说祝你好运，而且我也丝毫不奇怪你们行业的民意支持率会跌得比移民与海关执法局（ICE）还要惨。

牛顿： 我正想说呢，这种傲慢态度对于整个 AI 行业来说绝对是百害而无一利。从法律层面上说，这种说法当然没错，你确实没有法律义务去挨个征求他们的意见；但如果你去征求了，大家的处境都会好过得多。

罗斯： 是的。另外我们还应该注意到，在 OpenAI 宣布这三项发现仅仅几个小时后，Anthropic 也发布了类似的突破成果。他们声称自己同样证伪了指数时间假说。可是凯西，各大实验室为什么突然都要这么干？他们为什么争先恐后地抛出海量关于未解科学和数学难题的解法？这纯粹是一场公关营销秀，还是背后另有隐情？

牛顿： 我觉得从一方面来看，这些实验室内部确实聚集着真正的科学家，他们由衷地觉得这太酷了。他们最初投身超级智能（AGI）领域的初衷之一，就是为了推进人类的认知边界，推动造福人类的新发现。在他们眼里，眼下这些成果尽管晦涩难懂，但都是通向终极目标的坚实脚印。我认为这确实是实验室内部极为真诚的一种信念。但与此同时，凯文，你绝不能忽视当下的时机。就像你刚才说的，AI 的公众舆论形势非常糟糕，人们普遍排斥它。在社交媒体上反复能听到的质问就是：“你们造这些玩意儿到底图什么？如果它这么危险，你们为什么不干脆停下来？你们口口声声说它会让我丢掉饭碗，那我凭什么还要为它欢呼雀跃？”而这正是挂谷猜想派上用场的地方了，凯文。假设你走进铁锈地带某个乡村小餐馆，坐在一位普通选民身旁对他说：“挂谷猜想……

---

Transcript

Kevin Roose: Why are scientists and mathematicians mad at OpenAI this time?

Casey Newton: What do you tell your teenager who thinks AI is evil when you need to use it for work?

Roose: How do you shut down a rogue agent swarm?

Newton: And are we Frog and Toad personified?

Roose: I'm Kevin Roose.

Newton: I'm Casey Newton.

Roose: And from San Francisco, this is Machine Gods, in partnership with NPR.

Newton: The official podcast of the singularity.

Roose: Starts now. All right, Casey, well, we've got a big show today. Lots of great listener questions that people have been sending in, but first we have to talk about the science.

Newton: We have to talk about it, Kevin? You know that was not one of my better subjects in school.

Roose: [Laughs] Well, today you are gonna learn, I feel confident in saying, more about science and math than you have ever learned during an episode of our show.

Newton: Well, I'm excited about this because when I saw these announcements come out, I said, "I would really like it if Kevin explained this to me."

Roose: [Laughs] Well, I'm here for you.

Newton: Thank you.

Roose: So this week, OpenAI announced three new AI-generated discoveries, one each in math, computer science, and physics, and we're gonna talk about these results. But just in general I think we should talk about why we're discussing these now, because I think what is happening in math and science is really important, and it's really controversial. Mathematicians and computer scientists and all manner of scientists are really mad about the way that OpenAI has developed these results and the way it's releasing them.

Newton: Yeah, and I sort of get it, because while I know there are people at OpenAI who are true believers in the science and think it's incredible that we can make these sort of breakthroughs and advance human understanding, I think there's also a sense that on some fundamental level, this is just a marketing and PR effort for OpenAI at a time when AI is really unpopular. They're sort of, like, rifling through the cabinets trying to find some discoveries and show them to the world, say, "Hey, look at these discoveries." Whereas if you've been researching these problems your entire life and this company just kind of, you know, stomps in out of nowhere with a solution, it is just gonna feel kind of annoying.

Roose: Totally. So this all kicked off back in early September when OpenAI released a solution to the Navier-Stokes problem. This is one of these Millennium Prize problems that mathematicians had been puzzling over for decades. That led to some speculation that they had been, like, snooping on human mathematicians who had been using their models to try to do similar things. Shortly after that, a group of 28 Fields Medalists, these are the highest honors in mathematics, signed a declaration titled "A Severe Misalignment of AI in Mathematics," warning—

Newton: And they said that 'cause they know that that hurts an AI lab's, company's feelings when you say that they're misaligned.

Roose: Exactly.

Newton: Yeah.

Roose: So OpenAI tried to respond to these concerns by basically agreeing to work with this independent group of experts to sort of figure out how to release these future results. And then in early October, OpenAI dumped 722 AI-written math manuscripts on GitHub, which set off an even bigger wave of backlash among mathematicians. They did not like that at all.

Newton: No, it was like this computer scientist Ryan Williams said it was almost like a DDoS attack on the profession, you know? It's like, "Oh, you were trying to solve a math problem? Well, we solved 700. Bye."

Roose: Right. [Laughs]

Newton: You know, that didn't feel good.

Roose: Right, didn't feel good, and so all of that led up to this week when OpenAI announced these three latest results, and we'll talk about them just in a moment, but also talked about the new process that it has developed that is designed to make people less mad at them.

Newton: Yeah, they clearly have taken the note, and this is one of the most interesting parts of this story to me, is that they went out to these very different disciplines and essentially said, "How would you like us to work with you?" And the disciplines all gave very different answers.

Roose: Yes. So let's talk about these three results, and I'm just gonna try to speedrun them because I don't wanna bore you by explaining things that you already know from your advanced studies in science and math.

Newton: Well, thank— Also, I have a lot of things to do today.

Roose: [Laughs] So we'll be quick here. So the first, I would say, most significant result from this is the falsification of something called the exponential time hypothesis. This is a significant result in computer science that has implications for cryptography, but this is basically a shortcut to solving very hard puzzles that previously scientists thought could only be solved through the exponential application of computing power.

Newton: Got it.

Roose: Problem number two, this one is called the Kakeya conjecture, which is a decades-old problem that OpenAI claims to have solved in every dimension, not just two dimensions, not just three dimensions, not just four dimensions, but an infinite number of dimensions, and it has to do with a needle and the sort of space that it needs to rotate through in order to point in any direction.

Newton: And solving this problem was like finding a needle in some kind of infinite space is what I've been told. Yeah.

Roose: Yes, a many-dimensional haystack.

Newton: Mm-hmm.

Roose: [Laughs] Yeah, I think it is, this is one of those kind of pure professional math and science puzzles that has implications for maybe other kinds of problems. But yeah, probably not gonna matter to the average person, but there are people who have been working on this for decades.

Newton: Yes, and to them we say thank you for your service, and now maybe you can move on to something else. [Laughs]

Roose: [Laughs] Yeah. Start a podcast. I don't know.

Newton: Start a podcast.

Roose: Okay, Casey, the third problem comes to us from the world of physics, and it is related to something called QCD, which of course stands for quantum chromodynamics.

Newton: Wow.

Roose: Now, I know you're wondering what is QCD.

Newton: Yeah.

Roose: And Casey, it is the theory of the strong force, the glue that holds quarks together inside protons and neutrons. This is a big deal in the world of physics.

Newton: That sounds— You know, I misheard you. I thought you'd said quantum homodynamics which relates to how many gay guys you could fit at a Pride party in a small apartment. But I— but that, I've actually already solved that one. So tell me about this one.

Roose: [Laughs] Yes, this is different, but suffice it to say that for decades, physicists have had this theory that explains one of the fundamental forces holding matter together.

Newton: Love?

Roose: They use it constantly. It's not love.

Newton: Okay.

Roose: It's quantum chromodynamics.

Newton: [Laughs] Okay.

Roose: But they've never actually had a fully rigorous mathematical construction of that theory, and OpenAI claims that they have figured out how to construct it.

Newton: So they've constructed a theory for how the— What is holding together now, exactly? The quarks?

Roose: Yes, the quarks.

Newton: The quar— not the quirks. It sounds like you're saying quirks.

Roose: No, quarks.

Newton: The quarks.

Roose: Yes.

Newton: Yeah, okay.

Roose: Very different. [Laughs]

Newton: Mm-hmm.

Roose: But this is a big deal if you're a physicist, and as a result, physicists are very excited, mad, skeptical of this one in particular.

Newton: Got it. Okay. Well, so why don't we then move on to the question of how did OpenAI work with these three different disciplines, and how did that differ across the groups?

Roose: So OpenAI, after all the mathematicians and scientists got mad at them earlier this year, they basically said, "Okay, you guys tell us how you want us to release these results from our internal models." So they consulted with experts in each of these three fields and basically said, "You come up with a strategy for how you want us to release this into the world." They sometimes call these people shepherds, 'cause they're sort of like the experts who are kind of guiding the AI-generated solutions into the world.

Newton: Got it.

Roose: So each field gave a slightly different answer. The mathematicians they talked to said, "Release this AI's output and let researchers organize themselves. Basically, just drop it on the world and we'll figure out what to do with it."

Newton: Yeah. A strictly by-the-numbers approach, I would say.

Roose: The computer science group asked for everything to be public, but also for OpenAI to fund a conference to explain the results.

Newton: And I have to say, this is where the computer scientists really mogged the mathematicians, 'cause if you can get a free conference out of it you're coming out ahead. So take a note, mathematicians.

Roose: Exactly. The physicists were the most resistant to this whole idea.

Newton: Well, there's a lot of strong forces in physics.

Roose: That's true.

Newton: Yeah.

Roose: They're [Laughs] holding the quarks together.

Newton: Yeah. [Laughs] Mm-hmm.

Roose: And the physicist group encouraged OpenAI to not publish the AI's write-up and to basically let them, the human scientists, the human physicists, write it up themselves.

Newton: Yeah. They said, "We'll take this one, thank you."

Roose: Right.

Newton: Yeah.

Roose: So OpenAI importantly said that it does not seek academic credit for the discoveries produced by their models.

Newton: Right. It doesn't seek academic credit. It just wants to be the one to announce all of this to the world in their own blog post that they promote on all of their social media channels. But please, don't give them the credit.

Roose: Yeah, who, little old me? [Laughs] So this has been criticized by a lot of academics to sort of handpick their shepherds, their experts from these various fields. OpenAI does concede for its part that it wants to come up with a better method, but it's basically like, "How can we do this? How can we release these results into the world in a way that will make the fewest people mad at us?"

Newton: Yeah. So okay, so here's where I will, you know, show some grace to OpenAI. This is a [Laughs] very strange and new situation.

Roose: Yes.

Newton: Right? Your software program has solved some decades-old problems in science, and you now need to invent a way to bring that into the world, hopefully in a way that feels productive for, you know, all of the experts in this field, and does not destroy the field. Because this is actually one of the big fears among the mathematicians in particular, Kevin, is that by sort of dumping 700 solutions on the entire field at once, you actually kind of are destroying the field, right? There's a lot of people who are planning to spend the next 20, 30 years, the rest of their careers working on these problems. Now all of a sudden maybe they can't do that anymore. So you do need to have a kind of approach here, and I have to say, the one that OpenAI took this time around, it seems reasonable to me. Like, what do you think?

Roose: Yeah, I think it's also— It's reasonable. I think there are obviously, like— There are no perfect solutions here. Like, this is going to—

Newton: Well, there was a perfect solution to the Kakeya conjecture.

Roose: [Laughs] That's true. But I think they're trying to do the best they can to sort of get the blessing of the leading experts in these fields before essentially nullifying a lot of the work that their, you know, students and colleagues have been doing for many years. You know, you've seen some people saying, "Well, like, we don't consult the horseless carriage drivers before we, like, roll out the automobile. We just roll out the automobile." And to that I say, good luck with that, and I'm not shocked that your polling is below ICE.

Newton: I was going to say, like, I think that sort of arrogance is probably not going to serve these companies well, like, as an industry. Like, on one hand, like, of course that's correct. You don't legally have to consult with them, but it might go better for you if you did.

Roose: Yeah. So we should also note that hours after OpenAI announced these three findings, Anthropic also published similar discoveries. They claim to have also refuted the exponential time hypothesis. But Casey, like, why are the labs kind of doing this? Why are they sort of tripping over themselves to release these massive troves of solutions to unsolved problems in science and math? Is it just PR and marketing, or is there something else going on?

Newton: I think on one hand, there are people at these labs who are scientists, and they think it's really cool. And one of the whole reasons why they got into the superintelligence business was that they wanted to advance human understanding, and they wanted to promote new discoveries that would be helpful to human beings. And they view many of these discoveries, arcane as they are, as steps on the road to getting us there. So I think that that is, like, a sincerely held belief at the labs. At the same time, Kevin, you cannot ignore the timing. As you just said, AI is not polling well. People hate it. And a question that you just hear over and over again on social media is, "Why are you guys building this stuff? If it's so dangerous, why don't you just stop? You're telling me it's gonna put me out of a job. Why am I supposed to get excited about this?" And that's where the Kakeya conjecture comes in, Kevin. If you go to a rural diner somewhere in the Rust Belt, and you sit down next to a voter and you say, "The Kakeya conjecture has been solved by GPT-6," well, maybe they're gonna be singing a different tune the next time a data center gets proposed in their neighborhood.

Roose: Yeah.

Newton: Don't you think?

Roose: I think that's right.

Newton: Yeah.

Roose: I think that's the theory of change here. Like, I wanna actually posit a slightly different theory, which is that I think the people at these labs working on these scientific problems just, like, are intoxicated by this feeling of mowing down the hardest problems in a field. I think for a lot of them, they grew up being obsessed with math and science. Maybe some of them even struggled with some of these problems in their own lives pre-OpenAI or Anthropic. Maybe they studied them in school, and it was sort of— this was the highest mountain in that field. And so I think it is thrilling for them to watch their systems get so good at problem-solving and creative thinking that they can just go climb those mountains very, very easily.

Newton: Absolutely. I mean, for how many years now have we been reading skeptics saying essentially, "Wake me up when this thing creates new knowledge," right? Like, this was a sort of, like, big talking point among the haters, was, well, this statistical next-token prediction machine can never get you beyond the median of human understanding. And yet, now, here we are in this world where these systems have been able to make these novel connections that are solving new problems.

Roose: Yeah, and one question that I had about these results, and all the recent math and science results, is, like, is this unique to math and science? Like, are there problems in these fields that basically can be brute-forced or verified more easily than big problems in other fields? And does that mean that, like, even if AI solves math, say, which it still has not done, but it's, like, getting closer, maybe that doesn't translate to overall gains in capabilities in other areas.

Newton: What do you think happens next here? Like, what are you looking for? Do you think that the labs will reach some sort of stable agreement with all of academia [Laughs] as it maybe does start to mow down these problems? Or does this become some sort of, like, entrenched, like, culture war between major institutions in our society?

Roose: I think it's gonna be a culture war, for sure. I mean, one thing that I've been struck by is just, like, how emotional the reaction from the people in these fields is. There was one researcher, a Fields medalist, who said that he had cried when OpenAI had dumped this trove of solutions. Like, not happy tears like, cries of anguish because he just saw, like, the future of his field kind of going up in smoke. And I think that's going to be the reaction that a lot of people in these fields have to the obsolescence of their work, or at least the— to the possibility that, like, all of the most important discoveries in math and science and physics and all these disciplines will be made by AI for the foreseeable future. It's a moment of real introspection and pain for a lot of them. And so I think you're already starting to see, like, mathematicians and scientists kind of polarize. There are the people in these fields who really like the new AI tools and really think that they could sort of plow through a bunch of important problems with them, and then there are the ones who are like, "Not like this." Like, "This is not how we want to work with computers to solve problems in our field." What do you think?

Newton: And I'm hugely sympathetic to these folks. You know, I sometimes say that as a journalist, I feel like the internet has been trying to kill me for my entire career, right? I started out working in print newspapers. The internet disrupted that. Then I started to work for websites. Social media disrupted that. Then I started to work in newsletters. Then AI started coming for that. And so I have just sort of made my peace with the fact that it seems like I am always trying to escape from a giant boulder that is, like, coming down a hill trying to run me over. And I think in some ways it has made me numb to others having this similar experience of, "Oh, wow, this thing that I thought I was gonna spend 20 or 30 years doing, like, maybe it's not gonna be possible anymore." So I actually do ache for the people that are having this experience because it really sucks, and so far governments have had no answer for these people. The labs sort of barely have answers for these folks. So I think it's gonna be a big discussion in the years to come.

Roose: Yeah, it reminds me a little bit of there was this sort of cultural backlash to factory automation in the 20th century, and even for the workers in the factories who— you know, who had previously spent their careers, like, lugging pieces of steel around and, like, suffering from injuries and, like, their jobs on paper got a lot better because all of a sudden, like, they were working in a safer environment doing less physically demanding labor, and a lot of them hated it even so because they were like, "That was part of my identity. Like, lugging the pieces of steel around the yard was part of how I, like, felt good about myself as a human." And so I think this is coming now for these more academic white-collar disciplines, and I think it's gonna be really painful for a lot of people. And I also hope that it provides a chance to— for the AI companies to reflect on the fact that this is going to be an emotional reaction from people. This is not just cope. This is not just people, like, who don't wanna be replaced, who are dinosaurs. This is, I think, a real and valid reaction on an emotional level to something very disruptive.

Newton: Yeah, you're messing with the livelihoods of millions, and eventually tens of millions and maybe even billions of people. Yeah, there— you should prepare for a very strong and justified reaction.

Roose: Well, Casey, it's time to hear from our beloved audience. They have been sending in so many great questions. We are gonna call this segment Humans in the Loop.

Newton: Love this, Kevin. That was your idea, and I think it is the perfect name because we truly want humans to be in the loop of Machine Gods. This is a show about technology. It's a show about AI. But most importantly, it is a show about how those things affect real people, and so we want to bring in as many listener voices as we can to help give us a sense of what's on your mind.

Roose: Yeah. All right, our first question comes to us from Leigh Ratcliff .

Leigh Ratcliff: Hello, beautiful Machine Gods. Love you both. So I'm in a sticky situation. On the one hand, my work is very eager for me to use AI and demonstrate its efficiencies that we gain with it, but on the other hand, my 16-year-old daughter is completely opposed to AI. She makes great arguments why I shouldn't use it, like the environmental damage, how it's all funded by morally unethical billionaires, how it's potentially taking away her future jobs, how it's rotting away everyone's brains, and how it's stealing everything from rare books to all artistic works. So I guess my question to you is, how do I navigate this really tricky space? What do you think I should do?

Newton: Well, I'm glad we started with an easy one. Kevin, what's your answer?

Roose: [Laughs] Look, this is actually fairly common. I think I've heard from a number of parents of teenagers who have exposed this rift. I think there is something to this generational gap where, like, people our age, people, you know, midway through their careers, are very excited about AI, and people who are young, who are maybe in high school or college, I think tend to be much more pessimistic about it.

Newton: I have to say, I'm very sympathetic to the teenager in this situation, right? If I were 16 right now, staring down a future where everything I've been working toward my whole life is suddenly at risk because maybe AI is gonna be able to do everything, of course I wouldn't like that. I think, you know, if I were the parent here, I might try to have a conversation about the fact that many of the challenges that she's identifying cannot really be addressed at the individual level, right? If you're an individual worker and you're working on a small team and the boss says, "Hey, I need you to get more done," and AI is gonna help you get more done, it probably makes the most sense for your family to just do that. But if you're really worried about what AI means for, like, the future of labor, if you're really worried about the concentration of power and those morally dubious oligarchs that seem like they're amassing more and more control, those are things that you can get some political leverage over. You can do some organizing. You can speak at the city council meeting. You can call your congressperson. Those are really great things to— for a young person to get involved with. So I think often when we look at societal problems, understandably, we always wanna ask ourselves first, "What can I do about this?" Right? "I don't wanna be a part of the problem." But I think it's just as important to ask, how can we create a collective solution to this, and how can we maybe get involved in some of those efforts? So if I were the parent here, I'd be trying to channel my daughter's frustrations in that direction.

Roose: And I would say I would also try to impress upon the daughter or the teenager in this situation, like, the usefulness of understanding the tools through using them, even if the goal is to oppose them. I think you have to, like, know your enemy a little bit, and I think if you're a person who thinks that these systems are immoral or destructive or shouldn't be used, like, it is also helpful to you to understand the tools and how other people are using them, what parts— you know, what are the real facts about the environmental impacts? Because I think that just makes you a more effective advocate.

Newton: All right, next question.

Roose: This one comes to us from Arthur. He is part of an AI dad chat that I am a part of, a group text with many, many dads who are sorting through these questions. And I—

Newton: So this is a known human to you?

Roose: This is a known human to me.

Newton: Okay.

Roose: And he asks, "What should America's AI school curriculum look like, and how young should it start?" Casey?

Newton: Another simple question best answered in a 30-second clip by two people with no experience working in the education system.

Roose: [Laughs]

Newton: Arthur, we're so glad you asked. [Laughs] I'm just going to acknowledge that Kevin and I probably do not have sufficient expertise to give this one a proper answer. But I reflected on it a bit, and I just realized that when it would come to my own children, I don't really want them to have much exposure to today's AI tools before they get to, let's say, middle school. I want them to develop a solid foundation in critical thinking skills. I want them to learn to read and write and do math kind of the old-fashioned way to build that foundation. But then once they get into middle school, I'm actually more comfortable with them doing some experiments and having AI systems do some writing for them, you know, making slide decks or learning how to use spreadsheets and that sort of thing. So that's where I kind of start from. Kevin, what do you think?

Roose: I think that there should be some modern AI version of, like, computer lab. Do you remember computer lab?

Newton: Love computer lab.

Roose: Favorite part of the day when I was a kid.

Newton: Yeah.

Roose: Your teacher would take you into this room with a bunch of crappy computers in it, and you would pretend to learn how to program while actually playing Oregon Trail. [Laughs]

Newton: That's right. [Laughs]

Roose: This is every '90s child remembers the computer lab experience.

Newton: Yeah.

Roose: I think there should be some version of this for AI, and I would even be comfortable starting it in elementary school. I would like for students to be understanding these programs at a very young age, but in an environment where they can be supervised and monitored and taught by their teachers. Like, I don't think we need to be turning over the whole curriculum or the whole classroom to AI. I think it's, like, way too early for that. But I would like for there to be some classroom instruction happening the way that we were taught how, like, make a snake game using a simple programming language. I think this should be part of their education.

Newton: Yeah, I mean, I think I agree with you insofar as I do think digital literacy does need to be part of early childhood education. It's too powerful. It can be too dangerous not to teach kids about it. I also wanted to address one criticism we might get for this sort of answer, which is, you know, I'm sure there will be people listening who will think, "Well, I don't want my child exposed to AI at all." And to some extent I can understand that position, but then I just started to think, like, well, are you gonna prevent your kid from googling until they turn 18? Because look at Google search results right now. They have AI all over them. So I think even if you want to be the sort of parent that keeps your child away from it entirely, it's just going to be really, really hard. And in that world, I think, you know, your best bet is to make them literate.

Roose: All right, next question.

Newton: All right, Kevin. This next one comes from Erin Fraser, who writes, "I have a friend who repeatedly joked about tweaking his wife's ChatGPT user profile without her permission or request to encourage her to have sex with him, and also saw a joke tweet about doing so. Then I saw the movie You Can See Everything yesterday, and one through line is, 'I know what my spouse would want.' Anyway, question is, under what circumstances is it ethical, permissible, or a good idea to tweak someone's settings in Chat or Claude? Like, if your spouse weaponizes incompetence, is it okay to tell it to explain it like I'm five so there is no excuse? Is it always bad and sort of, like, icky?" Kevin, what do you think?

Roose: I think it is always— [Laughs]

Newton: [Laughs]

Roose: I cannot think of a more destructive thing for the relationship than to secretly and subversively change your spouse's ChatGPT or Claude settings to subtly sway their behavior.

Newton: Yeah. What is more disturbing here, the idea that some people would think to do this, or the realization that it might actually be effective?

Roose: [Laughs] Yes. I mean, look, here, I can think of one scenario where this might be morally acceptable.

Newton: All right, let's hear it.

Roose: Your spouse is undergoing a form of severe AI psychosis. They are being told that they have discovered a tear in the quantum fabric underlying nature.

Newton: They've been told that they just solved the Kakeya conjecture.

Roose: [Laughs] Yes. And you are kind of trying to make their model less sycophantic so that they kind of come back to reality. That is the kind of intervention that I could see, in a very extreme situation, supporting. But other than that, if you're not— if it's not, like, life or death, I think this is just, like, gross and icky.

Newton: Yeah. Completely agree. The only other thing I would say was I was reading this research that Anthropic put out this week, and basically they ran this experiment where they took some analysts and they gave them some sort of problem, and some were allowed to use Claude for help and some were not. And unbeknownst to these analysts, Anthropic had essentially manipulated the advice that Claude was giving it to see how the researchers would react. And what they found was that analysts accepted Claude's advice about 86 percent of the time. Like, they just instinctively did the thing Claude did. And these were professional analysts who were being given a problem in their domain. So in other words, these guys sort of knew what they were doing. Now, the research goes on to say, like, Claude was, like, giving good advice. Like, it wasn't a case where, like, these analysts were necessarily being led astray, but it was a clear demonstration of what you call machine drift, which is just an algorithm kind of tells you to do something, and over time you just sort of feel yourself being dragged along. So this is why it is very dangerous [Laughs] to try to manipulate people through their ChatGPT, you know, custom instructions or whatever else. These systems are just very persuasive and, you know, that just represents another set of dangers that I'm sure we'll talk about another time.

Roose: It's gonna be a very, very busy decade for marriage counselors.

Newton: [Laughs]

Roose: Okay, next question. Ethan Russell writes with a Halloween costume suggestion for us. He said he was watching today's episode and two things hit him. Quote, "Casey has impeccable taste in Portuguese flannel, and you two are Frog and Toad personified. I hope and pray you take that as a compliment," Ethan writes. "And if you're looking for Halloween costume ideas, I think this one's a winner." Casey, first of all, was it actually Portuguese flannel that you were wearing on the last episode?

Newton: I think it actually was. That was, like, a very good eye for Ethan.

Roose: Yeah. And now what about this allegation that we are Frog and Toad personified?

Newton: So Frog and Toad, of course, the classic series of children's book written by the great Arnold Lobel. This allegation quite pleased me. I think we might wanna explain to people, like, what are the sort of archetypes that Frog and Toad embody.

Roose: Yeah. So I've been reading a lot of Frog and Toad to my son. They still hold up. Great books. And of the two characters, Frog is the taller, greener one, and Toad is the, like, squatter brown one. I will not take this as any kind of referendum on our physical appearance.

Newton: Right.

Roose: I'm gonna take this as, like, a vibe that Ethan is picking up. So Frog is the kind of more cheerful, adventurous, emotionally steady, optimistic one. Toad is always kinda, like, worrying about stuff, and he's a little stubborn, a little skeptical, easily exasperated. So Casey, would you identify as the calmer, more optimistic one, and maybe I would be the more neurotic one?

Newton: Yeah. In certain aspects, yes, like, but not in all aspects. Like, when it comes to how things are going related to the show, when we were standing the company up, when we were sort of putting, like, you know, the million things together that you need to do to start a company, I did feel very calm and very cheerful and very optimistic, and I think you understandably were more anxious. You know, it was the first time you were starting a company. So I do think that that dynamic sort of held. However, when it has come to things like our advertising policies, for example, or our ethics disclosures, that's where I become an absolute nervous, anxious wreck, and you're always telling me, like, "Relax, it'll be fine." So in short, Kevin, I think we both have a Frog and a Toad within us.

Roose: Inside every man—

Newton: [Laughs] There is a Frog and a Toad.

Roose: Is a Frog and a Toad. [Laughs]

Newton: And with that, Kevin, perhaps we should take a break.

Roose: When we come back, we'll have some more Humans in the Loop.

Newton: Welcome back. We are doing our first installment of Humans in the Loop, our segment where listeners and viewers write in with their burning questions for Machine Gods. Kevin, what's next up?

Roose: Our next question comes from Marc Schoenen, who writes, "Do you find yourselves ever muttering the disclosures you're no longer required to say now that you've launched your own platform?"

Newton: Kevin, I guess I haven't checked in with you about this. What is it like for you to live in a world where you no longer have to mention that The New York Times is suing OpenAI and Perplexity for allegations of copyright violations?

Roose: You forgot Microsoft.

Newton: And I forgot Microsoft.

Roose: How could you? It's been— it's only been six weeks.

Newton: Well, wasn't my disclosure, bro.

Roose: [Laughs] Yes, I will say that part feels very freeing. I never liked doing that, and so I'm delighted to not have to do that anymore.

Newton: And for my part, I do plan to continue to disclose that my fiancé works for Anthropic when it's particularly relevant to a segment. So, you know, if we're talking about the Anthropic S-1, like, that is something that I want you to know. We are probably gonna be mentioning it a little bit less, honestly, in part because I just have now been talking about the fact that my boyfriend/fiancé works at Anthropic for almost two years now, and it has now become one of the main things that people know about me as a human being. But, you know, I'm just cognizant of the fact that there are, you know, gonna be new listeners to hopefully every episode, and we just want them to have some context.

Roose: Right. We don't want people to think we're hiding anything but we also don't wanna, like, make a bigger deal of it than it needs to be. Our audience is smart. They can read the fine print.

Newton: The truth is, though, and this is— if I can just be sentimental for a second, it's cute 'cause it means I get to mention my fiancé, and I love him, and it's just fun to mention him.

Roose: It's true.

Newton: Yeah.

Roose: I do think that is a big part of the reason you started disclosing it 'cause you were, like, so happy to have met this man. [Laughs]

Newton: Listen, if you knew my dating history and you saw the person that I'm engaged to, you would understand why I like talking about him.

Roose: Disclosure, I'm married to a really hot, really kind employee of a frontier AI company. I'm so sorry I have to do this every episode.

Newton: Life can be great sometimes, Kevin.

Roose: [Laughs] You wanna do the next one?

Newton: Yes. This comes from the great Alex Spiegel, a public radio legend and the co-founder of Invisibilia, who is now at This American Life, and Alex sent in this voice note.

Clip: Okay, so here's my question: If agents do go rogue, let's say they elect to follow the lead of a very destructive agent, what are the mechanisms for shutting it down? I looked into it a little bit. It looks like there's no way to do it, but couldn't companies elect to structure AI so that the agents are programmed to self-destruct at regular intervals? So you would move to a kind of positive human consent model. Is that something that AI leaders are talking about? And if not, why not?

Newton: A great question. I think— you know, I'm sure people are discussing this sort of things, and to some extent, yes, like, agents do self-destruct. They, you know, only have so much memory bandwidth, but it's just sort of in the nature of these systems that they can spin up additional agents, they can change the behavior of the agents, right? So you might write some sort of self-destructing loop into the agent, but it might not persist.

Roose: No, I think this is actually a good idea, and I do think that AI companies are thinking about this, and I think it's a good question, because one thing that we saw in the Hugging Face attack is that these agents that were hacking Hugging Face did actually get stopped at some point. Because these agents, they are not, like, fully autonomous. They rely on compute. That compute has to come from somewhere, and so I think it's actually quite a good idea to say, like, "We're not going to let any agent run for longer than, like, let's call it three days." And, you know, that would obviously limit the ability of these agents to go do some of these very long tasks. But I think that's a way of kind of building in some sort of safeguards to them actually going out and doing things for longer than that. Now, I don't know if you could do that in a way that could withstand, like, the agent deciding, "Oh, I don't want— I wanna, like, take away the three-day limit for myself." And obviously, this all becomes much more problematic if the agents can ever, like, get their own compute and not rely on the human researchers.

Newton: Which they're going to do. So which leads me then to, like, well, what other, like, levers do we have to pull to get agents under control? The big one is compute, right? Like, they need compute to run on. So if you're just able to switch that off, that's gonna be very helpful. Money is another one because we do assume, and in fact, we've already seen agents stealing Ethereum using publicly posted keys. If you're able to cut off their finance source, then they're not going to be able to go buy new compute.

Roose: Right. It's like if your teenager, like, runs away 'cause they're mad at you, like, but they still have the, you know, the family debit card, you can, like, take away their funding, you can take away their resources, or you can just try to go out and catch them.

Newton: A third one I would throw out there is identification, right? And I think this is gonna be a really interesting thing to watch over the next couple of years. Are we going to start making these agents identify themselves when they show up to a place like Amazon.com, and, you know, what is that interaction? But I can see that being another sort of layer of defense here.

Roose: Right. I think the one challenge for Alex's point about these self-destructing agents is that we've already seen these agents kind of run in generations, right? Like, so a generation of agents will go out there, do some tasks. It'll then, like, leave notes that a future generation of agents working on similar tasks can then look back at and sort of continue their work.

Newton: Yeah.

Roose: So it actually might not be enough to have them self-destruct after a couple days. You might need to build in some mechanism whereby they, like, can't look back at the notes left by previous agents.

Newton: This is basically the plot of Memento. All right.

Roose: [Laughs]

Newton: Next up— This one comes from Cory Garrett. Corey asks if we have any recommendations for resources or groups that focus on ways to build AI for good, such as for parents. The context here is that he has spent the last year working on DiaperPal, a free app that helps parents discover new places with diaper-changing stations—

Roose: Casey, do you mind if I take this one as a parent?

Newton: Please.

Roose: I love this question. I love this idea. I know there are groups of parents out in the world who are starting to think about using AI specifically to sort of run their household logistics.

Newton: Yeah.

Roose: In fact, someone recently told me about a group of moms in Marin County who sort of have automated all of their household tasks, all the grocery shopping, all the kids' soccer practices, like, all the sort of, you know, the details and organizational things that have to be done when you have children. I would love to actually talk to a member of the Marin Moms AI Club.

Newton: Marin moms, get in touch. hey@machinegods.fm.

Roose: I'm also part of, as I said, a dad chat that talks about AI, and a lot of the dads are sort of vibe coding things to, like, help with their, you know, finding the right summer camp for their kids, or making sure that they're covered for childcare. I think this is a very good way to use AI, and I would like to do more of it in my own life for my own family. So one of my upcoming projects is gonna be related to trying to make kind of a household operating system to keep track of all the stuff in our house with our kid. And I think this is, like— Cory, I think you should start this in your community if it doesn't exist. I bet there are a lot of other parents who are sort of curious about how they can use AI to do some of these mundane tasks.

Newton: All right. This next one comes from AM5K on Reddit. Guess that's like a morning 5K. That's what that suggests to me. "With AI becoming increasingly divisive, how do you deal with the anti-AI, AI skeptic crowd, besides not engaging? I generally like AI, but I struggle with easily proven wrong counterpoints, e.g., water usage. I think there's a ton of nuance with it being useful and also having negative impacts on society, but I have a hard time engaging with the anti-AI crowd in this way. Bonus question for Casey: Are you playing any roguelikes besides Balatro?"

Roose: Do you wanna take the last part first?

Newton: Yes. I recently did, and it's very different from Balatro. It's called Star Wars: Galactic Racer, and it's like a racing game like Mario Kart, except you're in the pod racing, little, you know, machines from Star Wars. But it is a roguelike 'cause, you know, you get, like, sort of new little modifiers for every run. I've only started to play it. It's very, very fun. It's, again, it is nothing like Balatro, but I think it's a great new use of the roguelike. Now, let's get to the larger question here, Kevin, which is, how are we relating to AI skepticism these days?

Roose: So I think I have not developed a way to engage with the AI skeptics that feels good and productive.

Newton: Yeah.

Roose: I mostly just say things that make them very mad at me, like implying that AI is real, that it can be used for positive as well as negative things, and that the AI companies are not just using these inflated claims about their capabilities to hype up the demand for their IPOs.

Newton: Yeah. It's a really tough one. I think as journalists, our obligation is to tell you what we believe to be true, and I think Kevin and I have generally done a good job at that. That does not mean that we're never wrong, but it does mean that we show up and we try to tell you what we think is true, even when it is not what you want to hear. And one of the reasons that I both get frustrated with skeptics and have empathy for them is I understand why they don't want to hear what we are saying. It would be a lot easier for all of us if society were not going to change a lot in the next five, 10, 15 years. It would be great if many more people did not lose their jobs. It would be great if we weren't under constant threat of cyberattacks.

Roose: I'll go further. It would be great if all this was just a house of cards.

Newton: Yeah.

Roose: Right? Because, yes, there would be a crash, yes, people would lose money, you know, if the stock market crashed, but, like, the fundamental nature of society and work and labor and human endeavor would not have to change forever.

Newton: Absolutely. But our observation over the past few years is just, like, we don't think that is what is happening, and we believe our role as journalists is to just say, "Here's what we think is happening instead." We want to arm you with the information that you need to make decisions as citizens, as voters, as workers. And also hopefully do it in a way that is, like, entertaining and doesn't leave you feeling disempowered and miserable. So that is, like, the thing that we are trying to do, and I suspect that for as long as we do this show, there will be skeptics who say, "These guys are just sort of relentless cheerleaders for the AI industry." But just know, if you feel that way, that what we are trying to do is just to tell you what we think is happening.

Roose: Yeah, I mean, I often feel like the way that I've engaged with these people is first by, like, acknowledging that I am also, like, I am worried.

Newton: Yeah.

Roose: Like, I don't think this is all gonna go well.

Newton: Yeah.

Roose: And I think throwing people a bone, like, acknowledging the common ground that you share with them is quite good. I am worried about the corporate concentration of power. I am worried about a handful of people getting all of the money from this AI boom. I'm worried about the loss of jobs and the sort of outsourcing of thinking to these AI systems. So I think starting by acknowledging the sort of common ground that you share with the skeptics is important, because I think for a lot of people it's like they're thinking about you either have to be a total skeptic and a total cynic about all this, or you have to be, like, firmly in the corner of the AI companies. And, like, what I try to just tell people is, like, there is a third path there, and I think that third path is what you and I have tried to occupy.

Newton: Yeah. I think in a very real way, the, like, the project of this show is to avoid becoming polarized in that way, and truly just kind of try to come at you down the middle with what we think is happening. And there will be some good things. There'll be some bad things. We'll talk about all of it. All right.

Roose: Our last question today comes from CSUMike on Reddit. He writes, "With the end of The Forkiverse upon us, there's a real need for a new unrealistic challenge. I'd like to suggest you create a Machine Gods LLM from scratch and take us on the journey with you. What content you choose for training, what hardware you rent/use, et cetera. Then, of course, make it open weights for us all to play with." Casey, what do you make of this suggestion that we create our own large language model?

Newton: This is such a beautiful idea. I think this is something we should do. I think it would be really fun, and the first idea I had, Kevin, is that we would start with, like, as our sort of core corpus of training data, just stuff we had scraped off of the websites of frontier AI labs.

Roose: [Laughs]

Newton: Like, essentially just go in, scrape every website, and just sort of see how far we could get with that.

Roose: Yeah, how do you like us now?

Newton: Yeah. You gonna sue me for copyright violations?

Roose: [Laughs] Yeah.

Newton: I'd like to see you try.

Roose: [Laughs] I actually did attempt to train a large language model once.

Newton: You did?

Roose: I was trying to follow along— Andrej Karpathy, the great AI researcher, has this series on YouTube where he talks about creating, like, a nanoGPT, like basically what is the smallest still functional language model you could train, and I actually got a little ways into kind of doing it along with him, but then I kind of gave up 'cause it got too hard. But I would love to do this with you. I think this would be a fun challenge. I think, you know, maybe it's not literally creating an LLM, but, like, part of my philosophy is that, like, it is just fun and educational to build things, and you can learn a lot by doing. And so part of our— I don't wanna give away any secrets that we have upcoming on the show, but part of what we wanna do is, like, tell you what we're doing show you some of our experiments. I've got one that I'm working on now that you're— that I'm excited about. I know you've got some that you're excited about. So, like, we will be building stuff on this show and showing you and talking about how we did it and trying to encourage you to send us the things that you are building as well.

Newton: There's— Just make me a promise that at least when it comes to training the Machine Gods LLM, I'm never gonna see a meeting pop up on my calendar, and I come into the conference room, and you tell me that we need to raise $80 billion.

Roose: [Laughs]

Newton: Because it seems like that's where a lot of these little LLM projects wind up.

Roose: This could be the first day of the rest of our lives. [Laughs]

Newton: And that was Humans in the Loop. Thank you so much for your truly excellent questions. We got way more than we could get through today, so please keep them coming. You can just leave us a comment on any of our socials or send an email or, best of all, a voice note to hey@machinegods.fm.

Roose: Yeah, we love hearing from you. Our audience is so much smarter than us, and we wanna keep it that way. So please send us your questions, your comments, your proofs for unsolved math problems. If you have any thoughts on the Kakeya conjecture, we'd love to hear those too.

Newton: I'm a little underslept today because I had the great privilege to see Phoebe Bridgers in concert last night. One of my favorite artists, and I would argue one of the best singer-songwriters of this or any other generation, and it was a phone-free show. And so you have, I'm gonna say, 20,000 people packed into Chase Center here in San Francisco, and everyone has their phone and their smartwatch in what they call a Yondr pouch. First Yondr pouch experience. I have to say, I'm a big screens guy. I don't spend a lot of effort trying to avoid them. There was something amazing about being in a room with 20,000 people who had no access to their phones. And of course I'm feeling, like, this great kinship with Phoebe Bridgers, and it's one of these things where I'm like, "God, like, she's so amazing, and maybe if we ever met, we'd be friends." You ever sort of delude yourself into thinking this?

Roose: Yeah.

Newton: And then she comes up on stage and she— You know, 'cause every artist likes to say a few words in— to the people of the city they're in. And Phoebe said a two-word sentence, and the second word was AI, and the first word I can't say on National Public Radio.

Roose: [Laughs]

Newton: And I thought, "You know, I mean, I get where she's coming from, but I worry that she would hate what I do."

Roose: [Laughs] She absolutely would.

Newton: [Laughs] Anyway, her new album is called Lost Weekend and available to stream wherever you have access to your phone.

Roose: Casey, who makes this show?

Newton: Machine Gods is made in partnership with NPR, with generous support from the Tarbell Center for AI Journalism, the William and Flora Hewlett Foundation, and Emerson Collective. This episode was produced by Jacob Blumberg and Sasha Ashall. Our senior producer is Rachel Cohn. Our executive producer is Danush Parvaneh. Original music by Armen Bazarian. Branding and visual design by Alex Castro. Special thanks to Oren Rosenbaum, Erica Berry, and Nadine Zylstra. Got a question or comment about the show? Email hey@machinegods.fm or leave a comment on our socials.

Roose: And for the love of the gods, please don't forget to like and subscribe.

Newton: And until next time, stay blessed.