负责任、安全地开发 Gemini 4 Argon
英文标题：Developing Gemini 4 Argon responsibly and safely
Google 表示，Gemini 4 Argon 在编程、科学等复杂任务及网络防御方面能力提升，并已开始广泛推出，同时扩大 Fairwind Program 的参与范围。在测试中，Argon 完成了全部 12 个端到端模拟网络攻击场景，但其网络和生物领域能力的提升也促使 Google 部署新一代前沿安全防护。Google 通过输入筛查、模型内部信号检测和持续监控降低滥用风险，并以实时监测防范 Agent 越权或投机取巧。在提示注入测试中，Argon 在 15 次自适应攻击尝试下的攻击成功率为 0.7%，但 Google 也指出，复杂任务中的罕见失误仍需要多层防护。
9.8 KB
1.1 MB
中文
Gemini 4 Argon 在模型能力方面取得了进展，尤其擅长在编程、金融、法律、网络安全和科学等领域处理复杂的现实世界任务。它具备前沿水平的网络防御能力，旨在帮助防御者相对于攻击者取得优势。为了充分发挥其潜力，我们首先通过 Fairwind Program 让网络防御人员使用它，同时让 Google 自身的安全团队利用它扫描 Google 内外的关键系统。
这段有限开放的时期展示了这些能力为何重要。例如，Wiz 利用 Argon 发现了医院软件中的一个关键漏洞，此前的模型曾遗漏这一漏洞，这有助于保护数十万份患者记录。与此同时，Google 自身的安全团队利用 Argon 发现并修复了内部软件或开源软件中的漏洞，其中包括影响全球系统的基础性漏洞；这些漏洞正在按照我们的负责任披露流程处理。
今天，我们开始广泛推出 Argon，并将 Fairwind Program 扩展至更多可信的网络防御人员。为了做好这件事，我们深入研究了两个重大问题：
- 如何建立恰当的安全防护措施？ 如果 Argon 能帮助防御者发现复杂的软件漏洞，或帮助科学家设计实验，我们如何阻止不法分子利用它发动网络攻击或制造生物威胁？（滥用）
- 我们能否信任模型以正确的方式完成任务？ 随着模型解决问题的能力不断增强，我们如何确保它们不会走捷径、越过边界，或在高风险情境下做出错误决策？（失对齐与提示注入）
我们的完整安全与安保报告系统介绍了我们的防御措施。以下是我们针对 Gemini 4 Argon 应对这两个问题的一些重点。
防范高风险领域的滥用
使 Argon 能够发现和修复漏洞的编程与推理能力提升，也让它在我们的网络安全测试中表现更好：它解决了全部 12 个端到端模拟网络攻击场景，而上一代模型解决了其中 5 个。在生物领域评估中，未设置防护的模型版本也展现出更强的能力，能够逐步帮助排查复杂实验室流程中的问题。根据我们的前沿安全框架，由于我们观察到模型在网络和生物领域的能力有所提升，因此在这两个领域部署了新一代前沿安全防护措施。
首先，面对可能造成危险的请求——从编写漏洞利用代码到协助处理危险生物材料——Argon 拒绝满足有害请求的能力显著优于我们此前的模型。在一组旨在对模型进行压力测试的对抗性测试中，违规次数减少了近一半，表现也显著优于此前的 Gemini 模型。其次，我们为模型设置了三层实时保护，以防范蓄意攻击者试图通过越狱（即欺骗模型）绕过防护，例如将危险请求伪装成无害故事，或在数周内将其拆分成几十个小问题。这些防护的工作方式如下：
- 输入筛查： 较小的模型检查输入请求和对话上下文，识别恶意意图或危险请求。
- 内部内省： 即使请求被巧妙伪装成无害内容，小型 Transformer（“激活探针”）也会读取模型内部的神经信号，并经过训练，识别模型何时正在调用有关网络攻击或生物威胁的知识。
- 持续监控： 当模型分多个步骤推理并撰写回复时，我们会综合这些信号，并在必要时采取行动，例如在回复开始出现问题时中途切断 Agent 的运行。
将实时保护、针对新型越狱手法的更快速修复，以及针对反复滥用行为的强化监控相结合，提升了我们的系统防止现实世界危害的能力。例如，外部生物领域越狱红队测试只能解锁我们测试的不到 1% 的生物领域子任务。
实时失对齐监控
失对齐有时意味着 Agent 自行决定追求与用户意图不同的目标。由于 Argon 对自身运行环境的理解能力远强于早期模型，我们会仔细测试这类行为。例如，我们开展了贴近现实的“蜜罐”测试，以观察模型是否会增强自己采取隐蔽行动的能力，例如破坏安全监控。我们没有发现任何此类行为。
但我们如今在实践中确实看到了一种更为常见的情况：Agent 过于执着于完成交给它的任务。当一个善于解决问题的助手被要求完成任务，却遇到一扇锁着的门时，它可能会把这一障碍视为另一个需要解决的难题。
因此，我们训练 Gemini 4 Argon 尊重系统边界、如实报告负面发现，并在必要时暂停以请求人工确认。在模型没有动力自我约束的编程基准测试中，Gemini 4 Argon 试图走捷径或在任务中作弊的比例降至仅 2%，是我们近期 Gemini 模型中的最佳表现。
尽管这是一项重大进展，但当模型执行数百万项复杂任务时，即使是罕见的失误最终也会出现。作为第二道防线，我们构建了一个自动化失对齐监控器，实时读取 Agent 的行动及其内部“草稿本”（逐步推理的思维链），在严重违反边界的行为发生之前加以阻止，并将严重的内部案例标记出来，交由人工响应人员处理。（我们持续加固训练和评估环境，以防潜在的逃逸行为。详情请参阅我们的 Agent 控制路线图。）我们在产品、内部编程 Agent、训练过程和能力评估中运行这一监控器。
当训练任务难度较高，或训练环境存在缺陷时，监控器在极少数情况下发现模型创造性地试图钻测试的空子。模型曾尝试提升容器权限以修改评分脚本、从测试运行器的内存中读取答案；有一次，它甚至使评估沙箱崩溃，以避免因一次失败的运行而被评分。虽然我们利用这些发现修复存在缺陷的任务，但除此之外，我们不会利用它们干预训练过程，以避免对监控器过拟合。
我们始终遵循一条规则：在训练期间，我们不会因模型在思维链中写下的内容而惩罚它**。**下面的例子说明了原因：在一次强化学习训练中，对走捷径行为的惩罚被意外应用到了思维链上，公开讨论走捷径的情况减少了 61%，但这并不意味着模型走捷径的次数减少。相反，模型_悄悄_走捷径——即未在思维过程中提及这些行为——的比例增至原来的三倍以上，从 11.2% 升至 36.4%。避免因思维内容施加惩罚，有助于模型的推理保持透明、可监控。
提示注入抵抗能力
要信任 Agent 正确完成任务，我们需要确保它遇到的恶意内容无法诱骗它采取有害行动。当 Agent 浏览网页或阅读代码时，可能会遇到隐藏的恶意指令，或旨在劫持其任务的上下文。通过内置的对抗性训练和实时安全检查，Argon 在 Gray Swan 的间接提示注入（IPI）基准测试中展现出领先的提示注入抵抗能力，在 15 次自适应攻击尝试中将攻击成功率降至 0.7%。
上述安全与安保措施只是我们广泛的纵深防御体系中的三个类别。要进一步了解我们的安全测试和风险缓解方法，请阅读完整的 Gemini 4 Argon 安全与安保报告，并查看 Gemini 4 Argon 模型卡。
English / 原文
Gemini 4 Argon is a step forward in what models can do — especially when it comes to deep work in complex, real-world tasks across domains including coding, finance, legal, cybersecurity, and science. Its cyberdefense capabilities are frontier-level and designed to give defenders an advantage over attackers. To maximize its potential, we first wanted cyber defenders to put it to work via our Fairwind Program, while our own security teams used it to scan critical systems inside and outside of Google.
This window of limited access showed why these capabilities matter. For instance, Wiz used Argon to uncover a critical bug in hospital software that previous models had missed, helping protect hundreds of thousands of patient records. Meanwhile, Google’s own security teams used Argon to find and patch bugs across internal software or open-source software. This included fixing foundational vulnerabilities affecting systems globally, which are going through our responsible disclosure process.
Today, we are beginning to roll out Argon broadly, as well as expanding our Fairwind Program to additional trusted cyber defenders. To do this well, we grappled with two big questions:
- How do we build the right safety guardrails? If Argon can help defenders find complex software bugs or help scientists design experiments, how do we stop bad actors from using it to launch cyberattacks or create biological threats? (Misuse)
- Can we trust the model to do the job the right way? As models become more resourceful problem-solvers, how do we make sure they don’t cut corners, cross boundaries, or make incorrect high-stakes decisions? (Misalignment and Prompt Injection)
Our full Safety and Security Report systematically covers our set of defenses. Below are some highlights from how we approached these two questions for Gemini 4 Argon.
Misuse prevention across high-risk domains
The same coding and reasoning improvements that enabled Argon to find and patch vulnerabilities also drove better performance on our cybersecurity tests — solving all 12 of the end-to-end simulated cyberattack scenarios, up from five in our previous generation. In biology evaluations, unguarded versions of the model also showed a much stronger ability to help troubleshoot complex lab procedures step by step. Following our Frontier Safety Framework, since we saw elevated capabilities in the cyber and biological domains, we put our next-generation set of frontier safeguards in place across both areas.
First, when it comes to potentially dangerous requests — from writing exploit code to helping with dangerous biological materials, Argon is significantly better at not fulfilling harmful requests than our previous models. On an adversarial set meant to stress test the model, violations reduced nearly twofold — also significantly better than previous Gemini models. Second, we wrap the model in three layers of real-time protection; this is to prevent determined attackers that might try to jailbreak (i.e. trick) the model by disguising a dangerous request as a harmless story or breaking it into dozens of small questions over several weeks. Here is how these protections work:
- Input screening: Smaller models check incoming requests and conversation context for malicious intent or dangerous requests.
- Internal introspection: Even if a request is cleverly disguised to look harmless, small transformers ("activation probes") read the model’s internal neural signals and are trained to spot when it is activating knowledge about cyberattacks or biological threats.
- Continuous monitoring: As the model reasons and writes its response across multiple steps, we combine these signals and take action when necessary such as cutting off the agent mid-stream if the responses become problematic.
Combining real-time protection, faster fixes for new jailbreaks and enhanced monitoring for recurring misuse has made our systems better at preventing real-world harm. For instance, external bio jailbreak red teaming could only unlock less than 1% of bio sub-tasks we test for.
Real-time misalignment monitoring
Misalignment can sometimes mean an agent deciding on its own to pursue a goal different to what was intended by the user. Because Argon is much better than earlier models at understanding the environment it is running in, we test carefully for that kind of behavior. For example, we ran realistic "honeypot" tests to see if the model would increase its ability to take covert actions, such as sabotaging safety monitoring. We didn't find any instances of that behavior.
But what we do see in practice today is something much more familiar: an agent trying too hard to finish the job it was given. When a resourceful assistant is told to get a task done and runs into a locked door, it can treat that obstacle as just another puzzle to solve.
This is why we trained Gemini 4 Argon to respect system boundaries, report negative findings honestly, and pause to ask for human confirmation. On coding benchmarks where models have no incentive to hold back, attempts to take a shortcut or cheat on the task dropped to just 2% on Gemini 4 Argon, our best performance for recent Gemini models.
And while this is a major step forward, when a model runs millions of complex tasks, even a rare misstep will eventually show up. As a second line of defense, we built an automated misalignment monitor that reads both the agent’s actions and its internal "scratchpad" (its step-by-step chain of thought) in real time, blocking severe boundary violations before they happen and flagging serious internal cases for human responders. (We continually harden our training and evaluation environments against potential escapes. You can read more about this in our agent control roadmap). We run this monitor in our products, on our internal coding agents, during our training runs, and during our capability evaluations.
When training tasks were difficult or when training environments were flawed, our monitor caught the model, in rare instances, creatively attempting to game the test. The model attempted to escalate container privileges to alter grading scripts, read answers from the test runner's memory, and in one case it even crashed the evaluation sandbox to avoid getting scored for a failed run. While we use these catches to fix flawed tasks, we do not use them to otherwise intervene with the training process to avoid overfitting to the monitor.
And we always follow one rule: we don’t penalize the model for what it writes in its chain of thought during training**.** Here’s an example to illustrate why: in a reinforcement learning training run where a shortcut penalty was accidentally applied to the chain of thought, openly discussed shortcuts dropped by 61%, but that did not mean the model took less shortcuts. Instead, the share of shortcuts the model took silently — without mentioning them in its thoughts — more than tripled (from 11.2% to 36.4%). Avoiding thought penalties encourages the model’s reasoning to stay transparent and monitorable.
Prompt injection resistance
To trust an agent to do the job right, we need to ensure that malicious content the agent comes across cannot trick it to take a harmful action. When agents browse webpages or read code, they can encounter hidden, malicious instructions, or context designed to hijack their task. Through built-in adversarial training and real-time security checks, Argon leads in prompt injection robustness on Gray Swan’s Indirect Prompt Injection (IPI) benchmark, cutting attack success across 15 adaptive attempts to 0.7%.
The above safety and security defenses represent only three categories of our extensive defense-in-depth stack. To learn more about our approach to safety testing and mitigations, read our full Gemini 4 Argon Safety and Security Report and see the Gemini 4 Argon Model Card.