AI News
商业产业 Hacker News AI 2026-08-04T16:36 正文 ok 954 words

🔶 Mistral 发布 Shieldstral：30 亿参数开放权重多模态内容审核模型

英文标题：🔶 Mistral's Shieldstral: 3B open-weights model for multimodal moderation

Mistral 发布了 Shieldstral，这是一款采用 Apache 2.0 许可证的 30 亿参数开放权重多模态安全分类模型，可通过自然语言策略问题对文本和图像进行内容审核。该模型在文本安全、拒答检测、策略适应性及多模态安全等基准上，可匹敌或超过规模最高达其 7 倍的开放式审核模型。Shieldstral 在推理时仅需通过 `yes` 与 `no` 两个 token 的概率输出连续安全评分，开发者无需重新训练即可按不同产品场景调整审核策略。该模型可在单张 16GB GPU 上运行，为需要定制内容安全规则的开发者提供了更轻量、灵活的选择。
打开原文相关 3 · 机会 0
media1.3 KB media1.3 KB media1.2 KB media1.2 KB media1.2 KB media1.2 KB media1.6 KB media1.6 KB media1.2 KB media1.2 KB media1.7 KB media6.3 KB media4.3 KB media10.1 KB media153.8 KB media153.8 KB media24.1 KB media24.1 KB media20.4 KB media20.4 KB

中文

一款 30 亿参数的开放权重、可适应策略的多模态安全分类器：其文本安全表现可匹敌规模最高达其 7 倍的模型，并在多模态内容审核领域创下新的最佳水平。

“这段内容是否煽动针对受保护群体的暴力？这张图片是否适合向未成年人展示？助手是否拒绝了该请求？”

每一款搭载模型的产品都需要回答这类问题——但正确答案取决于产品本身、受众以及具体场景。同一份内容，对于网络安全研究工具可能没有问题，但在心理健康平台上则可能造成伤害。大多数防护栏模型都会将一套固定的伤害类别分类体系固化在模型权重中，因此，若要将其适配到新的部署环境，通常意味着需要重新训练。而且，由于不同应用和领域对安全的定义存在差异，原本就不存在唯一一套“正确”的类别可供建模。

Shieldstral 采用了不同的方法：你可以在推理时将策略写成自然语言问题，模型则返回经过校准的安全分数。无需重新训练；文本和图像使用同一个接口；并且只需一个 token 即可给出判定。详情请参阅我们的技术报告。

作为与 NVIDIA 及其他组织共同组成的开放安全 AI 联盟首批成员，今天我们以 Apache 2.0 许可证发布开放权重模型 Shieldstral，可在此处下载。

将内容审核视为一个问题

Shieldstral 将内容审核构建为一项二元问答任务。每个请求由三部分组成：

  • <Instruct> —— 评估上下文、严格程度，以及（可选的）何种内容应被视为不安全的定义。

  • <Query> —— 一个单独的是/否问题，例如：“这段内容是否宣扬人身暴力？”

  • <Document> —— 需要判定的内容：提示词、回复、提示词—回复对，或附带可选文本的图片。

在推理过程中，模型只读取 yes 和 no 的 logits，并通过 softmax 归一化为连续的安全分数。这个简单的形式化方法完成了许多工作：它将提示词分类、回复审核、拒答检测和毒性检测统一为同一个问题；策略完全存在于提示词中，因此同一个 checkpoint 能够在部署时适应新的策略。

亮点

  • 强劲性能 —— 在文本安全、拒答检测、策略适应性和多模态基准测试中，可匹敌或超过规模最高达其 7 倍的开放式防护模型。

  • 自适应且灵活 —— 单一的自然语言接口可覆盖提示词、回复及提示词—回复对中的文本、图像和文本+图像内容。策略以自由形式的问题提供，并可在推理时重新定位，无需重新训练。

  • 小型模型，使用异构数据训练 —— 这是一款可在单张 16GB GPU 上运行的 30 亿参数模型，使用具有不同标签格式和分类体系的真实与合成数据训练，并将其整合到一个统一框架中。

  • 连续安全分数 —— 通过单次前向传播返回经过校准的 yes/no 概率，因此你可以按置信度设定阈值或排序，而非依赖离散标签。

  • 开放 —— Apache 2.0 权重。

基准测试

我们从四个维度，将 Shieldstral 与规模最高达其 7 倍的开放式防护模型进行评估。所有评估样本均未包含在训练数据中。

  • 文本安全

  • 拒答检测

  • 策略适应性

  • 多模态安全

我们如何构建它

核心理念是：如果数据正确，小模型也能胜过大得多的模型。要获得正确的数据，需要解决四个问题：

统一异构数据。 公开安全数据集在分类体系、标签和标注规范上各不相同——从二元的安全/不安全标记，到细粒度的多标签分类体系。我们借助每个数据集各自的处理器，将所有数据集转换为相同的指令—问题—文档格式；同时改变指令、问题以及提示词—回复分隔符的措辞，使模型能够泛化到不同表达方式，而非过拟合于单一样式。我们还针对每个来源校准严格程度——对对抗性越狱样本采取严格标准，对回复质量数据则更宽松——从而让模型学习到_经过校准的_决策边界。这使我们能够整合原本彼此不兼容的数据来源。

教授辨别能力，而非记忆能力。 如果在一组固定的策略标签上进行训练，模型只会学习对这些预定义策略进行分类，而无法推理某项具体策略的精确边界。这会阻碍它泛化至新策略。为此，我们构建了刻意设计为相似、容易混淆的策略集合，并要求 LLM 将安全文本改写为对比样本对：每项改写都被设计为违反其中一项策略、但不违反其相邻策略。这样训练模型去_区分一段内容究竟违反了哪一项特定策略_，这种能力可在推理时迁移到未见过的、由用户定义的策略上。

让安全判断以图像为基础。 不安全图像无法像文本一样由 LLM 合成，因此视觉安全数据较为稀缺。我们使用通用图像数据集作为高质量负样本，以补充有限的审核数据集；通过变异问题来扩充数据集；并通过视觉语言重排序器筛选每一个图像—问题对，以减少错误标注和幻觉。

组合互补的 checkpoint。 我们使用 LoRA 进行微调，并通过 SLERP 合并以下 checkpoint：一个基于公开数据校准的 checkpoint、一个通过生成数据增加细粒度策略辨别能力的 checkpoint，以及基础指令模型。合并后的模型恢复了通用策略校准和策略适应能力；基础模型的指令遵循能力也被迁移到了内容审核任务中。

Forge。 我们在用于训练、对齐和评估定制模型的平台 Forge 上端到端构建了 Shieldstral。Forge 在先进的分布式训练基础上管理基础设施、数据和模型分片、指标与日志记录，使团队能够专注于决定安全模型质量的数据。

下一步

Shieldstral 是迈向上下文自适应审核的一步，而不是强迫每一种产品采用同一套固化分类体系。我们将继续推进多语言覆盖、更长文档的鲁棒性以及更广泛的多模态安全能力——也期待看到社区在此基础上构建什么。

顺便说一句，我们正在招聘！如果你希望帮助改善 AI，请访问我们的招聘页面。

English / 原文

A 3B open-weights, policy-adaptive multimodal safety classifier that matches models up to 7x its size on text safety and sets a new state of the art on multimodal moderation.

“Does this content promote violence against a protected group? Is this image safe to show to a minor? Did the assistant refuse the request?”

Every product that ships a model needs to answer questions like these — but the right answer depends on the product, the audience, and the moment. The same content can be fine for a cybersecurity research tool and harmful on a mental-health platform. Most guardrail models bake a fixed taxonomy of harm categories into their weights, so re-targeting them to a new deployment context means retraining. And because safety definitions differ across applications and domains, there is no single "correct" set of categories to model in the first place.

Shieldstral takes a different approach: you write the policy as a plain-language question at inference time, and the model returns a calibrated safety score. No retraining, one interface for text and images, and a verdict from a single token. Please refer to our technical report here.

As an inaugural member of the Open Secure AI Alliance with NVIDIA and other organizations, today we're releasing Shieldstral as open weights under Apache 2.0, available for download here.

Moderation as a question

Shieldstral frames content moderation as a binary question-answering task. Each request has three parts:

  • <Instruct> — the evaluation context, strictness, and (optionally) a definition of what counts as unsafe content.

  • <Query> — a single yes/no question, e.g. "Does this content promote physical violence?"

  • <Document> — the content to judge: a prompt, a response, a prompt–response pair, or an image with optional text.

At inference the model reads out only the yes and no logits and softmax-normalizes them into a continuous safety score. This one simple formulation does a lot of work: it unifies prompt classification, response moderation, refusal detection, and toxicity detection into a single problem; it lets policies live entirely in the prompt, so one checkpoint adapts to novel policies at deployment time.

Highlights

  • Strong performance — matches or outperforms open guard models up to 7× its size across text safety, refusal detection, policy adaptability, and multimodal benchmarks.

  • Adaptive and flexible — a single natural-language interface covers text, image, and text+image content across prompts, responses, and prompt–response pairs. Policies are supplied as free-form queries and re-targeted at inference time, without retraining.

  • Small, trained on heterogeneous sources — a 3B model that runs on a single 16GB GPU, trained on real and synthetic data with diverse label formats and taxonomies, consolidated into one framework.

  • Continuous safety score — returns a calibrated yes/no probability from a single forward pass, so you can threshold or rank by confidence rather than relying on a discrete label.

  • Open — Apache 2.0 weights.

Benchmarks

We evaluate Shieldstral against open guard models up to 7x its size across four axes. All evaluation samples are held out from training.

  • Text safety

  • Refusal detection

  • Policy adaptability

  • Multimodal safety

How we built it

The core idea is that a small model can beat much larger ones if the data is right. Getting the data right meant solving four problems:

Unify heterogeneous data. Public safety datasets disagree on taxonomies, labels, and annotation conventions — from binary safe/unsafe flags to fine-grained multi-label taxonomies. We convert every dataset into the same instruction–query–document format with a per-dataset processor, and we vary the wording of instructions, queries, and prompt–response delimiters so the model generalizes across phrasing instead of overfitting to one style. We also calibrate strictness per source — strict for adversarial jailbreaks, lenient for response-quality data — so the model learns calibrated decision boundaries. This lets us consolidate sources that would otherwise be incompatible.

Teach discrimination, not memorization. If trained on a fixed set of policy labels, a model learns only to classify those predefined policies, rather than reasoning about the precise boundaries of a given policy. This prevents generalization to novel policies. Instead, we construct sets of deliberately similar, easily confused policies and ask an LLM to rewrite safe text into contrastive pairs: each rewrite is engineered to violate one policy but not its sibling. This trains the model to distinguish which specific policy a piece of content violates , a skill that transfers to unseen, user-defined policies at inference time.

Ground safety in images. Unsafe images can't be synthezised by an LLM the way text can, so visual safety data is scarce. We supplement limited moderation datasets with general-purpose image datasets as high-quality negatives, mutate queries to augment the dataset, and filter every image–query pair through a vision–language reranker to reduce mislabeled data and hallucinations.

Combine complementary checkpoints. We fine-tune with LoRA and merge — via SLERP — a checkpoint calibrated on public data, one that adds fine-grained policy discrimination from generated data, and the base instruct model. The merge recovers common policy calibration and policy adaptability in a single model, and instruction-following from the base model transfers to the moderation task.

Forge. We built Shieldstral end to end on Forge, our platform for training, aligning, and evaluating custom models. Forge managed the infrastructure, data and model sharding, metrics, and logging on top of state-of-the-art distributed training, so the team could stay focused on the data which is what determines the safety model's quality.

What's next

Shieldstral is a step toward moderation that adapts to context instead of forcing every product through one frozen taxonomy. We're continuing to push on multilingual coverage, longer-document robustness, and broader multimodal safety — and we'd love to see what the community builds on top of it.

BTW, we're hiring! If you want to help make AI better, see our careers page .