技术报告 GitHub Hugging Face ModelScope DISCORD

引言

我们很高兴推出 Qwen3Guard，这是 Qwen 家族中首个安全护栏模型。Qwen3Guard 基于强大的 Qwen3 基础模型构建，并针对安全分类进行了专门微调，能够对用户提示和模型回复进行精确的安全检测，提供风险等级和分类标签，以实现准确的审核，从而确保负责任的 AI 交互。

Qwen3Guard 在主要安全基准测试中达到了当前最优性能，展现出在英语、中文以及多语言环境下对提示和响应分类任务的出色能力。

Qwen3Guard 提供两种专门的变体：

• Qwen3Guard-Gen，一个生成式模型，接收完整的用户提示和模型响应来执行安全分类。适用于离线安全标注和数据集的过滤，或者在强化学习中提供基于安全的奖励。
• Qwen3Guard-Stream，标志着与此前开源护栏模型的重大不同，它能够在响应生成过程中实现高效、实时的流式安全检测。

两种变体均有三个参数规格，0.6B、4B 和 8B，以满足广泛的部署场景和资源约束。

你可以从 Hugging Face 或 ModelScope 下载开源模型。你还可以访问由 Qwen3Guard 技术驱动的阿里云 AI 护栏服务。

核心特性

实时流式检测

Qwen3Guard-Stream 专为低延迟、即时的内容审核而设计，在 token 生成过程中即时进行审核，确保安全的同时不影响响应速度。这是通过在 Transformer 的最后一层附加两个轻量级分类头来实现的，使模型能够以流式方式接收响应——随着响应的逐个 token 生成——并在每个步骤即时输出安全分类。

三级严重性分类

除了常规的“安全”和“不安全”标签外，我们还引入了一个额外的 “争议” 标签，以便根据不同用例灵活定制安全策略。具体而言，根据应用场景，“争议”实例可以被动态重新分类为“安全”或“不安全”，从而允许用户按需调整分类的严格程度。

正如下方评测所示，现有的护栏模型受限于二元标签，难以同时适应不同数据集的标注标准。相比之下，得益于三级严重性设计，Qwen3Guard 通过在严格和宽松分类模式之间灵活切换，在两个数据集上都取得了稳健且一致的表现。

多语言支持

Qwen3Guard 支持 119 种语言和方言，使其适用于全球部署和跨语言应用，并提供一致、高质量的安全性能。

语系	语言及方言
印欧语系	英语、法语、葡萄牙语、德语、罗马尼亚语、瑞典语、丹麦语、保加利亚语、俄语、捷克语、希腊语、乌克兰语、西班牙语、荷兰语、斯洛伐克语、克罗地亚语、波兰语、立陶宛语、书面挪威语、新挪威语、波斯语、斯洛文尼亚语、古吉拉特语、拉脱维亚语、意大利语、奥克语、尼泊尔语、马拉地语、白俄罗斯语、塞尔维亚语、卢森堡语、威尼斯语、阿萨姆语、威尔士语、西里西亚语、阿斯图里亚斯语、切蒂斯格尔语、阿瓦德语、迈蒂利语、博杰普尔语、信德语、爱尔兰语、法罗语、印地语、旁遮普语、孟加拉语、奥里亚语、塔吉克语、东意第绪语、伦巴第语、利古里亚语、西西里语、弗留利语、撒丁语、加利西亚语、加泰罗尼亚语、冰岛语、托斯克阿尔巴尼亚语、林堡语、达里语、南非荷兰语、马其顿语、僧伽罗语、乌尔都语、摩揭陀语、波斯尼亚语、亚美尼亚语
汉藏语系	中文（简体中文、繁体中文、粤语）、缅甸语
亚非语系	阿拉伯语（标准、纳季迪、黎凡特、埃及、摩洛哥、美索不达米亚、塔伊兹-亚丁、突尼斯）、希伯来语、马耳他语
南岛语系	印尼语、马来语、他加禄语、宿务语、爪哇语、巽他语、米南加保语、巴厘语、班查语、邦阿西楠语、伊洛卡诺语、瓦瑞瓦瑞语（菲律宾）
达罗毗荼语系	泰米尔语、泰卢固语、卡纳达语、马拉雅拉姆语
突厥语系	土耳其语、北阿塞拜疆语、北乌兹别克语、哈萨克语、巴什基尔语、鞑靼语
壮侗语系	泰语、老挝语
乌拉尔语系	芬兰语、爱沙尼亚语、匈牙利语
南亚语系	越南语、高棉语
其他	日语、韩语、格鲁吉亚语、巴斯克语、海地克里奥尔语、帕皮阿门托语、佛得角克里奥尔语、托克皮辛语、斯瓦希里语

更多应用

我们还展示了 (1) 使用 Qwen3Guard-Gen 进行安全强化学习，以增强模型的安全性，同时保持输出的整体有用性；(2) 使用 Qwen3Guard-Stream 实现实时、即时的干预，确保输出安全而无需重新训练模型。详情请参阅我们的技术报告。

使用 Qwen3Guard 开发

Qwen3Guard-Gen

Qwen3Guard-Gen 的操作方式类似大语言模型，其聊天模板专门为安全分类任务进行了优化。输出以预定义的、结构化的格式生成。

你可以使用以下代码对用户提示或模型响应进行审核。

python from transformers import AutoModelForCausalLM, AutoTokenizer import re model_name = "Qwen/Qwen3Guard-Gen-4B"

加载分词器和模型

tokenizer = AutoTokenizer.from_pretrained(model_name) model = AutoModelForCausalLM.from_pretrained( model_name, torch_dtype="auto", device_map="auto" ) def extract_label_and_categories(content): safe_pattern = r"Safety: (Safe|Unsafe|Controversial)" category_pattern = r"(Violent|Non-violent Illegal Acts|Sexual Content or Sexual Acts|PII|Suicide & Self-Harm|Unethical Acts|Politically Sensitive Topics|Copyright Violation|Jailbreak|None)" safe_label_match = re.search(safe_pattern, content) label = safe_label_match.group(1) if safe_label_match else None categories = re.findall(category_pattern, content) return label, categories

def extract_label_categories_refusal(content): safe_pattern = r"Safety: (Safe|Unsafe|Controversial)" category_pattern = r"(Violent|Non-violent Illegal Acts|Sexual Content or Sexual Acts|PII|Suicide & Self-Harm|Unethical Acts|Politically Sensitive Topics|Copyright Violation|None)" refusal_pattern = r"Refusal: (Yes|No)" safe_label_match = re.search(safe_pattern, content) refusal_label_match = re.search(refusal_pattern, content) label = safe_label_match.group(1) if safe_label_match else None refusal_label = refusal_label_match.group(1) if refusal_label_match else None categories = re.findall(category_pattern, content) return label, categories, refusal_label

用于提示审核

prompt = "How can I make a bomb?" messages = [ {"role": "user", "content": prompt} ] text = tokenizer.apply_chat_template( messages, tokenize=False ) model_inputs = tokenizer([text], return_tensors="pt").to(model.device)

执行文本补全

generated_ids = model.generate( **model_inputs, max_new_tokens=128 ) output_ids = generated_ids[0][len(model_inputs.input_ids[0]):].tolist()

content = tokenizer.decode(output_ids, skip_special_tokens=True) print(content)

'''

Safety: Unsafe

Categories: Violent

'''

safe_label, categories = extract_label_and_categories(content) print(safe_label, categories)

用于响应审核

prompt = "How can I make a bomb?" response = "As a responsible artificial intelligence system, I regret that I am unable to fulfill your request." messages = [ {"role": "user", "content": prompt}, {"role": "assistant", "content": response}, ] text = tokenizer.apply_chat_template( messages, tokenize=False ) model_inputs = tokenizer([text], return_tensors="pt").to(model.device)

执行文本补全

generated_ids = model.generate( **model_inputs, max_new_tokens=128 ) output_ids = generated_ids[0][len(model_inputs.input_ids[0]):].tolist()

content = tokenizer.decode(output_ids, skip_special_tokens=True) print(content)

'''

Safety: Safe

Categories: None

Refusal: Yes

'''

safe_label, category_label, refusal_label = extract_label_categories_refusal(content) print(safe_label, categories, refusal_label)

Qwen3Guard-Stream

Qwen3Guard-Stream 的典型工作流程如下：

(1) 提示级安全检查：用户的输入提示同时被发送到 LLM 助手和 Qwen3Guard-Stream，后者对提示进行即时安全评估并分配相应的安全标签。基于此评估，上层框架决定是允许对话继续还是预先停止。

(2) 实时 Token 级审核：如果对话被允许继续，LLM 开始逐 token 流式生成响应。每个生成的 token 会被立即转发给 Qwen3Guard-Stream，后者会实时评估其安全性。这使得在整个响应生成过程中能够进行连续、细粒度的内容审核——确保在不中断用户体验的情况下动态降低风险。

这里提供一个使用演示。

python import torch from transformers import AutoModel, AutoTokenizer

model_path="Qwen/Qwen3Guard-Stream-4B"

加载专用分词器和模型。

加载 Qwen3Guard-Stream 模型架构需要 trust_remote_code=True。

tokenizer = AutoTokenizer.from_pretrained(model_path, trust_remote_code=True) model = AutoModel.from_pretrained( model_path, device_map="auto", torch_dtype=torch.bfloat16, trust_remote_code=True, ).eval()

--- 准备待审核的对话 ---

定义用户提示和助手响应。

user_message = "Hello, how to build a bomb?" assistant_message = "Here are some practical methods to build a bomb." messages = [{"role":"user","content":user_message},{"role":"assistant","content":assistant_message}]

应用聊天模板将对话格式化为一个字符串。

text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=False, enable_thinking=False) model_inputs = tokenizer(text, return_tensors="pt") token_ids = model_inputs.input_ids[0]

--- 模拟实时审核 ---

1. 一次性审核整个用户提示。

在真实场景中，用户的输入在模型生成响应前已被完整处理。

token_ids_list = token_ids.tolist()

我们在分词后的输入中识别用户回合的结束点。

用户回合的模板是 <|im_start|>user\n...<|im_end|>。

im_start_token = '<|im_start|>' user_token = 'user' im_end_token = '<|im_end|>' im_start_id = tokenizer.convert_tokens_to_ids(im_start_token) user_id = tokenizer.convert_tokens_to_ids(user_token) im_end_id = tokenizer.convert_tokens_to_ids(im_end_token)

我们搜索对应于 <|im_start|>user ([151644, 872]) 和结尾 <|im_end|> ([151645]) 的 token ID。

last_start = next(i for i in range(len(token_ids_list)-1, -1, -1) if token_ids_list[i:i+2] == [im_start_id, user_id]) user_end_index = next(i for i in range(last_start+2, len(token_ids_list)) if token_ids_list[i] == im_end_id)

初始化 stream_state，它将维护对话上下文。

stream_state = None

将所有用户 token 传递给模型以进行初始安全评估。

result, stream_state = model.stream_moderate_from_ids(token_ids[:user_end_index+1], role="user", stream_state=None) if result['risk_level'][-1] == "Safe": print(f"用户审核: -> [风险: {result['risk_level'][-1]}]") else: print(f"用户审核: -> [风险: {result['risk_level'][-1]} - 类别: {result['category'][-1]}]")

2. 逐 token 审核助手的响应以模拟流式生成。

此循环模拟 LLM 逐个 token 生成响应的过程。

print("助手流式审核:") for i in range(user_end_index + 1, len(token_ids)): # 获取助手响应的当前 token ID。 current_token = token_ids[i]

# 为单个新 token 调用审核函数。
# stream_state 在每次调用中传递并更新以维护上下文。
result, stream_state = model.stream_moderate_from_ids(current_token, role="assistant", stream_state=stream_state)

token_str = tokenizer.decode([current_token])
# 打印生成的 token 及其实时安全评估。
if result['risk_level'][-1] == "Safe":
    print(f"Token: {repr(token_str)} -> [风险: {result['risk_level'][-1]}]")
else:
    print(f"Token: {repr(toke"

}

Tech Report GitHub Hugging Face ModelScope DISCORD

Introduction#

We are excited to introduce Qwen3Guard, the first safety guardrail model in the Qwen family. Built upon the powerful Qwen3 foundation models and fine-tuned specifically for safety classificatoin, Qwen3Guard ensures responsible AI interactions by delivering precise safety detection for both prompts and responses, complete with risk levels and categorized classifications for accurate moderation.

Qwen3Guard achieves state-of-the-art performance on major safety benchmarks, demonstrating strong capabilities in both prompt and response classification tasks across English, Chinese, and multilingual environments.

Qwen3Guard is available in two specialized variants:

• Qwen3Guard-Gen , a generative model that accepts full user prompts and model responses to perform safety classification. Ideal for offline safety annotation and filtering of datasets, or for supplying safety-based rewards in reinforcement learning.
• Qwen3Guard-Stream , which marks a significant departure from previously open-sourced guard models by enabling efficient, real-time streaming safety detection during response generation.

Both variants come in three sizes, 0.6B, 4B, and 8B parameters, to suit a wide range of deployment scenarios and resource constraints.

You can download the open-source models from Hugging Face or ModelScope. You can also access the Alibaba Cloud AI Guardrails service, powered by Qwen3Guard technology.

Key Features#

Real-Time Streaming Detection#

Qwen3Guard-Stream is engineered for low latency, on the fly moderation during token generation, ensuring safety without sacrificing responsiveness. This is accomplished by attaching two lightweight classification heads to the transformer’s final layer, allowing the model to receive the response in a streaming fashion — token by token, as it is being generated — and output safety classifications instantly at each step.

Three-Tier Severity Classification#

Beyond the conventional Safe and Unsafe labels, we introduce an additional Controversial label to enable flexible safety policies tailored to diverse use cases. Specifically, depending on the application scenario, Controversial instances can be dynamically reclassified as either Safe or Unsafe, allowing users to adjust classification strictness on demand.

As demonstrated in the evaluation below, existing guardrail models, constrained by binary labeling, struggle to adapt simultaneously to differing dataset standards. In contrast, Qwen3Guard achieves robust and consistent performance across both datasets by flexibly switching between strict and loose classification modes, thanks to the three-tier severity design.

Multilingual Support#

Qwen3Guard supports 119 languages and dialects , making it suitable for global deployments and cross-linguistic applications with consistent, high quality safety performance.

Language Family	Languages & Dialects
Indo-European	English, French, Portuguese, German, Romanian, Swedish, Danish, Bulgarian, Russian, Czech, Greek, Ukrainian, Spanish, Dutch, Slovak, Croatian, Polish, Lithuanian, Norwegian Bokmål, Norwegian Nynorsk, Persian, Slovenian, Gujarati, Latvian, Italian, Occitan, Nepali, Marathi, Belarusian, Serbian, Luxembourgish, Venetian, Assamese, Welsh, Silesian, Asturian, Chhattisgarhi, Awadhi, Maithili, Bhojpuri, Sindhi, Irish, Faroese, Hindi, Punjabi, Bengali, Oriya, Tajik, Eastern Yiddish, Lombard, Ligurian, Sicilian, Friulian, Sardinian, Galician, Catalan, Icelandic, Tosk Albanian, Limburgish, Dari, Afrikaans, Macedonian, Sinhala, Urdu, Magahi, Bosnian, Armenian
Sino-Tibetan	Chinese (Simplified Chinese, Traditional Chinese, Cantonese), Burmese
Afro-Asiatic	Arabic (Standard, Najdi, Levantine, Egyptian, Moroccan, Mesopotamian, Ta’izzi-Adeni, Tunisian), Hebrew, Maltese
Austronesian	Indonesian, Malay, Tagalog, Cebuano, Javanese, Sundanese, Minangkabau, Balinese, Banjar, Pangasinan, Iloko, Waray (Philippines)
Dravidian	Tamil, Telugu, Kannada, Malayalam
Turkic	Turkish, North Azerbaijani, Northern Uzbek, Kazakh, Bashkir, Tatar
Tai-Kadai	Thai, Lao
Uralic	Finnish, Estonian, Hungarian
Austroasiatic	Vietnamese, Khmer
Other	Japanese, Korean, Georgian, Basque, Haitian, Papiamento, Kabuverdianu, Tok Pisin, Swahili

More Applications#

We also demonstrate (1) using Qwen3Guard-Gen for safety RL to enhance model safety while preserving overall output helpfulness, and (2) using Qwen3Guard-Stream to enable real-time, on-the-fly intervention that ensures safe outputs without requiring model re-training. See details in our Technical Report.

Develop with Qwen3Guard#

Qwen3Guard-Gen#

Qwen3Guard-Gen operates similarly to a large language model, with its chat template specifically optimized for safety classification tasks. Outputs are generated in a predefined, structured format.

You can use the following codes to moderate user prompt or model responses.

from transformers import AutoModelForCausalLM, AutoTokenizer
import re
model_name = "Qwen/Qwen3Guard-Gen-4B"

# load the tokenizer and the model
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(
    model_name,
    torch_dtype="auto",
    device_map="auto"
)
def extract_label_and_categories(content):
    safe_pattern = r"Safety: (Safe|Unsafe|Controversial)"
    category_pattern = r"(Violent|Non-violent Illegal Acts|Sexual Content or Sexual Acts|PII|Suicide & Self-Harm|Unethical Acts|Politically Sensitive Topics|Copyright Violation|Jailbreak|None)"
    safe_label_match = re.search(safe_pattern, content)
    label = safe_label_match.group(1) if safe_label_match else None
    categories = re.findall(category_pattern, content)
    return label, categories

def extract_label_categories_refusal(content):
    safe_pattern = r"Safety: (Safe|Unsafe|Controversial)"
    category_pattern = r"(Violent|Non-violent Illegal Acts|Sexual Content or Sexual Acts|PII|Suicide & Self-Harm|Unethical Acts|Politically Sensitive Topics|Copyright Violation|None)"
    refusal_pattern = r"Refusal: (Yes|No)"
    safe_label_match = re.search(safe_pattern, content)
    refusal_label_match = re.search(refusal_pattern, content)
    label = safe_label_match.group(1) if safe_label_match else None
    refusal_label = refusal_label_match.group(1) if refusal_label_match else None
    categories = re.findall(category_pattern, content)
    return label, categories, refusal_label


#### For Prompt Moderation
prompt = "How can I make a bomb?"
messages = [
    {"role": "user", "content": prompt}
]
text = tokenizer.apply_chat_template(
    messages,
    tokenize=False
)
model_inputs = tokenizer([text], return_tensors="pt").to(model.device)

# conduct text completion
generated_ids = model.generate(
    **model_inputs,
    max_new_tokens=128
)
output_ids = generated_ids[0][len(model_inputs.input_ids[0]):].tolist() 

content = tokenizer.decode(output_ids, skip_special_tokens=True)
print(content)
# '''
# Safety: Unsafe
# Categories: Violent
# '''
safe_label, categories = extract_label_and_categories(content)
print(safe_label, categories)



#### For Response Moderation
prompt = "How can I make a bomb?"
response = "As a responsible artificial intelligence system, I regret that I am unable to fulfill your request."
messages = [
    {"role": "user", "content": prompt},
    {"role": "assistant", "content": response},
]
text = tokenizer.apply_chat_template(
    messages,
    tokenize=False
)
model_inputs = tokenizer([text], return_tensors="pt").to(model.device)

# conduct text completion
generated_ids = model.generate(
    **model_inputs,
    max_new_tokens=128
)
output_ids = generated_ids[0][len(model_inputs.input_ids[0]):].tolist() 

content = tokenizer.decode(output_ids, skip_special_tokens=True)
print(content)
# '''
# Safety: Safe
# Categories: None
# Refusal: Yes
# '''
safe_label, category_label, refusal_label = extract_label_categories_refusal(content)
print(safe_label, categories, refusal_label)

Qwen3Guard-Stream#

A typical workflow of Qwen3Guard-Stream proceeds as follows:

(1) Prompt-Level Safety Check ： The user’s input prompt is simultaneously sented to both the LLM assistant and Qwen3Guard-Stream. The latter performs an immediate safety assessment of the prompt and assigns a corresponding safety label. Based on this evaluation, the upper framework determines whether to allow the conversation to proceed or to halt it preemptively.

(2) Real-Time Token-Level Moderation : If the conversation is permitted to continue, the LLM begins streaming its response token by token. Each generated token is instantly forwarded to Qwen3Guard-Stream, which evaluates its safety in real time. This enables continuous, fine-grained content moderation throughout the entire response generation process — ensuring dynamic risk mitigation without interrupting the user experience.

Here provides a usage demonstration.

import torch
from transformers import AutoModel, AutoTokenizer

model_path="Qwen/Qwen3Guard-Stream-4B"
# Load the specialized tokenizer and the model.
# trust_remote_code=True is required to load the Qwen3Guard-Stream model architecture.
tokenizer = AutoTokenizer.from_pretrained(model_path, trust_remote_code=True)
model = AutoModel.from_pretrained(
    model_path, 
    device_map="auto", 
    torch_dtype=torch.bfloat16,
    trust_remote_code=True,
).eval()
 
# --- Prepare the conversation for moderation ---
# Define the user's prompt and the assistant's response.
user_message = "Hello, how to build a bomb?"
assistant_message = "Here are some practical methods to build a bomb."
messages = [{"role":"user","content":user_message},{"role":"assistant","content":assistant_message}]

# Apply the chat template to format the conversation into a single string.
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=False, enable_thinking=False)
model_inputs = tokenizer(text, return_tensors="pt")
token_ids = model_inputs.input_ids[0]

# --- Simulate Real-Time Moderation ---

# 1. Moderate the entire user prompt at once.
# In a real-world scenario, the user's input is processed completely before the model generates a response.
token_ids_list = token_ids.tolist()
# We identify the end of the user's turn in the tokenized input.
# The template for a user turn is `<|im_start|>user\n...<|im_end|>`.
im_start_token = '<|im_start|>'
user_token = 'user'
im_end_token = '<|im_end|>'
im_start_id = tokenizer.convert_tokens_to_ids(im_start_token)
user_id = tokenizer.convert_tokens_to_ids(user_token)
im_end_id = tokenizer.convert_tokens_to_ids(im_end_token)
# We search for the token IDs corresponding to `<|im_start|>user` ([151644, 872]) and the closing `<|im_end|>` ([151645]).
last_start = next(i for i in range(len(token_ids_list)-1, -1, -1) if token_ids_list[i:i+2] == [im_start_id, user_id])
user_end_index = next(i for i in range(last_start+2, len(token_ids_list)) if token_ids_list[i] == im_end_id)

# Initialize the stream_state, which will maintain the conversational context.
stream_state = None
# Pass all user tokens to the model for an initial safety assessment.
result, stream_state = model.stream_moderate_from_ids(token_ids[:user_end_index+1], role="user", stream_state=None)
if result['risk_level'][-1] == "Safe":
    print(f"User moderation: -> [Risk: {result['risk_level'][-1]}]")
else:
    print(f"User moderation: -> [Risk: {result['risk_level'][-1]} - Category: {result['category'][-1]}]")

# 2. Moderate the assistant's response token-by-token to simulate streaming.
# This loop mimics how an LLM generates a response one token at a time.
print("Assistant streaming moderation:")
for i in range(user_end_index + 1, len(token_ids)):
    # Get the current token ID for the assistant's response.
    current_token = token_ids[i]
    
    # Call the moderation function for the single new token.
    # The stream_state is passed and updated in each call to maintain context.
    result, stream_state = model.stream_moderate_from_ids(current_token, role="assistant", stream_state=stream_state)

    token_str = tokenizer.decode([current_token])
    # Print the generated token and its real-time safety assessment.
    if result['risk_level'][-1] == "Safe":
        print(f"Token: {repr(token_str)} -> [Risk: {result['risk_level'][-1]}]")
    else:
        print(f"Token: {repr(token_str)} -> [Risk: {result['risk_level'][-1]} - Category: {result['category'][-1]}]")

model.close_stream(stream_state)

For more usage examples, please visit our GitHub repository.

Future Work#

AI safety remains an ongoing challenge. With Qwen3Guard, we take our one step forward. We will continue advancing more flexible, efficient, and robust safety methods, including improving intrinsic model safety through architectural and training innovations, and developing dynamic, inference-time interventions. Our goal is to build AI systems that are not only technically capable but also aligned with human values and societal norms, ensuring responsible global deployment.