Viggle 发布 Qwen-Image-2.1-viggle-turbo v0.2.1:6 步生成,速度约提升 5 倍
英文标题:🤗 Viggle/Qwen-Image-2.1-viggle-turbo
Viggle 基于 Qwen/Qwen-Image-2.1 通过分布匹配蒸馏训练了 Qwen-Image-2.1-viggle-turbo,这款 7B 文生图与指令驱动图像编辑模型可在 6 次 Transformer 前向传递中完成生成,无需 classifier-free guidance。相较于 40 步基础模型,该模型端到端速度约快 5 倍,且在多数官方示例中质量接近;小而密集的文字渲染仍是基础模型更具优势的场景。v0.2.1 提供 rank 256 和 rank 128 两种 LoRA 适配器,并通过特定的 6 步 sigma 调度,在测试中将构图漂移降至 0%,样本多样性达到基础模型的 0.98 倍。开发者可通过固定版本的 diffusers、PEFT 和提供的 ComfyUI 工作流部署,但复杂多参考图编辑、换脸和身份保持编辑仍可能不及 40 步基础模型。
3.2 KB
411.3 KB
中文
Qwen-Image-2.1-viggle-turbo — v0.2.1
基于 Qwen 构建。这是 Qwen/Qwen-Image-2.1 的少步数蒸馏学生模型,由 Viggle 使用分布匹配蒸馏(Distribution Matching Distillation)训练。它同时支持文生图和使用 1–3 张参考图的指令驱动编辑,仅需 6 次 Transformer 前向传递,而不是 40 次,并且不使用 classifier-free guidance。
相较 40 步基础模型,端到端速度约快 5 倍,且质量与其极具竞争力:在官方 Qwen 示例中,两个模型在大多数提示词上的结果很难区分。最明显的差距在于小而密集的文字,基础模型在此方面仍然领先(使用 8 步可缩小差距)。可在演示 Space 的 Comparison 标签页自行查看:其中包含 Qwen/Qwen-Image-2.1 Space 的 32 个示例,在相同提示词、输入和随机种子下,以图像滑块对比 turbo 的 6 步结果(5 个密集文字示例使用 8 步)与基础模型的 40 步结果,每组各生成一个样本。复杂编辑的表现仍可能不及基础模型(已知限制)。
v0.2.1(2026-09-24)— 请使用此版本:
Qwen-Image-2.1-viggle-turbo-v0.2.1-6step-lora-r256.safetensors— LoRA 适配器(rank 256、alpha 256、bf16、1.3 GB),在运行时加载到基础 Transformer 之上。请使用随附 sigma 节点以 6 步进行采样:sigmas=[1.0, 0.9375, 0.875, 0.75, 0.5, 0.25](参见“使用方法”)。演示 Space 运行的正是该版本。Qwen-Image-2.1-viggle-turbo-v0.2.1-6step-lora-r128.safetensors— 将同一适配器裁剪至 rank 128(alpha 128、680 MB),由 ComfyUI 工作流使用。用法与 r256 相同;请参阅“Rank 128”了解其与 r256 的接近程度。
**ComfyUI:**自定义节点以及现成的文生图/编辑工作流位于 comfyui/;配置方法见 ComfyUI。坦率地说,我并不是 ComfyUI 的常规用户,这个移植版本大多是借助 AI 编程助手以“氛围编程”方式完成的。它已根据 diffusers pipeline 验证(数值见 ComfyUI 部分),并可端到端运行,但预计仍会存在一些粗糙之处。非常欢迎更熟悉 ComfyUI 的用户提交问题和修复方案。
v0.2.1 是同一次训练的 step-700 检查点;该训练的 step-600 检查点已于 2026-09-23 作为 v0.2 发布:增加了 100 个训练步数,配方和同一套 6 步调度均未改变。相较 v0.2,其清晰度略有提升(Laplacian 清晰度为 0.0199,对比 0.0187),多样性也略有提高(达到基础模型的 0.98×,对比 0.97×),同时保持 0% 的构图漂移。v0.2 文件(Qwen-Image-2.1-viggle-turbo-v0.2-5step-lora-r256.safetensors、r128 裁剪版、peft_v0.2/)仍原样保留在仓库中;请使用相同的 6 步节点采样(文件名中的 5step 指的是其首次发布时使用的调度,参见“重要规则”)。
相较 v0.1 的变化如下:基于我们保留的 96 条用户请求(文生图和编辑),与使用官方提示词增强的 40 步基础模型进行对比测量:
| v0.1 LoRA r64 | v0.1 全量微调 | v0.2.1 LoRA r256,6 步 | v0.2 LoRA r256,6 步 | (v0.2,5 步) | (v0.2,4 步) |
|---|---|---|---|---|---|
| 样本多样性,× 基础模型 ¹ | 0.75 | 0.72 | 0.98 | 0.97 | 0.93 |
| 相对基础模型的构图漂移 ² | −0.019 | −0.033 | +0.000 | −0.001 | +0.000 |
| 布局与基础模型不同的提示词占比 ³ | — | — | 0% | 0% | 4% |
¹ 在每条提示词的 8 个随机种子上计算平均的 DINOv2 patch 距离,涵盖 32 条提示词,并以 40 步基础模型的结果为比值(1.00 = 与基础模型同样多样)。v0.1 会使不同随机种子收缩至几乎相同的布局;v0.2 保留了基础模型的分布范围。² 对于相同提示词和随机种子,图像质心相对基础模型输出的平均水平漂移,以图像宽度为单位;0 表示学生模型将主体放置在与基础模型一致的位置。³ 96 条提示词中,学生模型在相同随机种子下与基础模型构图存在可见差异的比例(质心漂移超过 0.05 个图像宽度)。在实践中,v0.2 和 v0.2.1 对提示词和参考图的遵循明显优于 v0.1——被遗漏或合并的指令更少,重复主体的重影更少——并且达到基础模型的清晰度水平,而 v0.1 LoRA 更柔和。
v0.2 文件及 v0.1 产物(transformer/ 全量微调版和 r64 LoRA,均为 4 步)均原样保留在仓库中,以确保可复现性;没有理由优先使用它们。
**已知限制。**复杂编辑(多参考图构图、换脸、身份保持编辑、包含多项约束的指令)仍可能不及 40 步基础模型。随着蒸馏效果提升,我们将继续更新该仓库。
包含内容
Qwen-Image-2.1-viggle-turbo-v0.2.1-6step-lora-r256.safetensors # v0.2.1 LoRA (rank 256, alpha 256), diffusers key format, bf16 <- use this
peft_v0.2.1/ # the same v0.2.1 adapter in peft key format, F32 as trained
Qwen-Image-2.1-viggle-turbo-v0.2.1-6step-lora-r128.safetensors # v0.2.1 LoRA truncated to rank 128 (alpha 128), diffusers key format, bf16
comfyui/ # ComfyUI custom nodes (viggle_turbo.py), text-to-image and edit workflows, example inputs
scheduler/scheduler_config.json # base scheduler config with shift_terminal: null
Qwen-Image-2.1-viggle-turbo-v0.2-5step-lora-r256.safetensors # v0.2 LoRA (step 600 of the same run), kept for reference
Qwen-Image-2.1-viggle-turbo-v0.2-5step-lora-r128.safetensors # v0.2 LoRA truncated to rank 128, kept for reference
peft_v0.2/ # v0.2 adapter in peft key format, F32
Qwen-Image-2.1-viggle-turbo-4step-lora-r64.safetensors # v0.1 LoRA (rank 64), 4-step, kept for reference
peft/ # v0.1 adapter in peft key format, F32
transformer/ # v0.1 full fine-tuned transformer (bf16), 4-step, kept for reference
LICENSE, NOTICE, README.md
随附检查点:v0.2.1 是一次 LoRA 训练的 step-700 EMA 学生模型(v0.2 为 step 600)。该 LoRA 在注意力、image-MLP、调制和 timestep-embedder 投影上采用 rank 256,并在 DMD 目标中加入了 SenseFlow 风格的段内引导(学生模型在每个分段上的速度回归至教师模型的速度)以及经提示词增强的教师目标;它使用 4 步调度训练,并在采样时将第一个(最高噪声)分段切为三段(6 步),从而消除了普通 4 步 rollout 的细节损失、重影和构图漂移(5 步,即将该分段切为两段,是首次发布时的调度;在权重相同的前提下,6 步在我们跟踪的每项指标上都严格更优)。LoRA 从不合并进 Transformer——合并至 bf16 会产生损失,运行时加载则是精确的。文本编码器、VAE 和 processor 不会重新分发,而是从基础仓库加载。
安装
pip install -U torch "transformers>=5.17,<6" accelerate safetensors peft pillow
pip install "git+https://github.com/huggingface/diffusers.git@80c7ed262aeffbeb43ef13ae04baeb9b84515a69"
QwenImage21Pipeline 尚未进入正式发布版的 diffusers,因此需要固定到该 Git 版本安装。必须安装 peft。
使用方法
加载 v0.2.1 LoRA
import torch
from diffusers import QwenImage21Pipeline, FlowMatchEulerDiscreteScheduler
pipe = QwenImage21Pipeline.from_pretrained("Qwen/Qwen-Image-2.1", dtype=torch.bfloat16)
pipe.load_lora_weights(
"Viggle/Qwen-Image-2.1-viggle-turbo",
weight_name="Qwen-Image-2.1-viggle-turbo-v0.2.1-6step-lora-r256.safetensors",
)
pipe.scheduler = FlowMatchEulerDiscreteScheduler.from_pretrained(
"Viggle/Qwen-Image-2.1-viggle-turbo", subfolder="scheduler"
)
pipe.to("cuda")
STEPS, SIGMAS = 6, [1.0, 0.9375, 0.875, 0.75, 0.5, 0.25]
v0.1 学生模型(4 步,仅供参考)
from diffusers import QwenImage21Transformer2DModel
transformer = QwenImage21Transformer2DModel.from_pretrained(
"Viggle/Qwen-Image-2.1-viggle-turbo", subfolder="transformer", torch_dtype=torch.bfloat16
)
pipe = QwenImage21Pipeline.from_pretrained("Qwen/Qwen-Image-2.1", transformer=transformer, dtype=torch.bfloat16)
pipe.load_lora_weights("Viggle/Qwen-Image-2.1-viggle-turbo", weight_name="Qwen-Image-2.1-viggle-turbo-4step-lora-r64.safetensors")
切勿叠加两个学生模型(即在微调后的 Transformer 上再加载 LoRA,或同时加载两个 LoRA)。
文生图
image = pipe(
prompt="A studio portrait of an old fisherman mending a net, warm rim light, 85mm.",
height=1024,
width=1024,
num_inference_steps=STEPS,
sigmas=SIGMAS,
true_cfg_scale=1.0,
generator=torch.Generator("cuda").manual_seed(0),
).images[0]
image.save("out.png")
图像编辑(1–3 张参考图)
from diffusers.utils import load_image
image = pipe(
prompt="Replace the background with a sunset beach, keep the subject unchanged.",
image=[load_image("input.png")],
output_resolution=1024,
num_inference_steps=STEPS,
sigmas=SIGMAS,
true_cfg_scale=1.0,
generator=torch.Generator("cuda").manual_seed(0),
).images[0]
重要规则
- **
num_inference_steps=6,配合sigmas=[1.0, 0.9375, 0.875, 0.75, 0.5, 0.25]、true_cfg_scale=1.0,且不使用负面提示词。**这些是_raw_ sigma 节点:pipeline 会像处理默认节点一样,精确地对它们应用依赖分辨率的时间偏移,因此请在所有分辨率下按原样传入。它们是在 4 步训练节点linspace(1, 1/4, 4)的基础上,将第一个(最高噪声)分段1 → 0.75切为三段得到的。未设置sigmas=时使用num_inference_steps=4即为训练调度,也能正常工作,但细节和多样性会略少(见上表)。 - **如果改变步数,只能在高噪声端增加或减少步骤。**构图在原始 sigma 1 到 0.875 之间决定(偏移后,在 1024² 分辨率下大致是噪声范围的前 5%),在这一段进行一次大的 Euler 步会造成重影和布局漂移;低噪声节点
0.75, 0.5, 0.25是学生模型训练时需要抵达的节点,移动它们(例如 6 步的[1, 0.958, 0.917, 0.875, 0.625, 0.375])会使每张图都明显变软(Laplacian 清晰度为 0.014,而非 0.020)。因此,请保留0.875, 0.75, 0.5, 0.25,并将1 → 0.875均匀拆分:5 步为[1, 0.875, 0.75, 0.5, 0.25](首次发布调度,4% 的提示词出现漂移);6 步如上(0%);7 步为[1, 0.9583, 0.9167, 0.875, 0.75, 0.5, 0.25](同样为 0%,略微更清晰,但视觉上未优于 6 步)。采用其他步数的均匀linspace调度,以及 CFG,均无帮助。 - 使用随附的 scheduler 配置(或使用
FlowMatchEulerDiscreteScheduler.from_config(pipe.scheduler.config, shift_terminal=None))。基础配置的shift_terminal: 0.02会破坏最后一步。 - 将 LoRA scale 保持为 1.0(alpha 等于 rank)。
- 参考图的顺序决定提示词中
image 1/image 2所指代的图像。若不设置height/width,输出宽高比将跟随最后一张参考图(在 ComfyUI 中则跟随第一张)。训练中最多使用 3 张参考图。 - 建议使用官方 PE-T2I / PE-I2I 改写器进行提示词改写:学生模型是针对经提示词增强的教师目标蒸馏而成,改写器有助于构图和文字渲染。上述数据使用了这些改写器。原始提示词同样可用。
peft用户可直接加载peft_v0.2.1/:pipe.transformer.load_lora_adapter("Viggle/Qwen-Image-2.1-viggle-turbo", subfolder="peft_v0.2.1", weight_name="adapter_model.safetensors", prefix=None)—— 权重相同,只是键名不同;请选择其中一种路径,不要同时使用两种。
分辨率
训练尺寸遵循基础模型的 calculate_dimensions(area, aspect_ratio),并向上或向下取整至 32 的倍数。文生图训练使用 1024² 和 2048² 面积,编辑训练使用 1024² 和 1536² 面积(参考图始终按 1024² 面积编码)。其他尺寸也可用;这些是训练数据最集中的尺寸。
| 比例 | 1024² 面积(T2I + 编辑) | 1536² 面积(编辑) | 2048² 面积(T2I) |
|---|---|---|---|
| 1:1 | 1024 × 1024 | 1536 × 1536 | 2048 × 2048 |
| 4:3 | 1184 × 896 | 1760 × 1344 | 2368 × 1760 |
| 3:4 | 896 × 1184 | 1344 × 1760 | 1760 × 2368 |
| 16:9 | 1376 × 768 | 2048 × 1152 | 2720 × 1536 |
| 9:16 | 768 × 1376 | 1152 × 2048 | 1536 × 2720 |
| 3:2 | 1248 × 832 | 1888 × 1248 | 2496 × 1664 |
| 2:3 | 832 × 1248 | 1248 × 1888 | 1664 × 2496 |
Rank 128
...-lora-r128.safetensors 是通过对 r256 适配器更新量 ΔW = B A 进行逐层精确 SVD 截断得到的 r128 版本,也是最佳的 rank-128 近似(Eckart–Young 定理)。每层保留更新量能量的 83–100%(中位数为 95%);被舍弃的部分低于基础权重的 0.05%。在 96 条保留请求上,使用相同噪声和提示词时,其图像与 r256 适配器输出的 LPIPS 差异为 0.0244。对 r256 适配器进行完整 rank 的重新分解——除浮点舍入外不会改变任何内容——其图像已会产生 0.0239 的变化,因此 r128 处于这一噪声下限附近(rank 64:0.0267)。LPIPS 数值在 v0.2 配对版本上测得;v0.2.1 的截断具有相同谱特征(每层保留 83–100% 能量,中位数为 95%)。
ComfyUI
ComfyUI 移植版本大多采用“氛围编程”方式完成:我并不日常使用 ComfyUI,节点和工作流主要由 AI 编程助手协助编写。已验证的内容包括:sigma 调度与 diffusers 精确匹配到浮点精度,以及相对 diffusers 的图像差距处于
English / 原文
Qwen-Image-2.1-viggle-turbo — v0.2.1
Built with Qwen. A few-step distilled student of Qwen/Qwen-Image-2.1, trained by Viggle with Distribution Matching Distillation. It does both text-to-image and instruction-driven editing with 1–3 reference images in 6 transformer passes instead of 40 , with no classifier-free guidance.
About 5× faster than the 40-step base model end to end, and very competitive with it in quality : on the official Qwen examples the two are hard to tell apart on most prompts. The clearest gap is small, dense text, where the base model is still ahead (8 steps narrows it). See for yourself in the Comparison tab of the demo Space: 32 examples of the Qwen/Qwen-Image-2.1 Space, turbo in 6 steps (and 8 on the 5 dense-text examples) vs base in 40 steps, same prompt, inputs and seed, one sample each, in an image slider. Complicated edits can still fall short of the base model (Known limitations).
v0.2.1 (2026-09-24) — use this one:
Qwen-Image-2.1-viggle-turbo-v0.2.1-6step-lora-r256.safetensors— LoRA adapter (rank 256, alpha 256, bf16, 1.3 GB) loaded on top of the base transformer at runtime. Sample it with 6 steps on the shipped sigma nodessigmas=[1.0, 0.9375, 0.875, 0.75, 0.5, 0.25](see Usage). This is what the demo Space runs.Qwen-Image-2.1-viggle-turbo-v0.2.1-6step-lora-r128.safetensors— the same adapter cut to rank 128 (alpha 128, 680 MB), used by the ComfyUI workflows. Same usage as r256; see rank 128 for how close it is.
ComfyUI: custom nodes and ready-made text-to-image / edit workflows are in comfyui/; setup is in ComfyUI. Honest caveat: I am not a regular ComfyUI user, and the port was mostly vibe-coded with an AI coding assistant. It was checked against the diffusers pipeline (numbers in the ComfyUI section) and runs end to end, but expect rough edges. Issues and fixes from people who know ComfyUI better are very welcome.
v0.2.1 is the step-700 checkpoint of the run whose step-600 checkpoint shipped as v0.2 on 2026-09-23: 100 more training steps, same recipe, same 6-step schedule. Against v0.2 it is a little sharper (Laplacian sharpness 0.0199 vs 0.0187) and marginally more diverse (0.98× vs 0.97× the base model), with the same 0% composition drift. The v0.2 files (Qwen-Image-2.1-viggle-turbo-v0.2-5step-lora-r256.safetensors, the r128 cut, peft_v0.2/) stay in the repository unchanged; sample them with the same 6-step nodes (the 5step in their name is the schedule they launched with, see Rules that matter).
What changed against v0.1, measured on our held-out set of 96 user requests (text-to-image and editing) against the 40-step base model with its official prompt enhancement:
| v0.1 LoRA r64 | v0.1 full fine-tune | v0.2.1 LoRA r256, 6 steps | v0.2 LoRA r256, 6 steps | (v0.2 at 5 steps) | (v0.2 at 4 steps)
---|---|---|---|---|---|---
sample diversity, × base model ¹ | 0.75 | 0.72 | 0.98 | 0.97 | 0.93 | 0.89
composition drift vs base model ² | −0.019 | −0.033 | +0.000 | −0.001 | +0.000 | +0.002
prompts whose layout differs from the base model ³ | — | — | 0% | 0% | 4% | —
¹ mean intra-prompt DINOv2 patch distance over 8 seeds per prompt, 32 prompts, as a ratio to the 40-step base model (1.00 = as diverse as the base). v0.1 collapsed seeds onto near-identical layouts; v0.2 keeps the base model's spread. ² mean horizontal drift of the image centroid relative to the base model's output for the same prompt and seed, in image widths; 0 means the student places its subjects where the base model does. ³ share of the 96 prompts where the student's composition visibly differs from the base model's for the same seed (centroid drift above 0.05 image widths). In practice v0.2 and v0.2.1 follow the prompt and the reference images noticeably more faithfully than v0.1 — fewer dropped or merged instructions, less ghosting of duplicated subjects — and match the base model's sharpness where the v0.1 LoRA was softer.
The v0.2 files and the v0.1 artifacts (transformer/ full fine-tune and the r64 LoRA, both 4-step) stay in the repository unchanged for reproducibility; there is no reason to prefer them.
Known limits. Complicated edits (multi-reference composition, face swaps, identity-preserving edits, instructions with several constraints) can still fall short of the 40-step base model. We will keep updating this repository as the distillation improves.
What's inside
Qwen-Image-2.1-viggle-turbo-v0.2.1-6step-lora-r256.safetensors # v0.2.1 LoRA (rank 256, alpha 256), diffusers key format, bf16 <- use this
peft_v0.2.1/ # the same v0.2.1 adapter in peft key format, F32 as trained
Qwen-Image-2.1-viggle-turbo-v0.2.1-6step-lora-r128.safetensors # v0.2.1 LoRA truncated to rank 128 (alpha 128), diffusers key format, bf16
comfyui/ # ComfyUI custom nodes (viggle_turbo.py), text-to-image and edit workflows, example inputs
scheduler/scheduler_config.json # base scheduler config with shift_terminal: null
Qwen-Image-2.1-viggle-turbo-v0.2-5step-lora-r256.safetensors # v0.2 LoRA (step 600 of the same run), kept for reference
Qwen-Image-2.1-viggle-turbo-v0.2-5step-lora-r128.safetensors # v0.2 LoRA truncated to rank 128, kept for reference
peft_v0.2/ # v0.2 adapter in peft key format, F32
Qwen-Image-2.1-viggle-turbo-4step-lora-r64.safetensors # v0.1 LoRA (rank 64), 4-step, kept for reference
peft/ # v0.1 adapter in peft key format, F32
transformer/ # v0.1 full fine-tuned transformer (bf16), 4-step, kept for reference
LICENSE, NOTICE, README.md
Shipped checkpoint: v0.2.1 is the step-700 EMA student (v0.2 was step 600) of a LoRA run (rank 256 on the attention, image-MLP, modulation and timestep-embedder projections) that adds SenseFlow-style intra-segment guidance (the student's velocity over each of its segments is regressed on the teacher's) and prompt-enhanced teacher targets to the DMD objective; it was trained on the 4-step schedule and is sampled with the first (highest-noise) segment cut into three (6 steps), which removes the detail loss, ghosting and composition drift of the plain 4-step rollout (5 steps, the segment cut in two, was the launch schedule; 6 is strictly better on every metric we track, with the same weights). The LoRA is never merged into the transformer — merging into bf16 is lossy, loading it at runtime is exact. Text encoder, VAE and processor are not redistributed; they load from the base repo.
Install
pip install -U torch "transformers>=5.17,<6" accelerate safetensors peft pillow
pip install "git+https://github.com/huggingface/diffusers.git@80c7ed262aeffbeb43ef13ae04baeb9b84515a69"
QwenImage21Pipeline is not in a released diffusers yet, hence the pinned git install. peft is required.
Usage
Load the v0.2.1 LoRA
import torch
from diffusers import QwenImage21Pipeline, FlowMatchEulerDiscreteScheduler
pipe = QwenImage21Pipeline.from_pretrained("Qwen/Qwen-Image-2.1", dtype=torch.bfloat16)
pipe.load_lora_weights(
"Viggle/Qwen-Image-2.1-viggle-turbo",
weight_name="Qwen-Image-2.1-viggle-turbo-v0.2.1-6step-lora-r256.safetensors",
)
pipe.scheduler = FlowMatchEulerDiscreteScheduler.from_pretrained(
"Viggle/Qwen-Image-2.1-viggle-turbo", subfolder="scheduler"
)
pipe.to("cuda")
STEPS, SIGMAS = 6, [1.0, 0.9375, 0.875, 0.75, 0.5, 0.25]
v0.1 students (4-step, for reference)
from diffusers import QwenImage21Transformer2DModel
transformer = QwenImage21Transformer2DModel.from_pretrained(
"Viggle/Qwen-Image-2.1-viggle-turbo", subfolder="transformer", torch_dtype=torch.bfloat16
)
pipe = QwenImage21Pipeline.from_pretrained("Qwen/Qwen-Image-2.1", transformer=transformer, dtype=torch.bfloat16)
pipe.load_lora_weights("Viggle/Qwen-Image-2.1-viggle-turbo", weight_name="Qwen-Image-2.1-viggle-turbo-4step-lora-r64.safetensors")
Never stack two students (a LoRA on top of the fine-tuned transformer, or both LoRAs).
Text to image
image = pipe(
prompt="A studio portrait of an old fisherman mending a net, warm rim light, 85mm.",
height=1024,
width=1024,
num_inference_steps=STEPS,
sigmas=SIGMAS,
true_cfg_scale=1.0,
generator=torch.Generator("cuda").manual_seed(0),
).images[0]
image.save("out.png")
Image editing (1–3 reference images)
from diffusers.utils import load_image
image = pipe(
prompt="Replace the background with a sunset beach, keep the subject unchanged.",
image=[load_image("input.png")],
output_resolution=1024,
num_inference_steps=STEPS,
sigmas=SIGMAS,
true_cfg_scale=1.0,
generator=torch.Generator("cuda").manual_seed(0),
).images[0]
Rules that matter
num_inference_steps=6withsigmas=[1.0, 0.9375, 0.875, 0.75, 0.5, 0.25],true_cfg_scale=1.0, no negative prompt. These are raw sigma nodes: the pipeline applies its resolution-dependent time shift to them exactly as it does to its default nodes, so pass them as written for every resolution. They are the 4-step training nodeslinspace(1, 1/4, 4)with the first (highest-noise) segment1 → 0.75cut into three.num_inference_steps=4withoutsigmas=is the training schedule and also works, with slightly less detail and diversity (table above).- If you change the step count, add or remove steps at the high-noise end only. The composition is decided between raw sigma 1 and 0.875 (after the shift, roughly the first 5% of the noise range at 1024²), and one big Euler step there ghosts and drifts the layout; the low-noise nodes
0.75, 0.5, 0.25are the ones the student was trained to land on, and moving them (a 6-step[1, 0.958, 0.917, 0.875, 0.625, 0.375]) makes every image visibly softer (Laplacian sharpness 0.014 against 0.020). So keep0.875, 0.75, 0.5, 0.25and split1 → 0.875evenly: 5 steps[1, 0.875, 0.75, 0.5, 0.25](the launch schedule, 4% of prompts drift), 6 as above (0%), 7[1, 0.9583, 0.9167, 0.875, 0.75, 0.5, 0.25](also 0%, marginally sharper, not visibly better than 6). Uniformlinspaceschedules with other step counts, and CFG, do not help. - Use the shipped scheduler config (or
FlowMatchEulerDiscreteScheduler.from_config(pipe.scheduler.config, shift_terminal=None)). The base config'sshift_terminal: 0.02would wreck the last step. - Leave the LoRA scale at 1.0 (alpha equals rank).
- Reference-image order determines which image
image 1/image 2in the prompt refers to. Withoutheight/width, the output aspect ratio follows the last reference (in ComfyUI: the first). Up to 3 references were used in training. - Prompt rewriting with the official PE-T2I / PE-I2I rewriters is recommended: the student was distilled against prompt-enhanced teacher targets, and the rewriters help composition and rendered text. The numbers above use them. Raw prompts work too.
peftusers can loadpeft_v0.2.1/directly:pipe.transformer.load_lora_adapter("Viggle/Qwen-Image-2.1-viggle-turbo", subfolder="peft_v0.2.1", weight_name="adapter_model.safetensors", prefix=None)— same weights, different key names; pick one path, not both.
Resolutions
Training sizes follow the base model's calculate_dimensions(area, aspect_ratio) rounded to a multiple of 32. Text-to-image was trained at 1024² and 2048² area, editing at 1024² and 1536² area (references always encoded at 1024² area). Other sizes work; these are where the training mass sits.
| ratio | 1024² area (T2I + editing) | 1536² area (editing) | 2048² area (T2I) |
|---|---|---|---|
| 1:1 | 1024 × 1024 | 1536 × 1536 | 2048 × 2048 |
| 4:3 | 1184 × 896 | 1760 × 1344 | 2368 × 1760 |
| 3:4 | 896 × 1184 | 1344 × 1760 | 1760 × 2368 |
| 16:9 | 1376 × 768 | 2048 × 1152 | 2720 × 1536 |
| 9:16 | 768 × 1376 | 1152 × 2048 | 1536 × 2720 |
| 3:2 | 1248 × 832 | 1888 × 1248 | 2496 × 1664 |
| 2:3 | 832 × 1248 | 1248 × 1888 | 1664 × 2496 |
Rank 128
...-lora-r128.safetensors is the r256 adapter truncated by an exact per-layer SVD of its update ΔW = B A, which is the best rank-128 approximation (Eckart–Young). Per layer it keeps 83–100% of the update's energy (median 95%); what it drops is below 0.05% of the base weight. On the 96 held-out requests, with the same noise and prompts, its images differ from the r256 adapter's by LPIPS 0.0244. Re-factoring the r256 adapter at full rank, which changes nothing but float round-off, already moves them by 0.0239, so r128 sits at that noise floor (rank 64: 0.0267). The LPIPS numbers were measured on the v0.2 pair; the v0.2.1 truncation has the same spectrum (83–100% of the energy kept per layer, median 95%).
ComfyUI
The ComfyUI port is mostly vibe-coded: I don't use ComfyUI day to day, and the nodes and workflows were written largely with an AI coding assistant. What was verified: the sigma schedule matches diffusers to float precision, and the image gap to diffusers is in the table below. Workflow ergonomics and less common setups were not tested much; please open a discussion if something looks wrong.
Tested with ComfyUI 0.37.0 (frontend 1.53.6), which has native Qwen-Image-2.1 support. In comfyui/:
viggle_turbo.py— two custom nodes. Copy it intoComfyUI/custom_nodes/and restart ComfyUI.Qwen-Image-2.1-viggle-turbo-t2i.json,Qwen-Image-2.1-viggle-turbo-edit.json— the workflows (drag into ComfyUI). Each has a note with the settings and an optional prompt-enhancer group.input/woman2.webp,input/cat.webp— the edit workflow's example references; copy them intoComfyUI/input/. Like the reference photos in the demo Space, they come from the black-forest-labs/flux-klein-9b-kv Space.
Model files (the workflows carry their download links):
The base files are Comfy-Org's, the ones ComfyUI's own Qwen-Image-2.1 template uses. The workflows default to the int8 transformer and text encoder, the r128 LoRA and the prompt enhancer on; with every model kept loaded this peaks at 26 GB of VRAM on a 1248 × 832 image. The bf16 files are a drop-in swap.
The two nodes:
-
Viggle Turbo Sigmas — the 6-step schedule above with the pipeline's resolution-dependent shift (the
nodesfield takes any raw schedule; see Rules that matter before changing it). Use it instead of a KSampler scheduler; with euler andBasicGuider(no CFG, no negative prompt). -
Viggle Turbo LoRA (unmerged) — applies the LoRA at runtime as
W x + B A x, as diffusers does. ComfyUI's stock LoRA loaders merge it into the weights instead, and for this adapter the merge is lossy: round-to-nearest into bf16 keeps about 70% of the update on average (about 40% in some early MLP layers), and requantizing the int8 weights keeps it on average but adds noise about 4× its size. LPIPS distance from diffusers with the r256 LoRA, on 5 requests with the same noise:
| ComfyUI transformer | stock LoRA loader (merged) | Viggle Turbo LoRA (unmerged) | no LoRA on either side |
|---|---|---|---|
| bf16 | 0.093 | 0.052 | 0.033 |
| int8 | 0.086 | 0.038 | 0.045 |
The last column is the gap ComfyUI and diffusers have without any LoRA. The unmerged node costs about 10–25% more time per step. Keep its strength at 1.0.
Differences from diffusers: in the edit workflow the output size follows image 1 , at about 1 MP (diffusers follows the last reference). The prompt enhancer uses the Qwen3-VL text encoder with the official rewriter system prompts; the edit rewriter sees image 1 and image 2.
Troubleshooting: with comfy_kitchen 0.2.35 on an NVIDIA driver older than 580, TextGenerate (the prompt enhancer) fails with a CUDA driver error from comfy_kitchen's CUDA 13 kernels. Update the driver, or turn Enhance prompt off; image generation itself does not use that code path.
Known limitations
v0.2.1 is very competitive with the base model on text-to-image and on most edits, but still falls short of it on complicated editing. Specifically:
- Complicated edits degrade : multi-reference composition, face swaps and identity-document edits can still produce duplicated or ghosted figures; "keep everything the same" requests can drift in identity. Much rarer than in v0.1.
- Small or long rendered text can garble more often than with the 40-step base model.
- 2K output is compared with the base model only by eye (the text-to-image examples of the Comparison tab are ~4 MP); none of the numbers above are at 2K.
- RGBA output, mask-guided editing and edits with 5, 6 and 10 reference images work on the Qwen examples in the Comparison tab but are not measured.
- On the dense-text examples, 8 steps (
sigmas=[1, 0.9375, 0.875, 0.75, 0.625, 0.5, 0.25, 0.125]) prints small text noticeably cleaner than 6; the Comparison tab shows both. - The two metrics above are our own, computed on 96 / 32 held-out user requests; no standard benchmark is claimed.
License
This model is a derivative work of Qwen-Image-2.1 and is distributed under the Qwen RESEARCH LICENSE AGREEMENT (LICENSE): non-commercial use only — research or evaluation purposes. Commercial use requires a separate licence from the licensor (model-business@notice.qwencloud.com). See NOTICE for the required attribution.
Qwen is licensed under the Qwen RESEARCH LICENSE AGREEMENT, Copyright (c) 2026 Hangzhou Tongyi Laboratory Technology Co., Ltd. All Rights Reserved.
Relative to Qwen/Qwen-Image-2.1 this repository adds LoRA adapters (v0.2.1 and v0.2 at rank 256 and 128, v0.1), ComfyUI nodes, workflows and two example input photos, and a scheduler config with shift_terminal changed from 0.02 to null, and ships a modified transformer (transformer/, the v0.1 full fine-tune); text encoder, VAE and processor are not redistributed.
Distillation method: DMD2 (Yin et al., 2024) / SenseFlow-style distribution matching. Distillation and release by Viggle. Built with Qwen.