Skip to main content
Back

Viggle Shrinks Qwen-Image-2.1 Generation From 40 Steps to Five

Viggle's v0.2 LoRA compresses Qwen-Image-2.1 from 40 diffusion steps to 5, keeping sample diversity at 93% of the teacher.

Model, Full Access
  • Viggle released v0.2 of its turbo LoRA for Qwen-Image-2.1, cutting sampling from 40 to 5 steps.
  • Rank-256 adapter, 1.3 GB bf16, loaded at runtime with no CFG and no negative prompt.
  • Diversity hits 93% of the base model, composition drift measured at zero on 96 held-out prompts.
  • Handles both text-to-image and instruction editing with up to 3 reference images.
  • Distilled via DMD2 with SenseFlow-style intra-segment guidance and prompt-enhanced teacher targets.
  • Preview status; struggles with multi-reference edits, face swaps, and small rendered text. Qwen Research license, non-commercial.

Viggle has released v0.2 of its turbo LoRA for Qwen-Image-2.1, reducing the base model’s 40-step sampling process to five transformer passes. Viggle reports 93% of the base model’s sample diversity and no average composition drift on its internal benchmark.

A LoRA stores compact weight updates that modify selected model layers at runtime. Cutting 40 transformer evaluations to five can substantially reduce generation time and compute, although text encoding, reference-image processing, and VAE decoding prevent a full eightfold reduction in end-to-end latency. Viggle has not published hardware-specific latency or memory benchmarks.

Five passes, one adapter

The release consists of one 1.3 GB adapter file, Qwen-Image-2.1-viggle-turbo-v0.2-5step-lora-r256.safetensors, loaded over the base transformer through the adapter repository.

Viggle v0.2 release configuration
LoRA rank and alpha 256 and 256
Precision BF16
File size 1.3 GB
Inference steps 5
Sigma schedule [1.0, 0.875, 0.75, 0.5, 0.25]
Classifier-free guidance Disabled with true_cfg_scale=1.0
Supported tasks Text-to-image generation and instruction-guided editing
Reference images One to three during editing
License Qwen Research License

The repository lists the Qwen Research License, which permits noncommercial use. Commercial deployment requires a separate license from the Qwen team.

A 96-prompt benchmark

Viggle compared v0.2 with the v0.1 rank-64 LoRA and v0.1 full fine-tune on a held-out set of 96 user prompts. The reference was the 40-step base model using Qwen’s official prompt enhancement.

Viggle-reported results against the 40-step base model
Metric v0.1 LoRA r64 v0.1 full fine-tune v0.2 LoRA r256
Sample diversity ratio, base = 1.00 0.75 0.72 0.93
Composition drift, closer to 0 is better -0.019 -0.033 +0.000

Viggle calculated diversity from the mean DINOv2 patch distance among eight seeds per prompt across a 32-prompt subset. A score of 0.93 means the generated samples retained 93% of the spatial and visual variation measured in the 40-step reference, compared with 75% for the earlier LoRA.

The composition score measures changes in subject placement relative to the base model. The reported +0.000 is an aggregate value rounded to three decimal places, so it does not imply identical composition for every image.

These figures come from a small, vendor-run evaluation. Independent tests across broader prompt classes, resolutions, editing tasks, and hardware would provide a stronger account of quality and speed.

Teacher signals, shorter paths

Viggle trained the adapter with Distribution Matching Distillation plus a SenseFlow-style modification. The 40-step base model serves as the teacher, while the LoRA-equipped model learns a shorter denoising path as the student.

The objective aligns the student’s velocity prediction with the teacher’s prediction across each noise segment. In diffusion models, velocity describes the direction used to move a noisy latent toward the final image. Training targets came from prompts processed by Qwen’s prompt enhancer.

The released checkpoint captures an exponential moving average of the student weights at training step 600. The LoRA updates attention, image-MLP, modulation, and timestep-embedder projections while leaving the base checkpoint unchanged.

Training used four denoising segments. At inference time, Viggle splits the first, highest-noise segment into two passes, producing the five-value sigma schedule. The company reports that this extra early pass reduces detail loss and ghosting found in the four-pass rollout.

Keeping the adapter separate from the transformer also avoids merge-time rounding. Folding the LoRA updates into BF16 base weights can discard small values, while runtime loading preserves the adapter’s original updates before computation at the selected precision.

Reproduce the five-pass path

QwenImage21Pipeline requires a Git build of Diffusers because the class has not reached a tagged package release. The current source build can be installed with:

python -m pip install "git+https://github.com/huggingface/diffusers.git"
python -m pip install transformers accelerate safetensors

Reproducible deployments should replace the moving Git branch with a tested commit hash and record that hash alongside the model, adapter, and scheduler revisions.

The following text-to-image example loads the unmerged LoRA and its matching scheduler:

import torch
from diffusers import (
    FlowMatchEulerDiscreteScheduler,
    QwenImage21Pipeline,
)

base_id = "Qwen/Qwen-Image-2.1"
turbo_id = "Viggle/Qwen-Image-2.1-viggle-turbo"
weight_name = (
    "Qwen-Image-2.1-viggle-turbo-v0.2-5step-lora-r256.safetensors"
)

pipe = QwenImage21Pipeline.from_pretrained(
    base_id,
    torch_dtype=torch.bfloat16,
).to("cuda")

pipe.load_lora_weights(
    turbo_id,
    weight_name=weight_name,
)

pipe.scheduler = FlowMatchEulerDiscreteScheduler.from_pretrained(
    turbo_id,
    subfolder="scheduler",
)

result = pipe(
    prompt="A studio photograph of a red ceramic robot on a workbench",
    num_inference_steps=5,
    sigmas=[1.0, 0.875, 0.75, 0.5, 0.25],
    true_cfg_scale=1.0,
    generator=torch.Generator(device="cuda").manual_seed(0),
)

result.images[0].save("output.png")

Required inference settings

  • Use exactly five steps with [1.0, 0.875, 0.75, 0.5, 0.25].
  • Set true_cfg_scale=1.0 and omit the negative prompt.
  • Keep the LoRA scale at 1.0; its alpha equals its rank.
  • Load the adapter repository’s scheduler configuration.
  • Avoid the base scheduler’s shift_terminal: 0.02, which changes the final step and degrades output.
  • Keep the LoRA unmerged to avoid BF16 merge-time rounding.

Reference-image order controls the prompt tokens image 1, image 2, and image 3. Training covered up to three references, so applications should preserve order when constructing editing requests.

Where the preview breaks down

Viggle describes v0.2 as a preview and reports weaker results than the 40-step base model on multi-reference composition, face swaps, identity-preserving edits, and instructions containing several simultaneous constraints. Small or lengthy rendered text can also degrade more often.

The release has not validated 2K output against the base model. RGBA generation, more than three references, and mask-based local editing remain untested, which limits the evidence available for production editing pipelines.

ComfyUI changes the key map

A community-made ComfyUI conversion rewrites all 454 adapter keys with the diffusion_model.transformer. prefix expected by ComfyUI’s standard LoRA loader. Loading the original file through that node can fail silently because its keys use a different namespace.

Existing mixed-precision GGUF files target the v0.1 full fine-tune. Reported GGUF output differs from the upstream Safetensors checkpoint even at BF16-equivalent precision, so identical prompts and seeds can produce changes in composition or identity.

For interactive generation tools, v0.2 offers a five-pass route that retains most of the base model’s measured diversity and average layout behavior. Production evaluations should pin every model component, test representative editing workloads, measure end-to-end latency on target hardware, and resolve the commercial license before deployment.