Hugging Face Adds 4-Bit Nunchaku Loading To Diffusers

Diffusers gets a 4-bit loading path

A July 23 Hugging Face blog post introduced a Diffusers integration for Nunchaku Lite, giving text-to-image developers a way to load Nunchaku-style 4-bit checkpoints through the standard `from_pretrained()` workflow.

The post describes modern BF16 text-to-image pipelines as often requiring 20-30 GB of VRAM, while earlier Diffusers quantization options such as bitsandbytes, GGUF, torchao and Quanto mainly reduce model weight storage.

According to the same technical note, SVDQuant runs the main diffusion-transformer layers with 4-bit weights and activations, or W4A4, so the denoising loop can use less memory and run faster.

Kernels arrive through the Hub

Nunchaku Lite loads without a custom pipeline class, separate inference engine or local CUDA compilation, according to the post.

The runtime patches relevant `nn.Linear` modules in a stock Diffusers model with SVDQ or AWQ linear layers before loading the checkpoint, and CUDA kernels are downloaded through the `kernels` package.

The integration uses `svdq_w4a4` layers for attention and MLP projections, with INT4 and NVFP4 variants.

A second `awq_w4a16` path covers precision-sensitive normalization and modulation components in model families including FLUX and Qwen-Image.

The hardware table says NVFP4 checkpoints require NVIDIA Blackwell hardware, including RTX 50 series, RTX PRO 6000 and B200 GPUs, while INT4 variants support Turing, Ampere and Ada devices such as RTX 30 and 40 series cards, A100 and L40S.

Benchmarks stay vendor-owned

According to Hugging Face's example, one Nunchaku NVFP4 transformer paired with a bitsandbytes NF4 text encoder generated a square 1024-pixel image in roughly 1.7 seconds on an RTX 5090, while peak memory was about 12 GB; the same paragraph placed the BF16 pipeline at about 24 GB.

In a separate RTX PRO 6000 Blackwell benchmark, the blog listed a BF16 baseline at 3.00 seconds for the full pipeline and 31.1 GB peak VRAM.

The blog's benchmark table listed Nunchaku Lite NVFP4 at 2.27 seconds and 20.6 GB peak VRAM, Nunchaku Lite with `torch.compile` at 1.68 seconds and 20.6 GB, and Nunchaku Lite with an NF4 text encoder at 2.29 seconds and 16.0 GB.

The post characterized the result as up to 50% lower peak VRAM, roughly 30% better latency and as much as a 1.8x speedup with `torch.compile`; those figures remain vendor benchmark claims rather than independent lab measurements.

Quantization workflow widens

The post also points developers to `diffuse-compressor`, a companion toolkit for calibrating, quantizing, packaging and publishing new Diffusers models.

The FLUX.2 Klein 4B example workflow says the inspection stage should identify 100 SVDQ targets, 3 AWQ targets and 6 dense outer linears before quantization proceeds.

The remaining boundary is hardware and model structure.

The compatibility note excludes Volta and Hopper GPUs from current 4-bit kernel support, and the generic Nunchaku Lite path does not infer architecture-specific fused rewrites such as combined QKV projections.

That makes the integration useful as a standard Diffusers loading route, while specialized engines may still matter when a model family needs fused modules beyond the generic layer replacement path.

The post did not disclose production customer names for the new Diffusers path, so deployment proof is still limited to the published benchmark examples.