Hugging Face Adds 4-Bit Nunchaku Loading To Diffusers

Text-to-image pipelines that can require 20GB to 30GB of VRAM now have a 4-bit loading path in Hugging Face Diffusers.

Hugging Face said in a July 23 post that the Nunchaku Lite integration is available through the standard `from_pretrained()` workflow.

The change targets model loading rather than only reducing stored weight files.

Kernels arrive through the Hub

Nunchaku Lite loads without a custom pipeline class, separate inference engine or local CUDA compilation, according to the post.

The runtime patches relevant `nn.Linear` modules in a stock Diffusers model with SVDQ or AWQ linear layers before loading the checkpoint, and CUDA kernels are downloaded through the `kernels` package.

The integration uses `svdq_w4a4` layers for attention and MLP projections, with INT4 and NVFP4 variants.

A second `awq_w4a16` path covers precision-sensitive normalization and modulation components in model families including FLUX and Qwen-Image.

The hardware table says NVFP4 checkpoints require NVIDIA Blackwell hardware, including RTX 50 series, RTX PRO 6000 and B200 GPUs, while INT4 variants support Turing, Ampere and Ada devices such as RTX 30 and 40 series cards, A100 and L40S.

Benchmarks stay vendor-owned

According to Hugging Face's example, one Nunchaku NVFP4 transformer paired with a bitsandbytes NF4 text encoder generated a square 1024-pixel image in roughly 1.7 seconds on an RTX 5090, while peak memory was about 12 GB; the same paragraph placed the BF16 pipeline at about 24 GB.

In a separate RTX PRO 6000 Blackwell benchmark, the blog listed a BF16 baseline at 3.00 seconds for the full pipeline and 31.1 GB peak VRAM.

The blog's benchmark table listed Nunchaku Lite NVFP4 at 2.27 seconds and 20.6 GB peak VRAM, Nunchaku Lite with `torch.compile` at 1.68 seconds and 20.6 GB, and Nunchaku Lite with an NF4 text encoder at 2.29 seconds and 16.0 GB.

The post characterized the result as up to 50% lower peak VRAM, roughly 30% better latency and as much as a 1.8x speedup with `torch.compile`; those figures remain vendor benchmark claims rather than independent lab measurements.

Quantization workflow widens

The post also points developers to `diffuse-compressor`, a companion toolkit for calibrating, quantizing, packaging and publishing new Diffusers models.

The FLUX.2 Klein 4B example workflow says the inspection stage should identify 100 SVDQ targets, 3 AWQ targets and 6 dense outer linears before quantization proceeds.

The remaining boundary is hardware and model structure.

The compatibility note excludes Volta and Hopper GPUs from current 4-bit kernel support, and the generic Nunchaku Lite path does not infer architecture-specific fused rewrites such as combined QKV projections.

That makes the integration useful as a standard Diffusers loading route, while specialized engines may still matter when a model family needs fused modules beyond the generic layer replacement path.

The post did not disclose production customer names for the new Diffusers path, so deployment proof is still limited to the published benchmark examples.