Accelerating Stable Diffusion XL Inference with JAX on Cloud TPU v5e
Hugging Face Diffusers now supports running Stable Diffusion XL with JAX on Google Cloud TPU v5e, delivering 2.4x better performance per dollar than TPU v4 with sub-3-second image generation.
Hugging Face Brings SDXL to Google TPU v5e with JAX — Up to 2.4x Better Cost Efficiency
Generative AI models like Stable Diffusion XL (SDXL) have revolutionized the creation of high-quality, photorealistic images from text prompts. However, deploying these powerful models in production comes with significant computational challenges. SDXL's UNet component is roughly three times larger than its predecessor, placing heavy demands on memory and inference speed. Today, Hugging Face has announced that its Diffusers library now supports serving SDXL using JAX on Google Cloud TPU v5e, unlocking unprecedented performance and cost efficiency for AI image generation workloads.
Why JAX + TPU v5e for SDXL?
Google Cloud TPUs are custom-designed AI accelerators optimized for training and inference of large-scale models. The new TPU v5e is purpose-built to deliver high performance at a fraction of the cost — less than half the price of TPU v4 — making advanced AI inference accessible to more organizations.
Two key technologies make this integration especially powerful:
JIT Compilation
JAX's just-in-time (JIT) compiler traces the model's operations during the first inference run, generating highly optimized TPU binaries that are reused in subsequent calls. The catch? All input, intermediate, and output shapes must remain static — but image generation is ideally suited to this constraint. By always generating the same number of images at the same resolution, shapes stay constant, and the one-time compilation cost is quickly amortized over thousands of lightning-fast inferences.
Parallelism via pmap
JAX's pmap function expresses single-program, multiple-data (SPMD) programs that execute in parallel across multiple TPU chips. For text-to-image workloads, this means generating 8 images on an 8-chip TPU instance takes no longer than generating 1 image on a single chip. TPU v5e instances scale from 1 chip all the way up to 256 chips in a full pod, connected by ultra-fast inter-chip links (ICI).
Benchmark Performance
Hugging Face ran SDXL 1.0 base for 20 inference steps with the default Euler Discrete scheduler across both TPU v5e and TPU v4. The results speak for themselves:
- TPU v5e-4 (batch size 4): 2.33s latency, 21.46 perf/$ — 2.4x better than TPU v4
- TPU v5e-4 (batch size 8): 4.99s latency, 20.04 perf/$
- TPU v4-8 (batch size 4): 2.16s latency, 9.05 perf/$
- TPU v4-8 (batch size 8): 4.17s latency, 8.98 perf/$
The performance-per-dollar advantage of TPU v5e over v4 is a remarkable 2.4x improvement, driven primarily by the newer chip's better cost-efficiency.
Building an SDXL Pipeline with JAX
Getting started with SDXL on TPU is straightforward thanks to the Hugging Face Diffusers library. The pipeline follows a functional JAX paradigm where model parameters are returned separately and passed during inference. After loading the model from the Hugging Face Hub and converting weights to bfloat16 for efficiency, you can replicate parameters across devices using JAX's replicate() function.
The first inference run triggers JIT compilation — which takes about 3 minutes — but subsequent calls are blazing fast. In their demo deployment, Hugging Face serves four 1024×1024 images in approximately 2.3 seconds of generation time on a TPU v5e-4 instance.
The entire codebase is open source and available in Hugging Face Diffusers today. A live demo running on several TPU v5e-4 instances with a load-balancing server lets users experience the speed firsthand.
What This Means for AI Image Generation
This integration lowers the barrier to production-grade AI image generation. The combination of JAX's compilation-powered performance, TPU v5e's cost efficiency, and Hugging Face Diffusers' ease of use means that startups and enterprises alike can deploy SDXL at scale without breaking the bank. At 2.4x better performance per dollar than TPU v4 and sub-3-second image generation, the economics of running generative AI in production just got significantly more attractive.
Hugging Face has indicated that future posts will cover dynamic scaling solutions using Google Kubernetes Engine (GKE), suggesting an even more flexible infrastructure layer is on the horizon.
