FreeU: Free Lunch in Diffusion U-Net

In this paper, we uncover the untapped potential of dif-fusion U-Net, which serves as a “free lunch” that substan-tially improves the generation quality on the fly. We initially investigate the key contributions of the U-Net architecture to the denoising process and identify that its main backbone primarily contributes to denoising, whereas its skip connections mainly introduce high-frequency features into the de-coder module, causing the potential neglect of crucial functions intrinsic to the backbone network. Capitalizing on this discovery, we propose a simple yet effective method, termed “FreeU”, which enhances generation quality without additional training or finetuning. Our key insight is to…