Paint by Example: Exemplar-based Image Editing with Diffusion Models
University of Science and Technology of China · Microsoft Research Asia (China)
Abstract
Language-guided image editing has achieved great success recently. In this paper, we investigate exemplar-guided image editing for more precise control. We achieve this goal by leveraging self-supervised training to disentangle and re-organize the source image and the exemplar. However, the naive approach will cause obvious fusing artifacts. We carefully analyze it and propose a content bottleneck and strong augmentations to avoid the trivial solution of directly copying and pasting the exemplar image. Meanwhile, to ensure the controllability of the editing process, we design an arbitrary shape mask for the exemplar image and leverage the classifier-free guidance to increase the similarity to the exemplar…
Citation impact
- FWCI
- 34.86
- Percentile
- 100%
- References
- 100
Authors
8Topics & keywords
- Computer science
- Image editing
- Leverage (statistics)
- Artificial intelligence
- Source code
- Fidelity
- Controllability
- Computer vision