Inversion-based Style Transfer with Diffusion Models

The artistic style within a painting is the means of expression, which includes not only the painting material, colors, and brushstrokes, but also the high-level attributes, including semantic elements and object shapes. Previous arbitrary example-guided artistic image generation methods often fail to control shape changes or convey elements. Pre-trained text-to-image synthesis diffusion probabilistic models have achieved remarkable quality but often require extensive textual descriptions to accurately portray the attributes of a particular painting. The uniqueness of an artwork lies in the fact that it cannot be adequately explained with normal language. Our key idea is to learn the artistic style directly…

• NN
  National Natural Science Foundation of China

  Award: 61832016,U20B2070,62102162
• NS
  Natural Science Foundation of Beijing Municipality

  Award: L221013
• NK
  National Key Research and Development Program of China

  Award: 2020AAA0106200