DiffArtist: Towards Structure and Appearance Controllable Image Stylization

ACM MM 2025

Ruixiang Jiang, Changwen Chen*
The Hong Kong Polytechnic University, *Corresponding Author
arXiv Code Data Huggingface Demo

Gallery (Hover to Stylize)

Cyberpunk style painting, detailed
A precisionism painting, detailed, geometric shapes.
American comic, western style, detailed.
Rococo painting.
A Renaissance painting, detailed, traditional.
A abstract cubism painting, geometric shapes.
A ink-wash watercolor painting.
A fauvist painting, vivid colors.
A precisionism painting, detailed, geometric shapes.
An impressionism painting, claude monet, vibrant color
A traditional watercolor painting, colorful
A neo-expressionism painting, detailed brush strokes
A pixel art painting, detailed, 8-bit
A fauvist painting, vivid colors
An abstract cubist painting, geometric shapes
painting by Edward Munch.
A pixel art painting, detailed, 8-bit.
An ink-wash watercolor painting.
Dadaism style art.
A starry night-style painting, by Van Gogh.
A B&W line art drawing, bold lines.
An impressionism painting, claude monet, vibrant color.
An American comic style painting, detailed, comic book style
An traditional watercolor painting, colorful.

TL;DR: DiffArtist is a zero-shot text-driven stylization method. It requires no finetuning, nor additional ControlNets. Unlike existing stylization methods that entangles structure and appearance, the strength of them are highly controllable in DiffArtist.

Method overview

Method Overview

DiffArtist decompose the structure and appearance stylization with two two separate denoising processes, allowing fine-grained control for both terms.

Disentangled Structure and Appearance Control

Disentangled Structure and Appearance Control

DiffArtist Enables Disentangled Control over Structure and Appearance. Compared with previous methods, the unprecedented dual controllability allows for more customizable stylization.

Additional Control Visualizations

Disentangled Structure and Appearance Control
Disentangled Structure and Appearance Control

BibTeX

@article{jiang2024diffartist,
      title={DiffArtist: Towards Structure and Appearance Controllable Image Stylization},
      author={Jiang, Ruixiang and Chen, Changwen},
      journal={arXiv preprint arXiv:2407.15842},
      year={2024}
      }