Artist: Aesthetically Controllable Text-Driven Stylization without Training

Ruixiang Jiang, Changwen Chen*
The Hong Kong Polytechnic University, *Corresponding Author
arXiv Code Data Huggingface Demo
Example results

TL;DR: Artist stylizes images based on text prompts, preserving the original content while producing high aesthetic quality results. No finetuning, no ControlNets, it just works with your pretrained StableDiffusion model.

Method overview

Method Overview
Artist disentangle the content and style generation into two separate denoising processes with shared information, whose features are used to control the main stylization branch.

Aesthetic Control

Control

Artist allows the user to easily control the abstractness of the stylization, which includes but is not limited to stroke, shape, and color. results.


Gallery

Cyberpunk style painting, detailed
A precisionism painting, detailed, geometric shapes.
American comic, western style, detailed.
Rococo painting.
A Renaissance painting, detailed, traditional.
A abstract cubism painting, geometric shapes.
A ink-wash watercolor painting.
A fauvist painting, vivid colors.
A precisionism painting, detailed, geometric shapes.
An impressionism painting, claude monet, vibrant color
A traditional watercolor painting, colorful
A neo-expressionism painting, detailed brush strokes
A pixel art painting, detailed, 8-bit
A fauvist painting, vivid colors
An abstract cubist painting, geometric shapes
painting by Edward Munch.
A pixel art painting, detailed, 8-bit.
An ink-wash watercolor painting.
Dadaism style art.
A starry night-style painting, by Van Gogh.
A B&W line art drawing, bold lines.
An impressionism painting, claude monet, vibrant color.
An American comic style painting, detailed, comic book style
An traditional watercolor painting, colorful.

BibTeX

@misc{jiang2024artist,
      title={Artist: Aesthetically Controllable Text-Driven Stylization without Training},
      author={Ruixiang Jiang and Changwen Chen},
      year={2024},
      eprint={2407.15842},
      archivePrefix={arXiv},
      primaryClass={cs.CV}
      }
}