FastSDXL

This is an efficient implementation of Stable-Diffusion-XL. I make the following improvements:

• Reconstruct the architecture of UNet. This UNet implementation is faster than others (Diffusers, Fooocus, etc.). If you are interested in this implementation, please see FastSDXL/BlockUNet.py. The source code of this UNet is short and easy to understand. You can also use this component in your own projects.
• Use a trainable scheduler named OLSS. The implementation of OLSS is based on this project. I find this scheduler can improve the quality of generated images with given steps but we need to train it first (the training process requires a few minutes). To synthesize images with a specific style, OLSS is a good choice.

Usage

The code is headless. I developed this project based on diffusers==0.21.3. If you find it cannot run with another version of diffusers, please open an issue and tell me.

pip install diffusers safetensors torch gradio

To launch a webui without OLSS scheduler, please run the following command.

python launch_without_olss.py

To train an OLSS scheduler, please see run_olss.py for more details.

I trained an OLSS scheduler with the style of "Slightly Cinematic". You can use it by running the following command.

python launch_with_olss.py

Efficiency of the reconstructed UNet

I tested my code using NVidia 3060 laptop (6G, 85W). The resolution is 1024*1024, and the model is converted to float16 format.

• Diffusers: CUDA out of memory
• Fooocus: 1.78s/it
• FastSDXL (ours): 1.17s/it

Performance of OLSS scheduler

Here are some examples of OLSS.

Prompt template: cinematic still {prompt}. emotional, harmonious, vignette, highly detailed, high budget, bokeh, cinemascope, moody, epic, gorgeous, film grain, grainy Negative prompt: anime, cartoon, graphic, text, painting, crayon, graphite, abstract, glitch, deformed, mutated, ugly, disfigured

• a young girl, black hair, white clothes, in a garden, the background is red and white flowers

In the image generated by OLSS, the details of flowers on the right are more realistic.

DDIM	DPM	OLSS

• a forest in spring, birds

More birds.

DDIM	DPM	OLSS

• an orange cat and a pink ball on a white sofa

In this example, the image generated by OLSS is completely different with others.

DDIM	DPM	OLSS

• a robot, with blue lightsaber, in a city

Sometimes OLSS can modify the composition and fix the composition error.

DDIM	DPM	OLSS