Gallery
AI

Distribution Matching Distillation (DMD2): Photorealism in Lesser Steps (ComfyUI Workflow and LoRA Solution)

Distribution Matching Distillation (DMD2): Steps otimalization for SDXL

We have explored several solutions to distill diffusion models into efficient low-step generators. Distilled models produce rapid results, generally with reduced detail but often exhibiting improved compositional consistency and/or prompt adherence. Their application is promising for fast inference, batch dataset processing, and animation rendering.

DMD2 method distills a costly diffusion modelinto a one- or multi-step generator. For our use we will test the multistep generator version. We can use the technique either as full ComfyUI workflow, or as a LoRA version (used as standard lora, similar to Lightning destilled LoRAs) in ComfyUI or A1111.

Download and Installation

DMD2 Distribution Matching Distillation in SDXL
DMD2 unet model workflow (Lightning 8 Cinematix-L8 base model: 3 steps, 5 steps, 7 steps
How to optimize number of steps in SDXL models
DMD2 unet model workflow (Cinematix base model): 4 steps, 5 steps

SWARM and ComfyUI

In 2-pass technique (when you are using latent upscale), you may use around 1-3 steps for the first inference. Check test workflows for examples.

2pass inference in SDXL
1st pass 1step, 2ndpass 3 steps.

Using LoRA Version

Download dmd2_sdxl_4step_lora_fp16.safetensors LoRA from the same folder as before, in your standard \stable-diffusion-webui\models\Lora folder. Use it at full weight: 1. This is the fastest way how to test it in A1111.

Conclusion

DMD2 is surprisingly effective. While it can produce artifacts depending on the base model, it offers a new method for extracting high-quality outputs from SDXL base models, enabling the creation of realistic images and artistic effects. Compared to models like Flux (schnell), DMD2 has advantages such as compatibility with SDXL models and SDXL LoRAs.

You may also try NVIDIA AYS , a very good technique for fast rendering with SDXL.

Resouces

Updated:

Tagged in:

I am a multimedia artist, composer, computer scientist, and author. In my works, I often deal with the philosophy of art and science and its implications in the creative area. I focus on the research of quantum and particle physics and development of machine learning and artificial intelligence— and I like to write about it. My experimental multimedia projects tend to be on the frontiers of several scientific fields — neuroscience, applied physics, astrophysics, informatics, and data science. You will find my experimental AI/ML models on Civitai (airesearch) to freely test.

You may also like:

Stable diffusion digital art collage by Daniel Sandner

Ultimate Guide to SD 1.5 Photorealism

Although SDXL 1.0 has many benefits, I think SD 1.5 models will run side by side for some time. I have prepared this article to summarize my experiments and findings and show some tips and tricks for (not only) photorealism work with SD 1.5. I will use the Photomatix model and AUTOMATIC1111 GUI, but the principles will be the same for most SD 1.5-based models.

Read more

Subscribe

Stay connected to make sure you don’t miss anything. Join our newsletter community for artists, designers, and art and science enthusiasts.