Gallery
Stable Diffusion

Cinematix SDXL: Expert Tips and Advanced Techniques for Fine-Tuned Stable Diffusion Models 

Cinematix is the stable diffusion checkpoint merge based on SDXL 1.0 fine-tuned models. It is focused on photorealistic output for concept design, character portraits, and cinematic compositions. It is aimed for advanced users and it implicates workflows using ControlNet or other diffusion control techniques.

The purpose of this model merge is to create similar generative results to my previous Photomatix model, incorporating the advantages of the SDXL base model for style development and testing SDXL LoRAs and technologies (some new models and extensions are available for SDXL only). This article highlights the unique characteristics of this model and offers techniques for effectively working with it.

Portrait of woman and a jar, SDXL character portrait, Stable Diffusion image by Daniel Sandner
Portrait of woman reading a book, SDXL 1.0 character portrait, Stable Diffusion image by Daniel Sandner
Portrait of a man in a scifi costume, SDXL character portrait, Stable Diffusion image by Daniel Sandner

How to Install

In case of A1111, download the model to "\stable-diffusion-webui\models\Stable-diffusion". Download the styles.csv (optional) to "\stable-diffusion-webui\", backup the original styles.csv if you have any.

DOWNLOAD Cinematix SDXL from Civitai

DOWNLOAD CX/PX Styles from Github (Optional)

  • For development version of A1111, enter git checkout dev into console
  • To return back use git checkout main
  • If you have Forge installed as styles.csv, enter git checkout lllyasviel/main or git checkout forge/main , more on Forge in the Ultimate Guide to SDXL article.
  • After switching, pull latest changes with git pull

Performance Across Resolutions

Cinematix works best with standard SDXL resolutions (1024x1024, 1024x768, 960x1280, 1280x1472, 1280x1536), but it can render meaningful images in lower resolutions too (512x768, 768x768, 384x512, 384x256 or 256x192 with use of a proper upscaler, read below). You may experiment with various resolutions for different subjects.

Realistic synthetic photography with SDXL
Regional Prompter and DAT upscaling, DPM++ SDE

Samplers

For lower number of steps, I recommend these samplers:

DPM++ 2S a Karras, DPM2 Karras, DPM++ SDE, DPM++ 2S a, DPM++ 2M SDE Karras, DPM++ SDE Karras (8+ steps)

For standard workflows:

DPM++ 2M Karras, DPM++ SDE Karras, DPM++ 2M SDE Exponential,DPM++ 2M SDE Karras,Euler a,DPM2,DPM2 a,DPM++ 2S a,DPM++ SDE,DPM++ 3M SDE,DPM adaptive,LMS Karras,DPM2 Karras,DPM++ 2S a Karras,Restart,UniPC

Portrait of a man, SDXL character portrait, Stable Diffusion image by Daniel Sandner
Character expression
Study on expression, SDXL character portrait, Stable Diffusion image by Daniel Sandner
Character expression study
Study on expression, man, SDXL character portrait, Stable Diffusion image by Daniel Sandner
Character expression and light study

Lightning Versions

There are SDXL-Lightning versions of Cinematix avalable, which allow much faster generations. You will get the best results using "Euler A SGMUniform" sampling method with 7-11 steps, in one pass, or you may experiment with a sampler in first pass and fix the result in 2nd pass (f.i. with Hires fix). Read more about SDXL-Lightning in this article.

Tips and Workflows for Photorealism

Standard Workflow for Portraits

Standard SD workflow is still the best. At first pass, the image may be darker with more noise than usual, I recommend using this workflow especially when using SDXL concept LoRAs to make more natural looking final image.

  • You may use regular CFG 4+ and number of steps 
  • ADetailer or similar tool to add segmented details (for eyes, face, or person)
  • Hires fix option to upscale image during rendering

SDXL tend to produce very nice images without additional detailing, if the subject is not too small ("far from camera").

For an extensive list of SDXL tips for beginners, see this article on SDXL Photorealism.

Instant ID Workflow for Portraits

You need to adjust CFG to 2, number of steps around 16 should be enough for proper tones.

ControlNet Workflows

As addition to prompt engineering, you may modify the composition with ControlNet models and adapters, like IP-Adapter, Reference, and T2I Adapters. Note that this way you may generate scenes even without a prompt. Whe you get too much contrast in the image, try to lower the CFG.

Style Combination Workflow

When combining the styles, you may get duplicate tokens or unwanted effects. I advise utilizing style combinations for establishing the basic form of the scene and as a foundation for adjustments. Once styles are added, you can seamlessly integrate them into the prompt using the 'Apply all selected styles to prompt' button in A1111. You may then proceed with manual prompt adjustments.

Note that unchanged styles will load from saved file meta information.

Upscaling and Hires fix Using DAT

Upscale in Hires fix: original 256x256, Latent, DAT

For image upscaling and Hires. fix, you may use models set in A1111 or Forge (Latent Bicubic, DAT, or SwinIR) or get additional upscaler models and put them in proper model directories:

  • Look at  https://openmodeldb.info/ (you will find the following models there too)
  • 4x-ClearRealityV1 .pth goes into models/DAT (safe upscale 2x)
  • 4xFaceUpSharpDAT file goes into models/DAT (safe upscale 2x), good for face proportions
  • Other realistic upscale DAT files (spsr, 4xLSDIRDAT, 4xNomosUniDAT_otf) go into models/DAT
  • 4x_RealisticRescaler_100000_G .pth goes into models/realESRGAN
  • 4x-UltraSharp.pth and 4x-UltraScale9_V0.5 BETA.pth goes into models/ESRGAN
  • UltraMix, files go into models/ESRGAN

As a general rule, use Upscale by 1.25x, 1.5x (max 2x), Denoising scale 0.3-0.7.

Beauty of the world, sudanese model, SDXL portrait
Beauty of the world, european model, SDXL portrait
Beauty of the world, syrian model, SDXL portrait
Beauty of the world, european middle-aged model study, SDXL portrait
Beauty of the world, european elderly male model study, SDXL portrait

Comparison

Comparison of SDXL models
"epic valley", with CX Landscape Photo style, 8 steps, CFG 4

Cinematix creates slightly darker and noisier images, better for further processing. The anatomy is not always so good as on this random example, it tends to fail in complicated compositions and there is much to be improved. The other models in the comparison generally do very good images too. The same face and pose were inserted with ControlNet.

Face implanted with Instant ID , SDXL model comparison
Simple portrait of woman waving hand, the same face implanted with Instant ID

What to Make Better

  • eyes shape and details with correct anatomy
  • hand poses and handling objects
  • horizontal (landscape) orientation rendering of figures
  • proportions in composition

Cinematix will be gradually improved in a similar way as Photomatix, with added trained concepts, textures, and lightstyles.

Spacesuit, Scifi SDXL illustration by Daniel Sandner
Robot, Scifi SDXL illustration by Daniel Sandner, Cinematix model
Cyborg, Scifi SDXL illustration by Daniel Sandner
Snail Crab, Scifi SDXL illustration by Daniel Sandner, Cinematix model
Robot with glowing eyes, Scifi SDXL illustration by Daniel Sandner

Conclusion

Cinematix is a very fast realistic SDXL model. SDXL allows better consistency and more sane prompt engineering than previous models (read my review of the original SDXL base model without fine-tuning here). It also performs better with groups of characters or complex compositions. The Latent Consistency Model (LCM) and SDXL-Lightning methods allow faster generations but provide slightly fewer details, which can be remedied with other controlling techniques.

References

Cinematix was created by merging and experimenting with these models:

  • StabilityAI SDXL
  • OpenDalle
  • Hephaistos
  • RaffaelloXL
Inpainting with Cinematix SDXL and Regional Prompter, Sci-fi Concept Illustration by Daniel Sandner
Inpainting with Cinematix SDXL and Regional Prompter
Updated:

Tagged in:

I am a multimedia artist, composer, computer scientist, and author. In my works, I often deal with the philosophy of art and science and its implications in the creative area. I focus on the research of quantum and particle physics and development of machine learning and artificial intelligence— and I like to write about it. My experimental multimedia projects tend to be on the frontiers of several scientific fields — neuroscience, applied physics, astrophysics, informatics, and data science. You will find my experimental AI/ML models on Civitai (airesearch) to freely test.

You may also like:

Subscribe

Stay connected to make sure you don’t miss anything. Join our newsletter community for artists, designers, and art and science enthusiasts.