Gallery
FLUX

Latent Interpolate Upscale: Expanding FLUX and SDXL's Denoising Range

Latent Interpolate Upscale for Flux and SDXL, test scene
Latent Interpolate Upscale vs. Latent Upscale in Flux, by Daniel Sandner, 2024

When upscaling images with FLUX or SDXL models, a common challenge arises: low denoise values can introduce strange artifacts, while higher values (exceeding 0.6) may compromise the original image's composition, facial features, or overall aesthetic. Fortunately, several strategies can address these issues.

You can use a different model for the upscale (Ultrasharp, DAT, or some model from openmodeldb.info). You can also employ tile ControlNets with a good upscaling model. But we would like to use FLUX capabilities for precise details when creating hi-res image. Is there a solution without using a ControlNet or switching models?

FLUX: Latent Interpolate Upscale method workflow for upscaling stable diffusion images, by Daniel Sandner
Latent Interpolate Upscale in Flux: Artifacts occurring when denoise value is too low in a standard Latent Upscale workflow
Latent Interpolate Upscale: Adding details to FLUX image (upscaled by FLUX model) in Comfy UI
Latent Interpolate Upscale is adding details in FLUX image (upscaled by FLUX model)

After some experiments I have made a group node for Comfy, which is mixing two latents to overcome these issues a bit. In the images above, you may see how this method fixes the artifacts caused by low denoise value when using Flux.1 model (you will encounter similar effects in SDXL models too). Also note  how it keeps the composition from the initial image, because of lower denoise value. Explanation video tutorial will follow, meanwhile:

How Latent Interpolate Upscale Works

The workflow consists of taking the initial image, scaling it using a traditional algorithm (e.g., Lanczos), and incorporating it as a leading element for interpolation. The other element is the image upscaled by the latent upscaler node. These two latent representations are then interpolated in a variable ratio.

Benefits of the Method

  • Better rendering of diverse formats (wideangle, longer "landscape" ratios)
  • Better representation of characters/figures in the background 
  • Use of low denoising (named Denoise for Upscale in the node) value, 0.15-0.4 for upscaling
Flux upscaling: Latent Interpolate Upscale (LIU) screenshot node in Comfy
Latent Interpolate Upscale (LIU) node in Comfy

LIU Group node:
Scale by: Upscale amount

Denoise for Upscale: Amount of change (higher=more changes to the original image). Set to 0.15+ for LIU

Latent Interpolate RATIO: lower= latent upscale dominant, higher= Lanczos upscale dominant. Use 0.4+ to remove upscale artifacts

Results

A scene in Egypt, a demonstration of Flux upscaling
Click to enlarge
A scene in Egypt, a demonstration of Flux upscaling method
The characters in the background are better formed when using Latent Interpolate Upscale (LIU)
A comparison scene: demonstration of Flux upscaling method
Click to enlarge, latent artifacts in Flux, notice shadowing and pixelating
A comparison of latent upscales scene: demonstration of Flux upscaling method in Comfy
Click to enlarge, latent artifacts in Flux
A comparison of latent upscales and denoising values: demonstration of Flux upscaling method in Comfy, portrait of a woman
Removing artifacts, click to enlarge
FLUX, portrait of a woman: A comparison of latent upscales details and denoising values: demonstration of Flux upscaling method in Comfy
Notice the details which are kept from the original, as lightbulb type, boots etc.
A comparison of latent upscales in Flux
Notice how vertical artifacts imprints into vertical line objects and textures (shirt) on the right image

Variants Suggestions

Latent Interpolate Upscale is not much slower than Latent Upscale. Instead of fast Lanczos interpolation, you may also experiment with various upscaling models for the combination of latents.

Download

  • Latent Interpolate Upscale for Comfy Workflows on Github 
  • Drag&drop workflows or images into your ComfyUi  workplace window
  • Workflows are using Comfyroll Studio nodes for image comparison, install it via Comfy UI Manager

Note that there is a limit for FLUX when generating hi-res images—at higher resolution it is lacking performance and image quality. After your image composition and detailing is ready, use other upscaling methods and models to upscale to a very high resolution (using tiles).

Conclusion

'Latent Interpolate Upscale' brings some benefits when upscaling or generating hi-res wideangle images. By leveraging interpolation techniques, I have demonstrated the ability to expand the denoising range of FLUX and SDXL, enabling the creation images which are more close to the input, without using Controlnet (check an application of this upscale method in conjunction with CosXL edits). Latent Interpolate Upscale method is a kind of combination of img2img and latent upscale, bringing more details while retaining most of the original concept and achieving a reasonable speed. 

References

Updated:

Tagged in:

I am a multimedia artist, composer, computer scientist, and author. In my works, I often deal with the philosophy of art and science and its implications in the creative area. I focus on the research of quantum and particle physics and development of machine learning and artificial intelligence— and I like to write about it. My experimental multimedia projects tend to be on the frontiers of several scientific fields — neuroscience, applied physics, astrophysics, informatics, and data science. You will find my experimental AI/ML models on Civitai (airesearch) to freely test.

You may also like:

Subscribe

Stay connected to make sure you don’t miss anything. Join our newsletter community for artists, designers, and art and science enthusiasts.