Gallery
How to Achieve Hyperrealism with FLUX.1 using ComfyUI
Tutorial

Fast Hyperrealism with FLUX.1: Local Generation Guide

FLUX.1 is suite of text-to-image models using novel functional approach to image synthesis. FLUX is creation of AI researchers from The Black Forest Team (VQGAN, Latent Diffusion, Stable Diffusion XL, SVD). FLUX.1  offers better text render, better details and compositions, and improvements in rendering anatomy. In this article we will install FLUX.1 (schnell) for local image generation (you may use the tutorial also to install the dev version).

Introduction and System Requirements

You'll need an SDXL-capable machine. All workflows in this article were tested on an NVIDIA RTX A4000 with 16GB VRAM (system has 64GB RAM). I recommend at least 16GB VRAM for optimal performance of FLUX.1 (schnell), the more memory the better. FLUX.1 (Pro) version is more resource hungry and is available as an API at the time being.

If you are already using ComfyUI or SWARM, you may jump directly to FLUX.1 files download.

FLUX generative AI model photorealism and stylized hyperrealistic portraits
FLUX excels at hyperrealistic details and produces notably improved hand poses out of the box.

Portable Setup of ComfyUI

Download and Installation

Adjusting System Paths

If you are just starting your AI generation journey, just put your downloaded models into proper ComfyUI folders. If you have models already downloaded for A1111 or SWARM installations, you do not need to download or duplicate those files.

  • Find the file extra_model_paths.yaml.example in your ComfyUI, rename it to extra_model_paths.yaml. Then open the .yaml file and add:
  • In "a111: base_path:" section add the path to your models, f.i. C:/stable-diffusion-webui
  • At the end of the section (after controlnet: models/ControlNetentry), add two lines (needed for FLUX workflows): 
  • unet: models/unet
    clip: models/clip

Running ComfyUI and Updating

You will run Portable ComfyUI with run_nvidia_gpu.bat. You may update to the latest version in the future with update/update_comfyui.bat.

Installing ComfyUI-Manager

For an effective work with various nodes, install ComfyUI-Manager (it will appear as the Manager button in ComfyUI main menu)

  • Install git (you probably have it already)
  • Go to ComfyUI/custom_nodes directory.
  • From there, run cmd command to open the terminal
  • Enter git clone https://github.com/ltdrdata/ComfyUI-Manager.git
  • If you restart ComfyUI after the installation finishes (usually couple of seconds), ComfyUI-Manager should be ready.
FLUX.1 schnell: Fast realism in details
FLUX.1: Realism in details

GGUF: Installing Support for GGUF Format Models and Encoders for Portable ComfyUI

Currently ComfyUI does not support GGUF format yet. You can install custom nodes from here https://github.com/city96/ComfyUI-GGUF  (this way you can use GGUF dev and schnell quantized models for speed and lower filesize. Simplest installation for ComfyUI Portable:

  • go to your Comfy folder where run_nvidia_gpu.bat resides
  • run terminal cmd from this folder with the command:
git clone https://github.com/city96/ComfyUI-GGUF ComfyUI/custom_nodes/ComfyUI-GGUF
.\python_embeded\python.exe -s -m pip install -r .\ComfyUI\custom_nodes\ComfyUI-GGUF\requirements.txt

Now you can use custom GGUF nodes Unet Loader(GGUF) for models and DualCLIPLoader(GGUF) for encoders. Check examples in my test workflows https://github.com/sandner-art/ai-research/tree/main/FLUX

FLUX.1

Needed Files/Models

  1. Download text encoders https://huggingface.co/comfyanonymous/flux_text_encoders/tree/main . Files clip_l.safetensors and t5xxl_fp8_e4m3fn.safetensors will work with the provided workflows.The files will go to folder models/clip.
    Alternative text encoders for better text rendering in the image are here: https://huggingface.co/zer0int/CLIP-GmP-ViT-L-14/tree/main  (try ViT-L-14-TEXT-detail-improved-hiT-GmP-TE-only-HF or ViT-L-14-TEXT-detail-improved-hiT-GmP-HF.
  2. Download VAE https://huggingface.co/black-forest-labs/FLUX.1-schnell/tree/main/vae The file diffusion_pytorch_model.safetensors (rename it as you want) will go to folder models/VAE.
  3. Download the FLUX.1 model https://huggingface.co/Kijai/flux-fp8/tree/main The file flux1-schnell-fp8.safetensors will go to folder models/unet. You may, of course, experiment with the development (dev) version as well.

Note that the main model goes to models/unet folder (not the usual Stable Diffusion folder). If there is no unet folder in your SD models path, make unet folder manually).

Workflows

Run ComfyUI. Download the images or workflow from the test folder  https://github.com/sandner-art/ai-research/tree/main/FLUX and drag+drop into ComfyUI workplace in the browser. This will load the node workflow into ComfyUI. 

If you see some red nodes indicating an error, install missing nodes with ComfyUI-Manager. Open the MANAGER button from the menu and use "Install Missing Custom Nodes".

FLUX.1 LoRAs

FLUX.1 test LoRAs https://huggingface.co/XLabs-AI/flux-lora-collection/tree/main Workflows in references.

FLUX.1 ControlNet

ControlNet  Collections https://huggingface.co/XLabs-AI/flux-controlnet-collections/tree/main . Workflows in references.

Using X-LABS (flux-dev,) Controlnets in ComfyUI

For the XLABS test workflows to work, you need to put Controlnets models into the folder \ComfyUI\models\xlabs\controlnets, otherwise the workflow will not find the models!

Forge UI Supports FLUX.1 

You may run FLUX models also in A1111 Forge now, check this article for more details.

FLUX.1 (schnell) image generation for free on local machine
FLUX.1 (schnell)

Conclusion

FLUX.1's potential is truly impressive. Even as a base model, it generates astonishingly detailed images that often surpass expectations. While prompt engineering requires a bit more finesse due to the model's tendency towards hyperrealism, the quality is undeniable. The combination of specific details and careful style guidance is key.

Also the chronic pain of SD models—generating hands and limbs—is considerably better (even at quite crazy angles and poses).  However, character interaction with objects is not so great yet.

As we will surely see a fast evolution with custom LoRAs and finetuned versions of the base model, it seems most text-to-image models have found its contender. 

You can test FLUX.1 (Schnell, Dev, Pro) on Civitai, and also train LoRAs for FLUX.1 there.

References

Updated:

Tagged in:

I am a multimedia artist, composer, computer scientist, and author. In my works, I often deal with the philosophy of art and science and its implications in the creative area. I focus on the research of quantum and particle physics and development of machine learning and artificial intelligence— and I like to write about it. My experimental multimedia projects tend to be on the frontiers of several scientific fields — neuroscience, applied physics, astrophysics, informatics, and data science. You will find my experimental AI/ML models on Civitai (airesearch) to freely test.

You may also like:

Subscribe

Stay connected to make sure you don’t miss anything. Join our newsletter community for artists, designers, and art and science enthusiasts.