Not long ago, in a attempt to obtain Consistent portraits using IP-Adapters for SDXL, I shared a comparison between IP-Adapter-Plus-Face and IP-Adapter-FaceID. Today I’ll look at InstantID.

To be honest, I’ve been in two minds about AI models for faces, due to the concerns with deep fakes and using the likeness of living people. I will use only AI-generated images that I generated myself. I’ve also received emails asking me for my output images since those contain the workflow. When I first started my series of Stable Diffusion back in 2022, I did post some AI-images I generated. But honestly, AI images just pollute the Internet and as a result, future training sets may just comprise of fake imagery. Since I am only doing this for fun, I will not post generated images.

About InstantID

From the model card on HuggingFace:

InstantID is a new state-of-the-art tuning-free method to achieve ID-Preserving generation with only single image, supporting various downstream tasks.

Installing

As always, this will be ComfyUI Portable for Windows-specific, so the first step is to look for a suitable custom node. I will use ComfyUI InstantID (Native Support) as it is not just a wrapper, but integrates very well with ComfyUI, and requires less memory.

You need to install InsightFace for Windows. I detailed the steps in my previous post, which now I know originated with ComfyUI-Reactor-node. This time you also need to install omegaconf:

python_embeded\python.exe -m pip install insightface-0.7.3-cp311-cp311-win_amd64.whl omegaconf onnxruntime onnxruntime-gpu

Next, getting the InsightFace antelopev2 model. This is a bit tricky: it’s available in one of Furkan Gözükara’s HuggingFace tools folders or on someone’s Google Drive... (see this GitHub issue #1896)

  • Download antelopev2.zip
  • Extract the 5 .onnx files to ComfyUI\insightface\models\antelopev2

Last but not least, for InstantID, head over to InstantID on HuggingFace to get the model:

  • From the Files tab, download ip-adapter.bin to \ComfyUI\models\instantids
  • While here, download ControlNetModel/diffusion_pytorch_model.safetensors to ComfyUI\controlnet

I hope nothing is missing! :P

Workflow

As the author of ComfyUI_InstantID, noted:

The training data is full of watermarks, to avoid them to show up in your generations use a resolution slightly different from 1024×1024 (or the standard ones) for example 1016×1016 works pretty well.

But otherwise, this model is very good.

Consistent faces in ComfyUI with InstantID and RealVisXL

Use:

  • InstantIDModelLoader to load the ip-adapter.bin file,
  • InstantIDFaceAnalysis to load InsightFace - I use CUDA,
  • ControlNetLoader to load diffusion_pytorch_model.safetensors,
  • CheckpointLoader to load SDXL or one of its derivatives and create your positive and negative prompts,
  • ApplyInstantID to... apply InstantID, and finally
  • KSamplerAdvanced, per typical SDXL workflow.

I found that SDXL v1.0 Base produced more interesting but all with rather plastic-y faces, though the likeness is still clearly there.

Crystal Clear XL is better...

But RealVisXL v3.0 Turbo is the hands-down winner - it produced extremely life-like faces and specked, realistic skin. Just remember this is a turbo model, so 5-ish steps and CFG of 1.5 will suffice.

❮ Older
Newer ❯