Support union ControlNet

[huchenlei]

Closes #2986

https://huggingface.co/xinsir/controlnet-union-sdxl-1.0

Openpose

Do not use key map with face and hand as the union controlnet does not seem to be trained with hand/face annotation

Canny

Depth

SoftEdge

Normal

Segmentation

Note: ofade20k preprocessor does not seem to work

Multi-ControlNet

Currently the multi-controlnet is not working in the optimal way described in the original paper, but you can still try use it, as it can help you save VRAM by avoid loading another controlnet for different type of control.

Unit1 setup

Unit 2 setup

Generation result

Appendix

Input images

Mask

Generation parameters (Parameters including seed are fixed for all examples)

1girl, sitting on throne,, absurdres, highres, (masterpiece:1.2), (best quality, highest quality),
Negative prompt: (lowres, low quality, worst quality:1.2), (text:1.2), watermark, (frame:1.2), deformed, ugly, deformed eyes, blur, out of focus, blurry, deformed cat, deformed, photo, anthropomorphic cat, monochrome, photo, pet collar, gun, weapon, blue, 3d, drones, drone, buildings in background, green
Steps: 30, Sampler: DPM++ 2M, Schedule type: Karras, CFG scale: 7, Seed: 1413879340, Size: 1024x1024, Model hash: 5d973c1f8a, Model: animagine-xl-2.0, ControlNet 0: "Module: none, Model: controlnet++_union_sdxl [15e6ad5d], Weight: 1.0, Resize Mode: Crop and Resize, Processor Res: 512, Threshold A: 0.5, Threshold B: 0.5, Guidance Start: 0.0, Guidance End: 1.0, Pixel Perfect: False, Control Mode: Balanced", ControlNet 1: "Module: canny, Model: controlnet++_union_sdxl [15e6ad5d], Weight: 1.0, Resize Mode: Crop and Resize, Processor Res: 512, Threshold A: 100.0, Threshold B: 200.0, Guidance Start: 0.0, Guidance End: 1.0, Pixel Perfect: False, Control Mode: Balanced", Version: v1.9.4-249-g93c00b2a

[FurkanGozukara]

what should be the file name to use this? when downloading from hugging face

he also has canny model too what file name i should do?