Skip to content

Model download

Chenlei Hu edited this page Mar 30, 2024 · 13 revisions

This page documents multiple sources of models for the ControlNet extension.

Introduction

ControlNets allow yout to guide the image generation with user supplied images. The original implentation with examples can be found in https://github.com/lllyasviel/ControlNet.

Installation

After you download models, please put them under models/ControlNet. You need to download model files ending with ".pth" / ".safetensors" / ".bin". Do not right-click the filenames in HuggingFace website to download. Some users right-clicked those HuggingFace HTML websites and saved those HTML pages as PTH/YAML files. They are not downloading correct files. Instead, please click the small download arrow “↓” icon in HuggingFace to download.

Stable Diffusion 1.5 / 2.0

Stable Diffusion 1.5 and Stable Diffusion 2.0 ControlNet models are compatible with each other. There are three different type of models available of which one needs to be present for ControlNets to function

  • LARGE - these are the original models supplied by the author of ControlNet. Each of them is 1.45 GB large and can be found here.
  • MEDIUM - these have been shrunk to half their size by converting them to 16 bit. Each of them is 723 MB large and can be found here.
  • SMALL - these are LoRa implementations that only use 136 MB each and can be found here.

Stable Diffusion XL

There is no official SDXL ControlNet model. All models come from Stable Diffusion community. Here are some collections of SDXL models:

  • diffusers ControlNet
  • Controllllite
  • Stability AI Control LoRA
  • T2I Adapter
  • Tile realistic(NEW)
  • Tile anime
  • MLSD
  • DW pose
  • Normal
  • Recolor Luminance
  • Softedge
  • Segment animeface

Extensions

Other models supported in the ControlNet extension.

Animal Openpose [SD1.5] Original Project repo - Models

IPAdapter [SD1.5 / SDXL] Original Project repo - Models

IPAdapter FaceID [SD1.5 / SDXL] Original Project repo - Models

PhotoMaker [SDXL] Original Project repo - Models

InstantID [SDXL] Original Project repo - Follow instruction in here

Fooocus Inpaint [SDXL] patch - Needs a little more explanation

T2I Adapter

https://huggingface.co/TencentARC/T2I-Adapter/tree/main/models

Clone this wiki locally