ControlNet Web API

This page is a guide on how to use ControlNet's Web API.

List all available preprocessors

modules = requests.get("http://localhost:7860/controlnet/module_list").json()["module_list"]
"""
 [
    "None",
    "softedge_pidinet",
    "shuffle",
    "seg_ofade20k",
    "scribble_pidinet",
    "reference_only",
    "openpose_full",
    "normalbae",
    ...
 ]
"""

List all available models

modules = requests.get("http://localhost:7860/controlnet/model_list").json()["model_list"]
"""
  [
    "None",
    "control-lora-canny-rank256 [ec2dbbe4]",
    "control-lora-depth-rank128 [df51c1c8]",
    "control-lora-depth-rank256 [9fe0fd3b]",
    "control-lora-recolor-rank128 [c0bf9b31]",
    ...
  ]
"""

Generation (txt2img)

You need to pass parameters to ControlNet script, when you use A1111's txt2img/img2img generation API.

You may find https://github.com/huchenlei/sd-webui-api-payload-display helpful to reproduce an UI result via API.

import base64
import cv2
import requests

def read_image(img_path: Path) -> str:
    img = cv2.imread(str(img_path))
    _, bytes = cv2.imencode(".png", img)
    encoded_image = base64.b64encode(bytes).decode("utf-8")
    return encoded_image

# Note: You can no longer pass file path in payload as in sd-webui-controlnet.
base64image = read_image("D:/foo.png")
base64mask = read_image("D:/bar.png")

unit1 = {
    "image": base64image,
    "mask_image": base64mask,
    "control_mode": "Balanced",  # "Balanced", "My prompt is more important", "ControlNet is more important" 
    "enabled": True,
    "guidance_end": 1,
    "guidance_start": 0,
    "pixel_perfect": True,
    "processor_res": 512,
    "resize_mode": "Just Resize",  # "Just Resize", "Crop and Resize", "Resize and Fill"
    "threshold_a": 64,
    "threshold_b": 64,
    "weight": 1,
    "module": "canny",
    "model": "control-lora-canny-rank256 [ec2dbbe4]",
    "save_detected_map": True,
    "hr_option": "Both",  # "Both", "High res only", "Low res only"
}
unit2 = ...
unit3 = ...
payload = {
    "alwayson_scripts": {"ControlNet": {"args": [unit1, unit2, unit3]}},
    "batch_size": 1,
    "cfg_scale": 7,
    ... (other parameters)
}

resp = requests.post("http://localhost:7860/sdapi/v1/txt2img", json=payload)
imgs = resp.json()["images"]