Skip to content

xzuyn/OnnxDiffusersUI

 
 

Repository files navigation

xzuyn fork

Don't expect everything to work. This is my fork for testing/sharing my WIP stuff before it gets refined for a push to the main repo.

Also if you would like to try out my model merges, I have a photo merge which uses 7 models and an anime merge which uses 9 models.

if you feel like sending me a tip you can do so here; https://www.buymeacoffee.com/xzuyn. although your money is probably better spent elsewhere.

OnnxDiffusersUI

I’ve been helping people setup Stable Diffusion and run it on their AMD graphics card (or CPU) on Windows. I’ve also wrote a basic UI for the diffusers library version to make it more user friendly. This guide is a consolidation of what I’ve learned and hopefully will help other people setup their PC to run Stable Diffusion too.

The intent of this UI is to get people started running Stable Diffusion on Windows. As such this UI won't be as feature rich as other UI, nor will it be as fast as running Stable Diffusion on Linux and ROCm.

Example screenshot:
example screenshot

Example img2img loopback (first image is the input, then loopbacked at 0.25 denoise [16 step DEIS]): example screenshot using img2img loopback

Credits

A lot of this document is based on other guides. I've listed them below:

Prerequisites

You'll need to have a few things prepared first:

NOTE: For Windows 10/11 you need to do an additional step. Go to Settings -> search for "Manage app execution aliases" -> disable the App Installer for "python.exe" and "python3.exe".

To check if they’re installed properly open up command prompt and run the following commands:

python --version
git --version
pip --version

There shouldn't be any "not recognized as an internal or external command" errors.

Creating a Workspace

Start by creating a folder somewhere to store your project. I named mine stable_diff.

Download the setup.bat file and save it into your stable_diff folder: https://raw.githubusercontent.com/azuritecoin/OnnxDiffusersUI/main/setup.bat
NOTE: make sure you save the file as a .bat file and not a .bat.txt file.

Open command prompt and navigate to your stable_diff folder. Once there run the setup.bat file:

cd <path to your stable_diff folder>
.\setup.bat

The setup batch file should create a virtual envrionment and install the Python packages. It will also download Python scripts from online repositories.

Activate the virtual environment:
.\virtualenv\Scripts\activate.bat

At this point you should be in your virtual environment and your prompt should have a (virtualenv) at the begining of the line. To exit the virtual environment just run deactivate at any time.

To restart the virtual environment after closing the command prompt window, cd back into the stable_diff folder and run the .\virtualenv\Scripts\activate.bat batch file again.

Logging in Using Your Token

Login to huggingface:
huggingface-cli.exe login
step1

Go to the tokens page of the huggingface website and copy your token.
step2

Go back to the command prompt window. Make sure there's no characters selected (see below). Press Esc to de-select.
step3

Right click the Title Bar -> Edit -> Paste -> Press Enter. You should be logged in at this point.
step4

NOTE: when pasting, the command prompt will look like nothing has happened. This is normal behaviour, just press enter and it should update.

Download Model and Convert to ONNX

Go to https://huggingface.co/runwayml/stable-diffusion-v1-5 and accept the terms and conditions for the model.

Option 1

Run the Python script to download and convert:
python convert_stable_diffusion_checkpoint_to_onnx.py --model_path="runwayml/stable-diffusion-v1-5" --output_path="model/stable_diffusion_onnx"

NOTE: This may take a while depending on your internet connection speed.

Option 2

Althernatively, you could download the pre-converted version of the model using git:
git clone https://huggingface.co/runwayml/stable-diffusion-v1-5 --branch onnx --single-branch model/stable_diffusion_onnx

Basic Script and Setup Check

Run the txt2img_onnx.py script and check if any images were generated in the output folder. NOTE: some warnings may show up but it should be working as long as an output image is generated:
python txt2img_onnx.py

If an image was generated and it's not just a blank image then you're ready to generate art! You can use the txt2img_onnx.py script to input your own prompt for example:
python txt2img_onnx.py --prompt="tire swing hanging from a tree" --height=512 --width=512

Running The GUI

Run the UI script and wait for everything to load:
python onnxUI.py

Once you see "Running on local URL:" open up your browser and go to "http://127.0.0.1:7860". You should be able to generate images using the web UI. To close the program, go back to the command prompt and hit ctrl-C.

Using Other Models

Models Using Diffuers

If the model is on the hugging face website and it's using the diffusers library, then you can use the same convert script from the guide. Make sure you've accepted the terms for any new model. In this example I'll use waifu-diffusion.
python convert_stable_diffusion_checkpoint_to_onnx.py --model_path="hakurei/waifu-diffusion" --output_path="model/waifu_diffusion_onnx"

Models Using .ckpt

If the pretrained model is a .ckpt file, then you'll need to do a two step conversion. You first will need to convert from .ckpt to diffusers, then from diffusers to ONNX.

Download the .ckpt model of your choice and put them in your stable_diff folder. Run the first conversion script, using trinart2_step115000.ckpt in this example:
python convert_original_stable_diffusion_to_diffusers.py --checkpoint_path="./trinart2_step115000.ckpt" --dump_path="./trinart2_step115000_diffusers"
Then run the second conversion script:
python convert_stable_diffusion_checkpoint_to_onnx.py --model_path="./trinart2_step115000_diffusers" --output_path="model/trinart2_step115000_onnx"
NOTE: make sure the --dump_path in the first script and the --model_path is the same folder name.

After Conversion

Once you have your newly converted model, make sure the model is saved into the model folder (create one if it doesn't exist). Once you restarted the UI, select the model from the dropdown menu on the top left.

Running Stable Diffusion on CPUs

If you don't have a graphics card with enough VRAM or you only have onboard graphics, you can still run Stable Diffusion with the CPU. Simply add a --cpu-only flag to your command line:
python txt2img_onnx.py --cpu-only

Updating onnxUI.py

If you want to update the program, download the latest setup.bat and overwrite the old one: https://raw.githubusercontent.com/azuritecoin/OnnxDiffusersUI/main/setup.bat

Then run the following:
.\setup.bat -update

NOTE: if you're updating from diffusers v0.5.1 and below, you will need to re-convert your models.

Inpainting Fix

  • If inpainting does not work for you, please follow these steps from deinferno to fix it.
    • Within: virtualenv\lib\site-packages\diffusers\pipelines\stable_diffusion\pipeline_onnx_stable_diffusion_inpaint_legacy.py
    • Find (likely on line 402): sample=latent_model_input, timestep=np.array([t]), encoder_hidden_states=text_embeddings
    • Replace with: sample=latent_model_input, timestep=np.array([t], dtype="float32"), encoder_hidden_states=text_embeddings

About

UI for ONNX based diffusers

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • Python 97.9%
  • Batchfile 2.1%