ENFUGUE Server v0.2.2

[painebenjamin]

New Features

1. Depth ControlNet XL

ControlNet Depth XL has been added. Additionally, settings have been exposed that allow users to specify the path to HuggingFace repositories for all ControlNets.

• This will allow you to use ControlNets released on HuggingFace the moment they're available, without needing to wait for an update.

The ControlNet settings in the UI have empty settings for the unreleased XL ControlNets.

2. Task Transparency

Improved visibility into what the back-end is doing by added the current task to the UI and API.

The current task will change many times during extended diffusion plans.

3. Context Menus and Keyboard Shortcuts

Contextual menus have been added for the currently active canvas node and/or the currently visible image sample.

• These menus perform the same operations as the buttons in the header of the node or in the toolbar of the sample. This just provides another way to access these options that is always visible, without you needing to move the canvas to the buttons/toolbar.

Keyboard shortcuts have also been added for all menu items, including the contextual menus added above.

• Hold down the shift key to highlight the keys to press in the menu bar. Press the key corresponding to a menu item to expand the sub-menu, and show the keys to press for those items as well.
• When you aren't typing in a text input, pressing Shift+Enter will invoke the engine (i.e. click 'Enfugue.')
• When you are typing in the global prompt fields, press Ctrl+Enter to invoke the engine.

Showing context menus and their corresponding keyboard shortcuts.

4. Iterations Parameter

Iterations has been added as an invocation parameter in the UI and API. This allows you to generate more images using the same settings, without needing the VRAM to generate multiple samples.

• Upscaling will be performed on each sample separately.
• When using multiple samples, you can multiple your sample count by the iterations to determine the total number of images you will generate.

Generating 2 samples at a time over 3 iterations.

5. Secondary Prompts

Secondary prompts have been added to all prompt inputs. This allows you to enter two separate prompts into the primary and secondary text encodes of SDXL base models.

• This can have many varying affects, experimentation is encouraged.

The UI for secondary prompts.


The sample prompt with different dog breeds as secondary prompts produces visibly similar images with small modifications. Click the image for full-resolution (1024×1024)

6. Inpainting Options

More inpainting options have been made available to better control how you want to inpaint.

• Cropped inpainting is now optional, and you can increase or decrease feathering to better blend your inpaints.
• You can now select any model as an inpainting model, not just SD 1.5 inpainting pipelines. This includes XL models.
• An additional setting has been exposed to disable Enfugue's automated inpainting checkpoint creation. If you disable this, you will need to provide your own fine-tuned (9-channel) checkpoint for inpainting if you wish to use one.

Inpainting with cropping.


Inpainting without cropping.

7. Extended VAE Options

You can now select multiple VAE for various pipelines, to further enable mixing SD 1.5 and XL. Additionally, an "other" option is provided to allow selecting any VAE hosting on HuggingFace.

• Since this will download arbitrary resources, it is only enabled for administrators. Users with limited capabilities cannot select 'other.'

The VAE 'Other' option for the refiner pipeline.

8. Image Metadata

Metadata has been added to images generated by Enfugue. Drag and drop the image into Enfugue to load the same settings that generated that image into your UI.

• The metadata includes the state of all of your UI elements as well as the state generated from the backend.
• This does not include any details from your computer, your network, or anything else outside of what was needed to generate the image.
• At the present moment, if you copy and paste the image using your operating system's clipboard, the metadata is lost on all tested platforms. This is a limitation currently placed on transferring PNG's to your clipboard and cannot be worked around, so be sure to drag-and-drop or save-and-upload if you wish to share your settings.

An example of image metadata (click image to view full size.)

9. Upscale Pipeline Options

Provided the ability to select the pipeline to use when upscaling.

• The default behavior is the same as the previous behavior; it will use a refiner pipeline when one is available, otherwise it will use the base pipeline. You can now select to use the base pipeline regardless of whether or not you set a refiner pipeline.

The upscale pipeline option.

10. Intermediate Options

Added an option for how often to decode latents and generate intermediate images.

• It will also take longer to generate an image with intermediates than it will without. However, you will have a chance to see how the image is coming along and stop generation prior to completion if you enable them.
• The default settings is the recommended value - every 10 inference steps, an intermediate will be decoded.
• If you set this to 0, intermediates will be disabled, and as a result it will be possible to unload the decoder during denoising. This can help reduce VRAM consumption if you are hitting out-of-memory errors.
• This only applies during primary inference and refining. Intermediates are always disabled during upscaling.

The intermediate steps option.

11. Log Pause

Added a button to pause the log view, enabling you to take your time and read the entries rather than having to chase them when something is writing logs.

The log pause button.

12. Other

Further improved memory management resulting in lower VRAM overhead and overall faster inference. ControlNet's are now loaded to the GPU only when required, and VAE will be unloaded when no longer required. This means some users who have had issues with using the large XL ControlNet's may find them working better in this release.

Fixed Issues

1. The issue that previously limited multi-diffusion (i.e. chunked/sliced image generation) to a select number of schedulers has been remedied, and you can now use all* schedulers for multi-diffusion as well as single (unchunked) diffusion. For this reason, the setting for selecting different schedulers for single- and multi-diffusion has been removed.
   • There is one exception, and that is the DPM Solver SDE scheduler, which will fail when using chunking.
2. Fixed an issue whereby typing into the model picker could sort the results in a seemingly nonsensical way when using both configured models and checkpoints.
3. Fixed an issue whereby MacOS users on a trackpad could not move the canvas if it was entirely occupied by a scribble node or an image node with an inpaint mask.
   • Since there is no middle-mouse on MacOS trackpads, and Control+Click cannot be overridden due to MacOS, you must use the Options+Click method to pan the canvas in such scenarios.

Full Changelog: 0.2.1...0.2.2

How-To Guide

Installing and Running: Portable Distributions

Select a portable distribution if you'd like to avoid having to install other programs, or want to have an isolated executable file that doesn't interfere with other environments on your system.

Summary

Platform	Graphics API	File(s)	CUDA Version	Torch Version
MacOS	MPS	enfugue-server-0.2.2-macos-ventura-x86_64.tar.gz	N/A	2.1.0.dev20230720
Windows	CUDA	enfugue-server-0.2.2-win-cuda-x86_64.zip.001 enfugue-server-0.2.2-win-cuda-x86_64.zip.002	12.1.1	2.1.0.dev20230720
Windows	CUDA+TensorRT	enfugue-server-0.2.2-win-tensorrt-x86_64.zip.001 enfugue-server-0.2.2-win-tensorrt-x86_64.zip.002	11.7.1	1.13.1
Linux	CUDA	enfugue-server-0.2.2-manylinux-cuda-x86_64.tar.gz.0 enfugue-server-0.2.2-manylinux-cuda-x86_64.tar.gz.1 enfugue-server-0.2.2-manylinux-cuda-x86_64.tar.gz.2	12.1.1	2.1.0.dev20230720
Linux	CUDA+TensorRT	enfugue-server-0.2.2-manylinux-tensorrt-x86_64.tar.gz.0 enfugue-server-0.2.2-manylinux-tensorrt-x86_64.tar.gz.1 enfugue-server-0.2.2-manylinux-tensorrt-x86_64.tar.gz.2	11.7.1	1.13.1

Linux

First, decide which version you want - with or without TensorRT support. TensorRT requires a powerful, modern Nvidia GPU.
Then, download the appropriate manylinux files here, concatenate them and extract them. A console command to do that is:

cat enfugue-server-0.2.2* | tar -xvz

You are now ready to run the server with:

./enfugue-server/enfugue.sh

Press Ctrl+C to exit.

Windows

Download the win64 files here, and extract them using a program which allows extracting from multiple archives such as 7-Zip.

If you are using 7-Zip, you should not extract both files independently. If they are in the same directory when you unzip the first, 7-Zip will automatically unzip the second. The second file cannot be extracted on its own.

Locate the file enfugue-server.exe, and double-click it to run it. To exit, locate the icon in the bottom-right hand corner of your screen (the system tray) and right-click it, then select Quit.

MacOS

Download the the macos file here, then double-click it to extract the package. When you run the application using the command below, your Mac will warn you of running downloaded packages, and you will have to perform an administrator override to allow it to run - you will be prompted to do this. To avoid this, you can run an included command like so:

./enfugue-server/unquarantine.sh

This command finds all the files in the installation and removes the com.apple.quarantine xattr from the file. This does not require administrator privilege. After doing this (or if you will grant the override,) run the server with:

./enfugue-server/enfugue.sh

Note: while the MacOS packages are compiled on x86 machines, they are tested and designed for the new M1/M2 ARM machines thanks to Rosetta, Apple's machine code translation system.

Upgrading

To upgrade any distribution, download and extract the appropriate upgrade package on this release. Copy all files in the upgrade package into your Enfugue installation directory, overwriting any existing files.

Installing and Running: Conda

To install with the provided Conda environments, you need to install a version of Conda.

After installing Conda and configuring it so it is available to your shell or command-line, download one of the environment files depending on your platform and graphics API.

1. First, choose windows-, linux- or macos- based on your platform.
2. Then, choose your graphics API:
   • If you are on MacOS, you only have access to MPS.
   • If you have a powerful next-generation Nvidia GPU (3000 series and better with at least 12 GB of VRAM), use tensorrt for all of the capabilities of cuda with the added ability to compile TensorRT engines.
   • If you have any other Nvidia GPU or other CUDA-compatible device, select cuda.
   • Additional graphics APIs (rocm and directml) are being added and will be available soon.

Finally, using the file you downloaded, create your Conda environment:

conda env create -f <downloaded_file.yml>

You've now installed Enfugue and all dependencies. To run it, activate the environment and then run the installed binary.

conda activate enfugue
enfugue run

Installing and Running: Self-Managed Environment

If you would like to manage dependencies yourself, or want to install Enfugue into an environment to share with another Stable Diffusion UI, you can install enfugue via pip. This is the only method available for AMD GPU's at present.

pip install enfugue

If you are on Linux and want TensorRT support, execute:

pip install enfugue[tensorrt]

If you are on Windows and want TensorRT support, follow the steps detailed here.

Thank you!

---

This discussion was created from the release ENFUGUE Server v0.2.2.