Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

ControlNet and Kohya HRFix interop support #95

Open
1 task done
RedDestiny opened this issue Feb 7, 2024 · 1 comment
Open
1 task done

ControlNet and Kohya HRFix interop support #95

RedDestiny opened this issue Feb 7, 2024 · 1 comment
Labels
enhancement New feature or request

Comments

@RedDestiny
Copy link

Is there an existing issue for this?

  • I have searched the existing issues and checked the recent builds/commits

What would your feature do ?

我经常使用高分辨率的图生图，会使用CN的lineArt和Tile两个模型的样子。而且我发现，小分辨率的时候（1000px），SD生成的图，光影和结构更加正确、优秀，超过2000px的时候就会变得很平、很容易出错的样子，无论是结构上，还是光影。

然后我使用了内置的Kohya HRFix，它的工作机制就是先缩放分辨率迭代出正确的结构，然后再提升会原本的分辨率，而且可以控制参与的步骤的样子。在文生图（1500x1500，甚至2000x2000以上），能得到结构正确且非常优秀的画面。
我就想着试着搭配CN模型一起，进一步控制画面的样子，然后就出现了报错的样子，在Kohya HRFix起作用的阶段中，CN就会出现
warning control could not be applied torch.Size([2, 1280, 11, 8]) torch.Size([2, 1280, 32, 22])
warning control could not be applied torch.Size([2, 1280, 11, 8]) torch.Size([2, 1280, 32, 22])
warning control could not be applied torch.Size([2, 1280, 11, 8]) torch.Size([2, 1280, 32, 22])
warning control could not be applied torch.Size([2, 1280, 22, 15]) torch.Size([2, 1280, 64, 43])
warning control could not be applied torch.Size([2, 1280, 22, 15]) torch.Size([2, 1280, 64, 43])
warning control could not be applied torch.Size([2, 640, 22, 15]) torch.Size([2, 640, 64, 43])
warning control could not be applied torch.Size([2, 640, 43, 29]) torch.Size([2, 640, 128, 86])
warning control could not be applied torch.Size([2, 640, 43, 29]) torch.Size([2, 640, 128, 86])
然后并不起作用的样子。因为这个分辨率和CN设定的原本的分辨率不匹配的样子。

所以我特别希望CN也能自动匹配和Kohya HRFix的流程。

Proposed workflow

  1. 到图生图/文生图，写好参数和设置
  2. 启用Kohya HRFix Integrated，设置好参数
  3. 启用CN，设置好参数和模型
  4. CN能和HRFix一起正常运转

Additional information

No response
@RedDestiny RedDestiny added the enhancement New feature or request label Feb 7, 2024
@catboxanon catboxanon changed the title [功能请求]：希望CN支持和Kohya HRFix Integrated，一起启用。 ControlNet and Kohya HRFix interop support Feb 17, 2024
@Manchovies
Copy link

Also running into this. Seems to be Kohya HRFix that is messing this up. Happens on default fresh install as well.

2024-03-13 16:36:15,436 - ControlNet - INFO - ControlNet Input Mode: InputMode.SIMPLE 2024-03-13 16:36:15,437 - ControlNet - INFO - Using preprocessor: canny 2024-03-13 16:36:15,437 - ControlNet - INFO - preprocessor resolution = 1024 2024-03-13 16:36:15,942 - ControlNet - INFO - Current ControlNet ControlNetPatcher: D:\Users\Spencer\Downloads\StabilityMatrix-win-x64\Data\Packages\Stable Diffusion WebUI Forge New\models\ControlNet\ControlNet\TencentARC-t2i-adapter-canny-sdxl.safetensors NeverOOM Enabled for VAE (always tiled) 2024-03-13 16:36:15,951 - ControlNet - INFO - ControlNet Method canny patched. To load target model SDXL Begin to load 1 model Reuse 1 loaded models [Memory Management] Current Free GPU Memory (MB) = 3321.5380859375 [Memory Management] Model Memory (MB) = 0.0 [Memory Management] Minimal Inference Memory (MB) = 1024.0 [Memory Management] Estimated Remaining GPU Memory (MB) = 2297.5380859375 Moving model(s) has taken 0.04 seconds 0%| | 0/30 [00:00<?, ?it/s]warning control could not be applied torch.Size([2, 320, 36, 63]) torch.Size([1, 320, 72, 126]) warning control could not be applied torch.Size([2, 640, 36, 63]) torch.Size([1, 640, 72, 126]) warning control could not be applied torch.Size([2, 1280, 18, 32]) torch.Size([1, 1280, 36, 63]) warning control could not be applied torch.Size([2, 1280, 18, 32]) torch.Size([1, 1280, 36, 63]) warning control could not be applied torch.Size([1, 320, 36, 63]) torch.Size([1, 320, 72, 126]) warning control could not be applied torch.Size([1, 640, 36, 63]) torch.Size([1, 640, 72, 126]) warning control could not be applied torch.Size([1, 1280, 18, 32]) torch.Size([1, 1280, 36, 63]) warning control could not be applied torch.Size([1, 1280, 18, 32]) torch.Size([1, 1280, 36, 63]) 3%|▎ | 1/30 [00:00<00:28, 1.03it/s]warning control could not be applied torch.Size([2, 320, 36, 63]) torch.Size([1, 320, 72, 126]) warning control could not be applied torch.Size([2, 640, 36, 63]) torch.Size([1, 640, 72, 126]) warning control could not be applied torch.Size([2, 1280, 18, 32]) torch.Size([1, 1280, 36, 63]) warning control could not be applied torch.Size([2, 1280, 18, 32]) torch.Size([1, 1280, 36, 63]) warning control could not be applied torch.Size([1, 320, 36, 63]) torch.Size([1, 320, 72, 126]) warning control could not be applied torch.Size([1, 640, 36, 63]) torch.Size([1, 640, 72, 126]) warning control could not be applied torch.Size([1, 1280, 18, 32]) torch.Size([1, 1280, 36, 63]) warning control could not be applied torch.Size([1, 1280, 18, 32]) torch.Size([1, 1280, 36, 63]) 7%|▋ | 2/30 [00:01<00:24, 1.13it/s]warning control could not be applied torch.Size([2, 320, 36, 63]) torch.Size([1, 320, 72, 126]) warning control could not be applied torch.Size([2, 640, 36, 63]) torch.Size([1, 640, 72, 126]) warning control could not be applied torch.Size([2, 1280, 18, 32]) torch.Size([1, 1280, 36, 63]) warning control could not be applied torch.Size([2, 1280, 18, 32]) torch.Size([1, 1280, 36, 63]) warning control could not be applied torch.Size([1, 320, 36, 63]) torch.Size([1, 320, 72, 126]) warning control could not be applied torch.Size([1, 640, 36, 63]) torch.Size([1, 640, 72, 126]) warning control could not be applied torch.Size([1, 1280, 18, 32]) torch.Size([1, 1280, 36, 63]) warning control could not be applied torch.Size([1, 1280, 18, 32]) torch.Size([1, 1280, 36, 63]) 10%|█ | 3/30 [00:02<00:22, 1.19it/s]warning control could not be applied torch.Size([2, 320, 36, 63]) torch.Size([1, 320, 72, 126]) warning control could not be applied torch.Size([2, 640, 36, 63]) torch.Size([1, 640, 72, 126]) warning control could not be applied torch.Size([2, 1280, 18, 32]) torch.Size([1, 1280, 36, 63]) warning control could not be applied torch.Size([2, 1280, 18, 32]) torch.Size([1, 1280, 36, 63]) warning control could not be applied torch.Size([1, 320, 36, 63]) torch.Size([1, 320, 72, 126]) warning control could not be applied torch.Size([1, 640, 36, 63]) torch.Size([1, 640, 72, 126]) warning control could not be applied torch.Size([1, 1280, 18, 32]) torch.Size([1, 1280, 36, 63]) warning control could not be applied torch.Size([1, 1280, 18, 32]) torch.Size([1, 1280, 36, 63]) 13%|█▎ | 4/30 [00:03<00:21, 1.23it/s]warning control could not be applied torch.Size([2, 320, 36, 63]) torch.Size([1, 320, 72, 126]) warning control could not be applied torch.Size([2, 640, 36, 63]) torch.Size([1, 640, 72, 126]) warning control could not be applied torch.Size([2, 1280, 18, 32]) torch.Size([1, 1280, 36, 63]) warning control could not be applied torch.Size([2, 1280, 18, 32]) torch.Size([1, 1280, 36, 63]) warning control could not be applied torch.Size([1, 320, 36, 63]) torch.Size([1, 320, 72, 126]) warning control could not be applied torch.Size([1, 640, 36, 63]) torch.Size([1, 640, 72, 126]) warning control could not be applied torch.Size([1, 1280, 18, 32]) torch.Size([1, 1280, 36, 63]) warning control could not be applied torch.Size([1, 1280, 18, 32]) torch.Size([1, 1280, 36, 63]) 17%|█▋ | 5/30 [00:04<00:20, 1.25it/s]warning control could not be applied torch.Size([2, 320, 36, 63]) torch.Size([1, 320, 72, 126]) warning control could not be applied torch.Size([2, 640, 36, 63]) torch.Size([1, 640, 72, 126]) warning control could not be applied torch.Size([2, 1280, 18, 32]) torch.Size([1, 1280, 36, 63]) warning control could not be applied torch.Size([2, 1280, 18, 32]) torch.Size([1, 1280, 36, 63]) warning control could not be applied torch.Size([1, 320, 36, 63]) torch.Size([1, 320, 72, 126]) warning control could not be applied torch.Size([1, 640, 36, 63]) torch.Size([1, 640, 72, 126]) warning control could not be applied torch.Size([1, 1280, 18, 32]) torch.Size([1, 1280, 36, 63]) warning control could not be applied torch.Size([1, 1280, 18, 32]) torch.Size([1, 1280, 36, 63]) 20%|██ | 6/30 [00:04<00:19, 1.26it/s]warning control could not be applied torch.Size([2, 320, 36, 63]) torch.Size([1, 320, 72, 126]) warning control could not be applied torch.Size([2, 640, 36, 63]) torch.Size([1, 640, 72, 126]) warning control could not be applied torch.Size([2, 1280, 18, 32]) torch.Size([1, 1280, 36, 63]) warning control could not be applied torch.Size([2, 1280, 18, 32]) torch.Size([1, 1280, 36, 63]) warning control could not be applied torch.Size([1, 320, 36, 63]) torch.Size([1, 320, 72, 126]) warning control could not be applied torch.Size([1, 640, 36, 63]) torch.Size([1, 640, 72, 126]) warning control could not be applied torch.Size([1, 1280, 18, 32]) torch.Size([1, 1280, 36, 63]) warning control could not be applied torch.Size([1, 1280, 18, 32]) torch.Size([1, 1280, 36, 63]) 23%|██▎ | 7/30 [00:05<00:18, 1.26it/s]warning control could not be applied torch.Size([2, 320, 36, 63]) torch.Size([1, 320, 72, 126]) warning control could not be applied torch.Size([2, 640, 36, 63]) torch.Size([1, 640, 72, 126]) warning control could not be applied torch.Size([2, 1280, 18, 32]) torch.Size([1, 1280, 36, 63]) warning control could not be applied torch.Size([2, 1280, 18, 32]) torch.Size([1, 1280, 36, 63]) warning control could not be applied torch.Size([1, 320, 36, 63]) torch.Size([1, 320, 72, 126]) warning control could not be applied torch.Size([1, 640, 36, 63]) torch.Size([1, 640, 72, 126]) warning control could not be applied torch.Size([1, 1280, 18, 32]) torch.Size([1, 1280, 36, 63]) warning control could not be applied torch.Size([1, 1280, 18, 32]) torch.Size([1, 1280, 36, 63]) 27%|██▋ | 8/30 [00:06<00:17, 1.27it/s]warning control could not be applied torch.Size([2, 320, 36, 63]) torch.Size([1, 320, 72, 126]) warning control could not be applied torch.Size([2, 640, 36, 63]) torch.Size([1, 640, 72, 126]) warning control could not be applied torch.Size([2, 1280, 18, 32]) torch.Size([1, 1280, 36, 63]) warning control could not be applied torch.Size([2, 1280, 18, 32]) torch.Size([1, 1280, 36, 63]) warning control could not be applied torch.Size([1, 320, 36, 63]) torch.Size([1, 320, 72, 126]) warning control could not be applied torch.Size([1, 640, 36, 63]) torch.Size([1, 640, 72, 126]) warning control could not be applied torch.Size([1, 1280, 18, 32]) torch.Size([1, 1280, 36, 63]) warning control could not be applied torch.Size([1, 1280, 18, 32]) torch.Size([1, 1280, 36, 63]) 30%|███ | 9/30 [00:07<00:16, 1.27it/s]warning control could not be applied torch.Size([2, 320, 36, 63]) torch.Size([1, 320, 72, 126]) warning control could not be applied torch.Size([2, 640, 36, 63]) torch.Size([1, 640, 72, 126]) warning control could not be applied torch.Size([2, 1280, 18, 32]) torch.Size([1, 1280, 36, 63]) warning control could not be applied torch.Size([2, 1280, 18, 32]) torch.Size([1, 1280, 36, 63]) warning control could not be applied torch.Size([1, 320, 36, 63]) torch.Size([1, 320, 72, 126]) warning control could not be applied torch.Size([1, 640, 36, 63]) torch.Size([1, 640, 72, 126]) warning control could not be applied torch.Size([1, 1280, 18, 32]) torch.Size([1, 1280, 36, 63]) warning control could not be applied torch.Size([1, 1280, 18, 32]) torch.Size([1, 1280, 36, 63]) 33%|███▎ | 10/30 [00:08<00:15, 1.27it/s]warning control could not be applied torch.Size([2, 320, 36, 63]) torch.Size([1, 320, 72, 126]) warning control could not be applied torch.Size([2, 640, 36, 63]) torch.Size([1, 640, 72, 126]) warning control could not be applied torch.Size([2, 1280, 18, 32]) torch.Size([1, 1280, 36, 63]) warning control could not be applied torch.Size([2, 1280, 18, 32]) torch.Size([1, 1280, 36, 63]) warning control could not be applied torch.Size([1, 320, 36, 63]) torch.Size([1, 320, 72, 126]) warning control could not be applied torch.Size([1, 640, 36, 63]) torch.Size([1, 640, 72, 126]) warning control could not be applied torch.Size([1, 1280, 18, 32]) torch.Size([1, 1280, 36, 63]) warning control could not be applied torch.Size([1, 1280, 18, 32]) torch.Size([1, 1280, 36, 63])

This was referenced Mar 14, 2024
Closed
Closed
@jetjodh jetjodh mentioned this issue Apr 2, 2024
Closed
4 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

add support for Hrfix and CN jetjodh/stable-diffusion-webui-forge
2 participants
@RedDestiny @Manchovies