There should be a config to run this fork on 8GBVram faster with un-optimzed mode.

[mhnoni]

Hi, so this fork is great for small VRam, with only 2.4 GB, with turbo it takes 3.4GB, but its still slow vs un-optimized of other forks like hlky 49sec here vs hlky fork 37sec that takes 6.4GB. So it would be cool if there is a config to enable un-optimized for people who owns > 8Gb VRam.

I used my gtx 1070 8GB here for testing.

Thanks!

[bitRAKE]

The 1070 has faster FP32 (single-precision) floating-point performance than FP16 (half-precision) performance. By using the --precision full command line option you can take advantage of this older performance profile.

I'm in the same boat with my 1080ti. So, I understand what you are experiencing.

More modern cards have been designed with greater FP16 performance.

This wiki table also shows the legacy performance of the 10 Series cards:
https://en.wikipedia.org/wiki/GeForce_10_series#GeForce_10_(10xx)_series_for_desktops

Hard to believe it's only been five years. 😃

[bitRAKE]

Without any commands it takes 21 sec. But why can't I do a batch size of 8? VRAM ranges from 3.7 to 6.8, but a batch of 2 it jumps up to 9.8GB -- something is broken there. Batch size of 3 jumps to 10.1GB. Batch of four -- forget it!

This repo, I can do a batch of 25+.

My thinking is that the default of autocast works better in hlky\webui because of removal of explicit type casting on model use. On our machines it will default to FP32, and on modern machines default to FP16. This is because the memory use largely stays the same - regardless of parameters used.

My largest use case is generating images when I'm doing something else on the machine.

[bitRAKE]

Instead of hlky's broken repo use this one:
https://github.com/AUTOMATIC1111/stable-diffusion-webui

Note only better performance, but it correctly implements multi-headed attention to split the image. So, rendering larger dimensions works well with small VRAM. Best I can figure is hlky is copying features from this repo or some such craziness.

[mhnoni]

Instead of hlky's broken repo use this one: https://github.com/AUTOMATIC1111/stable-diffusion-webui

Note only better performance, but it correctly implements multi-headed attention to split the image. So, rendering larger dimensions works well with small VRAM. Best I can figure is hlky is copying features from this repo or some such craziness.

I have right now 4 forks on my hdd lol, performance with [AUTOMATIC1111 was the same as hlky, but I'm just not sure why precision full is slower on my side when it's faster on your side.

[bitRAKE]

Could be a slower memory bus on the 1070. I see some of the effects of transferring larger data to/from GFX card myself - precision full is actually 10x faster on the 1080ti, but because it involves larger data transfers most of that acceleration is lost (unless it stays on the GFX card)!

[mhnoni]

Could be a slower memory bus on the 1070. I see some of the effects of transferring larger data to/from GFX card myself - precision full is actually 10x faster on the 1080ti, but because it involves larger data transfers most of that acceleration is lost (unless it stays on the GFX card)!

Maybe I'm not doing it correctly, can you tell me how you enable the precision full with basujindal fork?
edit: nvm, you already told me in the first comment.