About Olive support. #149
Replies: 36 comments 141 replies
-
|
I get an error. I will delete the comment when fixed, or where else to put it? The Commands dont fix it. import torch_directml |
Beta Was this translation helpful? Give feedback.
-
i got same error, can you describe what torch commands do you use? |
Beta Was this translation helpful? Give feedback.
-
|
open windows command line (Windows button -> cmd -> Enter)
and if that didn't work then type this if you have it somewhere else:
so if your path to repository is:
|
Beta Was this translation helpful? Give feedback.
-
|
it work, but got new error my system spec: |
Beta Was this translation helpful? Give feedback.
-
|
I think ImportError will not appear anymore. |
Beta Was this translation helpful? Give feedback.
-
|
lshqqytiger |
Beta Was this translation helpful? Give feedback.
-
|
Bruh the model conversion is confusing, I lost my models and outputs on their folders. Then even i use XUI to convert my models it still won't work because it requires opt_config.json. And then one model (majicmix reality) is working which i take from huggingface but giving only black squares. |
Beta Was this translation helpful? Give feedback.
-
|
Yes. It is confusing but it works fine for me. I updated it with detailed descriptions. |
Beta Was this translation helpful? Give feedback.
-
|
Thank you brother |
Beta Was this translation helpful? Give feedback.
-
|
Here are some errors that I encountered when using Olive:
console log
console log
console log
|
Beta Was this translation helpful? Give feedback.
-
|
im unable to generate image, driver issue? console log
|
Beta Was this translation helpful? Give feedback.
-
|
how do i switch between gpu? currently it using IGPU screenshoot
|
Beta Was this translation helpful? Give feedback.
-
Some same issues are reported before. There was something went wrong during DirectML's operation. The conditions or environments that error occurs are unknown. I think because it is using iGPU, but not certain.
Olive does not support selecting specific device yet. I will have a look about it. |
Beta Was this translation helpful? Give feedback.
-
|
Now I think you can specify which device will be used by adding |
Beta Was this translation helpful? Give feedback.
-
It works, but the graph in the task manager shows spikes (unstable) and it generate black image screenshoot
task manager
webui
|
Beta Was this translation helpful? Give feedback.
-
|
Ive seen multiple updates for Olive.py over the last weeks, could you post any changelogs or something? I am very interested in Olive, since its way better for VRAM. |
Beta Was this translation helpful? Give feedback.
-
|
been trying to do this for 3 days now.,, still no luck. cant really understand where does the "Checkpoint file name" pul from so i can put my current folder from that folder?
|
Beta Was this translation helpful? Give feedback.
-
|
You can optimize the checkpoint file (.ckpt or .safetensors) which is located in |
Beta Was this translation helpful? Give feedback.
-
|
thanks. Im getting this now: Stable diffusion model failed to load
|
Beta Was this translation helpful? Give feedback.
-
|
also its running on my CPU and not GPU? When starting webui it says this also: |
Beta Was this translation helpful? Give feedback.
-
Let me know commandline arguments you used when you got this error. |
Beta Was this translation helpful? Give feedback.
-
There's no problem. |
Beta Was this translation helpful? Give feedback.
-
|
hi @lshqqytiger after successful conversion and optimization generating ang image gives me this error: NotImplemented: [ONNXRuntimeError] : 9 : NOT_IMPLEMENTED : Failed to find kernel for GroupNorm(1) (node GroupNorm_0). Kernel not found |
Beta Was this translation helpful? Give feedback.
-
|
UNet is not seemed to be optimized well. Is there an error while optimization? Did the optimization finish well? |
Beta Was this translation helpful? Give feedback.
-
|
hi again yes it said optimization complete |
Beta Was this translation helpful? Give feedback.
-
|
where can i find the log? tried looking for it |
Beta Was this translation helpful? Give feedback.
-
|
i will try to optimize again will |
Beta Was this translation helpful? Give feedback.
-
|
Done optimizing again heres the log back with NOT_IMPLEMENTED again even Use floats if turned off: venv "G:\lshqqytiger2\stable-diffusion-webui-directml\venv\Scripts\Python.exe"
|
Beta Was this translation helpful? Give feedback.
-
|
hi @lshqqytiger what does this error mean? Creating ONNX pipeline... |
Beta Was this translation helpful? Give feedback.
-
|
Just ignore that error |
Beta Was this translation helpful? Give feedback.
-
|
Still having this unet Error: ERROR LOG: *** Error completing request ==================================================== COMPLETE LOG AFTER ONNX OPTIMIZATION: Downloading (…)ain/model_index.json: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████| 541/541 [00:00<00:00, 462kB/s] Optimizing text_encoder Optimizing unet Optimizing vae_decoder Optimizing vae_encoder Creating ONNX pipeline... |
Beta Was this translation helpful? Give feedback.
-
|
It happened to me too, it because i uncheck |
Beta Was this translation helpful? Give feedback.
-
|
thanks will try to use half float then |
Beta Was this translation helpful? Give feedback.
-
|
Getting this error when trying to optimize any model |
Beta Was this translation helpful? Give feedback.
-
|
Did you provide "VAE Source Model ID"? Leave it empty and try again. It is failing to optimize VAE. |
Beta Was this translation helpful? Give feedback.
-
|
Tried both leaving it empty or linking the huggingface repos. Same error every time. Also tried models with the baked in vae to the same conclusion. |
Beta Was this translation helpful? Give feedback.
-
|
i made a new repo, fresh install only argument was --olive still same error, i cannot get onnx to work anymore. |
Beta Was this translation helpful? Give feedback.
-
|
im so impressed with your work! Where can i donate to you? wanna send 10€ via paypal! |
Beta Was this translation helpful? Give feedback.
-
|
I'm seeing the same error message. I note that at the start of the console is Further down I get this error: Adding --skip-install resulted in: |
Beta Was this translation helpful? Give feedback.
-
|
I already wrote the solution of last error on my post. I recommend to create another virtual environment for Olive to avoid such errors. |
Beta Was this translation helpful? Give feedback.
-
|
Just out of curiosity, does this change something for us amd users? "https://www.amd.com/en/developer/rocm-hub/hip-sdk.html" |
Beta Was this translation helpful? Give feedback.
-
|
It does for developers not for users at this time. The HIP SDK for Windows is released, but there's no framework that supports (is built with) it. About PyTorch (which WebUI strongly depends on), MIOpen, one of the build-time dependency of it, does not support Windows yet. So we have to wait until PyTorch built with ROCm for Windows is released. |
Beta Was this translation helpful? Give feedback.
-
|
Finally. |
Beta Was this translation helpful? Give feedback.
-
|
I'm getting an import error, but it's when starting up with olive. I tried conda list and it says i'm actually on version 21.0 for accelerate, but everytime I run this, it says i'm on 18..? I did downgrade to torch 1.13.1 and still get the same thing. It is important to note the guide i followed at first was this "https://community.amd.com/t5/gaming/how-to-running-optimized-automatic1111-stable-diffusion-webui-on/ba-p/625585" When I get to step 4, right after I run 'python stable_diffusion.py --interactive --num_images 2' (that worked too for clarification) I then follow these steps & get the error above. 'Run the Automatic1111 WebUI with the Optimized Model Launch a new Anaconda/Miniconda terminal window as soon as I run webui.bat --onnx --backend directml Any help would be appreciated. thanks a million. edit: I added where i found instructions. |
Beta Was this translation helpful? Give feedback.
-
|
Remove |
Beta Was this translation helpful? Give feedback.
-
|
I had the same problem :) What I found to be happening was webui.bat was, for whatever reason, downgrading my accelerate package to v0.18 every time it ran. All I had to do was:
|
Beta Was this translation helpful? Give feedback.
-
|
I added --skip-install and it ran Auto1111 but now I get this & it won't load the optimized model. Stable diffusion model failed to load |
Beta Was this translation helpful? Give feedback.
-
|
remove
Sorry, just getting around to this one. It gives me something totally different. (base) C:\Users\Xero\stable-diffusion-webui-directml>conda activate Automatic1111 (Automatic1111) C:\Users\Xero\stable-diffusion-webui-directml>launch.py --oliveINCOMPATIBLE PYTHON VERSION This program is tested with 3.10.6 Python, but you have 3.11.4. You can download 3.10 Python from here: https://www.python.org/downloads/release/python-3106/ Alternatively, use a binary release of WebUI: https://github.com/AUTOMATIC1111/stable-diffusion-webui/releases Use --skip-python-version-check to suppress this warning.fatal: No names found, cannot describe anything. Update Edit: I uninstalled everything and reinstalled python 3.10.6 as it said, just to weed out everything possible... |
Beta Was this translation helpful? Give feedback.
-
|
Ok, I restarted everything, and I get all the way back to where I am running 'webui.bat --onnx --backend directml --skip-install' I ran the commands to make sure I have accelerate, and all seems well, so I tried 'launch.py --olive --onnx --backend directml --skip-install' I then tried 'launch.py --onnx --backend directml' (Automatic1111) C:\Users\Xero\stable-diffusion-webui-directml>launch.py --onnx --backend directml stderr: ERROR: Could not install packages due to an OSError: [WinError 5] Access is denied: 'C:\Users\Xero\AppData\Local\Programs\Python\Python310\Lib\site-packages\onnxruntime\capi\onnxruntime_providers_shared.dll' I am running the prompt as admin, i tried the --user prompt but that failed (and i also read it's not good to do it anyway. one thing at a time ... i guess... but did you all have this much trouble? i have terrible luck xD |
Beta Was this translation helpful? Give feedback.
-
|
Okay... I don't know how, why, when , who, what universe, and what kind of magic changed... But I can run it with 'python launch.py --onnx --backend directml --skip-install' now. I need to investigate and play with it further but it's working now. I did randomly get curious and typed 'pip install accelerate==0.20.3' and it said the below part (which makes no sense). ANd then it gave me that 18.0 error all over again... I don't understand, AT ALL. but I will continue to test and see what comes of it...
Thanks again! |
Beta Was this translation helpful? Give feedback.
-
|
Oh nice!, Dang it! I wanted to try is to see if I could replicate AMD claims of 9.9X performance on a 7900XTX. After wasting some number of hours trying, eventually gave up. (My experienced recorded in #240 I also had the DLL Error & noticed the requirements_version.txt file conflicts with all the messages printed during upgrade/install. When you say you used: 'python launch.py --onnx --backend directml --skip-install' , where do the command line arguments go? |
Beta Was this translation helpful? Give feedback.
-
I wrote it in a miniconda window to launch it. |
Beta Was this translation helpful? Give feedback.
-
|
Anaconda... I need to learn a bit more about Python world, I'm primarily a SQL guy at my day job. I noticed a new updated was just published that changes a few things: Perhaps I'll try to get Olive/Onnx going again. |
Beta Was this translation helpful? Give feedback.
-
|
Getting this error whenever I try to optimize a model |
Beta Was this translation helpful? Give feedback.
-
|
May be fixed in 9c78894 (at least the error was gone for me) |
Beta Was this translation helpful? Give feedback.
-
|
I previously created an olive onxx version of standard 1.5 sd model, and was able to successfully use it once I copied it to "ONNX-Olive" folder, now there are a few errors before the first run but it seems everything is working allright, can I keep using this or should I recreate from scratch ?
Also is there any way or some place for people to get pre converted onnx models ? It would be very convenient. |
Beta Was this translation helpful? Give feedback.
-
These warnings are ignorable. You don't need to recreate it. Recreating it won't solve them.
Huggingface will be appropriate, I guess? But the problem is that the resolution of the result image is fixed when the model is optimized. |
Beta Was this translation helpful? Give feedback.
-
|
Have you tried sdxl from the same examples / directml / stable-diffusion_xl , it is on the olive github. "AssertionError" `(venv) D:\olive\examples\directml\stable_diffusion_xl>python stable_diffusion_xl.py --optimize Optimizing vae_encoder Optimizing vae_decoder Optimizing unet |
Beta Was this translation helpful? Give feedback.
-
|
I merged SDXL optimization into this repository. But I haven't tested it because I don't have enough system memory to optimize it. |
Beta Was this translation helpful? Give feedback.
-
I'd be happy to optimize some SDXL models and share, but I haven't had any luck yet. I don't understand how to setup the optimize screens, I suppose - do I include the full folder path? Only the names? Basic info: Model I want to optimize is here: The model's name is I successfully used the Olive tab with the following config in the Olive -> Optimize Checkpoint:
Console printed: Now, moving on to the Optimize ONNX model tab, I can't get this step to work correctly. I have the config like this. I assume the OUTPUT of the previous step should become the INPUT of this step, and the OUTPUT of this step will be a new folder where the final model is stored?
At first pass, I get these errors: I see that it says torch_dytype=torch.float32 requires safety checkfer false. I uncheck Safety Checker and try again, resulting in the following: A bit of a guess, so I try again and uncheck "Use Half Floats". Resulting in the following error message: Thank you for any help you can provide. |
Beta Was this translation helpful? Give feedback.
-
|
That's right. So I recommend to name optimized models like |
Beta Was this translation helpful? Give feedback.
-
|
I solved everything but vae always gives error with whatever model I use, some of the models have huggingface pages but I don't know how to add "model id" correctly to vae tab in the converter. For example this model : "https://huggingface.co/Lykon/dreamshaper-8/tree/main" How do I correctly point it to these files or where should I put them ? |
Beta Was this translation helpful? Give feedback.
-
|
Leave empty |
Beta Was this translation helpful? Give feedback.
-
|
thats great @pw405 that you have it working |
Beta Was this translation helpful? Give feedback.
-
|
I did everything , finally gave up and tried to tinker with that post of amd's , that is directly using microsoft's olive github project and finally learned how to optimize any model on huggingface with directory structure (meaning not only safetensors file) Have successfully did 2 models and uploaded one back for other people to use. Also for general information some benchmarks with this model 👍 Last note, while 768x512 and 512x768 is way faster than normal with 768x768 everything falls apart. I don't know how to specify default image dimensions with olive tools but this is the same with the other model I converted too /dreamshaper 8/ So maybe one dimension at least needs to be 512 ... Photon_V1 Olive Optimized : https://huggingface.co/patientxtr/photon_v1_onnx |
Beta Was this translation helpful? Give feedback.
-
|
안녕하세요 @lshqqytiger, 올리브 사용할려고 하는데 에러가 계속 뜨네요. torch_directml_native 에러 뜨면 requirements_olive.txt가 이제 리포에 없는데 혹시 이것때문인가요? |
Beta Was this translation helpful? Give feedback.
-
|
requirements_olive.txt는 requirements_onnx.txt로 통합되어서 이제 필요하지 않습니다. |
Beta Was this translation helpful? Give feedback.
-
|
I get the following when trying to convert an sdxl model on a 6950xt: nloaded weights 0.0s.
text encoder 2 is checked. the vae is in the vae folder. Not sure what else to do. Thank you. |
Beta Was this translation helpful? Give feedback.
-
|
I think this should be an issue. Please open an issue about this and use |
Beta Was this translation helpful? Give feedback.
-
I was thinking about this earlier today actually. I tried with the suggested settings (text encoder 2 checked, safety check not checked) and get the same error. I've been crazy busy the last two weeks and haven't been able to spend much time with Olive. |
Beta Was this translation helpful? Give feedback.
-
|
Hi. Firstly, thank you for your work on this endeavour. Quick query, set up successful, conversion successful, generation time ca. 18.6its/s to 20.2its/s @ 768x768, x4 upscale, about 1.2 sec per generation. Issue is, batch size or batch count can't be increased without triggering the below; InvalidArgument: [ONNXRuntimeError] : 2 : INVALID_ARGUMENT : Got invalid dimensions for input: encoder_hidden_states for the following indices index: 0 Got: 4 Expected: 2 Please fix either the inputs or the model. Any advice, for a complete novice, would be most gratefully accepted. Thanks a million. |
Beta Was this translation helpful? Give feedback.
-
|
@hisham-hchowdhu The delightful strangeness continues! Batch size up to 8 works occasionally, sometimes restricted to batch size 1, but batch count will work up to a at least 50! I have no clue. When size won't increase beyond 1, full restart of the UI usually resolves the issue. Below is a grid (resized in PS to reduce file size, but you get the idea) from earlier, B size 2, B count 50 - insane! What's even more insane, however, is that this grid was generated in 3 min 19.2 secs. Incredible really. Yes, quality is not fantastic, and Hires.Fix has decided not to play ball, but regardless, progress is progress. Kind regards. @pw405 No joy with SDXL models unfortunately. Great to see the potential here, think I will stick with SD.Next until a 'permanent' resolution to running on AMD is released, my ignorance and lack of understanding mean anything that requires more than a 'read this, tweak that, press this', is well beyond my capabilities. The speed increase via this onnx/olive/SD webui is incredible, and it's superb to see AMD GPUs potentially being able to compete relatively well with Nvidia - we shall await progress with optimistic anticipation. All the best |
Beta Was this translation helpful? Give feedback.
-
|
@dermottroy , for SDXL, I was able to optimize using Olive scripts on AMD 7900XTX 24GB Vram: https://github.com/microsoft/Olive/tree/main/examples/directml/stable_diffusion_xl. not sure if @lshqqytiger integrated into this branch or not. |
Beta Was this translation helpful? Give feedback.
-
|
@hisham-hchowdhu Wizard, thanks a million, will give it a shot in the coming days. |
Beta Was this translation helpful? Give feedback.
-
I tried that too on my 7900 XT, a while ago. Are you getting any good images out of your optimized SDXL? Mine kept looking like this. |
Beta Was this translation helpful? Give feedback.
-
|
Maybe you can try lower the Token Merge Ratio or just disable it. |
Beta Was this translation helpful? Give feedback.
-
|
win11 6800xt Creating ONNX pipeline... |
Beta Was this translation helpful? Give feedback.
-
|
Same here. Was working previously. Will open Bug later. |
Beta Was this translation helpful? Give feedback.
-
|
Should be solved in latest commit. |
Beta Was this translation helpful? Give feedback.
-
|
pulled the latest and converted 2 models with no problems. thank you lshqqytiger! |
Beta Was this translation helpful? Give feedback.
-
|
yeah but still sdxl conversion wont work. and even the converted sdxl model which is already optimized, dont work. What settings do i have to use to get the optimization working? |
Beta Was this translation helpful? Give feedback.
-
|
I can't test sdxl because my gpu does not have enough memory to optimize it. If you open an issue with detailed log and traceback, I'll have a look. |
Beta Was this translation helpful? Give feedback.
-
|
Is it expected that running with --nowebui (running API), does not work? I didn't see it in the features list, so maybe so. I get an Internal Server Error 500 when "GET /sdapi/v1/samplers HTTP/1.1" is called. I can see models and options. Cannot "POST /sdapi/v1/txt2img HTTP/1.1" due to 404 Not Found. |
Beta Was this translation helpful? Give feedback.
-
|
I should have said. I ran the API with "webui.bat --onnx --backend directml --nowebui" Seems the problem is that no sampler list can be pulled via api, so none can be set when a request is sent, resulting in the error. |
Beta Was this translation helpful? Give feedback.
-
|
have you solved this problem? |
Beta Was this translation helpful? Give feedback.
-
|
After doing some digging and realizing you need to use ARG --onnx and not --olive as stated in the Announcement, Olive/ONNX is working well, thank you. However, I'm noticing that Generation Info is not being populated into the output images nor text file though it should be set in the Settings. Is this a known issue? |
Beta Was this translation helpful? Give feedback.
-
|
Currently, I'm focusing on integrating ONNX and Olive into SD.Next. If you want more updated version and nice compatibility/functionality, I recommend using the Because Olive does not support the latest version of PyTorch, you should downgrade them. |
Beta Was this translation helpful? Give feedback.
-
|
Sadly sd next has been flakey for me, I tried a new install today and the one time it created an image it took ages on my 7900xtx, iteration around 0.05 and that was using direct line. I'll try Olive sd next in a few days when I by have some time, anyway it came over as whinging, really it's not, without your hard work I wouldn't even be able to play with ai art, so thank you. |
Beta Was this translation helpful? Give feedback.
-
Man, I cloned the SDNext repo, removed all ARGS except DirectML and had zero issues (other than kinda crazy-long first-run dependency setup time), zero other tinkering. It's been flying on my existing SD models/embeddings on a 7900XT, seems nearly as fast as Olive from this branch. |
Beta Was this translation helpful? Give feedback.
-
|
Mine used to months ago, a little while before the change to sd next, then overnight it got bad and for a time wouldn't launched, so I went back to tigers's directml version, every now and again I try sd next, at least now it loads (sometimes to a usable state), its obviously something on my system, I just don't know what. |
Beta Was this translation helpful? Give feedback.
-
|
I'm using a 6900 XT and having some issues when I try to optimize SD-XL-base-1.0 on Windows 11, I get these error's: *** Arguments: ('sd_xl_base_1.0_0.9vae.safetensors', '', 'vae', 'stabilityai/stable-diffusion-xl-base-1.0', 'stabilityai/stable-diffusion-xl-base-1.0', 'runwayml/stable-diffusion-v1-5', '', 'vae', 'stable-diffusion-v1-5', 'stable-diffusion-v1-5', True, True, True, True, True, False, True, True, True, True, 'euler', True, 1024, False, '', '', '') {}
Traceback (most recent call last):
File "C:\Users\markv\git\stable-diffusion-webui-directml\modules\call_queue.py", line 57, in f
res = list(func(*args, **kwargs))
File "C:\Users\markv\git\stable-diffusion-webui-directml\modules\call_queue.py", line 36, in f
res = func(*args, **kwargs)
File "C:\Users\markv\git\stable-diffusion-webui-directml\modules\ui.py", line 2080, in optimize
return optimize_sdxl_from_ckpt(
File "C:\Users\markv\git\stable-diffusion-webui-directml\modules\sd_olive_ui.py", line 155, in optimize_sdxl_from_ckpt
optimize(
File "C:\Users\markv\git\stable-diffusion-webui-directml\modules\sd_olive_ui.py", line 352, in optimize
with footprints_file_path.open("r") as footprint_file:
File "C:\Users\markv\.conda\envs\Automatic1111\lib\pathlib.py", line 1119, in open
return self._accessor.open(self, mode, buffering, encoding, errors,
FileNotFoundError: [Errno 2] No such file or directory: 'footprints\\unet_gpu-dml_footprints.json' |
Beta Was this translation helpful? Give feedback.
-
|
Same Error for me with an 7900 XTX |
Beta Was this translation helpful? Give feedback.
-
|
Looks related to this: microsoft/Olive#517 |
Beta Was this translation helpful? Give feedback.
-
|
@lshqqytiger Brother, there are no PNG info in every image generations. And no textual inversion file (.pt) are being loaded from Embeddings folder. |
Beta Was this translation helpful? Give feedback.
-
|
About EXIF, I left it empty and I'm planning to implement it after every main features are done. I welcome PR. |
Beta Was this translation helpful? Give feedback.
-
|
Today, I refactored whole onnx/olive codes. Please follow this instruction. (I updated original discussion comment too) Instruction
※ If you want img2img, change Extra instruction for DirectML users.
|
Beta Was this translation helpful? Give feedback.
-
|
The models will be automatically optimized when you generate image. |
Beta Was this translation helpful? Give feedback.
-
|
I get this error with every model : I dont know if it comes from the type of models I choose ? The issue is exactly the same with whatever model I use, even the included base model |
Beta Was this translation helpful? Give feedback.
-
|
Same for me, this seems to be this issue here: pytorch/pytorch#120559 |
Beta Was this translation helpful? Give feedback.
-
|
got the same problem |
Beta Was this translation helpful? Give feedback.
-
|
me too, same problem |
Beta Was this translation helpful? Give feedback.
-
Just to be clear - I want absolutely NOTHING to do with Onnx/Olive, etc. We appreciate your efforts though. I just want to run SD with DirectML like I was in previous versions. Do these special instructions for DirectML still apply? |
Beta Was this translation helpful? Give feedback.
-
|
No. Just for who wants ONNX or Olive. |
Beta Was this translation helpful? Give feedback.
-
|
Oh.... so how the heck do I get the new version to work? |
Beta Was this translation helpful? Give feedback.
-
|
There's no changes for normal DirectML users. Just |
Beta Was this translation helpful? Give feedback.
-
Odd... OK, will try again after work. I tried last night and didn't capture the error, but keeping --use-directml is what originally resulted in me having the error here: |
Beta Was this translation helpful? Give feedback.
-
|
This did not work either. Back to missing optimum package error message. |
Beta Was this translation helpful? Give feedback.
-
|
If the issue persists after removing |
Beta Was this translation helpful? Give feedback.
-
|
directml mode run OK. but I get the errors: |
Beta Was this translation helpful? Give feedback.
-
|
who can help me? |
Beta Was this translation helpful? Give feedback.
-
|
python launch.py --use-directml --skip-version-check --medvram To create a public link, set How do i solve this problem? i am not very adept in python and also kinda new to github in general |
Beta Was this translation helpful? Give feedback.
-
|
delete venv folder always a good idea. |
Beta Was this translation helpful? Give feedback.
-
|
now it is p.s. i have tried reinstalling tensorflow |
Beta Was this translation helpful? Give feedback.
-
|
Hello everyone, i'd like to use SDXL, i sucessfully use 1.5 pruned provided model but not SDXL, is there clear steps anywhere on how to add models working with olive ? i can't figure it out ... Many thanks. |
Beta Was this translation helpful? Give feedback.
-
|
To be honest I never had luck with Olive in SDXL. However, I tried using Zluda and it is amazing! I don't even touch DirectML anymore. Skim the thread here for info about how to configure: #191 (comment) I found performance with Zluda to be way better than DirectML, and more features work. |
Beta Was this translation helpful? Give feedback.
and others
-
I added the Olive support in ab1ef3e.
Requirements
RAM (not VRAM): 32GB recommended.
VRAM: 8GB recommended. At least 6GB.
Features
--device-idInstruction
olive-aiand requirements.Use ONNX Runtime instead of PyTorch implementation.Execution Providerto proper one.Enable Olive.Olive models to process.※ If you want img2img, change
Diffusers pipelinetoONNX Stable Diffusion Img2Img.Extra instruction for DirectML users.
onnxruntime-directmlmanually via running these commands below.--use-cpu-torch.Execution ProvidertoDmlExecutionProvider.About LoRA
You should merge LoRAs into the model before the optimization.
Performance
512x512 on RX 5700 XT
Before: 1.5it/s
After Olive: 2.03it/s
512x512 on RX 7900 XTX
Before: 5it/s
With ONNX: 7it/s
Olive without static dims: 11it/s
Olive with static dims: 22it/s
FAQ
I can't find my execution provider under
Execution Provideroption.Reinstall onnxruntime and onnxruntime-... packages via running these commands below.
For example, if you want
DmlExecutionProvider, runBeta Was this translation helpful? Give feedback.
All reactions