You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I've tried to use compile=True in load function, it is super slow to generate results, and I didn't see any GPU utilizition with nvidia-smi like below (a liitle memory ocuupied like 1GB):
after I commented out the code block below, GPU utilization turns normal, and result is generated much much faster:
I cant understand why the torch.compile function didn't work well. Does anyone know why?
The text was updated successfully, but these errors were encountered:
The compile only shows its effect when you run a generating process continuously with the same input shape. If you only run it once, it will slow down the generation.
pytorch version: 2.3.1+cu121
I've tried to use
![企业微信截图_17194899582803](https://private-user-images.githubusercontent.com/9158462/343763153-0a10755f-b254-4b28-9c95-5c58f69db372.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MTk3NDQ3NzYsIm5iZiI6MTcxOTc0NDQ3NiwicGF0aCI6Ii85MTU4NDYyLzM0Mzc2MzE1My0wYTEwNzU1Zi1iMjU0LTRiMjgtOWM5NS01YzU4ZjY5ZGIzNzIucG5nP1gtQW16LUFsZ29yaXRobT1BV1M0LUhNQUMtU0hBMjU2JlgtQW16LUNyZWRlbnRpYWw9QUtJQVZDT0RZTFNBNTNQUUs0WkElMkYyMDI0MDYzMCUyRnVzLWVhc3QtMSUyRnMzJTJGYXdzNF9yZXF1ZXN0JlgtQW16LURhdGU9MjAyNDA2MzBUMTA0NzU2WiZYLUFtei1FeHBpcmVzPTMwMCZYLUFtei1TaWduYXR1cmU9Y2I5NGMzMGUxZmU3ZGYwMmQzOGM3YzQ3NzJmOTI4ZjNlYzVhMmZkYzZmMmZlNTc2YzRkYTIxNjYzMDJhOGU5ZSZYLUFtei1TaWduZWRIZWFkZXJzPWhvc3QmYWN0b3JfaWQ9MCZrZXlfaWQ9MCZyZXBvX2lkPTAifQ.vhe52UXPTz6AGwZjo5ey8t94dpkejr5qqrfoj93uCZs)
compile=True
inload
function, it is super slow to generate results, and I didn't see any GPU utilizition withnvidia-smi
like below (a liitle memory ocuupied like 1GB):after I commented out the code block below, GPU utilization turns normal, and result is generated much much faster:
![企业微信截图_17194900322602](https://private-user-images.githubusercontent.com/9158462/343763522-d023b00f-bf35-4427-8b8e-d51e1d288f6e.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MTk3NDQ3NzYsIm5iZiI6MTcxOTc0NDQ3NiwicGF0aCI6Ii85MTU4NDYyLzM0Mzc2MzUyMi1kMDIzYjAwZi1iZjM1LTQ0MjctOGI4ZS1kNTFlMWQyODhmNmUucG5nP1gtQW16LUFsZ29yaXRobT1BV1M0LUhNQUMtU0hBMjU2JlgtQW16LUNyZWRlbnRpYWw9QUtJQVZDT0RZTFNBNTNQUUs0WkElMkYyMDI0MDYzMCUyRnVzLWVhc3QtMSUyRnMzJTJGYXdzNF9yZXF1ZXN0JlgtQW16LURhdGU9MjAyNDA2MzBUMTA0NzU2WiZYLUFtei1FeHBpcmVzPTMwMCZYLUFtei1TaWduYXR1cmU9ZWI0Y2MyN2Q0OGRmMzQ3MWRhZGY3MGZkMDg4MGYyMjIwYzgwNmRmMmQwOWMxZDI0NTk2MGM1NGZlZmIxMGQzMyZYLUFtei1TaWduZWRIZWFkZXJzPWhvc3QmYWN0b3JfaWQ9MCZrZXlfaWQ9MCZyZXBvX2lkPTAifQ.Y1561KZQ4ceIKkG-H614rhbk2I9aDmgwkOn2WIBTot0)
![企业微信截图_17194900872413](https://private-user-images.githubusercontent.com/9158462/343763801-6284a604-821a-4f83-96f5-37a383ce6b3e.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MTk3NDQ3NzYsIm5iZiI6MTcxOTc0NDQ3NiwicGF0aCI6Ii85MTU4NDYyLzM0Mzc2MzgwMS02Mjg0YTYwNC04MjFhLTRmODMtOTZmNS0zN2EzODNjZTZiM2UucG5nP1gtQW16LUFsZ29yaXRobT1BV1M0LUhNQUMtU0hBMjU2JlgtQW16LUNyZWRlbnRpYWw9QUtJQVZDT0RZTFNBNTNQUUs0WkElMkYyMDI0MDYzMCUyRnVzLWVhc3QtMSUyRnMzJTJGYXdzNF9yZXF1ZXN0JlgtQW16LURhdGU9MjAyNDA2MzBUMTA0NzU2WiZYLUFtei1FeHBpcmVzPTMwMCZYLUFtei1TaWduYXR1cmU9NTg0NGViMWQwNGVmODE3NzFjN2JkZGNlNTQ4NGVmYzI5YzY0YzAyMGUwMWRlZmUyMGU1NTFlMWNiMGM3ZDM1MSZYLUFtei1TaWduZWRIZWFkZXJzPWhvc3QmYWN0b3JfaWQ9MCZrZXlfaWQ9MCZyZXBvX2lkPTAifQ.iZjktC3sRnr18R7FxGgBtNlvKX_8r52lcg0DfUbmH58)
I cant understand why the torch.compile function didn't work well. Does anyone know why?
The text was updated successfully, but these errors were encountered: