Unload model after inference (?)

[crexlight]

I have Telegram bot, that receives link to YouTube audio and changing acapella voice to model using SVC, then send audio to user.
I do inferencing like this:

from so_vits_svc_fork.inference.main import infer

# ...

infer(input_path=input_path, output_path=output_path, model_path=model_path, config_path=config_path, recursive=False, auto_predict_f0=False, transpose=chosen_pitch, f0_method='harvest', chunk_seconds=20.0, device='cuda', speaker=0)

bot.send_audio(user_id, open(f"{user_id}_infered.mp3", "rb"), performer=chosen_model, timeout=60, title='@...Bot')

After first inference, SVC using GPU like 1.5-2 GB, and i do not have enough GPU memory (I have only 4GB) for audio separation (acapella, instrumental), audio separation just getting very slower, than with full GPU memory.
Maybe i do not understand something? Because i see, that SVC removing model/cache after inference, but GPU is 1.5-2 GB anyway. Thanks for any help.

[crexlight]

Example, but with Voice Message, not YouTube and audio separation.

[34j]

I don't know what to do any more, would calling torch.cuda.empty_cache() everywhere help?

[crexlight]

No

[34j]

Then you can use cpu inference, it's only 2x slower (probably)