RAM offloading like AutoGPTQ?

[manyotherfunctions]

Are there any plans to add the ability to split the model between VRAM and system RAM like AutoGPTQ does? For example the oobabooga webui, through AutoGPTQ, lets you load even a 65B parameter model on a 8GB VRAM GPU, where only 1GB is loaded in VRAM and the rest in RAM

[turboderp]

This would be horrendously slow without a dedicated CPU inference engine. And Llama.cpp is already very well optimized for this use case.