applications for not only LLMs

[lindakasabian]

Hi, this is an impressive work and a working alternative to LoRAs. However, can it be used to train selected layers of other models, not only transformer architecture? CNNs or pure Embedding and Gated MLPs?

[ehartford]

probably - but we have not experimented on that. Great opportunity for a future / related work