Distributed GPU serving API petals.ml integration

[sapcode]

Thank you very much for your work and publishing!!
@SigGravitas
could you please look into this http://petals.ml and this example https://github.com/bigscience-workshop/petals/blob/main/examples/prompt-tuning-personachat.ipynb
If you can integrate that, it will allow distributed GPU serving community free models without any paid API needs...
Here is a how to spawn own GPU swarm https://github.com/bigscience-workshop/petals/wiki/Launch-your-own-swarm

[lc0rp]

Issue - #4479

[Wladastic]

@p-i- very interesting topic!

[isaaclepes]

The site is now at https://petals.dev
Recently, CodeLlama became available on Petals.
https://health.petals.dev/