Cat on a Petals

This plugin will enable the use of Petals Config as LLM in Cheshire Cat AI.

Prerequisites

To use this plugin you need to switch to the features/petals branch of my Cheshire Cat AI core repository that updates all the dependencies to work with Petals.

Usage

1. Load the plugin in Cheshire Cat AI.
2. Go to the settings page.
3. Click on "Configure" under Large Language Model section.
4. Select the "Petals" model.
5. Insert the desired model
6. Insert the huggingface token
7. Save your settings.