Feat/aws bedrock (#34)

This PR adds support for Amazon Bedrock (using Boto3) which gives users access to an additional 15 LLM's.
The LLM's being added include:

AI21 Labs / Jurassic-2 Ultra
AI21 Labs / Jurassic-2 Mid
Amazon / Titan Text Lite
Amazon / Titan Text Express
Amazon / Titan Text Embeddings
Anthropic / Claude 2.1
Anthropic / Claude 2.0
Anthropic / Claude 1.3
Anthropic / Claude Instant
Cohere / Command
Cohere / Command Light
Cohere / Embed - English
Cohere / Embed - Multilingual
Meta / Llama-2-13b-chat
Meta / Llama-2-70b-chat

Author: alvxck

.envrc.example

@@ -2,4 +2,8 @@ export OPENAI_API_KEY=
 export COHERE_API_KEY=
 export EXTRA_RUN_OPTIONS=--reload
 
+export AWS_REGION=
+export AWS_PUBLIC_ACCESS_KEY=
+export AWS_PRIVATE_ACCESS_KEY=
+
 export DATABASE_URL=postgresql://postgres:postgres@postgres:5432/llm_gateway

README.md

@@ -12,10 +12,35 @@ Per OpenAI's non-API consumer products [data usage policy](https://help.openai.c
 
 Use `llm-gateway` to interact with OpenAI in a safe manner. The gateway also recreates the ChatGPT frontend using OpenAI's `/ChatCompletion` endpoint to keep all communication within the API.
 
+## 📦 Supported Models
+
+| Provider      | Model                     |
+|:--------------:|:-------------------------:|
+| OpenAI        | GPT 3.5 Turbo             |
+| OpenAI        | GPT 3.5 Turbo 16k         |
+| OpenAI        | GPT 4                     |
+| AI21 Labs     | Jurassic-2 Ultra          |
+| AI21 Labs     | Jurassic-2 Mid            |
+| Amazon        | Titan Text Lite           |
+| Amazon        | Titan Text Express        |
+| Amazon        | Titan Text Embeddings     |
+| Anthropic     | Claude 2.1                |
+| Anthropic     | Claude 2.0                |
+| Anthropic     | Claude 1.3                |
+| Anthropic     | Claude Instant            |
+| Cohere        | Command                   |
+| Cohere        | Command Light             |
+| Cohere        | Embed - English           |
+| Cohere        | Embed - Multilingual      |
+| Meta          | Llama-2-13b-chat          |
+| Meta          | Llama-2-70b-chat          |
+
 ## ⚒️ Usage
 
 The provider's API key needs to be saved as an environment variable (see setup further down). If you are communicating with OpenAI, set `OPENAI_API_KEY`.
 
+For step-by-step setup instructions with Cohere, OpenAI, and AWS Bedrock, click [here](llm_gateway/README.md).
+
 ### API Usage
 [OpenAI] Example cURL to `/completion` endpoint:
 ```

fixtures/init.sql

@@ -21,3 +21,15 @@ CREATE TABLE cohere_requests(
     created_at TIMESTAMP WITHOUT TIME ZONE,
     cohere_endpoint VARCHAR
 );
+
+CREATE TABLE awsbedrock_requests(
+    id serial primary key,
+    user_input VARCHAR,
+    user_email VARCHAR,
+    awsbedrock_response JSON,
+    awsbedrock_model VARCHAR,
+    temperature FLOAT,
+    created_at TIMESTAMP WITHOUT TIME ZONE,
+    awsbedrock_endpoint VARCHAR
+    extras JSON
+);

front_end/src/app/components/SettingsDialog/settingsDialog.tsx

@@ -122,6 +122,15 @@ export const ModelSettingsDialog: React.FC<Props> = ({
                 </b>
                 {modelChoices[selectedOption].maxTokensLimit} tokens <br />
               </li>
+              <li>
+                <b>
+                  <span role="img" aria-label="blink">
+                    ⚙️
+                  </span>{' '}
+                  Requirements:{' '}
+                </b>
+                {modelChoices[selectedOption].requirements} <br />
+              </li>
             </ul>
           </details>
           <details>

front_end/src/app/interfaces.tsx

@@ -46,7 +46,8 @@ export interface ModelInfo {
   advanceMetadata: ModelMetadata;
   supportFileUpload: boolean;
   initialPrompt: Message[];
-  requestBody: (req: IRequestBody) => OpenAIRequestBody | CohereRequestBody;
+  requirements: string,
+  requestBody: (req: IRequestBody) => OpenAIRequestBody | CohereRequestBody | AWSBedrockRequestBody;
   responseHandler: (res: any) => string;
 }
 
@@ -63,10 +64,24 @@ export interface CohereRequestBody {
   model: string;
 }
 
+export interface AWSBedrockRequestBody {
+  model: string;
+  temperature: number;
+  max_tokens: number;
+  prompt?: string;
+  embedding_texts?: string[];
+  instructions?: string;
+  model_kwargs?: ModelConfig;
+}
+
 interface ModelMetadata {
   [key: string]: string;
 }
 
+interface ModelConfig {
+  [key: string]: any;
+}
+
 export interface Models {
   [key: string]: ModelInfo;
 }