Amazon Bedrock Provider

content/providers/01-ai-sdk-providers/80-amazon-bedrock.mdx

@@ -0,0 +1,164 @@
+---
+title: Amazon Bedrock
+description: Learn how to use the Amazon Bedrock provider.
+---
+
+# Amazon Bedrock Provider
+
+The Amazon Bedrock provider for the [Vercel AI SDK](https://sdk.vercel.ai/docs) contains language model support for the [Amazon Bedrock](https://aws.amazon.com/bedrock) APIs.
+
+## Prerequisites
+
+Access to Amazon Bedrock foundation models isn't granted by default. In order to gain access to a foundation model, an IAM user with sufficient permissions needs to request access to it through the console. Once access is provided to a model, it is available for all users in the account.
+
+See the [Model Access Docs](https://docs.aws.amazon.com/bedrock/latest/userguide/model-access.html) for more information.
+
+## Authentication
+
+**Step 1: Creating AWS Access Key and Secret Key**
+
+To get started, you'll need to create an AWS access key and secret key. Here's how:
+
+**Login to AWS Management Console**
+
+- Go to the [AWS Management Console](https://console.aws.amazon.com/) and log in with your AWS account credentials.
+
+**Create an IAM User**
+
+- Navigate to the [IAM dashboard](https://console.aws.amazon.com/iam/home) and click on "Users" in the left-hand navigation menu.
+- Click on "Create user" and fill in the required details to create a new IAM user.
+- Make sure to select "Programmatic access" as the access type.
+
+**Create Access Key**
+
+- Click on the "Security credentials" tab and then click on "Create access key".
+- Click "Create access key" to generate a new access key pair.
+- Download the `.csv` file containing the access key ID and secret access key.
+
+**Step 2: Configuring the Access Key and Secret Key**
+
+Within your project add a `.env.local` file if you don't already have one. This file will be used to set the access key and secret key as environment variables. Add the following lines to the `.env.local` file:
+
+```makefile
+AWS_ACCESS_KEY_ID=YOUR_ACCESS_KEY_ID
+AWS_SECRET_ACCESS_KEY=YOUR_SECRET_ACCESS_KEY
+```
+
+Remember to replace `YOUR_ACCESS_KEY_ID` and `YOUR_SECRET_ACCESS_KEY` with the actual values from your AWS account.
+
+**Step 3: Using with @ai-sdk/amazon-bedrock**
+
+Here's an example of how to use the access key and secret key with the `@ai-sdk/amazon-bedrock`:
+
+```javascript
+import { createAmazonBedrock } from '@ai-sdk/amazon-bedrock';
+
+const bedrock = createAmazonBedrock({
+  region: 'your-region',
+  credentials: {
+    accessKeyId: process.env.AWS_ACCESS_KEY_ID,
+    secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY,
+  },
+});
+```
+
+## Setup
+
+The Bedrock provider is available in the `@ai-sdk/amazon-bedrock` module. You can install it with
+
+<Tabs items={['pnpm', 'npm', 'yarn']}>
+  <Tab>
+    <Snippet text="pnpm install @ai-sdk/amazon-bedrock" dark />
+  </Tab>
+  <Tab>
+    <Snippet text="npm install @ai-sdk/amazon-bedrock" dark />
+  </Tab>
+  <Tab>
+    <Snippet text="yarn add @ai-sdk/amazon-bedrock" dark />
+  </Tab>
+</Tabs>
+
+## Provider Instance
+
+You can import the default provider instance `bedrock` from `@ai-sdk/amazon-bedrock`:
+
+```ts
+import { bedrock } from '@ai-sdk/amazon-bedrock';
+```
+
+If you need a customized setup, you can import `createAmazonBedrock` from `@ai-sdk/amazon-bedrock` and create a provider instance with your settings:
+
+```ts
+import { createAmazonBedrock } from '@ai-sdk/amazon-bedrock';
+
+const bedrock = createAmazonBedrock({
+  region: 'us-east-1',
+  credentials: {
+    accessKeyId: 'xxxxxxxxx',
+    secretAccessKey: 'xxxxxxxxx',
+  },
+});
+```
+
+Documentation for additional settings can be found within the [Bedrock Runtime Documentation](https://docs.aws.amazon.com/AWSJavaScriptSDK/v3/latest/client/bedrock-runtime/).
+
+## Language Models
+
+You can create models that call the Bedrock API using the provider instance.
+The first argument is the model id, e.g. `meta.llama3-70b-instruct-v1:0`.
+
+```ts
+const model = bedrock('meta.llama3-70b-instruct-v1:0');
+```
+
+Amazon Bedrock models also support some model specific settings that are not part of the [standard call settings](/docs/ai-sdk-core/settings).
+You can pass them as an options argument:
+
+```ts
+const model = bedrock('anthropic.claude-3-sonnet-20240229-v1:0', {
+  additionalModelRequestFields: { top_k: 350 },
+});
+```
+
+Documentation for additional settings based on the selected model can be found within the [Amazon Bedrock Inference Parameter Documentation](https://docs.aws.amazon.com/bedrock/latest/userguide/model-parameters.html).
+
+### Example
+
+You can use Amazon Bedrock language models to generate text with the `generateText` function:
+
+```ts
+import { bedrock } from '@ai-sdk/amazon-bedrock';
+import { generateText } from 'ai'
+
+const { text } = await generateText({
+  model: bedrock('meta.llama3-70b-instruct-v1:0')
+  prompt: 'Write a vegetarian lasagna recipe for 4 people.'
+})
+```
+
+Amazon Bedrock language models can also be used in the `streamText` function
+(see [AI SDK Core](/docs/ai-sdk-core)).
+
+### Model Capabilities
+
+> Note: This model list is ever changing and may not be complete. Refer to the [Amazon Bedrock ](https://docs.aws.amazon.com/bedrock/latest/userguide/conversation-inference.html#conversation-inference-supported-models-features) documentation for up to date information.
+
+| Model                                       | Image Input         | Object Generation   | Tool Usage          | Tool Streaming      |
+| ------------------------------------------- | ------------------- | ------------------- | ------------------- | ------------------- |
+| `amazon.titan-tg1-large`                    | <Cross size={18} /> | <Cross size={18} /> | <Cross size={18} /> | <Cross size={18} /> |
+| `amazon.titan-text-express-v1`              | <Cross size={18} /> | <Cross size={18} /> | <Cross size={18} /> | <Cross size={18} /> |
+| `anthropic.claude-v2:1`                     | <Cross size={18} /> | <Cross size={18} /> | <Cross size={18} /> | <Cross size={18} /> |
+| `anthropic.claude-3-sonnet-20240229-v1:0`   | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> |
+| `anthropic.claude-3-haiku-20240307-v1:0`    | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> |
+| `anthropic.claude-3-opus-20240229-v1:0`     | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> |
+| `anthropic.claude-3-5-sonnet-20240620-v1:0` | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> |
+| `cohere.command-r-v1:0`                     | <Cross size={18} /> | <Cross size={18} /> | <Check size={18} /> | <Cross size={18} /> |
+| `cohere.command-r-plus-v1:0`                | <Cross size={18} /> | <Cross size={18} /> | <Check size={18} /> | <Cross size={18} /> |
+| `meta.llama2-13b-chat-v1`                   | <Cross size={18} /> | <Cross size={18} /> | <Cross size={18} /> | <Cross size={18} /> |
+| `meta.llama2-70b-chat-v1`                   | <Cross size={18} /> | <Cross size={18} /> | <Cross size={18} /> | <Cross size={18} /> |
+| `meta.llama3-8b-instruct-v1:0`              | <Cross size={18} /> | <Cross size={18} /> | <Cross size={18} /> | <Cross size={18} /> |
+| `meta.llama3-70b-instruct-v1:0`             | <Cross size={18} /> | <Cross size={18} /> | <Cross size={18} /> | <Cross size={18} /> |
+| `mistral.mistral-7b-instruct-v0:2`          | <Cross size={18} /> | <Cross size={18} /> | <Cross size={18} /> | <Cross size={18} /> |
+| `mistral.mixtral-8x7b-instruct-v0:1`        | <Cross size={18} /> | <Cross size={18} /> | <Cross size={18} /> | <Cross size={18} /> |
+| `mistral.mistral-large-2402-v1:0`           | <Cross size={18} /> | <Cross size={18} /> | <Cross size={18} /> | <Cross size={18} /> |
+| `mistral.mistral-small-2402-v1:0`           | <Cross size={18} /> | <Cross size={18} /> | <Cross size={18} /> | <Cross size={18} /> |

examples/ai-core/package.json

@@ -10,6 +10,7 @@
     "@ai-sdk/google-vertex": "latest",
     "@ai-sdk/mistral": "latest",
     "@ai-sdk/openai": "latest",
+    "@ai-sdk/amazon-bedrock": "latest",
     "ai": "latest",
     "dotenv": "16.4.5",
     "mathjs": "12.4.2",
@@ -24,4 +25,4 @@
     "tsx": "4.7.1",
     "typescript": "5.1.3"
   }
-}
+}

examples/ai-core/src/generate-text/bedrock-chatbot-with-tools.ts

@@ -0,0 +1,59 @@
+import { bedrock } from '@ai-sdk/amazon-bedrock';
+import { CoreMessage, generateText } from 'ai';
+import dotenv from 'dotenv';
+import * as readline from 'node:readline/promises';
+import { weatherTool } from '../tools/weather-tool';
+
+dotenv.config();
+
+const terminal = readline.createInterface({
+  input: process.stdin,
+  output: process.stdout,
+});
+
+const messages: CoreMessage[] = [];
+
+async function main() {
+  let toolResponseAvailable = false;
+
+  while (true) {
+    if (!toolResponseAvailable) {
+      const userInput = await terminal.question('You: ');
+      messages.push({ role: 'user', content: userInput });
+    }
+
+    const { text, toolCalls, toolResults, responseMessages } =
+      await generateText({
+        model: bedrock('anthropic.claude-3-haiku-20240307-v1:0'),
+        tools: { weatherTool },
+        system: `You are a helpful, respectful and honest assistant. If the weatehr is requested use the `,
+        messages,
+      });
+
+    toolResponseAvailable = false;
+
+    if (text) {
+      process.stdout.write(`\nAssistant: ${text}`);
+    }
+
+    for (const { toolName, args } of toolCalls) {
+      process.stdout.write(
+        `\nTool call: '${toolName}' ${JSON.stringify(args)}`,
+      );
+    }
+
+    for (const { toolName, result } of toolResults) {
+      process.stdout.write(
+        `\nTool response: '${toolName}' ${JSON.stringify(result)}`,
+      );
+    }
+
+    process.stdout.write('\n\n');
+
+    messages.push(...responseMessages);
+
+    toolResponseAvailable = toolCalls.length > 0;
+  }
+}
+
+main().catch(console.error);

examples/ai-core/src/generate-text/bedrock-multimodal-url.ts

@@ -0,0 +1,29 @@
+import { bedrock } from '@ai-sdk/amazon-bedrock';
+import { generateText } from 'ai';
+import dotenv from 'dotenv';
+
+dotenv.config();
+
+async function main() {
+  const result = await generateText({
+    model: bedrock('anthropic.claude-3-haiku-20240307-v1:0'),
+    maxTokens: 512,
+    messages: [
+      {
+        role: 'user',
+        content: [
+          { type: 'text', text: 'Describe the image in detail.' },
+          {
+            type: 'image',
+            image:
+              'https://github.com/vercel/ai/blob/main/examples/ai-core/data/comic-cat.png?raw=true',
+          },
+        ],
+      },
+    ],
+  });
+
+  console.log(result.text);
+}
+
+main().catch(console.error);

examples/ai-core/src/generate-text/bedrock-multimodal.ts

@@ -0,0 +1,26 @@
+import { bedrock } from '@ai-sdk/amazon-bedrock';
+import { generateText } from 'ai';
+import dotenv from 'dotenv';
+import fs from 'node:fs';
+
+dotenv.config();
+
+async function main() {
+  const result = await generateText({
+    model: bedrock('anthropic.claude-3-haiku-20240307-v1:0'),
+    maxTokens: 512,
+    messages: [
+      {
+        role: 'user',
+        content: [
+          { type: 'text', text: 'Describe the image in detail.' },
+          { type: 'image', image: fs.readFileSync('./data/comic-cat.png') },
+        ],
+      },
+    ],
+  });
+
+  console.log(result.text);
+}
+
+main().catch(console.error);

examples/ai-core/src/generate-text/bedrock-tool-call.ts

@@ -0,0 +1,60 @@
+import { generateText, tool } from 'ai';
+import dotenv from 'dotenv';
+import { z } from 'zod';
+import { weatherTool } from '../tools/weather-tool';
+import { bedrock } from '@ai-sdk/amazon-bedrock';
+
+dotenv.config();
+
+async function main() {
+  const result = await generateText({
+    model: bedrock('anthropic.claude-3-haiku-20240307-v1:0'),
+    maxTokens: 512,
+    tools: {
+      weather: weatherTool,
+      cityAttractions: tool({
+        parameters: z.object({ city: z.string() }),
+      }),
+    },
+    prompt:
+      'What is the weather in San Francisco and what attractions should I visit?',
+  });
+
+  // typed tool calls:
+  for (const toolCall of result.toolCalls) {
+    switch (toolCall.toolName) {
+      case 'cityAttractions': {
+        toolCall.args.city; // string
+        break;
+      }
+
+      case 'weather': {
+        toolCall.args.location; // string
+        break;
+      }
+    }
+  }
+
+  // typed tool results for tools with execute method:
+  for (const toolResult of result.toolResults) {
+    switch (toolResult.toolName) {
+      // NOT AVAILABLE (NO EXECUTE METHOD)
+      // case 'cityAttractions': {
+      //   toolResult.args.city; // string
+      //   toolResult.result;
+      //   break;
+      // }
+
+      case 'weather': {
+        toolResult.args.location; // string
+        toolResult.result.location; // string
+        toolResult.result.temperature; // number
+        break;
+      }
+    }
+  }
+
+  console.log(JSON.stringify(result, null, 2));
+}
+
+main().catch(console.error);

examples/ai-core/src/generate-text/bedrock-tool-choice.ts

@@ -0,0 +1,30 @@
+import { generateText, tool } from 'ai';
+import dotenv from 'dotenv';
+import { z } from 'zod';
+import { weatherTool } from '../tools/weather-tool';
+import { bedrock } from '@ai-sdk/amazon-bedrock';
+
+dotenv.config();
+
+async function main() {
+  const result = await generateText({
+    model: bedrock('anthropic.claude-3-haiku-20240307-v1:0'),
+    maxTokens: 512,
+    tools: {
+      weather: weatherTool,
+      cityAttractions: tool({
+        parameters: z.object({ city: z.string() }),
+      }),
+    },
+    toolChoice: {
+      type: 'tool',
+      toolName: 'weather',
+    },
+    prompt:
+      'What is the weather in San Francisco and what attractions should I visit?',
+  });
+
+  console.log(JSON.stringify(result, null, 2));
+}
+
+main().catch(console.error);