Skip to content

Latest commit

 

History

History
140 lines (117 loc) · 6.54 KB

README.md

File metadata and controls

140 lines (117 loc) · 6.54 KB

Rockhead.Extensions

Build & Test

Rockhead.Extensions is a set of extension methods for the AWS SDK for .NET Bedrock Runtime client. Amazon Bedrock is a serverless service making available through a unique API foundation models (FMs) from Amazon, AI21 Labs, Anthropic, Cohere, Meta, Mistral AI and Stability AI.

Rockhead.Extensions provides you extension methods to the low-level Bedrock Runtime API with strongly typed parameters and responses for each supported foundation models. It makes your developer life easier.

Rockhead.Extensions

Available on Nuget: https://www.nuget.org/packages/Rockhead.Extensions

  • AI21 Labs extension methods
    • InvokeJurassic2Async
      • An extension method to invoke Jurassic 2 models to generate text with strongly type parameters and response
      • Support Jurassic 2 Mid or Ultra models
  • Amazon extension methods
    • InvokeTitanImageGeneratorG1ForTextToImageAsync
      • An extension method to invoke Titan Image Generator G1 to generate images with strongly type parameters and response
    • InvokeTitanTextG1Async
      • An extension method to invoke Titan Text G1 models to generate text with strongly type parameters and response
      • Support Titan G1 Lite or Express
    • InvokeTitanTextG1WithResponseStreamAsync
      • An extension method to invoke Titan Text G1 models to generate text with strongly type parameters and returning an IAsyncEnumerable of strongly typed response
      • Support Titan G1 Lite or Express
    • InvokeTitanEmbeddingsG1TextAsync
      • An extension method to invoke Titan Embeddings G1 models to generate embeddings with strongly type parameters and response
    • InvokeTitanMultimodalEmbeddingsG1Async
      • An extension method to invoke Titan Multimodal Embeddings G1 models to generate embeddings with strongly type parameters and response
  • Anthropic extension methods
    • InvokeClaudeAsync
      • An extension method to invoke Claude models to generate text with strongly type parameters and response
      • Support Claude Instant v1, Claude v2 and Claude v2.1
    • InvokeClaudeWithResponseStreamAsync
      • An extension method to invoke Claude models to generate text with strongly type parameters and returning an IAsyncEnumerable of strongly typed response
      • Support Claude Instant v1, Claude v2 and Claude v2.1
  • Cohere extension methods
    • InvokeCommandV14Async
      • An extension method to invoke Command v14 models to generate text with strongly type parameters and response
      • Support Command v14 Text and Command v14 Light Text
    • InvokeCommandV14WithResponseStreamAsync
      • An extension method to invoke Command v14 models to generate text with strongly type parameters and returning an IAsyncEnumerable of strongly typed response
      • Support Command v14 Text and Command v14 Light Text
    • InvokeEmbedV3Async
  • Meta extension methods
    • InvokeLlamaAsync
      • An extension method to invoke Llama 2 models to generate text with strongly type parameters and response
      • Support Llama 2 13B Chat v1, Llama 2 70B Chat v1, Llama 3 8B Instruct and Llama 3 70B Instruct
    • InvokeLlamaWithResponseStreamAsync
      • An extension method to invoke Llama 2 models to generate text with strongly type parameters and returning an IAsyncEnumerable of strongly typed response
      • Support Llama 2 13B Chat v1, Llama 2 70B Chat v1, Llama 3 8B Instruct and Llama 3 70B Instruct
  • Stability AI extension methods
    • InvokeStableDiffusionXlForTextToImageAsync
      • An extension method to invoke Stable Diffusion XL to generate images with strongly type parameters and response
  • MistralAI extension methods
    • InvokeMistralAsync
      • An extension method to invoke Mistral AI models to generate text with strongly type parameters and response
      • Support Mistral AI 7B Instruct, Mistral AI 8x7B Instruct and Mistral Large
    • InvokeMistralWithResponseStreamAsync
      • An extension method to invoke Mistral AI models to generate text with strongly type parameters and returning an IAsyncEnumerable of strongly typed response
      • Support Mistral AI 7B Instruct, Mistral AI 8x7B Instruct and Mistral Large

Setup

dotnet add package rockhead.extensions

Usage

Below are a few examples of using the extension methods to invoke different models.

Invoke Claude v2.1 model

public async Task<string> GetLlmDescription()
{
    const string prompt = @"Human: Describe in one sentence what it a large language model\n\nAssistant:";
    var config = new ClaudeTextGenerationConfig()
    {
        MaxTokensToSample = 2048,
        Temperature = 0.8f
    };
    
    var response = await BedrockRuntime.InvokeClaudeAsync(new Model.ClaudeV2_1(), prompt, config);
    
    return response?.Completion ?? "";
}

Invoke Claude v2.1 model with a response stream

public async Task<string> GetLlmDescription()
    {
        const string prompt = @"Human: Describe in one sentence what it a large language model\n\nAssistant:";
        var config = new ClaudeTextGenerationConfig()
        {
            MaxTokensToSample = 2048,
            Temperature = 0.8f
        };

        var response = new StringBuilder();
        await foreach (var chunk in BedrockRuntime.InvokeClaudeWithResponseStreamAsync(new Model.ClaudeV2_1(), prompt, config))
        {
            response.Append(chunk.Completion);
        }
    
        return response.ToString();
    }

Invoke Llama 2 70B Chat

public async Task<string> GetLlmDescription()
{
    const string prompt = @"Describe in one sentence what it a large language model";
    var config = new LlamaTextGenerationConfig()
    {
        MaxGenLen = 2048,
        Temperature = 0.8f
    };
    
    var response = await BedrockRuntime.InvokeLlamaAsync(new Model.Llama270BChatV1(), prompt, config);
    
    return response?.Generation ?? "";
}

Learn More