Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update llm-tool.md and concept-connections.md to include Serverless Connection #2959

Merged
merged 11 commits into from
Apr 26, 2024
1 change: 1 addition & 0 deletions docs/concepts/concept-connections.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ Prompt flow provides a variety of pre-built connections, including Azure Open AI
| [Open AI](https://openai.com/) | LLM or Python |
| [Cognitive Search](https://azure.microsoft.com/en-us/products/search) | Vector DB Lookup or Python |
| [Serp](https://serpapi.com/) | Serp API or Python |
| [Serverless](https://learn.microsoft.com/en-us/azure/ai-studio/concepts/deployments-overview#deploy-models-with-model-as-a-service) | LLM or Python |
| Custom | Python |

By leveraging connections in prompt flow, you can easily establish and manage connections to external APIs and data sources, facilitating efficient data exchange and interaction within their AI applications.
Expand Down
19 changes: 13 additions & 6 deletions docs/reference/tools-reference/llm-tool.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# LLM

## Introduction
Prompt flow LLM tool enables you to leverage widely used large language models like [OpenAI](https://platform.openai.com/) or [Azure OpenAI (AOAI)](https://learn.microsoft.com/en-us/azure/cognitive-services/openai/overview) for natural language processing.
Prompt flow LLM tool enables you to leverage widely used large language models like [OpenAI](https://platform.openai.com/), [Azure OpenAI (AOAI)](https://learn.microsoft.com/en-us/azure/cognitive-services/openai/overview), and models in [Azure AI Studio model catalog](https://learn.microsoft.com/en-us/azure/ai-studio/how-to/model-catalog) for natural language processing.
> [!NOTE]
> The previous version of the LLM tool is now being deprecated. Please upgrade to latest [promptflow-tools](https://pypi.org/project/promptflow-tools/) package to consume new llm tools.

Expand All @@ -11,7 +11,7 @@ Prompt flow provides a few different LLM APIs:


## Prerequisite
Create OpenAI or Azure OpenAI resources:
Create OpenAI resources, Azure OpenAI resources or MaaS deployment with the LLM models (e.g.: llama2, mistral, cohere etc.) in Azure AI Studio model catalog:

- **OpenAI**

Expand All @@ -23,14 +23,21 @@ Create OpenAI or Azure OpenAI resources:

Create Azure OpenAI resources with [instruction](https://learn.microsoft.com/en-us/azure/cognitive-services/openai/how-to/create-resource?pivots=web-portal)

- **MaaS deployment**

Create MaaS deployment for models in Azure AI Studio model catalog with [instruction](https://learn.microsoft.com/en-us/azure/ai-studio/concepts/deployments-overview#deploy-models-with-model-as-a-service)

You can create serverless connection to use this MaaS deployment.

## **Connections**

Setup connections to provisioned resources in prompt flow.

| Type | Name | API KEY | API Type | API Version |
|-------------|----------|----------|----------|-------------|
| OpenAI | Required | Required | - | - |
| AzureOpenAI | Required | Required | Required | Required |
| Type | Name | API KEY | API BASE | API Type | API Version |
|-------------|----------|----------|----------|-----------|-------------|
| OpenAI | Required | Required | - | - | - |
| AzureOpenAI | Required | Required | Required | Required | Required |
| Serverless | Required | Required | Required | - | - |


## Inputs
Expand Down
Loading