New PF Tool: Open_Source_llm - Dev, Test, & Documentation

.cspell.json

@@ -125,7 +125,8 @@
     "aggr",
     "tcsetattr",
     "pysqlite",
-    "AADSTS700082"
+    "AADSTS700082",
+    "Mobius"
   ],
   "allowCompoundWords": true
 }

docs/reference/index.md

@@ -37,6 +37,7 @@ tools-reference/serp-api-tool
 tools-reference/faiss_index_lookup_tool
 tools-reference/vector_db_lookup_tool
 tools-reference/embedding_tool
+tools-reference/open_source_llm_tool
 ```
 
 ```{toctree}

docs/reference/tools-reference/open_source_llm_tool.md

@@ -0,0 +1,58 @@
+# Open Source LLM
+
+## Introduction
+
+The Prompt flow Open Source LLM tool enables you to utilize a variety of Open Source and Foundational Models, such as [Falcon](https://aka.ms/AAlc25c) or [Llama 2](https://aka.ms/AAlc258) for natural language processing, in PromptFlow.
+
+This Prompt flow supports two different LLM API types:
+
+- **Completion**: to generate text based on provided prompts.
+- **Chat**: to facilitate interactive conversations with text-based inputs and responses.
+
+## Prerequisite
+
+1. Pick the model to use with from the [Azure Machine Learning model catalog](https://ml.azure.com/model/catalog).
+2. Use the "Deploy" button to deploy the model to AzureML Inference endpoint.
+
+More detailed instructions can be found here [Deploying foundation models to endpoints for inferencing.](https://learn.microsoft.com/en-us/azure/machine-learning/how-to-use-foundation-models?view=azureml-api-2#deploying-foundation-models-to-endpoints-for-inferencing)
+
+## **Connections**
+
+Setup connections to provisioned resources in prompt flow.
+
+| Type        | Name     | API KEY  | API Type | API Version |
+|-------------|----------|----------|----------|-------------|
+| CustomConnection | Required | Required | -        | -           |
+
+Instructions to create a Custom Connection [can be found here.](https://learn.microsoft.com/en-us/azure/machine-learning/prompt-flow/how-to-integrate-with-langchain?view=azureml-api-2#create-a-connection)
+
+The keys to set are:
+
+1. endpoint_url
+2. endpoint_api_key
+3. model_family
+
+*These values can be found at the previously created Inferencing endpoint.*
+
+## Inputs
+
+| Name                   | Type        | Description                                                                             | Required |
+|------------------------|-------------|-----------------------------------------------------------------------------------------|----------|
+| api | string | this will be Completion or Chat, depending on the scenario selected | Yes |
+| connection | CustomConnection | the name of the connection which points to the Inferencing endpoint | Yes |
+| model_kwargs | dictionary | generic model configuration values, for example temperature | Yes |
+| prompt | string | text prompt that the language model will complete | Yes |
+
+## Outputs
+
+| API        | Return Type | Description                              |
+|------------|-------------|------------------------------------------|
+| Completion | string      | The text of one predicted completion     |
+| Chat       | string      | The text of one response of conversation |
+
+## How to use Open Source LLM Tool?
+
+1. Choose a Model from the catalog and deploy.
+2. Setup and select the connections to model deployment.
+3. Configure the model api and its parameters
+4. Prepare the Prompt with [guidance](./prompt-tool.md#how-to-write-prompt).

src/promptflow-tools/promptflow/tools/common.py

@@ -10,10 +10,13 @@
 from promptflow.tools.exception import ChatAPIInvalidRole, WrappedOpenAIError, LLMError, JinjaTemplateError, \
     ExceedMaxRetryTimes, ChatAPIInvalidFunctions, FunctionCallNotSupportedInStreamMode, \
     ChatAPIFunctionRoleInvalidFormat
+from typing import List
 
 
-def validate_role(role):
-    valid_roles = {"system", "user", "assistant", "function"}
+def validate_role(role: str, valid_roles: List[str] = None):
+    if not valid_roles:
+        valid_roles = {"system", "user", "assistant", "function"}
+
     if role not in valid_roles:
         valid_roles_str = ','.join([f'\'{role}:\\n\''for role in valid_roles])
         error_message = (

src/promptflow-tools/promptflow/tools/exception.py

@@ -134,3 +134,24 @@ class SerpAPIUserError(UserErrorException):
 
     def __init__(self, **kwargs):
         super().__init__(**kwargs, target=ErrorTarget.TOOL)
+
+
+class OpenSourceLLMOnlineEndpointError(UserErrorException):
+    """Base exception raised when the call to an online endpoint failed."""
+
+    def __init__(self, **kwargs):
+        super().__init__(**kwargs, target=ErrorTarget.TOOL)
+
+
+class OpenSourceLLMUserError(UserErrorException):
+    """Base exception raised when the call to Open Source LLM failed with a user error."""
+
+    def __init__(self, **kwargs):
+        super().__init__(**kwargs, target=ErrorTarget.TOOL)
+
+
+class OpenSourceLLMKeyValidationError(ToolValidationError):
+    """Base exception raised when failed to validate functions when call chat api."""
+
+    def __init__(self, **kwargs):
+        super().__init__(**kwargs)