diff --git a/docs/how-to-guides/develop-a-tool/create-custom-llm-tool.md b/docs/how-to-guides/develop-a-tool/create-custom-llm-tool.md index 93440f6eea5e..65cae14d2d37 100644 --- a/docs/how-to-guides/develop-a-tool/create-custom-llm-tool.md +++ b/docs/how-to-guides/develop-a-tool/create-custom-llm-tool.md @@ -4,60 +4,59 @@ In this document, we will guide you through the process of developing a tool wit Custom LLM is a large language model fine-tuned by yourself. If you find it has good performance and is useful, then you can follow this guidance to make it a tool so that it can be shared with other people to have more impact. ## Prerequisites -- Please ensure that your `Prompt flow` for VS Code is updated to version 1.2.0 or later. +- Please ensure that your [Prompt flow for VS Code](https://marketplace.visualstudio.com/items?itemName=prompt-flow.prompt-flow) is updated to version 1.2.0 or later. ## How to create a custom LLM tool Here we use [an existing tool package](../../../examples/tools/tool-package-quickstart/my_tool_package) as an example. If you want to create your own tool, please refer to [create and use tool package](create-and-use-tool-package.md). -### Add a `PromptTemplate` input to your tool, like in [this example](../../../examples/tools/tool-package-quickstart/my_tool_package/tools/tool_with_custom_llm_type.py) - -```python -from jinja2 import Template -from promptflow import tool -from promptflow.connections import CustomConnection -# 1. Import the PromptTemplate type. -from promptflow.contracts.types import PromptTemplate - - -# 2. Add a PromptTemplate input for your tool method. -@tool -def my_tool(connection: CustomConnection, prompt: PromptTemplate, **kwargs) -> str: - # 3. The prompt is used for your custom LLM to do inference, customize your own code to handle and use the prompt here. - message = Template(prompt, trim_blocks=True, keep_trailing_newline=True).render(**kwargs) - return message -``` - -### Generate custom LLM tool YAML - -Run the command below in your tool project directory to automatically generate your tool YAML, use _-t "custom_llm"_ or _--tool-type "custom_llm"_ to indicate this is a custom LLM tool: -``` -python \tool\generate_package_tool_meta.py -m -o -t "custom_llm" -``` -Here we use [an existing tool](../../../examples/tools/tool-package-quickstart/my_tool_package/yamls/tool_with_custom_llm_type.yaml) as an example. -``` -cd D:\proj\github\promptflow\examples\tools\tool-package-quickstart - -python D:\proj\github\promptflow\scripts\tool\generate_package_tool_meta.py -m my_tool_package.tools.tool_with_custom_llm_type -o my_tool_package\yamls\tool_with_custom_llm_type.yaml -n "My Custom LLM Tool" -d "This is a tool to demonstrate the custom_llm tool type" -t "custom_llm" -``` -This command will generate a YAML file as follows: - -```yaml -my_tool_package.tools.tool_with_custom_llm_type.my_tool: -name: My Custom LLM Tool -description: This is a tool to demonstrate the custom_llm tool type -# The type is custom_llm. -type: custom_llm -module: my_tool_package.tools.tool_with_custom_llm_type -function: my_tool -inputs: - connection: - type: - - CustomConnection -``` +1. Add a `PromptTemplate` input to your tool, like in [this example](../../../examples/tools/tool-package-quickstart/my_tool_package/tools/tool_with_custom_llm_type.py). + + ```python + from jinja2 import Template + from promptflow import tool + from promptflow.connections import CustomConnection + # 1. Import the PromptTemplate type. + from promptflow.contracts.types import PromptTemplate + + + # 2. Add a PromptTemplate input for your tool method. + @tool + def my_tool(connection: CustomConnection, prompt: PromptTemplate, **kwargs) -> str: + # 3. Customize your own code to handle and use the prompt here. + message = Template(prompt, trim_blocks=True, keep_trailing_newline=True).render(**kwargs) + return message + ``` + +2. Generate the custom LLM tool YAML. + Run the command below in your tool project directory to automatically generate your tool YAML, use _-t "custom_llm"_ or _--tool-type "custom_llm"_ to indicate this is a custom LLM tool: + ``` + python \tool\generate_package_tool_meta.py -m -o -t "custom_llm" + ``` + Here we use [an existing tool](../../../examples/tools/tool-package-quickstart/my_tool_package/yamls/tool_with_custom_llm_type.yaml) as an example. + ``` + cd D:\proj\github\promptflow\examples\tools\tool-package-quickstart + + python D:\proj\github\promptflow\scripts\tool\generate_package_tool_meta.py -m my_tool_package.tools.tool_with_custom_llm_type -o my_tool_package\yamls\tool_with_custom_llm_type.yaml -n "My Custom LLM Tool" -d "This is a tool to demonstrate the custom_llm tool type" -t "custom_llm" + ``` + This command will generate a YAML file as follows: + + ```yaml + my_tool_package.tools.tool_with_custom_llm_type.my_tool: + name: My Custom LLM Tool + description: This is a tool to demonstrate the custom_llm tool type + # The type is custom_llm. + type: custom_llm + module: my_tool_package.tools.tool_with_custom_llm_type + function: my_tool + inputs: + connection: + type: + - CustomConnection + ``` ## Use custom LLM tool in VS Code extension Follow the steps to [build and install your tool package](create-and-use-tool-package.md#build-and-share-the-tool-package) and [use your tool from VS Code extension](create-and-use-tool-package.md#use-your-tool-from-vscode-extension). Here we use an existing flow to demonstrate the experience, open [this flow](../../../examples/flows/standard/custom_llm_tool_showcase/flow.dag.yaml) in VS Code extension. -- There is a node named "custom_llm_tool" with a prompt template file. You can either use an existing file or create a new one as the prompt template file. -![use_custom_llm_tool](../../media/how-to-guides/develop-a-tool/use_custom_llm_tool.png) +- There is a node named "my_custom_llm_tool" with a prompt template file. You can either use an existing file or create a new one as the prompt template file. +![use_my_custom_llm_tool](../../media/how-to-guides/develop-a-tool/use_my_custom_llm_tool.png) diff --git a/docs/media/how-to-guides/develop-a-tool/use_custom_llm_tool.png b/docs/media/how-to-guides/develop-a-tool/use_custom_llm_tool.png deleted file mode 100644 index 1ab389aa3b15..000000000000 Binary files a/docs/media/how-to-guides/develop-a-tool/use_custom_llm_tool.png and /dev/null differ diff --git a/docs/media/how-to-guides/develop-a-tool/use_my_custom_llm_tool.png b/docs/media/how-to-guides/develop-a-tool/use_my_custom_llm_tool.png new file mode 100644 index 000000000000..4be0114944ca Binary files /dev/null and b/docs/media/how-to-guides/develop-a-tool/use_my_custom_llm_tool.png differ