Skip to content

Commit 2a6fa8b

Browse files
committed
resolve comments
update update update update
1 parent a84add5 commit 2a6fa8b

File tree

3 files changed

+47
-48
lines changed

3 files changed

+47
-48
lines changed

docs/how-to-guides/develop-a-tool/create-custom-llm-tool.md

+47-48
Original file line numberDiff line numberDiff line change
@@ -4,60 +4,59 @@ In this document, we will guide you through the process of developing a tool wit
44
Custom LLM is a large language model fine-tuned by yourself. If you find it has good performance and is useful, then you can follow this guidance to make it a tool so that it can be shared with other people to have more impact.
55

66
## Prerequisites
7-
- Please ensure that your `Prompt flow` for VS Code is updated to version 1.2.0 or later.
7+
- Please ensure that your [Prompt flow for VS Code](https://marketplace.visualstudio.com/items?itemName=prompt-flow.prompt-flow) is updated to version 1.2.0 or later.
88

99
## How to create a custom LLM tool
1010
Here we use [an existing tool package](../../../examples/tools/tool-package-quickstart/my_tool_package) as an example. If you want to create your own tool, please refer to [create and use tool package](create-and-use-tool-package.md).
1111

12-
### Add a `PromptTemplate` input to your tool, like in [this example](../../../examples/tools/tool-package-quickstart/my_tool_package/tools/tool_with_custom_llm_type.py)
13-
14-
```python
15-
from jinja2 import Template
16-
from promptflow import tool
17-
from promptflow.connections import CustomConnection
18-
# 1. Import the PromptTemplate type.
19-
from promptflow.contracts.types import PromptTemplate
20-
21-
22-
# 2. Add a PromptTemplate input for your tool method.
23-
@tool
24-
def my_tool(connection: CustomConnection, prompt: PromptTemplate, **kwargs) -> str:
25-
# 3. The prompt is used for your custom LLM to do inference, customize your own code to handle and use the prompt here.
26-
message = Template(prompt, trim_blocks=True, keep_trailing_newline=True).render(**kwargs)
27-
return message
28-
```
29-
30-
### Generate custom LLM tool YAML
31-
32-
Run the command below in your tool project directory to automatically generate your tool YAML, use _-t "custom_llm"_ or _--tool-type "custom_llm"_ to indicate this is a custom LLM tool:
33-
```
34-
python <path-to-scripts>\tool\generate_package_tool_meta.py -m <tool_module> -o <tool_yaml_path> -t "custom_llm"
35-
```
36-
Here we use [an existing tool](../../../examples/tools/tool-package-quickstart/my_tool_package/yamls/tool_with_custom_llm_type.yaml) as an example.
37-
```
38-
cd D:\proj\github\promptflow\examples\tools\tool-package-quickstart
39-
40-
python D:\proj\github\promptflow\scripts\tool\generate_package_tool_meta.py -m my_tool_package.tools.tool_with_custom_llm_type -o my_tool_package\yamls\tool_with_custom_llm_type.yaml -n "My Custom LLM Tool" -d "This is a tool to demonstrate the custom_llm tool type" -t "custom_llm"
41-
```
42-
This command will generate a YAML file as follows:
43-
44-
```yaml
45-
my_tool_package.tools.tool_with_custom_llm_type.my_tool:
46-
name: My Custom LLM Tool
47-
description: This is a tool to demonstrate the custom_llm tool type
48-
# The type is custom_llm.
49-
type: custom_llm
50-
module: my_tool_package.tools.tool_with_custom_llm_type
51-
function: my_tool
52-
inputs:
53-
connection:
54-
type:
55-
- CustomConnection
56-
```
12+
1. Develop tool code like in [this example](../../../examples/tools/tool-package-quickstart/my_tool_package/tools/tool_with_custom_llm_type.py).
13+
- Add a `CustomConnection` input to the tool, it is used for authenticating and establishing a connection to the customized large language model.
14+
- Add a `PromptTemplate` input to the tool, it is used to pass into the customized large language model as an argument.
15+
16+
```python
17+
from jinja2 import Template
18+
from promptflow import tool
19+
from promptflow.connections import CustomConnection
20+
from promptflow.contracts.types import PromptTemplate
21+
22+
23+
@tool
24+
def my_tool(connection: CustomConnection, prompt: PromptTemplate, **kwargs) -> str:
25+
# Customize your own code to use the connection and prompt here.
26+
rendered_prompt = Template(prompt, trim_blocks=True, keep_trailing_newline=True).render(**kwargs)
27+
return rendered_prompt
28+
```
29+
30+
2. Generate the custom LLM tool YAML.
31+
Run the command below in your tool project directory to automatically generate your tool YAML, use _-t "custom_llm"_ or _--tool-type "custom_llm"_ to indicate this is a custom LLM tool:
32+
```
33+
python <path-to-scripts>\tool\generate_package_tool_meta.py -m <tool_module> -o <tool_yaml_path> -t "custom_llm"
34+
```
35+
Here we use [an existing tool](../../../examples/tools/tool-package-quickstart/my_tool_package/yamls/tool_with_custom_llm_type.yaml) as an example.
36+
```
37+
cd D:\proj\github\promptflow\examples\tools\tool-package-quickstart
38+
39+
python D:\proj\github\promptflow\scripts\tool\generate_package_tool_meta.py -m my_tool_package.tools.tool_with_custom_llm_type -o my_tool_package\yamls\tool_with_custom_llm_type.yaml -n "My Custom LLM Tool" -d "This is a tool to demonstrate the custom_llm tool type" -t "custom_llm"
40+
```
41+
This command will generate a YAML file as follows:
42+
43+
```yaml
44+
my_tool_package.tools.tool_with_custom_llm_type.my_tool:
45+
name: My Custom LLM Tool
46+
description: This is a tool to demonstrate the custom_llm tool type
47+
# The type is custom_llm.
48+
type: custom_llm
49+
module: my_tool_package.tools.tool_with_custom_llm_type
50+
function: my_tool
51+
inputs:
52+
connection:
53+
type:
54+
- CustomConnection
55+
```
5756

5857
## Use custom LLM tool in VS Code extension
5958
Follow the steps to [build and install your tool package](create-and-use-tool-package.md#build-and-share-the-tool-package) and [use your tool from VS Code extension](create-and-use-tool-package.md#use-your-tool-from-vscode-extension).
6059

6160
Here we use an existing flow to demonstrate the experience, open [this flow](../../../examples/flows/standard/custom_llm_tool_showcase/flow.dag.yaml) in VS Code extension.
62-
- There is a node named "custom_llm_tool" with a prompt template file. You can either use an existing file or create a new one as the prompt template file.
63-
![use_custom_llm_tool](../../media/how-to-guides/develop-a-tool/use_custom_llm_tool.png)
61+
- There is a node named "my_custom_llm_tool" with a prompt template file. You can either use an existing file or create a new one as the prompt template file.
62+
![use_my_custom_llm_tool](../../media/how-to-guides/develop-a-tool/use_my_custom_llm_tool.png)
Binary file not shown.
Loading

0 commit comments

Comments
 (0)