-
Notifications
You must be signed in to change notification settings - Fork 98
FinanceAgent helm chart support #1063
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
2 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,46 @@ | ||
| # Copyright (C) 2024 Intel Corporation | ||
| # SPDX-License-Identifier: Apache-2.0 | ||
|
|
||
| apiVersion: v2 | ||
| name: financeagent | ||
| description: The Helm chart to deploy FinanceAgent | ||
| type: application | ||
| dependencies: | ||
| - name: agent | ||
| version: 0-latest | ||
| alias: finqa-agent | ||
| repository: "file://../common/agent" | ||
| - name: agent | ||
| version: 0-latest | ||
| alias: research-agent | ||
| repository: "file://../common/agent" | ||
| - name: agent | ||
| version: 0-latest | ||
| alias: supervisor | ||
| repository: "file://../common/agent" | ||
| - name: llm-uservice | ||
| version: 0-latest | ||
| alias: docsum | ||
| repository: "file://../common/llm-uservice" | ||
| - name: ui | ||
| alias: agent-ui | ||
| version: 0-latest | ||
| repository: "file://../common/ui" | ||
| - name: vllm | ||
| version: 0-latest | ||
| repository: "file://../common/vllm" | ||
| - name: tei | ||
| version: 0-latest | ||
| repository: "file://../common/tei" | ||
| - name: redis-vector-db | ||
| version: 0-latest | ||
| repository: "file://../common/redis-vector-db" | ||
| - name: redis-vector-db | ||
| version: 0-latest | ||
| alias: redis-kv-store | ||
| repository: "file://../common/redis-vector-db" | ||
| - name: data-prep | ||
| version: 0-latest | ||
| repository: "file://../common/data-prep" | ||
| version: 0-latest | ||
| appVersion: "v1.0" |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,97 @@ | ||
| # FinanceAgent | ||
|
|
||
| Helm chart for deploying FinanceAgent example. It demonstrates how agent works, using prepared data and questions. See [FinanceAgent Overview](https://github.com/opea-project/GenAIExamples/tree/main/FinanceAgent#overview) for the details. | ||
yongfengdu marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
|
||
| FinanceAgent depends on the following subcharts: | ||
|
|
||
| - [agent](../common/agent/README.md) | ||
| - [llm-uservice](../common/llm-uservice/README.md) | ||
| - [vllm](../common/vllm/README.md) | ||
| - [tei](../common/tei/README.md) | ||
| - [redis-vector-db](../common/redis-vector-db/README.md) | ||
| - [data-prep](../common/data-prep/README.md) | ||
|
|
||
| Check the [Chart.yaml](./Chart.yaml) file for how these subcharts are used. | ||
|
|
||
| Agent usually requires larger models to perform better, we used `meta-llama/Llama-3.3-70B-Instruct` for test, which requires 4x Gaudi devices for local deployment. | ||
|
|
||
| ## Deploy | ||
|
|
||
| The Deployment includes downloading tools and prompts for the agents, and data ingestion for testing. | ||
|
|
||
| ### Prerequisites | ||
|
|
||
| A volume is required to put tools and prompts used by agent. | ||
|
|
||
| We'll use hostPath in this README, which is convenient for single worker node deployment. PVC is recommended in a bigger cluster. If you want to use a PVC, comment out the `toolHostPath` and replace with `toolPVC` in the `values.yaml`. | ||
|
|
||
| Create the directory `/mnt/tools/financeagent` in the worker node, which is the default in `values.yaml`. We use the same directory for all 3 agents for easy configuration. | ||
|
|
||
| ``` | ||
| sudo mkdir /mnt/tools/financeagent | ||
| sudo chmod 777 /mnt/tools/financeagent | ||
| ``` | ||
|
|
||
| Download prompts, tools and the configuration to `/mnt/tools/financeagent` | ||
|
|
||
| ``` | ||
| # prompts used by 3 agents | ||
| wget https://raw.githubusercontent.com/opea-project/GenAIExamples/refs/heads/main/FinanceAgent/prompts/finqa_prompt.py -O /mnt/tools/financeagent/finqa_prompt.py | ||
| wget https://raw.githubusercontent.com/opea-project/GenAIExamples/refs/heads/main/FinanceAgent/prompts/research_prompt.py -O /mnt/tools/financeagent/research_prompt.py | ||
| wget https://raw.githubusercontent.com/opea-project/GenAIExamples/refs/heads/main/FinanceAgent/prompts/supervisor_prompt.py -O /mnt/tools/financeagent/supervisor_prompt.py | ||
|
|
||
| # tools and configurations used by the agents | ||
| wget https://raw.githubusercontent.com/opea-project/GenAIExamples/refs/heads/main/FinanceAgent/tools/finqa_agent_tools.yaml -O /mnt/tools/financeagent/finqa_agent_tools.yaml | ||
| wget https://raw.githubusercontent.com/opea-project/GenAIExamples/refs/heads/main/FinanceAgent/tools/finqa_tools.py -O /mnt/tools/financeagent/finqa_tools.py | ||
| wget https://raw.githubusercontent.com/opea-project/GenAIExamples/refs/heads/main/FinanceAgent/tools/research_agent_tools.yaml -O /mnt/tools/financeagent/research_agent_tools.yaml | ||
| wget https://raw.githubusercontent.com/opea-project/GenAIExamples/refs/heads/main/FinanceAgent/tools/research_tools.py -O /mnt/tools/financeagent/research_tools.py | ||
| wget https://raw.githubusercontent.com/opea-project/GenAIExamples/refs/heads/main/FinanceAgent/tools/supervisor_agent_tools.yaml -O /mnt/tools/financeagent/supervisor_agent_tools.yaml | ||
| wget https://raw.githubusercontent.com/opea-project/GenAIExamples/refs/heads/main/FinanceAgent/tools/supervisor_tools.py -O /mnt/tools/financeagent/supervisor_tools.py | ||
| wget https://raw.githubusercontent.com/opea-project/GenAIExamples/refs/heads/main/FinanceAgent/tools/sum_agent_tools.py -O /mnt/tools/financeagent/sum_agent_tools.py | ||
| wget https://raw.githubusercontent.com/opea-project/GenAIExamples/refs/heads/main/FinanceAgent/tools/redis_kv.py -O /mnt/tools/financeagent/redis_kv.py | ||
| wget https://raw.githubusercontent.com/opea-project/GenAIExamples/refs/heads/main/FinanceAgent/tools/utils.py -O /mnt/tools/financeagent/utils.py | ||
| ``` | ||
|
|
||
| ### Deploy with Helm chart | ||
|
|
||
| Deploy everything on Gaudi enabled Kubernetes cluster: | ||
|
|
||
| If you want to try with latest version, use `helm pull oci://ghcr.io/opea-project/charts/financeagent --version 0-latest --untar` | ||
|
|
||
| ``` | ||
| export HUGGINGFACEHUB_API_TOKEN="YourOwnToken" | ||
| export FINNHUB_API_KEY="YourOwnToken" | ||
| export FINANCIAL_DATASETS_API_KEY="YourOwnToken" | ||
| helm pull oci://ghcr.io/opea-project/charts/financeagent --untar | ||
| helm install financeagent financeagent -f financeagent/gaudi-values.yaml \ | ||
| --set global.HUGGINGFACEHUB_API_TOKEN=${HUGGINGFACEHUB_API_TOKEN} \ | ||
| --set research-agent.FINNHUB_API_KEY=${FINNHUB_API_KEY} \ | ||
| --set research-agent.FINANCIAL_DATASETS_API_KEY=${FINANCIAL_DATASETS_API_KEY} | ||
| ``` | ||
|
|
||
| ## Verify | ||
|
|
||
| To verify the installation, run the command `kubectl get pod` to make sure all pods are running. | ||
|
|
||
| ### Ingest data for RAG | ||
|
|
||
| Ingest data used by RAG. | ||
|
|
||
| ``` | ||
| ip_address=$(kubectl get svc financeagent-data-prep -o jsonpath='{.spec.clusterIP}) | ||
| curl "http://${ip_address}:6007/v1/dataprep/ingest" -X POST -H "Content-Type: application/x-www-form-urlencoded" \ | ||
| -d 'link_list=%5B%22https%3A%2F%2Fwww.fool.com%2Fearnings%2Fcall-transcripts%2F2025%2F03%2F06%2Fcostco-wholesale-cost-q2-2025-earnings-call-transc%2F%22%2C%22https%3A%2F%2Fwww.fool.com%2Fearnings%2Fcall-transcripts%2F2025%2F03%2F07%2Fgap-gap-q4-2024-earnings-call-transcript%2F%22%5D' | ||
| ``` | ||
|
|
||
| ### Verify the workload through curl command | ||
|
|
||
| Run the command `kubectl port-forward svc/financeagent-supervisor 9090:9090` to expose the service for access. | ||
|
|
||
| Open another terminal and run the following command to verify the service if working: | ||
|
|
||
| ```console | ||
| curl http://localhost:9090/v1/chat/completions \ | ||
| -X POST \ | ||
| -H "Content-Type: application/json" \ | ||
| -d '{"role": "user", "messages": "Can you summarize Costco 2025 Q2 earnings call?"}' | ||
| ``` | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,13 @@ | ||
| # Copyright (C) 2024 Intel Corporation | ||
| # SPDX-License-Identifier: Apache-2.0 | ||
|
|
||
| vllm: | ||
| accelDevice: "gaudi" | ||
| LLM_MODEL_ID: "meta-llama/Llama-3.3-70B-Instruct" | ||
| image: | ||
| repository: opea/vllm-gaudi | ||
| OMPI_MCA_btl_vader_single_copy_mechanism: none | ||
| extraCmdArgs: ["--tensor-parallel-size","4","--max-seq-len-to-capture","16384","--enable-auto-tool-choice","--tool-call-parser","llama3_json"] | ||
| resources: | ||
| limits: | ||
| habana.ai/gaudi: 4 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,64 @@ | ||
| {{/* | ||
| Expand the name of the chart. | ||
| */}} | ||
| {{- define "financeagent.name" -}} | ||
| {{- default .Chart.Name .Values.nameOverride | trunc 63 | trimSuffix "-" }} | ||
| {{- end }} | ||
|
|
||
| {{/* | ||
| Create a default fully qualified app name. | ||
| We truncate at 63 chars because some Kubernetes name fields are limited to this (by the DNS naming spec). | ||
| If release name contains chart name it will be used as a full name. | ||
| */}} | ||
| {{- define "financeagent.fullname" -}} | ||
| {{- if .Values.fullnameOverride }} | ||
| {{- .Values.fullnameOverride | trunc 63 | trimSuffix "-" }} | ||
| {{- else }} | ||
| {{- $name := default .Chart.Name .Values.nameOverride }} | ||
| {{- if contains $name .Release.Name }} | ||
| {{- .Release.Name | trunc 63 | trimSuffix "-" }} | ||
| {{- else }} | ||
| {{- printf "%s-%s" .Release.Name $name | trunc 63 | trimSuffix "-" }} | ||
| {{- end }} | ||
| {{- end }} | ||
| {{- end }} | ||
|
|
||
| {{/* | ||
| Create chart name and version as used by the chart label. | ||
| */}} | ||
| {{- define "financeagent.chart" -}} | ||
| {{- printf "%s-%s" .Chart.Name .Chart.Version | replace "+" "_" | trunc 63 | trimSuffix "-" }} | ||
| {{- end }} | ||
|
|
||
| {{/* | ||
| Common labels | ||
| */}} | ||
| {{- define "financeagent.labels" -}} | ||
| helm.sh/chart: {{ include "financeagent.chart" . }} | ||
| {{ include "financeagent.selectorLabels" . }} | ||
| {{- if .Chart.AppVersion }} | ||
| app.kubernetes.io/version: {{ .Chart.AppVersion | quote }} | ||
| {{- end }} | ||
| app.kubernetes.io/managed-by: {{ .Release.Service }} | ||
| {{- end }} | ||
|
|
||
| {{/* | ||
| Selector labels | ||
| */}} | ||
| {{- define "financeagent.selectorLabels" -}} | ||
| app.kubernetes.io/name: {{ include "financeagent.name" . }} | ||
| app.kubernetes.io/instance: {{ .Release.Name }} | ||
| {{- end }} | ||
|
|
||
| {{/* | ||
| Create the name of the service account to use | ||
| */}} | ||
| {{- define "financeagent.serviceAccountName" -}} | ||
| {{- if .Values.global.sharedSAName }} | ||
| {{- .Values.global.sharedSAName }} | ||
| {{- else if .Values.serviceAccount.create }} | ||
| {{- default (include "financeagent.fullname" .) .Values.serviceAccount.name }} | ||
| {{- else }} | ||
| {{- default "default" .Values.serviceAccount.name }} | ||
| {{- end }} | ||
| {{- end }} |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,16 @@ | ||
| # Copyright (C) 2024 Intel Corporation | ||
| # SPDX-License-Identifier: Apache-2.0 | ||
|
|
||
| {{- if .Values.serviceAccount.create }} | ||
| apiVersion: v1 | ||
| kind: ServiceAccount | ||
| metadata: | ||
| name: {{ include "financeagent.serviceAccountName" . }} | ||
| labels: | ||
| {{- include "financeagent.labels" . | nindent 4 }} | ||
| {{- with .Values.serviceAccount.annotations }} | ||
| annotations: | ||
| {{- toYaml . | nindent 4 }} | ||
| {{- end }} | ||
| automountServiceAccountToken: {{ .Values.serviceAccount.automount }} | ||
| {{- end }} |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.