Skip to content

Commit d3f60d6

Browse files
[pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
1 parent 6dbe635 commit d3f60d6

File tree

5 files changed

+38
-32
lines changed

5 files changed

+38
-32
lines changed

FinanceAgent/README.md

Lines changed: 8 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
# Finance Agent Example
1+
# Finance Agent Example
22

33
## Table of Contents
44

@@ -10,7 +10,6 @@
1010
- [Deployment Options](#deployment-options)
1111
- [Contribution](#contribution)
1212

13-
1413
## Overview
1514

1615
The Finance Agent exemplifies a hierarchical multi-agent system designed to streamline financial document processing and analysis for users. It offers three core functionalities: summarizing lengthy financial documents, answering queries related to these documents, and conducting research to generate investment reports on public companies.
@@ -19,8 +18,8 @@ Navigating and analyzing extensive financial documents can be both challenging a
1918

2019
Users interact with the system through a graphical user interface (UI), where a supervisor agent manages requests by delegating tasks to worker agents or the summarization microservice. The system also supports document uploads via the UI for processing.
2120

22-
2321
## Architecture
22+
2423
### High-Level Diagram
2524

2625
The architecture of this Finance Agent example is shown in the figure below. The agent is a hierarchical multi-agent system and has 3 main functions:
@@ -33,7 +32,8 @@ The user interacts with the supervisor agent through the graphical UI. The super
3332

3433
![Finance Agent Architecture](assets/finance_agent_arch.png)
3534

36-
### OPEA Microservices Diagram for Data Handling
35+
### OPEA Microservices Diagram for Data Handling
36+
3737
The architectural diagram of the `dataprep` microservice is shown below. We use [docling](https://github.com/docling-project/docling) to extract text from PDFs and URLs into markdown format. Both the full document content and tables are extracted. We then use an LLM to extract metadata from the document, including the company name, year, quarter, document type, and document title. The full document markdown then gets chunked, and LLM is used to summarize each chunk, and the summaries are embedded and saved to a vector database. Each table is also summarized by LLM and the summaries are embedded and saved to the vector database. The chunks and tables are also saved into a KV store. The pipeline is designed as such to improve retrieval accuracy of the `search_knowledge_base` tool used by the Question Answering worker agent.
3838

3939
![dataprep architecture](assets/fin_agent_dataprep.png)
@@ -52,17 +52,16 @@ The Question Answering worker agent uses `search_knowledge_base` tool to get rel
5252

5353
![finqa search tool arch](assets/finqa_tool.png)
5454

55-
5655
## Deployment Options
56+
5757
This Finance Agent example can be deployed manually on Docker Compose.
5858

59-
| Hardware | Deployment Mode | Guide Link |
60-
| :-------------- | :------------------- | :----------------------------------------------------------------------- |
59+
| Hardware | Deployment Mode | Guide Link |
60+
| :----------------------------- | :------------------- | :----------------------------------------------------------------------- |
6161
| Intel® Gaudi® AI Accelerator | Single Node (Docker) | [Gaudi Docker Compose Guide](./docker_compose/intel/hpu/gaudi/README.md) |
6262

6363
_Note: Building custom microservice images can be done using the resources in [GenAIComps](https://github.com/opea-project/GenAIComps)._
6464

65-
6665
## Contribution
67-
We welcome contributions to the OPEA project. Please refer to the [contribution guidelines](https://github.com/opea-project/docs/blob/main/community/CONTRIBUTING.md) for more information.
6866

67+
We welcome contributions to the OPEA project. Please refer to the [contribution guidelines](https://github.com/opea-project/docs/blob/main/community/CONTRIBUTING.md) for more information.

FinanceAgent/docker_compose/intel/hpu/gaudi/README.md

Lines changed: 26 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,5 @@
11
# Deploy Finance Agent on Intel® Gaudi® AI Accelerator with Docker Compose
2+
23
This README provides instructions for deploying the Finance Agent application using Docker Compose on systems equipped with Intel® Gaudi® AI Accelerators.
34

45
## Table of Contents
@@ -14,6 +15,7 @@ This README provides instructions for deploying the Finance Agent application us
1415
This guide focuses on running the pre-configured Finance Agent service using Docker Compose on Intel® Gaudi® AI Accelerators. It leverages containers optimized for Gaudi for the LLM serving component, along with CPU-based containers for other microservices like embedding, retrieval, data preparation and the UI.
1516

1617
## Prerequisites
18+
1719
- Docker and Docker Compose installed.
1820
- Intel® Gaudi® AI Accelerator(s) with the necessary drivers and software stack installed on the host system. (Refer to Intel Gaudi Documentation).
1921
- Git installed (for cloning repository).
@@ -31,8 +33,11 @@ Clone the GenAIExamples repository:
3133
```
3234

3335
## Start Deployment
36+
3437
This uses the default vLLM-based deployment profile (vllm-gaudi-server).
38+
3539
### Configure Environment
40+
3641
Set required environment variables in your shell:
3742

3843
```shell
@@ -41,12 +46,12 @@ Set required environment variables in your shell:
4146
# Path to your model cache
4247
export HF_CACHE_DIR="./data"
4348
# Go to https://finnhub.io/ to get your free api key
44-
export FINNHUB_API_KEY=<your-finnhub-api-key>
49+
export FINNHUB_API_KEY=<your-finnhub-api-key>
4550
# Go to https://docs.financialdatasets.ai/ to get your free api key
46-
export FINANCIAL_DATASETS_API_KEY=<your-api-key>
51+
export FINANCIAL_DATASETS_API_KEY=<your-api-key>
4752

4853
# Optional: Configure HOST_IP if needed
49-
# Replace with your host's external IP address (do not use localhost or 127.0.0.1).
54+
# Replace with your host's external IP address (do not use localhost or 127.0.0.1).
5055
# export HOST_IP=$(hostname -I | awk '{print $1}')
5156

5257
# Optional: Configure proxy if needed
@@ -60,19 +65,21 @@ Set required environment variables in your shell:
6065
Note: The compose file might read additional variables from set_env.sh. Ensure all required variables like ports (LLM_SERVICE_PORT, TEI_EMBEDDER_PORT, etc.) are set if not using defaults from the compose file. For instance, edit the set_env.sh to change the LLM model:
6166

6267
### Start Services
68+
6369
#### Deploy with Docker Compose
70+
6471
Below is the command to launch services
65-
- vllm-gaudi-server
66-
- tei-embedding-serving
67-
- redis-vector-db
68-
- redis-kv-store
69-
- dataprep-redis-server-finance
70-
- finqa-agent-endpoint
71-
- research-agent-endpoint
72-
- docsum-vllm-gaudi
73-
- supervisor-agent-endpoint
74-
- agent-ui
7572

73+
- vllm-gaudi-server
74+
- tei-embedding-serving
75+
- redis-vector-db
76+
- redis-kv-store
77+
- dataprep-redis-server-finance
78+
- finqa-agent-endpoint
79+
- research-agent-endpoint
80+
- docsum-vllm-gaudi
81+
- supervisor-agent-endpoint
82+
- agent-ui
7683

7784
```shell
7885
docker compose -f compose.yaml up -d
@@ -103,14 +110,16 @@ If deploy on Gaudi, also need to build vllm image.
103110
docker build --no-cache -f Dockerfile.hpu -t opea/vllm-gaudi:latest --shm-size=128g . --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy
104111
```
105112

106-
107113
## Validate Services
114+
108115
Wait several minutes for models to download and services to initialize (Gaudi initialization can take time). Check container logs (docker compose logs -f <service_name>, especially vllm-gaudi-server).
109116

110117
```bash
111118
docker logs --tail 2000 -f vllm-gaudi-server
112-
```
113-
> Expected output of the `vllm-gaudi-server` service is
119+
```
120+
121+
> Expected output of the `vllm-gaudi-server` service is
122+
114123
```
115124
INFO: Started server process [1]
116125
INFO: Waiting for application startup.
@@ -121,6 +130,7 @@ Wait several minutes for models to download and services to initialize (Gaudi in
121130
```
122131

123132
### Validate Data Services
133+
124134
Ingest data and retrieval from database
125135

126136
```bash

FinanceAgent/docker_compose/intel/hpu/gaudi/compose.yaml

Lines changed: 1 addition & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -118,7 +118,7 @@ services:
118118
REDIS_URL_KV: ${REDIS_URL_KV}
119119
TEI_EMBEDDING_ENDPOINT: ${TEI_EMBEDDING_ENDPOINT}
120120
LLM_ENDPOINT: ${LLM_ENDPOINT}
121-
LLM_MODEL: ${LLM_MODEL_ID}
121+
LLM_MODEL: ${LLM_MODEL_ID}
122122
HUGGINGFACEHUB_API_TOKEN: ${HF_TOKEN}
123123
HF_TOKEN: ${HF_TOKEN}
124124
LOGFLAG: true
@@ -215,5 +215,3 @@ services:
215215
ports:
216216
- "5175:8080"
217217
ipc: host
218-
219-

FinanceAgent/docker_compose/intel/set_env.sh

Lines changed: 2 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -23,7 +23,7 @@ check_var "HF_TOKEN"
2323
check_var "HOST_IP"
2424

2525
# VLLM configuration
26-
export VLLM_PORT="${VLLM_PORT:-8086}"
26+
export VLLM_PORT="${VLLM_PORT:-8086}"
2727
export VLLM_VOLUME="${VLLM_VOLUME:-/data2/huggingface}"
2828
export VLLM_IMAGE="${VLLM_IMAGE:-opea/vllm-gaudi:latest}"
2929
export LLM_MODEL_ID="${LLM_MODEL_ID:-meta-llama/Llama-3.3-70B-Instruct}"
@@ -70,7 +70,7 @@ export FINANCIAL_DATASETS_API_KEY="${FINANCIAL_DATASETS_API_KEY}"
7070

7171

7272
# Toolset and prompt paths
73-
if check_var "WORKDIR"; then
73+
if check_var "WORKDIR"; then
7474
export TOOLSET_PATH=$WORKDIR/GenAIExamples/FinanceAgent/tools/
7575
export PROMPT_PATH=$WORKDIR/GenAIExamples/FinanceAgent/prompts/
7676

@@ -87,4 +87,3 @@ if check_var "WORKDIR"; then
8787
fi
8888
done
8989
fi
90-

FinanceAgent/tests/test_compose_on_gaudi.sh

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -17,7 +17,7 @@ export HTTPS_PROXY="${https_proxy}"
1717

1818
# VLLM configuration
1919
MODEL=meta-llama/Llama-3.3-70B-Instruct
20-
export VLLM_PORT="${VLLM_PORT:-8086}"
20+
export VLLM_PORT="${VLLM_PORT:-8086}"
2121

2222
# export HF_CACHE_DIR="${HF_CACHE_DIR:-"./data"}"
2323
export HF_CACHE_DIR=${model_cache:-"./data2/huggingface"}

0 commit comments

Comments
 (0)