You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: FinanceAgent/README.md
+8-9Lines changed: 8 additions & 9 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,4 +1,4 @@
1
-
# Finance Agent Example
1
+
# Finance Agent Example
2
2
3
3
## Table of Contents
4
4
@@ -10,7 +10,6 @@
10
10
-[Deployment Options](#deployment-options)
11
11
-[Contribution](#contribution)
12
12
13
-
14
13
## Overview
15
14
16
15
The Finance Agent exemplifies a hierarchical multi-agent system designed to streamline financial document processing and analysis for users. It offers three core functionalities: summarizing lengthy financial documents, answering queries related to these documents, and conducting research to generate investment reports on public companies.
@@ -19,8 +18,8 @@ Navigating and analyzing extensive financial documents can be both challenging a
19
18
20
19
Users interact with the system through a graphical user interface (UI), where a supervisor agent manages requests by delegating tasks to worker agents or the summarization microservice. The system also supports document uploads via the UI for processing.
21
20
22
-
23
21
## Architecture
22
+
24
23
### High-Level Diagram
25
24
26
25
The architecture of this Finance Agent example is shown in the figure below. The agent is a hierarchical multi-agent system and has 3 main functions:
@@ -33,7 +32,8 @@ The user interacts with the supervisor agent through the graphical UI. The super
The architectural diagram of the `dataprep` microservice is shown below. We use [docling](https://github.com/docling-project/docling) to extract text from PDFs and URLs into markdown format. Both the full document content and tables are extracted. We then use an LLM to extract metadata from the document, including the company name, year, quarter, document type, and document title. The full document markdown then gets chunked, and LLM is used to summarize each chunk, and the summaries are embedded and saved to a vector database. Each table is also summarized by LLM and the summaries are embedded and saved to the vector database. The chunks and tables are also saved into a KV store. The pipeline is designed as such to improve retrieval accuracy of the `search_knowledge_base` tool used by the Question Answering worker agent.
| Intel® Gaudi® AI Accelerator | Single Node (Docker) |[Gaudi Docker Compose Guide](./docker_compose/intel/hpu/gaudi/README.md)|
62
62
63
63
_Note: Building custom microservice images can be done using the resources in [GenAIComps](https://github.com/opea-project/GenAIComps)._
64
64
65
-
66
65
## Contribution
67
-
We welcome contributions to the OPEA project. Please refer to the [contribution guidelines](https://github.com/opea-project/docs/blob/main/community/CONTRIBUTING.md) for more information.
68
66
67
+
We welcome contributions to the OPEA project. Please refer to the [contribution guidelines](https://github.com/opea-project/docs/blob/main/community/CONTRIBUTING.md) for more information.
Copy file name to clipboardExpand all lines: FinanceAgent/docker_compose/intel/hpu/gaudi/README.md
+26-16Lines changed: 26 additions & 16 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,4 +1,5 @@
1
1
# Deploy Finance Agent on Intel® Gaudi® AI Accelerator with Docker Compose
2
+
2
3
This README provides instructions for deploying the Finance Agent application using Docker Compose on systems equipped with Intel® Gaudi® AI Accelerators.
3
4
4
5
## Table of Contents
@@ -14,6 +15,7 @@ This README provides instructions for deploying the Finance Agent application us
14
15
This guide focuses on running the pre-configured Finance Agent service using Docker Compose on Intel® Gaudi® AI Accelerators. It leverages containers optimized for Gaudi for the LLM serving component, along with CPU-based containers for other microservices like embedding, retrieval, data preparation and the UI.
15
16
16
17
## Prerequisites
18
+
17
19
- Docker and Docker Compose installed.
18
20
- Intel® Gaudi® AI Accelerator(s) with the necessary drivers and software stack installed on the host system. (Refer to Intel Gaudi Documentation).
19
21
- Git installed (for cloning repository).
@@ -31,8 +33,11 @@ Clone the GenAIExamples repository:
31
33
```
32
34
33
35
## Start Deployment
36
+
34
37
This uses the default vLLM-based deployment profile (vllm-gaudi-server).
38
+
35
39
### Configure Environment
40
+
36
41
Set required environment variables in your shell:
37
42
38
43
```shell
@@ -41,12 +46,12 @@ Set required environment variables in your shell:
41
46
# Path to your model cache
42
47
export HF_CACHE_DIR="./data"
43
48
# Go to https://finnhub.io/ to get your free api key
44
-
export FINNHUB_API_KEY=<your-finnhub-api-key>
49
+
export FINNHUB_API_KEY=<your-finnhub-api-key>
45
50
# Go to https://docs.financialdatasets.ai/ to get your free api key
46
-
export FINANCIAL_DATASETS_API_KEY=<your-api-key>
51
+
export FINANCIAL_DATASETS_API_KEY=<your-api-key>
47
52
48
53
# Optional: Configure HOST_IP if needed
49
-
# Replace with your host's external IP address (do not use localhost or 127.0.0.1).
54
+
# Replace with your host's external IP address (do not use localhost or 127.0.0.1).
@@ -60,19 +65,21 @@ Set required environment variables in your shell:
60
65
Note: The compose file might read additional variables from set_env.sh. Ensure all required variables like ports (LLM_SERVICE_PORT, TEI_EMBEDDER_PORT, etc.) are set if not using defaults from the compose file. For instance, edit the set_env.sh to change the LLM model:
61
66
62
67
### Start Services
68
+
63
69
#### Deploy with Docker Compose
70
+
64
71
Below is the command to launch services
65
-
- vllm-gaudi-server
66
-
- tei-embedding-serving
67
-
- redis-vector-db
68
-
- redis-kv-store
69
-
- dataprep-redis-server-finance
70
-
- finqa-agent-endpoint
71
-
- research-agent-endpoint
72
-
- docsum-vllm-gaudi
73
-
- supervisor-agent-endpoint
74
-
- agent-ui
75
72
73
+
- vllm-gaudi-server
74
+
- tei-embedding-serving
75
+
- redis-vector-db
76
+
- redis-kv-store
77
+
- dataprep-redis-server-finance
78
+
- finqa-agent-endpoint
79
+
- research-agent-endpoint
80
+
- docsum-vllm-gaudi
81
+
- supervisor-agent-endpoint
82
+
- agent-ui
76
83
77
84
```shell
78
85
docker compose -f compose.yaml up -d
@@ -103,14 +110,16 @@ If deploy on Gaudi, also need to build vllm image.
Wait several minutes for models to download and services to initialize (Gaudi initialization can take time). Check container logs (docker compose logs -f <service_name>, especially vllm-gaudi-server).
109
116
110
117
```bash
111
118
docker logs --tail 2000 -f vllm-gaudi-server
112
-
```
113
-
> Expected output of the `vllm-gaudi-server` service is
119
+
```
120
+
121
+
> Expected output of the `vllm-gaudi-server` service is
122
+
114
123
```
115
124
INFO: Started server process [1]
116
125
INFO: Waiting for application startup.
@@ -121,6 +130,7 @@ Wait several minutes for models to download and services to initialize (Gaudi in
0 commit comments