Skip to content

Commit f33ba5a

Browse files
MSCetin37MSCetin37
committed
update readme
Signed-off-by: Mustafa <[email protected]>
1 parent 8875802 commit f33ba5a

File tree

3 files changed

+159
-29
lines changed

3 files changed

+159
-29
lines changed

‎CodeGen/README.md‎

Lines changed: 17 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -58,7 +58,7 @@ flowchart LR
5858
V_RET{{Retriever<br>service}}
5959
Ingest{{Ingest data}}
6060
DP([Data Preparation]):::blue
61-
LLM_gen{{LLM Service}}
61+
LLM_gen{{TGI Service}}
6262
GW([CodeGen GateWay]):::orange
6363
6464
%% Data Preparation flow
@@ -161,11 +161,25 @@ Refer to the [Gaudi Guide](./docker_compose/intel/hpu/gaudi/README.md) to build
161161

162162
Find the corresponding [compose.yaml](./docker_compose/intel/cpu/xeon/compose.yaml).
163163

164+
Start CodeGen based on TGI service:
165+
164166
```bash
165-
cd GenAIExamples/CodeGen/docker_compose/intel/cpu/xeon
166-
docker compose up -d
167+
cd GenAIExamples/CodeGen/docker_compose
168+
source set_env.sh
169+
cd intel/cpu/xeon
170+
docker compose --profile codegen-xeon-tgi up -d
167171
```
168172

173+
Start CodeGen based on vLLM service:
174+
175+
```bash
176+
cd GenAIExamples/CodeGen/docker_compose
177+
source set_env.sh
178+
cd intel/cpu/xeon
179+
docker compose --profile codegen-xeon-vllm up -d
180+
```
181+
182+
169183
Refer to the [Xeon Guide](./docker_compose/intel/cpu/xeon/README.md) for more instructions on building docker images from source.
170184

171185
### Deploy CodeGen on Kubernetes using Helm Chart

‎CodeGen/docker_compose/intel/cpu/xeon/README.md‎

Lines changed: 71 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -13,28 +13,77 @@ After launching your instance, you can connect to it using SSH (for Linux instan
1313

1414
## 🚀 Start Microservices and MegaService
1515

16-
The CodeGen megaservice manages a single microservice called LLM within a Directed Acyclic Graph (DAG). In the diagram above, the LLM microservice is a language model microservice that generates code snippets based on the user's input query. The TGI service serves as a text generation interface, providing a RESTful API for the LLM microservice. The CodeGen Gateway acts as the entry point for the CodeGen application, invoking the Megaservice to generate code snippets in response to the user's input query.
16+
The CodeGen megaservice manages a several microservices including 'Embedding MicroService', 'Retrieval MicroService' and 'LLM MicroService' within a Directed Acyclic Graph (DAG). In the diagram below, the LLM microservice is a language model microservice that generates code snippets based on the user's input query. The TGI service serves as a text generation interface, providing a RESTful API for the LLM microservice. Data Preparation allows users to save/update documents or online resources to the vector database. Users can upload files or provide URLs, and manage their saved resources. The CodeGen Gateway acts as the entry point for the CodeGen application, invoking the Megaservice to generate code snippets in response to the user's input query.
1717

1818
The mega flow of the CodeGen application, from user's input query to the application's output response, is as follows:
1919

2020
```mermaid
21+
---
22+
config:
23+
flowchart:
24+
nodeSpacing: 400
25+
rankSpacing: 100
26+
curve: linear
27+
themeVariables:
28+
fontSize: 25px
29+
---
2130
flowchart LR
22-
subgraph CodeGen
31+
%% Colors %%
32+
classDef blue fill:#ADD8E6,stroke:#ADD8E6,stroke-width:2px,fill-opacity:0.5
33+
classDef orange fill:#FBAA60,stroke:#ADD8E6,stroke-width:2px,fill-opacity:0.5
34+
classDef orchid fill:#C26DBC,stroke:#ADD8E6,stroke-width:2px,fill-opacity:0.5
35+
classDef invisible fill:transparent,stroke:transparent;
36+
style CodeGen-MegaService stroke:#000000
37+
%% Subgraphs %%
38+
subgraph CodeGen-MegaService["CodeGen-MegaService"]
2339
direction LR
24-
A[User] --> |Input query| B[CodeGen Gateway]
25-
B --> |Invoke| Megaservice
26-
subgraph Megaservice["Megaservice"]
27-
direction TB
28-
C((LLM<br>9000)) -. Post .-> D{{TGI Service<br>8028}}
29-
end
30-
Megaservice --> |Output| E[Response]
40+
EM([Embedding<br>MicroService]):::blue
41+
RET([Retrieval<br>MicroService]):::blue
42+
RER([Agents]):::blue
43+
LLM([LLM<br>MicroService]):::blue
3144
end
32-
33-
subgraph Legend
45+
subgraph User Interface
3446
direction LR
35-
G([Microservice]) ==> H([Microservice])
36-
I([Microservice]) -.-> J{{Server API}}
47+
a([Submit Query Tab]):::orchid
48+
UI([UI server]):::orchid
49+
Ingest([Manage Resources]):::orchid
3750
end
51+
52+
CLIP_EM{{Embedding<br>service}}
53+
VDB{{Vector DB}}
54+
V_RET{{Retriever<br>service}}
55+
Ingest{{Ingest data}}
56+
DP([Data Preparation]):::blue
57+
LLM_gen{{TGI Service}}
58+
GW([CodeGen GateWay]):::orange
59+
60+
%% Data Preparation flow
61+
%% Ingest data flow
62+
direction LR
63+
Ingest[Ingest data] --> UI
64+
UI --> DP
65+
DP <-.-> CLIP_EM
66+
67+
%% Questions interaction
68+
direction LR
69+
a[User Input Query] --> UI
70+
UI --> GW
71+
GW <==> CodeGen-MegaService
72+
EM ==> RET
73+
RET ==> RER
74+
RER ==> LLM
75+
76+
77+
%% Embedding service flow
78+
direction LR
79+
EM <-.-> CLIP_EM
80+
RET <-.-> V_RET
81+
LLM <-.-> LLM_gen
82+
83+
direction TB
84+
%% Vector DB interaction
85+
V_RET <-.->VDB
86+
DP <-.->VDB
3887
```
3988

4089
### Setup Environment Variables
@@ -111,6 +160,15 @@ docker compose --profile codegen-xeon-vllm up -d
111160
}'
112161
```
113162

163+
If the user wants a CodeGen service with RAG and Agents based on dedicated documentation.
164+
165+
```bash
166+
curl http://localhost:7778/v1/codegen \
167+
-H "Content-Type: application/json" \
168+
-d '{"agents_flag": "True", "index_name": "my_API_document", "messages": "Implement a high-level API for a TODO list application. The API takes as input an operation request and updates the TODO list in place. If the request is invalid, raise an exception."}'
169+
```
170+
171+
114172
## 🚀 Launch the UI
115173

116174
To access the frontend, open the following URL in your browser: `http://{host_ip}:5173`. By default, the UI runs on port 5173 internally. If you prefer to use a different host port to access the frontend, you can modify the port mapping in the `compose.yaml` file as shown below:

‎CodeGen/docker_compose/intel/hpu/gaudi/README.md‎

Lines changed: 71 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -6,28 +6,77 @@ The default pipeline deploys with vLLM as the LLM serving component. It also pro
66

77
## 🚀 Start MicroServices and MegaService
88

9-
The CodeGen megaservice manages a single microservice called LLM within a Directed Acyclic Graph (DAG). In the diagram above, the LLM microservice is a language model microservice that generates code snippets based on the user's input query. The TGI service serves as a text generation interface, providing a RESTful API for the LLM microservice. The CodeGen Gateway acts as the entry point for the CodeGen application, invoking the Megaservice to generate code snippets in response to the user's input query.
9+
The CodeGen megaservice manages a several microservices including 'Embedding MicroService', 'Retrieval MicroService' and 'LLM MicroService' within a Directed Acyclic Graph (DAG). In the diagram below, the LLM microservice is a language model microservice that generates code snippets based on the user's input query. The TGI service serves as a text generation interface, providing a RESTful API for the LLM microservice. Data Preparation allows users to save/update documents or online resources to the vector database. Users can upload files or provide URLs, and manage their saved resources. The CodeGen Gateway acts as the entry point for the CodeGen application, invoking the Megaservice to generate code snippets in response to the user's input query.
1010

1111
The mega flow of the CodeGen application, from user's input query to the application's output response, is as follows:
1212

1313
```mermaid
14+
---
15+
config:
16+
flowchart:
17+
nodeSpacing: 400
18+
rankSpacing: 100
19+
curve: linear
20+
themeVariables:
21+
fontSize: 25px
22+
---
1423
flowchart LR
15-
subgraph CodeGen
24+
%% Colors %%
25+
classDef blue fill:#ADD8E6,stroke:#ADD8E6,stroke-width:2px,fill-opacity:0.5
26+
classDef orange fill:#FBAA60,stroke:#ADD8E6,stroke-width:2px,fill-opacity:0.5
27+
classDef orchid fill:#C26DBC,stroke:#ADD8E6,stroke-width:2px,fill-opacity:0.5
28+
classDef invisible fill:transparent,stroke:transparent;
29+
style CodeGen-MegaService stroke:#000000
30+
%% Subgraphs %%
31+
subgraph CodeGen-MegaService["CodeGen-MegaService"]
1632
direction LR
17-
A[User] --> |Input query| B[CodeGen Gateway]
18-
B --> |Invoke| Megaservice
19-
subgraph Megaservice["Megaservice"]
20-
direction TB
21-
C((LLM<br>9000)) -. Post .-> D{{TGI Service<br>8028}}
22-
end
23-
Megaservice --> |Output| E[Response]
33+
EM([Embedding<br>MicroService]):::blue
34+
RET([Retrieval<br>MicroService]):::blue
35+
RER([Agents]):::blue
36+
LLM([LLM<br>MicroService]):::blue
2437
end
25-
26-
subgraph Legend
38+
subgraph User Interface
2739
direction LR
28-
G([Microservice]) ==> H([Microservice])
29-
I([Microservice]) -.-> J{{Server API}}
40+
a([Submit Query Tab]):::orchid
41+
UI([UI server]):::orchid
42+
Ingest([Manage Resources]):::orchid
3043
end
44+
45+
CLIP_EM{{Embedding<br>service}}
46+
VDB{{Vector DB}}
47+
V_RET{{Retriever<br>service}}
48+
Ingest{{Ingest data}}
49+
DP([Data Preparation]):::blue
50+
LLM_gen{{TGI Service}}
51+
GW([CodeGen GateWay]):::orange
52+
53+
%% Data Preparation flow
54+
%% Ingest data flow
55+
direction LR
56+
Ingest[Ingest data] --> UI
57+
UI --> DP
58+
DP <-.-> CLIP_EM
59+
60+
%% Questions interaction
61+
direction LR
62+
a[User Input Query] --> UI
63+
UI --> GW
64+
GW <==> CodeGen-MegaService
65+
EM ==> RET
66+
RET ==> RER
67+
RER ==> LLM
68+
69+
70+
%% Embedding service flow
71+
direction LR
72+
EM <-.-> CLIP_EM
73+
RET <-.-> V_RET
74+
LLM <-.-> LLM_gen
75+
76+
direction TB
77+
%% Vector DB interaction
78+
V_RET <-.->VDB
79+
DP <-.->VDB
3180
```
3281

3382
### Setup Environment Variables
@@ -104,6 +153,15 @@ docker compose --profile codegen-gaudi-vllm up -d
104153
}'
105154
```
106155

156+
If the user wants a CodeGen service with RAG and Agents based on dedicated documentation.
157+
158+
```bash
159+
curl http://localhost:7778/v1/codegen \
160+
-H "Content-Type: application/json" \
161+
-d '{"agents_flag": "True", "index_name": "my_API_document", "messages": "Implement a high-level API for a TODO list application. The API takes as input an operation request and updates the TODO list in place. If the request is invalid, raise an exception."}'
162+
```
163+
164+
107165
## 🚀 Launch the Svelte Based UI
108166

109167
To access the frontend, open the following URL in your browser: `http://{host_ip}:5173`. By default, the UI runs on port 5173 internally. If you prefer to use a different host port to access the frontend, you can modify the port mapping in the `compose.yaml` file as shown below:

0 commit comments

Comments
 (0)