|
3 | 3 | This document outlines the deployment process for a CodeGen application utilizing the [GenAIComps](https://github.com/opea-project/GenAIComps.git) microservice pipeline on Intel Xeon server. The steps include Docker images creation, container deployment via Docker Compose, and service execution to integrate microservices such as `llm`. We will publish the Docker images to Docker Hub soon, further simplifying the deployment process for this service. |
4 | 4 | The default pipeline deploys with vLLM as the LLM serving component. It also provides options of using TGI backend for LLM microservice. |
5 | 5 |
|
6 | | -## 🚀 Create an AWS Xeon Instance |
7 | | - |
8 | | -To run the example on an AWS Xeon instance, start by creating an AWS account if you don't have one already. Then, get started with the [EC2 Console](https://console.aws.amazon.com/ec2/v2/home). AWS EC2 M7i, C7i, C7i-flex and M7i-flex instances are 4th Generation Intel Xeon Scalable processors suitable for the task. |
9 | | - |
10 | | -For detailed information about these instance types, you can refer to [m7i](https://aws.amazon.com/ec2/instance-types/m7i/). Once you've chosen the appropriate instance type, proceed with configuring your instance settings, including network configurations, security groups, and storage options. |
11 | | - |
12 | | -After launching your instance, you can connect to it using SSH (for Linux instances) or Remote Desktop Protocol (RDP) (for Windows instances). From there, you'll have full access to your Xeon server, allowing you to install, configure, and manage your applications as needed. |
13 | | - |
14 | | -## 🚀 Start Microservices and MegaService |
15 | | - |
16 | | -The CodeGen megaservice manages a single microservice called LLM within a Directed Acyclic Graph (DAG). In the diagram above, the LLM microservice is a language model microservice that generates code snippets based on the user's input query. The TGI service serves as a text generation interface, providing a RESTful API for the LLM microservice. The CodeGen Gateway acts as the entry point for the CodeGen application, invoking the Megaservice to generate code snippets in response to the user's input query. |
17 | | - |
18 | | -The mega flow of the CodeGen application, from user's input query to the application's output response, is as follows: |
19 | | - |
20 | | -```mermaid |
21 | | -flowchart LR |
22 | | - subgraph CodeGen |
23 | | - direction LR |
24 | | - A[User] --> |Input query| B[CodeGen Gateway] |
25 | | - B --> |Invoke| Megaservice |
26 | | - subgraph Megaservice["Megaservice"] |
27 | | - direction TB |
28 | | - C((LLM<br>9000)) -. Post .-> D{{TGI Service<br>8028}} |
29 | | - end |
30 | | - Megaservice --> |Output| E[Response] |
31 | | - end |
32 | | -
|
33 | | - subgraph Legend |
34 | | - direction LR |
35 | | - G([Microservice]) ==> H([Microservice]) |
36 | | - I([Microservice]) -.-> J{{Server API}} |
37 | | - end |
38 | | -``` |
39 | | - |
40 | 6 | ### Setup Environment Variables |
41 | 7 |
|
42 | 8 | Since the `compose.yaml` will consume some environment variables, you need to setup them in advance as below. |
@@ -117,17 +83,12 @@ To access the frontend, open the following URL in your browser: `http://{host_ip |
117 | 83 |
|
118 | 84 | ```yaml |
119 | 85 | codegen-xeon-ui-server: |
120 | | - image: opea/codegen-ui:latest |
| 86 | + image: opea/codegen-gradio-ui:latest |
121 | 87 | ... |
122 | 88 | ports: |
123 | 89 | - "80:5173" |
124 | 90 | ``` |
125 | 91 |
|
126 | | - |
127 | | -
|
128 | | -Here is an example of running CodeGen in the UI: |
129 | | -
|
130 | | - |
131 | 92 |
|
132 | 93 | ## 🚀 Launch the React Based UI (Optional) |
133 | 94 |
|
@@ -256,20 +217,20 @@ cd GenAIExamples/CodeGen/ui |
256 | 217 | docker build -t opea/codegen-ui:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy -f ./docker/Dockerfile . |
257 | 218 | ``` |
258 | 219 |
|
259 | | -### 4. Build CodeGen React UI Docker Image (Optional) |
| 220 | +### 4. Build CodeGen Gradio UI Docker Image |
260 | 221 |
|
261 | | -Build react frontend Docker image via below command: |
| 222 | +Build gradio frontend Docker image via below command: |
262 | 223 |
|
263 | 224 | **Export the value of the public IP address of your Xeon server to the `host_ip` environment variable** |
264 | 225 |
|
265 | 226 | ```bash |
266 | 227 | cd GenAIExamples/CodeGen/ui |
267 | | -docker build --no-cache -t opea/codegen-react-ui:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy -f ./docker/Dockerfile.react . |
| 228 | +docker build --no-cache -t opea/codegen-gradio-ui:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy -f ./docker/Dockerfile.gradio . |
268 | 229 | ``` |
269 | 230 |
|
270 | 231 | Then run the command `docker images`, you will have the following Docker Images: |
271 | 232 |
|
272 | 233 | - `opea/llm-textgen:latest` |
273 | 234 | - `opea/codegen:latest` |
274 | 235 | - `opea/codegen-ui:latest` |
275 | | -- `opea/codegen-react-ui:latest` (optional) |
| 236 | +- `opea/codegen-gradio-ui:latest` |
0 commit comments