Skip to content

Commit 40386d9

Browse files
authored
remove vllm-on-ray (opea-project#1084)
Signed-off-by: Xinyao Wang <[email protected]>
1 parent fe97e88 commit 40386d9

File tree

5 files changed

+2
-369
lines changed

5 files changed

+2
-369
lines changed

ChatQnA/docker_compose/intel/hpu/gaudi/README.md

Lines changed: 2 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -26,7 +26,7 @@ To set up environment variables for deploying ChatQnA services, follow these ste
2626
export http_proxy="Your_HTTP_Proxy"
2727
export https_proxy="Your_HTTPs_Proxy"
2828
# Example: no_proxy="localhost, 127.0.0.1, 192.168.1.1"
29-
export no_proxy="Your_No_Proxy",chatqna-gaudi-ui-server,chatqna-gaudi-backend-server,dataprep-redis-service,tei-embedding-service,retriever,tei-reranking-service,tgi-service,vllm_service,vllm-ray-service,guardrails
29+
export no_proxy="Your_No_Proxy",chatqna-gaudi-ui-server,chatqna-gaudi-backend-server,dataprep-redis-service,tei-embedding-service,retriever,tei-reranking-service,tgi-service,vllm_service,guardrails
3030
```
3131

3232
3. Set up other environment variables:
@@ -227,7 +227,7 @@ For users in China who are unable to download models directly from Huggingface,
227227
export http_proxy="Your_HTTP_Proxy"
228228
export https_proxy="Your_HTTPs_Proxy"
229229
# Example: no_proxy="localhost, 127.0.0.1, 192.168.1.1"
230-
export no_proxy="Your_No_Proxy",chatqna-gaudi-ui-server,chatqna-gaudi-backend-server,dataprep-redis-service,tei-embedding-service,retriever,tei-reranking-service,tgi-service,vllm_service,vllm-ray-service,guardrails
230+
export no_proxy="Your_No_Proxy",chatqna-gaudi-ui-server,chatqna-gaudi-backend-server,dataprep-redis-service,tei-embedding-service,retriever,tei-reranking-service,tgi-service,vllm_service,guardrails
231231
```
232232

233233
3. Set up other environment variables:
@@ -257,12 +257,6 @@ If use vllm for llm backend.
257257
docker compose -f compose_vllm.yaml up -d
258258
```
259259

260-
If use vllm-on-ray for llm backend.
261-
262-
```bash
263-
docker compose -f compose_vllm_ray.yaml up -d
264-
```
265-
266260
If you want to enable guardrails microservice in the pipeline, please follow the below command instead:
267261

268262
```bash
@@ -351,13 +345,6 @@ For validation details, please refer to [how-to-validate_service](./how_to_valid
351345
}'
352346
```
353347
354-
```bash
355-
#vLLM-on-Ray Service
356-
curl http://${host_ip}:8006/v1/chat/completions \
357-
-H "Content-Type: application/json" \
358-
-d '{"model": "${LLM_MODEL_ID}", "messages": [{"role": "user", "content": "What is Deep Learning?"}]}'
359-
```
360-
361348
5. MegaService
362349
363350
```bash

ChatQnA/docker_compose/intel/hpu/gaudi/compose_vllm_ray.yaml

Lines changed: 0 additions & 164 deletions
This file was deleted.

ChatQnA/docker_image_build/build.yaml

Lines changed: 0 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -77,12 +77,6 @@ services:
7777
dockerfile: comps/llms/text-generation/vllm/langchain/Dockerfile
7878
extends: chatqna
7979
image: ${REGISTRY:-opea}/llm-vllm:${TAG:-latest}
80-
llm-vllm-ray-hpu:
81-
build:
82-
context: GenAIComps
83-
dockerfile: comps/llms/text-generation/vllm/ray/dependency/Dockerfile
84-
extends: chatqna
85-
image: ${REGISTRY:-opea}/llm-vllm-ray-hpu:${TAG:-latest}
8680
dataprep-redis:
8781
build:
8882
context: GenAIComps

0 commit comments

Comments
 (0)