@@ -26,7 +26,7 @@ To set up environment variables for deploying ChatQnA services, follow these ste
2626 export http_proxy=" Your_HTTP_Proxy"
2727 export https_proxy=" Your_HTTPs_Proxy"
2828 # Example: no_proxy="localhost, 127.0.0.1, 192.168.1.1"
29- export no_proxy=" Your_No_Proxy" ,chatqna-gaudi-ui-server,chatqna-gaudi-backend-server,dataprep-redis-service,tei-embedding-service,retriever,tei-reranking-service,tgi-service,vllm_service,vllm-ray-service, guardrails
29+ export no_proxy=" Your_No_Proxy" ,chatqna-gaudi-ui-server,chatqna-gaudi-backend-server,dataprep-redis-service,tei-embedding-service,retriever,tei-reranking-service,tgi-service,vllm_service,guardrails
3030 ```
3131
32323 . Set up other environment variables:
@@ -227,7 +227,7 @@ For users in China who are unable to download models directly from Huggingface,
227227 export http_proxy=" Your_HTTP_Proxy"
228228 export https_proxy=" Your_HTTPs_Proxy"
229229 # Example: no_proxy="localhost, 127.0.0.1, 192.168.1.1"
230- export no_proxy=" Your_No_Proxy" ,chatqna-gaudi-ui-server,chatqna-gaudi-backend-server,dataprep-redis-service,tei-embedding-service,retriever,tei-reranking-service,tgi-service,vllm_service,vllm-ray-service, guardrails
230+ export no_proxy=" Your_No_Proxy" ,chatqna-gaudi-ui-server,chatqna-gaudi-backend-server,dataprep-redis-service,tei-embedding-service,retriever,tei-reranking-service,tgi-service,vllm_service,guardrails
231231 ` ` `
232232
2332333. Set up other environment variables:
@@ -257,12 +257,6 @@ If use vllm for llm backend.
257257docker compose -f compose_vllm.yaml up -d
258258` ` `
259259
260- If use vllm-on-ray for llm backend.
261-
262- ` ` ` bash
263- docker compose -f compose_vllm_ray.yaml up -d
264- ` ` `
265-
266260If you want to enable guardrails microservice in the pipeline, please follow the below command instead:
267261
268262` ` ` bash
@@ -351,13 +345,6 @@ For validation details, please refer to [how-to-validate_service](./how_to_valid
351345 }'
352346 ```
353347
354- ```bash
355- #vLLM-on-Ray Service
356- curl http://${host_ip}:8006/v1/chat/completions \
357- -H "Content-Type: application/json" \
358- -d ' {" model" : " ${LLM_MODEL_ID} " , " messages" : [{" role" : " user" , " content" : " What is Deep Learning?" }]}'
359- ```
360-
3613485. MegaService
362349
363350 ```bash
0 commit comments