Adding a Dockerfile to build a TGI ROCm image with an unprivileged user in a container #1638
Conversation
chyundunovDatamonsters
requested review from
XinyaoWa,
ZePan110,
chensuyue,
letonghan and
lvliang-intel
as code owners
Build and upstream latest base image on push event opea-project#1314 Signed-off-by: chensuyue <[email protected]> Signed-off-by: Chingis Yundunov <[email protected]>
…ea-project#1329) * Add timeout param for DocSum and FaqGen to deal with long context Make timeout param configurable, solve issue opea-project/GenAIExamples#1481 Signed-off-by: Xinyao Wang <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Xinyao Wang <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Chingis Yundunov <[email protected]>
Signed-off-by: Eero Tamminen <[email protected]> Signed-off-by: Chingis Yundunov <[email protected]>
Signed-off-by: chensuyue <[email protected]> Signed-off-by: Chingis Yundunov <[email protected]>
- Fix permission issue for when ingesting pptx file with embedded image - Add more test coverage to the dataprep CI and unify common dataprep CI test code for DB backends: qdrant, milvus, redis, pgvector Signed-off-by: Lianhao Lu <[email protected]> Signed-off-by: Chingis Yundunov <[email protected]>
…ect#1374) Signed-off-by: Dina Suehiro Jones <[email protected]> Signed-off-by: Chingis Yundunov <[email protected]>
Signed-off-by: Raghava, Sharath <[email protected]> Signed-off-by: Chingis Yundunov <[email protected]>
Signed-off-by: jeanyu-habana <[email protected]> Signed-off-by: Chingis Yundunov <[email protected]>
* Add Dockerfile for build ROCm vLLM Docker image Signed-off-by: Chingis Yundunov <[email protected]> Signed-off-by: Chingis Yundunov <[email protected]>
* filter none test scripts in test matrix Signed-off-by: chensuyue <[email protected]> Signed-off-by: Chingis Yundunov <[email protected]>
…ge (opea-project#1376) opea-project/GenAIExamples#1436 --------- Signed-off-by: chensuyue <[email protected]> Signed-off-by: Chingis Yundunov <[email protected]>
…opea-project#1377) * [Bug: 1375] Fix Readme errors in dataprep component for all VectorDBs Fixes opea-project#1375 Signed-off-by: Piroozan, Nariman <[email protected]> Signed-off-by: Ghosh, Soumyadip <[email protected]> Signed-off-by: Jaini, Pallavi <[email protected]> Signed-off-by: Kavulya, Soila <[email protected]> Signed-off-by: Shifani Rajabose <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: Shifani Rajabose <[email protected]> * Improve dataprep CI and fix pptx file ingesting bug (opea-project#1334) - Fix permission issue for when ingesting pptx file with embedded image - Add more test coverage to the dataprep CI and unify common dataprep CI test code for DB backends: qdrant, milvus, redis, pgvector Signed-off-by: Lianhao Lu <[email protected]> Signed-off-by: Shifani Rajabose <[email protected]> * Fix docker compose command in embedding BridgeTower readme (opea-project#1374) Signed-off-by: Dina Suehiro Jones <[email protected]> Signed-off-by: Shifani Rajabose <[email protected]> * Changes to checkin text2graph microservice (opea-project#1357) Signed-off-by: Raghava, Sharath <[email protected]> Signed-off-by: Shifani Rajabose <[email protected]> * [Bug: 1375] Fix Readme errors in dataprep component for all VectorDBs Fixes opea-project#1375 Signed-off-by: Piroozan, Nariman <[email protected]> Signed-off-by: Ghosh, Soumyadip <[email protected]> Signed-off-by: Jaini, Pallavi <[email protected]> Signed-off-by: Kavulya, Soila <[email protected]> Signed-off-by: Shifani Rajabose <[email protected]> --------- Signed-off-by: Shifani Rajabose <[email protected]> Signed-off-by: Lianhao Lu <[email protected]> Signed-off-by: Dina Suehiro Jones <[email protected]> Signed-off-by: Raghava, Sharath <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Lianhao Lu <[email protected]> Co-authored-by: Dina Suehiro Jones <[email protected]> Co-authored-by: intelsharath <[email protected]> Co-authored-by: Liang Lv <[email protected]> Signed-off-by: Chingis Yundunov <[email protected]>
Signed-off-by: Lianhao Lu <[email protected]> Signed-off-by: Chingis Yundunov <[email protected]>
Signed-off-by: letonghan <[email protected]> Signed-off-by: Chingis Yundunov <[email protected]>
Signed-off-by: Jonathan Minkin <[email protected]> Signed-off-by: Chingis Yundunov <[email protected]>
Add aiofiles in requirements.txt of retriever, which is caused by cross-component function call of retriever neo4j. Signed-off-by: letonghan <[email protected]> Signed-off-by: Chingis Yundunov <[email protected]>
…pea-project#1251) * try to leverage existed env variable instead of introducing new one Signed-off-by: Tsai, Louie <[email protected]> * remove ENABLE_OPEA_TELEMETRY getenv Signed-off-by: Tsai, Louie <[email protected]> --------- Signed-off-by: Tsai, Louie <[email protected]> Signed-off-by: Chingis Yundunov <[email protected]>
…ct#1394) Signed-off-by: Sun, Xuehao <[email protected]> Signed-off-by: Chingis Yundunov <[email protected]>
…t#1391) Signed-off-by: minmin-intel <[email protected]> Signed-off-by: Chingis Yundunov <[email protected]>
* vLLM lvm integration - integrate vLLM LVMs and set vLLM as default - use OpenAI chat completions and cover single-image/text-only cases Signed-off-by: Chingis Yundunov <[email protected]>
opea-project#1380) * [Bug: 1378] Added Multimodal support for Milvus for dataprep component Fixes opea-project#1378 Co-authored-by: Jaini, Pallavi <[email protected]> Signed-off-by: Ghosh, Soumyadip <[email protected]> Signed-off-by: Piroozan, Nariman <[email protected]> Signed-off-by: Kavulya, Soila <[email protected]> Signed-off-by: Rajabose, Shifani <[email protected]> Signed-off-by: Shifani Rajabose <[email protected]> --------- Signed-off-by: Shifani Rajabose <[email protected]> Signed-off-by: pallavi.jaini <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: pallavi.jaini <[email protected]> Signed-off-by: Chingis Yundunov <[email protected]>
Signed-off-by: Zhu, Yongbo <[email protected]> Signed-off-by: Chingis Yundunov <[email protected]>
Signed-off-by: chensuyue <[email protected]> Signed-off-by: Chingis Yundunov <[email protected]>
…roject#1440) Signed-off-by: Dina Suehiro Jones <[email protected]> Signed-off-by: Chingis Yundunov <[email protected]>
* Add LLaMA Vision OH optimization - Add mllama OH optimization - Fix LVM README and fix steps - add UT - fix 422 request body issue from wrapper to dependency - add LOGFLAG - add healthcheck - Upgrade HPU driver version - Correct compose file mllama names Signed-off-by: Chingis Yundunov <[email protected]>
…er in a container. This is necessary to ensure that the best deployment practices in K8S are followed. Signed-off-by: Chingis Yundunov <[email protected]>
…er in a container. This is necessary to ensure that the best deployment practices in K8S are followed. Signed-off-by: Chingis Yundunov <[email protected]>
…er in a container. This is necessary to ensure that the best deployment practices in K8S are followed. Signed-off-by: Chingis Yundunov <[email protected]>
…er in a container. This is necessary to ensure that the best deployment practices in K8S are followed. Signed-off-by: Chingis Yundunov <[email protected]>
…er in a container. This is necessary to ensure that the best deployment practices in K8S are followed. Signed-off-by: Chingis Yundunov <[email protected]>
…er in a container. This is necessary to ensure that the best deployment practices in K8S are followed. Signed-off-by: Chingis Yundunov <[email protected]>
…er in a container. This is necessary to ensure that the best deployment practices in K8S are followed. Signed-off-by: Chingis Yundunov <[email protected]>
…er in a container. This is necessary to ensure that the best deployment practices in K8S are followed. Signed-off-by: Chingis Yundunov <[email protected]>
…er in a container. This is necessary to ensure that the best deployment practices in K8S are followed. Signed-off-by: Chingis Yundunov <[email protected]>
…er in a container. This is necessary to ensure that the best deployment practices in K8S are followed. Signed-off-by: Chingis Yundunov <[email protected]>
…er in a container. This is necessary to ensure that the best deployment practices in K8S are followed. Signed-off-by: Chingis Yundunov <[email protected]>
|
Please confirm the changes, they block adding changes to GenAIInfra and GenAIExamples. Thanks! |
|
The tgi in helm charts use this method to run default with NonRoot, https://github.com/opea-project/GenAIInfra/blob/1e20de1016dfeae9cbb1cc00476f43b3024f55aa/helm-charts/common/tgi/values.yaml#L56, do we need this PR specific for tgi-rocm? |
|
@lianhao @yongfengdu any comments? |
I've tried this approach, but I'll try again, maybe I did something wrong. |
I tried it. This solution does not work on ROCm images. |
|
Please make these changes |
ok, then would you also update images in compose.yaml to align with helm charts values? Or this image used for helm charts only. |
|
Thanks, could you paste the issue you encountered when you try to run ROCm with non-root user?
|
The changes have been tested for Compose and for Helm. However, their use is currently planned to be implemented only in Helm. |
I will create a problem in the TGI project and attach the link here. |
Have you create the issue? Please let us know the link. |
|
I am ok with this PR, but as a workaround, why not just keep use root? opea-project/GenAIInfra#949 (comment) |
|
20 participants
Description
Adding a Dockerfile to build a TGI ROCm image with an unprivileged user in a container. This is necessary to ensure that the best deployment practices in K8S are followed.
Issues
Type of change
Dependencies
Tests