Skip to content

Conversation

@chensuyue
Copy link
Collaborator

@chensuyue chensuyue commented Jun 11, 2025

Description

UV provides a drop-in replacement for common pip, pip-tools commands, deps installation experience a 10-100x speedup with the uv pip interface. https://docs.astral.sh/uv/.

  • Purpose
  1. Accelerate pip install with uv.
    For examples: docker build -t opea/build:test -f comps/retrievers/src/Dockerfile .
    uv install Vs. pip install is 30.1s Vs.173.6s.
# pip install logs
 => [5/7] RUN pip install --no-cache-dir --upgrade pip setuptools uv &&     if [ cpu = "cpu" ]; then       uv pip install --system --no-cache-dir torch torchvision --index-url https://download.pytorch.org/whl/cpu;       uv pip ins  30.1s
# uv install logs
 => [5/7] RUN pip install --no-cache-dir --upgrade pip setuptools &&     if [ cpu = "cpu" ]; then         PIP_EXTRA_INDEX_URL="--extra-index-url https://download.pytorch.org/whl/cpu";     else         PIP_EXTRA_INDEX_URL="";       173.6s
  1. Freeze deps version to avoid issues involved by regular dependency version update.
    For example, transformers, langchain, huggingface_hub updated with high frequency.
  • Steps
  1. Rename current requirements.txt to requirements.in as dependency template, including the first level dependencies.
  2. Run uv pip compile generate requirements.txt or requirements-cpu.txt and requirements-gpu.txt for different HW ENV. Full CMD is $ uv pip compile requirements.in --universal -o requirements.txt --upgrade, requirements.in works as a dependency template, --universal means cross platform, --upgrade means overwrite requirements.txt with latest version.
    CPU version CMD is $ uv pip compile --extra-index-url https://download.pytorch.org/whl/cpu --index-strategy unsafe-best-match requirements.in --universal -o requirements-cpu.txt --upgrade
  3. Update Dockerfile, use uv to install requirements.txt.
  • TODO
  1. Enable a CI/CD workflow to upgrade the deps version regularly.
  2. Remove some hardcode version limit in requirements.in and test with CI. Module dependency version differences #1775
  3. Add Readme of how to generate requirements.txt.

Issues

List the issue or RFC link this PR is working on. If there is no such link, please mark it as n/a.

Type of change

List the type of change like below. Please delete options that are not relevant.

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds new functionality)
  • Breaking change (fix or feature that would break existing design and interface)
  • Others (enhancement, documentation, validation, etc.)

Dependencies

List the newly introduced 3rd party dependency if exists.

Tests

Describe the tests that you ran to verify your changes.

@chensuyue chensuyue added the WIP label Jun 11, 2025
@chensuyue chensuyue marked this pull request as draft June 12, 2025 05:35
@chensuyue chensuyue force-pushed the suyue/uv branch 2 times, most recently from 6295b9e to aee7c1b Compare June 12, 2025 08:50
@chensuyue chensuyue marked this pull request as ready for review June 12, 2025 09:05
@chensuyue chensuyue marked this pull request as draft June 13, 2025 02:35
@joshuayao joshuayao added this to the v1.4 milestone Jun 13, 2025
@joshuayao joshuayao added this to OPEA Jun 13, 2025
@joshuayao joshuayao moved this to In progress in OPEA Jun 13, 2025
@chensuyue chensuyue marked this pull request as ready for review June 18, 2025 07:40
@chensuyue chensuyue marked this pull request as draft June 19, 2025 08:47
Signed-off-by: chensuyue <[email protected]>
Signed-off-by: chensuyue <[email protected]>
@chensuyue chensuyue marked this pull request as ready for review June 19, 2025 09:16
@chensuyue chensuyue marked this pull request as draft June 20, 2025 05:01
@chensuyue chensuyue marked this pull request as ready for review June 23, 2025 07:51
Copy link
Collaborator

@lkk12014402 lkk12014402 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@chensuyue
Copy link
Collaborator Author

image2image_on_intel_hpu
llms_text-generation_service_vllm_on_intel_hpu
Those 2 tests also failed with main branch, so it's known issue, will file ticket to trace. https://github.com/opea-project/GenAIComps/actions/runs/15813485072/job/44568234934?pr=1809

@chensuyue
Copy link
Collaborator Author

llms_doc-summarization_vllm_on_intel_hpu
llms_faq-generation_vllm
Those 2 tests failed with known issue, test got stuck. #1725

* Fix language_detection test

Signed-off-by: chensuyue <[email protected]>

* Fix lvms_llama_vision test

Signed-off-by: chensuyue <[email protected]>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Signed-off-by: chensuyue <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
@chensuyue
Copy link
Collaborator Author

chensuyue commented Jun 23, 2025

The 6 issues found in the last test round have been addressed, 4 known issues, 2 fixed in #1812, so this PR is ready for merge. https://github.com/opea-project/GenAIComps/actions/runs/15818449179
image

@chensuyue chensuyue merged commit a2669d4 into main Jun 24, 2025
18 of 20 checks passed
@chensuyue chensuyue deleted the suyue/uv branch June 24, 2025 01:41
@github-project-automation github-project-automation bot moved this from In progress to Done in OPEA Jun 24, 2025
Comment on lines +20 to +21
RUN pip install --no-cache-dir --upgrade pip setuptools uv && \
$uvpip -r /home/user/comps/agent/src/requirements.txt
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why this drops pytorch CPU version override, don't indirect deps bring it in any more?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This docker image does not include torch in its dependency, so I remove this --extra-index-url.

WenjiaoYue pushed a commit that referenced this pull request Jul 7, 2025
---------

Signed-off-by: chensuyue <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

6 participants