Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Core] Refactor Worker and ModelRunner to consolidate control plane communication #5408

Merged
merged 64 commits into from
Jun 26, 2024
Merged

[Core] Refactor Worker and ModelRunner to consolidate control plane communication #5408

merged 64 commits into from
Jun 26, 2024

Commits on Jun 11, 2024

  1. tmp 

    Signed-off-by: Stephanie Wang <[email protected]>
    @stephanie-wang
    stephanie-wang committed Jun 11, 2024
    Configuration menu
    Copy the full SHA
    725b0b2 View commit details
    Browse the repository at this point in the history

  2. fix 

    Signed-off-by: Stephanie Wang <[email protected]>
    @stephanie-wang
    stephanie-wang committed Jun 11, 2024
    Configuration menu
    Copy the full SHA
    b74eb10 View commit details
    Browse the repository at this point in the history

  3. ray and mp backends work 

    Signed-off-by: Stephanie Wang <[email protected]>
    @stephanie-wang
    stephanie-wang committed Jun 11, 2024
    Configuration menu
    Copy the full SHA
    38b0ddf View commit details
    Browse the repository at this point in the history

  4. embedding model runner works 

    Signed-off-by: Stephanie Wang <[email protected]>
    @stephanie-wang
    stephanie-wang committed Jun 11, 2024
    Configuration menu
    Copy the full SHA
    0d11e92 View commit details
    Browse the repository at this point in the history

  5. GPU executor works 

    Signed-off-by: Stephanie Wang <[email protected]>
    @stephanie-wang
    stephanie-wang committed Jun 11, 2024
    Configuration menu
    Copy the full SHA
    2cdc218 View commit details
    Browse the repository at this point in the history

  6. remove comment 

    Signed-off-by: Stephanie Wang <[email protected]>
    @stephanie-wang
    stephanie-wang committed Jun 11, 2024
    Configuration menu
    Copy the full SHA
    c728512 View commit details
    Browse the repository at this point in the history

  7. use the right ModelInput class 

    Signed-off-by: Stephanie Wang <[email protected]>
    @stephanie-wang
    stephanie-wang committed Jun 11, 2024
    Configuration menu
    Copy the full SHA
    2bf752b View commit details
    Browse the repository at this point in the history

  8. CPU worker 

    Signed-off-by: Stephanie Wang <[email protected]>
    @stephanie-wang
    stephanie-wang committed Jun 11, 2024
    Configuration menu
    Copy the full SHA
    f35a23f View commit details
    Browse the repository at this point in the history

  9. remove commented 

    Signed-off-by: Stephanie Wang <[email protected]>
    @stephanie-wang
    stephanie-wang committed Jun 11, 2024
    Configuration menu
    Copy the full SHA
    11133fe View commit details
    Browse the repository at this point in the history

  10. lint 

    Signed-off-by: Stephanie Wang <[email protected]>
    @stephanie-wang
    stephanie-wang committed Jun 11, 2024
    Configuration menu
    Copy the full SHA
    174bdb1 View commit details
    Browse the repository at this point in the history

  11. Worker.execute_model vs execute_model_local 

    Signed-off-by: Stephanie Wang <[email protected]>
    @stephanie-wang
    stephanie-wang committed Jun 11, 2024
    Configuration menu
    Copy the full SHA
    c0e98ca View commit details
    Browse the repository at this point in the history

  12. lint 

    Signed-off-by: Stephanie Wang <[email protected]>
    @stephanie-wang
    stephanie-wang committed Jun 11, 2024
    Configuration menu
    Copy the full SHA
    dccec95 View commit details
    Browse the repository at this point in the history

  13. neuron model runner 

    Signed-off-by: Stephanie Wang <[email protected]>
    @stephanie-wang
    stephanie-wang committed Jun 11, 2024
    Configuration menu
    Copy the full SHA
    dad94ba View commit details
    Browse the repository at this point in the history

  14. disallow distributed comms 

    Signed-off-by: Stephanie Wang <[email protected]>
    @stephanie-wang
    stephanie-wang committed Jun 11, 2024
    Configuration menu
    Copy the full SHA
    fca606e View commit details
    Browse the repository at this point in the history

Commits on Jun 12, 2024

  1. disable communication 

    Signed-off-by: Stephanie Wang <[email protected]>
    @stephanie-wang
    stephanie-wang committed Jun 12, 2024
    Configuration menu
    Copy the full SHA
    6ed3c2a View commit details
    Browse the repository at this point in the history

  2. Update worker.py

    @stephanie-wang
    stephanie-wang committed Jun 12, 2024
    Configuration menu
    Copy the full SHA
    1803e33 View commit details
    Browse the repository at this point in the history

  3. fix tests 

    Signed-off-by: Stephanie Wang <[email protected]>
    @stephanie-wang
    stephanie-wang committed Jun 12, 2024
    Configuration menu
    Copy the full SHA
    dde799e View commit details
    Browse the repository at this point in the history

  4. update 

    Signed-off-by: Stephanie Wang <[email protected]>
    @stephanie-wang
    stephanie-wang committed Jun 12, 2024
    Configuration menu
    Copy the full SHA
    0398631 View commit details
    Browse the repository at this point in the history

  5. Merge branch 'control-refactor-2' of github.com:stephanie-wang/vllm i… 

    …nto control-refactor-2
    @stephanie-wang
    stephanie-wang committed Jun 12, 2024
    Configuration menu
    Copy the full SHA
    5c41cc6 View commit details
    Browse the repository at this point in the history

  6. Merge remote-tracking branch 'upstream/main' into control-refactor-2

    @stephanie-wang
    stephanie-wang committed Jun 12, 2024
    Configuration menu
    Copy the full SHA
    72f0383 View commit details
    Browse the repository at this point in the history

  7. merge 

    Signed-off-by: Stephanie Wang <[email protected]>
    @stephanie-wang
    stephanie-wang committed Jun 12, 2024
    Configuration menu
    Copy the full SHA
    eef6623 View commit details
    Browse the repository at this point in the history

Commits on Jun 13, 2024

  1. update 

    Signed-off-by: Stephanie <[email protected]>
    Stephanie committed Jun 13, 2024
    Configuration menu
    Copy the full SHA
    3004ceb View commit details
    Browse the repository at this point in the history

  2. Merge remote-tracking branch 'upstream/main' into control-refactor-2

    @stephanie-wang
    stephanie-wang committed Jun 13, 2024
    Configuration menu
    Copy the full SHA
    8d852e9 View commit details
    Browse the repository at this point in the history

  3. fix 

    Signed-off-by: Stephanie Wang <[email protected]>
    @stephanie-wang
    stephanie-wang committed Jun 13, 2024
    Configuration menu
    Copy the full SHA
    9380ed8 View commit details
    Browse the repository at this point in the history

  4. fix 

    Signed-off-by: Stephanie Wang <[email protected]>
    @stephanie-wang
    stephanie-wang committed Jun 13, 2024
    Configuration menu
    Copy the full SHA
    3c4de6d View commit details
    Browse the repository at this point in the history

  5. fix 

    Signed-off-by: Stephanie Wang <[email protected]>
    @stephanie-wang
    stephanie-wang committed Jun 13, 2024
    Configuration menu
    Copy the full SHA
    5053f30 View commit details
    Browse the repository at this point in the history

Commits on Jun 14, 2024

  1. x 

    Signed-off-by: Stephanie Wang <[email protected]>
    @stephanie-wang
    stephanie-wang committed Jun 14, 2024
    Configuration menu
    Copy the full SHA
    db38556 View commit details
    Browse the repository at this point in the history

  2. rm 

    Signed-off-by: Stephanie Wang <[email protected]>
    @stephanie-wang
    stephanie-wang committed Jun 14, 2024
    Configuration menu
    Copy the full SHA
    456185d View commit details
    Browse the repository at this point in the history

  3. lint 

    Signed-off-by: Stephanie Wang <[email protected]>
    @stephanie-wang
    stephanie-wang committed Jun 14, 2024
    Configuration menu
    Copy the full SHA
    e860652 View commit details
    Browse the repository at this point in the history

  4. add missing 

    Signed-off-by: Stephanie Wang <[email protected]>
    @stephanie-wang
    stephanie-wang committed Jun 14, 2024
    Configuration menu
    Copy the full SHA
    3d4f242 View commit details
    Browse the repository at this point in the history

  5. revert 

    Signed-off-by: Stephanie Wang <[email protected]>
    @stephanie-wang
    stephanie-wang committed Jun 14, 2024
    Configuration menu
    Copy the full SHA
    11304cb View commit details
    Browse the repository at this point in the history

Commits on Jun 15, 2024

  1. refactor 

    Signed-off-by: Stephanie Wang <[email protected]>
    @stephanie-wang
    stephanie-wang committed Jun 15, 2024
    Configuration menu
    Copy the full SHA
    99f532e View commit details
    Browse the repository at this point in the history

  2. doc 

    Signed-off-by: Stephanie Wang <[email protected]>
    @stephanie-wang
    stephanie-wang committed Jun 15, 2024
    Configuration menu
    Copy the full SHA
    797a7cf View commit details
    Browse the repository at this point in the history

  3. revert spec decode and doc 

    Signed-off-by: Stephanie Wang <[email protected]>
    @stephanie-wang
    stephanie-wang committed Jun 15, 2024
    Configuration menu
    Copy the full SHA
    6ad2513 View commit details
    Browse the repository at this point in the history

  4. Merge remote-tracking branch 'upstream/main' into control-refactor-2

    @stephanie-wang
    stephanie-wang committed Jun 15, 2024
    Configuration menu
    Copy the full SHA
    97ec303 View commit details
    Browse the repository at this point in the history

  5. typing 

    Signed-off-by: Stephanie Wang <[email protected]>
    @stephanie-wang
    stephanie-wang committed Jun 15, 2024
    Configuration menu
    Copy the full SHA
    e10bace View commit details
    Browse the repository at this point in the history

Commits on Jun 18, 2024

  1. fix 

    Signed-off-by: Stephanie Wang <[email protected]>
    @stephanie-wang
    stephanie-wang committed Jun 18, 2024
    Configuration menu
    Copy the full SHA
    ce087ae View commit details
    Browse the repository at this point in the history

  2. Merge remote-tracking branch 'upstream/main' into control-refactor-2

    @stephanie-wang
    stephanie-wang committed Jun 18, 2024
    Configuration menu
    Copy the full SHA
    f851b00 View commit details
    Browse the repository at this point in the history

  3. XPU worker and rename 

    Signed-off-by: Stephanie Wang <[email protected]>
    @stephanie-wang
    stephanie-wang committed Jun 18, 2024
    Configuration menu
    Copy the full SHA
    0e2acc4 View commit details
    Browse the repository at this point in the history

  4. lint 

    Signed-off-by: Stephanie Wang <[email protected]>
    @stephanie-wang
    stephanie-wang committed Jun 18, 2024
    Configuration menu
    Copy the full SHA
    d318ec8 View commit details
    Browse the repository at this point in the history

  5. lint 

    Signed-off-by: Stephanie Wang <[email protected]>
    @stephanie-wang
    stephanie-wang committed Jun 18, 2024
    Configuration menu
    Copy the full SHA
    b48f783 View commit details
    Browse the repository at this point in the history

  6. Merge remote-tracking branch 'upstream/main' into control-refactor-2

    @stephanie-wang
    stephanie-wang committed Jun 18, 2024
    Configuration menu
    Copy the full SHA
    c93afc1 View commit details
    Browse the repository at this point in the history

  7. fix 

    Signed-off-by: Stephanie Wang <[email protected]>
    @stephanie-wang
    stephanie-wang committed Jun 18, 2024
    Configuration menu
    Copy the full SHA
    30ac400 View commit details
    Browse the repository at this point in the history

  8. x 

    Signed-off-by: Stephanie Wang <[email protected]>
    @stephanie-wang
    stephanie-wang committed Jun 18, 2024
    Configuration menu
    Copy the full SHA
    01688d5 View commit details
    Browse the repository at this point in the history

  9. fix 

    Signed-off-by: Stephanie Wang <[email protected]>
    @stephanie-wang
    stephanie-wang committed Jun 18, 2024
    Configuration menu
    Copy the full SHA
    7dbb646 View commit details
    Browse the repository at this point in the history

Commits on Jun 19, 2024

  1. fix 

    Signed-off-by: Stephanie Wang <[email protected]>
    @stephanie-wang
    stephanie-wang committed Jun 19, 2024
    Configuration menu
    Copy the full SHA
    d2e4c41 View commit details
    Browse the repository at this point in the history

  2. lint 

    Signed-off-by: Stephanie Wang <[email protected]>
    @stephanie-wang
    stephanie-wang committed Jun 19, 2024
    Configuration menu
    Copy the full SHA
    3e46253 View commit details
    Browse the repository at this point in the history

Commits on Jun 21, 2024

  1. Merge remote-tracking branch 'upstream/main' into control-refactor-2

    @stephanie-wang
    stephanie-wang committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    0a2890a View commit details
    Browse the repository at this point in the history

  2. merge 

    Signed-off-by: Stephanie Wang <[email protected]>
    @stephanie-wang
    stephanie-wang committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    36dfce1 View commit details
    Browse the repository at this point in the history

  3. Merge remote-tracking branch 'upstream/main' into control-refactor-2

    @stephanie-wang
    stephanie-wang committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    ea5412e View commit details
    Browse the repository at this point in the history

  4. x 

    Signed-off-by: Stephanie Wang <[email protected]>
    @stephanie-wang
    stephanie-wang committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    dc2f103 View commit details
    Browse the repository at this point in the history

Commits on Jun 22, 2024

  1. x 

    Signed-off-by: Stephanie Wang <[email protected]>
    @stephanie-wang
    stephanie-wang committed Jun 22, 2024
    Configuration menu
    Copy the full SHA
    fbf074d View commit details
    Browse the repository at this point in the history

Commits on Jun 23, 2024

  1. rename ModelInput -> ModelInputBase, override as_broadcastable_tensor… 

    …_dict and new

Signed-off-by: Stephanie Wang <[email protected]>
    @stephanie-wang
    stephanie-wang committed Jun 23, 2024
    Configuration menu
    Copy the full SHA
    660a8d5 View commit details
    Browse the repository at this point in the history

  2. fixes 

    Signed-off-by: Stephanie Wang <[email protected]>
    @stephanie-wang
    stephanie-wang committed Jun 23, 2024
    Configuration menu
    Copy the full SHA
    8cca634 View commit details
    Browse the repository at this point in the history

  3. rename 

    Signed-off-by: Stephanie Wang <[email protected]>
    @stephanie-wang
    stephanie-wang committed Jun 23, 2024
    Configuration menu
    Copy the full SHA
    0a25c19 View commit details
    Browse the repository at this point in the history

Commits on Jun 24, 2024

  1. fix 

    Signed-off-by: Stephanie Wang <[email protected]>
    @stephanie-wang
    stephanie-wang committed Jun 24, 2024
    Configuration menu
    Copy the full SHA
    0b26877 View commit details
    Browse the repository at this point in the history

  2. do not filter Nones 

    Signed-off-by: Stephanie Wang <[email protected]>
    @stephanie-wang
    stephanie-wang committed Jun 24, 2024
    Configuration menu
    Copy the full SHA
    e7052d5 View commit details
    Browse the repository at this point in the history

  3. dupe 

    Signed-off-by: Stephanie Wang <[email protected]>
    @stephanie-wang
    stephanie-wang committed Jun 24, 2024
    Configuration menu
    Copy the full SHA
    df5551f View commit details
    Browse the repository at this point in the history

Commits on Jun 25, 2024

  1. update 

    Signed-off-by: Stephanie Wang <[email protected]>
    @stephanie-wang
    stephanie-wang committed Jun 25, 2024
    Configuration menu
    Copy the full SHA
    6745b3b View commit details
    Browse the repository at this point in the history

  2. lint 

    Signed-off-by: Stephanie Wang <[email protected]>
    @stephanie-wang
    stephanie-wang committed Jun 25, 2024
    Configuration menu
    Copy the full SHA
    ebae970 View commit details
    Browse the repository at this point in the history

  3. revert 

    Signed-off-by: Stephanie Wang <[email protected]>
    @stephanie-wang
    stephanie-wang committed Jun 25, 2024
    Configuration menu
    Copy the full SHA
    5763621 View commit details
    Browse the repository at this point in the history

  4. Merge branch 'main' into control-refactor-2

    @stephanie-wang
    stephanie-wang committed Jun 25, 2024
    Configuration menu
    Copy the full SHA
    46d5b18 View commit details
    Browse the repository at this point in the history

  5. rm 

    Signed-off-by: Stephanie Wang <[email protected]>
    @stephanie-wang
    stephanie-wang committed Jun 25, 2024
    Configuration menu
    Copy the full SHA
    d16d5fe View commit details
    Browse the repository at this point in the history

  6. fix 

    Signed-off-by: Stephanie Wang <[email protected]>
    @stephanie-wang
    stephanie-wang committed Jun 25, 2024
    Configuration menu
    Copy the full SHA
    f6c6234 View commit details
    Browse the repository at this point in the history