Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Hardware][TPU] Implement tensor parallelism with Ray #5871

Merged
merged 72 commits into from
Jul 27, 2024
Merged

[Hardware][TPU] Implement tensor parallelism with Ray #5871

merged 72 commits into from
Jul 27, 2024

Commits on Jun 24, 2024

  1. Add & warnings

    @WoosukKwon
    WoosukKwon committed Jun 24, 2024
    Configuration menu
    Copy the full SHA
    76fc072 View commit details
    Browse the repository at this point in the history

  2. Add in dummy_run

    @WoosukKwon
    WoosukKwon committed Jun 24, 2024
    Configuration menu
    Copy the full SHA
    27a5ad8 View commit details
    Browse the repository at this point in the history

  3. Add is_driver_worker

    @WoosukKwon
    WoosukKwon committed Jun 24, 2024
    Configuration menu
    Copy the full SHA
    5ab6f65 View commit details
    Browse the repository at this point in the history

  4. Make TPUExecutor similar to GPUExecutor

    @WoosukKwon
    WoosukKwon committed Jun 24, 2024
    Configuration menu
    Copy the full SHA
    c4e79a0 View commit details
    Browse the repository at this point in the history

  5. Add multiprocessing-based TPU executor

    @WoosukKwon
    WoosukKwon committed Jun 24, 2024
    Configuration menu
    Copy the full SHA
    ff81993 View commit details
    Browse the repository at this point in the history

  6. Use TPU to initialize Ray cluster

    @WoosukKwon
    WoosukKwon committed Jun 24, 2024
    Configuration menu
    Copy the full SHA
    16e80b2 View commit details
    Browse the repository at this point in the history

  7. Add pjrt proc init

    @WoosukKwon
    WoosukKwon committed Jun 24, 2024
    Configuration menu
    Copy the full SHA
    05884ce View commit details
    Browse the repository at this point in the history

  8. Add Ray TPU executor

    @WoosukKwon
    WoosukKwon committed Jun 24, 2024
    Configuration menu
    Copy the full SHA
    20d23eb View commit details
    Browse the repository at this point in the history

  9. Use Ray TPU executor for tp

    @WoosukKwon
    WoosukKwon committed Jun 24, 2024
    Configuration menu
    Copy the full SHA
    5d4df21 View commit details
    Browse the repository at this point in the history

  10. Minor

    @WoosukKwon
    WoosukKwon committed Jun 24, 2024
    Configuration menu
    Copy the full SHA
    6b2c76c View commit details
    Browse the repository at this point in the history

  11. Fix TPUWorker.execute_model

    @WoosukKwon
    WoosukKwon committed Jun 24, 2024
    Configuration menu
    Copy the full SHA
    d91446b View commit details
    Browse the repository at this point in the history

  12. Add is_driver_worker & input broadcast

    @WoosukKwon
    WoosukKwon committed Jun 24, 2024
    Configuration menu
    Copy the full SHA
    ab1595d View commit details
    Browse the repository at this point in the history

  13. Call xm._init_world_size_ordinal

    @WoosukKwon
    WoosukKwon committed Jun 24, 2024
    Configuration menu
    Copy the full SHA
    4b45393 View commit details
    Browse the repository at this point in the history

  14. Bug fix on vocab

    @WoosukKwon
    WoosukKwon committed Jun 24, 2024
    Configuration menu
    Copy the full SHA
    86451a2 View commit details
    Browse the repository at this point in the history

  15. Use all gather for TPU

    @WoosukKwon
    WoosukKwon committed Jun 24, 2024
    Configuration menu
    Copy the full SHA
    0539299 View commit details
    Browse the repository at this point in the history

  16. Support TPU in GroupCoordinator

    @WoosukKwon
    WoosukKwon committed Jun 24, 2024
    Configuration menu
    Copy the full SHA
    b35917c View commit details
    Browse the repository at this point in the history

Commits on Jun 25, 2024

  1. Delete multiproc TPU executor

    @WoosukKwon
    WoosukKwon committed Jun 25, 2024
    Configuration menu
    Copy the full SHA
    b9a84bc View commit details
    Browse the repository at this point in the history

  2. Minor

    @WoosukKwon
    WoosukKwon committed Jun 25, 2024
    Configuration menu
    Copy the full SHA
    c756b76 View commit details
    Browse the repository at this point in the history

Commits on Jun 26, 2024

  1. [Bugfix][TPU] Fix CPU cache allocation & swapping

    @WoosukKwon
    WoosukKwon committed Jun 26, 2024
    Configuration menu
    Copy the full SHA
    16e9934 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'fix-tpu-swpa' into tpu-n

    @WoosukKwon
    WoosukKwon committed Jun 26, 2024
    Configuration menu
    Copy the full SHA
    e25f470 View commit details
    Browse the repository at this point in the history

  3. yapf

    @WoosukKwon
    WoosukKwon committed Jun 26, 2024
    Configuration menu
    Copy the full SHA
    ca6d1d6 View commit details
    Browse the repository at this point in the history

  4. Add Ray to TPU dependency

    @WoosukKwon
    WoosukKwon committed Jun 26, 2024
    Configuration menu
    Copy the full SHA
    cd4f68d View commit details
    Browse the repository at this point in the history

  5. Merge branch 'main' into tpu-n

    @WoosukKwon
    WoosukKwon committed Jun 26, 2024
    Configuration menu
    Copy the full SHA
    5df4164 View commit details
    Browse the repository at this point in the history

  6. Fix

    @WoosukKwon
    WoosukKwon committed Jun 26, 2024
    Configuration menu
    Copy the full SHA
    546987a View commit details
    Browse the repository at this point in the history

  7. Fix

    @WoosukKwon
    WoosukKwon committed Jun 26, 2024
    Configuration menu
    Copy the full SHA
    330be6e View commit details
    Browse the repository at this point in the history

Commits on Jun 29, 2024

  1. Merge branch 'main' into tpu-n

    @WoosukKwon
    WoosukKwon committed Jun 29, 2024
    Configuration menu
    Copy the full SHA
    b45ed24 View commit details
    Browse the repository at this point in the history

  2. Add use_all_gather to LoRA

    @WoosukKwon
    WoosukKwon committed Jun 29, 2024
    Configuration menu
    Copy the full SHA
    8fab9fd View commit details
    Browse the repository at this point in the history

  3. Fix

    @WoosukKwon
    WoosukKwon committed Jun 29, 2024
    Configuration menu
    Copy the full SHA
    c4cbe9f View commit details
    Browse the repository at this point in the history

Commits on Jun 30, 2024

  1. Merge branch 'main' into tpu-n

    @WoosukKwon
    WoosukKwon committed Jun 30, 2024
    Configuration menu
    Copy the full SHA
    2871c7c View commit details
    Browse the repository at this point in the history

  2. Add an assert for dim == -1

    @WoosukKwon
    WoosukKwon committed Jun 30, 2024
    Configuration menu
    Copy the full SHA
    db7adc7 View commit details
    Browse the repository at this point in the history

  3. is_tpu -> use_xla

    @WoosukKwon
    WoosukKwon committed Jun 30, 2024
    Configuration menu
    Copy the full SHA
    696790d View commit details
    Browse the repository at this point in the history

  4. Merge branch 'main' into tpu-n

    @WoosukKwon
    WoosukKwon committed Jun 30, 2024
    Configuration menu
    Copy the full SHA
    8a08896 View commit details
    Browse the repository at this point in the history

Commits on Jul 1, 2024

  1. Merge branch 'main' into tpu-n

    @WoosukKwon
    WoosukKwon committed Jul 1, 2024
    Configuration menu
    Copy the full SHA
    36f9070 View commit details
    Browse the repository at this point in the history

Commits on Jul 2, 2024

  1. yapf

    @WoosukKwon
    WoosukKwon committed Jul 2, 2024
    Configuration menu
    Copy the full SHA
    28afe56 View commit details
    Browse the repository at this point in the history

  2. Add hack in vocab

    @WoosukKwon
    WoosukKwon committed Jul 2, 2024
    Configuration menu
    Copy the full SHA
    60bf64d View commit details
    Browse the repository at this point in the history

Commits on Jul 7, 2024

  1. Merge branch 'main' into tpu-n

    @WoosukKwon
    WoosukKwon committed Jul 7, 2024
    Configuration menu
    Copy the full SHA
    0fbb050 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'main' into tpu-n

    @WoosukKwon
    WoosukKwon committed Jul 7, 2024
    Configuration menu
    Copy the full SHA
    ddf4cbe View commit details
    Browse the repository at this point in the history

Commits on Jul 9, 2024

  1. Fix multi-modal support

    @WoosukKwon
    WoosukKwon committed Jul 9, 2024
    Configuration menu
    Copy the full SHA
    cd4842d View commit details
    Browse the repository at this point in the history

  2. Merge branch 'main' into tpu-n

    @WoosukKwon
    WoosukKwon committed Jul 9, 2024
    Configuration menu
    Copy the full SHA
    54e637b View commit details
    Browse the repository at this point in the history

Commits on Jul 10, 2024

  1. Merge branch 'main' into tpu-n

    @WoosukKwon
    WoosukKwon committed Jul 10, 2024
    Configuration menu
    Copy the full SHA
    73ed611 View commit details
    Browse the repository at this point in the history

Commits on Jul 15, 2024

  1. Merge branch 'main' into tpu-n

    @WoosukKwon
    WoosukKwon committed Jul 15, 2024
    Configuration menu
    Copy the full SHA
    717b3fa View commit details
    Browse the repository at this point in the history

Commits on Jul 17, 2024

  1. Merge branch 'main' into tpu-n

    @WoosukKwon
    WoosukKwon committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    6b0c35d View commit details
    Browse the repository at this point in the history

Commits on Jul 18, 2024

  1. Merge branch 'main' into tpu-n

    @WoosukKwon
    WoosukKwon committed Jul 18, 2024
    Configuration menu
    Copy the full SHA
    7f583ba View commit details
    Browse the repository at this point in the history

  2. Remove unused

    @WoosukKwon
    WoosukKwon committed Jul 18, 2024
    Configuration menu
    Copy the full SHA
    106864d View commit details
    Browse the repository at this point in the history

  3. Minor

    @WoosukKwon
    WoosukKwon committed Jul 18, 2024
    Configuration menu
    Copy the full SHA
    223661f View commit details
    Browse the repository at this point in the history

Commits on Jul 21, 2024

  1. Merge branch 'main' into tpu-n

    @WoosukKwon
    WoosukKwon committed Jul 21, 2024
    Configuration menu
    Copy the full SHA
    5bd67bc View commit details
    Browse the repository at this point in the history

  2. Fix comm error

    @WoosukKwon
    WoosukKwon committed Jul 21, 2024
    Configuration menu
    Copy the full SHA
    ab7cccf View commit details
    Browse the repository at this point in the history

  3. Use custom inference_mode

    @WoosukKwon
    WoosukKwon committed Jul 21, 2024
    Configuration menu
    Copy the full SHA
    4e0c90a View commit details
    Browse the repository at this point in the history

  4. Remove hack in vocab embedding

    @WoosukKwon
    WoosukKwon committed Jul 21, 2024
    Configuration menu
    Copy the full SHA
    a2358ed View commit details
    Browse the repository at this point in the history

  5. Use patch

    @WoosukKwon
    WoosukKwon committed Jul 21, 2024
    Configuration menu
    Copy the full SHA
    ac21351 View commit details
    Browse the repository at this point in the history

  6. Update inference_mode

    @WoosukKwon
    WoosukKwon committed Jul 21, 2024
    Configuration menu
    Copy the full SHA
    ba76d9e View commit details
    Browse the repository at this point in the history

  7. use_all_gather -> use_gather

    @WoosukKwon
    WoosukKwon committed Jul 21, 2024
    Configuration menu
    Copy the full SHA
    452c321 View commit details
    Browse the repository at this point in the history

  8. Fix patch

    @WoosukKwon
    WoosukKwon committed Jul 21, 2024
    Configuration menu
    Copy the full SHA
    dcb63b7 View commit details
    Browse the repository at this point in the history

  9. Fix typo

    @WoosukKwon
    WoosukKwon committed Jul 21, 2024
    Configuration menu
    Copy the full SHA
    825cc44 View commit details
    Browse the repository at this point in the history

Commits on Jul 22, 2024

  1. Merge branch 'main' into tpu-n

    @WoosukKwon
    WoosukKwon committed Jul 22, 2024
    Configuration menu
    Copy the full SHA
    f27ef99 View commit details
    Browse the repository at this point in the history

  2. Remove inference_mode

    @WoosukKwon
    WoosukKwon committed Jul 22, 2024
    Configuration menu
    Copy the full SHA
    9730288 View commit details
    Browse the repository at this point in the history

Commits on Jul 23, 2024

  1. Add no_grad

    @WoosukKwon
    WoosukKwon committed Jul 23, 2024
    Configuration menu
    Copy the full SHA
    631b08b View commit details
    Browse the repository at this point in the history

  2. Merge branch 'main' into tpu-n

    @WoosukKwon
    WoosukKwon committed Jul 23, 2024
    Configuration menu
    Copy the full SHA
    d65a7d0 View commit details
    Browse the repository at this point in the history

Commits on Jul 24, 2024

  1. Merge branch 'main' into tpu-n

    @WoosukKwon
    WoosukKwon committed Jul 24, 2024
    Configuration menu
    Copy the full SHA
    755fe0b View commit details
    Browse the repository at this point in the history

Commits on Jul 26, 2024

  1. Merge branch 'main' into tpu-n

    @WoosukKwon
    WoosukKwon committed Jul 26, 2024
    Configuration menu
    Copy the full SHA
    d5fadfd View commit details
    Browse the repository at this point in the history

  2. [TPU] Support collective communications in XLA devices

    @WoosukKwon
    WoosukKwon committed Jul 26, 2024
    Configuration menu
    Copy the full SHA
    af3a259 View commit details
    Browse the repository at this point in the history

  3. Use current_platform

    @WoosukKwon
    WoosukKwon committed Jul 26, 2024
    Configuration menu
    Copy the full SHA
    0f2abea View commit details
    Browse the repository at this point in the history

  4. is_xla -> is_tpu

    @WoosukKwon
    WoosukKwon committed Jul 26, 2024
    Configuration menu
    Copy the full SHA
    8ebea7e View commit details
    Browse the repository at this point in the history

  5. Define TPU communicator

    @WoosukKwon
    WoosukKwon committed Jul 26, 2024
    Configuration menu
    Copy the full SHA
    782b182 View commit details
    Browse the repository at this point in the history

  6. Merge branch 'main' into tpu-n

    @WoosukKwon
    WoosukKwon committed Jul 26, 2024
    Configuration menu
    Copy the full SHA
    76fd300 View commit details
    Browse the repository at this point in the history

  7. Merge branch 'add-xla-comm' into tpu-n

    @WoosukKwon
    WoosukKwon committed Jul 26, 2024
    Configuration menu
    Copy the full SHA
    75f842b View commit details
    Browse the repository at this point in the history

  8. Fix

    @WoosukKwon
    WoosukKwon committed Jul 26, 2024
    Configuration menu
    Copy the full SHA
    8087227 View commit details
    Browse the repository at this point in the history

  9. Address comments

    @WoosukKwon
    WoosukKwon committed Jul 26, 2024
    Configuration menu
    Copy the full SHA
    f04e179 View commit details
    Browse the repository at this point in the history

  10. Device init

    @WoosukKwon
    WoosukKwon committed Jul 26, 2024
    Configuration menu
    Copy the full SHA
    f493c89 View commit details
    Browse the repository at this point in the history

  11. Fix patch

    @WoosukKwon
    WoosukKwon committed Jul 26, 2024
    Configuration menu
    Copy the full SHA
    f14b085 View commit details
    Browse the repository at this point in the history

  12. Merge branch 'add-xla-comm' into tpu-n

    @WoosukKwon
    WoosukKwon committed Jul 26, 2024
    Configuration menu
    Copy the full SHA
    1668582 View commit details
    Browse the repository at this point in the history

Commits on Jul 27, 2024

  1. Merge branch 'main' into tpu-n

    @WoosukKwon
    WoosukKwon committed Jul 27, 2024
    Configuration menu
    Copy the full SHA
    a05cf0f View commit details
    Browse the repository at this point in the history