Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Decouple TritonInferenceStage from pipeline mode #1402

Merged
merged 58 commits into from
Jan 8, 2024
Merged

Decouple TritonInferenceStage from pipeline mode #1402

merged 58 commits into from
Jan 8, 2024

Commits on Dec 1, 2023

  1. First pass at making needs_logits a constructor argument [no ci]

    @dagardner-nv
    dagardner-nv committed Dec 1, 2023
    Configuration menu
    Copy the full SHA
    f3265cb View commit details
    Browse the repository at this point in the history

  2. Add missing doc strings [no ci]

    @dagardner-nv
    dagardner-nv committed Dec 1, 2023
    Configuration menu
    Copy the full SHA
    0c0dd77 View commit details
    Browse the repository at this point in the history

  3. Update log parsing inference class to make needs_logits an argument [… 

    …no ci]
    @dagardner-nv
    dagardner-nv committed Dec 1, 2023
    Configuration menu
    Copy the full SHA
    ed4788c View commit details
    Browse the repository at this point in the history

  4. Update log parsing tests [no ci]

    @dagardner-nv
    dagardner-nv committed Dec 1, 2023
    Configuration menu
    Copy the full SHA
    ea0cabc View commit details
    Browse the repository at this point in the history

  5. Add more tests [no ci]

    @dagardner-nv
    dagardner-nv committed Dec 1, 2023
    Configuration menu
    Copy the full SHA
    7804987 View commit details
    Browse the repository at this point in the history

  6. Lint fixes

    @dagardner-nv
    dagardner-nv committed Dec 1, 2023
    Configuration menu
    Copy the full SHA
    474bc87 View commit details
    Browse the repository at this point in the history

Commits on Dec 11, 2023

  1. Allow for a worker-class argument

    @dagardner-nv
    dagardner-nv committed Dec 11, 2023
    Configuration menu
    Copy the full SHA
    8b0e1ce View commit details
    Browse the repository at this point in the history

  2. Use the public InferenceWorker as the type hint

    @dagardner-nv
    dagardner-nv committed Dec 11, 2023
    Configuration menu
    Copy the full SHA
    77e2a67 View commit details
    Browse the repository at this point in the history

  3. Specify worker class, since we now validate the worker class in the c… 

    …onstructor we need to specify one anytime we are using the other pipeline mode
    @dagardner-nv
    dagardner-nv committed Dec 11, 2023
    Configuration menu
    Copy the full SHA
    8d51010 View commit details
    Browse the repository at this point in the history

  4. Fix & update tests

    @dagardner-nv
    dagardner-nv committed Dec 11, 2023
    Configuration menu
    Copy the full SHA
    db5cc53 View commit details
    Browse the repository at this point in the history

  5. Lint fixes

    @dagardner-nv
    dagardner-nv committed Dec 11, 2023
    Configuration menu
    Copy the full SHA
    52a6778 View commit details
    Browse the repository at this point in the history

Commits on Dec 13, 2023

  1. wip

    @dagardner-nv
    dagardner-nv committed Dec 13, 2023
    Configuration menu
    Copy the full SHA
    de00f7a View commit details
    Browse the repository at this point in the history

  2. Merge branch 'branch-24.03' of github.com:nv-morpheus/Morpheus into t… 

    …riton-logits-1378 [no ci]
    @dagardner-nv
    dagardner-nv committed Dec 13, 2023
    Configuration menu
    Copy the full SHA
    ee18ee4 View commit details
    Browse the repository at this point in the history

Commits on Dec 14, 2023

  1. Remove the AE Triton worker which isn't supported, and remove the sta… 

    …ge from the AE pipeline [no ci]
    @dagardner-nv
    dagardner-nv committed Dec 14, 2023
    Configuration menu
    Copy the full SHA
    d852879 View commit details
    Browse the repository at this point in the history

  2. Rename _TritonInferenceWorker and make it public [no ci]

    @dagardner-nv
    dagardner-nv committed Dec 14, 2023
    Configuration menu
    Copy the full SHA
    de406e6 View commit details
    Browse the repository at this point in the history

  3. wip [no ci]

    @dagardner-nv
    dagardner-nv committed Dec 14, 2023
    Configuration menu
    Copy the full SHA
    fbbda24 View commit details
    Browse the repository at this point in the history

  4. Add expand_dims as constructor argument, and implement the _build_res… 

    …ponse method [no ci]
    @dagardner-nv
    dagardner-nv committed Dec 14, 2023
    Configuration menu
    Copy the full SHA
    aba1f96 View commit details
    Browse the repository at this point in the history

  5. update tests [no ci]

    @dagardner-nv
    dagardner-nv committed Dec 14, 2023
    Configuration menu
    Copy the full SHA
    10ff5cd View commit details
    Browse the repository at this point in the history

  6. Merge branch 'branch-24.03' of github.com:nv-morpheus/Morpheus into t… 

    …riton-logits-1378 [no ci]
    @dagardner-nv
    dagardner-nv committed Dec 14, 2023
    Configuration menu
    Copy the full SHA
    1171917 View commit details
    Browse the repository at this point in the history

  7. Fix inout_mapping [no ci]

    @dagardner-nv
    dagardner-nv committed Dec 14, 2023
    Configuration menu
    Copy the full SHA
    fd726e1 View commit details
    Browse the repository at this point in the history

  8. Fix CLI tests [no ci]

    @dagardner-nv
    dagardner-nv committed Dec 14, 2023
    Configuration menu
    Copy the full SHA
    2d3c213 View commit details
    Browse the repository at this point in the history

  9. Remove expand_dims constructor argument, just always perform it [no ci]

    @dagardner-nv
    dagardner-nv committed Dec 14, 2023
    Configuration menu
    Copy the full SHA
    e847536 View commit details
    Browse the repository at this point in the history

  10. Remove unused import [no ci]

    @dagardner-nv
    dagardner-nv committed Dec 14, 2023
    Configuration menu
    Copy the full SHA
    e6770df View commit details
    Browse the repository at this point in the history

  11. Remove redundant code by mazking LogParsingInferenceStage a subclass … 

    …of TritonInferenceStage [no ci]
    @dagardner-nv
    dagardner-nv committed Dec 14, 2023
    Configuration menu
    Copy the full SHA
    b35a21a View commit details
    Browse the repository at this point in the history

  12. wip [no ci]

    @dagardner-nv
    dagardner-nv committed Dec 14, 2023
    Configuration menu
    Copy the full SHA
    1eb6241 View commit details
    Browse the repository at this point in the history

  13. Allow subclassing a stage already registered with the CLI [no ci]

    @dagardner-nv
    dagardner-nv committed Dec 14, 2023
    Configuration menu
    Copy the full SHA
    d83a7c5 View commit details
    Browse the repository at this point in the history

  14. Remove unused imports [no ci]

    @dagardner-nv
    dagardner-nv committed Dec 14, 2023
    Configuration menu
    Copy the full SHA
    955e94f View commit details
    Browse the repository at this point in the history

  15. Add type hints [no ci]

    @dagardner-nv
    dagardner-nv committed Dec 14, 2023
    Configuration menu
    Copy the full SHA
    f785c90 View commit details
    Browse the repository at this point in the history

  16. cleanups [no ci]

    @dagardner-nv
    dagardner-nv committed Dec 14, 2023
    Configuration menu
    Copy the full SHA
    cd472c0 View commit details
    Browse the repository at this point in the history

  17. Add support for dictionary types by configuring the click type as a t… 

    …uple with multiple=True
    @dagardner-nv
    dagardner-nv committed Dec 14, 2023
    Configuration menu
    Copy the full SHA
    746ad6c View commit details
    Browse the repository at this point in the history

  18. Remove temportary print statement

    @dagardner-nv
    dagardner-nv committed Dec 14, 2023
    Configuration menu
    Copy the full SHA
    54c09e9 View commit details
    Browse the repository at this point in the history

  19. Document --inout_mapping flag

    @dagardner-nv
    dagardner-nv committed Dec 14, 2023
    Configuration menu
    Copy the full SHA
    4868d09 View commit details
    Browse the repository at this point in the history

  20. Update tests

    @dagardner-nv
    dagardner-nv committed Dec 14, 2023
    Configuration menu
    Copy the full SHA
    bf2508b View commit details
    Browse the repository at this point in the history

Commits on Dec 20, 2023

  1. Make INFERENCE_WORKER_DEFAULT_INOUT_MAPPING a class property

    @dagardner-nv
    dagardner-nv committed Dec 20, 2023
    Configuration menu
    Copy the full SHA
    4dd39f3 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'branch-24.03' of github.com:nv-morpheus/Morpheus into t… 

    …riton-logits-1378 [no ci]
    @dagardner-nv
    dagardner-nv committed Dec 20, 2023
    Configuration menu
    Copy the full SHA
    e9634a3 View commit details
    Browse the repository at this point in the history

  3. Replace ResponseMemoryLogParsing with ResponseMemory

    @dagardner-nv
    dagardner-nv committed Dec 20, 2023
    Configuration menu
    Copy the full SHA
    fd092ba View commit details
    Browse the repository at this point in the history

  4. Replace PostprocMemoryLogParsing with InferenceMemory [no ci]

    @dagardner-nv
    dagardner-nv committed Dec 20, 2023
    Configuration menu
    Copy the full SHA
    bfaef1c View commit details
    Browse the repository at this point in the history

  5. Remove unused MultiResponseLogParsingMessage [no ci]

    @dagardner-nv
    dagardner-nv committed Dec 20, 2023
    Configuration menu
    Copy the full SHA
    c40b0ff View commit details
    Browse the repository at this point in the history

  6. Remove un-needed property methods

    @dagardner-nv
    dagardner-nv committed Dec 20, 2023
    Configuration menu
    Copy the full SHA
    5cb45b8 View commit details
    Browse the repository at this point in the history

  7. Give a more descriptive message to the Attribute error

    @dagardner-nv
    dagardner-nv committed Dec 20, 2023
    Configuration menu
    Copy the full SHA
    eff5960 View commit details
    Browse the repository at this point in the history

  8. WIP: the dataclass decorator creates a constructor that doesn't call … 

    …the parent constructor, bypassing the checks in the MultiTensorMessage class. Manually construct a valid seq_ids array still need to detmine why the inference stage is emitting an inference message and not a response message [no ci]
    @dagardner-nv
    dagardner-nv committed Dec 20, 2023
    Configuration menu
    Copy the full SHA
    9a6f8bf View commit details
    Browse the repository at this point in the history

  9. Remove unused message class [no ci]

    @dagardner-nv
    dagardner-nv committed Dec 20, 2023
    Configuration menu
    Copy the full SHA
    a52e534 View commit details
    Browse the repository at this point in the history

  10. Update inference stage to emit the propper output messages [no ci]

    @dagardner-nv
    dagardner-nv committed Dec 20, 2023
    Configuration menu
    Copy the full SHA
    b85776c View commit details
    Browse the repository at this point in the history

  11. Update post-processing tests

    @dagardner-nv
    dagardner-nv committed Dec 20, 2023
    Configuration menu
    Copy the full SHA
    3821243 View commit details
    Browse the repository at this point in the history

  12. Replace usage of deprecated ResponseMemory class

    @dagardner-nv
    dagardner-nv committed Dec 20, 2023
    Configuration menu
    Copy the full SHA
    6ea799b View commit details
    Browse the repository at this point in the history

  13. Fix type-hint

    @dagardner-nv
    dagardner-nv committed Dec 20, 2023
    Configuration menu
    Copy the full SHA
    bee89d3 View commit details
    Browse the repository at this point in the history

  14. Update test

    @dagardner-nv
    dagardner-nv committed Dec 20, 2023
    Configuration menu
    Copy the full SHA
    ddd6572 View commit details
    Browse the repository at this point in the history

  15. Remove _get_worker_class method

    @dagardner-nv
    dagardner-nv committed Dec 20, 2023
    Configuration menu
    Copy the full SHA
    f100479 View commit details
    Browse the repository at this point in the history

  16. Remove _get_worker_class method

    @dagardner-nv
    dagardner-nv committed Dec 20, 2023
    Configuration menu
    Copy the full SHA
    342a004 View commit details
    Browse the repository at this point in the history

Commits on Dec 21, 2023

  1. Set include_nulls=True and na_rep=null to match Python json serializer

    @dagardner-nv
    dagardner-nv committed Dec 21, 2023
    Configuration menu
    Copy the full SHA
    6916db6 View commit details
    Browse the repository at this point in the history

  2. Support running the pipeline in C++ mode

    @dagardner-nv
    dagardner-nv committed Dec 21, 2023
    Configuration menu
    Copy the full SHA
    d6e6341 View commit details
    Browse the repository at this point in the history

  3. Fix test to enable c++ mode

    @dagardner-nv
    dagardner-nv committed Dec 21, 2023
    Configuration menu
    Copy the full SHA
    64b77c3 View commit details
    Browse the repository at this point in the history

  4. WIP: Need to fix tests to work in C++ mode

    @dagardner-nv
    dagardner-nv committed Dec 21, 2023
    Configuration menu
    Copy the full SHA
    1f5ae81 View commit details
    Browse the repository at this point in the history

Commits on Dec 26, 2023

  1. Merge branch 'branch-24.03' of github.com:nv-morpheus/Morpheus into t… 

    …riton-logits-1378
    @dagardner-nv
    dagardner-nv committed Dec 26, 2023
    Configuration menu
    Copy the full SHA
    923f4b5 View commit details
    Browse the repository at this point in the history

  2. Avoid redundant calls to get_tensor

    @dagardner-nv
    dagardner-nv committed Dec 26, 2023
    Configuration menu
    Copy the full SHA
    326fdb8 View commit details
    Browse the repository at this point in the history

  3. Fix setting the range of the output tensor

    @dagardner-nv
    dagardner-nv committed Dec 26, 2023
    Configuration menu
    Copy the full SHA
    bbc0ee3 View commit details
    Browse the repository at this point in the history

  4. Fix tests to opperate in C++ mode

    @dagardner-nv
    dagardner-nv committed Dec 26, 2023
    Configuration menu
    Copy the full SHA
    01f0fe8 View commit details
    Browse the repository at this point in the history

  5. Lint fixes

    @dagardner-nv
    dagardner-nv committed Dec 26, 2023
    Configuration menu
    Copy the full SHA
    2ad239f View commit details
    Browse the repository at this point in the history