Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add benchmark for completion pipeline #1414

Merged
merged 17 commits into from
Dec 11, 2023
Merged

Add benchmark for completion pipeline #1414

merged 17 commits into from
Dec 11, 2023

Conversation

dagardner-nv
Copy link
Contributor

@dagardner-nv dagardner-nv commented Dec 7, 2023 •
edited
Loading

Description

  • Executes the completion pipeline against mocked instances of nemollm and openai.
  • Mocked calls sleep for a set period of time, based upon observed times from my own machine.
  • Update completion pipeline example to optionally read country names from a file and optionally randomly sorting the order (These are changes I made to help with getting the timings, and don't really need to be merged).
  • Move the "inference" monitor to appear immediately after the inference stage.
  • Unrelated fix testing the results of tests/llm/test_agents_simple_pipe.py::test_agents_simple_pipe_integration_openai
  • Add CSV file containing the names of UN member nations

By Submitting this PR I confirm:

  • I am familiar with the Contributing Guidelines.
  • When the PR is ready for review, new or existing tests cover these changes.
  • When the PR is ready for review, the documentation is up to date with these changes.

@dagardner-nv
List of UN member nations based upon https://www.un.org/dgacm/sites/w…
4eaac3f 
…ww.un.org.dgacm/files/Documents_Protocol/officialnamesofcountries.pdf
@dagardner-nv
Add heading row
0c43db8
@dagardner-nv
fix heading row case
357e657
@dagardner-nv
Add flags to read in the input from a file, and optionally shuffle th…
b5f4b4a 
…e dataframe
@dagardner-nv
Move nemo and openai fixtures up a level to be shared with benchmarks
9e8d4ca
@dagardner-nv
Fix integration test, responses are prefixed with text
575c805
@dagardner-nv
Extend existing mock_chat_completion and mock_nemollm fixtures to ran…
a8876db 
…domly sleep
@dagardner-nv
Call asyncio.get_event_loop() rather than using the event_loop fixtur…
8e0f99c 
…e, avoids issue where the futures were created on a different loop than the llm engine
@dagardner-nv
Add missing --run_benchmark flag
e58aeb5
@dagardner-nv
Add benchmark for the completion pipeline
2918bd6
@dagardner-nv dagardner-nv requested a review from a team as a code owner December 7, 2023 00:28
@dagardner-nv dagardner-nv self-assigned this Dec 7, 2023
@dagardner-nv dagardner-nv added non-breaking Non-breaking change improvement Improvement to existing functionality labels Dec 7, 2023
mdemoret-nv
mdemoret-nv approved these changes Dec 11, 2023
View reviewed changes
tests/benchmarks/conftest.py Outdated Show resolved Hide resolved
@dagardner-nv
Copy link
Contributor Author

/merge

@rapids-bot rapids-bot bot merged commit 5d19b60 into nv-morpheus:branch-24.03 Dec 11, 2023
6 checks passed
@dagardner-nv dagardner-nv deleted the david-24.03-llm-bench branch December 11, 2023 19:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@mdemoret-nv mdemoret-nv mdemoret-nv approved these changes

Assignees

@dagardner-nv dagardner-nv

Labels
improvement Improvement to existing functionality non-breaking Non-breaking change
Projects
Morpheus Boards
Archived in project
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@dagardner-nv @mdemoret-nv