Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add benchmarks for stand-alone RAG & vdb upload pipelines #1421

Merged
merged 31 commits into from
Dec 14, 2023
Merged

Add benchmarks for stand-alone RAG & vdb upload pipelines #1421

merged 31 commits into from
Dec 14, 2023

Conversation

dagardner-nv
Copy link
Contributor

@dagardner-nv dagardner-nv commented Dec 11, 2023 •
edited
Loading

Description

  • Adds a benchmark for the vdb_upload & RAG stand-alone pipelines.
  • The vdb_upload benchmark mocks the HTTP requests performed by the RSS source and webscraper stages.
  • Update benchmark instructions in tests/benchmarks/README.md

By Submitting this PR I confirm:

  • I am familiar with the Contributing Guidelines.
  • When the PR is ready for review, new or existing tests cover these changes.
  • When the PR is ready for review, the documentation is up to date with these changes.

@dagardner-nv
List of UN member nations based upon https://www.un.org/dgacm/sites/w…
4eaac3f 
…ww.un.org.dgacm/files/Documents_Protocol/officialnamesofcountries.pdf
@dagardner-nv
Add heading row
0c43db8
@dagardner-nv
fix heading row case
357e657
@dagardner-nv
Add flags to read in the input from a file, and optionally shuffle th…
b5f4b4a 
…e dataframe
@dagardner-nv
Move nemo and openai fixtures up a level to be shared with benchmarks
9e8d4ca
@dagardner-nv
Fix integration test, responses are prefixed with text
575c805
@dagardner-nv
Extend existing mock_chat_completion and mock_nemollm fixtures to ran…
a8876db 
…domly sleep
@dagardner-nv
Call asyncio.get_event_loop() rather than using the event_loop fixtur…
8e0f99c 
…e, avoids issue where the futures were created on a different loop than the llm engine
@dagardner-nv
Add missing --run_benchmark flag
e58aeb5
@dagardner-nv
Add benchmark for the completion pipeline
2918bd6
@dagardner-nv
Rename benchmark to avoid name clash
477ca6d
@dagardner-nv
Remove unused importd
a03e659
@dagardner-nv
Specify string as a regex string
fe12b9c
@dagardner-nv
Rename benchmark to match existing names
e58cd92
@dagardner-nv
Rename test
ed212ae
@dagardner-nv
WIP: update docker run
9a60b14
@dagardner-nv
WIP: benchmark for vdb upload pipeline
4a0ec7e
@dagardner-nv
WIP
f97432d
@dagardner-nv
WIP
6e01776
@dagardner-nv
Cleanups
5771492
@dagardner-nv
Update docs
2325315
@dagardner-nv
Replace usage of random numbers with a fixed sleep based on average o…
4ee5e47 
…bserved sleep times
@dagardner-nv
Remove unused import
5acbb2e
@dagardner-nv
Merge branch 'david-24.03-llm-bench' into david-24.03-llm-bench-vdb_u…
bc675da 
…pload
@dagardner-nv dagardner-nv self-assigned this Dec 11, 2023
@dagardner-nv dagardner-nv requested a review from a team as a code owner December 11, 2023 18:17
@dagardner-nv dagardner-nv marked this pull request as draft December 11, 2023 18:17
@dagardner-nv dagardner-nv changed the title Draft: Add benchmark for vdb upload pipeline Add benchmark for vdb upload pipeline Dec 11, 2023
@dagardner-nv dagardner-nv marked this pull request as ready for review December 11, 2023 20:01
@dagardner-nv dagardner-nv added non-breaking Non-breaking change improvement Improvement to existing functionality labels Dec 11, 2023
@dagardner-nv dagardner-nv changed the title Add benchmark for vdb upload pipeline Add benchmark for stand-alone RAG & vdb upload pipelines Dec 11, 2023
@dagardner-nv dagardner-nv changed the title Add benchmark for stand-alone RAG & vdb upload pipelines Add benchmarks for stand-alone RAG & vdb upload pipelines Dec 11, 2023
@dagardner-nv dagardner-nv mentioned this pull request Dec 12, 2023
Merged
mdemoret-nv
mdemoret-nv approved these changes Dec 13, 2023
View reviewed changes
tests/benchmarks/test_bench_vdb_upload_pipeline.py Outdated Show resolved Hide resolved
tests/benchmarks/test_bench_vdb_upload_pipeline.py Outdated Show resolved Hide resolved
@dagardner-nv
Copy link
Contributor Author

/merge

@rapids-bot rapids-bot bot merged commit fb35a5e into nv-morpheus:branch-24.03 Dec 14, 2023
6 checks passed
rapids-bot bot pushed a commit that referenced this pull request Dec 14, 2023
@dagardner-nv
Add mocked test & benchmark for LLM agents pipeline (#1424)
fbc68de 
* Add a test for the LLM agents pipeline that doesn't communicate with OpenAI or SerpAPI.
* Add benchmark for LLM agents pipeline.

Includes changes from PR #1421

## By Submitting this PR I confirm:
- I am familiar with the [Contributing Guidelines](https://github.com/nv-morpheus/Morpheus/blob/main/docs/source/developer_guide/contributing.md).
- When the PR is ready for review, new or existing tests cover these changes.
- When the PR is ready for review, the documentation is up to date with these changes.

Authors:
  - David Gardner (https://github.com/dagardner-nv)

Approvers:
  - Michael Demoret (https://github.com/mdemoret-nv)

URL: #1424
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@mdemoret-nv mdemoret-nv mdemoret-nv approved these changes

Assignees

@dagardner-nv dagardner-nv

Labels
improvement Improvement to existing functionality non-breaking Non-breaking change
Projects
Morpheus Boards
Archived in project
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@dagardner-nv @mdemoret-nv