Add common perftest classes for data generation + use in confluence to demonstrate #1590
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
seanstory
reviewed
Sep 7, 2023
|
Started the migration of other sources in #1607. Did it for ABS and Dropbox there |
seanstory
previously approved these changes
Sep 8, 2023
|
Unclear to me why CI is failing, we'll need to dig into that. |
artem-shelkovnikov
deleted the
artem/add-common-perftest-class-and-use-in-confluence
branch
💔 Failed to create backport PR(s)The backport operation could not be completed due to the following error: The backport PRs will be merged automatically after passing CI. To backport manually run: |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Part of https://github.com/elastic/enterprise-search-team/issues/3397
Adding a series of classes that can be used to have common setup for performance tests.
WeightedFakeProviderclass allows to have a class that returns random small/medium/large/extra_large text/html files with given distribution.For example, by default distribution is
[0.58, 0.3, 0.1, 0.02], meaning that small items will be returned with 58% chance, medium-sized items with 30% chance, large with 10% and extra_large with 2%.The idea is to have this class used in all functional tests to have uniform test setup, so that we could do better job at estimating the memory consumption of each connector.
One thing to consider: I had to change all
python <FIXTURE_FILE.py>tobin/python <FIXTURE_FILE.py>for it to work, because the shared class is inside of the connectors library, and this library is not installed locally - it's installed inside the venv. So to use it, we need to use python that's distributed in venv as well. IMO it makes sense.Checklists
Pre-Review Checklist
v7.13.2,v7.14.0,v8.0.0)