Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] Test matrix difficult to grok #826

Open
dblock opened this issue Sep 23, 2024 · 0 comments
Open

[BUG] Test matrix difficult to grok #826

dblock opened this issue Sep 23, 2024 · 0 comments
Labels
bug Something isn't working

Comments

@dblock
Copy link
Member

dblock commented Sep 23, 2024

What is the bug?

Coming from #823 (comment), there's some code that defines "server tests" which makes it difficult to know what tests actually run in

ignores = [
"test_opensearchpy/test_server/",
"test_opensearchpy/test_server_secured/",
"test_opensearchpy/test_async/test_server/",
"test_opensearchpy/test_async/test_server_secured/",
]
.

So what tests actually run and when?

What is the expected behavior?

It should be easy to figure out what runs when. Test files shouldn't be including/excluding anything with nested if's or checking whether we are running with or without plugins.

Refactor the test setup.

  1. By default all test are unit tests and should all run without requiring an actual OpenSearch instance.
  2. A folder called integ should contain integration tests that can be run in bulk and require an OpenSearch instance.
  3. If we want to keep plugin tests separate, then there should be a plugins/<name of plugin> folder to run those.

CI/CD should invoke unit tests by default, and integration tests otherwise across a matrix of versions.

@dblock dblock added bug Something isn't working untriaged Need triage labels Sep 23, 2024
@saimedhi saimedhi removed the untriaged Need triage label Sep 25, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants