[rally] Add Rally role#3996
Conversation
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
|
Build failed (check pipeline). Post ✔️ openstack-k8s-operators-content-provider SUCCESS in 2h 16m 50s |
|
Build failed (check pipeline). Post ❌ openstack-k8s-operators-content-provider FAILURE in 7m 00s |
|
Build failed (check pipeline). Post ✔️ openstack-k8s-operators-content-provider SUCCESS in 2h 39m 19s |
|
Build failed (check pipeline). Post ✔️ openstack-k8s-operators-content-provider SUCCESS in 2h 23m 47s |
|
Build failed (check pipeline). Post ✔️ openstack-k8s-operators-content-provider SUCCESS in 2h 11m 55s |
|
Build failed (check pipeline). Post ✔️ openstack-k8s-operators-content-provider SUCCESS in 2h 20m 46s |
|
Build failed (check pipeline). Post ✔️ openstack-k8s-operators-content-provider SUCCESS in 2h 15m 22s |
|
Build failed (check pipeline). Post ✔️ openstack-k8s-operators-content-provider SUCCESS in 2h 22m 37s |
|
Build failed (check pipeline). Post ✔️ openstack-k8s-operators-content-provider SUCCESS in 2h 21m 42s |
Testproject MRA testproject MR has been added to validate this implementation end-to-end Test ResultsThis POC confirms that the implementation is working correctly. Cinder workload: One test failure observed, but it appears to be a Rally bug, not an issue with the integration itself. Manila workload: The failures seen are configuration-related only and are not caused by the Rally role or its wiring into the test_operator framework. |
|
Build failed (check pipeline). Post ✔️ openstack-k8s-operators-content-provider SUCCESS in 2h 31m 07s |
lkuchlan
force-pushed
the
add-rally-role
branch
2 times, most recently
from
43000e6 to
9ba4734
Compare
|
Merge Failed. This change or one of its cross-repo dependencies was unable to be automatically merged with the current state of its repository. Please rebase the change and upload a new patchset. |
| @@ -0,0 +1,31 @@ | |||
| --- | |||
There was a problem hiding this comment.
I'm not sure this is the right way for doing this. The test-operator role is a wrapper against test-operator, so it should only support test runners which are supported by test-operator. Rally is not supported by test-operator. So please keep everything in the separate rally role, outside test-operator.
There was a problem hiding this comment.
You're right!
I've removed rally_runner.yml from test_operator and all cifmw_test_operator_rally_* variables. Rally is now invoked entirely through the standalone rally role via hooks/playbooks/rally_run.yaml.
|
|
||
| ```yaml | ||
| cifmw_run_tests: true | ||
| cifmw_run_test_role: test_operator |
There was a problem hiding this comment.
see above, this rally role can't be used with the test-operator role if test-operator itself does not support rally
lkuchlan
changed the title
|
Build failed (check pipeline). Post ❌ openstack-k8s-operators-content-provider FAILURE in 3m 57s |
|
Build failed (check pipeline). Post ❌ openstack-k8s-operators-content-provider FAILURE in 4m 24s |
| @@ -0,0 +1,15 @@ | |||
| --- | |||
There was a problem hiding this comment.
couldn't hooks/playbooks/manila_create_default_resources.yml be used instead, after overriding some variables?
There was a problem hiding this comment.
yes, we can use it.
rally only requires the share type, the same as tempest, so we can reuse this playbook with the required variable overrides.
Thanks for the suggestion!
Add a new `rally` Ansible role that runs OpenStack Rally benchmarks
inside a podman container (quay.io/airshipit/xrally-openstack:3.0.0).
The role auto-discovers OpenStack credentials from the cluster's
KeystoneAPI resource and appends the deployment CA to the trust bundle
so SSL verification works without any manual configuration.
The role supports two execution modes:
* Single run (default): `cifmw_rally_runs: []` — uses the top-level
`cifmw_rally_*` variables, artifacts stored in
`cifmw_rally_artifacts_basedir/`.
* Multi-run: set `cifmw_rally_runs` to a list of dicts, each with a
`name` key and optional `cifmw_rally_*` overrides. Artifacts for each
run are namespaced under `cifmw_rally_artifacts_basedir/<name>/`.
This is modelled after `cifmw_test_operator_stages` and replaces the
former test_operator-based wiring (Rally is not a test-operator CRD).
A `hooks/playbooks/rally_run.yaml` entry point is provided so jobs can
invoke Rally via the ci-framework hooks system, e.g.:
post_tests_90_rally_run:
type: playbook
source: rally_run.yaml
Assisted-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Signed-off-by: lkuchlan <lkuchlan@redhat.com>
|
Build failed (check pipeline). Post ✔️ openstack-k8s-operators-content-provider SUCCESS in 52m 58s |
|
recheck |
| @@ -1,34 +1,37 @@ | |||
| --- | |||
There was a problem hiding this comment.
I see you're now making changes here in hooks/playbooks/manila_create_default_resources.yml. Most (if not all) of these changes are, to me, outside the scope of adding a rally role. I think if you want these changes merged, they would be better submitted as a separate pr.
If I'm missing something and there is some meaningful logic change happening here, I apologize! But if that is the case, I think
- we should minimize the noisy style changes happening here and leave those to a separate pr
- keep any meaningful logic changes that might be happening here to a separate commit
There was a problem hiding this comment.
@michburk
The changes to manila_create_default_resources.yml are required for running Manila workload Rally tests. Rally's bundled manila.yaml workload hardcodes a share type named dhss_true, so that share type must exist before Rally runs. Rather than adding a duplicate playbook (rally_manila_prerequisites.yaml) that does the same thing as the existing one, I generalized manila_create_default_resources.yml to accept the share type name and extra specs as overridable variables.
I can separate this into a different PR, but since it's a minor change that is already done and directly tied to making Manila Rally workloads work, I'm not sure it's worth the overhead of an additional PR.
There was a problem hiding this comment.
Sorry if this is a dumb question, but which changes were required to enable this? Couldn't you already pass different values for share_type_name or extra_specs?
There was a problem hiding this comment.
And by all means if there are meaningful changes that need to happen to this playbook that enable it to be used with your rally role, that's a fine thing to include in this pr. I'm just confused because these changes all appear to be aesthetic. Sorry again if I'm missing something obvious 😵💫
There was a problem hiding this comment.
No worries, thanks for the feedback!
Good catch, technically you could already override share_type_name and extra_specs via extra_vars with the original names, since extra_vars has the highest precedence.
The rename to _ prefix was driven by ansible-lint compliance: this project enforces a var_naming_pattern that requires variables to either follow the cifmw__ pattern or start with _.
3 participants
Add a new `rally` Ansible role that runs OpenStack Rally benchmarks inside a podman
container (`quay.io/airshipit/xrally-openstack:3.0.0`). The role auto-discovers OpenStack
credentials from the cluster's KeystoneAPI resource and appends the deployment CA to the
trust bundle so SSL verification works without any manual configuration.
The role supports two execution modes:
under `cifmw_rally_artifacts_basedir/`.
optional `cifmw_rally_*` overrides. Artifacts are namespaced under
`cifmw_rally_artifacts_basedir//`.
Jobs invoke Rally via the ci-framework hooks system using `hooks/playbooks/rally_run.yaml`:
Key design decisions:
container can verify the Keystone TLS endpoint
Assisted-By: Claude Sonnet 4.6 noreply@anthropic.com