Fix functional tests failures for STF tests by elfiesmelfie · Pull Request #166 · infrawatch/feature-verification-tests

elfiesmelfie · 2024-10-29T20:14:33Z

The issue may have been caused by the observability strategy tests causing both community and redhat monitoring components to be installed and shadowing the monitoring component definitions (prom, alertmanager, etc)

Depends-On: #177

elfiesmelfie · 2024-11-20T15:58:36Z

recheck

softwarefactory-project-zuul · 2024-12-05T13:51:51Z

Merge Failed.

This change or one of its cross-repo dependencies was unable to be automatically merged with the current state of its repository. Please rebase the change and upload a new patchset.
Warning:
Error merging github.com/infrawatch/feature-verification-tests for 166,2503c4afab8eaa204bbd76b3b965a57f513aa587

softwarefactory-project-zuul · 2024-12-06T15:03:34Z

Merge Failed.

This change or one of its cross-repo dependencies was unable to be automatically merged with the current state of its repository. Please rebase the change and upload a new patchset.
Warning:
Error merging github.com/infrawatch/feature-verification-tests for 166,a9a04c5158b0739624dadb794aa87b11cc5ac99b

softwarefactory-project-zuul · 2024-12-16T15:00:40Z

Merge Failed.

This change or one of its cross-repo dependencies was unable to be automatically merged with the current state of its repository. Please rebase the change and upload a new patchset.
Warning:
Error merging github.com/infrawatch/feature-verification-tests for 166,119f7197dd557f8a5d286d7fb156f21bafc4e272

softwarefactory-project-zuul · 2024-12-16T21:10:21Z

Merge Failed.

This change or one of its cross-repo dependencies was unable to be automatically merged with the current state of its repository. Please rebase the change and upload a new patchset.
Warning:
Error merging github.com/infrawatch/feature-verification-tests for 166,08ab06b0d20b820c0ec450c661ede3ba2f793bda

elfiesmelfie · 2025-03-05T12:45:20Z

  changed_when: false
  failed_when: container_nodes.stdout_lines|length != 0

+- name: Check what metrics are available to prometheus that relate to sensubility


@lnatapov I think that making this a new test would be a good idea.

It validates a pre-req for later sensubility tests i.e. There are sensubility metrics available

It would need a failed_when condition to make it useful

ayefimov-1

A few comments that have not been resolved but looks fine to me.

…e alarm

* Update the alarm to use FVT_TESTING in the alarm description so we can identify the alarm created by this test * Replace grep with an updated failed_when condition * Correct the failed_when condition to match the alarm message

* Retry the check for alarm creation TODO: * Add check for Prometheus rule deletion

Fix ansible-lint errors

This avoids some errors in parsing the alertmanager config, which caused delays and the alarm not to appear in the logs

…tch`

Increase retry count.

…n_alert_manager.yml

vkmc

LGTM

elfiesmelfie force-pushed the fix_functional_test_failures branch 2 times, most recently from b8ffcb7 to ff8ef96 Compare October 31, 2024 18:57

elfiesmelfie force-pushed the fix_functional_test_failures branch 3 times, most recently from 9f933df to 3fda2d0 Compare November 19, 2024 20:53

elfiesmelfie force-pushed the fix_functional_test_failures branch 6 times, most recently from 17863ff to 91990e9 Compare November 27, 2024 20:08

elfiesmelfie force-pushed the fix_functional_test_failures branch 6 times, most recently from 8353e87 to 2503c4a Compare December 5, 2024 13:51

elfiesmelfie force-pushed the fix_functional_test_failures branch from 2503c4a to a381325 Compare December 5, 2024 17:52

elfiesmelfie force-pushed the fix_functional_test_failures branch 4 times, most recently from de97212 to 2d46452 Compare December 10, 2024 18:41

elfiesmelfie force-pushed the fix_functional_test_failures branch 2 times, most recently from 410547e to 3190efb Compare December 17, 2024 18:07

elfiesmelfie force-pushed the fix_functional_test_failures branch from 3190efb to a771ce8 Compare January 10, 2025 19:13

elfiesmelfie commented Mar 5, 2025

View reviewed changes

Comment thread roles/test_sensubility/tasks/test_health_status.yml Outdated

elfiesmelfie commented Mar 5, 2025

View reviewed changes

elfiesmelfie force-pushed the fix_functional_test_failures branch from ea0fbf3 to c89d25f Compare June 3, 2025 14:26

elfiesmelfie commented Jun 3, 2025

View reviewed changes

Comment thread roles/test_sensubility/tasks/test_health_status.yml Outdated

Comment thread roles/test_sensubility/tasks/test_health_status.yml Outdated

elfiesmelfie commented Jun 3, 2025

View reviewed changes

Comment thread roles/test_snmp_traps/tasks/main.yml

elfiesmelfie marked this pull request as ready for review June 3, 2025 14:35

elfiesmelfie commented Jun 3, 2025

View reviewed changes

Comment thread roles/test_alerts/tasks/test_creating_a_standard_alert_route_in_alert_manager.yml Outdated

elfiesmelfie enabled auto-merge (squash) June 3, 2025 15:12

ayefimov-1 approved these changes Jun 4, 2025

View reviewed changes

elfiesmelfie mentioned this pull request Jun 4, 2025

[test_sensubility] Increase number of retries on metrics check #274

Merged

elfiesmelfie and others added 15 commits June 4, 2025 13:44

[test_snmp_traps] Remove grep and update failed_when

924bab0

[test_snmp_traps] Undo update to stf object post-test

0a3daeb

[test_snmp_traps] Wait up to two minutes for the snmp logs to show th…

ccb9713

…e alarm

[test_alerts] Update alertmanager test

1bc3af9

Update test_alerts

c34efe1

* Update the alarm to use FVT_TESTING in the alarm description so we can identify the alarm created by this test * Replace grep with an updated failed_when condition * Correct the failed_when condition to match the alarm message

[test_alerts] Update pre- and post-test tasks

79434f3

* Retry the check for alarm creation TODO: * Add check for Prometheus rule deletion

[test_alerts] Use command instead of shell

6411826

Fix ansible-lint errors

[test_verify_email] Add quotes to the receiver name

a82ed0e

This avoids some errors in parsing the alertmanager config, which caused delays and the alarm not to appear in the logs

[test_alerts][test_snmp_traps] Add meaningful changed_when for `oc pa…

926b77a

…tch`

[temp][test_sensubility] Add more detail to the healthcheck

5a70792

[test_sensubility] Check whether sensubility metrics are available

f502103

Update test_health_status.yml

23731ca

Increase retry count.

Remove unneeded comments

00d8ff8

Remove extra line

0099e23

Update roles/test_alerts/tasks/test_creating_a_standard_alert_route_i…

93eac82

…n_alert_manager.yml

elfiesmelfie force-pushed the fix_functional_test_failures branch from 25c8f8b to 93eac82 Compare June 4, 2025 12:44

elfiesmelfie requested a review from vkmc June 4, 2025 13:11

vkmc approved these changes Jun 4, 2025

View reviewed changes

elfiesmelfie merged commit 0142f05 into master Jun 4, 2025
9 checks passed

elfiesmelfie deleted the fix_functional_test_failures branch June 4, 2025 13:36

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix functional tests failures for STF tests#166

Fix functional tests failures for STF tests#166
elfiesmelfie merged 15 commits intomasterfrom
fix_functional_test_failures

elfiesmelfie commented Oct 29, 2024 •

edited

Loading

Uh oh!

elfiesmelfie commented Nov 20, 2024

Uh oh!

softwarefactory-project-zuul Bot commented Dec 5, 2024

Uh oh!

softwarefactory-project-zuul Bot commented Dec 6, 2024

Uh oh!

softwarefactory-project-zuul Bot commented Dec 16, 2024

Uh oh!

softwarefactory-project-zuul Bot commented Dec 16, 2024

Uh oh!

Uh oh!

elfiesmelfie Mar 5, 2025

Uh oh!

elfiesmelfie Mar 5, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ayefimov-1 left a comment

Uh oh!

vkmc left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Milestone

Development

Uh oh!

3 participants

Conversation

elfiesmelfie commented Oct 29, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

elfiesmelfie commented Nov 20, 2024

Uh oh!

softwarefactory-project-zuul Bot commented Dec 5, 2024

Uh oh!

softwarefactory-project-zuul Bot commented Dec 6, 2024

Uh oh!

softwarefactory-project-zuul Bot commented Dec 16, 2024

Uh oh!

softwarefactory-project-zuul Bot commented Dec 16, 2024

Uh oh!

Uh oh!

elfiesmelfie Mar 5, 2025

Choose a reason for hiding this comment

Uh oh!

elfiesmelfie Mar 5, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ayefimov-1 left a comment

Choose a reason for hiding this comment

Uh oh!

vkmc left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Milestone

Development

Uh oh!

3 participants

elfiesmelfie commented Oct 29, 2024 •

edited

Loading