Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Backport 2024.1] fix(nemesis.py): log nemesis start/end on db nodes logs #9797

Draft
wants to merge 1 commit into
base: branch-2024.1
Choose a base branch
from

Conversation

scylladbbot
Copy link

Add logging of the following nemesis details in logs of each db node of a cluster under test:

  • nemesis start with disruption/nemesis name
  • target node for the nemesis
  • nemesis end with status

Closes: #6088

Testing

Example of how the start/end is logged in:

  • sct.log
❯ egrep '>>>>|<<<<.*disruption' -irn sct-6288cf64.log
21138:< t:2025-01-13 16:05:06,583 f:nemesis.py      l:5469 c:sdcm.nemesis         p:DEBUG > sdcm.nemesis.SisyphusMonkey: >>>>>>>>>>>> Started disruption disrupt_kill_scylla (KillScylla nemesis) on the target node 'Node PR-provision-test-log-neme-db-node-6288cf64-2 [3.253.12.15 | 10.4.3.181]' >>>>>>>>>>>>
22492:< t:2025-01-13 16:05:30,440 f:nemesis.py      l:5556 c:sdcm.nemesis         p:DEBUG > sdcm.nemesis.SisyphusMonkey: <<<<<<<<<<<< Finished disruption disrupt_kill_scylla (KillScylla nemesis) with status 'succeeded' <<<<<<<<<<<<
  • db node log
❯ egrep '======.*disruption' -irn db-cluster-6288cf64/PR-provision-test-log-neme-db-node-6288cf64-*/system.log
db-cluster-6288cf64/PR-provision-test-log-neme-db-node-6288cf64-1/system.log:1145:Jan 13 16:05:07.963018 PR-provision-test-log-neme-db-node-6288cf64-1 scylla[5435]:  [shard 0:strm] api - /system/log: ============ Started disruption disrupt_kill_scylla (KillScylla nemesis) on the target node 'Node PR-provision-test-log-neme-db-node-6288cf64-2 [3.253.12.15 | 10.4.3.181]' ============
db-cluster-6288cf64/PR-provision-test-log-neme-db-node-6288cf64-1/system.log:1174:Jan 13 16:05:31.041321 PR-provision-test-log-neme-db-node-6288cf64-1 scylla[5435]:  [shard 0:strm] api - /system/log: ============ Finished disruption disrupt_kill_scylla (KillScylla nemesis) with status 'succeeded' ============
db-cluster-6288cf64/PR-provision-test-log-neme-db-node-6288cf64-2/system.log:1077:Jan 13 16:05:08.605344 PR-provision-test-log-neme-db-node-6288cf64-2 scylla[5447]:  [shard 0:strm] api - /system/log: ============ Started disruption disrupt_kill_scylla (KillScylla nemesis) on the target node 'Node PR-provision-test-log-neme-db-node-6288cf64-2 [3.253.12.15 | 10.4.3.181]' ============
db-cluster-6288cf64/PR-provision-test-log-neme-db-node-6288cf64-2/system.log:1626:Jan 13 16:05:31.620427 PR-provision-test-log-neme-db-node-6288cf64-2 scylla[5910]:  [shard 1:strm] api - /system/log: ============ Finished disruption disrupt_kill_scylla (KillScylla nemesis) with status 'succeeded' ============
db-cluster-6288cf64/PR-provision-test-log-neme-db-node-6288cf64-3/system.log:1058:Jan 13 16:05:09.198262 PR-provision-test-log-neme-db-node-6288cf64-3 scylla[5456]:  [shard 0:strm] api - /system/log: ============ Started disruption disrupt_kill_scylla (KillScylla nemesis) on the target node 'Node PR-provision-test-log-neme-db-node-6288cf64-2 [3.253.12.15 | 10.4.3.181]' ============
db-cluster-6288cf64/PR-provision-test-log-neme-db-node-6288cf64-3/system.log:1065:Jan 13 16:05:32.099599 PR-provision-test-log-neme-db-node-6288cf64-3 scylla[5456]:  [shard 0:strm] api - /system/log: ============ Finished disruption disrupt_kill_scylla (KillScylla nemesis) with status 'succeeded' ============

PR pre-checks (self review)

  • I added the relevant backport labels
  • I didn't leave commented-out/debugging code

Reminders

  • Add New configuration option and document them (in sdcm/sct_config.py)

  • Add unit tests to cover my changes (under unit-test/ folder)

  • Update the Readme/doc folder relevant to this change (if needed)

  • (cherry picked from commit da19d86)

Parent PR: #9774

Add logging of the following nemesis details in logs of each db node of
a cluster under test:
- nemesis start with disruption/nemesis name
- target node for the nemesis
- nemesis end with status

(cherry picked from commit da19d86)
@scylladbbot
Copy link
Author

@dimakr - This PR has conflicts, therefore it was moved to draft
Please resolve them and mark this PR as ready for review

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants