TestPurgeIntegration is flaky

[Michal-Leszczynski]

It looks like sometimes SM does not purge all manifests in TestPurgeIntegration:

2024-12-16T12:06:48.5510081Z --- Then: there should be 3 + 4 manifests
2024-12-16T12:06:48.5629775Z     service_backup_integration_test.go:1564: Expected 7 manifests (1 per each node) plus 4 generated, got 8 backup/meta/cluster/b9a74faf-dfef-498b-853f-943b81ca38cb/dc/dc1/node/03445f8d-a8f6-4397-861b-23ff71c1fb28/task_070e03d3-b12d-46c6-bb8c-a1f62183fbd2_tag_sm_20241215120647UTC_manifest.json.gz
2024-12-16T12:06:48.5633543Z         backup/meta/cluster/b9a74faf-dfef-498b-853f-943b81ca38cb/dc/dc1/node/03445f8d-a8f6-4397-861b-23ff71c1fb28/task_1f795853-fb1b-4c7a-8e80-928631431176_tag_sm_20241219120647UTC_manifest.json.gz
2024-12-16T12:06:48.5636397Z         backup/meta/cluster/b9a74faf-dfef-498b-853f-943b81ca38cb/dc/dc1/node/03445f8d-a8f6-4397-861b-23ff71c1fb28/task_2f2aaeb1-0922-4378-a217-8c7291df2014_tag_sm_20241216120647UTC_manifest.json.gz
2024-12-16T12:06:48.5638228Z         backup/meta/cluster/b9a74faf-dfef-498b-853f-943b81ca38cb/dc/dc1/node/03445f8d-a8f6-4397-861b-23ff71c1fb28/task_421ce0b2-3b0c-4719-8bb6-3b5259029f5e_tag_sm_20241215100647UTC_manifest.json.gz
2024-12-16T12:06:48.5640671Z         backup/meta/cluster/b9a74faf-dfef-498b-853f-943b81ca38cb/dc/dc1/node/03445f8d-a8f6-4397-861b-23ff71c1fb28/task_421ce0b2-3b0c-4719-8bb6-3b5259029f5e_tag_sm_20241216100647UTC_manifest.json.gz
2024-12-16T12:06:48.5642477Z         backup/meta/cluster/b9a74faf-dfef-498b-853f-943b81ca38cb/dc/dc1/node/41f1e4e1-a8a8-4324-949f-b621f774748a/task_2f2aaeb1-0922-4378-a217-8c7291df2014_tag_sm_20241216120647UTC_manifest.json.gz
2024-12-16T12:06:48.5644993Z         backup/meta/cluster/b9a74faf-dfef-498b-853f-943b81ca38cb/dc/dc1/node/4fffdd5c-430b-4144-bc84-8d632cb7f879/task_2f2aaeb1-0922-4378-a217-8c7291df2014_tag_sm_20241216120647UTC_manifest.json.gz
2024-12-16T12:06:48.5647135Z         backup/meta/cluster/b9a74faf-dfef-498b-853f-943b81ca38cb/dc/dc1/node/f8ba36d2-1919-47a7-8071-ad78e01c2630/task_2f2aaeb1-0922-4378-a217-8c7291df2014_tag_sm_20241216120647UTC_manifest.json.gz
2024-12-16T12:06:48.5648230Z --- FAIL: TestPurgeIntegration (6.13s)

First occurrence on an already merged PR: #4120 (comment)
Second occurrence on already merged master: https://github.com/scylladb/scylla-manager/actions/runs/12352036520/job/34468208926

Perhaps it's the minio lagging behind the test on gh actions machines?

[VAveryanov8]

Another occurrence in #4175

TestPurgeIntegration

--- Then: there should be 3 + 4 manifests
    service_backup_integration_test.go:1565: Expected 7 manifests (1 per each node) plus 4 generated, got 8 backup/meta/cluster/aa3bd91b-35d3-48e7-a0ee-7c4fc56996b1/dc/dc1/node/04b34d6d-efe9-4ed6-b19a-76e12cd0ac55/task_2e4e674d-977b-417e-9bb1-8324fa7421b7_tag_sm_20241220120743UTC_manifest.json.gz
        backup/meta/cluster/aa3bd91b-35d3-48e7-a0ee-7c4fc56996b1/dc/dc1/node/04b34d6d-efe9-4ed6-b19a-76e12cd0ac55/task_e960c937-d997-4043-9a2f-c75073d5f95f_tag_sm_20241216100743UTC_manifest.json.gz
        backup/meta/cluster/aa3bd91b-35d3-48e7-a0ee-7c4fc56996b1/dc/dc1/node/04b34d6d-efe9-4ed6-b19a-76e12cd0ac55/task_e960c937-d997-4043-9a2f-c75073d5f95f_tag_sm_20241217100743UTC_manifest.json.gz
        backup/meta/cluster/aa3bd91b-35d3-48e7-a0ee-7c4fc56996b1/dc/dc1/node/04b34d6d-efe9-4ed6-b19a-76e12cd0ac55/task_ee4153d4-fa91-4f76-bc2a-6821452eeba2_tag_sm_20241216120743UTC_manifest.json.gz
        backup/meta/cluster/aa3bd91b-35d3-48e7-a0ee-7c4fc56996b1/dc/dc1/node/04b34d6d-efe9-4ed6-b19a-76e12cd0ac55/task_f33d1d49-da25-4715-b0e0-d787ebf4dfde_tag_sm_20241217120743UTC_manifest.json.gz
        backup/meta/cluster/aa3bd91b-35d3-48e7-a0ee-7c4fc56996b1/dc/dc1/node/360fe131-e59b-46d9-acd3-4730947499ab/task_f33d1d49-da25-4715-b0e0-d787ebf4dfde_tag_sm_20241217120743UTC_manifest.json.gz
        backup/meta/cluster/aa3bd91b-35d3-48e7-a0ee-7c4fc56996b1/dc/dc1/node/ac764087-2004-4fa1-ba97-a3ca5bd22b30/task_f33d1d49-da25-4715-b0e0-d787ebf4dfde_tag_sm_20241217120743UTC_manifest.json.gz
        backup/meta/cluster/aa3bd91b-35d3-48e7-a0ee-7c4fc56996b1/dc/dc1/node/b5502919-fcb7-40d0-a066-caa98b12867e/task_f33d1d49-da25-4715-b0e0-d787ebf4dfde_tag_sm_20241217120743UTC_manifest.json.gz
--- FAIL: TestPurgeIntegration (7.63s) 

[Michal-Leszczynski]

Another occurrence: https://github.com/scylladb/scylla-manager/actions/runs/12392307925/job/34591202258?pr=4179