Problem
ShardAwarenessTest.correctShardInTracingTest intermittently fails in the 3.x Scylla integration-test matrix because the query trace sometimes does not include any event whose description contains querying locally.
Observed on PR #943, head efe29bf2e24b357417d7134b6ac39c9659f9cd38, in Scylla ITs (LATEST, 8):
https://github.com/scylladb/java-driver/actions/runs/28683913680/job/85072817714
Failure:
java.lang.AssertionError: No 'querying locally' trace event was observed for the query expected [true] but found [false]
at com.datastax.driver.core.ShardAwarenessTest.verifyCorrectShardSingleRow(ShardAwarenessTest.java:95)
at com.datastax.driver.core.ShardAwarenessTest.correctShardInTracingTest(ShardAwarenessTest.java:115)
The same surefire result shows that this failed only on the first attempt and then passed on retry:
Run 1: ShardAwarenessTest.correctShardInTracingTest:115->verifyCorrectShardSingleRow:95 No 'querying locally' trace event was observed for the query expected [true] but found [false]
Run 2: PASS
A similar failure was also seen in an earlier PR #943 run, Scylla ITs (LTS-LATEST, 8), where the first invocation failed at ShardAwarenessTest.java:113 and the retry passed:
https://github.com/scylladb/java-driver/actions/runs/28670230744/job/85031593100
Notes
This does not look directly related to PR #943: that PR changes only Connection, RequestHandler, and RequestHandlerTest, while the failing test is ShardAwarenessTest.
There is a related merged fix in #931 for a different ShardAwarenessTest.correctShardInTracingTest failure on Scylla 2025.1. That fix made the anyLocal assertion real, so this missing-querying locally case now fails visibly, but I did not find an existing GitHub or Jira issue tracking this exact flaky symptom.
Expected
ShardAwarenessTest.correctShardInTracingTest should be deterministic in CI, or the test should wait/retry trace-event collection if the querying locally event can appear asynchronously.
Problem
ShardAwarenessTest.correctShardInTracingTestintermittently fails in the 3.x Scylla integration-test matrix because the query trace sometimes does not include any event whose description containsquerying locally.Observed on PR #943, head
efe29bf2e24b357417d7134b6ac39c9659f9cd38, inScylla ITs (LATEST, 8):https://github.com/scylladb/java-driver/actions/runs/28683913680/job/85072817714
Failure:
The same surefire result shows that this failed only on the first attempt and then passed on retry:
A similar failure was also seen in an earlier PR #943 run,
Scylla ITs (LTS-LATEST, 8), where the first invocation failed atShardAwarenessTest.java:113and the retry passed:https://github.com/scylladb/java-driver/actions/runs/28670230744/job/85031593100
Notes
This does not look directly related to PR #943: that PR changes only
Connection,RequestHandler, andRequestHandlerTest, while the failing test isShardAwarenessTest.There is a related merged fix in #931 for a different
ShardAwarenessTest.correctShardInTracingTestfailure on Scylla 2025.1. That fix made theanyLocalassertion real, so this missing-querying locallycase now fails visibly, but I did not find an existing GitHub or Jira issue tracking this exact flaky symptom.Expected
ShardAwarenessTest.correctShardInTracingTestshould be deterministic in CI, or the test should wait/retry trace-event collection if thequerying locallyevent can appear asynchronously.