Skip to content

Commit

Permalink
Fix Flaky test GetShuffleReportForMultiPartTest (#241)
Browse files Browse the repository at this point in the history
### What changes were proposed in this pull request?
```
org.opentest4j.AssertionFailedError: expected: <0> but was: <-218>
	at org.junit.jupiter.api.AssertionUtils.fail(AssertionUtils.java:55)
	at org.junit.jupiter.api.AssertionUtils.failNotEqual(AssertionUtils.java:62)
	at org.junit.jupiter.api.AssertEquals.assertEquals(AssertEquals.java:150)
	at org.junit.jupiter.api.AssertEquals.assertEquals(AssertEquals.java:145)
	at org.junit.jupiter.api.Assertions.assertEquals(Assertions.java:527)
	at org.apache.uniffle.test.GetShuffleReportForMultiPartTest.validateRequestCount(GetShuffleReportForMultiPartTest.java:198)
	at org.apache.uniffle.test.GetShuffleReportForMultiPartTest.runTest(GetShuffleReportForMultiPartTest.java:185)
	at org.apache.uniffle.test.SparkIntegrationTestBase.runSparkApp(SparkIntegrationTestBase.java:74)
	at org.apache.uniffle.test.SparkIntegrationTestBase.run(SparkIntegrationTestBase.java:59)
	at org.apache.uniffle.test.GetShuffleReportForMultiPartTest.resultCompareTest(GetShuffleReportForMultiPartTest.java:141)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:725)
	at org.junit.jupiter.engine.execution.MethodInvocation.proceed(MethodInvocation.java:60)
	at 
```

### Why are the changes needed?
Fix flaky test


### Does this PR introduce _any_ user-facing change?
No

### How was this patch tested?
UT passed

Co-authored-by: leixianming <[email protected]>
  • Loading branch information
2 people authored and jerqi committed Oct 12, 2022
1 parent e9540e4 commit 7088e5f
Showing 1 changed file with 4 additions and 4 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -182,17 +182,17 @@ public int compare(String o1, String o2) {
int expectRequestNum = mockRssShuffleManager.getShuffleIdToPartitionNum().values().stream()
.mapToInt(x -> x.get()).sum();
// Validate getShuffleResultForMultiPart is correct before return result
validateRequestCount(expectRequestNum * replicateRead);
validateRequestCount(spark.sparkContext().applicationId(), expectRequestNum * replicateRead);
}
return map;
}

public void validateRequestCount(int expectRequestNum) {
public void validateRequestCount(String appId, int expectRequestNum) {
for (ShuffleServer shuffleServer : shuffleServers) {
MockedShuffleServerGrpcService service = ((MockedGrpcServer) shuffleServer.getServer()).getService();
Map<String, Map<Integer, AtomicInteger>> serviceRequestCount = service.getShuffleIdToPartitionRequest();
int requestNum = serviceRequestCount.entrySet().stream().flatMap(x -> x.getValue().values()
.stream()).mapToInt(AtomicInteger::get).sum();
int requestNum = serviceRequestCount.entrySet().stream().filter(x -> x.getKey().startsWith(appId))
.flatMap(x -> x.getValue().values().stream()).mapToInt(AtomicInteger::get).sum();
expectRequestNum -= requestNum;
}
assertEquals(0, expectRequestNum);
Expand Down

0 comments on commit 7088e5f

Please sign in to comment.