[Bug] Race condition when using the prefetchableClientReadHandler

[zuston]

Code of Conduct

• I agree to follow this project's Code of Conduct

Search before asking

• I have searched in the issues and found no similar issues.

Describe the bug

The bug happens on the code.

incubator-uniffle/storage/src/main/java/org/apache/uniffle/storage/handler/impl/DataSkippableReadHandler.java

Lines 90 to 110 in 124a901

	ShuffleDataResult result = null;
	while (segmentIndex < shuffleDataSegments.size()) {
	ShuffleDataSegment segment = shuffleDataSegments.get(segmentIndex);
	Roaring64NavigableMap blocksOfSegment = Roaring64NavigableMap.bitmapOf();
	segment.getBufferSegments().forEach(block -> blocksOfSegment.addLong(block.getBlockId()));
	// skip unexpected blockIds
	blocksOfSegment.and(expectBlockIds);
	if (!blocksOfSegment.isEmpty()) {
	// skip processed blockIds
	blocksOfSegment.or(processBlockIds);
	blocksOfSegment.xor(processBlockIds);
	if (!blocksOfSegment.isEmpty()) {
	result = readShuffleData(segment);
	segmentIndex++;
	break;
	}
	}
	segmentIndex++;
	}
	return result;
	}

The processedBlockId will be modified in the ShuffleReadClientImpl at the same time.

If so, we have to use synchronize to lock all related processBlockId operations.

Affects Version(s)

master

Uniffle Server Log Output

Uniffle Engine Log Output

Uniffle Server Configurations

Uniffle Engine Configurations

Additional context

cc @jerqi

Are you willing to submit PR?

• Yes I am willing to submit a PR!

[zuston]

cc @jerqi Do you have any ideas on this?

[jerqi]

cc @jerqi Do you have any ideas on this?

Maybe we can use concurrentHashMap instead of bitmap.

[summaryzb]

This is fixed by #2368