KAFKA-19648: Formatter refactoring #20707
Conversation
mannoopj
changed the title
There was a problem hiding this comment.
Thanks for the changes @mannoopj. Some high level comments:
| static void writeDynamicQuorumSnapshot( | ||
| String writeLogDir, | ||
| DynamicVoters initialControllers, | ||
| short kraftVersion, | ||
| String controllerListenerName | ||
| ) { | ||
| File parentDir = new File(writeLogDir); | ||
| File clusterMetadataDirectory = new File(parentDir, String.format("%s-%d", | ||
| CLUSTER_METADATA_TOPIC_PARTITION.topic(), | ||
| CLUSTER_METADATA_TOPIC_PARTITION.partition())); | ||
| VoterSet voterSet = initialControllers.toVoterSet(controllerListenerName); | ||
| RecordsSnapshotWriter.Builder builder = new RecordsSnapshotWriter.Builder(). | ||
| setLastContainedLogTimestamp(Time.SYSTEM.milliseconds()). | ||
| setMaxBatchSizeBytes(KafkaRaftClient.MAX_BATCH_SIZE_BYTES). | ||
| setRawSnapshotWriter(FileRawSnapshotWriter.create( | ||
| clusterMetadataDirectory.toPath(), | ||
| Paths.get(writeLogDir), | ||
| Snapshots.BOOTSTRAP_SNAPSHOT_ID)). | ||
| setKraftVersion(KRaftVersion.fromFeatureLevel(kraftVersion)). | ||
| setVoterSet(Optional.of(voterSet)); | ||
| try (RecordsSnapshotWriter<ApiMessageAndVersion> writer = builder.build(new MetadataRecordSerde())) { | ||
| writer.freeze(); | ||
| } | ||
| } |
There was a problem hiding this comment.
There is a clearer way to make these changes. This method is what writes the 0-0.checkpoint currently. We should pass the bootstrap metadata object here and append the metadata records using writer.append.append(bootstrapMetadata.records()) before calling writer.freeze().
| if (directoryTypes.get(writeLogDir).isDynamicMetadataDirectory()) { | ||
| writeDynamicQuorumSnapshot(writeLogDir, | ||
| writeDynamicQuorumSnapshot(clusterMetadataDirectory.getPath(), |
There was a problem hiding this comment.
We should rename writeDynamicQuorumSnapshot to writeZeroSnapshot, and write it when formatting metadata directories. The semantics that change here are that we should not write the KRaft control records (KRaft version and voter set) when !isDynamicMetadataDirectory().
| File createdBoostrapCheckpoint = new File(clusterMetadataDirectory.getPath() + "/" + BootstrapDirectory.BINARY_BOOTSTRAP_FILENAME); | ||
| File created000Checkpoint = new File(clusterMetadataDirectory.getPath() + "/" + BootstrapDirectory.BINARY_CHECKPOINT_FILENAME); | ||
| Files.write( | ||
| createdBoostrapCheckpoint.toPath(), | ||
| Files.readAllBytes(created000Checkpoint.toPath()), | ||
| StandardOpenOption.APPEND); | ||
| try { | ||
| created000Checkpoint.delete(); | ||
| createdBoostrapCheckpoint.renameTo(created000Checkpoint); | ||
| } catch (Exception ex) { | ||
| throw new RuntimeException("Failed operation to combine metadata and kraft records: ", ex); | ||
| } | ||
| } else { | ||
| File createdBoostrapCheckpoint = new File(clusterMetadataDirectory.getPath() + "/" + BootstrapDirectory.BINARY_BOOTSTRAP_FILENAME); | ||
| File created000Checkpoint = new File(clusterMetadataDirectory.getPath() + "/" + BootstrapDirectory.BINARY_CHECKPOINT_FILENAME); | ||
| try { | ||
| createdBoostrapCheckpoint.renameTo(created000Checkpoint); | ||
| } catch (Exception ex) { | ||
| throw new RuntimeException("Failed to rename file: ", ex); | ||
| } |
There was a problem hiding this comment.
This code is confusing. Instead of doing this renaming and deleting. We should instead remove the call to write the bootstrap metadata to disk on Line 447, since we're no longer writing bootstrap.checkpoint anymore, and follow the other comments for writing metadata records to 0-0.checkpoint.
We can check if an old bootstrap.checkpoint exists and delete it, since IIRC that was part of the KIP.
mannoopj
force-pushed
the
kip-1170-format
branch
from
42e105a to
a9c79a3
Compare
| try (RecordsSnapshotWriter<ApiMessageAndVersion> writer = builder.build(new MetadataRecordSerde(), Optional.of(bootstrapMetadata.records()))) { | ||
| writer.freeze(); |
There was a problem hiding this comment.
| try (RecordsSnapshotWriter<ApiMessageAndVersion> writer = builder.build(new MetadataRecordSerde(), Optional.of(bootstrapMetadata.records()))) { | |
| writer.freeze(); | |
| try (RecordsSnapshotWriter<ApiMessageAndVersion> writer = builder.build(new MetadataRecordSerde()))) { | |
| writer.append(bootstrapMetadata.records()); | |
| writer.freeze(); |
There was a problem hiding this comment.
Wouldn't this write the bootstrap records after the control records, since in RecordsSnapshotWriter.build() is where we append the kraft records and we would be calling that ahead of writer.append in this scenario? we want the bootstrap records ahead correct?
There was a problem hiding this comment.
It doesn't matter what order we write them in because KRaft only reads the control records, and the metadata module will only read the "data" records. When we read the 0-0.checkpoint back into memory, we only deal with either its control records or its data records, not both in the same code.
| .setKraftVersion(KRaftVersion.KRAFT_VERSION_1) | ||
| .setVoterSet(Optional.of(VoterSetTest.voterSet(VoterSetTest.voterMap(IntStream.of(1, 2, 3), true)))) | ||
| .build(MetadataRecordSerde.INSTANCE) | ||
| .build(MetadataRecordSerde.INSTANCE, emptyOptional) |
There was a problem hiding this comment.
We should be able to remove this. This applies to other instances where we build the RecordsSnapshotWriter.
| } | ||
| writeBoostrapSnapshot(writeLogDir, | ||
| bootstrapMetadata, | ||
| initialControllers.get(), |
There was a problem hiding this comment.
We need to pass the optional for initialControllers here. We can only do a .get() if initialControllers.isPresent().
| File clusterMetadataDirectory = new File(parentDir, String.format("%s-%d", | ||
| CLUSTER_METADATA_TOPIC_PARTITION.topic(), | ||
| CLUSTER_METADATA_TOPIC_PARTITION.partition())); | ||
| VoterSet voterSet = initialControllers.toVoterSet(controllerListenerName); |
There was a problem hiding this comment.
We should only construct the VoterSet object if initialControllers.isPresent().
2 participants
KIP-1170
Change formatter code to ensure that when the directories are formatted through the kafka storage command, metadata records normally written to the bootstrap.checkpoint file are instead located in the 00000000000000000000-0000000000.checkpoint file. Then if any kraft records generated during this format they will be located after the metadata records in the 00000000000000000000-0000000000.checkpoint file. This PR is currently a draft and opened to allow for reviews to code changes to formatter for this kip.