Skip to content

Conversation

BenPope
Copy link
Member

@BenPope BenPope commented Sep 24, 2025

KIP-514 suggests that Topic IDs should be rendered as base64 encoded, so switch logging to use base64.

Backports Required

  • none - not a bug fix
  • none - this is a backport
  • none - issue does not exist in previous branches
  • none - papercut/not impactful enough to backport
  • v25.2.x
  • v25.1.x
  • v24.3.x

Release Notes

  • none

@BenPope BenPope requested a review from a team September 24, 2025 23:10
@BenPope BenPope self-assigned this Sep 24, 2025
@BenPope BenPope requested review from IoannisRP, a team, Copilot and michael-redpanda and removed request for a team and IoannisRP September 24, 2025 23:10
Copy link
Contributor

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR implements base64 encoding for topic_id logging to align with KIP-514's recommendation for Topic ID rendering. The change affects how topic IDs are displayed in logs and formatted output throughout the system.

  • Adds base64 encoding for topic_id formatting via fmt::formatter specialization
  • Implements stream operator overloading for topic_id output consistency
  • Updates build dependencies to include the base64 utility module

Reviewed Changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.

File Description
src/v/model/model.cc Implements fmt::formatter for topic_id using base64 encoding and adds stream operator
src/v/model/fundamental.h Declares stream operator and fmt::formatter specialization for topic_id
src/v/model/BUILD Adds dependency on utils:base64 module

@BenPope BenPope force-pushed the core-13481-topic-id-base64 branch from 8a6643e to 4153519 Compare September 25, 2025 08:01
@vbotbuildovich
Copy link
Collaborator

vbotbuildovich commented Sep 25, 2025

CI test results

test results on build#72939
test_class test_method test_arguments test_kind job_url test_status passed reason test_history
ClusterRateQuotaTest test_client_group_produce_rate_throttle_mechanism null integration https://buildkite.com/redpanda/redpanda/builds/72939#0199801e-d7db-4748-a521-3b72bf017c40 FLAKY 16/21 upstream reliability is '83.38762214983714'. current run reliability is '76.19047619047619'. drift is 7.19715 and the allowed drift is set to 50. The test should PASS https://redpanda.metabaseapp.com/dashboard/87-tests?tab=142-dt-individual-test-history&test_class=ClusterRateQuotaTest&test_method=test_client_group_produce_rate_throttle_mechanism
ClusterRateQuotaTest test_client_response_throttle_mechanism_applies_to_next_request null integration https://buildkite.com/redpanda/redpanda/builds/72939#0199802a-4eb3-4d26-8e6b-df8f7b57cd80 FLAKY 15/21 upstream reliability is '82.9971181556196'. current run reliability is '71.42857142857143'. drift is 11.56855 and the allowed drift is set to 50. The test should PASS https://redpanda.metabaseapp.com/dashboard/87-tests?tab=142-dt-individual-test-history&test_class=ClusterRateQuotaTest&test_method=test_client_response_throttle_mechanism_applies_to_next_request
RecoveryModeTest test_rolling_restart null integration https://buildkite.com/redpanda/redpanda/builds/72939#0199802a-4eac-4777-ae1b-ba7028491ea2 FLAKY 16/21 upstream reliability is '94.48398576512456'. current run reliability is '76.19047619047619'. drift is 18.29351 and the allowed drift is set to 50. The test should PASS https://redpanda.metabaseapp.com/dashboard/87-tests?tab=142-dt-individual-test-history&test_class=RecoveryModeTest&test_method=test_rolling_restart
test results on build#72953
test_class test_method test_arguments test_kind job_url test_status passed reason test_history
ShadowLinkingReplicationTests test_replication_basic {"shuffle_leadership": false, "source_cluster_spec": {"cluster_type": "kafka", "kafka_quorum": "COMBINED_KRAFT", "kafka_version": "3.8.0"}} integration https://buildkite.com/redpanda/redpanda/builds/72953#01998160-c07d-4335-8163-0980697852b3 FLAKY 16/21 upstream reliability is '100.0'. current run reliability is '76.19047619047619'. drift is 23.80952 and the allowed drift is set to 50. The test should PASS https://redpanda.metabaseapp.com/dashboard/87-tests?tab=142-dt-individual-test-history&test_class=ShadowLinkingReplicationTests&test_method=test_replication_basic
ClusterRateQuotaTest test_client_group_consume_rate_throttle_mechanism null integration https://buildkite.com/redpanda/redpanda/builds/72953#0199813c-51e5-4942-b54b-bcb276d1f6c1 FLAKY 15/21 upstream reliability is '81.31229235880399'. current run reliability is '71.42857142857143'. drift is 9.88372 and the allowed drift is set to 50. The test should PASS https://redpanda.metabaseapp.com/dashboard/87-tests?tab=142-dt-individual-test-history&test_class=ClusterRateQuotaTest&test_method=test_client_group_consume_rate_throttle_mechanism
ClusterRateQuotaTest test_client_group_consume_rate_throttle_mechanism null integration https://buildkite.com/redpanda/redpanda/builds/72953#01998160-c078-4668-be67-e237d9cb69e4 FLAKY 13/21 upstream reliability is '81.31229235880399'. current run reliability is '61.904761904761905'. drift is 19.40753 and the allowed drift is set to 50. The test should PASS https://redpanda.metabaseapp.com/dashboard/87-tests?tab=142-dt-individual-test-history&test_class=ClusterRateQuotaTest&test_method=test_client_group_consume_rate_throttle_mechanism
ClusterRateQuotaTest test_client_group_produce_rate_throttle_mechanism null integration https://buildkite.com/redpanda/redpanda/builds/72953#0199813c-51e6-438c-8fa9-36d16da381f9 FLAKY 13/21 upstream reliability is '93.6768149882904'. current run reliability is '61.904761904761905'. drift is 31.77205 and the allowed drift is set to 50. The test should PASS https://redpanda.metabaseapp.com/dashboard/87-tests?tab=142-dt-individual-test-history&test_class=ClusterRateQuotaTest&test_method=test_client_group_produce_rate_throttle_mechanism
ClusterRateQuotaTest test_client_response_throttle_mechanism null integration https://buildkite.com/redpanda/redpanda/builds/72953#01998160-c07a-40f9-a228-9593087f50db FLAKY 15/21 upstream reliability is '81.8041634541249'. current run reliability is '71.42857142857143'. drift is 10.37559 and the allowed drift is set to 50. The test should PASS https://redpanda.metabaseapp.com/dashboard/87-tests?tab=142-dt-individual-test-history&test_class=ClusterRateQuotaTest&test_method=test_client_response_throttle_mechanism
ClusterRateQuotaTest test_client_response_throttle_mechanism_applies_to_next_request null integration https://buildkite.com/redpanda/redpanda/builds/72953#0199813c-51ea-4aa2-a3e9-a84eb65c6e73 FAIL 0/1 https://redpanda.metabaseapp.com/dashboard/87-tests?tab=142-dt-individual-test-history&test_class=ClusterRateQuotaTest&test_method=test_client_response_throttle_mechanism_applies_to_next_request
ClusterRateQuotaTest test_client_response_throttle_mechanism_applies_to_next_request null integration https://buildkite.com/redpanda/redpanda/builds/72953#01998160-c07b-48d7-83f6-5659822f77ad FLAKY 17/21 upstream reliability is '82.38434163701068'. current run reliability is '80.95238095238095'. drift is 1.43196 and the allowed drift is set to 50. The test should PASS https://redpanda.metabaseapp.com/dashboard/87-tests?tab=142-dt-individual-test-history&test_class=ClusterRateQuotaTest&test_method=test_client_response_throttle_mechanism_applies_to_next_request
DisablingPartitionsTest test_disable null integration https://buildkite.com/redpanda/redpanda/builds/72953#0199813c-51ee-4eff-863f-0dbe9d6c8095 FLAKY 8/21 upstream reliability is '86.02878916172735'. current run reliability is '38.095238095238095'. drift is 47.93355 and the allowed drift is set to 50. The test should PASS https://redpanda.metabaseapp.com/dashboard/87-tests?tab=142-dt-individual-test-history&test_class=DisablingPartitionsTest&test_method=test_disable
DisablingPartitionsTest test_disable null integration https://buildkite.com/redpanda/redpanda/builds/72953#01998160-c07d-4335-8163-0980697852b3 FLAKY 14/21 upstream reliability is '82.42375601926165'. current run reliability is '66.66666666666666'. drift is 15.75709 and the allowed drift is set to 50. The test should PASS https://redpanda.metabaseapp.com/dashboard/87-tests?tab=142-dt-individual-test-history&test_class=DisablingPartitionsTest&test_method=test_disable
WriteCachingFailureInjectionTest test_unavoidable_data_loss null integration https://buildkite.com/redpanda/redpanda/builds/72953#01998160-c078-4668-be67-e237d9cb69e4 FLAKY 18/21 upstream reliability is '95.22998296422487'. current run reliability is '85.71428571428571'. drift is 9.5157 and the allowed drift is set to 50. The test should PASS https://redpanda.metabaseapp.com/dashboard/87-tests?tab=142-dt-individual-test-history&test_class=WriteCachingFailureInjectionTest&test_method=test_unavoidable_data_loss
storage_test_fixture test_offset_range_size_incremental unit https://buildkite.com/redpanda/redpanda/builds/72953#01998103-1fe3-4f51-9389-52eada5e6f39 FAIL 0/1

Copy link
Contributor

@michael-redpanda michael-redpanda left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Now that we have base/format_to.h, I suggest using that instead of defining both an operator<< and a fmt::formatter

`uuid_t` is used elsewhere, `named_type` forwards `operator<<` to
the underlying type, so `topic_id` must become a seperate tpe in
order to customise printing.

Signed-off-by: Ben Pope <[email protected]>
@BenPope BenPope force-pushed the core-13481-topic-id-base64 branch from 4153519 to a4f5cdf Compare September 25, 2025 13:13
Copy link
Contributor

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants