v24.3.2

Features

• Improve the user messages when the
  topic_partitions_reserve_shard0 cluster config is used and a user tries to create a topic with more partitions than the core-based partition limit. by @pgellert in #24461

Bug Fixes

• Ensure redpanda_cloud_storage_cloud_log_size metric consistent across all replicas. We used to update it seldomly from the leader replica only which lead to inconsistent/stale values. by @nvartolomei in #24364
• Fixed a bug in which sliding window compaction may become stuck on failing to build an index map for a single segment. by @WillemKauf in #24424
• Fixes a bug in which a segment being rolled and closed could race, leading to a triggered vassert. by @WillemKauf in #24560
• Fixes a bug in which segments which may have tombstones in them were not considered eligible for self-compaction. by @WillemKauf in #24500
• Fixes a bug that could prevent topic recovery on ABS object storage when there are objects in a bucket from multiple clusters (e.g. following a whole cluster restore). by @andrwng in #24455
• Fixes a bug where rpk wasn't parsing --help when used alongside --redpanda-id in rpk cloud <provider> byoc apply by @r-vasquez in #24396
• Fixes a bug where serializing manifests for Iceberg topics with decimal fields could cause Redpanda to crash or upload invalid manifests by @oleiman in #24467
• Fixes a crash resulting from incorrect cleanup of log readers used for iceberg translation. by @bharathv in #24576
• Fixes a race that could prevent Iceberg translation from happening following a leadership change. by @andrwng in #24562
• Fixes accounting of iceberg commit lag metric that can remain erroneously high in some cases even though the translation if fully caught up. Additionally the change ensures that only partition leaders emit lag metrics while followers emit 0 lag. by @bharathv in #24575
• If a discrete disk is used for cloud storage cache Redpanda previously rejected writes if that disk (cache disk) was full (in degraded state). This is incorrect since the cache disk isn't in the way of writes. From now on, reject writes only if the data disk is full (in degraded state). by @nvartolomei in #24486
• #24428 Schema Registry: fixes a bug in the Avro compatibility check reader_field_missing_default_value where it was too lenient for missing default values of null-able types. by @pgellert in #24430
• #24587 Redpanda will now permit topics to be created with redpanda.remote.[read|write] set to true when a license is expired or missing provided that the cluster config cloud_storage_enabled is set to false. by @michael-redpanda in #24588

Improvements

• Adds additional debug log messages in the datalake coordinator regarding files to be committed to Iceberg. by @andrwng in #24563
• Beta version of Iceberg support was incorrectly classified as "enterprise only". by @oleiman in #24443
• Leader balancer: don't treat each core as independent and balance total number of leaders on each node as well. by @ztlpn in #24440
• Show leader id in /v1/cluster/partitions response. by @ztlpn in #24585
• #24539 Disable datalake services in recovery mode by @ztlpn in #24549
• rpk topic describe now supports the --format flag to display the output in either JSON or YAML. by @r-vasquez in #24438

Full Changelog: v24.3.1...v24.3.2