iceberg: use schema merging

[nvartolomei]

This is partial revert and resubmit of #27202 but this time the new mode is behind a feature gate which activates only after all nodes of the cluster and the cluster as a whole is upgraded.

Backports Required

• none - not a bug fix
• none - this is a backport
• none - issue does not exist in previous branches
• none - papercut/not impactful enough to backport
• v25.2.x
• v25.1.x
• v24.3.x

Release Notes

• none

[vbotbuildovich]

Retry command for Build#71805

please wait until all jobs are finished before running the slash command

/ci-repeat 1
tests/rptest/tests/compatibility/java_compression_test.py::JavaCompressionTest.test_upgrade_java_compression@{"compression_type":"zstd"}
tests/rptest/tests/compatibility/java_compression_test.py::JavaCompressionTest.test_upgrade_java_compression@{"compression_type":"gzip"}
tests/rptest/tests/compatibility/java_compression_test.py::JavaCompressionTest.test_upgrade_java_compression@{"compression_type":"lz4"}
tests/rptest/tests/compatibility/java_compression_test.py::JavaCompressionTest.test_upgrade_java_compression@{"compression_type":"snappy"}

[travisdowns]

https://redpandadata.slack.com/archives/C06MCRY75U7/p1757191660891239