Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

storage: compaction alignment test #24636

Open
wants to merge 3 commits into
base: dev
Choose a base branch
from

Conversation

WillemKauf
Copy link
Contributor

Minor enhancements to #24621

  • Sliding window compaction
  • Simulation of internal topic compaction

Backports Required

  • none - not a bug fix
  • none - this is a backport
  • none - issue does not exist in previous branches
  • none - papercut/not impactful enough to backport
  • v24.3.x
  • v24.2.x
  • v24.1.x

Release Notes

  • none

The test uses different segment layouts, runs compaction and compares
results.

Signed-off-by: Evgeny Lazin <[email protected]>
@WillemKauf WillemKauf changed the title E compaction alignment test storage: compaction alignment test Dec 20, 2024
@WillemKauf WillemKauf requested a review from Lazin December 20, 2024 22:30
Lazin
Lazin previously approved these changes Dec 20, 2024
compaction_config cfg, std::optional<model::offset> new_start_offset) {
return get_disk_log_impl().adjacent_merge_compact(cfg, new_start_offset);
}

ss::future<> disk_log_builder::apply_sliding_window_compaction(
compaction_config cfg, std::optional<model::offset> new_start_offset) {
return get_disk_log_impl().sliding_window_compact(cfg, new_start_offset);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sliding_window_compact returns future<bool> so it needs something like ignore_result at the end

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oops. That's what I get for walking away before my build finishes

Pushed

@vbotbuildovich
Copy link
Collaborator

vbotbuildovich commented Dec 21, 2024

CI test results

test results on build#60030
test_id test_kind job_url test_status passed
rptest.tests.datalake.partition_movement_test.PartitionMovementTest.test_cross_core_movements.cloud_storage_type=CloudStorageType.S3 ducktape https://buildkite.com/redpanda/redpanda/builds/60030#0193e6ed-9f27-4427-9221-edf95b330ebb FLAKY 1/6
storage_single_thread_rpunit.storage_single_thread_rpunit unit https://buildkite.com/redpanda/redpanda/builds/60030#0193e6a6-f454-437b-b938-aea99b04429d FAIL 0/2
storage_single_thread_rpunit.storage_single_thread_rpunit unit https://buildkite.com/redpanda/redpanda/builds/60030#0193e6a6-f455-405f-be53-9e777219169f FAIL 0/2
test results on build#60036
test_id test_kind job_url test_status passed
gtest_raft_rpunit.gtest_raft_rpunit unit https://buildkite.com/redpanda/redpanda/builds/60036#0193e7c7-36a5-42ac-9dbb-bb1b944f8d58 FLAKY 1/2
rptest.tests.datalake.partition_movement_test.PartitionMovementTest.test_cross_core_movements.cloud_storage_type=CloudStorageType.S3 ducktape https://buildkite.com/redpanda/redpanda/builds/60036#0193e81f-5818-4d78-87d3-26df330fce03 FLAKY 4/6
storage_single_thread_rpunit.storage_single_thread_rpunit unit https://buildkite.com/redpanda/redpanda/builds/60036#0193e7c7-36a3-44ad-b7cd-78fc94dbcee6 FAIL 0/2
storage_single_thread_rpunit.storage_single_thread_rpunit unit https://buildkite.com/redpanda/redpanda/builds/60036#0193e7c7-36a5-42ac-9dbb-bb1b944f8d58 FAIL 0/2
test results on build#60040
test_id test_kind job_url test_status passed
rptest.tests.random_node_operations_test.RandomNodeOperationsTest.test_node_operations.enable_failures=False.mixed_versions=False.with_tiered_storage=False.with_iceberg=False.cloud_storage_type=CloudStorageType.S3 ducktape https://buildkite.com/redpanda/redpanda/builds/60040#0193eacd-03cf-46fd-b8a3-3c28471a57d0 FAIL 0/1

@WillemKauf WillemKauf force-pushed the e_compaction_alignment_test branch from 80b0f72 to 6a7731e Compare December 21, 2024 05:52
@WillemKauf WillemKauf force-pushed the e_compaction_alignment_test branch from 6a7731e to d6833b7 Compare December 21, 2024 18:38
@vbotbuildovich
Copy link
Collaborator

Retry command for Build#60040

please wait until all jobs are finished before running the slash command

/ci-repeat 1
tests/rptest/tests/random_node_operations_test.py::RandomNodeOperationsTest.test_node_operations@{"cloud_storage_type":1,"enable_failures":false,"mixed_versions":false,"with_iceberg":false,"with_tiered_storage":false}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants