refactor(concurrent_delete): rename module + types to reflect N concurrent single-DELETE - #573
Merged
Merged
Conversation
…rrent single-DELETE Issue #572 — follow-up to issue #568 stage 6 + #570. Issue #568 stage 6 (PR #569) replaced the batch-XML DeleteObjects default path with rclone-style N concurrent single-DELETE workers. The module file `src/batched_delete.rs` and its types still carried the "batch" name from the retired implementation, which now read as misleading — there is no batching left, just an N-worker pool firing per-key DELETEs. This commit retitles everything to reflect the surviving design: src/batched_delete.rs -> src/concurrent_delete.rs struct BatchedDeleter -> ConcurrentDeleter module batched_delete -> concurrent_delete WorkerConfig field batched_delete_config -> concurrent_delete_config MntrsFs field batched_deleter -> concurrent_deleter tracing target mntrs::batched_delete -> mntrs::concurrent_delete prefix used in log strings "batched_delete: …" -> "concurrent_delete: …" prefix used in error strings "batched_delete error: …" -> "concurrent_delete error: …" Constants in src/concurrent_delete.rs: DEFAULT_BATCH_THRESHOLD -> removed (moved to lib.rs) Constants in src/lib.rs (above UnlinkBurstState): + DEFAULT_UNLINK_BURST_WINDOW_MS: u64 = 100 + DEFAULT_UNLINK_BURST_THRESHOLD: u32 = 32 (pub(crate)) UnlinkBurstState::from_env now reads the two new consts instead of the old `batched_delete::DEFAULT_BATCH_THRESHOLD` reference. Field doc comments on UnlinkBurstState (window_ms / threshold) and the burst-detector trace inside op.unlink are rewritten to drop the stale "batched_delete::DEFAULT_BATCH_THRESHOLD" noun. src/cmd/mount.rs: - tracing::info! payload retargeted to `mntrs::concurrent_delete` - tracing target string updated - delete_mode = "concurrent-single" already aligned in PR #569 Scripts and bench (no behavioral change to default policy): bench/unlink_ab.sh - strip dead MNTRS_BATCH_FLUSH_DELAY_MS=10 export - retarget tracing target to mntrs::concurrent_delete - rewrite the batch-XML metrics block (batch_size=N / multi-key lines) to count single-key DELETE / retry / fail events from the N concurrent single-DELETE worker pool scripts/micro_bench_rm.sh - header comment now reads "S3 deleter paths" (was "issue 541 fast-flush threshold verification") - strip dead MNTRS_BATCH_FLUSH_DELAY_MS=10, MNTRS_BATCH_THRESHOLD=0, MNTRS_BATCH_SIZE=20, MNTRS_BATCH_FAST_FLUSH_THRESHOLD=8 exports - retarget tracing target to mntrs::concurrent_delete - drop the now-obsolete 3-mode comparison block ("batched+fast-flush" mode is gone) bench/run_all.sh - rewrite knob doc block from the retired four-knob shape (MNTRS_BATCH_FLUSH_DELAY_MS / MNTRS_BATCH_THRESHOLD / MNTRS_BATCH_SIZE / MNTRS_BATCH_FAST_FLUSH_THRESHOLD / MNTRS_BATCH_PROFILE) to the surviving two knobs (MNTRS_BURST_WINDOW_MS=200, MNTRS_BURST_THRESHOLD=4) - strip the dead env exports from the mount command - retarget tracing target to mntrs::concurrent_delete Out of scope (preserved as-is): - MNTRS_BATCH_MNT / MNTRS_BATCH_DAEMON_LOG shell vars: mount path and log path, not runtime knobs - delete_mode = "concurrent-single" literal: already aligned in PR #569 - bench/delete_batch_poc.sh: standalone PoC for the retired DeleteObjects XML path, not invoked from the bench flow Historical references to "batched_delete" in explanatory comments (Issue #562 stage 0 workload rationale, the "Removed 2026-08-03" mkdir-deep test record, the dead-knobs archive comment in src/concurrent_delete.rs) are preserved verbatim — they document the design history that this rename is a follow-up to. Verification: cargo fmt --all -- --check green cargo clippy --workspace --all-targets -- -D warnings green cargo build green cargo test --workspace --lib 281 passed, 0 failed bash -n bench/run_all.sh bench/unlink_ab.sh scripts/micro_bench_rm.sh green git grep -E "batched_delete|BatchedDeleter|batched_deleter" in src/, bench/, scripts/ -> 0 hits in active code; remaining matches are historical doc-comments and the standalone delete_batch_poc.sh PoC (out of scope). Closes #572. Co-Authored-By: Claude <noreply@anthropic.com>
This was referenced Aug 14, 2026
This was referenced Aug 14, 2026
dyrnq
added a commit
that referenced
this pull request
Aug 14, 2026
PR #575 advanced Cargo.lock to reqsign-aws-v4 3.2.0, reqsign-core 3.3.0, reqsign-file-read-tokio 3.0.5 (plus a new transitive reqsign-aws-core 3.1.0 extracted from aws-v4 3.2). Cargo.toml manifest pins are still at the old 3.0/3.2/3.0 values — this PR brings them in line so a future `cargo update` doesn't have to climb across minor versions. The plan #64 comment is rewritten to point at the new module name (`concurrent_delete` after #573) and the new pin-policy ("track reqsign 3.x line" rather than "mirror opendal's transitive version"), since we are now deliberately ahead of opendal-service-s3 0.58.1's transitives by 2 minor versions. Verified: - cargo check --workspace clean - cargo clippy --all-targets -- -D warnings clean - cargo build --workspace clean (was verified in PR #575) - Cargo.lock unchanged from PR #575 head Refs: #574
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #572.
Follow-up to issue #568 stage 6 (PR #569) and #570 (PR #571). Issue #568 stage 6 replaced the batch-XML DeleteObjects default path with rclone-style N concurrent single-DELETE workers. The module file
src/batched_delete.rsand its types still carried the "batch" name from the retired implementation — there is no batching left, just an N-worker pool firing per-key DELETEs. This PR retitles everything to reflect the surviving design.Identifier renames
src/batched_delete.rssrc/concurrent_delete.rsstruct BatchedDeleterConcurrentDeletermodule batched_deleteconcurrent_deleteWorkerConfig::batched_delete_configWorkerConfig::concurrent_delete_configMntrsFs::batched_deleterMntrsFs::concurrent_deletertracingtargetmntrs::batched_deletemntrs::concurrent_delete"batched_delete: ...""concurrent_delete: ...""batched_delete error: ...""concurrent_delete error: ..."Constants
DEFAULT_BATCH_THRESHOLDremoved fromsrc/concurrent_delete.rs.src/lib.rs(aboveUnlinkBurstState):DEFAULT_UNLINK_BURST_WINDOW_MS: u64 = 100pub(crate) DEFAULT_UNLINK_BURST_THRESHOLD: u32 = 32UnlinkBurstState::from_envnow reads these two consts instead of the oldbatched_delete::DEFAULT_BATCH_THRESHOLDreference.Scripts / bench
bench/unlink_ab.sh— strip deadMNTRS_BATCH_FLUSH_DELAY_MS=10export; retarget tracing target; rewrite the batch-XML metrics block (countsbatch_size=N/ multi-key lines) to count single-key DELETE / retry / fail events from the N concurrent single-DELETE worker pool.scripts/micro_bench_rm.sh— header comment updated to "S3 deleter paths"; strip deadMNTRS_BATCH_FLUSH_DELAY_MS=10,MNTRS_BATCH_THRESHOLD=0,MNTRS_BATCH_SIZE=20,MNTRS_BATCH_FAST_FLUSH_THRESHOLD=8exports; retarget tracing target; drop the now-obsolete 3-mode comparison block ("batched+fast-flush" mode is gone).bench/run_all.sh— rewrite the knob doc block from the retired four-knob shape (MNTRS_BATCH_FLUSH_DELAY_MS/MNTRS_BATCH_THRESHOLD/MNTRS_BATCH_SIZE/MNTRS_BATCH_FAST_FLUSH_THRESHOLD/MNTRS_BATCH_PROFILE) to the surviving two knobs (MNTRS_BURST_WINDOW_MS=200,MNTRS_BURST_THRESHOLD=4); strip the dead env exports from the mount command; retarget tracing target.Out of scope (preserved as-is)
MNTRS_BATCH_MNT/MNTRS_BATCH_DAEMON_LOGshell vars: mount path and log path, not runtime knobs.delete_mode = "concurrent-single"literal: already aligned in PR batched_delete: N concurrent single-DELETE workers (issue #568) #569.bench/delete_batch_poc.sh: standalone PoC for the retired DeleteObjects XML path, not invoked from the bench flow.src/concurrent_delete.rs) are preserved verbatim — they document the design history that this rename is a follow-up to.Verification
cargo fmt --all -- --check— greencargo clippy --workspace --all-targets -- -D warnings— greencargo build(debug) — greencargo test --workspace --lib— 281 passed, 0 failedbash -n bench/run_all.sh bench/unlink_ab.sh scripts/micro_bench_rm.sh— greengit grep -E "batched_delete|BatchedDeleter|batched_deleter"insrc/,bench/,scripts/— 0 hits in active code; remaining matches are historical doc-comments and the standalonedelete_batch_poc.shPoC (out of scope).