From 91911760cf72702ea41fc99e0bbd4d9ed7293331 Mon Sep 17 00:00:00 2001 From: Ragnor Comerford Date: Sat, 5 Sep 2026 20:32:18 +0100 Subject: [PATCH 1/4] docs(rfc): propose bounded merge preparation and context reuse --- docs/rfcs/0053-bounded-merge-preparation.md | 474 ++++++++++++++++++ docs/rfcs/README.md | 3 +- .../0053-merge-preparation-prototype.zip | Bin 0 -> 3384 bytes .../assets/0053-merge-preparation-results.csv | 25 + 4 files changed, 501 insertions(+), 1 deletion(-) create mode 100644 docs/rfcs/0053-bounded-merge-preparation.md create mode 100644 docs/rfcs/assets/0053-merge-preparation-prototype.zip create mode 100644 docs/rfcs/assets/0053-merge-preparation-results.csv diff --git a/docs/rfcs/0053-bounded-merge-preparation.md b/docs/rfcs/0053-bounded-merge-preparation.md new file mode 100644 index 000000000..1123508f0 --- /dev/null +++ b/docs/rfcs/0053-bounded-merge-preparation.md @@ -0,0 +1,474 @@ +--- +rfc: "0053" +title: "Accepted-context reuse and bounded merge preparation" +track: maintainer +status: draft +implementation: not-started +authors: + - Codex +created: 2026-09-05 +updated: 2026-09-05 +discussion: null +supersedes: [] +superseded_by: [] +blocked_on: + - Qualify aggregate preparation accounting, width fallback, and scratch cleanup with wide and skewed tables. + - Pass the authority, deterministic-result, Blob, and existing recovery regression matrix described below. + - Reproduce the latency benefit in a release build with the bounded production scheduler and a checked-in benchmark driver. +--- + +# RFC 0053: Accepted-context reuse and bounded merge preparation + +## Summary + +Make two focused changes to branch merge: open an existing owned target from +the merge attempt's accepted context, and prepare up to four independent +non-Blob tables concurrently. Retain the existing three-way classifier, +combined graph validation, lock acquisition boundaries, recovery ownership, +serial durable table effects, and single graph publication. + +An isolated prototype measured **2.35× faster eight-table merges** and +**2.51× faster 29-table merges** with 17 ms added per physical storage request. +This RFC productionizes that bounded change. It requires no Lance upgrade, +storage migration, new transaction protocol, or public API. It does not make +merge cost constant in catalog size, history depth, or unrelated fragments. + +## Motivation + +Two repeated costs appear in the existing implementation: + +1. The existing-target preflight in + [merge.rs](../../crates/omnigraph/src/exec/merge.rs) calls the generic + [mutation opener](../../crates/omnigraph/src/db/omnigraph/table_ops.rs) + without a transaction. That route resolves the branch again and re-enters + [schema contract validation](../../crates/omnigraph/src/db/schema_state.rs) + for each such target, even though merge already captured accepted authority + and revalidates it before effects. +2. Independent table candidates are prepared sequentially. Their snapshot + reads, candidate hydration, three-way comparison, and local staging spend + substantial time waiting for storage. Preparing another table can overlap + that wait without changing which graph state is eventually published. + +The repeated-schema claim needs precise wording. The opener reads the schema +source, performs two existence checks, reads persisted IR and state, parses +the declarations, and compiles their semantic shape to validate the accepted +IR. It does not recompile every query. The number of requests per opener is +fixed; schema bytes and parsing work increase with schema size. + +Context reuse removed eight physical requests per touched existing target in +the measured fixtures: five schema requests and three branch-resolution +requests. Concurrency produced the larger latency reduction. Recording both +changes together makes the measured benefit and the pre-effect concurrency +contract reviewable; the opener refactor remains independently shippable. + +## User and operational behavior + +Existing branch merge commands, HTTP requests, actor checks, outcomes, and +graph visibility remain the contract. A successful merge still exposes all +table changes and lineage together. Genuine three-way conflicts and graph +constraint failures retain their typed outcomes and deterministic ordering. + +The expected benefit is lower latency when several tables require preparation +and storage wait dominates. A one-table merge has little work to overlap. +This proposal does not improve branch create/delete directly or establish +concurrent merges to different targets: the current exclusive coordination +remains. Existing index defects and merge-base bugs require their own fixes. + +Concurrency is an internal cap, initially four. There is no new CLI flag, +HTTP field, or environment-variable API. The prototype's +`OMNIGRAPH_DESIGN_VARIANT` switch is investigation instrumentation only. + +## Design + +### 1. Reuse the accepted attempt when opening an existing target + +Use the existing source and target `WriteTxn` captures, accepted catalog, +snapshots, and native branch bindings. Do not introduce another transaction +type, mutable cache, or independently refreshable copy of authority. + +At the current pre-arm existing-target preparation site, use a private helper +whose inputs come from that accepted attempt: + +- the target manifest entry, including stable table identity and dataset path; +- the target native branch binding, including its existing incarnation + semantics, rather than a branch name freshly resolved through the coordinator; +- the existing storage boundary used to open the physical target HEAD. + +The helper always returns an opened `SnapshotHandle`, full path, and native +binding for the existing `PreparedExistingMergeTarget` owner. It must check +that this is the existing-owned-target case. Main uses its existing native +binding representation. An inherited target requiring a first-touch fork +continues through the original deferred-fork path. + +Passing `Some(txn)` to the generic opener is insufficient: its accumulation +route may intentionally return `handle: None` for an owned/main target. The +merge preflight requires an actual handle with a physical baseline that can +be checked and carried through the effect phase. + +Keep every operation-level fence, including the schema-apply exclusion check, +the pending-recovery barriers, and `revalidate_merge_inputs`. Keep the final +target authority and manifest-version checks. The existing source-advance +rule remains: use the captured source commit, while validating its schema +and incarnation under the current protocol. + +After collecting all existing target handles, retain +`ensure_existing_effect_baseline` at the current last pre-arm boundary under +the full table gate envelope. Retain pure-insert target-incarnation +revalidation where applicable. Consume the same prepared handles in the +physical effect phase; do not reopen mutable HEADs after arming recovery. +An authority failure follows the existing typed failure/retry policy; any +new authority attempt must recapture its entire accepted view. + +### 2. Schedule existing scalar preparation with bounded concurrency + +Keep `ordered_table_keys` as the canonical order. Apply the existing unchanged +table checks and identity-compatibility checks before admitting work. +For each eligible non-Blob table, call the existing `classify_adopt` or +`stage_streaming_table_merge` path with the same base, source, target, catalog, +and native target binding used today. Preserve the existing adoption proofs, +lineage candidate selection, hydration, and fallback to ordered three-way +comparison. Continue comparing logical graph IDs; numeric Lance row IDs are +not unique across independently written branches. + +Each worker owns its temporary state and returns the table key, candidate, +and table-local conflicts. It does not mutate shared candidate/conflict +collections or graph authority. A bounded ordered stream polls at most four +table preparations. Collect results in the original table order, preserving +conflict ordering and deterministic error selection among concurrently +eligible scalar tables. Preparation results awaiting their turn count +toward the resource budget. Avoid eagerly constructing an unbounded queue +of materialized jobs. Admission-time identity/schema errors occupy their +ordered result slot too; do not surface a later table's precheck failure +ahead of an earlier table's preparation failure. + +Keep Blob descriptor preflight, shared external-payload budgets, payload +materialization, and their existing phase ordering on the original path. +Mixed Blob/scalar cases must preserve the original diagnostic ordering too; +the prototype alone is not evidence for that property. An implementation may +use ordered barriers at ineligible tables to preserve it, accepting less +overlap in mixed workloads. + +Preparation includes **local scratch writes**: `StagedTableWriter` creates +temporary Lance datasets. It is not a read-only operation. Workers may read +captured graph datasets and write exclusively owned staging directories, but +may not create/move graph-owned native refs, mutate graph tables, persist a +graph recovery intent, or publish graph content. Scratch ownership transfers +to the existing candidate owner only when the collector accepts the result. + +### 3. Bound the additional retained working set + +Four workers alone are not a byte bound. Preserve the existing per-row, +keyed-write, hydration, lineage-candidate, and operation-wide validation +limits. In particular, the existing `MergeValidationBudget` is shared across +the combined change set; it must not become four independent allowances. + +Add operation-local accounting to the preparation owners, not a new storage +buffer pool. The proposed initial parallel-preparation allowance is +`4 * KEYED_WRITE_MAX_BYTES` (currently **128 MiB**). Charge controlled retained +allocations in active and completed-but-uncollected workers: candidate IDs, +hydrated batches, copied rows, staging buffers, and worker-local result and +conflict payloads. Charge shared backing arrays conservatively; small slices +must not hide retained parent buffers. Transfer ownership/accounting exactly +once when a result joins the existing serial candidate/validation state. + +This is a cap on the speculative parallel working set, not a promise that +total process RSS fits 128 MiB. Lance/DataFusion pools, decoder allocations, +the already-required collected merge state, and local staging files have +their existing owners. Share the existing storage/session resources rather +than constructing a fresh memory pool per worker. Instrument peak accounted +bytes, in-flight batches, scratch bytes, and physical request concurrency; +four active tables can issue more than four physical requests. + +Use non-blocking reservations before controllable buffer growth. A worker +must not wait for permits while retaining the memory that prevents another +worker from finishing. On parallel-budget pressure, stop admission, settle +the current uncollected window, discard its candidates and scratch state, +and retry that window at width two, then width one. Already collected results +are retained once. This gives at most three width passes for a window and +does not retry logical conflicts, storage failures, or changed authority. + +At width one, use the existing serial limits and behavior. A row or merge +that previously fit those limits must not acquire a new refusal solely +because parallel work exhausted the additional allowance. All attempts use +the same immutable accepted inputs and stay before recovery arming; a width +fallback is a scheduling retry, not an authority retry. Expose fallback +counts in existing instrumentation. + +Decoder allocations that precede size discovery need separate qualification: +record the maximum in-flight overshoot, reduce batch/prefetch settings where +the existing API permits it, and disable concurrent preparation for shapes +whose overshoot cannot be bounded acceptably. The 128 MiB default and its +coverage are proposed policy, **not measured properties of the prototype**. +The resource gate blocks enabling width four until this accounting and +fallback preserve valid serial behavior without deadlock or unbounded queues. + +### 4. Preserve authority, validation, and publication sequencing + +The operation remains: + +1. Acquire the existing schema/lifecycle protection and capture merge inputs. +2. Hold the existing merge-exclusive coordination while preparing candidates, + with the bounded scheduling above. +3. Run the existing combined graph validation at its current boundary, + including adoption deltas, uniqueness, endpoints, cardinality, and deletion + effects. Retain the existing proven pure-insert fast-forward exemption; + add no new validation bypass. Independent table preparation does not + establish graph validity. +4. Acquire the existing conservative source/target table gate envelope in its + current global order; resolve/refuse relevant recovery and perform the + current final authority and physical-baseline checks. +5. Persist the existing complete recovery intent before graph-owned effects. +6. Apply durable table effects serially, confirm outcomes through the existing + recovery protocol, and publish every visible pointer and lineage update + through one `__manifest` publication. + +This preserves the actual distinction between preparation and the final +table-gated phase. It does not move preparation outside today's higher-level +locks or claim that the final table gates are held throughout preparation. + +### 5. Own failure and cancellation to completion + +Keep preparation futures inside the merge operation; do not detach workers. +On an error or budget fallback, stop admitting new tables, signal active +workers to stop at bounded checkpoints, await issued scratch operations, +and settle every worker before releasing its scratch owner or beginning a +replacement window. Cleanup must not race a still-running temporary writer. +Return the original typed failure after settlement; additional cleanup errors +remain observable under existing error reporting. + +Cooperative cancellation before arming must settle all preparations before +the operation returns, leaving no graph-owned effects or live preparation +task. A forced task drop or process death cannot promise asynchronous scratch +cleanup: preserve the existing staging ownership/abandoned-temp behavior and +prove that any residual files remain private and unreachable from graph +publication. Qualify the actual server cancellation path; if it cannot own +the drain, keep width one for that entry point until it can. Do not claim a +new general served-operation supervisor as part of this change. + +Cancellation after arming follows the unchanged existing recovery protocol. +This proposal does not introduce concurrent durable effects or rely on +dropping a remote write future to prove that it did not land. + +## Invariants + +The [architectural invariants](../dev/invariants.md) apply without exception. + +| Boundary | Preservation argument | +|---|---| +| Substrate ownership (1) | Lance still owns dataset reads, scratch datasets, writes, versions, and indexes; no dependency or storage format change. | +| Single publication and recovery (2, 4, 5) | Preparations cannot move graph-owned refs; durable effects remain serial under the existing intent and one graph publication. | +| Coherent accepted view and identity (3, 6, 12) | Workers and the private opener borrow the same accepted attempt; exact final authority, baseline, and incarnation checks remain. | +| Derived indexes and integrity (7, 8) | Reuse existing comparison and validation paths; index coverage does not become a precondition for success. | +| Typed semantics and trust (9, 10) | No new query/API semantics or actor bypass; all entry-point authorization stays in place. | +| Bounded resources and evidence (11, 13) | Cap workers and account additional retained data; settle retries and cancellation; qualify through existing test and instrumentation owners. | + +No deny-list exception is requested. In particular, this adds no WAL, shadow +manifest, public writable dataset handle, custom storage buffer pool, index +rebuild on the write path, or claim of distributed fencing. Existing +catalog/history/fragment scaling remains a known limitation against the +working-set objective; the speedup is not evidence that it has been resolved. + +## Compatibility and reversibility + +Storage and wire formats, recovery records, branch identities, merge outcomes, +and the Lance 11.0.0 dependency remain compatible. No migration or operator +maintenance is required. Local and object-storage execution use the same +engine path. Existing one-mutation-process limitations and Azure qualification +boundaries remain as documented in the architectural invariants. + +Either change can be reverted independently. Width one retains context reuse +while disabling parallel preparation. Test-only scoped controls may compare +widths and the old opener; production must not inherit the prototype's global +four-variant switch or repurpose `OMNIGRAPH_LOAD_CONCURRENCY`, which controls a +different write phase. + +## Alternatives + +| Alternative | Assessment | +|---|---| +| Keep current behavior | Lowest change risk, but retains demonstrated repeated reads and avoidable serial wait. | +| Ship context reuse alone | Valid first phase: about 1.13× in both multi-table fixtures. Smaller gain, independently useful. | +| Parallelize only | About 1.90×/2.01× for eight/29 tables, but leaves the repeated opener work. | +| Global schema cache | Introduces lifetime/invalidation concerns and does not by itself reuse the accepted native binding. Attempt-local reuse is sufficient. | +| Parallel durable commits or shorter locks | Changes recovery/interleaving obligations; unnecessary for the measured improvement. Requires a separate proposal. | +| Replace comparison with Lance merge-insert | Lance upsert does not supply graph ancestry, three-way conflict semantics, or multi-table validation/publication. | +| Persistent catalog/fragment trees and precomputed merge-base structures | Relevant to flat catalog/history/fragment cost, but requires substantially broader storage and authority changes. Not a dependency of this RFC. | + +## Evidence and tests + +### Measured evidence and limits + +The experiment used Omnigraph commit +`a09176d72601ce9c965de638cb4fc4179ab710c4`, Lance **11.0.0**, and upstream +commit `ab6b5bbe46009ed78746b444df8db59a8bc5d842`. The retained +[24-run CSV](assets/0053-merge-preparation-results.csv) and +[investigation patch archive](assets/0053-merge-preparation-prototype.zip) establish +which runs and code produced these observations. The archive preserves the +exact patch bytes; CSV line endings are normalized without changing cells. +The patch is evidence, not +the production implementation: it also contains server instrumentation and +a native-row-ID guard, and has no aggregate preparation accounting. + +| Touched tables | Control | Context reuse | Parallel preparation | Both | Combined gain | +|---|---:|---:|---:|---:|---:| +| 8 | 14.420 s | 12.749 s | 7.582 s | 6.124 s | 2.35× / 57.5% lower latency | +| 29 | 47.680 s | 42.208 s | 23.737 s | 18.992 s | 2.51× / 60.2% lower latency | + +These are medians of two repetitions per variant on a 121-table populated +catalog, with tiny disjoint source/target changes in each touched table. +The second repetition reversed variant order. The backend was local MinIO +behind an asynchronous proxy adding 17 ms per physical request. The proxy +preserved concurrency but buffered complete bodies; there was no bandwidth +cap, jitter, TLS, throttling, or provider tail-latency emulation in this run. + +One diagnostic executable supplied all four variants. It used the dev profile +with dependencies optimized at level two, `RUSTFLAGS=--cfg tokio_unstable`, +an Apple M5 Pro, and `LANCE_MEM_POOL_SIZE=1073741824`. Compilation completed +before timing. Lineage mode was enabled. Each timed merge used a fresh server +and the same point-read warmup; fixture setup and complete source/target +export checks were outside the timed interval. These are not production p95 +estimates, release-build results, or a cold-cache guarantee. + +At 29 tables, context reuse reduced requests from 2,647 to 2,415 and schema +requests from 157 to 12. At eight tables, the corresponding counts were +883 to 819 and 52 to 12. The remaining 12 schema requests are fixture-specific +operation-level work, not a universal allowance for every merge/retry path. +The variant named `schema` also reuses branch/table bindings; its gain cannot +be attributed to schema parsing alone. + +In the 29-table comparison, final revalidation fell from 7.196 s to 2.153 s; +serial physical publication stayed near 4.4 s and graph publication near +0.4 s. Both variants performed 87 hydration calls for 174 rows and 29 +upserts, with zero ordered full-table cursors. Per-table timing sums overlap +under concurrency and must not be added as elapsed phase time. + +Additional one-repetition controls found 1.07× for one touched table, 2.30× +for eight tables at 50 ms request delay, 1.05× for the 100,000-row fixture, +and 1.02× for the 256-history-update fixture. History and fragments grew +together in that last fixture. Neither these controls nor the multi-table +experiment proves flat cost as catalog, history, or unrelated fragments grow. + +All 24 measured merges returned HTTP 200. Full source and target exports +verified exact payloads, logical IDs, row/type counts, and edge endpoints. +The clean baseline reported 142 passing focused tests; each experimental +variant reported 143, including the additional sibling-branch row-ID guard. +Each local suite included one S3 guard that returned early without its bucket +setting; that guard subsequently passed explicitly against MinIO. The +three-way truth-table test executed 49 operation pairs; 51 unsupported +grammar cells were not successful executable cases. Expected-bug guards +remained expected-bug evidence. Full workspace CI and release qualification +were not performed for this prototype. + +Provenance SHA-256 values: + +```text +prototype.patch +42da0d84ec934b2d30b047ae1ba4c313cf7b519e35c1f2213707803bc6d435c7 +measured server executable +34e2fc158fc3c308ad49821c2d2762eb0f0bfb5314c912347cab63cdb74a0e6f +``` + +### Production acceptance gates + +Extend existing owners from [the testing map](../dev/testing.md). Do not +create a second merge semantics suite or rely on elapsed-time assertions in +ordinary CI. + +| Gate and owner | Required evidence | +|---|---| +| Opener cost: `merge_cost.rs`, existing instrumentation; pattern in `write_cost.rs` | Increasing touched existing targets adds no generic schema/branch re-resolution calls. Required capture/fence reads and physical baseline opens remain. Reproduce the fixture's eight-request-per-target reduction with physical traces. | +| Authority: `branching.rs`, `merge_fast_forward.rs`, `failpoints.rs`, existing server merge concurrency test | Target movement, source advance, same-name branch recreation, schema changes, stale physical heads, and first-touch targets retain exact outcomes. Verify the carried handle is the one checked before arming. | +| Semantics: `merge_truth_table.rs`, `merge_net_zero.rs`, `merge_projection_cache.rs` | Width one and four have identical full graph results, source preservation, conflict values/order, and typed failures across supported operation pairs, node/edge dependencies, and adoption/rewrite mixtures. | +| Blob and resource ownership: existing Blob cases in `branching.rs`, `merge_cost.rs` | Mixed scalar/Blob preparation keeps preflight limits and diagnostics. Wide values, a slow large table among small tables, dense conflicts, and pressured budgets demonstrate width fallback, bounded queues, no permit deadlock, and no new serial-valid refusal. | +| Cancellation/recovery: `failpoints.rs` and existing staging owners | Inject failures, cooperative cancellation, forced drop, and process death at preparation boundaries and fallback. Verify drain/cleanup where the operation can await, and private unreachable residual scratch where it cannot. Prove no graph-owned effects before arm. Run existing merge crash/reopen cases unchanged after arm. | +| Substrate: `lance_surface_guards.rs` | Retain pinned row-ID, historical-image, index, and native-branch guards. Label inherited known failures explicitly. Run required storage guards against a real configured disposable bucket with no early-return substitute. | +| Performance: `merge_cost.rs`, `write_cost_s3.rs`, shared instrumentation | Add a checked-in repeatable benchmark driver with exact fixture/build/store settings, separate elapsed and overlapping intervals, and export-based correctness checks. Record peak accounted bytes, allocation overshoot, active workers, requests/bytes, and fallback counts. | + +For the performance gate, use a release build and at least five paired, +alternating repetitions of baseline versus the combined production design. +With the same eight- and 29-table, 17 ms fixtures, require at least **2.0× +median speedup**. For the one-table fixture, require no more than **10% median +regression**. These are proposed promotion thresholds, not measured release +results. If they fail, context reuse may still ship independently; revise the +parallel proposal using the observed bottleneck instead of claiming 2.5×. + +Also qualify cold starts, 50 ms delay, bandwidth limits, jitter/throttling, +wide/skewed rows, indexes, catalog size, history depth, and unrelated fragments. +Vary history and fragmentation independently where the fixture permits, and +report limitations where it does not. Include a real deployed-store run +before a production latency claim. Record distributions; five repetitions +are not a reliable p95 estimate. Passing the speed gate cannot waive a +correctness/resource gate. + +### Lance sources and related tracking + +The investigation read the full upstream domains linked by +[the Lance reading protocol](../dev/lance.md), including transactions, +branches/tags/cleanup, read/write and distributed writing, row IDs, indexes, +object stores, performance, and Blob behavior. Public documentation explains +the model; the pinned code and executable guards decide the relied-on +behavior. Relevant entry points include the +[read/write guide](https://lance.org/guide/read_and_write/), +[distributed-write guide](https://lance.org/guide/distributed_write/), +[transaction specification](https://lance.org/format/table/transaction/), +[pinned merge-insert implementation](https://github.com/lance-format/lance/blob/ab6b5bbe46009ed78746b444df8db59a8bc5d842/rust/lance/src/dataset/write/merge_insert.rs), +[delta readers](https://github.com/lance-format/lance/blob/ab6b5bbe46009ed78746b444df8db59a8bc5d842/rust/lance/src/dataset/delta.rs), and +[commit conflict resolver](https://github.com/lance-format/lance/blob/ab6b5bbe46009ed78746b444df8db59a8bc5d842/rust/lance/src/io/commit/conflict_resolver.rs). + +Lance merge-insert is a target/source upsert; transaction rebase reconciles +compatible dataset commits. Neither substitutes for Omnigraph's graph +three-way merge. Distributed fragment writing is a separate, per-dataset +durable-write mechanism. This RFC changes none of those upstream mechanisms. + +The dev-graph inventory was read on 2026-09-05 from `modernrelay/dev` at +commit `01M1RQ45Q4SQVP9Q3TKK194J9G`, with a fresh schema before each query. +These are stable type/slug references, not invented web links: + +| Existing record or issue | Relationship to this decision | +|---|---| +| `Issue/iss-branch-ops-sprint`; `Learning/lrn-write-serial-backbone-measured` | Branch-operation latency umbrella and measured serial work. | +| `Gap/gap-write-path-rederivation`; `Issue/iss-write-txn`; `Issue/iss-write-cost-gate` | Accepted-context/rederivation history. Prior completion/cancellation does not close the measured merge opener path. | +| `Issue/iss-gh504-mutate-stages-tables-serially`; [Omnigraph #504](https://github.com/ModernRelay/omnigraph/issues/504) | Related mutation concurrency work; not evidence that branch merge preparation is already parallel. | +| `Issue/iss-gh384-merge-scales-target-table`; [Omnigraph #384](https://github.com/ModernRelay/omnigraph/issues/384) | Related target-size scaling, which this RFC does not eliminate. | +| [Lance #6444](https://github.com/lance-format/lance/issues/6444), [#7363](https://github.com/lance-format/lance/issues/7363), [#8853](https://github.com/lance-format/lance/issues/8853) | Upstream join/update/pruning context; not prerequisites for the two proposed changes. | +| [Lance #7263](https://github.com/lance-format/lance/issues/7263) | Native branch-merge feature request, not an available graph-merge replacement. | +| `Issue/iss-gh624-ff-merge-poisons-target`; `ExternalBlocker/blk-lance-7840`; [Omnigraph #624](https://github.com/ModernRelay/omnigraph/issues/624); [Lance #7840](https://github.com/lance-format/lance/issues/7840) | Reproduced inherited-index defect; an independent correctness issue, not fixed by this optimization. | + +These links establish related tracking, not closure of this proposal. The +investigation found no exact standalone GitHub issue for the repeated merge +opener. No separate issue has been created for this RFC; its discussion is tracked by the RFC PR. + +## Rollout + +1. Implement the private accepted-context opener and its cost/authority tests. + This can ship alone once its gates pass; mark implementation `partial`. +2. Implement the ordered scheduler, operation-local accounting, width + fallback, and cleanup in existing merge/staging owners. Qualify with scoped + test controls at widths one, two, and four while retaining serial effects. +3. Enable the internal width-four cap only after every relevant correctness, + resource, and release-benchmark gate passes. Update the measured evidence + and mark implementation `complete` only when both changes are delivered. + +If qualification requires broader storage or recovery changes, keep width +one and return that larger decision to review. Adoption of the persistent +tree redesign, new lock protocols, or parallel durable effects is not part +of this rollout. + +## Unresolved questions + +No architectural choice is intentionally delegated to implementation. The +initial 128 MiB parallel allowance, width-four cap, serial fallback, and +unchanged authority/effect protocol are the proposed decisions. Whether the +allowance covers the real retained allocations and preserves the measured +gain is an explicit evidence gate above; revise this document if evidence +requires a different policy. + +## Decision log + +- 2026-09-05: Drafted the narrow accepted-context and concurrent-preparation + proposal after the controlled four-variant experiment. Kept the measured + prototype separate from proposed resource accounting and production + qualification. The broader constant-cost redesign is outside this decision. + +- 2026-09-05: Allocated 0053 for publication after checking current main and + open RFC PRs; 0050 was already reserved by the engine-crate topology proposal. diff --git a/docs/rfcs/README.md b/docs/rfcs/README.md index 5f4c102dd..8cb575745 100644 --- a/docs/rfcs/README.md +++ b/docs/rfcs/README.md @@ -38,7 +38,7 @@ issue and implementation PR are usually enough. - Do not create `pre-merge`, `final`, `v2`, `internal`, or review-ledger copies. Revise the canonical file; preserve meaningful changes in its decision log. -The next available number is **0053**; lower gaps are historical and must +The next available number is **0054**; lower gaps are historical and must not be reused (0047 and 0048 are allocated by PR #606; 0050 by the `rfc/0050-engine-crate-topology` branch). @@ -178,3 +178,4 @@ This table is the human index for the canonical RFC corpus. | [0049](0049-control-plane-seams.md) | Control-plane seams: observe, readiness witness, bounded shutdown | maintainer | accepted | complete | | [0051](0051-json-output-via-arrow.md) | JSON output via Arrow | maintainer | draft | partial | | [0052](0052-managed-control-plane-cli.md) | Managed control-plane CLI | maintainer | accepted | complete | +| [0053](0053-bounded-merge-preparation.md) | Accepted-context reuse and bounded merge preparation | maintainer | draft | not-started | diff --git a/docs/rfcs/assets/0053-merge-preparation-prototype.zip b/docs/rfcs/assets/0053-merge-preparation-prototype.zip new file mode 100644 index 0000000000000000000000000000000000000000..b729404e9fd2b8d38231084da4889ea22a0363cf GIT binary patch literal 3384 zcmV-84af3OO9KQH000080000bUDAh|^SBHE001Wd01p5F0B~||bZ>NdaAhuVVRU0? zrCQr^<2Dj~_g7%JRSsFE=(|13xy4DGi7FY}HREJFtIH~ygj9Pk})9z zA<5*qBR6=Susd?Rw|5Lr8U*`vXaD8lVM2Cyc8(4XHb$e-^Vi+j+S>XXYk&JK8SU-w z9QgkxTl#bFC4siGnz&koN2t8dW0K~?NQfqa=J7O|F>%G*4N^bBjFu4-!aXVZsSt$- z!&DTr=(Z5oSwWLXus>L=ymzn6oQx?_R4OiJjJzQab3xdVOrUMk=ozs5is$Ur#;6{V zOoFx&5tZ(yV1HIjsz4T%h?!3|6@@P46B?^RL_G24^R>G4mU6YwHn0UWqW5lNtF2)v ziV2gTWn%HoVlc26*3CNQw2}8wa zEDXmWkLD^2^Wt`ChF}r|tx@2XtJ+Ans77SecITq-F^xf-6*+S~zuWoGjNKCA8B;H) zUxdb=25VMtXmc{5pCIGOYeaKQJB4jbu(s zIW54_0nb$t!R7^g2M@cC5-<~$5%kI^Jvvwl2&cG4 z*%*BRmr-#IBaFGw5c+dz26~hwQCtaG2pncS)@_euO!FwFStbKS3|DtAEkC_agbr^r zbV%63u;)n>#qBc$;;84zo7^k*41s#sDv1^NMX0$X@EC-;33xd4(6@e>lzR#2QXn0uqLX?gtqr-MD~qYmOK~yJz;+&*Q3VaQTbnwG%u%v~-Hl`wFVLx$-LZIEPwuA#%Y!T%I5?c_ zrZL?Mf=L=rj`q{t)oh?gP(Kmq8HgO5f&9SB*D=`7b>1(X+@ zX8bRcEm)QYdewG^o;44h?QJqX`|b48<@o){<@o%JoS*&j-;e;#K5IV{9!ShL8 zfa@@UBvp(p;J0#N;^Hdx zJOdo)Zi66yXRgWDFxPx}_FS;Hq%*)Sn)4K(4x9vW>(->rNb|_!WA8BBqHopp`pkh3 zo++MJtY`J761_+^f$hdIkX;uKzp*!Ar?}lvFiqD{|01qP@`iI26}xR)Qa%UE4N(WG#A@aFZchbXnIKd%llq6kz5itSE6?5F z(ZDCpsFXatk0>TP7qgd7nmFk!=`SL~(y~1f+kW&Z={Uh(^s24I-t`N5A}o`G9;i)e z$}6y=<`q39r!FRlhYXhO@}Y8Xdpiw<~e^_JGUl5F-w^Ps<0Y{^(SNO{fyNmgy%Y+NJ3_w{km2K}V8 z{Xa5&L!hmmcmp9aApiV23|~WnMzW}B`db$dM{}bvKY;$?roEn}o`Oknhjr)z0`}L- z_|*7dX$AVXp%KQ#5MT&}7BanHQUPUyk?z5wtlKHin3njdm@(HBBA&ixFv5czK+(0% z83I(-M$h2BL7)-c=iX*mfcTpy?&9qJOw}metc%Oc(eGjY9rFXy@C7}+S1a-ke4Vp% zpvK=@c(ZB8Y*CT|N=Y(dkR2H?Vx-L`h14WyynP;!?;w$OgxSfMJSbFMG8Vi9ZapgU z?4G1tNTjz?f)LBbp$Z>i$RZp8m}7Kz1Oh#;n5N|H4zkc(4Jh8O7?D%(B`V8E0Zi5s z_9{WXKrRY>Yhx>;m&dv?+TB0)k5?B)_%t`1?Qp2MgU-=^i%m35$SD>iBROd#_XJ`u zhyI&uT-JIAz8y^kA#sh~hcl_NQaKX|3uD`dt-Usam zLcs>3g7TDTgeMy7Bt)77Mi^u4PN_oO_)Nlgwo)9Pe*!TmA!KA=Dnyurag+o0m}_fY zVCcj}zJs7$DG(=@g;XOdX5cY3#RG|@sbDA*4+Ph61r_5@rzbxI-Fxfyqp>Hh1djCq z1T*kF2(07C++29JlpYp?Pk?Ldumot;=fiD+(XZ2XGJ#BOyU)v?d7jwJ?X3}RsmSU< zcuB*dWH2Um9eYb-d{~#`BU^npdDB@(0-j|ceMi))eOC}8e87#D7+>f@jaf!#XvoRw zf5x9K$7k(#&fpKbBNm9pcd#CUrUJcthfcKGYT~= zoL_x6ald|vagHdVcw7O9`y?s!tq7Ll7?(7Tt6634OH71|Z=~Jan=^z@G~b$4(CCe+ z>w6r$DgxAoOlk>rR0-~GYFufWaKb8g*3#SFTdF=R4|#?d8yL$f>a$QQHDxqe!)o!_ z9Eh$~HME9EK7#?`vp$}O=dY*z_-JeVpLnp=UMtI!&cJ_)3oFd!HkDBch2xj$?XiR0k~C(l)~p#MMX03N_zE#`Yw7u zIXKum+6#i?bn9-0#qsqoO{CaDFZ@d@H)$i7?Gb$A?8Q7cux-5KwQ7VZZ0gB&e<(4 zDUCSTEkHUb1Lq2|%&8~&A&ez!K8s4=Ba>l4H0U4Wsu9>gV@#rhKUNX2U=H<#zSMG?>lXcA0vz87D5#pj zKMYr#mC)waSLF@j<6y`wFc`W6ghv?j2XK7X(U1X-mHa9nVM$Tn5C%FDowf+9Fm-Z; z>Jp-7TIT~7WCyA+m*CCX-S^(D%VQJ~SgXQ#`)y&JjHRj`8$$uxyrOa)>~hS7gikO7 z+m-rujiQ9T(d(0?`<^3=mgqZkAkk0mX04@9v_`s614+BP-HrE8)a$}b{aH`l+gr97 z4o>kkO5a@{Qf${c!6$W~@WjJg{V(q3l6sy`+|8b~y|3*A*m{$IzP1Yl__ISIup!X@ zVqpSzxzpf=y9tWUsCwZ#9QX0f!(7#;ZyawaUgrU$G@H?~fE7jhPFJ_O1Afvbm-=NK z`}2jX#|Ch|^h;~JirHN?@(U>Fe6=rh^uJ)d=KFhZF-ozTc6YpvnOl;ykIa3xk@WHW zb2R=z%cA)`k~|>4F@=f}P#Wa^I?mP6!HE#`{%z~jGz-}Ci@3=dD{3NP)h*<6ay3h000O8001Rj(ubP! zxC{UQ04D$d4*&oF0000000000fB^si003}uZ**^Td2nSeaA9;~Xi!T91^@s600962 O06hQz0Obq-0000lp=NXd literal 0 HcmV?d00001 diff --git a/docs/rfcs/assets/0053-merge-preparation-results.csv b/docs/rfcs/assets/0053-merge-preparation-results.csv new file mode 100644 index 000000000..e9f3e8114 --- /dev/null +++ b/docs/rfcs/assets/0053-merge-preparation-results.csv @@ -0,0 +1,25 @@ +tag,case,delay_ms,variant,repeat,seconds,engine_seconds,requests,schema_requests,schema_bytes,schema_request_seconds_sum,response_bytes,request_bytes,max_inflight,outer_prepare_s,table_walk_sum_s,table_walk_max_s,final_revalidation_s,physical_publish_s,manifest_publish_s,ordered_cursors,hydration_calls,hydration_rows,hydration_bytes,upserts,full_state_verified +mf-history256-17-103-design-both,history256,17,both,103,4.506315124977846,4.504608709,3268,12,12354,0.23059308307711035,7807630,190034,155,1.356045,1.576193,1.576193,0.414298,0.154354,0.768484,0,3,6,3900,1,True +mf-history256-17-103-design-control,history256,17,control,103,4.6166733329882845,4.614486792,3276,17,18223,0.32761079008923844,7820327,190048,154,1.367538,1.546298,1.546298,0.561615,0.154022,0.748188,0,3,6,3900,1,True +mf-rows100000-17-103-design-both,rows100000,17,both,103,3.6363974999985658,3.631120625,280,12,12354,0.23641250288346782,4251050,147852,57,0.844895,1.652674,1.652674,0.426383,0.169493,0.307144,0,3,6,3900,1,True +mf-rows100000-17-103-design-control,rows100000,17,control,103,3.8167172499815933,3.809211167,288,17,18223,0.332609752076678,4259113,147866,57,0.820949,1.654837,1.654837,0.574106,0.169439,0.338702,0,3,6,3900,1,True +mf-tables1-17-103-design-both,tables1,17,both,103,3.1597234579967335,3.157258209,287,12,270018,0.23179762193467468,1895001,146334,63,0.839114,1.121338,1.121338,0.426482,0.132039,0.40905,0,3,6,3900,1,True +mf-tables1-17-103-design-control,tables1,17,control,103,3.3757650000043213,3.3731945420000002,295,17,404719,0.32982720935251564,2031641,146346,63,0.855633,1.130062,1.130062,0.589648,0.136538,0.412812,0,3,6,3900,1,True +mf-tables29-17-101-design-both,tables29,17,both,101,19.128344082972035,19.126079125,2415,12,270018,0.23767324996879324,21677564,2162902,63,0.867645,33.059305,1.158146,2.218299,4.486025,0.406418,0,87,174,113100,29,True +mf-tables29-17-101-design-control,tables29,17,control,101,47.4626567090163,47.461178708,2647,157,4176347,3.0793732540332712,25615735,2163207,63,0.838216,32.588371,1.149594,7.271002,4.369254,0.40052,0,87,174,113100,29,True +mf-tables29-17-101-design-parallel,tables29,17,parallel,101,23.680872333003208,23.679077959,2647,157,4176347,3.0251263777608983,25617328,2163292,63,0.858166,32.468671,1.129859,7.112904,4.324539,0.416836,0,87,174,113100,29,True +mf-tables29-17-101-design-schema,tables29,17,schema,101,42.647955875028856,42.646181375,2415,12,270018,0.22853383392794058,21680926,2163065,63,0.842358,32.687396,1.161162,2.158496,4.488483,0.426623,0,87,174,113100,29,True +mf-tables29-17-102-design-both,tables29,17,both,102,18.855217792035546,18.853374375,2415,12,270018,0.2296304169576615,21677564,2162877,63,0.870233,32.934806,1.14842,2.087216,4.41427,0.405024,0,87,174,113100,29,True +mf-tables29-17-102-design-control,tables29,17,control,102,47.89769941603299,47.895789375,2647,157,4176347,3.0267165048862807,25615654,2163199,63,0.853402,33.15239,1.179076,7.121083,4.366138,0.417353,0,87,174,113100,29,True +mf-tables29-17-102-design-parallel,tables29,17,parallel,102,23.793846999993548,23.792016875,2647,157,4176347,3.0236317519447766,25617102,2163300,63,0.843452,32.357292,1.136239,7.095678,4.474242,0.42677,0,87,174,113100,29,True +mf-tables29-17-102-design-schema,tables29,17,schema,102,41.76795929099899,41.766304417,2415,12,270018,0.22900904301786795,21680528,2163057,63,0.866501,32.357745,1.15761,2.027784,4.179852,0.393567,0,87,174,113100,29,True +mf-tables8-17-101-design-both,tables8,17,both,101,5.966756042034831,5.96545825,819,12,270018,0.22881470707943663,6840451,650518,63,0.834072,8.644619,1.090888,0.800393,1.166924,0.372305,0,24,48,31200,8,True +mf-tables8-17-101-design-control,tables8,17,control,101,14.220696875010617,14.219083167,883,52,1347626,1.0145289969514124,7927445,650582,63,0.870193,8.790255,1.134843,2.222311,1.237423,0.417701,0,24,48,31200,8,True +mf-tables8-17-101-design-parallel,tables8,17,parallel,101,7.582138541969471,7.580716,883,52,1347626,0.9995039480854757,7928287,650677,63,0.854949,9.118554,1.151903,2.197135,1.179581,0.404502,0,24,48,31200,8,True +mf-tables8-17-101-design-schema,tables8,17,schema,101,12.698209042020608,12.696734125,819,12,270018,0.23083620914258063,6841865,650551,63,0.857307,8.713635,1.12311,0.826865,1.229286,0.398702,0,24,48,31200,8,True +mf-tables8-17-102-design-both,tables8,17,both,102,6.2807058750186116,6.279442791,819,12,270018,0.23339808295713738,6840585,650511,63,0.872177,9.073251,1.147495,0.864218,1.211548,0.377913,0,24,48,31200,8,True +mf-tables8-17-102-design-control,tables8,17,control,102,14.618688249960542,14.617266459,883,52,1347626,1.0216614610981196,7927554,650590,63,0.867831,9.139606,1.155379,2.275409,1.232113,0.414123,0,24,48,31200,8,True +mf-tables8-17-102-design-parallel,tables8,17,parallel,102,7.58176758303307,7.580181166,883,52,1347626,1.0009401300922036,7928289,650673,63,0.841718,8.929906,1.121531,2.191743,1.219884,0.425394,0,24,48,31200,8,True +mf-tables8-17-102-design-schema,tables8,17,schema,102,12.79950933298096,12.79764825,819,12,270018,0.23102095705689862,6841727,650555,63,0.8335,8.836276,1.121132,0.853507,1.19554,0.409595,0,24,48,31200,8,True +mf-tables8-50-103-design-both,tables8,50,both,103,16.227784499991685,16.226013375,819,12,270018,0.6321440410683863,6840502,650512,63,2.213284,23.597487,2.95498,2.237872,3.072531,1.049183,0,24,48,31200,8,True +mf-tables8-50-103-design-control,tables8,50,control,103,37.35411174996989,37.351673083,883,52,1347626,2.7384456670843065,7927598,650587,63,2.215914,23.519453,2.960248,5.756919,3.07266,1.043436,0,24,48,31200,8,True From 644bac8e17acaaa607cf8b13228d1bdf408b6578 Mon Sep 17 00:00:00 2001 From: Ragnor Comerford Date: Sat, 5 Sep 2026 20:34:20 +0100 Subject: [PATCH 2/4] docs(rfc): link the merge preparation discussion --- docs/rfcs/0053-bounded-merge-preparation.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/rfcs/0053-bounded-merge-preparation.md b/docs/rfcs/0053-bounded-merge-preparation.md index 1123508f0..811c0f189 100644 --- a/docs/rfcs/0053-bounded-merge-preparation.md +++ b/docs/rfcs/0053-bounded-merge-preparation.md @@ -8,7 +8,7 @@ authors: - Codex created: 2026-09-05 updated: 2026-09-05 -discussion: null +discussion: https://github.com/ModernRelay/omnigraph/pull/638 supersedes: [] superseded_by: [] blocked_on: From 1b71a19ebe2433b5aeb682b512fb7f9efcaadc95 Mon Sep 17 00:00:00 2001 From: Ragnor Comerford Date: Sat, 5 Sep 2026 22:19:48 +0100 Subject: [PATCH 3/4] docs(rfc): use public issue references --- docs/rfcs/0053-bounded-merge-preparation.md | 23 +++++++++------------ 1 file changed, 10 insertions(+), 13 deletions(-) diff --git a/docs/rfcs/0053-bounded-merge-preparation.md b/docs/rfcs/0053-bounded-merge-preparation.md index 811c0f189..e6cd10dd8 100644 --- a/docs/rfcs/0053-bounded-merge-preparation.md +++ b/docs/rfcs/0053-bounded-merge-preparation.md @@ -420,23 +420,20 @@ compatible dataset commits. Neither substitutes for Omnigraph's graph three-way merge. Distributed fragment writing is a separate, per-dataset durable-write mechanism. This RFC changes none of those upstream mechanisms. -The dev-graph inventory was read on 2026-09-05 from `modernrelay/dev` at -commit `01M1RQ45Q4SQVP9Q3TKK194J9G`, with a fresh schema before each query. -These are stable type/slug references, not invented web links: - -| Existing record or issue | Relationship to this decision | +| Public issue | Relationship to this decision | |---|---| -| `Issue/iss-branch-ops-sprint`; `Learning/lrn-write-serial-backbone-measured` | Branch-operation latency umbrella and measured serial work. | -| `Gap/gap-write-path-rederivation`; `Issue/iss-write-txn`; `Issue/iss-write-cost-gate` | Accepted-context/rederivation history. Prior completion/cancellation does not close the measured merge opener path. | -| `Issue/iss-gh504-mutate-stages-tables-serially`; [Omnigraph #504](https://github.com/ModernRelay/omnigraph/issues/504) | Related mutation concurrency work; not evidence that branch merge preparation is already parallel. | -| `Issue/iss-gh384-merge-scales-target-table`; [Omnigraph #384](https://github.com/ModernRelay/omnigraph/issues/384) | Related target-size scaling, which this RFC does not eliminate. | +| [Omnigraph #310](https://github.com/ModernRelay/omnigraph/issues/310) | Earlier branch-operation request amplification work. The completed fork/delete changes do not establish that merge preparation is bounded. | +| [Omnigraph #504](https://github.com/ModernRelay/omnigraph/issues/504) | Related mutation concurrency work; not evidence that branch merge preparation is already parallel. | +| [Omnigraph #384](https://github.com/ModernRelay/omnigraph/issues/384) | Related target-size scaling, which this RFC does not eliminate. | +| [Omnigraph #641](https://github.com/ModernRelay/omnigraph/issues/641) | Remaining catalog, history, and fragment metadata costs, including fragmented historical merge bases after optimize. | +| [Omnigraph #642](https://github.com/ModernRelay/omnigraph/issues/642) | Fork/delete amplification with live sibling branches; separate from merge table preparation. | +| [Omnigraph #643](https://github.com/ModernRelay/omnigraph/issues/643) | Independent-target merge serialization. This RFC retains the existing graph-wide guards. | | [Lance #6444](https://github.com/lance-format/lance/issues/6444), [#7363](https://github.com/lance-format/lance/issues/7363), [#8853](https://github.com/lance-format/lance/issues/8853) | Upstream join/update/pruning context; not prerequisites for the two proposed changes. | | [Lance #7263](https://github.com/lance-format/lance/issues/7263) | Native branch-merge feature request, not an available graph-merge replacement. | -| `Issue/iss-gh624-ff-merge-poisons-target`; `ExternalBlocker/blk-lance-7840`; [Omnigraph #624](https://github.com/ModernRelay/omnigraph/issues/624); [Lance #7840](https://github.com/lance-format/lance/issues/7840) | Reproduced inherited-index defect; an independent correctness issue, not fixed by this optimization. | +| [Omnigraph #624](https://github.com/ModernRelay/omnigraph/issues/624); [Lance #7840](https://github.com/lance-format/lance/issues/7840) | Reproduced inherited-index defect; an independent correctness issue, not fixed by this optimization. | -These links establish related tracking, not closure of this proposal. The -investigation found no exact standalone GitHub issue for the repeated merge -opener. No separate issue has been created for this RFC; its discussion is tracked by the RFC PR. +Review this proposal in [RFC PR #638](https://github.com/ModernRelay/omnigraph/pull/638). +The issues above track related performance and correctness work separately. ## Rollout From 9ce043f21ce2d8f2f8bf90292eaa66379804ea1c Mon Sep 17 00:00:00 2001 From: aaltshuler Date: Sun, 6 Sep 2026 17:52:32 +0300 Subject: [PATCH 4/4] docs: accept canonical merge preparation RFC 0057 with measured gates --- ...n.md => 0057-bounded-merge-preparation.md} | 230 +- docs/rfcs/README.md | 6 +- .../0057-merge-preparation-diagnostics.json | 2377 +++++++++++++++++ ...7-merge-preparation-prototype-results.csv} | 0 ...p => 0057-merge-preparation-prototype.zip} | Bin 5 files changed, 2581 insertions(+), 32 deletions(-) rename docs/rfcs/{0053-bounded-merge-preparation.md => 0057-bounded-merge-preparation.md} (68%) create mode 100644 docs/rfcs/assets/0057-merge-preparation-diagnostics.json rename docs/rfcs/assets/{0053-merge-preparation-results.csv => 0057-merge-preparation-prototype-results.csv} (100%) rename docs/rfcs/assets/{0053-merge-preparation-prototype.zip => 0057-merge-preparation-prototype.zip} (100%) diff --git a/docs/rfcs/0053-bounded-merge-preparation.md b/docs/rfcs/0057-bounded-merge-preparation.md similarity index 68% rename from docs/rfcs/0053-bounded-merge-preparation.md rename to docs/rfcs/0057-bounded-merge-preparation.md index e6cd10dd8..f3b12d5c5 100644 --- a/docs/rfcs/0053-bounded-merge-preparation.md +++ b/docs/rfcs/0057-bounded-merge-preparation.md @@ -1,38 +1,46 @@ --- -rfc: "0053" +rfc: "0057" title: "Accepted-context reuse and bounded merge preparation" track: maintainer -status: draft -implementation: not-started +status: accepted +implementation: in-progress authors: - Codex created: 2026-09-05 -updated: 2026-09-05 +updated: 2026-09-06 discussion: https://github.com/ModernRelay/omnigraph/pull/638 supersedes: [] superseded_by: [] blocked_on: - - Qualify aggregate preparation accounting, width fallback, and scratch cleanup with wide and skewed tables. - - Pass the authority, deterministic-result, Blob, and existing recovery regression matrix described below. - - Reproduce the latency benefit in a release build with the bounded production scheduler and a checked-in benchmark driver. + - Native decoder allocation and serial-valid acceptance qualification + - Useful parallel failure, fallback, Blob, cancellation, and recovery coverage + - Release performance gate and deployed AWS benchmark evidence + - Request-independent operation ownership before enabling HTTP parallelism --- -# RFC 0053: Accepted-context reuse and bounded merge preparation +# RFC 0057: Accepted-context reuse and bounded merge preparation ## Summary -Make two focused changes to branch merge: open an existing owned target from -the merge attempt's accepted context, and prepare up to four independent -non-Blob tables concurrently. Retain the existing three-way classifier, +Prepare up to four eligible non-Blob tables concurrently, using the merge +attempt's accepted context. Retain the existing three-way classifier, combined graph validation, lock acquisition boundaries, recovery ownership, serial durable table effects, and single graph publication. -An isolated prototype measured **2.35× faster eight-table merges** and -**2.51× faster 29-table merges** with 17 ms added per physical storage request. -This RFC productionizes that bounded change. It requires no Lance upgrade, -storage migration, new transaction protocol, or public API. It does not make +The earlier isolated prototype motivated this work; its measurements below +are not results for the implementation in this RFC. This change requires no Lance upgrade, +storage migration, new transaction protocol, or wire API. It does not make merge cost constant in catalog size, history depth, or unrelated fragments. +The accepted-context opener is implemented in pending [PR #662](https://github.com/ModernRelay/omnigraph/pull/662), +at `86aa508580097cf46fb4c32b55f053d41f98e412`; it is not shipped by this RFC. +Pending [PR #668](https://github.com/ModernRelay/omnigraph/pull/668) adds the diagnostic scheduler +on that base. The opener's historical savings must not be counted again as +scheduler gains. That implementation extends the +existing scenario harness with small multi-table fixtures, scoped width controls +and delayed ObjectStore calls; that delay is an API-level diagnostic, not a +claim of wire-level S3 latency equivalence. + ## Motivation Two repeated costs appear in the existing implementation: @@ -74,7 +82,9 @@ This proposal does not improve branch create/delete directly or establish concurrent merges to different targets: the current exclusive coordination remains. Existing index defects and merge-base bugs require their own fixes. -Concurrency is an internal cap, initially four. There is no new CLI flag, +Concurrency is an internal cap of at most four. Production remains at one +until the qualification gates pass; HTTP has a separate hard ceiling of one +because its request owner cannot drain on disconnect. There is no new CLI flag, HTTP field, or environment-variable API. The prototype's `OMNIGRAPH_DESIGN_VARIANT` switch is investigation instrumentation only. @@ -157,6 +167,42 @@ to the existing candidate owner only when the collector accepts the result. ### 3. Bound the additional retained working set +The initial scheduler drains fixed ordered windows before refilling. A singleton +window uses serial preparation immediately. Source-state adoption is an ordered +serial barrier because it can enter transaction-history proofs and their source +normalizer; those allocation owners remain to be qualified. General non-Blob +cursor, lineage, comparison and staging routes use shared preparation accounting. Small metadata is charged conservatively +until collection, so a large merge may fall back earlier than its live bytes +alone require. Scratch telemetry counts successful logical Arrow payload, not +physical disk usage. Parallel scans set both the scanner and execution-context +concurrency to one, one fragment of read-ahead, and an 8 MiB I/O buffer per +scan. Typed stable row-ID masks keep hydration on the configured filtered-read +path instead of Lance's take shortcut. Serial scans retain their existing +tuning. The caller-owned allowance is not an RSS cap. + +Qualification admits only small, known V2.2 snapshots: at most 64 fragments, +one data file per fragment, 8 MiB aggregate encoded data, and 8,192 physical +rows. Unknown sizes/counts, overlays, any persisted index section and external +row-ID metadata use serial fallback. Inline row-ID metadata and deletion +counts are checked separately. The primary store and every referenced data or +deletion base must report I/O parallelism no greater than 64. Qualification +uses accepted metadata and existing store resolution; it does not HEAD unknown +files, rewrite native base identities, or construct replacement stores. + +Inherited branch files retain their original base IDs. Lance 11 ignores the +supplied 8 MiB scheduler for those files and constructs a native scheduler +with `32 MiB * io_parallelism` soft capacity instead. Their decoder options +still follow the configured reader path. The small-file gate narrows admitted +encoded input; compressed file size does not bound decoded memory. Native +decoder expansion and process RSS remain independent activation gates. + +The native resource envelope is separate: an ordered cursor retains its existing +150 MiB spill pool and 100 GiB scratch ceiling. Three cursors per table and four +workers can expose up to twelve such pools (1,800 MiB total capacity), not twelve +fully allocated buffers. Native I/O buffers apply soft backpressure; an oversized +page/request can exceed the requested 8 MiB. Qualification records process RSS +as well as controlled retained bytes, and does not claim a 128 MiB process bound. + Four workers alone are not a byte bound. Preserve the existing per-row, keyed-write, hydration, lineage-candidate, and operation-wide validation limits. In particular, the existing `MergeValidationBudget` is shared across @@ -287,8 +333,8 @@ different write phase. | Alternative | Assessment | |---|---| | Keep current behavior | Lowest change risk, but retains demonstrated repeated reads and avoidable serial wait. | -| Ship context reuse alone | Valid first phase: about 1.13× in both multi-table fixtures. Smaller gain, independently useful. | -| Parallelize only | About 1.90×/2.01× for eight/29 tables, but leaves the repeated opener work. | +| Ship context reuse alone | Valid first phase: about 1.13× in both historical prototype multi-table fixtures. Smaller gain, independently useful. | +| Parallelize only | Historical prototype: about 1.90×/2.01× for eight/29 tables, but leaves the repeated opener work. | | Global schema cache | Introduces lifetime/invalidation concerns and does not by itself reuse the accepted native binding. Attempt-local reuse is sufficient. | | Parallel durable commits or shorter locks | Changes recovery/interleaving obligations; unnecessary for the measured improvement. Requires a separate proposal. | | Replace comparison with Lance merge-insert | Lance upsert does not supply graph ancestry, three-way conflict semantics, or multi-table validation/publication. | @@ -296,13 +342,98 @@ different write phase. ## Evidence and tests -### Measured evidence and limits +### Current implementation evidence and disposition + +The design is accepted; its implementation remains in the pending PR stack. +The [portable diagnostic receipt](assets/0057-merge-preparation-diagnostics.json) +records the paired samples, fixture parameters, resource settings, source +identities, and executable hashes. These are local scenario-harness diagnostics, +not an authoritative benchmark archive or a production latency prediction. + +The serial source is [cbd66386](https://github.com/ModernRelay/omnigraph/commit/cbd66386015f890a885d2e22998c7d7834fce141), +based on PR #662's `86aa5085`, which already contains accepted-context reuse. +The scheduler source is [28488d7c](https://github.com/ModernRelay/omnigraph/commit/28488d7c39cb547e551158dbfb614703a042a257). +Both clean trees were built with the same bench release profile via +`cargo bench --locked -p omnigraph-engine --bench scenarios --no-run --jobs 1`. +No local compilation ran during measurement. Documentation corrections after +these commits do not change the measured implementation. + +Five alternating matched pairs per point used 121 populated node tables, +four rows per table (484 live rows per branch), and 17 ms delay per observed +ObjectStore API call. Setup, the timed merge, and complete result verification +ran in separate processes; the operation timer starts after graph open. +The scheduler used a scoped diagnostic width-four override. + +| Touched tables | Serial median | Diagnostic width-four median | Ratio of medians | +|---|---:|---:|---:| +| 1 | 0.802 s | 0.812 s | 0.987× | +| 8 | 3.619 s | 1.956 s | 1.85× | +| 29 | 12.604 s | 6.598 s | 1.91× | + +Both multi-table points **fail the 2× activation gate**. The one-table change +is approximately 1.34% slower, within the 10% regression allowance. Median +paired ratios are 1.86× and 1.90× at eight and 29 tables and reach the same +no-go conclusion. At 29 tables, candidate preparation takes 3.116 s and serial +physical publication 3.096 s (47% of total). Both variants issue 741 delayed +API calls: scheduling overlaps waits but does not remove this work. + +Two alternating pairs per history point used four populated/touched tables +and 16 live rows. H16/H64 add real current-format writes and restore logical +contents before divergence, keeping the live graph small. + +| Reachable history | Serial median | Diagnostic width-four median | Ratio of medians | +|---|---:|---:|---:| +| H0 | 2.022 s | 1.174 s | 1.72× | +| H16 | 2.651 s | 1.839 s | 1.44× | +| H64 | 4.353 s | 3.535 s | 1.23× | + +At H64, outer preparation (1.111 s) and manifest publication (1.391 s) consume +about 71% of the new operation time; candidate preparation takes 0.490 s. +Manifest reads rise from 13 at H0 to 269 at H64 in both variants. History +resolution and publication therefore remain separate optimization work. +These controls do not qualify every legacy physical format or migration path. + +All 58 timed merges passed separate setup and exact result verification, +including source preservation and single graph publication. Peak measured +whole-process RSS was 71.5 MiB and the largest persisted fixture 9.605 MiB. +Runs were sequential at nice 15, with two Lance CPU/I/O, Tokio, and Rayon +threads and `LANCE_MEM_POOL_SIZE=268435456`. Small-fixture RSS does not establish +a native decoder bound. Production-default controls used an eight-table +catalog: one touched table was 0.832 → 0.810 s; eight were 3.689 → 3.709 s +(+0.55%). Zero-delay controls used 121 populated tables: one touched table +was 24.456 → 24.171 ms; eight were 83.817 → 66.447 ms. Both controls used two +pairs per point, adding 16 merges to the 30 main-matrix and 12 history merges. +No production speedup is claimed. + +The delay and concurrency counters exclude GET body transfer, wire retries, +and unwrapped scratch I/O. This is not S3 latency emulation, a cold-cache +proof, or a p95 study. **The cloud AWS performance benchmark has not run.** +AWS-feature correctness CI is separate evidence. + +[Implementation CI](https://github.com/ModernRelay/omnigraph/actions/runs/34030407884) +at `28488d7c` passed 2,984 workspace tests (27 ignored, including a completed +genuine v0.9 upgrade case among the passes) and 371 AWS-server tests +(2 ignored). Local crate-local DST passed 78 tests with 30 ignored. +Focused cases prove useful width-four overlap; broad default tests and DST +do not qualify every parallel interleaving. + +Production therefore remains width one. Activation still requires native +pre-decode allocation/serial-valid acceptance proof, useful parallel overlap +in mixed scalar/Blob workloads while retaining the Blob serial barrier, +failure/fallback/recovery coverage, and the stated performance evidence. +Pinned Lance 11 exposes no supported pre-decode allocation cap for compressed +indivisible rows; encoded-file eligibility and post-decode accounting do not +supply that proof. HTTP additionally requires an operation owner that outlives +a request and participates in shutdown drain. A real TCP disconnect drops the +current request-owned future, so its hard width-one ceiling remains. + +### Historical prototype evidence and limits The experiment used Omnigraph commit `a09176d72601ce9c965de638cb4fc4179ab710c4`, Lance **11.0.0**, and upstream commit `ab6b5bbe46009ed78746b444df8db59a8bc5d842`. The retained -[24-run CSV](assets/0053-merge-preparation-results.csv) and -[investigation patch archive](assets/0053-merge-preparation-prototype.zip) establish +[24-run CSV](assets/0057-merge-preparation-prototype-results.csv) and +[investigation patch archive](assets/0057-merge-preparation-prototype.zip) establish which runs and code produced these observations. The archive preserves the exact patch bytes; CSV line endings are normalized without changing cells. The patch is evidence, not @@ -385,12 +516,16 @@ ordinary CI. | Performance: `merge_cost.rs`, `write_cost_s3.rs`, shared instrumentation | Add a checked-in repeatable benchmark driver with exact fixture/build/store settings, separate elapsed and overlapping intervals, and export-based correctness checks. Record peak accounted bytes, allocation overshoot, active workers, requests/bytes, and fallback counts. | For the performance gate, use a release build and at least five paired, -alternating repetitions of baseline versus the combined production design. +alternating repetitions against a frozen serial base. The current comparison +base already includes context reuse, so qualify the incremental scheduler gain +without counting that earlier benefit twice. With the same eight- and 29-table, 17 ms fixtures, require at least **2.0× median speedup**. For the one-table fixture, require no more than **10% median -regression**. These are proposed promotion thresholds, not measured release -results. If they fail, context reuse may still ship independently; revise the -parallel proposal using the observed bottleneck instead of claiming 2.5×. +regression**. These are activation thresholds. The current release diagnostics +pass the one-table regression control but fail both speedup thresholds. Context +reuse may still ship independently; keep production width one and address the +observed bottleneck instead of claiming the prototype's 2.5× or lowering the +threshold to fit the result. Also qualify cold starts, 50 ms delay, bandwidth limits, jitter/throttling, wide/skewed rows, indexes, catalog size, history depth, and unrelated fragments. @@ -437,11 +572,14 @@ The issues above track related performance and correctness work separately. ## Rollout -1. Implement the private accepted-context opener and its cost/authority tests. - This can ship alone once its gates pass; mark implementation `partial`. -2. Implement the ordered scheduler, operation-local accounting, width - fallback, and cleanup in existing merge/staging owners. Qualify with scoped - test controls at widths one, two, and four while retaining serial effects. +1. Review and land the private accepted-context opener in pending PR #662 + after its correctness dependency, PR #630. This can ship alone once its + gates pass; mark implementation `partial` when delivered. +2. Review pending PR #668's ordered scheduler, operation-local accounting, + width fallback, and scratch ownership. It remains a diagnostic implementation + with production width one. Qualify useful overlap at scoped widths two and + four, including the remaining failure and recovery cases, while retaining + serial effects. Accepting this RFC does not merge either implementation PR. 3. Enable the internal width-four cap only after every relevant correctness, resource, and release-benchmark gate passes. Update the measured evidence and mark implementation `complete` only when both changes are delivered. @@ -469,3 +607,35 @@ requires a different policy. - 2026-09-05: Allocated 0053 for publication after checking current main and open RFC PRs; 0050 was already reserved by the engine-crate topology proposal. + +- 2026-09-06: The maintainer approved implementing the bounded-preparation plan. + Reallocated this proposal to 0054 because the implementation base reserves + 0053 for retained merged ancestry; checked the registry and open RFC PRs. + Kept the existing accepted-context opener and publication protocol. The + resource, cancellation and release qualification gates remain required before + enabling the production cap. + +- 2026-09-06: A real TCP disconnect test confirmed that the pinned Axum/Hyper + HTTP/1 owner drops a pending merge service future. The HTTP entry therefore + retains a hard width-one ceiling, including under diagnostic overrides. + The same authorization and publication body serves both entry points. The + embedded path remains eligible for bounded preparation under the forced-drop + private-scratch contract above; a general served-operation supervisor is + outside this RFC. + +- 2026-09-06: Native-reader review found that inherited branch files bypass + the caller's I/O scheduler. Added conservative metadata eligibility rather + than altering Lance source identities. Small inherited fixtures can qualify + at diagnostic widths; unsupported layouts replay serially. This does not + establish a total-memory bound or waive production activation gates. + +- 2026-09-06: Consolidated the amended implementation proposal into this + canonical RFC under 0057. Main now owns 0053 and 0054 for other decisions; + 0055 is allocated and PR #670 reserves 0056. Retained merged ancestry in + PR #662 is a separate draft, reallocated to 0058. The earlier 0053/0054 + allocation entries above describe historical drafts, not current references. + Recorded the maintainer's approval to merge the design independently of + production activation. Current paired diagnostics supersede the prototype + as implementation evidence: the 2× gate is unmet, history work remains, and + AWS performance benchmarking has not run. Preserve the original prototype + archive bytes as historical evidence. diff --git a/docs/rfcs/README.md b/docs/rfcs/README.md index a89efe109..5f79b62de 100644 --- a/docs/rfcs/README.md +++ b/docs/rfcs/README.md @@ -38,9 +38,10 @@ issue and implementation PR are usually enough. - Do not create `pre-merge`, `final`, `v2`, `internal`, or review-ledger copies. Revise the canonical file; preserve meaningful changes in its decision log. -The next available number is **0056**; lower gaps are historical and must +The next available number is **0059**; lower gaps are historical and must not be reused (0047 and 0048 are allocated by PR #606; 0050 by the -`rfc/0050-engine-crate-topology` branch). +`rfc/0050-engine-crate-topology` branch; 0056 by PR #670; 0058 by +PR #662 for retained merged ancestry). ## Required frontmatter @@ -181,3 +182,4 @@ This table is the human index for the canonical RFC corpus. | [0053](0053-offline-data-token-verification.md) | Offline data-token verification | maintainer | accepted | complete | | [0054](0054-default-actor-provenance.md) | Default graph actor provenance | maintainer | accepted | complete | | [0055](0055-gq-branch-statements.md) | Branch statements in GQ | maintainer | draft | not-started | +| [0057](0057-bounded-merge-preparation.md) | Accepted-context reuse and bounded merge preparation | maintainer | accepted | in-progress | diff --git a/docs/rfcs/assets/0057-merge-preparation-diagnostics.json b/docs/rfcs/assets/0057-merge-preparation-diagnostics.json new file mode 100644 index 000000000..b6a46e8f6 --- /dev/null +++ b/docs/rfcs/assets/0057-merge-preparation-diagnostics.json @@ -0,0 +1,2377 @@ +{ + "schema_version": 1, + "title": "Bounded merge preparation: final paired local evidence", + "diagnostic_only": true, + "source_summary": { + "name": "paired-results.json", + "sha256": "eea941a1c98bc128d3da167c66c1b71f051c46d7391ceedd36ba5ce5a4e273ef", + "generated_utc": "2026-09-06T11:56:45.664170+00:00", + "complete": true, + "reported_issues": [] + }, + "counts": { + "acquisitions": 4, + "driver_invocations": 50, + "comparison_points": 10, + "pairs": 29, + "individual_measured_merges": 58, + "successful_child_phases": 174 + }, + "units": { + "time": "microseconds unless field ends in _ms", + "resource_bytes": "bytes", + "aggregate_statistic_tuple": [ + "available_samples", + "minimum", + "median", + "maximum" + ], + "speed_ratio": "A divided by B; greater than one favors B" + }, + "sources": { + "A": { + "source_commit": "cbd66386015f890a885d2e22998c7d7834fce141", + "source_tree": "b35f449cc1682068917dadef0ea59da344ad8b6f", + "source_clean_at_build_and_acquisition": true, + "binary_sha256": "8af0a00bd92c6b23fecbe1bb55eca70d3bd1bf6538c3e8a96bfc48b700cb094d", + "original_build_receipt_sha256": "e9376f907974531764dd9a246a7d0a24ce6256493750228017acfea054c4c578", + "build_exit_code": 0, + "compiler_profile": { + "opt_level": "2", + "debuginfo": 0, + "debug_assertions": false, + "overflow_checks": false, + "test": true + } + }, + "B": { + "source_commit": "28488d7c39cb547e551158dbfb614703a042a257", + "source_tree": "eeb8ae283fccb9a47fc1a7d89865c85cae66af75", + "source_clean_at_build_and_acquisition": true, + "binary_sha256": "8e63f808882b3d503192d924a79c6d910405af017e1a433b8af3a041991641af", + "original_build_receipt_sha256": "12ffd9bdb64b9512142304462f9b6c3b56bfe3c027f2380dc8410d001edf4407", + "build_exit_code": 0, + "compiler_profile": { + "opt_level": "2", + "debuginfo": 0, + "debug_assertions": false, + "overflow_checks": false, + "test": true + } + } + }, + "settings": { + "host": { + "os": "macos", + "arch": "aarch64", + "cores": 15 + }, + "build_command": [ + "cargo", + "bench", + "--locked", + "-p", + "omnigraph-engine", + "--bench", + "scenarios", + "--no-run", + "--jobs", + "1", + "--message-format=json-render-diagnostics" + ], + "build_workers": 1, + "runtime": { + "LANCE_MEM_POOL_SIZE": "268435456", + "TOKIO_WORKER_THREADS": "2", + "LANCE_CPU_THREADS": "2", + "LANCE_IO_THREADS": "2", + "RAYON_NUM_THREADS": "2", + "OMNIGRAPH_MERGE_LINEAGE": "on", + "LANG": "C", + "LC_ALL": "C", + "nice": 15, + "pause_seconds": 1.0 + }, + "scenario": "general-merge-updates", + "source_and_target_branches": 2, + "hard_process_memory_cap_requested": false, + "acquisition_sequential": true, + "individual_repetitions_per_point_per_driver": 1, + "synthetic_delay_scope": "merge only", + "verification": "all target/source rows in every populated type, unchanged source head and exact table pins, merged outcome" + }, + "acquisitions": [ + { + "id": "final-catalog121-d17", + "ledger_sha256": "9a8dd7aaeff868ea6b4481b94b4d12ea335b28d6350953a4f12297d3f52679d7", + "complete": true, + "driver_invocations": 30, + "individual_merges": 30, + "paired_points": 3, + "pairs_per_point": 5, + "current_mode": "diagnostic-width4", + "current_preparation_width_override": 4, + "history_sequence_per_driver": [0], + "driver_started_utc_min": "2026-09-06T11:51:47.102272+00:00", + "driver_completed_utc_max": "2026-09-06T11:55:00.332024+00:00" + }, + { + "id": "final-age4-d17", + "ledger_sha256": "88480dbeb89ac5206e311f0e9bc5be5123944a4e300b96f45c7f8117eb7e4ffd", + "complete": true, + "driver_invocations": 4, + "individual_merges": 12, + "paired_points": 3, + "pairs_per_point": 2, + "current_mode": "diagnostic-width4", + "current_preparation_width_override": 4, + "history_sequence_per_driver": [0,16,64], + "driver_started_utc_min": "2026-09-06T11:55:02.567560+00:00", + "driver_completed_utc_max": "2026-09-06T11:55:54.253780+00:00" + }, + { + "id": "final-catalog121-d0", + "ledger_sha256": "4a24021140b90da80ba19323737890dadd9e571d056071d4b620fec34cb9c813", + "complete": true, + "driver_invocations": 8, + "individual_merges": 8, + "paired_points": 2, + "pairs_per_point": 2, + "current_mode": "diagnostic-width4", + "current_preparation_width_override": 4, + "history_sequence_per_driver": [0], + "driver_started_utc_min": "2026-09-06T11:55:56.478281+00:00", + "driver_completed_utc_max": "2026-09-06T11:56:11.272627+00:00" + }, + { + "id": "final-default8-d17", + "ledger_sha256": "8fabd362403cacd1da5685c5cefb0215a03bdaa495bf52ef5372cfc3e674a89c", + "complete": true, + "driver_invocations": 8, + "individual_merges": 8, + "paired_points": 2, + "pairs_per_point": 2, + "current_mode": "production-default", + "current_preparation_width_override": null, + "history_sequence_per_driver": [0], + "driver_started_utc_min": "2026-09-06T11:56:13.480093+00:00", + "driver_completed_utc_max": "2026-09-06T11:56:43.565372+00:00" + } + ], + "points": [ + { + "id": "catalog4-touched4-h0-d17-diagnostic-width4", + "acquisition": "final-age4-d17", + "params": { + "cache_state": "cold", + "delta_rows": 2, + "dims": 4, + "history_commits": 0, + "io_delay_ms": 17, + "manifest_layout": "uncompacted", + "populated_tables": 4, + "retired_branches": 0, + "rows": 4, + "seed": 42, + "source_mode": "update", + "tables": 4, + "target_delta_rows": 2 + }, + "current_mode": "diagnostic-width4", + "current_preparation_width_override": 4, + "paired_count": 2, + "samples": [ + { + "pair": 0, + "order": [ + "A", + "B" + ], + "A_operation_us": 2001636, + "B_operation_us": 1177676, + "A_rss_bytes": 66846720, + "B_rss_bytes": 68861952, + "A_record_sha256": "8c4480a535a6e22965429c9091995b894088d2c76e0c769e7ed6bec185b815c6", + "B_record_sha256": "2d987e78d477bddf2164d065f13a91a75f017ae0499416a4e3c6c336e02c64f3" + }, + { + "pair": 1, + "order": [ + "B", + "A" + ], + "A_operation_us": 2042588, + "B_operation_us": 1170089, + "A_rss_bytes": 67190784, + "B_rss_bytes": 68730880, + "A_record_sha256": "59bb24af582e4eb433e85bdab805af480cf4e75eafe0892bea27bb8a7109f7f6", + "B_record_sha256": "524651919044335b86345cf1b7c7cde7da4a0dde9d1203e102c069bd2b486e59" + } + ], + "median_us": { + "A": 2022112.0, + "B": 1173882.5 + }, + "median_ms": { + "A": 2022.112, + "B": 1173.8825 + }, + "median_paired_A_over_B": 1.7226590293467747, + "ratio_of_medians_A_over_B": 1.7225846709530128, + "B_regression_percent_of_medians": -41.94770121536294, + "aggregates": { + "A": { + "operation_us": [2,2001636,2022112.0,2042588], + "phase_us": { + "outer_prepare": [2,141484,143295.0,145106], + "candidate_preparation": null, + "candidate_validation": [2,2532,2545.5,2559], + "final_revalidation": [2,62394,62573.0,62752], + "recovery_arm": [2,581,629.5,678], + "physical_publish": [2,407343,416215.5,425088], + "recovery_confirm": [2,889,1689.0,2489], + "manifest_publish": [2,103817,109504.5,115192], + "recovery_cleanup": [2,76,105.0,134], + "outer_restore_refresh": [2,0,0.0,0], + "proven_insert_history": [2,0,0.0,0], + "proven_insert_plan_scan": [2,0,0.0,0], + "keyed_stage_total": [2,92534,95382.5,98231], + "keyed_stage_max": [2,24658,25006.5,25355], + "keyed_commit_total": [2,311981,317179.5,322378], + "keyed_commit_max": [2,79315,80614.5,81914] + }, + "operation_minus_preparation_us": null, + "counters": { + "io_delay_calls": [2,116,116.0,116], + "operation_io_data_open_count": [2,20,20.0,20], + "operation_io_data_opener_reads": [2,28,28.0,28], + "operation_io_data_reads": [2,96,96.0,96], + "operation_io_data_scan_reads": [2,64,64.0,64], + "operation_io_internal_open_count": [2,4,4.0,4], + "operation_io_manifest_read_bytes": [2,34812,34812.5,34813], + "operation_io_manifest_reads": [2,13,13.0,13], + "operation_io_manifest_scan_count": [2,3,3.0,3], + "probe_completed_full_walk_classifications": [2,0,0.0,0], + "probe_completed_lineage_classifications": [2,4,4.0,4], + "probe_lineage_candidate_address_take_calls": [2,8,8.0,8], + "probe_lineage_candidate_address_take_rows": [2,16,16.0,16], + "probe_lineage_candidate_scan_rows": [2,16,16.0,16], + "probe_ordered_cursor_batch_bytes": [2,0,0.0,0], + "probe_ordered_cursor_batch_rows": [2,0,0.0,0], + "probe_ordered_cursor_scan_calls": [2,0,0.0,0], + "probe_proven_insert_history_read_calls": [2,0,0.0,0], + "probe_scan_staged_combined_calls": [2,0,0.0,0], + "probe_stage_append_calls": [2,0,0.0,0], + "probe_stage_append_rows": [2,0,0.0,0], + "probe_stage_fenced_insert_calls": [2,0,0.0,0], + "probe_stage_fenced_insert_rows": [2,0,0.0,0], + "probe_stage_known_present_update_calls": [2,4,4.0,4], + "probe_stage_known_present_update_rows": [2,8,8.0,8], + "probe_stage_merge_insert_calls": [2,0,0.0,0], + "probe_stage_merge_insert_rows": [2,0,0.0,0], + "probe_stage_vector_index_calls": [2,0,0.0,0], + "probe_strict_insert_preflight_calls": [2,0,0.0,0], + "probe_validation_scan_batches": [2,4,4.0,4], + "probe_validation_scan_projected_bytes": [2,1984,1984.0,1984] + }, + "resource": { + "operation_peak_rss_bytes": [2,66846720,67018752.0,67190784], + "fixture_bytes": [2,801696,801711.5,801727], + "fixture_files": [2,95,95.0,95] + }, + "preparation": null + }, + "B": { + "operation_us": [2,1170089,1173882.5,1177676], + "phase_us": { + "outer_prepare": [2,141081,141447.5,141814], + "candidate_preparation": [2,398565,400617.5,402670], + "candidate_validation": [2,2199,2247.5,2296], + "final_revalidation": [2,62417,62493.5,62570], + "recovery_arm": [2,475,480.0,485], + "physical_publish": [2,427566,429632.5,431699], + "recovery_confirm": [2,1962,2031.5,2101], + "manifest_publish": [2,107194,110500.5,113807], + "recovery_cleanup": [2,167,167.5,168], + "outer_restore_refresh": [2,0,0.0,0], + "proven_insert_history": [2,0,0.0,0], + "proven_insert_plan_scan": [2,0,0.0,0], + "keyed_stage_total": [2,99344,100012.0,100680], + "keyed_stage_max": [2,25335,25468.0,25601], + "keyed_commit_total": [2,324075,325499.5,326924], + "keyed_commit_max": [2,82118,82472.5,82827] + }, + "operation_minus_preparation_us": [2,767419,773265.0,779111], + "counters": { + "io_delay_calls": [2,116,116.0,116], + "io_wrapped_api_active": [2,0,0.0,0], + "io_wrapped_api_peak_active": [2,4,4.0,4], + "operation_io_data_open_count": [2,20,20.0,20], + "operation_io_data_opener_reads": [2,28,28.0,28], + "operation_io_data_reads": [2,96,96.0,96], + "operation_io_data_scan_reads": [2,64,64.0,64], + "operation_io_internal_open_count": [2,4,4.0,4], + "operation_io_manifest_read_bytes": [2,34812,34812.0,34812], + "operation_io_manifest_reads": [2,13,13.0,13], + "operation_io_manifest_scan_count": [2,3,3.0,3], + "probe_completed_full_walk_classifications": [2,0,0.0,0], + "probe_completed_lineage_classifications": [2,4,4.0,4], + "probe_lineage_candidate_address_take_calls": [2,8,8.0,8], + "probe_lineage_candidate_address_take_rows": [2,16,16.0,16], + "probe_lineage_candidate_scan_rows": [2,16,16.0,16], + "probe_ordered_cursor_batch_bytes": [2,0,0.0,0], + "probe_ordered_cursor_batch_rows": [2,0,0.0,0], + "probe_ordered_cursor_scan_calls": [2,0,0.0,0], + "probe_proven_insert_history_read_calls": [2,0,0.0,0], + "probe_scan_staged_combined_calls": [2,0,0.0,0], + "probe_stage_append_calls": [2,0,0.0,0], + "probe_stage_append_rows": [2,0,0.0,0], + "probe_stage_fenced_insert_calls": [2,0,0.0,0], + "probe_stage_fenced_insert_rows": [2,0,0.0,0], + "probe_stage_known_present_update_calls": [2,4,4.0,4], + "probe_stage_known_present_update_rows": [2,8,8.0,8], + "probe_stage_merge_insert_calls": [2,0,0.0,0], + "probe_stage_merge_insert_rows": [2,0,0.0,0], + "probe_stage_vector_index_calls": [2,0,0.0,0], + "probe_strict_insert_preflight_calls": [2,0,0.0,0], + "probe_validation_scan_batches": [2,4,4.0,4], + "probe_validation_scan_projected_bytes": [2,1984,1984.0,1984] + }, + "resource": { + "operation_peak_rss_bytes": [2,68730880,68796416.0,68861952], + "fixture_bytes": [2,801687,801688.5,801690], + "fixture_files": [2,95,95.0,95] + }, + "preparation": { + "accounted_bytes": [2,0,0.0,0], + "active": [2,0,0.0,0], + "admitted": [2,4,4.0,4], + "budget_fallbacks": [2,0,0.0,0], + "collected": [2,4,4.0,4], + "completed": [2,4,4.0,4], + "discarded": [2,0,0.0,0], + "peak_accounted_bytes": [2,285738,285738.0,285738], + "peak_active": [2,4,4.0,4], + "peak_ready": [2,4,4.0,4], + "peak_scratch_bytes": [2,2352,2352.0,2352], + "peak_scratch_owners": [2,8,8.0,8], + "peak_uncollected": [2,4,4.0,4], + "ready": [2,0,0.0,0], + "scratch_bytes": [2,0,0.0,0], + "scratch_owners": [2,0,0.0,0], + "uncollected": [2,0,0.0,0] + } + } + } + }, + { + "id": "catalog4-touched4-h16-d17-diagnostic-width4", + "acquisition": "final-age4-d17", + "params": { + "cache_state": "cold", + "delta_rows": 2, + "dims": 4, + "history_commits": 16, + "io_delay_ms": 17, + "manifest_layout": "uncompacted", + "populated_tables": 4, + "retired_branches": 0, + "rows": 4, + "seed": 42, + "source_mode": "update", + "tables": 4, + "target_delta_rows": 2 + }, + "current_mode": "diagnostic-width4", + "current_preparation_width_override": 4, + "paired_count": 2, + "samples": [ + { + "pair": 0, + "order": [ + "A", + "B" + ], + "A_operation_us": 2661332, + "B_operation_us": 1854326, + "A_rss_bytes": 67649536, + "B_rss_bytes": 69369856, + "A_record_sha256": "5bbafae4e8ebe39d498c2e4e3c9b8377e494f6c998037c58da51e1a0f2561f93", + "B_record_sha256": "91342cbe5d4fac231a2e3502794bf4212b40101f54b51e0155c39d2449c1331b" + }, + { + "pair": 1, + "order": [ + "B", + "A" + ], + "A_operation_us": 2640070, + "B_operation_us": 1823272, + "A_rss_bytes": 67780608, + "B_rss_bytes": 69500928, + "A_record_sha256": "855e259cd9f0c7aee74931636564d643ccf7a1b89c2a84c6d56b8edea368765f", + "B_record_sha256": "8f00228a07f29ab77fbc404ed0b4bd7dbc6dabafa7e07298eb10424926929a09" + } + ], + "median_us": { + "A": 2650701.0, + "B": 1838799.0 + }, + "median_ms": { + "A": 2650.701, + "B": 1838.799 + }, + "median_paired_A_over_B": 1.4415932574844255, + "ratio_of_medians_A_over_B": 1.441539287328305, + "B_regression_percent_of_medians": -30.629708895873208, + "aggregates": { + "A": { + "operation_us": [2,2640070,2650701.0,2661332], + "phase_us": { + "outer_prepare": [2,381806,382198.0,382590], + "candidate_preparation": null, + "candidate_validation": [2,2208,2416.0,2624], + "final_revalidation": [2,61329,61329.5,61330], + "recovery_arm": [2,651,656.0,661], + "physical_publish": [2,451996,453697.0,455398], + "recovery_confirm": [2,1951,2113.5,2276], + "manifest_publish": [2,426588,434351.5,442115], + "recovery_cleanup": [2,133,164.5,196], + "outer_restore_refresh": [2,0,0.0,0], + "proven_insert_history": [2,0,0.0,0], + "proven_insert_plan_scan": [2,0,0.0,0], + "keyed_stage_total": [2,122417,122761.0,123105], + "keyed_stage_max": [2,46414,47426.0,48438], + "keyed_commit_total": [2,325791,326604.5,327418], + "keyed_commit_max": [2,82747,83001.0,83255] + }, + "operation_minus_preparation_us": null, + "counters": { + "io_delay_calls": [2,185,185.0,185], + "operation_io_data_open_count": [2,20,20.0,20], + "operation_io_data_opener_reads": [2,27,27.0,27], + "operation_io_data_reads": [2,101,101.0,101], + "operation_io_data_scan_reads": [2,70,70.0,70], + "operation_io_internal_open_count": [2,4,4.0,4], + "operation_io_manifest_read_bytes": [2,255318,255319.0,255320], + "operation_io_manifest_reads": [2,77,77.0,77], + "operation_io_manifest_scan_count": [2,3,3.0,3], + "probe_completed_full_walk_classifications": [2,0,0.0,0], + "probe_completed_lineage_classifications": [2,4,4.0,4], + "probe_lineage_candidate_address_take_calls": [2,8,8.0,8], + "probe_lineage_candidate_address_take_rows": [2,15,15.0,15], + "probe_lineage_candidate_scan_rows": [2,17,17.0,17], + "probe_ordered_cursor_batch_bytes": [2,0,0.0,0], + "probe_ordered_cursor_batch_rows": [2,0,0.0,0], + "probe_ordered_cursor_scan_calls": [2,0,0.0,0], + "probe_proven_insert_history_read_calls": [2,0,0.0,0], + "probe_scan_staged_combined_calls": [2,0,0.0,0], + "probe_stage_append_calls": [2,0,0.0,0], + "probe_stage_append_rows": [2,0,0.0,0], + "probe_stage_fenced_insert_calls": [2,0,0.0,0], + "probe_stage_fenced_insert_rows": [2,0,0.0,0], + "probe_stage_known_present_update_calls": [2,4,4.0,4], + "probe_stage_known_present_update_rows": [2,8,8.0,8], + "probe_stage_merge_insert_calls": [2,0,0.0,0], + "probe_stage_merge_insert_rows": [2,0,0.0,0], + "probe_stage_vector_index_calls": [2,0,0.0,0], + "probe_strict_insert_preflight_calls": [2,0,0.0,0], + "probe_validation_scan_batches": [2,4,4.0,4], + "probe_validation_scan_projected_bytes": [2,1984,1984.0,1984] + }, + "resource": { + "operation_peak_rss_bytes": [2,67649536,67715072.0,67780608], + "fixture_bytes": [2,3054772,3054804.0,3054836], + "fixture_files": [2,208,208.0,208] + }, + "preparation": null + }, + "B": { + "operation_us": [2,1823272,1838799.0,1854326], + "phase_us": { + "outer_prepare": [2,383216,386069.5,388923], + "candidate_preparation": [2,478734,481409.5,484085], + "candidate_validation": [2,2296,2392.0,2488], + "final_revalidation": [2,60456,61371.5,62287], + "recovery_arm": [2,472,573.5,675], + "physical_publish": [2,436114,445150.5,454187], + "recovery_confirm": [2,2021,2731.5,3442], + "manifest_publish": [2,434505,435741.0,436977], + "recovery_cleanup": [2,128,163.0,198], + "outer_restore_refresh": [2,0,0.0,0], + "proven_insert_history": [2,0,0.0,0], + "proven_insert_plan_scan": [2,0,0.0,0], + "keyed_stage_total": [2,116809,118667.0,120525], + "keyed_stage_max": [2,44365,45337.5,46310], + "keyed_commit_total": [2,315099,322299.5,329500], + "keyed_commit_max": [2,81758,82437.0,83116] + }, + "operation_minus_preparation_us": [2,1344538,1357389.5,1370241], + "counters": { + "io_delay_calls": [2,185,185.0,185], + "io_wrapped_api_active": [2,0,0.0,0], + "io_wrapped_api_peak_active": [2,4,4.0,4], + "operation_io_data_open_count": [2,20,20.0,20], + "operation_io_data_opener_reads": [2,27,27.0,27], + "operation_io_data_reads": [2,101,101.0,101], + "operation_io_data_scan_reads": [2,70,70.0,70], + "operation_io_internal_open_count": [2,4,4.0,4], + "operation_io_manifest_read_bytes": [2,255316,255316.0,255316], + "operation_io_manifest_reads": [2,77,77.0,77], + "operation_io_manifest_scan_count": [2,3,3.0,3], + "probe_completed_full_walk_classifications": [2,0,0.0,0], + "probe_completed_lineage_classifications": [2,4,4.0,4], + "probe_lineage_candidate_address_take_calls": [2,8,8.0,8], + "probe_lineage_candidate_address_take_rows": [2,15,15.0,15], + "probe_lineage_candidate_scan_rows": [2,17,17.0,17], + "probe_ordered_cursor_batch_bytes": [2,0,0.0,0], + "probe_ordered_cursor_batch_rows": [2,0,0.0,0], + "probe_ordered_cursor_scan_calls": [2,0,0.0,0], + "probe_proven_insert_history_read_calls": [2,0,0.0,0], + "probe_scan_staged_combined_calls": [2,0,0.0,0], + "probe_stage_append_calls": [2,0,0.0,0], + "probe_stage_append_rows": [2,0,0.0,0], + "probe_stage_fenced_insert_calls": [2,0,0.0,0], + "probe_stage_fenced_insert_rows": [2,0,0.0,0], + "probe_stage_known_present_update_calls": [2,4,4.0,4], + "probe_stage_known_present_update_rows": [2,8,8.0,8], + "probe_stage_merge_insert_calls": [2,0,0.0,0], + "probe_stage_merge_insert_rows": [2,0,0.0,0], + "probe_stage_vector_index_calls": [2,0,0.0,0], + "probe_strict_insert_preflight_calls": [2,0,0.0,0], + "probe_validation_scan_batches": [2,4,4.0,4], + "probe_validation_scan_projected_bytes": [2,1984,1984.0,1984] + }, + "resource": { + "operation_peak_rss_bytes": [2,69369856,69435392.0,69500928], + "fixture_bytes": [2,3054773,3054775.0,3054777], + "fixture_files": [2,208,208.0,208] + }, + "preparation": { + "accounted_bytes": [2,0,0.0,0], + "active": [2,0,0.0,0], + "admitted": [2,4,4.0,4], + "budget_fallbacks": [2,0,0.0,0], + "collected": [2,4,4.0,4], + "completed": [2,4,4.0,4], + "discarded": [2,0,0.0,0], + "peak_accounted_bytes": [2,294117,294117.0,294117], + "peak_active": [2,4,4.0,4], + "peak_ready": [2,4,4.0,4], + "peak_scratch_bytes": [2,2352,2352.0,2352], + "peak_scratch_owners": [2,8,8.0,8], + "peak_uncollected": [2,4,4.0,4], + "ready": [2,0,0.0,0], + "scratch_bytes": [2,0,0.0,0], + "scratch_owners": [2,0,0.0,0], + "uncollected": [2,0,0.0,0] + } + } + } + }, + { + "id": "catalog4-touched4-h64-d17-diagnostic-width4", + "acquisition": "final-age4-d17", + "params": { + "cache_state": "cold", + "delta_rows": 2, + "dims": 4, + "history_commits": 64, + "io_delay_ms": 17, + "manifest_layout": "uncompacted", + "populated_tables": 4, + "retired_branches": 0, + "rows": 4, + "seed": 42, + "source_mode": "update", + "tables": 4, + "target_delta_rows": 2 + }, + "current_mode": "diagnostic-width4", + "current_preparation_width_override": 4, + "paired_count": 2, + "samples": [ + { + "pair": 0, + "order": [ + "A", + "B" + ], + "A_operation_us": 4363443, + "B_operation_us": 3535635, + "A_rss_bytes": 69517312, + "B_rss_bytes": 70828032, + "A_record_sha256": "d8ac41fcec922467d1bc423b97b90925deb29dd8d2493d4b3f439eb7366901c6", + "B_record_sha256": "8c5452ff3f1b9bc6ef4f61b6fe6d3cc9639df6e911f047fb420d4c3d779679f0" + }, + { + "pair": 1, + "order": [ + "B", + "A" + ], + "A_operation_us": 4342231, + "B_operation_us": 3534828, + "A_rss_bytes": 69189632, + "B_rss_bytes": 70811648, + "A_record_sha256": "87939761faa8776402c5acfc0edeac80e185bd1f1cc7e75b5fecbd4a6f186540", + "B_record_sha256": "998829aa6633f8093da752e74244df395b7a4e347ce59a82d5963d588425a06e" + } + ], + "median_us": { + "A": 4352837.0, + "B": 3535231.5 + }, + "median_ms": { + "A": 4352.837, + "B": 3535.2315 + }, + "median_paired_A_over_B": 1.231273212566258, + "ratio_of_medians_A_over_B": 1.2312735389464593, + "B_regression_percent_of_medians": -18.783278583599618, + "aggregates": { + "A": { + "operation_us": [2,4342231,4352837.0,4363443], + "phase_us": { + "outer_prepare": [2,1122304,1125285.0,1128266], + "candidate_preparation": null, + "candidate_validation": [2,1787,1880.0,1973], + "final_revalidation": [2,61871,62039.0,62207], + "recovery_arm": [2,416,475.0,534], + "physical_publish": [2,445582,449032.0,452482], + "recovery_confirm": [2,1937,2106.5,2276], + "manifest_publish": [2,1388843,1389820.0,1390797], + "recovery_cleanup": [2,202,234.0,266], + "outer_restore_refresh": [2,0,0.0,0], + "proven_insert_history": [2,0,0.0,0], + "proven_insert_plan_scan": [2,0,0.0,0], + "keyed_stage_total": [2,118120,120234.0,122348], + "keyed_stage_max": [2,45349,46035.0,46721], + "keyed_commit_total": [2,323827,325165.5,326504], + "keyed_commit_max": [2,82322,82525.0,82728] + }, + "operation_minus_preparation_us": null, + "counters": { + "io_delay_calls": [2,377,377.0,377], + "operation_io_data_open_count": [2,20,20.0,20], + "operation_io_data_opener_reads": [2,27,27.0,27], + "operation_io_data_reads": [2,101,101.0,101], + "operation_io_data_scan_reads": [2,70,70.0,70], + "operation_io_internal_open_count": [2,4,4.0,4], + "operation_io_manifest_read_bytes": [2,916873,916875.0,916877], + "operation_io_manifest_reads": [2,269,269.0,269], + "operation_io_manifest_scan_count": [2,3,3.0,3], + "probe_completed_full_walk_classifications": [2,0,0.0,0], + "probe_completed_lineage_classifications": [2,4,4.0,4], + "probe_lineage_candidate_address_take_calls": [2,8,8.0,8], + "probe_lineage_candidate_address_take_rows": [2,15,15.0,15], + "probe_lineage_candidate_scan_rows": [2,17,17.0,17], + "probe_ordered_cursor_batch_bytes": [2,0,0.0,0], + "probe_ordered_cursor_batch_rows": [2,0,0.0,0], + "probe_ordered_cursor_scan_calls": [2,0,0.0,0], + "probe_proven_insert_history_read_calls": [2,0,0.0,0], + "probe_scan_staged_combined_calls": [2,0,0.0,0], + "probe_stage_append_calls": [2,0,0.0,0], + "probe_stage_append_rows": [2,0,0.0,0], + "probe_stage_fenced_insert_calls": [2,0,0.0,0], + "probe_stage_fenced_insert_rows": [2,0,0.0,0], + "probe_stage_known_present_update_calls": [2,4,4.0,4], + "probe_stage_known_present_update_rows": [2,8,8.0,8], + "probe_stage_merge_insert_calls": [2,0,0.0,0], + "probe_stage_merge_insert_rows": [2,0,0.0,0], + "probe_stage_vector_index_calls": [2,0,0.0,0], + "probe_strict_insert_preflight_calls": [2,0,0.0,0], + "probe_validation_scan_batches": [2,4,4.0,4], + "probe_validation_scan_projected_bytes": [2,1984,1984.0,1984] + }, + "resource": { + "operation_peak_rss_bytes": [2,69189632,69353472.0,69517312], + "fixture_bytes": [2,10071158,10071208.5,10071259], + "fixture_files": [2,544,544.0,544] + }, + "preparation": null + }, + "B": { + "operation_us": [2,3534828,3535231.5,3535635], + "phase_us": { + "outer_prepare": [2,1101067,1111425.5,1121784], + "candidate_preparation": [2,487197,489740.5,492284], + "candidate_validation": [2,2885,3177.0,3469], + "final_revalidation": [2,60633,61456.5,62280], + "recovery_arm": [2,439,547.5,656], + "physical_publish": [2,447678,450879.5,454081], + "recovery_confirm": [2,2409,2431.0,2453], + "manifest_publish": [2,1385894,1390673.5,1395453], + "recovery_cleanup": [2,133,158.0,183], + "outer_restore_refresh": [2,0,0.0,0], + "proven_insert_history": [2,0,0.0,0], + "proven_insert_plan_scan": [2,0,0.0,0], + "keyed_stage_total": [2,116764,118050.5,119337], + "keyed_stage_max": [2,45603,46535.0,47467], + "keyed_commit_total": [2,326962,328769.0,330576], + "keyed_commit_max": [2,83291,83422.0,83553] + }, + "operation_minus_preparation_us": [2,3043351,3045491.0,3047631], + "counters": { + "io_delay_calls": [2,377,377.0,377], + "io_wrapped_api_active": [2,0,0.0,0], + "io_wrapped_api_peak_active": [2,4,4.0,4], + "operation_io_data_open_count": [2,20,20.0,20], + "operation_io_data_opener_reads": [2,27,27.0,27], + "operation_io_data_reads": [2,101,101.0,101], + "operation_io_data_scan_reads": [2,70,70.0,70], + "operation_io_internal_open_count": [2,4,4.0,4], + "operation_io_manifest_read_bytes": [2,916880,916881.5,916883], + "operation_io_manifest_reads": [2,269,269.0,269], + "operation_io_manifest_scan_count": [2,3,3.0,3], + "probe_completed_full_walk_classifications": [2,0,0.0,0], + "probe_completed_lineage_classifications": [2,4,4.0,4], + "probe_lineage_candidate_address_take_calls": [2,8,8.0,8], + "probe_lineage_candidate_address_take_rows": [2,15,15.0,15], + "probe_lineage_candidate_scan_rows": [2,17,17.0,17], + "probe_ordered_cursor_batch_bytes": [2,0,0.0,0], + "probe_ordered_cursor_batch_rows": [2,0,0.0,0], + "probe_ordered_cursor_scan_calls": [2,0,0.0,0], + "probe_proven_insert_history_read_calls": [2,0,0.0,0], + "probe_scan_staged_combined_calls": [2,0,0.0,0], + "probe_stage_append_calls": [2,0,0.0,0], + "probe_stage_append_rows": [2,0,0.0,0], + "probe_stage_fenced_insert_calls": [2,0,0.0,0], + "probe_stage_fenced_insert_rows": [2,0,0.0,0], + "probe_stage_known_present_update_calls": [2,4,4.0,4], + "probe_stage_known_present_update_rows": [2,8,8.0,8], + "probe_stage_merge_insert_calls": [2,0,0.0,0], + "probe_stage_merge_insert_rows": [2,0,0.0,0], + "probe_stage_vector_index_calls": [2,0,0.0,0], + "probe_strict_insert_preflight_calls": [2,0,0.0,0], + "probe_validation_scan_batches": [2,4,4.0,4], + "probe_validation_scan_projected_bytes": [2,1984,1984.0,1984] + }, + "resource": { + "operation_peak_rss_bytes": [2,70811648,70819840.0,70828032], + "fixture_bytes": [2,10071468,10071471.5,10071475], + "fixture_files": [2,544,544.0,544] + }, + "preparation": { + "accounted_bytes": [2,0,0.0,0], + "active": [2,0,0.0,0], + "admitted": [2,4,4.0,4], + "budget_fallbacks": [2,0,0.0,0], + "collected": [2,4,4.0,4], + "completed": [2,4,4.0,4], + "discarded": [2,0,0.0,0], + "peak_accounted_bytes": [2,294117,294117.0,294117], + "peak_active": [2,4,4.0,4], + "peak_ready": [2,4,4.0,4], + "peak_scratch_bytes": [2,2352,2352.0,2352], + "peak_scratch_owners": [2,8,8.0,8], + "peak_uncollected": [2,4,4.0,4], + "ready": [2,0,0.0,0], + "scratch_bytes": [2,0,0.0,0], + "scratch_owners": [2,0,0.0,0], + "uncollected": [2,0,0.0,0] + } + } + } + }, + { + "id": "catalog121-touched1-h0-d0-diagnostic-width4", + "acquisition": "final-catalog121-d0", + "params": { + "cache_state": "cold", + "delta_rows": 2, + "dims": 4, + "history_commits": 0, + "io_delay_ms": 0, + "manifest_layout": "uncompacted", + "populated_tables": 121, + "retired_branches": 0, + "rows": 4, + "seed": 42, + "source_mode": "update", + "tables": 1, + "target_delta_rows": 2 + }, + "current_mode": "diagnostic-width4", + "current_preparation_width_override": 4, + "paired_count": 2, + "samples": [ + { + "pair": 0, + "order": [ + "A", + "B" + ], + "A_operation_us": 25459, + "B_operation_us": 25192, + "A_rss_bytes": 69959680, + "B_rss_bytes": 70828032, + "A_record_sha256": "3e58ee137e4a538e92f2166881ce34a928c64a04fe64d1c32bd8a210480ccaf2", + "B_record_sha256": "7f583464854127fe8b6537f421603eb4fb108b8c8b6cfbdd03abb40968b28c72" + }, + { + "pair": 1, + "order": [ + "B", + "A" + ], + "A_operation_us": 23454, + "B_operation_us": 23150, + "A_rss_bytes": 69910528, + "B_rss_bytes": 70729728, + "A_record_sha256": "16c6e780aa8b5585c1e9b64a8d0e4cdec5825ab522768ea81b1582e691de80fc", + "B_record_sha256": "c02f30491266303b0799352aaff581b3c43d7ac39a7bbd198104670bc6b3d6fc" + } + ], + "median_us": { + "A": 24456.5, + "B": 24171.0 + }, + "median_ms": { + "A": 24.4565, + "B": 24.171 + }, + "median_paired_A_over_B": 1.0118651760955344, + "ratio_of_medians_A_over_B": 1.0118116751479045, + "B_regression_percent_of_medians": -1.1673788154478393, + "aggregates": { + "A": { + "operation_us": [2,23454,24456.5,25459], + "phase_us": { + "outer_prepare": [2,5041,5118.0,5195], + "candidate_preparation": null, + "candidate_validation": [2,307,351.5,396], + "final_revalidation": [2,2105,2122.5,2140], + "recovery_arm": [2,153,157.5,162], + "physical_publish": [2,3536,3591.5,3647], + "recovery_confirm": [2,354,362.5,371], + "manifest_publish": [2,4565,4740.5,4916], + "recovery_cleanup": [2,51,58.5,66], + "outer_restore_refresh": [2,0,0.0,0], + "proven_insert_history": [2,0,0.0,0], + "proven_insert_plan_scan": [2,0,0.0,0], + "keyed_stage_total": [2,2387,2398.0,2409], + "keyed_stage_max": [2,2387,2398.0,2409], + "keyed_commit_total": [2,908,939.5,971], + "keyed_commit_max": [2,908,939.5,971] + }, + "operation_minus_preparation_us": null, + "counters": { + "io_delay_calls": [2,44,44.0,44], + "operation_io_data_open_count": [2,5,5.0,5], + "operation_io_data_opener_reads": [2,7,7.0,7], + "operation_io_data_reads": [2,24,24.0,24], + "operation_io_data_scan_reads": [2,16,16.0,16], + "operation_io_internal_open_count": [2,4,4.0,4], + "operation_io_manifest_read_bytes": [2,45327,45327.0,45327], + "operation_io_manifest_reads": [2,16,16.0,16], + "operation_io_manifest_scan_count": [2,3,3.0,3], + "probe_completed_full_walk_classifications": [2,0,0.0,0], + "probe_completed_lineage_classifications": [2,1,1.0,1], + "probe_lineage_candidate_address_take_calls": [2,2,2.0,2], + "probe_lineage_candidate_address_take_rows": [2,4,4.0,4], + "probe_lineage_candidate_scan_rows": [2,4,4.0,4], + "probe_ordered_cursor_batch_bytes": [2,0,0.0,0], + "probe_ordered_cursor_batch_rows": [2,0,0.0,0], + "probe_ordered_cursor_scan_calls": [2,0,0.0,0], + "probe_proven_insert_history_read_calls": [2,0,0.0,0], + "probe_scan_staged_combined_calls": [2,0,0.0,0], + "probe_stage_append_calls": [2,0,0.0,0], + "probe_stage_append_rows": [2,0,0.0,0], + "probe_stage_fenced_insert_calls": [2,0,0.0,0], + "probe_stage_fenced_insert_rows": [2,0,0.0,0], + "probe_stage_known_present_update_calls": [2,1,1.0,1], + "probe_stage_known_present_update_rows": [2,2,2.0,2], + "probe_stage_merge_insert_calls": [2,0,0.0,0], + "probe_stage_merge_insert_rows": [2,0,0.0,0], + "probe_stage_vector_index_calls": [2,0,0.0,0], + "probe_strict_insert_preflight_calls": [2,0,0.0,0], + "probe_validation_scan_batches": [2,1,1.0,1], + "probe_validation_scan_projected_bytes": [2,496,496.0,496] + }, + "resource": { + "operation_peak_rss_bytes": [2,69910528,69935104.0,69959680], + "fixture_bytes": [2,1053852,1053856.5,1053861], + "fixture_files": [2,761,761.0,761] + }, + "preparation": null + }, + "B": { + "operation_us": [2,23150,24171.0,25192], + "phase_us": { + "outer_prepare": [2,4891,5024.5,5158], + "candidate_preparation": [2,6216,6842.5,7469], + "candidate_validation": [2,330,365.5,401], + "final_revalidation": [2,2032,2087.5,2143], + "recovery_arm": [2,142,146.0,150], + "physical_publish": [2,3564,3579.5,3595], + "recovery_confirm": [2,355,372.0,389], + "manifest_publish": [2,4634,4808.0,4982], + "recovery_cleanup": [2,54,57.0,60], + "outer_restore_refresh": [2,0,0.0,0], + "proven_insert_history": [2,0,0.0,0], + "proven_insert_plan_scan": [2,0,0.0,0], + "keyed_stage_total": [2,2231,2293.5,2356], + "keyed_stage_max": [2,2231,2293.5,2356], + "keyed_commit_total": [2,961,1013.5,1066], + "keyed_commit_max": [2,961,1013.5,1066] + }, + "operation_minus_preparation_us": [2,16934,17328.5,17723], + "counters": { + "io_delay_calls": [2,44,44.0,44], + "io_wrapped_api_active": [2,0,0.0,0], + "io_wrapped_api_peak_active": [2,2,2.0,2], + "operation_io_data_open_count": [2,5,5.0,5], + "operation_io_data_opener_reads": [2,7,7.0,7], + "operation_io_data_reads": [2,24,24.0,24], + "operation_io_data_scan_reads": [2,16,16.0,16], + "operation_io_internal_open_count": [2,4,4.0,4], + "operation_io_manifest_read_bytes": [2,45327,45327.0,45327], + "operation_io_manifest_reads": [2,16,16.0,16], + "operation_io_manifest_scan_count": [2,3,3.0,3], + "probe_completed_full_walk_classifications": [2,0,0.0,0], + "probe_completed_lineage_classifications": [2,1,1.0,1], + "probe_lineage_candidate_address_take_calls": [2,2,2.0,2], + "probe_lineage_candidate_address_take_rows": [2,4,4.0,4], + "probe_lineage_candidate_scan_rows": [2,4,4.0,4], + "probe_ordered_cursor_batch_bytes": [2,0,0.0,0], + "probe_ordered_cursor_batch_rows": [2,0,0.0,0], + "probe_ordered_cursor_scan_calls": [2,0,0.0,0], + "probe_proven_insert_history_read_calls": [2,0,0.0,0], + "probe_scan_staged_combined_calls": [2,0,0.0,0], + "probe_stage_append_calls": [2,0,0.0,0], + "probe_stage_append_rows": [2,0,0.0,0], + "probe_stage_fenced_insert_calls": [2,0,0.0,0], + "probe_stage_fenced_insert_rows": [2,0,0.0,0], + "probe_stage_known_present_update_calls": [2,1,1.0,1], + "probe_stage_known_present_update_rows": [2,2,2.0,2], + "probe_stage_merge_insert_calls": [2,0,0.0,0], + "probe_stage_merge_insert_rows": [2,0,0.0,0], + "probe_stage_vector_index_calls": [2,0,0.0,0], + "probe_strict_insert_preflight_calls": [2,0,0.0,0], + "probe_validation_scan_batches": [2,1,1.0,1], + "probe_validation_scan_projected_bytes": [2,496,496.0,496] + }, + "resource": { + "operation_peak_rss_bytes": [2,70729728,70778880.0,70828032], + "fixture_bytes": [2,1053852,1053856.5,1053861], + "fixture_files": [2,761,761.0,761] + }, + "preparation": { + "accounted_bytes": [2,0,0.0,0], + "active": [2,0,0.0,0], + "admitted": [2,1,1.0,1], + "budget_fallbacks": [2,0,0.0,0], + "collected": [2,1,1.0,1], + "completed": [2,1,1.0,1], + "discarded": [2,0,0.0,0], + "peak_accounted_bytes": [2,0,0.0,0], + "peak_active": [2,1,1.0,1], + "peak_ready": [2,1,1.0,1], + "peak_scratch_bytes": [2,588,588.0,588], + "peak_scratch_owners": [2,2,2.0,2], + "peak_uncollected": [2,1,1.0,1], + "ready": [2,0,0.0,0], + "scratch_bytes": [2,0,0.0,0], + "scratch_owners": [2,0,0.0,0], + "uncollected": [2,0,0.0,0] + } + } + } + }, + { + "id": "catalog121-touched8-h0-d0-diagnostic-width4", + "acquisition": "final-catalog121-d0", + "params": { + "cache_state": "cold", + "delta_rows": 2, + "dims": 4, + "history_commits": 0, + "io_delay_ms": 0, + "manifest_layout": "uncompacted", + "populated_tables": 121, + "retired_branches": 0, + "rows": 4, + "seed": 42, + "source_mode": "update", + "tables": 8, + "target_delta_rows": 2 + }, + "current_mode": "diagnostic-width4", + "current_preparation_width_override": 4, + "paired_count": 2, + "samples": [ + { + "pair": 0, + "order": [ + "A", + "B" + ], + "A_operation_us": 79880, + "B_operation_us": 62631, + "A_rss_bytes": 70631424, + "B_rss_bytes": 72187904, + "A_record_sha256": "6ba3f876fd78e66ed9f9c547113fe4e78fcc18a6db4af7d549ee8413d3626f1d", + "B_record_sha256": "e040242f78c91aac80d6865c61210e551d99ea57aa7c97392b07d33e861acfd8" + }, + { + "pair": 1, + "order": [ + "B", + "A" + ], + "A_operation_us": 87753, + "B_operation_us": 70263, + "A_rss_bytes": 71057408, + "B_rss_bytes": 72531968, + "A_record_sha256": "bd4b311337fdd9edcb9ed98db9da0cbadcccaf505b572a6691cb17c258a46543", + "B_record_sha256": "9e806ba7dda442353e6b1c9f52b426fe11ae94eec9c7c2a8b040a56fc0edce15" + } + ], + "median_us": { + "A": 83816.5, + "B": 66447.0 + }, + "median_ms": { + "A": 83.8165, + "B": 66.447 + }, + "median_paired_A_over_B": 1.2621643275735048, + "ratio_of_medians_A_over_B": 1.2614038256053697, + "B_regression_percent_of_medians": -20.723246616119738, + "aggregates": { + "A": { + "operation_us": [2,79880,83816.5,87753], + "phase_us": { + "outer_prepare": [2,4844,5042.5,5241], + "candidate_preparation": null, + "candidate_validation": [2,1545,1569.5,1594], + "final_revalidation": [2,2716,2858.5,3001], + "recovery_arm": [2,169,187.5,206], + "physical_publish": [2,20664,22481.0,24298], + "recovery_confirm": [2,1361,1368.5,1376], + "manifest_publish": [2,4592,5005.5,5419], + "recovery_cleanup": [2,49,57.0,65], + "outer_restore_refresh": [2,0,0.0,0], + "proven_insert_history": [2,0,0.0,0], + "proven_insert_plan_scan": [2,0,0.0,0], + "keyed_stage_total": [2,12190,13207.5,14225], + "keyed_stage_max": [2,2331,2334.0,2337], + "keyed_commit_total": [2,6724,7177.5,7631], + "keyed_commit_max": [2,1020,1078.5,1137] + }, + "operation_minus_preparation_us": null, + "counters": { + "io_delay_calls": [2,216,216.0,216], + "operation_io_data_open_count": [2,40,40.0,40], + "operation_io_data_opener_reads": [2,56,56.0,56], + "operation_io_data_reads": [2,192,192.0,192], + "operation_io_data_scan_reads": [2,128,128.0,128], + "operation_io_internal_open_count": [2,4,4.0,4], + "operation_io_manifest_read_bytes": [2,34830,34830.0,34830], + "operation_io_manifest_reads": [2,13,13.0,13], + "operation_io_manifest_scan_count": [2,3,3.0,3], + "probe_completed_full_walk_classifications": [2,0,0.0,0], + "probe_completed_lineage_classifications": [2,8,8.0,8], + "probe_lineage_candidate_address_take_calls": [2,16,16.0,16], + "probe_lineage_candidate_address_take_rows": [2,32,32.0,32], + "probe_lineage_candidate_scan_rows": [2,32,32.0,32], + "probe_ordered_cursor_batch_bytes": [2,0,0.0,0], + "probe_ordered_cursor_batch_rows": [2,0,0.0,0], + "probe_ordered_cursor_scan_calls": [2,0,0.0,0], + "probe_proven_insert_history_read_calls": [2,0,0.0,0], + "probe_scan_staged_combined_calls": [2,0,0.0,0], + "probe_stage_append_calls": [2,0,0.0,0], + "probe_stage_append_rows": [2,0,0.0,0], + "probe_stage_fenced_insert_calls": [2,0,0.0,0], + "probe_stage_fenced_insert_rows": [2,0,0.0,0], + "probe_stage_known_present_update_calls": [2,8,8.0,8], + "probe_stage_known_present_update_rows": [2,16,16.0,16], + "probe_stage_merge_insert_calls": [2,0,0.0,0], + "probe_stage_merge_insert_rows": [2,0,0.0,0], + "probe_stage_vector_index_calls": [2,0,0.0,0], + "probe_strict_insert_preflight_calls": [2,0,0.0,0], + "probe_validation_scan_batches": [2,8,8.0,8], + "probe_validation_scan_projected_bytes": [2,3968,3968.0,3968] + }, + "resource": { + "operation_peak_rss_bytes": [2,70631424,70844416.0,71057408], + "fixture_bytes": [2,2030407,2030413.0,2030419], + "fixture_files": [2,845,845.0,845] + }, + "preparation": null + }, + "B": { + "operation_us": [2,62631,66447.0,70263], + "phase_us": { + "outer_prepare": [2,5010,5011.5,5013], + "candidate_preparation": [2,22569,22579.5,22590], + "candidate_validation": [2,1535,1592.5,1650], + "final_revalidation": [2,2789,2849.0,2909], + "recovery_arm": [2,196,196.5,197], + "physical_publish": [2,21316,24138.0,26960], + "recovery_confirm": [2,1237,1343.0,1449], + "manifest_publish": [2,4583,5267.0,5951], + "recovery_cleanup": [2,50,56.5,63], + "outer_restore_refresh": [2,0,0.0,0], + "proven_insert_history": [2,0,0.0,0], + "proven_insert_plan_scan": [2,0,0.0,0], + "keyed_stage_total": [2,12805,14268.5,15732], + "keyed_stage_max": [2,2358,2528.5,2699], + "keyed_commit_total": [2,6758,7660.5,8563], + "keyed_commit_max": [2,977,1488.5,2000] + }, + "operation_minus_preparation_us": [2,40041,43867.5,47694], + "counters": { + "io_delay_calls": [2,216,216.0,216], + "io_wrapped_api_active": [2,0,0.0,0], + "io_wrapped_api_peak_active": [2,4,4.0,4], + "operation_io_data_open_count": [2,40,40.0,40], + "operation_io_data_opener_reads": [2,56,56.0,56], + "operation_io_data_reads": [2,192,192.0,192], + "operation_io_data_scan_reads": [2,128,128.0,128], + "operation_io_internal_open_count": [2,4,4.0,4], + "operation_io_manifest_read_bytes": [2,34828,34829.0,34830], + "operation_io_manifest_reads": [2,13,13.0,13], + "operation_io_manifest_scan_count": [2,3,3.0,3], + "probe_completed_full_walk_classifications": [2,0,0.0,0], + "probe_completed_lineage_classifications": [2,8,8.0,8], + "probe_lineage_candidate_address_take_calls": [2,16,16.0,16], + "probe_lineage_candidate_address_take_rows": [2,32,32.0,32], + "probe_lineage_candidate_scan_rows": [2,32,32.0,32], + "probe_ordered_cursor_batch_bytes": [2,0,0.0,0], + "probe_ordered_cursor_batch_rows": [2,0,0.0,0], + "probe_ordered_cursor_scan_calls": [2,0,0.0,0], + "probe_proven_insert_history_read_calls": [2,0,0.0,0], + "probe_scan_staged_combined_calls": [2,0,0.0,0], + "probe_stage_append_calls": [2,0,0.0,0], + "probe_stage_append_rows": [2,0,0.0,0], + "probe_stage_fenced_insert_calls": [2,0,0.0,0], + "probe_stage_fenced_insert_rows": [2,0,0.0,0], + "probe_stage_known_present_update_calls": [2,8,8.0,8], + "probe_stage_known_present_update_rows": [2,16,16.0,16], + "probe_stage_merge_insert_calls": [2,0,0.0,0], + "probe_stage_merge_insert_rows": [2,0,0.0,0], + "probe_stage_vector_index_calls": [2,0,0.0,0], + "probe_strict_insert_preflight_calls": [2,0,0.0,0], + "probe_validation_scan_batches": [2,8,8.0,8], + "probe_validation_scan_projected_bytes": [2,3968,3968.0,3968] + }, + "resource": { + "operation_peak_rss_bytes": [2,72187904,72359936.0,72531968], + "fixture_bytes": [2,2030428,2030428.0,2030428], + "fixture_files": [2,845,845.0,845] + }, + "preparation": { + "accounted_bytes": [2,0,0.0,0], + "active": [2,0,0.0,0], + "admitted": [2,8,8.0,8], + "budget_fallbacks": [2,0,0.0,0], + "collected": [2,8,8.0,8], + "completed": [2,8,8.0,8], + "discarded": [2,0,0.0,0], + "peak_accounted_bytes": [2,285738,285738.0,285738], + "peak_active": [2,4,4.0,4], + "peak_ready": [2,4,4.0,4], + "peak_scratch_bytes": [2,2352,2352.0,2352], + "peak_scratch_owners": [2,8,8.0,8], + "peak_uncollected": [2,4,4.0,4], + "ready": [2,0,0.0,0], + "scratch_bytes": [2,0,0.0,0], + "scratch_owners": [2,0,0.0,0], + "uncollected": [2,0,0.0,0] + } + } + } + }, + { + "id": "catalog121-touched1-h0-d17-diagnostic-width4", + "acquisition": "final-catalog121-d17", + "params": { + "cache_state": "cold", + "delta_rows": 2, + "dims": 4, + "history_commits": 0, + "io_delay_ms": 17, + "manifest_layout": "uncompacted", + "populated_tables": 121, + "retired_branches": 0, + "rows": 4, + "seed": 42, + "source_mode": "update", + "tables": 1, + "target_delta_rows": 2 + }, + "current_mode": "diagnostic-width4", + "current_preparation_width_override": 4, + "paired_count": 5, + "samples": [ + { + "pair": 0, + "order": [ + "A", + "B" + ], + "A_operation_us": 811800, + "B_operation_us": 814556, + "A_rss_bytes": 70254592, + "B_rss_bytes": 71073792, + "A_record_sha256": "2177b39f732558e45b6f9efc01a4ccf90039b724776b068abcb34af791e7ba02", + "B_record_sha256": "7efa77bcb1313a4cc739f2332e3119c523cf1f4c5a83cd9cb72eed5631052730" + }, + { + "pair": 1, + "order": [ + "B", + "A" + ], + "A_operation_us": 801587, + "B_operation_us": 805587, + "A_rss_bytes": 70057984, + "B_rss_bytes": 71335936, + "A_record_sha256": "a5fd6d5d0fe397a7a7cec2b49d98c51529e1117d4c8b213c94004e430437378c", + "B_record_sha256": "534a3030a76408c2d018d6121db2aa178e199683c664493624ad5d1832ccf868" + }, + { + "pair": 2, + "order": [ + "A", + "B" + ], + "A_operation_us": 792441, + "B_operation_us": 810284, + "A_rss_bytes": 70172672, + "B_rss_bytes": 70844416, + "A_record_sha256": "e7de536b1113484ef1ca278f31fdd0277bcc840ab0a337abaf21cb1eda50b23d", + "B_record_sha256": "761dfe694ffd7a374a96e047b15668c28477154ee78334277a992526f0990b5c" + }, + { + "pair": 3, + "order": [ + "B", + "A" + ], + "A_operation_us": 787291, + "B_operation_us": 813327, + "A_rss_bytes": 69877760, + "B_rss_bytes": 70877184, + "A_record_sha256": "0d7c9a36c462e9838c0d3aacc4b99cb70f2035ace15c664605b35a968d2f4088", + "B_record_sha256": "df77ccbb2c5e4afac6e30980705d80ad4f7b29c38c7280e36604a5c4fa5ef924" + }, + { + "pair": 4, + "order": [ + "A", + "B" + ], + "A_operation_us": 809641, + "B_operation_us": 812319, + "A_rss_bytes": 70172672, + "B_rss_bytes": 71024640, + "A_record_sha256": "a75b450f38c2fc485b0cc4c427eee5e62ff8372bc675ce8b3d75c43834be3220", + "B_record_sha256": "f2f6f95de0774289dd135986356cc3e2b0f9c535f37f1657be37010ecaba4685" + } + ], + "median_us": { + "A": 801587, + "B": 812319 + }, + "median_ms": { + "A": 801.587, + "B": 812.319 + }, + "median_paired_A_over_B": 0.9950346765774523, + "ratio_of_medians_A_over_B": 0.9867884414866573, + "B_regression_percent_of_medians": 1.3388440680799363, + "aggregates": { + "A": { + "operation_us": [5,787291,801587,811800], + "phase_us": { + "outer_prepare": [5,157551,159008,161893], + "candidate_preparation": null, + "candidate_validation": [5,521,919,1136], + "final_revalidation": [5,58748,62624,63460], + "recovery_arm": [5,232,498,616], + "physical_publish": [5,101862,105282,106852], + "recovery_confirm": [5,592,1186,1691], + "manifest_publish": [5,140432,146077,148998], + "recovery_cleanup": [5,128,142,182], + "outer_restore_refresh": [5,0,0,0], + "proven_insert_history": [5,0,0,0], + "proven_insert_plan_scan": [5,0,0,0], + "keyed_stage_total": [5,22467,24526,25761], + "keyed_stage_max": [5,22467,24526,25761], + "keyed_commit_total": [5,78637,79094,81033], + "keyed_commit_max": [5,78637,79094,81033] + }, + "operation_minus_preparation_us": null, + "counters": { + "io_delay_calls": [5,44,44,44], + "operation_io_data_open_count": [5,5,5,5], + "operation_io_data_opener_reads": [5,7,7,7], + "operation_io_data_reads": [5,24,24,24], + "operation_io_data_scan_reads": [5,16,16,16], + "operation_io_internal_open_count": [5,4,4,4], + "operation_io_manifest_read_bytes": [5,45324,45325,45327], + "operation_io_manifest_reads": [5,16,16,16], + "operation_io_manifest_scan_count": [5,3,3,3], + "probe_completed_full_walk_classifications": [5,0,0,0], + "probe_completed_lineage_classifications": [5,1,1,1], + "probe_lineage_candidate_address_take_calls": [5,2,2,2], + "probe_lineage_candidate_address_take_rows": [5,4,4,4], + "probe_lineage_candidate_scan_rows": [5,4,4,4], + "probe_ordered_cursor_batch_bytes": [5,0,0,0], + "probe_ordered_cursor_batch_rows": [5,0,0,0], + "probe_ordered_cursor_scan_calls": [5,0,0,0], + "probe_proven_insert_history_read_calls": [5,0,0,0], + "probe_scan_staged_combined_calls": [5,0,0,0], + "probe_stage_append_calls": [5,0,0,0], + "probe_stage_append_rows": [5,0,0,0], + "probe_stage_fenced_insert_calls": [5,0,0,0], + "probe_stage_fenced_insert_rows": [5,0,0,0], + "probe_stage_known_present_update_calls": [5,1,1,1], + "probe_stage_known_present_update_rows": [5,2,2,2], + "probe_stage_merge_insert_calls": [5,0,0,0], + "probe_stage_merge_insert_rows": [5,0,0,0], + "probe_stage_vector_index_calls": [5,0,0,0], + "probe_strict_insert_preflight_calls": [5,0,0,0], + "probe_validation_scan_batches": [5,1,1,1], + "probe_validation_scan_projected_bytes": [5,496,496,496] + }, + "resource": { + "operation_peak_rss_bytes": [5,69877760,70172672,70254592], + "fixture_bytes": [5,1053607,1053736,1053852], + "fixture_files": [5,761,761,761] + }, + "preparation": null + }, + "B": { + "operation_us": [5,805587,812319,814556], + "phase_us": { + "outer_prepare": [5,158747,159865,162156], + "candidate_preparation": [5,307799,310955,312324], + "candidate_validation": [5,942,1074,1220], + "final_revalidation": [5,62938,63430,64528], + "recovery_arm": [5,554,629,3398], + "physical_publish": [5,103845,104725,107442], + "recovery_confirm": [5,680,1086,1316], + "manifest_publish": [5,142237,147997,148592], + "recovery_cleanup": [5,154,164,330], + "outer_restore_refresh": [5,0,0,0], + "proven_insert_history": [5,0,0,0], + "proven_insert_plan_scan": [5,0,0,0], + "keyed_stage_total": [5,24669,25229,25914], + "keyed_stage_max": [5,24669,25229,25914], + "keyed_commit_total": [5,77638,78725,80531], + "keyed_commit_max": [5,77638,78725,80531] + }, + "operation_minus_preparation_us": [5,496264,499995,505528], + "counters": { + "io_delay_calls": [5,44,44,44], + "io_wrapped_api_active": [5,0,0,0], + "io_wrapped_api_peak_active": [5,2,2,2], + "operation_io_data_open_count": [5,5,5,5], + "operation_io_data_opener_reads": [5,7,7,7], + "operation_io_data_reads": [5,24,24,24], + "operation_io_data_scan_reads": [5,16,16,16], + "operation_io_internal_open_count": [5,4,4,4], + "operation_io_manifest_read_bytes": [5,45325,45327,45327], + "operation_io_manifest_reads": [5,16,16,16], + "operation_io_manifest_scan_count": [5,3,3,3], + "probe_completed_full_walk_classifications": [5,0,0,0], + "probe_completed_lineage_classifications": [5,1,1,1], + "probe_lineage_candidate_address_take_calls": [5,2,2,2], + "probe_lineage_candidate_address_take_rows": [5,4,4,4], + "probe_lineage_candidate_scan_rows": [5,4,4,4], + "probe_ordered_cursor_batch_bytes": [5,0,0,0], + "probe_ordered_cursor_batch_rows": [5,0,0,0], + "probe_ordered_cursor_scan_calls": [5,0,0,0], + "probe_proven_insert_history_read_calls": [5,0,0,0], + "probe_scan_staged_combined_calls": [5,0,0,0], + "probe_stage_append_calls": [5,0,0,0], + "probe_stage_append_rows": [5,0,0,0], + "probe_stage_fenced_insert_calls": [5,0,0,0], + "probe_stage_fenced_insert_rows": [5,0,0,0], + "probe_stage_known_present_update_calls": [5,1,1,1], + "probe_stage_known_present_update_rows": [5,2,2,2], + "probe_stage_merge_insert_calls": [5,0,0,0], + "probe_stage_merge_insert_rows": [5,0,0,0], + "probe_stage_vector_index_calls": [5,0,0,0], + "probe_strict_insert_preflight_calls": [5,0,0,0], + "probe_validation_scan_batches": [5,1,1,1], + "probe_validation_scan_projected_bytes": [5,496,496,496] + }, + "resource": { + "operation_peak_rss_bytes": [5,70844416,71024640,71335936], + "fixture_bytes": [5,1053808,1053849,1053858], + "fixture_files": [5,761,761,761] + }, + "preparation": { + "accounted_bytes": [5,0,0,0], + "active": [5,0,0,0], + "admitted": [5,1,1,1], + "budget_fallbacks": [5,0,0,0], + "collected": [5,1,1,1], + "completed": [5,1,1,1], + "discarded": [5,0,0,0], + "peak_accounted_bytes": [5,0,0,0], + "peak_active": [5,1,1,1], + "peak_ready": [5,1,1,1], + "peak_scratch_bytes": [5,588,588,588], + "peak_scratch_owners": [5,2,2,2], + "peak_uncollected": [5,1,1,1], + "ready": [5,0,0,0], + "scratch_bytes": [5,0,0,0], + "scratch_owners": [5,0,0,0], + "uncollected": [5,0,0,0] + } + } + } + }, + { + "id": "catalog121-touched8-h0-d17-diagnostic-width4", + "acquisition": "final-catalog121-d17", + "params": { + "cache_state": "cold", + "delta_rows": 2, + "dims": 4, + "history_commits": 0, + "io_delay_ms": 17, + "manifest_layout": "uncompacted", + "populated_tables": 121, + "retired_branches": 0, + "rows": 4, + "seed": 42, + "source_mode": "update", + "tables": 8, + "target_delta_rows": 2 + }, + "current_mode": "diagnostic-width4", + "current_preparation_width_override": 4, + "paired_count": 5, + "samples": [ + { + "pair": 0, + "order": [ + "A", + "B" + ], + "A_operation_us": 3621510, + "B_operation_us": 1969037, + "A_rss_bytes": 71106560, + "B_rss_bytes": 72679424, + "A_record_sha256": "d1fbee98d1590b628169fa938a16465f73c59f0e622d0b53cfda89f763eefad4", + "B_record_sha256": "2326e587c6d2f93897deb1cd9b6e835d7d8d2d13510515d733582fd3e447cf65" + }, + { + "pair": 1, + "order": [ + "B", + "A" + ], + "A_operation_us": 3619272, + "B_operation_us": 1963129, + "A_rss_bytes": 71041024, + "B_rss_bytes": 72695808, + "A_record_sha256": "962997dc64f3b331c60ed7ec45367877450f833ef6df6d0def4a7f948220091c", + "B_record_sha256": "89233e23928c84d834e0a092385065c0db35b786ca5a1687ae3319d1531ca380" + }, + { + "pair": 2, + "order": [ + "A", + "B" + ], + "A_operation_us": 3615444, + "B_operation_us": 1909966, + "A_rss_bytes": 70778880, + "B_rss_bytes": 72368128, + "A_record_sha256": "8db9fe5741539cf9d59e3a4b196c334871653b93fa8276ce5f5afcb5d0a719c5", + "B_record_sha256": "b390d9bd85061d74b56aa90e84f36085c111576023e1ba1ba58abeefe65dabac" + }, + { + "pair": 3, + "order": [ + "B", + "A" + ], + "A_operation_us": 3641681, + "B_operation_us": 1956090, + "A_rss_bytes": 71139328, + "B_rss_bytes": 72613888, + "A_record_sha256": "7d89bdee52c264c65e2945e308256910c20fcf8be2720034f6014e0b2257dcf8", + "B_record_sha256": "189d3060f67157540f4e0ceaf97318a2e47054c3409c7713eeebf4dc51398cfc" + }, + { + "pair": 4, + "order": [ + "A", + "B" + ], + "A_operation_us": 3617616, + "B_operation_us": 1934202, + "A_rss_bytes": 70696960, + "B_rss_bytes": 72728576, + "A_record_sha256": "2441bb73cbfe42b7af04146e01260b7528cc689cddf323782c3c33dcdce83569", + "B_record_sha256": "18ce72419449d762f656c3b97b54fb02498991fa9fb3ea35ccd5419b40439820" + } + ], + "median_us": { + "A": 3619272, + "B": 1956090 + }, + "median_ms": { + "A": 3619.272, + "B": 1956.09 + }, + "median_paired_A_over_B": 1.8617144405421018, + "ratio_of_medians_A_over_B": 1.8502584236921613, + "B_regression_percent_of_medians": -45.95349562011366, + "aggregates": { + "A": { + "operation_us": [5,3615444,3619272,3641681], + "phase_us": { + "outer_prepare": [5,141758,142473,142765], + "candidate_preparation": null, + "candidate_validation": [5,3655,4644,5534], + "final_revalidation": [5,62940,65093,66268], + "recovery_arm": [5,580,612,1185], + "physical_publish": [5,820650,834208,840043], + "recovery_confirm": [5,2886,3476,5275], + "manifest_publish": [5,103269,107980,112762], + "recovery_cleanup": [5,83,189,273], + "outer_restore_refresh": [5,0,0,0], + "proven_insert_history": [5,0,0,0], + "proven_insert_plan_scan": [5,0,0,0], + "keyed_stage_total": [5,186736,192657,196098], + "keyed_stage_max": [5,24872,25126,25728], + "keyed_commit_total": [5,626808,634019,635123], + "keyed_commit_max": [5,79897,80066,81325] + }, + "operation_minus_preparation_us": null, + "counters": { + "io_delay_calls": [5,216,216,216], + "operation_io_data_open_count": [5,40,40,40], + "operation_io_data_opener_reads": [5,56,56,56], + "operation_io_data_reads": [5,192,192,192], + "operation_io_data_scan_reads": [5,128,128,128], + "operation_io_internal_open_count": [5,4,4,4], + "operation_io_manifest_read_bytes": [5,34828,34828,34830], + "operation_io_manifest_reads": [5,13,13,13], + "operation_io_manifest_scan_count": [5,3,3,3], + "probe_completed_full_walk_classifications": [5,0,0,0], + "probe_completed_lineage_classifications": [5,8,8,8], + "probe_lineage_candidate_address_take_calls": [5,16,16,16], + "probe_lineage_candidate_address_take_rows": [5,32,32,32], + "probe_lineage_candidate_scan_rows": [5,32,32,32], + "probe_ordered_cursor_batch_bytes": [5,0,0,0], + "probe_ordered_cursor_batch_rows": [5,0,0,0], + "probe_ordered_cursor_scan_calls": [5,0,0,0], + "probe_proven_insert_history_read_calls": [5,0,0,0], + "probe_scan_staged_combined_calls": [5,0,0,0], + "probe_stage_append_calls": [5,0,0,0], + "probe_stage_append_rows": [5,0,0,0], + "probe_stage_fenced_insert_calls": [5,0,0,0], + "probe_stage_fenced_insert_rows": [5,0,0,0], + "probe_stage_known_present_update_calls": [5,8,8,8], + "probe_stage_known_present_update_rows": [5,16,16,16], + "probe_stage_merge_insert_calls": [5,0,0,0], + "probe_stage_merge_insert_rows": [5,0,0,0], + "probe_stage_vector_index_calls": [5,0,0,0], + "probe_strict_insert_preflight_calls": [5,0,0,0], + "probe_validation_scan_batches": [5,8,8,8], + "probe_validation_scan_projected_bytes": [5,3968,3968,3968] + }, + "resource": { + "operation_peak_rss_bytes": [5,70696960,71041024,71139328], + "fixture_bytes": [5,2030285,2030312,2030431], + "fixture_files": [5,845,845,845] + }, + "preparation": null + }, + "B": { + "operation_us": [5,1909966,1956090,1969037], + "phase_us": { + "outer_prepare": [5,140294,142460,143175], + "candidate_preparation": [5,765298,777212,785351], + "candidate_validation": [5,1664,3950,4646], + "final_revalidation": [5,61675,63993,65334], + "recovery_arm": [5,313,532,811], + "physical_publish": [5,811954,825796,855356], + "recovery_confirm": [5,1427,3612,4003], + "manifest_publish": [5,99987,105518,107588], + "recovery_cleanup": [5,58,99,198], + "outer_restore_refresh": [5,0,0,0], + "proven_insert_history": [5,0,0,0], + "proven_insert_plan_scan": [5,0,0,0], + "keyed_stage_total": [5,180674,188837,196437], + "keyed_stage_max": [5,24223,25621,25812], + "keyed_commit_total": [5,624669,627766,651156], + "keyed_commit_max": [5,80481,80730,101712] + }, + "operation_minus_preparation_us": [5,1143652,1170739,1197831], + "counters": { + "io_delay_calls": [5,216,216,216], + "io_wrapped_api_active": [5,0,0,0], + "io_wrapped_api_peak_active": [5,4,4,4], + "operation_io_data_open_count": [5,40,40,40], + "operation_io_data_opener_reads": [5,56,56,56], + "operation_io_data_reads": [5,192,192,192], + "operation_io_data_scan_reads": [5,128,128,128], + "operation_io_internal_open_count": [5,4,4,4], + "operation_io_manifest_read_bytes": [5,34828,34828,34828], + "operation_io_manifest_reads": [5,13,13,13], + "operation_io_manifest_scan_count": [5,3,3,3], + "probe_completed_full_walk_classifications": [5,0,0,0], + "probe_completed_lineage_classifications": [5,8,8,8], + "probe_lineage_candidate_address_take_calls": [5,16,16,16], + "probe_lineage_candidate_address_take_rows": [5,32,32,32], + "probe_lineage_candidate_scan_rows": [5,32,32,32], + "probe_ordered_cursor_batch_bytes": [5,0,0,0], + "probe_ordered_cursor_batch_rows": [5,0,0,0], + "probe_ordered_cursor_scan_calls": [5,0,0,0], + "probe_proven_insert_history_read_calls": [5,0,0,0], + "probe_scan_staged_combined_calls": [5,0,0,0], + "probe_stage_append_calls": [5,0,0,0], + "probe_stage_append_rows": [5,0,0,0], + "probe_stage_fenced_insert_calls": [5,0,0,0], + "probe_stage_fenced_insert_rows": [5,0,0,0], + "probe_stage_known_present_update_calls": [5,8,8,8], + "probe_stage_known_present_update_rows": [5,16,16,16], + "probe_stage_merge_insert_calls": [5,0,0,0], + "probe_stage_merge_insert_rows": [5,0,0,0], + "probe_stage_vector_index_calls": [5,0,0,0], + "probe_strict_insert_preflight_calls": [5,0,0,0], + "probe_validation_scan_batches": [5,8,8,8], + "probe_validation_scan_projected_bytes": [5,3968,3968,3968] + }, + "resource": { + "operation_peak_rss_bytes": [5,72368128,72679424,72728576], + "fixture_bytes": [5,2030384,2030395,2030416], + "fixture_files": [5,845,845,845] + }, + "preparation": { + "accounted_bytes": [5,0,0,0], + "active": [5,0,0,0], + "admitted": [5,8,8,8], + "budget_fallbacks": [5,0,0,0], + "collected": [5,8,8,8], + "completed": [5,8,8,8], + "discarded": [5,0,0,0], + "peak_accounted_bytes": [5,285738,285738,285738], + "peak_active": [5,4,4,4], + "peak_ready": [5,4,4,4], + "peak_scratch_bytes": [5,2352,2352,2352], + "peak_scratch_owners": [5,8,8,8], + "peak_uncollected": [5,4,4,4], + "ready": [5,0,0,0], + "scratch_bytes": [5,0,0,0], + "scratch_owners": [5,0,0,0], + "uncollected": [5,0,0,0] + } + } + } + }, + { + "id": "catalog121-touched29-h0-d17-diagnostic-width4", + "acquisition": "final-catalog121-d17", + "params": { + "cache_state": "cold", + "delta_rows": 2, + "dims": 4, + "history_commits": 0, + "io_delay_ms": 17, + "manifest_layout": "uncompacted", + "populated_tables": 121, + "retired_branches": 0, + "rows": 4, + "seed": 42, + "source_mode": "update", + "tables": 29, + "target_delta_rows": 2 + }, + "current_mode": "diagnostic-width4", + "current_preparation_width_override": 4, + "paired_count": 5, + "samples": [ + { + "pair": 0, + "order": [ + "A", + "B" + ], + "A_operation_us": 12436418, + "B_operation_us": 6564379, + "A_rss_bytes": 73302016, + "B_rss_bytes": 74973184, + "A_record_sha256": "9cc84018ba34fb411170b845a685f3afc055a7cfab6a5e8c6db3c2ceca561fd6", + "B_record_sha256": "f1f45e849bec8b76f50ae4206e4351ee93ae05417528fc21bf75e94cb01d3172" + }, + { + "pair": 1, + "order": [ + "B", + "A" + ], + "A_operation_us": 12604438, + "B_operation_us": 6570726, + "A_rss_bytes": 72646656, + "B_rss_bytes": 74891264, + "A_record_sha256": "be93716ee76f06e4fe1ac5f4f0d461fd035bb106c7c6fd1a4347236f38ec191d", + "B_record_sha256": "aa8a950154d38a3507847feee78d69afa4cb7ceb1bf87ecd84fe9247346c748f" + }, + { + "pair": 2, + "order": [ + "A", + "B" + ], + "A_operation_us": 12539394, + "B_operation_us": 6597668, + "A_rss_bytes": 68108288, + "B_rss_bytes": 72581120, + "A_record_sha256": "52e5b02fee9d3f713ad1f4655023778cd6f1143c055cc47418f84a3e25f1c333", + "B_record_sha256": "ecf8f5400606f4e5c032da1259d7ed3f5e328870817b304781834453da1ba480" + }, + { + "pair": 3, + "order": [ + "B", + "A" + ], + "A_operation_us": 12625051, + "B_operation_us": 6598779, + "A_rss_bytes": 72728576, + "B_rss_bytes": 74629120, + "A_record_sha256": "e264c8976d182473a779545c51ee20b007d49933f14b16d5f04b4e0c5799a360", + "B_record_sha256": "db55fae9883f4a0eb31fae4254ddf816ffbd39386cbab16490397145023c3a18" + }, + { + "pair": 4, + "order": [ + "A", + "B" + ], + "A_operation_us": 12627201, + "B_operation_us": 6647721, + "A_rss_bytes": 68124672, + "B_rss_bytes": 74530816, + "A_record_sha256": "cf1d295992951d01321a01844ad103ec3fee580d4b6e571dfae4836883458d44", + "B_record_sha256": "695960e1bdc787855837b93b418a7e61b02e550a8b1098844320cb0db59252d1" + } + ], + "median_us": { + "A": 12604438, + "B": 6597668 + }, + "median_ms": { + "A": 12604.438, + "B": 6597.668 + }, + "median_paired_A_over_B": 1.9005797199859102, + "ratio_of_medians_A_over_B": 1.910438354885393, + "B_regression_percent_of_medians": -47.65599227827531, + "aggregates": { + "A": { + "operation_us": [5,12436418,12604438,12627201], + "phase_us": { + "outer_prepare": [5,141164,146834,148503], + "candidate_preparation": null, + "candidate_validation": [5,13364,15047,16437], + "final_revalidation": [5,66876,67558,70300], + "recovery_arm": [5,349,473,577], + "physical_publish": [5,3094420,3095768,3103833], + "recovery_confirm": [5,4111,8621,10893], + "manifest_publish": [5,104732,113658,117261], + "recovery_cleanup": [5,57,176,207], + "outer_restore_refresh": [5,0,0,0], + "proven_insert_history": [5,0,0,0], + "proven_insert_plan_scan": [5,0,0,0], + "keyed_stage_total": [5,704945,710123,712775], + "keyed_stage_max": [5,25910,26417,30725], + "keyed_commit_total": [5,2353410,2360804,2362830], + "keyed_commit_max": [5,84155,84778,85991] + }, + "operation_minus_preparation_us": null, + "counters": { + "io_delay_calls": [5,741,741,741], + "operation_io_data_open_count": [5,145,145,145], + "operation_io_data_opener_reads": [5,203,203,203], + "operation_io_data_reads": [5,696,696,696], + "operation_io_data_scan_reads": [5,464,464,464], + "operation_io_internal_open_count": [5,4,4,4], + "operation_io_manifest_read_bytes": [5,34827,34828,34830], + "operation_io_manifest_reads": [5,13,13,13], + "operation_io_manifest_scan_count": [5,3,3,3], + "probe_completed_full_walk_classifications": [5,0,0,0], + "probe_completed_lineage_classifications": [5,29,29,29], + "probe_lineage_candidate_address_take_calls": [5,58,58,58], + "probe_lineage_candidate_address_take_rows": [5,116,116,116], + "probe_lineage_candidate_scan_rows": [5,116,116,116], + "probe_ordered_cursor_batch_bytes": [5,0,0,0], + "probe_ordered_cursor_batch_rows": [5,0,0,0], + "probe_ordered_cursor_scan_calls": [5,0,0,0], + "probe_proven_insert_history_read_calls": [5,0,0,0], + "probe_scan_staged_combined_calls": [5,0,0,0], + "probe_stage_append_calls": [5,0,0,0], + "probe_stage_append_rows": [5,0,0,0], + "probe_stage_fenced_insert_calls": [5,0,0,0], + "probe_stage_fenced_insert_rows": [5,0,0,0], + "probe_stage_known_present_update_calls": [5,29,29,29], + "probe_stage_known_present_update_rows": [5,58,58,58], + "probe_stage_merge_insert_calls": [5,0,0,0], + "probe_stage_merge_insert_rows": [5,0,0,0], + "probe_stage_vector_index_calls": [5,0,0,0], + "probe_strict_insert_preflight_calls": [5,0,0,0], + "probe_validation_scan_batches": [5,29,29,29], + "probe_validation_scan_projected_bytes": [5,14384,14384,14384] + }, + "resource": { + "operation_peak_rss_bytes": [5,68108288,72646656,73302016], + "fixture_bytes": [5,4958704,4958812,4958913], + "fixture_files": [5,1097,1097,1097] + }, + "preparation": null + }, + "B": { + "operation_us": [5,6564379,6597668,6647721], + "phase_us": { + "outer_prepare": [5,143374,146915,148520], + "candidate_preparation": [5,3079021,3115564,3152078], + "candidate_validation": [5,8131,10911,13602], + "final_revalidation": [5,65381,68923,72440], + "recovery_arm": [5,357,506,1118], + "physical_publish": [5,3089755,3096300,3101011], + "recovery_confirm": [5,6422,9247,10587], + "manifest_publish": [5,109091,110319,113788], + "recovery_cleanup": [5,126,166,1526], + "outer_restore_refresh": [5,0,0,0], + "proven_insert_history": [5,0,0,0], + "proven_insert_plan_scan": [5,0,0,0], + "keyed_stage_total": [5,707969,709988,720473], + "keyed_stage_max": [5,26550,27274,27914], + "keyed_commit_total": [5,2347502,2354945,2361312], + "keyed_commit_max": [5,83515,84095,86266] + }, + "operation_minus_preparation_us": [5,3475720,3482655,3495643], + "counters": { + "io_delay_calls": [5,741,741,741], + "io_wrapped_api_active": [5,0,0,0], + "io_wrapped_api_peak_active": [5,4,4,4], + "operation_io_data_open_count": [5,145,145,145], + "operation_io_data_opener_reads": [5,203,203,203], + "operation_io_data_reads": [5,696,696,696], + "operation_io_data_scan_reads": [5,464,464,464], + "operation_io_internal_open_count": [5,4,4,4], + "operation_io_manifest_read_bytes": [5,34828,34828,34830], + "operation_io_manifest_reads": [5,13,13,13], + "operation_io_manifest_scan_count": [5,3,3,3], + "probe_completed_full_walk_classifications": [5,0,0,0], + "probe_completed_lineage_classifications": [5,29,29,29], + "probe_lineage_candidate_address_take_calls": [5,58,58,58], + "probe_lineage_candidate_address_take_rows": [5,116,116,116], + "probe_lineage_candidate_scan_rows": [5,116,116,116], + "probe_ordered_cursor_batch_bytes": [5,0,0,0], + "probe_ordered_cursor_batch_rows": [5,0,0,0], + "probe_ordered_cursor_scan_calls": [5,0,0,0], + "probe_proven_insert_history_read_calls": [5,0,0,0], + "probe_scan_staged_combined_calls": [5,0,0,0], + "probe_stage_append_calls": [5,0,0,0], + "probe_stage_append_rows": [5,0,0,0], + "probe_stage_fenced_insert_calls": [5,0,0,0], + "probe_stage_fenced_insert_rows": [5,0,0,0], + "probe_stage_known_present_update_calls": [5,29,29,29], + "probe_stage_known_present_update_rows": [5,58,58,58], + "probe_stage_merge_insert_calls": [5,0,0,0], + "probe_stage_merge_insert_rows": [5,0,0,0], + "probe_stage_vector_index_calls": [5,0,0,0], + "probe_strict_insert_preflight_calls": [5,0,0,0], + "probe_validation_scan_batches": [5,29,29,29], + "probe_validation_scan_projected_bytes": [5,14384,14384,14384] + }, + "resource": { + "operation_peak_rss_bytes": [5,72581120,74629120,74973184], + "fixture_bytes": [5,4958732,4958870,4958963], + "fixture_files": [5,1097,1097,1097] + }, + "preparation": { + "accounted_bytes": [5,0,0,0], + "active": [5,0,0,0], + "admitted": [5,29,29,29], + "budget_fallbacks": [5,0,0,0], + "collected": [5,29,29,29], + "completed": [5,29,29,29], + "discarded": [5,0,0,0], + "peak_accounted_bytes": [5,285738,285738,285738], + "peak_active": [5,4,4,4], + "peak_ready": [5,4,4,4], + "peak_scratch_bytes": [5,2352,2352,2352], + "peak_scratch_owners": [5,8,8,8], + "peak_uncollected": [5,4,4,4], + "ready": [5,0,0,0], + "scratch_bytes": [5,0,0,0], + "scratch_owners": [5,0,0,0], + "uncollected": [5,0,0,0] + } + } + } + }, + { + "id": "catalog8-touched1-h0-d17-production-default", + "acquisition": "final-default8-d17", + "params": { + "cache_state": "cold", + "delta_rows": 2, + "dims": 4, + "history_commits": 0, + "io_delay_ms": 17, + "manifest_layout": "uncompacted", + "populated_tables": 8, + "retired_branches": 0, + "rows": 4, + "seed": 42, + "source_mode": "update", + "tables": 1, + "target_delta_rows": 2 + }, + "current_mode": "production-default", + "current_preparation_width_override": null, + "paired_count": 2, + "samples": [ + { + "pair": 0, + "order": [ + "A", + "B" + ], + "A_operation_us": 837579, + "B_operation_us": 801212, + "A_rss_bytes": 66568192, + "B_rss_bytes": 67534848, + "A_record_sha256": "11b67e65a711973f3c070e5480465925365162790739e447d36a4c3868a165c7", + "B_record_sha256": "347e2ef4b34720838a15201a28e7fe72d3e4ce2fc97515badc29eb8e39f8716a" + }, + { + "pair": 1, + "order": [ + "B", + "A" + ], + "A_operation_us": 825758, + "B_operation_us": 818907, + "A_rss_bytes": 66600960, + "B_rss_bytes": 67567616, + "A_record_sha256": "ed9a57bdf0862ed403f066d70c64c14f633588e4ebf2a469b472dff9919b5218", + "B_record_sha256": "853c13d3f2968efa375ae2af00ea36445b83df51a8d836bfb8e6b4fe03c58854" + } + ], + "median_us": { + "A": 831668.5, + "B": 810059.5 + }, + "median_ms": { + "A": 831.6685, + "B": 810.0595 + }, + "median_paired_A_over_B": 1.0268780067638685, + "ratio_of_medians_A_over_B": 1.0266758182577946, + "B_regression_percent_of_medians": -2.5982708254550957, + "aggregates": { + "A": { + "operation_us": [2,825758,831668.5,837579], + "phase_us": { + "outer_prepare": [2,163581,165267.5,166954], + "candidate_preparation": null, + "candidate_validation": [2,970,1610.5,2251], + "final_revalidation": [2,60061,62148.5,64236], + "recovery_arm": [2,503,593.0,683], + "physical_publish": [2,108236,108742.0,109248], + "recovery_confirm": [2,1223,1445.0,1667], + "manifest_publish": [2,149884,150105.5,150327], + "recovery_cleanup": [2,136,147.5,159], + "outer_restore_refresh": [2,0,0.0,0], + "proven_insert_history": [2,0,0.0,0], + "proven_insert_plan_scan": [2,0,0.0,0], + "keyed_stage_total": [2,25154,25551.0,25948], + "keyed_stage_max": [2,25154,25551.0,25948], + "keyed_commit_total": [2,81957,82143.0,82329], + "keyed_commit_max": [2,81957,82143.0,82329] + }, + "operation_minus_preparation_us": null, + "counters": { + "io_delay_calls": [2,44,44.0,44], + "operation_io_data_open_count": [2,5,5.0,5], + "operation_io_data_opener_reads": [2,7,7.0,7], + "operation_io_data_reads": [2,24,24.0,24], + "operation_io_data_scan_reads": [2,16,16.0,16], + "operation_io_internal_open_count": [2,4,4.0,4], + "operation_io_manifest_read_bytes": [2,45309,45309.5,45310], + "operation_io_manifest_reads": [2,16,16.0,16], + "operation_io_manifest_scan_count": [2,3,3.0,3], + "probe_completed_full_walk_classifications": [2,0,0.0,0], + "probe_completed_lineage_classifications": [2,1,1.0,1], + "probe_lineage_candidate_address_take_calls": [2,2,2.0,2], + "probe_lineage_candidate_address_take_rows": [2,4,4.0,4], + "probe_lineage_candidate_scan_rows": [2,4,4.0,4], + "probe_ordered_cursor_batch_bytes": [2,0,0.0,0], + "probe_ordered_cursor_batch_rows": [2,0,0.0,0], + "probe_ordered_cursor_scan_calls": [2,0,0.0,0], + "probe_proven_insert_history_read_calls": [2,0,0.0,0], + "probe_scan_staged_combined_calls": [2,0,0.0,0], + "probe_stage_append_calls": [2,0,0.0,0], + "probe_stage_append_rows": [2,0,0.0,0], + "probe_stage_fenced_insert_calls": [2,0,0.0,0], + "probe_stage_fenced_insert_rows": [2,0,0.0,0], + "probe_stage_known_present_update_calls": [2,1,1.0,1], + "probe_stage_known_present_update_rows": [2,2,2.0,2], + "probe_stage_merge_insert_calls": [2,0,0.0,0], + "probe_stage_merge_insert_rows": [2,0,0.0,0], + "probe_stage_vector_index_calls": [2,0,0.0,0], + "probe_strict_insert_preflight_calls": [2,0,0.0,0], + "probe_validation_scan_batches": [2,1,1.0,1], + "probe_validation_scan_projected_bytes": [2,496,496.0,496] + }, + "resource": { + "operation_peak_rss_bytes": [2,66568192,66584576.0,66600960], + "fixture_bytes": [2,405698,405713.0,405728], + "fixture_files": [2,83,83.0,83] + }, + "preparation": null + }, + "B": { + "operation_us": [2,801212,810059.5,818907], + "phase_us": { + "outer_prepare": [2,162957,163384.0,163811], + "candidate_preparation": [2,302987,311687.0,320387], + "candidate_validation": [2,712,979.5,1247], + "final_revalidation": [2,59670,60631.0,61592], + "recovery_arm": [2,215,254.0,293], + "physical_publish": [2,104976,105242.0,105508], + "recovery_confirm": [2,453,659.5,866], + "manifest_publish": [2,145095,146771.5,148448], + "recovery_cleanup": [2,61,117.5,174], + "outer_restore_refresh": [2,0,0.0,0], + "proven_insert_history": [2,0,0.0,0], + "proven_insert_plan_scan": [2,0,0.0,0], + "keyed_stage_total": [2,22601,22790.0,22979], + "keyed_stage_max": [2,22601,22790.0,22979], + "keyed_commit_total": [2,81673,81830.5,81988], + "keyed_commit_max": [2,81673,81830.5,81988] + }, + "operation_minus_preparation_us": [2,498225,498372.5,498520], + "counters": { + "io_delay_calls": [2,44,44.0,44], + "io_wrapped_api_active": [2,0,0.0,0], + "io_wrapped_api_peak_active": [2,2,2.0,2], + "operation_io_data_open_count": [2,5,5.0,5], + "operation_io_data_opener_reads": [2,7,7.0,7], + "operation_io_data_reads": [2,24,24.0,24], + "operation_io_data_scan_reads": [2,16,16.0,16], + "operation_io_internal_open_count": [2,4,4.0,4], + "operation_io_manifest_read_bytes": [2,45307,45308.5,45310], + "operation_io_manifest_reads": [2,16,16.0,16], + "operation_io_manifest_scan_count": [2,3,3.0,3], + "probe_completed_full_walk_classifications": [2,0,0.0,0], + "probe_completed_lineage_classifications": [2,1,1.0,1], + "probe_lineage_candidate_address_take_calls": [2,2,2.0,2], + "probe_lineage_candidate_address_take_rows": [2,4,4.0,4], + "probe_lineage_candidate_scan_rows": [2,4,4.0,4], + "probe_ordered_cursor_batch_bytes": [2,0,0.0,0], + "probe_ordered_cursor_batch_rows": [2,0,0.0,0], + "probe_ordered_cursor_scan_calls": [2,0,0.0,0], + "probe_proven_insert_history_read_calls": [2,0,0.0,0], + "probe_scan_staged_combined_calls": [2,0,0.0,0], + "probe_stage_append_calls": [2,0,0.0,0], + "probe_stage_append_rows": [2,0,0.0,0], + "probe_stage_fenced_insert_calls": [2,0,0.0,0], + "probe_stage_fenced_insert_rows": [2,0,0.0,0], + "probe_stage_known_present_update_calls": [2,1,1.0,1], + "probe_stage_known_present_update_rows": [2,2,2.0,2], + "probe_stage_merge_insert_calls": [2,0,0.0,0], + "probe_stage_merge_insert_rows": [2,0,0.0,0], + "probe_stage_vector_index_calls": [2,0,0.0,0], + "probe_strict_insert_preflight_calls": [2,0,0.0,0], + "probe_validation_scan_batches": [2,1,1.0,1], + "probe_validation_scan_projected_bytes": [2,496,496.0,496] + }, + "resource": { + "operation_peak_rss_bytes": [2,67534848,67551232.0,67567616], + "fixture_bytes": [2,405701,405714.5,405728], + "fixture_files": [2,83,83.0,83] + }, + "preparation": { + "accounted_bytes": [2,0,0.0,0], + "active": [2,0,0.0,0], + "admitted": [2,1,1.0,1], + "budget_fallbacks": [2,0,0.0,0], + "collected": [2,1,1.0,1], + "completed": [2,1,1.0,1], + "discarded": [2,0,0.0,0], + "peak_accounted_bytes": [2,0,0.0,0], + "peak_active": [2,1,1.0,1], + "peak_ready": [2,1,1.0,1], + "peak_scratch_bytes": [2,588,588.0,588], + "peak_scratch_owners": [2,2,2.0,2], + "peak_uncollected": [2,1,1.0,1], + "ready": [2,0,0.0,0], + "scratch_bytes": [2,0,0.0,0], + "scratch_owners": [2,0,0.0,0], + "uncollected": [2,0,0.0,0] + } + } + } + }, + { + "id": "catalog8-touched8-h0-d17-production-default", + "acquisition": "final-default8-d17", + "params": { + "cache_state": "cold", + "delta_rows": 2, + "dims": 4, + "history_commits": 0, + "io_delay_ms": 17, + "manifest_layout": "uncompacted", + "populated_tables": 8, + "retired_branches": 0, + "rows": 4, + "seed": 42, + "source_mode": "update", + "tables": 8, + "target_delta_rows": 2 + }, + "current_mode": "production-default", + "current_preparation_width_override": null, + "paired_count": 2, + "samples": [ + { + "pair": 0, + "order": [ + "A", + "B" + ], + "A_operation_us": 3700534, + "B_operation_us": 3714576, + "A_rss_bytes": 67829760, + "B_rss_bytes": 64880640, + "A_record_sha256": "7bf54cc3f562e29db31c6c2f39930012162618b410f9bfe2f51701aa22ace63c", + "B_record_sha256": "65e7bb0342c8aa1a4b5f90e5487d786c5865017a12d21d680d98be4c54368701" + }, + { + "pair": 1, + "order": [ + "B", + "A" + ], + "A_operation_us": 3676998, + "B_operation_us": 3703229, + "A_rss_bytes": 63078400, + "B_rss_bytes": 68485120, + "A_record_sha256": "fb279afb32987dd83614f6e78a2996cc69bcdf48125485e2307adfb9ddd2a556", + "B_record_sha256": "5b2b73009826a48dd105809407ef22c5b9d71e3b922e927a7d73ed6772a9b05e" + } + ], + "median_us": { + "A": 3688766.0, + "B": 3708902.5 + }, + "median_ms": { + "A": 3688.766, + "B": 3708.9025 + }, + "median_paired_A_over_B": 0.9945682395576346, + "ratio_of_medians_A_over_B": 0.9945707658802031, + "B_regression_percent_of_medians": 0.5458871611807359, + "aggregates": { + "A": { + "operation_us": [2,3676998,3688766.0,3700534], + "phase_us": { + "outer_prepare": [2,138948,140951.0,142954], + "candidate_preparation": null, + "candidate_validation": [2,5251,5501.0,5751], + "final_revalidation": [2,61478,62321.5,63165], + "recovery_arm": [2,524,539.5,555], + "physical_publish": [2,851655,857912.5,864170], + "recovery_confirm": [2,3358,3480.0,3602], + "manifest_publish": [2,107156,108788.5,110421], + "recovery_cleanup": [2,139,155.0,171], + "outer_restore_refresh": [2,0,0.0,0], + "proven_insert_history": [2,0,0.0,0], + "proven_insert_plan_scan": [2,0,0.0,0], + "keyed_stage_total": [2,197143,204181.0,211219], + "keyed_stage_max": [2,26950,32056.0,37162], + "keyed_commit_total": [2,644993,645732.0,646471], + "keyed_commit_max": [2,82342,82840.0,83338] + }, + "operation_minus_preparation_us": null, + "counters": { + "io_delay_calls": [2,216,216.0,216], + "operation_io_data_open_count": [2,40,40.0,40], + "operation_io_data_opener_reads": [2,56,56.0,56], + "operation_io_data_reads": [2,192,192.0,192], + "operation_io_data_scan_reads": [2,128,128.0,128], + "operation_io_internal_open_count": [2,4,4.0,4], + "operation_io_manifest_read_bytes": [2,34813,34813.0,34813], + "operation_io_manifest_reads": [2,13,13.0,13], + "operation_io_manifest_scan_count": [2,3,3.0,3], + "probe_completed_full_walk_classifications": [2,0,0.0,0], + "probe_completed_lineage_classifications": [2,8,8.0,8], + "probe_lineage_candidate_address_take_calls": [2,16,16.0,16], + "probe_lineage_candidate_address_take_rows": [2,32,32.0,32], + "probe_lineage_candidate_scan_rows": [2,32,32.0,32], + "probe_ordered_cursor_batch_bytes": [2,0,0.0,0], + "probe_ordered_cursor_batch_rows": [2,0,0.0,0], + "probe_ordered_cursor_scan_calls": [2,0,0.0,0], + "probe_proven_insert_history_read_calls": [2,0,0.0,0], + "probe_scan_staged_combined_calls": [2,0,0.0,0], + "probe_stage_append_calls": [2,0,0.0,0], + "probe_stage_append_rows": [2,0,0.0,0], + "probe_stage_fenced_insert_calls": [2,0,0.0,0], + "probe_stage_fenced_insert_rows": [2,0,0.0,0], + "probe_stage_known_present_update_calls": [2,8,8.0,8], + "probe_stage_known_present_update_rows": [2,16,16.0,16], + "probe_stage_merge_insert_calls": [2,0,0.0,0], + "probe_stage_merge_insert_rows": [2,0,0.0,0], + "probe_stage_vector_index_calls": [2,0,0.0,0], + "probe_strict_insert_preflight_calls": [2,0,0.0,0], + "probe_validation_scan_batches": [2,8,8.0,8], + "probe_validation_scan_projected_bytes": [2,3968,3968.0,3968] + }, + "resource": { + "operation_peak_rss_bytes": [2,63078400,65454080.0,67829760], + "fixture_bytes": [2,1382208,1382209.5,1382211], + "fixture_files": [2,167,167.0,167] + }, + "preparation": null + }, + "B": { + "operation_us": [2,3703229,3708902.5,3714576], + "phase_us": { + "outer_prepare": [2,138826,139878.5,140931], + "candidate_preparation": [2,2501087,2505194.0,2509301], + "candidate_validation": [2,4014,4732.0,5450], + "final_revalidation": [2,65135,65512.0,65889], + "recovery_arm": [2,737,1107.5,1478], + "physical_publish": [2,847516,851119.5,854723], + "recovery_confirm": [2,2950,3201.0,3452], + "manifest_publish": [2,108269,110348.0,112427], + "recovery_cleanup": [2,175,191.0,207], + "outer_restore_refresh": [2,0,0.0,0], + "proven_insert_history": [2,0,0.0,0], + "proven_insert_plan_scan": [2,0,0.0,0], + "keyed_stage_total": [2,193827,195703.0,197579], + "keyed_stage_max": [2,25407,26413.5,27420], + "keyed_commit_total": [2,643869,646965.5,650062], + "keyed_commit_max": [2,82559,82973.5,83388] + }, + "operation_minus_preparation_us": [2,1202142,1203708.5,1205275], + "counters": { + "io_delay_calls": [2,216,216.0,216], + "io_wrapped_api_active": [2,0,0.0,0], + "io_wrapped_api_peak_active": [2,2,2.0,2], + "operation_io_data_open_count": [2,40,40.0,40], + "operation_io_data_opener_reads": [2,56,56.0,56], + "operation_io_data_reads": [2,192,192.0,192], + "operation_io_data_scan_reads": [2,128,128.0,128], + "operation_io_internal_open_count": [2,4,4.0,4], + "operation_io_manifest_read_bytes": [2,34811,34811.5,34812], + "operation_io_manifest_reads": [2,13,13.0,13], + "operation_io_manifest_scan_count": [2,3,3.0,3], + "probe_completed_full_walk_classifications": [2,0,0.0,0], + "probe_completed_lineage_classifications": [2,8,8.0,8], + "probe_lineage_candidate_address_take_calls": [2,16,16.0,16], + "probe_lineage_candidate_address_take_rows": [2,32,32.0,32], + "probe_lineage_candidate_scan_rows": [2,32,32.0,32], + "probe_ordered_cursor_batch_bytes": [2,0,0.0,0], + "probe_ordered_cursor_batch_rows": [2,0,0.0,0], + "probe_ordered_cursor_scan_calls": [2,0,0.0,0], + "probe_proven_insert_history_read_calls": [2,0,0.0,0], + "probe_scan_staged_combined_calls": [2,0,0.0,0], + "probe_stage_append_calls": [2,0,0.0,0], + "probe_stage_append_rows": [2,0,0.0,0], + "probe_stage_fenced_insert_calls": [2,0,0.0,0], + "probe_stage_fenced_insert_rows": [2,0,0.0,0], + "probe_stage_known_present_update_calls": [2,8,8.0,8], + "probe_stage_known_present_update_rows": [2,16,16.0,16], + "probe_stage_merge_insert_calls": [2,0,0.0,0], + "probe_stage_merge_insert_rows": [2,0,0.0,0], + "probe_stage_vector_index_calls": [2,0,0.0,0], + "probe_strict_insert_preflight_calls": [2,0,0.0,0], + "probe_validation_scan_batches": [2,8,8.0,8], + "probe_validation_scan_projected_bytes": [2,3968,3968.0,3968] + }, + "resource": { + "operation_peak_rss_bytes": [2,64880640,66682880.0,68485120], + "fixture_bytes": [2,1382163,1382173.5,1382184], + "fixture_files": [2,167,167.0,167] + }, + "preparation": { + "accounted_bytes": [2,0,0.0,0], + "active": [2,0,0.0,0], + "admitted": [2,8,8.0,8], + "budget_fallbacks": [2,0,0.0,0], + "collected": [2,8,8.0,8], + "completed": [2,8,8.0,8], + "discarded": [2,0,0.0,0], + "peak_accounted_bytes": [2,0,0.0,0], + "peak_active": [2,1,1.0,1], + "peak_ready": [2,1,1.0,1], + "peak_scratch_bytes": [2,588,588.0,588], + "peak_scratch_owners": [2,2,2.0,2], + "peak_uncollected": [2,1,1.0,1], + "ready": [2,0,0.0,0], + "scratch_bytes": [2,0,0.0,0], + "scratch_owners": [2,0,0.0,0], + "uncollected": [2,0,0.0,0] + } + } + } + } + ], + "promotion_gate": { + "all_required_speed_points_passed": false, + "criteria": [ + { + "point": "catalog121-touched1-h0-d17-diagnostic-width4", + "requirement": "B median regression <= 10%", + "passed": true + }, + { + "point": "catalog121-touched8-h0-d17-diagnostic-width4", + "requirement": "median A/B speedup >= 2.0", + "passed": false + }, + { + "point": "catalog121-touched29-h0-d17-diagnostic-width4", + "requirement": "median A/B speedup >= 2.0", + "passed": false + } + ], + "production_parallel_activation_enabled": false + }, + "validation": { + "source_and_binary_inconsistencies": [], + "source_summary_statistics_recomputed_and_matched": 720, + "all_raw_records_pass_owning_admission": true, + "all_phase_exit_codes_zero": true, + "all_source_preservation_receipts_passed": true, + "all_B_end_gauges_zero": true, + "all_B_budget_fallbacks_zero": true, + "history_count_explanation": "The age ledger has four driver invocations, each containing H0/H16/H64: 12 individual merges. All ledgers total 50 driver invocations and 58 individual merges; those are 29 pairs, not 58 pairs.", + "selection": "Only the four final paired acquisitions; baseline smoke/build/test runs are excluded." + }, + "limitations": [ + "Local directional diagnostics only; no production latency or tail-latency claim, and no p95 estimate.", + "B production preparation default and its HTTP request-owned ceiling are one. Diagnostic points explicitly override embedded preparation width to four; default controls omit the override.", + "The five-pair 17 ms promotion threshold fails at both eight and 29 changed tables; production parallel activation remains disabled. One-table median regression is within 10%.", + "All measurements use asynchronous delay at wrapped graph ObjectStore API boundaries, not measured S3 network requests. No bandwidth limit, jitter, throttling, 50 ms delay, deployed-store, or broad workload qualification is represented.", + "Cold means a fresh operation process and graph open without metadata prewarm. OS page cache is uncontrolled.", + "Each graph retains four rows per populated type. History aging performs current-format updates restoring the original row values; it also grows physical history/fragments and does not isolate fragmentation or cover old file-format migrations.", + "Operation timing starts after graph open and optional prewarm. Setup and full verification are separate child processes.", + "Operation RSS is a whole-child high-water mark including runtime, open, prewarm, merge and output. The 256 MiB Lance pool is not a process RSS cap; no hard process memory cap was requested.", + "Accounted preparation bytes are not total allocations or allocation overshoot. Scratch bytes are logical staged Arrow payload, not physical scratch-disk usage or RSS.", + "API concurrency covers observed wrapped futures and list/delete streams through return/EOF/drop, including injected delay; excludes GET body transfer, wire retries and unwrapped local I/O.", + "Phase timers can overlap or nest. Keyed stage/commit totals belong inside physical publication; do not sum all phases. Operation minus preparation is a per-sample computed remainder. A preparation wall and concurrency gauges are unavailable, not zero.", + "Five pairs per main 17 ms point and two pairs per control are small samples. History points run H0,H16,H64 within each A or B driver invocation, so opposite variants are paired by matching history/pair ID but are not immediately adjacent at the individual-operation level.", + "All verification claims are the persisted benchmark verification receipts; temporary fixtures were not independently reopened by this consolidation." + ] +} diff --git a/docs/rfcs/assets/0053-merge-preparation-results.csv b/docs/rfcs/assets/0057-merge-preparation-prototype-results.csv similarity index 100% rename from docs/rfcs/assets/0053-merge-preparation-results.csv rename to docs/rfcs/assets/0057-merge-preparation-prototype-results.csv diff --git a/docs/rfcs/assets/0053-merge-preparation-prototype.zip b/docs/rfcs/assets/0057-merge-preparation-prototype.zip similarity index 100% rename from docs/rfcs/assets/0053-merge-preparation-prototype.zip rename to docs/rfcs/assets/0057-merge-preparation-prototype.zip