diff --git a/log-detailed.md b/log-detailed.md new file mode 100644 index 0000000000000..fba0542673d3d --- /dev/null +++ b/log-detailed.md @@ -0,0 +1,505 @@ +# Columnar Rendering: Detailed Work Log + + + +## Prompt 0.1: Introduce ColumnarCollection type alias and ColumnarBundle + +### What was done +- Added `ColumnarCollection` type alias in `typedefs.rs`, defined as `Collection::Timestamp, R)>>`. +- Added `columnar_collection` field to `CollectionBundle` (initially always `None`). +- Added `from_columnar_collections(oks, errs)` constructor. +- Added `columnar_collection()` accessor returning `Option<&(ColumnarCollection, VecCollection)>`. +- Added `ensure_vec_collection()` escape hatch that converts columnar → Vec using `timely::dataflow::operators::core::Map` and `columnar::Columnar::into_owned`. +- Updated `enter_region`, `leave_region`, `scope()`, and `update_id` to propagate the new field. +- All existing constructors (`from_collections`, `from_expressions`, `from_columns`) set `columnar_collection: None`. + +### Key decisions +- Error streams remain `VecCollection` even in the columnar variant, since `DataflowError` is not suited for columnar layout (per project notes). +- `ensure_vec_collection()` uses timely's `Map::map` to convert each columnar ref to owned `(Row, T, Diff)` tuples. This is item-at-a-time but is only the escape hatch. +- The columnar field is `Option` so it's fully backwards-compatible; no existing code paths are affected. + +### Files changed +- `src/compute/src/typedefs.rs` — Added `ColumnarCollection` type alias, `Column` and `Collection` imports. +- `src/compute/src/render/context.rs` — Added field, constructors, accessors, escape hatch, updated region/scope/update methods. + +### Issues +- Initial `flat_map` approach for `ensure_vec_collection` failed because timely's `Map::flat_map` operates per-item, not per-container. Switched to `Map::map` which correctly takes each `Ref<'_, (Row, T, Diff)>` item and converts to owned. + +## Prompt 0.2: Columnar ↔ Vec conversion utilities + +### What was done +- Implemented `vec_to_columnar` and `columnar_to_vec` free functions in new module `src/compute/src/render/columnar.rs`. +- `vec_to_columnar` uses `StreamCore::unary` with `ColumnBuilder<(Row, T, Diff)>` as the output container builder. Iterates Vec input and pushes each `(row, time, diff)` into the columnar session. ColumnBuilder handles batch sizing automatically (~2MB aligned containers). +- `columnar_to_vec` uses `StreamCore::unary` with `CapacityContainerBuilder>` as output. Iterates the columnar container via `data.borrow().into_index_iter()` and converts each ref to owned via `Columnar::into_owned`. +- Added two unit tests: `round_trip_vec_columnar_vec` (4 diverse rows including empty, multi-type, and multi-datum rows) and `round_trip_multiple_timestamps` (verifies timestamp preservation across time advances). + +### Key decisions +- Functions take collections by value (not reference) since `StreamCore::unary` consumes `self`. +- Batch sizing is delegated to `ColumnBuilder`'s built-in ~2MB alignment logic rather than a configurable constant, matching the existing codebase pattern. +- Used `Pipeline` pact (no exchange/repartitioning) since these are pure format conversions. +- Created a new module `render/columnar.rs` rather than putting utilities in `context.rs`, keeping conversion logic separate from the bundle management code. + +### Files changed +- `src/compute/src/render/columnar.rs` — New file with `vec_to_columnar`, `columnar_to_vec`, and tests. +- `src/compute/src/render.rs` — Added `pub(crate) mod columnar;` module declaration. + +### Issues +- Required `columnar::Index` trait import for `into_index_iter()` — the trait was implemented but not in scope. +- `Diff` is `Overflowing`, not `i64`, so tests needed `input.update(row, Diff::from(1))` instead of `input.insert(row)`. +- `Probe::probe()` returns a tuple `(ProbeHandle, Stream)`, requiring destructuring. + +## Prompt 1.1: Persist source emits columnar collections + +### What was done +- Investigated `persist_source::persist_source()` — it returns `StreamVec` (Vec-based containers). It does not use `ColumnBuilder` natively, so a `vec_to_columnar` conversion at the boundary is the practical approach. +- Modified both import loops in `render.rs` (recursive and non-recursive dataflow paths) to create columnar collections alongside the existing Vec collections. +- At each import boundary: clone the entered Vec collection, convert the clone to columnar via `vec_to_columnar`, and set both `collection` and `columnar_collection` fields on the `CollectionBundle`. +- All downstream operators continue to work unchanged since they access `self.collection` (the Vec variant), which is always populated. + +### Key decisions +- Set **both** `collection` and `columnar_collection` fields rather than only columnar. This avoids needing to modify `as_specific_collection` (which takes `&self`, not `&mut self`) and all downstream operators. The columnar collection is available for future operators to use. +- Applied `vec_to_columnar` **after** `enter`/`enter_region`, not before, because `Column<(Row, T, Diff)>` does not implement the `Enter` trait required by differential_dataflow's `enter()` method. +- The conversion cost is the overhead of `vec_to_columnar` at every import, even though no downstream operator uses the columnar variant yet. This will pay off as operators are converted in later prompts. + +### Files changed +- `src/compute/src/render.rs` — Modified both import loops (recursive at ~line 378, non-recursive at ~line 487) to create `CollectionBundle` with both Vec and columnar collections populated. + +### Issues +- `Column<(Row, T, Diff)>` does not implement `differential_dataflow::collection::containers::Enter`, so `enter()` cannot be called on a `ColumnarCollection`. Solved by converting to columnar after entering the region scope. +- Test binary linking OOMs in the constrained CI environment, but `cargo check` passes cleanly, confirming type correctness. + +## Prompt 2.1: Negate operator propagates columnar collections + +### What was done +- Added `negate_columnar` function in `render/columnar.rs` that takes a `ColumnarCollection` and produces a new one with all diffs negated. +- Modified the `Negate` match arm in `render.rs` to check for a columnar collection first. If present, uses `negate_columnar` to produce a columnar output bundle. Otherwise falls back to the existing Vec path. +- Added unit test `negate_columnar_flips_diffs` that verifies diffs are correctly negated through Vec→Columnar→Negate→Columnar→Vec round-trip. + +### Key decisions +- Used a `unary` operator with `ColumnBuilder` output (same pattern as `vec_to_columnar`) rather than trying to call `Collection::negate()` directly, since the columnar `Collection` type may not support `negate()` out of the box. +- When columnar is available, produces only a columnar output (no Vec). Downstream operators that need Vec will use `ensure_vec_collection()`. +- The `into_owned` conversion for the diff is necessary to apply `Neg`, but row and timestamp refs are passed directly to the `ColumnBuilder` session without conversion. + +### Files changed +- `src/compute/src/render/columnar.rs` — Added `negate_columnar` function and test. +- `src/compute/src/render.rs` — Modified Negate match arm to use columnar path when available. + +### Issues +- `Columnar::into_owned(r)` required explicit type annotation (`: Diff`) because the compiler couldn't infer the type through the `-` negation operator. + +## Prompt 2.2: Union operator propagates columnar collections + +### What was done +- Modified the `Union` match arm in `render.rs` to check if all inputs have columnar collections. If so, uses `differential_dataflow::collection::concatenate` on the columnar collections directly. +- When `consolidate_output` is true and inputs are columnar, converts to Vec for consolidation (which requires Vec-based batchers), then converts back to columnar. +- Falls back to the existing Vec path when any input lacks a columnar collection. +- Added unit test `union_columnar_concatenates` that verifies columnar concatenation preserves all rows from two input streams, including duplicate rows with different diffs. + +### Key decisions +- Used an "all or nothing" strategy: if all inputs have columnar, use columnar; otherwise fall back to Vec for all. This avoids the overhead of converting individual inputs. +- For `consolidate_output`, the consolidation uses `KeyBatcher` which requires Vec-based collections. Rather than implementing a columnar-native consolidation, we round-trip through Vec→consolidate→columnar. This is acceptable because consolidation is already expensive and the conversion overhead is small relative to the sort/merge. +- `concatenate` from `differential_dataflow` is generic over container types and works with `Column<...>` containers directly. + +### Files changed +- `src/compute/src/render.rs` — Modified Union match arm to support columnar path. +- `src/compute/src/render/columnar.rs` — Added `union_columnar_concatenates` test. + +### Issues +- None. `concatenate` worked out of the box with columnar collections since `Column` implements the required `Container` trait. + +## Prompt 2.3: Constant operator emits columnar collections + +### What was done +- Modified the `Constant` match arm in `render.rs` to also produce a columnar collection alongside the existing Vec collection. +- After creating the `ok_collection` (Vec-based), clones it and converts via `vec_to_columnar` to produce a columnar variant. +- Sets both `collection` and `columnar_collection` fields on the resulting `CollectionBundle`, so downstream operators can use either path. +- Added unit test `constant_rows_to_columnar` that simulates the Constant operator pattern: creates rows from an iterator, converts to stream, then round-trips through columnar. + +### Key decisions +- Set **both** Vec and columnar fields (same pattern as persist source in Prompt 1.1) rather than columnar-only, since downstream operators may still need the Vec path. +- Constants are typically small, so the `vec_to_columnar` overhead is negligible. This change is primarily for uniformity so that all source operators produce columnar collections. + +### Files changed +- `src/compute/src/render.rs` — Modified Constant match arm to also produce columnar collection. +- `src/compute/src/render/columnar.rs` — Added `constant_rows_to_columnar` test. + +### Issues +- None. Straightforward application of the established pattern. + +## Prompt 3.1: Columnar `as_collection_core` with Get/Mfp wiring + +### What was done +- Added `as_columnar_collection_core` method on `CollectionBundle` in `context.rs`. It delegates to the existing `as_collection_core` (row-at-a-time MFP evaluation) and converts the output to columnar via `vec_to_columnar`. +- When the bundle has only a columnar collection (no Vec), the method converts columnar→Vec first by creating a temporary bundle with Vec populated, then delegates to `as_collection_core`. +- Wired `render_plan_expr` in `render.rs` for `Get::Collection` and `Mfp` to check for columnar input and use `as_columnar_collection_core` when available, producing columnar-only output bundles. + +### Key decisions +- When only columnar input is available (no Vec collection), a temporary `CollectionBundle` is constructed with the Vec conversion so `as_collection_core` can proceed. This avoids modifying the immutable `&self`. +- Only `Get::Collection` and `Mfp` are wired to the columnar path. `Get::Arrangement` always uses arrangement access (key_val is Some) so columnar input is irrelevant there. +- The `Get::PassArrangements` path is unchanged since it only passes through arrangements. +- This is the incremental step: all MFP evaluation is still row-at-a-time. Vectorized evaluation will replace the inner loop in Prompt 3.2. + +### Files changed +- `src/compute/src/render/context.rs` — Added `as_columnar_collection_core` method. +- `src/compute/src/render.rs` — Modified `Get::Collection` and `Mfp` match arms to prefer columnar path. + +### Issues +- None. The temporary bundle clone is slightly wasteful but only occurs when the bundle lacks a Vec collection (columnar-only). Arrangement clones are cheap (reference-counted). + +## Prompt 4.1: Columnar FlatMap + +### What was done +- Modified `render_flat_map` to accept columnar input and emit columnar output. +- At the start of the method, checks if the input has a columnar collection (`has_columnar`). If the input has no Vec collection (columnar-only), calls `ensure_vec_collection()` to convert for the row-at-a-time table function evaluation. +- At the end, if the input was columnar, converts the Vec ok output to columnar via `vec_to_columnar` and returns a columnar-only bundle. Otherwise returns Vec as before. + +### Key decisions +- Table functions are inherently row-at-a-time (variable output rows per input), so no attempt is made to vectorize the table function evaluation itself. The columnar conversion is purely at the input/output boundaries. +- Used `ensure_vec_collection()` (the escape hatch from Prompt 0.1) to handle columnar-only inputs, which converts columnar→Vec in-place on the mutable bundle. +- Output follows the same pattern as Negate/Union: columnar-only when input was columnar, Vec-only otherwise. + +### Files changed +- `src/compute/src/render/flat_map.rs` — Modified `render_flat_map` to handle columnar input/output. + +### Issues +- None. Straightforward boundary conversion. + +## Prompt 5.1: Columnar input to arrangements + +### What was done +- Modified `ensure_collections` in `context.rs` to handle columnar-only inputs when creating arrangements. +- When the method needs a Vec collection (`form_raw_collection && self.collection.is_none()`) and the non-arrangement path is used (`input_key.is_none()`), it now calls `ensure_vec_collection()` to convert columnar→Vec before delegating to `as_collection_core`. +- This prevents a panic in `as_collection_core`/`as_specific_collection` when only columnar is available and no arrangement key is provided. + +### Key decisions +- Only convert columnar→Vec when `input_key.is_none()`, because when `input_key` is Some, `as_collection_core` uses the arrangement path which doesn't need `self.collection`. +- The arrangement spines (`RowRowSpine`) use `DatumContainer` which accepts Row input, so unpacking columnar→Row at this boundary is required regardless. The existing `arrange_collection` method works on the Vec collection unchanged. +- No changes needed to `extensions/arrange.rs` — the plumbing is entirely in `ensure_collections`. + +### Files changed +- `src/compute/src/render/context.rs` — Added columnar→Vec conversion guard in `ensure_collections`. + +### Issues +- None. Minimal change since the arrangement infrastructure already works with Vec collections. + +## Prompt 6.1: Columnar Reduce input + +### What was done +- Modified `render_reduce` in `reduce.rs` to handle columnar-only inputs. +- After `input.enter_region(inner)`, checks if the entered bundle has only columnar (no Vec collection). If so, calls `ensure_vec_collection()` to convert before the `flat_map` call that extracts keys and values. +- The key extraction, value extraction, and aggregation logic remain row-at-a-time. The columnar→Vec conversion happens at the operator boundary. + +### Key decisions +- Same pattern as FlatMap (Prompt 4.1) and ensure_collections (Prompt 5.1): convert at the boundary, process row-at-a-time internally. +- The conversion happens after `enter_region` since the entered bundle is what `flat_map` operates on. +- The reduce output (arrangements) remains unchanged — reduce inherently produces arrangements which are already efficient. + +### Files changed +- `src/compute/src/render/reduce.rs` — Added columnar→Vec conversion guard before `flat_map` call. + +### Issues +- None. Straightforward boundary conversion. + +## Prompt 6.2: Columnar TopK input + +### What was done +- Modified `render_topk` in `top_k.rs` to handle columnar-only inputs. +- Before calling `as_specific_collection(None, ...)`, checks if the bundle has only columnar (no Vec). If so, calls `ensure_vec_collection()` to convert. + +### Key decisions +- Same boundary conversion pattern as Reduce (6.1), FlatMap (4.1), and ensure_collections (5.1). + +### Files changed +- `src/compute/src/render/top_k.rs` — Added columnar→Vec conversion guard. + +### Issues +- None. + +## Prompt 6.3: Columnar Threshold input (verification only) + +### What was done +- Reviewed `render_threshold` and `build_threshold_basic` in `threshold.rs`. +- Confirmed that threshold operates entirely on arrangements via `input.arrangement(&key)` (line 84-86). It never accesses `self.collection` or calls `as_specific_collection`. +- No code changes needed. Columnar-only bundles work correctly because arrangements are stored in the separate `arranged` field of `CollectionBundle`. + +### Key decisions +- Verification-only prompt. The threshold operator is arrangement-native and does not interact with unarranged collections at all. + +### Files changed +- None (verification only). + +### Issues +- None. + +## Prompt 7.1: Columnar Linear Join input + +### What was done +- Modified `render_join_inner` in `linear_join.rs` to handle columnar-only inputs in the fallback path (no matching arrangement or initial closure present). +- Before calling `as_specific_collection`, checks if the source bundle has only columnar (no Vec) and `source_key` is None (no arrangement path). If so, clones the bundle, calls `ensure_vec_collection()`, and uses the converted bundle. +- Join stages continue to operate on arrangements unchanged. The columnar→Vec conversion only affects the initial collection extraction. + +### Key decisions +- Since `inputs` is borrowed by index (`&inputs[linear_plan.source_relation]`), we clone when conversion is needed rather than mutating in-place. The clone is cheap (stream handles are reference-counted). +- Only convert when `source_key` is None, because when a source_key is present, `as_specific_collection` uses the arrangement path which doesn't need `self.collection`. +- Did not implement columnar output accumulation for `LinearJoinImpl` — join stages inherently produce row-at-a-time output via closure application, and the existing Vec output path is sufficient. Columnar output can be added when downstream operators specifically benefit. + +### Files changed +- `src/compute/src/render/join/linear_join.rs` — Added columnar→Vec conversion guard in the fallback path of `render_join_inner`. + +### Issues +- None. + +## Prompt 7.2: Columnar Delta Join output + +### What was done +- Modified `render_delta_join` in `delta_join.rs` to produce columnar output when any input has a columnar collection. +- Delta joins operate entirely on arrangements for both inputs and join stages, so no input conversion is needed. +- At the output boundary, if any input had columnar, converts the Vec ok collection to columnar via `vec_to_columnar`. + +### Key decisions +- Used "any input has columnar" as the heuristic for producing columnar output, maintaining columnar flow through the dataflow graph. +- No changes to join internals — closure application is inherently per-result-tuple and stays Vec-based. + +### Files changed +- `src/compute/src/render/join/delta_join.rs` — Added columnar output conversion at the end of `render_delta_join`. + +### Issues +- None. + +## Prompt 8.1: Columnar sink input + +### What was done +- Modified `export_sink` in `sinks.rs` to handle columnar-only bundles at the sink boundary. +- Added a new branch: when `bundle.collection` is None but `bundle.columnar_collection` is Some, clones the bundle, calls `ensure_vec_collection()`, and uses the resulting Vec collection. +- The sink/persist export boundary requires Row-format data, so columnar→Vec conversion is necessary here. + +### Key decisions +- Sinks are the terminal boundary of the dataflow — no need to produce columnar output. The conversion to Vec is the correct behavior for writing to persist. +- Clone the bundle rather than mutating in-place since `bundle` is obtained from `lookup_id` which returns an owned value but the code structure requires the `if let` pattern. +- The arrangement fallback path (else branch) remains unchanged for bundles that have neither collection nor columnar. + +### Files changed +- `src/compute/src/render/sinks.rs` — Added columnar→Vec conversion branch in `export_sink`. + +### Issues +- None. + +## Prompt 9.1: Audit and remove dead Vec paths + +### What was done +- **Audit**: Identified all 19 `.collection` accesses across render modules. Found that all operators now handle columnar-only bundles via `ensure_vec_collection()` guards (Reduce, TopK, FlatMap, Linear Join, Sinks, ensure_collections). +- **Logging**: Added `tracing::debug!` in `ensure_vec_collection` to track when the Vec fallback is used at runtime. +- **Removed redundant Vec from persist source imports**: Both import loops (recursive and non-recursive) now produce columnar-only bundles via `from_columnar_collections` instead of setting both Vec and columnar. The Vec stream is still created (needed as input to `vec_to_columnar`) but not retained in the bundle. +- **Removed redundant Vec from Constant operator**: Now produces columnar-only bundle instead of both Vec and columnar. +- **Fixed assertion**: Updated `Get::PassArrangements` assertion to accept columnar collections as valid raw collections (line 1244: `keys.raw <= (collection.is_some() || columnar_collection.is_some())`). + +### Key decisions +- Source operators (persist, constant) now produce **columnar-only** bundles. All downstream operators already handle this via `ensure_vec_collection()` guards added in earlier prompts. +- The `collection` field in `CollectionBundle` is NOT removed yet — it's still needed as a transient state (populated by `ensure_vec_collection()` and used by arrangement creation, sinks, etc.). Removal is Prompt 9.2. +- Used `tracing::debug!` rather than metrics counters for simplicity. This can be upgraded to proper metrics if needed. + +### Files changed +- `src/compute/src/render/context.rs` — Added debug logging in `ensure_vec_collection`. +- `src/compute/src/render.rs` — Removed redundant Vec from persist source imports (both loops) and Constant operator; fixed PassArrangements assertion. + +### Issues +- The `Get::PassArrangements` assertion `keys.raw <= collection.collection.is_some()` would have panicked with columnar-only bundles. Fixed to also accept `columnar_collection.is_some()`. + +## Prompt 9.2: Remove `collection` field and `ensure_vec_collection` + +### What was done +- **Removed `collection` field** from `CollectionBundle` struct. Data now flows exclusively through `columnar_collection`. +- **Removed `ensure_vec_collection`** method entirely. +- **Modified `from_collections`** to automatically convert Vec→columnar via `vec_to_columnar`, so callers producing Vec output (TopK, Linear Join, LetRec, etc.) seamlessly convert to columnar. +- **Added `as_vec_collection`** method that converts columnar→Vec on demand, replacing the role of `ensure_vec_collection` + `.collection.clone()`. +- **Updated `as_specific_collection(None)`** and `flat_map(None, ...)` to use `as_vec_collection()` internally. +- **Simplified `as_columnar_collection_core`** — no longer needs special-case handling since there's only one collection type. +- **Rewrote `ensure_collections`** (ArrangeBy) to use a local `cached_vec` variable instead of `self.collection` for arrangement creation. After the loop, stores the passthrough as columnar if `collections.raw` is demanded. +- **Updated all callers**: removed `ensure_vec_collection()` calls from flat_map.rs, top_k.rs, reduce.rs, sinks.rs, linear_join.rs. +- **Fixed LetRec** code in render.rs to use `as_vec_collection()` instead of `.collection.unwrap()`. +- **Fixed hydration logging** to operate on `columnar_collection` instead of `collection`. +- **Updated `enter_region`/`leave_region`/`scope`/`update_id`** to only handle `columnar_collection`. + +### Key decisions +- `from_collections` auto-converts Vec→columnar, making the transition invisible to callers. This means operators that produce Vec output (TopK, reduce, join) don't need individual changes. +- `as_vec_collection()` converts on every call (no caching). This is acceptable because the conversion is only used at operator boundaries (arrangements, sinks, etc.) where the cost is amortized. +- The `from_expressions` constructor (arrangement-only bundles) sets `columnar_collection: None`, which is correct since arrangement-only bundles have no unarranged collection. + +### Files changed +- `src/compute/src/render/context.rs` — Struct field removal, method updates, new `as_vec_collection`. +- `src/compute/src/render.rs` — Fixed LetRec, PassArrangements assertion, hydration logging, debug prints. +- `src/compute/src/render/flat_map.rs` — Removed `ensure_vec_collection` call. +- `src/compute/src/render/top_k.rs` — Removed `ensure_vec_collection` call. +- `src/compute/src/render/reduce.rs` — Removed `ensure_vec_collection` call. +- `src/compute/src/render/sinks.rs` — Simplified to use `as_vec_collection`. +- `src/compute/src/render/join/linear_join.rs` — Removed `ensure_vec_collection` guard. + +### Issues +- The LetRec code (`render.rs` ~line 956, 1007) directly accessed `.collection.unwrap()` — needed conversion to `as_vec_collection()`. +- Hydration logging (`render.rs` ~line 1481) modified `.collection` in-place — updated to modify `columnar_collection` instead. +- `ensure_collections` loop used `self.collection.take()`/`self.collection = Some(...)` pattern — replaced with local `cached_vec` variable. + +## Prompt 10.1: Investigate columnar arrangement spines (research/design) + +### Current Architecture +- **`RowRowSpine`** (`row_spine.rs:35`): `Spine>>>` — uses `DatumContainer` for both keys and values. +- **`DatumContainer`** (`row_spine.rs:97-125`): Wraps `BytesContainer` which stores rows as concatenated byte sequences with offset-based indexing (`OffsetOptimized`). Each row's bytes are the same format as `Row`'s internal representation. +- **`DatumSeq`** (`row_spine.rs:205-287`): Borrowed view into `DatumContainer`. Implements `Iterator>` for sequential datum decoding from raw bytes. +- **`Row` Columnar impl** (`repr/src/row.rs:428-632`): `Row` implements `Columnar` with a `Rows` container (bounds: `Vec` + values: `Vec`). This separates offsets from data but is still **row-oriented** (each row's datums are contiguous). + +### Feasibility of Column-of-Datums Layout + +**Technically feasible but requires significant infrastructure changes:** + +1. **Schema propagation**: Arrangement spines don't know the relation schema. A column-of-datums layout requires knowing column count, types, and nullability. `RelationDesc` would need to be threaded through to `Layout`/`BatchContainer`. + +2. **New `BatchContainer` implementations**: Each column type (Int32, Int64, String, etc.) would need its own contiguous array. This means either: + - A `ColumnBatchContainer` with per-column typed arrays (like Arrow) + - Or a `Vec` approach matching the vectorized eval types from PR #35464 + +3. **Batch merging impact**: Spines merge batches during compaction. Column-oriented merge requires per-column merge logic. The current `BytesContainer` merge is simple byte-copy; columnar merge would need type-aware operations. + +4. **Cursor API compatibility**: The differential-dataflow cursor presents `(Key, Val, Time, Diff)` tuples. Column-oriented storage would need to reconstruct rows for the cursor API, negating some benefits. Alternatively, a columnar cursor extension could provide batch-level column access. + +5. **Variable-length types**: Strings, bytes, and nested types (arrays, maps, JSON) require offset+data separation within each column, adding complexity. + +### Recommendation + +A columnar arrangement spine is a **major undertaking** with prerequisites: +- Vectorized evaluation (Prompt 3.2 / PR #35464) must land first to provide `DatumColumn`/`ColumnDatum` types +- Schema propagation from the optimizer through to arrangements +- A new `ColumnBatchContainer` implementing differential-dataflow's `BatchContainer` trait +- Modified merge and cursor logic + +The current `DatumContainer` is already reasonably efficient (contiguous bytes, no per-row allocation, offset-based indexing). The biggest win from columnar spines would be **avoiding the Row→Column transpose at evaluation time**, which benchmarks show costs ~58μs/1024 rows. This savings only materializes when vectorized evaluation is available. + +**Suggested phased approach:** +1. Land vectorized MFP evaluation (Prompt 3.2) +2. Add schema metadata to arrangement creation +3. Prototype a `ColumnarDatumContainer` for a single column type (e.g., Int64) +4. Benchmark against `DatumContainer` for filter-heavy workloads +5. Generalize to all column types + +### Files reviewed (read-only) +- `src/compute/src/row_spine.rs` — `DatumContainer`, `DatumSeq`, `BytesContainer`, `RowRowLayout` +- `src/compute/src/typedefs.rs` — `RowRowSpine`, `RowRowAgent`, type aliases +- `src/repr/src/row.rs` — `Row`, `Rows` (Columnar impl), datum encoding +- `src/timely-util/src/columnar/builder.rs` — `ColumnBuilder` (not used in spines) + +### Issues +- None. Research/design prompt only. + +## Prompt 11.1: Columnar `flat_map` — direct &RowRef processing + +### What was done +- Added a columnar path to the `flat_map` method in `CollectionBundle` (context.rs). +- When `key_val` is `None` and `columnar_collection` is present, the new path iterates the columnar container directly using a bespoke `unary` operator named `ColumnarFlatMap`. +- Each columnar item yields `(&RowRef, T::Ref, Diff::Ref)` via `into_index_iter()`. The `&RowRef` is passed directly to `datums.borrow_with_limit(d, max_demand)` — no owned `Row` is ever allocated. +- Only the timestamp and diff are converted to owned via `Columnar::into_owned` (these are cheap scalar copies). +- The existing Vec fallback is retained for bundles that lack a columnar collection (e.g., arrangement-only bundles). + +### Key decisions +- Used `StreamCore::unary` with `CapacityContainerBuilder>` as the output builder, matching the return type `StreamVec`. This avoids changing the method signature. +- The `logic` closure signature (`FnMut(&mut DatumVecBorrow, T, Diff) -> I`) is unchanged. Callers (MFP evaluate, Reduce key/value extraction) work without modification because `DatumVecBorrow` is populated from `&RowRef` the same way as from `&Row`. +- The arrangement path (`key_val` is `Some`) is unchanged — it always uses the arrangement's own flat_map. + +### Files changed +- `src/compute/src/render/context.rs` — Added columnar branch in `flat_map` method. + +### Issues +- `unary` takes ownership of the stream, requiring `.clone()` on `col_oks.inner`. Stream clones are cheap (reference-counted handles). + +## Prompt 11.2: Columnar `as_specific_collection` (identity path) + +### What was done +- Added `as_specific_columnar_collection` method on `CollectionBundle` that returns `(ColumnarCollection, VecCollection)`. +- When `key` is `None`, returns the columnar collection directly by cloning the handles — no conversion at all. +- When `key` is `Some`, delegates to `as_specific_collection` (arrangement path) and converts the result to columnar. +- Optimized `as_columnar_collection_core` to detect identity MFPs and use `as_specific_columnar_collection` directly, eliminating the columnar→Vec→columnar round-trip. + +### Key decisions +- Added a new method rather than changing `as_specific_collection`'s return type, since many callers need `VecCollection` and changing the signature would be a larger refactor. +- The identity MFP detection in `as_columnar_collection_core` mirrors the same logic in `as_collection_core` (check `mfp_plan.is_identity() && !has_key_val`). +- The arrangement path (`key` is `Some`) still converts Vec→columnar since arrangement output is inherently Vec-based. + +### Files changed +- `src/compute/src/render/context.rs` — Added `as_specific_columnar_collection` method; optimized `as_columnar_collection_core` identity path. + +### Issues +- None. + +## Prompt 11.3: Columnar `as_collection_core` (MFP path) — verification + +### What was done +- Verified that `as_columnar_collection_core` no longer needs the Vec round-trip for identity MFPs (handled by 11.2's `as_specific_columnar_collection`). +- Verified that for non-identity MFPs, `flat_map` (11.1) iterates columnar data directly via `&RowRef` without allocating owned Rows. The output remains Vec-based due to `map_fallible`'s Ok/Err split, with a final `vec_to_columnar` conversion. +- Updated the doc comment on `as_columnar_collection_core` to accurately describe the current behavior. + +### Key decisions +- No further code changes needed beyond updating documentation. The Vec→columnar conversion on the non-identity MFP output path is inherent to `map_fallible` producing Vec and cannot be avoided without rewriting the Ok/Err split to produce columnar output directly. +- The important optimization (avoiding owned Row allocation) is already achieved by 11.1's columnar `flat_map`. + +### Files changed +- `src/compute/src/render/context.rs` — Updated doc comment on `as_columnar_collection_core`. + +### Issues +- None. Verification-only prompt. + +## Prompt 11.4: Columnar Reduce input (direct) — verification + +### What was done + +- Verified that `render_reduce` calls `entered.flat_map(input_key.map(|k| (k, None)), max_demand, ...)`. +- When `input_key` is `None`, `flat_map` uses the columnar path from 11.1 — iterating `&RowRef` directly without Vec conversion. +- When `input_key` is `Some`, `flat_map` uses the arrangement path (no collection conversion needed). +- The logic closure receives `DatumVecBorrow` populated from `&RowRef` for key/value expression evaluation. No owned Row allocation. + +### Key decisions +- No code changes needed. Reduce automatically benefits from 11.1's columnar `flat_map`. + +### Files changed +- None (verification only). + +### Issues +- None. + +## Prompt 11.5: Columnar FlatMap input (direct) + +### What was done +- Added a columnar path to `render_flat_map` that uses `unary_fallible` directly on the columnar inner stream (`Column<(Row, T, Diff)>`). +- `unary_fallible` accepts `Column<...>` because `Column` implements `Container + DrainContainer + Clone + Default`. +- The inner loop iterates columnar items via `data.borrow().into_index_iter()`, yielding `(&RowRef, T::Ref, Diff::Ref)`. The `&RowRef` is passed directly to `datums.borrow_with(row_ref)` for expression evaluation and to `drain_through_mfp(row_ref, ...)` for MFP application. +- Changed `drain_through_mfp` parameter from `&Row` to `&RowRef` (transparent since `Row: Deref`). +- The queue buffers `Column<...>` containers instead of `Vec<...>` containers. +- Vec fallback retained for arrangement key paths and non-columnar bundles. + +### Key decisions +- Created a full parallel columnar path rather than trying to make the existing code generic, because the iteration patterns differ (`for (row, t, d) in data` for Vec vs `for (ref, t_ref, d_ref) in data.borrow().into_index_iter()` for columnar). +- The columnar path does not use `'input` labeled break since columnar iteration doesn't yield owned items that can be pattern-matched the same way. Uses `continue` instead. +- Output is still Vec-based (`ConsolidatingContainerBuilder>`) for the ok/err streams, with a final `vec_to_columnar` conversion. The inner table function evaluation inherently produces owned Rows. + +### Files changed +- `src/compute/src/render/flat_map.rs` — Added columnar `unary_fallible` path; changed `drain_through_mfp` to accept `&RowRef`. + +### Issues +- `col_errs` needed `.clone()` for `concat` since it's behind a shared reference. + +## Prompt 11.6: Columnar ArrangeBy input (direct) + +### What was done +- Added `arrange_columnar_collection` method that takes `ColumnarCollection` and iterates `&RowRef` directly from columnar containers for key/value expression evaluation. +- The method mirrors `arrange_collection` but: iterates via `data.borrow().into_index_iter()` yielding `(&RowRef, T::Ref, Diff::Ref)`, passes `&RowRef` to `datums.borrow_with(row_ref)`, and produces a columnar passthrough via `ColumnBuilder<(Row, S::Timestamp, Diff)>`. +- Modified `ensure_collections` to detect when identity MFP + no input_key + columnar available, and use `arrange_columnar_collection` directly instead of converting columnar→Vec via `as_collection_core`. +- The columnar path tracks a `cached_col: Option<(ColumnarCollection, VecCollection)>` through the arrangement loop, keeping the passthrough columnar throughout. +- The existing Vec fallback path is retained for non-identity MFPs and arrangement-key cases. + +### Key decisions +- Only use the columnar direct path when MFP is identity and `input_key` is None. When MFP is non-identity, `as_collection_core` → `flat_map` already uses the columnar flat_map path from 11.1. +- The passthrough stream is columnar (`ColumnBuilder<(Row, T, Diff)>`), forwarding each item individually. This is slightly less efficient than the Vec path's `give_container` (which forwards entire containers), but avoids a columnar→Vec→columnar round-trip for subsequent arrangements. +- Key/value expression evaluation produces owned `Row`s via `key_buf.packer()` / `val_buf.packer()` — this is inherent to the arrangement format and unaffected by the input representation. + +### Files changed +- `src/compute/src/render/context.rs` — Added `arrange_columnar_collection` method; modified `ensure_collections` to prefer columnar path. + +### Issues +- None. diff --git a/log.md b/log.md new file mode 100644 index 0000000000000..213a58bd4d564 --- /dev/null +++ b/log.md @@ -0,0 +1,28 @@ +# Columnar Rendering: Work Log + +| # | Prompt | Status | Date | +|---|--------|--------|------| +| 1 | 0.1: Introduce ColumnarCollection type alias and ColumnarBundle | Done | 2026-03-24 | +| 2 | 0.2: Columnar ↔ Vec conversion utilities | Done | 2026-03-24 | +| 3 | 1.1: Persist source emits columnar collections | Done | 2026-03-24 | +| 4 | 2.1: Negate operator propagates columnar collections | Done | 2026-03-24 | +| 5 | 2.2: Union operator propagates columnar collections | Done | 2026-03-24 | +| 6 | 2.3: Constant operator emits columnar collections | Done | 2026-03-25 | +| 7 | 3.1: Columnar as_collection_core with Get/Mfp wiring | Done | 2026-03-25 | +| 8 | 4.1: Columnar FlatMap accepts columnar input, emits columnar output | Done | 2026-03-25 | +| 9 | 5.1: Columnar input to arrangements via ensure_collections | Done | 2026-03-25 | +| 10 | 6.1: Columnar Reduce input via ensure_vec_collection | Done | 2026-03-25 | +| 11 | 6.2: Columnar TopK input via ensure_vec_collection | Done | 2026-03-25 | +| 12 | 6.3: Columnar Threshold input — verified, no changes needed | Done | 2026-03-25 | +| 13 | 7.1: Columnar Linear Join input via ensure_vec_collection | Done | 2026-03-25 | +| 14 | 7.2: Columnar Delta Join output when inputs have columnar | Done | 2026-03-25 | +| 15 | 8.1: Columnar sink input via ensure_vec_collection | Done | 2026-03-25 | +| 16 | 9.1: Audit Vec paths; remove redundant Vec from sources; add logging | Done | 2026-03-25 | +| 17 | 9.2: Remove collection field and ensure_vec_collection from CollectionBundle | Done | 2026-03-25 | +| 18 | 10.1: Investigate columnar arrangement spines — research complete | Done | 2026-03-25 | +| 19 | 11.1: Columnar flat_map — iterate &RowRef directly without Vec conversion | Done | 2026-03-25 | +| 20 | 11.2: Columnar as_specific_collection — identity path returns columnar directly | Done | 2026-03-25 | +| 21 | 11.3: Columnar as_collection_core — verified, identity avoids round-trip | Done | 2026-03-25 | +| 22 | 11.4: Columnar Reduce input — verified, flat_map uses &RowRef directly | Done | 2026-03-25 | +| 23 | 11.5: Columnar FlatMap — direct &RowRef processing via unary_fallible on Column | Done | 2026-03-25 | +| 24 | 11.6: Columnar ArrangeBy — arrange_columnar_collection iterates &RowRef directly | Done | 2026-03-25 | diff --git a/prompts.md b/prompts.md new file mode 100644 index 0000000000000..8ddee898b0390 --- /dev/null +++ b/prompts.md @@ -0,0 +1,446 @@ +# Columnar Rendering: Incremental Migration Plan + +## Overview + +Convert Materialize's rendering layer from row-first (`Vec<(Row, T, Diff)>`) to column-first +representation using the `columnar` crate. The goal is to reduce pointer chasing, improve cache +locality, and enable vectorized scalar evaluation. + +### Architecture + +**Current state**: Data flows as `VecCollection` between operators. Each `Row` is a +separate allocation. Arrangements use `DatumContainer` (dictionary-compressed row bytes). +`CollectionBundle` wraps both unarranged collections and arrangements keyed by expression. + +**Target state**: Unarranged edges carry columnar containers (`Column<(Row, T, Diff)>` or a +future column-of-datums layout). Arrangements remain as-is initially (they already use +`DatumContainer`). Operators that apply scalar expressions can optionally use vectorized +evaluation when data is in columnar form. + +### Constraints + +- Each step must compile cleanly (`bin/lint`, `cargo clippy`, `cargo test`). +- Do not modify `src/storage` unless absolutely needed. +- Arrangements (Row→Row spines) are already reasonably efficient; focus on unarranged edges first. +- The vectorized evaluation PR (#35464) defines `ColumnDatum`/`DatumColumn` as the evaluation-time + layout. We don't need the same layout for edges, but conversion must be cheap. + +### Key types + +| Type | Location | Role | +|------|----------|------| +| `CollectionBundle` | `render/context.rs` | Edge between operators: optional collection + arrangements | +| `ArrangementFlavor` | `render/context.rs` | Local or imported arrangement (RowRow spine) | +| `VecCollection` | differential_dataflow | Unarranged data stream (row-first) | +| `Column` | `timely-util/src/columnar.rs` | Columnar container (typed/bytes/aligned) | +| `Col2ValBatcher` | `timely-util/src/columnar.rs` | Columnar merge batcher | +| `ColumnBuilder` | `timely-util/src/columnar/builder.rs` | Builds columnar containers | +| `ColumnDatum` / `DatumColumn` | PR #35464 `expr/src/vectorized.rs` | Vectorized eval layout | + +--- + +## Phase 0: Foundation + +### Prompt 0.1: Introduce `ColumnarCollection` type alias and `ColumnarBundle` + +[*] Introduce a type alias `ColumnarCollection` for a collection backed by columnar +containers (`StreamCore>` or equivalent). + +[*] Add a `columnar_collection` field to `CollectionBundle` alongside the existing +`collection` field. Initially always `None`. + +[*] Add helper methods: `from_columnar_collections(oks, errs)` and `columnar_collection()` that +return the columnar variant if present, falling back to converting the Vec variant. + +[*] Add a method `ensure_vec_collection()` that materializes the `VecCollection` from the columnar +collection if needed (the "escape hatch" for operators not yet converted). + +[*] Ensure all existing code continues to compile and pass tests unchanged. The new field is +always `None` at this point. + +**Files**: `src/compute/src/render/context.rs`, `src/compute/src/typedefs.rs` + +--- + +### Prompt 0.2: Columnar ↔ Vec conversion utilities + +[*] Implement `vec_to_columnar` and `columnar_to_vec` stream operators that convert between +`VecCollection` and the columnar equivalent. + +[*] The `vec_to_columnar` operator should batch rows into columnar containers using +`ColumnBuilder`. Use a configurable batch size (default 1024 or container-size-driven). + +[*] The `columnar_to_vec` operator should iterate the columnar container and emit individual +`(Row, T, Diff)` tuples. + +[*] Add unit tests that round-trip data through both conversions. + +**Files**: `src/compute/src/render/context.rs` or a new `src/compute/src/render/columnar.rs` + +--- + +## Phase 1: Source ingestion as columnar + +### Prompt 1.1: Persist source emits columnar collections + +[*] Investigate `persist_source::persist_source()` in `src/storage-operators/src/persist_source.rs`. +It already produces batches of rows. Determine whether it can emit `Column<(Row, T, Diff)>` +directly or whether a `vec_to_columnar` conversion at the boundary is more practical. + +[*] If the persist source already produces columnar-friendly batches (it uses `ColumnBuilder` in +PR #35464), wire the columnar output into the `imported_sources` in `render.rs` so that +`CollectionBundle` carries the columnar collection. + +[*] Ensure that all downstream operators still work by having `ensure_vec_collection()` as the +fallback. Run tests. + +**Files**: `src/compute/src/render.rs`, `src/storage-operators/src/persist_source.rs` (read-only +if possible) + +--- + +## Phase 2: Simple pass-through operators + +### Prompt 2.1: Negate + +[*] Convert the `Negate` operator to propagate columnar collections. Negation flips the sign of +`Diff`, which can be done in-place on a columnar container without unpacking rows. + +[*] If the input `CollectionBundle` has a columnar collection, produce a columnar output. Otherwise +fall back to the existing Vec path. + +**Files**: `src/compute/src/render.rs` (the Negate match arm in `render_plan_expr`) + +--- + +### Prompt 2.2: Union + +[*] Convert the `Union` operator to propagate columnar collections. Union concatenates streams, +which works identically for columnar containers. + +[*] If all inputs have columnar collections, produce a columnar output. If some inputs are Vec, +either convert them or fall back to Vec for all. + +**Files**: `src/compute/src/render.rs` (the Union match arm) + +--- + +### Prompt 2.3: Constant + +[*] Convert the `Constant` operator to emit columnar collections. Constants are small, so this +is primarily for uniformity. + +[*] Pack the constant rows into a columnar container and set the `columnar_collection` field. + +**Files**: `src/compute/src/render.rs` (the Constant match arm) + +--- + +## Phase 3: MFP (Map/Filter/Project) + +### Prompt 3.1: Columnar `as_collection_core` + +[*] This is the core method that applies `MapFilterProject` to collections or arrangements and +produces a `VecCollection`. Add a columnar variant `as_columnar_collection_core` that: + - Accepts columnar input + - For now, converts to Vec internally and applies the existing MfpPlan row-at-a-time + - Returns a columnar collection + - This is the incremental step; vectorized eval comes later + +[*] Wire `render_plan_expr` for `Get` and `Mfp` to prefer the columnar path when available. + +**Files**: `src/compute/src/render/context.rs` + +--- + +### Prompt 3.2: Vectorized MFP evaluation on columnar data + +[ ] Integrate the vectorized evaluation from PR #35464 into `as_columnar_collection_core`. +When the `MfpPlan` is suitable for vectorized evaluation (non-temporal, supported expression +types): + - Convert columnar `Row` batches to `Vec` (the `rows_to_columns` function) + - Evaluate using `MfpPlan::evaluate_batch` + - Convert results back to columnar `Row` containers + +[ ] Add a dyncfg flag to enable/disable vectorized evaluation, defaulting to off. + +[ ] Keep the row-at-a-time path as fallback for unsupported expressions. + +**Files**: `src/compute/src/render/context.rs`, `src/expr/src/vectorized.rs` + +--- + +## Phase 4: FlatMap + +### Prompt 4.1: Columnar FlatMap + +[*] Convert `render_flat_map` to accept columnar input. Since `FlatMap` applies table functions +that can produce variable numbers of output rows per input row, the output is naturally a +stream and may not benefit from columnar representation on the output side. + +[*] Accept columnar input, convert batch-at-a-time for table function evaluation, and emit +columnar output if practical. + +**Files**: `src/compute/src/render/flat_map.rs` + +--- + +## Phase 5: Arrangement creation (ArrangeBy) + +### Prompt 5.1: Columnar input to arrangements + +[*] The `ensure_collections` method in `CollectionBundle` arranges data by key expressions. It +currently calls `as_specific_collection` to get a `VecCollection` and then arranges it. + +[*] When columnar input is available, use it to feed into arrangement creation. The arrangement +spines (`RowRowSpine`) use `DatumContainer` which accepts `Row` input, so we need to unpack +columnar → Row at this boundary. + +[*] This is primarily a plumbing step: accept columnar, unpack to Row for arrangement. + +**Files**: `src/compute/src/render/context.rs`, `src/compute/src/extensions/arrange.rs` + +--- + +## Phase 6: Stateful operators + +### Prompt 6.1: Columnar Reduce input + +[*] `render_reduce` currently calls `flat_map` to selectively unpack demanded columns and +evaluate key/value plans. Convert to accept columnar input. + +[*] The reduce operator must create arrangements from its output. The key extraction and +aggregation logic operates row-at-a-time for now; the columnar input is unpacked at the +operator boundary. + +**Files**: `src/compute/src/render/reduce.rs` + +--- + +### Prompt 6.2: Columnar TopK input + +[*] `render_top_k` calls `as_specific_collection` to get a Vec collection. Convert to accept +columnar input, unpacking at the operator boundary. + +**Files**: `src/compute/src/render/top_k.rs` + +--- + +### Prompt 6.3: Columnar Threshold input + +[*] Threshold works directly on arrangements (`arrangement(&key)`), not on unarranged +collections. No changes needed for the main path. + +[*] Verify that threshold continues to work when upstream operators produce columnar collections +(it should, since it only uses arrangements). + +**Files**: `src/compute/src/render/threshold.rs` (verification only) + +--- + +## Phase 7: Joins + +### Prompt 7.1: Columnar Linear Join input + +[*] `render_linear_join` gets its initial input via `as_specific_collection`. Convert to accept +columnar input for the initial collection. + +[*] Join stages operate on arrangements, which remain unchanged. The initial closure application +and the final output can use columnar. + +[*] Update `LinearJoinImpl` to support columnar output accumulation. + +**Files**: `src/compute/src/render/join/linear_join.rs` + +--- + +### Prompt 7.2: Columnar Delta Join input + +[*] Delta joins build update streams from arrangements. The join output is accumulated in +collections. Convert the output path to produce columnar collections. + +[*] The join closure application happens per-result-tuple and may not benefit from columnar +within the join itself, but the output stream should be columnar. + +**Files**: `src/compute/src/render/join/delta_join.rs` + +--- + +## Phase 8: Sinks and exports + +### Prompt 8.1: Columnar sink input + +[*] Sinks receive data via `as_collection_core`. Convert to accept columnar input. + +[*] The sink export boundary (writing to persist) may need Row-format data. Unpack columnar +at this boundary. + +**Files**: `src/compute/src/render/sinks.rs` + +--- + +## Phase 9: Remove Vec fallbacks + +### Prompt 9.1: Audit and remove dead Vec paths + +[*] Once all operators produce and consume columnar collections, the `collection` field in +`CollectionBundle` (the Vec variant) should be unused for the unarranged data path. + +[*] Add metrics/logging to track how often the Vec fallback is used. + +[*] Gradually remove the Vec paths, starting with operators where the columnar path is proven +stable. + +**Files**: All render files + +--- + +### Prompt 9.2: Remove `ensure_vec_collection` calls + +[*] Once all operators are converted, `ensure_vec_collection` should have zero callers. +Remove it and the `collection` field from `CollectionBundle`. + +**Files**: `src/compute/src/render/context.rs`, all render files + +--- + +## Phase 10: Columnar-native arrangements (future) + +### Prompt 10.1: Investigate columnar arrangement spines + +[*] The current `RowRowSpine` uses `DatumContainer` for dictionary-compressed row bytes. This is +already reasonably cache-friendly but does not enable vectorized access by column. + +[*] Investigate whether arrangement spines can store data in a column-of-datums layout for +direct vectorized evaluation from arrangements without materializing collections. + +[*] This is a research/design step, not an implementation step. + +**Files**: `src/compute/src/row_spine.rs`, `src/compute/src/typedefs.rs` + +--- + +## Phase 11: Direct columnar processing (eliminate Vec escape hatches) + +The columnar `Ref<'_, Row>` is `&RowRef`, and `DatumVec::borrow_with` / `borrow_with_limit` +already accept `&RowRef`. Operators that unpack rows via `DatumVec` can iterate columnar +containers directly without materializing owned `Row` values. + +### Prompt 11.1: Columnar `flat_map` in `CollectionBundle` + +[*] The `flat_map` method in `CollectionBundle` (context.rs) is the core building block used by +MFP, Reduce, and other operators. When `key_val` is `None`, it currently calls +`as_vec_collection()` to get a `VecCollection`, then iterates `(Row, T, Diff)` tuples. + +[*] Add a columnar path: when `columnar_collection` is present and `key_val` is `None`, iterate +the columnar container directly using `into_index_iter()`. Each item yields `(&RowRef, &T, &Diff)`. +Pass `&RowRef` directly to `datums.borrow_with_limit(row_ref, max_demand)` — this already works +since `borrow_with_limit` accepts `&RowRef`. + +[*] The `logic` closure signature uses `DatumVecBorrow<'_>` which is populated from `&RowRef`, +so no changes to callers (MFP evaluate, Reduce key/value extraction) are needed. + +**Files**: `src/compute/src/render/context.rs` + +--- + +### Prompt 11.2: Columnar `as_specific_collection` (identity path) + +[*] `as_specific_collection(None)` currently calls `as_vec_collection()` which converts the +entire columnar stream to Vec. For the identity case (no arrangement key), this is pure overhead. + +[*] When the bundle has a columnar collection, return it directly as a `ColumnarCollection` +(or convert to Vec only at the caller boundary). This requires either changing the return type +to be generic over container, or providing a separate `as_specific_columnar_collection` method. + +[*] The callers that use this for identity passthrough (e.g., `as_collection_core` when MFP is +identity) should prefer the columnar variant. + +**Files**: `src/compute/src/render/context.rs` + +--- + +### Prompt 11.3: Columnar `as_collection_core` (MFP path) + +[*] `as_collection_core` calls `flat_map` (converted in 11.1) and then applies +`map_fallible` to split Ok/Err. With 11.1 done, this method already processes columnar +data without the Vec escape hatch when `key_val` is `None`. + +[*] For the identity MFP case, it currently calls `as_specific_collection` (converted in 11.2). +Wire this to return columnar directly. + +[*] Verify that `as_columnar_collection_core` no longer needs the Vec round-trip. + +**Files**: `src/compute/src/render/context.rs` + +--- + +### Prompt 11.4: Columnar Reduce input (direct) + +[*] `render_reduce` calls `flat_map` on the entered bundle for key/value extraction. With 11.1 +done, this already processes columnar data directly — the `flat_map` logic closure receives +`DatumVecBorrow` populated from `&RowRef`. + +[*] Verify that Reduce works end-to-end with columnar input without any Vec conversion. + +**Files**: `src/compute/src/render/reduce.rs` + +--- + +### Prompt 11.5: Columnar FlatMap input (direct) + +[*] `render_flat_map` calls `as_specific_collection` to get a Vec collection. With 11.2 done, +investigate whether the FlatMap inner loop can operate on columnar refs directly. + +[*] The FlatMap inner loop unpacks `input_row` via `datums.borrow_with(&input_row)` and evaluates +expressions. Since `borrow_with` accepts `&RowRef`, the loop body can work on columnar refs. +However, the `drain_through_mfp` helper also takes `&Row` — update it to accept `&RowRef`. + +[*] The FlatMap operator uses `unary_fallible` which expects a specific input container type. +Investigate whether it can accept `Column<(Row, T, Diff)>` directly or whether a columnar-aware +operator variant is needed. + +**Files**: `src/compute/src/render/flat_map.rs` + +--- + +### Prompt 11.6: Columnar ArrangeBy input (direct) + +[*] `arrange_collection` takes a `VecCollection` and iterates `(row, time, diff)` +to evaluate key/value expressions. The inner loop uses `datums.borrow_with(row)`. + +[*] Add a columnar variant `arrange_columnar_collection` that iterates `Column<(Row, T, Diff)>` +directly. Each `&RowRef` from the columnar container can be passed to `borrow_with` for +expression evaluation. The key/value `Row`s are built by `SharedRow::pack()` (owned output), +and the arrangement batcher accepts `((Row, Row), T, Diff)` — these owned outputs are unaffected. + +[*] Wire `ensure_collections` to prefer the columnar path when `columnar_collection` is present. + +**Files**: `src/compute/src/render/context.rs` + +--- + +## Notes + +### Conversion cost awareness + +The PR #35464 benchmarks show: +- Row→Column transpose: ~58μs per 1024 rows +- Column→Row packing: ~10μs per 1024 rows +- Vectorized arithmetic: ~0.7μs per 1024 rows + +The conversion cost dominates. The strategy is: +1. First, get columnar containers flowing through edges (even if Row-packed inside) +2. Then, reduce conversions by keeping data columnar across operator boundaries +3. Finally, enable vectorized evaluation where conversion is amortized + +### Arrangement boundary + +Arrangements remain Row-based throughout this plan. They use `DatumContainer` which stores +row bytes contiguously with dictionary compression. Converting arrangements to true columnar +is Phase 10 (future work) and depends on changes to differential-dataflow spine infrastructure. + +### Error streams + +Error streams (`VecCollection`) are kept as Vec throughout. Errors are +rare and the `DataflowError` type is not amenable to columnar representation. diff --git a/src/compute/src/render.rs b/src/compute/src/render.rs index 3ba65294ba9ce..08e36074bc690 100644 --- a/src/compute/src/render.rs +++ b/src/compute/src/render.rs @@ -166,6 +166,7 @@ use crate::render::continual_task::ContinualTaskCtx; use crate::row_spine::{DatumSeq, RowRowBatcher, RowRowBuilder}; use crate::typedefs::{ErrBatcher, ErrBuilder, ErrSpine, KeyBatcher, MzTimestamp}; +pub(crate) mod columnar; pub mod context; pub(crate) mod continual_task; mod errors; @@ -375,10 +376,11 @@ pub fn build_compute_dataflow( ); for (id, (oks, errs)) in imported_sources.into_iter() { - let bundle = crate::render::CollectionBundle::from_collections( - oks.enter(region), - errs.enter(region), - ); + let oks_entered = oks.enter(region); + let errs_entered = errs.enter(region); + let columnar_oks = crate::render::columnar::vec_to_columnar(oks_entered); + let bundle = + CollectionBundle::from_columnar_collections(columnar_oks, errs_entered); // Associate collection bundle with the source identifier. context.insert_id(id, bundle); } @@ -485,10 +487,11 @@ pub fn build_compute_dataflow( } else { oks }; - let bundle = crate::render::CollectionBundle::from_collections( - oks.enter_region(region), - errs.enter_region(region), - ); + let oks_entered = oks.enter_region(region); + let errs_entered = errs.enter_region(region); + let columnar_oks = crate::render::columnar::vec_to_columnar(oks_entered); + let bundle = + CollectionBundle::from_columnar_collections(columnar_oks, errs_entered); // Associate collection bundle with the source identifier. context.insert_id(id, bundle); } @@ -751,7 +754,10 @@ where compute_state.traces.set(idx_id, trace); } None => { - println!("collection available: {:?}", bundle.collection.is_none()); + println!( + "columnar_collection available: {:?}", + bundle.columnar_collection.is_some() + ); println!( "keys available: {:?}", bundle.arranged.keys().collect::>() @@ -847,7 +853,10 @@ where compute_state.traces.set(idx_id, trace); } None => { - println!("collection available: {:?}", bundle.collection.is_none()); + println!( + "columnar_collection available: {:?}", + bundle.columnar_collection.is_some() + ); println!( "keys available: {:?}", bundle.arranged.keys().collect::>() @@ -942,9 +951,8 @@ where let last = rec_iter.peek().is_none(); let binding = BindingInfo::LetRec { id, last }; let bundle = self.render_recursive_plan(object_id, level + 1, value, binding); - // We need to ensure that the raw collection exists, but do not have enough information - // here to cause that to happen. - let (oks, mut err) = bundle.collection.clone().unwrap(); + // Extract the raw collection as Vec for consolidation and variable setting. + let (oks, mut err) = bundle.as_vec_collection(); self.insert_id(Id::Local(id), bundle); let (oks_v, err_v) = variables.remove(&Id::Local(id)).unwrap(); @@ -996,7 +1004,7 @@ where // Now extract each of the rec bindings into the outer scope. for id in rec_ids.into_iter() { let bundle = self.remove_id(Id::Local(id)).unwrap(); - let (oks, err) = bundle.collection.unwrap(); + let (oks, err) = bundle.as_vec_collection(); self.insert_id( Id::Local(id), CollectionBundle::from_collections( @@ -1210,7 +1218,9 @@ where .to_stream(&mut self.scope) .as_collection(); - CollectionBundle::from_collections(ok_collection, err_collection) + // Produce a columnar-only collection for downstream operators. + let columnar_oks = crate::render::columnar::vec_to_columnar(ok_collection); + CollectionBundle::from_columnar_collections(columnar_oks, err_collection) } Get { id, keys, plan } => { // Recover the collection from `self` and then apply `mfp` to it. @@ -1226,7 +1236,7 @@ where .iter() .all(|(key, _, _)| collection.arranged.contains_key(key)) ); - assert!(keys.raw <= collection.collection.is_some()); + assert!(keys.raw <= collection.columnar_collection.is_some()); // Retain only those keys we want to import. collection.arranged.retain(|key, _value| { keys.arranged.iter().any(|(key2, _, _)| key2 == key) @@ -1243,13 +1253,23 @@ where CollectionBundle::from_collections(oks, errs) } mz_compute_types::plan::GetPlan::Collection(mfp) => { - let (oks, errs) = collection.as_collection_core( - mfp, - None, - self.until.clone(), - &self.config_set, - ); - CollectionBundle::from_collections(oks, errs) + if collection.columnar_collection.is_some() { + let (oks, errs) = collection.as_columnar_collection_core( + mfp, + None, + self.until.clone(), + &self.config_set, + ); + CollectionBundle::from_columnar_collections(oks, errs) + } else { + let (oks, errs) = collection.as_collection_core( + mfp, + None, + self.until.clone(), + &self.config_set, + ); + CollectionBundle::from_collections(oks, errs) + } } } } @@ -1262,6 +1282,14 @@ where // If `mfp` is non-trivial, we should apply it and produce a collection. if mfp.is_identity() { input + } else if input.columnar_collection.is_some() { + let (oks, errs) = input.as_columnar_collection_core( + mfp, + input_key_val, + self.until.clone(), + &self.config_set, + ); + CollectionBundle::from_columnar_collections(oks, errs) } else { let (oks, errs) = input.as_collection_core( mfp, @@ -1310,8 +1338,13 @@ where } Negate { input } => { let input = expect_input(input); - let (oks, errs) = input.as_specific_collection(None, &self.config_set); - CollectionBundle::from_collections(oks.negate(), errs) + if let Some((col_oks, col_errs)) = input.columnar_collection() { + let negated = crate::render::columnar::negate_columnar(col_oks.clone()); + CollectionBundle::from_columnar_collections(negated, col_errs.clone()) + } else { + let (oks, errs) = input.as_specific_collection(None, &self.config_set); + CollectionBundle::from_collections(oks.negate(), errs) + } } Threshold { input, @@ -1324,23 +1357,53 @@ where inputs, consolidate_output, } => { - let mut oks = Vec::new(); - let mut errs = Vec::new(); - for input in inputs.into_iter() { - let (os, es) = - expect_input(input).as_specific_collection(None, &self.config_set); - oks.push(os); - errs.push(es); - } - let mut oks = differential_dataflow::collection::concatenate(&mut self.scope, oks); - if consolidate_output { - oks = CollectionExt::consolidate_named::>( - oks, - "UnionConsolidation", - ) + let bundles: Vec<_> = inputs.into_iter().map(expect_input).collect(); + let all_columnar = bundles.iter().all(|b| b.columnar_collection().is_some()); + + if all_columnar { + let mut col_oks = Vec::new(); + let mut col_errs = Vec::new(); + for bundle in &bundles { + let (oks, errs) = bundle.columnar_collection().unwrap(); + col_oks.push(oks.clone()); + col_errs.push(errs.clone()); + } + let col_oks = + differential_dataflow::collection::concatenate(&mut self.scope, col_oks); + let col_errs = + differential_dataflow::collection::concatenate(&mut self.scope, col_errs); + if consolidate_output { + // Consolidation requires Vec-based collections; convert, consolidate, convert back. + let vec_oks = crate::render::columnar::columnar_to_vec(col_oks); + let vec_oks = CollectionExt::consolidate_named::>( + vec_oks, + "UnionConsolidation", + ); + let col_oks = crate::render::columnar::vec_to_columnar(vec_oks); + CollectionBundle::from_columnar_collections(col_oks, col_errs) + } else { + CollectionBundle::from_columnar_collections(col_oks, col_errs) + } + } else { + let mut oks = Vec::new(); + let mut errs = Vec::new(); + for bundle in bundles { + let (os, es) = bundle.as_specific_collection(None, &self.config_set); + oks.push(os); + errs.push(es); + } + let mut oks = + differential_dataflow::collection::concatenate(&mut self.scope, oks); + if consolidate_output { + oks = CollectionExt::consolidate_named::>( + oks, + "UnionConsolidation", + ) + } + let errs = + differential_dataflow::collection::concatenate(&mut self.scope, errs); + CollectionBundle::from_collections(oks, errs) } - let errs = differential_dataflow::collection::concatenate(&mut self.scope, errs); - CollectionBundle::from_collections(oks, errs) } ArrangeBy { input_key, @@ -1409,12 +1472,10 @@ where } } None => { - let (oks, _) = bundle - .collection - .as_mut() - .expect("CollectionBundle invariant"); - let stream = self.log_operator_hydration_inner(oks.inner.clone(), lir_id); - *oks = stream.as_collection(); + if let Some((oks, _)) = bundle.columnar_collection.as_mut() { + let stream = self.log_operator_hydration_inner(oks.inner.clone(), lir_id); + *oks = stream.as_collection(); + } } } } diff --git a/src/compute/src/render/columnar.rs b/src/compute/src/render/columnar.rs new file mode 100644 index 0000000000000..27043b7b75c61 --- /dev/null +++ b/src/compute/src/render/columnar.rs @@ -0,0 +1,417 @@ +// Copyright Materialize, Inc. and contributors. All rights reserved. +// +// Use of this software is governed by the Business Source License +// included in the LICENSE file. +// +// As of the Change Date specified in that file, in accordance with +// the Business Source License, use of this software will be governed +// by the Apache License, Version 2.0. + +//! Conversion and transformation utilities for columnar collections. + +use columnar::{Columnar, Index}; +use differential_dataflow::{AsCollection, VecCollection}; +use mz_repr::{Diff, Row}; +use mz_timely_util::columnar::builder::ColumnBuilder; +use timely::container::CapacityContainerBuilder; +use timely::dataflow::Scope; +use timely::dataflow::channels::pact::Pipeline; +use timely::dataflow::operators::Operator; +use timely::progress::Timestamp; + +use crate::typedefs::{ColumnarCollection, MzTimestamp}; + +/// Convert a `VecCollection` to a `ColumnarCollection`. +/// +/// This operator batches rows into columnar containers using `ColumnBuilder`. +/// The `ColumnBuilder` automatically determines batch sizes based on memory alignment +/// (approximately 2MB per container). +pub fn vec_to_columnar( + vec_collection: VecCollection, +) -> ColumnarCollection +where + S: Scope, + S::Timestamp: MzTimestamp, +{ + vec_collection + .inner + .unary::, _, _, _>( + Pipeline, + "VecToColumnar", + |_cap, _info| { + move |input, output| { + input.for_each(|time, data| { + let mut session = output.session_with_builder(&time); + for (row, time, diff) in data.iter() { + session.give((row, time, diff)); + } + }); + } + }, + ) + .as_collection() +} + +/// Convert a `ColumnarCollection` to a `VecCollection`. +/// +/// This operator iterates columnar containers and emits individual `(Row, T, Diff)` tuples +/// into Vec-based containers. +pub fn columnar_to_vec( + columnar_collection: ColumnarCollection, +) -> VecCollection +where + S: Scope, + S::Timestamp: MzTimestamp, +{ + columnar_collection + .inner + .unary::>, _, _, _>( + Pipeline, + "ColumnarToVec", + |_cap, _info| { + let mut row_buf = Row::default(); + let mut t_buf = S::Timestamp::minimum(); + let mut r_buf = Diff::default(); + move |input, output| { + input.for_each(|time, data| { + let mut session = output.session(&time); + for (d, t, r) in data.borrow().into_index_iter() { + row_buf.copy_from(d); + t_buf.copy_from(t); + r_buf.copy_from(r); + session.give((row_buf.clone(), t_buf.clone(), r_buf.clone())); + } + }); + } + }, + ) + .as_collection() +} + +/// Negate the diffs of a columnar collection, producing a new columnar collection. +/// +/// This is the columnar equivalent of `Collection::negate()`. It iterates the columnar +/// container and produces a new one with each diff negated. +pub fn negate_columnar( + collection: ColumnarCollection, +) -> ColumnarCollection +where + S: Scope, + S::Timestamp: MzTimestamp, +{ + collection + .inner + .unary::, _, _, _>( + Pipeline, + "NegateColumnar", + |_cap, _info| { + let mut r_buf = Diff::default(); + move |input, output| { + input.for_each(|time, data| { + let mut session = output.session_with_builder(&time); + for (d, t, r) in data.borrow().into_index_iter() { + r_buf.copy_from(r); + let neg_r = -r_buf; + session.give((d, t, &neg_r)); + } + }); + } + }, + ) + .as_collection() +} + +#[cfg(test)] +mod tests { + use super::*; + + use std::cell::RefCell; + use std::rc::Rc; + + use differential_dataflow::input::Input; + use mz_repr::{Datum, Diff, Row}; + use timely::dataflow::operators::Inspect; + use timely::dataflow::operators::probe::Probe; + + /// Round-trip data through vec_to_columnar and then columnar_to_vec, + /// verifying that all rows survive the conversion unchanged. + #[mz_ore::test] + fn round_trip_vec_columnar_vec() { + timely::execute_directly(|worker| { + let results: Rc>> = Rc::new(RefCell::new(Vec::new())); + let results_capture = Rc::clone(&results); + + let (mut input, probe) = worker.dataflow::(|scope| { + let (input, collection) = scope.new_collection::(); + + // Convert Vec -> Columnar -> Vec + let columnar = vec_to_columnar(collection); + let round_tripped = columnar_to_vec(columnar); + + let (probe, _stream) = round_tripped + .inner + .inspect(move |item: &(Row, u64, Diff)| { + results_capture + .borrow_mut() + .push((item.0.clone(), item.1, item.2)); + }) + .probe(); + + (input, probe) + }); + + let row1 = Row::pack_slice(&[Datum::Int32(42), Datum::String("hello")]); + let row2 = Row::pack_slice(&[Datum::Int64(100)]); + let row3 = Row::pack_slice(&[Datum::True, Datum::False, Datum::Null]); + let empty_row = Row::default(); + + let one = Diff::from(1); + input.update(row1.clone(), one); + input.update(row2.clone(), one); + input.update(row3.clone(), one); + input.update(empty_row.clone(), one); + input.advance_to(1); + input.flush(); + + worker.step_while(|| probe.less_than(&1)); + + let mut actual = results.borrow().clone(); + actual.sort_by(|a, b| a.0.cmp(&b.0)); + + let mut expected = [ + (row1, 0u64, one), + (row2, 0u64, one), + (row3, 0u64, one), + (empty_row, 0u64, one), + ]; + expected.sort_by(|a, b| a.0.cmp(&b.0)); + + assert_eq!(actual.len(), expected.len(), "Row count mismatch"); + assert_eq!(actual, expected); + }); + } + + /// Verify that vec_to_columnar and then back works with multiple timestamps. + #[mz_ore::test] + fn round_trip_multiple_timestamps() { + timely::execute_directly(|worker| { + let results: Rc>> = Rc::new(RefCell::new(Vec::new())); + let results_capture = Rc::clone(&results); + + let (mut input, probe) = worker.dataflow::(|scope| { + let (input, collection) = scope.new_collection::(); + + let columnar = vec_to_columnar(collection); + let round_tripped = columnar_to_vec(columnar); + + let (probe, _stream) = round_tripped + .inner + .inspect(move |item: &(Row, u64, Diff)| { + results_capture + .borrow_mut() + .push((item.0.clone(), item.1, item.2)); + }) + .probe(); + + (input, probe) + }); + + let row1 = Row::pack_slice(&[Datum::Int32(1)]); + let row2 = Row::pack_slice(&[Datum::Int32(2)]); + + let one = Diff::from(1); + input.update(row1.clone(), one); + input.advance_to(1); + input.update(row2.clone(), one); + input.advance_to(2); + input.flush(); + + worker.step_while(|| probe.less_than(&2)); + + let actual = results.borrow().clone(); + assert_eq!(actual.len(), 2); + assert!(actual.iter().any(|(r, t, _)| *r == row1 && *t == 0)); + assert!(actual.iter().any(|(r, t, _)| *r == row2 && *t == 1)); + }); + } + + /// Verify that negate_columnar flips the sign of all diffs. + #[mz_ore::test] + fn negate_columnar_flips_diffs() { + timely::execute_directly(|worker| { + let results: Rc>> = Rc::new(RefCell::new(Vec::new())); + let results_capture = Rc::clone(&results); + + let (mut input, probe) = worker.dataflow::(|scope| { + let (input, collection) = scope.new_collection::(); + + // Convert to columnar, negate, convert back to Vec for inspection + let columnar = vec_to_columnar(collection); + let negated = negate_columnar(columnar); + let result = columnar_to_vec(negated); + + let (probe, _stream) = result + .inner + .inspect(move |item: &(Row, u64, Diff)| { + results_capture + .borrow_mut() + .push((item.0.clone(), item.1, item.2)); + }) + .probe(); + + (input, probe) + }); + + let row1 = Row::pack_slice(&[Datum::Int32(42)]); + let row2 = Row::pack_slice(&[Datum::String("hello")]); + + input.update(row1.clone(), Diff::from(1)); + input.update(row2.clone(), Diff::from(3)); + input.advance_to(1); + input.flush(); + + worker.step_while(|| probe.less_than(&1)); + + let mut actual = results.borrow().clone(); + actual.sort_by(|a, b| a.0.cmp(&b.0)); + + // Diffs should be negated + assert_eq!(actual.len(), 2); + for (row, _t, diff) in &actual { + if *row == row1 { + assert_eq!(*diff, Diff::from(-1), "row1 diff should be negated"); + } else if *row == row2 { + assert_eq!(*diff, Diff::from(-3), "row2 diff should be negated"); + } else { + panic!("Unexpected row"); + } + } + }); + } + + /// Verify that concatenating columnar collections (union) preserves all rows. + #[mz_ore::test] + fn union_columnar_concatenates() { + timely::execute_directly(|worker| { + let results: Rc>> = Rc::new(RefCell::new(Vec::new())); + let results_capture = Rc::clone(&results); + + let (mut input1, mut input2, probe) = worker.dataflow::(|scope| { + let (input1, collection1) = scope.new_collection::(); + let (input2, collection2) = scope.new_collection::(); + + // Convert both to columnar, concatenate, convert back + let col1 = vec_to_columnar(collection1); + let col2 = vec_to_columnar(collection2); + let union = differential_dataflow::collection::concatenate(scope, vec![col1, col2]); + let result = columnar_to_vec(union); + + let (probe, _stream) = result + .inner + .inspect(move |item: &(Row, u64, Diff)| { + results_capture + .borrow_mut() + .push((item.0.clone(), item.1, item.2)); + }) + .probe(); + + (input1, input2, probe) + }); + + let row1 = Row::pack_slice(&[Datum::Int32(1)]); + let row2 = Row::pack_slice(&[Datum::Int32(2)]); + let row3 = Row::pack_slice(&[Datum::Int32(3)]); + + let one = Diff::from(1); + input1.update(row1.clone(), one); + input1.update(row2.clone(), one); + input2.update(row3.clone(), one); + input2.update(row1.clone(), Diff::from(2)); // duplicate row with different diff + input1.advance_to(1); + input2.advance_to(1); + input1.flush(); + input2.flush(); + + worker.step_while(|| probe.less_than(&1)); + + let mut actual = results.borrow().clone(); + actual.sort_by(|a, b| a.0.cmp(&b.0).then(a.2.cmp(&b.2))); + + assert_eq!(actual.len(), 4, "Should have 4 updates total"); + // row1 appears twice: once from input1 (diff=1) and once from input2 (diff=2) + let row1_entries: Vec<_> = actual.iter().filter(|(r, _, _)| *r == row1).collect(); + assert_eq!(row1_entries.len(), 2); + assert!(actual.iter().any(|(r, _, d)| *r == row2 && *d == one)); + assert!(actual.iter().any(|(r, _, d)| *r == row3 && *d == one)); + }); + } + + /// Verify that constant rows can be packed into a columnar collection and read back. + /// This simulates the Constant operator's columnar path. + #[mz_ore::test] + fn constant_rows_to_columnar() { + use timely::dataflow::operators::ToStream; + + timely::execute_directly(|worker| { + let results: Rc>> = Rc::new(RefCell::new(Vec::new())); + let results_capture = Rc::clone(&results); + + let probe = worker.dataflow::(|scope| { + // Simulate the Constant operator: create rows from an iterator, + // convert to a stream, then to columnar, then back to Vec. + let row1 = Row::pack_slice(&[Datum::Int32(42)]); + let row2 = Row::pack_slice(&[Datum::String("constant")]); + let row3 = Row::default(); + + let one = Diff::from(1); + let two = Diff::from(2); + + let constant_data: Vec<(Row, u64, Diff)> = + vec![(row1, 0, one), (row2, 0, two), (row3, 0, one)]; + + let vec_collection = constant_data.into_iter().to_stream(scope).as_collection(); + + let columnar = vec_to_columnar(vec_collection); + let result = columnar_to_vec(columnar); + + let (probe, _stream) = result + .inner + .inspect(move |item: &(Row, u64, Diff)| { + results_capture + .borrow_mut() + .push((item.0.clone(), item.1, item.2)); + }) + .probe(); + + probe + }); + + worker.step_while(|| probe.less_than(&1)); + + let actual = results.borrow().clone(); + assert_eq!(actual.len(), 3, "Should have 3 constant rows"); + + let row1 = Row::pack_slice(&[Datum::Int32(42)]); + let row2 = Row::pack_slice(&[Datum::String("constant")]); + let row3 = Row::default(); + let one = Diff::from(1); + let two = Diff::from(2); + + assert!( + actual + .iter() + .any(|(r, t, d)| *r == row1 && *t == 0 && *d == one) + ); + assert!( + actual + .iter() + .any(|(r, t, d)| *r == row2 && *t == 0 && *d == two) + ); + assert!( + actual + .iter() + .any(|(r, t, d)| *r == row3 && *t == 0 && *d == one) + ); + }); + } +} diff --git a/src/compute/src/render/context.rs b/src/compute/src/render/context.rs index 59e4bbd9d377b..98ab5643ece15 100644 --- a/src/compute/src/render/context.rs +++ b/src/compute/src/render/context.rs @@ -47,8 +47,8 @@ use crate::render::errors::ErrorLogger; use crate::render::{LinearJoinSpec, RenderTimestamp}; use crate::row_spine::{DatumSeq, RowRowBuilder}; use crate::typedefs::{ - ErrAgent, ErrBatcher, ErrBuilder, ErrEnter, ErrSpine, MzTimestamp, RowRowAgent, RowRowEnter, - RowRowSpine, + ColumnarCollection, ErrAgent, ErrBatcher, ErrBuilder, ErrEnter, ErrSpine, MzTimestamp, + RowRowAgent, RowRowEnter, RowRowSpine, }; /// Dataflow-local collections and arrangements. @@ -174,8 +174,8 @@ where .bindings .get_mut(&id) .expect("Binding verified to exist"); - if collection.collection.is_some() { - binding.collection = collection.collection; + if collection.columnar_collection.is_some() { + binding.columnar_collection = collection.columnar_collection; } for (key, flavor) in collection.arranged.into_iter() { binding.arranged.insert(key, flavor); @@ -308,7 +308,7 @@ where where I: IntoIterator, D: Data, - L: for<'a, 'b> FnMut(&'a mut DatumVecBorrow<'b>, S::Timestamp, Diff) -> I + 'static, + L: for<'a, 'b> FnMut(&'a mut DatumVecBorrow<'b>, &S::Timestamp, &Diff) -> I + 'static, { // Set a number of tuples after which the operator should yield. // This allows us to remain responsive even when enumerating a substantial @@ -316,7 +316,7 @@ where let refuel = 1000000; let mut datums = DatumVec::new(); - let logic = move |k: DatumSeq, v: DatumSeq, t, d| { + let logic = move |k: DatumSeq, v: DatumSeq, t: &S::Timestamp, d: &Diff| { let mut datums_borrow = datums.borrow(); datums_borrow.extend(k.to_datum_iter().take(max_demand)); let max_demand = max_demand.saturating_sub(datums_borrow.len()); @@ -399,8 +399,10 @@ where T: MzTimestamp, S::Timestamp: MzTimestamp + Refines, { - pub collection: Option<( - VecCollection, + /// Columnar variant of the unarranged collection. + /// Error streams remain Vec-based since `DataflowError` is not suited for columnar layout. + pub columnar_collection: Option<( + ColumnarCollection, VecCollection, )>, pub arranged: BTreeMap, ArrangementFlavor>, @@ -411,17 +413,56 @@ where T: MzTimestamp, S::Timestamp: MzTimestamp + Refines, { - /// Construct a new collection bundle from update streams. + /// Construct a new collection bundle from Vec update streams. + /// + /// Converts the Vec collection to columnar internally. pub fn from_collections( oks: VecCollection, errs: VecCollection, + ) -> Self { + let columnar_oks = crate::render::columnar::vec_to_columnar(oks); + Self { + columnar_collection: Some((columnar_oks, errs)), + arranged: BTreeMap::default(), + } + } + + /// Construct a new collection bundle from columnar update streams. + pub fn from_columnar_collections( + oks: ColumnarCollection, + errs: VecCollection, ) -> Self { Self { - collection: Some((oks, errs)), + columnar_collection: Some((oks, errs)), arranged: BTreeMap::default(), } } + /// Returns the columnar collection if present. + pub fn columnar_collection( + &self, + ) -> Option<&( + ColumnarCollection, + VecCollection, + )> { + self.columnar_collection.as_ref() + } + + /// Returns the collection as a Vec-based collection, converting from columnar if needed. + pub fn as_vec_collection( + &self, + ) -> ( + VecCollection, + VecCollection, + ) { + if let Some((col_oks, col_errs)) = &self.columnar_collection { + let vec_oks = crate::render::columnar::columnar_to_vec(col_oks.clone()); + (vec_oks, col_errs.clone()) + } else { + panic!("CollectionBundle contains no collection.") + } + } + /// Inserts arrangements by the expressions on which they are keyed. pub fn from_expressions( exprs: Vec, @@ -430,7 +471,7 @@ where let mut arranged = BTreeMap::new(); arranged.insert(exprs, arrangements); Self { - collection: None, + columnar_collection: None, arranged, } } @@ -449,7 +490,7 @@ where /// The scope containing the collection bundle. pub fn scope(&self) -> S { - if let Some((oks, _errs)) = &self.collection { + if let Some((oks, _errs)) = &self.columnar_collection { oks.inner.scope() } else { self.arranged @@ -466,7 +507,7 @@ where region: &Child<'a, S, S::Timestamp>, ) -> CollectionBundle, T> { CollectionBundle { - collection: self.collection.as_ref().map(|(oks, errs)| { + columnar_collection: self.columnar_collection.as_ref().map(|(oks, errs)| { ( oks.clone().enter_region(region), errs.clone().enter_region(region), @@ -489,8 +530,8 @@ where /// Extracts the collection bundle from a region. pub fn leave_region(&self) -> CollectionBundle { CollectionBundle { - collection: self - .collection + columnar_collection: self + .columnar_collection .as_ref() .map(|(oks, errs)| (oks.clone().leave_region(), errs.clone().leave_region())), arranged: self @@ -533,10 +574,7 @@ where // // If it doesn't, we panic. match key { - None => self - .collection - .clone() - .expect("The unarranged collection doesn't exist."), + None => self.as_vec_collection(), Some(key) => { let arranged = self.arranged.get(key).unwrap_or_else(|| { panic!("The collection arranged by {:?} doesn't exist.", key) @@ -544,7 +582,7 @@ where if ENABLE_COMPUTE_RENDER_FUELED_AS_SPECIFIC_COLLECTION.get(config_set) { // Decode all columns, pass max_demand as usize::MAX. let (ok, err) = arranged.flat_map(None, usize::MAX, |borrow, t, r| { - Some((SharedRow::pack(borrow.iter()), t, r)) + Some((SharedRow::pack(borrow.iter()), t.clone(), *r)) }); (ok.as_collection(), err) } else { @@ -555,6 +593,34 @@ where } } + /// Columnar variant of `as_specific_collection`. + /// + /// When `key` is `None`, returns the columnar collection directly (no conversion). + /// When `key` is `Some`, converts the arrangement to a columnar collection. + pub fn as_specific_columnar_collection( + &self, + key: Option<&[MirScalarExpr]>, + config_set: &ConfigSet, + ) -> ( + ColumnarCollection, + VecCollection, + ) { + match key { + None => { + let (col_oks, errs) = self + .columnar_collection + .as_ref() + .expect("Columnar collection doesn't exist."); + (col_oks.clone(), errs.clone()) + } + Some(_) => { + // Arrangement path: convert to Vec then columnar. + let (oks, errs) = self.as_specific_collection(key, config_set); + (crate::render::columnar::vec_to_columnar(oks), errs) + } + } + } + /// Constructs and applies logic to elements of a collection and returns the results. /// /// The function applies `logic` on elements. The logic conceptually receives @@ -579,7 +645,7 @@ where where I: IntoIterator, D: Data, - L: for<'a> FnMut(&'a mut DatumVecBorrow<'_>, S::Timestamp, Diff) -> I + 'static, + L: for<'a> FnMut(&'a mut DatumVecBorrow<'_>, &S::Timestamp, &Diff) -> I + 'static, { // If `key_val` is set, we should have to use the corresponding arrangement. // If there isn't one, that implies an error in the contract between @@ -588,15 +654,47 @@ where self.arrangement(&key) .expect("Should have ensured during planning that this arrangement exists.") .flat_map(val.as_ref(), max_demand, logic) + } else if let Some((col_oks, errs)) = &self.columnar_collection { + // Iterate columnar container directly, avoiding the columnar→Vec conversion. + // Each item yields (&RowRef, T::Ref, Diff::Ref) which we can pass to + // borrow_with_limit without materializing an owned Row. + use columnar::{Columnar, Index}; + use timely::dataflow::operators::Operator; + let oks = col_oks + .inner + .clone() + .unary::>, _, _, _>( + Pipeline, + "ColumnarFlatMap", + |_cap, _info| { + let mut datums = DatumVec::new(); + let mut t_buf = S::Timestamp::minimum(); + let mut r_buf = Diff::default(); + move |input, output| { + input.for_each(|time, data| { + let mut session = output.session(&time); + for (d, t, r) in data.borrow().into_index_iter() { + t_buf.copy_from(t); + r_buf.copy_from(r); + for item in logic( + &mut datums.borrow_with_limit(d, max_demand), + &t_buf, + &r_buf, + ) { + session.give(item); + } + } + }); + } + }, + ); + (oks, errs.clone()) } else { use timely::dataflow::operators::vec::Map; - let (oks, errs) = self - .collection - .clone() - .expect("Invariant violated: CollectionBundle contains no collection."); + let (oks, errs) = self.as_vec_collection(); let mut datums = DatumVec::new(); let oks = oks.inner.flat_map(move |(v, t, d)| { - logic(&mut datums.borrow_with_limit(&v, max_demand), t, d) + logic(&mut datums.borrow_with_limit(&v, max_demand), &t, &d) }); (oks, errs) } @@ -626,7 +724,7 @@ where + 'static, I: IntoIterator, D: Data, - L: FnMut(Tr::Key<'_>, Tr::Val<'_>, S::Timestamp, mz_repr::Diff) -> I + 'static, + L: FnMut(Tr::Key<'_>, Tr::Val<'_>, &S::Timestamp, &mz_repr::Diff) -> I + 'static, { use differential_dataflow::consolidation::ConsolidatingContainerBuilder as CB; let scope = trace.stream.scope(); @@ -756,7 +854,7 @@ where &mut datums_local, &temp_storage, event_time, - diff.clone(), + *diff, move |time| !until.less_equal(time), &mut row_builder, ) @@ -786,6 +884,42 @@ where (oks, errors.concat(errs)) } + + /// Columnar variant of `as_collection_core`. + /// + /// Applies `MapFilterProject` to the bundle and returns a columnar collection. + /// + /// For identity MFPs, returns the columnar collection directly (no conversion). + /// For non-identity MFPs, the `flat_map` path iterates columnar data directly via + /// `&RowRef` (no owned Row allocation), but the output is Vec-based (due to + /// `map_fallible` Ok/Err split) and converted back to columnar at the end. + pub fn as_columnar_collection_core( + &self, + mut mfp: MapFilterProject, + key_val: Option<(Vec, Option)>, + until: Antichain, + config_set: &ConfigSet, + ) -> ( + ColumnarCollection, + VecCollection, + ) { + mfp.optimize(); + let mfp_plan = mfp.clone().into_plan().unwrap(); + + // For identity MFPs without key_val seek, return the columnar collection + // directly — no Vec round-trip needed. + let has_key_val = matches!(&key_val, Some((_key, Some(_val)))); + if mfp_plan.is_identity() && !has_key_val { + let key = key_val.map(|(k, _v)| k); + return self.as_specific_columnar_collection(key.as_deref(), config_set); + } + + // Non-identity MFP: delegate to as_collection_core (uses columnar flat_map + // from 11.1 internally) and convert the Vec result back to columnar. + let (oks, errs) = self.as_collection_core(mfp, key_val, until, config_set); + (crate::render::columnar::vec_to_columnar(oks), errs) + } + pub fn ensure_collections( mut self, collections: AvailableCollections, @@ -797,13 +931,6 @@ where if collections == Default::default() { return self; } - // Cache collection to avoid reforming it each time. - // - // TODO(mcsherry): In theory this could be faster run out of another arrangement, - // as the `map_fallible` that follows could be run against an arrangement itself. - // - // Note(btv): If we ever do that, we would then only need to make the raw collection here - // if `collections.raw` is true. for (key, _, _) in collections.arranged.iter() { soft_assert_or_log!( @@ -818,8 +945,62 @@ where .arranged .iter() .any(|(key, _, _)| !self.arranged.contains_key(key)); - if form_raw_collection && self.collection.is_none() { - self.collection = Some(self.as_collection_core( + + // Determine if we can feed columnar input directly (identity MFP, no key). + let mfp_is_identity = { + let mut mfp = input_mfp.clone(); + mfp.optimize(); + mfp.into_plan().map_or(false, |p| p.is_identity()) + }; + let use_columnar_direct = + form_raw_collection && mfp_is_identity && input_key.is_none() && self.columnar_collection.is_some(); + + if use_columnar_direct { + let (col_oks, col_errs) = self + .columnar_collection + .as_ref() + .expect("checked above") + .clone(); + + // Track the columnar collection and errors through the arrangement loop. + let mut cached_col = Some((col_oks, col_errs)); + + for (key, _, thinning) in collections.arranged { + if !self.arranged.contains_key(&key) { + let name = format!("ArrangeBy[{:?}]", key); + let (col_oks, errs) = cached_col.take().expect("Collection constructed above"); + let (oks, errs_keyed, passthrough) = Self::arrange_columnar_collection( + &name, + col_oks, + key.clone(), + thinning.clone(), + ); + let errs_concat: KeyCollection<_, _, _> = + errs.clone().concat(errs_keyed).into(); + cached_col = Some((passthrough, errs)); + let errs = errs_concat + .mz_arrange::, ErrBuilder<_, _>, ErrSpine<_, _>>( + &format!("{}-errors", name), + ); + self.arranged + .insert(key, ArrangementFlavor::Local(oks, errs)); + } + } + if collections.raw { + if let Some((oks, errs)) = cached_col { + self.columnar_collection = Some((oks, errs)); + } + } + return self; + } + + // Fallback: materialize a Vec collection for arrangement creation. + let mut cached_vec: Option<( + VecCollection, + VecCollection, + )> = None; + if form_raw_collection { + cached_vec = Some(self.as_collection_core( input_mfp, input_key.map(|k| (k, None)), until, @@ -828,17 +1009,13 @@ where } for (key, _, thinning) in collections.arranged { if !self.arranged.contains_key(&key) { - // TODO: Consider allowing more expressive names. let name = format!("ArrangeBy[{:?}]", key); - let (oks, errs) = self - .collection - .take() - .expect("Collection constructed above"); + let (oks, errs) = cached_vec.take().expect("Collection constructed above"); let (oks, errs_keyed, passthrough) = Self::arrange_collection(&name, oks, key.clone(), thinning.clone()); let errs_concat: KeyCollection<_, _, _> = errs.clone().concat(errs_keyed).into(); - self.collection = Some((passthrough, errs)); + cached_vec = Some((passthrough, errs)); let errs = errs_concat.mz_arrange::, ErrBuilder<_, _>, ErrSpine<_, _>>( &format!("{}-errors", name), @@ -847,6 +1024,12 @@ where .insert(key, ArrangementFlavor::Local(oks, errs)); } } + if collections.raw { + if let Some((oks, errs)) = cached_vec { + self.columnar_collection = + Some((crate::render::columnar::vec_to_columnar(oks), errs)); + } + } self } @@ -934,6 +1117,89 @@ where passthrough_stream.as_collection(), ) } + + /// Like `arrange_collection`, but takes columnar input and produces a columnar passthrough. + /// + /// Iterates `&RowRef` directly from the columnar container without allocating owned Rows + /// for key/value expression evaluation. + fn arrange_columnar_collection( + name: &String, + oks: ColumnarCollection, + key: Vec, + thinning: Vec, + ) -> ( + Arranged>, + VecCollection, + ColumnarCollection, + ) { + let mut builder = + OperatorBuilder::new("FormArrangementKeyColumnar".to_string(), oks.inner.scope()); + let (ok_output, ok_stream) = builder.new_output(); + let mut ok_output = + OutputBuilder::<_, ColumnBuilder<((Row, Row), S::Timestamp, Diff)>>::from(ok_output); + let (err_output, err_stream) = builder.new_output(); + let mut err_output = OutputBuilder::from(err_output); + let (passthrough_output, passthrough_stream) = builder.new_output(); + let mut passthrough_output = + OutputBuilder::<_, ColumnBuilder<(Row, S::Timestamp, Diff)>>::from(passthrough_output); + let mut input = builder.new_input(oks.inner, Pipeline); + builder.set_notify(false); + builder.build(move |_capabilities| { + let mut key_buf = Row::default(); + let mut val_buf = Row::default(); + let mut datums = DatumVec::new(); + let mut temp_storage = RowArena::new(); + let mut t_buf = S::Timestamp::minimum(); + let mut r_buf = Diff::default(); + move |_frontiers| { + use columnar::{Columnar, Index}; + let mut ok_output = ok_output.activate(); + let mut err_output = err_output.activate(); + let mut passthrough_output = passthrough_output.activate(); + input.for_each(|time, data| { + let mut ok_session = ok_output.session_with_builder(&time); + let mut err_session = err_output.session(&time); + let mut pass_session = passthrough_output.session_with_builder(&time); + for (row_ref, t_ref, r_ref) in data.borrow().into_index_iter() { + t_buf.copy_from(t_ref); + r_buf.copy_from(r_ref); + temp_storage.clear(); + let datums = datums.borrow_with(row_ref); + let key_iter = key.iter().map(|k| k.eval(&datums, &temp_storage)); + match key_buf.packer().try_extend(key_iter) { + Ok(()) => { + let val_datum_iter = thinning.iter().map(|c| datums[*c]); + val_buf.packer().extend(val_datum_iter); + ok_session.give(((&*key_buf, &*val_buf), &t_buf, &r_buf)); + } + Err(e) => { + err_session.give((e.into(), t_buf.clone(), r_buf)); + } + } + pass_session.give((row_ref, &t_buf, &r_buf)); + } + }); + } + }); + + let oks = ok_stream + .mz_arrange_core::< + _, + Col2ValBatcher<_, _, _, _>, + RowRowBuilder<_, _>, + RowRowSpine<_, _>, + >( + ExchangeCore::, _>::new_core( + columnar_exchange::, + ), + name, + ); + ( + oks, + err_stream.as_collection(), + passthrough_stream.as_collection(), + ) + } } struct PendingWork @@ -967,7 +1233,7 @@ where ) where I: IntoIterator, D: Data, - L: FnMut(C::Key<'_>, C::Val<'_>, C::Time, C::Diff) -> I + 'static, + L: FnMut(C::Key<'_>, C::Val<'_>, &C::Time, &C::Diff) -> I + 'static, { use differential_dataflow::consolidation::consolidate; @@ -988,7 +1254,7 @@ where }); consolidate(&mut buffer); for (time, diff) in buffer.drain(..) { - for datum in logic(key, val, time, diff) { + for datum in logic(key, val, &time, &diff) { session.give(datum); work += 1; } @@ -1008,7 +1274,7 @@ where }); consolidate(&mut buffer); for (time, diff) in buffer.drain(..) { - for datum in logic(key, val, time, diff) { + for datum in logic(key, val, &time, &diff) { session.give(datum); work += 1; } diff --git a/src/compute/src/render/flat_map.rs b/src/compute/src/render/flat_map.rs index 9e5c0a6d64afd..0537f3435b02d 100644 --- a/src/compute/src/render/flat_map.rs +++ b/src/compute/src/render/flat_map.rs @@ -14,13 +14,14 @@ use mz_compute_types::dyncfgs::COMPUTE_FLAT_MAP_FUEL; use mz_expr::MfpPlan; use mz_expr::{MapFilterProject, MirScalarExpr, TableFunc}; use mz_repr::{DatumVec, RowArena, SharedRow}; -use mz_repr::{Diff, Row, Timestamp}; +use mz_repr::{Diff, Row, RowRef, Timestamp as MzTimestamp}; use mz_timely_util::operator::StreamExt; use timely::dataflow::Scope; use timely::dataflow::channels::pact::Pipeline; use timely::dataflow::operators::Capability; use timely::dataflow::operators::generic::Session; use timely::progress::Antichain; +use timely::progress::Timestamp; use crate::render::DataflowError; use crate::render::context::{CollectionBundle, Context}; @@ -41,10 +42,6 @@ where ) -> CollectionBundle { let until = self.until.clone(); let mfp_plan = mfp.into_plan().expect("MapFilterProject planning failed"); - let (ok_collection, err_collection) = - input.as_specific_collection(input_key.as_deref(), &self.config_set); - let stream = ok_collection.inner; - let scope = input.scope(); // Budget to limit the number of rows processed in a single invocation. // @@ -52,6 +49,108 @@ where // a batch. A `generate_series` can still cause unavailability if it generates many rows. let budget = COMPUTE_FLAT_MAP_FUEL.get(&self.config_set); + // When we have columnar input and no arrangement key, iterate the columnar + // container directly — each item yields (&RowRef, T::Ref, Diff::Ref) without + // allocating owned Rows. + if input_key.is_none() { + if let Some((col_oks, col_errs)) = &input.columnar_collection { + let scope = input.scope(); + let (oks, errs) = col_oks.inner.clone().unary_fallible( + Pipeline, + "FlatMapStageColumnar", + move |_, info| { + let activator = scope.activator_for(info.address); + let mut queue = VecDeque::new(); + let mut t_buf = G::Timestamp::minimum(); + let mut r_buf = Diff::default(); + Box::new(move |input, ok_output, err_output| { + use columnar::{Columnar, Index}; + let mut datums = DatumVec::new(); + let mut datums_mfp = DatumVec::new(); + let mut table_func_output = Vec::new(); + let mut budget = budget; + + input.for_each(|cap, data| { + queue.push_back(( + cap.retain(0), + cap.retain(1), + std::mem::take(data), + )) + }); + + while let Some((ok_cap, err_cap, data)) = queue.pop_front() { + let mut ok_session = ok_output.session_with_builder(&ok_cap); + let mut err_session = err_output.session_with_builder(&err_cap); + + for (row_ref, t_ref, r_ref) in data.borrow().into_index_iter() { + t_buf.copy_from(t_ref); + r_buf.copy_from(r_ref); + let temp_storage = RowArena::new(); + + let datums_local = datums.borrow_with(row_ref); + let args = exprs + .iter() + .map(|e| e.eval(&datums_local, &temp_storage)) + .collect::, _>>(); + let args = match args { + Ok(args) => args, + Err(e) => { + err_session + .give((e.into(), t_buf.clone(), r_buf)); + continue; + } + }; + let mut extensions = match func.eval(&args, &temp_storage) { + Ok(exts) => exts.fuse(), + Err(e) => { + err_session + .give((e.into(), t_buf.clone(), r_buf)); + continue; + } + }; + + while let Some((extension, output_diff)) = extensions.next() { + table_func_output.push((extension, output_diff)); + table_func_output.extend((&mut extensions).take(1023)); + drain_through_mfp( + row_ref, + &t_buf, + &r_buf, + &mut datums_mfp, + &table_func_output, + &mfp_plan, + &until, + &mut ok_session, + &mut err_session, + &mut budget, + ); + table_func_output.clear(); + } + } + if budget == 0 { + activator.activate(); + break; + } + } + }) + }, + ); + + use differential_dataflow::AsCollection; + let ok_collection = oks.as_collection(); + let new_err_collection = errs.as_collection(); + let err_collection = col_errs.clone().concat(new_err_collection); + let col_oks = crate::render::columnar::vec_to_columnar(ok_collection); + return CollectionBundle::from_columnar_collections(col_oks, err_collection); + } + } + + // Vec fallback: arrangement key or no columnar collection. + let (ok_collection, err_collection) = + input.as_specific_collection(input_key.as_deref(), &self.config_set); + let stream = ok_collection.inner; + let scope = input.scope(); + let (oks, errs) = stream.unary_fallible(Pipeline, "FlatMapStage", move |_, info| { let activator = scope.activator_for(info.address); let mut queue = VecDeque::new(); @@ -100,7 +199,6 @@ where while let Some((extension, output_diff)) = extensions.next() { table_func_output.push((extension, output_diff)); table_func_output.extend((&mut extensions).take(1023)); - // We could consolidate `table_func_output`, but it seems unlikely to be productive. drain_through_mfp( &input_row, &time, @@ -136,13 +234,13 @@ where /// /// The method decodes `input_row`, and should be amortized across non-trivial `extensions`. fn drain_through_mfp( - input_row: &Row, + input_row: &RowRef, input_time: &T, input_diff: &Diff, datum_vec: &mut DatumVec, extensions: &[(Row, Diff)], mfp_plan: &MfpPlan, - until: &Antichain, + until: &Antichain, ok_output: &mut Session< '_, '_, diff --git a/src/compute/src/render/join/delta_join.rs b/src/compute/src/render/join/delta_join.rs index dba7d0a212353..5449720b8661b 100644 --- a/src/compute/src/render/join/delta_join.rs +++ b/src/compute/src/render/join/delta_join.rs @@ -302,7 +302,15 @@ where differential_dataflow::collection::concatenate(inner, inner_errs).leave_region(), ) }); - CollectionBundle::from_collections(oks, errs) + // If any input had a columnar collection, produce columnar output for + // downstream operators. Delta joins operate entirely on arrangements, so + // the columnar→Vec conversion isn't needed for input; we only convert output. + if inputs.iter().any(|i| i.columnar_collection.is_some()) { + let col_oks = crate::render::columnar::vec_to_columnar(oks); + CollectionBundle::from_columnar_collections(col_oks, errs) + } else { + CollectionBundle::from_collections(oks, errs) + } } } diff --git a/src/compute/src/render/reduce.rs b/src/compute/src/render/reduce.rs index c5addf5ace878..5b807c7efa3e0 100644 --- a/src/compute/src/render/reduce.rs +++ b/src/compute/src/render/reduce.rs @@ -109,7 +109,8 @@ where let max_demand = demand.iter().max().map(|x| *x + 1).unwrap_or(0); let skips = mz_compute_types::plan::reduce::convert_indexes_to_skips(demand); - let (key_val_input, err_input) = input.enter_region(inner).flat_map( + let entered = input.enter_region(inner); + let (key_val_input, err_input) = entered.flat_map( input_key.map(|k| (k, None)), max_demand, move |row_datums, time, diff| { @@ -128,7 +129,7 @@ where key_plan.evaluate_into(&mut datums_local, &temp_storage, &mut row_builder); let key = match key { Err(e) => { - return Some((Err(DataflowError::from(e)), time.clone(), diff.clone())); + return Some((Err(DataflowError::from(e)), time.clone(), *diff)); } Ok(Some(key)) => key.clone(), Ok(None) => panic!("Row expected as no predicate was used"), @@ -141,13 +142,13 @@ where val_plan.evaluate_into(&mut datums_local, &temp_storage, &mut row_builder); let val = match val { Err(e) => { - return Some((Err(DataflowError::from(e)), time.clone(), diff.clone())); + return Some((Err(DataflowError::from(e)), time.clone(), *diff)); } Ok(Some(val)) => val.clone(), Ok(None) => panic!("Row expected as no predicate was used"), }; - Some((Ok((key, val)), time.clone(), diff.clone())) + Some((Ok((key, val)), time.clone(), *diff)) }, ); diff --git a/src/compute/src/render/sinks.rs b/src/compute/src/render/sinks.rs index d245c6d16b7fb..001ec8a4a6f20 100644 --- a/src/compute/src/render/sinks.rs +++ b/src/compute/src/render/sinks.rs @@ -71,8 +71,8 @@ where let bundle = self .lookup_id(mz_expr::Id::Global(sink.from)) .expect("Sink source collection not loaded"); - let (ok_collection, mut err_collection) = if let Some(collection) = &bundle.collection { - collection.clone() + let (ok_collection, mut err_collection) = if bundle.columnar_collection.is_some() { + bundle.as_vec_collection() } else { let (key, _arrangement) = bundle .arranged diff --git a/src/compute/src/typedefs.rs b/src/compute/src/typedefs.rs index 319614acb73de..d389004c66239 100644 --- a/src/compute/src/typedefs.rs +++ b/src/compute/src/typedefs.rs @@ -12,6 +12,7 @@ #![allow(dead_code, missing_docs)] use columnar::{Container, Ref}; +use differential_dataflow::Collection; use differential_dataflow::operators::arrange::Arranged; use differential_dataflow::operators::arrange::TraceAgent; use differential_dataflow::trace::implementations::chunker::ColumnationChunker; @@ -20,6 +21,7 @@ use differential_dataflow::trace::wrappers::enter::TraceEnter; use differential_dataflow::trace::wrappers::frontier::TraceFrontier; use mz_repr::Diff; use mz_storage_types::errors::DataflowError; +use mz_timely_util::columnar::Column; use timely::dataflow::ScopeParent; use crate::row_spine::RowValBuilder; @@ -122,6 +124,13 @@ pub type KeyBatcher = KeyValBatcher; pub type KeyValBatcher = MergeBatcher, ColumnationChunker<((K, V), T, D)>, ColMerger<(K, V), T, D>>; +/// A collection backed by columnar containers instead of `Vec`. +/// +/// This is the columnar equivalent of `VecCollection`. Data is stored in +/// `Column<(D, T, R)>` containers which provide better cache locality and enable +/// future vectorized evaluation. +pub type ColumnarCollection = Collection::Timestamp, R)>>; + /// Timestamp trait for rendering, constraint to support [`MzData`] and [timely::progress::Timestamp]. pub trait MzTimestamp: MzData + timely::progress::Timestamp + differential_dataflow::lattice::Lattice + std::hash::Hash