Skip to content

compute: Re-encode temporal-bucketing output as the columnar edge - #37787

Draft
antiguru wants to merge 3 commits into
columnar-p9-importsfrom
columnar-ta-temporal-bucket-reencode
Draft

compute: Re-encode temporal-bucketing output as the columnar edge#37787
antiguru wants to merge 3 commits into
columnar-p9-importsfrom
columnar-ta-temporal-bucket-reencode

Conversation

@antiguru

@antiguru antiguru commented Jul 21, 2026

Copy link
Copy Markdown
Member

Re-encode the temporal-bucketing output back to the columnar edge; the operator itself still works over Vec internally. Removes one mixed-variant edge source ahead of the concat_many teardown.

Columnar dataflow-edge migration. Design doc: doc/developer/design/20260720_columnar_dataflow_edges.md (#37744).

Part of CPU-51.

@antiguru
antiguru force-pushed the columnar-ta-temporal-bucket-reencode branch from 9978083 to 038ee6e Compare July 22, 2026 08:41
@linear-code

linear-code Bot commented Jul 22, 2026

Copy link
Copy Markdown

CPU-51

@antiguru
antiguru force-pushed the columnar-ta-temporal-bucket-reencode branch from 038ee6e to d5f9095 Compare July 22, 2026 16:24
@antiguru
antiguru force-pushed the columnar-ta-temporal-bucket-reencode branch from d5f9095 to 49c35f9 Compare July 22, 2026 17:51
@antiguru
antiguru force-pushed the columnar-ta-temporal-bucket-reencode branch 2 times, most recently from fd0f181 to aab0e0d Compare August 19, 2026 13:17
@antiguru
antiguru force-pushed the columnar-ta-temporal-bucket-reencode branch from aab0e0d to 8769666 Compare August 20, 2026 08:49
@antiguru
antiguru force-pushed the columnar-ta-temporal-bucket-reencode branch from 8769666 to 13ee1e0 Compare August 20, 2026 09:13
antiguru and others added 3 commits September 6, 2026 17:12
Temporal bucketing (`maybe_apply_temporal_bucketing`) is `Vec`-internal:
it consumes and produces a `StreamVec`. The four sites that drive it
previously decoded the edge with `into_vec` and then re-wrapped the `Vec`
result as `CollectionEdge::Vec`, leaving the last `Vec` producer that fed
a Union's `concat_many`.

Re-encode each bucketed result with `vec_to_columnar` so the output edge
is columnar, matching every other producer after the producer wave. The
bucketer stays on `Vec` internally; only its output boundary changes.
This is a non-consolidating leaf encode, since the bucketer output is not
consolidated and the prior `CollectionEdge::Vec` wrap was non-consolidating.

With every producer now columnar, a consolidating Union that mixes a
bucketed input with a `Direct` input feeds `concat_many` two columnar
edges instead of a mixed pair.

Adds runtime coverage to `temporal_bucketing.slt` under
`enable_compute_temporal_bucketing`: bucketed Reduce, bucketed TopK, and
an `EXCEPT ALL` whose Union carries `[TemporalBucketing, Direct]`.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Comment-only, no behavior change.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The comments named the migration plan's nodes, which mean nothing to a reader,
and described the union input in terms of a mixed-variant `concat_many` case
rather than what the code does.
@antiguru
antiguru force-pushed the columnar-ta-temporal-bucket-reencode branch from 13ee1e0 to 1bfb45e Compare September 6, 2026 18:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant