Further sectioned results performance improvements #6646
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This dramatically speeds up sorting TableViews with fairly dense sets of rows, and is performance-neutral for sparse sets with at most one row per leaf used.
Passing the Results to the SDK rather than Mixed allows it to use `.get<Obj>()` where applicable, which eliminates a redundant lookup by obj key when sectioning a table.
This was never actually used (Results().snapshot().sort() produces a live Results) and had a significant performance impact.
tgoyne
force-pushed
the
tg/sectioned-perf-2
branch
from
fa26a82
to
3467913
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
(But actually just performance improvements to the underlying things since that's where all the time is spent).
This eliminates TableRef checks in a pile of places where it very definitely will always be valid, generally because we get to the place via a call on a Table. TableRef checking isn't very expensive, but it adds up to a few percent of the runtime.
Checking if a TableView is up to date previously involved memory allocations, and it's an operation we do a lot. It now reuses a buffer between calls. Frozen table views now skip the check entirely as they're always up to date if they've ever been evaluated.
Sorting a TableView handled detached ObjKeys which would result from "snapshot" tableviews that we didn't re-evaluate. We never actually sort those after the fact, so we can just skip that handling and save some time.
Looking up an ObjKey in a Table is actually a kinda slow operation and was a meaningful portion of the runtime of sorting TableViews. The newly introduced ClusterTree::LeafCache optimizes lookups where subsequent lookups are on the same leaf, which will be fairly often in cache_first_column() due to it reading matching rows in table order. It is performance-neutral for sparse queries which never have multiple rows on the same leaf.