-
Notifications
You must be signed in to change notification settings - Fork 20
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[builder] enable Arrow Dictionary feature flag #1064
Conversation
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #1064 +/- ##
==========================================
+ Coverage 81.33% 81.35% +0.01%
==========================================
Files 73 73
Lines 5566 5566
==========================================
+ Hits 4527 4528 +1
+ Misses 1039 1038 -1
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
This PR enables the use of Arrow dictionary (aka TileDB enum, Pandas Categorical, ...) in building the Census. Affects various string columns in the
obs
dataframe which contain repetitive labels, such ascell_type
. Primary impact is more efficient memory use for end-user (reader) of Census obs dataframe.Fixes #604
Other changes: