Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Slow instantiation of ONE #140

Open
3 of 5 tasks
k1o0 opened this issue Oct 10, 2024 · 0 comments
Open
3 of 5 tasks

Slow instantiation of ONE #140

k1o0 opened this issue Oct 10, 2024 · 0 comments
Milestone
ONE3

Comments

@k1o0
Copy link
Collaborator

k1o0 commented Oct 10, 2024 •
edited
Loading

Multiple issues are compounding here:

  1. The dataset table is large and the download time is very long
  2. Loading parquet tables from disk takes a significant amount of time
  3. Once loaded, re-indexing, sorting, and casting QC column to a category takes time

Some solutions:
@k1o0 k1o0 mentioned this issue Oct 11, 2024
Merged
@k1o0 k1o0 added this to the ONE3 milestone Nov 1, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
ONE3
Development

No branches or pull requests
1 participant
@k1o0