Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature Request: Moving Quantile Window Operator #315

Open
joshdunnlime opened this issue Nov 22, 2023 · 1 comment
Open

Feature Request: Moving Quantile Window Operator #315

joshdunnlime opened this issue Nov 22, 2023 · 1 comment

Comments

@joshdunnlime
Copy link

As you might expect, moving quantiles are particularly slow by their nature. Standard python (e.g. Pandas, numpy) libraries offer moving/rolling/expanding quantiles but these are often slow and scale very poorly to 100,000 of data points.

Temporian seems like the perfect place to introduce both language specific speed-ups in C++ but also algorithmically superior operations. Here are some examples:
Heap based moving median: https://aakinshin.net/posts/partitioning-heaps-quantile-estimator/

Approximate moving median:
https://aakinshin.net/posts/mp2-quantile-estimator/
https://aakinshin.net/posts/partitioning-heaps-quantile-estimator2/

@ianspektor
Copy link
Collaborator

Thanks for the proposal @joshdunnlime! Agree that Temporian is a fitting home for this kind of implementation :)

Added it to our backlog and will work on it when we have the bandwidth! If you'd like to contribute to make it happen asap do let us know and we'll arrange it.

Cheers!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants