roll_time_series() speed up

[ItsJustIkenna]

Are there any suggestions on how to speed up this function? I am working with over 1 million rows of time series data so the function is entirely too slow. I read in the discussions/issues some people were talking about GPU acceleration or using numba? I think that was for the feature_extraction function and not roll_time_series. If anyone has ran into or solved this problem in the past, I'd love some help.

[nils-braun]

Hi @ItsJustIkenna - unfortunately I do not have a simple solution for you. If your data is large, have you thought about using an external system, such as e.g. Spark? It will potentially have a better support fir windowing in large data samples. What kind of windows do you want to achieve? If you are interested in only small window sizes, you might want to think about splitting your data into smaller pieces prior to the rolling. You might loose a few windows, but depending on your task that might not be so severe (?).