Skip to content

Importing OCELOT is too slow #313

Description

@st-walker

We need to speed it up. It takes nearly 3 seconds to import ocelot. This is not acceptable.

The original sin is that the top level init.py imports basically everything. I guess we can't change that as everyone's code will then break.

Slow:

  • importing some parts of scipy
  • precompiling with numba in math_op.py and anywhere else, we should always compile lazily.
  • importing pandas is slow
  • h5py is not that slow but it should be optional.

Solutions:

  • lazy compile all numba stuff
  • move scipy imports into function bodies
  • move to polars from pandas
  • rewrite __init__.py to not import the entire package.
  • h5py should be optional

with some the above stuff I got the import time down to 0.7s.

If we wrap the scipy imports then it would be faster still. Without moving all the scipy imports we can at least get down to 900ms import time or so.

Before, 2.7s:

Image

After 0.7s:

Image

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions