Skip to content

fix(tests): look for the engine where the runtime caches it - #109

Open
kevin9327 wants to merge 1 commit into
cactus-compute:mainfrom
kevin9327:fix/engine-test-gate
Open

fix(tests): look for the engine where the runtime caches it#109
kevin9327 wants to merge 1 commit into
cactus-compute:mainfrom
kevin9327:fix/engine-test-gate

Conversation

@kevin9327

Copy link
Copy Markdown

Problem

Every requires_engine test has been skipped since cd2f976, on machines that have the engine.

That commit moved the engine cache under a generation directory. needle._library_path now reads:

cache = os.path.join(os.path.expanduser("~"), ".cache", "cactus-needle",
                     f"v{generation}", version)

and needle fetch writes there, printing ~/.cache/cactus-needle/v2/2.0.4/. tests/conftest.py
was not part of that commit and still looks in the pre-split location:

cache = os.path.join(os.path.expanduser("~"), ".cache",
                     "cactus-needle", fetch.ENGINE_VERSION, name)

The v2 segment is missing, so the gate never finds a fetched engine and the six tests behind it
never run. It also misses the other two ways _library_path finds an engine that the same commit
introduced or kept: the NEEDLE2_LIB_PATH / NEEDLE_LIB_PATH override, and the generation-suffixed
in-package name (libneedle2.so).

The skip is silent, so nothing has been failing. The tests simply stopped covering anything.

Fix

_engine_available now mirrors _library_path's lookup order — override, in-package names,
generation-scoped cache — and stops short of the download that function ends with.

Tests

Two in tests/test_fetch.py, both failing before this change:

  • test_engine_gate_finds_the_cache_the_runtime_loads_from — with HOME pointed at a temp tree and
    the package directory empty, the gate is false, then true once an engine file exists at the path
    _library_path uses.
  • test_engine_gate_honours_the_library_overrideNEEDLE_LIB_PATH at a real file opens the gate;
    pointed at a missing one it does not.

How I tested

Windows 11, Python 3.12.10, pip install -e ".[test,train]", engine fetched with needle fetch
into C:\Users\...\.cache\cactus-needle\v2\2.0.4\libneedle.dll.

The six gated tests, before and after:

before   25 passed, 6 skipped   (all six "needle C++ engine not installed")
after    31 passed

Whole suite, pytest -q -m "not slow":

before   64 passed, 9 skipped
after    72 passed, 3 skipped

The six that stop skipping are the engine tests; the two extra passes are the tests above. The three
that still skip are the worker integration tests, which want a C compiler. No test changes from
passing to failing.

Both runs exclude test_lora.py, test_render.py, test_run.py, test_build.py and
test_finetune.py, which need flax. That will not install on this machine: orbax-checkpoint
ships paths past the Windows MAX_PATH limit and pip fails with WinError 206. Nothing here
touches that path, and CI runs it on Ubuntu.

Worth noting separately: the engine tests pass on Windows once they actually run, so the gate was
hiding working coverage rather than broken code.

Signed-off-by: kevin9327 <kevin9327@users.noreply.github.com>
@kevin9327
kevin9327 force-pushed the fix/engine-test-gate branch from 0cc0511 to a42199a Compare September 5, 2026 14:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant