fix(tests): look for the engine where the runtime caches it - #109
Open
kevin9327 wants to merge 1 commit into
Open
fix(tests): look for the engine where the runtime caches it#109kevin9327 wants to merge 1 commit into
kevin9327 wants to merge 1 commit into
Conversation
Signed-off-by: kevin9327 <kevin9327@users.noreply.github.com>
kevin9327
force-pushed
the
fix/engine-test-gate
branch
from
September 5, 2026 14:45
0cc0511 to
a42199a
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Every
requires_enginetest has been skipped sincecd2f976, on machines that have the engine.That commit moved the engine cache under a generation directory.
needle._library_pathnow reads:and
needle fetchwrites there, printing~/.cache/cactus-needle/v2/2.0.4/.tests/conftest.pywas not part of that commit and still looks in the pre-split location:
The
v2segment is missing, so the gate never finds a fetched engine and the six tests behind itnever run. It also misses the other two ways
_library_pathfinds an engine that the same commitintroduced or kept: the
NEEDLE2_LIB_PATH/NEEDLE_LIB_PATHoverride, and the generation-suffixedin-package name (
libneedle2.so).The skip is silent, so nothing has been failing. The tests simply stopped covering anything.
Fix
_engine_availablenow mirrors_library_path's lookup order — override, in-package names,generation-scoped cache — and stops short of the download that function ends with.
Tests
Two in
tests/test_fetch.py, both failing before this change:test_engine_gate_finds_the_cache_the_runtime_loads_from— withHOMEpointed at a temp tree andthe package directory empty, the gate is false, then true once an engine file exists at the path
_library_pathuses.test_engine_gate_honours_the_library_override—NEEDLE_LIB_PATHat a real file opens the gate;pointed at a missing one it does not.
How I tested
Windows 11, Python 3.12.10,
pip install -e ".[test,train]", engine fetched withneedle fetchinto
C:\Users\...\.cache\cactus-needle\v2\2.0.4\libneedle.dll.The six gated tests, before and after:
Whole suite,
pytest -q -m "not slow":The six that stop skipping are the engine tests; the two extra passes are the tests above. The three
that still skip are the worker integration tests, which want a C compiler. No test changes from
passing to failing.
Both runs exclude
test_lora.py,test_render.py,test_run.py,test_build.pyandtest_finetune.py, which needflax. That will not install on this machine:orbax-checkpointships paths past the Windows
MAX_PATHlimit and pip fails withWinError 206. Nothing heretouches that path, and CI runs it on Ubuntu.
Worth noting separately: the engine tests pass on Windows once they actually run, so the gate was
hiding working coverage rather than broken code.