Skip to content

reload_app_main() permanently swaps sys.modules app — pinned-app tests inherit stale drain state #2585

Description

@rmusser01

Problem

tldw_Server_API/tests/helpers/app_main_state.py::reload_app_main() replaces sys.modules["tldw_Server_API.app.main"] with a freshly imported module (new app object) and never restores the original, even though the helper ships snapshot_app_main() / restore_app_main() for exactly that.

Consequence chain (empirically verified during #2581 / PR #2583 review, with a probe plugin logging app identities — three distinct app objects observed in one Embeddings run):

  1. All test modules import app at collection time → they pin the ORIGINAL app object.
  2. Any test calling reload_app_main() (e.g. tests/Embeddings/test_backpressure_and_quotas.py:106,254; ~15 files repo-wide use main reloads) swaps sys.modules permanently.
  3. A later with TestClient(app) lifespan exit marks the pinned ORIGINAL app draining (mark_lifecycle_shutdown).
  4. The root conftest's autouse reset (tests/conftest.py::_reset_main_app_lifecycle_state_between_tests) re-imports app.main at reset time — so it resets only the NEW app. The drained original stays drained, and every subsequent request through a pinned client gets 503 {"status":"not_ready","reason":"shutdown_in_progress"} from DrainGateMiddleware.

This caused all 46 Embeddings directory-run failures (every file passed solo). PR #2583 adds a suite-local mitigation (tests/Embeddings/conftest.py::_reset_app_lifecycle_state resetting the pinned instance), but the hazard remains for ANY suite that mixes reload_app_main() callers with pinned-app lifespan tests.

Suggested fix

Make reload_app_main() callers restore the snapshot (fixture-style):

@pytest.fixture
def reloaded_app_main():
    snap = snapshot_app_main()
    try:
        yield reload_app_main()
    finally:
        restore_app_main(snap)

or have the root conftest reset track all app instances it has seen (e.g. a WeakSet registered at app construction) instead of only the current sys.modules one.

Repro

Remove the _reset_app_lifecycle_state fixture from tests/Embeddings/conftest.py and run:

pytest tldw_Server_API/tests/Embeddings -q  # 46 failed, all 503 shutdown_in_progress

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions