Skip to content

Split the artifact and Stage 12 cache precompute modules under the file-length limit - #938

Merged
anth-volk merged 1 commit into
mainfrom
file-length-split-precompute
Oct 27, 2026
Merged

anth-volk merged 1 commit into
mainfrom
file-length-split-precompute

Conversation

@anth-volk

Copy link
Copy Markdown
Contributor

Fixes #937

Same-Repository Draft PR Check

  • This PR is a draft.
  • This PR is from a branch in PolicyEngine/policyengine-sim-api, not a personal fork.

Summary

Sixth of nine stacked PRs; it builds on #936. It splits the artifact precompute, the Stage 12 cache precompute and the precompute tests without changing behaviour, and removes all three from the file-length baseline (count 7 to 4). For each source file, a script compared every top-level statement of the old module with the new modules: all appear exactly once and unchanged apart from the renames below, in their original order.

precompute.py (931 lines) and stage12_cache/precompute.py (844 lines)

Both are split the same way. The original module keeps the manifest functions, the functions dataclass and run_precompute, and re-exports every public name in __all__. The Modal precompute apps import the step functions from it inside their function bodies, so they, and the app tests that replace those functions on the module, are unchanged.

Module Artifact precompute Stage 12 cache
precompute.py: manifest, functions dataclass, run_precompute, re-exports 276 226
precompute_planning.py: cohorts or baseline identities, state dataset targets, plan_artifacts_impl, select_work 271 350
precompute_datasets.py: national and per-state dataset builds 276 241
precompute_baselines.py: computing baselines and checking determinism 207 134

Renames

Private names another module now uses are public: _cohort_resolution to cohort_resolution (artifact precompute), and _scoped_planned_simulation to scoped_planned_simulation (docstring added) and _baseline_identity to baseline_identity (cache precompute). stage12_worker_validation.py imported _scoped_planned_simulation from the cache precompute; it now imports scoped_planned_simulation from stage12_cache/precompute_planning.py.

Tests

  • tests/test_precompute.py (1,835 lines) keeps the schema, planning and orchestration tests (575 lines); the rest move to test_precompute_planning.py (428), test_precompute_datasets.py (443) and test_precompute_baselines.py (417). 71 tests before and after, with the same names; a script compared every class method and statement: all unchanged apart from patch targets.
  • Patch targets, each checked against the modules that read the name:
    • artifact precompute: TestPlanArtifactsImpl patches cohort_identity and state_dataset_targets in precompute_planning; TestBuildStateDatasetsImpl patches state_dataset_targets in precompute_datasets, where the state dataset build reads it; TestVerifyDeterminismImpl patches _prepare_cohort_baseline in precompute_baselines.
    • cache precompute (tests/test_stage12_cache_precompute.py, not split): the stubbed_planning fixture and test_states_on_the_default_dataset_add_no_state_datasets patch precompute_planning, which plan_artifacts_impl reads; test_state_targets_follow_how_a_request_for_the_state_resolves patches _country_module there too; the state_build fixture patches precompute_datasets; the determinism tests patch _ensure_dataset_file, scoped_planned_simulation and Stage12CacheStore in precompute_baselines; and the _use_store helper replaces Stage12CacheStore in all four modules, each of which opens the store. test_planned_baseline_file_is_the_one_the_worker_saves_and_loads calls scoped_planned_simulation from precompute_planning.
    • tests/test_stage12_worker_validation.py still patches state_dataset_targets on the cache precompute module, which the validation reads as a module attribute.

Environment variables

No change: the moved code reads the same variables, in the same functions, and no workflow, image or deployment file changes.

Testing

  • Executor, full suite: 1,873 passed, 22 skipped, as before.
  • Pyright for the executor: no errors.
  • Ruff 0.15.22 format and lint: clean.
  • python scripts/run_quality_guards.py and python -m scripts.rules guard --base-ref origin/file-length-split-stage12-runtime-tests: passed; python -m scripts.rules render leaves no diff.

Only the lowest PR in the stack runs full CI, so this PR's CI complete check fails by design until the PRs below it merge.

🤖 Generated with Claude Code

@anth-volk
anth-volk added this pull request to stack #928 October 27, 2026 13:34
@anth-volk
anth-volk force-pushed the file-length-split-precompute branch from b509b93 to 64f1b35 Compare October 27, 2026 13:45
@anth-volk
anth-volk force-pushed the file-length-split-precompute branch 4 times, most recently from 91e5eec to bafaee6 Compare October 27, 2026 16:30
Base automatically changed from file-length-split-stage12-runtime-tests to main October 27, 2026 16:52
precompute.py (931 lines), stage12_cache/precompute.py (844 lines) and
tests/test_precompute.py (1,835 lines) were over the file-length limits.
The code moves unchanged.

Each precompute module keeps its manifest functions, its functions
dataclass and run_precompute, and re-exports every public name in
__all__, so the Modal precompute apps, which import the step functions
from it when they run, are unchanged. The steps move to three modules
next to it:

- precompute_planning.py: cohorts or baseline identities, the state
  dataset targets, plan_artifacts_impl and select_work
- precompute_datasets.py: the national and per-state dataset builds
- precompute_baselines.py: computing the baselines and checking that
  they are deterministic

Private names another module now uses are public: the artifact
precompute's _cohort_resolution, and the cache precompute's
_scoped_planned_simulation (with a new docstring) and _baseline_identity.
stage12_worker_validation.py imports scoped_planned_simulation from the
cache's precompute_planning.py.

tests/test_precompute.py keeps the schema, planning and orchestration
tests; the rest move to test_precompute_planning.py,
test_precompute_datasets.py and test_precompute_baselines.py, all 71
tests unchanged apart from patch targets. Each patch now targets the
module whose code reads the name: the planning tests patch
state_dataset_targets and cohort_identity in precompute_planning, the
state dataset tests patch state_dataset_targets in precompute_datasets,
and the determinism tests patch _prepare_cohort_baseline in
precompute_baselines. In the cache tests, the planning fixture and the
state-target test patch precompute_planning, the state build fixture
patches precompute_datasets, the determinism tests patch
precompute_baselines, and the store helper replaces Stage12CacheStore
in all four modules, each of which opens the store.

The file-length baseline drops the three files (count 7 to 4).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@anth-volk
anth-volk force-pushed the file-length-split-precompute branch from bafaee6 to 710debe Compare October 27, 2026 16:52
@anth-volk
anth-volk marked this pull request as ready for review October 27, 2026 17:00
@anth-volk
anth-volk requested a review from a team as a code owner October 27, 2026 17:00
@anth-volk
anth-volk merged commit 7913229 into main Oct 27, 2026
23 checks passed
@anth-volk
anth-volk deleted the file-length-split-precompute branch October 27, 2026 17:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Split the artifact and Stage 12 cache precompute modules under the file-length limit

1 participant