Skip to content

Commit 710debe

Browse files
anth-volkclaude
andcommitted
Split the artifact and Stage 12 cache precompute modules
precompute.py (931 lines), stage12_cache/precompute.py (844 lines) and tests/test_precompute.py (1,835 lines) were over the file-length limits. The code moves unchanged. Each precompute module keeps its manifest functions, its functions dataclass and run_precompute, and re-exports every public name in __all__, so the Modal precompute apps, which import the step functions from it when they run, are unchanged. The steps move to three modules next to it: - precompute_planning.py: cohorts or baseline identities, the state dataset targets, plan_artifacts_impl and select_work - precompute_datasets.py: the national and per-state dataset builds - precompute_baselines.py: computing the baselines and checking that they are deterministic Private names another module now uses are public: the artifact precompute's _cohort_resolution, and the cache precompute's _scoped_planned_simulation (with a new docstring) and _baseline_identity. stage12_worker_validation.py imports scoped_planned_simulation from the cache's precompute_planning.py. tests/test_precompute.py keeps the schema, planning and orchestration tests; the rest move to test_precompute_planning.py, test_precompute_datasets.py and test_precompute_baselines.py, all 71 tests unchanged apart from patch targets. Each patch now targets the module whose code reads the name: the planning tests patch state_dataset_targets and cohort_identity in precompute_planning, the state dataset tests patch state_dataset_targets in precompute_datasets, and the determinism tests patch _prepare_cohort_baseline in precompute_baselines. In the cache tests, the planning fixture and the state-target test patch precompute_planning, the state build fixture patches precompute_datasets, the determinism tests patch precompute_baselines, and the store helper replaces Stage12CacheStore in all four modules, each of which opens the store. The file-length baseline drops the three files (count 7 to 4). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
1 parent 488a681 commit 710debe

18 files changed

Lines changed: 2910 additions & 2656 deletions

‎changelog_entry.yaml‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -16,6 +16,7 @@
1616
- Ran Stage 12 US national simulations as 20 region-group calls each, and built an independent, immutable Stage 12 cache of three annual US datasets and 60 current-law baselines before each deployment, publishing it only after a recorded determinism check and recording its release in the v2 version manifest under a new key, while the existing key keeps a copy that earlier readers accept
1717
- Made the budget refuse a household count outside its country's plausible range (above 200 million in the US or 60 million in the UK, negative or not finite), raising ImplausibleEntityCountError instead of publishing a count like #767's
1818
changed:
19+
- Split the artifact precompute (precompute.py, 931 lines) and the Stage 12 cache precompute (stage12_cache/precompute.py, 844 lines) each into planning, dataset-building and baseline modules, with the manifest and run_precompute left in the original module, which re-exports every public name, so the Modal precompute apps and their tests import from the same paths; the cache precompute's _scoped_planned_simulation and _baseline_identity are now public, and the worker validation imports scoped_planned_simulation from its new module; the 1,835-line precompute test file is split into four, and the cache precompute tests patch each name in the module whose code reads it
1920
- Split the 4,709-line Stage 12 runtime test file into thirteen test files by behaviour (dataset loading, provenance, the single worker and its failures, aggregates, the coordinator and its failures, runtime store retries, failure records, cancellation, child state errors and the wait window) and three support modules for the shared builders, fake stores and fake child calls, whose helpers are now public; the other Stage 12 tests import them from those modules, every test keeps its name and body, and two parametrized cases are named after the renamed bundle helper
2021
- Split Stage 12 report coordination (stage12_runtime/coordination.py, 1,091 lines) into modules for the report coordinator class, its child simulations' records, failures and cancellation, and the parent report's checks and claim, with coordinate_report and every public name still in coordination.py and child failures still logged under its logger; and turned the contract's stage12_execution.py (929 lines) into a stage12_execution package of modules for the shared field types and simulation input, report inputs and output plans, result artifacts, and comparison-run records, which re-exports every name from the same import path, with unchanged JSON schemas; models subclass the base models through the package path ruff.toml lists, so no ruff setting changes
2122
- Split the gateway's endpoints.py (1,454 lines) into modules for the shared router and request helpers, route resolution from the active routing state, Modal reads of job metadata and routing, bundle payloads, and the job submission, job status and version routes, which register their routes in the same order, so the OpenAPI document is unchanged; endpoints.py re-exports every public name, the routes keep logging under the policyengine_simulation_gateway.endpoints logger, and the private helpers the new modules share are now public; its 1,801-line test file is split into five test files, with the shared expected-response helpers in fixtures/gateway_responses.py

‎docs/generated/rules.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -66,7 +66,7 @@ Every rule that governs `policyengine-sim-api` is listed here with how it is enf
6666

6767
- `sim.ci.multiline-run-is-script`: 13 remaining in `rules/baselines/ci-multiline-run.txt`
6868
- `sim.microdata.weight-stripping-confined`: 0 remaining in `rules/baselines/sim-microdata-weight-stripping.txt`
69-
- `sim.quality.python-file-length`: 7 remaining in `rules/baselines/python-file-length.txt`
69+
- `sim.quality.python-file-length`: 4 remaining in `rules/baselines/python-file-length.txt`
7070

7171
### Exceptions and review dates
7272

0 commit comments

Comments
 (0)