Skip to content

cloud calibration updates#108

Draft
cdc-as81 wants to merge 18 commits intomainfrom
cdc-as81-cloud
Draft

cloud calibration updates#108
cdc-as81 wants to merge 18 commits intomainfrom
cdc-as81-cloud

Conversation

@cdc-as81
Copy link
Copy Markdown
Collaborator

Summary

Adds cloud and Docker-backed phase 1 calibration support for ixa-epi-covid, including reusable MRP runners, Azure Batch orchestration, synthetic population staging, auto-sizing, cleanup tooling, and expanded test coverage.

Changes

  • Add packaged phase 1 calibration entrypoints with direct, local MRP, Docker MRP, and Azure/cloud execution modes.
  • Add Dockerfile.cloud and MRP config files for local, Docker, cloud, and cloud-task execution.
  • Add cloud runner support for:
    • staging one shared synthetic population file per cloud session
    • rewriting task payloads for container paths
    • reusing Azure service clients
    • limiting parallel output downloads
    • optional RAM-based cloud auto-sizing
  • Add Azure cleanup CLI for listing and deleting cloud session resources.
  • Refactor model execution into shared helpers for importation generation, IXA execution, output reading, and canonical output.csv writing.
  • Update Makefile targets for cloud sync, Docker image builds, phase 1 calibration modes, checks, and help output.
  • Improve synthetic population generation by deriving household frames from sampled PUMS data and filtering invalid household weights.
  • Update FIPS/setting code capacity to support larger setting IDs.
  • Prefer packaged importation data when available, with cache/download fallback.
  • Add tests for cloud runner behavior, population asset staging, auto-sizing, MRP runners, cleanup wiring, phase 1 helpers, and model execution contracts.

@KOVALW
Copy link
Copy Markdown
Collaborator

KOVALW commented May 1, 2026

Commenting here that we'll keep this PR as a draft until calibrationtools has a front end that doesn't require more than a TOML and minimal argument changes to the sampler run calls in order to use Azure cloud

cdc-as81 added 3 commits May 5, 2026 12:36
run the Docker-backed MRP config directly through ixa_epi_covid.mrp_task_runner
override the IXA executable path for built-in local Docker calibration
@cdc-as81 cdc-as81 marked this pull request as draft May 6, 2026 14:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants