Summary
Add support for retroactive bandpass calibration: when a day's bandpass-calibrator transit yields a bad solution, re-solve the bandpass using the next day's transit MS and apply backward to the previous day's data.
Current state
- Same-day forward solve is enforced: the calibrator must be transiting in the MS being solved (
dsa110_continuum/calibration/runner.py::_validate_calibrator_transit).
- Day-level table borrowing is implemented and bidirectional: when the target date has no validated BP/G tables, the pipeline searches outward (±1, ±2, … days, default
max_borrow_days=30) for the nearest date with real tables and symlinks them onto the target date (dsa110_continuum/calibration/ensure.py::_find_nearest_real_tables).
What is not implemented: taking day N's data and running a fresh bandpass solve against day N+1's calibrator transit MS. The borrow path only symlinks existing tables — it never triggers a new calibration solve from a different day's MS.
Operational rationale
DSA-110 in drift-scan sees the same bandpass calibrator transit once per sidereal day. If today's transit is bad (RFI, system-temperature excursion, flagging artefact, etc.), the next viable solution comes from tomorrow's transit. Trade-off:
- Forward (today → today): low-latency output, lost data if today's solution is bad.
- Reverse (tomorrow → today): recovers data that would otherwise be discarded, adds ~24 h latency.
The pipeline should support either direction and pick by calibration QA verdict (i.e. only re-solve when the QA gate marks today's solution bad).
Acceptance criteria
- A function (or pipeline mode) that takes `(target_date, calibrator_mjd)` and produces BP/G tables for `target_date`, where `calibrator_mjd` may be in the future.
- Provenance sidecar records the cross-date origin (e.g. `solved_against_date: YYYY-MM-DD`).
- `scripts/batch_pipeline.py` orchestration to retry a previously-failed date when a fresher calibrator MS becomes available.
- Same-date verification before declaring acceptance: produce a `CLEAN` verdict for a known-bad-day-followed-by-good-day pair.
- Before/after dynamic-range comparison on a known case (per the calibration acceptance-criteria convention).
Related
- `CONTEXT.md` "Calibration table borrowing" entry — current behavior.
- `CONTEXT.md` "Bandpass cycle" entry — operational framing.
Summary
Add support for retroactive bandpass calibration: when a day's bandpass-calibrator transit yields a bad solution, re-solve the bandpass using the next day's transit MS and apply backward to the previous day's data.
Current state
dsa110_continuum/calibration/runner.py::_validate_calibrator_transit).max_borrow_days=30) for the nearest date with real tables and symlinks them onto the target date (dsa110_continuum/calibration/ensure.py::_find_nearest_real_tables).What is not implemented: taking day N's data and running a fresh bandpass solve against day N+1's calibrator transit MS. The borrow path only symlinks existing tables — it never triggers a new calibration solve from a different day's MS.
Operational rationale
DSA-110 in drift-scan sees the same bandpass calibrator transit once per sidereal day. If today's transit is bad (RFI, system-temperature excursion, flagging artefact, etc.), the next viable solution comes from tomorrow's transit. Trade-off:
The pipeline should support either direction and pick by calibration QA verdict (i.e. only re-solve when the QA gate marks today's solution bad).
Acceptance criteria
Related