-
Notifications
You must be signed in to change notification settings - Fork 358
PyCBC Live O4 development
Placeholder page to track ideas and proposals for PyCBC Live features we want in O4. Note that there is a PyCBC Live O3 development page as well, with open items that should be carried here. Here I am adopting a different format than the O3 page because I felt some items required more explanation than could fit in a table row.
The current example is quite minimal and does not test a variety of things that are used in a production search, notably:
- Different combinations of detectors.
- The state and DQ channels (#3261).
The O3 implementation of autogating still feels a bit clunky and should be more carefully characterized and checked. For example, what happens if a glitch is close to the boundary of the analysis chunk? What happens when the analysis chunk becomes much shorter (early-warning analysis)?
There are also potentially better ways to remove loud glitches, for example the inpainting method developed by IAS. Can one of these methods be used in low latency?
As explained in the O3 paper, including relatively insensitive detectors in the analysis leads to an increase of the trials factor due to how the current statistic is organized. Can this be improved, for example by excluding detectors from the trials based on their local sensitivity?
It would also be useful to introduce something similar to the single-detector trigger fits used by the offline search, as the background rate varies quite a lot over the search space.
The latency of the analysis is currently sensitive to the number of observing detectors, because each MPI worker has to process all detectors. Can we improve this by only processing one detector per MPI worker?
Right now each MPI worker requires access to h(t), and does the same conditioning to it. This has led to issues when the h(t) availability or timing becomes inconsistent across the cluster nodes. Can we read and prepare h(t) on the root process and broadcast it to the workers, while maintaining the same latency?
This has partially been done using O2 replay data and seems to work well, but it has to be looked at more carefully.
Can iDQ be used to improve the robustness of the search against glitches?
In O3, each upload was followed by a few immediate follow-up processes (for example adding plots and comments to GraceDB) which created noticeable spikes in the lag of the analysis. Can these operations be split off to separate threads in a nice way?
What is the effect of the SNR optimization on the skymaps generated by BAYESTAR? Are there ways to improve the speed or accuracy of pycbc_optimize_snr?
MPI has lots of little quirks and annoyances that make it somewhat inconvenient to use on the CIT cluster. Is there a different way to organize the multiprocess/multinode operation and communication, possibly using Condor?