Merge branch 'feature_module1_reflow' into develop

event_display/README.md

@@ -0,0 +1,17 @@
+# Flow File Event Displays
+
+## Single Module Event Displays
+
+### Overview 
+
+There are currently two Python-based event displays available for visualizing single module data and simulation files run through flow. The event displays are version-specific, with one intended for use on files run through `module0_flow` and one for use on files run through `proto_nd_flow`. The source code for each event display is available in the corresponding subdirectory, e.g. `event_display/proto_nd_flow`, and is adapted from [this older event display script](https://github.com/larpix/larpix-v2-testing-scripts/blob/master/event-display/module0_evd.py). The event displays here are set up to run inside Jupyter notebooks, and examples of such usage are also given in the corresponding subdirectories. The examples involve using files stored on NERSC, so if the notebooks are run outside of NERSC, you will need to download both the flow file you would like to visualize and a module geometry file.
+
+### File Locations
+
+In general, single module data files can be found [in this file system.](https://portal.nersc.gov/project/dune/data/) For the examples provided in the associated notebooks, the following files are used:
+
+ - [Module 1 geometry file](https://portal.nersc.gov/project/dune/data/Module1/TPC12/module1_layout-2.3.16.yaml)
+ - [Module 1 data file run through `module0_flow` (charge only)](https://portal.nersc.gov/project/dune/data/Module1/reco/charge_only/events_2022_02_09_17_23_09_CET.gz.h5)
+ - [Module 1 data file run through `proto_nd_flow` (charge only)](https://portal.nersc.gov/project/dune/data/Module1/TPC12/reflow-test/packet_2022_02_09_17_23_09_CET.module1_flow.h5)
+
+Additional Module 1 charge-only data files run through `module0_flow` can be found [here](https://portal.nersc.gov/project/dune/data/Module1/reco/charge_only/), and additional Module 1 charge-only data files run through `proto_nd_flow` can be found [here](https://portal.nersc.gov/project/dune/data/Module1/TPC12/reflow-test/). 

scripts/module1_flow/run_light_charge_data.sh

@@ -0,0 +1,43 @@
+LIGHT_INPUT_FILE=$1
+CHARGE_INPUT_FILE=$2
+
+LIGHT_OUTPUT_NAME=(${LIGHT_INPUT_FILE//"/"/ })
+LIGHT_OUTPUT_NAME=${LIGHT_OUTPUT_NAME[-1]}
+
+OUTPUT_DIR=`pwd` #!!! change me
+OUTPUT_NAME=(${CHARGE_INPUT_FILE//"/"/ })
+OUTPUT_NAME=${OUTPUT_NAME[-1]}
+OUTPUT_FILE="${OUTPUT_DIR}/${OUTPUT_NAME}"
+OUTPUT_FILE=${OUTPUT_FILE//.h5/_${LIGHT_OUTPUT_NAME}.module1_flow.h5}
+echo ${OUTPUT_FILE}
+
+H5FLOW_CMD='h5flow'
+
+HERE=`pwd`
+#cd ndlar_flow
+# assumes this is being run from ndlar_flow/scripts/module1_flow:
+cd ../../
+
+# avoid being asked if we want to overwrite the file if it exists.
+# this is us answering "yes".
+if [ -e $OUTPUT_FILE ]; then
+    rm -i $OUTPUT_FILE
+fi
+
+WORKFLOW1='yamls/module1_flow/workflows/light/light_event_building_adc64.yaml'
+WORKFLOW2='yamls/module1_flow/workflows/light/light_event_reconstruction.yaml'
+
+$H5FLOW_CMD -c $WORKFLOW1 $WORKFLOW2 -i $LIGHT_INPUT_FILE -o $OUTPUT_FILE
+
+WORKFLOW1='yamls/module1_flow/workflows/charge/charge_event_building.yaml'
+WORKFLOW2='yamls/module1_flow/workflows/charge/charge_event_reconstruction.yaml'
+WORKFLOW3='yamls/module1_flow/workflows/combined/combined_reconstruction.yaml'
+WORKFLOW4='yamls/module1_flow/workflows/charge/prompt_calibration.yaml'
+WORKFLOW5='yamls/module1_flow/workflows/charge/final_calibration.yaml'
+
+$H5FLOW_CMD -c $WORKFLOW1 $WORKFLOW2 $WORKFLOW3 $WORKFLOW4 $WORKFLOW5 -i $CHARGE_INPUT_FILE -o $OUTPUT_FILE
+
+echo "Done!"
+echo "Output can be found at $OUTPUT_FILE"
+
+cd ${HERE}

scripts/module1_flow/run_module1_flow_example.sh

@@ -0,0 +1,43 @@
+#!/bin/bash
+# Runs module1_flow on an example file.
+#
+
+INPUT_FILE=$1
+
+OUTPUT_DIR=`pwd` #!!! change me
+OUTPUT_NAME=(${INPUT_FILE//"/"/ })
+OUTPUT_NAME=${OUTPUT_NAME[-1]}
+OUTPUT_FILE="${OUTPUT_DIR}/${OUTPUT_NAME}"
+OUTPUT_FILE=${OUTPUT_FILE//.h5/.module1_flow.h5}
+echo ${OUTPUT_FILE}
+
+# for running on a login node
+H5FLOW_CMD='h5flow'
+# for running on a single compute node with 32 cores
+#H5FLOW_CMD='srun -n32 h5flow'
+
+# run all stages
+WORKFLOW1='yamls/module1_flow/workflows/charge/charge_event_building.yaml'
+WORKFLOW2='yamls/module1_flow/workflows/charge/charge_event_reconstruction.yaml'
+WORKFLOW3='yamls/module1_flow/workflows/combined/combined_reconstruction.yaml'
+WORKFLOW4='yamls/module1_flow/workflows/charge/prompt_calibration.yaml'
+WORKFLOW5='yamls/module1_flow/workflows/charge/final_calibration.yaml'
+
+HERE=`pwd`
+#cd ndlar_flow
+# assumes this is being run from ndlar_flow/scripts/proto_nd_flow:
+cd ../../
+
+# avoid being asked if we want to overwrite the file if it exists.
+# this is us answering "yes".
+if [ -e $OUTPUT_FILE ]; then
+    rm -i $OUTPUT_FILE
+fi
+
+$H5FLOW_CMD -c $WORKFLOW1 $WORKFLOW2 $WORKFLOW3 $WORKFLOW4 $WORKFLOW5 -i $INPUT_FILE -o $OUTPUT_FILE
+
+echo "Done!"
+echo "Output can be found at $OUTPUT_FILE"
+
+cd ${HERE}
+

src/proto_nd_flow/reco/light/timestamp_corrector.py

@@ -47,7 +47,7 @@ def init(self, source_name):
 
         events_dset = self.data_manager.get_dset(source_name)
 
-        self.t_ns_dtype = self.t_ns_dtype(*events_dset.dtype['tai_ns'].shape[0:2])
+        self.t_ns_dtype = self.t_ns_dtype(*events_dset.dtype['tai_ns'].shape[0:1])
 
         self.slope_array = np.zeros(self.t_ns_dtype['t_ns'].shape)
         for key, val in self.slope.items():

yamls/module1_flow/notes.rst

@@ -0,0 +1,137 @@
+============
+module1_flow
+============
+
+Resources
+=========
+* ``yamls/module1_flow/resources/RunData.yaml``
+
+  Copied and modified from ``yaml/proto_nd_flow/resources/RunData.yaml``
+
+* ``yamls/module1_flow/resources/Geometry.yaml``
+
+  Copied and modified from ``yamls/proto_nd_flow/resources/Geometry.yaml`` I set ``det_geometry_file`` to ``data/module1_flow/module0.yaml``. I don't know if that's alright, and I don't think module0 accepted that option. ``crs_geometry_file`` is set to what I found in previously flowed file's metadata. I kept ``lrs_geometry_file`` the same as in proto_nd_flow, but I don't know if that works. Also, some module workflows had ``network_agnostic`` True and others False; not sure which is correct. 
+
+* ``yamls/module1_flow/resources/LArData.yaml``
+
+  Copied and modified from ``yaml/proto_nd_flow/resources/LArData.yaml``.
+
+  Previously flowed files had an ``electron_lifetime`` of 900 us.
+
+  ``module0_flow`` has ``electron_lifetime_file`` which is not in ``proto_nd_flow``. It looks like modules 2 and 3 workflows set the lifetime to 2600 us. module0 workflow gave both a lifetime file and an electron_lifetime. module0 also gives a vdrift mode and LAr temperature. 
+
+
+Workflows
+=========
+
+Charge
+------
+1. ``yamls/module1_flow/workflows/charge/charge_event_building.yaml``
+
+   Originally copied and modified from ``ndlar_flow/yamls/proto_nd_flow/workflows/charge/charge_event_building.yaml``. Only difference is that the ``.yaml`` files now point to ``module1_flow`` specific files.
+
+* ``yamls/module1_flow/reco/charge/RawEventGenerator.yaml``
+
+  Copied and modified from ``yamls/proto_nd_flow/reco/charge/RawEventGenerator.yaml``.
+
+  Removed ``mc_tracks_dset_name``.
+
+  Set ``nhit_cuts`` to 5.
+
+2. ``yamls/module1_flow/workflows/charge/charge_event_reconstruction.yaml``
+
+   Copied and modified from ``ndlar_flow/yamls/proto_nd_flow/workflows/charge/charge_event_reconstruction.yaml``
+
+* ``yamls/module1_flow/reco/charge/TimestampCorrector.yaml``
+
+  Copied and modified from ``yamls/proto_nd_flow/reco/charge/TimestampCorrector.yaml``
+  Using numbers I found in previously flowed file's metadata.
+
+* ``yamls/module1_flow/reco/charge/ExternalTriggerFinder.yaml``
+
+  Copied from ``yamls/proto_nd_flow/reco/charge/ExternalTriggerFinder.yaml``. No changes made .
+
+* ``yamls/module1_flow/reco/charge/RawHitBuilder.yaml``
+
+  Copied and modified from ``yamls/proto_nd_flow/reco/charge/RawHitBuilder.yaml``
+
+  Included ``configuration_file`` and ``pedestal_file``, and set them to what I found in previously flowed file's metadata. I think this was renamed from ``yamls/module0_flow/reco/charge/HitBuilder.yaml`` in the ``module0_flow``. In module workflows 2 and 3, the option ``network_agnostic: True`` is set. Not sure if we want this here too. 
+
+* ``yamls/module1_flow/reco/charge/EventBuilder.yaml``
+
+  Copied from ``yamls/proto_nd_flow/reco/charge/EventBuilder.yaml``. No changes.
+
+
+3. ``yamls/module1_flow/workflows/combined/combined_reconstruction.yaml``
+
+   Copied and modified from ``yamls/proto_nd_flow/workflows/combined/combined_reconstruction.yaml``. Only difference is that the ``.yaml`` files point to ``module1_flow`` specific files. ``proto_nd_flow`` only had a ``t0_reco`` step, while module[0,2,3] workflows have ``drift_reco``, ``electron_lifetime_corr``, ``tracklet_reco``, and module[0,2] workflows have ``tracklet_merge``.
+
+* ``yamls/module1_flow/reco/combined/T0Reconstruction.yaml``
+
+  Copied from ``yamls/proto_nd_flow/reco/combined/T0Reconstruction.yaml``. No changes made. Has an extra parameter compared to module[0,2,3] workflows called ``raw_hits_dset_name: 'charge/raw_hits'``.
+
+4. ``yamls/module1_flow/workflows/charge/prompt_calibration.yaml``
+
+   Copied and modified from yamls/proto_nd_flow/workflows/charge/prompt_calibration.yaml. Only difference is that the ``.yaml`` files point to ``module1_flow`` specific files. I don't see a corresponding file for modules[0,2,3] workflows.
+
+* ``yamls/module1_flow/reco/charge/CalibHitBuilder.yaml``
+
+  Copied and modified from ``yamls/proto_nd_flow/reco/charge/CalibHitBuilder.yaml``.
+
+  Added option for ``pedestal_file`` and ``configuration_file``, using inputs found in previoulsy flowed file metadata. I don't see this file for module[0,2,3] workflows.
+
+5. ``yamls/module1_flow/workflows/charge/final_calibration.yaml``
+
+   Copied and modified from ``yamls/proto_nd_flow/workflows/charge/final_calibration.yaml``. Only difference is that ``.yaml`` files now point to ``module1_flow`` specific files. Don't see corresponding file for module[0,2,3] workflows.
+
+* ``yamls/module1_flow/reco/charge/CalibHitMerger.yaml``
+
+  Copied and modified from ``yamls/proto_nd_flow/reco/charge/CalibHitMerger.yaml``. Maybe corresponds to ``yamls/module0_flow/reco/charge/HitMerger.yaml`` in ``module0``? Doesn't exist for module[2,3] workflows.
+
+  Removed ``mc_hit_frac_dset_name``.
+
+Light
+-----
+1. ``yamls/module1_flow/workflows/light/light_event_building_adc64.yaml``
+
+   Copied and modified from ``yamls/module3_flow/workflows/light/light_event_building_adc64.yaml``. The equivalent file did not exist in ``proto_nd_flow``. Only difference is that the ``.yaml`` files now point to a ``module1_flow`` specific file.
+
+* ``yamls/module1_flow/reco/light/LightADC64EventGenerator.yaml``
+
+  Copied and modified from ``yamls/module3_flow/reco/light/LightADC64EventGenerator.yaml``. Set the ``sn_table`` arguments, I need to remember from where.
+
+2. ``yamls/module1_flow/workflows/light/light_event_reconstruction.yaml``
+
+   Copied and modified from ``yamls/proto_nd_flow/workflows/light/light_event_reconstruction.yaml``. Only difference is that the ``.yaml`` files now point to a ``module1_flow`` specific file. Compared to module 0 workflow, there are three extra steps: ``wvfm_calib``, ``sipm_hit_finder``, ``sum_hit_finder``.
+
+* ``yamls/module1_flow/reco/light/LightTimestampCorrector.yaml``
+
+  Copied and modified from ``yamls/proto_nd_flow/reco/light/LightTimestampCorrector.yaml``. Changed ``slope`` to only have two TPC values. I noticed that all other modules have slopes (0: -1.18e-7, 1: 1.18e-7), while I kept them set to 0. Not sure what module1 wants. 
+
+* ``yamls/module1_flow/reco/light/WaveformNoiseFilter.yaml``
+
+  Copied from ``yamls/proto_nd_flow/reco/light/WaveformNoiseFilter.yaml``. Option ``filter_channels`` differs from others modules.
+
+* ``yamls/module1_flow/reco/light/WaveformDeconvolution.yaml``
+
+  Copied and modified from ``yamls/proto_nd_flow/reco/light/WaveformDeconvolution.yaml``. ``noise_spectrum_filename``, ``signal_spectrum_filename``, ``signal_impulse_filename`` were set to ``module0`` files. Option ``filter_channels`` differs from other modules.
+
+* ``yamls/module1_flow/reco/light/WaveformAlign.yaml``
+
+  Copied from ``yamls/proto_nd_flow/reco/light/WaveformAlign.yaml``. Is ``sim_latency`` a simulation parameter that should be removed? Other module workflows have ``busy_channel: All: 0`` parameter. 
+
+* ``yamls/module1_flow/reco/light/WaveformCalib.yaml``
+
+  Copied from ``yamls/proto_nd_flow/reco/light/WaveformCalib.yaml``. Looks like it needs to be updated. Does it assume 8 TPCs? Does everything ``gain_mc`` need to be removed? And does ``gain`` need to be updated. File doesn't exist for other module workflows.
+
+* ``yamls/module1_flow/reco/light/WaveformSum.yaml``
+
+  Copied from ``yamls/proto_nd_flow/reco/light/WaveformSum.yaml``. Other module workflows have ``gain`` and ``gain_mc`` parameters. 
+
+* ``yamls/module1_flow/reco/light/SiPMHitFinder.yaml``
+
+  Copied from ``yamls/proto_nd_flow/reco/light/SiPMHitFinder.yaml``. ``near_sample`` parameter is different. Here, ``threshold`` is a single constant, while other module workflows point to a ``siplm_threshold.yaml`` file. 
+
+* ``yamls/module1_flow/reco/light/SumHitFinder.yaml``
+
+  Copied from ``yamls/proto_nd_flow/reco/light/SumHitFinder.yaml``. Does not exist for other module workflows. Is ``threshold`` assuming 8 TPCSs? 

yamls/module1_flow/reco/charge/CalibHitBuilder.yaml

@@ -0,0 +1,27 @@
+# Copied and modified from yamls/proto_nd_flow/reco/charge/CalibHitBuilder.yaml
+
+classname: CalibHitBuilder # reco/charge/calib_hit_builder.py
+path: proto_nd_flow.reco.charge.calib_prompt_hits
+requires:
+  - 'charge/events'
+  - 'charge/raw_hits'
+  - 'charge/packets'
+  - 'combined/t0'
+  - name: 'charge/packets_index'
+    path: ['charge/packets']
+    index_only: True
+params:
+  # inputs
+  events_dset_name: 'charge/events'
+  packets_dset_name: 'charge/packets'
+  packets_index_name: 'charge/packets_index'
+  raw_hits_dset_name: 'charge/raw_hits'
+  t0_dset_name: 'combined/t0'
+
+  # output
+  calib_hits_dset_name: 'charge/calib_prompt_hits'
+
+  # configuration parameters
+
+  pedestal_file: '/global/cfs/cdirs/dune/www/data/Module1/TPC12/dataRuns/Pedestal/packet_2022_02_08_01_40_31_CETevd_ped.json'
+  configuration_file: '/global/cfs/cdirs/dune/www/data/Module1/TPC12/config_22-02-08_13-37-39.json'

yamls/module1_flow/reco/charge/CalibHitMerger.yaml

@@ -0,0 +1,24 @@
+#Copied and modified from yamls/proto_nd_flow/workflows/charge/final_calibration.yaml
+
+classname: CalibHitMerger # reco/charge/calib_hit_merger.py
+path: proto_nd_flow.reco.charge.calib_hit_merger
+requires:
+  - 'charge/events'
+  - 'charge/calib_prompt_hits'
+  - name: 'packet_frac_backtrack'
+    path: ['charge/calib_prompt_hits','charge/packets','mc_truth/packet_fraction']
+  - name: 'packet_seg_backtrack'
+    path: ['charge/calib_prompt_hits','charge/packets','mc_truth/segments']
+
+
+params:
+  # inputs
+  events_dset_name: 'charge/events'
+  hits_name: 'charge/calib_prompt_hits'
+  merged_name: 'charge/calib_final_hits'
+  max_contrib_segments: 200
+  merge_cut: 65 # merge hits with delta t < merge_cut [CRS ticks]
+  max_merge_steps: 50 # max number of iterations when merging
+                       # adjacent packets in time on the same channel
+
+

yamls/module1_flow/reco/charge/EventBuilder.yaml

@@ -0,0 +1,14 @@
+# Copied from yamls/proto_nd_flow/reco/charge/EventBuilder.yaml
+
+classname: EventBuilder
+path: proto_nd_flow.reco.charge.event_builder
+requires:
+  - 'charge/raw_hits'
+  - 'charge/ext_trigs'
+params:
+  # inputs
+  hits_dset_name: 'charge/raw_hits'
+  ext_trigs_dset_name: 'charge/ext_trigs'
+
+  # output
+  events_dset_name: 'charge/events'

yamls/module1_flow/reco/charge/ExternalTriggerFinder.yaml

@@ -0,0 +1,19 @@
+# Copied from yamls/proto_nd_flow/reco/charge/ExternalTriggerFinder.yaml
+
+classname: ExternalTriggerFinder
+path: proto_nd_flow.reco.charge.external_trigger_finder
+requires:
+  - 'charge/packets'
+  - name: 'charge/packets_corr_ts'
+    path: ['charge/packets', 'charge/packets_corr_ts']
+params:
+  # inputs
+  packets_dset_name: 'charge/packets'
+  ts_dset_name: 'charge/packets_corr_ts'
+
+  # output
+  ext_trigs_dset_name: 'charge/ext_trigs'
+
+  # configuration parameters
+  pacman_trigger_enabled: True
+  pacman_trigger_word_filter: 2

yamls/module1_flow/reco/charge/RawEventGenerator.yaml

@@ -0,0 +1,19 @@
+# Copied and modified from yamls/proto_nd_flow/reco/charge/RawEventGenerator.yaml 
+
+classname: RawEventGenerator
+path: proto_nd_flow.reco.charge.raw_event_generator
+dset_name: 'charge/raw_events'
+params:
+  # secondary output
+  packets_dset_name: 'charge/packets'
+
+  # configuration parameters
+  buffer_size: 384000
+  nhit_cut: 5
+  sync_noise_cut: [1000000, 11000000] # 1e6 cut based on Brooke's study
+  sync_noise_cut_enabled: True
+  event_builder_class: 'SymmetricWindowRawEventBuilder'
+  event_builder_config:
+    window: 1000 # slightly more than 1/2 drift length at 500V/cm
+    threshold: 10
+    rollover_ticks: 10000000 # PPS = 1e7 ticks

yamls/module1_flow/reco/charge/RawHitBuilder.yaml

@@ -0,0 +1,20 @@
+# Copied and modified from yamls/proto_nd_flow/reco/charge/RawHitBuilder.yaml
+
+classname: RawHitBuilder # reco/charge/raw_hit_builder.py
+path: proto_nd_flow.reco.charge.raw_hit_builder
+requires:
+  - 'charge/packets'
+  - name: 'charge/packets_corr_ts'
+    path: ['charge/packets', 'charge/packets_corr_ts']
+params:
+  # inputs
+  packets_dset_name: 'charge/packets'
+  ts_dset_name: 'charge/packets_corr_ts'
+
+  # output
+  hits_dset_name: 'charge/raw_hits'
+
+  # configuration parameters
+
+  configuration_file: '/global/cfs/cdirs/dune/www/data/Module1/TPC12/config_22-02-08_13-37-39.json'
+  pedestal_file: '/global/cfs/cdirs/dune/www/data/Module1/TPC12/dataRuns/Pedestal/packet_2022_02_08_01_40_31_CETevd_ped.json'