Improvement suggestions

[falkamelung]

Here two suggestions
- run_files directory :
It would be good to have two directories:
miaplpy/run_files for run_01_* to run_03_* and
miaplpy/network_single_reference/run_files for run_04* to run_09_*

- interferogram.networkType option
Given that a directory network_sequential_3 or network_sequential_5 is generated, it would be helpful to have the ability to optionally specify the network as

interferogram.networkType=sequential_3
interferogram.networkType=sequential_5

with interferogram.numConn being read from the sequential_3 string. Similar for the Delaunay network options (which I have not tried). That would make it easier and cleaner to compare results for different networks

[falkamelung]

For MiNoPy there were some FAQs. We should transfer what is useful to MiaplPy.
geodesymiami/MiNoPy#60

[falkamelung]

Suggest to use a different symbol for the step number announcement then MintPy. This will make it easier to know whether it stalled in MiaplPy proper or MintPy. Currently we have:

****************** step 7 - ifgram_correction (MintPy) *********************
Generate /scratch/05861/tg851601/PichinchaSenDT142/miaplpy/network_sequential_3/run_files/run_07_mintpy_ifgram_correction
20221029:154352 * smallbaselineApp.py /scratch/05861/tg851601/PichinchaSenDT142/PichinchaSenDT142.txt --start modify_network --stop correct_unwrap_error --dir /scratch/05861/tg851601/PichinchaSenDT142/miaplpy/network_sequential_3


___________________________________________________________

  /##      /## /##             /##     /#######
 | ###    /###|__/            | ##    | ##__  ##
 | ####  /#### /## /#######  /######  | ##  \ ## /##   /##
 | ## ##/## ##| ##| ##__  ##|_  ##_/  | #######/| ##  | ##
 | ##  ###| ##| ##| ##  \ ##  | ##    | ##____/ | ##  | ##
 | ##\  # | ##| ##| ##  | ##  | ## /##| ##      | ##  | ##
 | ## \/  | ##| ##| ##  | ##  |  ####/| ##      |  #######
 |__/     |__/|__/|__/  |__/   \___/  |__/       \____  ##
                                                 /##  | ##
                                                |  ######/
   Miami InSAR Time-series software in Python    \______/
          MintPy 1.4.1-24, 2022-10-25
___________________________________________________________


read default template file: /scratch/05861/tg851601/PichinchaSenDT142/miaplpy/network_sequential_3/smallbaselineApp.cfg


******************** step - modify_network ********************

For MiaplPy we could use, e.g.:

########## step 7 - ifgram_correction (MintPy) ############

*#*#*#*#*#* step 7 - ifgram_correction (MintPy) *#*#*#*#*#*#*#*

or MiaplPy Step 7 - ifgram_correction (MintPy)

[falkamelung]

• We could simplify by moving everything network-related into the network_* folder, i.e. move
  /inverted/interferograms_single_reference into /network_single_reference/interferograms .

Edited 30 Oct:

• Can we generate /network_sequential_3/inputs when it is needed? Currently it is generated very early (at the beginning, I think) but stays empty for a long time

[falkamelung]

In the unwrapping step the *.o file may contain the NaN or infinity found in input float data error message below, which should be in the *.e file (Capturing the snaphu exit code may do it). See this issue.

cat run_05_miaplpy_unwrap_ifgram_0__10.o
20221029:213429 * unwrap_ifgram.py --ifg /scratch/05861/tg851601/MiamiSenAT48/miaplpy/inverted/interferograms_single_reference/20210104_20190912/filt_fine.int --coherence /scratch/05861/tg851601/MiamiSenAT48/miaplpy/inverted/tempCoh_average --unwrapped_ifg /scratch/05861/tg851601/MiamiSenAT48/miaplpy/inverted/interferograms_single_reference/20210104_20190912/filt_fine.unw --max_discontinuity 1.2 --init_method MCF --length 629 --width 1296 --height 698454.1600760925 --num_tiles 1 --earth_radius 6348066.309525905 --wavelength 0.05546576 -m /scratch/05861/tg851601/MiamiSenAT48/miaplpy/inverted/mask_unwrap --tmp
snaphu -f /scratch/05861/tg851601/MiamiSenAT48/miaplpy/inverted/interferograms_single_reference/20210104_20190912/config_all -d /scratch/05861/tg851601/MiamiSenAT48/miaplpy/inverted/interferograms_single_reference/20210104_20190912/filt_fine.int 1296 -o /scratch/05861/tg851601/MiamiSenAT48/miaplpy/inverted/interferograms_single_reference/20210104_20190912/filt_fine.unw
b'NaN or infinity found in input float data\nAbort\n'

[falkamelung]

miaplpyApp.py has the --tmp option but in the documentation we don't have much explanation on what it means (please excuse me if I have missed it). I would talk somewhere a bit about the IO issues (which to our knowledge are I issues only).

In the --help message it should say process on /tmp and not tmp.

Is it possible to have generate_ifgram.py and unwrap_ifgram.py using --tmp consistently, or possibly better, not at all? The first does not have the option and \tmp is part of a path which I would expect. However, the second has the option. I don't understand what it does. I would expect that /tmp is used only when given as part of a path.

[falkamelung]

When a previously run command is run again, are we consistent regarding overwriting? ISCE overwrites everything which is how it should be. generate_ifgram.py does not overwrite. If the output files exists already. The command

generate_ifgram.py --reference 20210104 --secondary 20190608 --output_dir /scratch/05861/tg851601/MiamiSenAT48/miaplpy/inverted/interferograms_single_reference/20210104_20190608 --azimuth_looks 1 --range_looks 1 --filter_strength 0 --stack_prefix tops --stack /tmp/phase_series.h5

only gives the message:

20221030:000724 * generate_ifgram.py --reference 20210104 --secondary 20190608 --output_dir /scratch/05861/tg851601/MiamiSenAT48/miaplpy/inverted/interferograms_single_reference/20210104_20190608 --azimuth_looks 1 --range_looks 1 --filter_strength 0 --stack_prefix tops --stack /tmp/phase_series.h5
/scratch/05861/tg851601/MiamiSenAT48/miaplpy/inverted/interferograms_single_reference/20210104_20190608

without doing anything.

[falkamelung]

Useful information to provide in --help, FAQs or elsewhere in the doc::

• Why is unwrap_ifgram.py given the height of the satellite and the earth radius? Some explanation in the --help would be helpful. Does this need to be given on the command line (instead of hardwired or in a constants file)? If that is something leftover and should be changed just say it.
• When you run miaplpyApp.py --jobfiles it says the following and then is silent for a few minutes. A message that it is doing something and what (writing job files or opening data files?) would be helpful:

input data files:
slc             : /scratch/05861/tg851601/MiamiSenAT48/merged/SLC/*/*.slc.full
number of slc             : 170

[falkamelung]

The order of arguments of generate_ifgram.py is counterintuitive. I would expect the input data and processing parameters at the beginning and the output_dir or file at the end.

generate_ifgram.py --reference 20170408 --secondary 20170806 --output_dir /scratch/05861/tg851601/PichinchaSenDT142/miaplpy/inverted/interferograms_single_reference/20170408_20170806 --azimuth_looks 1 --range_looks 1 --filter_strength 0 --stack_prefix tops --stack /scratch/05861/tg851601/PichinchaSenDT142/miaplpy/inverted/phase_series.h5

[falkamelung]

There are a few python scripts where the errors don't show up in stderr (*.e) file but in the *.o file. I suspect the error below is created by snaphu. Redirecting the error or checking for file existence prior to running snaphu may solve this.

unwrap_ifgram.py --ifg /scratch/05861/tg851601/MiamiTsxSMD14/miaplpy/inverted/interferograms_single_reference/20190407_20170923/filt_fine.int --coherence /scratch/05861/tg851601/MiamiTsxSMD14/miaplpy/inverted/tempCoh_average --unwrapped_ifg /scratch/05861/tg851601/MiamiTsxSMD14/miaplpy/inverted/interferograms_single_reference/20190407_20170923/filt_fine.unw --max_discontinuity 1.2 --init_method MCF --length 4058 --width 1420 --height 511035.8120019732 --num_tiles 1 --earth_radius 6348107.616403343 --wavelength 0.03106658115133631 -m /scratch/05861/tg851601/MiamiTsxSMD14/miaplpy/inverted/mask_unwrap --tmp
snaphu -f /scratch/05861/tg851601/MiamiTsxSMD14/miaplpy/inverted/interferograms_single_reference/20190407_20170923/config_all -d /scratch/05861/tg851601/MiamiTsxSMD14/miaplpy/inverted/interferograms_single_reference/20190407_20170923/filt_fine.int 1420 -o /scratch/05861/tg851601/MiamiTsxSMD14/miaplpy/inverted/interferograms_single_reference/20190407_20170923/filt_fine.unw
b"Can't open file /scratch/05861/tg851601/MiamiTsxSMD14/miaplpy/inverted/mask_unwrap\nAbort\n"

[falkamelung]

The number of connections is not consistent with ISCE naming: connNum versus --num_connections. So numConn would make more sense.

[falkamelung]

Make all options consistent with MIntPy. Instead of default: single_reference say auto for single_reference

miaplpy.interferograms.networkType             = auto     # [mini_stacks, single_reference, sequential, delaunay] default: single_reference

[falkamelung]

The Miami example data template files has some weired options. I would create an additional zenodo archive with a cleaned-up file:

• proper name (similar to Pichincha)
• no tropospheric correction
• use mintpy.reference.lalo
• use default values for window sizes (or change default?)
• no need to specify PsNumShp
• Are the mintpy unwrapError options important? If yes, explain somewhere. If not don't use them