2.1.8 (FIM 5.1) - Deployment Checklist

[RobHanna-NOAA]

TI

Data Inputs Received

• HAND
• ras2fim / Ripple (n/a - no new data for this release)
• FIM Performance
• Catfim Stage Data
• Catfim Flow Data
• usgs rating curve
• agg_nwm_recurr flows

Data Ingests/Loads

• Crosswalk
• ras2fim Model (just adjust table to add model_version field)
• ras2fim (no new data load required)
• ras2fim Boundaries (n/a for this release)
• AEP
• Catchments - Branch 0
• Catchments - GMS
• USGS Elev Table
• Hydrotable - staggered
• usgs rating curve - staggered
• Skill Metrics
• FIM Performance
• CatFIM Stage
• CatFIM Stage - FIM 30
• CatFIM Flow
• Bridge Centroids
• Clear HAND cache
• Push injest code to GIT / TF

QA/QC

• Run "hv-vpp-ti-trigger-apocalyptic-tests" Step Function without errors
• Run "Core\Manual_Workflows\Align MAPX dtypes with DB.ipynb" Notebook and commit/PR/merge any changes
• Run "Core\Manual_Workflows\helper_functions\sync_services_with_master_list.py" and commit/PR/merge any changes

(Add steps here, if they come up, to manually check services)

---

Dump and move tables to S3

These are done using the "hv-vpp-ti-db-dumps-to-s3" Step Function.

EGIS tables

• AEP (egis)

  • aep_fim.rf_2_inundation
  • aep_fim.rf_5_inundation
  • aep_fim.rf_10_inundation
  • aep_fim.rf_25_inundation
  • aep_fim.rf_50_inundation
  • aep_fim.rf_high_water_inundation

• Catchments - branch 0 (egis)

  • fim_catchments.catchments_0_branches
  • fim_catchments.catchments_0_branches_hi
  • fim_catchments.catchments_0_branches_prvi
  • Note: These three databases have been renamed slightly, moving the word "catchments" to the front from previous versions:

• Catchments - gms branch (egis)

  • fim_catchments.catchments_gms_branches Problem: Won't load via Step functions
  • fim_catchments.catchments_gms_branches_hi
  • fim_catchments.catchments_gms_branches_prvi
  • Note: These three databases have been renamed slightly, moving the word "catchments" to the front from previous db versions:

• FIM Performance (egis)

  • reference.fim_performance_polys
  • reference.fim_performance_points
  • reference.fim_performance_catchments

• CatFIM Stage (egis)

  • reference.stage_based_catfim
  • reference.stage_based_catfim_sites

• CatFIM Flow (egis)

  • reference.flow_based_catfim
  • reference.flow_based_catfim_sites

• CatFIM Stage - FIM 30 (egis) (note.. there is no flow based version of this (I think))

  • reference.stage_based_catfim_fim_30
  • reference.stage_based_catfim_sites_fim_30
  • NOTE: These two CatFIM fim 30 sites are DB renames. In earlier versions, the phrase "public" which is now replaced with "fim_30"

• Skills metrics (egis) (note: made in viz.derived, but copied over to egis.reference for deployment)

  • reference.src_skill

• Bridge Centroids (egis) (needs reloading.. osm_id column is incorrect

  • reference.bridge_centroids (Note: New service and DB)

• DB cleanup script (egis) - give sql script to UAT for cleanup up UAT db;s. ie) dropping old, now renamed tables plus cleaning up old legacy non-realvent tables from previous release (like catfim_public tables), etc.

VIZ tables

• Crosswalk (viz)

  • derived.fim4_featureid_crosswalk

• ras2fim (viz)

  • ras2fim.geocurves (updated and needs data dump)
    Note: ras2fim.max_geocurves was not reloaded and the 4.5.2.11 are still valid

• ras2fim boundaries -> SKIP. ras2fim.boundaries were not reloaded and the 4.5.2.11 are still valid

• usgs elev (viz)

  • derived.usgs_elev_table

• Hydrotable staggered (viz)

  • derived.hydrotable_staggered

• usgs rating curves staggered (viz)

  • derived.usgs_rating_curves_staggered

• HANDFIM_cache DBs (viz)

  • handfim_cache.hydrotable_cached
  • handfim_cache.hydrotable_cached_geo
  • handfim_cache.hydrotable_cached_max
  • handfim_cache.hydrotable_cached_zero_stage

• derived.channels_ak (renamed from derived.channels_alaska) : Note: Only needs a reload when a new NWM version comes out.

• DB cleanup script (viz) - give sql script to UAT for cleanup up UAT db;s. ie) dropping old, now renamed tables plus cleaning up old legacy non-realvent tables from previous release (like catfim_public tables), etc.

UAT

Pre-Deploy

• Move new FIM version data in S3 to UAT
• Move VIZ DB dump files to UAT
• Move EGIS DB dump files to UAT
• Run VIZ DB cleanup scripts
• Run EGIS DB cleanup scripts

Post-Deploy

• Manually (for now) execute hv-vpp-uat-viz-publish-service Lambda for all new/updated static services
• Manually execute SQL as documented in Add step to sync-wrds-db step function that recreates foreign tables #1050

Prod

Pre-Deploy

• Move VIZ DB dump files to Prod
• Move EGIS DB dump files to Prod
• Run VIZ DB cleanup scripts
• Run EGIS DB cleanup scripts

Post-Deploy

• Manually (for now) execute hv-vpp-prod-viz-publish-service Lambda for all new/updated static services
• If Add step to sync-wrds-db step function that recreates foreign tables #1050 hasn't been completed and its solution code deployed to prod, manually execute SQL as documented in Add step to sync-wrds-db step function that recreates foreign tables #1050