Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[5pt] PR: Incorporate Lidar-Informed elevations for bridges #1406

Open
wants to merge 55 commits into
base: dev
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
55 commits
Select commit Hold shift + click to select a range
289be25
added two new scripts to create lidar info for bridges
AliForghani-NOAA Dec 27, 2024
fdbdb6d
implementing bridges lidar info for healing REM
AliForghani-NOAA Dec 27, 2024
f59c182
assign crs to lidar rasters and redesigned creation of HUC6 elev diff…
AliForghani-NOAA Dec 28, 2024
0b69b12
fixed the alighment of rasters before making diff
AliForghani-NOAA Dec 28, 2024
4bcc6c3
added bridge lidar info into the FIM wrkflow
AliForghani-NOAA Jan 3, 2025
ae95b1a
added multiprocessing for making osm rasters using lidar las files
AliForghani-NOAA Jan 8, 2025
48a18dd
renamed the bridge data scripts and heal bridges elev using 3m cells …
AliForghani-NOAA Jan 14, 2025
204320a
added multiprocessing and logging for making dem diff
AliForghani-NOAA Jan 17, 2025
7d50bba
added logging into make osm rasters
AliForghani-NOAA Jan 17, 2025
8f754b1
added making vrt for diff raster files
AliForghani-NOAA Jan 18, 2025
fbd2ea5
made a copy of dem diff for branch zero
AliForghani-NOAA Jan 18, 2025
91aa6d4
temporarily used osm lines for an arbitrary HUC8 only for debugging
AliForghani-NOAA Jan 20, 2025
77e1056
Merge branch 'dev' into dev-lidar-bridge2
AliForghani-NOAA Jan 22, 2025
97ce7b5
applied linting
AliForghani-NOAA Jan 22, 2025
13a839a
updated pull osm to make two separate gpkg files for conus and alaska
AliForghani-NOAA Jan 22, 2025
0b2aca5
updated making raster s for alaska
AliForghani-NOAA Jan 22, 2025
b643993
updated dem diff to handle both HUC6 or HUC8 dems
AliForghani-NOAA Jan 23, 2025
e3a0a00
fixed projection issue in making lidar rasters
AliForghani-NOAA Jan 23, 2025
9142ea3
added a summary table showing classifications stats for all lidar data
AliForghani-NOAA Jan 23, 2025
bd5ec0a
fixed a linting
AliForghani-NOAA Jan 23, 2025
01e8993
added the dem diff raster links for Alaska
AliForghani-NOAA Jan 26, 2025
3eaed46
resolved merge conflict
AliForghani-NOAA Jan 27, 2025
f774856
changed clipping method for diff dem to near so neighboring cells do …
AliForghani-NOAA Jan 28, 2025
e9fc1d9
updated pre-clipping code with newalaska osm data
AliForghani-NOAA Jan 31, 2025
bc651f7
fix error msgs
RobHanna-NOAA Feb 3, 2025
b21650c
Merge Ali's update to Rob
RobHanna-NOAA Feb 3, 2025
6039452
Merge conflict fix
RobHanna-NOAA Feb 3, 2025
bd083e5
Merge branch 'dev' of https://github.com/NOAA-OWP/inundation-mapping …
RobHanna-NOAA Feb 3, 2025
bfbfa2d
linting fixes
RobHanna-NOAA Feb 3, 2025
d798914
linting
AliForghani-NOAA Feb 4, 2025
0577620
corrected no data
AliForghani-NOAA Feb 4, 2025
05e6b66
pull osm is good, pre-abandon fixes
RobHanna-NOAA Feb 4, 2025
76250cb
Merge branch 'dev-lidar-bridge2' of https://github.com/NOAA-OWP/inund…
RobHanna-NOAA Feb 4, 2025
c3dda2a
linting fixes
RobHanna-NOAA Feb 4, 2025
cdbdf17
linting fix plus bridge type changeds
RobHanna-NOAA Feb 5, 2025
72ec928
fixed zonal stats for small bridges
AliForghani-NOAA Feb 7, 2025
820b129
Merge branch 'dev-lidar-bridge2' of https://github.com/NOAA-OWP/inund…
AliForghani-NOAA Feb 7, 2025
4ba6774
linting bug fixed
AliForghani-NOAA Feb 7, 2025
5aa8278
pull osm done
RobHanna-NOAA Feb 8, 2025
dd41b01
Merge branch 'dev-lidar-bridge2' of https://github.com/NOAA-OWP/inund…
RobHanna-NOAA Feb 8, 2025
39c0dea
add more logs and progress bar for getting lidar
AliForghani-NOAA Feb 8, 2025
844dec3
added exception for getting lidar
AliForghani-NOAA Feb 8, 2025
5ae5728
used threshold terminalogy for non-lidar
AliForghani-NOAA Feb 10, 2025
a44754d
used threshold terminology in aggregate by huc
AliForghani-NOAA Feb 10, 2025
98df252
used threshold terminology in bridge_inundation.py
AliForghani-NOAA Feb 10, 2025
79a1484
create conda env tools and notes
RobHanna-NOAA Feb 10, 2025
3fefe88
udpate to ignore bridge txt file
RobHanna-NOAA Feb 10, 2025
05d85bc
linting fix
RobHanna-NOAA Feb 10, 2025
ecc87a1
fix .gitignore
RobHanna-NOAA Feb 12, 2025
bea7892
fix gitignore to skip txt and yml files
RobHanna-NOAA Feb 12, 2025
429402d
linting fixes
RobHanna-NOAA Feb 12, 2025
e70fced
linting exceptions
RobHanna-NOAA Feb 13, 2025
7207576
linting fixes
RobHanna-NOAA Feb 13, 2025
61b5f80
toml fixes
RobHanna-NOAA Feb 13, 2025
5667fd2
used ProcessPoolExecutor for parallel processing
AliForghani-NOAA Feb 13, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -16,3 +16,4 @@ config/*.swp
.private/
!tools/catfim/*.csv
!tools/catfim/catfim.env.template
!data/bridges/*.txt
278 changes: 278 additions & 0 deletions data/bridges/conda_fim_bridges_enviro.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,278 @@
name: fim_bridges
channels:
- conda-forge
- defaults
- https://repo.anaconda.com/pkgs/main
- https://repo.anaconda.com/pkgs/r
- https://repo.anaconda.com/pkgs/msys2
dependencies:
- _libavif_api=1.1.1=h57928b3_2
- affine=2.3.0=pyhd3eb1b0_0
- aom=3.9.1=he0c23c2_0
- attrs=24.3.0=py312haa95532_0
- aws-c-auth=0.8.1=hd11252f_0
- aws-c-cal=0.8.1=h099ea23_3
- aws-c-common=0.10.6=h2466b09_0
- aws-c-compression=0.3.0=h099ea23_5
- aws-c-event-stream=0.5.0=h85d8506_11
- aws-c-http=0.9.2=h3888f84_4
- aws-c-io=0.15.3=hc5a9e45_6
- aws-c-mqtt=0.11.0=h2c94728_12
- aws-c-s3=0.7.9=h6a47413_1
- aws-c-sdkutils=0.2.2=h099ea23_0
- aws-checksums=0.2.2=h099ea23_4
- aws-crt-cpp=0.29.9=he488853_2
- aws-sdk-cpp=1.11.489=h7d73209_0
- azure-core-cpp=1.14.0=haf5610f_0
- azure-identity-cpp=1.10.0=hd6deed7_0
- azure-storage-blobs-cpp=12.13.0=h3241184_1
- azure-storage-common-cpp=12.8.0=hd6deed7_1
- blas=1.0=mkl
- blosc=1.21.6=hfd34d9b_1
- boost-cpp=1.85.0=ha5ead02_4
- bottleneck=1.4.2=py312h4b0e54e_0
- branca=0.6.0=py312haa95532_0
- brotli-python=1.0.9=py312h5da7b33_9
- bzip2=1.0.8=h2bbff1b_6
- c-ares=1.34.4=h2466b09_0
- ca-certificates=2025.1.31=h56e8100_0
- cairo=1.18.2=h5782bbf_1
- capnproto=1.0.2=hb5d7e06_3
- ceres-solver=2.2.0=hd842749_5
- certifi=2025.1.31=py312haa95532_0
- cfitsio=4.5.0=h2b45a09_0
- charset-normalizer=3.3.2=pyhd3eb1b0_0
- click=8.1.7=py312haa95532_0
- click-plugins=1.1.1=pyhd3eb1b0_0
- cligj=0.7.2=pyhd3eb1b0_0
- colorama=0.4.6=py312haa95532_0
- contourpy=1.3.1=py312h214f63a_0
- cycler=0.11.0=pyhd3eb1b0_0
- dav1d=1.2.1=hcfcfb64_0
- draco=1.5.7=h181d51b_0
- eigen=3.4.0=h91493d7_0
- expat=2.6.4=h8ddb27b_0
- fiona=1.10.1=py312h6e88f47_3
- fmt=11.0.2=h7f575de_0
- folium=0.14.0=py312haa95532_0
- font-ttf-dejavu-sans-mono=2.37=hab24e00_0
- font-ttf-inconsolata=3.000=h77eed37_0
- font-ttf-source-code-pro=2.038=h77eed37_0
- font-ttf-ubuntu=0.83=h77eed37_3
- fontconfig=2.15.0=h765892d_1
- fonts-conda-ecosystem=1=0
- fonts-conda-forge=1=0
- fonttools=4.55.3=py312h827c3e9_0
- freetype=2.12.1=hdaf720e_2
- freexl=2.0.0=hd7a5696_0
- gdal=3.10.1=py312hc39d689_2
- geopandas=0.14.4=pyhd8ed1ab_0
- geopandas-base=0.14.4=pyha770c72_0
- geos=3.13.0=h5a68840_0
- geotiff=1.7.3=h496ac4d_3
- gflags=2.2.2=he0c23c2_1005
- glib=2.82.2=h3d4babf_1
- glib-tools=2.82.2=h4394cf3_1
- glog=0.7.1=h3ff59bf_0
- gmp=6.3.0=h537511b_0
- hdf4=4.2.15=h5557f11_7
- hdf5=1.14.3=nompi_hb2c4d47_109
- icc_rt=2022.1.0=h6049295_2
- icu=75.1=he0c23c2_0
- idna=3.7=py312haa95532_0
- intel-openmp=2023.1.0=h59b6b97_46320
- jinja2=3.1.5=py312haa95532_0
- joblib=1.4.2=py312haa95532_0
- kealib=1.6.1=hadd4d7f_0
- kiwisolver=1.4.8=py312h5da7b33_0
- krb5=1.21.3=hdf4eb48_0
- lcms2=2.17=hbcf6048_0
- lerc=4.0.0=h5da7b33_0
- libabseil=20240722.0=cxx17_h4eb7d71_4
- libaec=1.1.3=h63175ca_0
- libamd=3.3.3=ss783_h38ac50e
- libarchive=3.7.7=h979ed78_3
- libarrow=17.0.0=hc3bbc16_47_cpu
- libarrow-acero=17.0.0=h7d8d6a5_47_cpu
- libarrow-dataset=17.0.0=h7d8d6a5_47_cpu
- libavif16=1.1.1=h4d049a7_2
- libblas=3.9.0=1_h8933c1f_netlib
- libboost=1.85.0=h444863b_4
- libboost-devel=1.85.0=h91493d7_4
- libboost-headers=1.85.0=h57928b3_4
- libbrotlicommon=1.1.0=h2466b09_2
- libbrotlidec=1.1.0=h2466b09_2
- libbrotlienc=1.1.0=h2466b09_2
- libbtf=2.3.2=ss783_hde22806
- libcamd=3.3.3=ss783_hde22806
- libcblas=3.9.0=8_h719fc58_netlib
- libccolamd=3.3.4=ss783_hde22806
- libcholmod=5.3.0=ss783_h9a56889
- libcolamd=3.3.4=ss783_hde22806
- libcrc32c=1.1.2=h0e60522_0
- libcurl=8.11.1=h88aaa65_0
- libcxsparse=4.4.1=ss783_hde22806
- libde265=1.0.15=h91493d7_0
- libdeflate=1.23=h9062f6e_0
- libevent=2.1.12=h3671451_1
- libexpat=2.6.4=he0c23c2_0
- libffi=3.4.4=hd77b12b_1
- libgdal=3.10.1=hc25ceda_2
- libgdal-arrow-parquet=3.10.1=h9c6c97a_2
- libgdal-core=3.10.1=h095903c_2
- libgdal-fits=3.10.1=h20f9414_2
- libgdal-grib=3.10.1=h9921521_2
- libgdal-hdf4=3.10.1=hf9aff8f_2
- libgdal-hdf5=3.10.1=h7df419c_2
- libgdal-jp2openjpeg=3.10.1=h768cd86_2
- libgdal-kea=3.10.1=hfc54ade_2
- libgdal-netcdf=3.10.1=h3717446_2
- libgdal-pdf=3.10.1=h33ae9eb_2
- libgdal-pg=3.10.1=h5e54256_2
- libgdal-postgisraster=3.10.1=h5e54256_2
- libgdal-tiledb=3.10.1=h8d6a7ae_2
- libgdal-xls=3.10.1=hd0c044b_2
- libglib=2.82.2=h7025463_1
- libgoogle-cloud=2.34.0=h95c5cb2_0
- libgoogle-cloud-storage=2.34.0=he5eb982_0
- libgrpc=1.67.1=h0ac93cb_1
- libheif=1.19.5=gpl_hc631cee_100
- libiconv=1.17=hcfcfb64_2
- libintl=0.22.5=h5728263_3
- libintl-devel=0.22.5=h5728263_3
- libjpeg-turbo=3.0.0=hcfcfb64_1
- libklu=2.3.5=ss783_h77d05f4
- libkml=1.3.0=h538826c_1021
- liblapack=3.9.0=8_h719fc58_netlib
- libldl=3.3.2=ss783_hde22806
- liblzma=5.6.4=h2466b09_0
- libnetcdf=4.9.2=nompi_h008f77d_116
- libparquet=17.0.0=ha850022_47_cpu
- libparu=1.0.0=ss783_h21e6e03
- libpdal=2.8.3=h7c24d9f_1
- libpdal-arrow=2.8.3=ha7d42d9_1
- libpdal-core=2.8.3=h7b54269_1
- libpdal-draco=2.8.3=ha2c8d63_1
- libpdal-e57=2.8.3=hb58253e_1
- libpdal-hdf=2.8.3=hb9e256b_1
- libpdal-icebridge=2.8.3=hb9e256b_1
- libpdal-nitf=2.8.3=hd077b48_1
- libpdal-pgpointcloud=2.8.3=ha2ce333_1
- libpdal-tiledb=2.8.3=h0741228_1
- libpdal-trajectory=2.8.3=h1c12469_1
- libpng=1.6.46=had7236b_0
- libpq=17.2=h81f3393_1
- libprotobuf=5.28.3=h8309712_1
- librbio=4.3.4=ss783_hde22806
- libre2-11=2024.07.02=h4eb7d71_2
- librttopo=1.1.0=hd4c2148_17
- libspatialindex=1.9.3=h6c2663c_0
- libspatialite=5.1.0=h939089a_12
- libspex=3.2.1=ss783_hcfd7fc7
- libspqr=4.3.4=ss783_hc35ff37
- libsqlite=3.48.0=h67fdade_1
- libssh2=1.11.1=he619c9f_0
- libsuitesparseconfig=7.8.3=ss783_ha9923ec
- libthrift=0.21.0=hbe90ef8_0
- libtiff=4.7.0=h797046b_3
- libumfpack=6.3.5=ss783_h35348e5
- libutf8proc=2.10.0=hf9b99b7_0
- libwebp-base=1.5.0=h3b0e114_0
- libxcb=1.16=h013a479_1
- libxml2=2.13.5=he286e8c_1
- libxslt=1.1.39=h3df6e99_0
- libzip=1.11.2=h3135430_0
- libzlib=1.3.1=h2466b09_2
- lz4-c=1.10.0=h2466b09_1
- lzo=2.10=hcfcfb64_1001
- m2w64-gcc-libgfortran=5.3.0=6
- m2w64-gcc-libs=5.3.0=7
- m2w64-gcc-libs-core=5.3.0=7
- m2w64-gmp=6.1.0=2
- m2w64-libwinpthread-git=5.0.0.4634.697f757=2
- mapclassify=2.5.0=py312haa95532_0
- markupsafe=3.0.2=py312h827c3e9_0
- matplotlib-base=3.10.0=py312he19b0ae_0
- metis=5.1.0=h17e2fc9_1007
- minizip=4.0.7=h9fa1bad_3
- mkl=2023.1.0=h6b88ed4_46358
- mkl-service=2.4.0=py312h827c3e9_2
- mkl_fft=1.3.11=py312h827c3e9_0
- mkl_random=1.2.8=py312h0158946_0
- mpfr=4.2.1=h56c3642_0
- msys2-conda-epoch=20160418=1
- networkx=3.4.2=py312haa95532_0
- nitro=2.7.dev8=h1537add_0
- numexpr=2.10.1=py312h4cd664f_0
- numpy=1.26.4=py312hfd52020_0
- numpy-base=1.26.4=py312h4dde369_0
- openjpeg=2.5.3=h4d64b90_0
- openssl=3.4.0=ha4e3fda_1
- orc=2.0.3=haf104fe_2
- packaging=24.2=py312haa95532_0
- pandas=2.2.3=py312h5da7b33_0
- pcre2=10.44=h3d7b363_2
- pdal=2.8.3=hd8ed1ab_0
- pillow=10.4.0=py312h381445a_1
- pip=25.0=py312haa95532_0
- pixman=0.44.2=had0cd8c_0
- poppler=24.12.0=heaa0bce_2
- poppler-data=0.4.11=haa95532_1
- postgresql=17.2=h998eeb8_1
- proj=9.5.1=h4f671f6_0
- pthread-stubs=0.4=hcd874cb_1001
- pyparsing=3.2.0=py312haa95532_0
- pyproj=3.7.0=py312ha24589b_0
- pysocks=1.7.1=py312haa95532_0
- python=3.12.8=h3f84c4b_1_cpython
- python-dateutil=2.9.0post0=py312haa95532_2
- python-tzdata=2023.3=pyhd3eb1b0_0
- python_abi=3.12=5_cp312
- pytz=2024.1=py312haa95532_0
- qhull=2020.2=h59b6b97_2
- rasterio=1.4.3=py312hc0daee4_0
- rav1e=0.6.6=h975169c_2
- re2=2024.07.02=haf4117d_2
- requests=2.32.3=py312haa95532_1
- rioxarray=0.18.1=py312haa95532_0
- rtree=1.0.1=py312h2eaa2aa_0
- scikit-learn=1.6.1=py312h585ebfc_0
- scipy=1.15.1=py312hbb039d4_0
- setuptools=75.8.0=py312haa95532_0
- shapely=2.0.7=py312h0c580ee_0
- six=1.16.0=pyhd3eb1b0_1
- snappy=1.2.1=h500f7fa_1
- snuggs=1.4.7=pyhd3eb1b0_0
- spdlog=1.15.1=hf4138ee_0
- sqlite=3.45.3=h2bbff1b_0
- suitesparse=7.8.3=ss783_hf1e1ef2
- svt-av1=2.3.0=he0c23c2_0
- tbb=2021.8.0=h59b6b97_0
- threadpoolctl=3.5.0=py312hfc267ef_0
- tiledb=2.27.0=h13fc995_8
- tk=8.6.13=h5226925_1
- tqdm=4.67.1=py312hfc267ef_0
- tzdata=2025a=h04d1e81_0
- ucrt=10.0.22621.0=h57928b3_1
- unicodedata2=15.1.0=py312h827c3e9_1
- uriparser=0.9.8=h5a68840_0
- urllib3=2.3.0=py312haa95532_0
- vc=14.42=haa95532_4
- vc14_runtime=14.42.34433=h6356254_24
- vs2015_runtime=14.42.34433=hfef2bbc_24
- wheel=0.45.1=py312haa95532_0
- win_inet_pton=1.1.0=py312haa95532_0
- x265=3.5=h2d74725_3
- xarray=2024.11.0=py312haa95532_0
- xerces-c=3.2.5=he0c23c2_2
- xorg-libxau=1.0.11=hcd874cb_0
- xorg-libxdmcp=1.1.3=hcd874cb_0
- xyzservices=2022.9.0=py312haa95532_1
- xz=5.4.6=h8cc25b3_1
- zlib=1.3.1=h2466b09_2
- zstd=1.5.6=h0ea2cb4_0
- pip:
- laspy==2.5.4
- pdal-plugins==1.6.2
- python-pdal==0.0.1
prefix: C:\Users\rdp-user\anaconda3\envs\fim_bridges
Loading
Loading