Copernicus GLO-30 has now been released for Azerbaijan, Georgia, and Armenia

[EJFielding]

The bug

Copernicus GLO-30 has now been released for Azerbaijan, Georgia, and Armenia since December 2023. The scripts still download only the GLO-90 data for those cells.

To Reproduce

Run dem_stitcher for area including Azerbaijan. The scripts download GLO_90 there.

Opening glo_30 Datasets: 100%|███████████████████████████████████████████████████████████████████████████████| 42/42 [00:18<00:00, 2.25it/s]
Reading tile metadata: 100%|████████████████████████████████████████████████████████████████████████████████| 42/42 [00:00<00:00, 665.60it/s]
Reading tile imagery: 100%|██████████████████████████████████████████████████████████████████████████████████| 42/42 [01:15<00:00, 1.80s/it]
Opening glo_90_missing Datasets: 100%|███████████████████████████████████████████████████████████████████████| 12/12 [00:06<00:00, 1.89it/s]
Reading tile metadata: 100%|█████████████████████████████████████████████████████████████████████████████| 12/12 [00:00<00:00, 208845.01it/s]
Reading tile imagery: 100%|██████████████████████████████████████████████████████████████████████████████████| 12/12 [00:09<00:00, 1.28it/s]
fixing xml file path for file: isce_dem/full_res.dem.wgs84

Additional context

[EJFielding]

The lack of GLO_30 for these cells was reported in #34, but Copernius has now released GLO_30 there.

[cmarshak]

Thanks for letting me know.

The tiles for these areas are not in the glo_30 geojson collection and have to be added back. I will try to get to it this week - but can't promise.

[EJFielding]

Thanks for adding those GLO_30 cells to the collection when you have time.

[cmarshak]

It appears this is a bit more nuanced in that these new tiles seem to be available only through OpenTopography or through ESA [1, 2], not the bucket we are using. The links in [2] weren't working during the writing of this comment so had a hard time finding the tiles you were mentioning. I didn't have time to go into open topography and it would be a bigger lift to have open topography access here (assuming these tiles are there).

If you have more information about the location of these new tiles, that would be helpful, @EJFielding.

I double checked the available tiles from the bucket used in this library (i.e. the bucket copernicus-dem-30m) as demonstrated in notebook [3] and the tiles you mentioned have not been updated there. I checked via the command line the tiles that are not there:

aws s3 ls --no-sign-request s3://copernicus-dem-30m/Copernicus_DSM_COG_10_N41_00_E046_00_DEM/

However, if we look for the same tiles in the 90m collection:

aws s3 ls --no-sign-request s3://copernicus-dem-90m/Copernicus_DSM_COG_30_N41_00_E046_00_DEM/

We get some results. Similarly, going over west by 7 degrees in 30m collection to make sure I did correctly in 30m collection.

aws s3 ls --no-sign-request s3://copernicus-dem-30m/Copernicus_DSM_COG_10_N41_00_E039_00_DEM/

yields

                           PRE AUXFILES/
                           PRE INFO/
                           PRE PREVIEW/
2022-05-09 07:21:59      44713 Copernicus_DSM_10_N41_00_E039_00.xml
2022-05-09 07:21:59    2074839 Copernicus_DSM_COG_10_N41_00_E039_00_DEM.tif

A complete list of tiles in s3 bucket for 30 m resolution is here: https://copernicus-dem-30m.s3.amazonaws.com/tileList.txt

I was hoping that ASF did the hard work already and mirrored these new 30m DEMs for hyp3 on-demand plugins, too, but I am not sure they have yet: https://github.com/ASFHyP3/hyp3-gamma/blob/029e96061e3c7ab979b96b4dde06293642701b02/hyp3_gamma/dem.py#L13C25-L13C63

I looked at the geojson and this again appears to be putting the cop90 data there.

I looked in the ASF bucket using [3] and got all the "geojson" suffix files:

[{'Key': 'v2/cop30-2021-us-west-2-mirror.geojson',
  'LastModified': datetime.datetime(2022, 5, 27, 7, 4, 34, tzinfo=tzutc()),
  'ETag': '"c5709fc57c6ce55f9c95b470e822ef71"',
  'Size': 9474902,
  'StorageClass': 'INTELLIGENT_TIERING'},
 {'Key': 'v2/cop30-2021-with-cop90-us-west-2-mirror.geojson',
  'LastModified': datetime.datetime(2023, 9, 29, 17, 11, tzinfo=tzutc()),
  'ETag': '"848c1899f40fcd589c2568e947f8b9a8"',
  'Size': 14229924,
  'StorageClass': 'INTELLIGENT_TIERING'},
 {'Key': 'v2/cop30-2021.geojson',
  'LastModified': datetime.datetime(2022, 11, 22, 0, 12, 53, tzinfo=tzutc()),
  'ETag': '"1e39f7a00761324deb98c8383c8cb7b9-2"',
  'Size': 9263302,
  'StorageClass': 'INTELLIGENT_TIERING'},
 {'Key': 'v2/srtmgl1_coverage.geojson',
  'LastModified': datetime.datetime(2021, 1, 9, 3, 0, 35, tzinfo=tzutc()),
  'ETag': '"288fc1cd32708b2445e1c8b1f7085c48"',
  'Size': 175002,
  'StorageClass': 'INTELLIGENT_TIERING'}]

I haven't carefully inspected them all, but my guess is the new 30 m tiles are not there. We can ask @jhkennedy and @asjohnston-asf, when they are back from holiday/have time.

[1] https://portal.opentopography.org/raster?opentopoID=OTSDEM.032021.4326.3
[2] https://sentinels.copernicus.eu/web/sentinel/-/copernicus-dem-new-direct-data-download-access
[3] https://github.com/ACCESS-Cloud-Based-InSAR/dem-stitcher/blob/dev/notebooks/organize_tile_data/0_Format_and_Organize_Data.ipynb