Ahn missing links #262
Ahn missing links #262
Conversation
| laz_dir = ahn_laz_dir(context.resources.file_store.file_store.data_dir, 3) | ||
| laz_dir.mkdir(exist_ok=True, parents=True) | ||
| url_laz = tile_index_ahn[tile_id]["AHN3_LAZ"] | ||
| if url_laz is None or url_laz == "None": |
There was a problem hiding this comment.
It's more reliable if you use urllib to try to parse the url and check that the url is a valid url.
| url_laz = tile_index_ahn[tile_id]["AHN3_LAZ"] | ||
| if url_laz is None or url_laz == "None": | ||
| logger.error(f"Tile {tile_id} does not have a download link for AHN3.") | ||
| return Output( |
There was a problem hiding this comment.
We need to compute the file's hash (see below), because it needs to be added to the metadata table because the reconstruction depends on it.
|
This issue is related to #6 , because asset checks should verify is a file (still) exists on disk and it is not corrupted (eg hash matches the one in the metadata table). Having separate asset checks might allow us to decouple this file checking within the asset, so that the asset only deals with file downloads. I imagine that the asset check would run regularly to verify the files. If it succeeds, no download is needed, if it fails then the file must be downloaded again. |
No description provided.