Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is what caused the error.
When we started using AHN5 (about a year ago), we had noticed that pdal was failing for all the new tiles with this error:
(you can read more about this issue here )
Back then, we had decided that, even if pdal failed, the downloading should continue and only an empy dict should be stored in the DB.
However, this solution stopped working when we started using the pdal_info column from that table to get the filename from
I fixed this by storing the filename in the column, even if pdal fails - this will allow for the pipeline to continue.
TO LOOK INTO: Even the tiles that have a succesfull pdal info output they have this error:
TO LOOK INTO2: One more thing that I noticed is that we are running pdal info with the flag
--all
. This takes WAAAAY too long for each tile. Indicatively, it took 4' to download a single tile and 38 to get the metadata:Are we actually using this pdal output for anything else apart from retrieving the filename? Maybe in validation? I couldn't find anything. If not, I suggest removing it.
Also in this PR, I am building the new tools image.