This repository has been archived by the owner on Jun 30, 2023. It is now read-only.
Add target for a table of ScienceBase date last modified #98
Labels
documentation
Improvements or additions to documentation
pipeline
issues related to targets pipeline or tar_meta
There is a way to check the date that a ScienceBase item was last modified with
item_get_fields(<sb_id>, 'provenance')
.So, I think we can make a target for a table containing the provenance and other ScienceBase item information, and pass that table into the download target. In the download target, we can re-download only those items whose date changed.
Suggested steps:
p1_vars_of_interest
target, but I think a new target beforep1_vars_of_interest
would be better because the provenance should always be built when the pipeline is run. Usingcue = tar_cue(mode = "always')
should work to do that check.fetch_nhdv2_attributes_from_sb
to check the provenance column before downloading items from ScienceBase@lekoenig, @msleckman - tagging for your awareness of this option. Does this sound good to you?
The text was updated successfully, but these errors were encountered: