EDTF fields are not validated in migration

[bseeger]

EDTF fields have no content validated when they are migrated. If an invalid EDTF value slips in the system via migration, the system will throw an error later on when someone views the record.

There is, too, the bigger question of: what, if any, content validation happens during migration and what could/should we do for validation during migration?

[bseeger]

Here's an example of some work that was done by the Islandora community to validate EDTF dates.

Islandora/migrate_islandora_csv#41

This combined with turning on the validate flag in our yml files might cover most validation needs.

[birkland]

It's possible this may be solved merely by a config change (enabling validation).

[jordandukart]

Just some general notes / thoughts:

• Islandora's EDTF field definition doesn't provide a constraint plugin and relies on widget validation currently. In terms of the "correct" approach adding a constraint to the field is probably the way to go. With that in mind, I'm not sure why that field never had one defined to begin with.
• The migrate plugin mentioned above will perform validation that the widget does via the UI albeit it's not necessarily fully complete EDTF validation to specification. If the intent was to mirror what the field accepts via the UI leveraging this plugin would be sufficient.
• However, there is an ongoing effort to potentially refine the above but don't believe there is an official timeline for completion. The meta issue may provide some more insight. If this further work comes to fruition and the overall invocation does not change this plugin would not need to be updated to take advantage of it.

In terms of solutions and their timelines there would be two options (imo):

1. Leveraging the plugin would be as trivial as adding the below to any migration utilizing an EDTF field:

- plugin: valid_edtf
   sets: {true|false depending on the field config}
   intervals: {true|false depending on the field config}

2. In the event the constraint work should be undertook the actual constraint plugin would likely take only a few hours. However, potential time it may take to have it merged into Islandora core would be a bit of question mark as well some further research into why the constraint approach wasn't undertook in the original implementation. If this path was chosen leveraging validate: true for the entity should cause this validation to occur. Would also likely look out how to roll out an update to existing fields.

[birkland]

Blocked awaiting advice on required validation for launch

[birkland]

Follow-up: can we confirm from Katie/michelle/stakeholders that we want the plugin enabled, owing to a lack of a constraint?

[birkland]

Michelle: Yes, we need this plugin enabled

[birkland]

@mjanowiecki will investigate whether this needs to be implemented for launch