[Project Proposal]: Online simple DarwinCore data format standard checker

[MathewBiddle]

Project Description

The IOOS Marine Life Data Network recommends aligning data that observes a species at a location and time to the DarwinCore Standard. Currently, there are various tools and resources available to check if a package matches the DarwinCore standard:

• https://www.gbif.org/tools/data-validator
• https://github.com/iobis/obistools
• https://github.com/cioos-siooc/pyobistools
• https://gbif.es/en/software/darwin-test/

Most of these checkers have some prerequisite requirements of installing additional software or creating an account in a system. Something that could be useful to the IOOS DMAC community would be an online web checker for DarwinCore, similar to the IOOS Compliance Checker - https://compliance.ioos.us/index.html, that doesn't require a log in and checks for additional aquatic characteristics in a dataset.

The idea is that this could be a relatively simple checker for the community to use to ensure their workflows are aligning with the standard.

Expected Outcomes

An online checker that allows a user to:

• Upload a data file (occurrence, extended measurementorfact, event, DNA derived data or other applicable extensions.) or a DarwinCoreArchive package
• Checks for compliance with the DarwinCore standard. If DarwinCore Archive, checks compliance with that standard.
  • Adds additional checks for aquatic data (eg. on land? contains depth information? taxonomy aligns to WoRMS?)
  • If metadata (EML) is provided, checks against that standard.
• Provides the response back as various outputs (pdf, html, json, etc.),
• Provides a web enabled API, similar to the compliance-checker API
• A compliant data file could be directly loaded to an IPT by an OBIS Node manager.

Skills required

• Depending on the source package used; R, Python, or BASH
• Experience with front end data file loading
• Experience with API development

Expertise

Intermediate

Topic Lead(s)

No response

Relevant links

Resources:

• https://www.gbif.org/darwin-core
• https://dwc.tdwg.org/terms/
• https://discourse.gbif.org/t/how-i-check-darwin-core-datasets/3833
• https://ipt.gbif.org/manual/en/ipt/latest/dwca-guide
• https://rs.gbif.org/extensions.html
• https://manual.obis.org/darwin_core.html

Tools:

• https://github.com/cioos-siooc/pyobistools
• https://github.com/iobis/obistools
• https://github.com/iobis/obis-qc
• https://gbif.es/en/software/darwin-test/
• https://www.gbif.org/tools/data-validator
• https://www.datafix.com.au/darwin-core-checker/
• https://rshiny.lifewatch.be/BioCheck/
• https://github.com/EMODnet/EMODnetBiocheck

[MathewBiddle]

xref:

• OBIStools-style QC library in Python #7
• GHA for Darwin Core quick check? bio_data_guide#85