Skip to content
data-schema / 2.20.0

data-schema 2.20.0

Install from the command line:
Learn more about npm packages
$ npm install @datastreamapp/data-schema@2.20.0
Install via package.json:
"@datastreamapp/data-schema": "2.20.0"

About this version

DataStream Logo
DataStream Open Data Schema for Water Quality Data

Download the latest version in JSON Schema, Google Sheets, and Excel template formats.

DataStream

DataStream (DataStream.org) is an online open-access platform for sharing water quality data. Data is uploaded, stored and shared in DataStream’s Open Data Schema -- a model based on the WQX standard for the Exchange of Water Quality Data. DataStream is free to use and allows users to query, visualize, and download data in this standardized format. To date, over 15 million water quality observations have been published across DataStream’s four regional platforms (Mackenzie DataStream, Lake Winnipeg DataStream, Atlantic DataStream, and Great Lakes DataStream) by watershed groups, Indigenous organizations, researchers and governments at all levels.

DataStream was developed by The Gordon Foundation and is carried out in collaboration with regional partners and monitoring networks. Data contributors maintain ownership of their data which are published under open data licenses.

DataStream Open Data Schema (DS-WQX)

To ensure consistent formatting of water data and to avoid ambiguous or missing information, we developed an observation-level data schema based on the WQX standard for the Exchange of Water Quality Data.

The WQX schema was developed by the US Environmental Protection Agency (EPA) and the US Geological Society (USGS) and is an implementation of the ESAR (Environmental Sampling, Analysis and Results) data standard. It was designed to enable multiple monitoring entities to share results in a common format. In the US, the WQX schema is used on the US EPA’s Water Quality Portal to share over 340 million water quality data records data from 400 federal, state, tribal and other partners.

In 2018, The Gordon Foundation led a comprehensive review process to determine how this model could be adapted to best meet the needs of diverse water monitoring initiatives in Canada. DataStream’s science and technical advisory team, regional partners and collaborators, data contributors, government representatives as well as other members of the water community were engaged in this process.

DataStream’s open data schema (DS-WQX v1.0) was fully implemented across all regional DataStream platforms in 2019.

Changelog

The DataStream open data schema will continue to evolve to meet user needs and therefore is subject to various updates over time (e.g. addition of new allowed values). To view the most recent version number and record of changes please see the DataStream Upload Template.

Dataset Metadata

In addition to the observation-level information in DS-WQX, DataStream uses dataset level metadata. DataStream Metadata

Special Case Tests

In addition to our schema enforcing allowed values the column conditional logic; we have included additional check for common errors to the frontend flavour of our schema.

  • Dissolved oxygen (DO) should not be in %
  • Dissolved oxygen saturation should not be less than 0%
  • Hardness should not be less than or equal to 0
  • pH should be within 0 and 14
  • Temperature should be within -100 degC and 100 degC

Testing

We aim to have our test be as robust as possible. This is accomplished by having acceptance and rejection tests.

Install

You can download the compiled DS-WQX schema from above.

Alternately, you can build it from the source to include in your project.

# Public
$ npm i
$ npm run build
$ cat primary/index.json

# Private
$ npm i @gordonfn/schema

Use

CSV Template

The csv template follows R import/export best practices.

JavaScript

import validate from '@gordonfn/schema'
const data = {} // Single row of data
const valid = validate(data)
if (!valid) console.error(validate.errors)

Schema Flavours

Supports JSON Schema Draft 2019-09 Specification in non-strict mode. Strict mode removes if, then, additionalProperties from the schema.

  • primary: This includes only JSON schema specification supported parameters with strict allowed values and supplementary conditional checks
  • frontend: Includes loose allowed values with coercion and supplementary conditional checks
  • extract: Includes strict allowed values and supplementary conditional checks
  • backend: Includes all possible allowed values, value coercion and no supplementary conditional checks
  • quality-control: Includes additional conditional checks to be flagged as warnings

Publish

# change version in package.json
npm test
cd dist
npm publish

Contributing

Commenting on the Schema

If you wish to comment on the schema please open an issue.

You can also email us at [email protected] or visit us at:

Development

brew install nvm
nvm get 12
npm i

Publishing

# update version in `package.json`
npm run test
cd dist
npm publish

References

The Gordon Foundation Logo

Details


Assets

  • data-schema-2.20.0.tgz

Download activity

  • Total downloads 12
  • Last 30 days 0
  • Last week 0
  • Today 0