-
Notifications
You must be signed in to change notification settings - Fork 10
Adding reanalysis data #211
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from 12 commits
Commits
Show all changes
14 commits
Select commit
Hold shift + click to select a range
8c96188
automated copernicus product selection
j-atkins 04b50b9
extend plan tool time range back to start of reanalysis period
j-atkins 3ccdc67
remove zooplankton variable from CTD_BGC
j-atkins d540e8e
catch errors where schedule spans two non-overlapping products
j-atkins b1a0d0d
change non-overlapping handling to return analysis product instead
j-atkins b42b829
update copernicus catalogue error class
j-atkins e90293a
update test_cli and test_fetch for new logic
j-atkins 1152c71
static schedule period into reanalysis period
j-atkins dd96cd7
add tests for product selection and time range validation
j-atkins 56e400c
refactor tests to use xarray for mock dataset
j-atkins 0552d3d
update documentation, describing the different coperncius products used
j-atkins cf75a9e
fix broken link
j-atkins 62f140a
re-label copernicus product information as documentation, move away f…
j-atkins 5f77bc3
fix typo in User Guide & Documentation link
j-atkins File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,81 @@ | ||
| # Additional information | ||
|
|
||
| This file contains additional technical information and guidance not covered in the [Quickstart](https://virtualship.readthedocs.io/en/latest/user-guide/quickstart.html) guide or in the [Tutorials](https://virtualship.readthedocs.io/en/latest/user-guide/tutorials/index.html). | ||
|
|
||
| ### Copernicus Marine products | ||
|
|
||
| VirtualShip supports running experiments anywhere in the global ocean from 1993 through to the present day (and approximately two weeks into the future), using the suite of products available from the [Copernicus Marine Data Store](https://data.marine.copernicus.eu/products). | ||
|
|
||
| The data sourcing task is handled by the `virtualship fetch` command. The three products relied on by `fetch` to source data for all [VirtualShip instruments](https://virtualship.readthedocs.io/en/latest/user-guide/assignments/Research_proposal_intro.html#Measurement-Options) (both physical and biogeochemical) are: | ||
|
|
||
| 1. **Reanalysis** (or "hindcast" for biogeochemistry). | ||
| 2. **Renalysis interim** (or "hindcast interim" for biogeochemistry). | ||
| 3. **Analysis & Forecast**. | ||
|
|
||
| ```{tip} | ||
| The Copernicus Marine Service describe the differences between the three products in greater detail [here](https://help.marine.copernicus.eu/en/articles/4872705-what-are-the-main-differences-between-nearrealtime-and-multiyear-products). | ||
| ``` | ||
|
|
||
| As a general rule of thumb the three different products span different periods across the historical period to present and are intended to allow for continuity across the previous ~ 30 years. | ||
|
|
||
| ```{note} | ||
| The ethos for automated dataset selection in `virtualship fetch` is to prioritise the Reanalysis/Hindcast products where possible (the 'work horse'), then _interim products where possible for continuity, and finally filling the very near-present (and near-future) temporal range with the Analysis & Forecast products. | ||
| ``` | ||
|
|
||
| ```{warning} | ||
| In the rare situation where the start and end times of an expedition schedule span different products, which is possible in the case of the end time being in the **Reanalysis_interim** period and the start time in the **Reanalysis** period, the **Analysis & Forecast** product will be automatically selected, as this spans back enough in time for this niche case. | ||
| ``` | ||
|
|
||
| ### Data availability | ||
|
|
||
| The following tables summarise which Copernicus product is selected by `virtualship fetch` per combination of time period and variable (see legend below). | ||
|
|
||
| For biogeochemical variables `ph` and `phyc`, monthly products are required for hindcast and hindcast interim periods. For all other variables, daily products are available. | ||
|
|
||
| #### Physical products | ||
|
|
||
| | Period | Product ID | Temporal Resolution | Typical Years Covered | Variables | | ||
| | :------------------ | :--------------------------------------- | :------------------ | :---------------------------------- | :------------------------- | | ||
| | Reanalysis | `cmems_mod_glo_phy_my_0.083deg_P1D-m` | Daily | ~30 years ago to ~5 years ago | `uo`, `vo`, `so`, `thetao` | | ||
| | Reanalysis Interim | `cmems_mod_glo_phy_myint_0.083deg_P1D-m` | Daily | ~5 years ago to ~2 months ago | `uo`, `vo`, `so`, `thetao` | | ||
| | Analysis & Forecast | `cmems_mod_glo_phy_anfc_0.083deg_P1D-m` | Daily | ~2 months ago to ~2 weeks in future | `uo`, `vo`, `so`, `thetao` | | ||
|
|
||
| --- | ||
|
|
||
| #### Biogeochemical products | ||
|
|
||
| | Period | Product ID | Temporal Resolution | Typical Years Covered | Variables | Notes | | ||
| | :---------------------------- | :----------------------------------------- | :------------------ | :---------------------------------- | :-------------------------------- | :------------------------------------- | | ||
| | Hindcast | `cmems_mod_glo_bgc_my_0.25deg_P1D-m` | Daily | ~30 years ago to ~5 years ago | `o2`, `chl`, `no3`, `po4`, `nppv` | Most BGC variables except `ph`, `phyc` | | ||
| | Hindcast (monthly) | `cmems_mod_glo_bgc_my_0.25deg_P1M-m` | Monthly | ~30 years ago to ~5 years ago | `ph`, `phyc` | Only `ph`, `phyc` (monthly only) | | ||
| | Hindcast Interim | `cmems_mod_glo_bgc_myint_0.25deg_P1D-m` | Daily | ~5 years ago to ~2 months ago | `o2`, `chl`, `no3`, `po4`, `nppv` | Most BGC variables except `ph`, `phyc` | | ||
| | Hindcast Interim (monthly) | `cmems_mod_glo_bgc_myint_0.25deg_P1M-m` | Monthly | ~5 years ago to ~2 months ago | `ph`, `phyc` | Only `ph`, `phyc` (monthly only) | | ||
| | Analysis & Forecast (O2) | `cmems_mod_glo_bgc-bio_anfc_0.25deg_P1D-m` | Daily | ~2 months ago to ~2 weeks in future | `o2` | | | ||
| | Analysis & Forecast (Chl) | `cmems_mod_glo_bgc-pft_anfc_0.25deg_P1D-m` | Daily | ~2 months ago to ~2 weeks in future | `chl`, `phyc` | | | ||
| | Analysis & Forecast (NO3/PO4) | `cmems_mod_glo_bgc-nut_anfc_0.25deg_P1D-m` | Daily | ~2 months ago to ~2 weeks in future | `no3`, `po4` | | | ||
| | Analysis & Forecast (PH) | `cmems_mod_glo_bgc-car_anfc_0.25deg_P1D-m` | Daily | ~2 months ago to ~2 weeks in future | `ph` | | | ||
| | Analysis & Forecast (NPPV) | `cmems_mod_glo_bgc-bio_anfc_0.25deg_P1D-m` | Daily | ~2 months ago to ~2 weeks in future | `nppv` | | | ||
|
|
||
| --- | ||
|
|
||
| ```{note} | ||
| * "Typical Years Covered" are approximate and subject to change with new Copernicus data releases. | ||
| * For the most up-to-date information, always consult the Copernicus Marine product documentation. | ||
| * Certain BGC variables (`ph`, `phyc`) are only available as monthly products in hindcast and hindcast interim periods. | ||
| ``` | ||
|
|
||
| ##### CMEMS variables legend | ||
|
|
||
| | Variable Code | Full Variable Name | Category | | ||
| | :------------ | :------------------------------------------------------------ | :------------- | | ||
| | **uo** | Eastward Sea Water Velocity | Physical | | ||
| | **vo** | Northward Sea Water Velocity | Physical | | ||
| | **so** | Sea Water Salinity | Physical | | ||
| | **thetao** | Sea Water Potential Temperature | Physical | | ||
| | **o2** | Mole Concentration of Dissolved Molecular Oxygen in Sea Water | Biogeochemical | | ||
| | **chl** | Mass Concentration of Chlorophyll a in Sea Water | Biogeochemical | | ||
| | **no3** | Mole Concentration of Nitrate in Sea Water | Biogeochemical | | ||
| | **po4** | Mole Concentration of Phosphate in Sea Water | Biogeochemical | | ||
| | **nppv** | Net Primary Production of Biomass | Biogeochemical | | ||
| | **ph** | Sea Water pH | Biogeochemical | | ||
| | **phyc** | Mole Concentration of Phytoplankton | Biogeochemical | | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -6,4 +6,5 @@ | |
| quickstart | ||
| tutorials/index | ||
| assignments/index | ||
| additional_information | ||
|
||
| ``` | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -80,6 +80,10 @@ For the underway ADCP, there is a choice of using the 38 kHz OceanObserver or th | |
|
|
||
| ### Waypoint datetimes | ||
|
|
||
| ```{note} | ||
| VirtualShip supports simulating experiments in the years 1993 through to the present day (and up to two weeks in the future) by leveraging the suite of products available Copernicus Marine Data Store (see [Fetch the data](#fetch-the-data)). The data download is automated based on the time period selected in the schedule. Different periods will rely on different products from the Copernicus Marine catalogue (see [Additional information](additional_information.md)). | ||
|
||
| ``` | ||
|
|
||
| You will need to enter dates and times for each of the sampling stations/waypoints selected in the MFP route planning stage. This can be done under _Schedule Editor_ > _Waypoints & Instrument Selection_ in the planning tool. | ||
|
|
||
| Each waypoint has its own sub-panel for parameter inputs (click on it to expand the selection options). Here, the time for each waypoint can be inputted. There is also an option to adjust the latitude/longitude coordinates and you can add or remove waypoints. | ||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'd consider calling this Documentation. The quick start guide and tutorials should not be the main source of information in my opinion.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good point, thanks!