-
- 
+
+ 
+
- Iris is a powerful, format-agnostic, community-driven Python library for + Iris is a powerful, format-agnostic, community-driven Python package for analysing and visualising Earth science data
+| | | +|------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| ⚙️ CI | [](https://github.com/SciTools/iris/actions/workflows/ci-manifest.yml) [](https://github.com/SciTools/iris/actions/workflows/ci-tests.yml) [](https://github.com/SciTools/iris/actions/workflows/ci-wheels.yml) [](https://results.pre-commit.ci/latest/github/SciTools/iris/main) | +| 💬 Community | [](https://www.contributor-covenant.org/version/2/1/code_of_conduct/) [](https://github.com/SciTools/iris/discussions) [](https://twitter.com/scitools_iris) | +| 📖 Documentation | [](https://scitools-iris.readthedocs.io/en/latest/?badge=latest) | +| 📈 Health | [](https://codecov.io/gh/SciTools/iris) | +| ✨ Meta | [](https://github.com/astral-sh/ruff) [](https://numpy.org/neps/nep-0029-deprecation_policy.html) [](https://github.com/SciTools/iris/blob/main/LICENSE) [](https://anaconda.org/conda-forge/iris) | +| 📦 Package | [](https://doi.org/10.5281/zenodo.595182) [](https://anaconda.org/conda-forge/iris) [](https://pypi.org/project/scitools-iris/) [](https://pypi.org/project/scitools-iris/) | +| 🧰 Repo | [](https://github.com/SciTools/iris/commits/main) [](https://github.com/SciTools/iris/graphs/contributors) [](https://github.com/SciTools/iris/releases) | +| | +
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
+For documentation see the
+latest
+developer version or the most recent released
+stable version.
- - - -
Table of contents
- -[](TOC) - -+ [Overview](#overview) -+ [Documentation](#documentation) -+ [Installation](#installation) -+ [Copyright and licence](#copyright-and-licence) -+ [Get in touch](#get-in-touch) -+ [Contributing](#contributing) - -[](TOC) - -# Overview - -Iris implements a data model based on the [CF conventions](http://cfconventions.org/) -giving you a powerful, format-agnostic interface for working with your data. -It excels when working with multi-dimensional Earth Science data, where tabular -representations become unwieldy and inefficient. - -[CF Standard names](http://cfconventions.org/standard-names.html), -[units](https://github.com/SciTools/cf_units), and coordinate metadata -are built into Iris, giving you a rich and expressive interface for maintaining -an accurate representation of your data. Its treatment of data and - associated metadata as first-class objects includes: - - * a visualisation interface based on [matplotlib](https://matplotlib.org/) and - [cartopy](https://scitools.org.uk/cartopy/docs/latest/), - * unit conversion, - * subsetting and extraction, - * merge and concatenate, - * aggregations and reductions (including min, max, mean and weighted averages), - * interpolation and regridding (including nearest-neighbor, linear and area-weighted), and - * operator overloads (``+``, ``-``, ``*``, ``/``, etc.) - -A number of file formats are recognised by Iris, including CF-compliant NetCDF, GRIB, -and PP, and it has a plugin architecture to allow other formats to be added seamlessly. - -Building upon [NumPy](http://www.numpy.org/) and [dask](https://dask.pydata.org/en/latest/), -Iris scales from efficient single-machine workflows right through to multi-core clusters and HPC. -Interoperability with packages from the wider scientific Python ecosystem comes from Iris' -use of standard NumPy/dask arrays as its underlying data storage. +## [#ShowYourStripes](https://showyourstripes.info/s/globe) -# Documentation - - The documentation for *stable released versions* of Iris, including a user guide, example code, and gallery.
-
- The documentation for the *latest development version* of Iris.
-
-
-# Installation
-
-The easiest way to install Iris is with [conda](https://conda.io/miniconda.html):
-
- conda install -c conda-forge iris
-
-Detailed instructions, including information on installing from source,
-are available in [INSTALL](INSTALL).
-
-# Get in touch
-
- * Report bugs, or suggest new features using an Issue or Pull Request on [Github](https://github.com/SciTools/iris). You can also comment on existing Issues and Pull Requests.
- * For discussions from a user perspective you could join our [SciTools Users Google Group](https://groups.google.com/forum/#!forum/scitools-iris).
- * For those involved in developing Iris we also have an [Iris Developers Google Group](https://groups.google.com/forum/#!forum/scitools-iris-dev).
- * [StackOverflow](https://stackoverflow.com/questions/tagged/python-iris) For "How do I?".
+
+
+ 
+
-# Copyright and licence
+**Graphics and Lead Scientist**: [Ed Hawkins](https://www.met.reading.ac.uk/~ed/home/index.php), National Centre for Atmospheric Science, University of Reading.
-Iris may be freely distributed, modified and used commercially under the terms
-of its [GNU LGPLv3 license](COPYING.LESSER).
+**Data**: Berkeley Earth, NOAA, UK Met Office, MeteoSwiss, DWD, SMHI, UoR, Meteo France & ZAMG.
-# Contributing
-Information on how to contribute can be found in the [Iris developer guide](https://scitools.org.uk/iris/docs/latest/developers_guide/index.html).
+
+#ShowYourStripes is distributed under a
+Creative Commons Attribution 4.0 International License
+
+ 
+
+
+
+ Performance shifts
+ + ``` + {results_shifts} + ``` + +
+
+
+ Generated by GHA run {gha_run_link}
+ """
+ )
+ )
+ performance_report = performance_report.format(
+ commit_sha=commit_sha,
+ results_shifts=results_shifts,
+ results_full=results_full,
+ gha_run_link=gha_run_link,
+ )
+
+ if on_pull_request:
+ # Command to post the report as a comment on the active PR.
+ body_path.write_text(performance_report)
+ command = (
+ f"gh pr comment {pr_number} "
+ f"--body-file {body_path.absolute()} "
+ f"--repo {repo}"
+ )
+ command_path.write_text(command)
+
+ else:
+ # Command to post the report as new issue.
+ commit_msg = _subprocess_runner_capture(
+ f"git log {commit_sha}^! --oneline".split(" ")
+ )
+ # Intended for benchmarking commits on trunk - should include a PR
+ # number due to our squash policy.
+ pr_tag_match = re.search("#[0-9]*", commit_msg)
+
+ assignee = ""
+ pr_tag = "pull request number unavailable"
+ if pr_tag_match is not None:
+ pr_tag = pr_tag_match.group(0)
+
+ for login_type in ("author", "mergedBy"):
+ gh_query = f'.["{login_type}"]["login"]'
+ commandlist = shlex.split(
+ f"gh pr view {pr_tag[1:]} "
+ f"--json {login_type} -q '{gh_query}' "
+ f"--repo {repo}"
+ )
+ login = _subprocess_runner_capture(commandlist)
+
+ commandlist = [
+ "curl",
+ "-s",
+ f"https://api.github.com/users/{login}",
+ ]
+ login_info = _subprocess_runner_capture(commandlist)
+ is_user = '"type": "User"' in login_info
+ if is_user:
+ assignee = login
+ break
+
+ title = f"Performance Shift(s): `{commit_sha}`"
+ body = dedent(
+ (
+ f"""
+ Benchmark comparison has identified performance shifts at:
+
+ * commit {commit_sha} ({pr_tag}).
+
+ Full benchmark results
+ + ``` + {results_full} + ``` + ++ Please review the report below and + take corrective/congratulatory action as appropriate + :slightly_smiling_face: +
+ """ + ) + ) + body += performance_report + body_path.write_text(body) + + command = ( + "gh issue create " + f'--title "{title}" ' + f"--body-file {body_path.absolute()} " + '--label "Bot" ' + '--label "Type: Performance" ' + f"--repo {repo}" + ) + if assignee: + command += f" --assignee {assignee}" + command_path.write_text(command) + + +def _gh_post_reports() -> None: + """If running under GitHub Actions: post pre-prepared benchmark reports. + + Reports are prepared by :func:`_gh_create_reports`, which must be run + within a separate action to comply with GHA's security limitations. + """ + if "GITHUB_ACTIONS" not in environ: + # Only run when within GHA. + return + + commit_dirs = [x for x in GH_REPORT_DIR.iterdir() if x.is_dir()] + for commit_dir in commit_dirs: + command_path = commit_dir / "command.txt" + command = command_path.read_text() + + # Security: only accept certain commands to run. + assert command.startswith(("gh issue create", "gh pr comment")) + + _subprocess_runner(shlex.split(command)) + + +class _SubParserGenerator(ABC): + """Convenience for holding all the necessary argparse info in 1 place.""" + + name: str = NotImplemented + description: str = NotImplemented + epilog: str = NotImplemented + + class _SubParsersType(Protocol): + """Duck typing since argparse._SubParsersAction is private.""" + + def add_parser(self, name, **kwargs) -> argparse.ArgumentParser: ... + + def __init__(self, subparsers: _SubParsersType) -> None: + self.subparser = subparsers.add_parser( + self.name, + description=self.description, + epilog=self.epilog, + formatter_class=argparse.RawTextHelpFormatter, + ) + self.add_arguments() + self.add_asv_arguments() + self.subparser.set_defaults(func=self.func) + + @abstractmethod + def add_arguments(self) -> None: + """All custom self.subparser.add_argument() calls.""" + _ = NotImplemented + + def add_asv_arguments(self) -> None: + self.subparser.add_argument( + "asv_args", + nargs=argparse.REMAINDER, + help="Any number of arguments to pass down to the ASV benchmark command.", + ) + + @staticmethod + @abstractmethod + def func(args: argparse.Namespace): + """Return when the subparser is parsed. + + `func` is then called, performing the user's selected sub-command. + + """ + _ = args + return NotImplemented + + +class Overnight(_SubParserGenerator): + name = "overnight" + description = ( + "Benchmarks all commits between the input **first_commit** to ``HEAD``, " + "comparing each to its parent for performance shifts. If running on " + "GitHub Actions: performance shift(s) will be reported in a new issue.\n" + "Designed for checking the previous 24 hours' commits, typically in a " + "scheduled script.\n" + "Uses `asv run`." + ) + epilog = ( + "e.g. python bm_runner.py overnight a1b23d4\n" + "e.g. python bm_runner.py overnight a1b23d4 --bench=regridding" + ) + + def add_arguments(self) -> None: + self.subparser.add_argument( + "first_commit", + type=str, + help="The first commit in the benchmarking commit sequence.", + ) + + @staticmethod + def func(args: argparse.Namespace) -> None: + _setup_common() + + commit_range = f"{args.first_commit}^^.." + # git rev-list --first-parent is the command ASV uses. + git_command = shlex.split(f"git rev-list --first-parent {commit_range}") + commit_string = _subprocess_runner_capture(git_command) + commit_list = commit_string.split("\n") + + asv_command = shlex.split(ASV_HARNESS.format(posargs=commit_range)) + try: + _subprocess_runner([*asv_command, *args.asv_args], asv=True) + finally: + # Designed for long running - want to compare/post any valid + # results even if some are broken. + _asv_compare(*reversed(commit_list), overnight_mode=True) + + +class Branch(_SubParserGenerator): + name = "branch" + description = ( + "Performs the same operations as ``overnight``, but always on two commits " + "only - ``HEAD``, and ``HEAD``'s merge-base with the input " + "**base_branch**. If running on GitHub Actions: HEAD will be GitHub's " + "merge commit and merge-base will be the merge target. Performance " + "comparisons will be posted in a comment on the relevant pull request.\n" + "Designed " + "for testing if the active branch's changes cause performance shifts - " + "anticipating what would be caught by ``overnight`` once merged.\n\n" + "**For maximum accuracy, avoid using the machine that is running this " + "session. Run time could be >1 hour for the full benchmark suite.**\n" + "Uses `asv run`." + ) + epilog = ( + "e.g. python bm_runner.py branch upstream/main\n" + "e.g. python bm_runner.py branch upstream/main --bench=regridding" + ) + + def add_arguments(self) -> None: + self.subparser.add_argument( + "base_branch", + type=str, + help="A branch that has the merge-base with ``HEAD`` - ``HEAD`` will be benchmarked against that merge-base.", + ) + + @staticmethod + def func(args: argparse.Namespace) -> None: + _setup_common() + + git_command = shlex.split("git rev-parse HEAD") + head_sha = _subprocess_runner_capture(git_command)[:8] + + git_command = shlex.split(f"git merge-base {head_sha} {args.base_branch}") + merge_base = _subprocess_runner_capture(git_command)[:8] + + with NamedTemporaryFile("w") as hashfile: + hashfile.writelines([merge_base, "\n", head_sha]) + hashfile.flush() + commit_range = f"HASHFILE:{hashfile.name}" + asv_command = shlex.split(ASV_HARNESS.format(posargs=commit_range)) + _subprocess_runner([*asv_command, *args.asv_args], asv=True) + + _asv_compare(merge_base, head_sha) + + +class _CSPerf(_SubParserGenerator, ABC): + """Common code used by both CPerf and SPerf.""" + + description = ( + "Run the on-demand {} suite of benchmarks (part of the UK Met " + "Office NG-VAT project) for the ``HEAD`` of ``upstream/main`` only, " + "and publish the results to the input **publish_dir**, within a " + "unique subdirectory for this run.\n" + "Uses `asv run`." + ) + epilog = ( + "e.g. python bm_runner.py {0} my_publish_dir\n" + "e.g. python bm_runner.py {0} my_publish_dir --bench=regridding" + ) + + def add_arguments(self) -> None: + self.subparser.add_argument( + "publish_dir", + type=str, + help="HTML results will be published to a sub-dir in this dir.", + ) + + @staticmethod + def csperf(args: argparse.Namespace, run_type: Literal["cperf", "sperf"]) -> None: + _setup_common() + + publish_dir = Path(args.publish_dir) + if not publish_dir.is_dir(): + message = f"Input 'publish directory' is not a directory: {publish_dir}" + raise NotADirectoryError(message) + publish_subdir = ( + publish_dir / f"{run_type}_{datetime.now().strftime('%Y%m%d_%H%M%S')}" + ) + publish_subdir.mkdir() + + # Activate on demand benchmarks (C/SPerf are deactivated for + # 'standard' runs). + environ["ON_DEMAND_BENCHMARKS"] = "True" + commit_range = "upstream/main^!" + + asv_command_str = ( + ASV_HARNESS.format(posargs=commit_range) + f" --bench={run_type}" + ) + + # Only do a single round. + asv_command = shlex.split(re.sub(r"rounds=\d", "rounds=1", asv_command_str)) + try: + _subprocess_runner([*asv_command, *args.asv_args], asv=True) + except subprocess.CalledProcessError as err: + # C/SPerf benchmarks are much bigger than the CI ones: + # Don't fail the whole run if memory blows on 1 benchmark. + # ASV produces return code of 2 if the run includes crashes. + if err.returncode != 2: + raise + + asv_command = shlex.split(f"publish {commit_range} --html-dir={publish_subdir}") + _subprocess_runner(asv_command, asv=True) + + # Print completion message. + location = BENCHMARKS_DIR / ".asv" + echo( + f'New ASV results for "{run_type}".\n' + f'See "{publish_subdir}",' + f'\n or JSON files under "{location / "results"}".' + ) + + +class CPerf(_CSPerf): + name = "cperf" + description = _CSPerf.description.format("CPerf") + epilog = _CSPerf.epilog.format("cperf") + + @staticmethod + def func(args: argparse.Namespace) -> None: + _CSPerf.csperf(args, "cperf") + + +class SPerf(_CSPerf): + name = "sperf" + description = _CSPerf.description.format("SPerf") + epilog = _CSPerf.epilog.format("sperf") + + @staticmethod + def func(args: argparse.Namespace) -> None: + _CSPerf.csperf(args, "sperf") + + +class Custom(_SubParserGenerator): + name = "custom" + description = ( + "Run ASV with the input **ASV sub-command**, without any preset " + "arguments - must all be supplied by the user. So just like running " + "ASV manually, with the convenience of re-using the runner's " + "scripted setup steps." + ) + epilog = "e.g. python bm_runner.py custom continuous a1b23d4 HEAD --quick" + + def add_arguments(self) -> None: + self.subparser.add_argument( + "asv_sub_command", + type=str, + help="The ASV command to run.", + ) + + @staticmethod + def func(args: argparse.Namespace) -> None: + _setup_common() + _subprocess_runner([args.asv_sub_command, *args.asv_args], asv=True) + + +class TrialRun(_SubParserGenerator): + name = "trialrun" + description = ( + "Fast trial-run a given benchmark, to check it works : " + "in a provided or latest-lockfile environment, " + "with no repeats for accuracy of measurement." + ) + epilog = ( + "e.g. python bm_runner.py trialrun " + "MyBenchmarks.time_calc ${DATA_GEN_PYTHON}" + "\n\nNOTE: 'runpath' also replaces $DATA_GEN_PYTHON during the run." + ) + + def add_arguments(self) -> None: + self.subparser.add_argument( + "benchmark", + type=str, + help=( + "A benchmark name, possibly including wildcards, " + "as supported by the ASV '--bench' argument." + ), + ) + self.subparser.add_argument( + "runpath", + type=str, + help=( + "A path to an existing python executable, " + "to completely bypass environment building." + ), + ) + + @staticmethod + def func(args: argparse.Namespace) -> None: + if args.runpath: + # Shortcut creation of a data-gen environment + # - which is also the trial-run env. + python_path = Path(args.runpath).resolve() + environ["DATA_GEN_PYTHON"] = str(python_path) + _setup_common() + # get path of data-gen environment, setup by previous call + python_path = Path(environ["DATA_GEN_PYTHON"]) + # allow 'on-demand' benchmarks + environ["ON_DEMAND_BENCHMARKS"] = "1" + asv_command = [ + "run", + "--bench", + args.benchmark, + # no repeats for timing accuracy + "--quick", + "--show-stderr", + # do not build a unique env : run test in data-gen environment + "--environment", + f"existing:{python_path}", + ] + args.asv_args + _subprocess_runner(asv_command, asv=True) + + +class GhPost(_SubParserGenerator): + name = "_gh_post" + description = ( + "Used by GitHub Actions to post benchmark reports that were prepared " + "during previous actions. Separated to comply with GitHub's security " + "requirements." + ) + epilog = "Sole acceptable syntax: python bm_runner.py _gh_post" + + @staticmethod + def func(args: argparse.Namespace) -> None: + _gh_post_reports() + + # No arguments permitted for this subclass: + + def add_arguments(self) -> None: + pass + + def add_asv_arguments(self) -> None: + pass + + +def main(): + parser = argparse.ArgumentParser( + description="Run the Iris performance benchmarks (using Airspeed Velocity).", + epilog=( + "More help is available within each sub-command." + "\n\nNOTE(1): a separate python environment is created to " + "construct test files.\n Set $DATA_GEN_PYTHON to avoid the cost " + "of this." + "\nNOTE(2): iris-test-data is downloaded and cached within the " + "data generation environment.\n Set " + "$OVERRIDE_TEST_DATA_REPOSITORY to avoid the cost of this." + "\nNOTE(3): test data is cached within the " + "benchmarks code directory, and uses a lot of disk space " + "of disk space (Gb).\n Set $BENCHMARK_DATA to specify where this " + "space can be safely allocated." + ), + formatter_class=argparse.RawTextHelpFormatter, + ) + subparsers = parser.add_subparsers(required=True) + + for gen in (Overnight, Branch, CPerf, SPerf, Custom, TrialRun, GhPost): + _ = gen(subparsers).subparser + + parsed = parser.parse_args() + parsed.func(parsed) + + +if __name__ == "__main__": + main() diff --git a/benchmarks/custom_bms/README.md b/benchmarks/custom_bms/README.md new file mode 100644 index 0000000000..eea85d74fe --- /dev/null +++ b/benchmarks/custom_bms/README.md @@ -0,0 +1,11 @@ +# Iris custom benchmarks + +To be recognised by ASV, these benchmarks must be packaged and installed in +line with the +[ASV guidelines](https://asv.readthedocs.io/projects/asv-runner/en/latest/development/benchmark_plugins.html). +This is achieved using the custom build in [install.py](./install.py). + +Installation is into the environment where the benchmarks are run (i.e. not +the environment containing ASV + Nox, but the one built to the same +specifications as the Tests environment). This is done via `build_command` +in [asv.conf.json](../asv.conf.json). diff --git a/benchmarks/custom_bms/install.py b/benchmarks/custom_bms/install.py new file mode 100644 index 0000000000..59d27a0b43 --- /dev/null +++ b/benchmarks/custom_bms/install.py @@ -0,0 +1,55 @@ +# Copyright Iris contributors +# +# This file is part of Iris and is released under the BSD license. +# See LICENSE in the root of the repository for full licensing details. +"""Install Iris' custom benchmarks for detection by ASV. + +See the requirements for being detected as an ASV plugin: +https://asv.readthedocs.io/projects/asv-runner/en/latest/development/benchmark_plugins.html +""" + +from pathlib import Path +import shutil +from subprocess import run +from tempfile import TemporaryDirectory + +this_dir = Path(__file__).parent + + +def package_files(new_dir: Path) -> None: + """Package Iris' custom benchmarks for detection by ASV. + + Parameters + ---------- + new_dir : Path + The directory to package the custom benchmarks in. + """ + asv_bench_iris = new_dir / "asv_bench_iris" + benchmarks = asv_bench_iris / "benchmarks" + benchmarks.mkdir(parents=True) + (asv_bench_iris / "__init__.py").touch() + + for py_file in this_dir.glob("*.py"): + if py_file != Path(__file__): + shutil.copy2(py_file, benchmarks) + + # Create this on the fly, as having multiple pyproject.toml files in 1 + # project causes problems. + py_project = new_dir / "pyproject.toml" + py_project.write_text( + """ + [project] + name = "asv_bench_iris" + version = "0.1" + """ + ) + + +def main(): + with TemporaryDirectory() as temp_dir: + package_files(Path(temp_dir)) + run(["python", "-m", "pip", "install", temp_dir]) + + +if __name__ == "__main__": + main() diff --git a/benchmarks/custom_bms/tracemallocbench.py b/benchmarks/custom_bms/tracemallocbench.py new file mode 100644 index 0000000000..486c67aeb9 --- /dev/null +++ b/benchmarks/custom_bms/tracemallocbench.py @@ -0,0 +1,196 @@ +# Copyright Iris contributors +# +# This file is part of Iris and is released under the BSD license. +# See LICENSE in the root of the repository for full licensing details. + +"""Benchmark for growth in process resident memory, repeating for accuracy. + +Uses a modified version of the repeat logic in +:class:`asv_runner.benchmarks.time.TimeBenchmark`. +""" + +import re +from timeit import Timer +import tracemalloc +from typing import Callable + +from asv_runner.benchmarks.time import TimeBenchmark, wall_timer + + +class TracemallocBenchmark(TimeBenchmark): + """Benchmark for growth in process resident memory, repeating for accuracy. + + Obviously limited as to what it actually measures : Relies on the current + process not having significant unused (de-allocated) memory when the + tested codeblock runs, and only reliable when the code allocates a + significant amount of new memory. + + Benchmark operations prefixed with ``tracemalloc_`` or ``Tracemalloc`` will + use this benchmark class. + + Inherits behaviour from :class:`asv_runner.benchmarks.time.TimeBenchmark`, + with modifications for memory measurement. See the below Attributes section + and https://asv.readthedocs.io/en/stable/writing_benchmarks.html#timing-benchmarks. + + Attributes + ---------- + Mostly identical to :class:`asv_runner.benchmarks.time.TimeBenchmark`. See + https://asv.readthedocs.io/en/stable/benchmarks.html#timing-benchmarks + Make sure to use the inherited ``repeat`` attribute if greater accuracy + is needed. Below are the attributes where inherited behaviour is + overridden. + + number : int + The number of times the benchmarked operation will be called per + ``repeat``. Memory growth is measured after ALL calls - + i.e. `number` should make no difference to the result if the operation + has perfect garbage collection. The parent class's intelligent + modification of `number` is NOT inherited. A minimum value of ``1`` is + enforced. + warmup_time, sample_time, min_run_count, timer + Not used. + type : str = "tracemalloc" + The name of this benchmark type. + unit : str = "bytes" + The units of the measured metric (i.e. the growth in memory). + + """ + + name_regex = re.compile("^(Tracemalloc[A-Z_].+)|(tracemalloc_.+)$") + + param: tuple + + def __init__(self, name: str, func: Callable, attr_sources: list) -> None: + """Initialize a new instance of `TracemallocBenchmark`. + + Parameters + ---------- + name : str + The name of the benchmark. + func : callable + The function to benchmark. + attr_sources : list + A list of objects from which to draw attributes. + """ + super().__init__(name, func, attr_sources) + self.type = "tracemalloc" + self.unit = "bytes" + + def _load_vars(self): + """Load benchmark variables from attribute sources. + + Downstream handling of ``number`` is not the same as in the parent, so + need to make sure it is at least 1. + """ + super()._load_vars() + self.number = max(1, self.number) + + def run(self, *param: tuple) -> dict: + """Run the benchmark with the given parameters. + + Downstream handling of ``param`` is not the same as in the parent, so + need to store it now. + + Parameters + ---------- + *param : tuple + The parameters to pass to the benchmark function. + + Returns + ------- + dict + A dictionary with the benchmark results. It contains the samples + taken, and "the number of times the function was called in each + sample" - for this benchmark that is always ``1`` to avoid the + parent class incorrectly modifying the results. + """ + self.param = param + return super().run(*param) + + def benchmark_timing( + self, + timer: Timer, + min_repeat: int, + max_repeat: int, + max_time: float, + warmup_time: float, + number: int, + min_run_count: int, + ) -> tuple[list[int], int]: + """Benchmark the timing of the function execution. + + Heavily modified from the parent method + - Directly performs setup and measurement (parent used timeit). + - `number` used differently (see Parameters). + - No warmup phase. + + Parameters + ---------- + timer : timeit.Timer + Not used. + min_repeat : int + The minimum number of times to repeat the function execution. + max_repeat : int + The maximum number of times to repeat the function execution. + max_time : float + The maximum total time to spend on the benchmarking. + warmup_time : float + Not used. + number : int + The number of times the benchmarked operation will be called per + repeat. Memory growth is measured after ALL calls - i.e. `number` + should make no difference to the result if the operation + has perfect garbage collection. The parent class's intelligent + modification of `number` is NOT inherited. + min_run_count : int + Not used. + + Returns + ------- + list + A list of the measured memory growths, in bytes. + int = 1 + Part of the inherited return signature. Must be 1 to avoid + the parent incorrectly modifying the results. + """ + start_time = wall_timer() + samples: list[int] = [] + + def too_slow(num_samples) -> bool: + """Stop taking samples if limits exceeded. + + Parameters + ---------- + num_samples : int + The number of samples taken so far. + + Returns + ------- + bool + True if the benchmark should stop, False otherwise. + """ + if num_samples < min_repeat: + return False + return wall_timer() > start_time + max_time + + # Collect samples + while len(samples) < max_repeat: + self.redo_setup() + tracemalloc.start() + for _ in range(number): + __ = self.func(*self.param) + _, peak_mem_bytes = tracemalloc.get_traced_memory() + tracemalloc.stop() + + samples.append(peak_mem_bytes) + + if too_slow(len(samples)): + break + + # ``number`` is not used in the same way as in the parent class. Must + # be returned as 1 to avoid parent incorrectly modifying the results. + return samples, 1 + + +# https://asv.readthedocs.io/projects/asv-runner/en/latest/development/benchmark_plugins.html +export_as_benchmark = [TracemallocBenchmark] diff --git a/codecov.yml b/codecov.yml new file mode 100644 index 0000000000..a0efbb9997 --- /dev/null +++ b/codecov.yml @@ -0,0 +1,9 @@ +coverage: + # see https://docs.codecov.com/docs/commit-status + status: + project: + default: + target: auto + # coverage can drop by up to- Iris implements a data model based on the CF conventions - giving you a powerful, format-agnostic interface for working with your data. - It excels when working with multi-dimensional Earth Science data, where tabular - representations become unwieldy and inefficient. -
-- CF Standard names, - units, and coordinate metadata - are built into Iris, giving you a rich and expressive interface for maintaining - an accurate representation of your data. Its treatment of data and - associated metadata as first-class objects includes: -
--
-
- a visualisation interface based on matplotlib and - cartopy, -
- unit conversion, -
- subsetting and extraction, -
- merge and concatenate, -
- aggregations and reductions (including min, max, mean and weighted averages), -
- interpolation and regridding (including nearest-neighbor, linear and area-weighted), and -
- operator overloads (
+,-,*,/, etc.).
-
- A number of file formats are recognised by Iris, including CF-compliant NetCDF, GRIB, - and PP, and it has a plugin architecture to allow other formats to be added seamlessly. -
-- Building upon NumPy and - dask, - Iris scales from efficient single-machine workflows right through to multi-core - clusters and HPC. - Interoperability with packages from the wider scientific Python ecosystem comes from Iris' - use of standard NumPy/dask arrays as its underlying data storage. -
- -
-
-
-
-
-
-
-
-
-
-
-{% endblock %}
diff --git a/docs/iris/src/_templates/layout.html b/docs/iris/src/_templates/layout.html
deleted file mode 100644
index f854455f71..0000000000
--- a/docs/iris/src/_templates/layout.html
+++ /dev/null
@@ -1,71 +0,0 @@
-{% extends "!layout.html" %}
-
-{%- block extrahead %}
-{{ super() }}
-
-
-
-
-
-
-
-
-
-
-{% endblock %}
-
-
-{% block rootrellink %}
- -
-
-
-
Installation guide
-
- including dependency details
- -
-
User guide
-
- an introduction to Iris and its core concepts
- -
-
Reference documentation
-
- complete Iris package reference help
- -
-
Gallery
-
- a collection of images produced using Iris
- -
-
Developer's guide
-
- guide for SciTools developers
- -
-
Whitepapers
-
- extra information on specific technical issues
- -
-
What's new in Iris 3.0?
-
- recent changes in Iris's capabilities
-
-
-
-
-
-
- 
-
-
-
-{{ super() }}
-{% endblock %}
-
-
-
-{% block footer %}
-
-
-
-
-
-{% endblock %}
diff --git a/docs/iris/src/conf.py b/docs/iris/src/conf.py
deleted file mode 100644
index 6cdfe634c4..0000000000
--- a/docs/iris/src/conf.py
+++ /dev/null
@@ -1,332 +0,0 @@
-# (C) British Crown Copyright 2010 - 2018, Met Office
-#
-# This file is part of Iris.
-#
-# Iris is free software: you can redistribute it and/or modify it under
-# the terms of the GNU Lesser General Public License as published by the
-# Free Software Foundation, either version 3 of the License, or
-# (at your option) any later version.
-#
-# Iris is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-# GNU Lesser General Public License for more details.
-#
-# You should have received a copy of the GNU Lesser General Public License
-# along with Iris. If not, see
-
-- Iris v3.0 -
-- A powerful, format-agnostic, community-driven Python library for analysing and - visualising Earth science data. -
-Click on any image to see full size image and source code
-- -
-
-
- Gallery
-
-
-{}
-
-
'
- ''
- '{}'
- ' ')
-
- random_image_content_template = '''
-// This file was automatically generated by gen_gallery.py & should not be
-// modified directly.
-
-images = new Array();
-
-{}
-
-'''
-
- random_image_template = "['{thumbfile}', '{full_image}', '{link}'];"
- random_image_join = 'images[{}] = {}'
-
- dirs = ('General', 'Meteorology', 'Oceanography')
-
- for subdir in dirs:
- rows.append(header_template.format(subdir, subdir, subdir))
- toc_rows.append(toc_template.format(subdir, subdir))
-
- origdir = os.path.join(os.path.dirname(outdir), rootdir, subdir)
- if not os.path.exists(origdir):
- origdir = os.path.join(os.path.dirname(outdir), 'plot_directive',
- rootdir, subdir)
- thumbdir = os.path.join(outdir, rootdir, subdir, 'thumbnails')
- if not os.path.exists(thumbdir):
- os.makedirs(thumbdir)
-
- data = []
-
- for filename in sorted(glob.glob(os.path.join(origdir, '*.png'))):
- if filename.endswith('hires.png'):
- continue
-
- path, filename = os.path.split(filename)
- basename, ext = os.path.splitext(filename)
- if basename in skips:
- continue
-
- # Create thumbnails based on images in tmpdir, and place them
- # within the build tree.
- orig_path = str(os.path.join(origdir, filename))
- thumb_path = str(os.path.join(thumbdir, filename))
- if out_of_date(orig_path, thumb_path) or True:
- thumbnails[orig_path] = thumb_path
-
- m = multiimage.match(basename)
- if m is not None:
- basename = m.group(1)
-
- data.append((subdir, basename,
- os.path.join(rootdir, subdir, 'thumbnails',
- filename)))
-
- for (subdir, basename, thumbfile) in data:
- if thumbfile is not None:
- anchor = os.path.basename(thumbfile)
- anchor = os.path.splitext(anchor)[0].replace('_', '-')
- link = 'examples/{}/{}.html#{}'.format(
- subdir,
- basename,
- anchor)
- rows.append(link_template.format(
- href=link,
- thumb_file=thumbfile,
- alternative_text=basename))
- random_image.append(random_image_template.format(
- link=link,
- thumbfile=thumbfile,
- basename=basename,
- full_image='_images/' + os.path.basename(thumbfile)))
-
- if len(data) == 0:
- warnings.warn('No thumbnails were found in {}'.format(subdir))
-
- # Close out the ` method
-to calculate a statistic over a dimension.
-
-Let's use that method to calculate a mean of our air temperature time-series,
-which we'll then use to calculate a time mean anomaly and highlight the powerful
-benefits of cube broadcasting.
-
-First, let's remind ourselves of the shape of our air temperature time-series
-cube::
-
- >>> print(air_temp.summary(True))
- air_temperature / (K) (time: 240; latitude: 37; longitude: 49)
-
-Now, we'll calculate the time-series mean using the
-:meth:`Cube.collapsed ` method::
-
- >>> air_temp_mean = air_temp.collapsed('time', iris.analysis.MEAN)
- >>> print(air_temp_mean.summary(True))
- air_temperature / (K) (latitude: 37; longitude: 49)
-
-As expected the *time* dimension has been collapsed, reducing the
-dimensionality of the resultant *air_temp_mean* cube. This time-series mean can
-now be used to calculate the time mean anomaly against the original
-time-series::
-
- >>> anomaly = air_temp - air_temp_mean
- >>> print(anomaly.summary(True))
- unknown / (K) (time: 240; latitude: 37; longitude: 49)
-
-Notice that the calculation of the *anomaly* involves subtracting a
-*2d* cube from a *3d* cube to yield a *3d* result. This is only possible
-because cube broadcasting is performed during cube arithmetic operations.
-
-Cube broadcasting follows similar broadcasting rules as
-`NumPy `_, but
-the additional richness of Iris coordinate meta-data provides an enhanced
-capability beyond the basic broadcasting behaviour of NumPy.
-
-As the coordinate meta-data of a cube uniquely describes each dimension, it is
-possible to leverage this knowledge to identify the similar dimensions involved
-in a cube arithmetic operation. This essentially means that we are no longer
-restricted to performing arithmetic on cubes with identical shapes.
-
-This extended broadcasting behaviour is highlighted in the following
-examples. The first of these shows that it is possible to involve the
-transpose of the air temperature time-series in an arithmetic operation with
-itself.
-
-Let's first create the transpose of the air temperature time-series::
-
- >>> air_temp_T = air_temp.copy()
- >>> air_temp_T.transpose()
- >>> print(air_temp_T.summary(True))
- air_temperature / (K) (longitude: 49; latitude: 37; time: 240)
-
-Now add the transpose to the original time-series::
-
- >>> result = air_temp + air_temp_T
- >>> print(result.summary(True))
- unknown / (K) (time: 240; latitude: 37; longitude: 49)
-
-Notice that the *result* is the same dimensionality and shape as *air_temp*.
-Let's check that the arithmetic operation has calculated a result that
-we would intuitively expect::
-
- >>> result == 2 * air_temp
- True
-
-Let's extend this example slightly, by taking a slice from the middle
-*latitude* dimension of the transpose cube::
-
- >>> air_temp_T_slice = air_temp_T[:, 0, :]
- >>> print(air_temp_T_slice.summary(True))
- air_temperature / (K) (longitude: 49; time: 240)
-
-Compared to our original time-series, the *air_temp_T_slice* cube has one
-less dimension *and* it's shape if different. However, this doesn't prevent
-us from performing cube arithmetic with it, thanks to the extended cube
-broadcasting behaviour::
-
- >>> result = air_temp - air_temp_T_slice
- >>> print(result.summary(True))
- unknown / (K) (time: 240; latitude: 37; longitude: 49)
-
-Combining multiple phenomena to form a new one
-----------------------------------------------
-
-Combining cubes of potential-temperature and pressure we can calculate
-the associated temperature using the equation:
-
-.. math::
-
- T = \theta (\frac{p}{p_0}) ^ {(287.05 / 1005)}
-
-Where :math:`p` is pressure, :math:`\theta` is potential temperature,
-:math:`p_0` is the potential temperature reference pressure
-and :math:`T` is temperature.
-
-First, let's load pressure and potential temperature cubes::
-
- filename = iris.sample_data_path('colpex.pp')
- phenomenon_names = ['air_potential_temperature', 'air_pressure']
- pot_temperature, pressure = iris.load_cubes(filename, phenomenon_names)
-
-In order to calculate :math:`\frac{p}{p_0}` we can define a coordinate which
-represents the standard reference pressure of 1000 hPa::
-
- import iris.coords
- p0 = iris.coords.AuxCoord(1000.0,
- long_name='reference_pressure',
- units='hPa')
-
-We must ensure that the units of ``pressure`` and ``p0`` are the same,
-so convert the newly created coordinate using
-the :meth:`iris.coords.Coord.convert_units` method::
-
- p0.convert_units(pressure.units)
-
-Now we can combine all of this information to calculate the air temperature
-using the equation above::
-
- temperature = pot_temperature * ( (pressure / p0) ** (287.05 / 1005) )
-
-Finally, the cube we have created needs to be given a suitable name::
-
- temperature.rename('air_temperature')
-
-The result could now be plotted using the guidance provided in the
-:doc:`plotting_a_cube` section.
-
-.. htmlonly::
-
- A very similar example to this can be found in
- :doc:`/examples/Meteorology/deriving_phenomena`.
-
-.. latexonly::
-
- A very similar example to this can be found in the examples section,
- with the title "Deriving Exner Pressure and Air Temperature".
-
diff --git a/docs/iris/src/userguide/cube_statistics.rst b/docs/iris/src/userguide/cube_statistics.rst
deleted file mode 100644
index 3ca7d9a2e0..0000000000
--- a/docs/iris/src/userguide/cube_statistics.rst
+++ /dev/null
@@ -1,334 +0,0 @@
-.. _cube-statistics:
-
-===============
-Cube statistics
-===============
-
-.. _cube-statistics-collapsing:
-
-Collapsing entire data dimensions
----------------------------------
-
-.. testsetup::
-
- import iris
- filename = iris.sample_data_path('uk_hires.pp')
- cube = iris.load_cube(filename, 'air_potential_temperature')
-
- import iris.analysis.cartography
- cube.coord('grid_latitude').guess_bounds()
- cube.coord('grid_longitude').guess_bounds()
- grid_areas = iris.analysis.cartography.area_weights(cube)
-
-
-In the :doc:`subsetting_a_cube` section we saw how to extract a subset of a
-cube in order to reduce either its dimensionality or its resolution.
-Instead of simply extracting a sub-region of the data,
-we can produce statistical functions of the data values
-across a particular dimension,
-such as a 'mean over time' or 'minimum over latitude'.
-
-.. _cube-statistics_forecast_printout:
-
-For instance, suppose we have a cube:
-
- >>> import iris
- >>> filename = iris.sample_data_path('uk_hires.pp')
- >>> cube = iris.load_cube(filename, 'air_potential_temperature')
- >>> print(cube)
- air_potential_temperature / (K) (time: 3; model_level_number: 7; grid_latitude: 204; grid_longitude: 187)
- Dimension coordinates:
- time x - - -
- model_level_number - x - -
- grid_latitude - - x -
- grid_longitude - - - x
- Auxiliary coordinates:
- forecast_period x - - -
- level_height - x - -
- sigma - x - -
- surface_altitude - - x x
- Derived coordinates:
- altitude - x x x
- Scalar coordinates:
- forecast_reference_time: 2009-11-19 04:00:00
- Attributes:
- STASH: m01s00i004
- source: Data from Met Office Unified Model
- um_version: 7.3
-
-
-In this case we have a 4 dimensional cube;
-to mean the vertical (z) dimension down to a single valued extent
-we can pass the coordinate name and the aggregation definition to the
-:meth:`Cube.collapsed() ` method:
-
- >>> import iris.analysis
- >>> vertical_mean = cube.collapsed('model_level_number', iris.analysis.MEAN)
- >>> print(vertical_mean)
- air_potential_temperature / (K) (time: 3; grid_latitude: 204; grid_longitude: 187)
- Dimension coordinates:
- time x - -
- grid_latitude - x -
- grid_longitude - - x
- Auxiliary coordinates:
- forecast_period x - -
- surface_altitude - x x
- Derived coordinates:
- altitude - x x
- Scalar coordinates:
- forecast_reference_time: 2009-11-19 04:00:00
- level_height: 696.6666 m, bound=(0.0, 1393.3333) m
- model_level_number: 10, bound=(1, 19)
- sigma: 0.92292976, bound=(0.8458596, 1.0)
- Attributes:
- STASH: m01s00i004
- source: Data from Met Office Unified Model
- um_version: 7.3
- Cell methods:
- mean: model_level_number
-
-
-Similarly other analysis operators such as ``MAX``, ``MIN`` and ``STD_DEV``
-can be used instead of ``MEAN``, see :mod:`iris.analysis` for a full list
-of currently supported operators.
-
-For an example of using this functionality, the
-:ref:`Hovmoller diagram ` example found
-in the gallery takes a zonal mean of an ``XYT`` cube by using the
-``collapsed`` method with ``latitude`` and ``iris.analysis.MEAN`` as arguments.
-
-.. _cube-statistics-collapsing-average:
-
-Area averaging
-^^^^^^^^^^^^^^
-
-Some operators support additional keywords to the ``cube.collapsed`` method.
-For example, :func:`iris.analysis.MEAN ` supports
-a weights keyword which can be combined with
-:func:`iris.analysis.cartography.area_weights` to calculate an area average.
-
-Let's use the same data as was loaded in the previous example.
-Since ``grid_latitude`` and ``grid_longitude`` were both point coordinates
-we must guess bound positions for them
-in order to calculate the area of the grid boxes::
-
- import iris.analysis.cartography
- cube.coord('grid_latitude').guess_bounds()
- cube.coord('grid_longitude').guess_bounds()
- grid_areas = iris.analysis.cartography.area_weights(cube)
-
-These areas can now be passed to the ``collapsed`` method as weights:
-
-.. doctest::
-
- >>> new_cube = cube.collapsed(['grid_longitude', 'grid_latitude'], iris.analysis.MEAN, weights=grid_areas)
- >>> print(new_cube)
- air_potential_temperature / (K) (time: 3; model_level_number: 7)
- Dimension coordinates:
- time x -
- model_level_number - x
- Auxiliary coordinates:
- forecast_period x -
- level_height - x
- sigma - x
- Derived coordinates:
- altitude - x
- Scalar coordinates:
- forecast_reference_time: 2009-11-19 04:00:00
- grid_latitude: 1.5145501 degrees, bound=(0.14430022, 2.8848) degrees
- grid_longitude: 358.74948 degrees, bound=(357.494, 360.00497) degrees
- surface_altitude: 399.625 m, bound=(-14.0, 813.25) m
- Attributes:
- STASH: m01s00i004
- source: Data from Met Office Unified Model
- um_version: 7.3
- Cell methods:
- mean: grid_longitude, grid_latitude
-
-Several examples of area averaging exist in the gallery which may be of interest,
-including an example on taking a :ref:`global area-weighted mean
-`.
-
-.. _cube-statistics-aggregated-by:
-
-Partially reducing data dimensions
-----------------------------------
-
-Instead of completely collapsing a dimension, other methods can be applied
-to reduce or filter the number of data points of a particular dimension.
-
-
-Aggregation of grouped data
-^^^^^^^^^^^^^^^^^^^^^^^^^^^
-
-The :meth:`Cube.aggregated_by ` operation
-combines data for all points with the same value of a given coordinate.
-To do this, you need a coordinate whose points take on only a limited set
-of different values -- the *number* of these then determines the size of the
-reduced dimension.
-The :mod:`iris.coord_categorisation` module can be used to make such
-'categorical' coordinates out of ordinary ones: The most common use is
-to aggregate data over regular *time intervals*,
-such as by calendar month or day of the week.
-
-For example, let's create two new coordinates on the cube
-to represent the climatological seasons and the season year respectively::
-
- import iris
- import iris.coord_categorisation
-
- filename = iris.sample_data_path('ostia_monthly.nc')
- cube = iris.load_cube(filename, 'surface_temperature')
-
- iris.coord_categorisation.add_season(cube, 'time', name='clim_season')
- iris.coord_categorisation.add_season_year(cube, 'time', name='season_year')
-
-
-
-.. note::
-
- The 'season year' is not the same as year number, because (e.g.) the months
- Dec11, Jan12 + Feb12 all belong to 'DJF-12'.
- See :meth:`iris.coord_categorisation.add_season_year`.
-
-
-.. testsetup:: aggregation
-
- import datetime
- import iris
-
- filename = iris.sample_data_path('ostia_monthly.nc')
- cube = iris.load_cube(filename, 'surface_temperature')
-
- import iris.coord_categorisation
- iris.coord_categorisation.add_season(cube, 'time', name='clim_season')
- iris.coord_categorisation.add_season_year(cube, 'time', name='season_year')
-
- annual_seasonal_mean = cube.aggregated_by(
- ['clim_season', 'season_year'],
- iris.analysis.MEAN)
-
-
-Printing this cube now shows that two extra coordinates exist on the cube:
-
-.. doctest:: aggregation
-
- >>> print(cube)
- surface_temperature / (K) (time: 54; latitude: 18; longitude: 432)
- Dimension coordinates:
- time x - -
- latitude - x -
- longitude - - x
- Auxiliary coordinates:
- clim_season x - -
- forecast_reference_time x - -
- season_year x - -
- Scalar coordinates:
- forecast_period: 0 hours
- Attributes:
- Conventions: CF-1.5
- STASH: m01s00i024
- Cell methods:
- mean: month, year
-
-
-These two coordinates can now be used to aggregate by season and climate-year:
-
-.. doctest:: aggregation
-
- >>> annual_seasonal_mean = cube.aggregated_by(
- ... ['clim_season', 'season_year'],
- ... iris.analysis.MEAN)
- >>> print(repr(annual_seasonal_mean))
-
-
-The primary change in the cube is that the cube's data has been
-reduced in the 'time' dimension by aggregation (taking means, in this case).
-This has collected together all datapoints with the same values of season and
-season-year.
-The results are now indexed by the 19 different possible values of season and
-season-year in a new, reduced 'time' dimension.
-
-We can see this by printing the first 10 values of season+year
-from the original cube: These points are individual months,
-so adjacent ones are often in the same season:
-
-.. doctest:: aggregation
- :options: +NORMALIZE_WHITESPACE
-
- >>> for season, year in zip(cube.coord('clim_season')[:10].points,
- ... cube.coord('season_year')[:10].points):
- ... print(season + ' ' + str(year))
- mam 2006
- mam 2006
- jja 2006
- jja 2006
- jja 2006
- son 2006
- son 2006
- son 2006
- djf 2007
- djf 2007
-
-Compare this with the first 10 values of the new cube's coordinates:
-All the points now have distinct season+year values:
-
-.. doctest:: aggregation
- :options: +NORMALIZE_WHITESPACE
-
- >>> for season, year in zip(
- ... annual_seasonal_mean.coord('clim_season')[:10].points,
- ... annual_seasonal_mean.coord('season_year')[:10].points):
- ... print(season + ' ' + str(year))
- mam 2006
- jja 2006
- son 2006
- djf 2007
- mam 2007
- jja 2007
- son 2007
- djf 2008
- mam 2008
- jja 2008
-
-Because the original data started in April 2006 we have some incomplete seasons
-(e.g. there were only two months worth of data for 'mam-2006').
-In this case we can fix this by removing all of the resultant 'times' which
-do not cover a three month period (note: judged here as > 3*28 days):
-
-.. doctest:: aggregation
-
- >>> tdelta_3mth = datetime.timedelta(hours=3*28*24.0)
- >>> spans_three_months = lambda t: (t.bound[1] - t.bound[0]) > tdelta_3mth
- >>> three_months_bound = iris.Constraint(time=spans_three_months)
- >>> full_season_means = annual_seasonal_mean.extract(three_months_bound)
- >>> full_season_means
-
-
-The final result now represents the seasonal mean temperature for 17 seasons
-from jja-2006 to jja-2010:
-
-.. doctest:: aggregation
- :options: +NORMALIZE_WHITESPACE
-
- >>> for season, year in zip(full_season_means.coord('clim_season').points,
- ... full_season_means.coord('season_year').points):
- ... print(season + ' ' + str(year))
- jja 2006
- son 2006
- djf 2007
- mam 2007
- jja 2007
- son 2007
- djf 2008
- mam 2008
- jja 2008
- son 2008
- djf 2009
- mam 2009
- jja 2009
- son 2009
- djf 2010
- mam 2010
- jja 2010
-
diff --git a/docs/iris/src/userguide/end_of_userguide.rst b/docs/iris/src/userguide/end_of_userguide.rst
deleted file mode 100644
index c8f951a634..0000000000
--- a/docs/iris/src/userguide/end_of_userguide.rst
+++ /dev/null
@@ -1,15 +0,0 @@
-End of the user guide
-=====================
-
-If this was your first time reading the user guide, we hope you found it enjoyable and informative.
-It is advised that you now go back to the :doc:`start ` and try experimenting with your own data.
-
-
-
-Iris gallery
-------------
-It can be very daunting to start coding a project from an empty file, that is why you will find many in-depth
-examples in the Iris gallery which can be used as a goal driven reference to producing your own visualisations.
-
-If you produce a visualisation which you think would add value to the gallery, please get in touch with us and
-we will consider including it as an example for all to benefit from.
diff --git a/docs/iris/src/userguide/index.rst b/docs/iris/src/userguide/index.rst
deleted file mode 100644
index 8c0b24bec3..0000000000
--- a/docs/iris/src/userguide/index.rst
+++ /dev/null
@@ -1,41 +0,0 @@
-.. _user_guide_index:
-
-===============
-Iris user guide
-===============
-
-How to use the user guide
----------------------------
-If you are reading this user guide for the first time it is strongly recommended that you read the user guide
-fully before experimenting with your own data files.
-
-
-Much of the content has supplementary links to the reference documentation; you will not need to follow these
-links in order to understand the guide but they may serve as a useful reference for future exploration.
-
-.. htmlonly::
-
- Since later pages depend on earlier ones, try reading this user guide sequentially using the ``next`` and ``previous`` links.
-
-
-User guide table of contents
--------------------------------
-
-.. toctree::
- :maxdepth: 2
- :numbered:
-
- iris_cubes.rst
- loading_iris_cubes.rst
- saving_iris_cubes.rst
- navigating_a_cube.rst
- subsetting_a_cube.rst
- real_and_lazy_data.rst
- plotting_a_cube.rst
- interpolation_and_regridding.rst
- merge_and_concat.rst
- cube_statistics.rst
- cube_maths.rst
- citation.rst
- code_maintenance.rst
- end_of_userguide.rst
diff --git a/docs/iris/src/userguide/interpolation_and_regridding.rst b/docs/iris/src/userguide/interpolation_and_regridding.rst
deleted file mode 100644
index 565f9b61eb..0000000000
--- a/docs/iris/src/userguide/interpolation_and_regridding.rst
+++ /dev/null
@@ -1,412 +0,0 @@
-.. _interpolation_and_regridding:
-
-
-.. testsetup:: *
-
- import numpy as np
- import iris
- import warnings
- warnings.simplefilter('ignore')
-
-=================================
-Cube interpolation and regridding
-=================================
-
-Iris provides powerful cube-aware interpolation and regridding functionality,
-exposed through Iris cube methods. This functionality is provided by building
-upon existing interpolation schemes implemented by SciPy.
-
-In Iris we refer to the avaliable types of interpolation and regridding as
-`schemes`. The following are the interpolation schemes that are currently
-available in Iris:
-
- * linear interpolation (:class:`iris.analysis.Linear`), and
- * nearest-neighbour interpolation (:class:`iris.analysis.Nearest`).
-
-The following are the regridding schemes that are currently available in Iris:
-
- * linear regridding (:class:`iris.analysis.Linear`),
- * nearest-neighbour regridding (:class:`iris.analysis.Nearest`), and
- * area-weighted regridding (:class:`iris.analysis.AreaWeighted`, first-order conservative).
-
-
-.. _interpolation:
-
-Interpolation
--------------
-
-Interpolating a cube is achieved with the :meth:`~iris.cube.Cube.interpolate`
-method. This method expects two arguments:
-
- #. the sample points to interpolate, and
- #. the second argument being the interpolation scheme to use.
-
-The result is a new cube, interpolated at the sample points.
-
-Sample points must be defined as an iterable of ``(coord, value(s))`` pairs.
-The `coord` argument can be either a coordinate name or coordinate instance.
-The specified coordinate must exist on the cube being interpolated! For example:
-
- * coordinate names and scalar sample points: ``[('latitude', 51.48), ('longitude', 0)]``,
- * a coordinate instance and a scalar sample point: ``[(cube.coord('latitude'), 51.48)]``, and
- * a coordinate name and a NumPy array of sample points: ``[('longitude', np.linspace(-11, 2, 14))]``
-
-are all examples of valid sample points.
-
-The values for coordinates that correspond to date/times can be supplied as
-datetime.datetime or cftime.datetime instances,
-e.g. ``[('time', datetime.datetime(2009, 11, 19, 10, 30))]``).
-
-Let's take the air temperature cube we've seen previously:
-
- >>> air_temp = iris.load_cube(iris.sample_data_path('air_temp.pp'))
- >>> print(air_temp)
- air_temperature / (K) (latitude: 73; longitude: 96)
- Dimension coordinates:
- latitude x -
- longitude - x
- Scalar coordinates:
- forecast_period: 6477 hours, bound=(-28083.0, 6477.0) hours
- forecast_reference_time: 1998-03-01 03:00:00
- pressure: 1000.0 hPa
- time: 1998-12-01 00:00:00, bound=(1994-12-01 00:00:00, 1998-12-01 00:00:00)
- Attributes:
- STASH: m01s16i203
- source: Data from Met Office Unified Model
- Cell methods:
- mean within years: time
- mean over years: time
-
-We can interpolate specific values from the coordinates of the cube:
-
- >>> sample_points = [('latitude', 51.48), ('longitude', 0)]
- >>> print(air_temp.interpolate(sample_points, iris.analysis.Linear()))
- air_temperature / (K) (scalar cube)
- Scalar coordinates:
- forecast_period: 6477 hours, bound=(-28083.0, 6477.0) hours
- forecast_reference_time: 1998-03-01 03:00:00
- latitude: 51.48 degrees
- longitude: 0 degrees
- pressure: 1000.0 hPa
- time: 1998-12-01 00:00:00, bound=(1994-12-01 00:00:00, 1998-12-01 00:00:00)
- Attributes:
- STASH: m01s16i203
- source: Data from Met Office Unified Model
- Cell methods:
- mean within years: time
- mean over years: time
-
-As we can see, the resulting cube is scalar and has longitude and latitude coordinates with
-the values defined in our sample points.
-
-It isn't necessary to specify sample points for every dimension, only those that you
-wish to interpolate over:
-
- >>> result = air_temp.interpolate([('longitude', 0)], iris.analysis.Linear())
- >>> print('Original: ' + air_temp.summary(shorten=True))
- Original: air_temperature / (K) (latitude: 73; longitude: 96)
- >>> print('Interpolated: ' + result.summary(shorten=True))
- Interpolated: air_temperature / (K) (latitude: 73)
-
-The sample points for a coordinate can be an array of values. When multiple coordinates are
-provided with arrays instead of scalar sample points, the coordinates on the resulting cube
-will be orthogonal:
-
- >>> sample_points = [('longitude', np.linspace(-11, 2, 14)),
- ... ('latitude', np.linspace(48, 60, 13))]
- >>> result = air_temp.interpolate(sample_points, iris.analysis.Linear())
- >>> print(result.summary(shorten=True))
- air_temperature / (K) (latitude: 13; longitude: 14)
-
-
-Interpolating non-horizontal coordinates
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-
-Interpolation in Iris is not limited to horizontal-spatial coordinates - any
-coordinate satisfying the prerequisites of the chosen scheme may be interpolated
-over.
-
-For instance, the :class:`iris.analysis.Linear` scheme requires 1D numeric,
-monotonic, coordinates. Supposing we have a single column cube such as
-the one defined below:
-
- >>> cube = iris.load_cube(iris.sample_data_path('hybrid_height.nc'), 'air_potential_temperature')
- >>> column = cube[:, 0, 0]
- >>> print(column.summary(shorten=True))
- air_potential_temperature / (K) (model_level_number: 15)
-
-This cube has a "hybrid-height" vertical coordinate system, meaning that the vertical
-coordinate is unevenly spaced in altitude:
-
- >>> print(column.coord('altitude').points)
- [ 418.69836 434.5705 456.7928 485.3665 520.2933 561.5752
- 609.2145 663.2141 723.57697 790.30664 863.4072 942.8823
- 1028.737 1120.9764 1219.6051 ]
-
-We could regularise the vertical coordinate by defining 10 equally spaced altitude
-sample points between 400 and 1250 and interpolating our vertical coordinate onto
-these sample points:
-
- >>> sample_points = [('altitude', np.linspace(400, 1250, 10))]
- >>> new_column = column.interpolate(sample_points, iris.analysis.Linear())
- >>> print(new_column.summary(shorten=True))
- air_potential_temperature / (K) (model_level_number: 10)
-
-Let's look at the original data, the interpolation line and
-the new data in a plot. This will help us to see what is going on:
-
-.. plot:: userguide/regridding_plots/interpolate_column.py
-
-The red diamonds on the extremes of the altitude values show that we have
-extrapolated data beyond the range of the original data. In some cases this is
-desirable but in other cases it is not. For example, this column defines
-a surface altitude value of 414m, so extrapolating an "air potential temperature"
-at 400m makes little physical sense in this case.
-
-We can control the extrapolation mode when defining the interpolation scheme.
-Controlling the extrapolation mode allows us to avoid situations like the above where
-extrapolating values makes little physical sense.
-
-The extrapolation mode is controlled by the ``extrapolation_mode`` keyword.
-For the available interpolation schemes available in Iris, the ``extrapolation_mode``
-keyword must be one of:
-
- * ``extrapolate`` -- the extrapolation points will be calculated by extending the gradient of the closest two points,
- * ``error`` -- a ValueError exception will be raised, notifying an attempt to extrapolate,
- * ``nan`` -- the extrapolation points will be be set to NaN,
- * ``mask`` -- the extrapolation points will always be masked, even if the source data is not a MaskedArray, or
- * ``nanmask`` -- if the source data is a MaskedArray the extrapolation points will be masked. Otherwise they will be set to NaN.
-
-Using an extrapolation mode is achieved by constructing an interpolation scheme
-with the extrapolation mode keyword set as required. The constructed scheme
-is then passed to the :meth:`~iris.cube.Cube.interpolate` method.
-For example, to mask values that lie beyond the range of the original data:
-
- >>> scheme = iris.analysis.Linear(extrapolation_mode='mask')
- >>> new_column = column.interpolate(sample_points, scheme)
- >>> print(new_column.coord('altitude').points)
- [-- 494.44451904296875 588.888916015625 683.333251953125 777.77783203125
- 872.2222290039062 966.666748046875 1061.111083984375 1155.555419921875 --]
-
-
-.. _caching_an_interpolator:
-
-Caching an interpolator
-^^^^^^^^^^^^^^^^^^^^^^^
-
-If you need to interpolate a cube on multiple sets of sample points you can
-'cache' an interpolator to be used for each of these interpolations. This can
-shorten the execution time of your code as the most computationally
-intensive part of an interpolation is setting up the interpolator.
-
-To cache an interpolator you must set up an interpolator scheme and call the
-scheme's interpolator method. The interpolator method takes as arguments:
-
- #. a cube to be interpolated, and
- #. an iterable of coordinate names or coordinate instances of the coordinates that are to be interpolated over.
-
-For example:
-
- >>> air_temp = iris.load_cube(iris.sample_data_path('air_temp.pp'))
- >>> interpolator = iris.analysis.Nearest().interpolator(air_temp, ['latitude', 'longitude'])
-
-When this cached interpolator is called you must pass it an iterable of sample points
-that have the same form as the iterable of coordinates passed to the constructor.
-So, to use the cached interpolator defined above:
-
- >>> latitudes = np.linspace(48, 60, 13)
- >>> longitudes = np.linspace(-11, 2, 14)
- >>> for lat, lon in zip(latitudes, longitudes):
- ... result = interpolator([lat, lon])
-
-In each case ``result`` will be a cube interpolated from the ``air_temp`` cube we
-passed to interpolator.
-
-Note that you must specify the required extrapolation mode when setting up the cached interpolator.
-For example::
-
- >>> interpolator = iris.analysis.Nearest(extrapolation_mode='nan').interpolator(cube, coords)
-
-
-.. _regridding:
-
-Regridding
-----------
-
-Regridding is conceptually a very similar process to interpolation in Iris.
-The primary difference is that interpolation is based on sample points, while
-regridding is based on the **horizontal** grid of *another cube*.
-
-Regridding a cube is achieved with the :meth:`cube.regrid() ` method.
-This method expects two arguments:
-
- #. *another cube* that defines the target grid onto which the cube should be regridded, and
- #. the regridding scheme to use.
-
-.. note::
-
- Regridding is a common operation needed to allow comparisons of data on different grids.
- The powerful mapping functionality provided by cartopy, however, means that regridding
- is often not necessary if performed just for visualisation purposes.
-
-Let's load two cubes that have different grids and coordinate systems:
-
- >>> global_air_temp = iris.load_cube(iris.sample_data_path('air_temp.pp'))
- >>> rotated_psl = iris.load_cube(iris.sample_data_path('rotated_pole.nc'))
-
-We can visually confirm that they are on different grids by plotting the two cubes:
-
-.. plot:: userguide/regridding_plots/regridding_plot.py
-
-Let's regrid the ``global_air_temp`` cube onto a rotated pole grid
-using a linear regridding scheme. To achieve this we pass the ``rotated_psl``
-cube to the regridder to supply the target grid to regrid the ``global_air_temp``
-cube onto:
-
- >>> rotated_air_temp = global_air_temp.regrid(rotated_psl, iris.analysis.Linear())
-
-.. plot:: userguide/regridding_plots/regridded_to_rotated.py
-
-We could regrid the pressure values onto the global grid, but this will involve
-some form of extrapolation. As with interpolation, we can control the extrapolation
-mode when defining the regridding scheme.
-
-For the available regridding schemes in Iris, the ``extrapolation_mode`` keyword
-must be one of:
-
- * ``extrapolate`` --
-
- * for :class:`~iris.analysis.Linear` the extrapolation points will be calculated by extending the gradient of the closest two points.
- * for :class:`~iris.analysis.Nearest` the extrapolation points will take their value from the nearest source point.
-
- * ``nan`` -- the extrapolation points will be be set to NaN.
- * ``error`` -- a ValueError exception will be raised, notifying an attempt to extrapolate.
- * ``mask`` -- the extrapolation points will always be masked, even if the source data is not a MaskedArray.
- * ``nanmask`` -- if the source data is a MaskedArray the extrapolation points will be masked. Otherwise they will be set to NaN.
-
-The ``rotated_psl`` cube is defined on a limited area rotated pole grid. If we regridded
-the ``rotated_psl`` cube onto the global grid as defined by the ``global_air_temp`` cube
-any linearly extrapolated values would quickly become dominant and highly inaccurate.
-We can control this behaviour by defining the ``extrapolation_mode`` in the constructor
-of the regridding scheme to mask values that lie outside of the domain of the rotated
-pole grid:
-
- >>> scheme = iris.analysis.Linear(extrapolation_mode='mask')
- >>> global_psl = rotated_psl.regrid(global_air_temp, scheme)
-
-.. plot:: userguide/regridding_plots/regridded_to_global.py
-
-Notice that although we can still see the approximate shape of the rotated pole grid, the
-cells have now become rectangular in a plate carrée (equirectangular) projection.
-The spatial grid of the resulting cube is really global, with a large proportion of the
-data being masked.
-
-Area-weighted regridding
-^^^^^^^^^^^^^^^^^^^^^^^^
-
-It is often the case that a point-based regridding scheme (such as
-:class:`iris.analysis.Linear` or :class:`iris.analysis.Nearest`) is not
-appropriate when you need to conserve quantities when regridding. The
-:class:`iris.analysis.AreaWeighted` scheme is less general than
-:class:`~iris.analysis.Linear` or :class:`~iris.analysis.Nearest`, but is a
-conservative regridding scheme, meaning that the area-weighted total is
-approximately preserved across grids.
-
-With the :class:`~iris.analysis.AreaWeighted` regridding scheme, each target grid-box's
-data is computed as a weighted mean of all grid-boxes from the source grid. The weighting
-for any given target grid-box is the area of the intersection with each of the
-source grid-boxes. This scheme performs well when regridding from a high
-resolution source grid to a lower resolution target grid, since all source data
-points will be accounted for in the target grid.
-
-Let's demonstrate this with the global air temperature cube we saw previously,
-along with a limited area cube containing total concentration of volcanic ash:
-
- >>> global_air_temp = iris.load_cube(iris.sample_data_path('air_temp.pp'))
- >>> print(global_air_temp.summary(shorten=True))
- air_temperature / (K) (latitude: 73; longitude: 96)
- >>>
- >>> regional_ash = iris.load_cube(iris.sample_data_path('NAME_output.txt'))
- >>> regional_ash = regional_ash.collapsed('flight_level', iris.analysis.SUM)
- >>> print(regional_ash.summary(shorten=True))
- VOLCANIC_ASH_AIR_CONCENTRATION / (g/m3) (latitude: 214; longitude: 584)
-
-One of the key limitations of the :class:`~iris.analysis.AreaWeighted`
-regridding scheme is that the two input grids must be defined in the same
-coordinate system as each other. Both input grids must also contain monotonic,
-bounded, 1D spatial coordinates.
-
-.. note::
-
- The :class:`~iris.analysis.AreaWeighted` regridding scheme requires spatial
- areas, therefore the longitude and latitude coordinates must be bounded.
- If the longitude and latitude bounds are not defined in the cube we can
- guess the bounds based on the coordinates' point values:
-
- >>> global_air_temp.coord('longitude').guess_bounds()
- >>> global_air_temp.coord('latitude').guess_bounds()
-
-Using NumPy's masked array module we can mask any data that falls below a meaningful
-concentration:
-
- >>> regional_ash.data = np.ma.masked_less(regional_ash.data, 5e-6)
-
-Finally, we can regrid the data using the :class:`~iris.analysis.AreaWeighted`
-regridding scheme:
-
- >>> scheme = iris.analysis.AreaWeighted(mdtol=0.5)
- >>> global_ash = regional_ash.regrid(global_air_temp, scheme)
- >>> print(global_ash.summary(shorten=True))
- VOLCANIC_ASH_AIR_CONCENTRATION / (g/m3) (latitude: 73; longitude: 96)
-
-Note that the :class:`~iris.analysis.AreaWeighted` regridding scheme allows us
-to define a missing data tolerance (``mdtol``), which specifies the tolerated
-fraction of masked data in any given target grid-box. If the fraction of masked
-data within a target grid-box exceeds this value, the data in this target
-grid-box will be masked in the result.
-
-The fraction of masked data is calculated based on the area of masked source
-grid-boxes that overlaps with each target grid-box. Defining an ``mdtol`` in the
-:class:`~iris.analysis.AreaWeighted` regridding scheme allows fine control
-of masked data tolerance. It is worth remembering that defining an ``mdtol`` of
-anything other than 1 will prevent the scheme from being fully conservative, as
-some data will be disregarded if it lies close to masked data.
-
-To visualise the above regrid, let's plot the original data, along with 3 distinct
-``mdtol`` values to compare the result:
-
-.. plot:: userguide/regridding_plots/regridded_to_global_area_weighted.py
-
-
-.. _caching_a_regridder:
-
-Caching a regridder
-^^^^^^^^^^^^^^^^^^^
-
-If you need to regrid multiple cubes with a common source grid onto a common
-target grid you can 'cache' a regridder to be used for each of these regrids.
-This can shorten the execution time of your code as the most computationally
-intensive part of a regrid is setting up the regridder.
-
-To cache a regridder you must set up a regridder scheme and call the
-scheme's regridder method. The regridder method takes as arguments:
-
- #. a cube (that is to be regridded) defining the source grid, and
- #. a cube defining the target grid to regrid the source cube to.
-
-For example:
-
- >>> global_air_temp = iris.load_cube(iris.sample_data_path('air_temp.pp'))
- >>> rotated_psl = iris.load_cube(iris.sample_data_path('rotated_pole.nc'))
- >>> regridder = iris.analysis.Nearest().regridder(global_air_temp, rotated_psl)
-
-When this cached regridder is called you must pass it a cube on the same grid
-as the source grid cube (in this case ``global_air_temp``) that is to be
-regridded to the target grid. For example::
-
- >>> for cube in list_of_cubes_on_source_grid:
- ... result = regridder(cube)
-
-In each case ``result`` will be the input cube regridded to the grid defined by
-the target grid cube (in this case ``rotated_psl``) that we used to define the
-cached regridder.
diff --git a/docs/iris/src/userguide/iris_cubes.rst b/docs/iris/src/userguide/iris_cubes.rst
deleted file mode 100644
index dc423afba1..0000000000
--- a/docs/iris/src/userguide/iris_cubes.rst
+++ /dev/null
@@ -1,196 +0,0 @@
-.. _user_guide_introduction:
-
-===================
-Introduction
-===================
-
-.. _iris_data_structures:
-
-Iris data structures
---------------------
-The top level object in Iris is called a cube. A cube contains data and metadata about a phenomenon.
-
-In Iris, a cube is an interpretation of the *Climate and Forecast (CF) Metadata Conventions* whose purpose is to:
-
- *require conforming datasets to contain sufficient metadata that they are self-describing... including physical
- units if appropriate, and that each value can be located in space (relative to earth-based coordinates) and time.*
-
-Whilst the CF conventions are often mentioned alongside NetCDF, Iris implements several major format importers which can take
-files of specific formats and turn them into Iris cubes. Additionally, a framework is provided which allows users
-to extend Iris' import capability to cater for specialist or unimplemented formats.
-
-A single cube describes one and only one phenomenon, always has a name, a unit and
-an n-dimensional data array to represents the cube's phenomenon. In order to locate the
-data spatially, temporally, or in any other higher-dimensional space, a collection of *coordinates*
-exist on the cube.
-
-
-Coordinates
-===========
-
-A coordinate is a container to store metadata about some dimension(s) of a cube's data array and therefore,
-by definition, its phenomenon.
-
- * Each coordinate has a name and a unit.
- * When a coordinate is added to a cube, the data dimensions that it represents are also provided.
- * The shape of a coordinate is always the same as the shape of the associated data dimension(s) on the cube.
- * A dimension not explicitly listed signifies that the coordinate is independent of that dimension.
- * Each dimension of a coordinate must be mapped to a data dimension. The only coordinates with no mapping are
- scalar coordinates.
-
- * Depending on the underlying data that the coordinate is representing, its values may be discrete points or be
- bounded to represent interval extents (e.g. temperature at *point x* **vs** rainfall accumulation *between 0000-1200 hours*).
- * Coordinates have an attributes dictionary which can hold arbitrary extra metadata, excluding certain restricted CF names
- * More complex coordinates may contain a coordinate system which is necessary to fully interpret the values
- contained within the coordinate.
-
-There are two classes of coordinates:
-
- **DimCoord**
-
- * Numeric
- * Monotonic
- * Representative of, at most, a single data dimension (1d)
-
- **AuxCoord**
-
- * May be of any type, including strings
- * May represent multiple data dimensions (n-dimensional)
-
-
-Cube
-====
-A cube consists of:
-
- * a standard name and/or a long name and an appropriate unit
- * a data array who's values are representative of the phenomenon
- * a collection of coordinates and associated data dimensions on the cube's data array, which are split into two separate lists:
-
- * *dimension coordinates* - DimCoords which uniquely map to exactly one data dimension, ordered by dimension.
- * *auxiliary coordinates* - DimCoords or AuxCoords which map to as many data dimensions as the coordinate has dimensions.
-
- * an attributes dictionary which, other than some protected CF names, can hold arbitrary extra metadata.
- * a list of cell methods to represent operations which have already been applied to the data (e.g. "mean over time")
- * a list of coordinate "factories" used for deriving coordinates from the values of other coordinates in the cube
-
-
-Cubes in practice
------------------
-
-
-A simple cube example
-=====================
-
-Suppose we have some gridded data which has 24 air temperature readings (in Kelvin) which is located at
-4 different longitudes, 2 different latitudes and 3 different heights. Our data array can be represented pictorially:
-
-.. image:: multi_array.png
-
-Where dimensions 0, 1, and 2 have lengths 3, 2 and 4 respectively.
-
-The Iris cube to represent this data would consist of:
-
- * a standard name of ``air_temperature`` and a unit of ``kelvin``
- * a data array of shape ``(3, 2, 4)``
- * a coordinate, mapping to dimension 0, consisting of:
-
- * a standard name of ``height`` and unit of ``meters``
- * an array of length 3 representing the 3 ``height`` points
-
- * a coordinate, mapping to dimension 1, consisting of:
-
- * a standard name of ``latitude`` and unit of ``degrees``
- * an array of length 2 representing the 2 latitude points
- * a coordinate system such that the ``latitude`` points could be fully located on the globe
-
- * a coordinate, mapping to dimension 2, consisting of:
-
- * a standard name of ``longitude`` and unit of ``degrees``
- * an array of length 4 representing the 4 longitude points
- * a coordinate system such that the ``longitude`` points could be fully located on the globe
-
-
-
-
-Pictorially the cube has taken on more information than a simple array:
-
-
-.. image:: multi_array_to_cube.png
-
-
-Additionally further information may be optionally attached to the cube.
-For example, it is possible to attach any of the following:
-
- * a coordinate, not mapping to any data dimensions, consisting of:
-
- * a standard name of ``time`` and unit of ``days since 2000-01-01 00:00``
- * a data array of length 1 representing the time that the data array is valid for
-
- * an auxiliary coordinate, mapping to dimensions 1 and 2, consisting of:
-
- * a long name of ``place name`` and no unit
- * a 2d string array of shape ``(2, 4)`` with the names of the 8 places that the lat/lons correspond to
-
- * an auxiliary coordinate "factory", which can derive its own mapping, consisting of:
-
- * a standard name of ``height`` and a unit of ``feet``
- * knowledge of how data values for this coordinate can be calculated given the ``height in meters`` coordinate
-
- * a cell method of "mean" over "ensemble" to indicate that the data has been meaned over
- a collection of "ensembles" (i.e. multiple model runs).
-
-
-Printing a cube
-===============
-
-Every Iris cube can be printed to screen as you will see later in the user guide. It is worth familiarising yourself with the
-output as this is the quickest way of inspecting the contents of a cube. Here is the result of printing a real life cube:
-
-.. _hybrid_cube_printout:
-
-.. testcode::
- :hide:
-
- import iris
- filename = iris.sample_data_path('uk_hires.pp')
- # NOTE: Every time the output of this cube changes, the full list of deductions below should be re-assessed.
- print(iris.load_cube(filename, 'air_potential_temperature'))
-
-.. testoutput::
-
- air_potential_temperature / (K) (time: 3; model_level_number: 7; grid_latitude: 204; grid_longitude: 187)
- Dimension coordinates:
- time x - - -
- model_level_number - x - -
- grid_latitude - - x -
- grid_longitude - - - x
- Auxiliary coordinates:
- forecast_period x - - -
- level_height - x - -
- sigma - x - -
- surface_altitude - - x x
- Derived coordinates:
- altitude - x x x
- Scalar coordinates:
- forecast_reference_time: 2009-11-19 04:00:00
- Attributes:
- STASH: m01s00i004
- source: Data from Met Office Unified Model
- um_version: 7.3
-
-
-Using this output we can deduce that:
-
- * The cube represents air potential temperature.
- * There are 4 data dimensions, and the data has a shape of ``(3, 7, 204, 187)``
- * The 4 data dimensions are mapped to the ``time``, ``model_level_number``,
- ``grid_latitude``, ``grid_longitude`` coordinates respectively
- * There are three 1d auxiliary coordinates and one 2d auxiliary (``surface_altitude``)
- * There is a single ``altitude`` derived coordinate, which spans 3 data dimensions
- * There are 7 distinct values in the "model_level_number" coordinate. Similar inferences can
- be made for the other dimension coordinates.
- * There are 7, not necessarily distinct, values in the ``level_height`` coordinate.
- * There is a single ``forecast_reference_time`` scalar coordinate representing the entire cube.
- * The cube has one further attribute relating to the phenomenon.
- In this case the originating file format, PP, encodes information in a STASH code which in some cases can
- be useful for identifying advanced experiment information relating to the phenomenon.
diff --git a/docs/iris/src/userguide/loading_iris_cubes.rst b/docs/iris/src/userguide/loading_iris_cubes.rst
deleted file mode 100644
index 2cb3b9b259..0000000000
--- a/docs/iris/src/userguide/loading_iris_cubes.rst
+++ /dev/null
@@ -1,465 +0,0 @@
-.. _loading_iris_cubes:
-
-===================
-Loading Iris cubes
-===================
-
-To load a single file into a **list** of Iris cubes
-the :py:func:`iris.load` function is used::
-
- import iris
- filename = '/path/to/file'
- cubes = iris.load(filename)
-
-Iris will attempt to return **as few cubes as possible**
-by collecting together multiple fields with a shared standard name
-into a single multidimensional cube.
-
-The :py:func:`iris.load` function automatically recognises the format
-of the given files and attempts to produce Iris Cubes from their contents.
-
-.. note::
-
- Currently there is support for CF NetCDF, GRIB 1 & 2, PP and FieldsFiles
- file formats with a framework for this to be extended to custom formats.
-
-
-In order to find out what has been loaded, the result can be printed:
-
- >>> import iris
- >>> filename = iris.sample_data_path('uk_hires.pp')
- >>> cubes = iris.load(filename)
- >>> print(cubes)
- 0: air_potential_temperature / (K) (time: 3; model_level_number: 7; grid_latitude: 204; grid_longitude: 187)
- 1: surface_altitude / (m) (grid_latitude: 204; grid_longitude: 187)
-
-
-This shows that there were 2 cubes as a result of loading the file, they were:
-``air_potential_temperature`` and ``surface_altitude``.
-
-The ``surface_altitude`` cube was 2 dimensional with:
- * the two dimensions have extents of 204 and 187 respectively and are
- represented by the ``grid_latitude`` and ``grid_longitude`` coordinates.
-
-The ``air_potential_temperature`` cubes were 4 dimensional with:
- * the same length ``grid_latitude`` and ``grid_longitude`` dimensions as
- ``surface_altitide``
- * a ``time`` dimension of length 3
- * a ``model_level_number`` dimension of length 7
-
-.. note::
-
- The result of :func:`iris.load` is **always** a
- :class:`list of cubes `.
- Anything that can be done with a Python :class:`list` can be done
- with the resultant list of cubes. It is worth noting, however, that
- there is no inherent order to this
- :class:`list of cubes `.
- Because of this, indexing may be inconsistent. A more consistent way to
- extract a cube is by using the :class:`iris.Constraint` class as
- described in :ref:`constrained-loading`.
-
-.. hint::
-
- Throughout this user guide you will see the function
- ``iris.sample_data_path`` being used to get the filename for the resources
- used in the examples. The result of this function is just a string.
-
- Using this function allows us to provide examples which will work
- across platforms and with data installed in different locations,
- however in practice you will want to use your own strings::
-
- filename = '/path/to/file'
- cubes = iris.load(filename)
-
-To get the air potential temperature cube from the list of cubes
-returned by :py:func:`iris.load` in the previous example,
-list indexing can be used:
-
- >>> import iris
- >>> filename = iris.sample_data_path('uk_hires.pp')
- >>> cubes = iris.load(filename)
- >>> # get the first cube (list indexing is 0 based)
- >>> air_potential_temperature = cubes[0]
- >>> print(air_potential_temperature)
- air_potential_temperature / (K) (time: 3; model_level_number: 7; grid_latitude: 204; grid_longitude: 187)
- Dimension coordinates:
- time x - - -
- model_level_number - x - -
- grid_latitude - - x -
- grid_longitude - - - x
- Auxiliary coordinates:
- forecast_period x - - -
- level_height - x - -
- sigma - x - -
- surface_altitude - - x x
- Derived coordinates:
- altitude - x x x
- Scalar coordinates:
- forecast_reference_time: 2009-11-19 04:00:00
- Attributes:
- STASH: m01s00i004
- source: Data from Met Office Unified Model
- um_version: 7.3
-
-Notice that the result of printing a **cube** is a little more verbose than
-it was when printing a **list of cubes**. In addition to the very short summary
-which is provided when printing a list of cubes, information is provided
-on the coordinates which constitute the cube in question.
-This was the output discussed at the end of the :doc:`iris_cubes` section.
-
-.. note::
-
- Dimensioned coordinates will have a dimension marker ``x`` in the
- appropriate column for each cube data dimension that they describe.
-
-
-Loading multiple files
------------------------
-
-To load more than one file into a list of cubes, a list of filenames can be
-provided to :py:func:`iris.load`::
-
- filenames = [iris.sample_data_path('uk_hires.pp'),
- iris.sample_data_path('air_temp.pp')]
- cubes = iris.load(filenames)
-
-
-It is also possible to load one or more files with wildcard substitution
-using the expansion rules defined :py:mod:`fnmatch`.
-
-For example, to match **zero or more characters** in the filename,
-star wildcards can be used::
-
- filename = iris.sample_data_path('GloSea4', '*.pp')
- cubes = iris.load(filename)
-
-
-.. note::
-
- The cubes returned will not necessarily be in the same order as the
- order of the filenames.
-
-Lazy loading
-------------
-
-In fact when Iris loads data from most file types, it normally only reads the
-essential descriptive information or metadata : the bulk of the actual data
-content will only be loaded later, as it is needed.
-This is referred to as 'lazy' data. It allows loading to be much quicker, and to occupy less memory.
-
-For more on the benefits, handling and uses of lazy data, see :doc:`Real and Lazy Data `.
-
-
-.. _constrained-loading:
-
-Constrained loading
------------------------
-Given a large dataset, it is possible to restrict or constrain the load
-to match specific Iris cube metadata.
-Constrained loading provides the ability to generate a cube
-from a specific subset of data that is of particular interest.
-
-As we have seen, loading the following file creates several Cubes::
-
- filename = iris.sample_data_path('uk_hires.pp')
- cubes = iris.load(filename)
-
-Specifying a name as a constraint argument to :py:func:`iris.load` will mean
-only cubes with a matching :meth:`name `
-will be returned::
-
- filename = iris.sample_data_path('uk_hires.pp')
- cubes = iris.load(filename, 'specific_humidity')
-
-To constrain the load to multiple distinct constraints, a list of constraints
-can be provided. This is equivalent to running load once for each constraint
-but is likely to be more efficient::
-
- filename = iris.sample_data_path('uk_hires.pp')
- cubes = iris.load(filename, ['air_potential_temperature', 'specific_humidity'])
-
-The :class:`iris.Constraint` class can be used to restrict coordinate values
-on load. For example, to constrain the load to match
-a specific ``model_level_number``::
-
- filename = iris.sample_data_path('uk_hires.pp')
- level_10 = iris.Constraint(model_level_number=10)
- cubes = iris.load(filename, level_10)
-
-Constraints can be combined using ``&`` to represent a more restrictive
-constraint to ``load``::
-
- filename = iris.sample_data_path('uk_hires.pp')
- forecast_6 = iris.Constraint(forecast_period=6)
- level_10 = iris.Constraint(model_level_number=10)
- cubes = iris.load(filename, forecast_6 & level_10)
-
-As well as being able to combine constraints using ``&``,
-the :class:`iris.Constraint` class can accept multiple arguments,
-and a list of values can be given to constrain a coordinate to one of
-a collection of values::
-
- filename = iris.sample_data_path('uk_hires.pp')
- level_10_or_16_fp_6 = iris.Constraint(model_level_number=[10, 16], forecast_period=6)
- cubes = iris.load(filename, level_10_or_16_fp_6)
-
-A common requirement is to limit the value of a coordinate to a specific range,
-this can be achieved by passing the constraint a function::
-
- def bottom_16_levels(cell):
- # return True or False as to whether the cell in question should be kept
- return cell <= 16
-
- filename = iris.sample_data_path('uk_hires.pp')
- level_lt_16 = iris.Constraint(model_level_number=bottom_16_levels)
- cubes = iris.load(filename, level_lt_16)
-
-.. note::
-
- As with many of the examples later in this documentation, the
- simple function above can be conveniently written as a lambda function
- on a single line::
-
- bottom_16_levels = lambda cell: cell <= 16
-
-
-Note also the :ref:`warning on equality constraints with floating point coordinates `.
-
-
-Cube attributes can also be part of the constraint criteria. Supposing a
-cube attribute of ``STASH`` existed, as is the case when loading ``PP`` files,
-then specific STASH codes can be filtered::
-
- filename = iris.sample_data_path('uk_hires.pp')
- level_10_with_stash = iris.AttributeConstraint(STASH='m01s00i004') & iris.Constraint(model_level_number=10)
- cubes = iris.load(filename, level_10_with_stash)
-
-.. seealso::
-
- For advanced usage there are further examples in the
- :class:`iris.Constraint` reference documentation.
-
-
-Constraining a circular coordinate across its boundary
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-
-Occasionally you may need to constrain your cube with a region that crosses the
-boundary of a circular coordinate (this is often the meridian or the dateline /
-antimeridian). An example use-case of this is to extract the entire Pacific Ocean
-from a cube whose longitudes are bounded by the dateline.
-
-This functionality cannot be provided reliably using contraints. Instead you should use the
-functionality provided by :meth:`cube.intersection `
-to extract this region.
-
-
-.. _using-time-constraints:
-
-Constraining on Time
-^^^^^^^^^^^^^^^^^^^^
-Iris follows NetCDF-CF rules in representing time coordinate values as normalised,
-purely numeric, values which are normalised by the calendar specified in the coordinate's
-units (e.g. "days since 1970-01-01").
-However, when constraining by time we usually want to test calendar-related
-aspects such as hours of the day or months of the year, so Iris
-provides special features to facilitate this:
-
-Firstly, when Iris evaluates Constraint expressions, it will convert time-coordinate
-values (points and bounds) from numbers into :class:`~datetime.datetime`-like objects
-for ease of calendar-based testing.
-
- >>> filename = iris.sample_data_path('uk_hires.pp')
- >>> cube_all = iris.load_cube(filename, 'air_potential_temperature')
- >>> print('All times :\n' + str(cube_all.coord('time')))
- All times :
- DimCoord([2009-11-19 10:00:00, 2009-11-19 11:00:00, 2009-11-19 12:00:00], standard_name='time', calendar='gregorian')
- >>> # Define a function which accepts a datetime as its argument (this is simplified in later examples).
- >>> hour_11 = iris.Constraint(time=lambda cell: cell.point.hour == 11)
- >>> cube_11 = cube_all.extract(hour_11)
- >>> print('Selected times :\n' + str(cube_11.coord('time')))
- Selected times :
- DimCoord([2009-11-19 11:00:00], standard_name='time', calendar='gregorian')
-
-Secondly, the :class:`iris.time` module provides flexible time comparison
-facilities. An :class:`iris.time.PartialDateTime` object can be compared to
-objects such as :class:`datetime.datetime` instances, and this comparison will
-then test only those 'aspects' which the PartialDateTime instance defines:
-
- >>> import datetime
- >>> from iris.time import PartialDateTime
- >>> dt = datetime.datetime(2011, 3, 7)
- >>> print(dt > PartialDateTime(year=2010, month=6))
- True
- >>> print(dt > PartialDateTime(month=6))
- False
- >>>
-
-These two facilities can be combined to provide straightforward calendar-based
-time selections when loading or extracting data.
-
-The previous constraint example can now be written as:
-
- >>> the_11th_hour = iris.Constraint(time=iris.time.PartialDateTime(hour=11))
- >>> print(iris.load_cube(
- ... iris.sample_data_path('uk_hires.pp'),
- ... 'air_potential_temperature' & the_11th_hour).coord('time'))
- DimCoord([2009-11-19 11:00:00], standard_name='time', calendar='gregorian')
-
-It is common that a cube will need to be constrained between two given dates.
-In the following example we construct a time sequence representing the first
-day of every week for many years:
-
-.. testsetup:: timeseries_range
-
- import datetime
- import numpy as np
- from iris.time import PartialDateTime
- long_ts = iris.cube.Cube(np.arange(150), long_name='data', units='1')
- _mondays = iris.coords.DimCoord(7 * np.arange(150), standard_name='time', units='days since 2007-04-09')
- long_ts.add_dim_coord(_mondays, 0)
-
-
-.. doctest:: timeseries_range
- :options: +NORMALIZE_WHITESPACE, +ELLIPSIS
-
- >>> print(long_ts.coord('time'))
- DimCoord([2007-04-09 00:00:00, 2007-04-16 00:00:00, 2007-04-23 00:00:00,
- ...
- 2010-02-01 00:00:00, 2010-02-08 00:00:00, 2010-02-15 00:00:00],
- standard_name='time', calendar='gregorian')
-
-Given two dates in datetime format, we can select all points between them.
-
-.. doctest:: timeseries_range
- :options: +NORMALIZE_WHITESPACE, +ELLIPSIS
-
- >>> d1 = datetime.datetime.strptime('20070715T0000Z', '%Y%m%dT%H%MZ')
- >>> d2 = datetime.datetime.strptime('20070825T0000Z', '%Y%m%dT%H%MZ')
- >>> st_swithuns_daterange_07 = iris.Constraint(
- ... time=lambda cell: d1 <= cell.point < d2)
- >>> within_st_swithuns_07 = long_ts.extract(st_swithuns_daterange_07)
- >>> print(within_st_swithuns_07.coord('time'))
- DimCoord([2007-07-16 00:00:00, 2007-07-23 00:00:00, 2007-07-30 00:00:00,
- 2007-08-06 00:00:00, 2007-08-13 00:00:00, 2007-08-20 00:00:00],
- standard_name='time', calendar='gregorian')
-
-Alternatively, we may rewrite this using :class:`iris.time.PartialDateTime`
-objects.
-
-.. doctest:: timeseries_range
- :options: +NORMALIZE_WHITESPACE, +ELLIPSIS
-
- >>> pdt1 = PartialDateTime(year=2007, month=7, day=15)
- >>> pdt2 = PartialDateTime(year=2007, month=8, day=25)
- >>> st_swithuns_daterange_07 = iris.Constraint(
- ... time=lambda cell: pdt1 <= cell.point < pdt2)
- >>> within_st_swithuns_07 = long_ts.extract(st_swithuns_daterange_07)
- >>> print(within_st_swithuns_07.coord('time'))
- DimCoord([2007-07-16 00:00:00, 2007-07-23 00:00:00, 2007-07-30 00:00:00,
- 2007-08-06 00:00:00, 2007-08-13 00:00:00, 2007-08-20 00:00:00],
- standard_name='time', calendar='gregorian')
-
-A more complex example might require selecting points over an annually repeating
-date range. We can select points within a certain part of the year, in this case
-between the 15th of July through to the 25th of August. By making use of
-PartialDateTime this becomes simple:
-
-.. doctest:: timeseries_range
-
- >>> st_swithuns_daterange = iris.Constraint(
- ... time=lambda cell: PartialDateTime(month=7, day=15) <= cell < PartialDateTime(month=8, day=25))
- >>> within_st_swithuns = long_ts.extract(st_swithuns_daterange)
- ...
- >>> print(within_st_swithuns.coord('time'))
- DimCoord([2007-07-16 00:00:00, 2007-07-23 00:00:00, 2007-07-30 00:00:00,
- 2007-08-06 00:00:00, 2007-08-13 00:00:00, 2007-08-20 00:00:00,
- 2008-07-21 00:00:00, 2008-07-28 00:00:00, 2008-08-04 00:00:00,
- 2008-08-11 00:00:00, 2008-08-18 00:00:00, 2009-07-20 00:00:00,
- 2009-07-27 00:00:00, 2009-08-03 00:00:00, 2009-08-10 00:00:00,
- 2009-08-17 00:00:00, 2009-08-24 00:00:00], standard_name='time', calendar='gregorian')
-
-Notice how the dates printed are between the range specified in the ``st_swithuns_daterange``
-and that they span multiple years.
-
-
-Strict loading
---------------
-
-The :py:func:`iris.load_cube` and :py:func:`iris.load_cubes` functions are
-similar to :py:func:`iris.load` except they can only return
-*one cube per constraint*.
-The :func:`iris.load_cube` function accepts a single constraint and
-returns a single cube. The :func:`iris.load_cubes` function accepts any
-number of constraints and returns a list of cubes (as an `iris.cube.CubeList`).
-Providing no constraints to :func:`iris.load_cube` or :func:`iris.load_cubes`
-is equivalent to requesting exactly one cube of any type.
-
-A single cube is loaded in the following example::
-
- >>> filename = iris.sample_data_path('air_temp.pp')
- >>> cube = iris.load_cube(filename)
- >>> print(cube)
- air_temperature / (K) (latitude: 73; longitude: 96)
- Dimension coordinates:
- latitude x -
- longitude - x
- ...
- Cell methods:
- mean: time
-
-However, when attempting to load data which would result in anything other than
-one cube, an exception is raised::
-
- >>> filename = iris.sample_data_path('uk_hires.pp')
- >>> cube = iris.load_cube(filename)
- Traceback (most recent call last):
- ...
- iris.exceptions.ConstraintMismatchError: Expected exactly one cube, found 2.
-
-.. note::
-
- All the load functions share many of the same features, hence
- multiple files could be loaded with wildcard filenames
- or by providing a list of filenames.
-
-The strict nature of :func:`iris.load_cube` and :func:`iris.load_cubes`
-means that, when combined with constrained loading, it is possible to
-ensure that precisely what was asked for on load is given
-- otherwise an exception is raised.
-This fact can be utilised to make code only run successfully if
-the data provided has the expected criteria.
-
-For example, suppose that code needed ``air_potential_temperature``
-in order to run::
-
- import iris
- filename = iris.sample_data_path('uk_hires.pp')
- air_pot_temp = iris.load_cube(filename, 'air_potential_temperature')
- print(air_pot_temp)
-
-Should the file not produce exactly one cube with a standard name of
-'air_potential_temperature', an exception will be raised.
-
-Similarly, supposing a routine needed both 'surface_altitude' and
-'air_potential_temperature' to be able to run::
-
- import iris
- filename = iris.sample_data_path('uk_hires.pp')
- altitude_cube, pot_temp_cube = iris.load_cubes(filename, ['surface_altitude', 'air_potential_temperature'])
-
-The result of :func:`iris.load_cubes` in this case will be a list of 2 cubes
-ordered by the constraints provided. Multiple assignment has been used to put
-these two cubes into separate variables.
-
-.. note::
-
- In Python, lists of a pre-known length and order can be exploited
- using *multiple assignment*:
-
- >>> number_one, number_two = [1, 2]
- >>> print(number_one)
- 1
- >>> print(number_two)
- 2
-
diff --git a/docs/iris/src/userguide/merge.png b/docs/iris/src/userguide/merge.png
deleted file mode 100644
index cafaa370da..0000000000
Binary files a/docs/iris/src/userguide/merge.png and /dev/null differ
diff --git a/docs/iris/src/userguide/merge.svg b/docs/iris/src/userguide/merge.svg
deleted file mode 100644
index 9326bc332b..0000000000
--- a/docs/iris/src/userguide/merge.svg
+++ /dev/null
@@ -1,714 +0,0 @@
-
-
-
-
diff --git a/docs/iris/src/userguide/merge_and_concat.png b/docs/iris/src/userguide/merge_and_concat.png
deleted file mode 100644
index 48238287b4..0000000000
Binary files a/docs/iris/src/userguide/merge_and_concat.png and /dev/null differ
diff --git a/docs/iris/src/userguide/multi_array.png b/docs/iris/src/userguide/multi_array.png
deleted file mode 100644
index 54a2688d2a..0000000000
Binary files a/docs/iris/src/userguide/multi_array.png and /dev/null differ
diff --git a/docs/iris/src/userguide/multi_array.svg b/docs/iris/src/userguide/multi_array.svg
deleted file mode 100644
index d28f6d71d6..0000000000
--- a/docs/iris/src/userguide/multi_array.svg
+++ /dev/null
@@ -1,455 +0,0 @@
-
-
-
-
diff --git a/docs/iris/src/userguide/multi_array_to_cube.png b/docs/iris/src/userguide/multi_array_to_cube.png
deleted file mode 100644
index 1144ee6715..0000000000
Binary files a/docs/iris/src/userguide/multi_array_to_cube.png and /dev/null differ
diff --git a/docs/iris/src/userguide/multi_array_to_cube.svg b/docs/iris/src/userguide/multi_array_to_cube.svg
deleted file mode 100644
index a2fc2f5e26..0000000000
--- a/docs/iris/src/userguide/multi_array_to_cube.svg
+++ /dev/null
@@ -1,1378 +0,0 @@
-
-
-
-
diff --git a/docs/iris/src/userguide/plotting_examples/1d_quickplot_simple.py b/docs/iris/src/userguide/plotting_examples/1d_quickplot_simple.py
deleted file mode 100644
index 75462101a0..0000000000
--- a/docs/iris/src/userguide/plotting_examples/1d_quickplot_simple.py
+++ /dev/null
@@ -1,18 +0,0 @@
-
-from __future__ import (absolute_import, division, print_function)
-from six.moves import (filter, input, map, range, zip) # noqa
-
-import matplotlib.pyplot as plt
-
-import iris
-import iris.quickplot as qplt
-
-
-fname = iris.sample_data_path('air_temp.pp')
-temperature = iris.load_cube(fname)
-
-# Take a 1d slice using array style indexing.
-temperature_1d = temperature[5, :]
-
-qplt.plot(temperature_1d)
-plt.show()
diff --git a/docs/iris/src/userguide/plotting_examples/1d_simple.py b/docs/iris/src/userguide/plotting_examples/1d_simple.py
deleted file mode 100644
index 8cb3f45643..0000000000
--- a/docs/iris/src/userguide/plotting_examples/1d_simple.py
+++ /dev/null
@@ -1,18 +0,0 @@
-
-from __future__ import (absolute_import, division, print_function)
-from six.moves import (filter, input, map, range, zip) # noqa
-
-import matplotlib.pyplot as plt
-
-import iris
-import iris.plot as iplt
-
-
-fname = iris.sample_data_path('air_temp.pp')
-temperature = iris.load_cube(fname)
-
-# Take a 1d slice using array style indexing.
-temperature_1d = temperature[5, :]
-
-iplt.plot(temperature_1d)
-plt.show()
diff --git a/docs/iris/src/userguide/plotting_examples/1d_with_legend.py b/docs/iris/src/userguide/plotting_examples/1d_with_legend.py
deleted file mode 100644
index b0aee43c4a..0000000000
--- a/docs/iris/src/userguide/plotting_examples/1d_with_legend.py
+++ /dev/null
@@ -1,47 +0,0 @@
-
-from __future__ import (absolute_import, division, print_function)
-from six.moves import (filter, input, map, range, zip) # noqa
-
-import matplotlib.pyplot as plt
-
-import iris
-import iris.plot as iplt
-
-
-fname = iris.sample_data_path('air_temp.pp')
-
-# Load exactly one cube from the given file
-temperature = iris.load_cube(fname)
-
-# We are only interested in a small number of longitudes (the 4 after and
-# including the 5th element), so index them out
-temperature = temperature[5:9, :]
-
-for cube in temperature.slices('longitude'):
-
- # Create a string label to identify this cube (i.e. latitude: value)
- cube_label = 'latitude: %s' % cube.coord('latitude').points[0]
-
- # Plot the cube, and associate it with a label
- iplt.plot(cube, label=cube_label)
-
-# Match the longitude range to global
-max_lon = temperature.coord('longitude').points.max()
-min_lon = temperature.coord('longitude').points.min()
-plt.xlim(min_lon, max_lon)
-
-# Add the legend with 2 columns
-plt.legend(ncol=2)
-
-# Put a grid on the plot
-plt.grid(True)
-
-# Provide some axis labels
-plt.ylabel('Temerature / kelvin')
-plt.xlabel('Longitude / degrees')
-
-# And a sensible title
-plt.suptitle('Air Temperature', fontsize=20, y=0.9)
-
-# Finally, show it.
-plt.show()
diff --git a/docs/iris/src/userguide/plotting_examples/brewer.py b/docs/iris/src/userguide/plotting_examples/brewer.py
deleted file mode 100644
index 2d61e276ab..0000000000
--- a/docs/iris/src/userguide/plotting_examples/brewer.py
+++ /dev/null
@@ -1,27 +0,0 @@
-
-from __future__ import (absolute_import, division, print_function)
-from six.moves import (filter, input, map, range, zip) # noqa
-
-import matplotlib.pyplot as plt
-import numpy as np
-
-import iris.palette
-
-
-a = np.linspace(0, 1, 256).reshape(1, -1)
-a = np.vstack((a, a))
-
-maps = sorted(iris.palette.CMAP_BREWER)
-nmaps = len(maps)
-
-fig = plt.figure(figsize=(7, 10))
-fig.subplots_adjust(top=0.99, bottom=0.01, left=0.2, right=0.99)
-for i, m in enumerate(maps):
- ax = plt.subplot(nmaps, 1, i+1)
- plt.axis("off")
- plt.imshow(a, aspect='auto', cmap=plt.get_cmap(m), origin='lower')
- pos = list(ax.get_position().bounds)
- fig.text(pos[0] - 0.01, pos[1], m, fontsize=8,
- horizontalalignment='right')
-
-plt.show()
diff --git a/docs/iris/src/userguide/plotting_examples/cube_blockplot.py b/docs/iris/src/userguide/plotting_examples/cube_blockplot.py
deleted file mode 100644
index a6273a274c..0000000000
--- a/docs/iris/src/userguide/plotting_examples/cube_blockplot.py
+++ /dev/null
@@ -1,19 +0,0 @@
-
-from __future__ import (absolute_import, division, print_function)
-from six.moves import (filter, input, map, range, zip) # noqa
-
-import matplotlib.pyplot as plt
-
-import iris
-import iris.quickplot as qplt
-
-
-# Load the data for a single value of model level number.
-fname = iris.sample_data_path('hybrid_height.nc')
-temperature_cube = iris.load_cube(
- fname, iris.Constraint(model_level_number=1))
-
-# Draw the block plot.
-qplt.pcolormesh(temperature_cube)
-
-plt.show()
diff --git a/docs/iris/src/userguide/plotting_examples/cube_brewer_cite_contourf.py b/docs/iris/src/userguide/plotting_examples/cube_brewer_cite_contourf.py
deleted file mode 100644
index 9ba68fc47a..0000000000
--- a/docs/iris/src/userguide/plotting_examples/cube_brewer_cite_contourf.py
+++ /dev/null
@@ -1,29 +0,0 @@
-
-from __future__ import (absolute_import, division, print_function)
-from six.moves import (filter, input, map, range, zip) # noqa
-
-import matplotlib.pyplot as plt
-
-import iris
-import iris.quickplot as qplt
-import iris.plot as iplt
-
-
-fname = iris.sample_data_path('air_temp.pp')
-temperature_cube = iris.load_cube(fname)
-
-# Get the Purples "Brewer" palette.
-brewer_cmap = plt.get_cmap('brewer_Purples_09')
-
-# Draw the contours, with n-levels set for the map colours (9).
-# NOTE: needed as the map is non-interpolated, but matplotlib does not provide
-# any special behaviour for these.
-qplt.contourf(temperature_cube, brewer_cmap.N, cmap=brewer_cmap)
-
-# Add a citation to the plot.
-iplt.citation(iris.plot.BREWER_CITE)
-
-# Add coastlines to the map created by contourf.
-plt.gca().coastlines()
-
-plt.show()
diff --git a/docs/iris/src/userguide/plotting_examples/cube_brewer_contourf.py b/docs/iris/src/userguide/plotting_examples/cube_brewer_contourf.py
deleted file mode 100644
index aacf129c30..0000000000
--- a/docs/iris/src/userguide/plotting_examples/cube_brewer_contourf.py
+++ /dev/null
@@ -1,25 +0,0 @@
-
-from __future__ import (absolute_import, division, print_function)
-from six.moves import (filter, input, map, range, zip) # noqa
-
-import matplotlib.cm as mpl_cm
-import matplotlib.pyplot as plt
-
-import iris
-import iris.quickplot as qplt
-
-fname = iris.sample_data_path('air_temp.pp')
-temperature_cube = iris.load_cube(fname)
-
-# Load a Cynthia Brewer palette.
-brewer_cmap = mpl_cm.get_cmap('brewer_OrRd_09')
-
-# Draw the contours, with n-levels set for the map colours (9).
-# NOTE: needed as the map is non-interpolated, but matplotlib does not provide
-# any special behaviour for these.
-qplt.contourf(temperature_cube, brewer_cmap.N, cmap=brewer_cmap)
-
-# Add coastlines to the map created by contourf.
-plt.gca().coastlines()
-
-plt.show()
diff --git a/docs/iris/src/userguide/plotting_examples/cube_contour.py b/docs/iris/src/userguide/plotting_examples/cube_contour.py
deleted file mode 100644
index 0ec2fa7be4..0000000000
--- a/docs/iris/src/userguide/plotting_examples/cube_contour.py
+++ /dev/null
@@ -1,23 +0,0 @@
-
-from __future__ import (absolute_import, division, print_function)
-from six.moves import (filter, input, map, range, zip) # noqa
-
-import matplotlib.pyplot as plt
-
-import iris
-import iris.quickplot as qplt
-
-
-fname = iris.sample_data_path('air_temp.pp')
-temperature_cube = iris.load_cube(fname)
-
-# Add a contour, and put the result in a variable called contour.
-contour = qplt.contour(temperature_cube)
-
-# Add coastlines to the map created by contour.
-plt.gca().coastlines()
-
-# Add contour labels based on the contour we have just created.
-plt.clabel(contour, inline=False)
-
-plt.show()
diff --git a/docs/iris/src/userguide/plotting_examples/cube_contourf.py b/docs/iris/src/userguide/plotting_examples/cube_contourf.py
deleted file mode 100644
index 21ebb1a214..0000000000
--- a/docs/iris/src/userguide/plotting_examples/cube_contourf.py
+++ /dev/null
@@ -1,20 +0,0 @@
-
-from __future__ import (absolute_import, division, print_function)
-from six.moves import (filter, input, map, range, zip) # noqa
-
-import matplotlib.pyplot as plt
-
-import iris
-import iris.quickplot as qplt
-
-
-fname = iris.sample_data_path('air_temp.pp')
-temperature_cube = iris.load_cube(fname)
-
-# Draw the contour with 25 levels.
-qplt.contourf(temperature_cube, 25)
-
-# Add coastlines to the map created by contourf.
-plt.gca().coastlines()
-
-plt.show()
diff --git a/docs/iris/src/userguide/real_and_lazy_data.rst b/docs/iris/src/userguide/real_and_lazy_data.rst
deleted file mode 100644
index 84a35efa64..0000000000
--- a/docs/iris/src/userguide/real_and_lazy_data.rst
+++ /dev/null
@@ -1,235 +0,0 @@
-
-.. _real_and_lazy_data:
-
-
-.. testsetup:: *
-
- import dask.array as da
- import iris
- import numpy as np
-
-
-==================
-Real and Lazy Data
-==================
-
-We have seen in the :doc:`iris_cubes` section of the user guide that
-Iris cubes contain data and metadata about a phenomenon. The data element of a cube
-is always an array, but the array may be either "real" or "lazy".
-
-In this section of the user guide we will look specifically at the concepts of
-real and lazy data as they apply to the cube and other data structures in Iris.
-
-
-What is real and lazy data?
----------------------------
-
-In Iris, we use the term **real data** to describe data arrays that are loaded
-into memory. Real data is typically provided as a
-`NumPy array `_,
-which has a shape and data type that are used to describe the array's data points.
-Each data point takes up a small amount of memory, which means large NumPy arrays can
-take up a large amount of memory.
-
-Conversely, we use the term **lazy data** to describe data that is not loaded into memory.
-(This is sometimes also referred to as **deferred data**.)
-In Iris, lazy data is provided as a
-`dask array `_.
-A dask array also has a shape and data type
-but the dask array's data points remain on disk and only loaded into memory in
-small chunks when absolutely necessary. This has key performance benefits for
-handling large amounts of data, where both calculation time and storage
-requirements can be significantly reduced.
-
-In Iris, when actual data values are needed from a lazy data array, it is
-*'realised'* : this means that all the actual values are read in from the file,
-and a 'real'
-(i.e. `numpy `_)
-array replaces the lazy array within the Iris object.
-
-Following realisation, the Iris object just contains the actual ('real')
-data, so the time cost of reading all the data is not incurred again.
-From here on, access to the data is fast, but it now occupies its full memory space.
-
-In particular, any direct reference to a `cube.data` will realise the cube data
-content : any lazy content is lost as the data is read from file, and the cube
-content is replaced with a real array.
-This is also referred to simply as "touching" the data.
-
-See the section :ref:`when_real_data`
-for more examples of this.
-
-You can check whether a cube has real data or lazy data by using the method
-:meth:`~iris.cube.Cube.has_lazy_data`. For example::
-
- >>> cube = iris.load_cube(iris.sample_data_path('air_temp.pp'))
- >>> cube.has_lazy_data()
- True
- # Realise the lazy data.
- >>> cube.data
- >>> cube.has_lazy_data()
- False
-
-
-Benefits
---------
-
-The primary advantage of using lazy data is that it enables
-`out-of-core processing `_;
-that is, the loading and manipulating of datasets without loading the full data into memory.
-
-There are two key benefits from this :
-
-**Firstly**, the result of a calculation on a large dataset often occupies much
-less storage space than the source data -- such as for instance a maximum data
-value calculated over a large number of datafiles.
-In these cases the result can be computed in sections, without ever requiring the
-entire source dataset to be loaded, thus drastically reducing memory footprint.
-This strategy of task division can also enable reduced execution time through the effective
-use of parallel processing capabilities.
-
-**Secondly**, it is often simply convenient to form a calculation on a large
-dataset, of which only a certain portion is required at any one time
--- for example, plotting individual timesteps from a large sequence.
-In such cases, a required portion can be extracted and realised without calculating the entire result.
-
-.. _when_real_data:
-
-When does my data become real?
-------------------------------
-
-Certain operations, such as cube indexing and statistics, can be
-performed in a lazy fashion, producing a 'lazy' result from a lazy input, so
-that no realisation immediately occurs.
-However other operations, such as plotting or printing data values, will always
-trigger the 'realisation' of data.
-
-When you load a dataset using Iris the data array will almost always initially be
-a lazy array. This section details some operations that will realise lazy data
-as well as some operations that will maintain lazy data. We use the term **realise**
-to mean converting lazy data into real data.
-
-Most operations on data arrays can be run equivalently on both real and lazy data.
-If the data array is real then the operation will be run on the data array
-immediately. The results of the operation will be available as soon as processing is completed.
-If the data array is lazy then the operation will be deferred and the data array will
-remain lazy until you request the result (such as when you read from ``cube.data``)::
-
- >>> cube = iris.load_cube(iris.sample_data_path('air_temp.pp'))
- >>> cube.has_lazy_data()
- True
- >>> cube += 5
- >>> cube.has_lazy_data()
- True
-
-The process by which the operation is deferred until the result is requested is
-referred to as **lazy evaluation**.
-
-Certain operations, including regridding and plotting, can only be run on real data.
-Calling such operations on lazy data will automatically realise your lazy data.
-
-You can also realise (and so load into memory) your cube's lazy data if you 'touch' the data.
-To 'touch' the data means directly accessing the data by calling ``cube.data``,
-as in the previous example.
-
-Core data
-^^^^^^^^^
-
-Cubes have the concept of "core data". This returns the cube's data in its
-current state:
-
- * If a cube has lazy data, calling the cube's :meth:`~iris.cube.Cube.core_data` method
- will return the cube's lazy dask array. Calling the cube's
- :meth:`~iris.cube.Cube.core_data` method **will never realise** the cube's data.
- * If a cube has real data, calling the cube's :meth:`~iris.cube.Cube.core_data` method
- will return the cube's real NumPy array.
-
-For example::
-
- >>> cube = iris.load_cube(iris.sample_data_path('air_temp.pp'))
- >>> cube.has_lazy_data()
- True
-
- >>> the_data = cube.core_data()
- >>> type(the_data)
-
- >>> cube.has_lazy_data()
- True
-
- # Realise the lazy data.
- >>> cube.data
- >>> the_data = cube.core_data()
- >>> type(the_data)
-
- >>> cube.has_lazy_data()
- False
-
-
-Coordinates
------------
-
-In the same way that Iris cubes contain a data array, Iris coordinates contain a
-points array and an optional bounds array.
-Coordinate points and bounds arrays can also be real or lazy:
-
- * A :class:`~iris.coords.DimCoord` will only ever have **real** points and bounds
- arrays because of monotonicity checks that realise lazy arrays.
- * An :class:`~iris.coords.AuxCoord` can have **real or lazy** points and bounds.
- * An :class:`~iris.aux_factory.AuxCoordFactory` (or derived coordinate)
- can have **real or lazy** points and bounds. If all of the
- :class:`~iris.coords.AuxCoord` instances used to construct the derived coordinate
- have real points and bounds then the derived coordinate will have real points
- and bounds, otherwise the derived coordinate will have lazy points and bounds.
-
-Iris cubes and coordinates have very similar interfaces, which extends to accessing
-coordinates' lazy points and bounds:
-
-.. doctest::
-
- >>> cube = iris.load_cube(iris.sample_data_path('hybrid_height.nc'), 'air_potential_temperature')
-
- >>> dim_coord = cube.coord('model_level_number')
- >>> print(dim_coord.has_lazy_points())
- False
- >>> print(dim_coord.has_bounds())
- False
- >>> print(dim_coord.has_lazy_bounds())
- False
-
- >>> aux_coord = cube.coord('sigma')
- >>> print(aux_coord.has_lazy_points())
- True
- >>> print(aux_coord.has_bounds())
- True
- >>> print(aux_coord.has_lazy_bounds())
- True
-
- # Realise the lazy points. This will **not** realise the lazy bounds.
- >>> points = aux_coord.points
- >>> print(aux_coord.has_lazy_points())
- False
- >>> print(aux_coord.has_lazy_bounds())
- True
-
- >>> derived_coord = cube.coord('altitude')
- >>> print(derived_coord.has_lazy_points())
- True
- >>> print(derived_coord.has_bounds())
- True
- >>> print(derived_coord.has_lazy_bounds())
- True
-
-.. note::
- Printing a lazy :class:`~iris.coords.AuxCoord` will realise its points and bounds arrays!
-
-
-Dask processing options
------------------------
-
-Iris uses dask to provide lazy data arrays for both Iris cubes and coordinates,
-and for computing deferred operations on lazy arrays.
-
-Dask provides processing options to control how deferred operations on lazy arrays
-are computed. This is provided via the ``dask.set_options`` interface. See the
-`dask documentation `_
-for more information on setting dask processing options.
diff --git a/docs/iris/src/userguide/regridding_plots/interpolate_column.py b/docs/iris/src/userguide/regridding_plots/interpolate_column.py
deleted file mode 100644
index 201dcda4a1..0000000000
--- a/docs/iris/src/userguide/regridding_plots/interpolate_column.py
+++ /dev/null
@@ -1,63 +0,0 @@
-
-from __future__ import (absolute_import, division, print_function)
-from six.moves import (filter, input, map, range, zip) # noqa
-
-import iris
-import iris.quickplot as qplt
-import iris.analysis
-import matplotlib.pyplot as plt
-import numpy as np
-
-
-fname = iris.sample_data_path('hybrid_height.nc')
-column = iris.load_cube(fname, 'air_potential_temperature')[:, 0, 0]
-
-alt_coord = column.coord('altitude')
-
-# Interpolate the "perfect" linear interpolation. Really this is just
-# a high number of interpolation points, in this case 1000 of them.
-altitude_points = [('altitude', np.linspace(400, 1250, 1000))]
-scheme = iris.analysis.Linear(extrapolation_mode='mask')
-linear_column = column.interpolate(altitude_points, scheme)
-
-# Now interpolate the data onto 10 evenly spaced altitude levels,
-# as we did in the example.
-altitude_points = [('altitude', np.linspace(400, 1250, 10))]
-scheme = iris.analysis.Linear()
-new_column = column.interpolate(altitude_points, scheme)
-
-plt.figure(figsize=(5, 4), dpi=100)
-
-# Plot the black markers for the original data.
-qplt.plot(column, column.coord('altitude'),
- marker='o', color='black', linestyle='', markersize=3,
- label='Original values', zorder=2)
-
-# Plot the gray line to display the linear interpolation.
-qplt.plot(linear_column, linear_column.coord('altitude'),
- color='gray',
- label='Linear interpolation', zorder=0)
-
-# Plot the red markers for the new data.
-qplt.plot(new_column, new_column.coord('altitude'),
- marker='D', color='red', linestyle='',
- label='Interpolated values', zorder=1)
-
-ax = plt.gca()
-# Space the plot such that the labels appear correctly.
-plt.subplots_adjust(left=0.17, bottom=0.14)
-
-# Limit the plot to a maximum of 5 ticks.
-ax.xaxis.get_major_locator().set_params(nbins=5)
-
-# Prevent matplotlib from using "offset" notation on the xaxis.
-ax.xaxis.get_major_formatter().set_useOffset(False)
-
-# Put some space between the line and the axes.
-ax.margins(0.05)
-
-# Place gridlines and a legend.
-ax.grid()
-plt.legend(loc='lower right')
-
-plt.show()
diff --git a/docs/iris/src/userguide/regridding_plots/regridded_to_global.py b/docs/iris/src/userguide/regridding_plots/regridded_to_global.py
deleted file mode 100644
index c392de7a52..0000000000
--- a/docs/iris/src/userguide/regridding_plots/regridded_to_global.py
+++ /dev/null
@@ -1,26 +0,0 @@
-
-from __future__ import (absolute_import, division, print_function)
-from six.moves import (filter, input, map, range, zip) # noqa
-
-import iris
-import iris.analysis
-import iris.plot as iplt
-import matplotlib.pyplot as plt
-
-
-global_air_temp = iris.load_cube(iris.sample_data_path('air_temp.pp'))
-rotated_psl = iris.load_cube(iris.sample_data_path('rotated_pole.nc'))
-
-scheme = iris.analysis.Linear(extrapolation_mode='mask')
-global_psl = rotated_psl.regrid(global_air_temp, scheme)
-
-plt.figure(figsize=(4, 3))
-iplt.pcolormesh(global_psl)
-plt.title('Air pressure\n'
- 'on a global longitude latitude grid')
-ax = plt.gca()
-ax.coastlines()
-ax.gridlines()
-ax.set_extent([-90, 70, 10, 80])
-
-plt.show()
diff --git a/docs/iris/src/userguide/regridding_plots/regridded_to_global_area_weighted.py b/docs/iris/src/userguide/regridding_plots/regridded_to_global_area_weighted.py
deleted file mode 100644
index 067a4ee2d6..0000000000
--- a/docs/iris/src/userguide/regridding_plots/regridded_to_global_area_weighted.py
+++ /dev/null
@@ -1,51 +0,0 @@
-
-from __future__ import (absolute_import, division, print_function)
-from six.moves import (filter, input, map, range, zip) # noqa
-
-import iris
-import iris.analysis
-import iris.plot as iplt
-import matplotlib.pyplot as plt
-import matplotlib.colors
-import numpy as np
-
-global_air_temp = iris.load_cube(iris.sample_data_path('air_temp.pp'))
-
-regional_ash = iris.load_cube(iris.sample_data_path('NAME_output.txt'))
-regional_ash = regional_ash.collapsed('flight_level', iris.analysis.SUM)
-
-# Mask values so low that they are anomalous.
-regional_ash.data = np.ma.masked_less(regional_ash.data, 5e-6)
-
-norm = matplotlib.colors.LogNorm(5e-6, 0.0175)
-
-global_air_temp.coord('longitude').guess_bounds()
-global_air_temp.coord('latitude').guess_bounds()
-
-fig = plt.figure(figsize=(8, 4.5))
-
-plt.subplot(2, 2, 1)
-iplt.pcolormesh(regional_ash, norm=norm)
-plt.title('Volcanic ash total\nconcentration not regridded',
- size='medium')
-
-for subplot_num, mdtol in zip([2, 3, 4], [0, 0.5, 1]):
- plt.subplot(2, 2, subplot_num)
- scheme = iris.analysis.AreaWeighted(mdtol=mdtol)
- global_ash = regional_ash.regrid(global_air_temp, scheme)
- iplt.pcolormesh(global_ash, norm=norm)
- plt.title('Volcanic ash total concentration\n'
- 'regridded with AreaWeighted(mdtol={})'.format(mdtol),
- size='medium')
-
-plt.subplots_adjust(hspace=0, wspace=0.05,
- left=0.001, right=0.999, bottom=0, top=0.955)
-
-# Iterate over each of the figure's axes, adding coastlines, gridlines
-# and setting the extent.
-for ax in fig.axes:
- ax.coastlines('50m')
- ax.gridlines()
- ax.set_extent([-80, 40, 31, 75])
-
-plt.show()
diff --git a/docs/iris/src/userguide/regridding_plots/regridded_to_rotated.py b/docs/iris/src/userguide/regridding_plots/regridded_to_rotated.py
deleted file mode 100644
index f2c2b271df..0000000000
--- a/docs/iris/src/userguide/regridding_plots/regridded_to_rotated.py
+++ /dev/null
@@ -1,25 +0,0 @@
-
-from __future__ import (absolute_import, division, print_function)
-from six.moves import (filter, input, map, range, zip) # noqa
-
-import iris
-import iris.analysis
-import iris.plot as iplt
-import matplotlib.pyplot as plt
-
-global_air_temp = iris.load_cube(iris.sample_data_path('air_temp.pp'))
-rotated_psl = iris.load_cube(iris.sample_data_path('rotated_pole.nc'))
-
-rotated_air_temp = global_air_temp.regrid(rotated_psl, iris.analysis.Linear())
-
-
-plt.figure(figsize=(4, 3))
-
-iplt.pcolormesh(rotated_air_temp, norm=plt.Normalize(260, 300))
-plt.title('Air temperature\n'
- 'on a limited area rotated pole grid')
-ax = plt.gca()
-ax.coastlines(resolution='50m')
-ax.gridlines()
-
-plt.show()
diff --git a/docs/iris/src/userguide/regridding_plots/regridding_plot.py b/docs/iris/src/userguide/regridding_plots/regridding_plot.py
deleted file mode 100644
index 2419d47727..0000000000
--- a/docs/iris/src/userguide/regridding_plots/regridding_plot.py
+++ /dev/null
@@ -1,33 +0,0 @@
-
-from __future__ import (absolute_import, division, print_function)
-from six.moves import (filter, input, map, range, zip) # noqa
-
-import iris
-import iris.plot as iplt
-import matplotlib.pyplot as plt
-
-# Load the data.
-global_air_temp = iris.load_cube(iris.sample_data_path('air_temp.pp'))
-rotated_psl = iris.load_cube(iris.sample_data_path('rotated_pole.nc'))
-
-plt.figure(figsize=(9, 3.5))
-
-plt.subplot(1, 2, 1)
-iplt.pcolormesh(global_air_temp, norm=plt.Normalize(260, 300))
-plt.title('Air temperature\n'
- 'on a global longitude latitude grid')
-ax = plt.gca()
-ax.coastlines()
-ax.gridlines()
-
-plt.subplot(1, 2, 2)
-iplt.pcolormesh(rotated_psl)
-plt.title('Air pressure\n'
- 'on a limited area rotated pole grid')
-ax = plt.gca()
-ax.coastlines(resolution='50m')
-ax.gridlines()
-
-plt.tight_layout()
-
-plt.show()
diff --git a/docs/iris/src/userguide/subsetting_a_cube.rst b/docs/iris/src/userguide/subsetting_a_cube.rst
deleted file mode 100644
index b61f16a043..0000000000
--- a/docs/iris/src/userguide/subsetting_a_cube.rst
+++ /dev/null
@@ -1,210 +0,0 @@
-.. _subsetting_a_cube:
-
-=================
-Subsetting a Cube
-=================
-
-The :doc:`loading_iris_cubes` section of the user guide showed how to load data into multidimensional Iris cubes.
-However it is often necessary to reduce the dimensionality of a cube down to something more appropriate and/or manageable.
-
-Iris provides several ways of reducing both the amount of data and/or the number of dimensions in your cube depending on the circumstance.
-In all cases **the subset of a valid cube is itself a valid cube**.
-
-
-Cube extraction
-^^^^^^^^^^^^^^^^
-A subset of a cube can be "extracted" from a multi-dimensional cube in order to reduce its dimensionality:
-
- >>> import iris
- >>> filename = iris.sample_data_path('space_weather.nc')
- >>> cube = iris.load_cube(filename, 'electron density')
- >>> equator_slice = cube.extract(iris.Constraint(grid_latitude=0))
- >>> print(equator_slice)
- electron density / (1E11 e/m^3) (height: 29; grid_longitude: 31)
- Dimension coordinates:
- height x -
- grid_longitude - x
- Auxiliary coordinates:
- latitude - x
- longitude - x
- Scalar coordinates:
- grid_latitude: 0.0 degrees
- Attributes:
- Conventions: CF-1.5
-
-
-In this example we start with a 3 dimensional cube, with dimensions of ``height``, ``grid_latitude`` and ``grid_longitude``,
-and extract every point where the latitude is 0, resulting in a 2d cube with axes of ``height`` and ``grid_longitude``.
-
-
-.. _floating-point-warning:
-.. warning::
-
- Caution is required when using equality constraints with floating point coordinates such as ``grid_latitude``.
- Printing the points of a coordinate does not necessarily show the full precision of the underlying number and it
- is very easy return no matches to a constraint when one was expected.
- This can be avoided by using a function as the argument to the constraint::
-
- def near_zero(cell):
- """Returns true if the cell is between -0.1 and 0.1."""
- return -0.1 < cell < 0.1
-
- equator_constraint = iris.Constraint(grid_latitude=near_zero)
-
- Often you will see this construct in shorthand using a lambda function definition::
-
- equator_constraint = iris.Constraint(grid_latitude=lambda cell: -0.1 < cell < 0.1)
-
-
-The extract method could be applied again to the *equator_slice* cube to get a further subset.
-
-For example to get a ``height`` of 9000 metres at the equator the following line extends the previous example::
-
- equator_height_9km_slice = equator_slice.extract(iris.Constraint(height=9000))
- print(equator_height_9km_slice)
-
-The two steps required to get ``height`` of 9000 m at the equator can be simplified into a single constraint::
-
- equator_height_9km_slice = cube.extract(iris.Constraint(grid_latitude=0, height=9000))
- print(equator_height_9km_slice)
-
-As we saw in :doc:`loading_iris_cubes` the result of :func:`iris.load` is a :class:`CubeList `.
-The ``extract`` method also exists on a :class:`CubeList ` and behaves in exactly the
-same way as loading with constraints:
-
- >>> import iris
- >>> air_temp_and_fp_6 = iris.Constraint('air_potential_temperature', forecast_period=6)
- >>> level_10 = iris.Constraint(model_level_number=10)
- >>> filename = iris.sample_data_path('uk_hires.pp')
- >>> cubes = iris.load(filename).extract(air_temp_and_fp_6 & level_10)
- >>> print(cubes)
- 0: air_potential_temperature / (K) (grid_latitude: 204; grid_longitude: 187)
- >>> print(cubes[0])
- air_potential_temperature / (K) (grid_latitude: 204; grid_longitude: 187)
- Dimension coordinates:
- grid_latitude x -
- grid_longitude - x
- Auxiliary coordinates:
- surface_altitude x x
- Derived coordinates:
- altitude x x
- Scalar coordinates:
- forecast_period: 6.0 hours
- forecast_reference_time: 2009-11-19 04:00:00
- level_height: 395.0 m, bound=(360.0, 433.3332) m
- model_level_number: 10
- sigma: 0.9549927, bound=(0.9589389, 0.95068014)
- time: 2009-11-19 10:00:00
- Attributes:
- STASH: m01s00i004
- source: Data from Met Office Unified Model
- um_version: 7.3
-
-
-Cube iteration
-^^^^^^^^^^^^^^^
-A useful way of dealing with a Cube in its **entirety** is by iterating over its layers or slices.
-For example, to deal with a 3 dimensional cube (z,y,x) you could iterate over all 2 dimensional slices in y and x
-which make up the full 3d cube.::
-
- import iris
- filename = iris.sample_data_path('hybrid_height.nc')
- cube = iris.load_cube(filename)
- print(cube)
- for yx_slice in cube.slices(['grid_latitude', 'grid_longitude']):
- print(repr(yx_slice))
-
-As the original cube had the shape (15, 100, 100) there were 15 latitude longitude slices and hence the
-line ``print(repr(yx_slice))`` was run 15 times.
-
-.. note::
-
- The order of latitude and longitude in the list is important; had they been swapped the resultant cube slices
- would have been transposed.
-
- For further information see :py:meth:`Cube.slices `.
-
-
-This method can handle n-dimensional slices by providing more or fewer coordinate names in the list to **slices**::
-
- import iris
- filename = iris.sample_data_path('hybrid_height.nc')
- cube = iris.load_cube(filename)
- print(cube)
- for i, x_slice in enumerate(cube.slices(['grid_longitude'])):
- print(i, repr(x_slice))
-
-The Python function :py:func:`enumerate` is used in this example to provide an incrementing variable **i** which is
-printed with the summary of each cube slice. Note that there were 1500 1d longitude cubes as a result of
-slicing the 3 dimensional cube (15, 100, 100) by longitude (i starts at 0 and 1500 = 15 * 100).
-
-.. hint::
- It is often useful to get a single 2d slice from a multidimensional cube in order to develop a 2d plot function, for example.
- This can be achieved by using the ``next()`` function on the result of
- slices::
-
- first_slice = next(cube.slices(['grid_latitude', 'grid_longitude']))
-
- Once the your code can handle a 2d slice, it is then an easy step to loop over **all** 2d slices within the bigger
- cube using the slices method.
-
-
-Cube indexing
-^^^^^^^^^^^^^
-In the same way that you would expect a numeric multidimensional array to be **indexed** to take a subset of your
-original array, you can **index** a Cube for the same purpose.
-
-
-Here are some examples of array indexing in :py:mod:`numpy`::
-
- import numpy as np
- # create an array of 12 consecutive integers starting from 0
- a = np.arange(12)
- print(a)
-
- print(a[0]) # first element of the array
-
- print(a[-1]) # last element of the array
-
- print(a[0:4]) # first four elements of the array (the same as a[:4])
-
- print(a[-4:]) # last four elements of the array
-
- print(a[::-1]) # gives all of the array, but backwards
-
- # Make a 2d array by reshaping a
- b = a.reshape(3, 4)
- print(b)
-
- print(b[0, 0]) # first element of the first and second dimensions
-
- print(b[0]) # first element of the first dimension (+ every other dimension)
-
- # get the second element of the first dimension and all of the second dimension
- # in reverse, by steps of two.
- print(b[1, ::-2])
-
-
-Similarly, Iris cubes have indexing capability::
-
- import iris
- filename = iris.sample_data_path('hybrid_height.nc')
- cube = iris.load_cube(filename)
-
- print(cube)
-
- # get the first element of the first dimension (+ every other dimension)
- print(cube[0])
-
- # get the last element of the first dimension (+ every other dimension)
- print(cube[-1])
-
- # get the first 4 elements of the first dimension (+ every other dimension)
- print(cube[0:4])
-
- # Get the first element of the first and third dimension (+ every other dimension)
- print(cube[0, :, 0])
-
- # Get the second element of the first dimension and all of the second dimension
- # in reverse, by steps of two.
- print(cube[1, ::-2])
diff --git a/docs/iris/src/whatsnew/1.10.rst b/docs/iris/src/whatsnew/1.10.rst
deleted file mode 100644
index 26f21c0252..0000000000
--- a/docs/iris/src/whatsnew/1.10.rst
+++ /dev/null
@@ -1,176 +0,0 @@
-What's New in Iris 1.10
-***********************
-
-:Release: 1.10
-:Date: 5th September 2016
-
-This document explains the new/changed features of Iris in version 1.10
-(:doc:`View all changes `.)
-
-Iris 1.10 Features
-==================
-.. _iris_grib_added:
-
-* Support has now been added for the
- `iris_grib `_ package, which
- provides GRIB format support in an optional package, separate from Iris.
-
- * If ``iris_grib`` is available, it will always be used in place of the older
- iris module :mod:`iris.fileformats.grib`.
-
- * The capabilities of ``iris_grib`` are essentially the same as the existing
- :mod:`iris.fileformats.grib` when used with ``iris.FUTURE.strict_grib_load=True``,
- with only small detail differences.
-
- * The old :mod:`iris.fileformats.grib` module is now deprecated and may shortly be
- removed.
-
- * If you are already using the recommended :data:`iris.FUTURE` setting
- ``iris.FUTURE.strict_grib_load=True`` this should not cause problems, as
- the new package is all-but identical.
-
- * However, the option ``iris.FUTURE.strict_grib_load`` is itself now
- deprecated, so you should remove code that sets it.
-
- * If, however, your code is still using the older "non-strict" grib
- loading, then you may need to make code changes.
-
- * In particular, the ``field`` object passed to load callbacks is
- different.
- See :class:`iris.fileformats.grib.message.GribMessage` (the
- ``iris_grib.message.GribMessage`` class is the same as this, for now).
-
- * Please exercise your code with the new iris_grib module, and let us know of
- any problems you uncover, such as files that will no longer load with the
- new implementation.
-
-* :meth:`iris.experimental.regrid.PointInCell.regridder` now works across coordinate systems, including non latlon systems. Additionally, the requirement that the source data X and Y coordinates be 2D has been removed. NB: some aspects of this change are backwards incompatible.
-* Plotting non-Gregorian calendars is now supported. This adds `nc_time_axis `_ as a dependency.
-* Promoting a scalar coordinate to a dimension coordinate with :func:`iris.util.new_axis` no longer loads deferred data.
-* The parsing functionality for Cell Methods from netCDF files is available as part of the :mod:`iris.fileformats.netcdf` module as :func:`iris.fileformats.netcdf.parse_cell_methods`.
-* Support for the NameIII Version 2 file format has been added.
-* Loading netcdf data in Mercator and Stereographic projections now accepts optional extra projection parameter attributes (``false_easting``, ``false_northing`` and ``scale_factor_at_projection_origin``), if they match the default values.
-
- * NetCDF files which define a Mercator projection where the ``false_easting``, ``false_northing`` and ``scale_factor_at_projection_origin`` match the defaults will have the projection loaded correctly. Otherwise, a warning will be issued for each parameter that does not match the default and the projection will not be loaded.
- * NetCDF files which define a Steroegraphic projection where the ``scale_factor_at_projection_origin`` is equal to 1.0 will have the projection loaded correctly. Otherwise, a warning will be issued and the projection will not be loaded.
-
-* The :mod:`iris.plot` routines :func:`~iris.plot.contour`, :func:`~iris.plot.contourf`, :func:`~iris.plot.outline`, :func:`~iris.plot.pcolor`, :func:`~iris.plot.pcolormesh` and :func:`~iris.plot.points` now support plotting cubes with anonymous dimensions by specifying the *numeric index* of the anonymous dimension within the ``coords`` keyword argument.
-
- Note that the axis of the anonymous dimension will be plotted in index space.
-
-* NetCDF loading and saving now supports Cubes that use the LambertConformal coordinate system.
-* The experimental structured Fieldsfile loader :func:`~iris.experimental.fieldsfile.load` has been extended to also load structured PP files.
-
- Structured loading is a streamlined operation, offering the benefit of a significantly faster loading alternative to the more generic :func:`iris.load` mechanism.
-
- Note that structured loading is not an optimised wholesale replacement of :func:`iris.load`. Structured loading is restricted to input containing contiguously ordered fields for each phenomenon that repeat regularly over the same vertical levels and times. For further details, see :func:`~iris.experimental.fieldsfile.load`
-
-* :mod:`iris.experimental.regrid_conservative` is now compatible with ESMPy v7.
-* Saving zonal (i.e. longitudinal) means to PP files now sets the '64s' bit in LBPROC.
-* Loading of 'little-endian' PP files is now supported.
-* All appropriate :mod:`iris.plot` functions now handle an ``axes`` keyword, allowing use of the object oriented matplotlib interface rather than pyplot.
-* The ability to pass file format object lists into the rules based load pipeline, as used for GRIB, Fields Files and PP has been added. The :func:`iris.fileformats.pp.load_pairs_from_fields` and :func:`iris.fileformats.grib.load_pairs_from_fields` are provided to produce cubes from such lists. These lists may have been filtered or altered using the appropriate :mod:`iris.fileformats` modules.
-* Cubes can now have an 'hour' coordinate added with :meth:`iris.coord_categorisation.add_hour`.
-* Time coordinates from PP fields with an lbcode of the form 3xx23 are now correctly encoded with a 360-day calendar.
-* The loading from and saving to netCDF of CF cell_measure variables is supported, along with their representation within a Cube as :attr:`~iris.cube.Cube.cell_measures`.
-* Cubes with anonymous dimensions can now be concatenated. This can only occur along a dimension that is not anonymous.
-* NetCDF saving of ``valid_range``, ``valid_min`` and ``valid_max`` cube attributes is now allowed.
-
-Bugs Fixed
-==========
-* Altered Cell Methods to display coordinate's standard_name rather than var_name where appropriate to avoid human confusion.
-* Saving multiple cubes with netCDF4 protected attributes should now work as expected.
-* Concatenating cubes with singleton dimensions (dimensions of size one) now works properly.
-* Fixed the ``grid_mapping_name`` and ``secant_latitudes`` handling for the LambertConformal coordinate system.
-* Fixed bug in :func:`iris.analysis.cartography.project` where the output projection coordinates didn't have units.
-* Attempting to use :meth:`iris.sample_data_path` to access a file that isn't actually Iris sample data now raises a more descriptive error. A note about the appropriate use of `sample_data_path` has also been added to the documentation.
-* Fixed a bug where regridding or interpolation with the :class:`~iris.analysis.Nearest` scheme returned floating-point results even when the source data was integer typed. It now always returns the same type as the source data.
-* Fixed a bug where regridding circular data would ignore any source masking. This affected any regridding using the :class:`~iris.analysis.Linear` and :class:`~iris.analysis.Nearest` schemes, and also :func:`iris.analysis.interpolate.linear`.
-* The ``coord_name`` parameter to :func:`~iris.fileformats.rules.scalar_cell_method` is now checked correctly.
-* LBPROC is set correctly when a cube containing the minimum of a variable is saved to a PP file. The IA component of LBTIM is set correctly when saving maximum or minimum values.
-* The performance of :meth:`iris.cube.Cube.extract` when a list of values is given to an instance of :class:`iris.Constraint` has been improved considerably.
-* Fixed a bug with :meth:`iris.cube.Cube.data` where an :class:`numpy.ndarray` was not being returned for scalar cubes with lazy data.
-* When saving in netcdf format, the units of 'latitude' and 'longitude' coordinates specified in 'degrees' are saved as 'degrees_north' and 'degrees_east' respectively, as defined in the CF conventions for netCDF files: sections 4.1 and 4.2.
-* Fixed a bug with a class of pp files with lbyr == 0, where the date would cause errors when converting to a datetime object (e.g. when printing a cube).
-
- When processing a pp field with lbtim = 2x, lbyr == lbyrd == 0 and lbmon == lbmond, 'month' and 'month_number' coordinates are created instead of 'time'.
-
-* Fixed a bug in :meth:`~iris.analysis.calculus.curl` where the sign of the r-component for spherical coordinates was opposite to what was expected.
-* A bug that prevented cube printing in some cases has been fixed.
-* Fixed a bug where a deepcopy of a :class:`~iris.coords.DimCoord` would have writable ``points`` and ``bounds`` arrays. These arrays can now no longer be modified in-place.
-* Concatenation no longer occurs when the auxiliary coordinates of the cubes do not match. This check is not applied to AuxCoords that span the dimension the concatenation is occuring along. This behaviour can be switched off by setting the ``check_aux_coords`` kwarg in :meth:`iris.cube.CubeList.concatenate` to False.
-* Fixed a bug in :meth:`iris.cube.Cube.subset` where an exception would be thrown while trying to subset over a non-dimensional scalar coordinate.
-
-Incompatible Changes
-====================
-* The source and target for :meth:`iris.experimental.regrid.PointInCell.regridder` must now have defined coordinate systems (i.e. not ``None``). Additionally, the source data X and Y coordinates must have the same cube dimensions.
-
-Deprecations
-============
-* Deprecated the :class:`iris.Future` option
- ``iris.FUTURE.strict_grib_load``.
- This only affected the module :mod:`iris.fileformats.grib`, which is itself
- now deprecated.
- Please see :ref:`iris_grib package `, above.
-* Deprecated the module :mod:`iris.fileformats.grib`. The new package
- `iris_grib `_ replaces this
- fuctionality, which will shortly be removed.
- Please see :ref:`iris_grib package `, above.
-* The use of :data:`iris.config.SAMPLE_DATA_DIR` has been deprecated and replaced by the now importable `iris_sample_data `_ package.
-
-* Deprecated the module :mod:`iris.analysis.interpolate`.
- This contains the following public items, all of which are now deprecated and
- will be removed in a future release:
-
- * :func:`~iris.analysis.interpolate.linear`
- * :func:`~iris.analysis.interpolate.regrid`
- * :func:`~iris.analysis.interpolate.regrid_to_max_resolution`
- * :func:`~iris.analysis.interpolate.nearest_neighbour_indices`
- * :func:`~iris.analysis.interpolate.nearest_neighbour_data_value`
- * :func:`~iris.analysis.interpolate.extract_nearest_neighbour`
- * class :class:`~iris.analysis.interpolate.Linear1dExtrapolator`.
-
- Please use the replacement facilities individually noted in the module
- documentation for :mod:`iris.analysis.interpolate`
-* The method :meth:`iris.cube.Cube.regridded` has been deprecated.
- Please use :meth:`iris.cube.Cube.regrid` instead (see
- :meth:`~iris.cube.Cube.regridded` for details).
-* Deprecated :data:`iris.fileformats.grib.hindcast_workaround` and :class:`iris.fileformats.grib.GribWrapper`. The class :class:`iris.fileformats.grib.message.GribMessage` provides alternative means of working with GRIB message instances.
-* Deprecated the module :mod:`iris.fileformats.ff`. Please use the replacement
- facilities in module :mod:`iris.fileformats.um` :
-
- * :func:`iris.fileformats.um.um_to_pp` replaces :class:`iris.fileformats.ff.FF2PP`.
- * :func:`iris.fileformats.um.load_cubes` replaces :func:`iris.fileformats.ff.load_cubes`.
- * :func:`iris.fileformats.um.load_cubes_32bit_ieee` replaces :func:`iris.fileformats.ff.load_cubes_32bit_ieee`.
-
- All other public components are generally deprecated and will be removed in a future release.
-
-* The :func:`iris.fileformats.pp.as_pairs` and :func:`iris.fileformats.grib.as_pairs` are deprecated. These are replaced with :func:`iris.fileformats.pp.save_pairs_from_cube` and :func:`iris.fileformats.grib.save_pairs_from_cube`.
-* ``iris.fileformats.pp_packing`` has been deprecated. Please install the separate `mo_pack `_ package instead. This provides the same functionality.
-* Deprecated logging functions (currently used only for rules logging):
- :data:`iris.config.iris.config.RULE_LOG_DIR`,
- :data:`iris.config.iris.config.RULE_LOG_IGNORE` and
- :data:`iris.fileformats.rules.log`.
-
-* Deprecated all the remaining text rules mechanisms:
- :class:`iris.fileformats.rules.DebugString`,
- :class:`iris.fileformats.rules.CMAttribute`,
- :class:`iris.fileformats.rules.CMCustomAttribute`,
- :class:`iris.fileformats.rules.CoordAndDims`,
- :class:`iris.fileformats.rules.Rule`,
- :class:`iris.fileformats.rules.FunctionRule`,
- :class:`iris.fileformats.rules.ProcedureRule`,
- :class:`iris.fileformats.rules.RulesContainer` and
- :func:`iris.fileformats.rules.calculate_forecast_period`.
-
-* Deprecated the custom pp save rules mechanism implemented by the functions :func:`iris.fileformats.pp.add_save_rules` and :func:`iris.fileformats.pp.reset_save_rules`. The functions :func:`iris.fileformats.pp.as_fields`, :func:`iris.fileformats.pp.as_pairs` and :func:`iris.fileformats.pp.save_fields` provide alternative means of achieving the same ends.
-
-Documentation Changes
-=====================
-* It is now clear that repeated values will form a group under :meth:`iris.cube.Cube.aggregated_by` even if they aren't consecutive. Hence, the documentation for :mod:`iris.cube` has been changed to reflect this.
-* The documentation for :meth:`iris.analysis.calculus.curl` has been updated for clarity.
-* False claims about :meth:`iris.fileformats.pp.save`, :meth:`iris.fileformats.pp.as_pairs`, and :meth:`iris.fileformats.pp.as_fields` being able to take instances of :class:`iris.cube.CubeList` as inputs have been removed.
-* A :doc:`new code example <../examples/Meteorology/wind_speed>`, demonstrating the use of a quiver plot to display wind speeds over Lake Victoria, has been added.
-* The docstring for :data:`iris.analysis.SUM` has been updated to explicitly state that weights passed to it aren't normalised internally.
-* A note regarding the impossibility of partially collapsing multi-dimensional coordinates has been added to the user guide.
-
diff --git a/docs/iris/src/whatsnew/1.11.rst b/docs/iris/src/whatsnew/1.11.rst
deleted file mode 100644
index eb93ec2f8c..0000000000
--- a/docs/iris/src/whatsnew/1.11.rst
+++ /dev/null
@@ -1,31 +0,0 @@
-What's New in Iris 1.11
-***********************
-
-:Release: 1.11
-:Date: 2016-11-28
-
-This document explains the new/changed features of Iris in version 1.11
-(:doc:`View all changes `.)
-
-Iris 1.11 Features
-==================
-* If available, display the ``STASH`` code instead of ``unknown / (unknown)`` when printing cubes
- with no ``standard_name`` and no ``units``.
-* Support for saving to netCDF with data packing has been added.
-* The coordinate system :class:`iris.coord_systems.LambertAzimuthalEqualArea` has been added with NetCDF saving support.
-
-Bugs Fixed
-==========
-* Fixed a floating point tolerance bug in :func:`iris.experimental.regrid.regrid_area_weighted_rectilinear_src_and_grid`
- for wrapped longitudes.
-* Allow :func:`iris.util.new_axis` to promote the nominated scalar coordinate of a cube
- with a scalar masked constant data payload.
-* Fixed a bug where :func:`iris.util._is_circular` would erroneously return false
- when coordinate values are decreasing.
-* When saving to NetCDF, the existing behaviour of writing string attributes as ASCII has been
- maintained across known versions of netCDF4-python.
-
-Documentation Changes
-=====================
-* Fuller doc-string detail added to :func:`iris.analysis.cartography.unrotate_pole` and
- :func:`iris.analysis.cartography.rotate_pole`.
diff --git a/docs/iris/src/whatsnew/1.13.rst b/docs/iris/src/whatsnew/1.13.rst
deleted file mode 100644
index 532c160f13..0000000000
--- a/docs/iris/src/whatsnew/1.13.rst
+++ /dev/null
@@ -1,37 +0,0 @@
-What's New in Iris 1.13
-***********************
-
-:Release: 1.13
-:Date: 2017-05-17
-
-
-This document explains the new/changed features of Iris in version 1.13
-(:doc:`View all changes `.)
-
-Iris 1.13 Features
-==================
-
-* Allow the reading of NAME trajectories stored by time instead of by particle number.
-* An experimental link to python-stratify via :mod:`iris.experimental.stratify`.
-* Data arrays may be shared between cubes, and subsets of cubes, by using the :meth:`iris.cube.share_data` flag.
-
-
-Bug Fixes
-=========
-
-* The bounds are now set correctly on the longitude coordinate if a zonal mean diagnostic has been loaded from a PP file as per the CF Standard.
-* NetCDF loading will now determine whether there is a string-valued scalar label, i.e. a character variable that only has one dimension (the length of the string), and interpret this correctly.
-* A line plot of geographic coordinates (e.g. drawing a trajectory) wraps around the edge of the map cleanly, rather than plotting a segment straight across the map.
-* When saving to PP, lazy data is preserved when generating PP fields from cubes so that a list of cubes can be saved to PP without excessive memory requirements.
-* An error is now correctly raised if a user tries to perform an arithmetic operation on two cubes with mismatching coordinates. Previously these cases were caught by the add and subtract operators, and now it is also caught by the multiply and divide operators.
-* Limited area Rotated Pole datasets where the data range is ``0 <= lambda < 360``, for example as produced in New Zealand, are plotted over a sensible map extent by default.
-* Removed the potential for a RuntimeWarning: overflow encountered in ``int_scalars`` which was missed during collapsed calculations. This could trip up unwary users of limited data types, such as int32 for very large numbers (e.g. seconds since 1970).
-* The CF conventions state that certain ``formula_terms`` terms may be omitted and assumed to be zero (http://cfconventions.org/cf-conventions/v1.6.0/cf-conventions.html#dimensionless-v-coord) so Iris now allows factories to be constructed with missing terms.
-* In the User Guide's contour plot example, clabel inline is set to be False so that it renders correctly, avoiding spurious horizontal lines across plots, although this does make labels a little harder to see.
-* The computation of area weights has been changed to a more numerically stable form. The previous form converted latitude to colatitude and used difference of cosines in the cell area computation. This formulation uses latitude and difference of sines. The conversion from latitude to colatitude at lower precision causes errors when computing the cell areas.
-
-Testing
-=======
-
-* Iris has adopted conda-forge to provide environments for continuous integration testing.
-
diff --git a/docs/iris/src/whatsnew/1.4.rst b/docs/iris/src/whatsnew/1.4.rst
deleted file mode 100644
index 053a6e1096..0000000000
--- a/docs/iris/src/whatsnew/1.4.rst
+++ /dev/null
@@ -1,205 +0,0 @@
-What's new in Iris 1.4
-**********************
-
-:Release: 1.4.0
-:Date: 14 June 2013
-
-This document explains the new/changed features of Iris in version 1.4.
-(:doc:`View all changes `.)
-
-Iris 1.4 features
-=================
-
-A summary of the main features added with version 1.4:
-
-* Multiple cubes can now be exported to a NetCDF file.
-* Correct nearest-neighbour calculation with circular coords.
-* :ref:`Experimental regridding enhancements`.
-* :ref:`Iris-Pandas interoperability`.
-* NIMROD level type 12 (levels below ground) can now be loaded.
-* :ref:`Load cubes from the internet via OPeNDAP`.
-* :ref:`GeoTiff export (experimental)`.
-* :ref:`Cube merge update`.
-* :ref:`Unambiguous season year naming`.
-* NIMROD files with multiple fields and period of interest can now be loaded.
-* Missing values are now handled when loading GRIB messages.
-* PP export rule to calculate forecast period.
-* :func:`~iris.cube.Cube.aggregated_by` now maintains array masking.
-* IEEE 32bit fieldsfiles can now be loaded.
-* NetCDF transverse mercator and climatology data can now be loaded.
-* Polar stereographic GRIB data can now be loaded.
-* :ref:`Cubes with no vertical coord can now be exported to GRIB`.
-* :ref:`Simplified resource configuration`.
-* :ref:`Extended GRIB parameter translation`.
-* Added an optimisation for single-valued coordinate constraints.
-* :ref:`One dimensional linear interpolation fix`.
-* :ref:`Fix for iris.analysis.calculus.differentiate`.
-* Fixed pickling of cubes with 2D aux coords from NetCDF.
-* Fixed bug which ignored the "coords" keyword for certain plots.
-* Use the latest release of Cartopy, v0.8.0.
-
-
-Incompatible changes
---------------------
-* As part of simplifying the mechanism for accessing test data,
- :func:`iris.io.select_data_path`, :data:`iris.config.DATA_REPOSITORY`,
- :data:`iris.config.MASTER_DATA_REPOSITORY` and
- :data:`iris.config.RESOURCE_DIR` have been removed.
-
-Deprecations
-------------
-* The *add_custom_season_** functions from :mod:`~iris.coord_categorisation` have been deprecated in favour of adding their functionality to the *add_season_** functions
-
-
-.. _OPeNDAP: http://www.opendap.org/about
-
-
-.. _exp-regrid:
-
-Experimental regridding enhancements
-====================================
-
-Bilinear, area-weighted and area-conservative regridding functions are now available in
-:mod:`iris.experimental`. These functions support masked data and handle
-derived coordinates such as hybrid height. The final API is still in development.
-In the meantime:
-
-Bilinear rectilinear regridding
--------------------------------
-:func:`~iris.experimental.regrid.regrid_bilinear_rectilinear_src_and_grid`
-can be used to regrid a cube onto a horizontal grid defined in a different coordinate system.
-The data values are calculated using bilinear interpolation.
-
-For example::
-
- from iris.experimental.regrid import regrid_bilinear_rectilinear_src_and_grid
- regridded_cube = regrid_bilinear_rectilinear_src_and_grid(source_cube, target_grid_cube)
-
-Area-weighted regridding
-------------------------
-:func:`~iris.experimental.regrid.regrid_area_weighted_rectilinear_src_and_grid` can be used to regrid a cube
-such that the data values of the resulting cube are calculated using the
-area-weighted mean.
-
-For example::
-
- from iris.experimental.regrid import regrid_area_weighted_rectilinear_src_and_grid as regrid_area_weighted
- regridded_cube = regrid_area_weighted(source_cube, target_grid_cube)
-
-Area-conservative regridding
-----------------------------
-:func:`~iris.experimental.regrid_conservative.regrid_conservative_via_esmpy`
-can be used for area-conservative regridding between geographical coordinate systems.
-This uses the ESMF library functions, via the ESMPy interface.
-
-For example::
-
- from iris.experimental.regrid_conservative import regrid_conservative_via_esmpy
- regridded_cube = regrid_conservative_via_esmpy(source_cube, target_grid_cube)
-
-
-.. _iris-pandas:
-
-Iris-Pandas interoperablilty
-============================
-Conversion to and from Pandas Series_ and DataFrames_ is now available.
-See :mod:`iris.pandas` for more details.
-
-.. _Series: http://pandas.pydata.org/pandas-docs/stable/api.html#series
-.. _DataFrames: http://pandas.pydata.org/pandas-docs/stable/api.html#dataframe
-
-
-.. _load-opendap:
-
-Load cubes from the internet via OPeNDAP
-========================================
-Cubes can now be loaded directly from the internet, via OPeNDAP_.
-
-For example::
-
- cubes = iris.load("http://geoport.whoi.edu/thredds/dodsC/bathy/gom15")
-
-
-.. _geotiff_export:
-
-GeoTiff export
-==============
-With this experimental feature, two dimensional cubes can now be exported to GeoTiff files.
-
-For example::
-
- from iris.experimental.raster import export_geotiff
- export_geotiff(cube, filename)
-
-.. note::
-
- This is a raw data export only and does not save Iris plots.
-
-
-.. _cube-merge-update:
-
-Cube merge update
-=================
-Cube merging now favours numerical coordinates over string coordinates
-to describe a dimension, and :class:`~iris.coords.DimCoord` over
-:class:`~iris.coords.AuxCoord`. These modifications prevent the error:
-*"No functional relationship between separable and inseparable candidate dimensions"*.
-
-
-.. _season-year-name:
-
-Unambiguous season year naming
-==============================
-The default names of categorisation coordinates are now less ambiguous.
-For example, :func:`~iris.coord_categorisation.add_month_number` and
-:func:`~iris.coord_categorisation.add_month_fullname` now create
-"month_number" and "month_fullname" coordinates.
-
-
-.. _grib-novert:
-
-Cubes with no vertical coord can now be exported to GRIB
-========================================================
-Iris can now export cubes with no vertical coord to GRIB.
-The solution is still under discussion: See https://github.com/SciTools/iris/issues/519.
-
-
-.. _simple_cfg:
-
-Simplified resource configuration
-=================================
-A new configuration variable called :data:`iris.config.TEST_DATA_DIR`
-has been added, replacing the previous combination of
-:data:`iris.config.MASTER_DATA_REPOSITORY` and
-:data:`iris.config.DATA_REPOSITORY`. This constant should be the path
-to a directory containing the test data required by the unit tests. It can
-be set by adding a ``test_data_dir`` entry to the ``Resources`` section of
-``site.cfg``. See :mod:`iris.config` for more details.
-
-
-.. _grib_params:
-
-Extended GRIB parameter translation
-===================================
- - More GRIB2 params are recognised on input.
- - Now translates some codes on GRIB2 output.
- - Some GRIB2 params may load with a different standard_name.
-
-
-
-.. _one-d-linear:
-
-One dimensional linear interpolation fix
-========================================
-:func:`~iris.analysis.interpolate.linear` can now extrapolate from a single point
-assuming a gradient of zero. This prevents an issue when loading cross sections
-with a hybrid height coordinate, on a staggered grid and only a single orography field.
-
-
-.. _calc-diff-fix:
-
-Fix for iris.analysis.calculus.differentiate
-=============================================
-A bug in :func:`~iris.analysis.calculus.differentiate` that had the potential to cause
-the loss of coordinate metadata when calculating the curl or the derivative of a cube has been fixed.
-
diff --git a/docs/iris/src/whatsnew/1.7.rst b/docs/iris/src/whatsnew/1.7.rst
deleted file mode 100644
index 2f3a52fbb9..0000000000
--- a/docs/iris/src/whatsnew/1.7.rst
+++ /dev/null
@@ -1,248 +0,0 @@
-What's new in Iris 1.7
-**********************
-
-This document explains the new/changed features of Iris in version 1.7.
-(:doc:`View all changes `.)
-
-:Release: 1.7.4
-:Date: 15th April 2015
-
-Iris 1.7 features
-=================
-
-.. _showcase:
-
-.. admonition:: Showcase: Iris is making use of Biggus
-
- Iris is now making extensive use of `Biggus `_
- for virtual arrays and lazy array evaluation. In practice this means that analyses
- of cubes with data bigger than the available system memory are now possible.
-
- Other than the improved functionality the changes are mostly
- transparent; for example, before the introduction of biggus, MemoryErrors
- were likely for very large datasets::
-
- >>> result = extremely_large_cube.collapsed('time', iris.analyis.MEAN)
- MemoryError
-
- Now, for supported operations, the evaluation is lazy (i.e. it doesn't take
- place until the actual data is subsequently requested) and can handle data
- larger than available system memory::
-
- >>> result = extremely_large_cube.collapsed('time', iris.analyis.MEAN)
- >>> print(type(result))
-
-
- Memory is still a limiting factor if ever the data is desired as a NumPy array
- (e.g. via :data:`cube.data `), but additional methods have
- been added to the Cube to support querying and subsequently accessing the "lazy"
- data form (see :meth:`~iris.cube.Cube.has_lazy_data` and
- :meth:`~iris.cube.Cube.lazy_data`).
-
-.. admonition:: Showcase: New interpolation and regridding API
-
- New interpolation and regridding interfaces have been added which simplify and
- extend the existing functionality.
-
- The interfaces are exposed on the cube in the form of the
- :meth:`~iris.cube.Cube.interpolate` and :meth:`~iris.cube.Cube.regrid` methods.
- Conceptually the signatures of the methods are::
-
- interpolated_cube = cube.interpolate(interpolation_points, interpolation_scheme)
-
- and::
-
- regridded_cube = cube.regrid(target_grid_cube, regridding_scheme)
-
- Whilst not all schemes have been migrated to the new interface,
- :class:`iris.analysis.Linear` defines both linear interpolation and regridding,
- and :class:`iris.analysis.AreaWeighted` defines an area weighted regridding
- scheme.
-
-.. admonition:: Showcase: Merge and concatenate reporting
-
- Merge reporting is designed as an aid to the merge processes. Should merging
- a :class:`~iris.cube.CubeList` fail, merge reporting means that a descriptive
- error will be raised that details the differences between the cubes in the
- :class:`~iris.cube.CubeList` that prevented the merge from being successful.
-
- A new :class:`~iris.cube.CubeList` method, called
- :meth:`~iris.cube.CubeList.merge_cube`, has been introduced. Calling it on a
- :class:`~iris.cube.CubeList` will result in a single merged
- :class:`~iris.cube.Cube` being returned or an error message being raised
- that describes why the merge process failed.
-
- The following example demonstrates the error message that describes a merge
- failure caused by cubes having differing attributes::
-
- >>> cube_list = iris.cube.CubeList((c1, c2))
- >>> cube_list.merge_cube()
- Traceback (most recent call last):
- ...
- raise iris.exceptions.MergeError(msgs)
- iris.exceptions.MergeError: failed to merge into a single cube.
- cube.attributes keys differ: 'foo'
-
- The naming of this new method mirrors that of Iris load functions, where
- one would always expect a :class:`~iris.cube.CubeList` from :func:`iris.load`
- and a :class:`~iris.cube.Cube` from :func:`iris.load_cube`.
-
- Concatenate reporting is the equivalent process for concatenating a
- :class:`~iris.cube.CubeList`. It is accessed through the method
- :meth:`~iris.cube.CubeList.concatenate_cube`, which will return a single
- concatenated cube or produce an error message that describes why the
- concatenate process failed.
-
-.. admonition:: Showcase: Cube broadcasting
-
- When performing cube arithmetic, cubes now follow similar broadcasting rules
- as NumPy arrays.
-
- However, the additional richness of Iris coordinate meta-data provides an
- enhanced capability beyond the basic broadcasting behaviour of NumPy.
-
- This means that when performing cube arithmetic, the dimensionality and shape of
- cubes no longer need to match. For example, if the dimensionality of a cube is
- reduced by collapsing, then the result can be used to subtract from the original
- cube to calculate an anomaly::
-
- >>> time_mean = original_cube.collapsed('time', iris.analysis.MEAN)
- >>> mean_anomaly = original_cube - time_mean
-
- Given both broadcasting **and** coordinate meta-data, Iris can now perform
- arithmetic with cubes that have similar but not identical shape::
-
- >>> similar_cube = original_cube.copy()
- >>> similar_cube.transpose()
- >>> zero_cube = original_cube - similar_cube
-
-* Merge reporting that raises a descriptive error if the merge process fails.
-* Linear interpolation and regridding now make use of SciPy's RegularGridInterpolator
- for much faster linear interpolation.
-* NAME file loading now handles the "no time averaging" column and translates
- height/altitude above ground/sea-level columns into appropriate coordinate metadata.
-* The NetCDF saver has been extended to allow saving of cubes with hybrid pressure
- auxiliary factories.
-* PP/FF loading supports LBLEV of 9999.
-* Extended GRIB1 loading to support data on hybrid pressure levels.
-* :func:`iris.coord_categorisation.add_day_of_year` can be used to add categorised
- day of year coordinates based on time coordinates with non-Gregorian calendars.
-* Support for loading data on reduced grids from GRIB files in raw form without
- automatically interpolating to a regular grid.
-* The coordinate systems :class:`iris.coord_systems.Orthographic` and
- :class:`iris.coord_systems.VerticalPerspective` (for imagery from geostationary
- satellites) have been added.
-* Extended NetCDF loading to support the "ocean sigma over z" auxiliary coordinate
- factory.
-* Support added for loading CF-NetCDF data with bounds arrays that are missing a
- vertex dimension.
-* :meth:`iris.cube.Cube.rolling_window` can now be used with string-based
- :class:`iris.coords.AuxCoord` instances.
-* Loading of PP and FF files has been optimised through deferring creation of
- PPField attributes.
-* Automatic association of a coordinate's CF formula terms variable with the
- data variable associated with that coordinate.
-* PP loading translates cross-section height into a dimensional auxiliary coordinate.
-* String auxiliary coordinates can now be plotted with the Iris plotting wrappers.
-* :func:`iris.analysis.geometry.geometry_area_weights` now allows for the calculation of
- normalized cell weights.
-* Many new translations between the CF spec and STASH codes or GRIB2 parameter codes.
-* PP save rules add the data's UM Version to the attributes of the saved file
- when appropriate.
-* NetCDF reference surface variable promotion available through the
- :class:`iris.FUTURE` mechanism.
-* A speed improvement in calculation of :func:`iris.analysis.geometry.geometry_area_weights`.
-* The mdtol keyword was added to area-weighted regridding to allow control of the
- tolerance for missing data. For a further description of this concept, see
- :class:`iris.analysis.AreaWeighted`.
-* Handling for patching of the CF conventions global attribute via a defined
- cf_patch_conventions function.
-* Deferred GRIB data loading has been introduced for reduced memory consumption when
- loading GRIB files.
-* Concatenate reporting that raises a descriptive error if the concatenation
- process fails.
-* A speed improvement when loading PP or FF data and constraining on STASH code.
-
-Bugs fixed
-==========
-* Data containing more than one reference cube for constructing hybrid height
- coordinates can now be loaded.
-* Removed cause of increased margin of error when interpolating.
-* Changed floating-point precision used when wrapping points for interpolation.
-* Mappables that can be used to generate colorbars are now returned by Iris
- plotting wrappers.
-* NetCDF load ignores over-specified formula terms on bounded dimensionless vertical
- coordinates.
-* Auxiliary coordinate factory loading now correctly interprets formula term
- varibles for "atmosphere hybrid sigma pressure" coordinate data.
-* Corrected comparison of NumPy NaN values in cube merge process.
-* Fixes for :meth:`iris.cube.Cube.intersection` to correct calculating the intersection
- of a cube with split bounds, handling of circular coordinates, handling of
- monotonically descending bounded coordinats and for finding a wrapped two-point
- result and longitude tolerances.
-* A bug affecting :meth:`iris.cube.Cube.extract` and :meth:`iris.cube.CubeList.extract`
- that led to unexpected behaviour when operating on scalar cubes has been fixed.
-* Aggregate_by may now be passed single-value coordinates.
-* Making a copy of a :class:`iris.coords.DimCoord` no longer results in the writeable
- flag on the copied points and bounds arrays being set to True.
-* Can now save to PP a cube that has vertical levels but no orography.
-* Fix a bug causing surface altitude and surface pressure fields to not appear
- in cubes loaded with a STASH constraint.
-* Fixed support for :class:`iris.fileformats.pp.STASH` objects in STASH constraints.
-* A fix to avoid a problem where cube attribute names clash with NetCDF reserved attribute names.
-* A fix to allow :meth:`iris.cube.CubeList.concatenate` to deal with descending coordinate order.
-* Add missing NetCDF attribute `varname` when constructing a new :class:`iris.coords.AuxCoord`.
-* The datatype of time arrays converted with :func:`iris.util.unify_time_units` is now preserved.
-
-Bugs fixed in v1.7.3
-^^^^^^^^^^^^^^^^^^^^
-* Scalar dimension coordinates can now be concatenated with :meth:`iris.cube.CubeList.concatenate`.
-* Arbitrary names can no longer be set for elements of a :class:`iris.fileformats.pp.SplittableInt`.
-* Cubes that contain a pseudo-level coordinate can now be saved to PP.
-* Fixed a bug in the FieldsFile loader that prevented it always loading all available fields.
-
-Bugs fixed in v1.7.4
-^^^^^^^^^^^^^^^^^^^^
-* :meth:`Coord.guess_bounds` can now deal with circular coordinates.
-* :meth:`Coord.nearest_neighbour_index` can now work with descending bounds.
-* Passing `weights` to :meth:`Cube.rolling_window` no longer prevents other
- keyword arguments from being passed to the aggregator.
-* Several minor fixes to allow use of Iris on Windows.
-* Made use of the new standard_parallels keyword in Cartopy's LambertConformal
- projection (Cartopy v0.12). Older versions of Iris will not be able to
- create LambertConformal coordinate systems with Cartopy >= 0.12.
-
-Incompatible changes
-====================
-* Saving a cube with a STASH attribute to NetCDF now produces a variable
- with an attribute of "um_stash_source" rather than "ukmo__um_stash_source".
-* Cubes saved to NetCDF with a coordinate system referencing a spherical ellipsoid
- now result in the grid mapping variable containing only the "earth_radius" attribute,
- rather than the "semi_major_axis" and "semi_minor_axis".
-* Collapsing a cube over all of its dimensions now results in a scalar cube rather
- than a 1d cube.
-
-Deprecations
-============
-* :func:`iris.util.ensure_array` has been deprecated.
-* Deprecated the :func:`iris.fileformats.pp.reset_load_rules` and
- :func:`iris.fileformats.grib.reset_load_rules` functions.
-* Matplotlib is no longer a core Iris dependency.
-
-Documentation Changes
-=====================
-* New sections on :ref:`cube broadcasting ` and
- :doc:`regridding and interpolation `
- have been added to the :doc:`user guide `.
-* An example demonstrating custom log-scale colouring has been added.
- See :ref:`General-anomaly_log_colouring`.
-* An example demonstrating the creation of a custom
- :class:`iris.analysis.Aggregator` has been added.
- See :ref:`General-custom_aggregation`.
-* An example of reprojecting data from 2D auxiliary spatial coordinates
- (such as that from the ORCA grid) has been added. See :ref:`General-orca_projection`.
-* A clarification of the behaviour of :func:`iris.analysis.calculus.differentiate`.
-* A new :doc:`"whitepapers" ` section has been added to the documentation along
- with the addition of a paper providing an :doc:`overview of the load process for UM-like
- fileformats (e.g. PP and Fieldsfile) `.
-
diff --git a/docs/iris/src/whatsnew/1.8.rst b/docs/iris/src/whatsnew/1.8.rst
deleted file mode 100644
index c763411ed8..0000000000
--- a/docs/iris/src/whatsnew/1.8.rst
+++ /dev/null
@@ -1,181 +0,0 @@
-What's new in Iris 1.8
-**********************
-
-:Release: 1.8.1
-:Date: 3rd June 2015
-
-This document explains the new/changed features of Iris in version 1.8.
-(:doc:`View all changes `.)
-
-Iris 1.8 features
-=================
-
-.. _showcase:
-
-.. admonition:: Showcase: Rotate winds
-
- Iris can now rotate and unrotate wind vector data by transforming the wind
- vector data to another coordinate system.
-
- For example::
-
- >>> from iris.analysis.cartography import rotate_winds
- >>> u_cube = iris.load_cube('my_rotated_u_wind_cube.pp')
- >>> v_cube = iris.load_cube('my_rotated_v_wind_cube.pp')
- >>> target_cs = iris.coord_systems.GeogCS(6371229.0)
- >>> u_prime, v_prime = rotate_winds(u_cube, v_cube, target_cs)
-
-.. admonition:: Showcase: Nearest-neighbour scheme
-
- A nearest-neighbour scheme for interpolation and regridding has been added
- to Iris. This joins the existing :class:`~iris.analysis.Linear` and
- :class:`~iris.analysis.AreaWeighted` interpolation and regridding schemes.
-
- For example::
-
- >>> result = cube.interpolate(sample_points, iris.analysis.Nearest())
- >>> regridded_cube = cube.regrid(target_grid, iris.analysis.Nearest())
-
-.. admonition:: Showcase: Slices over a coordinate
-
- You can slice over one or more dimensions of a cube using :meth:`iris.cube.Cube.slices_over`.
- This provides similar functionality to :meth:`~iris.cube.Cube.slices` but with
- almost the opposite outcome.
-
- Using :meth:`~iris.cube.Cube.slices` to slice a cube on a selected dimension returns
- all possible slices of the cube with the selected dimension retaining its dimensionality.
- Using :meth:`~iris.cube.Cube.slices_over` to slice a cube on a selected
- dimension returns all possible slices of the cube over the selected dimension.
-
- To demonstrate this::
-
- >>> cube = iris.load(iris.sample_data_path('colpex.pp'))[0]
- >>> print(cube.summary(shorten=True))
- air_potential_temperature / (K) (time: 6; model_level_number: 10; grid_latitude: 83; grid_longitude: 83)
- >>> my_slice = next(cube.slices('time'))
- >>> my_slice_over = next(cube.slices_over('time'))
- >>> print(my_slice.summary(shorten=True))
- air_potential_temperature / (K) (time: 6)
- >>> print(my_slice_over.summary(shorten=True))
- air_potential_temperature / (K) (model_level_number: 10; grid_latitude: 83; grid_longitude: 83)
-
-
-* :func:`iris.cube.CubeList.concatenate` now works with `biggus `
- aggregator has been added.
-* A speed improvement in calculation of :func:`iris.analysis.cartography.area_weights`.
-* Experimental support for unstructured grids has been added with :func:`iris.experimental.ugrid`.
- This has been implemented using `UGRID `_.
-* :meth:`iris.cube.CubeList.extract_overlapping` supports extraction of cubes over
- regions where common coordinates overlap, over multiple coordinates.
-* Warnings raised due to invalid units in loaded data have been suppressed.
-* Experimental low-level read and write access for FieldsFile variants is now supported
- via :class:`iris.experimental.um.FieldsFileVariant`.
-* PP loader will return cubes for all fields prior to a field with a problematic
- header before raising an exception.
-* NetCDF loader skips invalid global attributes, raising a warning rather than raising an
- exception.
-* A warning is now raised rather than an exception when constructing an
- :class:`~iris.aux_factory.AuxCoordFactory` fails.
-* Supported :class:`aux coordinate factories `
- have been extended to include:
-
- * ``ocean sigma coordinate``,
- * ``ocean s coordinate``,
- * ``ocean s coordinate, generic form 1``, and
- * ``ocean s coordinate, generic form 2``.
-
-* :meth:`iris.cube.Cube.intersection` now supports taking a points-only intersection.
- Any bounds on intersected coordinates are ignored but retained.
-* The FF loader's known handled grids now includes ``Grid 21``.
-* A :class:`nearest neighbour ` scheme is now provided for
- :meth:`iris.cube.Cube.interpolate` and :meth:`iris.cube.Cube.regrid`.
-* :func:`iris.analysis.cartography.rotate_winds` supports transformation of wind vectors
- to a different coordinate system.
-* NumPy universal functions can now be applied to cubes using
- :func:`iris.analysis.maths.apply_ufunc`.
-* Generic functions can be applied to :class:`~iris.cube.Cube` instances using
- :class:`iris.analysis.maths.IFunc`.
-* The :class:`iris.analysis.Linear` scheme now supports regridding as well as interpolation.
- This enables :meth:`iris.cube.Cube.regrid` to perform bilinear regridding, which now
- replaces the experimental routine "iris.experimental.regrid.regrid_bilinear_rectilinear_src_and_grid".
-
-Bugs fixed
-==========
-
-1.8.0
-------
-* Fix in netCDF loader to correctly determine whether the longitude coordinate
- (including scalar coordinates) is circular.
-* :meth:`iris.cube.Cube.intersection` now supports bounds that extend slightly beyond 360
- degrees.
-* Lateral Boundary Condition (LBC) type FieldFiles are now handled correctly by the FF loader.
-* Making a copy of a scalar cube with no data now correctly copies the data array.
-* Height coordinates in NAME trajectory output files have been changed to match other
- NAME output file formats.
-* Fixed datatype when loading an ``integer_constants`` array from a FieldsFile.
-* FF/PP loader adds appropriate cell methods for ``lbtim.ib = 3`` intervals.
-* An exception is raised if the units of the latitude and longitude coordinates
- of the cube passed into :func:`iris.analysis.cartography.area_weights` are not
- convertible to radians.
-* GRIB1 loader now creates a time coordinate for a time range indicator of 2.
-* NetCDF loader now loads units that are empty strings as dimensionless.
-
-1.8.1
-------
-* The PP loader now carefully handles floating point errors in date time conversions to hours.
-* The handling fill values for lazy data loaded from NetCDF files is altered, such that the
- _FillValue set in the file is preserved through lazy operations.
-* The risk that cube intersections could return incorrect results due to floating point
- tolerances is reduced.
-* The new GRIB2 loading code is altered to enable the loading of various data representation
- templates; the data value unpacking is handled by the GRIB API.
-* Saving cube collections to NetCDF, where multiple similar aux-factories exist within the cubes,
- is now carefully handled such that extra file variables are created where required in some cases.
-
-1.8.2
------
-* A fix to prevent the error: *AttributeError: 'module' object has no attribute 'date2num'*.
- This was caused by the function :func:`netcdftime.date2num` being removed from the netCDF4
- package in recent versions.
-
-Deprecations
-============
-* The original GRIB loader has been deprecated and replaced with a new
- template-based GRIB loader.
-* Deprecated default NetCDF save behaviour of assigning the outermost
- dimension to be unlimited. Switch to the new behaviour with no auto
- assignment by setting :data:`iris.FUTURE.netcdf_no_unlimited` to True.
-* The former experimental method
- "iris.experimental.regrid.regrid_bilinear_rectilinear_src_and_grid" has been removed, as
- :class:`iris.analysis.Linear` now includes this functionality.
-
-Documentation Changes
-=====================
-* A chapter on :doc:`merge and concatenate ` has been
- added to the :doc:`user guide `.
-* A section on installing Iris using `conda `.)
-
-Iris 1.9 Features
-=================
-* Support for running on Python 3.4 has been added to the whole code base. Some features which
- depend on external libraries will not be available until they also support Python 3, namely:
-
- * gribapi does not yet provide a Python 3 interface
-
-* Added the UM pseudo level type to the information made available in the STASH_TRANS table in :mod:`iris.fileformats.um._ff_cross_references`
-* When reading "cell_methods" attributes from NetCDF files, allow optional whitespace before the colon.
- This is not strictly in the CF spec, but is a common occurrence.
-* Basic cube arithemetic (plus, minus, times, divide) now supports lazy evaluation.
-* :meth:`iris.analysis.cartography.rotate_winds` can now operate much faster on multi-layer (i.e. > 2-dimensional) cubes,
- as it calculates rotation coefficients only once and reuses them for additional layers.
-
-* Linear regridding of a multi-layer (i.e. > 2-dimensional) cube is now much faster,
- as it calculates transform coefficients just once and reuses them for additional layers.
-* Ensemble statistics can now be saved to GRIB2, using Product Definition Template 4.11.
-
-* Loading of NetCDF data with ocean vertical coordinates now returns a 'depth' in addition to an 'eta' cube.
- This operates on specific defined dimensionless coordinates : see CF spec version 1.6, Appendix D.
-
-* :func:`iris.analysis.stats.pearsonr` updates:
-
- * Cubes can now be different shapes, provided one is broadcastable to the
- other.
- * Accepts weights keyword for weighted correlations.
- * Accepts mdtol keyword for missing data tolerance level.
- * Accepts common_mask keyword for restricting calculation to unmasked pairs of
- cells.
-
-* Added a new point-in-cell regridding scheme, :class:`iris.experimental.regrid.PointInCell`.
-* Added :meth:`iris.analysis.WPERCENTILE` - a new weighted aggregator for calculating
- percentiles.
-* Added cell-method translations for LBPROC=64 and 192 in UM files, encoding 'zonal mean' and 'zonal+time mean'.
-
-* Support for loading GRIB2 messages defined on a Lambert conformal grid has been added to
- the GRIB2 loader.
-* Data on potential-temperature (theta) levels can now be saved to GRIB2, with a fixed surface type of 107.
-* Added several new helper functions for file-save customisation,
- (see also : :doc:`Saving Iris Cubes `):
-
- * :meth:`iris.fileformats.grib.as_pairs`
- * :meth:`iris.fileformats.grib.as_messages`
- * :meth:`iris.fileformats.grib.save_messages`
- * :meth:`iris.fileformats.pp.as_pairs`
- * :meth:`iris.fileformats.pp.as_fields`
- * :meth:`iris.fileformats.pp.save_fields`
-* Loading data from GRIB2 now supports most of the currently defined 'data representation templates' :
- code numbers 0, 1, 2, 3, 4, 40, 41, 50, 51 and 61.
-* When a Fieldsfile is opened for update as a :class:`iris.experimental.um.FieldsFileVariant`,
- unmodified packed data in the file can now be retained in the original form.
- Previously it could only be stored in an unpacked form.
-* When reading and writing NetCDF data, the CF 'flag' attributes,
- "flag_masks", "flag_meanings" and "flag_values" are now preserved through Iris load and save.
-* `mo_pack `_ was added as an optional dependency.
- It is used to encode and decode data in WGDOS packed form.
-* The :meth:`iris.experimental.um.Field.get_data` method can now be used to read Fieldsfile data
- after the original :class:`iris.experimental.um.FieldsFileVariant` has been closed.
-
-Bugs Fixed
-==========
-* Fixed a bug in :meth:`iris.unit.Unit.convert`
- (and the equivalent in `cf_units `_)
- so that it now converts data to the native endianness, without which udunits could not read it correctly.
-* Fixed a bug with loading WGDOS packed data in :mod:`iris.experimental.um`,
- which could occasionally crash, with some data.
-* Ignore non-numeric suffices in the numpy version string, which would otherwise crash some regridding routines.
-* fixed a bug in :mod:`iris.fileformats.um_cf_map` where the standard name
- for the stash code m01s12i187 was incorrectly set, such that it is inconsistent
- with the stated unit of measure, 'm s-1'. The different name, a long_name
- of 'change_over_time_in_upward_air_velocity_due_to_advection' with
- units of 'm s-1' is now used instead.
-* Fixed a bug in :meth:`iris.cube.Cube.intersection`.
- When edge points were at (base + period), intersection would unnecessarily wrap the data.
-* Fixed a bug in :mod:`iris.fileformats.pp`.
- A previous release removed the ability to pass a partial constraint on STASH attribute.
-* :meth:`iris.plot.default_projection_extent` now correctly raises an exception if a cube has X bounds but no Y bounds, or vice versa.
- Previously it never failed this, as the test was wrong.
-* When loading NetCDF data, a "units" attribute containing unicode characters is now transformed by backslash-replacement.
- Previously this caused a crash. Note: unicode units are *not supported in the CF conventions*.
-* When saving to NetCDF, factory-derived auxiliary coordinates are now correctly saved with different names when they are not identical.
- Previously, such coordinates could be saved with the same name, leading to errors.
-* Fixed a bug in :meth:`iris.experimental.um.FieldsFileVariant.close`,
- which now correctly allocates extra blocks for larger lookups when saving.
- Previously, when larger files open for update were closed, they could be written out with data overlapping the lookup table.
-* Fixed a bug in :class:`iris.aux_factory.OceanSigmaZFactory`
- which sometimes caused crashes when fetching the points of an "ocean sigma z" coordinate.
-
-Version 1.9.1
--------------
-* Fixed a unicode bug preventing standard names from being built cleanly when installing in Python3
-
-Version 1.9.2
--------------
-* New warning regarding data loss if writing to an open file which is also open to read, with lazy data.
-* Removal of a warning about data payload loading from concatenate.
-* Updates to concatenate documentation.
-* Fixed a bug with a name change in the netcdf4-python package.
-* Fixed a bug building the documentation examples.
-* Fixed a bug avoiding sorting classes directly when :meth:`iris.cube.Cube.coord_system` is used in Python3.
-* Fixed a bug regarding unsuccessful dot import.
-
-Incompatible Changes
-====================
-* GRIB message/file reading and writing may not be available for Python 3 due to GRIB API limitations.
-
-Deprecations
-============
-* Deprecated :mod:`iris.unit`, with unit functionality provided by `cf_units `_ instead.
-* When loading from NetCDF, a deprecation warning is emitted if there is vertical coordinate information
- that *would* produce extra result cubes if :data:`iris.FUTURE.netcdf_promote` were set,
- but it is *not* set.
-* Deprecated :class:`iris.aux_factory.LazyArray`
-
-Documentation Changes
-=====================
-* A chapter on :doc:`saving iris cubes ` has been
- added to the :doc:`user guide `.
-* Added script and documentation for building a what's new page from developer-submitted contributions.
- See :doc:`Contributing a "What's New" entry `.
diff --git a/docs/iris/src/whatsnew/2.3.rst b/docs/iris/src/whatsnew/2.3.rst
deleted file mode 100644
index c5a6060146..0000000000
--- a/docs/iris/src/whatsnew/2.3.rst
+++ /dev/null
@@ -1,215 +0,0 @@
-What's New in Iris 2.3.0
-************************
-
-:Release: 2.3.0
-:Date: 2019-10-04
-
-This document explains the new/changed features of Iris in version 2.3.0
-(:doc:`View all changes `.)
-
-Iris 2.3.0 Features
-===================
-.. _showcase:
-
-.. admonition:: Increased Support for CF 1.7
-
- We have introduced several changes that contribute to Iris's support for
- the CF Conventions, including some CF 1.7 additions. We are now able to
- support:
-
- * :ref:`Climatological Coordinates`
- * :ref:`Standard name modifiers`
- * :ref:`Geostationary projection`
-
- You can read more about each of these below.
-
- Additionally, the conventions attribute, added by Iris when saving to
- NetCDF, has been updated to "CF-1.7", accordingly.
-
-.. _climatological:
-.. admonition:: Climatological Coordinate Support
-
- Iris can now load, store and save `NetCDF climatological coordinates
- `_. Any cube time
- coordinate can be marked as a climatological time axis using the boolean
- property: ``climatological``. The climatological bounds are stored in the
- coordinate's ``bounds`` property.
-
- When an Iris climatological coordinate is saved in NetCDF, the NetCDF
- coordinate variable will be given a 'climatology' attribute, and the
- contents of the
- coordinate's ``bounds`` property are written to a NetCDF boundary variable
- called '_bounds'. These are in place of a standard
- 'bounds' attribute and accompanying boundary variable. See below
- for an `example adapted from CF conventions `_:
-
- .. code-block:: none
-
- dimensions:
- time=4;
- bnds=2;
- variables:
- float temperature(time,lat,lon);
- temperature:long_name="surface air temperature";
- temperature:cell_methods="time: minimum within years time: mean over years";
- temperature:units="K";
- double time(time);
- time:climatology="time_climatology";
- time:units="days since 1960-1-1";
- double time_climatology(time,bnds);
- data: // time coordinates translated to date/time format
- time="1960-4-16", "1960-7-16", "1960-10-16", "1961-1-16" ;
- time_climatology="1960-3-1", "1990-6-1",
- "1960-6-1", "1990-9-1",
- "1960-9-1", "1990-12-1",
- "1960-12-1", "1991-3-1" ;
-
- If a climatological time axis is detected when loading NetCDF -
- indicated by the format described above - the ``climatological`` property
- of the Iris coordinate will be set to ``True``.
-
-.. admonition:: New Chunking Strategy
-
- Iris now makes better choices of Dask chunk sizes when loading from NetCDF
- files: If a file variable has small, specified chunks, Iris will now choose
- Dask chunks which are a multiple of these up to a default target size.
-
- This is particularly relevant to files with an unlimited dimension, which
- previously could produce a large number of small chunks. This had an adverse
- effect on performance.
-
- In addition, Iris now takes its default chunksize from the default configured
- in Dask itself, i.e. ``dask.config.get('array.chunk-size')``.
-
-.. admonition:: Lazy Statistics
-
- Several statistical operations can now be done lazily, taking advantage of the
- performance improvements offered by Dask:
-
- * :meth:`~iris.cube.Cube.aggregated_by`
- * :class:`~iris.analysis.RMS` (more detail below)
- * :class:`~iris.analysis.MEAN`
-
-----
-
-.. _geostationary:
-.. _standard_name:
-.. _conventions_1.7:
-
-* Cube data equality testing (and hence cube equality) now uses a more
- relaxed
- tolerance : This means that some cubes may now test 'equal' that previously
- did not.
- Previously, Iris compared cube data arrays using:
- ``abs(a - b) < 1.e-8``
-
- We now apply the default operation of :func:`numpy.allclose` instead,
- which is equivalent to:
- ``abs(a - b) < (1.e-8 + 1.e-5 * b)``
-
-* Added support to render HTML for :class:`~iris.cube.CubeList` in Jupyter
- Notebooks and JupyterLab.
-* Loading CellMeasures with integer values is now supported.
-* New coordinate system: :class:`iris.coord_systems.Geostationary`,
- including load and save support, based on the `CF Geostationary projection
- definition `_.
-* :class:`iris.coord_systems.VerticalPerspective` can now be saved to and
- loaded from NetCDF files.
-* :class:`iris.experimental.regrid.PointInCell` moved to
- :class:`iris.analysis.PointInCell` to make this regridding scheme public
-* Iris now supports standard name modifiers. See `Appendix C, Standard Name Modifiers `_ for more information.
-* :meth:`iris.cube.Cube.remove_cell_measure` now also allows removal of a cell
- measure by its name (previously only accepted a CellMeasure object).
-* The :data:`iris.analysis.RMS` aggregator now supports a lazy calculation.
- However, the "weights" keyword is not currently supported by this, so a
- *weighted* calculation will still return a realised result, *and* force
- realisation of the original cube data.
-* Iris now supports NetCDF Climate and Forecast (CF) Metadata Conventions 1.7 (see `CF 1.7 Conventions Document `_ for more information)
-
-
-Iris 2.3.0 Dependency Updates
-=============================
-* Iris now supports Proj4 up to version 5, but not yet 6 or beyond, pending
- `fixes to some cartopy tests `_.
-* Iris now requires Dask >= 1.2 to allow for improved coordinate equality
- checks.
-
-
-Bugs Fixed
-==========
-* Cube equality of boolean data is now handled correctly.
-* Fixed a bug where cell measures were incorrect after a cube
- :meth:`~iris.cube.Cube.transpose` operation. Previously, this resulted in
- cell-measures that were no longer correctly mapped to the cube dimensions.
-* The :class:`~iris.coords.AuxCoord` disregarded masked points and bounds, as did the :class:`~iris.coords.DimCoord`.
- Fix permits an :class:`~iris.coords.AuxCoord` to contain masked points/bounds, and a TypeError exception is now
- raised when attempting to create or set the points/bounds of a
- :class:`~iris.coords.DimCoord` with arrays with missing points.
-* :class:`iris.coord_systems.VerticalPerspective` coordinate system now uses
- the `CF Vertical perspective definition `_; had been
- erroneously using Geostationary.
-* :class:`~iris.coords.CellMethod` will now only use valid `NetCDF name tokens `_ to reference the coordinates involved in the statistical operation.
-* The following var_name properties will now only allow valid `NetCDF name
- tokens
- `_ to
- reference the said NetCDF variable name. Note that names with a leading
- underscore are not permitted.
- - :attr:`iris.aux_factory.AuxCoordFactory.var_name`
- - :attr:`iris.coords.CellMeasure.var_name`
- - :attr:`iris.coords.Coord.var_name`
- - :attr:`iris.coords.AuxCoord.var_name`
- - :attr:`iris.cube.Cube.var_name`
-* Rendering a cube in Jupyter will no longer crash for a cube with
- attributes containing ``\n``.
-* NetCDF variables which reference themselves in their ``cell_measures``
- attribute can now be read.
-* :func:`~iris.plot.quiver` now handles circular coordinates.
-* The names of cubes loaded from abf/abl files have been corrected.
-* Fixed a bug in UM file loading, where any landsea-mask-compressed fields
- (i.e. with LBPACK=x2x) would cause an error later, when realising the data.
-* :meth:`iris.cube.Cube.collapsed` now handles partial collapsing of
- multidimensional coordinates that have bounds.
-* Fixed a bug in the :data:`~iris.analysis.PROPORTION` aggregator, where cube
- data in the form of a masked array with ``array.mask=False`` would cause an
- error, but possibly only later when the values are actually realised.
- ( Note: since netCDF4 version 1.4.0, this is now a common form for data
- loaded from netCDF files ).
-* Fixed a bug where plotting a cube with a
- :class:`iris.coord_systems.LambertConformal` coordinate system would result
- in an error. This would happen if the coordinate system was defined with one
- standard parallel, rather than two.
- In these cases, a call to
- :meth:`~iris.coord_systems.LambertConformal.as_cartopy_crs` would fail.
-* :meth:`iris.cube.Cube.aggregated_by` now gives correct values in points and
- bounds when handling multidimensional coordinates.
-* Fixed a bug in the :meth:`iris.cube.Cube.collapsed` operation, which caused
- the unexpected realization of any attached auxiliary coordinates that were
- *bounded*. It now correctly produces a lazy result and does not realise
- the original attached AuxCoords.
-
-
-Documentation Changes
-=====================
-* Added a gallery example showing `how to concatenate NEMO ocean model data
- <../examples/Oceanography/load_nemo.html>`_.
-* Added an example in the
- `Loading Iris Cubes: Constraining on Time <../userguide/loading_iris_cubes
- .html#constraining-on-time>`_
- Userguide section, demonstrating how to load data within a specified date
- range.
-* Added notes to the :func:`iris.load` documentation, and the userguide
- `Loading Iris Cubes <../userguide/loading_iris_cubes.html>`_
- chapter, emphasizing that the *order* of the cubes returned by an iris load
- operation is effectively random and unstable, and should not be relied on.
-* Fixed references in the documentation of
- :func:`iris.util.find_discontiguities` to a nonexistent
- "mask_discontiguities" routine : these now refer to
- :func:`~iris.util.mask_cube`.
-
diff --git a/docs/iris/src/whatsnew/aggregate_directory.py b/docs/iris/src/whatsnew/aggregate_directory.py
deleted file mode 100644
index fca098f4d4..0000000000
--- a/docs/iris/src/whatsnew/aggregate_directory.py
+++ /dev/null
@@ -1,322 +0,0 @@
-# (C) British Crown Copyright 2015 - 2019, Met Office
-#
-# This file is part of Iris.
-#
-# Iris is free software: you can redistribute it and/or modify it under
-# the terms of the GNU Lesser General Public License as published by the
-# Free Software Foundation, either version 3 of the License, or
-# (at your option) any later version.
-#
-# Iris is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-# GNU Lesser General Public License for more details.
-#
-# You should have received a copy of the GNU Lesser General Public License
-# along with Iris. If not, see ".
-Takes specified "xx.xx" as version, or latest found (alphabetic).
-Writes a file "<...whatsnew>/.rst".
-
-Valid contributions filenames are of the form:
- __summary.txt
-Where can be any valid chars, and
- is one of :
- "newfeature" "bugfix" "incompatiblechange" "deprecate" "docchange", and
- is in the style "2001-Jan-23".
-
-"""
-
-from __future__ import (absolute_import, division, print_function)
-from six.moves import (filter, input, map, range, zip) # noqa
-import datetime
-from glob import glob
-import os
-import re
-import argparse
-import warnings
-from operator import itemgetter
-from distutils import version
-
-# Regular expressions: CONTRIBUTION_REGEX matches the filenames of
-# contribution snippets. It is split into three sections separated by _
-# 0. String for the category. 1. ISO8601 date. 2. String for the feature name.
-# RELEASE_REGEX matches the directory names, returning the release.
-CONTRIBUTION_REGEX_STRING = r'(?P.*)'
-CONTRIBUTION_REGEX_STRING += r'_(?P\d{4}-\w{3}-\d{2})'
-CONTRIBUTION_REGEX_STRING += r'_(?P.*)\.txt$'
-CONTRIBUTION_REGEX = re.compile(CONTRIBUTION_REGEX_STRING)
-RELEASEDIR_PREFIX = r'contributions_'
-_RELEASEDIR_REGEX_STRING = RELEASEDIR_PREFIX + r'(?P.*)$'
-RELEASE_REGEX = re.compile(_RELEASEDIR_REGEX_STRING)
-SOFTWARE_NAME = 'Iris'
-EXTENSION = '.rst'
-VALID_CATEGORIES = [
- {'Prefix': 'newfeature', 'Title': 'Features'},
- {'Prefix': 'bugfix', 'Title': 'Bugs Fixed'},
- {'Prefix': 'incompatiblechange', 'Title': 'Incompatible Changes'},
- {'Prefix': 'deprecate', 'Title': 'Deprecations'},
- {'Prefix': 'docchange', 'Title': 'Documentation Changes'}
-]
-VALID_CATEGORY_PREFIXES = [cat['Prefix'] for cat in VALID_CATEGORIES]
-
-
-def _self_root_directory():
- return os.path.abspath(os.path.dirname(__file__))
-
-
-def _decode_contribution_filename(file_name):
- file_name_elements = CONTRIBUTION_REGEX.match(file_name)
- category = file_name_elements.group('category')
- if category not in VALID_CATEGORY_PREFIXES:
- # This is an error
- raise ValueError('Unknown category in contribution filename.')
- isodate = file_name_elements.group('isodate')
- date_of_item = datetime.datetime.strptime(isodate, '%Y-%b-%d').date()
- return category, isodate, date_of_item
-
-
-def is_release_directory(directory_name, release):
- '''Returns True if a given directory name matches the requested release.'''
- result = False
- directory_elements = RELEASE_REGEX.match(directory_name)
- try:
- release_string = directory_elements.group('release')
- directory_release = version.StrictVersion(release_string)
- except (AttributeError, ValueError):
- pass
- else:
- if directory_release == release:
- result = True
- return result
-
-
-def is_compiled_release(root_directory, release):
- '''Returns True if the requested release.rst file exists.'''
- result = False
- compiled_filename = '{!s}{}'.format(release, EXTENSION)
- compiled_filepath = os.path.join(root_directory, compiled_filename)
- if os.path.exists(compiled_filepath) and os.path.isfile(compiled_filepath):
- result = True
- return result
-
-
-def get_latest_release(root_directory=None):
- """
- Implement default=latest release identification.
-
- Returns a valid release code.
-
- """
- if root_directory is None:
- root_directory = _self_root_directory()
- directory_contents = os.listdir(root_directory)
- # Default release to latest visible dir.
- possible_release_dirs = [releasedir_name
- for releasedir_name in directory_contents
- if RELEASE_REGEX.match(releasedir_name)]
- if len(possible_release_dirs) == 0:
- dirspec = os.path.join(root_directory, RELEASEDIR_PREFIX + '*')
- msg = 'No valid release directories found, i.e. {!r}.'
- raise ValueError(msg.format(dirspec))
- release_dirname = sorted(possible_release_dirs)[-1]
- release = RELEASE_REGEX.match(release_dirname).group('release')
- return release
-
-
-def find_release_directory(root_directory, release=None,
- fail_on_existing=True):
- '''
- Returns the matching contribution directory or raises an exception.
-
- Defaults to latest-found release (from release directory names).
- Optionally, fail if the matching release file already exists.
- *Always* fail if no release directory exists.
-
- '''
- if release is None:
- # Default to latest release.
- release = get_latest_release(root_directory)
-
- if fail_on_existing:
- compiled_release = is_compiled_release(root_directory, release)
- if compiled_release:
- msg = ('Specified release {!r} is already compiled : '
- '{!r} already exists.')
- compiled_filename = str(release) + EXTENSION
- raise ValueError(msg.format(release, compiled_filename))
-
- directory_contents = os.listdir(root_directory)
- result = None
- for inode in directory_contents:
- node_path = os.path.join(root_directory, inode)
- if os.path.isdir(node_path):
- release_directory = is_release_directory(inode, release)
- if release_directory:
- result = os.path.join(root_directory, inode)
- break
- if not result:
- msg = 'Contribution folder for release {!s} does not exist : no {!r}.'
- release_dirname = RELEASEDIR_PREFIX + str(release) + '/'
- release_dirpath = os.path.join(root_directory, release_dirname)
- raise ValueError(msg.format(release, release_dirpath))
- return result
-
-
-def generate_header(release, unreleased=False):
- '''Return a list of text lines that make up a header for the document.'''
- if unreleased:
- isodatestamp = ''
- else:
- isodatestamp = datetime.date.today().strftime('%Y-%m-%d')
- header_text = []
- title_template = 'What\'s New in {} {!s}\n'
- title_line = title_template.format(SOFTWARE_NAME, release)
- title_underline = ('*' * (len(title_line) - 1)) + '\n'
- header_text.append(title_line)
- header_text.append(title_underline)
- header_text.append('\n')
- header_text.append(':Release: {!s}\n'.format(release))
- header_text.append(':Date: {}\n'.format(isodatestamp))
- header_text.append('\n')
- description_template = 'This document explains the new/changed features '\
- 'of {} in version {!s}\n'
- header_text.append(description_template.format(SOFTWARE_NAME, release))
- header_text.append('(:doc:`View all changes `.)')
- header_text.append('\n')
- return header_text
-
-
-def read_directory(directory_path):
- '''Parse the items in a specified directory and return their metadata.'''
- directory_contents = os.listdir(directory_path)
- compilable_files_unsorted = []
- misnamed_files = []
- for file_name in directory_contents:
- try:
- category, isodate, date_of_item = \
- _decode_contribution_filename(file_name)
- except (AttributeError, ValueError):
- misnamed_files.append(file_name)
- continue
- compilable_files_unsorted.append({'Category': category,
- 'Date': date_of_item,
- 'FileName': file_name})
- compilable_files = sorted(compilable_files_unsorted,
- key=itemgetter('Date'),
- reverse=True)
- if misnamed_files:
- msg = 'Found contribution file(s) with unexpected names :'
- for filename in misnamed_files:
- full_path = os.path.join(directory_path, filename)
- msg += '\n {}'.format(full_path)
- warnings.warn(msg, UserWarning)
-
- return compilable_files
-
-
-def compile_directory(directory, release, unreleased=False):
- '''Read in source files in date order and compile the text into a list.'''
- if unreleased:
- release = ''
- source_text = read_directory(directory)
- compiled_text = []
- header_text = generate_header(release, unreleased)
- compiled_text.extend(header_text)
- for count, category in enumerate(VALID_CATEGORIES):
- category_text = []
- subtitle_line = ''
- if count == 0:
- subtitle_line += '{} {!s} '.format(SOFTWARE_NAME, release)
- subtitle_line += category['Title'] + '\n'
- subtitle_underline = ('=' * (len(subtitle_line) - 1)) + '\n'
- category_text.append('\n')
- category_text.append(subtitle_line)
- category_text.append(subtitle_underline)
- category_items = [item for item in source_text
- if item['Category'] == category['Prefix']]
- if not category_items:
- continue
- for file_description in category_items:
- entry_path = os.path.join(directory, file_description['FileName'])
- with open(entry_path, 'r') as content_object:
- text = content_object.readlines()
- if not text[-1].endswith('\n'):
- text[-1] += '\n'
- category_text.extend(text)
- category_text.append('\n----\n\n')
- compiled_text.extend(category_text)
- return compiled_text
-
-
-def check_all_contributions_valid(release=None, quiet=False, unreleased=False):
- """"Scan the contributions directory for badly-named files."""
- root_directory = _self_root_directory()
- # Check there are *some* contributions directory(s), else silently pass.
- contribs_spec = os.path.join(root_directory, RELEASEDIR_PREFIX + '*')
- if len(glob(contribs_spec)) > 0:
- # There are some contributions directories: check latest / specified.
- if release is None:
- release = get_latest_release()
- if not quiet:
- msg = 'Checking whatsnew contributions for release "{!s}".'
- print(msg.format(release))
- release_directory = find_release_directory(root_directory, release,
- fail_on_existing=False)
- # Run the directory scan, but convert any warning into an error.
- with warnings.catch_warnings():
- warnings.simplefilter('error')
- compile_directory(release_directory, release, unreleased)
- if not quiet:
- print('done.')
-
-
-def run_compilation(release=None, quiet=False, unreleased=False):
- '''Write a draft release.rst file given a specified uncompiled release.'''
- if release is None:
- # This must exist !
- release = get_latest_release()
- if not quiet:
- msg = 'Building release document for release "{!s}".'
- print(msg.format(release))
- root_directory = _self_root_directory()
- release_directory = find_release_directory(root_directory, release)
- compiled_text = compile_directory(release_directory, release, unreleased)
- if unreleased:
- compiled_filename = 'latest' + EXTENSION
- else:
- compiled_filename = str(release) + EXTENSION
- compiled_filepath = os.path.join(root_directory, compiled_filename)
- with open(compiled_filepath, 'w') as output_object:
- for string_line in compiled_text:
- output_object.write(string_line)
- if not quiet:
- print('done.')
-
-
-if __name__ == '__main__':
- PARSER = argparse.ArgumentParser()
- PARSER.add_argument("release", help="Release number to be compiled",
- nargs='?', type=version.StrictVersion)
- PARSER.add_argument(
- '-c', '--checkonly', action='store_true',
- help="Check contribution file names, do not build.")
- PARSER.add_argument(
- '-u', '--unreleased', action='store_true',
- help=("Label the release version as '', "
- "and its date as ''."))
- PARSER.add_argument(
- '-q', '--quiet', action='store_true',
- help="Do not print progress messages.")
- ARGUMENTS = PARSER.parse_args()
- release = ARGUMENTS.release
- unreleased = ARGUMENTS.unreleased
- quiet = ARGUMENTS.quiet
- if ARGUMENTS.checkonly:
- check_all_contributions_valid(release, quiet=quiet,
- unreleased=unreleased)
- else:
- run_compilation(release, quiet=quiet, unreleased=unreleased)
diff --git a/docs/iris/src/whatsnew/contributions_3.0.0/deprecate_2019-Oct-14_remove_deprecated_future_flags.txt b/docs/iris/src/whatsnew/contributions_3.0.0/deprecate_2019-Oct-14_remove_deprecated_future_flags.txt
deleted file mode 100644
index 5bd2903e9b..0000000000
--- a/docs/iris/src/whatsnew/contributions_3.0.0/deprecate_2019-Oct-14_remove_deprecated_future_flags.txt
+++ /dev/null
@@ -1,3 +0,0 @@
-* The deprecated :class:`iris.Future` flags `cell_date_time_objects`,
- `netcfd_promote`, `netcdf_no_unlimited` and `clip_latitudes` have
- been removed.
\ No newline at end of file
diff --git a/docs/iris/src/whatsnew/index.rst b/docs/iris/src/whatsnew/index.rst
deleted file mode 100644
index 179216ccb5..0000000000
--- a/docs/iris/src/whatsnew/index.rst
+++ /dev/null
@@ -1,31 +0,0 @@
-.. _iris_whatsnew:
-
-What's new in Iris
-******************
-
-These "What's new" pages describe the important changes between major
-Iris versions.
-
-.. toctree::
- :maxdepth: 2
-
- latest.rst
- 3.0.rst
- 2.3.rst
- 2.2.rst
- 2.1.rst
- 2.0.rst
- 1.13.rst
- 1.12.rst
- 1.11.rst
- 1.10.rst
- 1.9.rst
- 1.8.rst
- 1.7.rst
- 1.6.rst
- 1.5.rst
- 1.4.rst
- 1.3.rst
- 1.2.rst
- 1.1.rst
- 1.0.rst
diff --git a/docs/iris/src/whitepapers/change_management.rst b/docs/iris/src/whitepapers/change_management.rst
deleted file mode 100644
index b279c91b96..0000000000
--- a/docs/iris/src/whitepapers/change_management.rst
+++ /dev/null
@@ -1,451 +0,0 @@
-.. _change_management:
-
-Change Management in Iris from the User's perspective
-*****************************************************
-
-As Iris changes, user code will need revising from time to time to keep it
-working, or to maintain best practice. At the very least, you are advised to
-review existing code to ensure it functions correctly with new releases.
-
-Here, we define ways to make this as easy as possible.
-
-.. include:: ../userguide/change_management_goals.txt
-
-
-Key principles you can rely on
-==============================
-
-Iris code editions are published as defined version releases, with a given
-major and minor version number in the version name, "major.minor.xxx",
-as explained in the :ref:`releases section ` below.
-
- * Code that currently works should **still work**, and have the same
- results and effects, in any subsequent sub-release with the same major
- release number.
-
- * The only time we will make changes that can break existing code is at
- a **major release**.
-
- * At a major release, code that works **and emits no deprecation warnings**
- in the latest previous (minor) release should still work, and have the
- same results and effects.
-
-
-**What can possibly go wrong ?**
-
-If your code produces :ref:`deprecation warnings `, then it
-*could* behave differently, or fail, at the next major release.
-
-
-
-User Actions : How you should respond to changes and releases
-=============================================================
-
-Checklist :
-
-* when a new **testing or candidate version** is announced
- if convenient, test your working legacy code against it and report any problems.
-
-* when a new **minor version is released**
-
- * review the 'Whats New' documentation to see if it introduces any
- deprecations that may affect you.
- * run your working legacy code and check for any deprecation warnings,
- indicating that modifications may be necessary at some point
- * when convenient :
-
- * review existing code for use of deprecated features
- * rewrite code to replace deprecated features
-
-* when a new major version is **announced**
- ensure your code runs, without producing deprecation warnings, in the
- previous minor release
-
-* when a new major version is **released**
- check for new deprecation warnings, as for a minor release
-
-
-Details
-=======
-
-The Iris change process aims to minimise the negative effects of change, by
-providing :
-
- * defined processes for release and change management
- * release versioning
- * backwards code compatibility through minor version releases
- * a way to ensure compatibility with a new major version release
- * deprecation notices and warnings to highlight all impending changes
-
-Our practices are intended be compatible with the principles defined in the
-`SemVer project `
- * :ref:`Backwards compatibility `
- * :ref:`Deprecation `
-
-
-.. _iris_backward_compatibility:
-
-Backwards compatibility
------------------------
-
-"Backwards-compatible" changes are those that leave any existing valid API
-usages unchanged (see :ref:`terminology ` below).
-Minor releases may only include backwards-compatible changes.
-
-The following are examples of backward-compatible changes :
-
- * changes to documentation
- * adding to a module : new submodules, functions, classes or properties
- * adding to a class : new methods or properties
- * adding to a function or method : new **optional** arguments or keywords
-
-The following are examples of **non-** backward-compatible changes :
-
- * removing (which includes *renaming*) any public module or submodule
- * removing any public component : a module, class, method, function or
- data object property of a public API component
- * removing any property of a public object
- * removing an argument or keyword from a method or function
- * adding a required argument to a method or function
- * removing a keyword (even one that has no effect)
- * changing the effect of *any* particular combination of arguments and/or
- keywords
-
-Note that it is also possible to modify the behaviour of an existing usage by
-making it depend on a newly-defined external control variable. This is
-effectively a change to the 'default behaviour' of a specific usage. Although
-this seems similar to adding a keyword, the cases where the new behaviour
-operates and where it does not are not distinguishable by a different code
-usage, which makes this somewhat dangerous. We do use this type of change,
-but any behaviour 'mode' controls of this sort are usually added as part of the
-:class:`iris.Future` definition.
-See :ref:`Usage of iris.FUTURE `, below.
-
-
-.. _iris_api:
-
-Terminology : API, features, usages and behaviours
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-
-The API is the components of the iris module and its submodules which are
-"public" : In Python, by convention, this normally means everything whose name
-does not have a single leading underscore "_".
-This applies to all public modules and their properties : submodules, contained
-public classes, data and properties, functions and methods.
-An exception is when a module or class defines an '__all__' property : In
-that case, the public aspects are just the ones listed there.
-
-Note: these are standard Python conventions, not specific to Iris.
-See: `PEP8 ..", available to code as
-:data:`iris.__version__` .
-
-This contains major and minor release numbers. The numbering and meaning of
-these are defined, following the `SemVer project .." arrangement
-are :
-
- * "", "" and "" are all integers, thus version
- 2.12 is later than 2.2 (i.e. it is "two point twelve", not "two point one
- two").
-
- * "." denote the software release version.
-
- * A non-zero "" denotes a bugfix version, thus a release "X.Y.0" may
- be followed by "X.Y.1", "X.Y.2" etc, which *only* differ by containing
- bugfixes. Any bugfix release supercedes its predecessors, and does not
- change any (valid) APIs or behaviour : hence, it is always advised to
- replace a given version with its latest bugfix successor, and there
- should be no reason not to.
-
- * "" is blank for formal releases. It used to indicate
- provisional software for testing : The version string in a development
- code branch is always labelled "-DEV", and release candidates for testing
- during the release process are labelled "-rc1", "-rc2" etc.
- For development code, the version number is that of the *next* release,
- which this code version is progressing towards, e.g. "1.2-DEV" for all
- code branches since the 1.1 release and intended for release in "1.2".
-
-.. note::
- Our use of "-" is typical, but does not follow strict SemVer
- principles.
-
-The code for a specific release is identified by a git tag which is the version
-string : see
-:ref:`Developer's Guide section on releases `.
-
-
-Major and Minor Releases
-^^^^^^^^^^^^^^^^^^^^^^^^
-
-The term "release" refers both to a specific state of the Iris code, which we
-have assigned a given version string, *and* the act of defining it
-(i.e. we "release a release").
-
-According to `SemVer `_.
-For the developer view of this, see
-:ref:`Developer's Guide section on deprecations `.
-
-Deprecation manages incompatible changes in a strictly controlled way.
-This allows APIs to evolve to the most effective form, even when that means
-that existing code could behave differently or fail : This is important
-because the freedom to remove features helps prevent the API becoming
-progressively cluttered, and modifying existing behaviours allows us to use
-the most natural code syntax for the most used features.
-
-We can only remove features or change behaviours at a major release. Thus, we
-first deprecate the feature in a minor release, to provide adequate warning
-that existing code may need to be modified.
-
-When we make a release that introduces a deprecation :
-
- * a deprecation notice appears in the
- :ref:`What's New section `
- * deprecation notes are included in all relevant parts of the :ref:`reference
- documentation `
- * a runtime warning is produced when the old feature is used or triggered
-
-In most cases, we also provide detailed advice in the documentation and/or
-warning messages on how to replace existing usage with a 'new' way of doing
-things.
-In all cases, we must provide a transitional period where both old and new
-features are available :
-
- * the 'old' style works as it did before
- * any usage of the 'old' features will emit a
- :class:`warnings.WarningMessage` message, noting that the feature is
- deprecated and what to use instead
- * the 'new' style can be adopted as soon as convenient
-
-This is to warn users :
-
- * not to use the deprecated features in any new code, *and*
- * eventually to rewrite old code to use the newer or better alternatives
-
-
-Deprecated features support through the Release cycle
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-
-The whole point of a deprecation is that the feature continues to work, but
-with a warning, for some time before an unavoidable change occurs.
-When a version that introduces a deprecation is released, the effects are as
-follows:
-
- * code that may be affected by the proposed change will result in
- deprecation warnings
- * code that currently works will, however, continue to work unchanged, at
- least until the next major release
- * you can avoid all deprecation warnings by suitable changes to your code
- * code which uses no deprecated features, and thus produces no deprecation
- warnings, will continue to work unchanged even at a **major** release
- * code that generates deprecation warnings may cease to work at the next
- **major** release.
-
-
-.. _iris_future_usage:
-
-Future options, `iris.FUTURE`
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-
-A special approach is needed where the replacement behaviour is not controlled
-by a distinct API usage.
-
-When we extend an API, or add a new feature, we usually add a new method or
-keyword. In those cases, code using the new feature is clearly distinct from
-any previous (valid) usage, so this is relatively simple to manage.
-However, sometimes we really need to change the *way* an API works, without
-modifying or extending (i.e. complicating) the existing user interface.
-In that case, existing user code might sometimes have *different* behaviour
-with the new release, which we obviously need to avoid.
-
-**For example :**
-
- We might decide there is a more useful way of loading cubes from files of a
- particular input data format.
-
- * the user code usage is simply by calls to "iris.load"
- * the change is not a bugfix, as the old way isn't actually "wrong"
- * we don't want to add an extra keyword into all the relevant calls
- * we don't see a longterm future for the existing behaviour : we
- expect everyone to adopt the new interpretation, eventually
-
-For changes of this sort, the release will define a new boolean property of the
-:data:`iris.FUTURE` object, as a control to select between the 'old' and 'new'
-behaviours, with values False='old' and True='new'.
-See :data:`iris.Future` for examples.
-
-In these cases, as any "deprecated usage" is not clearly distinguishable in the
-form of the user code, it is **especially** important to take note of any
-deprecation messages appearing when legacy code runs.
-
-
-**Sequence of changes to `iris.FUTURE`**
-
-To allow user code to avoid unexpected any behavioural changes, the
-:data:`iris.Future` controls follow a special management cycle, as follows
-(see also the relevant :ref:`Developer Guide section `):
-
-At (minor) release "...":
- * Changes to API:
- * the new behaviour is made available, alongside the old one
-
- * a new future option `iris.FUTURE.` is provided to switch
- between them.
-
- * the new option defaults to `iris.FUTURE.=False`, meaning
- the 'old' behaviour is the default.
-
- * when any relevant API call is made that invokes the old behaviour, a
- deprecation warning is emitted.
-
- * User actions:
-
- * If your code encounters the new deprecation warning, you should try
- enabling the new control option, and make any necessary rewrites to
- make it work. This will stop the deprecation warning appearing.
-
- * If you encounter problems making your code work with the new
- behaviour, and don't have time to fix them, you should make your
- code explicitly turn *off* the option for now, i.e. ::
- `iris.FUTURE. = False`.
- This locks you into the old behaviour, but your code will continue
- to work, even beyond the next major release when the default
- behaviour will change (see on).
-
-At (major) release ".0...":
- * Changes to API:
- * the control default is changed to `iris.FUTURE.=True`
-
- * the control property is *itself* deprecated, so that assigning to it
- now results in a deprecation warning.
-
- * when any affected API call is made, a deprecation warning is (still)
- emitted, if the old behaviour is in force. The "old" behaviour is,
- however, still available and functional.
-
- * User actions:
-
- * If your code is already using the "new" behaviour, it will now work
- without needing to set the Future option. *You should remove* the
- code which enables the option, as this will now emit a deprecation
- message. In the *next* major release, this would cause an error.
-
- * If your code is explicitly turning the option off, it will continue
- to work in the same way at this point, but obviously time is
- runnning out.
-
- * If your code is still using the old behaviour and *not* setting the
- control option at all, its behaviour might now have changed
- unexpectedly and you should review this.
-
-At (major) release "...":
- * Changes to API:
- * the control property is removed
- * the "old" behaviour is removed
-
-
-
-.. _iris_dependency_versions:
-
-Versions of Installed Dependencies
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-
-The dependencies of Iris (required and optional) are defined in
-:ref:`installing_iris`.
-
-Iris does not specify exact required versions for its dependencies, but it may
-specify a minimum acceptable version number. Iris is normally expected to be
-compatible with *any* version up to the latest current release.
-
-When a new release of a dependency is found to cause problems, Iris may define
-the supported version more precisely, but this would be a temporary fix which
-should be removed in a later release.
diff --git a/docs/iris/src/whitepapers/index.rst b/docs/iris/src/whitepapers/index.rst
deleted file mode 100644
index dd0876d257..0000000000
--- a/docs/iris/src/whitepapers/index.rst
+++ /dev/null
@@ -1,12 +0,0 @@
-.. _whitepapers_index:
-
-============================
-Iris technical 'Whitepapers'
-============================
-Extra information on specific technical issues.
-
-.. toctree::
- :maxdepth: 1
-
- um_files_loading.rst
- missing_data_handling.rst
diff --git a/docs/iris/src/IEP/IEP001.adoc b/docs/src/IEP/IEP001.adoc
similarity index 82%
rename from docs/iris/src/IEP/IEP001.adoc
rename to docs/src/IEP/IEP001.adoc
index d38b2e8478..2daef2363a 100644
--- a/docs/iris/src/IEP/IEP001.adoc
+++ b/docs/src/IEP/IEP001.adoc
@@ -119,7 +119,7 @@ cube.sel(height=1.5)
The semantics of position-based slices will continue to match that of normal Python slices. The start position is included, the end position is excluded.
-Value-based slices will be stricly inclusive, with both the start and end values included. This behaviour differs from normal Python slices but is in common with pandas.
+Value-based slices will be strictly inclusive, with both the start and end values included. This behaviour differs from normal Python slices but is in common with pandas.
Just as for normal Python slices, we do not need to provide the ability to control the include/exclude behaviour for slicing.
@@ -162,12 +162,12 @@ There is a risk that this topic could bog down when dealing with non-standard ca
* Boolean array indexing
* Lambdas?
* What to do about constrained loading?
-* Relationship to http://scitools.org.uk/iris/docs/v1.9.2/iris/iris/cube.html#iris.cube.Cube.intersection[iris.cube.Cube.intersection]?
+* Relationship to https://scitools.org.uk/iris/docs/v1.9.2/iris/iris/cube.html#iris.cube.Cube.intersection[iris.cube.Cube.intersection]?
* Relationship to interpolation (especially nearest-neighbour)?
** e.g. What to do about values that don't exist?
*** pandas throws a KeyError
-*** xarray supports (several) nearest-neighbour schemes via http://xarray.pydata.org/en/stable/indexing.html#nearest-neighbor-lookups[`data.sel()`]
-*** Apparently http://holoviews.org/[holoviews] does nearest-neighbour interpolation.
+*** xarray supports (several) nearest-neighbour schemes via https://xarray.pydata.org/en/stable/indexing.html#nearest-neighbor-lookups[`data.sel()`]
+*** Apparently https://holoviews.org/[holoviews] does nearest-neighbour interpolation.
* multi-dimensional coordinate => unroll?
* var_name only selection? `cube.vloc(t0=12)`
* Orthogonal only? Or also independent? `cube.loc_points(lon=[1, 1, 5], lat=[31, 33, 32])`
@@ -185,9 +185,9 @@ cube.interpolate(
## References
. Iris
- * http://scitools.org.uk/iris/docs/v1.9.2/iris/iris.html#iris.Constraint[iris.Constraint]
- * http://scitools.org.uk/iris/docs/v1.9.2/userguide/subsetting_a_cube.html[Subsetting a cube]
-. http://pandas.pydata.org/pandas-docs/stable/indexing.html[pandas indexing]
-. http://xarray.pydata.org/en/stable/indexing.html[xarray indexing]
-. http://legacy.python.org/dev/peps/pep-0472/[PEP 472 - Support for indexing with keyword arguments]
-. http://nbviewer.jupyter.org/gist/rsignell-usgs/13d7ce9d95fddb4983d4cbf98be6c71d[Time slicing NetCDF or OPeNDAP datasets] - Rich Signell's xarray/iris comparison focussing on time handling and performance
+ * https://scitools.org.uk/iris/docs/v1.9.2/iris/iris.html#iris.Constraint[iris.Constraint]
+ * https://scitools.org.uk/iris/docs/v1.9.2/userguide/subsetting_a_cube.html[Subsetting a cube]
+. https://pandas.pydata.org/pandas-docs/stable/indexing.html[pandas indexing]
+. https://xarray.pydata.org/en/stable/indexing.html[xarray indexing]
+. https://legacy.python.org/dev/peps/pep-0472/[PEP 472 - Support for indexing with keyword arguments]
+. https://nbviewer.jupyter.org/gist/rsignell-usgs/13d7ce9d95fddb4983d4cbf98be6c71d[Time slicing NetCDF or OPeNDAP datasets] - Rich Signell's xarray/iris comparison focussing on time handling and performance
diff --git a/docs/src/Makefile b/docs/src/Makefile
new file mode 100644
index 0000000000..8d652878f6
--- /dev/null
+++ b/docs/src/Makefile
@@ -0,0 +1,75 @@
+# Makefile for Sphinx documentation
+#
+
+# You can set these variables from the command line.
+SPHINXOPTS =
+SPHINXBUILD = sphinx-build
+PAPER =
+BUILDDIR = _build
+SRCDIR = .
+
+# See https://www.sphinx-doc.org/en/master/man/sphinx-build.html?highlight=--keep-going#cmdoption-sphinx-build-W
+WARNING_TO_ERROR = -W --keep-going
+
+# Internal variables.
+PAPEROPT_a4 = -D latex_paper_size=a4
+PAPEROPT_letter = -D latex_paper_size=letter
+ALLSPHINXOPTS = -d $(BUILDDIR)/doctrees $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) .
+
+.PHONY: help clean html html-noplot dirhtml singlehtml pickle json htmlhelp qthelp devhelp epub latex latexpdf text man changes linkcheck doctest show
+
+help:
+ @echo "Please use \`make ' where is one of"
+ @echo " help to view this help"
+ @echo " html to make standalone HTML files"
+ @echo " html-noplot to make standalone HTML files, skip gallery"
+ @echo " html-noapi to make standalone HTML files, skip the API"
+ @echo " html-quick to make standalone HTML files, skip the gallery and API"
+ @echo " clean to remove all built files"
+ @echo " doctest to run all doctests embedded in the documentation (if enabled)"
+ @echo " linkcheck to check all external links for integrity"
+ @echo " show to open the built documentation in the default browser"
+ @echo " livehtml to auto build the docs when any file changes are detected."
+ @echo " You need to install sphinx-autobuild first:"
+ @echo " conda install -c conda-forge sphinx-autobuild"
+
+html:
+ $(SPHINXBUILD) $(WARNING_TO_ERROR) -b html $(ALLSPHINXOPTS) $(BUILDDIR)/html
+ @echo
+ @echo "Build finished. The HTML pages are in $(BUILDDIR)/html"
+
+html-noplot:
+ $(SPHINXBUILD) -D plot_gallery=0 -b html $(ALLSPHINXOPTS) $(BUILDDIR)/html
+ @echo
+ @echo "Build finished. The HTML (no gallery docs) pages are in $(BUILDDIR)/html"
+
+html-noapi:
+ export SKIP_API=1; $(SPHINXBUILD) -b html $(ALLSPHINXOPTS) $(BUILDDIR)/html
+ @echo
+ @echo "Build finished. The HTML (no api docs) pages are in $(BUILDDIR)/html"
+
+html-quick:
+ export SKIP_API=1; $(SPHINXBUILD) -D plot_gallery=0 -b html $(ALLSPHINXOPTS) $(BUILDDIR)/html
+ @echo
+ @echo "Build finished. The HTML (no gallery or api docs) pages are in $(BUILDDIR)/html"
+
+clean:
+ -rm -rf $(BUILDDIR)
+ -rm -rf $(SRCDIR)/generated
+
+doctest:
+ $(SPHINXBUILD) -b doctest $(ALLSPHINXOPTS) $(BUILDDIR)/doctest
+ @echo "Testing of doctests in the sources finished, look at the "
+ @echo "results in $(BUILDDIR)/doctest/output.txt."
+
+linkcheck:
+ $(SPHINXBUILD) -b linkcheck $(ALLSPHINXOPTS) $(BUILDDIR)/linkcheck
+ @echo
+ @echo "Link check complete; look for any errors in the above output "
+ @echo "or in $(BUILDDIR)/linkcheck/output.txt."
+
+show:
+ @python -c "import webbrowser; webbrowser.open_new_tab('file://$(shell pwd)/$(BUILDDIR)/html/index.html')"
+
+livehtml:
+ sphinx-autobuild "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) --ignore generated $(O)
\ No newline at end of file
diff --git a/docs/src/_static/README.md b/docs/src/_static/README.md
new file mode 100644
index 0000000000..b9f2877a30
--- /dev/null
+++ b/docs/src/_static/README.md
@@ -0,0 +1,31 @@
+# Iris logos
+
+[](iris-logo-title.svg)
+
+Code for generating the logos is at:
+[SciTools/marketing/iris/logo/generate_logo.py](https://github.com/SciTools/marketing/blob/master/iris/logo/generate_logo.py)
+
+See the docstring of the `generate_logo()` function for more information.
+
+## Why a scripted logo?
+
+SVG logos are ideal for source-controlled projects:
+
+* Low file size, with infinitely scaling quality
+* Universally recognised vector format, editable by many software packages
+* XML-style content = human-readable diff when changes are made
+
+But Iris' logo is difficult to reproduce/edit using an SVG editor alone:
+
+* Includes correctly projected, low resolution coastlines
+* Needs precise alignment of the 'visual centre' of the iris with the centres
+ of the Earth and the image
+
+An SVG image is simply XML format, so can be easily assembled automatically
+with a script, which can also be engineered to address the above problems.
+
+Further advantages of using a script:
+
+* Parameterised text, making it easy to standardise the logo across all Iris
+ packages
+* Can generate an animated GIF/SVG of a rotating Earth
diff --git a/docs/src/_static/icon_api.svg b/docs/src/_static/icon_api.svg
new file mode 100644
index 0000000000..bf2f8d67bb
--- /dev/null
+++ b/docs/src/_static/icon_api.svg
@@ -0,0 +1,155 @@
+
+
+
+
\ No newline at end of file
diff --git a/docs/src/_static/icon_development.svg b/docs/src/_static/icon_development.svg
new file mode 100644
index 0000000000..dbc342688c
--- /dev/null
+++ b/docs/src/_static/icon_development.svg
@@ -0,0 +1,63 @@
+
+
diff --git a/docs/src/_static/icon_instructions.svg b/docs/src/_static/icon_instructions.svg
new file mode 100644
index 0000000000..62b3fc3620
--- /dev/null
+++ b/docs/src/_static/icon_instructions.svg
@@ -0,0 +1,162 @@
+
+
+
+
\ No newline at end of file
diff --git a/docs/src/_static/icon_new_product.svg b/docs/src/_static/icon_new_product.svg
new file mode 100644
index 0000000000..f222e1e066
--- /dev/null
+++ b/docs/src/_static/icon_new_product.svg
@@ -0,0 +1,182 @@
+
+
diff --git a/docs/src/_static/icon_shuttle.svg b/docs/src/_static/icon_shuttle.svg
new file mode 100644
index 0000000000..46ba64d2e0
--- /dev/null
+++ b/docs/src/_static/icon_shuttle.svg
@@ -0,0 +1,71 @@
+
+
diff --git a/docs/src/_static/icon_support.png b/docs/src/_static/icon_support.png
new file mode 100644
index 0000000000..567cdb1b2f
Binary files /dev/null and b/docs/src/_static/icon_support.png differ
diff --git a/docs/src/_static/icon_thumb.png b/docs/src/_static/icon_thumb.png
new file mode 100644
index 0000000000..6a14875e22
Binary files /dev/null and b/docs/src/_static/icon_thumb.png differ
diff --git a/docs/src/_static/iris-logo-title-dark.svg b/docs/src/_static/iris-logo-title-dark.svg
new file mode 100644
index 0000000000..b7358aafec
--- /dev/null
+++ b/docs/src/_static/iris-logo-title-dark.svg
@@ -0,0 +1,107 @@
+
+
\ No newline at end of file
diff --git a/docs/src/_static/iris-logo-title.svg b/docs/src/_static/iris-logo-title.svg
new file mode 100644
index 0000000000..98dd1a73d5
--- /dev/null
+++ b/docs/src/_static/iris-logo-title.svg
@@ -0,0 +1,107 @@
+
+
\ No newline at end of file
diff --git a/docs/src/_static/iris-logo.svg b/docs/src/_static/iris-logo.svg
new file mode 100644
index 0000000000..fe49411b45
--- /dev/null
+++ b/docs/src/_static/iris-logo.svg
@@ -0,0 +1,104 @@
+
+
\ No newline at end of file
diff --git a/docs/src/_static/theme_override.css b/docs/src/_static/theme_override.css
new file mode 100644
index 0000000000..355119f8a5
--- /dev/null
+++ b/docs/src/_static/theme_override.css
@@ -0,0 +1,28 @@
+/* import the standard theme css */
+@import url("styles/theme.css");
+
+/* now we can add custom css.... */
+
+/* Used for very strong warning */
+#slim-red-box-banner {
+ background: #ff0000;
+ box-sizing: border-box;
+ color: #ffffff;
+ font-weight: normal;
+ padding: 0.5em;
+}
+
+#slim-red-box-banner a {
+ color: #ffffff;
+ font-weight: normal;
+ text-decoration: underline;
+}
+
+/* bullet point list with green ticks */
+ul.squarelist {
+ /* https://developer.mozilla.org/en-US/docs/Web/CSS/list-style-type */
+ list-style-type: "\2705";
+ margin-left: 0;
+ text-indent: 1em;
+ padding-left: 5em;
+}
diff --git a/docs/src/_templates/custom_footer.html b/docs/src/_templates/custom_footer.html
new file mode 100644
index 0000000000..f81fcc583e
--- /dev/null
+++ b/docs/src/_templates/custom_footer.html
@@ -0,0 +1 @@
+-{{ version }}-gold?style=flat)
+
+ {% elif rtd_version == 'stable' %}
+
+ -{{ version }}-green?style=flat)
+
+ {% elif rtd_version_type == 'tag' %}
+ {# Covers builds for specific tags, including RC's. #}
+
+ -{{ version }}-red?style=?style=flat)
+
+ {% else %}
+ {# Anything else build by RTD will be the HEAD of an activated branch #}
+
+ 
+
+ {% endif %}
+{%- else %}
+ {# not on rtd #}
+
+ 
+
+{%- endif %}
diff --git a/docs/src/_templates/imagehash.html b/docs/src/_templates/imagehash.html
new file mode 100644
index 0000000000..8b0dac0cce
--- /dev/null
+++ b/docs/src/_templates/imagehash.html
@@ -0,0 +1,15 @@
+{% extends "!layout.html" %}
+
+{% block body %}
+
+`
+* `Twitter `_
+
+Interoperability
+----------------
+
+There's a big choice of Python tools out there! Each one has strengths and
+weaknesses in different areas, so we don't want to force a single choice for your
+whole workflow - we'd much rather make it easy for you to choose the right tool
+for the moment, switching whenever you need. Below are our ongoing efforts at
+smoother interoperability:
+
+.. not using toctree due to combination of child pages and cross-references.
+
+* The :mod:`iris.pandas` module
+* :doc:`iris_xarray`
+* :doc:`phrasebook`
+
+.. toctree::
+ :maxdepth: 1
+ :hidden:
+
+ iris_xarray
+ phrasebook
+
+Plugins
+-------
+
+Iris can be extended with **plugins**! See below for further information:
+
+.. toctree::
+ :maxdepth: 2
+
+ plugins
diff --git a/docs/src/community/iris_xarray.rst b/docs/src/community/iris_xarray.rst
new file mode 100644
index 0000000000..71585d8b9f
--- /dev/null
+++ b/docs/src/community/iris_xarray.rst
@@ -0,0 +1,200 @@
+.. include:: ../common_links.inc
+
+======================
+Iris ❤️ :term:`Xarray`
+======================
+
+There is a lot of overlap between Iris and :term:`Xarray`, but some important
+differences too. Below is a summary of the most important differences, so that
+you can be prepared, and to help you choose the best package for your use case.
+See :doc:`phrasebook` for a broad comparison of terminology.
+
+Overall Experience
+------------------
+
+Iris is the more specialised package, focused on making it as easy
+as possible to work with meteorological and climatological data. Iris
+is built to natively handle many key concepts, such as the CF conventions,
+coordinate systems and bounded coordinates. Iris offers a smaller toolkit of
+operations compared to Xarray, particularly around API for sophisticated
+computation such as array manipulation and multi-processing.
+
+Xarray's more generic data model and community-driven development give it a
+richer range of operations and broader possible uses. Using Xarray
+specifically for meteorology/climatology may require deeper knowledge
+compared to using Iris, and you may prefer to add Xarray plugins
+such as :ref:`cfxarray` to get the best experience. Advanced users can likely
+achieve better performance with Xarray than with Iris.
+
+Conversion
+----------
+There are multiple ways to convert between Iris and Xarray objects.
+
+* Xarray includes the :meth:`~xarray.DataArray.to_iris` and
+ :meth:`~xarray.DataArray.from_iris` methods - detailed in the
+ `Xarray IO notes on Iris`_. Since Iris evolves independently of Xarray, be
+ vigilant for concepts that may be lost during the conversion.
+* Because both packages are closely linked to the :term:`NetCDF Format`, it is
+ feasible to save a NetCDF file using one package then load that file using
+ the other package. This will be lossy in places, as both Iris and Xarray
+ are opinionated on how certain NetCDF concepts relate to their data models.
+* `ncdata `_ is a package which
+ the Iris development team have developed to manage netcdf data, which can act as an
+ improved 'bridge' between Iris and Xarray :
+
+Ncdata can convert Iris cubes to an Xarray dataset, or vice versa, with minimal
+overhead and as lossless as possible.
+
+For example :
+
+.. code-block:: python
+
+ from ncdata.iris_xarray import cubes_from_xarray, cubes_to_xarray
+ cubes = cubes_from_xarray(dataset)
+ xrds = cubes_to_xarray(cubes)
+
+Ncdata avoids the feature limitations previously mentioned regarding Xarray's
+:meth:`~xarray.DataArray.to_iris` and :meth:`~xarray.DataArray.from_iris`,
+because it doesn't replicate any logic of either Xarray or Iris.
+Instead, it uses the netcdf file interfaces of both to exchange data
+"as if" via a netcdf file. So, these conversions *behave* just like exchanging data
+via a file, but are far more efficient because they can transfer data without copying
+arrays or fetching lazy data.
+
+Regridding
+----------
+Iris and Xarray offer a range of regridding methods - both natively and via
+additional packages such as `iris-esmf-regrid`_ and `xESMF`_ - which overlap
+in places
+but tend to cover a different set of use cases (e.g. Iris handles unstructured
+meshes but offers access to fewer ESMF methods). The behaviour of these
+regridders also differs slightly (even between different regridders attached to
+the same package) so the appropriate package to use depends highly on the
+particulars of the use case.
+
+Plotting
+--------
+Xarray and Iris have a large overlap of functionality when creating
+:term:`Matplotlib` plots and both support the plotting of multidimensional
+coordinates. This means the experience is largely similar using either package.
+
+Xarray supports further plotting backends through external packages (e.g. Bokeh through `hvPlot`_)
+and, if a user is already familiar with `pandas`_, the interface should be
+familiar. It also supports some different plot types to Iris, and therefore can
+be used for a wider variety of plots. It also has benefits regarding "out of
+the box", quick customisations to plots. However, if further customisation is
+required, knowledge of matplotlib is still required.
+
+In both cases, :term:`Cartopy` is/can be used. Iris does more work
+automatically for the user here, creating Cartopy
+:class:`~cartopy.mpl.geoaxes.GeoAxes` for latitude and longitude coordinates,
+whereas the user has to do this manually in Xarray.
+
+Statistics
+----------
+Both libraries are quite comparable with generally similar capabilities,
+performance and laziness. Iris offers more specificity in some cases, such as
+some more specific unique functions and masked tolerance in most statistics.
+Xarray seems more approachable however, with some less unique but more
+convenient solutions (these tend to be wrappers to :term:`Dask` functions).
+
+Laziness and Multi-Processing with :term:`Dask`
+-----------------------------------------------
+Iris and Xarray both support lazy data and out-of-core processing through
+utilisation of Dask.
+
+While both Iris and Xarray expose :term:`NumPy` conveniences at the API level
+(e.g. the `ndim()` method), only Xarray exposes Dask conveniences. For example
+:attr:`xarray.DataArray.chunks`, which gives the user direct control
+over the underlying Dask array chunks. The Iris API instead takes control of
+such concepts and user control is only possible by manipulating the underlying
+Dask array directly (accessed via :meth:`iris.cube.Cube.core_data`).
+
+:class:`xarray.DataArray`\ s comply with `NEP-18`_, allowing NumPy arrays to be
+based on them, and they also include the necessary extra members for Dask
+arrays to be based on them too. Neither of these is currently possible with
+Iris :class:`~iris.cube.Cube`\ s, although an ambition for the future.
+
+NetCDF File Control
+-------------------
+(More info: :ref:`netcdf_io`)
+
+Unlike Iris, Xarray generally provides full control of major file structures,
+i.e. dimensions + variables, including their order in the file. It mostly
+respects these in a file input, and can reproduce them on output.
+However, attribute handling is not so complete: like Iris, it interprets and
+modifies some recognised aspects, and can add some extra attributes not in the
+input.
+
+Whereas Iris is primarily designed to handle netCDF data encoded according to
+`CF Conventions `_.
+
+In some particular aspects, CF data is not loaded well (or at all), and in many cases
+output is not fully CF compliant (as-per `the cf checker `_,
+ however date units may not always be saved correctly.
+* CF-style coordinate bounds variables are not fully understood. The CF approach
+ where bounds variables do not usually define their units or standard_names can cause
+ problems. Certain files containing bounds variables with more than 2 bounds (e.g.
+ unstructured data) may not load at all.
+* missing points are always represented as NaNs, as-per Pandas usage.
+ (See :ref:`xarray_missing_data` ).
+ This means that fill values are not preserved, and that masked integer data is
+ converted to floats.
+ The netCDF default fill-values are not supported, so that variables with no
+ "_FillValue" attribute will have missing points equal to the fill-value
+ in place of NaNs. By default, output variables generally have ``_FillValue = NaN``.
+
+Ultimately, however, nearly everything wanted in a particular desired result file
+**can** be achieved in Xarray, via provided override mechanisms (`loading keywords`_
+and the '`encoding`_' dictionaries).
+
+.. _xarray_missing_data:
+
+Missing Data
+------------
+Xarray uses :data:`numpy.nan` to represent missing values and this will support
+many simple use cases assuming the data are floats. Iris enables more
+sophisticated missing data handling by representing missing values as masks
+(:class:`numpy.ma.MaskedArray` for real data and :class:`dask.array.Array`
+for lazy data) which allows data to be any data type and to include either/both
+a mask and :data:`~numpy.nan`\ s.
+
+.. _cfxarray:
+
+`cf-xarray`_
+-------------
+Iris has a data model entirely based on :term:`CF Conventions`. Xarray has a
+data model based on :term:`NetCDF Format` with cf-xarray acting as translation
+into CF. Xarray/cf-xarray methods can be
+called and data accessed with CF like arguments (e.g. axis, standard name) and
+there are some CF specific utilities (similar
+to Iris utilities). Iris tends to cover more of and be stricter about CF.
+
+
+.. seealso::
+
+ * `Xarray IO notes on Iris`_
+ * `Xarray notes on other NetCDF libraries`_
+
+.. _Xarray IO notes on Iris: https://docs.xarray.dev/en/stable/user-guide/io.html#iris
+.. _Xarray notes on other NetCDF libraries: https://docs.xarray.dev/en/stable/getting-started-guide/faq.html#what-other-netcdf-related-python-libraries-should-i-know-about
+.. _loading keywords: https://docs.xarray.dev/en/stable/generated/xarray.open_dataset.html#xarray.open_dataset
+.. _encoding: https://docs.xarray.dev/en/stable/user-guide/io.html#writing-encoded-data
+.. _xESMF: https://github.com/pangeo-data/xESMF/
+.. _seaborn: https://seaborn.pydata.org/
+.. _hvPlot: https://hvplot.holoviz.org/
+.. _pandas: https://pandas.pydata.org/
+.. _NEP-18: https://numpy.org/neps/nep-0018-array-function-protocol.html
+.. _cf-xarray: https://github.com/xarray-contrib/cf-xarray
+.. _iris#4994: https://github.com/SciTools/iris/issues/4994
diff --git a/docs/src/community/phrasebook.rst b/docs/src/community/phrasebook.rst
new file mode 100644
index 0000000000..bcd91cca83
--- /dev/null
+++ b/docs/src/community/phrasebook.rst
@@ -0,0 +1,66 @@
+.. include:: ../common_links.inc
+
+.. _phrasebook:
+
+Package Phrasebook
+===================
+
+There are a number of similar packages to Iris, and a lot of these have their own
+terminology for similar things. Whether you're coming or going, we hope this might
+be a helpful guide to these differences!
+Definitions for each can be found in :ref:`glossary`. See also
+`Xarray terminology `_.
+
+.. list-table:: Phrasebook
+ :widths: 25 25 25 50
+ :header-rows: 1
+
+ * - Iris
+ - Xarray
+ - Example
+ - Notes
+ * - Non-Lazy
+ - Eager
+ -
+ - Used to relate to functions, rather than the data.
+ * - Cube
+ - DataArray
+ -
+ -
+ * - CubeList
+ - Dataset
+ -
+ - Though similar, a CubeList is a simpler object, and is
+ not a perfect comparison to a Dataset
+ * - Merge/ Concatenate
+ - Concatenate
+ - `Xarray concatenate `_
+ - Xarray's concatenate has the capability to largely do what both
+ Iris merge and Iris concatenate do. However, this is not a perfect comparison,
+ please see the link for more information.
+ * -
+ - Merge
+ - `Xarray merge `_
+ - Xarray's Merge function doesn't map neatly map to any Iris feature.
+ Please see the link for more information.
+ * - Scalar Coordinate
+ -
+ -
+ - Iris makes a distinction between scalar coordinates and non-scalar coordinates,
+ whereas xarray documentation makes a distinction between scalar and non-scalar *data*.
+ It is possible to make coordinates with scalar data in both Iris and xarray
+ but only Iris will label such coordinates.
+ * - AuxCoord
+ - Non-Dimensional Coordinate
+ -
+ - Coordinates in Iris and xarray are categorised using different rules,
+ and so are not a one-to-one match.
+ * - DimCoord
+ - Dimension Coordinate
+ -
+ - Coordinates in Iris and xarray are categorised using different rules,
+ and so are not a one-to-one match.
+
+----
+
+`To top `_
\ No newline at end of file
diff --git a/docs/src/community/plugins.rst b/docs/src/community/plugins.rst
new file mode 100644
index 0000000000..0d79d64623
--- /dev/null
+++ b/docs/src/community/plugins.rst
@@ -0,0 +1,68 @@
+.. _namespace package: https://packaging.python.org/en/latest/guides/packaging-namespace-packages/
+
+.. _community_plugins:
+
+Plugins
+=======
+
+Iris supports **plugins** under the ``iris.plugins`` `namespace package`_.
+This allows packages that extend Iris' functionality to be developed and
+maintained independently, while still being installed into ``iris.plugins``
+instead of a separate package. For example, a plugin may provide loaders or
+savers for additional file formats, or alternative visualisation methods.
+
+
+Using plugins
+-------------
+
+Once a plugin is installed, it can be used either via the
+:func:`iris.use_plugin` function, or by importing it directly:
+
+.. code-block:: python
+
+ import iris
+
+ iris.use_plugin("my_plugin")
+ # OR
+ import iris.plugins.my_plugin
+
+
+Creating plugins
+----------------
+
+The choice of a `namespace package`_ makes writing a plugin relatively
+straightforward: it simply needs to appear as a folder within ``iris/plugins``,
+then can be distributed in the same way as any other package. An example
+repository layout:
+
+.. code-block:: text
+
+ + lib
+ + iris
+ + plugins
+ + my_plugin
+ - __init__.py
+ - (more code...)
+ - README.md
+ - pyproject.toml
+ - setup.cfg
+ - (other project files...)
+
+In particular, note that there must **not** be any ``__init__.py`` files at
+higher levels than the plugin itself.
+
+The package name - how it is referred to by PyPI/conda, specified by
+``metadata.name`` in ``setup.cfg`` - is recommended to include both "iris" and
+the plugin name. Continuing this example, its ``setup.cfg`` should include, at
+minimum:
+
+.. code-block:: ini
+
+ [metadata]
+ name = iris-my-plugin
+
+ [options]
+ packages = find_namespace:
+
+ [options.packages.find]
+ where = lib
diff --git a/docs/src/conf.py b/docs/src/conf.py
new file mode 100644
index 0000000000..70b1063585
--- /dev/null
+++ b/docs/src/conf.py
@@ -0,0 +1,462 @@
+# Copyright Iris contributors
+#
+# This file is part of Iris and is released under the BSD license.
+# See LICENSE in the root of the repository for full licensing details.
+
+# -*- coding: utf-8 -*-
+#
+# Iris documentation build configuration file, created by
+# sphinx-quickstart on Tue May 25 13:26:23 2010.
+#
+# This file is execfile()d with the current directory set to its containing dir.
+#
+# Note that not all possible configuration values are present in this
+# autogenerated file.
+#
+# All configuration values have a default; values that are commented out
+# serve to show the default.
+# ----------------------------------------------------------------------------
+
+"""Config for sphinx."""
+
+import datetime
+from importlib.metadata import version as get_version
+from inspect import getsource
+import ntpath
+import os
+from pathlib import Path
+import re
+from subprocess import run
+import sys
+from tempfile import gettempdir
+from urllib.parse import quote
+import warnings
+
+
+# function to write useful output to stdout, prefixing the source.
+def autolog(message):
+ print("[{}] {}".format(ntpath.basename(__file__), message))
+
+
+# -- Check for dev make options to build quicker
+skip_api = os.environ.get("SKIP_API")
+
+# -- Are we running on the readthedocs server, if so do some setup -----------
+on_rtd = os.environ.get("READTHEDOCS") == "True"
+
+# This is the rtd reference to the version, such as: latest, stable, v3.0.1 etc
+rtd_version = os.environ.get("READTHEDOCS_VERSION")
+if rtd_version is not None:
+ # Make rtd_version safe for use in shields.io badges.
+ rtd_version = rtd_version.replace("_", "__")
+ rtd_version = rtd_version.replace("-", "--")
+ rtd_version = quote(rtd_version)
+
+# branch, tag, external (for pull request builds), or unknown.
+rtd_version_type = os.environ.get("READTHEDOCS_VERSION_TYPE")
+
+# For local testing purposes we can force being on RTD and the version
+# on_rtd = True # useful for testing
+# rtd_version = "latest" # useful for testing
+# rtd_version = "stable" # useful for testing
+# rtd_version_type = "tag" # useful for testing
+# rtd_version = "my_branch" # useful for testing
+
+if on_rtd:
+ autolog("Build running on READTHEDOCS server")
+
+ # list all the READTHEDOCS environment variables that may be of use
+ autolog("Listing all environment variables on the READTHEDOCS server...")
+
+ for item, value in os.environ.items():
+ autolog("[READTHEDOCS] {} = {}".format(item, value))
+
+# -- Path setup --------------------------------------------------------------
+
+# If extensions (or modules to document with autodoc) are in another directory,
+# add these directories to sys.path here. If the directory is relative to the
+# documentation root, use os.path.abspath to make it absolute, like shown here.
+
+# custom sphinx extensions
+sys.path.append(os.path.abspath("sphinxext"))
+
+# add some sample files from the developers guide..
+sys.path.append(os.path.abspath(os.path.join("developers_guide")))
+
+# why isn't the iris path added to it is discoverable too? We dont need to,
+# the sphinext to generate the api rst knows where the source is. If it
+# is added then the travis build will likely fail.
+
+# -- Project information -----------------------------------------------------
+
+project = "Iris"
+
+# define the copyright information for latex builds. Note, for html builds,
+# the copyright exists directly inside "_templates/layout.html"
+copyright_years = f"2010 - {datetime.datetime.now().year}"
+copyright = f"{copyright_years}, Iris Contributors"
+author = "Iris Developers"
+
+# The version info for the project you're documenting, acts as replacement for
+# |version|, also used in various other places throughout the built documents.
+version = get_version("scitools-iris")
+release = version
+autolog(f"Iris Version = {version}")
+autolog(f"Iris Release = {release}")
+
+# -- General configuration ---------------------------------------------------
+
+# Create a variable that can be inserted in the rst "|copyright_years|".
+# You can add more variables here if needed.
+
+build_python_version = ".".join([str(i) for i in sys.version_info[:3]])
+
+
+def _dotv(version):
+ result = version
+ match = re.match(r"^py(\d+)$", version)
+ if match:
+ digits = match.group(1)
+ if len(digits) > 1:
+ result = f"{digits[0]}.{digits[1:]}"
+ return result
+
+
+# Automate the discovery of the python versions tested with CI.
+python_support = sorted(
+ [fname.stem for fname in Path(".").glob("../../requirements/py*.yml")]
+)
+
+if not python_support:
+ python_support = "unknown Python versions"
+elif len(python_support) == 1:
+ python_support = f"Python {_dotv(python_support[0])}"
+else:
+ rest = ", ".join([_dotv(v) for v in python_support[:-1]])
+ last = _dotv(python_support[-1])
+ python_support = f"Python {rest} and {last}"
+
+rst_epilog = f"""
+.. |copyright_years| replace:: {copyright_years}
+.. |python_version| replace:: {build_python_version}
+.. |python_support| replace:: {python_support}
+.. |iris_version| replace:: v{version}
+.. |build_date| replace:: ({datetime.datetime.now().strftime('%d %b %Y')})
+"""
+
+# Add any Sphinx extension module names here, as strings. They can be
+# extensions coming with Sphinx (named "sphinx.ext.*") or your custom
+# ones.
+extensions = [
+ "sphinx.ext.todo",
+ "sphinx.ext.duration",
+ "sphinx.ext.coverage",
+ "sphinx.ext.viewcode",
+ "sphinx.ext.autosummary",
+ "sphinx.ext.doctest",
+ "sphinx.ext.extlinks",
+ "sphinx.ext.autodoc",
+ "sphinx.ext.intersphinx",
+ "sphinx_copybutton",
+ "sphinx.ext.napoleon",
+ "sphinx_design",
+ "sphinx_gallery.gen_gallery",
+ "matplotlib.sphinxext.mathmpl",
+ "matplotlib.sphinxext.plot_directive",
+]
+
+if skip_api == "1":
+ autolog("Skipping the API docs generation (SKIP_API=1)")
+else:
+ extensions.extend(["sphinxcontrib.apidoc"])
+ extensions.extend(["api_rst_formatting"])
+
+# -- Napoleon extension -------------------------------------------------------
+# See https://sphinxcontrib-napoleon.readthedocs.io/en/latest/sphinxcontrib.napoleon.html
+napoleon_google_docstring = True
+napoleon_numpy_docstring = True
+napoleon_include_init_with_doc = False
+napoleon_include_private_with_doc = False
+napoleon_include_special_with_doc = True # includes dunders in api doc
+napoleon_use_admonition_for_examples = False
+napoleon_use_admonition_for_notes = False
+napoleon_use_admonition_for_references = False
+napoleon_use_ivar = False
+napoleon_use_param = True
+napoleon_use_rtype = True
+napoleon_use_keyword = True
+napoleon_custom_sections = None
+
+# -- copybutton extension -----------------------------------------------------
+# See https://sphinx-copybutton.readthedocs.io/en/latest/
+copybutton_prompt_text = r">>> |\.\.\. "
+copybutton_prompt_is_regexp = True
+copybutton_line_continuation_character = "\\"
+
+# sphinx.ext.todo configuration -----------------------------------------------
+# See https://www.sphinx-doc.org/en/master/usage/extensions/todo.html
+todo_include_todos = False
+todo_emit_warnings = False
+
+# sphinx.ext.autodoc configuration --------------------------------------------
+# https://www.sphinx-doc.org/en/master/usage/extensions/autodoc.html#confval-autodoc_default_options
+autodoc_default_options = {
+ "members": True,
+ "member-order": "alphabetical",
+ "undoc-members": True,
+ "private-members": False,
+ "special-members": False,
+ "inherited-members": True,
+ "show-inheritance": True,
+}
+
+# https://www.sphinx-doc.org/en/master/usage/extensions/autodoc.html#confval-autodoc_typehints
+autodoc_typehints = "description"
+autosummary_generate = True
+autosummary_imported_members = True
+autopackage_name = ["iris"]
+autoclass_content = "both"
+modindex_common_prefix = ["iris"]
+
+# -- apidoc extension ---------------------------------------------------------
+# See https://github.com/sphinx-contrib/apidoc
+source_code_root = (Path(__file__).parents[2]).absolute()
+module_dir = source_code_root / "lib"
+apidoc_module_dir = str(module_dir)
+apidoc_output_dir = str(Path(__file__).parent / "generated/api")
+apidoc_toc_file = False
+
+apidoc_excluded_paths = [
+ str(module_dir / "iris/tests"),
+ str(module_dir / "iris/experimental/raster.*"), # gdal conflicts
+]
+
+apidoc_module_first = True
+apidoc_separate_modules = True
+apidoc_extra_args = []
+
+autolog(f"[sphinx-apidoc] source_code_root = {source_code_root}")
+autolog(f"[sphinx-apidoc] apidoc_excluded_paths = {apidoc_excluded_paths}")
+autolog(f"[sphinx-apidoc] apidoc_output_dir = {apidoc_output_dir}")
+
+# Add any paths that contain templates here, relative to this directory.
+templates_path = ["_templates"]
+
+# -- intersphinx extension ----------------------------------------------------
+# See https://www.sphinx-doc.org/en/master/usage/extensions/intersphinx.html
+intersphinx_mapping = {
+ "cartopy": ("https://scitools.org.uk/cartopy/docs/latest/", None),
+ "cf_units": ("https://cf-units.readthedocs.io/en/stable/", None),
+ "cftime": ("https://unidata.github.io/cftime/", None),
+ "dask": ("https://docs.dask.org/en/stable/", None),
+ "geovista": ("https://geovista.readthedocs.io/en/latest/", None),
+ "iris-esmf-regrid": ("https://iris-esmf-regrid.readthedocs.io/en/stable/", None),
+ "matplotlib": ("https://matplotlib.org/stable/", None),
+ "numpy": ("https://numpy.org/doc/stable/", None),
+ "pandas": ("https://pandas.pydata.org/docs/", None),
+ "python": ("https://docs.python.org/3/", None),
+ "pyvista": ("https://docs.pyvista.org/", None),
+ "scipy": ("https://docs.scipy.org/doc/scipy/", None),
+}
+
+# The name of the Pygments (syntax highlighting) style to use.
+pygments_style = "sphinx"
+
+# -- plot_directive extension -------------------------------------------------
+# See https://matplotlib.org/stable/api/sphinxext_plot_directive_api.html#options
+plot_formats = [
+ ("png", 100),
+]
+
+# -- Extlinks extension -------------------------------------------------------
+# See https://www.sphinx-doc.org/en/master/usage/extensions/extlinks.html
+
+extlinks = {
+ "issue": ("https://github.com/SciTools/iris/issues/%s", "Issue #%s"),
+ "pull": ("https://github.com/SciTools/iris/pull/%s", "PR #%s"),
+ "discussion": (
+ "https://github.com/SciTools/iris/discussions/%s",
+ "Discussion #%s",
+ ),
+}
+
+# -- Doctest ("make doctest")--------------------------------------------------
+
+doctest_global_setup = "import iris"
+
+# -- Options for HTML output --------------------------------------------------
+
+# The theme to use for HTML and HTML Help pages. See the documentation for
+# a list of builtin themes.
+#
+html_favicon = "_static/iris-logo.svg"
+html_theme = "pydata_sphinx_theme"
+
+# See https://pydata-sphinx-theme.readthedocs.io/en/latest/user_guide/configuring.html#configure-the-search-bar-position
+html_sidebars = {
+ "**": [
+ "custom_sidebar_logo_version",
+ "search-field",
+ "sidebar-nav-bs",
+ "sidebar-ethical-ads",
+ ]
+}
+
+# See https://pydata-sphinx-theme.readthedocs.io/en/latest/user_guide/configuring.html
+html_theme_options = {
+ "footer_start": ["copyright", "sphinx-version"],
+ "footer_end": ["custom_footer"],
+ "navigation_depth": 3,
+ "navigation_with_keys": False,
+ "show_toc_level": 2,
+ "show_prev_next": True,
+ "navbar_align": "content",
+ # removes the search box from the top bar
+ "navbar_persistent": [],
+ # TODO: review if 6 links is too crowded.
+ "header_links_before_dropdown": 6,
+ "github_url": "https://github.com/SciTools/iris",
+ "twitter_url": "https://twitter.com/scitools_iris",
+ # icons available: https://fontawesome.com/v5.15/icons?d=gallery&m=free
+ "icon_links": [
+ {
+ "name": "GitHub Discussions",
+ "url": "https://github.com/SciTools/iris/discussions",
+ "icon": "far fa-comments",
+ },
+ {
+ "name": "PyPI",
+ "url": "https://pypi.org/project/scitools-iris/",
+ "icon": "fas fa-box",
+ },
+ {
+ "name": "Conda",
+ "url": "https://anaconda.org/conda-forge/iris",
+ "icon": "fas fa-boxes",
+ },
+ ],
+ "use_edit_page_button": True,
+ # Omit `theme-switcher` from navbar_end below to disable it
+ # Info: https://pydata-sphinx-theme.readthedocs.io/en/stable/user_guide/light-dark.html#configure-default-theme-mode
+ # "navbar_end": ["navbar-icon-links"],
+ # https://pydata-sphinx-theme.readthedocs.io/en/v0.11.0/user_guide/branding.html#different-logos-for-light-and-dark-mode
+ "logo": {
+ "image_light": "_static/iris-logo-title.svg",
+ "image_dark": "_static/iris-logo-title-dark.svg",
+ },
+}
+
+# if we are building via Read The Docs and it is the latest (not stable)
+if on_rtd and rtd_version == "latest":
+ html_theme_options["announcement"] = f"""
+ You are viewing the latest unreleased documentation
+ {version}. You can switch to a
+ stable
+ version."""
+
+rev_parse = run(["git", "rev-parse", "--short", "HEAD"], capture_output=True)
+commit_sha = rev_parse.stdout.decode().strip()
+
+html_context = {
+ # pydata_theme
+ "github_repo": "iris",
+ "github_user": "scitools",
+ "github_version": "main",
+ "doc_path": "docs/src",
+ # default theme. Also disabled the button in the html_theme_options.
+ # Info: https://pydata-sphinx-theme.readthedocs.io/en/stable/user_guide/light-dark.html#configure-default-theme-mode
+ "default_mode": "auto",
+ # custom
+ "on_rtd": on_rtd,
+ "rtd_version": rtd_version,
+ "rtd_version_type": rtd_version_type,
+ "version": version,
+ "copyright_years": copyright_years,
+ "python_version": build_python_version,
+ "commit_sha": commit_sha,
+}
+
+# Add any paths that contain custom static files (such as style sheets) here,
+# relative to this directory. They are copied after the builtin static files,
+# so a file named "default.css" will overwrite the builtin "default.css".
+html_static_path = ["_static"]
+html_style = "theme_override.css"
+
+# url link checker. Some links work but report as broken, lets ignore them.
+# See https://www.sphinx-doc.org/en/1.2/config.html#options-for-the-linkcheck-builder
+linkcheck_ignore = [
+ "https://catalogue.ceda.ac.uk/uuid/82adec1f896af6169112d09cc1174499",
+ "https://cfconventions.org",
+ "https://code.google.com/p/msysgit/downloads/list",
+ "https://effbot.org",
+ "https://help.github.com",
+ "https://docs.github.com",
+ "https://github.com",
+ "https://www.personal.psu.edu/cab38/ColorBrewer/ColorBrewer_updates.html",
+ "https://scitools.github.com/cartopy",
+ "https://www.wmo.int/pages/prog/www/DPFS/documents/485_Vol_I_en_colour.pdf",
+ "https://software.ac.uk/how-cite-software",
+ "https://www.esrl.noaa.gov/psd/data/gridded/conventions/cdc_netcdf_standard.shtml",
+ "https://www.nationalarchives.gov.uk/doc/open-government-licence",
+ "https://www.metoffice.gov.uk/",
+ "https://biggus.readthedocs.io/",
+ "https://stickler-ci.com/",
+ "https://twitter.com/scitools_iris",
+ "https://stackoverflow.com/questions/tagged/python-iris",
+ "https://www.flaticon.com/",
+]
+
+# list of sources to exclude from the build.
+exclude_patterns = []
+
+# -- sphinx-gallery config ----------------------------------------------------
+# See https://sphinx-gallery.github.io/stable/configuration.html
+
+
+def reset_modules(gallery_conf, fname):
+ """Force re-registering of nc-time-axis with matplotlib for each example.
+
+ Required for sphinx-gallery>=0.11.0.
+ """
+ from sys import modules
+
+ _ = modules.pop("nc_time_axis", None)
+
+
+# https://sphinx-gallery.github.io/dev/configuration.html#importing-callables
+reset_modules_dir = Path(gettempdir()) / reset_modules.__name__
+reset_modules_dir.mkdir(exist_ok=True)
+(reset_modules_dir / f"{reset_modules.__name__}.py").write_text(
+ getsource(reset_modules)
+)
+sys.path.insert(0, str(reset_modules_dir))
+
+
+sphinx_gallery_conf = {
+ # path to your example scripts
+ "examples_dirs": ["../gallery_code"],
+ # path to where to save gallery generated output
+ "gallery_dirs": ["generated/gallery"],
+ # filename pattern for the files in the gallery
+ "filename_pattern": "/plot_",
+ # filename pattern to ignore in the gallery
+ "ignore_pattern": r"__init__\.py",
+ # force gallery building, unless overridden (see src/Makefile)
+ "plot_gallery": "'True'",
+ "reset_modules": f"{reset_modules.__name__}.{reset_modules.__name__}",
+}
+
+# -----------------------------------------------------------------------------
+# Remove warnings
+warnings.filterwarnings("ignore")
+
+# -- numfig options (built-in) ------------------------------------------------
+# Enable numfig.
+numfig = True
+
+numfig_format = {
+ "code-block": "Example %s",
+ "figure": "Figure %s",
+ "section": "Section %s",
+ "table": "Table %s",
+}
diff --git a/docs/src/copyright.rst b/docs/src/copyright.rst
new file mode 100644
index 0000000000..b0d68cfe8c
--- /dev/null
+++ b/docs/src/copyright.rst
@@ -0,0 +1,36 @@
+
+Iris Copyright, Licensing and Contributors
+==========================================
+
+Iris Code
+---------
+
+All Iris source code, unless explicitly stated, is ``Copyright Iris
+contributors`` and is licensed under the **BSD-3 License**.
+You should find all source files with the following header:
+
+.. admonition:: Code License
+
+ Copyright Iris contributors
+
+ This file is part of Iris and is released under the BSD license.
+ See LICENSE in the root of the repository for full licensing details.
+
+
+Iris Documentation and Examples
+-------------------------------
+
+All documentation, examples and sample data found on this website and in source repository
+are licensed under the UK's Open Government Licence:
+
+.. admonition:: Documentation, example and data license
+
+ (C) British Crown Copyright |copyright_years|
+
+ You may use and reuse the information featured on this website (not including logos) free of
+ charge in any format or medium, under the terms of the
+ `Open Government Licence `_.
+ We encourage users to establish hypertext links to this website.
+
+ Any email enquiries regarding the use and reuse of this information resource should be
+ sent to: psi@nationalarchives.gsi.gov.uk.
diff --git a/docs/src/developers_guide/assets/developer-settings-github-apps.png b/docs/src/developers_guide/assets/developer-settings-github-apps.png
new file mode 100644
index 0000000000..a63994d087
Binary files /dev/null and b/docs/src/developers_guide/assets/developer-settings-github-apps.png differ
diff --git a/docs/src/developers_guide/assets/download-pem.png b/docs/src/developers_guide/assets/download-pem.png
new file mode 100644
index 0000000000..cbceb1304d
Binary files /dev/null and b/docs/src/developers_guide/assets/download-pem.png differ
diff --git a/docs/src/developers_guide/assets/generate-key.png b/docs/src/developers_guide/assets/generate-key.png
new file mode 100644
index 0000000000..ac894dc71b
Binary files /dev/null and b/docs/src/developers_guide/assets/generate-key.png differ
diff --git a/docs/src/developers_guide/assets/gha-token-example.png b/docs/src/developers_guide/assets/gha-token-example.png
new file mode 100644
index 0000000000..cba1cf6935
Binary files /dev/null and b/docs/src/developers_guide/assets/gha-token-example.png differ
diff --git a/docs/src/developers_guide/assets/install-app.png b/docs/src/developers_guide/assets/install-app.png
new file mode 100644
index 0000000000..31259de588
Binary files /dev/null and b/docs/src/developers_guide/assets/install-app.png differ
diff --git a/docs/src/developers_guide/assets/install-iris-actions.png b/docs/src/developers_guide/assets/install-iris-actions.png
new file mode 100644
index 0000000000..db16dee55b
Binary files /dev/null and b/docs/src/developers_guide/assets/install-iris-actions.png differ
diff --git a/docs/src/developers_guide/assets/installed-app.png b/docs/src/developers_guide/assets/installed-app.png
new file mode 100644
index 0000000000..ab87032393
Binary files /dev/null and b/docs/src/developers_guide/assets/installed-app.png differ
diff --git a/docs/src/developers_guide/assets/iris-actions-secret.png b/docs/src/developers_guide/assets/iris-actions-secret.png
new file mode 100644
index 0000000000..f32456d0f2
Binary files /dev/null and b/docs/src/developers_guide/assets/iris-actions-secret.png differ
diff --git a/docs/src/developers_guide/assets/iris-github-apps.png b/docs/src/developers_guide/assets/iris-github-apps.png
new file mode 100644
index 0000000000..50753532b7
Binary files /dev/null and b/docs/src/developers_guide/assets/iris-github-apps.png differ
diff --git a/docs/src/developers_guide/assets/iris-secrets-created.png b/docs/src/developers_guide/assets/iris-secrets-created.png
new file mode 100644
index 0000000000..19b0ba11dc
Binary files /dev/null and b/docs/src/developers_guide/assets/iris-secrets-created.png differ
diff --git a/docs/src/developers_guide/assets/iris-security-actions.png b/docs/src/developers_guide/assets/iris-security-actions.png
new file mode 100644
index 0000000000..7cbe3a7dc2
Binary files /dev/null and b/docs/src/developers_guide/assets/iris-security-actions.png differ
diff --git a/docs/src/developers_guide/assets/iris-settings.png b/docs/src/developers_guide/assets/iris-settings.png
new file mode 100644
index 0000000000..70714235c2
Binary files /dev/null and b/docs/src/developers_guide/assets/iris-settings.png differ
diff --git a/docs/src/developers_guide/assets/org-perms-members.png b/docs/src/developers_guide/assets/org-perms-members.png
new file mode 100644
index 0000000000..99fd8985e2
Binary files /dev/null and b/docs/src/developers_guide/assets/org-perms-members.png differ
diff --git a/docs/src/developers_guide/assets/repo-perms-contents.png b/docs/src/developers_guide/assets/repo-perms-contents.png
new file mode 100644
index 0000000000..4c325c334d
Binary files /dev/null and b/docs/src/developers_guide/assets/repo-perms-contents.png differ
diff --git a/docs/src/developers_guide/assets/repo-perms-pull-requests.png b/docs/src/developers_guide/assets/repo-perms-pull-requests.png
new file mode 100644
index 0000000000..812f5ef951
Binary files /dev/null and b/docs/src/developers_guide/assets/repo-perms-pull-requests.png differ
diff --git a/docs/src/developers_guide/assets/scitools-settings.png b/docs/src/developers_guide/assets/scitools-settings.png
new file mode 100644
index 0000000000..8d7e728ab5
Binary files /dev/null and b/docs/src/developers_guide/assets/scitools-settings.png differ
diff --git a/docs/src/developers_guide/assets/user-perms.png b/docs/src/developers_guide/assets/user-perms.png
new file mode 100644
index 0000000000..607c7dcdb6
Binary files /dev/null and b/docs/src/developers_guide/assets/user-perms.png differ
diff --git a/docs/src/developers_guide/assets/webhook-active.png b/docs/src/developers_guide/assets/webhook-active.png
new file mode 100644
index 0000000000..538362f335
Binary files /dev/null and b/docs/src/developers_guide/assets/webhook-active.png differ
diff --git a/docs/src/developers_guide/asv_example_images/commits.png b/docs/src/developers_guide/asv_example_images/commits.png
new file mode 100644
index 0000000000..4e0d695322
Binary files /dev/null and b/docs/src/developers_guide/asv_example_images/commits.png differ
diff --git a/docs/src/developers_guide/asv_example_images/comparison.png b/docs/src/developers_guide/asv_example_images/comparison.png
new file mode 100644
index 0000000000..e146d30696
Binary files /dev/null and b/docs/src/developers_guide/asv_example_images/comparison.png differ
diff --git a/docs/src/developers_guide/asv_example_images/scalability.png b/docs/src/developers_guide/asv_example_images/scalability.png
new file mode 100644
index 0000000000..260c3ef536
Binary files /dev/null and b/docs/src/developers_guide/asv_example_images/scalability.png differ
diff --git a/docs/src/developers_guide/ci_checks.png b/docs/src/developers_guide/ci_checks.png
new file mode 100644
index 0000000000..54ab672b3c
Binary files /dev/null and b/docs/src/developers_guide/ci_checks.png differ
diff --git a/docs/src/developers_guide/contributing_benchmarks.rst b/docs/src/developers_guide/contributing_benchmarks.rst
new file mode 100644
index 0000000000..ccb9a50e39
--- /dev/null
+++ b/docs/src/developers_guide/contributing_benchmarks.rst
@@ -0,0 +1,64 @@
+.. include:: ../common_links.inc
+
+.. _contributing.benchmarks:
+
+Benchmarking
+============
+Iris includes architecture for benchmarking performance and other metrics of
+interest. This is done using the `Airspeed Velocity`_ (ASV) package.
+
+
+.. note:: Full detail on the setup and how to run or write benchmarks is in
+ `benchmarks/README.md`_ in the Iris repository.
+
+Continuous Integration
+----------------------
+The primary purpose of `Airspeed Velocity`_, and Iris' specific benchmarking
+setup, is to monitor for performance changes using statistical comparison
+between commits, and this forms part of Iris' continuous integration.
+
+Accurately assessing performance takes longer than functionality pass/fail
+tests, so the benchmark suite is not automatically run against open pull
+requests, instead it is **run overnight against each the commits of the
+previous day** to check if any commit has introduced performance shifts.
+Detected shifts are reported in a new Iris GitHub issue.
+
+.. _on_demand_pr_benchmark:
+
+If a pull request author/reviewer suspects their changes may cause performance
+shifts, they can manually order their pull request to be benchmarked by adding
+the ``benchmark_this`` label to the PR. Read more in `benchmarks/README.md`_.
+
+Other Uses
+----------
+Even when not statistically comparing commits, ASV's accurate execution time
+results - recorded using a sophisticated system of repeats - have other
+applications.
+
+* Absolute numbers can be interpreted providing they are recorded on a
+ dedicated resource.
+* Results for a series of commits can be visualised for an intuitive
+ understanding of when and why changes occurred.
+
+ .. image:: asv_example_images/commits.png
+ :width: 300
+
+* Parameterised benchmarks make it easy to visualise:
+
+ * Comparisons
+
+ .. image:: asv_example_images/comparison.png
+ :width: 300
+
+ * Scalability
+
+ .. image:: asv_example_images/scalability.png
+ :width: 300
+
+This also isn't limited to execution times. ASV can also measure memory demand,
+and even arbitrary numbers (e.g. file size, regridding accuracy), although
+without the repetition logic that execution timing has.
+
+
+.. _Airspeed Velocity: https://github.com/airspeed-velocity/asv
+.. _benchmarks/README.md: https://github.com/SciTools/iris/blob/main/benchmarks/README.md
diff --git a/docs/src/developers_guide/contributing_changes.rst b/docs/src/developers_guide/contributing_changes.rst
new file mode 100644
index 0000000000..48357874a7
--- /dev/null
+++ b/docs/src/developers_guide/contributing_changes.rst
@@ -0,0 +1,11 @@
+
+.. _contributing.changes:
+
+Contributing Your Changes
+=========================
+
+.. toctree::
+ :maxdepth: 3
+
+ documenting/whats_new_contributions
+ contributing_pull_request_checklist
diff --git a/docs/src/developers_guide/contributing_ci_tests.rst b/docs/src/developers_guide/contributing_ci_tests.rst
new file mode 100644
index 0000000000..542178c2ff
--- /dev/null
+++ b/docs/src/developers_guide/contributing_ci_tests.rst
@@ -0,0 +1,151 @@
+.. include:: ../common_links.inc
+
+.. _developer_testing_ci:
+
+Continuous Integration (CI) Testing
+===================================
+
+.. note:: Iris is currently supported and tested against |python_support|
+ running on Linux. We do not currently actively test on other
+ platforms such as Windows or macOS.
+
+The `Iris`_ GitHub repository is configured to run checks against all its
+branches automatically whenever a pull-request is created, updated or merged.
+The checks performed are:
+
+* :ref:`testing_gha`
+* :ref:`testing_cla`
+* :ref:`pre_commit_ci`
+
+
+.. _testing_gha:
+
+GitHub Actions
+**************
+
+Iris unit and integration tests are an essential mechanism to ensure
+that the Iris code base is working as expected. :ref:`developer_running_tests`
+may be performed manually by a developer locally. However Iris is configured to
+use `GitHub Actions`_ (GHA) for automated Continuous Integration (CI) testing.
+
+The Iris GHA YAML configuration files in the ``.github/workflows`` directory
+defines the CI tasks to be performed. For further details
+refer to the `GitHub Actions`_ documentation. The tasks performed during CI include:
+
+* running the system, integration and unit tests for Iris
+* ensuring the documentation gallery builds successfully
+* performing all doc-tests within the code base
+* checking all URL references within the code base and documentation are valid
+
+The above GHA tasks are run automatically against all `Iris`_ branches
+on GitHub whenever a pull-request is submitted, updated or merged. See the
+`Iris GitHub Actions`_ dashboard for details of recent past and active CI jobs.
+
+
+.. _gha_test_env:
+
+GitHub Actions Test Environment
+-------------------------------
+
+The CI test environments for our GHA is determined from the requirement files
+in ``requirements/pyXX.yml``. These are conda environment files list the top-level
+package dependencies for running and testing Iris.
+
+For reproducible test results, these environments are resolved for all their dependencies
+and stored as conda lock files in the ``requirements/locks`` directory. The test environments
+will not resolve the dependencies each time, instead they will use the lock files to reproduce the
+exact same environment each time.
+
+**If you have updated the requirement YAML files with new dependencies, you will need to
+generate new lock files.** To do this, run the command::
+
+ python tools/update_lockfiles.py -o requirements/locks requirements/py*.yml
+
+or simply::
+
+ make lockfiles
+
+and add the changed lockfiles to your pull request.
+
+New lockfiles are generated automatically each week to ensure that Iris continues to be
+tested against the latest available version of its dependencies.
+Each week the yaml files in ``requirements`` are resolved by a GitHub Action.
+If the resolved environment has changed, a pull request is created with the new lock files.
+The CI test suite will run on this pull request. If the tests fail, a developer
+will need to create a new branch based off the ``auto-update-lockfiles`` branch
+and add the required fixes to this new branch. If the fixes are made to the
+``auto-update-lockfiles`` branch these will be overwritten the next time the
+Github Action is run.
+
+
+GitHub Checklist
+----------------
+
+An example snapshot from a successful GitHub pull-request shows all tests
+passing:
+
+.. image:: ci_checks.png
+
+If any CI tasks fail, then the pull-request is unlikely to be merged to the
+Iris target branch by a core developer.
+
+
+.. _testing_cla:
+
+`CLA Assistant`_
+****************
+
+A bot which checks that the GitHub authors of the pull-request have signed the
+|SciTools Contributor's License Agreement (CLA)|_.
+
+
+.. _pre_commit_ci:
+
+pre-commit CI
+*************
+
+A CI service for the `pre-commit`_ framework that checks and auto fixes all
+pull-requests given the `Iris`_ GitHub repository `.pre-commit-config.yaml`_.
+
+See the `pre-commit.ci dashboard`_ for details of recent past and active Iris jobs.
+
+.. note::
+
+ The `codespell`_ ``pre-commit`` hook checks the spelling of the whole codebase
+ and documentation. This hook is configured in the ``[tool.codespell]`` section
+ of the ``pyproject.toml`` file.
+
+ Append to the ``ignore-words-list`` option any **valid words** that are
+ considered **not** a typo and should **not** be corrected by `codespell`_.
+
+ruff
+----
+As of **Iris 3.8** `ruff`_ has been adopted to ensure our codebase is using best
+practice. `ruff`_ is configured in the `Iris`_ GitHub repository using
+`.pre-commit-config.yaml`_.
+
+You can install and run `ruff`_ in your development **iris-dev** conda environment
+via::
+
+ conda activate iris-dev
+ pip install ruff
+ cd iris
+ ruff .
+
+.. note::
+
+ The `ruff`_ ``pre-commit`` hook checks for compliance of the whole codebase.
+ This hook is configured in the ``[tool.ruff]`` section
+ of the ``pyproject.toml`` file.
+
+ Edit the ``.ruff.toml`` file to include any *temporary* rules to be ignored. Edit the ``pyproject.toml`` to include any *permanent* rules to be ignored. We
+ aim to be fully `ruff`_ compliant as possible.
+
+For more information on how to use `ruff`_ please see the `ruff documentation`_.
+
+
+.. _.pre-commit-config.yaml: https://github.com/SciTools/iris/blob/main/.pre-commit-config.yaml
+.. _pre-commit.ci dashboard: https://results.pre-commit.ci/repo/github/5312648
+.. _CLA Assistant: https://github.com/cla-assistant/cla-assistant
+.. |SciTools Contributor's License Agreement (CLA)| replace:: **SciTools Contributor's License Agreement (CLA)**
+.. _ruff documentation: https://docs.astral.sh/ruff/tutorial/
diff --git a/docs/src/developers_guide/contributing_code_formatting.rst b/docs/src/developers_guide/contributing_code_formatting.rst
new file mode 100644
index 0000000000..bb3140e4f9
--- /dev/null
+++ b/docs/src/developers_guide/contributing_code_formatting.rst
@@ -0,0 +1,69 @@
+.. include:: ../common_links.inc
+
+.. _code_formatting:
+
+Code Formatting
+===============
+
+To ensure a consistent code format throughout Iris, we recommend using
+tools to check the source directly.
+
+* `black`_ for an opinionated coding auto-formatter
+* `flake8`_ linting checks
+
+The preferred way to run these tools automatically is to setup and configure
+`pre-commit`_.
+
+You can install ``pre-commit`` in your development environment using ``pip``::
+
+ $ pip install pre-commit
+
+or alternatively using ``conda``::
+
+ $ conda install -c conda-forge pre-commit
+
+.. note:: If you have setup your Python environment using the guide
+ :ref:`installing_from_source` then ``pre-commit`` should already
+ be present.
+
+In order to install the ``pre-commit`` git hooks defined in our
+``.pre-commit-config.yaml`` file, you must now run the following command from
+the root directory of Iris::
+
+ $ pre-commit install
+
+Upon performing a ``git commit``, your code will now be automatically formatted
+to the ``black`` configuration defined in our ``pyproject.toml`` file, and
+linted according to our ``.flake8`` configuration file. Note that,
+``pre-commit`` will automatically download and install the necessary packages
+for each ``.pre-commit-config.yaml`` git hook.
+
+Additionally, you may wish to enable ``black`` for your preferred
+`editor/IDE `_.
+
+With the ``pre-commit`` configured, the output of performing a ``git commit``
+will look similar to::
+
+ Check for added large files..............................................Passed
+ Check for merge conflicts................................................Passed
+ Debug Statements (Python)............................(no files to check)Skipped
+ Don't commit to branch...................................................Passed
+ black................................................(no files to check)Skipped
+ flake8...............................................(no files to check)Skipped
+ [contribution_overhaul c8513187] this is my commit message
+ 2 files changed, 10 insertions(+), 9 deletions(-)
+
+
+.. note:: You can also run `black`_ and `flake8`_ manually. Please see the
+ their officially documentation for more information.
+
+Type Hinting
+------------
+Iris is gradually adding
+`type hints `_ into the
+codebase. The reviewer will look for type hints in a pull request; if you're
+not confident with these, feel free to work together with the reviewer to
+add/improve them.
+
+
+.. _pre-commit: https://pre-commit.com/
diff --git a/docs/src/developers_guide/contributing_codebase_index.rst b/docs/src/developers_guide/contributing_codebase_index.rst
new file mode 100644
index 0000000000..b59a196ff0
--- /dev/null
+++ b/docs/src/developers_guide/contributing_codebase_index.rst
@@ -0,0 +1,13 @@
+.. _contributing.documentation.codebase:
+
+Working with the Code Base
+==========================
+
+.. toctree::
+ :maxdepth: 3
+
+ contributing_code_formatting
+ documenting/docstrings
+ documenting/rest_guide
+ contributing_deprecations
+ contributing_testing_index
diff --git a/docs/src/developers_guide/contributing_deprecations.rst b/docs/src/developers_guide/contributing_deprecations.rst
new file mode 100644
index 0000000000..8c5cb21feb
--- /dev/null
+++ b/docs/src/developers_guide/contributing_deprecations.rst
@@ -0,0 +1,120 @@
+.. _iris_development_deprecations:
+
+Deprecations
+************
+
+If you need to make a backwards-incompatible change to a public API
+[#public-api]_ that has been included in a release (e.g. deleting a
+method), then you must first deprecate the old behaviour in at least
+one release, before removing/updating it in the next
+`major release `
+
+ where you should replace `` with the major and minor version
+ of Iris in which this API is first deprecated. For example: `1.8`.
+
+ As with the deprecation warning, you should include advice on how to
+ avoid using the deprecated API within the content of this directive.
+ Feel free to include more detail in the updated docstring than in the
+ deprecation warning.
+- You should check the documentation for references to the deprecated
+ API and update them as appropriate.
+
+Changing a Default
+------------------
+
+When you need to change the default behaviour of a public API the
+situation is slightly more complex. The recommended solution is to use
+the :data:`iris.FUTURE` object. The :data:`iris.FUTURE` object provides
+boolean attributes that allow user code to control at run-time the
+default behaviour of corresponding public APIs. When a boolean attribute
+is set to `False` it causes the corresponding public API to use its
+deprecated default behaviour. When a boolean attribute is set to `True`
+it causes the corresponding public API to use its new default behaviour.
+
+The following points apply in addition to those for removing a public
+API:
+
+- You should add a new boolean attribute to :data:`iris.FUTURE` (by
+ modifying :class:`iris.Future`) that controls the default behaviour
+ of the public API that needs updating. The initial state of the new
+ boolean attribute should be `False`. You should name the new boolean
+ attribute to indicate that setting it to `True` will select the new
+ default behaviour.
+- You should include a reference to this :data:`iris.FUTURE` flag in your
+ deprecation warning and corresponding Sphinx deprecation directive.
+
+
+Removing a Deprecation
+======================
+
+When the time comes to make a new major release you should locate any
+deprecated APIs within the code that satisfy the one release
+minimum period described previously. Locating deprecated APIs can easily
+be done by searching for the Sphinx deprecation directives and/or
+deprecation warnings.
+
+Removing a Public API
+---------------------
+
+The deprecated API should be removed and any corresponding documentation
+and/or example code should be removed/updated as appropriate.
+
+.. _iris_developer_future:
+
+Changing a Default
+------------------
+
+- You should update the initial state of the relevant boolean attribute
+ of :data:`iris.FUTURE` to `True`.
+- You should deprecate setting the relevant boolean attribute of
+ :class:`iris.Future` in the same way as described in
+ :ref:`removing-a-public-api`.
+
+
+.. rubric:: Footnotes
+
+.. [#public-api] A name without a leading underscore in any of its
+ components, with the exception of the :mod:`iris.experimental` and
+ :mod:`iris.tests` packages.
+
+ Example public names are:
+ - `iris.this.`
+ - `iris.this.that`
+
+ Example private names are:
+ - `iris._this`
+ - `iris.this._that`
+ - `iris._this.that`
+ - `iris._this._that`
+ - `iris.experimental.something`
+ - `iris.tests.get_data_path`
diff --git a/docs/src/developers_guide/contributing_documentation.rst b/docs/src/developers_guide/contributing_documentation.rst
new file mode 100644
index 0000000000..e289b1548d
--- /dev/null
+++ b/docs/src/developers_guide/contributing_documentation.rst
@@ -0,0 +1,21 @@
+
+How to Contribute to the Documentation
+--------------------------------------
+
+Documentation is important and we encourage any improvements that can be made.
+If you believe the documentation is not clear please contribute a change to
+improve the documentation for all users.
+
+If you're confident diving right in, please head for
+:ref:`contributing.documentation_full`.
+
+If you're not then we've got a step-by-step guide here to walk you through it:
+:ref:`contributing.documentation_easy`
+
+.. toctree::
+ :maxdepth: 1
+ :hidden:
+
+ contributing_documentation_easy
+ contributing_documentation_full
+
\ No newline at end of file
diff --git a/docs/src/developers_guide/contributing_documentation_easy.rst b/docs/src/developers_guide/contributing_documentation_easy.rst
new file mode 100755
index 0000000000..a0513fe560
--- /dev/null
+++ b/docs/src/developers_guide/contributing_documentation_easy.rst
@@ -0,0 +1,103 @@
+
+.. include:: ../common_links.inc
+
+.. _contributing.documentation_easy:
+
+Contributing to the Documentation (the easy way)
+------------------------------------------------
+
+Documentation is important and we encourage any improvements that can be made.
+If you believe the documentation is not clear please contribute a change to
+improve the documentation for all users.
+
+The guide below is designed to be accessible to those with little-to-no
+knowledge of programming and GitHub. If you find that something doesn't work as
+described or could use more explanation then please let us know (or contribute
+the improvement yourself)!
+
+First Time Only Steps
+^^^^^^^^^^^^^^^^^^^^^
+
+1. Create a `GitHub `_ named
+ ``.readthedocs.yml``.
+
+
+.. _conf.py: https://github.com/SciTools/iris/blob/main/docs/src/conf.py
+
+
+.. _contributing.documentation.api:
+
+Generating API Documentation
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+In order to auto generate the API documentation based upon the docstrings a
+custom set of python scripts are used, these are located in the directory
+``docs/src/sphinxext``. Once the ``make html`` command has been run,
+the output of these scripts can be found in
+``docs/src/generated/api``.
+
+If there is a particularly troublesome module that breaks the ``make html`` you
+can exclude the module from the API documentation. Add the entry to the
+``exclude_modules`` tuple list in the
+``docs/src/sphinxext/generate_package_rst.py`` file.
+
+
+.. _contributing.documentation.gallery:
+
+Gallery
+~~~~~~~
+
+The Iris :ref:`gallery_index` uses a sphinx extension named
+`sphinx-gallery `_ documentation.
diff --git a/docs/src/developers_guide/contributing_getting_involved.rst b/docs/src/developers_guide/contributing_getting_involved.rst
new file mode 100644
index 0000000000..9da6cd13eb
--- /dev/null
+++ b/docs/src/developers_guide/contributing_getting_involved.rst
@@ -0,0 +1,65 @@
+.. include:: ../common_links.inc
+
+.. _development_where_to_start:
+.. _developers_guide:
+
+Developers Guide
+----------------
+
+Iris_ is an Open Source project hosted on Github and as such anyone with a
+GitHub account may create an `issue`_ on our `Iris GitHub`_ project page for
+raising:
+
+* bug reports
+* feature requests
+* documentation improvements
+
+The `Iris GitHub`_ project has been configured to use templates for each of
+the above issue types when creating a `new issue`_ to ensure the appropriate
+information is provided.
+
+Alternatively, **join the conversation** in Iris `GitHub Discussions`_, when
+you would like the opinions of the Iris community.
+
+A `pull request`_ may also be created by anyone who has become a
+|contributor|_ to Iris_. Permissions to merge pull requests to the
+``main`` branch are only given to |core developers|_ of Iris_, this is
+to ensure a measure of control. All authors on a pull request will
+automatically be asked to sign the
+`SciTools Contributor's License Agreement (CLA)`_, if they have not already
+done so.
+
+To get started we suggest reading recent `issues`_, `GitHub Discussions`_ and
+`pull requests`_ for Iris.
+
+If you are new to using GitHub we recommend reading the
+`GitHub getting started`_
+
+.. _GitHub getting started: https://docs.github.com/en/github/getting-started-with-github
+.. |contributor| replace:: **contributor**
+.. |core developers| replace:: **core developers**
+
+
+.. toctree::
+ :maxdepth: 1
+ :caption: Developers Guide
+ :name: development_index
+ :hidden:
+
+ gitwash/index
+ contributing_documentation
+ contributing_codebase_index
+ contributing_changes
+ github_app
+ release
+
+
+.. toctree::
+ :maxdepth: 1
+ :caption: Reference
+ :hidden:
+
+ ../generated/api/iris
+ ../whatsnew/index
+ ../copyright
+ ../voted_issues
diff --git a/docs/src/developers_guide/contributing_graphics_tests.rst b/docs/src/developers_guide/contributing_graphics_tests.rst
new file mode 100644
index 0000000000..1e42c35ae6
--- /dev/null
+++ b/docs/src/developers_guide/contributing_graphics_tests.rst
@@ -0,0 +1,142 @@
+.. include:: ../common_links.inc
+
+.. _testing.graphics:
+
+Adding or Updating Graphics Tests
+=================================
+
+.. note::
+
+ If a large number of images tests are failing due to an update to the
+ libraries used for image hashing, follow the instructions on
+ :ref:`refresh-imagerepo`.
+
+Generating New Results
+----------------------
+
+When you find that a graphics test in the Iris testing suite has failed,
+following changes in Iris or the run dependencies, this is the process
+you should follow:
+
+#. Create a new, empty directory to store temporary image results, at the path
+ ``lib/iris/tests/result_image_comparison`` in your Iris repository checkout.
+
+#. Run the relevant (failing) tests directly as python scripts, or using
+ ``pytest``.
+
+The results of the failing image tests will now be available in
+``lib/iris/tests/result_image_comparison``.
+
+.. note::
+
+ The ``result_image_comparison`` folder is covered by a project
+ ``.gitignore`` setting, so those files *will not show up* in a
+ ``git status`` check.
+
+Reviewing Failing Tests
+-----------------------
+
+#. Run ``iris/lib/iris/tests/graphics/idiff.py`` with python, e.g.::
+
+ python idiff.py
+
+ This will open a window for you to visually inspect
+ side-by-side **old**, **new** and **difference** images for each failed
+ graphics test. Hit a button to either :guilabel:`accept`,
+ :guilabel:`reject` or :guilabel:`skip` each new result.
+
+ If the change is **accepted**:
+
+ * the imagehash value of the new result image is added into the relevant
+ set of 'valid result hashes' in the image result database file,
+ ``tests/results/imagerepo.json``
+
+ * the relevant output file in ``tests/result_image_comparison`` is renamed
+ according to the test name. A copy of this new PNG file must then be added
+ into the ``iris-test-data`` repository, at
+ https://github.com/SciTools/iris-test-data (See below).
+
+ If a change is **skipped**:
+
+ * no further changes are made in the repo.
+
+ * when you run ``iris/tests/idiff.py`` again, the skipped choice will be
+ presented again.
+
+ If a change is **rejected**:
+
+ * the output image is deleted from ``result_image_comparison``.
+
+ * when you run ``iris/tests/idiff.py`` again, the skipped choice will not
+ appear, unless the relevant failing test is re-run.
+
+#. **Now re-run the tests**. The **new** result should now be recognised and the
+ relevant test should pass. However, some tests can perform *multiple*
+ graphics checks within a single test case function. In those cases, any
+ failing check will prevent the following ones from being run, so a test
+ re-run may encounter further (new) graphical test failures. If that
+ happens, simply repeat the check-and-accept process until all tests pass.
+
+#. You're now ready to :ref:`add-graphics-test-changes`
+
+
+Adding a New Image Test
+-----------------------
+
+If you attempt to run ``idiff.py`` when there are new graphical tests for which
+no baseline yet exists, you will get a warning that ``idiff.py`` is ``Ignoring
+unregistered test result...``. In this case,
+
+#. rename the relevant images from ``iris/tests/result_image_comparison`` by
+
+ * removing the ``result-`` prefix
+
+ * fully qualifying the test name if it isn't already (i.e. it should start
+ ``iris.tests...``or ``gallery_tests...``)
+
+#. run the tests in the mode that lets them create missing data (see
+ :ref:`create-missing`). This will update ``imagerepo.json`` with the new
+ test name and image hash.
+
+#. and then add them to the Iris test data as covered in
+ :ref:`add-graphics-test-changes`.
+
+
+.. _refresh-imagerepo:
+
+Refreshing the Stored Hashes
+----------------------------
+
+From time to time, a new version of the image hashing library will cause all
+image hashes to change. The image hashes stored in
+``tests/results/imagerepo.json`` can be refreshed using the baseline images
+stored in the ``iris-test-data`` repository (at
+https://github.com/SciTools/iris-test-data) using the script
+``tests/graphics/recreate_imagerepo.py``. Use the ``--help`` argument for the
+command line arguments.
+
+
+.. _add-graphics-test-changes:
+
+Add Your Changes to Iris
+------------------------
+
+To add your changes to Iris, you need to make two pull requests (PR).
+
+#. The first PR is made in the ``iris-test-data`` repository, at
+ https://github.com/SciTools/iris-test-data.
+
+ * Add all the newly-generated referenced PNG files into the
+ ``test_data/images`` directory. In your Iris repo, these files are to be found
+ in the temporary results folder ``iris/tests/result_image_comparison``.
+
+ * Create a PR proposing these changes, in the usual way.
+
+#. The second PR is the one that makes the changes you intend to the Iris_ repository.
+ The description box of this pull request should contain a reference to
+ the matching one in ``iris-test-data``.
+
+ * This PR should include updating the version of the test data in
+ ``.github/workflows/ci-tests.yml`` and
+ ``.github/workflows/ci-docs-tests.yml`` to the new version created by the
+ merging of your ``iris-test-data`` PR.
diff --git a/docs/src/developers_guide/contributing_pull_request_checklist.rst b/docs/src/developers_guide/contributing_pull_request_checklist.rst
new file mode 100644
index 0000000000..11d68ace46
--- /dev/null
+++ b/docs/src/developers_guide/contributing_pull_request_checklist.rst
@@ -0,0 +1,61 @@
+.. include:: ../common_links.inc
+
+.. _pr_check:
+
+Pull Request Checklist
+======================
+
+All pull request will be reviewed by a core developer who will manage the
+process of merging. It is the responsibility of the contributor submitting a
+pull request to do their best to deliver a pull request which meets the
+requirements of the project it is submitted to.
+
+This check list summarises criteria which will be checked before a pull request
+is merged. Before submitting a pull request please consider the following:
+
+
+#. **Provide a helpful description** of the Pull Request. This should include:
+
+ * The aim of the change / the problem addressed / a link to the issue.
+ * How the change has been delivered.
+
+#. **Include a "What's New" entry**, if appropriate.
+ See :ref:`whats_new_contributions`.
+
+#. **Check all tests pass**. This includes existing tests and any new tests
+ added for any new functionality. For more information see
+ :ref:`developer_running_tests`.
+
+#. **Check all modified and new source files conform to the required**
+ :ref:`code_formatting`.
+
+#. **Check all new dependencies added to the** `requirements`_ **yaml
+ files.** If dependencies have been added then new nox testing lockfiles
+ should be generated too, see :ref:`gha_test_env`.
+
+#. **Check the source documentation been updated to explain all new or changed
+ features**. Note, we now use numpydoc strings. Any touched code should
+ be updated to use the docstrings formatting. See :ref:`docstrings`.
+
+#. **Include code examples inside the docstrings where appropriate**. See
+ :ref:`contributing.documentation.testing`.
+
+#. **Check the documentation builds without warnings or errors**. See
+ :ref:`contributing.documentation.building`
+
+#. **Check for any new dependencies in the** `readthedocs.yml`_ **file**. This
+ file is used to build the documentation that is served from
+ https://scitools-iris.readthedocs.io/en/latest/
+
+#. **Check for updates needed for supporting projects for test or example
+ data**. For example:
+
+ * `iris-test-data`_ is a github project containing all the data to support
+ the tests.
+ * `iris-sample-data`_ is a github project containing all the data to support
+ the gallery and examples.
+
+ If new files are required by tests or code examples, they must be added to
+ the appropriate supporting project via a suitable pull-request. This pull
+ request should be referenced in the main Iris pull request and must be
+ accepted and merged before the Iris one can be.
diff --git a/docs/src/developers_guide/contributing_running_tests.rst b/docs/src/developers_guide/contributing_running_tests.rst
new file mode 100644
index 0000000000..f60cedba05
--- /dev/null
+++ b/docs/src/developers_guide/contributing_running_tests.rst
@@ -0,0 +1,197 @@
+.. include:: ../common_links.inc
+
+.. _developer_running_tests:
+
+Running the Tests
+*****************
+
+There are two options for running the tests:
+
+* Use an environment you created yourself. This requires more manual steps to
+ set up, but gives you more flexibility. For example, you can run a subset of
+ the tests or use ``python`` interactively to investigate any issues. See
+ :ref:`test manual env`.
+
+* Use ``nox``. This will automatically generate an environment and run test
+ sessions consistent with our GitHub continuous integration. See :ref:`using nox`.
+
+.. _test manual env:
+
+Testing Iris in a Manually Created Environment
+==============================================
+
+To create a suitable environment for running the tests, see :ref:`installing_from_source`.
+
+Many Iris tests will use data that may be defined in the test itself, however
+this is not always the case as sometimes example files may be used. Due to
+the size of some of the files used these are not kept in the Iris repository.
+A separate repository under the `SciTools`_ organisation is used, see
+https://github.com/SciTools/iris-test-data.
+
+In order to run the tests with **all** the test data you must clone the
+``iris-test-data`` repository and then ensure the Iris tests can access
+``iris-test-data/test_data``, using one of two methods:
+
+* Store the path in a shell environment variable named **OVERRIDE_TEST_DATA_REPOSITORY**.
+* Store the path in ``lib/iris/etc/site.cfg`` (see :mod:`iris.config` for more).
+
+The example command below uses ``~/projects`` as the parent directory::
+
+ cd ~/projects
+ git clone git@github.com:SciTools/iris-test-data.git
+ export OVERRIDE_TEST_DATA_REPOSITORY=~/projects/iris-test-data/test_data
+
+All the Iris tests may be run from the root ``iris`` project directory using
+``pytest``. For example::
+
+ pytest -n 2
+
+will run the tests across two processes. For more options, use the command
+``pytest -h``. Below is a trimmed example of the output::
+
+ ============================= test session starts ==============================
+ platform linux -- Python 3.10.5, pytest-7.1.2, pluggy-1.0.0
+ rootdir: /path/to/git/clone/iris, configfile: pyproject.toml, testpaths: lib/iris
+ plugins: xdist-2.5.0, forked-1.4.0
+ gw0 I / gw1 I
+ gw0 [6361] / gw1 [6361]
+
+ ........................................................................ [ 1%]
+ ........................................................................ [ 2%]
+ ........................................................................ [ 3%]
+ ...
+ .......................ssssssssssssssssss............................... [ 99%]
+ ........................ [100%]
+ =============================== warnings summary ===============================
+ ...
+ -- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
+ =========================== short test summary info ============================
+ SKIPPED [1] lib/iris/tests/experimental/test_raster.py:152: Test requires 'gdal'.
+ SKIPPED [1] lib/iris/tests/experimental/test_raster.py:155: Test requires 'gdal'.
+ ...
+ ========= 6340 passed, 21 skipped, 1659 warnings in 193.57s (0:03:13) ==========
+
+There may be some tests that have been **skipped**. This is due to a Python
+decorator being present in the test script that will intentionally skip a test
+if a certain condition is not met. In the example output above there are
+**21** skipped tests. At the point in time when this was run this was due to an
+experimental dependency not being present.
+
+.. tip::
+
+ The most common reason for tests to be skipped is when the directory for the
+ ``iris-test-data`` has not been set which would shows output such as::
+
+ SKIPPED [1] lib/iris/tests/unit/fileformats/test_rules.py:157: Test(s) require external data.
+ SKIPPED [1] lib/iris/tests/unit/fileformats/pp/test__interpret_field.py:97: Test(s) require external data.
+ SKIPPED [1] lib/iris/tests/unit/util/test_demote_dim_coord_to_aux_coord.py:29: Test(s) require external data.
+
+ All Python decorators that skip tests will be defined in
+ ``lib/iris/tests/__init__.py`` with a function name with a prefix of
+ ``skip_``.
+
+You can also run a specific test module. The example below runs the tests for
+mapping::
+
+ cd lib/iris/tests
+ python test_mapping.py
+
+When running the test directly as above you can view the command line options
+using the commands ``python test_mapping.py -h`` or
+``python test_mapping.py --help``.
+
+.. tip:: A useful command line option to use is ``-d``. This will display
+ matplotlib_ figures as the tests are run. For example::
+
+ python test_mapping.py -d
+
+.. _using nox:
+
+Using Nox for Testing Iris
+==========================
+
+The `nox`_ tool has for adopted for automated testing on `Iris GitHub Actions`_
+and also locally on the command-line for developers.
+
+`nox`_ is similar to `tox`_, but instead leverages the expressiveness and power of a Python
+configuration file rather than an `.ini` style file. As with `tox`_, `nox`_ can use `virtualenv`_
+to create isolated Python environments, but in addition also supports `conda`_ as a testing
+environment backend.
+
+
+Where is Nox Used?
+------------------
+
+Iris uses `nox`_ as a convenience to fully automate the process of executing the Iris tests, but also
+automates the process of:
+
+* building the documentation and executing the doc-tests
+* building the documentation gallery
+* running the documentation URL link check
+
+You can perform all of these tasks manually yourself, however the onus is on you to first ensure
+that all of the required package dependencies are installed and available in the testing environment.
+
+`Nox`_ has been configured to automatically do this for you, and provides a means to easily replicate
+the remote testing behaviour of `Iris GitHub Actions`_ locally for the developer.
+
+
+Installing Nox
+--------------
+
+We recommend installing `nox`_ using `conda`_. To install `nox`_ in a separate `conda`_ environment::
+
+ conda create -n nox -c conda-forge nox
+ conda activate nox
+
+To install `nox`_ in an existing active `conda`_ environment::
+
+ conda install -c conda-forge nox
+
+The `nox`_ package is also available on PyPI, however `nox`_ has been configured to use the `conda`_
+backend for Iris, so an installation of `conda`_ must always be available.
+
+
+Testing with Nox
+----------------
+
+The `nox`_ configuration file `noxfile.py` is available in the root ``iris`` project directory, and
+defines all the `nox`_ sessions (i.e., tasks) that may be performed. `nox`_ must always be executed
+from the ``iris`` root directory.
+
+To list the configured `nox`_ sessions for Iris::
+
+ nox --list
+
+To run the Iris tests for all configured versions of Python::
+
+ nox --session tests
+
+To build the Iris documentation specifically for Python 3.7::
+
+ nox --session doctest-3.7
+
+To run all the Iris `nox`_ sessions::
+
+ nox
+
+For further `nox`_ command-line options::
+
+ nox --help
+
+.. tip::
+ For `nox`_ sessions that use the `conda`_ backend, you can use the ``-v`` or ``--verbose``
+ flag to display the `nox`_ `conda`_ environment package details and environment info.
+ For example::
+
+ nox --session tests -- --verbose
+
+
+.. note:: `nox`_ will cache its testing environments in the `.nox` root ``iris`` project directory.
+
+
+.. _setuptools: https://setuptools.readthedocs.io/en/latest/
+.. _tox: https://tox.readthedocs.io/en/latest/
+.. _virtualenv: https://virtualenv.pypa.io/en/latest/
+.. _PyPI: https://pypi.org/project/nox/
+.. _v41.5.0: https://setuptools.readthedocs.io/en/latest/history.html#v41-5-0
diff --git a/docs/src/developers_guide/contributing_testing.rst b/docs/src/developers_guide/contributing_testing.rst
new file mode 100644
index 0000000000..a65bcebd55
--- /dev/null
+++ b/docs/src/developers_guide/contributing_testing.rst
@@ -0,0 +1,147 @@
+.. include:: ../common_links.inc
+
+.. _developer_test_categories:
+
+
+Test Categories
+***************
+
+There are two main categories of tests within Iris:
+
+- :ref:`testing.unit_test`
+- :ref:`testing.integration`
+
+Ideally, all code changes should be accompanied by one or more unit
+tests, and by zero or more integration tests.
+
+But if in any doubt about what tests to add or how to write them please
+feel free to submit a pull-request in any state and ask for assistance.
+
+
+.. _testing.unit_test:
+
+Unit Tests
+==========
+
+Code changes should be accompanied by enough unit tests to give a
+high degree of confidence that the change works as expected. In
+addition, the unit tests can help describe the intent behind a change.
+
+The docstring for each test module must state the unit under test.
+For example:
+
+ :literal:`"""Unit tests for the \`iris.experimental.raster.export_geotiff\` function."""`
+
+All unit tests must be placed and named according to the following
+structure:
+
+
+.. _testing.classes:
+
+Classes
+-------
+
+When testing a class all the tests must reside in the module:
+
+ :literal:`lib/iris/tests/unit//test_.py`
+
+Within this test module each tested method must have one or more
+corresponding test classes, for example:
+
+* ``Test_``
+* ``Test___``
+
+And within those test classes, the test methods must be named according
+to the aspect of the tested method which they address.
+
+**Examples**:
+
+All unit tests for :py:class:`iris.cube.Cube` must reside in:
+
+ :literal:`lib/iris/tests/unit/cube/test_Cube.py`
+
+Within that file the tests might look something like:
+
+.. code-block:: python
+
+ # Tests for the Cube.xml() method.
+ class Test_xml(tests.IrisTest):
+ def test_some_general_stuff(self):
+ ...
+
+
+ # Tests for the Cube.xml() method, focussing on the behaviour of
+ # the checksums.
+ class Test_xml__checksum(tests.IrisTest):
+ def test_checksum_ignores_masked_values(self):
+ ...
+
+
+ # Tests for the Cube.add_dim_coord() method.
+ class Test_add_dim_coord(tests.IrisTest):
+ def test_normal_usage(self):
+ ...
+
+ def test_coord_already_present(self):
+ ...
+
+
+.. _testing.functions:
+
+Functions
+---------
+
+When testing a function all the tests must reside in the module:
+
+ :literal:`lib/iris/tests/unit//test_.py`
+
+Within this test module there must be one or more test classes, for example:
+
+* ``Test``
+* ``TestAspectOfFunction``
+
+And within those test classes, the test methods must be named according
+to the aspect of the tested function which they address.
+
+**Examples**:
+
+All unit tests for :py:func:`iris.experimental.raster.export_geotiff`
+must reside in:
+
+ :literal:`lib/iris/tests/unit/experimental/raster/test_export_geotiff.py`
+
+Within that file the tests might look something like:
+
+.. code-block:: python
+
+ # Tests focussing on the handling of different data types.
+ class TestDtypeAndValues(tests.IrisTest):
+ def test_int16(self):
+ ...
+
+ def test_int16_big_endian(self):
+ ...
+
+
+ # Tests focussing on the handling of different projections.
+ class TestProjection(tests.IrisTest):
+ def test_no_ellipsoid(self):
+ ...
+
+
+.. _testing.integration:
+
+Integration Tests
+=================
+
+Some code changes may require tests which exercise several units in
+order to demonstrate an important consequence of their interaction which
+may not be apparent when considering the units in isolation.
+
+These tests must be placed in the ``lib/iris/tests/integration`` folder.
+Unlike unit tests, there is no fixed naming scheme for integration
+tests. But folders and files must be created as required to help
+developers locate relevant tests. It is recommended they are named
+according to the capabilities under test, e.g.
+``metadata/test_pp_preservation.py``, and not named according to the
+module(s) under test.
diff --git a/docs/src/developers_guide/contributing_testing_index.rst b/docs/src/developers_guide/contributing_testing_index.rst
new file mode 100644
index 0000000000..2f5ae411e8
--- /dev/null
+++ b/docs/src/developers_guide/contributing_testing_index.rst
@@ -0,0 +1,14 @@
+.. _testing:
+
+Testing
+=======
+
+.. toctree::
+ :maxdepth: 3
+
+ contributing_testing
+ testing_tools
+ contributing_graphics_tests
+ contributing_running_tests
+ contributing_ci_tests
+ contributing_benchmarks
diff --git a/docs/src/developers_guide/documenting/__init__.py b/docs/src/developers_guide/documenting/__init__.py
new file mode 100644
index 0000000000..6e031999e7
--- /dev/null
+++ b/docs/src/developers_guide/documenting/__init__.py
@@ -0,0 +1 @@
+# noqa: D104
diff --git a/docs/src/developers_guide/documenting/docstrings.rst b/docs/src/developers_guide/documenting/docstrings.rst
new file mode 100644
index 0000000000..86f2c839c1
--- /dev/null
+++ b/docs/src/developers_guide/documenting/docstrings.rst
@@ -0,0 +1,34 @@
+.. _docstrings:
+
+==========
+Docstrings
+==========
+
+Every public object in the Iris package should have an appropriate docstring.
+This is important as the docstrings are used by developers to understand
+the code and may be read directly in the source or via the
+:doc:`../../generated/api/iris`.
+
+.. note::
+ As of April 2022 we are looking to adopt `numpydoc`_ strings as standard.
+ We aim to complete the adoption over time as we do changes to the codebase.
+ For examples of use see `numpydoc`_ and `sphinxcontrib-napoleon`_
+
+For consistency always use:
+
+* ``"""triple double quotes"""`` around docstrings.
+* ``r"""raw triple double quotes"""`` if you use any backslashes in your
+ docstrings.
+* ``u"""Unicode triple-quoted string"""`` for Unicode docstrings
+
+All docstrings can use reST (reStructuredText) markup to augment the
+rendered formatting. See the :ref:`reST_quick_start` for more detail.
+
+For more information including examples pleasee see:
+
+* `numpydoc`_
+* `sphinxcontrib-napoleon`_
+
+
+.. _numpydoc: https://numpydoc.readthedocs.io/en/latest/format.html#style-guide
+.. _sphinxcontrib-napoleon: https://sphinxcontrib-napoleon.readthedocs.io/en/latest/example_numpy.html
\ No newline at end of file
diff --git a/docs/src/developers_guide/documenting/docstrings_attribute.py b/docs/src/developers_guide/documenting/docstrings_attribute.py
new file mode 100644
index 0000000000..1714373a62
--- /dev/null
+++ b/docs/src/developers_guide/documenting/docstrings_attribute.py
@@ -0,0 +1,38 @@
+"""Docstring attribute example."""
+
+
+class ExampleClass:
+ """Class Summary."""
+
+ def __init__(self, arg1, arg2):
+ """Purpose section description.
+
+ Description section text.
+
+ Parameters
+ ----------
+ arg1 : int
+ First argument description.
+ arg2 : float
+ Second argument description.
+
+ Returns
+ -------
+ bool
+
+ """
+ self.a = arg1
+ "Attribute arg1 docstring."
+ self.b = arg2
+ "Attribute arg2 docstring."
+
+ @property
+ def square(self):
+ """*(read-only)* Purpose section description.
+
+ Returns
+ -------
+ int
+
+ """
+ return self.a * self.a
diff --git a/docs/src/developers_guide/documenting/docstrings_sample_routine.py b/docs/src/developers_guide/documenting/docstrings_sample_routine.py
new file mode 100644
index 0000000000..7feec6dbd0
--- /dev/null
+++ b/docs/src/developers_guide/documenting/docstrings_sample_routine.py
@@ -0,0 +1,27 @@
+"""Docstring routine example."""
+
+
+def sample_routine(arg1, arg2, kwarg1="foo", kwarg2=None):
+ """Purpose section text goes here.
+
+ Description section longer text goes here.
+
+ Parameters
+ ----------
+ arg1 : numpy.ndarray
+ First argument description.
+ arg2 : numpy.ndarray
+ Second argument description.
+ kwarg1 : str, optional
+ The first keyword argument. This argument description
+ can be multi-lined.
+ **kwarg2 : bool, optional
+ The second keyword argument.
+
+ Returns
+ -------
+ numpy.ndarray
+ A numpy.ndarray of arg1 * arg2.
+
+ """
+ pass
diff --git a/docs/src/developers_guide/documenting/rest_guide.rst b/docs/src/developers_guide/documenting/rest_guide.rst
new file mode 100644
index 0000000000..9e8c1107b0
--- /dev/null
+++ b/docs/src/developers_guide/documenting/rest_guide.rst
@@ -0,0 +1,44 @@
+.. include:: ../../common_links.inc
+
+.. _reST_quick_start:
+
+================
+reST Quick Start
+================
+
+`reST`_ is used to create the documentation for Iris_. It is used to author
+all of the documentation content including use in docstrings where appropriate.
+For more information see :ref:`docstrings`.
+
+reST is a lightweight markup language intended to be highly readable in
+source format. This guide will cover some of the more frequently used advanced
+reST markup syntaxes, for the basics of reST the following links may be useful:
+
+* https://www.sphinx-doc.org/en/master/usage/restructuredtext/
+* https://packages.python.org/an_example_pypi_project/sphinx.html
+
+Reference documentation for reST can be found at https://docutils.sourceforge.net/rst.html.
+
+Creating Links
+--------------
+Basic links can be created with ```Text of the link `_``
+which will look like `Text of the link `_
+
+
+Documents in the same project can be cross referenced with the syntax
+``:doc:`document_name``` for example, to reference the "docstrings" page
+``:doc:`docstrings``` creates the following link :doc:`docstrings`
+
+
+References can be created between sections by first making a "label" where
+you would like the link to point to ``.. _name_of_reference::`` the
+appropriate link can now be created with ``:ref:`name_of_reference```
+(note the trailing underscore on the label)
+
+
+Cross referencing other reference documentation can be achieved with the
+syntax ``:py:class:`zipfile.ZipFile``` which will result in links such as
+:py:class:`zipfile.ZipFile` and :py:class:`numpy.ndarray`.
+
+
+.. _reST: https://en.wikipedia.org/wiki/ReStructuredText
diff --git a/docs/src/developers_guide/documenting/whats_new_contributions.rst b/docs/src/developers_guide/documenting/whats_new_contributions.rst
new file mode 100644
index 0000000000..82569e57a0
--- /dev/null
+++ b/docs/src/developers_guide/documenting/whats_new_contributions.rst
@@ -0,0 +1,145 @@
+.. include:: ../../common_links.inc
+
+.. _whats_new_contributions:
+
+=================================
+Contributing a "What's New" Entry
+=================================
+
+Iris uses a file named ``latest.rst`` to keep a draft of upcoming development
+changes that will form the next stable release. Contributions to the
+:ref:`iris_whatsnew` document are written by the developer most familiar
+with the change made. The contribution should be included as part of
+the Iris Pull Request that introduces the change.
+
+The ``latest.rst`` and the past release notes are kept in the
+``docs/src/whatsnew/`` directory. If you are writing the first contribution after
+an Iris release: **create the new** ``latest.rst`` by copying the content from
+``latest.rst.template`` in the same directory.
+
+Since the `Contribution categories`_ include Internal changes, **all** Iris
+Pull Requests should be accompanied by a "What's New" contribution.
+
+
+Git Conflicts
+=============
+
+If changes to ``latest.rst`` are being suggested in several simultaneous
+Iris Pull Requests, Git will likely encounter merge conflicts. If this
+situation is thought likely (large PR, high repo activity etc.):
+
+* PR author: Do not include a "What's New" entry. Mention in the PR text that a
+ "What's New" entry is pending
+
+* PR reviewer: Review the PR as normal. Once the PR is acceptable, ask that
+ a **new pull request** be created specifically for the "What's New" entry,
+ which references the main pull request and titled (e.g. for PR#9999):
+
+ What's New for #9999
+
+* PR author: create the "What's New" pull request
+
+* PR reviewer: once the "What's New" PR is created, **merge the main PR**.
+ (this will fix any `Iris GitHub Actions`_ linkcheck errors where the links in the
+ "What's New" PR reference new features introduced in the main PR)
+
+* PR reviewer: review the "What's New" PR, merge once acceptable
+
+These measures should mean the suggested ``latest.rst`` changes are outstanding
+for the minimum time, minimising conflicts and minimising the need to rebase or
+merge from trunk.
+
+
+Writing a Contribution
+======================
+
+A contribution is the short description of a change introduced to Iris
+which improved it in some way. As such, a single Iris Pull Request may
+contain multiple changes that are worth highlighting as contributions to the
+what's new document.
+
+The appropriate contribution for a pull request might in fact be an addition or
+change to an existing "What's New" entry.
+
+Each contribution will ideally be written as a single concise entry using a
+reStructuredText auto-enumerated list ``#.`` directive. Where possible do not
+exceed **column 80** and ensure that any subsequent lines of the same entry are
+aligned with the first. The content should target an Iris user as the audience.
+The required content, in order, is as follows:
+
+* Use your discretion to decide on the names of all those that you want to
+ acknowledge as part of your contribution. Also consider the efforts of the
+ reviewer. Please use GitHub user names that link to their GitHub profile
+ e.g.,
+
+ ```@tkknight`_ Lorem ipsum dolor sit amet ...``
+
+ Also add a full reference in the following section at the end of the ``latest.rst``::
+
+ .. comment
+ Whatsnew author names (@github name) in alphabetical order. Note that,
+ core dev names are automatically included by the common_links.inc:
+
+ .. _@tkknight: https://github.com/tkknight
+
+* A succinct summary of the new/changed behaviour.
+
+* Context to the change. Possible examples include: what this fixes, why
+ something was added, issue references (e.g. ``:issue:`9999```), more specific
+ detail on the change itself.
+
+* Pull request references, bracketed, following the final period e.g.,
+ ``(:pull:`1111`, :pull:`9999`)``
+
+* A trailing blank line (standard reStructuredText list format).
+
+For example::
+
+ #. `@tkknight `_ and
+ `@trexfeathers `_ (reviewer) changed
+ argument ``x`` to be optional in :class:`~iris.module.class` and
+ :meth:`iris.module.method`. This allows greater flexibility as requested in
+ :issue:`9999`. (:pull:`1111`, :pull:`9999`)
+
+
+The above example also demonstrates some of the possible syntax for including
+links to code. For more inspiration on possible content and references, please
+examine past what's :ref:`iris_whatsnew` entries.
+
+.. note:: The reStructuredText syntax will be checked as part of building
+ the documentation. Any warnings should be corrected. The
+ `Iris GitHub Actions`_ will automatically build the documentation when
+ creating a pull request, however you can also manually
+ :ref:`build ` the documentation.
+
+
+Contribution Categories
+=======================
+
+The structure of the what's new release note should be easy to read by
+users. To achieve this several categories may be used.
+
+**📢 Announcements**
+ General news and announcements to the Iris community.
+
+**✨ Features**
+ Features that are new or changed to add functionality.
+
+**🐛 Bug Fixes**
+ A bug fix.
+
+**💣 Incompatible Changes**
+ A change that causes an incompatibility with prior versions of Iris.
+
+**🔥 Deprecations**
+ Deprecations of functionality.
+
+**🔗 Dependencies**
+ Additions, removals and version changes in Iris' package dependencies.
+
+**📚 Documentation**
+ Changes to documentation.
+
+**💼 Internal**
+ Changes to any internal or development related topics, such as testing,
+ environment dependencies etc.
diff --git a/docs/src/developers_guide/edit_button.png b/docs/src/developers_guide/edit_button.png
new file mode 100755
index 0000000000..ee2e7858f0
Binary files /dev/null and b/docs/src/developers_guide/edit_button.png differ
diff --git a/docs/src/developers_guide/edit_on_github.png b/docs/src/developers_guide/edit_on_github.png
new file mode 100755
index 0000000000..f802ebd9d6
Binary files /dev/null and b/docs/src/developers_guide/edit_on_github.png differ
diff --git a/docs/src/developers_guide/find_main.png b/docs/src/developers_guide/find_main.png
new file mode 100755
index 0000000000..8a7af306cd
Binary files /dev/null and b/docs/src/developers_guide/find_main.png differ
diff --git a/docs/src/developers_guide/fork_banner.png b/docs/src/developers_guide/fork_banner.png
new file mode 100755
index 0000000000..0d140c9fc6
Binary files /dev/null and b/docs/src/developers_guide/fork_banner.png differ
diff --git a/docs/src/developers_guide/github_app.rst b/docs/src/developers_guide/github_app.rst
new file mode 100644
index 0000000000..402cfe0c75
--- /dev/null
+++ b/docs/src/developers_guide/github_app.rst
@@ -0,0 +1,281 @@
+.. include:: ../common_links.inc
+
+Token GitHub App
+----------------
+
+.. note::
+
+ This section of the documentation is applicable only to GitHub `SciTools`_
+ Organisation **owners** and **administrators**.
+
+.. note::
+
+ The ``iris-actions`` GitHub App has been rebranded with the more generic
+ name ``scitools-ci``, as the app can be used for any `SciTools`_ repository,
+ not just ``iris`` specifically.
+
+ All of the following instructions are still applicable.
+
+
+This section describes how to create, configure, install and use our `SciTools`_
+GitHub App for generating tokens for use with *GitHub Actions* (GHA).
+
+
+Background
+^^^^^^^^^^
+
+Our GitHub *Continuous Integration* (CI) workflows require fully reproducible
+`conda`_ environments to test ``iris`` and build our documentation.
+
+The ``iris`` `refresh-lockfiles`_ GHA workflow uses the `conda-lock`_ package to routinely
+generate a platform specific ``lockfile`` containing all the package dependencies
+required by ``iris`` for a specific version of ``python``.
+
+The environment lockfiles created by the `refresh-lockfiles`_ GHA are contributed
+back to ``iris`` though a pull-request that is automatically generated using the
+third-party `create-pull-request`_ GHA. By default, pull-requests created by such an
+action using the standard ``GITHUB_TOKEN`` **cannot** trigger other workflows, such
+as our CI.
+
+As a result, we use a dedicated authentication **GitHub App** to securely generate tokens
+for the `create-pull-request`_ GHA, which then permits our full suite of CI testing workflows
+to be triggered against the lockfiles pull-request. Ensuring that the CI is triggered gives us
+confidence that the proposed new lockfiles have not introduced a package level incompatibility
+or issue within ``iris``. See :ref:`use gha`.
+
+
+Create GitHub App
+^^^^^^^^^^^^^^^^^
+
+The **GitHub App** is created for the sole purpose of generating tokens for use with actions,
+and **must** be owned by the `SciTools`_ organisation.
+
+To create a minimal `GitHub App`_ for this purpose, perform the following steps:
+
+1. Click the `SciTools`_ organisation ``⚙️ Settings`` option.
+
+.. figure:: assets/scitools-settings.png
+ :alt: SciTools organisation Settings option
+ :align: center
+ :width: 75%
+
+2. Click the ``GitHub Apps`` option from the ``<> Developer settings``
+ section in the left hand sidebar.
+
+.. figure:: assets/developer-settings-github-apps.png
+ :alt: Developer settings, GitHub Apps option
+ :align: center
+ :width: 25%
+
+3. Now click the ``New GitHub App`` button to display the ``Register new GitHub App``
+ form.
+
+Within the ``Register new GitHub App`` form, complete the following fields:
+
+4. Set the **mandatory** ``GitHub App name`` field to be ``iris-actions``.
+5. Set the **mandatory** ``Homepage URL`` field to be ``https://github.com/SciTools/iris``
+6. Under the ``Webhook`` section, **uncheck** the ``Active`` checkbox.
+ Note that, **no** ``Webhook URL`` is required.
+
+.. figure:: assets/webhook-active.png
+ :alt: Webhook active checkbox
+ :align: center
+ :width: 75%
+
+7. Under the ``Repository permissions`` section, set the ``Contents`` field to
+ be ``Access: Read and write``.
+
+.. figure:: assets/repo-perms-contents.png
+ :alt: Repository permissions Contents option
+ :align: center
+ :width: 75%
+
+8. Under the ``Repository permissions`` section, set the ``Pull requests`` field
+ to be ``Access: Read and write``.
+
+.. figure:: assets/repo-perms-pull-requests.png
+ :alt: Repository permissions Pull requests option
+ :align: center
+ :width: 75%
+
+9. Under the ``Organization permissions`` section, set the ``Members`` field to
+ be ``Access: Read-only``.
+
+.. figure:: assets/org-perms-members.png
+ :alt: Organization permissions Members
+ :align: center
+ :width: 75%
+
+10. Under the ``User permissions`` section, for the ``Where can this GitHub App be installed?``
+ field, **check** the ``Only on this account`` radio-button i.e., only allow
+ this GitHub App to be installed on the **SciTools** account.
+
+.. figure:: assets/user-perms.png
+ :alt: User permissions
+ :align: center
+ :width: 75%
+
+11. Finally, click the ``Create GitHub App`` button.
+
+
+Configure GitHub App
+^^^^^^^^^^^^^^^^^^^^
+
+Creating the GitHub App will automatically redirect you to the ``SciTools settings / iris-actions``
+form for the newly created app.
+
+Perform the following GitHub App configuration steps:
+
+.. _app id:
+
+1. Under the ``About`` section, note of the GitHub ``App ID`` as this value is
+ required later. See :ref:`gha secrets`.
+2. Under the ``Display information`` section, optionally upload the ``iris`` logo
+ as a ``png`` image.
+3. Under the ``Private keys`` section, click the ``Generate a private key`` button.
+
+.. figure:: assets/generate-key.png
+ :alt: Private keys Generate a private key
+ :align: center
+ :width: 75%
+
+.. _private key:
+
+GitHub will automatically generate a private key to sign access token requests
+for the app. Also a separate browser pop-up window will appear with the GitHub
+App private key in ``OpenSSL PEM`` format.
+
+.. figure:: assets/download-pem.png
+ :alt: Download OpenSSL PEM file
+ :align: center
+ :width: 50%
+
+.. important::
+
+ Please ensure that you save the ``OpenSSL PEM`` file and **securely** archive
+ its contents. The private key within this file is required later.
+ See :ref:`gha secrets`.
+
+
+Install GitHub App
+^^^^^^^^^^^^^^^^^^
+
+To install the GitHub App:
+
+1. Select the ``Install App`` option from the top left menu of the
+ ``Scitools settings / iris-actions`` form, then click the ``Install`` button.
+
+.. figure:: assets/install-app.png
+ :alt: Private keys Generate a private key
+ :align: center
+ :width: 75%
+
+2. Select the ``Only select repositories`` radio-button from the ``Install iris-actions``
+ form, and choose the ``SciTools/iris`` repository.
+
+.. figure:: assets/install-iris-actions.png
+ :alt: Install iris-actions GitHub App
+ :align: center
+ :width: 75%
+
+3. Click the ``Install`` button.
+
+ The successfully installed ``iris-actions`` GitHub App is now available under
+ the ``GitHub Apps`` option in the ``Integrations`` section of the `SciTools`_
+ organisation ``Settings``. Note that, to reconfigure the installed app click
+ the ``⚙️ App settings`` option.
+
+.. figure:: assets/installed-app.png
+ :alt: Installed GitHub App
+ :align: center
+ :width: 80%
+
+4. Finally, confirm that the ``iris-actions`` GitHub App is now available within
+ the `SciTools/iris`_ repository by clicking the ``GitHub apps`` option in the
+ ``⚙️ Settings`` section.
+
+.. figure:: assets/iris-github-apps.png
+ :alt: Iris installed GitHub App
+ :align: center
+ :width: 80%
+
+
+.. _gha secrets:
+
+Create Repository Secrets
+^^^^^^^^^^^^^^^^^^^^^^^^^
+
+The GitHub Action that requests an access token from the ``iris-actions``
+GitHub App must be configured with the following information:
+
+* the ``App ID``, and
+* the ``OpenSSL PEM`` private key
+
+associated with the ``iris-actions`` GitHub App. This **sensitive** information is
+made **securely** available by creating `SciTools/iris`_ repository secrets:
+
+1. Click the `SciTools/iris`_ repository ``⚙️ Settings`` option.
+
+.. figure:: assets/iris-settings.png
+ :alt: Iris Settings
+ :align: center
+ :width: 75%
+
+2. Click the ``Actions`` option from the ``Security`` section in the left hand
+ sidebar.
+
+.. figure:: assets/iris-security-actions.png
+ :alt: Iris Settings Security Actions
+ :align: center
+ :width: 25%
+
+3. Click the ``New repository secret`` button.
+
+.. figure:: assets/iris-actions-secret.png
+ :alt: Iris Actions Secret
+ :align: center
+ :width: 75%
+
+4. Complete the ``Actions secrets / New secret`` form for the ``App ID``:
+
+ * Set the ``Name`` field to be ``AUTH_APP_ID``.
+ * Set the ``Value`` field to be the numerical ``iris-actions`` GitHub ``App ID``.
+ See :ref:`here `.
+ * Click the ``Add secret`` button.
+
+5. Click the ``New repository secret`` button again, and complete the form
+ for the ``OpenSSL PEM``:
+
+ * Set the ``Name`` field to be ``AUTH_APP_PRIVATE_KEY``.
+ * Set the ``Value`` field to be the entire contents of the ``OpenSSL PEM`` file.
+ See :ref:`here `.
+ * Click the ``Add secret`` button.
+
+A summary of the newly created `SciTools/iris`_ repository secrets is now available:
+
+.. figure:: assets/iris-secrets-created.png
+ :alt: Iris Secrets created
+ :align: center
+ :width: 75%
+
+
+.. _use gha:
+
+Use GitHub App
+^^^^^^^^^^^^^^
+
+The following example workflow shows how to use the `github-app-token`_ GHA
+to generate a token for use with the `create-pull-request`_ GHA:
+
+.. figure:: assets/gha-token-example.png
+ :alt: GitHub Action token example
+ :align: center
+ :width: 50%
+
+
+.. _GitHub App: https://docs.github.com/en/developers/apps/building-github-apps/creating-a-github-app
+.. _SciTools/iris: https://github.com/SciTools/iris
+.. _conda-lock: https://github.com/conda-incubator/conda-lock
+.. _create-pull-request: https://github.com/peter-evans/create-pull-request
+.. _github-app-token: https://github.com/tibdex/github-app-token
+.. _refresh-lockfiles: https://github.com/SciTools/iris/blob/main/.github/workflows/refresh-lockfiles.yml
diff --git a/docs/src/developers_guide/gitwash/LICENSE b/docs/src/developers_guide/gitwash/LICENSE
new file mode 100644
index 0000000000..cd8441c161
--- /dev/null
+++ b/docs/src/developers_guide/gitwash/LICENSE
@@ -0,0 +1,34 @@
+=========
+ LICENSE
+=========
+
+We release the documents under the Creative Commons attribution license:
+https://creativecommons.org/licenses/by/3.0/
+
+We release the code under the simplified BSD license:
+
+Copyright (c) 2010, Matthew Brett
+All rights reserved.
+
+Redistribution and use in source and binary forms, with or without
+modification, are permitted provided that the following conditions are
+met:
+
+* Redistributions of source code must retain the above copyright notice,
+ this list of conditions and the following disclaimer.
+* Redistributions in binary form must reproduce the above copyright
+ notice, this list of conditions and the following disclaimer in the
+ documentation and/or other materials provided with the distribution.
+
+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS
+IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
+TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
+PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED
+TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
+PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
+LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
+NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
+SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
diff --git a/docs/src/developers_guide/gitwash/branch_dropdown.png b/docs/src/developers_guide/gitwash/branch_dropdown.png
new file mode 100644
index 0000000000..6d74f3d643
Binary files /dev/null and b/docs/src/developers_guide/gitwash/branch_dropdown.png differ
diff --git a/docs/iris/src/developers_guide/gitwash/configure_git.rst b/docs/src/developers_guide/gitwash/configure_git.rst
similarity index 88%
rename from docs/iris/src/developers_guide/gitwash/configure_git.rst
rename to docs/src/developers_guide/gitwash/configure_git.rst
index 0e18b666d0..564ae51820 100644
--- a/docs/iris/src/developers_guide/gitwash/configure_git.rst
+++ b/docs/src/developers_guide/gitwash/configure_git.rst
@@ -1,8 +1,10 @@
+.. include:: links.inc
+
.. _configure-git:
-===============
- Configure git
-===============
+=============
+Configure Git
+=============
.. _git-config-basic:
@@ -49,13 +51,13 @@ command::
To set up on another computer, you can copy your ``~/.gitconfig`` file,
or run the commands above.
-In detail
+In Detail
=========
user.name and user.email
------------------------
-It is good practice to tell git_ who you are, for labeling any changes
+It is good practice to tell git_ who you are, for labelling any changes
you make to the code. The simplest way to do this is from the command
line::
@@ -122,7 +124,7 @@ Or from the command line::
.. _fancy-log:
-Fancy log output
+Fancy Log Output
----------------
This is a very nice alias to get a fancy log output; it should go in the
@@ -137,22 +139,18 @@ You use the alias with::
and it gives graph / text output something like this (but with color!)::
* 6d8e1ee - (HEAD, origin/my-fancy-feature, my-fancy-feature) NF - a fancy file (45 minutes ago) [Matthew Brett]
- * d304a73 - (origin/placeholder, placeholder) Merge pull request #48 from hhuuggoo/master (2 weeks ago) [Jonathan Terhorst]
- |\
+ * d304a73 - (origin/placeholder, placeholder) Merge pull request #48 from hhuuggoo/main (2 weeks ago) [Jonathan Terhorst]
+ |\
| * 4aff2a8 - fixed bug 35, and added a test in test_bugfixes (2 weeks ago) [Hugo]
- |/
+ |/
* a7ff2e5 - Added notes on discussion/proposal made during Data Array Summit. (2 weeks ago) [Corran Webster]
- * 68f6752 - Initial implimentation of AxisIndexer - uses 'index_by' which needs to be changed to a call on an Axes object - this is all very sketchy right now. (2 weeks ago) [Corr
- * 376adbd - Merge pull request #46 from terhorst/master (2 weeks ago) [Jonathan Terhorst]
- |\
+ * 68f6752 - Initial implementation of AxisIndexer - uses 'index_by' which needs to be changed to a call on an Axes object - this is all very sketchy right now. (2 weeks ago) [Corr
+ * 376adbd - Merge pull request #46 from terhorst/main (2 weeks ago) [Jonathan Terhorst]
+ |\
| * b605216 - updated joshu example to current api (3 weeks ago) [Jonathan Terhorst]
| * 2e991e8 - add testing for outer ufunc (3 weeks ago) [Jonathan Terhorst]
| * 7beda5a - prevent axis from throwing an exception if testing equality with non-axis object (3 weeks ago) [Jonathan Terhorst]
| * 65af65e - convert unit testing code to assertions (3 weeks ago) [Jonathan Terhorst]
- | * 956fbab - Merge remote-tracking branch 'upstream/master' (3 weeks ago) [Jonathan Terhorst]
- | |\
+ | * 956fbab - Merge remote-tracking branch 'upstream/main' (3 weeks ago) [Jonathan Terhorst]
+ | |\
| |/
-
-Thanks to Yury V. Zaytsev for posting it.
-
-.. include:: links.inc
diff --git a/docs/src/developers_guide/gitwash/development_workflow.rst b/docs/src/developers_guide/gitwash/development_workflow.rst
new file mode 100644
index 0000000000..8545a04308
--- /dev/null
+++ b/docs/src/developers_guide/gitwash/development_workflow.rst
@@ -0,0 +1,245 @@
+.. _development-workflow:
+
+####################
+Development Workflow
+####################
+
+You already have your own forked copy of the `iris`_ repository, by
+following :ref:`forking`. You have :ref:`set-up-fork`. You have configured
+git by following :ref:`configure-git`. Now you are ready for some real work.
+
+Workflow Summary
+================
+
+In what follows we'll refer to the upstream iris ``main`` branch, as
+"trunk".
+
+* Don't use your ``main`` (that is on your fork) branch for development.
+* When you are starting a new set of changes, fetch any changes from trunk,
+ and start a new *feature branch* from that.
+* Make a new branch for each separable set of changes |emdash| "one task, one
+ branch".
+* Name your branch for the purpose of the changes - e.g.
+ ``bugfix-for-issue-14`` or ``refactor-database-code``.
+* If you can possibly avoid it, avoid merging trunk or any other branches into
+ your feature branch while you are working.
+* If you do find yourself merging from trunk, consider :ref:`rebase-on-trunk`
+* Ask on the Iris `GitHub Discussions`_ if you get stuck.
+* Ask for code review!
+
+This way of working helps to keep work well organized, with readable history.
+This in turn makes it easier for project maintainers (that might be you) to see
+what you've done, and why you did it.
+
+See `linux git workflow`_ for some explanation.
+
+.. _update-mirror-trunk:
+
+Update the Mirror of Trunk
+==========================
+
+First make sure you have done :ref:`linking-to-upstream`.
+
+From time to time you should fetch the upstream (trunk) changes from github::
+
+ git fetch upstream
+
+This will pull down any commits you don't have, and set the remote branches to
+point to the right commit. For example, 'trunk' is the branch referred to by
+(remote/branchname) ``upstream/main`` - and if there have been commits since
+you last checked, ``upstream/main`` will change after you do the fetch.
+
+.. _make-feature-branch:
+
+Make a New Feature Branch
+=========================
+
+When you are ready to make some changes to the code, you should start a new
+branch. Branches that are for a collection of related edits are often called
+'feature branches'.
+
+Making an new branch for each set of related changes will make it easier for
+someone reviewing your branch to see what you are doing.
+
+Choose an informative name for the branch to remind yourself and the rest of us
+what the changes in the branch are for. For example ``add-ability-to-fly``, or
+``buxfix-for-issue-42``.
+
+::
+
+ # Update the mirror of trunk
+ git fetch upstream
+ # Make new feature branch starting at current trunk
+ git branch my-new-feature upstream/main
+ git checkout my-new-feature
+
+Generally, you will want to keep your feature branches on your public github_
+fork of `iris`_. To do this, you ``git push`` this new branch up to your
+github repo. Generally (if you followed the instructions in these pages, and by
+default), git will have a link to your github repo, called ``origin``. You push
+up to your own repo on github with::
+
+ git push origin my-new-feature
+
+In git >= 1.7 you can ensure that the link is correctly set by using the
+``--set-upstream`` option::
+
+ git push --set-upstream origin my-new-feature
+
+From now on git will know that ``my-new-feature`` is related to the
+``my-new-feature`` branch in the github repo.
+
+.. _edit-flow:
+
+The Editing Workflow
+====================
+
+Overview
+--------
+
+::
+
+ # hack hack
+ git add my_new_file
+ git commit -am 'NF - some message'
+ git push
+
+In More Detail
+--------------
+
+#. Make some changes
+#. See which files have changed with ``git status``.
+ You'll see a listing like this one::
+
+ # On branch ny-new-feature
+ # Changed but not updated:
+ # (use "git add ..." to update what will be committed)
+ # (use "git checkout -- ..." to discard changes in working directory)
+ #
+ # modified: README
+ #
+ # Untracked files:
+ # (use "git add ..." to include in what will be committed)
+ #
+ # INSTALL
+ no changes added to commit (use "git add" and/or "git commit -a")
+
+#. Check what the actual changes are with ``git diff``.
+#. Add any new files to version control ``git add new_file_name``.
+#. To commit all modified files into the local copy of your repo, do
+ ``git commit -am 'A commit message'``. Note the ``-am`` options to
+ ``commit``. The ``m`` flag just signals that you're going to type a
+ message on the command line. The ``a`` flag will automatically stage
+ all files that have been modified and deleted.
+#. To push the changes up to your forked repo on github, do a ``git
+ push``.
+
+
+Testing Your Changes
+====================
+
+Once you are happy with your changes, work thorough the :ref:`pr_check` and
+make sure your branch passes all the relevant tests.
+
+
+Ask for Your Changes to be Reviewed or Merged
+=============================================
+
+When you are ready to ask for someone to review your code and consider a merge:
+
+#. Go to the URL of your forked repo, say
+ ``https://github.com/your-user-name/iris``.
+#. Use the 'Switch Branches' dropdown menu near the top left of the page to
+ select the branch with your changes:
+
+ .. image:: branch_dropdown.png
+
+#. Click on the 'Pull request' button:
+
+ .. image:: pull_button.png
+
+ Enter a title for the set of changes, and some explanation of what you've
+ done. Say if there is anything you'd like particular attention for - like a
+ complicated change or some code you are not happy with.
+
+ If you don't think your request is ready to be merged, just say so in your
+ pull request message. This is still a good way of getting some preliminary
+ code review.
+
+Some Other Things you Might Want to do
+======================================
+
+Delete a Branch on Github
+-------------------------
+
+::
+
+ git checkout main
+ # delete branch locally
+ git branch -D my-unwanted-branch
+ # delete branch on github
+ git push origin :my-unwanted-branch
+
+Note the colon ``:`` before ``test-branch``. See also:
+https://github.com/guides/remove-a-remote-branch
+
+
+Several People Sharing a Single Repository
+------------------------------------------
+
+If you want to work on some stuff with other people, where you are all
+committing into the same repository, or even the same branch, then just
+share it via github.
+
+First fork iris into your account, as from :ref:`forking`.
+
+Then, go to your forked repository github page, say
+``https://github.com/your-user-name/iris``, select :guilabel:`Settings`,
+:guilabel:`Manage Access` and then :guilabel:`Invite collaborator`.
+
+.. note:: For more information on sharing your repository see the
+ GitHub documentation on `Inviting collaborators`_.
+
+
+.. _Inviting collaborators: https://docs.github.com/en/github/setting-up-and-managing-your-github-user-account/inviting-collaborators-to-a-personal-repository
+
+Now all those people can do::
+
+ git clone git@githhub.com:your-user-name/iris.git
+
+Remember that links starting with ``git@`` use the ssh protocol and are
+read-write; links starting with ``git://`` are read-only.
+
+Your collaborators can then commit directly into that repo with the
+usual::
+
+ git commit -am 'ENH - much better code'
+ git push origin main # pushes directly into your repo
+
+Explore Your Repository
+-----------------------
+
+To see a graphical representation of the repository branches and
+commits::
+
+ gitk --all
+
+To see a linear list of commits for this branch::
+
+ git log
+
+Finally the :ref:`fancy-log` ``lg`` alias will give you a reasonable text-based
+graph of the repository.
+
+
+.. _rebase-on-trunk:
+
+Rebasing on Trunk
+-----------------
+
+For more information please see the
+`official github documentation on git rebase`_.
+
+.. _official github documentation on git rebase: https://docs.github.com/en/github/using-git/about-git-rebase
+
+.. include:: links.inc
diff --git a/docs/src/developers_guide/gitwash/forking.rst b/docs/src/developers_guide/gitwash/forking.rst
new file mode 100644
index 0000000000..baeb243c86
--- /dev/null
+++ b/docs/src/developers_guide/gitwash/forking.rst
@@ -0,0 +1,36 @@
+.. include:: links.inc
+
+.. _forking:
+
+===================================
+Making Your own Copy (fork) of Iris
+===================================
+
+You need to do this only once. The instructions here are very similar
+to the instructions at https://help.github.com/forking/, please see
+that page for more detail. We're repeating some of it here just to give the
+specifics for the `Iris`_ project, and to suggest some default names.
+
+
+Set up and Configure a Github Account
+=====================================
+
+If you don't have a github account, go to the github page, and make one.
+
+You then need to configure your account to allow write access, see
+the `generating ssh keys for GitHub`_ help on `github help`_.
+
+
+Create Your own Forked Copy of Iris
+===================================
+
+#. Log into your github account.
+#. Go to the `Iris`_ github home at `Iris github`_.
+#. Click on the *fork* button:
+
+ .. image:: forking_button.png
+
+ Now, after a short pause, you should find yourself at the home page for
+ your own forked copy of `Iris`_.
+
+
diff --git a/docs/src/developers_guide/gitwash/forking_button.png b/docs/src/developers_guide/gitwash/forking_button.png
new file mode 100644
index 0000000000..1eb21051d1
Binary files /dev/null and b/docs/src/developers_guide/gitwash/forking_button.png differ
diff --git a/docs/src/developers_guide/gitwash/git_intro.rst b/docs/src/developers_guide/gitwash/git_intro.rst
new file mode 100644
index 0000000000..dfb64da872
--- /dev/null
+++ b/docs/src/developers_guide/gitwash/git_intro.rst
@@ -0,0 +1,15 @@
+.. include:: links.inc
+
+Introduction
+============
+
+These pages describe a git_ and github_ workflow for the `Iris`_
+project.
+
+This is not a comprehensive git reference, it's just a workflow for our
+own project. It's tailored to the github hosting service. You may well
+find better or quicker ways of getting stuff done with git, but these
+should get you started.
+
+.. tip:: Please see the official `git documentation`_ for a complete list of
+ git **commands** and **cheat sheets**.
\ No newline at end of file
diff --git a/docs/src/developers_guide/gitwash/git_links.inc b/docs/src/developers_guide/gitwash/git_links.inc
new file mode 100644
index 0000000000..3ced13703f
--- /dev/null
+++ b/docs/src/developers_guide/gitwash/git_links.inc
@@ -0,0 +1,18 @@
+.. This (-*- rst -*-) format file contains commonly used link targets
+ and name substitutions. It may be included in many files,
+ therefore it should only contain link targets and name
+ substitutions. Try grepping for "^\.\. _" to find plausible
+ candidates for this list.
+
+.. NOTE: reST targets are
+ __not_case_sensitive__, so only one target definition is needed for
+ nipy, NIPY, Nipy, etc...
+
+.. _git: https://git-scm.com/
+.. _github: https://github.com
+.. _github help: https://help.github.com
+.. _git documentation: https://git-scm.com/docs
+
+.. _linux git workflow: https://www.mail-archive.com/dri-devel@lists.sourceforge.net/msg39091.html
+
+.. |emdash| unicode:: U+02014
diff --git a/docs/src/developers_guide/gitwash/index.rst b/docs/src/developers_guide/gitwash/index.rst
new file mode 100644
index 0000000000..3cde622583
--- /dev/null
+++ b/docs/src/developers_guide/gitwash/index.rst
@@ -0,0 +1,14 @@
+.. _using-git:
+
+Working With Iris Source Code
+=============================
+
+.. toctree::
+ :maxdepth: 2
+
+ git_intro
+ forking
+ set_up_fork
+ configure_git
+ development_workflow
+
diff --git a/docs/src/developers_guide/gitwash/links.inc b/docs/src/developers_guide/gitwash/links.inc
new file mode 100644
index 0000000000..1d67f20d3a
--- /dev/null
+++ b/docs/src/developers_guide/gitwash/links.inc
@@ -0,0 +1,3 @@
+.. include:: ../../common_links.inc
+
+.. include:: git_links.inc
diff --git a/docs/src/developers_guide/gitwash/pull_button.png b/docs/src/developers_guide/gitwash/pull_button.png
new file mode 100644
index 0000000000..a088e19e95
Binary files /dev/null and b/docs/src/developers_guide/gitwash/pull_button.png differ
diff --git a/docs/src/developers_guide/gitwash/set_up_fork.rst b/docs/src/developers_guide/gitwash/set_up_fork.rst
new file mode 100644
index 0000000000..5318825488
--- /dev/null
+++ b/docs/src/developers_guide/gitwash/set_up_fork.rst
@@ -0,0 +1,70 @@
+.. include:: links.inc
+
+.. _set-up-fork:
+
+================
+Set up Your Fork
+================
+
+First you follow the instructions for :ref:`forking`.
+
+Overview
+========
+
+::
+
+ git clone git@github.com:your-user-name/iris.git
+ cd iris
+ git remote add upstream git@github.com/SciTools/iris.git
+
+In Detail
+=========
+
+Clone Your Fork
+---------------
+
+#. Clone your fork to the local computer with ``git clone
+ git@github.com:your-user-name/iris.git``
+#. Change directory to your new repo: ``cd iris``. Then
+ ``git branch -a`` to show you all branches. You'll get something
+ like::
+
+ * main
+ remotes/origin/main
+
+ This tells you that you are currently on the ``main`` branch, and
+ that you also have a ``remote`` connection to ``origin/main``.
+ What remote repository is ``remote/origin``? Try ``git remote -v`` to
+ see the URLs for the remote. They will point to your github fork.
+
+ Now you want to connect to the upstream `Iris github`_ repository, so
+ you can merge in changes from trunk.
+
+.. _linking-to-upstream:
+
+Linking Your Repository to the Upstream Repo
+--------------------------------------------
+
+::
+
+ cd iris
+ git remote add upstream git://github.com/SciTools/iris.git
+
+``upstream`` here is just the arbitrary name we're using to refer to the
+main `Iris`_ repository at `Iris github`_.
+
+Note that we've used ``git://`` for the URL rather than ``git@``. The
+``git://`` URL is read only. This means we that we can't accidentally
+(or deliberately) write to the upstream repo, and we are only going to
+use it to merge into our own code.
+
+Just for your own satisfaction, show yourself that you now have a new
+'remote', with ``git remote -v``, giving you something like::
+
+ upstream git://github.com/SciTools/iris.git (fetch)
+ upstream git://github.com/SciTools/iris.git (push)
+ origin git@github.com:your-user-name/iris.git (fetch)
+ origin git@github.com:your-user-name/iris.git (push)
+
+
+
diff --git a/docs/src/developers_guide/propose_changes.png b/docs/src/developers_guide/propose_changes.png
new file mode 100755
index 0000000000..d4e367ce1d
Binary files /dev/null and b/docs/src/developers_guide/propose_changes.png differ
diff --git a/docs/src/developers_guide/pull_request.png b/docs/src/developers_guide/pull_request.png
new file mode 100755
index 0000000000..bdc0698f54
Binary files /dev/null and b/docs/src/developers_guide/pull_request.png differ
diff --git a/docs/src/developers_guide/release.rst b/docs/src/developers_guide/release.rst
new file mode 100644
index 0000000000..c7ce230204
--- /dev/null
+++ b/docs/src/developers_guide/release.rst
@@ -0,0 +1,289 @@
+.. include:: ../common_links.inc
+
+.. _iris_development_releases:
+
+Releases
+========
+
+A release of Iris is a `tag on the SciTools/Iris`_ Github repository.
+
+Below is :ref:`iris_development_releases_steps`, followed by some prose on the
+main areas that constitute the release.
+
+
+.. _iris_development_releases_steps:
+
+How to Create an Iris Release
+-----------------------------
+
+The step-by-step process is walked-through by a script at:
+``/tools/release_do_nothing.py``, and also available here:
+:doc:`release_do_nothing`.
+
+
+.. _release_manager:
+
+Release Manager
+---------------
+
+A Release Manager will be nominated for each release of Iris. This role involves:
+
+* deciding which features and bug fixes should be included in the release
+* managing the `GitHub Projects`_ board for the release
+* using :discussion:`GitHub Discussion releases category `
+ for documenting intent and capturing any
+ discussion about the release
+* holding a developer retrospective post release, to look for potential
+ future improvements
+
+The Release Manager will make the release, ensuring that all the steps outlined
+on this page are completed.
+
+
+Versioning
+----------
+
+Iris' version numbers conform to `Semantic Versioning`_ (``MAJOR.MINOR.PATCH``)
+and `PEP 440`_.
+
+Iris uses `setuptools-scm`_ to automatically manage versioning based on Git
+tags. No manual versioning work is required within the files themselves.
+
+
+Release Candidate
+-----------------
+
+Prior to a release, a release candidate tag may be created, marked as a
+pre-release in GitHub, with a tag ending with :literal:`rc` followed by a
+number (0-based), e.g.,:
+
+ :literal:`v1.9.0rc0`
+
+If created, the pre-release shall be available for a minimum of 2 weeks
+prior to the release being cut. However a 4 week period should be the goal
+to allow user groups to be notified of the existence of the pre-release and
+encouraged to test the functionality.
+
+A pre-release is expected for a major or minor release, but not for a
+patch release.
+
+If new features are required for a release after a release candidate has been
+cut, a new pre-release shall be issued first.
+
+Release candidates are made available as a conda package on the
+`conda-forge Anaconda channel`_ using the `rc_iris`_ label. This is achieved via
+the `conda-forge iris-feedstock`_ following `CFEP-05`_. For further information
+see the `conda-forge User Documentation`_.
+
+
+Patch Releases
+--------------
+
+Patch releases may be implemented to fix problems with previous major or minor
+releases. E.g. ``v1.9.1`` to fix a problem in ``v1.9.0``, both being part of
+the ``v1.9`` series.
+
+New features shall not be included in a patch release, these are for bug fixes.
+
+A patch release does not require a release candidate, but the rest of the
+release process is to be followed.
+
+
+Before Release
+--------------
+
+Deprecations
+~~~~~~~~~~~~
+
+Any behaviour which has been deprecated for the correct number of
+previous releases is now finally changed. More detail, including the correct
+number of releases, is in :ref:`iris_development_deprecations`.
+
+Standard Names
+~~~~~~~~~~~~~~
+
+The file ``etc/cf-standard-name-table.xml`` is updated to the latest CF standard names,
+from the `latest CF standard names`_.
+( This is used during build to automatically generate the sourcefile
+``lib/iris/std_names.py``. )
+
+
+The Release
+-----------
+
+Release Branch
+~~~~~~~~~~~~~~
+
+Once the features intended for the release are on ``main``, a release branch
+should be created, in the ``SciTools/iris`` repository. This will have the name:
+
+ :literal:`v{major release number}.{minor release number}.x`
+
+for example:
+
+ :literal:`v1.9.x`
+
+This branch shall be used to finalise the release details in preparation for
+the release candidate.
+
+Changes for a **patch release** should target to the same release branch as the
+rest of the series. For example, a fix
+for a problem with the ``v1.9.0`` release will be merged into ``v1.9.x`` release
+branch, and then released with the tag ``v1.9.1``.
+
+Documentation
+~~~~~~~~~~~~~
+
+The documentation should include a dedicated What's New file for this release
+series (e.g. ``v1.9.rst``), incorporating all of the What's New entries for the release.
+This content should be reviewed and adapted as required, including highlights
+at the top of the What's New document.
+
+What's New entries for **patch releases** should be added to the existing file
+for that release series (e.g. ``v1.9.1`` section in the ``v1.9.rst`` file).
+
+A template for What's New formatting can be found in the
+``docs/src/whatsnew/latest.rst.template`` file.
+
+
+Tagging
+~~~~~~~
+
+Once all checks are complete, the release is published from the release
+branch - via the GitHub release functionality in the ``SciTools/iris``
+repository - which simultaneously creates a Git tag for the release.
+
+
+Post Release
+------------
+
+PyPI
+~~~~
+Iris is available on PyPI as ``scitools-iris``.
+
+Iris' Continuous-Integration (CI) includes the automatic building and publishing of
+PyPI artifacts in a dedicated GitHub Action.
+
+Legacy manual instructions are appended to this page for reference purposes
+(:ref:`update_pypi`)
+
+conda-forge
+~~~~~~~~~~~
+
+Iris is available on conda-forge as ``iris``.
+
+This is managed via the the Iris conda recipe on the
+`conda-forge iris-feedstock`_, which is updated after the release is cut on
+GitHub, followed by automatic build and publish of the
+conda package on the `conda-forge Anaconda channel`_.
+
+Announcement
+~~~~~~~~~~~~
+
+Iris uses Twitter (`@scitools_iris`_) to announce new releases, as well as any
+internal message boards that are accessible (e.g. at the UK Met Office).
+Announcements usually include a highlighted feature to hook readers' attention.
+
+Citation
+~~~~~~~~
+
+``docs/src/userguide/citation.rst`` is updated to include
+the latest [non-release-candidate] version, date and `Zenodo DOI`_
+of the new release. Ideally this would be updated before the release, but
+the DOI for the new version is only available once the release has been
+created in GitHub.
+
+Merge Back
+~~~~~~~~~~
+
+After any release is published, **including patch releases**, the changes from the
+release branch should be merged back onto the ``SciTools/iris`` ``main`` branch.
+
+
+Appendices
+----------
+
+.. _update_pypi:
+
+Updating PyPI Manually
+~~~~~~~~~~~~~~~~~~~~~~
+
+.. note::
+
+ As part of our Continuous-Integration (CI), the building and publishing of
+ PyPI artifacts is now automated by a dedicated GitHub Action.
+
+ The following instructions **no longer** require to be performed manually,
+ but remain part of the documentation for reference purposes only.
+
+Update the `scitools-iris`_ project on PyPI with the latest Iris release.
+
+To do this perform the following steps.
+
+Create a conda environment with the appropriate conda packages to build the
+source distribution (``sdist``) and pure Python wheel (``bdist_wheel``)::
+
+ > conda create -n iris-pypi -c conda-forge --yes build twine
+ > . activate iris-pypi
+
+Checkout the appropriate Iris ```` tag from the appropriate ````.
+For example, to checkout tag ``v1.0`` from ``upstream``::
+
+ > git fetch upstream --tags
+ > git checkout v1.0
+
+Build the source distribution and wheel from the Iris root directory::
+
+ > python -m build
+
+This ``./dist`` directory should now be populated with the source archive
+``.tar.gz`` file, and built distribution ``.whl`` file.
+
+Check that the package description will render properly on PyPI for each
+of the built artifacts::
+
+ > python -m twine check dist/*
+
+To list and check the contents of the binary wheel::
+
+ > python -m zipfile --list dist/*.whl
+
+If all seems well, sufficient maintainer privileges will be required to
+upload these artifacts to `scitools-iris`_ on PyPI::
+
+ > python -m twine upload --repository-url https://upload.pypi.org/legacy/ dist/*
+
+Ensure that the artifacts are successfully uploaded and available on
+`scitools-iris`_ before creating a conda test environment to install Iris
+from PyPI::
+
+ > conda deactivate
+ > conda env create --file ./requirements/iris.yml
+ > . activate iris-dev
+ > python -m pip install --no-deps scitools-iris
+
+For further details on how to test Iris, see :ref:`developer_running_tests`.
+
+.. seealso::
+
+ For further information on packaging and uploading a project to PyPI, please
+ refer to `Generating Distribution Archives`_ and `Packaging Your Project`_.
+
+.. _SciTools/iris: https://github.com/SciTools/iris
+.. _tag on the SciTools/Iris: https://github.com/SciTools/iris/releases
+.. _conda-forge Anaconda channel: https://anaconda.org/conda-forge/iris
+.. _conda-forge iris-feedstock: https://github.com/conda-forge/iris-feedstock
+.. _CFEP-05: https://github.com/conda-forge/cfep/blob/master/cfep-05.md
+.. _conda-forge User Documentation: https://conda-forge.org/docs/user/00_intro.html
+.. _Active Versions: https://readthedocs.org/projects/scitools-iris/versions/
+.. _Editing v3.0.0rc0: https://readthedocs.org/dashboard/scitools-iris/version/v3.0.0rc0/edit
+.. _rc_iris: https://anaconda.org/conda-forge/iris/labels
+.. _Generating Distribution Archives: https://packaging.python.org/tutorials/packaging-projects/#generating-distribution-archives
+.. _Packaging Your Project: https://packaging.python.org/guides/distributing-packages-using-setuptools/#packaging-your-project
+.. _latest CF standard names: https://cfconventions.org/Data/cf-standard-names/current/src/cf-standard-name-table.xml
+.. _setuptools-scm: https://github.com/pypa/setuptools_scm
+.. _Semantic Versioning: https://semver.org/
+.. _PEP 440: https://peps.python.org/pep-0440/
+.. _@scitools_iris: https://twitter.com/scitools_iris
+.. _GitHub Projects: https://github.com/SciTools/iris/projects
+.. _Zenodo DOI: https://doi.org/10.5281/zenodo.595182
diff --git a/docs/src/developers_guide/release_do_nothing.rst b/docs/src/developers_guide/release_do_nothing.rst
new file mode 100644
index 0000000000..1f72827184
--- /dev/null
+++ b/docs/src/developers_guide/release_do_nothing.rst
@@ -0,0 +1,12 @@
+:orphan:
+
+Release Do-Nothing Script
+-------------------------
+
+Rendered from the original ``/tools/release_do_nothing.py``.
+
+`Read more about do-nothing scripts
+`_ to
+parallelise the function calls. It's important that Dask is given the freedom
+to break the task down in an efficient manner - the function that is mapped
+across the bag should only load a single file, and the bag itself can
+iterate through the list of files. Here's the restructured script::
+
+ import glob
+ import multiprocessing
+ import os
+
+ import dask
+ import dask.bag as db
+ import iris
+
+ def callback(cube, field, fname):
+ if field.sections[5]['bitsPerValue'] == 0:
+ raise iris.exceptions.IgnoreCubeException
+ if field.sections[4]['parameterNumber'] == 20:
+ raise iris.exceptions.IgnoreCubeException
+ elif field.sections[4]['parameterNumber'] == 234:
+ cube.long_name = 'Icing Severity'
+
+ def func(fname):
+ return iris.load_cube(fname, callback=callback)
+
+ fpaths = list(glob.glob('20190416/*t18z*f???'))
+
+ # Determine the number of processors visible ..
+ cpu_count = multiprocessing.cpu_count()
+
+ # .. or as given by slurm allocation.
+ # Only relevant when using Slurm for job scheduling
+ if 'SLURM_NTASKS' in os.environ:
+ cpu_count = os.environ['SLURM_NTASKS']
+
+ # Do not exceed the number of CPUs available, leaving 1 for the system.
+ num_workers = cpu_count - 1
+ print('Using {} workers from {} CPUs...'.format(num_workers, cpu_count))
+
+ # Now do the parallel load.
+ with dask.config.set(num_workers=num_workers):
+ bag = db.from_sequence(fpaths).map(func)
+ cubes = iris.cube.CubeList(bag.compute()).merge()
+
+This achieves approximately a 10-fold improvement if enough CPUs are
+available to have one per file. See this benchmarking:
+
++---------------+-----------------------+---------------+---------------+
+| Machine | CPUs Available | CPUs Used | Time Taken |
++===============+=======================+===============+===============+
+| A | 4 | 3 | 4m 05s |
+| | +---------------+---------------+
+| | | 4 | 3m 22s |
++---------------+-----------------------+---------------+---------------+
+| B | 8 | 1 | 9m 10s |
+| | +---------------+---------------+
+| | | 7 | 2m 35s |
+| | +---------------+---------------+
+| | | 8 | 2m 20s |
++---------------+-----------------------+---------------+---------------+
+
+
+.. _examples_bags_greed_profile:
+
+3.3 Profiling
+^^^^^^^^^^^^^
+1m 10s is still a surprisingly long time. When faced with a mystery like
+this it is helpful to profile the script to see if there are any steps that
+are taking more time than we would expect. For this we use a tool called
+`kapture `_ to produce a
+flame chart visualising the time spent performing each call:
+
+.. image:: images/grib-bottleneck.png
+ :width: 1000
+ :align: center
+
+From this we see that 96% of the runtime is taken by this call::
+
+ res = gribapi.grib_get_array(self._message_id, key)
+
+This is the call being used during the ``callback`` function when it uses
+GRIB messages to filter out cubes with certain unwanted properties.
+
+3.4 Improving GRIB Key Handling
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+Even with parallelisation, we are still limited by the time it takes to run
+a single instance of a function. This is going to become much more important
+when running 7000 files instead of 11, since there will be nowhere near
+enough CPUs even on a large multi-processing system, meaning each CPU will be running many instances
+of the function. **Parallelisation can only go so far to solving speed issues** --
+it's effectively the 'brute force' method.
+
+:ref:`examples_bags_greed_profile` showed us where the major bottleneck is. To improve efficiency
+we can re-write the script to filter on GRIB messages *before* converting
+the GRIB file to a cube::
+
+ import dask
+ import dask.bag as db
+ import glob
+ import iris
+ import multiprocessing
+ import os
+
+ def func(fname):
+ import iris
+ from iris_grib import load_pairs_from_fields
+ from iris_grib.message import GribMessage # perform GRIB message level filtering...
+ filtered_messages = []
+ for message in GribMessage.messages_from_filename(fname):
+ if (message.sections[5]['bitsPerValue'] != 0 and
+ message.sections[4]['parameterNumber'] == 234):
+ filtered_messages.append(message) # now convert the messages to cubes...
+ cubes = [cube for cube, message in load_pairs_from_fields(filtered_messages)]
+ return iris.cube.CubeList(cubes).merge_cube()
+
+ fpaths = list(glob.glob('/scratch/frcz/ICING/GFS_DATA/20190416/*t18z*f???'))
+ cpu_count = multiprocessing.cpu_count()
+
+ # Only relevant when using Slurm for job scheduling
+ if 'SLURM_NTASKS' in os.environ:
+ cpu_count = os.environ['SLURM_NTASKS']
+
+ num_workers = cpu_count - 1
+
+ print('Using {} workers from {} CPUs...'.format(num_workers, cpu_count))
+ with dask.config.set(num_workers=num_workers):
+ bag = db.from_sequence(fpaths).map(func)
+ cubes = iris.cube.CubeList(bag.compute())
+
+This achieves a significant performance improvement - more than twice as
+fast as the previous benchmarks:
+
++---------------+-----------------------+---------------+---------------+-----------+
+| Machine | CPUs Available | CPUs Used | Previous Time | New Time |
++===============+=======================+===============+===============+===========+
+| Example | 8 | 7 | 2m 35s | 1m 05s |
+| | +---------------+---------------+-----------+
+| | | 8 | 2m 20s | 1m 03s |
++---------------+-----------------------+---------------+---------------+-----------+
+
+3.5 Managing External Factors
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+The speed will still need to be further improved before we can process 7000
+files. The main gains we can achieve are by making sure it is **only Dask**
+that manages multi-processing - if multi-processing is coming from more
+than one place there are predictable clashes.
+
+First, NumPy must be prevented from performing it's own multi-processing by
+adding the following **before** ``import numpy`` is called. You can read more
+about this in :ref:`numpy_threads`.
+
+::
+
+ import os
+
+ os.environ["OMP_NUM_THREADS"] = "1"
+ os.environ["OPENBLAS_NUM_THREADS"] = "1"
+ os.environ["MKL_NUM_THREADS"] = "1"
+ os.environ["VECLIB_MAXIMUM_THREADS"] = "1"
+ os.environ["NUMEXPR_NUM_THREADS"] = "1"
+
+Lastly, if you are using SLURM on the computing cluster then SLURM must be configured to prevent it
+optimising the number of cores necessary for the job. See the SLURM commands
+below, to be added before running the python script. It's important that
+``ntasks`` matches the number of CPUs specified in the python script. You
+can read more about these points in :ref:`multi-pro_slurm`.
+
+::
+
+ #SBATCH --ntasks=12
+ #SBATCH --ntasks-per-core=1
+
+This has all been based on a real example. Once all the above had been set
+up correctly, the completion time had dropped from an estimated **55 days**
+to **less than 1 day**.
+
+3.6 Lessons
+^^^^^^^^^^^
+* Dask isn't a magic switch - it's important to write your script so that
+ there is a way to create small sub-tasks. In this case by providing
+ dask.bag with the file list and the function separated
+* Parallelism is not the only performance improvement to try - the script
+ will still be slow if the individual function is slow
+* All multi-processing needs to be managed by Dask. Several other factors
+ may introduce multi-processing and these need to be configured not to
diff --git a/docs/src/further_topics/dask_best_practices/dask_parallel_loop.rst b/docs/src/further_topics/dask_best_practices/dask_parallel_loop.rst
new file mode 100644
index 0000000000..836503314c
--- /dev/null
+++ b/docs/src/further_topics/dask_best_practices/dask_parallel_loop.rst
@@ -0,0 +1,169 @@
+.. _examples_parallel_loop:
+
+2. Parallelising a Loop of Multiple Calls to a Third Party Library
+------------------------------------------------------------------
+
+Whilst Iris does provide extensive functionality for performing statistical and
+mathematical operations on your data, it is sometimes necessary to use a third
+party library.
+
+The following example describes a real world use case of how to parallelise
+multiple calls to a third party library using dask bags.
+
+2.1 The Problem - Parallelising a Loop
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+In this particular example, the user is calculating a sounding parcel for each
+column in their dataset. The cubes that are used are of shape::
+
+ (model_level_number: 20; grid_latitude: 1536; grid_longitude: 1536)
+
+As a sounding is calculated for each column, this means there are 1536x1536
+individual calculations.
+
+In Python, it is common practice to vectorize the calculation of for loops.
+Vectorising is done by using NumPy to operate on the whole array at once rather
+than a single element at a time. Unfortunately, not all operations are
+vectorisable, including the calculation in this example, and so we look to
+other methods to improve the performance.
+
+2.2 Original Code with Loop
+^^^^^^^^^^^^^^^^^^^^^^^^^^^
+We start out by loading cubes of pressure, temperature, dewpoint temperature and height::
+
+ import iris
+ import numpy as np
+ from skewt import SkewT as sk
+
+ pressure = iris.load_cube('a.press.19981109.pp')
+ temperature = iris.load_cube('a.temp.19981109.pp')
+ dewpoint = iris.load_cube('a.dewp.19981109.pp')
+ height = iris.load_cube('a.height.19981109.pp')
+
+We set up the NumPy arrays we will be filling with the output data::
+
+ output_arrays = [np.zeros(pressure.shape[0]) for _ in range(6)]
+ cape, cin, lcl, lfc, el, tpw = output_data
+
+Now we loop over the columns in the data to calculate the soundings::
+
+ for y in range(nlim):
+ for x in range(nlim):
+ mydata = {'pres': pressure[:, y, x],
+ 'temp': temperature[:, y, x],
+ 'dwpt': dewpoint[:, y, x],
+ 'hght': height[:, y, x]}
+
+ # Calculate the sounding with the selected column of data.
+ S = sk.Sounding(soundingdata=mydata)
+ try:
+ startp, startt, startdp, type_ = S.get_parcel(parcel_def)
+ P_lcl, P_lfc, P_el, CAPE, CIN = S.get_cape(
+ startp, startt, startdp, totalcape='tot')
+ TPW = S.precipitable_water()
+ except:
+ P_lcl, P_lfc, P_el, CAPE, CIN, TPW = [
+ np.ma.masked for _ in range(6)]
+
+ # Fill the output arrays with the results
+ cape[y,x] = CAPE
+ cin[y,x] = CIN
+ lcl[y,x] = P_lcl
+ lfc[y,x] = P_lfc
+ el[y,x] = P_el
+ tpw[y,x] = TPW
+
+2.3 Profiling the Code with Kapture
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+Kapture is a useful statistical profiler. For more information see `the
+Kapture repo `_.
+
+Results below:
+
+.. image:: images/loop_third_party_kapture_results.png
+ :width: 1000
+ :align: center
+
+As we can see above, (looking at the highlighted section of the red bar) it spends most of the time in the call to ::
+
+ S.get_parcel(parcel_def)
+
+As there are over two million columns in the data, we would greatly benefit
+from parallelising this work.
+
+2.4 Parallelising with Dask Bags
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+Dask bags are collections of Python objects that you can map a computation over
+in a parallel manner.
+
+For more information about dask bags, see the `Dask Bag Documentation
+`_.
+
+Dask bags work best with lightweight objects, so we will create a collection of
+indices into our data arrays.
+
+First, we put the loop into a function that takes a slice object to index the
+appropriate section of the array.::
+
+ def calculate_sounding(y_slice):
+ for y in range(y_slice.stop-y_slice.start):
+ for x in range(nlim):
+ mydata = {'pres': pressure[:, y_slice][:, y, x],
+ 'temp': temperature[:, y_slice][:, y, x],
+ 'dwpt': dewpoint[:, y_slice][:, y, x],
+ 'hght': height[:, y_slice][:, y, x]}
+
+ # Calculate the sounding with the selected column of data.
+ S = sk.Sounding(soundingdata=mydata)
+ try:
+ startp, startt, startdp, type_ = S.get_parcel(parcel_def)
+ P_lcl, P_lfc, P_el, CAPE, CIN = S.get_cape(
+ startp, startt, startdp, totalcape=total_cape)
+ TPW = S.precipitable_water()
+ except:
+ P_lcl, P_lfc, P_el, CAPE, CIN, TPW = [
+ np.ma.masked for _ in range(6)]
+
+ # Fill the output arrays with the results
+ cape[:, y_slice][y,x] = CAPE
+ cin[:, y_slice][y,x] = CIN
+ lcl[:, y_slice][y,x] = P_lcl
+ lfc[:, y_slice][y,x] = P_lfc
+ el[:, y_slice][y,x] = P_el
+ tpw[:, y_slice][y,x] = TPW
+
+Then we create a dask bag of slice objects that will create multiple partitions
+along the y axis.::
+
+ num_of_workers = 4
+ len_of_y_axis = pressure.shape[1]
+
+ part_loc = [int(loc) for loc in np.floor(np.linspace(0, len_of_y_axis,
+ num_of_workers + 1))]
+
+ dask_bag = db.from_sequence(
+ [slice(part_loc[i], part_loc[i+1]) for i in range(num_of_workers)])
+
+ with dask.config.set(scheduler='processes'):
+ dask_bag.map(calculate_sounding).compute()
+
+When this was run on a machine with 4 workers, a speedup of ~4x was achieved,
+as expected.
+
+Note that if using the processes scheduler this is some extra time spent
+serialising the data to pass it between workers. For more information on the
+different schedulers available in Dask, see `Dask Scheduler Overview
+`_.
+
+For more speed up, it is possible to run the same code on a multi-processing
+system where you will have access to more CPUs.
+
+In this particular example, we are handling multiple numpy arrays and so we use
+dask bags. If working with a single numpy array, it may be more appropriate to
+use Dask Arrays (see `Dask Arrays
+`_ for more information).
+
+
+2.5 Lessons
+^^^^^^^^^^^
+* If possible, dask bags should contain lightweight objects
+* Minimise the number of tasks that are created
diff --git a/docs/src/further_topics/dask_best_practices/dask_pp_to_netcdf.rst b/docs/src/further_topics/dask_best_practices/dask_pp_to_netcdf.rst
new file mode 100644
index 0000000000..28784154b4
--- /dev/null
+++ b/docs/src/further_topics/dask_best_practices/dask_pp_to_netcdf.rst
@@ -0,0 +1,92 @@
+.. _examples_pp_to_ff:
+
+1. Speed up Converting PP Files to NetCDF
+-----------------------------------------
+
+Here is an example of how dask objects can be tuned for better performance.
+
+1.1 The Problem - Slow Saving
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+We have ~300 PP files which we load as follows:
+
+.. code-block:: python
+
+ import iris
+ import glob
+
+ files = glob.glob("pp_files/*.pp")
+ cube = iris.load_cube(files, "mass_fraction_of_ozone_in_air")
+
+Note that loading here may also be parallelised in a similar manner as
+described in :ref:`examples_bags_greed`. Either way, the resulting cube looks
+as follows:
+
+.. code-block:: text
+
+ mass_fraction_of_ozone_in_air / (kg kg-1) (time: 276; model_level_number: 85; latitude: 144; longitude: 192)
+ Dimension coordinates:
+ time x - - -
+ model_level_number - x - -
+ latitude - - x -
+ longitude - - - x
+ Auxiliary coordinates:
+ forecast_period x - - -
+ level_height - x - -
+ sigma - x - -
+ Scalar coordinates:
+ forecast_reference_time: 1850-01-01 00:00:00
+ Attributes:
+ STASH: m01s34i001
+ source: Data from Met Office Unified Model
+ um_version: 10.9
+ Cell methods:
+ mean: time (1 hour)
+
+The cube is then immediately saved as a netCDF file.
+
+.. code-block:: python
+
+ nc_chunks = [chunk[0] for chunk in cube.lazy_data().chunks]
+ iris.save(cube, "outfile.nc", nc_chunks)
+
+This operation was taking longer than expected and we would like to improve
+performance. Note that when this cube is being saved, the data is still lazy,
+data is both read and written at the saving step and is done so in chunks.
+The way this data is divided into chunks can affect performance. By tweaking
+the way these chunks are structured it may be possible to improve performance
+when saving.
+
+
+.. _dask_rechunking:
+
+1.2 Rechunking
+^^^^^^^^^^^^^^
+We may inspect the cube's lazy data before saving:
+
+.. code-block:: python
+
+ # We can access the cubes Dask array
+ lazy_data = cube.lazy_data()
+ # We can find the shape of the chunks
+ # Note that the chunksize of a Dask array is the shape of the chunk
+ # as a tuple.
+ print(lazy_data.chunksize)
+
+Doing so, we find that the chunks currently have the shape::
+
+(1, 1, 144, 192)
+
+This is significantly smaller than the `size which Dask recommends
+`_. Bear in mind that the
+ideal chunk size depends on the platform you are running on (for this example,
+the code is being run on a desktop with 8 CPUs). In this case, we have 23460
+small chunks. We can reduce the number of chunks by rechunking before saving:
+
+.. code-block:: python
+
+ lazy_data = cube.lazy_data()
+ lazy_data = lazy_data.rechunk(1, 85, 144, 192)
+ cube.data = lazy_data
+
+We now have 276 moderately sized chunks. When we try saving again, we find
+that it is approximately 4 times faster, saving in 2m13s rather than 10m33s.
diff --git a/docs/src/further_topics/dask_best_practices/images/grib-bottleneck.png b/docs/src/further_topics/dask_best_practices/images/grib-bottleneck.png
new file mode 100644
index 0000000000..c029d57e5e
Binary files /dev/null and b/docs/src/further_topics/dask_best_practices/images/grib-bottleneck.png differ
diff --git a/docs/src/further_topics/dask_best_practices/images/loop_third_party_kapture_results.png b/docs/src/further_topics/dask_best_practices/images/loop_third_party_kapture_results.png
new file mode 100644
index 0000000000..8f388bb89c
Binary files /dev/null and b/docs/src/further_topics/dask_best_practices/images/loop_third_party_kapture_results.png differ
diff --git a/docs/src/further_topics/dask_best_practices/index.rst b/docs/src/further_topics/dask_best_practices/index.rst
new file mode 100644
index 0000000000..f126427d3f
--- /dev/null
+++ b/docs/src/further_topics/dask_best_practices/index.rst
@@ -0,0 +1,221 @@
+.. _dask best practices:
+
+Dask Best Practices
+*******************
+
+This section outlines some of the best practices when using Dask with Iris. These
+practices involve improving performance through rechunking, making the best use of
+computing clusters and avoiding parallelisation conflicts between Dask and NumPy.
+
+
+.. note::
+
+ Here, we have collated advice and a handful of examples, from the topics most
+ relevant when using Dask with Iris, that we hope will assist users to make
+ the best start when using Dask. It is *not* a fully comprehensive guide
+ encompassing all best practices. You can find more general dask information in the
+ `official Dask Documentation `_, so adding NumPy's 'competing' layer of
+ parallelisation could cause unpredictable performance.
+
+Therefore it is best to prevent NumPy performing its own parallelisation, `a
+suggestion made in Dask's own documentation `_.
+The following commands will ensure this in all scenarios:
+
+in Python...
+
+::
+
+ # Must be run before importing NumPy.
+ import os
+ os.environ["OMP_NUM_THREADS"] = "1"
+ os.environ["OPENBLAS_NUM_THREADS"] = "1"
+ os.environ["MKL_NUM_THREADS"] = "1"
+ os.environ["VECLIB_MAXIMUM_THREADS"] = "1"
+ os.environ["NUMEXPR_NUM_THREADS"] = "1"
+
+or in Linux command line...
+
+::
+
+ export OMP_NUM_THREADS=1
+ export OPENBLAS_NUM_THREADS=1
+ export MKL_NUM_THREADS=1
+ export VECLIB_MAXIMUM_THREADS=1
+ export NUMEXPR_NUM_THREADS=1
+
+
+.. _multi-pro_systems:
+
+Dask on Computing Clusters
+==========================
+
+Dask is well suited for use on computing clusters, but there are some important factors you must be
+aware of. In particular, you will always need to explicitly control parallel
+operation, both in Dask and likewise in NumPy.
+
+
+.. _multi-pro_slurm:
+
+CPU Allocation
+--------------
+
+When running on a computing cluster, unless configured otherwise, Dask will attempt to create
+one parallel 'worker' task for each CPU. However, when using a job scheduler such as Slurm, only *some* of
+these CPUs are actually accessible -- often, and by default, only one. This leads to a serious
+over-commitment unless it is controlled.
+
+So, **whenever Iris is used on a computing cluster, you must always control the number
+of dask workers to a sensible value**, matching the slurm allocation. You do
+this with::
+
+ dask.config.set(num_workers=N)
+
+For an example, see :doc:`dask_bags_and_greed`.
+
+Alternatively, when there is only one CPU allocated, it may actually be more
+efficient to use a "synchronous" scheduler instead, with::
+
+ dask.config.set(scheduler='synchronous')
+
+See the Dask documentation on `Single thread synchronous scheduler
+`_.
+
+
+.. _multi-pro_numpy:
+
+NumPy Threading
+---------------
+
+NumPy also interrogates the visible number of CPUs to multi-thread its operations.
+The large number of CPUs available in a computing cluster will thus cause confusion if NumPy
+attempts its own parallelisation, so this must be prevented. Refer back to
+:ref:`numpy_threads` for more detail.
+
+
+Distributed
+-----------
+
+Even though allocations on a computing cluster are generally restricted to a single node, there
+are still good reasons for using 'dask.distributed' in many cases. See `Single Machine: dask.distributed
+`_ in the Dask documentation.
+
+
+Chunking
+========
+
+Dask breaks down large data arrays into chunks, allowing efficient
+parallelisation by processing several smaller chunks simultaneously. For more
+information, see the documentation on
+`Dask Array `_.
+
+Iris provides a basic chunking shape to Dask, attempting to set the shape for
+best performance. The chunking that is used can depend on the file format that
+is being loaded. See below for how chunking is performed for:
+
+* :ref:`chunking_netcdf`
+* :ref:`chunking_pp_ff`
+
+It can in some cases be beneficial to re-chunk the arrays in Iris cubes.
+For information on how to do this, see :ref:`dask_rechunking`.
+
+
+.. _chunking_netcdf:
+
+NetCDF Files
+------------
+
+NetCDF files can include their own chunking specification. This is either
+specified when creating the file, or is automatically assigned if one or
+more of the dimensions is `unlimited `_.
+Importantly, netCDF chunk shapes are **not optimised for Dask
+performance**.
+
+Chunking can be set independently for any variable in a netCDF file.
+When a netCDF variable uses an unlimited dimension, it is automatically
+chunked: the chunking is the shape of the whole variable, but with '1' instead
+of the length in any unlimited dimensions.
+
+When chunking is specified for netCDF data, Iris will set the dask chunking
+to an integer multiple or fraction of that shape, such that the data size is
+near to but not exceeding the dask array chunk size.
+
+
+.. _chunking_pp_ff:
+
+PP and Fieldsfiles
+------------------
+
+PP and Fieldsfiles contain multiple 2D fields of data. When loading PP or
+Fieldsfiles into Iris cubes, the chunking will automatically be set to a chunk
+per field.
+
+For example, if a PP file contains 2D lat-lon fields for each of the
+85 model level numbers, it will load in a cube that looks as follows::
+
+ (model_level_number: 85; latitude: 144; longitude: 192)
+
+The data in this cube will be partitioned with chunks of shape
+:code:`(1, 144, 192)`.
+
+If the file(s) being loaded contain multiple fields, this can lead to an
+excessive amount of chunks which will result in poor performance.
+
+When the default chunking is not appropriate, it is possible to rechunk.
+:doc:`dask_pp_to_netcdf` provides a detailed demonstration of how Dask can optimise
+that process.
+
+
+Examples
+========
+
+We have written some examples of use cases for using Dask, that come with advice and
+explanations for why and how the tasks are performed the way they are.
+
+If you feel you have an example of a Dask best practice that you think may be helpful to others,
+please share them with us by raising a new `discussion on the Iris repository `: a warnings filter can be applied
+ either from the command line or from within Python.
+- :ref:`What`: a warnings filter accepts
+ various arguments to specify which Warnings are being filtered. Both broad
+ and narrow filters are possible.
+
+.. _warning-filter-application:
+
+**When** a Warnings Filter can be Applied
+-----------------------------------------
+
+- **Command line:** setting the :external+python:envvar:`PYTHONWARNINGS`
+ environment variable.
+- **Command line:** the `python -W `_
+ command line argument.
+- **Within Python:** use :func:`warnings.filterwarnings` .
+
+The :ref:`warning-filter-specificity` section demonstrates using
+:func:`warnings.filterwarnings`, and shows the equivalent **command line**
+approaches.
+
+
+.. _warning-filter-specificity:
+
+**What** Warnings will be Filtered
+----------------------------------
+
+.. note::
+
+ For all of these examples we are using the
+ :class:`~warnings.catch_warnings` context manager to ensure any changes to
+ settings are temporary.
+
+ This should always work fine for the ``ignore``
+ warning filter action, but note that some of the other actions
+ may not behave correctly with all Iris operations, as
+ :class:`~warnings.catch_warnings` is not thread-safe (e.g. using the
+ ``once`` action may cause 1 warning per chunk of lazy data).
+
+Specific Warnings
+~~~~~~~~~~~~~~~~~
+
+**When you do not want a specific warning, but still want all others.**
+
+You can target specific Warning messages, e.g.
+
+.. doctest:: filtering_warnings
+
+ >>> with warnings.catch_warnings():
+ ... warnings.filterwarnings("ignore", message="Discarding false_easting")
+ ... my_operation()
+ ...
+ iris/coord_systems.py:445: IrisUserWarning: Setting inverse_flattening does not affect other properties of the GeogCS object. To change other properties set them explicitly or create a new GeogCS instance.
+ warnings.warn(wmsg, category=iris.warnings.IrisUserWarning)
+
+::
+
+ python -W ignore:"Discarding false_easting"
+ export PYTHONWARNINGS=ignore:"Discarding false_easting"
+
+----
+
+Or you can target Warnings raised by specific lines of specific modules, e.g.
+
+.. doctest:: filtering_warnings
+
+ >>> with warnings.catch_warnings():
+ ... warnings.filterwarnings("ignore", module="iris.coord_systems", lineno=445)
+ ... my_operation()
+ ...
+ iris/coord_systems.py:771: IrisDefaultingWarning: Discarding false_easting and false_northing that are not used by Cartopy.
+ warnings.warn(
+
+::
+
+ python -W ignore:::iris.coord_systems:445
+ export PYTHONWARNINGS=ignore:::iris.coord_systems:445
+
+Warnings from a Common Source
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+**When you do not want ANY warnings raised by a module, or collection of
+modules.**
+
+E.g. filtering the ``coord_systems`` module:
+
+.. doctest:: filtering_warnings
+
+ >>> with warnings.catch_warnings():
+ ... warnings.filterwarnings("ignore", module="iris.coord_systems")
+ ... my_operation()
+
+::
+
+ python -W ignore:::iris.coord_systems
+ export PYTHONWARNINGS=ignore:::iris.coord_systems
+
+----
+
+If using :func:`warnings.filterwarnings` , you can also use partial
+definitions. The below example will ``ignore`` all Warnings from ``iris`` as a
+whole.
+
+.. doctest:: filtering_warnings
+
+ >>> with warnings.catch_warnings():
+ ... warnings.filterwarnings("ignore", module="iris")
+ ... my_operation()
+
+The above 'partial' filter is not available with the command line approaches.
+
+Warnings of a Common Type
+~~~~~~~~~~~~~~~~~~~~~~~~~
+
+**When you do not want any Warnings of the same nature, from anywhere in the
+code you are calling.**
+
+The below example will ``ignore`` any
+:class:`~iris.warnings.IrisDefaultingWarning` that gets raised by *any*
+module during execution:
+
+.. doctest:: filtering_warnings
+
+ >>> with warnings.catch_warnings():
+ ... warnings.filterwarnings(
+ ... "ignore",
+ ... category=iris.warnings.IrisDefaultingWarning
+ ... )
+ ... my_operation()
+ ...
+ iris/coord_systems.py:445: IrisUserWarning: Setting inverse_flattening does not affect other properties of the GeogCS object. To change other properties set them explicitly or create a new GeogCS instance.
+ warnings.warn(wmsg, category=iris.warnings.IrisUserWarning)
+
+----
+
+Using :class:`~iris.warnings.IrisUserWarning` in the filter will ``ignore``
+both Warnings, since :class:`~iris.warnings.IrisDefaultingWarning` subclasses
+:class:`~iris.warnings.IrisUserWarning` :
+
+.. doctest:: filtering_warnings
+
+ >>> with warnings.catch_warnings():
+ ... warnings.filterwarnings(
+ ... "ignore",
+ ... category=iris.warnings.IrisUserWarning
+ ... )
+ ... my_operation()
+
+----
+
+The command line approaches can only handle the built-in Warning
+categories (`cpython#66733`_)::
+
+ python -W ignore::UserWarning
+ export PYTHONWARNINGS=ignore::UserWarning
+
+----
+
+There are several built-in Python warning categories that can be used here
+(:class:`DeprecationWarning` being a popular example, see
+:external+python:mod:`warnings` for more). Since Iris has
+so many different warnings that might be raised, Iris subclasses
+:class:`UserWarning` to :class:`~iris.warnings.IrisUserWarning`, which itself
+has **many** specialised subclasses. These subclasses exist to give you more
+granularity in your warning filtering; you can see the full list by
+viewing the :mod:`iris.warnings` module.
+
+.. attention::
+
+ If you have ideas for adding/altering Iris' warning categories, please
+ :ref:`get in touch`! The categories exist to
+ make your life easier, and it is simple to make modifications.
+
+
+More Detail
+-----------
+
+Different people use Iris for very different purposes, from quick file
+visualisation to extract-transform-load to statistical analysis. These
+contrasting priorities mean disagreement on which Iris problems can be ignored
+and which are critically important.
+
+For problems that prevent Iris functioning: **Concrete Exceptions** are raised, which
+stop code from running any further - no debate here. For less catastrophic
+problems: **Warnings** are raised,
+which notify you (in ``stderr``) but allow code to continue running. The Warnings are
+there because Iris may **OR may not** function in the way you expect,
+depending on what you need - e.g. a problem might prevent data being saved to
+NetCDF, but statistical analysis will still work fine.
+
+Examples of Iris Warnings
+~~~~~~~~~~~~~~~~~~~~~~~~~
+
+- If you attempt to plot un-bounded point data as a ``pcolormesh``: Iris will
+ guess appropriate bounds around each point so that quadrilaterals can be
+ plotted. This permanently modifies the relevant coordinates, so the you are
+ warned in case downstream operations assume un-bounded coordinates.
+- If you load a NetCDF file where a CF variable references another variable -
+ e.g. ``my_var:coordinates = "depth_var" ;`` - but the referenced variable
+ (``depth_var``) is not in the file: Iris will still construct
+ its data model, but without this reference relationship. You are warned since
+ the file includes an error and the loaded result might therefore not be as
+ expected.
+
+
+.. testcleanup:: filtering_warnings
+
+ warnings.filterwarnings("ignore")
+
+
+.. _cpython#66733: https://github.com/python/cpython/issues/66733
diff --git a/docs/src/further_topics/index.rst b/docs/src/further_topics/index.rst
new file mode 100644
index 0000000000..73ce3d55e7
--- /dev/null
+++ b/docs/src/further_topics/index.rst
@@ -0,0 +1,21 @@
+.. _further_topics_index:
+
+
+Further Topics
+===============
+
+Extra information on specific technical issues.
+
+.. toctree::
+ :maxdepth: 1
+
+ filtering_warnings
+ metadata
+ lenient_metadata
+ lenient_maths
+ um_files_loading
+ missing_data_handling
+ netcdf_io
+ dask_best_practices/index
+ ugrid/index
+ which_regridder_to_use
\ No newline at end of file
diff --git a/docs/src/further_topics/lenient_maths.rst b/docs/src/further_topics/lenient_maths.rst
new file mode 100644
index 0000000000..51f77fb956
--- /dev/null
+++ b/docs/src/further_topics/lenient_maths.rst
@@ -0,0 +1,281 @@
+.. _lenient maths:
+
+Lenient Cube Maths
+******************
+
+This section provides an overview of lenient cube maths. In particular, it explains
+what lenient maths involves, clarifies how it differs from normal or strict cube
+maths, and demonstrates how you can exercise fine control over whether your cube
+maths operations are lenient or strict.
+
+Note that, lenient cube maths is the default behaviour of Iris from version
+``3.0.0``.
+
+
+Introduction
+============
+
+Lenient maths stands somewhat on the shoulders of giants. If you've not already
+done so, you may want to recap the material discussed in the following sections,
+
+- :ref:`cube maths`,
+- :ref:`metadata`,
+- :ref:`lenient metadata`
+
+In addition to this, cube maths leans heavily on the :mod:`~iris.common.resolve`
+module, which provides the necessary infrastructure required by Iris to analyse
+and combine each :class:`~iris.cube.Cube` operand involved in a maths operation
+into the resultant :class:`~iris.cube.Cube`. It may be worth while investing
+some time to understand how the :class:`~iris.common.resolve.Resolve` class
+underpins cube maths, and consider how it may be used in general to combine
+or resolve cubes together.
+
+Given these prerequisites, recall that :ref:`lenient behaviour `
+introduced and discussed the concept of lenient metadata; a more pragmatic and
+forgiving approach to :ref:`comparing `,
+:ref:`combining ` and understanding the
+:ref:`differences ` between your metadata
+(:ref:`metadata members table`). The lenient metadata philosophy introduced
+there is extended to cube maths, with the view to also preserving as much common
+coordinate (:ref:`metadata classes table`) information, as well as common
+metadata, between the participating :class:`~iris.cube.Cube` operands as possible.
+
+Let's consolidate our understanding of lenient and strict cube maths through
+a practical worked example, which we'll explore together next.
+
+
+.. _lenient example:
+
+Lenient Example
+===============
+
+.. testsetup:: lenient-example
+
+ import iris
+ from iris.common import LENIENT
+ experiment = iris.load_cube(iris.sample_data_path("hybrid_height.nc"), "air_potential_temperature")
+ control = experiment[0]
+ control.remove_aux_factory(control.aux_factory())
+ for coord in ["sigma", "forecast_reference_time", "forecast_period", "atmosphere_hybrid_height_coordinate", "surface_altitude"]:
+ control.remove_coord(coord)
+ control.attributes["Conventions"] = "CF-1.7"
+ experiment.attributes["experiment-id"] = "RT3 50"
+
+Consider the following :class:`~iris.cube.Cube` of ``air_potential_temperature``,
+which has an `atmosphere hybrid height parametric vertical coordinate`_, and
+represents the output of an low-resolution global atmospheric ``experiment``,
+
+.. doctest:: lenient-example
+
+ >>> print(experiment)
+ air_potential_temperature / (K) (model_level_number: 15; grid_latitude: 100; grid_longitude: 100)
+ Dimension coordinates:
+ model_level_number x - -
+ grid_latitude - x -
+ grid_longitude - - x
+ Auxiliary coordinates:
+ atmosphere_hybrid_height_coordinate x - -
+ sigma x - -
+ surface_altitude - x x
+ Derived coordinates:
+ altitude x x x
+ Scalar coordinates:
+ forecast_period 0.0 hours
+ forecast_reference_time 2009-09-09 17:10:00
+ time 2009-09-09 17:10:00
+ Attributes:
+ Conventions 'CF-1.5'
+ STASH m01s00i004
+ experiment-id 'RT3 50'
+ source 'Data from Met Office Unified Model 7.04'
+
+Consider also the following :class:`~iris.cube.Cube`, which has the same global
+spatial extent, and acts as a ``control``,
+
+.. doctest:: lenient-example
+
+ >>> print(control)
+ air_potential_temperature / (K) (grid_latitude: 100; grid_longitude: 100)
+ Dimension coordinates:
+ grid_latitude x -
+ grid_longitude - x
+ Scalar coordinates:
+ model_level_number 1
+ time 2009-09-09 17:10:00
+ Attributes:
+ Conventions 'CF-1.7'
+ STASH m01s00i004
+ source 'Data from Met Office Unified Model 7.04'
+
+Now let's subtract these cubes in order to calculate a simple ``difference``,
+
+.. doctest:: lenient-example
+
+ >>> difference = experiment - control
+ >>> print(difference)
+ unknown / (K) (model_level_number: 15; grid_latitude: 100; grid_longitude: 100)
+ Dimension coordinates:
+ model_level_number x - -
+ grid_latitude - x -
+ grid_longitude - - x
+ Auxiliary coordinates:
+ atmosphere_hybrid_height_coordinate x - -
+ sigma x - -
+ surface_altitude - x x
+ Derived coordinates:
+ altitude x x x
+ Scalar coordinates:
+ forecast_period 0.0 hours
+ forecast_reference_time 2009-09-09 17:10:00
+ time 2009-09-09 17:10:00
+ Attributes:
+ experiment-id 'RT3 50'
+ source 'Data from Met Office Unified Model 7.04'
+
+Note that, cube maths automatically takes care of broadcasting the
+dimensionality of the ``control`` up to that of the ``experiment``, in order to
+calculate the ``difference``. This is performed only after ensuring that both
+the **dimension coordinates** ``grid_latitude`` and ``grid_longitude`` are first
+:ref:`leniently equivalent `.
+
+As expected, the resultant ``difference`` contains the
+:class:`~iris.aux_factory.HybridHeightFactory` and all it's associated **auxiliary
+coordinates**. However, the **scalar coordinates** have been leniently combined to
+preserve as much coordinate information as possible, and the ``attributes``
+dictionaries have also been leniently combined. In addition, see what further
+:ref:`rationalisation ` is always performed by cube maths on
+the resultant metadata and coordinates.
+
+Also, note that the ``model_level_number`` **scalar coordinate** from the
+``control`` has be superseded by the similarly named **dimension coordinate**
+from the ``experiment`` in the resultant ``difference``.
+
+Now let's compare and contrast this lenient result with the strict alternative.
+But before we do so, let's first clarify how to control the behaviour of cube maths.
+
+
+Control the Behaviour
+=====================
+
+As stated earlier, lenient cube maths is the default behaviour from Iris ``3.0.0``.
+However, this behaviour may be controlled via the thread-safe ``LENIENT["maths"]``
+runtime option,
+
+.. doctest:: lenient-example
+
+ >>> from iris.common import LENIENT
+ >>> print(LENIENT)
+ Lenient(maths=True)
+
+Which may be set and applied globally thereafter for Iris within the current
+thread of execution,
+
+.. doctest:: lenient-example
+
+ >>> LENIENT["maths"] = False # doctest: +SKIP
+ >>> print(LENIENT) # doctest: +SKIP
+ Lenient(maths=False)
+
+Or alternatively, temporarily alter the behaviour of cube maths only within the
+scope of the ``LENIENT`` `context manager`_,
+
+.. doctest:: lenient-example
+
+ >>> print(LENIENT)
+ Lenient(maths=True)
+ >>> with LENIENT.context(maths=False):
+ ... print(LENIENT)
+ ...
+ Lenient(maths=False)
+ >>> print(LENIENT)
+ Lenient(maths=True)
+
+
+Strict Example
+==============
+
+Now that we know how to control the underlying behaviour of cube maths,
+let's return to our :ref:`lenient example `, but this
+time perform **strict** cube maths instead,
+
+.. doctest:: lenient-example
+
+ >>> with LENIENT.context(maths=False):
+ ... difference = experiment - control
+ ...
+ >>> print(difference)
+ unknown / (K) (model_level_number: 15; grid_latitude: 100; grid_longitude: 100)
+ Dimension coordinates:
+ model_level_number x - -
+ grid_latitude - x -
+ grid_longitude - - x
+ Auxiliary coordinates:
+ atmosphere_hybrid_height_coordinate x - -
+ sigma x - -
+ surface_altitude - x x
+ Derived coordinates:
+ altitude x x x
+ Scalar coordinates:
+ time 2009-09-09 17:10:00
+ Attributes:
+ source 'Data from Met Office Unified Model 7.04'
+
+Although the numerical result of this strict cube maths operation is identical,
+it is not as rich in metadata as the :ref:`lenient alternative `.
+In particular, it does not contain the ``forecast_period`` and ``forecast_reference_time``
+**scalar coordinates**, or the ``experiment-id`` in the ``attributes`` dictionary.
+
+This is because strict cube maths, in general, will only return common metadata
+and common coordinates that are :ref:`strictly equivalent `.
+
+
+Finer Detail
+============
+
+In general, if you want to preserve as much metadata and coordinate information as
+possible during cube maths, then opt to use the default lenient behaviour. Otherwise,
+favour the strict alternative if you require to enforce precise metadata and
+coordinate commonality.
+
+The following information may also help you decide whether lenient cube maths best
+suits your use case,
+
+- lenient behaviour uses :ref:`lenient equality ` to match the
+ metadata of coordinates, which is more tolerant to certain metadata differences,
+- lenient behaviour uses :ref:`lenient combination
{}' - '¶' - '
') - - toc_template = (' opened up at the top of this loop.
- rows.append('
')
-
- # Generate JS list of images for front page.
- random_image_content = '\n'.join([random_image_join.format(i, line)
- for i, line in enumerate(random_image)])
- random_image_content = random_image_content_template.format(
- random_image_content)
- random_image_script_path = os.path.join(app.builder.srcdir,
- '_static',
- 'random_image.js')
- with open(random_image_script_path, 'w') as fh:
- fh.write(random_image_content)
-
- content = template.format('\n'.join(toc_rows),
- '\n'.join(rows))
-
- # Only write out the file if the contents have actually changed.
- # Otherwise, this triggers a full rebuild of the docs.
-
- gallery_path = os.path.join(app.builder.srcdir,
- '_templates',
- 'gallery.html')
- if os.path.exists(gallery_path):
- with open(gallery_path, 'r') as fh:
- regenerate = fh.read() != content
- else:
- regenerate = True
- if regenerate:
- with open(gallery_path, 'w') as fh:
- fh.write(content)
-
- for key in status_iterator(thumbnails, 'generating thumbnails... ',
- length=len(thumbnails)):
- image.thumbnail(key, thumbnails[key], 0.3)
-
-
-def setup(app):
- app.connect('env-updated', gen_gallery)
diff --git a/docs/iris/src/sphinxext/generate_package_rst.py b/docs/iris/src/sphinxext/generate_package_rst.py
deleted file mode 100644
index f67efc7ebc..0000000000
--- a/docs/iris/src/sphinxext/generate_package_rst.py
+++ /dev/null
@@ -1,315 +0,0 @@
-# (C) British Crown Copyright 2010 - 2015, Met Office
-#
-# This file is part of Iris.
-#
-# Iris is free software: you can redistribute it and/or modify it under
-# the terms of the GNU Lesser General Public License as published by the
-# Free Software Foundation, either version 3 of the License, or
-# (at your option) any later version.
-#
-# Iris is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-# GNU Lesser General Public License for more details.
-#
-# You should have received a copy of the GNU Lesser General Public License
-# along with Iris. If not, see Built using Python {{ python_version }}.
diff --git a/docs/src/_templates/custom_sidebar_logo_version.html b/docs/src/_templates/custom_sidebar_logo_version.html new file mode 100644 index 0000000000..c9d9ac6e2e --- /dev/null +++ b/docs/src/_templates/custom_sidebar_logo_version.html @@ -0,0 +1,26 @@ +{% if on_rtd %} + {% if rtd_version == 'latest' %} + +
+
+ {% elif rtd_version == 'stable' %}
+
+
+
+ {% elif rtd_version_type == 'tag' %}
+ {# Covers builds for specific tags, including RC's. #}
+
+
+
+ {% else %}
+ {# Anything else build by RTD will be the HEAD of an activated branch #}
+
+
+
+ {% endif %}
+{%- else %}
+ {# not on rtd #}
+
+
+
+{%- endif %}
diff --git a/docs/src/_templates/imagehash.html b/docs/src/_templates/imagehash.html
new file mode 100644
index 0000000000..8b0dac0cce
--- /dev/null
+++ b/docs/src/_templates/imagehash.html
@@ -0,0 +1,15 @@
+{% extends "!layout.html" %}
+
+{% block body %}
+
+Test: {{ test }}
+ + +{% for hash, file in hashfiles %} +
+ 
+
+{% endfor %}
+
+{% endblock %}
\ No newline at end of file
diff --git a/docs/src/common_links.inc b/docs/src/common_links.inc
new file mode 100644
index 0000000000..a49a98bfa6
--- /dev/null
+++ b/docs/src/common_links.inc
@@ -0,0 +1,84 @@
+.. comment
+ Common resources in alphabetical order:
+
+.. _black: https://black.readthedocs.io/en/stable/
+.. _cartopy: https://github.com/SciTools/cartopy
+.. _flake8: https://flake8.pycqa.org/en/stable/
+.. _.flake8.yml: https://github.com/SciTools/iris/blob/main/.flake8
+.. _cirrus-ci: https://cirrus-ci.com/github/SciTools/iris
+.. _codespell: https://github.com/codespell-project/codespell
+.. _conda: https://docs.conda.io/en/latest/
+.. _contributor: https://github.com/SciTools/iris/graphs/contributors
+.. _core developers: https://github.com/orgs/SciTools/teams/iris-devs/members
+.. _generating ssh keys for GitHub: https://docs.github.com/en/github/authenticating-to-github/adding-a-new-ssh-key-to-your-github-account
+.. _GitHub Actions: https://docs.github.com/en/actions
+.. _GitHub Help Documentation: https://docs.github.com/en/github
+.. _GitHub Discussions: https://github.com/SciTools/iris/discussions
+.. _Iris: https://github.com/SciTools/iris
+.. _Iris GitHub: https://github.com/SciTools/iris
+.. _Iris GitHub Actions: https://github.com/SciTools/iris/actions
+.. _iris-sample-data: https://github.com/SciTools/iris-sample-data
+.. _iris-test-data: https://github.com/SciTools/iris-test-data
+.. _isort: https://pycqa.github.io/isort/
+.. _issue: https://github.com/SciTools/iris/issues
+.. _issues: https://github.com/SciTools/iris/issues
+.. _legacy documentation: https://github.com/SciTools/scitools.org.uk/tree/master/iris/docs/archive
+.. _matplotlib: https://matplotlib.org/stable/
+.. _napolean: https://sphinxcontrib-napoleon.readthedocs.io/en/latest/sphinxcontrib.napoleon.html
+.. _nox: https://nox.thea.codes/en/stable/
+.. _New Issue: https://github.com/scitools/iris/issues/new/choose
+.. _pre-commit: https://pre-commit.com/
+.. _pull request: https://github.com/SciTools/iris/pulls
+.. _pull requests: https://github.com/SciTools/iris/pulls
+.. _Read the Docs: https://scitools-iris.readthedocs.io/en/latest/
+.. _readthedocs.yml: https://github.com/SciTools/iris/blob/main/requirements/readthedocs.yml
+.. _ruff: https://github.com/astral-sh/ruff
+.. _SciTools: https://github.com/SciTools
+.. _scitools-iris: https://pypi.org/project/scitools-iris/
+.. _Shapely: https://shapely.readthedocs.io/en/stable/index.html
+.. _sphinx: https://www.sphinx-doc.org/en/master/
+.. _sphinx-apidoc: https://github.com/sphinx-contrib/apidoc
+.. _test-iris-imagehash: https://github.com/SciTools/test-iris-imagehash
+.. _using git: https://docs.github.com/en/github/using-git
+.. _requirements: https://github.com/SciTools/iris/tree/main/requirements
+.. _CF-UGRID: https://ugrid-conventions.github.io/ugrid-conventions/
+.. _issues on GitHub: https://github.com/SciTools/iris/issues?q=is%3Aopen+is%3Aissue+sort%3Areactions-%2B1-desc
+.. _python-stratify: https://github.com/SciTools/python-stratify
+.. _iris-esmf-regrid: https://github.com/SciTools-incubator/iris-esmf-regrid
+.. _netCDF4: https://github.com/Unidata/netcdf4-python
+.. _SciTools Contributor's License Agreement (CLA): https://cla-assistant.io/SciTools/
+
+
+.. comment
+ Core developers and prolific contributors (@github names) in alphabetical order:
+
+.. _@abooton: https://github.com/abooton
+.. _@alastair-gemmell: https://github.com/alastair-gemmell
+.. _@ajdawson: https://github.com/ajdawson
+.. _@bjlittle: https://github.com/bjlittle
+.. _@bouweandela: https://github.com/bouweandela
+.. _@bsherratt: https://github.com/bsherratt
+.. _@corinnebosley: https://github.com/corinnebosley
+.. _@cpelley: https://github.com/cpelley
+.. _@djkirkham: https://github.com/djkirkham
+.. _@DPeterK: https://github.com/DPeterK
+.. _@ESadek-MO: https://github.com/ESadek-MO
+.. _@esc24: https://github.com/esc24
+.. _@HGWright: https://github.com/HGWright
+.. _@jamesp: https://github.com/jamesp
+.. _@jonseddon: https://github.com/jonseddon
+.. _@jvegasbsc: https://github.com/jvegasbsc
+.. _@lbdreyer: https://github.com/lbdreyer
+.. _@marqh: https://github.com/marqh
+.. _@pelson: https://github.com/pelson
+.. _@pp-mo: https://github.com/pp-mo
+.. _@QuLogic: https://github.com/QuLogic
+.. _@rcomer: https://github.com/rcomer
+.. _@rhattersley: https://github.com/rhattersley
+.. _@schlunma: https://github.com/schlunma
+.. _@stephenworsley: https://github.com/stephenworsley
+.. _@tkknight: https://github.com/tkknight
+.. _@trexfeathers: https://github.com/trexfeathers
+.. _@ukmo-ccbunney: https://github.com/ukmo-ccbunney
+.. _@wjbenfold: https://github.com/wjbenfold
+.. _@zklaus: https://github.com/zklaus
diff --git a/docs/src/community/index.rst b/docs/src/community/index.rst
new file mode 100644
index 0000000000..ee227513b3
--- /dev/null
+++ b/docs/src/community/index.rst
@@ -0,0 +1,60 @@
+.. include:: ../common_links.inc
+
+.. todo:
+ consider scientific-python.org
+ consider scientific-python.org/specs/
+
+Iris in the Community
+=====================
+
+Iris aims to be a valuable member of the open source scientific Python
+community.
+
+We listen out for developments in our dependencies and neighbouring projects,
+and we reach out to them when we can solve problems together; please feel free
+to reach out to us!
+
+We are aware of our place in the user's wider 'toolbox' - offering unique
+functionality and interoperating smoothly with other packages.
+
+We welcome contributions from all; whether that's an opinion, a 1-line
+clarification, or a whole new feature 🙂
+
+Quick Links
+-----------
+
+* `GitHub Discussions`_
+* :ref:`Getting involved