From c8fc9575e2ee049db8a7a9ec756080cf01393a42 Mon Sep 17 00:00:00 2001 From: Matthew Bourque Date: Wed, 1 May 2024 09:27:14 -0600 Subject: [PATCH 1/5] Changed directory name to be consistent with how we call it in the docs --- docs/source/{development-guide => development}/doc-overview.rst | 0 docs/source/{development-guide => development}/docker.rst | 0 docs/source/{development-guide => development}/index.rst | 0 .../{development-guide => development}/release-workflow.rst | 0 .../style-guide/checklist-for-pull-requests.rst | 0 .../style-guide/git-and-github-workflow.rst | 0 .../style-guide/naming-conventions.rst | 0 .../style-guide/poetry-environment.rst | 0 .../style-guide/python-coding.rst | 0 .../style-guide/python-docstrings.rst | 0 .../style-guide/review-standards.rst | 0 .../{development-guide => development}/style-guide/security.rst | 0 .../style-guide/style-guide.rst | 0 .../style-guide/tools-and-library-recommendations.rst | 0 .../style-guide/versioning.rst | 0 docs/source/index.rst | 2 +- 16 files changed, 1 insertion(+), 1 deletion(-) rename docs/source/{development-guide => development}/doc-overview.rst (100%) rename docs/source/{development-guide => development}/docker.rst (100%) rename docs/source/{development-guide => development}/index.rst (100%) rename docs/source/{development-guide => development}/release-workflow.rst (100%) rename docs/source/{development-guide => development}/style-guide/checklist-for-pull-requests.rst (100%) rename docs/source/{development-guide => development}/style-guide/git-and-github-workflow.rst (100%) rename docs/source/{development-guide => development}/style-guide/naming-conventions.rst (100%) rename docs/source/{development-guide => development}/style-guide/poetry-environment.rst (100%) rename docs/source/{development-guide => development}/style-guide/python-coding.rst (100%) rename docs/source/{development-guide => development}/style-guide/python-docstrings.rst (100%) rename docs/source/{development-guide => development}/style-guide/review-standards.rst (100%) rename docs/source/{development-guide => development}/style-guide/security.rst (100%) rename docs/source/{development-guide => development}/style-guide/style-guide.rst (100%) rename docs/source/{development-guide => development}/style-guide/tools-and-library-recommendations.rst (100%) rename docs/source/{development-guide => development}/style-guide/versioning.rst (100%) diff --git a/docs/source/development-guide/doc-overview.rst b/docs/source/development/doc-overview.rst similarity index 100% rename from docs/source/development-guide/doc-overview.rst rename to docs/source/development/doc-overview.rst diff --git a/docs/source/development-guide/docker.rst b/docs/source/development/docker.rst similarity index 100% rename from docs/source/development-guide/docker.rst rename to docs/source/development/docker.rst diff --git a/docs/source/development-guide/index.rst b/docs/source/development/index.rst similarity index 100% rename from docs/source/development-guide/index.rst rename to docs/source/development/index.rst diff --git a/docs/source/development-guide/release-workflow.rst b/docs/source/development/release-workflow.rst similarity index 100% rename from docs/source/development-guide/release-workflow.rst rename to docs/source/development/release-workflow.rst diff --git a/docs/source/development-guide/style-guide/checklist-for-pull-requests.rst b/docs/source/development/style-guide/checklist-for-pull-requests.rst similarity index 100% rename from docs/source/development-guide/style-guide/checklist-for-pull-requests.rst rename to docs/source/development/style-guide/checklist-for-pull-requests.rst diff --git a/docs/source/development-guide/style-guide/git-and-github-workflow.rst b/docs/source/development/style-guide/git-and-github-workflow.rst similarity index 100% rename from docs/source/development-guide/style-guide/git-and-github-workflow.rst rename to docs/source/development/style-guide/git-and-github-workflow.rst diff --git a/docs/source/development-guide/style-guide/naming-conventions.rst b/docs/source/development/style-guide/naming-conventions.rst similarity index 100% rename from docs/source/development-guide/style-guide/naming-conventions.rst rename to docs/source/development/style-guide/naming-conventions.rst diff --git a/docs/source/development-guide/style-guide/poetry-environment.rst b/docs/source/development/style-guide/poetry-environment.rst similarity index 100% rename from docs/source/development-guide/style-guide/poetry-environment.rst rename to docs/source/development/style-guide/poetry-environment.rst diff --git a/docs/source/development-guide/style-guide/python-coding.rst b/docs/source/development/style-guide/python-coding.rst similarity index 100% rename from docs/source/development-guide/style-guide/python-coding.rst rename to docs/source/development/style-guide/python-coding.rst diff --git a/docs/source/development-guide/style-guide/python-docstrings.rst b/docs/source/development/style-guide/python-docstrings.rst similarity index 100% rename from docs/source/development-guide/style-guide/python-docstrings.rst rename to docs/source/development/style-guide/python-docstrings.rst diff --git a/docs/source/development-guide/style-guide/review-standards.rst b/docs/source/development/style-guide/review-standards.rst similarity index 100% rename from docs/source/development-guide/style-guide/review-standards.rst rename to docs/source/development/style-guide/review-standards.rst diff --git a/docs/source/development-guide/style-guide/security.rst b/docs/source/development/style-guide/security.rst similarity index 100% rename from docs/source/development-guide/style-guide/security.rst rename to docs/source/development/style-guide/security.rst diff --git a/docs/source/development-guide/style-guide/style-guide.rst b/docs/source/development/style-guide/style-guide.rst similarity index 100% rename from docs/source/development-guide/style-guide/style-guide.rst rename to docs/source/development/style-guide/style-guide.rst diff --git a/docs/source/development-guide/style-guide/tools-and-library-recommendations.rst b/docs/source/development/style-guide/tools-and-library-recommendations.rst similarity index 100% rename from docs/source/development-guide/style-guide/tools-and-library-recommendations.rst rename to docs/source/development/style-guide/tools-and-library-recommendations.rst diff --git a/docs/source/development-guide/style-guide/versioning.rst b/docs/source/development/style-guide/versioning.rst similarity index 100% rename from docs/source/development-guide/style-guide/versioning.rst rename to docs/source/development/style-guide/versioning.rst diff --git a/docs/source/index.rst b/docs/source/index.rst index df621d721..0b7f4de8e 100644 --- a/docs/source/index.rst +++ b/docs/source/index.rst @@ -20,7 +20,7 @@ The explicit code interfaces and structure are described in the :ref:`code-docum :maxdepth: 1 code-documentation/index - development-guide/index + development/index project-management/index external-tools/index data-access-api/index From e427b885fe6d397aa97a71fd498a496d95f8e21d Mon Sep 17 00:00:00 2001 From: Matthew Bourque Date: Wed, 8 May 2024 09:44:57 -0600 Subject: [PATCH 2/5] Initial commit of technology stack doc --- docs/source/development/technology-stack.rst | 128 +++++++++++++++++++ 1 file changed, 128 insertions(+) create mode 100644 docs/source/development/technology-stack.rst diff --git a/docs/source/development/technology-stack.rst b/docs/source/development/technology-stack.rst new file mode 100644 index 000000000..2dd0043da --- /dev/null +++ b/docs/source/development/technology-stack.rst @@ -0,0 +1,128 @@ +Technology Stack +---------------- + +This page lists the various technologies and libraries that the IMAP SDC +utilizes along with a few notes on what they are used for, why they were chosen, +and what other options were considered. It is worth noting that the development +of the SDC began in ~2022-2023, and so many of the technology decisions reflect +the technological ecosystem of that time. + +AWS +^^^ + +The SDC uses Amazon Web Services (AWS) as the solution for cloud infrastructure +used for data processing. Amongst cloud vendors, AWS was chosen mostly because +of its popularity within scientific software applications. Other cloud vendors +such as Microsoft Azure and Google Cloud were not really considered. + +AWS was chosen over an on-prem architecture due to the general movement within +the community to move data processing and storage to the cloud. + +The following sections describe the particular cloud services used in the SDC: + +CDK +""" + +Lambda +"""""" + +Batch +""""" + +RDS +""" + +RDS was famously chosen after an extremely lengthy discussion going +back-and-forth on pros and cons of various database technologies (OpenSearch and +DynamoDB were the other major considerations). RDS was ultimately chosen for +being the "old reliable" option, as it has been extensively used and is +extremely well documented. + +S3 +"" + + +Common Data Format +^^^^^^^^^^^^^^^^^^ + +The Common Data Format (CDF) was selected as the file format for IMAP data from +requirements. CDF is a widely used data format within the Heliophsyics +community. This decision was based purely on requirements. and arguably would +not be the top choice amongst IMAP SDC developers. As such, no other data +formats were considered. + +Docker +^^^^^^ + +GitHub +^^^^^^ + +The SDC uses GitHub for version controlling its software, as well as keeping +track of development tasks (i.e. GitHub Issues) and progress (i.e. GitHub +Projects), and performing code reviews. GitHub was chosen over other solutions +like GitLab and Bitbucket mainly for its collaborative features and unlimited +free public repositories. As the IMAP SDC strives to comply with the (relatively +new) NASA Open Source SPD-41a policies, this open-source collaborative GitHub +solution made the most sense. + +Poetry +^^^^^^ + + + +Pytest +^^^^^^ + +The ``pytest`` python library was chosen for writing and executing unit tests. +This library was chosen because it has a large and active community of users, is +well documented, and has several features that allow for a robust testing +framework (e.g. fixtures, parametrized testing, object mocking, etc.). There are +other tools/libraries for unit testing in Python (e.g. ``unittest``, ``nose``, +etc.), but those were not really considered. + + +Python +^^^^^^ + +The SDC uses Python as the primary programming language for the implementation +of the vast majority of the system and its ancillary tools. A few other +languages are supported for Level 3 algorithm development containers, and a lot +of the SDC cloud infrastructure is built with AWS CDK, but otherwise everything +is written in Python. Python was chosen mostly because at the time of +development, it was the most widely used and supported language for scientific +software development, and the main programming language used within the +Datasystems group at LASP. + + +Space Packet Parser +^^^^^^^^^^^^^^^^^^^ + +The SDC uses the ``space_packet_parser`` library for the decommutation of CCSDS +packets and processing L0-level data. This library was chosen for its support +of XTCE format for telemetry definitions. Another benefit to using this library +is that it was developed and is actively maintained in-house here in Datasystems +at LASP (Gavin Medley), and so the library can be updated to help meet the needs +of the IMAP SDC. The other option for packet decommutation would have been +CCSDSPy. + +Sphinx + ReadTheDocs +^^^^^^^^^^^^^^^^^^^^ + + +Sqlalchemy +^^^^^^^^^^ + +The Python ``sqlalchemy`` library is used as an object relational mapper (ORM) +for managing the relational database that the SDC uses. + +Xarray + cdflib +^^^^^^^^^^^^^^^ + +The Python ``xarray`` and ``cdflib`` libraries are used for creating data +structures for IMAP data and reading/writing those data structures to CDF files, +respectively. ``cdflib`` was chosen for CDF file I/O because of its convenient +``xarray_to_cdf()`` and ``cdf_to_xarray()`` functions. Additionally, the main +developer for ``cdflib`` (Bryan Harter) is also a developer for the IMAP SDC, +and so there is a lot of in-house knowledge of the library. ``xarray`` was +chosen for its support of data structures that closely match the format of CDF +files (i.e. use of data variables, data attributes, time coordinates, etc.). From c9ffa48a7b521c5518dae089cea31e310d631f8b Mon Sep 17 00:00:00 2001 From: Matthew Bourque Date: Wed, 8 May 2024 11:29:26 -0600 Subject: [PATCH 3/5] Added more content to tech stack doc --- docs/source/development/technology-stack.rst | 90 ++++++++++++-------- 1 file changed, 54 insertions(+), 36 deletions(-) diff --git a/docs/source/development/technology-stack.rst b/docs/source/development/technology-stack.rst index 2dd0043da..4a382895d 100644 --- a/docs/source/development/technology-stack.rst +++ b/docs/source/development/technology-stack.rst @@ -1,3 +1,6 @@ +`CDK deployment steps +`_ + Technology Stack ---------------- @@ -10,7 +13,8 @@ the technological ecosystem of that time. AWS ^^^ -The SDC uses Amazon Web Services (AWS) as the solution for cloud infrastructure +The SDC uses Amazon Web Services (AWS) as the solution for `cloud +infrastructure `_ used for data processing. Amongst cloud vendors, AWS was chosen mostly because of its popularity within scientific software applications. Other cloud vendors such as Microsoft Azure and Google Cloud were not really considered. @@ -32,11 +36,12 @@ Batch RDS """ -RDS was famously chosen after an extremely lengthy discussion going -back-and-forth on pros and cons of various database technologies (OpenSearch and -DynamoDB were the other major considerations). RDS was ultimately chosen for -being the "old reliable" option, as it has been extensively used and is -extremely well documented. +`RDS `_ was +famously chosen after an extremely lengthy discussion going back-and-forth on +pros and cons of various database technologies (OpenSearch and DynamoDB were the +other major considerations). RDS was ultimately chosen for being the "old +reliable" option, as it has been extensively used and is extremely well +documented. S3 "" @@ -47,9 +52,10 @@ Common Data Format The Common Data Format (CDF) was selected as the file format for IMAP data from requirements. CDF is a widely used data format within the Heliophsyics -community. This decision was based purely on requirements. and arguably would -not be the top choice amongst IMAP SDC developers. As such, no other data -formats were considered. +community. This decision was based purely on `requirements +`_ +and arguably would not be the top choice amongst IMAP SDC developers. As such, +no other data formats were considered. Docker ^^^^^^ @@ -57,28 +63,29 @@ Docker GitHub ^^^^^^ -The SDC uses GitHub for version controlling its software, as well as keeping -track of development tasks (i.e. GitHub Issues) and progress (i.e. GitHub -Projects), and performing code reviews. GitHub was chosen over other solutions -like GitLab and Bitbucket mainly for its collaborative features and unlimited -free public repositories. As the IMAP SDC strives to comply with the (relatively -new) NASA Open Source SPD-41a policies, this open-source collaborative GitHub -solution made the most sense. +The SDC uses `GitHub `_ for +version controlling its software, as well as keeping track of development tasks +(i.e. GitHub Issues) and progress (i.e. GitHub Projects), and performing code +reviews. GitHub was chosen over other solutions like GitLab and Bitbucket mainly +for its collaborative features and unlimited free public repositories. As the +IMAP SDC strives to comply with the (relatively new) `NASA SMD SPD-41a policies +`_, +this open-source collaborative GitHub solution made the most sense. Poetry ^^^^^^ - Pytest ^^^^^^ -The ``pytest`` python library was chosen for writing and executing unit tests. -This library was chosen because it has a large and active community of users, is -well documented, and has several features that allow for a robust testing -framework (e.g. fixtures, parametrized testing, object mocking, etc.). There are -other tools/libraries for unit testing in Python (e.g. ``unittest``, ``nose``, -etc.), but those were not really considered. +The `pytest `_ python library was chosen for writing +and executing unit tests. This library was chosen because it has a large and +active community of users, is well documented, and has several features that +allow for a robust testing framework (e.g. fixtures, parametrized testing, +object mocking, etc.). There are other tools/libraries for unit testing in +Python (e.g. ``unittest``, ``nose``, etc.), but those were not really +considered. Python @@ -86,9 +93,10 @@ Python The SDC uses Python as the primary programming language for the implementation of the vast majority of the system and its ancillary tools. A few other -languages are supported for Level 3 algorithm development containers, and a lot -of the SDC cloud infrastructure is built with AWS CDK, but otherwise everything -is written in Python. Python was chosen mostly because at the time of +languages are supported for Level 3 `algorithm development containers +`_, +and a lot of the SDC cloud infrastructure is built with AWS CDK, but otherwise +everything is written in Python. Python was chosen mostly because at the time of development, it was the most widely used and supported language for scientific software development, and the main programming language used within the Datasystems group at LASP. @@ -97,28 +105,38 @@ Datasystems group at LASP. Space Packet Parser ^^^^^^^^^^^^^^^^^^^ -The SDC uses the ``space_packet_parser`` library for the decommutation of CCSDS -packets and processing L0-level data. This library was chosen for its support -of XTCE format for telemetry definitions. Another benefit to using this library -is that it was developed and is actively maintained in-house here in Datasystems -at LASP (Gavin Medley), and so the library can be updated to help meet the needs -of the IMAP SDC. The other option for packet decommutation would have been -CCSDSPy. +The SDC uses the `space_packet_parser +`_ library for the +decommutation of CCSDS packets and processing L0-level data. This library was +chosen for its support of XTCE format for telemetry definitions. Another benefit +to using this library is that it was developed and is actively maintained +in-house here in Datasystems at LASP (Gavin Medley), and so the library can be +updated to help meet the needs of the IMAP SDC. The other option for packet +decommutation would have been `CCSDSPy `_. Sphinx + ReadTheDocs ^^^^^^^^^^^^^^^^^^^^ +The SDC uses the Python `sphinx `_ +library for generating project documentation as well as reference documentation +that gets automatically rendered from code docstrings. `ReadTheDocs +`_ is used to host the +documentation via the HTML files that ``sphinx`` generates. Together, these +tools product the 'official' `IMAP SDC documentation +`_. + +These tools were chosen because they are widely used and have great integration +with Poetry and GitHub. Sqlalchemy ^^^^^^^^^^ -The Python ``sqlalchemy`` library is used as an object relational mapper (ORM) -for managing the relational database that the SDC uses. Xarray + cdflib ^^^^^^^^^^^^^^^ -The Python ``xarray`` and ``cdflib`` libraries are used for creating data +The Python `xarray `_ and `cdflib +`_ libraries are used for creating data structures for IMAP data and reading/writing those data structures to CDF files, respectively. ``cdflib`` was chosen for CDF file I/O because of its convenient ``xarray_to_cdf()`` and ``cdf_to_xarray()`` functions. Additionally, the main From cc07fc03689ef35d6e69a8171d4954f07c27319c Mon Sep 17 00:00:00 2001 From: Matthew Bourque Date: Wed, 8 May 2024 13:33:04 -0600 Subject: [PATCH 4/5] Organized headings and subheadings --- docs/source/development/index.rst | 3 +- docs/source/development/technology-stack.rst | 32 +++++++++++--------- 2 files changed, 20 insertions(+), 15 deletions(-) diff --git a/docs/source/development/index.rst b/docs/source/development/index.rst index 9a085c76b..273d93ccc 100644 --- a/docs/source/development/index.rst +++ b/docs/source/development/index.rst @@ -38,4 +38,5 @@ A typical development workflow might look like the following: doc-overview docker release-workflow - style-guide/style-guide \ No newline at end of file + style-guide/style-guide + technology-stack \ No newline at end of file diff --git a/docs/source/development/technology-stack.rst b/docs/source/development/technology-stack.rst index 4a382895d..302bf6c87 100644 --- a/docs/source/development/technology-stack.rst +++ b/docs/source/development/technology-stack.rst @@ -1,6 +1,3 @@ -`CDK deployment steps -`_ - Technology Stack ---------------- @@ -11,7 +8,7 @@ of the SDC began in ~2022-2023, and so many of the technology decisions reflect the technological ecosystem of that time. AWS -^^^ +~~~ The SDC uses Amazon Web Services (AWS) as the solution for `cloud infrastructure `_ @@ -27,12 +24,19 @@ The following sections describe the particular cloud services used in the SDC: CDK """ + +API Gateway +""""""""""" + Lambda """""" Batch """"" +EventBridge +""""""""""" + RDS """ @@ -48,7 +52,7 @@ S3 Common Data Format -^^^^^^^^^^^^^^^^^^ +~~~~~~~~~~~~~~~~~~ The Common Data Format (CDF) was selected as the file format for IMAP data from requirements. CDF is a widely used data format within the Heliophsyics @@ -58,10 +62,10 @@ and arguably would not be the top choice amongst IMAP SDC developers. As such, no other data formats were considered. Docker -^^^^^^ +~~~~~~ GitHub -^^^^^^ +~~~~~~ The SDC uses `GitHub `_ for version controlling its software, as well as keeping track of development tasks @@ -73,11 +77,11 @@ IMAP SDC strives to comply with the (relatively new) `NASA SMD SPD-41a policies this open-source collaborative GitHub solution made the most sense. Poetry -^^^^^^ +~~~~~~ Pytest -^^^^^^ +~~~~~~ The `pytest `_ python library was chosen for writing and executing unit tests. This library was chosen because it has a large and @@ -89,7 +93,7 @@ considered. Python -^^^^^^ +~~~~~~ The SDC uses Python as the primary programming language for the implementation of the vast majority of the system and its ancillary tools. A few other @@ -103,7 +107,7 @@ Datasystems group at LASP. Space Packet Parser -^^^^^^^^^^^^^^^^^^^ +~~~~~~~~~~~~~~~~~~~ The SDC uses the `space_packet_parser `_ library for the @@ -115,7 +119,7 @@ updated to help meet the needs of the IMAP SDC. The other option for packet decommutation would have been `CCSDSPy `_. Sphinx + ReadTheDocs -^^^^^^^^^^^^^^^^^^^^ +~~~~~~~~~~~~~~~~~~~~ The SDC uses the Python `sphinx `_ library for generating project documentation as well as reference documentation @@ -129,11 +133,11 @@ These tools were chosen because they are widely used and have great integration with Poetry and GitHub. Sqlalchemy -^^^^^^^^^^ +~~~~~~~~~~ Xarray + cdflib -^^^^^^^^^^^^^^^ +~~~~~~~~~~~~~~~ The Python `xarray `_ and `cdflib `_ libraries are used for creating data From e333739ac924e5837c58ffd42670efcaba82d770 Mon Sep 17 00:00:00 2001 From: Matthew Bourque Date: Fri, 7 Jun 2024 12:38:04 -0600 Subject: [PATCH 5/5] Updates from PR comments/suggestions --- docs/source/development/technology-stack.rst | 41 ++++++++++---------- 1 file changed, 20 insertions(+), 21 deletions(-) diff --git a/docs/source/development/technology-stack.rst b/docs/source/development/technology-stack.rst index 302bf6c87..76c5a39cd 100644 --- a/docs/source/development/technology-stack.rst +++ b/docs/source/development/technology-stack.rst @@ -50,6 +50,19 @@ documented. S3 "" +cdflib + Xarray +~~~~~~~~~~~~~~~ + +The Python `xarray `_ and `cdflib +`_ libraries are used for creating data +structures for IMAP data and reading/writing those data structures to CDF files, +respectively. ``cdflib`` was chosen for CDF file I/O because of its convenient +``xarray_to_cdf()`` and ``cdf_to_xarray()`` functions. Additionally, the main +developer for ``cdflib`` (Bryan Harter) is also a developer for the IMAP SDC, +and so there is a lot of in-house knowledge of the library. ``xarray`` was +chosen for its support of data structures that closely match the format of CDF +files (i.e. use of data variables, data attributes, time coordinates, etc.). + Common Data Format ~~~~~~~~~~~~~~~~~~ @@ -57,9 +70,8 @@ Common Data Format The Common Data Format (CDF) was selected as the file format for IMAP data from requirements. CDF is a widely used data format within the Heliophsyics community. This decision was based purely on `requirements -`_ -and arguably would not be the top choice amongst IMAP SDC developers. As such, -no other data formats were considered. +`_. +As such, no other data formats were considered. Docker ~~~~~~ @@ -72,9 +84,10 @@ version controlling its software, as well as keeping track of development tasks (i.e. GitHub Issues) and progress (i.e. GitHub Projects), and performing code reviews. GitHub was chosen over other solutions like GitLab and Bitbucket mainly for its collaborative features and unlimited free public repositories. As the -IMAP SDC strives to comply with the (relatively new) `NASA SMD SPD-41a policies +IMAP SDC strives to comply with the `NASA SMD SPD-41a policies `_, -this open-source collaborative GitHub solution made the most sense. +(adopted in 2022) this open-source collaborative GitHub solution made the most +sense. Poetry ~~~~~~ @@ -103,7 +116,7 @@ and a lot of the SDC cloud infrastructure is built with AWS CDK, but otherwise everything is written in Python. Python was chosen mostly because at the time of development, it was the most widely used and supported language for scientific software development, and the main programming language used within the -Datasystems group at LASP. +Data Systems group at LASP. Space Packet Parser @@ -114,7 +127,7 @@ The SDC uses the `space_packet_parser decommutation of CCSDS packets and processing L0-level data. This library was chosen for its support of XTCE format for telemetry definitions. Another benefit to using this library is that it was developed and is actively maintained -in-house here in Datasystems at LASP (Gavin Medley), and so the library can be +in-house here in Data Systems at LASP (Gavin Medley), and so the library can be updated to help meet the needs of the IMAP SDC. The other option for packet decommutation would have been `CCSDSPy `_. @@ -134,17 +147,3 @@ with Poetry and GitHub. Sqlalchemy ~~~~~~~~~~ - - -Xarray + cdflib -~~~~~~~~~~~~~~~ - -The Python `xarray `_ and `cdflib -`_ libraries are used for creating data -structures for IMAP data and reading/writing those data structures to CDF files, -respectively. ``cdflib`` was chosen for CDF file I/O because of its convenient -``xarray_to_cdf()`` and ``cdf_to_xarray()`` functions. Additionally, the main -developer for ``cdflib`` (Bryan Harter) is also a developer for the IMAP SDC, -and so there is a lot of in-house knowledge of the library. ``xarray`` was -chosen for its support of data structures that closely match the format of CDF -files (i.e. use of data variables, data attributes, time coordinates, etc.).