From a285ba3ae0eccb2ddbf934a736a43a49be67f4a6 Mon Sep 17 00:00:00 2001 From: Greg Lucas Date: Wed, 21 Aug 2024 13:05:30 -0600 Subject: [PATCH] DOC: Update XTCE generator documentation --- .../tools/xtce-generator.rst | 210 ++++++++++++------ 1 file changed, 146 insertions(+), 64 deletions(-) diff --git a/docs/source/code-documentation/tools/xtce-generator.rst b/docs/source/code-documentation/tools/xtce-generator.rst index 452746a8d..cc16f5066 100644 --- a/docs/source/code-documentation/tools/xtce-generator.rst +++ b/docs/source/code-documentation/tools/xtce-generator.rst @@ -1,84 +1,166 @@ .. _xtce_generator: -Generating Telemetry XML with Python Script -=========================================== +XML Telemetric and Command Exchange (XTCE) +========================================== -Here is some info on `XTCE `_. This Green -Book introduces the main concepts of XML Telemetric and Command Exchange (XTCE), a -telemetry and telecommand database format for spacecraft monitoring -and control. +The `XTCE green book `_. +introduces the main concepts and specifications of XTCE. +The XTCE format is a specification for spacecraft monitoring and control data transfer. +It can be used to define how packets can be sent and received from the spacecraft, +which we then unpack on the ground using the XTCE format. -General -------- - -This document provides steps and information on how to use -`xtce_generator_template.py` script as a base for users to generate -telemetry XML files. The script is designed to simplify the process of creating -telemetry definitions for various packet types. - -The script is located in the `tools/xtce_generation` directory. The script is called -`xtce_generator_template.py`. The script is a ``template`` that can be modified to -generate telemetry XML files for different packet types. Your new file should be -called `xtce_generator_yourinstrument.py`. -An example of how to use the script is `xtce_generator_codice.py` which is also -located in the `tools/xtce_generation` directory. Before you Start ---------------- Generating XTCEs is only done whenever packet definitions get updated, and thus it -is not a part of the main processing package. To use it there are a few extra -dependencies like ``pandas`` that you can install with +is not regularly run as a part of processing. To use it there are a few extra +dependencies (like ``openpyxl`` for reading excel spreadsheets) that you +can install with the tools extra. .. code:: + # with poetry poetry install --extras tools + # or with pip + pip install imap_processing[tools] How to Use ---------- -Define the instrument name in the `main()` function by setting the `instrument_name` -variable to the name of your instrument. - -.. code:: - - instrument_name = "your_instrument_name" - -In the code, file paths are being configured. Make sure to change the file paths to -match your instrument's file structure. - -.. code:: - - current_directory = Path(__file__).parent - module_path = f"{current_directory}/../../imap_processing" - # This is the path of the output directory - packet_definition_path = f"{module_path}/{instrument_name}/packet_definitions" - # This is the path to the excel file that contains the telemetry definitions - path_to_excel_file = f"{current_directory}/your_packet.xlsx" - -Define packet names and `Application Process Identifiers (APIDs) -`_. -The packet names are **case sensitive** meaning the the packet names need to be exactly -what the tabs of the spreadsheet are. APID's must match the names and apIds in the -packet definition file. You can use as many packet names and apIds as you want. -The APID should be an integer (not hexadecimal). -Follow the format below. - -.. code:: - - packets = { - # Define packet names and associated Application IDs (apId) - "your_packet_A": ####, - "your_packet_B": ####, - # ... (other packet definitions) - } - -Generating Telemetry XML Files -------------------------------- - -Once you have your xtce processing file defined, you can run it with the -following command: +There is a command line utility ``imap_xtce`` that can be used to generate XTCE files +that is installed with the ``imap_processing`` package. +The utility takes in an excel file and generates XTCE files for each packet definition +in the excel file. If you don't provide an output file, it will generate the XTCE file +with the same name as the input Excel file but with the extension changed to ``.xml``. .. code:: - python xtce_generator_instrument_name.py + imap_xtce path/to/excel_packet_file.xlsx --output path/to/output_packet_definition.xml + + +Spreadsheet definitions +----------------------- + +The XTCE generator uses an excel spreadsheet to define the packet structure. +This is a commonly used spreadsheet format at the Laboratory for Atmospheric and Space Physics (LASP). +The required tabs are ``Subsystem``, ``Packets``, and whatever packet names you have. + +Subsystem tab +~~~~~~~~~~~~~ + +The ``Subsystem`` tab is used to define the instrument name and last updated date of the packet data. + +.. list-table:: Subsystem + :header-rows: 1 + + * - infoField + - infoValue + * - subsystem + - MY_INSTRUMENT + * - sheetReleaseDate + - 01/01/2010 + * - sheetReleaseRev + - 1.2.3 + +Packets tab +~~~~~~~~~~~ + +The packets tab contains the list of packets that you want to include within your XTCE +packet definition. You can remove rows from this to control which individual packet tabs +are read in later. The ``packetName`` column defines which other tabs to read in. So in +the following table, the generator will read in the ``MY_INSTRUMENT_HK`` and +``MY_INSTRUMENT_SCI`` tabs that contain the packet definitions. + +.. note:: + The generator will also work with tabs prefixed with ``P_``, so ``P_MY_INSTRUMENT_HK`` and + ``P_MY_INSTRUMENT_SCI`` tab names would also work. + +.. list-table:: Packets + :header-rows: 1 + + * - packetName + - apId + * - MY_INSTRUMENT_HK + - 123 + * - MY_INSTRUMENT_SCI + - 124 + +Individual packet tabs +~~~~~~~~~~~~~~~~~~~~~~ + +Each packet tab contains the contents that will create the XTCE packet definition. +The required columns are ``packetName``, ``mnemonic``, ``lengthInBits``, ``dataType``, +``convertAs``, with optional ``shortDescription`` and ``longDescription`` columns. + +Within the XTCE definition, the variable names will be ``packetName.mnemonic`` separated +with a period for easier distinguishing between packets and variables. For example, +the table below would have this XTCE parameter definition ``MY_INSTRUMENT_HK.VARIABLE1_UINT`` +for the first variable. If an analog conversion is required, the ``convertAs`` column +should be set to ``ANALOG``, which will then look at the ``AnalogConversions`` tab for +the conversion details. + +.. list-table:: MY_INSTRUMENT_HK + :header-rows: 1 + + * - packetName + - mnemonic + - lengthInBits + - dataType + - convertAs + - shortDescription + - longDescription + * - MY_INSTRUMENT_HK + - VARIABLE1_UINT + - 3 + - UINT + - NONE + - My short variable description + - My verbose variable description + * - MY_INSTRUMENT_HK + - VARIABLE2_CONVERTED + - 3 + - UINT + - ANALOG + - Apply an analog conversion + - + * - MY_INSTRUMENT_HK + - VARIABLE_LENGTH_BINARY_SCIENCE + - 100 + - BYTE + - NONE + - + - This variable size will be dynamic and based on the packet size + +AnalogConversions tab (optional) +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Packet parsing can also apply analog conversions to the data being read in. +For example, to change from a raw unsigned integer value to a temperature in Kelvin. +The ``AnalogConversions`` tab is used to define these conversions. +It currently only supports unsegmented polynomial conversions, and looks for the +coefficients defined from ``c0`` to ``c7`` to define the order of the polynomial. + +.. list-table:: AnalogConversions + :header-rows: 1 + + * - packetName + - mnemonic + - c0 + - c1 + - c2 + - c3 + - c4 + - c5 + - c6 + - c7 + * - MY_INSTRUMENT_HK + - VARIABLE2_CONVERTED + - 123.456 + - 0.234 + - + - + - + - + - + -