Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

ENH: Add Enumeration states to XTCE parser #783

Merged
merged 2 commits into from
Aug 30, 2024
Merged

ENH: Add Enumeration states to XTCE parser #783

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
8d922cf
ENH: Add Enumeration states to XTCE parser
greglucas Aug 26, 2024
6b8bff9
TST: Update unit tests for xtce creation
greglucas Aug 30, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 30 additions & 0 deletions docs/source/code-documentation/tools/xtce-generator.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -124,6 +124,13 @@ the conversion details.
- ANALOG
- Apply an analog conversion
-
* - MY_INSTRUMENT_HK
- VARIABLE_ENUMERATED
- 1
- UINT
- STATE
- Apply an enumeration state
-
* - MY_INSTRUMENT_HK
- VARIABLE_LENGTH_BINARY_SCIENCE
- 100
Expand Down Expand Up @@ -164,3 +171,26 @@ coefficients defined from ``c0`` to ``c7`` to define the order of the polynomial
-
-
-

States tab (optional)
~~~~~~~~~~~~~~~~~~~~~

Packet parsing can also apply enumeration/state conversions to the data being read in.
For example, to change from a raw unsigned integer value to a "VALID" / "INVALID" string.
The ``States`` tab is used to define these enumerations.

.. list-table:: States
:header-rows: 1

* - packetName
- mnemonic
- value
- state
* - MY_INSTRUMENT_HK
- VARIABLE_ENUMERATED
- 0
- INVALID
* - MY_INSTRUMENT_HK
- VARIABLE_ENUMERATED
- 1
- VALID
33 changes: 32 additions & 1 deletion imap_processing/ccsds/excel_to_xtce.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -251,7 +251,6 @@ def _add_parameter(self, row: pd.Series, total_packet_bits: int) -> None:
# Combine the packet name and mnemonic to create a unique parameter name
name = f"{row['packetName']}.{row['mnemonic']}"
parameter.attrib["name"] = name
# UINT8, ...
parameter.attrib["parameterTypeRef"] = name

# Add descriptions if they exist
Expand Down Expand Up @@ -329,6 +328,10 @@ def _add_parameter(self, row: pd.Series, total_packet_bits: int) -> None:
# Go look up the conversion in the AnalogConversions tab
# and add it to the encoding
self._add_analog_conversion(row, encoding)
elif row["convertAs"] == "STATE":
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't notice that this column and that tab was connected. I knew that they stored enums(derived) values in that tab but didn't connect those two together. Nice!

# Go look up the states in the States tab
# and add them to the parameter type
self._add_state_conversion(row, parameter_type)

def _add_analog_conversion(self, row: pd.Series, encoding: Et.Element) -> None:
"""
Expand Down Expand Up @@ -363,6 +366,34 @@ def _add_analog_conversion(self, row: pd.Series, encoding: Et.Element) -> None:
term.attrib["coefficient"] = str(conversion[col])
term.attrib["exponent"] = str(i)

def _add_state_conversion(self, row: pd.Series, parameter_type: Et.Element) -> None:
"""
Add a state conversion to the parameter type.

Changing from an IntegerParameterType to an EnumeratedParameterType. Adding
the list of state mappings to the parameter type.

Parameters
----------
row : pandas.Row
Row to be added to the XTCE file, containing mnemonic, packetName.
parameter_type : Element
The parameter type element to add the conversion to.
"""
# It is an EnumeratedParameterType rather than an IntegerParameterType
parameter_type.tag = "xtce:EnumeratedParameterType"
enumeration_list = Et.SubElement(parameter_type, "xtce:EnumerationList")
# Lookup the enumeration states for this parameter from the States sheet
state_sheet = self.sheets["States"]
state_sheet = state_sheet.loc[
(state_sheet["packetName"] == row["packetName"])
& (state_sheet["mnemonic"] == row["mnemonic"])
]
for _, state_row in state_sheet.iterrows():
enumeration = Et.SubElement(enumeration_list, "xtce:Enumeration")
enumeration.attrib["value"] = str(state_row["value"])
enumeration.attrib["label"] = str(state_row["state"])
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am wondering if this function produces something like this?

	        <xtce:EnumeratedParameterType name="QUARTER_CYCLE_ENUM" signed="false">
                <xtce:IntegerDataEncoding sizeInBits="5" encoding="unsigned"/>
                <xtce:EnumerationList>
                    <xtce:Enumeration label="FIRST" value="0"/>
                    <xtce:Enumeration label="SECOND" value="1"/>
		   <xtce:Enumeration label="THIRD" value="2"/>
                    <xtce:Enumeration label="FORTH" value="3"/>
                </xtce:EnumerationList>
            </xtce:EnumeratedParameterType>

I saw most of it but didn't see it add this line. I think we need that.

                <xtce:IntegerDataEncoding sizeInBits="5" encoding="unsigned"/>

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That is taken care of in the above _add_parameter_type(), which adds the encodings. Then here we modify that parameter type to be EnumeratedParameterType instead of IntegerParameterType. It should be there if you run this on the swapi or swe definitions (could you verify it works for you as well). This is where tests would be nice...

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I ran locally and it looks correct.


def to_xml(self, output_xml_path: Path) -> None:
"""
Create and output an XTCE file from the Element Tree representation.
Expand Down
Binary file removed imap_processing/tests/ccsds/test_data/excel_to_xtce_test_file.xlsx
View file
Open in desktop
Binary file not shown.
17 changes: 14 additions & 3 deletions imap_processing/tests/ccsds/test_data/expected_output.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@
<xtce:SizeInBits>
<xtce:DynamicValue>
<xtce:ParameterInstanceRef parameterRef="PKT_LEN" />
<xtce:LinearAdjustment slope="8" intercept="-70" />
<xtce:LinearAdjustment slope="8" intercept="-71" />
</xtce:DynamicValue>
</xtce:SizeInBits>
</xtce:BinaryDataEncoding>
Expand All @@ -59,6 +59,13 @@
<xtce:FloatParameterType name="TEST_PACKET.VAR_FLOAT">
<xtce:FloatDataEncoding sizeInBits="32" encoding="IEEE-754" />
</xtce:FloatParameterType>
<xtce:EnumeratedParameterType name="TEST_PACKET.VAR_STATE" signed="false">
<xtce:IntegerDataEncoding sizeInBits="1" encoding="unsigned" />
<xtce:EnumerationList>
<xtce:Enumeration value="0" label="OFF" />
<xtce:Enumeration value="1" label="ON" />
</xtce:EnumerationList>
</xtce:EnumeratedParameterType>
<xtce:IntegerParameterType name="TEST_PACKET2.SHCOARSE" signed="false">
<xtce:IntegerDataEncoding sizeInBits="32" encoding="unsigned" />
</xtce:IntegerParameterType>
Expand Down Expand Up @@ -109,11 +116,14 @@
<xtce:Parameter name="TEST_PACKET.VAR_FLOAT" parameterTypeRef="TEST_PACKET.VAR_FLOAT">
<xtce:LongDescription>Float data</xtce:LongDescription>
</xtce:Parameter>
<xtce:Parameter name="TEST_PACKET.VAR_STATE" parameterTypeRef="TEST_PACKET.VAR_STATE">
<xtce:LongDescription>State data</xtce:LongDescription>
</xtce:Parameter>
<xtce:Parameter name="TEST_PACKET2.SHCOARSE" parameterTypeRef="TEST_PACKET2.SHCOARSE">
<xtce:LongDescription>Mission elapsed time</xtce:LongDescription>
</xtce:Parameter>
<xtce:Parameter name="TEST_PACKET2.VAR1" parameterTypeRef="TEST_PACKET2.VAR1" shortDescription="Variable 1 short desc">
<xtce:LongDescription>Variable 1 - long desc</xtce:LongDescription>
<xtce:Parameter name="TEST_PACKET2.VAR1" parameterTypeRef="TEST_PACKET2.VAR1" shortDescription="Variable 1 short description">
<xtce:LongDescription>Variable 1 long description</xtce:LongDescription>
</xtce:Parameter>
</xtce:ParameterSet>
<xtce:ContainerSet>
Expand Down Expand Up @@ -142,6 +152,7 @@
<xtce:ParameterRefEntry parameterRef="TEST_PACKET.VAR_BYTE" />
<xtce:ParameterRefEntry parameterRef="TEST_PACKET.VAR_FILL" />
<xtce:ParameterRefEntry parameterRef="TEST_PACKET.VAR_FLOAT" />
<xtce:ParameterRefEntry parameterRef="TEST_PACKET.VAR_STATE" />
</xtce:EntryList>
</xtce:SequenceContainer>
<xtce:SequenceContainer name="TEST_PACKET2">
Expand Down
Loading
Loading