Enum encoding / decoding inconsistencies between platforms

[amsam0]

Hi, I was recently working on a project involving communication between a microcontroller and a host computer over UART. I used postcard on both sides to send extremely simple (less than 10 bytes) messages from the host computer to the microcontroller.

Here are enums equivalent to the messages I was sending:

#[derive(Serialize, Deserialize, Debug, Clone)]
pub enum Message {
    Hello,
    Start {
        a: (Parameter1, Parameter2),
        b: (Parameter1, Parameter2),
    },
    Stop,
}

#[derive(Serialize, Deserialize, Debug, Clone)]
pub enum Parameter1 {
    Option1,
    Option2,
}

#[derive(Serialize, Deserialize, Debug, Clone)]
pub enum Parameter2 {
    Option1,
    Option2,
}

The Parameter1 and Parameter2 enums were what caused issues. At first, the messages were failing to decode entirely; I was able to get them to decode after changing Parameter1 to a bool in Message. However, no matter which option I sent, postcard would always decode Parameter2 as Option1.

Eventually I was able to fix this by hand writing the Serialize and Deserialize implementations for Parameter2 to serialize and deserialize as a bool.

`Serialize` and `Deserialize` implementations for `Parameter2`

impl Serialize for Parameter2 {
    fn serialize<S>(&self, serializer: S) -> Result<S::Ok, S::Error>
    where
        S: serde::Serializer,
    {
        match self {
            Parameter2::Option1 => serializer.serialize_bool(true),
            Parameter2::Option2 => serializer.serialize_bool(false),
        }
    }
}

// Taken from serde::de
struct BoolVisitor;

impl<'de> Visitor<'de> for BoolVisitor {
    type Value = bool;

    fn expecting(&self, formatter: &mut core::fmt::Formatter) -> core::fmt::Result {
        formatter.write_str("a boolean")
    }

    fn visit_bool<E>(self, v: bool) -> Result<Self::Value, E>
    where
        E: serde::de::Error,
    {
        Ok(v)
    }
}

impl<'lt> Deserialize<'lt> for Parameter2 {
    fn deserialize<D>(deserializer: D) -> Result<Self, D::Error>
    where
        D: serde::Deserializer<'lt>,
    {
        Ok(match deserializer.deserialize_bool(BoolVisitor)? {
            true => Parameter2::Option1,
            false => Parameter2::Option2,
        })
    }
}

Do you have any ideas what was causing this? Perhaps how serde was generating the Serialize and Deserialize implementations?

Additional notes:

• I used postcard's COBS flavor to make communication easier. I have not yet tested this without COBS by sending the length of the message as the first byte.
• If the cause of the issue isn't immediately clear, I can send message dumps of what the computer sends the microcontroller and vice versa.
• I did not try using serde_repr's derive macros, but those probably would've fixed the issue.
• If this is a real issue, a flavor that appends the hash of the structure/enum before encoding and then allows checking that hash vs the decoded structure/enum might be useful.

(also, unrelated but do you think you could checkout jamesmunns/cobs.rs#20 and jamesmunns/cobs.rs#22?)