a new tsp sdk api for non-continuous transports

[wenjing]

Current tsp sdk offers the seal() and open() api that:

• seal() produces contiguous encoded data to be carried in a transport layer message where the encoded data is in a contiguous payload
• similarly, open() takes contiguous encoded data and verifies, and if encrypted, decrypts.

Some application scenarios however may use transport mechanisms where they wish to embed TSP envelope, payload, and signature in non-contiguous locations.

One such example is from the Matrix community where they may wish to use the TSP's signed but not encrypted messages to be carried over Matrix transparently (i.e. TSP messages being carried over Matrix messages). In this case, they would like to embed the TSP envelope and signature parts in Matrix's signature field, while leaving the unencrypted plaintext in its content field.

In such a scheme, the sender would still need to construct the encoded TSP message to generate the signature, and, in order for the receiver to verify such a Matrix message, it would need to reconstruct the contiguous TSP encoded message for signature and sender VID verification.

While the above Matrix example uses non-encrypted TSP messages, we could envision other applications where TSP encrypted messages are used instead. Therefore, it would be nice if we could design an api that can support both cases.

To better facilitate similar schemes, we propose to add a new api that:

• seal_nc() will generate correct signature in the same way as seal(), but the output will be in 3 parts: envelope, payload, signature. CESR encoding ensures that the 3 parts are concat'able - a simple concat() operation can reproduce the identical seal() output. The payload may be encrypted or non-encrypted.
• open_nc() will take in 3 parts: envelope, payload, signature, then it converts to a contiguous tsp message then verifies in the same way as open(). This should be possible again because CESR encoding. The payload may be encrypted or non-encrypted.
• tsp_payload_encode() is a helper function where the input is cleartext unencoded message (from Matrix) and output is encoded TSP payload which can be concat'ed to produce contiguous tsp message for verification.

The above Matrix application can then:

• On sending side, call seal_nc(). Concat envelope and signature to be carried in Matrix signature. Discard the encoded payload. Unencoded plaintext is already carried in normal Matrix message.
• On receive side, call tsp_payload_encode() to generate encoded payload from plaintext, then open_nc() to verify.

The above description is based on my basic understanding of its requirements. Please comment/edit as needed to make sure we have a more precise definition of this requirement. Note that the Matrix app MAY construct the above on its own by parsing/decoupling the CESR encoded message - but - I think an appropriately designed API could reduce work and error for application developers.

Below is an example of Matrix message in such use:

[kevinaboos]

(for any Matrix-related questions, cc me)

[michielp1807]

I think this can partly already be done by the functionality we currently expose:

• The open_message_into_parts function allows you to split a CESR-encoded TSP message into its separate parts.
• The encode_s_envelope encodes the envelope (including an optional non-confidential payload) into CESR, which could then be combined with CESR encoded encrypted payload (if any) and the CESR encoded signature to get the full TSP message.

Still it might be nice to have these methods for convenience.

[wenjing]

I think this can partly already be done by the functionality we currently expose:

• The open_message_into_parts function allows you to split a CESR-encoded TSP message into its separate parts.
• The encode_s_envelope encodes the envelope (including an optional non-confidential payload) into CESR, which could then be combined with CESR encoded encrypted payload (if any) and the CESR encoded signature to get the full TSP message.

Still it might be nice to have these methods for convenience.

I thought so too. This issue is mainly to keep track of all requirements and resolutions, and new functions, if added. The new new requirement, it seems, is to have an easy function to turn some matrix message data into CESR encoded data at the receiving side to ensure correct verification. I think CESR encoding does support this reconstruction at the receiving end. We can aid interoperability/standardization by providing a good/clear api function.

Please also note that Rev2 of the tsp spec is modifying the payload encoding structure a bit. I hope that change will be finalized next week for the SDK.

@kevinaboos also noted that sign_anycast() can be a starting point at least for the immediate need of unencrypted payload.

There is the question of what content types that Matrix/we need to support as well.

[kevinaboos]

I think this can partly already be done by the functionality we currently expose:

• The open_message_into_parts function allows you to split a CESR-encoded TSP message into its separate parts.
• The encode_s_envelope encodes the envelope (including an optional non-confidential payload) into CESR, which could then be combined with CESR encoded encrypted payload (if any) and the CESR encoded signature to get the full TSP message.

@michielp1807 I think these two methods are for decrypting an encrypted message, right? The functionality I'm looking to have is something like sign_anycast() return three separate buffers (or one buffer with three separate ranges) for:

1. the sender VID and whatever else is in the envelope (excluding the non-confidential message)
2. the non-confidential message data itself
3. the signature

The reasons/benefits for this are:

1. Matrix events have a size limit for the metadata, so we can't always just inject the entire signed message in the message metadata
2. it'd be quite wasteful to do so, even if we could fit it
3. the non-confidential message is already included in the Matrix event body, so we don't need to include it again.
   • we can simply reconstitute the buffer from each of the three parts, if needed, on the receiving side.

So, is it possible to modify sign_anycast() to return either extra ranges or a 3-part buffer? Or would you recommend a different means of achieving this?