From c35f10fcd54ffd57ee96472d52713c905d394dd7 Mon Sep 17 00:00:00 2001 From: Rusty Russell Date: Tue, 27 Jul 2021 13:17:37 +0930 Subject: [PATCH 1/4] option_websocket: allow transport over RFC6455 And allow transparent upgrade. This allows a wider range of software to interact with the lightning network. Signed-off-by: Rusty Russell --- .aspell.en.pws | 2 ++ 08-transport.md | 47 +++++++++++++++++++++++++++++++++++++++++++++-- 09-features.md | 2 ++ 3 files changed, 49 insertions(+), 2 deletions(-) diff --git a/.aspell.en.pws b/.aspell.en.pws index 2c0d09fdf..3fc6da7c4 100644 --- a/.aspell.en.pws +++ b/.aspell.en.pws @@ -387,3 +387,5 @@ CHECKSIGVERIFY IFDUP sats anysegwit +WebSocket +websocket diff --git a/08-transport.md b/08-transport.md index 49ca45e05..dc842f250 100644 --- a/08-transport.md +++ b/08-transport.md @@ -18,6 +18,7 @@ of a node. * [Handshake State](#handshake-state) * [Handshake State Initialization](#handshake-state-initialization) * [Handshake Exchange](#handshake-exchange) + * [Alternate Transport Layers: WebSocket](#websocket) * [Lightning Message Specification](#lightning-message-specification) * [Encrypting and Sending Messages](#encrypting-and-sending-messages) * [Receiving and Decrypting Messages](#receiving-and-decrypting-messages) @@ -100,7 +101,9 @@ indicates that no change is necessary, while a non-zero version indicate that th client has deviated from the protocol originally specified within this document. -Clients MUST reject handshake attempts initiated with an unknown version. +Clients MAY attempt to switch to an alternate interpretation if they +receive and invalid version on receipt of Act One, but otherwise +MUST reject handshake attempts initiated with an unknown version. ### Noise Protocol Instantiation @@ -253,7 +256,9 @@ and 16 bytes for the `poly1305` tag. * The raw bytes of the remote party's ephemeral public key (`re`) are to be deserialized into a point on the curve using affine coordinates as encoded by the key's serialized composed format. -3. If `v` is an unrecognized handshake version, then the responder MUST +3. If `v` is an unrecognized handshake version, and the responder supports + `option_websocket`, it MAY interpret the message as the initiation of + a [WebSocket connection](#websocket). Otherwise the responder MUST abort the connection attempt. 4. `h = SHA-256(h || re.serializeCompressed())` * The responder accumulates the initiator's ephemeral key into the authenticating @@ -402,6 +407,43 @@ construction, and 16 bytes for a final authenticating tag. 10. `rn = 0, sn = 0` * The sending and receiving nonces are initialized to 0. +## Alternate Transport Layers: WebSocket + +Normally the transport protocol defined here is performed over TCP/IP, +but it can also be performed over other underlying transports, such as +the WebSocket protocol as specified in +RFC6455[4](#reference-4). + +A client may connect to a node and initiate a WebSocket; this will +normally fail as the WebSocket protocol begins with a "GET" request, +which is trivially distinguishable from a valid handshake. However, +the node may also allow it and operate the protocol over binary +WebSocket frames. The `option_websocket` feature allows nodes to +advertise this, but not all nodes send node_announcements, so it is +not required before attempting a WebSocket connection. + + +### Requirements + +The initiator: +- MAY attempt to initiate an unencrypted WebSocket as specified in RFC6455[4](#reference-4): + - MUST send at least 50 bytes before awaiting a response. + - MUST abort the connection attempt if WebSocket upgrade fails. + - MUST begin the [Handshake Exchange](#handshake-exchange) as initiator + as soon as upgrade succeeds. + +The responder: +- if it supports `option_websocket`: + - SHOULD set `option_websocket` in its node announcements + - MUST attempt WebSocket upgrade if the Act 1 handshake it receives + is not valid. + - MUST abort the connection attempt if WebSocket upgrade fails. + +Both nodes, after upgrade: + - MUST use binary frames to send and receive messages. + - MUST NOT rely on WebSocket framing for message semantics. + + ## Lightning Message Specification At the conclusion of Act Three, both sides have derived the encryption keys, which @@ -779,6 +821,7 @@ TODO(roasbeef); fin 1. https://tools.ietf.org/html/rfc8439 2. http://noiseprotocol.org/noise.html 3. https://tools.ietf.org/html/rfc5869 +4. https://tools.ietf.org/html/rfc6455 # Authors diff --git a/09-features.md b/09-features.md index b508bac70..3a5abc9ed 100644 --- a/09-features.md +++ b/09-features.md @@ -41,6 +41,7 @@ The Context column decodes as follows: | 20/21 | `option_anchor_outputs` | Anchor outputs | IN | `option_static_remotekey` | [BOLT #3](03-transactions.md) | | 22/23 | `option_anchors_zero_fee_htlc_tx` | Anchor commitment type with zero fee HTLC transactions | IN | | [BOLT #3][bolt03-htlc-tx], [lightning-dev][ml-sighash-single-harmful]| | 26/27 | `option_shutdown_anysegwit` | Future segwit versions allowed in `shutdown` | IN | | [BOLT #2][bolt02-shutdown] | +| 42/43 | `option_websocket` | Can tunnel over WebSocket protocol | N | | [BOLT #8][bolt08-websocket] | ## Requirements @@ -88,4 +89,5 @@ This work is licensed under a [Creative Commons Attribution 4.0 International Li [bolt07-sync]: 07-routing-gossip.md#initial-sync [bolt07-query]: 07-routing-gossip.md#query-messages [bolt04-mpp]: 04-onion-routing.md#basic-multi-part-payments +[bolt08-websocket]: 08-transport.md#websocket [ml-sighash-single-harmful]: https://lists.linuxfoundation.org/pipermail/lightning-dev/2020-September/002796.html From 729d0eded1ab3cf34e071833e3b5c43d8dc25685 Mon Sep 17 00:00:00 2001 From: Rusty Russell Date: Mon, 30 Aug 2021 12:52:30 +0930 Subject: [PATCH 2/4] websocket: make it an address type, not an option. Everyone hated the overload. But it's not a full address type, so make it just a port number as a compromise. Signed-off-by: Rusty Russell --- 07-routing-gossip.md | 14 +++++++++++++- 08-transport.md | 27 ++++++++------------------- 09-features.md | 1 - 3 files changed, 21 insertions(+), 21 deletions(-) diff --git a/07-routing-gossip.md b/07-routing-gossip.md index b89c3d222..e879887b4 100644 --- a/07-routing-gossip.md +++ b/07-routing-gossip.md @@ -285,6 +285,7 @@ The following `address descriptor` types are defined: onion service addresses; Encodes: `[32:32_byte_ed25519_pubkey] || [2:checksum] || [1:version]`, where `checksum = sha3(".onion checksum" | pubkey || version)[:2]`. + * `5`: WebSocket port; data = `[2:port]` (length 2) ### Requirements @@ -306,12 +307,16 @@ The origin node: - MUST place address descriptors in ascending order. - SHOULD NOT place any zero-typed address descriptors anywhere. - SHOULD use placement only for aligning fields that follow `addresses`. - - MUST NOT create a `type 1` OR `type 2` address descriptor with `port` equal + - MUST NOT create a `type 1`, `type 2` or `type 5` address descriptor with `port` equal to 0. - SHOULD ensure `ipv4_addr` AND `ipv6_addr` are routable addresses. - MUST set `features` according to [BOLT #9](09-features.md#assigned-features-flags) - SHOULD set `flen` to the minimum length required to hold the `features` bits it sets. + - MUST NOT add a `type 5` address unless there is also at least one address of different type. + - if it adds a type 5 address: + - MUST allow unencrypted RFC6455[3](#reference-3) as a transport when a connection is made to at least one of the other addresses, with the type 5 `port` substituted for that address's `port` + - SHOULD allow this on ALL of the other addresses. The receiving node: - if `node_id` is NOT a valid compressed public key: @@ -359,6 +364,12 @@ to be ordered in ascending order, unknown ones can be safely ignored. Additional fields beyond `addresses` may also be added in the future—with optional padding within `addresses`, if they require certain alignment. +Websockets generally are run on adjacent ports (or even overloaded on +the same port) as existing "raw" transports, so including just the +port is a compromise which avoids replacating all the addresses. It's +ideal if all addresses support this, but it's not a hard requirement: +at least one must. + ### Security Considerations for Node Aliases Node aliases are user-defined and provide a potential avenue for injection @@ -1123,6 +1134,7 @@ above. 1. [RFC 1950 "ZLIB Compressed Data Format Specification version 3.3](https://www.ietf.org/rfc/rfc1950.txt) 2. [Maximum Compression Factor](https://zlib.net/zlib_tech.html) +3. [RFC 6455 "The WebSocket Protocol"](https://datatracker.ietf.org/doc/html/rfc6455) ![Creative Commons License](https://i.creativecommons.org/l/by/4.0/88x31.png "License CC-BY")
diff --git a/08-transport.md b/08-transport.md index dc842f250..6c654533d 100644 --- a/08-transport.md +++ b/08-transport.md @@ -101,9 +101,7 @@ indicates that no change is necessary, while a non-zero version indicate that th client has deviated from the protocol originally specified within this document. -Clients MAY attempt to switch to an alternate interpretation if they -receive and invalid version on receipt of Act One, but otherwise -MUST reject handshake attempts initiated with an unknown version. +Clients MUST reject handshake attempts initiated with an unknown version. ### Noise Protocol Instantiation @@ -256,9 +254,7 @@ and 16 bytes for the `poly1305` tag. * The raw bytes of the remote party's ephemeral public key (`re`) are to be deserialized into a point on the curve using affine coordinates as encoded by the key's serialized composed format. -3. If `v` is an unrecognized handshake version, and the responder supports - `option_websocket`, it MAY interpret the message as the initiation of - a [WebSocket connection](#websocket). Otherwise the responder MUST +3. If `v` is an unrecognized handshake version, then the responder MUST abort the connection attempt. 4. `h = SHA-256(h || re.serializeCompressed())` * The responder accumulates the initiator's ephemeral key into the authenticating @@ -412,31 +408,24 @@ construction, and 16 bytes for a final authenticating tag. Normally the transport protocol defined here is performed over TCP/IP, but it can also be performed over other underlying transports, such as the WebSocket protocol as specified in -RFC6455[4](#reference-4). +RFC6455[4](#reference-4) on ports so-advertized (in the +[node_announcement message](07-routing-gossip.md#the-node_announcement-message). -A client may connect to a node and initiate a WebSocket; this will -normally fail as the WebSocket protocol begins with a "GET" request, -which is trivially distinguishable from a valid handshake. However, -the node may also allow it and operate the protocol over binary -WebSocket frames. The `option_websocket` feature allows nodes to -advertise this, but not all nodes send node_announcements, so it is -not required before attempting a WebSocket connection. +A client may connect to this port node and initiate a WebSocket; and +operate the protocol over binary WebSocket frames instead of raw TCP/IP. ### Requirements The initiator: - MAY attempt to initiate an unencrypted WebSocket as specified in RFC6455[4](#reference-4): - - MUST send at least 50 bytes before awaiting a response. - MUST abort the connection attempt if WebSocket upgrade fails. - MUST begin the [Handshake Exchange](#handshake-exchange) as initiator as soon as upgrade succeeds. The responder: -- if it supports `option_websocket`: - - SHOULD set `option_websocket` in its node announcements - - MUST attempt WebSocket upgrade if the Act 1 handshake it receives - is not valid. +- if it supports WebSocket connections on a port: + - SHOULD advertize it using a type 5 address its node announcement. - MUST abort the connection attempt if WebSocket upgrade fails. Both nodes, after upgrade: diff --git a/09-features.md b/09-features.md index 3a5abc9ed..29949e191 100644 --- a/09-features.md +++ b/09-features.md @@ -41,7 +41,6 @@ The Context column decodes as follows: | 20/21 | `option_anchor_outputs` | Anchor outputs | IN | `option_static_remotekey` | [BOLT #3](03-transactions.md) | | 22/23 | `option_anchors_zero_fee_htlc_tx` | Anchor commitment type with zero fee HTLC transactions | IN | | [BOLT #3][bolt03-htlc-tx], [lightning-dev][ml-sighash-single-harmful]| | 26/27 | `option_shutdown_anysegwit` | Future segwit versions allowed in `shutdown` | IN | | [BOLT #2][bolt02-shutdown] | -| 42/43 | `option_websocket` | Can tunnel over WebSocket protocol | N | | [BOLT #8][bolt08-websocket] | ## Requirements From a174e0dabc695bd823e63a4be11f28be83f86607 Mon Sep 17 00:00:00 2001 From: Rusty Russell Date: Tue, 26 Oct 2021 06:34:42 +1030 Subject: [PATCH 3/4] fixup! websocket: make it an address type, not an option. Signed-off-by: Rusty Russell --- 07-routing-gossip.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/07-routing-gossip.md b/07-routing-gossip.md index e879887b4..cc8fad687 100644 --- a/07-routing-gossip.md +++ b/07-routing-gossip.md @@ -285,7 +285,7 @@ The following `address descriptor` types are defined: onion service addresses; Encodes: `[32:32_byte_ed25519_pubkey] || [2:checksum] || [1:version]`, where `checksum = sha3(".onion checksum" | pubkey || version)[:2]`. - * `5`: WebSocket port; data = `[2:port]` (length 2) + * `6`: WebSocket port; data = `[2:port]` (length 2) ### Requirements From 5f9449287dd2a75c114582339743c68e8a577bec Mon Sep 17 00:00:00 2001 From: Rusty Russell Date: Tue, 26 Oct 2021 07:01:49 +1030 Subject: [PATCH 4/4] fixup! fixup! websocket: make it an address type, not an option. --- 07-routing-gossip.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/07-routing-gossip.md b/07-routing-gossip.md index cc8fad687..a8b35e60c 100644 --- a/07-routing-gossip.md +++ b/07-routing-gossip.md @@ -307,15 +307,15 @@ The origin node: - MUST place address descriptors in ascending order. - SHOULD NOT place any zero-typed address descriptors anywhere. - SHOULD use placement only for aligning fields that follow `addresses`. - - MUST NOT create a `type 1`, `type 2` or `type 5` address descriptor with `port` equal + - MUST NOT create a `type 1`, `type 2` or `type 6` address descriptor with `port` equal to 0. - SHOULD ensure `ipv4_addr` AND `ipv6_addr` are routable addresses. - MUST set `features` according to [BOLT #9](09-features.md#assigned-features-flags) - SHOULD set `flen` to the minimum length required to hold the `features` bits it sets. - - MUST NOT add a `type 5` address unless there is also at least one address of different type. - - if it adds a type 5 address: - - MUST allow unencrypted RFC6455[3](#reference-3) as a transport when a connection is made to at least one of the other addresses, with the type 5 `port` substituted for that address's `port` + - MUST NOT add a `type 6` address unless there is also at least one address of different type. + - if it adds a type 6 address: + - MUST allow unencrypted RFC6455[3](#reference-3) as a transport when a connection is made to at least one of the other addresses, with the type 6 `port` substituted for that address's `port` - SHOULD allow this on ALL of the other addresses. The receiving node: