From f5eba72ab94504875a06406391e5d545929e82f5 Mon Sep 17 00:00:00 2001 From: Devguru Date: Fri, 22 May 2026 16:57:00 +0530 Subject: [PATCH 1/3] spec: document optimistic multistream-select (issue #643) Add new Working Draft specification for optimistic (lazy) multistream-select protocol negotiation, documenting the behavior already implemented in go-libp2p and rust-libp2p. - New: connections/optimistic-multistream.md - Modified: connections/README.md (ToC entry + cross-reference) Refs: multiformats/go-multistream#115, multiformats/go-multistream#87, multiformats/go-multistream#20, libp2p/go-libp2p#3038 --- connections/README.md | 11 + connections/optimistic-multistream.md | 307 ++++++++++++++++++++++++++ 2 files changed, 318 insertions(+) create mode 100644 connections/optimistic-multistream.md diff --git a/connections/README.md b/connections/README.md index b8ce295cb..f307a53be 100644 --- a/connections/README.md +++ b/connections/README.md @@ -27,6 +27,7 @@ and spec status. - [Definitions](#definitions) - [Protocol Negotiation](#protocol-negotiation) - [multistream-select](#multistream-select) + - [Optimistic Protocol Negotiation](#optimistic-protocol-negotiation) - [Upgrading Connections](#upgrading-connections) - [Opening New Streams Over a Connection](#opening-new-streams-over-a-connection) - [Practical Considerations](#practical-considerations) @@ -183,6 +184,15 @@ traffic over the channel will adhere to the rules of the agreed-upon protocol. If a peer receives a `"na"` response to a proposed protocol id, they can either try again with a different protocol id or close the channel. +### Optimistic Protocol Negotiation + +When the dialer has prior knowledge that the responder supports a given +protocol (e.g., via the [identify protocol][identify/push]), it can use an +optimistic variant of multistream-select that saves one round trip by sending +the protocol proposal and application data without waiting for the responder's +echo. For details, see the [Optimistic Protocol Negotiation][optimistic-ms] +specification. + ## Upgrading Connections @@ -427,3 +437,4 @@ updated to incorporate the changes. [resource-manager-issue]: https://github.com/libp2p/go-libp2p/issues/635 [hole-punching]: ./hole-punching.md [inlined-muxer-selection]: ./inlined-muxer-negotiation.md +[optimistic-ms]: ./optimistic-multistream.md diff --git a/connections/optimistic-multistream.md b/connections/optimistic-multistream.md new file mode 100644 index 000000000..df0a5c351 --- /dev/null +++ b/connections/optimistic-multistream.md @@ -0,0 +1,307 @@ +# Optimistic Protocol Negotiation + +| Lifecycle Stage | Maturity | Status | Latest Revision | +|-----------------|---------------|--------|-----------------| +| 1A | Working Draft | Active | r0, 2026-05-22 | + +Authors: [@Devguru-codes] + +Interest Group: [@marcopolo], [@marten-seemann] + +[@Devguru-codes]: https://github.com/Devguru-codes +[@marcopolo]: https://github.com/MarcoPolo +[@marten-seemann]: https://github.com/marten-seemann + +See the [lifecycle document][lifecycle-spec] for context about the maturity level +and spec status. + +[lifecycle-spec]: https://github.com/libp2p/specs/blob/master/00-framework-01-spec-lifecycle.md + +## Table of Contents + +- [Overview](#overview) +- [Applicability](#applicability) +- [Wire Format](#wire-format) + - [Standard Multistream-Select](#standard-multistream-select) + - [Optimistic Multistream-Select](#optimistic-multistream-select) +- [Prerequisites](#prerequisites) +- [Requirements](#requirements) + - [Dialer (Initiator) Requirements](#dialer-initiator-requirements) + - [Listener (Responder) Requirements](#listener-responder-requirements) +- [Known Limitations](#known-limitations) + - [Protocol Confusion on Negotiation Failure](#protocol-confusion-on-negotiation-failure) +- [Security Considerations](#security-considerations) +- [Interaction with Inlined Muxer Negotiation](#interaction-with-inlined-muxer-negotiation) +- [Implementation References](#implementation-references) + +## Overview + +Also known as "lazy multistream-select" or "lazy negotiation". + +In standard [multistream-select][mss] negotiation, the dialer (initiator) sends +its protocol proposal and **waits** for the listener (responder) to echo the +protocol ID back before sending any application data. This costs two full +round trips before application data can flow. + +Optimistic protocol negotiation eliminates one round trip by allowing the dialer +to send the multistream-select header, the protocol proposal, **and** the initial +application data all at once, without waiting for the listener's echo response. +The listener's echo response arrives asynchronously while the dialer is already +sending application data. + +This optimization is critical for latency-sensitive use cases such as +[Kademlia DHT][kad-dht] operations, which use a "one stream per RPC" pattern +and would otherwise pay the full two-round-trip cost on every request. + +Both [go-libp2p] and [rust-libp2p] use optimistic negotiation in production, +but it has not previously been documented in the libp2p specifications. This +document formalizes the optimization and documents the requirements +implementations must follow to use it correctly. + +## Applicability + +Optimistic protocol negotiation can be applied in two contexts: + +1. **Stream-level negotiation** (primary use case): When opening a new stream + over an existing connection, the dialer can optimistically propose a protocol + and begin sending application data immediately. This is the most common use + case and provides the greatest latency benefit for protocols that use + short-lived streams (e.g., DHT RPCs, Bitswap requests). + +2. **Connection upgrade negotiation**: When negotiating security or stream + multiplexing protocols during [connection establishment][connections]. This + use case is less common because [inlined muxer negotiation][inlined-muxer] + already reduces the round-trip cost of the connection upgrade process. + +## Wire Format + +The wire format for individual messages is unchanged from standard +[multistream-select][mss]. Messages are UTF-8 strings, newline-terminated, and +prefixed with their length as an [unsigned varint][uvarint]. The difference is +purely in the **sequencing** of messages. + +### Standard Multistream-Select + +In the standard (non-optimistic) flow, the dialer waits for the listener's echo +before sending application data: + +``` +Dialer Listener + | | + |--- /multistream/1.0.0 ---------------->| + |<-- /multistream/1.0.0 -----------------| + | | + |--- /my-protocol/1.0.0 --------------->| + |<-- /my-protocol/1.0.0 ---- (echo) ----| <- Dialer waits for echo + | | + |--- [application data] --------------->| <- Only then sends data + | | +``` + +**Cost**: 2 round trips before application data flows. + +### Optimistic Multistream-Select + +In the optimistic flow, the dialer sends the multistream header, protocol +proposal, and application data without waiting: + +``` +Dialer Listener + | | + |--- /multistream/1.0.0 ---------------->| + |--- /my-protocol/1.0.0 ---------------->| <- No wait for header echo + |--- [application data] ---------------->| <- Sent immediately + | | + |<-- /multistream/1.0.0 -----------------| <- Echo arrives later + |<-- /my-protocol/1.0.0 -----------------| + | | +``` + +**Cost**: 1 round trip saved. Application data is delivered to the listener +alongside the protocol proposal. The listener processes the negotiation and then +delivers the application data to the protocol handler. + +## Prerequisites + +Optimistic protocol negotiation is inherently **best-effort**. The dialer sends +application data before receiving confirmation that the listener supports the +requested protocol. If the listener does not support the protocol, the +negotiation will fail, and the application data sent optimistically will be +wasted. + +To minimize the risk of failed negotiations, dialers typically use the +[identify protocol][identify] to learn which protocols a peer supports before +using optimistic negotiation. The `protocols` field of the identify message +(see [identify spec][identify]) contains the list of protocols the remote peer +supports. + +However, it is important to note that this is still *optimistic* — a peer's +supported protocols can change dynamically at any time (e.g., via +[identify/push][identify-push] updates, or due to configuration changes). +In practice, protocol support is stable enough that optimistic negotiation +succeeds in the vast majority of cases. + +Implementations SHOULD NOT use optimistic negotiation on the **first** stream +to a peer when no prior protocol knowledge is available. + +## Requirements + +The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", +"SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be +interpreted as described in [RFC 2119]. + +### Dialer (Initiator) Requirements + +1. **Ordering**: Implementations MUST send the multistream protocol ID + (`/multistream/1.0.0`) and the application protocol ID before any + application data on the stream, without waiting for responses between + them. + +2. **Handshake completion on close**: Implementations SHOULD finish reading the + handshake response before closing the stream. If the dialer writes + application data and then closes the stream before the listener has echoed + back the protocol ID, the listener's echo write may fail on a closed stream, + potentially causing the listener to reset the stream and discard data that + was already received. + + If the application does not need to read any data from the stream (i.e., it + is a pure write/fire-and-forget pattern), the implementation MAY skip reading + the handshake response. + +3. **Prior knowledge**: Implementations SHOULD NOT use optimistic negotiation + without prior knowledge that the peer supports the requested protocol (e.g., + via a preceding [identify][identify] exchange). Using optimistic negotiation + without prior knowledge risks triggering the protocol confusion issue + described in [Known Limitations](#known-limitations). + +### Listener (Responder) Requirements + +1. **Tolerating echo write failures**: Implementations SHOULD ignore write + errors when echoing back the protocol ID during negotiation. With optimistic + negotiation, the dialer may have already closed its write side of the stream + after sending application data. The echo write failing does not indicate a + negotiation failure — the negotiated protocol has already been identified. + +2. **Stream delivery**: Implementations MUST still deliver the stream to the + application protocol handler even if the echo write fails. The handler can + then `Read()` any application data the dialer already sent. + +3. **Data boundary**: After a protocol has been successfully matched during + negotiation, implementations MUST NOT interpret subsequent bytes on the + stream as multistream-select messages. All bytes after the negotiated + protocol ID are application data and MUST be forwarded to the protocol + handler. + +## Known Limitations + +### Protocol Confusion on Negotiation Failure + +There is a known soundness issue with optimistic multistream-select that arises +when the negotiation **fails** — i.e., the listener does not support the +proposed protocol. + +Consider the following scenario: + +- Peer A supports `protocolA` +- Peer B supports `protocolB` (but **not** `protocolA`) + +Peer A sends optimistically: + +``` +/multistream/1.0.0\n /protocolA/1.0.0\n +``` + +Then immediately sends application data that happens to look like a valid +multistream-select protocol proposal: + +``` +/protocolB/1.0.0\n [other data] +``` + +Peer B does not support `protocolA`, so it responds with `"na"` and reads the +next message. It sees `/protocolB/1.0.0` — which it **does** support — and +interprets it as a new protocol proposal. It echoes back `/protocolB/1.0.0` and +starts interpreting `[other data]` as `protocolB` traffic. + +**Result**: Peer B is now speaking `protocolB` with data that was actually +`protocolA` application data. This is a protocol confusion vulnerability. + +This issue is extensively documented in [go-multistream#20]. As stated by the +original author: + +> "I don't know of a good fix that won't break everything (without upping +> the version and adding a mandatory round trip)." + +**Mitigations**: + +- The prerequisite of having prior protocol knowledge (via identify) makes this + scenario unlikely in practice, since the dialer would not propose a protocol + the listener does not support. +- Application protocol designers SHOULD ensure that their wire format cannot be + confused with multistream-select messages (i.e., application data should not + begin with a valid varint-prefixed, newline-terminated protocol path). +- Implementations SHOULD log or track negotiation failures to detect potential + protocol confusion. + +## Security Considerations + +The protocol confusion issue described in [Known Limitations](#known-limitations) +represents a potential security concern. If a malicious or misconfigured peer +provides incorrect information about which protocols it supports (e.g., via +identify responses), the dialer may optimistically propose a protocol the peer +does not actually support, triggering the protocol confusion scenario. + +Implementations SHOULD: + +- Validate identify responses and consider them advisory, not authoritative. +- Track and limit the rate of negotiation failures per peer. A high failure rate + may indicate a misconfigured peer or an attack. +- Ensure that protocol handlers validate incoming data before processing it, + regardless of how the stream was negotiated. + +The security impact is limited by the fact that libp2p connections are +authenticated and encrypted (via [Noise][noise] or [TLS 1.3][tls]). An attacker +cannot inject data into an existing connection. The protocol confusion risk only +applies within an already-authenticated peer relationship, where the remote peer +provides incorrect identify information. + +## Interaction with Inlined Muxer Negotiation + +For connection upgrades (as opposed to stream-level negotiation), the latency +benefit of optimistic negotiation is reduced by [inlined muxer negotiation][inlined-muxer], +which moves the stream multiplexer selection into the security handshake. When +inlined muxer negotiation is in use, the connection upgrade already avoids the +extra round trip for muxer selection, so the additional savings from optimistic +negotiation during the connection upgrade are minimal. + +Optimistic negotiation during connection upgrades is therefore most useful when +inlined muxer negotiation is **not** available (e.g., when connecting to older +peers that do not support the optimization). + +## Implementation References + +The following implementation artifacts motivated and informed this specification: + +| Reference | Description | +|-----------|-------------| +| [go-multistream#115] | Fix: finish reading handshake on `lazyConn` close | +| [go-multistream#87] | Fix: ignore error if can't write back echoed protocol | +| [go-multistream#20] | Issue: lazy negotiation soundness problem (protocol confusion) | +| [go-libp2p#3038] | Bug: WebTransport `StopSending` error caused by incomplete lazy handshake | + +[mss]: https://github.com/multiformats/multistream-select +[uvarint]: https://github.com/multiformats/unsigned-varint +[connections]: https://github.com/libp2p/specs/tree/master/connections +[inlined-muxer]: ./inlined-muxer-negotiation.md +[identify]: ../identify/README.md +[identify-push]: ../identify/README.md#identifypush +[kad-dht]: ../kad-dht/README.md +[noise]: ../noise/README.md +[tls]: ../tls/tls.md +[go-libp2p]: https://github.com/libp2p/go-libp2p +[rust-libp2p]: https://github.com/libp2p/rust-libp2p +[go-multistream#115]: https://github.com/multiformats/go-multistream/pull/115 +[go-multistream#87]: https://github.com/multiformats/go-multistream/pull/87 +[go-multistream#20]: https://github.com/multiformats/go-multistream/issues/20 +[go-libp2p#3038]: https://github.com/libp2p/go-libp2p/issues/3038 +[RFC 2119]: https://www.ietf.org/rfc/rfc2119.txt From 548f837298f7f1bf33405b842527516daae745d2 Mon Sep 17 00:00:00 2001 From: Devguru Date: Tue, 16 Jun 2026 19:40:53 +0530 Subject: [PATCH 2/3] address review: apply all feedback from MarcoPolo - Fix RTT count: standard negotiation is 1 RTT, not 2 (Comment 1, 4) - Fix both diagrams to show header+protocol sent together - Simplify wire format section (Comment 2) - Remove meta 'not previously documented' paragraph (Comment 3) - Change 'best-effort' to 'optimistic' (Comment 5) - Soften SHOULD NOT to MAY for prior knowledge (Comment 6, 7) - Remove obvious 'Data boundary' requirement (Comment 8) - Simplify inlined muxer section (Comment 9) - Remove Security Considerations section (Comment 10) --- connections/optimistic-multistream.md | 83 +++++++-------------------- 1 file changed, 20 insertions(+), 63 deletions(-) diff --git a/connections/optimistic-multistream.md b/connections/optimistic-multistream.md index df0a5c351..1a3fccb56 100644 --- a/connections/optimistic-multistream.md +++ b/connections/optimistic-multistream.md @@ -30,7 +30,6 @@ and spec status. - [Listener (Responder) Requirements](#listener-responder-requirements) - [Known Limitations](#known-limitations) - [Protocol Confusion on Negotiation Failure](#protocol-confusion-on-negotiation-failure) -- [Security Considerations](#security-considerations) - [Interaction with Inlined Muxer Negotiation](#interaction-with-inlined-muxer-negotiation) - [Implementation References](#implementation-references) @@ -40,10 +39,10 @@ Also known as "lazy multistream-select" or "lazy negotiation". In standard [multistream-select][mss] negotiation, the dialer (initiator) sends its protocol proposal and **waits** for the listener (responder) to echo the -protocol ID back before sending any application data. This costs two full -round trips before application data can flow. +protocol ID back before sending any application data. This costs one round +trip before application data can flow. -Optimistic protocol negotiation eliminates one round trip by allowing the dialer +Optimistic protocol negotiation eliminates this round trip by allowing the dialer to send the multistream-select header, the protocol proposal, **and** the initial application data all at once, without waiting for the listener's echo response. The listener's echo response arrives asynchronously while the dialer is already @@ -51,12 +50,7 @@ sending application data. This optimization is critical for latency-sensitive use cases such as [Kademlia DHT][kad-dht] operations, which use a "one stream per RPC" pattern -and would otherwise pay the full two-round-trip cost on every request. - -Both [go-libp2p] and [rust-libp2p] use optimistic negotiation in production, -but it has not previously been documented in the libp2p specifications. This -document formalizes the optimization and documents the requirements -implementations must follow to use it correctly. +and would otherwise pay the full round-trip cost on every request. ## Applicability @@ -76,9 +70,7 @@ Optimistic protocol negotiation can be applied in two contexts: ## Wire Format The wire format for individual messages is unchanged from standard -[multistream-select][mss]. Messages are UTF-8 strings, newline-terminated, and -prefixed with their length as an [unsigned varint][uvarint]. The difference is -purely in the **sequencing** of messages. +[multistream-select][mss]. ### Standard Multistream-Select @@ -89,16 +81,16 @@ before sending application data: Dialer Listener | | |--- /multistream/1.0.0 ---------------->| - |<-- /multistream/1.0.0 -----------------| + |--- /my-protocol/1.0.0 ---------------->| | | - |--- /my-protocol/1.0.0 --------------->| + |<-- /multistream/1.0.0 -----------------| |<-- /my-protocol/1.0.0 ---- (echo) ----| <- Dialer waits for echo | | |--- [application data] --------------->| <- Only then sends data | | ``` -**Cost**: 2 round trips before application data flows. +**Cost**: 1 round trip before application data flows. ### Optimistic Multistream-Select @@ -109,7 +101,7 @@ proposal, and application data without waiting: Dialer Listener | | |--- /multistream/1.0.0 ---------------->| - |--- /my-protocol/1.0.0 ---------------->| <- No wait for header echo + |--- /my-protocol/1.0.0 ---------------->| |--- [application data] ---------------->| <- Sent immediately | | |<-- /multistream/1.0.0 -----------------| <- Echo arrives later @@ -117,13 +109,13 @@ Dialer Listener | | ``` -**Cost**: 1 round trip saved. Application data is delivered to the listener +**Cost**: 0 round trips. Application data is delivered to the listener alongside the protocol proposal. The listener processes the negotiation and then delivers the application data to the protocol handler. ## Prerequisites -Optimistic protocol negotiation is inherently **best-effort**. The dialer sends +Optimistic protocol negotiation is inherently optimistic. The dialer sends application data before receiving confirmation that the listener supports the requested protocol. If the listener does not support the protocol, the negotiation will fail, and the application data sent optimistically will be @@ -141,8 +133,8 @@ supported protocols can change dynamically at any time (e.g., via In practice, protocol support is stable enough that optimistic negotiation succeeds in the vast majority of cases. -Implementations SHOULD NOT use optimistic negotiation on the **first** stream -to a peer when no prior protocol knowledge is available. +Implementations MAY skip optimistic negotiation when no prior protocol +knowledge is available. ## Requirements @@ -168,10 +160,10 @@ interpreted as described in [RFC 2119]. is a pure write/fire-and-forget pattern), the implementation MAY skip reading the handshake response. -3. **Prior knowledge**: Implementations SHOULD NOT use optimistic negotiation - without prior knowledge that the peer supports the requested protocol (e.g., - via a preceding [identify][identify] exchange). Using optimistic negotiation - without prior knowledge risks triggering the protocol confusion issue +3. **Prior knowledge**: Implementations MAY use optimistic negotiation without + prior knowledge that the peer supports the requested protocol. However, + using optimistic negotiation without prior knowledge (e.g., via a preceding + [identify][identify] exchange) risks triggering the protocol confusion issue described in [Known Limitations](#known-limitations). ### Listener (Responder) Requirements @@ -186,12 +178,6 @@ interpreted as described in [RFC 2119]. application protocol handler even if the echo write fails. The handler can then `Read()` any application data the dialer already sent. -3. **Data boundary**: After a protocol has been successfully matched during - negotiation, implementations MUST NOT interpret subsequent bytes on the - stream as multistream-select messages. All bytes after the negotiated - protocol ID are application data and MUST be forwarded to the protocol - handler. - ## Known Limitations ### Protocol Confusion on Negotiation Failure @@ -243,40 +229,11 @@ original author: - Implementations SHOULD log or track negotiation failures to detect potential protocol confusion. -## Security Considerations - -The protocol confusion issue described in [Known Limitations](#known-limitations) -represents a potential security concern. If a malicious or misconfigured peer -provides incorrect information about which protocols it supports (e.g., via -identify responses), the dialer may optimistically propose a protocol the peer -does not actually support, triggering the protocol confusion scenario. - -Implementations SHOULD: - -- Validate identify responses and consider them advisory, not authoritative. -- Track and limit the rate of negotiation failures per peer. A high failure rate - may indicate a misconfigured peer or an attack. -- Ensure that protocol handlers validate incoming data before processing it, - regardless of how the stream was negotiated. - -The security impact is limited by the fact that libp2p connections are -authenticated and encrypted (via [Noise][noise] or [TLS 1.3][tls]). An attacker -cannot inject data into an existing connection. The protocol confusion risk only -applies within an already-authenticated peer relationship, where the remote peer -provides incorrect identify information. - ## Interaction with Inlined Muxer Negotiation -For connection upgrades (as opposed to stream-level negotiation), the latency -benefit of optimistic negotiation is reduced by [inlined muxer negotiation][inlined-muxer], -which moves the stream multiplexer selection into the security handshake. When -inlined muxer negotiation is in use, the connection upgrade already avoids the -extra round trip for muxer selection, so the additional savings from optimistic -negotiation during the connection upgrade are minimal. - -Optimistic negotiation during connection upgrades is therefore most useful when -inlined muxer negotiation is **not** available (e.g., when connecting to older -peers that do not support the optimization). +For connection upgrades, [inlined muxer negotiation][inlined-muxer] is +preferred as it already eliminates the extra round trip for muxer selection +during the security handshake. ## Implementation References From 7f2227e6fb526f4916a8812ed6e3fe89a05070fb Mon Sep 17 00:00:00 2001 From: Devguru Date: Tue, 16 Jun 2026 19:57:04 +0530 Subject: [PATCH 3/3] addressing remaining comments --- connections/optimistic-multistream.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/connections/optimistic-multistream.md b/connections/optimistic-multistream.md index 1a3fccb56..a65c2a44a 100644 --- a/connections/optimistic-multistream.md +++ b/connections/optimistic-multistream.md @@ -127,7 +127,7 @@ using optimistic negotiation. The `protocols` field of the identify message (see [identify spec][identify]) contains the list of protocols the remote peer supports. -However, it is important to note that this is still *optimistic* — a peer's +However, it is important to note that this is still *optimistic* - a peer's supported protocols can change dynamically at any time (e.g., via [identify/push][identify-push] updates, or due to configuration changes). In practice, protocol support is stable enough that optimistic negotiation @@ -172,7 +172,7 @@ interpreted as described in [RFC 2119]. errors when echoing back the protocol ID during negotiation. With optimistic negotiation, the dialer may have already closed its write side of the stream after sending application data. The echo write failing does not indicate a - negotiation failure — the negotiated protocol has already been identified. + negotiation failure - the negotiated protocol has already been identified. 2. **Stream delivery**: Implementations MUST still deliver the stream to the application protocol handler even if the echo write fails. The handler can @@ -183,7 +183,7 @@ interpreted as described in [RFC 2119]. ### Protocol Confusion on Negotiation Failure There is a known soundness issue with optimistic multistream-select that arises -when the negotiation **fails** — i.e., the listener does not support the +when the negotiation **fails** - i.e., the listener does not support the proposed protocol. Consider the following scenario: @@ -205,7 +205,7 @@ multistream-select protocol proposal: ``` Peer B does not support `protocolA`, so it responds with `"na"` and reads the -next message. It sees `/protocolB/1.0.0` — which it **does** support — and +next message. It sees `/protocolB/1.0.0` - which it **does** support - and interprets it as a new protocol proposal. It echoes back `/protocolB/1.0.0` and starts interpreting `[other data]` as `protocolB` traffic.