spec: document optimistic multistream-select (issue #643) by Devguru-codes · Pull Request #721 · libp2p/specs

Devguru-codes · 2026-05-22T11:51:35Z

spec: Document optimistic multistream-select

Closes #643

Summary

This PR adds a new Working Draft specification documenting optimistic (lazy) multistream-select — the protocol negotiation optimization already implemented in go-libp2p and rust-libp2p but never formally specified.

What is optimistic multistream-select?

When a dialer knows (via identify) that a peer supports a given protocol, it can skip waiting for the multistream-select echo and send the header, protocol ID, and application data all at once — saving one round trip.

This is critical for latency-sensitive patterns like Kademlia DHT's "one stream per RPC" model.

Changes

New file

connections/optimistic-multistream.md — Full Working Draft specification covering:
- Wire format diagrams (standard vs optimistic flow)
- Prerequisites (identify-based prior knowledge)
- Dialer requirements (ordering, handshake completion on close, prior knowledge)
- Listener requirements (tolerating echo write failures, stream delivery, data boundaries)
- Known limitations (protocol confusion on negotiation failure — go-multistream#20)
- Security considerations
- Interaction with inlined muxer negotiation

Modified file

connections/README.md — Added ToC entry, cross-reference subsection under Protocol Negotiation, and link definition

Implementation references

Reference	Description
go-multistream#115	Fix: finish reading handshake on `lazyConn` close
go-multistream#87	Fix: ignore error if can't write back echoed protocol
go-multistream#20	Issue: lazy negotiation soundness problem (protocol confusion)
go-libp2p#3038	Bug: WebTransport `StopSending` caused by incomplete lazy handshake

Add new Working Draft specification for optimistic (lazy) multistream-select protocol negotiation, documenting the behavior already implemented in go-libp2p and rust-libp2p. - New: connections/optimistic-multistream.md - Modified: connections/README.md (ToC entry + cross-reference) Refs: multiformats/go-multistream#115, multiformats/go-multistream#87, multiformats/go-multistream#20, libp2p/go-libp2p#3038

Devguru-codes · 2026-05-22T11:55:37Z

Hello @MarcoPolo, I have created a draft pr so that you can review the work. I also had some doubts while I worked on this.
Three things I'd appreciate your input on:

RTT count - The spec says standard stream-level negotiation costs "2 round trips" (header exchange → wait → protocol proposal → wait for echo). Is that accurate for the stream-level case, or do implementations pipeline the header + protocol proposal together (making it 1 RTT)?
Interest Group - We need a 3rd member per the spec lifecycle. Currently listed: you and @marten-seemann. Could you suggest someone?
General review - Does the spec accurately capture the behavior from go-multistream PRs typo. #115, gossipsub spec #87, and issue relay: add note about in-progress review and update #20?

After the review and my doubts cleared - I will do final fixes and make the pr. Thank you.

Devguru-codes · 2026-05-30T20:28:02Z

hello @MarcoPolo , i am sorry to disturb you, please review my draft and help me with my doubts. Thank you.

Devguru-codes · 2026-06-09T20:42:54Z

hello @MarcoPolo, just a ping to review my work. Thank you.

MarcoPolo · 2026-06-15T17:42:48Z

+
+In standard [multistream-select][mss] negotiation, the dialer (initiator) sends
+its protocol proposal and **waits** for the listener (responder) to echo the
+protocol ID back before sending any application data. This costs two full


Just one round trip:

A -> <multistream-header> <protocol> -> B A <- <multistream-header> <protocol> <- B A -> Application data

This was my main doubt. I have now updated it to 1 RTT count

MarcoPolo · 2026-06-15T17:44:06Z

+[multistream-select][mss]. Messages are UTF-8 strings, newline-terminated, and
+prefixed with their length as an [unsigned varint][uvarint]. The difference is
+purely in the **sequencing** of messages.
+


Suggested change

[multistream-select][mss]. Messages are UTF-8 strings, newline-terminated, and

prefixed with their length as an [unsigned varint][uvarint]. The difference is

purely in the **sequencing** of messages.

[multistream-select][mss].

MarcoPolo · 2026-06-15T17:44:18Z

+Both [go-libp2p] and [rust-libp2p] use optimistic negotiation in production,
+but it has not previously been documented in the libp2p specifications. This
+document formalizes the optimization and documents the requirements
+implementations must follow to use it correctly.
+


Suggested change

Both [go-libp2p] and [rust-libp2p] use optimistic negotiation in production,

but it has not previously been documented in the libp2p specifications. This

document formalizes the optimization and documents the requirements

implementations must follow to use it correctly.

MarcoPolo · 2026-06-15T17:45:29Z

+Dialer                                  Listener
+  |                                        |
+  |--- /multistream/1.0.0 ---------------->|
+  |<-- /multistream/1.0.0 -----------------|


Dialer does not have have to wait to hear back /multistream/1.0.0

updated it.

MarcoPolo · 2026-06-15T17:45:55Z

+
+## Prerequisites
+
+Optimistic protocol negotiation is inherently **best-effort**. The dialer sends


Suggested change

Optimistic protocol negotiation is inherently **best-effort**. The dialer sends

Optimistic protocol negotiation is inherently optimistic. The dialer sends

updated this

MarcoPolo · 2026-06-15T17:47:38Z

+Implementations SHOULD NOT use optimistic negotiation on the **first** stream
+to a peer when no prior protocol knowledge is available.


Suggested change

Implementations SHOULD NOT use optimistic negotiation on the **first** stream

to a peer when no prior protocol knowledge is available.

Implementations MAY skip optimistic negotiation when no prior protocol knowledge is available.

updated this

MarcoPolo · 2026-06-15T17:49:54Z

+   is a pure write/fire-and-forget pattern), the implementation MAY skip reading
+   the handshake response.
+
+3. **Prior knowledge**: Implementations SHOULD NOT use optimistic negotiation


SHOULD NOT is too strong I think. This can be OPTIONAL or MAY

I updated it now

MarcoPolo · 2026-06-15T17:52:39Z

+applies within an already-authenticated peer relationship, where the remote peer
+provides incorrect identify information.
+
+## Interaction with Inlined Muxer Negotiation


This section should simply say Inlined Muxer Negotiation is preferred.

MarcoPolo · 2026-06-15T17:52:54Z

+- Implementations SHOULD log or track negotiation failures to detect potential
+  protocol confusion.
+
+## Security Considerations


This section doesn't add value. Consider cutting it.

I have also removed the security considerations section here. If you want me to keep - i will re-write it again. Thank you.

- Fix RTT count: standard negotiation is 1 RTT, not 2 (Comment 1, 4) - Fix both diagrams to show header+protocol sent together - Simplify wire format section (Comment 2) - Remove meta 'not previously documented' paragraph (Comment 3) - Change 'best-effort' to 'optimistic' (Comment 5) - Soften SHOULD NOT to MAY for prior knowledge (Comment 6, 7) - Remove obvious 'Data boundary' requirement (Comment 8) - Simplify inlined muxer section (Comment 9) - Remove Security Considerations section (Comment 10)

Devguru-codes · 2026-06-16T14:29:41Z

@MarcoPolo hello, I have addressed and made the updates you suggested in latest commits - i would like you to just see whether the changes are in accordance with what you suggested. Thank you.

github-project-automation Bot added this to libp2p Specs May 22, 2026

github-project-automation Bot moved this to Triage in libp2p Specs May 22, 2026

Devguru-codes marked this pull request as draft May 22, 2026 11:51

MarcoPolo requested changes Jun 15, 2026

View reviewed changes

Devguru-codes added 2 commits June 16, 2026 19:40

addressing remaining comments

7f2227e

Devguru-codes marked this pull request as ready for review June 24, 2026 04:15

Devguru-codes requested a review from MarcoPolo June 24, 2026 04:16


		## Prerequisites

		Optimistic protocol negotiation is inherently best-effort. The dialer sends

		Implementations SHOULD NOT use optimistic negotiation on the first stream
		to a peer when no prior protocol knowledge is available.

	Implementations SHOULD NOT use optimistic negotiation on the first stream
	to a peer when no prior protocol knowledge is available.
	Implementations MAY skip optimistic negotiation when no prior protocol knowledge is available.

Uh oh!

Conversation

Devguru-codes commented May 22, 2026

spec: Document optimistic multistream-select

Summary

What is optimistic multistream-select?

Changes

New file

Modified file

Implementation references

Uh oh!

Devguru-codes commented May 22, 2026

Uh oh!

Devguru-codes commented May 30, 2026

Uh oh!

Devguru-codes commented Jun 9, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Devguru-codes commented Jun 16, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants