Support graceful connection close

[oremanj]

RFC 7540 section 6.8 seems to indicate that traffic can continue flowing in both directions after a GOAWAY frame, with the only restriction being that the recipient can't open new streams after receiving the GOAWAY:

   A GOAWAY frame might not immediately precede closing of the
   connection; a receiver of a GOAWAY that has no more use for the
   connection SHOULD still send a GOAWAY frame before terminating the
   connection.
...
   Activity on streams numbered lower or equal to the last stream
   identifier might still complete successfully.  The sender of a GOAWAY
   frame might gracefully shut down a connection by sending a GOAWAY
   frame, maintaining the connection in an "open" state until all in-
   progress streams complete.

But, h2's connection state machine immediately transitions to CLOSED upon sending or receiving a GOAWAY, effectively prohibiting any further traffic. Would you be interested in a patch to fix this, or is there some disconnect I'm missing between the wording of the RFC and actual implementations?

[pgjones]

I'm not familiar enough with the code to know the answer to this, but I'm happy to review a patch if you research it.

[dimaqq]

Code in question: https://github.com/python-hyper/hyper-h2/blob/570dc7daa480d34bcf0676e88d241112c51b1796/h2/connection.py#L1825-L1844

TL;DR as far as I understand the RFC:

• having received GOAWAY with error code 0:
  • we can't create more streams
  • we are free to send and receive on the existing streams (up to GOAWAY's last stream id)
  • higher level library is free to retry requests with higher stream ids (peer either never saw them, or never parsed them)
• having receive GOAWAY with non-zero error:
  • we can't create new streams
  • we can't send more data on existing streams
  • we should probably keep the data already received for our user
  • we should treat this as ConnectionError and [dunno about response] and close the connection

Cross-link: https://github.com/encode/httpx/issues/828

[dimaqq]

Would it help to split the input two ways:

• RECV_GOAWAY for errors error_code != 0
• RECV_GRACEFUL_SHUTDOWN or RECV_LAST_STREAM_ID for error_code == 0?

[ech0-py]

+1 on this issue
Since such software like HAProxy terminates connections by sending GOAWAY first and HEADERS/DATA frames right after - the current transition states are unable to handle this.

[stephenc-pace]

Any updates on this?

[zanieb]

I've looked at fixing this and it seems like it'd be a lot of work. I'm very hesitant to dig into it without guidance from maintainers. It seems this is further complicated by two-stage GOAWAY handling but that does not appear to be well supported in general, for example https://mailman.nginx.org/pipermail/nginx-devel/2021-April/013956.html.