Skip to content

Commit 92f0918

Browse files
committed
Merge TASK-189: Repair WebSocket fixture capacity and lifecycle
2 parents 34c3543 + 3ffd1bc commit 92f0918

8 files changed

Lines changed: 377 additions & 17 deletions

File tree

‎.gitignore‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -138,3 +138,5 @@ __pycache__/
138138
!scripts/check-v3-runtime-coexistence.py
139139

140140
.debug/
141+
142+
!test/integ/native_websocket_regression.py
Lines changed: 62 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,62 @@
1+
# TASK-189 implementation evidence
2+
3+
Task worktree `/home/etr/progs/libhttpserver/.worktrees/TASK-189`, branch `task/TASK-189`, base `v3` at `34c354353f105bff810085c4a70092fcb487c760`. Manual Groundwork implementation; caller owns validation, finalization, merge and cleanup. Task card/index remain In Progress pending that handoff.
4+
5+
## Causal investigation and authorized scope
6+
7+
Invocation journal: `.debug/native-websocket-handshake-flood-koiz6ppd/journal.md`. Frozen TASK-185 normal and ASan/UBSan TLS-off/on raw receipts and pinned independent supplement were read without alteration. The normal/off/on and ASan/off raw failures occur inside the sequential handshake-split loop; ASan/on's recorded failure occurs during refusal probes. Independent websockets 15.0.1 completes echo/text/binary/ping/close and then times out opening `/flood`. The raw request contains valid CRLF, nonce and version headers.
8+
9+
Fresh task-local baseline reproduces the split-loop EOF. A bounded copied client prints exact request/101 bytes, split numbers and monotonic connect events; temporary engine instrumentation prints admission free space, effective limits and finalization IDs. It is removed after collecting the discriminator. Successful sessions restore the entire budget; this is not an accumulated leak. The failing overlap is explicit in `probe-instrumented.log`: upgrade 25 sees output capacity 4194304, reserves 4194048, and the next upgrade 26 sees only 256 free bytes before FINALIZE 25. It refuses `WebSocket queue storage exhausted`; the ordinary refusal cannot allocate its response either. No readiness or handshake writer repair is indicated.
10+
11+
The fresh `/flood` discriminator shows effective message limit 1039030 and logical output 1039040 despite the fixture requesting a 1048576-byte message and 1048586-byte output queue. Existing production accounting reserves four times logical output plus fixed queue/control storage against the fixture's default total 4194304-byte server output budget. The first MiB send is permanently `rejected` with `limit_exceeded`, not backpressured. The fixture treated rejection as retryable and repeatedly awaited an already-ready `writable()`, occupying route workers before 101 could flush. A copied diagnostic fixture which terminates on rejection immediately emits the complete 101 and the rejection reason. Original pinned independent failure and discriminator logs remain separate.
12+
13+
The caller explicitly amended the saved plan: correctly provision the fixture's total server storage for the original declared workload; retain every route message/queue limit, client predicate, frame count and deadline. This is authorized fixture configuration correction, not a production limit increase. Four live sessions cover the three-session isolation case and one retiring predecessor. Each input reservation is `5 * 1048576 + 16384 + 4096 + 256`. Each output reservation is `4 * 1048586 + 4096 + (8192 + 256) * 4`, with 256 bytes per handshake and 4096 bytes for ordinary HTTP response storage. Total input/output/message budgets are exactly **21054464 / 16934048 / 4194304 bytes**. Logical incoming/outgoing message counts remain one, input/message size remains one MiB and output remains one MiB plus ten frame-header bytes. Two workers and all timeout settings remain unchanged. The public low-budget negative probe still observes production admission refusal.
14+
15+
`src/detail/connection_engine.cpp`, `connection_engine_websocket.cpp` and the complete original client are byte-identical to the base; `.debug/.../unchanged-identities.json` records their hashes. No production code, ownership interface, dependency or default budget changes. Architecture updates are therefore unnecessary.
16+
17+
## RED/GREEN and fixture lifetime repair
18+
19+
The added public regression initially uses the unchanged retry loop and server storage. Both `red-rejected.log` and `red-flood.log` fail at the existing five-second socket handshake bound, exit 1; external timeout 25s is containment, not the assertion. The diagnostic copy independently establishes the exact typed rejection. The repair handles `rejected` as terminal, reports its reason and fails unexpected flood rejection instead of retrying writable. `/reject-send` intentionally sends one byte over the unchanged message limit and asserts rejection, complete valid 101 and clean Close.
20+
21+
A separate new Close-while-writable-stalled regression first fails `unexpected EOF` inside a pinned data frame (`supplement-regression.log`). The fixture returned on closed writable, destroying its owning session before the partially written frame and peer Close reply could drain. The minimal repair waits on its existing close notification gate on closed send/writable paths, as its receive path already does. `close-green.log` passes complete-frame and Close assertions. Production session/driver state is unchanged. A per-session notification witness catches duplicate callbacks; final fixture exit requires every accepted session to have exactly one callback after stop. Host readiness threads join before server destruction, while callbacks, route state and the server remain alive.
22+
23+
The supplemental public client validates the complete 101 status, Upgrade/Connection/Accept fields and absent Content-Length, Transfer-Encoding and extension headers before reading any data; the raw codec retains coalesced suffix bytes. It validates all 64 ordered one-MiB frames, observes an outstanding writable with reads paused, injects Ping and verifies Pong within three complete data frames after reads resume. Close under backpressure preserves complete pinned frames; rejection stays typed; fragmented text with interleaved Ping/Pong, exact/excess input size, parked receive cancellation, low-budget admission refusal and handler-safe request_stop are exercised. The unchanged raw client separately retains all 153 handshake boundaries, three coalesced messages, malformed handshakes, HTTP header timer removal, application return/throw, two-session isolation, writable cancellation and completed/expired/dropped drain tickets. The unchanged pinned independent flow remains a separate gate.
24+
25+
## Build and verification configuration
26+
27+
Fresh task-local normal and ASan/nonrecovering UBSan lanes, TLS-off/on, use `/usr/bin/clang-18`, `/usr/bin/clang++-18`, `-std=c++20 -pthread -O1 -g -fno-omit-frame-pointer`, and `--disable-examples`. TLS-on uses the existing `build/tooling/provider/include` and `lib` with `-lssl -lcrypto`; the provider is not rebuilt. ASan adds `-fsanitize=address,undefined -fno-sanitize-recover=undefined -fsanitize-address-use-after-scope` with `ASAN_OPTIONS=detect_leaks=1:detect_stack_use_after_return=1:halt_on_error=1` and `UBSAN_OPTIONS=halt_on_error=1:print_stacktrace=1`. Initial sandbox configure probes hit LeakSanitizer's ptrace restriction; their `initial-ptrace-config.log` remains preserved. Required sanitizer checks run outside ptrace with all checks enabled. No suppressions or diagnostic-disable options are added. No concurrent production path changed, so TSan is not a new required lane.
28+
29+
Task-local `build/task189/run_lane.py` records every configure/library/scoped-test/gate/install/consumer compile/runtime command, exit, duration and environment in each lane's `receipt.json`. Libraries build first; TESTS/check_PROGRAMS/noinst_PROGRAMS are scoped together. The complete existing 29-program H1/WS selection remains active, plus connection_engine, native_server, external_readiness_adapter, external_http1_e2e, server_budgets, consumer_v3_websocket and v3_header_hygiene (36 programs total). Both full raw and full `--independent` gate invocations run; the raw integration shell also runs the new regression. No smoke or restricted final gate substitutes for acceptance.
30+
31+
## Installed public witness and artifact ownership
32+
33+
The distributed public fixture source and public `examples/v3_host_poll.hpp` helper are copied into a standalone consumer directory. Each staged package compiles that copy against only its installed include tree and identified shared or static libhttpserver; no HTTPSERVER_COMPILATION, private headers, source includes or uninstalled v3core linkage. Compile/link commands, ldd results, library/executable and installed-header hashes are retained. Every linkage runs both managed and external readiness modes with the unchanged pinned original client and supplemental public regression. TLS-off/on denotes library configuration; the exercised loopback transport is plaintext HTTP/1, with no unexecuted wss claim.
34+
35+
Evidence root `build/task189`, journal root `.debug/native-websocket-handshake-flood-koiz6ppd`. Caller must archive both before cleanup. BSD/macOS/Windows and nonlocal backend runs remain unexecuted and CI/v3-PR owned under AGENTS.md. TASK-185's incomplete release status/frozen artifacts, TASK-200's timer teardown ownership and unrelated FEATURE work are preserved. This executor performs no validation-loop, merge, cleanup, push, tag or publication.
36+
37+
No executor memory store is configured. Reusable setup: bootstrap then out-of-tree configure; build src before scoped test targets; public copied fixtures use staged headers/lib and copied v3_host_poll.hpp; sanitizer execution requires outside-ptrace tools with leak/stack diagnostics retained. Global server storage budgets include codec/framing overhead and overlapping sessions, whereas route limits describe logical message/queue allowances. A rejected send cannot be repaired by writable; owning session lifetime must span pending close output.
38+
39+
## Final local receipts
40+
41+
All four final lane receipts pass every required local command. Each runs 36 affected programs, the complete existing H1/WS raw gate, the complete pinned independent gate, the raw integration shell including supplementary cases, installation, and shared/static installed witnesses in managed/external modes. All 16 installed linkage/mode/configuration combinations pass both unchanged original client (13 reported acceptance groups each) and supplementary client (6 groups each). These groups contain multiple assertions; they are not substituted for harness check counts. No final runtime log contains ASan, LeakSanitizer or UBSan diagnostics.
42+
43+
| Lane | Programs passing | Harness cases / checks / skips | Installed linkage and loops |
44+
|---|---:|---|---|
45+
| normal-off |36|426 /268637 /0|shared+static, managed+external pass|
46+
| normal-on |36|426 /268614 /0|shared+static, managed+external pass|
47+
| asan-off |36|426 /268630 /0|shared+static, managed+external pass|
48+
| asan-on |36|426 /268620 /0|shared+static, managed+external pass|
49+
50+
These are 30 littletest programs plus 6 standalone/compile consumers, whose separate exits are recorded. All 426 cases execute in every lane. Check-count differences occur solely in websocket_session_race's existing conditional assertions depending on which terminal-registration race wins; no skip or workload reduction. The complete upgrade suite remains 17 cases / 118 checks, including prior-HTTP-before-complete-101 ordering under partial writes, stalled101 deadlines, shutdown and queue-tail progress. Native protocol memory checks retain HTTP and WebSocket input/output plateaus and three one-second process-CPU samples per case; for example normal-off WebSocket output retains 462/462 bytes and CPU samples 0.0001314/0.0000641/0.0000736 seconds per approximately one-second wall interval. No busy-spin or unbounded queue is observed.
51+
52+
Each final installed source copy matches final repository bytes, and private fixture objects are newer than the final source. A nonfatal clock-skew warning in the ASan-off build is retained; object/source timestamps and exact source-copy comparisons prove the changed fixture was rebuilt, and all gates run its final bytes. Baseline unused-helper compiler warnings in unchanged units are retained. The initial excess-send external harness failure is preserved in normal-off/initial-excess-send-{receipt.json,regression.log}: excess length can reset TCP during sendall before its payload finishes, as well as during recv. The supplemental oracle accepts only those explicit reset/broken-pipe outcomes for that invalid message and still requires a typed terminal close and prior exact-limit echo. The original client remains byte-identical.
53+
54+
cpplint passes the changed C++ fixture; Python compilation, shell syntax and git diff --check pass. The littletest skip/result contract passes (77 all-skip, 0 mixed pass+skip, failure count fail+skip). No local required check remains incomplete. Groundwork validation, final task completion and merge/cleanup remain caller-owned.
55+
56+
`build/task189/final-summary.json` records per-program cases/checks/skips, all installed group counts, final runtime log hashes, source-copy checks and object freshness. Each receipt records complete installed header/library/executable hashes, runtime shared selection and static linkage with no libhttpserver shared dependency. `.debug/.../historical-identities.json` hashes the frozen receipts and baseline executable/library; `unchanged-identities.json` proves production engine and original client identity. Initial failed diagnostics and all RED/GREEN logs are retained separately.
57+
58+
Final changed runtime source identities:
59+
60+
- `test/integ/native_websocket_fixture.cpp`: `81808099602067ff01fa377c7fb2ba16c0c6cabb69bda3cc44c413b5fd708d17`.
61+
- `test/integ/native_websocket_client.py`: `ef94068b1152a7e8dc436b704310e3d0efdb1fd29bbc753a2986eb022dbfb33a`.
62+
- `test/integ/native_websocket_regression.py`: `381a0d0ec65687fe69eac5a5f92386fe56f045aab5e9f768271d2ebcb027ca0d`.

‎specs/tasks/M14-v3-release-repairs/TASK-189.md‎

Lines changed: 7 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -9,10 +9,10 @@
99
**Workflow:** Follow `../v3-release-repair-workflow.md`; helper-resolve TASK-189, require `base_branch=v3`, investigate and write failing regressions before production fixes, validate/merge/clean through Groundwork.
1010

1111
**Action Items:**
12-
- [ ] Read frozen H1/WebSocket and independent supplement traces. Reproduce with `native_websocket_fixture`, `native_websocket_e2e.sh` and `run-v3-http1-websocket-gates.sh --independent`. Distinguish malformed client request, accept/readiness loss, upgrade flush, admission and response flood ordering with exact wire bytes and bounded events.
13-
- [ ] Add regression for the demonstrated handshake trigger, including a client which waits for the complete 101 before consuming flood frames. Assert Sec-WebSocket-Accept, header ordering, payload/frame integrity, and that control/handshake bytes make progress while data is backpressured.
14-
- [ ] Repair the responsible owned H1/WS/transport code or invalid client fixture according to evidence. Preserve strict RFC handshake rejection, bounded output and masking rules. Reuse `connection_engine` and WS codec/session; do not substitute a test-only server.
15-
- [ ] Run real public API managed/external listener round trips, fragmentation/control/close, excess message/output limits, client cancellation and drain deadline; verify notification once and no stranded tasks/budgets. Retain the original independent predicates and timeout bounds.
12+
- [x] Read frozen H1/WebSocket and independent supplement traces. Reproduce with `native_websocket_fixture`, `native_websocket_e2e.sh` and `run-v3-http1-websocket-gates.sh --independent`. Distinguish malformed client request, accept/readiness loss, upgrade flush, admission and response flood ordering with exact wire bytes and bounded events.
13+
- [x] Add regression for the demonstrated handshake trigger, including a client which waits for the complete 101 before consuming flood frames. Assert Sec-WebSocket-Accept, header ordering, payload/frame integrity, and that control/handshake bytes make progress while data is backpressured.
14+
- [x] Repair the responsible owned H1/WS/transport code or invalid client fixture according to evidence. Preserve strict RFC handshake rejection, bounded output and masking rules. Reuse `connection_engine` and WS codec/session; do not substitute a test-only server.
15+
- [x] Run real public API managed/external listener round trips, fragmentation/control/close, excess message/output limits, client cancellation and drain deadline; verify notification once and no stranded tasks/budgets. Retain the original independent predicates and timeout bounds.
1616

1717
**Acceptance Criteria:**
1818
- Raw and pinned independent WebSocket clients complete the original handshake/flood cases, exchange validated frames and close successfully within existing deadlines.
@@ -24,4 +24,6 @@
2424
- Blocks: TASK-190, TASK-185
2525
**Related Requirements:** PRD-V3N-REQ-004, PRD-V3N-REQ-010, PRD-V3N-REQ-012, PRD-V3N-REQ-013, PRD-V3N-REQ-027, PRD-V3N-REQ-031, PRD-V3N-REQ-032, PRD-V3N-REQ-033.
2626
**Related Decisions:** DR-V3-001, DR-V3-004, DR-V3-006, DR-V3-008.
27-
**Status:** Not Started
27+
**Status:** Complete
28+
29+
**Finalization evidence:** Groundwork validation passed in one iteration with eight approvals, zero fixes and zero unworked findings (run `9e1cedb5dc61bed6acc9dedc2dae66c0`). Required local receipts and all implementation build/debug artifacts are retained in the primary worktree beneath `build/task189-evidence/implementation-e889ee0e/`, with a SHA256 archive manifest (4486 records).

‎specs/tasks/_index.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -336,7 +336,7 @@ Nominally: **13 sequential tasks**, each S–XL. Most other tasks parallelize of
336336
| TASK-186 | [Repair concurrency consumer lifetime and sanitizer-owned allocation coverage](M14-v3-release-repairs/TASK-186.md) | M14 | Complete | None |
337337
| TASK-187 | [Repair response-source and multipart coroutine lifetimes](M14-v3-release-repairs/TASK-187.md) | M14 | Complete | TASK-186 |
338338
| TASK-188 | [Eliminate peer-policy mutation starvation](M14-v3-release-repairs/TASK-188.md) | M14 | Complete | TASK-187 |
339-
| TASK-189 | [Repair native HTTP/1 WebSocket handshake and flood progress](M14-v3-release-repairs/TASK-189.md) | M14 | Not Started | TASK-188 |
339+
| TASK-189 | [Repair native HTTP/1 WebSocket handshake and flood progress](M14-v3-release-repairs/TASK-189.md) | M14 | Complete | TASK-188 |
340340
| TASK-190 | [Repair HTTP/2 conformance route-effect ordering](M14-v3-release-repairs/TASK-190.md) | M14 | Not Started | TASK-189 |
341341
| TASK-191 | [Repair independent HTTP/3 semantic drain completion](M14-v3-release-repairs/TASK-191.md) | M14 | Not Started | TASK-190 |
342342
| TASK-192 | [Supply current-v3 cookie parity witnesses](M14-v3-release-repairs/TASK-192.md) | M14 | Not Started | TASK-191 |

‎test/Makefile.am‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -753,6 +753,8 @@ websocket_progress_LDADD = $(top_builddir)/src/libhttpserver_v3core.la
753753
websocket_upgrade_decision_SOURCES = unit/websocket_upgrade_decision_test.cpp
754754
websocket_upgrade_decision_LDADD = $(top_builddir)/src/libhttpserver_v3core.la
755755
native_websocket_fixture_SOURCES = integ/native_websocket_fixture.cpp
756+
native_websocket_fixture_CPPFLAGS = $(AM_CPPFLAGS) -I$(top_srcdir)/examples
757+
EXTRA_DIST += integ/native_websocket_regression.py
756758
native_websocket_fixture_LDADD = $(top_builddir)/src/libhttpserver_v3core.la
757759
EXTRA_DIST += integ/native_websocket_client.py
758760

‎test/integ/native_websocket_e2e.sh‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,4 +2,5 @@
22
set -eu
33
# The default gate uses a stdlib-only independent wire implementation.
44
# The external websockets client is a separate explicitly provisioned lane.
5-
exec python3 "$(dirname "$0")/native_websocket_client.py" ./native_websocket_fixture
5+
python3 "$(dirname "$0")/native_websocket_client.py" ./native_websocket_fixture
6+
exec python3 "$(dirname "$0")/native_websocket_regression.py" ./native_websocket_fixture

0 commit comments

Comments
 (0)