Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
d4f27ac
test: prove pg-erd shared logs exclude sensitive request material
seonghobae Sep 2, 2026
8fcd040
docs: record pg-erd payload-free log acceptance
seonghobae Sep 2, 2026
d45c57d
docs: record pg-erd payload-free log contract
seonghobae Sep 2, 2026
48d45ad
docs: advance pg-erd observability gap baseline
seonghobae Sep 2, 2026
5cbe2d0
test: synchronize pg-erd log capture on completion metrics
seonghobae Sep 2, 2026
a857639
test: assert sensitive material is absent from all shared log output
seonghobae Sep 2, 2026
d81a0ef
test: wait for pg-erd completion log before shutdown
seonghobae Sep 2, 2026
bd66dd8
style(test): apply rustfmt to pg-erd observability fixture
seonghobae Sep 7, 2026
fe63626
test: hold observability listener reservations together
seonghobae Sep 8, 2026
ef3b30c
test: bound pg-erd observability origin reads
seonghobae Sep 8, 2026
caa1dfb
test: match pg-erd observability headers semantically
seonghobae Sep 8, 2026
08c0bd4
test: require exact observability metric sample
seonghobae Sep 8, 2026
e204e5c
merge: restack pg-erd payload-free observability on routed load
seonghobae Sep 8, 2026
b3b045e
docs: carry payload-free observability through restack
seonghobae Sep 8, 2026
68fff09
docs: define payload-free observability acceptance
seonghobae Sep 8, 2026
7006f45
docs: advance migration baseline through payload-free observability
seonghobae Sep 8, 2026
89aa294
style: apply hosted rustfmt for payload-free observability
seonghobae Sep 8, 2026
3695288
test: preserve case-sensitive observability request target
seonghobae Sep 8, 2026
0c2cfce
test: apply rustfmt to observability request-target assertion
seonghobae Sep 8, 2026
7c8f272
test: close observability review false-greens
seonghobae Sep 8, 2026
5b58bcf
test: apply rustfmt to exact completion oracle
seonghobae Sep 8, 2026
58d767e
chore(stack): adopt repaired routed-load parent and restore baseline …
seonghobae Sep 12, 2026
a5fb000
docs: bind payload-free observability child to repaired parent evidence
seonghobae Sep 12, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ All notable changes are tracked here. No release has been published yet.
- Kept migration admin parsing side-effect free for custom TLS trust material: exact transport-authority and upstream-contract validation happens during parse, while peer/trust-bundle materialization occurs once during `build_proxy` before listener creation. This removes an avoidable validate-then-reload trust-file window.
- Added a separate Ingress forwarding-policy boundary for the pg-erd migration. Request-controlled `Forwarded`, `X-Forwarded-*` and `X-Real-IP` values are removed, then the compatibility `X-Forwarded-For`, `X-Real-IP`, `X-Forwarded-Host`, `X-Forwarded-Port` and `X-Forwarded-Proto` fields are rebuilt from accepted transport metadata. The current characterized Traefik `web` entryPoint remains explicitly HTTP; TLS-derived scheme behavior is not claimed before a TLS listener contract exists.
- Added a shared `observability` bounded context so both `GatewayProxy` and `MigrationGatewayProxy` use the same low-cardinality request/error/body/backpressure counters and coarse access-log shape instead of duplicating telemetry. The public observation vocabulary contains only response status, `ok`/`error`, and observed request-body bytes; paths, query strings, headers/cookies, credentials, customer payloads and product identifiers stay out of the shared telemetry contract.
- Added dedicated compiled pg-erd payload-free access-log acceptance. A routed request carries unique URI/query, Host, Authorization, Cookie and product-context sentinels; the backend must receive those exact fields so the fixture is non-vacuous, while shared gateway stderr must emit only the bounded completion vocabulary and none of the sentinels. Test oracles use exact case-insensitive HTTP field matching and exact Prometheus sample-line matching so `X-Forwarded-Host` cannot satisfy `Host` and counter value `10` cannot satisfy the expected value `1`.
- Added mandatory positive `max_in_flight_requests` and `upstream_keepalive_pool_size` capacity budgets; Pingora's framework keepalive default is overridden from the validated edge contract.
- Added process-local fail-fast backpressure: non-health requests above the in-flight budget receive HTTP 503, health remains observable, rejection telemetry increments, and capacity is released after request completion or failure.
- Added dedicated compiled pg-erd traffic acceptance for streamed/chunked body overflow and routed in-flight saturation/recovery: the migration process must return 413 above the shared body budget, return 503 in less than one second above the in-flight budget, keep `/readyz` observable, expose the exact single-rejection Prometheus sample, and admit a later routed request after capacity is released.
Expand All @@ -40,4 +41,4 @@ All notable changes are tracked here. No release has been published yet.
- Added missing-public-rustdoc enforcement and documentation builds with warnings denied.
- Added DDD, product, technical, security, threat, test, operability, configuration, migration-gap, and primary-source traceability documentation.

Release remains blocked on the exact Pingora supplier disposition, including unmaintained `derivative 2.2.0` / `RUSTSEC-2024-0388`, restoration of authoritative dependency-review evidence, terminal exact-current-head CI/supply-chain/security/review evidence, representative pg-erd TLS/protocol/failure/concurrency performance, immutable registry/package identity with release-bound SBOM/provenance/reproducibility, rollback rehearsal, and protected-branch integration. Current parent #21 is `4d3cf712b89a0b607db1e5f80db2d60ed29f6e1c` after ordinary ancestry/single-writer repair and must independently reacquire its exact-head evidence; the changed #22 routed-load candidate must independently acquire unchanged exact-head CI/Supply Chain and review evidence before routed p95 credit is granted. Historical `51f1242...` and `3db4fe0...` GREEN receipts do not transfer. No consumer migration, canary, cutover, rollback, or legacy removal is claimed before those release and traffic-contract gates are satisfied.
Release remains blocked on the exact Pingora supplier disposition, including unmaintained `derivative 2.2.0` / `RUSTSEC-2024-0388`, restoration of authoritative dependency-review evidence, terminal exact-current-head CI/supply-chain/security/review evidence, representative pg-erd TLS/protocol/failure/concurrency performance, immutable registry/package identity with release-bound SBOM/provenance/reproducibility, rollback rehearsal, and protected-branch integration. Current parent #22 is `d74207828f3889393f8099948101e033721f7d7c` after ordinary ancestry/single-writer repair and must independently reacquire exact-head evidence; the current #23 payload-free observability child must independently acquire exact-head hosted and review evidence as well. Historical `3db4fe0...` and `5b58bcf...` GREEN receipts do not transfer. No consumer migration, canary, cutover, rollback, or legacy removal is claimed before those release and traffic-contract gates are satisfied.
8 changes: 5 additions & 3 deletions TEST_STRATEGY.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,11 +18,13 @@ The generic `load-contract` is deliberately separate from functional production-

Runtime and failure traffic are split by causal phase. `tests/pg_erd_runtime_isolation_traffic.rs` covers streamed body overflow plus in-flight saturation/recovery and exact rejection telemetry. `tests/pg_erd_upstream_failure_traffic.rs` creates deterministic `ECONNREFUSED`, requires bounded 502 recovery, exact request-error telemetry and later frontend success. `tests/pg_erd_read_stall_traffic.rs` keeps an accepted origin connection silent and open so read inactivity cannot be confused with origin closure. `tests/pg_erd_graceful_shutdown.rs` holds a routed request in flight, fixes one SIGTERM-relative external termination deadline, releases the response during the shared grace period, requires downstream 200 and clean process exit before that same deadline.

`tests/pg_erd_partial_response_traffic.rs` covers the post-header phase. The backend sends HTTP 200 with `Content-Length: 20` and only the seven-byte `partial` prefix, then waits until the downstream has observed the complete header block plus that exact prefix before closing. Acceptance requires the committed status/framing to remain visible without a fabricated second status or silent failover, `/readyz` to stay 200, exact `cwl_pingora_gateway_request_errors_total 1`, and an independent frontend recovery request. The framing oracle parses header lines, matches `Content-Length` field identity case-insensitively, trims field-value whitespace, requires exactly one value equal to `20`, and rejects lookalike or duplicate/conflicting fields. Current #21 exact `4d3cf712b89a0b607db1e5f80db2d60ed29f6e1c` is an ordinary ancestry/single-writer repair and must independently reacquire its exact-head CI/Supply Chain/review evidence; historical `51f1242...` GREEN does not transfer.
`tests/pg_erd_partial_response_traffic.rs` covers the post-header phase. The backend sends HTTP 200 with `Content-Length: 20` and only the seven-byte `partial` prefix, then waits until the downstream has observed the complete header block plus that exact prefix before closing. Acceptance requires the committed status/framing to remain visible without a fabricated second status or silent failover, `/readyz` to stay 200, exact `cwl_pingora_gateway_request_errors_total 1`, and an independent frontend recovery request. The framing oracle parses header lines, matches `Content-Length` field identity case-insensitively, trims field-value whitespace, requires exactly one value equal to `20`, and rejects lookalike or duplicate/conflicting fields. Current #21 exact `4d3cf712b89a0b607db1e5f80db2d60ed29f6e1c` must independently reacquire exact-head CI/Supply Chain/review evidence after its ordinary repair; no historical GREEN transfers.

`tests/pg_erd_payload_free_observability.rs` covers the shared gateway observability boundary through the compiled migration process. A real routed request carries unique URI/query, Host, Authorization, Cookie and product-context sentinels; the backend must receive those values so the test cannot pass vacuously, while the `cwl_pingora_gateway::observability` target may emit only the bounded completion vocabulary and none of the sentinels. Traffic and metrics listener reservations are held concurrently until process start, origin header reads are bounded by five seconds and 64 KiB, HTTP field identity is matched case-insensitively by exact field name with OWS trimming, and the Prometheus oracle requires a complete exact sample line so a counter value such as `10` cannot satisfy an expected value of `1`. This proves only the shared gateway observability contract; it does not claim authority over product-owned logging, tracing, identity or third-party logger configuration.

## Routed performance acceptance

#22 adds the first dedicated routed pg-erd concurrency/latency gate on the current #21 ancestry. `tests/load/load_origin.rs` is the only measured origin implementation: bounded std-only Rust, finite worker/queue capacity, 64 KiB header bound, deterministic Content-Length framing and direct parser/framing tests. The CI load lane compiles both admitted gateway binaries, runs `rustfmt` and `rustc -D warnings --test` on the origin, then builds the optimized fixture used by measured traffic.
PR `#22` adds the first dedicated routed pg-erd concurrency/latency gate on the current #21 ancestry. `tests/load/load_origin.rs` is the only measured origin implementation: bounded std-only Rust, finite worker/queue capacity, 64 KiB header bound, deterministic Content-Length framing and direct parser/framing tests. The CI load lane compiles both admitted gateway binaries, runs `rustfmt` and `rustc -D warnings --test` on the origin, then builds the optimized fixture used by measured traffic.

`tests/load/pg_erd_gateway_smoke.js` runs four VUs for 400 total iterations. It alternates `/api/load-contract` and `/load-contract`, tags each request `backend` or `frontend`, requires exact 200/body identity and zero HTTP failures, gates aggregate plus each route independently at p95 `<20 ms`, and requires at least 198 measured requests per route. `tests/pg_erd_routed_latency_contract.rs` freezes the route/path/body/tag binding and per-route thresholds/sample floors. `tests/rust_load_origin_workflow_contract.rs` prevents regression to interpreted measured-origin execution by inspecting only the `load-contract` job and requiring the bounded Rust build/test/start commands with no Python invocation.

Expand All @@ -38,4 +40,4 @@ Supply-chain evidence must remain exact-source-bound and include dependency audi

## Remaining gaps

Open acceptance still includes explicit TCP reset, broader streaming and WebSocket/Upgrade failure behavior, slow-drip/whole-response lifetime, downstream TLS/H2, H2→H1 Cookie handling, Extended CONNECT, explicit H3/QUIC disposition, tracing, property/fuzz testing, representative routed TLS/origin-capacity load, shadow/canary and rollback. Current #21 and #22 exact heads must independently reacquire their hosted evidence after the ordinary restack; no historical receipt transfers. Nginx/OpenResty/legacy removal is permitted only after parity, immutable release, canary/cutover and rollback evidence are all current on protected ancestry.
Open acceptance still includes explicit TCP reset, broader streaming and WebSocket/Upgrade failure behavior, slow-drip/whole-response lifetime, downstream TLS/H2, H2→H1 Cookie handling, Extended CONNECT, explicit H3/QUIC disposition, tracing, property/fuzz testing, representative routed TLS/origin-capacity load, shadow/canary and rollback. Current #22 and #23 exact heads must independently reacquire hosted/review evidence after ordinary restack; no historical receipt transfers. Nginx/OpenResty/legacy removal is permitted only after parity, immutable release, canary/cutover and rollback evidence are all current on protected ancestry.
Loading