Skip to content

fix(ci): prove load origin readiness before measured traffic - #63

Closed
seonghobae wants to merge 1 commit into
build/rust-1.98.1-vtable-fix-v1from
fix/load-origin-readiness-v1
Closed

fix(ci): prove load origin readiness before measured traffic#63
seonghobae wants to merge 1 commit into
build/rust-1.98.1-vtable-fix-v1from
fix/load-origin-readiness-v1

Conversation

@seonghobae

@seonghobae seonghobae commented Sep 6, 2026

Copy link
Copy Markdown
Contributor

Refs #58.

Executed root cause

Prior #62 exact d4a54d8d1337797c3763c3d915c674cdf483db1c exposed a real load-contract correctness RED: 5 first-second gateway 502s from Pingora Upstream ConnectRefused to 127.0.0.1:18081, followed by 395 successful requests. The unchanged sample was 400 requests / 4 VUs and p95 stayed below 20 ms, so the defect was origin-fixture startup admission rather than latency or supplier semantics.

Minimal causal repair

This branch is exact 190cd6aebc0b82b01f04b01ff095a741f831a0c8, ordinary child of #56 exact 18fb38b1ba70c4bf222642ef347f3d57a98379a2, and changes only .github/workflows/ci.yml (+14/-0). Before gateway /livez admission and k6, the workflow proves the test-only origin is directly reachable at 127.0.0.1:18081/fixture-ready and fails immediately if the fixture process exits. The probe does not traverse or warm the measured gateway /load-contract route.

The measured gateway route, exact candidate, 400 requests, 4 VUs, zero-failure thresholds, p95 <20 ms, Rust/Pingora source, dependency graph, compiler, and runtime policy remain unchanged.

Original exact GREEN

CI 34027297080 is terminal GREEN on unchanged exact head for test/load/OCI, and Supply Chain 34027296921 is terminal GREEN. Immutable artifact 9988273511, digest sha256:64500c71061e2df4a42693952fca72cdef5eac4df979b26bff3209722cf4276a, records 400/400 HTTP-200 checks and 400/400 body-identity checks, checks rate 1, http_req_failed rate 0, and http_req_duration p(95)=0.64378865 ms.

Fresh CodeRabbit review covered exact 18fb38b1...190cd6ae, selected only .github/workflows/ci.yml, and reported no actionable comments / minimal risk. This is technical review evidence, not an independent APPROVED review.

Verified complete succession — closure authority

The valid #63 delta was non-destructively adopted into #62 by exact two-parent merge 389801461e28f422c166fb0918a2805d7085d05a. Exact ancestry now proves #63 head 190cd6ae... is the merge base/ancestor of successor #62 (ahead 20 / behind 0), and compare 190cd6ae...38980146 contains only the two supplier-characterization files; .github/workflows/ci.yml has no later divergence. Conversely compare prior #62 d4a54d8d...38980146 contains exactly the +14/-0 workflow delta from this PR. The successor tree therefore carries this PR's complete valid workflow delta without force-push, destructive rebase, or partial cherry-pick.

Successor #62 exact 38980146... has now completed its own fresh hosted validation: CI 34030105237 is terminal GREEN for test 101477936704, oci-runtime 101477936789, and load-contract 101477936942; Supply Chain 34030105222 / candidate-evidence 101477936825 is terminal GREEN through committed dependency audit, exact image, SPDX SBOM, image scan, exact-source binding, and evidence upload. The exact successor load artifact 9988970232, digest sha256:781ee31569f536b3fdaef29c5c2740bbe8bd47d1fb6b0bb451fff790a50e7dc7, records 400/400 HTTP-200 and 400/400 body-identity checks, checks rate 1, http_req_failed rate 0, and http_req_duration p(95)=1.5580363 ms under the unchanged 400-request / 4-VU contract.

This PR is therefore closed only as verified complete succession into #62. No valid delta/test/fixture/contract/evidence is discarded. It is not treated as merged, and closure gives no supplier-fix, protected-main, release, canary, cutover, or legacy-removal credit.

@coderabbitai

coderabbitai Bot commented Sep 6, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Team

Run ID: 707758cd-fde3-4a7f-afdd-57b63a8bceeb

📥 Commits

Reviewing files that changed from the base of the PR and between 18fb38b and 190cd6a.

📒 Files selected for processing (1)
  • .github/workflows/ci.yml

Included review availability: Your plan provides up to 1 included review per hour; 0 remain after this review.


📝 Walkthrough

Walkthrough

load-contract 작업이 게이트웨이 상태 확인 전에 upstream fixture의 준비 상태를 확인합니다. 준비되지 않거나 프로세스가 종료되면 작업을 실패시키고, 준비되면 기존 게이트웨이 /livez 확인을 계속 수행합니다.

Changes

CI 준비 상태 확인

Layer / File(s) Summary
Upstream fixture 준비 대기
.github/workflows/ci.yml
load-contract 작업이 18081 포트의 /fixture-ready를 최대 80회 폴링합니다. upstream 프로세스가 준비 전에 종료되면 실패합니다. 준비 후 최종 readiness 확인을 수행하고 게이트웨이 /livez 확인으로 진행합니다.

Estimated code review effort: 1 (Trivial) | ~5 minutes

Merge Risk: ⚪ Minimal · up to 190cd

The CI load contract now waits for the upstream fixture before measuring gateway traffic, preventing startup-race 502s without changing application behavior or load thresholds. No merge-blocking risk remains.

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Analyzed 0 functions across 0…
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed PR 제목은 측정 트래픽 전에 load origin 준비 상태를 확인하도록 CI를 수정하는 주요 변경 사항을 정확하고 간결하게 설명합니다.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/load-origin-readiness-v1

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

Copy link
Copy Markdown
Contributor Author

@coderabbitai review

Please review exact head 190cd6aebc0b82b01f04b01ff095a741f831a0c8 against exact base 18fb38b1ba70c4bf222642ef347f3d57a98379a2, focusing on whether the direct origin-readiness probe is a bounded fixture-liveness repair rather than an application-route warm-up, whether cleanup/error propagation remains correct under set -euo pipefail, and whether the patch leaves the 400-request/4-VU/zero-failure/p95<20ms contract unchanged.

@coderabbitai

coderabbitai Bot commented Sep 6, 2026

Copy link
Copy Markdown
🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

base='18fb38b1ba70c4bf222642ef347f3d57a98379a2'
head='190cd6aebc0b82b01f04b01ff095a741f831a0c8'

echo '== Commit ancestry and changed files =='
git merge-base --is-ancestor "$base" "$head" && echo "base-is-ancestor-of-head=true"
git diff --name-status "$base" "$head"
echo
echo '== Exact diff =='
git diff --find-renames --find-copies --no-ext-diff --unified=80 "$base" "$head" -- .github/workflows/ci.yml

echo
echo '== Workflow outline and load-related sections at head =='
ast-grep outline .github/workflows/ci.yml --items all || true
rg -n -C 12 -e 'upstream_fixture|fixture-ready|livez|k6|load-contract|trap|set -e' .github/workflows/ci.yml

echo
echo '== Locate and inspect fixture and load-test definitions =='
fd -a -t f . tests/load | sort
for f in $(fd -a -t f . tests/load | sort); do
  echo
  echo "== $f =="
  case "$f" in
    *.py|*.js|*.json|*.yaml|*.yml|*.toml|*.sh)
      cat -n "$f"
      ;;
  esac
done

Length of output: 19319


@seonghobae Reviewed exact head 190cd6aebc0b82b01f04b01ff095a741f831a0c8 against exact base 18fb38b1ba70c4bf222642ef347f3d57a98379a2.

No findings.

  • The patch changes only .github/workflows/ci.yml.
  • The new probe calls 127.0.0.1:18081/fixture-ready directly. It does not call the gateway or /load-contract.
  • tests/load/upstream_fixture.py accepts this path and returns 200. The probe therefore proves that the fixture has bound and can serve HTTP.
  • The probe runs before the gateway /livez admission and before k6 run.
  • The retry loop is bounded to 80 attempts. Each attempt has --max-time 1.
  • If the fixture exits before readiness, the loop exits with status 1. If all attempts fail while the process remains alive, the final curl exits with status 1.
  • The existing EXIT trap is active before the probe. It terminates and waits for both processes. On failure, it emits both logs. The || true cleanup guards prevent cleanup errors from replacing the primary failure.
  • The measured traffic configuration is unchanged: vus: 4, iterations: 400, checks: ['rate==1'], http_req_failed: ['rate==0'], and http_req_duration: ['p(95)<20'].
  • Static review cannot verify the hosted CI result. The exact head must still run the required CI gates.
⚠️ Action not completed

Review rate limited.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

Copy link
Copy Markdown
Contributor Author

@coderabbitai review

Please review exact head 190cd6aebc0b82b01f04b01ff095a741f831a0c8. Focus on whether the direct origin-readiness probe can alter the measured gateway path or mask a runtime defect, whether failure/cleanup propagation remains fail-closed under set -euo pipefail, and whether the 400-request / 4-VU / zero-failure / p95<20ms contract is unchanged. Do not treat predecessor checks or command-analysis evidence as current formal review credit.

@coderabbitai

coderabbitai Bot commented Sep 6, 2026

Copy link
Copy Markdown
🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

base='18fb38b1ba70c4bf222642ef347f3d57a98379a2'
head='190cd6aebc0b82b01f04b01ff095a741f831a0c8'

git cat-file -e "${base}^{commit}"
git cat-file -e "${head}^{commit}"

echo '== ancestry and changed paths =='
git merge-base --is-ancestor "$base" "$head"
git diff --name-status "$base" "$head"
echo

echo '== exact workflow diff =='
git diff --no-ext-diff --unified=120 "$base" "$head" -- .github/workflows/ci.yml
echo

echo '== workflow symbols =='
ast-grep outline .github/workflows/ci.yml --items all || true
echo

echo '== relevant workflow sections at head =='
rg -n -C 30 --fixed-strings \
  -e 'set -euo pipefail' \
  -e 'upstream_fixture.py' \
  -e 'fixture-ready' \
  -e '/livez' \
  -e 'k6 run' \
  -e 'load-contract' \
  -e 'trap ' \
  .github/workflows/ci.yml
echo

echo '== fixture and k6 source locations =='
fd -a -t f . tests/load | sort
echo

echo '== fixture route implementation =='
rg -n -C 25 -P 'fixture-ready|BaseHTTPRequestHandler|do_GET|serve_forever|18081' tests/load
echo

echo '== measured load contract =='
rg -n -C 25 -P 'vus|iterations|checks|http_req_failed|http_req_duration|load-contract|http\.batch|http\.get|/load-contract' tests/load

Length of output: 24090


@seonghobae Reviewed exact head 190cd6aebc0b82b01f04b01ff095a741f831a0c8 against exact base 18fb38b1ba70c4bf222642ef347f3d57a98379a2.

No findings.

  • Only .github/workflows/ci.yml changes.
  • The readiness probe calls 127.0.0.1:18081/fixture-ready directly.
  • The probe does not call the gateway listener at 127.0.0.1:18080.
  • The probe does not call /load-contract.
  • The fixture handles the direct GET with its fixed 200 response. It has no request-dependent state. The probe cannot alter the response for measured gateway traffic.
  • The measured request remains GET ${gatewayUrl}/load-contract through the gateway.
  • The readiness loop has 80 attempts. Each request has --max-time 1.
  • If the fixture exits before readiness, kill -0 causes an explicit failure.
  • If readiness never succeeds, the final curl fails. Under set -euo pipefail, this fails the step.
  • The EXIT trap is active before the readiness probe. It terminates and waits for both child processes. The || true guards protect the original failure status during cleanup.
  • The k6 contract is unchanged: 4 VUs, 400 iterations, zero failed checks, zero failed HTTP requests, and p95 below 20 ms.

This review is static analysis of the exact Git objects. It is not hosted-CI evidence and is not formal approval credit. The exact head must still complete its required CI gates.

✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant