-
Notifications
You must be signed in to change notification settings - Fork 1.2k
docs(devlog): record the 2.50.0 regression audit and release train #4196
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
12 commits
Select commit
Hold shift + click to select a range
14d307a
docs(devlog): plan the 2.50.0 regression audit and release train
lidge-jun 480ecd7
docs(devlog): fold the roadmap audit findings into the 2.50.0 plan
lidge-jun ecd27aa
docs(devlog): pin the 2.50.0 promotion source and method
lidge-jun fb2118e
docs(devlog): write the wp3 triage and remediation protocol
lidge-jun 5110586
docs(devlog): specify the six audit lane dispatch packets
lidge-jun d206d3c
docs(devlog): close the packet and triage holes the third audit round…
lidge-jun b8f6db2
docs(devlog): record the six-lane regression audit result for 2.50.0
lidge-jun aa3dfcc
docs(devlog): record the 2.50.0 release-readiness decision
lidge-jun 2cf35c8
docs(devlog): write the 2.50.0 execution runbook
lidge-jun af190ef
docs(devlog): record the 2.50.0 promotion through main
lidge-jun aeeddfc
docs(devlog): record the verified 2.50.0 release artifacts
lidge-jun 188d925
docs(devlog): close the 2.50.0 unit with the delivery record
lidge-jun File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
57 changes: 57 additions & 0 deletions
57
devlog/_plan/260910_250_regression_audit_release/000_plan.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,57 @@ | ||
| # 2.50.0 regression audit and release — scope | ||
|
|
||
| ## Baseline and candidate | ||
|
|
||
| - Released baseline: `v2.49.0`, `main` at `2f3f736299dca38861f8fb9c4326a4b4d7c664bc`. | ||
| - Audit candidate: `origin/dev` at `12c248f52bed88ea13be5b284c79a238feb592d1`, `package.json` version `2.50.0`. | ||
| - Delta: 127 commits (`git rev-list --count 2f3f73629..origin/dev`), 1066 changed files | ||
| (`git diff --name-only 2f3f73629...origin/dev | wc -l`). | ||
|
|
||
| ### File counts, by `git diff --name-only 2f3f73629...origin/dev | cut -d/ -f1 | sort | uniq -c` | ||
|
|
||
| | Group | Files | Note | | ||
| | --- | --- | --- | | ||
| | `devlog` | 885 | No runtime. Not audited. | | ||
| | `src` | 62 | Audited: L1-L6 (L4 owns `src/server/management/*`, `src/server/{management-api,auth-cors,index}.ts`, `src/service.ts`) | | ||
| | `tests` | 42 | Read as evidence by every lane, not a lane of its own | | ||
| | `docs-site` | 28 | Not release-blocking on its own | | ||
| | `gui` | 27 | Audited: L4 | | ||
| | `readme` + root docs | 13 | `readme/` 8, plus `README.md`, `AGENTS.md`, `AGENTS_INSTALL.md`, `SECURITY.md`, `package.json` | | ||
| | `skills` | 3 | Audited: L6 (`skills/ocx` surface map) | | ||
| | `.github` | 3 | PR assets only, no workflow change | | ||
| | `structure` | 2 | Maintainer invariants | | ||
| | `scripts` | 1 | `scripts/test-layout/layout.json` | | ||
|
|
||
| `git diff --shortstat 2f3f73629...origin/dev -- src gui docs-site scripts .github` is | ||
| 121 files / +3322 / -280. That figure excludes `tests` and `package.json`; the full | ||
| non-`devlog` set is 181 files. An earlier revision of this doc attributed 121 to a | ||
| different folder set and derived the devlog count by subtraction; the reviewer | ||
| contradicted both with the commands above. | ||
|
|
||
| ## What this unit does | ||
|
|
||
| Audit the product delta for release-blocking regressions, remediate anything blocking, | ||
| then run the 2.50.0 train: pre-move `dev`, promote the frozen candidate to `main`, | ||
| publish to npm, and verify the artifacts independently. | ||
|
|
||
| ## Authorization in force | ||
|
|
||
| The user authorized parallel `xai/grok-4.6` subagents, a regression-audit PABCD cycle, | ||
| and the release itself. Subagents are read-only verifiers; the main session owns every | ||
| PABCD transition, every write, and every external action. | ||
|
|
||
| ## Out of scope | ||
|
|
||
| - Landing unrelated open pull requests. 74 are open against `dev` | ||
| (`gh api "repos/lidge-jun/opencodex/pulls?state=open&base=dev&per_page=100" --jq 'length'`); | ||
| none is a release prerequisite, and pulling one in moves the candidate mid-audit. | ||
| - Re-auditing anything already released in 2.49.0. | ||
| - Any change to `devlog/` history or to third-party accounts. | ||
|
|
||
| ## Terminal outcomes | ||
|
|
||
| - `DONE` — 2.50.0 on npm `latest` with `gitHead` matching the promoted `main` SHA, a | ||
| git tag, a GitHub release, and a recorded triage for every audit finding. | ||
| - `BLOCKED` — a release-blocking regression that cannot be fixed inside this scope, | ||
| or a missing external permission (npm trusted publishing, workflow dispatch). | ||
| - `NOOP` — the candidate is already published and verified. |
98 changes: 98 additions & 0 deletions
98
devlog/_plan/260910_250_regression_audit_release/010_audit_lanes.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,98 @@ | ||
| # Audit lanes | ||
|
|
||
| Six read-only lanes, each dispatched to an independent `xai/grok-4.6` subagent with a | ||
| fresh context. Read scopes are stated per lane so a finding traces to one owner; the | ||
| lanes never write, and the main session de-duplicates the returns. | ||
|
|
||
| Every lane compares `2f3f73629...origin/dev` and must return exact `path:line` anchors. | ||
| A lane that finds nothing returns "no blocker" with the files it actually read. | ||
|
|
||
| Lane coverage is checked mechanically: every path in | ||
| `git diff --name-only 2f3f73629...origin/dev -- src gui scripts skills package.json` | ||
| belongs to at least one lane. The first revision of this map left | ||
| `src/cli/{capabilities,index,models-runtime,observe}.ts` unowned, which is why L6 exists. | ||
|
|
||
| ## L1 — Responses and request pipeline | ||
|
|
||
| `src/server/responses/{core,compact,context-overflow,policy-fallback,codex-ws-wire}.ts`, | ||
| `src/server/{chat-completions,chat-native,claude-messages,images,search,request-decompress,request-log}.ts`, | ||
| `src/claude/inbound.ts`, `src/web-search/{passthrough-bridge,ollama-executor}.ts`. | ||
|
|
||
| Highest-risk lane: the new hosted web-search bridge (`passthrough-bridge.ts` +761), the | ||
| non-streaming context-overflow classification, agent-task recovery on mid-thread model | ||
| switches, and the configurable inbound body admission limit. | ||
|
|
||
| ## L2 — Codex accounts, quota, OAuth | ||
|
|
||
| `src/codex/{account-runtime-state,account-store,account-usability,auth-api,auth-context,inject,quota,quota-auto-refresh}.ts`, | ||
| `src/oauth/{health,index,token-guardian}.ts`, `src/cli/{account,account-api,account-auth,account-extended}.ts`. | ||
|
|
||
| Deferred validation, revoked pool grants, reauth-state clearing, the new account plan | ||
| field, and the token guardian. | ||
|
|
||
| ## L3 — Catalog, providers, combos, config | ||
|
|
||
| `src/codex/catalog/{parsing,provider-fetch,sync}.ts`, `src/providers/{registry,quota,google-ai-studio-model-discovery,opencode-zen-rate-limit}.ts`, | ||
| `src/combos/{index,resolve}.ts`, `src/config.ts`, `src/types.ts`, `src/types/{accounts,config,provider}.ts`, | ||
| `src/clients/config-export/zcode.ts`, `src/lib/errors.ts`. | ||
|
|
||
| Free-model pricing classification and filtering, quota-exhausted inactive marking, AI | ||
| Studio discovery restoration, cross-provider blocked-model redirects. | ||
|
|
||
| ## L4 — Management API, service, GUI | ||
|
|
||
| `src/server/management/*`, `src/server/{management-api,auth-cors,index}.ts`, `src/service.ts`, | ||
| `gui/src/**`, `gui/tests/**`. | ||
|
|
||
| The routed-account log label, the decode-rate column, management auth, the stale launchd | ||
| bootout recovery, and nine i18n locale files that must not contradict `en`. | ||
|
|
||
| ## L5 — Security, privacy, release surface | ||
|
|
||
| Cross-cutting read of `src/lib/privacy.ts`, the body-size admission path, web-search | ||
| bridge egress, `package.json`, | ||
| `scripts/test-layout/layout.json`, `structure/{02_config-and-codex-home,04_transports-and-sidecars}.md`, | ||
| and the repository invariants in `AGENTS.md`: the Lab/core import boundary, the | ||
| synchronous `startServer` window, no tracked gitlink, and no request-body or credential | ||
| logging. | ||
|
|
||
| The email-masking opt-out is the specific item to scrutinize: it deliberately weakens a | ||
| privacy default, so it must be off by default, must survive `bun run privacy:scan`, and | ||
| its CLI application in `src/cli/index.ts` (L6) must agree with the library default. | ||
|
|
||
| ## L6 — Operator CLI surface | ||
|
|
||
| `src/cli/{capabilities,index,models-runtime,observe}.ts`, `skills/ocx/**`, and the | ||
| generated surface map that `tests/ci-workflows/skill-ocx.test.ts` asserts. | ||
|
|
||
| `src/cli/index.ts` applies `privacy.maskEmails` to `ocx status` and is a | ||
| `service-lifecycle.yml` gate path. `capabilities.ts` adds a mutating `ocx account refresh` | ||
| with a consent warning. `models-runtime.ts` adds `--free-only`, which filters on | ||
| `pricingStatus === "free"` and therefore drops entries with no status. `observe.ts` adds | ||
| `--account` log filtering. | ||
|
|
||
| ## Blocker definition | ||
|
|
||
| A finding blocks the release when any of these hold. | ||
|
|
||
| 1. **Regression against 2.49.0** — behavior that worked in the released tree and does not now. | ||
| 2. **Crash, hang, or unbounded resource use** on any path a default install can reach. | ||
| 3. **New-path functional breakage.** A feature introduced in this delta that does not do | ||
| what it claims still blocks, even though it is not a regression. This covers the | ||
| web-search bridge returning wrong or empty results, `--free-only` silently dropping | ||
| models with an absent `pricingStatus`, and a no-op `ocx account refresh`. | ||
| 4. **Security or privacy weakening**, including a default that becomes less private, a | ||
| credential or request body reaching a log, or a loosened auth boundary. | ||
| 5. **User-consent or identity-spend bypass**, per `AGENTS.md` "User-consent actions": any | ||
| path that spends the user's identity, credits, or reputation without the code-level | ||
| gate, including a CLI path that mints its own dashboard session. | ||
| 6. **Core invariant violation**, per `AGENTS.md`: a Lab import reaching `src/router.ts`, | ||
| `src/server/lifecycle.ts`, or `src/server/responses/core.ts`, an `await` inside the | ||
| synchronous `startServer` activation window, or a tracked gitlink. | ||
| 7. **Upgrade-path breakage**, not only first-run. An existing 2.49.0 install that keeps a | ||
| stale launchd job, a stale config, or a stale service unit after upgrading blocks. | ||
| 8. **Broken release, packaging, or operator-surface contract**, including a `skills/ocx` | ||
| map that names a command the registry does not have. | ||
|
|
||
| Style, missing coverage for unchanged code, and defects that already shipped in 2.49.0 | ||
| do not block; they are recorded as non-blockers with the evidence that they predate the delta. | ||
77 changes: 77 additions & 0 deletions
77
devlog/_plan/260910_250_regression_audit_release/020_release_plan.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,77 @@ | ||
| # 2.50.0 release plan | ||
|
|
||
| Derived from `.github/workflows/release.yml` as it exists on `dev`, not from precedent. | ||
| The gates below are what the workflow actually enforces. | ||
|
|
||
| ## What release.yml requires | ||
|
|
||
| | Gate | Line | Requirement | | ||
| | --- | --- | --- | | ||
| | Branch | `release.yml:153-170` | Must run from `refs/heads/main` or `refs/heads/preview`. `main` refuses any version containing `-`; `preview` refuses any version that is not `*-preview.*`. | | ||
| | dist-tag | `release.yml:174-177` | `main` -> `latest`, `preview` -> `preview`. | | ||
| | CI | `release.yml:179-197` | A **successful `ci.yml` run with `--event push` on the release branch for `$GITHUB_SHA`**. A pull-request run is explicitly rejected, and a `workflow_dispatch` run on `dev` does not qualify. | | ||
| | Service lifecycle | `release.yml:225-237` | If any of `src/service.ts`, `src/cli.ts`, `src/cli/index.ts`, `src/lib/bun-runtime.ts`, `package.json`, `bun.lock`, `service-lifecycle.yml`, `release.yml` changed since the previous tag, a successful `service-lifecycle.yml` run for `$GITHUB_SHA` is required. This delta changes `src/cli/index.ts` and `package.json`, so the gate is armed. | | ||
| | dev ahead | `release.yml:242-249` | `bun scripts/version-line.ts assert-ahead <dev version> <release version>`. `dev` is currently `2.50.0`, so publishing 2.50.0 fails until `dev` is pre-moved. | | ||
| | Publish | `release.yml:22-26` | `dry-run` defaults to **true**. A real publish needs `dry-run=false`. `expected-sha` is required and must equal the branch head at dispatch. | | ||
|
|
||
| `$GITHUB_SHA` on `main` is the **promotion merge commit**, not the frozen `dev` SHA. | ||
| 2.49.0 published from merge `2f3f73629`, not from its promoted tree commit `62849dfa6`. | ||
| Both `ci.yml` (`push: branches: [main, preview, dev]`, `paths: src/**, gui/**, ...`) and | ||
| `service-lifecycle.yml` (`push`, paths including `package.json` and `src/cli/index.ts`) | ||
| fire automatically on that merge, so the required runs appear without a dispatch — but | ||
| they must be waited for on that exact SHA. | ||
|
|
||
| ## Order | ||
|
|
||
| 1. **Freeze the candidate.** Record the exact `dev` SHA. A `workflow_dispatch` `lane=all` | ||
| run on `dev` is audit evidence for the tree, not the release gate; it tells us whether | ||
| the tree is green before we spend a promotion on it. | ||
| 2. **Land blockers first.** Any wp3 fix goes to `dev` through a pull request, which moves | ||
| the candidate. Re-freeze and re-verify on the new SHA; old-head green is not evidence. | ||
| 3. **Pre-move `dev`.** Dispatch `dev-version-bump.yml` with `intended-version=2.50.0`, | ||
| `mode=pre-move`. It is `on: workflow_dispatch`, but `dev-version-bump.yml:79` refuses | ||
| a non-default ref, so dispatch it with `--ref main`. It opens a pull request and does | ||
| **not** push to `dev`, because the `Protect dev` ruleset requires review. Merge that PR | ||
| so `dev` reads 2.51.0 before the publish reaches `assert-ahead`. Use the workflow rather | ||
| than a hand-written one-file PR so its tag/npm/version-line proofs run. | ||
| 4. **Promote the frozen SHA to `main`, not current `dev`.** After step 3, `origin/dev` is | ||
| 2.51.0 and is no longer the candidate. Promotion always names the recorded freeze SHA | ||
| explicitly. | ||
|
|
||
| The freeze SHA is **not** an ancestor of `main`, and `main` carries commits `dev` does | ||
| not, so there is nothing to fast-forward. Replicate the 2.49.0 method: branch from | ||
| `main`, merge the freeze SHA into that branch as a single | ||
| `release: promote verified 2.50.0 product tree to main` commit, then open the PR into | ||
| `main`. For 2.49.0 that was branch `codex/release-249-main-01a08498`, promote commit | ||
| `62849dfa6` (parents `9a27e8699` = old `main`, `ad36c7be8` = the dev freeze), merged by | ||
| PR #4117 as `2f3f73629`. | ||
|
|
||
| The gate on this step is **tree equality**, not a green diff: after promotion, | ||
| `git rev-parse <main merge>^{tree}` must equal `git rev-parse <freeze SHA>^{tree}`. | ||
| For 2.49.0 all three of the promote commit, the dev freeze, and the merged `main` tip | ||
| resolved to tree `66294fb3eb15592afd732f8b8e29d0bcc644fe9e`. Any conflict resolution | ||
| that changes that tree means a different product shipped than the one audited. | ||
| Record the merge SHA. | ||
| 5. **Wait for the release-branch gates on the merge SHA.** Push-event `ci.yml` and | ||
| `service-lifecycle.yml` on `main` for that exact SHA, both successful. | ||
| 6. **Dry-run, then publish.** Dispatch `release.yml` with `--ref main`, | ||
| `version=2.50.0`, `tag=latest`, `expected-sha=<merge SHA>`, first with | ||
| `dry-run=true`, then with `dry-run=false` once the dry run is green. | ||
| 7. **Verify artifacts.** `npm view @bitkyc08/opencodex dist-tags`, the `2.50.0` | ||
| `gitHead` against the promoted `main` SHA, the git tag, the GitHub release, tarball | ||
| integrity, and SLSA provenance. npm propagation lag returns 404 or a stale `latest`; | ||
| poll, never republish. | ||
| 8. **`preview` is a separate line and is not part of this stable train.** | ||
| `origin/preview` is `2.49.0-preview.20260909`, and `release.yml:161-165` refuses a | ||
| preview publish whose version is not `*-preview.*`. Promoting the plain `2.50.0` tree | ||
| onto `preview` would break that branch's version line. If `preview` should carry this | ||
| tree, it needs its own `2.50.0-preview.<date>` commit, decided after the stable | ||
| release lands. A branch sync and a preview npm publication are distinct operations. | ||
|
|
||
| ## Known failure modes to expect | ||
|
|
||
| - Branch-keyed CI concurrency cancels an older run when a newer commit lands. A cancelled | ||
| aggregate is neither a product failure nor passing evidence. | ||
| - The registry-availability smoke can time out after npm already accepted the publish. | ||
| Inspect metadata, provenance, and tarball before considering a retry. | ||
| - `dev-version-bump.yml` rejects a dispatch from a non-default ref as an early warning. |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win
Use one immutable audit candidate in both lane documents.
devlog/_plan/260910_250_regression_audit_release/070_release_execution.mdmovesdevto 2.51.0 during the release sequence. A later audit run can therefore compare 2.49.0 with 2.51.0 while recording the result as 2.50.0 evidence.devlog/_plan/260910_250_regression_audit_release/010_audit_lanes.md#L7-L8: replaceorigin/devwith the immutable freeze SHA or a shared$FREEZE_SHA.devlog/_plan/260910_250_regression_audit_release/050_lane_packets.md#L16-L17: apply the same immutable comparison to every packet.📍 Affects 2 files
devlog/_plan/260910_250_regression_audit_release/010_audit_lanes.md#L7-L8(this comment)devlog/_plan/260910_250_regression_audit_release/050_lane_packets.md#L16-L17🤖 Prompt for AI Agents