Skip to content

Watch every standard's spec for changes, weekly - #9

Merged
liady merged 3 commits into
mainfrom
claude/ora-ax-guide-spec-adherence-vbf8pf
Oct 20, 2026
Merged

liady merged 3 commits into
mainfrom
claude/ora-ax-guide-spec-adherence-vbf8pf

Conversation

@liady

@liady liady commented Oct 20, 2026

Copy link
Copy Markdown
Contributor

What changed

Every standard now records where its spec lives and the exact state that was read (spec_sources, with a pin per source), and a weekly workflow opens one issue per standard whose spec changed since then, closing it once the pins move. The standard's page then says "The spec changed after this check." with a link to that issue, so readers learn of a change before anyone rechecks the page.

How it fits together

  • content/standards/*.md: a new required spec_sources field (validated by lib/schema.ts). Source kinds: a file or folder in a Git repo, a repo head, the newest dated revision folder, the newest release tag, an IETF draft revision, or a hash of a page's text. The template documents each.
  • spec-pins.json: generated from those fields by npm run pins (lib/spec-pins.ts); a test fails until it matches, the same pattern as npm run readme. Ora's watcher reads this file to compare its pins with ours.
  • scripts/spec-drift.mjs: the check, Node built-ins only. It is shared byte for byte with Ora's watcher (eralabs-ai/ora), which compares the two copies weekly.
  • .github/workflows/spec-drift-watch.yml: Mondays 05:17 UTC. A read-only check job reads the upstream specs; a separate report job writes the issues and pushes spec-drift.json to a spec-drift-status branch. Upstream text never reaches a shell or an issue: only hashes, revision names, digests and plain redirect paths do.
  • app/standards/[id]/page.tsx + lib/spec-drift.ts: the page reads that status file and shows the note in the "Last checked" footer, reusing the existing "May be out of date" style. A missing or broken status shows nothing. Standard pages now regenerate hourly (revalidate = 3600) instead of only at deploy, which also keeps the 90-day "May be out of date" mark current.
  • vercel.json: turns off Vercel deployments for the spec-drift-status branch, so the bot's status commits don't start preview builds.
  • CONTRIBUTING.md: a "Spec sources" section on handling a drift issue.

What the first run will do: open one issue, for Poppy. All three Draft 0.2 pages (spec, operations extension, guides) changed after the October 19 recheck. Every other source matches its pin.

Sources

Each pin is the state at the October 19 recheck (#8):

  • Git sources (ARD, AI Catalog, WebMCP, NLWeb, auth.md, ACP, AP2, x402): each repo's head last changed before that recheck, so the live state is the reviewed state. UCP: release v2026-08-25.
  • IETF: draft-ietf-webbotauth-httpsig-protocol-00 and draft-httpauth-payment-01, the revisions the pages name.
  • Poppy: the same October 19 page digests Ora pins.

Checklist

  • Every new or changed claim links a primary source. (No page text changed; the new field points at each spec's primary source.)
  • If I changed a status, the evidence meets the rules in content/contribute.md. (No status changed.)
  • I only changed last_verified on pages where I rechecked every claim. (No last_verified changed.)
  • Plain words, no em dashes.

Verification

  • npm run typecheck, npm run lint and npm test (17 tests) pass.
  • next build: all 12 standard pages prerender with a 3600 s revalidate, and the content files are bundled for regeneration.
  • The note: a build against a local status file showed the note and issue link on a changed standard. It dropped a link that was not an issue in this repo, and changed nothing on an unchanged standard.
  • Live run: node scripts/spec-drift.mjs check spec-pins.json against the real specs.
  • Issue and branch steps: run with a stand-in gh and a throwaway Git repo. The workflow itself has not run on GitHub Actions yet.

Merge order: please merge this before the Ora PR. Ora's watcher reads spec-pins.json from main here.


Generated by Claude Code

Each standard now lists its spec sources with the exact state that was read
(spec_sources), generated into spec-pins.json. A weekly workflow compares
them with the live specs, keeps one issue per changed spec, and publishes a
status file that makes the standard's page say "The spec changed after this
check." The check is scripts/spec-drift.mjs, shared with Ora's watcher.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VeACDNmxsBifFG4eSqRsUi
- Follow a page's redirects only within its own origin, and show only a
  plain path of the redirect target in issues.
- Read page bodies in chunks under the size cap.
- Normalize pages in linear time, with output identical to before, so the
  existing pins stay valid.
- Accept only plain release tags and well-shaped peer pins.
- Run the check in a job with a read-only token; only the reporting job can
  write issues and the status branch.
- Link an issue on the page only when it is exactly an issue of this repo.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VeACDNmxsBifFG4eSqRsUi
The lockstep comparison now also flags a source the peer watches that this
repo does not. The schema rejects a tag pattern that is not a valid regular
expression, and a test runs the drift script's own manifest check on
spec-pins.json, so neither can first fail in the weekly run.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VeACDNmxsBifFG4eSqRsUi
@vercel

vercel Bot commented Oct 20, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
standards Ready Ready Preview Oct 20, 2026 5:23am UTC

Request Review

@liady
liady merged commit 6eb7f08 into main Oct 20, 2026
3 checks passed
@liady
liady deleted the claude/ora-ax-guide-spec-adherence-vbf8pf branch October 20, 2026 07:00

This branch was successfully deployed

1 active deployment
Preview — b5c9bc32 Deployed Oct 20, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants