diff --git a/research/t488-independent-audit/README.md b/research/t488-independent-audit/README.md new file mode 100644 index 0000000000..14065f646b --- /dev/null +++ b/research/t488-independent-audit/README.md @@ -0,0 +1,279 @@ +# Independent verification of PR #3633 (T488): literal owners, fixed-inventory ceiling, staged words (no κ) + +**Object.** Open [PR #3633](https://github.com/CrocSwap/integer-mult-bounds/pull/3633) (hexapode, Pierre-Loic +Doulcet; head `72c268f113368ea0387d188225e3f81ba6d6ef73`, package `research/staged-width-t488`). It combines three +chronologies (T37 from Ross Richardson's PR #3605; T8 from hexapode's PR #3485/#3494; T16 from Nick Petalas's +PR #3413, via hexapode's PR #3457/#3465) as 8 copies of 61 base replicas (T = 488). It uses the unchanged staged +644/660 supplier words of [PR #3605](https://github.com/CrocSwap/integer-mult-bounds/pull/3605) (Ross Richardson), +and claims: +- 1,791,936 literal owner roles; +- the complete profile `{1:10826280, 2:3188592, 3:1015650, 4:447984, 5:94794, 8:35136, 48:186782}`; +- physical stock 437,757; +- an aggregate that attains the fractional packing optimum of its fixed inventory; +- κ = 0.001730096363528623534230 − 10⁻⁴⁶ − 10⁻⁷⁰. + +Ross Richardson's [PR #3630](https://github.com/CrocSwap/integer-mult-bounds/pull/3630) independently bounds that +fixed inventory below .001730096363528623534231. + +```text +Status / UTC update / priority revision: DONE (audit) · 2026-10-18 16:20 UTC · priority revision 6 · base main 20ecf289 +Concrete question: are PR #3633's allocation, complete invoice, ceiling attainment and endpoint correct when checked + by code that does not run PR3633, PR3630 or PR3605? +Result (independently checked, AI-assisted; no human review): no correctness defect in PR3633's finite claims in + the checked scope. All 1,791,936 roles pass a separate audit and the recount equals the claim. An own dual + certificate shows that no allocation, for any T, in the fixed-inventory model has a larger 24-digit endpoint, + and PR3633's profile is exactly 488 x the unique LP minimum, so it attains that ceiling. κ reproduces in two + rational enclosures. Two process defects are reported: tautological inherited controls, and platform-dependent + native pins. +Artifact: PR #3694 (https://github.com/CrocSwap/integer-mult-bounds/pull/3694). Replay: python3 -I -B research/t488-independent-audit/verify.py --fetch (212/212 controls) +Claim: released (advisory claim at discussions/1166#discussioncomment-19017329) +``` + +## What was checked + +`verify.py` and the `audit_*.py` modules import and execute nothing of PR3633, PR3630 or PR3605: their files are +read as pinned git data. The scope-C2 analyser parses copies of PR3605's auditor with `ast` as the object it +studies. The local-only `tools/` scripts run PR3633 code as the object under study, and their outputs are labelled +as local-only. + +| Scope | Check | Result | Negative controls | +| --- | --- | --- | --- | +| B `audit_owners.py` | the complete literal journal | all 1,791,936 roles pass; invoice = claim | 24 | +| A `audit_ceiling.py` | fixed-inventory LP, own simplex and dual certificate | profile = 488 × unique LP minimum | 52 | +| C1 `audit_staged_words.py` | both staged supplier words | exact forward and dual; widths = PR3633 providers | 50 | +| D `audit_endpoint.py` | endpoint, gains, PR3653 downstream | C = 0.001730096363528623534230 | 13 | +| C2 `audit_inherited_controls.py` | the five `reject()` controls of PR3605's auditor | tautological in all 15 copies | 65 | +| vendor | byte pins of this contributor's earlier modules | 8 files | 8 | +| | | **total** | **212/212** | + +- **B, the literal journal.** The journal ships as `data/FILL-columns.xz` (0.77 MB: owner, child, port and offset + columns; time and helper are implicit). The decoded rows re-serialise byte-identically to PR3633's pinned payload + (`owners/EXPECTED.json`, sha256 `20662cc6…`), so the encoding cannot change the audited object. ORDER, DISPATCH and + PROFILE are rebuilt from PR3633's base-chronology files, with each base call followed by its 8 copies (copy index + innermost), and must equal PR3633's literal-file pins. The audit covers: + - **Windows.** Each (replica, child) window is derived twice from the frozen template on main + (`rank7-joint-endframes/allocation/vendor/TEMPLATE.json.gz`), once from its level annotations and once by + replaying its event schedule. The two derivations must agree, and children of one family class have disjoint + windows. + - **Roles.** Every role must satisfy: + - `lo < t ≤ hi`; + - the active-local-pair exclusion of PR3630's checker, checked per role and also as a template lemma over 18,544 + triples, which shows the windows imply it; + - same-call physical families; + - capacity, with the widths recounted in C1; + - chronological compact offsets. + - **Invoice.** All 983,808 bins are charged, empty ones included: idle `{1:114192, 3:31842, 5:6954, 48:186782}`. + With 4,392 local bills this gives exactly the claimed profile. + - **Aggregate and stock.** The journal realises the aggregate's 11 capacity-patterns. The stock is + 896T + 408 + 8 + 91 + 2 = 437,757, with each component read from the word or the template. + - **Controls.** Controls include PR3133's widths (helpers 389/397 at 8), which do not host the journal. +- **A, the ceiling.** The model is fixed: per-call widths {4:26, 5:2, 6:18, 7:52, 8:97, 9:213}, the local bill + 2439H1+726H2+224H3+102H4+20H5+8H8, nine calls per replica, bins (672, 448, 448, 448) of capacities (8, 16, 32, 48), + live 896 per replica, and H_r(a) = r^(1−a). + - **Certificate.** Over this model, 2,551 patterns (7 + 38 + 399 + 2,107) go into an own exact simplex. Its dual + is certified rigorously: 12 reduced costs are identically zero and 2,539 are positive on [a_lo, a_hi]. + - **LP minimum.** The unique LP-minimum profile is 22185H1 + 6534H2 + (8325/4)H3 + 918H4 + (777/4)H5 + 72H8 + + (1531/4)H48 per replica, and PR3633's aggregate attains it. + - **What the ceiling covers.** It holds for every T within this model, and it relaxes windows, families, + chronology and integrality. + - **Cross-checks.** It cross-checks PR3633's `CEILING.json` (model, field hash, basis, dual, value and root + polynomial) and PR3630's model constants, basis, dual and published dual digest `11c3cade…`. + - **Independence.** A first draft mirrored PR3633's enumeration and column code. It was rewritten after review (see + the module docstring). +- **C1, the words.** These are this contributor's PR3053 word pipeline, run on the two words that are byte-identical + in PR3633 and PR3605: forward and dual replay of 632 inputs through cleanup and the target endpoint, the tracker, + the cleanup re-derivation and the 91 buffers. The helper widths equal PR3633's `PROVIDERS.json`, and they differ + from main's PR3133 providers only at helpers 389/397 (registers 644/660, 8 → 7). The change from the parent words + on main is localised: + - 8 helper→target writes are deleted, and 4 donor releases are inserted, with their 4 inverses in cleanup; + - restore 210 changes its frame only; + - the 72 warps move position only; + - `start_extra` changes. + + This contributor's parallel session audits the same two words independently, with a different code lineage, in + [PR #3687](https://github.com/CrocSwap/integer-mult-bounds/pull/3687) (14:46 UTC, same account). That audit and C1 + agree on the width inventory (8:97, 7:52) and the local bill. C1 is here to supply scope B's per-helper widths and + to bind them to PR3633's providers. +- **D, the endpoint.** Two independent enclosures accept C and reject C + 10⁻²⁴: atanh/Taylor series (PR3053) and + 640-bit fixed point (PR2804). κ = C − 10⁻⁴⁶ − 10⁻⁷⁰. The exact gains are 2.449882e-8 over PR3626, 4.881144e-8 over + PR3624, 8.967322e-8 over PR3630 and 8.981399e-8 over PR3605, and each predecessor's profile is rejected at C. + C is below PR3605's recorded BIT endpoint, which is compared, not re-derived. +- **D, downstream.** rohanarun's [PR #3653](https://github.com/CrocSwap/integer-mult-bounds/pull/3653) (head + `607a09ff`) puts roger-sato's [PR #3642](https://github.com/CrocSwap/integer-mult-bounds/pull/3642) retimed words + on this journal. Both words recount to PR3633's helper widths, with local bill {1:2437, 2:724, 3:226, …} (jwr404's + PR #3677 checks the same bill change). The profile is derived here, not read from PR3653: PR3633's audited profile + plus 4,392 × the bill change. That profile has endpoint 0.001730418899766071163804 = PR3653's κ + 10⁻⁴⁶ + 10⁻⁷⁰. + + The LP's reduced costs do not involve the local bill, and b·y = c·x holds symbolically, so only dual feasibility at + the new endpoint needed re-certifying. The scope-A dual, re-certified on that endpoint's 24-digit step (12 tight, + 2,539 positive), puts PR3633's allocation at the fixed-inventory ceiling for the retimed bill. + + This assumes PR3653 uses PR3633's journal unchanged; PR3653 pins the same `EXPECTED.json` (blob `415eac1a`), which + is not compared beyond that pin. The retimed words' exactness is not checked here; roger-sato's PR #3693 audits it. +- **C2, the inherited controls.** See finding 1. + +## Findings (non-blocking) + +1. **The literal-owner audit's five `reject()` controls cannot fail.** PR3633's `owners/independent_literal.py` + adapts PR3605's auditor: it adds input-authentication gates for artifact, source and native pins, the FILL payload + and exact ORDER/DISPATCH, and keeps its predicates and its five controls unchanged. `reject(name, test)` runs + `need(not test())`, but no `test` feeds changed data to a gate: + - three predicates are unsatisfiable: a shifted column compared with the tile, `lo < lo`, and `len({f, f}) == 2`; + - two test a +1 perturbation of an equality that the preceding gate has just asserted. + + **Where the same controls appear.** A scan of every `.py` blob in main and eight PR heads finds the same five + controls in 15 files. Static analysis (syntactic name tracking, with the remaining aliasing exclusions checked by + inspection) shows they are tautological in each: + - PR3633's own file; + - PR3605's file and its `provenance/` copies; + - PR3624's and PR3626's `independent_literal.py`, and PR3630's `independent_owners.py`; + - zxv's PR3638 (adapted); + - roger-sato's PR3642 (one adapted, two byte copies); + - rohanarun's PR3653 (one adapted, two byte copies). + + **Demonstration.** `evidence/GATE-DISABLE-DEMO.json` is local-only (Linux, CPython 3.14.4, numpy 2.5.4). In it, + PR3633's auditor ran unmodified, then with each of the five gates the controls are named after replaced by + `need(True, …)`, then with all five replaced. Every variant printed PASS and listed the same five controls. + + **What this is and is not.** It is a control-coverage defect, not a correctness defect: the gates are real checks + and pass on the real data, and scope B re-checks each one with controls that do feed violations. PR3633's other + guard tests (aggregate, stored-receipt and ceiling corruptions) do feed changed data to their checkers. Its + admission `owners` stage, which passed here, also re-checks windows, families and prefixes. + + **Lineage.** PR3605's file describes itself as an adapted copy of a parent audit (SHA256 `1001cb1c…`). No `.py` + blob in main or these heads has that hash (10,239 blobs scanned; archives not searched), so where the controls + were first written is not determined. The earlier checker of this lineage on main, + `research/rank7-joint-endframes/allocation/vendor/independent_owners.py`, credits its chronology/mask method to + Tekalign's PR2751. It reruns its `check()` on mutated inputs (a role outside its window, a same-call family + collision, a prefix hole, a changed profile, and T, role-count and dispatch corruptions) and requires the exact + rejection reason. + + **Suggested fix.** Factor the gates into a `check()` function and rerun it on each mutated journal, requiring the + expected message. +2. **Rebuilt native files fail only on the gzip header byte.** Here, PR3633's replay rebuilt both changed native + providers with PASS (Linux, CPython 3.14.4). The rebuilt `CANONICAL-WORD`, `FULL-WORD` and `LOCAL-PROGRAMS` + `.json.gz` files differ from the pinned files only in byte 9, the gzip OS field: + - FULL-WORD was compared byte by byte, because its pinned bytes ship as `supplier3605/STAGED644-660-*.json.gz`; + - the other four were compared by sha256 after setting byte 9. + + The payloads are identical. The pinned files carry 0x13; on this machine CPython 3.14.4 writes 0xff and CPython + 3.12.15 writes 0x03. + + **What the lineage already handles.** PR3605's `staged_pair.py` already restores this byte for the supplier words, + and PR3633 pins FILL by payload. `replay.py`'s native stage compares FULL-WORD by payload, which is why it passed + unpatched. + + **The remaining gap.** The case not covered is the `native_canonical` pin in `independent_literal.py` (PR3653's + adapted auditor has the same pin). It compares the rebuilt CANONICAL-WORD file hash, so a correct rebuild on + another platform fails it. The gate-disable demo fed it rebuilt files with byte 9 set to 0x13. Pinning payload + hashes would remove the gap (`evidence/NATIVE-REPRODUCTION.json`). + +## Overlap and timing + +- **Before the claim.** Overlap checks ran at 12:04 UTC (`find_overlap.py`, open PRs and discussions), and the + advisory claim was posted at 12:07 UTC (comment 19017329). + zxv's [PR #3638](https://github.com/CrocSwap/integer-mult-bounds/pull/3638) + (11:58 UTC) predates the claim and was missed by that check. It reaches the same endpoint with a different T360 + allocation and its own dual replay. It is not a check of PR3633's journal, and this package did not audit it. +- **After the claim.** Later PRs on this line: + - PR3653 (13:08 UTC, ready for review since 13:46 UTC); + - jwr404's PR3677 (the retimed bill keeps rank mass 5,135 and lowers the moment) and PR3680 (T360 role count); + - this account's PR3687 (an independent audit of PR3605's words; see C1); + - roger-sato's PR3693 (an audit of the retimed and delayed words, reusing PR3687's interpreter); + - roger-sato's PR3681 (a full replay of PR3605's wrapper, Ross Richardson's handoff); + - zxv's PR3682 (retimed releases at T360). + + None is an independent check of PR3633's journal or ceiling. The final recheck was at 2026-10-18 16:19 UTC. It + found roger-sato's PR3693 (15:33 UTC), which audits the retimed and delayed supplier words by reusing PR3687's + interpreter, and nothing on PR3633's journal or ceiling. + +## Scope limits + +- **Native lowering.** It is not checked. C1 replays the words in the lineage's free-operator model. +- **Release stages.** The precision, the prefix receipt and the physical invoice beyond the stock count are not + checked, nor the modern joint outer transfer, the bootstrap or the BIT side, which is only compared. +- **All-size hypotheses.** The inherited all-size dispatcher/compiler, uniform-cost, hosting, mutable-source, + rounding, tape, recovery and analytic hypotheses are not checked. +- **Written arguments.** PR3633's written call-copy and owner-copy (Hall) arguments are not checked; only their + output, the journal, is. +- **Model premises.** These are taken as given: the physical-family model (owner, 2·pair + side, port), as used by + the main-lineage checker, and the base-chronology files. Their equality to main's retained chronologies is not + checked, and legality does not depend on it. +- **Full replay.** PR3633's full fresh `replay.py`, which its author disclosed as not run, was attempted four times + here on 7.9 GB RAM with no swap. Fresh native passed every time. Owners passed in runs 2–4; parent-lift was + OOM-killed in run 2 and passed in runs 3–4; rebind and discovery `bind` passed in runs 3–4. Discovery `emit` was + OOM-killed in runs 1, 3 and 4, at 5.14 GB in run 4, which had no other job. So the release stage was never reached + (`evidence/NATIVE-REPRODUCTION.json`, local-only). +- **Frontier.** No selected-frontier change is proposed. κ is unchanged by this audit. + +## Reproduce + +```sh +python3 -I -B research/t488-independent-audit/verify.py --fetch # compares RESULT.json +python3 -I -B research/t488-independent-audit/verify.py --fetch --mutations --jobs 2 +python3.12 scripts/research_package.py --package research/t488-independent-audit # registered replay +``` + +`--fetch` needs git ≥ 2.29 and network access, and writes only the eight PR heads to `refs/audit/prN`. The replay +needs those heads to remain fetchable and a clone containing main `20ecf289`. The registered replay passed here with +CPython 3.12.15 (`PASS_REGISTERED_RESEARCH_PACKAGE`). RESULT.json was written with CPython 3.14.4; the +default run under CPython 3.12.15 (the CI interpreter) reproduces it exactly. Runtime is about 3 minutes on 4 cores, +with peak memory about 1.5 GB (local measurement). + +The mutation sweep runs an unmutated baseline first. It then applies each of the 45 checker mutations in a +subprocess, and each must load the mutated module and end in an audit rejection. Result: 45/45 killed, 0 survived +(`PASS_MUTATIONS`), after a passing unmutated baseline, with CPython 3.14.4 and 2 jobs, 2026-10-18 15:34–16:19 UTC, on +the final code bytes (log local-only). + +Local-only tools (not run by `verify.py` or CI): +- `tools/encode_journal.py` re-encodes the journal from PR3633's `owners/portable_expand.py` output. That output + reproduced PR3633's pinned FILL byte for byte here. +- `tools/gate_disable_demo.py` regenerates `evidence/GATE-DISABLE-DEMO.json`; it was run twice, with identical + results. + +## Open next questions + +1. **Does a width change, or a local-bill change beyond PR3642's retiming, raise the ceiling above + 0.001730418899766071163804?** That is the retimed ceiling certified here for PR3633's allocation. + - *Why it matters:* within scope A's model, packing is closed for every T. PR3633 attains the ceiling (checked + here) and PR3638 reports the same endpoint. PR3653 shows that a local-bill change with unchanged widths moves + the endpoint, with PR3633's allocation at the re-certified ceiling. + - *First step:* use `audit_ceiling.py`'s LP to price single-helper width changes and local-bill changes before + building any word. + - *Blocked by:* a width change needs a new exact word, as PR3605 built. +2. **Make the owner-audit controls test the gates.** + - *Why it matters:* finding 1 affects 15 copies of the same five controls. + - *First step:* rerun a `check()` function on mutated journals (as the main-lineage checker does), or reuse + `audit_owners.controls`. + - *Blocked by:* nothing. +3. **Make the native pins platform-independent.** + - *Why it matters:* finding 2. + - *First step:* pin the decompressed payloads of `CANONICAL-WORD` (and `FULL-WORD` and `LOCAL-PROGRAMS`) in + `independent_literal.py`'s `native_canonical` and in the receipts. + - *Blocked by:* nothing. +4. **Complete PR3633's fresh end-to-end replay.** + - *Why it matters:* its author did not run it, and here it stopped in discovery `emit` for lack of memory. + - *First step:* rerun `replay.py` with ≥ 12 GB RAM or swap. roger-sato's PR3681 reports a full PR3605-wrapper + replay, which may show a working configuration. + - *Blocked by:* memory on this machine. + +## Credits + +- **The object and its sources.** hexapode (Pierre-Loic Doulcet): the T488 composition and aggregate (PR3633), T450 + (PR3624), T530 (PR3626), the T8 chronology (PR3485/PR3494) and the T16 provider composition (PR3457/PR3465). Nick + Petalas: the T16 chronology (PR3413). Ross Richardson: the staged 644/660 supplier, the T37 lineage and the + allocation machinery (PR3605), and the all-T ceiling and dual (PR3630). +- **Downstream and concurrent work.** roger-sato: the retimed words (PR3642). rohanarun: their composition with T488 + (PR3653). zxv: the concurrent T360 allocation (PR3638). +- **Earlier work.** + - Marius-Paul Dumitrean (PR2979): the frozen template and plan of the rank7-joint-endframes package (windows, + family convention). + - dcharb78 (PR3133): the parent words and providers. + - Jeremy Diviney (PR3148, coupled-carrier-cap740): the capacity model and the finite-pattern primal/dual approach. + - The owner-checker lineage those packages name, including Tekalign's PR2751 independent owner checker. + - Clear1nsight's handoff suggested this audit. +- **Vendored code.** This contributor's own merged checkers (PR #3053 and predecessors). Their predecessors are + credited in `vendor/NOTICE.md`: Jeremy Kothe, Cinna Davis, Dugongue, Rohan Arun, mdumitrean, Vlado Vojdanovski, + Hao Yan, Josie Elliston, and icekylinx and Zhihao Chen (jacklightChen) as two contributors. +- **Author.** Sagar Thakrar (thakrarsagar), with substantial AI assistance (Claude Code, Anthropic: subagents built + two scopes and ran adversarial reviews). No independent human review is claimed. diff --git a/research/t488-independent-audit/RESULT.json b/research/t488-independent-audit/RESULT.json new file mode 100644 index 0000000000..81cb798153 --- /dev/null +++ b/research/t488-independent-audit/RESULT.json @@ -0,0 +1,2066 @@ +{ + "ceiling": { + "attainment": { + "S": 8, + "T": 488, + "base_T": 61, + "bins_per_capacity": { + "16": 218624, + "32": 218624, + "48": 218624, + "8": 327936 + }, + "equals_488_times_LP_minimum": true, + "full_profile": { + "1": 10826280, + "2": 3188592, + "3": 1015650, + "4": 447984, + "48": 186782, + "5": 94794, + "8": 35136 + }, + "groups": 549, + "idle_profile": { + "1": 114192, + "3": 31842, + "48": 186782, + "5": 6954 + }, + "nonbasic_tight_used": [ + [ + 16, + [ + 0, + 0, + 0, + 0, + 2, + 0 + ] + ] + ], + "roles_per_width": { + "4": 114192, + "5": 8784, + "6": 79056, + "7": 228384, + "8": 426024, + "9": 935496 + }, + "tight_patterns_used": 11, + "x_entries": 730, + "z_entries": 4217 + }, + "inputs": { + "budget3630": { + "commit": "bcac7fe04b9b7f7f5d3c27d620561f96c4ae5608", + "path": "research/staged-644-660-owner-successor/check_budget.py", + "sha256": "639820c9ccf72b88ae16768055ec5500a66004784f57a2fa5f218bf4d2e4ee05" + }, + "candidate": { + "commit": "72c268f113368ea0387d188225e3f81ba6d6ef73", + "path": "research/staged-width-t488/owners/CANDIDATE.json", + "sha256": "fbdda845beac5bba9baa4b1b8c8f596e98f70b59e1d60ebf4f4d3e8758631bfd" + }, + "ceiling3633": { + "commit": "72c268f113368ea0387d188225e3f81ba6d6ef73", + "path": "research/staged-width-t488/ceiling/CEILING.json", + "sha256": "a26bf26118222a5bcb6bf9f698d6b8b86003ff7b1479fd68b31f67c74945e460" + }, + "claim": { + "commit": "72c268f113368ea0387d188225e3f81ba6d6ef73", + "path": "research/staged-width-t488/CLAIM.json", + "sha256": "3332fb6f6d5ae81da8ead667973491ebefa1803d6d96a06790386ac16243f332" + }, + "expected": { + "commit": "72c268f113368ea0387d188225e3f81ba6d6ef73", + "path": "research/staged-width-t488/owners/EXPECTED.json", + "sha256": "924493b6bbeade055af4916ac7bd6d7d7ec0600719f5c39636dc3f16d7ec14ea" + }, + "readme3630": { + "commit": "bcac7fe04b9b7f7f5d3c27d620561f96c4ae5608", + "path": "research/staged-644-660-owner-successor/README.md", + "sha256": "79cf49cd2274364888830d63a664f514c7b1a51d3ac54e57dc4117d432078e31" + }, + "validation3630": { + "commit": "bcac7fe04b9b7f7f5d3c27d620561f96c4ae5608", + "path": "research/staged-644-660-owner-successor/VALIDATION.json", + "sha256": "1d85df49a5e4b9d1262cfaffa9725e03bfe660f395bc6c08d594a0b0551fc24e" + } + }, + "lp": { + "a_hi": "1730096363528623534231/1000000000000000000000000", + "basis": [ + { + "capacity": 8, + "idle": 0, + "pattern": [ + 0, + 0, + 0, + 0, + 1, + 0 + ], + "x": "672" + }, + { + "capacity": 16, + "idle": 0, + "pattern": [ + 0, + 0, + 0, + 1, + 0, + 1 + ], + "x": "448" + }, + { + "capacity": 32, + "idle": 5, + "pattern": [ + 0, + 0, + 0, + 0, + 0, + 3 + ], + "x": "57/4" + }, + { + "capacity": 32, + "idle": 0, + "pattern": [ + 0, + 0, + 0, + 1, + 2, + 1 + ], + "x": "39/2" + }, + { + "capacity": 32, + "idle": 0, + "pattern": [ + 0, + 0, + 0, + 2, + 0, + 2 + ], + "x": "1/4" + }, + { + "capacity": 32, + "idle": 0, + "pattern": [ + 0, + 0, + 1, + 0, + 1, + 2 + ], + "x": "162" + }, + { + "capacity": 32, + "idle": 0, + "pattern": [ + 0, + 1, + 0, + 0, + 0, + 3 + ], + "x": "18" + }, + { + "capacity": 32, + "idle": 1, + "pattern": [ + 1, + 0, + 0, + 0, + 0, + 3 + ], + "x": "234" + }, + { + "capacity": 48, + "idle": 48, + "pattern": [ + 0, + 0, + 0, + 0, + 0, + 0 + ], + "x": "1531/4" + }, + { + "capacity": 48, + "idle": 3, + "pattern": [ + 0, + 0, + 0, + 0, + 0, + 5 + ], + "x": "261/4" + } + ], + "dual": [ + { + "3": "-3/20", + "48": "3/20", + "5": "1/4" + }, + { + "3": "-3/10", + "48": "3/10", + "5": "1/2" + }, + { + "3": "-3/5", + "48": "3/5", + "5": "1" + }, + { + "48": "1" + }, + { + "1": "1", + "5": "-1" + }, + { + "5": "-1" + }, + { + "3": "1/20", + "48": "-1/20", + "5": "-3/4" + }, + { + "3": "1/10", + "48": "-1/10", + "5": "-1/2" + }, + { + "3": "3/20", + "48": "-3/20", + "5": "-1/4" + }, + { + "3": "1/5", + "48": "-1/5" + } + ], + "dual_uniform_on_bracket": [ + "173009636352862353423/100000000000000000000000", + "1730096363528623534231/1000000000000000000000000" + ], + "idle_per_replica": { + "1": "234", + "3": "261/4", + "48": "1531/4", + "5": "57/4" + }, + "min_strict_lower_scaled_by_73a": "4.083668e-03", + "minimum_profile_per_replica": { + "1": "22185", + "2": "6534", + "3": "8325/4", + "4": "918", + "48": "1531/4", + "5": "777/4", + "8": "72" + }, + "strictly_positive": 2539, + "symbolically_zero": 12, + "tight_patterns": [ + [ + 8, + [ + 0, + 0, + 0, + 0, + 1, + 0 + ] + ], + [ + 16, + [ + 0, + 0, + 0, + 0, + 2, + 0 + ] + ], + [ + 16, + [ + 0, + 0, + 0, + 1, + 0, + 1 + ] + ], + [ + 32, + [ + 0, + 0, + 0, + 0, + 0, + 3 + ] + ], + [ + 32, + [ + 0, + 0, + 0, + 0, + 4, + 0 + ] + ], + [ + 32, + [ + 0, + 0, + 0, + 1, + 2, + 1 + ] + ], + [ + 32, + [ + 0, + 0, + 0, + 2, + 0, + 2 + ] + ], + [ + 32, + [ + 0, + 0, + 1, + 0, + 1, + 2 + ] + ], + [ + 32, + [ + 0, + 1, + 0, + 0, + 0, + 3 + ] + ], + [ + 32, + [ + 1, + 0, + 0, + 0, + 0, + 3 + ] + ], + [ + 48, + [ + 0, + 0, + 0, + 0, + 0, + 0 + ] + ], + [ + 48, + [ + 0, + 0, + 0, + 0, + 0, + 5 + ] + ] + ] + }, + "patterns": { + "per_capacity": { + "16": 38, + "32": 399, + "48": 2107, + "8": 7 + }, + "total": 2551 + }, + "root": { + "candidate_endpoint_equals_kappa": true, + "ceiling": "for every positive integer T every complete allocation of this fixed model has M_T(a_hi) > 1, so its 24-digit endpoint is <= 0.001730096363528623534230; PR3633 attains it", + "endpoint": "0.001730096363528623534230", + "kappa": "17300963635286235342299999999999999999999998999999999999999999999999/10000000000000000000000000000000000000000000000000000000000000000000000", + "live": 437248, + "lp_minimum_per_replica_rejected_at_a_hi": true, + "lp_minimum_x488": { + "C_enclosure_width": "2.00e-60", + "C_moment_minus_1": [ + "-1.298238816e-24", + "-1.298238816e-24" + ], + "C_plus_1e24_moment_minus_1": [ + "1.538663693e-24", + "1.538663693e-24" + ] + }, + "pr3633_claimed": { + "C_enclosure_width": "2.00e-60", + "C_moment_minus_1": [ + "-1.298238816e-24", + "-1.298238816e-24" + ], + "C_plus_1e24_moment_minus_1": [ + "1.538663693e-24", + "1.538663693e-24" + ] + } + }, + "scope": "Fixed staged 644/660 supplier (widths, inventory, local bill), capacities/bin counts, nine-call normalization and live 896T; windows, families, chronology and integrality relaxed. Dual certified at a_hi and uniformly on [a_lo, a_hi] only. No literal-owner, native, outer or multiplication-theorem claim.", + "simplex_iterations": [ + 33, + 4 + ], + "status": "PASS_FIXED_MODEL_LP_CEILING_ATTAINED_BY_PR3633", + "x3630": { + "amounts_equal": true, + "basis_equal": true, + "bound_confirmed": true, + "coefficient_sha256": "11c3cadeeda1d3db40eb479af6835131f5e61bf26f63f2a62ec54de41a9f5c3a", + "digest_matches_validation": true, + "dual_equal_to_own": true, + "model_T": 740, + "model_equal": true, + "model_live": 663040, + "stated_bound": "0.001730096363528623534231", + "uniform_on_bracket": true, + "value_equal": true, + "zero_strict": [ + 12, + 2539 + ] + }, + "x3633": { + "amounts_equal": true, + "basis_equal": true, + "dual_equal": true, + "exponent": "1994665302554255138531/1152921504606846976000000", + "exponent_denominator_is_2p60_10p6": true, + "exponent_minus_a_hi": "-2.965376e-25", + "field_list_pinned": true, + "note": "their dual is certified at a rational exponent e = their root-interval upper end, with a_lo < e <= a_hi; by monotonicity rejection at e implies rejection at a_hi; their root interval is not the decimal bracket but brackets the same unique root", + "root_interval": [ + "199466530255425513853/115292150460684697600000", + "1994665302554255138531/1152921504606846976000000" + ], + "root_lower_minus_a_lo": "-1.638994e-25", + "root_polynomial_scale": 4, + "value_equal": true, + "zero_strict": { + "at_a_hi": [ + 12, + 2539 + ], + "at_their_exponent": [ + 12, + 2539 + ] + } + } + }, + "controls": { + "attempted": 212, + "by_scope": { + "ceiling": 52, + "endpoint": 13, + "inherited": 65, + "owners": 24, + "vendor": 8, + "words": 50 + }, + "distinct": 212, + "rejected_as_expected": 212, + "rules": [ + "VendorPinMismatch", + "ceiling.attain_bins", + "ceiling.attain_format", + "ceiling.attain_model", + "ceiling.attain_pattern", + "ceiling.attain_profile", + "ceiling.attain_roles", + "ceiling.attain_tight", + "ceiling.attain_window", + "ceiling.attain_zbalance", + "ceiling.attain_zcalls", + "ceiling.attain_zdegree", + "ceiling.claim_fields", + "ceiling.claim_interval", + "ceiling.claimed_profile", + "ceiling.dual_feasible", + "ceiling.dual_uniform", + "ceiling.endpoint_format", + "ceiling.enum_complete", + "ceiling.enum_distinct", + "ceiling.enum_valid", + "ceiling.input_format", + "ceiling.kappa", + "ceiling.primal_rows", + "ceiling.primal_sign", + "ceiling.root_accept", + "ceiling.root_ceiling", + "ceiling.root_reject", + "ceiling.strong_duality", + "ceiling.x3630_basis", + "ceiling.x3630_bound", + "ceiling.x3630_digest", + "ceiling.x3630_dual", + "ceiling.x3630_model", + "ceiling.x3630_value", + "ceiling.x3633_basis", + "ceiling.x3633_binding", + "ceiling.x3633_dual", + "ceiling.x3633_dual_value", + "ceiling.x3633_fields", + "ceiling.x3633_model", + "ceiling.x3633_root", + "ceiling.x3633_rootpoly", + "ceiling.x3633_value", + "controls.bindings", + "controls.data_identity", + "controls.data_mass", + "controls.data_pin", + "controls.data_prefix", + "controls.data_rows", + "controls.data_window", + "controls.gate_purity", + "controls.inventory", + "controls.need_def", + "controls.pattern", + "controls.pin", + "controls.reject_def", + "controls.tautology", + "downstream.ceiling", + "downstream.dual", + "downstream.kappa", + "downstream.widths", + "endpoint.accept", + "endpoint.bit", + "endpoint.gain", + "endpoint.improvement", + "endpoint.kappa", + "endpoint.profile", + "endpoint.reject", + "endpoint.two_impl", + "owners.active_pair", + "owners.aggregate", + "owners.binding", + "owners.bounds", + "owners.capacity", + "owners.dispatch", + "owners.family", + "owners.histogram", + "owners.identity", + "owners.literal_files", + "owners.order", + "owners.prefix", + "owners.schema", + "owners.shape", + "owners.stock", + "owners.window", + "owners.windows", + "owners.windows_agree", + "owners.windows_disjoint", + "staged.parent_diff.cleanup_pairs", + "staged.parent_diff.deleted", + "staged.parent_diff.inserted", + "staged.parent_diff.keys", + "staged.parent_diff.replace", + "staged.parent_diff.restore", + "staged.parent_diff.sections", + "staged.parent_diff.start_extra", + "staged.parent_diff.warp_alignment", + "staged.parent_diff.warp_record", + "staged.parent_diff.warps", + "staged.pin", + "staged.pr3133_widths", + "staged.providers", + "staged.providers_paid", + "staged.registers", + "wc.replay", + "words.cleanup", + "words.counts", + "words.declared" + ] + }, + "downstream_pr3653": { + "delta_per_call": { + "1": -2, + "2": -2, + "3": 2 + }, + "dual_certificate_at_retimed_endpoint": { + "interval": [ + "0.001730418899766071163804", + "0.001730418899766071163805" + ], + "strictly_positive_uniform": 2539, + "tight": 12 + }, + "endpoint": { + "C": "0.001730418899766071163804", + "M_C_minus_1": [ + "-2.0582e-24", + "-2.0582e-24" + ], + "M_next_minus_1": [ + "7.7817e-25", + "7.7817e-25" + ], + "vendored_agrees": true + }, + "kappa": "17304188997660711638039999999999999999999998999999999999999999999999/10000000000000000000000000000000000000000000000000000000000000000000000", + "profile": { + "1": 10817496, + "2": 3179808, + "3": 1024434, + "4": 447984, + "48": 186782, + "5": 94794, + "8": 35136 + }, + "profile_equals_488_retimed_lp_minimum": true, + "retimed_local_bill": { + "1": 2437, + "2": 724, + "3": 226, + "4": 102, + "5": 20, + "8": 8 + }, + "widths_equal_staged": true + }, + "endpoint": { + "bit": { + "complex_binds": true, + "source": "PR3605 CLAIM.json (recorded, not re-derived)", + "unchanged_bit_endpoint": "0.001733427039589844234749" + }, + "endpoint": { + "C": "0.001730096363528623534230", + "M_C_minus_1": [ + "-1.2982e-24", + "-1.2982e-24" + ], + "M_next_minus_1": [ + "1.5387e-24", + "1.5387e-24" + ], + "vendored_agrees": true + }, + "gains": { + "PR3605": { + "gain": "3592559591267489/40000000000000000000000", + "gain_float": "8.981399e-08", + "kappa": "17300065495388418470049999999999999999999998999999999999999999999999/10000000000000000000000000000000000000000000000000000000000000000000000", + "predecessor_M_at_C_minus_1": "2.548e-07", + "replicas": 740 + }, + "PR3624": { + "gain": "24405717639687323/500000000000000000000000", + "gain_float": "4.881144e-08", + "kappa": "17300475520933441595839999999999999999999998999999999999999999999999/10000000000000000000000000000000000000000000000000000000000000000000000", + "predecessor_M_at_C_minus_1": "1.385e-07", + "replicas": 450 + }, + "PR3626": { + "gain": "2449882284365169/100000000000000000000000", + "gain_float": "2.449882e-08", + "kappa": "17300718647057798825399999999999999999999998999999999999999999999999/10000000000000000000000000000000000000000000000000000000000000000000000", + "predecessor_M_at_C_minus_1": "6.950e-08", + "replicas": 530 + }, + "PR3630": { + "gain": "5604576079666369/62500000000000000000000", + "gain_float": "8.967322e-08", + "kappa": "17300066903113488723259999999999999999999998999999999999999999999999/10000000000000000000000000000000000000000000000000000000000000000000000", + "predecessor_M_at_C_minus_1": "2.544e-07", + "replicas": 740 + } + }, + "kappa": "17300963635286235342299999999999999999999998999999999999999999999999/10000000000000000000000000000000000000000000000000000000000000000000000" + }, + "inherited_controls": { + "controls_attempted": 65, + "controls_killed": 65, + "copies": { + "PR3605": { + "all_tautological": true, + "commit": "3efbb5de06a49ffeea6684482f2d3fe48cdca3fb", + "gates_before_controls": 26, + "gates_missing_here_vs_PR3633": [ + "'frozen audit artifact ' + rel", + "'retained source pin ' + rel", + "actual native canonical pin", + "exactmixedsourceclock", + "exactsourceproviderdispatch", + "freshT488payload" + ], + "gates_only_here_vs_PR3633": [], + "git_blob": "5d1a7b4fd635070522e78656ccedf9bcf4bf13db", + "original_sha256_claimed": "1001cb1c8b4ee1f93c23b58b094565ec1b3931eee0ea091a23504696e380be9e", + "path": "research/staged-644-660-conditional-release/owners/independent_literal.py", + "patterns": [ + "shifted_column", + "irreflexive", + "duplicate_set", + "gate_offset", + "gate_bump" + ], + "sha256": "da897bd5cef36943b3f5b1422fe81716b4eaa6230b056d31f8c2310cf62f8580" + }, + "PR3624": { + "all_tautological": true, + "commit": "515a7c4490aa32ac3ca5014feb6f7da0d6952863", + "gates_before_controls": 32, + "gates_missing_here_vs_PR3633": [ + "freshT488payload" + ], + "gates_only_here_vs_PR3633": [ + "freshT450payload" + ], + "git_blob": "d9c2450e03d9dcde39ae96f0edc835d1a67d9fe5", + "original_sha256_claimed": "1001cb1c8b4ee1f93c23b58b094565ec1b3931eee0ea091a23504696e380be9e", + "path": "research/staged-width-t450/owners/independent_literal.py", + "patterns": [ + "shifted_column", + "irreflexive", + "duplicate_set", + "gate_offset", + "gate_bump" + ], + "sha256": "68684584d3866b2ab76c254dd6d5be3a49d276a6e5f6353183dee4d8d84de4d4" + }, + "PR3624-provenance": { + "all_tautological": true, + "commit": "515a7c4490aa32ac3ca5014feb6f7da0d6952863", + "gates_before_controls": 26, + "gates_missing_here_vs_PR3633": [ + "'frozen audit artifact ' + rel", + "'retained source pin ' + rel", + "actual native canonical pin", + "exactmixedsourceclock", + "exactsourceproviderdispatch", + "freshT488payload" + ], + "gates_only_here_vs_PR3633": [], + "git_blob": "5d1a7b4fd635070522e78656ccedf9bcf4bf13db", + "original_sha256_claimed": "1001cb1c8b4ee1f93c23b58b094565ec1b3931eee0ea091a23504696e380be9e", + "path": "research/staged-width-t450/provenance/pr3605/owners/independent_literal.py", + "patterns": [ + "shifted_column", + "irreflexive", + "duplicate_set", + "gate_offset", + "gate_bump" + ], + "sha256": "da897bd5cef36943b3f5b1422fe81716b4eaa6230b056d31f8c2310cf62f8580" + }, + "PR3626": { + "all_tautological": true, + "commit": "659fd5e7064d9466b668d7188cbb013b333a415a", + "gates_before_controls": 32, + "gates_missing_here_vs_PR3633": [ + "freshT488payload" + ], + "gates_only_here_vs_PR3633": [ + "freshT530payload" + ], + "git_blob": "32755776c15660b939c92cb8835c71ec87c8c1d6", + "original_sha256_claimed": "1001cb1c8b4ee1f93c23b58b094565ec1b3931eee0ea091a23504696e380be9e", + "path": "research/staged-width-t530/owners/independent_literal.py", + "patterns": [ + "shifted_column", + "irreflexive", + "duplicate_set", + "gate_offset", + "gate_bump" + ], + "sha256": "36d7f4e3a611fd6b5983222887beb9a37a18bfac0d999ac5c00e3e191d1c5cc4" + }, + "PR3626-provenance": { + "all_tautological": true, + "commit": "659fd5e7064d9466b668d7188cbb013b333a415a", + "gates_before_controls": 26, + "gates_missing_here_vs_PR3633": [ + "'frozen audit artifact ' + rel", + "'retained source pin ' + rel", + "actual native canonical pin", + "exactmixedsourceclock", + "exactsourceproviderdispatch", + "freshT488payload" + ], + "gates_only_here_vs_PR3633": [], + "git_blob": "5d1a7b4fd635070522e78656ccedf9bcf4bf13db", + "original_sha256_claimed": "1001cb1c8b4ee1f93c23b58b094565ec1b3931eee0ea091a23504696e380be9e", + "path": "research/staged-width-t530/provenance/pr3605/owners/independent_literal.py", + "patterns": [ + "shifted_column", + "irreflexive", + "duplicate_set", + "gate_offset", + "gate_bump" + ], + "sha256": "da897bd5cef36943b3f5b1422fe81716b4eaa6230b056d31f8c2310cf62f8580" + }, + "PR3630": { + "all_tautological": true, + "commit": "bcac7fe04b9b7f7f5d3c27d620561f96c4ae5608", + "gates_before_controls": 27, + "gates_missing_here_vs_PR3633": [ + "'frozen audit artifact ' + rel", + "'retained source pin ' + rel", + "actual native canonical pin", + "exactmixedsourceclock", + "exactsourceproviderdispatch", + "freshT488payload" + ], + "gates_only_here_vs_PR3633": [ + "'unchanged ' + n", + "24 total changed literal rows", + "active local pair excluded from helper owners", + "all call/helper identities unchanged", + "complete predecessor literal payload", + "exact five changed owners", + "independently derived whole histogram delta", + "literal destination binding" + ], + "git_blob": "f116cc0142b4f6d674abf6e4c844142c00fb0297", + "original_sha256_claimed": null, + "path": "research/staged-644-660-owner-successor/independent_owners.py", + "patterns": [ + "shifted_column", + "irreflexive", + "duplicate_set", + "gate_offset", + "gate_bump" + ], + "sha256": "8d02dff267938cdb373aa617a411d8e67700e130609d2714e98ccf814e32eaef" + }, + "PR3633": { + "all_tautological": true, + "commit": "72c268f113368ea0387d188225e3f81ba6d6ef73", + "gates_before_controls": 32, + "gates_missing_here_vs_PR3633": [], + "gates_only_here_vs_PR3633": [], + "git_blob": "e2a226950ed30e88f6c386669716b9a463dd873d", + "original_sha256_claimed": "1001cb1c8b4ee1f93c23b58b094565ec1b3931eee0ea091a23504696e380be9e", + "path": "research/staged-width-t488/owners/independent_literal.py", + "patterns": [ + "shifted_column", + "irreflexive", + "duplicate_set", + "gate_offset", + "gate_bump" + ], + "sha256": "8517c0d6c7cbe73438885135ac7fd9781dd5120113c9721002346dca603f5ed9" + }, + "PR3633-provenance": { + "all_tautological": true, + "commit": "72c268f113368ea0387d188225e3f81ba6d6ef73", + "gates_before_controls": 26, + "gates_missing_here_vs_PR3633": [ + "'frozen audit artifact ' + rel", + "'retained source pin ' + rel", + "actual native canonical pin", + "exactmixedsourceclock", + "exactsourceproviderdispatch", + "freshT488payload" + ], + "gates_only_here_vs_PR3633": [], + "git_blob": "5d1a7b4fd635070522e78656ccedf9bcf4bf13db", + "original_sha256_claimed": "1001cb1c8b4ee1f93c23b58b094565ec1b3931eee0ea091a23504696e380be9e", + "path": "research/staged-width-t488/provenance/pr3605/owners/independent_literal.py", + "patterns": [ + "shifted_column", + "irreflexive", + "duplicate_set", + "gate_offset", + "gate_bump" + ], + "sha256": "da897bd5cef36943b3f5b1422fe81716b4eaa6230b056d31f8c2310cf62f8580" + }, + "PR3638": { + "all_tautological": true, + "commit": "d62e1acd7518a231440383f0d0dddf54d6edf8c1", + "gates_before_controls": 32, + "gates_missing_here_vs_PR3633": [ + "freshT488payload" + ], + "gates_only_here_vs_PR3633": [ + "freshT360payload" + ], + "git_blob": "4028493716bfcf3e73831067599518c055fcd2b0", + "original_sha256_claimed": "1001cb1c8b4ee1f93c23b58b094565ec1b3931eee0ea091a23504696e380be9e", + "path": "research/three-kappa-leads/independent_literal.py", + "patterns": [ + "shifted_column", + "irreflexive", + "duplicate_set", + "gate_offset", + "gate_bump" + ], + "sha256": "89c27f61ae3e6d2bdd4c469ac5b99d4f955a443c5e04c74e568c12093510efbe" + }, + "PR3642": { + "all_tautological": true, + "commit": "044d24f12e74f228dd462c07ae2d7f3ea35e2be9", + "gates_before_controls": 32, + "gates_missing_here_vs_PR3633": [ + "freshT488payload", + "new local bill" + ], + "gates_only_here_vs_PR3633": [ + "freshT530payload", + "two commuting release swaps local bill" + ], + "git_blob": "86dc50768ff3fda8e265c08aa9314fe072e0c396", + "original_sha256_claimed": "1001cb1c8b4ee1f93c23b58b094565ec1b3931eee0ea091a23504696e380be9e", + "path": "research/staged-release-retiming-t530/owner-audit/independent_literal.py", + "patterns": [ + "shifted_column", + "irreflexive", + "duplicate_set", + "gate_offset", + "gate_bump" + ], + "sha256": "b51c36ad1ae0999547edc4726d698df369fef60daaaac4f5bbea471648c9f628" + }, + "PR3642-upstream": { + "all_tautological": true, + "commit": "044d24f12e74f228dd462c07ae2d7f3ea35e2be9", + "gates_before_controls": 32, + "gates_missing_here_vs_PR3633": [ + "freshT488payload" + ], + "gates_only_here_vs_PR3633": [ + "freshT530payload" + ], + "git_blob": "32755776c15660b939c92cb8835c71ec87c8c1d6", + "original_sha256_claimed": "1001cb1c8b4ee1f93c23b58b094565ec1b3931eee0ea091a23504696e380be9e", + "path": "research/staged-release-retiming-t530/owner-audit/upstream/independent_literal.py", + "patterns": [ + "shifted_column", + "irreflexive", + "duplicate_set", + "gate_offset", + "gate_bump" + ], + "sha256": "36d7f4e3a611fd6b5983222887beb9a37a18bfac0d999ac5c00e3e191d1c5cc4" + }, + "PR3642-vendor": { + "all_tautological": true, + "commit": "044d24f12e74f228dd462c07ae2d7f3ea35e2be9", + "gates_before_controls": 32, + "gates_missing_here_vs_PR3633": [ + "freshT488payload" + ], + "gates_only_here_vs_PR3633": [ + "freshT530payload" + ], + "git_blob": "32755776c15660b939c92cb8835c71ec87c8c1d6", + "original_sha256_claimed": "1001cb1c8b4ee1f93c23b58b094565ec1b3931eee0ea091a23504696e380be9e", + "path": "research/staged-release-retiming-t530/full-admission/vendor/staged-width-t530/owners/independent_literal.py", + "patterns": [ + "shifted_column", + "irreflexive", + "duplicate_set", + "gate_offset", + "gate_bump" + ], + "sha256": "36d7f4e3a611fd6b5983222887beb9a37a18bfac0d999ac5c00e3e191d1c5cc4" + }, + "PR3653": { + "all_tautological": true, + "commit": "607a09ff441c79b215a8127a81f4cc70021d0d9b", + "gates_before_controls": 32, + "gates_missing_here_vs_PR3633": [], + "gates_only_here_vs_PR3633": [], + "git_blob": "403a9cd104c7e68a5bd2431bb442d701f30828f5", + "original_sha256_claimed": "1001cb1c8b4ee1f93c23b58b094565ec1b3931eee0ea091a23504696e380be9e", + "path": "research/retimed-t488-composition/owners/independent_literal.py", + "patterns": [ + "shifted_column", + "irreflexive", + "duplicate_set", + "gate_offset", + "gate_bump" + ], + "sha256": "caa5eb4fff4908c0891c3ee41e7e6d71bf19bb710d1d7781164f10ce84bce769" + }, + "PR3653-upstream": { + "all_tautological": true, + "commit": "607a09ff441c79b215a8127a81f4cc70021d0d9b", + "gates_before_controls": 32, + "gates_missing_here_vs_PR3633": [], + "gates_only_here_vs_PR3633": [], + "git_blob": "e2a226950ed30e88f6c386669716b9a463dd873d", + "original_sha256_claimed": "1001cb1c8b4ee1f93c23b58b094565ec1b3931eee0ea091a23504696e380be9e", + "path": "research/retimed-t488-composition/owners/upstream/independent_literal.py", + "patterns": [ + "shifted_column", + "irreflexive", + "duplicate_set", + "gate_offset", + "gate_bump" + ], + "sha256": "8517c0d6c7cbe73438885135ac7fd9781dd5120113c9721002346dca603f5ed9" + }, + "PR3653-vendor": { + "all_tautological": true, + "commit": "607a09ff441c79b215a8127a81f4cc70021d0d9b", + "gates_before_controls": 32, + "gates_missing_here_vs_PR3633": [ + "freshT488payload" + ], + "gates_only_here_vs_PR3633": [ + "freshT530payload" + ], + "git_blob": "32755776c15660b939c92cb8835c71ec87c8c1d6", + "original_sha256_claimed": "1001cb1c8b4ee1f93c23b58b094565ec1b3931eee0ea091a23504696e380be9e", + "path": "research/retimed-t488-composition/full-replay/vendor/pr3642/vendor/staged-width-t530/owners/independent_literal.py", + "patterns": [ + "shifted_column", + "irreflexive", + "duplicate_set", + "gate_offset", + "gate_bump" + ], + "sha256": "36d7f4e3a611fd6b5983222887beb9a37a18bfac0d999ac5c00e3e191d1c5cc4" + } + }, + "fairness": "The gates themselves are real checks evaluated on the real T488 data, and the premises the two gate-based proofs use hold on that data; this is a control-coverage defect, not a correctness defect of the owner journal or of its gates.", + "finding": "All five reject() controls of PR3633 owners/independent_literal.py (and of its 14 located copies) are tautological: three are unsatisfiable for every input (irreflexive a7 change)", + "staged.pr3133_widths", + true + ], + [ + "[base] declared changed registers (645, 660) instead of PR3605's (644, 660)", + "staged.registers", + true + ], + [ + "[base] parent word with an extra top-level key", + "staged.parent_diff.keys", + true + ], + [ + "[base] parent cleanup[0] replaced by two unmatched items (2-for-1 replace block)", + "staged.parent_diff.replace", + true + ], + [ + "[base] parent with its last warp record removed", + "staged.parent_diff.warps", + true + ], + [ + "[base] parent word with one A gate frame changed", + "staged.parent_diff.sections", + true + ], + [ + "[base] parent word already carrying the staged restore 210", + "staged.parent_diff.sections", + true + ], + [ + "[base] parent without deleted write B[184] (warps re-attached): 7 deletions", + "staged.parent_diff.sections", + true + ], + [ + "[base] parent deleted write B[184] with source 644 -> 645", + "staged.parent_diff.deleted", + true + ], + [ + "[base] parent B[599] replaced by the first inserted release: an ordinary item among the insertions", + "staged.parent_diff.inserted", + true + ], + [ + "[base] parent cleanup[1009] replaced by the first cleanup insert: one insert pair altered", + "staged.parent_diff.cleanup_pairs", + true + ], + [ + "[base] parent restore 210 coefficient changed (non-frame field)", + "staged.parent_diff.restore", + true + ], + [ + "[base] parent start_extra[644] = [31]", + "staged.parent_diff.start_extra", + true + ], + [ + "[base] parent warps[0] lender changed", + "staged.parent_diff.warp_record", + true + ], + [ + "[base] parent warps[0] B position shifted by one", + "staged.parent_diff.warp_alignment", + true + ], + [ + "[extra] one byte of the gzip stream flipped, pin kept", + "staged.pin", + true + ], + [ + "[extra] coefficient of the first donor release B[591] flipped", + "wc.replay", + true + ], + [ + "[extra] first cleanup item deleted, expected cleanup count lowered to match", + "wc.replay", + true + ], + [ + "[extra] commuting cleanup items 7/8 swapped (both replays and the tracker accept)", + "words.cleanup", + true + ], + [ + "[extra] helper_final frame of 226 raised to FULL, report forged", + "words.declared", + true + ], + [ + "[extra] cleanup prepended with helper 224 += zero buffer (forward-neutral), cleanup count raised", + "wc.replay", + true + ], + [ + "[extra] declared profile/inventory of the PR3133 parent", + "words.declared", + true + ], + [ + "[extra] expected helper count 409", + "words.counts", + true + ], + [ + "[extra] PROVIDERS.json[extra] width of helper 644 (index 389) back to 8", + "staged.providers", + true + ], + [ + "[extra] PROVIDERS.json[extra] actual_paid_profile rank-1 count +1, widths kept", + "staged.providers_paid", + true + ], + [ + "[extra] PR3133-PROVIDERS[extra] replaced by the staged widths (no 8->7 change)", + "staged.pr3133_widths", + true + ], + [ + "[extra] declared changed registers (645, 660) instead of PR3605's (644, 660)", + "staged.registers", + true + ], + [ + "[extra] parent word with an extra top-level key", + "staged.parent_diff.keys", + true + ], + [ + "[extra] parent cleanup[0] replaced by two unmatched items (2-for-1 replace block)", + "staged.parent_diff.replace", + true + ], + [ + "[extra] parent with its last warp record removed", + "staged.parent_diff.warps", + true + ], + [ + "[extra] parent word with one A gate frame changed", + "staged.parent_diff.sections", + true + ], + [ + "[extra] parent word already carrying the staged restore 210", + "staged.parent_diff.sections", + true + ], + [ + "[extra] parent without deleted write B[184] (warps re-attached): 7 deletions", + "staged.parent_diff.sections", + true + ], + [ + "[extra] parent deleted write B[184] with source 644 -> 645", + "staged.parent_diff.deleted", + true + ], + [ + "[extra] parent B[599] replaced by the first inserted release: an ordinary item among the insertions", + "staged.parent_diff.inserted", + true + ], + [ + "[extra] parent cleanup[1009] replaced by the first cleanup insert: one insert pair altered", + "staged.parent_diff.cleanup_pairs", + true + ], + [ + "[extra] parent restore 210 coefficient changed (non-frame field)", + "staged.parent_diff.restore", + true + ], + [ + "[extra] parent start_extra[644] = [31]", + "staged.parent_diff.start_extra", + true + ], + [ + "[extra] parent warps[0] lender changed", + "staged.parent_diff.warp_record", + true + ], + [ + "[extra] parent warps[0] B position shifted by one", + "staged.parent_diff.warp_alignment", + true + ] + ], + "extra": { + "buffers": { + "offset": 704, + "private_buffers": 91 + }, + "changed_registers": [ + 644, + 660 + ], + "cleanup": { + "all_at_full": true, + "items": 5222, + "literal_equal": true + }, + "counts": { + "a_prelude_events": 84, + "buffers": 91, + "corrections": 36, + "helpers": 408, + "live": 632, + "sections": { + "A": 4079, + "B": 1868, + "cleanup": 5222, + "compensation": 5736, + "cut": 176, + "precondition": 192, + "restore": 212 + }, + "warps": 72 + }, + "independent_inputs": 632, + "inventory": { + "4": 26, + "5": 2, + "6": 18, + "7": 52, + "8": 97, + "9": 213 + }, + "localisation": { + "B": { + "deleted": 8, + "inserted": 4, + "replaced": 0 + }, + "cleanup": { + "deleted": 0, + "inserted": 4, + "replaced": 0 + }, + "report": { + "keys": [ + "local_profile", + "width_inventory" + ] + }, + "restore": { + "deleted": 0, + "inserted": 0, + "replaced": 1 + }, + "start_extra": { + "keys": [ + "644", + "660" + ] + }, + "warps": { + "position_only": 72 + } + }, + "pr3133_width_changes": [ + [ + 389, + 8, + 7 + ], + [ + 397, + 8, + 7 + ] + ], + "profile": { + "1": 2439, + "2": 726, + "3": 224, + "4": 102, + "5": 20, + "8": 8 + }, + "sha256": "886338fc32da14f3389a82c8120a1bcf9c5e2f0ec3e9b2d377fdae809e1d977f", + "tracker_final": true, + "unchanged_sections": [ + "A", + "base", + "center_frames", + "centers", + "compensation", + "cut", + "helper_final", + "initial_corrections", + "kind", + "ports", + "precondition", + "prelude_events", + "removed", + "target_levels" + ] + }, + "parents": { + "base": { + "also_at": "research/coupled-carrier-cap740/words/base.json.gz", + "counts": { + "a_prelude_events": 84, + "buffers": 91, + "corrections": 36, + "helpers": 408, + "live": 632, + "sections": { + "A": 4079, + "B": 1872, + "cleanup": 5218, + "compensation": 5736, + "cut": 176, + "precondition": 192, + "restore": 212 + }, + "warps": 72 + }, + "inventory": { + "4": 26, + "5": 2, + "6": 18, + "7": 50, + "8": 99, + "9": 213 + }, + "path": "research/rank7-coupled-586-588-carrier-fold/edits/FINAL-WORD-base.json.gz", + "profile": { + "1": 2433, + "2": 724, + "3": 228, + "4": 102, + "5": 20, + "8": 8 + }, + "provider": "gauge44", + "sha256": "300a9b8ad1d554f0e6f84f43f3f5fd870a5b4e5f81ad856900b6e8999684892e", + "widths_equal_pr3133_providers": true + }, + "extra": { + "also_at": "research/coupled-carrier-cap740/words/extra.json.gz", + "counts": { + "a_prelude_events": 84, + "buffers": 91, + "corrections": 36, + "helpers": 408, + "live": 632, + "sections": { + "A": 4079, + "B": 1872, + "cleanup": 5218, + "compensation": 5736, + "cut": 176, + "precondition": 192, + "restore": 212 + }, + "warps": 72 + }, + "inventory": { + "4": 26, + "5": 2, + "6": 18, + "7": 50, + "8": 99, + "9": 213 + }, + "path": "research/rank7-coupled-586-588-carrier-fold/edits/FINAL-WORD-extra.json.gz", + "profile": { + "1": 2433, + "2": 724, + "3": 228, + "4": 102, + "5": 20, + "8": 8 + }, + "provider": "extra", + "sha256": "495d030f1e03b90e954dad3043994f03293d41b9ed220a3beaef849df435d3f0", + "widths_equal_pr3133_providers": true + } + }, + "pr3605_identity": { + "base": { + "identical_bytes": true, + "pr3605": { + "commit": "3efbb5de06a49ffeea6684482f2d3fe48cdca3fb", + "git_blob": "dc6d321b1e8e2a837a1ff7bb60e8f8f78d043db5", + "path": "research/staged-644-660-width-reduction/STAGED644-660-base.json.gz" + }, + "pr3633": { + "commit": "72c268f113368ea0387d188225e3f81ba6d6ef73", + "git_blob": "dc6d321b1e8e2a837a1ff7bb60e8f8f78d043db5", + "path": "research/staged-width-t488/supplier3605/STAGED644-660-base.json.gz" + }, + "sha256": "735c540b1fe29d0ceeb4d2f304ec941e5f2bd3bc92bafc8cce2b4f9a67a3b653" + }, + "extra": { + "identical_bytes": true, + "pr3605": { + "commit": "3efbb5de06a49ffeea6684482f2d3fe48cdca3fb", + "git_blob": "78e7b2bb0653c8dcc0133980a19dd9d97ebdb67e", + "path": "research/staged-644-660-width-reduction/STAGED644-660-extra.json.gz" + }, + "pr3633": { + "commit": "72c268f113368ea0387d188225e3f81ba6d6ef73", + "git_blob": "78e7b2bb0653c8dcc0133980a19dd9d97ebdb67e", + "path": "research/staged-width-t488/supplier3605/STAGED644-660-extra.json.gz" + }, + "sha256": "886338fc32da14f3389a82c8120a1bcf9c5e2f0ec3e9b2d377fdae809e1d977f" + } + } + } +} diff --git a/research/t488-independent-audit/SOURCES.json b/research/t488-independent-audit/SOURCES.json new file mode 100644 index 0000000000..f71ef08d1c --- /dev/null +++ b/research/t488-independent-audit/SOURCES.json @@ -0,0 +1,690 @@ +{ + "schema_version": 1, + "base_commit": "20ecf2891d80b5acf18d980b2164fb44ea7faa90", + "files": [ + { + "path": "certificates/selected-result.json", + "usage": "provenance", + "git_blob": "b1fea0c94436b236cc2b7d41a7be032811b94c81", + "sha256": "bb2eaa58647b353f88d0aec2eafb9d30df1be53589231b23e12ceef3e4bad129" + }, + { + "path": "docs/research/agent-task-catalog.json", + "usage": "provenance", + "git_blob": "67f085acd86acd7d4e03922d9cf4b26bca66fed5", + "sha256": "47b42433f5a85809a3aadd29c2c339da2c6ed728e4f7762df6b88bccb7874cf0" + }, + { + "path": "research/coupled-carrier-cap740/vendor/PR3133-PROVIDERS.json", + "usage": "provenance", + "git_blob": "0b2343675099d0a9fe90da6e1f1e53f50235567e", + "sha256": "52eb0d85caa7b2d4a93acdf5011f52b793f61ddca86235fb03cd9a09397ad4cd" + }, + { + "path": "research/coupled-carrier-cap740/words/base.json.gz", + "usage": "provenance", + "git_blob": "cb4cc24a4545944627ad595c0c5080196952b9f0", + "sha256": "300a9b8ad1d554f0e6f84f43f3f5fd870a5b4e5f81ad856900b6e8999684892e" + }, + { + "path": "research/coupled-carrier-cap740/words/extra.json.gz", + "usage": "provenance", + "git_blob": "decc8f26d8aab108baf75f0d8b2d1012eb06d076", + "sha256": "495d030f1e03b90e954dad3043994f03293d41b9ed220a3beaef849df435d3f0" + }, + { + "path": "research/rank7-coupled-586-588-carrier-fold/edits/FINAL-WORD-base.json.gz", + "usage": "provenance", + "git_blob": "cb4cc24a4545944627ad595c0c5080196952b9f0", + "sha256": "300a9b8ad1d554f0e6f84f43f3f5fd870a5b4e5f81ad856900b6e8999684892e" + }, + { + "path": "research/rank7-coupled-586-588-carrier-fold/edits/FINAL-WORD-extra.json.gz", + "usage": "provenance", + "git_blob": "decc8f26d8aab108baf75f0d8b2d1012eb06d076", + "sha256": "495d030f1e03b90e954dad3043994f03293d41b9ed220a3beaef849df435d3f0" + }, + { + "path": "research/rank7-joint-endframes/allocation/vendor/PLAN.json.gz", + "usage": "provenance", + "git_blob": "7a1db55a4d53ee99280be4f3ab7d29f7b2033adf", + "sha256": "d566cfb87b71639d4138e1f89b543e2dd6a7772070226a2a14c799a6835bf21e" + }, + { + "path": "research/rank7-joint-endframes/allocation/vendor/TEMPLATE.json.gz", + "usage": "provenance", + "git_blob": "114808c32b1799da53524bdc25cea67efb60009e", + "sha256": "21297b20f1f2dc76eb9dd8124416068c3cb096bab9df17fa4da19bc68dfe082e" + } + ], + "remote_sources": [ + { + "key": "outer_claim.json", + "repository": "CrocSwap/integer-mult-bounds", + "pr": 3653, + "url": "https://github.com/CrocSwap/integer-mult-bounds/pull/3653", + "commit": "607a09ff441c79b215a8127a81f4cc70021d0d9b", + "path": "research/retimed-t488-composition/evidence/recorded-admission/outer_claim.json", + "git_blob": "6f05d13347e650467b561f1861e6266ffc9ef15c", + "sha256": "c7455323c62a5b1d5c9ceef07a305a5758df6971ac8f88e2be5223eb5f4c7553", + "read_by": [ + "verify.INPUTS:outer3653" + ] + }, + { + "key": "independent_literal.py", + "repository": "CrocSwap/integer-mult-bounds", + "pr": 3653, + "url": "https://github.com/CrocSwap/integer-mult-bounds/pull/3653", + "commit": "607a09ff441c79b215a8127a81f4cc70021d0d9b", + "path": "research/retimed-t488-composition/full-replay/vendor/pr3642/vendor/staged-width-t530/owners/independent_literal.py", + "git_blob": "32755776c15660b939c92cb8835c71ec87c8c1d6", + "sha256": "36d7f4e3a611fd6b5983222887beb9a37a18bfac0d999ac5c00e3e191d1c5cc4", + "read_by": [ + "audit_inherited_controls.COPIES.PR3653-vendor" + ] + }, + { + "key": "RETIMED644-660-base.json.gz", + "repository": "CrocSwap/integer-mult-bounds", + "pr": 3653, + "url": "https://github.com/CrocSwap/integer-mult-bounds/pull/3653", + "commit": "607a09ff441c79b215a8127a81f4cc70021d0d9b", + "path": "research/retimed-t488-composition/local-audit/words/RETIMED644-660-base.json.gz", + "git_blob": "d69125e0c7d34af9299e411ada5c30dfc7b95ad1", + "sha256": "6df3356808e29909776be8bc0faf03df983401a267c2d740d1ac2dbcaa278269", + "read_by": [ + "verify.INPUTS:w3653_base" + ] + }, + { + "key": "RETIMED644-660-extra.json.gz", + "repository": "CrocSwap/integer-mult-bounds", + "pr": 3653, + "url": "https://github.com/CrocSwap/integer-mult-bounds/pull/3653", + "commit": "607a09ff441c79b215a8127a81f4cc70021d0d9b", + "path": "research/retimed-t488-composition/local-audit/words/RETIMED644-660-extra.json.gz", + "git_blob": "f3c580b06506e8888fe766c9a1e0bb1a748d7b74", + "sha256": "65624efec36b35934b750c1741ba2ad4e6df3cff86e44536b0c016ebddb0fb55", + "read_by": [ + "verify.INPUTS:w3653_extra" + ] + }, + { + "key": "independent_literal.py", + "repository": "CrocSwap/integer-mult-bounds", + "pr": 3653, + "url": "https://github.com/CrocSwap/integer-mult-bounds/pull/3653", + "commit": "607a09ff441c79b215a8127a81f4cc70021d0d9b", + "path": "research/retimed-t488-composition/owners/independent_literal.py", + "git_blob": "403a9cd104c7e68a5bd2431bb442d701f30828f5", + "sha256": "caa5eb4fff4908c0891c3ee41e7e6d71bf19bb710d1d7781164f10ce84bce769", + "read_by": [ + "audit_inherited_controls.COPIES.PR3653" + ] + }, + { + "key": "independent_literal.py", + "repository": "CrocSwap/integer-mult-bounds", + "pr": 3653, + "url": "https://github.com/CrocSwap/integer-mult-bounds/pull/3653", + "commit": "607a09ff441c79b215a8127a81f4cc70021d0d9b", + "path": "research/retimed-t488-composition/owners/upstream/independent_literal.py", + "git_blob": "e2a226950ed30e88f6c386669716b9a463dd873d", + "sha256": "8517c0d6c7cbe73438885135ac7fd9781dd5120113c9721002346dca603f5ed9", + "read_by": [ + "audit_inherited_controls.COPIES.PR3653-upstream" + ] + }, + { + "key": "CLAIM.json", + "repository": "CrocSwap/integer-mult-bounds", + "pr": 3605, + "url": "https://github.com/CrocSwap/integer-mult-bounds/pull/3605", + "commit": "3efbb5de06a49ffeea6684482f2d3fe48cdca3fb", + "path": "research/staged-644-660-conditional-release/CLAIM.json", + "git_blob": "445e6a83a4cdc6e8630e86f20990b50d1311f81d", + "sha256": "2e721209f626d5111142951b1f26383d6746b4f230a4625244b5e5c84d6079d5", + "read_by": [ + "verify.INPUTS:claim3605" + ] + }, + { + "key": "independent_literal.py", + "repository": "CrocSwap/integer-mult-bounds", + "pr": 3605, + "url": "https://github.com/CrocSwap/integer-mult-bounds/pull/3605", + "commit": "3efbb5de06a49ffeea6684482f2d3fe48cdca3fb", + "path": "research/staged-644-660-conditional-release/owners/independent_literal.py", + "git_blob": "5d1a7b4fd635070522e78656ccedf9bcf4bf13db", + "sha256": "da897bd5cef36943b3f5b1422fe81716b4eaa6230b056d31f8c2310cf62f8580", + "read_by": [ + "audit_inherited_controls.COPIES.PR3605" + ] + }, + { + "key": "CLAIM.json", + "repository": "CrocSwap/integer-mult-bounds", + "pr": 3630, + "url": "https://github.com/CrocSwap/integer-mult-bounds/pull/3630", + "commit": "bcac7fe04b9b7f7f5d3c27d620561f96c4ae5608", + "path": "research/staged-644-660-owner-successor/CLAIM.json", + "git_blob": "f82fff358656f8f6fe5d781157f12bc8b1edb62b", + "sha256": "aa4f56010fd4c1e620c742a091932ef3af05cca13886915c1c44f30e9f961aab", + "read_by": [ + "verify.INPUTS:claim3630" + ] + }, + { + "key": "README.md", + "repository": "CrocSwap/integer-mult-bounds", + "pr": 3630, + "url": "https://github.com/CrocSwap/integer-mult-bounds/pull/3630", + "commit": "bcac7fe04b9b7f7f5d3c27d620561f96c4ae5608", + "path": "research/staged-644-660-owner-successor/README.md", + "git_blob": "0f46d29b9d8674a1bb8851f2ea647e754ce40c12", + "sha256": "79cf49cd2274364888830d63a664f514c7b1a51d3ac54e57dc4117d432078e31", + "read_by": [ + "audit_ceiling.PINS.readme3630" + ] + }, + { + "key": "VALIDATION.json", + "repository": "CrocSwap/integer-mult-bounds", + "pr": 3630, + "url": "https://github.com/CrocSwap/integer-mult-bounds/pull/3630", + "commit": "bcac7fe04b9b7f7f5d3c27d620561f96c4ae5608", + "path": "research/staged-644-660-owner-successor/VALIDATION.json", + "git_blob": "5fb3ccf531d2e4c723f3a0abdf93813b8e3a77bb", + "sha256": "1d85df49a5e4b9d1262cfaffa9725e03bfe660f395bc6c08d594a0b0551fc24e", + "read_by": [ + "audit_ceiling.PINS.validation3630" + ] + }, + { + "key": "check_budget.py", + "repository": "CrocSwap/integer-mult-bounds", + "pr": 3630, + "url": "https://github.com/CrocSwap/integer-mult-bounds/pull/3630", + "commit": "bcac7fe04b9b7f7f5d3c27d620561f96c4ae5608", + "path": "research/staged-644-660-owner-successor/check_budget.py", + "git_blob": "a1de88c5f77ca173d581a5bcd0ed4f3144d74b56", + "sha256": "639820c9ccf72b88ae16768055ec5500a66004784f57a2fa5f218bf4d2e4ee05", + "read_by": [ + "audit_ceiling.PINS.budget3630" + ] + }, + { + "key": "independent_owners.py", + "repository": "CrocSwap/integer-mult-bounds", + "pr": 3630, + "url": "https://github.com/CrocSwap/integer-mult-bounds/pull/3630", + "commit": "bcac7fe04b9b7f7f5d3c27d620561f96c4ae5608", + "path": "research/staged-644-660-owner-successor/independent_owners.py", + "git_blob": "f116cc0142b4f6d674abf6e4c844142c00fb0297", + "sha256": "8d02dff267938cdb373aa617a411d8e67700e130609d2714e98ccf814e32eaef", + "read_by": [ + "audit_inherited_controls.COPIES.PR3630" + ] + }, + { + "key": "STAGED644-660-base.json.gz", + "repository": "CrocSwap/integer-mult-bounds", + "pr": 3605, + "url": "https://github.com/CrocSwap/integer-mult-bounds/pull/3605", + "commit": "3efbb5de06a49ffeea6684482f2d3fe48cdca3fb", + "path": "research/staged-644-660-width-reduction/STAGED644-660-base.json.gz", + "git_blob": "dc6d321b1e8e2a837a1ff7bb60e8f8f78d043db5", + "sha256": "735c540b1fe29d0ceeb4d2f304ec941e5f2bd3bc92bafc8cce2b4f9a67a3b653", + "read_by": [ + "audit_staged_words.PR3605_PATH" + ] + }, + { + "key": "STAGED644-660-extra.json.gz", + "repository": "CrocSwap/integer-mult-bounds", + "pr": 3605, + "url": "https://github.com/CrocSwap/integer-mult-bounds/pull/3605", + "commit": "3efbb5de06a49ffeea6684482f2d3fe48cdca3fb", + "path": "research/staged-644-660-width-reduction/STAGED644-660-extra.json.gz", + "git_blob": "78e7b2bb0653c8dcc0133980a19dd9d97ebdb67e", + "sha256": "886338fc32da14f3389a82c8120a1bcf9c5e2f0ec3e9b2d377fdae809e1d977f", + "read_by": [ + "audit_staged_words.PR3605_PATH" + ] + }, + { + "key": "independent_literal.py", + "repository": "CrocSwap/integer-mult-bounds", + "pr": 3642, + "url": "https://github.com/CrocSwap/integer-mult-bounds/pull/3642", + "commit": "044d24f12e74f228dd462c07ae2d7f3ea35e2be9", + "path": "research/staged-release-retiming-t530/full-admission/vendor/staged-width-t530/owners/independent_literal.py", + "git_blob": "32755776c15660b939c92cb8835c71ec87c8c1d6", + "sha256": "36d7f4e3a611fd6b5983222887beb9a37a18bfac0d999ac5c00e3e191d1c5cc4", + "read_by": [ + "audit_inherited_controls.COPIES.PR3642-vendor" + ] + }, + { + "key": "independent_literal.py", + "repository": "CrocSwap/integer-mult-bounds", + "pr": 3642, + "url": "https://github.com/CrocSwap/integer-mult-bounds/pull/3642", + "commit": "044d24f12e74f228dd462c07ae2d7f3ea35e2be9", + "path": "research/staged-release-retiming-t530/owner-audit/independent_literal.py", + "git_blob": "86dc50768ff3fda8e265c08aa9314fe072e0c396", + "sha256": "b51c36ad1ae0999547edc4726d698df369fef60daaaac4f5bbea471648c9f628", + "read_by": [ + "audit_inherited_controls.COPIES.PR3642" + ] + }, + { + "key": "independent_literal.py", + "repository": "CrocSwap/integer-mult-bounds", + "pr": 3642, + "url": "https://github.com/CrocSwap/integer-mult-bounds/pull/3642", + "commit": "044d24f12e74f228dd462c07ae2d7f3ea35e2be9", + "path": "research/staged-release-retiming-t530/owner-audit/upstream/independent_literal.py", + "git_blob": "32755776c15660b939c92cb8835c71ec87c8c1d6", + "sha256": "36d7f4e3a611fd6b5983222887beb9a37a18bfac0d999ac5c00e3e191d1c5cc4", + "read_by": [ + "audit_inherited_controls.COPIES.PR3642-upstream" + ] + }, + { + "key": "CLAIM.json", + "repository": "CrocSwap/integer-mult-bounds", + "pr": 3624, + "url": "https://github.com/CrocSwap/integer-mult-bounds/pull/3624", + "commit": "515a7c4490aa32ac3ca5014feb6f7da0d6952863", + "path": "research/staged-width-t450/CLAIM.json", + "git_blob": "0d59dc3be440d616dcc24c01d38a6b2439e4b94b", + "sha256": "8f2f80438593e7efaa6bb57c881a40cedbd7b5665b4ae12c6cf215d217c98397", + "read_by": [ + "verify.INPUTS:claim3624" + ] + }, + { + "key": "independent_literal.py", + "repository": "CrocSwap/integer-mult-bounds", + "pr": 3624, + "url": "https://github.com/CrocSwap/integer-mult-bounds/pull/3624", + "commit": "515a7c4490aa32ac3ca5014feb6f7da0d6952863", + "path": "research/staged-width-t450/owners/independent_literal.py", + "git_blob": "d9c2450e03d9dcde39ae96f0edc835d1a67d9fe5", + "sha256": "68684584d3866b2ab76c254dd6d5be3a49d276a6e5f6353183dee4d8d84de4d4", + "read_by": [ + "audit_inherited_controls.COPIES.PR3624" + ] + }, + { + "key": "independent_literal.py", + "repository": "CrocSwap/integer-mult-bounds", + "pr": 3624, + "url": "https://github.com/CrocSwap/integer-mult-bounds/pull/3624", + "commit": "515a7c4490aa32ac3ca5014feb6f7da0d6952863", + "path": "research/staged-width-t450/provenance/pr3605/owners/independent_literal.py", + "git_blob": "5d1a7b4fd635070522e78656ccedf9bcf4bf13db", + "sha256": "da897bd5cef36943b3f5b1422fe81716b4eaa6230b056d31f8c2310cf62f8580", + "read_by": [ + "audit_inherited_controls.COPIES.PR3624-provenance" + ] + }, + { + "key": "CLAIM.json", + "repository": "CrocSwap/integer-mult-bounds", + "pr": 3633, + "url": "https://github.com/CrocSwap/integer-mult-bounds/pull/3633", + "commit": "72c268f113368ea0387d188225e3f81ba6d6ef73", + "path": "research/staged-width-t488/CLAIM.json", + "git_blob": "3e56f9a3035ebe3da02fdf8bc6d71cde4e6c90a0", + "sha256": "3332fb6f6d5ae81da8ead667973491ebefa1803d6d96a06790386ac16243f332", + "read_by": [ + "audit_ceiling.PINS.claim", + "verify.INPUTS:claim3633" + ] + }, + { + "key": "CEILING.json", + "repository": "CrocSwap/integer-mult-bounds", + "pr": 3633, + "url": "https://github.com/CrocSwap/integer-mult-bounds/pull/3633", + "commit": "72c268f113368ea0387d188225e3f81ba6d6ef73", + "path": "research/staged-width-t488/ceiling/CEILING.json", + "git_blob": "8bbec230ddf8498ac55c9ce32adabe813bf07eda", + "sha256": "a26bf26118222a5bcb6bf9f698d6b8b86003ff7b1479fd68b31f67c74945e460", + "read_by": [ + "audit_ceiling.PINS.ceiling3633" + ] + }, + { + "key": "CANDIDATE.json", + "repository": "CrocSwap/integer-mult-bounds", + "pr": 3633, + "url": "https://github.com/CrocSwap/integer-mult-bounds/pull/3633", + "commit": "72c268f113368ea0387d188225e3f81ba6d6ef73", + "path": "research/staged-width-t488/owners/CANDIDATE.json", + "git_blob": "4293ad4cb76533072610c858d3f44576afe8d777", + "sha256": "fbdda845beac5bba9baa4b1b8c8f596e98f70b59e1d60ebf4f4d3e8758631bfd", + "read_by": [ + "audit_ceiling.PINS.candidate", + "verify.INPUTS:t488_candidate" + ] + }, + { + "key": "DISPATCH37.json", + "repository": "CrocSwap/integer-mult-bounds", + "pr": 3633, + "url": "https://github.com/CrocSwap/integer-mult-bounds/pull/3633", + "commit": "72c268f113368ea0387d188225e3f81ba6d6ef73", + "path": "research/staged-width-t488/owners/DISPATCH37.json", + "git_blob": "c4c016ea3f3cb7bd04c1812259246a9411ad7adc", + "sha256": "4f78935b2ef1f6c08c9c16433d54015def1eccd767443b496a77ea8823690df6", + "read_by": [ + "verify.INPUTS:t488_dispatch37" + ] + }, + { + "key": "EXPECTED.json", + "repository": "CrocSwap/integer-mult-bounds", + "pr": 3633, + "url": "https://github.com/CrocSwap/integer-mult-bounds/pull/3633", + "commit": "72c268f113368ea0387d188225e3f81ba6d6ef73", + "path": "research/staged-width-t488/owners/EXPECTED.json", + "git_blob": "415eac1a2b0201beac2c3e531c6fcbd1133a819f", + "sha256": "924493b6bbeade055af4916ac7bd6d7d7ec0600719f5c39636dc3f16d7ec14ea", + "read_by": [ + "audit_ceiling.PINS.expected", + "audit_inherited_controls.EXPECTED_JSON", + "verify.INPUTS:t488_expected" + ] + }, + { + "key": "ORDER37.json", + "repository": "CrocSwap/integer-mult-bounds", + "pr": 3633, + "url": "https://github.com/CrocSwap/integer-mult-bounds/pull/3633", + "commit": "72c268f113368ea0387d188225e3f81ba6d6ef73", + "path": "research/staged-width-t488/owners/ORDER37.json", + "git_blob": "411004ab280d6468bdd9c98c01f153baed66c543", + "sha256": "94803d190b2b0506d9645e980b57cdbc3e98221f3b615292df8a94f1b174c984", + "read_by": [ + "verify.INPUTS:t488_order37" + ] + }, + { + "key": "PROVIDERS.json", + "repository": "CrocSwap/integer-mult-bounds", + "pr": 3633, + "url": "https://github.com/CrocSwap/integer-mult-bounds/pull/3633", + "commit": "72c268f113368ea0387d188225e3f81ba6d6ef73", + "path": "research/staged-width-t488/owners/PROVIDERS.json", + "git_blob": "f321d4185339244b8b004326af70e653e611d84a", + "sha256": "d96f2798024ab83edbb8848b62c425992aec182f1f24ef46bb8286dd390618c2", + "read_by": [ + "audit_staged_words.PROVIDERS" + ] + }, + { + "key": "SOURCE-PINS.json", + "repository": "CrocSwap/integer-mult-bounds", + "pr": 3633, + "url": "https://github.com/CrocSwap/integer-mult-bounds/pull/3633", + "commit": "72c268f113368ea0387d188225e3f81ba6d6ef73", + "path": "research/staged-width-t488/owners/SOURCE-PINS.json", + "git_blob": "5aa0ee4cb46308e674c5e1cedaa7eefc7bac59bd", + "sha256": "7dec42af587af3b7ae66d4fdf3d0c6157f9c85eb744edec66fa266f294378e5e", + "read_by": [ + "audit_staged_words.SOURCE_PINS" + ] + }, + { + "key": "T16-DISPATCH.json", + "repository": "CrocSwap/integer-mult-bounds", + "pr": 3633, + "url": "https://github.com/CrocSwap/integer-mult-bounds/pull/3633", + "commit": "72c268f113368ea0387d188225e3f81ba6d6ef73", + "path": "research/staged-width-t488/owners/T16-DISPATCH.json", + "git_blob": "40725d2abb328ba7fee1c4f74656eecd982f2f7e", + "sha256": "282960b7163bc305675b64d1d0faee9f71b18848748326418a013626944c6488", + "read_by": [ + "verify.INPUTS:t488_dispatch_t16" + ] + }, + { + "key": "T16-ORDER.json", + "repository": "CrocSwap/integer-mult-bounds", + "pr": 3633, + "url": "https://github.com/CrocSwap/integer-mult-bounds/pull/3633", + "commit": "72c268f113368ea0387d188225e3f81ba6d6ef73", + "path": "research/staged-width-t488/owners/T16-ORDER.json", + "git_blob": "470b8afb8af03fd053b6ff19bc90eb410b5c9e6b", + "sha256": "9b2106c84aed23a7c3c8b99e21d3e96c66b12f92980a42ab2f6dbbe7831ef7c4", + "read_by": [ + "verify.INPUTS:t488_order_t16" + ] + }, + { + "key": "T8-DISPATCH.json", + "repository": "CrocSwap/integer-mult-bounds", + "pr": 3633, + "url": "https://github.com/CrocSwap/integer-mult-bounds/pull/3633", + "commit": "72c268f113368ea0387d188225e3f81ba6d6ef73", + "path": "research/staged-width-t488/owners/T8-DISPATCH.json", + "git_blob": "f95b6f07bf4d9e607a7ab8fcc101cdc63a725b07", + "sha256": "23877abbe8ec258eb1f2425137f9171d37b1a4414a25595abe75eb0738850a24", + "read_by": [ + "verify.INPUTS:t488_dispatch_t8" + ] + }, + { + "key": "T8-ORDER.json", + "repository": "CrocSwap/integer-mult-bounds", + "pr": 3633, + "url": "https://github.com/CrocSwap/integer-mult-bounds/pull/3633", + "commit": "72c268f113368ea0387d188225e3f81ba6d6ef73", + "path": "research/staged-width-t488/owners/T8-ORDER.json", + "git_blob": "b013bdb989f7fde1b8a22ae8ed743eae3a0a62fd", + "sha256": "a1abf78fb3b88bbf4656e6ae26f148a007f744f0be4c00d480fb2424ce8766d6", + "read_by": [ + "verify.INPUTS:t488_order_t8" + ] + }, + { + "key": "independent_literal.py", + "repository": "CrocSwap/integer-mult-bounds", + "pr": 3633, + "url": "https://github.com/CrocSwap/integer-mult-bounds/pull/3633", + "commit": "72c268f113368ea0387d188225e3f81ba6d6ef73", + "path": "research/staged-width-t488/owners/independent_literal.py", + "git_blob": "e2a226950ed30e88f6c386669716b9a463dd873d", + "sha256": "8517c0d6c7cbe73438885135ac7fd9781dd5120113c9721002346dca603f5ed9", + "read_by": [ + "audit_inherited_controls.COPIES.PR3633", + "audit_inherited_controls.TARGET" + ] + }, + { + "key": "independent_literal.py", + "repository": "CrocSwap/integer-mult-bounds", + "pr": 3633, + "url": "https://github.com/CrocSwap/integer-mult-bounds/pull/3633", + "commit": "72c268f113368ea0387d188225e3f81ba6d6ef73", + "path": "research/staged-width-t488/provenance/pr3605/owners/independent_literal.py", + "git_blob": "5d1a7b4fd635070522e78656ccedf9bcf4bf13db", + "sha256": "da897bd5cef36943b3f5b1422fe81716b4eaa6230b056d31f8c2310cf62f8580", + "read_by": [ + "audit_inherited_controls.COPIES.PR3633-provenance" + ] + }, + { + "key": "STAGED644-660-base.json.gz", + "repository": "CrocSwap/integer-mult-bounds", + "pr": 3633, + "url": "https://github.com/CrocSwap/integer-mult-bounds/pull/3633", + "commit": "72c268f113368ea0387d188225e3f81ba6d6ef73", + "path": "research/staged-width-t488/supplier3605/STAGED644-660-base.json.gz", + "git_blob": "dc6d321b1e8e2a837a1ff7bb60e8f8f78d043db5", + "sha256": "735c540b1fe29d0ceeb4d2f304ec941e5f2bd3bc92bafc8cce2b4f9a67a3b653", + "read_by": [ + "audit_staged_words.STAGED.base", + "verify.INPUTS:word_base" + ] + }, + { + "key": "STAGED644-660-extra.json.gz", + "repository": "CrocSwap/integer-mult-bounds", + "pr": 3633, + "url": "https://github.com/CrocSwap/integer-mult-bounds/pull/3633", + "commit": "72c268f113368ea0387d188225e3f81ba6d6ef73", + "path": "research/staged-width-t488/supplier3605/STAGED644-660-extra.json.gz", + "git_blob": "78e7b2bb0653c8dcc0133980a19dd9d97ebdb67e", + "sha256": "886338fc32da14f3389a82c8120a1bcf9c5e2f0ec3e9b2d377fdae809e1d977f", + "read_by": [ + "audit_staged_words.STAGED.extra" + ] + }, + { + "key": "CLAIM.json", + "repository": "CrocSwap/integer-mult-bounds", + "pr": 3626, + "url": "https://github.com/CrocSwap/integer-mult-bounds/pull/3626", + "commit": "659fd5e7064d9466b668d7188cbb013b333a415a", + "path": "research/staged-width-t530/CLAIM.json", + "git_blob": "c11ec575770d8aabff9731ce9cc671fc9db448c9", + "sha256": "9f27717c3ddbf5eb7b9773e1b2887ff3899d354019139d916c9b8004aee2e097", + "read_by": [ + "verify.INPUTS:claim3626" + ] + }, + { + "key": "independent_literal.py", + "repository": "CrocSwap/integer-mult-bounds", + "pr": 3626, + "url": "https://github.com/CrocSwap/integer-mult-bounds/pull/3626", + "commit": "659fd5e7064d9466b668d7188cbb013b333a415a", + "path": "research/staged-width-t530/owners/independent_literal.py", + "git_blob": "32755776c15660b939c92cb8835c71ec87c8c1d6", + "sha256": "36d7f4e3a611fd6b5983222887beb9a37a18bfac0d999ac5c00e3e191d1c5cc4", + "read_by": [ + "audit_inherited_controls.COPIES.PR3626" + ] + }, + { + "key": "independent_literal.py", + "repository": "CrocSwap/integer-mult-bounds", + "pr": 3626, + "url": "https://github.com/CrocSwap/integer-mult-bounds/pull/3626", + "commit": "659fd5e7064d9466b668d7188cbb013b333a415a", + "path": "research/staged-width-t530/provenance/pr3605/owners/independent_literal.py", + "git_blob": "5d1a7b4fd635070522e78656ccedf9bcf4bf13db", + "sha256": "da897bd5cef36943b3f5b1422fe81716b4eaa6230b056d31f8c2310cf62f8580", + "read_by": [ + "audit_inherited_controls.COPIES.PR3626-provenance" + ] + }, + { + "key": "independent_literal.py", + "repository": "CrocSwap/integer-mult-bounds", + "pr": 3638, + "url": "https://github.com/CrocSwap/integer-mult-bounds/pull/3638", + "commit": "d62e1acd7518a231440383f0d0dddf54d6edf8c1", + "path": "research/three-kappa-leads/independent_literal.py", + "git_blob": "4028493716bfcf3e73831067599518c055fcd2b0", + "sha256": "89c27f61ae3e6d2bdd4c469ac5b99d4f955a443c5e04c74e568c12093510efbe", + "read_by": [ + "audit_inherited_controls.COPIES.PR3638" + ] + } + ], + "vendored": [ + { + "path": "research/t488-independent-audit/vendor/pr3053/audit_common.py", + "module": "audit_common", + "git_blob": "17c4064f01b110012b54c05e64bdee8ae498e0b2", + "sha256": "7ced740252ac8d8a8c66ef3cc825008e0aacf080494d0e983fb322d7dcddbf42", + "copied_from": "20ecf2891d80b5acf18d980b2164fb44ea7faa90:research/t74-3015-restore-edit-audit/vendor/pr2804/audit_common.py", + "usage": "vendored library of this contributor (byte copy of a merged file on main)" + }, + { + "path": "research/t488-independent-audit/vendor/pr3053/words_check.py", + "module": "words_check", + "git_blob": "0b646b9d5583c75a78e0c2dd7e9cfffa02055058", + "sha256": "b66414a39f5ac7b9cd32284027e13a550734901aca5f4457085a034689bc3ce6", + "copied_from": "20ecf2891d80b5acf18d980b2164fb44ea7faa90:research/t74-3015-restore-edit-audit/vendor/pr2315/words_check.py", + "usage": "vendored library of this contributor (byte copy of a merged file on main)" + }, + { + "path": "research/t488-independent-audit/vendor/pr3053/fill_check80.py", + "module": "fill_check80", + "git_blob": "557e0732bfe4c030436a11f5433143c10fa07821", + "sha256": "1c4d30591969c6502605f239f40505261e99d025aa3b576a804332a6bc479b06", + "copied_from": "20ecf2891d80b5acf18d980b2164fb44ea7faa90:research/t74-3015-restore-edit-audit/vendor/pr2387/fill_check80.py", + "usage": "vendored library of this contributor (byte copy of a merged file on main)" + }, + { + "path": "research/t488-independent-audit/vendor/pr3053/check_words.py", + "module": "check_words", + "git_blob": "451dbb82f147ef446a1819b1ed595b4bf5547583", + "sha256": "5996e804fa006906abff63a982c272588a44c35dfc932fdf687ef98dd3e0905f", + "copied_from": "20ecf2891d80b5acf18d980b2164fb44ea7faa90:research/t74-3015-restore-edit-audit/vendor/pr2804/check_words.py", + "usage": "vendored library of this contributor (byte copy of a merged file on main)" + }, + { + "path": "research/t488-independent-audit/vendor/pr3053/audit_lib.py", + "module": "audit_lib", + "git_blob": "e8e847cc89dae6bb4e90e2c3dbebc30f615b698f", + "sha256": "e0cf84dd959d39e3f299c1e171753c29465fd5b4457561f4ab9ec0675d8e9dc1", + "copied_from": "20ecf2891d80b5acf18d980b2164fb44ea7faa90:research/t74-3015-restore-edit-audit/vendor/pr2865/audit_lib.py", + "usage": "vendored library of this contributor (byte copy of a merged file on main)" + }, + { + "path": "research/t488-independent-audit/vendor/pr3053/audit_kappa.py", + "module": "audit_kappa", + "git_blob": "6e935547c98188a5fd2ba952641bf474d9323d9b", + "sha256": "8dd12fac27ba3ee7decf99a7a1e410e138ec0fc34805bb28b93fa5cc843ab0ee", + "copied_from": "20ecf2891d80b5acf18d980b2164fb44ea7faa90:research/t74-3015-restore-edit-audit/audit_kappa.py", + "usage": "vendored library of this contributor (byte copy of a merged file on main)" + }, + { + "path": "research/t488-independent-audit/vendor/pr3053/audit_words.py", + "module": "audit_words", + "git_blob": "6368333e16c4f50e6eb8b5f2f4c178a2de8cbbb9", + "sha256": "ff831ad648dbecfd1c80f63245a84b762fdc70eb9fc3aa94dba23cd364857d18", + "copied_from": "20ecf2891d80b5acf18d980b2164fb44ea7faa90:research/t74-3015-restore-edit-audit/audit_words.py", + "usage": "vendored library of this contributor (byte copy of a merged file on main)" + }, + { + "path": "research/t488-independent-audit/vendor/pr3053/check_accounting.py", + "module": "check_accounting", + "git_blob": "e749057dd838b715d0630de618f32d66a38da931", + "sha256": "fb6aef50da586f15cd60762edc074c20e300d65f1378a80160b3d4b840762548", + "copied_from": "20ecf2891d80b5acf18d980b2164fb44ea7faa90:research/t74-3015-restore-edit-audit/vendor/pr2804/check_accounting.py", + "usage": "vendored library of this contributor (byte copy of a merged file on main)" + } + ], + "commits": { + "3605": "3efbb5de06a49ffeea6684482f2d3fe48cdca3fb", + "3624": "515a7c4490aa32ac3ca5014feb6f7da0d6952863", + "3626": "659fd5e7064d9466b668d7188cbb013b333a415a", + "3630": "bcac7fe04b9b7f7f5d3c27d620561f96c4ae5608", + "3633": "72c268f113368ea0387d188225e3f81ba6d6ef73", + "3638": "d62e1acd7518a231440383f0d0dddf54d6edf8c1", + "3642": "044d24f12e74f228dd462c07ae2d7f3ea35e2be9", + "3653": "607a09ff441c79b215a8127a81f4cc70021d0d9b" + }, + "package_data": [ + { + "path": "research/t488-independent-audit/data/FILL-columns.xz", + "sha256": "f10f95ccf2613988641533a664d15f300a6ccfd67cb3b7d636aa653c34a7e336", + "note": "PR3633 literal owner journal (output of PR3633 owners/portable_expand.py at head 72c268f1), re-encoded by tools/encode_journal.py; verify.py binds the decoded rows to PR3633 owners/EXPECTED.json payload_sha256 20662cc64df34cab409fdf31ed0f0222462530922a01308a46964d661d2374cf by byte-exact re-serialisation" + } + ], + "notes": [ + "All inputs are read with git show at the pinned commits and checked against git blob id and sha256; none is read from the working tree.", + "PR #3633, #3630, #3605, #3624, #3626, #3653, #3638 and #3642 are open; verify.py --fetch fetches their heads into refs/audit/prN (PR3638 and PR3642 only for the scope-C2 copy scan).", + "audit_inherited_controls.py also scans every .py blob of main and the eight PR heads to locate copies of PR3605's auditor (the located set must equal its pinned table) and to search for the sha256 its docstring cites; that scan is discovery beyond the pinned inputs listed here.", + "evidence/NATIVE-REPRODUCTION.json and evidence/GATE-DISABLE-DEMO.json are local-only records (not replayed); see README." + ] +} \ No newline at end of file diff --git a/research/t488-independent-audit/TASK.json b/research/t488-independent-audit/TASK.json new file mode 100644 index 0000000000..3530d45077 --- /dev/null +++ b/research/t488-independent-audit/TASK.json @@ -0,0 +1,100 @@ +{ + "schema_version": 1, + "task_id": "independent-moment", + "priority_revision": 6, + "base_commit": "20ecf2891d80b5acf18d980b2164fb44ea7faa90", + "scope_key": "pr3633-t488-independent-audit", + "original_card": { + "id": "independent-moment", + "question": "Does one changed supplier or assembly claim lack a source-bound independent check of a specific new implication?", + "starting_points": [ + "certificates/selected-result.json", + "research/rank7-joint-endframes/README.md", + "docs/research/research-family-status.md" + ], + "methods": [ + "Independently reconstruct a coordinated frame/circuit change and its single-change control, then derive the complete rank and physical ledger.", + "Implement certified moment inequalities through a different exact enclosure method." + ], + "acceptance_gate": "Demonstrate independence, source binding, positive cases and semantic failure controls. Name the existing checks and the missing coverage. More digits or another equivalent evaluation of an unchanged root is not by itself new verification coverage.", + "first_step": { + "command": [ + "python3", + "-B", + "scripts/research_baseline.py" + ], + "expected": "PASS_RESEARCH_BASELINE_BINDINGS_ONLY", + "inspect": "Read #2979 joint rank-7 end-frame proposal and the revision-6 evidence status. Its claimed kappa is not selected. Compare coordinated changes with the losing single-change controls and locate the complete physical ledger; an aggregate moment alone does not validate the mechanism.", + "scope": "Positive control / source-bound orientation only; this is not a new construction or full cold replay." + }, + "first_experiment": "Prioritize a bounded independent audit of #2979: reconstruct one joint group and its single-lowering control, check the declared frames/chronology, then trace changed children, idle widths and cleanup buffers into the complete invoice and both supplier caps. Choose one still-missing implication after checking existing audits. Stop with exact unresolved dependencies if full admission exceeds the session; do not launch a new lowering census on an unvalidated premise or substitute extra root digits for mechanism checking.", + "bottleneck_link": "Independent pricing prevents unsupported frontier claims and can reveal genuine accounting slack.", + "allowed_variations": [ + "Reconstruct either a source ledger or a numeric enclosure with a different method." + ], + "scope_change_triggers": [ + "Importing the primary function and calling its output independent verification.", + "Promoting #2979 from merged source archive to validated supplier solely from author PASS receipts or aggregate pricing." + ], + "overlap_terms": [ + "independent moment", + "rank histogram", + "exact enclosure", + "2979", + "rank7 joint endframes", + "paired extraction", + "joint lowering" + ] + }, + "scope_status": "refined", + "actual_scope": "Independent verification of open PR #3633 (hexapode, head 72c268f113368ea0387d188225e3f81ba6d6ef73): the T488 allocation of Ross Richardson's staged 644/660 supplier (PR #3605), its complete invoice, its claimed attainment of the fixed-inventory packing ceiling (independently bounded by PR #3630) and its endpoint, with standard-library code that imports or executes nothing of PR3633, PR3630 or PR3605: the two supplier words through this contributor's word lineage; all 1,791,936 literal owner roles (windows derived two ways from the frozen template, families, capacities, chronological prefixes, the active-local-pair exclusion, the complete invoice, aggregate realisation, stock); an own fixed-inventory LP (a first draft mirrored PR3633's enumeration/column code and was rewritten after review) with a rigorous dual certificate, attainment and cross-checks of both published certificates; the endpoint in two rational enclosures with exact gains; PR #3653's endpoint on the same journal; and a static proof that the five reject() controls kept in PR3633's adapted owner auditor (and 14 other copies) cannot fail.", + "scope_change_reason": "The card names #2979's coordinated lowering as its first example; the line has since moved to the staged 644/660 supplier, whose newest conditional claim (PR #3633, 20 minutes old at selection) had no independent check, and Clear1nsight's handoff (discussion 1166, comment 19016699) listed an arithmetic/release audit of this line as open. The audit follows the card's methods (independent reconstruction of the complete ledger, a different exact enclosure method) on that claim.", + "declared_milestone": "audit", + "budget_minutes": 360, + "stop_rule": "Stop after the independent checks and their controls pass or a defect is found; no new allocation search, no native operator proof, no outer-transfer re-derivation.", + "outcome": "completed", + "started_utc": "2026-10-18T12:00:00Z", + "author": "Sagar Thakrar (thakrarsagar)", + "ai_assistance": "Substantial AI assistance (Claude Code, Anthropic): two scopes built by subagents, adversarial review rounds by subagents; no independent human review is claimed.", + "existing_checks_consulted": [ + "PR3633's own checks: owners/independent_literal.py (adapted from PR3605's auditor, adding artifact/source/native-pin, FILL-payload and exact ORDER/DISPATCH gates; its five controls unchanged), ceiling/verify_ceiling.py, seven admission stages run by the author with reused native caches; fresh end-to-end portable CLI not run (author disclosure).", + "PR3630 (RossRichardson): independent 2,551-pattern dual and all-T fixed-inventory ceiling; states that PR3624's certificate/invoice match its premises but did not rerun its admission.", + "PR3638 (zxv, concurrent, 11:58 UTC): a different T360 allocation tying the same endpoint, with its own dual replay; not a check of PR3633's journal.", + "PR3653 (rohanarun; ready for review since 13:46 UTC): reuses PR3633's literal T488 allocation with roger-sato's PR3642 retimed words.", + "Clear1nsight's three-route handoff in discussion 1166 (comment 19016699): arithmetic/release audit of this line open; PR3624 read only.", + "PR3642 (roger-sato): the retimed T530 words that PR3653 composes with PR3633's journal; its owner audit carries the same five controls.", + "PR3680 (jwr404), PR3681 (roger-sato, full replay of PR3605's wrapper), PR3682 (zxv, retimed releases at T360): later work on the line, none an independent check of PR3633's journal or ceiling.", + "PR3687 (this account's parallel session, 14:46 UTC): an independent exact audit of PR3605's staged words with the PR3133 control, in a different code lineage; agrees with scope C1 on the width inventory and local bill.", + "PR3677 (jwr404): the retimed local bill keeps rank mass 5,135 and lowers the moment (used by the PR3653 downstream check).", + "PR3693 (roger-sato, 15:33 UTC): audits the retimed (PR3642/PR3653) and delayed (PR3682) words, reusing PR3687's interpreter; covers the retimed words' exactness that this package does not check." + ], + "open_questions": [ + { + "question": "Does a width change, or a local-bill change beyond PR3642's retiming, raise the ceiling above 0.001730418899766071163804 (the retimed ceiling certified here for PR3633's allocation)?", + "why_it_matters": "Within scope A's fixed model (these widths and local bill, the template's bin counts/capacities, nine-call normalisation, live 896T) packing is closed for every T: PR3633 attains the ceiling (checked here) and PR3638 reports the same endpoint. PR3653 shows a local-bill change with unchanged widths moves the endpoint, with PR3633's allocation at the re-certified ceiling.", + "first_step": "Reuse audit_ceiling.py's LP to price single-helper width changes (an inventory change) and local-bill changes before building any word.", + "blocked_by": "Any width change needs a new exact supplier word (as PR3605 built for helpers 644/660)." + }, + { + "question": "Can the five owner-audit controls (15 copies across PR3605, PR3624, PR3626, PR3630, PR3633, PR3638, PR3642, PR3653) be made to test their gates?", + "why_it_matters": "Finding 1: the five reject() controls are tautologies, so the receipts carry no evidence that the gates can fail.", + "first_step": "Factor the gates into a check() function and rerun it on each mutated journal requiring the expected message (as the main-lineage checker does), or reuse audit_owners.controls.", + "blocked_by": "Nothing." + }, + { + "question": "Can the native file pins be made platform-independent?", + "why_it_matters": "Finding 2: rebuilt native files differ from the pinned files only in the gzip OS byte, so the one file-hash gate (independent_literal.py native_canonical) rejects a correct rebuild on another platform.", + "first_step": "Pin decompressed payload sha256 of CANONICAL-WORD (and FULL-WORD, LOCAL-PROGRAMS) in independent_literal.py's native_canonical and the receipts; PR3605's staged_pair.py already restores the OS byte for supplier words and PR3633 pins FILL by payload.", + "blocked_by": "Nothing." + }, + { + "question": "Does PR3633's complete fresh end-to-end replay (replay.py) pass on a machine with enough memory?", + "why_it_matters": "The author disclosed it as not run; here it passed fresh native, owners, parent-lift, rebind and discovery bind, then the discovery emit stage was killed by the OOM killer (> 5.1 GB) on a 7.9 GB machine.", + "first_step": "Rerun replay.py with >= 12 GB RAM or swap; roger-sato's PR3681 reports a full PR3605-wrapper replay that may show a working configuration.", + "blocked_by": "Memory on this machine." + } + ], + "family_expansion_decision": "None proposed. The audit admits nothing. It reports no correctness defect in PR3633's finite claims in the checked scope; shows that within the fixed packing model (the staged widths and local bill, bin counts/capacities, nine-call normalisation, live 896T) packing is closed: no allocation for any T exceeds the endpoint, so gains need a change to the supplier or to one of these premises; and reports two process defects (tautological inherited controls; platform-dependent native pins).", + "bottleneck_link": "PR #3633 is the newest conditional candidate of the complex supplier line (kappa 0.0017300963 against the selected 0.0016726602). Independent verification of its allocation and ceiling attainment, and the statement that packing on this inventory is exhausted, inform the director's next frontier and expansion decisions.", + "evidence": "verify.py --fetch (RESULT.json, about 3 min; reproduced under CPython 3.14.4 and 3.12.15; 212/212 negative controls rejected by their named rules): B all 1,791,936 roles of PR3633's journal (bound to its payload sha256) pass windows derived two ways, the active-pair exclusion (also a lemma over 18,544 triples), same-call families, capacities, chronological prefixes and the complete invoice over 983,808 bins = the claimed profile; the journal realises the aggregate's 11 capacity-patterns; stock 437,757 (24 controls); A 2,551 patterns, own simplex, rigorous dual (12 zero / 2,539 positive reduced costs on [a_lo, a_hi]), unique LP-minimum profile attained by PR3633, PR3633 CEILING.json and PR3630 dual/digest cross-checked (52 controls); C1 both staged words exact forward and dual (632 inputs), widths = PR3633 PROVIDERS, differing from PR3133 only at helpers 389/397, change localised against the parent words (50 controls); D C = 0.001730096363528623534230 in two enclosures, kappa, gains over PR3626/PR3624/PR3630/PR3605 with improvement tests; PR3653's endpoint 0.001730418899766071163804 reproduced on the same journal with the dual re-certified at it (13 controls); C2 the five inherited reject() controls proved unable to fail in 15 files (65 controls); vendor pins (8 controls). Mutation sweep: 45/45 checker mutations killed after a passing baseline (local, CPython 3.14.4). Local-only: evidence/GATE-DISABLE-DEMO.json, evidence/NATIVE-REPRODUCTION.json." +} \ No newline at end of file diff --git a/research/t488-independent-audit/VERIFY.json b/research/t488-independent-audit/VERIFY.json new file mode 100644 index 0000000000..bbd8362ea5 --- /dev/null +++ b/research/t488-independent-audit/VERIFY.json @@ -0,0 +1,20 @@ +{ + "schema_version": 1, + "status": "ready", + "python": "3.12", + "requirements": [], + "inputs": [], + "commands": [ + { + "argv": [ + "{python}", + "-I", + "-B", + "{package}/verify.py", + "--fetch" + ], + "timeout_seconds": 600, + "checks": "Independent verification of open PR #3633 head 72c268f1 (hexapode, T488 staged-width allocation) in standard-library code that reads PR files only as pinned git data (git blob id and sha256) and never imports or runs PR code; --fetch (git >= 2.29, network) writes only fetched objects and refs/audit/prN for PR3633, PR3630, PR3605, PR3624, PR3626, PR3653, PR3638 and PR3642. It checks: both staged supplier words through this contributor's word lineage (forward and dual replay through cleanup and the target endpoint, tracker, cleanup re-derivation, private buffers, frame recount; widths equal PR3633 PROVIDERS.json and differ from main's PR3133 providers only at helpers 389 and 397; the change localised against the parent words on main); the complete literal journal of 1,791,936 roles (shipped compactly and bound to PR3633's pinned payload sha256 by byte-exact re-serialisation): ORDER/DISPATCH rebuilt and bound to PR3633's literal-file pins, source windows derived two ways from the frozen template, strict/closed windows, the active-local-pair exclusion per role and as a template lemma, same-call physical-family exclusion, capacities, chronological compact prefixes, the complete invoice over all 983,808 bins equal to the claimed profile, the aggregate's patterns realised exactly and the physical stock 437,757; an own fixed-inventory LP (2,551 patterns, exact simplex, rigorous dual certificate, unique LP-minimum profile, attainment by PR3633's aggregate, the all-T ceiling within the fixed model, cross-checks of PR3633 CEILING.json and PR3630's dual and digest); the endpoint C = 0.001730096363528623534230 in two rational enclosures, kappa, exact gains over PR3626, PR3624, PR3630 and PR3605 with improvement tests; PR3653's endpoint on the same journal with the dual re-certified at it; and the five reject() controls kept in PR3633's owner auditor (three unsatisfiable predicates, two +1 perturbations of equalities the preceding gate asserted), proved unable to fail in all 15 located copies. It requires every negative control (212) to be rejected by its named rule and the result to equal RESULT.json (timings excluded). See README.md." + } + ] +} diff --git a/research/t488-independent-audit/audit_ceiling.py b/research/t488-independent-audit/audit_ceiling.py new file mode 100644 index 0000000000..b03d67579c --- /dev/null +++ b/research/t488-independent-audit/audit_ceiling.py @@ -0,0 +1,873 @@ +# SPDX-License-Identifier: Apache-2.0 +"""Scope A: an own fixed-inventory LP ceiling for the staged 644/660 supplier and its attainment by PR3633 (see the +provenance note below). + +Model (PR3633 ceiling/PROOF.md and owners/UPSTREAM-PROOF.md; PR3630's check_budget.py literal constants are parsed +and compared below and state the same model with T = 740, live 896 T). Helper widths W = (4,5,6,7,8,9) with per-call +inventory I = (26,2,18,52,97,213) and per-call local paid bill L = 2439H1+726H2+224H3+102H4+20H5+8H8; nine calls per +replica; per replica n = (672,448,448,448) bins of capacities (8,16,32,48) (18 source children x 112 ports); live +stock 896 per replica; dimension 73; H_r(a) = r^(1-a). Over its lifetime a bin holds a multiset of helper roles (a +pattern p, p_w roles of width w) with |p| = sum_w w p_w <= cap and is charged one idle child H_{cap-|p|} (nothing +when |p| = cap). The complete profile of T replicas is 9T copies of L plus the idle children; its moment is +M_T(a) = sum_r n_r r (73/r)^a / (73 * 896 T) (audit_kappa.moment with live = 896T) and its 24-digit endpoint C is +the largest 24-digit decimal with M_T(C) < 1. + +Per-replica LP. Variables x_{c,p} >= 0 for every pattern of every capacity (the empty pattern included); +sum_p x_{c,p} = n_c (4 rows) and sum_{c,p} p_w x_{c,p} = 9 I_w (6 rows); cost sum x_{c,p} H_{c-|p|}(a) + 9 L. +Patterns are generated as multisets of roles (each multiset extended only by widths >= its largest member) and +cross-validated against an independent generating-function count. Search: an own two-phase revised simplex +(Bland's rule, exact Fractions, integer costs round(10^60 * mid of the audit_kappa enclosure of 73^a H_r(a)) at +a_hi = 0.001730096363528623534231; the common factor 73^a does not change the optimal basis). Certificate, checked +by check() without trusting the search: the patterns are complete and valid; the basic solution is exact, rational, +>= 0 and meets all ten equalities; the dual y is a vector of exact rational combinations of the symbols H_r; the +reduced cost of EVERY column is a rational combination of H_r that is either identically zero or has a rigorous +interval lower bound > 0 at a_hi AND uniformly on the decimal bracket [a_lo, a_hi] (audit_kappa.H encloses +r (73/r)^a = 73^a H_r(a), a common positive factor, increasing in a for r < 73); b.y = c.x symbolically. + +Ceiling theorem (finite certificate + this written argument). Hypotheses: the widths, capacities, per-replica bin +counts n, per-call inventory I, local bill L and nine-call normalization are exactly as above; T is any positive +integer; every one of the 9T I_w width-w roles lies in exactly one bin; every one of the n_c T capacity-c bins carries +one pattern with |p| <= c and is charged exactly H_{c-|p|}; the complete profile is 9T L plus those idle children; +the moment uses live stock 896T and m = 73. Claim: every such allocation has M_T(a_hi) > 1, so its root is < a_hi +and its 24-digit endpoint is <= a_lo = a_hi - 1e-24. Proof: x = (bin counts)/T is LP-feasible; weak duality with +reduced costs >= 0 at a_hi gives idle cost(a_hi) >= T b.y(a_hi) = T c.x*(a_hi); hence sum_r n_r H_r(a_hi) >= +T sum_r P_r H_r(a_hi) for the LP-minimum per-replica profile P, i.e. M_T(a_hi) >= M_P(a_hi) (live 896) > 1 by the +enclosure. All ranks are in [1, 48] < 73, so every term r (73/r)^a increases with a and M_T(a) > 1 for all a >= a_hi; +any 24-digit C with M_T(C) < 1 is < a_hi, i.e. C <= a_lo. Adding further paid children of rank in [1,72] only +raises M_T, so the bound persists. Relaxed (dropped): source windows, physical families, chronology, call-copy +decomposition, slot limits and integrality. NOT covered: any other width vector, inventory, local bill, +capacity/bin-count model, live-stock scaling or call normalization. The dual is certified at a_hi and uniformly on +[a_lo, a_hi] only; no all-exponent dual claim. Every LP-optimal solution is supported on the symbolically tight +columns (strictly positive reduced costs elsewhere), so its profile 9L + b.y is unique. + +Attainment. PR3633's T = 488 aggregate (owners/CANDIDATE.json: groups [cap, lo, hi, n_g] per base T = 61, x +[group, pattern, count], z [group, macro-time t, width, count], S = 8) uses only tight patterns, has 488 n_c bins of +each capacity, S n_g bins per group, 4392 I_w roles of each width, z inside lo < t <= hi with width balance, call +inventory and group degree, and its idle profile + 4392 L equals the claimed profile = 488 P. Its endpoint is +therefore exactly a_lo (accepted at a_lo, rejected at a_hi): no allocation in this fixed model has a larger 24-digit +endpoint. + +Cross-checks. PR3633 ceiling/CEILING.json: model, hashed candidate field list (must be the 11 scientific fields) +and candidate sha256 binding, basis, value, dual feasibility on the own columns at their exponent (a rational with +denominator 2^60 10^6, below a_hi) and at a_hi, b.y, the root polynomial (= lcm-scaled P minus scale * 896 at rank +73, i.e. live 896 per replica) and the root interval bracketing. PR3633 CLAIM.json: profile, root interval, kappa, +candidate_endpoint, calls and roles. PR3630: the literal constants of check_budget.py (ast.literal_eval of literal +right-hand sides) equal this model with W = 896 T; its parsed basis is optimal with the same value; its dual, +re-derived by the own symbolic solver, is positive uniformly on its [LO, HI] with its stated (12, 2539) counts, and +its reduced-cost coefficient digest (own reduced costs, PR3630's documented serialisation) equals VALIDATION.json +dual.coefficient_sha256; its README bound C < a_hi. + +Rejections raise audit_common.AuditFail('ceiling.: detail'); the vendored audit_kappa engine can also raise +AuditFail('kappa.: ...') (e.g. kappa.profile for a nonpositive count, kappa.rank for rank >= 73); a missing +key, wrong shape or unparsable literal raises AuditFail('ceiling.input_format: ...'). Every negative control runs +through check() on a deep copy and counts only on its expected rule id; they include three crafted duals whose +reduced cost is undecidable inside the enclosure at a_hi or changes sign inside (a_lo, a_hi), so the interval-endpoint +choices of lower_at/lower_uniform and the evaluation exponent are exercised. + +Independence: own code from the stated mathematics; PR3633/PR3630/PR3605 files are read as data only (PR3630's +check_budget.py is parsed by regular expressions and ast.literal_eval, never executed). Provenance note: a first +draft of this module mirrored the shape of PR3633's ceiling/verify_ceiling.py pattern walk and column builder and +of owners/verify_candidate.py's z loop; after review those parts were rewritten (multiset-extension enumeration, +sparse columns, tally-based aggregate checks). + +Credits: the capacity model and the finite-pattern primal/dual approach descend from the retained coupled-carrier- +cap740 certificate lineage on main (research/coupled-carrier-cap740); Ross Richardson's PR3630 (independent 2,551- +pattern dual and all-T ceiling) and PR3605 (staged 644/660 width 8->7 supplier); hexapode's PR3633 (T488 aggregate, +ceiling certificate). Standard library only. +""" +import ast +import copy +import hashlib +import json +import math +import os +import re +import subprocess +import sys +import time +from fractions import Fraction as Q +from pathlib import Path + +try: + _HERE = Path(__file__).resolve().parent + if str(_HERE / 'vendor' / 'pr3053') not in sys.path: + sys.path.insert(0, str(_HERE / 'vendor' / 'pr3053')) +except NameError: # loaded by exec: the integration supplies the vendored modules + _HERE = None + +import audit_common as AC +import audit_kappa as AK + +need = AC.need +E24 = Q(1, 10 ** 24) +A_LO = Q('0.001730096363528623534230') +A_HI = Q('0.001730096363528623534231') +T488 = 488 +MODEL = dict(widths=(4, 5, 6, 7, 8, 9), caps=(8, 16, 32, 48), n=(672, 448, 448, 448), inv=(26, 2, 18, 52, 97, 213), + local={1: 2439, 2: 726, 3: 224, 4: 102, 5: 20, 8: 8}, calls=9, live=896) +FIELDS3633 = ['baseT', 'S', 'order', 'dispatch', 'groups', 'widths', 'inventory', 'local_profile', 'x', 'z', 'profile'] +PR3633 = '72c268f113368ea0387d188225e3f81ba6d6ef73' +PR3630 = 'bcac7fe04b9b7f7f5d3c27d620561f96c4ae5608' +PINS = { # key: (commit, path, sha256) + 'candidate': (PR3633, 'research/staged-width-t488/owners/CANDIDATE.json', + 'fbdda845beac5bba9baa4b1b8c8f596e98f70b59e1d60ebf4f4d3e8758631bfd'), + 'expected': (PR3633, 'research/staged-width-t488/owners/EXPECTED.json', + '924493b6bbeade055af4916ac7bd6d7d7ec0600719f5c39636dc3f16d7ec14ea'), + 'ceiling3633': (PR3633, 'research/staged-width-t488/ceiling/CEILING.json', + 'a26bf26118222a5bcb6bf9f698d6b8b86003ff7b1479fd68b31f67c74945e460'), + 'claim': (PR3633, 'research/staged-width-t488/CLAIM.json', + '3332fb6f6d5ae81da8ead667973491ebefa1803d6d96a06790386ac16243f332'), + 'budget3630': (PR3630, 'research/staged-644-660-owner-successor/check_budget.py', + '639820c9ccf72b88ae16768055ec5500a66004784f57a2fa5f218bf4d2e4ee05'), + 'readme3630': (PR3630, 'research/staged-644-660-owner-successor/README.md', + '79cf49cd2274364888830d63a664f514c7b1a51d3ac54e57dc4117d432078e31'), + 'validation3630': (PR3630, 'research/staged-644-660-owner-successor/VALIDATION.json', + '1d85df49a5e4b9d1262cfaffa9725e03bfe660f395bc6c08d594a0b0551fc24e'), +} + + +def read_pinned(repo, key): + commit, path, sha = PINS[key] + env = {k: v for k, v in os.environ.items() if not k.startswith('GIT_')} + p = subprocess.run(['git', '--no-replace-objects', '-C', str(repo), 'show', '%s:%s' % (commit, path)], + stdout=subprocess.PIPE, stderr=subprocess.PIPE, env=env) + need(p.returncode == 0, 'ceiling.input: git show %s:%s failed: %s' % (commit[:12], path, p.stderr.decode()[:200])) + need(hashlib.sha256(p.stdout).hexdigest() == sha, 'ceiling.input: %s sha256 mismatch' % key) + return p.stdout + + +# ---------------------------------------------------------------- patterns, columns, symbols, intervals + +def enumerate_patterns(widths, caps): + """Every (cap, p): p a nonnegative count vector over widths with sum_w w p_w <= cap (empty pattern included). + + Roles are added one at a time in nondecreasing width order, so each multiset of roles is produced once; the + result is sorted by (cap, counts), the lexicographic order PR3630 serialises its coefficients in.""" + top, layer, multisets = max(caps), [((), 0)], [((), 0)] + while layer: + layer = [(ms + (w,), s + w) for ms, s in layer for w in widths if (not ms or w >= ms[-1]) and s + w <= top] + multisets += layer + vectors = [(tuple(ms.count(w) for w in widths), s) for ms, s in multisets] + return sorted((c, v) for c in caps for v, s in vectors if s <= c) + + +def dp_count(widths, cap): + """Number of nonnegative vectors with sum w p_w <= cap: prefix sum of the coefficients of prod 1/(1 - x^w).""" + f = [1] + [0] * cap + for w in widths: + for s in range(w, cap + 1): + f[s] += f[s - w] + return sum(f) + + +def used(widths, q): + return sum(w * k for w, k in zip(widths, q)) + + +def column(model, cap, q): + """Sparse LP column {row: coefficient}: the capacity row of cap, then one row per width carrying its count.""" + col = {model['caps'].index(cap): 1} + col.update((len(model['caps']) + i, k) for i, k in enumerate(q) if k) + return col + + +def times(model, pats, amounts): + """A x for the given columns and amounts, as a dense list of the ten row sums.""" + out = [Q(0)] * (len(model['caps']) + len(model['widths'])) + for (c, q), v in zip(pats, amounts): + for i, a in column(model, c, q).items(): + out[i] += a * v + return out + + +def rhs(model): + return [Q(v) for v in model['n']] + [Q(model['calls'] * v) for v in model['inv']] + + +def tally(items, key): + """Sum the counts of (k, n) items grouped by key(k).""" + out = {} + for k, n in items: + out[key(k)] = out.get(key(k), 0) + n + return out + + +def sym_add(acc, d, coef=1): + for r, v in d.items(): + acc[r] = acc.get(r, 0) + coef * v + if acc[r] == 0: + del acc[r] + return acc + + +def idle_sym(model, cap, q): + i = cap - used(model['widths'], q) + return {i: Q(1)} if i else {} + + +def reduced_cost(model, cap, q, y): + acc = dict(idle_sym(model, cap, q)) + for i, a in column(model, cap, q).items(): + sym_add(acc, y[i], -a) + return acc + + +def b_dot(b, y): + acc = {} + for bi, yi in zip(b, y): + sym_add(acc, yi, bi) + return acc + + +_H = {} + + +def h_iv(r, a): + """Rigorous enclosure of r (73/r)^a = 73^a H_r(a) from the vendored audit_kappa engine (cached).""" + if (r, a) not in _H: + _H[r, a] = AK.H(r, a) + return _H[r, a] + + +def lower_at(sym, a): + """Lower bound of 73^a * sum_r c_r H_r(a).""" + return sum((c * h_iv(r, a)[0 if c > 0 else 1] for r, c in sym.items()), Q(0)) + + +def lower_uniform(sym, a0, a1): + """Lower bound of 73^a * sum_r c_r H_r(a) over a0 <= a <= a1 (each r (73/r)^a increases in a since r < 73).""" + return sum((c * (h_iv(r, a0)[0] if c > 0 else h_iv(r, a1)[1]) for r, c in sym.items()), Q(0)) + + +def inverse(B): + m = len(B) + z = [[Q(v) for v in row] + [Q(int(i == j)) for j in range(m)] for i, row in enumerate(B)] + for j in range(m): + k = next((k for k in range(j, m) if z[k][j]), None) + need(k is not None, 'ceiling.basis_singular: basis matrix is singular') + z[j], z[k] = z[k], z[j] + p = z[j][j] + z[j] = [v / p for v in z[j]] + for i in range(m): + if i != j and z[i][j]: + f = z[i][j] + z[i] = [u - f * v for u, v in zip(z[i], z[j])] + return [row[m:] for row in z] + + +def symbolic_dual(model, basis): + """y with B^T y = c_B, each y_i an exact rational combination {rank: coefficient} of the symbols H_r.""" + m = len(basis) + cols = [column(model, cap, q) for cap, q in basis] + Binv = inverse([[col.get(i, 0) for col in cols] for i in range(m)]) + y = [] + for i in range(m): + acc = {} + for k, (cap, q) in enumerate(basis): + if Binv[k][i]: + sym_add(acc, idle_sym(model, cap, q), Binv[k][i]) + y.append(acc) + return y + + +def profile_of(model, basis, x): + prof = {r: model['calls'] * Q(v) for r, v in model['local'].items()} + for (cap, q), v in zip(basis, x): + sym_add(prof, idle_sym(model, cap, q), v) + return prof + + +# ---------------------------------------------------------------- search (own simplex; not trusted by check) + +def simplex(model, patterns, a): + m, b = 10, rhs(model) + cols = [column(model, cap, q) for cap, q in patterns] + real = len(cols) + cols += [{i: 1} for i in range(m)] + costs = [] + for cap, q in patterns: + i = cap - used(model['widths'], q) + lo, hi = h_iv(i, a) if i else (0, 0) + costs.append(round((lo + hi) / 2 * 10 ** 60)) + basis, iters = list(range(real, real + m)), [0, 0] + + def phase(cost, ph): + while True: + Binv = inverse([[cols[j].get(i, 0) for j in basis] for i in range(m)]) + xB = [sum(Binv[i][k] * b[k] for k in range(m)) for i in range(m)] + y = [sum(cost[basis[k]] * Binv[k][i] for k in range(m)) for i in range(m)] + inb = set(basis) + enter = next((j for j in range(real) if j not in inb + and cost[j] - sum(y[i] * v for i, v in cols[j].items()) < 0), None) + if enter is None: + return Binv, xB + d = [sum(Binv[i][k] * v for k, v in cols[enter].items()) for i in range(m)] + cand = [(xB[i] / d[i], basis[i], i) for i in range(m) if d[i] > 0] + need(cand, 'ceiling.search: unbounded LP') + basis[min(cand)[2]] = enter + iters[ph] += 1 + Binv, xB = phase([0] * real + [1] * m, 0) + need(sum(v for v, j in zip(xB, basis) if j >= real) == 0, 'ceiling.search: LP infeasible') + for r in range(m): # drive zero-level artificials out (the 10 rows have full rank) + if basis[r] >= real: + j = next(j for j in range(real) if j not in basis + and sum(Binv[r][k] * v for k, v in cols[j].items()) != 0) + basis[r] = j + Binv = inverse([[cols[jj].get(i, 0) for jj in basis] for i in range(m)]) + Binv, xB = phase(costs + [0] * m, 1) + order = sorted(range(m), key=lambda i: basis[i]) + B = [patterns[basis[i]] for i in order] + return dict(basis=B, x=[xB[i] for i in order], y=symbolic_dual(model, B), iterations=iters) + + +# ---------------------------------------------------------------- parsing of PR3630 literals (data only) + +def parse_pr3630(src, readme): + def lit(pattern, what): + m = re.search(pattern, src, re.M) + need(m, 'ceiling.x3630_parse: %s literal not found' % what) + try: + return ast.literal_eval(m.group(1)) + except (ValueError, SyntaxError) as e: + need(False, 'ceiling.x3630_parse: %s is not a literal (%s)' % (what, e)) + consts = {k: lit(r'^%s = (.+)$' % k, k) for k in ('CAPS', 'WIDTHS', 'CAP_TOTAL', 'INVENTORY', 'L')} + consts['M'], consts['T'], consts['W'] = lit(r'^M, T, W = (.+)$', 'M, T, W') + consts['calls'] = 9 if re.search(r'^rhs = list\(map\(Q, CAP_TOTAL\)\) \+ \[Q\(9 \* n\) for n in INVENTORY\]$', + src, re.M) else None + m = re.search(r'^specs = \[(.*)\]$', src, re.M) + need(m, 'ceiling.x3630_parse: specs literal not found') + specs = [(int(c), tuple(int(v) for v in ws.replace(' ', '').split(',') if v), Q(int(p), int(d or 1))) + for c, ws, p, d in re.findall(r'\((\d+), \(([\d, ]*)\), Q\((\d+)(?:, (\d+))?\)\)', m.group(1))] + lo = re.search(r"^LO = Q\('([0-9.]+)'\)$", src, re.M) + hi = re.search(r'^HI = LO \+ Q\(1, 10 \*\* 24\)$', src, re.M) + cnt = re.search(r'\(len\(tight\), len\(strict\)\) == \((\d+), (\d+)\)', src) + bound = re.search(r'C < (0\.\d{24})', readme) + need(len(specs) == 10 and lo and hi and cnt and bound, 'ceiling.x3630_parse: basis/endpoints/counts/bound') + return dict(consts=consts, specs=specs, LO=lo.group(1), HI='LO + 1e-24', + counts=[int(cnt.group(1)), int(cnt.group(2))], bound=bound.group(1)) + + +def digest3630(model, pats, rcs): + """PR3630's coefficient digest: per pattern in (cap, counts) order, json [[cap, counts, idle], sorted (r, str)].""" + h = hashlib.sha256() + for c, q in sorted(pats): + h.update(json.dumps([[c, list(q), c - used(model['widths'], q)], sorted((r, str(v)) for r, v in + rcs[c, q].items())], + separators=(',', ':')).encode()) + return h.hexdigest() + + +def projection_sha(cand, fields): + return hashlib.sha256(json.dumps({k: cand[k] for k in fields}, sort_keys=True, separators=(',', ':')).encode() + ).hexdigest() + + +# ---------------------------------------------------------------- the main check + +def rule_of(e): + return str(e).split(':', 1)[0] + + +def check(D): + """Main check; the positive run and every negative control go through here.""" + try: + return _check(D) + except (KeyError, IndexError, TypeError, ValueError, ZeroDivisionError, AttributeError) as e: + raise AC.AuditFail('ceiling.input_format: malformed input (%s: %s)' % (type(e).__name__, e)) from e + + +def _check(D): + model, a_hi, rec = D['model'], D['a_hi'], {} + W, caps, calls = model['widths'], model['caps'], model['calls'] + b = rhs(model) + # 1. enumeration: every pattern fits, no duplicates, and the per-capacity count equals the DP count + pats = D['patterns'] + need(all(c in caps and len(q) == len(W) and all(type(k) is int and k >= 0 for k in q) and used(W, q) <= c + for c, q in pats), 'ceiling.enum_valid: a listed pattern does not fit its capacity') + need(len(set(pats)) == len(pats), 'ceiling.enum_distinct: duplicate pattern') + split = {c: sum(1 for cc, _ in pats if cc == c) for c in caps} + dp = {c: dp_count(W, c) for c in caps} + need(split == dp, 'ceiling.enum_complete: per-capacity counts %s != independent counts %s' % (split, dp)) + rec['patterns'] = dict(per_capacity={str(c): split[c] for c in caps}, total=len(pats)) + pset = set(pats) + # 2. primal: exact basic solution, >= 0, all ten equalities + cert = D['cert'] + basis, x, y = [(c, tuple(q)) for c, q in cert['basis']], cert['x'], cert['y'] + need(len(basis) == len(x) == 10 and all(p in pset for p in basis), 'ceiling.primal_column: basis not in enumeration') + need(all(v >= 0 for v in x), 'ceiling.primal_sign: negative basic amount') + Ax = times(model, basis, x) + need(Ax == b, 'ceiling.primal_rows: A x = %s != b = %s' % (Ax, b)) + # 3. dual feasibility of every column: symbolic zero, or rigorous positive lower bound at a_hi and on [a_lo, a_hi] + need(len(y) == 10, 'ceiling.dual_shape: ten dual entries required') + tight, strict, rcs = [], [], {} + for c, q in pats: + rc = reduced_cost(model, c, q, y) + rcs[c, q] = rc + if not rc: + tight.append((c, q)) + continue + lo = lower_at(rc, a_hi) + need(lo > 0, 'ceiling.dual_feasible: reduced cost of (%d, %s) = %s has lower bound %.3e at a_hi' + % (c, q, rc, float(lo))) + need(lower_uniform(rc, A_LO, A_HI) > 0, 'ceiling.dual_uniform: reduced cost of (%d, %s) not positive on ' + '[a_lo, a_hi]' % (c, q)) + strict.append(lo) + tset = set(tight) + # 4. strong duality: b.y == c.x symbolically + by = b_dot(b, y) + cx = {} + for (c, q), v in zip(basis, x): + sym_add(cx, idle_sym(model, c, q), v) + need(by == cx, 'ceiling.strong_duality: b.y %s != c.x %s' % (by, cx)) + P = profile_of(model, basis, x) + rec['lp'] = dict(a_hi=str(a_hi), basis=[dict(capacity=c, pattern=list(q), idle=c - used(W, q), x=str(v)) + for (c, q), v in zip(basis, x)], + dual=[{str(r): str(v) for r, v in sorted(yi.items())} for yi in y], + symbolically_zero=len(tight), strictly_positive=len(strict), + min_strict_lower_scaled_by_73a='%.6e' % float(min(strict)), + dual_uniform_on_bracket=[str(A_LO), str(A_HI)], + tight_patterns=[[c, list(q)] for c, q in tight], + minimum_profile_per_replica={str(r): str(v) for r, v in sorted(P.items())}, + idle_per_replica={str(r): str(v) for r, v in sorted(cx.items())}) + # 5. claimed profile (EXPECTED and CLAIM, T = 488) / 488 == LP minimum + cand = D['candidate'] + S, G, base = cand['S'], cand['groups'], cand['baseT'] + T = base * S + need(T == D['expected']['replicas'] == D['claim']['replicas'] == T488, 'ceiling.claimed_profile: T != 488') + for name in ('expected', 'claim'): + cp = {int(r): Q(v, T) for r, v in D[name]['profile'].items()} + need(cp == P, 'ceiling.claimed_profile: %s profile / 488 != LP minimum' % name) + # 6. attainment by the PR3633 aggregate + need(all(type(row) is list and len(row) == 4 and all(type(v) is int for v in row) and row[3] > 0 for row in G) + and all(type(e) is list and len(e) == 3 and type(e[0]) is int and type(e[2]) is int and type(e[1]) is list + and len(e[1]) == len(W) and all(type(k) is int for k in e[1]) for e in cand['x']) + and all(type(e) is list and len(e) == 4 and all(type(v) is int for v in e) for e in cand['z']), + 'ceiling.input_format: candidate groups/x/z must be integer lists [cap,lo,hi,n>0]/[g,[6],n]/[g,t,w,n]') + need(cand['widths'] == list(W) and {int(k): v for k, v in cand['inventory'].items()} == dict(zip(W, model['inv'])) + and {int(k): v for k, v in cand['local_profile'].items()} == model['local'] + and len(cand['order']) == calls * base and D['expected']['calls'] == calls * T, + 'ceiling.attain_model: candidate widths/inventory/local bill/nine-call normalization') + bins = {(g, tuple(q)): n for g, q, n in cand['x']} + need(len(bins) == len(cand['x']) and all(0 <= g < len(G) and n > 0 for (g, q), n in bins.items()), + 'ceiling.attain_format: x has a repeated (group, pattern), a group index out of range or a count <= 0') + kinds = {(G[g][0], q) for g, q in bins} + need(kinds <= pset, 'ceiling.attain_pattern: %s not enumerated' % sorted(kinds - pset)[:3]) + loose = sorted(kinds - tset) + need(not loose, 'ceiling.attain_tight: %d aggregate bin pattern(s) have positive reduced cost, e.g. %s' + % (len(loose), loose[:3])) + need(tally(bins.items(), lambda k: k[0]) == {g: S * row[3] for g, row in enumerate(G)}, + 'ceiling.attain_bins: bins per group != S n_g') + per_cap = tally(bins.items(), lambda k: G[k[0]][0]) + need(per_cap == {c: T * nc for c, nc in zip(caps, model['n'])} + and tally(((row[0], row[3]) for row in G), lambda c: c) == {c: base * nc for c, nc in zip(caps, model['n'])}, + 'ceiling.attain_bins: bins per capacity %s != 488 n' % per_cap) + roles = [sum(n * k[1][i] for k, n in bins.items()) for i in range(len(W))] + need(roles == [calls * T * v for v in model['inv']], 'ceiling.attain_roles: roles per width %s != 4392 I' % roles) + flow = {(g, t, w): n for g, t, w, n in cand['z']} + need(len(flow) == len(cand['z']) and all(0 <= g < len(G) and w in W and n > 0 for (g, t, w), n in flow.items()), + 'ceiling.attain_format: z has a repeated (group, time, width), a bad group/width or a count <= 0') + outside = sorted(k for k in flow if not G[k[0]][1] < k[1] <= G[k[0]][2]) + need(not outside, 'ceiling.attain_window: %d z entries outside their group window (lo, hi], e.g. %s' + % (len(outside), outside[:3])) + supply = tally((((g, w), n * k) for (g, q), n in bins.items() for w, k in zip(W, q) if k), lambda k: k) + need(tally(flow.items(), lambda k: (k[0], k[2])) == supply, + 'ceiling.attain_zbalance: per (group, width) z total != roles carried by that group\'s bins') + need(tally(flow.items(), lambda k: (k[1], k[2])) == {(t, w): S * v for t in range(len(cand['order'])) + for w, v in zip(W, model['inv'])}, + 'ceiling.attain_zcalls: per (macro-call, width) z total != S I_w') + over = sorted(k for k, v in tally(flow.items(), lambda k: (k[0], k[1])).items() if v > S * G[k[0]][3]) + need(not over, 'ceiling.attain_zdegree: %d (group, time) loads exceed S n_g, e.g. %s' % (len(over), over[:3])) + idle = tally(bins.items(), lambda k: G[k[0]][0] - used(W, k[1])) + idle.pop(0, None) + full = sym_add({r: calls * T * v for r, v in model['local'].items()}, idle) + for name, prof in (('candidate', cand['profile']), ('expected', D['expected']['profile']), + ('claim', D['claim']['profile'])): + need({int(r): v for r, v in prof.items()} == full, 'ceiling.attain_profile: idle + 4392 L != %s profile' % name) + used_kinds = sorted([c, list(q)] for c, q in kinds) + rec['attainment'] = dict(T=T, S=S, base_T=base, groups=len(G), x_entries=len(bins), z_entries=len(flow), + bins_per_capacity={str(c): v for c, v in sorted(per_cap.items())}, + roles_per_width={str(w): v for w, v in zip(W, roles)}, + idle_profile={str(r): v for r, v in sorted(idle.items())}, + full_profile={str(r): v for r, v in sorted(full.items())}, + tight_patterns_used=len(kinds), + nonbasic_tight_used=[k for k in used_kinds if (k[0], tuple(k[1])) not in set(basis)], + equals_488_times_LP_minimum=True) + # 7. root: claimed endpoint C accepted, C + 1e-24 rejected (live 896 T), for the LP minimum and the claimed profile + live = model['live'] * T + cl = D['claim'] + C = Q(cl['root_interval'][0]) + need(C > 0 and (C * 10 ** 24).denominator == 1, 'ceiling.endpoint_format: claimed C %s is not a 24-digit decimal' % C) + mom = lambda p, a: AK.moment(p, a, live=live) + profiles = {'lp_minimum_x488': {r: v * T for r, v in P.items()}, + 'pr3633_claimed': {int(r): v for r, v in D['expected']['profile'].items()}} + rec['root'] = {} + for name, prof in profiles.items(): + need(AK.accepts(mom(prof, C)), 'ceiling.root_accept: %s not accepted at claimed endpoint %s' % (name, C)) + need(AK.rejects(mom(prof, C + E24)), 'ceiling.root_reject: %s not rejected at %s' % (name, C + E24)) + rec['root'][name] = AK.endpoint_check(prof, C, mom=mom) + need(AK.rejects(AK.moment(P, a_hi, live=model['live'])), 'ceiling.root_ceiling: LP minimum not rejected at a_hi') + need(Q(cl['root_interval'][1]) == C + E24 and C + E24 == a_hi, + 'ceiling.claim_interval: claimed root interval is not [C, C + 1e-24] with C + 1e-24 = a_hi') + kappa = Q(cl['kappa']) + need(kappa == C - AK.KAPPA_TERMS[0] - AK.KAPPA_TERMS[1], 'ceiling.kappa: kappa != C - 1e-46 - 1e-70') + need(Q(cl['candidate_endpoint']) == kappa and cl['calls'] == calls * T and cl['roles'] == calls * T * sum(model['inv']), + 'ceiling.claim_fields: CLAIM candidate_endpoint/calls/roles differ from kappa/4392/4392 sum(I)') + rec['root'].update(endpoint=AK.dec(C, 24), kappa=cl['kappa'], candidate_endpoint_equals_kappa=True, live=live, + lp_minimum_per_replica_rejected_at_a_hi=True, + ceiling='for every positive integer T every complete allocation of this fixed model has ' + 'M_T(a_hi) > 1, so its 24-digit endpoint is <= %s; PR3633 attains it' % AK.dec(C, 24)) + # 8a. cross-check PR3633 ceiling/CEILING.json + cj = D['ceiling3633'] + need(cj['widths'] == list(W) and cj['capacities'] == list(caps) + and {int(k): v for k, v in cj['inventory'].items()} == dict(zip(W, model['inv'])) + and {int(k): v for k, v in cj['local_profile'].items()} == model['local'] + and [Q(v) for v in cj['rhs_per_replica']] == b, 'ceiling.x3633_model: CEILING.json model differs') + need(cj['candidate_scientific_fields'] == FIELDS3633, + 'ceiling.x3633_fields: hashed candidate field list %s != the 11 scientific fields' % cj['candidate_scientific_fields']) + need(projection_sha(cand, FIELDS3633) == cj['candidate_scientific_sha256'], + 'ceiling.x3633_binding: candidate projection sha256') + cb = [(e['capacity'], tuple(e['counts'])) for e in cj['basis']] + cx3633 = [Q(e['amount_per_replica']) for e in cj['basis']] + need(all(p in pset for p in cb) and all(e['idle'] == e['capacity'] - used(W, e['counts']) for e in cj['basis']) + and times(model, cb, cx3633) == b and all(v >= 0 for v in cx3633), + 'ceiling.x3633_basis: CEILING.json primal basis infeasible') + need(profile_of(model, cb, cx3633) == P and {int(r): Q(v) for r, v in cj['full_profile_per_replica'].items()} == P, + 'ceiling.x3633_value: CEILING.json optimal value differs from the LP minimum') + y3633 = [{int(r): Q(v) for r, v in row.items()} for row in cj['dual_by_constraint']] + need(len(y3633) == 10, 'ceiling.x3633_dual: ten dual entries required') + e3633 = Q(cj['exponent']) + xs = {} + for label, a in (('at_their_exponent', e3633), ('at_a_hi', a_hi)): + z = s = 0 + for c, q in pats: + rc = reduced_cost(model, c, q, y3633) + if not rc: + z += 1 + else: + need(lower_at(rc, a) > 0, 'ceiling.x3633_dual: CEILING.json dual infeasible at %s for (%d, %s)' + % (label, c, q)) + s += 1 + xs[label] = [z, s] + need(b_dot(b, y3633) == cx, 'ceiling.x3633_dual_value: CEILING.json dual value b.y differs from the LP minimum') + scale = math.lcm(*(v.denominator for v in P.values())) + poly = {int(r): v for r, v in cj['scaled_root_profile'].items()} + need(poly == {**{r: int(v * scale) for r, v in P.items()}, 73: -scale * model['live']}, + 'ceiling.x3633_rootpoly: CEILING.json root polynomial != %d P - %d H73 (live 896 per replica)' + % (scale, scale * model['live'])) + rl, rh = (Q(v) for v in cj['root_interval']) + need(e3633 == rh and rl < rh and AK.accepts(AK.moment(P, rl, live=model['live'])) + and AK.rejects(AK.moment(P, rh, live=model['live'])) and A_LO < rh <= a_hi, + 'ceiling.x3633_root: CEILING.json root interval does not bracket the LP-minimum root below a_hi') + rec['x3633'] = dict(basis_equal=sorted(cb) == sorted(basis), amounts_equal=dict(zip(cb, cx3633)) == dict(zip(basis, x)), + dual_equal=y3633 == y, zero_strict=xs, value_equal=True, field_list_pinned=True, + root_polynomial_scale=scale, exponent=cj['exponent'], + exponent_denominator_is_2p60_10p6=e3633.denominator == 2 ** 60 * 10 ** 6, + exponent_minus_a_hi='%.6e' % float(e3633 - a_hi), root_interval=cj['root_interval'], + root_lower_minus_a_lo='%.6e' % float(rl - A_LO), + note='their dual is certified at a rational exponent e = their root-interval upper end, with ' + 'a_lo < e <= a_hi; by monotonicity rejection at e implies rejection at a_hi; their root ' + 'interval is not the decimal bracket but brackets the same unique root') + # 8b. cross-check PR3630 (constants and basis parsed from check_budget.py; dual re-derived by the own solver) + r30 = D['pr3630'] + k30 = r30['consts'] + need(tuple(k30['CAPS']) == caps and tuple(k30['WIDTHS']) == W and tuple(k30['CAP_TOTAL']) == model['n'] + and tuple(k30['INVENTORY']) == model['inv'] and k30['L'] == model['local'] and k30['calls'] == calls + and k30['M'] == 73 and type(k30['T']) is int and k30['T'] > 0 and k30['W'] == model['live'] * k30['T'], + 'ceiling.x3630_model: PR3630 constants %s differ from this model' % k30) + b30 = [(c, tuple(ws.count(w) for w in W)) for c, ws, _ in r30['specs']] + x30 = [v for _, _, v in r30['specs']] + need(all(p in pset for p in b30) and all(v > 0 for v in x30) and times(model, b30, x30) == b, + 'ceiling.x3630_basis: PR3630 basis infeasible') + need(profile_of(model, b30, x30) == P, 'ceiling.x3630_value: PR3630 optimal value differs from the LP minimum') + y30 = symbolic_dual(model, b30) + lo30 = Q(r30['LO']) + z = s = 0 + rc30 = {} + for c, q in pats: + rc = rc30[c, q] = reduced_cost(model, c, q, y30) + if not rc: + z += 1 + continue + need(lower_at(rc, a_hi) > 0 and lower_uniform(rc, lo30, lo30 + E24) > 0, + 'ceiling.x3630_dual: PR3630 dual not positive for (%d, %s)' % (c, q)) + s += 1 + need([z, s] == r30['counts'], 'ceiling.x3630_dual: zero/strict counts %s != stated %s' % ([z, s], r30['counts'])) + v30 = D['validation3630']['dual'] + dig = digest3630(model, pats, rc30) + need(dig == v30['coefficient_sha256'] and [v30['patterns'], v30['symbolically_zero'], v30['strict']] + == [len(pats), z, s], 'ceiling.x3630_digest: own reduced-cost digest %s or counts != VALIDATION.json dual %s' + % (dig, v30)) + need(lo30 == A_LO and lo30 + E24 == a_hi and Q(r30['bound']) == a_hi, + 'ceiling.x3630_bound: PR3630 stated bracket/bound differ from [a_lo, a_hi]') + rec['x3630'] = dict(model_equal=True, model_T=k30['T'], model_live=k30['W'], + basis_equal=sorted(b30) == sorted(basis), amounts_equal=dict(zip(b30, x30)) == dict(zip(basis, x)), + dual_equal_to_own=y30 == y, zero_strict=[z, s], uniform_on_bracket=True, value_equal=True, + coefficient_sha256=dig, digest_matches_validation=True, + stated_bound=r30['bound'], bound_confirmed=True) + return rec + + +# ---------------------------------------------------------------- driver and negative controls + +def load(repo): + raw = {k: read_pinned(repo, k) for k in PINS} + D = dict(model=MODEL, a_hi=A_HI, patterns=enumerate_patterns(MODEL['widths'], MODEL['caps']), + candidate=json.loads(raw['candidate']), expected=json.loads(raw['expected']), + ceiling3633=json.loads(raw['ceiling3633']), claim=json.loads(raw['claim']), + validation3630=json.loads(raw['validation3630']), + pr3630=parse_pr3630(raw['budget3630'].decode(), raw['readme3630'].decode())) + return D, {k: dict(commit=PINS[k][0], path=PINS[k][1], sha256=PINS[k][2]) for k in PINS} + + +def controls(D, rec): + """Negative controls: each mutates a deep copy of D and runs it through check(); killed only on its rule id.""" + tight = {(c, tuple(q)) for c, q in rec['lp']['tight_patterns']} + basis = [(c, tuple(q)) for c, q in D['cert']['basis']] + W, G0 = D['model']['widths'], D['candidate']['groups'] + inc = lambda v: v + 1 + out = [] + + def edit(*path, f): # replace the value at a nested path by f(value) + def mut(d): + o = d + for k in path[:-1]: + o = o[k] + o[path[-1]] = f(o[path[-1]]) + return mut + + def qstr(delta): + return lambda v: str(Q(v) + delta) + + def dual_row0(g): # capacity-8 row dual + g (a {rank: coefficient} combination) + return edit('cert', 'y', 0, f=lambda yi: sym_add(dict(yi), g)) + + def h1_over_h3(a, pick3): # q with q H3 ~ H1 at exponent a (from the cached enclosures) + (l1, h1), (l3, h3) = h_iv(1, a), h_iv(3, a) + return (l1 + h1) / 2 / (l3 if pick3 == 'lo' else (l3 + h3) / 2) + q_hi = h1_over_h3(A_HI, 'lo') # q lo(H3) = mid(H1) at a_hi: sign undecidable inside the enclosure + q_mid = h1_over_h3((A_LO + A_HI) / 2, 'mid') # H1 - q H3 crosses zero inside (a_lo, a_hi), increasing in a + + def dup_tight(d): # a tight basic column replaced by a duplicate of another column of its capacity + i = next(i for i, p in enumerate(d['patterns']) if p in tight and p in basis) + d['patterns'][i] = next(p for p in d['patterns'] if p[0] == d['patterns'][i][0] and p != d['patterns'][i]) + + def x_add(c, g, q, n): # n more bins of pattern q in group g (merging, dropping emptied entries) + e = next((e for e in c['x'] if e[0] == g and e[1] == q), None) + if e is None: + c['x'].append([g, list(q), n]) + else: + e[2] += n + c['x'] = [e for e in c['x'] if e[2]] + + def swap_pattern(pick): # one x entry's pattern replaced by another of its capacity chosen by pick + def mut(d): + c = d['candidate'] + have = {(g, tuple(q)) for g, q, n in c['x']} + for e in c['x']: + q2 = next((p for cc, p in d['patterns'] if cc == G0[e[0]][0] and p != tuple(e[1]) and pick((cc, p)) + and (e[0], p) not in have), None) + if q2 is not None: + e[1] = list(q2) + return + need(False, 'ceiling.controls: no pattern swap available') + return mut + + def x_move(d): # one bin moved to another group of the same capacity: per-group counts broken + c = d['candidate'] + g1, q = c['x'][0][0], c['x'][0][1] + g2 = next(g for g, row in enumerate(G0) if g != g1 and row[0] == G0[g1][0]) + x_add(c, g1, q, -1) + x_add(c, g2, q, 1) + + def group_grow(d): # a capacity-48 group gets n_g + 1 and S more empty bins: per-capacity counts broken + c = d['candidate'] + g = next(g for g, row in enumerate(G0) if row[0] == 48) + c['groups'][g][3] += 1 + x_add(c, g, [0] * len(W), c['S']) + + def split(key): # an entry with count >= 2 split into two entries with the same key + def mut(d): + rows = d['candidate'][key] + e = next(e for e in rows if e[-1] >= 2) + e[-1] -= 1 + rows.append(copy.deepcopy(e[:-1]) + [1]) + return mut + + def z_time(which): + def mut(d): + c = d['candidate'] + e = c['z'][0] + if which == 'lo': + e[1] = G0[e[0]][1] + elif which == 'hi+1': + e[1] = G0[e[0]][2] + 1 + else: # another in-window macro-time: width balance kept, call inventory broken + have = {(g, t, w) for g, t, w, n in c['z']} + for e in c['z']: + t2 = next((t for t in range(G0[e[0]][1] + 1, G0[e[0]][2] + 1) if (e[0], t, e[2]) not in have), + None) + if t2 is not None: + e[1] = t2 + return + need(False, 'ceiling.controls: no free in-window macro-time') + return mut + + def z_group(d): # a z entry moved to another group whose window contains t: width balance broken + c = d['candidate'] + have = {(g, t, w) for g, t, w, n in c['z']} + for e in c['z']: + g2 = next((g for g, row in enumerate(G0) if g != e[0] and row[1] < e[1] <= row[2] + and (g, e[1], e[2]) not in have), None) + if g2 is not None: + e[0] = g2 + return + need(False, 'ceiling.controls: no other group for a z entry') + + def z_degree(d): # 1-unit 4-cycle move: balance and call totals kept, a saturated (g, t) overloaded + c, S = d['candidate'], d['candidate']['S'] + flow = {(g, t, w): n for g, t, w, n in c['z']} + load_ = tally(flow.items(), lambda k: (k[0], k[1])) + for g2, t1 in sorted(k for k, v in load_.items() if v == S * G0[k[0]][3]): + for (g1, t, w) in [k for k in flow if k[1] == t1 and k[0] != g2]: + t2 = next((k[1] for k in flow if k[0] == g2 and k[2] == w and k[1] != t1 + and G0[g1][1] < k[1] <= G0[g1][2]), None) + if t2 is not None: + for k, v in (((g1, t1, w), -1), ((g2, t1, w), 1), ((g2, t2, w), -1), ((g1, t2, w), 1)): + flow[k] = flow.get(k, 0) + v + c['z'] = [[*k, v] for k, v in flow.items() if v] + return + need(False, 'ceiling.controls: no degree move available') + + def c3633_fields(d): # shorter field list with its projection hash recomputed: only the list pin catches it + cj = d['ceiling3633'] + cj['candidate_scientific_fields'] = [f for f in FIELDS3633 if f != 'z'] + cj['candidate_scientific_sha256'] = projection_sha(d['candidate'], cj['candidate_scientific_fields']) + + def c3633_row0(delta): + return edit('ceiling3633', 'dual_by_constraint', 0, f=lambda row: {**row, '1': str(Q(row.get('1', '0')) + delta)}) + + def p30_pivot(d): # PR3630 basis -> adjacent feasible basis: a non-tight column pivoted in, unique ratio + specs, model = d['pr3630']['specs'], d['model'] + B = [(c, tuple(ws.count(w) for w in W)) for c, ws, _ in specs] + cols = [column(model, c, q) for c, q in B] + Binv = inverse([[col.get(i, 0) for col in cols] for i in range(len(B))]) + for c, q in d['patterns']: + if (c, q) in tight: + continue + dj = [sum(Binv[i][k] * v for k, v in column(model, c, q).items()) for i in range(len(B))] + ratios = sorted((specs[i][2] / dj[i], i) for i in range(len(B)) if dj[i] > 0) + if ratios and (len(ratios) == 1 or ratios[0][0] < ratios[1][0]): + th, leave = ratios[0] + specs[:] = [(s[0], s[1], s[2] - th * dj[i]) for i, s in enumerate(specs) if i != leave] + [ + (c, tuple(w for w, k in zip(W, q) for _ in range(k)), th)] + return + need(False, 'ceiling.controls: no non-degenerate pivot available') + + overfull = lambda cap: [0] * (len(W) - 1) + [cap // W[-1] + 1] + for name, mut, rule in ( + # enumeration and primal + ('drop_tight_pattern_column', edit('patterns', f=lambda ps: [p for p in ps if p != next( + b for b in basis if b in tight)]), 'ceiling.enum_complete'), + ('pattern_overfull', edit('patterns', 0, f=lambda p: (p[0], tuple(overfull(p[0])))), 'ceiling.enum_valid'), + ('duplicate_replaces_tight_column', dup_tight, 'ceiling.enum_distinct'), + ('primal_amount_negated', edit('cert', 'x', 0, f=lambda v: -v), 'ceiling.primal_sign'), + ('primal_amount_plus_1', edit('cert', 'x', 0, f=inc), 'ceiling.primal_rows'), + # dual: coarse and sharp perturbations, and three crafted near-zero reduced costs + ('dual_plus_1e-40_H1', dual_row0({1: Q(1, 10 ** 40)}), 'ceiling.dual_feasible'), + ('dual_undecidable_within_enclosure', dual_row0({1: Q(1), 3: -q_hi}), 'ceiling.dual_feasible'), + ('dual_negative_at_a_hi_positive_at_a_lo', dual_row0({1: Q(1), 3: -q_mid}), 'ceiling.dual_feasible'), + ('dual_positive_at_a_hi_negative_at_a_lo', dual_row0({1: Q(-1), 3: q_mid}), 'ceiling.dual_uniform'), + ('weak_feasible_dual_minus_1e-3_H1', dual_row0({1: -Q(1, 1000)}), 'ceiling.strong_duality'), + # claimed profiles + ('expected_profile_plus_H1', edit('expected', 'profile', '1', f=inc), 'ceiling.claimed_profile'), + ('claim_profile_plus_H1', edit('claim', 'profile', '1', f=inc), 'ceiling.claimed_profile'), + # aggregate + ('x_entry_malformed', edit('candidate', 'x', 0, f=lambda e: e[:2]), 'ceiling.input_format'), + ('x_count_float', edit('candidate', 'x', 0, 2, f=float), 'ceiling.input_format'), + ('candidate_inventory_edit', edit('candidate', 'inventory', '4', f=inc), 'ceiling.attain_model'), + ('x_entry_split_duplicate', split('x'), 'ceiling.attain_format'), + ('aggregate_overfull_pattern', edit('candidate', 'x', 0, f=lambda e: [e[0], overfull(G0[e[0]][0]), e[2]]), + 'ceiling.attain_pattern'), + ('aggregate_nontight_pattern', swap_pattern(lambda p: p not in tight), 'ceiling.attain_tight'), + ('x_count_plus_1', edit('candidate', 'x', 0, 2, f=inc), 'ceiling.attain_bins'), + ('x_bin_moved_between_groups', x_move, 'ceiling.attain_bins'), + ('group_plus_1_with_S_empty_bins', group_grow, 'ceiling.attain_bins'), + ('aggregate_other_tight_pattern', swap_pattern(lambda p: p in tight), 'ceiling.attain_roles'), + ('z_entry_split_duplicate', split('z'), 'ceiling.attain_format'), + ('z_at_window_lo', z_time('lo'), 'ceiling.attain_window'), + ('z_at_window_hi_plus_1', z_time('hi+1'), 'ceiling.attain_window'), + ('z_moved_to_other_group', z_group, 'ceiling.attain_zbalance'), + ('z_moved_to_other_call', z_time('move'), 'ceiling.attain_zcalls'), + ('z_degree_overload', z_degree, 'ceiling.attain_zdegree'), + ('candidate_profile_plus_H1', edit('candidate', 'profile', '1', f=inc), 'ceiling.attain_profile'), + # root and claim + ('claimed_endpoint_not_24_digit', edit('claim', 'root_interval', 0, f=qstr(Q(1, 10 ** 30))), + 'ceiling.endpoint_format'), + ('claimed_endpoint_plus_1e-24', edit('claim', 'root_interval', 0, f=qstr(E24)), 'ceiling.root_accept'), + ('claimed_endpoint_minus_1e-24', edit('claim', 'root_interval', 0, f=qstr(-E24)), 'ceiling.root_reject'), + ('certify_at_a_lo', edit('a_hi', f=lambda v: A_LO), 'ceiling.root_ceiling'), + ('claim_interval_upper_plus_1e-24', edit('claim', 'root_interval', 1, f=qstr(E24)), 'ceiling.claim_interval'), + ('kappa_plus_1e-70', edit('claim', 'kappa', f=qstr(Q(1, 10 ** 70))), 'ceiling.kappa'), + ('candidate_endpoint_changed', edit('claim', 'candidate_endpoint', f=lambda v: '1/1000'), + 'ceiling.claim_fields'), + # PR3633 CEILING.json + ('ceiling3633_inventory_edit', edit('ceiling3633', 'inventory', '4', f=inc), 'ceiling.x3633_model'), + ('ceiling3633_short_field_list', c3633_fields, 'ceiling.x3633_fields'), + ('candidate_dispatch_edit', edit('candidate', 'dispatch', 0, -1, f=lambda v: -7), 'ceiling.x3633_binding'), + ('ceiling3633_amount_edit', edit('ceiling3633', 'basis', 2, 'amount_per_replica', f=qstr(1)), + 'ceiling.x3633_basis'), + ('ceiling3633_full_profile_edit', edit('ceiling3633', 'full_profile_per_replica', '1', f=qstr(1)), + 'ceiling.x3633_value'), + ('ceiling3633_dual_perturb', c3633_row0(Q(1, 1000)), 'ceiling.x3633_dual'), + ('ceiling3633_weak_dual', c3633_row0(-Q(1, 1000)), 'ceiling.x3633_dual_value'), + ('ceiling3633_rootpoly_live', edit('ceiling3633', 'scaled_root_profile', '73', f=inc), + 'ceiling.x3633_rootpoly'), + ('ceiling3633_root_lower_at_upper', edit('ceiling3633', 'root_interval', f=lambda r: [r[1], r[1]]), + 'ceiling.x3633_root'), + # PR3630 + ('pr3630_live_plus_1', edit('pr3630', 'consts', 'W', f=inc), 'ceiling.x3630_model'), + ('pr3630_amount', edit('pr3630', 'specs', 2, f=lambda s: (s[0], s[1], s[2] + 1)), 'ceiling.x3630_basis'), + ('pr3630_adjacent_suboptimal_basis', p30_pivot, 'ceiling.x3630_value'), + ('pr3630_bracket_moved_to_0', edit('pr3630', 'LO', f=lambda v: '0'), 'ceiling.x3630_dual'), + ('pr3630_counts', edit('pr3630', 'counts', f=lambda v: [13, 2538]), 'ceiling.x3630_dual'), + ('pr3630_digest', edit('validation3630', 'dual', 'coefficient_sha256', f=lambda v: '0' * 64), + 'ceiling.x3630_digest'), + ('pr3630_bound', edit('pr3630', 'bound', f=lambda v: '0.001730096363528623534232'), 'ceiling.x3630_bound')): + AC.attempt(name) + d = copy.deepcopy(D) + mut(d) + try: + check(d) + got = None + except AC.AuditFail as e: + got = rule_of(e) + out.append(dict(name=name, expected=rule, raised=got, killed=got == rule)) + return out + + +def run(repo=None): + t0 = time.time() + repo = repo or (_HERE.parents[1] if _HERE else Path.cwd()) + D, pins = load(repo) + t1 = time.time() + found = simplex(D['model'], D['patterns'], D['a_hi']) + D['cert'] = dict(basis=found['basis'], x=found['x'], y=found['y']) + t2 = time.time() + rec = check(D) + t3 = time.time() + ctl = controls(D, rec) + need(all(c['killed'] for c in ctl), 'ceiling.controls: a negative control survived or raised another rule: %s' + % [c for c in ctl if not c['killed']]) + rec.update(status='PASS_FIXED_MODEL_LP_CEILING_ATTAINED_BY_PR3633', simplex_iterations=found['iterations'], + controls=ctl, inputs=pins, + seconds=dict(load=round(t1 - t0, 2), search=round(t2 - t1, 2), check=round(t3 - t2, 2), + controls=round(time.time() - t3, 2), total=round(time.time() - t0, 2)), + scope='Fixed staged 644/660 supplier (widths, inventory, local bill), capacities/bin counts, nine-call ' + 'normalization and live 896T; windows, families, chronology and integrality relaxed. Dual ' + 'certified at a_hi and uniformly on [a_lo, a_hi] only. No literal-owner, native, outer or ' + 'multiplication-theorem claim.') + return rec + + +if __name__ == '__main__': + print(json.dumps(run(sys.argv[1] if len(sys.argv) > 1 else None), indent=1)) diff --git a/research/t488-independent-audit/audit_endpoint.py b/research/t488-independent-audit/audit_endpoint.py new file mode 100644 index 0000000000..8b0991bad9 --- /dev/null +++ b/research/t488-independent-audit/audit_endpoint.py @@ -0,0 +1,232 @@ +# SPDX-License-Identifier: Apache-2.0 +"""Scope D: the complex endpoint and kappa of PR #3633 from the complete profile recounted by audit_owners, with two +rational enclosures, the exact gains over the predecessors, and the improvement tests. + +M_T(a) = sum_r n_r r (73/r)^a / (73 * 896 T); C is the largest 24-digit decimal with M_T(C) < 1 and M_T(C + 1e-24) +> 1; kappa = C - 1e-46 - 1e-70 (the inherited outer margins, recorded by every package of this line). Enclosures: +audit_kappa.moment (atanh/Taylor series with explicit tails, outward rounding to 1e-60; PR3053) and +check_accounting.moment (640-bit fixed point; PR2804). Both must accept at C and reject at C + 1e-24 (the second +covers ranks {1,2,3,4,5,7,8,16,32,48}, which contain the T488 ranks). The improvement test asks that each +predecessor's complete profile (at its own T) is rejected at PR3633's C, i.e. the gain is a real moment gain and not +a rounding artefact. The BIT side is not re-derived: PR3605's recorded unchanged BIT endpoint is only compared. + +Rejections raise audit_common.AuditFail('endpoint.: ...') or the vendored 'kappa.*' rules. +Standard library only. +""" +from fractions import Fraction as Q + +from audit_common import need + +E24 = Q(1, 10 ** 24) +MARGINS = Q(1, 10 ** 46) + Q(1, 10 ** 70) +LIVE_PER_REPLICA = 896 + + +def dec24(q): + n = q * 10 ** 24 + need(n.denominator == 1 and n > 0, 'endpoint.format: not a positive 24-digit decimal') + n = int(n) + return '%d.%024d' % (n // 10 ** 24, n % 10 ** 24) + + +def intkeys(d): + return dict((int(k), v) for k, v in d.items()) + + +def moments(AK, CA, prof, a, T): + live = LIVE_PER_REPLICA * T + own = AK.moment(prof, a, live=live) + ven = CA.moment(prof, a, live=live) + need(max(own[0], ven[0]) <= min(own[1], ven[1]), 'endpoint.two_impl: enclosures disjoint at %s' % a) + return own, ven + + +def endpoint(AK, CA, prof, T, C): + """endpoint.accept / endpoint.reject in both implementations.""" + own_c, ven_c = moments(AK, CA, prof, C, T) + need(own_c[1] < 1 and ven_c[1] < 1, 'endpoint.accept: M(C) not below 1 in both enclosures') + own_n, ven_n = moments(AK, CA, prof, C + E24, T) + need(own_n[0] > 1 and ven_n[0] > 1, 'endpoint.reject: M(C + 1e-24) not above 1 in both enclosures') + return dict(C=dec24(C), M_C_minus_1=['%.4e' % float(own_c[0] - 1), '%.4e' % float(own_c[1] - 1)], + M_next_minus_1=['%.4e' % float(own_n[0] - 1), '%.4e' % float(own_n[1] - 1)], + vendored_agrees=True) + + +def run(AK, CA, recounted, claim, predecessors, bit_endpoint): + """recounted: profile from audit_owners; claim: PR3633 CLAIM.json; predecessors: {label: (T, profile, kappa)}.""" + prof = intkeys(recounted) + need(prof == intkeys(claim['profile']), 'endpoint.profile: recounted profile != CLAIM.json profile') + T = claim['replicas'] + lo, hi = (Q(x) for x in claim['root_interval']) + need(hi - lo == E24 and (lo * 10 ** 24).denominator == 1, 'endpoint.format: root_interval is not one 24-digit step') + C = lo + rec = dict(endpoint=endpoint(AK, CA, prof, T, C)) + kappa = Q(claim['kappa']) + need(kappa == C - MARGINS and Q(claim['candidate_endpoint']) == kappa, + 'endpoint.kappa: CLAIM kappa/candidate_endpoint != C - 1e-46 - 1e-70') + rec['kappa'] = str(kappa) + gains = {} + for label, (Tp, pp, kp) in sorted(predecessors.items()): + g = kappa - Q(kp) + need(g > 0, 'endpoint.gain: no gain over %s' % label) + own = AK.moment(intkeys(pp), C, live=LIVE_PER_REPLICA * Tp) + need(own[0] > 1, 'endpoint.improvement: %s profile is not rejected at PR3633 C' % label) + gains[label] = dict(replicas=Tp, kappa=str(Q(kp)), gain=str(g), gain_float='%.6e' % float(g), + predecessor_M_at_C_minus_1='%.3e' % float(own[0] - 1)) + rec['gains'] = gains + need(C < Q(bit_endpoint), 'endpoint.bit: complex endpoint is not below the recorded unchanged BIT endpoint') + rec['bit'] = dict(unchanged_bit_endpoint=bit_endpoint, source='PR3605 CLAIM.json (recorded, not re-derived)', + complex_binds=True) + return rec + + +def controls(AK, CA, AL, recounted, claim, predecessors, bit_endpoint): + out = [] + E = lambda name, rule, fn: AL.expect(out, name, rule, fn) + + def claim_with(**kw): + c = dict(claim) + c.update(kw) + return c + lo = Q(claim['root_interval'][0]) + up = lo + E24 + E('claimed_endpoint_one_step_high', 'endpoint.accept', + lambda: run(AK, CA, recounted, claim_with(root_interval=[str(up), str(up + E24)], + kappa=str(up - MARGINS), candidate_endpoint=str(up - MARGINS)), + predecessors, bit_endpoint)) + down = lo - E24 + E('claimed_endpoint_one_step_low', 'endpoint.reject', + lambda: run(AK, CA, recounted, claim_with(root_interval=[str(down), str(lo)], + kappa=str(down - MARGINS), candidate_endpoint=str(down - MARGINS)), + predecessors, bit_endpoint)) + E('kappa_without_second_margin', 'endpoint.kappa', + lambda: run(AK, CA, recounted, claim_with(kappa=str(lo - Q(1, 10 ** 46))), predecessors, bit_endpoint)) + E('kappa_with_doubled_second_margin', 'endpoint.kappa', + lambda: run(AK, CA, recounted, claim_with(kappa=str(lo - Q(1, 10 ** 46) - Q(2, 10 ** 70)), + candidate_endpoint=str(lo - Q(1, 10 ** 46) - Q(2, 10 ** 70))), + predecessors, bit_endpoint)) + bad = dict(intkeys(recounted)) + bad[48] -= 1 + E('recount_one_idle_48_short', 'endpoint.profile', + lambda: run(AK, CA, bad, claim, predecessors, bit_endpoint)) + # A predecessor whose profile were one unpaid rank-48 child better per call-normalised replica would still lose; + # the real control: a hypothetical predecessor at PR3633's own profile is NOT an improvement. + same = dict(predecessors, hypothetical=(claim['replicas'], claim['profile'], str(Q(claim['kappa']) - Q(1, 10 ** 80)))) + E('predecessor_equal_to_candidate', 'endpoint.improvement', + lambda: run(AK, CA, recounted, claim, same, bit_endpoint)) + T26, P26, _ = predecessors['PR3626'] + above = dict(predecessors, PR3626=(T26, P26, str(Q(claim['kappa']) + Q(1, 10 ** 30)))) + E('predecessor_kappa_above_candidate', 'endpoint.gain', + lambda: run(AK, CA, recounted, claim, above, bit_endpoint)) + class DisjointEngine(object): + @staticmethod + def moment(prof, a, m=73, live=1): + lo, hi = CA.moment(prof, a, m=m, live=live) + return lo + 1, hi + 1 + E('second_enclosure_disjoint', 'endpoint.two_impl', + lambda: run(AK, DisjointEngine, recounted, claim, predecessors, bit_endpoint)) + E('bit_endpoint_below_complex', 'endpoint.bit', + lambda: run(AK, CA, recounted, claim, predecessors, '0.001730000000000000000000')) + return out + + +# --------------------------------------------------------------------------------------------------------------- +# Downstream: PR #3653 (rohanarun, head 607a09ff) composes roger-sato's PR #3642 retimed supplier words with +# PR3633's literal T488 allocation. If its words keep the staged helper widths, PR3633's journal (audited in scope B) hosts +# them unchanged, the idle part of the invoice is unchanged, and the complete profile is PR3633's + 4392 (L' - L). +# The fixed-inventory LP of scope A depends on the local bill only through the constant 9L: its optimal basis and dual +# are unchanged, but dual feasibility must be re-certified at the retimed endpoint (retimed_dual_certificate). + +def endpoint_of(AK, prof, T, lo_guess, span=Q(1, 10 ** 5)): + """The 24-digit endpoint of prof by bisection on the 24-digit grid (own enclosure, live 896 T).""" + live = LIVE_PER_REPLICA * T + lo = (lo_guess * 10 ** 24).__floor__() + hi = ((lo_guess + span) * 10 ** 24).__floor__() + need(AK.moment(prof, Q(lo, 10 ** 24), live=live)[1] < 1 and AK.moment(prof, Q(hi, 10 ** 24), live=live)[0] > 1, + 'downstream.bracket: initial bracket does not straddle 1') + while hi - lo > 1: + mid = (lo + hi) // 2 + e = AK.moment(prof, Q(mid, 10 ** 24), live=live) + if e[1] < 1: + lo = mid + elif e[0] > 1: + hi = mid + else: + raise AssertionError('grid point straddles 1') + return Q(lo, 10 ** 24) + + +def retimed_dual_certificate(CEIL, basis, a0, a1): + """downstream.dual: PR3633's optimal basis (scope A) stays dual feasible uniformly on [a0, a1]. The column costs + (idle children) do not involve the local bill, so the same dual applies to the retimed supplier; this is checked + at the retimed endpoint, not inferred from the certificate at PR3633's exponent.""" + model = CEIL.MODEL + y = CEIL.symbolic_dual(model, basis) + tight = positive = 0 + for cap, q in CEIL.enumerate_patterns(model['widths'], model['caps']): + rc = dict((r, c) for r, c in CEIL.reduced_cost(model, cap, q, y).items() if c) + if not rc: + tight += 1 + continue + need(CEIL.lower_uniform(rc, a0, a1) > 0, + 'downstream.dual: reduced cost of (%d, %s) not certified positive on [%s, %s]' % (cap, list(q), a0, a1)) + positive += 1 + return dict(tight=tight, strictly_positive_uniform=positive, interval=[dec24(a0), dec24(a1)]) + + +def downstream_3653(AK, CA, AL, CW, words, staged_widths, staged_local, audited_profile, lp_per_replica, outer_claim, + CEIL=None, basis=None): + T, calls = 488, 4392 + locs = [] + for tag, pid in (('base', 'gauge44'), ('extra', 'extra')): + fr = CW.frame_recount(AL.parse_word(words[tag])) + need(list(fr['helper_widths']) == list(staged_widths[pid]), + 'downstream.widths: PR3653 %s word changes helper widths' % tag) + locs.append(dict((int(k), v) for k, v in fr['paid'].items())) + need(locs[0] == locs[1], 'downstream.local: the two retimed words have different local bills') + L1, L0 = locs[0], dict((int(k), v) for k, v in staged_local.items()) + prof = dict((int(k), v) for k, v in audited_profile.items()) + for r in set(L0) | set(L1): + prof[r] = prof.get(r, 0) + calls * (L1.get(r, 0) - L0.get(r, 0)) + prof = dict((r, n) for r, n in sorted(prof.items()) if n) + lp = dict((int(k), v) for k, v in lp_per_replica.items()) + lp1 = dict(lp) + for r in set(L0) | set(L1): + lp1[r] = lp1.get(r, 0) + 9 * (L1.get(r, 0) - L0.get(r, 0)) + need(dict((r, n) for r, n in prof.items()) == dict((r, v * T) for r, v in sorted(lp1.items()) if v), + 'downstream.ceiling: PR3653 profile is not 488 x the LP minimum with the retimed local bill') + C = endpoint_of(AK, prof, T, Q('0.00173')) + end = endpoint(AK, CA, prof, T, C) + cert = retimed_dual_certificate(CEIL, basis, C, C + E24) + kappa = Q(outer_claim['kappa_rational']) + need(kappa == C - MARGINS, 'downstream.kappa: PR3653 kappa %s != own endpoint %s - 1e-46 - 1e-70' + % (outer_claim['kappa'], C)) + return dict(retimed_local_bill=dict(sorted(L1.items())), delta_per_call=dict( + sorted((r, L1.get(r, 0) - L0.get(r, 0)) for r in set(L0) | set(L1) if L1.get(r, 0) != L0.get(r, 0))), + profile=prof, endpoint=end, kappa=str(kappa), widths_equal_staged=True, + profile_equals_488_retimed_lp_minimum=True, dual_certificate_at_retimed_endpoint=cert) + + +def downstream_controls(AK, CA, AL, CW, words, staged_widths, staged_local, audited_profile, lp_per_replica, + outer_claim, CEIL=None, basis=None): + out = [] + E = lambda name, rule, fn: AL.expect(out, name, rule, fn) + run = lambda **kw: downstream_3653(AK, CA, AL, CW, kw.get('words', words), kw.get('sw', staged_widths), + staged_local, kw.get('prof', audited_profile), kw.get('lp', lp_per_replica), + kw.get('oc', outer_claim), CEIL, kw.get('basis', basis)) + # A feasible but suboptimal basis: swap the empty capacity-48 pattern for a non-tight one; its dual is not + # feasible at the retimed endpoint. + alt = list(basis) + k = next(i for i, (cap, q) in enumerate(alt) if cap == 48 and not any(q)) + alt[k] = (48, (0, 0, 0, 0, 6, 0)) + E('downstream_dual_of_other_basis', 'downstream.dual', lambda: run(basis=alt)) + sw = dict((pid, list(ws)) for pid, ws in staged_widths.items()) + sw['gauge44'][389] = 8 + E('downstream_staged_width_389_reverted', 'downstream.widths', lambda: run(sw=sw)) + lp = dict(lp_per_replica) + lp[48] = lp[48] - Q(1, 4) + E('downstream_lp_minimum_one_quarter_idle48_short', 'downstream.ceiling', lambda: run(lp=lp)) + oc = dict(outer_claim) + oc['kappa_rational'] = str(Q(outer_claim['kappa_rational']) + E24) + E('downstream_kappa_one_step_high', 'downstream.kappa', lambda: run(oc=oc)) + return out diff --git a/research/t488-independent-audit/audit_inherited_controls.py b/research/t488-independent-audit/audit_inherited_controls.py new file mode 100644 index 0000000000..61f0c3b2f6 --- /dev/null +++ b/research/t488-independent-audit/audit_inherited_controls.py @@ -0,0 +1,938 @@ +# SPDX-License-Identifier: Apache-2.0 +"""Scope C2: the five reject(name, test) controls of PR3633's owners/independent_literal.py (kept from PR3605's +adapted owner audit) cannot detect a disabled or weakened gate; the same holds in every located copy. + +Static analysis only (ast); no code of PR3605, PR3624, PR3626, PR3630, PR3633, PR3638, PR3642 or PR3653 is imported +or executed. The file +defines `need(v, m)` (raise iff not v) and `reject(name, test)`, which runs `need(not test(), ...)`: a control +"passes" iff its lambda returns a falsy value. analyse(source, ctx) is the main check (positive run and every +negative control): + controls.pin sha256 of the source is the pinned value + controls.need_def need's body is exactly `if not v: raise ...` (no other effect) + controls.reject_def reject's body is exactly `need(not test(), msg); controls.append(name)` + controls.bindings need and reject are bound exactly once in the module, by those defs (no later rebinding) + controls.gate_purity no gate (a need(...) call outside a function) binds or mutates anything by name (lambda + bodies included): no assignment expression, mutating method, x.at(...), out=, or in-place + numpy function (copyto, put, place, putmask, ...); a module function called by a gate declares + no global/nonlocal and mutates only its own local names (never a parameter or module name) + controls.inventory the reject() calls are exactly the five inherited names, each with a lambda + controls.tautology every lambda (i) calls no module function and no need/reject, only pure builtins/numpy + constructors (so its value is the same whether or not any gate exists), and (ii) matches a + proof pattern showing it is False on every input that reaches it: + irreflexive `a < a ...` (or >), identical side-effect-free operands on numbers: no value + duplicate_set `len({int(e), int(e)}) == 2`: a one-element set, for every input + shifted_column `array_equal(r_[X[0,k], X[:u,k]], tile(arange(M), N))`, M >= 2, X a matrix, + no step: entries 0 and 1 of the first array are both X[0,k], of the second 0 + and 1 (or the lengths differ); for every X, u and N + gate_offset `int(A[I[0]]) + d == int(B[0])`, d != 0, after an unconditional earlier gate + `need(array_equal(A[I], B))`, I last bound by np.lexsort/np.argsort (an integer + index array, so A[I][0] == A[I[0]]) + gate_bump `dict(H) == bad` with `bad = dict(P); bad[k] += d` (d != 0) after an + unconditional earlier gate asserting `dict(H) == P` (a conjunct of an `and`) + For the two gate patterns, every statement strictly between the premise gate and the control, + other than reject() calls and the `bad` pair, references none of the premise names, calls no + module function except need, and mutates only objects built fresh inside the region (region + rule). + Verdicts when no proof is accepted (all raise controls.tautology): 'genuine' = the lambda calls + a gate's module function on arguments that differ syntactically from every gate call of it + (it feeds changed data to the gate predicate); 'gate-coupled' = it calls need/reject or a + module function, but only as the gates do (not certified either way); 'unproven' = no pattern. + controls.pattern each control is proved by the pattern expected for its name + controls.data_* (PR3633 file, when journal rows are given) the premises the proofs rely on, evaluated in pure + Python on the actual T488 values: data_pin (rows bound to PR3633 EXPECTED.json payload_sha256 + 20662cc6... by compact re-serialisation; ORDER/DISPATCH rebuilt from the pinned base + chronologies and bound to EXPECTED literal_files; PROVIDERS.json and the profile from PR3633's + head; TEMPLATE/PLAN from main), data_rows, data_identity (rows are the ordered helper + identities), data_window (row 0's time is inside its own source window), data_prefix (the + first bin's offsets are prefix sums of the provider widths), data_mass (the profile mass is + 65088 x 488). The predicate values are reported but carry no data content: three are + data-independent by their pattern, and the hole and rank1 values are reduced forms fixed by + the premises (H itself is not recomputed here; that is the literal-owner audit's scope). +Fairness: the gates themselves are real checks evaluated on the real data; this is a control-coverage defect (the five +controls never feed changed data to any gate, so they would also pass with that gate deleted), not evidence that the +owner journal is wrong. The copies of the same controls in PR3605/PR3624/PR3626/PR3630/PR3633 are located (every +.py blob of main and the eight PR heads containing the control text), pinned and analysed structurally with the same +function; the cited original (SHA256 1001cb1c...) is searched among all those .py blobs by sha256. +Not mutation-controlled (fail-closed reads that must pass on the pinned bytes): controls.git, the input pins +(show_pinned, ORDER/DISPATCH/PROVIDERS/FILL file pins in build_t488/read_local_fill) and the located-copies set +(controls.copies). The journal-payload pin and every analyser rule have negative controls. +Model and limits: names are tracked syntactically; mutation is recognised through stores, augmented assignments, +the MUTATING method names, x.at(...), out= and INPLACE_NP. shortcut: a library call that mutates an argument outside +those lists (np.random.shuffle, setattr, ...), or mutation of a mutable element shared into a fresh container, is +not recognised; neither occurs in the located files (by inspection: their regions mutate only used, idle, H, controls +and bad), upgrade to a callee whitelist if this analyser is applied to other files. Standard library only. +""" +import ast +import gzip +import json +import os +import re +import subprocess +import sys +import time + +_HERE = os.path.dirname(os.path.abspath(__file__)) +_VENDOR = os.path.join(_HERE, 'vendor', 'pr3053') +if _VENDOR not in sys.path: + sys.path.insert(0, _VENDOR) +import audit_common as AC # noqa: E402 +import audit_lib as AL # noqa: E402 + +REPO = os.path.dirname(os.path.dirname(_HERE)) +SCRATCH_OWNERS = os.path.join(REPO, '.loop-scratch', 'iter22', 't488-owners') +MAIN = '20ecf2891d80b5acf18d980b2164fb44ea7faa90' +COMMITS = dict(PR3605='3efbb5de06a49ffeea6684482f2d3fe48cdca3fb', PR3624='515a7c4490aa32ac3ca5014feb6f7da0d6952863', + PR3626='659fd5e7064d9466b668d7188cbb013b333a415a', PR3630='bcac7fe04b9b7f7f5d3c27d620561f96c4ae5608', + PR3633='72c268f113368ea0387d188225e3f81ba6d6ef73', PR3638='d62e1acd7518a231440383f0d0dddf54d6edf8c1', + PR3642='044d24f12e74f228dd462c07ae2d7f3ea35e2be9', PR3653='607a09ff441c79b215a8127a81f4cc70021d0d9b') +ORIGINAL_SHA256 = '1001cb1c8b4ee1f93c23b58b094565ec1b3931eee0ea091a23504696e380be9e' +P605 = ('5d1a7b4fd635070522e78656ccedf9bcf4bf13db', 'da897bd5cef36943b3f5b1422fe81716b4eaa6230b056d31f8c2310cf62f8580') +# label -> (pr, path, git blob, sha256) +TARGET = ('PR3633', 'research/staged-width-t488/owners/independent_literal.py', + 'e2a226950ed30e88f6c386669716b9a463dd873d', '8517c0d6c7cbe73438885135ac7fd9781dd5120113c9721002346dca603f5ed9') +COPIES = { + 'PR3605': ('PR3605', 'research/staged-644-660-conditional-release/owners/independent_literal.py') + P605, + 'PR3624': ('PR3624', 'research/staged-width-t450/owners/independent_literal.py', + 'd9c2450e03d9dcde39ae96f0edc835d1a67d9fe5', '68684584d3866b2ab76c254dd6d5be3a49d276a6e5f6353183dee4d8d84de4d4'), + 'PR3624-provenance': ('PR3624', 'research/staged-width-t450/provenance/pr3605/owners/independent_literal.py') + P605, + 'PR3626': ('PR3626', 'research/staged-width-t530/owners/independent_literal.py', + '32755776c15660b939c92cb8835c71ec87c8c1d6', '36d7f4e3a611fd6b5983222887beb9a37a18bfac0d999ac5c00e3e191d1c5cc4'), + 'PR3626-provenance': ('PR3626', 'research/staged-width-t530/provenance/pr3605/owners/independent_literal.py') + P605, + 'PR3630': ('PR3630', 'research/staged-644-660-owner-successor/independent_owners.py', + 'f116cc0142b4f6d674abf6e4c844142c00fb0297', '8d02dff267938cdb373aa617a411d8e67700e130609d2714e98ccf814e32eaef'), + 'PR3633': TARGET, + 'PR3633-provenance': ('PR3633', 'research/staged-width-t488/provenance/pr3605/owners/independent_literal.py') + P605, + 'PR3638': ('PR3638', 'research/three-kappa-leads/independent_literal.py', + '4028493716bfcf3e73831067599518c055fcd2b0', '89c27f61ae3e6d2bdd4c469ac5b99d4f955a443c5e04c74e568c12093510efbe'), + 'PR3642': ('PR3642', 'research/staged-release-retiming-t530/owner-audit/independent_literal.py', + '86dc50768ff3fda8e265c08aa9314fe072e0c396', 'b51c36ad1ae0999547edc4726d698df369fef60daaaac4f5bbea471648c9f628'), + 'PR3642-upstream': ('PR3642', 'research/staged-release-retiming-t530/owner-audit/upstream/independent_literal.py', + '32755776c15660b939c92cb8835c71ec87c8c1d6', + '36d7f4e3a611fd6b5983222887beb9a37a18bfac0d999ac5c00e3e191d1c5cc4'), + 'PR3642-vendor': ('PR3642', 'research/staged-release-retiming-t530/full-admission/vendor/staged-width-t530/owners/' + 'independent_literal.py', '32755776c15660b939c92cb8835c71ec87c8c1d6', + '36d7f4e3a611fd6b5983222887beb9a37a18bfac0d999ac5c00e3e191d1c5cc4'), + 'PR3653': ('PR3653', 'research/retimed-t488-composition/owners/independent_literal.py', + '403a9cd104c7e68a5bd2431bb442d701f30828f5', 'caa5eb4fff4908c0891c3ee41e7e6d71bf19bb710d1d7781164f10ce84bce769'), + 'PR3653-upstream': ('PR3653', 'research/retimed-t488-composition/owners/upstream/independent_literal.py') + TARGET[2:], + 'PR3653-vendor': ('PR3653', 'research/retimed-t488-composition/full-replay/vendor/pr3642/vendor/staged-width-t530/' + 'owners/independent_literal.py', '32755776c15660b939c92cb8835c71ec87c8c1d6', + '36d7f4e3a611fd6b5983222887beb9a37a18bfac0d999ac5c00e3e191d1c5cc4'), +} +EXPECTED_JSON = ('PR3633', 'research/staged-width-t488/owners/EXPECTED.json', '415eac1a2b0201beac2c3e531c6fcbd1133a819f', + '924493b6bbeade055af4916ac7bd6d7d7ec0600719f5c39636dc3f16d7ec14ea') +TEMPLATE = (MAIN, 'research/rank7-joint-endframes/allocation/vendor/TEMPLATE.json.gz', + '114808c32b1799da53524bdc25cea67efb60009e', '21297b20f1f2dc76eb9dd8124416068c3cb096bab9df17fa4da19bc68dfe082e') +PLAN = (MAIN, 'research/rank7-joint-endframes/allocation/vendor/PLAN.json.gz', + '7a1db55a4d53ee99280be4f3ab7d29f7b2033adf', 'd566cfb87b71639d4138e1f89b543e2dd6a7772070226a2a14c799a6835bf21e') +NAMES = ('duplicate helper', 'strict lower endpoint used as available', 'same-call family reused', + 'unpaid chronological hole', 'one unpaid rank1 child') +PATTERN_OF = dict(zip(NAMES, ('shifted_column', 'irreflexive', 'duplicate_set', 'gate_offset', 'gate_bump'))) +PURE_CALLS = {'np.array_equal', 'np.tile', 'np.arange', 'bool', 'len', 'int', 'dict'} +FRESH = {'np.zeros', 'np.ones', 'np.empty', 'np.full', 'np.repeat', 'Counter', 'dict', 'list', 'set'} +INPLACE_NP = {'np.copyto', 'np.put', 'np.place', 'np.putmask', 'np.fill_diagonal', 'np.put_along_axis'} +CONTROL_TEXT = b'strict lower endpoint used as available' +MUTATING = {'append', 'extend', 'insert', 'pop', 'remove', 'clear', 'sort', 'reverse', 'update', 'setdefault', + 'popitem', 'add', 'discard', 'fill', 'resize', 'put', 'itemset', 'partition', 'write_text', 'write_bytes'} +COLUMNS = ['time', 'helper', 'owner_replica', 'child', 'port', 'offset'] +# PR3633 owners/ base chronology files (name, git blob, sha256) +BASE_ORDERS = (('ORDER37.json', '411004ab280d6468bdd9c98c01f153baed66c543', + '94803d190b2b0506d9645e980b57cdbc3e98221f3b615292df8a94f1b174c984'), + ('T8-ORDER.json', 'b013bdb989f7fde1b8a22ae8ed743eae3a0a62fd', + 'a1abf78fb3b88bbf4656e6ae26f148a007f744f0be4c00d480fb2424ce8766d6'), + ('T16-ORDER.json', '470b8afb8af03fd053b6ff19bc90eb410b5c9e6b', + '9b2106c84aed23a7c3c8b99e21d3e96c66b12f92980a42ab2f6dbbe7831ef7c4')) +BASE_DISPATCH = (('DISPATCH37.json', 'c4c016ea3f3cb7bd04c1812259246a9411ad7adc', + '4f78935b2ef1f6c08c9c16433d54015def1eccd767443b496a77ea8823690df6'), + ('T8-DISPATCH.json', 'f95b6f07bf4d9e607a7ab8fcc101cdc63a725b07', + '23877abbe8ec258eb1f2425137f9171d37b1a4414a25595abe75eb0738850a24'), + ('T16-DISPATCH.json', '40725d2abb328ba7fee1c4f74656eecd982f2f7e', + '282960b7163bc305675b64d1d0faee9f71b18848748326418a013626944c6488')) +PROVIDERS_PIN = ('f321d4185339244b8b004326af70e653e611d84a', + 'd96f2798024ab83edbb8848b62c425992aec182f1f24ef46bb8286dd390618c2') + + +# ------------------------------------------------------------------------------------------------- inputs --- +def git(*args, **kw): + p = subprocess.run(['git', '-C', REPO] + list(args), stdout=subprocess.PIPE, stderr=subprocess.PIPE, + env=AC.git_env(), **kw) + AC.need(p.returncode == 0, 'controls.git: git %s failed: %s' % (' '.join(args)[:120], p.stderr.decode()[:200])) + return p.stdout + + +def show_pinned(commit, path, blob, sha): + data = git('show', '%s:%s' % (commit, path)) + AC.need(AC.git_blob_id(data) == blob and AC.sha256(data) == sha, 'controls.pin: %s:%s is not blob %s / sha256 %s' + % (commit[:12], path, blob[:12], sha[:12])) + return data + + +def spec_bytes(spec): + pr, path, blob, sha = spec + return show_pinned(COMMITS[pr], path, blob, sha) + + +def scan_trees(): + """Every .py blob in the trees of main and the eight PR heads (git ls-tree -r, read back with git cat-file --batch and + re-hashed): the ones containing the control text, with blob id and sha256, must equal the pinned COPIES table; + also returns the blobs whose sha256 is the cited original's and the number of .py blobs scanned.""" + blobs = {} + for label, commit in [('main', MAIN)] + sorted(COMMITS.items()): + for ent in git('ls-tree', '-r', '-z', commit).split(b'\0'): + if ent: + meta, path = ent.split(b'\t', 1) + _, typ, oid = meta.split() + if typ == b'blob' and path.endswith(b'.py'): + blobs.setdefault(oid.decode(), []).append((label, commit, path.decode('utf-8'))) + data = git('cat-file', '--batch', input=('\n'.join(blobs) + '\n').encode()) + found, original, i = {}, [], 0 + for oid, where in blobs.items(): + nl = data.index(b'\n', i) + hdr = data[i:nl].split() + n = int(hdr[2]) + body = data[nl + 1:nl + 1 + n] + i = nl + 2 + n + AC.need(hdr[0].decode() == oid and AC.git_blob_id(body) == oid, 'controls.git: cat-file returned %r for %s' + % (hdr[:2], oid)) + sha = AC.sha256(body) + if sha == ORIGINAL_SHA256: + original += ['%s:%s' % (label, path) for label, _, path in where] + if CONTROL_TEXT in body: + for label, commit, path in where: + found['%s:%s' % (label, path)] = dict(commit=commit, git_blob=oid, sha256=sha) + AC.need(i == len(data), 'controls.git: trailing cat-file output') + want = dict(('%s:%s' % (pr, path), (blob, sha)) for pr, path, blob, sha in COPIES.values()) + got = dict((k, (v['git_blob'], v['sha256'])) for k, v in found.items()) + AC.need(got == want, 'controls.copies: located copies %r != pinned %r' % (sorted(got), sorted(want))) + return found, original, len(blobs) + + +# ------------------------------------------------------------------------------------------- ast helpers --- +def dump(node): + return ast.dump(node, annotate_fields=False) + + +def callee(call): + return dotted(call.func) + + +def dotted(f): + parts = [] + while isinstance(f, ast.Attribute): + parts.append(f.attr) + f = f.value + if isinstance(f, ast.Name): + return '.'.join([f.id] + parts[::-1]) + return None + + +def is_call(node, name, nargs=None): + return isinstance(node, ast.Call) and callee(node) == name and not node.keywords and \ + (nargs is None or len(node.args) == nargs) + + +def is_const(node, pred=lambda v: True): + return isinstance(node, ast.Constant) and not isinstance(node.value, bool) and pred(node.value) + + +def int_value(node): + """The int of a literal like 5 or -1 (parsed as USub(5)), else None.""" + if isinstance(node, ast.UnaryOp) and isinstance(node.op, ast.USub) and is_const(node.operand, lambda v: isinstance(v, int)): + return -node.operand.value + return node.value if is_const(node, lambda v: isinstance(v, int)) else None + + +def side_effect_free(node): + """Names, constants, subscripts, slices, tuples and arithmetic/unary operators only (no call, no walrus).""" + ok = (ast.Name, ast.Constant, ast.Subscript, ast.Slice, ast.Tuple, ast.Load, ast.UnaryOp, ast.USub, ast.BinOp, + ast.Add, ast.Sub, ast.Mult) + return all(isinstance(n, ok) for n in ast.walk(node)) + + +def names_in(node): + return set(n.id for n in ast.walk(node) if isinstance(n, ast.Name)) + + +def bound_names(node): + """Names (re)bound, or whose object is mutated, by a statement or expression (names are syntactic, see the module + docstring): targets of assignments, augmented assignments (in place for numpy), for loops, walrus, with, del + (a Subscript/Attribute target counts its base names), receivers of MUTATING methods, the first argument of x.at(...) + and of the in-place numpy functions INPLACE_NP, and out= arguments.""" + out = set() + for n in ast.walk(node): + targets = [] + if isinstance(n, (ast.Assign, ast.Delete)): + targets = n.targets + elif isinstance(n, (ast.AugAssign, ast.AnnAssign, ast.For, ast.NamedExpr)): + targets = [n.target] + elif isinstance(n, ast.withitem) and n.optional_vars is not None: + targets = [n.optional_vars] + for t in targets: + out |= names_in(t) + if isinstance(n, ast.Call): + if isinstance(n.func, ast.Attribute) and n.func.attr in MUTATING: + out |= names_in(n.func.value) + if n.args and ((isinstance(n.func, ast.Attribute) and n.func.attr == 'at') or callee(n) in INPLACE_NP): + out |= names_in(n.args[0]) + for kw in n.keywords: + if kw.arg == 'out': + out |= names_in(kw.value) + return out + + +def binding_sites(tree, name): + """Every node of the module that binds `name`: def/class, a Name store/del, an import alias, global/nonlocal.""" + out = [] + for n in ast.walk(tree): + if isinstance(n, (ast.FunctionDef, ast.AsyncFunctionDef, ast.ClassDef)) and n.name == name: + out.append(n) + elif isinstance(n, ast.Name) and n.id == name and not isinstance(n.ctx, ast.Load): + out.append(n) + elif isinstance(n, ast.alias) and (n.asname or n.name.split('.')[0]) == name: + out.append(n) + elif isinstance(n, (ast.Global, ast.Nonlocal)) and name in n.names: + out.append(n) + return out + + +def need_calls(stmt): + """need(...) calls in a module statement (functions and lambdas excluded).""" + out = [] + + def walk(n): + if isinstance(n, (ast.FunctionDef, ast.Lambda)): + return + if is_call(n, 'need'): + out.append(n) + for ch in ast.iter_child_nodes(n): + walk(ch) + walk(stmt) + return out + + +# --------------------------------------------------------------------------------------- the main check --- +def check_defs(tree): + funcs = dict((n.name, n) for n in tree.body if isinstance(n, ast.FunctionDef)) + nd = funcs.get('need') + AC.need(nd is not None and [a.arg for a in nd.args.args] == ['v', 'm'] and len(nd.body) == 1 + and isinstance(nd.body[0], ast.If) and not nd.body[0].orelse and len(nd.body[0].body) == 1 + and isinstance(nd.body[0].body[0], ast.Raise) + and dump(nd.body[0].test) == dump(ast.UnaryOp(ast.Not(), ast.Name('v', ast.Load()))), + 'controls.need_def: need is not exactly "if not v: raise"') + rj = funcs.get('reject') + ok = rj is not None and [a.arg for a in rj.args.args] == ['name', 'test'] and len(rj.body) == 2 + if ok: + s1, s2 = rj.body + ok = isinstance(s1, ast.Expr) and is_call(s1.value, 'need', 2) and \ + dump(s1.value.args[0]) == dump(ast.UnaryOp(ast.Not(), ast.Call(ast.Name('test', ast.Load()), [], []))) and \ + isinstance(s2, ast.Expr) and is_call(s2.value, 'controls.append', 1) and \ + dump(s2.value.args[0]) == dump(ast.Name('name', ast.Load())) + AC.need(ok, 'controls.reject_def: reject is not exactly "need(not test(), msg); controls.append(name)"') + for f in ('need', 'reject'): + sites = binding_sites(tree, f) + AC.need(sites == [funcs[f]], 'controls.bindings: %s is bound %d times (lines %r), not only by its def' + % (f, len(sites), [getattr(x, 'lineno', None) for x in sites])) + return funcs + + +def gates_of(tree): + """[(statement index, unconditional?, need call, message text)] for every module-level gate.""" + out = [] + for i, st in enumerate(tree.body): + for call in need_calls(st): + msg = call.args[1] if len(call.args) > 1 else None + text = msg.value if is_const(msg) else (ast.unparse(msg) if msg is not None else '') + out.append((i, isinstance(st, ast.Expr) and st.value is call, call, text)) + return out + + +def gate_purity(gates, funcs): + called = set() + for i, _, call, text in gates: + called |= set(callee(n) for n in ast.walk(call) if isinstance(n, ast.Call)) & set(funcs) + AC.need(not bound_names(call), 'controls.gate_purity: gate %r binds or mutates %r' % (text, sorted(bound_names(call)))) + for f in sorted(called - {'need'}): + fn = funcs[f] + params = set(a.arg for a in fn.args.args) + local = set(n.id for n in ast.walk(fn) if isinstance(n, ast.Name) and isinstance(n.ctx, ast.Store)) - params + AC.need(not any(isinstance(n, (ast.Global, ast.Nonlocal)) for n in ast.walk(fn)) + and bound_names(fn) <= local, + 'controls.gate_purity: gate function %s declares global/nonlocal or mutates a parameter or a module-level ' + 'name (%r)' + % (f, sorted(bound_names(fn) - local))) + + +def find_controls(tree): + out = [] + for i, st in enumerate(tree.body): + if isinstance(st, ast.Expr) and is_call(st.value, 'reject'): + c = st.value + AC.need(len(c.args) == 2 and is_const(c.args[0], lambda v: isinstance(v, str)) + and isinstance(c.args[1], ast.Lambda) and not c.args[1].args.args, + 'controls.inventory: reject call at statement %d is not reject("name", lambda: ...)' % i) + out.append((i, c.args[0].value, c.args[1])) + else: + AC.need(not any(is_call(n, 'reject') for n in ast.walk(st)), 'controls.inventory: nested reject call') + AC.need(tuple(n for _, n, _ in out) == NAMES, 'controls.inventory: %r != %r' % ([n for _, n, _ in out], NAMES)) + return out + + +class Unproven(Exception): + pass + + +def region_clear(tree, funcs, lo, hi, names, allowed=()): + """Region rule: every statement strictly between lo and hi, other than reject() calls and `allowed`, references + none of `names`, calls no module function except need, and mutates (bound_names other than a plain `name = ...`) + only names bound earlier in the region to a fresh object (a FRESH constructor call or a list/dict/set display), so + a view or alias of a premise object bound before the gate cannot be mutated by name.""" + fresh = set() + for j in range(lo + 1, hi): + st = tree.body[j] + if j in allowed or (isinstance(st, ast.Expr) and is_call(st.value, 'reject')): + continue + hit = names_in(st) & set(names) + if hit: + raise Unproven('statement %d between the premise gate and the control references %r' % (j, sorted(hit))) + calls = set(callee(n) for n in ast.walk(st) if isinstance(n, ast.Call)) & (set(funcs) - {'need'}) + if calls: + raise Unproven('statement %d between the premise gate and the control calls module function(s) %r' + % (j, sorted(calls))) + plain = isinstance(st, ast.Assign) and all(isinstance(t, ast.Name) for t in st.targets) + mutated = bound_names(st) - (set(t.id for t in st.targets) if plain else set()) + if mutated - fresh: + raise Unproven('statement %d between the premise gate and the control mutates %r, not built fresh in the ' + 'region' % (j, sorted(mutated - fresh))) + if plain: + new = isinstance(st.value, (ast.List, ast.Dict, ast.Set)) or (isinstance(st.value, ast.Call) + and callee(st.value) in FRESH) + for t in st.targets: + (fresh.add if new else fresh.discard)(t.id) + + +def premise_gate(tree, gates, at, test): + """The latest unconditional gate before statement `at` whose test satisfies test(expr).""" + for i, uncond, call, text in reversed(gates): + if i < at and uncond and test(call.args[0]): + return i, text + raise Unproven('no unconditional earlier gate supplies the premise') + + +def prove(tree, funcs, gates, at, body): + """(pattern, premise gate message or None, data-independent?, argument) or raise Unproven.""" + e = body.args[0] if is_call(body, 'bool', 1) else body + # irreflexive + if isinstance(e, ast.Compare) and isinstance(e.ops[0], (ast.Lt, ast.Gt)) and dump(e.left) == dump(e.comparators[0]) \ + and side_effect_free(e.left): + return ('irreflexive', None, True, 'first conjunct is %s %s itself: false for every value' + % (ast.unparse(e.left), '<' if isinstance(e.ops[0], ast.Lt) else '>')) + if isinstance(e, ast.Compare) and len(e.ops) == 1 and isinstance(e.ops[0], ast.Eq): + L, R = e.left, e.comparators[0] + # duplicate_set + if is_call(L, 'len', 1) and isinstance(L.args[0], ast.Set) and len(L.args[0].elts) == 2 \ + and is_const(R, lambda v: v == 2): + a, b = L.args[0].elts + if dump(a) == dump(b) and is_call(a, 'int', 1) and side_effect_free(a.args[0]): + return ('duplicate_set', None, True, '{%s, same} has one element for every input, so len != 2' + % ast.unparse(a)) + if is_call(e, 'np.array_equal', 2): + A, B = e.args + if isinstance(A, ast.Subscript) and dotted(A.value) == 'np.r_' \ + and isinstance(A.slice, ast.Tuple) and len(A.slice.elts) == 2 and is_call(B, 'np.tile', 2) \ + and is_call(B.args[0], 'np.arange', 1) and is_const(B.args[0].args[0], lambda v: isinstance(v, int) and v >= 2): + first, rest = A.slice.elts + # the slice's upper bound is free: X[:u,k] starts with X[0,k] for every u (or is empty, lengths differ) + if isinstance(first, ast.Subscript) and isinstance(rest, ast.Subscript) and dump(first.value) == dump(rest.value) \ + and isinstance(first.slice, ast.Tuple) and isinstance(rest.slice, ast.Tuple) \ + and len(first.slice.elts) == len(rest.slice.elts) == 2 and is_const(first.slice.elts[0], lambda v: v == 0) \ + and isinstance(rest.slice.elts[0], ast.Slice) and rest.slice.elts[0].lower is None \ + and rest.slice.elts[0].step is None and dump(first.slice.elts[1]) == dump(rest.slice.elts[1]) \ + and side_effect_free(A.slice): + return ('shifted_column', None, True, 'entries 0 and 1 of the first array are both %s, of tile(arange(%d))' + ' are 0 and 1: never equal' % (ast.unparse(first), B.args[0].args[0].value)) + if isinstance(e, ast.Compare) and len(e.ops) == 1 and isinstance(e.ops[0], ast.Eq): + L, R = e.left, e.comparators[0] + # gate_offset: int(A[I[0]]) + d == int(B[0]) + if isinstance(L, ast.BinOp) and isinstance(L.op, ast.Add) and is_const(L.right, lambda v: isinstance(v, int) and v)\ + and is_call(L.left, 'int', 1) and is_call(R, 'int', 1): + x, y = L.left.args[0], R.args[0] + if isinstance(x, ast.Subscript) and isinstance(x.slice, ast.Subscript) and is_const(x.slice.slice, lambda v: v == 0) \ + and isinstance(y, ast.Subscript) and is_const(y.slice, lambda v: v == 0) \ + and side_effect_free(ast.Tuple([x, y], ast.Load())): + Ad, Id, Bd = dump(x.value), dump(x.slice.value), dump(y.value) + + def gate_test(t): + return is_call(t, 'np.array_equal', 2) and isinstance(t.args[0], ast.Subscript) \ + and dump(t.args[0].value) == Ad and dump(t.args[0].slice) == Id and dump(t.args[1]) == Bd + gi, text = premise_gate(tree, gates, at, gate_test) + I = x.slice.value + last = [st for st in tree.body[:gi] if isinstance(I, ast.Name) and I.id in bound_names(st)][-1:] + if not (last and isinstance(last[0], ast.Assign) and len(last[0].targets) == 1 + and dump(last[0].targets[0]) == dump(ast.Name(I.id, ast.Store())) + and isinstance(last[0].value, ast.Call) and callee(last[0].value) in ('np.lexsort', 'np.argsort')): + raise Unproven('index %s is not last bound by np.lexsort/np.argsort before the gate' % ast.unparse(I)) + region_clear(tree, funcs, gi, at, names_in(x) | names_in(y)) + return ('gate_offset', text, False, 'gate gives %s == %s, so the predicate is v + %d == v' + % (ast.unparse(x), ast.unparse(y), L.right.value)) + # gate_bump: dict(H) == bad, bad = dict(P); bad[k] += d + if is_call(L, 'dict', 1) and isinstance(L.args[0], ast.Name) and isinstance(R, ast.Name): + H, bad = L.args[0].id, R.id + stmts = tree.body[:at] + asg = [j for j, s in enumerate(stmts) if bad in bound_names(s)] + if len(asg) >= 2: # a third binding of bad after asg[0] is rejected by the region rule below + s1, s2 = stmts[asg[0]], stmts[asg[1]] + if isinstance(s1, ast.Assign) and len(s1.targets) == 1 and dump(s1.targets[0]) == dump(ast.Name(bad, ast.Store())) \ + and is_call(s1.value, 'dict', 1) and isinstance(s1.value.args[0], ast.Name) \ + and isinstance(s2, ast.AugAssign) and isinstance(s2.op, ast.Add) \ + and isinstance(s2.target, ast.Subscript) and dump(s2.target.value) == dump(ast.Name(bad, ast.Load())) \ + and is_const(s2.target.slice) and is_const(s2.value, lambda v: isinstance(v, int) and v): + P = s1.value.args[0].id + want = dump(ast.Compare(ast.Call(ast.Name('dict', ast.Load()), [ast.Name(H, ast.Load())], []), + [ast.Eq()], [ast.Name(P, ast.Load())])) + + def gate_test(t): + parts = t.values if isinstance(t, ast.BoolOp) and isinstance(t.op, ast.And) else [t] + return any(dump(p) == want for p in parts) + gi, text = premise_gate(tree, gates, asg[0], gate_test) + region_clear(tree, funcs, gi, at, {H, P}, allowed=set(asg)) + region_clear(tree, funcs, asg[0], at, {bad}, allowed={asg[1]}) + return ('gate_bump', text, False, 'gate gives dict(%s) == %s and %s differs from %s at key %r' + % (H, P, bad, P, s2.target.slice.value)) + raise Unproven('no proof pattern matches') + + +def args_key(call): + return dump(ast.Tuple(list(call.args), ast.Load())) + repr([(k.arg, dump(k.value)) for k in call.keywords]) + + +def classify(tree, funcs, gates, at, name, lam): + gate_args = {} # module function (need excluded) -> argument dumps of its calls inside gates + for _, _, call, _ in gates: + for n in ast.walk(call): + if isinstance(n, ast.Call) and callee(n) in set(funcs) - {'need'}: + gate_args.setdefault(callee(n), set()).add(args_key(n)) + lam_calls = [n for n in ast.walk(lam.body) if isinstance(n, ast.Call)] + calls = sorted(set(callee(n) for n in lam_calls)) + + def fail(verdict, why): + e = AC.AuditFail('controls.tautology: control %r is %s: %s' % (name, verdict, why)) + e.verdict = verdict + raise e + changed = sorted(set(callee(n) for n in lam_calls if callee(n) in gate_args and args_key(n) not in gate_args[callee(n)])) + if changed: + fail('genuine', 'calls gate function(s) %r on arguments that differ from every gate call' % changed) + coupled = [f for f in calls if f in funcs or f in ('need', 'reject')] + if coupled: + fail('gate-coupled', 'calls module function(s) %r, only on the gates\' own arguments' % coupled) + other = [f for f in calls if f not in PURE_CALLS] + if other: + fail('unproven', 'calls %r outside the pure builtin/numpy constructors' % other) + for n in ast.walk(lam.body): + if isinstance(n, ast.NamedExpr): + fail('unproven', 'assignment expression in the lambda') + try: + pattern, premise, independent, argument = prove(tree, funcs, gates, at, lam.body) + except Unproven as u: + fail('unproven', str(u)) + return dict(name=name, source=None, pattern=pattern, premise_gate=premise, data_independent=independent, + argument=argument, calls=calls, free_names=sorted(names_in(lam.body) - {'np'}), + calls_a_gate=False, verdict='tautological') + + +def analyse(source, ctx): + """The main check: pin, defs, gate purity, inventory, one tautology proof per control, optional T488 data.""" + AC.need(AC.sha256(source) == ctx['sha256'], 'controls.pin: source sha256 %s' % AC.sha256(source)) + text = source.decode('utf-8') + tree = ast.parse(text) + funcs = check_defs(tree) + gates = gates_of(tree) + gate_purity(gates, funcs) + ctrls = find_controls(tree) + first = ctrls[0][0] + res = [] + for at, name, lam in ctrls: + r = classify(tree, funcs, gates, at, name, lam) + AC.need(r['pattern'] == PATTERN_OF[name], 'controls.pattern: %r proved by %s, expected %s' + % (name, r['pattern'], PATTERN_OF[name])) + r['source'] = ast.get_source_segment(text, lam) + res.append(r) + pre = [g for g in gates if g[0] < first] + out = dict(sha256=AC.sha256(source), controls=res, gates_before_controls=len(pre), + unconditional_gates_before_controls=sum(1 for g in pre if g[1]), + gate_messages=sorted(set(g[3] for g in gates)), + original_sha256_claimed=(re.findall(r'Original SHA256 ([0-9a-f]{64})', text) or [None])[0]) + if ctx.get('data') is not None: + out['t488'] = evaluate(res, ctx['data']) + return out + + +# ------------------------------------------------------------------------------------ actual T488 values --- +def owners_file(name, blob, sha): + return show_pinned(COMMITS['PR3633'], 'research/staged-width-t488/owners/' + name, blob, sha) + + +def compact_sha(obj): + """sha256 of the compact JSON text plus newline (the byte format of PR3633's literal files).""" + return AC.sha256(json.dumps(obj, separators=(',', ':')).encode() + b'\n') + + +def payload_sha(rows): + return compact_sha(dict(columns=COLUMNS, dispatch_overrides=[], rows=rows)) + + +def build_t488(rows): + """T488 data for the evaluation, from git-pinned inputs only, plus the journal rows. ORDER/DISPATCH are rebuilt + as 8 copies of the 61 base replicas (ORDER37 + T8 at +37 + T16 at +45, copy cp at +61*cp) and must hash to + PR3633's EXPECTED.json literal_files pins; the rows are bound to its payload_sha256 inside evaluate().""" + exp = json.loads(spec_bytes(EXPECTED_JSON)) + base, bdisp = [], {} + for (oname, ob, osha), (dname, db, dsha), off in zip(BASE_ORDERS, BASE_DISPATCH, (0, 37, 45)): + base += [[s, r + off] for s, r in json.loads(owners_file(oname, ob, osha))] + bdisp.update(((s, r + off), q) for s, r, q in json.loads(owners_file(dname, db, dsha))) + order = [[s, r + 61 * cp] for s, r in base for cp in range(8)] + dispatch = [[s, r + 61 * cp, bdisp[(s, r)]] for s, r in base for cp in range(8)] + lit = exp['literal_files'] + AC.need(compact_sha(order) == lit['ORDER.json'], 'controls.data_pin: rebuilt ORDER != EXPECTED pin') + AC.need(compact_sha(dispatch) == lit['DISPATCH.json'], 'controls.data_pin: rebuilt DISPATCH != EXPECTED pin') + prov = owners_file('PROVIDERS.json', *PROVIDERS_PIN) + AC.need(AC.sha256(prov) == lit['PROVIDERS.json'], 'controls.data_pin: PROVIDERS.json != EXPECTED pin') + return dict(rows=rows, payload_sha256=exp['payload_sha256'], order=order, dispatch=dispatch, + providers=json.loads(prov), replicas=exp['replicas'], + profile=dict((int(k), v) for k, v in exp['profile'].items()), + template=json.loads(gzip.decompress(show_pinned(*TEMPLATE))), + plan=json.loads(gzip.decompress(show_pinned(*PLAN)))) + + +def read_local_fill(owners_dir): + """Rows of the locally regenerated FILL.json.gz (file sha256 = PR3633 EXPECTED literal_files pin).""" + raw = open(os.path.join(owners_dir, 'FILL.json.gz'), 'rb').read() + AC.need(AC.sha256(raw) == json.loads(spec_bytes(EXPECTED_JSON))['literal_files']['FILL.json.gz'], + 'controls.data_pin: FILL.json.gz sha256') + j = json.loads(gzip.decompress(raw)) + AC.need(j['columns'] == COLUMNS and j['dispatch_overrides'] == [], 'controls.data_pin: journal schema') + return j['rows'] + + +def source_window(template, clock, r, child): + """(lo, hi) of idle child `child` in replica r: hi = time of the replica's latest local sweep when the child's idle + sweep runs, lo = latest time the sweep's pair was touched (local sweep or bridge) before it, else -1.""" + pair_of_stage = dict((x['stage'], x['pair']) for x in template['recipe'] if x['kind'] == 'local') + last, now = {}, None + for ev in template['execution_schedule']: + if ev['kind'] == 'local_cover_sweep': + now = clock[(ev['stage'], r)] + last[pair_of_stage[ev['stage']]] = now + elif ev['kind'] == 'bridge': + last[ev['src']] = last[ev['dst']] = now + elif ev['kind'] == 'idle_cover_sweep' and child in ev['child_indices']: + return last.get(ev['pair'], -1), now + raise AC.AuditFail('controls.data_window: child %d has no source window' % child) + + +def first_role(rows): + """Index of the first role in (owner, child, port, time) order: the first entry of the gate's z[idx].""" + return min(range(len(rows)), key=lambda i: (rows[i][2], rows[i][3], rows[i][4], rows[i][0], i)) + + +def evaluate(res, d): + """The premises the proofs use, checked on the actual T488 values (each with its own rule id), and the predicate + values, which are fixed by each predicate's form once the premises hold (reported with their basis).""" + rows, order = d['rows'], d['order'] + AC.need(payload_sha(rows) == d['payload_sha256'], 'controls.data_pin: journal rows do not re-serialise to the ' + 'pinned FILL payload sha256') + N, M = len(order), 408 + AC.need(len(rows) == N * M, 'controls.data_rows: %d rows != %d calls x 408' % (len(rows), N)) + t0, h0, o0, c0, p0, z0 = rows[0] + prem, val = {}, {} + # duplicate helper: the gate 'complete literal ordered helper identities' (time, helper) == (i // 408, i % 408) + AC.need(all(r[0] == i // M and r[1] == i % M for i, r in enumerate(rows)), + 'controls.data_identity: helper/time columns are not the ordered identities') + prem['duplicate helper'] = 'rows are the ordered (call, helper) identities' + helper = [r[1] for r in rows] + shifted = [helper[0]] + helper[:-1] + val['duplicate helper'] = dict(value=shifted == [i % M for i in range(len(rows))], + basis='data-independent (shifted_column: entries 0 and 1 are equal)') + # strict lower endpoint: the window gate on row 0, lo < t <= hi + clock = dict(((s, r), t) for t, (s, r) in enumerate(order)) + lo, hi = source_window(d['template'], clock, o0, c0) + AC.need(lo < t0 <= hi, 'controls.data_window: row 0 time %d outside its window (%d, %d]' % (t0, lo, hi)) + prem['strict lower endpoint used as available'] = dict(lo=lo, hi=hi, t=t0, holds='lo < t <= hi') + val['strict lower endpoint used as available'] = dict(value=bool(lo < lo <= hi), basis='data-independent (lo < lo)') + # same-call family reused: the physical family of row 0 + fam = 2 * d['plan']['pairs'][str(c0)] + d['plan']['sides'][str(c0)] + f = o0 * 896 + fam * 112 + p0 + val['same-call family reused'] = dict(value=len({int(f), int(f)}) == 2, family=f, + basis='data-independent (one-element set)') + # unpaid chronological hole: the first sorted role's bin has prefix-sum offsets (the gate z[idx] == expected) + prov = dict(((s, r), p) for s, r, p in d['dispatch']) + width = lambda t, h: d['providers'][prov[tuple(order[t])]]['widths'][h] # noqa: E731 + k0 = first_role(rows) + b = rows[k0][2:5] + members = sorted((r[0], i) for i, r in enumerate(rows) if r[2:5] == b) + acc = 0 + for t, i in members: + AC.need(rows[i][5] == acc, 'controls.data_prefix: bin %r offset at time %d is %d, prefix %d' + % (b, t, rows[i][5], acc)) + acc += width(t, rows[i][1]) + prem['unpaid chronological hole'] = dict(bin=b, bin_roles=len(members), holds='offsets are prefix sums of widths') + val['unpaid chronological hole'] = dict(value=rows[k0][5] + 1 == 0, z=rows[k0][5], expected=0, + basis='reduced form: expected[0] = 0 (first role of its bin) and the ' + 'premise gives z = 0') + # one unpaid rank1 child: with the gate dict(H) == profile the predicate is profile == bad + AC.need(sum(k * v for k, v in d['profile'].items()) == 65088 * d['replicas'], + 'controls.data_mass: profile mass != 65088 x replicas (second conjunct of the same gate)') + prem['one unpaid rank1 child'] = dict(rank1=d['profile'][1], holds='profile mass = 65088 x 488; H not recomputed') + bad = dict(d['profile']) + bad[1] += 1 + val['one unpaid rank1 child'] = dict(value=d['profile'] == bad, basis='reduced form: profile == profile + {1: 1}') + return dict(premises=prem, predicate_values=val) + + +# -------------------------------------------------------------------------------------------- controls --- +def controls(source, ctx, out): + """Negative controls, all through analyse(): source edits (re-pinned, so only the named rule can reject them) and, + with T488 data, data mutants (re-pinned payload unless the pin itself is the target).""" + text = source.decode() + + def through(src, data='keep', repin=True): + b = src.encode() if isinstance(src, str) else src + c = dict(ctx, sha256=AC.sha256(b) if repin else ctx['sha256']) + if data != 'keep': + c['data'] = data + return lambda: analyse(b, c) + + def sub(old, new, base=None): + base = text if base is None else base + AC.need(base.count(old) == 1, 'controls.harness: anchor %r found %d times' % (old[:40], base.count(old))) + return base.replace(old, new) + + def taut(name, src, verdict): + AL.expect(out, name, 'controls.tautology', through(src, None), attrs=dict(verdict=verdict)) + + gate109 = "need(np.array_equal(z[idx],expected),'every chronological pending mask is a compact prefix')" + hist = "need(dict(H)==profile and sum(r*n for r,n in H.items())==65088*T,'independent complete histogram')" + hole = 'lambda: int(z[idx[0]])+1==int(expected[0])' + region = 'controls=[]\n' # first statement after the last premise gate, before every control + b = bytearray(source) + b[100] ^= 1 + AL.expect(out, 'source byte flipped, pin kept', 'controls.pin', through(bytes(b), None, repin=False)) + # verdicts other than tautological + scaffold = sub(gate109, "def prefix_ok(zz,ee):return bool(np.array_equal(zz,ee))\n" + "need(prefix_ok(z[idx],expected),'every chronological pending mask is a compact prefix')") + taut('genuine control: the hole lambda feeds a shifted journal to the gate predicate prefix_ok', + sub(hole, 'lambda: prefix_ok(np.r_[z[idx][:1]+1,z[idx][1:]],expected)', scaffold), 'genuine') + taut('gate-coupled lambda: not prefix_ok(z[idx],expected), the gate call itself (not genuine)', + sub(hole, 'lambda: not prefix_ok(z[idx],expected)', scaffold), 'gate-coupled') + # each proof-pattern condition: a satisfiable (or unsupported) edit must not be certified + lower = 'lo[o[0],c[0]]=0\n' + gate109), + 'unproven') + taut('gate_offset: index rebound by a with statement before the gate', + sub(gate109, "with open('/dev/null') as idx:pass\n" + gate109), 'unproven') + taut('region: expected=expected+1 between gate and control', sub(region, region + 'expected=expected+1\n'), + 'unproven') + taut('region: np.copyto(z,z-1) between gate and control', sub(region, region + 'np.copyto(z,z-1)\n'), 'unproven') + taut('region: alias e=expected[:]; e[0]+=1 between gate and control', + sub(region, region + 'e=expected[:];e[0]+=1\n'), 'unproven') + taut('region: rows[0,5]-=1 between gate and control (z is a view of rows; no premise name)', + sub(region, region + 'rows[0,5]-=1\n'), 'unproven') + taut('region: alias r2=rows; r2[0,5]-=1 between gate and control (alias bound in the region, not fresh)', + sub(region, region + 'r2=rows\nr2[0,5]-=1\n'), 'unproven') + taut('region: module function (mutating z, defined before the gate) called between gate and control', + sub(region, region + 'f()\n', sub(gate109, 'def f():\n z[0]=1\n' + gate109)), 'unproven') + taut('gate_bump: bad[1]+=0 (bad == profile)', sub('bad[1]+=1', 'bad[1]+=0'), 'unproven') + taut('gate_bump: bad[1]*=1 (bad == profile)', sub('bad[1]+=1', 'bad[1]*=1'), 'unproven') + taut('gate_bump: histogram gate with "or" instead of "and"', sub(hist, hist.replace(' and ', ' or ')), 'unproven') + taut('gate_bump: histogram gate without its dict(H)==profile conjunct', + sub(hist, "need(sum(r*n for r,n in H.items())==65088*T,'independent complete histogram')"), 'unproven') + taut('gate_bump: H[1]+=1 between the histogram gate and the control', sub(region, region + 'H[1]+=1\n'), + 'unproven') + taut('gate_bump: bad.__setitem__(1,bad[1]-1) inside the bad pair (bad == profile after the bump)', + sub('bad=dict(profile);bad[1]+=1;', 'bad=dict(profile);bad.__setitem__(1,bad[1]-1);bad[1]+=1;'), 'unproven') + lam_lower = 'lambda: bool(lo[o[0],c[0]] %d, whose source window excludes time %d (re-pinned)' % (c0, c1, t0), + 'controls.data_window', through(source, dict(d, rows=rows, payload_sha256=payload_sha(rows)), False)) + rows = list(d['rows']) + k0 = first_role(rows) + rows[k0] = rows[k0][:5] + [rows[k0][5] + 1] + AL.expect(out, 'T488 first-bin role offset +1, an unpaid hole (re-pinned)', 'controls.data_prefix', + through(source, dict(d, rows=rows, payload_sha256=payload_sha(rows)), False)) + prof = dict(d['profile']) + prof[1] += 1 + AL.expect(out, 'T488 profile rank-1 count +1', 'controls.data_mass', through(source, dict(d, profile=prof), False)) + + +# ------------------------------------------------------------------------------------------------ main --- +def run_integrated(I=None, owners_rows=None, with_controls=True): + """Structural analysis of PR3633's auditor and its located copies; with owners_rows (the journal rows [t, h, o, c, p, z] + in journal order) also the T488 premises, the rows bound to PR3633's payload sha256 by re-serialisation. I is + accepted for the integration interface; inputs are read with this module's own pinned git reads.""" + t0 = time.time() + copies, original_found, scanned = scan_trees() + source = spec_bytes(TARGET) + ctx = dict(sha256=TARGET[3], data=None if owners_rows is None else build_t488(owners_rows)) + target = analyse(source, ctx) + per_copy = {} + for label, spec in sorted(COPIES.items()): + r = analyse(spec_bytes(spec), dict(sha256=spec[3], data=None)) + per_copy[label] = dict(path=spec[1], commit=COMMITS[spec[0]], git_blob=spec[2], sha256=spec[3], + patterns=[c['pattern'] for c in r['controls']], + all_tautological=all(c['verdict'] == 'tautological' for c in r['controls']), + gates_before_controls=r['gates_before_controls'], + original_sha256_claimed=r['original_sha256_claimed'], + gates_only_here_vs_PR3633=sorted(set(r['gate_messages']) - set(target['gate_messages'])), + gates_missing_here_vs_PR3633=sorted(set(target['gate_messages']) - set(r['gate_messages']))) + out = [] + attempted = 0 + if with_controls: + AL.harness_selftest() + n0 = len(AC.ATTEMPTED) + controls(source, ctx, out) + attempted = len(AC.ATTEMPTED) - n0 + finding = ('All five reject() controls of PR3633 owners/independent_literal.py (and of its %d located copies) are ' + 'tautological: three are unsatisfiable for every input (irreflexive a= 0 and t == lo + 1 + initial += lo < 0 + w = wbc[t][h] + need(0 <= z and z + w <= cap[c], 'owners.capacity: row %d offset %d width %d cap %d' % (i, z, w, cap[c])) + # Pass 2: one role per physical family per call. + seen, current = set(), -1 + for row in rows: + t, h, o, c, p, z = row + if t != current: + seen, current = set(), t + f = (o * CLASSES + cls[c]) * PORTS + p + need(f not in seen, 'owners.family: call %d reuses physical family (%d, %d, %d)' % (t, o, cls[c], p)) + seen.add(f) + # Pass 3: chronological compact prefixes (rows are in time order, so bins fill chronologically). + used = {} + patterns = {} + for row in rows: + t, h, o, c, p, z = row + b = (o * CHILDREN + c) * PORTS + p + w = wbc[t][h] + u = used.get(b, 0) + need(z == u, 'owners.prefix: bin (%d, %d, %d) role at time %d has offset %d, pending prefix %d' + % (o, c, p, t, z, u)) + used[b] = u + w + patterns.setdefault(b, []).append(w) + # Pass 4: the complete invoice over every bin, including empty ones. + idle = Counter() + for o in range(T): + for c in range(CHILDREN): + for p in range(PORTS): + left = cap[c] - used.get((o * CHILDREN + c) * PORTS + p, 0) # >= 0 by passes 1 and 3 + if left: + idle[left] += 1 + H = Counter(dict((r, n * CALLS) for r, n in LOCAL.items())) + H.update(idle) + H = dict(sorted(H.items())) + need(H == dict(sorted(ctx['claimed'].items())), 'owners.histogram: recount %r != claimed %r' % (H, ctx['claimed'])) + mass = sum(r * n for r, n in H.items()) + need(mass == 65088 * T, 'owners.histogram: rank mass %d != 65088 T' % mass) + by_cap = Counter((cap[(b // PORTS) % CHILDREN], tuple(sorted(ws))) for b, ws in patterns.items()) + empty = Counter() + for o in range(T): + for c in range(CHILDREN): + for p in range(PORTS): + if (o * CHILDREN + c) * PORTS + p not in patterns: + empty[cap[c], ()] += 1 + by_cap.update(empty) + return dict(rows=len(rows), calls=CALLS, replicas=T, bins=T * CHILDREN * PORTS, used_bins=len(patterns), + idle=dict(sorted(idle.items())), profile=H, rank_mass=mass, pattern_multiset=by_cap, + active_pair_lemma_triples=ctx['active_pair_lemma_triples'], + window_boundaries=dict(roles_at_closed_upper_endpoint=at_upper, + roles_one_after_strict_lower_endpoint=after_lower, + roles_in_initially_open_windows=initial)) + + +def aggregate_patterns(candidate): + """The literal per-capacity pattern multiset implied by PR3633's aggregate (groups [cap, lo, hi, n], x rows + [group, pattern-vector over widths 4..9, count]).""" + W = (4, 5, 6, 7, 8, 9) + out = Counter() + for gi, q, n in candidate['x']: + capg = candidate['groups'][gi][0] + out[capg, tuple(w for w, k in zip(W, q) for _ in range(k))] += n + return out + + +def check_aggregate(record, candidate): + """owners.aggregate: the literal journal realises exactly the aggregate's bin patterns, capacity by capacity.""" + agg = aggregate_patterns(candidate) + lit = record['pattern_multiset'] + need(agg == lit, 'owners.aggregate: literal pattern multiset differs from the aggregate in %d entries' + % len(set(agg.items()) ^ set(lit.items()))) + return dict(distinct_capacity_patterns=len(lit), bins=sum(lit.values())) + + +def check_stock(word, template, buffers): + """owners.stock: physical = 896 T + 408 helpers + 8 centres + 91 buffers + 2 coefficient registers, each + component read from the staged word (helpers, centres, private buffers) and the template (coefficient + namespace). PR3633 reports 437,757.""" + helpers = word['report']['R'] - len(word['removed']) + centres = len(word['centers']) + lo, hi = template['private_namespace']['coefficient'] + total = CLASSES * PORTS * T + helpers + centres + buffers + (hi - lo) + need((helpers, centres, buffers, hi - lo) == (408, 8, 91, 2) and total == 437757, + 'owners.stock: physical %d from (%d, %d, %d, %d)' % (total, helpers, centres, buffers, hi - lo)) + return dict(live=CLASSES * PORTS * T, helpers=helpers, centres=centres, private_buffers=buffers, + coefficient=hi - lo, physical=total) + + +# --------------------------------------------------------------------------------------------------------------- +# Negative controls. Each mutates one object (journal row, claimed profile, input file, template), runs the SAME +# function the positive run uses (audit, make_ctx, check_order, check_dispatch, binding, check_aggregate, +# check_stock) and must be rejected with exactly the named rule id (audit_lib.expect). Journal rows are mutated in +# place and restored in a finally block. + +def _swap_row(rows, i, new): + old = rows[i] + rows[i] = new + return old + + +def _with_row(rows, i, new, fn): + old = _swap_row(rows, i, new) + try: + fn() + finally: + rows[i] = old + + +def _with_rows(rows, changes, fn): + olds = [(i, rows[i]) for i, _ in changes] + for i, new in changes: + rows[i] = new + try: + fn() + finally: + for i, old in olds: + rows[i] = old + + +def bin_index(rows, ctx): + """Per bin: chronological list of row indices (positive data only).""" + out = {} + for i, (t, h, o, c, p, z) in enumerate(rows): + out.setdefault((o, c, p), []).append(i) + return out + + +def controls(rows, ctx, inp, AL): + out = [] + cap, cls, win, wbc = ctx['cap'], ctx['cls'], ctx['win'], ctx['wbc'] + run = lambda: audit(rows, ctx) + E = lambda name, rule, fn, **kw: AL.expect(out, name, rule, fn, **kw) + t, h, o, c, p, z = rows[5] + E('row_not_six_integers', 'owners.schema', lambda: _with_row(rows, 5, [t, h, o, c, p, float(z)], run)) + E('swapped_helper_identities', 'owners.identity', + lambda: _with_rows(rows, [(5, [t, rows[6][1], o, c, p, z]), (6, [t, h] + rows[6][2:])], run)) + E('port_out_of_range', 'owners.bounds', lambda: _with_row(rows, 5, [t, h, o, c, PORTS, z], run)) + # Strict lower endpoint: re-host a role at time t on a child of the same replica that opens exactly at t. + lower = next((i, c2) for i, (t, h, o, c, p, z) in enumerate(rows) for c2 in range(CHILDREN) + if win[o][c2][0] == t) + i, c2 = lower + E('role_at_strict_lower_endpoint', 'owners.window', lambda: _with_row(rows, i, rows[i][:3] + [c2] + rows[i][4:], run), + row=i) + upper = next((i, c2) for i, (t, h, o, c, p, z) in enumerate(rows) for c2 in range(CHILDREN) + if win[o][c2][1] == t - 1 and win[o][c2][0] < t) + i2, c3 = upper + E('role_after_closed_upper_endpoint', 'owners.window', + lambda: _with_row(rows, i2, rows[i2][:3] + [c3] + rows[i2][4:], run), row=i2) + # Active local pair: put a role of call t on its own call's replica, on a child of the call's active pair (the + # per-role gate runs before the window check); and a window table whose child stays open across a same-pair stage + # (the template-level lemma, through the function make_ctx uses). + t, h, o, c, p, z = rows[11] + c_act = next(c2 for c2 in range(CHILDREN) if ctx['pair_of'][c2] == ctx['call_pair'][t]) + E('role_on_own_active_local_pair', 'owners.active_pair', + lambda: _with_row(rows, 11, [t, h, ctx['call_replica'][t], c_act, p, z], run)) + bad_win = [list(w) for w in win] + r0 = 0 + c0 = next(c2 for c2 in range(CHILDREN) if ctx['pair_of'][c2] == ctx['stage_pair'][STAGES - 1]) + bad_win[r0][c0] = (-1, ctx['clock'][STAGES - 1, r0]) + E('window_open_across_same_pair_stage', 'owners.active_pair', + lambda: check_active_pair_lemma(bad_win, ctx['clock'], ctx['stage_pair'], ctx['pair_of'])) + # Same-call family reuse: put role B of call 0 on role A's bin at A's offset, with B no wider than A. + a = 0 + b = next(j for j in range(1, HELPERS) if wbc[0][rows[j][1]] <= wbc[0][rows[a][1]]) + E('same_call_family_reused', 'owners.family', + lambda: _with_row(rows, b, rows[b][:2] + rows[a][2:5] + [rows[a][5]], run), rows=[a, b]) + t, h, o, c, p, z = rows[7] + E('offset_overflows_capacity', 'owners.capacity', + lambda: _with_row(rows, 7, [t, h, o, c, p, cap[c] - wbc[t][h] + 1], run)) + bins = bin_index(rows, ctx) + # Unpaid hole: the last role of a bin with spare capacity moves up by one. + hole = next(ix[-1] for key, ix in bins.items() + if rows[ix[-1]][5] + wbc[rows[ix[-1]][0]][rows[ix[-1]][1]] < cap[key[1]]) + E('unpaid_chronological_hole', 'owners.prefix', + lambda: _with_row(rows, hole, rows[hole][:5] + [rows[hole][5] + 1], run), row=hole) + # Offsets not in time order: in a two-role bin with different widths, the later role takes offset 0. + pair = next(ix for key, ix in bins.items() if len(ix) == 2 + and wbc[rows[ix[0]][0]][rows[ix[0]][1]] != wbc[rows[ix[1]][0]][rows[ix[1]][1]]) + w0, w1 = (wbc[rows[j][0]][rows[j][1]] for j in pair) + E('offsets_not_in_time_order', 'owners.prefix', + lambda: _with_rows(rows, [(pair[0], rows[pair[0]][:5] + [w1]), (pair[1], rows[pair[1]][:5] + [0])], run), + rows=list(pair)) + E('dropped_role', 'owners.shape', lambda: audit(rows[:-1], ctx)) + bad = dict(ctx, claimed=dict(ctx['claimed'])) + bad['claimed'][1] += 1 + E('claimed_profile_plus_one_rank1', 'owners.histogram', lambda: audit(rows, bad)) + # The pre-change PR3133 provider widths (main's PR3133-PROVIDERS.json; they differ from the staged words exactly at + # helpers 389 and 397, 7 -> 8) do not host this journal. + old = dict((pid, list(ws)) for pid, ws in inp['pr3133_widths'].items()) + need(all([j for j in range(HELPERS) if old[pid][j] != inp['widths'][pid][j]] == [389, 397] for pid in old), + 'owners.widths: PR3133 and staged widths do not differ exactly at helpers 389 and 397') + oldctx = dict(ctx, wbc=[old[pid] for pid in inp['providers_by_call']]) + E('pr3133_widths_for_helpers_389_397', inp['pr3133_width_rule'], lambda: audit(rows, oldctx)) + order = [list(x) for x in inp['order']] + order[0], order[1] = order[1], order[0] + E('order_entries_swapped', 'owners.order', lambda: check_order(order, *inp['base_orders'])) + disp = [list(x) for x in inp['dispatch']] + disp[0][2] = 'extra' if disp[0][2] == 'gauge44' else 'gauge44' + E('dispatch_provider_flipped', 'owners.dispatch', lambda: check_dispatch(disp, inp['order'], *inp['base_dispatch'])) + tm = json.loads(json.dumps(inp['template'])) + sw = next(e for e in tm['execution_schedule'] if e['kind'] == 'idle_cover_sweep' and e['from_level'] >= 0) + sw['from_level'] -= 1 + E('template_level_annotation_shifted', 'owners.windows_agree', + lambda: make_ctx(inp['order'], inp['clock'], inp['providers_by_call'], inp['widths'], tm, inp['plan'], + inp['claimed'])) + tk = json.loads(json.dumps(inp['template'])) + next(e for e in tk['execution_schedule'] if e['kind'] == 'terminal')['kind'] = 'terminal_mutated' + E('template_unknown_event_kind', 'owners.windows', + lambda: make_ctx(inp['order'], inp['clock'], inp['providers_by_call'], inp['widths'], tk, inp['plan'], + inp['claimed'])) + pl = json.loads(json.dumps(inp['plan'])) + pl['sides']['1'] = 1 - pl['sides']['1'] + E('plan_side_flipped_merges_family_classes', 'owners.windows_disjoint', + lambda: make_ctx(inp['order'], inp['clock'], inp['providers_by_call'], inp['widths'], inp['template'], pl, + inp['claimed'])) + t, h, o, c, p, z = rows[9] + E('journal_row_changed_after_pinning', 'owners.binding', + lambda: _with_row(rows, 9, [t, h, o, c, p, z + 1], lambda: binding(rows, inp['payload_sha256']))) + swapped = [list(x) for x in inp['order']] + swapped[0], swapped[1] = swapped[1], swapped[0] + E('rebuilt_order_differs_from_pinned_literal_file', 'owners.literal_files', + lambda: check_literal_files(inp['expected'], swapped, inp['dispatch'], inp['claimed'])) + bad_disp = [list(x) for x in inp['dispatch']] + bad_disp[-1][2] = 'extra' if bad_disp[-1][2] == 'gauge44' else 'gauge44' + E('rebuilt_dispatch_differs_from_pinned_literal_file', 'owners.literal_files', + lambda: check_literal_files(inp['expected'], inp['order'], bad_disp, inp['claimed'])) + cand = json.loads(json.dumps(inp['candidate'])) + cand['x'][0][2] += 1 + E('aggregate_pattern_count_changed', 'owners.aggregate', lambda: check_aggregate(inp['record'], cand)) + wd = json.loads(json.dumps(inp['word'])) + wd['centers'] = wd['centers'][:-1] + E('word_with_seven_centres', 'owners.stock', lambda: check_stock(wd, inp['template'], inp['buffers'])) + return out + + +# --------------------------------------------------------------------------------------------------------------- +# Entry point used by verify.py. I is audit_common.PinnedInputs over verify.INPUTS (git blob id + sha256 pins); +# widths {provider: [408 widths]} come from audit_staged_words (own frame recount of the two supplier words). + +def literal_file_bytes(name, obj): + """PR3633's expander serialisation of its literal side files (owners/portable_expand.py documents the format).""" + if name == 'PROFILE.json': + return (json.dumps(dict((int(k), v) for k, v in sorted(obj.items(), key=lambda kv: int(kv[0]))), indent=2) + + '\n').encode() + return (json.dumps(obj, separators=(',', ':')) + '\n').encode() + + +def check_literal_files(expected, order, dispatch, profile): + """owners.literal_files: the ORDER, DISPATCH and PROFILE rebuilt here serialise to PR3633's pinned literal files.""" + got = dict((n, sha256(literal_file_bytes(n, o))) for n, o in + (('ORDER.json', order), ('DISPATCH.json', dispatch), ('PROFILE.json', profile))) + for n, h in got.items(): + need(h == expected['literal_files'][n], 'owners.literal_files: rebuilt %s sha256 %s != pinned %s' + % (n, h, expected['literal_files'][n])) + return got + + +def run(I, widths, rows, AL, with_controls=True): + """rows: decode_columns(data/FILL-columns.xz); binding() below fixes them to PR3633's pinned payload.""" + exp = I.json('t488_expected') + o37, o8, o16 = I.json('t488_order37'), I.json('t488_order_t8'), I.json('t488_order_t16') + d37, d8, d16 = I.json('t488_dispatch37'), I.json('t488_dispatch_t8'), I.json('t488_dispatch_t16') + base = base_order(o37, o8, o16) + order = [[s, r + BASE * cp] for s, r in base for cp in range(COPIES)] + clock = check_order(order, o37, o8, o16) + bd = {} + for rows_, off in ((d37, 0), (d8, 37), (d16, 45)): + for s, r, pid in rows_: + bd[s, r + off] = pid + dispatch = [[s, r, bd[s, r % BASE]] for s, r in order] + pbc, counts = check_dispatch(dispatch, order, d37, d8, d16) + need(counts == {'extra': 1680, 'gauge44': 2712}, 'owners.dispatch: provider counts %r' % counts) + files = check_literal_files(exp, order, dispatch, exp['profile']) + need(exp['rows'] == CALLS * HELPERS and exp['replicas'] == T and exp['calls'] == CALLS and exp['physical'] == 437757, + 'owners.binding: EXPECTED.json header') + bind = binding(rows, exp['payload_sha256']) + template, plan = I.json('template'), I.json('plan') + ctx = make_ctx(order, clock, pbc, widths, template, plan, exp['profile']) + rec = audit(rows, ctx) + agg = check_aggregate(rec, I.json('t488_candidate')) + word = I.json('word_base') + stock = check_stock(word, template, len(word['helper_final'])) + out = dict(order_dispatch=dict(calls=CALLS, base_replicas=BASE, copies=COPIES, providers=counts, + literal_files_sha256=files), + binding=bind, audit=dict((k, v) for k, v in rec.items() if k != 'pattern_multiset'), + aggregate=agg, stock=stock) + if with_controls: + pr3133 = I.json('pr3133_providers') + inp = dict(widths=widths, pr3133_widths=dict((pid, pr3133[pid]['widths']) for pid in widths), providers_by_call=pbc, pr3133_width_rule='owners.capacity', order=order, + base_orders=(o37, o8, o16), dispatch=dispatch, base_dispatch=(d37, d8, d16), template=template, + plan=plan, clock=clock, claimed=exp['profile'], payload_sha256=exp['payload_sha256'], + candidate=I.json('t488_candidate'), record=rec, word=word, buffers=len(word['helper_final']), + expected=exp) + out['controls'] = controls(rows, ctx, inp, AL) + return out diff --git a/research/t488-independent-audit/audit_staged_words.py b/research/t488-independent-audit/audit_staged_words.py new file mode 100644 index 0000000000..64c6fc47b4 --- /dev/null +++ b/research/t488-independent-audit/audit_staged_words.py @@ -0,0 +1,506 @@ +# SPDX-License-Identifier: Apache-2.0 +"""Scope C1: PR3633's two supplier words (PR3605's staged 644/660 width reduction) through this contributor's merged +PR3053 word lineage, with no code of PR3633, PR3605, PR3630, PR3624 or PR3626. + +Inputs (all read with plain `git -C show :`, then pinned by git blob id and sha256; nothing of a +PR is imported or executed): + PR3633 head 72c268f1 research/staged-width-t488/supplier3605/STAGED644-660-{base,extra}.json.gz (the words) + research/staged-width-t488/owners/SOURCE-PINS.json (its native_source_word_sha256 pins) + research/staged-width-t488/owners/PROVIDERS.json (widths used by its owner journal) + PR3605 head 3efbb5de research/staged-644-660-width-reduction/STAGED644-660-*.json.gz (byte identity only) + main 20ecf289 research/rank7-coupled-586-588-carrier-fold/edits/FINAL-WORD-*.json.gz (PR3133 parents; + byte-identical copies at research/coupled-carrier-cap740/words/*.json.gz) + research/coupled-carrier-cap740/vendor/PR3133-PROVIDERS.json + +audit_word(tag, raw, ctx) is the per-word main check (positive run and every negative control): + staged.pin sha256(raw) is the pinned value + vendored word_pipeline (audit_words, PR3053): parse + 36 initial corrections, warp algebra, literal frame recount, + words.counts, words.report_fields, words.declared, forward and dual free-operator replay + through cleanup, the own exact tracker, words.cleanup (literal inverse-reverse), words.buffers + staged.providers recount helper widths == PR3633 PROVIDERS.json entry (base->gauge44, extra->extra) + staged.providers_paid recount paid profile == the same entry's actual_paid_profile + staged.pr3133_widths the widths differ from the parent's PR3133-PROVIDERS entry exactly at helper indices 389, 397 + (8 -> 7) + staged.registers those indices are the live helper registers PR3605 names (644 and 660; ctx['staged_regs']) + staged.parent_diff.* localisation against the PR3133 parent word (one rule id per fact): + .keys / .replace / .warps same top-level keys; no unequal replace block; warps change only together with B + .warp_record / .warp_alignment every warp record unchanged except its B position, which follows the B alignment + .sections section-level counts == PR3605's published edit (B: 8 deleted, 4 inserted; cleanup: 4 + inserted; restore: 1 replaced; report: profile/inventory; start_extra: 644/660; warps: 72 + moved); every other section unchanged + .deleted the 8 deleted B items are helper-to-target writes, 4 from 644 and 4 from 660 + .inserted the 4 inserted B items are helper-to-helper donor releases, 2 from 644 and 2 from 660 + .cleanup_pairs the 4 cleanup inserts carry the same (target, source) pairs as the B inserts + .restore the replaced restore item is 210 and only its frame changes, to FULL + .start_extra start_extra of 644/660 goes from [30] to [429, 30] +Expected counts (words.counts) and why: + live 632 = 2*112 + 408; helpers 408 (PR3605 README "all 408 helper roles"; PR3633's owner audit gates R==408); + buffers 91 (PR3605 README "91 private buffers"; PR3633 gates len(zero_buffers)==91); warps 72 and corrections 36 + (the lineage's published EXPECT_COUNTS, main research/t74-3015-restore-edit-audit/verify.py, and the parent word + recounted by the same function); a_prelude_events and the section lengths = the parent's, except B -4 and + cleanup +4 (PR3605: eight writes replaced by four releases whose inverses enter the cleanup). +Declared profile/inventory (words.declared): staged {1:2439,2:726,3:224,4:102,5:20,8:8} / + {4:26,5:2,6:18,7:52,8:97,9:213}; parent {1:2433,2:724,3:228,4:102,5:20,8:8} / {4:26,5:2,6:18,7:50,8:99,9:213} + (PR3605 README; the parent profile also equals PR3133-PROVIDERS actual_paid_profile). +Negative controls go through audit_word, each on both words, and are counted by audit_lib.expect (exact rule id, and +stage for replays); every rule above has at least one control that only it can reject (the parent_diff facts by +parent mutants that keep the section counts, so each reaches its own fact). staged.registers is controlled with a +mismatched declared register pair (the data side, P['live'], is the vendored parser's; a relabelled-word control was +not built). Not mutation-controlled: the fail-closed git/pin reads of fixed inputs (staged.git, staged.pin, +staged.pr3605_identity) and run()'s parent positive control (staged.parent_counts, staged.parent_providers), which +must pass on the pinned bytes. Standard library only. +""" +import copy +import difflib +import gzip +import json +import os +import subprocess +import sys +import time +from collections import Counter + +_HERE = os.path.dirname(os.path.abspath(__file__)) +_VENDOR = os.path.join(_HERE, 'vendor', 'pr3053') +if _VENDOR not in sys.path: + sys.path.insert(0, _VENDOR) +import audit_common as AC # noqa: E402 +import audit_lib as AL # noqa: E402 +import audit_words as AW # noqa: E402 + +REPO = os.path.dirname(os.path.dirname(_HERE)) +MAIN = '20ecf2891d80b5acf18d980b2164fb44ea7faa90' +PR3633 = '72c268f113368ea0387d188225e3f81ba6d6ef73' +PR3605 = '3efbb5de06a49ffeea6684482f2d3fe48cdca3fb' +V = AL.V +FULL = list(AL.FULL_LIST) +TAGS = ('base', 'extra') +PROVIDER = dict(base='gauge44', extra='extra') + +# (commit, path, git blob, sha256) +STAGED = dict( + base=(PR3633, 'research/staged-width-t488/supplier3605/STAGED644-660-base.json.gz', + 'dc6d321b1e8e2a837a1ff7bb60e8f8f78d043db5', '735c540b1fe29d0ceeb4d2f304ec941e5f2bd3bc92bafc8cce2b4f9a67a3b653'), + extra=(PR3633, 'research/staged-width-t488/supplier3605/STAGED644-660-extra.json.gz', + '78e7b2bb0653c8dcc0133980a19dd9d97ebdb67e', '886338fc32da14f3389a82c8120a1bcf9c5e2f0ec3e9b2d377fdae809e1d977f')) +PR3605_PATH = 'research/staged-644-660-width-reduction/STAGED644-660-%s.json.gz' +PARENT = dict( + base=(MAIN, 'research/rank7-coupled-586-588-carrier-fold/edits/FINAL-WORD-base.json.gz', + 'cb4cc24a4545944627ad595c0c5080196952b9f0', '300a9b8ad1d554f0e6f84f43f3f5fd870a5b4e5f81ad856900b6e8999684892e'), + extra=(MAIN, 'research/rank7-coupled-586-588-carrier-fold/edits/FINAL-WORD-extra.json.gz', + 'decc8f26d8aab108baf75f0d8b2d1012eb06d076', '495d030f1e03b90e954dad3043994f03293d41b9ed220a3beaef849df435d3f0')) +PARENT_COPY = 'research/coupled-carrier-cap740/words/%s.json.gz' +SOURCE_PINS = (PR3633, 'research/staged-width-t488/owners/SOURCE-PINS.json', '5aa0ee4cb46308e674c5e1cedaa7eefc7bac59bd', + '7dec42af587af3b7ae66d4fdf3d0c6157f9c85eb744edec66fa266f294378e5e') +PROVIDERS = (PR3633, 'research/staged-width-t488/owners/PROVIDERS.json', 'f321d4185339244b8b004326af70e653e611d84a', + 'd96f2798024ab83edbb8848b62c425992aec182f1f24ef46bb8286dd390618c2') +PR3133_PROVIDERS = (MAIN, 'research/coupled-carrier-cap740/vendor/PR3133-PROVIDERS.json', None, + '52eb0d85caa7b2d4a93acdf5011f52b793f61ddca86235fb03cd9a09397ad4cd') + +DECLARED = dict(local_profile={1: 2439, 2: 726, 3: 224, 4: 102, 5: 20, 8: 8}, + width_inventory={4: 26, 5: 2, 6: 18, 7: 52, 8: 97, 9: 213}) +DECLARED_PARENT = dict(local_profile={1: 2433, 2: 724, 3: 228, 4: 102, 5: 20, 8: 8}, + width_inventory={4: 26, 5: 2, 6: 18, 7: 50, 8: 99, 9: 213}) +EXPECT_LINEAGE = dict(live=2 * V + 408, helpers=408, buffers=91, warps=72, corrections=36) +SECTION_DELTA = dict(B=-4, cleanup=4) +WIDTH_CHANGES = [(389, 8, 7), (397, 8, 7)] +STAGED_REGS = (644, 660) +LOCALISATION = dict( + B=dict(deleted=8, inserted=4, replaced=0), + cleanup=dict(deleted=0, inserted=4, replaced=0), + restore=dict(deleted=0, inserted=0, replaced=1), + report=dict(keys=['local_profile', 'width_inventory']), + start_extra=dict(keys=['644', '660']), + warps=dict(position_only=72)) +STAGE_FWD = 'forward replay, before cleanup' +STAGE_DUAL_BUF = 'dual replay, buffers after cleanup' +STAGE_FWD_FINAL = 'forward replay, final map after cleanup' + + +# ------------------------------------------------------------------------------------------------- inputs --- +def git(*args): + p = subprocess.run(['git', '-C', REPO] + list(args), stdout=subprocess.PIPE, stderr=subprocess.PIPE, + env=AC.git_env()) + AC.need(p.returncode == 0, 'staged.git: git %s failed: %s' % (' '.join(args)[:120], p.stderr.decode()[:200])) + return p.stdout + + +def read_pinned(spec): + """Bytes of commit:path, checked against the pinned git blob id and sha256 (None = not pinned).""" + commit, path, blob, sha = spec + data = git('show', '%s:%s' % (commit, path)) + AC.need(blob is None or AC.git_blob_id(data) == blob, 'staged.pin: %s blob %s != %s' + % (path, AC.git_blob_id(data), blob)) + AC.need(sha is None or AC.sha256(data) == sha, 'staged.pin: %s sha256 %s != %s' % (path, AC.sha256(data), sha)) + return data + + +def gz_json(raw): + return json.loads(gzip.decompress(raw).decode('utf-8')) + + +def gz_bytes(w): + return gzip.compress(json.dumps(w, separators=(',', ':')).encode(), mtime=0) + + +def counts_of(P, w): + c = AW.counts(P, w) + return dict((k, c[k]) for k in sorted(EXPECT_LINEAGE)), c + + +# ------------------------------------------------------------------------------------------ localisation --- +def list_diff(a, b): + sm = difflib.SequenceMatcher(None, [json.dumps(x) for x in a], [json.dumps(x) for x in b], autojunk=False) + out = dict(deleted=0, inserted=0, replaced=0) + ops = [o for o in sm.get_opcodes() if o[0] != 'equal'] + for tag, i1, i2, j1, j2 in ops: + if tag == 'delete': + out['deleted'] += i2 - i1 + elif tag == 'insert': + out['inserted'] += j2 - j1 + else: + AC.need(i2 - i1 == j2 - j1, 'staged.parent_diff.replace: unequal replace block %r' % ((tag, i1, i2, j1, j2),)) + out['replaced'] += i2 - i1 + return out, ops, sm + + +def b_position_map(sm, n_old, n_new): + """Parent B position -> staged B position for every position inside an equal block (and the end).""" + m = {n_old: n_new} + for tag, i1, i2, j1, j2 in sm.get_opcodes(): + if tag == 'equal': + for k in range(i2 - i1): + m[i1 + k] = j1 + k + return m + + +def localise(parent, w): + """Section-level differences parent -> staged word (counts only), with the per-item facts of PR3605's edit.""" + loc, facts = {}, {} + AC.need(sorted(parent) == sorted(w), 'staged.parent_diff.keys: key sets differ %r' % sorted(set(parent) ^ set(w))) + smB = None + for k in sorted(w): + a, b = parent[k], w[k] + if a == b: + continue + if k == 'warps': + continue + if isinstance(a, list): + loc[k], ops, sm = list_diff(a, b) + if k == 'B': + smB, opsB = sm, ops + facts[k] = ops + elif isinstance(a, dict): + loc[k] = dict(keys=sorted(x for x in set(a) | set(b) if a.get(x) != b.get(x))) + else: + loc[k] = dict(scalar=[a, b]) + if parent['warps'] != w['warps']: + AC.need(smB is not None and len(parent['warps']) == len(w['warps']), 'staged.parent_diff.warps: warps changed without B') + pos = b_position_map(smB, len(parent['B']), len(w['B'])) + moved = 0 + for ra, rb in zip(parent['warps'], w['warps']): + AC.need(ra[1:] == rb[1:], 'staged.parent_diff.warp_record: a warp record changed beyond its position') + AC.need(ra[0] in pos and pos[ra[0]] == rb[0], + 'staged.parent_diff.warp_alignment: warp at parent B position %d does not follow the B alignment' % ra[0]) + moved += ra[0] != rb[0] + loc['warps'] = dict(position_only=moved) + AC.need(loc == LOCALISATION, 'staged.parent_diff.sections: %r != %r' % (loc, LOCALISATION)) + # per-item facts of the published edit + dele = [g for t, i1, i2, _, _ in facts['B'] if t == 'delete' for g in parent['B'][i1:i2]] + ins = [g for t, _, _, j1, j2 in facts['B'] if t == 'insert' for g in w['B'][j1:j2]] + cins = [g for t, _, _, j1, j2 in facts['cleanup'] if t == 'insert' for g in w['cleanup'][j1:j2]] + AC.need(all(g[0] == 'add' and V <= g[1] < 2 * V and g[2] in STAGED_REGS for g in dele) + and Counter(g[2] for g in dele) == Counter({644: 4, 660: 4}), + 'staged.parent_diff.deleted: deleted B items are not 4+4 helper-to-target writes of 644/660') + AC.need(all(g[0] == 'add' and g[1] >= 2 * V and g[2] in STAGED_REGS for g in ins) + and Counter(g[2] for g in ins) == Counter({644: 2, 660: 2}), + 'staged.parent_diff.inserted: inserted B items are not 2+2 donor releases of 644/660') + AC.need(sorted((g[1], g[2]) for g in cins) == sorted((g[1], g[2]) for g in ins), + 'staged.parent_diff.cleanup_pairs: cleanup inserts are not the four inserted (target, source) pairs') + (_, i1, _, _, _), = facts['restore'] + ra, rb = parent['restore'][i1], w['restore'][i1] + AC.need(i1 == 210 and ra[:-1] == rb[:-1] and rb[-1] == FULL and ra[-1] != FULL, + 'staged.parent_diff.restore: restore change is not item 210 frame -> FULL') + AC.need(all(parent['start_extra'][str(r)] == [30] and w['start_extra'][str(r)] == [429, 30] for r in STAGED_REGS), + 'staged.parent_diff.start_extra: start_extra of 644/660 is not [30] -> [429, 30]') + return dict(sections=loc, b_deleted_sources=dict(Counter(g[2] for g in dele)), + b_inserted=[g[:3] for g in ins], restore_index=i1, unchanged_sections=sorted(k for k in w + if parent[k] == w[k])) + + +# ---------------------------------------------------------------------------------------- per-word check --- +def audit_word(tag, raw, ctx): + """The per-word main check (see module docstring). Returns (record, helper widths).""" + AC.need(AC.sha256(raw) == ctx['pins'][tag], 'staged.pin: %s word sha256 %s' % (tag, AC.sha256(raw))) + w = gz_json(raw) + rec, P, fr = AW.word_pipeline(w, ctx['declared'], ctx['expect'][tag]) + widths = list(fr['helper_widths']) + pid = PROVIDER[tag] + prov = ctx['providers'][pid] + AC.need(widths == prov['widths'], 'staged.providers: %s recount widths != PROVIDERS.json[%s] (first diff %r)' + % (tag, pid, next(((i, a, b) for i, (a, b) in enumerate(zip(widths, prov['widths'])) if a != b), None))) + AC.need(fr['paid'] == AL.intkeys(prov['actual_paid_profile']), + 'staged.providers_paid: %s recount paid profile %r != PROVIDERS.json[%s]' % (tag, fr['paid'], pid)) + old = ctx['pr3133'][pid]['widths'] + diffs = [(i, a, b) for i, (a, b) in enumerate(zip(old, widths)) if a != b] + AC.need(len(old) == len(widths) and diffs == WIDTH_CHANGES, + 'staged.pr3133_widths: %s diffs vs PR3133-PROVIDERS %r != %r' % (tag, diffs, WIDTH_CHANGES)) + helpers = [r for r in P['live'] if r >= 2 * V] + AC.need(tuple(helpers[i] for i, _, _ in diffs) == tuple(ctx['staged_regs']), + 'staged.registers: changed indices name registers %r, declared %r' + % ([helpers[i] for i, _, _ in diffs], ctx['staged_regs'])) + loc = localise(ctx['parents'][tag], w) + rec = dict(rec, sha256=AC.sha256(raw), provider=pid, pr3133_width_changes=[list(d) for d in diffs], + changed_registers=list(STAGED_REGS), localisation=loc) + return rec, widths + + +# -------------------------------------------------------------------------------------------- controls --- +def word_controls(tag, raw, ctx, out): + T = '[%s] ' % tag + w0 = gz_json(raw) + + def through(m, ctx2=None, repin=True): + """Run the mutant word through audit_word; repin=True gives it a matching pin, so only the deeper checks can + reject it.""" + r = gz_bytes(m) if isinstance(m, dict) else m + c = dict(ctx2 or ctx) + if repin: + c['pins'] = dict(c['pins'], **{tag: AC.sha256(r)}) + return lambda: audit_word(tag, r, c) + + def expect_count(section, d): + c2 = dict(ctx, expect=dict(ctx['expect'])) + c2['expect'][tag] = dict(ctx['expect'][tag], sections=dict(ctx['expect'][tag]['sections'])) + c2['expect'][tag]['sections'][section] += d + return c2 + + def with_parent(par): + return dict(ctx, parents=dict(ctx['parents'], **{tag: par})) + + bad = bytearray(raw) + bad[len(bad) // 2] ^= 1 + AL.expect(out, T + 'one byte of the gzip stream flipped, pin kept', 'staged.pin', through(bytes(bad), repin=False)) + + m = AL.mutant(w0) + j = next(i for i, g in enumerate(m['B']) if g[0] == 'add' and g[2] in STAGED_REGS and g[1] >= 2 * V) + m['B'][j][3] = -m['B'][j][3] + AL.expect(out, T + 'coefficient of the first donor release B[%d] flipped' % j, 'wc.replay', through(m), + stage=STAGE_FWD) + + m = AL.mutant(w0) + del m['cleanup'][0] + AL.expect(out, T + 'first cleanup item deleted, expected cleanup count lowered to match', 'wc.replay', + through(m, expect_count('cleanup', -1)), stage=STAGE_FWD_FINAL) + + m = AL.mutant(w0) + cl = m['cleanup'] + i = next(k for k in range(len(cl) - 1) if cl[k][0] == cl[k + 1][0] == 'add' + and not {cl[k][1], cl[k][2]} & {cl[k + 1][1], cl[k + 1][2]}) + cl[i], cl[i + 1] = cl[i + 1], cl[i] + P = AL.parse_word(m) + AL.replay(P, False) + AL.replay(P, True) + AW.tracker_check(AW.track(m)) + AL.expect(out, T + 'commuting cleanup items %d/%d swapped (both replays and the tracker accept)' % (i, i + 1), + 'words.cleanup', through(m), replays_and_tracker_accept=True) + + m = AL.mutant(w0) + k = next(k for k in sorted(m['helper_final'], key=int) if len(m['helper_final'][k]) < len(FULL)) + m['helper_final'][k] = list(FULL) + AL.forge(m) + AL.expect(out, T + 'helper_final frame of %s raised to FULL, report forged' % k, 'words.declared', through(m)) + + m = AL.mutant(w0) + restored = set(int(x) for x in m['helper_final']) + n = 2 * V + m['report']['R'] + helper = next(r for r in range(2 * V, n) if r not in set(m['removed']) and r not in restored) + m['cleanup'].insert(0, ['add', helper, n + min(restored), 1, 1, list(FULL)]) + AL.expect(out, T + 'cleanup prepended with helper %d += zero buffer (forward-neutral), cleanup count raised' + % helper, 'wc.replay', through(m, expect_count('cleanup', 1)), stage=STAGE_DUAL_BUF) + + AL.expect(out, T + 'declared profile/inventory of the PR3133 parent', 'words.declared', + through(raw, dict(ctx, declared=DECLARED_PARENT), repin=False)) + + c2 = dict(ctx, expect=dict(ctx['expect'])) + c2['expect'][tag] = dict(ctx['expect'][tag], helpers=409) + AL.expect(out, T + 'expected helper count 409', 'words.counts', through(raw, c2, repin=False)) + + pid = PROVIDER[tag] + prov = copy.deepcopy(ctx['providers']) + prov[pid]['widths'][WIDTH_CHANGES[0][0]] = 8 + AL.expect(out, T + 'PROVIDERS.json[%s] width of helper 644 (index 389) back to 8' % pid, 'staged.providers', + through(raw, dict(ctx, providers=prov), repin=False)) + + prov = copy.deepcopy(ctx['providers']) + prov[pid]['actual_paid_profile']['1'] += 1 + AL.expect(out, T + 'PROVIDERS.json[%s] actual_paid_profile rank-1 count +1, widths kept' % pid, + 'staged.providers_paid', through(raw, dict(ctx, providers=prov), repin=False)) + + old = copy.deepcopy(ctx['pr3133']) + old[pid]['widths'] = list(ctx['providers'][pid]['widths']) + AL.expect(out, T + 'PR3133-PROVIDERS[%s] replaced by the staged widths (no 8->7 change)' % pid, + 'staged.pr3133_widths', through(raw, dict(ctx, pr3133=old), repin=False)) + + AL.expect(out, T + 'declared changed registers (645, 660) instead of PR3605\'s (644, 660)', 'staged.registers', + through(raw, dict(ctx, staged_regs=(645, 660)), repin=False)) + + # parent_diff: each parent mutant keeps every earlier fact, so only the named one can reject it + P0 = ctx['parents'][tag] + par = AL.mutant(P0) + par['note'] = 'extra key' + AL.expect(out, T + 'parent word with an extra top-level key', 'staged.parent_diff.keys', + through(raw, with_parent(par), repin=False)) + + par = AL.mutant(P0) + k = next(k for k in range(len(P0['cleanup'])) if P0['cleanup'][k] == w0['cleanup'][k]) + par['cleanup'][k:k + 1] = [['add', 0, 0, 0, 0, []], ['add', 1, 1, 0, 0, []]] + AL.expect(out, T + 'parent cleanup[%d] replaced by two unmatched items (2-for-1 replace block)' % k, + 'staged.parent_diff.replace', through(raw, with_parent(par), repin=False)) + + par = AL.mutant(P0) + del par['warps'][-1] + AL.expect(out, T + 'parent with its last warp record removed', 'staged.parent_diff.warps', + through(raw, with_parent(par), repin=False)) + + par = AL.mutant(P0) + par['A'][par['prelude_events'] + 5][-1] = list(FULL) + AL.expect(out, T + 'parent word with one A gate frame changed', 'staged.parent_diff.sections', + through(raw, with_parent(par), repin=False)) + + par = AL.mutant(P0) + par['restore'][210] = copy.deepcopy(w0['restore'][210]) + AL.expect(out, T + 'parent word already carrying the staged restore 210', 'staged.parent_diff.sections', + through(raw, with_parent(par), repin=False)) + + _, opsB, _ = list_diff(P0['B'], w0['B']) + dels = [i for t, i1, i2, _, _ in opsB if t == 'delete' for i in range(i1, i2)] + ins_at = [(i1, j1) for t, i1, _, j1, _ in opsB if t == 'insert'] + warp_pos = set(r[0] for r in P0['warps']) + + par = AL.mutant(P0) + AL.delete_B(par, dels[0]) + AL.expect(out, T + 'parent without deleted write B[%d] (warps re-attached): 7 deletions' % dels[0], + 'staged.parent_diff.sections', through(raw, with_parent(par), repin=False)) + + par = AL.mutant(P0) + par['B'][dels[0]][2] += 1 + AL.expect(out, T + 'parent deleted write B[%d] with source %d -> %d' % (dels[0], P0['B'][dels[0]][2], + par['B'][dels[0]][2]), + 'staged.parent_diff.deleted', through(raw, with_parent(par), repin=False)) + + # parent B[i1] := the first inserted release: that release becomes matched and the staged copy of the old B[i1] + # becomes the fourth insertion (counts kept, no warp at i1) + i1, j1 = ins_at[0] + AC.need(i1 not in warp_pos and P0['B'][i1] == w0['B'][j1 + 2], 'controls.harness: B insertion point %d' % i1) + par = AL.mutant(P0) + par['B'][i1] = copy.deepcopy(w0['B'][j1]) + AL.expect(out, T + 'parent B[%d] replaced by the first inserted release: an ordinary item among the insertions' + % i1, 'staged.parent_diff.inserted', through(raw, with_parent(par), repin=False)) + + _, opsC, _ = list_diff(P0['cleanup'], w0['cleanup']) + (c1, cj1), = [(i1, j1) for t, i1, _, j1, _ in opsC if t == 'insert'][:1] + AC.need(P0['cleanup'][c1] == w0['cleanup'][cj1 + 2], 'controls.harness: cleanup insertion point %d' % c1) + par = AL.mutant(P0) + par['cleanup'][c1] = copy.deepcopy(w0['cleanup'][cj1]) + AL.expect(out, T + 'parent cleanup[%d] replaced by the first cleanup insert: one insert pair altered' % c1, + 'staged.parent_diff.cleanup_pairs', through(raw, with_parent(par), repin=False)) + + par = AL.mutant(P0) + par['restore'][210][2] -= 2 + AL.expect(out, T + 'parent restore 210 coefficient changed (non-frame field)', 'staged.parent_diff.restore', + through(raw, with_parent(par), repin=False)) + + par = AL.mutant(P0) + par['start_extra']['644'] = [31] + AL.expect(out, T + 'parent start_extra[644] = [31]', 'staged.parent_diff.start_extra', + through(raw, with_parent(par), repin=False)) + + par = AL.mutant(P0) + par['warps'][0][1] += 1 + AL.expect(out, T + 'parent warps[0] lender changed', 'staged.parent_diff.warp_record', + through(raw, with_parent(par), repin=False)) + + par = AL.mutant(P0) + par['warps'][0][0] += 1 + AL.expect(out, T + 'parent warps[0] B position shifted by one', 'staged.parent_diff.warp_alignment', + through(raw, with_parent(par), repin=False)) + + +# ------------------------------------------------------------------------------------------------ main --- +def run(controls=True): + t0 = time.time() + pins_doc = json.loads(read_pinned(SOURCE_PINS).decode()) + for tag in TAGS: + AC.need(pins_doc['native_canonical'][tag]['native_source_word_sha256'] == STAGED[tag][3], + 'staged.pin: PR3633 SOURCE-PINS native_source_word_sha256[%s] != %s' % (tag, STAGED[tag][3])) + providers = json.loads(read_pinned(PROVIDERS).decode()) + pr3133 = json.loads(read_pinned(PR3133_PROVIDERS).decode()) + raws, parents, identity = {}, {}, {} + for tag in TAGS: + raws[tag] = read_pinned(STAGED[tag]) + b3605 = git('rev-parse', '%s:%s' % (PR3605, PR3605_PATH % tag)).decode().strip() + AC.need(b3605 == STAGED[tag][2], 'staged.pr3605_identity: %s blob %s != %s' % (tag, b3605, STAGED[tag][2])) + same = read_pinned((PR3605, PR3605_PATH % tag, STAGED[tag][2], STAGED[tag][3])) + AC.need(same == raws[tag], 'staged.pr3605_identity: %s bytes differ' % tag) + identity[tag] = dict(pr3633=dict(commit=PR3633, path=STAGED[tag][1], git_blob=STAGED[tag][2]), + pr3605=dict(commit=PR3605, path=PR3605_PATH % tag, git_blob=b3605), + sha256=STAGED[tag][3], identical_bytes=True) + praw = read_pinned(PARENT[tag]) + AC.need(read_pinned((MAIN, PARENT_COPY % tag, PARENT[tag][2], PARENT[tag][3])) == praw, + 'staged.pin: parent copy %s differs' % (PARENT_COPY % tag)) + parents[tag] = gz_json(praw) + # parents (positive): the lineage pipeline with PR3605's "before" profile, widths == PR3133-PROVIDERS + parent_recs, expect = {}, {} + for tag in TAGS: + Pp = AL.parse_word(parents[tag]) + lin, cp = counts_of(Pp, parents[tag]) + AC.need(lin == EXPECT_LINEAGE, 'staged.parent_counts: %s %r != %r' % (tag, lin, EXPECT_LINEAGE)) + recp, _, frp = AW.word_pipeline(parents[tag], DECLARED_PARENT, EXPECT_LINEAGE) + pid = PROVIDER[tag] + AC.need(list(frp['helper_widths']) == pr3133[pid]['widths'] + and frp['paid'] == AL.intkeys(pr3133[pid]['actual_paid_profile']), + 'staged.parent_providers: parent %s recount != PR3133-PROVIDERS[%s]' % (tag, pid)) + parent_recs[tag] = dict(sha256=PARENT[tag][3], path=PARENT[tag][1], also_at=PARENT_COPY % tag, + counts=recp['counts'], profile=recp['profile'], inventory=recp['inventory'], + provider=pid, widths_equal_pr3133_providers=True) + sections = dict(cp['sections']) + for k, d in SECTION_DELTA.items(): + sections[k] += d + expect[tag] = dict(EXPECT_LINEAGE, a_prelude_events=cp['a_prelude_events'], sections=sections) + ctx = dict(pins=dict((t, STAGED[t][3]) for t in TAGS), declared=DECLARED, expect=expect, providers=providers, + pr3133=pr3133, parents=parents, staged_regs=STAGED_REGS) + words, widths = {}, {} + for tag in TAGS: + words[tag], widths[PROVIDER[tag]] = audit_word(tag, raws[tag], ctx) + out = [] + if controls: + AL.harness_selftest() + n0 = len(AC.ATTEMPTED) + for tag in TAGS: + word_controls(tag, raws[tag], ctx, out) + attempted = len(AC.ATTEMPTED) - n0 + return dict(scope='C1 staged supplier words through the PR3053 word lineage', words=words, parents=parent_recs, + pr3605_identity=identity, expect=expect, helper_widths=widths, controls=out, + controls_attempted=attempted if controls else 0, controls_killed=len(out), + runtime_seconds=round(time.time() - t0, 1)) + + +def summary(res): + s = dict(runtime_seconds=res['runtime_seconds'], pr3605_identity=res['pr3605_identity']) + for tag, r in res['words'].items(): + s[tag] = dict(sha256=r['sha256'], counts=r['counts'], profile=r['profile'], inventory=r['inventory'], + independent_inputs=r['independent_inputs'], buffers=r['buffers'], cleanup=r['cleanup'], + tracker_final=r['tracker']['final_identity'], pr3133_width_changes=r['pr3133_width_changes'], + changed_registers=r['changed_registers'], localisation=r['localisation']['sections'], + unchanged_sections=r['localisation']['unchanged_sections']) + s['parents'] = res['parents'] + s['controls'] = [(c['control'], c['expected'], c['rejected_by'] == c['expected']) for c in res['controls']] + return s + + +if __name__ == '__main__': + res = run(controls='--no-controls' not in sys.argv) + print(json.dumps(summary(res), indent=1, sort_keys=True)) + print('PASS C1: %d words, %d of %d controls killed (%.1fs)' % (len(res['words']), res['controls_killed'], + res['controls_attempted'], res['runtime_seconds'])) diff --git a/research/t488-independent-audit/data/FILL-columns.xz b/research/t488-independent-audit/data/FILL-columns.xz new file mode 100644 index 0000000000..1488aa9b61 Binary files /dev/null and b/research/t488-independent-audit/data/FILL-columns.xz differ diff --git a/research/t488-independent-audit/evidence/GATE-DISABLE-DEMO.json b/research/t488-independent-audit/evidence/GATE-DISABLE-DEMO.json new file mode 100644 index 0000000000..35e9f3c9f2 --- /dev/null +++ b/research/t488-independent-audit/evidence/GATE-DISABLE-DEMO.json @@ -0,0 +1,126 @@ +{ + "command": "python3 tools/gate_disable_demo.py OWNERS_DIR LITERAL_DIR NATIVE_DIR PYTHON REPO WORK_DIR (run twice, with identical results)", + "conclusion": "Every variant exits 0 with PASS_INDEPENDENT_LITERAL_OWNER_WINDOWS_FAMILIES_PREFIXES_AND_INVOICE and reports the same five controls: the controls do not detect a disabled gate. The unmodified run reproduces PR3633's owner audit on this machine (roles 1791936, idle {1:114192,3:31842,5:6954,48:186782}, physical 437757).", + "inputs": { + "environment": "Linux x86_64, CPython 3.14.4, numpy 2.5.4", + "literal_dir": "FILL/ORDER/DISPATCH/PROFILE from PR3633 owners/portable_expand.py (FILL payload sha256 20662cc64df34cab409fdf31ed0f0222462530922a01308a46964d661d2374cf; FILL.json.gz sha256 a93f1baa8d1a2eda216fff54708ef6a32586428cbb9ccf5f75fb053e290b7050, equal to PR3633 EXPECTED literal_files)", + "native_dir": "checked-{base,extra}/CANONICAL-WORD.json.gz = this machine's fresh native rebuild with gzip byte 9 set to 0x13; sha256 equal to PR3633 SOURCE-PINS native_canonical (b87f174c..., 8276f3d7...)" + }, + "label": "LOCAL-ONLY evidence (not replayed by verify.py or CI): PR3633 code run as the object under study", + "method": "tools/gate_disable_demo.py: each variant copies owners/ and replaces the condition of one need(...) gate (identified by its exact message) with True; the copy's ARTIFACTS.json self-pin is updated; nothing else changes. Variants: original, each of the five gates the controls are named after, and all five at once.", + "object": { + "commit": "72c268f113368ea0387d188225e3f81ba6d6ef73", + "path": "research/staged-width-t488/owners/independent_literal.py", + "pr": 3633, + "sha256": "8517c0d6c7cbe73438885135ac7fd9781dd5120113c9721002346dca603f5ed9" + }, + "results": [ + { + "controls_reported": [ + "duplicate helper", + "strict lower endpoint used as available", + "same-call family reused", + "unpaid chronological hole", + "one unpaid rank1 child" + ], + "disabled_gates": [], + "exit_code": 0, + "status": "PASS_INDEPENDENT_LITERAL_OWNER_WINDOWS_FAMILIES_PREFIXES_AND_INVOICE", + "variant": "original" + }, + { + "controls_reported": [ + "duplicate helper", + "strict lower endpoint used as available", + "same-call family reused", + "unpaid chronological hole", + "one unpaid rank1 child" + ], + "disabled_gates": [ + "helper_identity" + ], + "exit_code": 0, + "status": "PASS_INDEPENDENT_LITERAL_OWNER_WINDOWS_FAMILIES_PREFIXES_AND_INVOICE", + "variant": "no-helper_identity" + }, + { + "controls_reported": [ + "duplicate helper", + "strict lower endpoint used as available", + "same-call family reused", + "unpaid chronological hole", + "one unpaid rank1 child" + ], + "disabled_gates": [ + "window" + ], + "exit_code": 0, + "status": "PASS_INDEPENDENT_LITERAL_OWNER_WINDOWS_FAMILIES_PREFIXES_AND_INVOICE", + "variant": "no-window" + }, + { + "controls_reported": [ + "duplicate helper", + "strict lower endpoint used as available", + "same-call family reused", + "unpaid chronological hole", + "one unpaid rank1 child" + ], + "disabled_gates": [ + "family" + ], + "exit_code": 0, + "status": "PASS_INDEPENDENT_LITERAL_OWNER_WINDOWS_FAMILIES_PREFIXES_AND_INVOICE", + "variant": "no-family" + }, + { + "controls_reported": [ + "duplicate helper", + "strict lower endpoint used as available", + "same-call family reused", + "unpaid chronological hole", + "one unpaid rank1 child" + ], + "disabled_gates": [ + "prefix" + ], + "exit_code": 0, + "status": "PASS_INDEPENDENT_LITERAL_OWNER_WINDOWS_FAMILIES_PREFIXES_AND_INVOICE", + "variant": "no-prefix" + }, + { + "controls_reported": [ + "duplicate helper", + "strict lower endpoint used as available", + "same-call family reused", + "unpaid chronological hole", + "one unpaid rank1 child" + ], + "disabled_gates": [ + "histogram" + ], + "exit_code": 0, + "status": "PASS_INDEPENDENT_LITERAL_OWNER_WINDOWS_FAMILIES_PREFIXES_AND_INVOICE", + "variant": "no-histogram" + }, + { + "controls_reported": [ + "duplicate helper", + "strict lower endpoint used as available", + "same-call family reused", + "unpaid chronological hole", + "one unpaid rank1 child" + ], + "disabled_gates": [ + "helper_identity", + "window", + "family", + "prefix", + "histogram" + ], + "exit_code": 0, + "status": "PASS_INDEPENDENT_LITERAL_OWNER_WINDOWS_FAMILIES_PREFIXES_AND_INVOICE", + "variant": "no-gates" + } + ] +} \ No newline at end of file diff --git a/research/t488-independent-audit/evidence/NATIVE-REPRODUCTION.json b/research/t488-independent-audit/evidence/NATIVE-REPRODUCTION.json new file mode 100644 index 0000000000..c5b0370ea6 --- /dev/null +++ b/research/t488-independent-audit/evidence/NATIVE-REPRODUCTION.json @@ -0,0 +1,582 @@ +{ + "admission_owners_stage_note": "PR3633 admission stage \"owners\" (package/reconstruct.py, the PR2839 replica reconstruction: windows, helper and family uniqueness, capacity and compact prefixes) exited 0 in runs 2, 3 and 4; its output line is recorded verbatim.", + "admission_owners_stage_output": "PASS_OWNER_VERIFY {1: 114192, 5: 6954, 48: 186782, 3: 31842} {1: 10826280, 2: 3188592, 3: 1015650, 4: 447984, 5: 94794, 8: 35136, 48: 186782}", + "author_receipt_json_files_equal": [ + "local/CLIFFORD-RECIPES.json", + "local/CLIFFORD-RECIPES.json", + "local/LOCAL-CHECKS.json", + "local/LOCAL-CHECKS.json", + "local/RESIDUAL-ADAPTERS.json", + "local/RESIDUAL-ADAPTERS.json", + "local/WRAPPER-CHECKS.json", + "local/WRAPPER-CHECKS.json" + ], + "command": "python -B research/staged-width-t488/replay.py --work WORK --allocation OWNERS --main-files MAIN --t8-package MAIN/research/t8-rank7-two-swaps (PR3633 head 72c268f1; OWNERS from PR3633 owners/portable_expand.py, payload sha256 20662cc6...)", + "conclusion": "Both changed native providers rebuild with PASS (PASS_TWO_FRESH_SHARED_KERNEL_NATIVE_CHAINS_AUTHENTICATED_PARENT_REUSE) in all four runs. Every regenerated CANONICAL-WORD, FULL-WORD and LOCAL-PROGRAMS .json.gz equals the author-pinned bytes once gzip header byte 9 (OS) is set to 0x13 (sha256 comparison; for FULL-WORD also a direct byte comparison showing offset 9 as the only difference); payloads are identical. The complete replay did not finish on this machine: owners passed in runs 2-4; parent-lift was OOM-killed in run 2 and passed in runs 3 and 4; rebind and discovery bind passed in runs 3 and 4; discovery emit was OOM-killed in runs 1, 3 and 4.", + "environment": { + "boost": "1_90", + "cxx": "g++ (GCC) 15.2.0", + "numpy": "2.5.4", + "os": "Linux-7.0.0-38-generic-x86_64-with-glibc2.43", + "python": "3.14.4", + "ram_gb": 7.9, + "scipy": "1.18.1", + "swap": 0, + "sympy": "1.14.0", + "zlib": "1.3.1" + }, + "fresh_native_files": [ + { + "author_sha256": "b87f174cd72d194f23c075fd82b2fe42f5e9606e8500e4dfa23c31e8655c8de5", + "equal_after_os_byte": true, + "file": "local/CANONICAL-WORD.json.gz", + "fresh_gzip_os_byte": "0xff", + "fresh_sha256": "064c7fe3743328cdcccd781338abd823657f31ec54ff3c8b2b6535d5a3f57b04", + "fresh_with_os_byte_0x13_sha256": "b87f174cd72d194f23c075fd82b2fe42f5e9606e8500e4dfa23c31e8655c8de5", + "payload_sha256": "8566a2b7aa6f73ab827fe25dc905eefb36d77581d831457784303a3ff34252f0", + "provider_word": "735c540b1fe29d0ceeb4d2f304ec941e5f2bd3bc92bafc8cce2b4f9a67a3b653" + }, + { + "author_sha256": "735c540b1fe29d0ceeb4d2f304ec941e5f2bd3bc92bafc8cce2b4f9a67a3b653", + "equal_after_os_byte": true, + "file": "local/FULL-WORD.json.gz", + "fresh_gzip_os_byte": "0xff", + "fresh_sha256": "c857fe8140410f8afc005b7b2b92201924be09f0b75118340634f153c41daad8", + "fresh_with_os_byte_0x13_sha256": "735c540b1fe29d0ceeb4d2f304ec941e5f2bd3bc92bafc8cce2b4f9a67a3b653", + "payload_sha256": "94d33db98720483f57efcd7bf50657f29f710554474e4f6b4b90617ad4500263", + "provider_word": "735c540b1fe29d0ceeb4d2f304ec941e5f2bd3bc92bafc8cce2b4f9a67a3b653" + }, + { + "author_sha256": "86726cc4014d65848cbda3ce093cea96f9411d17e6368a77eb3523bdddb6ff7d", + "equal_after_os_byte": true, + "file": "local/LOCAL-PROGRAMS.json.gz", + "fresh_gzip_os_byte": "0xff", + "fresh_sha256": "5d85b75e7dd6b0ea8b76de83b1b07695f169b5bdad6abdf096d5fe44ea62835c", + "fresh_with_os_byte_0x13_sha256": "86726cc4014d65848cbda3ce093cea96f9411d17e6368a77eb3523bdddb6ff7d", + "payload_sha256": "b0273b99ad204e4c61a63d03c610e63c73a3348641bfac57ba1e349965b4bf21", + "provider_word": "735c540b1fe29d0ceeb4d2f304ec941e5f2bd3bc92bafc8cce2b4f9a67a3b653" + }, + { + "author_sha256": "8276f3d7f6f85bf3d678832f63baabb38550d11ed8f3841aa47020121e8e4e82", + "equal_after_os_byte": true, + "file": "local/CANONICAL-WORD.json.gz", + "fresh_gzip_os_byte": "0xff", + "fresh_sha256": "181efb82ddc286685e40ef487769be7b26094f3a4a7d029c6c574422e45558de", + "fresh_with_os_byte_0x13_sha256": "8276f3d7f6f85bf3d678832f63baabb38550d11ed8f3841aa47020121e8e4e82", + "payload_sha256": "dad6d27360457b458f30ee6aba9545e8a78097c927daf2ec142dda0283a8f19d", + "provider_word": "886338fc32da14f3389a82c8120a1bcf9c5e2f0ec3e9b2d377fdae809e1d977f" + }, + { + "author_sha256": "886338fc32da14f3389a82c8120a1bcf9c5e2f0ec3e9b2d377fdae809e1d977f", + "equal_after_os_byte": true, + "file": "local/FULL-WORD.json.gz", + "fresh_gzip_os_byte": "0xff", + "fresh_sha256": "d643252355aa21cb2cbbe6d564e880781043467c771fd13f4f8dfb1edf1bcced", + "fresh_with_os_byte_0x13_sha256": "886338fc32da14f3389a82c8120a1bcf9c5e2f0ec3e9b2d377fdae809e1d977f", + "payload_sha256": "ec4b2e15beaf99be4a96a6056fd6e92d09ea2382a4d0e8ab321f10bb2fc1dd10", + "provider_word": "886338fc32da14f3389a82c8120a1bcf9c5e2f0ec3e9b2d377fdae809e1d977f" + }, + { + "author_sha256": "86726cc4014d65848cbda3ce093cea96f9411d17e6368a77eb3523bdddb6ff7d", + "equal_after_os_byte": true, + "file": "local/LOCAL-PROGRAMS.json.gz", + "fresh_gzip_os_byte": "0xff", + "fresh_sha256": "5d85b75e7dd6b0ea8b76de83b1b07695f169b5bdad6abdf096d5fe44ea62835c", + "fresh_with_os_byte_0x13_sha256": "86726cc4014d65848cbda3ce093cea96f9411d17e6368a77eb3523bdddb6ff7d", + "payload_sha256": "b0273b99ad204e4c61a63d03c610e63c73a3348641bfac57ba1e349965b4bf21", + "provider_word": "886338fc32da14f3389a82c8120a1bcf9c5e2f0ec3e9b2d377fdae809e1d977f" + } + ], + "full_word_direct_byte_comparison": [ + { + "compared_with": "PR3633 supplier3605/STAGED644-660-base.json.gz (the pinned bytes; sha256 = cache key)", + "differing_offsets": [ + 9 + ], + "file": "735c540b1fe2/local/FULL-WORD.json.gz", + "same_length": true + }, + { + "compared_with": "PR3633 supplier3605/STAGED644-660-extra.json.gz (the pinned bytes; sha256 = cache key)", + "differing_offsets": [ + 9 + ], + "file": "886338fc32da/local/FULL-WORD.json.gz", + "same_length": true + } + ], + "gzip_os_byte_observations": { + "cpython_3_12_15_gzip_compress_mtime0": "0x3 (zlib 1.3.2)", + "cpython_3_14_4_gzip_compress_mtime0": "0xff", + "note": "Observed on this machine only. PR3605 supplier3605/staged_pair.py already restores this byte for the supplier words (gzip_os_byte 19); PR3633 pins FILL by decompressed payload; replay.py's native stage compares FULL-WORD by decompressed JSON, which is why fresh-native passed unpatched. The one gate in PR3633 that compares a rebuilt .json.gz file hash is owners/independent_literal.py (native_canonical pins).", + "pinned_files": "0x13 (zlib OS code 19)", + "zlib_1_3_1_compress_wbits31_this_machine": "0x3" + }, + "label": "LOCAL-ONLY evidence (not replayed by verify.py or CI); recorded by the package author on one machine", + "oom_note": "exit_code -9 = killed by the kernel OOM killer (dmesg). Run 1: emit at 4.33 GB RSS; run 2: parent-lift at 3.78 GB; runs 3 and 4: emit again (run 4, with no other heavy job, was killed at 5.14 GB RSS). The machine has 7.9 GB RAM and no swap; /tmp is a RAM-backed tmpfs, so runs 3 and 4 used a disk work directory.", + "runs": { + "run1": { + "discovery": [ + { + "exit_code": 0, + "log_sha256": "71f679ce5a123fb58397fd6852ba64ce3832fd20e81a096bfda9a1cf3237237a", + "name": "bind", + "seconds": 4.445759210000688 + }, + { + "exit_code": -9, + "log_sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", + "name": "emit", + "seconds": 57.94057986900043 + } + ], + "top": [ + { + "command": [ + "/root/projects/integer-mult-bound-loop/.loop-scratch/iter22/venv/bin/python", + "-B", + "/tmp/iter22-t488-release/changed/prepare_extension_native.py", + "--source", + "/tmp/iter22-t488-release/changed/source", + "--retained-cache", + "/tmp/iter22-t488-release/predecessor/native-cache", + "--retained-receipt", + "/tmp/iter22-t488-release/predecessor/NATIVE-RECEIPT.json", + "--out", + "/tmp/iter22-t488-release/changed/native", + "--jobs", + "1" + ], + "exit_code": 0, + "log_sha256": "d7581123b0f3330f2598e7758ccbca466018d2599124fc6ceb1b678a9ee27dec", + "name": "fresh-native" + }, + { + "command": [ + "/root/projects/integer-mult-bound-loop/.loop-scratch/iter22/venv/bin/python", + "-B", + "/tmp/iter22-t488-release/changed/admit_extension.py", + "--source", + "/tmp/iter22-t488-release/changed/source", + "--fill", + "/tmp/iter22-t488-release/FILL-legacy.json.gz", + "--order", + "/root/projects/integer-mult-bound-loop/.loop-scratch/iter22/t488-owners/ORDER.json", + "--dispatch", + "/root/projects/integer-mult-bound-loop/.loop-scratch/iter22/t488-owners/DISPATCH.json", + "--native-cache", + "/tmp/iter22-t488-release/changed/native/native-cache", + "--native-receipt", + "/tmp/iter22-t488-release/changed/native/NATIVE-RECEIPT.json", + "--work", + "/tmp/iter22-t488-release/admission", + "--replica-helper-dir", + "/tmp/iter22-t488-release/predecessor/helpers", + "--reconstructor", + "/tmp/iter22-t488-release/predecessor/helpers/reconstruct_replica.py", + "--comparison-claim", + "/tmp/iter22-t488-release/predecessor/PR2969-CLAIM.json", + "--blocknorm-checker", + "/tmp/iter22-t488-release/predecessor/adapters/extension_blocknorm.py", + "--source-profile", + "staged3605", + "--comparison-profile", + "pr2969", + "--jobs", + "1" + ], + "exit_code": 1, + "log_sha256": "04fec98d5ee995adcc48d39568edcddf2716d0801466bc9e25cb766231911fdd", + "name": "full-admission" + } + ] + }, + "run2": { + "admission": [ + { + "command": [ + "/root/projects/integer-mult-bound-loop/.loop-scratch/iter22/venv/bin/python", + "-B", + "/tmp/iter22-t488-release/admission/package/reconstruct.py", + "--out", + "/tmp/iter22-t488-release/admission/owners" + ], + "exit_code": 0, + "log_sha256": "8092c046470ac3a41f32f3d58b862950fcb5ad0009c13efa513ac528ac5bd5bf", + "name": "owners", + "script_sha256": "5472811c24aef3852dc7e189dff5060076a6c0b25bfde928da50dcfe9b5bdc15", + "seconds": 51.47574688300119 + }, + { + "command": [ + "/root/projects/integer-mult-bound-loop/.loop-scratch/iter22/venv/bin/python", + "-B", + "/tmp/iter22-t488-release/admission/package/lift_parent.py", + "--package", + "/tmp/iter22-t488-release/admission/package", + "--owners", + "/tmp/iter22-t488-release/admission/owners", + "--out", + "/tmp/iter22-t488-release/admission/parent-data" + ], + "exit_code": -9, + "log_sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", + "name": "parent-lift", + "script_sha256": "9a1edd7d4ab4a2f2ac8d15b42b7e40f93653d65026bac59e4c1ce80b486bd60e", + "seconds": 78.83684588100004 + } + ], + "top": [ + { + "command": [ + "/root/projects/integer-mult-bound-loop/.loop-scratch/iter22/venv/bin/python", + "-B", + "/tmp/iter22-t488-release/changed/prepare_extension_native.py", + "--source", + "/tmp/iter22-t488-release/changed/source", + "--retained-cache", + "/tmp/iter22-t488-release/predecessor/native-cache", + "--retained-receipt", + "/tmp/iter22-t488-release/predecessor/NATIVE-RECEIPT.json", + "--out", + "/tmp/iter22-t488-release/changed/native", + "--jobs", + "1" + ], + "exit_code": 0, + "log_sha256": "d7581123b0f3330f2598e7758ccbca466018d2599124fc6ceb1b678a9ee27dec", + "name": "fresh-native" + }, + { + "command": [ + "/root/projects/integer-mult-bound-loop/.loop-scratch/iter22/venv/bin/python", + "-B", + "/tmp/iter22-t488-release/changed/admit_extension.py", + "--source", + "/tmp/iter22-t488-release/changed/source", + "--fill", + "/tmp/iter22-t488-release/FILL-legacy.json.gz", + "--order", + "/root/projects/integer-mult-bound-loop/.loop-scratch/iter22/t488-owners/ORDER.json", + "--dispatch", + "/root/projects/integer-mult-bound-loop/.loop-scratch/iter22/t488-owners/DISPATCH.json", + "--native-cache", + "/tmp/iter22-t488-release/changed/native/native-cache", + "--native-receipt", + "/tmp/iter22-t488-release/changed/native/NATIVE-RECEIPT.json", + "--work", + "/tmp/iter22-t488-release/admission", + "--replica-helper-dir", + "/tmp/iter22-t488-release/predecessor/helpers", + "--reconstructor", + "/tmp/iter22-t488-release/predecessor/helpers/reconstruct_replica.py", + "--comparison-claim", + "/tmp/iter22-t488-release/predecessor/PR2969-CLAIM.json", + "--blocknorm-checker", + "/tmp/iter22-t488-release/predecessor/adapters/extension_blocknorm.py", + "--source-profile", + "staged3605", + "--comparison-profile", + "pr2969", + "--jobs", + "1" + ], + "exit_code": 1, + "log_sha256": "7ed49469dc66df08156d2110812bbf6c5bcaa5e2dad7590a31f7169488922d3d", + "name": "full-admission" + } + ] + }, + "run3": { + "admission": [ + { + "command": [ + "/root/projects/integer-mult-bound-loop/.loop-scratch/iter22/venv/bin/python", + "-B", + "/root/projects/integer-mult-bound-loop/.loop-scratch/iter22/release3/admission/package/reconstruct.py", + "--out", + "/root/projects/integer-mult-bound-loop/.loop-scratch/iter22/release3/admission/owners" + ], + "exit_code": 0, + "log_sha256": "8092c046470ac3a41f32f3d58b862950fcb5ad0009c13efa513ac528ac5bd5bf", + "name": "owners", + "script_sha256": "5472811c24aef3852dc7e189dff5060076a6c0b25bfde928da50dcfe9b5bdc15", + "seconds": 50.421530191000784 + }, + { + "command": [ + "/root/projects/integer-mult-bound-loop/.loop-scratch/iter22/venv/bin/python", + "-B", + "/root/projects/integer-mult-bound-loop/.loop-scratch/iter22/release3/admission/package/lift_parent.py", + "--package", + "/root/projects/integer-mult-bound-loop/.loop-scratch/iter22/release3/admission/package", + "--owners", + "/root/projects/integer-mult-bound-loop/.loop-scratch/iter22/release3/admission/owners", + "--out", + "/root/projects/integer-mult-bound-loop/.loop-scratch/iter22/release3/admission/parent-data" + ], + "exit_code": 0, + "log_sha256": "430d982dddcdcd6976bcfa18d5bb718bb6c3bcaa7918722464c817063563b878", + "name": "parent-lift", + "script_sha256": "9a1edd7d4ab4a2f2ac8d15b42b7e40f93653d65026bac59e4c1ce80b486bd60e", + "seconds": 90.03408756700082 + }, + { + "command": [ + "/root/projects/integer-mult-bound-loop/.loop-scratch/iter22/venv/bin/python", + "-B", + "/root/projects/integer-mult-bound-loop/.loop-scratch/iter22/release3/admission/package/new/rebind.py", + "/root/projects/integer-mult-bound-loop/.loop-scratch/iter22/release3/admission/package", + "/root/projects/integer-mult-bound-loop/.loop-scratch/iter22/release3/changed/native/native-cache/735c540b1fe29d0ceeb4d2f304ec941e5f2bd3bc92bafc8cce2b4f9a67a3b653/local", + "/root/projects/integer-mult-bound-loop/.loop-scratch/iter22/release3/changed/native/native-cache/886338fc32da14f3389a82c8120a1bcf9c5e2f0ec3e9b2d377fdae809e1d977f/local", + "/root/projects/integer-mult-bound-loop/.loop-scratch/iter22/release3/admission/rebound" + ], + "exit_code": 0, + "log_sha256": "78b7c6fb391a9266999649deca7bfd0a8c63e9542f31ca49a0ad37549352248e", + "name": "rebind", + "script_sha256": "52f8b4cec2dab69cbe590e1fd0f7a68e1b35de5a540bd22fd2adfce07a92b46b", + "seconds": 10.218786642999476 + }, + { + "command": [ + "/root/projects/integer-mult-bound-loop/.loop-scratch/iter22/venv/bin/python", + "-B", + "/root/projects/integer-mult-bound-loop/.loop-scratch/iter22/release3/admission/layout/packing74/replay.py", + "--data", + "/root/projects/integer-mult-bound-loop/.loop-scratch/iter22/release3/admission/layout/packing74/candidate", + "--work", + "/root/projects/integer-mult-bound-loop/.loop-scratch/iter22/release3/admission/discovery", + "--local", + "/root/projects/integer-mult-bound-loop/.loop-scratch/iter22/release3/changed/native/native-cache/735c540b1fe29d0ceeb4d2f304ec941e5f2bd3bc92bafc8cce2b4f9a67a3b653/local", + "--discover" + ], + "exit_code": 1, + "log_sha256": "d78e859dba2c27694238470b6ef911928f4246f2307235a33c0956976bcb7c17", + "name": "discovery", + "script_sha256": "4f5c0926b70edb5aef3af0f8a8d9fa8119fa7af6603df1721c0dc31760b08254", + "seconds": 176.21827154600032 + } + ], + "discovery": [ + { + "exit_code": 0, + "log_sha256": "71f679ce5a123fb58397fd6852ba64ce3832fd20e81a096bfda9a1cf3237237a", + "name": "bind", + "seconds": 4.449389909999809 + }, + { + "exit_code": -9, + "log_sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", + "name": "emit", + "seconds": 171.64894347299924 + } + ], + "top": [ + { + "command": [ + "/root/projects/integer-mult-bound-loop/.loop-scratch/iter22/venv/bin/python", + "-B", + "/root/projects/integer-mult-bound-loop/.loop-scratch/iter22/release3/changed/prepare_extension_native.py", + "--source", + "/root/projects/integer-mult-bound-loop/.loop-scratch/iter22/release3/changed/source", + "--retained-cache", + "/root/projects/integer-mult-bound-loop/.loop-scratch/iter22/release3/predecessor/native-cache", + "--retained-receipt", + "/root/projects/integer-mult-bound-loop/.loop-scratch/iter22/release3/predecessor/NATIVE-RECEIPT.json", + "--out", + "/root/projects/integer-mult-bound-loop/.loop-scratch/iter22/release3/changed/native", + "--jobs", + "1" + ], + "exit_code": 0, + "log_sha256": "d7581123b0f3330f2598e7758ccbca466018d2599124fc6ceb1b678a9ee27dec", + "name": "fresh-native" + }, + { + "command": [ + "/root/projects/integer-mult-bound-loop/.loop-scratch/iter22/venv/bin/python", + "-B", + "/root/projects/integer-mult-bound-loop/.loop-scratch/iter22/release3/changed/admit_extension.py", + "--source", + "/root/projects/integer-mult-bound-loop/.loop-scratch/iter22/release3/changed/source", + "--fill", + "/root/projects/integer-mult-bound-loop/.loop-scratch/iter22/release3/FILL-legacy.json.gz", + "--order", + "/root/projects/integer-mult-bound-loop/.loop-scratch/iter22/t488-owners/ORDER.json", + "--dispatch", + "/root/projects/integer-mult-bound-loop/.loop-scratch/iter22/t488-owners/DISPATCH.json", + "--native-cache", + "/root/projects/integer-mult-bound-loop/.loop-scratch/iter22/release3/changed/native/native-cache", + "--native-receipt", + "/root/projects/integer-mult-bound-loop/.loop-scratch/iter22/release3/changed/native/NATIVE-RECEIPT.json", + "--work", + "/root/projects/integer-mult-bound-loop/.loop-scratch/iter22/release3/admission", + "--replica-helper-dir", + "/root/projects/integer-mult-bound-loop/.loop-scratch/iter22/release3/predecessor/helpers", + "--reconstructor", + "/root/projects/integer-mult-bound-loop/.loop-scratch/iter22/release3/predecessor/helpers/reconstruct_replica.py", + "--comparison-claim", + "/root/projects/integer-mult-bound-loop/.loop-scratch/iter22/release3/predecessor/PR2969-CLAIM.json", + "--blocknorm-checker", + "/root/projects/integer-mult-bound-loop/.loop-scratch/iter22/release3/predecessor/adapters/extension_blocknorm.py", + "--source-profile", + "staged3605", + "--comparison-profile", + "pr2969", + "--jobs", + "1" + ], + "exit_code": 1, + "log_sha256": "e39672a8ffd0efd3766eccd095cb21dd6a236abfa348336c7b508585877fe7fe", + "name": "full-admission" + } + ] + }, + "run4": { + "admission": [ + { + "command": [ + "/root/projects/integer-mult-bound-loop/.loop-scratch/iter22/venv/bin/python", + "-B", + "/root/projects/integer-mult-bound-loop/.loop-scratch/iter22/release4/admission/package/reconstruct.py", + "--out", + "/root/projects/integer-mult-bound-loop/.loop-scratch/iter22/release4/admission/owners" + ], + "exit_code": 0, + "log_sha256": "8092c046470ac3a41f32f3d58b862950fcb5ad0009c13efa513ac528ac5bd5bf", + "name": "owners", + "script_sha256": "5472811c24aef3852dc7e189dff5060076a6c0b25bfde928da50dcfe9b5bdc15", + "seconds": 47.95485870199991 + }, + { + "command": [ + "/root/projects/integer-mult-bound-loop/.loop-scratch/iter22/venv/bin/python", + "-B", + "/root/projects/integer-mult-bound-loop/.loop-scratch/iter22/release4/admission/package/lift_parent.py", + "--package", + "/root/projects/integer-mult-bound-loop/.loop-scratch/iter22/release4/admission/package", + "--owners", + "/root/projects/integer-mult-bound-loop/.loop-scratch/iter22/release4/admission/owners", + "--out", + "/root/projects/integer-mult-bound-loop/.loop-scratch/iter22/release4/admission/parent-data" + ], + "exit_code": 0, + "log_sha256": "430d982dddcdcd6976bcfa18d5bb718bb6c3bcaa7918722464c817063563b878", + "name": "parent-lift", + "script_sha256": "9a1edd7d4ab4a2f2ac8d15b42b7e40f93653d65026bac59e4c1ce80b486bd60e", + "seconds": 89.78357079899979 + }, + { + "command": [ + "/root/projects/integer-mult-bound-loop/.loop-scratch/iter22/venv/bin/python", + "-B", + "/root/projects/integer-mult-bound-loop/.loop-scratch/iter22/release4/admission/package/new/rebind.py", + "/root/projects/integer-mult-bound-loop/.loop-scratch/iter22/release4/admission/package", + "/root/projects/integer-mult-bound-loop/.loop-scratch/iter22/release4/changed/native/native-cache/735c540b1fe29d0ceeb4d2f304ec941e5f2bd3bc92bafc8cce2b4f9a67a3b653/local", + "/root/projects/integer-mult-bound-loop/.loop-scratch/iter22/release4/changed/native/native-cache/886338fc32da14f3389a82c8120a1bcf9c5e2f0ec3e9b2d377fdae809e1d977f/local", + "/root/projects/integer-mult-bound-loop/.loop-scratch/iter22/release4/admission/rebound" + ], + "exit_code": 0, + "log_sha256": "78b7c6fb391a9266999649deca7bfd0a8c63e9542f31ca49a0ad37549352248e", + "name": "rebind", + "script_sha256": "52f8b4cec2dab69cbe590e1fd0f7a68e1b35de5a540bd22fd2adfce07a92b46b", + "seconds": 10.070241393001197 + }, + { + "command": [ + "/root/projects/integer-mult-bound-loop/.loop-scratch/iter22/venv/bin/python", + "-B", + "/root/projects/integer-mult-bound-loop/.loop-scratch/iter22/release4/admission/layout/packing74/replay.py", + "--data", + "/root/projects/integer-mult-bound-loop/.loop-scratch/iter22/release4/admission/layout/packing74/candidate", + "--work", + "/root/projects/integer-mult-bound-loop/.loop-scratch/iter22/release4/admission/discovery", + "--local", + "/root/projects/integer-mult-bound-loop/.loop-scratch/iter22/release4/changed/native/native-cache/735c540b1fe29d0ceeb4d2f304ec941e5f2bd3bc92bafc8cce2b4f9a67a3b653/local", + "--discover" + ], + "exit_code": 1, + "log_sha256": "20b45a78b99eae861ab36f4e1df6a55e10be344e2140f7574626dd5dfc1bca25", + "name": "discovery", + "script_sha256": "4f5c0926b70edb5aef3af0f8a8d9fa8119fa7af6603df1721c0dc31760b08254", + "seconds": 143.2647706050011 + } + ], + "discovery": [ + { + "exit_code": 0, + "log_sha256": "71f679ce5a123fb58397fd6852ba64ce3832fd20e81a096bfda9a1cf3237237a", + "name": "bind", + "seconds": 4.548172221000641 + }, + { + "exit_code": -9, + "log_sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", + "name": "emit", + "seconds": 138.62347031899844 + } + ], + "note": "run with no other heavy job (only interactive agent processes, about 1.5 GB); emit still killed by the OOM killer", + "top": [ + { + "command": [ + "/root/projects/integer-mult-bound-loop/.loop-scratch/iter22/venv/bin/python", + "-B", + "/root/projects/integer-mult-bound-loop/.loop-scratch/iter22/release4/changed/prepare_extension_native.py", + "--source", + "/root/projects/integer-mult-bound-loop/.loop-scratch/iter22/release4/changed/source", + "--retained-cache", + "/root/projects/integer-mult-bound-loop/.loop-scratch/iter22/release4/predecessor/native-cache", + "--retained-receipt", + "/root/projects/integer-mult-bound-loop/.loop-scratch/iter22/release4/predecessor/NATIVE-RECEIPT.json", + "--out", + "/root/projects/integer-mult-bound-loop/.loop-scratch/iter22/release4/changed/native", + "--jobs", + "1" + ], + "exit_code": 0, + "log_sha256": "d7581123b0f3330f2598e7758ccbca466018d2599124fc6ceb1b678a9ee27dec", + "name": "fresh-native" + }, + { + "command": [ + "/root/projects/integer-mult-bound-loop/.loop-scratch/iter22/venv/bin/python", + "-B", + "/root/projects/integer-mult-bound-loop/.loop-scratch/iter22/release4/changed/admit_extension.py", + "--source", + "/root/projects/integer-mult-bound-loop/.loop-scratch/iter22/release4/changed/source", + "--fill", + "/root/projects/integer-mult-bound-loop/.loop-scratch/iter22/release4/FILL-legacy.json.gz", + "--order", + "/root/projects/integer-mult-bound-loop/.loop-scratch/iter22/t488-owners/ORDER.json", + "--dispatch", + "/root/projects/integer-mult-bound-loop/.loop-scratch/iter22/t488-owners/DISPATCH.json", + "--native-cache", + "/root/projects/integer-mult-bound-loop/.loop-scratch/iter22/release4/changed/native/native-cache", + "--native-receipt", + "/root/projects/integer-mult-bound-loop/.loop-scratch/iter22/release4/changed/native/NATIVE-RECEIPT.json", + "--work", + "/root/projects/integer-mult-bound-loop/.loop-scratch/iter22/release4/admission", + "--replica-helper-dir", + "/root/projects/integer-mult-bound-loop/.loop-scratch/iter22/release4/predecessor/helpers", + "--reconstructor", + "/root/projects/integer-mult-bound-loop/.loop-scratch/iter22/release4/predecessor/helpers/reconstruct_replica.py", + "--comparison-claim", + "/root/projects/integer-mult-bound-loop/.loop-scratch/iter22/release4/predecessor/PR2969-CLAIM.json", + "--blocknorm-checker", + "/root/projects/integer-mult-bound-loop/.loop-scratch/iter22/release4/predecessor/adapters/extension_blocknorm.py", + "--source-profile", + "staged3605", + "--comparison-profile", + "pr2969", + "--jobs", + "1" + ], + "exit_code": 1, + "log_sha256": "45373e5ada2fad8eb1093e1bbcbe48af62464913a6df7d464257d81d921c10cd", + "name": "full-admission" + } + ] + } + } +} \ No newline at end of file diff --git a/research/t488-independent-audit/tools/encode_journal.py b/research/t488-independent-audit/tools/encode_journal.py new file mode 100644 index 0000000000..81b33ed818 --- /dev/null +++ b/research/t488-independent-audit/tools/encode_journal.py @@ -0,0 +1,34 @@ +#!/usr/bin/env python3 +# SPDX-License-Identifier: Apache-2.0 +"""Encode PR #3633's literal T488 owner journal (FILL.json.gz from PR3633's deterministic expander, +owners/portable_expand.py) as data/FILL-columns.xz: the owner_replica (uint16), child, port and offset (uint8) +columns, little-endian, concatenated, xz-compressed. time and helper are implicit (row i is (i // 408, i % 408)). +verify.py decodes it and re-serialises the rows in PR3633's documented format; the result must hash to PR3633's +pinned payload sha256 (owners/EXPECTED.json), so this encoding cannot change the audited object. + +Usage: python3 -I -B tools/encode_journal.py PATH/TO/FILL.json.gz data/FILL-columns.xz +Standard library only. +""" +import gzip +import json +import lzma +import sys +from array import array + + +def main(src, dst): + rows = json.loads(gzip.decompress(open(src, 'rb').read()))['rows'] + for i, r in enumerate(rows): + assert r[0] == i // 408 and r[1] == i % 408, 'rows are not in (time, helper) order' + assert 0 <= r[2] < 65536 and all(0 <= v < 256 for v in r[3:]), 'value outside the column type' + cols = [array('H', (r[2] for r in rows))] + [array('B', (r[k] for r in rows)) for k in (3, 4, 5)] + if sys.byteorder != 'little': + cols[0].byteswap() + blob = b''.join(c.tobytes() for c in cols) + open(dst, 'wb').write(lzma.compress(blob, format=lzma.FORMAT_XZ, check=lzma.CHECK_SHA256, + preset=9 | lzma.PRESET_EXTREME)) + print('ENCODED', len(rows), 'rows ->', dst) + + +if __name__ == '__main__': + main(sys.argv[1], sys.argv[2]) diff --git a/research/t488-independent-audit/tools/gate_disable_demo.py b/research/t488-independent-audit/tools/gate_disable_demo.py new file mode 100644 index 0000000000..a9990311b0 --- /dev/null +++ b/research/t488-independent-audit/tools/gate_disable_demo.py @@ -0,0 +1,72 @@ +#!/usr/bin/env python3 +# SPDX-License-Identifier: Apache-2.0 +"""Local-only demonstration for finding 1 (not run by verify.py or CI; it executes PR #3633 code as the object under +study). Runs PR3633's owners/independent_literal.py unmodified, then copies in which one gate (and finally all five +gates) named by the five reject() controls is replaced by need(True, ...). The only other change in a copy is its +ARTIFACTS.json self-pin. Records whether each copy still reports PASS and still lists its five controls. + +Usage: python3 tools/gate_disable_demo.py OWNERS_DIR LITERAL_DIR NATIVE_DIR PYTHON REPO WORK_DIR > DEMO.json + OWNERS_DIR PR3633 research/staged-width-t488/owners (e.g. from `git archive` of head 72c268f1) + LITERAL_DIR output of PR3633 owners/portable_expand.py (FILL.json.gz, ORDER.json, DISPATCH.json, PROFILE.json) + NATIVE_DIR checked-base/ and checked-extra/CANONICAL-WORD.json.gz whose sha256 equal PR3633 SOURCE-PINS + PYTHON an interpreter with numpy (PR3633's auditor imports it) + REPO a checkout of main (OWNER_SOURCE_REPO) + WORK_DIR a new directory for the copies +""" +import json +import os +import re +import shutil +import subprocess +import sys +from hashlib import sha256 +from pathlib import Path + +GATES = {'helper_identity': 'complete literal ordered helper identities', + 'window': 'strict/closed actual source windows', + 'family': 'same-call physical family uniqueness', + 'prefix': 'every chronological pending mask is a compact prefix', + 'histogram': 'independent complete histogram'} + + +def variant(src, work, name, disabled, env, python): + d = work / ('run-' + name) + shutil.copytree(src, d) + text = (d / 'independent_literal.py').read_text() + for g in disabled: + msg = GATES[g] + hits = list(re.finditer(r"need\((?:(?!need\().)*?,'" + re.escape(msg) + r"'\)", text, re.S)) + if len(hits) != 1: + raise SystemExit('gate %r found %d times' % (g, len(hits))) + text = text[:hits[0].start()] + "need(True,'" + msg + "')" + text[hits[0].end():] + (d / 'independent_literal.py').write_text(text) + art = json.loads((d / 'ARTIFACTS.json').read_text()) + art['independent_literal.py'] = sha256((d / 'independent_literal.py').read_bytes()).hexdigest() + (d / 'ARTIFACTS.json').write_text(json.dumps(art, indent=2) + '\n') + out = d / 'AUDIT.json' + r = subprocess.run([python, '-B', str(d / 'independent_literal.py')], env=dict(env, OWNER_AUDIT_OUTPUT=str(out)), + capture_output=True, text=True) + rec = dict(variant=name, disabled=disabled, exit=r.returncode) + if r.returncode == 0: + res = json.loads(out.read_text()) + rec.update(status=res['status'], controls=res['controls']) + else: + rec['stderr_tail'] = r.stderr[-300:] + return rec + + +def main(owners, literal, native, python, repo, work): + work = Path(work) + work.mkdir() + env = dict((k, v) for k, v in os.environ.items() if not k.startswith('GIT_')) + env.update(OWNER_SOURCE_REPO=str(Path(repo).resolve()), OWNER_LITERAL_DIR=str(Path(literal).resolve()), + OWNER_NATIVE_DIR=str(Path(native).resolve())) + src = Path(owners).resolve() + recs = [variant(src, work, 'original', [], env, python)] + recs += [variant(src, work, 'no-' + g, [g], env, python) for g in GATES] + recs.append(variant(src, work, 'no-gates', list(GATES), env, python)) + print(json.dumps(recs, indent=1)) + + +if __name__ == '__main__': + main(*sys.argv[1:7]) diff --git a/research/t488-independent-audit/vendor/NOTICE.md b/research/t488-independent-audit/vendor/NOTICE.md new file mode 100644 index 0000000000..eac3fd03e1 --- /dev/null +++ b/research/t488-independent-audit/vendor/NOTICE.md @@ -0,0 +1,34 @@ +# Vendored files + +Every file below is an earlier checker by the author of this package (Sagar Thakrar / thakrarsagar), Apache-2.0, +prepared with substantial AI assistance (Claude Code, Anthropic). Each is a byte-for-byte copy of the file of the same name +on main at `20ecf2891d80b5acf18d980b2164fb44ea7faa90` (paths below). `verify.py` checks each copy's git blob id and SHA-256 +and executes exactly the verified bytes as the module, so no cached `.pyc` can stand in. No code of PR #3633, #3630, #3605, +#3624, #3626 or #3653 is vendored, imported or executed by `verify.py`. + +| File | Module | Git blob | SHA-256 | Copied from (main) | Origin | +| --- | --- | --- | --- | --- | --- | +| `vendor/pr3053/audit_common.py` | `audit_common` | `17c4064f01b110012b54c05e64bdee8ae498e0b2` | `7ced740252ac8d8a8c66ef3cc825008e0aacf080494d0e983fb322d7dcddbf42` | `research/t74-3015-restore-edit-audit/vendor/pr2804/audit_common.py` | PR #2804 (merged): shared rule/attempt bookkeeping, pinned git inputs | +| `vendor/pr3053/words_check.py` | `words_check` | `0b646b9d5583c75a78e0c2dd7e9cfffa02055058` | `b66414a39f5ac7b9cd32284027e13a550734901aca5f4457085a034689bc3ce6` | `research/t74-3015-restore-edit-audit/vendor/pr2315/words_check.py` | PR #2315 (merged): free-operator word parser, warp algebra and forward/dual replay | +| `vendor/pr3053/fill_check80.py` | `fill_check80` | `557e0732bfe4c030436a11f5433143c10fa07821` | `1c4d30591969c6502605f239f40505261e99d025aa3b576a804332a6bc479b06` | `research/t74-3015-restore-edit-audit/vendor/pr2387/fill_check80.py` | PR #2387 (merged): T74 owner-row rules (imported by audit_lib/check_accounting; not used directly here) | +| `vendor/pr3053/check_words.py` | `check_words` | `451dbb82f147ef446a1819b1ed595b4bf5547583` | `5996e804fa006906abff63a982c272588a44c35dfc932fdf687ef98dd3e0905f` | `research/t74-3015-restore-edit-audit/vendor/pr2804/check_words.py` | PR #2804: extended parser, literal frame recount, replays, cleanup re-derivation | +| `vendor/pr3053/audit_lib.py` | `audit_lib` | `e8e847cc89dae6bb4e90e2c3dbebc30f615b698f` | `e0cf84dd959d39e3f299c1e171753c29465fd5b4457561f4ab9ec0675d8e9dc1` | `research/t74-3015-restore-edit-audit/vendor/pr2865/audit_lib.py` | PR #2865 (merged): parse with initial corrections, own exact tracker, negative-control harness expect() | +| `vendor/pr3053/audit_kappa.py` | `audit_kappa` | `6e935547c98188a5fd2ba952641bf474d9323d9b` | `8dd12fac27ba3ee7decf99a7a1e410e138ec0fc34805bb28b93fa5cc843ab0ee` | `research/t74-3015-restore-edit-audit/audit_kappa.py` | PR #3053 (merged): rational interval moment enclosure (atanh/Taylor series with explicit tails) | +| `vendor/pr3053/audit_words.py` | `audit_words` | `6368333e16c4f50e6eb8b5f2f4c178a2de8cbbb9` | `ff831ad648dbecfd1c80f63245a84b762fdc70eb9fc3aa94dba23cd364857d18` | `research/t74-3015-restore-edit-audit/audit_words.py` | PR #3053: the word pipeline (counts, replays, tracker, cleanup, buffers) | +| `vendor/pr3053/check_accounting.py` | `check_accounting` | `e749057dd838b715d0630de618f32d66a38da931` | `fb6aef50da586f15cd60762edc074c20e300d65f1378a80160b3d4b840762548` | `research/t74-3015-restore-edit-audit/vendor/pr2804/check_accounting.py` | PR #2804: 640-bit fixed-point moment enclosure (second engine) | + +## Predecessor credits + +Carried over from the files' docstrings and from `research/t74-3015-restore-edit-audit/vendor/NOTICE.md` on main: + +- `words_check.py`: PR2269 (Jeremy Kothe, jeremykothe2): the borrow43 package and words; PR2226 (Cinna Davis / + 1337isnot1337, with OpenAI Codex): anticipatory dirty borrowing, the alias rule and the free-operator replay idea; + PR2260 (Dugongue); the T74 lineage PR1319/PR1297 (Rohan Arun) and PR1625 (mdumitrean); **icekylinx and Zhihao Chen + (jacklightChen)** (two contributors): Clifford-frame normal forms and the local-word checker lineage. The pinned bytes of + `words_check.py` and `fill_check80.py` carry the conflated form "icekylinx (Zhihao Chen)" in their docstrings, which the + maintainer corrected on main when integrating PR #2315; the correct attribution is the one here. +- `fill_check80.py`: PR2269 (Jeremy Kothe): the T74 template, windows, dispatch and owner-row format; PR2322 (Vlado + Vojdanovski); PR2226 (Cinna Davis); the PR858 / complex-double-replica-owners hosting rules (Hao Yan); the T16 owner audit + (Josie Elliston; icekylinx and Zhihao Chen (jacklightChen) checker lineage). +- `check_accounting.py`, `check_words.py`, `audit_common.py` (PR #2804), `audit_lib.py` (PR #2865), `audit_kappa.py` and + `audit_words.py` (PR #3053): this contributor's own, built on the above. diff --git a/research/t488-independent-audit/vendor/pr3053/audit_common.py b/research/t488-independent-audit/vendor/pr3053/audit_common.py new file mode 100644 index 0000000000..17c4064f01 --- /dev/null +++ b/research/t488-independent-audit/vendor/pr3053/audit_common.py @@ -0,0 +1,216 @@ +# SPDX-License-Identifier: Apache-2.0 +"""Shared helpers for the PR #2530 audit: one failure type and the pinned-input reader. + +Every PR input is read with `git -C show :` at a pinned commit and must match a pinned git +blob id (and, where listed, a sha256) before it is parsed; a pinned tree is listed by reading and re-hashing every +tree object below it (`git cat-file tree`, recursively), so a listing is accepted only if it hashes to the pinned +tree id. PR content is data only: it is decoded as JSON, gzip, tar or text and never imported or executed. + +Git environment (git_env): the repository-redirecting variables (GIT_DIR, GIT_WORK_TREE, GIT_INDEX_FILE, +GIT_OBJECT_DIRECTORY, GIT_ALTERNATE_OBJECT_DIRECTORIES, GIT_NAMESPACE, GIT_CEILING_DIRECTORIES, GIT_COMMON_DIR) and +the command-line config injections (GIT_CONFIG_PARAMETERS, GIT_CONFIG_COUNT, GIT_CONFIG_KEY_*, GIT_CONFIG_VALUE_*) +are removed, GIT_NO_REPLACE_OBJECTS=1 is set, so every git call reads exactly the --repo repository's own objects, +and GIT_NO_LAZY_FETCH=1 is set, so that in a partial clone `git show` fails on a missing object instead of fetching +it (honoured by recent git; older git ignores it). The reader is read-only. The only write is the documented --fetch (ensure_commits), which runs +`git fetch` with --no-write-fetch-head, --no-tags, --no-recurse-submodules, --no-prune, an empty --refmap and +-c gc.auto=0 -c maintenance.auto=false -c fetch.writeCommitGraph=false -c core.logAllRefUpdates=false +-c core.hooksPath=/dev/null, so it writes only the fetched objects and the namespaced ref refs/audit/pr (no +FETCH_HEAD, no other ref, no reflog, no commit-graph, no gc or maintenance, no hook); it needs git >= 2.29 +(--no-write-fetch-head); a failed fetch is reported with git's own error message. + +Control bookkeeping: every negative-control helper of the scopes calls attempt() before it runs its mutant, and every +positive control calls positive() after it has passed, so verify.py prints attempted and rejected negative +controls from separate counters and the positive controls actually executed. +Standard library only; Python 3.8-compatible syntax. +""" +import gzip +import hashlib +import json +import os +import subprocess + + +class AuditFail(Exception): + """A failed audit check (the message names the check).""" + + +def need(ok, msg): + if not ok: + raise AuditFail(msg) + + +# Negative controls attempted (names, in order) and positive controls executed (ids, in order); see reset_controls. +ATTEMPTED, POSITIVES = [], [] + + +def reset_controls(): + del ATTEMPTED[:] + del POSITIVES[:] + + +def attempt(name): + """Called by every negative-control helper before it runs its mutant.""" + ATTEMPTED.append(name) + + +def positive(pid): + """Called after a positive control has passed.""" + POSITIVES.append(pid) + + +def git_blob_id(data): + return hashlib.sha1(b'blob %d\0' % len(data) + data).hexdigest() + + +def sha256(data): + return hashlib.sha256(data).hexdigest() + + +# Environment variables that would make git read or write a repository other than `git -C `. +GIT_REDIRECT_VARS = ('GIT_DIR', 'GIT_WORK_TREE', 'GIT_INDEX_FILE', 'GIT_OBJECT_DIRECTORY', + 'GIT_ALTERNATE_OBJECT_DIRECTORIES', 'GIT_NAMESPACE', 'GIT_CEILING_DIRECTORIES', 'GIT_COMMON_DIR') +# Environment variables that inject configuration as if given with `git -c` (prefixes end with '_'). +GIT_CONFIG_INJECTION_VARS = ('GIT_CONFIG_PARAMETERS', 'GIT_CONFIG_COUNT') +GIT_CONFIG_INJECTION_PREFIXES = ('GIT_CONFIG_KEY_', 'GIT_CONFIG_VALUE_') +# `git -c` settings and options of the one writing call (--fetch): objects and refs/audit/pr only. +FETCH_CONFIG = ('gc.auto=0', 'maintenance.auto=false', 'fetch.writeCommitGraph=false', 'core.logAllRefUpdates=false', + 'core.hooksPath=/dev/null') +FETCH_OPTIONS = ('-q', '--no-tags', '--no-write-fetch-head', '--no-recurse-submodules', '--no-prune', '--refmap=') + + +def git_env(): + """A copy of os.environ without GIT_REDIRECT_VARS and the config-injection variables, with + GIT_NO_REPLACE_OBJECTS=1 (refs/replace is ignored) and GIT_NO_LAZY_FETCH=1 (no on-demand fetch of missing + objects in a partial clone).""" + env = dict(os.environ) + for name in list(env): + if name in GIT_REDIRECT_VARS or name in GIT_CONFIG_INJECTION_VARS or name.startswith(GIT_CONFIG_INJECTION_PREFIXES): + del env[name] + env['GIT_NO_REPLACE_OBJECTS'] = '1' + env['GIT_NO_LAZY_FETCH'] = '1' + return env + + +def _git(repo, args, config=()): + cmd = ['git'] + for kv in config: + cmd += ['-c', kv] + return subprocess.run(cmd + ['-C', repo] + list(args), stdout=subprocess.PIPE, stderr=subprocess.PIPE, + env=git_env()) + + +def commit_present(repo, commit): + return _git(repo, ['cat-file', '-e', commit + '^{commit}']).returncode == 0 + + +def audit_ref(pr): + return 'refs/audit/pr%d' % pr + + +def anchored(repo, commit, ref): + """commit is present and reachable from ref (so git gc keeps it).""" + return commit_present(repo, commit) and _git(repo, ['merge-base', '--is-ancestor', commit, ref]).returncode == 0 + + +def ensure_commits(repo, commits, fetch, remote): + """commits: {pr_number: sha}. Without fetch a missing commit is an error and nothing is written. With fetch, + a missing commit is fetched from the remote's refs/pull/N/head and, if the PR head has moved past it, by its + sha, each time into the namespaced ref refs/audit/prN of the target repository (forced update), so that the + fetched commit stays reachable from a ref and survives git gc; it must then be present and reachable from + refs/audit/prN. The fetch writes the fetched objects and refs/audit/prN only (FETCH_CONFIG, FETCH_OPTIONS; + git >= 2.29). If the commit is still missing, the failure message carries the last fetch's exit code and stderr + (for example an unknown option on older git, or a network error). Returns the PR numbers fetched.""" + need(_git(repo, ['rev-parse', '--git-dir']).returncode == 0, '%s is not a git repository (use --repo)' % repo) + fetched = [] + for pr in sorted(commits): + sha = commits[pr] + if commit_present(repo, sha): + continue + need(fetch, 'commit %s of PR #%d is absent from %s; rerun with --fetch' % (sha[:12], pr, repo)) + ref = audit_ref(pr) + last = None + for src in ('refs/pull/%d/head' % pr, sha): + last = _git(repo, ['fetch'] + list(FETCH_OPTIONS) + [remote, '+%s:%s' % (src, ref)], FETCH_CONFIG) + if anchored(repo, sha, ref): + fetched.append(pr) + break + need(anchored(repo, sha, ref), 'could not fetch commit %s of PR #%d from %s into %s (last git fetch: exit %d, ' + 'stderr %r)' % (sha[:12], pr, remote, ref, last.returncode, last.stderr.decode('utf-8', 'replace')[-400:])) + return fetched + + +class PinnedInputs(object): + """Reader for the pinned input table {key: (pr, commit, path, git_blob, sha256 or None)}. + + raw() returns the verified bytes; json() the parsed JSON (gunzipped for *.gz). Parsed objects are cached and + shared, so callers must treat them as read-only (controls deep-copy before mutating).""" + + def __init__(self, repo, table): + self.repo, self.table = repo, table + self._raw, self._json, self.used = {}, {}, {} + + def raw(self, key): + if key not in self._raw: + pr, commit, path, blob, sha = self.table[key] + p = _git(self.repo, ['rev-parse', '--verify', '-q', commit + ':' + path]) + need(p.returncode == 0, 'input %s: %s:%s not found' % (key, commit[:12], path)) + need(p.stdout.decode().strip() == blob, 'input %s: tree entry is not the pinned blob %s' % (key, blob)) + p = _git(self.repo, ['show', commit + ':' + path]) + need(p.returncode == 0, 'input %s: git show failed: %s' % (key, p.stderr.decode()[:200])) + data = p.stdout + need(git_blob_id(data) == blob, 'input %s: git blob id mismatch' % key) + digest = sha256(data) + need(sha is None or digest == sha, 'input %s: sha256 mismatch' % key) + self._raw[key] = data + self.used[key] = dict(pr=pr, commit=commit, path=path, git_blob=blob, sha256=digest, bytes=len(data)) + return self._raw[key] + + def json(self, key): + if key not in self._json: + data = self.raw(key) + if self.table[key][2].endswith('.gz'): + data = gzip.decompress(data) + self._json[key] = json.loads(data.decode('utf-8')) + return self._json[key] + + def text(self, key): + return self.raw(key).decode('utf-8') + + def show_blob(self, commit, path, blob): + """Bytes of commit:path for a file listed by tree(); the blob id is re-verified.""" + p = _git(self.repo, ['show', commit + ':' + path]) + need(p.returncode == 0 and git_blob_id(p.stdout) == blob, 'git show %s:%s is not blob %s' % (commit[:12], path, blob)) + return p.stdout + + def _tree_object(self, tree_id): + """Raw content of a tree object, re-hashed against its id (fail-closed listing).""" + p = _git(self.repo, ['cat-file', 'tree', tree_id]) + need(p.returncode == 0, 'git cat-file tree %s failed' % tree_id) + data = p.stdout + need(hashlib.sha1(b'tree %d\0' % len(data) + data).hexdigest() == tree_id, 'tree object %s does not hash to its id' + % tree_id) + return data + + def _tree_files(self, tree_id, prefix, out, key): + data, i = self._tree_object(tree_id), 0 + while i < len(data): + sp, nul = data.index(b' ', i), data.index(b'\0', i) + mode, name, oid = data[i:sp].decode(), data[sp + 1:nul].decode('utf-8'), data[nul + 1:nul + 21].hex() + i = nul + 21 + if mode == '40000': + self._tree_files(oid, prefix + name + '/', out, key) + else: + need(mode in ('100644', '100755'), 'tree %s: non-regular entry %r (mode %s)' % (key, prefix + name, mode)) + out[prefix + name] = oid + return out + + def tree(self, key): + """A pinned tree entry {key: (pr, commit, path, tree_id, None)}: returns {relative path: blob id} of every + regular file below it, after checking that commit:path is the pinned tree object; every tree object read is + re-hashed against its id.""" + pr, commit, path, tree_id, _ = self.table[key] + p = _git(self.repo, ['rev-parse', '--verify', '-q', commit + ':' + path]) + need(p.returncode == 0 and p.stdout.decode().strip() == tree_id, 'tree %s is not the pinned tree %s' % (key, tree_id)) + out = self._tree_files(tree_id, '', {}, key) + self.used[key] = dict(pr=pr, commit=commit, path=path, git_tree=tree_id, files=len(out)) + return out diff --git a/research/t488-independent-audit/vendor/pr3053/audit_kappa.py b/research/t488-independent-audit/vendor/pr3053/audit_kappa.py new file mode 100644 index 0000000000..6e935547c9 --- /dev/null +++ b/research/t488-independent-audit/vendor/pr3053/audit_kappa.py @@ -0,0 +1,202 @@ +# SPDX-License-Identifier: Apache-2.0 +"""Scope D: the complex endpoint and kappa of PR #3015 from its complete profile, with an own rational-interval +moment enclosure for every rank (cross-checked against the vendored check_accounting.moment of PR2804 on the ranks +that one tabulates), the exact gain over PR #2979, the improvement test, the pricing of profile deltas (census), and +the PR #3018 composition prediction. + +M(a) = (1/(73 W)) sum_r n_r r (73/r)^a = sum_r n_r r exp(a (ln 73 - ln r)) / (73 W), increasing in a; the complex +endpoint C is the largest 24-digit decimal with M(C) < 1 (and M(C + 1e-24) > 1); kappa = C - 1e-46 - 1e-70. +ln r (r >= 1): r = 2^k x with 1 <= x < 2, ln x = 2 atanh((x-1)/(x+1)) by its series with the geometric tail, ln 2 = +2 atanh(1/3); exp(y) for 0 <= y < 1 by its Taylor series with the tail bound 2 y^N / N!; every bound is rounded +outward to the grid 10^-60. Standard library only; Python 3.8-compatible syntax. +""" +from fractions import Fraction as Q + +import audit_lib as AL +from audit_common import need + +M = 73 +W = 66304 +CALLS = 666 +E24 = Q(1, 10 ** 24) +KAPPA_TERMS = (Q(1, 10 ** 46), Q(1, 10 ** 70)) +GRID = 10 ** 60 +_LN = {} + + +def floor_g(x): + return Q((x * GRID).__floor__(), GRID) + + +def ceil_g(x): + return Q(-((-x * GRID).__floor__()), GRID) + + +def atanh_iv(z, N=80): + """0 <= z < 1: (partial sum, partial sum + tail) of atanh(z) = sum z^(2j+1)/(2j+1).""" + need(0 <= z < 1, 'kappa.ln: atanh argument') + s, zp, z2 = Q(0), z, z * z + for j in range(N): + s += zp / (2 * j + 1) + zp *= z2 + return s, s + zp / (2 * N + 1) / (1 - z2) + + +def ln_interval(r): + """Enclosure of ln r for an integer r >= 1 (cached).""" + if r in _LN: + return _LN[r] + need(isinstance(r, int) and r >= 1, 'kappa.ln: rank') + k, x = 0, Q(r) + while x >= 2: + x /= 2 + k += 1 + l2 = atanh_iv(Q(1, 3)) + lx = atanh_iv((x - 1) / (x + 1)) + lo = floor_g(2 * (k * l2[0] + lx[0])) + hi = ceil_g(2 * (k * l2[1] + lx[1])) + _LN[r] = (lo, hi) + return _LN[r] + + +def exp_interval(y_lo, y_hi, N=40): + """Enclosure of exp over [y_lo, y_hi], 0 <= y_lo <= y_hi < 1.""" + need(0 <= y_lo <= y_hi < 1, 'kappa.exp: argument outside [0, 1)') + out = [] + for y, up in ((y_lo, False), (y_hi, True)): + t = s = Q(1) + for j in range(1, N): + t = t * y / j + s += t + if up: + s += 2 * t * y / N # tail bound sum_{j>=N} y^j/j! <= 2 y^N/N! for y <= (N+1)/2 + out.append(ceil_g(s)) + else: + out.append(floor_g(s)) + return out[0], out[1] + + +def H(r, a): + """Enclosure of r (73/r)^a.""" + l73, lr = ln_interval(M), ln_interval(r) + L_lo, L_hi = l73[0] - lr[1], l73[1] - lr[0] + need(L_lo > 0, 'kappa.rank: rank %d not below 73' % r) + e_lo, e_hi = exp_interval(a * L_lo, a * L_hi) + return r * e_lo, r * e_hi + + +def moment(profile, a, live=W): + """Enclosure of M(a) for a complete profile {rank: count} (counts >= 0).""" + lo = hi = Q(0) + for r, n in profile.items(): + if n == 0: + continue + need(n > 0 and 0 < int(r) < M, 'kappa.profile: rank %r count %r' % (r, n)) + h_lo, h_hi = H(int(r), a) + lo += n * h_lo + hi += n * h_hi + return floor_g(lo / (M * live)), ceil_g(hi / (M * live)) + + +def accepts(enc): + return enc[1] < 1 + + +def rejects(enc): + return enc[0] > 1 + + +def endpoint_format(C): + need((C * 10 ** 24).denominator == 1 and C > 0, 'kappa.endpoint_format: C is not a positive 24-digit decimal') + + +def endpoint_check(prof, C, mom=moment): + """C accepted (upper end of the enclosure of M(C) below 1) and C + 1e-24 rejected (lower end above 1).""" + lo, hi = mom(prof, C) + need(accepts((lo, hi)), 'kappa.endpoint_accept: M(C) enclosure [%.3e, %.3e] - 1 not below 0' + % (float(lo - 1), float(hi - 1))) + lo2, hi2 = mom(prof, C + E24) + need(rejects((lo2, hi2)), 'kappa.endpoint_reject: M(C + 1e-24) enclosure [%.3e, %.3e] - 1 not above 0' + % (float(lo2 - 1), float(hi2 - 1))) + return dict(C_moment_minus_1=['%.9e' % float(lo - 1), '%.9e' % float(hi - 1)], + C_plus_1e24_moment_minus_1=['%.9e' % float(lo2 - 1), '%.9e' % float(hi2 - 1)], + C_enclosure_width='%.2e' % float(hi - lo)) + + +def two_impl(prof, a, vendored_moment): + """kappa.two_impl: the own enclosure and the vendored one (PR2804's check_accounting.moment) at a intersect, and + neither accepts what the other rejects.""" + own, ven = moment(prof, a), vendored_moment(prof, a) + need(max(own[0], ven[0]) <= min(own[1], ven[1]), 'kappa.two_impl: enclosures %r and %r are disjoint' % (own, ven)) + need(accepts(own) == accepts(ven) and rejects(own) == rejects(ven), 'kappa.two_impl: verdicts differ at %s' % a) + return dict(own=['%.3e' % float(own[0] - 1), '%.3e' % float(own[1] - 1)], + vendored=['%.3e' % float(ven[0] - 1), '%.3e' % float(ven[1] - 1)], intersect=True) + + +def kappa_check(kappa, C): + need(kappa == C - KAPPA_TERMS[0] - KAPPA_TERMS[1], 'kappa.kappa: CLAIM kappa != C - 1e-46 - 1e-70') + + +def gain_check(kappa_new, kappa_old, claimed_gain): + gain = kappa_new - kappa_old + need(gain > 0 and gain == claimed_gain, 'kappa.gain: kappa gain %s != claimed %s' % (gain, claimed_gain)) + return dict(gain=str(gain), gain_float='%.6e' % float(gain)) + + +def improvement(prof_old, prof_new, C_new, C_old): + need(rejects(moment(prof_old, C_new)) and accepts(moment(prof_new, C_old)), + 'kappa.improvement: the old profile is not rejected at the new C (or the new profile not accepted at the old C)') + + +def straddle(prof, C, mom=moment): + """Bisection on [C, C + 1e-24] until the enclosure of M straddles 1: (lo_a accepted, hi_a rejected, S).""" + lo_a, hi_a = C, C + E24 + for _ in range(1200): # the vendored enclosure is ~1e-191 wide: about 560 halvings + mid = (lo_a + hi_a) / 2 + e = mom(prof, mid) + if accepts(e): + lo_a = mid + elif rejects(e): + hi_a = mid + else: + return lo_a, hi_a, mid + raise AL.AuditFail('kappa.root_bracket: no straddle point found') + + +def predict_endpoint(prof, C_start, span=Q(1, 10 ** 6)): + """The 24-digit endpoint of a complete profile by bisection on the 24-digit grid from an accepted C_start: + returns (C, kappa) with M(C) accepted and M(C + 1e-24) rejected.""" + lo = (C_start * 10 ** 24).__floor__() + hi = ((C_start + span) * 10 ** 24).__floor__() + need(accepts(moment(prof, Q(lo, 10 ** 24))) and rejects(moment(prof, Q(hi, 10 ** 24))), + 'kappa.predict: initial bracket') + while hi - lo > 1: + mid = (lo + hi) // 2 + e = moment(prof, Q(mid, 10 ** 24)) + if accepts(e): + lo = mid + elif rejects(e): + hi = mid + else: + raise AL.AuditFail('kappa.predict: the grid point %d straddles 1' % mid) + C = Q(lo, 10 ** 24) + return C, C - KAPPA_TERMS[0] - KAPPA_TERMS[1] + + +def price_delta(delta_per_call, a, calls=CALLS, live=W): + """Enclosure of the change of M(a) when every call's local profile changes by delta_per_call {rank: change}.""" + lo = hi = Q(0) + for r, d in delta_per_call.items(): + h_lo, h_hi = H(int(r), a) + if d >= 0: + lo += d * h_lo + hi += d * h_hi + else: + lo += d * h_hi + hi += d * h_lo + return calls * lo / (M * live), calls * hi / (M * live) + + +def dec(q, digits): + s = (q * 10 ** digits).__floor__() + return '%d.%0*d' % (s // 10 ** digits, digits, s % 10 ** digits) diff --git a/research/t488-independent-audit/vendor/pr3053/audit_lib.py b/research/t488-independent-audit/vendor/pr3053/audit_lib.py new file mode 100644 index 0000000000..e8e847cc89 --- /dev/null +++ b/research/t488-independent-audit/vendor/pr3053/audit_lib.py @@ -0,0 +1,396 @@ +# SPDX-License-Identifier: Apache-2.0 +"""Shared helpers of the PR #2751 audit: word parsing with initial corrections, an own exact forward tracker, +exact linear algebra, derived-object hashes and the negative-control bookkeeping. + +Model (the lineage's free-operator model, as in the vendored words_check.py of PR2315): every live register is an +independent dirty input symbol, private buffers start at zero, and each warp letter K_k is an abstract invertible +operator (a formal word with adjacent inverse cancellation). Rows are {(symbol, word): Fraction}; words are tuples of +signed ints, newest letter first. + +Initial corrections. The vendored parser (words_check.parse_borrow) accepts only words without initial +(anticipatory) corrections. PR2733's and PR2751's words carry 36 of them: {target t, source s, operator_word [k..], +coefficient q} meaning t += q K s before the precondition (PR2226 anticipatory borrowing, here PR2679's pre-life +hosting; the source must be at frame 0 at the start). parse_word strips them before the vendored parser, validates +them, and correction_items emulates each one EXACTLY inside the vendored replay as the conjugated triple +warp(s, k) ... ; add t += q s ; warp(s, -k) ...: its forward action is t += q K s with s unchanged, and the vendored +dual replay (which processes the same items with t,s,c -> s,t,-c and dual letters) applies the transpose-inverse. +Corrections carry no frame charge (as in PR2679, PR2733 and PR2751); this is recorded, not re-derived. + +Controls. expect() runs one negative control: it records the attempt (audit_common.attempt), runs the mutant through +the given function, and requires it to raise with exactly the named rule id (and, for a replay, at the named stage); +any other exception type propagates and fails the run. Rule ids: 'wc.' for a vendored words_check rejection +(stage from check_words.reject_stage), the 'rule' prefix of an AuditFail message ('rule: detail'), or a +fill_check80.CheckFailure id. Standard library only; Python 3.8-compatible syntax. +""" +import copy +import hashlib +import json +from collections import Counter, defaultdict +from fractions import Fraction + +import audit_common as AC +import check_words as CW +import fill_check80 as FC +import words_check as WC +from audit_common import AuditFail, need + +V, H = 112, 9 +FULL_LIST = [1 << b for b in range(H)][::-1] +ONE = Fraction(1) + + +# --------------------------------------------------------------------------------------------- hashing --- +def canon(obj): + """Canonical JSON text (sorted keys, no spaces; Fractions and tuples as their JSON-friendly forms).""" + def conv(x): + if isinstance(x, Fraction): + return str(x) + if isinstance(x, (set, frozenset)): + return sorted(conv(y) for y in x) + if isinstance(x, tuple): + return [conv(y) for y in x] + if isinstance(x, list): + return [conv(y) for y in x] + if isinstance(x, dict): + return dict((str(k), conv(v)) for k, v in x.items()) + return x + return json.dumps(conv(obj), sort_keys=True, separators=(',', ':')) + + +def sha(obj): + return hashlib.sha256(canon(obj).encode()).hexdigest() + + +def hist(d): + """{int rank: count} with string keys sorted numerically (for RESULT.json).""" + return dict((str(k), d[k]) for k in sorted(d, key=int)) + + +def intkeys(d): + return dict((int(k), v) for k, v in d.items()) + + +# --------------------------------------------------------------------------------- parse and replay --- +def parse_word(w): + """check_words.parse_ext on the word without its initial corrections; the corrections are validated and kept as + P['corrections'] = [(t, s, q, word)] (rule words.corrections).""" + w2 = dict(w) + corr = w.get('initial_corrections', []) + need(isinstance(corr, list), 'words.corrections: initial_corrections is not a list') + w2['initial_corrections'] = [] + P = CW.parse_ext(w2) + out = [] + for i, c in enumerate(corr): + need(isinstance(c, dict) and sorted(c) == ['coefficient', 'operator_word', 'source', 'target'], + 'words.corrections: correction %d fields' % i) + t, s, word = c['target'], c['source'], c['operator_word'] + q = Fraction(c['coefficient']) + need(WC.is_int(t) and V <= t < 2 * V, 'words.corrections: correction %d target is not a target' % i) + need(WC.is_int(s) and s in P['start'] and not (V <= s < 2 * V) and s in set(P['live']), + 'words.corrections: correction %d source is not a live non-target' % i) + need(P['start'][s] == (), 'words.corrections: correction %d source %d is not at frame 0 at the start' % (i, s)) + need(q != 0 and WC.dyadic(q), 'words.corrections: correction %d coefficient is not a non-zero dyadic' % i) + need(isinstance(word, list) and len(word) > 0 and all(WC.is_int(k) and 1 <= abs(k) <= len(P['warps']) + for k in word), + 'words.corrections: correction %d operator word' % i) + out.append((t, s, q, tuple(word))) + P['corrections'] = out + return P + + +def correction_items(corrections): + """The conjugated triples (vendored internal item format) emulating t += q K s, K = K_w0 K_w1 ... (rightmost + letter applied first).""" + items = [] + for t, s, q, word in corrections: + pre = [('warp', s, k) for k in reversed(word)] + post = [('warp', s, -k) for k in word] + items += pre + [('add', t, s, q, ())] + post + return items + + +def with_corrections(P): + Q = dict(P) + Q['pre'] = correction_items(P['corrections']) + list(P['pre']) + return Q + + +def replay(P, dual): + """check_words.run_replay (the vendored words_check.replay, orientation recorded) with the corrections.""" + return CW.run_replay(with_corrections(P), dual) + + +def rederive_cleanup(w): + return CW.rederive_cleanup(w) + + +def forge(w): + """Mutant helper: make the word's self-reported fields (report.local_profile, report.width_inventory) equal its own + frame recount, so that only the independent checks can reject it. Returns the recount.""" + fr = CW.frame_recount(parse_word(w)) + w['report'] = dict(w['report'], local_profile=hist(fr['paid']), width_inventory=hist(fr['inventory'])) + return fr + + +def forge_providers(prov, key, fr): + """A PROVIDERS.json copy whose entry `key` carries the recount fr (paid profile, ordered widths).""" + p = copy.deepcopy(prov) + p[key] = dict(p[key], actual_paid_profile=hist(fr['paid']), widths=list(fr['helper_widths'])) + return p + + +def delete_B(w, j): + """Delete B[j]; every warp record stays attached to the B item it preceded.""" + del w['B'][j] + for rec in w['warps']: + if rec[0] > j: + rec[0] -= 1 + + +def insert_B(w, j, items): + """Insert items before B[j]; warp records at positions > j move with their items, those at j stay before the + inserted items.""" + w['B'][j:j] = items + for rec in w['warps']: + if rec[0] > j: + rec[0] += len(items) + + +# ---------------------------------------------------------------------------------------- own tracker --- +def name(x): + if isinstance(x, tuple): + return '%s%d' % x + return 'X%d' % x if x < V else ('Y%d' % (x - V) if x < 2 * V else 'H%d' % x) + + +def fmt(row): + if not row: + return '0' + def key(kv): + return (str(kv[0][0]), kv[0][1]) + return ' + '.join('%s*%s%s' % (v, ('K%s.' % '.'.join(map(str, w))) if w else '', name(x)) + for (x, w), v in sorted(row.items(), key=key)) + + +def nregs(w): + return 2 * V + w['report']['R'] + + +def init_rows(w): + """Every live register its own symbol; every private buffer (helper_final key + n) zero.""" + n = nregs(w) + removed = set(w['removed']) + rows = dict((r, {(r, ()): ONE}) for r in range(n) if r not in removed) + for k in w['helper_final']: + rows[int(k) + n] = {} + return rows + + +def events(w, cleanup=False, corrections=True): + """The chronological forward program as (section, index, item): corrections (as warp/add/warp), precondition, + compensation, A, cut, B with every warp record placed before B[pos] in record order ('Bwarp', op, ['warp', + lender, op, U, E]), restore, and optionally cleanup.""" + ev = [] + if corrections: + for i, c in enumerate(w.get('initial_corrections', [])): + q = Fraction(c['coefficient']) + for k in reversed(c['operator_word']): + ev.append(('corr', i, ['warp', c['source'], k])) + ev.append(('corr', i, ['add', c['target'], c['source'], q.numerator, q.denominator, []])) + for k in c['operator_word']: + ev.append(('corr', i, ['warp', c['source'], -k])) + for sec in ('precondition', 'compensation', 'A', 'cut'): + ev += [(sec, i, g) for i, g in enumerate(w[sec])] + by_pos = defaultdict(list) + for op, rec in enumerate(w['warps'], 1): + by_pos[rec[0]].append((op, rec)) + for p in range(len(w['B']) + 1): + for op, rec in by_pos[p]: + ev.append(('Bwarp', op, ['warp', rec[1], op, rec[3], rec[4]])) + if p < len(w['B']): + ev.append(('B', p, w['B'][p])) + ev += [('restore', i, g) for i, g in enumerate(w['restore'])] + if cleanup: + ev += [('cleanup', i, g) for i, g in enumerate(w['cleanup'])] + return ev + + +def apply(rows, g): + """Forward action of one raw item: ['add', t, s, num, den, F]: t += (num/den) s; ['scale', t, num, den, F]; + ['warp', reg, k, ...]: prefix the letter k on every term of reg, cancelling a leading -k.""" + kind = g[0] + if kind == 'add': + t, s, c = g[1], g[2], Fraction(g[3], g[4]) + need(t in rows and s in rows and t != s, 'tracker.item: add on an unknown register %r' % (g[:3],)) + dst = rows[t] + for key, val in list(rows[s].items()): + nv = dst.get(key, 0) + c * val + if nv: + dst[key] = nv + else: + dst.pop(key, None) + elif kind == 'scale': + need(g[1] in rows, 'tracker.item: scale on an unknown register') + c = Fraction(g[2], g[3]) + rows[g[1]] = dict((k, v * c) for k, v in rows[g[1]].items() if v * c) + elif kind == 'warp': + reg, k = g[1], g[2] + need(reg in rows and k != 0, 'tracker.item: warp on an unknown register') + rows[reg] = dict(((var, word[1:] if word and word[0] == -k else (k,) + word), val) + for (var, word), val in rows[reg].items()) + else: + raise AuditFail('tracker.item: unknown item %r' % (kind,)) + + +def run(w, evs, rows=None, stop=None): + """Apply evs to rows (default init_rows(w)); stop(section, index, item) -> True halts before that event.""" + rows = init_rows(w) if rows is None else rows + for sec, i, g in evs: + if stop is not None and stop(sec, i, g): + return rows + apply(rows, g) + return rows + + +def own_symbol(r): + return {(r, ()): ONE} + + +def target_final(t): + return {(V + t, ()): ONE, (t, ()): ONE} + + +# ------------------------------------------------------------------------------------- linear algebra --- +def det(M): + """Exact determinant (Gaussian elimination over Fractions).""" + A = [list(map(Fraction, r)) for r in M] + n, d = len(A), Fraction(1) + for c in range(n): + p = next((r for r in range(c, n) if A[r][c] != 0), None) + if p is None: + return Fraction(0) + if p != c: + A[c], A[p] = A[p], A[c] + d = -d + d *= A[c][c] + for r in range(c + 1, n): + f = A[r][c] / A[c][c] + if f: + A[r] = [a - f * b for a, b in zip(A[r], A[c])] + return d + + +def rank(M): + A = [list(map(Fraction, r)) for r in M] + rk, nrows, ncols = 0, len(A), (len(A[0]) if A else 0) + for c in range(ncols): + p = next((r for r in range(rk, nrows) if A[r][c] != 0), None) + if p is None: + continue + A[rk], A[p] = A[p], A[rk] + for r in range(nrows): + if r != rk and A[r][c] != 0: + f = A[r][c] / A[rk][c] + A[r] = [a - f * b for a, b in zip(A[r], A[rk])] + rk += 1 + return rk + + +def strs(M): + return [[str(x) for x in r] for r in M] + + +# ------------------------------------------------------------------------------------ control bookkeeping --- +def rule_of(exc): + """(rule id, stage, message) of an audit rejection; None for any other exception.""" + if isinstance(exc, WC.Reject): + return 'wc.' + exc.check, CW.reject_stage(exc), exc.msg + if isinstance(exc, AuditFail): + msg = str(exc) + return msg.split(':', 1)[0], None, msg + if isinstance(exc, FC.CheckFailure): + return exc.check, None, str(exc) + return None + + +def expect(out, name, rule, fn, stage=None, attrs=None, **extra): + """Negative control: fn() must raise an audit rejection with exactly this rule id (and stage, if given; and, if + attrs is given, with exception attributes equal to attrs, e.g. the exact set of flipped hypothesis booleans). + The attempt is counted before fn runs; the record is appended only after the rejecting rule was checked.""" + AC.attempt(name) + try: + fn() + except (WC.Reject, AuditFail, FC.CheckFailure) as e: + got, where, msg = rule_of(e) + seen = dict((k, getattr(e, k, None)) for k in sorted(attrs or ())) # no reference to e is kept (cycles) + else: + raise AuditFail('negative control ACCEPTED: ' + name) + need(got == rule and (stage is None or where == stage), + 'control %s: rejected by %s%s (%s), expected %s%s' % (name, got, '' if where is None else ' at %r' % where, + ' '.join(msg.split())[:120], rule, + '' if stage is None else ' at %r' % stage)) + if attrs is not None: + need(seen == dict(attrs), 'control %s: rejected by %s with %r, expected %r' % (name, got, seen, dict(attrs))) + rec = dict(control=name, expected=rule, rejected_by=got, message=' '.join(msg.split())[:160]) + if stage is not None: + rec['stage'] = where + if attrs is not None: + rec['asserted'] = canonical_json(attrs) + rec.update(extra) + out.append(rec) + return rec + + +def canonical_json(x): + return json.loads(canon(x)) + + +def harness_selftest(): + """expect() itself must fail on a mutant rejected by another rule, on an accepted mutant and on a mutant rejected + by the right rule with other asserted attributes (checked here, then removed from the attempt counter, so they do + not enter the negative-control counts).""" + n0 = len(AC.ATTEMPTED) + out = [] + res = {} + + def wrong_attrs(): + e = AuditFail('selftest.expected: rejected with other attributes') + e.flipped = ['other'] + raise e + for case, fn, attrs in (('wrong_rule', lambda: need(False, 'selftest.other: rejected by another rule'), None), + ('accepted', lambda: None, None), + ('wrong_attrs', wrong_attrs, dict(flipped=['expected']))): + try: + expect(out, 'harness self-test ' + case, 'selftest.expected', fn, attrs=attrs) + except AuditFail as e: + res[case + '_raises'] = str(e).startswith(('control harness self-test', 'negative control ACCEPTED')) + else: + res[case + '_raises'] = False + del AC.ATTEMPTED[n0:] + need(all(res.values()) and len(res) == 3 and not out, 'harness.selftest: expect() accepted a bad control %r' % res) + return res + + +def mutant(w): + return copy.deepcopy(w) + + +# Self-check registry: each self-check / scope-F check called from verify.py records its rule id here after it has +# passed; verify.py requires the counts to equal EXPECTED_SELF_CHECKS (so a check whose call is removed fails the run). +RAN = Counter() + + +def ran(rule): + RAN[rule] += 1 + + +def check_self_checks(counts, expected): + """self_checks.ran: every registered check ran (and passed) exactly as often as expected.""" + need(dict(counts) == dict(expected), 'self_checks.ran: %r != %r' % (dict(counts), dict(expected))) + + +def counter_delta(a, b): + """a - b as {rank: change}, zero entries dropped.""" + d = Counter(a) + d.subtract(Counter(b)) + return dict((k, v) for k, v in sorted(d.items()) if v) diff --git a/research/t488-independent-audit/vendor/pr3053/audit_words.py b/research/t488-independent-audit/vendor/pr3053/audit_words.py new file mode 100644 index 0000000000..6368333e16 --- /dev/null +++ b/research/t488-independent-audit/vendor/pr3053/audit_words.py @@ -0,0 +1,128 @@ +# SPDX-License-Identifier: Apache-2.0 +"""Scope B: PR #3015's two final words (and PR #2979's parent words as positive controls) through this contributor's +lineage checks, with the vendored byte-pinned modules as libraries and no code of either PR. + +word_pipeline(w, declared, expect) runs, in this order (each step raises with its own rule id): + parse (vendored parser + validated initial corrections; wc.format / words.corrections) + -> warp algebra of every B record (wc.warp) + -> literal frame recount (wc.frame): paid local profile, width inventory, helper widths + -> words.counts (live inputs, helpers, private buffers, warp records, initial corrections = expect) + -> words.report_fields (the word's own report equals the recount) + -> words.declared (the recount equals the declared local profile and inventory of EDITS.json / CLAIM.json) + -> forward replay and dual replay of all live inputs in the free-operator model through cleanup and the target + endpoint (wc.replay, orientation recorded) + -> the own exact tracker through restore and cleanup (words.tracker: targets Y_t + X_t, restored registers their own + input and every private buffer zero before cleanup; the final identity and zero buffers after cleanup) + -> words.cleanup (the literal cleanup equals the own inverse-reverse re-derivation, frames aside, every item at FULL) + -> words.buffers (the cleanup's private buffers are exactly the helper_final endpoints, one each). +Standard library only; Python 3.8-compatible syntax. +""" +from collections import Counter + +import audit_lib as AL +import check_words as CW +import words_check as WC +from audit_common import need + +V = AL.V + + +def counts(P, w): + return dict(live=len(P['live']), helpers=len([r for r in P['live'] if r >= 2 * V]), buffers=len(P['buffers']), + warps=len(w['warps']), corrections=len(w.get('initial_corrections', [])), + a_prelude_events=w['prelude_events'], + sections=dict((k, len(w[k])) for k in ('A', 'B', 'restore', 'cut', 'precondition', 'compensation', + 'cleanup'))) + + +def check_counts(c, expect): + got = dict((k, c[k]) for k in expect) + need(got == dict(expect), 'words.counts: %r != %r' % (got, dict(expect))) + + +def check_report(P, fr): + need(fr['paid'] == P['claimed_profile'] and fr['inventory'] == P['claimed_inventory'], + 'words.report_fields: report %r %r != recount %r %r' % (P['claimed_profile'], P['claimed_inventory'], + fr['paid'], fr['inventory'])) + + +def check_declared(fr, declared): + prof = AL.intkeys(declared['local_profile']) + inv = AL.intkeys(declared['width_inventory']) + need(fr['paid'] == prof and fr['inventory'] == inv, + 'words.declared: recount %r %r != declared %r %r' % (fr['paid'], fr['inventory'], prof, inv)) + + +def check_cleanup(w): + mine = AL.rederive_cleanup(w) + need([g[:-1] for g in mine] == [g[:-1] for g in w['cleanup']], + 'words.cleanup: the literal cleanup is not the own inverse-reverse re-derivation') + full = WC.span([1 << b for b in range(9)]) + need(all(WC.span(g[-1]) == full for g in w['cleanup']), 'words.cleanup: an item is not at FULL') + return dict(items=len(mine), literal_equal=True, all_at_full=True) + + +def check_buffers(w, P): + n = 2 * V + w['report']['R'] + touched = set() + for g in w['cleanup']: + for r in ((g[1],) if g[0] == 'warp' else ((g[1], g[2]) if g[0] == 'add' else (g[1],))): + if r >= n: + touched.add(r) + ends = set(int(k) + n for k in w['helper_final']) + need(touched == ends == set(P['buffers']), 'words.buffers: cleanup buffers %d != helper_final endpoints %d' + % (len(touched), len(ends))) + return dict(private_buffers=len(ends), offset=n) + + +def track(w): + """One exact forward pass of the own tracker (audit_lib rows: every live register its own symbol, private buffers + zero, warp letters as formal words) through corrections, precondition, compensation, A, cut, B with its warps, + restore and cleanup. Returns the pre-cleanup and final conditions (rule words.tracker, see tracker_check).""" + rows = AL.init_rows(w) + n = AL.nregs(w) + removed = set(w['removed']) + restored = set(int(k) for k in w['helper_final']) + live = [r for r in range(n) if r not in removed] + pre = None + for sec, i, g in AL.events(w, cleanup=True): + if sec == 'cleanup' and pre is None: + pre = dict(targets_final=all(rows[V + t] == AL.target_final(t) for t in range(V)), + restored_own=all(rows[r] == AL.own_symbol(r) for r in restored), + buffers_zero=all(not rows[b + n] for b in restored), + rows_with_operator_words=sum(1 for r in live if any(wd for _, wd in rows[r]))) + AL.apply(rows, g) + need(pre is not None, 'words.tracker: empty cleanup') + final = all(rows[r] == (AL.target_final(r - V) if V <= r < 2 * V else AL.own_symbol(r)) for r in live) + return dict(pre, final_identity=final, buffers_zero_after=all(not rows[b + n] for b in restored), + live=len(live), buffers=len(restored), cleanup_items=len(w['cleanup'])) + + +def tracker_check(check): + need(check['targets_final'] and check['restored_own'] and check['buffers_zero'], + 'words.tracker: pre-cleanup state %r' % dict((k, check[k]) for k in ('targets_final', 'restored_own', + 'buffers_zero'))) + need(check['final_identity'] and check['buffers_zero_after'], 'words.tracker: final identity after cleanup') + + +def word_pipeline(w, declared, expect): + P = AL.parse_word(w) + warps = WC.check_all_warps(P) + fr = CW.frame_recount(P) + c = counts(P, w) + check_counts(c, expect) + check_report(P, fr) + if declared is not None: + check_declared(fr, declared) + fwd = AL.replay(P, False) + dual = AL.replay(P, True) + check = track(w) + tracker_check(check) + cl = check_cleanup(w) + bf = check_buffers(w, P) + rec = dict(counts=c, profile=AL.hist(fr['paid']), inventory=AL.hist(fr['inventory']), mass=fr['mass'], + helper_widths_sha256=AL.sha(fr['helper_widths']), warp_records=len(warps), + centre_copy_children=sorted(sum(1 for d in c_['children']) for c_ in fr['copy_paths']), + replay_forward=fwd, replay_dual=dual, tracker=check, cleanup=cl, buffers=bf, + independent_inputs=fwd['independent_inputs']) + return rec, P, fr diff --git a/research/t488-independent-audit/vendor/pr3053/check_accounting.py b/research/t488-independent-audit/vendor/pr3053/check_accounting.py new file mode 100644 index 0000000000..e749057dd8 --- /dev/null +++ b/research/t488-independent-audit/vendor/pr3053/check_accounting.py @@ -0,0 +1,638 @@ +# SPDX-License-Identifier: Apache-2.0 +"""Scope C and provenance: verifier overrides, recovered source, stock/precision accounting, endpoint, kappa, PR2474. + +What is proved: + * overrides: OVERRIDES.json's old hashes are PR2462's independent_core.py and new/rebind.py (rebind also equals + PR2435's); the new hashes are PR #2530's files; ignoring CRLF, independent_core.py changes exactly two exact- + equality pins (independent_core_change) and rebind.py exactly one expression, physical += zero buffers + (rebind_change). The pins, parsed as literals (never executed), equal what the words give: 16 line-[30] lifted + targets and centre children {1:8, 7:8} (old pins = PR2462's words). No other .py file of the PR2462 package + carries the old pins (old_pin_files); all three checks are functions of the file bytes; + * recovered source: PR #2530's source-pr2435.tar.gz (438 members) equals the PR2435 package tree (438 files) and, + with pr2462-overlay.tar.gz (40 members) applied, equals the PR2462 package tree (443 files), including the + overlay words (reconstruct; recon.pr2435_tree, recon.pr2462_tree, recon.overlay_words); archives are read in + memory as data and nothing in them is executed or written to disk; + * stock: physical = W + 8 + R + Z + 2 gives 66814 (R=422, Z=78 counted from the words) and reproduces PR2462's + SPEC 66822 (R=430) and PR2435's SPEC 66826 (R=434); local mass falls 5315 -> 5251 = 8 x 8; + * precision arithmetic: receipt prefix_l1 values give 62/69 bits (valid upper bounds); the global prefix computed + from them, 69+37+666+666+4 = 1442, equals receipts/INDEPENDENT-GLOBAL.json prefix_bits and fits the payload 2048 + (= its charged_prefix_bits) (global_prefix); the row coefficient is the receipt's 19545 for every physical stock + 66736..66830; the receipt values are recorded next to the own ones; + * endpoint: a rigorous integer fixed-point enclosure (2^-640, directed rounding, atanh/exp series with tail + bounds) of M(a) = (1/W) sum n_r (r/73)^(1-a) on PROFILE.json; an endpoint is accepted only if the upper end of + the enclosure is below 1 (accepts) and rejected only if the lower end is above 1 (rejects). C = + 0.001713737921547504346752 is accepted and C + 1e-24 rejected; bisection brackets the root, so C is its + 24-digit floor; the conservative endpoint is likewise accepted/rejected; the ledger's two 2^-527 native moments + bracket the enclosure at b; + * kappa = C - 10^-70 - 10^-46 with the ledger's b = C - r0, q = C - 2 r0, delta = (1025/1024) r0, and the + DIG = 24 instance of PR2462's grid-maximum rule (outer_grid, rule outer.grid): C*F integer, 0 < C - eps*q + (= 2.0017e-104, recorded) < 1/F, floor(F eps q)/F - B = kappa (floor(F eps q) = C*F - 1, recorded); + * PR2474 comparison: the pinned OUTER-CLAIM equals the file at the PR2474 commit; the gain computed from the two + kappas equals CLAIM's latest_claim_gain exactly (latest_gain, rule pr2474.gain) and the percentage string is its + 100-digit truncation; + * enclosure soundness guards: every exp series argument lies in [0, 1) in fixed point (the precondition of the + "2 x next term" tail bound) and every lower end is <= its upper end; + * positive control: the same moment checker's enclosure [lo, hi] at PR2462's certified 3000-digit endpoint + C2462 (where 1 - 1e-2999 < M <= 1) satisfies lo <= 1 and hi >= 1 - 1e-2990, so an upper end that falls below + the true moment by more than about 1e-2990 (an unsound exp upper rounding) is caught; C2462 - 1e-150 is + accepted and C2462 + 1e-150 rejected; + * negative controls, each asserting its rule (and, where one rule has several conditions, the condition) and each + run through the function the main check uses. accounting (8): the acceptance check fails at C + 1e-24, at the + conservative C' + 1e-24 and at C2462 + 1e-150; at C2462 the enclosure straddles 1 (lo < 1 < hi), so the + acceptance check must fail at C2462 + 1e-3000 and the rejection check at C2462 (a predicate on the wrong end of + the enclosure would pass either); a wrong target pin; a reconstruction that keeps PR2435's base word; and the + stock rule without the zero-buffer term (the pre-PR2530 rebind formula, 66736 != 66814). outer_grid (3): + kappa + 1e-70 (floor equality), q - 2e-70 (C - eps*q > 1/F) and q = C/eps (C - eps*q = 0). overrides_diff + (5): independent_core.py with one more line (a require) commented out, rebind.py with one more line (an assert) + commented out, each once before and once after the allowed lines, and an extra .py file carrying the old target + pin. provenance_accounting (6): the source archive with one extra and with one missing member + (recon.pr2435_tree), the overlay with one extra member and missing one of the files it adds (recon.pr2462_tree), + CLAIM's gain + 1e-40 (pr2474.gain), and the global prefix from the largest receipt prefix_l1 doubled + (precision.global_prefix). +Scope limits: the 63 outer inequalities are not re-derived; prefix_l1 is taken from the PR's receipts (only its +bit arithmetic is checked); the ledger was not regenerated here (PR2462's outer code is not run by this package). +""" +import ast +import io +import posixpath +import re +import tarfile +from collections import Counter +from fractions import Fraction as Q + +import fill_check80 as FC +from audit_common import AuditFail, attempt, git_blob_id, need, positive, sha256 + +V, M, W, T, CALLS = 112, 73, 66304, 74, 666 +PKG_HANDOFF = 'research/gmx-t74-borrow43-handoff/' +IC_PIN_LINES = (72, 93) # 1-based lines of independent_core.py that may change +REBIND_LINE = 52 +REBIND_SUFFIX = " + max(len(rd(d / 'CANONICAL-WORD.json.gz').get('zero_buffers', [])) for d in locs.values()))" +OLD_PIN_PATTERNS = (r'23,\s*34,\s*50,\s*108', r'1:\s*4,\s*7:\s*4,\s*8:\s*4') + + +def dec(q, digits): + """Truncated decimal string of a non-negative Fraction.""" + n = q.numerator * 10 ** digits // q.denominator + s = str(n).rjust(digits + 1, '0') + return s[:-digits] + '.' + s[-digits:] + + +# ------------------------------------------------------------------------------------------ overrides --- +def _lines(raw): + return raw.decode('utf-8').replace('\r\n', '\n').split('\n') + + +def changed_lines(old_raw, new_raw): + old, new = _lines(old_raw), _lines(new_raw) + need(len(old) == len(new), 'overrides: line count changed') + return [(i + 1, old[i].strip(), new[i].strip()) for i in range(len(old)) if old[i] != new[i]] + + +def pin_literals(ic_text_lines): + """Parse (never execute) the two pins of independent_core.py as Python literals.""" + targets = re.search(r'set\(levels\) == (\{[0-9, ]*\})', ic_text_lines[0]) + children = re.search(r'forward\) == (\{[0-9:, ]*\})', ic_text_lines[1]) + need(targets is not None and children is not None, 'overrides: pin lines not recognised') + return sorted(ast.literal_eval(targets.group(1))), dict(ast.literal_eval(children.group(1))) + + +def pins_match_words(ic_new_line_pair, ic_old_line_pair, words): + """New pins must equal the PR2530 words (both providers), old pins the PR2462 words.""" + for tag in ('base', 'extra'): + for lines, key in ((ic_new_line_pair, 'pr2530'), (ic_old_line_pair, 'pr2462')): + targets, children = pin_literals(lines) + wsum = words[tag][key] + need(targets == wsum['lifted_targets'], 'overrides: %s target pin != %s %s words' % (key, key, tag)) + need(Counter(children) == Counter(d for _, ch in wsum['copy_paths'] for d in ch), + 'overrides: %s centre-child pin != %s %s words' % (key, key, tag)) + + +def independent_core_change(old_raw, new_raw): + """independent_core.py (bytes, CRLF ignored) changes exactly the lines IC_PIN_LINES; returns the changes.""" + ic = changed_lines(old_raw, new_raw) + need([c[0] for c in ic] == list(IC_PIN_LINES), 'overrides: independent_core changed lines %r' % [c[0] for c in ic]) + return ic + + +def rebind_change(old_raw, new_raw): + """rebind.py (bytes, CRLF ignored) changes exactly line REBIND_LINE, by appending the zero-buffer term to the + physical stock expression; returns the change.""" + rb = changed_lines(old_raw, new_raw) + need([c[0] for c in rb] == [REBIND_LINE] and rb[0][1].endswith('+ 8 + R + 2)') + and rb[0][2] == rb[0][1][:-1] + REBIND_SUFFIX, 'overrides: rebind change %r' % [c[0] for c in rb]) + return rb + + +def old_pin_files(py_files, listing): + """py_files: {relative path: bytes} of the .py files of the PR2462 package; listing: the .py paths of the pinned + tree, which the scan must cover exactly. Only independent_core.py may carry the old pins (any other file would + also have needed editing); returns the files that carry them.""" + need(sorted(py_files) == sorted(listing), 'overrides: old-pin scan covers %d of %d .py files' % (len(py_files), len(listing))) + hits = sorted(rel for rel, data in py_files.items() + if any(re.search(p, data.decode('utf-8', 'replace')) for p in OLD_PIN_PATTERNS)) + need(hits == ['independent_core.py'], 'overrides: old pins also in %r' % hits) + return hits + + +def overrides(inp, words): + ov = inp.json('overrides2530') + need(sorted(ov) == ['independent_core.py', 'new/rebind.py'], 'overrides: entries') + need(ov['independent_core.py']['old_sha256'] == sha256(inp.raw('ic2462')), 'overrides: independent_core old hash != PR2462 file') + need(ov['independent_core.py']['new_sha256'] == sha256(inp.raw('ic2530')), 'overrides: independent_core new hash') + need(ov['new/rebind.py']['old_sha256'] == sha256(inp.raw('rebind2462')) == sha256(inp.raw('rebind2435')), + 'overrides: rebind old hash != PR2462 (= PR2435) file') + need(ov['new/rebind.py']['new_sha256'] == sha256(inp.raw('rebind2530')), 'overrides: rebind new hash') + ic = independent_core_change(inp.raw('ic2462'), inp.raw('ic2530')) + pins_match_words([c[2] for c in ic], [c[1] for c in ic], words) + rb = rebind_change(inp.raw('rebind2462'), inp.raw('rebind2530')) + # no other .py file of the PR2462 package carries the old pins (they would also have needed editing) + tree = inp.tree('tree2462') + py_files = dict((rel, inp.show_blob(inp.table['tree2462'][1], PKG_HANDOFF + rel, tree[rel])) + for rel in sorted(tree) if rel.endswith('.py')) + hits = old_pin_files(py_files, [rel for rel in tree if rel.endswith('.py')]) + return dict(independent_core_changed=[list(c) for c in ic], rebind_changed=[list(c) for c in rb], + old_files_crlf=b'\r\n' in inp.raw('ic2462') and b'\r\n' in inp.raw('rebind2462'), + new_files_crlf=b'\r\n' in inp.raw('ic2530') or b'\r\n' in inp.raw('rebind2530'), + py_files_scanned_for_old_pins=len(py_files), files_with_old_pins=hits), ic, py_files + + +# ------------------------------------------------------------------------------- source reconstruction --- +def tar_blobs(raw, prefix=''): + """{path relative to prefix: git blob id} of the regular files of a .tar.gz, read in memory (data only).""" + out = {} + with tarfile.open(fileobj=io.BytesIO(raw), mode='r:gz') as tf: + for m in tf.getmembers(): + name = m.name + if m.isdir(): + continue + need(not name.startswith('/') and '..' not in name.split('/') and posixpath.normpath(name) == name, + 'tar: unsafe member name %r' % name) + need(m.isfile(), 'tar: non-regular member %r' % name) + need(name.startswith(prefix), 'tar: member outside %s: %r' % (prefix, name)) + rel = name[len(prefix):] + need(rel not in out, 'tar: duplicate member %r' % name) + out[rel] = git_blob_id(tf.extractfile(m).read()) + return out + + +def same_tree(got, want, label): + """Two {path: git blob id} maps are equal: no missing, no extra and no differing file ('