README.md

Python examples

Generating Vakint's rule packages

generate_vakint_artifacts.py is Python steering of the generic release CLI and a small Rust example using the existing public terminal-normalization API. The algebra, search, native encoding and loading remain in RustRed/Symbolica. It generates rules from family inputs; it never imports a saved rule package as a generation source or invokes FORM.

Initialize the dependency and its required patch following the Symbolica setup instructions. Build once from the repository root, with SYMBOLICA_LICENSE already set:

mkdir -p TMP
export TMPDIR="$PWD/TMP" TMP="$PWD/TMP" TEMP="$PWD/TMP"
nix develop --command cargo build --release --locked -p rustred-app \
  --bin rustred --example normalize_candidate_terminals
nix develop --command python examples/python/generate_vakint_artifacts.py \
  --loops 1 2 3 --workers 6 --output-directory TMP/vakint-lower-generated --execute
nix develop --command python examples/python/generate_vakint_artifacts.py \
  --loops 4 --workers 6 --output-directory TMP/vakint-four-generated --execute

Omit --execute for a dry-run plan, select any subset with --loops, or use --loops 4 --families fg for one four-loop parent. The destination must not exist. --executable and --normalizer select already-built binaries; the script never compiles during a run. Individual native command logs and wall times are retained. There is no runtime or numerator-rank cutoff; --workers is a compute setting, not a RAM guard. Apply your own resource envelope for large generations.

Expected output tree (selecting all four loop counts):

OUTPUT/
  unit_mass_vacuum_k1.rrbin
  unit_mass_vacuum_k3.rrbin
  unit_mass_vacuum_k6.rrbin
  unit_mass_vacuum_k*.inspection.toml
  unit_mass_vacuum_k*.reduction.toml
  four_loop/
    h.candidates.rrbin.gz        # also fg, bmw, x
    h.rrnorm.bin                # exact terminal normalization
    h.generation.toml
    h.normalization.json
    h.normalization-replay.json
  logs/
  plan.json
  inputs.json
  result.json

Uncompressed native programs are retained too. A failed command preserves its logs and writes failure.json instead of a success receipt. One/two-loop artifacts have one/two explicit masters; the current three-loop recipe has 623 generated rules, 5,640 guarded cells, 38 masters and 26 zero sectors. Each lower-loop file is inspected and applied in fresh processes. Four-loop programs retain candidate status: cold loading and exact normalization replay are not independent certification of their IBP sources or unrestricted closure.

The September 24 release run produced the following unrestricted four-loop inventories. Raw terminal counts include symmetry-related or reducible numerator forms; the last column counts the actual catalog outputs after normalization.

Parent	Solved nonzero sectors	Generated rules	Raw terminals	Catalog outputs
H	314	21,318	386	22
FG	124	9,266	145	16
BMW	134	9,018	179	17
X	328	19,907	445	19
Total	900	59,509	1,155	74

Fresh generation commands took approximately 19.4/25.8/54.0/101.2 seconds respectively on six pinned cores in that run, including process startup and native encoding but excluding compilation, the subsequent cold checks and normalization. These are single-run observations, not portable timing promises or full-family certification measurements.

The four-loop input files live in ../input/vakint/. Their exact denominator ordering and metadata match Vakint; the recipes leave the auxiliary ISP indices nonpositive and impose no total rank bound. H/X have one auxiliary index, FG/BMW two. They are input data, not engine dispatch. This generation is distinct from the restricted renormalizable starting-domain walk used to assess the five-loop campaign.

Master values are not generated by this script. To include the existing precomputed Vakint catalog, add:

--catalog-directory FOR_REFERENCE_ONLY_DO_NOT_PUSH/gammaloop/crates/vakint/data/rustred/four_loop

Use the actual four-loop asset directory in your checkout (see its README). The script validates the catalog's family identity and exact normalized terminal key set before copying it. It does not calculate values, request extra precision or silently invent missing masters. All four parent catalogs together retain 74 entries; this is not a claim of a minimal independent basis.

Optionally supply --reference-directory PATH_TO_VAKINT_NATIVE_DATA to compare fresh rule programs with the corresponding shipped native programs. That directory must contain unit_mass_vacuum_k{1,3,6}.rrbin and four_loop/, for example FOR_REFERENCE_ONLY_DO_NOT_PUSH/gammaloop/crates/vakint/data/rustred. Reference files are used only after generation. Comparison checks exact native structures and coefficient meanings rather than ambient Symbolica registry bytes. A difference fails explicitly; it is never silently accepted or used as a fallback. The script does not install anything into Vakint: changing shipped assets still requires its numerical regression tests.

Saved-owner production campaign

The current prepared five-loop input snapshot lives in campaigns/five-loop-saved-coarse-cover/inputs. It contains the saved owners, the unchanged 67-root base query document and 67 ordinary auxiliary owner regions for reuse. Python does no algebra or topology dispatch. The previous campaigns/five-loop-saved run is paused and retains its own checkpoint; its inputs must not be substituted when resuming that checkpoint. The full campaign is an explicit launch, not an example smoke test. The current snapshot is already running in Zellij session rustred, tab five_loop_vacuum; do not start a second copy. For a separately staged new campaign (replace NEW_CAMPAIGN with its directory):

mkdir -p TMP
export TMPDIR="$PWD/TMP" TMP="$PWD/TMP" TEMP="$PWD/TMP"
export CARGO_HOME="$PWD/TMP/cargo-home" CARGO_TARGET_DIR="$PWD/target"
nix develop --command cargo build --release --locked -p rustred-app --bin rustred -j 16
nix develop --command python examples/python/production_saved_owner_campaign.py \
  --campaign-directory campaigns/NEW_CAMPAIGN \
  --executable target/release/rustred --start

The launcher freezes exact executable bytes and the original steering policy on first use. Omit --start to freeze/prepare and print the command without launching. The production preset has no solve timeout or cumulative work stop; it retains bounded worker buffers, input/scratch/algebra admission, at most 256 workers (never more than the permitted CPU affinity; the default is 50) and a default 500 GB decimal requested RAM ceiling. Hourly native checkpoints are the resume authority.

The frozen steering file is rustred.production-steering.v2: workers, the CPU set, checkpoint interval, RAM policy, --publication-policy (default ready for new campaigns; ordered remains selectable and is required for physical subdivision), --transfer-unreserved-lookahead (default 256) and optional --inspection-workers N (default: the native split). Older v1 steering files stay readable with their recorded values (Ordered, 256, native split). Resume refuses any change to a frozen option except the RAM overrides below. --cpus accepts comma lists and ranges (128-177, 0-3,8); the frozen value is the sorted comma list. The Python RAM guard requests checkpoint-and-stop at 95% of the effective hard ceiling, earlier under host/cgroup pressure. Configure --max-memory-bytes (any positive byte count, for example 700000000000) or --ram-guard-margin-percent on first preparation or as per-invocation resume overrides. No additional address-space cap is imposed. Sampled RSS cannot strictly prevent between-sample overshoot; hard emergencies preserve the last completed save.

Ctrl-C requests a graceful saved pause and prints the exact restart command. Wait for completion of that save. The production resume command automatically reuses the frozen workers, CPU set and optional subdivision policy:

nix develop --command python examples/python/production_saved_owner_campaign.py \
  --campaign-directory campaigns/five-loop-saved-coarse-cover --resume --start
nix develop --command python examples/python/campaign_monitor.py \
  campaigns/five-loop-saved-coarse-cover/runs/RUN_NAME

For a later resume requesting 700 GB, add --max-memory-bytes 700000000000. This does not modify a running process, the original frozen steering.json, executable or native solver policy. Wait for the current process's saved pause before resuming. Repeat the RAM override on each production resume, or use the supervisor's emitted exact restart command; omitted values use the original frozen RAM defaults. Host/cgroup protection can still admit less RAM, and the 95% checkpoint threshold uses that effective ceiling (665 GB only when the full 700 GB is admitted). Plans and receipts distinguish these policies.

active-run.json identifies the latest run. Each run has atomic status.json, PID/start/boot metadata, bounded-tail event monitoring and separate native stdout/stderr. The aligned, colored, overwriting TTY dashboard and append-only JSON redirected summaries show measured CPU activity separately from worker reservations, finite initial-entry publication separately from descendant work, and checkpoint writing/completion. The progress bar counts recursively closed initial domains; the publication counter stays separate. Neither is a percentage of all eventual work, and the closure ETA stays unknown. Use --once or --json with the monitor for a read-only snapshot; NO_COLOR disables color. Stale heartbeats/process identities are reported explicitly. The dashboard adapts to terminal width/height (additional detail is visible in taller terminals). The full normalized frame is always saved, even when a short terminal hides secondary detail. Rates come from status.json's additive derived block: completions per hour, stall share (fraction of wall time in heartbeat intervals of at least 5 s or 20 s with no completion), coordinator duty, pending growth per completion, RSS per discovered domain, computing inspectors, max scheduled finite rank, checkpoint generation/bytes/seconds/duty and roots closed. They are deltas of native counters over the last hour of heartbeats (two hours retained), never estimates; absent fields print unknown, and no line carries an ETA. heartbeat_metrics.py EVENTS.jsonl --start S --end E recomputes the same numbers offline for any elapsed window, so a pilot can be compared with the live run at matched elapsed time.

Discovery/closure shows D/C: newly discovered domains divided by newly recursively closed domains in the same trailing-hour window. For example, 2.000 means two discoveries per observed recursive closure. Positive discoveries with zero closures display ∞ in red; an empty 0/0 window, unavailable telemetry, invalid endpoints or resets display unknown neutrally. Both this ratio and the unchanged pending ... per completion metric are red strictly above 2, yellow strictly above 1 through 2, and green at 1 or below. CPU colors are unchanged: red below 50% of reserved cores, yellow from 50% to below 75%, green at 75% or more. The display includes actual sample span and startup warm-up; Closure snapshot shows age, staleness and whether the closure scan advanced. Closure counts update in batches and are conservative lower bounds: the ratio and its colors are observations, not convergence proof, a bound on closure throughput, or an ETA. Epoch supplies these snapshots through its existing duty-throttled, cancellable scan at complete merge boundaries, never by forcing a graph scan for each heartbeat or checkpoint. The rolling window restarts on each launcher invocation, including resume or automatic rescue; persisted closure counts do not restore an hour of samples.

New supervisor runs also append telemetry.jsonl in the run directory, including with --no-progress. It records raw local-completion and recursive-closure counts, their distinct measured rates/deltas, actual window endpoints, warm-up, scan age and reset/missing-data state. Frames are bounded to 64 KiB; recording also preserves checkpoint milestones for consumers reading only this stream. The new rates.discovery_per_recursive_closure_1h object records a finite value or null, an explicit infinite boolean, and window/state metadata. Null with infinite: false means unavailable; infinity is never written as a nonstandard JSON number. Displays derive the ratio from raw paired deltas, never reinterpret historical discovery_closure_balance values. The raw signed discovery-minus-closure rate remains in rates.discovery_minus_closure. Local-completion warm-up begins at the first usable heartbeat, not at the start of a potentially long preparation phase. The normalized recording uses constant memory and cannot stop a campaign if the stream fails (the error is exposed under status.json.telemetry_stream). A normalized data producer (campaign_telemetry.py) is independent of the terminal/JSON consumer (campaign_dashboard.py), so future dashboards need not change the producer.

Plot unresolved domains and their raw net rate without a plotting package:

nix develop --command python examples/python/plot_campaign_rates.py \
  campaigns/NAME/runs/RUN_NAME/telemetry.jsonl --output campaign-rates.svg

The SVG is unchanged: its left axis shows total discovered-but-not-recursively- closed domains; its right axis shows the raw signed discovery-minus-closure rate in domains/second, not the dimensionless D/C dashboard ratio. A negative raw rate means the recorded unresolved gap shrank over that sampled window. Exact sampled endpoints, covered spans and warm-up flags remain in the JSON stream; the plot does not label each point's window. Graph-dirty snapshots stay visible as hollow markers; larger markers identify an advanced scan. Missing/invalid observations and stale heartbeats are gaps, never fabricated zeros. --start S --end E selects run elapsed seconds; the default bounded extrema envelope keeps at most 20,000 plot points. The SVG is a read-only visualization, not a closure certificate.

The new standalone monitor can read an existing run's status.json without restarting or modifying that run. Old supervisors do not acquire a new telemetry.jsonl retroactively; this stream starts with the updated supervisor. When freezing/copying steering scripts, keep campaign_monitor.py, campaign_telemetry.py, campaign_dashboard.py and plot_campaign_rates.py together in the same directory. There are no new Nix/Python dependencies.

--prepare-from SOURCE --queries NEW.json --attach FILE ... stages a new, verified query document (schema v2, only the six native row fields, owners drawn from the source selection) beside the copied owner payloads, and copies each attached planner receipt read-only into inputs/; the input receipt and the printed plan record every attachment's name, size and SHA-256, and the plan's entry_plan_receipt names an attached entry-plan-receipt.json. With --queries the default order is preserve; the planner's order is data.

For another supplied snapshot, stage_saved_owner_campaign.py --help describes the generic staging helper. Its default is byte-preserving. Optional --anchor-max-numerator-rank R appends one ordinary owner-orthant query per selected manifest owner, in manifest order, with no positive-power or D bound. Add --anchor-max-positive-power A to bound anchor positive power; optionally restrict that bound to --anchor-positive-power-owners MASK1,MASK2 (otherwise it applies to every anchor). These parameters are input data, with no loop-specific defaults. All original query objects and their order remain unchanged, and queries-original.json retains the exact supplied bytes. The receipt and production plan record the additional obligations. Anchors are not certified coverage, and escaping descendants remain ordinary required work. Native admission remains authoritative, including for original diagnostic queries. Choose a fresh staging/campaign destination: input planning never changes an existing campaign or its checkpoint/resume policy.

The Nix flake also exports campaign, campaign-production, campaign-monitor and campaign-stage apps. See the driver documentation for resource accounting, genuine native resume and low-level command examples.

Profiling controls and walk audits

These scripts measure and check completed walks; none of them launches a campaign or claims closure.

• walk_control_matrix.py runs a JSON matrix (rustred.walk-control-matrix.json.v1; cases name an executable, manifest, queries, owner base, workers, CPU range such as 192-197, publication policy, optional inspection workers, extra supervisor options, RAM ceiling and checkpoint interval) sequentially through shared_owner_campaign.py under nice -n 5 taskset -c CPUS. Each case gets command.json, the supervisor run/ receipts and summary.json (whole-command wall/CPU/max RSS from wait4, native preparation and traversal seconds, inspections by phase, aliases, events, max scheduled finite rank, checkpoint seconds, peak sampled RSS, exit status); the matrix gets RESULTS.md and matrix-receipt.json with executable and input digests. It refuses CPUs outside its own affinity mask; --dry-run only writes commands and --audit streams the audit below over every result.
• audit_owner_domain_walk.py streams a possibly multi-gigabyte result.json once with bounded memory and writes audit.json: aliases resolve to same-phase, same-owner completed native representatives; Apply/Route statistics are consistent (zero problems and missing routes, successor sums); queue, ledger and pool are drained; zero frontiers; initial and partial-anchor obligations discharged; input queries preserved; the durable checkpoint manifest matches the report. Any violation gives a nonzero exit. It checks recorded completion, not IBP identities or family termination. A --resume run may report the in-flight attempts its paused session returned but never published (resume_reinspects_unfinished_part); each must be published natively in the result, and they are the only allowed uncommitted entries. After a resume the pool's returned_inspections must be at least native records plus carried attempts: results that were finished but unpolled at a periodic save, followed by a crash, count once more; the surplus is reported as resumed_unpublished_returned_inspections. Every alias is also re-checked to be an exact integer-set subset of its direct representative and every partial record's D >= cut slice of its anchor (interval predicate, cross-checked by lattice-point enumeration on small sets); accepted_events must equal stats.events per record; frontiers, failures and refusal provenance must be explicit in records (parity); no record with a frontier or error may be descendant_closed; helper roots and physics queries are reported separately (--helper-pattern). --require-closure additionally requires every root and record closed. Edge-based closure and reference re-inspection coverage (F10) are checked by rustred walk-verify-closure --command RUN/command.json.
• oracle_mutation_matrix.py injects the six oracle mutations (dropped edge, retargeted alias, dropped frontier record, seal with a frontier, seal with an error, injected false hit) into a drained run and a frontier-bearing run, for the audit (mutated result.json copies) and for rustred walk-verify-closure --mutate; every mutation must FAIL and the unmutated baselines must behave as required.
• ready_resume_control.py is the fresh-process Ready multi-prefix resume gate: one control argv run as an uninterrupted Ready baseline, as a run with RUSTRED_WALK_DIAGNOSTIC_PAUSE=ready-multi-prefix (the walk force-saves and exits 4 at its first state with two unfinished accepted prefixes and a published hole) and as a --resume of that checkpoint in a new process. PASS needs the pause evidence (receipt, manifest label, journaled trigger), both audits and native inspections and events within 2% of the baseline. --baseline OLDOUT/baseline reuses an earlier run's baseline only when its argv matches apart from transport options and OLDOUT/report.json records a 0 exit under the same binary SHA-256. Neither campaign supervisor (shared_owner_campaign.py, rustred campaign shards) passes the variable on, and the walk refuses it on a --resume.
• compare_walk_records.py compares two reports: --mode strict (Ordered, old vs new binary) demands identical record geometry, native/guard/dependency counters and outcomes apart from timing, checkpoint bookkeeping and scheduling diagnostics; --mode multiset (Ready or cross-policy) demands equal multisets of (phase, owner, lower, upper, rank, power_bounds, outcome) and native counts within --native-tolerance. --shape discharged (default) makes the outcome policy-independent (error-free, obligation discharged natively or by delegation); --shape kind adds the record kind for same-policy runs; --shape geometry drops the outcome. Measured on the FG baseline: the Ordered walk and its repeat are strictly identical (98,909 records), while the Ready walk holds 28 fewer logical records, so cross-policy multisets differ even in geometry; a difference is a finding, not closure.
• heartbeat_metrics.py is the offline twin of the supervisor's derived status block (see above).

nix develop --command python examples/python/walk_control_matrix.py matrix.json \
  --output TMP/fable51-controls/new-binary --audit
nix develop --command python examples/python/compare_walk_records.py --mode strict \
  TMP/fable51-controls/baseline/fg/run/result.json TMP/fable51-controls/new-binary/fg/run/result.json

Other examples

one_loop_single_mass_vacuum.py uses the public import rustred API to generate, cold-inspect, and apply the complete one-loop tadpole artifact:

maturin develop --release --features extension-module
python examples/python/one_loop_single_mass_vacuum.py

It asserts the single generated source and guarded rule, then reduces I(3) to the tadpole master I(1) with the expected common-mass power.

two_loop_single_mass_vacuum.py uses the public import rustred API to generate, inspect, and apply the complete two-loop closing artifact.

uv venv .venv
. .venv/bin/activate
maturin develop --release --features extension-module
python examples/python/two_loop_single_mass_vacuum.py

The script asserts four generated source rows, five guarded rule cells, and the two expected master/mass-power pairs before printing the generation and reduction TOML documents.

three_loop_k6_foundry_campaign.py runs the separate K=6 foundry investigation through the release Python API. Its default is the reviewed external-search-hint document; --mode autonomous selects the strictly no-hint control:

uv venv .venv
. .venv/bin/activate
maturin develop --release --locked
mkdir k6-external-run k6-autonomous-run
python examples/python/three_loop_k6_foundry_campaign.py \
  --mode external-hints --n-cores 4 \
  --output k6-external-run/report.toml \
  --measurements-output k6-external-run/measurements.toml \
  --artifact-output k6-external-run/artifact.rribp
python examples/python/three_loop_k6_foundry_campaign.py \
  --mode autonomous --n-cores 4 \
  --output k6-autonomous-run/report.toml \
  --measurements-output k6-autonomous-run/measurements.toml \
  --artifact-output k6-autonomous-run/artifact.rribp

The external document carries 55 raw anchor/axis rectangles. Authenticated K4 routing reduces them to their semantic representatives during the run; no recurrence row, right-hand side, coefficient, support, owner, terminal, or master value is present in the input. These are deliberately sizable bounded release experiments, not quick smoke tests, and an incomplete report is a valid non-closing result. The optional artifact file is created only after exact closure, deterministic encoding, and an independent cold-load replay succeed. All three destinations are create-if-absent and require an existing parent directory, so each release run should use a fresh directory. Omitting --output sends only the semantic report to stdout; measurements are emitted only when --measurements-output is supplied and are never concatenated with the report.

k6_closing_artifact.py generates the completed autonomous K=6 artifact through rustred.family_close, then cold-inspects the written bytes and applies the generic reducer. The mathematical family is supplied in three_loop_k6.toml, with the same mathematical family and propagator order used by Vakint. Its identifier-safe metadata name rustred_three_loop_unit_mass_vacuum_k6_v1 intentionally changes the old hyphenated-name fingerprint: regenerated Vakint assets and their loader identity must migrate together. Terminal keys are unchanged. All 64 sectors are requested; the input contains no authored IBPs or oracle hints.

After building/installing the release Python extension, generation needs no specialized Rust example binary or recompilation:

python examples/python/k6_closing_artifact.py k6.rr --generate --workers 6
python examples/python/k6_closing_artifact.py k6.rr

The first command refuses to overwrite any existing artifact and requires its parent directory to exist. Generation reports 38 solved sectors, 26 proved-zero sectors, 623 generated rules and 5,640 installed rule cells. Both commands check the 6-coordinate family, 38 terminals, and the 30-term exact reduction of [2,1,1,1,1,1]. The second command consumes supplied bytes only: no generation, compilation, or FORM dependency. Run it as a separate process to demonstrate fresh-process cold loading after generation. --workers controls generation only and defaults to one.