Skip to content

Latest commit

 

History

History
383 lines (344 loc) · 28.7 KB

python-notebook-status.md

File metadata and controls

383 lines (344 loc) · 28.7 KB

Python API and Higgs-plus-jet notebook status

Snapshot: 2026-10-06, after native publication and supplied-boundary WASM acceptance. The Python API and native Higgs-plus-jet numerical milestone have passed; the broader AMFlow/DiffExp implementation is not feature-complete. This page distinguishes the demonstrated calculation from remaining generality and packaging work.

Plan item Implemented and checked Remaining acceptance
Descriptive Python API Optional PyO3 bindings in this crate, registered under symbolica.community.hep.integration; shared native HEPKit/Symbolica objects, scoped reductions, precision evidence, progress/cancellation and generated stubs. New core bindings expose supplied finite endpoints, exact rational endpoint constraints, native cut-diagram evaluation, explicit deformed propagator slots, prescribed physical continuation and general dense registered-root connections These newer interfaces are not in the frozen publication wheel; constrained registered-root endpoints support Gaussian-rational leading constants, while other coefficient fields and dimensional-sector selection remain open
Unified numerical evaluation Automatic AMF, Euclidean FT, finite-epsilon sampling and Laurent reconstruction share the series machinery used for physical transport Full AMFlow/DiffExp generality and performance parity remain open; see the comparison table
Original AMFlow acceptance All four two-loop paper targets through epsilon zero passed the requested 20-digit comparison and independent sampling/refinement That completed gate does not establish full upstream feature coverage
Growing boundary cache Binary persistence, compatible-source selection, intermediate points, source fingerprints and retained uncertainty; completed samples are stored separately. Final-wheel forced recomputation, interruption/resume and nearby reuse passed. Warm controller persistence improved from 49.88 s to 3.35 s Further serialization and source-preparation performance work remains
Exact Higgs-plus-jet inputs Native physical families, certified supplied 48/61 canonical basis maps and parent connections, exact normalization and kinematics-dependent form-factor projections Top-level RustRed derivation of those parent connections is not demonstrated; general multiloop reduction coverage remains open
Fresh physical boundaries and amplitude All 16 native 30-digit configurations from empty numerical caches, all 4,360 transport coefficients, eight W/Z form factors and three coherent observables passed. Independent 40-digit regeneration, interruption/resume, uncertainty consistency and nearby-point checks also passed External EW-square comparison remains limited to 19 relative digits; no matched full-amplitude reference timing
Independent Euclidean anchors Publication wheel passed all 545 comparisons at 20 digits with fresh native boundaries This prerequisite alone does not certify the physical calculation
Marimo notebook symbolica-community/examples/hep/gg_hg.py shows the native transport, projection, amplitude and binary-cache API calls directly. Supplied-boundary native cold/warm and actual single-core Chromium runs pass; code and populated results are visible, with no browser console errors Fresh boundary generation and cancellation retain separate native/controller acceptance; browser files live only for the current page
Packaging The publication-era Rust core suite passed 534 tests, with 12 opt-in tests ignored; newer library release coverage passes 651 tests and one doctest after a test-only schema repair. Strict Python-feature Clippy and formatting passed. Original native wheel component suite: 206 passed, one inherited C++ export failure. Corrected tensor stubs: seven checks passed. Isolated tensor fix passes native and actual Pyodide checks with/without NumPy Existing full browser wheel still contains the inherited tensor list-output bug; its fix has not been integrated into the frozen host. The whole browser suite has not been rerun with that fix

Live demonstration with supplied starting values

The newer live-notebook work packages the independently refined native starting values instead of regenerating them interactively. The community data commit contains all sixteen configurations in a 1,054,943-byte gzip bundle. Every number retains its exact dyadic value, working precision, original comparison error and 40-digit input cap. A mathematical fingerprint checks the ordered bases, equations, normalization and root conventions before admission through the ordinary supplied-boundary API. Source-sensitive binary caches retain their separate compatibility requirements. No destination integrals or amplitudes are included in the numerical bundle.

The native import adapter passed its focused Python, cache and stub gates. The portable-number experiment verified all values and errors in actual single-core Pyodide in 0.78 seconds, excluding startup/downloads. That experiment checks numeric portability, not browser transport, amplitude assembly or native-cache admission. A subsequent native shared-host controller run passed all 4,360 transported coefficients, eight form factors, three observables, binary restart and warm reuse from the shipped starting values. Import took 0.66 seconds, initial transport including checkpoint I/O took 101.77 seconds, and warm transport/amplitude took 0.52/2.68 seconds on one core. That development extension predates the tensor optimization below; its initial amplitude stage took 38.38 seconds. The subsequent WASM acceptance below uses a different, explicitly identified wheel with the staged contraction optimization.

The initial physical transport comparison uses the same frozen native executable, exact starting values and growing cache across all sixteen physical configurations:

Initial guard/order All sixteen transports Final working bits/order
60 / 96 270.213 s 349 / 128
40 / 64 163.129 s 282 / 96
30 / 32 91.512 s 249 / 64

Requested accuracy remains 20 digits, with the adaptive checks unchanged. Two exact-rational checkers verify all 4,360 final coefficients, inserted cache points, eight form factors and three observables. The reported observable accuracy estimates remain 35/36/47 relative digits. The separate EW reference still supports 19 comparison digits. Each profile has one native run on the same shared host; setup, Python/UI overhead and checkpoint I/O are excluded. That transport experiment also measured about 34 seconds for first-use amplitude kernel construction and about 2.5 seconds for projection and numerical observable evaluation. These are native component measurements, not end-to-end notebook or WASM timings.

A subsequent native tensor-contraction optimization reduces first-use kernel construction from 33.00–33.08 to 8.53–8.60 seconds in two single-core comparisons. It runs the existing Idenso simplifier before expansion and again afterward, requiring completed contractions before scalar extraction. All three symbolic kernels are exactly identical; observable values, propagated errors, arithmetic-change bounds and binary precisions are unchanged. The native amplitude regression, both feature configurations of strict release Clippy and formatting pass. This is a separate constructor measurement, not a new complete notebook or browser timing.

A subsequent smaller-order comparison reduces the native solver time to 68.106 seconds with guard 20/order 16. All 4,360 final coefficients, 8,720 inserted coefficients, eight form factors and three observables pass independent exact-arithmetic checks. The master accuracy estimate remains 28 digits and observable estimates remain 35/36/47. A guard-10 trial took 64.383 seconds but left the master estimate at 20 digits; the notebook retains guard 20. The complete native controller rerun passed with 75.295-second initial transport including checkpoint I/O, 0.626-second seed import and 0.498-second warm transport. That extension predates the staged amplitude optimization; these are not browser measurements.

The complete actual Pyodide calculation now passes on one CPU without license credentials: all 4,360 master coefficients, eight form factors and three observables, exact cross-process restart and warm reuse. An independent host checker uses exact integers and rational numbers. The sixteen unique initial calls take 275.747 s, including checkpoint I/O; first amplitude assembly takes 24.113 s. Warm transport/amplitude take 1.477/7.776 s. These Node Pyodide measurements exclude rendering/downloads and keep substantial benchmark verification/serialization costs separate.

The code-first notebook validation also passes. The notebook shows system.evaluate(cache, destination, root_sheets, options=...), native form-factor projection and amplitude evaluation directly, without a stage-button dashboard. Editing kinematics preserves model/kernel preparation and the growing cache. Tables wait for numerical work to finish, resolving earlier Marimo RPC timeouts. Actual Chromium completes the page and binary repeat in 339.705 s on one CPU, with 280.67 s displayed transport time, populated result tables and no console, page or network errors. The browser wheel is pinned to RustFlow d81dae9; it evaluates supplied boundaries live and does not generate them in the browser.

Native execution of the visible cells separately passes all scientific checks, with 76.070 s cold transport and 1.354 s warm transport, sixteen exact hits and zero ODE steps. That run uses the older 7096ba8 development extension; full app.run() reconstructs its older amplitude kernel, so whole-notebook timings are not warm-reactivity measurements or a matched native/WASM comparison. The notebook keeps guard 20/order 16 for physical transport. Full native AMF boundary generation, forced recomputation and cancellation remain covered by the separate long acceptance runner.

Frozen publication run

The final publication run completed all 16 physical starting configurations at 30 verified digits, with 1,248 completed finite-epsilon samples. Fresh boundary generation took 6,467.54 seconds. The cold physical calculation also passed transport and native amplitude comparison, with the same 25/26/47 propagated relative-digit estimates as the earlier build. The complete publication acceptance subsequently passed independent 40-digit regeneration, all uncertainty checks, interruption/resume and nearby reuse. The root process exited with code zero; final runtime attestation and post-completion source hashes passed before any presentation or compatibility follow-up was applied.

The independent scope audit distinguishes the exact supplied parent equations from computed numerical data. Generic common-mass transformations use those certified equations to close the parent flow. Native region enumeration, recursive boundary-family IBPs, analytic terminals and finite-epsilon reconstruction then generate the boundary values. This route admits supplied symbolic reduction results; it does not establish that RustRed independently rederived all 48/61 parent master equations.

Completed cold calculation and precision scope

The historical cold report records native generation without reference-seed fallback, with two independent sample sets per source and 1,248 completed finite-epsilon samples. All 4,360 transport coefficients passed the 20-digit mixed absolute/relative comparison. The native propagated relative-digit estimates were 25 for the coherent EW square, 26 for interference and 47 for the infinite-top HEFT square. Reference uncertainties support only 19, 20 and 39 relative digits respectively; the EW reference comparison therefore does not independently establish 20 digits.

Stage Historical build Publication build
All 16 fresh native boundaries 7,399.44 s 6,467.54 s
All 16 physical transports 405.37 s 400.07 s
Native amplitude assembly 50.75 s 35.96 s
Binary reload and repeated transport 75.39 s 73.88 s
Warm transport, 16 exact hits and zero ODE steps 69.44 s 68.43 s
Warm amplitude assembly 4.06 s 4.03 s
Resumed independent 40-digit boundaries 10,317.13 s 5,503.79 s
Refined transport 426.26 s 412.67 s
Refined amplitude 51.64 s 36.88 s
Nearby transport, all 16 reuse accumulated points 374.08 s 363.54 s
Nearby amplitude 4.23 s 4.02 s

The historical build subsequently completed its entire acceptance with exit code zero. All sixteen independent 40-digit boundary fits passed; all 4,360 transported coefficients were consistent with the cold run's uncertainty estimates, and the refined form factors and observables passed their checks. Resumed boundary generation took 10,317.13 seconds after a 299.42-second intentionally interrupted stage; refined transport and amplitude took 426.26 and 51.64 seconds. Nearby transport reused accumulated physical points for all sixteen configurations, inserted 32 points, and took 374.08 seconds, followed by 4.23 seconds for amplitude assembly. Completed sample hashes survived the interruption and resume; final runtime attestation passed. The publication wheel now independently passes the same complete gates: 1,248 initial and 1,376 refined samples, all 4,360 stable coefficients, all eight form factors and three observables, and 32 points inserted by nearby transport. Each report retains its own runtime provenance.

These observations used four boundary workers sharing a 256-sample-worker budget on a busy cluster. Warm controller timings include persistence after each query; they are not isolated cache-lookup timings. The cache audit identified repeated serialization. A subsequent copied-bank measurement took 0.268 seconds for sixteen exact evaluations without saving, 56.221 seconds for sixteen unchanged saves and 55.918 seconds for the original combined controller path. Values, errors, provenance and cached evidence agreed, and the original archived files stayed unchanged. This one sequence ran before amplitude preparation in a fresh process on a busy host; it measures a persistence bottleneck but establishes no optimization speedup. A matched full-application reference timing is not yet available.

The integrated cache-save optimization reuses native Symbolica exports within each snapshot. Four new release tests, forty existing cache integration tests, formatting and strict release Clippy pass. A paired profile with the real canonical topology and synthetic numerical values measured 6.23–20.47× faster payload serialization with byte-identical output. These are isolated serialization results, not a notebook speedup; the publication runtime remains unchanged. The review also produced a standalone Symbolica export-race reproducer. Its validated owner fix is proposed in Symbolica PR #52 and is not part of the tested runtime. Native import/export regressions, standalone reproducers, generator reentrancy checks and strict Clippy passed in isolation.

The integrated native-arithmetic follow-up uses official Symbolica's explicit rounded scalar operations in the complex arithmetic layer. It preserves the working precision and operation order; 24,192 bit comparisons, 76 selected regressions, formatting and strict Clippy passed. Its report makes no new scientific acceptance or performance claim, and it has not changed the publication runtime.

Both changes are now published on RustFlow main, with their own PRs closed after integration. Their combined release gate passed 541 unit/integration tests and one doctest, formatting and strict Python/stub-generation Clippy. The subsequent linear/endpoint gate passed 74 targeted integration tests and the same static checks. These newer library validations do not replace the frozen publication-wheel scientific run.

The interactive notebook check served the unchanged notebook with native Marimo and clicked transport, amplitude, binary reload and repeated amplitude controls in Chromium. All completed without browser errors. A separate panel capture confirmed the populated transport and eight-row form-factor tables and native diagram display. Both checks used copies of the committed cold bank; its originals, runtime and steering sources were verified unchanged. Click-to-completion timings include UI refresh and persistence, and are not isolated solver measurements.

A documentation-only notebook follow-up was validated at community commit 059ec1faf663e38ccb4def8bfeb368eb7b5edc8e, on codex/gg-hg-notebook-finalization. Its exact-input provenance, comparison precision and explicit loop measure passed static Chromium rendering and notebook smoke checks. The measure display was visually inspected. This empty-cache presentation check starts no numerical work. Its wording and the separately tested Python compatibility changes were applied after the full numerical run's final attestation and source rechecks.

Publication and remaining work

RustFlow milestones and validation reports are pushed to its repository. The community notebook/API changes and dependency pins were squash-merged through PR #19, which is closed. Community main commit 66a67bb0bc7b01ae53757c274aba92fc2ef234f8 is authored and committed by Ben Ruijl ben@ruijl.ch. It selects RustFlow 92cfc9d, Symbolica community 6defcca9 and HEPKit owner b96600b, and includes regenerated integration stubs. GitHub retained the original PR author during its squash, so the new main tip received a leased metadata-only correction; its tree, parent and message are unchanged. The full numerical run used 8758b93d1e1a6fed2aace41493a91be569f98a98 and its earlier native graph; its extension bytes and acceptance data remain historical. Presentation, polling compatibility and documented CI repairs include matching native graph bindings for Python 3.10. The newer controller also avoids repeated binary writes after exact transport hits, as measured below. Package release and browser follow-ups remain separate. The notebook's repository-relative destination is examples/hep/gg_hg.py; the local publication checkout is /common/dev/symbolica-community/loop-integration-publication.

The host-update report records the rebuilt wheel, shared dependency checks, native graph identity, regenerated stubs and six typing fixtures. All 57 focused gg→hg notebook/controller checks pass. The CI-equivalent suite recorded 1,146 passes and 172 skips; its one failure was a repository source check following a machine-local development symlink. That check now excludes external sources and all five affected tests pass. GitHub's Linux/macOS Python 3.10 jobs were still running at merge time.

A broader optional notebook run with Marimo preinstalled is not green: 131 initial failures/errors reproduce as API getter/slot mismatches or stale cell arguments with both old and updated wheels. The separate source-guard failure was corrected. These optional examples and their later physics need follow-up; they do not invalidate the separately checked gg→hg notebook. The full scientific calculation has not been repeated on the new dependencies.

The controller persistence comparison uses the unchanged publication extension and copies of its completed 80-entry bank. Three alternating runs on one CPU reduce median warm transport from 49.88 s to 3.35 s by replacing sixteen full-bank writes with one. Every run checks all 4,360 coefficients, uncertainties, root sheets and provenance for exact equality; binary reload preserves the full bank evidence. There are zero ODE steps and no inserted points. This is a controller-stage improvement, not a numerical-kernel or upstream speed ratio. The original scientific banks and acceptance reports are unchanged. All 51 lightweight controller tests and six notebook/render checks pass, including recovery from a failed save. New points still checkpoint after each successful configuration, while the initial save also persists valid in-memory points left by an earlier checkpoint failure.

The inherited tensor list-output failure has a separate draft owner PR #126. Focused native tests pass with and without NumPy on that isolated fix. Its actual Pyodide validation also passes both output types. The live wheel has not been changed; the isolated tensor host is not a rerun of the full community browser suite.

The final wheel's forced cancellation after one fresh 40-digit sample passed; regeneration resumed in a fresh session and preserved completed sample hashes. All sixteen refined sources passed, using 1,376 new-generation samples. Refined physical transport and observable stability passed, and all sixteen nearby configurations reused accumulated physical points with preserved provenance. The complete report records the final integrity attestation; its summary includes the authoritative process exit rather than inferring success from a progress file.

Separate Python 3.10 checks found and fixed the headless runner's use of the built-in timeout exception: Future.result raises a distinct exception on that Python version. The follow-up preserves pending polling, worker exception identity and final failure progress. It passed 72 focused checks on Python 3.10 and 70 on Python 3.14 with identical native extension bytes; the difference is two tests for exception classes that become aliases on newer Python. A separate conditional tomli dependency fixes the inherited Python 3.10 test-collection failure. Both commits are pushed on codex/python310-toml-tests, ending at d36b9f4fe2b2b0766f313dab05237e90ea544640, and are now included in the prepared publication branch. The final applied checkout also passed all 70 focused Python 3.14 controller, anchor and notebook checks. See the compatibility evidence and applied-steering checks.

The full Python 3.10 suite before the polling fix recorded 1,095 passes, 36 failures and 172 skips. Its source-provenance audit identifies 29 missing native Linnet dependency failures, inherited documentation and accessor issues, an unrelated showcase symlink issue, the polling defect and the known C++ export defect. These are being repaired or tracked separately; the focused green checks do not mean the entire community suite is green. The subsequent documentation/accessor repair passed 653 documentation examples and 32 focused checks, retaining only the same 29 missing-Linnet failures in that documentation run. API signatures were unchanged.

The subsequent native graph packaging milestone builds the standalone Linnet Python wheel from the exact HEPKit owner used by the shared extension, checks its source pin and wheel/native hashes, and installs only that local artifact with no dependency or index lookup. This is not the unrelated PyPI package with the same name. An isolated Python 3.10 installation passed native Graph/Subgraph identity, physical selection, rendering and cycle collection, plus all 682 documentation checks and 11 package guards. The full previous Linux CI had 1,106 passes and 30 failures: those 29 missing-Linnet cases and the inherited Symbolica C++ complex exporter. The subsequent Linux/macOS workflow passed native graph build/identity checks and 1,145 tests on each platform, with two failures each. One is the inherited exporter defect; the other was an offline-rendering check that assumed no earlier test had imported Linnet. Community 5cde08fd runs that unchanged checker in a fresh interpreter, preserving all no-import assertions. All 17 graph/display tests passed locally with Linnet deliberately imported in the parent. The follow-up workflow passed 1,146 tests on Linux, with only the inherited exporter failure. macOS passed 1,145 tests and exposed a second failure: the interruption test's mock writer could publish an empty sample before finishing its payload. The native writer already uses atomic replacement. Test-only commit 330f0a4b follows that contract, explicitly keeps a partial temporary invisible to the monitor and retains all hash-preservation assertions; all 51 controller tests pass locally. The follow-up run passes 1,148 tests on each platform, with only the inherited exporter failure on each. Rendering and interruption fixtures pass both full platform suites. Native numerical dependencies and the completed publication runtime are unchanged, and the whole community suite is not yet green.

The later general registered-root Python milestone adds exact roots declarations to the dense KinematicTransport constructor, including coupled epsilon-independent matrices. Its isolated release gate passes 43 tests covering analytic values on both sheets, prescribed paths, cache restart, nearby-source selection, finite endpoints, cancellation and native stubs. It uses the existing native algebraic transport owner and does not modify the frozen community extension. The newer rational correlated endpoint constraints are currently Rust-only.

Broader work remains on general linear/cut recursion, constrained algebraic singular endpoints, causal paths, nondiagonal epsilon transformations, advanced Python solver access, and performance across the reference benchmark suite. Numerical success for the demonstrated families is not a claim of arbitrary-family parity.

Evidence for this packaging milestone is in the native core report, wheel component report, Euclidean anchors, initial notebook export, native diagram display, tensor stubs and browser checks.