From 35b33b8aaa4bd87f98d7eb0c11964190acbce100 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Roberto=20Di=20Remigio=20Eik=C3=A5s?= Date: Fri, 18 Sep 2026 18:44:46 +0200 Subject: [PATCH 01/64] =?UTF-8?q?docs(parallelism):=20=F0=9F=93=9D=20add?= =?UTF-8?q?=20rank-local=20OpenMP=20design=20and=20implementation=20plan?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Assisted-by: Pi:gpt-6-astra --- .../plans/2026-09-18-rank-local-openmp.md | 1633 +++++++++++++++++ .../2026-09-18-rank-local-openmp-design.md | 321 ++++ 2 files changed, 1954 insertions(+) create mode 100644 docs/superpowers/plans/2026-09-18-rank-local-openmp.md create mode 100644 docs/superpowers/specs/2026-09-18-rank-local-openmp-design.md diff --git a/docs/superpowers/plans/2026-09-18-rank-local-openmp.md b/docs/superpowers/plans/2026-09-18-rank-local-openmp.md new file mode 100644 index 00000000..52c1165d --- /dev/null +++ b/docs/superpowers/plans/2026-09-18-rank-local-openmp.md @@ -0,0 +1,1633 @@ +# Rank-local OpenMP Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or +> superpowers:executing-plans to implement this plan task-by-task. Read both documents completely; explicit full-plan +> approval is required before execution. Do not infer missing contracts. Checkboxes track future work. + +**Goal:** Replace thread-owned partitions with one compact operator store per MPI rank and OpenMP worksharing, accepting +the replacement only after correctness, runtime, and peak-memory parity have been demonstrated. + +**Architecture:** Keep the packed position-list rows, keyless hash index, inverted index, graph representation, and MPI +routing algorithms/settings. Setting the router's partition dimension to one can change historical physical-rank +ownership under splitmix routing; Task 10 makes that compatibility consequence explicit. Workers traverse frozen +structures or update preallocated disjoint ranges; the controlling thread owns structural publication and MPI. Prototype +fixed-graph replay first, then deterministic graph construction, and delete the old runtime last. + +**Tech Stack:** C++23, OpenMP CXX, existing Boost and optional MPI, nanobind, Python 3.11+, uv/scikit-build-core, +Boost.Test, pytest. Monoprop's direct hwloc dependency is removed with the legacy partition runtime. + +**Spec:** [Rank-local OpenMP design](../specs/2026-09-18-rank-local-openmp-design.md). Read both documents before +execution. + +**Status:** Planning only. Refreshed against branch `perf/linear-routing-on-wire`, HEAD +`290112c8289ab8015eb9a2c7651ac409839c5a88`. The owner approved removing monoprop's direct hwloc dependency and making +process/thread counts and placement user responsibilities; the full refactor is not authorized for execution. No +implementation, builds, tests, or benchmarks were performed in preparing this plan. All future code/test/build commands +below are planned, not executed. The refresh authorizes only these two Markdown documents. EXISTING labels name current +contracts; PROPOSED labels name work requiring full-plan approval. + +## Global Constraints + +- C++23; retain the GCC 14 / Clang 18 minimum compiler versions and Python 3.11 floor. +- MPI remains optional and OFF by default; retain working non-MPI wheels. +- OpenMP becomes a required implementation dependency for the replacement runtime; support Linux x86_64, Linux aarch64, + and macOS builds covered by the current project. +- Each MPI rank owns exactly one operator store and one graph; threads are never communicator ranks. +- Remove monoprop's direct hwloc dependency and topology/affinity machinery; do not replace it with another + library-owned hardware-discovery or pinning mechanism. +- Users choose P processes and T threads per process and configure resource allocation, binding and pinning; monoprop + does not clamp T to physical cores or validate NUMA placement. +- Preserve packed operator rows, the keyless hash index, fixed uint32_t TermIndex, sparse state storage, and + inverted-index cosine recomputation. +- Do not introduce boost::concurrent_flat_map, hidden shards, a dense duplicate store, a second persistent key store, a + full per-thread operator, or full-vector coefficient double buffering. +- Preserve Majorana and Pauli bases, Heisenberg and Schrödinger pictures, cutoffs, graph build/replay/paring, gradients, + partial contraction, copying, and initial-operator updates. +- MPI calls execute on the controlling thread outside library-created OpenMP worksharing regions; retain + MPI_THREAD_SERIALIZED rather than changing the initialization contract. +- No exception escapes an OpenMP structured region; distributed operations fail coherently rather than leaving peers in + collectives. +- No global omp_set_num_threads/omp_set_dynamic/omp_set_nested mutation; each worksharing region receives an explicit + per-object thread budget. +- Use bitmap traversal or anticommutation pass for the existing operation; reserve prefix offsets for cumulative counts + used to allocate output ranges. +- Do not change numerical tolerances, truncation rules, generator order, source ordering, or MPI routing to make a + regression disappear. +- Do not declare success without target-hardware runtime and peak-memory measurements at the same rank count, CPU + allocation, workload, and fixed 32-bit TermIndex as the baseline. +- No automatic regression allowance: the acceptance target is candidate/baseline <= 1.00 for the median runtime and + median peak memory of every required workload cell. Noise triggers more measurement, not a silent waiver. +- Update AGENTS.md, README.md, and affected user/contributor docs in the same implementation changes that alter their + documented behavior. +- No permanent legacy/new-backend selector. Temporary coexistence is confined to the development branch; remove it + before accepting the replacement. + +Additional execution rules: + +- **Do not start until the owner authorizes implementation.** Approval must include the proposed compatibility policy, + required OpenMP dependency, floating-point policy, and strict performance gates in the design. If any is rejected, + revise these documents first. +- P (called R in existing code/benchmark geometry) is the user-selected process count; T is the requested threads per + process, normally set with `OMP_NUM_THREADS`. Users must allocate enough resources and configure rank/thread binding + with OpenMP, `srun --cpu-bind`, and/or `mpiexec` options. Each rank's workers are assumed to fit its allocation within + one NUMA domain. Do not create a topology service/NUMA scheduler, silently change P/T, or attempt to prevent + user-created oversubscription. +- Do not touch pre-existing `.codegraph/` or `.direnv/`. Use an isolated implementation checkout and a separately built + baseline. Follow the worktree/GitButler skills for version-control writes; no raw git write commands. +- New behavior follows RED → minimal implementation → GREEN → diff review. Run tests before production edits and record + actual results. Preservation fixtures can already pass after an earlier task; do not manufacture a failure or remove + completed plumbing. Each threading task also needs a kernel-specific worker-participation check, not only equal + serial/threaded numerical output. For configuration/documentation use a failing integration check. +- Commit only when authorized, after the task gate. Use `(): ` and the required + `Assisted-by: Pi:` trailer. Suggested boundaries are the numbered tasks, not every checkbox. +- Do not use a broad formatter over unrelated files. Public declarations receive a one-line summary; header + declarations/members follow the repository's documentation convention. + +## Execution map and gates + +| Stage | Tasks | Gate before proceeding | +| --- | --- | --- | +| Baseline | 1 | Reproducible numerical/performance baseline and approved policies | +| Small prototype | 2–4 | OpenMP packaging works; one-store serial and threaded replay agree | +| Shared-store construction | 5–7 | Stable queries/IDs, safety, construction scaling and scratch measured | +| Remaining evaluation kernels | 8–9 | Replay, derivative, retained closures and reductions validated | +| Integration cutover | 10–11 | Both APIs and MPI×threads pass; benchmark parity at final geometry | +| Removal/release | 12–13 | Old runtime removed, final full matrix and parity rerun | + +One writer per checkout. Kernel and storage tasks are ordered: do not have agents concurrently edit `Engine.h`, +`Evolution.cpp`, or `MonomialPropagator.inl`. Independent read-only reviews are safe. Do not batch past a failed gate. + +Expected RED/GREEN checkpoints (record real results; these are planned conditions, not claims of execution): + +| Task | RED before the task's change | GREEN required | +| --- | --- | --- | +| 1 | exactness/shape schema and new driver tests fail | complete baseline/validator evidence; no parity claim | +| 2 | missing Workshare/OpenMP exported dependency | helper and installed consumer pass | +| 3 | options overload/config/failure tests fail | serial behavior preserved; failures join/abort promptly | +| 4 | cosine/fused work-range participation remains serial; preservation tests may pass | exact coefficients and measured replay scaling | +| 5 | ordered range/window/self-stage tests fail | byte/order/ID equivalence and bounded scratch | +| 6 | new position helper/checked-boundary tests fail; Task 3 options already exist | position/hash probe and ordered serial publication | +| 7 | arithmetic-only capacity/new guard tests fail | uint32 count/ID guards; optional fill passes only if measured | +| 8 | endpoint work-range participation remains serial; preservation tests may pass | current stable derivatives and sparse transport preserved | +| 9 | named threaded reductions absent | fixed-block association approved and deterministic | +| 10 | one-store/default/compatibility tests fail | approved S=1 API/resource semantics | +| 11 | absent/incomplete evidence is not a pass | every runtime/peak cell <=1.00 after prescribed repetitions | +| 12 | static legacy/direct-dependency checks fail | runtime/topology removed LAST; parity rerun | +| 13 | clean wheel/Nix/consumer/docs checks expose missing integration | complete evidence and independent reviews | + +## Baseline-drift checkpoint and handoff + +Before any authorized implementation, record HEAD, status and these EXISTING anchors. Line numbers identify this +snapshot; symbol names are the durable anchors. If source differs semantically, stop, report the changed contract and +refresh/approve the plan before coding. Missing PROPOSED symbols are expected RED, not drift. + +| EXISTING anchor | Must match before proceeding | +| --- | --- | +| `cpp/monoprop/detail/mpi/Routing.h:177–260,315–326` | boolean linear/splitmix, default linear, unsupported geometry throws | +| `cpp/monoprop/detail/evolution/layer_build/Scan.h:208–253` | `FusedScanResult`, window, precomputed `gen_shift`, two self stages | +| `cpp/monoprop/detail/evolution/layer_build/Resolve.h:90–183` | `probe_incoming_queries(incoming,op,QueryForm)`, positions and hashes | +| `cpp/monoprop/detail/evolution/layer_build/Engine.h:486–565` | position/hash deferred publication and direct bounded self probe | +| `cpp/monoprop/detail/operator/OperatorIndex.h:282–288` | five-span `find_batch_positions`, optional hash output | +| `cpp/monoprop/TypeAliases.h:39`; `cpp/include/monoprop/Evolution.h:54–63` | uint32_t; derivative has `LayerAngle` and optional `CosRecordView` | +| `cpp/include/monoprop/MonomialPropagator.h:286–305` | virtual update/clone, nonvirtual apply helper | +| `justfile:75–122,169–171,371–397`; `pyproject.toml:162–168` | labels, centralized recipes, MPI env build override | +| `benches/bench_random.py:22–139`; `benches/conftest.py:248–274,411–493` | four operation families, shape guard, existing op/outer windows | +| `packages/monoprop-bench-tools/src/monoprop_bench_tools/memory/cpu.py:75–107,213–233` | nested peak preservation already implemented | + +Handoff checklist: exact revision/binary and overlay hashes; owner approvals; last passed task; actual RED and GREEN +commands/results; remaining gate/capacity/placement blockers; fixed routing and workload manifest; raw artifact paths. +Never hand off only “tests pass”. No builds/tests/imports/installs/VCS writes are authorized by this document refresh. + +## File responsibilities + +Paths in task lists are relative to the repository root. + +| Area | Existing anchors | New files | +| --- | --- | --- | +| Runtime configuration | `detail/EnvConfig.h`, constructor/copy in `MonomialPropagator.inl`; delete `detail/partition/CpuTopology.*`, do not move it | `cpp/monoprop/detail/parallel/{Options.h,Workshare.h,ThreadBudget.h,ThreadBudget.cpp,CMakeLists.txt}` (budget validation only; no hardware discovery) | +| Replay | `Evolution.cpp`, `detail/evolution/CosineRecompute.h`, `MPFunctions.cpp` | no executor or new graph layer | +| Construction | `detail/evolution/layer_build/{Scan,Resolve,Engine,FusedApply}.h` | range helpers stay beside current kernels | +| Storage | `detail/operator/{OperatorIndex,MPOperator,InvertedIndex}.h` | no replacement map | +| MPI failure | `detail/mpi/{Comm,MPICompat,MPIUtils,Exchange}.h`, `MPICompat.cpp` | `cpp/monoprop/detail/mpi/OperationFailure.h` | +| Tests | existing suites listed per task | `cpp/tests/{openmp_workshare_tests,openmp_runtime_tests,openmp_kernel_tests,openmp_equivalence_tests}.cpp`, `tests/test_openmp_config.py`, `cpp/tests/mpi_failure_driver.cpp` | +| Benchmark evidence | `benches/conftest.py`, both benchmark-tool renderers | `tools/benchmark-rank-local-openmp.py`, `tools/rank-local-openmp-workloads.json`, `tests/test_rank_local_openmp_benchmark.py` | +| Documentation | `AGENTS.md`, `README.md`, docs listed in Task 13 | `docs/content/docs/openmp-migration.mdx` | + +`detail/...` in the table means `cpp/monoprop/detail/...`. Task file lists below use full paths. New installed headers +must enter CMake FILE_SETs, not merely compiler include paths. + +## Shared invariants: reviewer checklist + +1. `OperatorIndex::find_batch_positions` (and dense `find_batch`) is read-only **only while rows, overflow storage, and + hash slots are frozen**. `set()` is not a disjoint-row parallel setter: it touches a shared overflow map, even + through `erase` for inline rows. +2. `MPOperator::inverted_index() const` and `InvertedIndex::row_parity_words() const` can mutate caches. Prepare them on + the controlling thread before a team. Never retain a parity pointer across store growth. +3. Distinct inverted-index rows or graph records can share a packed word. Keep bitmap append and graph phase packing + serial initially. +4. Leaders finish local resolution, remote resolution/insertion, responses, and matched marks **before** follower work. + Deferred self misses are published after both passes. Physical insertion order is remote leader misses → remote + follower misses → self leader misses → self follower misses. +5. XOR by one fixed generator is injective, and source terms are unique across owners. This is the reason query + keys/targets are unique; the hash table does not deduplicate bulk input. Debug tests must verify this invariant. + Generic duplicate input to a helper requiring distinct keys must not be silently accepted. +6. Query words, source IDs and captured values form parallel streams. Merge each owner/leader-or-follower stream in + source-range order, not worker completion order. +7. Fused cosine scaling captures old source values before multiplying. Each rotation record owns its entire coefficient + pair. No coefficient atomics. +8. A graph layer's `scaled_count` currently follows post-insertion finalization. Preserve it; the pre-gate source count + is a different boundary. +9. Self derivative entries pair `k` with `k + count/2`, not `2*k` with `2*k+1`. Snapshot both required pre-update values + before writing either endpoint. +10. Gate loops, reverse-gradient loops, repeated-parameter accumulation, graph append, store publication, and MPI remain + ordered on the caller. +11. Empty local stores still follow the same distributed collective sequence. Runtime thread budgets may differ by rank + without changing ownership. +12. Ordinary kernels fall back to serial inside an existing OpenMP region. This does **not** authorize concurrent MPI + calls or concurrent calls on one propagator/retained functional. + +## Build/test command library + +These commands are to run **during implementation**, not evidence of passing tests now. Run from the relevant checkout. +Unset stale sanitizer/build-type and MPI override settings before Release commands; R explicitly uses +`monoprop_ENABLE_MPI=OFF` as well as its CMake OFF setting. A configuration change requires forced reinstall and +regenerated tests; `uv sync` alone does not reliably relink the C++ executable. For ordinary suite runs explicitly set +`OMP_NUM_THREADS=1`; this is test-runner policy, not a library multi-rank default. Dedicated parallel tests override +their object budget or launch fresh processes with a selected OMP_NUM_THREADS on an adequate allocation. + +### R: non-MPI Release + +```bash +monoprop_ENABLE_MPI=OFF uv sync --all-extras --group workspace-test --group bench \ + --reinstall-package monoprop --no-cache \ + --config-settings-package='monoprop:cmake.define.monoprop_ENABLE_MPI=OFF' -v +cmake --build --preset skbuild-release +OMP_NUM_THREADS=1 just test-cpp +OMP_NUM_THREADS=1 just test-py +``` + +For a targeted C++ gate after rebuilding: + +```bash +ctest --test-dir build/editable/Release --label-exclude mpi --no-tests=error \ + --output-on-failure -R 'openmp_|operator_index|fused_|combined_recompute' +``` + +Use `ctest --test-dir build/editable/Release -N -V` to verify discovery. A regex matching zero tests is not a pass. New +Boost cases must be flat `BOOST_AUTO_TEST_CASE` names; CMake discovers `cpp/tests/*.cpp` automatically after +regeneration. + +### M: MPI Release + +```bash +# Environment switch also adds mpi4py>=4.1.0 to the isolated build requirements. +monoprop_ENABLE_MPI=ON uv sync --all-extras --group workspace-test --group bench \ + --reinstall-package monoprop --no-cache \ + --config-settings-package='monoprop:cmake.define.monoprop_MPI_TEST_PROCS=1;2;3;4' -v +uv run --no-sync python -c 'import monoprop; assert monoprop.has_mpi, monoprop.__file__' +cmake --build --preset skbuild-release +OMP_NUM_THREADS=1 just test-cpp +# Every success matrix including three ranks uses explicit splitmix. +monoprop_ROUTING=splitmix OMP_NUM_THREADS=1 just test-cpp-mpi +monoprop_ROUTING=splitmix OMP_NUM_THREADS=1 just test-py-mpi '1;2;3;4' +# Rebuild/register a separate default-linear success matrix, excluding unsupported R=3. +monoprop_ENABLE_MPI=ON uv sync --all-extras --group workspace-test --group bench \ + --reinstall-package monoprop --no-cache \ + --config-settings-package='monoprop:cmake.define.monoprop_MPI_TEST_PROCS=1;2;4' -v +cmake --build --preset skbuild-release +monoprop_ROUTING=linear OMP_NUM_THREADS=1 just test-cpp-mpi +monoprop_ROUTING=linear OMP_NUM_THREADS=1 just test-py-mpi '1;2;4' +``` + +Run only on an allocation supporting those ranks; smaller local runs are smoke tests, not substitutes. Do not apply the +serial CTest fabric exclusions to real MPI/performance runs. `just bench-build-mpi` is the existing MPI benchmark build +recipe; `just test-mpi` is the centralized full test recipe. Verify executable mtimes after sync; use the explicit +`cmake --build` above before trusting C++ results (or update the recipe to guarantee relinking in its authorized task). +If direct CMake settings are used instead of the MPI environment switch, also supply +`--config-settings-package='monoprop:build.requires=mpi4py>=4.1.0'`. `--all-extras` supplies runtime extras, not this +isolated-build requirement. Use `--label-exclude mpi` for the non-MPI leg: the export probe is labeled `unit`, not +`serial`. Test linear R=3 rejection directly with `Router::for_modes`, not by treating a full-suite failed launch as +success. The `just bench-mpi` companion forwards extra arguments to **mpiexec**, not pytest. + +There is no alternate index-width build. `cpp/monoprop/TypeAliases.h:39` fixes TermIndex to uint32_t; +`operator_index_tests.cpp:31–34` asserts it. Inspect cache and imported binary identity for every configuration. + +### S: sanitizer profiles + +```bash +SKBUILD_CMAKE_BUILD_TYPE=AsanUbsan \ +monoprop_ENABLE_MPI=OFF SKBUILD_CMAKE_DEFINE='monoprop_SANITIZER=asan-ubsan;monoprop_ENABLE_MPI=OFF' \ +uv sync --group workspace-test --all-extras --reinstall-package monoprop --no-cache -v +cmake --build build/editable/AsanUbsan --target monoprop_unit_tests.x +ASAN_OPTIONS=detect_leaks=1 UBSAN_OPTIONS=halt_on_error=1:print_stacktrace=1 \ + ctest --test-dir build/editable/AsanUbsan --output-on-failure +LD_PRELOAD="$(g++ -print-file-name=libasan.so):$(g++ -print-file-name=libstdc++.so.6)" \ +ASAN_OPTIONS=detect_leaks=0 uv run --no-sync python -m pytest -s + +SKBUILD_CMAKE_BUILD_TYPE=Tsan monoprop_ENABLE_MPI=OFF \ +SKBUILD_CMAKE_DEFINE='monoprop_SANITIZER=tsan;monoprop_ENABLE_MPI=OFF' \ +uv sync --group workspace-test --all-extras --reinstall-package monoprop --no-cache -v +cmake --build build/editable/Tsan --target monoprop_unit_tests.x +TSAN_OPTIONS=halt_on_error=1:history_size=4 \ + ctest --test-dir build/editable/Tsan --output-on-failure --no-tests=error -R 'openmp_' +``` + +Migrate `justfile::test-cpp-tsan` from `(partition_|shm_comm_)` to retained concurrency tests plus `openmp_`, with +`--no-tests=error`, before deletion; `.github/workflows/qa-analysis.yml` invokes that recipe. Direct selection above is +a focused gate, not a replacement for CI migration. Use existing recipes in workflows; +`tools/check-workflow-commands.py` (enforced by `prek.toml`) forbids duplicating ordinary workflow build/test commands. + +ASan and TSan are separate Linux profiles. Qualify the compiler/OpenMP-runtime TSan combination using a known-racy and +known-safe small test first; unsupported runtime reports are a tooling limitation, not race-free evidence. No +stock-Python TSan gate, no blanket suppression of engine frames, and no sanitizer performance measurements. + +--- + +### Task 1: Freeze baselines and repair measurement trustworthiness + +**Files:** +- Modify: `benches/conftest.py` (exactness and shared shape preflight); preserve existing windows and timed operations + in `benches/{bench_random,bench_models}.py`. No historical node renaming or restoration of removed pytest nodes. +- Modify: `packages/monoprop-bench-tools/src/monoprop_bench_tools/{report,bmf}.py`. +- Test: `packages/monoprop-bench-tools/tests/{test_memory,test_report,test_bmf}.py`. +- Create: `tools/benchmark-rank-local-openmp.py`, `tools/rank-local-openmp-workloads.json`, + `tests/test_rank_local_openmp_benchmark.py`. +- Reference/migrate guidance: `benches/LADDER.md`; supplement with `tools/capture-baseline.py`. +- Modify: `AGENTS.md`, `README.md`, `docs/content/docs/benchmarks.mdx` where measurement commands/interpretations are + documented. + +**EXISTING:** The four pytest families in each benchmark file are `build_graph`, `propagate`, `energy`, `gradient`. All +have operation windows. `propagate` is graph-free, not a separate inplace node. Functional creation in energy/gradient +is outside their timed window. Nested peaks are preserved by `memory/cpu.py::_fold_open_windows`; keep its tests. + +**PROPOSED interfaces:** One bounded driver and shared overlay used unchanged on both arms, preserving historical node +IDs. Add independent `opmemexact[nodeid]` and `memhwmexact[nodeid]` booleans (all ranks exact in that window); absent +means unknown. The separate construction worker records its own `construction_exact`. Each fresh process writes one +kind-specific artifact; `compare` joins a timed/construction pair into one observation as specified below. The new +paring-construction measurement below is deliberate additional measurement work requiring full-plan approval, not +instrumentation-only. + +- [ ] Obtain owner approval of the full design policies, bounded new paring-construction measurement and target + allocation. Record baseline revision, CPU allocation, compiler/flags, MPI version, index width, OpenMP runtime, + allocator environment, imported extension path/hash, seed and complete workload configuration. +- [ ] Before changing engine code, separately build the baseline using R/M. Preserve its environment and binary. Apply + only the identical measurement-harness changes to both arms; record this overlay's diff/hash. Baseline + numerical/performance failures stop this task. +- [ ] Add renderer tests independently varying operation and outer exactness: true/false/absent and mixed flags. + `bmf.py::build_bmf` publishes `peak-memory` from `memhwm`, so gate it on `memhwmexact`; `operation-memory` comes from + `opmemdelta`, so gate it on `opmemexact`. `report.py::build_report` must label each corresponding metric correctly. + False means non-exact warning and no exact metric for that window; absent means legacy/unknown, never assumed true. + RED: current renderers claim exactness without flags. GREEN: no cross-window certification; time/term fields survive. +- [ ] In `OpMemory.close`, collectively reduce `HighWaterMark.exact` across every rank into `opmemexact`; independently + do the same in `record_memory` for `memhwmexact`. Retain peak/base/delta sum/max, outer memhwm/memhwm_max, opbytes, + opmembreak, opsize and existing provenance. Tiny integration checks require complete fields for all four families. No + rank-zero-only collective. Extend existing nested transient/fallback tests, not the reset implementation. +- [ ] Migrate `_require_shape` in the identical shared overlay. Add explicit manifest/CLI runtime declaration + `--runtime-shape=partitions|openmp` to pytest and driver; this is PROPOSED measurement configuration, not a backend + selector. Baseline declares partitions with `monoprop_PARTITIONS=T` (and `monoprop_NUM_THREADS=T`); candidate declares + openmp with `OMP_NUM_THREADS=T` and both library overrides unset. Require positive declared R/T and equal allocation. + Record declared and observed shape separately; a renamed `--arm` label must not bypass validation. Before acceptance, + diagnostic runs must verify actual baseline partition participation and candidate worker participation on + representative large kernels, accounting for documented small/nested/runtime-limit paths. Reject missing or + contradictory evidence. Tests: missing baseline declaration, missing candidate OMP setting, candidate deprecated + override, wrong observed R/T, mislabeled arm, and valid equal-allocation declarations. RED: current multi-rank guard + rejects candidate configuration. +- [ ] Separate whole-construction measurement is intentional policy, not a repair for nesting. A fresh untimed worker, + without pytest memory fixtures/session graphs, opens one `HighWaterMark`, constructs inputs/propagator/graph/callable + as required, runs the operation and retains outputs until close. Record peak sum/max, floor, delta and exactness. Use + existing model builders; no duplicate physics or general benchmark framework. +- [ ] Freeze an owner-approved campaign inventory BEFORE collecting baseline evidence. Its JSON schema is + `{"schema_version":1,"workloads_path":str,"workloads_sha256":str,"cells":[cell]}`. Each cell has unique slug `id`, + `profile`, `node_id`, `operation`, `measurement_kind` (`pytest` or `driver`), `identity`, `config_digest` and + `parameters_digest`. `identity` contains `ranks`, requested `threads`, `index_bits=32`, `cpu_allocation`, + `compiler_flags`, resolved `config`, and `routing` (`mode`, derived `bits`, numeric `seed`). Expand the complete + required workload/profile × approved geometry/routing inventory from the matrix below; archive its digest with owner + approval. This file, not the observations supplied later, is the authoritative expected-cell list. A full cell missing + from BOTH arms must fail completeness. Reject duplicate/extra cell IDs and mismatched identities. Do not regenerate or + shrink the inventory to make observations pass. +- [ ] `tools/rank-local-openmp-workloads.json` maps `profiles[profile][node_id]` to `operation`, `measurement_kind`, + `basis`, `picture`, complete `config`, deterministic `parameters`, and `pare_threshold`. `reference`/`tiny` contain + unpared four-family entries; `reference-pared`/`tiny-pared` contain threshold-1e-10 energy/gradient entries and both + driver construction identities. Record profile overrides explicitly; no ambiguity within one profile/node pair. Define + config/parameter digests as SHA-256 of UTF-8 + `json.dumps(value, sort_keys=True, separators=(",", ":"), allow_nan=False)`. File digests are SHA-256 of exact file + bytes. Resolve relative JSON paths against the containing file's directory. Do not put binary or placement-evidence + hashes inside cross-arm `identity`: those legitimately differ, while allocation/routing/workload must match. +- [ ] Give the driver exactly three argparse modes: `observe` requires `--arm baseline|candidate`, + `--runtime-shape partitions|openmp`, `--campaign FILE`, `--cell-id CELL`, `--sample-id SAMPLE`, `--placement FILE`, + `--output DIR`, plus optional `--whole-process`; `validate` takes the same options except + `--sample-id`/`--whole-process`; `compare --campaign FILE --manifest FILE --output FILE`. Resolve profile, historical + node ID and complete workload from the campaign cell and digest-checked workload file; reject an observed + geometry/routing/configuration that differs. The pytest overlay also takes `--runtime-shape`; never rewrite historical + node IDs. Environment/launcher/allocation stay external; the driver installs nothing. +- [ ] Define the external placement JSON consumed by `--placement`: `schema_version=1`, `artifact_kind="placement"`, + `arm`, `cell_id`, `binary_hash`, `identity`, `runtime_shape`, `declared`, and `observed`. `declared` contains `ranks`, + `threads`, `cpu_allocation` and `settings` (the exact OpenMP/library environment). `observed` contains `ranks`, + `threads_per_rank` and `cpu_allocation`, from Task 11's binary-linked representative large-kernel diagnostic. Preserve + raw launcher/affinity evidence alongside this summary. Baseline shape is `partitions`; candidate shape is `openmp`; + both must match the campaign R/T/allocation. A limited-team diagnostic cannot establish the requested full-team + benchmark cell. Small-kernel serial fallback remains allowed and reported. Each artifact records `placement_path` and + its exact `placement_sha256`; compare reads and validates that file, not just a caller boolean. Cross-arm + placement-file hashes differ legitimately. Diagnostic provenance can be reused for matching samples of the same + cell/arm/binary, never substituted for measured operation or construction artifacts. +- [ ] Normal `observe` writes `DIR/ARM/CELL/SAMPLE/timed.json`; `observe --whole-process` writes + `DIR/ARM/CELL/SAMPLE/construction.json` in a SEPARATE fresh process with the same sample ID. Cell/sample IDs are + path-safe slugs. Create files exclusively; reject overwrite. Each invocation generates a fresh unique `run_id` and + records schema version, artifact kind, arm, cell/sample IDs, campaign digest, binary hash, profile/node/operation, + identity, config/parameter digests and placement reference/hash. Keep pytest raw files in that sample's timed + subdirectory so labels cannot overwrite another sample. Timed artifacts contain runtime, operation and outer-window + fields only; construction artifacts contain construction-window fields only. Do not invent timing for the untimed + worker. `validate` writes a separate uniquely named validation artifact under `DIR/ARM/CELL`, without sample ID. +- [ ] Define comparison input as + `{"schema_version":1,"campaign_sha256":str,"cells":[{"id":str,"baseline":[sample],"candidate":[sample]}]}`. Each + `sample` contains `sample_id`, `timed_path`, `construction_path`, `validation_path`. `compare` first verifies the + campaign/workload digests and exact expected cell inventory. It then reads the three references and joins only if arm, + cell, binary hash, profile/node/operation, identity, configuration/parameter digests, campaign digest and placement + evidence agree; timed/construction sample IDs must also agree. Binary hashes need not match across arms. Require five + distinct timed runs AND five distinct construction runs per arm/cell initially, ten after prescribed repetition. + Reject missing/mismatched pairs, duplicate sample IDs within an arm/cell, reused measurement paths or run IDs, + including reuse across cells/arms. A validation artifact may serve multiple samples only for exactly the same + cell/arm/binary/configuration/parameters. Never count copied paths or five references to one run as five samples. +- [ ] The joined observation exposes `runtime_seconds`, operation `peak_sum_bytes`/`peak_max_bytes`, + `construction_peak_sum_bytes`/`construction_peak_max_bytes`, and `op_exact`/`construction_exact`. Both exactness flags + must be true for parity. Preserve `outer_peak_sum_bytes`/`outer_peak_max_bytes` and independent `outer_exact` as + DIAGNOSTICS, not parity gates. Outer false/unknown must be labeled non-exact/unknown, not upgraded by op_exact; it + does not invalidate otherwise exact operation/construction evidence. Missing outer flags in legacy raw files map to + unknown; missing required operation/construction exactness fails acceptance. Include operation/construction + floor/delta sum/max and persistent bytes as diagnostics, never substitutes for absolute peaks. Keep original raw + artifact metric names: operation peaks come from opmempeak; outer peaks from memhwm/memhwm_max. Compare exactly FIVE + per-cell medians: runtime, operation peak sum/max, construction peak sum/max. All five ratios must be <=1.00. Report + outer ratios separately; an outer-only regression cannot fail this five-metric gate. Reject missing/non-finite + required measurements rather than use zeros. Derive numerical success by reading validation products below; a + caller-supplied boolean is insufficient. Exit 0 means all cells pass, 1 means unmet gates (including incomplete + campaign/sample coverage), and 2 means malformed/incompatible evidence. Output `failed_cells`, missing coverage, five + gate ratios and separately labeled diagnostic ratios. +- [ ] Implement `validate` in a **separate untimed/unmeasured process**, reconstructing the same + model/configuration/operation with existing builders. Include the shared provenance above with + `artifact_kind="validation"` and its own run ID; record `schema_version`, `node_id`, `operation` (one of the five + operation rows below), `config_digest`, `parameters_digest`, `binary_hash`, `basis`, `picture`, finite `energy`, full + `gradient` when relevant, `global_term_count`, `graph_layers`, and `term_files` paths to decoded term/coefficient + files. Each sorted term file is JSON Lines with `{"key":[indices],"real":number,"imag":number}`; gradient arrays are + in optimizer parameter order. `pare_functional_construct` and pared evaluation artifacts omit term files because the + pared callable owns a subgraph rather than an independently exported operator; validate its scalar/full gradient + against the baseline pared callable. Use deterministic input parameters; a graph benchmark replays at those parameters + for validation. The required products are: + +| Operation | Validation products | +| --- | --- | +| build_graph | energy, gradient, decoded evolved term/value map and graph layer count after replay | +| propagate | immediate energy and decoded evolved term/value map; independent graph-replay comparison on the same inputs where both paths use the same cutoff settings | +| energy | scalar result plus graph decoded map if unpared; pared scalar against same-threshold baseline | +| gradient | scalar/full gradient plus graph decoded map if unpared; pared scalar/full gradient otherwise | +| pare_functional_construct | evaluate the returned pared energy/gradient callable at the recorded parameters; compare to the baseline pared callable at the same threshold | + +For random graph-free propagation, EXISTING `build_random_propagator(..., lower_atol=None)` is the current default; +record it explicitly. Fixed Hubbard/Pauli atols are below. Freeze separate energy/gradient profiles with +`pare_threshold=None` and `pare_threshold=1e-10`; the latter times evaluation, not construction. Pared profiles validate +the same returned scalar/full-gradient callable on both arms, not an unpared operator map. Do not compare different +truncation settings. Existing tiny/exact and finite-difference tests remain independent physics oracles; large-cell +evidence compares baseline/candidate of the identical operation, not an infeasible dense exact simulator. +- [ ] Implement exactly one NEW bounded measurement identity per picture/profile: + `driver::pare_functional_construct[heisenberg]` and `driver::pare_functional_construct[schrodinger]`. In the workload + manifest set `operation="pare_functional_construct"`, `measurement_kind="driver"`, + `functional="expectation_value_and_gradient_functional"`, `pare_threshold=1e-10`, picture, full random config and + parameter digest. Use the proposed `observe`/`validate` interface: `--cell-id` resolves this node/profile from the + frozen campaign, not a new framework or pytest family. Workload key is `(profile,node_id)`; a campaign cell also fixes + geometry/routing and gets a unique slug. Freeze tiny-pared and reference-pared entries before baseline collection. + Example planned invocation after selecting the matching cell and binary-linked placement file (launcher/allocation + supplied externally): + +```bash +OMP_NUM_THREADS="$T" uv run --no-sync python tools/benchmark-rank-local-openmp.py observe \ + --arm candidate --runtime-shape openmp --campaign "$CAMPAIGN" --cell-id "$CELL" \ + --sample-id s01 --placement "$PLACEMENT" --output "$RESULTS" +# Repeat in a separate fresh process with the same arguments plus --whole-process. +# Run validate separately; omit --sample-id and --whole-process. +``` + + Unset both library overrides before this candidate command; if launched at three ranks, explicitly export + `monoprop_ROUTING=splitmix` to every rank. Use `--runtime-shape partitions` and both legacy T declarations for + baseline. In normal driver observe, build random inputs first, then open an outer `HighWaterMark` before + propagator/graph construction (the driver analogue of pytest `record_memory`). Construct graph fully before barrier, + timer and operation `HighWaterMark(settle=False)`. Close operation then outer window, retaining graph/callable through + both; publish outer and operation exactness independently. Time ONLY + `built_graph.expectation_value_and_gradient_functional(1e-10)`, using existing barrier semantics (entry untimed, + completion included); one call per fresh process, retain returned callable until the window closes. Record time as + rank maximum and peaks/floors/deltas as rank sum/max with independent exactness. Do not evaluate inside timing. A + separate `validate` process invokes the retained callable at deterministic parameters and records scalar/full gradient + plus graph/config/binary provenance; compare the same pared callable on both arms. No fabricated term export from a + private pared graph. `observe --whole-process` opens before random inputs/propagator/graph construction, builds the + same callable and keeps graph/callable live until close; it is untimed and records construction peaks separately. + Tests instrument event order and callable lifetime, forbid evaluation during timing, check exact CLI/manifest + identity, threshold/config parity, missing outputs and numerical mismatch. RED: driver identities absent. GREEN: + bounded driver and validation artifacts complete on both arms. This retains original paring-construction runtime/peak + coverage. +- [ ] Export maps with the existing low-level `propagator._simulator.evolved_operator(parameters, 0.0)`; use an empty + parameter vector after graph-free propagation and the repeated circuit parameters for graph replay. Sort raw encoded + index tuples and store real/imaginary coefficient components in per-rank files. These exported coefficients are + currently rounded to 1e-12; state that limitation and retain unrounded scalar/gradient checks. Stream-merge sorted + rank files when comparing global maps instead of requiring historical rank ownership. For Heisenberg, the binding adds + the replicated identity/core term on each rank: retain one copy after checking agreement, **do not sum it R times**. + Other duplicated nonidentity terms fail unique ownership; Schrödinger identity follows ordinary owner semantics. + Require equal global key sets, including zero-coefficient stored terms; compare values with the existing + `test_utils::near` rule from `cpp/tests/TestUtilities.h.in`: `abs(a-b) <= 1e-9 + 1e-7*max(abs(a),abs(b))`, + componentwise for complex values and gradients. Reject missing/non-finite outputs. Preserve all stricter existing + unit-test tolerances; this benchmark comparison does not weaken them. If full validation export cannot fit, stop for + an approved smaller equal-arm profile, not an unchecked `numerical_ok=true`. +- [ ] Link every performance observation to a successful validation artifact for the same binary/configuration/operation + and compare baseline/candidate validation artifacts. Add tests with wrong energy, a changed gradient entry, one + missing/extra term, duplicate ownership, replicated identity, non-finite data, stale binary hash and absent + validation. A completed pytest benchmark with only type/length assertions is not numerical evidence. +- [ ] Test the comparator without running benchmarks: + +```python +import hashlib +import json +from pathlib import Path +import subprocess +import sys + + +def test_comparison_does_not_hide_one_regression(tmp_path): + script = Path(__file__).resolve().parents[1] / "tools/benchmark-rank-local-openmp.py" + + def encoded(value): + return json.dumps(value, sort_keys=True, separators=(",", ":"), allow_nan=False).encode() + + def digest(value): + return hashlib.sha256(encoded(value)).hexdigest() + + def save(name, value): + path = tmp_path / name + path.write_bytes(encoded(value)) + return str(path) + + config = {"gen_length": 4, "obs_terms": 16, "num_generators": 8, "num_modes": 8, + "cutoff": 6, "seed": 0, "lower_atol": None} + parameters = [0.0] * 8 + identity = {"ranks": 1, "threads": 2, "index_bits": 32, "cpu_allocation": "test-cpus", + "compiler_flags": "test", "config": config, + "routing": {"mode": "linear", "bits": 0, "seed": 0}} + expected, entries = [], {} + for name in ("build", "replay"): + node = f"energy-fixture-{name}" + entries[node] = {"operation": "energy", "measurement_kind": "pytest", "basis": "majorana", + "picture": "heisenberg", "config": config, "parameters": parameters, + "pare_threshold": None} + expected.append({"id": name, "profile": "tiny", "node_id": node, "operation": "energy", + "measurement_kind": "pytest", "identity": identity, + "config_digest": digest(identity["config"]), "parameters_digest": digest(parameters)}) + workloads = {"schema_version": 1, "profiles": {"tiny": entries}} + campaign = {"schema_version": 1, "workloads_path": save("workloads.json", workloads), + "workloads_sha256": digest(workloads), "cells": expected} + campaign_path = save("campaign.json", campaign) + cells = [] + for target, candidate_time in zip(expected, (10.1, 5.0), strict=True): + name = target["id"] + cell = {"id": name} + for arm, elapsed in (("baseline", 10.0), ("candidate", candidate_time)): + shape = "partitions" if arm == "baseline" else "openmp" + binary_hash = ("b" if arm == "baseline" else "c") * 64 + settings = {"OMP_NUM_THREADS": "2", "OMP_DYNAMIC": "FALSE"} + if arm == "baseline": + settings.update(monoprop_PARTITIONS="2", monoprop_NUM_THREADS="2") + placement = {"schema_version": 1, "artifact_kind": "placement", "arm": arm, + "cell_id": name, "binary_hash": binary_hash, "identity": identity, + "runtime_shape": shape, + "declared": {"ranks": 1, "threads": 2, "cpu_allocation": "test-cpus", + "settings": settings}, + "observed": {"ranks": 1, "threads_per_rank": [2], "cpu_allocation": "test-cpus"}} + placement_path = save(f"{name}-{arm}-placement.json", placement) + common = {"schema_version": 1, "campaign_sha256": digest(campaign), "arm": arm, + "cell_id": name, "profile": target["profile"], "node_id": target["node_id"], + "operation": "energy", "binary_hash": binary_hash, "identity": identity, + "config_digest": target["config_digest"], "parameters_digest": target["parameters_digest"], + "placement_path": placement_path, "placement_sha256": digest(placement)} + validation = {**common, "artifact_kind": "validation", "run_id": f"{name}-{arm}-validation", + "basis": "majorana", "picture": "heisenberg", "energy": 0.0, + "global_term_count": 0, "graph_layers": 0, "term_files": []} + validation_path = save(f"{name}-{arm}-validation.json", validation) + cell[arm] = [] + for i in range(5): + sample_id = f"{name}-{arm}-{i}" + timed = {**common, "artifact_kind": "timed", "sample_id": sample_id, + "run_id": f"{sample_id}-timed", "runtime_seconds": elapsed, + "peak_sum_bytes": 1000, "peak_max_bytes": 1000, "op_exact": True, + "outer_peak_sum_bytes": 1200, "outer_peak_max_bytes": 1200, "outer_exact": True, + "operation_floor_sum_bytes": 800, "operation_floor_max_bytes": 800, + "operation_delta_sum_bytes": 200, "operation_delta_max_bytes": 200} + construction = {**common, "artifact_kind": "construction", "sample_id": sample_id, + "run_id": f"{sample_id}-construction", "construction_exact": True, + "construction_peak_sum_bytes": 1200, "construction_peak_max_bytes": 1200, + "construction_floor_sum_bytes": 800, "construction_floor_max_bytes": 800, + "construction_delta_sum_bytes": 400, "construction_delta_max_bytes": 400} + cell[arm].append({"sample_id": sample_id, + "timed_path": save(f"{sample_id}-timed.json", timed), + "construction_path": save(f"{sample_id}-construction.json", construction), + "validation_path": validation_path}) + cells.append(cell) + manifest = save("manifest.json", {"schema_version": 1, "campaign_sha256": digest(campaign), "cells": cells}) + output = tmp_path / "comparison.json" + result = subprocess.run([sys.executable, str(script), "compare", "--campaign", campaign_path, + "--manifest", manifest, "--output", str(output)], check=False) + assert result.returncode == 1 + assert json.loads(output.read_text())["failed_cells"] == ["build"] +``` + +These two miniature inventory entries use empty-map energy fixtures, not real benchmark measurements. They follow the +same schema, reference loading and provenance checks as real artifacts; no synthetic-mode bypass is allowed. The +comparison test does not run physics, acquire placement, or establish actual hardware participation. Add tests for a +whole required cell deleted from both arms, missing samples, mismatched timed/construction pairs, reused measurement +files/run IDs, missing placement provenance (exit 2), each false/absent exactness flag, unequal +shape/configuration/index width, and sum-versus-max peaks. For only an outer-peak regression or unknown outer exactness, +report the diagnostic accurately while allowing the five-metric gate to pass. Operation/construction inexactness still +fails. The comparator reports that five more observations are needed on first ratio failure; it launches no jobs. +- [ ] Freeze these required cells in `tools/rank-local-openmp-workloads.json`: fixed Hubbard and Pauli models and random + Heisenberg/Schrödinger × build_graph/propagate/energy/gradient. Energy/gradient have separate unpared and + threshold-1e-10 profiles; do not change node IDs to distinguish the profile. Also require random Heisenberg and + Schrödinger `pare_functional_construct` driver cells; no removed pytest node is revived. Reference configurations at + the inspected HEAD are: + +```json +{ + "hubbard": {"num_sites": 60, "hopping": 1.0, "interaction": -2.0, + "chemical_potential": 0.0, "trotter_dt": 0.2, "trotter_steps": 29, + "observable_site": 46, "observable_spin": "up", "neel_start_spin": "down", + "cutoff": 6, "lower_atol": 0.0001}, + "pauli": {"num_qubits": 127, "num_layers": 20, "observable_qubit": 62, + "theta": 0.7853981633974483, "coupling": 0.7853981633974483, + "cutoff": 8, "lower_atol": 0.0001}, + "random": {"gen_length": 4, "obs_terms": 10000, "num_generators": 100, + "num_modes": 128, "cutoff": 6, "seed": 0, "lower_atol": null} +} +``` + +Add a tiny profile with random options `--num-generators 8 --num-modes 8 --cutoff 6 --obs-terms 16`; other random values +stay fixed. Both fixed models use their existing picture; random cases cover both pictures. Keep these complete +configurations in artifacts, not just a profile label. +- [ ] Collect node IDs with `uv run --no-sync python -m pytest benches --collect-only -q`; record exact IDs and run one + node per process. The existing fixed-model guard skips retained graphs with more than two repeated steps + (`MAX_GRAPH_STEPS=2`), so the 29-step Hubbard reference is intentionally a capacity decision: use + `monoprop_BENCH_ALLOW_BIG_GRAPH=1` only on adequately provisioned hardware, or obtain approval for a fixed lower + `--hubbard-trotter-steps` profile on both arms before collecting any baseline. A skipped cell is not a pass. Do not + silently omit graph-heavy cells or bypass the guard on an unverified allocation. +- [ ] Run baseline geometries `(R,T)=(1,1),(1,Tmax),(Rmax,1),(Rmax,Tmax)` on the approved allocation, deduplicating + equal cells. Old throughput arm uses `R` real MPI ranks × `T` partitions; record an additional old single-store serial + diagnostic. Never compare only against forced-serial old code. + +- [ ] Reuse `benches/LADDER.md` L1–L4 labels/provenance and term-count fields as historical calibration, never + acceptance evidence from old runs. Where a full-node single rank crosses NUMA domains, do not copy that historical + allocation: obtain an approved within-one-domain profile, equal on both arms, before baseline collection. Preserve + historical nodes and fields while adding runtime-shape/observed-team metadata. Term counts must agree for the + identical operation. +- [ ] Use `just capture-baseline` / `just diff-baseline` and `tools/capture-baseline.py --compare` only as supplementary + small-fixture regression evidence. Its keys include rank, tolerance differs and it lacks operation + gradients/provenance; it cannot replace the ownership-aware global-map comparator or independent + exact/finite-difference tests. + +**Gate:** Benchmark tools' unit tests pass; baseline numerical suites pass; baseline observations and placement evidence +are archived. Historical benchmark keys unchanged. No parity claim yet. + +### Task 2: Add the smallest OpenMP primitive and its build dependency + +**Files:** +- Create: `cpp/monoprop/detail/parallel/{Options.h,Workshare.h,CMakeLists.txt}`. +- Modify: `CMakeLists.txt`, `cpp/monoprop/CMakeLists.txt`, `cpp/monoprop/detail/CMakeLists.txt`, + `cmake/monopropConfig.cmake.in`. +- Create/test: `cpp/tests/openmp_workshare_tests.cpp`. +- Modify: `pyproject.toml`, `tools/install-deps.sh`, `tools/packages/{apt,brew}.txt`, + `.github/actions/setup/action.yml`, `.devcontainer/Dockerfile`, `nix/{monoprop,devshell}.nix`, `flake.nix`, + `.github/workflows/{test,nix,deploy}.yml` for required OpenMP across supported dependency routes. +- Extend: `cpp/tests/find_package_smoke/CMakeLists.txt`, `cpp/tests/link_export_probe/link_export_probe.cpp`; reuse + `just test-find-package`, not a second consumer harness. +- Modify: `AGENTS.md`, `README.md`, `docs/content/docs/building.mdx` for required OpenMP. + +**PROPOSED interfaces:** In namespace `monoprop::detail::parallel`: + +```cpp +struct Options { + int threads = 1; //!< Validated positive per-object budget; not a global runtime setting. +}; + +template +auto for_blocks(size_t count, Options options, Fn &&body) -> void; +// body(block_id), one invocation per logical block; returns only after all work joins. +// No MPI, no shared append, no escaping worker exception. Serial if nested or budget==1. +``` + +`Options.h` contains no OpenMP include; `Workshare.h` includes ``. No pool, scheduler, task queue, type-erased +jobs, or persistent worker-owned operator state. + +- [ ] Add tests that require the new helper: every block visited exactly once; count 0/1/19; budget 1/2/3; more + requested threads than blocks; nested invocation; original runtime max-thread setting unchanged; a worker throws + `std::runtime_error` and caller catches it after all work has joined. +- [ ] Use this test pattern; each element has one writer: + +```cpp +BOOST_AUTO_TEST_CASE(openmp_blocks_visit_once) { + std::vector visits(19); + monoprop::detail::parallel::for_blocks(19, {.threads = 3}, [&](size_t b) { + ++visits[b]; + }); + for (const auto n : visits) { BOOST_TEST(n == 1); } +} +``` + +For the exception test also record visited/completed blocks without asserting which ones finish after a failure. Never +use `BOOST_TEST` inside workers; store results and assert on the caller. +- [ ] Run the new test build: expect missing header/helper. Add required dependency integration: + +```cmake +# Root discovery +find_package(OpenMP REQUIRED COMPONENTS CXX) +# Compilation and link/export requirements are separate: the shared library is +# assembled from TARGET_OBJECTS and does not inherit the object target's linkage. +target_link_libraries(monoprop-objs PUBLIC OpenMP::OpenMP_CXX) +target_link_libraries(monoprop PUBLIC OpenMP::OpenMP_CXX) +# Installed cmake/monopropConfig.cmake.in: +find_dependency(OpenMP REQUIRED COMPONENTS CXX) +``` + +Register `Options.h`/`Workshare.h` in the existing `headers` FILE_SET via the new subdirectory. Use imported target +flags, not hard-coded `-fopenmp` or `-lgomp`. There is no OpenMP-disabled build in this design; a one-thread build/run +is still OpenMP-enabled. +- [ ] Implement serial and parallel paths. Serial path calls `body(b)` in ascending order without allocating error + slots. Parallel path requests `min(options.threads,count)` workers, uses + `#pragma omp parallel for schedule(static) num_threads(team)`, catches into preallocated `std::exception_ptr` slots + **indexed by actual worker ID**, joins, then rethrows the first populated slot. Allocate at most `team` errors, not + one per row/block. Validate positive budget and loop-bound representability before the region. Correctness must not + depend on actual team size equaling the request. +- [ ] Do not invoke throwing setup outside the worker's catch. Exceptions from allocation of error slots occur before a + region and propagate normally. The helper cannot provide transactional rollback. +- [ ] Add `libomp` to the Homebrew manifest and the matching Clang OpenMP development package to the apt/setup route; + GCC uses its selected libgomp. `.github/actions/setup/action.yml` and `.devcontainer/Dockerfile` consume the shared + package manifests: update the source of truth rather than duplicate installs. Wire compiler-compatible OpenMP inputs + into both Nix package and devshell, preserving Nix's explicit sandbox build-time mpi4py provisioning. Extend + `flake.nix` package/MPI package/check/devshell checks and `.github/workflows/nix.yml` smoke validation. Verify OpenMP + configure/link/run in both Nix packages and shell. Hwloc stays until Task 12; do not delete it early. +- [ ] Install `libomp` for macOS and Clang Linux jobs; verify keg-only Homebrew discovery with the actual toolchain. + Keep GCC's selected libgomp when using GCC. Linux wheel dependency repair and macOS delocation must resolve the chosen + runtime in a clean environment. Keep hwloc only while the legacy partition runtime still requires it; Task 12 deletes + that dependency and its topology code rather than porting them. Audit Threads separately for remaining non-partition + uses; OpenMP discovery/linking must not add a direct hwloc requirement. +- [ ] Run R and the new helper tests with `OMP_DYNAMIC=TRUE` and `FALSE`, plus `OMP_THREAD_LIMIT=1`. A limit of one must + still visit every block. Add a test-only worker-ID observation proving more than one worker participates when + hardware/runtime permits; report a skip when fewer than two are available, not a false success. + +**Gate:** Helper tests and `just test-find-package` pass; extend its shared link-export probe source to instantiate an +OpenMP template and both basis APIs using installed targets only. RED: missing imported OpenMP dependency/header; GREEN: +clean consumer finds/links/runs with no manual flags or source includes. No engine behavior changes yet. + +### Task 3: Plumb per-object budgets and distributed failure boundaries + +**Files:** +- Create: `cpp/monoprop/detail/parallel/{ThreadBudget.h,ThreadBudget.cpp}`. +- Create: `cpp/monoprop/detail/mpi/OperationFailure.h`. +- Modify: `cpp/monoprop/detail/parallel/CMakeLists.txt`, `cpp/monoprop/detail/mpi/CMakeLists.txt`. +- Modify: `cpp/include/monoprop/{MonomialPropagator,Evolution,MPFunctions}.h`. +- Modify: `cpp/monoprop/detail/monomial_propagator/MonomialPropagator.inl`, `cpp/monoprop/{Evolution,MPFunctions}.cpp`. +- Modify: `cpp/monoprop/detail/evolution/{CosineRecompute,CosineRecomputeCallbacks}.h`, + `cpp/monoprop/detail/evolution/layer_build/{Scan,Resolve,Engine,FusedApply}.h`. +- Modify: `cpp/monoprop/detail/mpi/MPICompat.cpp`, `cpp/monoprop/detail/EnvConfig.h`, `src/monoprop/bindings/binder.h`. +- Create/test: `cpp/tests/openmp_runtime_tests.cpp`, `cpp/tests/mpi_failure_driver.cpp`, `tests/test_openmp_config.py`; + modify `cpp/tests/CMakeLists.txt` to exclude the failure-driver main from the unit-runner glob and build a separate + executable. + +**PROPOSED interfaces:** + +```cpp +// ThreadBudget.h; pure count validation, no topology or mutable global setting. +namespace monoprop::detail::parallel { +// requested==0 selects runtime_default; otherwise use requested exactly. +// Throw std::invalid_argument if the selected value is not in [1, INT_MAX]. +auto resolve_thread_budget(size_t requested, int runtime_default) -> Options; +} +// Constructor: append after existing child_factory argument: +// size_t num_threads = 0 +// Object member: parallel::Options parallel_; logically immutable after construction. +// EvalRequest: append detail::parallel::Options parallel = {}; +``` + +All the following existing functions gain a trailing `parallel::Options options = {}` (fully qualify namespace in public +headers): `build_layer`, `fused_find_and_collect`, `probe_incoming_queries`, `resolve_incoming`, `apply_fused_contract`, +`scale_cos_lazy`, `accumulate_cos_lazy`, `scale_cos_mask`, `accumulate_cos_mask`, `evolve_step`, `evolve_operator`, +`state_operator_derivative_local`, and `build_cos_callbacks`. Private helpers receive/forward the same value. Keep +callback types `LayerCosScale`/`LayerCosAccumulate`/`LayerCosIndices` unchanged: closures capture options. EXISTING +derivative signature in `cpp/include/monoprop/Evolution.h:54–63` is the following signature (retain its existing export +annotation): + +```cpp +auto state_operator_derivative_local(VecD&, VecD&, const MPGraphView&, size_t, + LayerAngle, mpi::Comm, const detail::LayerCosAccumulate&, + const detail::CosRecordView& = {}) -> double; +``` +PROPOSED options follow `record`, not `angle` or `cos_acc`; preserve `LayerAngle{.gen_coeff=g,.param=theta}` and record +forwarding. `MPFunctions.cpp:44–172` records pre-layer values based on accumulated amplification AND vanishing-cosine +criteria; `Evolution.cpp:450–477` snapshots self endpoints, predivides, accumulates, restores and combines +pre-layer-unit sums. Keep record append/predivide/restore serial because duplicate indices are allowed. Do not replace +this stable protocol with a division-only reverse pass. + +- [ ] Add compile/integration tests for old low-level numerical call sites (their defaults stay serial; constructor + default selection changes in Task 10), two objects with budgets 1/3, copied budget, retained functional budget + capture, and rejection of simultaneous prototype multi-partition + threaded kernels. +- [ ] During coexistence, only explicit new `num_threads>0` enables the new runtime; require resolved legacy + `partitions==1`. Use explicit `partitions=1` plus final `num_threads` in C++ and low-level Python tests. Legacy + children always receive `{.threads=1}`. Do not reinterpret environment variables until Task 10. +- [ ] Expose the low-level binding constructor keyword `num_threads=0` after `partitions`. Regenerate by running R or M: + `src/monoprop/bindings/CMakeLists.txt` already invokes the generator at configure time and dispatch generation at + install time. For a manual regeneration of the default Release configuration only, its exact equivalent is: + +```bash +uv run --no-sync python tools/generate-binders.py --max-num-modes 1024 \ + --batch-size 8 --output-dir build/editable/Release/src/monoprop/bindings/generated \ + --binding-kind core +``` + +For a nondefault maximum use the actual `monoprop_MAX_NUM_MODES` cache value (current default 1024). Generated files +stay in the build tree and are not committed. Never hand-edit generated bindings, invent a mode-width policy, or modify +`tools/_binding_layout.py` for this task. +- [ ] Implement `resolve_thread_budget(requested,runtime_default)` without any hardware queries: when requested is + nonzero, check `requested <= static_cast(INT_MAX)` before conversion and keep that value exactly; when zero, + require `runtime_default > 0` and use it. Throw `std::invalid_argument` for unrepresentable/nonpositive selected + values; translate configuration errors at the public constructor boundary to `PropagatorConfigError`. Tests cover + `(0,1)->1`, `(0,3)->3`, `(8,2)->8`, `(1,0)->1`, `(0,0)`/`(0,-1)` rejection, INT_MAX acceptance and SIZE_MAX rejection + when larger than INT_MAX. These are pure count tests, never requests to create huge teams. In the prototype resolve + only explicitly enabled threaded objects; Task 10 enables the default path. Copy the resolved options unchanged. +- [ ] At cutover, when all explicit/compatibility overrides are absent, pass `omp_get_max_threads()` as runtime_default + once at construction. This captures the OpenMP calling context's default (normally `OMP_NUM_THREADS`), not actual team + size or available hardware. Do not parse OMP_NUM_THREADS independently or derive T from `omp_get_num_procs`, + `std::thread::hardware_concurrency`, affinity masks, OpenMP places, hwloc, sysfs or NUMA discovery. A requested count + may exceed the rank's allocation; preventing that is the user's responsibility. Small-work/nested serial fallback and + OpenMP limits may still reduce a region's actual team. +- [ ] In this prototype task test explicit budget validation/copy/callable capture and runtime-limited teams; + default-selection/precedence subprocess tests belong to Task 10, when that path is enabled. Verify library calls leave + OpenMP settings unchanged and issue no affinity-setting calls. A no-affinity-change test must disable OpenMP binding + (`OMP_PROC_BIND=FALSE`), so it does not mistake the runtime's user-requested placement for library pinning. No + hardware-topology query is needed by these tests; Task 11 placement checks are external benchmark-operator work. +- [ ] Forward options through ALL paths: graph-only build; graph-with-coefficients stored-mask closure; immediate fused + apply; replay; retained functional capture and `EvalRequest`; `ev`/`ev_and_grad` → `prepare_evolved_operator` → + evolution; derivative callback; partial contraction. `build_cos_callbacks` captures `[cache, sc, options]`. Keep + parity pointers ephemeral. Do not alter the GIL policy or external callable lifetime contract. +- [ ] Implement a caller-side `mpi::operation_failed(comm, std::exception_ptr)` as a `[[noreturn]]` helper. For an + ordinary real MPI communicator with size>1, report rank/error and call `MPI_Abort`; if MPI_Abort returns, terminate. + For size one/non-MPI, rethrow the original exception. Keep transitional legacy communicator failures on their existing + poison/abort paths until Task 12. +- [ ] Guard every newly throwable distributed phase, not merely the outermost function. Especially catch a worker + failure in overlapped cosine work **inside the lifetime of an active exchange ticket** and abort before stack + unwinding can wait on peers that never posted. Normal successful paths still explicitly wait. Do not rely on a ticket + destructor to coordinate distributed failure; do not add a per-kernel allreduce to agree on allocation failure. Wrap + constructor/build/replay/evaluation phases that can strand another rank, including retained-callable entry points. + Single-rank failures preserve exception type; failed mutated objects must not be reused. +- [ ] Validate already initialized MPI with `MPI_Query_thread`, not only monoprop-owned initialization. Reject a level + below `MPI_THREAD_SERIALIZED` with an actionable diagnostic before distributed work. MPI worker calls are forbidden; + low-level numerical kernels may serialize under a nested OpenMP call, but distributed public operations from an + external OpenMP team are unsupported and must reject before communication. +- [ ] Add a separate failure driver with command cases `worker-throw`, `before-exchange`, `active-ticket`, + `insufficient-thread-level`. On rank zero inject the failure; peers execute the normal phase. Run from a subprocess + with timeout 30s under two ranks. Expected outcome is prompt nonzero failure, **not** timeout or a successful exit. Do + not put intentional MPI_Abort cases inside the ordinary suite executable. Test single-rank worker throws separately. + +**Gate:** R/M compile and pass with kernels still serial. Focused gates include `combined_recompute_equivalence.cpp` +record restoration and `tests/test_deep_circuit_gradient.py` deep non-singular, vanishing-cosine and no-record cases, +before and after threading. Options reach every intended entry point; runtime global settings unchanged; failure tests +do not hang. Public/exported signature ABI change is recorded for a coordinated library/bindings rebuild. + +### Task 4: Prototype cosine and fused updates without structural concurrency + +**Files:** +- Modify: `cpp/monoprop/detail/evolution/CosineRecompute.h`, `cpp/monoprop/detail/evolution/layer_build/FusedApply.h`. +- Create/test: `cpp/tests/openmp_kernel_tests.cpp`. +- Extend: `cpp/tests/{combined_recompute_equivalence,fused_cos_sweep_tests}.cpp`. + +**PROPOSED interfaces:** Consume Task 3 options. No further public interface. `for_blocks` handles region/exception +joining; caller chooses semantic blocks and the serial threshold. + +- [ ] First add exact vector-equivalence tests using a stored mask spanning 4097 words; final mask word has only bit + zero. Test budgets 1/2/3/4, zero words, 63/64/65-row tails and more workers than work. + +```cpp +BOOST_AUTO_TEST_CASE(openmp_cos_mask_exact) { + monoprop::CosMask mask; + for (size_t w = 0; w < 4097; ++w) { + const auto bits = w == 4096 ? uint64_t{1} : uint64_t{0x5555555555555555}; + mask.blocks.emplace_back(w * 64, bits); + mask.total_count += static_cast(std::popcount(bits)); + } + monoprop::VecD s(4097 * 64, 1.25), p = s; + monoprop::detail::scale_cos_mask(s.data(), mask, 0.625, {.threads = 1}); + monoprop::detail::scale_cos_mask(p.data(), mask, 0.625, {.threads = 3}); + BOOST_CHECK_EQUAL_COLLECTIONS(s.begin(), s.end(), p.begin(), p.end()); +} +``` + +- [ ] Add lazy-mask tests in `combined_recompute_equivalence.cpp` with an `InvertedIndex<8>` exceeding three + `kColumnBlockWords` blocks, mixed dense/sparse columns, odd/even Majorana, Pauli transformed fold, empty columns, zero + generator, and partial `scaled_count`. Clear `row_parity_` before the **parallel-first** call to expose cold-cache + races. Compare coefficients exactly against the existing serial oracle. +- [ ] Parallelize `scale_cos_lazy` over existing 1024-word fold blocks after caller-side parity preparation. Obtain + scratch inside each worker invocation; use a fixed `std::array` for no-allocation fold + scratch or the existing TLS under the helper's catch. Never capture a reference to the caller's TLS buffer in the team + lambda. +- [ ] Parallelize `scale_cos_mask` over disjoint mask-block ranges. Keep accumulate/reduction routines serial in this + task. Keep `fold_to_cos_mask` and `fold_to_indices` serial to preserve diagnostics/ordering cheaply. +- [ ] In fused apply, size snapshot arrays on the caller, parallel gather insert-record snapshots, join, complete + optional cosine phase, join, then workshare hit/insert/cross-half records. Each iteration performs the entire existing + rotation formula; retain phase order. In debug tests record all updated row IDs and assert exactly one add-owner per + destination before using this invariant. +- [ ] Keep first prototype thresholds conservative: at least two logical work blocks; a cosine block is 1024 words and a + record block is 1024 records. Tiny cases run serial. Thresholds may be tuned only by Task 11 evidence, not semantic + changes. Preservation fixtures above may already pass with Task 3's serial options plumbing. For RED/GREEN of actual + threading, use test-build-only observations at each affected kernel work-range body: pre-size one worker-ID slot per + logical range, have its sole owner record `omp_get_thread_num()`, and check distinct IDs after join on adequate + hardware. Do not count a team created elsewhere as this kernel's participation. A runtime-limited skip is not scaling + evidence. These observations must not become a production callback API or persistent per-term storage. +- [ ] Run R, M targeted fused/cosine tests, S, and repeated stress with runtime-limited teams. Measure serial versus + threaded one-store replay before investing in construction. Archive timings/peak/worker scratch; this prototype cannot + establish construction scalability. + +**Gate:** Exact coefficient agreement and measured replay scaling on large cases; no coefficient-sized additional +persistent buffer. If replay is already bandwidth-bound with regressions, stop/profile rather than proceed on assumed +speedup. + +### Task 5: Deterministic parallel bitmap traversal and query generation + +**Files:** +- Modify: `cpp/monoprop/detail/evolution/layer_build/{Scan,Engine}.h`, `cpp/monoprop/algebra/AlgebraCommon.h`. +- Extend: `cpp/tests/{fused_cos_sweep_tests,sparse_query_tests,pauli_build_layer_tests}.cpp`, + `cpp/tests/{exact_upper_atol_rescue,evolution_detail_tests}.cpp`. + +**PROPOSED interfaces:** Add `CutoffEvaluator::parallel_safe() const noexcept -> bool`, true only for existing typed +`LengthCutoff`/`SupportCutoff` targets. Unknown callbacks, including current basis-change closures, use serial +traversal. Do not require arbitrary existing user callbacks to become thread-safe. + +EXISTING `Scan.h::fused_find_and_collect` returns `FusedScanResult` and takes `SlotWindow window`, `my_rank`, +router, precomputed `gen_shift`, then capture/scaling arguments. `Engine.h` derives the plan/window once from that +shift. PROPOSED: extract the following range helper in `Scan.h`; preserve these inputs and result type, prepare lazy +data once, and merge private results. Do not reintroduce rank-count-shaped query vectors. The range helper must never +call a lazy accessor itself: + +```cpp +template +auto fused_find_and_collect_range(const MPOperator &op, + const Monomial &gen, const CutoffEvaluator &cutoff_eval, + const CutoffContext &cut_st, const VecD &coeffs, + std::optional only_rotate_len_k, mpi::SlotWindow window, size_t my_rank, + const routing::Router &router, size_t gen_shift, const InvertedIndex &index, + const uint64_t *row_parity, size_t wlo, size_t whi, + bool capture_values, double *fused_scale_coeffs, + double fused_scale_cos) -> FusedScanResult; +``` + +Recomputing the small immutable generator context per range is acceptable initially; do not introduce a persistent +prepared-state cache. Tail masking uses the full index's final word, not each range's final word. + +- [ ] Before changing traversal, add a serial/threaded result comparison helper checking each owner's leader/follower + query words, source IDs, captured values, cosine indices, coefficients and both self stages (logical position spans, + offsets, counts and phases). Include nonzero window bases, linear zero/nonzero shifts and splitmix dense windows. + Cases must exercise dense and sparse pivots, odd Majorana correction even when selected fold columns are empty, Pauli + J(G), no length cap, cap 0/positive, lower-atol equality, upper-atol rescue, cos=0 fallback, capture-values on/off, + and no emitted queries. +- [ ] Add a stateful custom cutoff counting calls and checking caller thread identity. Expected behavior remains serial. + A basis-change regression test must retain its existing mathematical result, even though this first version leaves its + traversal serial. +- [ ] On caller: validate arguments, prepare algebra/generator columns, obtain router once, initialize lazy + index/parity, snapshot row/word count, check skip/tail conditions. Partition words into at most + `min(options.threads,ceil(words/1024))` contiguous nonempty ranges. Range IDs increase with source word index and do + not depend on actual worker count. +- [ ] Move `EvenParityNzWord` scratch, `CosineWordBuilder`, counters and the `emit` lambda **inside** each range + invocation. Reuse `even_parity_scan_pass1(wlo,whi)` then immediately emit its results. Workers read/update only their + assigned source rows. They may not resolve target coefficients or grow stores. +- [ ] If cutoff is not parallel-safe or work is small, call the original serial body without chunk metadata. Otherwise + `for_blocks(range_count,options,...)` fills one `FusedScanResult` per range. No shared push_back/critical section + and no per-bitmap-block × rank vector table. +- [ ] Join, calculate checked total lengths, reserve once and merge reachable slots in ascending range ID using + `mpi::WindowIndex{k}` or `.at_slot(flat_slot)`; never use a flat slot as a window index. All pieces share the exact + `window` (base and count). Move the first useful buffer where safe and release drained buffers immediately. + `leader_queries/src/val` and follower equivalents preserve each stream's source-order subsequence. Wire records have + variable boundaries, not a fixed stride; use `QueryWire` and preserve Plain/Fused semantics. Self wire stays empty. +- [ ] Merge `leader_self` and `follower_self` separately, aligned with the self slot's src/val streams. EXISTING + `SelfQueryStage` in `PartnerMerge.h:81–140` uses logical `size()`/`positions()`, not backing vector sizes. Use its + existing `push(span,phase)` in source order initially: for each q below `piece.size()`, pass the span at + `piece.pos_off[q]` of length `piece.k_of[q]` and `piece.phase_of[q]`. This appends to the logical position end and + rebases offsets correctly without a new helper or accessing private counters. Check total queries/positions and + reserve bounds first. Do not copy capacity tails. Assert self stage length equals self src length; values match only + when capture is enabled. Keep cosine word seam handling and ascending disjoint cos_blocks in Engine. + +```cpp +// PROPOSED merge loop shape; append concrete streams as specified above. +for (size_t k = 0; k < window.count; ++k) { + const mpi::WindowIndex wi{k}; + const auto flat_slot = window.slot(wi); + for (size_t range = 0; range < pieces.size(); ++range) { + // pieces[range].leader_queries[wi] belongs to flat_slot, not flat slot k. + // Append leader src/val in the same range order; handle followers separately. + // If flat_slot==my_rank, merge self positions, never synthesize self wire. + } +} +``` + + Metadata bound is O(T*window.count), payload O(total emitted records), never per-block×R or per-thread full-store + dense keys. At S=1, linear zero shift is all self; nonzero shift has exactly one reachable remote slot. +- [ ] Keep leader/follower resolution unchanged in this task. Compare graph data and row IDs at fixed rank count after + full construction, not just energy. +- [ ] Measure simultaneous private+merged query capacities and worker TLS. If merge duplication fails peak parity, use a + two-pass exact-fill variant **only for pure cutoffs**: count with no scaling/capture mutation, caller prefix/reserve, + then fill and scale once. Count and fill must use identical pre-scale coefficients and predicates; do not scale in + both passes. If this CPU/memory tradeoff still fails, stop. Do not replace compact storage to fix output buffering. + +**Gate:** RED: new range helper absent; Task 3 options already exist. Exercise the new multi-range merge path in ordered +window/self regressions, not only the old serial path. GREEN: exact serial/threaded query words, self positions, IDs and +graph fields at fixed geometry. Ordered query/row/graph equivalence, no callback-thread contract change, no lazy-cache +races. Construction bitmap-traversal time and peak scratch are recorded separately from resolution. + +### Task 6: Parallel frozen lookup while retaining serial insertion + +**Files:** +- Modify: `cpp/monoprop/detail/evolution/layer_build/{Resolve,Engine,QueryWire}.h`. +- Extend: `cpp/tests/{operator_index_tests,sparse_resolve_tests,sparse_query_tests,mpi_fresh_insert_equivalence}.cpp`. + +**PROPOSED interfaces:** + +```cpp +// PROPOSED Resolve.h helper; per-query spans have equal lengths; store frozen until return. +template +auto probe_frozen_positions(const OperatorIndex &store, + std::span::PosT> pos_flat, + std::span pos_off, std::span k_of, + std::span out, std::span hash_out, + parallel::Options options = {}) -> void; +``` + +EXISTING `OperatorIndex::find_batch_positions(pos_flat,pos_off,k_of,out,hash_out={}) const -> void` +(`OperatorIndex.h:282–288`) consumes position spans; use 256-query blocks, with absolute pos_off values into the shared +pos_flat and sliced query/output/hash spans. Retain batched lookup and cached hashes, not scalar find or dense keys. +Change incoming `miss_g` from `vector` to `vector` because it contains query ordinals, not term IDs. +Count that memory change explicitly. + +- [ ] Add this preservation test to `sparse_resolve_tests.cpp`, using its EXISTING `make_op` and `positions_of` + helpers/includes. Task 3 already added the trailing options argument, so this test may pass before Task 6. Its purpose + is exact predicted/published IDs and nonzero-window addressing, not a fabricated RED. Use the missing + `probe_frozen_positions` helper and malformed-boundary tests as this task's RED targets. Public Majorana indices are + reversed when encoded; `QueryWire::push` needs raw ascending bit positions obtained with `positions_of`: + +```cpp +BOOST_AUTO_TEST_CASE(openmp_incoming_miss_order) { + using namespace monoprop; + const auto a = indices_to_bitset<8>({0, 1}); + const auto b = indices_to_bitset<8>({2, 3}); + const auto c = indices_to_bitset<8>({4, 5}); + for (const auto threads : {1, 3}) { + auto op = make_op<8>(std::vector>{a}); + const mpi::SlotWindow window{.base = 4, .count = 3}; + mpi::WindowVec incoming(window); + detail::QueryWire<8>::push(incoming.at_slot(4), positions_of<8>(b), +1); + detail::QueryWire<8>::push(incoming.at_slot(6), positions_of<8>(a), -1); + detail::QueryWire<8>::push(incoming.at_slot(6), positions_of<8>(c), +1); + const auto pr = detail::probe_incoming_queries<8>( + incoming, op, detail::QueryForm::Plain, {.threads = threads}); + BOOST_TEST(pr.sender_slot(0) == 4U); + BOOST_TEST(pr.sender_slot(1) == 6U); + const std::vector offsets{0, 1, 1, 3}, ids{1, 0, 2}, misses{0, 2}; + BOOST_TEST(pr.base == 1U); + BOOST_CHECK_EQUAL_COLLECTIONS(pr.goff.begin(), pr.goff.end(), offsets.begin(), offsets.end()); + BOOST_CHECK_EQUAL_COLLECTIONS(pr.idx_of.begin(), pr.idx_of.end(), ids.begin(), ids.end()); + BOOST_CHECK_EQUAL_COLLECTIONS(pr.miss_g.begin(), pr.miss_g.end(), misses.begin(), misses.end()); + detail::insert_incoming_misses(op, pr); + BOOST_TEST(op.size() == 3U); + BOOST_REQUIRE(op.store->find(a).has_value()); + BOOST_REQUIRE(op.store->find(b).has_value()); + BOOST_REQUIRE(op.store->find(c).has_value()); + BOOST_TEST(op.store->find(a).value() == 0U); + BOOST_TEST(op.store->find(b).value() == 1U); + BOOST_TEST(op.store->find(c).value() == 2U); + } +} +``` + +Repeat at window base zero, empty/all-hit/all-miss inputs, >256 queries and short tails, fused-value codec, overflow +rows, and a prebuilt inverted index. +- [ ] Prepare safe decode on caller BEFORE workers. EXISTING `QueryWire::count_queries`/`read_query` assume valid input; + do not use an unchecked walk as a safety check. In `Resolve.h::probe_incoming_queries`, walk each sender by window + index, requiring off::header_bits_for(h.k)` before calculating payload size; reject noncanonical escaped + counts below `kKEscape`. Otherwise `words_of_header` can undercount the actual header consumed by `read_positions`. + Compute payload words using the validated header, add the Fused value word with checked arithmetic, and require the + entire record to fit `buffer.size()-off`. Reject malformed/truncated streams; require final off==buffer.size. Add a + pre-worker rejection fixture for `QueryWire<128>`: `VecZ{size_t{0x8037e}}`, one Plain word with phase +1, escaped k=6, + gw=8 and zero payload bits. Its actual decode needs 68 bits but a canonical k=6 header implies 59; it must be rejected + before any payload read. Keep valid escaped-count and Fused coverage. Checked-prefix query and decoded-position + totals, plus goff, off_of, pos_off, k_of and sender_wi on caller. Check uint32 sender/count narrowing and vector + max_size. Allocate all output arrays once; no worker vector growth. +- [ ] Decode disjoint records into their prepared `pos_flat` spans via `read_query(buf,form,off,span)`, preserving + phase_of and Fused value offsets. Check returned next matches the prepared boundary. Extend EXISTING + `QueryWire::read_positions` to check cumulative decoded size_t positions before narrowing to PosT: each must be + strictly ascending and below 2*N. Checking only already-narrowed bytes could miss overflow. Headers already bound + payload reads; invalid positions throw inside the worker catch before hashing/publication. No dense key + reconstruction. Join and surface exceptions through Task 3's failure boundary, then `probe_frozen_positions`, join, + normalize not-found and assign misses serially in ascending sender-window/query order. Retain hash_of for every query. + Check ID/count limits before exposing predicted IDs. RED malformed-boundary/overflow tests must reject before worker + decode; GREEN includes empty, multiword, escaped-popcount, Fused, overflow-row and partial-block cases without dense + reconstruction. +- [ ] Keep `insert_incoming_misses` publication serial: grow → `set_positions(base+j,pr.positions_at(g))` → + `bulk_insert_hashed(...pr.hash_of[g])` → `reindex_after_growth`. Preserve phase-3 scatter before insertion because it + reads pre-insert coefficients. All ordinary set/overflow/hash/inverted/state-extension mutation stays single-writer. + Existing paired-only `IncomingProbe::mono_at` for Schrödinger state scoring is a bounded per-record exception already + in ContractSink; preserve it, never turn it into a dense batch/store. Keep that scoring path serial initially. +- [ ] Parallelize incoming scatter only after `responses` sizing and sink `prepare`. PROPOSED capability: + `static constexpr bool parallel_resolve = true` in `GraphSink`/`ContractSink`; it certifies only independent + `on_resolved` writes after prepare, not other sink methods. Missing/false capability means serial for generic sinks; + use nested `if constexpr (requires { Sink::parallel_resolve; })` then test its value, avoiding absent-member lookup. + For `ContractSink`, also require `!sink.schrodinger` initially; scoring remains serial. Keep shared + `ContractSink::on_response_block` push_back serial. Matched leader marks are unique uint16 elements and bounded by + pre-gate `combined_size`; join before followers read them. +- [ ] For self lookup, EXISTING `Engine.h::resolve_range_` uses 64-query batches of gathered offsets/counts into + `SelfQueryStage` and returns cached hashes. PROPOSED: use bounded windows up to 4096 queries for parallel work; serial + stable follower filtering → gather absolute position offsets/counts → parallel position probe → serial hit/deferred + emission in original order. Leaders skip filtering. No self wire encode/decode, no full-store keys, and no store + insertion between windows. Preserve deferred positions AND hashes, reuse bounded scratch and retain recording-sink + compatibility. +- [ ] Add assertions/tests for mixed self hit/miss, remote hit/miss, and leader/follower ordering. Do not coalesce all + insertion into a final phase. Keep remote-leader then remote-follower insertion and both-pass deferred self + publication. + +**Gate:** R/M lookup and fresh-insert tests pass with exact fixed-geometry IDs. Report +probe/prefix/publication/inverted-index times separately. If serial hash publication dominates, report an Amdahl limit +instead of inventing a concurrent table. + +### Task 7: Harden rank-wide capacity and optionally parallelize append-only row fill + +**Files:** +- Modify: `cpp/monoprop/detail/operator/{OperatorIndex,MPOperator}.h`. +- Modify only if row-fill optimization is justified: `cpp/monoprop/detail/evolution/layer_build/{Resolve,Engine}.h`. +- Extend: + `cpp/tests/{operator_index_tests,inverted_index_tests,evolution_detail_tests,exchange_layout_precondition_tests}.cpp`, + `cpp/tests/{graph_encoding_tests,ctor_validation_tests,majorana_cutoff_tests}.cpp`. + +EXISTING `grow_rows_geometric` already checks append counts (`OperatorIndex.h:111–122,544–563`); +`operator_index_tests.cpp:68–99` checks ceiling/wrap refusal without huge allocation. Extend, do not replace, these +tests. + +**PROPOSED required interface:** static `OperatorIndex::checked_append_end(size_t base,size_t count) -> size_t` +validates row-count/index limits without allocating. Keep capacity arithmetic distinct from valid row IDs. + +**PROPOSED optional measured interface:** + +```cpp +// PROPOSED OperatorIndex members, only if measured row fill justifies them. +[[nodiscard]] auto inline_width() const noexcept -> size_t; +auto set_new_inline_positions(size_t i, std::span positions) noexcept -> void; +auto reserve_index_for_size(size_t final_size) -> void; +``` + +- [ ] Add boundary tests before mutation: `(ceiling-1,1)` succeeds; `(ceiling-1,2)`, `(ceiling,1)` and arithmetic + overflow fail; `(ceiling,0)` succeeds as a count boundary. No billion-row allocations. Extend the existing + fixed-uint32 guard tests. Operator row IDs exclude the sentinel ceiling while some graph count fields can equal their + representable max: keep those domains separate. +- [ ] Implement checked append addition as `base>ceiling || count>ceiling-base` before evaluating `base+count`. Check + row-storage multiplication, vector max_size, table load-factor sizing and next-power-of-two rounding before + allocations. Geometric spare capacity cannot wrap. Update diagnostics from per-partition to per-rank store at cutover. +- [ ] Use these checks before incoming predicted IDs and deferred insertion. Audit existing MPI checked counts without + changing transport or count width. A rank-wide store can exceed a ceiling that each old partition fit; this is a + **capacity compatibility blocker**, not permission to widen only the candidate or introduce hidden shards. +- [ ] Profile row fill. If it is not significant, leave row initialization and publication serial and mark the optional + steps below skipped with evidence; the required capacity checks still ship. +- [ ] If justified, add tests proving simultaneous new inline writes do not modify old overflow entries. + `set_new_inline_positions` requires already-grown new rows, each with one owner and position count<=inline width; it + must contain no overflow lookup/erase, allocation, resize, table operation, or exception. Keep ordinary `set` serial. +- [ ] Do not introduce a dense KeyAt/Monomial append layer. Operate on EXISTING `IncomingProbe::positions_at(g)` and + `deferred_pos_flat_` spans, preserving cached `hash_of[g]` / `deferred_self_misses[j].hash`. Check ranges before + growth; input positions must outlive growth and never alias invalidated store views. +- [ ] Apply this order directly at `Resolve.h::insert_incoming_misses` and `Engine.h::insert_deferred_self_misses`: + serial checked reserve/grow → parallel inline-position fill → join → serial `set_positions` for overflow rows → serial + deferred `sink.emit_deferred` in k order (no incoming metadata emission) → serial `bulk_insert_hashed` using retained + hashes → serial `reindex_after_growth`. No readers before publication; no shared overflow lookup/erase from workers. + There is no production assign_row callback here to remove. Preserve the independent dense reference in + `sparse_resolve_tests.cpp`; do not change unrelated `insert_absent_terms` callback contracts. +- [ ] Preserve streaming initialization during capacity changes: `for_each_paired_monomial` enumeration order, + `paired_op_size` saturation guard, local reserve estimate and pre-worker rejection. At S=1 reserve/check against real + rank share, not T. Keep `majorana_cutoff_tests.cpp` enumeration/count tests, `ctor_validation_tests.cpp` oversized + constructor rejection and initial-basis partition-invariance coverage when migrating it to thread budgets. +- [ ] Compare all rows, keys, hash lookup, inverted bits/parity, sparse/dense state extension and `scaled_count` against + serial append after every growth. Include boundaries 0/1/15/16/17/63/64/65/127/128/129 and appends starting at + unaligned rows. + +**Gate:** Capacity guards pass R/M. Optional fill must improve measured bottleneck without peak regression. Hash +insertion, overflow mutation, packed graph phases and inverted append remain serial. + +### Task 8: Parallel replay endpoints and derivative pairs + +**Files:** +- Modify: `cpp/monoprop/Evolution.cpp`, `cpp/include/monoprop/Evolution.h` only for signature consistency. +- Extend: `cpp/tests/{combined_recompute_equivalence,evolution_detail_tests}.cpp`, + `cpp/tests/{mpi_distributed_layer_equivalence,flat_exchange_tests}.cpp`, `tests/test_deep_circuit_gradient.py`. + +**PROPOSED interfaces:** Use Task 3 options in snapshot, pack, remote apply and self-pair helpers. If range flattening +needs a descriptor, define local `EndpointRange { size_t slot; size_t begin; size_t end; }` in `Evolution.cpp`; this is +proposed transient metadata, not a new graph field. Flatten **ranges within active slots**; merely parallelizing ranks +underutilizes sparse linear-routing peers. + +- [ ] Write serial/threaded forward and derivative comparisons with multiple self pairs, nonadjacent row IDs, + remote+self slots, empty peers, zero-angle, deep non-singular circuits, vanishing-cosine and no-record paths. Retain + exact recorded restoration, including duplicate indices, indices outside the cosine set and cos rounding to one. + Compare energy/gradient to current finite-difference/exact oracles. Assert even self endpoint count and unique output + ownership. +- [ ] Record baseline PASS for existing mathematical/record fixtures where appropriate. Add test-build-only worker-ID + observations inside the endpoint work-range bodies, one preallocated slot per range with one writer. Their multiworker + requirement is RED while those bodies are serial; check after joining on an adequate allocation. Keep observations out + of production APIs and distinguish runtime-limited skips from demonstrated participation. +- [ ] Pre-size snapshots/payloads on caller. Parallel gather and pack disjoint ranges, join before posting MPI. Preserve + forward ordering: snapshot → pack → post → cosine → wait → remote apply → self apply. Do not reorder for overlap in + the same patch. +- [ ] For self derivative workshare over `k in [0,pairs)`, keep endpoint `k+pairs` in the same iteration. Read both + values required by the original formula before either overwrite. Never sort entries or independently workshare the two + halves. +- [ ] Retain serial scalar accumulations in this task; if a loop both mutates arrays and produces a scalar, isolate its + independent update portion only where the original formula permits it. Task 9 defines deterministic partials for the + full loop. +- [ ] Keep layer loops and gradient reverse order unchanged. No worker may call `mpi::rank`, `router_for`, exchange + begin/wait, or allreduce. Query rank/layout on caller. +- [ ] Cover worker exceptions after exchange posting using Task 3 active-ticket failure case. Buffers cannot resize or + die while requests are outstanding. Preserve EXISTING replay `wire_bits` selection: plain MPI uses dense + PeerPlan/nonzero pairwise legs, HybridComm derives its plan from all local partition rows. Do not introduce a new + generator-derived replay plan. Run `flat_exchange_tests.cpp` dense, pairwise, dense-layout-on-pairwise and + empty-layout cases after every transport edit. + +- [ ] Keep `LayerAngle`, `CosRecordView` and `LayerCosIndices` intact. Serial forward record append, predivide/restore + and snapshot-before-restore ordering are mandatory even if record ordinals differ: index duplication is legal. + Preserve `A = cos_acc(...)*sec` and `-g*(sin*(A-ep.cos_terms)-ep.sin_terms)` in pre-layer units from Evolution.cpp; + threading must not reintroduce division-only inversion or move cancellation after multiplication by sec. + +**Gate:** RED endpoint participation checks; preservation tests need not fail first. GREEN includes the entire +deep-circuit gradient suite at budgets 1/2/3/4. Full replay and immediate-contraction equivalence across both +pictures/bases, gradients and MPI; useful active-slot worksharing even with few peers. No full-vector snapshot beyond +current algorithmic requirements. + +### Task 9: Deterministic reductions without silently breaking serial dot contracts + +**Files:** +- Modify: `cpp/include/monoprop/MPFunctions.h`, `cpp/monoprop/MPFunctions.cpp`, `cpp/monoprop/Evolution.cpp`, + `cpp/monoprop/detail/evolution/CosineRecompute.h`. +- Extend: `cpp/tests/{mpfunctions,combined_recompute_equivalence,openmp_kernel_tests}.cpp`. + +**PROPOSED interfaces:** Keep `inner_product(v,w)` and `EvalState::dot(op)` unchanged. Add explicitly named +`inner_product_threaded(v,w,parallel::Options)` and `EvalState::dot_threaded(op,parallel::Options) const`. They have +documented fixed-block association, not historical left-to-right association. `ev`/`ev_and_grad` use the new helpers +through `EvalRequest.parallel` after approval of this numerical policy. + +- [ ] Add cancellation-heavy finite vectors, zero/signed-zero, sparse gapped rows, very short arrays, >4096 rows, + non-finite inputs, and repeated parameter indices. Check old serial dot tests unchanged; check new helper results + bit-identical across thread budgets at fixed geometry and within existing oracle tolerances. +- [ ] Define fixed logical dot blocks of **4096 row positions**, independent of worker count or dense/sparse + representation. Dense blocks sum in increasing row order; sparse blocks use lower_bound to locate entries in that same + row range and sum in ascending row order. Fold partial doubles on caller in increasing block ID. Use the same + partition even at budget one. Document non-finite/signed-zero limitations; do not strengthen sparse/dense guarantees + beyond what tests and current finite-input contract support. +- [ ] `EvalState::sparse` currently does not enforce all documented ascending/unique conditions. Do not introduce unsafe + parallel scatter. Keep `scatter_into` serial; for the new dot helper detect non-strictly-increasing rows and fall back + to existing `dot` behavior. This preserves historical duplicate/unsorted-input behavior without adding a new public + rejection in this refactor. +- [ ] For cosine accumulators use fixed 1024-word logical bitmap blocks; each block computes its scalar partial and + performs its disjoint existing coefficient updates. Caller folds partials in block order. Stored masks use logical + word ranges, not mask-vector ordinal ranges, when comparing to lazy recomputation. Leave old standalone serial + accumulator oracles in tests; compare according to existing tolerances except binary-exact fixtures. +- [ ] For endpoint derivative contributions keep two partial doubles per fixed 1024-record block. A block owns full self + pairs. Sum both fields in block order. Do not use OpenMP `reduction`, atomics, critical floating accumulation, or O(n) + product materialization to recreate serial sum order. +- [ ] Keep `indices_above`, parameter mapping and export loops serial initially. If a later profile shows + `indices_above` dominant, use count → caller offsets → disjoint exact fill, with logical row order, strict `>`, + negative threshold admitting zeros, and NaN behavior unchanged. No new global threshold policy. +- [ ] Run `tests/test_deep_circuit_gradient.py` and `combined_recompute_equivalence.cpp` across budgets 1/2/3/4, + including deep non-singular amplification, vanishing cosine, no-record and duplicate-record restoration cases. +- [ ] Run retained/direct evaluation, paring, partial contraction, repeated parameter mapping, both pictures/bases, + fixed-rank deterministic repeats, and near-cutoff retained-key comparisons. Cross-rank-count sums need not be bitwise + equal; don't widen tolerances to pass them. + +**Gate:** Existing serial API guarantees remain intact; new numerical association is documented and deterministic by +team size. If near-cutoff retained terms change outside existing intended equivalence, stop for owner decision, not a +tolerance adjustment. + +### Task 10: Cut over configuration and API semantics to one store + +**Files:** +- Modify: `cpp/include/monoprop/MonomialPropagator.h`, `cpp/monoprop/detail/monomial_propagator/MonomialPropagator.inl`, + `cpp/monoprop/detail/EnvConfig.h`, `cpp/monoprop/detail/parallel/{ThreadBudget.h,ThreadBudget.cpp}`. +- Modify: `src/monoprop/bindings/binder.h`, `src/monoprop/monomial_propagator.py` docstrings. +- Create/extend: `cpp/tests/openmp_equivalence_tests.cpp`, `tests/test_openmp_config.py`, + `cpp/tests/{env_config_tests,routing_tests,mpi_distributed_layer_equivalence}.cpp`, + `cpp/tests/{mpi_fresh_insert_equivalence,unit_tests}.cpp`, `tests/test_monoprop_smoke.py`. +- Modify: `AGENTS.md`, `README.md`, `docs/content/docs/features/parallelism.mdx`, + `docs/content/docs/openmp-migration.mdx` (new). + +**PROPOSED interfaces/policy:** Implement the design's deprecation aliases, not the alternative rejection policy +suggested by some investigations. No second permanent backend setting. + +- [ ] In fresh subprocesses test the precedence table: + +| Input | Result | +| --- | --- | +| explicit `num_threads=N`, no positive partitions | N exactly, provided 1 <= N <= INT_MAX; no hardware clamp | +| positive `partitions` and positive `num_threads` | `PropagatorConfigError` | +| explicit positive `partitions=N` alone | warned deprecated thread budget N; reject if above INT_MAX | +| `monoprop_PARTITIONS=off` | warned, budget 1 | +| `monoprop_PARTITIONS=N` | warned, N exactly; reject a positive numeric request above INT_MAX | +| `monoprop_PARTITIONS=auto` or malformed | warned, fall through to valid monoprop_NUM_THREADS, otherwise OpenMP default | +| no partition override, valid `monoprop_NUM_THREADS=N` | N exactly (existing parser/range retained) | +| no library override, any MPI rank count | capture omp_get_max_threads() at construction | +| OMP_NUM_THREADS unset and no library override | capture runtime default; no hardware-suitability promise | +| non-null child factory | `PropagatorConfigError`, factory never invoked | + +Public `num_threads=0` is the no-explicit-request sentinel. Positive explicit num_threads ignores environment aliases; +warn only for the legacy setting actually used. Warn once per process per deprecated partitions spelling, using an +existing diagnostic style or a small `std::once_flag`, not a Python-only warning that C++ users miss. Invalid +monoprop_NUM_THREADS keeps current parser behavior. Reject oversized positive partition requests before narrowing; do +not reinterpret them as malformed fallback input. Normally recommend OMP_NUM_THREADS=T with library overrides unset. It +supplies the OpenMP default but does not override an explicit library budget. Runtime limits such as OMP_THREAD_LIMIT +and dynamic-team adjustment may reduce the actual team. There is no topology-based fallback or special multi-rank +default of one thread. +- [ ] Add default-selection subprocess tests with library overrides unset: OMP_NUM_THREADS=1/3, list-valued + OMP_NUM_THREADS=2,3 (capture the runtime's outer-level default), and OMP_NUM_THREADS unset (compare the captured count + with omp_get_max_threads, not an assumed core count). Repeat at one and multiple MPI ranks; prove there is no library + multi-rank serial fallback. Test explicit/library-environment precedence and OMP_THREAD_LIMIT-reduced actual teams + separately from stored requests. Copies and retained closures preserve their captured budget; caller-driven + runtime-setting changes affect only subsequently constructed objects. Update prototype constructors/helpers to pass + `partitions=0` with an explicit num_threads, since the final API rejects both positive arguments. +- [ ] Obtain approval for changed historical splitmix physical ownership: `floor((hash % (R*S))/S)` becomes `hash % R`. + Linear rank mapping remains unchanged; S=1 removes only the partition hash. Preserve boolean mode/seed, + `linear_requested()`, `linear_bits_for(R)`, `make_router(R,1)`, generator `rank_shift` and `dest_from_shift` peer + selection. No virtual shards. Exports/load/reduction order can change under splitmix; validate global term/value maps + and resulting per-rank peaks at equal R/T/routing settings. Candidate ownership never depends on T. +- [ ] Extend `routing_tests.cpp`, `mpi_distributed_layer_equivalence.cpp` and `mpi_fresh_insert_equivalence.cpp`: + default linear success at R=1/2/4; explicit splitmix success at R=1/2/3/4. Record `monoprop_ROUTING`, + `monoprop_ROUTE_SEED` and resolved `linear_bits()` as derived metadata, not a requested bit count. Separately assert + `Router::for_modes<8>(3,1,true)` throws `UnroutableGeometry` and splitmix construction succeeds. Keep fresh-process + default tests distinct from splitmix matrix runs; never expect an automatic fallback. Assert unique global ownership, + generator-peer agreement, zero-payload participation, fixed-candidate ordering across budgets and global numerical + equivalence. Do not require old/new rank-local maps equal for splitmix. +- [ ] Make constructor create exactly one rank-local `MPOperator` and graph, with `comm_` representing only real MPI + ranks. Remove facade branches from every public method and internal dispatch, but leave now-unused transport files + until Task 12. Retain `PartitionChildFactory` declaration/argument temporarily to produce actionable failure, and keep + `MultiPartitionUnsupported` type temporarily for source migration even though single-store accessors no longer throw + it. +- [ ] Preserve rank-local size/bytes accounting (do not allreduce twice), core term handling, copy independence, + immutable graph-core sharing, virtual destructor, virtual `clone_`, overridden virtual `update_initial_operator`, and + protected nonvirtual `apply_initial_operator_`. Do not add override to or make that helper virtual. Copy budget and + ordinary MPI communicator. Raw accessors return the one store/graph at all budgets. +- [ ] Preserve `MonomialPropagator.inl:126–151,197–227` streamed paired-basis construction and local reservation: + cardinality rejection remains before worker startup and uses real ranks at S=1, never T. Do not materialize + `generate_paired_op` globally per rank. Migrate `partition_equivalence_tests.cpp` initial-basis invariance alongside + enumeration/cardinality tests from `majorana_cutoff_tests.cpp` and `ctor_validation_tests.cpp`. +- [ ] Migrate numerical tests from partition counts to explicit budgets 1/2/3/4; distinguish stored requests from actual + team sizes under OpenMP limits. Library budget selection never clamps to affinity. Run parallel gates on a user/CI + allocation supporting the requested teams and report limited-team skips honestly; pure budget tests need no extra + CPUs. The default white-box harness selects a serial library budget (not CPU pinning); explicit tests override it. + Test default OpenMP selection in fresh processes without that harness override. +- [ ] Test public Majorana/Pauli constructors unchanged. Keep binding-only `partitions`/`num_threads`; do not add + unrelated public constructor parameters. Test copy, operator update, retained callbacks after growth, + cutoff/basis-change update, paring, Schrödinger state extension and independent repeated evaluation. +- [ ] Regenerate bindings via the configured generator. Preserve 32-mode template dispatch. Document + low-level/source/ABI breaks and unspecified coefficient-order changes; compare decoded term/value mappings rather than + sorted coefficients alone. + +**Gate:** R/M API/configuration tests pass; `R ranks × T threads` owns exactly R stores. Full compatibility ledger is +approved before deleting extension hooks. + +### Task 11: Measure parity and apply only bounded optimizations + +**Files:** Task 1 benchmark files and, only if profiling justifies, the kernels named below. Do not rename `bench_*` +functions or move benchmark files. + +- [ ] Add metadata for `has_mpi`, requested/effective team size, OpenMP runtime, + OMP_NUM_THREADS/PLACES/PROC_BIND/DYNAMIC/THREAD_LIMIT, any library override, routing mode/effective linear bits/seed, + launcher command, rank/worker CPU masks, NUMA IDs, revision/dirty overlay and extension path/hash. Collect placement + in separate diagnostic runs of the same binary/configuration with OpenMP runtime diagnostics + (`OMP_DISPLAY_AFFINITY=TRUE`, `OMP_DISPLAY_ENV=VERBOSE`), launcher binding reports, and external OS/scheduler tools + (on Linux, `lscpu -e=CPU,CORE,SOCKET,NODE` and `/proc//task/*/status`). Observe actual active-worker masks/team + size, not merely omp_get_max_threads; if a runtime cannot supply that evidence, use external sampling or test-only + observation rather than adding a production topology API. Preserve old single-CPU-thread fields as historical data: + OpenMP core places may include SMT siblings. No required hwloc dependency for the library or measurement helper. +- [ ] The benchmark operator, not monoprop construction, verifies that rank masks and actual worker places stay in one + NUMA domain and within the reserved allocation, with no unintended overlap between ranks. Do not require one logical + CPU per worker mask. Record idle/small-kernel serial fallback versus representative large-kernel actual team size + separately. An unverifiable placement blocks that benchmark result, not normal library construction. +- [ ] Compare old partitions vs final new threads at identical R,T and routing settings, using Task 1 frozen profiles. + Freeze the production routing mode in that matrix; additionally require the random reference profile under explicit + splitmix at R=3 and R=4, plus default linear at R=4 so ownership-dependent imbalance is measured, not only + unit-tested. Run five alternating **fresh-process** observations per arm/cell; external repetition only, + `--bench-rounds=1`. Include first-call/cold cache and separately measured repeated warm calls without overlapping + object construction. + +Example candidate Open MPI invocation after selecting actual values for `R` (= P), `T`, `LABEL`, and absolute `RESULTS`. +This demonstrates one raw pytest run, not a complete parity observation. The Task 1 driver uses the same launcher +contract, unique per-sample raw directory, frozen campaign and paired construction/validation artifacts for acceptance. +Unset compatibility overrides so this exercises normal OMP_NUM_THREADS configuration: + +```bash +unset monoprop_PARTITIONS monoprop_NUM_THREADS +# This example chooses splitmix explicitly, including if R=3; use linear only in a separate R=1/2/4 cell. +export monoprop_ROUTING=splitmix +mkdir -p "$RESULTS" +monoprop_BENCH_LABEL="$LABEL" monoprop_BENCH_RESULTS="$RESULTS" \ +OMP_NUM_THREADS="$T" OMP_DYNAMIC=FALSE \ +OMP_PLACES=cores OMP_PROC_BIND=close \ +uv run --no-sync mpiexec -n "$R" --map-by "ppr:1:numa:PE=$T" \ + --bind-to core --report-bindings \ + -x monoprop_BENCH_LABEL -x monoprop_BENCH_RESULTS -x monoprop_ROUTING \ + -x OMP_NUM_THREADS -x OMP_DYNAMIC -x OMP_PLACES -x OMP_PROC_BIND \ + python -m pytest 'benches/bench_models.py::test_model_propagate[pauli]' \ + -o filterwarnings=default --bench-rounds=1 --runtime-shape=openmp \ + --benchmark-json="$RESULTS/time-$LABEL.json" +uv run --no-sync monoprop-bench-report "$RESULTS" +uv run --no-sync monoprop-bench-bmf "$RESULTS" "$LABEL" +``` + +This mapping syntax is Open MPI-specific. Verify scheduler/launcher support; do not escape the allocation. For the +**baseline only**, set `monoprop_PARTITIONS="$T"` and `monoprop_NUM_THREADS="$T"`, export both to ranks with +`-x monoprop_PARTITIONS -x monoprop_NUM_THREADS`, and replace `--runtime-shape=openmp` with +`--runtime-shape=partitions`. Keep OMP settings, routing, allocation and shared overlay identical. The candidate must +leave both library overrides unset. The shared preflight is a required Task 1 change: the existing guard would reject +this candidate above one rank. Verify declared versus observed shape with binary-linked diagnostics; labels alone cannot +certify shape. After M build assert monoprop.has_mpi before invoking any launcher. For Slurm document the corresponding +user contract (`srun --ntasks=P --cpus-per-task=T --cpu-bind=cores` with OMP_NUM_THREADS=T and suitable +OMP_PLACES/OMP_PROC_BIND), while making clear that site CPU/SMT allocation rules and single-NUMA containment must be +verified. Launcher binding must leave each rank enough CPUs for its workers; merely setting OMP_NUM_THREADS does not +enlarge a one-core rank mask. +- [ ] Report runtime, exact operation peak/floor/delta, outer peak sum/max, separate whole-construction peak, persistent + operator/graph bytes, query/snapshot/TLS capacities, term/graph sizes, and numerical checks. Rank peak sums are upper + bounds on aggregate footprint, not node provisioning estimates; retain maximum rank peak as another statistic. Reject + non-exact operation/construction windows as parity evidence; outer exactness controls diagnostic labeling only. +- [ ] Compare medians per cell using exactly the five Task 1 gates: runtime, operation peak sum/max and construction + peak sum/max. Outer peak ratios are diagnostics only. All five required ratios must be <=1.00. If any fail, collect + five additional observations per arm for those cells and compare all ten. Do not replace the strict gate with a + geometric mean, a made-up 5% tolerance, or a changed problem size. +- [ ] Profile phases before optimization: bitmap traversal, query merge, decode/probe, missing-ID assignment, row fill, + hash publication/rehash, inverted-index maintenance, graph packing, MPI pack/wait, cosine, pairs, dot. Record miss + fraction, overflow fraction and bytes, not just aggregate speed. +- [ ] Allowed bounded optimizations, in order of measured relevance: + 1. Serial thresholds and block sizes; reduce unnecessary tiny teams, keeping logical **reduction** partitions fixed + for numerical determinism. + 2. Ordered query exact-fill from Task 5; bounded self-probe windows from Task 6; optional inline fill from Task 7. + 3. Incoming missing-ID prefix: parallel fixed-block miss counts, serial exclusive offsets, exact resize, parallel + ascending fill. IDs remain sender/query order; scratch O(block count), not threads×queries. + 4. Serial index pre-reserve to reduce rehash (measure simultaneous old/new table capacity), without concurrent + insertion. + 5. Packed graph phases by **whole-word ownership** only: serial validate/count/allocate; each worker constructs and + assigns one uint64 word, including slot seams and zero padding. No endpoint-wise `|=`. Test slots 63/2/65 and + nonbinary fallback; preserve derived D permutation instead of adding persistent D indices. + 6. Inverted-index append only if measured dominant: bounded windows of 64-row-aligned blocks; per-block/per-column + integer counts, serial column offsets/allocations, disjoint sparse element writes and whole dense-word ownership, + serial handling of an unaligned first word, column-owned promotion after fill. Scratch O(window_blocks×2N), never + full-height bitmap per worker. Tests must compare dense/sparse promotion, parity and sorted posting order against + serial after every append. +- [ ] Each optional optimization gets its own failing regression tests, focused review and full affected parity rerun. + Incoming whole-batch streaming/windowing, concurrent hash publication, new transport, new executor, rank + redistribution, full per-thread stores or a persistent key map require a new design; stop rather than improvise. + +**Gate:** All required performance/memory cells pass with attached raw observations and numerical/placement evidence. +Missing hardware, capacity failure, persistent serial-publication bottleneck or repeated ratio>1.00 blocks +removal/acceptance. Report it honestly; the design is not a promise of achievable parity. + +### Task 12: Remove legacy partition runtime, topology machinery and direct hwloc dependency + +**Files:** +- Delete: `cpp/monoprop/detail/partition/PartitionGroup.h`, + `cpp/monoprop/detail/mpi/{ShmComm,HybridComm,PartitionBarrier}.h`. +- Delete `cpp/monoprop/detail/mpi/CpuRelax.h` only if no retained caller remains. +- Delete: `cpp/monoprop/detail/partition/CpuTopology.{h,cpp}`, `cpp/monoprop/detail/partition/CMakeLists.txt`, + `cpp/tests/cpu_topology_tests.cpp`. Do not create replacement CpuTopology files under parallel/ or another path. +- Modify for dependency removal: `cpp/monoprop/CMakeLists.txt`, `cpp/tests/CMakeLists.txt`, `pyproject.toml`, + `tools/install-deps.sh`, `tools/packages/{apt,brew}.txt`, `.github/actions/setup/action.yml`, + `.devcontainer/Dockerfile`, `nix/{monoprop,devshell}.nix`, `flake.nix`, `justfile`, + `.github/workflows/{test,qa-analysis,docpages,copilot-setup-steps,bench,nix,deploy}.yml`. +- Modify migration/build guidance in the same change: `AGENTS.md`, `README.md`, + `docs/content/docs/{building,benchmarks,openmp-migration}.mdx`, `docs/content/docs/features/parallelism.mdx`. +- Modify: `cpp/monoprop/detail/CMakeLists.txt`, `cpp/monoprop/detail/{mpi,parallel}/CMakeLists.txt`, + `cpp/include/monoprop/MonomialPropagator.h`, `cpp/monoprop/detail/monomial_propagator/MonomialPropagator.inl`, + `cpp/monoprop/detail/mpi/{Comm,MPICompat,MPIUtils,Exchange,CheckedCount,Routing}.h`, + `cpp/monoprop/detail/mpi/MPICompat.cpp`, `cpp/monoprop/MPFunctions.cpp`. +- Migrate/retire test files according to ledger below; update `cpp/tests/README.md` and architecture docs in the same + change. + +**PROPOSED interface:** Simplify `mpi::Comm` to the existing implicit MPI_Comm wrapper with `.mpi`; remove +Kind/Shm/Hybrid and associated pointers/constructors/dispatch. Keep non-MPI behavior, ordinary MPI, PeerPlan routing and +existing typed exchanges. Do not collapse it to a raw alias if that changes overload behavior unnecessarily. + +- [ ] Start only after Task 11 parity and explicit owner approval of the C++ extension-interface break. Confirm no + runtime path constructs old groups before deleting definitions. +- [ ] Remove `partition_group_`, friendship, fan-out implementations, `resolve_partition_count_`, `is_partition_facade`, + and all protected partition-only helpers. Keep `clone_` and virtual lifecycle/operator-update hooks. No compatibility + facade that invokes old callbacks once with silently different semantics. +- [ ] Remove non-MPI-thread transport branches from utilities/tickets. Production router geometry is `(real_ranks,1)`. + Preserve EXISTING boolean linear/splitmix default/rejection contract and generator shift/peer rules; no + routing-default repair is needed. Keep ordinary MPI sparse query routing and sparse on-wire replay, sender order, + count checks, communicator lifetime and serialized thread-level validation (including external initialization). + `MPIUtils.h` routing agreement uses fixed S=1, never per-rank OpenMP budgets. Keep both MPI and non-MPI + implementations. +- [ ] Delete the entire topology/affinity implementation and its legacy call sites, including automatic physical-core + counting, L3 dealing, pinning and mandatory placement diagnostics. Do not move them into ThreadBudget or reimplement + them with OpenMP-place unions, OS affinity, sysfs or another topology library. User-controlled OpenMP and launcher + behavior replaces these policies, not another internal runtime. +- [ ] Remove `find_package(PkgConfig REQUIRED QUIET)`, `pkg_check_modules(HWLOC ...)`, the hwloc status message and both + `PkgConfig::HWLOC` links in `cpp/monoprop/CMakeLists.txt`; remove the unit-runner's hwloc link in + `cpp/tests/CMakeLists.txt`. PkgConfig is currently required there only for hwloc; do not require it on monoprop's + behalf once that use is gone. Audit installed target/config output for residual direct hwloc requirements. Keep OpenMP + target discovery/link/export intact; a transitive MPI/OpenMP-runtime dependency on hwloc is allowed and is not a + monoprop dependency to remove. +- [ ] Remove hwloc installer implementation/configuration/help/output from `tools/install-deps.sh`, including + `--skip-hwloc`; remove explicit hwloc/libhwloc-dev from `tools/packages/{apt,brew}.txt`, wheel hooks and any direct CI + installs. Shared setup and devcontainer consume those manifests, so do not add duplicate replacement install blocks. + Remove direct hwloc arguments/buildInputs in both Nix files and replace the Nix workflow's + `pkg-config --modversion hwloc` smoke requirement with OpenMP configure/link/run validation via existing recipes. Keep + required OpenMP across flake package/MPI package/devshell checks and the deploy wheel matrix. Remove documentation + claiming hwloc/pkg-config is a direct monoprop prerequisite or that the library pins threads. Do not uninstall or + strip a library needed by the selected MPI/OpenMP runtime, or remove another dependency's legitimate pkg-config use. +- [ ] Migrate tests before deleting obsolete files: + +| Old coverage | Replacement/retirement | +| --- | --- | +| `partition_equivalence_tests.cpp`: energies, gradients, propagation, determinism, copies, Pauli, setters, partial contraction | `openmp_equivalence_tests.cpp`, budgets 1/2/3/4, both pictures, decoded term/value equivalence | +| facade raw-accessor errors | single-store raw-accessor consistency at every budget | +| factory throw/child construction | deprecated factory rejected before invocation; no worker terminate | +| `partition_group_clone_tests.cpp`: derived children | retire child-existence assertions; retain derived clone/update-initial-operator independence in `simulator_copy_tests.cpp` and `update_initial_operator.cpp` | +| `shm_comm_tests.cpp` / `hybrid_comm_tests.cpp`: private barrier algorithms | retire only after ownership/source-order/empty-payload/count/error assertions map to MPI/OpenMP tests | +| `hybrid_comm_sparse_plan_on_the_plain_mpi_path` | move unchanged semantics to ordinary MPI tests; unknown and known receive counts | +| plain-MPI half of `hybrid_comm_known_recv_counts_are_masked_through_the_plan` | preserve masked non-peer counts and sparse peer payload assertions | +| `flat_exchange_tests.cpp` replay dense/pairwise/dense-layout/empty tests | retain and run under ordinary MPI after deletion | +| hybrid MPI energy and size case | retain as MPI×OpenMP matrix case in `mpi_distributed_layer_equivalence.cpp` | +| poisoned waiter release | Task 3 MPI failure-driver and local joined-exception tests | +| `cpu_topology_tests.cpp`: topology enumeration, mask classification, pinning/L3 dealing | retire these library-owned policy tests; Task 3/10 count/precedence/no-affinity-mutation tests cover the retained library contract, and Task 11 verifies placement externally | + +Record individual retired case names and replacement names in the implementation acceptance artifact; do not merely +delete files until green. +- [ ] Audit references: + +```bash +rg -n 'PartitionGroup|ShmComm|HybridComm|PartitionBarrier|CpuRelax|is_partition_facade|run_on_all' cpp src +rg -n 'partition_|partitions|monoprop_PARTITIONS' cpp src README.md AGENTS.md docs/content/docs +rg -n -i 'hwloc|CpuTopology|PkgConfig::HWLOC|libhwloc-dev' \ + cpp cmake CMakeLists.txt pyproject.toml tools .github .devcontainer nix flake.nix justfile \ + README.md AGENTS.md docs/content/docs +``` + +First command should have no production matches except deliberately retained migration diagnostics/type names where +applicable; second must resolve to approved deprecated API/docs, not active ownership/fan-out. The hwloc audit must find +no direct discovery/link/install requirement or CpuTopology implementation; migration notes about removal and +third-party/transitive dependencies are allowed. Inspect each match rather than deleting mathematical/MPI partition +terminology indiscriminately. +- [ ] Update `justfile::test-cpp-tsan` and `.github/workflows/qa-analysis.yml` selection before retiring files; require + nonempty `openmp_` coverage. Keep `just test-cpp`'s unit-only export probe and `just test-find-package` consumer. +- [ ] Run R/M/S, C++ installed-consumer checks and full Python suite. Re-run Task 11 parity after deletion: removal can + change layout, initialization and measurements. Preserve archived baseline; do not make candidate its own baseline. + +**Gate:** No in-process communication runtime, library-owned worker threads, custom barriers, topology/affinity +subsystem, direct hwloc dependency or permanent legacy backend remain. Build/install/import a non-MPI candidate in an +environment without hwloc development headers/pkg-config metadata; inspect direct linkage instead of requiring the +absence of a runtime's own transitive hwloc dependency. One store per rank is verified by construction, tests and memory +accounting, not merely a flag. + +### Task 13: Final packaging, documentation and independent acceptance + +**Files:** +- Modify as required: `.github/workflows/{test,bench,bench_bare_metal,deploy,nix,qa-analysis}.yml`, `pyproject.toml`, + `tools/install-deps.sh`, `tools/packages/{apt,brew}.txt`, `.github/actions/setup/action.yml`, `justfile`, + `nix/{monoprop,devshell}.nix`, `flake.nix`, `.devcontainer/Dockerfile`. +- Extend existing consumer/probe: `cpp/tests/find_package_smoke/CMakeLists.txt`, + `cpp/tests/link_export_probe/link_export_probe.cpp`. +- Modify: `AGENTS.md`, `README.md`, `docs/content/docs/{building,benchmarks,openmp-migration}.mdx`, + `docs/content/docs/features/parallelism.mdx`, `docs/content/docs/concepts/algorithm.mdx`, `cpp/tests/README.md`, + public Python docstrings affected by ordering/runtime guidance. +- Test: existing wheel jobs; benchmark renderer tests; whole C++/Python suite. + +- [ ] Validate the Nix package, explicit MPI package and devshell paths after dependency removal. Current + `.github/workflows/nix.yml` owns these entrypoints; keep its OpenMP smoke gate consistent with the shared recipes. + Planned commands (separate candidate environment; none were executed in this refresh): + +```bash +nix flake check --print-build-logs +nix build --print-build-logs .#monoprop .#monoprop-mpi +nix run . -- -c 'import monoprop; print(monoprop.__version__)' +nix develop --command bash -euo pipefail -c ' + monoprop_ENABLE_MPI=OFF just build --reinstall-package monoprop --no-cache + cmake --build --preset skbuild-release + just test-find-package +' +``` + + Require installed runtime resolution and the extended OpenMP/both-basis consumer, not a pkg-config hwloc probe. + Archive MPI package import/smoke evidence too; the current default flake check alone covers only non-MPI packaging. +- [ ] Build/install/repair wheels through `.github/workflows/deploy.yml::build-wheels` and the `pyproject.toml` + cibuildwheel hooks for the currently advertised Linux x86_64/aarch64 and macOS matrix. Validate import and a threaded + propagation in a clean environment outside source checkout, MPI disabled. Inspect ELF/dylib dependencies and bundled + runtime licensing; check coexistence with NumPy/other imported OpenMP users. Do not claim Windows or unsupported + sanitizer/runtime combinations tested. +- [ ] Run `just test-find-package` against the installed package, extending its existing shared probe source rather than + creating a duplicate harness; compile a template kernel using OpenMP and both Majorana/Pauli APIs. The consumer must + not inherit source-tree include paths or manual linker flags. +- [ ] Document one store/rank and the resource-ownership contract explicitly: users choose P and T for their resources + and bind/pin ranks/threads with OMP_PLACES+OMP_PROC_BIND, srun --cpu-bind and/or mpiexec options. Normally use + OMP_NUM_THREADS=T; explicit library overrides have the documented precedence. Describe construction-time capture of + the OpenMP default at any rank count, no library hardware clamp/topology discovery/NUMA validation, possible OpenMP + team-size limits, and the one-NUMA-per-rank allocation assumption. Show matching allocation/rank-binding/thread-place + examples, including the single-core-rank-mask pitfall. Retain MPI serialized policy, no concurrent object/communicator + calls, callable lifetime, failed-operation non-reuse, capacity and numerical contracts. State that external placement + verification for benchmarks does not add a required topology library. +- [ ] State exact breaks: protected partition helpers removed, factory rejected, old automatic physical-core + selection/multi-rank serial default and library pinning removed in favor of OpenMP/user configuration, direct hwloc + prerequisite removed, raw coefficient order can differ, ABI requires rebuild. Explicit thread requests are not + hardware-clamped. Preserve high-level Python mathematical signatures. In prose/docstrings use `[Symbol][]` references, + not hard-coded API URLs. +- [ ] Run `prek run --all-files` before any separately authorized push, plus relevant tests, `just gen-api` and + `just build-docs`. Targeted hooks during development do not replace the all-files gate. Keep generated bindings/API + artifacts consistent; no hand-edited dispatch. +- [ ] Obtain fresh-context read-only spec/correctness and performance-memory reviews. Reviewers receive both documents, + implementation diff, retired-test ledger, raw baseline/candidate results, placement/capacity report, build logs and + known tool limitations. Any concurrency, ordering, memory or unsupported-claim finding blocks completion until + fixed/retested. +- [ ] Produce a final acceptance artifact containing: revision identities and overlay hashes; configuration matrix; + actual executed commands/results; numerical cases; maximum observed store sizes; MPI failure tests; runtime/library + identities; old/new line/file/runtime-component summary; per-cell time/peak medians and ratios; raw artifact paths; + reviewer findings/fixes; residual risks. Include actual output/artifact paths when delegated, not filenames mentioned + only in prompts. + +**Final acceptance checklist:** + +- [ ] Mathematical features, near-cutoff term retention, copies/updates/closures and graph paring pass existing oracles. +- [ ] Thread-count-stable ordering and approved deterministic reductions pass; no shared append, lazy-cache, packed-word + or MPI-worker race remains. +- [ ] Fixed uint32_t TermIndex and optional-MPI/non-MPI packaging pass. +- [ ] Rank-wide limits and real placement satisfy the selected baseline geometries. +- [ ] Every required runtime and exact peak-memory cell passes Task 11; unsupported or missing measurements are not + marked passed. +- [ ] No legacy runtime, direct hwloc dependency or replacement topology/pinning service remains; no equivalent + scheduler/concurrent store was introduced. +- [ ] Documentation, benchmark schema consumers and API generation agree. Operation/construction exactness gates the + five parity metrics; outer-window exactness controls diagnostic labels only. Frozen campaign coverage, unique paired + measurement artifacts and matching validation/placement provenance pass, including all four historical pytest families + and new bounded paring-construction cells. +- [ ] Shared preflight proves declared/observed equal-allocation shape. Every successful R=3 launch uses explicit + splitmix; linear rejection is tested separately. Current stable-gradient and streamed-initialization regressions pass. +- [ ] Required OpenMP works through shared package/setup, wheel/deploy, Nix package/devshell/check and + installed-consumer routes; centralized TSan selection remains nonempty. `prek run --all-files` is complete before an + authorized push. +- [ ] Owner receives evidence and integration choices; no unrequested commit/push/PR or merge is performed. + +## Stop-and-report template + +Use this instead of making an unapproved design change: + +```text +Blocked task/gate: +Checkout/revision/configuration: +Command or workload cell: +Expected contract: +Observed failure or ratio (raw artifact path): +Smallest reproducer / phase profile: +Changes already made and worktree status: +Approved bounded options attempted: +Decision required from owner: +``` + +Examples requiring a stop: a full rank cannot fit fixed uint32_t TermIndex, serial publication prevents parity, query +buffering exceeds peak baseline, the benchmark operator cannot verify the target allocation/placement externally, +unsupported OpenMP packaging, failure injection hangs, numerical changes alter retained terms, or the owner declines a +compatibility break. None authorizes a concurrent map, extra shards, altered MPI geometry, weakened oracle, or a +performance waiver. diff --git a/docs/superpowers/specs/2026-09-18-rank-local-openmp-design.md b/docs/superpowers/specs/2026-09-18-rank-local-openmp-design.md new file mode 100644 index 00000000..37eefee1 --- /dev/null +++ b/docs/superpowers/specs/2026-09-18-rank-local-openmp-design.md @@ -0,0 +1,321 @@ +# Rank-local OpenMP parallelization design + +## Status and authority + +This is the design accompanying the implementation plan at `../plans/2026-09-18-rank-local-openmp.md`. It records the +direction discussed with the owner; it is not a claim that the proposed implementation meets its performance gates. + +Source inspected: branch `perf/linear-routing-on-wire`, commit `290112c8289ab8015eb9a2c7651ac409839c5a88`. + +The owner requested a written implementation plan, not implementation or publication. The owner has approved removing +monoprop's direct hwloc dependency and making process/thread counts and placement user responsibilities. Approval of the +full plan is still required before executing it; the remaining compatibility policy is not an assertion that a breaking +C++ API change has already been approved. + +## Objective + +Replace thread-owned operator partitions and their in-process communication protocols with one compact operator store +per MPI rank, operated on by OpenMP worksharing. Preserve the mathematical features, memory footprint, and performance +of the existing library. Each MPI rank's assigned CPUs are assumed to lie within one NUMA domain. The user chooses the +number of processes P and requested threads per process T for the resources available, and configures the launcher and +OpenMP runtime to bind/pin processes and threads correctly. The library neither discovers topology nor repairs +placement. + +## Global constraints + +- C++23; retain the GCC 14 / Clang 18 minimum compiler versions and Python 3.11 floor. +- MPI remains optional and OFF by default; retain working non-MPI wheels. +- OpenMP becomes a required implementation dependency for the replacement runtime; support Linux x86_64, Linux aarch64, + and macOS builds covered by the current project. +- Each MPI rank owns exactly one operator store and one graph; threads are never communicator ranks. +- Remove monoprop's direct hwloc dependency and topology/affinity machinery; do not replace it with another + library-owned hardware-discovery or pinning mechanism. +- Users choose P processes and T threads per process and configure resource allocation, binding and pinning; monoprop + does not clamp T to physical cores or validate NUMA placement. +- Preserve packed operator rows, the keyless hash index, fixed uint32_t TermIndex, sparse state storage, and + inverted-index cosine recomputation. +- Do not introduce boost::concurrent_flat_map, hidden shards, a dense duplicate store, a second persistent key store, a + full per-thread operator, or full-vector coefficient double buffering. +- Preserve Majorana and Pauli bases, Heisenberg and Schrödinger pictures, cutoffs, graph build/replay/paring, gradients, + partial contraction, copying, and initial-operator updates. +- MPI calls execute on the controlling thread outside library-created OpenMP worksharing regions; retain + MPI_THREAD_SERIALIZED rather than changing the initialization contract. +- No exception escapes an OpenMP structured region; distributed operations fail coherently rather than leaving peers in + collectives. +- No global omp_set_num_threads/omp_set_dynamic/omp_set_nested mutation; each worksharing region receives an explicit + per-object thread budget. +- Use bitmap traversal or anticommutation pass for the existing operation; reserve prefix offsets for cumulative counts + used to allocate output ranges. +- Do not change numerical tolerances, truncation rules, generator order, source ordering, or MPI routing to make a + regression disappear. +- Do not declare success without target-hardware runtime and peak-memory measurements at the same rank count, CPU + allocation, workload, and fixed 32-bit TermIndex as the baseline. +- No automatic regression allowance: the acceptance target is candidate/baseline <= 1.00 for the median runtime and + median peak memory of every required workload cell. Noise triggers more measurement, not a silent waiver. +- Update AGENTS.md, README.md, and affected user/contributor docs in the same implementation changes that alter their + documented behavior. +- No permanent legacy/new-backend selector. Temporary coexistence is confined to the development branch; remove it + before accepting the replacement. + +## Architecture + +One rank owns its `MPOperator`, `MPGraph`, and ordinary MPI communicator. A small header-only worksharing helper +supplies deterministic block IDs, explicit thread budgets, and exception collection. It owns no threads, queues, +communicators, or operator data. OpenMP owns worker teams. + +Local kernels read an immutable structure or write preallocated, non-overlapping output ranges. The controlling thread +resizes shared containers and publishes structural changes between phases. Initially, insertion into the keyless hash +table and inverted-index maintenance remain single-writer. This is an explicit performance gate, not an assumption that +insertion is negligible. + +Only recognized pure `LengthCutoff`/`SupportCutoff` targets may run in worker traversal initially. Arbitrary cutoff +callbacks and current basis-change closures retain serial invocation; a const `std::function` does not establish that +its captured state is thread-safe. + +A gate remains sequential with respect to other gates: + +1. Prepare lazy caches and snapshot the pre-gate row count. +2. Traverse disjoint inverted-index word ranges, preserving the fused cosine update and capturing pre-update source + coefficients in the existing record streams. +3. Resolve the leader pass: local resolution, remote query exchange, owner resolution/insertion, and response handling + in the existing order. +4. Resolve the follower pass after leader matches are visible. +5. Publish deferred local misses after both passes, preserving existing insertion ordering. +6. Finalize the graph layer or apply disjoint fused rotation records. + +The freeze boundary in step 1 controls which source rows generate work. Do not change the current post-insertion +`LayerCore::scaled_count` convention when finalizing a graph layer. + +Replay uses the existing graph and coefficient arrays. Parallelize cosine blocks, endpoint packing, local rotation +pairs, remote endpoint application, and ordered local reductions. Do not parallelize gate order or parameter order. Keep +MPI replay transport behavior unchanged. EXISTING: `Evolution.cpp` derives rank-uniform `wire_bits`; +`detail/mpi/Exchange.h::post_flat_alltoallv` uses a dense `PeerPlan` but posts only nonzero Isend/Irecv legs when +`wire_bits>0`, otherwise MPI_Ialltoallv. HybridComm derives its sparse plan from all local partitions' rows. Preserve +plain-MPI sparse on-wire replay at S=1; a new generator-derived replay plan is outside this refactor. + +## Ownership rules + +| Structure | Parallel access allowed | Structural mutation owner | +| --- | --- | --- | +| Packed rows / keyless table | Frozen `find_batch_positions` with disjoint ID/hash outputs | Controlling thread initially | +| Inverted index / row parity | Reads after all lazy initialization finishes | Controlling thread initially | +| Existing coefficients | Disjoint cosine indices or proven-disjoint rotation records | Caller reserves/resizes; workers update assigned elements | +| Query records | Reachable-window and self-position streams; merged in source-range order | Each private buffer's owner, then controlling thread | +| Incoming query probe arrays | Disjoint deserialization/probe output ranges | Caller sizes once | +| Missing-row IDs | Assigned in canonical sender/query or source/query order | Prefix offsets and allocation on controlling thread | +| MatchedEpochSet | Distinct element writes by leader records, followed by a barrier before follower reads | Caller grows/resets; workers may mark unique targets | +| Graph endpoint arrays | Disjoint preallocated records | Caller sizes and finalizes | +| Packed phase words | Not concurrently written merely because record IDs differ | Keep packing serial initially | +| MPI buffers and requests | Pack before posting; do not mutate in-flight buffers | Controlling thread posts/waits and sizes buffers | + +EXISTING: `FusedScanResult` in `Scan.h:208–227` has six `WindowVec` streams plus `leader_self`/`follower_self`. +`SlotWindow` indices are not flat slots, including when base is nonzero. Preserve the precomputed `gen_shift`, variable +`QueryWire` records and `QueryForm`; self stages are positions, never serialized wire. Merge only their logical lengths +and rebase position offsets while preserving source/value alignment. Private query buffers have at most +O(T*window.count) empty-container metadata, not O(number_of_bitmap_blocks*R). Per-record payload across those buffers +must be O(total emitted records), never O(T*operator_size). Release/move drained temporary buffers before starting the +next memory-heavy phase. Before parallel decode, the caller checks variable record boundaries, canonical escaped headers +(`h.bits == QueryWire::header_bits_for(h.k)`) and position-count prefix sums, sizes arrays once, then workers decode +into disjoint spans. Frozen probes retain `find_batch_positions` and cached hashes; publication remains serial +`set_positions` → `bulk_insert_hashed` → reindex. No per-rank or full-store dense monomial reconstruction is allowed. +Existing bounded paired-only state scoring may still reconstruct its one required monomial; do not turn that exception +into a batch/store. Task 6 defines the checked preparation sequence. + +## Determinism and numerical contract + +- Bitmap traversal output is concatenated by increasing source-word range, never completion order. +- Each destination rank's query order remains its source-order subsequence. +- Incoming misses retain sender-rank/query order; local misses retain source-query order. +- Stable row IDs do not use an atomic allocation counter whose results depend on scheduling. +- Reductions use fixed logical blocks independent of thread count and combine partials in block order. +- Repeated calls with a fixed rank geometry must be deterministic. Cross-thread/rank numerical equivalence uses the + existing project tolerances; do not promise bitwise equality to the old floating-point reduction tree. +- Preserve the exact serial behavior of standalone `inner_product` and `EvalState::dot`; introduce explicitly named + `inner_product_threaded` / `EvalState::dot_threaded` helpers rather than silently changing their existing contracts. + Evaluation adopts the new association only after the numerical policy gate. Dot blocks span fixed logical row ranges + in both sparse and dense representations; cosine blocks span fixed logical word ranges in both lazy and stored-mask + representations. +- Preserve `LayerAngle`, `CosRecordView`, all three `CosCallbacks` (`LayerCosScale`, `LayerCosAccumulate`, + `LayerCosIndices`) and current pre-layer-unit derivative formulas. Forward record append and reverse predivide/restore + remain serial: record indices may duplicate. Threaded gates include deep non-singular circuits, vanishing cosine and + no-record cases from `tests/test_deep_circuit_gradient.py`, plus exact record restoration in + `cpp/tests/combined_recompute_equivalence.cpp`. +- Near-cutoff tests compare retained operator terms, not just final energies. +- Preserve streamed `for_each_paired_monomial` initialization order and local reservation/cardinality checks in + `MonomialPropagator.inl:126–151,197–227`. At S=1 estimates use real ranks, never T. Reject impossible cardinality + before workers; never materialize the global paired basis per rank. + +## Resource ownership and deployment + +- The user is responsible for choosing P processes and T threads per process to match available resources, including any + deliberate SMT use. Normally set P with the scheduler/MPI launcher and T with `OMP_NUM_THREADS=T`. +- The user is responsible for binding/pinning processes and threads to the correct hardware using, for example, + `OMP_PLACES` + `OMP_PROC_BIND`, `srun --cpu-bind`, and/or `mpiexec` options. Scheduler allocation, rank binding and + OpenMP places must agree: binding a rank to one core while requesting many threads does not allocate additional cores. + Each rank's allocation must accommodate its workers within one NUMA domain; avoid unintended overlap with other ranks. +- Monoprop validates thread-count representability, not hardware suitability. It does not discover physical cores, + choose SMT policy, redistribute CPUs, change affinity, validate NUMA containment or prevent oversubscription. OpenMP's + own limits and small-work/nested serial paths may use fewer workers; they do not certify adequate resource allocation. +- Delete `CpuTopology` with the partition runtime and remove direct hwloc discovery/link/install requirements. Do not + retain hwloc for mandatory diagnostics. Benchmark operators still verify placement with runtime/launcher diagnostics + and external tools; those are evidence requirements, not library dependencies. MPI or the OpenMP runtime may + independently depend on hwloc. + +## Thread selection and compatibility policy + +The replacement stores an immutable `parallel::Options` per propagator; copies copy that budget. Without a library +override, capture the calling context's OpenMP default via `omp_get_max_threads()` at construction, normally configured +by `OMP_NUM_THREADS`. Do not parse that environment variable independently; OpenMP also supports list-valued settings +and application-level runtime configuration. Later runtime-setting changes do not alter an existing object's captured +budget. A worksharing region may use fewer workers than the budget. Thread count never changes ownership or graph +layout. MPI ranks may use different budgets. + +During the prototype only, add a final C++ constructor argument `size_t num_threads = 0` after the existing +`child_factory` argument. A nonzero value is allowed only when the constructor resolves one legacy partition; it enables +threaded kernels on that single store. Existing legacy children use one kernel thread. This gives a testable transition +without nesting runtimes. + +At cutover: + +1. Keep the existing `partitions` constructor slot and low-level binding keyword for a deprecation window. Explicit + positive values become a deprecated thread-budget request, with a once-per-process warning. `partitions=1` remains + serial. There are no actual thread partitions. +2. Keep `monoprop_PARTITIONS=off|auto|N` as a deprecated, warned thread-budget alias. `off` means one thread; positive N + requests that budget. `auto` and malformed values fall through to a valid `monoprop_NUM_THREADS` override, otherwise + the OpenMP default. They no longer discover hardware. +3. A positive explicit C++ `num_threads` wins over environment settings, but supplying it alongside positive explicit + `partitions` is an error. Otherwise precedence is: explicit positive partitions, then monoprop_PARTITIONS + off/positive N, then valid monoprop_NUM_THREADS, then the captured OpenMP default. Preserve the existing + monoprop_NUM_THREADS parser/range; recommend OMP_NUM_THREADS for normal configuration and unset library overrides + when using it. +4. The OpenMP-default path is the same for one or multiple MPI ranks. Do not retain the old automatic physical-core + selection or multi-rank default of one thread. If OMP_NUM_THREADS is unset, use the runtime's default, without a + library promise about that default's suitability. Explicit positive requests must fit `int`; reject unrepresentable + counts instead of truncating or hardware-clamping them. Public constructor zero remains the no-explicit-request + sentinel; resolved Options always contains a positive count. Document the changed default-selection policy. +5. `OMP_PLACES` and `OMP_PROC_BIND` control OpenMP placement; monoprop never modifies affinity. Explicit library + overrides take precedence over the runtime's default thread budget. Runtime limits such as OMP_THREAD_LIMIT and + dynamic-team adjustment may reduce actual teams even when a larger count was requested; correctness must not depend + on receiving exactly T workers. No place-union/physical-core query is needed to select or validate T. Verify actual + placement externally for performance acceptance, not during ordinary propagator construction. +6. Retain the `PartitionChildFactory` type and constructor argument temporarily for source diagnostics, but reject a + non-null factory with `PropagatorConfigError` explaining that thread-owned children no longer exist. Do not ignore it + or invoke it once with changed semantics. +7. Remove protected partition-only helpers and facade-specific accessor errors. This is an explicit C++ + extension-interface breaking change requiring a release note; do not describe the refactor as universally source/ABI + compatible. Keep the virtual destructor, virtual `update_initial_operator`, its protected nonvirtual + `apply_initial_operator_` helper, and independent virtual `clone_` hook (`MonomialPropagator.h:286–305`). +8. Python mathematical constructors/methods remain unchanged. Low-level `_core` gains the optional `num_threads` keyword + used by tests; the public frontends continue to use environment configuration. +9. Returned coefficient/term ordering can differ from the old partition-block order. Candidate query/row ordering is + deterministic across thread budgets at fixed rank geometry; this does not promise the old raw export layout. Validate + key/value associations, not coefficient-only sorted lists. +10. EXISTING routing is boolean linear/splitmix (`Routing.h::linear_requested`, `linear_bits_for`, `make_router`). + Linear is the default and requires power-of-two R; unsupported geometry throws `UnroutableGeometry`, not a fallback. + Preserve linear rank mapping, generator `rank_shift`/`dest_from_shift` and peer selection. Splitmix previously + placed a term on `floor((hash % (R*S))/S)`; S=1 uses `hash % R`. Obtain explicit approval of that ownership change, + including rank-local export/load/reduction effects; do not retain hidden shards. New thread budgets never change + ownership. Test default-linear R=3 rejection separately. Every successful R=3 suite or benchmark explicitly sets + `monoprop_ROUTING=splitmix`. Compare global maps and per-rank loads/peaks at identical routing/seed/rank geometry. + +The plan's final removal task may proceed only after the owner approves this policy as part of the plan, or provides a +replacement policy. Implementation agents must not invent another migration. + +## Safety and failure policy + +The worksharing helper catches each worker exception, joins the region, and rethrows on the calling thread. Caller-side +distributed phase guards print the underlying failure and call MPI_Abort when communicator size exceeds one. Catch +failures inside the lifetime of active exchange tickets, before stack unwinding can wait on peers that never posted; an +outer catch alone is insufficient. Successful paths explicitly wait as before. For single-rank operation, rethrow the +original exception. An operation that failed after mutations is not transactional and the propagator must not be reused. +No rank-local early return may bypass a collective peers will enter. Validate MPI_THREAD_SERIALIZED also when MPI was +initialized externally. Distributed entry from an external OpenMP team is not a supported calling contract; numerical +helpers may serialize when nested, but this does not authorize MPI from workers or concurrent evaluation on a shared +communicator. + +Lazy initialization, allocation, and cache mutation must be outside worker bodies unless storage is strictly +worker-private. Do not assume `const` means thread-safe: both the inverted index and parity cache have lazy mutation +today. + +## Performance acceptance and stopping + +Keep the baseline revision separately built. The benchmark operator must set P/T and verify the allocation and actual +worker placement using OpenMP/launcher diagnostics and external tools, not a monoprop topology service. Use identical +MPI rank placement and physical cores; changing ranks per domain or widening indices is not a valid comparison. Measure +at least five fresh-process observations per required cell, alternating baseline/candidate. Each observation has one +benchmark round. Compare medians per cell, both runtime and kernel VmHWM (sum across ranks and maximum rank peak). +EXISTING operation windows and nested-window peak preservation already work (`benches/conftest.py` and benchmark +`memory/cpu.py`). Extend them, do not reimplement them. Separate fresh-process whole-construction measurements remain a +deliberate policy to include graph creation without unrelated resident session graphs, not a workaround for broken +nesting. PROPOSED artifacts independently aggregate `opmemexact` for operation metrics, `memhwmexact` for outer +`memhwm`/`memhwm_max`, and `construction_exact` for the separate construction window. Never certify outer peaks with +`opmemexact`; legacy absent flags are unknown. Renderers label each metric using its own provenance. The parity gate has +exactly FIVE metrics: runtime, operation peak sum/max, and whole-construction peak sum/max. Operation and construction +exactness must both be true; exit-RSS fallbacks cannot pass. Outer peaks/ratios are diagnostics only: non-exact or +unknown outer evidence must be labeled honestly but cannot fail an otherwise passing five-metric comparison. All five +required ratios must be <=1.00. Rank-peak sums are upper bounds, not simultaneous node-footprint measurements. + +If a ratio exceeds 1.00, repeat the cell for five additional observations on each arm. If the combined median still +exceeds 1.00, the gate is not passed. Profile and use only the bounded optimization menu in the plan; otherwise stop +with a regression report. No geometric-mean improvement cancels a failing required cell. Missing target hardware is a +validation blocker, not permission to claim parity. + +Benchmark type/length assertions do not establish numerical parity. A separate untimed validation process records +energies, gradients where applicable and decoded global term/value maps, compared against the identical baseline +operation with the existing project's tolerance policy. Paring compares the same pared callable/threshold on both arms. +Missing/non-finite evidence fails acceptance; replicated Heisenberg core terms are checked once rather than summed +across ranks. Keep exact small-system and finite-difference tests as independent oracles. Preserve all four current +pytest families: build_graph, graph-free propagate, energy and gradient. Record random `lower_atol=None`, fixed-model +cutoffs/atols and both unpared/default and `pare_threshold=1e-10` evaluation profiles. Keep original paring-construction +performance coverage through an explicitly NEW bounded driver measurement, `pare_functional_construct`, specified in +Task 1: graph ready before timing/op window, threshold 1e-10, returned gradient callable retained through close, untimed +scalar/full-gradient validation, and separate whole-construction peak scope. This is proposed measurement work under +full-plan approval, not an existing pytest node or instrumentation-only edit. Use identical driver code on both arms; do +not build a general benchmark framework. + +Freeze and archive an owner-approved campaign inventory and digest before baseline collection. It lists every required +profile/node/geometry/routing cell independently of the observations. Comparator input must match that inventory, so +removing a whole cell from both arms cannot pass. Each sample references one fresh timed artifact, one separate fresh +construction artifact, and a matching untimed validation artifact. Join only matching cell/arm/binary/configuration, +parameters, geometry/routing/allocation and campaign provenance. Timed/construction run IDs and files cannot be reused; +require five distinct pairs per arm/cell, or ten after prescribed repetition. Validation and placement diagnostics can +be reused only for the same cell/arm/binary/configuration. Task 1 specifies the CLI, file schema, identity checks and +independent placement references; no caller boolean or synthetic-data bypass substitutes for that evidence. + +The shared shape-preflight overlay must accept declared baseline `monoprop_PARTITIONS=T` and candidate +`OMP_NUM_THREADS=T` with library overrides unset, on the same allocation. Verify declarations against observed shape; +arm labels alone cannot establish shape. Extend existing ladder/provenance/term-count fields without renaming historical +nodes. `tools/capture-baseline.py` supplies supplementary small-fixture evidence only; it cannot replace global, +ownership-aware benchmark validation with gradients and provenance. + +The full-rank store must fit the fixed uint32_t TermIndex ceiling (`TypeAliases.h:39`). Row IDs exclude UINT32_MAX, +which is the index sentinel; row counts and some graph counts may equal it. Extend existing overflow guards/tests rather +than claim there are two index-width builds. The store must also fit capacity arithmetic and MPI count limits. If not, +report that capacity incompatibility; do not widen indices, change rank geometry, or add local shards without a new +design decision. + +## Non-goals + +No new executor framework, custom concurrent hash table, GPU backend, load-balancing/repartitioning algorithm, +asynchronous gate pipeline, GIL-policy change, graph-format redesign, new replay transport, or replacement +topology/affinity-management subsystem. Do not remove useful lifetime/ordering comments merely to lower line count. + +## Execution handoff and supported build routes + +All future code/test/build commands in the companion plan are planned, not executed during this document refresh. Only +these two Markdown documents are authorized now. Before implementation, recheck exact HEAD and the plan's baseline-drift +table; stop on changed signatures, routing, capacity, numerical or benchmark contracts. Keep 13 gated tasks and record +RED/GREEN evidence for new behavior, plus baseline results for preservation tests that may already pass. Do not +manufacture RED by removing earlier plumbing. Use kernel-specific worker observations for threading participation. Do +not execute past a failed gate or infer approval from this refresh. + +Required OpenMP must link PUBLIC on both `monoprop-objs` and the shared `monoprop` target, plus installed +`find_dependency(OpenMP REQUIRED COMPONENTS CXX)`; INTERFACE alone does not link the shared library itself. Cover wheel +hooks in `pyproject.toml`, actual wheel jobs in `.github/workflows/deploy.yml`, `tools/install-deps.sh`, shared +`tools/packages/{apt,brew}.txt` and setup action, `.devcontainer/Dockerfile`, `nix/{monoprop,devshell}.nix`, `flake.nix` +package/check entrypoints and the Nix workflow. Remove direct hwloc requirements/checks LAST, with the partition +runtime; legitimate transitive dependencies remain. Use environment-driven MPI recipes from `justfile` so isolated +builds receive mpi4py; `--all-extras` alone cannot do so. Migrate centralized TSan selection before retiring partition +tests, preserve the unit-only export probe, and extend `just test-find-package` rather than create a second consumer +harness. MPI OFF clean wheels remain mandatory. Before any separately authorized push, run `prek run --all-files`, +relevant tests and docs generation/build. From 9bede897d99ac7e2a53442885629d5032f1b7df4 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Roberto=20Di=20Remigio=20Eik=C3=A5s?= Date: Tue, 22 Sep 2026 15:51:56 +0200 Subject: [PATCH 02/64] chore: spec and plan revision after grill-with-docs --- CONTEXT.md | 24 + .../plans/2026-09-18-rank-local-openmp.md | 584 ++++++++++++------ .../2026-09-18-rank-local-openmp-design.md | 322 +++++++--- 3 files changed, 654 insertions(+), 276 deletions(-) create mode 100644 CONTEXT.md diff --git a/CONTEXT.md b/CONTEXT.md new file mode 100644 index 00000000..258eac93 --- /dev/null +++ b/CONTEXT.md @@ -0,0 +1,24 @@ +# Monoprop + +Terminology for retained operators and their numerical comparison. + +## Language + +**Propagator instance**: +An independent propagation context for an operator and its evolution. A copy is a separate instance with independent +operator state. +_Avoid_: Partition when referring to a copied propagator + +**Rank-local operator**: +The portion of one propagator instance's global retained operator assigned to one rank. +_Avoid_: The process's operator when several independent propagator instances are present + +**Global retained operator**: +The collection of retained term keys and associated coefficients for one propagated operator, independent of how those +terms are distributed. +_Avoid_: Energy or gradient when referring to the operator itself + +**Retained-operator equivalence**: +Identical global retained term keys, with corresponding coefficients equal within the applicable numerical tolerances. +Agreement of energies or gradients alone does not establish retained-operator equivalence. +_Avoid_: Observable agreement as a synonym diff --git a/docs/superpowers/plans/2026-09-18-rank-local-openmp.md b/docs/superpowers/plans/2026-09-18-rank-local-openmp.md index 52c1165d..05a3456a 100644 --- a/docs/superpowers/plans/2026-09-18-rank-local-openmp.md +++ b/docs/superpowers/plans/2026-09-18-rank-local-openmp.md @@ -1,8 +1,9 @@ # Rank-local OpenMP Implementation Plan > **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or -> superpowers:executing-plans to implement this plan task-by-task. Read both documents completely; explicit full-plan -> approval is required before execution. Do not infer missing contracts. Checkboxes track future work. +> superpowers:executing-plans to implement this plan task-by-task. Read both documents completely; full-plan execution +> requires explicit approval. A separately authorized baseline-only calibration may precede campaign approval, as Task 1 +> describes. Do not infer missing contracts. Checkboxes track future work. **Goal:** Replace thread-owned partitions with one compact operator store per MPI rank and OpenMP worksharing, accepting the replacement only after correctness, runtime, and peak-memory parity have been demonstrated. @@ -19,12 +20,23 @@ Boost.Test, pytest. Monoprop's direct hwloc dependency is removed with the legac **Spec:** [Rank-local OpenMP design](../specs/2026-09-18-rank-local-openmp-design.md). Read both documents before execution. -**Status:** Planning only. Refreshed against branch `perf/linear-routing-on-wire`, HEAD -`290112c8289ab8015eb9a2c7651ac409839c5a88`. The owner approved removing monoprop's direct hwloc dependency and making -process/thread counts and placement user responsibilities; the full refactor is not authorized for execution. No -implementation, builds, tests, or benchmarks were performed in preparing this plan. All future code/test/build commands -below are planned, not executed. The refresh authorizes only these two Markdown documents. EXISTING labels name current -contracts; PROPOSED labels name work requiring full-plan approval. +**Status:** Planning only. Source baseline: `perf/linear-routing-on-wire` at +`290112c8289ab8015eb9a2c7651ac409839c5a88`; published planning commit: `refactor-parallelization` at +`38eb13d80a95dd1ac60e762f14f68c9a5925f755`. This documentation-only interview revision records approved runtime, +compatibility, numerical and failure policies. OpenMP is mandatory; partition configuration is removed immediately; +thread budgets are launch-environment-only and captured at construction. The fixed32 capacity reduction is accepted. +The owner selected one AWS EC2 `c8a.metal-24xl` for implementation/testing and initial acceptance. Production targets +are dual-socket Sapphire Rapids and AMD Zen3/Zen4/Zen5 HPC nodes, commonly four NUMA domains/socket. Hardware SMT may +be on or off, but SMT workers are not recommended: always configure one worker per physical core. Passing the frozen +single-instance EC2 campaign is sufficient for initial acceptance; HPC performance remains separately unqualified. +Multi-node qualification is pending a proper HPC cluster/interconnect, not a cloud-network substitute. Record actual EC2 +topology/software and agree representative workload sizes before freezing the campaign or collecting acceptance data. +The owner approved baseline-only, unscored sizing trials before the freeze, once separately authorized. Small/medium +profiles cover the full geometry sweep; large profiles cover only the full-node one- and two-ranks-per-domain cases. + +No implementation, engine builds/tests or benchmarks are authorized by this revision. All future execution commands +below are planned, not evidence of passing gates. EXISTING labels describe current source; PROPOSED labels describe +future work. Full implementation and any publication of this revision require separate authorization. ## Global Constraints @@ -32,11 +44,15 @@ contracts; PROPOSED labels name work requiring full-plan approval. - MPI remains optional and OFF by default; retain working non-MPI wheels. - OpenMP becomes a required implementation dependency for the replacement runtime; support Linux x86_64, Linux aarch64, and macOS builds covered by the current project. -- Each MPI rank owns exactly one operator store and one graph; threads are never communicator ranks. +- Each independent propagator instance owns one packed operator store per MPI rank; threads are never communicator + ranks. Preserve independent copies, immutable graph-core sharing, retained coefficient/state snapshots and pared + graphs. - Remove monoprop's direct hwloc dependency and topology/affinity machinery; do not replace it with another library-owned hardware-discovery or pinning mechanism. - Users choose P processes and T threads per process and configure resource allocation, binding and pinning; monoprop does not clamp T to physical cores or validate NUMA placement. +- SMT use is not recommended: allocate at most one monoprop worker per physical core, including on SMT-enabled hosts. + Enforce this through user launch/binding settings and acceptance evidence, not a library hardware policy. - Preserve packed operator rows, the keyless hash index, fixed uint32_t TermIndex, sparse state storage, and inverted-index cosine recomputation. - Do not introduce boost::concurrent_flat_map, hidden shards, a dense duplicate store, a second persistent key store, a @@ -46,7 +62,7 @@ contracts; PROPOSED labels name work requiring full-plan approval. - MPI calls execute on the controlling thread outside library-created OpenMP worksharing regions; retain MPI_THREAD_SERIALIZED rather than changing the initialization contract. - No exception escapes an OpenMP structured region; distributed operations fail coherently rather than leaving peers in - collectives. + collectives. Enforce invalidation after failed mutation; dependent functionals cannot reuse an invalid owner. - No global omp_set_num_threads/omp_set_dynamic/omp_set_nested mutation; each worksharing region receives an explicit per-object thread budget. - Use bitmap traversal or anticommutation pass for the existing operation; reserve prefix offsets for cumulative counts @@ -64,14 +80,18 @@ contracts; PROPOSED labels name work requiring full-plan approval. Additional execution rules: -- **Do not start until the owner authorizes implementation.** Approval must include the proposed compatibility policy, - required OpenMP dependency, floating-point policy, and strict performance gates in the design. If any is rejected, - revise these documents first. +- **No runtime work or implementation is authorized by the design interview.** Obtain separate authorization for a + bounded baseline-only calibration/build phase; it may precede campaign approval. Candidate engine changes require + implementation authorization and an owner-approved frozen campaign. Formal baseline collection also waits for that + freeze. Simplification without regressions is the goal; reject the architecture if it cannot earn acceptance. The + campaign bounds performance claims, not mathematical correctness across the supported API. - P (called R in existing code/benchmark geometry) is the user-selected process count; T is the requested threads per - process, normally set with `OMP_NUM_THREADS`. Users must allocate enough resources and configure rank/thread binding + process, normally set with `monoprop_NUM_THREADS`. Only an unset override selects the OpenMP runtime default. Users + must allocate enough resources and configure rank/thread binding with OpenMP, `srun --cpu-bind`, and/or `mpiexec` options. Each rank's workers are assumed to fit its allocation within - one NUMA domain. Do not create a topology service/NUMA scheduler, silently change P/T, or attempt to prevent - user-created oversubscription. + one NUMA domain. Documentation recommends against SMT workers: allocate one worker per physical core, including when + hardware SMT is enabled. Do not create a topology service/NUMA scheduler, silently change P/T, or enforce that + recommendation through library clamping or affinity changes. - Do not touch pre-existing `.codegraph/` or `.direnv/`. Use an isolated implementation checkout and a separately built baseline. Follow the worktree/GitButler skills for version-control writes; no raw git write commands. - New behavior follows RED → minimal implementation → GREEN → diff review. Run tests before production edits and record @@ -87,7 +107,7 @@ Additional execution rules: | Stage | Tasks | Gate before proceeding | | --- | --- | --- | -| Baseline | 1 | Reproducible numerical/performance baseline and approved policies | +| Baseline | 1 | Authorized baseline-only calibration, approved frozen inventory, fresh formal baseline evidence | | Small prototype | 2–4 | OpenMP packaging works; one-store serial and threaded replay agree | | Shared-store construction | 5–7 | Stable queries/IDs, safety, construction scaling and scratch measured | | Remaining evaluation kernels | 8–9 | Replay, derivative, retained closures and reductions validated | @@ -138,6 +158,37 @@ Handoff checklist: exact revision/binary and overlay hashes; owner approvals; la commands/results; remaining gate/capacity/placement blockers; fixed routing and workload manifest; raw artifact paths. Never hand off only “tests pass”. No builds/tests/imports/installs/VCS writes are authorized by this document refresh. +### Remote Task 1 entry point + +The baseline-only pilot belongs to Task 1. It is not a prerequisite that must run on the planning machine before handing +this plan to an implementer. Run it on the authorized execution host, currently c8a.metal-24xl. If the host differs, +clarify the hardware decision before treating its sizing results as applicable to the target campaign. +No remote access, provisioning or runtime execution is authorized merely by reading these documents. + +Transfer the latest spec, plan and CONTEXT.md with the source checkout. The historical planning commit alone does not +include subsequent interview decisions. Verify document contents/digests on the execution host, including the 2-hour, +15-minute and 75% calibration limits. Documentation-only commits above the source anchor are not engine drift; inspect +semantic source/build/benchmark changes against the baseline-drift checkpoint rather than require HEAD to equal the old +planning commit. Keep the measured baseline source pinned to 290112c8289ab8015eb9a2c7651ac409839c5a88. + +A bounded starter authorization may cover only host/source preflight, documented project-local setup, an isolated +baseline build, existing baseline API/benchmark sizing trials and their report. It does not authorize candidate engine +changes, formal acceptance samples, commits, pushes or a PR. Read both documents completely before starting; distinguish +EXISTING baseline behavior from PROPOSED replacement behavior rather than making the baseline conform to the new design. +Do not infer subagent/delegation permission from the plan's choice of execution skills. + +Before expensive commands, state the observed host/allocation, source/doc identities, permitted scope, resource limits +and a durable artifact directory outside the source checkout. Preserve the source checkout and record any approved +measurement overlay separately. On interruption, retain commands/results, trial outcomes and remaining budget in a +checkpoint; resuming does not reset the pilot limits. + +The pilot handoff contains absolute artifact/report paths and digests; host/toolchain/binary provenance; verified +placement; all tried configurations and outcomes; time/memory observations including export/validation; proposed exact +small/medium/large workload and geometry/routing cells; and the estimated time/cost of the formal campaign, including +prescribed extra repetitions. State blockers and unmeasured items explicitly. Stop for owner approval of that campaign +and budget, and authorization for the remaining work. Task 1 is not complete until its tools/tests and fresh post-freeze +formal baseline evidence also pass. No pilot result is a parity claim or an acceptance sample. + ## File responsibilities Paths in task lists are relative to the repository root. @@ -181,10 +232,13 @@ must enter CMake FILE_SETs, not merely compiler include paths. before writing either endpoint. 10. Gate loops, reverse-gradient loops, repeated-parameter accumulation, graph append, store publication, and MPI remain ordered on the caller. -11. Empty local stores still follow the same distributed collective sequence. Runtime thread budgets may differ by rank - without changing ownership. -12. Ordinary kernels fall back to serial inside an existing OpenMP region. This does **not** authorize concurrent MPI - calls or concurrent calls on one propagator/retained functional. +11. Empty local stores still follow the same distributed collective sequence. Launch-time thread budgets may differ by + rank without changing ownership; each object's captured budget is immutable. +12. Internal numerical kernels fall back to serial inside an existing OpenMP region. The supported host enters through + one controlling thread per rank, outside active host OpenMP teams, with no overlapping dependent-state or host MPI + calls. The fallback does not extend that calling contract. +13. A post-mutation failure invalidates the affected owner and its dependent functionals. Check validity before later + state access or copying; independent earlier copies remain valid. Pre-mutation validation errors do not poison. ## Build/test command library @@ -192,12 +246,15 @@ These commands are to run **during implementation**, not evidence of passing tes Unset stale sanitizer/build-type and MPI override settings before Release commands; R explicitly uses `monoprop_ENABLE_MPI=OFF` as well as its CMake OFF setting. A configuration change requires forced reinstall and regenerated tests; `uv sync` alone does not reliably relink the C++ executable. For ordinary suite runs explicitly set -`OMP_NUM_THREADS=1`; this is test-runner policy, not a library multi-rank default. Dedicated parallel tests override -their object budget or launch fresh processes with a selected OMP_NUM_THREADS on an adequate allocation. +`monoprop_NUM_THREADS=1 OMP_NUM_THREADS=1`; this is test-runner policy, not a library multi-rank default. Dedicated +kernel tests pass internal Options; propagator tests launch fresh processes with a selected environment budget on an +adequate allocation. Unset removed partition settings in candidate launches. OpenMP-fallback tests specifically unset +monoprop_NUM_THREADS. ### R: non-MPI Release ```bash +export monoprop_NUM_THREADS=1 OMP_NUM_THREADS=1 monoprop_ENABLE_MPI=OFF uv sync --all-extras --group workspace-test --group bench \ --reinstall-package monoprop --no-cache \ --config-settings-package='monoprop:cmake.define.monoprop_ENABLE_MPI=OFF' -v @@ -220,6 +277,7 @@ regeneration. ### M: MPI Release ```bash +export monoprop_NUM_THREADS=1 OMP_NUM_THREADS=1 # Environment switch also adds mpi4py>=4.1.0 to the isolated build requirements. monoprop_ENABLE_MPI=ON uv sync --all-extras --group workspace-test --group bench \ --reinstall-package monoprop --no-cache \ @@ -255,6 +313,7 @@ There is no alternate index-width build. `cpp/monoprop/TypeAliases.h:39` fixes T ### S: sanitizer profiles ```bash +export monoprop_NUM_THREADS=1 OMP_NUM_THREADS=1 SKBUILD_CMAKE_BUILD_TYPE=AsanUbsan \ monoprop_ENABLE_MPI=OFF SKBUILD_CMAKE_DEFINE='monoprop_SANITIZER=asan-ubsan;monoprop_ENABLE_MPI=OFF' \ uv sync --group workspace-test --all-extras --reinstall-package monoprop --no-cache -v @@ -307,12 +366,55 @@ kind-specific artifact; `compare` joins a timed/construction pair into one obser paring-construction measurement below is deliberate additional measurement work requiring full-plan approval, not instrumentation-only. -- [ ] Obtain owner approval of the full design policies, bounded new paring-construction measurement and target - allocation. Record baseline revision, CPU allocation, compiler/flags, MPI version, index width, OpenMP runtime, - allocator environment, imported extension path/hash, seed and complete workload configuration. +**Selected platform, not observed topology:** AWS's +[exact-size table](https://docs.aws.amazon.com/ec2/latest/instancetypes/co.html#co_hardware) specifies c8a.metal-24xl as +AMD EPYC 9R45, 96 physical cores/96 vCPUs, one thread/core and 192 GiB. Its +[C8a description](https://aws.amazon.com/ec2/instance-types/c8a/) explicitly says no SMT. These are published facts, not +measurements. Socket/NUMA counts remain unverified. Do not label oversubscription as SMT coverage or assume this machine +has production's usual eight domains. Once execution is authorized, archive the following read-only observations from +inside the actual allocation; these commands have not been run here: + +```bash +lscpu +lscpu -e=CPU,NODE,SOCKET,CORE,ONLINE +numactl --hardware +grep -E 'Cpus_allowed_list|Mems_allowed_list' /proc/self/status +``` + +Use these observations and per-rank placement evidence to select NUMA-contained geometries; a capacity-based 1x96 shape +is not automatically admissible. Missing diagnostic tools require another external observation method, not a new +library topology dependency. Record hardware SMT state, but configure only one worker per physical core. Do not add +SMT-sibling worker profiles; production-HPC and multi-node qualification remain separate and pending. + +- [ ] Obtain separate authorization for baseline build/calibration work. On c8a.metal-24xl, record sockets/NUMA domains, + physical cores, hardware SMT state, memory allocation and software stack before choosing geometries. Agree + representative workloads/sizes and single-instance routing/rank/thread shapes. Initial acceptance is the EC2 campaign, + including a two-ranks-per-domain case; all workers use distinct physical cores. Production-HPC performance and + multi-node qualification on a proper interconnect remain pending, not prerequisites for initial EC2 acceptance. + Production's common eight-domain layout is not a hard-coded EC2 topology. Historical profiles below are candidates, + not an approved campaign. + Confirm the bounded new paring-construction measurement in that campaign. Record baseline + revision, allocation, compiler/flags, MPI version, index width, OpenMP runtime, allocator environment, imported + extension path/hash, seed and complete workload configuration before collecting evidence. - [ ] Before changing engine code, separately build the baseline using R/M. Preserve its environment and binary. Apply - only the identical measurement-harness changes to both arms; record this overlay's diff/hash. Baseline - numerical/performance failures stop this task. + only the identical measurement-harness changes to both arms; record this overlay's diff/hash. Baseline numerical + failures stop this task. After the freeze, any required baseline failure blocks collection, not permission to resize + or remove the cell. +- [ ] Once authorized, run unscored baseline-only feasibility trials before freezing workload sizes. Use existing + baseline APIs/benchmark entrypoints; add no fourth driver mode or general calibration framework. Check elapsed + time and memory for construction, evaluation and numerical export/validation, including their temporary storage. + The approved pilot budget is 2 hours total, excluding builds/setup, and 15 minutes per trial including all those + phases. If insufficient, stop and report; do not silently extend either limit. Target at most 75% of observed usable + allocation RAM across all ranks, including construction and validation/export temporary storage. Record the RAM basis + and byte ceiling before trials; do not keep recomputing it from falling free memory. Exceeding this target requires + owner review before larger trials. This is operator-side sizing, not a library memory clamp or an OOM guarantee. + Record every attempted configuration, geometry/placement, baseline binary/overlay identity, elapsed time, memory + observations, outcome and reason for retaining/rejecting a size. Keep raw trials in a separate calibration archive, + without acceptance sample IDs. Capacity/time failures may inform pre-freeze sizing; numerical inconsistencies stop. + Do not run/consult candidate performance to select the inventory. Estimate formal campaign time/cost from the pilot, + then seek owner approval of sizes, coverage and campaign expenditure before the freeze and formal collection. Do not + reduce required samples/coverage to fit an unapproved budget. Collect fresh formal baseline samples after the freeze. + Never promote calibration runs, even with unchanged parameters. Approval of these limits is not execution permission. - [ ] Add renderer tests independently varying operation and outer exactness: true/false/absent and mixed flags. `bmf.py::build_bmf` publishes `peak-memory` from `memhwm`, so gate it on `memhwmexact`; `operation-memory` comes from `opmemdelta`, so gate it on `opmemexact`. `report.py::build_report` must label each corresponding metric correctly. @@ -324,31 +426,40 @@ instrumentation-only. rank-zero-only collective. Extend existing nested transient/fallback tests, not the reset implementation. - [ ] Migrate `_require_shape` in the identical shared overlay. Add explicit manifest/CLI runtime declaration `--runtime-shape=partitions|openmp` to pytest and driver; this is PROPOSED measurement configuration, not a backend - selector. Baseline declares partitions with `monoprop_PARTITIONS=T` (and `monoprop_NUM_THREADS=T`); candidate declares - openmp with `OMP_NUM_THREADS=T` and both library overrides unset. Require positive declared R/T and equal allocation. + selector. Baseline declares partitions with `monoprop_PARTITIONS=T` and `monoprop_NUM_THREADS=T`; candidate declares + openmp with `monoprop_NUM_THREADS=T` and removed partition settings unset. OMP_NUM_THREADS may also be T for the host + runtime. Accept the documented candidate fallback with monoprop_NUM_THREADS unset and OpenMP resolving T; verify it + separately in configuration tests. Require positive declared R/T and equal allocation. Record declared and observed shape separately; a renamed `--arm` label must not bypass validation. Before acceptance, diagnostic runs must verify actual baseline partition participation and candidate worker participation on representative large kernels, accounting for documented small/nested/runtime-limit paths. Reject missing or - contradictory evidence. Tests: missing baseline declaration, missing candidate OMP setting, candidate deprecated - override, wrong observed R/T, mislabeled arm, and valid equal-allocation declarations. RED: current multi-rank guard - rejects candidate configuration. + contradictory evidence. Tests: missing baseline declaration, invalid candidate thread override, leftover candidate + partition setting in the measurement environment, wrong observed R/T, mislabeled arm, and valid explicit/fallback + candidate declarations. This strict measurement preflight does not restore a removed production getenv reader. + RED: current multi-rank guard rejects candidate configuration. - [ ] Separate whole-construction measurement is intentional policy, not a repair for nesting. A fresh untimed worker, without pytest memory fixtures/session graphs, opens one `HighWaterMark`, constructs inputs/propagator/graph/callable as required, runs the operation and retains outputs until close. Record peak sum/max, floor, delta and exactness. Use existing model builders; no duplicate physics or general benchmark framework. -- [ ] Freeze an owner-approved campaign inventory BEFORE collecting baseline evidence. Its JSON schema is +- [ ] After unscored calibration, freeze an owner-approved campaign inventory BEFORE formal baseline collection. Its + JSON schema is `{"schema_version":1,"workloads_path":str,"workloads_sha256":str,"cells":[cell]}`. Each cell has unique slug `id`, `profile`, `node_id`, `operation`, `measurement_kind` (`pytest` or `driver`), `identity`, `config_digest` and `parameters_digest`. `identity` contains `ranks`, requested `threads`, `index_bits=32`, `cpu_allocation`, `compiler_flags`, resolved `config`, and `routing` (`mode`, derived `bits`, numeric `seed`). Expand the complete - required workload/profile × approved geometry/routing inventory from the matrix below; archive its digest with owner - approval. This file, not the observations supplied later, is the authoritative expected-cell list. A full cell missing + size-band-specific inventory below: small/medium profiles use the full geometry sweep, with the existing routing + coverage requirements; selected large profiles use only full-node one- and two-ranks-per-domain geometries. Do not + generate an all-sizes Cartesian product. Archive the inventory/digest with owner approval. This file, not later + observations, is the authoritative expected-cell list. A full cell missing from BOTH arms must fail completeness. Reject duplicate/extra cell IDs and mismatched identities. Do not regenerate or shrink the inventory to make observations pass. - [ ] `tools/rank-local-openmp-workloads.json` maps `profiles[profile][node_id]` to `operation`, `measurement_kind`, - `basis`, `picture`, complete `config`, deterministic `parameters`, and `pare_threshold`. `reference`/`tiny` contain - unpared four-family entries; `reference-pared`/`tiny-pared` contain threshold-1e-10 energy/gradient entries and both - driver construction identities. Record profile overrides explicitly; no ambiguity within one profile/node pair. Define + `basis`, `picture`, complete `config`, deterministic `parameters`, and `pare_threshold`. `tiny` is the small band and + calibrated `reference` is medium; each band's `-pared` variant shares its geometry policy. Preserve the existing + unpared four-family and threshold-1e-10 energy/gradient plus driver-construction coverage. Add `large`/`large-pared` + entries for the owner-selected large workloads, restricted to the two full-node decompositions. Profile names do not + freeze the historical reference sizes: record every calibrated override explicitly, with no ambiguity within a + profile/node pair. Define config/parameter digests as SHA-256 of UTF-8 `json.dumps(value, sort_keys=True, separators=(",", ":"), allow_nan=False)`. File digests are SHA-256 of exact file bytes. Resolve relative JSON paths against the containing file's directory. Do not put binary or placement-evidence @@ -433,19 +544,21 @@ evidence compares baseline/candidate of the identical operation, not an infeasib `functional="expectation_value_and_gradient_functional"`, `pare_threshold=1e-10`, picture, full random config and parameter digest. Use the proposed `observe`/`validate` interface: `--cell-id` resolves this node/profile from the frozen campaign, not a new framework or pytest family. Workload key is `(profile,node_id)`; a campaign cell also fixes - geometry/routing and gets a unique slug. Freeze tiny-pared and reference-pared entries before baseline collection. + geometry/routing and gets a unique slug. Freeze tiny-pared, reference-pared and selected large-pared entries before + formal baseline collection, respecting each band's geometry coverage. Example planned invocation after selecting the matching cell and binary-linked placement file (launcher/allocation supplied externally): ```bash -OMP_NUM_THREADS="$T" uv run --no-sync python tools/benchmark-rank-local-openmp.py observe \ +monoprop_NUM_THREADS="$T" OMP_NUM_THREADS="$T" \ +uv run --no-sync python tools/benchmark-rank-local-openmp.py observe \ --arm candidate --runtime-shape openmp --campaign "$CAMPAIGN" --cell-id "$CELL" \ --sample-id s01 --placement "$PLACEMENT" --output "$RESULTS" # Repeat in a separate fresh process with the same arguments plus --whole-process. # Run validate separately; omit --sample-id and --whole-process. ``` - Unset both library overrides before this candidate command; if launched at three ranks, explicitly export + Unset removed partition settings before this candidate command; if launched at three ranks, explicitly export `monoprop_ROUTING=splitmix` to every rank. Use `--runtime-shape partitions` and both legacy T declarations for baseline. In normal driver observe, build random inputs first, then open an outer `HighWaterMark` before propagator/graph construction (the driver analogue of pytest `record_memory`). Construct graph fully before barrier, @@ -528,9 +641,9 @@ def test_comparison_does_not_hide_one_regression(tmp_path): for arm, elapsed in (("baseline", 10.0), ("candidate", candidate_time)): shape = "partitions" if arm == "baseline" else "openmp" binary_hash = ("b" if arm == "baseline" else "c") * 64 - settings = {"OMP_NUM_THREADS": "2", "OMP_DYNAMIC": "FALSE"} + settings = {"monoprop_NUM_THREADS": "2", "OMP_NUM_THREADS": "2", "OMP_DYNAMIC": "FALSE"} if arm == "baseline": - settings.update(monoprop_PARTITIONS="2", monoprop_NUM_THREADS="2") + settings.update(monoprop_PARTITIONS="2") placement = {"schema_version": 1, "artifact_kind": "placement", "arm": arm, "cell_id": name, "binary_hash": binary_hash, "identity": identity, "runtime_shape": shape, @@ -577,16 +690,18 @@ def test_comparison_does_not_hide_one_regression(tmp_path): These two miniature inventory entries use empty-map energy fixtures, not real benchmark measurements. They follow the same schema, reference loading and provenance checks as real artifacts; no synthetic-mode bypass is allowed. The comparison test does not run physics, acquire placement, or establish actual hardware participation. Add tests for a -whole required cell deleted from both arms, missing samples, mismatched timed/construction pairs, reused measurement -files/run IDs, missing placement provenance (exit 2), each false/absent exactness flag, unequal +whole required cell deleted from both arms (including a large-profile cell), missing samples, mismatched +timed/construction pairs, reused measurement files/run IDs, raw calibration output offered as an acceptance artifact, +missing placement provenance (exit 2), each false/absent exactness flag, unequal shape/configuration/index width, and sum-versus-max peaks. For only an outer-peak regression or unknown outer exactness, report the diagnostic accurately while allowing the five-metric gate to pass. Operation/construction inexactness still fails. The comparator reports that five more observations are needed on first ratio failure; it launches no jobs. -- [ ] Freeze these required cells in `tools/rank-local-openmp-workloads.json`: fixed Hubbard and Pauli models and random +- [ ] Use the baseline-only trials and observed allocation to freeze owner-approved sizes for these coverage families in + `tools/rank-local-openmp-workloads.json`: fixed Hubbard and Pauli models and random Heisenberg/Schrödinger × build_graph/propagate/energy/gradient. Energy/gradient have separate unpared and threshold-1e-10 profiles; do not change node IDs to distinguish the profile. Also require random Heisenberg and - Schrödinger `pare_functional_construct` driver cells; no removed pytest node is revived. Reference configurations at - the inspected HEAD are: + Schrödinger `pare_functional_construct` driver cells; no removed pytest node is revived. The following are EXISTING + source-reference configurations, not the selected acceptance workloads: ```json { @@ -609,15 +724,21 @@ configurations in artifacts, not just a profile label. node per process. The existing fixed-model guard skips retained graphs with more than two repeated steps (`MAX_GRAPH_STEPS=2`), so the 29-step Hubbard reference is intentionally a capacity decision: use `monoprop_BENCH_ALLOW_BIG_GRAPH=1` only on adequately provisioned hardware, or obtain approval for a fixed lower - `--hubbard-trotter-steps` profile on both arms before collecting any baseline. A skipped cell is not a pass. Do not + `--hubbard-trotter-steps` profile on both arms before formal baseline collection. A skipped cell is not a pass. Do not silently omit graph-heavy cells or bypass the guard on an unverified allocation. -- [ ] Run baseline geometries `(R,T)=(1,1),(1,Tmax),(Rmax,1),(Rmax,Tmax)` on the approved allocation, deduplicating - equal cells. Old throughput arm uses `R` real MPI ranks × `T` partitions; record an additional old single-store serial - diagnostic. Never compare only against forced-serial old code. +- [ ] Select geometries from observed physical cores, not vCPU/SMT-sibling counts. For D allocated NUMA domains with + C selected physical cores each, the small/medium sweep is `(R,T)=(1,1),(1,C),(D,1),(D,C)`, deduplicated. R=1 uses + only one domain. Add `(2*D,floor(C/2))` with non-overlapping physical-core subsets when C>=2; record unused cores. + Apply existing routing coverage requirements too. Selected large profiles use only `(D,C)` and `(2*D,floor(C/2))` + from the full-node allocation, not the one-worker/partial-node sweep. Freeze the exact allowed cells per profile; + do not reclassify profiles or drop cells after a regression. If domains/allocation are uneven, agree explicit balanced + subsets before freezing rather than assume C=96/D. Each baseline/candidate pair has identical R/T/core allocation. + All these cells are single-instance; network-HPC qualification stays pending. Old throughput uses R ranks x T + partitions; retain an old single-store serial diagnostic but never compare only against forced-serial old code. - [ ] Reuse `benches/LADDER.md` L1–L4 labels/provenance and term-count fields as historical calibration, never acceptance evidence from old runs. Where a full-node single rank crosses NUMA domains, do not copy that historical - allocation: obtain an approved within-one-domain profile, equal on both arms, before baseline collection. Preserve + allocation: freeze a within-one-domain profile, identical on both arms, before formal baseline collection. Preserve historical nodes and fields while adding runtime-shape/observed-team metadata. Term counts must agree for the identical operation. - [ ] Use `just capture-baseline` / `just diff-baseline` and `tools/capture-baseline.py --compare` only as supplementary @@ -625,8 +746,9 @@ configurations in artifacts, not just a profile label. gradients/provenance; it cannot replace the ownership-aware global-map comparator or independent exact/finite-difference tests. -**Gate:** Benchmark tools' unit tests pass; baseline numerical suites pass; baseline observations and placement evidence -are archived. Historical benchmark keys unchanged. No parity claim yet. +**Gate:** Calibration archive and owner-approved size-band inventory/digest are retained; benchmark tools' unit tests +and baseline numerical suites pass. Fresh post-freeze baseline observations and placement evidence cover every required +cell; no calibration sample is counted. Historical benchmark keys remain unchanged. No parity claim yet. ### Task 2: Add the smallest OpenMP primitive and its build dependency @@ -728,7 +850,7 @@ clean consumer finds/links/runs with no manual flags or source includes. No engi - Modify: `cpp/monoprop/detail/monomial_propagator/MonomialPropagator.inl`, `cpp/monoprop/{Evolution,MPFunctions}.cpp`. - Modify: `cpp/monoprop/detail/evolution/{CosineRecompute,CosineRecomputeCallbacks}.h`, `cpp/monoprop/detail/evolution/layer_build/{Scan,Resolve,Engine,FusedApply}.h`. -- Modify: `cpp/monoprop/detail/mpi/MPICompat.cpp`, `cpp/monoprop/detail/EnvConfig.h`, `src/monoprop/bindings/binder.h`. +- Modify: `cpp/monoprop/detail/mpi/MPICompat.cpp`; the final environment/configuration cleanup is in Task 10. - Create/test: `cpp/tests/openmp_runtime_tests.cpp`, `cpp/tests/mpi_failure_driver.cpp`, `tests/test_openmp_config.py`; modify `cpp/tests/CMakeLists.txt` to exclude the failure-driver main from the unit-runner glob and build a separate executable. @@ -736,15 +858,18 @@ clean consumer finds/links/runs with no manual flags or source includes. No engi **PROPOSED interfaces:** ```cpp -// ThreadBudget.h; pure count validation, no topology or mutable global setting. +// ThreadBudget.h; include , and Options.h. namespace monoprop::detail::parallel { -// requested==0 selects runtime_default; otherwise use requested exactly. -// Throw std::invalid_argument if the selected value is not in [1, INT_MAX]. -auto resolve_thread_budget(size_t requested, int runtime_default) -> Options; +// nullopt selects runtime_default; a present value must be decimal digits and in [1, INT_MAX]. +// Throw std::invalid_argument on invalid input; no hardware query or global runtime mutation. +auto resolve_thread_budget(std::optional configured, int runtime_default) -> Options; +// Read monoprop_NUM_THREADS once and obtain omp_get_max_threads(); called only during construction. +auto capture_thread_budget() -> Options; } -// Constructor: append after existing child_factory argument: -// size_t num_threads = 0 -// Object member: parallel::Options parallel_; logically immutable after construction. +// No new public constructor argument or binding keyword. +// Private object members: parallel::Options parallel_; bool invalid_ = false; +// Private helpers: auto require_valid_() const -> void; auto invalidate_() noexcept -> void; +// MonomialPropagator.h: declare monoprop::InvalidPropagatorError deriving std::runtime_error. // EvalRequest: append detail::parallel::Options parallel = {}; ``` @@ -767,15 +892,17 @@ criteria; `Evolution.cpp:450–477` snapshots self endpoints, predivides, accumu pre-layer-unit sums. Keep record append/predivide/restore serial because duplicate indices are allowed. Do not replace this stable protocol with a division-only reverse pass. -- [ ] Add compile/integration tests for old low-level numerical call sites (their defaults stay serial; constructor - default selection changes in Task 10), two objects with budgets 1/3, copied budget, retained functional budget - capture, and rejection of simultaneous prototype multi-partition + threaded kernels. -- [ ] During coexistence, only explicit new `num_threads>0` enables the new runtime; require resolved legacy - `partitions==1`. Use explicit `partitions=1` plus final `num_threads` in C++ and low-level Python tests. Legacy - children always receive `{.threads=1}`. Do not reinterpret environment variables until Task 10. -- [ ] Expose the low-level binding constructor keyword `num_threads=0` after `partitions`. Regenerate by running R or M: - `src/monoprop/bindings/CMakeLists.txt` already invokes the generator at configure time and dispatch generation at - install time. For a manual regeneration of the default Release configuration only, its exact equivalent is: +- [ ] Add compile/integration tests for old low-level numerical call sites (their default Options stay serial), + prototype budgets 1/3 in separately launched processes, copied budget and retained-functional capture. Prove legacy + children stay serial even when their process environment requests multiple threads. +- [ ] During coexistence, initialize `parallel_` to `{.threads=1}`. Only when the EXISTING constructor has explicit + `partitions==1` AND `comm.kind==mpi::Comm::Kind::Mpi`, capture the environment budget before work. This also covers + non-MPI builds, whose ordinary Comm has Kind::Mpi. Shm/Hybrid children never activate OpenMP; multi-partition facades + keep their legacy path. Prototype tests use existing `partitions=1` and launch-time monoprop_NUM_THREADS; no new + constructor keyword, runtime switch or test-only production setter is introduced. Task 10 removes the old arguments + and makes capture unconditional for independent instances. +- [ ] Rebuild bindings through R/M without adding a thread keyword. Configuration already invokes binder generation + and installation invokes dispatch generation. For manual regeneration of the default Release configuration only: ```bash uv run --no-sync python tools/generate-binders.py --max-num-modes 1024 \ @@ -786,21 +913,25 @@ uv run --no-sync python tools/generate-binders.py --max-num-modes 1024 \ For a nondefault maximum use the actual `monoprop_MAX_NUM_MODES` cache value (current default 1024). Generated files stay in the build tree and are not committed. Never hand-edit generated bindings, invent a mode-width policy, or modify `tools/_binding_layout.py` for this task. -- [ ] Implement `resolve_thread_budget(requested,runtime_default)` without any hardware queries: when requested is - nonzero, check `requested <= static_cast(INT_MAX)` before conversion and keep that value exactly; when zero, - require `runtime_default > 0` and use it. Throw `std::invalid_argument` for unrepresentable/nonpositive selected - values; translate configuration errors at the public constructor boundary to `PropagatorConfigError`. Tests cover - `(0,1)->1`, `(0,3)->3`, `(8,2)->8`, `(1,0)->1`, `(0,0)`/`(0,-1)` rejection, INT_MAX acceptance and SIZE_MAX rejection - when larger than INT_MAX. These are pure count tests, never requests to create huge teams. In the prototype resolve - only explicitly enabled threaded objects; Task 10 enables the default path. Copy the resolved options unchanged. -- [ ] At cutover, when all explicit/compatibility overrides are absent, pass `omp_get_max_threads()` as runtime_default - once at construction. This captures the OpenMP calling context's default (normally `OMP_NUM_THREADS`), not actual team - size or available hardware. Do not parse OMP_NUM_THREADS independently or derive T from `omp_get_num_procs`, - `std::thread::hardware_concurrency`, affinity masks, OpenMP places, hwloc, sysfs or NUMA discovery. A requested count - may exceed the rank's allocation; preventing that is the user's responsibility. Small-work/nested serial fallback and - OpenMP limits may still reduce a region's actual team. -- [ ] In this prototype task test explicit budget validation/copy/callable capture and runtime-limited teams; - default-selection/precedence subprocess tests belong to Task 10, when that path is enabled. Verify library calls leave +- [ ] Implement `resolve_thread_budget(configured,runtime_default)` as a pure parser: nullopt selects a positive runtime + default; present input must be nonempty ASCII decimal digits, with the whole value in [1, INT_MAX]. Use checked + parsing before narrowing; reject signs, whitespace, lists, junk, zero and overflow. Leading zeros are harmless. Do not + retain the old arbitrary million-thread parser cap. Translate std::invalid_argument to PropagatorConfigError at the + constructor boundary, then apply the coherent distributed failure policy if peers can be stranded. Tests cover nullopt + with 1/3, nullopt with 0/-1 rejection, "8" with default 2 returning 8, and "1" with default 0 returning 1. + Reject empty, "0", "-1", "abc", "2,3", " 3" and "+3". Test INT_MAX text acceptance and overflow rejection without + creating teams. +- [ ] `capture_thread_budget()` reads getenv("monoprop_NUM_THREADS") once, distinguishes absent from present-empty, + and calls the pure resolver with omp_get_max_threads(). No function-static budget cache or operation-time reload. + During coexistence keep the old config parser solely for legacy partition selection; remove that cached budget path + in Task 10. Copy resolved options unchanged. The supported application fixes ranks/threads at launch; no live + configuration mechanism is provided or required. +- [ ] OpenMP supplies the fallback at any rank count, not an actual team size or hardware count. Do not independently + parse OMP_NUM_THREADS or derive T from omp_get_num_procs, hardware_concurrency, affinity masks, places, hwloc, sysfs + or NUMA. A request may exceed the allocation; resource suitability is the user's responsibility. Small-work/nested + internal-helper fallback and OpenMP limits may reduce actual teams without changing the captured budget. +- [ ] In this task test parser/copy/callable capture and runtime-limited teams through the one-store prototype; + final constructor surface and full default-selection subprocess tests belong to Task 10. Verify library calls leave OpenMP settings unchanged and issue no affinity-setting calls. A no-affinity-change test must disable OpenMP binding (`OMP_PROC_BIND=FALSE`), so it does not mistake the runtime's user-requested placement for library pinning. No hardware-topology query is needed by these tests; Task 11 placement checks are external benchmark-operator work. @@ -817,11 +948,29 @@ stay in the build tree and are not committed. Never hand-edit generated bindings unwinding can wait on peers that never posted. Normal successful paths still explicitly wait. Do not rely on a ticket destructor to coordinate distributed failure; do not add a per-kernel allreduce to agree on allocation failure. Wrap constructor/build/replay/evaluation phases that can strand another rank, including retained-callable entry points. - Single-rank failures preserve exception type; failed mutated objects must not be reused. + Single-rank failures preserve the original exception type. Add private `invalid_`, `invalidate_()` and + `require_valid_()` to enforce non-reuse. The latter throws InvalidPropagatorError with a stable recreate-the-object + diagnostic. On the caller, separate validation from the mutation phase using a local `mutation_started` flag; set it + before the first mutable cache/state/workspace change. A catch after that boundary calls invalidate_() before the + existing abort/rethrow path, including inside active-ticket lifetime. Successful calls never invalidate; pre-mutation + validation errors preserve validity. No atomic flag, rollback framework or new persistent workspace is needed. +- [ ] Call require_valid_() before state-consuming/mutating public operations and dependent functional invocations. + Retained closures capture the existing non-owning owner reference and check it before touching caches/snapshots; + they invalidate that owner on their own post-mutation failure. Do not change C++ callable lifetime or Python owner + retention. Validate a copy/clone source before any store/graph copy; cloning must not revive an invalid object. + Already independent copies remain valid. Destruction is always allowed; immutable configuration diagnostics need + not inspect invalid computational state. No production reset/setter clears invalid_. +- [ ] Add single-rank test-only failure injection at a real mutation boundary and during functional evaluation. Catch + the original exception; then assert build/replay/update/functional creation, a previously retained functional and + copy/clone reject the invalid owner. An independent earlier copy still evaluates. A validation-only failure leaves + the original usable. RED: reuse currently reaches state; GREEN: entry guards reject before further work. Injection + remains test-only and no production arbitrary-callback API is added. - [ ] Validate already initialized MPI with `MPI_Query_thread`, not only monoprop-owned initialization. Reject a level below `MPI_THREAD_SERIALIZED` with an actionable diagnostic before distributed work. MPI worker calls are forbidden; low-level numerical kernels may serialize under a nested OpenMP call, but distributed public operations from an - external OpenMP team are unsupported and must reject before communication. + external OpenMP team are unsupported and must reject before communication. The host contract is one controlling + caller per rank, no overlapping use of one propagator/dependent functionals and no overlapping host MPI calls. + Do not add a process-wide locking service or nested-parallel execution machinery. - [ ] Add a separate failure driver with command cases `worker-throw`, `before-exchange`, `active-ticket`, `insufficient-thread-level`. On rank zero inject the failure; peers execute the normal phase. Run from a subprocess with timeout 30s under two ranks. Expected outcome is prompt nonzero failure, **not** timeout or a successful exit. Do @@ -896,9 +1045,12 @@ speedup. - Extend: `cpp/tests/{fused_cos_sweep_tests,sparse_query_tests,pauli_build_layer_tests}.cpp`, `cpp/tests/{exact_upper_atol_rescue,evolution_detail_tests}.cpp`. -**PROPOSED interfaces:** Add `CutoffEvaluator::parallel_safe() const noexcept -> bool`, true only for existing typed -`LengthCutoff`/`SupportCutoff` targets. Unknown callbacks, including current basis-change closures, use serial -traversal. Do not require arbitrary existing user callbacks to become thread-safe. +**PROPOSED interfaces:** Add `CutoffEvaluator::parallel_safe() const noexcept -> bool`, initially true only for existing +typed `LengthCutoff`/`SupportCutoff` targets. Ordinary Python/C++ constructors take cutoff settings and basis-change +data, not arbitrary functions. Opaque predicates remain possible through protected C++ `cutoff_fn_` or low-level +helpers; keep their serial path. Current basis-change closures capture data by value, with no mutable capture found; +their initial serial path is conservative, not evidence of unsafety. Do not add a new callback API or claim a const +std::function alone proves safety. EXISTING `Scan.h::fused_find_and_collect` returns `FusedScanResult` and takes `SlotWindow window`, `my_rank`, router, precomputed `gen_shift`, then capture/scaling arguments. `Engine.h` derives the plan/window once from that @@ -927,9 +1079,11 @@ prepared-state cache. Tail masking uses the full index's final word, not each ra Cases must exercise dense and sparse pivots, odd Majorana correction even when selected fold columns are empty, Pauli J(G), no length cap, cap 0/positive, lower-atol equality, upper-atol rescue, cos=0 fallback, capture-values on/off, and no emitted queries. -- [ ] Add a stateful custom cutoff counting calls and checking caller thread identity. Expected behavior remains serial. - A basis-change regression test must retain its existing mathematical result, even though this first version leaves its - traversal serial. +- [ ] Extend the EXISTING stateless opaque C++ lambda fixture in `majorana_cutoff_tests.cpp:171–196`: evaluator + classification must be non-parallel-safe and the kernel's test-only work-range observations must show its serial + path. Do not invent a stateful Python callback or a callback-heavy performance cell for a nonexistent constructor + surface. Preserve actual basis-change results; this first version leaves that traversal serial. Audit real cosine + callbacks/parity-cache preparation separately. Serial fallback is not an exemption from approved campaign gates. - [ ] On caller: validate arguments, prepare algebra/generator columns, obtain router once, initialize lazy index/parity, snapshot row/word count, check skip/tail conditions. Partition words into at most `min(options.threads,ceil(words/1024))` contiguous nonempty ranges. Range IDs increase with source word index and do @@ -1127,8 +1281,11 @@ auto reserve_index_for_size(size_t final_size) -> void; row-storage multiplication, vector max_size, table load-factor sizing and next-power-of-two rounding before allocations. Geometric spare capacity cannot wrap. Update diagnostics from per-partition to per-rank store at cutover. - [ ] Use these checks before incoming predicted IDs and deferred insertion. Audit existing MPI checked counts without - changing transport or count width. A rank-wide store can exceed a ceiling that each old partition fit; this is a - **capacity compatibility blocker**, not permission to widen only the candidate or introduce hidden shards. + changing transport or count width. The owner accepts that consolidating old partitions reduces aggregate per-rank + addressability. A required campaign cell above the new ceiling is still a **capacity acceptance blocker**, not + permission to widen only the candidate, change ranks or introduce hidden shards. Do not infer a universal RAM limit + from the illustrative 512-GiB/eight-domain machine. C8a has 192 GiB advertised RAM; per-rank budgets still + require the observed allocation and actual memory demands, not a hard-coded equal split. - [ ] Profile row fill. If it is not significant, leave row initialization and publication serial and mark the optional steps below skipped with evidence; the required capacity checks still ship. - [ ] If justified, add tests proving simultaneous new inline writes do not modify old overflow entries. @@ -1212,7 +1369,7 @@ current algorithmic requirements. **PROPOSED interfaces:** Keep `inner_product(v,w)` and `EvalState::dot(op)` unchanged. Add explicitly named `inner_product_threaded(v,w,parallel::Options)` and `EvalState::dot_threaded(op,parallel::Options) const`. They have documented fixed-block association, not historical left-to-right association. `ev`/`ev_and_grad` use the new helpers -through `EvalRequest.parallel` after approval of this numerical policy. +through `EvalRequest.parallel` under the approved numerical policy; implementation remains separately gated. - [ ] Add cancellation-heavy finite vectors, zero/signed-zero, sparse gapped rows, very short arrays, >4096 rows, non-finite inputs, and repeated parameter indices. Check old serial dot tests unchanged; check new helper results @@ -1243,8 +1400,8 @@ through `EvalRequest.parallel` after approval of this numerical policy. equal; don't widen tolerances to pass them. **Gate:** Existing serial API guarantees remain intact; new numerical association is documented and deterministic by -team size. If near-cutoff retained terms change outside existing intended equivalence, stop for owner decision, not a -tolerance adjustment. +team size. Global retained term keys must agree, including near cutoffs and stored zeros; matching energies/gradients +alone is insufficient. A key-set change blocks the gate. Stop and report, never widen tolerances or change truncation. ### Task 10: Cut over configuration and API semantics to one store @@ -1254,44 +1411,49 @@ tolerance adjustment. - Modify: `src/monoprop/bindings/binder.h`, `src/monoprop/monomial_propagator.py` docstrings. - Create/extend: `cpp/tests/openmp_equivalence_tests.cpp`, `tests/test_openmp_config.py`, `cpp/tests/{env_config_tests,routing_tests,mpi_distributed_layer_equivalence}.cpp`, - `cpp/tests/{mpi_fresh_insert_equivalence,unit_tests}.cpp`, `tests/test_monoprop_smoke.py`. + `cpp/tests/{mpi_fresh_insert_equivalence,unit_tests,ctor_validation_tests,hybrid_comm_tests}.cpp`, + `cpp/tests/link_export_probe/link_export_probe.cpp`, `tests/{test_monoprop_smoke,test_parameter_validation}.py`. +- Migrate now: `cpp/tests/{partition_equivalence_tests,partition_group_clone_tests}.cpp`; preserve useful coverage in + the OpenMP equivalence suite and `cpp/tests/{simulator_copy_tests,update_initial_operator}.cpp` before retiring + obsolete facade/factory assertions. Update `cpp/tests/README.md`; do not defer broken constructor users to Task 12. - Modify: `AGENTS.md`, `README.md`, `docs/content/docs/features/parallelism.mdx`, `docs/content/docs/openmp-migration.mdx` (new). -**PROPOSED interfaces/policy:** Implement the design's deprecation aliases, not the alternative rejection policy -suggested by some investigations. No second permanent backend setting. +**PROPOSED interfaces/policy:** Implement immediate removal, as approved in the design interview. No aliases, +deprecation window, public thread-count constructor argument or permanent backend selector. -- [ ] In fresh subprocesses test the precedence table: +- [ ] In fresh subprocesses test the final environment-only policy: | Input | Result | | --- | --- | -| explicit `num_threads=N`, no positive partitions | N exactly, provided 1 <= N <= INT_MAX; no hardware clamp | -| positive `partitions` and positive `num_threads` | `PropagatorConfigError` | -| explicit positive `partitions=N` alone | warned deprecated thread budget N; reject if above INT_MAX | -| `monoprop_PARTITIONS=off` | warned, budget 1 | -| `monoprop_PARTITIONS=N` | warned, N exactly; reject a positive numeric request above INT_MAX | -| `monoprop_PARTITIONS=auto` or malformed | warned, fall through to valid monoprop_NUM_THREADS, otherwise OpenMP default | -| no partition override, valid `monoprop_NUM_THREADS=N` | N exactly (existing parser/range retained) | -| no library override, any MPI rank count | capture omp_get_max_threads() at construction | -| OMP_NUM_THREADS unset and no library override | capture runtime default; no hardware-suitability promise | -| non-null child factory | `PropagatorConfigError`, factory never invoked | - -Public `num_threads=0` is the no-explicit-request sentinel. Positive explicit num_threads ignores environment aliases; -warn only for the legacy setting actually used. Warn once per process per deprecated partitions spelling, using an -existing diagnostic style or a small `std::once_flag`, not a Python-only warning that C++ users miss. Invalid -monoprop_NUM_THREADS keeps current parser behavior. Reject oversized positive partition requests before narrowing; do -not reinterpret them as malformed fallback input. Normally recommend OMP_NUM_THREADS=T with library overrides unset. It -supplies the OpenMP default but does not override an explicit library budget. Runtime limits such as OMP_THREAD_LIMIT -and dynamic-team adjustment may reduce the actual team. There is no topology-based fallback or special multi-rank -default of one thread. -- [ ] Add default-selection subprocess tests with library overrides unset: OMP_NUM_THREADS=1/3, list-valued - OMP_NUM_THREADS=2,3 (capture the runtime's outer-level default), and OMP_NUM_THREADS unset (compare the captured count - with omp_get_max_threads, not an assumed core count). Repeat at one and multiple MPI ranks; prove there is no library - multi-rank serial fallback. Test explicit/library-environment precedence and OMP_THREAD_LIMIT-reduced actual teams - separately from stored requests. Copies and retained closures preserve their captured budget; caller-driven - runtime-setting changes affect only subsequently constructed objects. Update prototype constructors/helpers to pass - `partitions=0` with an explicit num_threads, since the final API rejects both positive arguments. -- [ ] Obtain approval for changed historical splitmix physical ownership: `floor((hash % (R*S))/S)` becomes `hash % R`. +| `monoprop_NUM_THREADS=N`, valid decimal 1..INT_MAX | capture N exactly; no hardware clamp | +| valid override and a different OpenMP default | monoprop_NUM_THREADS wins | +| present empty/zero/negative/malformed/overflowing override | actionable PropagatorConfigError; no fallback | +| override unset, any MPI rank count | capture omp_get_max_threads() at construction | +| both thread environment variables unset | capture runtime default; no hardware-suitability promise | +| stale `monoprop_PARTITIONS` in application environment | no production reader, alias or effect | +| C++ partition/factory constructor arguments | removed; old source must migrate | +| low-level Python `partitions` or `num_threads` keyword | rejected as an unknown keyword | + +Only absent monoprop_NUM_THREADS selects the OpenMP default; zero is not a public sentinel. Use Task 3's strict parser +rather than the old cached permissive one. Remove Settings::num_threads and its legacy accessor/parser once their +partition consumers are gone; remove resolve_partition_count_ declaration/definition in this task too, so its old parser +references do not survive as ill-formed dead template code. Retain independent routing settings/cache. Scan readers for +remaining partition-only knobs and remove them. At the inspected source the only partition-specific runtime variable is +monoprop_PARTITIONS; monoprop_ROUTING and monoprop_ROUTE_SEED remain MPI configuration. Do not add a removed-variable +reader for warning or rejection. Candidate benchmark preflight may reject a stale launch environment independently. + +The supported application fixes ranks/threads at launch. Capture once per constructor, never reload during operations or +add a setter. Copies/functionals preserve their budget. OpenMP limits or small-work paths may use fewer actual workers; +this is not a budget change. There is no topology-based fallback or special multi-rank default of one thread. +- [ ] Test defaults in fresh processes with monoprop_NUM_THREADS unset: OMP_NUM_THREADS=1/3, list-valued 2,3 (use the + runtime's outer-level default), and unset OMP_NUM_THREADS (compare against omp_get_max_threads, not assumed cores). + Repeat at one and multiple ranks. Test override precedence, all invalid-input classes from Task 3 and limited teams. + Invalid settings on one rank must fail coherently rather than strand peers. Add a bounded white-box test that changes + the source setting after construction solely to prove existing objects/copies/closures do not reread it; this is not + a supported hot-configuration API. Full-budget matrix tests use separate launches, not mid-calculation changes. + Remove prototype partitions=1 arguments; public tests use environment configuration, internal kernels use Options. +- [ ] Apply the approved splitmix physical ownership change: `floor((hash % (R*S))/S)` becomes `hash % R`. Linear rank mapping remains unchanged; S=1 removes only the partition hash. Preserve boolean mode/seed, `linear_requested()`, `linear_bits_for(R)`, `make_router(R,1)`, generator `rank_shift` and `dest_from_shift` peer selection. No virtual shards. Exports/load/reduction order can change under splitmix; validate global term/value maps @@ -1303,33 +1465,43 @@ default of one thread. default tests distinct from splitmix matrix runs; never expect an automatic fallback. Assert unique global ownership, generator-peer agreement, zero-payload participation, fixed-candidate ordering across budgets and global numerical equivalence. Do not require old/new rank-local maps equal for splitmix. -- [ ] Make constructor create exactly one rank-local `MPOperator` and graph, with `comm_` representing only real MPI - ranks. Remove facade branches from every public method and internal dispatch, but leave now-unused transport files - until Task 12. Retain `PartitionChildFactory` declaration/argument temporarily to produce actionable failure, and keep - `MultiPartitionUnsupported` type temporarily for source migration even though single-store accessors no longer throw - it. +- [ ] Migrate every constructor/helper/binding test using removed partition arguments in this same task. Use a source + search over cpp/tests, tests and src; include the installed export probe, constructor-validation cases and Python + parameter-validation/smoke tests. Old calls failing to compile or obsolete keywords still being accepted are + meaningful RED conditions. Move useful numerical/copy/update/empty-rank coverage before retiring facade/factory-only + cases; preserve ordinary-MPI cases currently housed in hybrid suites. Record each retired case's replacement in the + Task 12 ledger. Remove unit_tests.cpp's monoprop_PARTITIONS override and use the serial launch budget instead. + R/M must compile and pass here, not become green only after Task 12 deletes test files. +- [ ] Each independent propagator constructor creates one rank-local MPOperator and primary graph; comm_ represents + only real MPI ranks. Remove facade branches, partitions/child_factory arguments, PartitionChildFactory and + partition-specific error types such as MultiPartitionUnsupported. No diagnostic compatibility shims. Leave now-unused + transport implementation files until Task 12. Independently copied propagators keep their own stores; retained + coefficient/state snapshots, shared immutable graph cores and pared graphs remain legitimate existing objects. - [ ] Preserve rank-local size/bytes accounting (do not allreduce twice), core term handling, copy independence, immutable graph-core sharing, virtual destructor, virtual `clone_`, overridden virtual `update_initial_operator`, and protected nonvirtual `apply_initial_operator_`. Do not add override to or make that helper virtual. Copy budget and - ordinary MPI communicator. Raw accessors return the one store/graph at all budgets. + ordinary MPI communicator. Raw accessors expose that instance's rank-local store/primary graph at every budget. + Preserve Task 3 invalidation checks: invalid sources cannot be cloned; an already independent copy stays usable. - [ ] Preserve `MonomialPropagator.inl:126–151,197–227` streamed paired-basis construction and local reservation: cardinality rejection remains before worker startup and uses real ranks at S=1, never T. Do not materialize `generate_paired_op` globally per rank. Migrate `partition_equivalence_tests.cpp` initial-basis invariance alongside enumeration/cardinality tests from `majorana_cutoff_tests.cpp` and `ctor_validation_tests.cpp`. -- [ ] Migrate numerical tests from partition counts to explicit budgets 1/2/3/4; distinguish stored requests from actual - team sizes under OpenMP limits. Library budget selection never clamps to affinity. Run parallel gates on a user/CI +- [ ] Migrate numerical tests from partition counts to launch-time budgets 1/2/3/4 in fresh processes; lower-level + kernel tests may pass explicit internal Options. Distinguish stored requests from actual team sizes under OpenMP + limits. Library budget selection never clamps to affinity. Run parallel gates on a user/CI allocation supporting the requested teams and report limited-team skips honestly; pure budget tests need no extra - CPUs. The default white-box harness selects a serial library budget (not CPU pinning); explicit tests override it. - Test default OpenMP selection in fresh processes without that harness override. -- [ ] Test public Majorana/Pauli constructors unchanged. Keep binding-only `partitions`/`num_threads`; do not add - unrelated public constructor parameters. Test copy, operator update, retained callbacks after growth, + CPUs. The default white-box harness selects a serial library budget (not CPU pinning); dedicated launches override + it before construction. Test OpenMP-default selection in fresh processes with that library override absent. +- [ ] Test public Majorana/Pauli mathematical constructors unchanged. Remove binding-only partitions and do not add + num_threads or any other public thread-count argument. Test copy, operator update, retained callbacks after growth, cutoff/basis-change update, paring, Schrödinger state extension and independent repeated evaluation. - [ ] Regenerate bindings via the configured generator. Preserve 32-mode template dispatch. Document low-level/source/ABI breaks and unspecified coefficient-order changes; compare decoded term/value mappings rather than sorted coefficients alone. -**Gate:** R/M API/configuration tests pass; `R ranks × T threads` owns exactly R stores. Full compatibility ledger is -approved before deleting extension hooks. +**Gate:** R/M API/configuration tests pass; one independent propagator across R ranks owns R packed stores, regardless +of T. Copies and retained snapshots/graphs obey the qualified ownership rule. Removed configuration/constructor surfaces +stay absent; invalid objects cannot be reused. Record the approved immediate compatibility breaks in the release ledger. ### Task 11: Measure parity and apply only bounded optimizations @@ -1344,35 +1516,44 @@ functions or move benchmark files. (on Linux, `lscpu -e=CPU,CORE,SOCKET,NODE` and `/proc//task/*/status`). Observe actual active-worker masks/team size, not merely omp_get_max_threads; if a runtime cannot supply that evidence, use external sampling or test-only observation rather than adding a production topology API. Preserve old single-CPU-thread fields as historical data: - OpenMP core places may include SMT siblings. No required hwloc dependency for the library or measurement helper. + OpenMP core places may contain both SMT siblings, but at most one worker may occupy each physical core in the + recommended/acceptance configuration. A wider affinity mask is not evidence of extra worker capacity. Record distinct + physical-core ownership from verified sibling topology across active ranks/workers, not only disjoint logical CPU IDs. + No required hwloc dependency for the library or measurement helper. - [ ] The benchmark operator, not monoprop construction, verifies that rank masks and actual worker places stay in one - NUMA domain and within the reserved allocation, with no unintended overlap between ranks. Do not require one logical - CPU per worker mask. Record idle/small-kernel serial fallback versus representative large-kernel actual team size - separately. An unverifiable placement blocks that benchmark result, not normal library construction. -- [ ] Compare old partitions vs final new threads at identical R,T and routing settings, using Task 1 frozen profiles. + NUMA domain and within the reserved allocation, with no unintended overlap between ranks. Advise at least one MPI + process per allocated NUMA domain, with disjoint CPU subsets when several processes share a domain. Use actual OS/site + topology, not a constant four domains/socket; partial-node runs cover only their allocated domains. + Do not require one logical CPU per worker mask. Record idle/small-kernel serial fallback versus representative + large-kernel actual team size separately. Unverifiable placement blocks benchmark evidence, not library construction. +- [ ] Compare old partitions vs final new threads at identical R,T and routing settings, using Task 1's frozen + size-band-specific inventory. Small/medium profiles cover the full sweep; large profiles only the two full-node + decompositions. Do not add/remove required cells or reinterpret profile sizes after seeing candidate performance. Freeze the production routing mode in that matrix; additionally require the random reference profile under explicit splitmix at R=3 and R=4, plus default linear at R=4 so ownership-dependent imbalance is measured, not only unit-tested. Run five alternating **fresh-process** observations per arm/cell; external repetition only, `--bench-rounds=1`. Include first-call/cold cache and separately measured repeated warm calls without overlapping object construction. -Example candidate Open MPI invocation after selecting actual values for `R` (= P), `T`, `LABEL`, and absolute `RESULTS`. +Example candidate Open MPI invocation after selecting R (= P), K (ranks/domain), T, LABEL and absolute RESULTS. +Use K=1 or K=2 for the agreed decomposition profiles. Choose a compatible placement for routing-specific R=3/R=4 cells; +do not force ppr:1:numa onto an allocation with too few NUMA domains. All placements require observed disjoint cores. This demonstrates one raw pytest run, not a complete parity observation. The Task 1 driver uses the same launcher contract, unique per-sample raw directory, frozen campaign and paired construction/validation artifacts for acceptance. -Unset compatibility overrides so this exercises normal OMP_NUM_THREADS configuration: +Unset removed partition configuration and use the supported library override. The OpenMP fallback is tested separately: ```bash -unset monoprop_PARTITIONS monoprop_NUM_THREADS +unset monoprop_PARTITIONS # This example chooses splitmix explicitly, including if R=3; use linear only in a separate R=1/2/4 cell. export monoprop_ROUTING=splitmix mkdir -p "$RESULTS" monoprop_BENCH_LABEL="$LABEL" monoprop_BENCH_RESULTS="$RESULTS" \ -OMP_NUM_THREADS="$T" OMP_DYNAMIC=FALSE \ +monoprop_NUM_THREADS="$T" OMP_NUM_THREADS="$T" OMP_DYNAMIC=FALSE \ OMP_PLACES=cores OMP_PROC_BIND=close \ -uv run --no-sync mpiexec -n "$R" --map-by "ppr:1:numa:PE=$T" \ - --bind-to core --report-bindings \ +uv run --no-sync mpiexec -n "$R" --map-by "ppr:${K}:numa:PE=${T}" \ + --bind-to core --nooversubscribe --report-bindings \ -x monoprop_BENCH_LABEL -x monoprop_BENCH_RESULTS -x monoprop_ROUTING \ - -x OMP_NUM_THREADS -x OMP_DYNAMIC -x OMP_PLACES -x OMP_PROC_BIND \ + -x monoprop_NUM_THREADS -x OMP_NUM_THREADS -x OMP_DYNAMIC -x OMP_PLACES -x OMP_PROC_BIND \ python -m pytest 'benches/bench_models.py::test_model_propagate[pauli]' \ -o filterwarnings=default --bench-rounds=1 --runtime-shape=openmp \ --benchmark-json="$RESULTS/time-$LABEL.json" @@ -1383,14 +1564,15 @@ uv run --no-sync monoprop-bench-bmf "$RESULTS" "$LABEL" This mapping syntax is Open MPI-specific. Verify scheduler/launcher support; do not escape the allocation. For the **baseline only**, set `monoprop_PARTITIONS="$T"` and `monoprop_NUM_THREADS="$T"`, export both to ranks with `-x monoprop_PARTITIONS -x monoprop_NUM_THREADS`, and replace `--runtime-shape=openmp` with -`--runtime-shape=partitions`. Keep OMP settings, routing, allocation and shared overlay identical. The candidate must -leave both library overrides unset. The shared preflight is a required Task 1 change: the existing guard would reject -this candidate above one rank. Verify declared versus observed shape with binary-linked diagnostics; labels alone cannot -certify shape. After M build assert monoprop.has_mpi before invoking any launcher. For Slurm document the corresponding -user contract (`srun --ntasks=P --cpus-per-task=T --cpu-bind=cores` with OMP_NUM_THREADS=T and suitable -OMP_PLACES/OMP_PROC_BIND), while making clear that site CPU/SMT allocation rules and single-NUMA containment must be -verified. Launcher binding must leave each rank enough CPUs for its workers; merely setting OMP_NUM_THREADS does not -enlarge a one-core rank mask. +`--runtime-shape=partitions`. Keep OMP settings, routing, allocation and shared overlay identical. The candidate uses +monoprop_NUM_THREADS but never monoprop_PARTITIONS. The shared preflight is a required Task 1 change: the existing +guard would reject this candidate above one rank. Verify declared versus observed shape with binary-linked diagnostics; +labels alone cannot certify shape. After M build assert monoprop.has_mpi before invoking any launcher. Follow the +[physical-core launch recipes](../specs/2026-09-18-rank-local-openmp-design.md#physical-core-launch-recipes) for both +Open MPI and Slurm. Document srun's --threads-per-core=1, compatible core binding and OpenMP budgets/places, and the +required NUMA verification. Do not combine conflicting --hint options or claim that socket/block distribution implies +NUMA containment. Explicit-mask fallbacks require eligible full-CPU step allocation, not merely an exclusive job. +OMP_PLACES=cores is not a worker-count cap, and merely setting OMP_NUM_THREADS does not enlarge a one-core rank mask. - [ ] Report runtime, exact operation peak/floor/delta, outer peak sum/max, separate whole-construction peak, persistent operator/graph bytes, query/snapshot/TLS capacities, term/graph sizes, and numerical checks. Rank peak sums are upper bounds on aggregate footprint, not node provisioning estimates; retain maximum rank peak as another statistic. Reject @@ -1398,7 +1580,8 @@ enlarge a one-core rank mask. - [ ] Compare medians per cell using exactly the five Task 1 gates: runtime, operation peak sum/max and construction peak sum/max. Outer peak ratios are diagnostics only. All five required ratios must be <=1.00. If any fail, collect five additional observations per arm for those cells and compare all ten. Do not replace the strict gate with a - geometric mean, a made-up 5% tolerance, or a changed problem size. + geometric mean, a made-up 5% tolerance, or a changed problem size. A noisy failure means parity was not demonstrated, + not proof of architectural slowdown; it still blocks acceptance. Do not drop the cell or quietly waive it. - [ ] Profile phases before optimization: bitmap traversal, query merge, decode/probe, missing-ID assignment, row fill, hash publication/rehash, inverted-index maintenance, graph packing, MPI pack/wait, cosine, pairs, dot. Record miss fraction, overflow fraction and bytes, not just aggregate speed. @@ -1423,7 +1606,7 @@ enlarge a one-core rank mask. redistribution, full per-thread stores or a persistent key map require a new design; stop rather than improvise. **Gate:** All required performance/memory cells pass with attached raw observations and numerical/placement evidence. -Missing hardware, capacity failure, persistent serial-publication bottleneck or repeated ratio>1.00 blocks +Missing required EC2 evidence, capacity failure, persistent serial-publication bottleneck or repeated ratio>1.00 blocks removal/acceptance. Report it honestly; the design is not a promise of achievable parity. ### Task 12: Remove legacy partition runtime, topology machinery and direct hwloc dependency @@ -1451,11 +1634,12 @@ removal/acceptance. Report it honestly; the design is not a promise of achievabl Kind/Shm/Hybrid and associated pointers/constructors/dispatch. Keep non-MPI behavior, ordinary MPI, PeerPlan routing and existing typed exchanges. Do not collapse it to a raw alias if that changes overload behavior unnecessarily. -- [ ] Start only after Task 11 parity and explicit owner approval of the C++ extension-interface break. Confirm no - runtime path constructs old groups before deleting definitions. -- [ ] Remove `partition_group_`, friendship, fan-out implementations, `resolve_partition_count_`, `is_partition_facade`, - and all protected partition-only helpers. Keep `clone_` and virtual lifecycle/operator-update hooks. No compatibility - facade that invokes old callbacks once with silently different semantics. +- [ ] Start only after Task 11 parity under separately authorized implementation. The owner approved the immediate + partition-related C++ extension/ABI break; confirm no runtime path constructs old groups before deleting definitions. +- [ ] Remove any remaining `partition_group_`, friendship and unused fan-out implementation. Confirm Task 10 already + removed resolve_partition_count_, public partition/factory arguments and facade-only test users; remove remaining + is_partition_facade/protected partition-only helpers. Keep `clone_` and virtual lifecycle/operator-update hooks. No + compatibility facade that invokes old callbacks once with silently different semantics. - [ ] Remove non-MPI-thread transport branches from utilities/tickets. Production router geometry is `(real_ranks,1)`. Preserve EXISTING boolean linear/splitmix default/rejection contract and generator shift/peer rules; no routing-default repair is needed. Keep ordinary MPI sparse query routing and sparse on-wire replay, sender order, @@ -1480,20 +1664,21 @@ existing typed exchanges. Do not collapse it to a raw alias if that changes over required OpenMP across flake package/MPI package/devshell checks and the deploy wheel matrix. Remove documentation claiming hwloc/pkg-config is a direct monoprop prerequisite or that the library pins threads. Do not uninstall or strip a library needed by the selected MPI/OpenMP runtime, or remove another dependency's legitimate pkg-config use. -- [ ] Migrate tests before deleting obsolete files: +- [ ] Complete this migration ledger before deleting obsolete files. Constructor/facade-dependent cases must already + have moved in Task 10; retain transport-only coverage until this task removes its implementation. | Old coverage | Replacement/retirement | | --- | --- | | `partition_equivalence_tests.cpp`: energies, gradients, propagation, determinism, copies, Pauli, setters, partial contraction | `openmp_equivalence_tests.cpp`, budgets 1/2/3/4, both pictures, decoded term/value equivalence | | facade raw-accessor errors | single-store raw-accessor consistency at every budget | -| factory throw/child construction | deprecated factory rejected before invocation; no worker terminate | +| factory throw/child construction | factory API removed; Task 3 validates joined worker exceptions without terminate | | `partition_group_clone_tests.cpp`: derived children | retire child-existence assertions; retain derived clone/update-initial-operator independence in `simulator_copy_tests.cpp` and `update_initial_operator.cpp` | | `shm_comm_tests.cpp` / `hybrid_comm_tests.cpp`: private barrier algorithms | retire only after ownership/source-order/empty-payload/count/error assertions map to MPI/OpenMP tests | | `hybrid_comm_sparse_plan_on_the_plain_mpi_path` | move unchanged semantics to ordinary MPI tests; unknown and known receive counts | | plain-MPI half of `hybrid_comm_known_recv_counts_are_masked_through_the_plan` | preserve masked non-peer counts and sparse peer payload assertions | | `flat_exchange_tests.cpp` replay dense/pairwise/dense-layout/empty tests | retain and run under ordinary MPI after deletion | | hybrid MPI energy and size case | retain as MPI×OpenMP matrix case in `mpi_distributed_layer_equivalence.cpp` | -| poisoned waiter release | Task 3 MPI failure-driver and local joined-exception tests | +| poisoned waiter release | Task 3 MPI failure-driver, joined exceptions and enforced invalid-owner/functionals tests | | `cpu_topology_tests.cpp`: topology enumeration, mask classification, pinning/L3 dealing | retire these library-owned policy tests; Task 3/10 count/precedence/no-affinity-mutation tests cover the retained library contract, and Task 11 verifies placement externally | Record individual retired case names and replacement names in the implementation acceptance artifact; do not merely @@ -1508,8 +1693,9 @@ rg -n -i 'hwloc|CpuTopology|PkgConfig::HWLOC|libhwloc-dev' \ README.md AGENTS.md docs/content/docs ``` -First command should have no production matches except deliberately retained migration diagnostics/type names where -applicable; second must resolve to approved deprecated API/docs, not active ownership/fan-out. The hwloc audit must find +First command should have no production legacy-runtime matches; second must resolve only to mathematical/MPI +partition terminology or historical migration notes, never a retained constructor alias, getenv reader or fan-out. +The hwloc audit must find no direct discovery/link/install requirement or CpuTopology implementation; migration notes about removal and third-party/transitive dependencies are allowed. Inspect each match rather than deleting mathematical/MPI partition terminology indiscriminately. @@ -1521,8 +1707,8 @@ terminology indiscriminately. **Gate:** No in-process communication runtime, library-owned worker threads, custom barriers, topology/affinity subsystem, direct hwloc dependency or permanent legacy backend remain. Build/install/import a non-MPI candidate in an environment without hwloc development headers/pkg-config metadata; inspect direct linkage instead of requiring the -absence of a runtime's own transitive hwloc dependency. One store per rank is verified by construction, tests and memory -accounting, not merely a flag. +absence of a runtime's own transitive hwloc dependency. One packed store per independent propagator per rank is verified +by construction, copy/functional tests and memory accounting, not merely a flag. ### Task 13: Final packaging, documentation and independent acceptance @@ -1562,17 +1748,24 @@ nix develop --command bash -euo pipefail -c ' - [ ] Run `just test-find-package` against the installed package, extending its existing shared probe source rather than creating a duplicate harness; compile a template kernel using OpenMP and both Majorana/Pauli APIs. The consumer must not inherit source-tree include paths or manual linker flags. -- [ ] Document one store/rank and the resource-ownership contract explicitly: users choose P and T for their resources - and bind/pin ranks/threads with OMP_PLACES+OMP_PROC_BIND, srun --cpu-bind and/or mpiexec options. Normally use - OMP_NUM_THREADS=T; explicit library overrides have the documented precedence. Describe construction-time capture of - the OpenMP default at any rank count, no library hardware clamp/topology discovery/NUMA validation, possible OpenMP - team-size limits, and the one-NUMA-per-rank allocation assumption. Show matching allocation/rank-binding/thread-place - examples, including the single-core-rank-mask pitfall. Retain MPI serialized policy, no concurrent object/communicator - calls, callable lifetime, failed-operation non-reuse, capacity and numerical contracts. State that external placement - verification for benchmarks does not add a required topology library. -- [ ] State exact breaks: protected partition helpers removed, factory rejected, old automatic physical-core - selection/multi-rank serial default and library pinning removed in favor of OpenMP/user configuration, direct hwloc - prerequisite removed, raw coefficient order can differ, ABI requires rebuild. Explicit thread requests are not +- [ ] Document one packed store per independent propagator per rank, preserving copies, existing functional snapshots + and pared graphs. Users choose P/T at launch and bind with OMP_PLACES+OMP_PROC_BIND, srun --cpu-bind and/or mpiexec. + Normally use monoprop_NUM_THREADS=T; only unset selects the OpenMP runtime default. Present-invalid is an error. + Capture once during construction, with no reload/setter. Describe no hardware clamp/topology discovery/NUMA + validation, possible smaller OpenMP teams, and the within-one-domain allocation assumption. Show matching allocation, + rank-binding and thread-place examples, including the one-core-rank-mask pitfall. Document one controlling host caller + outside host OpenMP teams, no overlapping use of dependent state or host MPI calls, serialized MPI, callable lifetime, + enforced failed-object invalidation, accepted capacity reduction and numerical contracts. Placement verification for + benchmarks does not add a topology dependency. Advise at least one MPI process per allocated NUMA domain and identify + actual topology externally; production's common four-domains/socket layout is not universal. Distinguish the selected + c8a.metal-24xl initial acceptance from pending Intel/AMD HPC qualification. Explicitly recommend against SMT workers, + even when hardware SMT is enabled, and demonstrate launcher plus OpenMP settings for one worker per physical core. + Multi-node performance qualification remains pending a proper HPC cluster/interconnect; do not claim it from EC2 + same-instance MPI tests. +- [ ] State exact immediate breaks: partitions/child_factory arguments, PartitionChildFactory, partition-only helpers, + errors and environment controls removed without aliases; no replacement public num_threads argument. Old automatic + physical-core selection, multi-rank serial default, library pinning and direct hwloc requirement are gone. OpenMP is + mandatory even without MPI. Raw order/splitmix physical ownership may differ; ABI requires rebuild. Requests are not hardware-clamped. Preserve high-level Python mathematical signatures. In prose/docstrings use `[Symbol][]` references, not hard-coded API URLs. - [ ] Run `prek run --all-files` before any separately authorized push, plus relevant tests, `just gen-api` and @@ -1590,11 +1783,16 @@ nix develop --command bash -euo pipefail -c ' **Final acceptance checklist:** -- [ ] Mathematical features, near-cutoff term retention, copies/updates/closures and graph paring pass existing oracles. +- [ ] Mathematical features, equal global retained term keys, copies/updates/closures and graph paring pass existing + oracles; matching energies/gradients alone is insufficient. +- [ ] Environment-only immutable budgets, strict invalid-input errors, removed partition interfaces and enforced + invalid-object/dependent-functional rejection pass; host calling assumptions and immediate breaks are documented. - [ ] Thread-count-stable ordering and approved deterministic reductions pass; no shared append, lazy-cache, packed-word or MPI-worker race remains. - [ ] Fixed uint32_t TermIndex and optional-MPI/non-MPI packaging pass. -- [ ] Rank-wide limits and real placement satisfy the selected baseline geometries. +- [ ] Rank-wide limits and verified one-worker-per-physical-core placement satisfy the single-instance campaign, + including its two-ranks-per-domain case. HPC-family performance and proper-interconnect multi-node qualification are + explicitly pending, not falsely passed or silently included in the initial acceptance claim. - [ ] Every required runtime and exact peak-memory cell passes Task 11; unsupported or missing measurements are not marked passed. - [ ] No legacy runtime, direct hwloc dependency or replacement topology/pinning service remains; no equivalent diff --git a/docs/superpowers/specs/2026-09-18-rank-local-openmp-design.md b/docs/superpowers/specs/2026-09-18-rank-local-openmp-design.md index 37eefee1..6effa311 100644 --- a/docs/superpowers/specs/2026-09-18-rank-local-openmp-design.md +++ b/docs/superpowers/specs/2026-09-18-rank-local-openmp-design.md @@ -7,16 +7,29 @@ direction discussed with the owner; it is not a claim that the proposed implemen Source inspected: branch `perf/linear-routing-on-wire`, commit `290112c8289ab8015eb9a2c7651ac409839c5a88`. -The owner requested a written implementation plan, not implementation or publication. The owner has approved removing -monoprop's direct hwloc dependency and making process/thread counts and placement user responsibilities. Approval of the -full plan is still required before executing it; the remaining compatibility policy is not an assertion that a breaking -C++ API change has already been approved. +The planning documents were published in `refactor-parallelization` at `38eb13d80a95dd1ac60e762f14f68c9a5925f755`; +this policy revision follows the owner's design interview. The owner approved mandatory OpenMP, removal of direct hwloc +and partition configuration without deprecation aliases, the associated API/ABI and capacity breaks, environment-only +construction-time thread budgets, and the failure/calling contracts below. These are design approvals, not +implementation or publication authority for this revision. + +The owner selected one AWS EC2 `c8a.metal-24xl` for implementation/testing and the initial acceptance campaign. +Production deployments are dual-socket HPC nodes using Intel Sapphire Rapids or AMD Zen3/Zen4/Zen5. Hardware SMT may be +on or off, but the recommended configuration always uses one worker per physical core, never extra SMT-sibling workers. +Passing the frozen EC2 campaign is sufficient for initial acceptance; production-HPC performance remains unverified +until separately qualified. Multi-node qualification is pending a proper HPC cluster/interconnect, not a second EC2 +instance. Observed EC2 topology/software and representative workload sizes still need recording/approval before freezing +that campaign. The owner approved baseline-only, unscored feasibility calibration before the freeze, once execution is +separately authorized. Small/medium profiles cover the full geometry sweep; large profiles cover only the full-node +one- and two-ranks-per-domain configurations. This revision authorizes no runtime work or implementation. ## Objective Replace thread-owned operator partitions and their in-process communication protocols with one compact operator store per MPI rank, operated on by OpenMP worksharing. Preserve the mathematical features, memory footprint, and performance -of the existing library. Each MPI rank's assigned CPUs are assumed to lie within one NUMA domain. The user chooses the +of the existing library. Simplification without regressions is the objective; the one-store/OpenMP candidate must earn +acceptance, not be rescued with prohibited replacement machinery if its gates fail. Each MPI rank's assigned CPUs are +assumed to lie within one NUMA domain. The user chooses the number of processes P and requested threads per process T for the resources available, and configures the launcher and OpenMP runtime to bind/pin processes and threads correctly. The library neither discovers topology nor repairs placement. @@ -27,11 +40,15 @@ placement. - MPI remains optional and OFF by default; retain working non-MPI wheels. - OpenMP becomes a required implementation dependency for the replacement runtime; support Linux x86_64, Linux aarch64, and macOS builds covered by the current project. -- Each MPI rank owns exactly one operator store and one graph; threads are never communicator ranks. +- Each independent propagator instance owns one packed operator store per MPI rank; threads are never communicator + ranks. Preserve independent copies, immutable graph-core sharing, retained coefficient/state snapshots and pared + graphs. - Remove monoprop's direct hwloc dependency and topology/affinity machinery; do not replace it with another library-owned hardware-discovery or pinning mechanism. - Users choose P processes and T threads per process and configure resource allocation, binding and pinning; monoprop does not clamp T to physical cores or validate NUMA placement. +- SMT use is not recommended: allocate at most one monoprop worker per physical core, including on SMT-enabled hosts. + Enforce this through user launch/binding settings and acceptance evidence, not a library hardware policy. - Preserve packed operator rows, the keyless hash index, fixed uint32_t TermIndex, sparse state storage, and inverted-index cosine recomputation. - Do not introduce boost::concurrent_flat_map, hidden shards, a dense duplicate store, a second persistent key store, a @@ -41,7 +58,7 @@ placement. - MPI calls execute on the controlling thread outside library-created OpenMP worksharing regions; retain MPI_THREAD_SERIALIZED rather than changing the initialization contract. - No exception escapes an OpenMP structured region; distributed operations fail coherently rather than leaving peers in - collectives. + collectives. Enforce invalidation after failed mutation; dependent functionals cannot reuse an invalid owner. - No global omp_set_num_threads/omp_set_dynamic/omp_set_nested mutation; each worksharing region receives an explicit per-object thread budget. - Use bitmap traversal or anticommutation pass for the existing operation; reserve prefix offsets for cumulative counts @@ -59,7 +76,11 @@ placement. ## Architecture -One rank owns its `MPOperator`, `MPGraph`, and ordinary MPI communicator. A small header-only worksharing helper +Each independent propagator has its rank-local `MPOperator`, primary `MPGraph`, and ordinary MPI communicator. This is +not a process-wide singleton: copies own independent packed stores; retained functionals may own coefficient/state +snapshots and pared graphs without cloning the packed store. Keep existing C++ owner-lifetime requirements and Python +owner retention. The prohibition on redundant stores/buffering concerns new parallelization machinery, not these +existing mathematical objects. A small header-only worksharing helper supplies deterministic block IDs, explicit thread budgets, and exception collection. It owns no threads, queues, communicators, or operator data. OpenMP owns worker teams. @@ -68,9 +89,13 @@ resizes shared containers and publishes structural changes between phases. Initi table and inverted-index maintenance remain single-writer. This is an explicit performance gate, not an assumption that insertion is negligible. -Only recognized pure `LengthCutoff`/`SupportCutoff` targets may run in worker traversal initially. Arbitrary cutoff -callbacks and current basis-change closures retain serial invocation; a const `std::function` does not establish that -its captured state is thread-safe. +Only recognized pure `LengthCutoff`/`SupportCutoff` targets may run in worker traversal initially. Ordinary Python and +C++ constructors accept built-in cutoff configuration and basis-change data, not arbitrary cutoff functions. Opaque +predicates are reachable through the existing protected C++ `cutoff_fn_` extension point and lower-level helpers; +retain their conservative serial path without inventing a Python callback API or an artificial callback benchmark. +Current basis-change closures capture data by value; audit their callees rather than claim a mutable capture exists. +Their initial serial path is a conservative implementation choice, not proof that they are unsafe or a performance +exemption. Actual lazy cosine/parity caches still require caller-side preparation. A gate remains sequential with respect to other gates: @@ -147,79 +172,154 @@ into a batch/store. Task 6 defines the checked preparation sequence. ## Resource ownership and deployment -- The user is responsible for choosing P processes and T threads per process to match available resources, including any - deliberate SMT use. Normally set P with the scheduler/MPI launcher and T with `OMP_NUM_THREADS=T`. +### Selected implementation/test platform + +AWS documents `c8a.metal-24xl` as AMD EPYC 9R45, 96 physical cores/96 vCPUs, one thread per core and 192 GiB RAM in its +[exact-size specifications](https://docs.aws.amazon.com/ec2/latest/instancetypes/co.html#co_hardware). The +[C8a page](https://aws.amazon.com/ec2/instance-types/c8a/) identifies fifth-generation EPYC (Turin) and explicitly +states that there is no SMT. Do not call a 192-worker run on this instance an SMT-on test; it would oversubscribe its +physical cores. The exact socket/NUMA layout was not established by the reviewed AWS specifications and must be observed +on the actual allocation before selecting rank/thread geometries. In particular, do not assume eight NUMA domains or +approve one rank spanning all 96 cores without checking containment. + +These published specifications are not measurements or evidence of parity. C8a does not establish performance on +Sapphire Rapids, Zen3/Zen4 or HPC hosts with hardware SMT enabled. Those remain separate qualification targets using the +same one-worker-per-physical-core recommendation; no SMT-sibling worker sweep is required. Initial acceptance is scoped +to the frozen single-instance C8a campaign. Multi-node qualification requires a proper HPC cluster/interconnect and +remains pending; do not substitute a cloud-network experiment or claim HPC fabric scaling from local MPI. + +### User-owned allocation and placement + +- The user chooses P processes and T threads per process for the allocated physical cores. SMT use is not recommended: + do not count SMT siblings as extra worker capacity. Set P with the launcher and normally T with + `monoprop_NUM_THREADS=T` at launch. + When it is unset, use the OpenMP runtime default, normally configured by `OMP_NUM_THREADS`. No runtime reconfiguration + API or environment reload is provided for an existing object. - The user is responsible for binding/pinning processes and threads to the correct hardware using, for example, `OMP_PLACES` + `OMP_PROC_BIND`, `srun --cpu-bind`, and/or `mpiexec` options. Scheduler allocation, rank binding and OpenMP places must agree: binding a rank to one core while requesting many threads does not allocate additional cores. Each rank's allocation must accommodate its workers within one NUMA domain; avoid unintended overlap with other ranks. -- Monoprop validates thread-count representability, not hardware suitability. It does not discover physical cores, - choose SMT policy, redistribute CPUs, change affinity, validate NUMA containment or prevent oversubscription. OpenMP's + Documentation will advise at least one MPI process per NUMA domain used by the allocation; multiple processes may + share a domain using disjoint CPU allocations. The production nodes commonly have four domains per socket, hence + eight per full dual-socket node, but use the actual configured topology rather than a hard-coded count. Do not infer + the EC2 topology from that HPC convention. Partial-node runs cover only their allocated domains. +- Monoprop validates thread-count representability, not hardware suitability. The executable does not discover physical + cores, enforce the documented no-SMT-worker recommendation, redistribute CPUs, change affinity, validate NUMA + containment or prevent oversubscription. OpenMP's own limits and small-work/nested serial paths may use fewer workers; they do not certify adequate resource allocation. - Delete `CpuTopology` with the partition runtime and remove direct hwloc discovery/link/install requirements. Do not retain hwloc for mandatory diagnostics. Benchmark operators still verify placement with runtime/launcher diagnostics and external tools; those are evidence requirements, not library dependencies. MPI or the OpenMP runtime may independently depend on hwloc. -## Thread selection and compatibility policy +### Physical-core launch recipes + +These are documentation templates, not executed/qualified launch commands. T is the number of assigned physical cores +per rank and the total OpenMP team size, including its primary thread. Set both launch-time budgets explicitly: + +```bash +export monoprop_NUM_THREADS="$T" OMP_NUM_THREADS="$T" +export OMP_PLACES=cores OMP_PROC_BIND=close OMP_DYNAMIC=FALSE +``` + +A core place may contain both hardware threads of a physical core. With exactly T distinct core places and T workers, +`close` assigns one worker per place; it does not launch a worker on each sibling. `OMP_PLACES=cores` alone is not a +thread-count limit: requesting more workers than places can share cores. Verify the runtime's actual places and worker +masks, and remove conflicting runtime/site affinity overrides. Do not recommend SMT with `OMP_PLACES=threads`. +See the [OpenMP places](https://openmp.org/spec-html/5.1/openmpse62.html) and +[close binding](https://openmp.org/spec-html/5.1/openmpsu41.html) definitions. + +**Open MPI 4.1/5.x:** for K=1 or K=2 ranks per participating NUMA domain and R total ranks, each domain must provide at +least K*T available physical cores. Obtain a matching allocation and select R/K/T consistently with its actual topology: + +```bash +mpirun -n "$R" --map-by "ppr:${K}:numa:PE=${T}" \ + --bind-to core --nooversubscribe --report-bindings ./application +``` + +Processing elements are cores by default. Do not use `--use-hwthread-cpus`, `HWTCPUS` or binding-overload options. +Open MPI 5 documents `CORECPUS`; it is absent from the 4.1 manual, so the shared example omits it and requires the +core-counting default without conflicting overrides. `--nooversubscribe` checks process slots, not proof of disjoint +physical-core ownership. Verify binding reports: T distinct cores per rank, no physical core shared across ranks, and +one NUMA domain per rank. Do not assume a partial allocation exposes exactly the desired domains. Sources: +[Open MPI 4.1](https://www.open-mpi.org/doc/v4.1/man1/mpirun.1.php) and +[Open MPI 5.0](https://docs.open-mpi.org/en/v5.0.0/man-openmpi/man1/mpirun.1.html). + +**Slurm:** the following single-node template requires matching job/step CPU requests, task/affinity support and a +site-approved MPI plugin (select MPI_PLUGIN with site guidance; `srun --mpi=list` lists installed choices). It is valid +only when observed site ordering/allocation makes each resulting T-core rank mask NUMA-local: + +```bash +srun --nodes=1 --ntasks="$R" --ntasks-per-node="$R" --mpi="$MPI_PLUGIN" \ + --cpus-per-task="$T" --threads-per-core=1 \ + --distribution=block:block --cpu-bind=verbose,cores ./application +``` + +`--threads-per-core=1` restricts task layout to one thread/core; explicit core binding overrides its implied thread +binding. Core masks may still include sibling CPUs, but the OpenMP settings above use one worker/core. Do not combine +these explicit options with `--hint=nomultithread`: Slurm documents conflicts/overrides. That hint is an alternative, +not an extra safeguard. Sources: [srun](https://slurm.schedmd.com/srun.html) and its +[CPU-binding options](https://slurm.schedmd.com/srun.html#OPT_cpu-bind). + +Slurm's block distribution describes nodes/sockets/cores, **not NUMA domains**. Neither this command nor adding a third +`block` field proves containment on four-domains-per-socket machines. If masks cross domains, use a site-supported +NUMA-aware launcher or replace automatic binding with `--cpu-bind=verbose,mask_cpu:$MASKS`. Build hexadecimal masks from +observed CPU/sibling topology: T distinct physical cores within one domain, with disjoint physical cores across +ranks. Slurm requires every available CPU on the node to belong to the **job step** for explicit masks to be honored; +job-level exclusivity alone is insufficient, and a reduced `--cpus-per-task` step is not automatically eligible. Resolve +that allocation with the site before using masks. Mask lists repeat by node-local rank, so identical masks cannot be +assumed valid on differently numbered/configured nodes. Binding two ranks to the same entire locality domain does not +partition its cores. No automatic topology/binding fallback is added to monoprop. + +Archive observed rank/worker masks, physical-core sibling identities and NUMA containment before accepting measurements. +These recipes document HPC deployment; they do not convert pending proper-interconnect multi-node qualification into +completed evidence. -The replacement stores an immutable `parallel::Options` per propagator; copies copy that budget. Without a library -override, capture the calling context's OpenMP default via `omp_get_max_threads()` at construction, normally configured -by `OMP_NUM_THREADS`. Do not parse that environment variable independently; OpenMP also supports list-valued settings -and application-level runtime configuration. Later runtime-setting changes do not alter an existing object's captured -budget. A worksharing region may use fewer workers than the budget. Thread count never changes ownership or graph -layout. MPI ranks may use different budgets. +## Thread selection and compatibility policy -During the prototype only, add a final C++ constructor argument `size_t num_threads = 0` after the existing -`child_factory` argument. A nonzero value is allowed only when the constructor resolves one legacy partition; it enables -threaded kernels on that single store. Existing legacy children use one kernel thread. This gives a testable transition -without nesting runtimes. +The replacement stores an immutable `parallel::Options` per propagator; copies and retained functionals preserve it. +The supported deployment is launch-configured: capture the request exclusively at construction, never reread it during +operations, and provide no setter or hot-reconfiguration mechanism. A worksharing region may use fewer workers than the +budget; this is not a budget change. Thread count never changes ownership or graph layout. MPI ranks may use different +launch-time budgets. At cutover: -1. Keep the existing `partitions` constructor slot and low-level binding keyword for a deprecation window. Explicit - positive values become a deprecated thread-budget request, with a once-per-process warning. `partitions=1` remains - serial. There are no actual thread partitions. -2. Keep `monoprop_PARTITIONS=off|auto|N` as a deprecated, warned thread-budget alias. `off` means one thread; positive N - requests that budget. `auto` and malformed values fall through to a valid `monoprop_NUM_THREADS` override, otherwise - the OpenMP default. They no longer discover hardware. -3. A positive explicit C++ `num_threads` wins over environment settings, but supplying it alongside positive explicit - `partitions` is an error. Otherwise precedence is: explicit positive partitions, then monoprop_PARTITIONS - off/positive N, then valid monoprop_NUM_THREADS, then the captured OpenMP default. Preserve the existing - monoprop_NUM_THREADS parser/range; recommend OMP_NUM_THREADS for normal configuration and unset library overrides - when using it. -4. The OpenMP-default path is the same for one or multiple MPI ranks. Do not retain the old automatic physical-core - selection or multi-rank default of one thread. If OMP_NUM_THREADS is unset, use the runtime's default, without a - library promise about that default's suitability. Explicit positive requests must fit `int`; reject unrepresentable - counts instead of truncating or hardware-clamping them. Public constructor zero remains the no-explicit-request - sentinel; resolved Options always contains a positive count. Document the changed default-selection policy. -5. `OMP_PLACES` and `OMP_PROC_BIND` control OpenMP placement; monoprop never modifies affinity. Explicit library - overrides take precedence over the runtime's default thread budget. Runtime limits such as OMP_THREAD_LIMIT and - dynamic-team adjustment may reduce actual teams even when a larger count was requested; correctness must not depend - on receiving exactly T workers. No place-union/physical-core query is needed to select or validate T. Verify actual - placement externally for performance acceptance, not during ordinary propagator construction. -6. Retain the `PartitionChildFactory` type and constructor argument temporarily for source diagnostics, but reject a - non-null factory with `PropagatorConfigError` explaining that thread-owned children no longer exist. Do not ignore it - or invoke it once with changed semantics. -7. Remove protected partition-only helpers and facade-specific accessor errors. This is an explicit C++ - extension-interface breaking change requiring a release note; do not describe the refactor as universally source/ABI - compatible. Keep the virtual destructor, virtual `update_initial_operator`, its protected nonvirtual - `apply_initial_operator_` helper, and independent virtual `clone_` hook (`MonomialPropagator.h:286–305`). -8. Python mathematical constructors/methods remain unchanged. Low-level `_core` gains the optional `num_threads` keyword - used by tests; the public frontends continue to use environment configuration. -9. Returned coefficient/term ordering can differ from the old partition-block order. Candidate query/row ordering is - deterministic across thread budgets at fixed rank geometry; this does not promise the old raw export layout. Validate - key/value associations, not coefficient-only sorted lists. -10. EXISTING routing is boolean linear/splitmix (`Routing.h::linear_requested`, `linear_bits_for`, `make_router`). - Linear is the default and requires power-of-two R; unsupported geometry throws `UnroutableGeometry`, not a fallback. - Preserve linear rank mapping, generator `rank_shift`/`dest_from_shift` and peer selection. Splitmix previously - placed a term on `floor((hash % (R*S))/S)`; S=1 uses `hash % R`. Obtain explicit approval of that ownership change, - including rank-local export/load/reduction effects; do not retain hidden shards. New thread budgets never change - ownership. Test default-linear R=3 rejection separately. Every successful R=3 suite or benchmark explicitly sets - `monoprop_ROUTING=splitmix`. Compare global maps and per-rank loads/peaks at identical routing/seed/rank geometry. - -The plan's final removal task may proceed only after the owner approves this policy as part of the plan, or provides a -replacement policy. Implementation agents must not invent another migration. +1. Remove `partitions` from the C++ constructor and low-level binding, `PartitionChildFactory`, partition-specific + helpers/error types, and every partition configuration environment variable. There is no deprecation window, alias, + compatibility warning shim or new public `num_threads` constructor argument. Existing `monoprop_PARTITIONS` has no + effect in the candidate: do not retain a getenv reader merely to diagnose it. Remove it from launch scripts/tests. +2. A present `monoprop_NUM_THREADS` must contain a positive decimal integer representable as `int`. Reject empty, + malformed, zero, negative and overflowing values with an actionable configuration error; never silently fall back. + Only an unset variable selects `omp_get_max_threads()`. Preserve the chosen positive value exactly, without hardware + clamping. Replace the old cached permissive parser for this setting; routing settings retain their own contract. +3. Do not independently parse `OMP_NUM_THREADS`. OpenMP supports list-valued settings and application initialization; + its runtime default is the fallback at any MPI rank count. If it is also unset, use that default without promising + hardware suitability. Do not preserve automatic physical-core discovery or a multi-rank default of one thread. +4. `OMP_PLACES` and `OMP_PROC_BIND` control placement; monoprop never modifies affinity. OpenMP limits/dynamic-team + adjustment may reduce actual teams; correctness must not depend on receiving exactly T workers. Verify actual + placement and participation externally for performance acceptance, not during ordinary construction. +5. Keep the virtual destructor, virtual `update_initial_operator`, its protected nonvirtual `apply_initial_operator_` + helper, and independent virtual `clone_` hook (`MonomialPropagator.h:286–305`). The owner accepts immediate C++ + source/ABI breaks for partition removal in this 0.x library; record release notes and rebuild library/bindings. + High-level Python mathematical signatures remain unchanged; `_core` loses `partitions` and gains no thread keyword. +6. Returned term/coefficient ordering may differ from old partition-block order. Validate identical global retained term + keys and coefficient associations within existing tolerances, not coefficient-only sorted lists. Equal energies and + gradients alone are insufficient. Keep candidate ordering deterministic across budgets at fixed rank geometry. +7. Preserve boolean linear/splitmix routing, default linear rejection of unsupported R, generator shifts and peer rules. + The owner accepts the splitmix ownership change from `floor((hash % (R*S))/S)` to `hash % R`; do not emulate S with + hidden shards. Keep `monoprop_ROUTING` and `monoprop_ROUTE_SEED`: these control MPI routing, not thread partitions. + Test default-linear R=3 rejection separately; successful R=3 suites/benchmarks explicitly use splitmix. Compare + global maps and per-rank loads/peaks at identical routing/seed/rank geometry. + +Development-only coexistence uses the EXISTING explicit `partitions=1` argument plus `comm.kind==mpi::Comm::Kind::Mpi` +to select the one-store prototype and capture its environment budget. Legacy Shm/Hybrid children and multi-partition +facades retain serial kernel options. No new constructor keyword, environment switch or backend selector is needed. +Task 10 removes that transitional selection and captures the budget for every independent instance. Tests use explicit +internal kernel Options or separately launched processes, not a production budget setter. + +These compatibility policies are approved. Execution and final removal still require implementation authorization and +all earlier gates; they are not permission to bypass performance or packaging failures. ## Safety and failure policy @@ -227,11 +327,16 @@ The worksharing helper catches each worker exception, joins the region, and reth distributed phase guards print the underlying failure and call MPI_Abort when communicator size exceeds one. Catch failures inside the lifetime of active exchange tickets, before stack unwinding can wait on peers that never posted; an outer catch alone is insufficient. Successful paths explicitly wait as before. For single-rank operation, rethrow the -original exception. An operation that failed after mutations is not transactional and the propagator must not be reused. +original exception. Enforce post-mutation invalidation on the affected propagator: later state-consuming/mutating calls, +copy/clone and dependent retained functionals reject it before touching state. Existing independent copies remain valid. +Pre-mutation validation failures leave a valid object usable; destruction remains safe. There is no rollback or reset +that revives a failed object. Task 3 defines the invalid flag, entry checks and failure-injection tests. + No rank-local early return may bypass a collective peers will enter. Validate MPI_THREAD_SERIALIZED also when MPI was -initialized externally. Distributed entry from an external OpenMP team is not a supported calling contract; numerical -helpers may serialize when nested, but this does not authorize MPI from workers or concurrent evaluation on a shared -communicator. +initialized externally. The owner confirms one controlling caller per rank, outside active host OpenMP teams, no +overlapping calls on the same propagator/dependent functionals and no overlapping host MPI calls. Distributed entry +from an external OpenMP team remains unsupported; numerical helpers may serialize when nested, but that does not +expand the public calling contract. Add no process-wide locking service or nested-parallel execution machinery. Lazy initialization, allocation, and cache mutation must be outside worker bodies unless storage is strictly worker-private. Do not assume `const` means thread-safe: both the inverted index and parity cache have lazy mutation @@ -239,6 +344,40 @@ today. ## Performance acceptance and stopping +The initial acceptance gate is a frozen campaign on one c8a.metal-24xl, with at most one worker per physical core. +Record the actual socket/NUMA/core layout before selecting geometries. Include one-rank-per-allocated-domain operation +and an additional two-ranks-per-domain configuration with disjoint physical cores, where resources permit. Both arms use +identical decomposition/allocation in every comparison; changing ranks only on the candidate cannot rescue a regression. +Source-reference workloads remain candidates, not approved production sizes. After separate execution authorization, +use baseline-only, unscored calibration to choose feasible sizes. The approved pilot budget is 2 hours total, excluding +builds/setup, with 15 minutes per trial including construction, evaluation and numerical export/validation. If the time +budget is insufficient, stop and report rather than silently extend it. Target at most 75% of observed usable allocation +RAM across all ranks, including construction and validation/export temporary storage. Record the RAM basis and byte +ceiling before the pilot; do not continuously rescale it from falling free memory during a trial. Exceeding the memory +target requires owner review before larger trials. These are operator-side sizing limits, not library memory/clamping +policies or guarantees against out-of-memory failures. Estimate formal campaign time/cost from the pilot before seeking +approval; do not reduce required samples or coverage to meet an unapproved budget. + +Archive each trial's config, placement, binary identity, elapsed time, memory observations and outcome, including +rejected sizes. Do not consult candidate performance when selecting the campaign. Baseline numerical inconsistencies +stop calibration rather than justify a smaller test. Resource failures may inform pre-freeze sizing; they are not failed +acceptance cells. The approved calibration limits do not authorize builds, setup or runtime execution. + +Small/medium profiles cover the full geometry sweep with the existing routing requirements. Large profiles cover only +the full-node one- and two-ranks-per-NUMA-domain configurations, not one-worker runs. This coverage is chosen before +formal measurements, not used to +excuse a failed cell later. Select the actual large workloads and sizes from calibration and obtain owner approval. +Freeze the complete size-band-specific inventory before formal baseline collection. Calibration results are not +acceptance samples: collect fresh formal baseline runs after the freeze, even for unchanged configurations. Every +required cell must pass, but this does not claim parity for omitted workload/geometry combinations or CPU families. +Correctness still covers the supported API. A noisy failure means parity was not demonstrated, not proof of a slowdown, +and still blocks acceptance without a waiver. + +Intel/AMD HPC qualification remains separate from initial acceptance and must not be reported as already passed. +Multi-node performance qualification is pending access to a proper HPC cluster and interconnect; same-instance MPI +correctness/performance does not substitute for that evidence. Do not add two-instance EC2 networking to the initial +campaign or require sibling-worker SMT profiles. + Keep the baseline revision separately built. The benchmark operator must set P/T and verify the allocation and actual worker placement using OpenMP/launcher diagnostics and external tools, not a monoprop topology service. Use identical MPI rank placement and physical cores; changing ranks per domain or widening indices is not a valid comparison. Measure @@ -273,8 +412,9 @@ scalar/full-gradient validation, and separate whole-construction peak scope. Thi full-plan approval, not an existing pytest node or instrumentation-only edit. Use identical driver code on both arms; do not build a general benchmark framework. -Freeze and archive an owner-approved campaign inventory and digest before baseline collection. It lists every required -profile/node/geometry/routing cell independently of the observations. Comparator input must match that inventory, so +Freeze and archive an owner-approved campaign inventory and digest after unscored calibration and before formal baseline +collection. It lists every required profile/node/geometry/routing cell for its size band, independently of observations. +Comparator input must match that inventory, so removing a whole cell from both arms cannot pass. Each sample references one fresh timed artifact, one separate fresh construction artifact, and a matching untimed validation artifact. Join only matching cell/arm/binary/configuration, parameters, geometry/routing/allocation and campaign provenance. Timed/construction run IDs and files cannot be reused; @@ -283,16 +423,25 @@ be reused only for the same cell/arm/binary/configuration. Task 1 specifies the independent placement references; no caller boolean or synthetic-data bypass substitutes for that evidence. The shared shape-preflight overlay must accept declared baseline `monoprop_PARTITIONS=T` and candidate -`OMP_NUM_THREADS=T` with library overrides unset, on the same allocation. Verify declarations against observed shape; +`monoprop_NUM_THREADS=T` with removed partition settings unset, on the same allocation. Test the candidate's +unset-library fallback to OpenMP separately; both configuration paths must resolve the declared budget. Verify +declarations against observed shape; arm labels alone cannot establish shape. Extend existing ladder/provenance/term-count fields without renaming historical nodes. `tools/capture-baseline.py` supplies supplementary small-fixture evidence only; it cannot replace global, ownership-aware benchmark validation with gradients and provenance. -The full-rank store must fit the fixed uint32_t TermIndex ceiling (`TypeAliases.h:39`). Row IDs exclude UINT32_MAX, -which is the index sentinel; row counts and some graph counts may equal it. Extend existing overflow guards/tests rather -than claim there are two index-width builds. The store must also fit capacity arithmetic and MPI count limits. If not, -report that capacity incompatibility; do not widen indices, change rank geometry, or add local shards without a new -design decision. +The owner accepts reduced per-rank capacity: one independent propagator's store must fit fixed uint32_t TermIndex +(`TypeAliases.h:39`), whereas several old partitions could collectively exceed that ceiling. Row IDs exclude +UINT32_MAX, the index sentinel; row counts and some graph counts may equal it. Extend existing guards, including +allocation arithmetic and MPI counts. A required campaign cell that does not fit still blocks acceptance; never widen +indices, change ranks or add shards silently. + +The representative 512-GiB/eight-domain example does not select the campaign or impose a hardware limit. If evenly +budgeted at 64 GiB/rank, current coefficient-bearing storage is memory-limited before the ceiling. Approximately 2^32 +four-byte IDs occupy 16 GiB, not a full operator: at that row count, a consolidated current hash table alone is 64 GiB, +coefficients 32 GiB, and packed rows at least 8 GiB, before other storage. Across old partitions the corresponding +optimistic aggregate lower bound exceeds 85 GiB. These are source-derived bounds from `OperatorIndex.h`/`MPOperator.h`, +not measurements; CPU binding alone does not enforce a RAM budget. ## Non-goals @@ -302,13 +451,20 @@ topology/affinity-management subsystem. Do not remove useful lifetime/ordering c ## Execution handoff and supported build routes -All future code/test/build commands in the companion plan are planned, not executed during this document refresh. Only -these two Markdown documents are authorized now. Before implementation, recheck exact HEAD and the plan's baseline-drift +All future code/test/build commands in the companion plan are planned, not executed during this document refresh. +This revision changes planning documentation and the agreed glossary only; no implementation or publication is +authorized. Before implementation, recheck exact HEAD and the plan's baseline-drift table; stop on changed signatures, routing, capacity, numerical or benchmark contracts. Keep 13 gated tasks and record RED/GREEN evidence for new behavior, plus baseline results for preservation tests that may already pass. Do not manufacture RED by removing earlier plumbing. Use kernel-specific worker observations for threading participation. Do not execute past a failed gate or infer approval from this refresh. +The baseline-only pilot travels with Task 1 to the authorized execution machine; it need not run on the planning host +before handoff. Transfer the latest spec, plan and CONTEXT.md, not only the historical published planning commit. +The plan's remote entry point defines a bounded first phase ending with a proposed campaign/budget for owner approval, +not candidate engine work or a completed Task 1 gate. A different execution host does not silently change the agreed +c8a.metal-24xl acceptance target. No machine provisioning, execution or publication is authorized by this clarification. + Required OpenMP must link PUBLIC on both `monoprop-objs` and the shared `monoprop` target, plus installed `find_dependency(OpenMP REQUIRED COMPONENTS CXX)`; INTERFACE alone does not link the shared library itself. Cover wheel hooks in `pyproject.toml`, actual wheel jobs in `.github/workflows/deploy.yml`, `tools/install-deps.sh`, shared From 35f359f095fa3f1d628ca2fdc66c93fef6ea7564 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Roberto=20Di=20Remigio=20Eik=C3=A5s?= Date: Thu, 24 Sep 2026 11:59:52 +0200 Subject: [PATCH 03/64] chore: spec and plan revision after rebase, add mini-app experiment --- .../plans/2026-09-18-rank-local-openmp.md | 699 ++++++++++++++---- .../2026-09-18-rank-local-openmp-design.md | 329 +++++++-- 2 files changed, 843 insertions(+), 185 deletions(-) diff --git a/docs/superpowers/plans/2026-09-18-rank-local-openmp.md b/docs/superpowers/plans/2026-09-18-rank-local-openmp.md index 05a3456a..984ff781 100644 --- a/docs/superpowers/plans/2026-09-18-rank-local-openmp.md +++ b/docs/superpowers/plans/2026-09-18-rank-local-openmp.md @@ -2,17 +2,19 @@ > **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or > superpowers:executing-plans to implement this plan task-by-task. Read both documents completely; full-plan execution -> requires explicit approval. A separately authorized baseline-only calibration may precede campaign approval, as Task 1 -> describes. Do not infer missing contracts. Checkboxes track future work. +> requires explicit approval. Task 0 now precedes all downstream work and ends with an owner architecture decision. +> After that gate, separately authorized baseline-only calibration may precede campaign approval, as Task 1 describes. +> Do not infer missing contracts. Checkboxes track future work. -**Goal:** Replace thread-owned partitions with one compact operator store per MPI rank and OpenMP worksharing, accepting -the replacement only after correctness, runtime, and peak-memory parity have been demonstrated. +**Goal:** First compare MPI-off index ownership/scaling with a mini-app. Reopen the one-store-per-rank choice using its +evidence; implement the conditional OpenMP replacement only after architecture approval and full-library validation. -**Architecture:** Keep the packed position-list rows, keyless hash index, inverted index, graph representation, and MPI -routing algorithms/settings. Setting the router's partition dimension to one can change historical physical-rank +**Conditional architecture (Tasks 1–13):** Keep packed position-list rows, keyless indexing, inverted indices, graphs +and MPI routing algorithms/settings. Setting the router's partition dimension to one can change historical physical-rank ownership under splitmix routing; Task 10 makes that compatibility consequence explicit. Workers traverse frozen -structures or update preallocated disjoint ranges; the controlling thread owns structural publication and MPI. Prototype -fixed-graph replay first, then deterministic graph construction, and delete the old runtime last. +structures or update preallocated disjoint ranges; the caller initially owns structural publication. A bounded Boost +concurrent-index trial may change hash publication, not row/graph ownership or canonical IDs. Final MPI calls belong to +MPI's initializing thread under FUNNELED. Prototype replay first, then construction, and delete the old runtime last. **Tech Stack:** C++23, OpenMP CXX, existing Boost and optional MPI, nanobind, Python 3.11+, uv/scikit-build-core, Boost.Test, pytest. Monoprop's direct hwloc dependency is removed with the legacy partition runtime. @@ -20,11 +22,17 @@ Boost.Test, pytest. Monoprop's direct hwloc dependency is removed with the legac **Spec:** [Rank-local OpenMP design](../specs/2026-09-18-rank-local-openmp-design.md). Read both documents before execution. -**Status:** Planning only. Source baseline: `perf/linear-routing-on-wire` at -`290112c8289ab8015eb9a2c7651ac409839c5a88`; published planning commit: `refactor-parallelization` at -`38eb13d80a95dd1ac60e762f14f68c9a5925f755`. This documentation-only interview revision records approved runtime, -compatibility, numerical and failure policies. OpenMP is mandatory; partition configuration is removed immediately; -thread budgets are launch-environment-only and captured at construction. The fixed32 capacity reduction is accepted. +**Status:** Planning only. Refreshed source baseline: current `origin/main` at +`90d57177c2b0cd93503f88dd931d2f8dd409aa23`, inspected in rebased `refactor-parallelization` at +`9bede897d99ac7e2a53442885629d5032f1b7df4`. Only CONTEXT.md and these planning documents differ between those commits. +The original planning publication was `38eb13d80a95dd1ac60e762f14f68c9a5925f755`. This owner-requested current-main +refresh preserves the subsequent approved runtime, compatibility, numerical and failure policies and uncommitted edits. +The owner has now explicitly reopened the one-store architecture decision through Task 0. Tasks 1–13 retain the +previously approved one-store proposal as a CONDITIONAL path, not an automatic follow-up. Stop after the mini-app for +an owner decision; reconcile both documents before proceeding, including the required MPI-off performance coverage. +On the retained one-store path, OpenMP is mandatory; partition configuration is removed immediately; thread budgets are +launch-environment-only and captured at construction. The fixed32 capacity reduction remains an explicit consequence, +not a restriction that predetermines Task 0's architectural conclusion. The owner selected one AWS EC2 `c8a.metal-24xl` for implementation/testing and initial acceptance. Production targets are dual-socket Sapphire Rapids and AMD Zen3/Zen4/Zen5 HPC nodes, commonly four NUMA domains/socket. Hardware SMT may be on or off, but SMT workers are not recommended: always configure one worker per physical core. Passing the frozen @@ -35,11 +43,24 @@ The owner approved baseline-only, unscored sizing trials before the freeze, once profiles cover the full geometry sweep; large profiles cover only the full-node one- and two-ranks-per-domain cases. No implementation, engine builds/tests or benchmarks are authorized by this revision. All future execution commands -below are planned, not evidence of passing gates. EXISTING labels describe current source; PROPOSED labels describe -future work. Full implementation and any publication of this revision require separate authorization. +below are planned, not evidence of passing gates. EXISTING labels describe the inspected baseline; PROPOSED labels +describe future work. Full implementation and any publication of this revision require separate authorization. + +**Source-drift audit completed:** this refresh replaces the superseded +`290112c8289ab8015eb9a2c7651ac409839c5a88` pin with the main revision above, as requested by the owner. All 43 changed +paths were audited; affected contracts, source anchors, examples and preservation tests are updated below. The previous +d93d63f source-refresh blocker is resolved, not carried forward merely because HEAD contains planning commits. No +runtime gate is thereby passed or authorized. Both arms must use the refreshed pre-OpenMP engine as their common +starting point; samples from superseded engines cannot establish its baseline. Recheck subsequent source drift before +execution, and retain separate build/calibration authorization, campaign freeze and implementation gates. ## Global Constraints +The following constraints describe the conditional one-store path in Tasks 1–13. Task 0 alone permits the explicit +per-thread-store comparison and requires full-machine MPI-off measurements, including cross-NUMA placement. This is +not permission to add shards to production or to waive any existing acceptance gate. Reconcile these constraints with +the owner's post-experiment architecture decision before proceeding. + - C++23; retain the GCC 14 / Clang 18 minimum compiler versions and Python 3.11 floor. - MPI remains optional and OFF by default; retain working non-MPI wheels. - OpenMP becomes a required implementation dependency for the replacement runtime; support Linux x86_64, Linux aarch64, @@ -53,14 +74,18 @@ future work. Full implementation and any publication of this revision require se does not clamp T to physical cores or validate NUMA placement. - SMT use is not recommended: allocate at most one monoprop worker per physical core, including on SMT-enabled hosts. Enforce this through user launch/binding settings and acceptance evidence, not a library hardware policy. -- Preserve packed operator rows, the keyless hash index, fixed uint32_t TermIndex, sparse state storage, and - inverted-index cosine recomputation. -- Do not introduce boost::concurrent_flat_map, hidden shards, a dense duplicate store, a second persistent key store, a - full per-thread operator, or full-vector coefficient double buffering. +- Preserve packed rows, keyless indexing through row references, fixed uint32_t TermIndex, sparse state storage, and + inverted-index cosine recomputation. A bounded Boost concurrent-index experiment may replace the index, not rows. +- Do not introduce hidden operator shards, a dense duplicate store, a second persistent full-key store, per-thread + operators, or full-vector coefficient double buffering. Select any concurrent index only after correctness/memory/time + evidence; its container name is neither a rejection criterion nor proof of suitability. +- Required performance cells keep each rank within one NUMA domain. Full-node cross-NUMA L2a is diagnostic-only and + cannot replace, excuse or average away any required cell; its numerical correctness is still checked. - Preserve Majorana and Pauli bases, Heisenberg and Schrödinger pictures, cutoffs, graph build/replay/paring, gradients, partial contraction, copying, and initial-operator updates. -- MPI calls execute on the controlling thread outside library-created OpenMP worksharing regions; retain - MPI_THREAD_SERIALIZED rather than changing the initialization contract. +- Final MPI calls execute on the MPI-initializing thread, outside library-created OpenMP regions. Request/validate at + least MPI_THREAD_FUNNELED, accepting higher provided levels without relaxing the caller rule. Retain + MPI_THREAD_SERIALIZED during legacy Hybrid coexistence; lower the minimum only with its removal in Task 12. - No exception escapes an OpenMP structured region; distributed operations fail coherently rather than leaving peers in collectives. Enforce invalidation after failed mutation; dependent functionals cannot reuse an invalid owner. - No global omp_set_num_threads/omp_set_dynamic/omp_set_nested mutation; each worksharing region receives an explicit @@ -80,16 +105,19 @@ future work. Full implementation and any publication of this revision require se Additional execution rules: -- **No runtime work or implementation is authorized by the design interview.** Obtain separate authorization for a - bounded baseline-only calibration/build phase; it may precede campaign approval. Candidate engine changes require +- **No runtime work or implementation is authorized by this planning amendment.** Task 0 needs its own implementation, + host and finite measurement-budget authorization; it is not part of Task 1's two-hour baseline-only pilot. After its + architecture gate, obtain separate authorization for bounded baseline calibration/build work before campaign approval. + Candidate engine changes require implementation authorization and an owner-approved frozen campaign. Formal baseline collection also waits for that freeze. Simplification without regressions is the goal; reject the architecture if it cannot earn acceptance. The campaign bounds performance claims, not mathematical correctness across the supported API. - P (called R in existing code/benchmark geometry) is the user-selected process count; T is the requested threads per process, normally set with `monoprop_NUM_THREADS`. Only an unset override selects the OpenMP runtime default. Users must allocate enough resources and configure rank/thread binding - with OpenMP, `srun --cpu-bind`, and/or `mpiexec` options. Each rank's workers are assumed to fit its allocation within - one NUMA domain. Documentation recommends against SMT workers: allocate one worker per physical core, including when + with OpenMP, `srun --cpu-bind`, and/or `mpiexec` options. Required production cells keep each rank within one NUMA + domain; full-node cross-NUMA L2a is a labelled diagnostic exception, never a parity waiver. Do not call a smaller + NUMA-local run L2a. Documentation recommends one worker per physical core, including when hardware SMT is enabled. Do not create a topology service/NUMA scheduler, silently change P/T, or enforce that recommendation through library clamping or affinity changes. - Do not touch pre-existing `.codegraph/` or `.direnv/`. Use an isolated implementation checkout and a separately built @@ -107,7 +135,8 @@ Additional execution rules: | Stage | Tasks | Gate before proceeding | | --- | --- | --- | -| Baseline | 1 | Authorized baseline-only calibration, approved frozen inventory, fresh formal baseline evidence | +| Architecture experiment | 0 | Small MPI-off mini-app, correct three-way comparison, owner architecture decision and reconciled documents | +| Baseline (conditional) | 1 | Task 0 decision, authorized baseline-only calibration, frozen inventory, fresh formal baseline evidence | | Small prototype | 2–4 | OpenMP packaging works; one-store serial and threaded replay agree | | Shared-store construction | 5–7 | Stable queries/IDs, safety, construction scaling and scratch measured | | Remaining evaluation kernels | 8–9 | Replay, derivative, retained closures and reductions validated | @@ -121,6 +150,7 @@ Expected RED/GREEN checkpoints (record real results; these are planned condition | Task | RED before the task's change | GREEN required | | --- | --- | --- | +| 0 | standalone target/self-tests and three variants absent | exact index results, honest scaling/memory evidence, owner decision; no library-parity claim | | 1 | exactness/shape schema and new driver tests fail | complete baseline/validator evidence; no parity claim | | 2 | missing Workshare/OpenMP exported dependency | helper and installed consumer pass | | 3 | options overload/config/failure tests fail | serial behavior preserved; failures join/abort promptly | @@ -137,39 +167,65 @@ Expected RED/GREEN checkpoints (record real results; these are planned condition ## Baseline-drift checkpoint and handoff -Before any authorized implementation, record HEAD, status and these EXISTING anchors. Line numbers identify this -snapshot; symbol names are the durable anchors. If source differs semantically, stop, report the changed contract and -refresh/approve the plan before coding. Missing PROPOSED symbols are expected RED, not drift. +Before authorized runtime work or implementation, record HEAD, status and these EXISTING anchors at source pin +90d57177c2b0cd93503f88dd931d2f8dd409aa23. Line numbers identify that snapshot; symbol names are durable anchors. The +current audit is complete. If later source differs semantically, report the changed contract and refresh/approve the +plan before coding. Missing PROPOSED symbols are expected RED, not drift. + +The audit compared the superseded pin with current main across routing/MPI, construction, Python API/tests, build/CI, +version and docs tooling. Packed storage/QueryWire, fixed32, streamed paired initialization, stable-gradient formulas, +benchmark families/windows, dependency manifests and justfile recipes retain their earlier contracts. Main's routing +basis, communicator agreement/cache, request ownership and Python incremental-axis changes are now baseline behavior, +not optimizations to implement or credit to OpenMP. VERSION is 0.9.2a1; record source/binary hashes, not just that +label. | EXISTING anchor | Must match before proceeding | | --- | --- | -| `cpp/monoprop/detail/mpi/Routing.h:177–260,315–326` | boolean linear/splitmix, default linear, unsupported geometry throws | -| `cpp/monoprop/detail/evolution/layer_build/Scan.h:208–253` | `FusedScanResult`, window, precomputed `gen_shift`, two self stages | -| `cpp/monoprop/detail/evolution/layer_build/Resolve.h:90–183` | `probe_incoming_queries(incoming,op,QueryForm)`, positions and hashes | -| `cpp/monoprop/detail/evolution/layer_build/Engine.h:486–565` | position/hash deferred publication and direct bounded self probe | -| `cpp/monoprop/detail/operator/OperatorIndex.h:282–288` | five-span `find_batch_positions`, optional hash output | -| `cpp/monoprop/TypeAliases.h:39`; `cpp/include/monoprop/Evolution.h:54–63` | uint32_t; derivative has `LayerAngle` and optional `CosRecordView` | -| `cpp/include/monoprop/MonomialPropagator.h:286–305` | virtual update/clone, nonvirtual apply helper | -| `justfile:75–122,169–171,371–397`; `pyproject.toml:162–168` | labels, centralized recipes, MPI env build override | +| `cpp/monoprop/detail/mpi/Routing.h:100–164,265–296` | full-rank basis draw, width-bound boolean router, default linear, Config and pure routes_pairwise predicate | +| `cpp/monoprop/detail/mpi/Pairwise.h:70–145`; `MPICompat.cpp:91–130` in the same directory | exact mode/partitions/seed agreement; communicator-attribute transport cache; request pre-sizing | +| `cpp/monoprop/detail/mpi/Comm.h:70–175`; `MPICompat.h:135–263` in the same directory | typed window iteration; sparse-plan validation; move-only PendingAlltoallv; WindowVec-only blocks; known-count width check | +| `cpp/monoprop/Evolution.cpp:102–117`; `cpp/monoprop/detail/mpi/Exchange.h:88–148` | rank-uniform routes_pairwise boolean, dense peer walk/nonempty legs, owned request vector | +| `cpp/monoprop/detail/mpi/HybridComm.h::derived_wire_plan_ / exchange_payload_` | fold every partition's send/receive peers; multi-peer fallback retains pairwise transport | +| `cpp/monoprop/detail/evolution/layer_build/Scan.h:208–249` | FusedScanResult has per-stream windows, no result.window; precomputed gen_shift and two self stages | +| `cpp/monoprop/detail/evolution/layer_build/Resolve.h:37–182` | sender_index plus window.slot, position/hash probing and ordered publication | +| `cpp/monoprop/detail/evolution/layer_build/Engine.h:348–372,470–574,600–612` | explicit plan/window constructor, position/hash deferred publication and bounded self probe | +| `cpp/monoprop/detail/operator/OperatorIndex.h:282–288` | five-span find_batch_positions, optional hash output | +| `cpp/monoprop/TypeAliases.h:39`; `cpp/include/monoprop/Evolution.h:54–63` | uint32_t; derivative has LayerAngle and optional CosRecordView | +| `cpp/monoprop/Evolution.cpp:437–464`; `cpp/monoprop/MPFunctions.cpp:44–172` | stable derivative snapshot/predivide/restore and record decisions | +| `cpp/include/monoprop/MonomialPropagator.h:288–306` | virtual update/clone, nonvirtual apply helper | +| `src/monoprop/monomial_propagator.py:231–270`; `tests/test_circuit.py:768–1008` | picture-correct incremental axis and post-call seed, both bases and shared-index gates | +| `justfile:75–122,169–171,371–397`; `pyproject.toml:162–168`; `cpp/tests/boost-test.cmake:1–9` | centralized recipes/MPI build override; CTest default 2;4, distinct from just's default 2 | | `benches/bench_random.py:22–139`; `benches/conftest.py:248–274,411–493` | four operation families, shape guard, existing op/outer windows | | `packages/monoprop-bench-tools/src/monoprop_bench_tools/memory/cpu.py:75–107,213–233` | nested peak preservation already implemented | +The updated CI action versions and docs package/lockfile are part of this source snapshot, not runtime-design changes. +Qiskit conversion also gained sparse-observable/single-Pauli and Pauli-product-rotation support: retain its existing +conversion tests and version-gated optional coverage; this OpenMP work neither rewrites conversion nor raises its floor. + Handoff checklist: exact revision/binary and overlay hashes; owner approvals; last passed task; actual RED and GREEN commands/results; remaining gate/capacity/placement blockers; fixed routing and workload manifest; raw artifact paths. Never hand off only “tests pass”. No builds/tests/imports/installs/VCS writes are authorized by this document refresh. -### Remote Task 1 entry point +### Remote entry point: Task 0, then conditional Task 1 -The baseline-only pilot belongs to Task 1. It is not a prerequisite that must run on the planning machine before handing -this plan to an implementer. Run it on the authorized execution host, currently c8a.metal-24xl. If the host differs, +The initial handoff is Task 0's standalone mini-app, not a full monoprop build or the baseline pilot. Transfer both +planning documents and CONTEXT.md; verify the source pin, authorize the small implementation and a finite execution +budget, and use the selected c8a.metal-24xl unless the owner approves another host. Return results and stop for the +architecture decision. No Task 1 command below runs until that decision and the corresponding document reconciliation. +No delegation, provisioning, commits or publication are implied by this handoff. + +After that gate, the baseline-only pilot belongs to Task 1. It need not run on the planning machine before handing this +plan to an implementer. Run it on the authorized execution host, currently c8a.metal-24xl. If the host differs, clarify the hardware decision before treating its sizing results as applicable to the target campaign. No remote access, provisioning or runtime execution is authorized merely by reading these documents. Transfer the latest spec, plan and CONTEXT.md with the source checkout. The historical planning commit alone does not include subsequent interview decisions. Verify document contents/digests on the execution host, including the 2-hour, 15-minute and 75% calibration limits. Documentation-only commits above the source anchor are not engine drift; inspect -semantic source/build/benchmark changes against the baseline-drift checkpoint rather than require HEAD to equal the old -planning commit. Keep the measured baseline source pinned to 290112c8289ab8015eb9a2c7651ac409839c5a88. +semantic source/build/benchmark changes against the baseline-drift checkpoint rather than require HEAD to equal a +planning commit. The refreshed source pin is 90d57177c2b0cd93503f88dd931d2f8dd409aa23. Build the baseline from that +engine, with separately recorded measurement overlays; do not auto-retarget to a later HEAD or reuse superseded-engine +measurements. The owner-requested current-main source refresh is complete; execution authorization is still required. A bounded starter authorization may cover only host/source preflight, documented project-local setup, an isolated baseline build, existing baseline API/benchmark sizing trials and their report. It does not authorize candidate engine @@ -195,10 +251,11 @@ Paths in task lists are relative to the repository root. | Area | Existing anchors | New files | | --- | --- | --- | +| Preliminary index study | `detail/operator/OperatorIndex.h`, `core/Monomial.h`; read-only reuse | `tools/index-miniapp/{CMakeLists.txt,main.cpp,README.md}`; no installed interface or engine changes | | Runtime configuration | `detail/EnvConfig.h`, constructor/copy in `MonomialPropagator.inl`; delete `detail/partition/CpuTopology.*`, do not move it | `cpp/monoprop/detail/parallel/{Options.h,Workshare.h,ThreadBudget.h,ThreadBudget.cpp,CMakeLists.txt}` (budget validation only; no hardware discovery) | | Replay | `Evolution.cpp`, `detail/evolution/CosineRecompute.h`, `MPFunctions.cpp` | no executor or new graph layer | | Construction | `detail/evolution/layer_build/{Scan,Resolve,Engine,FusedApply}.h` | range helpers stay beside current kernels | -| Storage | `detail/operator/{OperatorIndex,MPOperator,InvertedIndex}.h` | no replacement map | +| Storage | `detail/operator/{OperatorIndex,MPOperator,InvertedIndex}.h` | bounded Boost index variant only; no duplicate full-key store | | MPI failure | `detail/mpi/{Comm,MPICompat,MPIUtils,Exchange}.h`, `MPICompat.cpp` | `cpp/monoprop/detail/mpi/OperationFailure.h` | | Tests | existing suites listed per task | `cpp/tests/{openmp_workshare_tests,openmp_runtime_tests,openmp_kernel_tests,openmp_equivalence_tests}.cpp`, `tests/test_openmp_config.py`, `cpp/tests/mpi_failure_driver.cpp` | | Benchmark evidence | `benches/conftest.py`, both benchmark-tool renderers | `tools/benchmark-rank-local-openmp.py`, `tools/rank-local-openmp-workloads.json`, `tests/test_rank_local_openmp_benchmark.py` | @@ -230,13 +287,15 @@ must enter CMake FILE_SETs, not merely compiler include paths. is a different boundary. 9. Self derivative entries pair `k` with `k + count/2`, not `2*k` with `2*k+1`. Snapshot both required pre-update values before writing either endpoint. -10. Gate loops, reverse-gradient loops, repeated-parameter accumulation, graph append, store publication, and MPI remain - ordered on the caller. +10. Gate loops, reverse-gradient loops, repeated-parameter accumulation, graph append and publication phase boundaries + remain caller-ordered. Only the bounded index trial may parallelize hash publication within a phase; MPI stays on + its initializing thread in the final runtime. 11. Empty local stores still follow the same distributed collective sequence. Launch-time thread budgets may differ by rank without changing ownership; each object's captured budget is immutable. 12. Internal numerical kernels fall back to serial inside an existing OpenMP region. The supported host enters through - one controlling thread per rank, outside active host OpenMP teams, with no overlapping dependent-state or host MPI - calls. The fallback does not extend that calling contract. + one controlling caller per rank: MPI's initializing thread in MPI builds, outside active host OpenMP teams, with no + overlapping dependent-state or host MPI calls. Higher provided support does not relax this rule. The fallback does + not extend that calling contract. 13. A post-mutation failure invalidates the affected owner and its dependent functionals. Check validity before later state access or copying; independent earlier copies remain valid. Pre-mutation validation errors do not poison. @@ -301,6 +360,8 @@ Run only on an allocation supporting those ranks; smaller local runs are smoke t serial CTest fabric exclusions to real MPI/performance runs. `just bench-build-mpi` is the existing MPI benchmark build recipe; `just test-mpi` is the centralized full test recipe. Verify executable mtimes after sync; use the explicit `cmake --build` above before trusting C++ results (or update the recipe to guarantee relinking in its authorized task). +The CTest default is `2;4`, but `just test-mpi`/`just test-py-mpi` default to `2` unless given ranks or the environment +setting. Supply the intended matrix explicitly; a default two-rank run is not coverage for four-rank multi-peer replay. If direct CMake settings are used instead of the MPI environment switch, also supply `--config-settings-package='monoprop:build.requires=mpi4py>=4.1.0'`. `--all-extras` supplies runtime extras, not this isolated-build requirement. Use `--label-exclude mpi` for the non-MPI leg: the export probe is labeled `unit`, not @@ -342,8 +403,233 @@ stock-Python TSan gate, no blanket suppression of engine frames, and no sanitize --- +### Task 0: MPI-off index mini-app and architecture decision + +**Question:** For one process using T physical cores, how do per-thread stores, a shared single-writer index and a +shared concurrent index scale under monoprop-like batches? The owner explicitly permits the answer to overturn the +one-store-per-rank choice. This is an index/ownership experiment, NOT a propagation benchmark or production cutover. + +**Size boundary and files:** Create only `tools/index-miniapp/{CMakeLists.txt,main.cpp,README.md}`. One translation unit +contains the generator, three small adapters, driver and `--self-test`; split no framework out of it. Reuse +`cpp/monoprop/detail/operator/OperatorIndex.h`, `cpp/monoprop/core/Monomial.h` and their header-only dependencies +without editing them. Do not build/link/import monoprop, add Python bindings, use MPI/ShmComm, copy the engine, +simulate gates, coefficients/inverted indices/graphs, capture library traces, or implement a scheduler/container +framework. Aim for a few hundred lines of C++, not a second library. If it needs a production storage extraction or +substantial additional machinery, stop and simplify or request a narrower experiment. It must not depend on Tasks 1–3 +being implemented first. + +**PROPOSED interface:** `index-miniapp --self-test`, or one variant/case per fresh process: + +```bash +OMP_NUM_THREADS="$T" OMP_DYNAMIC=FALSE OMP_PLACES=cores OMP_PROC_BIND=close \ + "$BUILD/index-miniapp" --variant shared-serial --mode grow --rows 65536 \ + --batch 4096 --batches 16 --miss-percent 10 --key-length 6 --modes 128 --seed 0 +``` + +This is a proposed smoke-size invocation, not a frozen measurement size or an executed command. Supported variants are +`per-thread`, `shared-serial`, `shared-concurrent`; modes are `lookup` and `grow`; template widths are 128 and 1024; +key lengths are 6 (inline) and 16 (spilled), with inline width fixed at 11. Capture omp_get_max_threads once and record +the actual team. Optional `--profile` enables phase timers in a separate diagnostic run; normal runs time complete +batches only, leaving phase fields empty rather than reporting invented zeros. No library environment parser, global +OpenMP mutation or new public monoprop option. Reject malformed numbers, out-of-range percentages, impossible key/row +counts and checked-arithmetic overflow before allocation. Keep study sizes below the common one-store fixed32 ceiling; +report the architectural capacity difference without billion-row stress tests. A worker failure is caught inside its +region, all workers join, then the process reports failure; do not emit a success row. + +- [ ] Obtain separate approval for implementation, host/allocation and finite wall-time/RAM limits before execution. + Record source/header and executable hashes, compiler/options, Boost/OpenMP versions, complete commands and placement. + Task 0 neither consumes nor extends Task 1's two-hour pilot. Keep its CSV/logs outside the checkout and separate from + calibration/acceptance artifacts. No runtime measurements or setup are authorized by adding this task to the plan. +- [ ] Add the standalone self-test entry before implementing variants. RED is the absent target/adapter or incorrect + expected result, not a broken production test. Use ordinary runtime checks that remain active in Release, not assert + alone. The same driver executes tiny cases through all three variants at requested teams 1/2/3/4 on suitable hardware: + +| Tiny fixture | Required result | +| --- | --- | +| Initial A={0,1}; query B={2,3}, A, C={4,5}; publish misses | Hit mask false/true/false; shared IDs 1/0/2; final three exact keys | +| Next batch C,B,A | All hits; shared IDs 2/1/0, including with the concurrent index | +| Repeated existing A,A | Both queries resolve correctly and preserve their output ordinals | +| Repeated absent B,B in a distinct-miss publication fixture | Reject the invalid fixture; do not silently create duplicate rows | +| Empty input/store, all hits, all misses; 15/16/17 and 63/64/65 query tails | Correct sentinels, no dropped work and correct growth | +| Two distinct keys with equal cached 32-bit hash | Exact comparison distinguishes them before and after table growth | +| Spilled rows and a reallocation of their backing store | Earlier lookups still work; no dangling key/query views | + + Across shards compare decoded keys and owner/local-ID pairs, not global numeric IDs. Also compare each variant with + its own serial execution at fixed shard count, so scheduling cannot change IDs. Check every tiny result against a + simple serial key oracle; keep that oracle out of measured processes. Test runtime-limited teams, joined worker + exceptions and actual participation separately; fewer observed workers cannot be reported as T-core scaling. +- [ ] Add only this independent build configuration, enforcing the repository's compiler floors as well. It uses no + root add_subdirectory, binding generation or monoprop runtime dependency; Task 2 still owns production OpenMP setup: + +```cmake +cmake_minimum_required(VERSION 3.28...4.0) +project(index_miniapp LANGUAGES CXX) +if(CMAKE_CXX_COMPILER_ID STREQUAL "GNU" AND CMAKE_CXX_COMPILER_VERSION VERSION_LESS 14) + message(FATAL_ERROR "GCC 14 or newer is required") +elseif(CMAKE_CXX_COMPILER_ID MATCHES "Clang" AND CMAKE_CXX_COMPILER_VERSION VERSION_LESS 18) + message(FATAL_ERROR "Clang 18 or newer is required") +endif() +find_package(Boost 1.85 CONFIG REQUIRED) +find_package(OpenMP REQUIRED COMPONENTS CXX) +find_package(Threads REQUIRED) +add_executable(index-miniapp main.cpp) +target_compile_features(index-miniapp PRIVATE cxx_std_23) +target_include_directories(index-miniapp PRIVATE "${CMAKE_CURRENT_LIST_DIR}/../../cpp") +target_link_libraries(index-miniapp PRIVATE Boost::boost OpenMP::OpenMP_CXX Threads::Threads) +enable_testing() +add_test(NAME index-miniapp-self-test COMMAND index-miniapp --self-test) +``` + + Keep monoprop_ENABLE_MPI undefined; reject a build where it is defined. Confirm no MPI library is linked. Verify the + needed Boost APIs at the supported 1.85 floor, not only on the experiment host. If unavailable, report the concurrent + arm as unavailable and stop for a decision; do not silently raise the floor or substitute a different container. +- [ ] Generate identical deterministic global keys/query batches for every variant and T. Use an injective synthetic + encoding of logical key ordinals into sorted positions: divide the 2*NumModes positions into key_length disjoint + bands, encode ordinal digits in that radix, and apply a seed-derived cyclic offset within each band. Reject an + ordinal that does not fit; each band contributes one position. This avoids an O(N) key-deduplication table or trace + corpus. Stream initial rows and retain only one bounded query batch. For hits choose distinct earlier ordinals; for + misses use fresh ordinals, unique within each publication batch. Use floor(batch*miss_percent/100) misses, report the + actual count, and reject a requested distinct-hit count above the existing population. Mix order deterministically. + Repeated hits are a separate self-test, not an accidentally shrinking working set. In lookup mode misses stay absent; + in grow mode they become available to later batches. This structured synthetic family is NOT a claim about real + workload ratios or ownership skew; report those limits rather than adding synthetic physics or a trace service. +- [ ] Implement exactly these ownership variants using the same OpenMP harness. T means one team including its primary + thread; no multiple teams, per-NUMA shared stores, extra processes or fourth hybrid architecture in this task: + +| Variant | Required implementation | +| --- | --- | +| per-thread | T disjoint current OperatorIndex stores; owner is monomial_hash(key)%T, as in R=1 routing; each owner is a single writer | +| shared-serial | One current OperatorIndex; parallel disjoint find_batch_positions output ranges; caller-ordered missing-ID assignment, grow_rows_geometric/set_positions and bulk_insert_hashed | +| shared-concurrent | One packed row backing and one boost::concurrent_flat_map index using compact row handles/cached hashes; ordered IDs and caller row initialization, then parallel index publication | + + Reuse public row access on OperatorIndex for the concurrent adapter; call grow_rows_geometric/set_positions but never + populate its reference table or call reserve(), which would grow that unused table. Its constant empty 16-slot table + is permitted ONLY as measured mini-app overhead, not a second populated index or a production design. OperatorIndex + is noncopyable/nonmovable: keep stores in unique_ptr objects with stable addresses. Store row IDs, not raw row + pointers or query spans; transparent hash/equality consult that same frozen backing store and compare monomial + identity exactly. Retain cached hashes for publication. Never re-enter Boost from a visitor or + assume bulk visitors arrive in query order. A bounded per-query visitor is sufficient; no bulk-API tuning project. + Instantiate only the selected variant. Boost also changes table layout/probing, so its difference from the reference + is not a pure measurement of locking overhead; report its T=1 overhead as well as parallel scaling. +- [ ] Replay only the index phase: prepare one query batch outside timing, then route/bucket → frozen probe → ordered + missing-ID assignment → grow/fill packed rows → publish → return results in original query order → join. Hashing and + ownership calculation belong inside timing, not input preparation; retain/reuse hashes across probe/publication. + In lookup mode omit growth/publication, retaining hit/miss lookup work. Reference variants keep find_batch_positions + and its internal 16-query prefetch pipeline. For shared lookup use balanced contiguous query spans over the actual + team, including short spans/tails; use the same spans for Boost. Do not artificially cap participation at batch/256 + and mistake that chunking limit for map scalability. Small batches may have fewer active workers; record that. + Preserve stable per-owner query subsequences for the sharded arm. That arm may grow/fill/publish different stores in + parallel; shared rows/overflow remain caller-written even in the concurrent arm. Do not overlap reads/inserts or + allocate IDs by scheduling. Use native table growth; time/report any reserve. No free final-capacity + reservation for only one arm. Row initialization completes before concurrent map operations, and joins precede any + row growth/relocation or dependent reads. +- [ ] Model owner bucketing with bounded flat arrays of ordinals/offsets and result scatter, not a copy of ShmComm. + Parallel source chunks may use O(T*T) counts/offsets plus O(batch) records; use ordered prefixes, not one serial + dispatcher doing all work or a T-fold copy of the keys. No producer-consumer queues or custom barriers. Include this + overhead in complete-batch timing, and report index-only probe/publication timings separately. Label bucketing as a + simplified shared-memory model; neither its cost nor its omission from kernel timing predicts legacy transport time. +- [ ] Check results outside timing without a second resident operator: every declared hit/miss, returned row's exact + key, expected owner, final count and insertion order. A final streaming lookup of every expected key plus the exact + row count checks the retained set without an O(N) shadow map. Checksums may prevent dead-code elimination and help + compare runs, but are not the correctness oracle. A timed run with incorrect results is invalid, never fast evidence. + Run separate ASan/UBSan checks and a qualified OpenMP race check on tiny cases; unsupported tooling stays a reported + limitation. Do not use sanitizer timings for scaling or expand into the whole library's test matrix. +- [ ] Start with a compact strong-scaling study: fixed GLOBAL initial rows and total queries across T, never N rows per + thread. At modes=128/key-length=6/batch=4096/seed=0, compare lookup at 0% and 50% misses, and growth at 10% and 100% + misses, using one small and one memory-resident size chosen before comparative timing. Verify the larger footprint + exceeds the relevant last-level cache if claiming DRAM scaling. Choose batches to exercise that footprint, and record + initial/final sizes. Sweep 1/2/4/... physical cores plus the exact domain and allocation endpoints, deduplicated. + Repeat within one domain and across the allocated machine; a one-domain host needs no duplicate series. Use the same + core sets and memory policy for all three variants at a given T. The full-machine MPI-off series is required for this + decision, even across NUMA domains; do not replace it with an MPI run or downgrade it to optional L2a diagnostics. +- [ ] Add only bounded sensitivity checks at T=1, domain-full and machine-full: the same mixed-growth case with + length-16 spilled rows, one with modes=1024 (wider PosT), and one batch=64 case. Run all three variants on the same + inputs; + do not expand every combination into a Cartesian product. If conclusions depend on these cases or are noisy, report + that and request targeted follow-up rather than adding containers, weak-scaling campaigns or a trace subsystem. +- [ ] Verify active worker masks against physical-core/NUMA topology using external tools and OpenMP diagnostics. + Record first-touch behavior: per-owner construction naturally places shard pages differently from caller-built + shared rows. Do not mistake NUMA placement effects for hash-table locking. A matched externally interleaved-memory + rerun may diagnose a reversal, only within the approved budget; never apply it silently to one variant. No internal + affinity setter, hwloc dependency or automatic geometry repair. Unknown placement makes scaling evidence incomplete. +- [ ] Emit a plain CSV row per process with full knobs, requested/actual team sizes, initial/final rows, hits/misses, + complete-batch and optional phase seconds, queries/second, min/max shard rows, row/spill/index byte estimates where + available, and validation outcome. Report setup separately. Measure complete batches without fine-grained timers, + then collect --profile diagnostics in separate matching processes to avoid timer overhead bias. Label per-owner + phase sums/maxima as nonadditive diagnostics when owners overlap; do not present their sum as elapsed batch time. + Profiles also record active-worker counts from the actual probe/publication bodies, not merely team creation. + Use three fresh-process observations per variant/cell with rotated order; retain every sample, median and range. + Report speedup versus each arm's T=1 and time/memory ratios versus per-thread at the SAME T; do not select from + throughput alone or hide losing cases. + Capture whole-process peak RSS with an external tool in fresh processes, including setup/growth. Input generation + and checking use bounded scratch, not a large reference map that masks index peaks. RSS includes allocator/runtime + overhead; current memory_bytes/index_estimated_memory_bytes are estimates, not exact allocations or Task 1 windows. + Archive compiler/runtime/placement metadata beside CSV rather than building a report schema/framework into the app. + +**Planned standalone smoke commands, only after authorization:** BUILD is an absolute, isolated build directory; the +selected allocation must support the requested workers. No uv sync, monoprop imports or engine build is needed: + +```bash +cmake -S tools/index-miniapp -B "$BUILD" -DCMAKE_BUILD_TYPE=Release +cmake --build "$BUILD" --parallel +OMP_NUM_THREADS=1 ctest --test-dir "$BUILD" --output-on-failure --no-tests=error +for T in 1 2 3 4; do + OMP_NUM_THREADS="$T" OMP_DYNAMIC=FALSE "$BUILD/index-miniapp" --self-test +done +``` + +**Gate and handoff:** Deliver the three-file mini-app, self-test evidence, raw CSV/peak-RSS/placement logs and a short +comparison with phase bottlenecks, uncertainty and omitted costs. Distinguish an ownership-model result from a measured +current-library baseline. Do not claim propagation, replay, gradients, MPI or full-library parity from this study. +If an arm is unavailable, placement is unverifiable, correctness fails or the budget ends, report incomplete evidence; +never declare the remaining arm the winner by default. Stop for the owner's decision to retain a shared-store path, +retain/reconsider sharding, or gather specified additional evidence. Record that decision and revise the spec and +Tasks 1–13 accordingly BEFORE proceeding, including MPI-off performance requirements. No fixed mini-app speedup +threshold replaces that decision or weakens the five full-library gates. Its outputs are never formal Task 1 samples. + ### Task 1: Freeze baselines and repair measurement trustworthiness +**Prerequisite:** Task 0 results have been reviewed by the owner, the architecture selected and both documents +reconciled. The following remains the conditional one-store campaign, not an automatic continuation of the mini-app. + +**Benchmark roadmap: use [benches/LADDER.md](../../../benches/LADDER.md), not a new suite.** N is nodes, C is usable +physical cores/node, R is ranks/node, and the ladder's P is old partitions/rank (not the generic process-count P used +elsewhere in this plan). Map old P to candidate T workers/rank. On the single target instance N=1: + +| Rung/control | Candidate geometry | Classification | +| --- | --- | --- | +| L1 | R=1, T=1 | Required small/medium single-thread control | +| NUMA-local shared-memory control | R=1, T=C_dom | Required small/medium; not canonical full-node L2a | +| MPI-only control | R=D, T=1 | Required small/medium, with fewer active cores than full-node L2b | +| L2b | R=D, T=C_dom | Required one-rank/domain production shape | +| L2b variant | R=2D, T=floor(C_dom/2) | Required two-ranks/domain shape when C_dom>=2 | +| L2a | R=1, T=C | Full-node diagnostic; cross-NUMA allocation expressly allowed | +| L3/L4 | N>1 | Pending proper-interconnect HPC qualification | + +D is allocated NUMA domains, C_dom the selected cores/domain, C=D*C_dom. Record actual balanced subsets and unused cores +rather than infer topology. If D=1 makes L2a identical to required L2b, deduplicate the shape +without downgrading its required status. Large acceptance profiles use the two L2b decompositions only. A corresponding +L2a diagnostic must retain the same workload flags even if it cannot fit; report that limitation instead of quietly +shrinking it. The NUMA-local control cannot substitute for full-node L2a when interpreting L2a/L2b differences. + +The practical workflow is baseline-only ladder sizing → owner-approved frozen cells/flags → the same pytest nodes on +both arms at matched shapes → existing reports plus the five parity metrics and numerical evidence below. Prefer the +ladder's Hubbard/Pauli lower_atol, random-Heisenberg obs_terms and random-Schrödinger num_generators sizing controls; +record seed/structure and the resulting gradient length. These pre-freeze workload choices never relax oracle +comparison tolerances or permit post-regression retuning. Size graph rows using the largest build/energy/gradient peak, +including export/validation resources, within the already agreed pilot limits. + +The ladder recommends grouped graph runs to save setup. Use that for pilot/diagnostic work where fixtures actually +share a graph; audit the fixtures rather than assume all models share build output. Formal per-operation observations +still use fresh processes and --bench-rounds=1 to avoid prior live graphs and fixture order affecting their peaks. Their +separate construction worker measures setup explicitly. Reuse the existing functions, builders and report files; +observe/validate/compare only supply the missing evidence joins/checks, not another benchmark framework. + +Keep cross-NUMA L2a in a separately labelled diagnostic inventory/output directory, outside the frozen required-cell +manifest supplied to compare. Never reclassify a failed required cell as diagnostic or use L2a ratios to waive it. +Numerical mismatches in diagnostic runs still require investigation. Historical ladder results are sizing examples, +not target-machine acceptance evidence. Update its shape/declaration guidance with the candidate cutover. + **Files:** - Modify: `benches/conftest.py` (exactness and shared shape preflight); preserve existing windows and timed operations in `benches/{bench_random,bench_models}.py`. No historical node renaming or restoration of removed pytest nodes. @@ -381,10 +667,11 @@ numactl --hardware grep -E 'Cpus_allowed_list|Mems_allowed_list' /proc/self/status ``` -Use these observations and per-rank placement evidence to select NUMA-contained geometries; a capacity-based 1x96 shape -is not automatically admissible. Missing diagnostic tools require another external observation method, not a new -library topology dependency. Record hardware SMT state, but configure only one worker per physical core. Do not add -SMT-sibling worker profiles; production-HPC and multi-node qualification remain separate and pending. +Use these observations and per-rank placement evidence to select NUMA-contained acceptance geometries. A 1x96 shape is +canonical L2a only with all 96 allocated physical cores; if it crosses domains, label it diagnostic-only. Missing +diagnostic tools require another external observation method, not a library topology dependency. Record hardware SMT +state, but configure one worker per physical core. Do not add SMT-sibling profiles; HPC/multi-node qualification remains +separate and pending. - [ ] Obtain separate authorization for baseline build/calibration work. On c8a.metal-24xl, record sockets/NUMA domains, physical cores, hardware SMT state, memory allocation and software stack before choosing geometries. Agree @@ -397,9 +684,10 @@ SMT-sibling worker profiles; production-HPC and multi-node qualification remain revision, allocation, compiler/flags, MPI version, index width, OpenMP runtime, allocator environment, imported extension path/hash, seed and complete workload configuration before collecting evidence. - [ ] Before changing engine code, separately build the baseline using R/M. Preserve its environment and binary. Apply - only the identical measurement-harness changes to both arms; record this overlay's diff/hash. Baseline numerical - failures stop this task. After the freeze, any required baseline failure blocks collection, not permission to resize - or remove the cell. + only the identical measurement-harness changes to both arms; record this overlay's diff/hash. Use the refreshed + main pin for the baseline and the candidate's starting engine, not the former routing-branch snapshot. Main's basis + redraw, transport and parameter-axis changes must not be attributed to OpenMP. Baseline numerical failures stop this + task. After the freeze, any required baseline failure blocks collection, not permission to resize or remove the cell. - [ ] Once authorized, run unscored baseline-only feasibility trials before freezing workload sizes. Use existing baseline APIs/benchmark entrypoints; add no fourth driver mode or general calibration framework. Check elapsed time and memory for construction, evaluation and numerical export/validation, including their temporary storage. @@ -726,21 +1014,21 @@ configurations in artifacts, not just a profile label. `monoprop_BENCH_ALLOW_BIG_GRAPH=1` only on adequately provisioned hardware, or obtain approval for a fixed lower `--hubbard-trotter-steps` profile on both arms before formal baseline collection. A skipped cell is not a pass. Do not silently omit graph-heavy cells or bypass the guard on an unverified allocation. -- [ ] Select geometries from observed physical cores, not vCPU/SMT-sibling counts. For D allocated NUMA domains with - C selected physical cores each, the small/medium sweep is `(R,T)=(1,1),(1,C),(D,1),(D,C)`, deduplicated. R=1 uses - only one domain. Add `(2*D,floor(C/2))` with non-overlapping physical-core subsets when C>=2; record unused cores. - Apply existing routing coverage requirements too. Selected large profiles use only `(D,C)` and `(2*D,floor(C/2))` - from the full-node allocation, not the one-worker/partial-node sweep. Freeze the exact allowed cells per profile; - do not reclassify profiles or drop cells after a regression. If domains/allocation are uneven, agree explicit balanced - subsets before freezing rather than assume C=96/D. Each baseline/candidate pair has identical R/T/core allocation. - All these cells are single-instance; network-HPC qualification stays pending. Old throughput uses R ranks x T - partitions; retain an old single-store serial diagnostic but never compare only against forced-serial old code. - -- [ ] Reuse `benches/LADDER.md` L1–L4 labels/provenance and term-count fields as historical calibration, never - acceptance evidence from old runs. Where a full-node single rank crosses NUMA domains, do not copy that historical - allocation: freeze a within-one-domain profile, identical on both arms, before formal baseline collection. Preserve - historical nodes and fields while adding runtime-shape/observed-team metadata. Term counts must agree for the - identical operation. +- [ ] Select required geometries from observed physical cores, not vCPU/SMT-sibling counts. For D allocated domains with + C_dom cores each, use `(R,T)=(1,1),(1,C_dom),(D,1),(D,C_dom)`, deduplicated, for small/medium profiles. Required R=1 + controls use one domain. Add `(2*D,floor(C_dom/2))` with disjoint cores when C_dom>=2; record unused cores. Apply + existing routing coverage requirements too. Selected large acceptance profiles use only `(D,C_dom)` and + `(2*D,floor(C_dom/2))` from the full-node allocation. Freeze allowed cells per profile; + do not reclassify profiles or drop cells after a regression. Uneven domains require approved balanced subsets, not + assumed C_dom=96/D. Each old/new pair has identical R/T/core allocation. Old throughput uses R ranks x T partitions; + retain an old single-store serial control but never compare only against forced-serial old code. +- [ ] Run canonical full-node L2a separately as a labelled diagnostic using the same approved model flags at R=1,T=C. + It may cross NUMA domains; verify its complete allocation and one worker/core rather than asserting containment. + Use separately budgeted existing-suite runs/reports, not required-cell artifacts or an extra driver mode. Do not + substitute NUMA-local R=1,T=C_dom and still call it L2a. Record failed/unavailable diagnostics honestly; no diagnostic + ratio is a parity gate or waiver. Required L2b geometry remains NUMA-contained; L3/L4 remain pending. Preserve the + existing node IDs/term-count fields and add runtime-shape/actual-team metadata. Counts must agree, but the global + term-map/value checks below remain necessary. - [ ] Use `just capture-baseline` / `just diff-baseline` and `tools/capture-baseline.py --compare` only as supplementary small-fixture regression evidence. Its keys include rank, tolerance differs and it lacks operation gradients/provenance; it cannot replace the ownership-aware global-map comparator or independent @@ -850,7 +1138,9 @@ clean consumer finds/links/runs with no manual flags or source includes. No engi - Modify: `cpp/monoprop/detail/monomial_propagator/MonomialPropagator.inl`, `cpp/monoprop/{Evolution,MPFunctions}.cpp`. - Modify: `cpp/monoprop/detail/evolution/{CosineRecompute,CosineRecomputeCallbacks}.h`, `cpp/monoprop/detail/evolution/layer_build/{Scan,Resolve,Engine,FusedApply}.h`. -- Modify: `cpp/monoprop/detail/mpi/MPICompat.cpp`; the final environment/configuration cleanup is in Task 10. +- Modify: `cpp/monoprop/detail/mpi/MPICompat.{h,cpp}`; audit request/collective boundaries in `Exchange.h` and + `Pairwise.h`. + Final environment cleanup is Task 10, MPI-level cutover Task 12. - Create/test: `cpp/tests/openmp_runtime_tests.cpp`, `cpp/tests/mpi_failure_driver.cpp`, `tests/test_openmp_config.py`; modify `cpp/tests/CMakeLists.txt` to exclude the failure-driver main from the unit-runner glob and build a separate executable. @@ -888,7 +1178,7 @@ auto state_operator_derivative_local(VecD&, VecD&, const MPGraphView&, size_t, ``` PROPOSED options follow `record`, not `angle` or `cos_acc`; preserve `LayerAngle{.gen_coeff=g,.param=theta}` and record forwarding. `MPFunctions.cpp:44–172` records pre-layer values based on accumulated amplification AND vanishing-cosine -criteria; `Evolution.cpp:450–477` snapshots self endpoints, predivides, accumulates, restores and combines +criteria; `Evolution.cpp:437–464` snapshots self endpoints, predivides, accumulates, restores and combines pre-layer-unit sums. Keep record append/predivide/restore serial because duplicate indices are allowed. Do not replace this stable protocol with a division-only reverse pass. @@ -945,7 +1235,11 @@ stay in the build tree and are not committed. Never hand-edit generated bindings poison/abort paths until Task 12. - [ ] Guard every newly throwable distributed phase, not merely the outermost function. Especially catch a worker failure in overlapped cosine work **inside the lifetime of an active exchange ticket** and abort before stack - unwinding can wait on peers that never posted. Normal successful paths still explicitly wait. Do not rely on a ticket + unwinding can wait on peers that never posted. This covers BOTH replay `Exchange.h::Ticket` and construction + `MPICompat.h::PendingAlltoallv`: both are already move-only/self-draining; move assignment drains the old destination. + Preserve unified request-vector ownership, moved-from posted=0 and pre-sized request storage. Normal successful paths + still explicitly wait. Extend the active-ticket driver scenario to both handle types; source lifetime tests alone do + not establish coherent distributed failure. Do not rely on a ticket destructor to coordinate distributed failure; do not add a per-kernel allreduce to agree on allocation failure. Wrap constructor/build/replay/evaluation phases that can strand another rank, including retained-callable entry points. Single-rank failures preserve the original exception type. Add private `invalid_`, `invalidate_()` and @@ -965,21 +1259,41 @@ stay in the build tree and are not committed. Never hand-edit generated bindings copy/clone reject the invalid owner. An independent earlier copy still evaluates. A validation-only failure leaves the original usable. RED: reuse currently reaches state; GREEN: entry guards reject before further work. Injection remains test-only and no production arbitrary-callback API is added. -- [ ] Validate already initialized MPI with `MPI_Query_thread`, not only monoprop-owned initialization. Reject a level - below `MPI_THREAD_SERIALIZED` with an actionable diagnostic before distributed work. MPI worker calls are forbidden; - low-level numerical kernels may serialize under a nested OpenMP call, but distributed public operations from an - external OpenMP team are unsupported and must reject before communication. The host contract is one controlling - caller per rank, no overlapping use of one propagator/dependent functionals and no overlapping host MPI calls. - Do not add a process-wide locking service or nested-parallel execution machinery. -- [ ] Add a separate failure driver with command cases `worker-throw`, `before-exchange`, `active-ticket`, - `insufficient-thread-level`. On rank zero inject the failure; peers execute the normal phase. Run from a subprocess - with timeout 30s under two ranks. Expected outcome is prompt nonzero failure, **not** timeout or a successful exit. Do - not put intentional MPI_Abort cases inside the ordinary suite executable. Test single-rank worker throws separately. +- [ ] Validate host-owned and library-owned MPI with MPI_Query_thread. During coexistence, keep the required/requested + MPI_THREAD_SERIALIZED level: Hybrid still makes MPI calls from non-initializing workers. Task 12 lowers both checks + to MPI_THREAD_FUNNELED only after removing that path. Do not reinitialize host-owned MPI or assume the provided level + equals the requested one. Higher levels are valid; no speedup is presumed from requesting a lower minimum. +- [ ] Add an internal `mpi::require_initializing_thread()` guard in MPICompat, using MPI_Is_thread_main after MPI is + initialized. Apply it before MPI-using constructor/operation/retained-functional/cleanup work on the new ordinary-MPI + one-store path, including implicit communicator work before constructor bodies. Do not impose it on still-supported + legacy Shm/Hybrid worker dispatch. MPI-off is a no-op; public operations still enter outside active host OpenMP teams. + Keep `mpi::routes_pairwise` behind the caller guard on the ordinary-MPI path: first use may allreduce, warm use + still calls MPI_Comm_get_attr, and a new/duplicated communicator must agree independently. Preserve the + communicator-attribute + cache and Hybrid's constructor agreement; do not replace them with per-gate agreement or a raw-handle global map. + At Task 10 every new propagator uses this contract, even if provided support is SERIALIZED or MULTIPLE. Do not capture + whichever thread first touched an object as a substitute for MPI's actual initializing thread. +- [ ] On wrong-thread entry, print an actionable initializing-thread diagnostic and fail fast locally without calling + MPI_Abort, communicator queries, collectives or the normal operation_failed helper from that thread. This unsupported + host-contract violation is distinct from valid-caller distributed failures. Test it under a launcher configured to + terminate peers on a process's abnormal exit; do not promise peer cleanup under an arbitrary launcher policy. Add no + process-wide lock, dispatch service or nested-execution machinery. Preserve no-overlap rules for dependent state and + host MPI calls; an internal helper's nested fallback does not widen the public contract. +- [ ] Add a separate failure driver with `worker-throw`, `before-exchange`, `active-ticket`, `insufficient-thread-level` + and `wrong-thread-entry`. Inject on rank zero while peers follow the normal phase. For wrong-thread-entry, invoke a + real guarded entry from a host std::thread (outside OpenMP) after main-thread MPI initialization, at supplied + SERIALIZED/MULTIPLE now; add FUNNELED in Task 12 after lowering the minimum. Never relax the caller rule. + Use a 30s subprocess timeout under two ranks, with launcher abort-on-bad-exit enabled for local-fatal cases. Require + prompt nonzero failure with the expected diagnostic, not timeout. Keep abort cases outside the ordinary unit runner. + Test all required/provided-level decisions in a pure validator; real initialization tests record the actual provided + value rather than assuming a request guarantees an insufficient level. Test single-rank worker throws separately. **Gate:** R/M compile and pass with kernels still serial. Focused gates include `combined_recompute_equivalence.cpp` record restoration and `tests/test_deep_circuit_gradient.py` deep non-singular, vanishing-cosine and no-record cases, -before and after threading. Options reach every intended entry point; runtime global settings unchanged; failure tests -do not hang. Public/exported signature ABI change is recorded for a coordinated library/bindings rebuild. +before and after threading. Preserve `tests/test_circuit.py` incremental-axis/seed behavior too; its existing passing +cases are baseline preservation evidence, not RED for a feature already shipped on main. Options reach every intended +entry point; runtime global settings unchanged; failure tests do not hang. Public/exported signature ABI change is +recorded for a coordinated library/bindings rebuild. ### Task 4: Prototype cosine and fused updates without structural concurrency @@ -1053,10 +1367,13 @@ their initial serial path is conservative, not evidence of unsafety. Do not add std::function alone proves safety. EXISTING `Scan.h::fused_find_and_collect` returns `FusedScanResult` and takes `SlotWindow window`, `my_rank`, -router, precomputed `gen_shift`, then capture/scaling arguments. `Engine.h` derives the plan/window once from that -shift. PROPOSED: extract the following range helper in `Scan.h`; preserve these inputs and result type, prepare lazy -data once, and merge private results. Do not reintroduce rank-count-shaped query vectors. The range helper must never -call a lazy accessor itself: +router, precomputed `gen_shift`, then capture/scaling arguments. `Engine.h::build_layer` derives the plan/window once +from that shift and passes BOTH explicitly to LayerBuildEngine after the sink argument. Keep that constructor contract +in direct-engine fixtures; it no longer derives geometry itself or supplies a default plan. FusedScanResult has no +separate window member: compare each populated stream's window(), with value streams empty when capture is disabled. +PROPOSED: extract the following range helper in Scan.h; preserve these inputs/result type, prepare lazy data once, and +merge private results. Do not reintroduce rank-count-shaped query vectors. The range helper must never call a lazy +accessor itself: ```cpp template @@ -1095,7 +1412,7 @@ prepared-state cache. Tail masking uses the full index's final word, not each ra `for_blocks(range_count,options,...)` fills one `FusedScanResult` per range. No shared push_back/critical section and no per-bitmap-block × rank vector table. - [ ] Join, calculate checked total lengths, reserve once and merge reachable slots in ascending range ID using - `mpi::WindowIndex{k}` or `.at_slot(flat_slot)`; never use a flat slot as a window index. All pieces share the exact + `window.indices()` and `.at_slot(flat_slot)`; never use a flat slot as a window index. All pieces share the exact `window` (base and count). Move the first useful buffer where safe and release drained buffers immediately. `leader_queries/src/val` and follower equivalents preserve each stream's source-order subsequence. Wire records have variable boundaries, not a fixed stride; use `QueryWire` and preserve Plain/Fused semantics. Self wire stays empty. @@ -1109,11 +1426,10 @@ prepared-state cache. Tail masking uses the full index's final word, not each ra ```cpp // PROPOSED merge loop shape; append concrete streams as specified above. -for (size_t k = 0; k < window.count; ++k) { - const mpi::WindowIndex wi{k}; +for (const auto wi : window.indices()) { const auto flat_slot = window.slot(wi); for (size_t range = 0; range < pieces.size(); ++range) { - // pieces[range].leader_queries[wi] belongs to flat_slot, not flat slot k. + // pieces[range].leader_queries[wi] belongs to flat_slot, not flat slot wi.value. // Append leader src/val in the same range order; handle followers separately. // If flat_slot==my_rank, merge self positions, never synthesize self wire. } @@ -1179,8 +1495,8 @@ BOOST_AUTO_TEST_CASE(openmp_incoming_miss_order) { detail::QueryWire<8>::push(incoming.at_slot(6), positions_of<8>(c), +1); const auto pr = detail::probe_incoming_queries<8>( incoming, op, detail::QueryForm::Plain, {.threads = threads}); - BOOST_TEST(pr.sender_slot(0) == 4U); - BOOST_TEST(pr.sender_slot(1) == 6U); + BOOST_TEST(pr.window.slot(pr.sender_index(0)) == 4U); + BOOST_TEST(pr.window.slot(pr.sender_index(1)) == 6U); const std::vector offsets{0, 1, 1, 3}, ids{1, 0, 2}, misses{0, 2}; BOOST_TEST(pr.base == 1U); BOOST_CHECK_EQUAL_COLLECTIONS(pr.goff.begin(), pr.goff.end(), offsets.begin(), offsets.end()); @@ -1245,8 +1561,8 @@ rows, and a prebuilt inverted index. publication. **Gate:** R/M lookup and fresh-insert tests pass with exact fixed-geometry IDs. Report -probe/prefix/publication/inverted-index times separately. If serial hash publication dominates, report an Amdahl limit -instead of inventing a concurrent table. +probe/prefix/publication/inverted-index times separately. If serial hash publication dominates, report its Amdahl limit +and carry the evidence into the bounded Task 11 Boost trial. Do not improvise a custom concurrent table here. ### Task 7: Harden rank-wide capacity and optionally parallelize append-only row fill @@ -1309,7 +1625,7 @@ auto reserve_index_for_size(size_t final_size) -> void; unaligned rows. **Gate:** Capacity guards pass R/M. Optional fill must improve measured bottleneck without peak regression. Hash -insertion, overflow mutation, packed graph phases and inverted append remain serial. +insertion, overflow mutation, packed graph phases and inverted append remain serial at this stage. ### Task 8: Parallel replay endpoints and derivative pairs @@ -1324,7 +1640,8 @@ proposed transient metadata, not a new graph field. Flatten **ranges within acti underutilizes sparse linear-routing peers. - [ ] Write serial/threaded forward and derivative comparisons with multiple self pairs, nonadjacent row IDs, - remote+self slots, empty peers, zero-angle, deep non-singular circuits, vanishing-cosine and no-record paths. Retain + remote+self slots, multiple remote peers, empty peers, zero-angle, deep non-singular circuits, vanishing-cosine and + no-record paths. Retain exact recorded restoration, including duplicate indices, indices outside the cosine set and cos rounding to one. Compare energy/gradient to current finite-difference/exact oracles. Assert even self endpoint count and unique output ownership. @@ -1344,10 +1661,16 @@ underutilizes sparse linear-routing peers. - [ ] Keep layer loops and gradient reverse order unchanged. No worker may call `mpi::rank`, `router_for`, exchange begin/wait, or allreduce. Query rank/layout on caller. - [ ] Cover worker exceptions after exchange posting using Task 3 active-ticket failure case. Buffers cannot resize or - die while requests are outstanding. Preserve EXISTING replay `wire_bits` selection: plain MPI uses dense - PeerPlan/nonzero pairwise legs, HybridComm derives its plan from all local partition rows. Do not introduce a new - generator-derived replay plan. Run `flat_exchange_tests.cpp` dense, pairwise, dense-layout-on-pairwise and - empty-layout cases after every transport edit. + die while requests are outstanding. Preserve EXISTING `mpi::routes_pairwise(comm)` selection and the boolean + `post_flat_alltoallv(..., pairwise)` argument. Plain MPI walks a dense PeerPlan and posts only nonempty legs; + `sparse_pairwise` sizes requests in its own pre-pass, with no active-leg estimate argument. Hybrid folds send AND + receive summaries across every partition. Several peers or incompatible summaries retain every peer on the pairwise + arm; that local fallback must NEVER switch transport to MPI_Alltoallv. Empty layouts post nothing on that same arm. + Keep communicator-wide agreement/cache on the caller, not workers or a per-layer layout predicate. Do not introduce a + new generator-derived replay plan. Run `flat_exchange_tests.cpp` dense, pairwise, dense-layout-on-pairwise and + empty-layout cases after every transport edit, plus + `hybrid_comm_derived_wire_plan_reads_every_partitions_row` and + `hybrid_comm_derived_wire_plan_falls_back_to_every_peer` during coexistence. The latter needs at least four ranks. - [ ] Keep `LayerAngle`, `CosRecordView` and `LayerCosIndices` intact. Serial forward record append, predivide/restore and snapshot-before-restore ordering are mandatory even if record ordinals differ: index duplication is legal. @@ -1396,7 +1719,9 @@ through `EvalRequest.parallel` under the approved numerical policy; implementati - [ ] Run `tests/test_deep_circuit_gradient.py` and `combined_recompute_equivalence.cpp` across budgets 1/2/3/4, including deep non-singular amplification, vanishing cosine, no-record and duplicate-record restoration cases. - [ ] Run retained/direct evaluation, paring, partial contraction, repeated parameter mapping, both pictures/bases, - fixed-rank deterministic repeats, and near-cutoff retained-key comparisons. Cross-rank-count sums need not be bitwise + fixed-rank deterministic repeats, and near-cutoff retained-key comparisons. Include the incremental-axis and + post-call-seed tests from `tests/test_circuit.py`: gradients must use the equivalent composed circuit's parameter + order, including multi-monomial gates sharing one index. Cross-rank-count sums need not be bitwise equal; don't widen tolerances to pass them. **Gate:** Existing serial API guarantees remain intact; new numerical association is documented and deterministic by @@ -1410,9 +1735,10 @@ alone is insufficient. A key-set change blocks the gate. Stop and report, never `cpp/monoprop/detail/EnvConfig.h`, `cpp/monoprop/detail/parallel/{ThreadBudget.h,ThreadBudget.cpp}`. - Modify: `src/monoprop/bindings/binder.h`, `src/monoprop/monomial_propagator.py` docstrings. - Create/extend: `cpp/tests/openmp_equivalence_tests.cpp`, `tests/test_openmp_config.py`, - `cpp/tests/{env_config_tests,routing_tests,mpi_distributed_layer_equivalence}.cpp`, + `cpp/tests/{env_config_tests,routing_tests,mpi_utils_tests,mpi_distributed_layer_equivalence}.cpp`, `cpp/tests/{mpi_fresh_insert_equivalence,unit_tests,ctor_validation_tests,hybrid_comm_tests}.cpp`, - `cpp/tests/link_export_probe/link_export_probe.cpp`, `tests/{test_monoprop_smoke,test_parameter_validation}.py`. + `cpp/tests/link_export_probe/link_export_probe.cpp`, + `tests/{test_monoprop_smoke,test_parameter_validation,test_circuit}.py`. - Migrate now: `cpp/tests/{partition_equivalence_tests,partition_group_clone_tests}.cpp`; preserve useful coverage in the OpenMP equivalence suite and `cpp/tests/{simulator_copy_tests,update_initial_operator}.cpp` before retiring obsolete facade/factory assertions. Update `cpp/tests/README.md`; do not defer broken constructor users to Task 12. @@ -1454,9 +1780,13 @@ this is not a budget change. There is no topology-based fallback or special mult a supported hot-configuration API. Full-budget matrix tests use separate launches, not mid-calculation changes. Remove prototype partitions=1 arguments; public tests use environment configuration, internal kernels use Options. - [ ] Apply the approved splitmix physical ownership change: `floor((hash % (R*S))/S)` becomes `hash % R`. - Linear rank mapping remains unchanged; S=1 removes only the partition hash. Preserve boolean mode/seed, - `linear_requested()`, `linear_bits_for(R)`, `make_router(R,1)`, generator `rank_shift` and `dest_from_shift` peer - selection. No virtual shards. Exports/load/reduction order can change under splitmix; validate global term/value maps + Linear rank mapping remains unchanged relative to the REFRESHED main baseline; S=1 removes only the partition hash. + Preserve boolean mode/seed, `linear_requested()`, `make_router(R,1)`, router.linear_bits(), generator + rank_shift and dest_from_shift peer selection. Keep the deterministic full-rank basis draw (up to 1024 attempts) and + width bound log2(R)<=min(2*NumModes,64), where NumModes is the template width, not necessarily logical_num_modes_. + The old transposed-plane/geometry-bit helper is gone; do not restore it. The pure routing::routes_pairwise predicate + chooses transport for an agreed Config; its false result does not waive Router's invalid-geometry rejection. + No virtual shards. Exports/load/reduction order can change under splitmix; validate global term/value maps and resulting per-rank peaks at equal R/T/routing settings. Candidate ownership never depends on T. - [ ] Extend `routing_tests.cpp`, `mpi_distributed_layer_equivalence.cpp` and `mpi_fresh_insert_equivalence.cpp`: default linear success at R=1/2/4; explicit splitmix success at R=1/2/3/4. Record `monoprop_ROUTING`, @@ -1464,7 +1794,25 @@ this is not a budget change. There is no topology-based fallback or special mult `Router::for_modes<8>(3,1,true)` throws `UnroutableGeometry` and splitmix construction succeeds. Keep fresh-process default tests distinct from splitmix matrix runs; never expect an automatic fallback. Assert unique global ownership, generator-peer agreement, zero-payload participation, fixed-candidate ordering across budgets and global numerical - equivalence. Do not require old/new rank-local maps equal for splitmix. + equivalence. Do not require old/new rank-local maps equal for splitmix. Preserve current + `routing_more_rank_bits_than_modes_is_refused`, `routing_basis_columns_are_independent_at_any_seed`, + `routing_fibres_are_equal_sized`, `routing_dest_is_bit_identical_to_the_reference_bit_walk` and + `routing_sparse_peer_plans_are_validated_before_use`. Run seed variations in separate processes because EnvConfig + and the basis are cached; do not add environment reload to make tests convenient. +- [ ] Keep exact routing agreement on the caller: `routing::Config` carries raw mode, partitions and seed; + `Pairwise.h::agree_routes_pairwise` reduces each value and its complement with MPI_MAX in one collective. + `MPIUtils.h::check_routing_agreement` delegates to `mpi::routes_pairwise`, whose ordinary-MPI result is cached on + that communicator with no attribute-copy callback. At cutover partitions=1 is fixed and budgets are not agreement + inputs. Add ordinary-MPI tests in mpi_utils_tests.cpp: explicit matching Configs agree, differing valid mode/seed/part + Configs throw routing::RoutingDisagreement on every rank, and cold/warm calls plus duplicated/freed/new communicators + preserve independent agreement. Use explicit test Configs or fresh process environments, not production cache reset. + Keep all ranks in the same cold-use sequence, including empty local stores. Do not reintroduce probabilistic digest + agreement, per-gate allreduces or a cache keyed only by recycled MPI_Comm values. +- [ ] Preserve EnvConfig's separate route-seed parser: uint64 decimal via from_chars; unset/empty uses the default; + signs, whitespace, trailing junk and overflow reject. Do not conflate it with the new present-empty thread error. + Preserve COMMROUTE's ranks_per_coset/rank_cosets diagnostic and invalid-gate retry behavior. These describe reachable + cosets, not observed idle ranks. Equal fibres over all possible monomials do not prove actual retained-term balance; + per-rank load/peak measurements remain required. - [ ] Migrate every constructor/helper/binding test using removed partition arguments in this same task. Use a source search over cpp/tests, tests and src; include the installed export probe, constructor-validation cases and Python parameter-validation/smoke tests. Old calls failing to compile or obsolete keywords still being accepted are @@ -1495,6 +1843,18 @@ this is not a budget change. There is no topology-based fallback or special mult - [ ] Test public Majorana/Pauli mathematical constructors unchanged. Remove binding-only partitions and do not add num_threads or any other public thread-count argument. Test copy, operator update, retained callbacks after growth, cutoff/basis-change update, paring, Schrödinger state extension and independent repeated evaluation. +- [ ] Preserve Python incremental graph construction exactly: build_graph(a); build_graph(b) denotes b+a in Heisenberg + and a+b in Schrödinger. New Heisenberg blocks take low parameter indices and lift existing ones, without renumbering + gate arrival IDs or splitting a multi-monomial gate's shared index. seed_parameters is on the POST-call axis; Python + rotates it for the existing C++ build, then relabels the successful graph. C++ build_graph still takes its mapping + as given. Keep this Python adaptation rather than shifting gates/parameters inside worker loops. Run these existing + tests at budgets 1/2/3/4 in fresh launches, with no tolerance relaxation: + `test_incremental_build_wires_the_picture_s_equivalent_circuit`, + `test_incremental_heisenberg_moves_each_block_to_the_front_of_the_axis`, + `test_incremental_reindex_keeps_a_multi_monomial_gate_on_one_index`, + `test_incremental_build_wires_the_equivalent_circuit_for_pauli`, and + `test_extend_seed_parameters_are_read_on_the_post_call_axis`. Compare decoded retained maps for seeded truncation + as well as sizes/scalars; preserve failed-build retry and retained-functional invalidation coverage. - [ ] Regenerate bindings via the configured generator. Preserve 32-mode template dispatch. Document low-level/source/ABI breaks and unspecified coefficient-order changes; compare decoded term/value mappings rather than sorted coefficients alone. @@ -1505,10 +1865,12 @@ stay absent; invalid objects cannot be reused. Record the approved immediate com ### Task 11: Measure parity and apply only bounded optimizations -**Files:** Task 1 benchmark files and, only if profiling justifies, the kernels named below. Do not rename `bench_*` -functions or move benchmark files. +**Files:** Task 1 benchmark files and, for bounded optimizations, the kernels below. The concurrent-index trial may +modify `cpp/monoprop/detail/operator/OperatorIndex.h`, its immediate MPOperator/Resolve/Engine callers and +`cpp/tests/{operator_index_tests,sparse_resolve_tests,simulator_copy_tests}.cpp`. Keep any adapter private and narrow; +no container-plugin framework. Do not rename `bench_*` functions or move benchmark files. -- [ ] Add metadata for `has_mpi`, requested/effective team size, OpenMP runtime, +- [ ] Add metadata for `has_mpi`, MPI requested/provided thread support, requested/effective team size, OpenMP runtime, OMP_NUM_THREADS/PLACES/PROC_BIND/DYNAMIC/THREAD_LIMIT, any library override, routing mode/effective linear bits/seed, launcher command, rank/worker CPU masks, NUMA IDs, revision/dirty overlay and extension path/hash. Collect placement in separate diagnostic runs of the same binary/configuration with OpenMP runtime diagnostics @@ -1520,9 +1882,10 @@ functions or move benchmark files. recommended/acceptance configuration. A wider affinity mask is not evidence of extra worker capacity. Record distinct physical-core ownership from verified sibling topology across active ranks/workers, not only disjoint logical CPU IDs. No required hwloc dependency for the library or measurement helper. -- [ ] The benchmark operator, not monoprop construction, verifies that rank masks and actual worker places stay in one - NUMA domain and within the reserved allocation, with no unintended overlap between ranks. Advise at least one MPI - process per allocated NUMA domain, with disjoint CPU subsets when several processes share a domain. Use actual OS/site +- [ ] For required cells, the benchmark operator verifies that rank masks and worker places stay within one NUMA domain + and the reserved allocation, with no overlap. The separately labelled full-node L2a diagnostic may cross domains; + record actual domain/core masks instead of falsely certifying containment. Neither check is library policy. Advise at + least one MPI process per allocated NUMA domain, with disjoint cores when several processes share a domain. Use actual topology, not a constant four domains/socket; partial-node runs cover only their allocated domains. Do not require one logical CPU per worker mask. Record idle/small-kernel serial fallback versus representative large-kernel actual team size separately. Unverifiable placement blocks benchmark evidence, not library construction. @@ -1591,8 +1954,8 @@ OMP_PLACES=cores is not a worker-count cap, and merely setting OMP_NUM_THREADS d 2. Ordered query exact-fill from Task 5; bounded self-probe windows from Task 6; optional inline fill from Task 7. 3. Incoming missing-ID prefix: parallel fixed-block miss counts, serial exclusive offsets, exact resize, parallel ascending fill. IDs remain sender/query order; scratch O(block count), not threads×queries. - 4. Serial index pre-reserve to reduce rehash (measure simultaneous old/new table capacity), without concurrent - insertion. + 4. Serial index pre-reserve to reduce rehash (measure simultaneous old/new table capacity); the separate bounded + concurrent-index trial below is also allowed, not assumed to win. 5. Packed graph phases by **whole-word ownership** only: serial validate/count/allocate; each worker constructs and assigns one uint64 word, including slot seams and zero padding. No endpoint-wise `|=`. Test slots 63/2/65 and nonbinary fallback; preserve derived D permutation instead of adding persistent D indices. @@ -1602,8 +1965,53 @@ OMP_PLACES=cores is not a worker-count cap, and merely setting OMP_NUM_THREADS d full-height bitmap per worker. Tests must compare dense/sparse promotion, parity and sorted posting order against serial after every append. - [ ] Each optional optimization gets its own failing regression tests, focused review and full affected parity rerun. - Incoming whole-batch streaming/windowing, concurrent hash publication, new transport, new executor, rank - redistribution, full per-thread stores or a persistent key map require a new design; stop rather than improvise. + Beyond the bounded index trial below, whole-batch streaming/windowing, a custom concurrent table, new transport, + executor, rank redistribution, full per-thread stores or a persistent duplicate full-key map require a new design. + +#### Bounded packed-index versus Boost concurrent-index trial + +Task 0 already investigates ownership/index scaling in isolation. This task validates integration of the same Boost +candidate only if retained by the owner; it is not a second container search. Reuse its feasibility findings and tiny +fixtures, not its timings as full-library evidence. If the architecture changed, revise this task before executing it. + +This is one development-only index variant, not an unbounded container search or a permanent backend choice. +It needs authorized implementation/measurement budget; it is not part of the two-hour baseline-only sizing pilot. +Investigate feasibility from source, then compare focused lookup/insertion/growth measurements and end-to-end build. If +inadmissible, record the concrete invariant/API/version obstacle, not an unsupported claim that concurrent maps lose. + +- [ ] Verify the project's supported Boost versions provide the APIs used; do not silently raise the dependency floor + from the version available on the experiment host. Use `boost::concurrent_flat_map` as an index into the packed rows, + with compact stored row references and heterogeneous position-query lookup. Full monomials, query-buffer spans and a + second persistent reference index are forbidden as stored duplicate keys. Hash/equality must represent monomial + identity, not compare row IDs as if they were term keys. Retain exact collision checks and cached query hashes. +- [ ] Preserve the current lookup/batched result contract. Bulk visitation callbacks are not an ordered query/result + zip: test hits, misses, repeated queries and output ordinals explicitly. A bounded per-query visitor implementation + is an acceptable first trial; do not invent side-effecting hash/equality functions to recover callback order. Never + re-enter the same concurrent container from its visitor. Audit index iteration callers: canonical row order comes + from row IDs, not concurrent bucket iteration order. +- [ ] Complete frozen probing and deterministic missing-ID assignment before any publication. Caller checks fixed32 and + allocation limits, reserves rows/table, initializes packed rows and overflow entries, then workers publish only the + index entries for preassigned IDs. No row growth, coefficient resize, overflow mutation, inverted-index mutation, + graph packing or external-key relocation while map workers run. Indexed row contents remain immutable until the + index is cleared/rebuilt. Join before dependent reads, structural maintenance or freeing scratch. Preserve remote + leader → remote follower → deferred self leader → deferred self follower publication phase order and matched timing. +- [ ] Keep one index per independent packed store; no reference index alongside Boost. Copy/move/clone must bind + row-aware hash/equality state to the destination's rows, never a destroyed source. Container synchronization does not + make external-row lifetimes safe. Keep exceptions inside Workshare, join, then use existing caller invalidation and + distributed-failure boundaries, including during active-ticket lifetime. No worker MPI or scheduling-based ID counter. +- [ ] Extend tests before adoption: deliberate hash collisions with distinct keys; repeated lookup queries; mixed + hit/miss batches and exact row IDs at budgets 1/2/3/4; inline and spilled rows; reserve/growth boundaries; duplicate + insertion contract; clone lookup after source mutation/destruction; allocation/publication failure invalidation; and + end-to-end term/graph equivalence at the same geometry. Preservation tests may already pass; new variant-path and + worker-participation tests provide the new-path evidence. Run qualified race checks without blanket suppressions. +- [ ] Compare separate build variants on identical frozen inputs/allocation. Measure hit/miss lookup, insertion, + reserve/rehash time and transient peaks, row/index/overflow bytes, total construction peak and build/propagate time. + Then rerun every affected formal cell's five metrics and numerical checks against Task 1's refreshed-main baseline; + report the comparison with the packed-index candidate too. No new sizes, higher memory allowance or acceptance waiver. + Microbenchmark gains do not override end-to-end regressions. Obtain approval for any spend beyond the approved budget. +- [ ] If selected, remove the development selector/reference variant before final acceptance and rerun affected gates + on that final binary. If rejected, keep the packed index and archive the evidence. If neither can pass, stop: the + experiment is permission to investigate an existing container, not to rescue the architecture at any cost. **Gate:** All required performance/memory cells pass with attached raw observations and numerical/placement evidence. Missing required EC2 evidence, capacity failure, persistent serial-publication bottleneck or repeated ratio>1.00 blocks @@ -1625,7 +2033,7 @@ removal/acceptance. Report it honestly; the design is not a promise of achievabl `docs/content/docs/{building,benchmarks,openmp-migration}.mdx`, `docs/content/docs/features/parallelism.mdx`. - Modify: `cpp/monoprop/detail/CMakeLists.txt`, `cpp/monoprop/detail/{mpi,parallel}/CMakeLists.txt`, `cpp/include/monoprop/MonomialPropagator.h`, `cpp/monoprop/detail/monomial_propagator/MonomialPropagator.inl`, - `cpp/monoprop/detail/mpi/{Comm,MPICompat,MPIUtils,Exchange,CheckedCount,Routing}.h`, + `cpp/monoprop/detail/mpi/{Comm,MPICompat,MPIUtils,Exchange,Pairwise,CheckedCount,Routing}.h`, `cpp/monoprop/detail/mpi/MPICompat.cpp`, `cpp/monoprop/MPFunctions.cpp`. - Migrate/retire test files according to ledger below; update `cpp/tests/README.md` and architecture docs in the same change. @@ -1643,9 +2051,23 @@ existing typed exchanges. Do not collapse it to a raw alias if that changes over - [ ] Remove non-MPI-thread transport branches from utilities/tickets. Production router geometry is `(real_ranks,1)`. Preserve EXISTING boolean linear/splitmix default/rejection contract and generator shift/peer rules; no routing-default repair is needed. Keep ordinary MPI sparse query routing and sparse on-wire replay, sender order, - count checks, communicator lifetime and serialized thread-level validation (including external initialization). - `MPIUtils.h` routing agreement uses fixed S=1, never per-rank OpenMP budgets. Keep both MPI and non-MPI + count checks, communicator lifetime and initializing-thread guards (including external initialization). + `MPIUtils.h::check_routing_agreement` still delegates to mpi::routes_pairwise; retain exact Config agreement, + communicator attributes/no-copy lifetime semantics and fixed S=1, never per-rank OpenMP budgets. Preserve + require_routable validation, the flat known_recv_counts width check, and WindowVec-only begin_alltoallv/wait_into. + Keep cpp/tests/SlotBlocks.h as the test-only whole-world adapter where migrated tests use it; do not restore + production vector-of-vectors overloads. Ticket/PendingAlltoallv move/drain semantics survive deletion. Keep both MPI + and non-MPI implementations. +- [ ] With the last Hybrid caller removed, change MPICompat's requested and required minimum to MPI_THREAD_FUNNELED. + Preserve MPI_Query_thread validation for externally initialized MPI and Task 3's caller guard even at higher levels. + Test library-owned initialization; external FUNNELED, SERIALIZED and MULTIPLE; rejection of actual SINGLE; + wrong-thread entry at higher levels; and valid caller execution with workers. Record requested/provided support + and distinguish unavailable runtime scenarios from exercised cases. In Python subprocesses, configure mpi4py before + importing MPI/monoprop; its default MULTIPLE is accepted, not downgraded. Audit finalization/cleanup and failure paths + as well as hot kernels: no worker MPI calls, including MPI_Abort. Update diagnostics/docs/tests in the same task, then + rerun MPI correctness/failure tests and affected parity cells on the final binary. Changing only Init_thread's + constant does not complete this gate. - [ ] Delete the entire topology/affinity implementation and its legacy call sites, including automatic physical-core counting, L3 dealing, pinning and mandatory placement diagnostics. Do not move them into ThreadBudget or reimplement them with OpenMP-place unions, OS affinity, sysfs or another topology library. User-controlled OpenMP and launcher @@ -1675,8 +2097,11 @@ existing typed exchanges. Do not collapse it to a raw alias if that changes over | `partition_group_clone_tests.cpp`: derived children | retire child-existence assertions; retain derived clone/update-initial-operator independence in `simulator_copy_tests.cpp` and `update_initial_operator.cpp` | | `shm_comm_tests.cpp` / `hybrid_comm_tests.cpp`: private barrier algorithms | retire only after ownership/source-order/empty-payload/count/error assertions map to MPI/OpenMP tests | | `hybrid_comm_sparse_plan_on_the_plain_mpi_path` | move unchanged semantics to ordinary MPI tests; unknown and known receive counts | -| plain-MPI half of `hybrid_comm_known_recv_counts_are_masked_through_the_plan` | preserve masked non-peer counts and sparse peer payload assertions | -| `flat_exchange_tests.cpp` replay dense/pairwise/dense-layout/empty tests | retain and run under ordinary MPI after deletion | +| plain-MPI half of `hybrid_comm_known_recv_counts_are_masked_through_the_plan` | preserve masked non-peer counts and sparse peer payload assertions; add short known-count rejection before posting | +| `hybrid_comm_unroutable_peer_plan_is_refused` | move ordinary-MPI invalid-shift/count-entry checks; retain routing_sparse_peer_plans_are_validated_before_use | +| `hybrid_comm_pending_alltoallv_owns_its_requests` | move ordinary-MPI move/noncopyable/destructor-drain coverage unchanged; extend active-destination move assignment with live buffers | +| `hybrid_comm_derived_wire_plan_reads_every_partitions_row` / `hybrid_comm_derived_wire_plan_falls_back_to_every_peer` | retire only partition-row folding; preserve empty, one-peer and multi-peer replay under ordinary MPI, including four-rank fixtures | +| `flat_exchange_tests.cpp` replay dense/pairwise/dense-layout/empty tests | retain and run under ordinary MPI after deletion; dense gate test is flat_exchange_dense_gate_always_takes_the_collective | | hybrid MPI energy and size case | retain as MPI×OpenMP matrix case in `mpi_distributed_layer_equivalence.cpp` | | poisoned waiter release | Task 3 MPI failure-driver, joined exceptions and enforced invalid-owner/functionals tests | | `cpu_topology_tests.cpp`: topology enumeration, mask classification, pinning/L3 dealing | retire these library-owned policy tests; Task 3/10 count/precedence/no-affinity-mutation tests cover the retained library contract, and Task 11 verifies placement externally | @@ -1719,7 +2144,8 @@ by construction, copy/functional tests and memory accounting, not merely a flag. - Extend existing consumer/probe: `cpp/tests/find_package_smoke/CMakeLists.txt`, `cpp/tests/link_export_probe/link_export_probe.cpp`. - Modify: `AGENTS.md`, `README.md`, `docs/content/docs/{building,benchmarks,openmp-migration}.mdx`, - `docs/content/docs/features/parallelism.mdx`, `docs/content/docs/concepts/algorithm.mdx`, `cpp/tests/README.md`, + `docs/content/docs/features/{parallelism,evaluation}.mdx`, `docs/content/docs/concepts/algorithm.mdx`, + `cpp/tests/README.md`, public Python docstrings affected by ordering/runtime guidance. - Test: existing wheel jobs; benchmark renderer tests; whole C++/Python suite. @@ -1753,8 +2179,9 @@ nix develop --command bash -euo pipefail -c ' Normally use monoprop_NUM_THREADS=T; only unset selects the OpenMP runtime default. Present-invalid is an error. Capture once during construction, with no reload/setter. Describe no hardware clamp/topology discovery/NUMA validation, possible smaller OpenMP teams, and the within-one-domain allocation assumption. Show matching allocation, - rank-binding and thread-place examples, including the one-core-rank-mask pitfall. Document one controlling host caller - outside host OpenMP teams, no overlapping use of dependent state or host MPI calls, serialized MPI, callable lifetime, + rank-binding and thread-place examples, including the one-core-rank-mask pitfall and separately labelled cross-NUMA + L2a diagnostic exception. Document one controlling host caller on MPI's initializing thread outside host OpenMP teams, + no overlapping use of dependent state or host MPI calls, FUNNELED-or-higher support, caller checks, callable lifetime, enforced failed-object invalidation, accepted capacity reduction and numerical contracts. Placement verification for benchmarks does not add a topology dependency. Advise at least one MPI process per allocated NUMA domain and identify actual topology externally; production's common four-domains/socket layout is not universal. Distinguish the selected @@ -1766,7 +2193,11 @@ nix develop --command bash -euo pipefail -c ' errors and environment controls removed without aliases; no replacement public num_threads argument. Old automatic physical-core selection, multi-rank serial default, library pinning and direct hwloc requirement are gone. OpenMP is mandatory even without MPI. Raw order/splitmix physical ownership may differ; ABI requires rebuild. Requests are not - hardware-clamped. Preserve high-level Python mathematical signatures. In prose/docstrings use `[Symbol][]` references, + hardware-clamped. Record the stronger initializing-thread MPI caller rule as a compatibility break, even at supplied + SERIALIZED/MULTIPLE. Preserve Python mathematical signatures and current incremental-axis/post-call seed semantics + in the evaluation guide/docstrings. Keep Qiskit conversion and its optional-version tests unchanged; do not advertise + those already-shipped additions as OpenMP work. Explain routing geometry and COMMROUTE cosets without claiming a + balanced basis guarantees a balanced retained operator. Use `[Symbol][]` references in prose/docstrings, not hard-coded API URLs. - [ ] Run `prek run --all-files` before any separately authorized push, plus relevant tests, `just gen-api` and `just build-docs`. Targeted hooks during development do not replace the all-files gate. Keep generated bindings/API @@ -1783,6 +2214,8 @@ nix develop --command bash -euo pipefail -c ' **Final acceptance checklist:** +- [ ] Task 0 evidence and the owner's architecture decision are archived; this conditional implementation and its + MPI-off acceptance coverage match the reconciled spec/plan, not a superseded assumption of one store. - [ ] Mathematical features, equal global retained term keys, copies/updates/closures and graph paring pass existing oracles; matching energies/gradients alone is insufficient. - [ ] Environment-only immutable budgets, strict invalid-input errors, removed partition interfaces and enforced @@ -1795,8 +2228,9 @@ nix develop --command bash -euo pipefail -c ' explicitly pending, not falsely passed or silently included in the initial acceptance claim. - [ ] Every required runtime and exact peak-memory cell passes Task 11; unsupported or missing measurements are not marked passed. -- [ ] No legacy runtime, direct hwloc dependency or replacement topology/pinning service remains; no equivalent - scheduler/concurrent store was introduced. +- [ ] No legacy runtime, direct hwloc dependency, replacement topology/pinning service or scheduler remains. A selected + Boost concurrent index passed its bounded tests and affected parity gates, with no duplicate operator/full-key store. + MPI requests/requires FUNNELED and checks the initializing caller even at higher provided levels. - [ ] Documentation, benchmark schema consumers and API generation agree. Operation/construction exactness gates the five parity metrics; outer-window exactness controls diagnostic labels only. Frozen campaign coverage, unique paired measurement artifacts and matching validation/placement provenance pass, including all four historical pytest families @@ -1827,5 +2261,6 @@ Decision required from owner: Examples requiring a stop: a full rank cannot fit fixed uint32_t TermIndex, serial publication prevents parity, query buffering exceeds peak baseline, the benchmark operator cannot verify the target allocation/placement externally, unsupported OpenMP packaging, failure injection hangs, numerical changes alter retained terms, or the owner declines a -compatibility break. None authorizes a concurrent map, extra shards, altered MPI geometry, weakened oracle, or a -performance waiver. +compatibility break. Task 0 permits the three standalone index variants and reopens the architecture with the owner; +Task 11 permits only the retained candidate's bounded integration trial. Neither silently authorizes production shards, +an unbounded/custom concurrent store, altered acceptance geometry, a weakened oracle or a waiver. diff --git a/docs/superpowers/specs/2026-09-18-rank-local-openmp-design.md b/docs/superpowers/specs/2026-09-18-rank-local-openmp-design.md index 6effa311..77fd3f9c 100644 --- a/docs/superpowers/specs/2026-09-18-rank-local-openmp-design.md +++ b/docs/superpowers/specs/2026-09-18-rank-local-openmp-design.md @@ -5,13 +5,22 @@ This is the design accompanying the implementation plan at `../plans/2026-09-18-rank-local-openmp.md`. It records the direction discussed with the owner; it is not a claim that the proposed implementation meets its performance gates. -Source inspected: branch `perf/linear-routing-on-wire`, commit `290112c8289ab8015eb9a2c7651ac409839c5a88`. - -The planning documents were published in `refactor-parallelization` at `38eb13d80a95dd1ac60e762f14f68c9a5925f755`; -this policy revision follows the owner's design interview. The owner approved mandatory OpenMP, removal of direct hwloc -and partition configuration without deprecation aliases, the associated API/ABI and capacity breaks, environment-only -construction-time thread budgets, and the failure/calling contracts below. These are design approvals, not -implementation or publication authority for this revision. +Source inspected for this owner-requested current-main refresh: `origin/main` at +`90d57177c2b0cd93503f88dd931d2f8dd409aa23`, present in rebased `refactor-parallelization` at +`9bede897d99ac7e2a53442885629d5032f1b7df4`. Only CONTEXT.md and the two planning documents differ between those commits. +EXISTING means this refreshed source snapshot; PROPOSED means future implementation. + +The original planning publication was `38eb13d80a95dd1ac60e762f14f68c9a5925f755` in `refactor-parallelization`. +This source refresh preserves the subsequent owner-approved policy revisions and uncommitted edits. The owner approved +mandatory OpenMP, removal of direct hwloc and partition configuration without deprecation aliases, the associated +API/ABI and capacity breaks, environment-only construction-time thread budgets, and the failure/calling contracts below. +These are design approvals, not implementation or publication authority for this revision. + +**Architecture decision reopened:** the owner now requires Task 0, a small MPI-off index mini-app comparing per-thread +stores, one shared non-concurrent store and one shared concurrent index. Its evidence can overturn the one-store choice. +The one-store design and Tasks 1–13 below are conditional, not an instruction to proceed regardless of that result. +After Task 0, stop for the owner's architecture decision and reconcile both documents before any downstream execution. +This amendment authorizes planning only, not mini-app implementation, measurements or remote expenditure. The owner selected one AWS EC2 `c8a.metal-24xl` for implementation/testing and the initial acceptance campaign. Production deployments are dual-socket HPC nodes using Intel Sapphire Rapids or AMD Zen3/Zen4/Zen5. Hardware SMT may be @@ -23,19 +32,100 @@ that campaign. The owner approved baseline-only, unscored feasibility calibratio separately authorized. Small/medium profiles cover the full geometry sweep; large profiles cover only the full-node one- and two-ranks-per-domain configurations. This revision authorizes no runtime work or implementation. +**Source-drift audit completed:** the owner-requested refresh replaces the superseded +`290112c8289ab8015eb9a2c7651ac409839c5a88` pin with the main revision above. The audit covered all 43 changed paths, +including routing/MPI, construction, Python parameter-axis semantics, tests, version/build tooling and documentation. +The previous d93d63f source-refresh blocker is resolved by this audit and the corresponding plan updates, not by the +rebase alone. Both measurement arms must start from this refreshed pre-OpenMP engine; do not reuse superseded-engine +samples as its baseline. Baseline build/calibration authorization, campaign freeze and implementation approval remain +separate gates. Later semantic source drift still requires review against the plan's refreshed anchor table. + +### Findings incorporated from current main + +- Routing walks a width-bound, full-rank GF(2) basis; it no longer uses a transposed plane table. Linear geometry + requires power-of-two R and log2(R)<=min(2*NumModes,64). A deterministic bounded redraw can change the basis + relative to the superseded source, so old physical-rank maps are not a reference for this refactor. Preserve the + current map/seed. +- MPI replay selects a communicator-agreed boolean via mpi::routes_pairwise, not a recomputed bit-count selector. + Exact agreement, communicator-scoped caching, sparse-plan validation and multi-peer replay must survive cutover. +- Construction exchanges accept WindowVec blocks only. LayerBuildEngine receives both plan and window explicitly; + each FusedScanResult stream carries its own window. IncomingProbe exposes sender_index, not sender_slot. +- Both replay Ticket and construction PendingAlltoallv own and drain their requests. Failure guards must protect both + lifetimes, including move-assignment and destruction, rather than accidentally wait during exception unwinding. +- Python incremental Heisenberg graph builds now renumber the parameter axis to match the equivalent composed circuit; + preserve this behavior, post-call seed interpretation and the new regression tests for both Python propagators. +- Packed storage, fixed32 indices, streamed initialization, stable-gradient formulas, benchmark families/windows and + centralized build recipes are unchanged from the inspected source. The plan retains their gates and adds the current + transport/routing/axis regressions. Source inspection is not runtime, race-freedom or performance evidence. + ## Objective -Replace thread-owned operator partitions and their in-process communication protocols with one compact operator store -per MPI rank, operated on by OpenMP worksharing. Preserve the mathematical features, memory footprint, and performance -of the existing library. Simplification without regressions is the objective; the one-store/OpenMP candidate must earn -acceptance, not be rescued with prohibited replacement machinery if its gates fail. Each MPI rank's assigned CPUs are -assumed to lie within one NUMA domain. The user chooses the -number of processes P and requested threads per process T for the resources available, and configures the launcher and +First investigate MPI-off index scaling with Task 0 before choosing the ownership architecture. The conditional +proposal is to replace thread-owned operator partitions and their in-process communication protocols with one compact +operator store per MPI rank, operated on by OpenMP worksharing. Preserve mathematical features, memory footprint and +performance. Simplification without regressions is the objective; neither one-store variant is a predetermined winner. +If the owner retains the one-store proposal, its production acceptance assumes each +rank's CPUs lie within one NUMA domain. Full-node, cross-NUMA L2a is an explicitly allowed diagnostic exception, not a +required performance gate. The user chooses processes P and threads per process T, and configures the launcher and OpenMP runtime to bind/pin processes and threads correctly. The library neither discovers topology nor repairs placement. +## Preliminary architecture experiment (Task 0) + +Keep the mini-app an index exercise, not a second propagation engine: one C++ translation unit, standalone CMake file +and short README under tools/index-miniapp/. Reuse the current OperatorIndex and monomial hashing headers. No library +build/import, MPI, ShmComm, graph, coefficients, inverted index, physics/cutoff logic, trace capture/replay service, +custom scheduler or benchmark framework. A few local adapter functions, flat batch buffers, self-tests and CSV output +are sufficient. Stop and reduce scope if implementing the experiment requires a production storage refactor. + +Use one process and one OpenMP team of T workers, including its primary thread. Compare exactly three variants: + +| Variant | Stores for one global input | Lookup and publication | +| --- | --- | --- | +| per-thread | T disjoint current OperatorIndex stores, not T copies of all keys | Each owner probes and inserts serially; owners run in parallel | +| shared-serial | One current OperatorIndex | Parallel frozen probes; caller assigns IDs, initializes rows and inserts | +| shared-concurrent | One packed row store and one Boost concurrent index | Parallel probes; caller assigns IDs/initializes rows; parallel index publication | + +The per-thread variant models current storage ownership, not the legacy runtime's performance. All variants use the +same OpenMP harness and global keys/queries. Keep the real packed-row representation, batched position lookup, cached +32-bit hashes, exact collision checks and fixed32 row references. For Boost use stable row handles and heterogeneous +query lookup, not persistent full monomial keys. A row-only use of OperatorIndex may retain its constant empty +16-slot table, accounted as harness overhead; never populate it or allocate a second size-dependent index. This narrow +mini-app convenience is not an approved production layout. If the supported Boost floor cannot express the adapter, +report that limitation rather than building another table or silently changing dependencies. + +Exercise frozen lookup and repeated probe → ordered missing-ID assignment → row growth/fill → publication → join +batches. Keep rows immutable during concurrent index operations; do not benchmark simultaneous readers/inserters when +the proposed engine uses separate phases. Include a small flat-buffer owner-bucketing/result-scatter path for the +sharded variant and report its cost separately. Do not simulate the old message transport or claim those transfers are +free. Local IDs may differ between sharding geometries; decoded keys/results and within-geometry ID order must agree. + +Use deterministic synthetic position lists: fixed global row/query counts across thread counts, controlled hit/miss +ratios, short and spilled rows, and natural growth. No full trace corpus or synthetic physics. Begin with a compact +strong-scaling sweep, not a Cartesian product of every knob. Measure both within one NUMA domain and across the +allocated machine with one worker per physical core. Full-machine MPI-off results are essential Task 0 evidence, not +optional merely because the conditional production campaign calls cross-NUMA L2a diagnostic-only. Verify placement +externally and record first-touch/memory policy; do not add topology or affinity management to the app. + +Report complete-batch time, phase times, throughput, speedup against each variant's own T=1 and the per-thread variant +at the same T, actual workers, per-store imbalance, and fresh-process peak RSS including construction/growth. State +which costs are omitted, which byte counts are estimates, and when memory placement or noise prevents a conclusion. +Synthetic index results cannot establish propagation/replay/gradient scaling or full-library parity. + +Task 0 ends with raw results and an owner decision: retain a shared-store path, retain/reconsider sharding, or declare +the evidence inconclusive. Do not auto-select the fastest lookup kernel or invent a fourth architecture. Any revised +production design needs an updated spec/plan and approval, including MPI-off acceptance coverage. Keep the existing +strict library gates as written unless the owner explicitly revises them. Task 0 needs a separately approved execution +budget; it neither consumes Task 1's two-hour baseline pilot nor supplies its calibration or acceptance samples. The +later Task 11 trial is integration validation of a retained candidate, not a second container search. + ## Global constraints +The following constraints describe the conditional one-store path in Tasks 1–13. Task 0 alone permits the explicit +per-thread-store comparison and requires full-machine MPI-off measurements, including cross-NUMA placement. This is +not permission to add shards to production or to waive any existing acceptance gate. Reconcile these constraints with +the owner's post-experiment architecture decision before proceeding. + - C++23; retain the GCC 14 / Clang 18 minimum compiler versions and Python 3.11 floor. - MPI remains optional and OFF by default; retain working non-MPI wheels. - OpenMP becomes a required implementation dependency for the replacement runtime; support Linux x86_64, Linux aarch64, @@ -49,14 +139,18 @@ placement. does not clamp T to physical cores or validate NUMA placement. - SMT use is not recommended: allocate at most one monoprop worker per physical core, including on SMT-enabled hosts. Enforce this through user launch/binding settings and acceptance evidence, not a library hardware policy. -- Preserve packed operator rows, the keyless hash index, fixed uint32_t TermIndex, sparse state storage, and - inverted-index cosine recomputation. -- Do not introduce boost::concurrent_flat_map, hidden shards, a dense duplicate store, a second persistent key store, a - full per-thread operator, or full-vector coefficient double buffering. +- Preserve packed rows, keyless indexing through row references, fixed uint32_t TermIndex, sparse state storage, and + inverted-index cosine recomputation. A bounded Boost concurrent-index experiment may replace the index, not rows. +- Do not introduce hidden operator shards, a dense duplicate store, a second persistent full-key store, per-thread + operators, or full-vector coefficient double buffering. Select any concurrent index only after correctness/memory/time + evidence; its container name is neither a rejection criterion nor proof of suitability. +- Required performance cells keep each rank within one NUMA domain. Full-node cross-NUMA L2a is diagnostic-only and + cannot replace, excuse or average away any required cell; its numerical correctness is still checked. - Preserve Majorana and Pauli bases, Heisenberg and Schrödinger pictures, cutoffs, graph build/replay/paring, gradients, partial contraction, copying, and initial-operator updates. -- MPI calls execute on the controlling thread outside library-created OpenMP worksharing regions; retain - MPI_THREAD_SERIALIZED rather than changing the initialization contract. +- Final MPI calls execute on the MPI-initializing thread, outside library-created OpenMP regions. Request/validate at + least MPI_THREAD_FUNNELED, accepting higher provided levels without relaxing the caller rule. Retain + MPI_THREAD_SERIALIZED during legacy Hybrid coexistence; lower the minimum only with its removal in Task 12. - No exception escapes an OpenMP structured region; distributed operations fail coherently rather than leaving peers in collectives. Enforce invalidation after failed mutation; dependent functionals cannot reuse an invalid owner. - No global omp_set_num_threads/omp_set_dynamic/omp_set_nested mutation; each worksharing region receives an explicit @@ -76,6 +170,8 @@ placement. ## Architecture +This section describes the one-store candidate, conditional on the Task 0 decision. + Each independent propagator has its rank-local `MPOperator`, primary `MPGraph`, and ordinary MPI communicator. This is not a process-wide singleton: copies own independent packed stores; retained functionals may own coefficient/state snapshots and pared graphs without cloning the packed store. Keep existing C++ owner-lifetime requirements and Python @@ -89,6 +185,29 @@ resizes shared containers and publishes structural changes between phases. Initi table and inverted-index maintenance remain single-writer. This is an explicit performance gate, not an assumption that insertion is negligible. +### Bounded concurrent-index experiment + +The previous blanket exclusion of boost::concurrent_flat_map is superseded, not a measured rejection. Task 0 first +compares the three ownership/index designs in isolation. If retained by the owner, compare the packed keyless path with +that same bounded Boost variant in Task 11 after the full-library baseline campaign is frozen. This later trial tests +integration and the unchanged acceptance gates; it does not repeat container selection or bypass Task 0. A direct +`concurrent_flat_map` duplicates full keys; investigate stable row handles and heterogeneous lookup +into packed rows. Container synchronization does not protect coefficients, overflow storage, inverted indices or +graph packing. The +[Boost concurrency guide](https://www.boost.org/doc/libs/1_89_0/libs/unordered/doc/html/unordered/concurrent.html) +describes visitation and blocking rehash; neither is a drop-in substitute for the current batched query/result contract. + +Assign IDs in canonical order, reserve all needed storage on the caller, initialize rows before publication, then join +before dependent reads. Only the index's insertion/publication is allowed to become concurrent in this experiment; +other shared structural mutation remains separately governed. Prove exact collision handling, stable external-row +lifetimes, per-query result alignment and clone ownership. Never store query-buffer views as long-lived keys, allocate +IDs by worker arrival order, or add a second persistent index beside the reference implementation. Compare separate +build variants, not two resident stores or a permanent backend selector. Measure lookup/insertion, growth peaks and +end-to-end construction plus the full affected acceptance cells; an insertion microbenchmark alone cannot select it. +Task 11 specifies the bounded trial and its tests. Keep or reject it from evidence, without weakening parity gates. + +### Kernel and phase sequencing + Only recognized pure `LengthCutoff`/`SupportCutoff` targets may run in worker traversal initially. Ordinary Python and C++ constructors accept built-in cutoff configuration and basis-change data, not arbitrary cutoff functions. Opaque predicates are reachable through the existing protected C++ `cutoff_fn_` extension point and lower-level helpers; @@ -113,10 +232,21 @@ The freeze boundary in step 1 controls which source rows generate work. Do not c Replay uses the existing graph and coefficient arrays. Parallelize cosine blocks, endpoint packing, local rotation pairs, remote endpoint application, and ordered local reductions. Do not parallelize gate order or parameter order. Keep -MPI replay transport behavior unchanged. EXISTING: `Evolution.cpp` derives rank-uniform `wire_bits`; -`detail/mpi/Exchange.h::post_flat_alltoallv` uses a dense `PeerPlan` but posts only nonzero Isend/Irecv legs when -`wire_bits>0`, otherwise MPI_Ialltoallv. HybridComm derives its sparse plan from all local partitions' rows. Preserve -plain-MPI sparse on-wire replay at S=1; a new generator-derived replay plan is outside this refactor. +MPI replay transport behavior unchanged. EXISTING: `Evolution.cpp::begin_flat_exchange` passes +`mpi::routes_pairwise(comm)` to `detail/mpi/Exchange.h::post_flat_alltoallv(..., bool pairwise=false)`. This +rank-uniform boolean selects Isend/Irecv or MPI_Ialltoallv; a local layout must never select the transport. Plain MPI +walks a dense PeerPlan and posts only nonempty remote legs, copying self locally. HybridComm folds every partition's +send/receive peer summaries: one compatible peer narrows the set, while multiple/disagreeing peers retain every peer +on the SAME pairwise transport. Empty traffic must not send one rank into a collective. Preserve multi-peer/hand-built +layer replay and S=1 sparse on-wire replay; a new generator-derived replay plan is outside this refactor. + +EXISTING: `Pairwise.h::agree_routes_pairwise` checks raw routing mode, partition count and seed exactly with one +MPI_MAX allreduce over values and complements. `MPICompat.cpp::routes_pairwise` caches ordinary-MPI results as a +communicator attribute with no copy callback; new/duplicated communicators agree independently rather than inheriting +stale raw-handle cache entries. HybridComm agrees during construction; Shm/non-MPI/single-rank ordinary MPI return +false. `MPIUtils.h::check_routing_agreement` delegates to this path. First use can be collective, and even a cache hit +queries MPI attributes: all of it belongs on the caller, never workers. At S=1 agreement still covers routing/seed, +not per-object thread budgets. The error type is `routing::RoutingDisagreement`. ## Ownership rules @@ -133,18 +263,27 @@ plain-MPI sparse on-wire replay at S=1; a new generator-derived replay plan is o | Packed phase words | Not concurrently written merely because record IDs differ | Keep packing serial initially | | MPI buffers and requests | Pack before posting; do not mutate in-flight buffers | Controlling thread posts/waits and sizes buffers | -EXISTING: `FusedScanResult` in `Scan.h:208–227` has six `WindowVec` streams plus `leader_self`/`follower_self`. -`SlotWindow` indices are not flat slots, including when base is nonzero. Preserve the precomputed `gen_shift`, variable -`QueryWire` records and `QueryForm`; self stages are positions, never serialized wire. Merge only their logical lengths -and rebase position offsets while preserving source/value alignment. Private query buffers have at most +EXISTING: `FusedScanResult` in `Scan.h:208–226` has six `WindowVec` streams plus `leader_self`/`follower_self`, +with no separate result.window member. `build_layer` supplies the same explicit plan/window to scan and engine; +value streams remain empty when capture is disabled. `SlotWindow::indices()` yields typed WindowIndex values in order; +they are not flat slots, including when base is nonzero. Resolve a sender as `pr.window.slot(pr.sender_index(g))`. +Preserve the precomputed `gen_shift`, variable `QueryWire` records and `QueryForm`; self stages are positions, never +serialized wire. Production `begin_alltoallv`/`wait_into` accept WindowVec blocks, not vector-of-vectors; the latter's +adapter is test-only in `cpp/tests/SlotBlocks.h`. Known receive counts remain flat-slot indexed and must cover the +window's stop; short arrays throw rather than silently truncate. Sparse PeerPlan validation rejects invalid geometry +or shifts before indexing/posting. Do not remove these checks when retiring Hybrid. + +Merge self stages by their logical lengths and rebase position offsets while preserving source/value alignment. +Private query buffers have at most O(T*window.count) empty-container metadata, not O(number_of_bitmap_blocks*R). Per-record payload across those buffers must be O(total emitted records), never O(T*operator_size). Release/move drained temporary buffers before starting the next memory-heavy phase. Before parallel decode, the caller checks variable record boundaries, canonical escaped headers (`h.bits == QueryWire::header_bits_for(h.k)`) and position-count prefix sums, sizes arrays once, then workers decode -into disjoint spans. Frozen probes retain `find_batch_positions` and cached hashes; publication remains serial -`set_positions` → `bulk_insert_hashed` → reindex. No per-rank or full-store dense monomial reconstruction is allowed. -Existing bounded paired-only state scoring may still reconstruct its one required monomial; do not turn that exception -into a batch/store. Task 6 defines the checked preparation sequence. +into disjoint spans. Frozen probes retain `find_batch_positions` and cached hashes; the initial reference publication is +serial `set_positions` → `bulk_insert_hashed` → reindex. The bounded Task 11 index trial must preserve these phase and +row-ID contracts even if hash publication becomes concurrent. No per-rank or full-store dense monomial reconstruction is +allowed. Existing bounded paired-only scoring may reconstruct its one required monomial; do not turn that exception into +a batch/store. Task 6 defines the checked preparation sequence. ## Determinism and numerical contract @@ -165,6 +304,13 @@ into a batch/store. Task 6 defines the checked preparation sequence. remain serial: record indices may duplicate. Threaded gates include deep non-singular circuits, vanishing cosine and no-record cases from `tests/test_deep_circuit_gradient.py`, plus exact record restoration in `cpp/tests/combined_recompute_equivalence.cpp`. +- Preserve Python's incremental parameter-axis contract (`src/monoprop/monomial_propagator.py::build_graph`): + build_graph(a) then build_graph(b) denotes b+a in Heisenberg and a+b in Schrödinger. Heisenberg moves each new block + to the low indices, lifting the existing indices; seed_parameters uses the post-call axis. Python rotates that seed + for the low-level build and relabels after success. Gate arrival IDs do not change, and multiple monomial layers of + one gate keep their shared index. C++ build_graph still consumes parameter_mapping as given; do not move Python's + renumbering into it. Preserve tests/test_circuit.py axis, off-build-angle value/gradient, Pauli and seeded-truncation + regressions across budgets. This behavior is already in the baseline, not a new OpenMP compatibility break. - Near-cutoff tests compare retained operator terms, not just final energies. - Preserve streamed `for_each_paired_monomial` initialization order and local reservation/cardinality checks in `MonomialPropagator.inl:126–151,197–227`. At S=1 estimates use real ranks, never T. Reject impossible cardinality @@ -198,7 +344,8 @@ remains pending; do not substitute a cloud-network experiment or claim HPC fabri - The user is responsible for binding/pinning processes and threads to the correct hardware using, for example, `OMP_PLACES` + `OMP_PROC_BIND`, `srun --cpu-bind`, and/or `mpiexec` options. Scheduler allocation, rank binding and OpenMP places must agree: binding a rank to one core while requesting many threads does not allocate additional cores. - Each rank's allocation must accommodate its workers within one NUMA domain; avoid unintended overlap with other ranks. + Production acceptance keeps each rank's workers within one NUMA domain; avoid overlap with other ranks. Full-node + L2a may span domains only as the labelled diagnostic exception described below. Documentation will advise at least one MPI process per NUMA domain used by the allocation; multiple processes may share a domain using disjoint CPU allocations. The production nodes commonly have four domains per socket, hence eight per full dual-socket node, but use the actual configured topology rather than a hard-coded count. Do not infer @@ -271,7 +418,8 @@ that allocation with the site before using masks. Mask lists repeat by node-loca assumed valid on differently numbered/configured nodes. Binding two ranks to the same entire locality domain does not partition its cores. No automatic topology/binding fallback is added to monoprop. -Archive observed rank/worker masks, physical-core sibling identities and NUMA containment before accepting measurements. +Archive rank/worker masks, physical-core sibling identities and NUMA containment for acceptance. For cross-NUMA L2a, +record the full-node allocation and actual multi-domain placement explicitly; do not claim single-domain containment. These recipes document HPC deployment; they do not convert pending proper-interconnect multi-node qualification into completed evidence. @@ -300,17 +448,24 @@ At cutover: adjustment may reduce actual teams; correctness must not depend on receiving exactly T workers. Verify actual placement and participation externally for performance acceptance, not during ordinary construction. 5. Keep the virtual destructor, virtual `update_initial_operator`, its protected nonvirtual `apply_initial_operator_` - helper, and independent virtual `clone_` hook (`MonomialPropagator.h:286–305`). The owner accepts immediate C++ + helper, and independent virtual `clone_` hook (`MonomialPropagator.h:288–306`). The owner accepts immediate C++ source/ABI breaks for partition removal in this 0.x library; record release notes and rebuild library/bindings. High-level Python mathematical signatures remain unchanged; `_core` loses `partitions` and gains no thread keyword. 6. Returned term/coefficient ordering may differ from old partition-block order. Validate identical global retained term keys and coefficient associations within existing tolerances, not coefficient-only sorted lists. Equal energies and gradients alone are insufficient. Keep candidate ordering deterministic across budgets at fixed rank geometry. 7. Preserve boolean linear/splitmix routing, default linear rejection of unsupported R, generator shifts and peer rules. + Keep the full-rank basis draw and width bound log2(R)<=min(2*NumModes,64); the bound uses the C++ template width. The owner accepts the splitmix ownership change from `floor((hash % (R*S))/S)` to `hash % R`; do not emulate S with hidden shards. Keep `monoprop_ROUTING` and `monoprop_ROUTE_SEED`: these control MPI routing, not thread partitions. + Preserve the seed's strict decimal uint64 parser (no sign/whitespace; unset/empty selects its default), separately + from the proposed present-empty thread-budget error. COMMROUTE reports ranks_per_coset/rank_cosets, not measured + idle ranks; a full-rank basis or spanning generator set alone does not prove an actual retained operator is balanced. Test default-linear R=3 rejection separately; successful R=3 suites/benchmarks explicitly use splitmix. Compare global maps and per-rank loads/peaks at identical routing/seed/rank geometry. +8. MPI-using entry must run on MPI's initializing thread, even with SERIALIZED or MULTIPLE support. This deliberately + removes the old freedom to move the controlling caller between threads while serializing calls. It does not add a + dispatcher, change the GIL policy or permit overlapping dependent-state calls. Development-only coexistence uses the EXISTING explicit `partitions=1` argument plus `comm.kind==mpi::Comm::Kind::Mpi` to select the one-store prototype and capture its environment budget. Legacy Shm/Hybrid children and multi-partition @@ -325,18 +480,33 @@ all earlier gates; they are not permission to bypass performance or packaging fa The worksharing helper catches each worker exception, joins the region, and rethrows on the calling thread. Caller-side distributed phase guards print the underlying failure and call MPI_Abort when communicator size exceeds one. Catch -failures inside the lifetime of active exchange tickets, before stack unwinding can wait on peers that never posted; an -outer catch alone is insufficient. Successful paths explicitly wait as before. For single-rank operation, rethrow the -original exception. Enforce post-mutation invalidation on the affected propagator: later state-consuming/mutating calls, -copy/clone and dependent retained functionals reject it before touching state. Existing independent copies remain valid. +failures inside the lifetime of both `Exchange.h::Ticket` and `MPICompat.h::PendingAlltoallv`, before stack unwinding +can wait on peers that never posted; an outer catch alone is insufficient. Both handles are move-only and self-draining; +move assignment drains the destination before adopting incoming requests. Keep request/buffer ownership together and +never resize live request storage. `Pairwise.h::sparse_pairwise` pre-sizes from actual nonempty remote send/receive +legs, not a caller-supplied active-leg estimate. Successful paths explicitly wait as before. For single-rank operation, +rethrow the original exception. Enforce post-mutation invalidation on the affected propagator: later +state-consuming/mutating calls, copy/clone and dependent retained functionals reject it before touching state. +Existing independent copies remain valid. Pre-mutation validation failures leave a valid object usable; destruction remains safe. There is no rollback or reset that revives a failed object. Task 3 defines the invalid flag, entry checks and failure-injection tests. -No rank-local early return may bypass a collective peers will enter. Validate MPI_THREAD_SERIALIZED also when MPI was -initialized externally. The owner confirms one controlling caller per rank, outside active host OpenMP teams, no -overlapping calls on the same propagator/dependent functionals and no overlapping host MPI calls. Distributed entry -from an external OpenMP team remains unsupported; numerical helpers may serialize when nested, but that does not -expand the public calling contract. Add no process-wide locking service or nested-parallel execution machinery. +No rank-local early return may bypass a collective peers will enter. In the final runtime, validate provided support +with MPI_Query_thread, including externally initialized MPI, against MPI_THREAD_FUNNELED. Higher levels are accepted, +but the owner-approved contract remains one controlling caller per rank: the thread that initialized MPI, outside active +host OpenMP teams. Serialize dependent-state use and host MPI calls. A different thread making calls one at a time is +not FUNNELED. Check caller identity before MPI-using entry, including constructors, retained functionals and cleanup. +Wrong-thread entry is a host-contract violation, not an ordinary distributed operation failure: diagnose and fail fast +locally without calling MPI_Abort or querying communicators from that thread. Task 3 defines the launcher-supervised +negative tests. Valid-caller operation failures retain the join/abort/invalidation rules above. + +Keep the legacy SERIALIZED requirement while Hybrid workers still call MPI. Task 12 removes that path and lowers the +requested/required level, updating tests in the same change. Never reinitialize or downgrade host-owned MPI. mpi4py +normally requests MULTIPLE; record actual provided support rather than claim FUNNELED from the request alone. Numerical +helpers may serialize when nested, but public distributed entry from an active host OpenMP team remains unsupported. +Add no process-wide locking service, caller-thread dispatcher or nested-parallel execution machinery. See the +[MPI initialization/thread contract](https://www.mpi-forum.org/docs/mpi-4.1/mpi41-report/node267.htm), including +MPI_Is_thread_main; identifying a wrong caller does not permit that caller to communicate or invoke MPI_Abort. Lazy initialization, allocation, and cache mutation must be outside worker bodies unless storage is strictly worker-private. Do not assume `const` means thread-safe: both the inverted index and parity cache have lazy mutation @@ -344,8 +514,56 @@ today. ## Performance acceptance and stopping +### Run the existing ladder on both revisions + +[benches/LADDER.md](../../../benches/LADDER.md) is the measurement roadmap, not merely a source of historical labels. +Reuse its benchmark functions, model builders, CLI sizing controls and report outputs. The additional driver is a thin +execution/validation wrapper for the agreed evidence requirements, not a replacement benchmark suite. + +Here use ladder notation: N nodes, C usable physical cores/node, D NUMA domains/node, R ranks/node and P old partitions +per rank. The candidate uses T OpenMP workers/rank; matched comparisons use T=P. For balanced allocations: + +| Rung/control | Single-node candidate shape | Role | +| --- | --- | --- | +| L1 | R=1, T=1 | Required single-thread control for small/medium profiles | +| NUMA-local shared-memory control | R=1, T=C/D | Required small/medium control; do not call it full-node L2a | +| MPI-only control | R=D, T=1 | Required small/medium control, not a fixed-core comparison with L1 | +| L2b | R=D, T=C/D | Required production shape: one rank/domain | +| L2b decomposition variant | R=2D, T=floor(C/(2D)) | Required two-ranks/domain comparison where resources permit | +| L2a | R=1, T=C | Full-node diagnostic, allowed to span NUMA domains | +| L3/L4 | Multiple nodes | Pending proper HPC cluster/interconnect qualification | + +Use actual balanced subsets when allocation is uneven and record unused cores. Small/medium profiles span the required +controls; large acceptance profiles use only the two full-node L2b decompositions. Diagnostic L2a may use the same large +rows, without changing their flags to conceal a capacity/performance failure. If D=1, L2a and L2b coincide: deduplicate +that shape without removing its required acceptance gate. + +After the Task 0 architecture decision and reconciliation of these conditional production gates, the sequence is: + +1. On the baseline only, calibrate each ladder row using its preferred control: Hubbard/Pauli `lower_atol`, random + Heisenberg `obs_terms`, and random Schrödinger `num_generators` (or the documented mode-count sweep). Keep model + structure/seed fixed within each sweep; changing generator count also changes gradient length. Record every override. +2. Size graph rows against the largest build/energy/gradient peak, not build alone; also account for numerical export + and validation. Use existing grouping where it actually shares a graph, not an assumed sharing property. +3. Obtain approval and freeze exact flags, seeds, routing, placements and required cells. Apply the same flags to each + matched rung/arm; pre-freeze workload sizing is not permission to relax oracle tolerances or tune after a regression. +4. Run the existing pytest nodes on baseline and candidate with unique labels, fresh processes and `--bench-rounds=1`. + Preserve the plan's per-operation isolation for formal observations; grouped pilot/diagnostic runs do not certify + fresh construction peaks. Add separate construction and numerical-validation passes using the same model builders. +5. Compare five pairs per required cell, adding five more after a failure, using the five metrics defined below. Reuse + existing reports and record actual term maps/values as well as counts; a term-count match is not numerical proof. + +Keep cross-NUMA L2a output in a separately labelled diagnostic inventory/directory, outside the frozen required-cell +manifest consumed by the acceptance comparator. Its ratios are informative, never a waiver or an acceptance verdict. +Diagnose numerical mismatches even there; diagnostic-only describes the performance gate, not permission for wrong +results. Full-node L2a/L2b contrasts include process/decomposition and local MPI effects. Replacing L2a with a smaller +NUMA-local run changes resources and cannot claim that same contrast. Historical ladder timings/sizes are examples, +not measurements on the target machine. + +### Acceptance evidence and stopping rules + The initial acceptance gate is a frozen campaign on one c8a.metal-24xl, with at most one worker per physical core. -Record the actual socket/NUMA/core layout before selecting geometries. Include one-rank-per-allocated-domain operation +Record the actual socket/NUMA/core layout before selecting required geometries. Include one-rank-per-domain operation and an additional two-ranks-per-domain configuration with disjoint physical cores, where resources permit. Both arms use identical decomposition/allocation in every comparison; changing ranks only on the candidate cannot rescue a regression. Source-reference workloads remain candidates, not approved production sizes. After separate execution authorization, @@ -445,24 +663,29 @@ not measurements; CPU binding alone does not enforce a RAM budget. ## Non-goals -No new executor framework, custom concurrent hash table, GPU backend, load-balancing/repartitioning algorithm, -asynchronous gate pipeline, GIL-policy change, graph-format redesign, new replay transport, or replacement +Beyond the bounded existing-library index experiment, no custom concurrent hash table or new executor framework. +No GPU backend, load-balancing/repartitioning algorithm, asynchronous gate pipeline, GIL-policy change, graph-format +redesign, new replay transport, or replacement topology/affinity-management subsystem. Do not remove useful lifetime/ordering comments merely to lower line count. ## Execution handoff and supported build routes All future code/test/build commands in the companion plan are planned, not executed during this document refresh. -This revision changes planning documentation and the agreed glossary only; no implementation or publication is -authorized. Before implementation, recheck exact HEAD and the plan's baseline-drift -table; stop on changed signatures, routing, capacity, numerical or benchmark contracts. Keep 13 gated tasks and record +This current-main refresh changes the two planning documents only; the existing glossary remains applicable. No +implementation or publication is authorized. The previously reported source drift is audited and incorporated above. +Before runtime work, verify the pinned main revision and the plan's refreshed anchor table; stop on subsequent changes +to signatures, routing, capacity, numerical or benchmark contracts. Keep Task 0 followed by conditional Tasks 1–13; +record RED/GREEN evidence for new behavior, plus baseline results for preservation tests that may already pass. Do not manufacture RED by removing earlier plumbing. Use kernel-specific worker observations for threading participation. Do not execute past a failed gate or infer approval from this refresh. -The baseline-only pilot travels with Task 1 to the authorized execution machine; it need not run on the planning host -before handoff. Transfer the latest spec, plan and CONTEXT.md, not only the historical published planning commit. -The plan's remote entry point defines a bounded first phase ending with a proposed campaign/budget for owner approval, -not candidate engine work or a completed Task 1 gate. A different execution host does not silently change the agreed +The first execution handoff is now the separately authorized Task 0 mini-app, ending at the architecture decision. +Only after that gate does the baseline-only pilot travel with Task 1 to the authorized execution machine; neither +experiment must run on the planning host before handoff. Transfer the latest spec, plan and CONTEXT.md, not only the +historical published planning commit. The Task 1 portion of the remote entry point defines the subsequent baseline-only +phase ending with a proposed campaign/budget for owner approval, not candidate engine work or a completed Task 1 gate. +A different execution host does not silently change the agreed c8a.metal-24xl acceptance target. No machine provisioning, execution or publication is authorized by this clarification. Required OpenMP must link PUBLIC on both `monoprop-objs` and the shared `monoprop` target, plus installed From e88a43600503cf50039d09bd381ab5a1f703d6a1 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Roberto=20Di=20Remigio=20Eik=C3=A5s?= Date: Thu, 24 Sep 2026 12:50:06 +0200 Subject: [PATCH 04/64] chore: further refinement of spec and plan --- .../plans/2026-09-18-rank-local-openmp.md | 279 +++++++++++------- .../2026-09-18-rank-local-openmp-design.md | 144 +++++---- 2 files changed, 265 insertions(+), 158 deletions(-) diff --git a/docs/superpowers/plans/2026-09-18-rank-local-openmp.md b/docs/superpowers/plans/2026-09-18-rank-local-openmp.md index 984ff781..b075327d 100644 --- a/docs/superpowers/plans/2026-09-18-rank-local-openmp.md +++ b/docs/superpowers/plans/2026-09-18-rank-local-openmp.md @@ -24,12 +24,14 @@ execution. **Status:** Planning only. Refreshed source baseline: current `origin/main` at `90d57177c2b0cd93503f88dd931d2f8dd409aa23`, inspected in rebased `refactor-parallelization` at -`9bede897d99ac7e2a53442885629d5032f1b7df4`. Only CONTEXT.md and these planning documents differ between those commits. +`35f359f095fa3f1d628ca2fdc66c93fef6ea7564`. Only CONTEXT.md and these planning documents differ between those commits. The original planning publication was `38eb13d80a95dd1ac60e762f14f68c9a5925f755`. This owner-requested current-main refresh preserves the subsequent approved runtime, compatibility, numerical and failure policies and uncommitted edits. The owner has now explicitly reopened the one-store architecture decision through Task 0. Tasks 1–13 retain the previously approved one-store proposal as a CONDITIONAL path, not an automatic follow-up. Stop after the mini-app for -an owner decision; reconcile both documents before proceeding, including the required MPI-off performance coverage. +an owner decision; reconcile both documents before proceeding. The follow-up interview fixed full-machine MPI-off +library acceptance for small/medium and selected large profiles before seeing results. Task 0 is a staged index-phase +pipeline study, not an isolated hash-container contest or a replacement for those library gates. On the retained one-store path, OpenMP is mandatory; partition configuration is removed immediately; thread budgets are launch-environment-only and captured at construction. The fixed32 capacity reduction remains an explicit consequence, not a restriction that predetermines Task 0's architectural conclusion. @@ -40,7 +42,8 @@ single-instance EC2 campaign is sufficient for initial acceptance; HPC performan Multi-node qualification is pending a proper HPC cluster/interconnect, not a cloud-network substitute. Record actual EC2 topology/software and agree representative workload sizes before freezing the campaign or collecting acceptance data. The owner approved baseline-only, unscored sizing trials before the freeze, once separately authorized. Small/medium -profiles cover the full geometry sweep; large profiles cover only the full-node one- and two-ranks-per-domain cases. +profiles cover the full geometry sweep plus MPI-off full-machine operation. Selected large profiles cover the two +full-node MPI decompositions and MPI-off full-machine operation, not a new all-sizes Cartesian product. No implementation, engine builds/tests or benchmarks are authorized by this revision. All future execution commands below are planned, not evidence of passing gates. EXISTING labels describe the inspected baseline; PROPOSED labels @@ -56,10 +59,10 @@ execution, and retain separate build/calibration authorization, campaign freeze ## Global Constraints -The following constraints describe the conditional one-store path in Tasks 1–13. Task 0 alone permits the explicit -per-thread-store comparison and requires full-machine MPI-off measurements, including cross-NUMA placement. This is -not permission to add shards to production or to waive any existing acceptance gate. Reconcile these constraints with -the owner's post-experiment architecture decision before proceeding. +The storage constraints describe the conditional one-store path in Tasks 1–13. Task 0 alone permits the explicit +per-thread-store experiment, not production shards. Full-machine MPI-off measurements are required both in Task 0 +and in full-library acceptance, including cross-NUMA placement. Reconcile the storage design with the owner's +post-experiment decision without silently weakening that coverage or any numerical/performance gate. - C++23; retain the GCC 14 / Clang 18 minimum compiler versions and Python 3.11 floor. - MPI remains optional and OFF by default; retain working non-MPI wheels. @@ -79,8 +82,9 @@ the owner's post-experiment architecture decision before proceeding. - Do not introduce hidden operator shards, a dense duplicate store, a second persistent full-key store, per-thread operators, or full-vector coefficient double buffering. Select any concurrent index only after correctness/memory/time evidence; its container name is neither a rejection criterion nor proof of suitability. -- Required performance cells keep each rank within one NUMA domain. Full-node cross-NUMA L2a is diagnostic-only and - cannot replace, excuse or average away any required cell; its numerical correctness is still checked. +- Required MPI-enabled cells keep each rank within one NUMA domain; MPI-enabled cross-NUMA L2a is diagnostic-only. + Actual MPI-disabled full-machine operation is required for small/medium and selected large profiles, even across + NUMA domains. Diagnostic results cannot replace, excuse or average away any required cell; correctness still applies. - Preserve Majorana and Pauli bases, Heisenberg and Schrödinger pictures, cutoffs, graph build/replay/paring, gradients, partial contraction, copying, and initial-operator updates. - Final MPI calls execute on the MPI-initializing thread, outside library-created OpenMP regions. Request/validate at @@ -105,9 +109,10 @@ the owner's post-experiment architecture decision before proceeding. Additional execution rules: -- **No runtime work or implementation is authorized by this planning amendment.** Task 0 needs its own implementation, - host and finite measurement-budget authorization; it is not part of Task 1's two-hour baseline-only pilot. After its - architecture gate, obtain separate authorization for bounded baseline calibration/build work before campaign approval. +- **No runtime work or implementation is authorized by this planning amendment.** Task 0's 2-hour and + min(8 GiB, 25% RAM) study limits are approved, not permission to launch. Authorize implementation, host/allocation and + setup/build separately. Task 1 retains its independent 2-hour/15-minute/75% baseline-only pilot. After Task 0's + architecture gate, obtain separate authorization for that baseline calibration/build work before campaign approval. Candidate engine changes require implementation authorization and an owner-approved frozen campaign. Formal baseline collection also waits for that freeze. Simplification without regressions is the goal; reject the architecture if it cannot earn acceptance. The @@ -115,10 +120,11 @@ Additional execution rules: - P (called R in existing code/benchmark geometry) is the user-selected process count; T is the requested threads per process, normally set with `monoprop_NUM_THREADS`. Only an unset override selects the OpenMP runtime default. Users must allocate enough resources and configure rank/thread binding - with OpenMP, `srun --cpu-bind`, and/or `mpiexec` options. Required production cells keep each rank within one NUMA - domain; full-node cross-NUMA L2a is a labelled diagnostic exception, never a parity waiver. Do not call a smaller - NUMA-local run L2a. Documentation recommends one worker per physical core, including when - hardware SMT is enabled. Do not create a topology service/NUMA scheduler, silently change P/T, or enforce that + with OpenMP, `srun --cpu-bind`, and/or `mpiexec` options. Required MPI-enabled cells keep each rank within one NUMA + domain; MPI-enabled cross-NUMA L2a stays diagnostic. Required MPI-off full-machine cells instead span the allocation + in one process. Neither permits a parity waiver or calling a smaller NUMA-local run full-machine. Recommend one + worker per physical core, including when hardware SMT is enabled. Do not create a topology service/NUMA scheduler, + silently change P/T, or enforce that recommendation through library clamping or affinity changes. - Do not touch pre-existing `.codegraph/` or `.direnv/`. Use an isolated implementation checkout and a separately built baseline. Follow the worktree/GitButler skills for version-control writes; no raw git write commands. @@ -209,9 +215,10 @@ Never hand off only “tests pass”. No builds/tests/imports/installs/VCS write ### Remote entry point: Task 0, then conditional Task 1 The initial handoff is Task 0's standalone mini-app, not a full monoprop build or the baseline pilot. Transfer both -planning documents and CONTEXT.md; verify the source pin, authorize the small implementation and a finite execution -budget, and use the selected c8a.metal-24xl unless the owner approves another host. Return results and stop for the -architecture decision. No Task 1 command below runs until that decision and the corresponding document reconciliation. +planning documents and CONTEXT.md; verify the source pin, authorize the small implementation and execution within +Task 0's 2-hour/min(8 GiB, 25% RAM) limits, accounting for setup/build separately. Use c8a.metal-24xl unless +the owner approves another host. Return results and stop for the architecture decision. No Task 1 command below runs +until that decision and the corresponding document reconciliation. No delegation, provisioning, commits or publication are implied by this handoff. After that gate, the baseline-only pilot belongs to Task 1. It need not run on the planning machine before handing this @@ -407,7 +414,11 @@ stock-Python TSan gate, no blanket suppression of engine frames, and no sanitize **Question:** For one process using T physical cores, how do per-thread stores, a shared single-writer index and a shared concurrent index scale under monoprop-like batches? The owner explicitly permits the answer to overturn the -one-store-per-rank choice. This is an index/ownership experiment, NOT a propagation benchmark or production cutover. +one-store-per-rank choice. Compare the index-phase pipeline: ownership transfers and packed-row initialization, +not isolated containers. Owners initialize private rows in parallel; both shared variants initialize rows on the caller. +The concurrent index only parallelizes publication, so it need not fix a serial row-initialization bottleneck. +This is NOT a propagation benchmark or production cutover. Exact key/ID checks establish index-result agreement, not +retained-operator equivalence, which also requires coefficients. **Size boundary and files:** Create only `tools/index-miniapp/{CMakeLists.txt,main.cpp,README.md}`. One translation unit contains the generator, three small adapters, driver and `--self-test`; split no framework out of it. Reuse @@ -427,8 +438,10 @@ OMP_NUM_THREADS="$T" OMP_DYNAMIC=FALSE OMP_PLACES=cores OMP_PROC_BIND=close \ ``` This is a proposed smoke-size invocation, not a frozen measurement size or an executed command. Supported variants are -`per-thread`, `shared-serial`, `shared-concurrent`; modes are `lookup` and `grow`; template widths are 128 and 1024; -key lengths are 6 (inline) and 16 (spilled), with inline width fixed at 11. Capture omp_get_max_threads once and record +`per-thread`, `shared-serial`, `shared-concurrent`; modes are `lookup` and `grow`. The first performance pass uses only +modes=128, key-length=6 and inline width=11. Width 1024 and spilled length-16 fixtures remain small correctness checks; +their performance studies are deferred, not automatic extra cells. Keep any supporting dispatch minimal, not a generic +container/type framework. Capture omp_get_max_threads once and record the actual team. Optional `--profile` enables phase timers in a separate diagnostic run; normal runs time complete batches only, leaving phase fields empty rather than reporting invented zeros. No library environment parser, global OpenMP mutation or new public monoprop option. Reject malformed numbers, out-of-range percentages, impossible key/row @@ -436,10 +449,18 @@ counts and checked-arithmetic overflow before allocation. Keep study sizes below report the architectural capacity difference without billion-row stress tests. A worker failure is caught inside its region, all workers join, then the process reports failure; do not emit a success row. -- [ ] Obtain separate approval for implementation, host/allocation and finite wall-time/RAM limits before execution. - Record source/header and executable hashes, compiler/options, Boost/OpenMP versions, complete commands and placement. - Task 0 neither consumes nor extends Task 1's two-hour pilot. Keep its CSV/logs outside the checkout and separate from - calibration/acceptance artifacts. No runtime measurements or setup are authorized by adding this task to the plan. +- [ ] Obtain separate implementation, host/allocation and execution authorization. The approved Task 0 study limit is + 2 hours total for calibration and measurements, including process initialization, input generation, validation and + phase profiles. Account for setup/build time and costs separately; they need separate authorization. Limit each fresh + process's peak RSS, with setup/growth/checks, to min(8 GiB, 25% of observed usable allocation RAM). Record the + RAM basis and fixed byte ceiling before trials; do not recompute it from falling free memory. Use external monitoring + and + stop on a limit; add no library resource clamp. Keep all attempts and consumed/remaining time across interruptions. + If limits prevent completion, report incomplete evidence; do not extend them or omit losing cells. + Record source/header and executable hashes, compiler/options, Boost/OpenMP versions, commands and placement. Task 0 + neither consumes nor extends Task 1's pilot; its 8-GiB cap does not restrict Task 1's calibrated workloads. + Keep CSV/logs outside the checkout, separate from full-library calibration/acceptance. These limits do not authorize + runtime measurements, setup or launch. - [ ] Add the standalone self-test entry before implementing variants. RED is the absent target/adapter or incorrect expected result, not a broken production test. Use ordinary runtime checks that remain active in Release, not assert alone. The same driver executes tiny cases through all three variants at requested teams 1/2/3/4 on suitable hardware: @@ -534,24 +555,27 @@ add_test(NAME index-miniapp-self-test COMMAND index-miniapp --self-test) compare runs, but are not the correctness oracle. A timed run with incorrect results is invalid, never fast evidence. Run separate ASan/UBSan checks and a qualified OpenMP race check on tiny cases; unsupported tooling stays a reported limitation. Do not use sanitizer timings for scaling or expand into the whole library's test matrix. -- [ ] Start with a compact strong-scaling study: fixed GLOBAL initial rows and total queries across T, never N rows per - thread. At modes=128/key-length=6/batch=4096/seed=0, compare lookup at 0% and 50% misses, and growth at 10% and 100% - misses, using one small and one memory-resident size chosen before comparative timing. Verify the larger footprint - exceeds the relevant last-level cache if claiming DRAM scaling. Choose batches to exercise that footprint, and record - initial/final sizes. Sweep 1/2/4/... physical cores plus the exact domain and allocation endpoints, deduplicated. - Repeat within one domain and across the allocated machine; a one-domain host needs no duplicate series. Use the same - core sets and memory policy for all three variants at a given T. The full-machine MPI-off series is required for this - decision, even across NUMA domains; do not replace it with an MPI run or downgrade it to optional L2a diagnostics. -- [ ] Add only bounded sensitivity checks at T=1, domain-full and machine-full: the same mixed-growth case with - length-16 spilled rows, one with modes=1024 (wider PosT), and one batch=64 case. Run all three variants on the same - inputs; - do not expand every combination into a Cartesian product. If conclusions depend on these cases or are noisy, report - that and request targeted follow-up rather than adding containers, weak-scaling campaigns or a trace subsystem. +- [ ] Run the approved first pass: fixed GLOBAL initial rows and total queries across T, never N rows per thread. + At modes=128/key-length=6/batch=4096/seed=0, use exactly three cases: frozen all-hit lookup, growth with 10% misses, + and growth with 100% misses. Select one small and one larger memory-resident size before comparative timing, within + the common study cap including growth/checking. Verify the accessed footprint exceeds the relevant last-level cache + if claiming DRAM scaling; if the cap prevents that, state the limitation rather than enlarging the budget. Choose + batches to exercise the footprint and record initial/final sizes. Sweep 1/2/4/... physical cores plus exact domain + and allocation endpoints, deduplicated. Repeat within one domain and across the allocated machine; a one-domain host + needs no duplicate series. Use the same core sets and external memory policy for all variants at a given T. The + full-machine MPI-off series is required, even across NUMA domains; an MPI run cannot replace it. +- [ ] Report first-pass results before expanding scope. Defer wider-key, spilled-row performance, batch=64 and other + hit/miss-distribution sensitivities until those results justify an owner-approved follow-up. Keep tiny correctness + fixtures for collisions, both position widths, spilled rows and backing growth; deferring performance cases does not + permit incorrect adapters. Qualify conclusions to measured inputs. Do not add a Cartesian product, weak-scaling + campaign, containers or a trace subsystem; follow-up cannot silently extend the remaining time/RAM limits. - [ ] Verify active worker masks against physical-core/NUMA topology using external tools and OpenMP diagnostics. - Record first-touch behavior: per-owner construction naturally places shard pages differently from caller-built - shared rows. Do not mistake NUMA placement effects for hash-table locking. A matched externally interleaved-memory - rerun may diagnose a reversal, only within the approved budget; never apply it silently to one variant. No internal - affinity setter, hwloc dependency or automatic geometry repair. Unknown placement makes scaling evidence incomplete. + Record natural first-touch: per-owner and caller-built rows can occupy memory differently under the same external + policy. Count that difference as a real ownership/construction consequence; do not normalize away an architectural + disadvantage. Separate locality effects from hash-table synchronization in the report. An explicitly matched + interleaved-memory rerun may diagnose a result within the budget, but cannot replace the original comparison or apply + only to one arm. No internal affinity setter, hwloc dependency or automatic geometry repair. Unknown placement makes + scaling evidence incomplete. - [ ] Emit a plain CSV row per process with full knobs, requested/actual team sizes, initial/final rows, hits/misses, complete-batch and optional phase seconds, queries/second, min/max shard rows, row/spill/index byte estimates where available, and validation outcome. Report setup separately. Measure complete batches without fine-grained timers, @@ -584,8 +608,10 @@ current-library baseline. Do not claim propagation, replay, gradients, MPI or fu If an arm is unavailable, placement is unverifiable, correctness fails or the budget ends, report incomplete evidence; never declare the remaining arm the winner by default. Stop for the owner's decision to retain a shared-store path, retain/reconsider sharding, or gather specified additional evidence. Record that decision and revise the spec and -Tasks 1–13 accordingly BEFORE proceeding, including MPI-off performance requirements. No fixed mini-app speedup -threshold replaces that decision or weakens the five full-library gates. Its outputs are never formal Task 1 samples. +Tasks 1–13 accordingly BEFORE proceeding. Preserve the already-approved required MPI-off full-machine library coverage +across all three size tiers; do not choose protected configurations after seeing which architecture wins. No fixed +mini-app speedup threshold replaces the owner decision or weakens the five full-library gates. Task 0 outputs are never +formal Task 1 samples, and its whole-process RSS cannot certify the library's operation/construction windows. ### Task 1: Freeze baselines and repair measurement trustworthiness @@ -603,14 +629,17 @@ elsewhere in this plan). Map old P to candidate T workers/rank. On the single ta | MPI-only control | R=D, T=1 | Required small/medium, with fewer active cores than full-node L2b | | L2b | R=D, T=C_dom | Required one-rank/domain production shape | | L2b variant | R=2D, T=floor(C_dom/2) | Required two-ranks/domain shape when C_dom>=2 | -| L2a | R=1, T=C | Full-node diagnostic; cross-NUMA allocation expressly allowed | +| L2a (MPI-enabled) | R=1, T=C | Full-node diagnostic; cross-NUMA allocation expressly allowed | +| MPI-off full machine | R=1, T=C, has_mpi=false | Required small/medium and selected large profiles, including cross-NUMA | | L3/L4 | N>1 | Pending proper-interconnect HPC qualification | D is allocated NUMA domains, C_dom the selected cores/domain, C=D*C_dom. Record actual balanced subsets and unused cores -rather than infer topology. If D=1 makes L2a identical to required L2b, deduplicate the shape -without downgrading its required status. Large acceptance profiles use the two L2b decompositions only. A corresponding -L2a diagnostic must retain the same workload flags even if it cannot fit; report that limitation instead of quietly -shrinking it. The NUMA-local control cannot substitute for full-node L2a when interpreting L2a/L2b differences. +rather than infer topology. If D=1 makes MPI-enabled L2a identical to required L2b, deduplicate within that build mode +without downgrading required status. Never deduplicate MPI-off against MPI-enabled evidence, even at the same R/T. +Selected large profiles use both L2b decompositions AND actual MPI-off full-machine operation. Calibrate common profile +sizes using the MPI-off baseline too; a post-freeze capacity failure blocks acceptance, not permission to shrink a +workload or drop its cell. MPI-enabled diagnostic L2a retains the same flags even if it cannot fit. The NUMA-local +control cannot substitute for full-machine operation in either build mode. The practical workflow is baseline-only ladder sizing → owner-approved frozen cells/flags → the same pytest nodes on both arms at matched shapes → existing reports plus the five parity metrics and numerical evidence below. Prefer the @@ -625,8 +654,10 @@ still use fresh processes and --bench-rounds=1 to avoid prior live graphs and fi separate construction worker measures setup explicitly. Reuse the existing functions, builders and report files; observe/validate/compare only supply the missing evidence joins/checks, not another benchmark framework. -Keep cross-NUMA L2a in a separately labelled diagnostic inventory/output directory, outside the frozen required-cell -manifest supplied to compare. Never reclassify a failed required cell as diagnostic or use L2a ratios to waive it. +Keep MPI-enabled cross-NUMA L2a in a separately labelled diagnostic inventory/output directory, outside the frozen +required-cell manifest supplied to compare. MPI-off full-machine cells belong IN that manifest, with unchanged +numerical checks and all five parity gates. Never reclassify a failed required cell as diagnostic or use L2a ratios +to waive it. Numerical mismatches in diagnostic runs still require investigation. Historical ladder results are sizing examples, not target-machine acceptance evidence. Update its shape/declaration guidance with the candidate cutover. @@ -667,8 +698,9 @@ numactl --hardware grep -E 'Cpus_allowed_list|Mems_allowed_list' /proc/self/status ``` -Use these observations and per-rank placement evidence to select NUMA-contained acceptance geometries. A 1x96 shape is -canonical L2a only with all 96 allocated physical cores; if it crosses domains, label it diagnostic-only. Missing +Use these observations to select NUMA-contained MPI acceptance and full-allocation MPI-off geometries. A 1x96 shape is +full-machine only with all 96 allocated physical cores. If an MPI-enabled run spans domains, it is diagnostic L2a; +in an actual MPI-disabled build, full-machine operation is required whether or not it crosses domains. Missing diagnostic tools require another external observation method, not a library topology dependency. Record hardware SMT state, but configure one worker per physical core. Do not add SMT-sibling profiles; HPC/multi-node qualification remains separate and pending. @@ -676,13 +708,14 @@ separate and pending. - [ ] Obtain separate authorization for baseline build/calibration work. On c8a.metal-24xl, record sockets/NUMA domains, physical cores, hardware SMT state, memory allocation and software stack before choosing geometries. Agree representative workloads/sizes and single-instance routing/rank/thread shapes. Initial acceptance is the EC2 campaign, - including a two-ranks-per-domain case; all workers use distinct physical cores. Production-HPC performance and + including a two-ranks-per-domain case and actual MPI-off full-machine operation in all three selected size tiers; + all workers use distinct physical cores. Production-HPC performance and multi-node qualification on a proper interconnect remain pending, not prerequisites for initial EC2 acceptance. Production's common eight-domain layout is not a hard-coded EC2 topology. Historical profiles below are candidates, not an approved campaign. Confirm the bounded new paring-construction measurement in that campaign. Record baseline - revision, allocation, compiler/flags, MPI version, index width, OpenMP runtime, allocator environment, imported - extension path/hash, seed and complete workload configuration before collecting evidence. + revision, allocation, compiler/flags, has_mpi, MPI version where applicable, index width, OpenMP runtime, allocator + environment, imported extension path/hash, seed and complete workload configuration before collecting evidence. - [ ] Before changing engine code, separately build the baseline using R/M. Preserve its environment and binary. Apply only the identical measurement-harness changes to both arms; record this overlay's diff/hash. Use the refreshed main pin for the baseline and the candidate's starting engine, not the former routing-branch snapshot. Main's basis @@ -702,7 +735,9 @@ separate and pending. Do not run/consult candidate performance to select the inventory. Estimate formal campaign time/cost from the pilot, then seek owner approval of sizes, coverage and campaign expenditure before the freeze and formal collection. Do not reduce required samples/coverage to fit an unapproved budget. Collect fresh formal baseline samples after the freeze. - Never promote calibration runs, even with unchanged parameters. Approval of these limits is not execution permission. + Never promote calibration runs, even with unchanged parameters. Include actual MPI-disabled baseline feasibility + for small/medium and selected large profiles before freezing shared sizes. No candidate-influenced resizing or + removal after freeze. Task 1 limits are independent of Task 0's 8-GiB study; approval is not execution permission. - [ ] Add renderer tests independently varying operation and outer exactness: true/false/absent and mixed flags. `bmf.py::build_bmf` publishes `peak-memory` from `memhwm`, so gate it on `memhwmexact`; `operation-memory` comes from `opmemdelta`, so gate it on `opmemexact`. `report.py::build_report` must label each corresponding metric correctly. @@ -725,6 +760,15 @@ separate and pending. partition setting in the measurement environment, wrong observed R/T, mislabeled arm, and valid explicit/fallback candidate declarations. This strict measurement preflight does not restore a removed production getenv reader. RED: current multi-rank guard rejects candidate configuration. +- [ ] Include the build mode in shared preflight and evidence. Read monoprop.has_mpi from the imported extension, + record it with the extension hash, and require it to equal identity.has_mpi for both arms. Actual MPI-off cells have + has_mpi=false and R=1; a single rank of an MPI-enabled binary cannot substitute. The current conftest imports mpi4py + whenever installed: gate that import on monoprop.has_mpi BEFORE touching mpi4py.MPI. For MPI-off, use comm=None and + launch directly, with no MPI launcher/initialization or invented MPI thread-support/version values. Apply the same + guard in the new driver. Use the identical overlay on both arms, without changing the installed Python API. + Tests must cover mpi4py present/absent with has_mpi=false and a trap on attempted MPI import, unchanged MPI-enabled + fixture behavior, requested build-mode mismatch, missing build-mode evidence and MPI-off declarations with R>1. + Neither an arm label nor absence of mpiexec proves the build mode. - [ ] Separate whole-construction measurement is intentional policy, not a repair for nesting. A fresh untimed worker, without pytest memory fixtures/session graphs, opens one `HighWaterMark`, constructs inputs/propagator/graph/callable as required, runs the operation and retains outputs until close. Record peak sum/max, floor, delta and exactness. Use @@ -733,20 +777,20 @@ separate and pending. JSON schema is `{"schema_version":1,"workloads_path":str,"workloads_sha256":str,"cells":[cell]}`. Each cell has unique slug `id`, `profile`, `node_id`, `operation`, `measurement_kind` (`pytest` or `driver`), `identity`, `config_digest` and - `parameters_digest`. `identity` contains `ranks`, requested `threads`, `index_bits=32`, `cpu_allocation`, - `compiler_flags`, resolved `config`, and `routing` (`mode`, derived `bits`, numeric `seed`). Expand the complete - size-band-specific inventory below: small/medium profiles use the full geometry sweep, with the existing routing - coverage requirements; selected large profiles use only full-node one- and two-ranks-per-domain geometries. Do not - generate an all-sizes Cartesian product. Archive the inventory/digest with owner approval. This file, not later - observations, is the authoritative expected-cell list. A full cell missing + `parameters_digest`. `identity` contains boolean `has_mpi`, `ranks`, requested `threads`, `index_bits=32`, + `cpu_allocation`, `compiler_flags`, resolved `config`, and `routing` (`mode`, derived `bits`, numeric `seed`). Expand + the size-band-specific inventory below: small/medium profiles use the full geometry sweep and existing routing + requirements plus MPI-off full-machine operation. Selected large profiles use the two full-node MPI decompositions + and MPI-off full-machine operation. Do not generate an all-sizes Cartesian product. Archive the inventory/digest + with owner approval. This file, not later observations, is the authoritative expected-cell list. A full cell missing from BOTH arms must fail completeness. Reject duplicate/extra cell IDs and mismatched identities. Do not regenerate or shrink the inventory to make observations pass. - [ ] `tools/rank-local-openmp-workloads.json` maps `profiles[profile][node_id]` to `operation`, `measurement_kind`, `basis`, `picture`, complete `config`, deterministic `parameters`, and `pare_threshold`. `tiny` is the small band and calibrated `reference` is medium; each band's `-pared` variant shares its geometry policy. Preserve the existing unpared four-family and threshold-1e-10 energy/gradient plus driver-construction coverage. Add `large`/`large-pared` - entries for the owner-selected large workloads, restricted to the two full-node decompositions. Profile names do not - freeze the historical reference sizes: record every calibrated override explicitly, with no ambiguity within a + entries for the owner-selected large workloads at both full-node MPI decompositions and MPI-off full-machine shape. + Profile names do not freeze historical reference sizes: record every calibrated override, with no ambiguity within a profile/node pair. Define config/parameter digests as SHA-256 of UTF-8 `json.dumps(value, sort_keys=True, separators=(",", ":"), allow_nan=False)`. File digests are SHA-256 of exact file @@ -761,8 +805,8 @@ separate and pending. node IDs. Environment/launcher/allocation stay external; the driver installs nothing. - [ ] Define the external placement JSON consumed by `--placement`: `schema_version=1`, `artifact_kind="placement"`, `arm`, `cell_id`, `binary_hash`, `identity`, `runtime_shape`, `declared`, and `observed`. `declared` contains `ranks`, - `threads`, `cpu_allocation` and `settings` (the exact OpenMP/library environment). `observed` contains `ranks`, - `threads_per_rank` and `cpu_allocation`, from Task 11's binary-linked representative large-kernel diagnostic. Preserve + `threads`, `cpu_allocation` and `settings` (the exact OpenMP/library environment). `observed` contains `has_mpi`, + `ranks`, `threads_per_rank` and `cpu_allocation`, from Task 11's binary-linked large-kernel diagnostic. Preserve raw launcher/affinity evidence alongside this summary. Baseline shape is `partitions`; candidate shape is `openmp`; both must match the campaign R/T/allocation. A limited-team diagnostic cannot establish the requested full-team benchmark cell. Small-kernel serial fallback remains allowed and reported. Each artifact records `placement_path` and @@ -775,8 +819,10 @@ separate and pending. records schema version, artifact kind, arm, cell/sample IDs, campaign digest, binary hash, profile/node/operation, identity, config/parameter digests and placement reference/hash. Keep pytest raw files in that sample's timed subdirectory so labels cannot overwrite another sample. Timed artifacts contain runtime, operation and outer-window - fields only; construction artifacts contain construction-window fields only. Do not invent timing for the untimed - worker. `validate` writes a separate uniquely named validation artifact under `DIR/ARM/CELL`, without sample ID. + fields only; construction artifacts contain construction-window fields only. Every timed, construction and validation + artifact records the imported binary's observed has_mpi as well as the expected identity; compare both to the cell and + placement evidence. Do not invent timing for the untimed worker. `validate` writes a separate uniquely named + validation artifact under `DIR/ARM/CELL`, without sample ID. - [ ] Define comparison input as `{"schema_version":1,"campaign_sha256":str,"cells":[{"id":str,"baseline":[sample],"candidate":[sample]}]}`. Each `sample` contains `sample_id`, `timed_path`, `construction_path`, `validation_path`. `compare` first verifies the @@ -795,7 +841,8 @@ separate and pending. unknown; missing required operation/construction exactness fails acceptance. Include operation/construction floor/delta sum/max and persistent bytes as diagnostics, never substitutes for absolute peaks. Keep original raw artifact metric names: operation peaks come from opmempeak; outer peaks from memhwm/memhwm_max. Compare exactly FIVE - per-cell medians: runtime, operation peak sum/max, construction peak sum/max. All five ratios must be <=1.00. Report + per-cell medians: runtime, operation peak sum/max, construction peak sum/max. All five ratios must be <=1.00. At R=1 + with MPI disabled, sum and max coincide; retain all five fields/gates and the same numerical oracles. Report outer ratios separately; an outer-only regression cannot fail this five-metric gate. Reject missing/non-finite required measurements rather than use zeros. Derive numerical success by reading validation products below; a caller-supplied boolean is insufficient. Exit 0 means all cells pass, 1 means unmet gates (including incomplete @@ -873,8 +920,9 @@ uv run --no-sync python tools/benchmark-rank-local-openmp.py observe \ Require equal global key sets, including zero-coefficient stored terms; compare values with the existing `test_utils::near` rule from `cpp/tests/TestUtilities.h.in`: `abs(a-b) <= 1e-9 + 1e-7*max(abs(a),abs(b))`, componentwise for complex values and gradients. Reject missing/non-finite outputs. Preserve all stricter existing - unit-test tolerances; this benchmark comparison does not weaken them. If full validation export cannot fit, stop for - an approved smaller equal-arm profile, not an unchecked `numerical_ok=true`. + unit-test tolerances; this benchmark comparison does not weaken them. During pre-freeze calibration, export failures + may require an approved smaller equal-arm profile. After freeze they block acceptance, not permission to resize or + supply an unchecked `numerical_ok=true`. - [ ] Link every performance observation to a successful validation artifact for the same binary/configuration/operation and compare baseline/candidate validation artifacts. Add tests with wrong energy, a changed gradient entry, one missing/extra term, duplicate ownership, replicated identity, non-finite data, stale binary hash and absent @@ -906,7 +954,7 @@ def test_comparison_does_not_hide_one_regression(tmp_path): config = {"gen_length": 4, "obs_terms": 16, "num_generators": 8, "num_modes": 8, "cutoff": 6, "seed": 0, "lower_atol": None} parameters = [0.0] * 8 - identity = {"ranks": 1, "threads": 2, "index_bits": 32, "cpu_allocation": "test-cpus", + identity = {"has_mpi": False, "ranks": 1, "threads": 2, "index_bits": 32, "cpu_allocation": "test-cpus", "compiler_flags": "test", "config": config, "routing": {"mode": "linear", "bits": 0, "seed": 0}} expected, entries = [], {} @@ -937,9 +985,10 @@ def test_comparison_does_not_hide_one_regression(tmp_path): "runtime_shape": shape, "declared": {"ranks": 1, "threads": 2, "cpu_allocation": "test-cpus", "settings": settings}, - "observed": {"ranks": 1, "threads_per_rank": [2], "cpu_allocation": "test-cpus"}} + "observed": {"has_mpi": False, "ranks": 1, "threads_per_rank": [2], + "cpu_allocation": "test-cpus"}} placement_path = save(f"{name}-{arm}-placement.json", placement) - common = {"schema_version": 1, "campaign_sha256": digest(campaign), "arm": arm, + common = {"schema_version": 1, "campaign_sha256": digest(campaign), "arm": arm, "has_mpi": False, "cell_id": name, "profile": target["profile"], "node_id": target["node_id"], "operation": "energy", "binary_hash": binary_hash, "identity": identity, "config_digest": target["config_digest"], "parameters_digest": target["parameters_digest"], @@ -978,9 +1027,11 @@ def test_comparison_does_not_hide_one_regression(tmp_path): These two miniature inventory entries use empty-map energy fixtures, not real benchmark measurements. They follow the same schema, reference loading and provenance checks as real artifacts; no synthetic-mode bypass is allowed. The comparison test does not run physics, acquire placement, or establish actual hardware participation. Add tests for a -whole required cell deleted from both arms (including a large-profile cell), missing samples, mismatched -timed/construction pairs, reused measurement files/run IDs, raw calibration output offered as an acceptance artifact, -missing placement provenance (exit 2), each false/absent exactness flag, unequal +whole required cell deleted from both arms (including a large MPI-off full-machine cell), missing samples, mismatched +timed/construction pairs, reused files/run IDs and calibration output offered as acceptance evidence. Also reject +missing/mismatched has_mpi between inventory, artifacts, placement and binaries; MPI-off declared at R>1; and a D=1 +attempt to deduplicate MPI-off cells against MPI-enabled cells. Check missing placement provenance (exit 2), +each false/absent exactness flag, unequal shape/configuration/index width, and sum-versus-max peaks. For only an outer-peak regression or unknown outer exactness, report the diagnostic accurately while allowing the five-metric gate to pass. Operation/construction inexactness still fails. The comparator reports that five more observations are needed on first ratio failure; it launches no jobs. @@ -1015,20 +1066,23 @@ configurations in artifacts, not just a profile label. `--hubbard-trotter-steps` profile on both arms before formal baseline collection. A skipped cell is not a pass. Do not silently omit graph-heavy cells or bypass the guard on an unverified allocation. - [ ] Select required geometries from observed physical cores, not vCPU/SMT-sibling counts. For D allocated domains with - C_dom cores each, use `(R,T)=(1,1),(1,C_dom),(D,1),(D,C_dom)`, deduplicated, for small/medium profiles. Required R=1 - controls use one domain. Add `(2*D,floor(C_dom/2))` with disjoint cores when C_dom>=2; record unused cores. Apply - existing routing coverage requirements too. Selected large acceptance profiles use only `(D,C_dom)` and - `(2*D,floor(C_dom/2))` from the full-node allocation. Freeze allowed cells per profile; + C_dom cores each, keep `(R,T)=(1,1),(1,C_dom),(D,1),(D,C_dom)` for small/medium profiles. These R=1 controls use one + domain. Add `(2*D,floor(C_dom/2))` with disjoint cores when C_dom>=2; record unused cores. Preserve MPI routing + coverage. Add actual MPI-off `(1,C)` for every small/medium profile and the owner-selected large/large-pared profiles. + Selected large acceptance uses `(D,C_dom)` and `(2*D,floor(C_dom/2))` with MPI, plus `(1,C)` without MPI. Freeze + has_mpi and geometry per cell; deduplicate only within the same build mode and complete identity. In particular, + D=1 never lets an MPI-enabled sample replace required MPI-off evidence. Freeze allowed cells per profile; do not reclassify profiles or drop cells after a regression. Uneven domains require approved balanced subsets, not - assumed C_dom=96/D. Each old/new pair has identical R/T/core allocation. Old throughput uses R ranks x T partitions; - retain an old single-store serial control but never compare only against forced-serial old code. -- [ ] Run canonical full-node L2a separately as a labelled diagnostic using the same approved model flags at R=1,T=C. - It may cross NUMA domains; verify its complete allocation and one worker/core rather than asserting containment. - Use separately budgeted existing-suite runs/reports, not required-cell artifacts or an extra driver mode. Do not - substitute NUMA-local R=1,T=C_dom and still call it L2a. Record failed/unavailable diagnostics honestly; no diagnostic - ratio is a parity gate or waiver. Required L2b geometry remains NUMA-contained; L3/L4 remain pending. Preserve the - existing node IDs/term-count fields and add runtime-shape/actual-team metadata. Counts must agree, but the global - term-map/value checks below remain necessary. + assumed C_dom=96/D. Each pair has identical build mode, R/T/cores and inputs. Baseline uses T partitions per process; + never compare only against forced-serial old code. Launch MPI-off directly after verifying not monoprop.has_mpi; + its one process spans the allocation without an MPI launcher or fictitious NUMA containment. +- [ ] Run MPI-enabled canonical full-node L2a separately as a labelled diagnostic with the same model flags at R=1,T=C. + It may cross NUMA domains; verify complete allocation and one worker/core rather than claiming containment. Keep its + separately budgeted outputs outside required-cell artifacts, not in an extra driver mode. Do not substitute a smaller + NUMA-local run or relabel the required MPI-off `(1,C)` cell as diagnostic. Record failed/unavailable diagnostics; + no diagnostic ratio is a parity gate or waiver. Required MPI L2b stays NUMA-contained; L3/L4 remain pending. Preserve + existing node IDs/term-count fields and add runtime-shape/build-mode/actual-team metadata. Counts alone cannot replace + global term-map/value checks. - [ ] Use `just capture-baseline` / `just diff-baseline` and `tools/capture-baseline.py --compare` only as supplementary small-fixture regression evidence. Its keys include rank, tolerance differs and it lacks operation gradients/provenance; it cannot replace the ownership-aware global-map comparator or independent @@ -1882,22 +1936,32 @@ no container-plugin framework. Do not rename `bench_*` functions or move benchma recommended/acceptance configuration. A wider affinity mask is not evidence of extra worker capacity. Record distinct physical-core ownership from verified sibling topology across active ranks/workers, not only disjoint logical CPU IDs. No required hwloc dependency for the library or measurement helper. -- [ ] For required cells, the benchmark operator verifies that rank masks and worker places stay within one NUMA domain - and the reserved allocation, with no overlap. The separately labelled full-node L2a diagnostic may cross domains; - record actual domain/core masks instead of falsely certifying containment. Neither check is library policy. Advise at - least one MPI process per allocated NUMA domain, with disjoint cores when several processes share a domain. Use actual - topology, not a constant four domains/socket; partial-node runs cover only their allocated domains. +- [ ] For required MPI-enabled cells, verify that rank masks and worker places stay within one NUMA domain and the + reserved allocation, with no overlap. Required MPI-off full-machine cells and diagnostic MPI-enabled cross-NUMA L2a + instead record the complete allocation and actual domain/core masks; do not falsely certify containment. In every + case verify distinct physical-core use. These checks are measurement requirements, not library policy. For MPI + deployments advise at least one process per allocated NUMA domain, with disjoint cores when several share a domain. + Use actual topology, not a constant four domains/socket; partial-node runs cover only their allocated domains. Do not require one logical CPU per worker mask. Record idle/small-kernel serial fallback versus representative large-kernel actual team size separately. Unverifiable placement blocks benchmark evidence, not library construction. - [ ] Compare old partitions vs final new threads at identical R,T and routing settings, using Task 1's frozen - size-band-specific inventory. Small/medium profiles cover the full sweep; large profiles only the two full-node - decompositions. Do not add/remove required cells or reinterpret profile sizes after seeing candidate performance. + size-band-specific inventory and equal has_mpi. Small/medium cover the full sweep plus MPI-off full-machine operation; + selected large profiles cover both full-node MPI decompositions and MPI-off full-machine operation. Do not add/remove + required cells, change build modes or reinterpret profile sizes after seeing candidate performance. Freeze the production routing mode in that matrix; additionally require the random reference profile under explicit splitmix at R=3 and R=4, plus default linear at R=4 so ownership-dependent imbalance is measured, not only unit-tested. Run five alternating **fresh-process** observations per arm/cell; external repetition only, `--bench-rounds=1`. Include first-call/cold cache and separately measured repeated warm calls without overlapping object construction. +For required MPI-off full-machine observations, select the R (MPI-disabled) build on both arms and verify with +`uv run --no-sync python -c 'import monoprop, sys; sys.exit(int(monoprop.has_mpi))'` (exit 0 required). +Use Task 1's direct driver invocation without mpiexec. Set T=C with matching physical-core allocation and OpenMP places; +the baseline still declares both +legacy T variables. The shared overlay must avoid importing mpi4py.MPI even if it is installed. Record has_mpi=false +from the imported binary in timed/construction/validation/placement evidence, not just the requested build setting. +The usual five fresh observation pairs, numerical oracles and all five ratios apply, including selected large profiles. + Example candidate Open MPI invocation after selecting R (= P), K (ranks/domain), T, LABEL and absolute RESULTS. Use K=1 or K=2 for the agreed decomposition profiles. Choose a compatible placement for routing-specific R=3/R=4 cells; do not force ppr:1:numa onto an allocation with too few NUMA domains. All placements require observed disjoint cores. @@ -2178,13 +2242,15 @@ nix develop --command bash -euo pipefail -c ' and pared graphs. Users choose P/T at launch and bind with OMP_PLACES+OMP_PROC_BIND, srun --cpu-bind and/or mpiexec. Normally use monoprop_NUM_THREADS=T; only unset selects the OpenMP runtime default. Present-invalid is an error. Capture once during construction, with no reload/setter. Describe no hardware clamp/topology discovery/NUMA - validation, possible smaller OpenMP teams, and the within-one-domain allocation assumption. Show matching allocation, - rank-binding and thread-place examples, including the one-core-rank-mask pitfall and separately labelled cross-NUMA - L2a diagnostic exception. Document one controlling host caller on MPI's initializing thread outside host OpenMP teams, + validation and possible smaller OpenMP teams. Distinguish NUMA-contained required MPI cells, MPI-enabled cross-NUMA + L2a diagnostics, and required MPI-off full-machine operation spanning domains. Show matching allocation, rank-binding + and thread-place examples, including the one-core-rank-mask pitfall and direct MPI-off execution. Document one + controlling host caller on MPI's initializing thread outside host OpenMP teams, no overlapping use of dependent state or host MPI calls, FUNNELED-or-higher support, caller checks, callable lifetime, enforced failed-object invalidation, accepted capacity reduction and numerical contracts. Placement verification for - benchmarks does not add a topology dependency. Advise at least one MPI process per allocated NUMA domain and identify - actual topology externally; production's common four-domains/socket layout is not universal. Distinguish the selected + benchmarks does not add a topology dependency. For MPI deployments advise at least one process per allocated NUMA + domain; do not tell MPI-off users to add unavailable MPI ranks to avoid regressions. Identify topology externally; + production's common four-domains/socket layout is not universal. Distinguish the selected c8a.metal-24xl initial acceptance from pending Intel/AMD HPC qualification. Explicitly recommend against SMT workers, even when hardware SMT is enabled, and demonstrate launcher plus OpenMP settings for one worker per physical core. Multi-node performance qualification remains pending a proper HPC cluster/interconnect; do not claim it from EC2 @@ -2226,8 +2292,9 @@ nix develop --command bash -euo pipefail -c ' - [ ] Rank-wide limits and verified one-worker-per-physical-core placement satisfy the single-instance campaign, including its two-ranks-per-domain case. HPC-family performance and proper-interconnect multi-node qualification are explicitly pending, not falsely passed or silently included in the initial acceptance claim. -- [ ] Every required runtime and exact peak-memory cell passes Task 11; unsupported or missing measurements are not - marked passed. +- [ ] Every required runtime and exact peak-memory cell passes Task 11, including actual MPI-off full-machine + small/medium and selected large profiles. All five metrics and numerical checks apply; a NUMA-local win cannot + excuse a full-machine regression. Unsupported or missing measurements are not marked passed. - [ ] No legacy runtime, direct hwloc dependency, replacement topology/pinning service or scheduler remains. A selected Boost concurrent index passed its bounded tests and affected parity gates, with no duplicate operator/full-key store. MPI requests/requires FUNNELED and checks the initializing caller even at higher provided levels. diff --git a/docs/superpowers/specs/2026-09-18-rank-local-openmp-design.md b/docs/superpowers/specs/2026-09-18-rank-local-openmp-design.md index 77fd3f9c..956c44b6 100644 --- a/docs/superpowers/specs/2026-09-18-rank-local-openmp-design.md +++ b/docs/superpowers/specs/2026-09-18-rank-local-openmp-design.md @@ -7,7 +7,7 @@ direction discussed with the owner; it is not a claim that the proposed implemen Source inspected for this owner-requested current-main refresh: `origin/main` at `90d57177c2b0cd93503f88dd931d2f8dd409aa23`, present in rebased `refactor-parallelization` at -`9bede897d99ac7e2a53442885629d5032f1b7df4`. Only CONTEXT.md and the two planning documents differ between those commits. +`35f359f095fa3f1d628ca2fdc66c93fef6ea7564`. Only CONTEXT.md and the two planning documents differ between those commits. EXISTING means this refreshed source snapshot; PROPOSED means future implementation. The original planning publication was `38eb13d80a95dd1ac60e762f14f68c9a5925f755` in `refactor-parallelization`. @@ -20,6 +20,8 @@ These are design approvals, not implementation or publication authority for this stores, one shared non-concurrent store and one shared concurrent index. Its evidence can overturn the one-store choice. The one-store design and Tasks 1–13 below are conditional, not an instruction to proceed regardless of that result. After Task 0, stop for the owner's architecture decision and reconcile both documents before any downstream execution. +The follow-up interview approved a staged index-phase study and required full-machine MPI-off library acceptance, +including selected large profiles. These requirements are fixed before results, not deferred to the winning design. This amendment authorizes planning only, not mini-app implementation, measurements or remote expenditure. The owner selected one AWS EC2 `c8a.metal-24xl` for implementation/testing and the initial acceptance campaign. @@ -29,8 +31,9 @@ Passing the frozen EC2 campaign is sufficient for initial acceptance; production until separately qualified. Multi-node qualification is pending a proper HPC cluster/interconnect, not a second EC2 instance. Observed EC2 topology/software and representative workload sizes still need recording/approval before freezing that campaign. The owner approved baseline-only, unscored feasibility calibration before the freeze, once execution is -separately authorized. Small/medium profiles cover the full geometry sweep; large profiles cover only the full-node -one- and two-ranks-per-domain configurations. This revision authorizes no runtime work or implementation. +separately authorized. Small/medium profiles cover the full geometry sweep plus full-machine MPI-off operation; +selected large profiles cover the two full-node MPI decompositions and full-machine MPI-off operation. This revision +authorizes no runtime work or implementation. **Source-drift audit completed:** the owner-requested refresh replaces the superseded `290112c8289ab8015eb9a2c7651ac409839c5a88` pin with the main revision above. The audit covered all 43 changed paths, @@ -64,16 +67,17 @@ First investigate MPI-off index scaling with Task 0 before choosing the ownershi proposal is to replace thread-owned operator partitions and their in-process communication protocols with one compact operator store per MPI rank, operated on by OpenMP worksharing. Preserve mathematical features, memory footprint and performance. Simplification without regressions is the objective; neither one-store variant is a predetermined winner. -If the owner retains the one-store proposal, its production acceptance assumes each -rank's CPUs lie within one NUMA domain. Full-node, cross-NUMA L2a is an explicitly allowed diagnostic exception, not a -required performance gate. The user chooses processes P and threads per process T, and configures the launcher and -OpenMP runtime to bind/pin processes and threads correctly. The library neither discovers topology nor repairs -placement. +Required MPI-enabled acceptance cells keep each rank's CPUs within one NUMA domain; MPI-enabled full-node cross-NUMA +L2a remains a diagnostic. Separately, a real MPI-disabled build using the full machine is a required library gate, +even across NUMA domains. A NUMA-local win cannot excuse an MPI-off full-machine regression. The user chooses +processes P and threads per process T and configures the launcher and OpenMP runtime to bind/pin correctly. The library +neither discovers topology nor repairs placement. ## Preliminary architecture experiment (Task 0) -Keep the mini-app an index exercise, not a second propagation engine: one C++ translation unit, standalone CMake file -and short README under tools/index-miniapp/. Reuse the current OperatorIndex and monomial hashing headers. No library +Keep the mini-app an index-phase pipeline experiment, not a second propagation engine: one C++ translation unit, +standalone CMake file and short README under tools/index-miniapp/. Reuse the current OperatorIndex and monomial hashing +headers. No library build/import, MPI, ShmComm, graph, coefficients, inverted index, physics/cutoff logic, trace capture/replay service, custom scheduler or benchmark framework. A few local adapter functions, flat batch buffers, self-tests and CSV output are sufficient. Stop and reduce scope if implementing the experiment requires a production storage refactor. @@ -87,7 +91,9 @@ Use one process and one OpenMP team of T workers, including its primary thread. | shared-concurrent | One packed row store and one Boost concurrent index | Parallel probes; caller assigns IDs/initializes rows; parallel index publication | The per-thread variant models current storage ownership, not the legacy runtime's performance. All variants use the -same OpenMP harness and global keys/queries. Keep the real packed-row representation, batched position lookup, cached +same OpenMP harness and global keys/queries. Ownership transfers and packed-row initialization belong in batch timing; +this is not an isolated container-throughput contest. Shared rows remain caller-initialized, so a concurrent index +need not fix a row-initialization bottleneck. Keep the real packed-row representation, batched position lookup, cached 32-bit hashes, exact collision checks and fixed32 row references. For Boost use stable row handles and heterogeneous query lookup, not persistent full monomial keys. A row-only use of OperatorIndex may retain its constant empty 16-slot table, accounted as harness overhead; never populate it or allocate a second size-dependent index. This narrow @@ -100,31 +106,45 @@ the proposed engine uses separate phases. Include a small flat-buffer owner-buck sharded variant and report its cost separately. Do not simulate the old message transport or claim those transfers are free. Local IDs may differ between sharding geometries; decoded keys/results and within-geometry ID order must agree. -Use deterministic synthetic position lists: fixed global row/query counts across thread counts, controlled hit/miss -ratios, short and spilled rows, and natural growth. No full trace corpus or synthetic physics. Begin with a compact -strong-scaling sweep, not a Cartesian product of every knob. Measure both within one NUMA domain and across the -allocated machine with one worker per physical core. Full-machine MPI-off results are essential Task 0 evidence, not -optional merely because the conditional production campaign calls cross-NUMA L2a diagnostic-only. Verify placement -externally and record first-touch/memory policy; do not add topology or affinity management to the app. +Stage the study. The first performance pass uses 128 modes, inline length-six keys, and exactly three cases: frozen +all-hit lookup, growth with 10% misses, and growth with 100% misses. Use two working-set sizes, fixed global row/query +counts across thread counts, deterministic synthetic position lists and natural growth. No trace corpus or synthetic +physics. Keep small collision, spilled-row and row-growth correctness checks; defer wider-key, spilled-row performance, +small-batch and other distribution sensitivities until initial results justify an owner-approved follow-up. + +Use a compact physical-core sweep within one NUMA domain and across the allocated machine. Both series matter; do not +replace the full-machine MPI-off series with an MPI run. Verify placement externally and use identical external memory +policies across variants. Natural first-touch differences are real consequences of storage ownership/construction and +count in the comparison, not noise to normalize away. Report locality separately from synchronization effects. An +explicitly matched interleaved-memory rerun may diagnose a result, but cannot replace it. No app-owned affinity service. Report complete-batch time, phase times, throughput, speedup against each variant's own T=1 and the per-thread variant at the same T, actual workers, per-store imbalance, and fresh-process peak RSS including construction/growth. State which costs are omitted, which byte counts are estimates, and when memory placement or noise prevents a conclusion. -Synthetic index results cannot establish propagation/replay/gradient scaling or full-library parity. +Call exact key/ID lookup checks index-result agreement, not retained-operator equivalence: the latter also requires +coefficients. Synthetic index results cannot establish propagation/replay/gradient scaling or full-library parity. + +Task 0 has an approved calibration/measurement limit of 2 hours total, including each process's initialization, input +generation and validation, and phase-profile runs. Setup/build time and costs are accounted separately and need their +own authorization. Cap each fresh process's peak RSS at min(8 GiB, 25% of observed usable allocation RAM), including +setup/growth/checking. Record a fixed RAM basis and byte ceiling before trials; retain consumed/remaining time across +resumption. Stop with incomplete evidence if a limit prevents completion; do not silently extend the budget or claim +DRAM scaling if the permitted working set cannot establish it. These are external study limits, not a library clamp. +Approval of limits does not authorize execution. Task 0 ends with raw results and an owner decision: retain a shared-store path, retain/reconsider sharding, or declare the evidence inconclusive. Do not auto-select the fastest lookup kernel or invent a fourth architecture. Any revised -production design needs an updated spec/plan and approval, including MPI-off acceptance coverage. Keep the existing -strict library gates as written unless the owner explicitly revises them. Task 0 needs a separately approved execution -budget; it neither consumes Task 1's two-hour baseline pilot nor supplies its calibration or acceptance samples. The -later Task 11 trial is integration validation of a retained candidate, not a second container search. +production design needs an updated spec/plan and approval; it must preserve the now-required MPI-off acceptance +coverage and strict library gates unless the owner explicitly revises them. Task 0 neither consumes Task 1's separate +two-hour baseline pilot nor supplies its calibration or acceptance samples. The later Task 11 trial is integration +validation of a retained candidate, not a second container search. ## Global constraints -The following constraints describe the conditional one-store path in Tasks 1–13. Task 0 alone permits the explicit -per-thread-store comparison and requires full-machine MPI-off measurements, including cross-NUMA placement. This is -not permission to add shards to production or to waive any existing acceptance gate. Reconcile these constraints with -the owner's post-experiment architecture decision before proceeding. +The storage constraints describe the conditional one-store path in Tasks 1–13. Task 0 alone permits the explicit +per-thread-store experiment, not production shards. Full-machine MPI-off measurements are required both in Task 0 +and in full-library acceptance, including cross-NUMA placement. Reconcile the storage design with the owner's +post-experiment decision without silently weakening that coverage or any numerical/performance gate. - C++23; retain the GCC 14 / Clang 18 minimum compiler versions and Python 3.11 floor. - MPI remains optional and OFF by default; retain working non-MPI wheels. @@ -144,8 +164,9 @@ the owner's post-experiment architecture decision before proceeding. - Do not introduce hidden operator shards, a dense duplicate store, a second persistent full-key store, per-thread operators, or full-vector coefficient double buffering. Select any concurrent index only after correctness/memory/time evidence; its container name is neither a rejection criterion nor proof of suitability. -- Required performance cells keep each rank within one NUMA domain. Full-node cross-NUMA L2a is diagnostic-only and - cannot replace, excuse or average away any required cell; its numerical correctness is still checked. +- Required MPI-enabled cells keep each rank within one NUMA domain; MPI-enabled cross-NUMA L2a is diagnostic-only. + Actual MPI-disabled full-machine operation is required for small/medium and selected large profiles, even across + NUMA domains. Diagnostic results cannot replace, excuse or average away any required cell; correctness still applies. - Preserve Majorana and Pauli bases, Heisenberg and Schrödinger pictures, cutoffs, graph build/replay/paring, gradients, partial contraction, copying, and initial-operator updates. - Final MPI calls execute on the MPI-initializing thread, outside library-created OpenMP regions. Request/validate at @@ -325,8 +346,9 @@ AWS documents `c8a.metal-24xl` as AMD EPYC 9R45, 96 physical cores/96 vCPUs, one [C8a page](https://aws.amazon.com/ec2/instance-types/c8a/) identifies fifth-generation EPYC (Turin) and explicitly states that there is no SMT. Do not call a 192-worker run on this instance an SMT-on test; it would oversubscribe its physical cores. The exact socket/NUMA layout was not established by the reviewed AWS specifications and must be observed -on the actual allocation before selecting rank/thread geometries. In particular, do not assume eight NUMA domains or -approve one rank spanning all 96 cores without checking containment. +on the actual allocation before selecting rank/thread geometries. Do not assume eight NUMA domains or claim a +96-core process is NUMA-contained. Verify actual placement for required MPI-off full-machine operation and classify +MPI-enabled cross-NUMA L2a as diagnostic; both still require distinct physical cores. These published specifications are not measurements or evidence of parity. C8a does not establish performance on Sapphire Rapids, Zen3/Zen4 or HPC hosts with hardware SMT enabled. Those remain separate qualification targets using the @@ -344,9 +366,10 @@ remains pending; do not substitute a cloud-network experiment or claim HPC fabri - The user is responsible for binding/pinning processes and threads to the correct hardware using, for example, `OMP_PLACES` + `OMP_PROC_BIND`, `srun --cpu-bind`, and/or `mpiexec` options. Scheduler allocation, rank binding and OpenMP places must agree: binding a rank to one core while requesting many threads does not allocate additional cores. - Production acceptance keeps each rank's workers within one NUMA domain; avoid overlap with other ranks. Full-node - L2a may span domains only as the labelled diagnostic exception described below. - Documentation will advise at least one MPI process per NUMA domain used by the allocation; multiple processes may + Required MPI-enabled acceptance keeps each rank's workers within one NUMA domain; avoid overlap with other ranks. + MPI-enabled cross-NUMA L2a remains diagnostic. Required MPI-off full-machine operation spans the allocated domains + in one process and must verify distinct physical-core use without claiming NUMA containment. + For MPI deployments, advise at least one MPI process per NUMA domain used by the allocation; multiple processes may share a domain using disjoint CPU allocations. The production nodes commonly have four domains per socket, hence eight per full dual-socket node, but use the actual configured topology rather than a hard-coded count. Do not infer the EC2 topology from that HPC convention. Partial-node runs cover only their allocated domains. @@ -418,8 +441,9 @@ that allocation with the site before using masks. Mask lists repeat by node-loca assumed valid on differently numbered/configured nodes. Binding two ranks to the same entire locality domain does not partition its cores. No automatic topology/binding fallback is added to monoprop. -Archive rank/worker masks, physical-core sibling identities and NUMA containment for acceptance. For cross-NUMA L2a, -record the full-node allocation and actual multi-domain placement explicitly; do not claim single-domain containment. +Archive rank/worker masks, physical-core sibling identities and, for required MPI-enabled cells, NUMA containment. +For required MPI-off full-machine runs and diagnostic MPI-enabled cross-NUMA L2a, record the full allocation and actual +multi-domain placement explicitly; do not claim single-domain containment or substitute multiple MPI ranks. These recipes document HPC deployment; they do not convert pending proper-interconnect multi-node qualification into completed evidence. @@ -530,13 +554,17 @@ per rank. The candidate uses T OpenMP workers/rank; matched comparisons use T=P. | MPI-only control | R=D, T=1 | Required small/medium control, not a fixed-core comparison with L1 | | L2b | R=D, T=C/D | Required production shape: one rank/domain | | L2b decomposition variant | R=2D, T=floor(C/(2D)) | Required two-ranks/domain comparison where resources permit | -| L2a | R=1, T=C | Full-node diagnostic, allowed to span NUMA domains | +| L2a (MPI-enabled) | R=1, T=C | Full-node diagnostic, allowed to span NUMA domains | +| MPI-off full machine | R=1, T=C, has_mpi=false | Required small/medium and selected large profiles, including cross-NUMA | | L3/L4 | Multiple nodes | Pending proper HPC cluster/interconnect qualification | Use actual balanced subsets when allocation is uneven and record unused cores. Small/medium profiles span the required -controls; large acceptance profiles use only the two full-node L2b decompositions. Diagnostic L2a may use the same large -rows, without changing their flags to conceal a capacity/performance failure. If D=1, L2a and L2b coincide: deduplicate -that shape without removing its required acceptance gate. +controls plus MPI-off full-machine operation. Selected large profiles use the two full-node MPI decompositions and +MPI-off full-machine operation, not a new all-sizes Cartesian product. Calibrate shared profiles against the actual +MPI-disabled baseline too; after freeze, a required baseline/candidate capacity failure blocks acceptance, not grounds +to shrink a workload or drop a cell. MPI-enabled diagnostic L2a retains the same flags even if it cannot fit. +If D=1, L2a and L2b coincide: deduplicate within the same build mode without removing a required gate. Never deduplicate +MPI-off evidence against an MPI-enabled run, even at identical R/T/allocation. After the Task 0 architecture decision and reconciliation of these conditional production gates, the sequence is: @@ -553,8 +581,10 @@ After the Task 0 architecture decision and reconciliation of these conditional p 5. Compare five pairs per required cell, adding five more after a failure, using the five metrics defined below. Reuse existing reports and record actual term maps/values as well as counts; a term-count match is not numerical proof. -Keep cross-NUMA L2a output in a separately labelled diagnostic inventory/directory, outside the frozen required-cell -manifest consumed by the acceptance comparator. Its ratios are informative, never a waiver or an acceptance verdict. +Keep MPI-enabled cross-NUMA L2a output in a separately labelled diagnostic inventory/directory, outside the frozen +required-cell manifest consumed by the acceptance comparator. Its ratios are informative, never a waiver or an +acceptance verdict. MPI-off full-machine cells belong IN that required manifest and pass the same five metrics and +numerical checks; they must not inherit L2a's diagnostic classification merely from their R=1,T=C geometry. Diagnose numerical mismatches even there; diagnostic-only describes the performance gate, not permission for wrong results. Full-node L2a/L2b contrasts include process/decomposition and local MPI effects. Replacing L2a with a smaller NUMA-local run changes resources and cannot claim that same contrast. Historical ladder timings/sizes are examples, @@ -566,8 +596,10 @@ The initial acceptance gate is a frozen campaign on one c8a.metal-24xl, with at Record the actual socket/NUMA/core layout before selecting required geometries. Include one-rank-per-domain operation and an additional two-ranks-per-domain configuration with disjoint physical cores, where resources permit. Both arms use identical decomposition/allocation in every comparison; changing ranks only on the candidate cannot rescue a regression. -Source-reference workloads remain candidates, not approved production sizes. After separate execution authorization, -use baseline-only, unscored calibration to choose feasible sizes. The approved pilot budget is 2 hours total, excluding +Also require full-machine single-process MPI-disabled operation in every size tier's selected profiles; an MPI-enabled +binary at one rank cannot substitute. Source-reference workloads remain candidates, not approved production sizes. +After separate execution authorization, use baseline-only, unscored calibration to choose feasible sizes. The approved +pilot budget is 2 hours total, excluding builds/setup, with 15 minutes per trial including construction, evaluation and numerical export/validation. If the time budget is insufficient, stop and report rather than silently extend it. Target at most 75% of observed usable allocation RAM across all ranks, including construction and validation/export temporary storage. Record the RAM basis and byte @@ -581,10 +613,11 @@ rejected sizes. Do not consult candidate performance when selecting the campaign stop calibration rather than justify a smaller test. Resource failures may inform pre-freeze sizing; they are not failed acceptance cells. The approved calibration limits do not authorize builds, setup or runtime execution. -Small/medium profiles cover the full geometry sweep with the existing routing requirements. Large profiles cover only -the full-node one- and two-ranks-per-NUMA-domain configurations, not one-worker runs. This coverage is chosen before -formal measurements, not used to -excuse a failed cell later. Select the actual large workloads and sizes from calibration and obtain owner approval. +Small/medium profiles cover the full geometry sweep with the existing routing requirements, plus MPI-off full-machine +operation. Selected large profiles cover the full-node one- and two-ranks-per-domain MPI configurations and MPI-off +full-machine operation, not one-worker runs. This coverage is fixed before results, not chosen by the winning design. +Select the actual large workloads and common sizes using baseline-only calibration, including the MPI-off baseline, +and obtain owner approval. Task 1 keeps its 2-hour/15-minute/75% pilot limits; Task 0's 8-GiB cap does not apply here. Freeze the complete size-band-specific inventory before formal baseline collection. Calibration results are not acceptance samples: collect fresh formal baseline runs after the freeze, even for unchanged configurations. Every required cell must pass, but this does not claim parity for omitted workload/geometry combinations or CPU families. @@ -611,6 +644,8 @@ exactly FIVE metrics: runtime, operation peak sum/max, and whole-construction pe exactness must both be true; exit-RSS fallbacks cannot pass. Outer peaks/ratios are diagnostics only: non-exact or unknown outer evidence must be labeled honestly but cannot fail an otherwise passing five-metric comparison. All five required ratios must be <=1.00. Rank-peak sums are upper bounds, not simultaneous node-footprint measurements. +For MPI-off single-process cells, sum and max peaks coincide; retain all five fields and gates rather than weakening +the schema or substituting Task 0's whole-process RSS measurements. If a ratio exceeds 1.00, repeat the cell for five additional observations on each arm. If the combined median still exceeds 1.00, the gate is not passed. Profile and use only the bounded optimization menu in the plan; otherwise stop @@ -631,9 +666,13 @@ full-plan approval, not an existing pytest node or instrumentation-only edit. Us not build a general benchmark framework. Freeze and archive an owner-approved campaign inventory and digest after unscored calibration and before formal baseline -collection. It lists every required profile/node/geometry/routing cell for its size band, independently of observations. -Comparator input must match that inventory, so -removing a whole cell from both arms cannot pass. Each sample references one fresh timed artifact, one separate fresh +collection. It lists every required profile/node/build-mode/geometry/routing cell for its size band, independently of +observations. Include has_mpi in cross-arm identity and verify it from monoprop.has_mpi in each imported binary, with +binary-linked provenance. Missing/mismatched build modes fail; a single observed rank does not prove MPI is disabled. +Gate the shared harness's mpi4py import on has_mpi so an installed MPI package cannot initialize MPI in an MPI-off run. +MPI support/version fields are not applicable there, not invented FUNNELED results. Comparator input must match the +inventory, so removing a whole cell from both arms cannot pass. Each sample references one fresh timed artifact, one +separate fresh construction artifact, and a matching untimed validation artifact. Join only matching cell/arm/binary/configuration, parameters, geometry/routing/allocation and campaign provenance. Timed/construction run IDs and files cannot be reused; require five distinct pairs per arm/cell, or ten after prescribed repetition. Validation and placement diagnostics can @@ -680,8 +719,9 @@ RED/GREEN evidence for new behavior, plus baseline results for preservation test manufacture RED by removing earlier plumbing. Use kernel-specific worker observations for threading participation. Do not execute past a failed gate or infer approval from this refresh. -The first execution handoff is now the separately authorized Task 0 mini-app, ending at the architecture decision. -Only after that gate does the baseline-only pilot travel with Task 1 to the authorized execution machine; neither +The first execution handoff is the separately authorized Task 0 mini-app, ending at the architecture decision. Apply +its staged first pass and 2-hour/min(8 GiB, 25% RAM) limits; account for setup/build separately. These approvals do not +authorize launch. Only after that gate does the baseline pilot travel with Task 1 to the authorized host; neither experiment must run on the planning host before handoff. Transfer the latest spec, plan and CONTEXT.md, not only the historical published planning commit. The Task 1 portion of the remote entry point defines the subsequent baseline-only phase ending with a proposed campaign/budget for owner approval, not candidate engine work or a completed Task 1 gate. From 700765850c1c35b0f01d96b583f5cb9d92d0dc0c Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Roberto=20Di=20Remigio=20Eik=C3=A5s?= Date: Fri, 25 Sep 2026 12:53:52 +0000 Subject: [PATCH 05/64] =?UTF-8?q?feat(index-miniapp):=20=F0=9F=A7=AA=20add?= =?UTF-8?q?=20Task=200=20MPI-off=20index=20mini-app?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Standalone CMake experiment for Task 0 of the rank-local OpenMP plan. One process and one OpenMP team compare three ownership/index designs over the current packed OperatorIndex rows: per-thread disjoint stores, one shared store with frozen parallel lookup and caller-owned insertion, and one shared packed backing store indexed by boost::concurrent_flat_map with per-query or bulk lookup. It reuses the header-only storage/hash code without changes and does not link or import monoprop. Release-active self-tests cover exact IDs, sharded owner/local IDs, collisions, spilled rows, tails, duplicate-miss rejection, runtime-limited teams, worker participation and joined exceptions. A BOOST_UNORDERED_ENABLE_STATS diagnostic build adds Boost's own container statistics to the CSV output. Assisted-by: Pi:claude-opus-5-5 --- tools/index-miniapp/CMakeLists.txt | 53 + tools/index-miniapp/README.md | 96 ++ tools/index-miniapp/main.cpp | 1483 ++++++++++++++++++++++++++++ 3 files changed, 1632 insertions(+) create mode 100644 tools/index-miniapp/CMakeLists.txt create mode 100644 tools/index-miniapp/README.md create mode 100644 tools/index-miniapp/main.cpp diff --git a/tools/index-miniapp/CMakeLists.txt b/tools/index-miniapp/CMakeLists.txt new file mode 100644 index 00000000..21be8ea6 --- /dev/null +++ b/tools/index-miniapp/CMakeLists.txt @@ -0,0 +1,53 @@ +cmake_minimum_required(VERSION 3.28...4.0) +project(index_miniapp LANGUAGES CXX) +if( + CMAKE_CXX_COMPILER_ID + STREQUAL + "GNU" + AND + CMAKE_CXX_COMPILER_VERSION + VERSION_LESS + 14 +) + message(FATAL_ERROR "GCC 14 or newer is required") +elseif( + CMAKE_CXX_COMPILER_ID + MATCHES + "Clang" + AND + CMAKE_CXX_COMPILER_VERSION + VERSION_LESS + 18 +) + message(FATAL_ERROR "Clang 18 or newer is required") +endif() +# The study is MPI-off by construction; it must never inherit the library's MPI switch. +if(DEFINED monoprop_ENABLE_MPI) + message( + FATAL_ERROR + "index-miniapp is MPI-off only; do not define monoprop_ENABLE_MPI" + ) +endif() +find_package(Boost 1.85 CONFIG REQUIRED) +find_package(OpenMP REQUIRED COMPONENTS CXX) +find_package(Threads REQUIRED) +add_executable(index-miniapp main.cpp) +target_compile_features(index-miniapp PRIVATE cxx_std_23) +target_include_directories( + index-miniapp + PRIVATE + "${CMAKE_CURRENT_LIST_DIR}/../../cpp" +) +target_link_libraries( + index-miniapp + PRIVATE + Boost::boost + OpenMP::OpenMP_CXX + Threads::Threads +) +enable_testing() +add_test( + NAME index-miniapp-self-test + COMMAND + index-miniapp --self-test +) diff --git a/tools/index-miniapp/README.md b/tools/index-miniapp/README.md new file mode 100644 index 00000000..4681909e --- /dev/null +++ b/tools/index-miniapp/README.md @@ -0,0 +1,96 @@ +# index-miniapp (Task 0) + +A standalone MPI-off experiment for the rank-local OpenMP plan +(`docs/superpowers/plans/2026-09-18-rank-local-openmp.md`, Task 0). One process with one OpenMP team of T workers +compares three ownership/index designs over monoprop's current packed rows. The program reuses the header-only +`OperatorIndex` and `Monomial` code without changing it. It does not build, link or import monoprop. + +The app checks exact keys and IDs, which establishes **index-result agreement only**. That is not retained-operator +equivalence, which also requires coefficients. It is not a propagation, replay, gradient, MPI or full-library benchmark, +and it does not measure the current library. + +## Variants + +| `--variant` | Stores | Batch pipeline | +| --- | --- | --- | +| `per-thread` | T disjoint `OperatorIndex` stores; owner = `monomial_hash % T` (R=1 routing) | route/count → ordered prefixes → stable scatter to owner buckets → per owner: `find_batch_positions`, missing IDs in bucket order, `grow_rows_geometric`/`set_positions`, `bulk_insert_hashed` into bucket slots → each source chunk pulls its own results | +| `shared-serial` | one `OperatorIndex` | parallel `find_batch_positions` over balanced contiguous spans → caller assigns missing IDs in query order, grows/fills rows, `bulk_insert_hashed` | +| `shared-concurrent` | one packed row backing (`OperatorIndex` rows only) + `boost::concurrent_flat_map` | parallel hash + per-query `cvisit` → caller assigns IDs and fills rows → join → parallel `emplace` of preassigned IDs | + +`--lookup bulk` (only with `shared-concurrent`) probes through Boost's bulk `cvisit` in 16-query chunks. The visitor +receives only the matched element, never the query, and Boost documents no callback order. So the visitor records +`(hash, row)` pairs, and the results are matched back to queries after each call, outside Boost's locks. A query that is +alone with that hash in its chunk is the match; several such queries (duplicates or a 32-bit collision) are resolved by +exact comparison. Misses produce no callback and stay absent. The default is `--lookup per-query`. + +The concurrent index stores `{row ID, cached 32-bit fold hash}` handles, never monomials or query views. Transparent +hash and equality compare query positions against the packed row exactly, using a dense comparison for spilled rows. The +backing `OperatorIndex` keeps its constant empty 16-slot table (`harness_bytes_est`); the app never populates or +reserves it. Boost changes table layout and probing as well as synchronization, so its T=1 result is a +different-container result, not a pure locking cost. + +Shared IDs are canonical: misses get IDs in query order. Per-thread IDs are owner-local and follow per-owner query +order, so the numeric IDs differ between shard counts while the decoded keys and (owner, local ID) order agree. +Nothing reserves capacity; every table grows natively. + +## Build and self-test + +```bash +cmake -S tools/index-miniapp -B "$BUILD" -DCMAKE_BUILD_TYPE=Release +cmake --build "$BUILD" --parallel +OMP_NUM_THREADS=1 ctest --test-dir "$BUILD" --output-on-failure --no-tests=error +for T in 1 2 3 4; do OMP_NUM_THREADS="$T" OMP_DYNAMIC=FALSE "$BUILD/index-miniapp" --self-test; done +``` + +The build needs GCC 14+ or Clang 18+, Boost 1.85+ and OpenMP. It refuses a `monoprop_ENABLE_MPI` definition. The +self-test runs every fixture through all three variants at requested teams 1–4, with 128 and 1024 modes (both +position widths). It also runs per-thread at a fixed three shards. A serial key oracle that exists only in the +self-test checks the results. + +The fixtures cover the plan's A/B/C hit masks and IDs, repeated queries, duplicate-miss rejection, empty +inputs and stores, 15/16/17 and 63/64/65 query tails, and an equal-fold-hash collision pair before and after table +growth. They also cover spilled length-16 rows across backing reallocations, actual probe/publication worker +participation and joined worker exceptions. The checks use `require`, not `assert`, so they stay active in Release. +Under `OMP_THREAD_LIMIT` the fixtures still pass, and participation is reported as `SKIP`. + +A diagnostic build with `-DCMAKE_CXX_FLAGS=-DBOOST_UNORDERED_ENABLE_STATS` (Boost 1.86 or newer) adds Boost's own container +statistics for `shared-concurrent`: `bs_*` CSV columns covering probe lengths and comparisons for hits, misses and +insertions. The statistics cover the measured batches. Boost also counts its own traffic: every `emplace` performs an +absence lookup, and rehash transfers count as insertions. Only the hit counts match adapter probes one-to-one. The +concurrent counters serialize on a lock, so never time a stats build. + +Sanitizers use command-line flags, for example +`-DCMAKE_CXX_FLAGS="-fsanitize=address,undefined -fno-omit-frame-pointer"`. Qualify a TSan/OpenMP-runtime pair with a +known-racy and a known-safe program first. GCC TSan with an uninstrumented libgomp reports false positives. Sanitizer +timings are not performance evidence. + +## Measurement run + +```bash +OMP_NUM_THREADS="$T" OMP_DYNAMIC=FALSE OMP_PLACES=cores OMP_PROC_BIND=close \ + "$BUILD/index-miniapp" --variant shared-serial --mode grow --rows 65536 \ + --batch 4096 --batches 16 --miss-percent 10 --key-length 6 --modes 128 --seed 0 [--profile] +``` + +- The team is `omp_get_max_threads()`, read once at startup. Every region requests exactly that many workers. The app + never calls `omp_set_*`, sets affinity or discovers topology. Verify placement externally, for example with + `OMP_DISPLAY_AFFINITY` or `/proc//task/*/status`. +- Keys are an injective synthetic encoding: `2*modes` positions split into `key-length` bands, with one ordinal digit + per band after a seed-derived cyclic offset. The app streams the initial rows through the variant's own grow path. + Each timed batch contains `batch - floor(batch*miss/100)` distinct existing ordinals and fresh ordinals for the + misses. In `lookup` mode the misses stay absent. In `grow` mode they are published. This structured synthetic + family makes no claim about real hit ratios or ownership skew. +- The app prepares each batch outside the timer. The timer covers the complete batch: hashing, ownership, + bucketing/scatter, probing, ID assignment, row growth and fill, publication and the join. `--profile` adds phase + timers. Profile a separate matching process: normal runs leave the phase fields empty, and so do phases that a + variant or mode does not have. For per-thread, `route`, `prefix`, `scatter`, `owner_elapsed` and `gather` are + elapsed and additive. Its `probe`…`publish` columns are summed across owners that run concurrently, so they are + **nonadditive**. `owner_max` sums the slowest owner of each batch. +- Validation runs outside the timer and keeps no shadow map. Each batch checks every declared hit and miss, the exact + key of every returned row, the owner and the per-owner publication order. At the end, the app checks the exact row + count and then looks up every expected key again, in order. It prints one CSV row with `validation=ok` only on + success. A failure prints `FAILED` and no CSV row. +- `row_bytes_est`, `index_bytes_est` and `harness_bytes_est` are estimates. The concurrent index figure is a formula + over `bucket_count()`. Take peak memory from an external whole-process RSS measurement. +- Boost APIs used: the `concurrent_flat_map(n, hash, eq)` constructor, `emplace`, heterogeneous single and bulk + `cvisit`, `size` and `bucket_count`. The code rejects Boost older than 1.85. diff --git a/tools/index-miniapp/main.cpp b/tools/index-miniapp/main.cpp new file mode 100644 index 00000000..08e15086 --- /dev/null +++ b/tools/index-miniapp/main.cpp @@ -0,0 +1,1483 @@ +// Copyright 2026 Algorithmiq +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +/* + * Task 0 index mini-app: one MPI-off process, one OpenMP team, three ownership/index designs over the + * current packed rows (see README.md). Exact key/ID checks establish index-result agreement only, never + * retained-operator equivalence or propagation performance. + */ + +#include + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include +#include + +#include "monoprop/core/Monomial.h" +#include "monoprop/detail/operator/OperatorIndex.h" + +#ifdef monoprop_ENABLE_MPI +#error "index-miniapp is MPI-off only" +#endif +static_assert(BOOST_VERSION >= 108500, "the project's Boost floor is 1.85"); + +namespace { + +using monoprop::Monomial; +using monoprop::TermIndex; +using monoprop::detail::OperatorIndex; +using Clock = std::chrono::steady_clock; + +constexpr size_t kNotFound = OperatorIndex<128>::kNotFound; + +// Release-active check: correctness gates must not vanish with NDEBUG. +auto require(bool ok, std::string_view what) -> void { + if (!ok) { + throw std::runtime_error(std::string(what)); + } +} + +auto lap(Clock::time_point &t) -> double { + const auto now = Clock::now(); + const double s = std::chrono::duration(now - t).count(); + t = now; + return s; +} + +auto splitmix(uint64_t &state) -> uint64_t { + uint64_t z = (state += 0x9E3779B97F4A7C15ULL); + z = (z ^ (z >> 30)) * 0xBF58476D1CE4E5B9ULL; + z = (z ^ (z >> 27)) * 0x94D049BB133111EBULL; + return z ^ (z >> 31); +} + +// Self-test-only fault injection: the worker with this thread number throws inside its region. +std::atomic g_fault_tid{-1}; +auto maybe_inject(int tid) -> void { + if (tid == g_fault_tid.load(std::memory_order_relaxed)) { + throw std::runtime_error("injected worker fault"); + } +} + +// Balanced contiguous span [lo, hi) of n items for worker tid; short spans and empty tails are allowed. +auto span_of(size_t n, int team, int tid) -> std::pair { + const auto t = static_cast(team); + return {n * static_cast(tid) / t, n * static_cast(tid + 1) / t}; +} + +auto rethrow_first(const std::vector &errors) -> void { + for (const auto &e : errors) { + if (e) { + std::rethrow_exception(e); + } + } +} + +// Per-owner sub-phases of the sharded arm overlap in time: their sums are nonadditive diagnostics. +enum Phase : uint8_t { kRoute, kPrefix, kScatter, kProbe, kAssign, kFill, kPublish, kGather, kOwnerElapsed, kOwnerMax }; +constexpr size_t kNumPhases = 10; +constexpr std::array kPhaseNames{"route", + "prefix", + "scatter", + "probe", + "assign", + "fill", + "publish", + "gather", + "owner_elapsed", + "owner_max"}; + +struct BatchObs { + int team = 0; //!< Smallest actual team among this batch's regions. + int probe_active = 0; //!< Workers that probed at least one query. + int publish_active = 0; //!< Workers that published at least one row; 0 when nothing was published. + size_t bulk_calls = 0; //!< Bulk cvisit calls issued by the probe (shared-concurrent bulk lookup only). + std::array phase{}; +}; + +// Flat query batch: query q is pos[off[q], off[q]+k[q]), strictly ascending positions. +template +struct Batch { + using PosT = typename OperatorIndex::PosT; + std::vector pos; + std::vector off; + std::vector k; + + [[nodiscard]] auto size() const -> size_t { return off.size(); } + [[nodiscard]] auto key(size_t q) const -> std::span { return {pos.data() + off[q], k[q]}; } + auto clear() -> void { + pos.clear(); + off.clear(); + k.clear(); + } + auto push(std::span p) -> void { + off.push_back(pos.size()); + k.push_back(static_cast(p.size())); + pos.insert(pos.end(), p.begin(), p.end()); + } +}; + +// Results in original query order; owner is 0 for the shared variants. +struct Results { + std::vector id; + std::vector owner; + std::vector hit; + auto resize(size_t n) -> void { + id.assign(n, kNotFound); + owner.assign(n, 0); + hit.assign(n, 0); + } +}; + +template +auto monomial_of(std::span::PosT> pos) -> Monomial { + Monomial m; + for (const auto p : pos) { + m.set(p); + } + return m; +} + +// Exact identity through public row access; a spilled row has no position array and compares densely. +template +auto row_matches(const OperatorIndex &s, size_t row, std::span::PosT> q) -> bool { + const auto rp = s.row_positions(row); + return rp.inlined() ? std::ranges::equal(rp.pos, q) : s.row(row) == monomial_of(q); +} + +template +auto rows_equal(const OperatorIndex &s, size_t a, size_t b) -> bool { + const auto ra = s.row_positions(a); + const auto rb = s.row_positions(b); + return (ra.inlined() && rb.inlined()) ? std::ranges::equal(ra.pos, rb.pos) : s.row(a) == s.row(b); +} + +// bulk_insert_hashed never deduplicates, so a publication list holding one key twice must be refused before +// any row grows. Self-tests enable this; measured inputs are distinct by construction and validated afterwards. +template +auto require_distinct(const Batch &b, std::vector> hq) -> void { + std::ranges::sort(hq, [&](const auto &x, const auto &y) { + return x.first != y.first ? x.first < y.first + : std::ranges::lexicographical_compare(b.key(x.second), b.key(y.second)); + }); + for (size_t i = 1; i < hq.size(); ++i) { + require(hq[i].first != hq[i - 1].first || !std::ranges::equal(b.key(hq[i].second), b.key(hq[i - 1].second)), + "invalid publication fixture: duplicate missing key"); + } +} + +// --------------------------------------------------------------------------------------------------------------- +// Variant 1: T disjoint current stores. Owner = monomial_hash % shards (R=1 routing); each owner is the single +// writer of its store. Bucketing is a simplified shared-memory model, not the legacy transport. +template +class PerThread { +public: + using Store = OperatorIndex; + using PosT = typename Store::PosT; + static constexpr bool kIdsAreOrdinals = false; + static constexpr std::array kPhases{true, true, true, true, true, true, true, true, true, true}; + + PerThread(int shards, int threads, size_t width) : threads_(threads) { + for (int s = 0; s < shards; ++s) { + stores_.push_back(std::make_unique(width)); + } + } + bool check_distinct = false; //!< Refuse duplicate misses before publication (self-test fixtures). + + [[nodiscard]] auto owners() const -> size_t { return stores_.size(); } + [[nodiscard]] auto owner_size(size_t o) const -> size_t { return stores_[o]->size(); } + [[nodiscard]] auto owner_of(std::span key) const -> uint32_t { + return static_cast(monoprop::monomial_hash(monomial_of(key)) % owners()); + } + [[nodiscard]] auto matches(size_t o, size_t id, std::span key) const -> bool { + return id < stores_[o]->size() && row_matches(*stores_[o], id, key); + } + [[nodiscard]] auto row_bytes() const -> size_t { + return sum_([](const Store &s) { return s.memory_bytes(); }); + } + [[nodiscard]] auto index_bytes() const -> size_t { + return sum_([](const Store &s) { return s.index_estimated_memory_bytes(); }); + } + [[nodiscard]] static auto harness_bytes() -> size_t { return 0; } + + // route/count -> ordered prefixes -> stable scatter into owner buckets -> per-owner probe, ordered ID + // assignment, row fill and publication into bucket slots -> each source chunk pulls its own results. Owners + // never write query-order arrays, so no two owners share a result cache line. One team; phases are separated + // by barriers, and a failed phase only skips later work so every worker still reaches every barrier and join. + auto run(const Batch &b, bool grow, Results &r, BatchObs &obs, bool profile) -> void { + const size_t n = b.size(); + const size_t shards = owners(); + const auto threads = static_cast(threads_); + r.resize(n); + owner_q_.resize(n); + perm_.resize(n); + boff_.resize(n); + bk_.resize(n); + lout_.resize(n); + lhash_.resize(n); + miss_.resize(n); + slot_.resize(n); + bhit_.resize(n); + count_.assign(threads * shards, 0); + cursor_.assign(threads * shards, 0); + start_.assign(shards + 1, 0); + probed_.assign(threads, 0); + published_.assign(threads, 0); + owner_t_.assign(shards, {}); + std::vector errors(threads); + std::atomic failed{false}; + auto guarded = [&](int tid, auto &&fn) { + if (failed.load(std::memory_order_relaxed)) { + return; + } + try { + fn(); + } + catch (...) { + errors[static_cast(tid)] = std::current_exception(); + failed.store(true, std::memory_order_relaxed); + } + }; + int team_seen = 0; + auto t = Clock::now(); +#pragma omp parallel num_threads(threads_) + { + const int tid = omp_get_thread_num(); + const int team = omp_get_num_threads(); + const auto [lo, hi] = span_of(n, team, tid); + if (tid == 0) { + team_seen = team; + } + guarded(tid, [&] { + size_t *cnt = &count_[static_cast(tid) * shards]; + for (size_t q = lo; q < hi; ++q) { + owner_q_[q] = static_cast(monoprop::monomial_hash(monomial_of(b.key(q))) % shards); + ++cnt[owner_q_[q]]; + } + }); +#pragma omp barrier + if (profile && tid == 0) { + obs.phase[kRoute] += lap(t); + } + guarded(tid, [&] { + for (auto o = static_cast(tid); o < shards; o += static_cast(team)) { + size_t total = 0; + for (size_t c = 0; c < static_cast(team); ++c) { + cursor_[(c * shards) + o] = total; + total += count_[(c * shards) + o]; + } + start_[o + 1] = total; + } + }); +#pragma omp barrier +#pragma omp single + guarded(tid, [&] { std::partial_sum(start_.begin(), start_.end(), start_.begin()); }); + if (profile && tid == 0) { + obs.phase[kPrefix] += lap(t); + } + guarded(tid, [&] { + size_t *cur = &cursor_[static_cast(tid) * shards]; + for (size_t q = lo; q < hi; ++q) { + const size_t p = start_[owner_q_[q]] + cur[owner_q_[q]]++; + perm_[p] = q; + slot_[q] = p; + boff_[p] = b.off[q]; + bk_[p] = b.k[q]; + } + }); +#pragma omp barrier + if (profile && tid == 0) { + obs.phase[kScatter] += lap(t); + } + guarded(tid, [&] { + maybe_inject(tid); + for (auto o = static_cast(tid); o < shards; o += static_cast(team)) { + owner_work_(o, tid, b, grow, profile); + } + }); +#pragma omp barrier + if (profile && tid == 0) { + obs.phase[kOwnerElapsed] += lap(t); + } + guarded(tid, [&] { + for (size_t q = lo; q < hi; ++q) { + r.id[q] = lout_[slot_[q]]; + r.hit[q] = bhit_[slot_[q]]; + r.owner[q] = owner_q_[q]; + } + }); + } + rethrow_first(errors); + obs.team = team_seen; + obs.probe_active = static_cast(std::ranges::count(probed_, 1)); + obs.publish_active = static_cast(std::ranges::count(published_, 1)); + if (profile) { + obs.phase[kGather] += lap(t); + double owner_max = 0; + for (const auto &ot : owner_t_) { + double sum = 0; + for (size_t p = kProbe; p <= kPublish; ++p) { + obs.phase[p] += ot[p]; + sum += ot[p]; + } + owner_max = std::max(owner_max, sum); + } + obs.phase[kOwnerMax] += owner_max; + } + } + +private: + auto owner_work_(size_t o, int tid, const Batch &b, bool grow, bool profile) -> void { + const size_t lo = start_[o]; + const size_t count = start_[o + 1] - lo; + if (count == 0) { + return; + } + probed_[static_cast(tid)] = 1; + Store &st = *stores_[o]; + auto t = Clock::now(); + auto tick = [&](Phase p) { + if (profile) { + owner_t_[o][p] += lap(t); + } + }; + st.find_batch_positions(b.pos, + std::span(boff_).subspan(lo, count), + std::span(bk_).subspan(lo, count), + std::span(lout_).subspan(lo, count), + std::span(lhash_).subspan(lo, count)); + tick(kProbe); + size_t m = 0; + for (size_t j = lo; j < lo + count; ++j) { + bhit_[j] = lout_[j] != kNotFound ? 1 : 0; + if (grow && lout_[j] == kNotFound) { + miss_[lo + m++] = j; + } + } + if (m > 0) { + if (check_distinct) { + std::vector> hq; + for (size_t i = 0; i < m; ++i) { + hq.emplace_back(lhash_[miss_[lo + i]], perm_[miss_[lo + i]]); + } + require_distinct(b, std::move(hq)); + } + tick(kAssign); + const size_t base = st.grow_rows_geometric(m); + for (size_t i = 0; i < m; ++i) { + st.set_positions(base + i, b.key(perm_[miss_[lo + i]])); + } + tick(kFill); + st.bulk_insert_hashed(m, base, [&](size_t i) { return lhash_[miss_[lo + i]]; }); + for (size_t i = 0; i < m; ++i) { + lout_[miss_[lo + i]] = base + i; + } + published_[static_cast(tid)] = 1; + tick(kPublish); + } + } + + template + [[nodiscard]] auto sum_(Fn fn) const -> size_t { + size_t total = 0; + for (const auto &s : stores_) { + total += fn(*s); + } + return total; + } + + int threads_; + std::vector> stores_; + // Batch scratch: O(batch) records plus O(threads*shards) counters, reused across batches. + std::vector owner_q_, bk_, lhash_; + std::vector perm_, slot_, boff_, lout_, miss_, count_, cursor_, start_; + std::vector bhit_, probed_, published_; + std::vector> owner_t_; +}; + +// --------------------------------------------------------------------------------------------------------------- +// Variant 2: one current store. Parallel frozen probes into disjoint output spans; the caller assigns IDs in +// query order, grows/initializes rows and publishes with bulk_insert_hashed. +template +class SharedSerial { +public: + using Store = OperatorIndex; + using PosT = typename Store::PosT; + static constexpr bool kIdsAreOrdinals = true; + static constexpr std::array + kPhases{false, false, false, true, true, true, true, false, false, false}; + + SharedSerial(int threads, size_t width) : threads_(threads), store_(std::make_unique(width)) {} + bool check_distinct = false; //!< Refuse duplicate misses before publication (self-test fixtures). + + [[nodiscard]] static auto owners() -> size_t { return 1; } + [[nodiscard]] auto owner_size(size_t /*o*/) const -> size_t { return store_->size(); } + [[nodiscard]] static auto owner_of(std::span /*key*/) -> uint32_t { return 0; } + [[nodiscard]] auto matches(size_t /*o*/, size_t id, std::span key) const -> bool { + return id < store_->size() && row_matches(*store_, id, key); + } + [[nodiscard]] auto row_bytes() const -> size_t { return store_->memory_bytes(); } + [[nodiscard]] auto index_bytes() const -> size_t { return store_->index_estimated_memory_bytes(); } + [[nodiscard]] static auto harness_bytes() -> size_t { return 0; } + + auto run(const Batch &b, bool grow, Results &r, BatchObs &obs, bool profile) -> void { + const size_t n = b.size(); + r.resize(n); + hash_.resize(n); + probed_.assign(static_cast(threads_), 0); + std::vector errors(static_cast(threads_)); + int team_seen = 0; + auto t = Clock::now(); +#pragma omp parallel num_threads(threads_) + { + const int tid = omp_get_thread_num(); + const int team = omp_get_num_threads(); + if (tid == 0) { + team_seen = team; + } + try { + maybe_inject(tid); + const auto [lo, hi] = span_of(n, team, tid); + if (lo < hi) { + probed_[static_cast(tid)] = 1; + store_->find_batch_positions(b.pos, + std::span(b.off).subspan(lo, hi - lo), + std::span(b.k).subspan(lo, hi - lo), + std::span(r.id).subspan(lo, hi - lo), + std::span(hash_).subspan(lo, hi - lo)); + } + } + catch (...) { + errors[static_cast(tid)] = std::current_exception(); + } + } + rethrow_first(errors); + obs.team = team_seen; + obs.probe_active = static_cast(std::ranges::count(probed_, 1)); + if (profile) { + obs.phase[kProbe] += lap(t); + } + miss_.clear(); + for (size_t q = 0; q < n; ++q) { + r.hit[q] = r.id[q] != kNotFound ? 1 : 0; + if (grow && r.id[q] == kNotFound) { + miss_.push_back(q); + } + } + if (miss_.empty()) { + return; + } + if (check_distinct) { + std::vector> hq; + for (const auto q : miss_) { + hq.emplace_back(hash_[q], q); + } + require_distinct(b, std::move(hq)); + } + if (profile) { + obs.phase[kAssign] += lap(t); + } + const size_t base = store_->grow_rows_geometric(miss_.size()); + for (size_t i = 0; i < miss_.size(); ++i) { + store_->set_positions(base + i, b.key(miss_[i])); + } + if (profile) { + obs.phase[kFill] += lap(t); + } + store_->bulk_insert_hashed(miss_.size(), base, [&](size_t i) { return hash_[miss_[i]]; }); + for (size_t i = 0; i < miss_.size(); ++i) { + r.id[miss_[i]] = base + i; + } + obs.publish_active = 1; + if (profile) { + obs.phase[kPublish] += lap(t); + } + } + +private: + int threads_; + std::unique_ptr store_; + std::vector hash_; + std::vector miss_; + std::vector probed_; +}; + +// --------------------------------------------------------------------------------------------------------------- +// Variant 3: one packed row backing plus boost::concurrent_flat_map keyed by compact row handles. The backing +// OperatorIndex keeps its constant empty 16-slot table (harness overhead); it is never populated or reserved. +template +class SharedConcurrent { +public: + using Store = OperatorIndex; + using PosT = typename Store::PosT; + static constexpr bool kIdsAreOrdinals = true; + static constexpr std::array + kPhases{false, false, false, true, true, true, true, false, false, false}; + + SharedConcurrent(int threads, size_t width) + : threads_(threads), + rows_(std::make_unique(width)), + map_(0, KeyHash{}, KeyEqual{rows_.get()}) {} + SharedConcurrent(const SharedConcurrent &) = delete; + auto operator=(const SharedConcurrent &) -> SharedConcurrent & = delete; + bool check_distinct = false; //!< Refuse duplicate misses before publication (self-test fixtures). + bool bulk_lookup = false; //!< Probe through Boost's bulk cvisit in 16-query chunks. + + [[nodiscard]] static auto owners() -> size_t { return 1; } + [[nodiscard]] auto owner_size(size_t /*o*/) const -> size_t { return rows_->size(); } + [[nodiscard]] static auto owner_of(std::span /*key*/) -> uint32_t { return 0; } + [[nodiscard]] auto matches(size_t /*o*/, size_t id, std::span key) const -> bool { + return id < rows_->size() && row_matches(*rows_, id, key); + } + [[nodiscard]] auto index_entries() const -> size_t { return map_.size(); } + [[nodiscard]] auto row_bytes() const -> size_t { return rows_->memory_bytes(); } + // Estimate: FOA slot array plus 16 metadata bytes per 15-slot group and its concurrent group-access word. + [[nodiscard]] auto index_bytes() const -> size_t { + const size_t buckets = map_.bucket_count(); + return sizeof(Map) + (buckets * sizeof(typename Map::value_type)) + ((buckets / 15 + 1) * (16 + 8)); + } + [[nodiscard]] auto harness_bytes() const -> size_t { return rows_->index_estimated_memory_bytes(); } +#ifdef BOOST_UNORDERED_ENABLE_STATS + //! Boost's container statistics; diagnostic builds only (the concurrent counters serialize on one lock). + [[nodiscard]] auto boost_stats() const { return map_.get_stats(); } + //! Clears Boost's container statistics. + auto reset_boost_stats() -> void { map_.reset_stats(); } +#endif + + // Parallel hash + per-query visit (frozen map) -> caller-ordered IDs and row fill -> join -> parallel + // publication of preassigned IDs. Rows are immutable while the map runs; joins precede any row growth. + auto run(const Batch &b, bool grow, Results &r, BatchObs &obs, bool profile) -> void { + const size_t n = b.size(); + r.resize(n); + hash_.resize(n); + probed_.assign(static_cast(threads_), 0); + published_.assign(static_cast(threads_), 0); + std::vector errors(static_cast(threads_)); + std::vector bulk_calls(static_cast(threads_), 0); + int team_seen = threads_; + auto t = Clock::now(); +#pragma omp parallel num_threads(threads_) + { + const int tid = omp_get_thread_num(); + const int team = omp_get_num_threads(); + if (tid == 0) { + team_seen = team; + } + try { + maybe_inject(tid); + const auto [lo, hi] = span_of(n, team, tid); + if (bulk_lookup) { + for (size_t c = lo; c < hi; c += kBulkChunk) { + probe_chunk_(b, r, c, std::min(hi, c + kBulkChunk)); + ++bulk_calls[static_cast(tid)]; + } + } + else { + for (size_t q = lo; q < hi; ++q) { + hash_[q] = Store::fold_hash_positions(b.key(q)); + size_t found = kNotFound; + // One bounded visitor per query keeps results aligned; it never re-enters the map. + map_.cvisit(QueryKey{b.key(q), hash_[q]}, [&found](const auto &kv) { found = kv.second; }); + r.id[q] = found; + } + } + probed_[static_cast(tid)] = lo < hi ? 1 : 0; + } + catch (...) { + errors[static_cast(tid)] = std::current_exception(); + } + } + rethrow_first(errors); + obs.team = team_seen; + obs.probe_active = static_cast(std::ranges::count(probed_, 1)); + obs.bulk_calls = std::accumulate(bulk_calls.begin(), bulk_calls.end(), size_t{0}); + if (profile) { + obs.phase[kProbe] += lap(t); + } + miss_.clear(); + for (size_t q = 0; q < n; ++q) { + r.hit[q] = r.id[q] != kNotFound ? 1 : 0; + if (grow && r.id[q] == kNotFound) { + miss_.push_back(q); + } + } + if (miss_.empty()) { + return; + } + if (check_distinct) { + std::vector> hq; + for (const auto q : miss_) { + hq.emplace_back(hash_[q], q); + } + require_distinct(b, std::move(hq)); + } + if (profile) { + obs.phase[kAssign] += lap(t); + } + // grow_rows_geometric refuses an append past the fixed32 ceiling, so every ID below fits TermIndex. + const size_t base = rows_->grow_rows_geometric(miss_.size()); + for (size_t i = 0; i < miss_.size(); ++i) { + rows_->set_positions(base + i, b.key(miss_[i])); + } + if (profile) { + obs.phase[kFill] += lap(t); + } + const size_t m = miss_.size(); +#pragma omp parallel num_threads(threads_) + { + const int tid = omp_get_thread_num(); + const int team = omp_get_num_threads(); + if (tid == 0) { + team_seen = std::min(team_seen, team); + } + try { + const auto [lo, hi] = span_of(m, team, tid); + for (size_t i = lo; i < hi; ++i) { + const auto id = static_cast(base + i); + require(map_.emplace(RowKey{id, hash_[miss_[i]]}, id), + "invalid publication: duplicate missing key"); + r.id[miss_[i]] = id; + } + published_[static_cast(tid)] = lo < hi ? 1 : 0; + } + catch (...) { + errors[static_cast(tid)] = std::current_exception(); + } + } + rethrow_first(errors); + obs.team = team_seen; + obs.publish_active = static_cast(std::ranges::count(published_, 1)); + if (profile) { + obs.phase[kPublish] += lap(t); + } + } + +private: + // Boost processes bulk visits in groups of 16 (its internal bulk_visit_size); one chunk per call keeps the + // result matching below a scan of at most 16 queries. + static constexpr size_t kBulkChunk = 16; + + // Bulk probe of queries [c, e). The visitor receives only the matched element, never the query, and Boost + // documents no callback order, so it records (hash, row) pairs and matching to queries happens after the call, + // outside Boost's group locks. Boost invokes it only for an element exactly equal to some chunk query, and + // equal keys have equal hashes: a lone query with that hash is the match, while several (duplicates or a + // 32-bit collision) are separated by exact comparison. Misses produce no call and keep kNotFound. + auto probe_chunk_(const Batch &b, Results &r, size_t c, size_t e) -> void { + std::array keys{}; + for (size_t q = c; q < e; ++q) { + hash_[q] = Store::fold_hash_positions(b.key(q)); + keys[q - c] = QueryKey{b.key(q), hash_[q]}; + } + std::array found{}; + size_t nfound = 0; + map_.cvisit(keys.begin(), keys.begin() + static_cast(e - c), [&](const auto &kv) { + if (nfound < kBulkChunk) { + found[nfound] = RowKey{kv.second, kv.first.hash}; + } + ++nfound; + }); + require(nfound <= e - c, "bulk visit reported more matches than queries"); + for (size_t i = 0; i < nfound; ++i) { + size_t candidates = 0; + size_t only = kNotFound; + for (size_t q = c; q < e; ++q) { + if (hash_[q] == found[i].hash) { + ++candidates; + only = q; + } + } + if (candidates == 1) { + r.id[only] = found[i].row; + continue; + } + for (size_t q = c; q < e; ++q) { + if (hash_[q] == found[i].hash && row_matches(*rows_, found[i].row, b.key(q))) { + r.id[q] = found[i].row; + } + } + } + } + + // Compact handle into the packed rows with its cached 32-bit fold hash; never a copied monomial. + struct RowKey { + TermIndex row; + uint32_t hash; + }; + // Heterogeneous probe key: a view of one query's positions, valid only for the probe call. + struct QueryKey { + std::span pos; + uint32_t hash; + }; + struct KeyHash { + using is_transparent = void; + auto operator()(const RowKey &k) const noexcept -> size_t { return k.hash; } + auto operator()(const QueryKey &k) const noexcept -> size_t { return k.hash; } + }; + struct KeyEqual { + using is_transparent = void; + const Store *rows; + auto operator()(const RowKey &a, const RowKey &b) const -> bool { + return a.row == b.row || (a.hash == b.hash && rows_equal(*rows, a.row, b.row)); + } + auto operator()(const QueryKey &q, const RowKey &k) const -> bool { + return q.hash == k.hash && row_matches(*rows, k.row, q.pos); + } + auto operator()(const RowKey &k, const QueryKey &q) const -> bool { return (*this)(q, k); } + }; + using Map = boost::concurrent_flat_map; + + int threads_; + std::unique_ptr rows_; + Map map_; + std::vector hash_; + std::vector miss_; + std::vector probed_, published_; +}; + +// --------------------------------------------------------------------------------------------------------------- +// Injective synthetic keys: 2N positions split into `length` disjoint bands of `radix` positions; digit i of the +// ordinal (radix `radix`) selects one position of band i after a seed-derived cyclic offset. No dedup table. +template +class KeyGen { +public: + using PosT = typename OperatorIndex::PosT; + + KeyGen(size_t length, uint64_t seed) : length_(length), radix_(length == 0 ? 0 : 2 * N / length) { + require(length >= 1 && radix_ >= 2, "key length must be in [1, modes]"); + uint64_t state = seed ^ 0x6A09E667F3BCC909ULL; + for (size_t i = 0; i < length_; ++i) { + offset_.push_back(splitmix(state) % radix_); + } + capacity_ = 1; + for (size_t i = 0; i < length_ && capacity_ != UINT64_MAX; ++i) { + capacity_ = capacity_ > UINT64_MAX / radix_ ? UINT64_MAX : capacity_ * radix_; + } + } + [[nodiscard]] auto length() const -> size_t { return length_; } + [[nodiscard]] auto capacity() const -> uint64_t { return capacity_; } + auto key(uint64_t ordinal, PosT *out) const -> void { + require(ordinal < capacity_, "ordinal does not fit the key encoding"); + for (size_t i = 0; i < length_; ++i) { + out[i] = static_cast((i * radix_) + (((ordinal % radix_) + offset_[i]) % radix_)); + ordinal /= radix_; + } + } + +private: + size_t length_; + uint64_t radix_; + uint64_t capacity_ = 0; + std::vector offset_; +}; + +// One mixed batch: `hits` distinct existing ordinals (a coprime stride over [0, population)) and `misses` fresh +// ordinals fresh_base, fresh_base+1, ... assigned in query order, so shared IDs equal ordinals after growth. +template +auto make_batch(const KeyGen &gen, + uint64_t seed, + uint64_t batch_index, + uint64_t population, + uint64_t fresh_base, + size_t hits, + size_t misses, + Batch &b, + std::vector &ordinal, + std::vector &declared_hit) -> void { + require(hits <= population, "more distinct hits requested than existing rows"); + const size_t n = hits + misses; + declared_hit.assign(n, 0); + std::fill_n(declared_hit.begin(), hits, uint8_t{1}); + uint64_t state = (seed * 0xD1B54A32D192ED03ULL) ^ (batch_index + 1); + for (size_t i = n; i > 1; --i) { + std::swap(declared_hit[i - 1], declared_hit[splitmix(state) % i]); + } + const uint64_t start = population == 0 ? 0 : splitmix(state) % population; + uint64_t stride = std::max(1, static_cast(static_cast(population) * 0.6180339887)); + while (population > 1 && std::gcd(stride, population) != 1) { + ++stride; + } + b.clear(); + ordinal.resize(n); + std::vector::PosT> buf(gen.length()); + uint64_t j = 0; + uint64_t k = 0; + for (size_t q = 0; q < n; ++q) { + ordinal[q] = declared_hit[q] != 0 + ? static_cast((start + (static_cast(j++) * stride)) % population) + : fresh_base + k++; + gen.key(ordinal[q], buf.data()); + b.push(buf); + } +} + +// Sequential ordinals [first, first+count) in order, for initial streaming fills and final validation. +template +auto make_range(const KeyGen &gen, uint64_t first, size_t count, Batch &b) -> void { + b.clear(); + std::vector::PosT> buf(gen.length()); + for (size_t i = 0; i < count; ++i) { + gen.key(first + i, buf.data()); + b.push(buf); + } +} + +// --------------------------------------------------------------------------------------------------------------- +// Measurement driver: one variant/case per fresh process, one CSV row on success. +struct Config { + std::string variant; + std::string lookup = "per-query"; //!< shared-concurrent probe kernel: per-query or bulk. + bool grow = false; + uint64_t rows = 0; + uint64_t batch = 4096; + uint64_t batches = 16; + uint64_t miss_percent = 0; + uint64_t key_length = 6; + uint64_t modes = 128; + uint64_t seed = 0; + uint64_t inline_width = 11; + bool profile = false; +}; + +struct Totals { + int team_min = 1 << 30, team_max = 0, probe_min = 1 << 30, probe_max = 0, publish_min = 1 << 30, publish_max = 0; + std::array phase{}; + auto add(const BatchObs &o) -> void { + team_min = std::min(team_min, o.team); + team_max = std::max(team_max, o.team); + probe_min = std::min(probe_min, o.probe_active); + probe_max = std::max(probe_max, o.probe_active); + if (o.publish_active > 0) { + publish_min = std::min(publish_min, o.publish_active); + publish_max = std::max(publish_max, o.publish_active); + } + for (size_t p = 0; p < kNumPhases; ++p) { + phase[p] += o.phase[p]; + } + } +}; + +template +auto check_batch(const V &v, + const Batch &b, + const std::vector &ordinal, + const std::vector &declared_hit, + const Results &r, + bool grow, + std::vector &next_new) -> void { + for (size_t q = 0; q < b.size(); ++q) { + const auto key = b.key(q); + const uint32_t o = v.owner_of(key); + require(r.hit[q] == declared_hit[q], "hit/miss disagrees with the declared workload"); + require(r.owner[q] == o, "result owner disagrees with the routing owner"); + if (!declared_hit[q] && !grow) { + require(r.id[q] == kNotFound, "lookup miss returned a row"); + continue; + } + require(v.matches(o, r.id[q], key), "returned row does not hold the exact query key"); + if constexpr (V::kIdsAreOrdinals) { + require(r.id[q] == ordinal[q], "shared ID is not the canonical ordinal"); + } + if (!declared_hit[q]) { + require(r.id[q] == next_new[o]++, "published ID breaks per-owner query order"); + } + } +} + +template +auto measure(V &v, const Config &c, int threads) -> int { + const KeyGen gen(c.key_length, c.seed); + const uint64_t misses = c.batch * c.miss_percent / 100; + const uint64_t hits = c.batch - misses; + const uint64_t final_rows = c.grow ? c.rows + (c.batches * misses) : c.rows; + Batch b; + Results r; + BatchObs scratch; + std::vector ordinal; + std::vector declared; + std::vector next_new(v.owners(), 0); + constexpr uint64_t kFillChunk = 65536; + const auto t_setup = Clock::now(); + for (uint64_t first = 0; first < c.rows; first += kFillChunk) { + const size_t count = std::min(kFillChunk, c.rows - first); + make_range(gen, first, count, b); + v.run(b, true, r, scratch, false); + ordinal.resize(count); + std::iota(ordinal.begin(), ordinal.end(), first); + declared.assign(count, 0); + check_batch(v, b, ordinal, declared, r, true, next_new); + } + const double setup_s = std::chrono::duration(Clock::now() - t_setup).count(); + +#ifdef BOOST_UNORDERED_ENABLE_STATS + // Statistics cover the measured batches only: reset after setup, snapshot before the final validation. + if constexpr (requires { v.reset_boost_stats(); }) { + v.reset_boost_stats(); + } +#endif + Totals tot; + double batch_s = 0; + double validate_s = 0; + uint64_t checksum = 0; + uint64_t population = c.rows; + uint64_t fresh = c.rows; + for (uint64_t bi = 0; bi < c.batches; ++bi) { + make_batch(gen, c.seed, bi, population, fresh, hits, misses, b, ordinal, declared); + BatchObs obs; + const auto t0 = Clock::now(); + v.run(b, c.grow, r, obs, c.profile); + batch_s += std::chrono::duration(Clock::now() - t0).count(); + tot.add(obs); + const auto tv = Clock::now(); + check_batch(v, b, ordinal, declared, r, c.grow, next_new); + for (const auto id : r.id) { + checksum = (checksum * 0x100000001B3ULL) ^ id; + } + validate_s += std::chrono::duration(Clock::now() - tv).count(); + fresh += misses; + population += c.grow ? misses : 0; + } + + std::string stats_header; + std::string stats_row; +#ifdef BOOST_UNORDERED_ENABLE_STATS + if constexpr (requires { v.boost_stats(); }) { + const auto s = v.boost_stats(); + stats_header = ",bs_insert_count,bs_insert_probe_mean,bs_hit_count,bs_hit_probe_mean,bs_hit_cmp_mean," + "bs_miss_count,bs_miss_probe_mean,bs_miss_cmp_mean"; + stats_row = std::format(",{},{:.4f},{},{:.4f},{:.4f},{},{:.4f},{:.4f}", + s.insertion.count, + s.insertion.probe_length.average, + s.successful_lookup.count, + s.successful_lookup.probe_length.average, + s.successful_lookup.num_comparisons.average, + s.unsuccessful_lookup.count, + s.unsuccessful_lookup.probe_length.average, + s.unsuccessful_lookup.num_comparisons.average); + } +#endif + + // Retained-set check without a shadow map: exact count, then a streaming lookup of every expected key. + // Ordinal order is insertion order within every owner, so expected owner-local IDs are one counter each. + const auto tv = Clock::now(); + size_t total = 0; + size_t min_rows = SIZE_MAX; + size_t max_rows = 0; + for (size_t o = 0; o < v.owners(); ++o) { + total += v.owner_size(o); + min_rows = std::min(min_rows, v.owner_size(o)); + max_rows = std::max(max_rows, v.owner_size(o)); + } + require(total == final_rows, "final row count differs from the workload"); + if constexpr (requires { v.index_entries(); }) { + require(v.index_entries() == final_rows, "concurrent index size differs from the row count"); + } + std::vector expect(v.owners(), 0); + for (uint64_t first = 0; first < final_rows; first += kFillChunk) { + const size_t count = std::min(kFillChunk, final_rows - first); + make_range(gen, first, count, b); + v.run(b, false, r, scratch, false); + for (size_t q = 0; q < count; ++q) { + const uint32_t o = v.owner_of(b.key(q)); + require(r.hit[q] == 1 && r.owner[q] == o && r.id[q] == expect[o]++, "retained key has a wrong owner/ID"); + if constexpr (V::kIdsAreOrdinals) { + require(r.id[q] == first + q, "retained shared ID is not canonical"); + } + require(v.matches(o, r.id[q], b.key(q)), "retained row does not hold its exact key"); + } + } + validate_s += std::chrono::duration(Clock::now() - tv).count(); + + const uint64_t queries = c.batch * c.batches; + std::string header = + "variant,mode,modes,key_length,inline_width,rows,batch,batches,miss_percent,seed,requested_threads,team_min," + "team_max,probe_active_min,probe_active_max,publish_active_min,publish_active_max,initial_rows,final_rows," + "hits,misses,setup_s,batch_s,queries_per_s"; + std::string row = std::format("{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{:.6f},{:.6f},{:.1f}", + c.variant, + c.grow ? "grow" : "lookup", + c.modes, + c.key_length, + c.inline_width, + c.rows, + c.batch, + c.batches, + c.miss_percent, + c.seed, + threads, + tot.team_min, + tot.team_max, + tot.probe_min, + tot.probe_max, + tot.publish_max == 0 ? 0 : tot.publish_min, + tot.publish_max, + c.rows, + final_rows, + hits * c.batches, + misses * c.batches, + setup_s, + batch_s, + static_cast(queries) / batch_s); + for (size_t p = 0; p < kNumPhases; ++p) { + header += std::format(",{}_s", kPhaseNames[p]); + // Phases a variant or mode does not have stay empty rather than reporting invented zeros. + const bool applies = V::kPhases[p] && (c.grow || (p != kAssign && p != kFill && p != kPublish)); + row += c.profile && applies ? std::format(",{:.6f}", tot.phase[p]) : std::string(","); + } + header += ",min_shard_rows,max_shard_rows,row_bytes_est,index_bytes_est,harness_bytes_est,validate_s,validation," + "checksum,profile,lookup"; + row += std::format(",{},{},{},{},{},{:.6f},ok,{:016x},{},{}", + min_rows, + max_rows, + v.row_bytes(), + v.index_bytes(), + v.harness_bytes(), + validate_s, + checksum, + c.profile ? 1 : 0, + c.lookup); + std::println("{}{}\n{}{}", header, stats_header, row, stats_row); + return 0; +} + +template +auto measure_variant(const Config &c) -> int { + const int threads = omp_get_max_threads(); // captured once; every region requests exactly this team + if (c.variant == "per-thread") { + PerThread v(threads, threads, c.inline_width); + return measure(v, c, threads); + } + if (c.variant == "shared-serial") { + SharedSerial v(threads, c.inline_width); + return measure(v, c, threads); + } + SharedConcurrent v(threads, c.inline_width); + v.bulk_lookup = c.lookup == "bulk"; + return measure(v, c, threads); +} + +// --------------------------------------------------------------------------------------------------------------- +// Self-tests. The serial key oracle below exists only here, never in measured processes. +template +struct Oracle { + using PosT = typename OperatorIndex::PosT; + std::vector, size_t>> ids; + explicit Oracle(size_t owners) : ids(owners) {} + [[nodiscard]] auto owner_of(std::span key) const -> uint32_t { + return ids.size() == 1 ? 0 + : static_cast(monoprop::monomial_hash(monomial_of(key)) % ids.size()); + } + // Probes the frozen pre-batch state, then assigns misses in query order per owner. + auto apply(const Batch &b, bool grow) -> Results { + Results e; + e.resize(b.size()); + for (size_t q = 0; q < b.size(); ++q) { + const auto o = owner_of(b.key(q)); + const auto it = ids[o].find({b.key(q).begin(), b.key(q).end()}); + e.owner[q] = o; + e.hit[q] = it != ids[o].end(); + e.id[q] = e.hit[q] != 0 ? it->second : kNotFound; + } + for (size_t q = 0; grow && q < b.size(); ++q) { + if (e.hit[q] == 0) { + auto &m = ids[e.owner[q]]; + e.id[q] = m.size(); + m.emplace(std::vector(b.key(q).begin(), b.key(q).end()), e.id[q]); + } + } + return e; + } +}; + +template +auto batch_of(std::initializer_list::PosT>> keys) -> Batch { + Batch b; + for (const auto &k : keys) { + b.push(k); + } + return b; +} + +// Runs one batch through the variant and the oracle; results and every returned row's exact key must agree. +template +auto step(V &v, Oracle &oracle, const Batch &b, bool grow, std::string_view what) -> Results { + Results r; + BatchObs obs; + v.run(b, grow, r, obs, true); + const Results e = oracle.apply(b, grow); + for (size_t q = 0; q < b.size(); ++q) { + require(r.hit[q] == e.hit[q] && r.id[q] == e.id[q] && r.owner[q] == e.owner[q], + std::format("{}: query {} got hit={} owner={} id={}, expected hit={} owner={} id={}", + what, + q, + r.hit[q], + r.owner[q], + r.id[q], + e.hit[q], + e.owner[q], + e.id[q])); + require(r.id[q] == kNotFound || v.matches(r.owner[q], r.id[q], b.key(q)), + std::format("{}: wrong row key", what)); + } + return r; +} + +// Two distinct keys with the same cached 32-bit fold hash, found by a birthday search over synthetic keys. +template +auto hash_collision() -> std::pair, std::vector> { + const KeyGen gen(6, 7); + std::unordered_map seen; + std::vector::PosT> a(6); + std::vector::PosT> c(6); + for (uint64_t ord = 0;; ++ord) { + gen.key(ord, a.data()); + const auto [it, fresh] = seen.emplace(OperatorIndex::fold_hash_positions(a), ord); + if (!fresh) { + gen.key(it->second, c.data()); + return {{c.begin(), c.end()}, {a.begin(), a.end()}}; + } + } +} + +template +auto fixtures(V &v, size_t owners, std::string_view name) -> void { + using PosT = typename OperatorIndex::PosT; + Oracle oracle(owners); + const std::vector a{0, 1}, bk{2, 3}, c{4, 5}, d{6, 7}; + const std::vector far{0, static_cast(2 * N - 1)}; + v.check_distinct = true; + step(v, oracle, Batch{}, true, std::format("{} empty batch/empty store", name)); + step(v, oracle, batch_of({a, far}), false, std::format("{} lookup in empty store", name)); + step(v, oracle, batch_of({a}), true, std::format("{} initial A", name)); + const Results r1 = step(v, oracle, batch_of({bk, a, c}), true, std::format("{} B,A,C", name)); + const Results r2 = step(v, oracle, batch_of({c, bk, a}), false, std::format("{} C,B,A", name)); + if (owners == 1) { + require(r1.hit == std::vector{0, 1, 0} && r1.id == std::vector{1, 0, 2}, "B,A,C shared IDs"); + require(r2.id == std::vector{2, 1, 0}, "C,B,A shared IDs"); + } + step(v, oracle, batch_of({a, a}), false, std::format("{} repeated A,A", name)); + step(v, oracle, batch_of({a, a}), true, std::format("{} repeated A,A (grow)", name)); + const size_t before = [&] { + size_t s = 0; + for (size_t o = 0; o < v.owners(); ++o) { + s += v.owner_size(o); + } + return s; + }(); + bool rejected = false; + try { + Results r; + BatchObs obs; + v.run(batch_of({d, d}), true, r, obs, false); + } + catch (const std::runtime_error &e) { + rejected = std::string_view(e.what()).find("duplicate") != std::string_view::npos; + } + size_t after = 0; + for (size_t o = 0; o < v.owners(); ++o) { + after += v.owner_size(o); + } + require(rejected && after == before, std::format("{}: duplicate-miss fixture not rejected cleanly", name)); + step(v, oracle, Batch{}, false, std::format("{} empty batch/non-empty store", name)); + + // Tails around the 16-query probe pipeline and 64-query blocks, then all-hit and all-miss repeats. + const KeyGen gen(6, 11); + uint64_t next = 1000; + for (const size_t n : {1UL, 15UL, 16UL, 17UL, 63UL, 64UL, 65UL, 4096UL}) { + Batch fresh; + make_range(gen, next, n, fresh); + Batch absent; + make_range(gen, next + n, n, absent); + next += 2 * n; + step(v, oracle, fresh, true, std::format("{} tail {} all-miss grow", name, n)); + step(v, oracle, fresh, false, std::format("{} tail {} all-hit", name, n)); + step(v, oracle, absent, false, std::format("{} tail {} all-miss lookup", name, n)); + } + // Mixed batches from the measured generator. + std::vector ord; + std::vector declared; + for (uint64_t bi = 0; bi < 4; ++bi) { + Batch mixed; + make_batch(gen, 3, bi, 1000, next, 700, 300, mixed, ord, declared); + next += 300; + step(v, oracle, mixed, bi % 2 == 0, std::format("{} mixed batch {}", name, bi)); + } + + // Equal cached hashes: exact comparison must separate the pair before and after table growth. + const auto [x16, y16] = hash_collision(); + const std::vector x(x16.begin(), x16.end()), y(y16.begin(), y16.end()); + require(OperatorIndex::fold_hash_positions(x) == OperatorIndex::fold_hash_positions(y) && x != y, + "collision search"); + step(v, oracle, batch_of({x}), true, std::format("{} collision X", name)); + step(v, oracle, batch_of({y}), false, std::format("{} collision Y absent", name)); + step(v, oracle, batch_of({y, x}), true, std::format("{} collision Y publish", name)); + Batch filler; + make_range(gen, next, 3000, filler); + next += 3000; + step(v, oracle, filler, true, std::format("{} collision growth", name)); + step(v, oracle, batch_of({x, y}), false, std::format("{} collision after growth", name)); + + // Spilled length-16 rows beside inline rows, across repeated backing reallocations. + const KeyGen spill_gen(16, 5); + std::vector> earlier; + for (uint64_t g = 0; g < 12; ++g) { + Batch spill; + make_range(spill_gen, g * 250, 250, spill); + step(v, oracle, spill, true, std::format("{} spill growth {}", name, g)); + earlier.push_back(std::move(spill)); + for (const auto &e : earlier) { + step(v, oracle, e, false, std::format("{} spill re-lookup {}", name, g)); + } + } +} + +// Exercises the three variants at one requested team size. +template +auto self_test_team(int team) -> void { + { + PerThread v(team, team, 11); + fixtures(v, v.owners(), std::format("per-thread N={} T={} S={}", N, team, team)); + } + { + PerThread v(3, team, 11); // fixed shard count: IDs must not depend on the team + fixtures(v, 3, std::format("per-thread N={} T={} S=3", N, team)); + } + { + SharedSerial v(team, 11); + fixtures(v, 1, std::format("shared-serial N={} T={}", N, team)); + } + { + SharedConcurrent v(team, 11); + fixtures(v, 1, std::format("shared-concurrent N={} T={}", N, team)); + } + { + SharedConcurrent v(team, 11); + v.bulk_lookup = true; + fixtures(v, 1, std::format("shared-concurrent/bulk N={} T={}", N, team)); + } +} + +// Actual participation in probe/publication bodies and joined worker exceptions, at a team of four. +template +auto self_test_team_behaviour(std::string_view name, V &&make) -> void { + Batch<128> b; + make_range(KeyGen<128>(6, 1), 0, 4096, b); + Results r; + BatchObs obs; + auto v = make(); + v->run(b, true, r, obs, true); + if constexpr (requires { v->bulk_lookup; }) { + // 4096 queries in 16-query chunks: the bulk path issues exactly one call per chunk at any team size. + require(!v->bulk_lookup || obs.bulk_calls == 4096 / 16, std::format("{}: bulk path not taken", name)); + } + if (obs.team < 4) { + std::println("SKIP participation {}: runtime limited the team to {} (not T-core evidence)", name, obs.team); + } + else { + require(obs.probe_active == 4, std::format("{}: only {} probe workers active", name, obs.probe_active)); + std::println("participation {}: team={} probe_active={} publish_active={}", + name, + obs.team, + obs.probe_active, + obs.publish_active); + } + if (obs.team < 2) { + std::println("SKIP worker-exception {}: team of one", name); + return; + } + auto w = make(); + g_fault_tid.store(1); + bool caught = false; + try { + w->run(b, true, r, obs, false); + } + catch (const std::runtime_error &e) { + caught = std::string_view(e.what()) == "injected worker fault"; + } + g_fault_tid.store(-1); + require(caught, std::format("{}: worker exception was not joined and rethrown", name)); + std::println("worker-exception {}: joined and rethrown on the caller", name); +} + +#ifdef BOOST_UNORDERED_ENABLE_STATS +// Diagnostic builds only: Boost's own counters must see exactly the index traffic the adapter issues. +auto self_test_boost_stats() -> void { + for (const bool bulk : {false, true}) { + SharedConcurrent<128> v(1, 11); + v.bulk_lookup = bulk; + Batch<128> b; + make_range(KeyGen<128>(6, 2), 0, 100, b); + Results r; + BatchObs obs; + v.reset_boost_stats(); + v.run(b, true, r, obs, false); // 100 misses, then 100 inserts + v.run(b, false, r, obs, false); // 100 hits + const auto s = v.boost_stats(); + // Boost counts its own traffic too: each emplace performs an absence lookup (an unsuccessful lookup, retried + // after a rehash), and rehash transfers count as insertions. Only hits map one-to-one to adapter probes. + require(s.successful_lookup.count == 100 && s.unsuccessful_lookup.count >= 200 && s.insertion.count >= 100, + std::format("boost stats (bulk={}): miss={} insert={} hit={}", + bulk, + s.unsuccessful_lookup.count, + s.insertion.count, + s.successful_lookup.count)); + } + std::println("boost container stats: hit counts match the adapter's probes"); +} +#endif + +auto self_test() -> int { +#ifdef BOOST_UNORDERED_ENABLE_STATS + self_test_boost_stats(); +#endif + for (const int team : {1, 2, 3, 4}) { + self_test_team<128>(team); + self_test_team<1024>(team); + std::println("fixtures ok at requested team {}", team); + } + self_test_team_behaviour("per-thread", [] { return std::make_unique>(4, 4, 11); }); + self_test_team_behaviour("shared-serial", [] { return std::make_unique>(4, 11); }); + self_test_team_behaviour("shared-concurrent", [] { return std::make_unique>(4, 11); }); + self_test_team_behaviour("shared-concurrent/bulk", [] { + auto v = std::make_unique>(4, 11); + v->bulk_lookup = true; + return v; + }); + std::println("self-test passed (Boost {}.{}, _OPENMP {})", + BOOST_VERSION / 100000, + BOOST_VERSION / 100 % 1000, + _OPENMP); + return 0; +} + +auto parse_u64(std::string_view flag, std::string_view text) -> uint64_t { + uint64_t value = 0; + const auto [end, ec] = std::from_chars(text.data(), text.data() + text.size(), value); + require(ec == std::errc{} && end == text.data() + text.size() && !text.empty(), + std::format("{}: expected a non-negative decimal integer, got '{}'", flag, text)); + return value; +} + +auto checked_mul(uint64_t a, uint64_t b) -> uint64_t { + uint64_t out = 0; + require(!__builtin_mul_overflow(a, b, &out), "workload size overflows 64-bit arithmetic"); + return out; +} + +auto parse(int argc, char **argv) -> Config { + Config c; + std::string mode; + for (int i = 1; i < argc; ++i) { + const std::string_view flag = argv[i]; + if (flag == "--profile") { + c.profile = true; + continue; + } + require(i + 1 < argc, std::format("{}: missing value", flag)); + const std::string_view value = argv[++i]; + if (flag == "--variant") { + c.variant = value; + } + else if (flag == "--lookup") { + c.lookup = value; + } + else if (flag == "--mode") { + mode = value; + } + else if (flag == "--rows") { + c.rows = parse_u64(flag, value); + } + else if (flag == "--batch") { + c.batch = parse_u64(flag, value); + } + else if (flag == "--batches") { + c.batches = parse_u64(flag, value); + } + else if (flag == "--miss-percent") { + c.miss_percent = parse_u64(flag, value); + } + else if (flag == "--key-length") { + c.key_length = parse_u64(flag, value); + } + else if (flag == "--modes") { + c.modes = parse_u64(flag, value); + } + else if (flag == "--seed") { + c.seed = parse_u64(flag, value); + } + else if (flag == "--inline-width") { + c.inline_width = parse_u64(flag, value); + } + else { + require(false, std::format("unknown option {}", flag)); + } + } + require(c.variant == "per-thread" || c.variant == "shared-serial" || c.variant == "shared-concurrent", + "--variant must be per-thread, shared-serial or shared-concurrent"); + require(mode == "lookup" || mode == "grow", "--mode must be lookup or grow"); + require(c.lookup == "per-query" || (c.lookup == "bulk" && c.variant == "shared-concurrent"), + "--lookup must be per-query, or bulk with --variant shared-concurrent"); + c.grow = mode == "grow"; + require(c.modes == 128 || c.modes == 1024, "--modes must be 128 or 1024"); + require(c.miss_percent <= 100, "--miss-percent must be in [0, 100]"); + require(c.batch >= 1 && c.batches >= 1, "--batch and --batches must be positive"); + require(c.inline_width >= 1 && c.inline_width <= OperatorIndex<128>::kMaxInlinePositions, + "--inline-width must be in [1, 32]"); + require(c.key_length >= 1 && c.key_length <= c.modes, "--key-length must be in [1, modes]"); + const uint64_t misses = checked_mul(c.batch, c.miss_percent) / 100; + require(c.batch - misses <= c.rows, "more distinct hits per batch than initial rows"); + const uint64_t added = checked_mul(c.batches, misses); + require(added <= UINT64_MAX - c.rows, "workload size overflows 64-bit arithmetic"); + // Lookup misses use fresh never-inserted ordinals too, so both modes need rows+added encodable keys. + require(c.rows + added < OperatorIndex<128>::kIndexCeiling, "study sizes must stay below the fixed32 ceiling"); + const uint64_t capacity = + c.modes == 128 ? KeyGen<128>(c.key_length, 0).capacity() : KeyGen<1024>(c.key_length, 0).capacity(); + require(c.rows + added <= capacity, "key length cannot encode this many distinct keys"); + return c; +} + +} // namespace + +auto main(int argc, char **argv) -> int { + try { + if (argc == 2 && std::string_view(argv[1]) == "--self-test") { + return self_test(); + } + const Config c = parse(argc, argv); + return c.modes == 128 ? measure_variant<128>(c) : measure_variant<1024>(c); + } + catch (const std::exception &e) { + std::println(stderr, "index-miniapp: FAILED: {}", e.what()); + return 1; + } +} From 1ebd05f3ba662195f37e0decebf7153a0cdd0e2a Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Roberto=20Di=20Remigio=20Eik=C3=A5s?= Date: Fri, 25 Sep 2026 12:53:52 +0000 Subject: [PATCH 06/64] =?UTF-8?q?docs(parallelism):=20=F0=9F=93=9D=20recor?= =?UTF-8?q?d=20Task=200=20outcome=20and=20defer=20the=20ownership=20decisi?= =?UTF-8?q?on?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Record the owner's decision after Task 0: the evidence was inconclusive for the ownership axis, so one store per rank remains the working hypothesis. Summarize the mini-app, in-library counter, one-core profile and Boost-index trial results, and record the waiver of Boost 1.85 compile checks. Move the ownership decision to Task 11's full-library comparison against the partition runtime, add a diagnostic go/no-go after Task 6 before the Task 10 API cutover, and make Task 12 conditional on confirming the single store. Replace the bounded Boost trial with a shared-serial versus shared-concurrent index A/B, and add an invariant that keeps OperatorIndex swappable. Assisted-by: Pi:claude-opus-5-5 --- .../plans/2026-09-18-rank-local-openmp.md | 160 ++++++++++++------ .../2026-09-18-rank-local-openmp-design.md | 113 +++++++++---- 2 files changed, 188 insertions(+), 85 deletions(-) diff --git a/docs/superpowers/plans/2026-09-18-rank-local-openmp.md b/docs/superpowers/plans/2026-09-18-rank-local-openmp.md index b075327d..417043bc 100644 --- a/docs/superpowers/plans/2026-09-18-rank-local-openmp.md +++ b/docs/superpowers/plans/2026-09-18-rank-local-openmp.md @@ -78,7 +78,7 @@ post-experiment decision without silently weakening that coverage or any numeric - SMT use is not recommended: allocate at most one monoprop worker per physical core, including on SMT-enabled hosts. Enforce this through user launch/binding settings and acceptance evidence, not a library hardware policy. - Preserve packed rows, keyless indexing through row references, fixed uint32_t TermIndex, sparse state storage, and - inverted-index cosine recomputation. A bounded Boost concurrent-index experiment may replace the index, not rows. + inverted-index cosine recomputation. The Task 11 index A/B may replace the index, not rows. - Do not introduce hidden operator shards, a dense duplicate store, a second persistent full-key store, per-thread operators, or full-vector coefficient double buffering. Select any concurrent index only after correctness/memory/time evidence; its container name is neither a rejection criterion nor proof of suitability. @@ -141,13 +141,13 @@ Additional execution rules: | Stage | Tasks | Gate before proceeding | | --- | --- | --- | -| Architecture experiment | 0 | Small MPI-off mini-app, correct three-way comparison, owner architecture decision and reconciled documents | +| Architecture experiment | 0 | Done: evidence inconclusive for ownership; one store per rank is the working hypothesis (see Task 0 outcome) | | Baseline (conditional) | 1 | Task 0 decision, authorized baseline-only calibration, frozen inventory, fresh formal baseline evidence | | Small prototype | 2–4 | OpenMP packaging works; one-store serial and threaded replay agree | -| Shared-store construction | 5–7 | Stable queries/IDs, safety, construction scaling and scratch measured | +| Shared-store construction | 5–7 | Stable queries/IDs, safety, construction scaling and scratch measured; post-Task-6 architecture go/no-go | | Remaining evaluation kernels | 8–9 | Replay, derivative, retained closures and reductions validated | -| Integration cutover | 10–11 | Both APIs and MPI×threads pass; benchmark parity at final geometry | -| Removal/release | 12–13 | Old runtime removed, final full matrix and parity rerun | +| Integration cutover | 10–11 | Go/no-go passed before Task 10; parity at final geometry decides ownership; index A/B in Task 11 | +| Removal/release | 12–13 | Only after Task 11 confirms the single store; old runtime removed, final full matrix and parity rerun | One writer per checkout. Kernel and storage tasks are ordered: do not have agents concurrently edit `Engine.h`, `Evolution.cpp`, or `MonomialPropagator.inl`. Independent read-only reviews are safe. Do not batch past a failed gate. @@ -305,6 +305,12 @@ must enter CMake FILE_SETs, not merely compiler include paths. not extend that calling contract. 13. A post-mutation failure invalidates the affected owner and its dependent functionals. Check validity before later state access or copying; independent earlier copies remain valid. Pre-mutation validation errors do not poison. +14. Keep the index swappable for Task 11's A/B trial: + - Nothing outside `OperatorIndex` touches the hash table's internals (slot layout, `insert_slot_`, probe chains). + - No caller depends on the table's iteration order. Clone and memory accounting go through the public API. + - Publication goes through `bulk_insert_hashed`, whose parallel-options argument the packed index may ignore. + - The duplicate-key behaviour of `insert_absent_terms` / `bulk_insert` is pinned by a test. Today the packed index + indexes both rows; any change is an explicit, reviewed contract change. ## Build/test command library @@ -501,9 +507,9 @@ enable_testing() add_test(NAME index-miniapp-self-test COMMAND index-miniapp --self-test) ``` - Keep monoprop_ENABLE_MPI undefined; reject a build where it is defined. Confirm no MPI library is linked. Verify the - needed Boost APIs at the supported 1.85 floor, not only on the experiment host. If unavailable, report the concurrent - arm as unavailable and stop for a decision; do not silently raise the floor or substitute a different container. + Keep monoprop_ENABLE_MPI undefined; reject a build where it is defined. Confirm no MPI library is linked. The owner + waived compile/test verification against the Boost 1.85 floor after the first pass; building with the host's Boost + (>= 1.85, as `find_package` requires) suffices. Do not silently raise the floor or substitute a different container. - [ ] Generate identical deterministic global keys/query batches for every variant and T. Use an injective synthetic encoding of logical key ordinals into sorted positions: divide the 2*NumModes positions into key_length disjoint bands, encode ordinal digits in that radix, and apply a seed-derived cyclic offset within each band. Reject an @@ -613,10 +619,29 @@ across all three size tiers; do not choose protected configurations after seeing mini-app speedup threshold replaces the owner decision or weakens the five full-library gates. Task 0 outputs are never formal Task 1 samples, and its whole-process RSS cannot certify the library's operation/construction windows. +**Task 0 outcome (owner decision):** the evidence is inconclusive for the ownership axis. One packed store per rank +remains the working hypothesis for Tasks 1–13, and the per-thread (sharded) design stays available through the +existing partition runtime until Task 12. The ownership decision moves to Task 11's full-library comparison against the +old partitions, with an earlier diagnostic go/no-go after Task 6. The index axis — current packed index with serial +publication versus `boost::concurrent_flat_map` with parallel publication — stays open until Task 11's A/B trial. +Evidence (measured on the target host, not acceptance samples): +- The mini-app favoured neither design uniformly. Per-thread won growth with 100% new terms; the shared stores won + lookups; the concurrent index lost 1.2–2.1× on one core. +- Instrumented `propagate` on the benchmark models: lookups hit 80–89% on Hubbard and Pauli, and 0% on the random + circuits. The scan takes 66–95% of one-core time; lookups take 19–21% on the physics models and 2% on random. +- In-library Boost trial: +15–18% end-to-end on one core for Hubbard/Pauli at L1 size. Parallel publication gave no gain + at per-gate batch sizes; the scan was still serial, so there is no full-machine result. + +The results, overlays and budget ledger are archived outside the repository with the Task 0 report. The owner waived +the Boost 1.85 compile check, as stated above. + ### Task 1: Freeze baselines and repair measurement trustworthiness -**Prerequisite:** Task 0 results have been reviewed by the owner, the architecture selected and both documents -reconciled. The following remains the conditional one-store campaign, not an automatic continuation of the mini-app. +**Prerequisite:** Task 0 results have been reviewed by the owner and recorded (see the Task 0 outcome), and both +documents have been reconciled. The following remains the conditional one-store campaign, not an automatic continuation +of the mini-app. The formal baseline observations of the partition runtime, including MPI-off full-machine cells, +are also the sharded reference for the post-Task-6 go/no-go and Task 11's ownership decision. No extra measurement +framework is needed. **Benchmark roadmap: use [benches/LADDER.md](../../../benches/LADDER.md), not a new suite.** N is nodes, C is usable physical cores/node, R is ranks/node, and the ladder's P is old partitions/rank (not the generic process-count P used @@ -1616,7 +1641,17 @@ rows, and a prebuilt inverted index. **Gate:** R/M lookup and fresh-insert tests pass with exact fixed-geometry IDs. Report probe/prefix/publication/inverted-index times separately. If serial hash publication dominates, report its Amdahl limit -and carry the evidence into the bounded Task 11 Boost trial. Do not improvise a custom concurrent table here. +and carry the evidence into the Task 11 index A/B. Do not improvise a custom concurrent table here. Give +`bulk_insert_hashed` a trailing `parallel::Options` argument now (the packed index publishes serially and ignores it), +so the Task 11 variant needs no call-site changes (shared invariant 14). + +**Architecture go/no-go (diagnostic, after Task 6 and before Task 10):** with the Task 1 harness, compare the +one-store prototype against the frozen partition baseline at matched geometries. The prototype has threaded replay +(Task 4), traversal (Task 5) and frozen lookup (Task 6). Cover L1 (R=1,T=1), the NUMA-local control and MPI-off full +machine for representative small/medium profiles. These are fresh diagnostic observations, never formal samples, and +they cannot waive a Task 11 cell. Report per-phase time and peak memory. If the one-store path trails clearly and no +bounded Task 11 optimization explains a route to parity, stop for the owner's decision before Task 10 removes the +partition API. This is the last cheap point to revert: only the additive one-store path would be dropped. ### Task 7: Harden rank-wide capacity and optionally parallelize append-only row fill @@ -1784,6 +1819,9 @@ alone is insufficient. A key-set change blocks the gate. Stop and report, never ### Task 10: Cut over configuration and API semantics to one store +**Prerequisite:** the post-Task-6 go/no-go passed, or the owner explicitly chose to continue. Reverting this task +later means reverting the API/configuration cutover; the partition runtime itself remains until Task 12. + **Files:** - Modify: `cpp/include/monoprop/MonomialPropagator.h`, `cpp/monoprop/detail/monomial_propagator/MonomialPropagator.inl`, `cpp/monoprop/detail/EnvConfig.h`, `cpp/monoprop/detail/parallel/{ThreadBudget.h,ThreadBudget.cpp}`. @@ -1953,6 +1991,10 @@ no container-plugin framework. Do not rename `bench_*` functions or move benchma unit-tested. Run five alternating **fresh-process** observations per arm/cell; external repetition only, `--bench-rounds=1`. Include first-call/cold cache and separately measured repeated warm calls without overlapping object construction. +- [ ] Treat this comparison as the ownership decision recorded in Task 0: one store per rank against the sharded + partition runtime, on the frozen campaign with all five gates. If a required cell still fails after the prescribed + repetitions, the bounded optimizations and the index A/B below, keep the partition runtime and stop with a + regression report. Task 12 does not start, and the owner decides whether to revise the design. For required MPI-off full-machine observations, select the R (MPI-disabled) build on both arms and verify with `uv run --no-sync python -c 'import monoprop, sys; sys.exit(int(monoprop.has_mpi))'` (exit 0 required). @@ -2032,52 +2074,70 @@ OMP_PLACES=cores is not a worker-count cap, and merely setting OMP_NUM_THREADS d Beyond the bounded index trial below, whole-batch streaming/windowing, a custom concurrent table, new transport, executor, rank redistribution, full per-thread stores or a persistent duplicate full-key map require a new design. -#### Bounded packed-index versus Boost concurrent-index trial - -Task 0 already investigates ownership/index scaling in isolation. This task validates integration of the same Boost -candidate only if retained by the owner; it is not a second container search. Reuse its feasibility findings and tiny -fixtures, not its timings as full-library evidence. If the architecture changed, revise this task before executing it. - -This is one development-only index variant, not an unbounded container search or a permanent backend choice. -It needs authorized implementation/measurement budget; it is not part of the two-hour baseline-only sizing pilot. -Investigate feasibility from source, then compare focused lookup/insertion/growth measurements and end-to-end build. If -inadmissible, record the concrete invariant/API/version obstacle, not an unsupported claim that concurrent maps lose. - -- [ ] Verify the project's supported Boost versions provide the APIs used; do not silently raise the dependency floor - from the version available on the experiment host. Use `boost::concurrent_flat_map` as an index into the packed rows, - with compact stored row references and heterogeneous position-query lookup. Full monomials, query-buffer spans and a - second persistent reference index are forbidden as stored duplicate keys. Hash/equality must represent monomial - identity, not compare row IDs as if they were term keys. Retain exact collision checks and cached query hashes. -- [ ] Preserve the current lookup/batched result contract. Bulk visitation callbacks are not an ordered query/result - zip: test hits, misses, repeated queries and output ordinals explicitly. A bounded per-query visitor implementation - is an acceptable first trial; do not invent side-effecting hash/equality functions to recover callback order. Never - re-enter the same concurrent container from its visitor. Audit index iteration callers: canonical row order comes - from row IDs, not concurrent bucket iteration order. +#### Index A/B: shared-serial versus shared-concurrent + +Compare two build variants of the same one-store engine, with everything else identical: +- **shared-serial:** the current packed `OperatorIndex`, parallel frozen probes and serial publication. +- **shared-concurrent:** the same packed rows indexed by `boost::concurrent_flat_map` over `{row ID, cached hash}` + handles, with bulk lookup and parallel publication of preassigned IDs. + +The variants differ only inside `OperatorIndex` and at its publication call sites; shared invariant 14 keeps them +local. This is a development-only A/B between an existing container and the reference, not a container search or a +permanent backend selector. It needs authorized implementation/measurement budget. Task 0's in-library trial is the +starting point (archived overlay patch); its timings are not acceptance evidence. + +- [ ] Prerequisite: Tasks 5–6 are complete, so traversal and lookup are parallel. With a serial scan (66–95% of one-core + time in Task 0) the variants cannot be told apart at full machine. +- [ ] Build the variants from one development-branch compile-time option; never keep two resident indices. Run the full + C++/Python suites on both during development. Task 0 found bitwise-identical term maps and energies, and one + duplicate-key fixture that depends on the contract pinned by invariant 14. +- [ ] Boost: the owner waived compile/test verification against the 1.85 floor. Bulk visitation exists at 1.85. + Container statistics (`BOOST_UNORDERED_ENABLE_STATS`) need Boost 1.86 or newer, are for diagnostic builds only, and + serialize on one lock: never time them. +- [ ] Use `boost::concurrent_flat_map` as an index into the packed rows, with compact stored row references and + heterogeneous position-query lookup. Full monomials, query-buffer spans and a second persistent reference index are + forbidden as stored duplicate keys. Hash/equality must represent monomial identity, not compare row IDs as if they + were term keys. Retain exact collision checks and cached query hashes. +- [ ] Preserve the batched lookup/result contract. + - Use bulk visitation in chunks of at most 16 queries. The visitor receives only matched elements, in no documented + order, so it only records `(cached hash, row ID)` pairs. + - After the call and outside Boost's locks, match them to queries: a query alone with that hash in its chunk is the + match; several such queries (duplicates or a collision) are separated by exact row comparison. Misses produce no + call. + - Never use side-effecting hash/equality, re-enter the container from a visitor, or assume callback order. A + per-query visitor is an acceptable fallback. + - Canonical row order comes from row IDs, not bucket iteration. - [ ] Complete frozen probing and deterministic missing-ID assignment before any publication. Caller checks fixed32 and allocation limits, reserves rows/table, initializes packed rows and overflow entries, then workers publish only the index entries for preassigned IDs. No row growth, coefficient resize, overflow mutation, inverted-index mutation, graph packing or external-key relocation while map workers run. Indexed row contents remain immutable until the index is cleared/rebuilt. Join before dependent reads, structural maintenance or freeing scratch. Preserve remote leader → remote follower → deferred self leader → deferred self follower publication phase order and matched timing. -- [ ] Keep one index per independent packed store; no reference index alongside Boost. Copy/move/clone must bind - row-aware hash/equality state to the destination's rows, never a destroyed source. Container synchronization does not - make external-row lifetimes safe. Keep exceptions inside Workshare, join, then use existing caller invalidation and - distributed-failure boundaries, including during active-ticket lifetime. No worker MPI or scheduling-based ID counter. + Callbacks supplied by callers of `bulk_insert` (construction) are not assumed thread-safe: publish those serially. +- [ ] Keep one index per independent packed store. Copy/move/clone must bind row-aware hash/equality state to the + destination's rows, never a destroyed source. Container synchronization does not make external-row lifetimes safe. + Keep exceptions inside Workshare, join, then use existing caller invalidation and distributed-failure boundaries, + including during active-ticket lifetime. No worker MPI or scheduling-based ID counter. - [ ] Extend tests before adoption: deliberate hash collisions with distinct keys; repeated lookup queries; mixed - hit/miss batches and exact row IDs at budgets 1/2/3/4; inline and spilled rows; reserve/growth boundaries; duplicate - insertion contract; clone lookup after source mutation/destruction; allocation/publication failure invalidation; and - end-to-end term/graph equivalence at the same geometry. Preservation tests may already pass; new variant-path and - worker-participation tests provide the new-path evidence. Run qualified race checks without blanket suppressions. -- [ ] Compare separate build variants on identical frozen inputs/allocation. Measure hit/miss lookup, insertion, - reserve/rehash time and transient peaks, row/index/overflow bytes, total construction peak and build/propagate time. - Then rerun every affected formal cell's five metrics and numerical checks against Task 1's refreshed-main baseline; - report the comparison with the packed-index candidate too. No new sizes, higher memory allowance or acceptance waiver. - Microbenchmark gains do not override end-to-end regressions. Obtain approval for any spend beyond the approved budget. -- [ ] If selected, remove the development selector/reference variant before final acceptance and rerun affected gates - on that final binary. If rejected, keep the packed index and archive the evidence. If neither can pass, stop: the - experiment is permission to investigate an existing container, not to rescue the architecture at any cost. + hit/miss batches and exact row IDs at budgets 1/2/3/4; inline and spilled rows; reserve/growth boundaries; the pinned + duplicate-key contract; clone lookup after source mutation/destruction; allocation/publication failure invalidation; + and end-to-end term/graph equivalence at the same geometry. Add a worker-participation test on the variant path. Run + qualified race checks without blanket suppressions. +- [ ] Measure both variants with the Task 1 harness on identical frozen inputs, allocation and placement: fresh-process + pairs, alternating order, all five metrics, plus per-phase lookup/publication/reindex times and peaks. Cover at least: + - L1 (R=1,T=1) cells: Task 0 measured shared-concurrent 15–18% slower on Hubbard/Pauli L1; + - MPI-off full machine; + - the all-new-terms random profiles, where serial publication is the concern; + - the physics models (80–89% hits). + + No new sizes, higher memory allowance or acceptance waiver. Microbenchmark gains do not override end-to-end + regressions. Obtain approval for any spend beyond the approved budget. +- [ ] Decide from the required cells, not from index-phase speedups. Record the choice, remove the losing variant and + the build option before final acceptance, and rerun the affected gates on the final binary. If neither variant can + pass, stop: the A/B is permission to compare an existing container, not to rescue the architecture at any cost. **Gate:** All required performance/memory cells pass with attached raw observations and numerical/placement evidence. +The ownership decision and the index A/B outcome are recorded, and only the chosen variant remains. Missing required EC2 evidence, capacity failure, persistent serial-publication bottleneck or repeated ratio>1.00 blocks removal/acceptance. Report it honestly; the design is not a promise of achievable parity. @@ -2106,7 +2166,9 @@ removal/acceptance. Report it honestly; the design is not a promise of achievabl Kind/Shm/Hybrid and associated pointers/constructors/dispatch. Keep non-MPI behavior, ordinary MPI, PeerPlan routing and existing typed exchanges. Do not collapse it to a raw alias if that changes overload behavior unnecessarily. -- [ ] Start only after Task 11 parity under separately authorized implementation. The owner approved the immediate +- [ ] Start only after Task 11 parity, and only if Task 11's ownership decision confirmed the single store, under + separately authorized implementation. This is the irreversible step: afterwards, returning to sharding means + restoring the partition runtime, transport, topology code and hwloc dependency. The owner approved the immediate partition-related C++ extension/ABI break; confirm no runtime path constructs old groups before deleting definitions. - [ ] Remove any remaining `partition_group_`, friendship and unused fan-out implementation. Confirm Task 10 already removed resolve_partition_count_, public partition/factory arguments and facade-only test users; remove remaining diff --git a/docs/superpowers/specs/2026-09-18-rank-local-openmp-design.md b/docs/superpowers/specs/2026-09-18-rank-local-openmp-design.md index 956c44b6..8609d4cd 100644 --- a/docs/superpowers/specs/2026-09-18-rank-local-openmp-design.md +++ b/docs/superpowers/specs/2026-09-18-rank-local-openmp-design.md @@ -16,12 +16,16 @@ mandatory OpenMP, removal of direct hwloc and partition configuration without de API/ABI and capacity breaks, environment-only construction-time thread budgets, and the failure/calling contracts below. These are design approvals, not implementation or publication authority for this revision. -**Architecture decision reopened:** the owner now requires Task 0, a small MPI-off index mini-app comparing per-thread -stores, one shared non-concurrent store and one shared concurrent index. Its evidence can overturn the one-store choice. -The one-store design and Tasks 1–13 below are conditional, not an instruction to proceed regardless of that result. -After Task 0, stop for the owner's architecture decision and reconcile both documents before any downstream execution. -The follow-up interview approved a staged index-phase study and required full-machine MPI-off library acceptance, -including selected large profiles. These requirements are fixed before results, not deferred to the winning design. +**Architecture decision (Task 0 outcome):** Task 0 ran on the target c8a.metal-24xl, with owner-approved follow-ups. +It compared per-thread stores, one shared non-concurrent store and one shared concurrent index, then instrumented and +trialled the real library. The owner judged the evidence inconclusive for the ownership axis. The one-store design is +the working hypothesis for Tasks 1–13, not a settled choice. +- Ownership is decided by Task 11's full-library comparison against the existing partition runtime, which is the + sharded design. A diagnostic go/no-go after Task 6 is the last cheap point to revert before Task 10's API cutover. +- The index axis (current packed index with serial publication versus a Boost concurrent index) stays open until the + Task 11 A/B. +- The follow-up interview approved a staged index-phase study and required full-machine MPI-off library acceptance, + including selected large profiles. These requirements are fixed before results, not deferred to the winning design. This amendment authorizes planning only, not mini-app implementation, measurements or remote expenditure. The owner selected one AWS EC2 `c8a.metal-24xl` for implementation/testing and the initial acceptance campaign. @@ -63,9 +67,10 @@ separate gates. Later semantic source drift still requires review against the pl ## Objective -First investigate MPI-off index scaling with Task 0 before choosing the ownership architecture. The conditional -proposal is to replace thread-owned operator partitions and their in-process communication protocols with one compact -operator store per MPI rank, operated on by OpenMP worksharing. Preserve mathematical features, memory footprint and +Task 0 investigated MPI-off index scaling; its evidence did not settle the ownership architecture, so full-library +evidence decides it (Task 11) before the legacy runtime is removed. The working proposal is to replace thread-owned +operator partitions and their in-process communication protocols with one compact operator store per MPI rank, operated +on by OpenMP worksharing. Preserve mathematical features, memory footprint and performance. Simplification without regressions is the objective; neither one-store variant is a predetermined winner. Required MPI-enabled acceptance cells keep each rank's CPUs within one NUMA domain; MPI-enabled full-node cross-NUMA L2a remains a diagnostic. Separately, a real MPI-disabled build using the full machine is a required library gate, @@ -136,15 +141,33 @@ Task 0 ends with raw results and an owner decision: retain a shared-store path, the evidence inconclusive. Do not auto-select the fastest lookup kernel or invent a fourth architecture. Any revised production design needs an updated spec/plan and approval; it must preserve the now-required MPI-off acceptance coverage and strict library gates unless the owner explicitly revises them. Task 0 neither consumes Task 1's separate -two-hour baseline pilot nor supplies its calibration or acceptance samples. The later Task 11 trial is integration -validation of a retained candidate, not a second container search. +two-hour baseline pilot nor supplies its calibration or acceptance samples. The later Task 11 index A/B is integration +validation of retained candidates, not a second container search. + +**Recorded outcome.** The mini-app favoured no design uniformly: +- per-thread won growth where 100% of lookups create new terms; +- the shared stores won lookups; +- the concurrent index was 1.2–2.1× slower on one core. + +Counters in the library's layer builder on the benchmark models showed: +- lookups hit an existing term 80–89% of the time on Hubbard and Pauli, and almost never on the random circuits; +- the scan takes 66–95% of one-core time. + +An in-library trial replacing the index with `boost::concurrent_flat_map` was 15–18% slower end-to-end on one core for +Hubbard/Pauli at L1 size. Parallel publication gave no gain at per-gate batch sizes. The scan stayed serial, so the +trial gives no full-machine result. + +The owner judged this inconclusive, kept one store per rank as the working hypothesis, and waived compile/test +verification against the Boost 1.85 floor. The evidence and patches are archived outside the repository with the Task 0 +report; none of it is acceptance evidence. ## Global constraints The storage constraints describe the conditional one-store path in Tasks 1–13. Task 0 alone permits the explicit per-thread-store experiment, not production shards. Full-machine MPI-off measurements are required both in Task 0 -and in full-library acceptance, including cross-NUMA placement. Reconcile the storage design with the owner's -post-experiment decision without silently weakening that coverage or any numerical/performance gate. +and in full-library acceptance, including cross-NUMA placement. The storage design follows the recorded Task 0 outcome: +the single store is the working hypothesis, and sharding stays available through the partition runtime until Task 11 +confirms the single store. Neither step may silently weaken that coverage or any numerical/performance gate. - C++23; retain the GCC 14 / Clang 18 minimum compiler versions and Python 3.11 floor. - MPI remains optional and OFF by default; retain working non-MPI wheels. @@ -160,7 +183,7 @@ post-experiment decision without silently weakening that coverage or any numeric - SMT use is not recommended: allocate at most one monoprop worker per physical core, including on SMT-enabled hosts. Enforce this through user launch/binding settings and acceptance evidence, not a library hardware policy. - Preserve packed rows, keyless indexing through row references, fixed uint32_t TermIndex, sparse state storage, and - inverted-index cosine recomputation. A bounded Boost concurrent-index experiment may replace the index, not rows. + inverted-index cosine recomputation. The Task 11 index A/B may replace the index, not rows. - Do not introduce hidden operator shards, a dense duplicate store, a second persistent full-key store, per-thread operators, or full-vector coefficient double buffering. Select any concurrent index only after correctness/memory/time evidence; its container name is neither a rejection criterion nor proof of suitability. @@ -191,7 +214,12 @@ post-experiment decision without silently weakening that coverage or any numeric ## Architecture -This section describes the one-store candidate, conditional on the Task 0 decision. +This section describes the one-store candidate, the working hypothesis after Task 0. Task 11 confirms or rejects it +against the partition runtime before Task 12 deletes that runtime. The diagnostic go/no-go after Task 6 precedes the +Task 10 API cutover. Keep `OperatorIndex` swappable for the Task 11 index A/B: +- no code outside it depends on table internals or iteration order; +- publication goes through `bulk_insert_hashed` with a parallel-options argument the packed index may ignore; +- the duplicate-key behaviour of bulk insertion is pinned by a test. Each independent propagator has its rank-local `MPOperator`, primary `MPGraph`, and ordinary MPI communicator. This is not a process-wide singleton: copies own independent packed stores; retained functionals may own coefficient/state @@ -206,26 +234,39 @@ resizes shared containers and publishes structural changes between phases. Initi table and inverted-index maintenance remain single-writer. This is an explicit performance gate, not an assumption that insertion is negligible. -### Bounded concurrent-index experiment +### Index A/B: shared-serial versus shared-concurrent + +The previous blanket exclusion of boost::concurrent_flat_map is superseded, not a measured rejection. Task 11 compares +two development-branch build variants of the same one-store engine: +- **shared-serial:** the current packed keyless index, parallel frozen probes and serial publication. +- **shared-concurrent:** `boost::concurrent_flat_map` over compact `{row ID, cached hash}` handles into the same packed + rows, with heterogeneous exact lookup and parallel publication of preassigned IDs. + +The A/B runs only after traversal and lookup are parallel (Tasks 5–6). With a serial scan the variants cannot be told +apart at full machine. It tests integration and the unchanged acceptance gates; it is not a container search and does +not bypass the recorded Task 0 outcome. Task 0 measured the concurrent variant 15–18% slower on one core on the physics +models; the A/B must include those single-thread cells. -The previous blanket exclusion of boost::concurrent_flat_map is superseded, not a measured rejection. Task 0 first -compares the three ownership/index designs in isolation. If retained by the owner, compare the packed keyless path with -that same bounded Boost variant in Task 11 after the full-library baseline campaign is frozen. This later trial tests -integration and the unchanged acceptance gates; it does not repeat container selection or bypass Task 0. A direct -`concurrent_flat_map` duplicates full keys; investigate stable row handles and heterogeneous lookup -into packed rows. Container synchronization does not protect coefficients, overflow storage, inverted indices or -graph packing. The +A direct `concurrent_flat_map` duplicates full keys and is not permitted. Container synchronization +does not protect coefficients, overflow storage, inverted indices or graph packing. The [Boost concurrency guide](https://www.boost.org/doc/libs/1_89_0/libs/unordered/doc/html/unordered/concurrent.html) -describes visitation and blocking rehash; neither is a drop-in substitute for the current batched query/result contract. +describes visitation and blocking rehash. Bulk visitation reports matched elements only, in no documented order. The +lookup contract is kept by recording `(hash, row)` pairs in the visitor and matching them to queries after the call, +with exact row comparison when several queries share a hash. Never use side-effecting hash/equality, re-enter the +container from a visitor, or assume callback order. Assign IDs in canonical order, reserve all needed storage on the caller, initialize rows before publication, then join -before dependent reads. Only the index's insertion/publication is allowed to become concurrent in this experiment; -other shared structural mutation remains separately governed. Prove exact collision handling, stable external-row -lifetimes, per-query result alignment and clone ownership. Never store query-buffer views as long-lived keys, allocate -IDs by worker arrival order, or add a second persistent index beside the reference implementation. Compare separate -build variants, not two resident stores or a permanent backend selector. Measure lookup/insertion, growth peaks and -end-to-end construction plus the full affected acceptance cells; an insertion microbenchmark alone cannot select it. -Task 11 specifies the bounded trial and its tests. Keep or reject it from evidence, without weakening parity gates. +before dependent reads. Only the index's insertion/publication may become concurrent; other shared structural mutation +remains separately governed. Construction-time key callbacks are not assumed thread-safe and publish serially. Prove +exact collision handling, stable external-row lifetimes, per-query result alignment and clone ownership. Never store +query-buffer views as long-lived keys, allocate IDs by worker arrival order, or add a second persistent index beside +the reference implementation. + +Build both variants from one compile-time option, never two resident indices or a permanent backend selector, and run +the full test suites on both. Decide from the required acceptance cells (five metrics, numerical checks), covering at +least L1, MPI-off full machine, the all-new-terms random profiles and the physics models. Index-phase speedups alone +cannot select a variant. Remove the losing variant and the option before acceptance. Task 11 specifies the trial and its +tests. ### Kernel and phase sequencing @@ -301,7 +342,7 @@ must be O(total emitted records), never O(T*operator_size). Release/move drained next memory-heavy phase. Before parallel decode, the caller checks variable record boundaries, canonical escaped headers (`h.bits == QueryWire::header_bits_for(h.k)`) and position-count prefix sums, sizes arrays once, then workers decode into disjoint spans. Frozen probes retain `find_batch_positions` and cached hashes; the initial reference publication is -serial `set_positions` → `bulk_insert_hashed` → reindex. The bounded Task 11 index trial must preserve these phase and +serial `set_positions` → `bulk_insert_hashed` → reindex. The Task 11 index A/B must preserve these phase and row-ID contracts even if hash publication becomes concurrent. No per-rank or full-store dense monomial reconstruction is allowed. Existing bounded paired-only scoring may reconstruct its one required monomial; do not turn that exception into a batch/store. Task 6 defines the checked preparation sequence. @@ -566,7 +607,7 @@ to shrink a workload or drop a cell. MPI-enabled diagnostic L2a retains the same If D=1, L2a and L2b coincide: deduplicate within the same build mode without removing a required gate. Never deduplicate MPI-off evidence against an MPI-enabled run, even at identical R/T/allocation. -After the Task 0 architecture decision and reconciliation of these conditional production gates, the sequence is: +After the recorded Task 0 outcome and the reconciliation of these production gates, the sequence is: 1. On the baseline only, calibrate each ladder row using its preferred control: Hubbard/Pauli `lower_atol`, random Heisenberg `obs_terms`, and random Schrödinger `num_generators` (or the documented mode-count sweep). Keep model @@ -719,9 +760,9 @@ RED/GREEN evidence for new behavior, plus baseline results for preservation test manufacture RED by removing earlier plumbing. Use kernel-specific worker observations for threading participation. Do not execute past a failed gate or infer approval from this refresh. -The first execution handoff is the separately authorized Task 0 mini-app, ending at the architecture decision. Apply -its staged first pass and 2-hour/min(8 GiB, 25% RAM) limits; account for setup/build separately. These approvals do not -authorize launch. Only after that gate does the baseline pilot travel with Task 1 to the authorized host; neither +The first execution handoff was the separately authorized Task 0 mini-app, now completed with its outcome recorded +above. It ran within its staged first pass, the owner-extended 9,200 s study budget and the min(8 GiB, 25% RAM) cap, +with setup/build accounted separately. The baseline pilot travels with Task 1 to the authorized host; neither experiment must run on the planning host before handoff. Transfer the latest spec, plan and CONTEXT.md, not only the historical published planning commit. The Task 1 portion of the remote entry point defines the subsequent baseline-only phase ending with a proposed campaign/budget for owner approval, not candidate engine work or a completed Task 1 gate. From 79628642faa44b62bd359e3a623633d3880a99a8 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Roberto=20Di=20Remigio=20Eik=C3=A5s?= Date: Sat, 26 Sep 2026 19:53:51 +0000 Subject: [PATCH 07/64] =?UTF-8?q?feat(bench):=20=E2=9C=A8=20add=20rank-loc?= =?UTF-8?q?al=20OpenMP=20parity=20evidence=20tooling?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Task 1 measurement layer for the rank-local OpenMP plan, applied identically to both arms. - benches/conftest.py records operation and outer memory-window exactness independently (all-rank reductions), imports mpi4py only for MPI-enabled extensions, records has_mpi and the extension hash, and adds --runtime-shape/--build-mode preflight while keeping the legacy multi-rank guard. - monoprop_bench_tools.preflight holds the shape/build-mode checks shared with the driver; the report and BMF renderers label or gate each memory metric by its own window's flag. - tools/benchmark-rank-local-openmp.py has exactly three modes: observe (historical pytest nodes in-process, the new bounded driver::pare_functional_construct measurement, and a separate whole-construction worker), validate (untimed numerical products with gzip JSON Lines term files), and compare (frozen-inventory joins, placement checks and the five median gates). - Docs describe the options, exactness labels, the VmHWM shortfall observed on many-core hosts, and the driver. Assisted-by: Pi:claude-opus-5-5 --- AGENTS.md | 4 + benches/LADDER.md | 6 +- benches/README.md | 5 + benches/conftest.py | 105 +- docs/content/docs/benchmarks.mdx | 103 +- .../src/monoprop_bench_tools/bmf.py | 31 +- .../src/monoprop_bench_tools/preflight.py | 221 ++ .../src/monoprop_bench_tools/report.py | 33 +- .../monoprop-bench-tools/tests/test_bmf.py | 82 +- .../tests/test_preflight.py | 180 ++ .../monoprop-bench-tools/tests/test_report.py | 64 + tests/test_rank_local_openmp_benchmark.py | 2394 +++++++++++++++++ tools/benchmark-rank-local-openmp.py | 1915 +++++++++++++ 13 files changed, 5114 insertions(+), 29 deletions(-) create mode 100644 packages/monoprop-bench-tools/src/monoprop_bench_tools/preflight.py create mode 100644 packages/monoprop-bench-tools/tests/test_preflight.py create mode 100644 tests/test_rank_local_openmp_benchmark.py create mode 100755 tools/benchmark-rank-local-openmp.py diff --git a/AGENTS.md b/AGENTS.md index 0ecd12b5..679abdec 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -56,6 +56,10 @@ just build-docs - Python cases live in `tests/cases.py` (`load_problem()` reads `tests/data/*.msgpack`); C++ uses `test_utils::load_case()` from `cpp/tests/TestData.h`. - Pytest's fd capture hides C++ stderr such as `COMMPROF`; rerun with `-s`. +- Benchmark evidence: `pytest benches --runtime-shape=... --build-mode=...` refuses contradictory + runs. `tools/benchmark-rank-local-openmp.py` has exactly three modes, `observe`, `validate` and + `compare` (see `docs/content/docs/benchmarks.mdx`). Launch MPI-off runs directly: `mpi4py` is + never imported when `monoprop.has_mpi` is false. - `uv sync` does not relink `bin/monoprop_unit_tests.x` -- compare mtimes and use the standalone recipe in `docs/content/docs/building.mdx`. - Slow CTest startup in MPI builds is `MPI_Init` fabric probing; see diff --git a/benches/LADDER.md b/benches/LADDER.md index 36701a09..5b660d7a 100644 --- a/benches/LADDER.md +++ b/benches/LADDER.md @@ -55,8 +55,10 @@ export monoprop_NUM_THREADS= ``` For multi-rank runs this is mandatory. The engine otherwise defaults to one partition per rank. -Before comparing results, check `ranks`, `nodes`, `ranks_per_node`, `partitions_env`, and -`monoprop_threads` in the report's Configuration table. +Pass `--runtime-shape=partitions` (and `--build-mode=mpi` or `--build-mode=mpi-off`) to have the +suite refuse a run whose environment, rank count or imported build contradicts that declaration. +Before comparing results, check `ranks`, `nodes`, `ranks_per_node`, `partitions_env`, +`monoprop_threads`, `declared_shape` and `has_mpi` in the run metadata. ## Calibrating model size diff --git a/benches/README.md b/benches/README.md index 852d1ac2..1c19eef0 100644 --- a/benches/README.md +++ b/benches/README.md @@ -27,6 +27,11 @@ Everything reusable is in the `monoprop-bench-tools` package memory instrumentation, the model builders, and the two renderers that turn a run's artifacts into `REPORT.md` and Bencher Metric Format JSON. +`../tools/benchmark-rank-local-openmp.py` drives these benchmarks for the rank-local OpenMP parity +campaign. It runs a node, a separate whole-construction measurement and an untimed numerical +validation in fresh processes, then compares two arms against a frozen inventory. See the +Benchmarks documentation. + Cross-engine comparisons against other propagation libraries live in [`../packages/bench-third-party`](../packages/bench-third-party), a standalone uv project with its own lockfile. diff --git a/benches/conftest.py b/benches/conftest.py index 40ab612c..434492f8 100644 --- a/benches/conftest.py +++ b/benches/conftest.py @@ -25,9 +25,18 @@ so a change to ``_RESULTS`` has to land in both places. Recording is on only when the ``just bench`` recipe exports ``monoprop_BENCH_LABEL`` / ``monoprop_BENCH_RESULTS``. -The ``bench_comm`` fixture yields ``MPI.COMM_WORLD`` when ``mpi4py`` is available -(``None`` otherwise). Operations are barrier-wrapped so the timed cost reflects -the slowest rank, and only rank 0 writes results. +The ``bench_comm`` fixture yields ``MPI.COMM_WORLD`` when the imported extension was built +with MPI and ``mpi4py`` is available (``None`` otherwise). An MPI-off extension never imports +``mpi4py``: importing ``mpi4py.MPI`` initializes MPI, which an MPI-off measurement must not do. +Operations are barrier-wrapped so the timed cost reflects the slowest rank, and only rank 0 +writes results. + +Each memory window records whether it was exact on every rank, independently: +``opmemexact`` for the operation window and ``memhwmexact`` for the outer ``memhwm`` window. +``--runtime-shape`` (``partitions`` or ``openmp``) and ``--build-mode`` (``mpi`` or +``mpi-off``) declare the measured shape; the preflight in +:mod:`monoprop_bench_tools.preflight` checks them against the environment, the rank count and +the imported extension before anything is measured. """ from __future__ import annotations @@ -52,6 +61,7 @@ build_random_propagator, make_random_problem, ) +from monoprop_bench_tools.preflight import PreflightError, declared_shape, import_mpi import monoprop @@ -62,12 +72,13 @@ from monoprop import MajoranaPropagator -try: - from mpi4py import MPI -except (ImportError, OSError, RuntimeError): # pragma: no cover - optional MPI build - # mpi4py may be absent, or (the ABI wheel) present but unable to dlopen libmpi - # on a serial node with no MPI module loaded. - MPI = None +# The build mode of the extension actually imported; ``None`` if it cannot be read. +_HAS_MPI: bool | None = getattr(monoprop, "has_mpi", None) + +# Gated on the build mode before mpi4py is touched: an MPI-off run must not initialize MPI even +# when mpi4py is installed. For an MPI build, mpi4py may still be absent, or (the ABI wheel) +# present but unable to dlopen libmpi on a serial node; the run is then serial. +MPI = import_mpi(has_mpi=bool(_HAS_MPI)) def _rank() -> int: @@ -118,6 +129,11 @@ def _gather_lists(comm: Any, values: list[int]) -> list[list[int]]: return gathered if gathered is not None else [] +def _all_ranks(comm: Any, *, flag: bool) -> bool: + """Return whether ``flag`` holds on every rank. Collective; serial returns ``flag``.""" + return _reduce_min(comm, int(flag)) == 1 + + def _spread(comm: Any, value: int) -> dict[str, int]: """Reduce a per-rank number to ``sum`` (bounds the job) and ``max`` (bounds a node).""" return {"sum": _reduce_sum(comm, value), "max": _reduce_max(comm, value)} @@ -153,6 +169,9 @@ def _spread(comm: Any, value: int) -> dict[str, int]: "opmembase": {}, # node id -> that floor "opbytes": {}, # node id -> {"operator": n, "graph": n} "opmembreak": {}, # node id -> operator memory split (bytes) + # Per-window exactness, each true only when every rank's window was exact. + "opmemexact": {}, # node id -> the operation window (opmem*) + "memhwmexact": {}, # node id -> the outer window (memhwm, memhwm_max) } @@ -177,6 +196,20 @@ def pytest_addoption(parser: pytest.Parser) -> None: for name, kind, default, help_text in _RANDOM_OPTIONS: group.addoption(f"--{name}", type=kind, default=default, help=help_text) + shape = parser.getgroup("monoprop-shape", "monoprop measurement preflight") + shape.addoption( + "--runtime-shape", + choices=("partitions", "openmp"), + default=None, + help="Declared process layout, checked against the environment and rank count.", + ) + shape.addoption( + "--build-mode", + choices=("mpi", "mpi-off"), + default=None, + help="Required build mode, checked against the imported extension's has_mpi.", + ) + models = parser.getgroup("monoprop-models", "monoprop fixed-model overrides") for model, (config_cls, _builder, _steps) in MODELS.items(): for field in fields(config_cls): @@ -188,6 +221,15 @@ def pytest_addoption(parser: pytest.Parser) -> None: ) +def _core_sha256() -> str: + """Return the SHA-256 of the extension module this run imported (see :func:`_core_md5`).""" + try: + path = Path(monoprop._core.__file__) + return hashlib.sha256(path.read_bytes()).hexdigest() + except (AttributeError, OSError, TypeError): + return "unavailable" + + def _core_md5() -> str: """Return the md5 of the extension module this run imported. @@ -219,6 +261,9 @@ def _meta(nodes: int, ranks_per_node: int) -> dict[str, Any]: "hostname": socket.gethostname(), "monoprop_version": monoprop.__version__, "monoprop_core_md5": _core_md5(), + "monoprop_core_sha256": _core_sha256(), + "monoprop_core_path": str(getattr(monoprop._core, "__file__", "unavailable")), + "has_mpi": _HAS_MPI, "monoprop_variant": monoprop.__variant__, "monoprop_compiler_flags": monoprop.__compiler_flags__, "python_version": platform.python_version(), @@ -245,14 +290,37 @@ def _params(config: pytest.Config) -> dict[str, Any]: } -def _require_shape() -> None: - """Fail a multi-rank run that has not declared its partition count. +def _require_shape(config: pytest.Config) -> dict[str, Any] | None: + """Check the declared measurement shape, or the legacy guard when none is declared. - ``resolve_partition_count_`` defaults to ``ranks == 1 ? cores : 1``, so an unset knob - under MPI measures one partition per rank -- a single-threaded run at a plausible wall - time, with nothing in the timing to say so. Every rank raises, so no rank is left in a - collective. Serial runs take the engine default and are unaffected. + With ``--runtime-shape`` the shared preflight validates the declaration against the + environment, the observed rank count and the imported build mode (plus ``--build-mode``), + and returns it for the run metadata. Without one, the legacy guard remains: a multi-rank run + must declare its partition count, because ``resolve_partition_count_`` defaults to + ``ranks == 1 ? cores : 1`` and an unset knob under MPI measures one partition per rank -- a + single-threaded run at a plausible wall time, with nothing in the timing to say so. Every + rank raises, so no rank is left in a collective. Serial runs take the engine default. """ + runtime_shape = config.getoption("--runtime-shape") + build_mode = config.getoption("--build-mode") + expected_has_mpi = None if build_mode is None else build_mode == "mpi" + if expected_has_mpi is not None and _HAS_MPI is not expected_has_mpi: + msg = ( + f"--build-mode={build_mode} requires has_mpi={expected_has_mpi}, but the imported " + f"extension has has_mpi={_HAS_MPI}." + ) + raise pytest.UsageError(msg) + if runtime_shape is not None: + try: + return declared_shape( + runtime_shape, + os.environ, + ranks=_size(), + has_mpi=_HAS_MPI, + expected_has_mpi=expected_has_mpi, + ) + except PreflightError as exc: + raise pytest.UsageError(str(exc)) from exc if _size() > 1 and not os.environ.get("monoprop_PARTITIONS"): # noqa: SIM112 msg = ( "monoprop_PARTITIONS is unset on a run of " @@ -261,6 +329,7 @@ def _require_shape() -> None: "for 8 ranks per 128-core node." ) raise pytest.UsageError(msg) + return None @pytest.hookimpl(trylast=True) @@ -271,9 +340,10 @@ def pytest_configure(config: pytest.Config) -> None: ``trylast`` so the terminal reporter exists before non-root ranks unregister it. """ nodes, ranks_per_node = _nodes() # collective; every rank must call this - _require_shape() + shape = _require_shape(config) if _rank() == 0: _RESULTS["meta"] = _meta(nodes, ranks_per_node) + _RESULTS["meta"]["declared_shape"] = shape _RESULTS["params"] = _params(config) return @@ -418,6 +488,7 @@ def close(self, propagator: Any = None) -> None: _record("opmemdelta", self._key, _spread(self._comm, window.delta_bytes)) _record("opmempeak", self._key, _spread(self._comm, window.peak_bytes)) _record("opmembase", self._key, _spread(self._comm, window.baseline_bytes)) + _record("opmemexact", self._key, _all_ranks(self._comm, flag=window.exact)) _record_placement(self._comm) @@ -487,6 +558,8 @@ def record_memory(request: pytest.FixtureRequest, bench_comm: Any) -> Iterator[N key = request.node.nodeid.split("/")[-1] hwm = _reduce_sum(bench_comm, window.peak_bytes) hwm_max = _reduce_max(bench_comm, window.peak_bytes) + # Its own flag: an exact operation window inside it certifies nothing about this one. + _record("memhwmexact", key, _all_ranks(bench_comm, flag=window.exact)) if hwm: _record("memhwm", key, hwm) if hwm_max: diff --git a/docs/content/docs/benchmarks.mdx b/docs/content/docs/benchmarks.mdx index 06369bf3..3da85067 100644 --- a/docs/content/docs/benchmarks.mdx +++ b/docs/content/docs/benchmarks.mdx @@ -238,11 +238,110 @@ even when they never coexisted. `monoprop_PARTITIONS` is mandatory above one rank — the engine's default is `ranks == 1 ? cores : 1`, which would silently measure one partition per rank — and the declared shape is recorded in `meta`. +`mpi4py` is imported only when the imported extension reports `monoprop.has_mpi`: importing +`mpi4py.MPI` initializes MPI, which an MPI-off run must never do, even with `mpi4py` installed. An +MPI-off build runs one process, launched directly. + +Two options declare what a run measures and fail before anything is measured when the process +contradicts them. `meta` records the declaration, `has_mpi` and the extension's SHA-256: + +- `--runtime-shape=partitions` requires `monoprop_PARTITIONS` and `monoprop_NUM_THREADS`, equal and + positive. `--runtime-shape=openmp` requires `monoprop_PARTITIONS` unset, and a positive + `monoprop_NUM_THREADS`, or with it unset a single positive `OMP_NUM_THREADS`. MPI-off builds must + run one rank. +- `--build-mode=mpi` or `--build-mode=mpi-off` must match the imported extension's `has_mpi`. + +Without `--runtime-shape` the multi-rank `monoprop_PARTITIONS` check above still applies. + ```bash just bench-build-mpi # build once (MPI on) monoprop_PARTITIONS=2 monoprop_NUM_THREADS=2 just bench-mpi r5t2 5 --map-by slot:PE=2 --bind-to core ``` +### Memory exactness + +Each memory window records whether its kernel reset worked on every rank, independently: +`opmemexact` for the operation window (`opmempeak`, `opmembase`, `opmemdelta`) and `memhwmexact` for +the outer window (`memhwm`, `memhwm_max`). One never certifies the other. + +In `REPORT.md`, a cell from a `false` window reads *non-exact* (the peak fell back to the RSS at +exit, a lower bound). A cell with no flag, as in artifacts from before the flags existed, reads +*exactness unknown*. `monoprop-bench-bmf` publishes `peak-memory` only when `memhwmexact` is true +and `operation-memory` only when `opmemexact` is true, and warns on stderr otherwise. Latency and +`terms` are unaffected. + +An exact window is not byte-exact on a many-core host. The kernel keeps `VmHWM` from per-CPU RSS +counters that it reads approximately. When threads on many CPUs fault in memory, the recorded peak +can fall short of a transient that really was resident, by tens of MiB. On a 96-core EC2 host the +shortfall ranged from 2 to 51 MiB, independent of the transient's size. A single thread loses +under 1 MiB, and `VmRSS` itself stays exact. Size memory comparisons so that peaks are well above +this bound. + +### Parity evidence for the rank-local OpenMP refactor + +`tools/benchmark-rank-local-openmp.py` wraps the pytest suite above to collect the evidence the +rank-local OpenMP plan (`docs/superpowers/plans/2026-09-18-rank-local-openmp.md`) requires. It adds no benchmarks of its own, apart from the paring-construction measurement below. +The launcher, CPU binding and environment stay external. The driver checks them against a +campaign inventory and an external placement file, and refuses to run when they disagree. + +It has three modes, each run in a fresh process: + +- `observe` measures one campaign cell and writes `DIR/ARM/CELL/SAMPLE/timed.json`. A historical + node runs in-process through pytest, and its raw artifacts are kept under `SAMPLE/timed/`. Add + `--whole-process` for a separate, untimed run that writes `construction.json`. That window opens + before any input is built and closes after the operation's outputs, graph and callable are all + still alive. +- `validate` rebuilds the cell's workload in an untimed process. It records the energy, the full + gradient where the operation has one, and, for unpared operations, per-rank term files. Each + term file is sorted, gzip-compressed JSON Lines (`.jsonl.gz`, level 1, with a reproducible + header), `{"key": [...], "real": x, "imag": y}`, exported with + `evolved_operator(parameters, 0.0)`, so coefficients are rounded to `1e-12`. Pared operations + record scalars only. For `propagate` without coefficient truncation, validation also checks the + result against a graph replay of the same inputs. +- `compare --campaign FILE --manifest FILE --output FILE` joins baseline and candidate artifacts + against the inventory. It exits `0` when every cell passes, `1` for an unmet gate or incomplete + coverage, and `2` for malformed or incompatible evidence. + +`compare` applies exactly five gates per cell, each a median ratio that must be at most 1.00: + +- runtime; +- operation peak, summed over ranks and per-rank maximum; +- whole-construction peak, summed over ranks and per-rank maximum. + +Both exactness flags must be true. Outer-window peaks are reported as labelled diagnostics and never +gate. `compare` wants five paired observations per arm and cell, or ten after the prescribed +repetition, and says when a first failure needs five more. + +It rejects evidence that cannot be joined: + +- run IDs or measurement files reused anywhere; +- mismatched identities or build modes; +- placement files that do not certify the full team; +- artifacts from another campaign, such as calibration runs. + +Numerically, it merges each arm's rank files into a global map and requires equal key sets, with +coefficients agreeing under `abs(a-b) <= 1e-9 + 1e-7*max(|a|,|b|)`. The Heisenberg identity term, +replicated on every rank, is checked for agreement and counted once. + +`driver::pare_functional_construct[heisenberg|schrodinger]` is the one measurement the driver owns. +It times only `expectation_value_and_gradient_functional(1e-10)` on a graph built beforehand, and +keeps the returned callable alive until both memory windows close. `validate` then evaluates that +callable at the recorded parameters. + +```bash +# Baseline, MPI-off full machine: direct launch, no mpiexec. +monoprop_PARTITIONS=96 monoprop_NUM_THREADS=96 OMP_NUM_THREADS=96 taskset -c 0-95 \ + .venv/bin/python tools/benchmark-rank-local-openmp.py observe --arm baseline \ + --runtime-shape partitions --campaign "$CAMPAIGN" --cell-id "$CELL" --sample-id s01 \ + --placement "$PLACEMENT" --output "$RESULTS" +# Repeat with --whole-process in a new process; run `validate` without --sample-id. +``` + +`tools/rank-local-openmp-workloads.json` holds each profile's complete configurations and +parameter vectors. `tools/rank-local-openmp-campaign.json` is the owner-approved, frozen inventory of +required cells, each with its identity and digests. `compare` treats that file as authoritative, so +evidence cannot pass by regenerating either file. + ### Benchmarking ladder `benches/LADDER.md` holds the development ladder: five groups of configurations, each row's flags @@ -271,8 +370,8 @@ monoprop-bench-bmf benches/results l2b # results as Bencher Metric Format JSO | Measure | Unit | Source | Threshold | |---|---|---|---| | `latency` | nanoseconds | pytest-benchmark mean ± 1σ | Student's t-test, upper 0.99, ≥12 samples | -| `peak-memory` | bytes | the process's peak RSS (`VmHWM`) while the row ran, summed over ranks | +10% | -| `operation-memory` | bytes | what the timed call added above its own floor, summed over ranks | +10% | +| `peak-memory` | bytes | the process's peak RSS (`VmHWM`) while the row ran, summed over ranks; only when `memhwmexact` | +10% | +| `operation-memory` | bytes | what the timed call added above its own floor, summed over ranks; only when `opmemexact` | +10% | | `terms` | count | terms in the evolved operator | any change | The `~GiB` columns below are `peak-memory`. `terms` is deterministic for a fixed seed and size, so diff --git a/packages/monoprop-bench-tools/src/monoprop_bench_tools/bmf.py b/packages/monoprop-bench-tools/src/monoprop_bench_tools/bmf.py index 37dedee6..cdacd39e 100644 --- a/packages/monoprop-bench-tools/src/monoprop_bench_tools/bmf.py +++ b/packages/monoprop-bench-tools/src/monoprop_bench_tools/bmf.py @@ -39,6 +39,12 @@ window opens inside ``setup``, after construction, so it excludes both the build transient and anything a previous row left resident -- the figure to compare across the rows of one plot. +Each memory measure is published only when its own window was exact on every rank: +``peak-memory`` needs ``memhwmexact`` and ``operation-memory`` needs ``opmemexact``. A false +flag means the window fell back to a lower bound; an absent flag (legacy artifact) means the +exactness is unknown. Either way the measure is withheld with a warning on stderr rather than +seeding Bencher's history with an uncertified number, and one window never certifies the other. + ``terms`` (count) Terms in the evolved operator. Deterministic for a fixed seed and problem size, so it is held to an exact match: it is the only check that a timing @@ -145,6 +151,20 @@ def _timings(results_dir: Path, label: str) -> Iterator[tuple[str, Metric]]: yield benchmark_name(bench["fullname"].split("/")[-1]), _latency(bench["stats"]) +def _exact(flags: dict[str, bool], node_id: str, metric: str) -> bool: + """Return whether ``metric`` of ``node_id`` is certified by its own window's flag. + + Warns on stderr when it is not: ``False`` is a known lower-bound fallback, absence is a + legacy artifact whose exactness is unknown and is never assumed to be ``True``. + """ + flag = flags.get(node_id) + if flag is True: + return True + reason = "is not exact" if flag is False else "has unknown exactness" + print(f"bmf: {metric} for {node_id} {reason}; not published", file=sys.stderr) + return False + + def build_bmf(results_dir: Path, label: str) -> Bmf: """Return the BMF JSON object for ``label``'s artifacts in ``results_dir``.""" results = _read_json(results_dir / f"{label}.json") @@ -156,15 +176,20 @@ def measure(benchmark: str, name: str, value: float) -> None: for benchmark, latency in _timings(results_dir, label): bmf.setdefault(benchmark, {})["latency"] = latency - # ``memhwm`` is the kernel's exact peak RSS per operation, summed over ranks under MPI. + # ``memhwm`` is the kernel's peak RSS per operation, summed over ranks under MPI. Its + # window spans setup, so only its own ``memhwmexact`` flag can certify it. + outer_exact = results.get("memhwmexact", {}) for benchmark, peak in results.get("memhwm", {}).items(): - measure(benchmark_name(benchmark), "peak-memory", peak) + if _exact(outer_exact, benchmark, "peak-memory"): + measure(benchmark_name(benchmark), "peak-memory", peak) # ``opmemdelta`` is spread over the ranks rather than reduced to one number: ``sum`` # matches what ``peak-memory`` uploads, and ``max`` is the per-node bound that # ``memhwm_max`` is for the footprint. Only rows that ran a timed call have one. + op_exact = results.get("opmemexact", {}) for benchmark, delta in results.get("opmemdelta", {}).items(): - measure(benchmark_name(benchmark), "operation-memory", delta["sum"]) + if _exact(op_exact, benchmark, "operation-memory"): + measure(benchmark_name(benchmark), "operation-memory", delta["sum"]) # A node-id key names the benchmark that built the operator, so its count belongs on # that benchmark, beside the latency and peak memory of the same call. Where a shared diff --git a/packages/monoprop-bench-tools/src/monoprop_bench_tools/preflight.py b/packages/monoprop-bench-tools/src/monoprop_bench_tools/preflight.py new file mode 100644 index 00000000..5a418e6a --- /dev/null +++ b/packages/monoprop-bench-tools/src/monoprop_bench_tools/preflight.py @@ -0,0 +1,221 @@ +# Copyright 2026 Algorithmiq +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"""Runtime-shape and build-mode preflight shared by the benchmark suite and the parity driver. + +A measurement declares how its process is laid out -- ``partitions`` (the legacy runtime, which +reads ``monoprop_PARTITIONS``) or ``openmp`` (one store per rank, sized by +``monoprop_NUM_THREADS`` or, when that is unset, the OpenMP runtime default). The preflight checks +that declaration against the launch environment, the observed rank count and the build mode of +the extension actually imported, so that neither an arm label nor the absence of ``mpiexec`` can +stand in for evidence. + +This is measurement policy, not library configuration: the library parses its own settings. +""" + +from __future__ import annotations + +from typing import TYPE_CHECKING, Any + +if TYPE_CHECKING: + from collections.abc import Mapping + +#: The two process layouts a measurement may declare. +RUNTIME_SHAPES = ("partitions", "openmp") + +#: The runtime shape each comparison arm must declare. +ARM_SHAPES = {"baseline": "partitions", "candidate": "openmp"} + +# Launch settings that change what a measurement runs; everything else in the environment is +# irrelevant to the comparison and is kept out of recorded evidence. +_SETTING_PREFIXES = ("monoprop_", "OMP_", "GOMP_", "KMP_", "MALLOC_") +# Output routing only; ``monoprop_BENCH_ALLOW_BIG_GRAPH`` stays because it changes what runs. +_NOT_SETTINGS = ("monoprop_BENCH_LABEL", "monoprop_BENCH_RESULTS") + + +class PreflightError(ValueError): + """A declared measurement shape or build mode contradicts the observed process.""" + + +def _positive_decimal(name: str, value: str) -> int: + """Return ``value`` as a positive decimal integer, rejecting signs, spaces and lists.""" + if not value.isascii() or not value.isdigit() or int(value) < 1: + msg = f"{name}={value!r} is not a positive decimal integer." + raise PreflightError(msg) + return int(value) + + +def settings_snapshot(env: Mapping[str, str]) -> dict[str, str]: + """Return the library, OpenMP and allocator settings present in ``env``.""" + return { + key: value + for key, value in sorted(env.items()) + if key.startswith(_SETTING_PREFIXES) and key not in _NOT_SETTINGS + } + + +def require_arm_shape(arm: str, runtime_shape: str) -> None: + """Reject an arm label that does not match its runtime shape. + + Raises: + PreflightError: If ``arm`` is unknown or declares the other arm's shape. + """ + expected = ARM_SHAPES.get(arm) + if expected is None: + msg = f"Unknown arm {arm!r}; expected one of {sorted(ARM_SHAPES)}." + raise PreflightError(msg) + if runtime_shape != expected: + msg = ( + f"The {arm} arm must declare runtime shape {expected!r}, not " + f"{runtime_shape!r}; relabelling an arm cannot change what it measures." + ) + raise PreflightError(msg) + + +def _partition_threads(env: Mapping[str, str]) -> tuple[int, str]: + """Return the legacy runtime's declared per-rank thread count.""" + partitions = env.get("monoprop_PARTITIONS") + threads = env.get("monoprop_NUM_THREADS") + if partitions is None or threads is None: + msg = ( + "Runtime shape 'partitions' needs both monoprop_PARTITIONS=T and " + "monoprop_NUM_THREADS=T; the legacy engine otherwise picks its own count." + ) + raise PreflightError(msg) + count = _positive_decimal("monoprop_PARTITIONS", partitions) + if _positive_decimal("monoprop_NUM_THREADS", threads) != count: + msg = ( + f"Runtime shape 'partitions' needs monoprop_PARTITIONS ({partitions}) equal to " + f"monoprop_NUM_THREADS ({threads})." + ) + raise PreflightError(msg) + return count, "monoprop_PARTITIONS" + + +def _openmp_threads(env: Mapping[str, str]) -> tuple[int, str]: + """Return the one-store runtime's declared per-rank team budget.""" + if "monoprop_PARTITIONS" in env: + msg = ( + "Runtime shape 'openmp' has no partitions: unset monoprop_PARTITIONS rather than " + "leave it in the measurement environment." + ) + raise PreflightError(msg) + configured = env.get("monoprop_NUM_THREADS") + if configured is not None: + return _positive_decimal( + "monoprop_NUM_THREADS", configured + ), "monoprop_NUM_THREADS" + fallback = env.get("OMP_NUM_THREADS", "") + if not fallback.isascii() or not fallback.isdigit() or int(fallback) < 1: + msg = ( + "Runtime shape 'openmp' with monoprop_NUM_THREADS unset cannot establish the " + f"declared team size from OMP_NUM_THREADS={fallback!r}; set either to one " + "positive integer." + ) + raise PreflightError(msg) + return int(fallback), "OMP_NUM_THREADS" + + +def declared_shape( + runtime_shape: str, + env: Mapping[str, str], + *, + ranks: int, + has_mpi: bool | None, + expected_has_mpi: bool | None = None, + expected_ranks: int | None = None, + expected_threads: int | None = None, +) -> dict[str, Any]: + """Validate a declared runtime shape and return it with its evidence. + + Args: + runtime_shape: ``"partitions"`` or ``"openmp"``. + env: The launch environment of this process. + ranks: The observed number of ranks in the measurement communicator. + has_mpi: ``monoprop.has_mpi`` of the extension actually imported; ``None`` when it could + not be read. + expected_has_mpi: The build mode the measurement requires, if declared. + expected_ranks: The rank count the measurement requires, if declared. + expected_threads: The per-rank thread count the measurement requires, if declared. + + Returns: + ``runtime_shape``, ``ranks``, ``threads``, ``threads_source``, ``has_mpi`` and the + relevant ``settings`` of ``env``. + + Raises: + PreflightError: On any missing, malformed or contradictory declaration. + """ + if runtime_shape not in RUNTIME_SHAPES: + msg = f"Unknown runtime shape {runtime_shape!r}; expected one of {RUNTIME_SHAPES}." + raise PreflightError(msg) + if has_mpi is None: + msg = "The imported extension's has_mpi could not be read; the build mode is unproven." + raise PreflightError(msg) + if expected_has_mpi is not None and has_mpi != expected_has_mpi: + msg = ( + f"The imported extension has has_mpi={has_mpi}, but the measurement requires " + f"has_mpi={expected_has_mpi}." + ) + raise PreflightError(msg) + if ranks < 1: + msg = f"Observed {ranks} ranks; a measurement needs at least one." + raise PreflightError(msg) + if not has_mpi and ranks != 1: + msg = f"An MPI-off binary cannot run {ranks} ranks; MPI-off cells have R=1." + raise PreflightError(msg) + if expected_ranks is not None and ranks != expected_ranks: + msg = f"Observed {ranks} ranks, but the measurement declares {expected_ranks} ranks." + raise PreflightError(msg) + + reader = _partition_threads if runtime_shape == "partitions" else _openmp_threads + try: + threads, source = reader(env) + except PreflightError as exc: + if str(exc).startswith("Runtime shape"): + raise + msg = f"Runtime shape '{runtime_shape}': {exc}" + raise PreflightError(msg) from exc + if expected_threads is not None and threads != expected_threads: + msg = ( + f"The environment declares {threads} threads per rank, but the measurement " + f"requires {expected_threads} threads." + ) + raise PreflightError(msg) + return { + "runtime_shape": runtime_shape, + "ranks": ranks, + "threads": threads, + "threads_source": source, + "has_mpi": has_mpi, + "settings": settings_snapshot(env), + } + + +def import_mpi(*, has_mpi: bool) -> Any: + """Return ``mpi4py.MPI`` for an MPI build, without touching mpi4py otherwise. + + Importing ``mpi4py.MPI`` initializes MPI, so an MPI-off run must never reach it, even when + the package is installed. An MPI build whose mpi4py is absent or cannot load libmpi runs + serially, as before. + + Args: + has_mpi: ``monoprop.has_mpi`` of the imported extension. + """ + if not has_mpi: + return None + try: + from mpi4py import MPI # noqa: PLC0415 - deliberately gated on the build mode + except (ImportError, OSError, RuntimeError): + return None + return MPI diff --git a/packages/monoprop-bench-tools/src/monoprop_bench_tools/report.py b/packages/monoprop-bench-tools/src/monoprop_bench_tools/report.py index df683664..da9c2584 100644 --- a/packages/monoprop-bench-tools/src/monoprop_bench_tools/report.py +++ b/packages/monoprop-bench-tools/src/monoprop_bench_tools/report.py @@ -115,6 +115,18 @@ def _delta_sum(spread: dict[str, int] | None) -> int | None: return None if spread is None else spread["sum"] +def _fmt_exact_mem(num_bytes: int | None, *, flag: bool | None) -> str: + """Format a memory cell labelled by the exactness flag of the window it came from. + + ``False`` marks a lower-bound fallback and ``None`` (flag absent, as in legacy artifacts) an + unknown exactness; neither is presented as an exact peak. Empty cells stay unlabelled. + """ + text = _fmt_mem(num_bytes) + if num_bytes is None or flag is True: + return text + return f"{text} (non-exact)" if flag is False else f"{text} (exactness unknown)" + + def _fmt_config(value: object) -> str: """Format a config field value compactly (floats via ``g``, else ``str``).""" return format(value, "g") if isinstance(value, float) else str(value) @@ -292,6 +304,8 @@ def sec(name: str) -> dict[str, dict]: sec("memhwm_max"), sec("opmemdelta"), ) + # Each window has its own flag; the outer peaks never borrow the operation's, or vice versa. + outer_exact, op_exact = sec("memhwmexact"), sec("opmemexact") all_ops = sorted( {op for table in timings.values() for op in table} @@ -330,7 +344,10 @@ def ops_section(name: str, picture: str) -> list[str]: "Operation", ops, labels, - lambda lbl, op: _fmt_mem(memory.get(lbl, {}).get(op)), + lambda lbl, op: _fmt_exact_mem( + memory.get(lbl, {}).get(op), + flag=outer_exact.get(lbl, {}).get(op), + ), level=3, ), *_section( @@ -339,7 +356,10 @@ def ops_section(name: str, picture: str) -> list[str]: "Operation", ops, labels, - lambda lbl, op: _fmt_mem(memory_max.get(lbl, {}).get(op)), + lambda lbl, op: _fmt_exact_mem( + memory_max.get(lbl, {}).get(op), + flag=outer_exact.get(lbl, {}).get(op), + ), level=3, ), *_section( @@ -348,7 +368,10 @@ def ops_section(name: str, picture: str) -> list[str]: "Operation", ops, labels, - lambda lbl, op: _fmt_mem(_delta_sum(op_delta.get(lbl, {}).get(op))), + lambda lbl, op: _fmt_exact_mem( + _delta_sum(op_delta.get(lbl, {}).get(op)), + flag=op_exact.get(lbl, {}).get(op), + ), level=3, ), ] @@ -363,7 +386,9 @@ def ops_section(name: str, picture: str) -> list[str]: "upper bound on the job total); the max figure is the single worst rank. The " "delta figure is what the timed call added above its own floor, which is the one " "to compare between operations -- the peaks include whatever an earlier " - "operation left resident.", + "operation left resident. A memory cell marked non-exact fell back to a lower " + "bound because the kernel window could not be reset; one marked exactness " + "unknown comes from an artifact that predates the per-window flags.", "", *_config_table(labels, results), *_section( diff --git a/packages/monoprop-bench-tools/tests/test_bmf.py b/packages/monoprop-bench-tools/tests/test_bmf.py index e045e425..6eabfb8c 100644 --- a/packages/monoprop-bench-tools/tests/test_bmf.py +++ b/packages/monoprop-bench-tools/tests/test_bmf.py @@ -74,7 +74,7 @@ def test_latency_lower_bound_floors_at_zero(tmp_path: Path) -> None: def test_memory_joins_the_benchmark_of_the_same_operation(tmp_path: Path) -> None: - _write(tmp_path, memhwm={_ENERGY: 1024}) + _write(tmp_path, memhwm={_ENERGY: 1024}, memhwmexact={_ENERGY: True}) # conftest keys memhwm by the same node id pytest-benchmark reports, so both # measures must land on one benchmark rather than splitting into two. @@ -88,7 +88,9 @@ def test_operation_memory_reads_the_rank_sum_of_the_spread(tmp_path: Path) -> No _write( tmp_path, memhwm={_ENERGY: 4096}, + memhwmexact={_ENERGY: True}, opmemdelta={_ENERGY: {"sum": 1024, "max": 512}}, + opmemexact={_ENERGY: True}, ) # The footprint and the operation's own cost are separate measures on one benchmark: @@ -101,13 +103,89 @@ def test_operation_memory_reads_the_rank_sum_of_the_spread(tmp_path: Path) -> No def test_operation_memory_is_absent_where_no_window_was_opened(tmp_path: Path) -> None: - _write(tmp_path, memhwm={_ENERGY: 4096}) + _write(tmp_path, memhwm={_ENERGY: 4096}, memhwmexact={_ENERGY: True}) # ``record_memory`` is autouse, ``op_memory`` is not, so a row can have a peak and no # delta. Uploading a zero there would read as an operation that allocated nothing. assert "operation-memory" not in bmf.build_bmf(tmp_path, "ci")[_ENERGY_NAME] +# Each window's metric is certified only by its own exactness flag: ``memhwmexact`` for the +# outer ``memhwm`` peak, ``opmemexact`` for the operation window's ``opmemdelta``. +_LATENCY = {"value": 0.5e9, "lower_value": 0.49e9, "upper_value": 0.51e9} + + +@pytest.mark.parametrize( + ("outer_flag", "op_flag", "expected"), + [ + (True, True, {"peak-memory", "operation-memory"}), + (True, False, {"peak-memory"}), + (False, True, {"operation-memory"}), + (False, False, set()), + (None, True, {"operation-memory"}), + (True, None, {"peak-memory"}), + (None, None, set()), + ], +) +def test_each_memory_metric_is_gated_on_its_own_window( + tmp_path: Path, + capsys: pytest.CaptureFixture[str], + outer_flag: bool | None, # noqa: FBT001 - parametrized + op_flag: bool | None, # noqa: FBT001 + expected: set[str], +) -> None: + sections: dict[str, object] = { + "memhwm": {_ENERGY: 4096}, + "opmemdelta": {_ENERGY: {"sum": 1024, "max": 512}}, + } + if outer_flag is not None: + sections["memhwmexact"] = {_ENERGY: outer_flag} + if op_flag is not None: + sections["opmemexact"] = {_ENERGY: op_flag} + _write(tmp_path, **sections) + + result = bmf.build_bmf(tmp_path, "ci")[_ENERGY_NAME] + + assert set(result) - {"latency"} == expected + # Timing is never withheld because a memory window was not exact. + assert result["latency"] == _LATENCY + warnings = capsys.readouterr().err + for flag, metric in ((outer_flag, "peak-memory"), (op_flag, "operation-memory")): + if flag is False: + assert f"{metric} for {_ENERGY} is not exact" in warnings + elif flag is None: + assert f"{metric} for {_ENERGY} has unknown exactness" in warnings + + +def test_operation_exactness_never_certifies_the_outer_peak(tmp_path: Path) -> None: + # An exact operation window says nothing about the setup-spanning outer window. + _write( + tmp_path, + memhwm={_ENERGY: 4096}, + opmemdelta={_ENERGY: {"sum": 1024, "max": 512}}, + opmemexact={_ENERGY: True}, + memhwmexact={_ENERGY: False}, + ) + + assert "peak-memory" not in bmf.build_bmf(tmp_path, "ci")[_ENERGY_NAME] + + +def test_exactness_is_per_node(tmp_path: Path) -> None: + other = "bench_random.py::test_random_gradient[heisenberg]" + _write( + tmp_path, + memhwm={_ENERGY: 4096, other: 8192}, + memhwmexact={_ENERGY: True, other: False}, + opsize={_ENERGY: {"terms": 34}}, + ) + result = bmf.build_bmf(tmp_path, "ci") + + assert result[_ENERGY_NAME]["peak-memory"] == {"value": 4096.0} + assert "peak-memory" not in result.get("random/gradient[heisenberg]", {}) + # Term counts are not memory evidence and survive regardless of any flag. + assert result[_ENERGY_NAME]["terms"] == {"value": 34.0} + + def test_operator_metrics_are_grouped_per_picture_and_model(tmp_path: Path) -> None: _write( tmp_path, diff --git a/packages/monoprop-bench-tools/tests/test_preflight.py b/packages/monoprop-bench-tools/tests/test_preflight.py new file mode 100644 index 00000000..b608661f --- /dev/null +++ b/packages/monoprop-bench-tools/tests/test_preflight.py @@ -0,0 +1,180 @@ +# Copyright 2026 Algorithmiq +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"""Unit tests for the shared runtime-shape and build-mode preflight.""" + +from __future__ import annotations + +import sys +import types + +import pytest +from monoprop_bench_tools import preflight +from monoprop_bench_tools.preflight import PreflightError + +_BASELINE = {"monoprop_PARTITIONS": "4", "monoprop_NUM_THREADS": "4"} + + +def _declare(shape: str, env: dict[str, str], **kwargs: object) -> dict: + kwargs.setdefault("ranks", 1) + kwargs.setdefault("has_mpi", False) + return preflight.declared_shape(shape, env, **kwargs) # type: ignore[arg-type] + + +def test_baseline_declares_both_legacy_thread_variables() -> None: + shape = _declare("partitions", {**_BASELINE, "OMP_NUM_THREADS": "4"}) + + assert shape["runtime_shape"] == "partitions" + assert (shape["ranks"], shape["threads"]) == (1, 4) + assert shape["settings"] == {**_BASELINE, "OMP_NUM_THREADS": "4"} + + +@pytest.mark.parametrize( + "env", + [ + {}, + {"monoprop_NUM_THREADS": "4"}, + {"monoprop_PARTITIONS": "4"}, + {"monoprop_PARTITIONS": "4", "monoprop_NUM_THREADS": "2"}, + {"monoprop_PARTITIONS": "auto", "monoprop_NUM_THREADS": "4"}, + ], +) +def test_baseline_declaration_must_be_complete_and_consistent(env: dict) -> None: + with pytest.raises(PreflightError, match="partitions"): + _declare("partitions", env) + + +def test_candidate_explicit_thread_budget() -> None: + shape = _declare("openmp", {"monoprop_NUM_THREADS": "8", "OMP_NUM_THREADS": "8"}) + + assert (shape["threads"], shape["threads_source"]) == (8, "monoprop_NUM_THREADS") + + +def test_candidate_openmp_fallback_resolves_the_declared_budget() -> None: + shape = _declare("openmp", {"OMP_NUM_THREADS": "6"}) + + assert (shape["threads"], shape["threads_source"]) == (6, "OMP_NUM_THREADS") + + +@pytest.mark.parametrize("value", ["", "0", "-1", "abc", "2,3", " 3", "+3", "3 "]) +def test_candidate_invalid_thread_override_is_rejected(value: str) -> None: + with pytest.raises(PreflightError, match="monoprop_NUM_THREADS"): + _declare("openmp", {"monoprop_NUM_THREADS": value, "OMP_NUM_THREADS": "4"}) + + +def test_candidate_without_any_resolvable_budget_is_rejected() -> None: + # A list-valued OMP_NUM_THREADS cannot declare one team size for the preflight. + with pytest.raises(PreflightError, match="cannot establish"): + _declare("openmp", {"OMP_NUM_THREADS": "4,2"}) + + +def test_leftover_partition_setting_in_candidate_environment_is_rejected() -> None: + env = {"monoprop_NUM_THREADS": "4", "monoprop_PARTITIONS": "4"} + with pytest.raises(PreflightError, match="monoprop_PARTITIONS"): + _declare("openmp", env) + + +def test_unknown_runtime_shape_is_rejected() -> None: + with pytest.raises(PreflightError, match="runtime shape"): + _declare("hybrid", _BASELINE) + + +def test_wrong_observed_ranks_or_threads_is_rejected() -> None: + with pytest.raises(PreflightError, match="ranks"): + _declare("partitions", _BASELINE, ranks=2, has_mpi=True, expected_ranks=1) + with pytest.raises(PreflightError, match="threads"): + _declare("partitions", _BASELINE, expected_threads=8) + + +def test_nonpositive_ranks_are_rejected() -> None: + with pytest.raises(PreflightError, match="ranks"): + _declare("partitions", _BASELINE, ranks=0) + + +def test_mpi_off_declaration_with_several_ranks_is_rejected() -> None: + with pytest.raises(PreflightError, match="MPI-off"): + _declare("partitions", _BASELINE, ranks=2, has_mpi=False) + + +def test_requested_build_mode_must_match_the_imported_binary() -> None: + with pytest.raises(PreflightError, match="has_mpi"): + _declare("partitions", _BASELINE, has_mpi=True, expected_has_mpi=False) + with pytest.raises(PreflightError, match="has_mpi"): + _declare("partitions", _BASELINE, has_mpi=False, expected_has_mpi=True) + + +def test_missing_build_mode_evidence_is_rejected() -> None: + with pytest.raises(PreflightError, match="has_mpi"): + _declare("partitions", _BASELINE, has_mpi=None) + + +@pytest.mark.parametrize( + ("arm", "shape"), [("baseline", "openmp"), ("candidate", "partitions")] +) +def test_a_relabelled_arm_cannot_bypass_the_shape(arm: str, shape: str) -> None: + with pytest.raises(PreflightError, match="arm"): + preflight.require_arm_shape(arm, shape) + + +def test_matching_arm_and_shape_pass() -> None: + preflight.require_arm_shape("baseline", "partitions") + preflight.require_arm_shape("candidate", "openmp") + + +def test_mpi_off_never_imports_mpi4py(monkeypatch: pytest.MonkeyPatch) -> None: + # A trap standing in for an installed mpi4py: importing it at all would initialize MPI. + trap = types.ModuleType("mpi4py") + + def _explode(name: str) -> None: + msg = f"attempted to import mpi4py.{name} in an MPI-off run" + raise AssertionError(msg) + + trap.__getattr__ = _explode # type: ignore[method-assign] + monkeypatch.setitem(sys.modules, "mpi4py", trap) + monkeypatch.setitem(sys.modules, "mpi4py.MPI", None) + + assert preflight.import_mpi(has_mpi=False) is None + + +def test_mpi_build_without_mpi4py_stays_serial(monkeypatch: pytest.MonkeyPatch) -> None: + monkeypatch.setitem(sys.modules, "mpi4py", None) + + assert preflight.import_mpi(has_mpi=True) is None + + +def test_mpi_build_imports_mpi4py(monkeypatch: pytest.MonkeyPatch) -> None: + fake_mpi = types.ModuleType("mpi4py.MPI") + package = types.ModuleType("mpi4py") + package.MPI = fake_mpi # type: ignore[attr-defined] + monkeypatch.setitem(sys.modules, "mpi4py", package) + monkeypatch.setitem(sys.modules, "mpi4py.MPI", fake_mpi) + + assert preflight.import_mpi(has_mpi=True) is fake_mpi + + +def test_settings_snapshot_keeps_only_runtime_configuration() -> None: + env = { + "monoprop_NUM_THREADS": "4", + "OMP_PLACES": "cores", + "HOME": "/root", + "monoprop_BENCH_LABEL": "x", + "monoprop_BENCH_RESULTS": "results/r", + "monoprop_BENCH_ALLOW_BIG_GRAPH": "1", + } + + assert preflight.settings_snapshot(env) == { + "OMP_PLACES": "cores", + "monoprop_BENCH_ALLOW_BIG_GRAPH": "1", + "monoprop_NUM_THREADS": "4", + } diff --git a/packages/monoprop-bench-tools/tests/test_report.py b/packages/monoprop-bench-tools/tests/test_report.py index 2fd86b32..a97bb05d 100644 --- a/packages/monoprop-bench-tools/tests/test_report.py +++ b/packages/monoprop-bench-tools/tests/test_report.py @@ -233,6 +233,70 @@ def test_build_report_includes_memory(tmp_path: Path) -> None: assert "40.00 MiB" in md +_HEIS = "bench_random.py::test_random_energy[heisenberg]" + + +def _memory_rows(md: str, title: str) -> str: + """Return one memory table of the Heisenberg section.""" + start = md.index(title) + end = md.index("###", start + len(title)) + return md[start:end] + + +def test_report_labels_each_memory_metric_by_its_own_exactness(tmp_path: Path) -> None: + _write_timings(tmp_path) + _write_results( + tmp_path, + memhwm={_HEIS: 52428800}, + memhwm_max={_HEIS: 41943040}, + memhwmexact={_HEIS: False}, + opmemdelta={_HEIS: {"sum": 10485760, "max": 10485760}}, + opmemexact={_HEIS: True}, + ) + md = _collapse(report.build_report(tmp_path)) + + summed = _memory_rows(md, "Memory (peak RSS, summed across ranks)") + worst = _memory_rows(md, "Memory (peak RSS, max across ranks)") + delta = _memory_rows(md, "Memory (operation delta, summed across ranks)") + assert "50.00 MiB (non-exact)" in summed + assert "40.00 MiB (non-exact)" in worst + # The exact operation window does not upgrade the outer peaks, and vice versa. + assert "10.00 MiB |" in delta + assert "(non-exact)" not in delta + assert "unknown" not in delta + + +def test_report_marks_absent_exactness_as_unknown(tmp_path: Path) -> None: + _write_timings(tmp_path) + _write_results( + tmp_path, + memhwm={_HEIS: 52428800}, + opmemdelta={_HEIS: {"sum": 10485760, "max": 10485760}}, + memhwmexact={_HEIS: True}, + ) + md = _collapse(report.build_report(tmp_path)) + + summed = _memory_rows(md, "Memory (peak RSS, summed across ranks)") + delta = _memory_rows(md, "Memory (operation delta, summed across ranks)") + assert "50.00 MiB |" in summed + assert "10.00 MiB (exactness unknown)" in delta + + +def test_report_keeps_times_when_memory_is_not_exact(tmp_path: Path) -> None: + _write_timings(tmp_path) + _write_results( + tmp_path, + memhwm={_HEIS: 52428800}, + memhwmexact={_HEIS: False}, + opmemexact={_HEIS: False}, + opsize={"heisenberg": {"terms": 132220}}, + ) + md = _collapse(report.build_report(tmp_path)) + + assert "1.000 ms" in md + assert "| Heisenberg | 132,220 |" in md + + def test_build_report_sorts_labels_numerically(tmp_path: Path) -> None: for label in ("np1", "np2", "np10"): _write_timings(tmp_path, label) diff --git a/tests/test_rank_local_openmp_benchmark.py b/tests/test_rank_local_openmp_benchmark.py new file mode 100644 index 00000000..fd8f1419 --- /dev/null +++ b/tests/test_rank_local_openmp_benchmark.py @@ -0,0 +1,2394 @@ +# Copyright 2026 Algorithmiq +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"""Tests for the rank-local OpenMP parity evidence: benchmark overlay and parity driver. + +The benchmark-suite tests launch ``pytest benches`` at tiny sizes in fresh processes; they check +the evidence fields the suite writes, not performance. Everything else exercises +``tools/benchmark-rank-local-openmp.py`` on synthetic artifacts that follow the real schema. None +of it is runtime evidence or a parity claim. +""" + +from __future__ import annotations + +import dataclasses +import gzip +import hashlib +import importlib.util +import json +import os +import shutil +import subprocess +import sys +from pathlib import Path + +import pytest + +import monoprop + +_ROOT = Path(__file__).resolve().parents[1] +_SCRIPT = _ROOT / "tools/benchmark-rank-local-openmp.py" + +# Tiny sizes for every benchmark family; the fixed models are shrunk through their own overrides. +_TINY = [ + "--num-generators=8", + "--num-modes=8", + "--cutoff=6", + "--obs-terms=16", + "--bench-rounds=1", + "--hubbard-num-sites=4", + "--hubbard-observable-site=1", + "--hubbard-trotter-steps=2", + "--pauli-num-layers=1", + "--pauli-cutoff=2", + "--pauli-lower-atol=0.1", +] +_FAMILIES = ("build_graph", "propagate", "energy", "gradient") + +# These tests launch their own processes (pytest, the driver, mpiexec). Inside a multi-rank MPI test +# job every rank would repeat those launches nested in the parent job, so they run outside one. +_IN_MPI_JOB = ( + int(os.environ.get("OMPI_COMM_WORLD_SIZE", os.environ.get("PMI_SIZE", "1"))) > 1 +) +_launches = pytest.mark.skipif( + _IN_MPI_JOB, reason="launches its own processes; run outside a multi-rank MPI job" +) + + +def _bench_env(**extra: str) -> dict[str, str]: + """Return a clean launch environment for a benchmark subprocess.""" + # Launcher variables are dropped so a child never joins an enclosing MPI job. + dropped = ("monoprop_", "OMP_", "PYTEST_", "OMPI_", "PMIX_", "PRTE_", "PMI_") + env = { + key: value for key, value in os.environ.items() if not key.startswith(dropped) + } + env.update(extra) + return env + + +def _run_benches( + *args: str, env: dict[str, str] | None = None +) -> subprocess.CompletedProcess[str]: + """Run ``pytest benches`` in a fresh process from the repository root.""" + return subprocess.run( # noqa: S603 - trusted: this interpreter, repository paths + [sys.executable, "-m", "pytest", "benches", "-p", "no:cacheprovider", *args], + cwd=_ROOT, + env=env if env is not None else _bench_env(), + capture_output=True, + text=True, + timeout=600, + check=False, + ) + + +def _needs_bench_tools() -> None: + pytest.importorskip("monoprop_bench_tools") + pytest.importorskip("pytest_benchmark") + + +@_launches +def test_suite_records_independent_exactness_for_all_four_families( + tmp_path: Path, +) -> None: + _needs_bench_tools() + results = tmp_path / "results" + results.mkdir() + env = _bench_env( + monoprop_BENCH_LABEL="tiny", + monoprop_BENCH_RESULTS=str(results), + monoprop_PARTITIONS="1", + monoprop_NUM_THREADS="1", + ) + proc = _run_benches( + *_TINY, + f"--benchmark-json={results / 'time-tiny.json'}", + env=env, + ) + assert proc.returncode == 0, proc.stdout[-4000:] + proc.stderr[-4000:] + data = json.loads((results / "tiny.json").read_text()) + + nodes = [ + f"bench_random.py::test_random_{family}[{picture}]" + for family in _FAMILIES + for picture in ("heisenberg", "schrodinger") + ] + [ + f"bench_models.py::test_model_{family}[{model}]" + for family in _FAMILIES + for model in ("hubbard", "pauli") + ] + for node in nodes: + for section in ("opmempeak", "opmembase", "opmemdelta"): + assert set(data[section][node]) == {"sum", "max"}, (section, node) + assert data["opmemexact"][node] is True, node + assert data["memhwmexact"][node] is True, node + assert data["memhwm"][node] >= data["opmempeak"][node]["max"] > 0, node + assert data["meta"]["has_mpi"] is monoprop.has_mpi + assert len(data["meta"]["monoprop_core_sha256"]) == 64 + + +def _mpi4py_trap(tmp_path: Path) -> tuple[Path, Path]: + """Return a PYTHONPATH entry whose ``mpi4py`` records any import, and that marker file.""" + marker = tmp_path / "mpi4py-imported" + package = tmp_path / "trap" / "mpi4py" + package.mkdir(parents=True) + (package / "__init__.py").write_text( + f"from pathlib import Path\nPath({str(marker)!r}).write_text('imported')\n" + "raise RuntimeError('mpi4py must not be imported in an MPI-off run')\n" + ) + return package.parent, marker + + +@_launches +@pytest.mark.skipif(monoprop.has_mpi, reason="needs an MPI-off extension") +def test_mpi_off_suite_never_imports_mpi4py(tmp_path: Path) -> None: + _needs_bench_tools() + trap, marker = _mpi4py_trap(tmp_path) + env = _bench_env(PYTHONPATH=os.pathsep.join([str(trap), *sys.path])) + proc = _run_benches("--collect-only", "-q", *_TINY, env=env) + + assert proc.returncode == 0, proc.stdout[-4000:] + proc.stderr[-4000:] + assert not marker.exists() + + +@_launches +@pytest.mark.skipif(monoprop.has_mpi, reason="needs an MPI-off extension") +@pytest.mark.parametrize( + ("args", "env", "message"), + [ + (["--runtime-shape=partitions"], {"monoprop_NUM_THREADS": "2"}, "partitions"), + ( + ["--runtime-shape=openmp"], + {"monoprop_NUM_THREADS": "2", "monoprop_PARTITIONS": "2"}, + "monoprop_PARTITIONS", + ), + (["--runtime-shape=openmp"], {"monoprop_NUM_THREADS": "0"}, "positive"), + ( + ["--runtime-shape=partitions", "--build-mode=mpi"], + {"monoprop_PARTITIONS": "2", "monoprop_NUM_THREADS": "2"}, + "has_mpi", + ), + (["--build-mode=mpi"], {}, "has_mpi"), + ], +) +def test_suite_rejects_contradictory_declarations( + args: list[str], env: dict[str, str], message: str +) -> None: + _needs_bench_tools() + proc = _run_benches("--collect-only", "-q", *args, env=_bench_env(**env)) + + assert proc.returncode != 0 + assert message in proc.stdout + proc.stderr + + +@_launches +@pytest.mark.skipif(monoprop.has_mpi, reason="needs an MPI-off extension") +@pytest.mark.parametrize( + ("args", "env"), + [ + ( + ["--runtime-shape=partitions", "--build-mode=mpi-off"], + {"monoprop_PARTITIONS": "2", "monoprop_NUM_THREADS": "2"}, + ), + (["--runtime-shape=openmp"], {"monoprop_NUM_THREADS": "2"}), + (["--runtime-shape=openmp"], {"OMP_NUM_THREADS": "2"}), + (["--build-mode=mpi-off"], {}), + ([], {}), + ], +) +def test_suite_accepts_valid_declarations(args: list[str], env: dict[str, str]) -> None: + _needs_bench_tools() + proc = _run_benches("--collect-only", "-q", *args, env=_bench_env(**env)) + + assert proc.returncode == 0, proc.stdout[-4000:] + proc.stderr[-4000:] + + +@_launches +@pytest.mark.skipif( + not monoprop.has_mpi or shutil.which("mpiexec") is None, + reason="needs an MPI extension and mpiexec", +) +@pytest.mark.parametrize( + ("args", "env", "ok"), + [ + # The legacy guard alone would reject this candidate shape above one rank. + (["--runtime-shape=openmp"], {"monoprop_NUM_THREADS": "1"}, True), + ( + ["--runtime-shape=partitions"], + {"monoprop_PARTITIONS": "1", "monoprop_NUM_THREADS": "1"}, + True, + ), + (["--runtime-shape=partitions"], {"monoprop_NUM_THREADS": "1"}, False), + ([], {}, False), + ], +) +def test_multi_rank_shape_preflight( + args: list[str], + env: dict[str, str], + ok: bool, # noqa: FBT001 - parametrized +) -> None: + _needs_bench_tools() + exported = [flag for key in env for flag in ("-x", key)] + proc = subprocess.run( # noqa: S603 - trusted: this interpreter, repository paths + [ + shutil.which("mpiexec"), + "--map-by", + ":OVERSUBSCRIBE", + "-n", + "2", + *exported, + sys.executable, + "-m", + "pytest", + "benches", + "-p", + "no:cacheprovider", + "--collect-only", + "-q", + *args, + ], + cwd=_ROOT, + env=_bench_env(**env), + capture_output=True, + text=True, + timeout=600, + check=False, + ) + assert (proc.returncode == 0) is ok, proc.stdout[-4000:] + proc.stderr[-4000:] + + +# --------------------------------------------------------------------------------- comparator +# +# The plan's miniature inventory: two empty-map energy fixtures following the real schema, +# reference loading and provenance checks. No synthetic-mode bypass exists in the driver. + + +@_launches +def test_comparison_does_not_hide_one_regression(tmp_path): + pytest.importorskip("monoprop_bench_tools") + script = ( + Path(__file__).resolve().parents[1] / "tools/benchmark-rank-local-openmp.py" + ) + + def encoded(value): + return json.dumps( + value, sort_keys=True, separators=(",", ":"), allow_nan=False + ).encode() + + def digest(value): + return hashlib.sha256(encoded(value)).hexdigest() + + def save(name, value): + path = tmp_path / name + path.write_bytes(encoded(value)) + return str(path) + + config = { + "gen_length": 4, + "obs_terms": 16, + "num_generators": 8, + "num_modes": 8, + "cutoff": 6, + "seed": 0, + "lower_atol": None, + } + parameters = [0.0] * 8 + identity = { + "has_mpi": False, + "ranks": 1, + "threads": 2, + "index_bits": 32, + "cpu_allocation": "test-cpus", + "compiler_flags": "test", + "config": config, + "routing": {"mode": "linear", "bits": 0, "seed": 0}, + } + expected, entries = [], {} + for name in ("build", "replay"): + node = f"energy-fixture-{name}" + entries[node] = { + "operation": "energy", + "measurement_kind": "pytest", + "basis": "majorana", + "picture": "heisenberg", + "config": config, + "parameters": parameters, + "pare_threshold": None, + } + expected.append( + { + "id": name, + "profile": "tiny", + "node_id": node, + "operation": "energy", + "measurement_kind": "pytest", + "identity": identity, + "config_digest": digest(identity["config"]), + "parameters_digest": digest(parameters), + } + ) + workloads = {"schema_version": 1, "profiles": {"tiny": entries}} + campaign = { + "schema_version": 1, + "workloads_path": save("workloads.json", workloads), + "workloads_sha256": digest(workloads), + "cells": expected, + } + campaign_path = save("campaign.json", campaign) + cells = [] + for target, candidate_time in zip(expected, (10.1, 5.0), strict=True): + name = target["id"] + cell = {"id": name} + for arm, elapsed in (("baseline", 10.0), ("candidate", candidate_time)): + shape = "partitions" if arm == "baseline" else "openmp" + binary_hash = ("b" if arm == "baseline" else "c") * 64 + settings = { + "monoprop_NUM_THREADS": "2", + "OMP_NUM_THREADS": "2", + "OMP_DYNAMIC": "FALSE", + } + if arm == "baseline": + settings.update(monoprop_PARTITIONS="2") + placement = { + "schema_version": 1, + "artifact_kind": "placement", + "arm": arm, + "cell_id": name, + "binary_hash": binary_hash, + "identity": identity, + "runtime_shape": shape, + "declared": { + "ranks": 1, + "threads": 2, + "cpu_allocation": "test-cpus", + "settings": settings, + }, + "observed": { + "has_mpi": False, + "ranks": 1, + "threads_per_rank": [2], + "cpu_allocation": "test-cpus", + }, + } + placement_path = save(f"{name}-{arm}-placement.json", placement) + common = { + "schema_version": 1, + "campaign_sha256": digest(campaign), + "arm": arm, + "has_mpi": False, + "cell_id": name, + "profile": target["profile"], + "node_id": target["node_id"], + "operation": "energy", + "binary_hash": binary_hash, + "identity": identity, + "config_digest": target["config_digest"], + "parameters_digest": target["parameters_digest"], + "placement_path": placement_path, + "placement_sha256": digest(placement), + } + validation = { + **common, + "artifact_kind": "validation", + "run_id": f"{name}-{arm}-validation", + "basis": "majorana", + "picture": "heisenberg", + "energy": 0.0, + "global_term_count": 0, + "graph_layers": 0, + "term_files": [], + } + validation_path = save(f"{name}-{arm}-validation.json", validation) + cell[arm] = [] + for i in range(5): + sample_id = f"{name}-{arm}-{i}" + timed = { + **common, + "artifact_kind": "timed", + "sample_id": sample_id, + "run_id": f"{sample_id}-timed", + "runtime_seconds": elapsed, + "peak_sum_bytes": 1000, + "peak_max_bytes": 1000, + "op_exact": True, + "outer_peak_sum_bytes": 1200, + "outer_peak_max_bytes": 1200, + "outer_exact": True, + "operation_floor_sum_bytes": 800, + "operation_floor_max_bytes": 800, + "operation_delta_sum_bytes": 200, + "operation_delta_max_bytes": 200, + } + construction = { + **common, + "artifact_kind": "construction", + "sample_id": sample_id, + "run_id": f"{sample_id}-construction", + "construction_exact": True, + "construction_peak_sum_bytes": 1200, + "construction_peak_max_bytes": 1200, + "construction_floor_sum_bytes": 800, + "construction_floor_max_bytes": 800, + "construction_delta_sum_bytes": 400, + "construction_delta_max_bytes": 400, + } + cell[arm].append( + { + "sample_id": sample_id, + "timed_path": save(f"{sample_id}-timed.json", timed), + "construction_path": save( + f"{sample_id}-construction.json", construction + ), + "validation_path": validation_path, + } + ) + cells.append(cell) + manifest = save( + "manifest.json", + {"schema_version": 1, "campaign_sha256": digest(campaign), "cells": cells}, + ) + output = tmp_path / "comparison.json" + result = subprocess.run( # noqa: S603 - trusted: this interpreter, repository paths + [ + sys.executable, + str(script), + "compare", + "--campaign", + campaign_path, + "--manifest", + manifest, + "--output", + str(output), + ], + check=False, + ) + assert result.returncode == 1 + assert json.loads(output.read_text())["failed_cells"] == ["build"] + + +# ------------------------------------------------------------------- evidence builder (negatives) + + +def _driver(): + """Import the driver script as a module (it is a tool, not a package).""" + pytest.importorskip("monoprop_bench_tools") + spec = importlib.util.spec_from_file_location("rank_local_driver", _SCRIPT) + module = importlib.util.module_from_spec(spec) + sys.modules[spec.name] = module + spec.loader.exec_module(module) + return module + + +def _encoded(value, *, allow_nan: bool = False) -> bytes: + return json.dumps( + value, sort_keys=True, separators=(",", ":"), allow_nan=allow_nan + ).encode() + + +def _digest(value) -> str: + return hashlib.sha256(_encoded(value)).hexdigest() + + +_RANDOM = { + "gen_length": 4, + "obs_terms": 16, + "num_generators": 8, + "num_modes": 8, + "cutoff": 6, + "seed": 0, + "lower_atol": None, +} + + +def _write_gz(path: Path, text: str) -> None: + """Write ``text`` as a gzip member, as the driver writes term files.""" + with gzip.open(path, "wt") as handle: + handle.write(text) + + +class Evidence: + """Writes a complete, valid campaign/manifest; tests then break one thing at a time.""" + + def __init__(self, root: Path) -> None: + self.root = root + self.cells: list[dict] = [] + self.entries: dict[str, dict] = {"tiny": {}, "large": {}} + + def save(self, name: str, value) -> str: + path = self.root / name + path.parent.mkdir(parents=True, exist_ok=True) + # Finite documents encode exactly as their canonical digest; non-finite ones exist only + # to be rejected. + path.write_bytes(_encoded(value, allow_nan=True)) + return str(path) + + def add_cell( + self, + cell_id: str, + *, + profile: str = "tiny", + node: str = "bench_random.py::test_random_energy[heisenberg]", + operation: str = "energy", + has_mpi: bool = False, + ranks: int = 1, + threads: int = 2, + allocation: str = "0-1", + routing: dict | None = None, + threshold: float | None = None, + kind: str = "pytest", + ) -> dict: + entry = { + "operation": operation, + "measurement_kind": kind, + "basis": "majorana", + "picture": "heisenberg", + "config": _RANDOM, + "parameters": [0.5] * 8, + "pare_threshold": threshold, + } + if operation == "pare_functional_construct": + entry["functional"] = "expectation_value_and_gradient_functional" + self.entries[profile][node] = entry + identity = { + "has_mpi": has_mpi, + "ranks": ranks, + "threads": threads, + "index_bits": 32, + "cpu_allocation": allocation, + "compiler_flags": "-O3", + "config": _RANDOM, + "routing": routing + or {"mode": "linear", "bits": ranks.bit_length() - 1, "seed": 7}, + } + cell = { + "id": cell_id, + "profile": profile, + "node_id": node, + "operation": operation, + "measurement_kind": kind, + "identity": identity, + "config_digest": _digest(_RANDOM), + "parameters_digest": _digest(entry["parameters"]), + } + self.cells.append(cell) + return cell + + def write_campaign(self) -> str: + workloads = {"schema_version": 1, "profiles": self.entries} + self.campaign = { + "schema_version": 1, + "workloads_path": "workloads.json", + "workloads_sha256": _digest(workloads), + "cells": self.cells, + } + self.save("workloads.json", workloads) + self.campaign_path = self.save("campaign.json", self.campaign) + return self.campaign_path + + def placement(self, cell: dict, which: str, /, **override) -> str: + arm = which + identity = cell["identity"] + threads = str(identity["threads"]) + settings = {"monoprop_NUM_THREADS": threads, "OMP_NUM_THREADS": threads} + if arm == "baseline": + settings["monoprop_PARTITIONS"] = threads + value = { + "schema_version": 1, + "artifact_kind": "placement", + "arm": arm, + "cell_id": cell["id"], + "binary_hash": self.binary(arm, identity), + "identity": identity, + "runtime_shape": "partitions" if arm == "baseline" else "openmp", + "declared": { + "ranks": identity["ranks"], + "threads": identity["threads"], + "cpu_allocation": identity["cpu_allocation"], + "settings": settings, + }, + "observed": { + "has_mpi": identity["has_mpi"], + "ranks": identity["ranks"], + "threads_per_rank": [identity["threads"]] * identity["ranks"], + "cpu_allocation": identity["cpu_allocation"], + }, + } + for key, item in override.items(): + if key in ("declared", "observed"): + value[key] = {**value[key], **item} + else: + value[key] = item + return self.save(f"{cell['id']}/{arm}/placement.json", value) + + @staticmethod + def binary(arm: str, identity: dict) -> str: + tag = ("b" if arm == "baseline" else "c") + ( + "m" if identity["has_mpi"] else "o" + ) + return (tag * 32)[:64] + + def common(self, cell: dict, arm: str, placement: str) -> dict: + return { + "schema_version": 1, + "campaign_sha256": _digest(self.campaign), + "arm": arm, + "has_mpi": cell["identity"]["has_mpi"], + "cell_id": cell["id"], + "profile": cell["profile"], + "node_id": cell["node_id"], + "operation": cell["operation"], + "binary_hash": self.binary(arm, cell["identity"]), + "identity": cell["identity"], + "config_digest": cell["config_digest"], + "parameters_digest": cell["parameters_digest"], + "placement_path": placement, + "placement_sha256": _digest(json.loads(Path(placement).read_bytes())), + } + + def terms(self, cell: dict, arm: str, ranks: list[list[tuple]]) -> list[str]: + paths = [] + for rank, terms in enumerate(ranks): + path = self.root / cell["id"] / arm / f"terms-{rank}.jsonl.gz" + path.parent.mkdir(parents=True, exist_ok=True) + _write_gz( + path, + "".join( + json.dumps({"key": list(key), "real": real, "imag": imag}) + "\n" + for key, real, imag in sorted(terms) + ), + ) + paths.append(str(path)) + return paths + + def arm_samples( + self, + cell: dict, + arm: str, + *, + n: int = 5, + runtime: float = 10.0, + peak: float = 1000.0, + construction: float = 1200.0, + outer: float = 1500.0, + timed_extra: dict | None = None, + construction_extra: dict | None = None, + validation_extra: dict | None = None, + placement: str | None = None, + ) -> list[dict]: + placement = placement or self.placement(cell, arm) + common = self.common(cell, arm, placement) + validation = { + **common, + "artifact_kind": "validation", + "run_id": f"{cell['id']}-{arm}-validation", + "basis": "majorana", + "picture": "heisenberg", + "energy": -1.25, + "gradient": [0.1] * 8, + "global_term_count": 0, + "graph_layers": 3, + "term_files": [], + **(validation_extra or {}), + } + validation_path = self.save(f"{cell['id']}/{arm}/validation.json", validation) + samples = [] + for i in range(n): + sample_id = f"s{i:02d}" + timed = { + **common, + "artifact_kind": "timed", + "sample_id": sample_id, + "run_id": f"{cell['id']}-{arm}-{sample_id}-timed", + "runtime_seconds": runtime, + "peak_sum_bytes": peak, + "peak_max_bytes": peak, + "op_exact": True, + "outer_peak_sum_bytes": outer, + "outer_peak_max_bytes": outer, + "outer_exact": True, + **(timed_extra or {}), + } + built = { + **common, + "artifact_kind": "construction", + "sample_id": sample_id, + "run_id": f"{cell['id']}-{arm}-{sample_id}-construction", + "construction_exact": True, + "construction_peak_sum_bytes": construction, + "construction_peak_max_bytes": construction, + **(construction_extra or {}), + } + samples.append( + { + "sample_id": sample_id, + "timed_path": self.save( + f"{cell['id']}/{arm}/{sample_id}/timed.json", timed + ), + "construction_path": self.save( + f"{cell['id']}/{arm}/{sample_id}/construction.json", built + ), + "validation_path": validation_path, + } + ) + return samples + + def manifest(self, cells: list[dict], campaign_sha: str | None = None) -> str: + return self.save( + "manifest.json", + { + "schema_version": 1, + "campaign_sha256": campaign_sha or _digest(self.campaign), + "cells": cells, + }, + ) + + def compare(self, manifest: str) -> tuple[int, dict]: + driver = _driver() + output = self.root / "comparison.json" + code = driver.main( + [ + "compare", + "--campaign", + self.campaign_path, + "--manifest", + manifest, + "--output", + str(output), + ] + ) + return code, json.loads(output.read_text()) + + def passing(self, cell: dict, **arms: dict) -> dict: + return { + "id": cell["id"], + "baseline": self.arm_samples(cell, "baseline", **arms.get("baseline", {})), + "candidate": self.arm_samples( + cell, "candidate", runtime=9.0, **arms.get("candidate", {}) + ), + } + + +def _one_cell(tmp_path: Path, **cell_kwargs) -> tuple[Evidence, dict]: + evidence = Evidence(tmp_path) + cell = evidence.add_cell("tiny-energy", **cell_kwargs) + evidence.write_campaign() + return evidence, cell + + +def test_complete_passing_campaign_exits_zero(tmp_path: Path) -> None: + evidence, cell = _one_cell(tmp_path) + code, report = evidence.compare(evidence.manifest([evidence.passing(cell)])) + + assert code == 0, report + assert report["passed"] is True + gates = report["cells"]["tiny-energy"]["gates"] + assert set(gates) == { + "runtime", + "operation_peak_sum", + "operation_peak_max", + "construction_peak_sum", + "construction_peak_max", + } + assert gates["runtime"]["ratio"] == pytest.approx(0.9) + + +def test_a_whole_cell_missing_from_both_arms_fails_completeness(tmp_path: Path) -> None: + evidence = Evidence(tmp_path) + kept = evidence.add_cell("tiny-energy") + evidence.add_cell( + "large-mpi-off-full", + profile="large", + node="bench_random.py::test_random_gradient[heisenberg]", + operation="gradient", + threads=96, + allocation="0-95", + ) + evidence.write_campaign() + code, report = evidence.compare(evidence.manifest([evidence.passing(kept)])) + + assert code == 1 + assert report["missing_cells"] == ["large-mpi-off-full"] + + +@pytest.mark.parametrize("n", [4, 6]) +def test_wrong_sample_counts_are_incomplete(tmp_path: Path, n: int) -> None: + evidence, cell = _one_cell(tmp_path) + manifest = evidence.manifest( + [ + { + "id": cell["id"], + "baseline": evidence.arm_samples(cell, "baseline", n=n), + "candidate": evidence.arm_samples(cell, "candidate", n=n), + } + ] + ) + code, report = evidence.compare(manifest) + + assert code == 1 + assert "tiny-energy" in report["incomplete_cells"] + + +def test_unequal_arm_counts_are_incomplete(tmp_path: Path) -> None: + evidence, cell = _one_cell(tmp_path) + manifest = evidence.manifest( + [ + { + "id": cell["id"], + "baseline": evidence.arm_samples(cell, "baseline", n=10), + "candidate": evidence.arm_samples(cell, "candidate", n=5), + } + ] + ) + code, report = evidence.compare(manifest) + + assert code == 1 + assert "tiny-energy" in report["incomplete_cells"] + + +def test_ten_samples_after_repetition_are_judged_on_all_ten(tmp_path: Path) -> None: + evidence, cell = _one_cell(tmp_path) + manifest = evidence.manifest( + [ + { + "id": cell["id"], + "baseline": evidence.arm_samples(cell, "baseline", n=10), + "candidate": evidence.arm_samples( + cell, "candidate", n=10, runtime=10.5 + ), + } + ] + ) + code, report = evidence.compare(manifest) + + assert code == 1 + assert report["failed_cells"] == ["tiny-energy"] + assert report["cells"]["tiny-energy"]["needs_additional_samples"] == 0 + + +def test_first_failure_requests_five_more_observations(tmp_path: Path) -> None: + evidence, cell = _one_cell(tmp_path) + manifest = evidence.manifest([evidence.passing(cell, candidate={"peak": 1001.0})]) + code, report = evidence.compare(manifest) + + assert code == 1 + result = report["cells"]["tiny-energy"] + assert result["gates"]["operation_peak_sum"]["passed"] is False + assert result["needs_additional_samples"] == 5 + + +@pytest.mark.parametrize( + ("arm_kwargs", "gate"), + [ + ({"runtime": 10.01}, "runtime"), + ({"peak": 1000.5}, "operation_peak_max"), + ({"construction": 1200.5}, "construction_peak_sum"), + ({"construction": 1200.5}, "construction_peak_max"), + ], +) +def test_each_of_the_five_gates_fails_on_its_own( + tmp_path: Path, arm_kwargs: dict, gate: str +) -> None: + evidence, cell = _one_cell(tmp_path) + manifest = evidence.manifest( + [ + { + "id": cell["id"], + "baseline": evidence.arm_samples(cell, "baseline"), + "candidate": evidence.arm_samples(cell, "candidate", **arm_kwargs), + } + ] + ) + code, report = evidence.compare(manifest) + + assert code == 1 + assert report["cells"]["tiny-energy"]["gates"][gate]["passed"] is False + + +def test_sum_and_max_peaks_are_separate_gates(tmp_path: Path) -> None: + evidence, cell = _one_cell(tmp_path, has_mpi=True, ranks=2, allocation="0-3") + manifest = evidence.manifest( + [ + { + "id": cell["id"], + "baseline": evidence.arm_samples(cell, "baseline"), + "candidate": evidence.arm_samples( + cell, + "candidate", + timed_extra={"peak_sum_bytes": 900, "peak_max_bytes": 1100}, + ), + } + ] + ) + code, report = evidence.compare(manifest) + gates = report["cells"]["tiny-energy"]["gates"] + + assert code == 1 + assert gates["operation_peak_sum"]["passed"] is True + assert gates["operation_peak_max"]["passed"] is False + + +def test_outer_only_regression_is_a_diagnostic(tmp_path: Path) -> None: + evidence, cell = _one_cell(tmp_path) + manifest = evidence.manifest( + [ + evidence.passing( + cell, candidate={"outer": 9000.0, "timed_extra": {"outer_exact": None}} + ) + ] + ) + code, report = evidence.compare(manifest) + diagnostics = report["cells"]["tiny-energy"]["diagnostics"] + + assert code == 0 + assert diagnostics["outer_peak_sum"]["ratio"] == pytest.approx(6.0) + assert diagnostics["outer_peak_sum"]["exactness"] == "unknown" + + +def test_outer_non_exact_is_labelled_without_failing(tmp_path: Path) -> None: + evidence, cell = _one_cell(tmp_path) + manifest = evidence.manifest( + [evidence.passing(cell, baseline={"timed_extra": {"outer_exact": False}})] + ) + code, report = evidence.compare(manifest) + + assert code == 0 + assert ( + report["cells"]["tiny-energy"]["diagnostics"]["outer_peak_max"]["exactness"] + == "non-exact" + ) + + +@pytest.mark.parametrize( + ("where", "flag", "value"), + [ + ("timed_extra", "op_exact", False), + ("timed_extra", "op_exact", None), + ("construction_extra", "construction_exact", False), + ("construction_extra", "construction_exact", None), + ], +) +def test_inexact_required_windows_fail( + tmp_path: Path, where: str, flag: str, value +) -> None: + evidence, cell = _one_cell(tmp_path) + extra = {flag: value} + manifest = evidence.manifest([evidence.passing(cell, candidate={where: extra})]) + code, report = evidence.compare(manifest) + + assert code == 1 + assert report["failed_cells"] == ["tiny-energy"] + assert any(flag in reason for reason in report["cells"]["tiny-energy"]["reasons"]) + + +def test_absent_exactness_flag_fails(tmp_path: Path) -> None: + evidence, cell = _one_cell(tmp_path) + samples = evidence.arm_samples(cell, "candidate", runtime=9.0) + for sample in samples: + path = Path(sample["timed_path"]) + artifact = json.loads(path.read_text()) + del artifact["op_exact"] + path.write_text(json.dumps(artifact)) + manifest = evidence.manifest( + [ + { + "id": cell["id"], + "baseline": evidence.arm_samples(cell, "baseline"), + "candidate": samples, + } + ] + ) + code, _report = evidence.compare(manifest) + + assert code == 1 + + +@pytest.mark.parametrize("value", [None, float("nan"), 0, -1.0]) +def test_missing_or_non_finite_measurements_are_malformed( + tmp_path: Path, value +) -> None: + evidence, cell = _one_cell(tmp_path) + samples = evidence.arm_samples(cell, "candidate") + path = Path(samples[2]["timed_path"]) + artifact = json.loads(path.read_text()) + artifact["runtime_seconds"] = value + path.write_text(json.dumps(artifact)) + manifest = evidence.manifest( + [ + { + "id": cell["id"], + "baseline": evidence.arm_samples(cell, "baseline"), + "candidate": samples, + } + ] + ) + code, report = evidence.compare(manifest) + + assert code == 2 + assert report["errors"] + + +def test_duplicate_sample_ids_are_rejected(tmp_path: Path) -> None: + # Two genuinely distinct runs that both claim sample s00. + evidence, cell = _one_cell(tmp_path) + samples = evidence.arm_samples(cell, "candidate") + for kind in ("timed_path", "construction_path"): + path = Path(samples[1][kind]) + artifact = json.loads(path.read_text()) + artifact["sample_id"] = samples[0]["sample_id"] + path.write_text(json.dumps(artifact)) + samples[1] = {**samples[1], "sample_id": samples[0]["sample_id"]} + manifest = evidence.manifest( + [ + { + "id": cell["id"], + "baseline": evidence.arm_samples(cell, "baseline"), + "candidate": samples, + } + ] + ) + code, report = evidence.compare(manifest) + + assert code == 2 + assert "twice" in report["errors"][0] + + +def test_five_references_to_one_run_are_not_five_samples(tmp_path: Path) -> None: + evidence, cell = _one_cell(tmp_path) + samples = evidence.arm_samples(cell, "candidate") + reused = [{**samples[0], "sample_id": f"s{i:02d}"} for i in range(5)] + manifest = evidence.manifest( + [ + { + "id": cell["id"], + "baseline": evidence.arm_samples(cell, "baseline"), + "candidate": reused, + } + ] + ) + code, report = evidence.compare(manifest) + + assert code == 2 + assert "already used" in report["errors"][0] + + +def test_copied_artifacts_reusing_a_run_id_are_rejected(tmp_path: Path) -> None: + evidence, cell = _one_cell(tmp_path) + samples = evidence.arm_samples(cell, "candidate") + copy = Path(samples[0]["timed_path"]).read_text() + duplicate = tmp_path / "copy-timed.json" + duplicate.write_text(copy) + artifact = json.loads(copy) + artifact["sample_id"] = samples[1]["sample_id"] + duplicate.write_text(json.dumps(artifact)) + samples[1] = {**samples[1], "timed_path": str(duplicate)} + manifest = evidence.manifest( + [ + { + "id": cell["id"], + "baseline": evidence.arm_samples(cell, "baseline"), + "candidate": samples, + } + ] + ) + + assert evidence.compare(manifest)[0] == 2 + + +def test_measurement_reused_across_cells_is_rejected(tmp_path: Path) -> None: + evidence = Evidence(tmp_path) + first = evidence.add_cell("tiny-energy") + second = evidence.add_cell( + "tiny-gradient", + node="bench_random.py::test_random_gradient[heisenberg]", + operation="gradient", + ) + evidence.write_campaign() + one = evidence.passing(first) + two = evidence.passing(second) + two["candidate"][0] = { + **two["candidate"][0], + "timed_path": one["candidate"][0]["timed_path"], + } + code, report = evidence.compare(evidence.manifest([one, two])) + + assert code == 2 + assert "already used" in report["errors"][0] + + +def test_mismatched_timed_and_construction_pair_is_rejected(tmp_path: Path) -> None: + evidence, cell = _one_cell(tmp_path) + samples = evidence.arm_samples(cell, "candidate") + samples[0], samples[1] = ( + {**samples[0], "construction_path": samples[1]["construction_path"]}, + {**samples[1], "construction_path": samples[0]["construction_path"]}, + ) + manifest = evidence.manifest( + [ + { + "id": cell["id"], + "baseline": evidence.arm_samples(cell, "baseline"), + "candidate": samples, + } + ] + ) + + assert evidence.compare(manifest)[0] == 2 + + +@pytest.mark.parametrize( + "field", + [ + "identity", + "config_digest", + "parameters_digest", + "node_id", + "binary_hash", + "has_mpi", + "cell_id", + ], +) +def test_mismatched_identity_fields_are_rejected(tmp_path: Path, field: str) -> None: + evidence, cell = _one_cell(tmp_path) + changed = { + "identity": {**cell["identity"], "threads": 4}, + "config_digest": "0" * 64, + "parameters_digest": "0" * 64, + "node_id": "bench_random.py::test_random_gradient[heisenberg]", + "binary_hash": "f" * 64, + "has_mpi": True, + "cell_id": "other-cell", + }[field] + manifest = evidence.manifest( + [evidence.passing(cell, candidate={"construction_extra": {field: changed}})] + ) + + assert evidence.compare(manifest)[0] == 2 + + +def test_unequal_index_width_is_rejected(tmp_path: Path) -> None: + evidence, cell = _one_cell(tmp_path) + identity = {**cell["identity"], "index_bits": 64} + manifest = evidence.manifest( + [evidence.passing(cell, candidate={"timed_extra": {"identity": identity}})] + ) + + assert evidence.compare(manifest)[0] == 2 + + +def test_calibration_output_is_not_acceptance_evidence(tmp_path: Path) -> None: + # Pilot artifacts belong to a different (draft) campaign digest. + evidence, cell = _one_cell(tmp_path) + samples = evidence.arm_samples( + cell, "candidate", timed_extra={"campaign_sha256": "a" * 64} + ) + manifest = evidence.manifest( + [ + { + "id": cell["id"], + "baseline": evidence.arm_samples(cell, "baseline"), + "candidate": samples, + } + ] + ) + + assert evidence.compare(manifest)[0] == 2 + + +def test_manifest_for_another_campaign_is_rejected(tmp_path: Path) -> None: + evidence, cell = _one_cell(tmp_path) + manifest = evidence.manifest([evidence.passing(cell)], campaign_sha="d" * 64) + + assert evidence.compare(manifest)[0] == 2 + + +def test_extra_and_duplicate_manifest_cells_are_rejected(tmp_path: Path) -> None: + evidence, cell = _one_cell(tmp_path) + good = evidence.passing(cell) + assert evidence.compare(evidence.manifest([good, good]))[0] == 2 + assert ( + evidence.compare(evidence.manifest([good, {**good, "id": "not-in-inventory"}]))[ + 0 + ] + == 2 + ) + + +def test_campaign_with_duplicate_cell_ids_is_rejected(tmp_path: Path) -> None: + evidence = Evidence(tmp_path) + cell = evidence.add_cell("tiny-energy") + evidence.cells.append(dict(cell)) + evidence.write_campaign() + + assert evidence.compare(evidence.manifest([]))[0] == 2 + + +def test_tampered_workloads_file_is_rejected(tmp_path: Path) -> None: + evidence, cell = _one_cell(tmp_path) + manifest = evidence.manifest([evidence.passing(cell)]) + (tmp_path / "workloads.json").write_text("{}") + + assert evidence.compare(manifest)[0] == 2 + + +def test_mpi_off_cell_declared_at_several_ranks_is_rejected(tmp_path: Path) -> None: + evidence, _cell = _one_cell(tmp_path, has_mpi=False, ranks=2) + + assert evidence.compare(evidence.manifest([]))[0] == 2 + + +def test_linear_routing_at_three_ranks_is_rejected(tmp_path: Path) -> None: + evidence, _cell = _one_cell( + tmp_path, + has_mpi=True, + ranks=3, + routing={"mode": "linear", "bits": 1, "seed": 7}, + ) + + assert evidence.compare(evidence.manifest([]))[0] == 2 + + +def test_d1_mpi_enabled_samples_cannot_stand_in_for_mpi_off_cells( + tmp_path: Path, +) -> None: + # On a one-domain host the MPI-enabled R=1,T=C cell has the same geometry as the required + # MPI-off full-machine cell; the inventory keeps both and evidence cannot be shared. + evidence = Evidence(tmp_path) + mpi_off = evidence.add_cell( + "reference-mpi-off-full", has_mpi=False, threads=96, allocation="0-95" + ) + mpi_on = evidence.add_cell( + "reference-l2b", has_mpi=True, threads=96, allocation="0-95" + ) + evidence.write_campaign() + on = evidence.passing(mpi_on) + borrowed = { + "id": mpi_off["id"], + "baseline": on["baseline"], + "candidate": on["candidate"], + } + + assert evidence.compare(evidence.manifest([on, borrowed]))[0] == 2 + + +def test_campaign_listing_the_same_cell_twice_under_two_ids_is_rejected( + tmp_path: Path, +) -> None: + evidence = Evidence(tmp_path) + evidence.add_cell("a") + evidence.add_cell("b") + evidence.write_campaign() + + assert evidence.compare(evidence.manifest([]))[0] == 2 + + +@pytest.mark.parametrize( + "override", + [ + {"runtime_shape": "openmp"}, + {"arm": "candidate"}, + {"declared": {"threads": 1}}, + {"declared": {"settings": {"monoprop_NUM_THREADS": "2"}}}, + {"observed": {"threads_per_rank": [1]}}, + {"observed": {"has_mpi": True}}, + {"observed": {"cpu_allocation": "0"}}, + {"binary_hash": "e" * 64}, + ], +) +def test_invalid_placement_evidence_is_rejected(tmp_path: Path, override: dict) -> None: + evidence, cell = _one_cell(tmp_path) + bad = evidence.placement(cell, "baseline", **override) + manifest = evidence.manifest( + [ + { + "id": cell["id"], + "baseline": evidence.arm_samples(cell, "baseline", placement=bad), + "candidate": evidence.arm_samples(cell, "candidate"), + } + ] + ) + + assert evidence.compare(manifest)[0] == 2 + + +def test_candidate_placement_with_leftover_partitions_is_rejected( + tmp_path: Path, +) -> None: + evidence, cell = _one_cell(tmp_path) + bad = evidence.placement( + cell, + "candidate", + declared={ + "settings": {"monoprop_NUM_THREADS": "2", "monoprop_PARTITIONS": "2"} + }, + ) + manifest = evidence.manifest( + [ + { + "id": cell["id"], + "baseline": evidence.arm_samples(cell, "baseline"), + "candidate": evidence.arm_samples(cell, "candidate", placement=bad), + } + ] + ) + + assert evidence.compare(manifest)[0] == 2 + + +def test_missing_placement_provenance_is_malformed(tmp_path: Path) -> None: + evidence, cell = _one_cell(tmp_path) + samples = evidence.arm_samples(cell, "candidate") + Path( + json.loads(Path(samples[0]["timed_path"]).read_text())["placement_path"] + ).unlink() + manifest = evidence.manifest( + [ + { + "id": cell["id"], + "baseline": evidence.arm_samples(cell, "baseline"), + "candidate": samples, + } + ] + ) + + assert evidence.compare(manifest)[0] == 2 + + +def test_absent_validation_is_malformed(tmp_path: Path) -> None: + evidence, cell = _one_cell(tmp_path) + samples = evidence.arm_samples(cell, "candidate") + Path(samples[0]["validation_path"]).unlink() + manifest = evidence.manifest( + [ + { + "id": cell["id"], + "baseline": evidence.arm_samples(cell, "baseline"), + "candidate": samples, + } + ] + ) + + assert evidence.compare(manifest)[0] == 2 + + +def test_validation_for_another_binary_is_rejected(tmp_path: Path) -> None: + evidence, cell = _one_cell(tmp_path) + manifest = evidence.manifest( + [ + evidence.passing( + cell, candidate={"validation_extra": {"binary_hash": "9" * 64}} + ) + ] + ) + + assert evidence.compare(manifest)[0] == 2 + + +def _numerical( + tmp_path: Path, baseline: dict, candidate: dict, **cell_kwargs +) -> tuple[int, dict]: + evidence, cell = _one_cell(tmp_path, **cell_kwargs) + for extra, arm in ((baseline, "baseline"), (candidate, "candidate")): + if "terms" in extra: + ranks = extra.pop("terms") + extra["term_files"] = evidence.terms(cell, arm, ranks) + manifest = evidence.manifest( + [ + evidence.passing( + cell, + baseline={"validation_extra": baseline}, + candidate={"validation_extra": candidate}, + ) + ] + ) + return evidence.compare(manifest) + + +_TERMS = [ + [((), 0.5, 0.0), ((0, 1), 0.25, 0.0)], + [((), 0.5, 0.0), ((2, 3), -0.125, 0.0)], +] + + +def test_matching_global_maps_pass_across_different_ownership(tmp_path: Path) -> None: + # Candidate ownership differs; the replicated Heisenberg identity is kept once per arm. + moved = [ + [((), 0.5, 0.0), ((2, 3), -0.125, 0.0), ((0, 1), 0.25, 0.0)], + [((), 0.5, 0.0)], + ] + code, report = _numerical( + tmp_path, + {"terms": [list(t) for t in _TERMS], "global_term_count": 3}, + {"terms": moved, "global_term_count": 3}, + has_mpi=True, + ranks=2, + allocation="0-3", + ) + + assert code == 0, report + assert ( + report["cells"]["tiny-energy"]["numerical"]["pairs"][0]["terms"]["common_terms"] + == 3 + ) + + +@pytest.mark.parametrize( + ("baseline", "candidate"), + [ + ({"energy": -1.25}, {"energy": -1.2501}), + ({"gradient": [0.1] * 8}, {"gradient": [0.1] * 7 + [0.11]}), + ({"gradient": [0.1] * 8}, {"gradient": [0.1] * 7}), + ({"energy": -1.25}, {"energy": float("inf")}), + ( + {"terms": [[((0, 1), 0.25, 0.0)]], "global_term_count": 1}, + { + "terms": [[((0, 1), 0.25, 0.0), ((4, 5), 0.0, 0.0)]], + "global_term_count": 2, + }, + ), + ( + { + "terms": [[((0, 1), 0.25, 0.0), ((4, 5), 0.0, 0.0)]], + "global_term_count": 2, + }, + {"terms": [[((0, 1), 0.25, 0.0)]], "global_term_count": 1}, + ), + ( + {"terms": [[((0, 1), 0.25, 0.0)]], "global_term_count": 1}, + {"terms": [[((0, 1), 0.2500001, 0.0)]], "global_term_count": 1}, + ), + ( + {"terms": [[((0, 1), 0.25, 0.0)]], "global_term_count": 1}, + { + "terms": [[((0, 1), 0.25, 0.0)], [((0, 1), 0.25, 0.0)]], + "global_term_count": 1, + }, + ), + ( + {"terms": [[((0, 1), 0.25, 0.0)]], "global_term_count": 1}, + {"terms": [[((0, 1), 0.25, 0.0)]], "global_term_count": 2}, + ), + ( + {"terms": [[((0, 1), 0.25, 0.0)]], "global_term_count": 1}, + {"term_files": []}, + ), + ], + ids=[ + "energy", + "gradient-entry", + "gradient-length", + "non-finite", + "extra-term", + "missing-term", + "value", + "duplicate-ownership", + "count", + "one-arm-map", + ], +) +def test_numerical_mismatches_fail_the_cell( + tmp_path: Path, baseline: dict, candidate: dict +) -> None: + code, report = _numerical(tmp_path, baseline, candidate) + + assert code == 1, report + assert report["cells"]["tiny-energy"]["numerical"]["ok"] is False + + +def test_replicated_identity_disagreement_fails(tmp_path: Path) -> None: + bad = [[((), 0.5, 0.0), ((0, 1), 0.25, 0.0)], [((), 0.75, 0.0)]] + code, report = _numerical( + tmp_path, + {"terms": [list(t) for t in _TERMS], "global_term_count": 3}, + {"terms": bad, "global_term_count": 2}, + has_mpi=True, + ranks=2, + allocation="0-3", + ) + + assert code == 1 + assert "identity" in report["cells"]["tiny-energy"]["numerical"]["reason"] + + +def test_schrodinger_identity_is_an_ordinary_owned_term(tmp_path: Path) -> None: + doubled = [[((), 0.5, 0.0)], [((), 0.5, 0.0)]] + code, report = _numerical( + tmp_path, + {"terms": doubled, "global_term_count": 1, "picture": "schrodinger"}, + { + "terms": [list(t) for t in doubled], + "global_term_count": 1, + "picture": "schrodinger", + }, + has_mpi=True, + ranks=2, + allocation="0-3", + ) + + assert code == 1 + assert "owned by 2 ranks" in report["cells"]["tiny-energy"]["numerical"]["reason"] + + +def test_unsorted_term_file_is_malformed(tmp_path: Path) -> None: + evidence, cell = _one_cell(tmp_path) + files = evidence.terms( + cell, "baseline", [[((0, 1), 0.25, 0.0), ((2, 3), 0.5, 0.0)]] + ) + _write_gz( + Path(files[0]), + '{"key": [2, 3], "real": 0.5, "imag": 0.0}\n{"key": [0, 1], "real": 0.25, "imag": 0.0}\n', + ) + manifest = evidence.manifest( + [ + evidence.passing( + cell, + baseline={ + "validation_extra": {"term_files": files, "global_term_count": 2} + }, + candidate={ + "validation_extra": {"term_files": files, "global_term_count": 2} + }, + ) + ] + ) + + assert evidence.compare(manifest)[0] == 2 + + +def test_pared_evidence_must_not_carry_term_files(tmp_path: Path) -> None: + evidence, cell = _one_cell(tmp_path, threshold=1e-10) + files = evidence.terms(cell, "baseline", [[((0, 1), 0.25, 0.0)]]) + manifest = evidence.manifest( + [ + evidence.passing( + cell, + baseline={ + "validation_extra": {"term_files": files, "pare_threshold": 1e-10} + }, + candidate={"validation_extra": {"pare_threshold": 1e-10}}, + ) + ] + ) + + assert evidence.compare(manifest)[0] == 2 + + +def test_failed_graph_replay_check_fails_the_cell(tmp_path: Path) -> None: + evidence, cell = _one_cell( + tmp_path, + node="bench_random.py::test_random_propagate[heisenberg]", + operation="propagate", + ) + manifest = evidence.manifest( + [ + evidence.passing( + cell, + candidate={ + "validation_extra": { + "graph_replay_check": {"applicable": True, "ok": False} + } + }, + ) + ] + ) + code, report = evidence.compare(manifest) + + assert code == 1 + assert "replay" in report["cells"]["tiny-energy"]["numerical"]["reason"] + + +def test_pare_construction_validates_the_callable_gradient(tmp_path: Path) -> None: + evidence, cell = _one_cell( + tmp_path, + node="driver::pare_functional_construct[heisenberg]", + operation="pare_functional_construct", + threshold=1e-10, + kind="driver", + ) + manifest = evidence.manifest( + [ + evidence.passing( + cell, + baseline={"validation_extra": {"pare_threshold": 1e-10}}, + candidate={ + "validation_extra": {"pare_threshold": 1e-10, "gradient": None} + }, + ) + ] + ) + code, report = evidence.compare(manifest) + + assert code == 1 + assert "gradient" in report["cells"]["tiny-energy"]["numerical"]["reason"] + + +# ------------------------------------------------------------- measurement routines (with fakes) + + +class _FakeWindow: + """Records open/close in the shared event log; fixed byte counts.""" + + def __init__(self, log: list[str], name: str, *, settle: bool = True) -> None: + self.log, self.name, self.settle = log, name, settle + self.peak_bytes, self.baseline_bytes, self.exact = 2000, 500, True + + @property + def delta_bytes(self) -> int: + return self.peak_bytes - self.baseline_bytes + + def start(self) -> None: + self.log.append(f"{self.name}:start(settle={self.settle})") + + def stop(self) -> None: + self.log.append(f"{self.name}:stop") + + +class _FakeCallable: + def __init__(self, log: list[str]) -> None: + self.log = log + + def __call__(self, parameters): + self.log.append("EVALUATED") + return -1.0, [0.0] * len(parameters) + + def __del__(self) -> None: + self.log.append("callable-dead") + + +class _FakePropagator: + def __init__(self, log: list[str]) -> None: + self.log = log + simulator = type("Sim", (), {})() + simulator.operator_memory_bytes = lambda: 10 + simulator.graph_memory_bytes = lambda: 20 + self._simulator = simulator + + def build_graph(self, _circuit) -> None: + self.log.append("build_graph") + + def propagate(self, _circuit) -> None: + self.log.append("propagate") + + def expectation_value_and_gradient_functional(self, threshold): + self.log.append(f"functional({threshold})") + return _FakeCallable(self.log) + + def expectation_value_functional(self, threshold): + self.log.append(f"energy-functional({threshold})") + return _FakeCallable(self.log) + + +def _fake_context(driver): + return driver.Context( + comm=None, + rank=0, + size=1, + has_mpi=False, + binary_path="x", + binary_hash="0" * 64, + declared={}, + cpu_allocation="0", + placement_path="p", + placement_sha256="1" * 64, + ) + + +def _fake_problem(driver, log: list[str], steps: int = 2): + return driver.Problem( + make_propagator=lambda: ( + log.append("propagator-built") or _FakePropagator(log), + "circuit", + ), + steps=steps, + parameters=[0.1, 0.2], + ) + + +def test_pare_construction_times_only_the_callable_and_keeps_it_alive() -> None: + driver = _driver() + log: list[str] = [] + windows = iter(["outer", "operation"]) + ticks = iter([100.0, 103.5]) + + def clock() -> float: + log.append("clock") + return next(ticks) + + result = driver.timed_pare_construct( + _fake_problem(driver, log), + _fake_context(driver), + window_factory=lambda **kw: _FakeWindow(log, next(windows), **kw), + clock=clock, + events=log, + ) + + assert result["runtime_seconds"] == pytest.approx(3.5) + assert "EVALUATED" not in log + order = [ + "outer:start(settle=True)", + "propagator-built", + "build_graph", + "graph-built", + "operation:start(settle=False)", + "clock", + "functional(1e-10)", + "clock", + "operation:stop", + "outer:stop", + "callable-dead", + "callable-released", + ] + positions = [log.index(event) for event in order[:6]] + [ + len(log) - 1 - log[::-1].index(event) for event in order[6:] + ] + assert positions == sorted(positions), log + assert log.count("build_graph") == 2 # every step before timing starts + assert result["op_exact"] is True + assert result["outer_exact"] is True + assert result["peak_sum_bytes"] == result["peak_max_bytes"] == 2000 + + +def test_construction_window_opens_before_inputs_and_outlives_outputs() -> None: + driver = _driver() + log: list[str] = [] + spec = driver.parse_node("bench_random.py::test_random_gradient[heisenberg]") + + def build(): + log.append("inputs-built") + return _fake_problem(driver, log, steps=1) + + result = driver.construction_worker( + spec, + {"pare_threshold": None}, + _fake_context(driver), + window_factory=lambda **kw: _FakeWindow(log, "construction", **kw), + build=build, + events=log, + ) + + assert log.index("construction:start(settle=True)") < log.index("inputs-built") + assert log.index("EVALUATED") < log.index("construction:stop") + assert log.index("construction:stop") < log.index("callable-dead") + assert result["construction_exact"] is True + assert "runtime_seconds" not in result # the untimed worker invents no timing + + +def test_pare_construction_worker_builds_but_never_evaluates_the_callable() -> None: + driver = _driver() + log: list[str] = [] + spec = driver.parse_node("driver::pare_functional_construct[schrodinger]") + driver.construction_worker( + spec, + {"pare_threshold": 1e-10}, + _fake_context(driver), + window_factory=lambda **kw: _FakeWindow(log, "construction", **kw), + build=lambda: _fake_problem(driver, log, steps=1), + events=log, + ) + + assert "functional(1e-10)" in log + assert "EVALUATED" not in log + assert log.index("construction:stop") < log.index("callable-dead") + + +def test_propagate_construction_runs_every_step_without_a_graph() -> None: + driver = _driver() + log: list[str] = [] + spec = driver.parse_node("bench_models.py::test_model_propagate[hubbard]") + driver.construction_worker( + spec, + {"pare_threshold": None}, + _fake_context(driver), + window_factory=lambda **kw: _FakeWindow(log, "construction", **kw), + build=lambda: _fake_problem(driver, log, steps=3), + events=log, + ) + + assert log.count("propagate") == 3 + assert "build_graph" not in log + + +# ------------------------------------------------------------------ workload and CLI identity + + +def test_driver_has_exactly_three_modes() -> None: + driver = _driver() + parser = driver.build_parser() + modes = next(a for a in parser._actions if a.dest == "mode") + + assert set(modes.choices) == {"observe", "validate", "compare"} + observe = modes.choices["observe"] + required = {a.dest for a in observe._actions if a.required} + assert required == { + "arm", + "runtime_shape", + "campaign", + "cell_id", + "sample_id", + "placement", + "output", + } + validate = modes.choices["validate"] + assert "sample_id" not in {a.dest for a in validate._actions} + assert "whole_process" not in {a.dest for a in validate._actions} + + +@pytest.mark.parametrize( + ("node", "expected"), + [ + ( + "bench_random.py::test_random_energy[schrodinger]", + ("random", "energy", "schrodinger", "pytest"), + ), + ( + "bench_models.py::test_model_propagate[pauli]", + ("pauli", "propagate", "heisenberg", "pytest"), + ), + ( + "driver::pare_functional_construct[heisenberg]", + ("random", "pare_functional_construct", "heisenberg", "driver"), + ), + ], +) +def test_node_ids_resolve_to_historical_benchmarks(node: str, expected: tuple) -> None: + spec = _driver().parse_node(node) + + assert ( + spec.family, + spec.operation, + spec.picture, + spec.measurement_kind, + ) == expected + + +@pytest.mark.parametrize( + "node", + [ + "bench_random.py::test_random_inplace[heisenberg]", + "bench_models.py::test_random_energy[hubbard]", + "driver::pare_functional_construct[hubbard]", + "bench_random.py::test_random_energy", + ], +) +def test_unknown_node_ids_are_rejected(node: str) -> None: + driver = _driver() + with pytest.raises(driver.EvidenceError): + driver.parse_node(node) + + +def _pare_entry(**changes) -> dict: + entry = { + "operation": "pare_functional_construct", + "measurement_kind": "driver", + "functional": "expectation_value_and_gradient_functional", + "basis": "majorana", + "picture": "heisenberg", + "config": _RANDOM, + "parameters": [0.1] * 8, + "pare_threshold": 1e-10, + } + entry.update(changes) + return entry + + +@pytest.mark.parametrize( + "changes", + [ + {"pare_threshold": 1e-9}, + {"pare_threshold": None}, + {"functional": "expectation_value_functional"}, + {"measurement_kind": "pytest"}, + {"picture": "schrodinger"}, + {"basis": "pauli"}, + {"parameters": [float("nan")]}, + ], +) +def test_pare_construction_manifest_identity_is_exact(changes: dict) -> None: + driver = _driver() + with pytest.raises(driver.EvidenceError): + driver.check_workload_entry( + "tiny-pared", + "driver::pare_functional_construct[heisenberg]", + _pare_entry(**changes), + ) + + +def test_pare_construction_manifest_entry_is_accepted() -> None: + driver = _driver() + spec = driver.check_workload_entry( + "tiny-pared", "driver::pare_functional_construct[heisenberg]", _pare_entry() + ) + + assert spec.operation == "pare_functional_construct" + + +def test_pytest_flags_reproduce_the_workload_config() -> None: + driver = _driver() + spec = driver.parse_node("bench_models.py::test_model_energy[hubbard]") + entry = { + "config": {"num_sites": 4, "lower_atol": 0.0001, "observable_spin": "up"}, + "pare_threshold": 1e-10, + } + + assert driver.pytest_flags(spec, entry) == [ + "--hubbard-num-sites=4", + "--hubbard-lower-atol=0.0001", + "--hubbard-observable-spin=up", + "--pare-threshold=1e-10", + ] + + +def test_random_pytest_cells_must_use_the_benchmark_default_lower_atol() -> None: + driver = _driver() + entry = { + "operation": "energy", + "measurement_kind": "pytest", + "basis": "majorana", + "picture": "heisenberg", + "config": {**_RANDOM, "lower_atol": 1e-6}, + "parameters": [0.1] * 8, + "pare_threshold": None, + } + with pytest.raises(driver.EvidenceError, match="lower_atol"): + driver.check_workload_entry( + "tiny", "bench_random.py::test_random_energy[heisenberg]", entry + ) + + +# --------------------------------------------------------------- end to end on the real engine +# +# Baseline-arm artifacts only: a baseline binary is never relabelled as candidate evidence, even +# in a test. These runs check schema and provenance, not performance. + +_E2E_NODES = { + "e2e-energy": ("tiny", "bench_random.py::test_random_energy[heisenberg]", None), + "e2e-propagate": ( + "tiny", + "bench_random.py::test_random_propagate[heisenberg]", + None, + ), + "e2e-pare": ("tiny-pared", "driver::pare_functional_construct[schrodinger]", 1e-10), + "e2e-hubbard": ("tiny", "bench_models.py::test_model_build_graph[hubbard]", None), +} +_TINY_HUBBARD = {"num_sites": 4, "observable_site": 1, "trotter_steps": 2} +_TINY_RANDOM = {**_RANDOM, "obs_terms": 16} + + +def _e2e_campaign(root: Path, *, has_mpi: bool = False) -> tuple[str, dict]: + # Optional workspace dependency, absent when the suite tests a built wheel. + from monoprop_bench_tools.models import ( # noqa: PLC0415 + HubbardConfig, + build_hubbard_problem, + make_random_problem, + ) + + driver = _driver() + profiles: dict[str, dict] = {"tiny": {}, "tiny-pared": {}} + cells = [] + allocation = driver.format_cpu_list(os.sched_getaffinity(0)) + for cell_id, (profile, node, threshold) in _E2E_NODES.items(): + spec = driver.parse_node(node) + if spec.family == "random": + config = dict(_TINY_RANDOM) + problem = make_random_problem( + **{k: v for k, v in config.items() if k != "lower_atol"} + ) + parameters = [float(p) for p in problem.parameters] + else: + hubbard = HubbardConfig(**_TINY_HUBBARD) + config = dataclasses.asdict(hubbard) + # Rank-local construction: an MPI build must not be handed the world implicitly. + _propagator, circuit = build_hubbard_problem(hubbard, comm=_serial_comm()) + parameters = [float(p) for p in circuit.parameters] * hubbard.trotter_steps + entry = { + "operation": spec.operation, + "measurement_kind": spec.measurement_kind, + "basis": "majorana", + "picture": spec.picture, + "config": config, + "parameters": parameters, + "pare_threshold": threshold, + } + if spec.measurement_kind == "driver": + entry["functional"] = "expectation_value_and_gradient_functional" + profiles[profile][node] = entry + identity = { + "has_mpi": has_mpi, + "ranks": 1, + "threads": 1, + "index_bits": 32, + "cpu_allocation": allocation, + "compiler_flags": "recorded-by-operator", + "config": config, + "routing": {"mode": "linear", "bits": 0, "seed": driver.DEFAULT_ROUTE_SEED}, + } + cells.append( + { + "id": cell_id, + "profile": profile, + "node_id": node, + "operation": spec.operation, + "measurement_kind": spec.measurement_kind, + "identity": identity, + "config_digest": _digest(config), + "parameters_digest": _digest(parameters), + } + ) + workloads = {"schema_version": 1, "profiles": profiles} + (root / "workloads.json").write_bytes(_encoded(workloads)) + campaign = { + "schema_version": 1, + "workloads_path": "workloads.json", + "workloads_sha256": _digest(workloads), + "cells": cells, + } + path = root / "campaign.json" + path.write_bytes(_encoded(campaign)) + binary = hashlib.sha256(Path(monoprop._core.__file__).read_bytes()).hexdigest() + placements = {} + for cell in cells: + placement = { + "schema_version": 1, + "artifact_kind": "placement", + "arm": "baseline", + "cell_id": cell["id"], + "binary_hash": binary, + "identity": cell["identity"], + "runtime_shape": "partitions", + "declared": { + "ranks": 1, + "threads": 1, + "cpu_allocation": allocation, + "settings": dict(_E2E_SETTINGS), + }, + "observed": { + "has_mpi": has_mpi, + "ranks": 1, + "threads_per_rank": [1], + "cpu_allocation": allocation, + }, + } + placement_path = root / f"placement-{cell['id']}.json" + placement_path.write_bytes(_encoded(placement)) + placements[cell["id"]] = str(placement_path) + return str(path), placements + + +def _serial_comm(): + """Return COMM_SELF for an MPI extension, ``None`` otherwise (never imports mpi4py MPI-off).""" + if not monoprop.has_mpi: + return None + from mpi4py import MPI # noqa: PLC0415 - gated on the build mode + + return MPI.COMM_SELF + + +_E2E_SETTINGS = { + "monoprop_NUM_THREADS": "1", + "monoprop_PARTITIONS": "1", + "OMP_NUM_THREADS": "1", +} + + +def _driver_run( + *args: str, env: dict[str, str] | None = None +) -> subprocess.CompletedProcess[str]: + return subprocess.run( # noqa: S603 - trusted: this interpreter, repository paths + [sys.executable, str(_SCRIPT), *args], + cwd=_ROOT, + env=env if env is not None else _bench_env(**_E2E_SETTINGS), + capture_output=True, + text=True, + timeout=600, + check=False, + ) + + +def _observe_args( + campaign: str, placements: dict, cell: str, out: Path, *extra: str +) -> list[str]: + return [ + "--arm", + "baseline", + "--runtime-shape", + "partitions", + "--campaign", + campaign, + "--cell-id", + cell, + "--placement", + placements[cell], + "--output", + str(out), + *extra, + ] + + +@_launches +@pytest.mark.skipif(monoprop.has_mpi, reason="MPI-off end-to-end run") +def test_observe_and_validate_write_complete_joined_artifacts(tmp_path: Path) -> None: + _needs_bench_tools() + driver = _driver() + campaign, placements = _e2e_campaign(tmp_path) + out = tmp_path / "results" + loaded = driver.load_campaign(Path(campaign)) + placement_cache: dict = {} + for cell_id in _E2E_NODES: + base = _observe_args(campaign, placements, cell_id, out) + for extra in ([], ["--whole-process"]): + proc = _driver_run("observe", *base, "--sample-id", "s00", *extra) + assert proc.returncode == 0, proc.stdout[-3000:] + proc.stderr[-3000:] + proc = _driver_run("validate", *base) + assert proc.returncode == 0, proc.stdout[-3000:] + proc.stderr[-3000:] + + cell = loaded.cells[cell_id] + timed = json.loads( + (out / "baseline" / cell_id / "s00" / "timed.json").read_text() + ) + built = json.loads( + (out / "baseline" / cell_id / "s00" / "construction.json").read_text() + ) + (validation_path,) = (out / "baseline" / cell_id).glob("validation-*.json") + validation = json.loads(validation_path.read_text()) + expected = { + "arm": "baseline", + "cell_id": cell_id, + "campaign_sha256": loaded.sha256, + "binary_hash": timed["binary_hash"], + "profile": cell["profile"], + "node_id": cell["node_id"], + "operation": cell["operation"], + "identity": cell["identity"], + "config_digest": cell["config_digest"], + "parameters_digest": cell["parameters_digest"], + "has_mpi": False, + "placement_sha256": timed["placement_sha256"], + } + for artifact in (timed, built, validation): + driver._check_join(artifact, expected, cell_id, placement_cache) + assert artifact["has_mpi"] is False + assert timed["runtime_seconds"] > 0 + assert timed["op_exact"] is True + assert timed["outer_exact"] is True + assert timed["peak_sum_bytes"] == timed["peak_max_bytes"] > 0 + assert built["construction_exact"] is True + assert built["construction_peak_sum_bytes"] > 0 + assert "runtime_seconds" not in built + assert driver.compare_validations(validation, validation)["ok"] is True + if cell_id == "e2e-pare": + assert validation["term_files"] == [] + assert len(validation["gradient"]) == len(loaded.entry(cell)["parameters"]) + else: + assert validation["global_term_count"] > 0 + assert len(validation["term_files"]) == 1 + assert validation["term_files"][0].endswith(".jsonl.gz") + if cell_id == "e2e-propagate": + assert validation["graph_replay_check"]["applicable"] is True + assert validation["graph_replay_check"]["ok"] is True + if cell_id in ("e2e-energy", "e2e-hubbard"): + assert timed["raw_dir"].endswith("s00/timed") + + # Artifacts are created exclusively: rerunning a sample cannot overwrite it. + again = _driver_run( + "observe", + *_observe_args(campaign, placements, "e2e-energy", out), + "--sample-id", + "s00", + ) + assert again.returncode == 2 + assert "never overwritten" in again.stderr + + +@_launches +@pytest.mark.skipif(monoprop.has_mpi, reason="MPI-off end-to-end run") +@pytest.mark.parametrize( + ("change", "message"), + [ + ({"arm": "candidate"}, "arm"), + ({"env": {"monoprop_NUM_THREADS": "1", "OMP_NUM_THREADS": "1"}}, "partitions"), + ({"env": {**_E2E_SETTINGS, "OMP_PLACES": "cores"}}, "settings differ"), + ({"env": {**_E2E_SETTINGS, "monoprop_ROUTING": "splitmix"}}, "routing"), + ({"has_mpi": True}, "has_mpi"), + ({"cell": "not-a-cell"}, "no cell"), + ], +) +def test_observe_rejects_contradictory_evidence( + tmp_path: Path, change: dict, message: str +) -> None: + _needs_bench_tools() + campaign, placements = _e2e_campaign(tmp_path, has_mpi=change.get("has_mpi", False)) + cell = change.get("cell", "e2e-energy") + args = _observe_args( + campaign, {**placements, cell: placements["e2e-energy"]}, cell, tmp_path / "r" + ) + if "arm" in change: + args[1] = change["arm"] + env = _bench_env(**change.get("env", _E2E_SETTINGS)) + proc = _driver_run("observe", *args, "--sample-id", "s00", env=env) + + assert proc.returncode == 2 + assert message in proc.stderr + assert not (tmp_path / "r" / "baseline" / cell / "s00" / "timed.json").exists() + + +@_launches +@pytest.mark.skipif( + not monoprop.has_mpi or shutil.which("mpiexec") is None, + reason="needs an MPI extension and mpiexec", +) +def test_two_rank_observe_and_validate_merge_the_replicated_identity( + tmp_path: Path, +) -> None: + _needs_bench_tools() + driver = _driver() + campaign_path, placements = _e2e_campaign(tmp_path, has_mpi=True) + # Re-issue the energy cell at R=2 (linear, one rank bit) over the whole host mask. + campaign = json.loads(Path(campaign_path).read_text()) + cell = next(c for c in campaign["cells"] if c["id"] == "e2e-energy") + allocation = driver.format_cpu_list(range(os.cpu_count() or 1)) + cell["identity"] = { + **cell["identity"], + "ranks": 2, + "cpu_allocation": allocation, + "routing": {"mode": "linear", "bits": 1, "seed": driver.DEFAULT_ROUTE_SEED}, + } + campaign["cells"] = [cell] + Path(campaign_path).write_bytes(_encoded(campaign)) + placement = json.loads(Path(placements["e2e-energy"]).read_text()) + placement["identity"] = cell["identity"] + placement["declared"] |= {"ranks": 2, "cpu_allocation": allocation} + placement["observed"] |= { + "ranks": 2, + "threads_per_rank": [1, 1], + "cpu_allocation": allocation, + } + Path(placements["e2e-energy"]).write_bytes(_encoded(placement)) + out = tmp_path / "results" + base = _observe_args(campaign_path, placements, "e2e-energy", out) + launcher = [shutil.which("mpiexec"), "--bind-to", "none", "-n", "2"] + exported = [flag for key in _E2E_SETTINGS for flag in ("-x", key)] + for args in (["observe", *base, "--sample-id", "s00"], ["validate", *base]): + proc = subprocess.run( # noqa: S603 - trusted: this interpreter, repository paths + [*launcher, *exported, sys.executable, str(_SCRIPT), *args], + cwd=_ROOT, + env=_bench_env(**_E2E_SETTINGS), + capture_output=True, + text=True, + timeout=600, + check=False, + ) + assert proc.returncode == 0, proc.stdout[-3000:] + proc.stderr[-3000:] + + timed = json.loads( + (out / "baseline" / "e2e-energy" / "s00" / "timed.json").read_text() + ) + (validation_path,) = (out / "baseline" / "e2e-energy").glob("validation-*.json") + validation = json.loads(validation_path.read_text()) + assert timed["has_mpi"] is True + assert timed["observed"]["ranks"] == 2 + assert timed["peak_sum_bytes"] > timed["peak_max_bytes"] + assert len(validation["term_files"]) == 2 + # The identity sits on both ranks' files but counts once in the global map. + merged = list(driver.merged_terms(validation["term_files"], "heisenberg")) + assert [key for key, _r, _i in merged].count(()) == 1 + assert validation["global_term_count"] == len(merged) + assert driver.compare_validations(validation, validation)["ok"] is True + + +def test_term_lines_encode_exactly_as_json(tmp_path: Path) -> None: + driver = _driver() + terms = { + (): complex(0.0, -0.0), + (0, 1, 2, 6): complex(-2.9288912e-05, 1e300), + (3, 250): complex(0.1 + 0.2, -123456789.125), + } + path = tmp_path / "terms.jsonl.gz" + assert driver._write_terms(path, terms) == (3, True) + expected = "".join( + json.dumps({"key": list(key), "real": terms[key].real, "imag": terms[key].imag}) + + "\n" + for key in sorted(terms) + ) + # A gzip member whose decompressed lines are exactly the JSON encoding. + assert path.read_bytes()[:2] == b"\x1f\x8b" + assert gzip.decompress(path.read_bytes()).decode() == expected + # Reproducible bytes: MTIME (header bytes 4-7) is zero and no file name is stored. + assert path.read_bytes()[4:8] == bytes(4) + assert b"terms" not in path.read_bytes()[:32] + again = tmp_path / "again.jsonl.gz" + driver._write_terms(again, terms) + assert again.read_bytes() == path.read_bytes() + + +@pytest.mark.parametrize("name", ["terms.jsonl", "terms.txt"]) +def test_term_files_must_be_gzip_json_lines(tmp_path: Path, name: str) -> None: + driver = _driver() + with pytest.raises(driver.EvidenceError, match=r"\.jsonl\.gz"): + driver._write_terms(tmp_path / name, {(0, 1): 0.5 + 0j}) + + +def test_uncompressed_term_file_is_malformed(tmp_path: Path) -> None: + evidence, cell = _one_cell(tmp_path) + plain = tmp_path / "plain.jsonl" + plain.write_text('{"key": [0, 1], "real": 0.25, "imag": 0.0}\n') + extra = {"term_files": [str(plain)], "global_term_count": 1} + manifest = evidence.manifest( + [ + evidence.passing( + cell, + baseline={"validation_extra": extra}, + candidate={"validation_extra": extra}, + ) + ] + ) + code, report = evidence.compare(manifest) + + assert code == 2 + assert ".jsonl.gz" in report["errors"][0] + + +def test_corrupt_gzip_term_file_is_malformed(tmp_path: Path) -> None: + evidence, cell = _one_cell(tmp_path) + (files,) = [evidence.terms(cell, "baseline", [[((0, 1), 0.25, 0.0)]])] + data = Path(files[0]).read_bytes() + Path(files[0]).write_bytes(data[: len(data) // 2]) + extra = {"term_files": files, "global_term_count": 1} + manifest = evidence.manifest( + [ + evidence.passing( + cell, + baseline={"validation_extra": extra}, + candidate={"validation_extra": extra}, + ) + ] + ) + + assert evidence.compare(manifest)[0] == 2 + + +def test_replay_stream_comparison_needs_no_second_file(tmp_path: Path) -> None: + driver = _driver() + own = tmp_path / "rank-00000.jsonl.gz" + driver._write_terms(own, {(0, 1): 0.5 + 0j, (2, 3): -0.25 + 0j}) + + same = driver.compare_sorted_streams( + driver._read_terms(own), + driver._sorted_items({(2, 3): -0.25 + 0j, (0, 1): 0.5 + 0j}), + ) + moved = driver.compare_sorted_streams( + driver._read_terms(own), + driver._sorted_items({(0, 1): 0.5 + 0j, (4, 5): -0.25 + 0j}), + ) + + assert same["ok"] is True + assert moved["ok"] is False + assert list(tmp_path.iterdir()) == [own] diff --git a/tools/benchmark-rank-local-openmp.py b/tools/benchmark-rank-local-openmp.py new file mode 100755 index 00000000..949f49c4 --- /dev/null +++ b/tools/benchmark-rank-local-openmp.py @@ -0,0 +1,1915 @@ +#!/usr/bin/env python3 +# Copyright 2026 Algorithmiq +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"""Evidence driver for the rank-local OpenMP parity campaign. + +A thin execution/validation wrapper around the existing benchmark suite (``benches/``), not a +benchmark framework. It has exactly three modes: + +``observe`` + One fresh-process measurement of one campaign cell. The default writes + ``DIR/ARM/CELL/SAMPLE/timed.json``: a historical pytest node run in this process (raw pytest + artifacts kept under ``SAMPLE/timed/``), or the driver-owned + ``driver::pare_functional_construct[]`` measurement. ``--whole-process`` instead + writes ``DIR/ARM/CELL/SAMPLE/construction.json`` from a separate, untimed process that opens + one memory window before any input is built. +``validate`` + A separate untimed process that rebuilds the cell's workload and records its numerical + products (energy, full gradient, decoded per-rank term files) as + ``DIR/ARM/CELL/validation-.json``. +``compare`` + Joins baseline/candidate evidence against a frozen campaign inventory and applies the five + parity gates -- runtime, operation peak sum/max, construction peak sum/max -- each median + ratio <= 1.00, plus the numerical comparison of validation products. Exit 0: every cell + passes; 1: an unmet gate or incomplete coverage; 2: malformed or incompatible evidence. + +The launcher, allocation, binding and environment are external: ``observe`` and ``validate`` +run under whatever ``mpiexec`` (MPI builds) or direct launch (MPI-off builds) supplies them, and +check what they observe against the campaign cell and the placement evidence. +""" + +from __future__ import annotations + +import argparse +import gzip +import hashlib +import heapq +import io +import json +import math +import os +import re +import socket +import statistics +import sys +import time +import uuid +from dataclasses import dataclass +from pathlib import Path +from typing import TYPE_CHECKING, Any + +if TYPE_CHECKING: + from collections.abc import Callable, Iterable, Iterator + +SCHEMA_VERSION = 1 +REPO_ROOT = Path(__file__).resolve().parents[1] + +ARMS = ("baseline", "candidate") +ARM_SHAPES = {"baseline": "partitions", "candidate": "openmp"} +OPERATIONS = ( + "build_graph", + "propagate", + "energy", + "gradient", + "pare_functional_construct", +) +MEASUREMENT_KINDS = ("pytest", "driver") +ROUTING_MODES = ("linear", "splitmix") +PARE_FUNCTIONAL = "expectation_value_and_gradient_functional" +PARE_THRESHOLD = 1e-10 + +#: Default of ``monoprop_ROUTE_SEED`` (``routing::kDefaultSeed``). +DEFAULT_ROUTE_SEED = 0x5DEE_CE66_D0C6_2517 + +#: Observations per arm/cell: five initially, ten after the prescribed repetition. +INITIAL_SAMPLES = 5 +REPEATED_SAMPLES = 10 + +#: Term files are gzip-compressed JSON Lines; level 1 cuts them ~4-16x at several hundred MB/s. +TERM_SUFFIX = ".jsonl.gz" +TERM_GZIP_LEVEL = 1 + +#: ``test_utils::near`` from cpp/tests/TestUtilities.h.in, applied componentwise. +ABS_TOL = 1e-9 +REL_TOL = 1e-7 + +EXIT_PASS = 0 +EXIT_UNMET = 1 +EXIT_MALFORMED = 2 + +#: The five parity gates: (name, artifact kind, field). +GATES = ( + ("runtime", "timed", "runtime_seconds"), + ("operation_peak_sum", "timed", "peak_sum_bytes"), + ("operation_peak_max", "timed", "peak_max_bytes"), + ("construction_peak_sum", "construction", "construction_peak_sum_bytes"), + ("construction_peak_max", "construction", "construction_peak_max_bytes"), +) +#: Reported, never gated: the setup-spanning outer window of the timed process. +DIAGNOSTICS = ( + ("outer_peak_sum", "outer_peak_sum_bytes"), + ("outer_peak_max", "outer_peak_max_bytes"), +) + +_SLUG = re.compile(r"^[A-Za-z0-9][A-Za-z0-9._-]*$") +_RANDOM_FIELDS = ( + "gen_length", + "obs_terms", + "num_generators", + "num_modes", + "cutoff", + "seed", + "lower_atol", +) +_PICTURES = ("heisenberg", "schrodinger") +_NODE = re.compile( + r"^(?:bench_(?Prandom|models)\.py::test_(?Prandom|model)_(?P[a-z_]+)" + r"|driver::(?Ppare_functional_construct))\[(?P[a-z]+)\]$" +) + + +class EvidenceError(Exception): + """Malformed, incompatible or unverifiable evidence (exit 2).""" + + +# ------------------------------------------------------------------------------------ encoding + + +def canonical(value: object) -> bytes: + """Return the canonical JSON encoding that configuration digests hash.""" + return json.dumps( + value, sort_keys=True, separators=(",", ":"), allow_nan=False + ).encode() + + +def digest(value: object) -> str: + """Return the SHA-256 of ``value``'s canonical JSON encoding.""" + return hashlib.sha256(canonical(value)).hexdigest() + + +def file_sha256(path: Path) -> str: + """Return the SHA-256 of the exact bytes of ``path``.""" + sha = hashlib.sha256() + with path.open("rb") as handle: + for block in iter(lambda: handle.read(1 << 20), b""): + sha.update(block) + return sha.hexdigest() + + +def load_json(path: Path, what: str) -> Any: # noqa: ANN401 - arbitrary JSON + """Return the JSON document at ``path``, raising :class:`EvidenceError` if unreadable.""" + try: + return json.loads(path.read_text()) + except (OSError, UnicodeDecodeError, json.JSONDecodeError) as exc: + msg = f"cannot read {what} {path}: {exc}" + raise EvidenceError(msg) from exc + + +def require_slug(value: object, what: str) -> str: + """Return ``value`` if it is a path-safe slug.""" + if not isinstance(value, str) or not _SLUG.match(value): + msg = f"{what} {value!r} is not a path-safe slug ([A-Za-z0-9._-], no leading symbol)" + raise EvidenceError(msg) + return value + + +def _require(mapping: object, keys: Iterable[str], what: str) -> dict[str, Any]: + """Return ``mapping`` if it is a dict containing every key in ``keys``.""" + if not isinstance(mapping, dict): + msg = f"{what} is not a JSON object" + raise EvidenceError(msg) + missing = [key for key in keys if key not in mapping] + if missing: + msg = f"{what} lacks {missing}" + raise EvidenceError(msg) + return mapping + + +def near(a: float, b: float) -> bool: + """Return whether ``a`` and ``b`` agree under ``test_utils::near``.""" + return abs(a - b) <= ABS_TOL + REL_TOL * max(abs(a), abs(b)) + + +def _finite_number(value: object) -> bool: + return ( + isinstance(value, (int, float)) + and not isinstance(value, bool) + and math.isfinite(value) + ) + + +def format_cpu_list(cpus: Iterable[int]) -> str: + """Return a kernel-style CPU list (``0-3,8``) for ``cpus``.""" + ordered = sorted(set(cpus)) + runs: list[str] = [] + start = prev = None + for cpu in ordered: + if start is None: + start = prev = cpu + elif cpu == prev + 1: + prev = cpu + else: + runs.append(f"{start}-{prev}" if start != prev else f"{start}") + start = prev = cpu + if start is not None: + runs.append(f"{start}-{prev}" if start != prev else f"{start}") + return ",".join(runs) + + +# ----------------------------------------------------------------------------------- workloads + + +@dataclass(frozen=True) +class NodeSpec: + """What a historical pytest node id or a driver node id denotes.""" + + node_id: str + family: str # "random" or a fixed-model name + operation: str + picture: str + measurement_kind: str + + @property + def pytest_path(self) -> str: + """Return the pytest selector for a historical node.""" + return f"benches/{self.node_id}" + + +def parse_node(node_id: str) -> NodeSpec: + """Return the family, operation and picture of a campaign node id.""" + match = _NODE.match(node_id) if isinstance(node_id, str) else None + if match is None: + msg = f"unrecognised node id {node_id!r}" + raise EvidenceError(msg) + param = match["param"] + if match["driver"]: + if param not in _PICTURES: + msg = f"driver node {node_id!r} must name a picture" + raise EvidenceError(msg) + return NodeSpec(node_id, "random", match["driver"], param, "driver") + operation = match["op"] + if operation not in OPERATIONS[:4] or (match["file"] == "random") != ( + match["family"] == "random" + ): + msg = f"unrecognised node id {node_id!r}" + raise EvidenceError(msg) + if match["family"] == "random": + if param not in _PICTURES: + msg = f"random node {node_id!r} must name a picture" + raise EvidenceError(msg) + return NodeSpec(node_id, "random", operation, param, "pytest") + # The fixed models are Heisenberg-only (bench_models.py). + return NodeSpec(node_id, param, operation, "heisenberg", "pytest") + + +def _basis_of(family: str) -> str: + return "pauli" if family == "pauli" else "majorana" + + +def check_workload_schema(profile: str, node_id: str, entry: object) -> dict[str, Any]: + """Validate the schema of one ``profiles[profile][node_id]`` workload entry.""" + what = f"workload {profile}/{node_id}" + keys = ("operation", "measurement_kind", "basis", "picture", "config", "parameters") + entry = _require(entry, (*keys, "pare_threshold"), what) + allowed = { + "operation": OPERATIONS, + "measurement_kind": MEASUREMENT_KINDS, + "basis": ("majorana", "pauli"), + "picture": _PICTURES, + } + for key, values in allowed.items(): + if entry[key] not in values: + msg = f"{what}: {key} {entry[key]!r} is not one of {values}" + raise EvidenceError(msg) + driver = entry["operation"] == "pare_functional_construct" + if driver != (entry["measurement_kind"] == "driver"): + msg = f"{what}: only pare_functional_construct is a driver measurement" + raise EvidenceError(msg) + if not isinstance(entry["config"], dict): + msg = f"{what} config is not an object" + raise EvidenceError(msg) + params = entry["parameters"] + if not isinstance(params, list) or not all(_finite_number(p) for p in params): + msg = f"{what} parameters must be a list of finite numbers" + raise EvidenceError(msg) + threshold = entry["pare_threshold"] + if entry["operation"] in ("build_graph", "propagate") and threshold is not None: + msg = f"{what}: {entry['operation']} has no pare threshold" + raise EvidenceError(msg) + if driver: + if entry.get("functional") != PARE_FUNCTIONAL or threshold != PARE_THRESHOLD: + msg = ( + f"{what} must set functional={PARE_FUNCTIONAL!r} and " + f"pare_threshold={PARE_THRESHOLD}" + ) + raise EvidenceError(msg) + elif threshold is not None and threshold != PARE_THRESHOLD: + msg = f"{what}: pared profiles use pare_threshold={PARE_THRESHOLD}" + raise EvidenceError(msg) + return entry + + +def check_workload_entry(profile: str, node_id: str, entry: object) -> NodeSpec: + """Validate that a workload entry is runnable: its node id resolves and agrees with it.""" + what = f"workload {profile}/{node_id}" + entry = check_workload_schema(profile, node_id, entry) + spec = parse_node(node_id) + expected = { + "operation": spec.operation, + "measurement_kind": spec.measurement_kind, + "basis": _basis_of(spec.family), + "picture": spec.picture, + } + for key, value in expected.items(): + if entry[key] != value: + msg = f"{what} has {key}={entry[key]!r}, but its node id implies {value!r}" + raise EvidenceError(msg) + # bench_random.py builds every random propagator with the default lower_atol=None. + random_pytest = spec.family == "random" and spec.measurement_kind == "pytest" + if random_pytest and entry["config"].get("lower_atol") is not None: + msg = f"{what}: the random benchmarks build with lower_atol=None" + raise EvidenceError(msg) + return spec + + +def pytest_flags(spec: NodeSpec, entry: dict[str, Any]) -> list[str]: + """Return the benchmark-suite CLI options that reproduce ``entry``.""" + config = entry["config"] + flags: list[str] = [] + if spec.family == "random": + unknown = set(config) - set(_RANDOM_FIELDS) + if unknown: + msg = f"random config has unknown fields {sorted(unknown)}" + raise EvidenceError(msg) + for name in _RANDOM_FIELDS: + if name == "lower_atol": + continue + flags.append(f"--{name.replace('_', '-')}={config[name]}") + else: + for name, value in config.items(): + text = repr(value) if isinstance(value, float) else str(value) + flags.append(f"--{spec.family}-{name.replace('_', '-')}={text}") + if entry["pare_threshold"] is not None: + flags.append(f"--pare-threshold={entry['pare_threshold']!r}") + return flags + + +# ------------------------------------------------------------------------------------ campaign + + +@dataclass +class Campaign: + """A frozen campaign inventory and its digest-checked workload file.""" + + path: Path + sha256: str + workloads_path: Path + workloads_sha256: str + workloads: dict[str, Any] + cells: dict[str, dict[str, Any]] + + def entry(self, cell: dict[str, Any]) -> dict[str, Any]: + """Return the workload entry a cell resolves to.""" + return self.workloads["profiles"][cell["profile"]][cell["node_id"]] + + +def _check_identity(identity: object, what: str) -> dict[str, Any]: + keys = ( + "has_mpi", + "ranks", + "threads", + "index_bits", + "cpu_allocation", + "compiler_flags", + ) + identity = _require(identity, (*keys, "config", "routing"), what) + if not isinstance(identity["has_mpi"], bool): + msg = f"{what}: has_mpi must be a boolean" + raise EvidenceError(msg) + for key in ("ranks", "threads"): + value = identity[key] + if not isinstance(value, int) or isinstance(value, bool) or value < 1: + msg = f"{what}: {key} must be a positive integer" + raise EvidenceError(msg) + if identity["index_bits"] != 32: + msg = f"{what}: index_bits must be 32 (fixed uint32_t TermIndex)" + raise EvidenceError(msg) + if not identity["has_mpi"] and identity["ranks"] != 1: + msg = f"{what}: an MPI-off cell has R=1, not {identity['ranks']}" + raise EvidenceError(msg) + routing = _require(identity["routing"], ("mode", "bits", "seed"), f"{what} routing") + if routing["mode"] not in ROUTING_MODES: + msg = f"{what}: routing mode must be one of {ROUTING_MODES}" + raise EvidenceError(msg) + if routing != expected_routing(routing["mode"], identity["ranks"], routing["seed"]): + msg = f"{what}: routing {routing} is not what mode/ranks/seed derive" + raise EvidenceError(msg) + return identity + + +def expected_routing(mode: str, ranks: int, seed: object) -> dict[str, Any]: + """Return the routing identity the engine derives for ``mode`` at ``ranks``. + + Linear routing takes log2(R) rank bits and needs power-of-two R; splitmix and R=1 have none. + """ + if not isinstance(seed, int) or isinstance(seed, bool) or not 0 <= seed < 1 << 64: + msg = f"routing seed {seed!r} is not a uint64" + raise EvidenceError(msg) + if mode == "linear" and ranks & (ranks - 1): + msg = f"linear routing cannot route R={ranks}; successful R={ranks} cells use splitmix" + raise EvidenceError(msg) + bits = ranks.bit_length() - 1 if mode == "linear" and ranks > 1 else 0 + return {"mode": mode, "bits": bits, "seed": seed} + + +def _load_workloads( + campaign_path: Path, data: dict[str, Any] +) -> tuple[Path, str, dict]: + """Return the digest-checked workload file a campaign references.""" + workloads_path = (campaign_path.parent / data["workloads_path"]).resolve() + if not workloads_path.is_file(): + msg = f"campaign workloads file {workloads_path} does not exist" + raise EvidenceError(msg) + workloads_sha = file_sha256(workloads_path) + if workloads_sha != data["workloads_sha256"]: + msg = f"workloads file {workloads_path} does not match the campaign's workloads_sha256" + raise EvidenceError(msg) + workloads = _require( + load_json(workloads_path, "workloads"), + ("schema_version", "profiles"), + "workloads", + ) + profiles = workloads["profiles"] + if workloads["schema_version"] != SCHEMA_VERSION or not isinstance(profiles, dict): + msg = "workloads file has an unsupported schema" + raise EvidenceError(msg) + return workloads_path, workloads_sha, workloads + + +def _check_cell(raw: object, workloads: dict[str, Any]) -> dict[str, Any]: + """Validate one campaign cell against its workload entry.""" + keys = ("id", "profile", "node_id", "operation", "measurement_kind", "identity") + cell = _require(raw, (*keys, "config_digest", "parameters_digest"), "campaign cell") + what = f"cell {require_slug(cell['id'], 'cell id')}" + entry = workloads["profiles"].get(cell["profile"], {}).get(cell["node_id"]) + if entry is None: + msg = f"{what}: workloads have no {cell['profile']}/{cell['node_id']}" + raise EvidenceError(msg) + # Runnability of the node id is checked where a node runs (observe/validate). + check_workload_schema(cell["profile"], cell["node_id"], entry) + identity = _check_identity(cell["identity"], what) + expected = { + "operation": entry["operation"], + "measurement_kind": entry["measurement_kind"], + "config_digest": digest(entry["config"]), + "parameters_digest": digest(entry["parameters"]), + } + for key, value in expected.items(): + if cell[key] != value: + msg = f"{what}: {key} {cell[key]!r} does not match its workload entry" + raise EvidenceError(msg) + if identity["config"] != entry["config"]: + msg = f"{what}: identity config differs from the workload config" + raise EvidenceError(msg) + return cell + + +def load_campaign(path: Path) -> Campaign: + """Load and verify a campaign inventory and its workload file.""" + path = path.resolve() + data = _require( + load_json(path, "campaign"), + ("schema_version", "workloads_path", "workloads_sha256", "cells"), + "campaign", + ) + if data["schema_version"] != SCHEMA_VERSION: + msg = f"campaign schema_version {data['schema_version']!r} is not {SCHEMA_VERSION}" + raise EvidenceError(msg) + workloads_path, workloads_sha, workloads = _load_workloads(path, data) + if not isinstance(data["cells"], list) or not data["cells"]: + msg = "campaign lists no cells" + raise EvidenceError(msg) + cells: dict[str, dict[str, Any]] = {} + seen: dict[str, str] = {} + for raw in data["cells"]: + cell = _check_cell(raw, workloads) + if cell["id"] in cells: + msg = f"campaign lists cell id {cell['id']!r} twice" + raise EvidenceError(msg) + # Duplicates collapse only within one build mode and complete identity; a D=1 + # MPI-enabled cell never stands in for the MPI-off one: has_mpi is part of this key. + key = digest([cell["profile"], cell["node_id"], cell["identity"]]) + if key in seen: + msg = f"cell {cell['id']} duplicates cell {seen[key]} (same profile, node, identity)" + raise EvidenceError(msg) + seen[key] = cell["id"] + cells[cell["id"]] = cell + return Campaign( + path=path, + sha256=file_sha256(path), + workloads_path=workloads_path, + workloads_sha256=workloads_sha, + workloads=workloads, + cells=cells, + ) + + +# ----------------------------------------------------------------------------------- placement + + +def check_placement( + placement: object, + *, + arm: str, + cell_id: str, + binary_hash: str, + identity: dict[str, Any], +) -> dict[str, Any]: + """Validate external placement evidence for one cell/arm/binary. + + The runtime shape is fixed by the arm; the declared settings must resolve the campaign's + thread count through the shared preflight, and the observed shape must be the full team on + every rank of the campaign allocation. A limited team cannot certify a full-team cell. + """ + keys = ( + "schema_version", + "artifact_kind", + "arm", + "cell_id", + "binary_hash", + "identity", + ) + placement = _require( + placement, + (*keys, "runtime_shape", "declared", "observed"), + "placement evidence", + ) + what = f"placement evidence for {arm}/{cell_id}" + checks = { + "schema_version": SCHEMA_VERSION, + "artifact_kind": "placement", + "arm": arm, + "cell_id": cell_id, + "binary_hash": binary_hash, + "identity": identity, + "runtime_shape": ARM_SHAPES[arm], + } + for key, value in checks.items(): + if placement[key] != value: + msg = f"{what}: {key} does not match (got {placement[key]!r})" + raise EvidenceError(msg) + _check_declared_placement(placement["declared"], arm, identity, what) + _check_observed_placement(placement["observed"], identity, what) + return placement + + +def _check_declared_placement( + declared: object, arm: str, identity: dict[str, Any], what: str +) -> None: + """Require the declared launch shape and settings to resolve the campaign cell's shape.""" + from monoprop_bench_tools.preflight import ( # noqa: PLC0415 - optional at import + PreflightError, + declared_shape, + ) + + declared = _require( + declared, ("ranks", "threads", "cpu_allocation", "settings"), what + ) + for key in ("ranks", "threads", "cpu_allocation"): + if declared[key] != identity[key]: + msg = f"{what}: declared {key} {declared[key]!r} differs from {identity[key]!r}" + raise EvidenceError(msg) + if not isinstance(declared["settings"], dict): + msg = f"{what}: declared settings must be an object" + raise EvidenceError(msg) + try: + declared_shape( + ARM_SHAPES[arm], + declared["settings"], + ranks=identity["ranks"], + has_mpi=identity["has_mpi"], + expected_threads=identity["threads"], + ) + except PreflightError as exc: + msg = f"{what}: {exc}" + raise EvidenceError(msg) from exc + + +def _check_observed_placement( + observed: object, identity: dict[str, Any], what: str +) -> None: + """Require the observed shape to be the full team on every rank of the cell's allocation.""" + observed = _require( + observed, ("has_mpi", "ranks", "threads_per_rank", "cpu_allocation"), what + ) + for key in ("has_mpi", "ranks", "cpu_allocation"): + if observed[key] != identity[key] or type(observed[key]) is not type( + identity[key] + ): + msg = f"{what}: observed {key} {observed[key]!r} differs from {identity[key]!r}" + raise EvidenceError(msg) + teams = observed["threads_per_rank"] + if not isinstance(teams, list) or len(teams) != identity["ranks"]: + msg = f"{what}: threads_per_rank must list one team size per rank" + raise EvidenceError(msg) + if any(team != identity["threads"] for team in teams): + msg = ( + f"{what}: observed teams {teams} are not the full {identity['threads']}-thread " + "team; a limited-team diagnostic cannot certify this cell" + ) + raise EvidenceError(msg) + + +# ---------------------------------------------------------------------------- process context + + +@dataclass +class Context: + """The observed process: communicator, build mode, binary and declared shape.""" + + comm: Any + rank: int + size: int + has_mpi: bool + binary_path: str + binary_hash: str + declared: dict[str, Any] + cpu_allocation: str + placement_path: str + placement_sha256: str + + def barrier(self) -> None: + """Synchronize ranks (serial: no-op).""" + if self.comm is not None and self.size > 1: + self.comm.Barrier() + + def reduce(self, value: float, op: str) -> float: + """Reduce ``value`` over ranks with ``op`` (``sum``, ``max`` or ``min``).""" + if self.comm is None or self.size == 1: + return value + mpi = sys.modules["mpi4py.MPI"] + return self.comm.allreduce( + value, op={"sum": mpi.SUM, "max": mpi.MAX, "min": mpi.MIN}[op] + ) + + def gather(self, value: object) -> list[Any]: + """Gather ``value`` to rank 0 (other ranks receive ``[]``).""" + if self.comm is None or self.size == 1: + return [value] + gathered = self.comm.gather(value, root=0) + return gathered if gathered is not None else [] + + def spread(self, value: int) -> dict[str, int]: + """Return ``value`` reduced to its rank sum and maximum.""" + return { + "sum": int(self.reduce(value, "sum")), + "max": int(self.reduce(value, "max")), + } + + def all_true(self, *, flag: bool) -> bool: + """Return whether ``flag`` holds on every rank.""" + return self.reduce(int(flag), "min") == 1 + + +def _routing_from_env(ranks: int) -> dict[str, Any]: + """Return the routing identity the launch environment selects (engine defaults applied).""" + # The library's own variable names; unset or empty selects the engine defaults. + mode = os.environ.get("monoprop_ROUTING") or "linear" # noqa: SIM112 + seed_text = os.environ.get("monoprop_ROUTE_SEED") or str(DEFAULT_ROUTE_SEED) # noqa: SIM112 + if mode not in ROUTING_MODES or not seed_text.isdigit(): + msg = ( + f"invalid routing environment monoprop_ROUTING={mode!r} seed={seed_text!r}" + ) + raise EvidenceError(msg) + return expected_routing(mode, ranks, int(seed_text)) + + +def open_context(args: argparse.Namespace, cell: dict[str, Any]) -> Context: + """Import the extension, gate MPI on its build mode and check the declared shape. + + Raises: + EvidenceError: If the process contradicts the cell, its arm, or the placement evidence. + """ + from monoprop_bench_tools.preflight import ( # noqa: PLC0415 + PreflightError, + declared_shape, + import_mpi, + require_arm_shape, + settings_snapshot, + ) + + import monoprop # noqa: PLC0415 + + identity = cell["identity"] + try: + require_arm_shape(args.arm, args.runtime_shape) + except PreflightError as exc: + raise EvidenceError(str(exc)) from exc + has_mpi = getattr(monoprop, "has_mpi", None) + # Gated before any mpi4py import: an MPI-off process must never initialize MPI. + mpi = import_mpi(has_mpi=bool(has_mpi)) + comm = mpi.COMM_WORLD if mpi is not None else None + rank = 0 if comm is None else comm.Get_rank() + size = 1 if comm is None else comm.Get_size() + try: + declared = declared_shape( + args.runtime_shape, + os.environ, + ranks=size, + has_mpi=has_mpi, + expected_has_mpi=identity["has_mpi"], + expected_ranks=identity["ranks"], + expected_threads=identity["threads"], + ) + except PreflightError as exc: + raise EvidenceError(str(exc)) from exc + if _routing_from_env(size) != identity["routing"]: + msg = f"routing environment {_routing_from_env(size)} differs from {identity['routing']}" + raise EvidenceError(msg) + + binary = Path(monoprop._core.__file__) + binary_hash = file_sha256(binary) + local_cpus = sorted(os.sched_getaffinity(0)) + everyone = ( + local_cpus + if comm is None + else [c for cpus in comm.allgather(local_cpus) for c in cpus] + ) + cpu_allocation = format_cpu_list(everyone) + if cpu_allocation != identity["cpu_allocation"]: + msg = f"process allocation {cpu_allocation!r} differs from {identity['cpu_allocation']!r}" + raise EvidenceError(msg) + + placement_path = Path(args.placement).resolve() + placement = check_placement( + load_json(placement_path, "placement evidence"), + arm=args.arm, + cell_id=cell["id"], + binary_hash=binary_hash, + identity=identity, + ) + if placement["declared"]["settings"] != settings_snapshot(os.environ): + msg = "the launch settings differ from those the placement evidence declares" + raise EvidenceError(msg) + return Context( + comm=comm, + rank=rank, + size=size, + has_mpi=bool(has_mpi), + binary_path=str(binary), + binary_hash=binary_hash, + declared=declared, + cpu_allocation=cpu_allocation, + placement_path=str(placement_path), + placement_sha256=file_sha256(placement_path), + ) + + +def common_fields( + kind: str, + args: argparse.Namespace, + campaign: Campaign, + cell: dict[str, Any], + ctx: Context, +) -> dict[str, Any]: + """Return the provenance every artifact carries.""" + return { + "schema_version": SCHEMA_VERSION, + "artifact_kind": kind, + "run_id": f"{kind}-{uuid.uuid4().hex}", + "arm": args.arm, + "runtime_shape": args.runtime_shape, + "cell_id": cell["id"], + "campaign_path": str(campaign.path), + "campaign_sha256": campaign.sha256, + "workloads_sha256": campaign.workloads_sha256, + "profile": cell["profile"], + "node_id": cell["node_id"], + "operation": cell["operation"], + "measurement_kind": cell["measurement_kind"], + "identity": cell["identity"], + "config_digest": cell["config_digest"], + "parameters_digest": cell["parameters_digest"], + "binary_path": ctx.binary_path, + "binary_hash": ctx.binary_hash, + "has_mpi": ctx.has_mpi, + "observed": {"ranks": ctx.size, "cpu_allocation": ctx.cpu_allocation}, + "declared": ctx.declared, + "placement_path": ctx.placement_path, + "placement_sha256": ctx.placement_sha256, + "host": socket.gethostname(), + "driver_sha256": file_sha256(Path(__file__)), + } + + +def _write_exclusive(path: Path, value: dict[str, Any]) -> None: + """Create ``path`` with ``value``; an existing artifact is never overwritten.""" + path.parent.mkdir(parents=True, exist_ok=True) + with path.open("x") as handle: + json.dump(value, handle, indent=1, allow_nan=False) + handle.write("\n") + + +# -------------------------------------------------------------------------------- measurements + + +@dataclass +class Problem: + """A workload rebuilt with the existing model builders.""" + + make_propagator: Callable[[], tuple[Any, Any]] # () -> (propagator, circuit) + steps: int + parameters: list[float] + + +def build_problem(spec: NodeSpec, entry: dict[str, Any], comm: Any) -> Problem: # noqa: ANN401 + """Rebuild a workload's inputs and check its parameter vector against the manifest. + + Random inputs are generated here (before any propagator exists); fixed models construct their + circuit inside ``make_propagator``, exactly as the benchmark fixtures do. + """ + from monoprop_bench_tools.models import ( # noqa: PLC0415 + MODELS, + build_random_propagator, + make_random_problem, + ) + + config = entry["config"] + if spec.family == "random": + problem = make_random_problem( + **{k: v for k, v in config.items() if k != "lower_atol"} + ) + parameters = [float(p) for p in problem.parameters] + schrodinger = spec.picture == "schrodinger" + + def make() -> tuple[Any, Any]: + return build_random_propagator( + problem, + comm=comm, + lower_atol=config["lower_atol"], + schrodinger=schrodinger, + ) + + steps = 1 + else: + config_cls, build_fn, steps_fn = MODELS[spec.family] + model_config = config_cls(**config) + steps = int(steps_fn(model_config)) + + def make() -> tuple[Any, Any]: + return build_fn(model_config, comm=comm) + + # The builders return the circuit only beside its (single-term) propagator. + propagator, circuit = make() + parameters = [float(p) for p in circuit.parameters] * steps + del propagator, circuit + if parameters != [float(p) for p in entry["parameters"]]: + msg = f"{spec.node_id}: rebuilt parameters differ from the workload's parameter vector" + raise EvidenceError(msg) + return Problem(make_propagator=make, steps=steps, parameters=parameters) + + +def timed_pare_construct( + problem: Problem, + ctx: Context, + *, + window_factory: Callable[..., Any], + clock: Callable[[], float] = time.perf_counter, + events: list[str] | None = None, +) -> dict[str, Any]: + """Measure ``expectation_value_and_gradient_functional(1e-10)`` on a ready graph. + + The outer window (the driver analogue of pytest's ``record_memory``) opens after the random + inputs exist and before the propagator is built. The graph is complete before the operation + window, the entry barrier and the timer; the timer covers only the functional construction and + the completion barrier. The returned callable is never evaluated here and stays alive until + both windows have closed. + """ + log = events if events is not None else [] + outer = window_factory() + outer.start() + log.append("outer-open") + propagator, circuit = problem.make_propagator() + for _ in range(problem.steps): + propagator.build_graph(circuit) + log.append("graph-built") + operation = window_factory(settle=False) + operation.start() + log.append("operation-open") + ctx.barrier() # untimed entry barrier + start = clock() + log.append("timer-start") + functional = propagator.expectation_value_and_gradient_functional(PARE_THRESHOLD) + ctx.barrier() # completion is part of the measured makespan + elapsed = clock() - start + log.append("timer-stop") + operation.stop() + log.append("operation-close") + outer.stop() + log.append("outer-close") + runtime = ctx.reduce(elapsed, "max") + op_spread = { + "peak": ctx.spread(operation.peak_bytes), + "floor": ctx.spread(operation.baseline_bytes), + "delta": ctx.spread(operation.delta_bytes), + } + outer_spread = ctx.spread(outer.peak_bytes) + result = { + "runtime_seconds": runtime, + "peak_sum_bytes": op_spread["peak"]["sum"], + "peak_max_bytes": op_spread["peak"]["max"], + "op_exact": ctx.all_true(flag=operation.exact), + "outer_peak_sum_bytes": outer_spread["sum"], + "outer_peak_max_bytes": outer_spread["max"], + "outer_exact": ctx.all_true(flag=outer.exact), + "operation_floor_sum_bytes": op_spread["floor"]["sum"], + "operation_floor_max_bytes": op_spread["floor"]["max"], + "operation_delta_sum_bytes": op_spread["delta"]["sum"], + "operation_delta_max_bytes": op_spread["delta"]["max"], + "persistent_bytes": { + "operator": int( + ctx.reduce(propagator._simulator.operator_memory_bytes(), "sum") + ), + "graph": int(ctx.reduce(propagator._simulator.graph_memory_bytes(), "sum")), + }, + } + # Only now may the callable go: it had to outlive both windows. + del functional + log.append("callable-released") + return result + + +def construction_worker( + spec: NodeSpec, + entry: dict[str, Any], + ctx: Context, + *, + window_factory: Callable[..., Any], + build: Callable[[], Problem], + events: list[str] | None = None, +) -> dict[str, Any]: + """Measure one whole construction, inputs included, in an untimed fresh process. + + One window opens before the random inputs are generated; the propagator, graph and callable + are built and the operation runs; every output stays alive until the window closes. + """ + log = events if events is not None else [] + window = window_factory() + window.start() + log.append("window-open") + problem = build() + propagator, circuit = problem.make_propagator() + retained: list[Any] = [problem, propagator, circuit] + threshold = entry["pare_threshold"] + if spec.operation == "propagate": + for _ in range(problem.steps): + propagator.propagate(circuit) + else: + for _ in range(problem.steps): + propagator.build_graph(circuit) + if spec.operation == "energy": + functional = propagator.expectation_value_functional(threshold) + retained += [functional, functional(problem.parameters)] + elif spec.operation == "gradient": + functional = propagator.expectation_value_and_gradient_functional(threshold) + retained += [functional, functional(problem.parameters)] + elif spec.operation == "pare_functional_construct": + retained.append(propagator.expectation_value_and_gradient_functional(threshold)) + log.append("operation-done") + window.stop() + log.append("window-close") + peak, floor, delta = ( + ctx.spread(window.peak_bytes), + ctx.spread(window.baseline_bytes), + ctx.spread(window.delta_bytes), + ) + result = { + "construction_exact": ctx.all_true(flag=window.exact), + "construction_peak_sum_bytes": peak["sum"], + "construction_peak_max_bytes": peak["max"], + "construction_floor_sum_bytes": floor["sum"], + "construction_floor_max_bytes": floor["max"], + "construction_delta_sum_bytes": delta["sum"], + "construction_delta_max_bytes": delta["max"], + } + del retained + log.append("outputs-released") + return result + + +def timed_pytest( + spec: NodeSpec, + entry: dict[str, Any], + args: argparse.Namespace, + ctx: Context, + raw: Path, +) -> dict[str, Any]: + """Run one historical pytest node in this process and read back its raw artifacts.""" + import pytest # noqa: PLC0415 + + label = "timed" + raw.mkdir(parents=True, exist_ok=True) + os.environ["monoprop_BENCH_LABEL"] = label # noqa: SIM112 - the suite's names + os.environ["monoprop_BENCH_RESULTS"] = str(raw) # noqa: SIM112 + build_mode = "mpi" if ctx.has_mpi else "mpi-off" + time_json = raw / f"time-{label}.json" + argv = [ + spec.pytest_path, + "-p", + "no:cacheprovider", + "-o", + "filterwarnings=default", + "--bench-rounds=1", + f"--runtime-shape={args.runtime_shape}", + f"--build-mode={build_mode}", + *pytest_flags(spec, entry), + f"--benchmark-json={time_json}", + ] + cwd = Path.cwd() + os.chdir(REPO_ROOT) + try: + status = int(pytest.main(argv)) + finally: + os.chdir(cwd) + if status != 0: + msg = f"pytest exited {status} for {spec.node_id}; a skipped or failed node is not a pass" + raise EvidenceError(msg) + if ctx.rank != 0: + return {} + results = load_json(raw / f"{label}.json", "raw benchmark results") + timings = load_json(time_json, "raw benchmark timings") + benchmarks = [ + b for b in timings.get("benchmarks", []) if b["fullname"].endswith(spec.node_id) + ] + if len(benchmarks) != 1 or benchmarks[0]["stats"].get("rounds") != 1: + msg = f"raw timings do not hold exactly one single-round run of {spec.node_id}" + raise EvidenceError(msg) + node = spec.node_id + + def field(section: str) -> Any: # noqa: ANN401 - raw JSON value + if node not in results.get(section, {}): + msg = f"raw results lack {section}[{node}]" + raise EvidenceError(msg) + return results[section][node] + + peak, floor, delta = field("opmempeak"), field("opmembase"), field("opmemdelta") + return { + "runtime_seconds": benchmarks[0]["stats"]["mean"], + "peak_sum_bytes": peak["sum"], + "peak_max_bytes": peak["max"], + "op_exact": field("opmemexact"), + "outer_peak_sum_bytes": field("memhwm"), + "outer_peak_max_bytes": field("memhwm_max"), + "outer_exact": field("memhwmexact"), + "operation_floor_sum_bytes": floor["sum"], + "operation_floor_max_bytes": floor["max"], + "operation_delta_sum_bytes": delta["sum"], + "operation_delta_max_bytes": delta["max"], + "persistent_bytes": results.get("opbytes", {}).get(node), + "terms": results.get("opsize", {}).get(node, {}).get("terms"), + "raw_dir": str(raw), + "raw_meta": results.get("meta", {}), + } + + +def observe(args: argparse.Namespace) -> int: + """Run one timed or whole-construction observation of one campaign cell.""" + from monoprop_bench_tools.memory.cpu import HighWaterMark # noqa: PLC0415 + + campaign = load_campaign(Path(args.campaign)) + cell = campaign.cells.get(require_slug(args.cell_id, "cell id")) + if cell is None: + msg = f"campaign has no cell {args.cell_id!r}" + raise EvidenceError(msg) + sample = require_slug(args.sample_id, "sample id") + kind = "construction" if args.whole_process else "timed" + target = ( + Path(args.output).resolve() / args.arm / cell["id"] / sample / f"{kind}.json" + ) + if target.exists(): + msg = f"{target} exists; artifacts are never overwritten" + raise EvidenceError(msg) + ctx = open_context(args, cell) + common = common_fields(kind, args, campaign, cell, ctx) + common["sample_id"] = sample + entry = campaign.entry(cell) + spec = parse_node(cell["node_id"]) + started = time.time() + if args.whole_process: + fields = construction_worker( + spec, + entry, + ctx, + window_factory=HighWaterMark, + build=lambda: build_problem(spec, entry, ctx.comm), + ) + elif spec.measurement_kind == "driver": + problem = build_problem(spec, entry, ctx.comm) + fields = timed_pare_construct(problem, ctx, window_factory=HighWaterMark) + else: + fields = timed_pytest(spec, entry, args, ctx, target.parent / "timed") + if ctx.rank == 0: + _write_exclusive( + target, + { + **common, + **fields, + "started": started, + "wall_seconds": time.time() - started, + }, + ) + ctx.barrier() + return EXIT_PASS + + +# ---------------------------------------------------------------------------------- validation + + +def _sorted_items( + terms: dict[tuple[int, ...], complex], +) -> Iterator[tuple[tuple[int, ...], float, float]]: + """Yield ``(key, real, imag)`` in sorted key order, rejecting non-finite coefficients.""" + for key in sorted(terms): + value = complex(terms[key]) + if not (math.isfinite(value.real) and math.isfinite(value.imag)): + msg = f"non-finite coefficient for term {key}" + raise EvidenceError(msg) + yield key, value.real, value.imag + + +def _require_term_suffix(path: Path) -> None: + if not path.name.endswith(TERM_SUFFIX): + msg = f"term file {path} must be gzip-compressed JSON Lines ({TERM_SUFFIX})" + raise EvidenceError(msg) + + +def _write_terms(path: Path, terms: dict[tuple[int, ...], complex]) -> tuple[int, bool]: + """Write ``terms`` as sorted, gzip-compressed JSON Lines; return the count and identity flag. + + Lines are formatted directly rather than through ``json.dumps``: for finite floats ``%r`` is + the same shortest round-trip repr, so the decompressed text is identical and faster to write. + The gzip header carries no timestamp or file name, so equal maps give equal bytes. + """ + _require_term_suffix(path) + path.parent.mkdir(parents=True, exist_ok=True) + with ( + path.open("xb") as raw, + gzip.GzipFile( + filename="", fileobj=raw, mode="wb", compresslevel=TERM_GZIP_LEVEL, mtime=0 + ) as packed, + io.TextIOWrapper(packed, encoding="ascii", newline="\n") as handle, + ): + write = handle.write + for key, real, imag in _sorted_items(terms): + indices = ", ".join(map(str, key)) + write('{"key": [%s], "real": %r, "imag": %r}\n' % (indices, real, imag)) # noqa: UP031 + return len(terms), () in terms + + +def _evaluate( + spec: NodeSpec, + threshold: float | None, + problem: Problem, + propagator: Any, # noqa: ANN401 - either propagator front-end + circuit: object, +) -> tuple[dict[str, Any], list[float]]: + """Run the operation and return its scalar products and the export parameter vector.""" + if spec.operation == "propagate": + for _ in range(problem.steps): + propagator.propagate(circuit) + return {"energy": float(propagator.expectation_value())}, [] + for _ in range(problem.steps): + propagator.build_graph(circuit) + if spec.operation == "build_graph": + value, gradient = propagator.expectation_value_and_gradient(problem.parameters) + elif spec.operation == "energy": + energy = propagator.expectation_value_functional(threshold)(problem.parameters) + return {"energy": float(energy)}, problem.parameters + else: + functional = propagator.expectation_value_and_gradient_functional(threshold) + value, gradient = functional(problem.parameters) + return { + "energy": float(value), + "gradient": [float(g) for g in gradient], + }, problem.parameters + + +def _export_terms( + spec: NodeSpec, + propagator: Any, # noqa: ANN401 - either propagator front-end + parameters: list[float], + ctx: Context, + term_dir: Path, +) -> dict[str, Any]: + """Write this rank's decoded term map and return the gathered file list and global count.""" + terms = propagator._simulator.evolved_operator(list(parameters), 0.0) + path = term_dir / f"rank-{ctx.rank:05d}{TERM_SUFFIX}" + count, identity = _write_terms(path, terms) + del terms + gathered = ctx.gather((str(path), count, identity)) + if ctx.rank != 0: + return {} + counts = [c for _, c, _ in gathered] + # The binding adds the Heisenberg core term on every rank; it counts once globally. + copies = ( + sum(1 for *_, has in gathered if has) if spec.picture == "heisenberg" else 0 + ) + return { + "term_files": [p for p, _, _ in gathered], + "term_counts": counts, + "global_term_count": sum(counts) - max(0, copies - 1), + } + + +def _graph_replay_check( + entry: dict[str, Any], problem: Problem, ctx: Context, term_dir: Path +) -> dict[str, Any]: + """Compare graph-free propagation with a graph replay of the same inputs, per rank. + + Only meaningful without coefficient truncation, when both paths apply the cutoff alone. The + replayed map is streamed against this rank's already written term file, not written again. + """ + if entry["config"].get("lower_atol") is not None: + return {"applicable": False} + replay, circuit = problem.make_propagator() + for _ in range(problem.steps): + replay.build_graph(circuit) + replayed = replay._simulator.evolved_operator(list(problem.parameters), 0.0) + del replay + own = term_dir / f"rank-{ctx.rank:05d}{TERM_SUFFIX}" + local = compare_sorted_streams(_read_terms(own), _sorted_items(replayed)) + del replayed + return { + "applicable": True, + "ok": ctx.all_true(flag=local["ok"]), + "detail": ctx.gather(local), + } + + +def validation_products( + spec: NodeSpec, + entry: dict[str, Any], + problem: Problem, + ctx: Context, + term_dir: Path, +) -> dict[str, Any]: + """Compute the operation's numerical products at the recorded parameters.""" + threshold = entry["pare_threshold"] + unpared = threshold is None and spec.operation != "pare_functional_construct" + propagator, circuit = problem.make_propagator() + products, export_parameters = _evaluate( + spec, threshold, problem, propagator, circuit + ) + layers = int(propagator.graph_layers) + if ctx.reduce(layers, "max") != ctx.reduce(layers, "min"): + msg = "ranks disagree on the graph layer count" + raise EvidenceError(msg) + products |= {"graph_layers": layers, "term_files": []} + if unpared: + products |= _export_terms(spec, propagator, export_parameters, ctx, term_dir) + else: + products["global_term_count"] = int(ctx.reduce(propagator.size(), "sum")) + del propagator + if spec.operation == "propagate": + products["graph_replay_check"] = _graph_replay_check( + entry, problem, ctx, term_dir + ) + return products + + +def validate(args: argparse.Namespace) -> int: + """Rebuild a cell's workload in an untimed process and record its numerical products.""" + from monoprop_bench_tools.memory.cpu import HighWaterMark # noqa: PLC0415 + + campaign = load_campaign(Path(args.campaign)) + cell = campaign.cells.get(require_slug(args.cell_id, "cell id")) + if cell is None: + msg = f"campaign has no cell {args.cell_id!r}" + raise EvidenceError(msg) + ctx = open_context(args, cell) + common = common_fields("validation", args, campaign, cell, ctx) + run_id = ( + ctx.comm.bcast(common["run_id"], root=0) if ctx.size > 1 else common["run_id"] + ) + common["run_id"] = run_id + base = Path(args.output).resolve() / args.arm / cell["id"] + target = base / f"validation-{run_id}.json" + entry = campaign.entry(cell) + spec = parse_node(cell["node_id"]) + started = time.time() + with HighWaterMark() as window: + problem = build_problem(spec, entry, ctx.comm) + products = validation_products( + spec, entry, problem, ctx, base / f"{run_id}-terms" + ) + diagnostics = { + "wall_seconds": ctx.reduce(time.time() - started, "max"), + "peak_sum_bytes": int(ctx.reduce(window.peak_bytes, "sum")), + "peak_max_bytes": int(ctx.reduce(window.peak_bytes, "max")), + "exact": ctx.all_true(flag=window.exact), + } + if ctx.rank == 0: + artifact = { + **common, + "basis": entry["basis"], + "picture": entry["picture"], + "pare_threshold": entry["pare_threshold"], + "parameters": problem.parameters, + **products, + "diagnostics": diagnostics, + } + _write_exclusive(target, artifact) + ctx.barrier() + return EXIT_PASS + + +# ------------------------------------------------------------------------- numerical comparison + + +def _read_terms(path: Path) -> Iterator[tuple[tuple[int, ...], float, float]]: + """Yield ``(key, real, imag)`` from a sorted term file, rejecting disorder and bad values.""" + _require_term_suffix(path) + previous: tuple[int, ...] | None = None + try: + with gzip.open(path, "rt", encoding="ascii") as handle: + for number, line in enumerate(handle, 1): + try: + record = json.loads(line) + key = tuple(int(i) for i in record["key"]) + real, imag = float(record["real"]), float(record["imag"]) + except (ValueError, KeyError, TypeError) as exc: + msg = f"{path}:{number}: malformed term record" + raise EvidenceError(msg) from exc + if previous is not None and key <= previous: + msg = f"{path}:{number}: term keys are not strictly increasing" + raise EvidenceError(msg) + if not (math.isfinite(real) and math.isfinite(imag)): + msg = f"{path}:{number}: non-finite coefficient" + raise EvidenceError(msg) + previous = key + yield key, real, imag + except (OSError, EOFError, gzip.BadGzipFile) as exc: + msg = f"cannot read term file {path}: {exc}" + raise EvidenceError(msg) from exc + + +def merged_terms( + paths: list[str], picture: str +) -> Iterator[tuple[tuple[int, ...], float, float]]: + """Merge per-rank term files into the global map. + + Ownership is unique: a key on two ranks fails, except the Heisenberg identity/core term, which + the binding replicates on every rank; it is kept once after checking the copies agree. The + Schrödinger identity is an ordinary owned term. + """ + streams = [_read_terms(Path(p)) for p in paths] + merged = heapq.merge(*streams, key=lambda term: term[0]) + pending: list[tuple[tuple[int, ...], float, float]] = [] + + def flush() -> Iterator[tuple[tuple[int, ...], float, float]]: + if not pending: + return + key = pending[0][0] + if len(pending) > 1: + if key != () or picture != "heisenberg": + raise NumericalMismatchError( + f"term {list(key)} is owned by {len(pending)} ranks" + ) + real, imag = pending[0][1], pending[0][2] + if not all(near(r, real) and near(i, imag) for _, r, i in pending): + raise NumericalMismatchError( + "replicated Heisenberg identity copies disagree" + ) + yield pending[0] + + for term in merged: + if pending and term[0] != pending[0][0]: + yield from flush() + pending.clear() + pending.append(term) + yield from flush() + + +class NumericalMismatchError(Exception): + """The validation products of two arms disagree (a failed numerical check).""" + + +def compare_sorted_streams( + left: Iterable[tuple[tuple[int, ...], float, float]], + right: Iterable[tuple[tuple[int, ...], float, float]], +) -> dict[str, Any]: + """Compare two sorted term streams: equal key sets, componentwise ``near`` values.""" + missing = extra = mismatched = count = 0 + worst = 0.0 + first: list[str] = [] + a_iter, b_iter = iter(left), iter(right) + a, b = next(a_iter, None), next(b_iter, None) + while a is not None or b is not None: + if b is None or (a is not None and a[0] < b[0]): + missing += 1 + first.append(f"only in first: {list(a[0])}") + a = next(a_iter, None) + elif a is None or b[0] < a[0]: + extra += 1 + first.append(f"only in second: {list(b[0])}") + b = next(b_iter, None) + else: + count += 1 + diff = max(abs(a[1] - b[1]), abs(a[2] - b[2])) + worst = max(worst, diff) + if not (near(a[1], b[1]) and near(a[2], b[2])): + mismatched += 1 + first.append(f"value differs at {list(a[0])}") + a, b = next(a_iter, None), next(b_iter, None) + return { + "ok": missing == extra == mismatched == 0, + "common_terms": count, + "only_first": missing, + "only_second": extra, + "value_mismatches": mismatched, + "max_abs_diff": worst, + "examples": first[:5], + } + + +def _check_validation(artifact: dict[str, Any], what: str) -> None: + """Require a validation artifact's products to be present and finite.""" + operation = artifact["operation"] + pared = ( + artifact.get("pare_threshold") is not None + or operation == "pare_functional_construct" + ) + if not _finite_number(artifact.get("energy")): + msg = f"{what}: energy is missing or non-finite" + raise NumericalMismatchError(msg) + if operation in ("build_graph", "gradient", "pare_functional_construct"): + gradient = artifact.get("gradient") + if not isinstance(gradient, list) or not all( + _finite_number(g) for g in gradient + ): + msg = f"{what}: full gradient is missing or non-finite" + raise NumericalMismatchError(msg) + files = artifact.get("term_files") + if not isinstance(files, list): + msg = f"{what}: term_files must be a list" + raise EvidenceError(msg) + if pared and files: + # A pared callable owns a subgraph, not an independently exported operator. + msg = f"{what}: pared evidence must not carry term files" + raise EvidenceError(msg) + replay = artifact.get("graph_replay_check") + if replay is not None and replay.get("applicable") and not replay.get("ok"): + msg = f"{what}: graph replay disagrees with graph-free propagation" + raise NumericalMismatchError(msg) + + +def _compare_gradients(baseline: dict[str, Any], candidate: dict[str, Any]) -> int: + """Require equal-length, componentwise-near gradients; return their length.""" + left, right = baseline.get("gradient"), candidate.get("gradient") + if left is None or right is None or len(left) != len(right): + msg = "gradient missing or of different length" + raise NumericalMismatchError(msg) + bad = [ + i for i, (x, y) in enumerate(zip(left, right, strict=True)) if not near(x, y) + ] + if bad: + msg = f"gradient differs at parameters {bad[:5]}" + raise NumericalMismatchError(msg) + return len(left) + + +def _compare_term_maps( + baseline: dict[str, Any], candidate: dict[str, Any] +) -> dict[str, Any]: + """Merge each arm's per-rank term files and compare the two global maps.""" + if not (baseline["term_files"] and candidate["term_files"]): + msg = "only one arm exported a term map" + raise NumericalMismatchError(msg) + result = compare_sorted_streams( + merged_terms(baseline["term_files"], baseline["picture"]), + merged_terms(candidate["term_files"], candidate["picture"]), + ) + if not result["ok"]: + msg = f"global term maps differ: {result['examples']}" + raise NumericalMismatchError(msg) + for side, artifact in (("baseline", baseline), ("candidate", candidate)): + if artifact.get("global_term_count") != result["common_terms"]: + msg = f"{side} global_term_count disagrees with its merged term files" + raise NumericalMismatchError(msg) + return result + + +def compare_validations( + baseline: dict[str, Any], candidate: dict[str, Any] +) -> dict[str, Any]: + """Compare two arms' validation products of the identical operation. + + Both artifacts already joined on the same parameters digest. + """ + for side, artifact in (("baseline", baseline), ("candidate", candidate)): + _check_validation(artifact, f"{side} validation {artifact['run_id']}") + if not near(baseline["energy"], candidate["energy"]): + msg = f"energy {baseline['energy']!r} vs {candidate['energy']!r}" + raise NumericalMismatchError(msg) + detail: dict[str, Any] = {} + if "gradient" in baseline or "gradient" in candidate: + detail["gradient_length"] = _compare_gradients(baseline, candidate) + if baseline["term_files"] or candidate["term_files"]: + detail["terms"] = _compare_term_maps(baseline, candidate) + return {"ok": True, **detail} + + +# ------------------------------------------------------------------------------------- compare + + +_JOIN_FIELDS = ( + "arm", + "cell_id", + "campaign_sha256", + "binary_hash", + "profile", + "node_id", + "operation", + "identity", + "config_digest", + "parameters_digest", + "has_mpi", + "placement_sha256", +) + + +class _Evidence: + """Loads referenced artifacts once and enforces cross-sample uniqueness.""" + + def __init__(self) -> None: + self.cache: dict[Path, dict[str, Any]] = {} + self.measurement_paths: dict[Path, str] = {} + self.run_ids: dict[str, str] = {} + + def load(self, path_text: object, kind: str, owner: str) -> dict[str, Any]: + if not isinstance(path_text, str): + msg = f"{owner}: {kind} path must be a string" + raise EvidenceError(msg) + path = Path(path_text).resolve() + if kind in ("timed", "construction"): + if path in self.measurement_paths: + msg = ( + f"{owner}: {path} is already used by {self.measurement_paths[path]}" + ) + raise EvidenceError(msg) + self.measurement_paths[path] = owner + if path not in self.cache: + artifact = _require( + load_json(path, f"{kind} artifact"), ("artifact_kind",), str(path) + ) + if ( + artifact["artifact_kind"] != kind + or artifact.get("schema_version") != SCHEMA_VERSION + ): + msg = f"{path} is not a schema-{SCHEMA_VERSION} {kind} artifact" + raise EvidenceError(msg) + run_id = artifact.get("run_id") + if not isinstance(run_id, str) or not run_id: + msg = f"{path} has no run_id" + raise EvidenceError(msg) + if run_id in self.run_ids: + msg = f"{path} reuses run_id {run_id!r} of {self.run_ids[run_id]}" + raise EvidenceError(msg) + self.run_ids[run_id] = str(path) + self.cache[path] = artifact + return self.cache[path] + + +def _check_join( + artifact: dict[str, Any], + expected: dict[str, Any], + what: str, + placement_cache: dict[tuple[str, str], dict[str, Any]], +) -> None: + """Require an artifact to belong to exactly this cell/arm/binary/configuration.""" + for key in _JOIN_FIELDS: + if key not in artifact: + msg = f"{what} lacks {key}" + raise EvidenceError(msg) + if artifact[key] != expected[key]: + msg = f"{what}: {key} {artifact[key]!r} does not match {expected[key]!r}" + raise EvidenceError(msg) + path_text = artifact.get("placement_path") + if not isinstance(path_text, str): + msg = f"{what} lacks placement_path" + raise EvidenceError(msg) + key = (path_text, artifact["placement_sha256"]) + if key not in placement_cache: + path = Path(path_text) + if not path.is_file() or file_sha256(path) != artifact["placement_sha256"]: + msg = f"{what}: placement evidence {path} is missing or does not match its hash" + raise EvidenceError(msg) + placement_cache[key] = check_placement( + load_json(path, "placement evidence"), + arm=expected["arm"], + cell_id=expected["cell_id"], + binary_hash=expected["binary_hash"], + identity=expected["identity"], + ) + + +def _median(values: list[float]) -> float: + return float(statistics.median(values)) + + +def _measurement(artifact: dict[str, Any], field: str, what: str) -> float: + value = artifact.get(field) + if not _finite_number(value) or value <= 0: + msg = f"{what}: {field} is missing, non-finite or not positive" + raise EvidenceError(msg) + return float(value) + + +def compare(args: argparse.Namespace) -> int: + """Join evidence against the frozen inventory and apply the five parity gates.""" + output = Path(args.output) + report: dict[str, Any] = { + "schema_version": SCHEMA_VERSION, + "passed": False, + "failed_cells": [], + "missing_cells": [], + "incomplete_cells": {}, + "cells": {}, + "errors": [], + } + try: + status = _compare(Path(args.campaign), Path(args.manifest), report) + except EvidenceError as exc: + report["errors"].append(str(exc)) + status = EXIT_MALFORMED + report["passed"] = status == EXIT_PASS + output.write_text(json.dumps(report, indent=1, allow_nan=False) + "\n") + return status + + +def _load_manifest(path: Path, campaign: Campaign) -> dict[str, dict[str, Any]]: + """Return the manifest's cells by id, rejecting foreign, duplicate or unknown cells.""" + manifest = _require( + load_json(path, "manifest"), + ("schema_version", "campaign_sha256", "cells"), + "manifest", + ) + if manifest["schema_version"] != SCHEMA_VERSION: + msg = "manifest has an unsupported schema_version" + raise EvidenceError(msg) + if manifest["campaign_sha256"] != campaign.sha256: + msg = "manifest was assembled for a different campaign (digest mismatch)" + raise EvidenceError(msg) + if not isinstance(manifest["cells"], list): + msg = "manifest cells must be a list" + raise EvidenceError(msg) + offered: dict[str, dict[str, Any]] = {} + for raw in manifest["cells"]: + cell = _require(raw, ("id", *ARMS), "manifest cell") + if cell["id"] in offered: + msg = f"manifest lists cell {cell['id']!r} twice" + raise EvidenceError(msg) + if cell["id"] not in campaign.cells: + msg = f"manifest cell {cell['id']!r} is not in the frozen inventory" + raise EvidenceError(msg) + offered[cell["id"]] = cell + return offered + + +def _compare(campaign_path: Path, manifest_path: Path, report: dict[str, Any]) -> int: + campaign = load_campaign(campaign_path) + report["campaign_sha256"] = campaign.sha256 + offered = _load_manifest(manifest_path, campaign) + evidence = _Evidence() + placements: dict[tuple[str, str], dict[str, Any]] = {} + for cell_id, cell in campaign.cells.items(): + if cell_id not in offered: + report["missing_cells"].append(cell_id) + continue + result = _compare_cell(campaign, cell, offered[cell_id], evidence, placements) + report["cells"][cell_id] = result + if result.get("incomplete"): + report["incomplete_cells"][cell_id] = result["incomplete"] + elif not result["passed"]: + report["failed_cells"].append(cell_id) + unmet = ( + report["missing_cells"] or report["incomplete_cells"] or report["failed_cells"] + ) + return EXIT_UNMET if unmet else EXIT_PASS + + +def _join_arm( + campaign: Campaign, + cell: dict[str, Any], + arm: str, + samples: object, + evidence: _Evidence, + placements: dict[tuple[str, str], dict[str, Any]], +) -> dict[str, Any]: + """Load and join one arm's samples of one cell; return its timed/construction/validations.""" + cell_id = cell["id"] + if not isinstance(samples, list): + msg = f"manifest {cell_id}/{arm} samples must be a list" + raise EvidenceError(msg) + joined: dict[str, Any] = {"timed": [], "construction": [], "validations": {}} + binary_hash: str | None = None + sample_ids: set[str] = set() + for index, raw in enumerate(samples): + what = f"{cell_id}/{arm} sample {index}" + keys = ("sample_id", "timed_path", "construction_path", "validation_path") + sample = _require(raw, keys, what) + sample_id = require_slug(sample["sample_id"], f"{what} sample id") + if sample_id in sample_ids: + msg = f"{cell_id}/{arm} lists sample id {sample_id!r} twice" + raise EvidenceError(msg) + sample_ids.add(sample_id) + timed = evidence.load(sample["timed_path"], "timed", what) + built = evidence.load(sample["construction_path"], "construction", what) + validation = evidence.load(sample["validation_path"], "validation", what) + binary_hash = binary_hash or timed.get("binary_hash") + expected = { + "arm": arm, + "cell_id": cell_id, + "campaign_sha256": campaign.sha256, + "binary_hash": binary_hash, + "profile": cell["profile"], + "node_id": cell["node_id"], + "operation": cell["operation"], + "identity": cell["identity"], + "config_digest": cell["config_digest"], + "parameters_digest": cell["parameters_digest"], + "has_mpi": cell["identity"]["has_mpi"], + "placement_sha256": timed.get("placement_sha256"), + } + for artifact, kind in ((timed, "timed"), (built, "construction")): + _check_join(artifact, expected, f"{what} {kind}", placements) + if artifact.get("sample_id") != sample_id: + msg = f"{what} {kind} artifact belongs to sample {artifact.get('sample_id')!r}" + raise EvidenceError(msg) + # Validation is reusable across samples of the same cell/arm/binary/configuration only. + own = {**expected, "placement_sha256": validation.get("placement_sha256")} + _check_join(validation, own, f"{what} validation", placements) + joined["validations"][validation["run_id"]] = validation + joined["timed"].append(timed) + joined["construction"].append(built) + return joined + + +def _gates( + cell_id: str, per_arm: dict[str, dict[str, Any]] +) -> tuple[dict[str, Any], list[str]]: + """Return the five median-ratio gates and the reasons any of them, or exactness, fails.""" + reasons: list[str] = [] + for arm in ARMS: + for kind, flag in ( + ("timed", "op_exact"), + ("construction", "construction_exact"), + ): + flags = [artifact.get(flag) for artifact in per_arm[arm][kind]] + if any(f is not True for f in flags): + state = "unknown" if any(f is None for f in flags) else "false" + reasons.append(f"{arm} {flag} is {state} for some observations") + gates: dict[str, Any] = {} + for name, kind, field in GATES: + medians = { + arm: _median( + [ + _measurement(a, field, f"{cell_id}/{arm} {kind}") + for a in per_arm[arm][kind] + ] + ) + for arm in ARMS + } + ratio = medians["candidate"] / medians["baseline"] + gates[name] = { + "baseline_median": medians["baseline"], + "candidate_median": medians["candidate"], + "ratio": ratio, + "passed": ratio <= 1.0, + } + if ratio > 1.0: + reasons.append(f"{name} ratio {ratio:.4f} > 1.00") + return gates, reasons + + +def _diagnostics(per_arm: dict[str, dict[str, Any]]) -> dict[str, Any]: + """Return outer-peak ratios, labelled by the outer windows' own exactness; never gated.""" + flags = [a.get("outer_exact") for arm in ARMS for a in per_arm[arm]["timed"]] + unknown = "unknown" if any(f is None for f in flags) else "non-exact" + label = "exact" if all(f is True for f in flags) else unknown + diagnostics: dict[str, Any] = {} + for name, field in DIAGNOSTICS: + medians: dict[str, float | None] = {} + for arm in ARMS: + values = [a.get(field) for a in per_arm[arm]["timed"]] + finite = all(_finite_number(v) for v in values) + medians[arm] = _median([float(v) for v in values]) if finite else None + base, cand = medians["baseline"], medians["candidate"] + diagnostics[name] = { + "baseline_median": base, + "candidate_median": cand, + "ratio": cand / base if base and cand is not None else None, + "exactness": label, + } + return diagnostics + + +def _numerical(per_arm: dict[str, dict[str, Any]]) -> dict[str, Any]: + """Compare every baseline validation with every candidate validation of the cell.""" + pairs = [] + try: + for base in per_arm["baseline"]["validations"].values(): + for cand in per_arm["candidate"]["validations"].values(): + result = compare_validations(base, cand) + pairs.append( + {"baseline": base["run_id"], "candidate": cand["run_id"], **result} + ) + except NumericalMismatchError as exc: + return {"ok": False, "reason": str(exc)} + return {"ok": True, "pairs": pairs} + + +def _compare_cell( + campaign: Campaign, + cell: dict[str, Any], + offered: dict[str, Any], + evidence: _Evidence, + placements: dict[tuple[str, str], dict[str, Any]], +) -> dict[str, Any]: + """Join, gate and numerically compare one campaign cell.""" + per_arm = { + arm: _join_arm(campaign, cell, arm, offered[arm], evidence, placements) + for arm in ARMS + } + counts = {arm: len(per_arm[arm]["timed"]) for arm in ARMS} + result: dict[str, Any] = {"samples": counts, "passed": False} + allowed = (INITIAL_SAMPLES, REPEATED_SAMPLES) + if counts["baseline"] != counts["candidate"] or counts["baseline"] not in allowed: + result["incomplete"] = ( + f"needs {INITIAL_SAMPLES} (or {REPEATED_SAMPLES} after repetition) distinct paired " + f"observations per arm, got {counts}" + ) + return result + gates, reasons = _gates(cell["id"], per_arm) + numerical = _numerical(per_arm) + if not numerical["ok"]: + reasons.append(f"numerical mismatch: {numerical['reason']}") + ratio_failed = any(not gate["passed"] for gate in gates.values()) + first_round = counts["baseline"] == INITIAL_SAMPLES + result |= { + "gates": gates, + "diagnostics": _diagnostics(per_arm), + "numerical": numerical, + "needs_additional_samples": ( + REPEATED_SAMPLES - INITIAL_SAMPLES if ratio_failed and first_round else 0 + ), + "reasons": reasons, + "passed": not reasons, + } + return result + + +# ------------------------------------------------------------------------------------------ CLI + + +def build_parser() -> argparse.ArgumentParser: + """Return the driver's three-mode argument parser.""" + parser = argparse.ArgumentParser(description=__doc__.split("\n\n")[0]) + modes = parser.add_subparsers(dest="mode", required=True) + for name in ("observe", "validate"): + mode = modes.add_parser(name) + mode.add_argument("--arm", required=True, choices=ARMS) + mode.add_argument( + "--runtime-shape", required=True, choices=tuple(ARM_SHAPES.values()) + ) + mode.add_argument("--campaign", required=True) + mode.add_argument("--cell-id", required=True) + if name == "observe": + mode.add_argument("--sample-id", required=True) + mode.add_argument("--placement", required=True) + mode.add_argument("--output", required=True) + if name == "observe": + mode.add_argument("--whole-process", action="store_true") + mode = modes.add_parser("compare") + mode.add_argument("--campaign", required=True) + mode.add_argument("--manifest", required=True) + mode.add_argument("--output", required=True) + return parser + + +def main(argv: list[str] | None = None) -> int: + """Run one driver mode; malformed evidence exits 2.""" + args = build_parser().parse_args(argv) + if args.mode == "compare": + return compare(args) + try: + return observe(args) if args.mode == "observe" else validate(args) + except EvidenceError as exc: + print(f"benchmark-rank-local-openmp: {exc}", file=sys.stderr) # noqa: T201 + return EXIT_MALFORMED + + +if __name__ == "__main__": + raise SystemExit(main()) From 8fc256474009a93f017622ba9c414493b6dfd48d Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Roberto=20Di=20Remigio=20Eik=C3=A5s?= Date: Sat, 26 Sep 2026 19:54:00 +0000 Subject: [PATCH 08/64] =?UTF-8?q?docs(parallelism):=20=F0=9F=93=9D=20freez?= =?UTF-8?q?e=20the=20Task=201=20baseline=20campaign?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Record the owner's decisions after the unscored baseline pilot and freeze the campaign. - tools/rank-local-openmp-campaign.json (250 cells) and tools/rank-local-openmp-workloads.json are the owner-approved inventory: tiny/reference (+pared) across MPI 1x1, 1x96, 2x48 and MPI-off 1x96, reference routing cells at 3x32/4x24, and large (+pared) at MPI 1x96, 2x48 and MPI-off 1x96. - Term files are gzip-compressed JSON Lines instead of adding storage (4.4-16x smaller measured). - Hubbard graph cells use 2 Trotter steps within the size guard; the 29-step graph did not fit. - The measured VmHWM shortfall is accepted as documented measurement noise; gates stay <=1.00. - prek exempts the frozen workloads file, which carries full parameter vectors by design, from the large-file check. Assisted-by: Pi:claude-opus-5-5 --- .../plans/2026-09-18-rank-local-openmp.md | 64 ++++++++++++++----- prek.toml | 3 +- tools/rank-local-openmp-campaign.json | 1 + tools/rank-local-openmp-workloads.json | 1 + 4 files changed, 51 insertions(+), 18 deletions(-) create mode 100644 tools/rank-local-openmp-campaign.json create mode 100644 tools/rank-local-openmp-workloads.json diff --git a/docs/superpowers/plans/2026-09-18-rank-local-openmp.md b/docs/superpowers/plans/2026-09-18-rank-local-openmp.md index 417043bc..cb25648b 100644 --- a/docs/superpowers/plans/2026-09-18-rank-local-openmp.md +++ b/docs/superpowers/plans/2026-09-18-rank-local-openmp.md @@ -878,11 +878,12 @@ separate and pending. `artifact_kind="validation"` and its own run ID; record `schema_version`, `node_id`, `operation` (one of the five operation rows below), `config_digest`, `parameters_digest`, `binary_hash`, `basis`, `picture`, finite `energy`, full `gradient` when relevant, `global_term_count`, `graph_layers`, and `term_files` paths to decoded term/coefficient - files. Each sorted term file is JSON Lines with `{"key":[indices],"real":number,"imag":number}`; gradient arrays are - in optimizer parameter order. `pare_functional_construct` and pared evaluation artifacts omit term files because the - pared callable owns a subgraph rather than an independently exported operator; validate its scalar/full gradient - against the baseline pared callable. Use deterministic input parameters; a graph benchmark replays at those parameters - for validation. The required products are: + files. Each sorted term file is gzip-compressed JSON Lines (`.jsonl.gz`) with + `{"key":[indices],"real":number,"imag":number}`; gradient arrays are in optimizer parameter order. Uncompressed, + truncated or otherwise unreadable term files are malformed evidence. `pare_functional_construct` and pared evaluation + artifacts omit term files because the pared callable owns a subgraph rather than an independently exported operator; + validate its scalar/full gradient against the baseline pared callable. Use deterministic input parameters; a graph + benchmark replays at those parameters for validation. The required products are: | Operation | Validation products | | --- | --- | @@ -937,17 +938,20 @@ uv run --no-sync python tools/benchmark-rank-local-openmp.py observe \ coverage. - [ ] Export maps with the existing low-level `propagator._simulator.evolved_operator(parameters, 0.0)`; use an empty parameter vector after graph-free propagation and the repeated circuit parameters for graph replay. Sort raw encoded - index tuples and store real/imaginary coefficient components in per-rank files. These exported coefficients are - currently rounded to 1e-12; state that limitation and retain unrounded scalar/gradient checks. Stream-merge sorted - rank files when comparing global maps instead of requiring historical rank ownership. For Heisenberg, the binding adds - the replicated identity/core term on each rank: retain one copy after checking agreement, **do not sum it R times**. - Other duplicated nonidentity terms fail unique ownership; Schrödinger identity follows ordinary owner semantics. - Require equal global key sets, including zero-coefficient stored terms; compare values with the existing - `test_utils::near` rule from `cpp/tests/TestUtilities.h.in`: `abs(a-b) <= 1e-9 + 1e-7*max(abs(a),abs(b))`, - componentwise for complex values and gradients. Reject missing/non-finite outputs. Preserve all stricter existing - unit-test tolerances; this benchmark comparison does not weaken them. During pre-freeze calibration, export failures - may require an approved smaller equal-arm profile. After freeze they block acceptance, not permission to resize or - supply an unchecked `numerical_ok=true`. + index tuples and store real/imaginary coefficient components in per-rank gzip files: level 1, with no timestamp or + file name in the gzip header, so equal maps give equal bytes. The line format, sort order and rounding are unchanged + by compression. The owner approved compression instead of adding storage. On the pilot's term files it measured + 4.4-16x smaller, and the draft campaign needs about 24 GB per arm instead of about 122 GB on the host's 128 GB volume. + These exported coefficients are currently rounded to 1e-12; state that limitation and retain unrounded scalar/gradient + checks. Stream-merge sorted rank files when comparing global maps instead of requiring historical rank ownership. For + Heisenberg, the binding adds the replicated identity/core term on each rank: retain one copy after checking agreement, + **do not sum it R times**. Other duplicated nonidentity terms fail unique ownership; Schrödinger identity follows + ordinary owner semantics. Require equal global key sets, including zero-coefficient stored terms; compare values with + the existing `test_utils::near` rule from `cpp/tests/TestUtilities.h.in`: `abs(a-b) <= 1e-9 + + 1e-7*max(abs(a),abs(b))`, componentwise for complex values and gradients. Reject missing/non-finite outputs. Preserve + all stricter existing unit-test tolerances; this benchmark comparison does not weaken them. During pre-freeze + calibration, export failures may require an approved smaller equal-arm profile. After freeze they block acceptance, + not permission to resize or supply an unchecked `numerical_ok=true`. - [ ] Link every performance observation to a successful validation artifact for the same binary/configuration/operation and compare baseline/candidate validation artifacts. Add tests with wrong energy, a changed gradient entry, one missing/extra term, duplicate ownership, replicated identity, non-finite data, stale binary hash and absent @@ -1089,7 +1093,11 @@ configurations in artifacts, not just a profile label. (`MAX_GRAPH_STEPS=2`), so the 29-step Hubbard reference is intentionally a capacity decision: use `monoprop_BENCH_ALLOW_BIG_GRAPH=1` only on adequately provisioned hardware, or obtain approval for a fixed lower `--hubbard-trotter-steps` profile on both arms before formal baseline collection. A skipped cell is not a pass. Do not - silently omit graph-heavy cells or bypass the guard on an unverified allocation. + silently omit graph-heavy cells or bypass the guard on an unverified allocation. Owner decision after the pilot: + Hubbard graph cells (`build_graph`, `energy`, `gradient`) use 2 Trotter steps on smaller lattices, within the guard. + Hubbard `propagate` keeps its 29 steps. The 29-step graph did not fit this host: the pilot killed it above the + 75% memory target, with at least 147 GiB resident. It may be checked separately on a larger machine; that check is + not part of this campaign. - [ ] Select required geometries from observed physical cores, not vCPU/SMT-sibling counts. For D allocated domains with C_dom cores each, keep `(R,T)=(1,1),(1,C_dom),(D,1),(D,C_dom)` for small/medium profiles. These R=1 controls use one domain. Add `(2*D,floor(C_dom/2))` with disjoint cores when C_dom>=2; record unused cores. Preserve MPI routing @@ -1113,6 +1121,28 @@ configurations in artifacts, not just a profile label. gradients/provenance; it cannot replace the ownership-aware global-map comparator or independent exact/finite-difference tests. +**Frozen campaign (owner-approved 2026-09-25):** `tools/rank-local-openmp-campaign.json` (sha256 +`ffde870c77a66c8ce37fb2cc07b39abef976e9d7938b7cea59f43cb407de068a`) lists 250 cells over +`tools/rank-local-openmp-workloads.json` (sha256 `508b65f1f0eb2171fc4fbd79feba2381054c6dcdd38a614600f5b957ab814416`). +Neither file may be regenerated or shrunk to make observations pass. The unscored pilot and its size decisions are +archived outside the repository with the Task 1 handoff report; no pilot output is a formal sample. +- The observed host has one NUMA domain (D=1, C=C_dom=96). Within each build mode, L1 and the MPI-only control + coincide at 1x1, and the NUMA-local control, L2b and L2a coincide at 1x96. There is therefore no separate L2a + diagnostic inventory. MPI-off 1x96 cells stay separate and required. The single-thread control runs in the MPI + build. +- `tiny` and `reference` (plus their `-pared` variants) run at MPI 1x1, 1x96 and 2x48 and at MPI-off 1x96. + - Each band has 16 unpared nodes. The `-pared` variants cover energy/gradient at threshold 1e-10 for all four + families, plus `driver::pare_functional_construct` in both pictures. + - Random reference `propagate` and `build_graph`, in both pictures, add 3x32 splitmix, 4x24 splitmix and + 4x24 linear cells. +- `large` and `large-pared` run at MPI 1x96 and 2x48 and at MPI-off 1x96. Their sizes (13-68M terms) are set by the + cost of validation export, not by RAM. +- Owner decision on memory: the measured `VmHWM` shortfall is accepted as documented measurement noise for the + five gates. With threads on many CPUs, a window can miss up to about 50 MiB of a real transient (see + `docs/content/docs/benchmarks.mdx`, "Memory exactness"). The strict <=1.00 gates are not relaxed. +- The pilot killed one trial above the 75% memory target: the rejected 29-step Hubbard graph. The owner reviewed it; + no cell in the campaign needs that graph. + **Gate:** Calibration archive and owner-approved size-band inventory/digest are retained; benchmark tools' unit tests and baseline numerical suites pass. Fresh post-freeze baseline observations and placement evidence cover every required cell; no calibration sample is counted. Historical benchmark keys remain unchanged. No parity claim yet. diff --git a/prek.toml b/prek.toml index b20664e7..e8749aed 100644 --- a/prek.toml +++ b/prek.toml @@ -5,7 +5,8 @@ default_stages = ["pre-commit", "pre-push"] [[repos]] repo = "builtin" hooks = [ - { id = "check-added-large-files" }, + # The frozen parity workloads carry every node's full parameter vector by design (plan, Task 1). + { id = "check-added-large-files", exclude = "^tools/rank-local-openmp-workloads\\.json$" }, { id = "check-case-conflict" }, { id = "check-merge-conflict" }, { id = "check-symlinks" }, diff --git a/tools/rank-local-openmp-campaign.json b/tools/rank-local-openmp-campaign.json new file mode 100644 index 00000000..b2636857 --- /dev/null +++ b/tools/rank-local-openmp-campaign.json @@ -0,0 +1 @@ +{"cells":[{"config_digest":"fb5da6a0444ab95f40ba4e8446729f3139a2105f1a9eb4f7cc5a3d5e6e2afd21","id":"tiny--build-graph-heis--mpi-1x1","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":8,"num_modes":8,"obs_terms":16,"seed":0},"cpu_allocation":"0","has_mpi":true,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":1},"measurement_kind":"pytest","node_id":"bench_random.py::test_random_build_graph[heisenberg]","operation":"build_graph","parameters_digest":"4a68849272707590968775797bd7e6a107e692378ce2789cf5c8ffd5dd0f65f3","profile":"tiny"},{"config_digest":"fb5da6a0444ab95f40ba4e8446729f3139a2105f1a9eb4f7cc5a3d5e6e2afd21","id":"tiny--build-graph-heis--mpi-1x96","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":8,"num_modes":8,"obs_terms":16,"seed":0},"cpu_allocation":"0-95","has_mpi":true,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":96},"measurement_kind":"pytest","node_id":"bench_random.py::test_random_build_graph[heisenberg]","operation":"build_graph","parameters_digest":"4a68849272707590968775797bd7e6a107e692378ce2789cf5c8ffd5dd0f65f3","profile":"tiny"},{"config_digest":"fb5da6a0444ab95f40ba4e8446729f3139a2105f1a9eb4f7cc5a3d5e6e2afd21","id":"tiny--build-graph-heis--mpi-2x48","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":8,"num_modes":8,"obs_terms":16,"seed":0},"cpu_allocation":"0-95","has_mpi":true,"index_bits":32,"ranks":2,"routing":{"bits":1,"mode":"linear","seed":6768574230969066775},"threads":48},"measurement_kind":"pytest","node_id":"bench_random.py::test_random_build_graph[heisenberg]","operation":"build_graph","parameters_digest":"4a68849272707590968775797bd7e6a107e692378ce2789cf5c8ffd5dd0f65f3","profile":"tiny"},{"config_digest":"fb5da6a0444ab95f40ba4e8446729f3139a2105f1a9eb4f7cc5a3d5e6e2afd21","id":"tiny--build-graph-heis--off-1x96","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":8,"num_modes":8,"obs_terms":16,"seed":0},"cpu_allocation":"0-95","has_mpi":false,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":96},"measurement_kind":"pytest","node_id":"bench_random.py::test_random_build_graph[heisenberg]","operation":"build_graph","parameters_digest":"4a68849272707590968775797bd7e6a107e692378ce2789cf5c8ffd5dd0f65f3","profile":"tiny"},{"config_digest":"fb5da6a0444ab95f40ba4e8446729f3139a2105f1a9eb4f7cc5a3d5e6e2afd21","id":"tiny--build-graph-schr--mpi-1x1","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":8,"num_modes":8,"obs_terms":16,"seed":0},"cpu_allocation":"0","has_mpi":true,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":1},"measurement_kind":"pytest","node_id":"bench_random.py::test_random_build_graph[schrodinger]","operation":"build_graph","parameters_digest":"4a68849272707590968775797bd7e6a107e692378ce2789cf5c8ffd5dd0f65f3","profile":"tiny"},{"config_digest":"fb5da6a0444ab95f40ba4e8446729f3139a2105f1a9eb4f7cc5a3d5e6e2afd21","id":"tiny--build-graph-schr--mpi-1x96","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":8,"num_modes":8,"obs_terms":16,"seed":0},"cpu_allocation":"0-95","has_mpi":true,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":96},"measurement_kind":"pytest","node_id":"bench_random.py::test_random_build_graph[schrodinger]","operation":"build_graph","parameters_digest":"4a68849272707590968775797bd7e6a107e692378ce2789cf5c8ffd5dd0f65f3","profile":"tiny"},{"config_digest":"fb5da6a0444ab95f40ba4e8446729f3139a2105f1a9eb4f7cc5a3d5e6e2afd21","id":"tiny--build-graph-schr--mpi-2x48","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":8,"num_modes":8,"obs_terms":16,"seed":0},"cpu_allocation":"0-95","has_mpi":true,"index_bits":32,"ranks":2,"routing":{"bits":1,"mode":"linear","seed":6768574230969066775},"threads":48},"measurement_kind":"pytest","node_id":"bench_random.py::test_random_build_graph[schrodinger]","operation":"build_graph","parameters_digest":"4a68849272707590968775797bd7e6a107e692378ce2789cf5c8ffd5dd0f65f3","profile":"tiny"},{"config_digest":"fb5da6a0444ab95f40ba4e8446729f3139a2105f1a9eb4f7cc5a3d5e6e2afd21","id":"tiny--build-graph-schr--off-1x96","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":8,"num_modes":8,"obs_terms":16,"seed":0},"cpu_allocation":"0-95","has_mpi":false,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":96},"measurement_kind":"pytest","node_id":"bench_random.py::test_random_build_graph[schrodinger]","operation":"build_graph","parameters_digest":"4a68849272707590968775797bd7e6a107e692378ce2789cf5c8ffd5dd0f65f3","profile":"tiny"},{"config_digest":"251e899349b55ed62a6e51e2fad26381b0d2701d4431f0630732f6dad001ca58","id":"tiny--build-graph-hubb--mpi-1x1","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"chemical_potential":0.0,"cutoff":6,"hopping":1.0,"interaction":-2.0,"lower_atol":0.0001,"neel_start_spin":"down","num_sites":4,"observable_site":1,"observable_spin":"up","trotter_dt":0.2,"trotter_steps":2},"cpu_allocation":"0","has_mpi":true,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":1},"measurement_kind":"pytest","node_id":"bench_models.py::test_model_build_graph[hubbard]","operation":"build_graph","parameters_digest":"2742159d9e8aa1810c6cbf63993fd0612a3b5c62f68b31871ad0784764a60e20","profile":"tiny"},{"config_digest":"251e899349b55ed62a6e51e2fad26381b0d2701d4431f0630732f6dad001ca58","id":"tiny--build-graph-hubb--mpi-1x96","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"chemical_potential":0.0,"cutoff":6,"hopping":1.0,"interaction":-2.0,"lower_atol":0.0001,"neel_start_spin":"down","num_sites":4,"observable_site":1,"observable_spin":"up","trotter_dt":0.2,"trotter_steps":2},"cpu_allocation":"0-95","has_mpi":true,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":96},"measurement_kind":"pytest","node_id":"bench_models.py::test_model_build_graph[hubbard]","operation":"build_graph","parameters_digest":"2742159d9e8aa1810c6cbf63993fd0612a3b5c62f68b31871ad0784764a60e20","profile":"tiny"},{"config_digest":"251e899349b55ed62a6e51e2fad26381b0d2701d4431f0630732f6dad001ca58","id":"tiny--build-graph-hubb--mpi-2x48","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"chemical_potential":0.0,"cutoff":6,"hopping":1.0,"interaction":-2.0,"lower_atol":0.0001,"neel_start_spin":"down","num_sites":4,"observable_site":1,"observable_spin":"up","trotter_dt":0.2,"trotter_steps":2},"cpu_allocation":"0-95","has_mpi":true,"index_bits":32,"ranks":2,"routing":{"bits":1,"mode":"linear","seed":6768574230969066775},"threads":48},"measurement_kind":"pytest","node_id":"bench_models.py::test_model_build_graph[hubbard]","operation":"build_graph","parameters_digest":"2742159d9e8aa1810c6cbf63993fd0612a3b5c62f68b31871ad0784764a60e20","profile":"tiny"},{"config_digest":"251e899349b55ed62a6e51e2fad26381b0d2701d4431f0630732f6dad001ca58","id":"tiny--build-graph-hubb--off-1x96","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"chemical_potential":0.0,"cutoff":6,"hopping":1.0,"interaction":-2.0,"lower_atol":0.0001,"neel_start_spin":"down","num_sites":4,"observable_site":1,"observable_spin":"up","trotter_dt":0.2,"trotter_steps":2},"cpu_allocation":"0-95","has_mpi":false,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":96},"measurement_kind":"pytest","node_id":"bench_models.py::test_model_build_graph[hubbard]","operation":"build_graph","parameters_digest":"2742159d9e8aa1810c6cbf63993fd0612a3b5c62f68b31871ad0784764a60e20","profile":"tiny"},{"config_digest":"33f7cbf55f96127177eb5adb68bf2a4ae45a486da9b5ddb1dd7eb76fae4d8d5e","id":"tiny--build-graph-paul--mpi-1x1","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"coupling":0.7853981633974483,"cutoff":2,"lower_atol":0.1,"num_layers":1,"num_qubits":127,"observable_qubit":62,"theta":0.7853981633974483},"cpu_allocation":"0","has_mpi":true,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":1},"measurement_kind":"pytest","node_id":"bench_models.py::test_model_build_graph[pauli]","operation":"build_graph","parameters_digest":"70d31cd57471b39e5025b99cce04a23349f951ee380aa34fffb80cbe33626ab0","profile":"tiny"},{"config_digest":"33f7cbf55f96127177eb5adb68bf2a4ae45a486da9b5ddb1dd7eb76fae4d8d5e","id":"tiny--build-graph-paul--mpi-1x96","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"coupling":0.7853981633974483,"cutoff":2,"lower_atol":0.1,"num_layers":1,"num_qubits":127,"observable_qubit":62,"theta":0.7853981633974483},"cpu_allocation":"0-95","has_mpi":true,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":96},"measurement_kind":"pytest","node_id":"bench_models.py::test_model_build_graph[pauli]","operation":"build_graph","parameters_digest":"70d31cd57471b39e5025b99cce04a23349f951ee380aa34fffb80cbe33626ab0","profile":"tiny"},{"config_digest":"33f7cbf55f96127177eb5adb68bf2a4ae45a486da9b5ddb1dd7eb76fae4d8d5e","id":"tiny--build-graph-paul--mpi-2x48","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"coupling":0.7853981633974483,"cutoff":2,"lower_atol":0.1,"num_layers":1,"num_qubits":127,"observable_qubit":62,"theta":0.7853981633974483},"cpu_allocation":"0-95","has_mpi":true,"index_bits":32,"ranks":2,"routing":{"bits":1,"mode":"linear","seed":6768574230969066775},"threads":48},"measurement_kind":"pytest","node_id":"bench_models.py::test_model_build_graph[pauli]","operation":"build_graph","parameters_digest":"70d31cd57471b39e5025b99cce04a23349f951ee380aa34fffb80cbe33626ab0","profile":"tiny"},{"config_digest":"33f7cbf55f96127177eb5adb68bf2a4ae45a486da9b5ddb1dd7eb76fae4d8d5e","id":"tiny--build-graph-paul--off-1x96","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"coupling":0.7853981633974483,"cutoff":2,"lower_atol":0.1,"num_layers":1,"num_qubits":127,"observable_qubit":62,"theta":0.7853981633974483},"cpu_allocation":"0-95","has_mpi":false,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":96},"measurement_kind":"pytest","node_id":"bench_models.py::test_model_build_graph[pauli]","operation":"build_graph","parameters_digest":"70d31cd57471b39e5025b99cce04a23349f951ee380aa34fffb80cbe33626ab0","profile":"tiny"},{"config_digest":"fb5da6a0444ab95f40ba4e8446729f3139a2105f1a9eb4f7cc5a3d5e6e2afd21","id":"tiny--propagate-heis--mpi-1x1","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":8,"num_modes":8,"obs_terms":16,"seed":0},"cpu_allocation":"0","has_mpi":true,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":1},"measurement_kind":"pytest","node_id":"bench_random.py::test_random_propagate[heisenberg]","operation":"propagate","parameters_digest":"4a68849272707590968775797bd7e6a107e692378ce2789cf5c8ffd5dd0f65f3","profile":"tiny"},{"config_digest":"fb5da6a0444ab95f40ba4e8446729f3139a2105f1a9eb4f7cc5a3d5e6e2afd21","id":"tiny--propagate-heis--mpi-1x96","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":8,"num_modes":8,"obs_terms":16,"seed":0},"cpu_allocation":"0-95","has_mpi":true,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":96},"measurement_kind":"pytest","node_id":"bench_random.py::test_random_propagate[heisenberg]","operation":"propagate","parameters_digest":"4a68849272707590968775797bd7e6a107e692378ce2789cf5c8ffd5dd0f65f3","profile":"tiny"},{"config_digest":"fb5da6a0444ab95f40ba4e8446729f3139a2105f1a9eb4f7cc5a3d5e6e2afd21","id":"tiny--propagate-heis--mpi-2x48","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":8,"num_modes":8,"obs_terms":16,"seed":0},"cpu_allocation":"0-95","has_mpi":true,"index_bits":32,"ranks":2,"routing":{"bits":1,"mode":"linear","seed":6768574230969066775},"threads":48},"measurement_kind":"pytest","node_id":"bench_random.py::test_random_propagate[heisenberg]","operation":"propagate","parameters_digest":"4a68849272707590968775797bd7e6a107e692378ce2789cf5c8ffd5dd0f65f3","profile":"tiny"},{"config_digest":"fb5da6a0444ab95f40ba4e8446729f3139a2105f1a9eb4f7cc5a3d5e6e2afd21","id":"tiny--propagate-heis--off-1x96","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":8,"num_modes":8,"obs_terms":16,"seed":0},"cpu_allocation":"0-95","has_mpi":false,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":96},"measurement_kind":"pytest","node_id":"bench_random.py::test_random_propagate[heisenberg]","operation":"propagate","parameters_digest":"4a68849272707590968775797bd7e6a107e692378ce2789cf5c8ffd5dd0f65f3","profile":"tiny"},{"config_digest":"fb5da6a0444ab95f40ba4e8446729f3139a2105f1a9eb4f7cc5a3d5e6e2afd21","id":"tiny--propagate-schr--mpi-1x1","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":8,"num_modes":8,"obs_terms":16,"seed":0},"cpu_allocation":"0","has_mpi":true,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":1},"measurement_kind":"pytest","node_id":"bench_random.py::test_random_propagate[schrodinger]","operation":"propagate","parameters_digest":"4a68849272707590968775797bd7e6a107e692378ce2789cf5c8ffd5dd0f65f3","profile":"tiny"},{"config_digest":"fb5da6a0444ab95f40ba4e8446729f3139a2105f1a9eb4f7cc5a3d5e6e2afd21","id":"tiny--propagate-schr--mpi-1x96","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":8,"num_modes":8,"obs_terms":16,"seed":0},"cpu_allocation":"0-95","has_mpi":true,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":96},"measurement_kind":"pytest","node_id":"bench_random.py::test_random_propagate[schrodinger]","operation":"propagate","parameters_digest":"4a68849272707590968775797bd7e6a107e692378ce2789cf5c8ffd5dd0f65f3","profile":"tiny"},{"config_digest":"fb5da6a0444ab95f40ba4e8446729f3139a2105f1a9eb4f7cc5a3d5e6e2afd21","id":"tiny--propagate-schr--mpi-2x48","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":8,"num_modes":8,"obs_terms":16,"seed":0},"cpu_allocation":"0-95","has_mpi":true,"index_bits":32,"ranks":2,"routing":{"bits":1,"mode":"linear","seed":6768574230969066775},"threads":48},"measurement_kind":"pytest","node_id":"bench_random.py::test_random_propagate[schrodinger]","operation":"propagate","parameters_digest":"4a68849272707590968775797bd7e6a107e692378ce2789cf5c8ffd5dd0f65f3","profile":"tiny"},{"config_digest":"fb5da6a0444ab95f40ba4e8446729f3139a2105f1a9eb4f7cc5a3d5e6e2afd21","id":"tiny--propagate-schr--off-1x96","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":8,"num_modes":8,"obs_terms":16,"seed":0},"cpu_allocation":"0-95","has_mpi":false,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":96},"measurement_kind":"pytest","node_id":"bench_random.py::test_random_propagate[schrodinger]","operation":"propagate","parameters_digest":"4a68849272707590968775797bd7e6a107e692378ce2789cf5c8ffd5dd0f65f3","profile":"tiny"},{"config_digest":"251e899349b55ed62a6e51e2fad26381b0d2701d4431f0630732f6dad001ca58","id":"tiny--propagate-hubb--mpi-1x1","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"chemical_potential":0.0,"cutoff":6,"hopping":1.0,"interaction":-2.0,"lower_atol":0.0001,"neel_start_spin":"down","num_sites":4,"observable_site":1,"observable_spin":"up","trotter_dt":0.2,"trotter_steps":2},"cpu_allocation":"0","has_mpi":true,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":1},"measurement_kind":"pytest","node_id":"bench_models.py::test_model_propagate[hubbard]","operation":"propagate","parameters_digest":"2742159d9e8aa1810c6cbf63993fd0612a3b5c62f68b31871ad0784764a60e20","profile":"tiny"},{"config_digest":"251e899349b55ed62a6e51e2fad26381b0d2701d4431f0630732f6dad001ca58","id":"tiny--propagate-hubb--mpi-1x96","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"chemical_potential":0.0,"cutoff":6,"hopping":1.0,"interaction":-2.0,"lower_atol":0.0001,"neel_start_spin":"down","num_sites":4,"observable_site":1,"observable_spin":"up","trotter_dt":0.2,"trotter_steps":2},"cpu_allocation":"0-95","has_mpi":true,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":96},"measurement_kind":"pytest","node_id":"bench_models.py::test_model_propagate[hubbard]","operation":"propagate","parameters_digest":"2742159d9e8aa1810c6cbf63993fd0612a3b5c62f68b31871ad0784764a60e20","profile":"tiny"},{"config_digest":"251e899349b55ed62a6e51e2fad26381b0d2701d4431f0630732f6dad001ca58","id":"tiny--propagate-hubb--mpi-2x48","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"chemical_potential":0.0,"cutoff":6,"hopping":1.0,"interaction":-2.0,"lower_atol":0.0001,"neel_start_spin":"down","num_sites":4,"observable_site":1,"observable_spin":"up","trotter_dt":0.2,"trotter_steps":2},"cpu_allocation":"0-95","has_mpi":true,"index_bits":32,"ranks":2,"routing":{"bits":1,"mode":"linear","seed":6768574230969066775},"threads":48},"measurement_kind":"pytest","node_id":"bench_models.py::test_model_propagate[hubbard]","operation":"propagate","parameters_digest":"2742159d9e8aa1810c6cbf63993fd0612a3b5c62f68b31871ad0784764a60e20","profile":"tiny"},{"config_digest":"251e899349b55ed62a6e51e2fad26381b0d2701d4431f0630732f6dad001ca58","id":"tiny--propagate-hubb--off-1x96","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"chemical_potential":0.0,"cutoff":6,"hopping":1.0,"interaction":-2.0,"lower_atol":0.0001,"neel_start_spin":"down","num_sites":4,"observable_site":1,"observable_spin":"up","trotter_dt":0.2,"trotter_steps":2},"cpu_allocation":"0-95","has_mpi":false,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":96},"measurement_kind":"pytest","node_id":"bench_models.py::test_model_propagate[hubbard]","operation":"propagate","parameters_digest":"2742159d9e8aa1810c6cbf63993fd0612a3b5c62f68b31871ad0784764a60e20","profile":"tiny"},{"config_digest":"33f7cbf55f96127177eb5adb68bf2a4ae45a486da9b5ddb1dd7eb76fae4d8d5e","id":"tiny--propagate-paul--mpi-1x1","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"coupling":0.7853981633974483,"cutoff":2,"lower_atol":0.1,"num_layers":1,"num_qubits":127,"observable_qubit":62,"theta":0.7853981633974483},"cpu_allocation":"0","has_mpi":true,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":1},"measurement_kind":"pytest","node_id":"bench_models.py::test_model_propagate[pauli]","operation":"propagate","parameters_digest":"70d31cd57471b39e5025b99cce04a23349f951ee380aa34fffb80cbe33626ab0","profile":"tiny"},{"config_digest":"33f7cbf55f96127177eb5adb68bf2a4ae45a486da9b5ddb1dd7eb76fae4d8d5e","id":"tiny--propagate-paul--mpi-1x96","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"coupling":0.7853981633974483,"cutoff":2,"lower_atol":0.1,"num_layers":1,"num_qubits":127,"observable_qubit":62,"theta":0.7853981633974483},"cpu_allocation":"0-95","has_mpi":true,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":96},"measurement_kind":"pytest","node_id":"bench_models.py::test_model_propagate[pauli]","operation":"propagate","parameters_digest":"70d31cd57471b39e5025b99cce04a23349f951ee380aa34fffb80cbe33626ab0","profile":"tiny"},{"config_digest":"33f7cbf55f96127177eb5adb68bf2a4ae45a486da9b5ddb1dd7eb76fae4d8d5e","id":"tiny--propagate-paul--mpi-2x48","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"coupling":0.7853981633974483,"cutoff":2,"lower_atol":0.1,"num_layers":1,"num_qubits":127,"observable_qubit":62,"theta":0.7853981633974483},"cpu_allocation":"0-95","has_mpi":true,"index_bits":32,"ranks":2,"routing":{"bits":1,"mode":"linear","seed":6768574230969066775},"threads":48},"measurement_kind":"pytest","node_id":"bench_models.py::test_model_propagate[pauli]","operation":"propagate","parameters_digest":"70d31cd57471b39e5025b99cce04a23349f951ee380aa34fffb80cbe33626ab0","profile":"tiny"},{"config_digest":"33f7cbf55f96127177eb5adb68bf2a4ae45a486da9b5ddb1dd7eb76fae4d8d5e","id":"tiny--propagate-paul--off-1x96","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"coupling":0.7853981633974483,"cutoff":2,"lower_atol":0.1,"num_layers":1,"num_qubits":127,"observable_qubit":62,"theta":0.7853981633974483},"cpu_allocation":"0-95","has_mpi":false,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":96},"measurement_kind":"pytest","node_id":"bench_models.py::test_model_propagate[pauli]","operation":"propagate","parameters_digest":"70d31cd57471b39e5025b99cce04a23349f951ee380aa34fffb80cbe33626ab0","profile":"tiny"},{"config_digest":"fb5da6a0444ab95f40ba4e8446729f3139a2105f1a9eb4f7cc5a3d5e6e2afd21","id":"tiny--energy-heis--mpi-1x1","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":8,"num_modes":8,"obs_terms":16,"seed":0},"cpu_allocation":"0","has_mpi":true,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":1},"measurement_kind":"pytest","node_id":"bench_random.py::test_random_energy[heisenberg]","operation":"energy","parameters_digest":"4a68849272707590968775797bd7e6a107e692378ce2789cf5c8ffd5dd0f65f3","profile":"tiny"},{"config_digest":"fb5da6a0444ab95f40ba4e8446729f3139a2105f1a9eb4f7cc5a3d5e6e2afd21","id":"tiny--energy-heis--mpi-1x96","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":8,"num_modes":8,"obs_terms":16,"seed":0},"cpu_allocation":"0-95","has_mpi":true,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":96},"measurement_kind":"pytest","node_id":"bench_random.py::test_random_energy[heisenberg]","operation":"energy","parameters_digest":"4a68849272707590968775797bd7e6a107e692378ce2789cf5c8ffd5dd0f65f3","profile":"tiny"},{"config_digest":"fb5da6a0444ab95f40ba4e8446729f3139a2105f1a9eb4f7cc5a3d5e6e2afd21","id":"tiny--energy-heis--mpi-2x48","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":8,"num_modes":8,"obs_terms":16,"seed":0},"cpu_allocation":"0-95","has_mpi":true,"index_bits":32,"ranks":2,"routing":{"bits":1,"mode":"linear","seed":6768574230969066775},"threads":48},"measurement_kind":"pytest","node_id":"bench_random.py::test_random_energy[heisenberg]","operation":"energy","parameters_digest":"4a68849272707590968775797bd7e6a107e692378ce2789cf5c8ffd5dd0f65f3","profile":"tiny"},{"config_digest":"fb5da6a0444ab95f40ba4e8446729f3139a2105f1a9eb4f7cc5a3d5e6e2afd21","id":"tiny--energy-heis--off-1x96","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":8,"num_modes":8,"obs_terms":16,"seed":0},"cpu_allocation":"0-95","has_mpi":false,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":96},"measurement_kind":"pytest","node_id":"bench_random.py::test_random_energy[heisenberg]","operation":"energy","parameters_digest":"4a68849272707590968775797bd7e6a107e692378ce2789cf5c8ffd5dd0f65f3","profile":"tiny"},{"config_digest":"fb5da6a0444ab95f40ba4e8446729f3139a2105f1a9eb4f7cc5a3d5e6e2afd21","id":"tiny--energy-schr--mpi-1x1","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":8,"num_modes":8,"obs_terms":16,"seed":0},"cpu_allocation":"0","has_mpi":true,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":1},"measurement_kind":"pytest","node_id":"bench_random.py::test_random_energy[schrodinger]","operation":"energy","parameters_digest":"4a68849272707590968775797bd7e6a107e692378ce2789cf5c8ffd5dd0f65f3","profile":"tiny"},{"config_digest":"fb5da6a0444ab95f40ba4e8446729f3139a2105f1a9eb4f7cc5a3d5e6e2afd21","id":"tiny--energy-schr--mpi-1x96","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":8,"num_modes":8,"obs_terms":16,"seed":0},"cpu_allocation":"0-95","has_mpi":true,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":96},"measurement_kind":"pytest","node_id":"bench_random.py::test_random_energy[schrodinger]","operation":"energy","parameters_digest":"4a68849272707590968775797bd7e6a107e692378ce2789cf5c8ffd5dd0f65f3","profile":"tiny"},{"config_digest":"fb5da6a0444ab95f40ba4e8446729f3139a2105f1a9eb4f7cc5a3d5e6e2afd21","id":"tiny--energy-schr--mpi-2x48","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":8,"num_modes":8,"obs_terms":16,"seed":0},"cpu_allocation":"0-95","has_mpi":true,"index_bits":32,"ranks":2,"routing":{"bits":1,"mode":"linear","seed":6768574230969066775},"threads":48},"measurement_kind":"pytest","node_id":"bench_random.py::test_random_energy[schrodinger]","operation":"energy","parameters_digest":"4a68849272707590968775797bd7e6a107e692378ce2789cf5c8ffd5dd0f65f3","profile":"tiny"},{"config_digest":"fb5da6a0444ab95f40ba4e8446729f3139a2105f1a9eb4f7cc5a3d5e6e2afd21","id":"tiny--energy-schr--off-1x96","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":8,"num_modes":8,"obs_terms":16,"seed":0},"cpu_allocation":"0-95","has_mpi":false,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":96},"measurement_kind":"pytest","node_id":"bench_random.py::test_random_energy[schrodinger]","operation":"energy","parameters_digest":"4a68849272707590968775797bd7e6a107e692378ce2789cf5c8ffd5dd0f65f3","profile":"tiny"},{"config_digest":"251e899349b55ed62a6e51e2fad26381b0d2701d4431f0630732f6dad001ca58","id":"tiny--energy-hubb--mpi-1x1","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"chemical_potential":0.0,"cutoff":6,"hopping":1.0,"interaction":-2.0,"lower_atol":0.0001,"neel_start_spin":"down","num_sites":4,"observable_site":1,"observable_spin":"up","trotter_dt":0.2,"trotter_steps":2},"cpu_allocation":"0","has_mpi":true,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":1},"measurement_kind":"pytest","node_id":"bench_models.py::test_model_energy[hubbard]","operation":"energy","parameters_digest":"2742159d9e8aa1810c6cbf63993fd0612a3b5c62f68b31871ad0784764a60e20","profile":"tiny"},{"config_digest":"251e899349b55ed62a6e51e2fad26381b0d2701d4431f0630732f6dad001ca58","id":"tiny--energy-hubb--mpi-1x96","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"chemical_potential":0.0,"cutoff":6,"hopping":1.0,"interaction":-2.0,"lower_atol":0.0001,"neel_start_spin":"down","num_sites":4,"observable_site":1,"observable_spin":"up","trotter_dt":0.2,"trotter_steps":2},"cpu_allocation":"0-95","has_mpi":true,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":96},"measurement_kind":"pytest","node_id":"bench_models.py::test_model_energy[hubbard]","operation":"energy","parameters_digest":"2742159d9e8aa1810c6cbf63993fd0612a3b5c62f68b31871ad0784764a60e20","profile":"tiny"},{"config_digest":"251e899349b55ed62a6e51e2fad26381b0d2701d4431f0630732f6dad001ca58","id":"tiny--energy-hubb--mpi-2x48","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"chemical_potential":0.0,"cutoff":6,"hopping":1.0,"interaction":-2.0,"lower_atol":0.0001,"neel_start_spin":"down","num_sites":4,"observable_site":1,"observable_spin":"up","trotter_dt":0.2,"trotter_steps":2},"cpu_allocation":"0-95","has_mpi":true,"index_bits":32,"ranks":2,"routing":{"bits":1,"mode":"linear","seed":6768574230969066775},"threads":48},"measurement_kind":"pytest","node_id":"bench_models.py::test_model_energy[hubbard]","operation":"energy","parameters_digest":"2742159d9e8aa1810c6cbf63993fd0612a3b5c62f68b31871ad0784764a60e20","profile":"tiny"},{"config_digest":"251e899349b55ed62a6e51e2fad26381b0d2701d4431f0630732f6dad001ca58","id":"tiny--energy-hubb--off-1x96","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"chemical_potential":0.0,"cutoff":6,"hopping":1.0,"interaction":-2.0,"lower_atol":0.0001,"neel_start_spin":"down","num_sites":4,"observable_site":1,"observable_spin":"up","trotter_dt":0.2,"trotter_steps":2},"cpu_allocation":"0-95","has_mpi":false,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":96},"measurement_kind":"pytest","node_id":"bench_models.py::test_model_energy[hubbard]","operation":"energy","parameters_digest":"2742159d9e8aa1810c6cbf63993fd0612a3b5c62f68b31871ad0784764a60e20","profile":"tiny"},{"config_digest":"33f7cbf55f96127177eb5adb68bf2a4ae45a486da9b5ddb1dd7eb76fae4d8d5e","id":"tiny--energy-paul--mpi-1x1","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"coupling":0.7853981633974483,"cutoff":2,"lower_atol":0.1,"num_layers":1,"num_qubits":127,"observable_qubit":62,"theta":0.7853981633974483},"cpu_allocation":"0","has_mpi":true,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":1},"measurement_kind":"pytest","node_id":"bench_models.py::test_model_energy[pauli]","operation":"energy","parameters_digest":"70d31cd57471b39e5025b99cce04a23349f951ee380aa34fffb80cbe33626ab0","profile":"tiny"},{"config_digest":"33f7cbf55f96127177eb5adb68bf2a4ae45a486da9b5ddb1dd7eb76fae4d8d5e","id":"tiny--energy-paul--mpi-1x96","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"coupling":0.7853981633974483,"cutoff":2,"lower_atol":0.1,"num_layers":1,"num_qubits":127,"observable_qubit":62,"theta":0.7853981633974483},"cpu_allocation":"0-95","has_mpi":true,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":96},"measurement_kind":"pytest","node_id":"bench_models.py::test_model_energy[pauli]","operation":"energy","parameters_digest":"70d31cd57471b39e5025b99cce04a23349f951ee380aa34fffb80cbe33626ab0","profile":"tiny"},{"config_digest":"33f7cbf55f96127177eb5adb68bf2a4ae45a486da9b5ddb1dd7eb76fae4d8d5e","id":"tiny--energy-paul--mpi-2x48","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"coupling":0.7853981633974483,"cutoff":2,"lower_atol":0.1,"num_layers":1,"num_qubits":127,"observable_qubit":62,"theta":0.7853981633974483},"cpu_allocation":"0-95","has_mpi":true,"index_bits":32,"ranks":2,"routing":{"bits":1,"mode":"linear","seed":6768574230969066775},"threads":48},"measurement_kind":"pytest","node_id":"bench_models.py::test_model_energy[pauli]","operation":"energy","parameters_digest":"70d31cd57471b39e5025b99cce04a23349f951ee380aa34fffb80cbe33626ab0","profile":"tiny"},{"config_digest":"33f7cbf55f96127177eb5adb68bf2a4ae45a486da9b5ddb1dd7eb76fae4d8d5e","id":"tiny--energy-paul--off-1x96","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"coupling":0.7853981633974483,"cutoff":2,"lower_atol":0.1,"num_layers":1,"num_qubits":127,"observable_qubit":62,"theta":0.7853981633974483},"cpu_allocation":"0-95","has_mpi":false,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":96},"measurement_kind":"pytest","node_id":"bench_models.py::test_model_energy[pauli]","operation":"energy","parameters_digest":"70d31cd57471b39e5025b99cce04a23349f951ee380aa34fffb80cbe33626ab0","profile":"tiny"},{"config_digest":"fb5da6a0444ab95f40ba4e8446729f3139a2105f1a9eb4f7cc5a3d5e6e2afd21","id":"tiny--gradient-heis--mpi-1x1","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":8,"num_modes":8,"obs_terms":16,"seed":0},"cpu_allocation":"0","has_mpi":true,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":1},"measurement_kind":"pytest","node_id":"bench_random.py::test_random_gradient[heisenberg]","operation":"gradient","parameters_digest":"4a68849272707590968775797bd7e6a107e692378ce2789cf5c8ffd5dd0f65f3","profile":"tiny"},{"config_digest":"fb5da6a0444ab95f40ba4e8446729f3139a2105f1a9eb4f7cc5a3d5e6e2afd21","id":"tiny--gradient-heis--mpi-1x96","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":8,"num_modes":8,"obs_terms":16,"seed":0},"cpu_allocation":"0-95","has_mpi":true,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":96},"measurement_kind":"pytest","node_id":"bench_random.py::test_random_gradient[heisenberg]","operation":"gradient","parameters_digest":"4a68849272707590968775797bd7e6a107e692378ce2789cf5c8ffd5dd0f65f3","profile":"tiny"},{"config_digest":"fb5da6a0444ab95f40ba4e8446729f3139a2105f1a9eb4f7cc5a3d5e6e2afd21","id":"tiny--gradient-heis--mpi-2x48","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":8,"num_modes":8,"obs_terms":16,"seed":0},"cpu_allocation":"0-95","has_mpi":true,"index_bits":32,"ranks":2,"routing":{"bits":1,"mode":"linear","seed":6768574230969066775},"threads":48},"measurement_kind":"pytest","node_id":"bench_random.py::test_random_gradient[heisenberg]","operation":"gradient","parameters_digest":"4a68849272707590968775797bd7e6a107e692378ce2789cf5c8ffd5dd0f65f3","profile":"tiny"},{"config_digest":"fb5da6a0444ab95f40ba4e8446729f3139a2105f1a9eb4f7cc5a3d5e6e2afd21","id":"tiny--gradient-heis--off-1x96","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":8,"num_modes":8,"obs_terms":16,"seed":0},"cpu_allocation":"0-95","has_mpi":false,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":96},"measurement_kind":"pytest","node_id":"bench_random.py::test_random_gradient[heisenberg]","operation":"gradient","parameters_digest":"4a68849272707590968775797bd7e6a107e692378ce2789cf5c8ffd5dd0f65f3","profile":"tiny"},{"config_digest":"fb5da6a0444ab95f40ba4e8446729f3139a2105f1a9eb4f7cc5a3d5e6e2afd21","id":"tiny--gradient-schr--mpi-1x1","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":8,"num_modes":8,"obs_terms":16,"seed":0},"cpu_allocation":"0","has_mpi":true,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":1},"measurement_kind":"pytest","node_id":"bench_random.py::test_random_gradient[schrodinger]","operation":"gradient","parameters_digest":"4a68849272707590968775797bd7e6a107e692378ce2789cf5c8ffd5dd0f65f3","profile":"tiny"},{"config_digest":"fb5da6a0444ab95f40ba4e8446729f3139a2105f1a9eb4f7cc5a3d5e6e2afd21","id":"tiny--gradient-schr--mpi-1x96","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":8,"num_modes":8,"obs_terms":16,"seed":0},"cpu_allocation":"0-95","has_mpi":true,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":96},"measurement_kind":"pytest","node_id":"bench_random.py::test_random_gradient[schrodinger]","operation":"gradient","parameters_digest":"4a68849272707590968775797bd7e6a107e692378ce2789cf5c8ffd5dd0f65f3","profile":"tiny"},{"config_digest":"fb5da6a0444ab95f40ba4e8446729f3139a2105f1a9eb4f7cc5a3d5e6e2afd21","id":"tiny--gradient-schr--mpi-2x48","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":8,"num_modes":8,"obs_terms":16,"seed":0},"cpu_allocation":"0-95","has_mpi":true,"index_bits":32,"ranks":2,"routing":{"bits":1,"mode":"linear","seed":6768574230969066775},"threads":48},"measurement_kind":"pytest","node_id":"bench_random.py::test_random_gradient[schrodinger]","operation":"gradient","parameters_digest":"4a68849272707590968775797bd7e6a107e692378ce2789cf5c8ffd5dd0f65f3","profile":"tiny"},{"config_digest":"fb5da6a0444ab95f40ba4e8446729f3139a2105f1a9eb4f7cc5a3d5e6e2afd21","id":"tiny--gradient-schr--off-1x96","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":8,"num_modes":8,"obs_terms":16,"seed":0},"cpu_allocation":"0-95","has_mpi":false,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":96},"measurement_kind":"pytest","node_id":"bench_random.py::test_random_gradient[schrodinger]","operation":"gradient","parameters_digest":"4a68849272707590968775797bd7e6a107e692378ce2789cf5c8ffd5dd0f65f3","profile":"tiny"},{"config_digest":"251e899349b55ed62a6e51e2fad26381b0d2701d4431f0630732f6dad001ca58","id":"tiny--gradient-hubb--mpi-1x1","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"chemical_potential":0.0,"cutoff":6,"hopping":1.0,"interaction":-2.0,"lower_atol":0.0001,"neel_start_spin":"down","num_sites":4,"observable_site":1,"observable_spin":"up","trotter_dt":0.2,"trotter_steps":2},"cpu_allocation":"0","has_mpi":true,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":1},"measurement_kind":"pytest","node_id":"bench_models.py::test_model_gradient[hubbard]","operation":"gradient","parameters_digest":"2742159d9e8aa1810c6cbf63993fd0612a3b5c62f68b31871ad0784764a60e20","profile":"tiny"},{"config_digest":"251e899349b55ed62a6e51e2fad26381b0d2701d4431f0630732f6dad001ca58","id":"tiny--gradient-hubb--mpi-1x96","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"chemical_potential":0.0,"cutoff":6,"hopping":1.0,"interaction":-2.0,"lower_atol":0.0001,"neel_start_spin":"down","num_sites":4,"observable_site":1,"observable_spin":"up","trotter_dt":0.2,"trotter_steps":2},"cpu_allocation":"0-95","has_mpi":true,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":96},"measurement_kind":"pytest","node_id":"bench_models.py::test_model_gradient[hubbard]","operation":"gradient","parameters_digest":"2742159d9e8aa1810c6cbf63993fd0612a3b5c62f68b31871ad0784764a60e20","profile":"tiny"},{"config_digest":"251e899349b55ed62a6e51e2fad26381b0d2701d4431f0630732f6dad001ca58","id":"tiny--gradient-hubb--mpi-2x48","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"chemical_potential":0.0,"cutoff":6,"hopping":1.0,"interaction":-2.0,"lower_atol":0.0001,"neel_start_spin":"down","num_sites":4,"observable_site":1,"observable_spin":"up","trotter_dt":0.2,"trotter_steps":2},"cpu_allocation":"0-95","has_mpi":true,"index_bits":32,"ranks":2,"routing":{"bits":1,"mode":"linear","seed":6768574230969066775},"threads":48},"measurement_kind":"pytest","node_id":"bench_models.py::test_model_gradient[hubbard]","operation":"gradient","parameters_digest":"2742159d9e8aa1810c6cbf63993fd0612a3b5c62f68b31871ad0784764a60e20","profile":"tiny"},{"config_digest":"251e899349b55ed62a6e51e2fad26381b0d2701d4431f0630732f6dad001ca58","id":"tiny--gradient-hubb--off-1x96","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"chemical_potential":0.0,"cutoff":6,"hopping":1.0,"interaction":-2.0,"lower_atol":0.0001,"neel_start_spin":"down","num_sites":4,"observable_site":1,"observable_spin":"up","trotter_dt":0.2,"trotter_steps":2},"cpu_allocation":"0-95","has_mpi":false,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":96},"measurement_kind":"pytest","node_id":"bench_models.py::test_model_gradient[hubbard]","operation":"gradient","parameters_digest":"2742159d9e8aa1810c6cbf63993fd0612a3b5c62f68b31871ad0784764a60e20","profile":"tiny"},{"config_digest":"33f7cbf55f96127177eb5adb68bf2a4ae45a486da9b5ddb1dd7eb76fae4d8d5e","id":"tiny--gradient-paul--mpi-1x1","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"coupling":0.7853981633974483,"cutoff":2,"lower_atol":0.1,"num_layers":1,"num_qubits":127,"observable_qubit":62,"theta":0.7853981633974483},"cpu_allocation":"0","has_mpi":true,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":1},"measurement_kind":"pytest","node_id":"bench_models.py::test_model_gradient[pauli]","operation":"gradient","parameters_digest":"70d31cd57471b39e5025b99cce04a23349f951ee380aa34fffb80cbe33626ab0","profile":"tiny"},{"config_digest":"33f7cbf55f96127177eb5adb68bf2a4ae45a486da9b5ddb1dd7eb76fae4d8d5e","id":"tiny--gradient-paul--mpi-1x96","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"coupling":0.7853981633974483,"cutoff":2,"lower_atol":0.1,"num_layers":1,"num_qubits":127,"observable_qubit":62,"theta":0.7853981633974483},"cpu_allocation":"0-95","has_mpi":true,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":96},"measurement_kind":"pytest","node_id":"bench_models.py::test_model_gradient[pauli]","operation":"gradient","parameters_digest":"70d31cd57471b39e5025b99cce04a23349f951ee380aa34fffb80cbe33626ab0","profile":"tiny"},{"config_digest":"33f7cbf55f96127177eb5adb68bf2a4ae45a486da9b5ddb1dd7eb76fae4d8d5e","id":"tiny--gradient-paul--mpi-2x48","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"coupling":0.7853981633974483,"cutoff":2,"lower_atol":0.1,"num_layers":1,"num_qubits":127,"observable_qubit":62,"theta":0.7853981633974483},"cpu_allocation":"0-95","has_mpi":true,"index_bits":32,"ranks":2,"routing":{"bits":1,"mode":"linear","seed":6768574230969066775},"threads":48},"measurement_kind":"pytest","node_id":"bench_models.py::test_model_gradient[pauli]","operation":"gradient","parameters_digest":"70d31cd57471b39e5025b99cce04a23349f951ee380aa34fffb80cbe33626ab0","profile":"tiny"},{"config_digest":"33f7cbf55f96127177eb5adb68bf2a4ae45a486da9b5ddb1dd7eb76fae4d8d5e","id":"tiny--gradient-paul--off-1x96","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"coupling":0.7853981633974483,"cutoff":2,"lower_atol":0.1,"num_layers":1,"num_qubits":127,"observable_qubit":62,"theta":0.7853981633974483},"cpu_allocation":"0-95","has_mpi":false,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":96},"measurement_kind":"pytest","node_id":"bench_models.py::test_model_gradient[pauli]","operation":"gradient","parameters_digest":"70d31cd57471b39e5025b99cce04a23349f951ee380aa34fffb80cbe33626ab0","profile":"tiny"},{"config_digest":"fb5da6a0444ab95f40ba4e8446729f3139a2105f1a9eb4f7cc5a3d5e6e2afd21","id":"tiny-pared--energy-heis--mpi-1x1","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":8,"num_modes":8,"obs_terms":16,"seed":0},"cpu_allocation":"0","has_mpi":true,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":1},"measurement_kind":"pytest","node_id":"bench_random.py::test_random_energy[heisenberg]","operation":"energy","parameters_digest":"4a68849272707590968775797bd7e6a107e692378ce2789cf5c8ffd5dd0f65f3","profile":"tiny-pared"},{"config_digest":"fb5da6a0444ab95f40ba4e8446729f3139a2105f1a9eb4f7cc5a3d5e6e2afd21","id":"tiny-pared--energy-heis--mpi-1x96","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":8,"num_modes":8,"obs_terms":16,"seed":0},"cpu_allocation":"0-95","has_mpi":true,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":96},"measurement_kind":"pytest","node_id":"bench_random.py::test_random_energy[heisenberg]","operation":"energy","parameters_digest":"4a68849272707590968775797bd7e6a107e692378ce2789cf5c8ffd5dd0f65f3","profile":"tiny-pared"},{"config_digest":"fb5da6a0444ab95f40ba4e8446729f3139a2105f1a9eb4f7cc5a3d5e6e2afd21","id":"tiny-pared--energy-heis--mpi-2x48","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":8,"num_modes":8,"obs_terms":16,"seed":0},"cpu_allocation":"0-95","has_mpi":true,"index_bits":32,"ranks":2,"routing":{"bits":1,"mode":"linear","seed":6768574230969066775},"threads":48},"measurement_kind":"pytest","node_id":"bench_random.py::test_random_energy[heisenberg]","operation":"energy","parameters_digest":"4a68849272707590968775797bd7e6a107e692378ce2789cf5c8ffd5dd0f65f3","profile":"tiny-pared"},{"config_digest":"fb5da6a0444ab95f40ba4e8446729f3139a2105f1a9eb4f7cc5a3d5e6e2afd21","id":"tiny-pared--energy-heis--off-1x96","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":8,"num_modes":8,"obs_terms":16,"seed":0},"cpu_allocation":"0-95","has_mpi":false,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":96},"measurement_kind":"pytest","node_id":"bench_random.py::test_random_energy[heisenberg]","operation":"energy","parameters_digest":"4a68849272707590968775797bd7e6a107e692378ce2789cf5c8ffd5dd0f65f3","profile":"tiny-pared"},{"config_digest":"fb5da6a0444ab95f40ba4e8446729f3139a2105f1a9eb4f7cc5a3d5e6e2afd21","id":"tiny-pared--energy-schr--mpi-1x1","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":8,"num_modes":8,"obs_terms":16,"seed":0},"cpu_allocation":"0","has_mpi":true,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":1},"measurement_kind":"pytest","node_id":"bench_random.py::test_random_energy[schrodinger]","operation":"energy","parameters_digest":"4a68849272707590968775797bd7e6a107e692378ce2789cf5c8ffd5dd0f65f3","profile":"tiny-pared"},{"config_digest":"fb5da6a0444ab95f40ba4e8446729f3139a2105f1a9eb4f7cc5a3d5e6e2afd21","id":"tiny-pared--energy-schr--mpi-1x96","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":8,"num_modes":8,"obs_terms":16,"seed":0},"cpu_allocation":"0-95","has_mpi":true,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":96},"measurement_kind":"pytest","node_id":"bench_random.py::test_random_energy[schrodinger]","operation":"energy","parameters_digest":"4a68849272707590968775797bd7e6a107e692378ce2789cf5c8ffd5dd0f65f3","profile":"tiny-pared"},{"config_digest":"fb5da6a0444ab95f40ba4e8446729f3139a2105f1a9eb4f7cc5a3d5e6e2afd21","id":"tiny-pared--energy-schr--mpi-2x48","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":8,"num_modes":8,"obs_terms":16,"seed":0},"cpu_allocation":"0-95","has_mpi":true,"index_bits":32,"ranks":2,"routing":{"bits":1,"mode":"linear","seed":6768574230969066775},"threads":48},"measurement_kind":"pytest","node_id":"bench_random.py::test_random_energy[schrodinger]","operation":"energy","parameters_digest":"4a68849272707590968775797bd7e6a107e692378ce2789cf5c8ffd5dd0f65f3","profile":"tiny-pared"},{"config_digest":"fb5da6a0444ab95f40ba4e8446729f3139a2105f1a9eb4f7cc5a3d5e6e2afd21","id":"tiny-pared--energy-schr--off-1x96","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":8,"num_modes":8,"obs_terms":16,"seed":0},"cpu_allocation":"0-95","has_mpi":false,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":96},"measurement_kind":"pytest","node_id":"bench_random.py::test_random_energy[schrodinger]","operation":"energy","parameters_digest":"4a68849272707590968775797bd7e6a107e692378ce2789cf5c8ffd5dd0f65f3","profile":"tiny-pared"},{"config_digest":"251e899349b55ed62a6e51e2fad26381b0d2701d4431f0630732f6dad001ca58","id":"tiny-pared--energy-hubb--mpi-1x1","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"chemical_potential":0.0,"cutoff":6,"hopping":1.0,"interaction":-2.0,"lower_atol":0.0001,"neel_start_spin":"down","num_sites":4,"observable_site":1,"observable_spin":"up","trotter_dt":0.2,"trotter_steps":2},"cpu_allocation":"0","has_mpi":true,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":1},"measurement_kind":"pytest","node_id":"bench_models.py::test_model_energy[hubbard]","operation":"energy","parameters_digest":"2742159d9e8aa1810c6cbf63993fd0612a3b5c62f68b31871ad0784764a60e20","profile":"tiny-pared"},{"config_digest":"251e899349b55ed62a6e51e2fad26381b0d2701d4431f0630732f6dad001ca58","id":"tiny-pared--energy-hubb--mpi-1x96","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"chemical_potential":0.0,"cutoff":6,"hopping":1.0,"interaction":-2.0,"lower_atol":0.0001,"neel_start_spin":"down","num_sites":4,"observable_site":1,"observable_spin":"up","trotter_dt":0.2,"trotter_steps":2},"cpu_allocation":"0-95","has_mpi":true,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":96},"measurement_kind":"pytest","node_id":"bench_models.py::test_model_energy[hubbard]","operation":"energy","parameters_digest":"2742159d9e8aa1810c6cbf63993fd0612a3b5c62f68b31871ad0784764a60e20","profile":"tiny-pared"},{"config_digest":"251e899349b55ed62a6e51e2fad26381b0d2701d4431f0630732f6dad001ca58","id":"tiny-pared--energy-hubb--mpi-2x48","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"chemical_potential":0.0,"cutoff":6,"hopping":1.0,"interaction":-2.0,"lower_atol":0.0001,"neel_start_spin":"down","num_sites":4,"observable_site":1,"observable_spin":"up","trotter_dt":0.2,"trotter_steps":2},"cpu_allocation":"0-95","has_mpi":true,"index_bits":32,"ranks":2,"routing":{"bits":1,"mode":"linear","seed":6768574230969066775},"threads":48},"measurement_kind":"pytest","node_id":"bench_models.py::test_model_energy[hubbard]","operation":"energy","parameters_digest":"2742159d9e8aa1810c6cbf63993fd0612a3b5c62f68b31871ad0784764a60e20","profile":"tiny-pared"},{"config_digest":"251e899349b55ed62a6e51e2fad26381b0d2701d4431f0630732f6dad001ca58","id":"tiny-pared--energy-hubb--off-1x96","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"chemical_potential":0.0,"cutoff":6,"hopping":1.0,"interaction":-2.0,"lower_atol":0.0001,"neel_start_spin":"down","num_sites":4,"observable_site":1,"observable_spin":"up","trotter_dt":0.2,"trotter_steps":2},"cpu_allocation":"0-95","has_mpi":false,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":96},"measurement_kind":"pytest","node_id":"bench_models.py::test_model_energy[hubbard]","operation":"energy","parameters_digest":"2742159d9e8aa1810c6cbf63993fd0612a3b5c62f68b31871ad0784764a60e20","profile":"tiny-pared"},{"config_digest":"33f7cbf55f96127177eb5adb68bf2a4ae45a486da9b5ddb1dd7eb76fae4d8d5e","id":"tiny-pared--energy-paul--mpi-1x1","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"coupling":0.7853981633974483,"cutoff":2,"lower_atol":0.1,"num_layers":1,"num_qubits":127,"observable_qubit":62,"theta":0.7853981633974483},"cpu_allocation":"0","has_mpi":true,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":1},"measurement_kind":"pytest","node_id":"bench_models.py::test_model_energy[pauli]","operation":"energy","parameters_digest":"70d31cd57471b39e5025b99cce04a23349f951ee380aa34fffb80cbe33626ab0","profile":"tiny-pared"},{"config_digest":"33f7cbf55f96127177eb5adb68bf2a4ae45a486da9b5ddb1dd7eb76fae4d8d5e","id":"tiny-pared--energy-paul--mpi-1x96","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"coupling":0.7853981633974483,"cutoff":2,"lower_atol":0.1,"num_layers":1,"num_qubits":127,"observable_qubit":62,"theta":0.7853981633974483},"cpu_allocation":"0-95","has_mpi":true,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":96},"measurement_kind":"pytest","node_id":"bench_models.py::test_model_energy[pauli]","operation":"energy","parameters_digest":"70d31cd57471b39e5025b99cce04a23349f951ee380aa34fffb80cbe33626ab0","profile":"tiny-pared"},{"config_digest":"33f7cbf55f96127177eb5adb68bf2a4ae45a486da9b5ddb1dd7eb76fae4d8d5e","id":"tiny-pared--energy-paul--mpi-2x48","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"coupling":0.7853981633974483,"cutoff":2,"lower_atol":0.1,"num_layers":1,"num_qubits":127,"observable_qubit":62,"theta":0.7853981633974483},"cpu_allocation":"0-95","has_mpi":true,"index_bits":32,"ranks":2,"routing":{"bits":1,"mode":"linear","seed":6768574230969066775},"threads":48},"measurement_kind":"pytest","node_id":"bench_models.py::test_model_energy[pauli]","operation":"energy","parameters_digest":"70d31cd57471b39e5025b99cce04a23349f951ee380aa34fffb80cbe33626ab0","profile":"tiny-pared"},{"config_digest":"33f7cbf55f96127177eb5adb68bf2a4ae45a486da9b5ddb1dd7eb76fae4d8d5e","id":"tiny-pared--energy-paul--off-1x96","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"coupling":0.7853981633974483,"cutoff":2,"lower_atol":0.1,"num_layers":1,"num_qubits":127,"observable_qubit":62,"theta":0.7853981633974483},"cpu_allocation":"0-95","has_mpi":false,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":96},"measurement_kind":"pytest","node_id":"bench_models.py::test_model_energy[pauli]","operation":"energy","parameters_digest":"70d31cd57471b39e5025b99cce04a23349f951ee380aa34fffb80cbe33626ab0","profile":"tiny-pared"},{"config_digest":"fb5da6a0444ab95f40ba4e8446729f3139a2105f1a9eb4f7cc5a3d5e6e2afd21","id":"tiny-pared--gradient-heis--mpi-1x1","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":8,"num_modes":8,"obs_terms":16,"seed":0},"cpu_allocation":"0","has_mpi":true,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":1},"measurement_kind":"pytest","node_id":"bench_random.py::test_random_gradient[heisenberg]","operation":"gradient","parameters_digest":"4a68849272707590968775797bd7e6a107e692378ce2789cf5c8ffd5dd0f65f3","profile":"tiny-pared"},{"config_digest":"fb5da6a0444ab95f40ba4e8446729f3139a2105f1a9eb4f7cc5a3d5e6e2afd21","id":"tiny-pared--gradient-heis--mpi-1x96","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":8,"num_modes":8,"obs_terms":16,"seed":0},"cpu_allocation":"0-95","has_mpi":true,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":96},"measurement_kind":"pytest","node_id":"bench_random.py::test_random_gradient[heisenberg]","operation":"gradient","parameters_digest":"4a68849272707590968775797bd7e6a107e692378ce2789cf5c8ffd5dd0f65f3","profile":"tiny-pared"},{"config_digest":"fb5da6a0444ab95f40ba4e8446729f3139a2105f1a9eb4f7cc5a3d5e6e2afd21","id":"tiny-pared--gradient-heis--mpi-2x48","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":8,"num_modes":8,"obs_terms":16,"seed":0},"cpu_allocation":"0-95","has_mpi":true,"index_bits":32,"ranks":2,"routing":{"bits":1,"mode":"linear","seed":6768574230969066775},"threads":48},"measurement_kind":"pytest","node_id":"bench_random.py::test_random_gradient[heisenberg]","operation":"gradient","parameters_digest":"4a68849272707590968775797bd7e6a107e692378ce2789cf5c8ffd5dd0f65f3","profile":"tiny-pared"},{"config_digest":"fb5da6a0444ab95f40ba4e8446729f3139a2105f1a9eb4f7cc5a3d5e6e2afd21","id":"tiny-pared--gradient-heis--off-1x96","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":8,"num_modes":8,"obs_terms":16,"seed":0},"cpu_allocation":"0-95","has_mpi":false,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":96},"measurement_kind":"pytest","node_id":"bench_random.py::test_random_gradient[heisenberg]","operation":"gradient","parameters_digest":"4a68849272707590968775797bd7e6a107e692378ce2789cf5c8ffd5dd0f65f3","profile":"tiny-pared"},{"config_digest":"fb5da6a0444ab95f40ba4e8446729f3139a2105f1a9eb4f7cc5a3d5e6e2afd21","id":"tiny-pared--gradient-schr--mpi-1x1","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":8,"num_modes":8,"obs_terms":16,"seed":0},"cpu_allocation":"0","has_mpi":true,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":1},"measurement_kind":"pytest","node_id":"bench_random.py::test_random_gradient[schrodinger]","operation":"gradient","parameters_digest":"4a68849272707590968775797bd7e6a107e692378ce2789cf5c8ffd5dd0f65f3","profile":"tiny-pared"},{"config_digest":"fb5da6a0444ab95f40ba4e8446729f3139a2105f1a9eb4f7cc5a3d5e6e2afd21","id":"tiny-pared--gradient-schr--mpi-1x96","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":8,"num_modes":8,"obs_terms":16,"seed":0},"cpu_allocation":"0-95","has_mpi":true,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":96},"measurement_kind":"pytest","node_id":"bench_random.py::test_random_gradient[schrodinger]","operation":"gradient","parameters_digest":"4a68849272707590968775797bd7e6a107e692378ce2789cf5c8ffd5dd0f65f3","profile":"tiny-pared"},{"config_digest":"fb5da6a0444ab95f40ba4e8446729f3139a2105f1a9eb4f7cc5a3d5e6e2afd21","id":"tiny-pared--gradient-schr--mpi-2x48","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":8,"num_modes":8,"obs_terms":16,"seed":0},"cpu_allocation":"0-95","has_mpi":true,"index_bits":32,"ranks":2,"routing":{"bits":1,"mode":"linear","seed":6768574230969066775},"threads":48},"measurement_kind":"pytest","node_id":"bench_random.py::test_random_gradient[schrodinger]","operation":"gradient","parameters_digest":"4a68849272707590968775797bd7e6a107e692378ce2789cf5c8ffd5dd0f65f3","profile":"tiny-pared"},{"config_digest":"fb5da6a0444ab95f40ba4e8446729f3139a2105f1a9eb4f7cc5a3d5e6e2afd21","id":"tiny-pared--gradient-schr--off-1x96","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":8,"num_modes":8,"obs_terms":16,"seed":0},"cpu_allocation":"0-95","has_mpi":false,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":96},"measurement_kind":"pytest","node_id":"bench_random.py::test_random_gradient[schrodinger]","operation":"gradient","parameters_digest":"4a68849272707590968775797bd7e6a107e692378ce2789cf5c8ffd5dd0f65f3","profile":"tiny-pared"},{"config_digest":"251e899349b55ed62a6e51e2fad26381b0d2701d4431f0630732f6dad001ca58","id":"tiny-pared--gradient-hubb--mpi-1x1","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"chemical_potential":0.0,"cutoff":6,"hopping":1.0,"interaction":-2.0,"lower_atol":0.0001,"neel_start_spin":"down","num_sites":4,"observable_site":1,"observable_spin":"up","trotter_dt":0.2,"trotter_steps":2},"cpu_allocation":"0","has_mpi":true,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":1},"measurement_kind":"pytest","node_id":"bench_models.py::test_model_gradient[hubbard]","operation":"gradient","parameters_digest":"2742159d9e8aa1810c6cbf63993fd0612a3b5c62f68b31871ad0784764a60e20","profile":"tiny-pared"},{"config_digest":"251e899349b55ed62a6e51e2fad26381b0d2701d4431f0630732f6dad001ca58","id":"tiny-pared--gradient-hubb--mpi-1x96","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"chemical_potential":0.0,"cutoff":6,"hopping":1.0,"interaction":-2.0,"lower_atol":0.0001,"neel_start_spin":"down","num_sites":4,"observable_site":1,"observable_spin":"up","trotter_dt":0.2,"trotter_steps":2},"cpu_allocation":"0-95","has_mpi":true,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":96},"measurement_kind":"pytest","node_id":"bench_models.py::test_model_gradient[hubbard]","operation":"gradient","parameters_digest":"2742159d9e8aa1810c6cbf63993fd0612a3b5c62f68b31871ad0784764a60e20","profile":"tiny-pared"},{"config_digest":"251e899349b55ed62a6e51e2fad26381b0d2701d4431f0630732f6dad001ca58","id":"tiny-pared--gradient-hubb--mpi-2x48","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"chemical_potential":0.0,"cutoff":6,"hopping":1.0,"interaction":-2.0,"lower_atol":0.0001,"neel_start_spin":"down","num_sites":4,"observable_site":1,"observable_spin":"up","trotter_dt":0.2,"trotter_steps":2},"cpu_allocation":"0-95","has_mpi":true,"index_bits":32,"ranks":2,"routing":{"bits":1,"mode":"linear","seed":6768574230969066775},"threads":48},"measurement_kind":"pytest","node_id":"bench_models.py::test_model_gradient[hubbard]","operation":"gradient","parameters_digest":"2742159d9e8aa1810c6cbf63993fd0612a3b5c62f68b31871ad0784764a60e20","profile":"tiny-pared"},{"config_digest":"251e899349b55ed62a6e51e2fad26381b0d2701d4431f0630732f6dad001ca58","id":"tiny-pared--gradient-hubb--off-1x96","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"chemical_potential":0.0,"cutoff":6,"hopping":1.0,"interaction":-2.0,"lower_atol":0.0001,"neel_start_spin":"down","num_sites":4,"observable_site":1,"observable_spin":"up","trotter_dt":0.2,"trotter_steps":2},"cpu_allocation":"0-95","has_mpi":false,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":96},"measurement_kind":"pytest","node_id":"bench_models.py::test_model_gradient[hubbard]","operation":"gradient","parameters_digest":"2742159d9e8aa1810c6cbf63993fd0612a3b5c62f68b31871ad0784764a60e20","profile":"tiny-pared"},{"config_digest":"33f7cbf55f96127177eb5adb68bf2a4ae45a486da9b5ddb1dd7eb76fae4d8d5e","id":"tiny-pared--gradient-paul--mpi-1x1","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"coupling":0.7853981633974483,"cutoff":2,"lower_atol":0.1,"num_layers":1,"num_qubits":127,"observable_qubit":62,"theta":0.7853981633974483},"cpu_allocation":"0","has_mpi":true,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":1},"measurement_kind":"pytest","node_id":"bench_models.py::test_model_gradient[pauli]","operation":"gradient","parameters_digest":"70d31cd57471b39e5025b99cce04a23349f951ee380aa34fffb80cbe33626ab0","profile":"tiny-pared"},{"config_digest":"33f7cbf55f96127177eb5adb68bf2a4ae45a486da9b5ddb1dd7eb76fae4d8d5e","id":"tiny-pared--gradient-paul--mpi-1x96","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"coupling":0.7853981633974483,"cutoff":2,"lower_atol":0.1,"num_layers":1,"num_qubits":127,"observable_qubit":62,"theta":0.7853981633974483},"cpu_allocation":"0-95","has_mpi":true,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":96},"measurement_kind":"pytest","node_id":"bench_models.py::test_model_gradient[pauli]","operation":"gradient","parameters_digest":"70d31cd57471b39e5025b99cce04a23349f951ee380aa34fffb80cbe33626ab0","profile":"tiny-pared"},{"config_digest":"33f7cbf55f96127177eb5adb68bf2a4ae45a486da9b5ddb1dd7eb76fae4d8d5e","id":"tiny-pared--gradient-paul--mpi-2x48","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"coupling":0.7853981633974483,"cutoff":2,"lower_atol":0.1,"num_layers":1,"num_qubits":127,"observable_qubit":62,"theta":0.7853981633974483},"cpu_allocation":"0-95","has_mpi":true,"index_bits":32,"ranks":2,"routing":{"bits":1,"mode":"linear","seed":6768574230969066775},"threads":48},"measurement_kind":"pytest","node_id":"bench_models.py::test_model_gradient[pauli]","operation":"gradient","parameters_digest":"70d31cd57471b39e5025b99cce04a23349f951ee380aa34fffb80cbe33626ab0","profile":"tiny-pared"},{"config_digest":"33f7cbf55f96127177eb5adb68bf2a4ae45a486da9b5ddb1dd7eb76fae4d8d5e","id":"tiny-pared--gradient-paul--off-1x96","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"coupling":0.7853981633974483,"cutoff":2,"lower_atol":0.1,"num_layers":1,"num_qubits":127,"observable_qubit":62,"theta":0.7853981633974483},"cpu_allocation":"0-95","has_mpi":false,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":96},"measurement_kind":"pytest","node_id":"bench_models.py::test_model_gradient[pauli]","operation":"gradient","parameters_digest":"70d31cd57471b39e5025b99cce04a23349f951ee380aa34fffb80cbe33626ab0","profile":"tiny-pared"},{"config_digest":"fb5da6a0444ab95f40ba4e8446729f3139a2105f1a9eb4f7cc5a3d5e6e2afd21","id":"tiny-pared--pare-construct-heis--mpi-1x1","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":8,"num_modes":8,"obs_terms":16,"seed":0},"cpu_allocation":"0","has_mpi":true,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":1},"measurement_kind":"driver","node_id":"driver::pare_functional_construct[heisenberg]","operation":"pare_functional_construct","parameters_digest":"4a68849272707590968775797bd7e6a107e692378ce2789cf5c8ffd5dd0f65f3","profile":"tiny-pared"},{"config_digest":"fb5da6a0444ab95f40ba4e8446729f3139a2105f1a9eb4f7cc5a3d5e6e2afd21","id":"tiny-pared--pare-construct-heis--mpi-1x96","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":8,"num_modes":8,"obs_terms":16,"seed":0},"cpu_allocation":"0-95","has_mpi":true,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":96},"measurement_kind":"driver","node_id":"driver::pare_functional_construct[heisenberg]","operation":"pare_functional_construct","parameters_digest":"4a68849272707590968775797bd7e6a107e692378ce2789cf5c8ffd5dd0f65f3","profile":"tiny-pared"},{"config_digest":"fb5da6a0444ab95f40ba4e8446729f3139a2105f1a9eb4f7cc5a3d5e6e2afd21","id":"tiny-pared--pare-construct-heis--mpi-2x48","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":8,"num_modes":8,"obs_terms":16,"seed":0},"cpu_allocation":"0-95","has_mpi":true,"index_bits":32,"ranks":2,"routing":{"bits":1,"mode":"linear","seed":6768574230969066775},"threads":48},"measurement_kind":"driver","node_id":"driver::pare_functional_construct[heisenberg]","operation":"pare_functional_construct","parameters_digest":"4a68849272707590968775797bd7e6a107e692378ce2789cf5c8ffd5dd0f65f3","profile":"tiny-pared"},{"config_digest":"fb5da6a0444ab95f40ba4e8446729f3139a2105f1a9eb4f7cc5a3d5e6e2afd21","id":"tiny-pared--pare-construct-heis--off-1x96","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":8,"num_modes":8,"obs_terms":16,"seed":0},"cpu_allocation":"0-95","has_mpi":false,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":96},"measurement_kind":"driver","node_id":"driver::pare_functional_construct[heisenberg]","operation":"pare_functional_construct","parameters_digest":"4a68849272707590968775797bd7e6a107e692378ce2789cf5c8ffd5dd0f65f3","profile":"tiny-pared"},{"config_digest":"fb5da6a0444ab95f40ba4e8446729f3139a2105f1a9eb4f7cc5a3d5e6e2afd21","id":"tiny-pared--pare-construct-schr--mpi-1x1","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":8,"num_modes":8,"obs_terms":16,"seed":0},"cpu_allocation":"0","has_mpi":true,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":1},"measurement_kind":"driver","node_id":"driver::pare_functional_construct[schrodinger]","operation":"pare_functional_construct","parameters_digest":"4a68849272707590968775797bd7e6a107e692378ce2789cf5c8ffd5dd0f65f3","profile":"tiny-pared"},{"config_digest":"fb5da6a0444ab95f40ba4e8446729f3139a2105f1a9eb4f7cc5a3d5e6e2afd21","id":"tiny-pared--pare-construct-schr--mpi-1x96","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":8,"num_modes":8,"obs_terms":16,"seed":0},"cpu_allocation":"0-95","has_mpi":true,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":96},"measurement_kind":"driver","node_id":"driver::pare_functional_construct[schrodinger]","operation":"pare_functional_construct","parameters_digest":"4a68849272707590968775797bd7e6a107e692378ce2789cf5c8ffd5dd0f65f3","profile":"tiny-pared"},{"config_digest":"fb5da6a0444ab95f40ba4e8446729f3139a2105f1a9eb4f7cc5a3d5e6e2afd21","id":"tiny-pared--pare-construct-schr--mpi-2x48","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":8,"num_modes":8,"obs_terms":16,"seed":0},"cpu_allocation":"0-95","has_mpi":true,"index_bits":32,"ranks":2,"routing":{"bits":1,"mode":"linear","seed":6768574230969066775},"threads":48},"measurement_kind":"driver","node_id":"driver::pare_functional_construct[schrodinger]","operation":"pare_functional_construct","parameters_digest":"4a68849272707590968775797bd7e6a107e692378ce2789cf5c8ffd5dd0f65f3","profile":"tiny-pared"},{"config_digest":"fb5da6a0444ab95f40ba4e8446729f3139a2105f1a9eb4f7cc5a3d5e6e2afd21","id":"tiny-pared--pare-construct-schr--off-1x96","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":8,"num_modes":8,"obs_terms":16,"seed":0},"cpu_allocation":"0-95","has_mpi":false,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":96},"measurement_kind":"driver","node_id":"driver::pare_functional_construct[schrodinger]","operation":"pare_functional_construct","parameters_digest":"4a68849272707590968775797bd7e6a107e692378ce2789cf5c8ffd5dd0f65f3","profile":"tiny-pared"},{"config_digest":"8a194c1d11504cba158ffde10e81136303cf6cc37c82cc3966bdf8abaa6d7f53","id":"reference--build-graph-heis--mpi-1x1","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":400,"num_modes":128,"obs_terms":400000,"seed":0},"cpu_allocation":"0","has_mpi":true,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":1},"measurement_kind":"pytest","node_id":"bench_random.py::test_random_build_graph[heisenberg]","operation":"build_graph","parameters_digest":"2ee633f84e9f2df140694f3366370774debc7cf4931449a109e276cb6b0f7dcf","profile":"reference"},{"config_digest":"8a194c1d11504cba158ffde10e81136303cf6cc37c82cc3966bdf8abaa6d7f53","id":"reference--build-graph-heis--mpi-1x96","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":400,"num_modes":128,"obs_terms":400000,"seed":0},"cpu_allocation":"0-95","has_mpi":true,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":96},"measurement_kind":"pytest","node_id":"bench_random.py::test_random_build_graph[heisenberg]","operation":"build_graph","parameters_digest":"2ee633f84e9f2df140694f3366370774debc7cf4931449a109e276cb6b0f7dcf","profile":"reference"},{"config_digest":"8a194c1d11504cba158ffde10e81136303cf6cc37c82cc3966bdf8abaa6d7f53","id":"reference--build-graph-heis--mpi-2x48","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":400,"num_modes":128,"obs_terms":400000,"seed":0},"cpu_allocation":"0-95","has_mpi":true,"index_bits":32,"ranks":2,"routing":{"bits":1,"mode":"linear","seed":6768574230969066775},"threads":48},"measurement_kind":"pytest","node_id":"bench_random.py::test_random_build_graph[heisenberg]","operation":"build_graph","parameters_digest":"2ee633f84e9f2df140694f3366370774debc7cf4931449a109e276cb6b0f7dcf","profile":"reference"},{"config_digest":"8a194c1d11504cba158ffde10e81136303cf6cc37c82cc3966bdf8abaa6d7f53","id":"reference--build-graph-heis--off-1x96","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":400,"num_modes":128,"obs_terms":400000,"seed":0},"cpu_allocation":"0-95","has_mpi":false,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":96},"measurement_kind":"pytest","node_id":"bench_random.py::test_random_build_graph[heisenberg]","operation":"build_graph","parameters_digest":"2ee633f84e9f2df140694f3366370774debc7cf4931449a109e276cb6b0f7dcf","profile":"reference"},{"config_digest":"84d8747d842ac221af6a1bb980015748fd049857489d1cfe3c87569509dc440e","id":"reference--build-graph-schr--mpi-1x1","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":400,"num_modes":128,"obs_terms":10000,"seed":0},"cpu_allocation":"0","has_mpi":true,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":1},"measurement_kind":"pytest","node_id":"bench_random.py::test_random_build_graph[schrodinger]","operation":"build_graph","parameters_digest":"73a43144ad39c4dc12e9e0d4695e09a80ac75511776587ba072a83daf596782a","profile":"reference"},{"config_digest":"84d8747d842ac221af6a1bb980015748fd049857489d1cfe3c87569509dc440e","id":"reference--build-graph-schr--mpi-1x96","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":400,"num_modes":128,"obs_terms":10000,"seed":0},"cpu_allocation":"0-95","has_mpi":true,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":96},"measurement_kind":"pytest","node_id":"bench_random.py::test_random_build_graph[schrodinger]","operation":"build_graph","parameters_digest":"73a43144ad39c4dc12e9e0d4695e09a80ac75511776587ba072a83daf596782a","profile":"reference"},{"config_digest":"84d8747d842ac221af6a1bb980015748fd049857489d1cfe3c87569509dc440e","id":"reference--build-graph-schr--mpi-2x48","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":400,"num_modes":128,"obs_terms":10000,"seed":0},"cpu_allocation":"0-95","has_mpi":true,"index_bits":32,"ranks":2,"routing":{"bits":1,"mode":"linear","seed":6768574230969066775},"threads":48},"measurement_kind":"pytest","node_id":"bench_random.py::test_random_build_graph[schrodinger]","operation":"build_graph","parameters_digest":"73a43144ad39c4dc12e9e0d4695e09a80ac75511776587ba072a83daf596782a","profile":"reference"},{"config_digest":"84d8747d842ac221af6a1bb980015748fd049857489d1cfe3c87569509dc440e","id":"reference--build-graph-schr--off-1x96","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":400,"num_modes":128,"obs_terms":10000,"seed":0},"cpu_allocation":"0-95","has_mpi":false,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":96},"measurement_kind":"pytest","node_id":"bench_random.py::test_random_build_graph[schrodinger]","operation":"build_graph","parameters_digest":"73a43144ad39c4dc12e9e0d4695e09a80ac75511776587ba072a83daf596782a","profile":"reference"},{"config_digest":"b6f2b0b1a3be6d446330fedbc3ed742b798474823d26390c52ba0de7a888e922","id":"reference--build-graph-hubb--mpi-1x1","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"chemical_potential":0.0,"cutoff":6,"hopping":1.0,"interaction":-2.0,"lower_atol":3e-05,"neel_start_spin":"down","num_sites":24,"observable_site":18,"observable_spin":"up","trotter_dt":0.2,"trotter_steps":2},"cpu_allocation":"0","has_mpi":true,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":1},"measurement_kind":"pytest","node_id":"bench_models.py::test_model_build_graph[hubbard]","operation":"build_graph","parameters_digest":"c0c7d11ee1836009d0abe437246dd4021c6ac7401c7f1f5c8b11a1bc4a8567a0","profile":"reference"},{"config_digest":"b6f2b0b1a3be6d446330fedbc3ed742b798474823d26390c52ba0de7a888e922","id":"reference--build-graph-hubb--mpi-1x96","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"chemical_potential":0.0,"cutoff":6,"hopping":1.0,"interaction":-2.0,"lower_atol":3e-05,"neel_start_spin":"down","num_sites":24,"observable_site":18,"observable_spin":"up","trotter_dt":0.2,"trotter_steps":2},"cpu_allocation":"0-95","has_mpi":true,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":96},"measurement_kind":"pytest","node_id":"bench_models.py::test_model_build_graph[hubbard]","operation":"build_graph","parameters_digest":"c0c7d11ee1836009d0abe437246dd4021c6ac7401c7f1f5c8b11a1bc4a8567a0","profile":"reference"},{"config_digest":"b6f2b0b1a3be6d446330fedbc3ed742b798474823d26390c52ba0de7a888e922","id":"reference--build-graph-hubb--mpi-2x48","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"chemical_potential":0.0,"cutoff":6,"hopping":1.0,"interaction":-2.0,"lower_atol":3e-05,"neel_start_spin":"down","num_sites":24,"observable_site":18,"observable_spin":"up","trotter_dt":0.2,"trotter_steps":2},"cpu_allocation":"0-95","has_mpi":true,"index_bits":32,"ranks":2,"routing":{"bits":1,"mode":"linear","seed":6768574230969066775},"threads":48},"measurement_kind":"pytest","node_id":"bench_models.py::test_model_build_graph[hubbard]","operation":"build_graph","parameters_digest":"c0c7d11ee1836009d0abe437246dd4021c6ac7401c7f1f5c8b11a1bc4a8567a0","profile":"reference"},{"config_digest":"b6f2b0b1a3be6d446330fedbc3ed742b798474823d26390c52ba0de7a888e922","id":"reference--build-graph-hubb--off-1x96","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"chemical_potential":0.0,"cutoff":6,"hopping":1.0,"interaction":-2.0,"lower_atol":3e-05,"neel_start_spin":"down","num_sites":24,"observable_site":18,"observable_spin":"up","trotter_dt":0.2,"trotter_steps":2},"cpu_allocation":"0-95","has_mpi":false,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":96},"measurement_kind":"pytest","node_id":"bench_models.py::test_model_build_graph[hubbard]","operation":"build_graph","parameters_digest":"c0c7d11ee1836009d0abe437246dd4021c6ac7401c7f1f5c8b11a1bc4a8567a0","profile":"reference"},{"config_digest":"0f48481659cbfa2c63674a43f462d55b1555fbfa8dd00e42f74de2784623b0d4","id":"reference--build-graph-paul--mpi-1x1","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"coupling":0.7853981633974483,"cutoff":8,"lower_atol":3e-06,"num_layers":20,"num_qubits":127,"observable_qubit":62,"theta":0.7853981633974483},"cpu_allocation":"0","has_mpi":true,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":1},"measurement_kind":"pytest","node_id":"bench_models.py::test_model_build_graph[pauli]","operation":"build_graph","parameters_digest":"971c364c3cae8d46d276b9c9bffb1e14e23e023429863d9c674b92945e648a42","profile":"reference"},{"config_digest":"0f48481659cbfa2c63674a43f462d55b1555fbfa8dd00e42f74de2784623b0d4","id":"reference--build-graph-paul--mpi-1x96","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"coupling":0.7853981633974483,"cutoff":8,"lower_atol":3e-06,"num_layers":20,"num_qubits":127,"observable_qubit":62,"theta":0.7853981633974483},"cpu_allocation":"0-95","has_mpi":true,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":96},"measurement_kind":"pytest","node_id":"bench_models.py::test_model_build_graph[pauli]","operation":"build_graph","parameters_digest":"971c364c3cae8d46d276b9c9bffb1e14e23e023429863d9c674b92945e648a42","profile":"reference"},{"config_digest":"0f48481659cbfa2c63674a43f462d55b1555fbfa8dd00e42f74de2784623b0d4","id":"reference--build-graph-paul--mpi-2x48","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"coupling":0.7853981633974483,"cutoff":8,"lower_atol":3e-06,"num_layers":20,"num_qubits":127,"observable_qubit":62,"theta":0.7853981633974483},"cpu_allocation":"0-95","has_mpi":true,"index_bits":32,"ranks":2,"routing":{"bits":1,"mode":"linear","seed":6768574230969066775},"threads":48},"measurement_kind":"pytest","node_id":"bench_models.py::test_model_build_graph[pauli]","operation":"build_graph","parameters_digest":"971c364c3cae8d46d276b9c9bffb1e14e23e023429863d9c674b92945e648a42","profile":"reference"},{"config_digest":"0f48481659cbfa2c63674a43f462d55b1555fbfa8dd00e42f74de2784623b0d4","id":"reference--build-graph-paul--off-1x96","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"coupling":0.7853981633974483,"cutoff":8,"lower_atol":3e-06,"num_layers":20,"num_qubits":127,"observable_qubit":62,"theta":0.7853981633974483},"cpu_allocation":"0-95","has_mpi":false,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":96},"measurement_kind":"pytest","node_id":"bench_models.py::test_model_build_graph[pauli]","operation":"build_graph","parameters_digest":"971c364c3cae8d46d276b9c9bffb1e14e23e023429863d9c674b92945e648a42","profile":"reference"},{"config_digest":"8a194c1d11504cba158ffde10e81136303cf6cc37c82cc3966bdf8abaa6d7f53","id":"reference--propagate-heis--mpi-1x1","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":400,"num_modes":128,"obs_terms":400000,"seed":0},"cpu_allocation":"0","has_mpi":true,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":1},"measurement_kind":"pytest","node_id":"bench_random.py::test_random_propagate[heisenberg]","operation":"propagate","parameters_digest":"2ee633f84e9f2df140694f3366370774debc7cf4931449a109e276cb6b0f7dcf","profile":"reference"},{"config_digest":"8a194c1d11504cba158ffde10e81136303cf6cc37c82cc3966bdf8abaa6d7f53","id":"reference--propagate-heis--mpi-1x96","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":400,"num_modes":128,"obs_terms":400000,"seed":0},"cpu_allocation":"0-95","has_mpi":true,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":96},"measurement_kind":"pytest","node_id":"bench_random.py::test_random_propagate[heisenberg]","operation":"propagate","parameters_digest":"2ee633f84e9f2df140694f3366370774debc7cf4931449a109e276cb6b0f7dcf","profile":"reference"},{"config_digest":"8a194c1d11504cba158ffde10e81136303cf6cc37c82cc3966bdf8abaa6d7f53","id":"reference--propagate-heis--mpi-2x48","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":400,"num_modes":128,"obs_terms":400000,"seed":0},"cpu_allocation":"0-95","has_mpi":true,"index_bits":32,"ranks":2,"routing":{"bits":1,"mode":"linear","seed":6768574230969066775},"threads":48},"measurement_kind":"pytest","node_id":"bench_random.py::test_random_propagate[heisenberg]","operation":"propagate","parameters_digest":"2ee633f84e9f2df140694f3366370774debc7cf4931449a109e276cb6b0f7dcf","profile":"reference"},{"config_digest":"8a194c1d11504cba158ffde10e81136303cf6cc37c82cc3966bdf8abaa6d7f53","id":"reference--propagate-heis--off-1x96","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":400,"num_modes":128,"obs_terms":400000,"seed":0},"cpu_allocation":"0-95","has_mpi":false,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":96},"measurement_kind":"pytest","node_id":"bench_random.py::test_random_propagate[heisenberg]","operation":"propagate","parameters_digest":"2ee633f84e9f2df140694f3366370774debc7cf4931449a109e276cb6b0f7dcf","profile":"reference"},{"config_digest":"84d8747d842ac221af6a1bb980015748fd049857489d1cfe3c87569509dc440e","id":"reference--propagate-schr--mpi-1x1","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":400,"num_modes":128,"obs_terms":10000,"seed":0},"cpu_allocation":"0","has_mpi":true,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":1},"measurement_kind":"pytest","node_id":"bench_random.py::test_random_propagate[schrodinger]","operation":"propagate","parameters_digest":"73a43144ad39c4dc12e9e0d4695e09a80ac75511776587ba072a83daf596782a","profile":"reference"},{"config_digest":"84d8747d842ac221af6a1bb980015748fd049857489d1cfe3c87569509dc440e","id":"reference--propagate-schr--mpi-1x96","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":400,"num_modes":128,"obs_terms":10000,"seed":0},"cpu_allocation":"0-95","has_mpi":true,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":96},"measurement_kind":"pytest","node_id":"bench_random.py::test_random_propagate[schrodinger]","operation":"propagate","parameters_digest":"73a43144ad39c4dc12e9e0d4695e09a80ac75511776587ba072a83daf596782a","profile":"reference"},{"config_digest":"84d8747d842ac221af6a1bb980015748fd049857489d1cfe3c87569509dc440e","id":"reference--propagate-schr--mpi-2x48","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":400,"num_modes":128,"obs_terms":10000,"seed":0},"cpu_allocation":"0-95","has_mpi":true,"index_bits":32,"ranks":2,"routing":{"bits":1,"mode":"linear","seed":6768574230969066775},"threads":48},"measurement_kind":"pytest","node_id":"bench_random.py::test_random_propagate[schrodinger]","operation":"propagate","parameters_digest":"73a43144ad39c4dc12e9e0d4695e09a80ac75511776587ba072a83daf596782a","profile":"reference"},{"config_digest":"84d8747d842ac221af6a1bb980015748fd049857489d1cfe3c87569509dc440e","id":"reference--propagate-schr--off-1x96","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":400,"num_modes":128,"obs_terms":10000,"seed":0},"cpu_allocation":"0-95","has_mpi":false,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":96},"measurement_kind":"pytest","node_id":"bench_random.py::test_random_propagate[schrodinger]","operation":"propagate","parameters_digest":"73a43144ad39c4dc12e9e0d4695e09a80ac75511776587ba072a83daf596782a","profile":"reference"},{"config_digest":"f110f1224a43dc3b26de2eaa3b813137cad147029b0afad58bb9de46701da7ff","id":"reference--propagate-hubb--mpi-1x1","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"chemical_potential":0.0,"cutoff":6,"hopping":1.0,"interaction":-2.0,"lower_atol":3e-05,"neel_start_spin":"down","num_sites":60,"observable_site":46,"observable_spin":"up","trotter_dt":0.2,"trotter_steps":29},"cpu_allocation":"0","has_mpi":true,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":1},"measurement_kind":"pytest","node_id":"bench_models.py::test_model_propagate[hubbard]","operation":"propagate","parameters_digest":"9613b53e27984ebea2313ec73a38d3142d8f96212f3c713be92a05afa57aa809","profile":"reference"},{"config_digest":"f110f1224a43dc3b26de2eaa3b813137cad147029b0afad58bb9de46701da7ff","id":"reference--propagate-hubb--mpi-1x96","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"chemical_potential":0.0,"cutoff":6,"hopping":1.0,"interaction":-2.0,"lower_atol":3e-05,"neel_start_spin":"down","num_sites":60,"observable_site":46,"observable_spin":"up","trotter_dt":0.2,"trotter_steps":29},"cpu_allocation":"0-95","has_mpi":true,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":96},"measurement_kind":"pytest","node_id":"bench_models.py::test_model_propagate[hubbard]","operation":"propagate","parameters_digest":"9613b53e27984ebea2313ec73a38d3142d8f96212f3c713be92a05afa57aa809","profile":"reference"},{"config_digest":"f110f1224a43dc3b26de2eaa3b813137cad147029b0afad58bb9de46701da7ff","id":"reference--propagate-hubb--mpi-2x48","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"chemical_potential":0.0,"cutoff":6,"hopping":1.0,"interaction":-2.0,"lower_atol":3e-05,"neel_start_spin":"down","num_sites":60,"observable_site":46,"observable_spin":"up","trotter_dt":0.2,"trotter_steps":29},"cpu_allocation":"0-95","has_mpi":true,"index_bits":32,"ranks":2,"routing":{"bits":1,"mode":"linear","seed":6768574230969066775},"threads":48},"measurement_kind":"pytest","node_id":"bench_models.py::test_model_propagate[hubbard]","operation":"propagate","parameters_digest":"9613b53e27984ebea2313ec73a38d3142d8f96212f3c713be92a05afa57aa809","profile":"reference"},{"config_digest":"f110f1224a43dc3b26de2eaa3b813137cad147029b0afad58bb9de46701da7ff","id":"reference--propagate-hubb--off-1x96","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"chemical_potential":0.0,"cutoff":6,"hopping":1.0,"interaction":-2.0,"lower_atol":3e-05,"neel_start_spin":"down","num_sites":60,"observable_site":46,"observable_spin":"up","trotter_dt":0.2,"trotter_steps":29},"cpu_allocation":"0-95","has_mpi":false,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":96},"measurement_kind":"pytest","node_id":"bench_models.py::test_model_propagate[hubbard]","operation":"propagate","parameters_digest":"9613b53e27984ebea2313ec73a38d3142d8f96212f3c713be92a05afa57aa809","profile":"reference"},{"config_digest":"0f48481659cbfa2c63674a43f462d55b1555fbfa8dd00e42f74de2784623b0d4","id":"reference--propagate-paul--mpi-1x1","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"coupling":0.7853981633974483,"cutoff":8,"lower_atol":3e-06,"num_layers":20,"num_qubits":127,"observable_qubit":62,"theta":0.7853981633974483},"cpu_allocation":"0","has_mpi":true,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":1},"measurement_kind":"pytest","node_id":"bench_models.py::test_model_propagate[pauli]","operation":"propagate","parameters_digest":"971c364c3cae8d46d276b9c9bffb1e14e23e023429863d9c674b92945e648a42","profile":"reference"},{"config_digest":"0f48481659cbfa2c63674a43f462d55b1555fbfa8dd00e42f74de2784623b0d4","id":"reference--propagate-paul--mpi-1x96","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"coupling":0.7853981633974483,"cutoff":8,"lower_atol":3e-06,"num_layers":20,"num_qubits":127,"observable_qubit":62,"theta":0.7853981633974483},"cpu_allocation":"0-95","has_mpi":true,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":96},"measurement_kind":"pytest","node_id":"bench_models.py::test_model_propagate[pauli]","operation":"propagate","parameters_digest":"971c364c3cae8d46d276b9c9bffb1e14e23e023429863d9c674b92945e648a42","profile":"reference"},{"config_digest":"0f48481659cbfa2c63674a43f462d55b1555fbfa8dd00e42f74de2784623b0d4","id":"reference--propagate-paul--mpi-2x48","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"coupling":0.7853981633974483,"cutoff":8,"lower_atol":3e-06,"num_layers":20,"num_qubits":127,"observable_qubit":62,"theta":0.7853981633974483},"cpu_allocation":"0-95","has_mpi":true,"index_bits":32,"ranks":2,"routing":{"bits":1,"mode":"linear","seed":6768574230969066775},"threads":48},"measurement_kind":"pytest","node_id":"bench_models.py::test_model_propagate[pauli]","operation":"propagate","parameters_digest":"971c364c3cae8d46d276b9c9bffb1e14e23e023429863d9c674b92945e648a42","profile":"reference"},{"config_digest":"0f48481659cbfa2c63674a43f462d55b1555fbfa8dd00e42f74de2784623b0d4","id":"reference--propagate-paul--off-1x96","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"coupling":0.7853981633974483,"cutoff":8,"lower_atol":3e-06,"num_layers":20,"num_qubits":127,"observable_qubit":62,"theta":0.7853981633974483},"cpu_allocation":"0-95","has_mpi":false,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":96},"measurement_kind":"pytest","node_id":"bench_models.py::test_model_propagate[pauli]","operation":"propagate","parameters_digest":"971c364c3cae8d46d276b9c9bffb1e14e23e023429863d9c674b92945e648a42","profile":"reference"},{"config_digest":"8a194c1d11504cba158ffde10e81136303cf6cc37c82cc3966bdf8abaa6d7f53","id":"reference--energy-heis--mpi-1x1","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":400,"num_modes":128,"obs_terms":400000,"seed":0},"cpu_allocation":"0","has_mpi":true,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":1},"measurement_kind":"pytest","node_id":"bench_random.py::test_random_energy[heisenberg]","operation":"energy","parameters_digest":"2ee633f84e9f2df140694f3366370774debc7cf4931449a109e276cb6b0f7dcf","profile":"reference"},{"config_digest":"8a194c1d11504cba158ffde10e81136303cf6cc37c82cc3966bdf8abaa6d7f53","id":"reference--energy-heis--mpi-1x96","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":400,"num_modes":128,"obs_terms":400000,"seed":0},"cpu_allocation":"0-95","has_mpi":true,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":96},"measurement_kind":"pytest","node_id":"bench_random.py::test_random_energy[heisenberg]","operation":"energy","parameters_digest":"2ee633f84e9f2df140694f3366370774debc7cf4931449a109e276cb6b0f7dcf","profile":"reference"},{"config_digest":"8a194c1d11504cba158ffde10e81136303cf6cc37c82cc3966bdf8abaa6d7f53","id":"reference--energy-heis--mpi-2x48","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":400,"num_modes":128,"obs_terms":400000,"seed":0},"cpu_allocation":"0-95","has_mpi":true,"index_bits":32,"ranks":2,"routing":{"bits":1,"mode":"linear","seed":6768574230969066775},"threads":48},"measurement_kind":"pytest","node_id":"bench_random.py::test_random_energy[heisenberg]","operation":"energy","parameters_digest":"2ee633f84e9f2df140694f3366370774debc7cf4931449a109e276cb6b0f7dcf","profile":"reference"},{"config_digest":"8a194c1d11504cba158ffde10e81136303cf6cc37c82cc3966bdf8abaa6d7f53","id":"reference--energy-heis--off-1x96","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":400,"num_modes":128,"obs_terms":400000,"seed":0},"cpu_allocation":"0-95","has_mpi":false,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":96},"measurement_kind":"pytest","node_id":"bench_random.py::test_random_energy[heisenberg]","operation":"energy","parameters_digest":"2ee633f84e9f2df140694f3366370774debc7cf4931449a109e276cb6b0f7dcf","profile":"reference"},{"config_digest":"84d8747d842ac221af6a1bb980015748fd049857489d1cfe3c87569509dc440e","id":"reference--energy-schr--mpi-1x1","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":400,"num_modes":128,"obs_terms":10000,"seed":0},"cpu_allocation":"0","has_mpi":true,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":1},"measurement_kind":"pytest","node_id":"bench_random.py::test_random_energy[schrodinger]","operation":"energy","parameters_digest":"73a43144ad39c4dc12e9e0d4695e09a80ac75511776587ba072a83daf596782a","profile":"reference"},{"config_digest":"84d8747d842ac221af6a1bb980015748fd049857489d1cfe3c87569509dc440e","id":"reference--energy-schr--mpi-1x96","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":400,"num_modes":128,"obs_terms":10000,"seed":0},"cpu_allocation":"0-95","has_mpi":true,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":96},"measurement_kind":"pytest","node_id":"bench_random.py::test_random_energy[schrodinger]","operation":"energy","parameters_digest":"73a43144ad39c4dc12e9e0d4695e09a80ac75511776587ba072a83daf596782a","profile":"reference"},{"config_digest":"84d8747d842ac221af6a1bb980015748fd049857489d1cfe3c87569509dc440e","id":"reference--energy-schr--mpi-2x48","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":400,"num_modes":128,"obs_terms":10000,"seed":0},"cpu_allocation":"0-95","has_mpi":true,"index_bits":32,"ranks":2,"routing":{"bits":1,"mode":"linear","seed":6768574230969066775},"threads":48},"measurement_kind":"pytest","node_id":"bench_random.py::test_random_energy[schrodinger]","operation":"energy","parameters_digest":"73a43144ad39c4dc12e9e0d4695e09a80ac75511776587ba072a83daf596782a","profile":"reference"},{"config_digest":"84d8747d842ac221af6a1bb980015748fd049857489d1cfe3c87569509dc440e","id":"reference--energy-schr--off-1x96","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":400,"num_modes":128,"obs_terms":10000,"seed":0},"cpu_allocation":"0-95","has_mpi":false,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":96},"measurement_kind":"pytest","node_id":"bench_random.py::test_random_energy[schrodinger]","operation":"energy","parameters_digest":"73a43144ad39c4dc12e9e0d4695e09a80ac75511776587ba072a83daf596782a","profile":"reference"},{"config_digest":"b6f2b0b1a3be6d446330fedbc3ed742b798474823d26390c52ba0de7a888e922","id":"reference--energy-hubb--mpi-1x1","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"chemical_potential":0.0,"cutoff":6,"hopping":1.0,"interaction":-2.0,"lower_atol":3e-05,"neel_start_spin":"down","num_sites":24,"observable_site":18,"observable_spin":"up","trotter_dt":0.2,"trotter_steps":2},"cpu_allocation":"0","has_mpi":true,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":1},"measurement_kind":"pytest","node_id":"bench_models.py::test_model_energy[hubbard]","operation":"energy","parameters_digest":"c0c7d11ee1836009d0abe437246dd4021c6ac7401c7f1f5c8b11a1bc4a8567a0","profile":"reference"},{"config_digest":"b6f2b0b1a3be6d446330fedbc3ed742b798474823d26390c52ba0de7a888e922","id":"reference--energy-hubb--mpi-1x96","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"chemical_potential":0.0,"cutoff":6,"hopping":1.0,"interaction":-2.0,"lower_atol":3e-05,"neel_start_spin":"down","num_sites":24,"observable_site":18,"observable_spin":"up","trotter_dt":0.2,"trotter_steps":2},"cpu_allocation":"0-95","has_mpi":true,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":96},"measurement_kind":"pytest","node_id":"bench_models.py::test_model_energy[hubbard]","operation":"energy","parameters_digest":"c0c7d11ee1836009d0abe437246dd4021c6ac7401c7f1f5c8b11a1bc4a8567a0","profile":"reference"},{"config_digest":"b6f2b0b1a3be6d446330fedbc3ed742b798474823d26390c52ba0de7a888e922","id":"reference--energy-hubb--mpi-2x48","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"chemical_potential":0.0,"cutoff":6,"hopping":1.0,"interaction":-2.0,"lower_atol":3e-05,"neel_start_spin":"down","num_sites":24,"observable_site":18,"observable_spin":"up","trotter_dt":0.2,"trotter_steps":2},"cpu_allocation":"0-95","has_mpi":true,"index_bits":32,"ranks":2,"routing":{"bits":1,"mode":"linear","seed":6768574230969066775},"threads":48},"measurement_kind":"pytest","node_id":"bench_models.py::test_model_energy[hubbard]","operation":"energy","parameters_digest":"c0c7d11ee1836009d0abe437246dd4021c6ac7401c7f1f5c8b11a1bc4a8567a0","profile":"reference"},{"config_digest":"b6f2b0b1a3be6d446330fedbc3ed742b798474823d26390c52ba0de7a888e922","id":"reference--energy-hubb--off-1x96","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"chemical_potential":0.0,"cutoff":6,"hopping":1.0,"interaction":-2.0,"lower_atol":3e-05,"neel_start_spin":"down","num_sites":24,"observable_site":18,"observable_spin":"up","trotter_dt":0.2,"trotter_steps":2},"cpu_allocation":"0-95","has_mpi":false,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":96},"measurement_kind":"pytest","node_id":"bench_models.py::test_model_energy[hubbard]","operation":"energy","parameters_digest":"c0c7d11ee1836009d0abe437246dd4021c6ac7401c7f1f5c8b11a1bc4a8567a0","profile":"reference"},{"config_digest":"0f48481659cbfa2c63674a43f462d55b1555fbfa8dd00e42f74de2784623b0d4","id":"reference--energy-paul--mpi-1x1","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"coupling":0.7853981633974483,"cutoff":8,"lower_atol":3e-06,"num_layers":20,"num_qubits":127,"observable_qubit":62,"theta":0.7853981633974483},"cpu_allocation":"0","has_mpi":true,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":1},"measurement_kind":"pytest","node_id":"bench_models.py::test_model_energy[pauli]","operation":"energy","parameters_digest":"971c364c3cae8d46d276b9c9bffb1e14e23e023429863d9c674b92945e648a42","profile":"reference"},{"config_digest":"0f48481659cbfa2c63674a43f462d55b1555fbfa8dd00e42f74de2784623b0d4","id":"reference--energy-paul--mpi-1x96","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"coupling":0.7853981633974483,"cutoff":8,"lower_atol":3e-06,"num_layers":20,"num_qubits":127,"observable_qubit":62,"theta":0.7853981633974483},"cpu_allocation":"0-95","has_mpi":true,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":96},"measurement_kind":"pytest","node_id":"bench_models.py::test_model_energy[pauli]","operation":"energy","parameters_digest":"971c364c3cae8d46d276b9c9bffb1e14e23e023429863d9c674b92945e648a42","profile":"reference"},{"config_digest":"0f48481659cbfa2c63674a43f462d55b1555fbfa8dd00e42f74de2784623b0d4","id":"reference--energy-paul--mpi-2x48","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"coupling":0.7853981633974483,"cutoff":8,"lower_atol":3e-06,"num_layers":20,"num_qubits":127,"observable_qubit":62,"theta":0.7853981633974483},"cpu_allocation":"0-95","has_mpi":true,"index_bits":32,"ranks":2,"routing":{"bits":1,"mode":"linear","seed":6768574230969066775},"threads":48},"measurement_kind":"pytest","node_id":"bench_models.py::test_model_energy[pauli]","operation":"energy","parameters_digest":"971c364c3cae8d46d276b9c9bffb1e14e23e023429863d9c674b92945e648a42","profile":"reference"},{"config_digest":"0f48481659cbfa2c63674a43f462d55b1555fbfa8dd00e42f74de2784623b0d4","id":"reference--energy-paul--off-1x96","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"coupling":0.7853981633974483,"cutoff":8,"lower_atol":3e-06,"num_layers":20,"num_qubits":127,"observable_qubit":62,"theta":0.7853981633974483},"cpu_allocation":"0-95","has_mpi":false,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":96},"measurement_kind":"pytest","node_id":"bench_models.py::test_model_energy[pauli]","operation":"energy","parameters_digest":"971c364c3cae8d46d276b9c9bffb1e14e23e023429863d9c674b92945e648a42","profile":"reference"},{"config_digest":"8a194c1d11504cba158ffde10e81136303cf6cc37c82cc3966bdf8abaa6d7f53","id":"reference--gradient-heis--mpi-1x1","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":400,"num_modes":128,"obs_terms":400000,"seed":0},"cpu_allocation":"0","has_mpi":true,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":1},"measurement_kind":"pytest","node_id":"bench_random.py::test_random_gradient[heisenberg]","operation":"gradient","parameters_digest":"2ee633f84e9f2df140694f3366370774debc7cf4931449a109e276cb6b0f7dcf","profile":"reference"},{"config_digest":"8a194c1d11504cba158ffde10e81136303cf6cc37c82cc3966bdf8abaa6d7f53","id":"reference--gradient-heis--mpi-1x96","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":400,"num_modes":128,"obs_terms":400000,"seed":0},"cpu_allocation":"0-95","has_mpi":true,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":96},"measurement_kind":"pytest","node_id":"bench_random.py::test_random_gradient[heisenberg]","operation":"gradient","parameters_digest":"2ee633f84e9f2df140694f3366370774debc7cf4931449a109e276cb6b0f7dcf","profile":"reference"},{"config_digest":"8a194c1d11504cba158ffde10e81136303cf6cc37c82cc3966bdf8abaa6d7f53","id":"reference--gradient-heis--mpi-2x48","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":400,"num_modes":128,"obs_terms":400000,"seed":0},"cpu_allocation":"0-95","has_mpi":true,"index_bits":32,"ranks":2,"routing":{"bits":1,"mode":"linear","seed":6768574230969066775},"threads":48},"measurement_kind":"pytest","node_id":"bench_random.py::test_random_gradient[heisenberg]","operation":"gradient","parameters_digest":"2ee633f84e9f2df140694f3366370774debc7cf4931449a109e276cb6b0f7dcf","profile":"reference"},{"config_digest":"8a194c1d11504cba158ffde10e81136303cf6cc37c82cc3966bdf8abaa6d7f53","id":"reference--gradient-heis--off-1x96","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":400,"num_modes":128,"obs_terms":400000,"seed":0},"cpu_allocation":"0-95","has_mpi":false,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":96},"measurement_kind":"pytest","node_id":"bench_random.py::test_random_gradient[heisenberg]","operation":"gradient","parameters_digest":"2ee633f84e9f2df140694f3366370774debc7cf4931449a109e276cb6b0f7dcf","profile":"reference"},{"config_digest":"84d8747d842ac221af6a1bb980015748fd049857489d1cfe3c87569509dc440e","id":"reference--gradient-schr--mpi-1x1","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":400,"num_modes":128,"obs_terms":10000,"seed":0},"cpu_allocation":"0","has_mpi":true,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":1},"measurement_kind":"pytest","node_id":"bench_random.py::test_random_gradient[schrodinger]","operation":"gradient","parameters_digest":"73a43144ad39c4dc12e9e0d4695e09a80ac75511776587ba072a83daf596782a","profile":"reference"},{"config_digest":"84d8747d842ac221af6a1bb980015748fd049857489d1cfe3c87569509dc440e","id":"reference--gradient-schr--mpi-1x96","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":400,"num_modes":128,"obs_terms":10000,"seed":0},"cpu_allocation":"0-95","has_mpi":true,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":96},"measurement_kind":"pytest","node_id":"bench_random.py::test_random_gradient[schrodinger]","operation":"gradient","parameters_digest":"73a43144ad39c4dc12e9e0d4695e09a80ac75511776587ba072a83daf596782a","profile":"reference"},{"config_digest":"84d8747d842ac221af6a1bb980015748fd049857489d1cfe3c87569509dc440e","id":"reference--gradient-schr--mpi-2x48","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":400,"num_modes":128,"obs_terms":10000,"seed":0},"cpu_allocation":"0-95","has_mpi":true,"index_bits":32,"ranks":2,"routing":{"bits":1,"mode":"linear","seed":6768574230969066775},"threads":48},"measurement_kind":"pytest","node_id":"bench_random.py::test_random_gradient[schrodinger]","operation":"gradient","parameters_digest":"73a43144ad39c4dc12e9e0d4695e09a80ac75511776587ba072a83daf596782a","profile":"reference"},{"config_digest":"84d8747d842ac221af6a1bb980015748fd049857489d1cfe3c87569509dc440e","id":"reference--gradient-schr--off-1x96","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":400,"num_modes":128,"obs_terms":10000,"seed":0},"cpu_allocation":"0-95","has_mpi":false,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":96},"measurement_kind":"pytest","node_id":"bench_random.py::test_random_gradient[schrodinger]","operation":"gradient","parameters_digest":"73a43144ad39c4dc12e9e0d4695e09a80ac75511776587ba072a83daf596782a","profile":"reference"},{"config_digest":"b6f2b0b1a3be6d446330fedbc3ed742b798474823d26390c52ba0de7a888e922","id":"reference--gradient-hubb--mpi-1x1","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"chemical_potential":0.0,"cutoff":6,"hopping":1.0,"interaction":-2.0,"lower_atol":3e-05,"neel_start_spin":"down","num_sites":24,"observable_site":18,"observable_spin":"up","trotter_dt":0.2,"trotter_steps":2},"cpu_allocation":"0","has_mpi":true,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":1},"measurement_kind":"pytest","node_id":"bench_models.py::test_model_gradient[hubbard]","operation":"gradient","parameters_digest":"c0c7d11ee1836009d0abe437246dd4021c6ac7401c7f1f5c8b11a1bc4a8567a0","profile":"reference"},{"config_digest":"b6f2b0b1a3be6d446330fedbc3ed742b798474823d26390c52ba0de7a888e922","id":"reference--gradient-hubb--mpi-1x96","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"chemical_potential":0.0,"cutoff":6,"hopping":1.0,"interaction":-2.0,"lower_atol":3e-05,"neel_start_spin":"down","num_sites":24,"observable_site":18,"observable_spin":"up","trotter_dt":0.2,"trotter_steps":2},"cpu_allocation":"0-95","has_mpi":true,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":96},"measurement_kind":"pytest","node_id":"bench_models.py::test_model_gradient[hubbard]","operation":"gradient","parameters_digest":"c0c7d11ee1836009d0abe437246dd4021c6ac7401c7f1f5c8b11a1bc4a8567a0","profile":"reference"},{"config_digest":"b6f2b0b1a3be6d446330fedbc3ed742b798474823d26390c52ba0de7a888e922","id":"reference--gradient-hubb--mpi-2x48","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"chemical_potential":0.0,"cutoff":6,"hopping":1.0,"interaction":-2.0,"lower_atol":3e-05,"neel_start_spin":"down","num_sites":24,"observable_site":18,"observable_spin":"up","trotter_dt":0.2,"trotter_steps":2},"cpu_allocation":"0-95","has_mpi":true,"index_bits":32,"ranks":2,"routing":{"bits":1,"mode":"linear","seed":6768574230969066775},"threads":48},"measurement_kind":"pytest","node_id":"bench_models.py::test_model_gradient[hubbard]","operation":"gradient","parameters_digest":"c0c7d11ee1836009d0abe437246dd4021c6ac7401c7f1f5c8b11a1bc4a8567a0","profile":"reference"},{"config_digest":"b6f2b0b1a3be6d446330fedbc3ed742b798474823d26390c52ba0de7a888e922","id":"reference--gradient-hubb--off-1x96","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"chemical_potential":0.0,"cutoff":6,"hopping":1.0,"interaction":-2.0,"lower_atol":3e-05,"neel_start_spin":"down","num_sites":24,"observable_site":18,"observable_spin":"up","trotter_dt":0.2,"trotter_steps":2},"cpu_allocation":"0-95","has_mpi":false,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":96},"measurement_kind":"pytest","node_id":"bench_models.py::test_model_gradient[hubbard]","operation":"gradient","parameters_digest":"c0c7d11ee1836009d0abe437246dd4021c6ac7401c7f1f5c8b11a1bc4a8567a0","profile":"reference"},{"config_digest":"0f48481659cbfa2c63674a43f462d55b1555fbfa8dd00e42f74de2784623b0d4","id":"reference--gradient-paul--mpi-1x1","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"coupling":0.7853981633974483,"cutoff":8,"lower_atol":3e-06,"num_layers":20,"num_qubits":127,"observable_qubit":62,"theta":0.7853981633974483},"cpu_allocation":"0","has_mpi":true,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":1},"measurement_kind":"pytest","node_id":"bench_models.py::test_model_gradient[pauli]","operation":"gradient","parameters_digest":"971c364c3cae8d46d276b9c9bffb1e14e23e023429863d9c674b92945e648a42","profile":"reference"},{"config_digest":"0f48481659cbfa2c63674a43f462d55b1555fbfa8dd00e42f74de2784623b0d4","id":"reference--gradient-paul--mpi-1x96","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"coupling":0.7853981633974483,"cutoff":8,"lower_atol":3e-06,"num_layers":20,"num_qubits":127,"observable_qubit":62,"theta":0.7853981633974483},"cpu_allocation":"0-95","has_mpi":true,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":96},"measurement_kind":"pytest","node_id":"bench_models.py::test_model_gradient[pauli]","operation":"gradient","parameters_digest":"971c364c3cae8d46d276b9c9bffb1e14e23e023429863d9c674b92945e648a42","profile":"reference"},{"config_digest":"0f48481659cbfa2c63674a43f462d55b1555fbfa8dd00e42f74de2784623b0d4","id":"reference--gradient-paul--mpi-2x48","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"coupling":0.7853981633974483,"cutoff":8,"lower_atol":3e-06,"num_layers":20,"num_qubits":127,"observable_qubit":62,"theta":0.7853981633974483},"cpu_allocation":"0-95","has_mpi":true,"index_bits":32,"ranks":2,"routing":{"bits":1,"mode":"linear","seed":6768574230969066775},"threads":48},"measurement_kind":"pytest","node_id":"bench_models.py::test_model_gradient[pauli]","operation":"gradient","parameters_digest":"971c364c3cae8d46d276b9c9bffb1e14e23e023429863d9c674b92945e648a42","profile":"reference"},{"config_digest":"0f48481659cbfa2c63674a43f462d55b1555fbfa8dd00e42f74de2784623b0d4","id":"reference--gradient-paul--off-1x96","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"coupling":0.7853981633974483,"cutoff":8,"lower_atol":3e-06,"num_layers":20,"num_qubits":127,"observable_qubit":62,"theta":0.7853981633974483},"cpu_allocation":"0-95","has_mpi":false,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":96},"measurement_kind":"pytest","node_id":"bench_models.py::test_model_gradient[pauli]","operation":"gradient","parameters_digest":"971c364c3cae8d46d276b9c9bffb1e14e23e023429863d9c674b92945e648a42","profile":"reference"},{"config_digest":"8a194c1d11504cba158ffde10e81136303cf6cc37c82cc3966bdf8abaa6d7f53","id":"reference-pared--energy-heis--mpi-1x1","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":400,"num_modes":128,"obs_terms":400000,"seed":0},"cpu_allocation":"0","has_mpi":true,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":1},"measurement_kind":"pytest","node_id":"bench_random.py::test_random_energy[heisenberg]","operation":"energy","parameters_digest":"2ee633f84e9f2df140694f3366370774debc7cf4931449a109e276cb6b0f7dcf","profile":"reference-pared"},{"config_digest":"8a194c1d11504cba158ffde10e81136303cf6cc37c82cc3966bdf8abaa6d7f53","id":"reference-pared--energy-heis--mpi-1x96","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":400,"num_modes":128,"obs_terms":400000,"seed":0},"cpu_allocation":"0-95","has_mpi":true,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":96},"measurement_kind":"pytest","node_id":"bench_random.py::test_random_energy[heisenberg]","operation":"energy","parameters_digest":"2ee633f84e9f2df140694f3366370774debc7cf4931449a109e276cb6b0f7dcf","profile":"reference-pared"},{"config_digest":"8a194c1d11504cba158ffde10e81136303cf6cc37c82cc3966bdf8abaa6d7f53","id":"reference-pared--energy-heis--mpi-2x48","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":400,"num_modes":128,"obs_terms":400000,"seed":0},"cpu_allocation":"0-95","has_mpi":true,"index_bits":32,"ranks":2,"routing":{"bits":1,"mode":"linear","seed":6768574230969066775},"threads":48},"measurement_kind":"pytest","node_id":"bench_random.py::test_random_energy[heisenberg]","operation":"energy","parameters_digest":"2ee633f84e9f2df140694f3366370774debc7cf4931449a109e276cb6b0f7dcf","profile":"reference-pared"},{"config_digest":"8a194c1d11504cba158ffde10e81136303cf6cc37c82cc3966bdf8abaa6d7f53","id":"reference-pared--energy-heis--off-1x96","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":400,"num_modes":128,"obs_terms":400000,"seed":0},"cpu_allocation":"0-95","has_mpi":false,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":96},"measurement_kind":"pytest","node_id":"bench_random.py::test_random_energy[heisenberg]","operation":"energy","parameters_digest":"2ee633f84e9f2df140694f3366370774debc7cf4931449a109e276cb6b0f7dcf","profile":"reference-pared"},{"config_digest":"84d8747d842ac221af6a1bb980015748fd049857489d1cfe3c87569509dc440e","id":"reference-pared--energy-schr--mpi-1x1","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":400,"num_modes":128,"obs_terms":10000,"seed":0},"cpu_allocation":"0","has_mpi":true,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":1},"measurement_kind":"pytest","node_id":"bench_random.py::test_random_energy[schrodinger]","operation":"energy","parameters_digest":"73a43144ad39c4dc12e9e0d4695e09a80ac75511776587ba072a83daf596782a","profile":"reference-pared"},{"config_digest":"84d8747d842ac221af6a1bb980015748fd049857489d1cfe3c87569509dc440e","id":"reference-pared--energy-schr--mpi-1x96","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":400,"num_modes":128,"obs_terms":10000,"seed":0},"cpu_allocation":"0-95","has_mpi":true,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":96},"measurement_kind":"pytest","node_id":"bench_random.py::test_random_energy[schrodinger]","operation":"energy","parameters_digest":"73a43144ad39c4dc12e9e0d4695e09a80ac75511776587ba072a83daf596782a","profile":"reference-pared"},{"config_digest":"84d8747d842ac221af6a1bb980015748fd049857489d1cfe3c87569509dc440e","id":"reference-pared--energy-schr--mpi-2x48","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":400,"num_modes":128,"obs_terms":10000,"seed":0},"cpu_allocation":"0-95","has_mpi":true,"index_bits":32,"ranks":2,"routing":{"bits":1,"mode":"linear","seed":6768574230969066775},"threads":48},"measurement_kind":"pytest","node_id":"bench_random.py::test_random_energy[schrodinger]","operation":"energy","parameters_digest":"73a43144ad39c4dc12e9e0d4695e09a80ac75511776587ba072a83daf596782a","profile":"reference-pared"},{"config_digest":"84d8747d842ac221af6a1bb980015748fd049857489d1cfe3c87569509dc440e","id":"reference-pared--energy-schr--off-1x96","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":400,"num_modes":128,"obs_terms":10000,"seed":0},"cpu_allocation":"0-95","has_mpi":false,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":96},"measurement_kind":"pytest","node_id":"bench_random.py::test_random_energy[schrodinger]","operation":"energy","parameters_digest":"73a43144ad39c4dc12e9e0d4695e09a80ac75511776587ba072a83daf596782a","profile":"reference-pared"},{"config_digest":"b6f2b0b1a3be6d446330fedbc3ed742b798474823d26390c52ba0de7a888e922","id":"reference-pared--energy-hubb--mpi-1x1","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"chemical_potential":0.0,"cutoff":6,"hopping":1.0,"interaction":-2.0,"lower_atol":3e-05,"neel_start_spin":"down","num_sites":24,"observable_site":18,"observable_spin":"up","trotter_dt":0.2,"trotter_steps":2},"cpu_allocation":"0","has_mpi":true,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":1},"measurement_kind":"pytest","node_id":"bench_models.py::test_model_energy[hubbard]","operation":"energy","parameters_digest":"c0c7d11ee1836009d0abe437246dd4021c6ac7401c7f1f5c8b11a1bc4a8567a0","profile":"reference-pared"},{"config_digest":"b6f2b0b1a3be6d446330fedbc3ed742b798474823d26390c52ba0de7a888e922","id":"reference-pared--energy-hubb--mpi-1x96","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"chemical_potential":0.0,"cutoff":6,"hopping":1.0,"interaction":-2.0,"lower_atol":3e-05,"neel_start_spin":"down","num_sites":24,"observable_site":18,"observable_spin":"up","trotter_dt":0.2,"trotter_steps":2},"cpu_allocation":"0-95","has_mpi":true,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":96},"measurement_kind":"pytest","node_id":"bench_models.py::test_model_energy[hubbard]","operation":"energy","parameters_digest":"c0c7d11ee1836009d0abe437246dd4021c6ac7401c7f1f5c8b11a1bc4a8567a0","profile":"reference-pared"},{"config_digest":"b6f2b0b1a3be6d446330fedbc3ed742b798474823d26390c52ba0de7a888e922","id":"reference-pared--energy-hubb--mpi-2x48","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"chemical_potential":0.0,"cutoff":6,"hopping":1.0,"interaction":-2.0,"lower_atol":3e-05,"neel_start_spin":"down","num_sites":24,"observable_site":18,"observable_spin":"up","trotter_dt":0.2,"trotter_steps":2},"cpu_allocation":"0-95","has_mpi":true,"index_bits":32,"ranks":2,"routing":{"bits":1,"mode":"linear","seed":6768574230969066775},"threads":48},"measurement_kind":"pytest","node_id":"bench_models.py::test_model_energy[hubbard]","operation":"energy","parameters_digest":"c0c7d11ee1836009d0abe437246dd4021c6ac7401c7f1f5c8b11a1bc4a8567a0","profile":"reference-pared"},{"config_digest":"b6f2b0b1a3be6d446330fedbc3ed742b798474823d26390c52ba0de7a888e922","id":"reference-pared--energy-hubb--off-1x96","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"chemical_potential":0.0,"cutoff":6,"hopping":1.0,"interaction":-2.0,"lower_atol":3e-05,"neel_start_spin":"down","num_sites":24,"observable_site":18,"observable_spin":"up","trotter_dt":0.2,"trotter_steps":2},"cpu_allocation":"0-95","has_mpi":false,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":96},"measurement_kind":"pytest","node_id":"bench_models.py::test_model_energy[hubbard]","operation":"energy","parameters_digest":"c0c7d11ee1836009d0abe437246dd4021c6ac7401c7f1f5c8b11a1bc4a8567a0","profile":"reference-pared"},{"config_digest":"0f48481659cbfa2c63674a43f462d55b1555fbfa8dd00e42f74de2784623b0d4","id":"reference-pared--energy-paul--mpi-1x1","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"coupling":0.7853981633974483,"cutoff":8,"lower_atol":3e-06,"num_layers":20,"num_qubits":127,"observable_qubit":62,"theta":0.7853981633974483},"cpu_allocation":"0","has_mpi":true,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":1},"measurement_kind":"pytest","node_id":"bench_models.py::test_model_energy[pauli]","operation":"energy","parameters_digest":"971c364c3cae8d46d276b9c9bffb1e14e23e023429863d9c674b92945e648a42","profile":"reference-pared"},{"config_digest":"0f48481659cbfa2c63674a43f462d55b1555fbfa8dd00e42f74de2784623b0d4","id":"reference-pared--energy-paul--mpi-1x96","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"coupling":0.7853981633974483,"cutoff":8,"lower_atol":3e-06,"num_layers":20,"num_qubits":127,"observable_qubit":62,"theta":0.7853981633974483},"cpu_allocation":"0-95","has_mpi":true,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":96},"measurement_kind":"pytest","node_id":"bench_models.py::test_model_energy[pauli]","operation":"energy","parameters_digest":"971c364c3cae8d46d276b9c9bffb1e14e23e023429863d9c674b92945e648a42","profile":"reference-pared"},{"config_digest":"0f48481659cbfa2c63674a43f462d55b1555fbfa8dd00e42f74de2784623b0d4","id":"reference-pared--energy-paul--mpi-2x48","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"coupling":0.7853981633974483,"cutoff":8,"lower_atol":3e-06,"num_layers":20,"num_qubits":127,"observable_qubit":62,"theta":0.7853981633974483},"cpu_allocation":"0-95","has_mpi":true,"index_bits":32,"ranks":2,"routing":{"bits":1,"mode":"linear","seed":6768574230969066775},"threads":48},"measurement_kind":"pytest","node_id":"bench_models.py::test_model_energy[pauli]","operation":"energy","parameters_digest":"971c364c3cae8d46d276b9c9bffb1e14e23e023429863d9c674b92945e648a42","profile":"reference-pared"},{"config_digest":"0f48481659cbfa2c63674a43f462d55b1555fbfa8dd00e42f74de2784623b0d4","id":"reference-pared--energy-paul--off-1x96","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"coupling":0.7853981633974483,"cutoff":8,"lower_atol":3e-06,"num_layers":20,"num_qubits":127,"observable_qubit":62,"theta":0.7853981633974483},"cpu_allocation":"0-95","has_mpi":false,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":96},"measurement_kind":"pytest","node_id":"bench_models.py::test_model_energy[pauli]","operation":"energy","parameters_digest":"971c364c3cae8d46d276b9c9bffb1e14e23e023429863d9c674b92945e648a42","profile":"reference-pared"},{"config_digest":"8a194c1d11504cba158ffde10e81136303cf6cc37c82cc3966bdf8abaa6d7f53","id":"reference-pared--gradient-heis--mpi-1x1","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":400,"num_modes":128,"obs_terms":400000,"seed":0},"cpu_allocation":"0","has_mpi":true,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":1},"measurement_kind":"pytest","node_id":"bench_random.py::test_random_gradient[heisenberg]","operation":"gradient","parameters_digest":"2ee633f84e9f2df140694f3366370774debc7cf4931449a109e276cb6b0f7dcf","profile":"reference-pared"},{"config_digest":"8a194c1d11504cba158ffde10e81136303cf6cc37c82cc3966bdf8abaa6d7f53","id":"reference-pared--gradient-heis--mpi-1x96","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":400,"num_modes":128,"obs_terms":400000,"seed":0},"cpu_allocation":"0-95","has_mpi":true,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":96},"measurement_kind":"pytest","node_id":"bench_random.py::test_random_gradient[heisenberg]","operation":"gradient","parameters_digest":"2ee633f84e9f2df140694f3366370774debc7cf4931449a109e276cb6b0f7dcf","profile":"reference-pared"},{"config_digest":"8a194c1d11504cba158ffde10e81136303cf6cc37c82cc3966bdf8abaa6d7f53","id":"reference-pared--gradient-heis--mpi-2x48","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":400,"num_modes":128,"obs_terms":400000,"seed":0},"cpu_allocation":"0-95","has_mpi":true,"index_bits":32,"ranks":2,"routing":{"bits":1,"mode":"linear","seed":6768574230969066775},"threads":48},"measurement_kind":"pytest","node_id":"bench_random.py::test_random_gradient[heisenberg]","operation":"gradient","parameters_digest":"2ee633f84e9f2df140694f3366370774debc7cf4931449a109e276cb6b0f7dcf","profile":"reference-pared"},{"config_digest":"8a194c1d11504cba158ffde10e81136303cf6cc37c82cc3966bdf8abaa6d7f53","id":"reference-pared--gradient-heis--off-1x96","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":400,"num_modes":128,"obs_terms":400000,"seed":0},"cpu_allocation":"0-95","has_mpi":false,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":96},"measurement_kind":"pytest","node_id":"bench_random.py::test_random_gradient[heisenberg]","operation":"gradient","parameters_digest":"2ee633f84e9f2df140694f3366370774debc7cf4931449a109e276cb6b0f7dcf","profile":"reference-pared"},{"config_digest":"84d8747d842ac221af6a1bb980015748fd049857489d1cfe3c87569509dc440e","id":"reference-pared--gradient-schr--mpi-1x1","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":400,"num_modes":128,"obs_terms":10000,"seed":0},"cpu_allocation":"0","has_mpi":true,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":1},"measurement_kind":"pytest","node_id":"bench_random.py::test_random_gradient[schrodinger]","operation":"gradient","parameters_digest":"73a43144ad39c4dc12e9e0d4695e09a80ac75511776587ba072a83daf596782a","profile":"reference-pared"},{"config_digest":"84d8747d842ac221af6a1bb980015748fd049857489d1cfe3c87569509dc440e","id":"reference-pared--gradient-schr--mpi-1x96","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":400,"num_modes":128,"obs_terms":10000,"seed":0},"cpu_allocation":"0-95","has_mpi":true,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":96},"measurement_kind":"pytest","node_id":"bench_random.py::test_random_gradient[schrodinger]","operation":"gradient","parameters_digest":"73a43144ad39c4dc12e9e0d4695e09a80ac75511776587ba072a83daf596782a","profile":"reference-pared"},{"config_digest":"84d8747d842ac221af6a1bb980015748fd049857489d1cfe3c87569509dc440e","id":"reference-pared--gradient-schr--mpi-2x48","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":400,"num_modes":128,"obs_terms":10000,"seed":0},"cpu_allocation":"0-95","has_mpi":true,"index_bits":32,"ranks":2,"routing":{"bits":1,"mode":"linear","seed":6768574230969066775},"threads":48},"measurement_kind":"pytest","node_id":"bench_random.py::test_random_gradient[schrodinger]","operation":"gradient","parameters_digest":"73a43144ad39c4dc12e9e0d4695e09a80ac75511776587ba072a83daf596782a","profile":"reference-pared"},{"config_digest":"84d8747d842ac221af6a1bb980015748fd049857489d1cfe3c87569509dc440e","id":"reference-pared--gradient-schr--off-1x96","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":400,"num_modes":128,"obs_terms":10000,"seed":0},"cpu_allocation":"0-95","has_mpi":false,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":96},"measurement_kind":"pytest","node_id":"bench_random.py::test_random_gradient[schrodinger]","operation":"gradient","parameters_digest":"73a43144ad39c4dc12e9e0d4695e09a80ac75511776587ba072a83daf596782a","profile":"reference-pared"},{"config_digest":"b6f2b0b1a3be6d446330fedbc3ed742b798474823d26390c52ba0de7a888e922","id":"reference-pared--gradient-hubb--mpi-1x1","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"chemical_potential":0.0,"cutoff":6,"hopping":1.0,"interaction":-2.0,"lower_atol":3e-05,"neel_start_spin":"down","num_sites":24,"observable_site":18,"observable_spin":"up","trotter_dt":0.2,"trotter_steps":2},"cpu_allocation":"0","has_mpi":true,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":1},"measurement_kind":"pytest","node_id":"bench_models.py::test_model_gradient[hubbard]","operation":"gradient","parameters_digest":"c0c7d11ee1836009d0abe437246dd4021c6ac7401c7f1f5c8b11a1bc4a8567a0","profile":"reference-pared"},{"config_digest":"b6f2b0b1a3be6d446330fedbc3ed742b798474823d26390c52ba0de7a888e922","id":"reference-pared--gradient-hubb--mpi-1x96","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"chemical_potential":0.0,"cutoff":6,"hopping":1.0,"interaction":-2.0,"lower_atol":3e-05,"neel_start_spin":"down","num_sites":24,"observable_site":18,"observable_spin":"up","trotter_dt":0.2,"trotter_steps":2},"cpu_allocation":"0-95","has_mpi":true,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":96},"measurement_kind":"pytest","node_id":"bench_models.py::test_model_gradient[hubbard]","operation":"gradient","parameters_digest":"c0c7d11ee1836009d0abe437246dd4021c6ac7401c7f1f5c8b11a1bc4a8567a0","profile":"reference-pared"},{"config_digest":"b6f2b0b1a3be6d446330fedbc3ed742b798474823d26390c52ba0de7a888e922","id":"reference-pared--gradient-hubb--mpi-2x48","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"chemical_potential":0.0,"cutoff":6,"hopping":1.0,"interaction":-2.0,"lower_atol":3e-05,"neel_start_spin":"down","num_sites":24,"observable_site":18,"observable_spin":"up","trotter_dt":0.2,"trotter_steps":2},"cpu_allocation":"0-95","has_mpi":true,"index_bits":32,"ranks":2,"routing":{"bits":1,"mode":"linear","seed":6768574230969066775},"threads":48},"measurement_kind":"pytest","node_id":"bench_models.py::test_model_gradient[hubbard]","operation":"gradient","parameters_digest":"c0c7d11ee1836009d0abe437246dd4021c6ac7401c7f1f5c8b11a1bc4a8567a0","profile":"reference-pared"},{"config_digest":"b6f2b0b1a3be6d446330fedbc3ed742b798474823d26390c52ba0de7a888e922","id":"reference-pared--gradient-hubb--off-1x96","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"chemical_potential":0.0,"cutoff":6,"hopping":1.0,"interaction":-2.0,"lower_atol":3e-05,"neel_start_spin":"down","num_sites":24,"observable_site":18,"observable_spin":"up","trotter_dt":0.2,"trotter_steps":2},"cpu_allocation":"0-95","has_mpi":false,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":96},"measurement_kind":"pytest","node_id":"bench_models.py::test_model_gradient[hubbard]","operation":"gradient","parameters_digest":"c0c7d11ee1836009d0abe437246dd4021c6ac7401c7f1f5c8b11a1bc4a8567a0","profile":"reference-pared"},{"config_digest":"0f48481659cbfa2c63674a43f462d55b1555fbfa8dd00e42f74de2784623b0d4","id":"reference-pared--gradient-paul--mpi-1x1","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"coupling":0.7853981633974483,"cutoff":8,"lower_atol":3e-06,"num_layers":20,"num_qubits":127,"observable_qubit":62,"theta":0.7853981633974483},"cpu_allocation":"0","has_mpi":true,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":1},"measurement_kind":"pytest","node_id":"bench_models.py::test_model_gradient[pauli]","operation":"gradient","parameters_digest":"971c364c3cae8d46d276b9c9bffb1e14e23e023429863d9c674b92945e648a42","profile":"reference-pared"},{"config_digest":"0f48481659cbfa2c63674a43f462d55b1555fbfa8dd00e42f74de2784623b0d4","id":"reference-pared--gradient-paul--mpi-1x96","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"coupling":0.7853981633974483,"cutoff":8,"lower_atol":3e-06,"num_layers":20,"num_qubits":127,"observable_qubit":62,"theta":0.7853981633974483},"cpu_allocation":"0-95","has_mpi":true,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":96},"measurement_kind":"pytest","node_id":"bench_models.py::test_model_gradient[pauli]","operation":"gradient","parameters_digest":"971c364c3cae8d46d276b9c9bffb1e14e23e023429863d9c674b92945e648a42","profile":"reference-pared"},{"config_digest":"0f48481659cbfa2c63674a43f462d55b1555fbfa8dd00e42f74de2784623b0d4","id":"reference-pared--gradient-paul--mpi-2x48","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"coupling":0.7853981633974483,"cutoff":8,"lower_atol":3e-06,"num_layers":20,"num_qubits":127,"observable_qubit":62,"theta":0.7853981633974483},"cpu_allocation":"0-95","has_mpi":true,"index_bits":32,"ranks":2,"routing":{"bits":1,"mode":"linear","seed":6768574230969066775},"threads":48},"measurement_kind":"pytest","node_id":"bench_models.py::test_model_gradient[pauli]","operation":"gradient","parameters_digest":"971c364c3cae8d46d276b9c9bffb1e14e23e023429863d9c674b92945e648a42","profile":"reference-pared"},{"config_digest":"0f48481659cbfa2c63674a43f462d55b1555fbfa8dd00e42f74de2784623b0d4","id":"reference-pared--gradient-paul--off-1x96","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"coupling":0.7853981633974483,"cutoff":8,"lower_atol":3e-06,"num_layers":20,"num_qubits":127,"observable_qubit":62,"theta":0.7853981633974483},"cpu_allocation":"0-95","has_mpi":false,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":96},"measurement_kind":"pytest","node_id":"bench_models.py::test_model_gradient[pauli]","operation":"gradient","parameters_digest":"971c364c3cae8d46d276b9c9bffb1e14e23e023429863d9c674b92945e648a42","profile":"reference-pared"},{"config_digest":"8a194c1d11504cba158ffde10e81136303cf6cc37c82cc3966bdf8abaa6d7f53","id":"reference-pared--pare-construct-heis--mpi-1x1","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":400,"num_modes":128,"obs_terms":400000,"seed":0},"cpu_allocation":"0","has_mpi":true,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":1},"measurement_kind":"driver","node_id":"driver::pare_functional_construct[heisenberg]","operation":"pare_functional_construct","parameters_digest":"2ee633f84e9f2df140694f3366370774debc7cf4931449a109e276cb6b0f7dcf","profile":"reference-pared"},{"config_digest":"8a194c1d11504cba158ffde10e81136303cf6cc37c82cc3966bdf8abaa6d7f53","id":"reference-pared--pare-construct-heis--mpi-1x96","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":400,"num_modes":128,"obs_terms":400000,"seed":0},"cpu_allocation":"0-95","has_mpi":true,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":96},"measurement_kind":"driver","node_id":"driver::pare_functional_construct[heisenberg]","operation":"pare_functional_construct","parameters_digest":"2ee633f84e9f2df140694f3366370774debc7cf4931449a109e276cb6b0f7dcf","profile":"reference-pared"},{"config_digest":"8a194c1d11504cba158ffde10e81136303cf6cc37c82cc3966bdf8abaa6d7f53","id":"reference-pared--pare-construct-heis--mpi-2x48","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":400,"num_modes":128,"obs_terms":400000,"seed":0},"cpu_allocation":"0-95","has_mpi":true,"index_bits":32,"ranks":2,"routing":{"bits":1,"mode":"linear","seed":6768574230969066775},"threads":48},"measurement_kind":"driver","node_id":"driver::pare_functional_construct[heisenberg]","operation":"pare_functional_construct","parameters_digest":"2ee633f84e9f2df140694f3366370774debc7cf4931449a109e276cb6b0f7dcf","profile":"reference-pared"},{"config_digest":"8a194c1d11504cba158ffde10e81136303cf6cc37c82cc3966bdf8abaa6d7f53","id":"reference-pared--pare-construct-heis--off-1x96","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":400,"num_modes":128,"obs_terms":400000,"seed":0},"cpu_allocation":"0-95","has_mpi":false,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":96},"measurement_kind":"driver","node_id":"driver::pare_functional_construct[heisenberg]","operation":"pare_functional_construct","parameters_digest":"2ee633f84e9f2df140694f3366370774debc7cf4931449a109e276cb6b0f7dcf","profile":"reference-pared"},{"config_digest":"84d8747d842ac221af6a1bb980015748fd049857489d1cfe3c87569509dc440e","id":"reference-pared--pare-construct-schr--mpi-1x1","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":400,"num_modes":128,"obs_terms":10000,"seed":0},"cpu_allocation":"0","has_mpi":true,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":1},"measurement_kind":"driver","node_id":"driver::pare_functional_construct[schrodinger]","operation":"pare_functional_construct","parameters_digest":"73a43144ad39c4dc12e9e0d4695e09a80ac75511776587ba072a83daf596782a","profile":"reference-pared"},{"config_digest":"84d8747d842ac221af6a1bb980015748fd049857489d1cfe3c87569509dc440e","id":"reference-pared--pare-construct-schr--mpi-1x96","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":400,"num_modes":128,"obs_terms":10000,"seed":0},"cpu_allocation":"0-95","has_mpi":true,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":96},"measurement_kind":"driver","node_id":"driver::pare_functional_construct[schrodinger]","operation":"pare_functional_construct","parameters_digest":"73a43144ad39c4dc12e9e0d4695e09a80ac75511776587ba072a83daf596782a","profile":"reference-pared"},{"config_digest":"84d8747d842ac221af6a1bb980015748fd049857489d1cfe3c87569509dc440e","id":"reference-pared--pare-construct-schr--mpi-2x48","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":400,"num_modes":128,"obs_terms":10000,"seed":0},"cpu_allocation":"0-95","has_mpi":true,"index_bits":32,"ranks":2,"routing":{"bits":1,"mode":"linear","seed":6768574230969066775},"threads":48},"measurement_kind":"driver","node_id":"driver::pare_functional_construct[schrodinger]","operation":"pare_functional_construct","parameters_digest":"73a43144ad39c4dc12e9e0d4695e09a80ac75511776587ba072a83daf596782a","profile":"reference-pared"},{"config_digest":"84d8747d842ac221af6a1bb980015748fd049857489d1cfe3c87569509dc440e","id":"reference-pared--pare-construct-schr--off-1x96","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":400,"num_modes":128,"obs_terms":10000,"seed":0},"cpu_allocation":"0-95","has_mpi":false,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":96},"measurement_kind":"driver","node_id":"driver::pare_functional_construct[schrodinger]","operation":"pare_functional_construct","parameters_digest":"73a43144ad39c4dc12e9e0d4695e09a80ac75511776587ba072a83daf596782a","profile":"reference-pared"},{"config_digest":"8a194c1d11504cba158ffde10e81136303cf6cc37c82cc3966bdf8abaa6d7f53","id":"reference--propagate-heis--mpi-3x32-splitmix","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":400,"num_modes":128,"obs_terms":400000,"seed":0},"cpu_allocation":"0-95","has_mpi":true,"index_bits":32,"ranks":3,"routing":{"bits":0,"mode":"splitmix","seed":6768574230969066775},"threads":32},"measurement_kind":"pytest","node_id":"bench_random.py::test_random_propagate[heisenberg]","operation":"propagate","parameters_digest":"2ee633f84e9f2df140694f3366370774debc7cf4931449a109e276cb6b0f7dcf","profile":"reference"},{"config_digest":"8a194c1d11504cba158ffde10e81136303cf6cc37c82cc3966bdf8abaa6d7f53","id":"reference--propagate-heis--mpi-4x24-splitmix","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":400,"num_modes":128,"obs_terms":400000,"seed":0},"cpu_allocation":"0-95","has_mpi":true,"index_bits":32,"ranks":4,"routing":{"bits":0,"mode":"splitmix","seed":6768574230969066775},"threads":24},"measurement_kind":"pytest","node_id":"bench_random.py::test_random_propagate[heisenberg]","operation":"propagate","parameters_digest":"2ee633f84e9f2df140694f3366370774debc7cf4931449a109e276cb6b0f7dcf","profile":"reference"},{"config_digest":"8a194c1d11504cba158ffde10e81136303cf6cc37c82cc3966bdf8abaa6d7f53","id":"reference--propagate-heis--mpi-4x24-linear","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":400,"num_modes":128,"obs_terms":400000,"seed":0},"cpu_allocation":"0-95","has_mpi":true,"index_bits":32,"ranks":4,"routing":{"bits":2,"mode":"linear","seed":6768574230969066775},"threads":24},"measurement_kind":"pytest","node_id":"bench_random.py::test_random_propagate[heisenberg]","operation":"propagate","parameters_digest":"2ee633f84e9f2df140694f3366370774debc7cf4931449a109e276cb6b0f7dcf","profile":"reference"},{"config_digest":"8a194c1d11504cba158ffde10e81136303cf6cc37c82cc3966bdf8abaa6d7f53","id":"reference--build-graph-heis--mpi-3x32-splitmix","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":400,"num_modes":128,"obs_terms":400000,"seed":0},"cpu_allocation":"0-95","has_mpi":true,"index_bits":32,"ranks":3,"routing":{"bits":0,"mode":"splitmix","seed":6768574230969066775},"threads":32},"measurement_kind":"pytest","node_id":"bench_random.py::test_random_build_graph[heisenberg]","operation":"build_graph","parameters_digest":"2ee633f84e9f2df140694f3366370774debc7cf4931449a109e276cb6b0f7dcf","profile":"reference"},{"config_digest":"8a194c1d11504cba158ffde10e81136303cf6cc37c82cc3966bdf8abaa6d7f53","id":"reference--build-graph-heis--mpi-4x24-splitmix","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":400,"num_modes":128,"obs_terms":400000,"seed":0},"cpu_allocation":"0-95","has_mpi":true,"index_bits":32,"ranks":4,"routing":{"bits":0,"mode":"splitmix","seed":6768574230969066775},"threads":24},"measurement_kind":"pytest","node_id":"bench_random.py::test_random_build_graph[heisenberg]","operation":"build_graph","parameters_digest":"2ee633f84e9f2df140694f3366370774debc7cf4931449a109e276cb6b0f7dcf","profile":"reference"},{"config_digest":"8a194c1d11504cba158ffde10e81136303cf6cc37c82cc3966bdf8abaa6d7f53","id":"reference--build-graph-heis--mpi-4x24-linear","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":400,"num_modes":128,"obs_terms":400000,"seed":0},"cpu_allocation":"0-95","has_mpi":true,"index_bits":32,"ranks":4,"routing":{"bits":2,"mode":"linear","seed":6768574230969066775},"threads":24},"measurement_kind":"pytest","node_id":"bench_random.py::test_random_build_graph[heisenberg]","operation":"build_graph","parameters_digest":"2ee633f84e9f2df140694f3366370774debc7cf4931449a109e276cb6b0f7dcf","profile":"reference"},{"config_digest":"84d8747d842ac221af6a1bb980015748fd049857489d1cfe3c87569509dc440e","id":"reference--propagate-schr--mpi-3x32-splitmix","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":400,"num_modes":128,"obs_terms":10000,"seed":0},"cpu_allocation":"0-95","has_mpi":true,"index_bits":32,"ranks":3,"routing":{"bits":0,"mode":"splitmix","seed":6768574230969066775},"threads":32},"measurement_kind":"pytest","node_id":"bench_random.py::test_random_propagate[schrodinger]","operation":"propagate","parameters_digest":"73a43144ad39c4dc12e9e0d4695e09a80ac75511776587ba072a83daf596782a","profile":"reference"},{"config_digest":"84d8747d842ac221af6a1bb980015748fd049857489d1cfe3c87569509dc440e","id":"reference--propagate-schr--mpi-4x24-splitmix","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":400,"num_modes":128,"obs_terms":10000,"seed":0},"cpu_allocation":"0-95","has_mpi":true,"index_bits":32,"ranks":4,"routing":{"bits":0,"mode":"splitmix","seed":6768574230969066775},"threads":24},"measurement_kind":"pytest","node_id":"bench_random.py::test_random_propagate[schrodinger]","operation":"propagate","parameters_digest":"73a43144ad39c4dc12e9e0d4695e09a80ac75511776587ba072a83daf596782a","profile":"reference"},{"config_digest":"84d8747d842ac221af6a1bb980015748fd049857489d1cfe3c87569509dc440e","id":"reference--propagate-schr--mpi-4x24-linear","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":400,"num_modes":128,"obs_terms":10000,"seed":0},"cpu_allocation":"0-95","has_mpi":true,"index_bits":32,"ranks":4,"routing":{"bits":2,"mode":"linear","seed":6768574230969066775},"threads":24},"measurement_kind":"pytest","node_id":"bench_random.py::test_random_propagate[schrodinger]","operation":"propagate","parameters_digest":"73a43144ad39c4dc12e9e0d4695e09a80ac75511776587ba072a83daf596782a","profile":"reference"},{"config_digest":"84d8747d842ac221af6a1bb980015748fd049857489d1cfe3c87569509dc440e","id":"reference--build-graph-schr--mpi-3x32-splitmix","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":400,"num_modes":128,"obs_terms":10000,"seed":0},"cpu_allocation":"0-95","has_mpi":true,"index_bits":32,"ranks":3,"routing":{"bits":0,"mode":"splitmix","seed":6768574230969066775},"threads":32},"measurement_kind":"pytest","node_id":"bench_random.py::test_random_build_graph[schrodinger]","operation":"build_graph","parameters_digest":"73a43144ad39c4dc12e9e0d4695e09a80ac75511776587ba072a83daf596782a","profile":"reference"},{"config_digest":"84d8747d842ac221af6a1bb980015748fd049857489d1cfe3c87569509dc440e","id":"reference--build-graph-schr--mpi-4x24-splitmix","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":400,"num_modes":128,"obs_terms":10000,"seed":0},"cpu_allocation":"0-95","has_mpi":true,"index_bits":32,"ranks":4,"routing":{"bits":0,"mode":"splitmix","seed":6768574230969066775},"threads":24},"measurement_kind":"pytest","node_id":"bench_random.py::test_random_build_graph[schrodinger]","operation":"build_graph","parameters_digest":"73a43144ad39c4dc12e9e0d4695e09a80ac75511776587ba072a83daf596782a","profile":"reference"},{"config_digest":"84d8747d842ac221af6a1bb980015748fd049857489d1cfe3c87569509dc440e","id":"reference--build-graph-schr--mpi-4x24-linear","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":400,"num_modes":128,"obs_terms":10000,"seed":0},"cpu_allocation":"0-95","has_mpi":true,"index_bits":32,"ranks":4,"routing":{"bits":2,"mode":"linear","seed":6768574230969066775},"threads":24},"measurement_kind":"pytest","node_id":"bench_random.py::test_random_build_graph[schrodinger]","operation":"build_graph","parameters_digest":"73a43144ad39c4dc12e9e0d4695e09a80ac75511776587ba072a83daf596782a","profile":"reference"},{"config_digest":"382aef8b540cf9b9b38fe376a08d199b65345511cb112659abc8552cad1b9312","id":"large--propagate-heis--mpi-1x96","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":400,"num_modes":128,"obs_terms":2000000,"seed":0},"cpu_allocation":"0-95","has_mpi":true,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":96},"measurement_kind":"pytest","node_id":"bench_random.py::test_random_propagate[heisenberg]","operation":"propagate","parameters_digest":"dd73f7973d0d03487692201e103588d92f6c51ed1fdae8585e1adbf8c925c43f","profile":"large"},{"config_digest":"382aef8b540cf9b9b38fe376a08d199b65345511cb112659abc8552cad1b9312","id":"large--propagate-heis--mpi-2x48","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":400,"num_modes":128,"obs_terms":2000000,"seed":0},"cpu_allocation":"0-95","has_mpi":true,"index_bits":32,"ranks":2,"routing":{"bits":1,"mode":"linear","seed":6768574230969066775},"threads":48},"measurement_kind":"pytest","node_id":"bench_random.py::test_random_propagate[heisenberg]","operation":"propagate","parameters_digest":"dd73f7973d0d03487692201e103588d92f6c51ed1fdae8585e1adbf8c925c43f","profile":"large"},{"config_digest":"382aef8b540cf9b9b38fe376a08d199b65345511cb112659abc8552cad1b9312","id":"large--propagate-heis--off-1x96","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":400,"num_modes":128,"obs_terms":2000000,"seed":0},"cpu_allocation":"0-95","has_mpi":false,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":96},"measurement_kind":"pytest","node_id":"bench_random.py::test_random_propagate[heisenberg]","operation":"propagate","parameters_digest":"dd73f7973d0d03487692201e103588d92f6c51ed1fdae8585e1adbf8c925c43f","profile":"large"},{"config_digest":"382aef8b540cf9b9b38fe376a08d199b65345511cb112659abc8552cad1b9312","id":"large--build-graph-heis--mpi-1x96","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":400,"num_modes":128,"obs_terms":2000000,"seed":0},"cpu_allocation":"0-95","has_mpi":true,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":96},"measurement_kind":"pytest","node_id":"bench_random.py::test_random_build_graph[heisenberg]","operation":"build_graph","parameters_digest":"dd73f7973d0d03487692201e103588d92f6c51ed1fdae8585e1adbf8c925c43f","profile":"large"},{"config_digest":"382aef8b540cf9b9b38fe376a08d199b65345511cb112659abc8552cad1b9312","id":"large--build-graph-heis--mpi-2x48","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":400,"num_modes":128,"obs_terms":2000000,"seed":0},"cpu_allocation":"0-95","has_mpi":true,"index_bits":32,"ranks":2,"routing":{"bits":1,"mode":"linear","seed":6768574230969066775},"threads":48},"measurement_kind":"pytest","node_id":"bench_random.py::test_random_build_graph[heisenberg]","operation":"build_graph","parameters_digest":"dd73f7973d0d03487692201e103588d92f6c51ed1fdae8585e1adbf8c925c43f","profile":"large"},{"config_digest":"382aef8b540cf9b9b38fe376a08d199b65345511cb112659abc8552cad1b9312","id":"large--build-graph-heis--off-1x96","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":400,"num_modes":128,"obs_terms":2000000,"seed":0},"cpu_allocation":"0-95","has_mpi":false,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":96},"measurement_kind":"pytest","node_id":"bench_random.py::test_random_build_graph[heisenberg]","operation":"build_graph","parameters_digest":"dd73f7973d0d03487692201e103588d92f6c51ed1fdae8585e1adbf8c925c43f","profile":"large"},{"config_digest":"382aef8b540cf9b9b38fe376a08d199b65345511cb112659abc8552cad1b9312","id":"large--gradient-heis--mpi-1x96","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":400,"num_modes":128,"obs_terms":2000000,"seed":0},"cpu_allocation":"0-95","has_mpi":true,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":96},"measurement_kind":"pytest","node_id":"bench_random.py::test_random_gradient[heisenberg]","operation":"gradient","parameters_digest":"dd73f7973d0d03487692201e103588d92f6c51ed1fdae8585e1adbf8c925c43f","profile":"large"},{"config_digest":"382aef8b540cf9b9b38fe376a08d199b65345511cb112659abc8552cad1b9312","id":"large--gradient-heis--mpi-2x48","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":400,"num_modes":128,"obs_terms":2000000,"seed":0},"cpu_allocation":"0-95","has_mpi":true,"index_bits":32,"ranks":2,"routing":{"bits":1,"mode":"linear","seed":6768574230969066775},"threads":48},"measurement_kind":"pytest","node_id":"bench_random.py::test_random_gradient[heisenberg]","operation":"gradient","parameters_digest":"dd73f7973d0d03487692201e103588d92f6c51ed1fdae8585e1adbf8c925c43f","profile":"large"},{"config_digest":"382aef8b540cf9b9b38fe376a08d199b65345511cb112659abc8552cad1b9312","id":"large--gradient-heis--off-1x96","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":400,"num_modes":128,"obs_terms":2000000,"seed":0},"cpu_allocation":"0-95","has_mpi":false,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":96},"measurement_kind":"pytest","node_id":"bench_random.py::test_random_gradient[heisenberg]","operation":"gradient","parameters_digest":"dd73f7973d0d03487692201e103588d92f6c51ed1fdae8585e1adbf8c925c43f","profile":"large"},{"config_digest":"a52c48a911878c1caf5abd2d693d17dee9b597ef5313635a50552b36b13cd807","id":"large--propagate-schr--mpi-1x96","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":1000,"num_modes":160,"obs_terms":10000,"seed":0},"cpu_allocation":"0-95","has_mpi":true,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":96},"measurement_kind":"pytest","node_id":"bench_random.py::test_random_propagate[schrodinger]","operation":"propagate","parameters_digest":"0b0892a9f5ab8eea64986cd9b0a0dddad24f94a21d42e0ce163162496630536f","profile":"large"},{"config_digest":"a52c48a911878c1caf5abd2d693d17dee9b597ef5313635a50552b36b13cd807","id":"large--propagate-schr--mpi-2x48","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":1000,"num_modes":160,"obs_terms":10000,"seed":0},"cpu_allocation":"0-95","has_mpi":true,"index_bits":32,"ranks":2,"routing":{"bits":1,"mode":"linear","seed":6768574230969066775},"threads":48},"measurement_kind":"pytest","node_id":"bench_random.py::test_random_propagate[schrodinger]","operation":"propagate","parameters_digest":"0b0892a9f5ab8eea64986cd9b0a0dddad24f94a21d42e0ce163162496630536f","profile":"large"},{"config_digest":"a52c48a911878c1caf5abd2d693d17dee9b597ef5313635a50552b36b13cd807","id":"large--propagate-schr--off-1x96","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":1000,"num_modes":160,"obs_terms":10000,"seed":0},"cpu_allocation":"0-95","has_mpi":false,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":96},"measurement_kind":"pytest","node_id":"bench_random.py::test_random_propagate[schrodinger]","operation":"propagate","parameters_digest":"0b0892a9f5ab8eea64986cd9b0a0dddad24f94a21d42e0ce163162496630536f","profile":"large"},{"config_digest":"36dfcf30020d448795f9d47b434e366d04759733621d2da58e2086ed2e6bb873","id":"large--propagate-hubb--mpi-1x96","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"chemical_potential":0.0,"cutoff":6,"hopping":1.0,"interaction":-2.0,"lower_atol":5e-06,"neel_start_spin":"down","num_sites":60,"observable_site":46,"observable_spin":"up","trotter_dt":0.2,"trotter_steps":29},"cpu_allocation":"0-95","has_mpi":true,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":96},"measurement_kind":"pytest","node_id":"bench_models.py::test_model_propagate[hubbard]","operation":"propagate","parameters_digest":"9613b53e27984ebea2313ec73a38d3142d8f96212f3c713be92a05afa57aa809","profile":"large"},{"config_digest":"36dfcf30020d448795f9d47b434e366d04759733621d2da58e2086ed2e6bb873","id":"large--propagate-hubb--mpi-2x48","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"chemical_potential":0.0,"cutoff":6,"hopping":1.0,"interaction":-2.0,"lower_atol":5e-06,"neel_start_spin":"down","num_sites":60,"observable_site":46,"observable_spin":"up","trotter_dt":0.2,"trotter_steps":29},"cpu_allocation":"0-95","has_mpi":true,"index_bits":32,"ranks":2,"routing":{"bits":1,"mode":"linear","seed":6768574230969066775},"threads":48},"measurement_kind":"pytest","node_id":"bench_models.py::test_model_propagate[hubbard]","operation":"propagate","parameters_digest":"9613b53e27984ebea2313ec73a38d3142d8f96212f3c713be92a05afa57aa809","profile":"large"},{"config_digest":"36dfcf30020d448795f9d47b434e366d04759733621d2da58e2086ed2e6bb873","id":"large--propagate-hubb--off-1x96","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"chemical_potential":0.0,"cutoff":6,"hopping":1.0,"interaction":-2.0,"lower_atol":5e-06,"neel_start_spin":"down","num_sites":60,"observable_site":46,"observable_spin":"up","trotter_dt":0.2,"trotter_steps":29},"cpu_allocation":"0-95","has_mpi":false,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":96},"measurement_kind":"pytest","node_id":"bench_models.py::test_model_propagate[hubbard]","operation":"propagate","parameters_digest":"9613b53e27984ebea2313ec73a38d3142d8f96212f3c713be92a05afa57aa809","profile":"large"},{"config_digest":"54593abf486541c62665fd182a866f73c6b3988d7170054d90fcc57255732daf","id":"large--gradient-hubb--mpi-1x96","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"chemical_potential":0.0,"cutoff":6,"hopping":1.0,"interaction":-2.0,"lower_atol":3e-05,"neel_start_spin":"down","num_sites":40,"observable_site":30,"observable_spin":"up","trotter_dt":0.2,"trotter_steps":2},"cpu_allocation":"0-95","has_mpi":true,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":96},"measurement_kind":"pytest","node_id":"bench_models.py::test_model_gradient[hubbard]","operation":"gradient","parameters_digest":"91e736e1b1415ee4e1a6fb4e76103c7d7c27cd7cd2772b790453ade426c33a38","profile":"large"},{"config_digest":"54593abf486541c62665fd182a866f73c6b3988d7170054d90fcc57255732daf","id":"large--gradient-hubb--mpi-2x48","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"chemical_potential":0.0,"cutoff":6,"hopping":1.0,"interaction":-2.0,"lower_atol":3e-05,"neel_start_spin":"down","num_sites":40,"observable_site":30,"observable_spin":"up","trotter_dt":0.2,"trotter_steps":2},"cpu_allocation":"0-95","has_mpi":true,"index_bits":32,"ranks":2,"routing":{"bits":1,"mode":"linear","seed":6768574230969066775},"threads":48},"measurement_kind":"pytest","node_id":"bench_models.py::test_model_gradient[hubbard]","operation":"gradient","parameters_digest":"91e736e1b1415ee4e1a6fb4e76103c7d7c27cd7cd2772b790453ade426c33a38","profile":"large"},{"config_digest":"54593abf486541c62665fd182a866f73c6b3988d7170054d90fcc57255732daf","id":"large--gradient-hubb--off-1x96","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"chemical_potential":0.0,"cutoff":6,"hopping":1.0,"interaction":-2.0,"lower_atol":3e-05,"neel_start_spin":"down","num_sites":40,"observable_site":30,"observable_spin":"up","trotter_dt":0.2,"trotter_steps":2},"cpu_allocation":"0-95","has_mpi":false,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":96},"measurement_kind":"pytest","node_id":"bench_models.py::test_model_gradient[hubbard]","operation":"gradient","parameters_digest":"91e736e1b1415ee4e1a6fb4e76103c7d7c27cd7cd2772b790453ade426c33a38","profile":"large"},{"config_digest":"b2d9d5c71afce2da492c3d7c687b61a0915ae95a682c78f5a0b30b66967d8fd8","id":"large--propagate-paul--mpi-1x96","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"coupling":0.7853981633974483,"cutoff":12,"lower_atol":0.0001,"num_layers":20,"num_qubits":127,"observable_qubit":62,"theta":0.7853981633974483},"cpu_allocation":"0-95","has_mpi":true,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":96},"measurement_kind":"pytest","node_id":"bench_models.py::test_model_propagate[pauli]","operation":"propagate","parameters_digest":"971c364c3cae8d46d276b9c9bffb1e14e23e023429863d9c674b92945e648a42","profile":"large"},{"config_digest":"b2d9d5c71afce2da492c3d7c687b61a0915ae95a682c78f5a0b30b66967d8fd8","id":"large--propagate-paul--mpi-2x48","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"coupling":0.7853981633974483,"cutoff":12,"lower_atol":0.0001,"num_layers":20,"num_qubits":127,"observable_qubit":62,"theta":0.7853981633974483},"cpu_allocation":"0-95","has_mpi":true,"index_bits":32,"ranks":2,"routing":{"bits":1,"mode":"linear","seed":6768574230969066775},"threads":48},"measurement_kind":"pytest","node_id":"bench_models.py::test_model_propagate[pauli]","operation":"propagate","parameters_digest":"971c364c3cae8d46d276b9c9bffb1e14e23e023429863d9c674b92945e648a42","profile":"large"},{"config_digest":"b2d9d5c71afce2da492c3d7c687b61a0915ae95a682c78f5a0b30b66967d8fd8","id":"large--propagate-paul--off-1x96","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"coupling":0.7853981633974483,"cutoff":12,"lower_atol":0.0001,"num_layers":20,"num_qubits":127,"observable_qubit":62,"theta":0.7853981633974483},"cpu_allocation":"0-95","has_mpi":false,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":96},"measurement_kind":"pytest","node_id":"bench_models.py::test_model_propagate[pauli]","operation":"propagate","parameters_digest":"971c364c3cae8d46d276b9c9bffb1e14e23e023429863d9c674b92945e648a42","profile":"large"},{"config_digest":"382aef8b540cf9b9b38fe376a08d199b65345511cb112659abc8552cad1b9312","id":"large-pared--gradient-heis--mpi-1x96","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":400,"num_modes":128,"obs_terms":2000000,"seed":0},"cpu_allocation":"0-95","has_mpi":true,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":96},"measurement_kind":"pytest","node_id":"bench_random.py::test_random_gradient[heisenberg]","operation":"gradient","parameters_digest":"dd73f7973d0d03487692201e103588d92f6c51ed1fdae8585e1adbf8c925c43f","profile":"large-pared"},{"config_digest":"382aef8b540cf9b9b38fe376a08d199b65345511cb112659abc8552cad1b9312","id":"large-pared--gradient-heis--mpi-2x48","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":400,"num_modes":128,"obs_terms":2000000,"seed":0},"cpu_allocation":"0-95","has_mpi":true,"index_bits":32,"ranks":2,"routing":{"bits":1,"mode":"linear","seed":6768574230969066775},"threads":48},"measurement_kind":"pytest","node_id":"bench_random.py::test_random_gradient[heisenberg]","operation":"gradient","parameters_digest":"dd73f7973d0d03487692201e103588d92f6c51ed1fdae8585e1adbf8c925c43f","profile":"large-pared"},{"config_digest":"382aef8b540cf9b9b38fe376a08d199b65345511cb112659abc8552cad1b9312","id":"large-pared--gradient-heis--off-1x96","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":400,"num_modes":128,"obs_terms":2000000,"seed":0},"cpu_allocation":"0-95","has_mpi":false,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":96},"measurement_kind":"pytest","node_id":"bench_random.py::test_random_gradient[heisenberg]","operation":"gradient","parameters_digest":"dd73f7973d0d03487692201e103588d92f6c51ed1fdae8585e1adbf8c925c43f","profile":"large-pared"},{"config_digest":"382aef8b540cf9b9b38fe376a08d199b65345511cb112659abc8552cad1b9312","id":"large-pared--pare-construct-heis--mpi-1x96","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":400,"num_modes":128,"obs_terms":2000000,"seed":0},"cpu_allocation":"0-95","has_mpi":true,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":96},"measurement_kind":"driver","node_id":"driver::pare_functional_construct[heisenberg]","operation":"pare_functional_construct","parameters_digest":"dd73f7973d0d03487692201e103588d92f6c51ed1fdae8585e1adbf8c925c43f","profile":"large-pared"},{"config_digest":"382aef8b540cf9b9b38fe376a08d199b65345511cb112659abc8552cad1b9312","id":"large-pared--pare-construct-heis--mpi-2x48","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":400,"num_modes":128,"obs_terms":2000000,"seed":0},"cpu_allocation":"0-95","has_mpi":true,"index_bits":32,"ranks":2,"routing":{"bits":1,"mode":"linear","seed":6768574230969066775},"threads":48},"measurement_kind":"driver","node_id":"driver::pare_functional_construct[heisenberg]","operation":"pare_functional_construct","parameters_digest":"dd73f7973d0d03487692201e103588d92f6c51ed1fdae8585e1adbf8c925c43f","profile":"large-pared"},{"config_digest":"382aef8b540cf9b9b38fe376a08d199b65345511cb112659abc8552cad1b9312","id":"large-pared--pare-construct-heis--off-1x96","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":400,"num_modes":128,"obs_terms":2000000,"seed":0},"cpu_allocation":"0-95","has_mpi":false,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":96},"measurement_kind":"driver","node_id":"driver::pare_functional_construct[heisenberg]","operation":"pare_functional_construct","parameters_digest":"dd73f7973d0d03487692201e103588d92f6c51ed1fdae8585e1adbf8c925c43f","profile":"large-pared"},{"config_digest":"a52c48a911878c1caf5abd2d693d17dee9b597ef5313635a50552b36b13cd807","id":"large-pared--pare-construct-schr--mpi-1x96","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":1000,"num_modes":160,"obs_terms":10000,"seed":0},"cpu_allocation":"0-95","has_mpi":true,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":96},"measurement_kind":"driver","node_id":"driver::pare_functional_construct[schrodinger]","operation":"pare_functional_construct","parameters_digest":"0b0892a9f5ab8eea64986cd9b0a0dddad24f94a21d42e0ce163162496630536f","profile":"large-pared"},{"config_digest":"a52c48a911878c1caf5abd2d693d17dee9b597ef5313635a50552b36b13cd807","id":"large-pared--pare-construct-schr--mpi-2x48","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":1000,"num_modes":160,"obs_terms":10000,"seed":0},"cpu_allocation":"0-95","has_mpi":true,"index_bits":32,"ranks":2,"routing":{"bits":1,"mode":"linear","seed":6768574230969066775},"threads":48},"measurement_kind":"driver","node_id":"driver::pare_functional_construct[schrodinger]","operation":"pare_functional_construct","parameters_digest":"0b0892a9f5ab8eea64986cd9b0a0dddad24f94a21d42e0ce163162496630536f","profile":"large-pared"},{"config_digest":"a52c48a911878c1caf5abd2d693d17dee9b597ef5313635a50552b36b13cd807","id":"large-pared--pare-construct-schr--off-1x96","identity":{"compiler_flags":"gcc-15.2.0 Release -O3 -DNDEBUG -march=native(znver5)","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":1000,"num_modes":160,"obs_terms":10000,"seed":0},"cpu_allocation":"0-95","has_mpi":false,"index_bits":32,"ranks":1,"routing":{"bits":0,"mode":"linear","seed":6768574230969066775},"threads":96},"measurement_kind":"driver","node_id":"driver::pare_functional_construct[schrodinger]","operation":"pare_functional_construct","parameters_digest":"0b0892a9f5ab8eea64986cd9b0a0dddad24f94a21d42e0ce163162496630536f","profile":"large-pared"}],"schema_version":1,"workloads_path":"rank-local-openmp-workloads.json","workloads_sha256":"508b65f1f0eb2171fc4fbd79feba2381054c6dcdd38a614600f5b957ab814416"} \ No newline at end of file diff --git a/tools/rank-local-openmp-workloads.json b/tools/rank-local-openmp-workloads.json new file mode 100644 index 00000000..e2e184ef --- /dev/null +++ b/tools/rank-local-openmp-workloads.json @@ -0,0 +1 @@ +{"profiles":{"large":{"bench_models.py::test_model_gradient[hubbard]":{"basis":"majorana","config":{"chemical_potential":0.0,"cutoff":6,"hopping":1.0,"interaction":-2.0,"lower_atol":3e-05,"neel_start_spin":"down","num_sites":40,"observable_site":30,"observable_spin":"up","trotter_dt":0.2,"trotter_steps":2},"measurement_kind":"pytest","operation":"gradient","parameters":[0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2],"pare_threshold":null,"picture":"heisenberg"},"bench_models.py::test_model_propagate[hubbard]":{"basis":"majorana","config":{"chemical_potential":0.0,"cutoff":6,"hopping":1.0,"interaction":-2.0,"lower_atol":5e-06,"neel_start_spin":"down","num_sites":60,"observable_site":46,"observable_spin":"up","trotter_dt":0.2,"trotter_steps":29},"measurement_kind":"pytest","operation":"propagate","parameters":[0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2],"pare_threshold":null,"picture":"heisenberg"},"bench_models.py::test_model_propagate[pauli]":{"basis":"pauli","config":{"coupling":0.7853981633974483,"cutoff":12,"lower_atol":0.0001,"num_layers":20,"num_qubits":127,"observable_qubit":62,"theta":0.7853981633974483},"measurement_kind":"pytest","operation":"propagate","parameters":[-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483],"pare_threshold":null,"picture":"heisenberg"},"bench_random.py::test_random_build_graph[heisenberg]":{"basis":"majorana","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":400,"num_modes":128,"obs_terms":2000000,"seed":0},"measurement_kind":"pytest","operation":"build_graph","parameters":[0.016400693043662398,0.05330219374461546,-0.0705585365990327,0.08476754745855934,-0.15193480882210897,-0.10129757289247394,0.0060443043275890005,0.0033006704879674394,0.17067988985270965,-0.08916685645374621,0.12613189835435187,-0.16222910655384135,0.040891271856525566,-0.1149045977068274,-0.05755222577163754,-0.12461566371227287,0.11774995001240425,0.03816598855107006,0.047555032869730784,-0.037665979696370644,0.07385086233087486,-0.05161060791550898,0.04282412057545871,-0.10388061544685463,-0.04656080441867633,-0.0029855177702576637,-0.10044530887179909,-0.007895569464808176,0.0708071867627511,0.09719084988871696,-0.03468737354302981,0.043013228204003995,0.0020010210255856893,-0.016863596767489743,-0.030937337128652737,0.11215984008436618,-0.22114731539727248,0.004086440358185286,-0.03520485106159149,-0.09021747968583696,-0.08572221069218883,0.013000094477226862,-0.13807311956012583,0.058160922962470746,-0.08175154625854542,0.003453209207847275,0.04089703519675422,0.003669508272555036,-0.12162447297958479,-0.06773854210337246,0.08070419531416607,-0.03218016234924814,-0.14906871097296168,0.04680321949393105,0.10302503823644318,-0.046380534186498165,-0.09449489099255681,0.06785468222107302,-0.11714870278510087,-0.06486539896765105,-0.11177345913505171,0.23746744912228307,0.030201985842687545,-0.18870509656509427,-0.20596448063263184,0.05853611513904732,0.012801172058615005,-0.06591351855745625,-0.06194068363028791,0.06821970647325686,0.029337966148215197,-0.11406292379426981,-0.04953735334304769,0.13669521026857187,-0.07338888957192352,0.17368281984797243,-0.01045244901510901,0.13367542508520355,-0.02448947161886537,0.007298895915872514,-0.11316662809120451,-0.019263475903101013,0.12462930631640301,-0.13434412884754396,0.017504781838779748,0.131682291679371,-0.042690174099431516,0.007544922399804037,0.11585811545602419,0.1130997717451217,-0.0005907849943751003,-0.006922018996999964,0.23036964972608667,0.09032244963124984,0.0501388301880246,-0.1301662977596475,0.04731459157257271,0.18576505500150753,0.075022879350012,0.058944213303005005,-0.041834020851077236,0.12978549893014532,0.0719361596514538,-0.0300851117170453,-0.06777999312990489,0.03928450028085989,0.03477793380909691,0.12226368606889648,-0.13274713408121458,-0.05524281048951133,-0.0189581679073641,0.16591228554997536,-0.13027531761515265,-0.05283220999217206,0.046459879640193484,-0.02044877769279148,0.11086636130910539,0.05275380231870326,0.11900879241654763,0.06076851266550035,0.008904936146376522,0.06349297834303429,0.01218103813012849,0.00041851956555569744,-0.09891555446328601,0.033849603095663705,-0.1850724022162653,0.05298388644630856,-0.10587366007901472,-0.009391604898195323,-0.0375707319776898,0.10554440252900106,0.1630172688385757,0.11275709609260479,-0.16540825113672053,0.08113262789956423,-0.026720595466532222,0.0443753353719741,0.007341852274681296,-0.12051664168854091,0.06188392804442369,0.04135931590249106,-0.04685210558651132,-0.09649784533930134,0.12714530030393717,-0.17422327591998577,0.05388901098024321,0.003144153968725154,0.0896730286841024,0.09615834776464798,-0.05404867587753102,0.1729287697288421,0.10211891127378668,0.1875337505678078,-0.0569683674585049,-0.02892526930122943,-0.02066087053671811,0.038198251282201896,-0.08556113879611502,0.09388052204298739,0.06140379567415778,0.11520079237437607,-0.05550634350357095,-0.1404310342362654,0.1211827206685111,0.024793632249527804,0.04862960182465336,-0.048515618953787906,-0.056633322476744574,-0.09569523308155137,-0.06025854122835222,0.01826453419177222,0.021750071675342465,0.0487236328972563,0.09896340027797944,0.1258921868088697,-0.21528654881923037,0.23744711970622925,0.07794430436522615,0.07562261205000631,0.0506778562116543,-0.02254790958661484,-0.03571557369773109,-0.039985784203548126,0.05425167155819899,-0.049536578413245665,0.19691403997362794,-0.08163944411468854,-0.037405261569227216,0.18918269359267037,0.07984206857477427,0.03809024162161545,-0.014898828041645962,0.01971162682033886,0.06724055831980186,0.17261930830681269,-0.025195506241274375,-0.1102600287318478,0.09521803134672806,0.005753896322139205,-0.1182360074468223,0.030768979529344067,0.07146022468010177,-0.06694873827800595,-0.028627917493452427,-0.0311439472465524,0.0992456524761879,0.18488102972597473,-0.032063688764073624,0.13872310377935976,-0.0073768114557231776,0.12962007024499148,-0.11828424798063351,-0.1334268794278036,-0.012498537756610903,0.04382123858602198,0.24885824172975043,0.06943051553862198,0.09103117581271185,0.05014294039954016,0.05552344652827205,0.09878339510357041,-0.05146748461222698,-0.0818833745671812,0.022183747829827682,0.010332045255538507,0.08857731193313047,-0.03392591502267615,0.05281141265088473,-0.14073430351172528,-0.030984329640816145,-0.045153666960562715,0.012134489525199767,-0.011241530709619293,0.049372412843166145,0.044903941605246564,0.10415619551293515,0.10691391983119487,-0.18190013887416495,-0.018248780479126644,0.026244674363329026,0.07223196059000313,-0.0282115873166085,-0.07697543000062129,0.009783991304479385,-0.03311344350740197,0.005173985164112646,0.15694428222905352,-0.004344193338604331,-0.016437742906754118,0.06900816820080378,-0.04215296151589517,-0.06274649935579278,0.035873443028829846,-0.1692324240142118,0.17462207013180245,-0.10217810902308222,-0.20179109147129393,0.08647563449167345,0.04891595114815255,-0.05301889410793609,-0.03195647740567347,-0.07810576559146204,0.01432774626108516,-0.04018899577856525,-0.026465187411549768,-0.017163474432223328,-0.007286885205411046,0.024366190661721555,-0.012676664886555509,0.060132173689510286,-0.0021225498977851427,-0.20199151520926578,-0.0589854714721007,-0.14073405270216338,0.04283894052409212,0.11620346390480635,-0.06811162810051763,-0.021327261208735623,0.2823491223862139,0.09857725268276941,0.1300272855361974,0.10162854008263147,-0.07284494230324078,0.0007409665951316078,-0.08303770911182705,0.02156446928404665,0.044556095036232604,-0.0036043675599702683,-0.12560093923705534,-0.02444595951480936,0.05807315499357597,-0.04353881296852248,-0.099618141049875,0.08285572124071197,0.13267871336724446,-0.09010425244291062,0.17771137250857158,0.04540604047026477,0.1833915885783089,-0.01444292991029257,-0.02749034062041287,0.025512571654009997,-0.15822735006952574,0.015801393378892362,0.1338664764790573,-0.07186415049857346,-0.08710422188730478,0.0423162461647298,0.02830699305259258,-0.15491273732031147,-0.1424759252179176,-0.027055741439198422,0.055657068737031905,-0.029444797516596555,0.09618676326244543,0.16128106518274896,0.08618051184576907,-0.05175626000697086,0.06815599854054319,-0.08670758931341853,0.006083061650996331,0.07302501161435006,-0.09341947709185561,0.09393226958376752,0.12564715732294632,0.004872416738449033,0.1273734137045372,0.14910571326417854,-0.0732509026073625,0.048459705783293804,-0.0458205486604196,-0.042556988046132393,0.1365550929414173,0.08821200722403609,0.09289997670896066,-0.06182144707771201,-0.12186678081623477,-0.007502098543791167,0.121208726887009,-0.04053964280052427,-0.13721899041569685,-0.06709262637109265,0.004935934756346951,0.12000618387999645,0.08092682074492252,-0.026949667229747395,-0.1454949851543202,-0.030997103665956025,-0.052049775907766375,0.08667587004741813,0.18340604132980282,0.06502073151043598,0.06849603778440697,0.05247484595845428,0.09705169679124963,0.0017041388785504201,-0.08835682487054249,0.13411730507553205,-0.026794429523946773,0.1684961689565902,0.013899553514746553,-0.12529917578851865,-0.10577144764195075,0.0889472182165657,0.02835905953681845,0.17076385971354408,0.13666465007999382,-0.0710102520115172,0.09181490536976147,0.004345283880384389,-0.12127189267957823,0.06770555453268857,-0.027612732559700072,0.1142636191432179,0.012353450287781925,0.04924137609197693,0.21485686854869013,-0.01628134359731778,0.035800114325442146,0.05193994641501318,0.11809571107448924,-0.04882722687249083,0.05479733137433027,0.19570829053705988,-0.13782224372065852,0.031232786845975337,0.032768443257354264,0.09968037138271345,-0.04593275595486472,0.008673141230843336,0.10370113596925015,0.00969590770957512,-0.01721745991106537,-0.18955442804026992,-0.02639685953766667,0.02094681471258093,-0.05733565707392492,0.06555042886274136,-0.15107773060231727],"pare_threshold":null,"picture":"heisenberg"},"bench_random.py::test_random_gradient[heisenberg]":{"basis":"majorana","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":400,"num_modes":128,"obs_terms":2000000,"seed":0},"measurement_kind":"pytest","operation":"gradient","parameters":[0.016400693043662398,0.05330219374461546,-0.0705585365990327,0.08476754745855934,-0.15193480882210897,-0.10129757289247394,0.0060443043275890005,0.0033006704879674394,0.17067988985270965,-0.08916685645374621,0.12613189835435187,-0.16222910655384135,0.040891271856525566,-0.1149045977068274,-0.05755222577163754,-0.12461566371227287,0.11774995001240425,0.03816598855107006,0.047555032869730784,-0.037665979696370644,0.07385086233087486,-0.05161060791550898,0.04282412057545871,-0.10388061544685463,-0.04656080441867633,-0.0029855177702576637,-0.10044530887179909,-0.007895569464808176,0.0708071867627511,0.09719084988871696,-0.03468737354302981,0.043013228204003995,0.0020010210255856893,-0.016863596767489743,-0.030937337128652737,0.11215984008436618,-0.22114731539727248,0.004086440358185286,-0.03520485106159149,-0.09021747968583696,-0.08572221069218883,0.013000094477226862,-0.13807311956012583,0.058160922962470746,-0.08175154625854542,0.003453209207847275,0.04089703519675422,0.003669508272555036,-0.12162447297958479,-0.06773854210337246,0.08070419531416607,-0.03218016234924814,-0.14906871097296168,0.04680321949393105,0.10302503823644318,-0.046380534186498165,-0.09449489099255681,0.06785468222107302,-0.11714870278510087,-0.06486539896765105,-0.11177345913505171,0.23746744912228307,0.030201985842687545,-0.18870509656509427,-0.20596448063263184,0.05853611513904732,0.012801172058615005,-0.06591351855745625,-0.06194068363028791,0.06821970647325686,0.029337966148215197,-0.11406292379426981,-0.04953735334304769,0.13669521026857187,-0.07338888957192352,0.17368281984797243,-0.01045244901510901,0.13367542508520355,-0.02448947161886537,0.007298895915872514,-0.11316662809120451,-0.019263475903101013,0.12462930631640301,-0.13434412884754396,0.017504781838779748,0.131682291679371,-0.042690174099431516,0.007544922399804037,0.11585811545602419,0.1130997717451217,-0.0005907849943751003,-0.006922018996999964,0.23036964972608667,0.09032244963124984,0.0501388301880246,-0.1301662977596475,0.04731459157257271,0.18576505500150753,0.075022879350012,0.058944213303005005,-0.041834020851077236,0.12978549893014532,0.0719361596514538,-0.0300851117170453,-0.06777999312990489,0.03928450028085989,0.03477793380909691,0.12226368606889648,-0.13274713408121458,-0.05524281048951133,-0.0189581679073641,0.16591228554997536,-0.13027531761515265,-0.05283220999217206,0.046459879640193484,-0.02044877769279148,0.11086636130910539,0.05275380231870326,0.11900879241654763,0.06076851266550035,0.008904936146376522,0.06349297834303429,0.01218103813012849,0.00041851956555569744,-0.09891555446328601,0.033849603095663705,-0.1850724022162653,0.05298388644630856,-0.10587366007901472,-0.009391604898195323,-0.0375707319776898,0.10554440252900106,0.1630172688385757,0.11275709609260479,-0.16540825113672053,0.08113262789956423,-0.026720595466532222,0.0443753353719741,0.007341852274681296,-0.12051664168854091,0.06188392804442369,0.04135931590249106,-0.04685210558651132,-0.09649784533930134,0.12714530030393717,-0.17422327591998577,0.05388901098024321,0.003144153968725154,0.0896730286841024,0.09615834776464798,-0.05404867587753102,0.1729287697288421,0.10211891127378668,0.1875337505678078,-0.0569683674585049,-0.02892526930122943,-0.02066087053671811,0.038198251282201896,-0.08556113879611502,0.09388052204298739,0.06140379567415778,0.11520079237437607,-0.05550634350357095,-0.1404310342362654,0.1211827206685111,0.024793632249527804,0.04862960182465336,-0.048515618953787906,-0.056633322476744574,-0.09569523308155137,-0.06025854122835222,0.01826453419177222,0.021750071675342465,0.0487236328972563,0.09896340027797944,0.1258921868088697,-0.21528654881923037,0.23744711970622925,0.07794430436522615,0.07562261205000631,0.0506778562116543,-0.02254790958661484,-0.03571557369773109,-0.039985784203548126,0.05425167155819899,-0.049536578413245665,0.19691403997362794,-0.08163944411468854,-0.037405261569227216,0.18918269359267037,0.07984206857477427,0.03809024162161545,-0.014898828041645962,0.01971162682033886,0.06724055831980186,0.17261930830681269,-0.025195506241274375,-0.1102600287318478,0.09521803134672806,0.005753896322139205,-0.1182360074468223,0.030768979529344067,0.07146022468010177,-0.06694873827800595,-0.028627917493452427,-0.0311439472465524,0.0992456524761879,0.18488102972597473,-0.032063688764073624,0.13872310377935976,-0.0073768114557231776,0.12962007024499148,-0.11828424798063351,-0.1334268794278036,-0.012498537756610903,0.04382123858602198,0.24885824172975043,0.06943051553862198,0.09103117581271185,0.05014294039954016,0.05552344652827205,0.09878339510357041,-0.05146748461222698,-0.0818833745671812,0.022183747829827682,0.010332045255538507,0.08857731193313047,-0.03392591502267615,0.05281141265088473,-0.14073430351172528,-0.030984329640816145,-0.045153666960562715,0.012134489525199767,-0.011241530709619293,0.049372412843166145,0.044903941605246564,0.10415619551293515,0.10691391983119487,-0.18190013887416495,-0.018248780479126644,0.026244674363329026,0.07223196059000313,-0.0282115873166085,-0.07697543000062129,0.009783991304479385,-0.03311344350740197,0.005173985164112646,0.15694428222905352,-0.004344193338604331,-0.016437742906754118,0.06900816820080378,-0.04215296151589517,-0.06274649935579278,0.035873443028829846,-0.1692324240142118,0.17462207013180245,-0.10217810902308222,-0.20179109147129393,0.08647563449167345,0.04891595114815255,-0.05301889410793609,-0.03195647740567347,-0.07810576559146204,0.01432774626108516,-0.04018899577856525,-0.026465187411549768,-0.017163474432223328,-0.007286885205411046,0.024366190661721555,-0.012676664886555509,0.060132173689510286,-0.0021225498977851427,-0.20199151520926578,-0.0589854714721007,-0.14073405270216338,0.04283894052409212,0.11620346390480635,-0.06811162810051763,-0.021327261208735623,0.2823491223862139,0.09857725268276941,0.1300272855361974,0.10162854008263147,-0.07284494230324078,0.0007409665951316078,-0.08303770911182705,0.02156446928404665,0.044556095036232604,-0.0036043675599702683,-0.12560093923705534,-0.02444595951480936,0.05807315499357597,-0.04353881296852248,-0.099618141049875,0.08285572124071197,0.13267871336724446,-0.09010425244291062,0.17771137250857158,0.04540604047026477,0.1833915885783089,-0.01444292991029257,-0.02749034062041287,0.025512571654009997,-0.15822735006952574,0.015801393378892362,0.1338664764790573,-0.07186415049857346,-0.08710422188730478,0.0423162461647298,0.02830699305259258,-0.15491273732031147,-0.1424759252179176,-0.027055741439198422,0.055657068737031905,-0.029444797516596555,0.09618676326244543,0.16128106518274896,0.08618051184576907,-0.05175626000697086,0.06815599854054319,-0.08670758931341853,0.006083061650996331,0.07302501161435006,-0.09341947709185561,0.09393226958376752,0.12564715732294632,0.004872416738449033,0.1273734137045372,0.14910571326417854,-0.0732509026073625,0.048459705783293804,-0.0458205486604196,-0.042556988046132393,0.1365550929414173,0.08821200722403609,0.09289997670896066,-0.06182144707771201,-0.12186678081623477,-0.007502098543791167,0.121208726887009,-0.04053964280052427,-0.13721899041569685,-0.06709262637109265,0.004935934756346951,0.12000618387999645,0.08092682074492252,-0.026949667229747395,-0.1454949851543202,-0.030997103665956025,-0.052049775907766375,0.08667587004741813,0.18340604132980282,0.06502073151043598,0.06849603778440697,0.05247484595845428,0.09705169679124963,0.0017041388785504201,-0.08835682487054249,0.13411730507553205,-0.026794429523946773,0.1684961689565902,0.013899553514746553,-0.12529917578851865,-0.10577144764195075,0.0889472182165657,0.02835905953681845,0.17076385971354408,0.13666465007999382,-0.0710102520115172,0.09181490536976147,0.004345283880384389,-0.12127189267957823,0.06770555453268857,-0.027612732559700072,0.1142636191432179,0.012353450287781925,0.04924137609197693,0.21485686854869013,-0.01628134359731778,0.035800114325442146,0.05193994641501318,0.11809571107448924,-0.04882722687249083,0.05479733137433027,0.19570829053705988,-0.13782224372065852,0.031232786845975337,0.032768443257354264,0.09968037138271345,-0.04593275595486472,0.008673141230843336,0.10370113596925015,0.00969590770957512,-0.01721745991106537,-0.18955442804026992,-0.02639685953766667,0.02094681471258093,-0.05733565707392492,0.06555042886274136,-0.15107773060231727],"pare_threshold":null,"picture":"heisenberg"},"bench_random.py::test_random_propagate[heisenberg]":{"basis":"majorana","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":400,"num_modes":128,"obs_terms":2000000,"seed":0},"measurement_kind":"pytest","operation":"propagate","parameters":[0.016400693043662398,0.05330219374461546,-0.0705585365990327,0.08476754745855934,-0.15193480882210897,-0.10129757289247394,0.0060443043275890005,0.0033006704879674394,0.17067988985270965,-0.08916685645374621,0.12613189835435187,-0.16222910655384135,0.040891271856525566,-0.1149045977068274,-0.05755222577163754,-0.12461566371227287,0.11774995001240425,0.03816598855107006,0.047555032869730784,-0.037665979696370644,0.07385086233087486,-0.05161060791550898,0.04282412057545871,-0.10388061544685463,-0.04656080441867633,-0.0029855177702576637,-0.10044530887179909,-0.007895569464808176,0.0708071867627511,0.09719084988871696,-0.03468737354302981,0.043013228204003995,0.0020010210255856893,-0.016863596767489743,-0.030937337128652737,0.11215984008436618,-0.22114731539727248,0.004086440358185286,-0.03520485106159149,-0.09021747968583696,-0.08572221069218883,0.013000094477226862,-0.13807311956012583,0.058160922962470746,-0.08175154625854542,0.003453209207847275,0.04089703519675422,0.003669508272555036,-0.12162447297958479,-0.06773854210337246,0.08070419531416607,-0.03218016234924814,-0.14906871097296168,0.04680321949393105,0.10302503823644318,-0.046380534186498165,-0.09449489099255681,0.06785468222107302,-0.11714870278510087,-0.06486539896765105,-0.11177345913505171,0.23746744912228307,0.030201985842687545,-0.18870509656509427,-0.20596448063263184,0.05853611513904732,0.012801172058615005,-0.06591351855745625,-0.06194068363028791,0.06821970647325686,0.029337966148215197,-0.11406292379426981,-0.04953735334304769,0.13669521026857187,-0.07338888957192352,0.17368281984797243,-0.01045244901510901,0.13367542508520355,-0.02448947161886537,0.007298895915872514,-0.11316662809120451,-0.019263475903101013,0.12462930631640301,-0.13434412884754396,0.017504781838779748,0.131682291679371,-0.042690174099431516,0.007544922399804037,0.11585811545602419,0.1130997717451217,-0.0005907849943751003,-0.006922018996999964,0.23036964972608667,0.09032244963124984,0.0501388301880246,-0.1301662977596475,0.04731459157257271,0.18576505500150753,0.075022879350012,0.058944213303005005,-0.041834020851077236,0.12978549893014532,0.0719361596514538,-0.0300851117170453,-0.06777999312990489,0.03928450028085989,0.03477793380909691,0.12226368606889648,-0.13274713408121458,-0.05524281048951133,-0.0189581679073641,0.16591228554997536,-0.13027531761515265,-0.05283220999217206,0.046459879640193484,-0.02044877769279148,0.11086636130910539,0.05275380231870326,0.11900879241654763,0.06076851266550035,0.008904936146376522,0.06349297834303429,0.01218103813012849,0.00041851956555569744,-0.09891555446328601,0.033849603095663705,-0.1850724022162653,0.05298388644630856,-0.10587366007901472,-0.009391604898195323,-0.0375707319776898,0.10554440252900106,0.1630172688385757,0.11275709609260479,-0.16540825113672053,0.08113262789956423,-0.026720595466532222,0.0443753353719741,0.007341852274681296,-0.12051664168854091,0.06188392804442369,0.04135931590249106,-0.04685210558651132,-0.09649784533930134,0.12714530030393717,-0.17422327591998577,0.05388901098024321,0.003144153968725154,0.0896730286841024,0.09615834776464798,-0.05404867587753102,0.1729287697288421,0.10211891127378668,0.1875337505678078,-0.0569683674585049,-0.02892526930122943,-0.02066087053671811,0.038198251282201896,-0.08556113879611502,0.09388052204298739,0.06140379567415778,0.11520079237437607,-0.05550634350357095,-0.1404310342362654,0.1211827206685111,0.024793632249527804,0.04862960182465336,-0.048515618953787906,-0.056633322476744574,-0.09569523308155137,-0.06025854122835222,0.01826453419177222,0.021750071675342465,0.0487236328972563,0.09896340027797944,0.1258921868088697,-0.21528654881923037,0.23744711970622925,0.07794430436522615,0.07562261205000631,0.0506778562116543,-0.02254790958661484,-0.03571557369773109,-0.039985784203548126,0.05425167155819899,-0.049536578413245665,0.19691403997362794,-0.08163944411468854,-0.037405261569227216,0.18918269359267037,0.07984206857477427,0.03809024162161545,-0.014898828041645962,0.01971162682033886,0.06724055831980186,0.17261930830681269,-0.025195506241274375,-0.1102600287318478,0.09521803134672806,0.005753896322139205,-0.1182360074468223,0.030768979529344067,0.07146022468010177,-0.06694873827800595,-0.028627917493452427,-0.0311439472465524,0.0992456524761879,0.18488102972597473,-0.032063688764073624,0.13872310377935976,-0.0073768114557231776,0.12962007024499148,-0.11828424798063351,-0.1334268794278036,-0.012498537756610903,0.04382123858602198,0.24885824172975043,0.06943051553862198,0.09103117581271185,0.05014294039954016,0.05552344652827205,0.09878339510357041,-0.05146748461222698,-0.0818833745671812,0.022183747829827682,0.010332045255538507,0.08857731193313047,-0.03392591502267615,0.05281141265088473,-0.14073430351172528,-0.030984329640816145,-0.045153666960562715,0.012134489525199767,-0.011241530709619293,0.049372412843166145,0.044903941605246564,0.10415619551293515,0.10691391983119487,-0.18190013887416495,-0.018248780479126644,0.026244674363329026,0.07223196059000313,-0.0282115873166085,-0.07697543000062129,0.009783991304479385,-0.03311344350740197,0.005173985164112646,0.15694428222905352,-0.004344193338604331,-0.016437742906754118,0.06900816820080378,-0.04215296151589517,-0.06274649935579278,0.035873443028829846,-0.1692324240142118,0.17462207013180245,-0.10217810902308222,-0.20179109147129393,0.08647563449167345,0.04891595114815255,-0.05301889410793609,-0.03195647740567347,-0.07810576559146204,0.01432774626108516,-0.04018899577856525,-0.026465187411549768,-0.017163474432223328,-0.007286885205411046,0.024366190661721555,-0.012676664886555509,0.060132173689510286,-0.0021225498977851427,-0.20199151520926578,-0.0589854714721007,-0.14073405270216338,0.04283894052409212,0.11620346390480635,-0.06811162810051763,-0.021327261208735623,0.2823491223862139,0.09857725268276941,0.1300272855361974,0.10162854008263147,-0.07284494230324078,0.0007409665951316078,-0.08303770911182705,0.02156446928404665,0.044556095036232604,-0.0036043675599702683,-0.12560093923705534,-0.02444595951480936,0.05807315499357597,-0.04353881296852248,-0.099618141049875,0.08285572124071197,0.13267871336724446,-0.09010425244291062,0.17771137250857158,0.04540604047026477,0.1833915885783089,-0.01444292991029257,-0.02749034062041287,0.025512571654009997,-0.15822735006952574,0.015801393378892362,0.1338664764790573,-0.07186415049857346,-0.08710422188730478,0.0423162461647298,0.02830699305259258,-0.15491273732031147,-0.1424759252179176,-0.027055741439198422,0.055657068737031905,-0.029444797516596555,0.09618676326244543,0.16128106518274896,0.08618051184576907,-0.05175626000697086,0.06815599854054319,-0.08670758931341853,0.006083061650996331,0.07302501161435006,-0.09341947709185561,0.09393226958376752,0.12564715732294632,0.004872416738449033,0.1273734137045372,0.14910571326417854,-0.0732509026073625,0.048459705783293804,-0.0458205486604196,-0.042556988046132393,0.1365550929414173,0.08821200722403609,0.09289997670896066,-0.06182144707771201,-0.12186678081623477,-0.007502098543791167,0.121208726887009,-0.04053964280052427,-0.13721899041569685,-0.06709262637109265,0.004935934756346951,0.12000618387999645,0.08092682074492252,-0.026949667229747395,-0.1454949851543202,-0.030997103665956025,-0.052049775907766375,0.08667587004741813,0.18340604132980282,0.06502073151043598,0.06849603778440697,0.05247484595845428,0.09705169679124963,0.0017041388785504201,-0.08835682487054249,0.13411730507553205,-0.026794429523946773,0.1684961689565902,0.013899553514746553,-0.12529917578851865,-0.10577144764195075,0.0889472182165657,0.02835905953681845,0.17076385971354408,0.13666465007999382,-0.0710102520115172,0.09181490536976147,0.004345283880384389,-0.12127189267957823,0.06770555453268857,-0.027612732559700072,0.1142636191432179,0.012353450287781925,0.04924137609197693,0.21485686854869013,-0.01628134359731778,0.035800114325442146,0.05193994641501318,0.11809571107448924,-0.04882722687249083,0.05479733137433027,0.19570829053705988,-0.13782224372065852,0.031232786845975337,0.032768443257354264,0.09968037138271345,-0.04593275595486472,0.008673141230843336,0.10370113596925015,0.00969590770957512,-0.01721745991106537,-0.18955442804026992,-0.02639685953766667,0.02094681471258093,-0.05733565707392492,0.06555042886274136,-0.15107773060231727],"pare_threshold":null,"picture":"heisenberg"},"bench_random.py::test_random_propagate[schrodinger]":{"basis":"majorana","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":1000,"num_modes":160,"obs_terms":10000,"seed":0},"measurement_kind":"pytest","operation":"propagate","parameters":[0.06276272766274849,-0.22687133805788584,-0.25107438577838603,-0.09677052490016542,-0.0768869602555572,0.15588376877677446,-0.06424349886973883,0.3173793275673169,0.02292820677316353,-0.07103322587570442,-0.12536315249015878,-0.07783013990818642,0.0812219862162197,0.07384491630065919,0.09692215372640231,-0.058441970213209606,-0.1882687568693983,-0.1013907789537107,-0.044067202037032016,-0.08436498813583376,0.023069294505968483,-0.12263524153943746,0.024522343574866573,-0.018876609494189355,-0.14376027944668926,-0.07776249419512249,-0.08257852994546848,0.18518041834187568,0.12278247077436039,0.07709848798665908,0.0013980501870870766,0.054197150674083164,0.06138268183868248,-0.11218162848941715,-0.09257758502871483,0.07835025053251038,-0.23283024306936217,-0.03637238612608577,0.061022767113721094,0.14925076227267092,0.12601266831141103,0.13430713483919132,-0.19850890895081724,-0.03740167178722114,-0.22060910190052885,0.06816291252550757,-0.11610566399108056,-0.04048684532387989,0.06681344911274748,0.13519687395401872,-0.026613764938776548,0.0643918265353292,0.12244395975673428,0.08724706391408271,0.013111874642129158,-0.10802290324013786,0.07700329657791315,0.1362040998630115,0.09766101175706404,0.03610730783130695,-0.05021932213979196,-0.06998801176227355,-0.017504094567420456,0.034341457491017906,0.18425915352979647,-0.03243523151481797,-0.061043447595918135,-0.0651985618903383,-0.016738636970275167,-0.006343264208759926,-0.3435898683215483,-0.04584456438455309,0.11515943170552606,0.03690016812634455,-0.059662457187573875,0.07603454652097243,-0.13108723609025472,0.08094048164889978,-0.03305525661466989,-0.053459638459425335,0.061422954304962245,-0.2361865712612381,0.04972795561776804,-0.008663511556076904,-0.07224422038606591,0.1712015172455634,-0.13612414625910044,0.20701819606643773,0.04330824574780291,0.10546747742916218,0.0024713848313154554,-0.0010271523659349166,0.0994183206124781,-0.010284709971628886,-0.20010120727011343,0.03637799611958672,-0.0009459388124698413,0.012782171099906404,0.09281141803232992,-0.0008368188479920023,-0.1192077289779303,-0.030359797293891225,-0.005582478864044986,0.0496679012831155,-0.17633335896951383,-0.1285506743889473,0.0902474625775841,-0.11415700212078837,-0.02583700354099643,-0.12210181556481117,-0.1093537941302389,-0.03644268692230384,-0.023235673809399927,0.06825227750129913,0.051819351059577835,0.00721494406398302,-0.13037313440545836,0.08768801073157384,0.08806034997795176,0.08766002071791226,-0.022753675946119406,-0.020097003836525976,0.08643822815366907,0.11277445670912867,0.055096265694647095,0.06140940944027082,0.14666070464504763,-0.04598226896809146,-0.09039766966533806,-0.015454741568079126,-0.048829308235195164,0.07353368362821668,0.05876922651983877,0.16014236058822495,0.024420808529096824,-0.09878803899653621,-0.06308696601268102,-0.06614664779644445,-0.12412050453467627,-0.0951049985104442,-0.07747543818838261,-0.10740184134844051,-0.11452704866127446,-0.10516166930239078,-0.21945815808588254,0.04431562421217815,0.13633071091941773,0.05666268193834978,-0.05556362607753662,-0.09611021105799235,-0.007315765638685897,-0.17274850315161908,0.13863517898102376,0.00935328366709223,-0.19976587612519586,0.1353295427175246,-0.1543136670505187,-0.0031255305572637708,0.11661455786257875,0.08130786940078792,0.005440063471758646,-0.16696171784979846,0.07327742927231254,0.10653083228715671,-0.0252884933849329,0.08148988367182947,0.0760276147574616,0.015446567026381797,0.24903335547945638,0.03546520965316171,-0.09615542694737728,0.08800882048406919,0.01280139989990979,0.020989878633498774,0.042864713123378376,0.07296203205558324,-0.02296559680976527,0.027133739799264363,-0.1553179366271642,0.08265767447156505,0.04296212882088783,0.13274898550633366,0.0716739536157238,0.04161903533781064,-0.09558908277483183,-0.08524921490475086,-0.0903061504724615,0.09445265515424968,0.013984977196329152,-0.1106208353418456,-0.21855158744986422,-0.015271519235740489,-0.13590314842918563,-0.04760164236494814,-0.14865481105561487,-0.0025558180929022983,0.05385973127464486,-0.05954531722846558,0.09771323732157962,0.04287758307951439,-0.019126035244953623,-0.19248655772788414,-0.010917334862613199,-0.018406352955598386,0.10262279003039934,-0.012793031398259842,0.0768071791957383,-0.009249009117021817,0.08321204572024274,-0.10823863178608238,0.1374747295045929,0.24354335778024186,0.04058738348807797,-0.048039633272225796,-0.1121766857989379,-0.11799419670067393,0.22754276450209454,-0.14546131797942977,0.03867542136353724,-0.04023639260870505,-0.09449997282739675,-0.0979820022295657,0.12546534966197684,-0.08797453028341676,-0.06534612222116108,-0.11440728430981593,0.08663795644396793,0.18020218605882393,0.07000584419966656,-0.057992813280183214,-0.15630872801165144,0.11966628485716478,0.05340254741900327,0.23780001607156379,-0.05016595182476339,0.07784226230588195,-0.08247705739325724,-0.08964584854585411,-0.022722077601874693,0.035551860308917556,-0.08932535014758965,0.019169609008638372,-0.01087113883521514,-0.010544015027172993,0.14607874127382556,0.16803857275361458,0.06292578552571965,-0.0767878652116652,0.03052783979832214,0.056327224982507876,0.036735000621115683,-0.03104377204097222,0.04622672432010137,-0.10094205816667796,0.053981510646019415,-0.027332522857129213,-0.04906120409482551,0.06642502017976502,-0.07719572532664998,0.06515868452686698,0.059448804997544236,-0.08123135278284248,-0.03177696530443739,-0.11764056361468399,-0.10431700209282568,-0.050695565769386546,-0.016323186423288033,0.10004425633838072,0.06847297485193375,-0.029856048241370548,-0.006627731129939755,0.00040038132552957986,0.14183377812871575,0.008267512806017804,0.04435443376664319,0.012967513398568876,0.03252220679860737,-0.0003913153207884925,0.17094553913503197,0.01898839975222227,0.08928386848946213,0.07000120902769709,0.036476206806316004,0.17725362729166114,0.11484071514804753,0.10653536662541885,0.12879190210000244,-0.1200038428243611,0.0374976652640413,-0.012282609548483623,-0.09403141265212324,-0.030905151689040084,-0.05704323692553618,0.1035307030633321,0.06311587365175637,-0.07699397287288551,-0.023958658974864076,0.037906818682203025,0.1385576670762419,0.18273916132584678,-0.0016654756326588479,0.14945427334128591,-0.18412382433919447,-0.023542061227968886,0.06259200290358802,0.19421437811141928,-0.01257219686249554,-0.084787712189934,0.0692689235579865,0.04010437041513535,0.038469608156980466,-0.31268923920341407,0.013686206019353242,-0.00623851946499252,0.0863255651117361,0.0028028297115834495,0.09686661295677647,0.005136295655841398,-0.04257446916084448,-0.12283314665983557,-0.1502479316976437,-0.12059661639744637,-0.05025492712810057,-0.06810791809905885,0.05713354476194441,0.025444939677748236,-0.14747016487058393,0.03958494353081173,0.08041381684969137,0.07640853575291244,0.043874081558348474,0.16721666790219472,0.16004645934333694,-0.1936153693546914,-0.1959543795330284,0.11388686591225328,0.0384217332679265,0.12794796373328163,-0.13531959723406325,-0.025134133141378384,0.02078492743332607,-0.05774294536278882,0.0129855522511809,-0.004711711865549823,0.08305130521302456,-0.04996638802760931,0.018289917794050047,0.062195361411023144,0.09007941158419147,-0.20426967453612802,-0.020632232037755972,0.026460196079060153,-0.11174803603006056,-0.004124911166063076,0.007916345938963818,0.08616409749621094,-0.04906698039110374,-0.047727380044708764,0.04292032035388957,-0.16529173220122098,0.044125206693123156,-0.06264343488957308,-0.06245184130205708,0.12666861074334027,0.11166469895597803,0.02181855967709821,0.11573445853352363,0.08815930209319889,0.13755718359904312,0.038175732200796106,-0.050346045926522545,0.039437674117072025,0.16476253280331302,-0.0879632715532659,0.016675240721986086,-0.025117801330188396,0.008994621746602736,0.06788556204772116,0.04419279247821052,0.03327512041787936,0.11892435724165472,0.031796657880267046,0.02056053655953996,-0.022756586335563957,0.07341792095351166,0.04527583313829037,0.037573683003254,-0.05753663856312268,-0.04339256839686785,0.062129660515415845,-0.0358289482780926,0.18590775162220208,-0.059137207662189044,-0.0029603745962606485,0.08231343041650153,0.06320264944590682,-0.036663591441645986,-0.06754377046919105,-0.1198671532910771,0.10849820823817868,-0.054365962015024305,-0.024344826739070947,-0.11978454102930912,-0.08751138478543875,0.10096765781361901,0.1592553317734185,0.1049044216460918,0.058353399542260424,0.16216641591277237,-0.17198990292750066,-0.007062917080313863,0.1521400735365031,0.11829342369956707,0.10133411582333612,0.051730783422424875,-0.08347512599052101,0.020369167996479712,-0.08884099859580673,0.21654698187314586,0.03894154076960041,0.09416643971005627,-0.16812131799990732,0.13996059406871625,-0.054983418766623174,0.03402532557280094,-0.03040371644820248,-0.07185734462260501,-0.16913570339082065,0.08673102732644461,-0.0013327833201409631,-0.010346814618200782,-0.12939658226981374,-0.11383167941947843,0.1264552409603924,0.004478209347693592,-0.03126673626067476,-0.08736033404313467,0.18613770993407067,-0.05019536018619178,-0.012763200702420409,0.21833625098345355,0.0031415343090802415,0.03214585936238269,-0.19992439058036954,0.018898255136305407,-0.18159234200610702,-0.052566158010671986,-0.08518631006209691,0.05439233950638141,-0.14947554771131635,-0.0753357975713261,0.06533608655651728,-0.03148845441409269,-0.15377216759815082,0.08171981232759805,0.06601896387894199,-0.08151610759750655,0.0557058437601177,0.12579823592046266,0.12356064076397244,0.051071203519584524,0.025945341856343836,0.18143397174950227,-0.07071046334785201,0.1433740459770921,-0.18430235092598657,-0.027609620049223033,0.05387074197106803,0.21336265281374966,-0.051367327251379324,-0.0023666398253829753,0.049599391408693905,-0.12711212099803618,-0.15406773747557295,0.002167613522097274,-0.07983683504432056,0.04541698889642274,0.03312388966717149,-0.21968157827654014,-0.04279851312408997,-0.07890121238954599,0.056516991840726595,0.04181414322707895,-0.015531835744269227,-0.08744550882590894,-0.08011557144078246,-0.08573933982348687,-0.025300166962526523,0.07738017671315045,-0.029581042776431823,0.13031946225149543,-0.142048298171351,-0.018758416230905982,0.05147261608210521,0.0064004556570420425,0.022577719107429062,0.0396500138842135,-0.16168061368095094,-0.17624158785641753,0.16255120984378366,0.11015215044527989,0.08003558880272359,0.021817096820655725,0.06787428734306573,-0.1287965100496475,0.017502807210445958,0.19383933085016433,0.07190743845628454,-0.24208634031967247,0.04335606554014287,-0.041717044255931424,-0.011322533685899822,-0.018624770469814738,-0.00970924003874039,0.06892341702762476,-0.025865512346863462,0.08316010061209371,-0.006272589306787965,0.174423970536427,-0.11326813877148766,0.06489320855412054,-0.16541842752647445,0.008261642738960037,-0.00836928657239852,0.03201442626110271,0.12021720134250767,0.035659262874604615,0.03587933684112218,-0.028508168886511694,-0.04557782330912097,-0.0314890020115514,0.18517036200962744,-0.03508077191419653,-0.2345105337505559,0.0826992677588738,-0.09340381597684011,-0.15956167298026425,-0.0004400591583051728,0.05915325275872283,-0.07118383227048762,0.12086389637690223,0.012818660645969813,-0.04616566129472341,-0.19181319255983897,-0.1536058207021391,0.029601639420800685,0.145714709292847,0.050168229332723004,-0.11145371693833046,0.00564670992202313,0.044899136704619036,0.0008862438898512453,0.11505909010788669,0.16152680701826272,0.0065780433879503416,-0.09181572541104183,-0.17374901198705717,0.12378213818535132,0.11395519763844895,0.14748461214987474,-0.06662685899293873,0.01551505195819542,-0.19439520314957695,-0.20815501251994442,-0.015955456787968957,0.0036190036598666023,-0.022156310424324442,0.04283773749201175,-0.20619499342346687,-0.12870818173728565,-0.08138598042959827,-0.07829865444977535,-0.025149130274242107,0.0766133724051146,0.14871612612785676,-0.19654243862138956,-0.010798447136809206,0.13222192142657307,-0.02927081900660914,0.10351332185865497,-0.12697339292674434,-0.08633490093299673,0.01843914560269134,0.07831805868044639,0.03136494151790882,0.06832837798767553,0.11015836877884924,0.08768606860753664,-0.10010869653467805,-0.12282213015182207,0.018780150146897254,-0.004817478353349955,0.049014687769521205,0.085513139696511,-0.0035945229108303722,-0.014555548065848693,-0.13746460665765944,0.07397832032728119,-0.04498679753937875,0.04434434677861934,-0.029475622868182106,-0.09299219600574608,-0.024447804090667092,-0.0177612559681759,0.08395658072999457,0.07269083568287427,0.07438164644971174,-0.05647700523749073,-0.0883431432100969,-0.17048730377977683,-0.07471450164216163,0.011958920771599973,-0.10457362520843257,-0.01599436664424899,-0.055634581748593925,0.015551649556405749,-0.08091461835186348,-0.06761212376340443,0.18077098202840478,0.039240072936793866,0.1203835032709618,-0.0013608211112286755,-0.041499513748177334,-0.11817568836055997,0.025763475351521237,0.021079135788591483,-0.05289605134209744,-0.062144222933871424,0.11035039961768499,0.059760137583236464,-0.13860172672192947,-0.03183545637617876,0.0493493418502237,-0.038187059552925055,-0.09780220405993975,-0.11333367792907652,0.04043691953392767,0.04642169090501244,-0.04527489244697058,0.020357835775801747,-0.04185657006687821,-0.06756195497207482,-0.1101415684208914,0.10844155021610935,-0.04609002659870931,-0.03907005350617264,-0.11116790745086227,-0.03987611775583941,0.019216383610989398,0.11614620600311915,-0.11518806473067658,0.006281303716553359,-0.04564041305458383,0.1093579830057281,0.1342993394886974,0.12340768415048549,0.1267777051513554,0.03835357945121847,0.12253054869570229,-0.07918572372879261,0.0777153560911588,-0.03848168143882725,-0.010855994650898874,-0.07830647935253518,-0.011618609254385383,0.13873313363304182,-0.11579264937841507,-0.2367323450485221,-0.15862269322029945,0.027401411728591993,-0.18675939665788843,-0.12002999676997964,0.05397306845591109,0.04971618410973413,-0.09988723267620188,-0.1841846220789959,-0.0800332936153636,0.021726542434816924,0.044614855781237785,0.14523247654594337,-0.027294541155590504,-0.07553536953486262,-0.09838085835050402,-0.13081983974217584,-0.09415415713650438,-0.0274999562429024,-0.046061382087644626,0.1612293515874996,0.1321809646616843,-0.09774763854433201,-0.029861256170780018,0.03735835074850183,0.09515589776802405,0.0819689431414217,0.09779381001208708,0.18147323725245262,0.0094328063250593,-0.05745389829724219,0.033760123072992525,-0.1871040168986563,-0.09227225740270387,0.010057940066828636,0.19376569853450898,-0.018285738762673892,-0.008241924887968008,-0.08150206880575511,0.03679848187947773,0.15967309767217883,0.06112726250139562,0.00699475333599775,-0.21558675380533,-0.1738584569354492,0.0005732778213747429,-0.005782887242626333,-0.04554664057657038,0.06627813118741502,0.0845772216506309,-0.07015217490388241,0.15515029066519065,-0.09948087310677357,0.05585498104891351,0.1418374095423812,0.02144527033404603,0.13772131149527675,-0.1679606942450375,-0.021550207386317455,-0.06389234553182785,0.046004161350740765,0.036703610371877485,0.20104863251058083,0.03233225931939441,-0.07794083727474331,-0.0022071257291329444,0.07085401420429162,-0.006355525076133531,0.07840096887525895,0.07487782242965475,0.15964787434371314,-0.11061012209853366,-0.0683479785828875,0.20891776560652012,0.03451726671877159,-0.08479041434197489,0.10502313099785515,0.11530310400473996,0.02428077695396679,0.07446242583113068,0.007218528216356839,0.0484837700055454,0.030881128968563928,-0.1020330527683294,0.15880174463081057,-0.10268313092098819,0.06800235675623091,0.16335061072477441,-0.08065054183838283,-0.01966661180125237,0.11161461701996248,0.009896800509713451,-0.05936131706769511,-0.026948518784022477,0.1565411035281179,0.0652408019401025,0.08143906451858968,0.07223254531398397,0.09178280057455607,-0.04434994585539389,-0.016236740163136212,-0.09815362643622287,0.011323276626718733,0.12922672414802824,0.09761459092916874,-0.04581776253275588,-0.10487696390037388,0.07524238388767557,0.10161630878381936,-0.03784332237029631,0.05438104964519358,0.12022333980028815,-0.09785817266488119,-0.045412321564340156,0.11448602567725381,0.08611692813763289,0.04254426451870566,0.048383086770820614,0.17920759702679206,-0.19035829983290264,0.21300297693798254,0.08662188675069189,0.04566247454038605,-0.13431105526079692,0.027268601927646116,-0.07278115031673667,0.1004410744480942,-0.2084271842590659,-0.10349908239071746,0.02845546455515921,-0.22334881746574953,0.03152153961461469,0.03002443769141083,0.06667283194193661,0.02342430059644278,-0.014476021401424462,-0.15733156979335905,0.04783622684090224,0.07431069306705987,-0.017691824788874985,0.09905205517569893,0.00833735536897295,0.17614677584488034,0.2201581295596142,0.10979836499676075,-0.0017513769337368203,0.008989017604878758,-0.07097408480244023,0.06421704595150812,-0.047963902733947045,0.033719179414828146,-0.07764484769639887,-0.014537651446404629,-0.11373141917285826,0.10155765021556813,-0.09949675568892846,0.1069386693428084,-0.06848211872373694,-0.014697889288646302,-0.021581114929070208,-0.2995159730723255,0.02198905047020407,-0.17332771347995246,0.005133672629540125,-0.028640957403942315,0.125648027960912,0.04078530639465178,0.00012728004522073163,-0.09906804042516798,0.0016294530031185682,0.025349215673843545,-0.10157970567550614,0.01957880643398501,-0.09137747673308441,-0.1506800719170716,0.17395244613940533,-0.19180439859280823,-0.08177379666604478,0.18819703901403773,-0.1525574385651186,-0.006694898675801525,0.07180940779474952,-0.05290534693258395,-0.03699707542331403,-0.06914502015298808,-0.0018101857393414353,0.12801597150630858,-0.03471212193213878,-0.11174575602062707,0.03590771869937087,0.07759335764075306,0.23016031995578243,0.1498311485587772,0.020039958089782928,-0.07869699538590262,-0.035115890510380623,0.1914057491770485,0.09191994656423602,-0.07727250322474635,0.0009421782344111768,-0.10524994362520167,0.0029321955985496346,0.1442918693807178,-0.03467189225853997,0.12944237987484342,0.08213483498015912,-0.07173206948558407,-0.013468167494366615,0.005427280542090472,-0.029165655558106754,-0.08327785391984875,0.08126185109898947,-0.009933630200775196,0.09735864643354517,-0.0615005935253543,-0.06174430829494161,-0.03867074995673561,0.17867498863381304,-0.14062926576170762,0.21641164291018986,0.051107843038230884,0.3580370658583978,0.024484739369756488,0.06645416596376418,0.08344289238752278,0.007613325445165436,0.19476219952218646,-0.11752054591540012,-0.00722626443634542,0.06515638108399038,0.11295416569154114,-0.07826906722751585,-0.060435967853682895,-0.03715890861511448,-0.08799622684310571,-0.09206358444092062,-0.02930704778486198,0.12445575222279603,-0.04938203040359851,0.016324553113430108,-0.041694126182578096,0.05107994022761724,0.08133728571733945,0.1306128525123853,-0.09130646341524418,-0.08633753476811608,-0.02459774899382118,0.1777874957794574,-0.08408380539881352,0.04224299760542824,-0.030035851797606178,-0.03550815837375678,-0.13222860736825057,0.03228444099872796,-0.0696121293747679,-0.04314950216938583,-0.01945962440623235,-0.22415735893315542,-0.00022383622799218556,-0.07445051356265081,-0.15093623282331964,-0.008190912294800363,-0.06291471504782097,-0.0833935682382325,-0.08244607279888878,0.02137695384978772,0.04953572812431484,0.04094371750382065,0.03780551111118961,-0.04964200861500929,-0.011818163073588267,-0.02441686577309903,-0.009408787680404295,0.09674763868818198,0.1267120861158286,-0.04461503212468937,0.011450269532123063,-0.07715406438971911,0.053882518378086774,0.1260060131568069,-0.053342631062027836,0.060550579414734566,0.12526730587777346,0.07183839924491008,0.06840447922778163,0.09132594879501013,-0.09346935969273211,0.030688121875914,-0.10751298611151039,0.03401601464668943,-0.0009652687198926032,0.026996722463152007,-0.053358219857272765,-0.09784095392286335,-0.14183379356683726,-0.011259541510243378,-0.006931866864744481,0.08231102784802186,0.04340508364988904,-0.11514222459204825,-0.03774338441692881,-0.0181880216053515,0.008689365886367968,-0.1707329042699843,0.04917322053519837,0.13001609959093124,0.1022418982470996,0.24717260586905043,-0.09022923478116973,-0.001559033227319535,0.181487373507162,0.1472451940433803,-0.06482772594683155,0.03811945014217169,0.05766920566521172,-0.06115063607495113,0.1947449314645693,0.05597530543455928,-0.03499931285006407,-0.12057027650952415,0.14542262508175519,0.019495876104451627,-0.089395853792459,-0.02763789566759399,-0.07416794422621088,0.1416564491013777,-0.057051036804628454,0.08483339848186261,-0.02883477954799167,0.09587349017290778,0.022464430888841855,-0.13856214868051567,0.022416008135631205],"pare_threshold":null,"picture":"schrodinger"}},"large-pared":{"bench_random.py::test_random_gradient[heisenberg]":{"basis":"majorana","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":400,"num_modes":128,"obs_terms":2000000,"seed":0},"measurement_kind":"pytest","operation":"gradient","parameters":[0.016400693043662398,0.05330219374461546,-0.0705585365990327,0.08476754745855934,-0.15193480882210897,-0.10129757289247394,0.0060443043275890005,0.0033006704879674394,0.17067988985270965,-0.08916685645374621,0.12613189835435187,-0.16222910655384135,0.040891271856525566,-0.1149045977068274,-0.05755222577163754,-0.12461566371227287,0.11774995001240425,0.03816598855107006,0.047555032869730784,-0.037665979696370644,0.07385086233087486,-0.05161060791550898,0.04282412057545871,-0.10388061544685463,-0.04656080441867633,-0.0029855177702576637,-0.10044530887179909,-0.007895569464808176,0.0708071867627511,0.09719084988871696,-0.03468737354302981,0.043013228204003995,0.0020010210255856893,-0.016863596767489743,-0.030937337128652737,0.11215984008436618,-0.22114731539727248,0.004086440358185286,-0.03520485106159149,-0.09021747968583696,-0.08572221069218883,0.013000094477226862,-0.13807311956012583,0.058160922962470746,-0.08175154625854542,0.003453209207847275,0.04089703519675422,0.003669508272555036,-0.12162447297958479,-0.06773854210337246,0.08070419531416607,-0.03218016234924814,-0.14906871097296168,0.04680321949393105,0.10302503823644318,-0.046380534186498165,-0.09449489099255681,0.06785468222107302,-0.11714870278510087,-0.06486539896765105,-0.11177345913505171,0.23746744912228307,0.030201985842687545,-0.18870509656509427,-0.20596448063263184,0.05853611513904732,0.012801172058615005,-0.06591351855745625,-0.06194068363028791,0.06821970647325686,0.029337966148215197,-0.11406292379426981,-0.04953735334304769,0.13669521026857187,-0.07338888957192352,0.17368281984797243,-0.01045244901510901,0.13367542508520355,-0.02448947161886537,0.007298895915872514,-0.11316662809120451,-0.019263475903101013,0.12462930631640301,-0.13434412884754396,0.017504781838779748,0.131682291679371,-0.042690174099431516,0.007544922399804037,0.11585811545602419,0.1130997717451217,-0.0005907849943751003,-0.006922018996999964,0.23036964972608667,0.09032244963124984,0.0501388301880246,-0.1301662977596475,0.04731459157257271,0.18576505500150753,0.075022879350012,0.058944213303005005,-0.041834020851077236,0.12978549893014532,0.0719361596514538,-0.0300851117170453,-0.06777999312990489,0.03928450028085989,0.03477793380909691,0.12226368606889648,-0.13274713408121458,-0.05524281048951133,-0.0189581679073641,0.16591228554997536,-0.13027531761515265,-0.05283220999217206,0.046459879640193484,-0.02044877769279148,0.11086636130910539,0.05275380231870326,0.11900879241654763,0.06076851266550035,0.008904936146376522,0.06349297834303429,0.01218103813012849,0.00041851956555569744,-0.09891555446328601,0.033849603095663705,-0.1850724022162653,0.05298388644630856,-0.10587366007901472,-0.009391604898195323,-0.0375707319776898,0.10554440252900106,0.1630172688385757,0.11275709609260479,-0.16540825113672053,0.08113262789956423,-0.026720595466532222,0.0443753353719741,0.007341852274681296,-0.12051664168854091,0.06188392804442369,0.04135931590249106,-0.04685210558651132,-0.09649784533930134,0.12714530030393717,-0.17422327591998577,0.05388901098024321,0.003144153968725154,0.0896730286841024,0.09615834776464798,-0.05404867587753102,0.1729287697288421,0.10211891127378668,0.1875337505678078,-0.0569683674585049,-0.02892526930122943,-0.02066087053671811,0.038198251282201896,-0.08556113879611502,0.09388052204298739,0.06140379567415778,0.11520079237437607,-0.05550634350357095,-0.1404310342362654,0.1211827206685111,0.024793632249527804,0.04862960182465336,-0.048515618953787906,-0.056633322476744574,-0.09569523308155137,-0.06025854122835222,0.01826453419177222,0.021750071675342465,0.0487236328972563,0.09896340027797944,0.1258921868088697,-0.21528654881923037,0.23744711970622925,0.07794430436522615,0.07562261205000631,0.0506778562116543,-0.02254790958661484,-0.03571557369773109,-0.039985784203548126,0.05425167155819899,-0.049536578413245665,0.19691403997362794,-0.08163944411468854,-0.037405261569227216,0.18918269359267037,0.07984206857477427,0.03809024162161545,-0.014898828041645962,0.01971162682033886,0.06724055831980186,0.17261930830681269,-0.025195506241274375,-0.1102600287318478,0.09521803134672806,0.005753896322139205,-0.1182360074468223,0.030768979529344067,0.07146022468010177,-0.06694873827800595,-0.028627917493452427,-0.0311439472465524,0.0992456524761879,0.18488102972597473,-0.032063688764073624,0.13872310377935976,-0.0073768114557231776,0.12962007024499148,-0.11828424798063351,-0.1334268794278036,-0.012498537756610903,0.04382123858602198,0.24885824172975043,0.06943051553862198,0.09103117581271185,0.05014294039954016,0.05552344652827205,0.09878339510357041,-0.05146748461222698,-0.0818833745671812,0.022183747829827682,0.010332045255538507,0.08857731193313047,-0.03392591502267615,0.05281141265088473,-0.14073430351172528,-0.030984329640816145,-0.045153666960562715,0.012134489525199767,-0.011241530709619293,0.049372412843166145,0.044903941605246564,0.10415619551293515,0.10691391983119487,-0.18190013887416495,-0.018248780479126644,0.026244674363329026,0.07223196059000313,-0.0282115873166085,-0.07697543000062129,0.009783991304479385,-0.03311344350740197,0.005173985164112646,0.15694428222905352,-0.004344193338604331,-0.016437742906754118,0.06900816820080378,-0.04215296151589517,-0.06274649935579278,0.035873443028829846,-0.1692324240142118,0.17462207013180245,-0.10217810902308222,-0.20179109147129393,0.08647563449167345,0.04891595114815255,-0.05301889410793609,-0.03195647740567347,-0.07810576559146204,0.01432774626108516,-0.04018899577856525,-0.026465187411549768,-0.017163474432223328,-0.007286885205411046,0.024366190661721555,-0.012676664886555509,0.060132173689510286,-0.0021225498977851427,-0.20199151520926578,-0.0589854714721007,-0.14073405270216338,0.04283894052409212,0.11620346390480635,-0.06811162810051763,-0.021327261208735623,0.2823491223862139,0.09857725268276941,0.1300272855361974,0.10162854008263147,-0.07284494230324078,0.0007409665951316078,-0.08303770911182705,0.02156446928404665,0.044556095036232604,-0.0036043675599702683,-0.12560093923705534,-0.02444595951480936,0.05807315499357597,-0.04353881296852248,-0.099618141049875,0.08285572124071197,0.13267871336724446,-0.09010425244291062,0.17771137250857158,0.04540604047026477,0.1833915885783089,-0.01444292991029257,-0.02749034062041287,0.025512571654009997,-0.15822735006952574,0.015801393378892362,0.1338664764790573,-0.07186415049857346,-0.08710422188730478,0.0423162461647298,0.02830699305259258,-0.15491273732031147,-0.1424759252179176,-0.027055741439198422,0.055657068737031905,-0.029444797516596555,0.09618676326244543,0.16128106518274896,0.08618051184576907,-0.05175626000697086,0.06815599854054319,-0.08670758931341853,0.006083061650996331,0.07302501161435006,-0.09341947709185561,0.09393226958376752,0.12564715732294632,0.004872416738449033,0.1273734137045372,0.14910571326417854,-0.0732509026073625,0.048459705783293804,-0.0458205486604196,-0.042556988046132393,0.1365550929414173,0.08821200722403609,0.09289997670896066,-0.06182144707771201,-0.12186678081623477,-0.007502098543791167,0.121208726887009,-0.04053964280052427,-0.13721899041569685,-0.06709262637109265,0.004935934756346951,0.12000618387999645,0.08092682074492252,-0.026949667229747395,-0.1454949851543202,-0.030997103665956025,-0.052049775907766375,0.08667587004741813,0.18340604132980282,0.06502073151043598,0.06849603778440697,0.05247484595845428,0.09705169679124963,0.0017041388785504201,-0.08835682487054249,0.13411730507553205,-0.026794429523946773,0.1684961689565902,0.013899553514746553,-0.12529917578851865,-0.10577144764195075,0.0889472182165657,0.02835905953681845,0.17076385971354408,0.13666465007999382,-0.0710102520115172,0.09181490536976147,0.004345283880384389,-0.12127189267957823,0.06770555453268857,-0.027612732559700072,0.1142636191432179,0.012353450287781925,0.04924137609197693,0.21485686854869013,-0.01628134359731778,0.035800114325442146,0.05193994641501318,0.11809571107448924,-0.04882722687249083,0.05479733137433027,0.19570829053705988,-0.13782224372065852,0.031232786845975337,0.032768443257354264,0.09968037138271345,-0.04593275595486472,0.008673141230843336,0.10370113596925015,0.00969590770957512,-0.01721745991106537,-0.18955442804026992,-0.02639685953766667,0.02094681471258093,-0.05733565707392492,0.06555042886274136,-0.15107773060231727],"pare_threshold":1e-10,"picture":"heisenberg"},"driver::pare_functional_construct[heisenberg]":{"basis":"majorana","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":400,"num_modes":128,"obs_terms":2000000,"seed":0},"functional":"expectation_value_and_gradient_functional","measurement_kind":"driver","operation":"pare_functional_construct","parameters":[0.016400693043662398,0.05330219374461546,-0.0705585365990327,0.08476754745855934,-0.15193480882210897,-0.10129757289247394,0.0060443043275890005,0.0033006704879674394,0.17067988985270965,-0.08916685645374621,0.12613189835435187,-0.16222910655384135,0.040891271856525566,-0.1149045977068274,-0.05755222577163754,-0.12461566371227287,0.11774995001240425,0.03816598855107006,0.047555032869730784,-0.037665979696370644,0.07385086233087486,-0.05161060791550898,0.04282412057545871,-0.10388061544685463,-0.04656080441867633,-0.0029855177702576637,-0.10044530887179909,-0.007895569464808176,0.0708071867627511,0.09719084988871696,-0.03468737354302981,0.043013228204003995,0.0020010210255856893,-0.016863596767489743,-0.030937337128652737,0.11215984008436618,-0.22114731539727248,0.004086440358185286,-0.03520485106159149,-0.09021747968583696,-0.08572221069218883,0.013000094477226862,-0.13807311956012583,0.058160922962470746,-0.08175154625854542,0.003453209207847275,0.04089703519675422,0.003669508272555036,-0.12162447297958479,-0.06773854210337246,0.08070419531416607,-0.03218016234924814,-0.14906871097296168,0.04680321949393105,0.10302503823644318,-0.046380534186498165,-0.09449489099255681,0.06785468222107302,-0.11714870278510087,-0.06486539896765105,-0.11177345913505171,0.23746744912228307,0.030201985842687545,-0.18870509656509427,-0.20596448063263184,0.05853611513904732,0.012801172058615005,-0.06591351855745625,-0.06194068363028791,0.06821970647325686,0.029337966148215197,-0.11406292379426981,-0.04953735334304769,0.13669521026857187,-0.07338888957192352,0.17368281984797243,-0.01045244901510901,0.13367542508520355,-0.02448947161886537,0.007298895915872514,-0.11316662809120451,-0.019263475903101013,0.12462930631640301,-0.13434412884754396,0.017504781838779748,0.131682291679371,-0.042690174099431516,0.007544922399804037,0.11585811545602419,0.1130997717451217,-0.0005907849943751003,-0.006922018996999964,0.23036964972608667,0.09032244963124984,0.0501388301880246,-0.1301662977596475,0.04731459157257271,0.18576505500150753,0.075022879350012,0.058944213303005005,-0.041834020851077236,0.12978549893014532,0.0719361596514538,-0.0300851117170453,-0.06777999312990489,0.03928450028085989,0.03477793380909691,0.12226368606889648,-0.13274713408121458,-0.05524281048951133,-0.0189581679073641,0.16591228554997536,-0.13027531761515265,-0.05283220999217206,0.046459879640193484,-0.02044877769279148,0.11086636130910539,0.05275380231870326,0.11900879241654763,0.06076851266550035,0.008904936146376522,0.06349297834303429,0.01218103813012849,0.00041851956555569744,-0.09891555446328601,0.033849603095663705,-0.1850724022162653,0.05298388644630856,-0.10587366007901472,-0.009391604898195323,-0.0375707319776898,0.10554440252900106,0.1630172688385757,0.11275709609260479,-0.16540825113672053,0.08113262789956423,-0.026720595466532222,0.0443753353719741,0.007341852274681296,-0.12051664168854091,0.06188392804442369,0.04135931590249106,-0.04685210558651132,-0.09649784533930134,0.12714530030393717,-0.17422327591998577,0.05388901098024321,0.003144153968725154,0.0896730286841024,0.09615834776464798,-0.05404867587753102,0.1729287697288421,0.10211891127378668,0.1875337505678078,-0.0569683674585049,-0.02892526930122943,-0.02066087053671811,0.038198251282201896,-0.08556113879611502,0.09388052204298739,0.06140379567415778,0.11520079237437607,-0.05550634350357095,-0.1404310342362654,0.1211827206685111,0.024793632249527804,0.04862960182465336,-0.048515618953787906,-0.056633322476744574,-0.09569523308155137,-0.06025854122835222,0.01826453419177222,0.021750071675342465,0.0487236328972563,0.09896340027797944,0.1258921868088697,-0.21528654881923037,0.23744711970622925,0.07794430436522615,0.07562261205000631,0.0506778562116543,-0.02254790958661484,-0.03571557369773109,-0.039985784203548126,0.05425167155819899,-0.049536578413245665,0.19691403997362794,-0.08163944411468854,-0.037405261569227216,0.18918269359267037,0.07984206857477427,0.03809024162161545,-0.014898828041645962,0.01971162682033886,0.06724055831980186,0.17261930830681269,-0.025195506241274375,-0.1102600287318478,0.09521803134672806,0.005753896322139205,-0.1182360074468223,0.030768979529344067,0.07146022468010177,-0.06694873827800595,-0.028627917493452427,-0.0311439472465524,0.0992456524761879,0.18488102972597473,-0.032063688764073624,0.13872310377935976,-0.0073768114557231776,0.12962007024499148,-0.11828424798063351,-0.1334268794278036,-0.012498537756610903,0.04382123858602198,0.24885824172975043,0.06943051553862198,0.09103117581271185,0.05014294039954016,0.05552344652827205,0.09878339510357041,-0.05146748461222698,-0.0818833745671812,0.022183747829827682,0.010332045255538507,0.08857731193313047,-0.03392591502267615,0.05281141265088473,-0.14073430351172528,-0.030984329640816145,-0.045153666960562715,0.012134489525199767,-0.011241530709619293,0.049372412843166145,0.044903941605246564,0.10415619551293515,0.10691391983119487,-0.18190013887416495,-0.018248780479126644,0.026244674363329026,0.07223196059000313,-0.0282115873166085,-0.07697543000062129,0.009783991304479385,-0.03311344350740197,0.005173985164112646,0.15694428222905352,-0.004344193338604331,-0.016437742906754118,0.06900816820080378,-0.04215296151589517,-0.06274649935579278,0.035873443028829846,-0.1692324240142118,0.17462207013180245,-0.10217810902308222,-0.20179109147129393,0.08647563449167345,0.04891595114815255,-0.05301889410793609,-0.03195647740567347,-0.07810576559146204,0.01432774626108516,-0.04018899577856525,-0.026465187411549768,-0.017163474432223328,-0.007286885205411046,0.024366190661721555,-0.012676664886555509,0.060132173689510286,-0.0021225498977851427,-0.20199151520926578,-0.0589854714721007,-0.14073405270216338,0.04283894052409212,0.11620346390480635,-0.06811162810051763,-0.021327261208735623,0.2823491223862139,0.09857725268276941,0.1300272855361974,0.10162854008263147,-0.07284494230324078,0.0007409665951316078,-0.08303770911182705,0.02156446928404665,0.044556095036232604,-0.0036043675599702683,-0.12560093923705534,-0.02444595951480936,0.05807315499357597,-0.04353881296852248,-0.099618141049875,0.08285572124071197,0.13267871336724446,-0.09010425244291062,0.17771137250857158,0.04540604047026477,0.1833915885783089,-0.01444292991029257,-0.02749034062041287,0.025512571654009997,-0.15822735006952574,0.015801393378892362,0.1338664764790573,-0.07186415049857346,-0.08710422188730478,0.0423162461647298,0.02830699305259258,-0.15491273732031147,-0.1424759252179176,-0.027055741439198422,0.055657068737031905,-0.029444797516596555,0.09618676326244543,0.16128106518274896,0.08618051184576907,-0.05175626000697086,0.06815599854054319,-0.08670758931341853,0.006083061650996331,0.07302501161435006,-0.09341947709185561,0.09393226958376752,0.12564715732294632,0.004872416738449033,0.1273734137045372,0.14910571326417854,-0.0732509026073625,0.048459705783293804,-0.0458205486604196,-0.042556988046132393,0.1365550929414173,0.08821200722403609,0.09289997670896066,-0.06182144707771201,-0.12186678081623477,-0.007502098543791167,0.121208726887009,-0.04053964280052427,-0.13721899041569685,-0.06709262637109265,0.004935934756346951,0.12000618387999645,0.08092682074492252,-0.026949667229747395,-0.1454949851543202,-0.030997103665956025,-0.052049775907766375,0.08667587004741813,0.18340604132980282,0.06502073151043598,0.06849603778440697,0.05247484595845428,0.09705169679124963,0.0017041388785504201,-0.08835682487054249,0.13411730507553205,-0.026794429523946773,0.1684961689565902,0.013899553514746553,-0.12529917578851865,-0.10577144764195075,0.0889472182165657,0.02835905953681845,0.17076385971354408,0.13666465007999382,-0.0710102520115172,0.09181490536976147,0.004345283880384389,-0.12127189267957823,0.06770555453268857,-0.027612732559700072,0.1142636191432179,0.012353450287781925,0.04924137609197693,0.21485686854869013,-0.01628134359731778,0.035800114325442146,0.05193994641501318,0.11809571107448924,-0.04882722687249083,0.05479733137433027,0.19570829053705988,-0.13782224372065852,0.031232786845975337,0.032768443257354264,0.09968037138271345,-0.04593275595486472,0.008673141230843336,0.10370113596925015,0.00969590770957512,-0.01721745991106537,-0.18955442804026992,-0.02639685953766667,0.02094681471258093,-0.05733565707392492,0.06555042886274136,-0.15107773060231727],"pare_threshold":1e-10,"picture":"heisenberg"},"driver::pare_functional_construct[schrodinger]":{"basis":"majorana","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":1000,"num_modes":160,"obs_terms":10000,"seed":0},"functional":"expectation_value_and_gradient_functional","measurement_kind":"driver","operation":"pare_functional_construct","parameters":[0.06276272766274849,-0.22687133805788584,-0.25107438577838603,-0.09677052490016542,-0.0768869602555572,0.15588376877677446,-0.06424349886973883,0.3173793275673169,0.02292820677316353,-0.07103322587570442,-0.12536315249015878,-0.07783013990818642,0.0812219862162197,0.07384491630065919,0.09692215372640231,-0.058441970213209606,-0.1882687568693983,-0.1013907789537107,-0.044067202037032016,-0.08436498813583376,0.023069294505968483,-0.12263524153943746,0.024522343574866573,-0.018876609494189355,-0.14376027944668926,-0.07776249419512249,-0.08257852994546848,0.18518041834187568,0.12278247077436039,0.07709848798665908,0.0013980501870870766,0.054197150674083164,0.06138268183868248,-0.11218162848941715,-0.09257758502871483,0.07835025053251038,-0.23283024306936217,-0.03637238612608577,0.061022767113721094,0.14925076227267092,0.12601266831141103,0.13430713483919132,-0.19850890895081724,-0.03740167178722114,-0.22060910190052885,0.06816291252550757,-0.11610566399108056,-0.04048684532387989,0.06681344911274748,0.13519687395401872,-0.026613764938776548,0.0643918265353292,0.12244395975673428,0.08724706391408271,0.013111874642129158,-0.10802290324013786,0.07700329657791315,0.1362040998630115,0.09766101175706404,0.03610730783130695,-0.05021932213979196,-0.06998801176227355,-0.017504094567420456,0.034341457491017906,0.18425915352979647,-0.03243523151481797,-0.061043447595918135,-0.0651985618903383,-0.016738636970275167,-0.006343264208759926,-0.3435898683215483,-0.04584456438455309,0.11515943170552606,0.03690016812634455,-0.059662457187573875,0.07603454652097243,-0.13108723609025472,0.08094048164889978,-0.03305525661466989,-0.053459638459425335,0.061422954304962245,-0.2361865712612381,0.04972795561776804,-0.008663511556076904,-0.07224422038606591,0.1712015172455634,-0.13612414625910044,0.20701819606643773,0.04330824574780291,0.10546747742916218,0.0024713848313154554,-0.0010271523659349166,0.0994183206124781,-0.010284709971628886,-0.20010120727011343,0.03637799611958672,-0.0009459388124698413,0.012782171099906404,0.09281141803232992,-0.0008368188479920023,-0.1192077289779303,-0.030359797293891225,-0.005582478864044986,0.0496679012831155,-0.17633335896951383,-0.1285506743889473,0.0902474625775841,-0.11415700212078837,-0.02583700354099643,-0.12210181556481117,-0.1093537941302389,-0.03644268692230384,-0.023235673809399927,0.06825227750129913,0.051819351059577835,0.00721494406398302,-0.13037313440545836,0.08768801073157384,0.08806034997795176,0.08766002071791226,-0.022753675946119406,-0.020097003836525976,0.08643822815366907,0.11277445670912867,0.055096265694647095,0.06140940944027082,0.14666070464504763,-0.04598226896809146,-0.09039766966533806,-0.015454741568079126,-0.048829308235195164,0.07353368362821668,0.05876922651983877,0.16014236058822495,0.024420808529096824,-0.09878803899653621,-0.06308696601268102,-0.06614664779644445,-0.12412050453467627,-0.0951049985104442,-0.07747543818838261,-0.10740184134844051,-0.11452704866127446,-0.10516166930239078,-0.21945815808588254,0.04431562421217815,0.13633071091941773,0.05666268193834978,-0.05556362607753662,-0.09611021105799235,-0.007315765638685897,-0.17274850315161908,0.13863517898102376,0.00935328366709223,-0.19976587612519586,0.1353295427175246,-0.1543136670505187,-0.0031255305572637708,0.11661455786257875,0.08130786940078792,0.005440063471758646,-0.16696171784979846,0.07327742927231254,0.10653083228715671,-0.0252884933849329,0.08148988367182947,0.0760276147574616,0.015446567026381797,0.24903335547945638,0.03546520965316171,-0.09615542694737728,0.08800882048406919,0.01280139989990979,0.020989878633498774,0.042864713123378376,0.07296203205558324,-0.02296559680976527,0.027133739799264363,-0.1553179366271642,0.08265767447156505,0.04296212882088783,0.13274898550633366,0.0716739536157238,0.04161903533781064,-0.09558908277483183,-0.08524921490475086,-0.0903061504724615,0.09445265515424968,0.013984977196329152,-0.1106208353418456,-0.21855158744986422,-0.015271519235740489,-0.13590314842918563,-0.04760164236494814,-0.14865481105561487,-0.0025558180929022983,0.05385973127464486,-0.05954531722846558,0.09771323732157962,0.04287758307951439,-0.019126035244953623,-0.19248655772788414,-0.010917334862613199,-0.018406352955598386,0.10262279003039934,-0.012793031398259842,0.0768071791957383,-0.009249009117021817,0.08321204572024274,-0.10823863178608238,0.1374747295045929,0.24354335778024186,0.04058738348807797,-0.048039633272225796,-0.1121766857989379,-0.11799419670067393,0.22754276450209454,-0.14546131797942977,0.03867542136353724,-0.04023639260870505,-0.09449997282739675,-0.0979820022295657,0.12546534966197684,-0.08797453028341676,-0.06534612222116108,-0.11440728430981593,0.08663795644396793,0.18020218605882393,0.07000584419966656,-0.057992813280183214,-0.15630872801165144,0.11966628485716478,0.05340254741900327,0.23780001607156379,-0.05016595182476339,0.07784226230588195,-0.08247705739325724,-0.08964584854585411,-0.022722077601874693,0.035551860308917556,-0.08932535014758965,0.019169609008638372,-0.01087113883521514,-0.010544015027172993,0.14607874127382556,0.16803857275361458,0.06292578552571965,-0.0767878652116652,0.03052783979832214,0.056327224982507876,0.036735000621115683,-0.03104377204097222,0.04622672432010137,-0.10094205816667796,0.053981510646019415,-0.027332522857129213,-0.04906120409482551,0.06642502017976502,-0.07719572532664998,0.06515868452686698,0.059448804997544236,-0.08123135278284248,-0.03177696530443739,-0.11764056361468399,-0.10431700209282568,-0.050695565769386546,-0.016323186423288033,0.10004425633838072,0.06847297485193375,-0.029856048241370548,-0.006627731129939755,0.00040038132552957986,0.14183377812871575,0.008267512806017804,0.04435443376664319,0.012967513398568876,0.03252220679860737,-0.0003913153207884925,0.17094553913503197,0.01898839975222227,0.08928386848946213,0.07000120902769709,0.036476206806316004,0.17725362729166114,0.11484071514804753,0.10653536662541885,0.12879190210000244,-0.1200038428243611,0.0374976652640413,-0.012282609548483623,-0.09403141265212324,-0.030905151689040084,-0.05704323692553618,0.1035307030633321,0.06311587365175637,-0.07699397287288551,-0.023958658974864076,0.037906818682203025,0.1385576670762419,0.18273916132584678,-0.0016654756326588479,0.14945427334128591,-0.18412382433919447,-0.023542061227968886,0.06259200290358802,0.19421437811141928,-0.01257219686249554,-0.084787712189934,0.0692689235579865,0.04010437041513535,0.038469608156980466,-0.31268923920341407,0.013686206019353242,-0.00623851946499252,0.0863255651117361,0.0028028297115834495,0.09686661295677647,0.005136295655841398,-0.04257446916084448,-0.12283314665983557,-0.1502479316976437,-0.12059661639744637,-0.05025492712810057,-0.06810791809905885,0.05713354476194441,0.025444939677748236,-0.14747016487058393,0.03958494353081173,0.08041381684969137,0.07640853575291244,0.043874081558348474,0.16721666790219472,0.16004645934333694,-0.1936153693546914,-0.1959543795330284,0.11388686591225328,0.0384217332679265,0.12794796373328163,-0.13531959723406325,-0.025134133141378384,0.02078492743332607,-0.05774294536278882,0.0129855522511809,-0.004711711865549823,0.08305130521302456,-0.04996638802760931,0.018289917794050047,0.062195361411023144,0.09007941158419147,-0.20426967453612802,-0.020632232037755972,0.026460196079060153,-0.11174803603006056,-0.004124911166063076,0.007916345938963818,0.08616409749621094,-0.04906698039110374,-0.047727380044708764,0.04292032035388957,-0.16529173220122098,0.044125206693123156,-0.06264343488957308,-0.06245184130205708,0.12666861074334027,0.11166469895597803,0.02181855967709821,0.11573445853352363,0.08815930209319889,0.13755718359904312,0.038175732200796106,-0.050346045926522545,0.039437674117072025,0.16476253280331302,-0.0879632715532659,0.016675240721986086,-0.025117801330188396,0.008994621746602736,0.06788556204772116,0.04419279247821052,0.03327512041787936,0.11892435724165472,0.031796657880267046,0.02056053655953996,-0.022756586335563957,0.07341792095351166,0.04527583313829037,0.037573683003254,-0.05753663856312268,-0.04339256839686785,0.062129660515415845,-0.0358289482780926,0.18590775162220208,-0.059137207662189044,-0.0029603745962606485,0.08231343041650153,0.06320264944590682,-0.036663591441645986,-0.06754377046919105,-0.1198671532910771,0.10849820823817868,-0.054365962015024305,-0.024344826739070947,-0.11978454102930912,-0.08751138478543875,0.10096765781361901,0.1592553317734185,0.1049044216460918,0.058353399542260424,0.16216641591277237,-0.17198990292750066,-0.007062917080313863,0.1521400735365031,0.11829342369956707,0.10133411582333612,0.051730783422424875,-0.08347512599052101,0.020369167996479712,-0.08884099859580673,0.21654698187314586,0.03894154076960041,0.09416643971005627,-0.16812131799990732,0.13996059406871625,-0.054983418766623174,0.03402532557280094,-0.03040371644820248,-0.07185734462260501,-0.16913570339082065,0.08673102732644461,-0.0013327833201409631,-0.010346814618200782,-0.12939658226981374,-0.11383167941947843,0.1264552409603924,0.004478209347693592,-0.03126673626067476,-0.08736033404313467,0.18613770993407067,-0.05019536018619178,-0.012763200702420409,0.21833625098345355,0.0031415343090802415,0.03214585936238269,-0.19992439058036954,0.018898255136305407,-0.18159234200610702,-0.052566158010671986,-0.08518631006209691,0.05439233950638141,-0.14947554771131635,-0.0753357975713261,0.06533608655651728,-0.03148845441409269,-0.15377216759815082,0.08171981232759805,0.06601896387894199,-0.08151610759750655,0.0557058437601177,0.12579823592046266,0.12356064076397244,0.051071203519584524,0.025945341856343836,0.18143397174950227,-0.07071046334785201,0.1433740459770921,-0.18430235092598657,-0.027609620049223033,0.05387074197106803,0.21336265281374966,-0.051367327251379324,-0.0023666398253829753,0.049599391408693905,-0.12711212099803618,-0.15406773747557295,0.002167613522097274,-0.07983683504432056,0.04541698889642274,0.03312388966717149,-0.21968157827654014,-0.04279851312408997,-0.07890121238954599,0.056516991840726595,0.04181414322707895,-0.015531835744269227,-0.08744550882590894,-0.08011557144078246,-0.08573933982348687,-0.025300166962526523,0.07738017671315045,-0.029581042776431823,0.13031946225149543,-0.142048298171351,-0.018758416230905982,0.05147261608210521,0.0064004556570420425,0.022577719107429062,0.0396500138842135,-0.16168061368095094,-0.17624158785641753,0.16255120984378366,0.11015215044527989,0.08003558880272359,0.021817096820655725,0.06787428734306573,-0.1287965100496475,0.017502807210445958,0.19383933085016433,0.07190743845628454,-0.24208634031967247,0.04335606554014287,-0.041717044255931424,-0.011322533685899822,-0.018624770469814738,-0.00970924003874039,0.06892341702762476,-0.025865512346863462,0.08316010061209371,-0.006272589306787965,0.174423970536427,-0.11326813877148766,0.06489320855412054,-0.16541842752647445,0.008261642738960037,-0.00836928657239852,0.03201442626110271,0.12021720134250767,0.035659262874604615,0.03587933684112218,-0.028508168886511694,-0.04557782330912097,-0.0314890020115514,0.18517036200962744,-0.03508077191419653,-0.2345105337505559,0.0826992677588738,-0.09340381597684011,-0.15956167298026425,-0.0004400591583051728,0.05915325275872283,-0.07118383227048762,0.12086389637690223,0.012818660645969813,-0.04616566129472341,-0.19181319255983897,-0.1536058207021391,0.029601639420800685,0.145714709292847,0.050168229332723004,-0.11145371693833046,0.00564670992202313,0.044899136704619036,0.0008862438898512453,0.11505909010788669,0.16152680701826272,0.0065780433879503416,-0.09181572541104183,-0.17374901198705717,0.12378213818535132,0.11395519763844895,0.14748461214987474,-0.06662685899293873,0.01551505195819542,-0.19439520314957695,-0.20815501251994442,-0.015955456787968957,0.0036190036598666023,-0.022156310424324442,0.04283773749201175,-0.20619499342346687,-0.12870818173728565,-0.08138598042959827,-0.07829865444977535,-0.025149130274242107,0.0766133724051146,0.14871612612785676,-0.19654243862138956,-0.010798447136809206,0.13222192142657307,-0.02927081900660914,0.10351332185865497,-0.12697339292674434,-0.08633490093299673,0.01843914560269134,0.07831805868044639,0.03136494151790882,0.06832837798767553,0.11015836877884924,0.08768606860753664,-0.10010869653467805,-0.12282213015182207,0.018780150146897254,-0.004817478353349955,0.049014687769521205,0.085513139696511,-0.0035945229108303722,-0.014555548065848693,-0.13746460665765944,0.07397832032728119,-0.04498679753937875,0.04434434677861934,-0.029475622868182106,-0.09299219600574608,-0.024447804090667092,-0.0177612559681759,0.08395658072999457,0.07269083568287427,0.07438164644971174,-0.05647700523749073,-0.0883431432100969,-0.17048730377977683,-0.07471450164216163,0.011958920771599973,-0.10457362520843257,-0.01599436664424899,-0.055634581748593925,0.015551649556405749,-0.08091461835186348,-0.06761212376340443,0.18077098202840478,0.039240072936793866,0.1203835032709618,-0.0013608211112286755,-0.041499513748177334,-0.11817568836055997,0.025763475351521237,0.021079135788591483,-0.05289605134209744,-0.062144222933871424,0.11035039961768499,0.059760137583236464,-0.13860172672192947,-0.03183545637617876,0.0493493418502237,-0.038187059552925055,-0.09780220405993975,-0.11333367792907652,0.04043691953392767,0.04642169090501244,-0.04527489244697058,0.020357835775801747,-0.04185657006687821,-0.06756195497207482,-0.1101415684208914,0.10844155021610935,-0.04609002659870931,-0.03907005350617264,-0.11116790745086227,-0.03987611775583941,0.019216383610989398,0.11614620600311915,-0.11518806473067658,0.006281303716553359,-0.04564041305458383,0.1093579830057281,0.1342993394886974,0.12340768415048549,0.1267777051513554,0.03835357945121847,0.12253054869570229,-0.07918572372879261,0.0777153560911588,-0.03848168143882725,-0.010855994650898874,-0.07830647935253518,-0.011618609254385383,0.13873313363304182,-0.11579264937841507,-0.2367323450485221,-0.15862269322029945,0.027401411728591993,-0.18675939665788843,-0.12002999676997964,0.05397306845591109,0.04971618410973413,-0.09988723267620188,-0.1841846220789959,-0.0800332936153636,0.021726542434816924,0.044614855781237785,0.14523247654594337,-0.027294541155590504,-0.07553536953486262,-0.09838085835050402,-0.13081983974217584,-0.09415415713650438,-0.0274999562429024,-0.046061382087644626,0.1612293515874996,0.1321809646616843,-0.09774763854433201,-0.029861256170780018,0.03735835074850183,0.09515589776802405,0.0819689431414217,0.09779381001208708,0.18147323725245262,0.0094328063250593,-0.05745389829724219,0.033760123072992525,-0.1871040168986563,-0.09227225740270387,0.010057940066828636,0.19376569853450898,-0.018285738762673892,-0.008241924887968008,-0.08150206880575511,0.03679848187947773,0.15967309767217883,0.06112726250139562,0.00699475333599775,-0.21558675380533,-0.1738584569354492,0.0005732778213747429,-0.005782887242626333,-0.04554664057657038,0.06627813118741502,0.0845772216506309,-0.07015217490388241,0.15515029066519065,-0.09948087310677357,0.05585498104891351,0.1418374095423812,0.02144527033404603,0.13772131149527675,-0.1679606942450375,-0.021550207386317455,-0.06389234553182785,0.046004161350740765,0.036703610371877485,0.20104863251058083,0.03233225931939441,-0.07794083727474331,-0.0022071257291329444,0.07085401420429162,-0.006355525076133531,0.07840096887525895,0.07487782242965475,0.15964787434371314,-0.11061012209853366,-0.0683479785828875,0.20891776560652012,0.03451726671877159,-0.08479041434197489,0.10502313099785515,0.11530310400473996,0.02428077695396679,0.07446242583113068,0.007218528216356839,0.0484837700055454,0.030881128968563928,-0.1020330527683294,0.15880174463081057,-0.10268313092098819,0.06800235675623091,0.16335061072477441,-0.08065054183838283,-0.01966661180125237,0.11161461701996248,0.009896800509713451,-0.05936131706769511,-0.026948518784022477,0.1565411035281179,0.0652408019401025,0.08143906451858968,0.07223254531398397,0.09178280057455607,-0.04434994585539389,-0.016236740163136212,-0.09815362643622287,0.011323276626718733,0.12922672414802824,0.09761459092916874,-0.04581776253275588,-0.10487696390037388,0.07524238388767557,0.10161630878381936,-0.03784332237029631,0.05438104964519358,0.12022333980028815,-0.09785817266488119,-0.045412321564340156,0.11448602567725381,0.08611692813763289,0.04254426451870566,0.048383086770820614,0.17920759702679206,-0.19035829983290264,0.21300297693798254,0.08662188675069189,0.04566247454038605,-0.13431105526079692,0.027268601927646116,-0.07278115031673667,0.1004410744480942,-0.2084271842590659,-0.10349908239071746,0.02845546455515921,-0.22334881746574953,0.03152153961461469,0.03002443769141083,0.06667283194193661,0.02342430059644278,-0.014476021401424462,-0.15733156979335905,0.04783622684090224,0.07431069306705987,-0.017691824788874985,0.09905205517569893,0.00833735536897295,0.17614677584488034,0.2201581295596142,0.10979836499676075,-0.0017513769337368203,0.008989017604878758,-0.07097408480244023,0.06421704595150812,-0.047963902733947045,0.033719179414828146,-0.07764484769639887,-0.014537651446404629,-0.11373141917285826,0.10155765021556813,-0.09949675568892846,0.1069386693428084,-0.06848211872373694,-0.014697889288646302,-0.021581114929070208,-0.2995159730723255,0.02198905047020407,-0.17332771347995246,0.005133672629540125,-0.028640957403942315,0.125648027960912,0.04078530639465178,0.00012728004522073163,-0.09906804042516798,0.0016294530031185682,0.025349215673843545,-0.10157970567550614,0.01957880643398501,-0.09137747673308441,-0.1506800719170716,0.17395244613940533,-0.19180439859280823,-0.08177379666604478,0.18819703901403773,-0.1525574385651186,-0.006694898675801525,0.07180940779474952,-0.05290534693258395,-0.03699707542331403,-0.06914502015298808,-0.0018101857393414353,0.12801597150630858,-0.03471212193213878,-0.11174575602062707,0.03590771869937087,0.07759335764075306,0.23016031995578243,0.1498311485587772,0.020039958089782928,-0.07869699538590262,-0.035115890510380623,0.1914057491770485,0.09191994656423602,-0.07727250322474635,0.0009421782344111768,-0.10524994362520167,0.0029321955985496346,0.1442918693807178,-0.03467189225853997,0.12944237987484342,0.08213483498015912,-0.07173206948558407,-0.013468167494366615,0.005427280542090472,-0.029165655558106754,-0.08327785391984875,0.08126185109898947,-0.009933630200775196,0.09735864643354517,-0.0615005935253543,-0.06174430829494161,-0.03867074995673561,0.17867498863381304,-0.14062926576170762,0.21641164291018986,0.051107843038230884,0.3580370658583978,0.024484739369756488,0.06645416596376418,0.08344289238752278,0.007613325445165436,0.19476219952218646,-0.11752054591540012,-0.00722626443634542,0.06515638108399038,0.11295416569154114,-0.07826906722751585,-0.060435967853682895,-0.03715890861511448,-0.08799622684310571,-0.09206358444092062,-0.02930704778486198,0.12445575222279603,-0.04938203040359851,0.016324553113430108,-0.041694126182578096,0.05107994022761724,0.08133728571733945,0.1306128525123853,-0.09130646341524418,-0.08633753476811608,-0.02459774899382118,0.1777874957794574,-0.08408380539881352,0.04224299760542824,-0.030035851797606178,-0.03550815837375678,-0.13222860736825057,0.03228444099872796,-0.0696121293747679,-0.04314950216938583,-0.01945962440623235,-0.22415735893315542,-0.00022383622799218556,-0.07445051356265081,-0.15093623282331964,-0.008190912294800363,-0.06291471504782097,-0.0833935682382325,-0.08244607279888878,0.02137695384978772,0.04953572812431484,0.04094371750382065,0.03780551111118961,-0.04964200861500929,-0.011818163073588267,-0.02441686577309903,-0.009408787680404295,0.09674763868818198,0.1267120861158286,-0.04461503212468937,0.011450269532123063,-0.07715406438971911,0.053882518378086774,0.1260060131568069,-0.053342631062027836,0.060550579414734566,0.12526730587777346,0.07183839924491008,0.06840447922778163,0.09132594879501013,-0.09346935969273211,0.030688121875914,-0.10751298611151039,0.03401601464668943,-0.0009652687198926032,0.026996722463152007,-0.053358219857272765,-0.09784095392286335,-0.14183379356683726,-0.011259541510243378,-0.006931866864744481,0.08231102784802186,0.04340508364988904,-0.11514222459204825,-0.03774338441692881,-0.0181880216053515,0.008689365886367968,-0.1707329042699843,0.04917322053519837,0.13001609959093124,0.1022418982470996,0.24717260586905043,-0.09022923478116973,-0.001559033227319535,0.181487373507162,0.1472451940433803,-0.06482772594683155,0.03811945014217169,0.05766920566521172,-0.06115063607495113,0.1947449314645693,0.05597530543455928,-0.03499931285006407,-0.12057027650952415,0.14542262508175519,0.019495876104451627,-0.089395853792459,-0.02763789566759399,-0.07416794422621088,0.1416564491013777,-0.057051036804628454,0.08483339848186261,-0.02883477954799167,0.09587349017290778,0.022464430888841855,-0.13856214868051567,0.022416008135631205],"pare_threshold":1e-10,"picture":"schrodinger"}},"reference":{"bench_models.py::test_model_build_graph[hubbard]":{"basis":"majorana","config":{"chemical_potential":0.0,"cutoff":6,"hopping":1.0,"interaction":-2.0,"lower_atol":3e-05,"neel_start_spin":"down","num_sites":24,"observable_site":18,"observable_spin":"up","trotter_dt":0.2,"trotter_steps":2},"measurement_kind":"pytest","operation":"build_graph","parameters":[0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2],"pare_threshold":null,"picture":"heisenberg"},"bench_models.py::test_model_build_graph[pauli]":{"basis":"pauli","config":{"coupling":0.7853981633974483,"cutoff":8,"lower_atol":3e-06,"num_layers":20,"num_qubits":127,"observable_qubit":62,"theta":0.7853981633974483},"measurement_kind":"pytest","operation":"build_graph","parameters":[-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483],"pare_threshold":null,"picture":"heisenberg"},"bench_models.py::test_model_energy[hubbard]":{"basis":"majorana","config":{"chemical_potential":0.0,"cutoff":6,"hopping":1.0,"interaction":-2.0,"lower_atol":3e-05,"neel_start_spin":"down","num_sites":24,"observable_site":18,"observable_spin":"up","trotter_dt":0.2,"trotter_steps":2},"measurement_kind":"pytest","operation":"energy","parameters":[0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2],"pare_threshold":null,"picture":"heisenberg"},"bench_models.py::test_model_energy[pauli]":{"basis":"pauli","config":{"coupling":0.7853981633974483,"cutoff":8,"lower_atol":3e-06,"num_layers":20,"num_qubits":127,"observable_qubit":62,"theta":0.7853981633974483},"measurement_kind":"pytest","operation":"energy","parameters":[-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483],"pare_threshold":null,"picture":"heisenberg"},"bench_models.py::test_model_gradient[hubbard]":{"basis":"majorana","config":{"chemical_potential":0.0,"cutoff":6,"hopping":1.0,"interaction":-2.0,"lower_atol":3e-05,"neel_start_spin":"down","num_sites":24,"observable_site":18,"observable_spin":"up","trotter_dt":0.2,"trotter_steps":2},"measurement_kind":"pytest","operation":"gradient","parameters":[0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2],"pare_threshold":null,"picture":"heisenberg"},"bench_models.py::test_model_gradient[pauli]":{"basis":"pauli","config":{"coupling":0.7853981633974483,"cutoff":8,"lower_atol":3e-06,"num_layers":20,"num_qubits":127,"observable_qubit":62,"theta":0.7853981633974483},"measurement_kind":"pytest","operation":"gradient","parameters":[-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483],"pare_threshold":null,"picture":"heisenberg"},"bench_models.py::test_model_propagate[hubbard]":{"basis":"majorana","config":{"chemical_potential":0.0,"cutoff":6,"hopping":1.0,"interaction":-2.0,"lower_atol":3e-05,"neel_start_spin":"down","num_sites":60,"observable_site":46,"observable_spin":"up","trotter_dt":0.2,"trotter_steps":29},"measurement_kind":"pytest","operation":"propagate","parameters":[0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2],"pare_threshold":null,"picture":"heisenberg"},"bench_models.py::test_model_propagate[pauli]":{"basis":"pauli","config":{"coupling":0.7853981633974483,"cutoff":8,"lower_atol":3e-06,"num_layers":20,"num_qubits":127,"observable_qubit":62,"theta":0.7853981633974483},"measurement_kind":"pytest","operation":"propagate","parameters":[-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483],"pare_threshold":null,"picture":"heisenberg"},"bench_random.py::test_random_build_graph[heisenberg]":{"basis":"majorana","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":400,"num_modes":128,"obs_terms":400000,"seed":0},"measurement_kind":"pytest","operation":"build_graph","parameters":[-0.0890874666997349,-0.18546478544594966,0.11530378524586189,0.09205503938041594,-0.04563737995115475,0.001733519464636122,0.16766083911233587,0.015906451204065943,-0.035335020936261376,-0.15408243487908768,0.06829336807511036,-0.09875456154423223,-0.09420659079934764,-0.13527046390030678,0.00032446012231942357,-0.08055153496093498,-0.07026793455847383,-0.16058044224920034,0.07800112374861702,-0.08917937491740122,0.10113674111769128,0.04078484540120101,-0.04838800845045549,-0.032941978230042045,-0.13743969066960246,0.04795773804369145,-0.04817213480178123,0.0922502468741554,0.08891839657471055,0.015148926559246434,0.029529785131937925,0.13915625508827403,-0.05915121300113967,-0.1721588692016708,0.04653487982462562,-0.04067082394459168,-0.02781581485374118,-0.03212701319877228,-0.10071955733142322,0.11542594804946232,-0.12014960175007597,0.15314417370885783,0.058188270193482605,-0.09042159699295656,-0.004376778231361494,-0.04101134553535272,-0.025174800230267736,0.04430038493251029,-0.07489774417794269,0.0476372310871176,-0.17202251118913703,0.07555640731779886,-0.010663963085352179,0.10588469256042365,-0.078394163158033,-0.21668027931856537,0.1113358686772602,-0.048817185825212966,0.09872469767299243,-0.0792036091629742,0.13772312242390494,-0.1042634933215457,-0.04640909746036348,0.14239438928058076,-0.09862648505712415,-0.011384462634295489,0.09156760709220976,-0.04474273967465235,0.04070936090857963,0.16046091900862924,-0.028335890245733256,0.034871528426287074,-0.04576902420616219,0.007053743509268884,0.09386941654071462,0.04452765130300167,0.16318608109591703,-0.032833304312874644,-0.015243199120535778,-0.04790142809545422,0.10994052087680833,0.042147467705845126,0.026961174465468044,0.02254642159569981,-0.0759321983713304,-0.03481189380279137,0.13324239106585972,0.11623609963830212,-0.0472276713901843,0.05123815718070845,-0.0680724838672439,0.22511199745617416,0.12374932713032648,0.09033595533694411,0.11918598620419814,-0.13643055599304252,0.09720094663385238,-0.05046012807597048,0.07580761412927266,0.0993583979513917,-0.05683581440605167,-0.062487357918517006,-0.009021135022060098,-0.029875071843633052,-0.030830109973128322,-0.10165196220737958,0.0825123841687644,-0.0027989438116668894,-0.07107374618031712,-0.015273335324415988,-0.011477001042720017,-0.11766860892804282,0.08259616215186905,-0.06401228130912244,0.15037250642819652,-0.07578795885990713,-0.13658021460199368,-0.05514202798819469,-0.15647726765568065,-0.0023845967728761616,-0.09666669379002862,-0.10425454326974425,0.04724233682704434,0.1269507835816496,0.08166858083660285,-0.04907598062522172,-0.011262794433605412,-0.10934280590870285,0.10752981680234273,-0.13957983453604317,0.023566507212558856,-0.05671152298320114,0.023774795035375187,-0.13167629503411202,0.07512442751075103,-0.04121427403094613,0.06277656983057854,0.004543518596217104,0.06044101727265512,0.1422438344495671,0.016420064424940324,0.0033381073596776543,-0.24234655728023138,0.13059549006802115,0.14909079646099657,0.026050239464906796,0.037162226172543264,-0.1333773902839095,0.06814161961258128,-0.012248160731059166,0.04337568254756503,0.13442050521779486,0.10448293032484424,-0.0382833599978901,-0.05606856687261135,-0.07821880014389376,-0.11038697791565365,0.162862719579702,-0.052358864550370324,-0.03983658753220438,0.046918653399639826,-0.20032230822116387,-0.15227562805572584,0.10455854050313973,-0.19308561797349275,-0.1351833836049526,-0.04770892358562029,-0.08158580605717293,0.20600367453290558,0.1490809684138873,-0.011648677628165363,0.055659466216815195,-0.0016674676206011045,-0.1447953637144113,0.05051978035662302,0.03152227812718284,0.026544866756992653,-0.06568971221197095,-0.04109098224630695,-0.12706134825321733,-0.16936461274659775,0.0931568919235316,0.2356966162438123,0.16100233689783544,-0.047468357141855544,0.03465752020783921,0.06592392415819469,0.06874752726128501,0.03927253992718121,-0.012583019215118965,-0.15322345348044122,-0.19788733052280702,0.10597664066227946,0.05586775653925852,0.08997349312459904,-0.07521727035218978,-0.0728973703173678,0.058172895714934315,-0.03183401489387498,-0.03782428697477435,0.15586337971773612,-0.17439397917678431,-0.22499283548255453,-0.12680424521787412,-0.03191469665644928,-0.059489063106776735,0.0005176370372658732,0.05914427122528237,-0.09189408429030793,0.1060244322316331,-0.04332586685847601,-0.09224233678563501,0.10440857700717475,0.03719525483409568,-0.06654287729297802,-0.08198593251352013,0.002363967787441058,0.2118517578544168,-0.04498785096396881,-0.09729142359573209,-0.095911070073797,0.006192877809233909,0.09592762285869111,-0.10140519551810741,0.007197425086942265,-0.04784407962721418,-0.12023484580086988,-0.0626322823914419,0.049299870525285366,-0.1499874563563304,0.06035911713661988,0.029550708953764094,-0.04940377673588078,0.09207687890724953,0.18305591357676754,-0.0984386077352897,0.12085328678257419,0.06231218771243483,-0.07412429531916709,-0.1047422711556157,-0.09360384099835178,0.09241093409626018,-0.13183846137925034,-0.0972721483201835,0.08968790155448197,-0.15590741997653682,0.11818248502653801,0.1147959621010341,0.05955294914560261,-0.21546636157612886,0.054848261235656684,-0.09264568802332213,-0.008191004720525079,0.1648338446001526,-0.07532107009485864,-0.030505240539493623,-0.06545017758297193,-0.16865218824286965,0.20255259388760738,-0.057665460726581845,-0.11254758036064694,0.10243041700782643,-0.243397237975634,-0.11445035404111492,-0.10924934909361839,-0.00562712041357458,-0.17217161675496168,-0.006278701168404551,-0.05336695644107105,-0.07938475147363878,0.33309510550299826,0.043968203979775626,-0.025602263830126495,0.04058514998804157,0.05014793609943289,-0.012685247779452628,-0.012900770600027495,0.2717524499041873,0.04188253246349927,0.05890498540909638,-0.02744379138405556,0.06700560680635273,0.09690937018597491,0.12244911182981869,-0.07028758284169366,-0.06644128494843506,0.13976492880805755,-0.07837449500034811,-0.0877381145978071,-0.05744391264757614,0.05188097352912799,0.001855959638506978,0.02800981738050803,0.08115964850080082,-0.15606246816866395,-0.19211404826896397,0.006601489076866402,0.05456730582303986,0.09913572095150547,-0.09609592692018036,-0.03142460015021777,0.010046031290633556,-0.2295507511833769,-0.06374948899959614,-0.0724862028962266,0.20754544668250352,0.12754046860794419,0.1170428859782735,-0.1066940447280706,0.10295280197475681,0.028351816784711784,-0.012924787782112396,0.2099453090838468,0.05593094808171548,-0.23469151516820508,-0.16466508164106,-0.09670375908769191,-0.1231510758421144,-0.09596439945792823,0.04988981426395063,0.021916014191161706,-0.1957375440130406,0.09833774122421912,0.08483230072027081,0.0682112706370084,-0.038024884707414594,-0.04727707895995233,-0.06531707433744781,-0.14594365988905186,-0.034874820602021184,-0.1379854906866556,-0.011318689768268284,0.06771350752230382,0.13118145713399762,-0.13425560884271381,0.06757395179523433,-0.015570342520249759,-0.0942987340432553,-0.05065649847286044,-0.05228210238746951,-0.06643297667287897,0.012428537816613775,-0.03598850382017678,0.12747486125538174,0.002944716881140423,-0.08197168659817991,0.07779017958081508,0.04879402419577096,-0.11547317797073486,0.026739606738835325,0.036132263988933995,-0.1212731233956646,-0.05055816091070684,-0.060011096042585844,-0.05866295314472206,-0.21160438461209302,-0.19405258421000715,-0.08662631742640342,0.00015806514670594984,-0.22391354024671606,0.10762153381092539,0.0409515246268395,-0.04977078352009195,-0.117338203410967,-0.08469487602993139,0.022631021869743473,-0.06890266422673769,-0.021026280312454723,-0.1282331386098067,-0.06699810269227373,0.14194370942159973,-0.006631494184976331,-0.037646417281089924,0.05410717086669751,0.08269698897152511,-0.01309707185135621,0.09039227672591284,-0.21236464809358557,-0.056338101094111706,0.08830892138383754,0.10710453334280996,0.0691890257415017,-0.06594512219260516,0.07847004356588017,-0.1255945537422462,-0.10529268011861069,0.02396054040307971,-0.048776168233594576,0.032647504353052796,0.07498018793640919,0.14212764933637376,-0.06780353290189946,0.04390064143023965,-0.10206261109486772,0.005559866783096345,0.18979666060549116,-0.11423984555346632,-0.028774029605737602,0.12449203535945763,-0.07694833283004784],"pare_threshold":null,"picture":"heisenberg"},"bench_random.py::test_random_build_graph[schrodinger]":{"basis":"majorana","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":400,"num_modes":128,"obs_terms":10000,"seed":0},"measurement_kind":"pytest","operation":"build_graph","parameters":[-0.10523340542759485,-0.08094137974967036,-0.06211595669994035,-0.08826667666730753,0.01577990223114918,0.08842021633067204,0.008764144630662368,-0.023338043483467942,-0.10123826606905634,-0.08083056169051887,-0.024483151549459133,0.19999587419240034,0.034402450495715046,-0.060740944460963814,-0.028372883375546682,-0.08123197657142059,0.054598417011416184,0.13226565264832876,0.1793829185275495,0.24500502342044142,0.016533138965112933,0.06398693841926537,-0.14071297616875642,-0.03582049725701987,-0.00829798504984494,-0.01989172269036455,-0.0890516639083282,0.03499511691944299,0.07842812126606673,0.013411144350238583,-0.0065272353626342474,-0.009933884573162247,-0.07130892691243998,-0.05462419968503196,0.05305440937501408,-0.16304998019855138,-0.033019373414405986,-0.12993026546956915,-0.018704197153130025,-0.0027448315477394914,0.027526252158962795,0.022111909469319724,-0.06693585658874013,0.07698158394797196,0.026521370076793685,0.07532162570586155,0.1379370993805433,-0.041699627186328886,0.03985001364123631,-0.10179778154977051,-0.08737644929329814,-0.05380890530176533,0.06318706740090722,-0.025949884263762563,-0.06148101314198309,0.06112432511268341,-0.01818615038827541,-0.050237547334459044,-0.03495661871365907,-0.16188998902660967,-0.060077798881129574,0.10667620519047481,-0.06962111156981816,-0.028749781758810607,0.017852773723701875,0.0787870777524101,-0.026547227979639355,0.19643236750966347,-0.003860402650380958,-0.10640764381079792,0.03370477002953416,0.10033302304501035,0.14497668277100265,0.012699918516428953,-0.047500966560800245,0.11387542005001526,0.03681447397268443,0.13268427823457848,0.19802289486958669,-0.10648417868874525,-0.1180630037398055,-0.10690756918016098,0.09876089717466784,0.08878273104161043,0.10031715253481406,0.09025533919979599,0.005619418535165455,0.10224702372001543,0.07042981647935002,0.07380342507193559,0.052056087034874424,-0.15832075587354974,0.15652289255233173,0.04798536387357363,0.006583519693100686,0.05519485520358874,-0.005808178493392487,-0.054414491623919994,0.09800899380380651,0.0007580460863512288,0.09877494476788876,-0.08322143393177485,0.12968259865322757,0.040166145726451156,0.13851020153237129,-0.07318155902699117,-0.05069211096908072,-0.011431279988824736,0.12038565312947776,0.09675248394729692,-0.020676130255665923,0.19937295294057358,-0.012253605707711245,0.1200007769714421,0.06076993421691586,-0.022209177216871566,0.0912040054245478,0.012296926819538985,-0.09127973610635993,0.06224516291358034,0.0925927318604553,-0.10493314928240707,-0.07923218230027412,-0.0002653231059715322,0.08593637911813679,0.12279512921150058,0.10937099005675983,-0.1423342298397086,0.10891864781121346,-0.04936298164773581,-0.04076803756295305,-0.08461463900825827,-0.01487385496575952,0.013855597169472595,0.011218175863522018,-0.07593378235863349,-0.020229517013817783,-0.0895784410380972,0.19782826851567517,-0.13909730849863666,0.08166197031784965,0.04617142010280429,0.008061062732140833,0.17919363095734697,-0.019496165364105866,-0.06364340056169956,0.2697046947954294,0.07419704867967905,0.08362910014866365,0.003699039883658526,0.08332288327560508,-0.06618332782239543,0.03479098298890444,0.09497854317778413,0.06187092273695474,-0.07746605899470924,-0.034022007171120076,-0.09972229600906303,-0.04992464701789866,0.11686620622841158,0.10282349844994852,0.11351699248306656,-0.09705515823400813,0.009908627904673596,-0.013585942798689105,0.04560729045169355,0.09707847038453056,-0.051240501180525644,-0.04996795185614452,0.1589160668959215,0.07392249083884464,-0.12107626565880228,0.05774357445707268,0.014757976336802016,0.25447976752326096,0.11422572518800539,0.013756692700837207,-0.07944551829280541,-0.08137450516773338,-0.07435315046291195,-0.05112227603565842,-0.03447050874636209,-0.13283255090881205,-0.09102970224852679,0.021807637740561294,-0.06208065445112028,-0.025317125969109424,-0.17591851134924535,0.23210253315613555,-0.08263221230933254,-0.06278766068647927,0.013579267414203542,-0.06500932008157909,0.12772468695594472,0.04063049227732398,-0.015668820672189564,0.029157767113167407,-0.0031122754321551266,-0.00334216123900371,0.12233145897166023,0.09506269625144646,0.10651030534322053,-0.04644062433963333,0.23480578970528848,0.11228442443075295,0.07698332555813456,0.17353825498695397,0.08725987776930247,-0.07463448275969331,0.07932281405532177,0.06145851921299405,0.03858208749745545,0.025642759176322263,-0.0469425534980268,0.17301852055435166,-0.09886777574194905,-0.07237998166424277,0.13464779769958665,-0.027228060372084546,-0.019277986477240685,-0.17795469174076958,0.1563135977092122,-0.08117125255680013,-0.09264691927909507,0.004536304279842132,0.0008860638629985287,-0.058087652819835,-0.0321934813060711,0.0019115894402543339,0.03557678522322349,-0.05562435443125942,0.022950161549708067,0.08646233258553271,-0.004598738243267859,-0.12760796938145713,-0.020859819916942003,-0.060057571800486646,0.05155087450201268,0.03457543563770801,0.16963634146736892,-0.023567107988920907,-0.0030039741671125907,0.03224314682493049,-0.05752676344112634,-0.06339305531763027,0.1761036830591626,-0.1395425461883431,-0.04199834295282049,-0.03987373875708988,0.04608634718373789,0.016400549341834006,-0.0296677251450764,0.1412902320205028,-0.14979458449883776,-0.08367398259233477,0.05843496141832834,0.09527081078155063,-0.01579054361549966,0.11270514706811362,0.03791389093329849,0.11373737959082018,-0.1436781267771055,-0.10118661405422243,-0.18073307102587235,-0.11997324584501234,0.0712774307251998,-0.021892258291943578,-0.02211352018449934,-0.03432059348785439,-0.018205405708011418,0.16932120202964476,0.04201886129213857,-0.10694128002393022,0.010975197993096299,0.1367131741994031,3.594379665730761e-05,-0.06783325840196479,-0.06502185513518512,0.034499829341458,0.004346579470375118,-0.0006053965820215505,0.04908374708464368,-0.13300035168236565,-0.08111286360949103,0.020152463819115868,0.054041477626494305,-0.02833680984081713,-0.01112808282760254,0.023569276110329735,-0.0715612452501808,0.034619146102981085,-0.06979541076825363,-0.08745912215659571,0.011046034292088417,0.02075852068485466,-0.043554076268817556,-0.16227923026761154,-0.053769039778167396,0.01500159100577821,-0.20103107311268806,-0.1963648174348411,0.049973345111997185,0.08860716511737221,0.017733471981668695,0.08951126941576142,-0.018238109415638384,-0.08038822264777062,-0.00569259630111204,0.07869350519807561,0.12312589300679651,0.15960947784472165,0.029853290634047975,-0.06580599198831669,0.15752576869119472,-0.01717554653450146,0.10097717033172518,0.07747923624936776,0.15487636241563757,-0.027908553406802435,0.08147803388732411,0.048879130310388985,0.08671082257965242,0.2957506819507744,0.09237929717423665,0.10642615471394044,0.027636145911902693,0.026817434550932936,0.07774753888931099,-0.16201184672538546,-0.03591035453837232,-0.0325000453859792,0.009225014061417921,-0.020693579859150775,-0.13717644071875332,-0.18477911926081758,-0.12172581533535208,-0.1334271627989746,-0.05382058901560284,0.17453497857495073,0.03264957154524198,0.051958568801748664,-0.03928178482942598,0.0026983474625661115,0.07669124819921451,-0.11008024556842873,-0.14613206722566724,0.04501809955263127,0.02128746148974579,-0.015510620049398256,-0.11367425165870917,0.09241077102411667,-0.01317280182320589,0.02782068478330976,-0.07972281307977037,0.024825883235338162,0.1523649515383286,-0.04654508846642184,0.03422461190210632,-0.025703881921715008,0.025350875685893132,0.1325151209985144,0.12625095888057677,0.02101661712380698,0.001503206784237066,-0.08711235100328199,-0.07986454182214933,-0.008376178730586025,0.1268798072043726,-0.023055980671505796,-0.08741503853062876,-0.1711030547413609,0.03434690591861681,0.21191675927767695,-0.011521095778679186,-0.03490135006870497,0.11365171325062196,0.0374178462644883,0.008348809074067513,-0.08428525051868793,-0.030581935457419492,-0.09803521685896925,0.0615682116968893,-0.062306658769597914,-0.009119007161406515,0.06502960935173034,-0.0266140654116749,-0.07917971644027381,0.015143567180107544,-0.018971143173674473,-0.0572915949379865,0.1841040937517825,0.13328607969270875,0.10792713265702464,-0.04497510148666434,0.18621618974887613,0.005994159306986166,-0.009452355550903995,0.11187824678851102,-0.07145892316115895,-0.016729257872477366],"pare_threshold":null,"picture":"schrodinger"},"bench_random.py::test_random_energy[heisenberg]":{"basis":"majorana","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":400,"num_modes":128,"obs_terms":400000,"seed":0},"measurement_kind":"pytest","operation":"energy","parameters":[-0.0890874666997349,-0.18546478544594966,0.11530378524586189,0.09205503938041594,-0.04563737995115475,0.001733519464636122,0.16766083911233587,0.015906451204065943,-0.035335020936261376,-0.15408243487908768,0.06829336807511036,-0.09875456154423223,-0.09420659079934764,-0.13527046390030678,0.00032446012231942357,-0.08055153496093498,-0.07026793455847383,-0.16058044224920034,0.07800112374861702,-0.08917937491740122,0.10113674111769128,0.04078484540120101,-0.04838800845045549,-0.032941978230042045,-0.13743969066960246,0.04795773804369145,-0.04817213480178123,0.0922502468741554,0.08891839657471055,0.015148926559246434,0.029529785131937925,0.13915625508827403,-0.05915121300113967,-0.1721588692016708,0.04653487982462562,-0.04067082394459168,-0.02781581485374118,-0.03212701319877228,-0.10071955733142322,0.11542594804946232,-0.12014960175007597,0.15314417370885783,0.058188270193482605,-0.09042159699295656,-0.004376778231361494,-0.04101134553535272,-0.025174800230267736,0.04430038493251029,-0.07489774417794269,0.0476372310871176,-0.17202251118913703,0.07555640731779886,-0.010663963085352179,0.10588469256042365,-0.078394163158033,-0.21668027931856537,0.1113358686772602,-0.048817185825212966,0.09872469767299243,-0.0792036091629742,0.13772312242390494,-0.1042634933215457,-0.04640909746036348,0.14239438928058076,-0.09862648505712415,-0.011384462634295489,0.09156760709220976,-0.04474273967465235,0.04070936090857963,0.16046091900862924,-0.028335890245733256,0.034871528426287074,-0.04576902420616219,0.007053743509268884,0.09386941654071462,0.04452765130300167,0.16318608109591703,-0.032833304312874644,-0.015243199120535778,-0.04790142809545422,0.10994052087680833,0.042147467705845126,0.026961174465468044,0.02254642159569981,-0.0759321983713304,-0.03481189380279137,0.13324239106585972,0.11623609963830212,-0.0472276713901843,0.05123815718070845,-0.0680724838672439,0.22511199745617416,0.12374932713032648,0.09033595533694411,0.11918598620419814,-0.13643055599304252,0.09720094663385238,-0.05046012807597048,0.07580761412927266,0.0993583979513917,-0.05683581440605167,-0.062487357918517006,-0.009021135022060098,-0.029875071843633052,-0.030830109973128322,-0.10165196220737958,0.0825123841687644,-0.0027989438116668894,-0.07107374618031712,-0.015273335324415988,-0.011477001042720017,-0.11766860892804282,0.08259616215186905,-0.06401228130912244,0.15037250642819652,-0.07578795885990713,-0.13658021460199368,-0.05514202798819469,-0.15647726765568065,-0.0023845967728761616,-0.09666669379002862,-0.10425454326974425,0.04724233682704434,0.1269507835816496,0.08166858083660285,-0.04907598062522172,-0.011262794433605412,-0.10934280590870285,0.10752981680234273,-0.13957983453604317,0.023566507212558856,-0.05671152298320114,0.023774795035375187,-0.13167629503411202,0.07512442751075103,-0.04121427403094613,0.06277656983057854,0.004543518596217104,0.06044101727265512,0.1422438344495671,0.016420064424940324,0.0033381073596776543,-0.24234655728023138,0.13059549006802115,0.14909079646099657,0.026050239464906796,0.037162226172543264,-0.1333773902839095,0.06814161961258128,-0.012248160731059166,0.04337568254756503,0.13442050521779486,0.10448293032484424,-0.0382833599978901,-0.05606856687261135,-0.07821880014389376,-0.11038697791565365,0.162862719579702,-0.052358864550370324,-0.03983658753220438,0.046918653399639826,-0.20032230822116387,-0.15227562805572584,0.10455854050313973,-0.19308561797349275,-0.1351833836049526,-0.04770892358562029,-0.08158580605717293,0.20600367453290558,0.1490809684138873,-0.011648677628165363,0.055659466216815195,-0.0016674676206011045,-0.1447953637144113,0.05051978035662302,0.03152227812718284,0.026544866756992653,-0.06568971221197095,-0.04109098224630695,-0.12706134825321733,-0.16936461274659775,0.0931568919235316,0.2356966162438123,0.16100233689783544,-0.047468357141855544,0.03465752020783921,0.06592392415819469,0.06874752726128501,0.03927253992718121,-0.012583019215118965,-0.15322345348044122,-0.19788733052280702,0.10597664066227946,0.05586775653925852,0.08997349312459904,-0.07521727035218978,-0.0728973703173678,0.058172895714934315,-0.03183401489387498,-0.03782428697477435,0.15586337971773612,-0.17439397917678431,-0.22499283548255453,-0.12680424521787412,-0.03191469665644928,-0.059489063106776735,0.0005176370372658732,0.05914427122528237,-0.09189408429030793,0.1060244322316331,-0.04332586685847601,-0.09224233678563501,0.10440857700717475,0.03719525483409568,-0.06654287729297802,-0.08198593251352013,0.002363967787441058,0.2118517578544168,-0.04498785096396881,-0.09729142359573209,-0.095911070073797,0.006192877809233909,0.09592762285869111,-0.10140519551810741,0.007197425086942265,-0.04784407962721418,-0.12023484580086988,-0.0626322823914419,0.049299870525285366,-0.1499874563563304,0.06035911713661988,0.029550708953764094,-0.04940377673588078,0.09207687890724953,0.18305591357676754,-0.0984386077352897,0.12085328678257419,0.06231218771243483,-0.07412429531916709,-0.1047422711556157,-0.09360384099835178,0.09241093409626018,-0.13183846137925034,-0.0972721483201835,0.08968790155448197,-0.15590741997653682,0.11818248502653801,0.1147959621010341,0.05955294914560261,-0.21546636157612886,0.054848261235656684,-0.09264568802332213,-0.008191004720525079,0.1648338446001526,-0.07532107009485864,-0.030505240539493623,-0.06545017758297193,-0.16865218824286965,0.20255259388760738,-0.057665460726581845,-0.11254758036064694,0.10243041700782643,-0.243397237975634,-0.11445035404111492,-0.10924934909361839,-0.00562712041357458,-0.17217161675496168,-0.006278701168404551,-0.05336695644107105,-0.07938475147363878,0.33309510550299826,0.043968203979775626,-0.025602263830126495,0.04058514998804157,0.05014793609943289,-0.012685247779452628,-0.012900770600027495,0.2717524499041873,0.04188253246349927,0.05890498540909638,-0.02744379138405556,0.06700560680635273,0.09690937018597491,0.12244911182981869,-0.07028758284169366,-0.06644128494843506,0.13976492880805755,-0.07837449500034811,-0.0877381145978071,-0.05744391264757614,0.05188097352912799,0.001855959638506978,0.02800981738050803,0.08115964850080082,-0.15606246816866395,-0.19211404826896397,0.006601489076866402,0.05456730582303986,0.09913572095150547,-0.09609592692018036,-0.03142460015021777,0.010046031290633556,-0.2295507511833769,-0.06374948899959614,-0.0724862028962266,0.20754544668250352,0.12754046860794419,0.1170428859782735,-0.1066940447280706,0.10295280197475681,0.028351816784711784,-0.012924787782112396,0.2099453090838468,0.05593094808171548,-0.23469151516820508,-0.16466508164106,-0.09670375908769191,-0.1231510758421144,-0.09596439945792823,0.04988981426395063,0.021916014191161706,-0.1957375440130406,0.09833774122421912,0.08483230072027081,0.0682112706370084,-0.038024884707414594,-0.04727707895995233,-0.06531707433744781,-0.14594365988905186,-0.034874820602021184,-0.1379854906866556,-0.011318689768268284,0.06771350752230382,0.13118145713399762,-0.13425560884271381,0.06757395179523433,-0.015570342520249759,-0.0942987340432553,-0.05065649847286044,-0.05228210238746951,-0.06643297667287897,0.012428537816613775,-0.03598850382017678,0.12747486125538174,0.002944716881140423,-0.08197168659817991,0.07779017958081508,0.04879402419577096,-0.11547317797073486,0.026739606738835325,0.036132263988933995,-0.1212731233956646,-0.05055816091070684,-0.060011096042585844,-0.05866295314472206,-0.21160438461209302,-0.19405258421000715,-0.08662631742640342,0.00015806514670594984,-0.22391354024671606,0.10762153381092539,0.0409515246268395,-0.04977078352009195,-0.117338203410967,-0.08469487602993139,0.022631021869743473,-0.06890266422673769,-0.021026280312454723,-0.1282331386098067,-0.06699810269227373,0.14194370942159973,-0.006631494184976331,-0.037646417281089924,0.05410717086669751,0.08269698897152511,-0.01309707185135621,0.09039227672591284,-0.21236464809358557,-0.056338101094111706,0.08830892138383754,0.10710453334280996,0.0691890257415017,-0.06594512219260516,0.07847004356588017,-0.1255945537422462,-0.10529268011861069,0.02396054040307971,-0.048776168233594576,0.032647504353052796,0.07498018793640919,0.14212764933637376,-0.06780353290189946,0.04390064143023965,-0.10206261109486772,0.005559866783096345,0.18979666060549116,-0.11423984555346632,-0.028774029605737602,0.12449203535945763,-0.07694833283004784],"pare_threshold":null,"picture":"heisenberg"},"bench_random.py::test_random_energy[schrodinger]":{"basis":"majorana","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":400,"num_modes":128,"obs_terms":10000,"seed":0},"measurement_kind":"pytest","operation":"energy","parameters":[-0.10523340542759485,-0.08094137974967036,-0.06211595669994035,-0.08826667666730753,0.01577990223114918,0.08842021633067204,0.008764144630662368,-0.023338043483467942,-0.10123826606905634,-0.08083056169051887,-0.024483151549459133,0.19999587419240034,0.034402450495715046,-0.060740944460963814,-0.028372883375546682,-0.08123197657142059,0.054598417011416184,0.13226565264832876,0.1793829185275495,0.24500502342044142,0.016533138965112933,0.06398693841926537,-0.14071297616875642,-0.03582049725701987,-0.00829798504984494,-0.01989172269036455,-0.0890516639083282,0.03499511691944299,0.07842812126606673,0.013411144350238583,-0.0065272353626342474,-0.009933884573162247,-0.07130892691243998,-0.05462419968503196,0.05305440937501408,-0.16304998019855138,-0.033019373414405986,-0.12993026546956915,-0.018704197153130025,-0.0027448315477394914,0.027526252158962795,0.022111909469319724,-0.06693585658874013,0.07698158394797196,0.026521370076793685,0.07532162570586155,0.1379370993805433,-0.041699627186328886,0.03985001364123631,-0.10179778154977051,-0.08737644929329814,-0.05380890530176533,0.06318706740090722,-0.025949884263762563,-0.06148101314198309,0.06112432511268341,-0.01818615038827541,-0.050237547334459044,-0.03495661871365907,-0.16188998902660967,-0.060077798881129574,0.10667620519047481,-0.06962111156981816,-0.028749781758810607,0.017852773723701875,0.0787870777524101,-0.026547227979639355,0.19643236750966347,-0.003860402650380958,-0.10640764381079792,0.03370477002953416,0.10033302304501035,0.14497668277100265,0.012699918516428953,-0.047500966560800245,0.11387542005001526,0.03681447397268443,0.13268427823457848,0.19802289486958669,-0.10648417868874525,-0.1180630037398055,-0.10690756918016098,0.09876089717466784,0.08878273104161043,0.10031715253481406,0.09025533919979599,0.005619418535165455,0.10224702372001543,0.07042981647935002,0.07380342507193559,0.052056087034874424,-0.15832075587354974,0.15652289255233173,0.04798536387357363,0.006583519693100686,0.05519485520358874,-0.005808178493392487,-0.054414491623919994,0.09800899380380651,0.0007580460863512288,0.09877494476788876,-0.08322143393177485,0.12968259865322757,0.040166145726451156,0.13851020153237129,-0.07318155902699117,-0.05069211096908072,-0.011431279988824736,0.12038565312947776,0.09675248394729692,-0.020676130255665923,0.19937295294057358,-0.012253605707711245,0.1200007769714421,0.06076993421691586,-0.022209177216871566,0.0912040054245478,0.012296926819538985,-0.09127973610635993,0.06224516291358034,0.0925927318604553,-0.10493314928240707,-0.07923218230027412,-0.0002653231059715322,0.08593637911813679,0.12279512921150058,0.10937099005675983,-0.1423342298397086,0.10891864781121346,-0.04936298164773581,-0.04076803756295305,-0.08461463900825827,-0.01487385496575952,0.013855597169472595,0.011218175863522018,-0.07593378235863349,-0.020229517013817783,-0.0895784410380972,0.19782826851567517,-0.13909730849863666,0.08166197031784965,0.04617142010280429,0.008061062732140833,0.17919363095734697,-0.019496165364105866,-0.06364340056169956,0.2697046947954294,0.07419704867967905,0.08362910014866365,0.003699039883658526,0.08332288327560508,-0.06618332782239543,0.03479098298890444,0.09497854317778413,0.06187092273695474,-0.07746605899470924,-0.034022007171120076,-0.09972229600906303,-0.04992464701789866,0.11686620622841158,0.10282349844994852,0.11351699248306656,-0.09705515823400813,0.009908627904673596,-0.013585942798689105,0.04560729045169355,0.09707847038453056,-0.051240501180525644,-0.04996795185614452,0.1589160668959215,0.07392249083884464,-0.12107626565880228,0.05774357445707268,0.014757976336802016,0.25447976752326096,0.11422572518800539,0.013756692700837207,-0.07944551829280541,-0.08137450516773338,-0.07435315046291195,-0.05112227603565842,-0.03447050874636209,-0.13283255090881205,-0.09102970224852679,0.021807637740561294,-0.06208065445112028,-0.025317125969109424,-0.17591851134924535,0.23210253315613555,-0.08263221230933254,-0.06278766068647927,0.013579267414203542,-0.06500932008157909,0.12772468695594472,0.04063049227732398,-0.015668820672189564,0.029157767113167407,-0.0031122754321551266,-0.00334216123900371,0.12233145897166023,0.09506269625144646,0.10651030534322053,-0.04644062433963333,0.23480578970528848,0.11228442443075295,0.07698332555813456,0.17353825498695397,0.08725987776930247,-0.07463448275969331,0.07932281405532177,0.06145851921299405,0.03858208749745545,0.025642759176322263,-0.0469425534980268,0.17301852055435166,-0.09886777574194905,-0.07237998166424277,0.13464779769958665,-0.027228060372084546,-0.019277986477240685,-0.17795469174076958,0.1563135977092122,-0.08117125255680013,-0.09264691927909507,0.004536304279842132,0.0008860638629985287,-0.058087652819835,-0.0321934813060711,0.0019115894402543339,0.03557678522322349,-0.05562435443125942,0.022950161549708067,0.08646233258553271,-0.004598738243267859,-0.12760796938145713,-0.020859819916942003,-0.060057571800486646,0.05155087450201268,0.03457543563770801,0.16963634146736892,-0.023567107988920907,-0.0030039741671125907,0.03224314682493049,-0.05752676344112634,-0.06339305531763027,0.1761036830591626,-0.1395425461883431,-0.04199834295282049,-0.03987373875708988,0.04608634718373789,0.016400549341834006,-0.0296677251450764,0.1412902320205028,-0.14979458449883776,-0.08367398259233477,0.05843496141832834,0.09527081078155063,-0.01579054361549966,0.11270514706811362,0.03791389093329849,0.11373737959082018,-0.1436781267771055,-0.10118661405422243,-0.18073307102587235,-0.11997324584501234,0.0712774307251998,-0.021892258291943578,-0.02211352018449934,-0.03432059348785439,-0.018205405708011418,0.16932120202964476,0.04201886129213857,-0.10694128002393022,0.010975197993096299,0.1367131741994031,3.594379665730761e-05,-0.06783325840196479,-0.06502185513518512,0.034499829341458,0.004346579470375118,-0.0006053965820215505,0.04908374708464368,-0.13300035168236565,-0.08111286360949103,0.020152463819115868,0.054041477626494305,-0.02833680984081713,-0.01112808282760254,0.023569276110329735,-0.0715612452501808,0.034619146102981085,-0.06979541076825363,-0.08745912215659571,0.011046034292088417,0.02075852068485466,-0.043554076268817556,-0.16227923026761154,-0.053769039778167396,0.01500159100577821,-0.20103107311268806,-0.1963648174348411,0.049973345111997185,0.08860716511737221,0.017733471981668695,0.08951126941576142,-0.018238109415638384,-0.08038822264777062,-0.00569259630111204,0.07869350519807561,0.12312589300679651,0.15960947784472165,0.029853290634047975,-0.06580599198831669,0.15752576869119472,-0.01717554653450146,0.10097717033172518,0.07747923624936776,0.15487636241563757,-0.027908553406802435,0.08147803388732411,0.048879130310388985,0.08671082257965242,0.2957506819507744,0.09237929717423665,0.10642615471394044,0.027636145911902693,0.026817434550932936,0.07774753888931099,-0.16201184672538546,-0.03591035453837232,-0.0325000453859792,0.009225014061417921,-0.020693579859150775,-0.13717644071875332,-0.18477911926081758,-0.12172581533535208,-0.1334271627989746,-0.05382058901560284,0.17453497857495073,0.03264957154524198,0.051958568801748664,-0.03928178482942598,0.0026983474625661115,0.07669124819921451,-0.11008024556842873,-0.14613206722566724,0.04501809955263127,0.02128746148974579,-0.015510620049398256,-0.11367425165870917,0.09241077102411667,-0.01317280182320589,0.02782068478330976,-0.07972281307977037,0.024825883235338162,0.1523649515383286,-0.04654508846642184,0.03422461190210632,-0.025703881921715008,0.025350875685893132,0.1325151209985144,0.12625095888057677,0.02101661712380698,0.001503206784237066,-0.08711235100328199,-0.07986454182214933,-0.008376178730586025,0.1268798072043726,-0.023055980671505796,-0.08741503853062876,-0.1711030547413609,0.03434690591861681,0.21191675927767695,-0.011521095778679186,-0.03490135006870497,0.11365171325062196,0.0374178462644883,0.008348809074067513,-0.08428525051868793,-0.030581935457419492,-0.09803521685896925,0.0615682116968893,-0.062306658769597914,-0.009119007161406515,0.06502960935173034,-0.0266140654116749,-0.07917971644027381,0.015143567180107544,-0.018971143173674473,-0.0572915949379865,0.1841040937517825,0.13328607969270875,0.10792713265702464,-0.04497510148666434,0.18621618974887613,0.005994159306986166,-0.009452355550903995,0.11187824678851102,-0.07145892316115895,-0.016729257872477366],"pare_threshold":null,"picture":"schrodinger"},"bench_random.py::test_random_gradient[heisenberg]":{"basis":"majorana","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":400,"num_modes":128,"obs_terms":400000,"seed":0},"measurement_kind":"pytest","operation":"gradient","parameters":[-0.0890874666997349,-0.18546478544594966,0.11530378524586189,0.09205503938041594,-0.04563737995115475,0.001733519464636122,0.16766083911233587,0.015906451204065943,-0.035335020936261376,-0.15408243487908768,0.06829336807511036,-0.09875456154423223,-0.09420659079934764,-0.13527046390030678,0.00032446012231942357,-0.08055153496093498,-0.07026793455847383,-0.16058044224920034,0.07800112374861702,-0.08917937491740122,0.10113674111769128,0.04078484540120101,-0.04838800845045549,-0.032941978230042045,-0.13743969066960246,0.04795773804369145,-0.04817213480178123,0.0922502468741554,0.08891839657471055,0.015148926559246434,0.029529785131937925,0.13915625508827403,-0.05915121300113967,-0.1721588692016708,0.04653487982462562,-0.04067082394459168,-0.02781581485374118,-0.03212701319877228,-0.10071955733142322,0.11542594804946232,-0.12014960175007597,0.15314417370885783,0.058188270193482605,-0.09042159699295656,-0.004376778231361494,-0.04101134553535272,-0.025174800230267736,0.04430038493251029,-0.07489774417794269,0.0476372310871176,-0.17202251118913703,0.07555640731779886,-0.010663963085352179,0.10588469256042365,-0.078394163158033,-0.21668027931856537,0.1113358686772602,-0.048817185825212966,0.09872469767299243,-0.0792036091629742,0.13772312242390494,-0.1042634933215457,-0.04640909746036348,0.14239438928058076,-0.09862648505712415,-0.011384462634295489,0.09156760709220976,-0.04474273967465235,0.04070936090857963,0.16046091900862924,-0.028335890245733256,0.034871528426287074,-0.04576902420616219,0.007053743509268884,0.09386941654071462,0.04452765130300167,0.16318608109591703,-0.032833304312874644,-0.015243199120535778,-0.04790142809545422,0.10994052087680833,0.042147467705845126,0.026961174465468044,0.02254642159569981,-0.0759321983713304,-0.03481189380279137,0.13324239106585972,0.11623609963830212,-0.0472276713901843,0.05123815718070845,-0.0680724838672439,0.22511199745617416,0.12374932713032648,0.09033595533694411,0.11918598620419814,-0.13643055599304252,0.09720094663385238,-0.05046012807597048,0.07580761412927266,0.0993583979513917,-0.05683581440605167,-0.062487357918517006,-0.009021135022060098,-0.029875071843633052,-0.030830109973128322,-0.10165196220737958,0.0825123841687644,-0.0027989438116668894,-0.07107374618031712,-0.015273335324415988,-0.011477001042720017,-0.11766860892804282,0.08259616215186905,-0.06401228130912244,0.15037250642819652,-0.07578795885990713,-0.13658021460199368,-0.05514202798819469,-0.15647726765568065,-0.0023845967728761616,-0.09666669379002862,-0.10425454326974425,0.04724233682704434,0.1269507835816496,0.08166858083660285,-0.04907598062522172,-0.011262794433605412,-0.10934280590870285,0.10752981680234273,-0.13957983453604317,0.023566507212558856,-0.05671152298320114,0.023774795035375187,-0.13167629503411202,0.07512442751075103,-0.04121427403094613,0.06277656983057854,0.004543518596217104,0.06044101727265512,0.1422438344495671,0.016420064424940324,0.0033381073596776543,-0.24234655728023138,0.13059549006802115,0.14909079646099657,0.026050239464906796,0.037162226172543264,-0.1333773902839095,0.06814161961258128,-0.012248160731059166,0.04337568254756503,0.13442050521779486,0.10448293032484424,-0.0382833599978901,-0.05606856687261135,-0.07821880014389376,-0.11038697791565365,0.162862719579702,-0.052358864550370324,-0.03983658753220438,0.046918653399639826,-0.20032230822116387,-0.15227562805572584,0.10455854050313973,-0.19308561797349275,-0.1351833836049526,-0.04770892358562029,-0.08158580605717293,0.20600367453290558,0.1490809684138873,-0.011648677628165363,0.055659466216815195,-0.0016674676206011045,-0.1447953637144113,0.05051978035662302,0.03152227812718284,0.026544866756992653,-0.06568971221197095,-0.04109098224630695,-0.12706134825321733,-0.16936461274659775,0.0931568919235316,0.2356966162438123,0.16100233689783544,-0.047468357141855544,0.03465752020783921,0.06592392415819469,0.06874752726128501,0.03927253992718121,-0.012583019215118965,-0.15322345348044122,-0.19788733052280702,0.10597664066227946,0.05586775653925852,0.08997349312459904,-0.07521727035218978,-0.0728973703173678,0.058172895714934315,-0.03183401489387498,-0.03782428697477435,0.15586337971773612,-0.17439397917678431,-0.22499283548255453,-0.12680424521787412,-0.03191469665644928,-0.059489063106776735,0.0005176370372658732,0.05914427122528237,-0.09189408429030793,0.1060244322316331,-0.04332586685847601,-0.09224233678563501,0.10440857700717475,0.03719525483409568,-0.06654287729297802,-0.08198593251352013,0.002363967787441058,0.2118517578544168,-0.04498785096396881,-0.09729142359573209,-0.095911070073797,0.006192877809233909,0.09592762285869111,-0.10140519551810741,0.007197425086942265,-0.04784407962721418,-0.12023484580086988,-0.0626322823914419,0.049299870525285366,-0.1499874563563304,0.06035911713661988,0.029550708953764094,-0.04940377673588078,0.09207687890724953,0.18305591357676754,-0.0984386077352897,0.12085328678257419,0.06231218771243483,-0.07412429531916709,-0.1047422711556157,-0.09360384099835178,0.09241093409626018,-0.13183846137925034,-0.0972721483201835,0.08968790155448197,-0.15590741997653682,0.11818248502653801,0.1147959621010341,0.05955294914560261,-0.21546636157612886,0.054848261235656684,-0.09264568802332213,-0.008191004720525079,0.1648338446001526,-0.07532107009485864,-0.030505240539493623,-0.06545017758297193,-0.16865218824286965,0.20255259388760738,-0.057665460726581845,-0.11254758036064694,0.10243041700782643,-0.243397237975634,-0.11445035404111492,-0.10924934909361839,-0.00562712041357458,-0.17217161675496168,-0.006278701168404551,-0.05336695644107105,-0.07938475147363878,0.33309510550299826,0.043968203979775626,-0.025602263830126495,0.04058514998804157,0.05014793609943289,-0.012685247779452628,-0.012900770600027495,0.2717524499041873,0.04188253246349927,0.05890498540909638,-0.02744379138405556,0.06700560680635273,0.09690937018597491,0.12244911182981869,-0.07028758284169366,-0.06644128494843506,0.13976492880805755,-0.07837449500034811,-0.0877381145978071,-0.05744391264757614,0.05188097352912799,0.001855959638506978,0.02800981738050803,0.08115964850080082,-0.15606246816866395,-0.19211404826896397,0.006601489076866402,0.05456730582303986,0.09913572095150547,-0.09609592692018036,-0.03142460015021777,0.010046031290633556,-0.2295507511833769,-0.06374948899959614,-0.0724862028962266,0.20754544668250352,0.12754046860794419,0.1170428859782735,-0.1066940447280706,0.10295280197475681,0.028351816784711784,-0.012924787782112396,0.2099453090838468,0.05593094808171548,-0.23469151516820508,-0.16466508164106,-0.09670375908769191,-0.1231510758421144,-0.09596439945792823,0.04988981426395063,0.021916014191161706,-0.1957375440130406,0.09833774122421912,0.08483230072027081,0.0682112706370084,-0.038024884707414594,-0.04727707895995233,-0.06531707433744781,-0.14594365988905186,-0.034874820602021184,-0.1379854906866556,-0.011318689768268284,0.06771350752230382,0.13118145713399762,-0.13425560884271381,0.06757395179523433,-0.015570342520249759,-0.0942987340432553,-0.05065649847286044,-0.05228210238746951,-0.06643297667287897,0.012428537816613775,-0.03598850382017678,0.12747486125538174,0.002944716881140423,-0.08197168659817991,0.07779017958081508,0.04879402419577096,-0.11547317797073486,0.026739606738835325,0.036132263988933995,-0.1212731233956646,-0.05055816091070684,-0.060011096042585844,-0.05866295314472206,-0.21160438461209302,-0.19405258421000715,-0.08662631742640342,0.00015806514670594984,-0.22391354024671606,0.10762153381092539,0.0409515246268395,-0.04977078352009195,-0.117338203410967,-0.08469487602993139,0.022631021869743473,-0.06890266422673769,-0.021026280312454723,-0.1282331386098067,-0.06699810269227373,0.14194370942159973,-0.006631494184976331,-0.037646417281089924,0.05410717086669751,0.08269698897152511,-0.01309707185135621,0.09039227672591284,-0.21236464809358557,-0.056338101094111706,0.08830892138383754,0.10710453334280996,0.0691890257415017,-0.06594512219260516,0.07847004356588017,-0.1255945537422462,-0.10529268011861069,0.02396054040307971,-0.048776168233594576,0.032647504353052796,0.07498018793640919,0.14212764933637376,-0.06780353290189946,0.04390064143023965,-0.10206261109486772,0.005559866783096345,0.18979666060549116,-0.11423984555346632,-0.028774029605737602,0.12449203535945763,-0.07694833283004784],"pare_threshold":null,"picture":"heisenberg"},"bench_random.py::test_random_gradient[schrodinger]":{"basis":"majorana","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":400,"num_modes":128,"obs_terms":10000,"seed":0},"measurement_kind":"pytest","operation":"gradient","parameters":[-0.10523340542759485,-0.08094137974967036,-0.06211595669994035,-0.08826667666730753,0.01577990223114918,0.08842021633067204,0.008764144630662368,-0.023338043483467942,-0.10123826606905634,-0.08083056169051887,-0.024483151549459133,0.19999587419240034,0.034402450495715046,-0.060740944460963814,-0.028372883375546682,-0.08123197657142059,0.054598417011416184,0.13226565264832876,0.1793829185275495,0.24500502342044142,0.016533138965112933,0.06398693841926537,-0.14071297616875642,-0.03582049725701987,-0.00829798504984494,-0.01989172269036455,-0.0890516639083282,0.03499511691944299,0.07842812126606673,0.013411144350238583,-0.0065272353626342474,-0.009933884573162247,-0.07130892691243998,-0.05462419968503196,0.05305440937501408,-0.16304998019855138,-0.033019373414405986,-0.12993026546956915,-0.018704197153130025,-0.0027448315477394914,0.027526252158962795,0.022111909469319724,-0.06693585658874013,0.07698158394797196,0.026521370076793685,0.07532162570586155,0.1379370993805433,-0.041699627186328886,0.03985001364123631,-0.10179778154977051,-0.08737644929329814,-0.05380890530176533,0.06318706740090722,-0.025949884263762563,-0.06148101314198309,0.06112432511268341,-0.01818615038827541,-0.050237547334459044,-0.03495661871365907,-0.16188998902660967,-0.060077798881129574,0.10667620519047481,-0.06962111156981816,-0.028749781758810607,0.017852773723701875,0.0787870777524101,-0.026547227979639355,0.19643236750966347,-0.003860402650380958,-0.10640764381079792,0.03370477002953416,0.10033302304501035,0.14497668277100265,0.012699918516428953,-0.047500966560800245,0.11387542005001526,0.03681447397268443,0.13268427823457848,0.19802289486958669,-0.10648417868874525,-0.1180630037398055,-0.10690756918016098,0.09876089717466784,0.08878273104161043,0.10031715253481406,0.09025533919979599,0.005619418535165455,0.10224702372001543,0.07042981647935002,0.07380342507193559,0.052056087034874424,-0.15832075587354974,0.15652289255233173,0.04798536387357363,0.006583519693100686,0.05519485520358874,-0.005808178493392487,-0.054414491623919994,0.09800899380380651,0.0007580460863512288,0.09877494476788876,-0.08322143393177485,0.12968259865322757,0.040166145726451156,0.13851020153237129,-0.07318155902699117,-0.05069211096908072,-0.011431279988824736,0.12038565312947776,0.09675248394729692,-0.020676130255665923,0.19937295294057358,-0.012253605707711245,0.1200007769714421,0.06076993421691586,-0.022209177216871566,0.0912040054245478,0.012296926819538985,-0.09127973610635993,0.06224516291358034,0.0925927318604553,-0.10493314928240707,-0.07923218230027412,-0.0002653231059715322,0.08593637911813679,0.12279512921150058,0.10937099005675983,-0.1423342298397086,0.10891864781121346,-0.04936298164773581,-0.04076803756295305,-0.08461463900825827,-0.01487385496575952,0.013855597169472595,0.011218175863522018,-0.07593378235863349,-0.020229517013817783,-0.0895784410380972,0.19782826851567517,-0.13909730849863666,0.08166197031784965,0.04617142010280429,0.008061062732140833,0.17919363095734697,-0.019496165364105866,-0.06364340056169956,0.2697046947954294,0.07419704867967905,0.08362910014866365,0.003699039883658526,0.08332288327560508,-0.06618332782239543,0.03479098298890444,0.09497854317778413,0.06187092273695474,-0.07746605899470924,-0.034022007171120076,-0.09972229600906303,-0.04992464701789866,0.11686620622841158,0.10282349844994852,0.11351699248306656,-0.09705515823400813,0.009908627904673596,-0.013585942798689105,0.04560729045169355,0.09707847038453056,-0.051240501180525644,-0.04996795185614452,0.1589160668959215,0.07392249083884464,-0.12107626565880228,0.05774357445707268,0.014757976336802016,0.25447976752326096,0.11422572518800539,0.013756692700837207,-0.07944551829280541,-0.08137450516773338,-0.07435315046291195,-0.05112227603565842,-0.03447050874636209,-0.13283255090881205,-0.09102970224852679,0.021807637740561294,-0.06208065445112028,-0.025317125969109424,-0.17591851134924535,0.23210253315613555,-0.08263221230933254,-0.06278766068647927,0.013579267414203542,-0.06500932008157909,0.12772468695594472,0.04063049227732398,-0.015668820672189564,0.029157767113167407,-0.0031122754321551266,-0.00334216123900371,0.12233145897166023,0.09506269625144646,0.10651030534322053,-0.04644062433963333,0.23480578970528848,0.11228442443075295,0.07698332555813456,0.17353825498695397,0.08725987776930247,-0.07463448275969331,0.07932281405532177,0.06145851921299405,0.03858208749745545,0.025642759176322263,-0.0469425534980268,0.17301852055435166,-0.09886777574194905,-0.07237998166424277,0.13464779769958665,-0.027228060372084546,-0.019277986477240685,-0.17795469174076958,0.1563135977092122,-0.08117125255680013,-0.09264691927909507,0.004536304279842132,0.0008860638629985287,-0.058087652819835,-0.0321934813060711,0.0019115894402543339,0.03557678522322349,-0.05562435443125942,0.022950161549708067,0.08646233258553271,-0.004598738243267859,-0.12760796938145713,-0.020859819916942003,-0.060057571800486646,0.05155087450201268,0.03457543563770801,0.16963634146736892,-0.023567107988920907,-0.0030039741671125907,0.03224314682493049,-0.05752676344112634,-0.06339305531763027,0.1761036830591626,-0.1395425461883431,-0.04199834295282049,-0.03987373875708988,0.04608634718373789,0.016400549341834006,-0.0296677251450764,0.1412902320205028,-0.14979458449883776,-0.08367398259233477,0.05843496141832834,0.09527081078155063,-0.01579054361549966,0.11270514706811362,0.03791389093329849,0.11373737959082018,-0.1436781267771055,-0.10118661405422243,-0.18073307102587235,-0.11997324584501234,0.0712774307251998,-0.021892258291943578,-0.02211352018449934,-0.03432059348785439,-0.018205405708011418,0.16932120202964476,0.04201886129213857,-0.10694128002393022,0.010975197993096299,0.1367131741994031,3.594379665730761e-05,-0.06783325840196479,-0.06502185513518512,0.034499829341458,0.004346579470375118,-0.0006053965820215505,0.04908374708464368,-0.13300035168236565,-0.08111286360949103,0.020152463819115868,0.054041477626494305,-0.02833680984081713,-0.01112808282760254,0.023569276110329735,-0.0715612452501808,0.034619146102981085,-0.06979541076825363,-0.08745912215659571,0.011046034292088417,0.02075852068485466,-0.043554076268817556,-0.16227923026761154,-0.053769039778167396,0.01500159100577821,-0.20103107311268806,-0.1963648174348411,0.049973345111997185,0.08860716511737221,0.017733471981668695,0.08951126941576142,-0.018238109415638384,-0.08038822264777062,-0.00569259630111204,0.07869350519807561,0.12312589300679651,0.15960947784472165,0.029853290634047975,-0.06580599198831669,0.15752576869119472,-0.01717554653450146,0.10097717033172518,0.07747923624936776,0.15487636241563757,-0.027908553406802435,0.08147803388732411,0.048879130310388985,0.08671082257965242,0.2957506819507744,0.09237929717423665,0.10642615471394044,0.027636145911902693,0.026817434550932936,0.07774753888931099,-0.16201184672538546,-0.03591035453837232,-0.0325000453859792,0.009225014061417921,-0.020693579859150775,-0.13717644071875332,-0.18477911926081758,-0.12172581533535208,-0.1334271627989746,-0.05382058901560284,0.17453497857495073,0.03264957154524198,0.051958568801748664,-0.03928178482942598,0.0026983474625661115,0.07669124819921451,-0.11008024556842873,-0.14613206722566724,0.04501809955263127,0.02128746148974579,-0.015510620049398256,-0.11367425165870917,0.09241077102411667,-0.01317280182320589,0.02782068478330976,-0.07972281307977037,0.024825883235338162,0.1523649515383286,-0.04654508846642184,0.03422461190210632,-0.025703881921715008,0.025350875685893132,0.1325151209985144,0.12625095888057677,0.02101661712380698,0.001503206784237066,-0.08711235100328199,-0.07986454182214933,-0.008376178730586025,0.1268798072043726,-0.023055980671505796,-0.08741503853062876,-0.1711030547413609,0.03434690591861681,0.21191675927767695,-0.011521095778679186,-0.03490135006870497,0.11365171325062196,0.0374178462644883,0.008348809074067513,-0.08428525051868793,-0.030581935457419492,-0.09803521685896925,0.0615682116968893,-0.062306658769597914,-0.009119007161406515,0.06502960935173034,-0.0266140654116749,-0.07917971644027381,0.015143567180107544,-0.018971143173674473,-0.0572915949379865,0.1841040937517825,0.13328607969270875,0.10792713265702464,-0.04497510148666434,0.18621618974887613,0.005994159306986166,-0.009452355550903995,0.11187824678851102,-0.07145892316115895,-0.016729257872477366],"pare_threshold":null,"picture":"schrodinger"},"bench_random.py::test_random_propagate[heisenberg]":{"basis":"majorana","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":400,"num_modes":128,"obs_terms":400000,"seed":0},"measurement_kind":"pytest","operation":"propagate","parameters":[-0.0890874666997349,-0.18546478544594966,0.11530378524586189,0.09205503938041594,-0.04563737995115475,0.001733519464636122,0.16766083911233587,0.015906451204065943,-0.035335020936261376,-0.15408243487908768,0.06829336807511036,-0.09875456154423223,-0.09420659079934764,-0.13527046390030678,0.00032446012231942357,-0.08055153496093498,-0.07026793455847383,-0.16058044224920034,0.07800112374861702,-0.08917937491740122,0.10113674111769128,0.04078484540120101,-0.04838800845045549,-0.032941978230042045,-0.13743969066960246,0.04795773804369145,-0.04817213480178123,0.0922502468741554,0.08891839657471055,0.015148926559246434,0.029529785131937925,0.13915625508827403,-0.05915121300113967,-0.1721588692016708,0.04653487982462562,-0.04067082394459168,-0.02781581485374118,-0.03212701319877228,-0.10071955733142322,0.11542594804946232,-0.12014960175007597,0.15314417370885783,0.058188270193482605,-0.09042159699295656,-0.004376778231361494,-0.04101134553535272,-0.025174800230267736,0.04430038493251029,-0.07489774417794269,0.0476372310871176,-0.17202251118913703,0.07555640731779886,-0.010663963085352179,0.10588469256042365,-0.078394163158033,-0.21668027931856537,0.1113358686772602,-0.048817185825212966,0.09872469767299243,-0.0792036091629742,0.13772312242390494,-0.1042634933215457,-0.04640909746036348,0.14239438928058076,-0.09862648505712415,-0.011384462634295489,0.09156760709220976,-0.04474273967465235,0.04070936090857963,0.16046091900862924,-0.028335890245733256,0.034871528426287074,-0.04576902420616219,0.007053743509268884,0.09386941654071462,0.04452765130300167,0.16318608109591703,-0.032833304312874644,-0.015243199120535778,-0.04790142809545422,0.10994052087680833,0.042147467705845126,0.026961174465468044,0.02254642159569981,-0.0759321983713304,-0.03481189380279137,0.13324239106585972,0.11623609963830212,-0.0472276713901843,0.05123815718070845,-0.0680724838672439,0.22511199745617416,0.12374932713032648,0.09033595533694411,0.11918598620419814,-0.13643055599304252,0.09720094663385238,-0.05046012807597048,0.07580761412927266,0.0993583979513917,-0.05683581440605167,-0.062487357918517006,-0.009021135022060098,-0.029875071843633052,-0.030830109973128322,-0.10165196220737958,0.0825123841687644,-0.0027989438116668894,-0.07107374618031712,-0.015273335324415988,-0.011477001042720017,-0.11766860892804282,0.08259616215186905,-0.06401228130912244,0.15037250642819652,-0.07578795885990713,-0.13658021460199368,-0.05514202798819469,-0.15647726765568065,-0.0023845967728761616,-0.09666669379002862,-0.10425454326974425,0.04724233682704434,0.1269507835816496,0.08166858083660285,-0.04907598062522172,-0.011262794433605412,-0.10934280590870285,0.10752981680234273,-0.13957983453604317,0.023566507212558856,-0.05671152298320114,0.023774795035375187,-0.13167629503411202,0.07512442751075103,-0.04121427403094613,0.06277656983057854,0.004543518596217104,0.06044101727265512,0.1422438344495671,0.016420064424940324,0.0033381073596776543,-0.24234655728023138,0.13059549006802115,0.14909079646099657,0.026050239464906796,0.037162226172543264,-0.1333773902839095,0.06814161961258128,-0.012248160731059166,0.04337568254756503,0.13442050521779486,0.10448293032484424,-0.0382833599978901,-0.05606856687261135,-0.07821880014389376,-0.11038697791565365,0.162862719579702,-0.052358864550370324,-0.03983658753220438,0.046918653399639826,-0.20032230822116387,-0.15227562805572584,0.10455854050313973,-0.19308561797349275,-0.1351833836049526,-0.04770892358562029,-0.08158580605717293,0.20600367453290558,0.1490809684138873,-0.011648677628165363,0.055659466216815195,-0.0016674676206011045,-0.1447953637144113,0.05051978035662302,0.03152227812718284,0.026544866756992653,-0.06568971221197095,-0.04109098224630695,-0.12706134825321733,-0.16936461274659775,0.0931568919235316,0.2356966162438123,0.16100233689783544,-0.047468357141855544,0.03465752020783921,0.06592392415819469,0.06874752726128501,0.03927253992718121,-0.012583019215118965,-0.15322345348044122,-0.19788733052280702,0.10597664066227946,0.05586775653925852,0.08997349312459904,-0.07521727035218978,-0.0728973703173678,0.058172895714934315,-0.03183401489387498,-0.03782428697477435,0.15586337971773612,-0.17439397917678431,-0.22499283548255453,-0.12680424521787412,-0.03191469665644928,-0.059489063106776735,0.0005176370372658732,0.05914427122528237,-0.09189408429030793,0.1060244322316331,-0.04332586685847601,-0.09224233678563501,0.10440857700717475,0.03719525483409568,-0.06654287729297802,-0.08198593251352013,0.002363967787441058,0.2118517578544168,-0.04498785096396881,-0.09729142359573209,-0.095911070073797,0.006192877809233909,0.09592762285869111,-0.10140519551810741,0.007197425086942265,-0.04784407962721418,-0.12023484580086988,-0.0626322823914419,0.049299870525285366,-0.1499874563563304,0.06035911713661988,0.029550708953764094,-0.04940377673588078,0.09207687890724953,0.18305591357676754,-0.0984386077352897,0.12085328678257419,0.06231218771243483,-0.07412429531916709,-0.1047422711556157,-0.09360384099835178,0.09241093409626018,-0.13183846137925034,-0.0972721483201835,0.08968790155448197,-0.15590741997653682,0.11818248502653801,0.1147959621010341,0.05955294914560261,-0.21546636157612886,0.054848261235656684,-0.09264568802332213,-0.008191004720525079,0.1648338446001526,-0.07532107009485864,-0.030505240539493623,-0.06545017758297193,-0.16865218824286965,0.20255259388760738,-0.057665460726581845,-0.11254758036064694,0.10243041700782643,-0.243397237975634,-0.11445035404111492,-0.10924934909361839,-0.00562712041357458,-0.17217161675496168,-0.006278701168404551,-0.05336695644107105,-0.07938475147363878,0.33309510550299826,0.043968203979775626,-0.025602263830126495,0.04058514998804157,0.05014793609943289,-0.012685247779452628,-0.012900770600027495,0.2717524499041873,0.04188253246349927,0.05890498540909638,-0.02744379138405556,0.06700560680635273,0.09690937018597491,0.12244911182981869,-0.07028758284169366,-0.06644128494843506,0.13976492880805755,-0.07837449500034811,-0.0877381145978071,-0.05744391264757614,0.05188097352912799,0.001855959638506978,0.02800981738050803,0.08115964850080082,-0.15606246816866395,-0.19211404826896397,0.006601489076866402,0.05456730582303986,0.09913572095150547,-0.09609592692018036,-0.03142460015021777,0.010046031290633556,-0.2295507511833769,-0.06374948899959614,-0.0724862028962266,0.20754544668250352,0.12754046860794419,0.1170428859782735,-0.1066940447280706,0.10295280197475681,0.028351816784711784,-0.012924787782112396,0.2099453090838468,0.05593094808171548,-0.23469151516820508,-0.16466508164106,-0.09670375908769191,-0.1231510758421144,-0.09596439945792823,0.04988981426395063,0.021916014191161706,-0.1957375440130406,0.09833774122421912,0.08483230072027081,0.0682112706370084,-0.038024884707414594,-0.04727707895995233,-0.06531707433744781,-0.14594365988905186,-0.034874820602021184,-0.1379854906866556,-0.011318689768268284,0.06771350752230382,0.13118145713399762,-0.13425560884271381,0.06757395179523433,-0.015570342520249759,-0.0942987340432553,-0.05065649847286044,-0.05228210238746951,-0.06643297667287897,0.012428537816613775,-0.03598850382017678,0.12747486125538174,0.002944716881140423,-0.08197168659817991,0.07779017958081508,0.04879402419577096,-0.11547317797073486,0.026739606738835325,0.036132263988933995,-0.1212731233956646,-0.05055816091070684,-0.060011096042585844,-0.05866295314472206,-0.21160438461209302,-0.19405258421000715,-0.08662631742640342,0.00015806514670594984,-0.22391354024671606,0.10762153381092539,0.0409515246268395,-0.04977078352009195,-0.117338203410967,-0.08469487602993139,0.022631021869743473,-0.06890266422673769,-0.021026280312454723,-0.1282331386098067,-0.06699810269227373,0.14194370942159973,-0.006631494184976331,-0.037646417281089924,0.05410717086669751,0.08269698897152511,-0.01309707185135621,0.09039227672591284,-0.21236464809358557,-0.056338101094111706,0.08830892138383754,0.10710453334280996,0.0691890257415017,-0.06594512219260516,0.07847004356588017,-0.1255945537422462,-0.10529268011861069,0.02396054040307971,-0.048776168233594576,0.032647504353052796,0.07498018793640919,0.14212764933637376,-0.06780353290189946,0.04390064143023965,-0.10206261109486772,0.005559866783096345,0.18979666060549116,-0.11423984555346632,-0.028774029605737602,0.12449203535945763,-0.07694833283004784],"pare_threshold":null,"picture":"heisenberg"},"bench_random.py::test_random_propagate[schrodinger]":{"basis":"majorana","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":400,"num_modes":128,"obs_terms":10000,"seed":0},"measurement_kind":"pytest","operation":"propagate","parameters":[-0.10523340542759485,-0.08094137974967036,-0.06211595669994035,-0.08826667666730753,0.01577990223114918,0.08842021633067204,0.008764144630662368,-0.023338043483467942,-0.10123826606905634,-0.08083056169051887,-0.024483151549459133,0.19999587419240034,0.034402450495715046,-0.060740944460963814,-0.028372883375546682,-0.08123197657142059,0.054598417011416184,0.13226565264832876,0.1793829185275495,0.24500502342044142,0.016533138965112933,0.06398693841926537,-0.14071297616875642,-0.03582049725701987,-0.00829798504984494,-0.01989172269036455,-0.0890516639083282,0.03499511691944299,0.07842812126606673,0.013411144350238583,-0.0065272353626342474,-0.009933884573162247,-0.07130892691243998,-0.05462419968503196,0.05305440937501408,-0.16304998019855138,-0.033019373414405986,-0.12993026546956915,-0.018704197153130025,-0.0027448315477394914,0.027526252158962795,0.022111909469319724,-0.06693585658874013,0.07698158394797196,0.026521370076793685,0.07532162570586155,0.1379370993805433,-0.041699627186328886,0.03985001364123631,-0.10179778154977051,-0.08737644929329814,-0.05380890530176533,0.06318706740090722,-0.025949884263762563,-0.06148101314198309,0.06112432511268341,-0.01818615038827541,-0.050237547334459044,-0.03495661871365907,-0.16188998902660967,-0.060077798881129574,0.10667620519047481,-0.06962111156981816,-0.028749781758810607,0.017852773723701875,0.0787870777524101,-0.026547227979639355,0.19643236750966347,-0.003860402650380958,-0.10640764381079792,0.03370477002953416,0.10033302304501035,0.14497668277100265,0.012699918516428953,-0.047500966560800245,0.11387542005001526,0.03681447397268443,0.13268427823457848,0.19802289486958669,-0.10648417868874525,-0.1180630037398055,-0.10690756918016098,0.09876089717466784,0.08878273104161043,0.10031715253481406,0.09025533919979599,0.005619418535165455,0.10224702372001543,0.07042981647935002,0.07380342507193559,0.052056087034874424,-0.15832075587354974,0.15652289255233173,0.04798536387357363,0.006583519693100686,0.05519485520358874,-0.005808178493392487,-0.054414491623919994,0.09800899380380651,0.0007580460863512288,0.09877494476788876,-0.08322143393177485,0.12968259865322757,0.040166145726451156,0.13851020153237129,-0.07318155902699117,-0.05069211096908072,-0.011431279988824736,0.12038565312947776,0.09675248394729692,-0.020676130255665923,0.19937295294057358,-0.012253605707711245,0.1200007769714421,0.06076993421691586,-0.022209177216871566,0.0912040054245478,0.012296926819538985,-0.09127973610635993,0.06224516291358034,0.0925927318604553,-0.10493314928240707,-0.07923218230027412,-0.0002653231059715322,0.08593637911813679,0.12279512921150058,0.10937099005675983,-0.1423342298397086,0.10891864781121346,-0.04936298164773581,-0.04076803756295305,-0.08461463900825827,-0.01487385496575952,0.013855597169472595,0.011218175863522018,-0.07593378235863349,-0.020229517013817783,-0.0895784410380972,0.19782826851567517,-0.13909730849863666,0.08166197031784965,0.04617142010280429,0.008061062732140833,0.17919363095734697,-0.019496165364105866,-0.06364340056169956,0.2697046947954294,0.07419704867967905,0.08362910014866365,0.003699039883658526,0.08332288327560508,-0.06618332782239543,0.03479098298890444,0.09497854317778413,0.06187092273695474,-0.07746605899470924,-0.034022007171120076,-0.09972229600906303,-0.04992464701789866,0.11686620622841158,0.10282349844994852,0.11351699248306656,-0.09705515823400813,0.009908627904673596,-0.013585942798689105,0.04560729045169355,0.09707847038453056,-0.051240501180525644,-0.04996795185614452,0.1589160668959215,0.07392249083884464,-0.12107626565880228,0.05774357445707268,0.014757976336802016,0.25447976752326096,0.11422572518800539,0.013756692700837207,-0.07944551829280541,-0.08137450516773338,-0.07435315046291195,-0.05112227603565842,-0.03447050874636209,-0.13283255090881205,-0.09102970224852679,0.021807637740561294,-0.06208065445112028,-0.025317125969109424,-0.17591851134924535,0.23210253315613555,-0.08263221230933254,-0.06278766068647927,0.013579267414203542,-0.06500932008157909,0.12772468695594472,0.04063049227732398,-0.015668820672189564,0.029157767113167407,-0.0031122754321551266,-0.00334216123900371,0.12233145897166023,0.09506269625144646,0.10651030534322053,-0.04644062433963333,0.23480578970528848,0.11228442443075295,0.07698332555813456,0.17353825498695397,0.08725987776930247,-0.07463448275969331,0.07932281405532177,0.06145851921299405,0.03858208749745545,0.025642759176322263,-0.0469425534980268,0.17301852055435166,-0.09886777574194905,-0.07237998166424277,0.13464779769958665,-0.027228060372084546,-0.019277986477240685,-0.17795469174076958,0.1563135977092122,-0.08117125255680013,-0.09264691927909507,0.004536304279842132,0.0008860638629985287,-0.058087652819835,-0.0321934813060711,0.0019115894402543339,0.03557678522322349,-0.05562435443125942,0.022950161549708067,0.08646233258553271,-0.004598738243267859,-0.12760796938145713,-0.020859819916942003,-0.060057571800486646,0.05155087450201268,0.03457543563770801,0.16963634146736892,-0.023567107988920907,-0.0030039741671125907,0.03224314682493049,-0.05752676344112634,-0.06339305531763027,0.1761036830591626,-0.1395425461883431,-0.04199834295282049,-0.03987373875708988,0.04608634718373789,0.016400549341834006,-0.0296677251450764,0.1412902320205028,-0.14979458449883776,-0.08367398259233477,0.05843496141832834,0.09527081078155063,-0.01579054361549966,0.11270514706811362,0.03791389093329849,0.11373737959082018,-0.1436781267771055,-0.10118661405422243,-0.18073307102587235,-0.11997324584501234,0.0712774307251998,-0.021892258291943578,-0.02211352018449934,-0.03432059348785439,-0.018205405708011418,0.16932120202964476,0.04201886129213857,-0.10694128002393022,0.010975197993096299,0.1367131741994031,3.594379665730761e-05,-0.06783325840196479,-0.06502185513518512,0.034499829341458,0.004346579470375118,-0.0006053965820215505,0.04908374708464368,-0.13300035168236565,-0.08111286360949103,0.020152463819115868,0.054041477626494305,-0.02833680984081713,-0.01112808282760254,0.023569276110329735,-0.0715612452501808,0.034619146102981085,-0.06979541076825363,-0.08745912215659571,0.011046034292088417,0.02075852068485466,-0.043554076268817556,-0.16227923026761154,-0.053769039778167396,0.01500159100577821,-0.20103107311268806,-0.1963648174348411,0.049973345111997185,0.08860716511737221,0.017733471981668695,0.08951126941576142,-0.018238109415638384,-0.08038822264777062,-0.00569259630111204,0.07869350519807561,0.12312589300679651,0.15960947784472165,0.029853290634047975,-0.06580599198831669,0.15752576869119472,-0.01717554653450146,0.10097717033172518,0.07747923624936776,0.15487636241563757,-0.027908553406802435,0.08147803388732411,0.048879130310388985,0.08671082257965242,0.2957506819507744,0.09237929717423665,0.10642615471394044,0.027636145911902693,0.026817434550932936,0.07774753888931099,-0.16201184672538546,-0.03591035453837232,-0.0325000453859792,0.009225014061417921,-0.020693579859150775,-0.13717644071875332,-0.18477911926081758,-0.12172581533535208,-0.1334271627989746,-0.05382058901560284,0.17453497857495073,0.03264957154524198,0.051958568801748664,-0.03928178482942598,0.0026983474625661115,0.07669124819921451,-0.11008024556842873,-0.14613206722566724,0.04501809955263127,0.02128746148974579,-0.015510620049398256,-0.11367425165870917,0.09241077102411667,-0.01317280182320589,0.02782068478330976,-0.07972281307977037,0.024825883235338162,0.1523649515383286,-0.04654508846642184,0.03422461190210632,-0.025703881921715008,0.025350875685893132,0.1325151209985144,0.12625095888057677,0.02101661712380698,0.001503206784237066,-0.08711235100328199,-0.07986454182214933,-0.008376178730586025,0.1268798072043726,-0.023055980671505796,-0.08741503853062876,-0.1711030547413609,0.03434690591861681,0.21191675927767695,-0.011521095778679186,-0.03490135006870497,0.11365171325062196,0.0374178462644883,0.008348809074067513,-0.08428525051868793,-0.030581935457419492,-0.09803521685896925,0.0615682116968893,-0.062306658769597914,-0.009119007161406515,0.06502960935173034,-0.0266140654116749,-0.07917971644027381,0.015143567180107544,-0.018971143173674473,-0.0572915949379865,0.1841040937517825,0.13328607969270875,0.10792713265702464,-0.04497510148666434,0.18621618974887613,0.005994159306986166,-0.009452355550903995,0.11187824678851102,-0.07145892316115895,-0.016729257872477366],"pare_threshold":null,"picture":"schrodinger"}},"reference-pared":{"bench_models.py::test_model_energy[hubbard]":{"basis":"majorana","config":{"chemical_potential":0.0,"cutoff":6,"hopping":1.0,"interaction":-2.0,"lower_atol":3e-05,"neel_start_spin":"down","num_sites":24,"observable_site":18,"observable_spin":"up","trotter_dt":0.2,"trotter_steps":2},"measurement_kind":"pytest","operation":"energy","parameters":[0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2],"pare_threshold":1e-10,"picture":"heisenberg"},"bench_models.py::test_model_energy[pauli]":{"basis":"pauli","config":{"coupling":0.7853981633974483,"cutoff":8,"lower_atol":3e-06,"num_layers":20,"num_qubits":127,"observable_qubit":62,"theta":0.7853981633974483},"measurement_kind":"pytest","operation":"energy","parameters":[-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483],"pare_threshold":1e-10,"picture":"heisenberg"},"bench_models.py::test_model_gradient[hubbard]":{"basis":"majorana","config":{"chemical_potential":0.0,"cutoff":6,"hopping":1.0,"interaction":-2.0,"lower_atol":3e-05,"neel_start_spin":"down","num_sites":24,"observable_site":18,"observable_spin":"up","trotter_dt":0.2,"trotter_steps":2},"measurement_kind":"pytest","operation":"gradient","parameters":[0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2],"pare_threshold":1e-10,"picture":"heisenberg"},"bench_models.py::test_model_gradient[pauli]":{"basis":"pauli","config":{"coupling":0.7853981633974483,"cutoff":8,"lower_atol":3e-06,"num_layers":20,"num_qubits":127,"observable_qubit":62,"theta":0.7853981633974483},"measurement_kind":"pytest","operation":"gradient","parameters":[-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483],"pare_threshold":1e-10,"picture":"heisenberg"},"bench_random.py::test_random_energy[heisenberg]":{"basis":"majorana","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":400,"num_modes":128,"obs_terms":400000,"seed":0},"measurement_kind":"pytest","operation":"energy","parameters":[-0.0890874666997349,-0.18546478544594966,0.11530378524586189,0.09205503938041594,-0.04563737995115475,0.001733519464636122,0.16766083911233587,0.015906451204065943,-0.035335020936261376,-0.15408243487908768,0.06829336807511036,-0.09875456154423223,-0.09420659079934764,-0.13527046390030678,0.00032446012231942357,-0.08055153496093498,-0.07026793455847383,-0.16058044224920034,0.07800112374861702,-0.08917937491740122,0.10113674111769128,0.04078484540120101,-0.04838800845045549,-0.032941978230042045,-0.13743969066960246,0.04795773804369145,-0.04817213480178123,0.0922502468741554,0.08891839657471055,0.015148926559246434,0.029529785131937925,0.13915625508827403,-0.05915121300113967,-0.1721588692016708,0.04653487982462562,-0.04067082394459168,-0.02781581485374118,-0.03212701319877228,-0.10071955733142322,0.11542594804946232,-0.12014960175007597,0.15314417370885783,0.058188270193482605,-0.09042159699295656,-0.004376778231361494,-0.04101134553535272,-0.025174800230267736,0.04430038493251029,-0.07489774417794269,0.0476372310871176,-0.17202251118913703,0.07555640731779886,-0.010663963085352179,0.10588469256042365,-0.078394163158033,-0.21668027931856537,0.1113358686772602,-0.048817185825212966,0.09872469767299243,-0.0792036091629742,0.13772312242390494,-0.1042634933215457,-0.04640909746036348,0.14239438928058076,-0.09862648505712415,-0.011384462634295489,0.09156760709220976,-0.04474273967465235,0.04070936090857963,0.16046091900862924,-0.028335890245733256,0.034871528426287074,-0.04576902420616219,0.007053743509268884,0.09386941654071462,0.04452765130300167,0.16318608109591703,-0.032833304312874644,-0.015243199120535778,-0.04790142809545422,0.10994052087680833,0.042147467705845126,0.026961174465468044,0.02254642159569981,-0.0759321983713304,-0.03481189380279137,0.13324239106585972,0.11623609963830212,-0.0472276713901843,0.05123815718070845,-0.0680724838672439,0.22511199745617416,0.12374932713032648,0.09033595533694411,0.11918598620419814,-0.13643055599304252,0.09720094663385238,-0.05046012807597048,0.07580761412927266,0.0993583979513917,-0.05683581440605167,-0.062487357918517006,-0.009021135022060098,-0.029875071843633052,-0.030830109973128322,-0.10165196220737958,0.0825123841687644,-0.0027989438116668894,-0.07107374618031712,-0.015273335324415988,-0.011477001042720017,-0.11766860892804282,0.08259616215186905,-0.06401228130912244,0.15037250642819652,-0.07578795885990713,-0.13658021460199368,-0.05514202798819469,-0.15647726765568065,-0.0023845967728761616,-0.09666669379002862,-0.10425454326974425,0.04724233682704434,0.1269507835816496,0.08166858083660285,-0.04907598062522172,-0.011262794433605412,-0.10934280590870285,0.10752981680234273,-0.13957983453604317,0.023566507212558856,-0.05671152298320114,0.023774795035375187,-0.13167629503411202,0.07512442751075103,-0.04121427403094613,0.06277656983057854,0.004543518596217104,0.06044101727265512,0.1422438344495671,0.016420064424940324,0.0033381073596776543,-0.24234655728023138,0.13059549006802115,0.14909079646099657,0.026050239464906796,0.037162226172543264,-0.1333773902839095,0.06814161961258128,-0.012248160731059166,0.04337568254756503,0.13442050521779486,0.10448293032484424,-0.0382833599978901,-0.05606856687261135,-0.07821880014389376,-0.11038697791565365,0.162862719579702,-0.052358864550370324,-0.03983658753220438,0.046918653399639826,-0.20032230822116387,-0.15227562805572584,0.10455854050313973,-0.19308561797349275,-0.1351833836049526,-0.04770892358562029,-0.08158580605717293,0.20600367453290558,0.1490809684138873,-0.011648677628165363,0.055659466216815195,-0.0016674676206011045,-0.1447953637144113,0.05051978035662302,0.03152227812718284,0.026544866756992653,-0.06568971221197095,-0.04109098224630695,-0.12706134825321733,-0.16936461274659775,0.0931568919235316,0.2356966162438123,0.16100233689783544,-0.047468357141855544,0.03465752020783921,0.06592392415819469,0.06874752726128501,0.03927253992718121,-0.012583019215118965,-0.15322345348044122,-0.19788733052280702,0.10597664066227946,0.05586775653925852,0.08997349312459904,-0.07521727035218978,-0.0728973703173678,0.058172895714934315,-0.03183401489387498,-0.03782428697477435,0.15586337971773612,-0.17439397917678431,-0.22499283548255453,-0.12680424521787412,-0.03191469665644928,-0.059489063106776735,0.0005176370372658732,0.05914427122528237,-0.09189408429030793,0.1060244322316331,-0.04332586685847601,-0.09224233678563501,0.10440857700717475,0.03719525483409568,-0.06654287729297802,-0.08198593251352013,0.002363967787441058,0.2118517578544168,-0.04498785096396881,-0.09729142359573209,-0.095911070073797,0.006192877809233909,0.09592762285869111,-0.10140519551810741,0.007197425086942265,-0.04784407962721418,-0.12023484580086988,-0.0626322823914419,0.049299870525285366,-0.1499874563563304,0.06035911713661988,0.029550708953764094,-0.04940377673588078,0.09207687890724953,0.18305591357676754,-0.0984386077352897,0.12085328678257419,0.06231218771243483,-0.07412429531916709,-0.1047422711556157,-0.09360384099835178,0.09241093409626018,-0.13183846137925034,-0.0972721483201835,0.08968790155448197,-0.15590741997653682,0.11818248502653801,0.1147959621010341,0.05955294914560261,-0.21546636157612886,0.054848261235656684,-0.09264568802332213,-0.008191004720525079,0.1648338446001526,-0.07532107009485864,-0.030505240539493623,-0.06545017758297193,-0.16865218824286965,0.20255259388760738,-0.057665460726581845,-0.11254758036064694,0.10243041700782643,-0.243397237975634,-0.11445035404111492,-0.10924934909361839,-0.00562712041357458,-0.17217161675496168,-0.006278701168404551,-0.05336695644107105,-0.07938475147363878,0.33309510550299826,0.043968203979775626,-0.025602263830126495,0.04058514998804157,0.05014793609943289,-0.012685247779452628,-0.012900770600027495,0.2717524499041873,0.04188253246349927,0.05890498540909638,-0.02744379138405556,0.06700560680635273,0.09690937018597491,0.12244911182981869,-0.07028758284169366,-0.06644128494843506,0.13976492880805755,-0.07837449500034811,-0.0877381145978071,-0.05744391264757614,0.05188097352912799,0.001855959638506978,0.02800981738050803,0.08115964850080082,-0.15606246816866395,-0.19211404826896397,0.006601489076866402,0.05456730582303986,0.09913572095150547,-0.09609592692018036,-0.03142460015021777,0.010046031290633556,-0.2295507511833769,-0.06374948899959614,-0.0724862028962266,0.20754544668250352,0.12754046860794419,0.1170428859782735,-0.1066940447280706,0.10295280197475681,0.028351816784711784,-0.012924787782112396,0.2099453090838468,0.05593094808171548,-0.23469151516820508,-0.16466508164106,-0.09670375908769191,-0.1231510758421144,-0.09596439945792823,0.04988981426395063,0.021916014191161706,-0.1957375440130406,0.09833774122421912,0.08483230072027081,0.0682112706370084,-0.038024884707414594,-0.04727707895995233,-0.06531707433744781,-0.14594365988905186,-0.034874820602021184,-0.1379854906866556,-0.011318689768268284,0.06771350752230382,0.13118145713399762,-0.13425560884271381,0.06757395179523433,-0.015570342520249759,-0.0942987340432553,-0.05065649847286044,-0.05228210238746951,-0.06643297667287897,0.012428537816613775,-0.03598850382017678,0.12747486125538174,0.002944716881140423,-0.08197168659817991,0.07779017958081508,0.04879402419577096,-0.11547317797073486,0.026739606738835325,0.036132263988933995,-0.1212731233956646,-0.05055816091070684,-0.060011096042585844,-0.05866295314472206,-0.21160438461209302,-0.19405258421000715,-0.08662631742640342,0.00015806514670594984,-0.22391354024671606,0.10762153381092539,0.0409515246268395,-0.04977078352009195,-0.117338203410967,-0.08469487602993139,0.022631021869743473,-0.06890266422673769,-0.021026280312454723,-0.1282331386098067,-0.06699810269227373,0.14194370942159973,-0.006631494184976331,-0.037646417281089924,0.05410717086669751,0.08269698897152511,-0.01309707185135621,0.09039227672591284,-0.21236464809358557,-0.056338101094111706,0.08830892138383754,0.10710453334280996,0.0691890257415017,-0.06594512219260516,0.07847004356588017,-0.1255945537422462,-0.10529268011861069,0.02396054040307971,-0.048776168233594576,0.032647504353052796,0.07498018793640919,0.14212764933637376,-0.06780353290189946,0.04390064143023965,-0.10206261109486772,0.005559866783096345,0.18979666060549116,-0.11423984555346632,-0.028774029605737602,0.12449203535945763,-0.07694833283004784],"pare_threshold":1e-10,"picture":"heisenberg"},"bench_random.py::test_random_energy[schrodinger]":{"basis":"majorana","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":400,"num_modes":128,"obs_terms":10000,"seed":0},"measurement_kind":"pytest","operation":"energy","parameters":[-0.10523340542759485,-0.08094137974967036,-0.06211595669994035,-0.08826667666730753,0.01577990223114918,0.08842021633067204,0.008764144630662368,-0.023338043483467942,-0.10123826606905634,-0.08083056169051887,-0.024483151549459133,0.19999587419240034,0.034402450495715046,-0.060740944460963814,-0.028372883375546682,-0.08123197657142059,0.054598417011416184,0.13226565264832876,0.1793829185275495,0.24500502342044142,0.016533138965112933,0.06398693841926537,-0.14071297616875642,-0.03582049725701987,-0.00829798504984494,-0.01989172269036455,-0.0890516639083282,0.03499511691944299,0.07842812126606673,0.013411144350238583,-0.0065272353626342474,-0.009933884573162247,-0.07130892691243998,-0.05462419968503196,0.05305440937501408,-0.16304998019855138,-0.033019373414405986,-0.12993026546956915,-0.018704197153130025,-0.0027448315477394914,0.027526252158962795,0.022111909469319724,-0.06693585658874013,0.07698158394797196,0.026521370076793685,0.07532162570586155,0.1379370993805433,-0.041699627186328886,0.03985001364123631,-0.10179778154977051,-0.08737644929329814,-0.05380890530176533,0.06318706740090722,-0.025949884263762563,-0.06148101314198309,0.06112432511268341,-0.01818615038827541,-0.050237547334459044,-0.03495661871365907,-0.16188998902660967,-0.060077798881129574,0.10667620519047481,-0.06962111156981816,-0.028749781758810607,0.017852773723701875,0.0787870777524101,-0.026547227979639355,0.19643236750966347,-0.003860402650380958,-0.10640764381079792,0.03370477002953416,0.10033302304501035,0.14497668277100265,0.012699918516428953,-0.047500966560800245,0.11387542005001526,0.03681447397268443,0.13268427823457848,0.19802289486958669,-0.10648417868874525,-0.1180630037398055,-0.10690756918016098,0.09876089717466784,0.08878273104161043,0.10031715253481406,0.09025533919979599,0.005619418535165455,0.10224702372001543,0.07042981647935002,0.07380342507193559,0.052056087034874424,-0.15832075587354974,0.15652289255233173,0.04798536387357363,0.006583519693100686,0.05519485520358874,-0.005808178493392487,-0.054414491623919994,0.09800899380380651,0.0007580460863512288,0.09877494476788876,-0.08322143393177485,0.12968259865322757,0.040166145726451156,0.13851020153237129,-0.07318155902699117,-0.05069211096908072,-0.011431279988824736,0.12038565312947776,0.09675248394729692,-0.020676130255665923,0.19937295294057358,-0.012253605707711245,0.1200007769714421,0.06076993421691586,-0.022209177216871566,0.0912040054245478,0.012296926819538985,-0.09127973610635993,0.06224516291358034,0.0925927318604553,-0.10493314928240707,-0.07923218230027412,-0.0002653231059715322,0.08593637911813679,0.12279512921150058,0.10937099005675983,-0.1423342298397086,0.10891864781121346,-0.04936298164773581,-0.04076803756295305,-0.08461463900825827,-0.01487385496575952,0.013855597169472595,0.011218175863522018,-0.07593378235863349,-0.020229517013817783,-0.0895784410380972,0.19782826851567517,-0.13909730849863666,0.08166197031784965,0.04617142010280429,0.008061062732140833,0.17919363095734697,-0.019496165364105866,-0.06364340056169956,0.2697046947954294,0.07419704867967905,0.08362910014866365,0.003699039883658526,0.08332288327560508,-0.06618332782239543,0.03479098298890444,0.09497854317778413,0.06187092273695474,-0.07746605899470924,-0.034022007171120076,-0.09972229600906303,-0.04992464701789866,0.11686620622841158,0.10282349844994852,0.11351699248306656,-0.09705515823400813,0.009908627904673596,-0.013585942798689105,0.04560729045169355,0.09707847038453056,-0.051240501180525644,-0.04996795185614452,0.1589160668959215,0.07392249083884464,-0.12107626565880228,0.05774357445707268,0.014757976336802016,0.25447976752326096,0.11422572518800539,0.013756692700837207,-0.07944551829280541,-0.08137450516773338,-0.07435315046291195,-0.05112227603565842,-0.03447050874636209,-0.13283255090881205,-0.09102970224852679,0.021807637740561294,-0.06208065445112028,-0.025317125969109424,-0.17591851134924535,0.23210253315613555,-0.08263221230933254,-0.06278766068647927,0.013579267414203542,-0.06500932008157909,0.12772468695594472,0.04063049227732398,-0.015668820672189564,0.029157767113167407,-0.0031122754321551266,-0.00334216123900371,0.12233145897166023,0.09506269625144646,0.10651030534322053,-0.04644062433963333,0.23480578970528848,0.11228442443075295,0.07698332555813456,0.17353825498695397,0.08725987776930247,-0.07463448275969331,0.07932281405532177,0.06145851921299405,0.03858208749745545,0.025642759176322263,-0.0469425534980268,0.17301852055435166,-0.09886777574194905,-0.07237998166424277,0.13464779769958665,-0.027228060372084546,-0.019277986477240685,-0.17795469174076958,0.1563135977092122,-0.08117125255680013,-0.09264691927909507,0.004536304279842132,0.0008860638629985287,-0.058087652819835,-0.0321934813060711,0.0019115894402543339,0.03557678522322349,-0.05562435443125942,0.022950161549708067,0.08646233258553271,-0.004598738243267859,-0.12760796938145713,-0.020859819916942003,-0.060057571800486646,0.05155087450201268,0.03457543563770801,0.16963634146736892,-0.023567107988920907,-0.0030039741671125907,0.03224314682493049,-0.05752676344112634,-0.06339305531763027,0.1761036830591626,-0.1395425461883431,-0.04199834295282049,-0.03987373875708988,0.04608634718373789,0.016400549341834006,-0.0296677251450764,0.1412902320205028,-0.14979458449883776,-0.08367398259233477,0.05843496141832834,0.09527081078155063,-0.01579054361549966,0.11270514706811362,0.03791389093329849,0.11373737959082018,-0.1436781267771055,-0.10118661405422243,-0.18073307102587235,-0.11997324584501234,0.0712774307251998,-0.021892258291943578,-0.02211352018449934,-0.03432059348785439,-0.018205405708011418,0.16932120202964476,0.04201886129213857,-0.10694128002393022,0.010975197993096299,0.1367131741994031,3.594379665730761e-05,-0.06783325840196479,-0.06502185513518512,0.034499829341458,0.004346579470375118,-0.0006053965820215505,0.04908374708464368,-0.13300035168236565,-0.08111286360949103,0.020152463819115868,0.054041477626494305,-0.02833680984081713,-0.01112808282760254,0.023569276110329735,-0.0715612452501808,0.034619146102981085,-0.06979541076825363,-0.08745912215659571,0.011046034292088417,0.02075852068485466,-0.043554076268817556,-0.16227923026761154,-0.053769039778167396,0.01500159100577821,-0.20103107311268806,-0.1963648174348411,0.049973345111997185,0.08860716511737221,0.017733471981668695,0.08951126941576142,-0.018238109415638384,-0.08038822264777062,-0.00569259630111204,0.07869350519807561,0.12312589300679651,0.15960947784472165,0.029853290634047975,-0.06580599198831669,0.15752576869119472,-0.01717554653450146,0.10097717033172518,0.07747923624936776,0.15487636241563757,-0.027908553406802435,0.08147803388732411,0.048879130310388985,0.08671082257965242,0.2957506819507744,0.09237929717423665,0.10642615471394044,0.027636145911902693,0.026817434550932936,0.07774753888931099,-0.16201184672538546,-0.03591035453837232,-0.0325000453859792,0.009225014061417921,-0.020693579859150775,-0.13717644071875332,-0.18477911926081758,-0.12172581533535208,-0.1334271627989746,-0.05382058901560284,0.17453497857495073,0.03264957154524198,0.051958568801748664,-0.03928178482942598,0.0026983474625661115,0.07669124819921451,-0.11008024556842873,-0.14613206722566724,0.04501809955263127,0.02128746148974579,-0.015510620049398256,-0.11367425165870917,0.09241077102411667,-0.01317280182320589,0.02782068478330976,-0.07972281307977037,0.024825883235338162,0.1523649515383286,-0.04654508846642184,0.03422461190210632,-0.025703881921715008,0.025350875685893132,0.1325151209985144,0.12625095888057677,0.02101661712380698,0.001503206784237066,-0.08711235100328199,-0.07986454182214933,-0.008376178730586025,0.1268798072043726,-0.023055980671505796,-0.08741503853062876,-0.1711030547413609,0.03434690591861681,0.21191675927767695,-0.011521095778679186,-0.03490135006870497,0.11365171325062196,0.0374178462644883,0.008348809074067513,-0.08428525051868793,-0.030581935457419492,-0.09803521685896925,0.0615682116968893,-0.062306658769597914,-0.009119007161406515,0.06502960935173034,-0.0266140654116749,-0.07917971644027381,0.015143567180107544,-0.018971143173674473,-0.0572915949379865,0.1841040937517825,0.13328607969270875,0.10792713265702464,-0.04497510148666434,0.18621618974887613,0.005994159306986166,-0.009452355550903995,0.11187824678851102,-0.07145892316115895,-0.016729257872477366],"pare_threshold":1e-10,"picture":"schrodinger"},"bench_random.py::test_random_gradient[heisenberg]":{"basis":"majorana","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":400,"num_modes":128,"obs_terms":400000,"seed":0},"measurement_kind":"pytest","operation":"gradient","parameters":[-0.0890874666997349,-0.18546478544594966,0.11530378524586189,0.09205503938041594,-0.04563737995115475,0.001733519464636122,0.16766083911233587,0.015906451204065943,-0.035335020936261376,-0.15408243487908768,0.06829336807511036,-0.09875456154423223,-0.09420659079934764,-0.13527046390030678,0.00032446012231942357,-0.08055153496093498,-0.07026793455847383,-0.16058044224920034,0.07800112374861702,-0.08917937491740122,0.10113674111769128,0.04078484540120101,-0.04838800845045549,-0.032941978230042045,-0.13743969066960246,0.04795773804369145,-0.04817213480178123,0.0922502468741554,0.08891839657471055,0.015148926559246434,0.029529785131937925,0.13915625508827403,-0.05915121300113967,-0.1721588692016708,0.04653487982462562,-0.04067082394459168,-0.02781581485374118,-0.03212701319877228,-0.10071955733142322,0.11542594804946232,-0.12014960175007597,0.15314417370885783,0.058188270193482605,-0.09042159699295656,-0.004376778231361494,-0.04101134553535272,-0.025174800230267736,0.04430038493251029,-0.07489774417794269,0.0476372310871176,-0.17202251118913703,0.07555640731779886,-0.010663963085352179,0.10588469256042365,-0.078394163158033,-0.21668027931856537,0.1113358686772602,-0.048817185825212966,0.09872469767299243,-0.0792036091629742,0.13772312242390494,-0.1042634933215457,-0.04640909746036348,0.14239438928058076,-0.09862648505712415,-0.011384462634295489,0.09156760709220976,-0.04474273967465235,0.04070936090857963,0.16046091900862924,-0.028335890245733256,0.034871528426287074,-0.04576902420616219,0.007053743509268884,0.09386941654071462,0.04452765130300167,0.16318608109591703,-0.032833304312874644,-0.015243199120535778,-0.04790142809545422,0.10994052087680833,0.042147467705845126,0.026961174465468044,0.02254642159569981,-0.0759321983713304,-0.03481189380279137,0.13324239106585972,0.11623609963830212,-0.0472276713901843,0.05123815718070845,-0.0680724838672439,0.22511199745617416,0.12374932713032648,0.09033595533694411,0.11918598620419814,-0.13643055599304252,0.09720094663385238,-0.05046012807597048,0.07580761412927266,0.0993583979513917,-0.05683581440605167,-0.062487357918517006,-0.009021135022060098,-0.029875071843633052,-0.030830109973128322,-0.10165196220737958,0.0825123841687644,-0.0027989438116668894,-0.07107374618031712,-0.015273335324415988,-0.011477001042720017,-0.11766860892804282,0.08259616215186905,-0.06401228130912244,0.15037250642819652,-0.07578795885990713,-0.13658021460199368,-0.05514202798819469,-0.15647726765568065,-0.0023845967728761616,-0.09666669379002862,-0.10425454326974425,0.04724233682704434,0.1269507835816496,0.08166858083660285,-0.04907598062522172,-0.011262794433605412,-0.10934280590870285,0.10752981680234273,-0.13957983453604317,0.023566507212558856,-0.05671152298320114,0.023774795035375187,-0.13167629503411202,0.07512442751075103,-0.04121427403094613,0.06277656983057854,0.004543518596217104,0.06044101727265512,0.1422438344495671,0.016420064424940324,0.0033381073596776543,-0.24234655728023138,0.13059549006802115,0.14909079646099657,0.026050239464906796,0.037162226172543264,-0.1333773902839095,0.06814161961258128,-0.012248160731059166,0.04337568254756503,0.13442050521779486,0.10448293032484424,-0.0382833599978901,-0.05606856687261135,-0.07821880014389376,-0.11038697791565365,0.162862719579702,-0.052358864550370324,-0.03983658753220438,0.046918653399639826,-0.20032230822116387,-0.15227562805572584,0.10455854050313973,-0.19308561797349275,-0.1351833836049526,-0.04770892358562029,-0.08158580605717293,0.20600367453290558,0.1490809684138873,-0.011648677628165363,0.055659466216815195,-0.0016674676206011045,-0.1447953637144113,0.05051978035662302,0.03152227812718284,0.026544866756992653,-0.06568971221197095,-0.04109098224630695,-0.12706134825321733,-0.16936461274659775,0.0931568919235316,0.2356966162438123,0.16100233689783544,-0.047468357141855544,0.03465752020783921,0.06592392415819469,0.06874752726128501,0.03927253992718121,-0.012583019215118965,-0.15322345348044122,-0.19788733052280702,0.10597664066227946,0.05586775653925852,0.08997349312459904,-0.07521727035218978,-0.0728973703173678,0.058172895714934315,-0.03183401489387498,-0.03782428697477435,0.15586337971773612,-0.17439397917678431,-0.22499283548255453,-0.12680424521787412,-0.03191469665644928,-0.059489063106776735,0.0005176370372658732,0.05914427122528237,-0.09189408429030793,0.1060244322316331,-0.04332586685847601,-0.09224233678563501,0.10440857700717475,0.03719525483409568,-0.06654287729297802,-0.08198593251352013,0.002363967787441058,0.2118517578544168,-0.04498785096396881,-0.09729142359573209,-0.095911070073797,0.006192877809233909,0.09592762285869111,-0.10140519551810741,0.007197425086942265,-0.04784407962721418,-0.12023484580086988,-0.0626322823914419,0.049299870525285366,-0.1499874563563304,0.06035911713661988,0.029550708953764094,-0.04940377673588078,0.09207687890724953,0.18305591357676754,-0.0984386077352897,0.12085328678257419,0.06231218771243483,-0.07412429531916709,-0.1047422711556157,-0.09360384099835178,0.09241093409626018,-0.13183846137925034,-0.0972721483201835,0.08968790155448197,-0.15590741997653682,0.11818248502653801,0.1147959621010341,0.05955294914560261,-0.21546636157612886,0.054848261235656684,-0.09264568802332213,-0.008191004720525079,0.1648338446001526,-0.07532107009485864,-0.030505240539493623,-0.06545017758297193,-0.16865218824286965,0.20255259388760738,-0.057665460726581845,-0.11254758036064694,0.10243041700782643,-0.243397237975634,-0.11445035404111492,-0.10924934909361839,-0.00562712041357458,-0.17217161675496168,-0.006278701168404551,-0.05336695644107105,-0.07938475147363878,0.33309510550299826,0.043968203979775626,-0.025602263830126495,0.04058514998804157,0.05014793609943289,-0.012685247779452628,-0.012900770600027495,0.2717524499041873,0.04188253246349927,0.05890498540909638,-0.02744379138405556,0.06700560680635273,0.09690937018597491,0.12244911182981869,-0.07028758284169366,-0.06644128494843506,0.13976492880805755,-0.07837449500034811,-0.0877381145978071,-0.05744391264757614,0.05188097352912799,0.001855959638506978,0.02800981738050803,0.08115964850080082,-0.15606246816866395,-0.19211404826896397,0.006601489076866402,0.05456730582303986,0.09913572095150547,-0.09609592692018036,-0.03142460015021777,0.010046031290633556,-0.2295507511833769,-0.06374948899959614,-0.0724862028962266,0.20754544668250352,0.12754046860794419,0.1170428859782735,-0.1066940447280706,0.10295280197475681,0.028351816784711784,-0.012924787782112396,0.2099453090838468,0.05593094808171548,-0.23469151516820508,-0.16466508164106,-0.09670375908769191,-0.1231510758421144,-0.09596439945792823,0.04988981426395063,0.021916014191161706,-0.1957375440130406,0.09833774122421912,0.08483230072027081,0.0682112706370084,-0.038024884707414594,-0.04727707895995233,-0.06531707433744781,-0.14594365988905186,-0.034874820602021184,-0.1379854906866556,-0.011318689768268284,0.06771350752230382,0.13118145713399762,-0.13425560884271381,0.06757395179523433,-0.015570342520249759,-0.0942987340432553,-0.05065649847286044,-0.05228210238746951,-0.06643297667287897,0.012428537816613775,-0.03598850382017678,0.12747486125538174,0.002944716881140423,-0.08197168659817991,0.07779017958081508,0.04879402419577096,-0.11547317797073486,0.026739606738835325,0.036132263988933995,-0.1212731233956646,-0.05055816091070684,-0.060011096042585844,-0.05866295314472206,-0.21160438461209302,-0.19405258421000715,-0.08662631742640342,0.00015806514670594984,-0.22391354024671606,0.10762153381092539,0.0409515246268395,-0.04977078352009195,-0.117338203410967,-0.08469487602993139,0.022631021869743473,-0.06890266422673769,-0.021026280312454723,-0.1282331386098067,-0.06699810269227373,0.14194370942159973,-0.006631494184976331,-0.037646417281089924,0.05410717086669751,0.08269698897152511,-0.01309707185135621,0.09039227672591284,-0.21236464809358557,-0.056338101094111706,0.08830892138383754,0.10710453334280996,0.0691890257415017,-0.06594512219260516,0.07847004356588017,-0.1255945537422462,-0.10529268011861069,0.02396054040307971,-0.048776168233594576,0.032647504353052796,0.07498018793640919,0.14212764933637376,-0.06780353290189946,0.04390064143023965,-0.10206261109486772,0.005559866783096345,0.18979666060549116,-0.11423984555346632,-0.028774029605737602,0.12449203535945763,-0.07694833283004784],"pare_threshold":1e-10,"picture":"heisenberg"},"bench_random.py::test_random_gradient[schrodinger]":{"basis":"majorana","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":400,"num_modes":128,"obs_terms":10000,"seed":0},"measurement_kind":"pytest","operation":"gradient","parameters":[-0.10523340542759485,-0.08094137974967036,-0.06211595669994035,-0.08826667666730753,0.01577990223114918,0.08842021633067204,0.008764144630662368,-0.023338043483467942,-0.10123826606905634,-0.08083056169051887,-0.024483151549459133,0.19999587419240034,0.034402450495715046,-0.060740944460963814,-0.028372883375546682,-0.08123197657142059,0.054598417011416184,0.13226565264832876,0.1793829185275495,0.24500502342044142,0.016533138965112933,0.06398693841926537,-0.14071297616875642,-0.03582049725701987,-0.00829798504984494,-0.01989172269036455,-0.0890516639083282,0.03499511691944299,0.07842812126606673,0.013411144350238583,-0.0065272353626342474,-0.009933884573162247,-0.07130892691243998,-0.05462419968503196,0.05305440937501408,-0.16304998019855138,-0.033019373414405986,-0.12993026546956915,-0.018704197153130025,-0.0027448315477394914,0.027526252158962795,0.022111909469319724,-0.06693585658874013,0.07698158394797196,0.026521370076793685,0.07532162570586155,0.1379370993805433,-0.041699627186328886,0.03985001364123631,-0.10179778154977051,-0.08737644929329814,-0.05380890530176533,0.06318706740090722,-0.025949884263762563,-0.06148101314198309,0.06112432511268341,-0.01818615038827541,-0.050237547334459044,-0.03495661871365907,-0.16188998902660967,-0.060077798881129574,0.10667620519047481,-0.06962111156981816,-0.028749781758810607,0.017852773723701875,0.0787870777524101,-0.026547227979639355,0.19643236750966347,-0.003860402650380958,-0.10640764381079792,0.03370477002953416,0.10033302304501035,0.14497668277100265,0.012699918516428953,-0.047500966560800245,0.11387542005001526,0.03681447397268443,0.13268427823457848,0.19802289486958669,-0.10648417868874525,-0.1180630037398055,-0.10690756918016098,0.09876089717466784,0.08878273104161043,0.10031715253481406,0.09025533919979599,0.005619418535165455,0.10224702372001543,0.07042981647935002,0.07380342507193559,0.052056087034874424,-0.15832075587354974,0.15652289255233173,0.04798536387357363,0.006583519693100686,0.05519485520358874,-0.005808178493392487,-0.054414491623919994,0.09800899380380651,0.0007580460863512288,0.09877494476788876,-0.08322143393177485,0.12968259865322757,0.040166145726451156,0.13851020153237129,-0.07318155902699117,-0.05069211096908072,-0.011431279988824736,0.12038565312947776,0.09675248394729692,-0.020676130255665923,0.19937295294057358,-0.012253605707711245,0.1200007769714421,0.06076993421691586,-0.022209177216871566,0.0912040054245478,0.012296926819538985,-0.09127973610635993,0.06224516291358034,0.0925927318604553,-0.10493314928240707,-0.07923218230027412,-0.0002653231059715322,0.08593637911813679,0.12279512921150058,0.10937099005675983,-0.1423342298397086,0.10891864781121346,-0.04936298164773581,-0.04076803756295305,-0.08461463900825827,-0.01487385496575952,0.013855597169472595,0.011218175863522018,-0.07593378235863349,-0.020229517013817783,-0.0895784410380972,0.19782826851567517,-0.13909730849863666,0.08166197031784965,0.04617142010280429,0.008061062732140833,0.17919363095734697,-0.019496165364105866,-0.06364340056169956,0.2697046947954294,0.07419704867967905,0.08362910014866365,0.003699039883658526,0.08332288327560508,-0.06618332782239543,0.03479098298890444,0.09497854317778413,0.06187092273695474,-0.07746605899470924,-0.034022007171120076,-0.09972229600906303,-0.04992464701789866,0.11686620622841158,0.10282349844994852,0.11351699248306656,-0.09705515823400813,0.009908627904673596,-0.013585942798689105,0.04560729045169355,0.09707847038453056,-0.051240501180525644,-0.04996795185614452,0.1589160668959215,0.07392249083884464,-0.12107626565880228,0.05774357445707268,0.014757976336802016,0.25447976752326096,0.11422572518800539,0.013756692700837207,-0.07944551829280541,-0.08137450516773338,-0.07435315046291195,-0.05112227603565842,-0.03447050874636209,-0.13283255090881205,-0.09102970224852679,0.021807637740561294,-0.06208065445112028,-0.025317125969109424,-0.17591851134924535,0.23210253315613555,-0.08263221230933254,-0.06278766068647927,0.013579267414203542,-0.06500932008157909,0.12772468695594472,0.04063049227732398,-0.015668820672189564,0.029157767113167407,-0.0031122754321551266,-0.00334216123900371,0.12233145897166023,0.09506269625144646,0.10651030534322053,-0.04644062433963333,0.23480578970528848,0.11228442443075295,0.07698332555813456,0.17353825498695397,0.08725987776930247,-0.07463448275969331,0.07932281405532177,0.06145851921299405,0.03858208749745545,0.025642759176322263,-0.0469425534980268,0.17301852055435166,-0.09886777574194905,-0.07237998166424277,0.13464779769958665,-0.027228060372084546,-0.019277986477240685,-0.17795469174076958,0.1563135977092122,-0.08117125255680013,-0.09264691927909507,0.004536304279842132,0.0008860638629985287,-0.058087652819835,-0.0321934813060711,0.0019115894402543339,0.03557678522322349,-0.05562435443125942,0.022950161549708067,0.08646233258553271,-0.004598738243267859,-0.12760796938145713,-0.020859819916942003,-0.060057571800486646,0.05155087450201268,0.03457543563770801,0.16963634146736892,-0.023567107988920907,-0.0030039741671125907,0.03224314682493049,-0.05752676344112634,-0.06339305531763027,0.1761036830591626,-0.1395425461883431,-0.04199834295282049,-0.03987373875708988,0.04608634718373789,0.016400549341834006,-0.0296677251450764,0.1412902320205028,-0.14979458449883776,-0.08367398259233477,0.05843496141832834,0.09527081078155063,-0.01579054361549966,0.11270514706811362,0.03791389093329849,0.11373737959082018,-0.1436781267771055,-0.10118661405422243,-0.18073307102587235,-0.11997324584501234,0.0712774307251998,-0.021892258291943578,-0.02211352018449934,-0.03432059348785439,-0.018205405708011418,0.16932120202964476,0.04201886129213857,-0.10694128002393022,0.010975197993096299,0.1367131741994031,3.594379665730761e-05,-0.06783325840196479,-0.06502185513518512,0.034499829341458,0.004346579470375118,-0.0006053965820215505,0.04908374708464368,-0.13300035168236565,-0.08111286360949103,0.020152463819115868,0.054041477626494305,-0.02833680984081713,-0.01112808282760254,0.023569276110329735,-0.0715612452501808,0.034619146102981085,-0.06979541076825363,-0.08745912215659571,0.011046034292088417,0.02075852068485466,-0.043554076268817556,-0.16227923026761154,-0.053769039778167396,0.01500159100577821,-0.20103107311268806,-0.1963648174348411,0.049973345111997185,0.08860716511737221,0.017733471981668695,0.08951126941576142,-0.018238109415638384,-0.08038822264777062,-0.00569259630111204,0.07869350519807561,0.12312589300679651,0.15960947784472165,0.029853290634047975,-0.06580599198831669,0.15752576869119472,-0.01717554653450146,0.10097717033172518,0.07747923624936776,0.15487636241563757,-0.027908553406802435,0.08147803388732411,0.048879130310388985,0.08671082257965242,0.2957506819507744,0.09237929717423665,0.10642615471394044,0.027636145911902693,0.026817434550932936,0.07774753888931099,-0.16201184672538546,-0.03591035453837232,-0.0325000453859792,0.009225014061417921,-0.020693579859150775,-0.13717644071875332,-0.18477911926081758,-0.12172581533535208,-0.1334271627989746,-0.05382058901560284,0.17453497857495073,0.03264957154524198,0.051958568801748664,-0.03928178482942598,0.0026983474625661115,0.07669124819921451,-0.11008024556842873,-0.14613206722566724,0.04501809955263127,0.02128746148974579,-0.015510620049398256,-0.11367425165870917,0.09241077102411667,-0.01317280182320589,0.02782068478330976,-0.07972281307977037,0.024825883235338162,0.1523649515383286,-0.04654508846642184,0.03422461190210632,-0.025703881921715008,0.025350875685893132,0.1325151209985144,0.12625095888057677,0.02101661712380698,0.001503206784237066,-0.08711235100328199,-0.07986454182214933,-0.008376178730586025,0.1268798072043726,-0.023055980671505796,-0.08741503853062876,-0.1711030547413609,0.03434690591861681,0.21191675927767695,-0.011521095778679186,-0.03490135006870497,0.11365171325062196,0.0374178462644883,0.008348809074067513,-0.08428525051868793,-0.030581935457419492,-0.09803521685896925,0.0615682116968893,-0.062306658769597914,-0.009119007161406515,0.06502960935173034,-0.0266140654116749,-0.07917971644027381,0.015143567180107544,-0.018971143173674473,-0.0572915949379865,0.1841040937517825,0.13328607969270875,0.10792713265702464,-0.04497510148666434,0.18621618974887613,0.005994159306986166,-0.009452355550903995,0.11187824678851102,-0.07145892316115895,-0.016729257872477366],"pare_threshold":1e-10,"picture":"schrodinger"},"driver::pare_functional_construct[heisenberg]":{"basis":"majorana","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":400,"num_modes":128,"obs_terms":400000,"seed":0},"functional":"expectation_value_and_gradient_functional","measurement_kind":"driver","operation":"pare_functional_construct","parameters":[-0.0890874666997349,-0.18546478544594966,0.11530378524586189,0.09205503938041594,-0.04563737995115475,0.001733519464636122,0.16766083911233587,0.015906451204065943,-0.035335020936261376,-0.15408243487908768,0.06829336807511036,-0.09875456154423223,-0.09420659079934764,-0.13527046390030678,0.00032446012231942357,-0.08055153496093498,-0.07026793455847383,-0.16058044224920034,0.07800112374861702,-0.08917937491740122,0.10113674111769128,0.04078484540120101,-0.04838800845045549,-0.032941978230042045,-0.13743969066960246,0.04795773804369145,-0.04817213480178123,0.0922502468741554,0.08891839657471055,0.015148926559246434,0.029529785131937925,0.13915625508827403,-0.05915121300113967,-0.1721588692016708,0.04653487982462562,-0.04067082394459168,-0.02781581485374118,-0.03212701319877228,-0.10071955733142322,0.11542594804946232,-0.12014960175007597,0.15314417370885783,0.058188270193482605,-0.09042159699295656,-0.004376778231361494,-0.04101134553535272,-0.025174800230267736,0.04430038493251029,-0.07489774417794269,0.0476372310871176,-0.17202251118913703,0.07555640731779886,-0.010663963085352179,0.10588469256042365,-0.078394163158033,-0.21668027931856537,0.1113358686772602,-0.048817185825212966,0.09872469767299243,-0.0792036091629742,0.13772312242390494,-0.1042634933215457,-0.04640909746036348,0.14239438928058076,-0.09862648505712415,-0.011384462634295489,0.09156760709220976,-0.04474273967465235,0.04070936090857963,0.16046091900862924,-0.028335890245733256,0.034871528426287074,-0.04576902420616219,0.007053743509268884,0.09386941654071462,0.04452765130300167,0.16318608109591703,-0.032833304312874644,-0.015243199120535778,-0.04790142809545422,0.10994052087680833,0.042147467705845126,0.026961174465468044,0.02254642159569981,-0.0759321983713304,-0.03481189380279137,0.13324239106585972,0.11623609963830212,-0.0472276713901843,0.05123815718070845,-0.0680724838672439,0.22511199745617416,0.12374932713032648,0.09033595533694411,0.11918598620419814,-0.13643055599304252,0.09720094663385238,-0.05046012807597048,0.07580761412927266,0.0993583979513917,-0.05683581440605167,-0.062487357918517006,-0.009021135022060098,-0.029875071843633052,-0.030830109973128322,-0.10165196220737958,0.0825123841687644,-0.0027989438116668894,-0.07107374618031712,-0.015273335324415988,-0.011477001042720017,-0.11766860892804282,0.08259616215186905,-0.06401228130912244,0.15037250642819652,-0.07578795885990713,-0.13658021460199368,-0.05514202798819469,-0.15647726765568065,-0.0023845967728761616,-0.09666669379002862,-0.10425454326974425,0.04724233682704434,0.1269507835816496,0.08166858083660285,-0.04907598062522172,-0.011262794433605412,-0.10934280590870285,0.10752981680234273,-0.13957983453604317,0.023566507212558856,-0.05671152298320114,0.023774795035375187,-0.13167629503411202,0.07512442751075103,-0.04121427403094613,0.06277656983057854,0.004543518596217104,0.06044101727265512,0.1422438344495671,0.016420064424940324,0.0033381073596776543,-0.24234655728023138,0.13059549006802115,0.14909079646099657,0.026050239464906796,0.037162226172543264,-0.1333773902839095,0.06814161961258128,-0.012248160731059166,0.04337568254756503,0.13442050521779486,0.10448293032484424,-0.0382833599978901,-0.05606856687261135,-0.07821880014389376,-0.11038697791565365,0.162862719579702,-0.052358864550370324,-0.03983658753220438,0.046918653399639826,-0.20032230822116387,-0.15227562805572584,0.10455854050313973,-0.19308561797349275,-0.1351833836049526,-0.04770892358562029,-0.08158580605717293,0.20600367453290558,0.1490809684138873,-0.011648677628165363,0.055659466216815195,-0.0016674676206011045,-0.1447953637144113,0.05051978035662302,0.03152227812718284,0.026544866756992653,-0.06568971221197095,-0.04109098224630695,-0.12706134825321733,-0.16936461274659775,0.0931568919235316,0.2356966162438123,0.16100233689783544,-0.047468357141855544,0.03465752020783921,0.06592392415819469,0.06874752726128501,0.03927253992718121,-0.012583019215118965,-0.15322345348044122,-0.19788733052280702,0.10597664066227946,0.05586775653925852,0.08997349312459904,-0.07521727035218978,-0.0728973703173678,0.058172895714934315,-0.03183401489387498,-0.03782428697477435,0.15586337971773612,-0.17439397917678431,-0.22499283548255453,-0.12680424521787412,-0.03191469665644928,-0.059489063106776735,0.0005176370372658732,0.05914427122528237,-0.09189408429030793,0.1060244322316331,-0.04332586685847601,-0.09224233678563501,0.10440857700717475,0.03719525483409568,-0.06654287729297802,-0.08198593251352013,0.002363967787441058,0.2118517578544168,-0.04498785096396881,-0.09729142359573209,-0.095911070073797,0.006192877809233909,0.09592762285869111,-0.10140519551810741,0.007197425086942265,-0.04784407962721418,-0.12023484580086988,-0.0626322823914419,0.049299870525285366,-0.1499874563563304,0.06035911713661988,0.029550708953764094,-0.04940377673588078,0.09207687890724953,0.18305591357676754,-0.0984386077352897,0.12085328678257419,0.06231218771243483,-0.07412429531916709,-0.1047422711556157,-0.09360384099835178,0.09241093409626018,-0.13183846137925034,-0.0972721483201835,0.08968790155448197,-0.15590741997653682,0.11818248502653801,0.1147959621010341,0.05955294914560261,-0.21546636157612886,0.054848261235656684,-0.09264568802332213,-0.008191004720525079,0.1648338446001526,-0.07532107009485864,-0.030505240539493623,-0.06545017758297193,-0.16865218824286965,0.20255259388760738,-0.057665460726581845,-0.11254758036064694,0.10243041700782643,-0.243397237975634,-0.11445035404111492,-0.10924934909361839,-0.00562712041357458,-0.17217161675496168,-0.006278701168404551,-0.05336695644107105,-0.07938475147363878,0.33309510550299826,0.043968203979775626,-0.025602263830126495,0.04058514998804157,0.05014793609943289,-0.012685247779452628,-0.012900770600027495,0.2717524499041873,0.04188253246349927,0.05890498540909638,-0.02744379138405556,0.06700560680635273,0.09690937018597491,0.12244911182981869,-0.07028758284169366,-0.06644128494843506,0.13976492880805755,-0.07837449500034811,-0.0877381145978071,-0.05744391264757614,0.05188097352912799,0.001855959638506978,0.02800981738050803,0.08115964850080082,-0.15606246816866395,-0.19211404826896397,0.006601489076866402,0.05456730582303986,0.09913572095150547,-0.09609592692018036,-0.03142460015021777,0.010046031290633556,-0.2295507511833769,-0.06374948899959614,-0.0724862028962266,0.20754544668250352,0.12754046860794419,0.1170428859782735,-0.1066940447280706,0.10295280197475681,0.028351816784711784,-0.012924787782112396,0.2099453090838468,0.05593094808171548,-0.23469151516820508,-0.16466508164106,-0.09670375908769191,-0.1231510758421144,-0.09596439945792823,0.04988981426395063,0.021916014191161706,-0.1957375440130406,0.09833774122421912,0.08483230072027081,0.0682112706370084,-0.038024884707414594,-0.04727707895995233,-0.06531707433744781,-0.14594365988905186,-0.034874820602021184,-0.1379854906866556,-0.011318689768268284,0.06771350752230382,0.13118145713399762,-0.13425560884271381,0.06757395179523433,-0.015570342520249759,-0.0942987340432553,-0.05065649847286044,-0.05228210238746951,-0.06643297667287897,0.012428537816613775,-0.03598850382017678,0.12747486125538174,0.002944716881140423,-0.08197168659817991,0.07779017958081508,0.04879402419577096,-0.11547317797073486,0.026739606738835325,0.036132263988933995,-0.1212731233956646,-0.05055816091070684,-0.060011096042585844,-0.05866295314472206,-0.21160438461209302,-0.19405258421000715,-0.08662631742640342,0.00015806514670594984,-0.22391354024671606,0.10762153381092539,0.0409515246268395,-0.04977078352009195,-0.117338203410967,-0.08469487602993139,0.022631021869743473,-0.06890266422673769,-0.021026280312454723,-0.1282331386098067,-0.06699810269227373,0.14194370942159973,-0.006631494184976331,-0.037646417281089924,0.05410717086669751,0.08269698897152511,-0.01309707185135621,0.09039227672591284,-0.21236464809358557,-0.056338101094111706,0.08830892138383754,0.10710453334280996,0.0691890257415017,-0.06594512219260516,0.07847004356588017,-0.1255945537422462,-0.10529268011861069,0.02396054040307971,-0.048776168233594576,0.032647504353052796,0.07498018793640919,0.14212764933637376,-0.06780353290189946,0.04390064143023965,-0.10206261109486772,0.005559866783096345,0.18979666060549116,-0.11423984555346632,-0.028774029605737602,0.12449203535945763,-0.07694833283004784],"pare_threshold":1e-10,"picture":"heisenberg"},"driver::pare_functional_construct[schrodinger]":{"basis":"majorana","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":400,"num_modes":128,"obs_terms":10000,"seed":0},"functional":"expectation_value_and_gradient_functional","measurement_kind":"driver","operation":"pare_functional_construct","parameters":[-0.10523340542759485,-0.08094137974967036,-0.06211595669994035,-0.08826667666730753,0.01577990223114918,0.08842021633067204,0.008764144630662368,-0.023338043483467942,-0.10123826606905634,-0.08083056169051887,-0.024483151549459133,0.19999587419240034,0.034402450495715046,-0.060740944460963814,-0.028372883375546682,-0.08123197657142059,0.054598417011416184,0.13226565264832876,0.1793829185275495,0.24500502342044142,0.016533138965112933,0.06398693841926537,-0.14071297616875642,-0.03582049725701987,-0.00829798504984494,-0.01989172269036455,-0.0890516639083282,0.03499511691944299,0.07842812126606673,0.013411144350238583,-0.0065272353626342474,-0.009933884573162247,-0.07130892691243998,-0.05462419968503196,0.05305440937501408,-0.16304998019855138,-0.033019373414405986,-0.12993026546956915,-0.018704197153130025,-0.0027448315477394914,0.027526252158962795,0.022111909469319724,-0.06693585658874013,0.07698158394797196,0.026521370076793685,0.07532162570586155,0.1379370993805433,-0.041699627186328886,0.03985001364123631,-0.10179778154977051,-0.08737644929329814,-0.05380890530176533,0.06318706740090722,-0.025949884263762563,-0.06148101314198309,0.06112432511268341,-0.01818615038827541,-0.050237547334459044,-0.03495661871365907,-0.16188998902660967,-0.060077798881129574,0.10667620519047481,-0.06962111156981816,-0.028749781758810607,0.017852773723701875,0.0787870777524101,-0.026547227979639355,0.19643236750966347,-0.003860402650380958,-0.10640764381079792,0.03370477002953416,0.10033302304501035,0.14497668277100265,0.012699918516428953,-0.047500966560800245,0.11387542005001526,0.03681447397268443,0.13268427823457848,0.19802289486958669,-0.10648417868874525,-0.1180630037398055,-0.10690756918016098,0.09876089717466784,0.08878273104161043,0.10031715253481406,0.09025533919979599,0.005619418535165455,0.10224702372001543,0.07042981647935002,0.07380342507193559,0.052056087034874424,-0.15832075587354974,0.15652289255233173,0.04798536387357363,0.006583519693100686,0.05519485520358874,-0.005808178493392487,-0.054414491623919994,0.09800899380380651,0.0007580460863512288,0.09877494476788876,-0.08322143393177485,0.12968259865322757,0.040166145726451156,0.13851020153237129,-0.07318155902699117,-0.05069211096908072,-0.011431279988824736,0.12038565312947776,0.09675248394729692,-0.020676130255665923,0.19937295294057358,-0.012253605707711245,0.1200007769714421,0.06076993421691586,-0.022209177216871566,0.0912040054245478,0.012296926819538985,-0.09127973610635993,0.06224516291358034,0.0925927318604553,-0.10493314928240707,-0.07923218230027412,-0.0002653231059715322,0.08593637911813679,0.12279512921150058,0.10937099005675983,-0.1423342298397086,0.10891864781121346,-0.04936298164773581,-0.04076803756295305,-0.08461463900825827,-0.01487385496575952,0.013855597169472595,0.011218175863522018,-0.07593378235863349,-0.020229517013817783,-0.0895784410380972,0.19782826851567517,-0.13909730849863666,0.08166197031784965,0.04617142010280429,0.008061062732140833,0.17919363095734697,-0.019496165364105866,-0.06364340056169956,0.2697046947954294,0.07419704867967905,0.08362910014866365,0.003699039883658526,0.08332288327560508,-0.06618332782239543,0.03479098298890444,0.09497854317778413,0.06187092273695474,-0.07746605899470924,-0.034022007171120076,-0.09972229600906303,-0.04992464701789866,0.11686620622841158,0.10282349844994852,0.11351699248306656,-0.09705515823400813,0.009908627904673596,-0.013585942798689105,0.04560729045169355,0.09707847038453056,-0.051240501180525644,-0.04996795185614452,0.1589160668959215,0.07392249083884464,-0.12107626565880228,0.05774357445707268,0.014757976336802016,0.25447976752326096,0.11422572518800539,0.013756692700837207,-0.07944551829280541,-0.08137450516773338,-0.07435315046291195,-0.05112227603565842,-0.03447050874636209,-0.13283255090881205,-0.09102970224852679,0.021807637740561294,-0.06208065445112028,-0.025317125969109424,-0.17591851134924535,0.23210253315613555,-0.08263221230933254,-0.06278766068647927,0.013579267414203542,-0.06500932008157909,0.12772468695594472,0.04063049227732398,-0.015668820672189564,0.029157767113167407,-0.0031122754321551266,-0.00334216123900371,0.12233145897166023,0.09506269625144646,0.10651030534322053,-0.04644062433963333,0.23480578970528848,0.11228442443075295,0.07698332555813456,0.17353825498695397,0.08725987776930247,-0.07463448275969331,0.07932281405532177,0.06145851921299405,0.03858208749745545,0.025642759176322263,-0.0469425534980268,0.17301852055435166,-0.09886777574194905,-0.07237998166424277,0.13464779769958665,-0.027228060372084546,-0.019277986477240685,-0.17795469174076958,0.1563135977092122,-0.08117125255680013,-0.09264691927909507,0.004536304279842132,0.0008860638629985287,-0.058087652819835,-0.0321934813060711,0.0019115894402543339,0.03557678522322349,-0.05562435443125942,0.022950161549708067,0.08646233258553271,-0.004598738243267859,-0.12760796938145713,-0.020859819916942003,-0.060057571800486646,0.05155087450201268,0.03457543563770801,0.16963634146736892,-0.023567107988920907,-0.0030039741671125907,0.03224314682493049,-0.05752676344112634,-0.06339305531763027,0.1761036830591626,-0.1395425461883431,-0.04199834295282049,-0.03987373875708988,0.04608634718373789,0.016400549341834006,-0.0296677251450764,0.1412902320205028,-0.14979458449883776,-0.08367398259233477,0.05843496141832834,0.09527081078155063,-0.01579054361549966,0.11270514706811362,0.03791389093329849,0.11373737959082018,-0.1436781267771055,-0.10118661405422243,-0.18073307102587235,-0.11997324584501234,0.0712774307251998,-0.021892258291943578,-0.02211352018449934,-0.03432059348785439,-0.018205405708011418,0.16932120202964476,0.04201886129213857,-0.10694128002393022,0.010975197993096299,0.1367131741994031,3.594379665730761e-05,-0.06783325840196479,-0.06502185513518512,0.034499829341458,0.004346579470375118,-0.0006053965820215505,0.04908374708464368,-0.13300035168236565,-0.08111286360949103,0.020152463819115868,0.054041477626494305,-0.02833680984081713,-0.01112808282760254,0.023569276110329735,-0.0715612452501808,0.034619146102981085,-0.06979541076825363,-0.08745912215659571,0.011046034292088417,0.02075852068485466,-0.043554076268817556,-0.16227923026761154,-0.053769039778167396,0.01500159100577821,-0.20103107311268806,-0.1963648174348411,0.049973345111997185,0.08860716511737221,0.017733471981668695,0.08951126941576142,-0.018238109415638384,-0.08038822264777062,-0.00569259630111204,0.07869350519807561,0.12312589300679651,0.15960947784472165,0.029853290634047975,-0.06580599198831669,0.15752576869119472,-0.01717554653450146,0.10097717033172518,0.07747923624936776,0.15487636241563757,-0.027908553406802435,0.08147803388732411,0.048879130310388985,0.08671082257965242,0.2957506819507744,0.09237929717423665,0.10642615471394044,0.027636145911902693,0.026817434550932936,0.07774753888931099,-0.16201184672538546,-0.03591035453837232,-0.0325000453859792,0.009225014061417921,-0.020693579859150775,-0.13717644071875332,-0.18477911926081758,-0.12172581533535208,-0.1334271627989746,-0.05382058901560284,0.17453497857495073,0.03264957154524198,0.051958568801748664,-0.03928178482942598,0.0026983474625661115,0.07669124819921451,-0.11008024556842873,-0.14613206722566724,0.04501809955263127,0.02128746148974579,-0.015510620049398256,-0.11367425165870917,0.09241077102411667,-0.01317280182320589,0.02782068478330976,-0.07972281307977037,0.024825883235338162,0.1523649515383286,-0.04654508846642184,0.03422461190210632,-0.025703881921715008,0.025350875685893132,0.1325151209985144,0.12625095888057677,0.02101661712380698,0.001503206784237066,-0.08711235100328199,-0.07986454182214933,-0.008376178730586025,0.1268798072043726,-0.023055980671505796,-0.08741503853062876,-0.1711030547413609,0.03434690591861681,0.21191675927767695,-0.011521095778679186,-0.03490135006870497,0.11365171325062196,0.0374178462644883,0.008348809074067513,-0.08428525051868793,-0.030581935457419492,-0.09803521685896925,0.0615682116968893,-0.062306658769597914,-0.009119007161406515,0.06502960935173034,-0.0266140654116749,-0.07917971644027381,0.015143567180107544,-0.018971143173674473,-0.0572915949379865,0.1841040937517825,0.13328607969270875,0.10792713265702464,-0.04497510148666434,0.18621618974887613,0.005994159306986166,-0.009452355550903995,0.11187824678851102,-0.07145892316115895,-0.016729257872477366],"pare_threshold":1e-10,"picture":"schrodinger"}},"tiny":{"bench_models.py::test_model_build_graph[hubbard]":{"basis":"majorana","config":{"chemical_potential":0.0,"cutoff":6,"hopping":1.0,"interaction":-2.0,"lower_atol":0.0001,"neel_start_spin":"down","num_sites":4,"observable_site":1,"observable_spin":"up","trotter_dt":0.2,"trotter_steps":2},"measurement_kind":"pytest","operation":"build_graph","parameters":[0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2],"pare_threshold":null,"picture":"heisenberg"},"bench_models.py::test_model_build_graph[pauli]":{"basis":"pauli","config":{"coupling":0.7853981633974483,"cutoff":2,"lower_atol":0.1,"num_layers":1,"num_qubits":127,"observable_qubit":62,"theta":0.7853981633974483},"measurement_kind":"pytest","operation":"build_graph","parameters":[-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483],"pare_threshold":null,"picture":"heisenberg"},"bench_models.py::test_model_energy[hubbard]":{"basis":"majorana","config":{"chemical_potential":0.0,"cutoff":6,"hopping":1.0,"interaction":-2.0,"lower_atol":0.0001,"neel_start_spin":"down","num_sites":4,"observable_site":1,"observable_spin":"up","trotter_dt":0.2,"trotter_steps":2},"measurement_kind":"pytest","operation":"energy","parameters":[0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2],"pare_threshold":null,"picture":"heisenberg"},"bench_models.py::test_model_energy[pauli]":{"basis":"pauli","config":{"coupling":0.7853981633974483,"cutoff":2,"lower_atol":0.1,"num_layers":1,"num_qubits":127,"observable_qubit":62,"theta":0.7853981633974483},"measurement_kind":"pytest","operation":"energy","parameters":[-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483],"pare_threshold":null,"picture":"heisenberg"},"bench_models.py::test_model_gradient[hubbard]":{"basis":"majorana","config":{"chemical_potential":0.0,"cutoff":6,"hopping":1.0,"interaction":-2.0,"lower_atol":0.0001,"neel_start_spin":"down","num_sites":4,"observable_site":1,"observable_spin":"up","trotter_dt":0.2,"trotter_steps":2},"measurement_kind":"pytest","operation":"gradient","parameters":[0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2],"pare_threshold":null,"picture":"heisenberg"},"bench_models.py::test_model_gradient[pauli]":{"basis":"pauli","config":{"coupling":0.7853981633974483,"cutoff":2,"lower_atol":0.1,"num_layers":1,"num_qubits":127,"observable_qubit":62,"theta":0.7853981633974483},"measurement_kind":"pytest","operation":"gradient","parameters":[-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483],"pare_threshold":null,"picture":"heisenberg"},"bench_models.py::test_model_propagate[hubbard]":{"basis":"majorana","config":{"chemical_potential":0.0,"cutoff":6,"hopping":1.0,"interaction":-2.0,"lower_atol":0.0001,"neel_start_spin":"down","num_sites":4,"observable_site":1,"observable_spin":"up","trotter_dt":0.2,"trotter_steps":2},"measurement_kind":"pytest","operation":"propagate","parameters":[0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2],"pare_threshold":null,"picture":"heisenberg"},"bench_models.py::test_model_propagate[pauli]":{"basis":"pauli","config":{"coupling":0.7853981633974483,"cutoff":2,"lower_atol":0.1,"num_layers":1,"num_qubits":127,"observable_qubit":62,"theta":0.7853981633974483},"measurement_kind":"pytest","operation":"propagate","parameters":[-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483],"pare_threshold":null,"picture":"heisenberg"},"bench_random.py::test_random_build_graph[heisenberg]":{"basis":"majorana","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":8,"num_modes":8,"obs_terms":16,"seed":0},"measurement_kind":"pytest","operation":"build_graph","parameters":[-0.07130680950592722,0.062101785354009856,-0.2250141173574592,0.03863695975663059,-0.05816408364095032,0.010927969747781388,-0.007570152622082312,0.020211439504395988],"pare_threshold":null,"picture":"heisenberg"},"bench_random.py::test_random_build_graph[schrodinger]":{"basis":"majorana","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":8,"num_modes":8,"obs_terms":16,"seed":0},"measurement_kind":"pytest","operation":"build_graph","parameters":[-0.07130680950592722,0.062101785354009856,-0.2250141173574592,0.03863695975663059,-0.05816408364095032,0.010927969747781388,-0.007570152622082312,0.020211439504395988],"pare_threshold":null,"picture":"schrodinger"},"bench_random.py::test_random_energy[heisenberg]":{"basis":"majorana","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":8,"num_modes":8,"obs_terms":16,"seed":0},"measurement_kind":"pytest","operation":"energy","parameters":[-0.07130680950592722,0.062101785354009856,-0.2250141173574592,0.03863695975663059,-0.05816408364095032,0.010927969747781388,-0.007570152622082312,0.020211439504395988],"pare_threshold":null,"picture":"heisenberg"},"bench_random.py::test_random_energy[schrodinger]":{"basis":"majorana","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":8,"num_modes":8,"obs_terms":16,"seed":0},"measurement_kind":"pytest","operation":"energy","parameters":[-0.07130680950592722,0.062101785354009856,-0.2250141173574592,0.03863695975663059,-0.05816408364095032,0.010927969747781388,-0.007570152622082312,0.020211439504395988],"pare_threshold":null,"picture":"schrodinger"},"bench_random.py::test_random_gradient[heisenberg]":{"basis":"majorana","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":8,"num_modes":8,"obs_terms":16,"seed":0},"measurement_kind":"pytest","operation":"gradient","parameters":[-0.07130680950592722,0.062101785354009856,-0.2250141173574592,0.03863695975663059,-0.05816408364095032,0.010927969747781388,-0.007570152622082312,0.020211439504395988],"pare_threshold":null,"picture":"heisenberg"},"bench_random.py::test_random_gradient[schrodinger]":{"basis":"majorana","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":8,"num_modes":8,"obs_terms":16,"seed":0},"measurement_kind":"pytest","operation":"gradient","parameters":[-0.07130680950592722,0.062101785354009856,-0.2250141173574592,0.03863695975663059,-0.05816408364095032,0.010927969747781388,-0.007570152622082312,0.020211439504395988],"pare_threshold":null,"picture":"schrodinger"},"bench_random.py::test_random_propagate[heisenberg]":{"basis":"majorana","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":8,"num_modes":8,"obs_terms":16,"seed":0},"measurement_kind":"pytest","operation":"propagate","parameters":[-0.07130680950592722,0.062101785354009856,-0.2250141173574592,0.03863695975663059,-0.05816408364095032,0.010927969747781388,-0.007570152622082312,0.020211439504395988],"pare_threshold":null,"picture":"heisenberg"},"bench_random.py::test_random_propagate[schrodinger]":{"basis":"majorana","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":8,"num_modes":8,"obs_terms":16,"seed":0},"measurement_kind":"pytest","operation":"propagate","parameters":[-0.07130680950592722,0.062101785354009856,-0.2250141173574592,0.03863695975663059,-0.05816408364095032,0.010927969747781388,-0.007570152622082312,0.020211439504395988],"pare_threshold":null,"picture":"schrodinger"}},"tiny-pared":{"bench_models.py::test_model_energy[hubbard]":{"basis":"majorana","config":{"chemical_potential":0.0,"cutoff":6,"hopping":1.0,"interaction":-2.0,"lower_atol":0.0001,"neel_start_spin":"down","num_sites":4,"observable_site":1,"observable_spin":"up","trotter_dt":0.2,"trotter_steps":2},"measurement_kind":"pytest","operation":"energy","parameters":[0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2],"pare_threshold":1e-10,"picture":"heisenberg"},"bench_models.py::test_model_energy[pauli]":{"basis":"pauli","config":{"coupling":0.7853981633974483,"cutoff":2,"lower_atol":0.1,"num_layers":1,"num_qubits":127,"observable_qubit":62,"theta":0.7853981633974483},"measurement_kind":"pytest","operation":"energy","parameters":[-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483],"pare_threshold":1e-10,"picture":"heisenberg"},"bench_models.py::test_model_gradient[hubbard]":{"basis":"majorana","config":{"chemical_potential":0.0,"cutoff":6,"hopping":1.0,"interaction":-2.0,"lower_atol":0.0001,"neel_start_spin":"down","num_sites":4,"observable_site":1,"observable_spin":"up","trotter_dt":0.2,"trotter_steps":2},"measurement_kind":"pytest","operation":"gradient","parameters":[0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2],"pare_threshold":1e-10,"picture":"heisenberg"},"bench_models.py::test_model_gradient[pauli]":{"basis":"pauli","config":{"coupling":0.7853981633974483,"cutoff":2,"lower_atol":0.1,"num_layers":1,"num_qubits":127,"observable_qubit":62,"theta":0.7853981633974483},"measurement_kind":"pytest","operation":"gradient","parameters":[-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.39269908169872414,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483,-0.7853981633974483],"pare_threshold":1e-10,"picture":"heisenberg"},"bench_random.py::test_random_energy[heisenberg]":{"basis":"majorana","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":8,"num_modes":8,"obs_terms":16,"seed":0},"measurement_kind":"pytest","operation":"energy","parameters":[-0.07130680950592722,0.062101785354009856,-0.2250141173574592,0.03863695975663059,-0.05816408364095032,0.010927969747781388,-0.007570152622082312,0.020211439504395988],"pare_threshold":1e-10,"picture":"heisenberg"},"bench_random.py::test_random_energy[schrodinger]":{"basis":"majorana","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":8,"num_modes":8,"obs_terms":16,"seed":0},"measurement_kind":"pytest","operation":"energy","parameters":[-0.07130680950592722,0.062101785354009856,-0.2250141173574592,0.03863695975663059,-0.05816408364095032,0.010927969747781388,-0.007570152622082312,0.020211439504395988],"pare_threshold":1e-10,"picture":"schrodinger"},"bench_random.py::test_random_gradient[heisenberg]":{"basis":"majorana","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":8,"num_modes":8,"obs_terms":16,"seed":0},"measurement_kind":"pytest","operation":"gradient","parameters":[-0.07130680950592722,0.062101785354009856,-0.2250141173574592,0.03863695975663059,-0.05816408364095032,0.010927969747781388,-0.007570152622082312,0.020211439504395988],"pare_threshold":1e-10,"picture":"heisenberg"},"bench_random.py::test_random_gradient[schrodinger]":{"basis":"majorana","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":8,"num_modes":8,"obs_terms":16,"seed":0},"measurement_kind":"pytest","operation":"gradient","parameters":[-0.07130680950592722,0.062101785354009856,-0.2250141173574592,0.03863695975663059,-0.05816408364095032,0.010927969747781388,-0.007570152622082312,0.020211439504395988],"pare_threshold":1e-10,"picture":"schrodinger"},"driver::pare_functional_construct[heisenberg]":{"basis":"majorana","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":8,"num_modes":8,"obs_terms":16,"seed":0},"functional":"expectation_value_and_gradient_functional","measurement_kind":"driver","operation":"pare_functional_construct","parameters":[-0.07130680950592722,0.062101785354009856,-0.2250141173574592,0.03863695975663059,-0.05816408364095032,0.010927969747781388,-0.007570152622082312,0.020211439504395988],"pare_threshold":1e-10,"picture":"heisenberg"},"driver::pare_functional_construct[schrodinger]":{"basis":"majorana","config":{"cutoff":6,"gen_length":4,"lower_atol":null,"num_generators":8,"num_modes":8,"obs_terms":16,"seed":0},"functional":"expectation_value_and_gradient_functional","measurement_kind":"driver","operation":"pare_functional_construct","parameters":[-0.07130680950592722,0.062101785354009856,-0.2250141173574592,0.03863695975663059,-0.05816408364095032,0.010927969747781388,-0.007570152622082312,0.020211439504395988],"pare_threshold":1e-10,"picture":"schrodinger"}}},"schema_version":1,"status":"frozen: owner-approved 2026-09-25 (Task 1 campaign freeze)"} \ No newline at end of file From af430daa8afbd3434812c8dd1b09ab6e5d4eb633 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Roberto=20Di=20Remigio=20Eik=C3=A5s?= Date: Sun, 27 Sep 2026 11:19:28 +0000 Subject: [PATCH 09/64] =?UTF-8?q?docs(parallelism):=20=F0=9F=93=9D=20recor?= =?UTF-8?q?d=20where=20Task=200=20and=20Task=201=20artifacts=20live?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Name the host directories that hold the uncommitted Task 0 results and the Task 1 handoff report, formal baseline evidence, baseline index, collection scripts and pilot, so Task 11 can locate the baseline half of its comparison. The artifacts stay uncommitted and will be removed before merging. Assisted-by: Pi:claude-opus-5-5 --- .../plans/2026-09-18-rank-local-openmp.md | 19 +++++++++++++++++-- 1 file changed, 17 insertions(+), 2 deletions(-) diff --git a/docs/superpowers/plans/2026-09-18-rank-local-openmp.md b/docs/superpowers/plans/2026-09-18-rank-local-openmp.md index cb25648b..05f12837 100644 --- a/docs/superpowers/plans/2026-09-18-rank-local-openmp.md +++ b/docs/superpowers/plans/2026-09-18-rank-local-openmp.md @@ -632,8 +632,10 @@ Evidence (measured on the target host, not acceptance samples): - In-library Boost trial: +15–18% end-to-end on one core for Hubbard/Pauli at L1 size. Parallel publication gave no gain at per-gate batch sizes; the scan was still serial, so there is no full-machine result. -The results, overlays and budget ledger are archived outside the repository with the Task 0 report. The owner waived -the Boost 1.85 compile check, as stated above. +The results, overlays and budget ledger are archived outside the repository with the Task 0 report, in +`/home/ubuntu/task0-artifacts/` on the c8a.metal-24xl host (`report.md`, `ledger.md`, `budget.jsonl`). They are +intentionally uncommitted and will be removed before merging. The owner waived the Boost 1.85 compile check, as stated +above. ### Task 1: Freeze baselines and repair measurement trustworthiness @@ -1126,6 +1128,19 @@ configurations in artifacts, not just a profile label. `tools/rank-local-openmp-workloads.json` (sha256 `508b65f1f0eb2171fc4fbd79feba2381054c6dcdd38a614600f5b957ab814416`). Neither file may be regenerated or shrunk to make observations pass. The unscored pilot and its size decisions are archived outside the repository with the Task 1 handoff report; no pilot output is a formal sample. +- Artifact locations: all Task 1 artifacts are in `/home/ubuntu/task1-artifacts/` on the c8a.metal-24xl host. + They are intentionally uncommitted and will be removed before merging. `ledger.md` indexes them and records every + hash. + - `report.md`: the handoff report. `preflight/`, `provenance/` (overlay patches and hashes) and `findings/` + (VmHWM shortfall, gzip measurement) hold its supporting evidence. + - `formal/results/baseline//`: the formal baseline evidence (s01..s05 timed and construction artifacts plus + one validation per cell, about 20 GB). `formal/placements/` and `formal/diagnostics/` hold the per-cell placement + files and their raw evidence. + - `formal/baseline-index.json` (sha256 `82479d6ff83deb3113306ac1caa1720d25f15a400cbc89b5693f85ffad47fff5`): the + baseline half of the Task 11 comparison manifest. + - `formal/formal.py`, `formal/verify.py`, `formal/consistency.py` and `pilot/pilot.py`: the operator-side code + that collected and checked the evidence, kept as run rather than as reusable tooling. + - `pilot/` and `proposal/`: the unscored pilot and the inventory proposal. - The observed host has one NUMA domain (D=1, C=C_dom=96). Within each build mode, L1 and the MPI-only control coincide at 1x1, and the NUMA-local control, L2b and L2a coincide at 1x96. There is therefore no separate L2a diagnostic inventory. MPI-off 1x96 cells stay separate and required. The single-thread control runs in the MPI From ec59ad9cd479eb9cc4c43995e7548f98f7ec30be Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Roberto=20Di=20Remigio=20Eik=C3=A5s?= Date: Sun, 27 Sep 2026 18:02:16 +0000 Subject: [PATCH 10/64] =?UTF-8?q?feat(parallel):=20=E2=9C=A8=20add=20the?= =?UTF-8?q?=20OpenMP=20block=20worksharing=20primitive=20and=20require=20O?= =?UTF-8?q?penMP?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Task 2 of the rank-local OpenMP plan: the smallest worksharing helper and the mandatory OpenMP dependency across build routes. No engine code uses the helper yet. - detail/parallel/Options.h holds the per-region thread budget and does not include omp.h. detail/parallel/Workshare.h::for_blocks runs each logical block once: - it rejects non-positive budgets and counts above PTRDIFF_MAX before any work; - it runs serially in ascending order for a budget or count of one, or inside any OpenMP region; - otherwise it opens one static-schedule region requesting min(threads, count) workers; - worker exceptions stay in per-worker slots and are rethrown on the caller after the join; - it changes no OpenMP runtime setting. - CMake requires OpenMP CXX in every configuration, links OpenMP::OpenMP_CXX PUBLIC on both monoprop-objs and the shared monoprop target (which is built from TARGET_OBJECTS), and the package config calls find_dependency(OpenMP). - openmp_workshare_tests.cpp covers: - exactly-once visits for counts 0/1/19 and budgets 1/2/3, and more workers than blocks; - serial and nested fallbacks, and budget and bound validation; - unchanged runtime settings and joined worker exceptions; - observed multi-worker participation, skipped when the runtime cannot give two workers. - The shared link-export probe, which just test-find-package also builds, instantiates for_blocks, requires the imported target's OpenMP flags and drives the Pauli basis as well as Majorana. - Dependency routes: - Homebrew libomp with OpenMP_ROOT, in the setup action, the macOS wheel config and the docs; - a matching libomp--dev for Linux Clang lanes, and libomp-dev in the devcontainer; - llvmPackages.openmp for Clang-based Nix builds, monoprop-mpi in the flake checks, and a dev-shell consumer step in nix.yml; - clang-tidy's setup now receives libomp. Hwloc stays until its removal task. Assisted-by: Pi:claude-opus-5-5 --- .devcontainer/Dockerfile | 1 + .github/actions/setup/action.yml | 8 + .github/workflows/nix.yml | 17 +- .github/workflows/qa-analysis.yml | 3 + AGENTS.md | 3 + CMakeLists.txt | 4 + README.md | 6 +- cmake/monopropConfig.cmake.in | 7 + cpp/monoprop/CMakeLists.txt | 4 + cpp/monoprop/detail/CMakeLists.txt | 1 + cpp/monoprop/detail/parallel/CMakeLists.txt | 9 + cpp/monoprop/detail/parallel/Options.h | 31 ++ cpp/monoprop/detail/parallel/Workshare.h | 99 ++++++ cpp/tests/README.md | 6 + .../link_export_probe/link_export_probe.cpp | 55 +++ cpp/tests/openmp_workshare_tests.cpp | 324 ++++++++++++++++++ docs/content/docs/building.mdx | 20 +- flake.nix | 3 +- nix/devshell.nix | 7 +- nix/monoprop.nix | 5 + pyproject.toml | 5 +- tools/install-deps.sh | 5 + tools/packages/apt.txt | 4 + tools/packages/brew.txt | 1 + 24 files changed, 617 insertions(+), 11 deletions(-) create mode 100644 cpp/monoprop/detail/parallel/CMakeLists.txt create mode 100644 cpp/monoprop/detail/parallel/Options.h create mode 100644 cpp/monoprop/detail/parallel/Workshare.h create mode 100644 cpp/tests/openmp_workshare_tests.cpp diff --git a/.devcontainer/Dockerfile b/.devcontainer/Dockerfile index 8208d73f..8b6e56dc 100644 --- a/.devcontainer/Dockerfile +++ b/.devcontainer/Dockerfile @@ -9,6 +9,7 @@ RUN apt-get update && export DEBIAN_FRONTEND=noninteractive \ | grep -v '^$' \ | xargs apt-get -y install \ clang \ + libomp-dev \ clang-tidy \ clang-format \ cmake \ diff --git a/.github/actions/setup/action.yml b/.github/actions/setup/action.yml index 54a4f2b0..5dd25f9f 100644 --- a/.github/actions/setup/action.yml +++ b/.github/actions/setup/action.yml @@ -69,8 +69,16 @@ runs: esac fi + # OpenMP is required. g++ ships libgomp; a Linux Clang needs the libomp of its own + # version, even when the runner already provides the compiler. + if [[ "$RUNNER_OS" != "macOS" && "${CXX:-}" == clang++* ]]; then + packages+=("libomp${CXX#clang++}-dev") + fi + if [[ "$RUNNER_OS" == "macOS" ]]; then brew install "${packages[@]}" + # Homebrew's libomp is keg-only; CMake's FindOpenMP searches OpenMP_ROOT. + echo "OpenMP_ROOT=$(brew --prefix libomp)" >> "$GITHUB_ENV" else sudo apt-get update sudo apt-get install -y "${packages[@]}" diff --git a/.github/workflows/nix.yml b/.github/workflows/nix.yml index 615daef1..89bfcfd4 100644 --- a/.github/workflows/nix.yml +++ b/.github/workflows/nix.yml @@ -16,6 +16,10 @@ on: - VERSION - pyproject.toml - CMakeLists.txt + - cmake/** + - cpp/**/CMakeLists.txt + - cpp/tests/find_package_smoke/** + - cpp/tests/link_export_probe/** - src/monoprop/bindings/CMakeLists.txt concurrency: @@ -41,8 +45,8 @@ jobs: - name: Install Nix uses: cachix/install-nix-action@v31.11.1 - # Builds `checks.monoprop`, which is the non-MPI package, so the build below - # reuses this store path rather than compiling twice. + # Builds `checks.monoprop` and `checks.monoprop-mpi`, so the build below reuses + # both store paths rather than compiling twice. - name: Check flake run: nix flake check --print-build-logs @@ -66,6 +70,15 @@ jobs: mpicc --version ' + # Configures, links and runs the installed-package consumer, which instantiates the + # OpenMP worksharing template, with the shell's compiler and OpenMP runtime. + - name: Check the OpenMP consumer from the dev shell + run: | + nix develop --command bash -euo pipefail -c ' + monoprop_ENABLE_MPI=OFF just build --group workspace-test + just test-find-package + ' + flake-result: name: All flake checks if: always() diff --git a/.github/workflows/qa-analysis.yml b/.github/workflows/qa-analysis.yml index 78e08c63..4d5e7c22 100644 --- a/.github/workflows/qa-analysis.yml +++ b/.github/workflows/qa-analysis.yml @@ -239,6 +239,9 @@ jobs: - name: Set up the build environment uses: ./.github/actions/setup + # The build below uses clang++, so the setup installs its matching libomp. + env: + CXX: clang++ with: cache-suffix: clang-tidy-mpi mpi: "on" diff --git a/AGENTS.md b/AGENTS.md index 679abdec..8f438960 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -62,6 +62,9 @@ just build-docs never imported when `monoprop.has_mpi` is false. - `uv sync` does not relink `bin/monoprop_unit_tests.x` -- compare mtimes and use the standalone recipe in `docs/content/docs/building.mdx`. +- OpenMP is a required build dependency in every configuration (`OpenMP::OpenMP_CXX`, linked PUBLIC on + both `monoprop-objs` and `monoprop`, `find_dependency` in the package config). The worksharing + primitive is `detail/parallel/Workshare.h::for_blocks`. Never call `omp_set_*` from the library. - Slow CTest startup in MPI builds is `MPI_Init` fabric probing; see `monoprop_TEST_EXCLUDE_MPI_FABRIC` in `cpp/tests/CMakeLists.txt`. - Sanitizer rebuild and test: diff --git a/CMakeLists.txt b/CMakeLists.txt index 1442722c..03c3403e 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -65,6 +65,10 @@ find_package( find_package(Threads REQUIRED) +# Required in every configuration, including MPI-off and single-thread builds: there is no +# OpenMP-disabled variant of the library. +find_package(OpenMP REQUIRED COMPONENTS CXX) + if(monoprop_ENABLE_MPI) find_package(MPI REQUIRED COMPONENTS CXX) endif() diff --git a/README.md b/README.md index 2b7914aa..acd11b24 100644 --- a/README.md +++ b/README.md @@ -113,7 +113,9 @@ GitHub Actions calls, so a lane can be reproduced locally. Full instructions — prerequisites, MPI options, and running the example executable — are in the [building guide](https://docs.monoprop.algorithmiq.tech/building). In particular, from-source builds require `hwloc` and `pkg-config` so CMake can -locate `hwloc`. +locate `hwloc`, and OpenMP for C++ in every configuration, including builds without +MPI. GCC ships its OpenMP runtime; Clang needs the matching `libomp`, and Apple Clang +needs Homebrew's `libomp` with `OpenMP_ROOT="$(brew --prefix libomp)"` set. ## Running the tests @@ -179,7 +181,7 @@ The repository is a [Nix flake](https://wiki.nixos.org/wiki/Flakes), so on Nix o NixOS none of the prerequisites have to be installed by hand: ```bash -nix develop # dev shell: C++ toolchain, hwloc, MPI, uv, just, node +nix develop # dev shell: C++ toolchain, OpenMP, hwloc, MPI, uv, just, node nix build .#monoprop # build the package (`.#monoprop-mpi` for the MPI build) nix run # Python interpreter with monoprop importable ``` diff --git a/cmake/monopropConfig.cmake.in b/cmake/monopropConfig.cmake.in index 89296ac2..a4af9c79 100644 --- a/cmake/monopropConfig.cmake.in +++ b/cmake/monopropConfig.cmake.in @@ -31,6 +31,13 @@ find_dependency( Threads REQUIRED ) +# Always required: the exported target links OpenMP::OpenMP_CXX in every configuration. +find_dependency( + OpenMP + REQUIRED + COMPONENTS + CXX +) set(monoprop_ENABLE_MPI "@monoprop_ENABLE_MPI@") if(monoprop_ENABLE_MPI) diff --git a/cpp/monoprop/CMakeLists.txt b/cpp/monoprop/CMakeLists.txt index e2bb624c..93f27e07 100644 --- a/cpp/monoprop/CMakeLists.txt +++ b/cpp/monoprop/CMakeLists.txt @@ -43,12 +43,15 @@ target_include_directories( $ ) +# OpenMP is linked on both targets: the shared library is assembled from TARGET_OBJECTS and does not +# inherit monoprop-objs' link requirements, so it needs its own link against the runtime. target_link_libraries( monoprop-objs PUBLIC monoprop-sanitizers Boost::boost Threads::Threads + OpenMP::OpenMP_CXX $<$:MPI::MPI_CXX> PRIVATE PkgConfig::HWLOC @@ -105,6 +108,7 @@ target_compile_features(monoprop INTERFACE cxx_std_23) target_link_libraries( monoprop PUBLIC + OpenMP::OpenMP_CXX $<$:MPI::MPI_CXX> INTERFACE Boost::boost diff --git a/cpp/monoprop/detail/CMakeLists.txt b/cpp/monoprop/detail/CMakeLists.txt index f75f51c5..9747a7da 100644 --- a/cpp/monoprop/detail/CMakeLists.txt +++ b/cpp/monoprop/detail/CMakeLists.txt @@ -13,5 +13,6 @@ add_subdirectory(graph_encoding) add_subdirectory(monomial_propagator) add_subdirectory(mpi) add_subdirectory(operator) +add_subdirectory(parallel) add_subdirectory(pare) add_subdirectory(partition) diff --git a/cpp/monoprop/detail/parallel/CMakeLists.txt b/cpp/monoprop/detail/parallel/CMakeLists.txt new file mode 100644 index 00000000..2237ed3d --- /dev/null +++ b/cpp/monoprop/detail/parallel/CMakeLists.txt @@ -0,0 +1,9 @@ +target_sources( + monoprop + PUBLIC + FILE_SET headers + TYPE HEADERS + FILES + "Options.h" + "Workshare.h" +) diff --git a/cpp/monoprop/detail/parallel/Options.h b/cpp/monoprop/detail/parallel/Options.h new file mode 100644 index 00000000..8c025a71 --- /dev/null +++ b/cpp/monoprop/detail/parallel/Options.h @@ -0,0 +1,31 @@ +// Copyright 2026 Algorithmiq +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +#pragma once + +// Deliberately free of : signatures can carry Options without exposing OpenMP. + +namespace monoprop::detail::parallel { + +/*! + * \brief Thread budget for one worksharing region. + * + * The budget is an upper bound on the workers a region requests, never a global OpenMP runtime setting. + * The runtime may provide fewer workers; correctness never depends on receiving the full budget. + */ +struct Options { + int threads = 1; //!< Validated positive per-object budget; not a global runtime setting. +}; + +} // namespace monoprop::detail::parallel diff --git a/cpp/monoprop/detail/parallel/Workshare.h b/cpp/monoprop/detail/parallel/Workshare.h new file mode 100644 index 00000000..d477b690 --- /dev/null +++ b/cpp/monoprop/detail/parallel/Workshare.h @@ -0,0 +1,99 @@ +// Copyright 2026 Algorithmiq +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +#pragma once + +#include + +#include +#include +#include +#include +#include +#include +#include + +#include "monoprop/detail/parallel/Options.h" + +namespace monoprop::detail::parallel { + +/*! + * \brief Invoke `body(block)` once for every logical block in `[0, count)`, then return. + * + * Runs serially, in ascending block order, when the budget or `count` is one, or when called from inside + * any OpenMP region (active or not). Otherwise opens one region requesting `min(options.threads, count)` + * workers with a static schedule; the runtime may provide fewer, and every block still runs exactly once. + * + * Worker exceptions never leave the region. Each worker keeps its first exception and stops taking + * blocks; after all workers join, the exception of the lowest-numbered failing worker is rethrown on + * the caller. The serial path lets an exception propagate from the failing block directly. After a + * failure, which other blocks ran is unspecified, and nothing is rolled back. + * + * The helper owns no threads and changes no OpenMP runtime setting. + * + * \param count Number of logical blocks; must not exceed `PTRDIFF_MAX`. + * \param options Thread budget; `options.threads` must be positive. + * \param body Callable taking a `std::size_t` block ID. Distinct blocks may run concurrently, so it + * must only write state owned by its block and must not call MPI. + * \throws std::invalid_argument if `options.threads` is not positive, before any block runs. + * \throws std::length_error if `count` exceeds `PTRDIFF_MAX`, before any block runs. + */ +template +auto for_blocks(std::size_t count, Options options, Fn &&body) -> void { + if (options.threads < 1) { + throw std::invalid_argument( + std::format("parallel::for_blocks: the thread budget must be positive, got {}", options.threads)); + } + // The region iterates a signed index, as the most portable OpenMP loop form requires. + constexpr auto max_count = static_cast(std::numeric_limits::max()); + if (count > max_count) { + throw std::length_error( + std::format("parallel::for_blocks: {} blocks exceed the loop bound {}", count, max_count)); + } + + const auto team = std::min(static_cast(options.threads), count); + if (team <= 1 || omp_get_level() > 0) { + for (std::size_t block = 0; block < count; ++block) { + body(block); + } + return; + } + + // Allocated before the region, so allocation failure propagates normally. The actual team never + // exceeds the num_threads request, so every worker ID indexes a slot. + std::vector errors(team); + const auto blocks = static_cast(count); + +#pragma omp parallel for schedule(static) num_threads(static_cast(team)) + for (std::ptrdiff_t block = 0; block < blocks; ++block) { + auto &error = errors[static_cast(omp_get_thread_num())]; + if (error) { + continue; + } + try { + body(static_cast(block)); + } + catch (...) { + error = std::current_exception(); + } + } + + for (const auto &error : errors) { + if (error) { + std::rethrow_exception(error); + } + } +} + +} // namespace monoprop::detail::parallel diff --git a/cpp/tests/README.md b/cpp/tests/README.md index c69941de..7b67d3eb 100644 --- a/cpp/tests/README.md +++ b/cpp/tests/README.md @@ -122,6 +122,12 @@ name and cannot address suite-nested cases, tests use flat rails + MPGraph bounds). - **Exchange layout preconditions**: `exchange_layout_precondition_tests.cpp` (the layout width check, driven through ShmComm). +- **OpenMP worksharing**: `openmp_workshare_tests.cpp` (`parallel::for_blocks`: + exactly-once visits, serial and nested fallbacks, budget and bound validation, + unchanged runtime settings, joined worker exceptions, and actual multi-worker + participation, which is skipped when the runtime cannot provide two workers). + Run it in fresh processes under `OMP_DYNAMIC=TRUE`, `OMP_DYNAMIC=FALSE` and + `OMP_THREAD_LIMIT=1` when changing the helper. New `*.cpp` files are auto-discovered on the next configure — no CMake edit needed. diff --git a/cpp/tests/link_export_probe/link_export_probe.cpp b/cpp/tests/link_export_probe/link_export_probe.cpp index f579e42c..9864eb7b 100644 --- a/cpp/tests/link_export_probe/link_export_probe.cpp +++ b/cpp/tests/link_export_probe/link_export_probe.cpp @@ -26,11 +26,18 @@ // build_graph(), graph_memory_usage(), and expectation_value_and_gradient(). // (b) the partition path (detail/partition/CpuTopology.h), via a partitions > 1 construction, which is // the only way to make PartitionGroup actually place and pin partition-worker threads. +// It also checks the installed usage requirements a consumer inherits: +// (c) the OpenMP worksharing template (detail/parallel/Workshare.h), instantiated in this translation +// unit, so OpenMP compile and link flags must reach the consumer through the imported target alone; +// (d) the Pauli basis, alongside the Majorana graph chain in (a). #include "monoprop/MonomialPropagator.h" #include "monoprop/detail/mpi/MPICompat.h" +#include "monoprop/detail/parallel/Workshare.h" #include +#include +#include #include #include #include @@ -76,6 +83,52 @@ auto run_graph_build_chain() -> void { grad.size()); } +// Without the imported target's OpenMP compile flags the pragmas in Workshare.h would silently compile +// out, so check for them explicitly rather than rely on a link failure. +#ifndef _OPENMP +#error "monoprop::monoprop did not propagate the OpenMP compile flags to this consumer" +#endif + +// Instantiates for_blocks here: its region, and the omp_* calls it makes, are compiled and linked in +// the consumer. The body only writes its own element; library calls stay outside the region. +auto run_openmp_workshare_chain() -> void { + constexpr size_t kBlocks = 257; + std::vector squares(kBlocks, 0); + monoprop::detail::parallel::for_blocks(kBlocks, {.threads = 2}, [&](size_t b) { squares[b] = b * b; }); + size_t sum = 0; + for (const auto s : squares) { + sum += s; + } + const size_t expected = (kBlocks - 1) * kBlocks * (2 * kBlocks - 1) / 6; + std::println(stderr, "[link_export_probe] openmp chain: blocks={} sum={} expected={}", kBlocks, sum, expected); + if (sum != expected) { + std::println(stderr, "[link_export_probe] openmp chain: FAILED"); + std::exit(1); + } +} + +// The Pauli basis: slots 2q/2q+1 are qubit q's x/z planes, so {0, 1} is Z on qubit 0 and {0} is X. +auto run_pauli_chain() -> void { + constexpr size_t kQubits = 3; + OperatorDict observable; + observable[VecZ{0, 1}] = std::complex{1.0, 0.0}; + + MonomialPropagator sim(observable, + /*cutoff=*/2 * kQubits, + VecZ{}, + /*schrodinger_cutoff=*/std::nullopt, + MPI_COMM_SELF, + /*lower_atol=*/std::nullopt, + /*upper_atol=*/std::nullopt, + CutoffType::Support, + /*basis_change=*/std::nullopt, + /*logical_num_modes=*/kQubits, + Basis::Pauli); + sim.propagate(std::vector{{0}}, VecZ{0}, VecD{1.0}, VecD{0.3}); + + std::println(stderr, "[link_export_probe] pauli chain: size={}", sim.size()); +} + // Drives PartitionGroup's constructor: partitions > 1 is required for the facade to actually exist, so // enumerate_physical_cores / affinity_mask_words / summarize_masks / format_place_line / partition_cpusets / // pin_this_thread all run for real (not merely compiled) on the master threads it spawns. @@ -106,6 +159,8 @@ auto main() -> int { monoprop::mpi::init(); run_graph_build_chain(); run_partition_chain(); + run_openmp_workshare_chain(); + run_pauli_chain(); monoprop::mpi::finalize(); std::println(stderr, "[link_export_probe] OK"); return 0; diff --git a/cpp/tests/openmp_workshare_tests.cpp b/cpp/tests/openmp_workshare_tests.cpp new file mode 100644 index 00000000..89093f47 --- /dev/null +++ b/cpp/tests/openmp_workshare_tests.cpp @@ -0,0 +1,324 @@ +// Copyright 2026 Algorithmiq +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +// Workers never call BOOST_TEST: each block writes only its own preallocated element (or an atomic), +// and every assertion runs on the caller after for_blocks has returned. + +#include + +#include + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include "monoprop/detail/parallel/Options.h" +#include "monoprop/detail/parallel/Workshare.h" + +using monoprop::detail::parallel::for_blocks; +using monoprop::detail::parallel::Options; + +namespace { + +// The OpenMP runtime settings (ICVs) that a library helper must never change. +struct RuntimeSettings { + int max_threads = 0; + int dynamic = 0; + int max_active_levels = 0; + int thread_limit = 0; + omp_sched_t schedule = omp_sched_static; + int chunk = 0; + omp_proc_bind_t proc_bind = omp_proc_bind_false; + int level = 0; + + auto operator==(const RuntimeSettings &) const -> bool = default; +}; + +auto runtime_settings() -> RuntimeSettings { + RuntimeSettings s; + s.max_threads = omp_get_max_threads(); + s.dynamic = omp_get_dynamic(); + s.max_active_levels = omp_get_max_active_levels(); + s.thread_limit = omp_get_thread_limit(); + omp_get_schedule(&s.schedule, &s.chunk); + s.proc_bind = omp_get_proc_bind(); + s.level = omp_get_level(); + return s; +} + +// A test-side probe, not library behavior: can this runtime give a two-thread region right now? +auto runtime_offers_two_workers(boost::unit_test::test_unit_id) -> boost::test_tools::assertion_result { + if (omp_get_thread_limit() < 2) { + return {false}; + } + int team = 0; +#pragma omp parallel num_threads(2) + { +#pragma omp single + team = omp_get_num_threads(); + } + boost::test_tools::assertion_result result(team >= 2); + result.message() << "the OpenMP runtime provided " << team << " worker(s) for a two-thread request"; + return result; +} + +} // namespace + +BOOST_AUTO_TEST_CASE(openmp_blocks_visit_once) { + std::vector visits(19); + for_blocks(19, {.threads = 3}, [&](size_t b) { ++visits[b]; }); + for (const auto n : visits) { + BOOST_TEST(n == 1); + } +} + +BOOST_AUTO_TEST_CASE(openmp_blocks_visit_once_across_counts_and_budgets) { + for (const size_t count : {size_t{0}, size_t{1}, size_t{19}}) { + for (const int threads : {1, 2, 3}) { + std::vector visits(count, 0); + std::vector worker(count, -1); + for_blocks(count, {.threads = threads}, [&](size_t b) { + ++visits[b]; + worker[b] = omp_get_thread_num(); + }); + const auto requested = std::min(static_cast(threads), count); + for (size_t b = 0; b < count; ++b) { + BOOST_TEST(visits[b] == 1, "count=" << count << " threads=" << threads << " block=" << b); + BOOST_TEST(worker[b] >= 0); + BOOST_TEST(static_cast(worker[b]) < requested); + } + } + } +} + +BOOST_AUTO_TEST_CASE(openmp_blocks_more_requested_workers_than_blocks) { + constexpr size_t count = 2; + std::vector visits(count, 0); + std::vector team(count, 0); + std::vector worker(count, -1); + for_blocks(count, {.threads = 8}, [&](size_t b) { + ++visits[b]; + team[b] = omp_get_num_threads(); + worker[b] = omp_get_thread_num(); + }); + for (size_t b = 0; b < count; ++b) { + BOOST_TEST(visits[b] == 1); + // The region requests min(threads, count) workers, so no idle third worker is created. + BOOST_TEST(team[b] >= 1); + BOOST_TEST(team[b] <= static_cast(count)); + BOOST_TEST(worker[b] < static_cast(count)); + } +} + +BOOST_AUTO_TEST_CASE(openmp_blocks_budget_one_runs_ascending_on_the_caller) { + constexpr size_t count = 19; + const auto caller = std::this_thread::get_id(); + std::vector order; + bool on_caller = true; + int level = -1; + for_blocks(count, {.threads = 1}, [&](size_t b) { + order.push_back(b); + on_caller = on_caller && std::this_thread::get_id() == caller; + level = omp_get_level(); + }); + BOOST_REQUIRE_EQUAL(order.size(), count); + for (size_t b = 0; b < count; ++b) { + BOOST_TEST(order[b] == b); + } + BOOST_TEST(on_caller); + BOOST_TEST(level == 0); // no OpenMP region was opened +} + +// The serial path runs the body directly, so its first exception ends the loop at once. +BOOST_AUTO_TEST_CASE(openmp_blocks_budget_one_propagates_at_the_failing_block) { + std::vector order; + BOOST_CHECK_EXCEPTION(for_blocks(19, + {.threads = 1}, + [&](size_t b) { + order.push_back(b); + if (b == 5) { + throw std::runtime_error("block 5"); + } + }), + std::runtime_error, + [](const std::runtime_error &e) { return std::string(e.what()) == "block 5"; }); + const std::vector expected{0, 1, 2, 3, 4, 5}; + BOOST_CHECK_EQUAL_COLLECTIONS(order.begin(), order.end(), expected.begin(), expected.end()); +} + +BOOST_AUTO_TEST_CASE(openmp_blocks_nested_invocation_runs_serially_in_order) { + constexpr size_t outer = 4; + constexpr size_t inner = 5; + std::vector visits(outer * inner, 0); + std::vector> order(outer); + std::vector outer_level(outer, -1); + std::vector inner_level(outer * inner, -1); + for_blocks(outer, {.threads = 2}, [&](size_t o) { + outer_level[o] = omp_get_level(); + for_blocks(inner, {.threads = 3}, [&](size_t i) { + ++visits[o * inner + i]; + order[o].push_back(i); // one writer: the inner loop runs on outer block o's worker + inner_level[o * inner + i] = omp_get_level(); + }); + }); + for (size_t o = 0; o < outer; ++o) { + BOOST_REQUIRE_EQUAL(order[o].size(), inner); + for (size_t i = 0; i < inner; ++i) { + BOOST_TEST(visits[o * inner + i] == 1); + BOOST_TEST(order[o][i] == i); + // No nested region: the inner body runs at the outer body's nesting level. + BOOST_TEST(inner_level[o * inner + i] == outer_level[o]); + } + } +} + +BOOST_AUTO_TEST_CASE(openmp_blocks_nested_invocation_propagates_at_the_failing_block) { + std::vector> order(2); + BOOST_CHECK_THROW(for_blocks(2, + {.threads = 2}, + [&](size_t o) { + for_blocks(6, {.threads = 3}, [&](size_t i) { + order[o].push_back(i); + if (i == 2) { + throw std::runtime_error("inner"); + } + }); + }), + std::runtime_error); + // Each outer block that ran stopped its serial inner loop at the failing block. + for (const auto &seen : order) { + BOOST_TEST(seen.size() <= 3U); + for (size_t i = 0; i < seen.size(); ++i) { + BOOST_TEST(seen[i] == i); + } + } +} + +BOOST_AUTO_TEST_CASE(openmp_blocks_invalid_budget_is_rejected_before_work) { + for (const int threads : {0, -1, INT_MIN}) { + for (const size_t count : {size_t{0}, size_t{5}}) { + int calls = 0; + BOOST_CHECK_THROW(for_blocks(count, {.threads = threads}, [&](size_t) { ++calls; }), std::invalid_argument); + BOOST_TEST(calls == 0); + } + } +} + +BOOST_AUTO_TEST_CASE(openmp_blocks_unrepresentable_count_is_rejected_before_work) { + const auto first_unrepresentable = static_cast(std::numeric_limits::max()) + 1U; + for (const size_t count : {first_unrepresentable, std::numeric_limits::max()}) { + for (const int threads : {1, 3}) { + int calls = 0; + BOOST_CHECK_THROW(for_blocks(count, {.threads = threads}, [&](size_t) { ++calls; }), std::length_error); + BOOST_TEST(calls == 0); + } + } +} + +BOOST_AUTO_TEST_CASE(openmp_blocks_leave_runtime_settings_unchanged) { + const auto before = runtime_settings(); + std::vector sink(64, 0); + for_blocks(sink.size(), {.threads = 1}, [&](size_t b) { sink[b] = 1; }); + for_blocks(sink.size(), {.threads = 3}, [&](size_t b) { sink[b] = 2; }); + for_blocks(sink.size(), {.threads = 1000}, [&](size_t b) { sink[b] = 3; }); + for_blocks(4, {.threads = 2}, [&](size_t o) { + for_blocks(16, {.threads = 4}, [&](size_t i) { sink[o * 16 + i] = 4; }); + }); + BOOST_CHECK_THROW(for_blocks(8, {.threads = 3}, [](size_t) { throw std::runtime_error("fail"); }), + std::runtime_error); + BOOST_CHECK_THROW(for_blocks(8, {.threads = 0}, [](size_t) {}), std::invalid_argument); + const auto after = runtime_settings(); + BOOST_TEST((before == after)); + BOOST_TEST(after.max_threads == before.max_threads); + BOOST_TEST(after.dynamic == before.dynamic); + BOOST_TEST(after.max_active_levels == before.max_active_levels); + BOOST_TEST(std::count(sink.begin(), sink.end(), 4) == 64); +} + +// A worker failure is caught inside the region, every worker joins, and only then does the caller +// see the exception. After a failure no particular subset of blocks is required to have run. +BOOST_AUTO_TEST_CASE(openmp_blocks_worker_exception_is_rethrown_after_join) { + constexpr size_t count = 19; + std::atomic in_flight{0}; + std::vector finished(count, 0); + // Block 7 opens the second worker's static chunk (19 blocks over 3 workers: 7/6/6), so the + // exception is raised while the other workers are still sleeping inside their bodies. + constexpr size_t failing = 7; + int in_flight_at_catch = -1; + try { + for_blocks(count, {.threads = 3}, [&](size_t b) { + in_flight.fetch_add(1); + if (b == failing) { + in_flight.fetch_sub(1); + throw std::runtime_error("block 7"); + } + std::this_thread::sleep_for(std::chrono::milliseconds(2)); + finished[b] = 1; + in_flight.fetch_sub(1); + }); + BOOST_FAIL("for_blocks returned normally after a worker threw"); + } + catch (const std::runtime_error &e) { + in_flight_at_catch = in_flight.load(); + BOOST_TEST(std::string(e.what()) == "block 7"); + } + BOOST_TEST(in_flight_at_catch == 0); + BOOST_TEST(finished[failing] == 0); +} + +// Every block throws; worker 0 always receives block 0 first under a static schedule, whatever the +// actual team size, so the first populated slot is worker 0's and it holds block 0's exception. +BOOST_AUTO_TEST_CASE(openmp_blocks_first_populated_worker_slot_is_rethrown) { + for (const int threads : {2, 3}) { + BOOST_CHECK_EXCEPTION( + for_blocks(19, {.threads = threads}, [](size_t b) { throw std::runtime_error(std::to_string(b)); }), + std::runtime_error, + [](const std::runtime_error &e) { return std::string(e.what()) == "0"; }); + } +} + +// Test-only worker observation: each block records its actual OpenMP worker ID. A request for four +// workers is not evidence of participation; the recorded IDs are. +BOOST_AUTO_TEST_CASE(openmp_blocks_multiple_workers_participate, + *boost::unit_test::precondition(runtime_offers_two_workers)) { + constexpr size_t count = 64; + constexpr int threads = 4; + std::vector worker(count, -1); + std::vector team(count, 0); + for_blocks(count, {.threads = threads}, [&](size_t b) { + worker[b] = omp_get_thread_num(); + team[b] = omp_get_num_threads(); + }); + const auto actual = *std::max_element(team.begin(), team.end()); + const std::set distinct(worker.begin(), worker.end()); + BOOST_TEST(actual >= 1); + BOOST_TEST(actual <= threads); + // A static schedule with count >= team hands every actual worker at least one block. + BOOST_TEST(distinct.size() == static_cast(actual)); + BOOST_TEST(*distinct.begin() == 0); + BOOST_TEST(*distinct.rbegin() == actual - 1); + BOOST_WARN_MESSAGE( + actual >= 2, + "multi-worker participation NOT demonstrated: the runtime gave this region " << actual << " worker(s)"); + BOOST_TEST_MESSAGE("openmp_blocks_multiple_workers_participate: observed " + << distinct.size() << " distinct workers of " << threads << " requested"); +} diff --git a/docs/content/docs/building.mdx b/docs/content/docs/building.mdx index 5b469a2f..a4156736 100644 --- a/docs/content/docs/building.mdx +++ b/docs/content/docs/building.mdx @@ -26,6 +26,15 @@ without MPI, so a from-source build is required for multi-rank runs. - Python 3.11 or newer and the `uv` package manager (for the bindings) - an MPI implementation such as Open MPI (only for MPI builds) - `hwloc` (version 2.9+) and `pkg-config` (required so CMake can locate `hwloc`) +- OpenMP for C++, in every build — with or without MPI, and even when you only + run one thread; there is no OpenMP-disabled build. Use the runtime that matches + your compiler: + - **GCC** ships its own `libgomp`; nothing extra is needed; + - **Clang on Linux** needs the `libomp` of the same LLVM version, e.g. + `libomp-18-dev` for `clang++-18` on Debian/Ubuntu; + - **Apple Clang** needs Homebrew's `libomp`. It is keg-only, so point CMake at + it with `export OpenMP_ROOT="$(brew --prefix libomp)"` before building, and + before configuring a project that consumes the installed package. The repository ships a [DevContainer](https://containers.dev/) with all of the above pre-configured; opening the folder in VS Code and rebuilding the container is @@ -42,6 +51,10 @@ The repository is also a [Nix flake](https://wiki.nixos.org/wiki/Flakes): | `nix build .#monoprop-mpi` | the same package with `monoprop_ENABLE_MPI=ON` | | `nix run` | a Python interpreter with `monoprop` importable | +The packages and the shell take OpenMP from the compiler: GCC's `libgomp` on Linux, +and `llvmPackages.openmp` wherever the standard environment's compiler is Clang +(macOS). `nix flake check` builds both packages. + Inside `nix develop` the `uv sync` and `just` workflows below apply unchanged. The shell sets two variables that only matter on NixOS: `UV_PYTHON_PREFERENCE=only-system`, because uv's managed interpreters expect a loader NixOS does not provide, and @@ -63,7 +76,7 @@ The packaged build deviates from the `uv` build in three places, all in Use the exported overlay when monoprop must share a downstream project's Python and native dependency package set. Following the downstream `nixpkgs` input avoids -introducing a second Python, Boost, hwloc, or MPI closure: +introducing a second Python, Boost, hwloc, OpenMP, or MPI closure: ```nix { @@ -222,7 +235,10 @@ link dependencies merely because the consuming project already has an `MPI::MPI_ package through `find_package(monoprop CONFIG)` — and runs it. That is the only leg that exercises `monopropConfig.cmake`, so it is what catches a missing `find_dependency` or a usage requirement the package fails to export. It compiles the same source as the in-tree -probe, so the two cannot drift. +probe, so the two cannot drift. The probe drives both the Majorana and the Pauli basis and +instantiates the internal OpenMP worksharing template in its own translation unit, so the +consumer only builds if the imported `monoprop::monoprop` target carries OpenMP's compile +and link flags — the consumer project adds none of its own. ### Debug tree diff --git a/flake.nix b/flake.nix index 3344afb5..82924376 100644 --- a/flake.nix +++ b/flake.nix @@ -70,8 +70,9 @@ devShells.default = pkgs.callPackage ./nix/devshell.nix { inherit python; }; + # Both packages configure against the required OpenMP and are import-checked. checks = { - inherit monoprop; + inherit monoprop monoprop-mpi; }; formatter = pkgs.nixfmt-tree; diff --git a/nix/devshell.nix b/nix/devshell.nix index 22ec6347..2da14d5d 100644 --- a/nix/devshell.nix +++ b/nix/devshell.nix @@ -16,6 +16,7 @@ # C++ dependencies boost, hwloc, + llvmPackages, openmpi, # workflow tooling @@ -43,12 +44,14 @@ mkShell { ]; # Host inputs, so that CMake's setup hook puts them on NIXPKGS_CMAKE_PREFIX_PATH - # and PKG_CONFIG_PATH for the CMake run scikit-build-core drives. + # and PKG_CONFIG_PATH for the CMake run scikit-build-core drives. OpenMP is required: + # the GCC stdenv ships libgomp, while Clang (the Darwin stdenv) needs the LLVM runtime. buildInputs = [ boost hwloc openmpi - ]; + ] + ++ lib.optionals stdenv.cc.isClang [ llvmPackages.openmp ]; shellHook = '' # `[tool.uv] python-preference = "only-managed"` pulls prebuilt interpreters diff --git a/nix/monoprop.nix b/nix/monoprop.nix index 83d5b63d..20ead8ec 100644 --- a/nix/monoprop.nix +++ b/nix/monoprop.nix @@ -1,5 +1,6 @@ { lib, + stdenv, buildPythonPackage, python, @@ -15,6 +16,7 @@ # C++ dependencies boost, hwloc, + llvmPackages, # runtime dependencies msgpack, @@ -81,10 +83,13 @@ buildPythonPackage { # variable is deliberately left unset -- see `env` below. ++ lib.optionals withMPI [ mpi4py ]; + # OpenMP is required. GCC's libgomp comes with the stdenv compiler; Clang (the Darwin + # stdenv) needs the matching LLVM runtime. buildInputs = [ boost hwloc ] + ++ lib.optionals stdenv.cc.isClang [ llvmPackages.openmp ] ++ lib.optionals withMPI [ mpi ]; dependencies = [ diff --git a/pyproject.toml b/pyproject.toml index 5ca96222..325fd097 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -403,5 +403,6 @@ before-all = ["./tools/install-deps.sh --skip-boost-test --skip-msgpack"] environment = { SKBUILD_CMAKE_ARGS = "-Dmonoprop_ENABLE_ARCH_FLAGS=OFF;-Dmonoprop_ENABLE_CXX_UNIT_TESTS=OFF;-Dmonoprop_ENABLE_MPI=OFF" } [tool.cibuildwheel.macos] -before-all = "brew install boost hwloc" -environment = { MACOSX_DEPLOYMENT_TARGET = "15.0", SKBUILD_CMAKE_ARGS = "-Dmonoprop_ENABLE_ARCH_FLAGS=OFF;-Dmonoprop_ENABLE_CXX_UNIT_TESTS=OFF;-Dmonoprop_ENABLE_MPI=OFF" } +before-all = "brew install boost hwloc libomp" +# Homebrew's libomp is keg-only; delocate then bundles it into the wheel. +environment = { MACOSX_DEPLOYMENT_TARGET = "15.0", OpenMP_ROOT = "$(brew --prefix libomp)", SKBUILD_CMAKE_ARGS = "-Dmonoprop_ENABLE_ARCH_FLAGS=OFF;-Dmonoprop_ENABLE_CXX_UNIT_TESTS=OFF;-Dmonoprop_ENABLE_MPI=OFF" } diff --git a/tools/install-deps.sh b/tools/install-deps.sh index 027c6d01..6895f388 100755 --- a/tools/install-deps.sh +++ b/tools/install-deps.sh @@ -14,6 +14,10 @@ This script can install Boost Unordered, Boost Test, msgpack-cxx, and hwloc. Each component can be skipped with the corresponding option. The default installation prefix is /usr/local. +OpenMP is also required but is not installed here: it comes with the compiler. +GCC ships libgomp; Clang needs the matching libomp (e.g. libomp-18-dev for +clang++-18, or Homebrew's libomp with OpenMP_ROOT pointing at it). + Arguments: INSTALL_PREFIX Directory to install dependencies (default: /usr/local) @@ -215,6 +219,7 @@ echo echo "Dependencies installation completed successfully!" echo "Install location: $INSTALL_PREFIX" echo "Make sure to set CMAKE_PREFIX_PATH=$INSTALL_PREFIX when building monoprop" +echo "OpenMP is not installed by this script; it must come with your compiler (see --help)" # Show what was installed echo diff --git a/tools/packages/apt.txt b/tools/packages/apt.txt index ca01bbd3..ce1db6c7 100644 --- a/tools/packages/apt.txt +++ b/tools/packages/apt.txt @@ -4,3 +4,7 @@ libboost-dev # Boost.Unordered and friends, a PUBLIC dependency libboost-test-dev # Boost.Test, the C++ unit-test framework libmsgpack-cxx-dev # reads the tests/data/*.msgpack fixtures libhwloc-dev # CpuTopology.cpp: core discovery and partition pinning + +# OpenMP is required but not listed: g++ ships its own libgomp. A Clang lane needs the +# libomp matching its compiler version (libomp-18-dev for clang++-18), which +# .github/actions/setup derives from CXX. diff --git a/tools/packages/brew.txt b/tools/packages/brew.txt index 6b0738ac..542b00b5 100644 --- a/tools/packages/brew.txt +++ b/tools/packages/brew.txt @@ -3,3 +3,4 @@ boost # Boost.Unordered and Boost.Test msgpack-cxx # reads the tests/data/*.msgpack fixtures hwloc # CpuTopology.cpp: core discovery and partition pinning +libomp # OpenMP runtime for Apple Clang; keg-only, so set OpenMP_ROOT From d749787e45605828705eddc5d1c88bda9b67a1d8 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Roberto=20Di=20Remigio=20Eik=C3=A5s?= Date: Sun, 27 Sep 2026 18:02:23 +0000 Subject: [PATCH 11/64] =?UTF-8?q?docs(parallelism):=20=F0=9F=93=9D=20recor?= =?UTF-8?q?d=20the=20Task=202=20outcome?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Record what Task 2 delivered and what later tasks inherit: - the helper's implemented contract: exception types, serial and nested fallback conditions, and per-worker stop-on-first-failure; - RED/GREEN and participation evidence under libgomp and libomp, and the load-bearing find_dependency; - the observation that GNU ld's --as-needed drops libgomp while the engine has no OpenMP code, so wheel runtime bundling is still unverified; - which dependency routes were verified on Linux x86_64 and which await CI or a suitable host; - a build-route note: rebuild through uv sync, since the adopted tree's cached ninja is gone, and keep M and Clang trees in their own venvs; - where the uncommitted Task 2 artifacts live. Assisted-by: Pi:claude-opus-5-5 --- .../plans/2026-09-18-rank-local-openmp.md | 45 +++++++++++++++++++ 1 file changed, 45 insertions(+) diff --git a/docs/superpowers/plans/2026-09-18-rank-local-openmp.md b/docs/superpowers/plans/2026-09-18-rank-local-openmp.md index 05f12837..7a741526 100644 --- a/docs/superpowers/plans/2026-09-18-rank-local-openmp.md +++ b/docs/superpowers/plans/2026-09-18-rank-local-openmp.md @@ -1252,6 +1252,51 @@ is still OpenMP-enabled. OpenMP template and both basis APIs using installed targets only. RED: missing imported OpenMP dependency/header; GREEN: clean consumer finds/links/runs with no manual flags or source includes. No engine behavior changes yet. +**Task 2 outcome (2026-09-27):** gate passed on GCC 15.2 and Clang 18.1.8 Linux; other platforms pending. +- Helper contract as implemented, which later tasks rely on: + - `for_blocks` throws `std::invalid_argument` for a budget below one and `std::length_error` for + `count > PTRDIFF_MAX`, before any block runs, on every path. + - It runs serially and in ascending order when `min(threads, count) <= 1` or `omp_get_level() > 0`, so any + enclosing region counts, active or not. + - In the parallel path each worker stops taking blocks after its own first exception. The lowest worker ID's + exception is rethrown after the join. +- Tests: `cpp/tests/openmp_workshare_tests.cpp` has 13 flat cases. The participation case is skipped by a + Boost precondition when the runtime cannot provide a two-worker region. + - RED: missing header, then `undefined reference to omp_get_thread_num` without the CMake dependency, then + the installed consumer missing `Workshare.h`. + - GREEN: 4 of 4 requested workers observed under `OMP_DYNAMIC=TRUE/FALSE` with both libgomp and libomp; + `OMP_THREAD_LIMIT=1` stays correct with the participation case skipped; hundreds of fresh processes + without failure. Injected helper mutations were all caught. +- R and M suites pass: splitmix R=1–4 and linear R=1,2,4, in C++ and Python. `just test-find-package` passes + for the R, M, Clang and repaired-wheel installs. A copy of the installed config without + `find_dependency(OpenMP)` fails to configure, so that line is load-bearing. +- The engine has no OpenMP code yet, and GNU ld's default `--as-needed` drops libgomp. The GCC `_core.abi3.so` + and `libmonoprop.so` are byte-identical to the Task 1 baselines, and a locally repaired wheel bundles no + OpenMP runtime. Verify runtime bundling and delocation once library code uses OpenMP (Task 4 onward, and + Task 13). +- Dependency routes: + - Homebrew `libomp`, found through `OpenMP_ROOT=$(brew --prefix libomp)` in the setup action, the macOS + cibuildwheel environment and the docs. Downstream macOS consumers need the same variable. + - The setup action installs `libomp-dev` whenever `CXX=clang++` on Linux. The devcontainer adds + `libomp-dev` for its default clang. + - Nix adds `llvmPackages.openmp` when `stdenv.cc.isClang`, and the flake checks include `monoprop-mpi`. + `nix.yml` gains a dev-shell `just build` + `just test-find-package` step. + - The clang-tidy job in `qa-analysis.yml` sets `CXX: clang++` for setup, so it receives libomp. + - Hwloc is unchanged. +- Verified only on Linux x86_64 (GCC, Clang, local wheel with auditwheel). Nix, macOS/Homebrew, the devcontainer + and cibuildwheel/delocate have not run; they are pending CI or a suitable host. +- Build-route note: `cmake --build --preset skbuild-release` cannot rebuild a tree that `uv sync` created, + because its cached `CMAKE_MAKE_PROGRAM` points at the deleted build-isolation ninja. Rebuild through the R/M + `uv sync` commands instead, and check binary mtimes and hashes. + - Keep M and Clang builds apart from R with `UV_PROJECT_ENVIRONMENT=` and + `--config-settings-package='monoprop:build-dir=build//{build_type}'`. + - Use `--all-extras` for comparable pytest counts: without mpi4py the communicator-parametrized cases are + not collected. +- Artifacts: `/home/ubuntu/task2-artifacts/` on the c8a.metal-24xl host. `report.md` indexes the numbered + logs, build scripts, mutant runs and the wheel. They are intentionally uncommitted and will be removed before + merging. The frozen campaign/workload files and the Task 1 artifacts and baseline binaries were verified + unchanged. + ### Task 3: Plumb per-object budgets and distributed failure boundaries **Files:** From 8cd7b62f716036aa9a2a2051562cd8e39f8df44f Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Roberto=20Di=20Remigio=20Eik=C3=A5s?= Date: Mon, 28 Sep 2026 07:42:11 +0000 Subject: [PATCH 12/64] =?UTF-8?q?feat(parallel):=20=E2=9C=A8=20plumb=20per?= =?UTF-8?q?-object=20thread=20budgets=20and=20distributed=20failure=20boun?= =?UTF-8?q?daries?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Task 3 of the rank-local OpenMP plan: configuration plumbing and safety boundaries. Kernels stay serial. - detail/parallel/ThreadBudget.{h,cpp} (exported, installed): - resolve_thread_budget is a pure parser. Unset monoprop_NUM_THREADS takes a positive omp_get_max_threads(); a set value must be ASCII digits in [1, INT_MAX], with leading zeros accepted. Empty, signed, padded, list, zero and overflowing values are rejected. There is no thread cap. - capture_thread_budget reads the environment once per call. - MonomialPropagator gains a private parallel::Options. It is serial except on the one-store prototype (explicit partitions=1 on an ordinary communicator), which captures the budget once at construction. Copies and retained functionals keep it; Shm/Hybrid children and facades stay serial. An invalid budget is a PropagatorConfigError on one rank and aborts on several. - The trailing Options reaches every path: graph-only build, stored-mask callbacks, immediate fused application, replay, retained functionals (EvalRequest::parallel), prepare_evolved_operator, the derivative (after record) and partial contraction. build_cos_callbacks closures capture it and the callback types are unchanged. - Failure boundaries: - mpi::operation_failed reports the error and calls MPI_Abort on a multi-rank ordinary communicator, and rethrows the original exception otherwise. - Guards sit inside the replay Ticket and construction PendingAlltoallv lifetimes, and around local work before each allreduce. - Invalid owners: each operation separates validation from mutation. A failure after mutation starts invalidates the object. Later operations, copy/clone and retained functionals then throw InvalidPropagatorError; earlier copies keep working. Interleaved index validation now runs first. - MPI: the requested and required level stay MPI_THREAD_SERIALIZED, and the level actually provided is checked for host- and library-initialized MPI. require_initializing_thread (MPI_Is_thread_main) makes prototype entries from another thread print a diagnostic and abort locally, with no MPI call. - Tests: - openmp_runtime_tests.cpp, plus fresh-process openmp_env_* CTest entries; - tests/test_openmp_config.py and a routes_pairwise cache case; - mpi_failure_driver.cpp, outside the unit runner: eight two-rank mpi_failure_* scenarios with a 30 s timeout, covering worker throw, before exchange, both request owners, insufficient level and wrong thread; - the link-export probe now covers the prototype chain. - Docs cover the prototype, the failed-operation rules, the ABI rebuild, OpenMP runtime linkage and the new test entries. Assisted-by: Pi:claude-opus-5-5 --- AGENTS.md | 9 + cpp/include/monoprop/Evolution.h | 17 +- cpp/include/monoprop/MPFunctions.h | 17 +- cpp/include/monoprop/MonomialPropagator.h | 103 ++- cpp/monoprop/Evolution.cpp | 90 ++- cpp/monoprop/MPFunctions.cpp | 78 +- .../detail/evolution/CosineRecompute.h | 24 +- .../detail/evolution/layer_build/Engine.h | 29 +- .../detail/evolution/layer_build/FusedApply.h | 6 +- .../detail/evolution/layer_build/Resolve.h | 9 +- .../detail/evolution/layer_build/Scan.h | 4 +- .../MonomialPropagator.inl | 638 +++++++++++------ cpp/monoprop/detail/mpi/CMakeLists.txt | 1 + cpp/monoprop/detail/mpi/MPICompat.cpp | 68 +- cpp/monoprop/detail/mpi/MPICompat.h | 41 ++ cpp/monoprop/detail/mpi/OperationFailure.h | 113 +++ cpp/monoprop/detail/parallel/CMakeLists.txt | 3 + cpp/monoprop/detail/parallel/ThreadBudget.cpp | 76 ++ cpp/monoprop/detail/parallel/ThreadBudget.h | 54 ++ cpp/tests/CMakeLists.txt | 140 ++++ cpp/tests/PropagatorTestAccess.h | 57 ++ cpp/tests/README.md | 21 +- .../link_export_probe/link_export_probe.cpp | 33 +- cpp/tests/mpi_failure_driver.cpp | 292 ++++++++ cpp/tests/mpi_utils_tests.cpp | 23 + cpp/tests/openmp_runtime_tests.cpp | 666 ++++++++++++++++++ cpp/tests/run_mpi_failure_scenario.cmake | 86 +++ docs/content/docs/building.mdx | 16 +- docs/content/docs/features/parallelism.mdx | 40 ++ docs/content/docs/testing.mdx | 22 + tests/test_openmp_config.py | 130 ++++ 31 files changed, 2563 insertions(+), 343 deletions(-) create mode 100644 cpp/monoprop/detail/mpi/OperationFailure.h create mode 100644 cpp/monoprop/detail/parallel/ThreadBudget.cpp create mode 100644 cpp/monoprop/detail/parallel/ThreadBudget.h create mode 100644 cpp/tests/PropagatorTestAccess.h create mode 100644 cpp/tests/mpi_failure_driver.cpp create mode 100644 cpp/tests/openmp_runtime_tests.cpp create mode 100644 cpp/tests/run_mpi_failure_scenario.cmake create mode 100644 tests/test_openmp_config.py diff --git a/AGENTS.md b/AGENTS.md index 8f438960..d1d71270 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -65,6 +65,15 @@ just build-docs - OpenMP is a required build dependency in every configuration (`OpenMP::OpenMP_CXX`, linked PUBLIC on both `monoprop-objs` and `monoprop`, `find_dependency` in the package config). The worksharing primitive is `detail/parallel/Workshare.h::for_blocks`. Never call `omp_set_*` from the library. + The library calls the runtime (`ThreadBudget.cpp`), so `import monoprop` loads it: with `OMP_PLACES` + set and binding on, the importing thread is bound to the first place. +- Transitional one-store prototype: only explicit `partitions=1` on an ordinary comm captures + `monoprop_NUM_THREADS` (strict parser) and enforces MPI's initializing thread. Every state operation + separates validation from mutation (`run_operation_`): a failure after mutation starts invalidates the + object and aborts a multi-rank ordinary communicator (`mpi::operation_failed`). Catch failures inside + posted `Ticket`/`PendingAlltoallv` lifetimes, never only at the outermost level. +- `cpp/tests/mpi_failure_driver.cpp` has its own `main()`, excluded from the unit-runner glob; + `ctest -L mpi_failure` runs its two-rank abort scenarios (30 s timeout; a timeout is a hang). - Slow CTest startup in MPI builds is `MPI_Init` fabric probing; see `monoprop_TEST_EXCLUDE_MPI_FABRIC` in `cpp/tests/CMakeLists.txt`. - Sanitizer rebuild and test: diff --git a/cpp/include/monoprop/Evolution.h b/cpp/include/monoprop/Evolution.h index 23f4d190..b40aaaf8 100644 --- a/cpp/include/monoprop/Evolution.h +++ b/cpp/include/monoprop/Evolution.h @@ -17,6 +17,7 @@ #include "monoprop/TypeAliases.h" #include "monoprop/detail/evolution/CosineRecomputeCallbacks.h" #include "monoprop/detail/mpi/MPICompat.h" +#include "monoprop/detail/parallel/Options.h" #include "monoprop/monopropExport.h" namespace monoprop { @@ -29,12 +30,17 @@ struct LayerAngle { double param = 0.0; ///< the optimizer parameter driving this layer }; +// Each entry below takes a trailing thread budget, `options`, for its local kernels; the default is serial. +// A rank-local failure after the entry has started communicating goes to mpi::operation_failed(): it aborts +// a multi-rank ordinary MPI communicator and rethrows the original exception otherwise. + /// Forward-evolve `op` through one layer; each rank owns its local coefficients, cross-rank cycles are communicated. monoprop_EXPORT auto evolve_step(VecD &op, const Layer &layer, double param, mpi::Comm comm, - const detail::LayerCosScale &cos_scale) -> void; + const detail::LayerCosScale &cos_scale, + detail::parallel::Options options = {}) -> void; /// Forward-evolve `op` through one layer of `graph`; `layer_idx` also selects the layer's cosine set. monoprop_EXPORT auto evolve_step(VecD &op, @@ -42,14 +48,16 @@ monoprop_EXPORT auto evolve_step(VecD &op, double param, size_t layer_idx, mpi::Comm comm, - const detail::LayerCosScale &cos_scale) -> void; + const detail::LayerCosScale &cos_scale, + detail::parallel::Options options = {}) -> void; /// Forward-evolve `coeffs` through every layer of `graph`, returning this rank's evolved coefficients. monoprop_EXPORT auto evolve_operator(VecD &&coeffs, const MPGraphView &graph, const VecD ¶ms, mpi::Comm comm, - const detail::LayerCosScale &cos_scale) -> VecD; + const detail::LayerCosScale &cos_scale, + detail::parallel::Options options = {}) -> VecD; /// Reverse-mode derivative of one layer: inverse-rotates (state, op) in place and returns the gradient term. /// `record` must be the pre-layer coefficients the forward pass kept for this same `layer_idx`. @@ -60,5 +68,6 @@ monoprop_EXPORT auto state_operator_derivative_local(VecD &state, LayerAngle angle, mpi::Comm comm, const detail::LayerCosAccumulate &cos_acc, - const detail::CosRecordView &record = {}) -> double; + const detail::CosRecordView &record = {}, + detail::parallel::Options options = {}) -> double; } // namespace monoprop diff --git a/cpp/include/monoprop/MPFunctions.h b/cpp/include/monoprop/MPFunctions.h index 9816ede6..7d124cef 100644 --- a/cpp/include/monoprop/MPFunctions.h +++ b/cpp/include/monoprop/MPFunctions.h @@ -25,6 +25,7 @@ #include "monoprop/detail/evolution/CosineRecomputeCallbacks.h" #include "monoprop/detail/mpi/MPICompat.h" #include "monoprop/detail/mpi/MPIUtils.h" +#include "monoprop/detail/parallel/Options.h" #include "monoprop/monopropExport.h" namespace monoprop { @@ -83,17 +84,19 @@ monoprop_EXPORT auto map_params(const VecD ¶meters, /// into it. Built at the call site and consumed there -- every member but `graph` is a non-owning /// reference, and `graph` is itself a view over layers the caller keeps alive. struct EvalRequest { - double e_core; ///< the identity term, added to the summed expectation value - const EvalState &state; ///< the contraction partner; see EvalState - const VecD &op; ///< un-evolved operator coefficients, one per term on this rank - const VecZ ¶meter_mapping; ///< optimizer order: which parameter drives graph layer i - const VecD &gen_coeffs; ///< optimizer order, parallel to parameter_mapping - MPGraphView graph; ///< replay window; MPGraph callers pass graph.replay_view() - const VecD ¶ms; ///< the optimizer's parameter vector, indexed by parameter_mapping + double e_core; ///< the identity term, added to the summed expectation value + const EvalState &state; ///< the contraction partner; see EvalState + const VecD &op; ///< un-evolved operator coefficients, one per term on this rank + const VecZ ¶meter_mapping; ///< optimizer order: which parameter drives graph layer i + const VecD &gen_coeffs; ///< optimizer order, parallel to parameter_mapping + MPGraphView graph; ///< replay window; MPGraph callers pass graph.replay_view() + const VecD ¶ms; ///< the optimizer's parameter vector, indexed by parameter_mapping + detail::parallel::Options parallel = {}; ///< the owning propagator's thread budget; serial by default }; /// Expectation value of the evolved operator against `request.state` plus `request.e_core`, summed over all /// ranks. `cos.scale` is required if `request.params` is non-empty; `cos.accumulate` is ignored here. +/// A rank-local failure past argument validation goes to mpi::operation_failed() before the allreduce. monoprop_EXPORT auto ev(const EvalRequest &request, mpi::Comm comm = MPI_COMM_WORLD, const detail::CosCallbacks &cos = {}) -> double; diff --git a/cpp/include/monoprop/MonomialPropagator.h b/cpp/include/monoprop/MonomialPropagator.h index 03e52973..0274bf7a 100644 --- a/cpp/include/monoprop/MonomialPropagator.h +++ b/cpp/include/monoprop/MonomialPropagator.h @@ -19,6 +19,7 @@ #include #include #include +#include #include #include #include @@ -45,10 +46,14 @@ #include "monoprop/detail/mpi/MPICompat.h" #include "monoprop/detail/mpi/MPIUtils.h" #include "monoprop/detail/operator/MPOperator.h" +#include "monoprop/detail/parallel/Options.h" namespace monoprop { namespace detail { struct FusedContract; +// White-box test access; declared here and defined only by the test suite. +template +struct PropagatorTestAccess; namespace partition { template class PartitionGroup; @@ -63,6 +68,15 @@ class PropagatorConfigError : public std::runtime_error { using std::runtime_error::runtime_error; }; +/// The propagator failed part-way through changing its state and can no longer be used. +// Thrown before any state is touched by every later state-consuming or mutating call, by copying or +// cloning it, and by the retained functionals it created. Nothing resets it: destroy the object and +// construct a new one, or keep using an independent copy made before the failure. +class InvalidPropagatorError : public std::runtime_error { +public: + using std::runtime_error::runtime_error; +}; + /// A raw-layout accessor was called on a multi-partition propagator, which owns no operator or graph. class MultiPartitionUnsupported : public std::runtime_error { public: @@ -93,6 +107,8 @@ class MonomialPropagator { /// Deep copy: clones the operator store, shares the immutable graph cores, and clones the whole /// partition group on a facade. The virtual destructor suppresses implicit moves, so a "move" deep-copies. + /// The copy keeps the source's thread budget. Throws InvalidPropagatorError, before copying anything, + /// if the source is invalid. MonomialPropagator(const MonomialPropagator &other); auto operator=(const MonomialPropagator &) -> MonomialPropagator & = delete; @@ -102,31 +118,39 @@ class MonomialPropagator { auto logical_num_modes() const -> size_t { return logical_num_modes_; } /// Term count on this rank (allreduce for global). - auto size() const -> size_t { return partition_group_ ? partitioned_size_() : mp_op_.size(); } + auto size() const -> size_t { + require_valid_(); + return partition_group_ ? partitioned_size_() : mp_op_.size(); + } /// (cosine-only indices, cycles) on this rank; cosine-only = cos-scaled but not a rotation endpoint. auto graph_size() const -> std::pair { + require_valid_(); return partition_group_ ? partitioned_graph_size_() : std::pair{cos_index_count_(), graph_.total_cycles()}; } /// Local to this rank. Single-partition only — see require_single_partition_. auto graph() const -> const MPGraph & { + require_valid_(); require_single_partition_("graph()"); return graph_; } /// This rank's operator storage. Single-partition only — see require_single_partition_. auto mp_op() -> detail::MPOperator & { + require_valid_(); require_single_partition_("mp_op()"); return mp_op_; } auto mp_op() const -> const detail::MPOperator & { + require_valid_(); require_single_partition_("mp_op()"); return mp_op_; } // The breakdown fields are additive over the disjoint hash partitions, so a facade sums them. auto graph_memory_usage() const -> GraphMemoryBreakdown { + require_valid_(); if (partition_group_) { return partitioned_graph_memory_usage_(); } @@ -134,6 +158,7 @@ class MonomialPropagator { } auto operator_memory_usage() const -> detail::MPOperatorMemoryBreakdown { + require_valid_(); if (partition_group_) { return partitioned_operator_memory_usage_(); } @@ -144,13 +169,19 @@ class MonomialPropagator { return breakdown; } - auto graph_layers() const -> size_t { return partition_group_ ? partitioned_graph_layers_() : graph_.layers(); } + auto graph_layers() const -> size_t { + require_valid_(); + return partition_group_ ? partitioned_graph_layers_() : graph_.layers(); + } /// A multi-term gate spans several layers, so n_gates() <= graph_layers(). auto n_gates() const -> size_t; /// Per-layer, in optimizer order (length = graph_layers()). - auto parameter_mapping() const -> VecZ { return graph_gate_arrays_().first; } + auto parameter_mapping() const -> VecZ { + require_valid_(); + return graph_gate_arrays_().first; + } /// Re-wire which parameter drives each graph layer, in place. Accepts a per-layer mapping (length /// graph_layers(), optimizer order) or a per-gate one (length n_gates()); on a tie, per-layer wins. @@ -158,10 +189,12 @@ class MonomialPropagator { /// This rank's monomial → coefficient index. Single-partition only — see require_single_partition_. auto indexing() -> detail::OperatorIndex & { + require_valid_(); require_single_partition_("indexing()"); return *mp_op_.store; } auto indexing() const -> const detail::OperatorIndex & { + require_valid_(); require_single_partition_("indexing()"); return *mp_op_.store; } @@ -175,6 +208,7 @@ class MonomialPropagator { auto graph_data() const -> std::vector; auto update_lower_atol(std::optional new_lower_atol) -> void { + require_valid_(); if (upper_atol_.has_value() && new_lower_atol.has_value() && (new_lower_atol.value() > upper_atol_.value())) { throw PropagatorConfigError( std::format("New lower_atol ({}) must be less than or equal to current upper_atol ({}).", @@ -185,6 +219,7 @@ class MonomialPropagator { } auto update_upper_atol(std::optional new_upper_atol) -> void { + require_valid_(); if (lower_atol_.has_value() && new_upper_atol.has_value() && new_upper_atol.value() < lower_atol_.value()) { throw PropagatorConfigError( std::format("New upper_atol ({}) must be greater than or equal to current lower_atol ({}).", @@ -203,6 +238,7 @@ class MonomialPropagator { } auto update_cutoff_type(CutoffType new_cutoff_type) -> void { + require_valid_(); validate_cutoff_config_(new_cutoff_type, basis_change_); update_setting_([&](MonomialPropagator &p) { p.cutoff_type_ = new_cutoff_type; @@ -212,6 +248,7 @@ class MonomialPropagator { /// The basis the cutoff is measured in; nullopt ⇒ the native basis. auto update_basis_change(std::optional> new_basis_change) -> void { + require_valid_(); validate_cutoff_config_(cutoff_type_, new_basis_change); update_setting_([&](MonomialPropagator &p) { p.basis_change_ = new_basis_change; @@ -223,7 +260,10 @@ class MonomialPropagator { auto basis() const -> Basis { return basis_; } - auto core_term() const -> double { return partition_group_ ? partitioned_core_term_() : core_term_; } + auto core_term() const -> double { + require_valid_(); + return partition_group_ ? partitioned_core_term_() : core_term_; + } auto cutoff() const -> unsigned int { return cutoff_; } @@ -382,6 +422,49 @@ class MonomialPropagator { std::unique_ptr> partition_group_; // PartitionGroup rebinds a cloned partition's comm_ to its own transport during a deep copy. friend class detail::partition::PartitionGroup; + friend struct detail::PropagatorTestAccess; + + // This object's kernel thread budget, fixed at construction and kept by copies and retained + // functionals. Serial, except on the one-store prototype (explicit partitions == 1 on an ordinary + // communicator), which captures monoprop_NUM_THREADS once; Task 10 makes that capture unconditional. + detail::parallel::Options parallel_{}; + // Set when an operation fails after it started changing state; never cleared (see require_valid_). + bool invalid_{false}; + // The one-store prototype, the only path that enforces the initializing-thread rule during coexistence. + bool one_store_{false}; + + /// Throw InvalidPropagatorError if a failed operation left this object's state unusable. + auto require_valid_() const -> void; + + /// Mark this object unusable. Called from a failure path, before the failure is reported or rethrown. + auto invalidate_() noexcept -> void { invalid_ = true; } + + /// Entry check for a state operation: the initializing-thread rule (one-store prototype, MPI-using + /// operations only), then validity. + auto enter_operation_(bool uses_mpi) const -> void; + + /// A failure after mutation started: invalidate, then apply the distributed failure policy + /// (mpi::operation_failed) on an ordinary propagator, or rethrow on a legacy partition facade. + [[noreturn]] auto mutation_failed_(std::exception_ptr error) -> void; + + /// Run one state operation. `body(mutation_started)` validates first and sets `mutation_started` + /// immediately before its first change to any cache, state or workspace. A failure before that + /// rethrows and leaves the object usable; a failure after it goes to mutation_failed_(). + template + auto run_operation_(bool uses_mpi, Body &&body) -> decltype(auto); + + /// Capture the prototype's budget and check MPI's thread support, applying the distributed failure + /// policy on failure: a rank that throws here would strand peers entering the routing agreement. + auto capture_one_store_budget_() -> void; + + /// The argument, epoch and graph checks a facade functional runs before its partitions, as a callable. + auto facade_functional_check_() -> std::function; + + /// Reject generator indices outside the system before any gate runs. + auto validate_generators_(const std::vector &majoranas) const -> void; + + /// `other`, once checked valid; used first in the copy constructor's member initializers. + static auto checked_source_(const MonomialPropagator &other) -> const MonomialPropagator &; // A facade's own graph_/mp_op_ are never populated, so handing them out would return plausible-looking // empty state; there is no meaningful merge either, since the callers want one partition's raw layout. @@ -408,9 +491,15 @@ class MonomialPropagator { // Validation stays at the call site, so a rejected value throws before anything is mutated, and // the partitions do not re-validate what the facade already checked against identical field values. auto update_setting_(const std::function &mutate) -> void { - mutate(*this); - if (partition_group_) { - for_each_partition_(mutate); + require_valid_(); + try { + mutate(*this); + if (partition_group_) { + for_each_partition_(mutate); + } + } + catch (...) { + mutation_failed_(std::current_exception()); } } diff --git a/cpp/monoprop/Evolution.cpp b/cpp/monoprop/Evolution.cpp index b477c131..35b834dd 100644 --- a/cpp/monoprop/Evolution.cpp +++ b/cpp/monoprop/Evolution.cpp @@ -26,6 +26,7 @@ #include "monoprop/detail/evolution/CosineRecomputeCallbacks.h" #include "monoprop/detail/mpi/Exchange.h" #include "monoprop/detail/mpi/MPICompat.h" +#include "monoprop/detail/mpi/OperationFailure.h" namespace monoprop { namespace { @@ -424,38 +425,47 @@ auto state_operator_derivative_local(VecD &state, LayerAngle angle, mpi::Comm comm, const detail::LayerCosAccumulate &cos_acc, - const detail::CosRecordView &record) -> double { + const detail::CosRecordView &record, + [[maybe_unused]] detail::parallel::Options options) -> double { const TrigValues trig(angle.param, angle.gen_coeff); const auto layer = graph.get_layer_traversal(layer_idx); const auto my_rank = static_cast(mpi::rank(comm)); const size_t R = layer.cross_rank_rank_count(); auto &snap = derivative_snapshot_scratch(); - snapshot_remote_endpoints(state, op, layer, my_rank, snap); - - // The self slot reads its entry op off the post-cos slots, which a record overwrites first. - const bool self_pre = record.count > 0 && my_rank < R; - if (self_pre) { - const auto slot = layer.cross_rank_self_slot(); - snap.self_recv_op.resize(slot.sin_send_count); - for (size_t k = 0; k < slot.sin_send_count; ++k) { - snap.self_recv_op[k] = op[detail::slot_sin_recv_index(slot, k)]; + // Snapshots and packing may throw on this rank alone before it posts: guarded, since peers post regardless. + auto in_flight = mpi::guard_distributed(comm, [&] { + snapshot_remote_endpoints(state, op, layer, my_rank, snap); + // The self slot reads its entry op off the post-cos slots, which a record overwrites first. + if (record.count > 0 && my_rank < R) { + const auto slot = layer.cross_rank_self_slot(); + snap.self_recv_op.resize(slot.sin_send_count); + for (size_t k = 0; k < slot.sin_send_count; ++k) { + snap.self_recv_op[k] = op[detail::slot_sin_recv_index(slot, k)]; + } } - } - - // No-op at single rank; the transfer touches only buffers, so the cos pass below may mutate state/op. - auto in_flight = begin_cross_rank_derivative_exchange(snap, layer, comm); - - // A = Σ s_old·h_pre over all anticommuting indices, endpoints included — hence the subtraction below. - // Pre-dividing lets the kernel's own ×sec land back on the recorded value; restore fixes up the rest. - detail::predivide_cos_record(op.data(), record, trig.cos_val); - const double A = cos_acc(layer_idx, state.data(), op.data(), trig.cos_val, trig.sec_val) * trig.sec_val; - detail::restore_cos_record(op.data(), record); + // No-op at single rank; the transfer touches only buffers, so the cos pass below may mutate state/op. + return begin_cross_rank_derivative_exchange(snap, layer, comm); + }); + const bool self_pre = record.count > 0 && my_rank < R; + // Caught here, while in_flight still owns posted requests: unwinding past it would wait on peers. EndpointContrib ep; - if (my_rank < R) { - ep = apply_self_slot_derivative_paired(state, op, layer, trig, self_pre ? snap.self_recv_op.data() : nullptr); - } + const double A = mpi::guard_distributed(comm, [&] { + // A = Σ s_old·h_pre over all anticommuting indices, endpoints included — hence the subtraction below. + // Pre-dividing lets the kernel's own ×sec land back on the recorded value; restore fixes up the rest. + detail::predivide_cos_record(op.data(), record, trig.cos_val); + const double a = cos_acc(layer_idx, state.data(), op.data(), trig.cos_val, trig.sec_val) * trig.sec_val; + detail::restore_cos_record(op.data(), record); + if (my_rank < R) { + ep = apply_self_slot_derivative_paired(state, + op, + layer, + trig, + self_pre ? snap.self_recv_op.data() : nullptr); + } + return a; + }); const auto remote = finish_cross_rank_derivative_exchange(state, op, layer, snap, trig, in_flight); ep = combine_endpoint_contrib(ep, remote); @@ -481,14 +491,17 @@ auto evolve_step_traversal_impl(VecD &op, const auto self_slot = layer.cross_rank_self_slot(); const size_t self_b_count = self_slot.sin_send_count; VecD self_b_snapshot; - self_b_snapshot.resize(self_b_count); - for (size_t k = 0; k < self_b_count; ++k) { - self_b_snapshot[k] = op[detail::slot_sin_send_index(self_slot, k)]; - } - // Pack + start the exchange before the cos scan so partner values are pre-cos and the transfer overlaps. - auto in_flight = begin_cross_rank_evolution_exchange(op, layer, comm); - cos_scale(layer_idx, op_data, cos_val); + // A throw before posting is guarded too: the peers post and wait regardless. + auto in_flight = mpi::guard_distributed(comm, [&] { + self_b_snapshot.resize(self_b_count); + for (size_t k = 0; k < self_b_count; ++k) { + self_b_snapshot[k] = op[detail::slot_sin_send_index(self_slot, k)]; + } + return begin_cross_rank_evolution_exchange(op, layer, comm); + }); + // Caught here, while in_flight still owns posted requests: unwinding past it would wait on peers. + mpi::guard_distributed(comm, [&] { cos_scale(layer_idx, op_data, cos_val); }); finish_cross_rank_evolution_exchange(op, layer, sin_val, in_flight); // Self-slot sin_recv entries: op[i] is already cos-scaled, so only the sine term is added. @@ -499,18 +512,24 @@ auto evolve_step_traversal_impl(VecD &op, } } // namespace +// The kernels are still serial, so `options` is accepted but not yet consumed below this level. auto evolve_step(VecD &op, const MPGraphView &graph, double param, size_t layer_idx, mpi::Comm comm, - const detail::LayerCosScale &cos_scale) -> void { + const detail::LayerCosScale &cos_scale, + [[maybe_unused]] detail::parallel::Options options) -> void { evolve_step_traversal_impl(op, graph.get_layer_traversal(layer_idx), param, layer_idx, comm, cos_scale); } // A standalone Layer replays as a one-layer graph, so layer_idx 0 is the only cosine set to select. -auto evolve_step(VecD &op, const Layer &layer, double param, mpi::Comm comm, const detail::LayerCosScale &cos_scale) - -> void { +auto evolve_step(VecD &op, + const Layer &layer, + double param, + mpi::Comm comm, + const detail::LayerCosScale &cos_scale, + [[maybe_unused]] detail::parallel::Options options) -> void { evolve_step_traversal_impl(op, layer.traversal(), param, 0, comm, cos_scale); } @@ -518,11 +537,12 @@ auto evolve_operator(VecD &&coeffs, const MPGraphView &graph, const VecD ¶ms, mpi::Comm comm, - const detail::LayerCosScale &cos_scale) -> VecD { + const detail::LayerCosScale &cos_scale, + detail::parallel::Options options) -> VecD { // Evolved in place in the caller's moved-from vector, then handed back: no per-layer copy. VecD evolved = std::move(coeffs); for (size_t i = 0; i < graph.layers(); ++i) { - evolve_step(evolved, graph, params[i], i, comm, cos_scale); + evolve_step(evolved, graph, params[i], i, comm, cos_scale, options); } return evolved; } diff --git a/cpp/monoprop/MPFunctions.cpp b/cpp/monoprop/MPFunctions.cpp index c355e06e..deac5bf4 100644 --- a/cpp/monoprop/MPFunctions.cpp +++ b/cpp/monoprop/MPFunctions.cpp @@ -22,6 +22,7 @@ #include "monoprop/Evolution.h" #include "monoprop/detail/evolution/CosineRecomputeCallbacks.h" +#include "monoprop/detail/mpi/OperationFailure.h" namespace monoprop { @@ -154,22 +155,30 @@ auto prepare_evolved_operator(const EvalRequest &request, mpi::Comm comm, const } auto &scratch = eval_scratch(); fill_mapped_params(scratch.mapped_params, request.params, request.parameter_mapping, request.gen_coeffs, 1.0, true); - scratch.op = request.op; - if (!record) { - scratch.op = evolve_operator(std::move(scratch.op), request.graph, scratch.mapped_params, comm, cos.scale); - return; - } - - // Step layer by layer so each record is taken while op still holds that layer's pre-layer values. + // Decided from the replicated parameters alone, so every rank rejects a missing callback together. const size_t layers = request.graph.layers(); - if (layout_cos_records(layers, scratch) && !cos.indices) { + if (record && layout_cos_records(layers, scratch) && !cos.indices) { throw MissingLayerCallback("A layer whose cosine vanishes requires a cos_indices (reverse) callback."); } - for (size_t i = 0; i < layers; ++i) { - record_pre_layer(scratch, request.graph, cos.indices, i); - evolve_step(scratch.op, request.graph, scratch.mapped_params[i], i, comm, cos.scale); - } - scratch.cos_offset[layers] = scratch.cos_values.size(); + // Local work between the per-layer exchanges; each exchange guards its own in-flight phase. + mpi::guard_distributed(comm, [&] { + scratch.op = request.op; + if (!record) { + scratch.op = evolve_operator(std::move(scratch.op), + request.graph, + scratch.mapped_params, + comm, + cos.scale, + request.parallel); + return; + } + // Step layer by layer so each record is taken while op still holds that layer's pre-layer values. + for (size_t i = 0; i < layers; ++i) { + record_pre_layer(scratch, request.graph, cos.indices, i); + evolve_step(scratch.op, request.graph, scratch.mapped_params[i], i, comm, cos.scale, request.parallel); + } + scratch.cos_offset[layers] = scratch.cos_values.size(); + }); } } // namespace @@ -288,17 +297,20 @@ auto ev(const EvalRequest &request, mpi::Comm comm, const detail::CosCallbacks & // The allreduce is unconditional -- ShmComm's is barrier-synced, so short-circuiting an empty local // sum past it would deadlock every peer. if (request.params.empty()) { - return request.e_core + mpi::allreduce_sum(request.state.dot(request.op), comm); + const double local = mpi::guard_distributed(comm, [&] { return request.state.dot(request.op); }); + return request.e_core + mpi::allreduce_sum(local, comm); } prepare_evolved_operator(request, comm, cos, /*record=*/false); - return request.e_core + mpi::allreduce_sum(request.state.dot(eval_scratch().op), comm); + const double local = mpi::guard_distributed(comm, [&] { return request.state.dot(eval_scratch().op); }); + return request.e_core + mpi::allreduce_sum(local, comm); } auto ev_and_grad(const EvalRequest &request, mpi::Comm comm, const detail::CosCallbacks &cos) -> std::pair { if (request.params.empty()) { - return {request.e_core + mpi::allreduce_sum(request.state.dot(request.op), comm), VecD(0)}; + const double local = mpi::guard_distributed(comm, [&] { return request.state.dot(request.op); }); + return {request.e_core + mpi::allreduce_sum(local, comm), VecD(0)}; } // cos.scale is checked in prepare_evolved_operator, shared by both paths; cos.accumulate is this path's own. @@ -310,7 +322,7 @@ auto ev_and_grad(const EvalRequest &request, mpi::Comm comm, const detail::CosCa // this path needs it dense. Densifying into the shared scratch keeps energy-only runs from ever // building one. auto &scratch = eval_scratch(); - request.state.scatter_into(scratch.state); + mpi::guard_distributed(comm, [&] { request.state.scatter_into(scratch.state); }); prepare_evolved_operator(request, comm, cos, /*record=*/true); auto &state_ = scratch.state; @@ -318,20 +330,24 @@ auto ev_and_grad(const EvalRequest &request, mpi::Comm comm, const detail::CosCa const auto expectation_value = mpi::allreduce_sum(inner_product(state_, op_), comm); const auto ¶meter_mapping = request.parameter_mapping; - scratch.gradient.assign(request.params.size(), 0.0); - for (size_t i = 0; i < parameter_mapping.size(); ++i) { - const auto idx = parameter_mapping.size() - 1 - i; - const auto param_ind = parameter_mapping[i]; - scratch.gradient[param_ind] += - state_operator_derivative_local(state_, - op_, - request.graph, - idx, - {.gen_coeff = request.gen_coeffs[i], .param = request.params[param_ind]}, - comm, - cos.accumulate, - layer_record_in(scratch, idx)); - } + // Local work between the per-layer exchanges and before the gradient allreduce. + mpi::guard_distributed(comm, [&] { + scratch.gradient.assign(request.params.size(), 0.0); + for (size_t i = 0; i < parameter_mapping.size(); ++i) { + const auto idx = parameter_mapping.size() - 1 - i; + const auto param_ind = parameter_mapping[i]; + scratch.gradient[param_ind] += state_operator_derivative_local( + state_, + op_, + request.graph, + idx, + {.gen_coeff = request.gen_coeffs[i], .param = request.params[param_ind]}, + comm, + cos.accumulate, + layer_record_in(scratch, idx), + request.parallel); + } + }); mpi::allreduce_sum_inplace(scratch.gradient, comm); return {request.e_core + expectation_value, scratch.gradient}; diff --git a/cpp/monoprop/detail/evolution/CosineRecompute.h b/cpp/monoprop/detail/evolution/CosineRecompute.h index 653f7da4..930bfe80 100644 --- a/cpp/monoprop/detail/evolution/CosineRecompute.h +++ b/cpp/monoprop/detail/evolution/CosineRecompute.h @@ -36,6 +36,7 @@ #include "monoprop/detail/evolution/CosineRecomputeCallbacks.h" #include "monoprop/detail/evolution/layer_build/Scan.h" #include "monoprop/detail/operator/InvertedIndex.h" +#include "monoprop/detail/parallel/Options.h" namespace monoprop::detail { @@ -207,8 +208,11 @@ inline auto cos_indices_mask(const CosMask &cos, std::vector &out) -> } template -auto scale_cos_lazy(const InvertedIndex &sc, const LazyFold &r, double *coeff, double cos_val) - -> void { +auto scale_cos_lazy(const InvertedIndex &sc, + const LazyFold &r, + double *coeff, + double cos_val, + [[maybe_unused]] parallel::Options options = {}) -> void { const size_t mask_words = r.fold.mask_words; const uint64_t *row_parity = fold_row_parity(sc, r.fold); std::vector &blk = column_block_scratch(); @@ -229,7 +233,8 @@ auto accumulate_cos_lazy(const InvertedIndex &sc, double *state, double *ham, double cos_val, - double sec_val) -> double { + double sec_val, + [[maybe_unused]] parallel::Options options = {}) -> double { const size_t mask_words = r.fold.mask_words; const uint64_t *row_parity = fold_row_parity(sc, r.fold); double loc = 0.0; @@ -248,15 +253,22 @@ auto accumulate_cos_lazy(const InvertedIndex &sc, return loc; } -inline auto scale_cos_mask(double *coeff, const CosMask &cos, double cos_val) -> void { +inline auto scale_cos_mask(double *coeff, + const CosMask &cos, + double cos_val, + [[maybe_unused]] parallel::Options options = {}) -> void { const size_t n = cos.blocks.size(); for (size_t k = 0; k < n; ++k) { const auto [base, bits] = cos.blocks[k]; for_each_cos_index(base, bits, [&](size_t i) { coeff[i] *= cos_val; }); } } -inline auto accumulate_cos_mask(double *state, double *ham, const CosMask &cos, double cos_val, double sec_val) - -> double { +inline auto accumulate_cos_mask(double *state, + double *ham, + const CosMask &cos, + double cos_val, + double sec_val, + [[maybe_unused]] parallel::Options options = {}) -> double { const size_t n = cos.blocks.size(); double loc = 0.0; for (size_t k = 0; k < n; ++k) { diff --git a/cpp/monoprop/detail/evolution/layer_build/Engine.h b/cpp/monoprop/detail/evolution/layer_build/Engine.h index 12ff3ff0..00813e67 100644 --- a/cpp/monoprop/detail/evolution/layer_build/Engine.h +++ b/cpp/monoprop/detail/evolution/layer_build/Engine.h @@ -36,8 +36,10 @@ #include "monoprop/detail/evolution/layer_build/Scan.h" #include "monoprop/detail/graph_encoding/MPGraphEncodingStorage.h" #include "monoprop/detail/mpi/MPIUtils.h" +#include "monoprop/detail/mpi/OperationFailure.h" #include "monoprop/detail/operator/MPOperator.h" #include "monoprop/detail/operator/RowAccess.h" +#include "monoprop/detail/parallel/Options.h" namespace monoprop::detail { @@ -347,6 +349,8 @@ struct LayerBuildEngine { // This gate's destination ranks (mpi::PeerPlan), derived once in build_layer. mpi::PeerPlan plan; Sink sink; + // The owning propagator's thread budget, forwarded to the resolve kernels. + parallel::Options options; LayerBuildEngine(MPOperator &local_op_, mpi::Comm comm_, @@ -356,7 +360,8 @@ struct LayerBuildEngine { size_t combined_size_, Sink &&sink_, mpi::PeerPlan plan_, - mpi::SlotWindow window_) + mpi::SlotWindow window_, + parallel::Options options_ = {}) : local_op(local_op_), comm(comm_), R(R_), @@ -365,7 +370,8 @@ struct LayerBuildEngine { combined_size(combined_size_), window(window_), plan(plan_), - sink(std::move(sink_)) { + sink(std::move(sink_)), + options(options_) { assert(window.stop() <= R && window.count != 0); queries_r.reset(window); src_idx_r.reset(window); @@ -421,12 +427,16 @@ struct LayerBuildEngine { } mpi::WindowVec &send = sink.send_buffer(queries_r, src_val_r, combined_qv_); mpi::WindowVec inc_q; - mpi::begin_alltoallv(send, comm, /*skip_self=*/false, /*known_recv_counts=*/nullptr, plan).wait_into(inc_q); - auto resp = resolve_incoming(inc_q, local_op, is_leader_pass, matched, combined_size, sink); + // Each round completes inside its handle's lifetime and under the guard, so a failure after posting + // never reaches the handle's draining destructor while a peer is still inside the round. + auto query_round = mpi::begin_alltoallv(send, comm, /*skip_self=*/false, /*known_recv_counts=*/nullptr, plan); + mpi::guard_distributed(comm, [&] { query_round.wait_into(inc_q); }); + auto resp = resolve_incoming(inc_q, local_op, is_leader_pass, matched, combined_size, sink, options); std::vector resp_recv = response_recv_counts(); mpi::WindowVec> inc_r; // Answers retrace the queries over an XOR involution, so the same plan holds. - mpi::begin_alltoallv(resp, comm, /*skip_self=*/false, &resp_recv, plan).wait_into(inc_r); + auto response_round = mpi::begin_alltoallv(resp, comm, /*skip_self=*/false, &resp_recv, plan); + mpi::guard_distributed(comm, [&] { response_round.wait_into(inc_r); }); process_responses(inc_r, src_idx_r, queries_r, my_rank, sink); } @@ -598,7 +608,8 @@ auto build_layer(MPOperator &local_op, bool schrodinger = false, VecD *fused_scale_coeffs = nullptr, bool *fused_scale_out = nullptr, - Basis basis = Basis::Majorana) -> std::shared_ptr { + Basis basis = Basis::Majorana, + parallel::Options options = {}) -> std::shared_ptr { validate_only_rotate_len_k_(only_rotate_len_k, 2 * NumModes); const size_t my_rank = static_cast(mpi::rank(comm)); const size_t R = static_cast(mpi::size(comm)); @@ -650,7 +661,8 @@ auto build_layer(MPOperator &local_op, gen_shift, /*capture_values=*/use_fused, sweep_ptr, - cos_build); + cos_build, + options); }); if (fused.cos_blocks.size() == 1) { // The serial scan produces a single cosine block set — take it wholesale. @@ -675,7 +687,8 @@ auto build_layer(MPOperator &local_op, /*combined_size=*/local_op.store->size(), std::move(sink), plan, - scan_window); + scan_window, + options); if (!identity_gen) { eng.run_exchange(/*is_leader_pass=*/true, std::move(fused.leader_queries), diff --git a/cpp/monoprop/detail/evolution/layer_build/FusedApply.h b/cpp/monoprop/detail/evolution/layer_build/FusedApply.h index fdd002de..6978e667 100644 --- a/cpp/monoprop/detail/evolution/layer_build/FusedApply.h +++ b/cpp/monoprop/detail/evolution/layer_build/FusedApply.h @@ -19,6 +19,7 @@ #include "monoprop/TypeAliases.h" #include "monoprop/detail/evolution/CosineRecompute.h" #include "monoprop/detail/evolution/layer_build/Common.h" +#include "monoprop/detail/parallel/Options.h" namespace monoprop::detail { @@ -34,7 +35,8 @@ inline auto apply_fused_contract(FusedContract &fc, const CosMask &cos, double param, bool schrodinger, - bool fused_scale) -> void { + bool fused_scale, + parallel::Options options = {}) -> void { // (1) insert records: v_tgt is the freshly-inserted term's pre-cos coeff, readable only now op_coeffs // is extended. Needed only in Schrödinger — a Heisenberg fresh insert has coeff 0, so skip the gather. if (schrodinger) { @@ -48,7 +50,7 @@ inline auto apply_fused_contract(FusedContract &fc, const double sin_val = std::sin(2 * param); double *const c = op_coeffs.data(); if (!fused_scale) { - scale_cos_mask(c, cos, cos_val); + scale_cos_mask(c, cos, cos_val, options); } // (3) One pass over hits ++ inserts ++ cross_half. Each op slot is touched by exactly one add (pivot diff --git a/cpp/monoprop/detail/evolution/layer_build/Resolve.h b/cpp/monoprop/detail/evolution/layer_build/Resolve.h index ac06dc6c..a50b6a0c 100644 --- a/cpp/monoprop/detail/evolution/layer_build/Resolve.h +++ b/cpp/monoprop/detail/evolution/layer_build/Resolve.h @@ -28,6 +28,7 @@ #include "monoprop/detail/mpi/Comm.h" #include "monoprop/detail/operator/MPOperator.h" #include "monoprop/detail/operator/RowAccess.h" +#include "monoprop/detail/parallel/Options.h" namespace monoprop::detail { @@ -88,7 +89,8 @@ struct IncomingProbe { template auto probe_incoming_queries(const mpi::WindowVec &incoming, // serialized, one VecZ per sender slot MPOperator &op, - QueryForm form) -> IncomingProbe { + QueryForm form, + [[maybe_unused]] parallel::Options options = {}) -> IncomingProbe { using QW = QueryWire; using PosT = typename IncomingProbe::PosT; IncomingProbe pr; @@ -195,9 +197,10 @@ auto resolve_incoming(const mpi::WindowVec &incoming, // serialized, one V bool is_leader_pass, MatchedEpochSet &matched, size_t combined_size, // pre-layer op size: bounds the matched set - Sink &sink) -> mpi::WindowVec> { + Sink &sink, + parallel::Options options = {}) -> mpi::WindowVec> { using Resp = typename Sink::Response; - const IncomingProbe pr = probe_incoming_queries(incoming, op, sink.incoming_form()); + const IncomingProbe pr = probe_incoming_queries(incoming, op, sink.incoming_form(), options); // The pairing is an XOR involution, so the response window is the query window. mpi::WindowVec> responses(pr.window); for (const auto wi : pr.window.indices()) { diff --git a/cpp/monoprop/detail/evolution/layer_build/Scan.h b/cpp/monoprop/detail/evolution/layer_build/Scan.h index fcf691ba..0bb2958b 100644 --- a/cpp/monoprop/detail/evolution/layer_build/Scan.h +++ b/cpp/monoprop/detail/evolution/layer_build/Scan.h @@ -37,6 +37,7 @@ #include "monoprop/detail/mpi/MPIUtils.h" #include "monoprop/detail/operator/InvertedIndex.h" #include "monoprop/detail/operator/MPOperator.h" +#include "monoprop/detail/parallel/Options.h" namespace monoprop::detail { @@ -245,7 +246,8 @@ auto fused_find_and_collect(const MPOperator &op, size_t gen_shift, bool capture_values = false, double *fused_scale_coeffs = nullptr, - double fused_scale_cos = 1.0) -> FusedScanResult { + double fused_scale_cos = 1.0, + [[maybe_unused]] parallel::Options options = {}) -> FusedScanResult { validate_only_rotate_len_k_(only_rotate_len_k, 2 * NumModes); const size_t gen_pop = gen.count(); const size_t rank_count = router.flat_world(); diff --git a/cpp/monoprop/detail/monomial_propagator/MonomialPropagator.inl b/cpp/monoprop/detail/monomial_propagator/MonomialPropagator.inl index 5e985685..7961aa87 100644 --- a/cpp/monoprop/detail/monomial_propagator/MonomialPropagator.inl +++ b/cpp/monoprop/detail/monomial_propagator/MonomialPropagator.inl @@ -40,6 +40,8 @@ #include "monoprop/detail/evolution/LayerBuilder.h" #include "monoprop/detail/evolution/layer_build/FusedApply.h" #include "monoprop/detail/monomial_propagator/MonomialPropagatorCommon.h" +#include "monoprop/detail/mpi/OperationFailure.h" +#include "monoprop/detail/parallel/ThreadBudget.h" #include "monoprop/detail/partition/PartitionGroup.h" namespace monoprop { @@ -98,6 +100,14 @@ MonomialPropagator::MonomialPropagator(const OperatorDict &initial_ope cutoff_type_{cutoff_type}, basis_change_{basis_change}, basis_{basis} { + // Transitional one-store prototype: explicit partitions=1 on an ordinary communicator (also the non-MPI + // build's only kind). Shm/Hybrid partition children and facades keep serial options and their legacy + // configuration. Checked before any MPI work in this constructor. + if (partitions == 1 && comm.kind == mpi::Comm::Kind::Mpi) { + one_store_ = true; + mpi::require_initializing_thread(); + capture_one_store_budget_(); + } if (logical_num_modes_ == 0 || logical_num_modes_ > NumModes) { throw PropagatorConfigError( std::format("logical_num_modes ({}) must be in the range [1, {}].", logical_num_modes_, NumModes)); @@ -238,7 +248,7 @@ MonomialPropagator::~MonomialPropagator() = default; template MonomialPropagator::MonomialPropagator(const MonomialPropagator &other) - : schrodinger_(other.schrodinger_), + : schrodinger_(checked_source_(other).schrodinger_), comm_(other.comm_), cutoff_fn_(other.cutoff_fn_), mp_op_(other.mp_op_), @@ -256,7 +266,82 @@ MonomialPropagator::MonomialPropagator(const MonomialPropagator &other basis_(other.basis_), partition_group_(other.partition_group_ ? std::make_unique>(*other.partition_group_) - : nullptr) {} + : nullptr), + parallel_(other.parallel_), + one_store_(other.one_store_) {} + +template +auto MonomialPropagator::checked_source_(const MonomialPropagator &other) -> const MonomialPropagator & { + other.require_valid_(); + return other; +} + +template +auto MonomialPropagator::require_valid_() const -> void { + if (invalid_) { + throw InvalidPropagatorError( + "This propagator is no longer usable: an earlier operation failed after it had started changing the " + "operator, graph or cached state, so that state is incomplete. Construct a new propagator (or use a " + "copy made before the failure); the failed object can only be destroyed."); + } +} + +template +auto MonomialPropagator::enter_operation_(bool uses_mpi) const -> void { + if (uses_mpi && one_store_) { + mpi::require_initializing_thread(); + } + require_valid_(); +} + +template +auto MonomialPropagator::mutation_failed_(std::exception_ptr error) -> void { + invalidate_(); + // A facade's failure has already passed through its partitions' own poison/abort paths. + if (!partition_group_) { + mpi::operation_failed(comm_, error); + } + std::rethrow_exception(error); +} + +template +template +auto MonomialPropagator::run_operation_(bool uses_mpi, Body &&body) -> decltype(auto) { + enter_operation_(uses_mpi); + bool mutation_started = false; + try { + return std::forward(body)(mutation_started); + } + catch (...) { + if (!mutation_started) { + throw; + } + mutation_failed_(std::current_exception()); + } +} + +template +auto MonomialPropagator::capture_one_store_budget_() -> void { + try { + mpi::require_thread_support(); + try { + parallel_ = detail::parallel::capture_thread_budget(); + } + catch (const std::invalid_argument &e) { + throw PropagatorConfigError(e.what()); + } + } + catch (...) { + mpi::operation_failed(comm_, std::current_exception()); + } +} + +template +auto MonomialPropagator::validate_generators_(const std::vector &majoranas) const -> void { + for (const auto &gate : majoranas) { + (void)indices_to_bitset_checked(gate, 2 * logical_num_modes_); + } +} template auto MonomialPropagator::resolve_partition_count_(size_t requested, mpi::Comm comm) -> size_t { @@ -405,33 +490,47 @@ auto MonomialPropagator::packed_inline_width_() const -> size_t { template auto MonomialPropagator::apply_initial_operator_(const OperatorDict &op_dict) -> std::pair, VecD> { - ++initial_operator_epoch_; - if (partition_group_) { - // The facade holds no local terms of its own, so the return is empty. - for_each_partition_([&](MonomialPropagator &s) { s.update_initial_operator(op_dict); }); - return {}; - } - const routing::Router router = router_for(comm_); // hoisted, never per term - const size_t my_rank = static_cast(mpi::rank(comm_)); - - OperatorDict new_op; - for (const auto &[ind, coeff] : op_dict) { - const auto mono = indices_to_bitset_checked(ind, 2 * logical_num_modes_); - if (ind.empty()) { // Core term, store in all - core_term_ = algebra_encode_coeff(basis_, coeff, mono); - continue; + return run_operation_(true, [&](bool &mutation_started) -> std::pair, VecD> { + if (partition_group_) { + for (const auto &[ind, coeff] : op_dict) { + (void)indices_to_bitset_checked(ind, 2 * logical_num_modes_); + } + mutation_started = true; + ++initial_operator_epoch_; + // The facade holds no local terms of its own, so the return is empty. + for_each_partition_([&](MonomialPropagator &s) { s.update_initial_operator(op_dict); }); + return {}; } - if (my_rank == find_rank(mono, router)) { - const auto mono_indices = bitset_to_indices(mono); - new_op[mono_indices] = coeff; + const routing::Router router = router_for(comm_); // hoisted, never per term + const size_t my_rank = static_cast(mpi::rank(comm_)); + + // Read-only pass: every index is checked before anything, the epoch included, changes. + OperatorDict new_op; + std::optional new_core_term; + for (const auto &[ind, coeff] : op_dict) { + const auto mono = indices_to_bitset_checked(ind, 2 * logical_num_modes_); + if (ind.empty()) { // Core term, store in all + new_core_term = algebra_encode_coeff(basis_, coeff, mono); + continue; + } + if (my_rank == find_rank(mono, router)) { + const auto mono_indices = bitset_to_indices(mono); + new_op[mono_indices] = coeff; + } } - } - return mp_op_.update_initial_operator(new_op, schrodinger_); + mutation_started = true; + ++initial_operator_epoch_; + if (new_core_term) { + core_term_ = *new_core_term; + } + return mp_op_.update_initial_operator(new_op, schrodinger_); + }); } template auto MonomialPropagator::graph_data() const -> std::vector { + require_valid_(); require_single_partition_("graph_data()"); std::vector layers; const auto num_layers = graph_.layers(); @@ -524,6 +623,12 @@ auto MonomialPropagator::validate_cutoff_config_(CutoffType cutoff_typ 2 * logical_num_modes_, basis_change->size())); } + // regenerate_cutoff_fn_ would reject these too, but only after an update started changing settings. + if (basis_change.has_value()) { + for (const auto &row : *basis_change) { + (void)indices_to_bitset_checked(row, 2 * logical_num_modes_); + } + } } template @@ -630,10 +735,10 @@ auto MonomialPropagator::evolve_mode_graph_with_coeffs_(const std::vec extend_coeffs_from_current_picture_if_needed_(coeffs); Layer layer(std::move(storage)); - detail::LayerCosScale cos_scale = [cos](size_t, double *c, double v) { - detail::scale_cos_mask(c, *cos, v); + detail::LayerCosScale cos_scale = [cos, options = parallel_](size_t, double *c, double v) { + detail::scale_cos_mask(c, *cos, v, options); }; - evolve_step(coeffs, layer, apply_angle, comm_, cos_scale); + evolve_step(coeffs, layer, apply_angle, comm_, cos_scale, parallel_); }); } @@ -659,7 +764,7 @@ auto MonomialPropagator::evolve_mode_contract_immediately_(const std:: bool fused_scale = false; build_evolve_result_(mono, rot_len, std::cref(*op_coeffs), build_angle, &cos, &fc, op_coeffs, &fused_scale); extend_coeffs_from_current_picture_if_needed_(*op_coeffs); - detail::apply_fused_contract(fc, *op_coeffs, cos, apply_angle, schrodinger_, fused_scale); + detail::apply_fused_contract(fc, *op_coeffs, cos, apply_angle, schrodinger_, fused_scale, parallel_); }); } @@ -670,61 +775,63 @@ auto MonomialPropagator::build_graph(const std::vector &majorana std::optional gate_indices, std::optional parameters, std::optional only_rotate_len_k) -> void { - validate_only_rotate_len_k_(only_rotate_len_k, 2 * logical_num_modes_); - if (partition_group_) { - for_each_partition_([&](MonomialPropagator &s) { - s.build_graph(majoranas, parameter_mapping, gen_coeffs, gate_indices, parameters, only_rotate_len_k); - }); - return; - } - if (majoranas.empty()) { - return; - } - validate_coefficient_lengths(parameter_mapping, gen_coeffs); - - VecZ local_gates; - if (gate_indices.has_value()) { - local_gates = std::move(*gate_indices); - } - else { - local_gates.resize(majoranas.size()); - std::iota(local_gates.begin(), local_gates.end(), size_t{0}); - } - validate_gate_indices(local_gates, majoranas.size()); - const size_t gate_offset = n_gates(); - for (auto &g : local_gates) { - g += gate_offset; - } - - if (!parameters.has_value()) { - evolve_mode_build_graph_(majoranas, parameter_mapping, gen_coeffs, local_gates, only_rotate_len_k); - } - else { - // map_params() indexes `parameters` by parameter_mapping, so a too-short vector reads out of bounds. - validate_parameters_length(*parameters, parameter_mapping); - // Coefficient-informed build: seed by contracting the existing graph so atol truncation sees - // realistic coefficients. That graph covers the parameter prefix [0, m). - VecD seed; - if (graph_layers() > 0) { - const auto existing = graph_gate_arrays_(); - const size_t m = expected_num_params(existing.first); - // The per-mapping check above only covers this call's indices, which may all sit above the - // prefix the stored graph needs. Truncating instead would replay the existing graph at a silently - // different point on the axis, and map_params would fail one layer down on the sliced vector. - if (parameters->size() < m) { - throw SeedParametersTooShort( - std::format("Coefficient-informed build_graph() needs at least {} parameter value(s) to replay the " - "existing {}-layer graph as a seed, but got {}.", - m, - graph_layers(), - parameters->size())); - } - const VecD existing_params(parameters->begin(), parameters->begin() + static_cast(m)); - seed = contract_partially(existing_params, false); + run_operation_(true, [&](bool &mutation_started) { + // Validation: identical on every rank and partition, and before any state changes. + validate_only_rotate_len_k_(only_rotate_len_k, 2 * logical_num_modes_); + if (majoranas.empty()) { + return; + } + validate_coefficient_lengths(parameter_mapping, gen_coeffs); + validate_generators_(majoranas); + VecZ local_gates; + if (gate_indices.has_value()) { + local_gates = *gate_indices; } else { - seed = current_picture_coeffs_(); + local_gates.resize(majoranas.size()); + std::iota(local_gates.begin(), local_gates.end(), size_t{0}); } + validate_gate_indices(local_gates, majoranas.size()); + std::optional seed_params; + if (parameters.has_value()) { + // map_params() indexes `parameters` by parameter_mapping, so a too-short vector reads out of bounds. + validate_parameters_length(*parameters, parameter_mapping); + if (graph_layers() > 0) { + // Coefficient-informed build: seed by contracting the existing graph so atol truncation sees + // realistic coefficients. That graph covers the parameter prefix [0, m). + const size_t m = expected_num_params(graph_gate_arrays_().first); + // The per-mapping check above only covers this call's indices, which may all sit above the + // prefix the stored graph needs. Truncating instead would replay the existing graph at a + // silently different point on the axis, and map_params would fail one layer down on the + // sliced vector. + if (parameters->size() < m) { + throw SeedParametersTooShort(std::format( + "Coefficient-informed build_graph() needs at least {} parameter value(s) to replay the " + "existing {}-layer graph as a seed, but got {}.", + m, + graph_layers(), + parameters->size())); + } + seed_params.emplace(parameters->begin(), parameters->begin() + static_cast(m)); + } + } + + mutation_started = true; + if (partition_group_) { + for_each_partition_([&](MonomialPropagator &s) { + s.build_graph(majoranas, parameter_mapping, gen_coeffs, gate_indices, parameters, only_rotate_len_k); + }); + return; + } + const size_t gate_offset = n_gates(); + for (auto &g : local_gates) { + g += gate_offset; + } + if (!parameters.has_value()) { + evolve_mode_build_graph_(majoranas, parameter_mapping, gen_coeffs, local_gates, only_rotate_len_k); + return; + } + const VecD seed = seed_params ? contract_partially(*seed_params, false) : current_picture_coeffs_(); evolve_mode_graph_with_coeffs_(majoranas, parameter_mapping, gen_coeffs, @@ -732,7 +839,7 @@ auto MonomialPropagator::build_graph(const std::vector &majorana *parameters, seed, only_rotate_len_k); - } + }); } template @@ -741,26 +848,31 @@ auto MonomialPropagator::propagate(const std::vector &majoranas, const VecD &gen_coeffs, const VecD ¶meters, std::optional only_rotate_len_k) -> void { - validate_only_rotate_len_k_(only_rotate_len_k, 2 * logical_num_modes_); - if (partition_group_) { - for_each_partition_([&](MonomialPropagator &s) { - s.propagate(majoranas, parameter_mapping, gen_coeffs, parameters, only_rotate_len_k); - }); - return; - } - if (majoranas.empty()) { - return; - } - validate_coefficient_lengths(parameter_mapping, gen_coeffs); - validate_parameters_length(parameters, parameter_mapping); - if (graph_layers() > 0) { - throw GraphStateConflict(std::format("Cannot propagate() on top of a non-empty graph of {} layer(s): " - "propagate() evolves and contracts in place and assumes no stored graph. " - "Call contract_partially() to fold the existing graph first, or use " - "build_graph() to extend it.", - graph_layers())); - } - evolve_mode_contract_immediately_(majoranas, parameter_mapping, gen_coeffs, parameters, only_rotate_len_k); + run_operation_(true, [&](bool &mutation_started) { + validate_only_rotate_len_k_(only_rotate_len_k, 2 * logical_num_modes_); + if (majoranas.empty()) { + return; + } + validate_coefficient_lengths(parameter_mapping, gen_coeffs); + validate_parameters_length(parameters, parameter_mapping); + validate_generators_(majoranas); + if (graph_layers() > 0) { + throw GraphStateConflict( + std::format("Cannot propagate() on top of a non-empty graph of {} layer(s): " + "propagate() evolves and contracts in place and assumes no stored graph. " + "Call contract_partially() to fold the existing graph first, or use " + "build_graph() to extend it.", + graph_layers())); + } + mutation_started = true; + if (partition_group_) { + for_each_partition_([&](MonomialPropagator &s) { + s.propagate(majoranas, parameter_mapping, gen_coeffs, parameters, only_rotate_len_k); + }); + return; + } + evolve_mode_contract_immediately_(majoranas, parameter_mapping, gen_coeffs, parameters, only_rotate_len_k); + }); } template @@ -849,7 +961,8 @@ auto MonomialPropagator::build_evolve_result_(const VecZ &gen_vec, schrodinger_, fused_scale_coeffs, fused_scale, - basis_); + basis_, + parallel_); } template @@ -863,10 +976,13 @@ auto MonomialPropagator::propagate_one_(const VecZ &gen_vec, graph_.append(build_evolve_result_(gen_vec, only_rotate_len_k, coeffs, param), param_index, gen_coeff, gate_index); } +/// Per-layer cosine callbacks replaying `graph` against `inverted_index`; each closure captures `options` +/// for its kernels. The closures keep raw pointers into the index, so they must not outlive it. template auto build_cos_callbacks(const detail::InvertedIndex &inverted_index, const MPGraphView &graph, - Basis basis = Basis::Majorana) -> detail::CosCallbacks; + Basis basis = Basis::Majorana, + detail::parallel::Options options = {}) -> detail::CosCallbacks; template auto MonomialPropagator::evolve_operator_with_recompute_(VecD &&coeffs, @@ -874,12 +990,13 @@ auto MonomialPropagator::evolve_operator_with_recompute_(VecD &&coeffs const VecD ¶ms) -> VecD { const auto &inverted_index = mp_op_.inverted_index(); // Only the scale side is consumed; build both through the shared builder for consistency. - auto cos_scale = build_cos_callbacks(inverted_index, graph, basis_).scale; - return evolve_operator(std::move(coeffs), graph, params, comm_, cos_scale); + auto cos_scale = build_cos_callbacks(inverted_index, graph, basis_, parallel_).scale; + return evolve_operator(std::move(coeffs), graph, params, comm_, cos_scale, parallel_); } template auto MonomialPropagator::n_gates() const -> size_t { + require_valid_(); if (partition_group_) { return first_partition_().n_gates(); } @@ -896,45 +1013,43 @@ auto MonomialPropagator::n_gates() const -> size_t { template auto MonomialPropagator::set_parameter_mapping(const VecZ ¶meter_mapping) -> void { - if (partition_group_) { - for_each_partition_([&](MonomialPropagator &s) { s.set_parameter_mapping(parameter_mapping); }); - return; - } - const size_t count = graph_.layers(); - const size_t gates = n_gates(); - - // The LayerCore is shared and immutable, so relabelling copies it and replaces the layer's core. - auto relabel = [this](size_t layer, size_t new_param_index) { - auto &target = graph_.get_layer(layer); - auto new_core = std::make_shared(target.core()); - new_core->param_index = new_param_index; - if (const CosMask *pruned = target.pruned_cos()) { - target = Layer(std::move(new_core), *pruned); + run_operation_(false, [&](bool &mutation_started) { + const size_t count = graph_layers(); + const size_t gates = n_gates(); + const bool per_layer = parameter_mapping.size() == count; + if (!per_layer && parameter_mapping.size() != gates) { + throw GraphStateConflict(std::format("parameter_mapping has {} entries; expected {} (per graph " + "layer) or {} (per gate).", + parameter_mapping.size(), + count, + gates)); } - else { - target = Layer(std::move(new_core)); + mutation_started = true; + if (partition_group_) { + for_each_partition_([&](MonomialPropagator &s) { s.set_parameter_mapping(parameter_mapping); }); + return; } - }; - if (parameter_mapping.size() == count) { - // Per-layer mapping in optimizer order. - for (size_t layer = 0; layer < count; ++layer) { - relabel(layer, parameter_mapping[count - 1 - layer]); - } - } - else if (parameter_mapping.size() == gates) { - // Per-gate mapping, indexed by absolute gate index. + // The LayerCore is shared and immutable, so relabelling copies it and replaces the layer's core. + auto relabel = [this](size_t layer, size_t new_param_index) { + auto &target = graph_.get_layer(layer); + auto new_core = std::make_shared(target.core()); + new_core->param_index = new_param_index; + if (const CosMask *pruned = target.pruned_cos()) { + target = Layer(std::move(new_core), *pruned); + } + else { + target = Layer(std::move(new_core)); + } + }; + for (size_t layer = 0; layer < count; ++layer) { - relabel(layer, parameter_mapping[graph_.get_layer_traversal(layer).gate_index()]); + // Per-layer mapping in optimizer order; per-gate mapping indexed by absolute gate index. + relabel(layer, + per_layer ? parameter_mapping[count - 1 - layer] + : parameter_mapping[graph_.get_layer_traversal(layer).gate_index()]); } - } - else { - throw GraphStateConflict(std::format("parameter_mapping has {} entries; expected {} (per graph " - "layer) or {} (per gate).", - parameter_mapping.size(), - count, - gates)); - } + }); } template @@ -956,8 +1071,10 @@ auto MonomialPropagator::graph_gate_arrays_() const -> std::pair -auto build_cos_callbacks(const detail::InvertedIndex &inverted_index, const MPGraphView &graph, Basis basis) - -> detail::CosCallbacks { +auto build_cos_callbacks(const detail::InvertedIndex &inverted_index, + const MPGraphView &graph, + Basis basis, + detail::parallel::Options options) -> detail::CosCallbacks { struct LayerCos { bool recomputes_cos = false; detail::LazyFold recipe{}; // used iff recomputes_cos @@ -982,21 +1099,22 @@ auto build_cos_callbacks(const detail::InvertedIndex &inverted_index, } const auto *sc = &inverted_index; - detail::LayerCosScale cos_scale = [cache, sc](size_t i, double *c, double v) { + // Parity words are fetched inside each call, never cached here: they move whenever the store grows. + detail::LayerCosScale cos_scale = [cache, sc, options](size_t i, double *c, double v) { const auto &e = (*cache)[i]; if (!e.recomputes_cos) { - detail::scale_cos_mask(c, *e.filtered, v); + detail::scale_cos_mask(c, *e.filtered, v, options); } else { - detail::scale_cos_lazy(*sc, e.recipe, c, v); + detail::scale_cos_lazy(*sc, e.recipe, c, v, options); } }; - detail::LayerCosAccumulate cos_acc = [cache, sc](size_t i, double *s, double *h, double v, double sec) { + detail::LayerCosAccumulate cos_acc = [cache, sc, options](size_t i, double *s, double *h, double v, double sec) { const auto &e = (*cache)[i]; if (!e.recomputes_cos) { - return detail::accumulate_cos_mask(s, h, *e.filtered, v, sec); + return detail::accumulate_cos_mask(s, h, *e.filtered, v, sec, options); } - return detail::accumulate_cos_lazy(*sc, e.recipe, s, h, v, sec); + return detail::accumulate_cos_lazy(*sc, e.recipe, s, h, v, sec, options); }; detail::LayerCosIndices cos_inds = [cache, sc](size_t i, std::vector &out) { const auto &e = (*cache)[i]; @@ -1034,10 +1152,12 @@ auto MonomialPropagator::make_functional_(Fn &&func, std::optionalcore_term(); const auto comm = comm_; + const auto options = parallel_; const auto expected_layers = graph_layers(); - // Aliased rather than copied: the check below needs the live counter, like graph->layers(). - const auto *epoch = &initial_operator_epoch_; + // The owner is referenced, not copied: the checks below need its live validity and epoch counter, + // like graph->layers(). The callable already must not outlive it (see build_cos_callbacks below). + auto *owner = this; const auto expected_epoch = initial_operator_epoch_; const auto &inverted_index = mp_op_.inverted_index(); @@ -1063,7 +1183,7 @@ auto MonomialPropagator::make_functional_(Fn &&func, std::optional(inverted_index, graph->replay_view(), basis_); + auto cos = build_cos_callbacks(inverted_index, graph->replay_view(), basis_, options); return [func = std::move(func), core_term, @@ -1073,121 +1193,182 @@ auto MonomialPropagator::make_functional_(Fn &&func, std::optional R { - validate_expected_initial_operator(*epoch, expected_epoch); + comm, + options](const VecD ¶ms) -> R { + owner->enter_operation_(true); + validate_expected_initial_operator(owner->initial_operator_epoch_, expected_epoch); validate_functional_call(params, num_params); validate_expected_graph_layers(graph->layers(), expected_layers); - return func(EvalRequest{.e_core = core_term, - .state = state, - .op = op, - .parameter_mapping = parameter_mapping, - .gen_coeffs = gen_coeffs, - .graph = graph->replay_view(), - .params = params}, - comm, - cos); + // Evaluation warms the owner's lazy caches (inverted-index parity) and per-thread scratch, so any + // failure from here on invalidates the owner. + try { + return func(EvalRequest{.e_core = core_term, + .state = state, + .op = op, + .parameter_mapping = parameter_mapping, + .gen_coeffs = gen_coeffs, + .graph = graph->replay_view(), + .params = params, + .parallel = options}, + comm, + cos); + } + catch (...) { + owner->mutation_failed_(std::current_exception()); + } }; } template auto MonomialPropagator::expectation_value_functional(std::optional pare_threshold) -> std::function { - if (partition_group_) { - // Each partition allreduces internally, so partition 0 is the global value. The group is captured by - // raw pointer, so the returned callable must not outlive this propagator. - auto fns = std::make_shared>>( - map_partitions_([&](MonomialPropagator &s) { return s.expectation_value_functional(pare_threshold); })); - auto *grp = partition_group_.get(); - return [grp, fns](const VecD ¶ms) -> double { - return detail::partition::collect_on_all(*grp, - [&](int r) { return (*fns)[static_cast(r)](params); })[0]; - }; - } - return make_functional_(ev_fn, pare_threshold); + return run_operation_(true, [&](bool &mutation_started) -> std::function { + // Building a functional warms lazy caches and may pare the graph, which communicates. + mutation_started = true; + if (partition_group_) { + // Each partition allreduces internally, so partition 0 is the global value. The group is captured + // by raw pointer, so the returned callable must not outlive this propagator. + auto fns = std::make_shared>>( + map_partitions_([&](MonomialPropagator &s) { return s.expectation_value_functional(pare_threshold); })); + auto *grp = partition_group_.get(); + return [this, grp, fns, check = facade_functional_check_()](const VecD ¶ms) -> double { + check(params); + try { + return detail::partition::collect_on_all(*grp, [&](int r) { + return (*fns)[static_cast(r)](params); + })[0]; + } + catch (...) { + mutation_failed_(std::current_exception()); + } + }; + } + return make_functional_(ev_fn, pare_threshold); + }); } template auto MonomialPropagator::expectation_value_and_gradient_functional(std::optional pare_threshold) -> std::function(const VecD &)> { - if (partition_group_) { - auto fns = std::make_shared(const VecD &)>>>(map_partitions_( - [&](MonomialPropagator &s) { return s.expectation_value_and_gradient_functional(pare_threshold); })); - auto *grp = partition_group_.get(); - return [grp, fns](const VecD ¶ms) -> std::pair { - return detail::partition::collect_on_all(*grp, - [&](int r) { return (*fns)[static_cast(r)](params); })[0]; - }; - } - return make_functional_(ev_and_grad_fn, pare_threshold); + using Functional = std::function(const VecD &)>; + return run_operation_(true, [&](bool &mutation_started) -> Functional { + mutation_started = true; + if (partition_group_) { + auto fns = std::make_shared>(map_partitions_( + [&](MonomialPropagator &s) { return s.expectation_value_and_gradient_functional(pare_threshold); })); + auto *grp = partition_group_.get(); + return [this, grp, fns, check = facade_functional_check_()](const VecD ¶ms) -> std::pair { + check(params); + try { + return detail::partition::collect_on_all(*grp, [&](int r) { + return (*fns)[static_cast(r)](params); + })[0]; + } + catch (...) { + mutation_failed_(std::current_exception()); + } + }; + } + return make_functional_(ev_and_grad_fn, pare_threshold); + }); +} + +template +auto MonomialPropagator::facade_functional_check_() -> std::function { + // The partitions' own functionals repeat these checks; doing them here first keeps a rejected + // argument from invalidating the facade. + const auto num_params = expected_num_params(graph_gate_arrays_().first); + const auto expected_layers = graph_layers(); + const auto expected_epoch = initial_operator_epoch_; + return [this, num_params, expected_layers, expected_epoch](const VecD ¶ms) { + enter_operation_(true); + validate_expected_initial_operator(initial_operator_epoch_, expected_epoch); + validate_functional_call(params, num_params); + validate_expected_graph_layers(graph_layers(), expected_layers); + }; } template auto MonomialPropagator::expectation_value(const VecD ¶meters) -> double { if (partition_group_) { - // Each partition allreduces internally, so every partition returns the global value; take partition 0. - return map_partitions_([&](MonomialPropagator &s) { return s.expectation_value(parameters); })[0]; + return run_operation_(true, [&](bool &mutation_started) -> double { + validate_functional_call(parameters, expected_num_params(graph_gate_arrays_().first)); + mutation_started = true; + // Each partition allreduces internally, so every partition returns the global value; take partition 0. + return map_partitions_([&](MonomialPropagator &s) { return s.expectation_value(parameters); })[0]; + }); } + // The functional carries the entry checks and the failure policy for both steps. return expectation_value_functional(std::nullopt)(parameters); } template auto MonomialPropagator::expectation_value_and_gradient(const VecD ¶meters) -> std::pair { if (partition_group_) { - // As in expectation_value(): the gradient is allreduced inside each partition. - return map_partitions_([&](MonomialPropagator &s) { return s.expectation_value_and_gradient(parameters); })[0]; + return run_operation_(true, [&](bool &mutation_started) -> std::pair { + validate_functional_call(parameters, expected_num_params(graph_gate_arrays_().first)); + mutation_started = true; + // As in expectation_value(): the gradient is allreduced inside each partition. + return map_partitions_( + [&](MonomialPropagator &s) { return s.expectation_value_and_gradient(parameters); })[0]; + }); } return expectation_value_and_gradient_functional(std::nullopt)(parameters); } template auto MonomialPropagator::contract_partially(const VecD ¶meters, bool inplace) -> VecD { - if (partition_group_) { - return concat_partitions_([&](MonomialPropagator &s) { return s.contract_partially(parameters, inplace); }); - } - const auto gate_arrays = graph_gate_arrays_(); - const auto ¶meter_mapping = gate_arrays.first; - const auto &gen_coeffs = gate_arrays.second; - validate_parameters_length(parameters, parameter_mapping); + return run_operation_(true, [&](bool &mutation_started) -> VecD { + const auto gate_arrays = graph_gate_arrays_(); + const auto ¶meter_mapping = gate_arrays.first; + const auto &gen_coeffs = gate_arrays.second; + validate_parameters_length(parameters, parameter_mapping); + // Past validation every path reads lazily built caches or consumes the graph. + mutation_started = true; + if (partition_group_) { + return concat_partitions_([&](MonomialPropagator &s) { return s.contract_partially(parameters, inplace); }); + } - if (parameters.empty()) { - return current_picture_coeffs_(); - } + if (parameters.empty()) { + return current_picture_coeffs_(); + } - const size_t num_majoranas = parameter_mapping.size(); - // Inplace slicing produces an owned MPGraph that must be bound to a named local before viewing - // (never view a temporary); slice_view() views this graph's still-live layers directly. - if (schrodinger_) { - const auto &state = mp_op_.dense_state(); - const auto mapped_params = map_params(parameters, parameter_mapping, gen_coeffs, -1.0); - VecD evolved_state; + const size_t num_majoranas = parameter_mapping.size(); + // Inplace slicing produces an owned MPGraph that must be bound to a named local before viewing + // (never view a temporary); slice_view() views this graph's still-live layers directly. + if (schrodinger_) { + const auto &state = mp_op_.dense_state(); + const auto mapped_params = map_params(parameters, parameter_mapping, gen_coeffs, -1.0); + VecD evolved_state; + if (inplace) { + const MPGraph sliced = graph_.slice_graph(num_majoranas, true); + evolved_state = evolve_operator_with_recompute_(VecD(state), sliced.replay_view(), mapped_params); + mp_op_.state_coeffs = evolved_state; + } + else { + evolved_state = + evolve_operator_with_recompute_(VecD(state), graph_.slice_view(num_majoranas), mapped_params); + } + return evolved_state; + } + + const auto &op = mp_op_.get_operator(); + const auto mapped_params = map_params(parameters, parameter_mapping, gen_coeffs, 1.0, true); + VecD evolved_op; if (inplace) { const MPGraph sliced = graph_.slice_graph(num_majoranas, true); - evolved_state = evolve_operator_with_recompute_(VecD(state), sliced.replay_view(), mapped_params); - mp_op_.state_coeffs = evolved_state; + evolved_op = evolve_operator_with_recompute_(VecD(op), sliced.replay_view(), mapped_params); + mp_op_.op_coeffs = evolved_op; } else { - evolved_state = - evolve_operator_with_recompute_(VecD(state), graph_.slice_view(num_majoranas), mapped_params); + evolved_op = evolve_operator_with_recompute_(VecD(op), graph_.slice_view(num_majoranas), mapped_params); } - return evolved_state; - } - - const auto &op = mp_op_.get_operator(); - const auto mapped_params = map_params(parameters, parameter_mapping, gen_coeffs, 1.0, true); - VecD evolved_op; - if (inplace) { - const MPGraph sliced = graph_.slice_graph(num_majoranas, true); - evolved_op = evolve_operator_with_recompute_(VecD(op), sliced.replay_view(), mapped_params); - mp_op_.op_coeffs = evolved_op; - } - else { - evolved_op = evolve_operator_with_recompute_(VecD(op), graph_.slice_view(num_majoranas), mapped_params); - } - return evolved_op; + return evolved_op; + }); } template @@ -1214,10 +1395,15 @@ auto MonomialPropagator::evolved_operator_terms(const VecD ¶meters }); return terms; }; - if (!partition_group_) { - return collect(*this); - } - return concat_partitions_(collect); + return run_operation_(true, [&](bool &mutation_started) -> std::vector { + // Checked here as well as in contract_partially, so that a rejected argument leaves this object usable. + validate_parameters_length(parameters, graph_gate_arrays_().first); + mutation_started = true; + if (!partition_group_) { + return collect(*this); + } + return concat_partitions_(collect); + }); } } // namespace monoprop diff --git a/cpp/monoprop/detail/mpi/CMakeLists.txt b/cpp/monoprop/detail/mpi/CMakeLists.txt index 9128c8ef..13d82bcb 100644 --- a/cpp/monoprop/detail/mpi/CMakeLists.txt +++ b/cpp/monoprop/detail/mpi/CMakeLists.txt @@ -11,6 +11,7 @@ target_sources( "HybridComm.h" "MPICompat.h" "MPIUtils.h" + "OperationFailure.h" "Pairwise.h" "PartitionBarrier.h" "Routing.h" diff --git a/cpp/monoprop/detail/mpi/MPICompat.cpp b/cpp/monoprop/detail/mpi/MPICompat.cpp index 0ed8cbcb..43a8bdaf 100644 --- a/cpp/monoprop/detail/mpi/MPICompat.cpp +++ b/cpp/monoprop/detail/mpi/MPICompat.cpp @@ -16,6 +16,8 @@ #include #include +#include +#include #include #include #include @@ -31,17 +33,22 @@ namespace monoprop::mpi { auto init(int *argc, char ***argv) -> void { auto initialized = 0; MPI_Initialized(&initialized); + int provided = MPI_THREAD_SINGLE; if (!initialized) { // serialized (not funneled): under the hybrid the one-at-a-time MPI calls come from each rank's // partition-0 master, not the main thread. mpi4py already requests >= serialized. - auto required = MPI_THREAD_SERIALIZED; - auto provided = 0; - MPI_Init_thread(argc, argv, required, &provided); - if (provided < required) { - std::print("Sorry, the MPI library does not provide MPI_THREAD_SERIALIZED support, which is required " - "by the partition/MPI hybrid transport.\n"); - MPI_Abort(MPI_COMM_WORLD, 1); - } + MPI_Init_thread(argc, argv, kRequiredThreadLevel, &provided); + } + else { + // Host-initialized: never reinitialize, and never assume the host's request was granted. + MPI_Query_thread(&provided); + } + if (!thread_level_satisfies(provided, kRequiredThreadLevel)) { + std::print("Sorry, the MPI library provides {} but monoprop requires {}, which the partition/MPI hybrid " + "transport needs.\n", + thread_level_name(provided), + thread_level_name(kRequiredThreadLevel)); + MPI_Abort(MPI_COMM_WORLD, 1); } } @@ -54,6 +61,51 @@ auto finalize() -> void { } #endif // monoprop_ENABLE_MPI +auto require_thread_support() -> void { +#ifdef monoprop_ENABLE_MPI + int initialized = 0; + MPI_Initialized(&initialized); + if (initialized == 0) { + return; + } + int provided = MPI_THREAD_SINGLE; + MPI_Query_thread(&provided); + if (!thread_level_satisfies(provided, kRequiredThreadLevel)) { + throw MpiThreadLevelUnsupported( + std::format("monoprop requires {} support from MPI, but the initialized MPI library provides {}. " + "Initialize MPI with MPI_Init_thread requesting {} or MPI_THREAD_MULTIPLE (mpi4py requests " + "MPI_THREAD_MULTIPLE by default).", + thread_level_name(kRequiredThreadLevel), + thread_level_name(provided), + thread_level_name(kRequiredThreadLevel))); + } +#endif +} + +auto require_initializing_thread() -> void { +#ifdef monoprop_ENABLE_MPI + // MPI_Initialized, MPI_Finalized and MPI_Is_thread_main may be called from any thread; nothing else is. + int initialized = 0; + int finalized = 0; + MPI_Initialized(&initialized); + MPI_Finalized(&finalized); + if (initialized == 0 || finalized != 0) { + return; + } + int is_main = 0; + MPI_Is_thread_main(&is_main); + if (is_main == 0) { + std::fputs("monoprop: an operation that uses MPI was called from a thread other than the thread that " + "initialized MPI. monoprop requires every such call to come from the thread that initialized " + "MPI, whatever thread support MPI provides. This process stops here without communicating; " + "the launcher ends its peers.\n", + stderr); + std::fflush(stderr); + std::abort(); + } +#endif +} + auto rank(const Comm &comm) -> int { if (comm.kind == Comm::Kind::Shm) { return comm.shm_rank; diff --git a/cpp/monoprop/detail/mpi/MPICompat.h b/cpp/monoprop/detail/mpi/MPICompat.h index b72a7c1a..46a48455 100644 --- a/cpp/monoprop/detail/mpi/MPICompat.h +++ b/cpp/monoprop/detail/mpi/MPICompat.h @@ -43,6 +43,35 @@ namespace monoprop::mpi { #ifdef monoprop_ENABLE_MPI +/// The MPI thread support monoprop requests and requires. SERIALIZED, not FUNNELED, while the legacy +/// Hybrid transport still calls MPI from its partition-0 workers. +inline constexpr int kRequiredThreadLevel = MPI_THREAD_SERIALIZED; + +/// Whether \a provided thread support satisfies \a required; MPI orders the levels +/// SINGLE < FUNNELED < SERIALIZED < MULTIPLE. A higher level never relaxes the initializing-thread rule. +constexpr auto thread_level_satisfies(int provided, int required) noexcept -> bool { + return provided >= required; +} + +/// The MPI constant naming a thread support level, for diagnostics. +constexpr auto thread_level_name(int level) noexcept -> const char * { + if (level == MPI_THREAD_SINGLE) { + return "MPI_THREAD_SINGLE"; + } + if (level == MPI_THREAD_FUNNELED) { + return "MPI_THREAD_FUNNELED"; + } + if (level == MPI_THREAD_SERIALIZED) { + return "MPI_THREAD_SERIALIZED"; + } + if (level == MPI_THREAD_MULTIPLE) { + return "MPI_THREAD_MULTIPLE"; + } + return "an unknown MPI thread level"; +} + +/// Initialize MPI at kRequiredThreadLevel unless the host already did, then check the level actually +/// provided in either case, aborting if it is too low. Host-initialized MPI is never reinitialized. monoprop_EXPORT auto init(int *argc = nullptr, char ***argv = nullptr) -> void; monoprop_EXPORT auto finalize() -> void; @@ -90,6 +119,18 @@ inline auto init(int * /*argc*/ = nullptr, char *** /*argv*/ = nullptr) -> void inline auto finalize() -> void {} #endif // monoprop_ENABLE_MPI +/// Check that initialized MPI provides at least kRequiredThreadLevel, whoever initialized it. +/// \throws MpiThreadLevelUnsupported if it does not. A no-op without MPI or before MPI is initialized. +monoprop_EXPORT auto require_thread_support() -> void; + +/// Fail fast unless the caller is the thread that initialized MPI. +/// +/// A call from any other thread breaks the host contract, whatever thread support MPI provides: it prints a +/// diagnostic and calls std::abort without calling MPI_Abort or any other communication, so the launcher +/// must end the peers. A no-op without MPI or while MPI is not initialized. Guard the one-store path only; +/// the legacy Hybrid transport legitimately calls MPI from its partition workers. +monoprop_EXPORT auto require_initializing_thread() -> void; + monoprop_EXPORT auto rank(const Comm &comm) -> int; monoprop_EXPORT auto size(const Comm &comm) -> int; diff --git a/cpp/monoprop/detail/mpi/OperationFailure.h b/cpp/monoprop/detail/mpi/OperationFailure.h new file mode 100644 index 00000000..db86a52b --- /dev/null +++ b/cpp/monoprop/detail/mpi/OperationFailure.h @@ -0,0 +1,113 @@ +// Copyright 2026 Algorithmiq +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +#pragma once + +#include +#include +#include +#include +#include + +#include "monoprop/detail/mpi/Comm.h" + +namespace monoprop::mpi { + +/*! + * \brief The message carried by \a error, for a diagnostic. + * + * Never throws; an exception that is not a `std::exception` reads as "unknown exception". + */ +inline auto describe_exception(const std::exception_ptr &error) noexcept -> std::string { + try { + std::rethrow_exception(error); + } + catch (const std::exception &e) { + try { + return e.what(); + } + catch (...) { + return {}; + } + } + catch (...) { + return "unknown exception"; + } +} + +/*! + * \brief Handle a failure on the calling rank during a distributed operation, from the controlling thread. + * + * On an ordinary MPI communicator with more than one rank, the peers may already be committed to a + * communication this rank will never join, and no exception can release them. So this reports the rank + * and the underlying error on stderr and calls `MPI_Abort` on \a comm; if `MPI_Abort` returns, it calls + * `std::terminate`. On one rank, without MPI, or on a legacy in-process communicator (whose partition + * runtime poisons its own transport), it rethrows \a error unchanged, preserving the original type. + * + * Callers catch failures where they happen, inside the lifetime of any posted request handle + * (mpi::Ticket, mpi::PendingAlltoallv), so that unwinding never drains requests peers will not complete. + * Never call it from an OpenMP worker or from a thread other than MPI's initializing thread. + * + * \param comm The communicator the operation runs on. + * \param error The failure; must not be null. + */ +[[noreturn]] inline auto operation_failed(const Comm &comm, std::exception_ptr error) -> void { +#ifdef monoprop_ENABLE_MPI + if (comm.kind == Comm::Kind::Mpi) { + int initialized = 0; + int finalized = 0; + MPI_Initialized(&initialized); + MPI_Finalized(&finalized); + int size = 1; + if (initialized != 0 && finalized == 0 && MPI_Comm_size(comm.mpi, &size) == MPI_SUCCESS && size > 1) { + int rank = 0; + MPI_Comm_rank(comm.mpi, &rank); + const auto line = + std::format("monoprop: rank {} of {}: a distributed operation failed on this rank: {}. Its peers may " + "already be waiting on communication this rank will not complete, so monoprop is " + "aborting the communicator rather than leaving them blocked.\n", + rank, + size, + describe_exception(error)); + std::fputs(line.c_str(), stderr); + std::fflush(stderr); + MPI_Abort(comm.mpi, 1); + std::terminate(); + } + } +#else + (void)comm; +#endif + std::rethrow_exception(std::move(error)); +} + +/*! + * \brief Run \a fn and send any exception it throws to operation_failed(). + * + * Wrap each phase that can throw while peers may be committed to communication, and place the wrapper + * inside the lifetime of any request handle the phase overlaps. + * + * \return Whatever \a fn returns. + */ +template +auto guard_distributed(const Comm &comm, Fn &&fn) -> decltype(std::forward(fn)()) { + try { + return std::forward(fn)(); + } + catch (...) { + operation_failed(comm, std::current_exception()); + } +} + +} // namespace monoprop::mpi diff --git a/cpp/monoprop/detail/parallel/CMakeLists.txt b/cpp/monoprop/detail/parallel/CMakeLists.txt index 2237ed3d..0bf96603 100644 --- a/cpp/monoprop/detail/parallel/CMakeLists.txt +++ b/cpp/monoprop/detail/parallel/CMakeLists.txt @@ -5,5 +5,8 @@ target_sources( TYPE HEADERS FILES "Options.h" + "ThreadBudget.h" "Workshare.h" ) + +target_sources(monoprop-objs PRIVATE ThreadBudget.cpp) diff --git a/cpp/monoprop/detail/parallel/ThreadBudget.cpp b/cpp/monoprop/detail/parallel/ThreadBudget.cpp new file mode 100644 index 00000000..341d1f49 --- /dev/null +++ b/cpp/monoprop/detail/parallel/ThreadBudget.cpp @@ -0,0 +1,76 @@ +// Copyright 2026 Algorithmiq +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +#include "monoprop/detail/parallel/ThreadBudget.h" + +#include + +#include +#include +#include +#include +#include +#include +#include +#include + +namespace monoprop::detail::parallel { +namespace { + +constexpr const char *kVariable = "monoprop_NUM_THREADS"; + +[[noreturn]] auto reject(std::string_view text) -> void { + throw std::invalid_argument(std::format("{}=\"{}\" is not a thread count: it must be a decimal integer from 1 " + "to {} with no sign or spaces. Set it to the threads per process, or " + "unset it to use the OpenMP runtime default.", + kVariable, + text, + INT_MAX)); +} + +} // namespace + +auto resolve_thread_budget(std::optional configured, int runtime_default) -> Options { + if (!configured.has_value()) { + if (runtime_default < 1) { + throw std::invalid_argument(std::format("{} is unset and the OpenMP runtime default thread count is {}, " + "not positive; set {} explicitly.", + kVariable, + runtime_default, + kVariable)); + } + return {.threads = runtime_default}; + } + const std::string_view text = *configured; + // from_chars alone would accept a partial parse; requiring digits throughout also rejects signs, + // whitespace and lists before any narrowing. + if (text.empty() || !std::ranges::all_of(text, [](char c) { return c >= '0' && c <= '9'; })) { + reject(text); + } + std::uint64_t value = 0; + const auto [end, ec] = std::from_chars(text.data(), text.data() + text.size(), value, 10); + if (ec != std::errc{} || end != text.data() + text.size() || value < 1 + || value > static_cast(INT_MAX)) { + reject(text); + } + return {.threads = static_cast(value)}; +} + +auto capture_thread_budget() -> Options { + const char *configured = std::getenv(kVariable); + return resolve_thread_budget(configured != nullptr ? std::optional(configured) : std::nullopt, + omp_get_max_threads()); +} + +} // namespace monoprop::detail::parallel diff --git a/cpp/monoprop/detail/parallel/ThreadBudget.h b/cpp/monoprop/detail/parallel/ThreadBudget.h new file mode 100644 index 00000000..5236459b --- /dev/null +++ b/cpp/monoprop/detail/parallel/ThreadBudget.h @@ -0,0 +1,54 @@ +// Copyright 2026 Algorithmiq +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +#pragma once + +#include +#include + +#include "monoprop/detail/parallel/Options.h" +#include "monoprop/monopropExport.h" + +namespace monoprop::detail::parallel { + +/*! + * \brief Resolve a per-object thread budget from a configured value and the OpenMP runtime default. + * + * Pure: no environment read, hardware query or runtime mutation. The value is never clamped to the + * hardware or the allocation. + * + * \param configured The `monoprop_NUM_THREADS` text, or `std::nullopt` when the variable is unset. A + * present value must be nonempty ASCII decimal digits whose value lies in + * `[1, INT_MAX]`; leading zeros are accepted. A valid value wins over any fallback. + * \param runtime_default The OpenMP default (`omp_get_max_threads()`), used only when `configured` is + * absent; it must then be positive. + * \return The resolved budget. + * \throws std::invalid_argument for an empty, signed, padded, list-valued, non-digit, zero or overflowing + * configured value, or for an absent value with a non-positive runtime default. + */ +monoprop_EXPORT auto resolve_thread_budget(std::optional configured, int runtime_default) -> Options; + +/*! + * \brief Read `monoprop_NUM_THREADS` once and resolve it against `omp_get_max_threads()`. + * + * Called while constructing a propagator, never during an operation; each call rereads the + * environment and nothing is cached. An unset variable selects the OpenMP runtime default, while a + * present but empty one is an error. + * + * \return The resolved budget. + * \throws std::invalid_argument as resolve_thread_budget() does. + */ +monoprop_EXPORT auto capture_thread_budget() -> Options; + +} // namespace monoprop::detail::parallel diff --git a/cpp/tests/CMakeLists.txt b/cpp/tests/CMakeLists.txt index 50d224e7..6c3f83be 100644 --- a/cpp/tests/CMakeLists.txt +++ b/cpp/tests/CMakeLists.txt @@ -23,6 +23,8 @@ file( CONFIGURE_DEPENDS "${CMAKE_CURRENT_LIST_DIR}/*.cpp" ) +# Its own main() ends every scenario by aborting the job, so it stays out of the unit runner. +list(FILTER _tests_cpps EXCLUDE REGEX "/mpi_failure_driver\\.cpp$") add_executable(monoprop_unit_tests.x ${_tests_cpps}) @@ -84,4 +86,142 @@ discover_tests( ${_monoprop_serial_env_entries} ) +# Fresh processes for the construction-time thread budget: each entry reruns one unit case under its own +# launch environment, since an object captures monoprop_NUM_THREADS once, when it is constructed. +function(monoprop_add_launch_env_case NAME CASE) + cmake_parse_arguments("" "" "" "ENVIRONMENT;ENVIRONMENT_MODIFICATION" ${ARGN}) + add_test( + NAME ${NAME} + COMMAND + monoprop_unit_tests.x --run_test=${CASE} + ) + set( + _environment + ${_monoprop_serial_env_entries} + ${_ENVIRONMENT} + ) + set_tests_properties( + ${NAME} + PROPERTIES + LABELS + "unit;openmp_env" + ENVIRONMENT + "${_environment}" + ) + if(_ENVIRONMENT_MODIFICATION) + set_tests_properties( + ${NAME} + PROPERTIES + ENVIRONMENT_MODIFICATION + "${_ENVIRONMENT_MODIFICATION}" + ) + endif() +endfunction() + +monoprop_add_launch_env_case( + openmp_env_prototype_budget_1 + openmp_prototype_budget_matches_the_launch_environment + ENVIRONMENT "monoprop_NUM_THREADS=1;monoprop_TEST_EXPECT_BUDGET=1" +) +monoprop_add_launch_env_case( + openmp_env_prototype_budget_3 + openmp_prototype_budget_matches_the_launch_environment + ENVIRONMENT "monoprop_NUM_THREADS=3;monoprop_TEST_EXPECT_BUDGET=3" +) +# Unset library override: the OpenMP runtime default decides. +monoprop_add_launch_env_case( + openmp_env_prototype_budget_openmp_default_3 + openmp_prototype_budget_matches_the_launch_environment + ENVIRONMENT "OMP_NUM_THREADS=3;monoprop_TEST_EXPECT_BUDGET=3" + ENVIRONMENT_MODIFICATION "monoprop_NUM_THREADS=unset:" +) +monoprop_add_launch_env_case( + openmp_env_runtime_thread_limit_2 + openmp_runtime_limited_team_keeps_the_captured_budget + ENVIRONMENT "monoprop_NUM_THREADS=3;OMP_THREAD_LIMIT=2;monoprop_TEST_EXPECT_BUDGET=3" +) +monoprop_add_launch_env_case( + openmp_env_no_affinity_change + openmp_propagator_work_does_not_change_affinity + ENVIRONMENT "OMP_PROC_BIND=false" +) + +# Distributed failure scenarios, each a separate two-rank launch with a 30 s timeout. Labelled mpi, so +# only the MPI test leg runs them. +if(monoprop_ENABLE_MPI AND _monoprop_mpiexec) + add_executable(monoprop_mpi_failure_driver.x mpi_failure_driver.cpp) + target_include_directories( + monoprop_mpi_failure_driver.x + PRIVATE + ${CMAKE_CURRENT_LIST_DIR} + ${CMAKE_CURRENT_BINARY_DIR} + ) + target_link_libraries( + monoprop_mpi_failure_driver.x + PRIVATE + monoprop-sanitizers + monoprop-objs + PkgConfig::HWLOC + ) + + set(_monoprop_abort_diagnostic "aborting the communicator") + function( + monoprop_add_failure_scenario + NAME + SCENARIO + OPTION + EXPECT + FORBID + ) + add_test( + NAME mpi_failure_${NAME} + COMMAND + ${CMAKE_COMMAND} "-DMPIEXEC=${_monoprop_mpiexec}" + "-DNUMPROC_FLAG=${_monoprop_mpiexec_numproc_flag}" + "-DPREFLAGS=${MPIEXEC_PREFLAGS}" + "-DDRIVER=$" + "-DSCENARIO=${SCENARIO}" "-DOPTION=${OPTION}" "-DEXPECT=${EXPECT}" + "-DFORBID=${FORBID}" -DTIMEOUT_SECONDS=30 -P + "${CMAKE_CURRENT_LIST_DIR}/run_mpi_failure_scenario.cmake" + ) + set_tests_properties( + mpi_failure_${NAME} + PROPERTIES + LABELS + "mpi;mpi_failure" + TIMEOUT + 90 + SKIP_RETURN_CODE + 77 + # As for the suite's MPI variants: CI containers may run as root. + ENVIRONMENT + "OMPI_ALLOW_RUN_AS_ROOT=1;OMPI_ALLOW_RUN_AS_ROOT_CONFIRM=1" + ) + endfunction() + + monoprop_add_failure_scenario(worker_throw worker-throw "" "injected worker failure" "UNGUARDED") + monoprop_add_failure_scenario( + before_exchange before-exchange "" "injected failure in the scan" "UNGUARDED|NOT REACHED" + ) + monoprop_add_failure_scenario( + active_ticket active-ticket ticket "injected failure while the replay Ticket" "UNGUARDED" + ) + monoprop_add_failure_scenario( + active_pending active-ticket pending "injected failure while a PendingAlltoallv" "UNGUARDED" + ) + monoprop_add_failure_scenario( + insufficient_thread_level_single insufficient-thread-level single "requires MPI_THREAD_SERIALIZED" "UNGUARDED|despite" + ) + monoprop_add_failure_scenario( + insufficient_thread_level_funneled insufficient-thread-level funneled "requires MPI_THREAD_SERIALIZED" "UNGUARDED|despite" + ) + # Local fail-fast: the offending thread must not call MPI_Abort (or anything else in MPI). + monoprop_add_failure_scenario( + wrong_thread_entry_serialized wrong-thread-entry serialized "thread that initialized MPI" "${_monoprop_abort_diagnostic}|completed" + ) + monoprop_add_failure_scenario( + wrong_thread_entry_multiple wrong-thread-entry multiple "thread that initialized MPI" "${_monoprop_abort_diagnostic}|completed" + ) +endif() + add_subdirectory(link_export_probe) diff --git a/cpp/tests/PropagatorTestAccess.h b/cpp/tests/PropagatorTestAccess.h new file mode 100644 index 00000000..f8f05984 --- /dev/null +++ b/cpp/tests/PropagatorTestAccess.h @@ -0,0 +1,57 @@ +// Copyright 2026 Algorithmiq +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +#pragma once + +// White-box access for tests only. MonomialPropagator befriends this template but the library never +// defines it, so it adds no production API: tests use it to observe the captured thread budget and to +// inject failures at real mutation and evaluation boundaries through existing protected members. + +#include +#include +#include + +#include "monoprop/MonomialPropagator.h" +#include "monoprop/detail/parallel/Options.h" + +namespace monoprop::detail { + +template +struct PropagatorTestAccess { + using Propagator = MonomialPropagator; + + static auto options(const Propagator &p) -> parallel::Options { return p.parallel_; } + + static auto is_invalid(const Propagator &p) -> bool { return p.invalid_; } + + // The existing protected cutoff predicate: every scan evaluates it, so a throwing one fails a real + // build after the operation has started mutating. + static auto set_cutoff_fn(Propagator &p, CutoffFn fn) -> void { p.cutoff_fn_ = std::move(fn); } + + // The existing protected functional factory, with a caller-supplied evaluation body. + template + static auto make_functional(Propagator &p, Fn &&fn, std::optional pare_threshold = std::nullopt) { + return p.make_functional_(std::forward(fn), pare_threshold); + } + + static auto clone(const Propagator &p) -> std::unique_ptr { return p.clone_(); } + + static auto partition_count(const Propagator &p) -> int { + return p.partition_group_ ? p.partition_group_->partition_count() : 0; + } + + static auto partition(const Propagator &p, int r) -> const Propagator & { return p.partition_group_->partition(r); } +}; + +} // namespace monoprop::detail diff --git a/cpp/tests/README.md b/cpp/tests/README.md index 7b67d3eb..f3dd71ca 100644 --- a/cpp/tests/README.md +++ b/cpp/tests/README.md @@ -76,6 +76,10 @@ name and cannot address suite-nested cases, tests use flat - **`dense_query_reference.h`**: the retired dense query record, frozen as the independent oracle for `sparse_query_tests.cpp`. Test-only, and not kept in sync with the wire format. +- **`PropagatorTestAccess.h`**: white-box access that `MonomialPropagator` befriends but the library + never defines. Tests use it to read an object's captured thread budget and validity, and to + inject failures through existing protected members: a throwing cutoff predicate, or a custom + functional body. - **`TestData.{h,cpp}`**: the `CaseData` struct and msgpack fixture loader. - **`boost-test.cmake` / `boostAddTests.cmake`**: CMake test discovery. @@ -128,9 +132,20 @@ name and cannot address suite-nested cases, tests use flat participation, which is skipped when the runtime cannot provide two workers). Run it in fresh processes under `OMP_DYNAMIC=TRUE`, `OMP_DYNAMIC=FALSE` and `OMP_THREAD_LIMIT=1` when changing the helper. - -New `*.cpp` files are auto-discovered on the next configure — no CMake edit -needed. +- **Thread budgets and failed-owner rule**: `openmp_runtime_tests.cpp` covers the budget parser and + its capture, per-object budgets through copies and retained functionals, and legacy paths that + stay serial. It also checks that OpenMP settings and affinity are unchanged, that old low-level + calls default to serial options, and that a failed mutation or evaluation invalidates the owner + while validation errors do not. `cpp/tests/CMakeLists.txt` reruns some of these cases as + `openmp_env_*` entries, each in a fresh process with a fixed launch environment. +- **Distributed failure driver**: `mpi_failure_driver.cpp` is not part of the unit runner (the glob + excludes it). The MPI build compiles it into `monoprop_mpi_failure_driver.x`, and + `run_mpi_failure_scenario.cmake` runs each scenario on two ranks with a 30 s timeout, as the + `mpi_failure_*` CTest entries. + +New `*.cpp` files are auto-discovered on the next configure, with no CMake edit +needed. A file with its own `main()` must be excluded from the glob, as +`mpi_failure_driver.cpp` is. ## MPI Test Configuration diff --git a/cpp/tests/link_export_probe/link_export_probe.cpp b/cpp/tests/link_export_probe/link_export_probe.cpp index 9864eb7b..ea0f1b9c 100644 --- a/cpp/tests/link_export_probe/link_export_probe.cpp +++ b/cpp/tests/link_export_probe/link_export_probe.cpp @@ -29,10 +29,14 @@ // It also checks the installed usage requirements a consumer inherits: // (c) the OpenMP worksharing template (detail/parallel/Workshare.h), instantiated in this translation // unit, so OpenMP compile and link flags must reach the consumer through the imported target alone; -// (d) the Pauli basis, alongside the Majorana graph chain in (a). +// (d) the Pauli basis, alongside the Majorana graph chain in (a); +// (e) the one-store prototype (partitions = 1), whose constructor template calls the out-of-line +// capture_thread_budget(), require_thread_support() and require_initializing_thread(), and whose +// operations reach the exported evolve/derivative entry points with their trailing Options. #include "monoprop/MonomialPropagator.h" #include "monoprop/detail/mpi/MPICompat.h" +#include "monoprop/detail/parallel/ThreadBudget.h" #include "monoprop/detail/parallel/Workshare.h" #include @@ -153,6 +157,32 @@ auto run_partition_chain() -> void { std::println(stderr, "[link_export_probe] partition chain: size={}", sim.size()); } +auto run_one_store_prototype_chain() -> void { + constexpr size_t kModes = 2; + OperatorDict ham; + ham[VecZ{0, 1}] = std::complex{0.0, 1.0}; + MonomialPropagator sim(ham, + 2 * kModes, + VecZ{0, 1}, + /*schrodinger_cutoff=*/std::nullopt, + MPI_COMM_SELF, + /*lower_atol=*/std::nullopt, + /*upper_atol=*/std::nullopt, + CutoffType::Length, + /*basis_change=*/std::nullopt, + /*logical_num_modes=*/kModes, + Basis::Majorana, + /*partitions=*/1); + sim.build_graph(std::vector{{0, 2}, {1, 3}}, VecZ{0, 1}, VecD{1.0, 1.0}); + const auto [value, grad] = sim.expectation_value_and_gradient(VecD{0.1, 0.2}); + const auto budget = monoprop::detail::parallel::capture_thread_budget(); + std::println(stderr, + "[link_export_probe] one-store prototype chain: budget={} value={} grad_size={}", + budget.threads, + value, + grad.size()); +} + } // namespace auto main() -> int { @@ -161,6 +191,7 @@ auto main() -> int { run_partition_chain(); run_openmp_workshare_chain(); run_pauli_chain(); + run_one_store_prototype_chain(); monoprop::mpi::finalize(); std::println(stderr, "[link_export_probe] OK"); return 0; diff --git a/cpp/tests/mpi_failure_driver.cpp b/cpp/tests/mpi_failure_driver.cpp new file mode 100644 index 00000000..326b00c3 --- /dev/null +++ b/cpp/tests/mpi_failure_driver.cpp @@ -0,0 +1,292 @@ +// Copyright 2026 Algorithmiq +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +// Launcher-supervised distributed failure scenarios. Its own main(), outside the unit runner: every +// scenario ends the whole job. run_mpi_failure_scenario.cmake launches it on two ranks with a 30 s +// timeout and requires a prompt nonzero exit carrying the expected diagnostic. +// +// Rank 0 fails while its peers follow the normal phase. A failure the library does not handle escapes to +// main(), which models a host that catches it and shuts down: rank 0 finalizes while its peers are still +// inside the operation, so the job hangs until the timeout. Every scenario therefore passes only through +// the library's own abort or fail-fast path. +// +// worker-throw worksharing worker throws inside a retained-functional evaluation +// before-exchange a build fails in the scan, before the first query exchange +// active-ticket ticket|pending a failure while a replay Ticket / construction PendingAlltoallv is posted +// insufficient-thread-level single|funneled host-initialized MPI below the required level +// wrong-thread-entry serialized|multiple a guarded entry called from a non-initializing std::thread + +#include +#include + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include "PropagatorTestAccess.h" +#include "monoprop/Evolution.h" +#include "monoprop/MPFunctions.h" +#include "monoprop/MonomialPropagator.h" +#include "monoprop/detail/mpi/MPICompat.h" +#include "monoprop/detail/mpi/OperationFailure.h" +#include "monoprop/detail/parallel/Workshare.h" + +namespace { + +using namespace monoprop; + +constexpr size_t kModes = 6; +using Propagator = MonomialPropagator; +using Access = detail::PropagatorTestAccess; + +// Exit code CTest reads as "skipped": the MPI library gave more support than the scenario needs. +constexpr int kNotExercised = 77; + +auto say(int rank, std::string_view text) -> void { + std::fputs(std::format("[driver] rank {}: {}\n", rank, text).c_str(), stderr); + std::fflush(stderr); +} + +auto world_rank() -> int { + int rank = 0; + MPI_Comm_rank(MPI_COMM_WORLD, &rank); + return rank; +} + +// Twenty terms anticommuting with the {0, 1} generator, so both ranks own scan work under any hash split. +auto hamiltonian() -> OperatorDict { + OperatorDict ham; + for (size_t j = 2; j < 2 * kModes; ++j) { + ham[VecZ{0, j}] = std::complex{0.0, 0.1 * static_cast(j)}; + ham[VecZ{1, j}] = std::complex{0.0, 0.05 * static_cast(j)}; + } + return ham; +} + +auto make_prototype() -> Propagator { + return Propagator(hamiltonian(), + 2 * kModes, + VecZ{0, 1}, + std::nullopt, + MPI_COMM_WORLD, + std::nullopt, + std::nullopt, + CutoffType::Length, + std::nullopt, + kModes, + Basis::Majorana, + /*partitions=*/1); +} + +const std::vector kGates{{0, 1}, {2, 3}, {1, 4}}; +const VecZ kMapping{0, 1, 2}; +const VecD kGenCoeffs{1.0, 1.0, 1.0}; +const VecD kParams{0.3, 0.2, 0.1}; + +auto build(Propagator &sim) -> void { + sim.build_graph(kGates, kMapping, kGenCoeffs); +} + +auto worker_throw(int rank) -> void { + auto sim = make_prototype(); + build(sim); + auto functional = Access::make_functional( + sim, + [rank](const EvalRequest &request, mpi::Comm comm, const detail::CosCallbacks &cos) -> double { + if (rank == 0) { + detail::parallel::for_blocks(4, {.threads = 2}, [](size_t block) { + if (block == 3) { + throw std::runtime_error( + std::format("injected worker failure on OpenMP worker {}", omp_get_thread_num())); + } + }); + } + return ev(request, comm, cos); + }); + (void)functional(kParams); +} + +auto before_exchange(int rank) -> void { + auto sim = make_prototype(); + if (rank == 0) { + Access::set_cutoff_fn(sim, [](const Monomial &) -> bool { + throw std::runtime_error("injected failure in the scan, before the query exchange"); + }); + } + build(sim); + if (rank == 0) { + say(rank, "INJECTION NOT REACHED: the scan never evaluated the cutoff"); + } +} + +auto active_ticket(int rank) -> void { + auto sim = make_prototype(); + build(sim); + VecD coeffs = sim.mp_op().get_operator(); + const auto view = sim.graph().replay_view(); + // Splitmix replays through MPI_Ialltoallv, so the Ticket always holds a posted request here. + const detail::LayerCosScale scale = [rank](size_t, double *, double) { + if (rank == 0) { + throw std::runtime_error("injected failure while the replay Ticket is in flight"); + } + }; + say(rank, "posting the replay exchange"); + evolve_step(coeffs, view, 0.3, 0, MPI_COMM_WORLD, scale); +} + +auto active_pending(int rank, int size) -> void { + const mpi::SlotWindow window{.base = 0, .count = static_cast(size)}; + mpi::WindowVec> send(window); + for (const auto wi : window.indices()) { + send[wi].assign(4, static_cast(rank)); + } + auto pending = mpi::begin_alltoallv(send, MPI_COMM_WORLD); + say(rank, std::format("PendingAlltoallv posted {} request(s)", pending.posted)); + if (rank == 0) { + mpi::guard_distributed(MPI_COMM_WORLD, [] { + throw std::runtime_error("injected failure while a PendingAlltoallv is in flight"); + }); + } + mpi::WindowVec> recv; + pending.wait_into(recv); +} + +auto insufficient_level(int rank) -> void { + // The constructor validates the provided level before it communicates. + const auto sim = make_prototype(); + say(rank, std::format("constructed with {} terms despite the insufficient level", sim.size())); +} + +auto wrong_thread(int rank) -> void { + auto sim = make_prototype(); + build(sim); + if (rank == 0) { + std::thread host([&sim] { (void)sim.expectation_value(kParams); }); + host.join(); + } + else { + (void)sim.expectation_value(kParams); + } +} + +auto parse_level(std::string_view name) -> std::optional { + if (name == "single") { + return MPI_THREAD_SINGLE; + } + if (name == "funneled") { + return MPI_THREAD_FUNNELED; + } + if (name == "serialized") { + return MPI_THREAD_SERIALIZED; + } + if (name == "multiple") { + return MPI_THREAD_MULTIPLE; + } + return std::nullopt; +} + +} // namespace + +auto main(int argc, char **argv) -> int { + if (argc < 2) { + std::fputs("usage: monoprop_mpi_failure_driver.x [option]\n", stderr); + return 2; + } + const std::string_view scenario = argv[1]; + const std::string_view option = argc > 2 ? argv[2] : ""; + // Dense collectives on every exchange, so each peer is committed to the round rank 0 abandons. + ::setenv("monoprop_ROUTING", "splitmix", 1); + + const bool host_owned = scenario == "insufficient-thread-level" || scenario == "wrong-thread-entry"; + std::optional requested; + if (host_owned) { + requested = parse_level(option); + if (!requested) { + std::fputs("a host-initialized scenario needs a thread level\n", stderr); + return 2; + } + int provided = MPI_THREAD_SINGLE; + MPI_Init_thread(&argc, &argv, *requested, &provided); + } + else { + mpi::init(&argc, &argv); + } + const int rank = world_rank(); + int size = 1; + MPI_Comm_size(MPI_COMM_WORLD, &size); + int provided = MPI_THREAD_SINGLE; + MPI_Query_thread(&provided); + say(rank, + std::format("scenario={} {} size={} MPI {} requested={} provided={}", + scenario, + option, + size, + host_owned ? "host-initialized" : "library-initialized", + requested ? mpi::thread_level_name(*requested) : mpi::thread_level_name(mpi::kRequiredThreadLevel), + mpi::thread_level_name(provided))); + + if (scenario == "insufficient-thread-level" && mpi::thread_level_satisfies(provided, mpi::kRequiredThreadLevel)) { + say(rank, + std::format("NOT EXERCISED: requested {} but the MPI library provided {}", + mpi::thread_level_name(*requested), + mpi::thread_level_name(provided))); + MPI_Finalize(); + return kNotExercised; + } + + try { + if (scenario == "worker-throw") { + worker_throw(rank); + } + else if (scenario == "before-exchange") { + before_exchange(rank); + } + else if (scenario == "active-ticket" && option == "ticket") { + active_ticket(rank); + } + else if (scenario == "active-ticket" && option == "pending") { + active_pending(rank, size); + } + else if (scenario == "insufficient-thread-level") { + insufficient_level(rank); + } + else if (scenario == "wrong-thread-entry") { + wrong_thread(rank); + } + else { + say(rank, std::format("unknown scenario '{}' '{}'", scenario, option)); + MPI_Abort(MPI_COMM_WORLD, 2); + } + } + catch (const std::exception &e) { + say(rank, std::format("UNGUARDED: an exception escaped the library: {}", e.what())); + // A host that handled the error and shut down; peers still inside the operation hang. + MPI_Finalize(); + return 3; + } + + // The next collective of the peers' normal phase. + MPI_Barrier(MPI_COMM_WORLD); + say(rank, "scenario completed without a failure"); + MPI_Finalize(); + return 0; +} diff --git a/cpp/tests/mpi_utils_tests.cpp b/cpp/tests/mpi_utils_tests.cpp index bba4f5dc..946174e3 100644 --- a/cpp/tests/mpi_utils_tests.cpp +++ b/cpp/tests/mpi_utils_tests.cpp @@ -28,6 +28,7 @@ #include "monoprop/detail/evolution/CutoffContext.h" #include "monoprop/detail/evolution/layer_build/Scan.h" #include "monoprop/detail/mpi/Comm.h" +#include "monoprop/detail/mpi/MPICompat.h" #include "monoprop/detail/mpi/MPIUtils.h" #include "monoprop/detail/mpi/Routing.h" #include "monoprop/detail/operator/MPOperator.h" @@ -320,3 +321,25 @@ BOOST_AUTO_TEST_CASE(mpi_utils_paired_enumeration_partitions_by_find_rank) { } } } + +#ifdef monoprop_ENABLE_MPI +// The transport decision is agreed once per communicator and cached on it as an attribute. Warm calls +// repeat the agreed answer, and a duplicated or freshly created communicator agrees on its own instead of +// inheriting an entry keyed by a recycled raw handle. Every rank runs the same call sequence. +BOOST_AUTO_TEST_CASE(mpi_utils_routes_pairwise_is_cached_per_communicator) { + const bool cold = mpi::routes_pairwise(MPI_COMM_WORLD); + BOOST_TEST(mpi::routes_pairwise(MPI_COMM_WORLD) == cold); + for (int round = 0; round < 2; ++round) { + MPI_Comm dup = MPI_COMM_NULL; + MPI_Comm_dup(MPI_COMM_WORLD, &dup); + BOOST_TEST(mpi::routes_pairwise(dup) == cold); + BOOST_TEST(mpi::routes_pairwise(dup) == cold); + MPI_Comm_free(&dup); + } + int size = 1; + MPI_Comm_size(MPI_COMM_WORLD, &size); + if (size == 1) { + BOOST_TEST(!cold); // no peer to pair with + } +} +#endif diff --git a/cpp/tests/openmp_runtime_tests.cpp b/cpp/tests/openmp_runtime_tests.cpp new file mode 100644 index 00000000..2d569529 --- /dev/null +++ b/cpp/tests/openmp_runtime_tests.cpp @@ -0,0 +1,666 @@ +// Copyright 2026 Algorithmiq +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +// Construction-time thread budgets, their per-object plumbing, and the invalid-owner rule. +// +// Budgets are captured from monoprop_NUM_THREADS when an object is constructed. Cases that need a specific +// launch environment are also registered as separate fresh-process CTest entries (see CMakeLists.txt); +// the rest set the variable around one construction with ScopedEnv and restore it afterwards. Kernels are +// still serial here, so these tests check what reaches each object, never parallel execution. + +#include + +#include +#ifdef __linux__ +#include +#endif + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include "PropagatorTestAccess.h" +#include "TestUtilities.h" +#include "monoprop/Evolution.h" +#include "monoprop/MPFunctions.h" +#include "monoprop/MonomialPropagator.h" +#include "monoprop/detail/evolution/CosineRecompute.h" +#include "monoprop/detail/evolution/layer_build/FusedApply.h" +#include "monoprop/detail/mpi/MPICompat.h" +#include "monoprop/detail/mpi/OperationFailure.h" +#include "monoprop/detail/parallel/Options.h" +#include "monoprop/detail/parallel/ThreadBudget.h" +#include "monoprop/detail/parallel/Workshare.h" + +namespace { + +using namespace monoprop; +using monoprop::detail::parallel::capture_thread_budget; +using monoprop::detail::parallel::Options; +using monoprop::detail::parallel::resolve_thread_budget; + +constexpr size_t kNumModes = 8; +constexpr unsigned int kCutoff = 4; +constexpr const char *kBudgetVariable = "monoprop_NUM_THREADS"; + +using Access = monoprop::detail::PropagatorTestAccess; +using Propagator = MonomialPropagator; + +// Sets (or unsets, for nullopt) one environment variable for a scope, then restores it. +class ScopedEnv { +public: + ScopedEnv(const char *name, std::optional value) : name_(name) { + if (const char *old = std::getenv(name)) { + old_ = std::string(old); + } + if (value) { + ::setenv(name, value->c_str(), 1); + } + else { + ::unsetenv(name); + } + } + ScopedEnv(const ScopedEnv &) = delete; + auto operator=(const ScopedEnv &) -> ScopedEnv & = delete; + ~ScopedEnv() { + if (old_) { + ::setenv(name_, old_->c_str(), 1); + } + else { + ::unsetenv(name_); + } + } + +private: + const char *name_; + std::optional old_; +}; + +// A distinct type, so a test can tell the original exception from anything the library substitutes. +class InjectedFailure : public std::runtime_error { +public: + using std::runtime_error::runtime_error; +}; + +auto make_sim(const test_utils::CaseData &data, + size_t partitions, + std::optional schrodinger = std::nullopt) -> Propagator { + return Propagator(data.hamiltonian, + kCutoff, + data.initial_state, + schrodinger, + MPI_COMM_SELF, + std::nullopt, + std::nullopt, + CutoffType::Length, + std::nullopt, + kNumModes, + Basis::Majorana, + partitions); +} + +// The one-store prototype: explicit partitions=1 on an ordinary communicator. +auto make_prototype(const test_utils::CaseData &data) -> Propagator { + return make_sim(data, 1); +} + +auto load() -> test_utils::CaseData { + return test_utils::load_case_data("random_exact.msgpack"); +} + +auto build_all(Propagator &sim, const test_utils::CaseData &data) -> void { + sim.build_graph(data.majoranas, data.param_inds, data.gen_coeffs); +} + +// What the launch environment should resolve to, independently of the library's capture. +auto expected_launch_budget() -> int { + if (const char *expected = std::getenv("monoprop_TEST_EXPECT_BUDGET")) { + return std::stoi(expected); + } + const char *configured = std::getenv(kBudgetVariable); + return resolve_thread_budget(configured ? std::optional(configured) : std::nullopt, + omp_get_max_threads()) + .threads; +} + +struct RuntimeSettings { + int max_threads = 0; + int dynamic = 0; + int max_active_levels = 0; + int thread_limit = 0; + omp_sched_t schedule = omp_sched_static; + int chunk = 0; + omp_proc_bind_t proc_bind = omp_proc_bind_false; + + auto operator==(const RuntimeSettings &) const -> bool = default; +}; + +auto runtime_settings() -> RuntimeSettings { + RuntimeSettings s; + s.max_threads = omp_get_max_threads(); + s.dynamic = omp_get_dynamic(); + s.max_active_levels = omp_get_max_active_levels(); + s.thread_limit = omp_get_thread_limit(); + omp_get_schedule(&s.schedule, &s.chunk); + s.proc_bind = omp_get_proc_bind(); + return s; +} + +auto binding_disabled(boost::unit_test::test_unit_id) -> boost::test_tools::assertion_result { + boost::test_tools::assertion_result result(omp_get_proc_bind() == omp_proc_bind_false); + result.message() << "OpenMP binding is enabled (OMP_PROC_BIND); run with OMP_PROC_BIND=false"; + return result; +} + +// Every state-consuming entry an invalid owner must refuse, each run on its own. +template +auto for_each_state_entry(Propagator &sim, const test_utils::CaseData &data, Check check) -> void { + check("build_graph", [&] { sim.build_graph(data.majoranas, data.param_inds, data.gen_coeffs); }); + check("propagate", [&] { sim.propagate(data.majoranas, data.param_inds, data.gen_coeffs, data.parameters); }); + check("expectation_value", [&] { (void)sim.expectation_value(data.parameters); }); + check("expectation_value_and_gradient", [&] { (void)sim.expectation_value_and_gradient(data.parameters); }); + check("expectation_value_functional", [&] { (void)sim.expectation_value_functional(); }); + check("expectation_value_and_gradient_functional", [&] { (void)sim.expectation_value_and_gradient_functional(); }); + check("contract_partially", [&] { (void)sim.contract_partially(data.parameters, false); }); + check("evolved_operator_terms", [&] { (void)sim.evolved_operator_terms(data.parameters, 0.0); }); + check("update_initial_operator", [&] { sim.update_initial_operator(data.hamiltonian); }); + check("set_parameter_mapping", [&] { sim.set_parameter_mapping(sim.parameter_mapping()); }); + check("update_cutoff", [&] { sim.update_cutoff(kCutoff); }); + check("size", [&] { (void)sim.size(); }); + check("graph_size", [&] { (void)sim.graph_size(); }); + check("graph_data", [&] { (void)sim.graph_data(); }); + check("copy", [&] { const Propagator copy(sim); }); + check("clone", [&] { (void)Access::clone(sim); }); +} + +auto require_invalid(Propagator &sim, const test_utils::CaseData &data) -> void { + for_each_state_entry(sim, data, [](const char *what, auto &&call) { + BOOST_TEST_CONTEXT(what) { + BOOST_CHECK_THROW(call(), InvalidPropagatorError); + } + }); +} + +} // namespace + +// --- resolve_thread_budget: the pure parser -------------------------------------------------------------- + +BOOST_AUTO_TEST_CASE(openmp_budget_absent_uses_runtime_default) { + BOOST_TEST(resolve_thread_budget(std::nullopt, 1).threads == 1); + BOOST_TEST(resolve_thread_budget(std::nullopt, 3).threads == 3); +} + +BOOST_AUTO_TEST_CASE(openmp_budget_absent_rejects_a_nonpositive_runtime_default) { + BOOST_CHECK_THROW((void)resolve_thread_budget(std::nullopt, 0), std::invalid_argument); + BOOST_CHECK_THROW((void)resolve_thread_budget(std::nullopt, -1), std::invalid_argument); +} + +BOOST_AUTO_TEST_CASE(openmp_budget_explicit_value_overrides_the_default) { + BOOST_TEST(resolve_thread_budget("8", 2).threads == 8); + // A valid explicit value wins even over an invalid fallback. + BOOST_TEST(resolve_thread_budget("1", 0).threads == 1); + BOOST_TEST(resolve_thread_budget("1", -1).threads == 1); + // Leading zeros are harmless, however many. + BOOST_TEST(resolve_thread_budget("0008", 2).threads == 8); + BOOST_TEST(resolve_thread_budget("0000000000000000000000000000000000000003", 1).threads == 3); +} + +BOOST_AUTO_TEST_CASE(openmp_budget_rejects_malformed_values) { + for (const std::string_view text : + {"", "0", "00", "-1", "-0", "abc", "2,3", " 3", "3 ", "+3", "\t3", "3\n", "0x10", "1e3", "3.0", "3a"}) { + BOOST_TEST_CONTEXT("value=\"" << text << "\"") { + BOOST_CHECK_EXCEPTION((void)resolve_thread_budget(text, 2), + std::invalid_argument, + [](const std::invalid_argument &e) { + return std::string_view(e.what()).find(kBudgetVariable) != std::string_view::npos; + }); + } + } + // Non-ASCII digits (Arabic-Indic three, U+0663) are not decimal digits here. + BOOST_CHECK_THROW((void)resolve_thread_budget("\xd9\xa3", 2), std::invalid_argument); +} + +BOOST_AUTO_TEST_CASE(openmp_budget_accepts_int_max_without_creating_a_team) { + // Parsing only: no region is opened, so no INT_MAX-sized team is ever requested. + BOOST_TEST(resolve_thread_budget("2147483647", 1).threads == INT_MAX); + BOOST_TEST(resolve_thread_budget("0002147483647", 1).threads == INT_MAX); +} + +BOOST_AUTO_TEST_CASE(openmp_budget_rejects_overflow) { + for (const std::string_view text : + {"2147483648", "4294967297", "18446744073709551617", "99999999999999999999999"}) { + BOOST_TEST_CONTEXT("value=" << text) { + BOOST_CHECK_THROW((void)resolve_thread_budget(text, 2), std::invalid_argument); + } + } +} + +BOOST_AUTO_TEST_CASE(openmp_budget_has_no_arbitrary_thread_cap) { + // The legacy parser silently ignored anything above one million. + BOOST_TEST(resolve_thread_budget("1000001", 1).threads == 1000001); +} + +// --- capture_thread_budget: one environment read per call --------------------------------------------- + +BOOST_AUTO_TEST_CASE(openmp_capture_reads_the_environment) { + { + const ScopedEnv env(kBudgetVariable, "5"); + BOOST_TEST(capture_thread_budget().threads == 5); + } + { + const ScopedEnv env(kBudgetVariable, std::nullopt); + BOOST_TEST(capture_thread_budget().threads == omp_get_max_threads()); + } + { + // Present but empty is an error, never the fallback. + const ScopedEnv env(kBudgetVariable, ""); + BOOST_CHECK_THROW((void)capture_thread_budget(), std::invalid_argument); + } + { + const ScopedEnv env(kBudgetVariable, "two"); + BOOST_CHECK_THROW((void)capture_thread_budget(), std::invalid_argument); + } +} + +// --- per-object capture ------------------------------------------------------------------------------------ + +// Registered again as fresh processes with monoprop_NUM_THREADS=1, =3, and unset with OMP_NUM_THREADS=3. +BOOST_AUTO_TEST_CASE(openmp_prototype_budget_matches_the_launch_environment) { + const auto data = load(); + const auto sim = make_prototype(data); + BOOST_TEST(Access::options(sim).threads == expected_launch_budget()); +} + +BOOST_AUTO_TEST_CASE(openmp_prototype_captures_once_and_copies_preserve_it) { + const auto data = load(); + std::optional sim; + { + const ScopedEnv env(kBudgetVariable, "3"); + sim.emplace(make_prototype(data)); + } + BOOST_TEST(Access::options(*sim).threads == 3); + build_all(*sim, data); + + // A later environment change reaches new objects only. + const ScopedEnv env(kBudgetVariable, "5"); + const Propagator copy(*sim); + BOOST_TEST(Access::options(copy).threads == 3); + BOOST_TEST(Access::options(*Access::clone(*sim)).threads == 3); + BOOST_TEST(Access::options(make_prototype(data)).threads == 5); + BOOST_TEST(Access::options(*sim).threads == 3); +} + +BOOST_AUTO_TEST_CASE(openmp_retained_functionals_carry_the_owner_budget) { + const auto data = load(); + std::optional sim; + { + const ScopedEnv env(kBudgetVariable, "3"); + sim.emplace(make_prototype(data)); + } + build_all(*sim, data); + auto copy = *sim; + + int seen = 0; + const auto record = + [&seen](const EvalRequest &request, mpi::Comm, const monoprop::detail::CosCallbacks &) -> double { + seen = request.parallel.threads; + return 0.0; + }; + const ScopedEnv env(kBudgetVariable, "5"); + auto from_owner = Access::make_functional(*sim, record); + auto from_copy = Access::make_functional(copy, record); + (void)from_owner(data.parameters); + BOOST_TEST(seen == 3); + seen = 0; + (void)from_copy(data.parameters); + BOOST_TEST(seen == 3); + // The public functionals evaluate through the same request and still agree with a serial object. + auto serial = [&] { + const ScopedEnv one(kBudgetVariable, "1"); + return make_prototype(data); + }(); + build_all(serial, data); + BOOST_TEST(test_utils::near(sim->expectation_value_functional()(data.parameters), + serial.expectation_value_functional()(data.parameters))); +} + +BOOST_AUTO_TEST_CASE(openmp_invalid_budget_is_a_config_error_at_construction) { + const auto data = load(); + for (const char *value : {"0", "abc", "", "-2", "2147483648"}) { + const ScopedEnv env(kBudgetVariable, value); + BOOST_TEST_CONTEXT("monoprop_NUM_THREADS=\"" << value << "\"") { + BOOST_CHECK_EXCEPTION((void)make_prototype(data), + PropagatorConfigError, + [](const PropagatorConfigError &e) { + return std::string_view(e.what()).find(kBudgetVariable) != std::string_view::npos; + }); + } + } +} + +BOOST_AUTO_TEST_CASE(openmp_legacy_paths_keep_serial_options) { + const auto data = load(); + const ScopedEnv env(kBudgetVariable, "3"); + // A two-partition facade and its Shm children (built with partitions=1 over a non-ordinary comm). + const auto facade = make_sim(data, 2); + BOOST_TEST(Access::options(facade).threads == 1); + BOOST_REQUIRE(Access::partition_count(facade) == 2); + for (int r = 0; r < 2; ++r) { + BOOST_TEST(Access::options(Access::partition(facade, r)).threads == 1); + } + // Automatic partition selection keeps the legacy path whatever it resolves to. + const auto automatic = make_sim(data, 0); + BOOST_TEST(Access::options(automatic).threads == 1); + for (int r = 0; r < Access::partition_count(automatic); ++r) { + BOOST_TEST(Access::options(Access::partition(automatic, r)).threads == 1); + } +} + +// Registered again with OMP_THREAD_LIMIT=2 and monoprop_NUM_THREADS=3. +BOOST_AUTO_TEST_CASE(openmp_runtime_limited_team_keeps_the_captured_budget) { + const auto data = load(); + const auto sim = make_prototype(data); + const auto options = Access::options(sim); + BOOST_TEST(options.threads == expected_launch_budget()); + + constexpr size_t kBlocks = 64; + std::vector worker(kBlocks, -1); + monoprop::detail::parallel::for_blocks(kBlocks, options, [&](size_t b) { worker[b] = omp_get_thread_num(); }); + const std::set distinct(worker.begin(), worker.end()); + BOOST_TEST(!distinct.contains(-1)); + BOOST_TEST(distinct.size() <= static_cast(std::min(options.threads, omp_get_thread_limit()))); + // A smaller actual team is not a budget change. + BOOST_TEST(Access::options(sim).threads == options.threads); +} + +BOOST_AUTO_TEST_CASE(openmp_propagator_work_leaves_openmp_settings_unchanged) { + const auto data = load(); + const auto before = runtime_settings(); + { + const ScopedEnv env(kBudgetVariable, "3"); + auto sim = make_prototype(data); + build_all(sim, data); + (void)sim.expectation_value_and_gradient(data.parameters); + const auto copy = sim; + (void)sim.contract_partially(data.parameters, true); + } + BOOST_TEST((runtime_settings() == before)); +} + +// Registered again with OMP_PROC_BIND=false; skipped where the runtime was asked to bind. +BOOST_AUTO_TEST_CASE(openmp_propagator_work_does_not_change_affinity, + *boost::unit_test::precondition(binding_disabled)) { +#ifdef __linux__ + cpu_set_t before; + CPU_ZERO(&before); + BOOST_REQUIRE(sched_getaffinity(0, sizeof(before), &before) == 0); + { + const auto data = load(); + const ScopedEnv env(kBudgetVariable, "3"); + auto sim = make_prototype(data); + build_all(sim, data); + (void)sim.expectation_value_and_gradient(data.parameters); + monoprop::detail::parallel::for_blocks(8, Access::options(sim), [](size_t) {}); + } + cpu_set_t after; + CPU_ZERO(&after); + BOOST_REQUIRE(sched_getaffinity(0, sizeof(after), &after) == 0); + BOOST_TEST(CPU_EQUAL(&before, &after)); +#else + BOOST_TEST_MESSAGE("affinity masks are only compared on Linux"); +#endif +} + +// --- old low-level call sites keep compiling and default to serial Options --------------------------------- + +BOOST_AUTO_TEST_CASE(openmp_low_level_calls_default_to_serial_options) { + const auto data = load(); + auto sim = make_prototype(data); + build_all(sim, data); + const auto &graph = sim.graph(); + const auto view = graph.replay_view(); + const auto mapped = map_params(data.parameters, sim.parameter_mapping(), VecD(sim.graph_layers(), 1.0), 1.0, true); + const auto cos = build_cos_callbacks(sim.mp_op().inverted_index(), view, Basis::Majorana); + const auto cos_threaded = + build_cos_callbacks(sim.mp_op().inverted_index(), view, Basis::Majorana, Options{.threads = 3}); + + const VecD op = sim.mp_op().get_operator(); + const VecD by_default = evolve_operator(VecD(op), view, mapped, MPI_COMM_SELF, cos.scale); + const VecD explicit_serial = evolve_operator(VecD(op), view, mapped, MPI_COMM_SELF, cos.scale, Options{}); + const VecD threaded = + evolve_operator(VecD(op), view, mapped, MPI_COMM_SELF, cos_threaded.scale, Options{.threads = 3}); + BOOST_CHECK_EQUAL_COLLECTIONS(by_default.begin(), by_default.end(), explicit_serial.begin(), explicit_serial.end()); + BOOST_CHECK_EQUAL_COLLECTIONS(by_default.begin(), by_default.end(), threaded.begin(), threaded.end()); + + VecD s1(op.size(), 0.5), h1 = by_default, s2 = s1, h2 = h1; + const size_t last = sim.graph_layers() - 1; + const LayerAngle angle{.gen_coeff = 1.0, .param = mapped[last]}; + const double d1 = state_operator_derivative_local(s1, h1, view, last, angle, MPI_COMM_SELF, cos.accumulate); + const double d2 = state_operator_derivative_local(s2, + h2, + view, + last, + angle, + MPI_COMM_SELF, + cos_threaded.accumulate, + {}, + Options{.threads = 3}); + BOOST_TEST(d1 == d2); + BOOST_CHECK_EQUAL_COLLECTIONS(h1.begin(), h1.end(), h2.begin(), h2.end()); + + CosMask mask; + mask.blocks.emplace_back(0, uint64_t{0b1011}); + mask.total_count = 3; + VecD a(64, 2.0), b = a; + monoprop::detail::scale_cos_mask(a.data(), mask, 0.5); + monoprop::detail::scale_cos_mask(b.data(), mask, 0.5, Options{.threads = 3}); + BOOST_CHECK_EQUAL_COLLECTIONS(a.begin(), a.end(), b.begin(), b.end()); + + EvalState state = EvalState::dense(VecD(op.size(), 0.25)); + const auto mapping = sim.parameter_mapping(); + const VecD gens(mapping.size(), 1.0); + const EvalRequest serial_request{.e_core = 0.0, + .state = state, + .op = op, + .parameter_mapping = mapping, + .gen_coeffs = gens, + .graph = view, + .params = data.parameters}; + BOOST_TEST(serial_request.parallel.threads == 1); +} + +// --- invalid-owner enforcement ------------------------------------------------------------------------------ + +BOOST_AUTO_TEST_CASE(openmp_failed_build_invalidates_the_owner) { + const auto data = load(); + const size_t half = data.majoranas.size() / 2; + BOOST_REQUIRE(half > 0); + const std::vector first(data.majoranas.begin(), data.majoranas.begin() + static_cast(half)); + const std::vector rest(data.majoranas.begin() + static_cast(half), data.majoranas.end()); + const VecZ first_map(data.param_inds.begin(), data.param_inds.begin() + static_cast(half)); + const VecZ rest_map(data.param_inds.begin() + static_cast(half), data.param_inds.end()); + const VecD first_gen(data.gen_coeffs.begin(), data.gen_coeffs.begin() + static_cast(half)); + const VecD rest_gen(data.gen_coeffs.begin() + static_cast(half), data.gen_coeffs.end()); + + auto sim = make_prototype(data); + sim.build_graph(first, first_map, first_gen); + const VecD params(expected_num_params(sim.parameter_mapping()), 0.3); + const double before = sim.expectation_value(params); + Propagator early(sim); + auto retained = sim.expectation_value_functional(); + + Access::set_cutoff_fn(sim, [](const Monomial &) -> bool { + throw InjectedFailure("injected cutoff failure"); + }); + // The original exception reaches a single-rank caller unchanged. + BOOST_CHECK_THROW(sim.build_graph(rest, rest_map, rest_gen), InjectedFailure); + BOOST_TEST(Access::is_invalid(sim)); + + require_invalid(sim, data); + BOOST_CHECK_THROW((void)retained(params), InvalidPropagatorError); + // Immutable configuration stays readable. + BOOST_TEST(sim.cutoff() == kCutoff); + BOOST_TEST(!sim.schrodinger()); + // An independent earlier copy is unaffected. + BOOST_TEST(test_utils::near(early.expectation_value(params), before)); +} + +BOOST_AUTO_TEST_CASE(openmp_failed_functional_evaluation_invalidates_the_owner) { + const auto data = load(); + auto sim = make_prototype(data); + build_all(sim, data); + const double before = sim.expectation_value(data.parameters); + Propagator early(sim); + auto retained = sim.expectation_value_and_gradient_functional(); + + auto failing = + Access::make_functional(sim, + [](const EvalRequest &, mpi::Comm, const monoprop::detail::CosCallbacks &) -> double { + throw InjectedFailure("injected evaluation failure"); + }); + BOOST_CHECK_THROW((void)failing(data.parameters), InjectedFailure); + BOOST_TEST(Access::is_invalid(sim)); + BOOST_CHECK_THROW((void)retained(data.parameters), InvalidPropagatorError); + require_invalid(sim, data); + BOOST_TEST(test_utils::near(early.expectation_value(data.parameters), before)); +} + +// A worker exception from the worksharing helper, inside a real evaluation, on one rank. +BOOST_AUTO_TEST_CASE(openmp_single_rank_worker_throw_keeps_the_original_exception) { + const auto data = load(); + auto sim = make_prototype(data); + build_all(sim, data); + auto failing = Access::make_functional( + sim, + [](const EvalRequest &request, mpi::Comm, const monoprop::detail::CosCallbacks &) -> double { + monoprop::detail::parallel::for_blocks(8, Options{.threads = 2}, [](size_t block) { + if (block == 5) { + throw InjectedFailure("injected worker failure"); + } + }); + return request.e_core; + }); + BOOST_CHECK_THROW((void)failing(data.parameters), InjectedFailure); + BOOST_TEST(Access::is_invalid(sim)); + BOOST_CHECK_THROW((void)sim.expectation_value(data.parameters), InvalidPropagatorError); +} + +BOOST_AUTO_TEST_CASE(openmp_validation_errors_leave_the_owner_usable) { + const auto data = load(); + auto sim = make_prototype(data); + build_all(sim, data); + const double before = sim.expectation_value(data.parameters); + const auto check_usable = [&](const char *what) { + BOOST_TEST_CONTEXT(what) { + BOOST_TEST(!Access::is_invalid(sim)); + BOOST_TEST(test_utils::near(sim.expectation_value(data.parameters), before)); + } + }; + + BOOST_CHECK_THROW(sim.build_graph(data.majoranas, VecZ{0}, data.gen_coeffs), std::exception); + check_usable("mismatched mapping length"); + + // A generator index past the system: rejected before any gate runs, even behind a valid gate. + std::vector bad_gates{data.majoranas.front(), VecZ{0, 2 * kNumModes}}; + BOOST_CHECK_THROW(sim.build_graph(bad_gates, VecZ{0, 0}, VecD{1.0, 1.0}), std::exception); + check_usable("out-of-range generator index in build_graph"); + auto fresh = make_prototype(data); + BOOST_CHECK_THROW(fresh.propagate(bad_gates, VecZ{0, 0}, VecD{1.0, 1.0}, VecD{0.1}), std::exception); + BOOST_TEST(!Access::is_invalid(fresh)); + + BOOST_CHECK_THROW(sim.propagate(data.majoranas, data.param_inds, data.gen_coeffs, data.parameters), std::exception); + check_usable("propagate on a stored graph"); + + BOOST_CHECK_THROW((void)sim.expectation_value(VecD{0.1}), std::exception); + check_usable("wrong parameter length"); + auto retained = sim.expectation_value_functional(); + BOOST_CHECK_THROW((void)retained(VecD{0.1}), std::exception); + check_usable("wrong parameter length through a retained functional"); + + sim.update_upper_atol(0.5); + BOOST_CHECK_THROW(sim.update_lower_atol(1.0), PropagatorConfigError); + sim.update_upper_atol(std::nullopt); + check_usable("crossed atol pair"); + BOOST_CHECK_THROW(sim.set_parameter_mapping(VecZ{0}), std::exception); + check_usable("wrong mapping length"); + + OperatorDict bad_operator = data.hamiltonian; + bad_operator[VecZ{0, 2 * kNumModes}] = 1.0; + BOOST_CHECK_THROW(sim.update_initial_operator(bad_operator), std::exception); + check_usable("out-of-range initial-operator index"); + BOOST_TEST(test_utils::near(retained(data.parameters), before)); +} + +BOOST_AUTO_TEST_CASE(openmp_invalid_owner_can_still_be_destroyed) { + const auto data = load(); + auto owner = std::make_unique(make_prototype(data)); + build_all(*owner, data); + Access::set_cutoff_fn(*owner, [](const Monomial &) -> bool { throw InjectedFailure("injected"); }); + BOOST_CHECK_THROW(build_all(*owner, data), InjectedFailure); + BOOST_CHECK_NO_THROW(owner.reset()); +} + +// --- failure boundary helper ----------------------------------------------------------------------------- + +BOOST_AUTO_TEST_CASE(openmp_operation_failed_rethrows_the_original_on_one_rank) { + const auto error = std::make_exception_ptr(InjectedFailure("original")); + BOOST_CHECK_EXCEPTION(mpi::operation_failed(MPI_COMM_SELF, error), InjectedFailure, [](const InjectedFailure &e) { + return std::string_view(e.what()) == "original"; + }); + int calls = 0; + BOOST_CHECK_THROW(mpi::guard_distributed(MPI_COMM_SELF, + [&] { + ++calls; + throw InjectedFailure("guarded"); + }), + InjectedFailure); + BOOST_TEST(calls == 1); + BOOST_TEST(mpi::guard_distributed(MPI_COMM_SELF, [] { return 7; }) == 7); +} + +#ifdef monoprop_ENABLE_MPI +BOOST_AUTO_TEST_CASE(openmp_thread_level_decisions_are_pure) { + const int levels[] = {MPI_THREAD_SINGLE, MPI_THREAD_FUNNELED, MPI_THREAD_SERIALIZED, MPI_THREAD_MULTIPLE}; + for (const int provided : levels) { + BOOST_TEST_CONTEXT("provided=" << mpi::thread_level_name(provided)) { + BOOST_TEST(mpi::thread_level_satisfies(provided, MPI_THREAD_SERIALIZED) + == (provided >= MPI_THREAD_SERIALIZED)); + BOOST_TEST(mpi::thread_level_satisfies(provided, MPI_THREAD_FUNNELED) == (provided >= MPI_THREAD_FUNNELED)); + } + } + // Coexistence keeps SERIALIZED as the requested and required level until Hybrid is removed. + BOOST_TEST(mpi::kRequiredThreadLevel == MPI_THREAD_SERIALIZED); + BOOST_TEST(std::string(mpi::thread_level_name(MPI_THREAD_FUNNELED)) == "MPI_THREAD_FUNNELED"); +} + +BOOST_AUTO_TEST_CASE(openmp_initialized_mpi_meets_the_required_level) { + int provided = MPI_THREAD_SINGLE; + MPI_Query_thread(&provided); + BOOST_TEST_MESSAGE("MPI provided thread support: " << mpi::thread_level_name(provided)); + BOOST_TEST(mpi::thread_level_satisfies(provided, mpi::kRequiredThreadLevel)); + BOOST_CHECK_NO_THROW(mpi::require_thread_support()); + int is_main = 0; + MPI_Is_thread_main(&is_main); + BOOST_TEST(is_main != 0); + // The initializing thread passes the guard; a wrong thread is covered by mpi_failure_driver. + mpi::require_initializing_thread(); +} +#endif diff --git a/cpp/tests/run_mpi_failure_scenario.cmake b/cpp/tests/run_mpi_failure_scenario.cmake new file mode 100644 index 00000000..3f416c68 --- /dev/null +++ b/cpp/tests/run_mpi_failure_scenario.cmake @@ -0,0 +1,86 @@ +# Runs one mpi_failure_driver scenario under the MPI launcher and checks that the job fails promptly. +# +# Inputs (-D): MPIEXEC, NUMPROC_FLAG, PREFLAGS (MPIEXEC_PREFLAGS, may be empty), DRIVER, SCENARIO, OPTION +# (may be empty), EXPECT (regex that must appear in the combined output), FORBID (optional regex that must +# not), TIMEOUT_SECONDS (default 30). +# +# Passing requires a nonzero exit before the timeout with EXPECT in the output. A timeout is a failure, +# never a pass: it means some rank was left blocked. The launcher must end the peers of a rank that exits +# abnormally (Open MPI's default); the wrong-thread case depends on that policy. + +if(NOT DEFINED TIMEOUT_SECONDS) + set(TIMEOUT_SECONDS 30) +endif() + +set( + _command + "${MPIEXEC}" + "${NUMPROC_FLAG}" + 2 + ${PREFLAGS} + "${DRIVER}" + "${SCENARIO}" +) +if(NOT "${OPTION}" STREQUAL "") + list(APPEND _command "${OPTION}") +endif() + +string(TIMESTAMP _start "%s" UTC) +execute_process( + COMMAND + ${_command} + TIMEOUT ${TIMEOUT_SECONDS} + RESULT_VARIABLE _result + OUTPUT_VARIABLE _stdout + ERROR_VARIABLE _stderr +) +string(TIMESTAMP _stop "%s" UTC) +math(EXPR _elapsed "${_stop} - ${_start}") + +message("command: ${_command}") +message( + "elapsed: ${_elapsed} s (timeout ${TIMEOUT_SECONDS} s); result: ${_result}" +) +message("--- stdout ---\n${_stdout}--- stderr ---\n${_stderr}--------------") + +set(_output "${_stdout}${_stderr}") +if(_result MATCHES "timeout") + message( + FATAL_ERROR + "FAILED: the job did not terminate within ${TIMEOUT_SECONDS} s" + ) +endif() +if(_output MATCHES "NOT EXERCISED") + message( + "NOT EXERCISED: the MPI library supplied more thread support than requested" + ) + if(CMAKE_VERSION VERSION_LESS 3.29) + message(FATAL_ERROR "NOT EXERCISED (CMake < 3.29 cannot report a skip)") + endif() + cmake_language(EXIT 77) +endif() +if("${_result}" STREQUAL "0") + message( + FATAL_ERROR + "FAILED: the job exited 0; the injected failure did not end it" + ) +endif() +if(NOT _output MATCHES "${EXPECT}") + message(FATAL_ERROR "FAILED: expected diagnostic /${EXPECT}/ not found") +endif() +if( + DEFINED + FORBID + AND + NOT + "${FORBID}" + STREQUAL + "" + AND + _output + MATCHES + "${FORBID}" +) + message(FATAL_ERROR "FAILED: forbidden output /${FORBID}/ found") +endif() +message("PASSED: nonzero exit after ${_elapsed} s with the expected diagnostic") diff --git a/docs/content/docs/building.mdx b/docs/content/docs/building.mdx index a4156736..6cb5b70a 100644 --- a/docs/content/docs/building.mdx +++ b/docs/content/docs/building.mdx @@ -36,6 +36,12 @@ without MPI, so a from-source build is required for multi-rank runs. it with `export OpenMP_ROOT="$(brew --prefix libomp)"` before building, and before configuring a project that consumes the installed package. + The library calls the OpenMP runtime itself, so both `libmonoprop` and the Python + extension link it. The runtime therefore starts when monoprop loads. If you set + `OMP_PLACES` and leave binding enabled, it binds the importing thread to the first + place at that point. That is OpenMP applying your placement request, not monoprop: + monoprop never changes affinity. Set `OMP_PROC_BIND=false` if you want no binding. + The repository ships a [DevContainer](https://containers.dev/) with all of the above pre-configured; opening the folder in VS Code and rebuilding the container is the quickest route to a working environment. @@ -238,7 +244,15 @@ usage requirement the package fails to export. It compiles the same source as th probe, so the two cannot drift. The probe drives both the Majorana and the Pauli basis and instantiates the internal OpenMP worksharing template in its own translation unit, so the consumer only builds if the imported `monoprop::monoprop` target carries OpenMP's compile -and link flags — the consumer project adds none of its own. +and link flags — the consumer project adds none of its own. It also constructs the one-store +prototype (`partitions=1`), whose constructor template calls the library's out-of-line thread +budget and MPI thread checks, so an unexported helper fails the consumer's link. + +The exported low-level entry points (`evolve_step`, `evolve_operator` and +`state_operator_derivative_local`) take a trailing thread budget with a serial default. Existing +source keeps compiling, but the exported symbols changed. Rebuild C++ consumers, `libmonoprop` and +the Python bindings together, from the same monoprop build. The installed headers now include +`detail/parallel/ThreadBudget.h` and `detail/mpi/OperationFailure.h`. ### Debug tree diff --git a/docs/content/docs/features/parallelism.mdx b/docs/content/docs/features/parallelism.mdx index a56727ca..92cf5e81 100644 --- a/docs/content/docs/features/parallelism.mdx +++ b/docs/content/docs/features/parallelism.mdx @@ -46,6 +46,46 @@ Which MPI rank owns a term is set separately; see [Rank routing](#rank-routing). export monoprop_NUM_THREADS=8 ``` +### One-store prototype (transitional) + +monoprop is moving from thread-owned partitions to one operator store per MPI rank, worked on by +OpenMP threads. While both runtimes coexist, the new path is selected only by the low-level +`partitions=1` argument (the C++ constructor, or `monoprop._core.MonomialPropagatorNNN(..., +partitions=1)`) on an ordinary communicator, with or without MPI. The default constructors and +multi-partition runs are unchanged. + +- **Thread budget.** The prototype reads `monoprop_NUM_THREADS` once, when it is constructed. A set + value must be decimal digits for a number from 1 to 2147483647 (leading zeros are fine). An empty + value, a sign, spaces, a list, zero or an overflowing value raises a `RuntimeError` naming the + variable, and never falls back. When the variable is unset, the OpenMP runtime default + (`omp_get_max_threads()`, normally `OMP_NUM_THREADS`) applies. The budget is not clamped to the + cores you have; placement belongs to `OMP_PLACES`, `OMP_PROC_BIND` and your launcher. Copies and + retained callables keep the budget, and a later change to the environment only affects new objects. +- **Serial kernels, for now.** The budget reaches every kernel, but no kernel runs in parallel yet. +- **Calling thread.** Every call that can use MPI (construction, operations, retained callables) + must come from the thread that initialized MPI, whatever thread support MPI provides. A call from + another thread prints a diagnostic and aborts that process without communicating; the launcher + then ends the other ranks. +- **MPI thread support.** Construction checks that MPI provides at least `MPI_THREAD_SERIALIZED`, + whether monoprop or your program (mpi4py, for example) initialized it. mpi4py's default, + `MPI_THREAD_MULTIPLE`, is fine. +- **Invalid settings in a multi-rank run** abort the job: the other ranks are already on their way + into collective setup. + +## Failed operations + +An operation that fails after it started changing a propagator (building or extending a graph, +propagating, evaluating, contracting, or updating the initial operator) leaves the propagator +unusable. Every later operation on it, copying it, and every callable it returned then raise a +`RuntimeError` that asks you to construct a new propagator. Copies made before the failure keep +working. Argument errors (wrong lengths, out-of-range indices, crossed tolerances and so on) are +raised before anything changes, so the propagator stays usable. + +With MPI and more than one rank, a failure on one rank cannot be handed back to your program +without leaving the other ranks blocked in communication. monoprop therefore prints the rank and the +error to stderr and calls `MPI_Abort`. On a single rank, and without MPI, you get the original +exception. Multi-partition runs keep their existing handling. + ## Placement report Building a propagator writes one `COMMPLACE` line per rank to stderr, naming the diff --git a/docs/content/docs/testing.mdx b/docs/content/docs/testing.mdx index fd84defc..30a547b6 100644 --- a/docs/content/docs/testing.mdx +++ b/docs/content/docs/testing.mdx @@ -109,6 +109,28 @@ one entry per rank count rather than per case, because the ranks have to reach the same collectives. Each MPI entry times out after 600 seconds, preventing a collective deadlock from occupying the runner indefinitely. +A few cases also run again as separate `openmp_env_*` entries, each a fresh process with its own +launch environment. They cover `monoprop_NUM_THREADS=1` and `=3`, the OpenMP default with the +variable unset, `OMP_THREAD_LIMIT=2`, and `OMP_PROC_BIND=false` for the affinity check. They are +needed because a propagator reads its thread budget once, when it is constructed. + +The MPI build also registers `mpi_failure_*` entries, labelled `mpi` and `mpi_failure`, from +`cpp/tests/mpi_failure_driver.cpp`. That driver has its own `main()`, so the unit-runner glob +excludes it. Each entry launches two ranks through `cpp/tests/run_mpi_failure_scenario.cmake`, with +a 30-second timeout. Rank 0 fails in one of these ways while its peer follows the normal phase: + +- a worksharing worker throws; +- the scan fails before the first exchange; +- a failure happens while a replay `Ticket` or a construction `PendingAlltoallv` is posted; +- host-initialized MPI provides only `SINGLE` or `FUNNELED`; +- a guarded call arrives from a `std::thread` under `SERIALIZED` or `MULTIPLE`. + +An entry passes only on a prompt nonzero exit with the expected diagnostic. A timeout means a rank +was left blocked, and it fails the entry. The wrong-thread cases rely on the launcher ending the +peers of a process that aborts, which is Open MPI's default. `ctest -L mpi_failure` runs just +these entries. If the MPI library grants more thread support than a scenario requests, that +scenario reports itself as skipped instead of passing. + CI runs the non-MPI and MPI labels separately with `--no-tests=error`, so an MPI lane fails if its MPI variant was not registered. Standalone MPI-enabled C++ test executables initialize MPI before constructing propagators and destroy the diff --git a/tests/test_openmp_config.py b/tests/test_openmp_config.py new file mode 100644 index 00000000..9df94aa4 --- /dev/null +++ b/tests/test_openmp_config.py @@ -0,0 +1,130 @@ +# Copyright 2026 Algorithmiq +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"""Construction-time thread budgets, observed from fresh Python processes. + +A propagator reads ``monoprop_NUM_THREADS`` once, when it is constructed, so every case launches its own +interpreter with the environment under test. During the transition only the one-store prototype (the +low-level ``partitions=1`` argument on an ordinary communicator) captures a budget; the default and +multi-partition paths keep their legacy configuration. +""" + +from __future__ import annotations + +import json +import os +import subprocess +import sys + +import pytest + +# Environment a launcher gives its ranks: a child that inherits it would try to join the parent's job. +_LAUNCHER_PREFIXES = ("OMPI_", "PMIX_", "PRTE_", "PMI_", "HYDRA_", "MPIR_") + +_PROBE = r""" +import json +import sys + +from monoprop import _core + +args = json.loads(sys.argv[1]) +try: + sim = _core.MonomialPropagator032( + initial_operator={(0, 1): 1j, (2, 3): 0.5j, (1, 2): 0.25j}, + cutoff=4, + initial_state=[0, 1], + **args, + ) + sim.build_graph([[0, 2], [1, 3]], [0, 1], [1.0, 1.0]) + print(json.dumps({"ok": True, "energy": sim.expectation_value([0.3, 0.2])})) +except Exception as exc: # noqa: BLE001 - the parent inspects whatever the constructor raised + print(json.dumps({"ok": False, "type": type(exc).__name__, "message": str(exc)})) +""" + + +def _inside_multirank_job() -> bool: + size = os.environ.get("OMPI_COMM_WORLD_SIZE") or os.environ.get("PMI_SIZE") or "1" + return int(size) > 1 + + +pytestmark = pytest.mark.skipif( + _inside_multirank_job(), + reason="launches fresh interpreters, which cannot run inside a multi-rank job", +) + + +def _probe( + budget: str | None, extra_env: dict[str, str] | None = None, **args: object +) -> dict: + env = {k: v for k, v in os.environ.items() if not k.startswith(_LAUNCHER_PREFIXES)} + env.pop("monoprop_NUM_THREADS", None) + env.pop("monoprop_PARTITIONS", None) + if budget is not None: + env["monoprop_NUM_THREADS"] = budget + env.update(extra_env or {}) + result = subprocess.run( # noqa: S603 - trusted: this interpreter and a fixed probe + [sys.executable, "-c", _PROBE, json.dumps(args)], + env=env, + capture_output=True, + text=True, + timeout=120, + check=False, + ) + assert result.returncode == 0, result.stderr + return json.loads(result.stdout.strip().splitlines()[-1]) + + +@pytest.mark.parametrize( + "budget", ["", "0", "-1", "abc", "2,3", " 3", "3 ", "+3", "2147483648"] +) +def test_prototype_rejects_an_invalid_budget(budget: str) -> None: + outcome = _probe(budget, partitions=1) + assert not outcome["ok"] + assert outcome["type"] == "RuntimeError" + assert "monoprop_NUM_THREADS" in outcome["message"] + + +def test_prototype_accepts_valid_budgets_without_changing_results() -> None: + energies = [] + for budget in ["1", "3", "0003", None]: + outcome = _probe(budget, {"OMP_NUM_THREADS": "2"}, partitions=1) + assert outcome["ok"], (budget, outcome) + energies.append(outcome["energy"]) + # Kernels are still serial, so every budget gives the same numbers. + assert all(e == energies[0] for e in energies) + + +@pytest.mark.parametrize( + ("args", "extra_env"), + [ + pytest.param({"partitions": 2}, {}, id="two-partition-facade"), + pytest.param( + {"partitions": 0}, + {"monoprop_PARTITIONS": "off"}, + id="default-single-partition", + ), + ], +) +def test_legacy_paths_do_not_capture_the_budget( + args: dict, extra_env: dict[str, str] +) -> None: + # The same value the prototype rejects is never read on the legacy paths. + outcome = _probe("abc", extra_env, **args) + assert outcome["ok"], outcome + + +def test_no_thread_keyword_is_accepted() -> None: + outcome = _probe("1", partitions=1, num_threads=2) + assert not outcome["ok"] + assert outcome["type"] == "TypeError" From ef394889d5d191a10325537277c892a7f3ccbe85 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Roberto=20Di=20Remigio=20Eik=C3=A5s?= Date: Mon, 28 Sep 2026 07:42:11 +0000 Subject: [PATCH 13/64] =?UTF-8?q?fix(bench):=20=F0=9F=90=9B=20read=20the?= =?UTF-8?q?=20process=20allocation=20before=20importing=20monoprop?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit monoprop now links the OpenMP runtime. With OMP_PLACES set and binding on, libgomp binds the importing thread to the first place when the library loads, so the observe preflight read one CPU instead of the allocation. It then rejected the documented candidate launch on GCC builds. The allocation is now read before the import. For the baseline binary, which loads no OpenMP runtime, both orders read the same mask, so collected evidence is unaffected. Assisted-by: Pi:claude-opus-5-5 --- tools/benchmark-rank-local-openmp.py | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/tools/benchmark-rank-local-openmp.py b/tools/benchmark-rank-local-openmp.py index 949f49c4..9e654a7c 100755 --- a/tools/benchmark-rank-local-openmp.py +++ b/tools/benchmark-rank-local-openmp.py @@ -700,6 +700,10 @@ def open_context(args: argparse.Namespace, cell: dict[str, Any]) -> Context: settings_snapshot, ) + # Read before importing monoprop: loading it starts the OpenMP runtime, which (with OMP_PLACES set and + # binding on) may bind this thread to the first place, so a later read sees that place, not the allocation. + local_cpus = sorted(os.sched_getaffinity(0)) + import monoprop # noqa: PLC0415 identity = cell["identity"] @@ -731,7 +735,6 @@ def open_context(args: argparse.Namespace, cell: dict[str, Any]) -> Context: binary = Path(monoprop._core.__file__) binary_hash = file_sha256(binary) - local_cpus = sorted(os.sched_getaffinity(0)) everyone = ( local_cpus if comm is None From 1f87c019fc7adbdf807ec257236f3f8373beddb1 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Roberto=20Di=20Remigio=20Eik=C3=A5s?= Date: Mon, 28 Sep 2026 07:42:11 +0000 Subject: [PATCH 14/64] =?UTF-8?q?docs(parallelism):=20=F0=9F=93=9D=20recor?= =?UTF-8?q?d=20the=20Task=203=20outcome?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Record what Task 3 delivered and what later tasks inherit: - the implemented budget, invalidation, failure-boundary and MPI thread contracts; - RED/GREEN and mutation evidence, the R/M/Clang results and the provided MPI thread levels; - the ABI change, and the OpenMP runtime that the library now links; - the harness allocation fix, and the matching edit still due in the baseline worktrees; - where the uncommitted Task 3 artifacts live. Assisted-by: Pi:claude-opus-5-5 --- .../plans/2026-09-18-rank-local-openmp.md | 64 +++++++++++++++++++ 1 file changed, 64 insertions(+) diff --git a/docs/superpowers/plans/2026-09-18-rank-local-openmp.md b/docs/superpowers/plans/2026-09-18-rank-local-openmp.md index 7a741526..5aa39f4e 100644 --- a/docs/superpowers/plans/2026-09-18-rank-local-openmp.md +++ b/docs/superpowers/plans/2026-09-18-rank-local-openmp.md @@ -1464,6 +1464,70 @@ cases are baseline preservation evidence, not RED for a feature already shipped entry point; runtime global settings unchanged; failure tests do not hang. Public/exported signature ABI change is recorded for a coordinated library/bindings rebuild. +**Task 3 outcome (2026-09-27):** gate passed on GCC 15.2 (R and M) and Clang 18.1.8/libomp (R) Linux, after one +fix to the Task 1 harness that the change required (see below). Kernels are still serial. Other platforms pending. +- Contracts as implemented, which later tasks rely on: + - `resolve_thread_budget`/`capture_thread_budget` (exported, installed) follow the parser rules above. Only + explicit `partitions == 1` on a `Kind::Mpi` communicator captures a budget, once per object. Everything else, + including Shm/Hybrid children and facades, keeps `{.threads = 1}`. Copies and retained functionals keep the + captured value. A parse error is `PropagatorConfigError` on one rank and `MPI_Abort` on several. + - `run_operation_` separates validation from mutation with a `mutation_started` flag. A later failure calls + `invalidate_()` and then `mpi::operation_failed()`. That applies to every non-facade object on an ordinary + communicator, not only the prototype. It aborts when the communicator has more than one rank, and rethrows + the original exception otherwise. Facades rethrow through their legacy path. Validation that used to be + interleaved with mutation now runs first: generator, initial-operator and basis-change indices. + - Guards sit inside the `Ticket` lifetime in `evolve_step` and `state_operator_derivative_local`, around local + work before each allreduce in `ev`/`ev_and_grad`, and around each `wait_into` inside its `PendingAlltoallv` + lifetime in `run_exchange`. `build_layer` has no whole-body guard: the propagator's operation guard covers it. + - The requested and required MPI level stay `MPI_THREAD_SERIALIZED`. `mpi::init` checks the level actually + provided whether MPI was host- or library-initialized, and the prototype constructor calls + `require_thread_support()`. + - `require_initializing_thread()` uses `MPI_Is_thread_main`. On a wrong thread it prints a diagnostic and calls + `std::abort()` with no MPI calls. It guards only the MPI-using prototype entries, retained functionals included. + - White-box access for tests goes through a befriended `detail::PropagatorTestAccess`, which the library never + defines. +- Tests: + - `openmp_runtime_tests.cpp` has 23 cases. + - Five fresh-process `openmp_env_*` CTest entries cover budgets 1 and 3, the OpenMP default, + `OMP_THREAD_LIMIT=2`, and `OMP_PROC_BIND=false` for the affinity check. + - `tests/test_openmp_config.py` has 13 cases, and `mpi_utils_tests.cpp` gains a `routes_pairwise` cache case. + - `mpi_failure_driver.cpp` is excluded from the unit-runner glob and registered as eight `mpi_failure_*` entries. + - RED: invalid budgets were silently accepted (9 Python failures), and the C++ build failed on missing headers. + - Six R mutants and three failure-driver mutants were all killed. Without the abort, or without the guard + inside the Ticket lifetime, the scenarios hang until the 30 s timeout. +- Final R and M runs on fresh rebuilds: + - R: ctest 333/333; pytest 840 passed, 13 skipped (after the harness fix). + - M serial: ctest 363/363; pytest 827 passed, 26 skipped. + - Splitmix R=1-4 and linear R=1,2,4: C++ suites and failure scenarios pass; `--with-mpi` pytest passes. + - `just test-find-package` passes on R and M, including a new prototype chain. + - Clang: 333/333 and 840 passed. + - Generated bindings are byte-identical to Task 2's. +- Failure scenarios, two ranks with a 30 s timeout: all eight end with a nonzero exit in 2 s or less and print + the expected diagnostic. + - Provided levels equal the requested ones (SINGLE, FUNNELED, SERIALIZED, MULTIPLE), so the + insufficient-level case really runs; mpi4py 4.1.2 provides MULTIPLE. + - A wrong-thread entry exits with SIGABRT (134) and prints no `MPI_Abort` text. +- The exported `evolve_step`, `evolve_operator` and `state_operator_derivative_local` changed signature (trailing + `Options`, serial default). Rebuild the library, the bindings and C++ consumers together (`building.mdx`). +- Linkage: `_core.abi3.so` and `libmonoprop.so` now NEED the OpenMP runtime (`libgomp.so.1` or `libomp.so.5`), + so Task 2's "no libgomp" finding no longer holds. Check wheel bundling and delocation again in Task 13. +- Harness finding, fixed with owner approval: + - With `OMP_PLACES` set and binding on, the runtime binds the initial thread to the first place. libgomp does + it when the library loads, libomp at the first OpenMP call. This is OpenMP applying the user's placement; + monoprop calls no affinity API. + - `tools/benchmark-rank-local-openmp.py` read `sched_getaffinity(0)` after `import monoprop`, so it rejected the + documented candidate launch on GCC builds, and `test_observe_rejects_contradictory_evidence[change2]` failed. + - `open_context` now reads the allocation before importing monoprop. Afterwards R pytest gives 840 passed; the + harness tests pass on R, Clang, M and MPI with 2 ranks. + - For the baseline binary, which loads no OpenMP runtime, both orders read the same mask, so collected evidence is + unaffected. Apply the same edit to the baseline worktrees' copy of the tool before any Task 11 baseline reruns. + - Still open: `memory/cpu.py` reports the diagnostic `affinity_cpus` after the import, so under libgomp binding it + reports 1. It is metadata, not a gate. +- Not run: `just build-docs`, the Nix, macOS/Homebrew and devcontainer routes, and the wheel repair routes. +- Artifacts: `/home/ubuntu/task3-artifacts/` on the c8a.metal-24xl host (`report.md`, `ledger.md`, `logs/`). + They are intentionally uncommitted. The frozen campaign and workload files, Task 1 artifacts and baseline + binaries were verified unchanged. + ### Task 4: Prototype cosine and fused updates without structural concurrency **Files:** From e96af6ea01e9a38cef8315a86eedde9f1126f439 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Roberto=20Di=20Remigio=20Eik=C3=A5s?= Date: Mon, 28 Sep 2026 07:45:22 +0000 Subject: [PATCH 15/64] =?UTF-8?q?docs(parallelism):=20=F0=9F=93=9D=20recor?= =?UTF-8?q?d=20the=20baseline=20tool=20sync=20and=20add=20a=20Task=2011=20?= =?UTF-8?q?check?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The allocation fix is now applied to the tool copy in both baseline worktrees, so both arms run the same measurement code (sha256 2207529f…). Task 11 gains a checklist item to verify that hash before measuring, because those copies are uncommitted and could drift. Assisted-by: Pi:claude-opus-5-5 --- docs/superpowers/plans/2026-09-18-rank-local-openmp.md | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/docs/superpowers/plans/2026-09-18-rank-local-openmp.md b/docs/superpowers/plans/2026-09-18-rank-local-openmp.md index 5aa39f4e..2ced6812 100644 --- a/docs/superpowers/plans/2026-09-18-rank-local-openmp.md +++ b/docs/superpowers/plans/2026-09-18-rank-local-openmp.md @@ -1520,7 +1520,8 @@ fix to the Task 1 harness that the change required (see below). Kernels are stil - `open_context` now reads the allocation before importing monoprop. Afterwards R pytest gives 840 passed; the harness tests pass on R, Clang, M and MPI with 2 ranks. - For the baseline binary, which loads no OpenMP runtime, both orders read the same mask, so collected evidence is - unaffected. Apply the same edit to the baseline worktrees' copy of the tool before any Task 11 baseline reruns. + unaffected. The same edit is applied to the tool copy in both baseline worktrees, which now match the committed + tool (sha256 `2207529f…`). Their harness tests pass, including MPI with 2 ranks, and their binaries are unchanged. - Still open: `memory/cpu.py` reports the diagnostic `affinity_cpus` after the import, so under libgomp binding it reports 1. It is metadata, not a gate. - Not run: `just build-docs`, the Nix, macOS/Homebrew and devcontainer routes, and the wheel repair routes. @@ -2116,6 +2117,9 @@ modify `cpp/monoprop/detail/operator/OperatorIndex.h`, its immediate MPOperator/ `cpp/tests/{operator_index_tests,sparse_resolve_tests,simulator_copy_tests}.cpp`. Keep any adapter private and narrow; no container-plugin framework. Do not rename `bench_*` functions or move benchmark files. +- [ ] Before measuring, check that both arms run identical measurement code. The baseline worktrees hold uncommitted + copies of `tools/benchmark-rank-local-openmp.py`. Their sha256 must equal the candidate's committed copy (after + Task 3: `2207529f…`, which reads the allocation before importing monoprop). Re-sync them after any later tool change. - [ ] Add metadata for `has_mpi`, MPI requested/provided thread support, requested/effective team size, OpenMP runtime, OMP_NUM_THREADS/PLACES/PROC_BIND/DYNAMIC/THREAD_LIMIT, any library override, routing mode/effective linear bits/seed, launcher command, rank/worker CPU masks, NUMA IDs, revision/dirty overlay and extension path/hash. Collect placement From ea42df6616d2779ed888f1a967d1c04e3b0ca411 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Roberto=20Di=20Remigio=20Eik=C3=A5s?= Date: Mon, 28 Sep 2026 18:53:03 +0000 Subject: [PATCH 16/64] =?UTF-8?q?feat(parallel):=20=E2=9C=A8=20thread=20co?= =?UTF-8?q?sine=20scaling=20and=20fused=20coefficient=20updates?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Task 4 of the rank-local OpenMP plan. Two kernels now use the object's thread budget. Structure (scan, lookup, insertion, graph packing) stays serial. - scale_cos_lazy runs one logical range per 1024 fold words. Row parity is fetched on the caller for each call. Each worker keeps its scratch on its own stack, never the caller's thread_local buffer. Tail masking stays global. - scale_cos_mask runs one range per 1024 stored mask blocks. - apply_fused_contract keeps its phases, each joined before the next: gather insert snapshots into the existing v_tgt field, run the optional cosine phase, then apply records in ranges of 1024. - Every record range calls one out-of-line apply_fused_record_range, so the compiler's FMA contraction of cos*c + sin*phi*v is the same at every budget. Serial and threaded results are bitwise equal, and match the previous release bitwise on the paths checked. - Debug builds assert that every coefficient slot has exactly one writing record. - Fewer than two ranges stays serial. The thresholds are fixed until the evidence-driven tuning task. - The kernels take a defaulted, test-only range observer; production passes an empty one. Tests: - openmp_kernel_tests.cpp covers exact agreement across budgets, small work staying serial, worker exceptions joined before rethrow, and participation observed inside each kernel's own ranges. - Synthetic multi-block lazy folds (tails, cold parity, Majorana/Pauli, empty columns, index growth) are checked against the materialised-fold oracle. - Records from the real build path have one add-owner per slot, in both pictures and with cross-rank halves under MPI. Threaded prototypes match serial ones bitwise. - Fused and two-pass application still agree with graph replay at the existing tolerances. - openmp_env_kernels_* rerun the kernel cases under OMP_THREAD_LIMIT=2 and OMP_DYNAMIC=TRUE. - The failure driver gains active-ticket cosine-worker: a kernel worker throws while a replay Ticket is posted, and the job must abort rather than hang. Docs: the one-store prototype section lists which kernels are threaded. building.mdx explains that TSan evidence for the OpenMP kernels needs Clang + libomp + Archer, since GCC's libgomp reports false races. Assisted-by: Pi:claude-opus-5-5 --- AGENTS.md | 3 + cpp/monoprop/Evolution.cpp | 3 +- .../detail/evolution/CosineRecompute.h | 106 +++- .../detail/evolution/layer_build/FusedApply.h | 142 +++-- cpp/tests/CMakeLists.txt | 20 + cpp/tests/KernelTestSupport.h | 329 +++++++++++ cpp/tests/PropagatorTestAccess.h | 36 ++ cpp/tests/README.md | 14 +- cpp/tests/combined_recompute_equivalence.cpp | 155 ++++- cpp/tests/fused_cos_sweep_tests.cpp | 292 ++++++++++ cpp/tests/mpi_failure_driver.cpp | 41 ++ cpp/tests/openmp_kernel_tests.cpp | 536 ++++++++++++++++++ cpp/tests/openmp_runtime_tests.cpp | 4 +- docs/content/docs/building.mdx | 20 + docs/content/docs/features/parallelism.mdx | 7 +- tests/test_openmp_config.py | 3 +- 16 files changed, 1656 insertions(+), 55 deletions(-) create mode 100644 cpp/tests/KernelTestSupport.h create mode 100644 cpp/tests/openmp_kernel_tests.cpp diff --git a/AGENTS.md b/AGENTS.md index d1d71270..f123d01b 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -72,6 +72,9 @@ just build-docs separates validation from mutation (`run_operation_`): a failure after mutation starts invalidates the object and aborts a multi-rank ordinary communicator (`mpi::operation_failed`). Catch failures inside posted `Ticket`/`PendingAlltoallv` lifetimes, never only at the outermost level. +- Threaded kernels (`scale_cos_*`, `apply_fused_contract`) take a defaulted test-only range observer; + participation tests read it (`cpp/tests/KernelTestSupport.h`). Serial and threaded runs must agree + bitwise. TSan evidence needs Clang + libomp + Archer; GCC's libgomp gives false reports (`building.mdx`). - `cpp/tests/mpi_failure_driver.cpp` has its own `main()`, excluded from the unit-runner glob; `ctest -L mpi_failure` runs its two-rank abort scenarios (30 s timeout; a timeout is a hang). - Slow CTest startup in MPI builds is `MPI_Init` fabric probing; see diff --git a/cpp/monoprop/Evolution.cpp b/cpp/monoprop/Evolution.cpp index 35b834dd..e1086be7 100644 --- a/cpp/monoprop/Evolution.cpp +++ b/cpp/monoprop/Evolution.cpp @@ -512,7 +512,8 @@ auto evolve_step_traversal_impl(VecD &op, } } // namespace -// The kernels are still serial, so `options` is accepted but not yet consumed below this level. +// The layer's cosine scaling runs threaded through the budget captured in `cos_scale`; the endpoint work +// below it is still serial, so `options` itself is not yet consumed at this level. auto evolve_step(VecD &op, const MPGraphView &graph, double param, diff --git a/cpp/monoprop/detail/evolution/CosineRecompute.h b/cpp/monoprop/detail/evolution/CosineRecompute.h index 930bfe80..a41b1480 100644 --- a/cpp/monoprop/detail/evolution/CosineRecompute.h +++ b/cpp/monoprop/detail/evolution/CosineRecompute.h @@ -29,6 +29,7 @@ #include #include #include +#include #include #include @@ -37,9 +38,41 @@ #include "monoprop/detail/evolution/layer_build/Scan.h" #include "monoprop/detail/operator/InvertedIndex.h" #include "monoprop/detail/parallel/Options.h" +#include "monoprop/detail/parallel/Workshare.h" namespace monoprop::detail { +/*! + * \brief The kinds of logical work range a threaded kernel reports to its range observer (see NoRangeObserver). + */ +enum class KernelRange { + cos_lazy, //!< scale_cos_lazy: one range per kColumnBlockWords fold words. + cos_mask, //!< scale_cos_mask: one range per kCosMaskRangeBlocks stored mask blocks. + fused_gather, //!< apply_fused_contract insert snapshots: one range per kFusedRecordRange records. + fused_apply, //!< apply_fused_contract rotations: one range per kFusedRecordRange records. +}; + +/*! + * \brief The observer production kernels use: it does nothing and compiles away. + * + * A kernel calls `prepare(kind, ranges)` once on the calling thread before any range runs, then + * `visit(kind, range)` at the start of each logical range, from whichever thread owns that range. Tests + * substitute an observer that records the worker of each range; that seam is the only reason the kernels + * take an observer, and no production caller passes one. + */ +struct NoRangeObserver { + auto prepare(KernelRange /*kind*/, size_t /*ranges*/) const noexcept -> void {} + auto visit(KernelRange /*kind*/, size_t /*range*/) const noexcept -> void {} +}; + +//! Stored mask blocks (one 64-bit word each) per scale_cos_mask range: as many words as a fold block. +inline constexpr size_t kCosMaskRangeBlocks = kColumnBlockWords; + +//! Number of `per_range`-sized logical ranges covering `items`. +inline constexpr auto logical_ranges(size_t items, size_t per_range) -> size_t { + return (items + per_range - 1) / per_range; +} + // Reconstruct a layer's generator Monomial from the raw words stored on its LayerCore. template inline auto generator_from_words(const std::vector &gw) -> Monomial { @@ -207,24 +240,47 @@ inline auto cos_indices_mask(const CosMask &cos, std::vector &out) -> } } -template +/*! + * \brief Multiply every coefficient in a layer's recomputed cosine set by `cos_val`. + * + * Each kColumnBlockWords fold block is one logical range, so the ranges touch disjoint coefficients. + * + * \param sc The inverted index the fold reads; must not change during the call. + * \param r The layer's fold recipe. + * \param coeff Coefficients, covering at least `r.fold.mask_words * 64` rows or the index rows. + * \param cos_val The factor. + * \param options Thread budget. + * \param observer Test seam; see NoRangeObserver. + */ +template auto scale_cos_lazy(const InvertedIndex &sc, const LazyFold &r, double *coeff, double cos_val, - [[maybe_unused]] parallel::Options options = {}) -> void { + parallel::Options options = {}, + const Observer &observer = {}) -> void { const size_t mask_words = r.fold.mask_words; - const uint64_t *row_parity = fold_row_parity(sc, r.fold); - std::vector &blk = column_block_scratch(); - for (size_t bb = 0; bb < mask_words; bb += kColumnBlockWords) { + // Fetched here, on the caller, for this call only: row_parity_words() builds its cache lazily, so it must + // never run in a worker, and the pointer moves whenever the index grows. + const uint64_t *const row_parity = fold_row_parity(sc, r.fold); + const std::span columns{r.columns.data(), r.columns.size()}; + const size_t ranges = logical_ranges(mask_words, kColumnBlockWords); + observer.prepare(KernelRange::cos_lazy, ranges); + parallel::for_blocks(ranges, options, [&](size_t range) { + observer.visit(KernelRange::cos_lazy, range); + // Worker-private: never the caller's thread_local column_block_scratch(). combine_columns_block + // writes every word of [bb, be) before any is read. + std::array blk; + const size_t bb = range * kColumnBlockWords; const size_t be = std::min(bb + kColumnBlockWords, mask_words); - combine_columns_block(sc, {r.columns.data(), r.columns.size()}, blk.data(), bb, be); + combine_columns_block(sc, columns, blk.data(), bb, be); + // apply_fold_mask compares wi with the fold's global last word, so only the real tail is masked. for (size_t wi = bb; wi < be; ++wi) { for_each_cos_index(wi * 64, recipe_fold_word(r, blk.data(), bb, wi, row_parity), [&](size_t i) { coeff[i] *= cos_val; }); } - } + }); } template @@ -253,15 +309,35 @@ auto accumulate_cos_lazy(const InvertedIndex &sc, return loc; } -inline auto scale_cos_mask(double *coeff, - const CosMask &cos, - double cos_val, - [[maybe_unused]] parallel::Options options = {}) -> void { +/*! + * \brief Multiply every coefficient in a stored cosine mask by `cos_val`, leaving all others untouched. + * + * Each kCosMaskRangeBlocks run of mask blocks is one logical range. CosMask blocks are ascending with + * distinct word-aligned bases, so the ranges touch disjoint coefficients. + * + * \param coeff Coefficients, covering every index in `cos`. + * \param cos The mask. + * \param cos_val The factor. + * \param options Thread budget. + * \param observer Test seam; see NoRangeObserver. + */ +template +auto scale_cos_mask(double *coeff, + const CosMask &cos, + double cos_val, + parallel::Options options = {}, + const Observer &observer = {}) -> void { const size_t n = cos.blocks.size(); - for (size_t k = 0; k < n; ++k) { - const auto [base, bits] = cos.blocks[k]; - for_each_cos_index(base, bits, [&](size_t i) { coeff[i] *= cos_val; }); - } + const size_t ranges = logical_ranges(n, kCosMaskRangeBlocks); + observer.prepare(KernelRange::cos_mask, ranges); + parallel::for_blocks(ranges, options, [&](size_t range) { + observer.visit(KernelRange::cos_mask, range); + const size_t hi = std::min(n, (range + 1) * kCosMaskRangeBlocks); + for (size_t k = range * kCosMaskRangeBlocks; k < hi; ++k) { + const auto [base, bits] = cos.blocks[k]; + for_each_cos_index(base, bits, [&](size_t i) { coeff[i] *= cos_val; }); + } + }); } inline auto accumulate_cos_mask(double *state, double *ham, diff --git a/cpp/monoprop/detail/evolution/layer_build/FusedApply.h b/cpp/monoprop/detail/evolution/layer_build/FusedApply.h index 6978e667..aa0b1b2e 100644 --- a/cpp/monoprop/detail/evolution/layer_build/FusedApply.h +++ b/cpp/monoprop/detail/evolution/layer_build/FusedApply.h @@ -14,53 +14,63 @@ #pragma once +#include +#include #include +#include +#include #include "monoprop/TypeAliases.h" #include "monoprop/detail/evolution/CosineRecompute.h" #include "monoprop/detail/evolution/layer_build/Common.h" #include "monoprop/detail/parallel/Options.h" +#include "monoprop/detail/parallel/Workshare.h" namespace monoprop::detail { -// The drain paired with build_layer's fused emission: complete each rotation by adding its sine term -// directly to op_coeffs (the ContractImmediately forward path at all rank counts). The gate's cosine -// scale reaches the coefficients two ways: -// • fused_scale (no length cap, default): the scan already scaled every anticommuting coeff, so no cos pass runs -// here; slots born after that sweep (fresh inserts) fold cos in via their apply arm below. -// • two-pass (length cap / cos==0 fallback): scale_cos_mask runs here, then every arm is a plain add. -// At R>1 each rank applies only the add to the slot it owns (half rotations in fc.cross_half). -inline auto apply_fused_contract(FusedContract &fc, - VecD &op_coeffs, - const CosMask &cos, - double param, - bool schrodinger, - bool fused_scale, - parallel::Options options = {}) -> void { - // (1) insert records: v_tgt is the freshly-inserted term's pre-cos coeff, readable only now op_coeffs - // is extended. Needed only in Schrödinger — a Heisenberg fresh insert has coeff 0, so skip the gather. - if (schrodinger) { - for (size_t k = 0; k < fc.inserts.size(); ++k) { - fc.inserts[k].v_tgt = op_coeffs[fc.inserts[k].tgt]; +//! Records per apply_fused_contract logical range, for both the insert gather and the rotation pass. +inline constexpr size_t kFusedRecordRange = 1024; + +/*! + * \brief Whether every coefficient slot the contract writes has exactly one writing record. + * + * The slots are the sources and targets of `hits` and `inserts` and the local slots of `cross_half`. + * apply_fused_contract's unsynchronized parallel pass relies on this; debug builds assert it on every + * call. Sorts a transient copy of the slot IDs: O(records log records) time and O(records) memory. + */ +inline auto fused_add_owners_unique(const FusedContract &fc) -> bool { + std::vector slots; + slots.reserve(2 * (fc.hits.size() + fc.inserts.size()) + fc.cross_half.size()); + for (const auto *records : {&fc.hits, &fc.inserts}) { + for (const RotationRec &r : *records) { + slots.push_back(r.src); + slots.push_back(r.tgt); } } - - // (2) Two-pass mode only: cos scale over all anticommuting endpoints (inserts included). - const double cos_val = std::cos(2 * param); - const double sin_val = std::sin(2 * param); - double *const c = op_coeffs.data(); - if (!fused_scale) { - scale_cos_mask(c, cos, cos_val, options); + for (const HalfRotationRec &h : fc.cross_half) { + slots.push_back(h.local_idx); } + std::ranges::sort(slots); + return std::ranges::adjacent_find(slots) == slots.end(); +} - // (3) One pass over hits ++ inserts ++ cross_half. Each op slot is touched by exactly one add (pivot - // split + ⊕G-injective targets + drop_matched_cross_rank_followers), which is what makes the plain += - // safe. In fused_scale mode a slot born after the sweep (insert targets, resolver miss halves) folds - // the gate's cos in here (c = cos·c + sin). +/*! + * \brief Apply records `[lo, hi)` of hits ++ inserts ++ cross_half, each its complete rotation formula. + * + * Out of line on purpose: the serial and every threaded range call this one compiled body, so however the + * compiler contracts `cos·c + sin·φ·v` into a fused multiply-add, the choice is the same at every budget + * and serial and threaded results stay bitwise equal. The cost is one call per kFusedRecordRange records. + */ +[[gnu::noinline]] inline auto apply_fused_record_range(const FusedContract &fc, + double *c, + size_t lo, + size_t hi, + double cos_val, + double sin_val, + bool fused_scale) -> void { const size_t n_hit = fc.hits.size(); const size_t n_full = n_hit + fc.inserts.size(); - const size_t n_cross = fc.cross_half.size(); - for (size_t k = 0; k < n_full + n_cross; ++k) { + for (size_t k = lo; k < hi; ++k) { if (k < n_full) { const bool is_insert = k >= n_hit; const RotationRec &r = is_insert ? fc.inserts[k - n_hit] : fc.hits[k]; @@ -84,4 +94,72 @@ inline auto apply_fused_contract(FusedContract &fc, } } +/*! + * \brief The drain paired with build_layer's fused emission: complete each rotation by adding its sine term + * directly to op_coeffs (the ContractImmediately forward path at all rank counts). + * + * The gate's cosine scale reaches the coefficients two ways: + * - fused_scale (no length cap, default): the scan already scaled every anticommuting coeff, so no cos pass + * runs here; slots born after that sweep (fresh inserts) fold cos in via their apply arm below. + * - two-pass (length cap / cos==0 fallback): scale_cos_mask runs here, then every arm is a plain add. + * At R>1 each rank applies only the add to the slot it owns (half rotations in fc.cross_half). + * + * The caller has already sized and extended op_coeffs. The three phases below run in order, each joined + * before the next, and each is split into fixed logical ranges that run on up to `options.threads` workers. + * + * \param observer Test seam; see NoRangeObserver. + */ +template +auto apply_fused_contract(FusedContract &fc, + VecD &op_coeffs, + const CosMask &cos, + double param, + bool schrodinger, + bool fused_scale, + parallel::Options options = {}, + const Observer &observer = {}) -> void { + double *const c = op_coeffs.data(); + + // (1) insert records: v_tgt is the freshly-inserted term's pre-cos coeff, readable only now op_coeffs + // is extended. Needed only in Schrödinger — a Heisenberg fresh insert has coeff 0, so skip the gather. + if (schrodinger) { + RotationRec *const inserts = fc.inserts.data(); + const size_t n_inserts = fc.inserts.size(); + const size_t ranges = logical_ranges(n_inserts, kFusedRecordRange); + observer.prepare(KernelRange::fused_gather, ranges); + // Each range writes only its own records' snapshot field and reads coefficients nothing writes yet. + parallel::for_blocks(ranges, options, [&](size_t range) { + observer.visit(KernelRange::fused_gather, range); + const size_t hi = std::min(n_inserts, (range + 1) * kFusedRecordRange); + for (size_t k = range * kFusedRecordRange; k < hi; ++k) { + inserts[k].v_tgt = c[inserts[k].tgt]; + } + }); + } + + // (2) Two-pass mode only: cos scale over all anticommuting endpoints (inserts included). + const double cos_val = std::cos(2 * param); + const double sin_val = std::sin(2 * param); + if (!fused_scale) { + scale_cos_mask(c, cos, cos_val, options, observer); + } + + // (3) One pass over hits ++ inserts ++ cross_half. Each op slot is touched by exactly one add (pivot + // split + ⊕G-injective targets + drop_matched_cross_rank_followers), which is what makes the plain += + // safe. In fused_scale mode a slot born after the sweep (insert targets, resolver miss halves) folds + // the gate's cos in here (c = cos·c + sin). + assert(fused_add_owners_unique(fc)); + const size_t n_records = fc.hits.size() + fc.inserts.size() + fc.cross_half.size(); + const size_t ranges = logical_ranges(n_records, kFusedRecordRange); + observer.prepare(KernelRange::fused_apply, ranges); + // Distinct records write distinct slots (the one-add-owner property above), and every value a record + // reads is its own snapshot or its own slot, so the ranges need no synchronization. Never paper over a + // violation with atomics: it would mean the scan emitted a slot twice. + parallel::for_blocks(ranges, options, [&](size_t range) { + observer.visit(KernelRange::fused_apply, range); + const size_t lo = range * kFusedRecordRange; + apply_fused_record_range(fc, c, lo, std::min(n_records, lo + kFusedRecordRange), cos_val, sin_val, fused_scale); + }); +} + } // namespace monoprop::detail diff --git a/cpp/tests/CMakeLists.txt b/cpp/tests/CMakeLists.txt index 6c3f83be..4d89a5a8 100644 --- a/cpp/tests/CMakeLists.txt +++ b/cpp/tests/CMakeLists.txt @@ -145,6 +145,22 @@ monoprop_add_launch_env_case( openmp_propagator_work_does_not_change_affinity ENVIRONMENT "OMP_PROC_BIND=false" ) +# The threaded cosine and fused-apply kernels under runtime-limited and dynamic teams: exact results and +# participation must not depend on receiving the full budget. +set( + _monoprop_kernel_cases + "openmp_cos_mask_exact,openmp_cos_mask_workers_participate,openmp_cos_lazy_workers_participate,openmp_fused_apply_exact_on_disjoint_records,openmp_fused_apply_workers_participate,openmp_kernel_worker_failures_join_before_rethrow,combined_threaded_lazy_fold_matches_oracle,fused_real_records_have_one_add_owner_and_apply_exactly,fused_threaded_prototype_matches_serial_prototype_exactly" +) +monoprop_add_launch_env_case( + openmp_env_kernels_thread_limit_2 + "${_monoprop_kernel_cases}" + ENVIRONMENT "OMP_THREAD_LIMIT=2" +) +monoprop_add_launch_env_case( + openmp_env_kernels_dynamic_teams + "${_monoprop_kernel_cases}" + ENVIRONMENT "OMP_DYNAMIC=TRUE" +) # Distributed failure scenarios, each a separate two-rank launch with a 30 s timeout. Labelled mpi, so # only the MPI test leg runs them. @@ -206,6 +222,10 @@ if(monoprop_ENABLE_MPI AND _monoprop_mpiexec) monoprop_add_failure_scenario( active_ticket active-ticket ticket "injected failure while the replay Ticket" "UNGUARDED" ) + # Worker 1 of the kernel's two-worker team owns range 2, so the failure starts off the calling thread. + monoprop_add_failure_scenario( + active_ticket_cosine_worker active-ticket cosine-worker "cosine range 2 on OpenMP worker 1 while the replay Ticket" "UNGUARDED" + ) monoprop_add_failure_scenario( active_pending active-ticket pending "injected failure while a PendingAlltoallv" "UNGUARDED" ) diff --git a/cpp/tests/KernelTestSupport.h b/cpp/tests/KernelTestSupport.h new file mode 100644 index 00000000..1286d82c --- /dev/null +++ b/cpp/tests/KernelTestSupport.h @@ -0,0 +1,329 @@ +// Copyright 2026 Algorithmiq +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +#pragma once + +// Test-only support for the threaded cosine and fused-apply kernels: a range observer that records which +// OpenMP worker ran each logical range, a synthetic inverted index large enough to span several fold +// blocks, and an independent add-owner count over fused rotation records. +// +// Workers never assert. RangeLog slots are sized by the kernel's prepare() on the calling thread, before +// any worker starts, and each slot is written only by the sole owner of its range; tests inspect them +// after the kernel has joined. + +#include + +#include + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include "monoprop/detail/evolution/CosineRecompute.h" +#include "monoprop/detail/evolution/layer_build/Common.h" +#include "monoprop/detail/operator/InvertedIndex.h" + +namespace kernel_test { + +using monoprop::detail::KernelRange; + +inline constexpr size_t kKinds = 4; + +// What one kernel invocation reported for one kind of range. +struct RangeSlots { + size_t prepares = 0; // prepare() calls, on the calling thread + std::vector worker; // omp_get_thread_num() of each range's owner, -1 if never visited + std::vector team; // omp_get_num_threads() seen by that owner + std::vector level; // omp_get_level() seen by that owner (0: not inside any region) + std::vector visits; // visits per range; each element has one writer +}; + +struct RangeLog { + std::array slots{}; + auto operator[](KernelRange kind) -> RangeSlots & { return slots[static_cast(kind)]; } +}; + +// Records the worker that owns each logical range. Optionally throws from one range, on its worker, to +// exercise the helper's capture-and-join path from inside a real kernel body. +struct RecordingObserver { + RangeLog *log; + KernelRange throw_kind = KernelRange::cos_lazy; + size_t throw_range = SIZE_MAX; + + auto prepare(KernelRange kind, size_t ranges) const -> void { + auto &s = (*log)[kind]; + ++s.prepares; + s.worker.assign(ranges, -1); + s.team.assign(ranges, 0); + s.level.assign(ranges, 0); + s.visits.assign(ranges, 0); + } + auto visit(KernelRange kind, size_t range) const -> void { + auto &s = (*log)[kind]; + s.worker[range] = omp_get_thread_num(); + s.team[range] = omp_get_num_threads(); + s.level[range] = omp_get_level(); + ++s.visits[range]; + if (kind == throw_kind && range == throw_range) { + throw std::runtime_error("injected kernel worker failure"); + } + } +}; + +// Evidence that one kernel's own ranges ran on several workers of a region the kernel opened. +struct Participation { + size_t ranges = 0; + size_t distinct_workers = 0; + int team = 0; + bool every_range_once = false; + bool inside_region = false; // every range saw omp_get_level() >= 1 +}; + +inline auto participation(const RangeSlots &s) -> Participation { + Participation p; + p.ranges = s.worker.size(); + p.every_range_once = std::ranges::all_of(s.visits, [](int v) { return v == 1; }); + p.inside_region = !s.level.empty() && std::ranges::all_of(s.level, [](int l) { return l >= 1; }); + const std::set distinct(s.worker.begin(), s.worker.end()); + p.distinct_workers = distinct.size(); + p.team = s.team.empty() ? 0 : *std::ranges::max_element(s.team); + return p; +} + +// Whether a two-worker team is a fair expectation: a dynamic runtime or a thread limit of one may shrink +// the region, which is then reported, never counted as participation. +inline auto team_may_shrink() -> bool { + return omp_get_dynamic() != 0 || omp_get_thread_limit() < 2; +} + +// Checks on the caller that the kernel's ranges ran inside its own region, each exactly once, on as many +// distinct workers as the region had, and on at least two of them unless the runtime may shrink teams. +inline auto check_participation(const RangeSlots &s, size_t expected_ranges, const char *label) -> void { + const auto p = participation(s); + BOOST_TEST_CONTEXT(label << ": ranges=" << p.ranges << " distinct_workers=" << p.distinct_workers + << " team=" << p.team) { + BOOST_TEST(s.prepares == 1u); + BOOST_TEST(p.ranges == expected_ranges); + BOOST_TEST(p.every_range_once); + BOOST_TEST(p.inside_region); + // A static schedule over at least as many ranges as workers gives every worker a range. + BOOST_TEST(p.distinct_workers == static_cast(p.team)); + if (team_may_shrink()) { + BOOST_WARN_MESSAGE(p.team >= 2, + label << ": multi-worker participation NOT demonstrated (team " << p.team + << " under a dynamic or limited runtime)"); + } + else { + BOOST_TEST(p.team >= 2); + } + } + BOOST_TEST_MESSAGE(label << ": " << p.distinct_workers << " distinct workers over " << p.ranges << " ranges"); +} + +// Checks that a kernel stayed on the calling thread: every range visited once, outside any region. +inline auto check_serial(const RangeSlots &s, size_t expected_ranges, const char *label) -> void { + const auto p = participation(s); + BOOST_TEST_CONTEXT(label) { + BOOST_TEST(s.prepares == 1u); + BOOST_TEST(p.ranges == expected_ranges); + BOOST_TEST(p.every_range_once); + BOOST_TEST(std::ranges::all_of(s.level, [](int l) { return l == 0; })); + } +} + +// A test-side probe, not library behavior: can this runtime give a two-thread region right now? +inline auto runtime_offers_two_workers(boost::unit_test::test_unit_id) -> boost::test_tools::assertion_result { + if (omp_get_thread_limit() < 2) { + boost::test_tools::assertion_result result(false); + result.message() << "OMP_THREAD_LIMIT is below two"; + return result; + } + int team = 0; +#pragma omp parallel num_threads(2) + { +#pragma omp single + team = omp_get_num_threads(); + } + boost::test_tools::assertion_result result(team >= 2); + result.message() << "the OpenMP runtime provided " << team << " worker(s) for a two-thread request"; + return result; +} + +// Sets monoprop_NUM_THREADS for one scope, so an object constructed inside captures that budget, then +// restores the previous value. Only the one-store prototype (explicit partitions=1 on an ordinary +// communicator) captures it; tests confirm selection through PropagatorTestAccess::options. +class ScopedBudget { +public: + explicit ScopedBudget(int threads) { + if (const char *old = std::getenv(kVariable)) { + old_ = std::string(old); + } + ::setenv(kVariable, std::to_string(threads).c_str(), 1); + } + ScopedBudget(const ScopedBudget &) = delete; + auto operator=(const ScopedBudget &) -> ScopedBudget & = delete; + ~ScopedBudget() { + if (old_) { + ::setenv(kVariable, old_->c_str(), 1); + } + else { + ::unsetenv(kVariable); + } + } + +private: + static constexpr const char *kVariable = "monoprop_NUM_THREADS"; + std::optional old_; +}; + +// Deterministic, library-independent pseudo-random words (splitmix64). +struct SplitMix { + uint64_t state; + auto next() -> uint64_t { + uint64_t z = (state += 0x9e3779b97f4a7c15ULL); + z = (z ^ (z >> 30)) * 0xbf58476d1ce4e5b9ULL; + z = (z ^ (z >> 27)) * 0x94d049bb133111ebULL; + return z ^ (z >> 31); + } + auto below(uint64_t n) -> uint64_t { return next() % n; } +}; + +inline constexpr size_t kIndexModes = 8; // 16 columns + +// A synthetic InvertedIndex<8> over `rows` rows with every column tier a fold can meet: dense random +// columns, sparse columns (including rows straddling fold-block edges), a dense-but-sparse column and +// empty columns. Consistent with an operator whose row r touches column c iff bit r of column c is set, +// so its lazily built row parity is well defined. Bits beyond `rows` stay clear, as in the real index. +inline auto make_synthetic_index(size_t rows, uint64_t seed) -> monoprop::detail::InvertedIndex { + using Index = monoprop::detail::InvertedIndex; + Index index; + index.row_count = rows; + const size_t words = index.words(); + const uint64_t last_mask = rows % 64 == 0 ? ~uint64_t{0} : (uint64_t{1} << (rows % 64)) - 1; + SplitMix rng{seed}; + const auto dense = [&](size_t c, int sparsity_shift) { + auto &col = index.cols[c]; + col.is_dense = true; + col.words.assign(words, 0); + for (auto &w : col.words) { + w = rng.next(); + for (int s = 0; s < sparsity_shift; ++s) { + w &= rng.next(); + } + } + if (words != 0) { + col.words.back() &= last_mask; + } + }; + const auto sparse = [&](size_t c, uint64_t one_in) { + auto &col = index.cols[c]; + for (size_t r = 0; r < rows; ++r) { + if (rng.below(one_in) == 0) { + col.set_rows.push_back(static_cast(r)); + } + } + }; + for (size_t c = 0; c < 6; ++c) { + dense(c, 0); // density 1/2 + } + sparse(6, 200); + sparse(7, 997); + // Rows straddling every fold-block edge, so a block boundary splits a sparse column's row list. + { + auto &col = index.cols[8]; + const size_t block_rows = monoprop::detail::kColumnBlockWords * 64; + for (size_t edge = block_rows; edge < rows; edge += block_rows) { + for (size_t r = edge - 3; r < std::min(rows, edge + 3); ++r) { + col.set_rows.push_back(static_cast(r)); + } + } + if (rows != 0) { + col.set_rows.push_back(static_cast(rows - 1)); // the global tail row + } + std::ranges::sort(col.set_rows); + col.set_rows.erase(std::ranges::unique(col.set_rows).begin(), col.set_rows.end()); + } + dense(9, 4); // dense representation, density 1/32 + // Columns 10 and 11 stay empty. + sparse(12, 64); + dense(13, 1); // density 1/4 + sparse(14, 5000); + dense(15, 0); + return index; +} + +// Append rows [index.rows(), new_rows) the way operator growth does: new bits for dense columns (including +// the old tail word's unused bits), ascending new rows for sparse ones. Existing rows keep their columns. +// Does not touch row_parity_; the caller decides whether to drop or keep it. +inline auto grow_synthetic_index(monoprop::detail::InvertedIndex &index, size_t new_rows, uint64_t seed) + -> void { + const size_t old_rows = index.row_count; + index.row_count = new_rows; + const size_t words = index.words(); + SplitMix rng{seed}; + for (auto &col : index.cols) { + if (col.is_dense) { + col.words.resize(words, 0); + for (size_t r = old_rows; r < new_rows; ++r) { + if (rng.below(3) == 0) { + col.words[r >> 6] |= uint64_t{1} << (r & 63U); + } + } + } + else if (!col.set_rows.empty()) { + for (size_t r = old_rows; r < new_rows; ++r) { + if (rng.below(150) == 0) { + col.set_rows.push_back(static_cast(r)); + } + } + } + } +} + +// A Majorana/Pauli generator over the synthetic index's 16 columns. +inline auto generator(std::initializer_list columns) -> monoprop::Monomial { + monoprop::Monomial gen{}; + for (const auto c : columns) { + gen.set(c); + } + return gen; +} + +// Independent add-owner count: every slot a fused contract writes (sources and targets of full +// rotations, local slots of half rotations) mapped to how many records write it. +inline auto add_owner_counts(const monoprop::detail::FusedContract &fc) -> std::map { + std::map owners; + for (const auto &r : fc.hits) { + ++owners[r.src]; + ++owners[r.tgt]; + } + for (const auto &r : fc.inserts) { + ++owners[r.src]; + ++owners[r.tgt]; + } + for (const auto &h : fc.cross_half) { + ++owners[h.local_idx]; + } + return owners; +} + +} // namespace kernel_test diff --git a/cpp/tests/PropagatorTestAccess.h b/cpp/tests/PropagatorTestAccess.h index f8f05984..c3b5ed1a 100644 --- a/cpp/tests/PropagatorTestAccess.h +++ b/cpp/tests/PropagatorTestAccess.h @@ -23,6 +23,7 @@ #include #include "monoprop/MonomialPropagator.h" +#include "monoprop/detail/evolution/layer_build/Common.h" #include "monoprop/detail/parallel/Options.h" namespace monoprop::detail { @@ -47,6 +48,41 @@ struct PropagatorTestAccess { static auto clone(const Propagator &p) -> std::unique_ptr { return p.clone_(); } + // The live coefficients of the propagator's picture: the operator (Heisenberg) or the state (Schrödinger). + static auto picture_coeffs(const Propagator &p) -> const VecD & { + return p.schrodinger_ ? p.mp_op_.state_coeffs : p.mp_op_.op_coeffs; + } + + // One gate of the ContractImmediately path (evolve_mode_contract_immediately_) up to, not including, + // apply_fused_contract: the real build and coefficient extension, so a test can inspect the records the + // build produced and run the apply itself on the returned coefficient vector. + struct ContractedGate { + FusedContract fc; + CosMask cos; + bool fused_scale = false; + VecD *coeffs = nullptr; // the picture's live coefficients, already extended + double apply_angle = 0.0; + }; + static auto contract_gate(Propagator &p, + const VecZ &gate, + std::optional only_rotate_len_k, + double build_angle) -> ContractedGate { + (void)p.current_picture_coeffs_(); + ContractedGate g; + g.coeffs = p.schrodinger_ ? &p.mp_op_.state_coeffs : &p.mp_op_.op_coeffs; + (void)p.build_evolve_result_(gate, + only_rotate_len_k, + std::cref(*g.coeffs), + build_angle, + &g.cos, + &g.fc, + g.coeffs, + &g.fused_scale); + p.extend_coeffs_from_current_picture_if_needed_(*g.coeffs); + g.apply_angle = p.schrodinger_ ? -build_angle : build_angle; + return g; + } + static auto partition_count(const Propagator &p) -> int { return p.partition_group_ ? p.partition_group_->partition_count() : 0; } diff --git a/cpp/tests/README.md b/cpp/tests/README.md index f3dd71ca..312fdbea 100644 --- a/cpp/tests/README.md +++ b/cpp/tests/README.md @@ -138,10 +138,22 @@ name and cannot address suite-nested cases, tests use flat calls default to serial options, and that a failed mutation or evaluation invalidates the owner while validation errors do not. `cpp/tests/CMakeLists.txt` reruns some of these cases as `openmp_env_*` entries, each in a fresh process with a fixed launch environment. +- **Threaded kernels**: `openmp_kernel_tests.cpp` covers the threaded cosine scaling + (`scale_cos_mask`, `scale_cos_lazy`) and fused rotation apply (`apply_fused_contract`): bitwise + agreement of every budget with budget one, small work staying serial, joined worker exceptions, + and participation. Participation is read from a test-only range observer (`KernelTestSupport.h`) + that each kernel calls at the start of its own logical ranges; production code passes the empty + `NoRangeObserver`, so a team opened anywhere else never counts. Lazy folds over a synthetic + multi-block inverted index are checked against the materialised-fold oracle in + `combined_recompute_equivalence.cpp` (cold parity cache, tails, index growth), and fused records + from the real build path (one add-owner per slot, both pictures, cross-rank halves under MPI) in + `fused_cos_sweep_tests.cpp`. `openmp_env_kernels_*` rerun them under `OMP_THREAD_LIMIT=2` and + `OMP_DYNAMIC=TRUE`. - **Distributed failure driver**: `mpi_failure_driver.cpp` is not part of the unit runner (the glob excludes it). The MPI build compiles it into `monoprop_mpi_failure_driver.x`, and `run_mpi_failure_scenario.cmake` runs each scenario on two ranks with a 30 s timeout, as the - `mpi_failure_*` CTest entries. + `mpi_failure_*` CTest entries. `active-ticket cosine-worker` throws from a worker of the threaded + cosine kernel while a replay Ticket is posted. New `*.cpp` files are auto-discovered on the next configure, with no CMake edit needed. A file with its own `main()` must be excluded from the glob, as diff --git a/cpp/tests/combined_recompute_equivalence.cpp b/cpp/tests/combined_recompute_equivalence.cpp index 6196a0a8..bc66a12a 100644 --- a/cpp/tests/combined_recompute_equivalence.cpp +++ b/cpp/tests/combined_recompute_equivalence.cpp @@ -24,6 +24,7 @@ #include #include +#include "KernelTestSupport.h" #include "TestUtilities.h" #include "monoprop/detail/evolution/CosineRecompute.h" @@ -113,11 +114,15 @@ BOOST_AUTO_TEST_CASE(combined_scale_cache_equals_recompute) { std::vector a = baseline; std::vector b = baseline; + std::vector threaded = baseline; scale_cos_cached(prepared, a.data(), cos_val); monoprop::detail::scale_cos_lazy(inverted_index, recipe, b.data(), cos_val); + monoprop::detail::scale_cos_lazy(inverted_index, recipe, threaded.data(), cos_val, {.threads = 3}); BOOST_TEST_INFO("layer " << li); BOOST_TEST(std::memcmp(a.data(), b.data(), n * sizeof(double)) == 0); + BOOST_TEST_INFO("layer " << li << " threads 3"); + BOOST_TEST(std::memcmp(a.data(), threaded.data(), n * sizeof(double)) == 0); } // The fixture must actually exercise the odd-|G| parity correction, or the guardrail is hollow. BOOST_TEST(odd_layers > 0u); @@ -341,9 +346,155 @@ BOOST_AUTO_TEST_CASE(lazy_fold_survives_operator_growth) { auto prepared = monoprop::detail::make_fold_cache(sim.mp_op().inverted_index(), gen, scaled_count, kBasis); std::vector expected = baseline; - std::vector actual = baseline; scale_cos_cached(prepared, expected.data(), cos_val); - monoprop::detail::scale_cos_lazy(sim.mp_op().inverted_index(), recipe, actual.data(), cos_val); + for (const int threads : {1, 4}) { + std::vector actual = baseline; + monoprop::detail::scale_cos_lazy(sim.mp_op().inverted_index(), + recipe, + actual.data(), + cos_val, + {.threads = threads}); + BOOST_TEST_INFO("threads " << threads); + BOOST_TEST(std::memcmp(expected.data(), actual.data(), n * sizeof(double)) == 0); + } +} +// --- threaded lazy folds over several fold blocks ----------------------------------------------------------- +// +// The real fixtures above fit in one kColumnBlockWords block, so scale_cos_lazy stays serial on them. These +// use a synthetic InvertedIndex<8> spanning more than three blocks, with dense, sparse, dense-but-sparse and +// empty columns (KernelTestSupport.h), and require every budget to match the materialised-fold oracle +// bitwise. + +namespace { + +struct FoldCase { + const char *label; + Monomial gen; + Basis basis; +}; + +auto fold_cases() -> std::vector { + using kernel_test::generator; + return { + {"majorana even, dense", generator({1, 4}), Basis::Majorana}, + {"majorana even, mixed tiers", generator({0, 2, 9, 13}), Basis::Majorana}, + {"majorana odd, dense + sparse", generator({0, 3, 7}), Basis::Majorana}, + {"majorana odd, sparse + block edges", generator({6, 8, 14}), Basis::Majorana}, + {"majorana even, empty columns", generator({10, 11}), Basis::Majorana}, + {"majorana odd, one empty column (parity only)", generator({10}), Basis::Majorana}, + {"zero generator", generator({}), Basis::Majorana}, + {"pauli J(G), two columns", generator({0, 5}), Basis::Pauli}, + {"pauli J(G), three columns", generator({3, 8, 15}), Basis::Pauli}, + }; +} + +// Threaded first, on a cold parity cache: the kernel must build row parity on the caller before its team +// starts. Only then are the one-thread run and the oracle (whose make_fold_cache warms the cache) computed. +auto check_threaded_lazy_fold(const monoprop::detail::InvertedIndex &index, + const FoldCase &fc, + uint64_t scaled_count) -> void { + constexpr size_t kModes = kernel_test::kIndexModes; + const size_t n = index.rows(); + const auto recipe = monoprop::detail::make_lazy_fold(index, fc.gen, scaled_count, fc.basis); + const std::vector baseline = [n] { + std::vector v(n); + for (size_t i = 0; i < n; ++i) { + v[i] = 1.0 + static_cast(i) * 1e-3; + } + return v; + }(); + const double cos_val = 0.6234; + + std::vector> threaded; + for (const int threads : {2, 3, 4}) { + index.row_parity_.clear(); + threaded.push_back(baseline); + monoprop::detail::scale_cos_lazy(index, recipe, threaded.back().data(), cos_val, {.threads = threads}); + } + index.row_parity_.clear(); + std::vector serial = baseline; + monoprop::detail::scale_cos_lazy(index, recipe, serial.data(), cos_val, {.threads = 1}); + + const auto prepared = monoprop::detail::make_fold_cache(index, fc.gen, scaled_count, fc.basis); + std::vector oracle = baseline; + scale_cos_cached(prepared, oracle.data(), cos_val); + + BOOST_TEST_CONTEXT(fc.label << " rows=" << n << " scaled_count=" << scaled_count) { + BOOST_TEST(std::memcmp(oracle.data(), serial.data(), n * sizeof(double)) == 0); + for (size_t t = 0; t < threaded.size(); ++t) { + BOOST_TEST_INFO("threads " << t + 2); + BOOST_TEST(std::memcmp(oracle.data(), threaded[t].data(), n * sizeof(double)) == 0); + } + // Rows at or beyond scaled_count are never scaled. + size_t scaled_beyond = 0; + for (size_t i = std::min(scaled_count, n); i < n; ++i) { + scaled_beyond += static_cast(serial[i] != baseline[i]); + } + BOOST_TEST(scaled_beyond == 0u); + } +} + +} // namespace + +BOOST_AUTO_TEST_CASE(combined_threaded_lazy_fold_matches_oracle) { + constexpr size_t block_rows = monoprop::detail::kColumnBlockWords * 64; + // More than three fold blocks, with 63-, 64- and 65-row tails past a word boundary. + const size_t base = 3 * block_rows + 7 * 64; + for (const size_t tail : {63, 64, 65}) { + const size_t rows = base + tail; + const auto index = kernel_test::make_synthetic_index(rows, 1000 + tail); + size_t odd_folds = 0; + for (const auto &fc : fold_cases()) { + odd_folds += static_cast( + monoprop::algebra_fold_needs_odd_correction(fc.basis, fc.gen)); + // Full; a partial final word; fewer blocks than the index; a single word in the second range; + // nothing at all. + for (const uint64_t scaled_count : {uint64_t{rows}, + uint64_t{rows - 100}, + uint64_t{2 * block_rows + 17}, + uint64_t{block_rows + 1}, + uint64_t{0}}) { + check_threaded_lazy_fold(index, fc, scaled_count); + } + } + // The odd-|G| parity correction and the Pauli fold are both exercised. + BOOST_TEST(odd_folds > 0u); + } +} + +// A retained LazyFold across index growth, threaded: the parity words the workers read must be fetched for +// this call, from the grown index, never retained from before the growth (see LazyFold). +BOOST_AUTO_TEST_CASE(combined_threaded_lazy_fold_survives_index_growth) { + constexpr size_t kModes = kernel_test::kIndexModes; + constexpr size_t block_rows = monoprop::detail::kColumnBlockWords * 64; + const size_t old_rows = 2 * block_rows + 500; + auto index = kernel_test::make_synthetic_index(old_rows, 77); + const auto gen = kernel_test::generator({0, 6, 8}); + const uint64_t scaled_count = old_rows - 3; + const auto recipe = monoprop::detail::make_lazy_fold(index, gen, scaled_count, Basis::Majorana); + + const uint64_t *before = index.row_parity_words(); + BOOST_REQUIRE(before != nullptr); + // Keep the old allocation alive and poisoned, so a stale pointer would be observably wrong. + auto old_row_parity = std::move(index.row_parity_); + for (auto &word : old_row_parity) { + word = ~word; + } + kernel_test::grow_synthetic_index(index, 4 * block_rows + 123, 78); + const size_t n = index.rows(); + + std::vector baseline(n); + for (size_t i = 0; i < n; ++i) { + baseline[i] = 1.0 + static_cast(i) * 1e-3; + } + const double cos_val = 0.6234; + std::vector actual = baseline; + monoprop::detail::scale_cos_lazy(index, recipe, actual.data(), cos_val, {.threads = 4}); + BOOST_TEST(index.row_parity_words() != before); + + const auto prepared = monoprop::detail::make_fold_cache(index, gen, scaled_count, Basis::Majorana); + std::vector expected = baseline; + scale_cos_cached(prepared, expected.data(), cos_val); BOOST_TEST(std::memcmp(expected.data(), actual.data(), n * sizeof(double)) == 0); } diff --git a/cpp/tests/fused_cos_sweep_tests.cpp b/cpp/tests/fused_cos_sweep_tests.cpp index a8d36ea2..fd982a79 100644 --- a/cpp/tests/fused_cos_sweep_tests.cpp +++ b/cpp/tests/fused_cos_sweep_tests.cpp @@ -14,12 +14,27 @@ #include +#include +#include +#include +#include +#include +#include #include +#include +#include "KernelTestSupport.h" +#include "PropagatorTestAccess.h" #include "TestUtilities.h" +#include "monoprop/MPFunctions.h" +#include "monoprop/detail/evolution/layer_build/FusedApply.h" // The fused cos sweep's one deliberate FP deviation from the two-pass path (≤1 ulp per hit endpoint, // from resolve's stored·(1/cos) recovery), against the build_graph()+replay evaluation as oracle. +// +// The threaded cases below run the one-store prototype (explicit partitions=1, budget captured from +// monoprop_NUM_THREADS at construction). Serial and threaded runs of the same algorithm must agree bitwise; +// fused-versus-replay comparisons keep the tolerances above. namespace { @@ -47,6 +62,26 @@ auto graph_energy(const CaseData &data, const SimulatorConfig &cfg) -> double { return fn(data.parameters); } +template +auto build_prototype(const CaseData &data, const SimulatorConfig &cfg, int budget) -> MonomialPropagator { + const kernel_test::ScopedBudget scoped(budget); + MonomialPropagator sim(data.hamiltonian, + static_cast(2 * NumModes), + data.initial_state, + cfg.schrodinger_cutoff, + cfg.comm, + cfg.atol, + cfg.upper_atol, + cfg.cutoff_type, + cfg.basis_change, + NumModes, + Basis::Majorana, + /*partitions=*/1); + // Selection is checked, not assumed: only the prototype captures the environment budget. + BOOST_TEST_REQUIRE(monoprop::detail::PropagatorTestAccess::options(sim).threads == budget); + return sim; +} + void check_agreement(const CaseData &data, const SimulatorConfig &cfg, const char *label) { const double inplace = inplace_energy(data, cfg); const double graph = graph_energy(data, cfg); @@ -56,6 +91,80 @@ void check_agreement(const CaseData &data, const SimulatorConfig &cfg, const cha } } +// check_agreement on threaded prototypes, with the same tolerances. +void check_prototype_agreement(const CaseData &data, const SimulatorConfig &cfg, int budget, const char *label) { + constexpr size_t N = ExampleDataFix::n_modes; + auto inplace_sim = build_prototype(data, cfg, budget); + inplace_sim.propagate(data.majoranas, data.param_inds, data.gen_coeffs, data.parameters); + const double inplace = inplace_sim.expectation_value_functional(std::nullopt)(VecD{}); + auto graph_sim = build_prototype(data, cfg, budget); + graph_sim.build_graph(data.majoranas, data.param_inds, data.gen_coeffs); + const double graph = graph_sim.expectation_value_functional(std::nullopt)(data.parameters); + BOOST_TEST_CONTEXT(label << " budget=" << budget << " inplace=" << inplace << " graph=" << graph) { + BOOST_CHECK_SMALL(inplace - graph, kAgreeAtol); + BOOST_CHECK_SMALL(inplace - data.actual_expval, kExactAtol); + } +} + +// --- a workload large enough for the threaded kernels --------------------------------------------------------- + +constexpr size_t kBig = 10; // 20 Majoranas +using BigAccess = monoprop::detail::PropagatorTestAccess; + +// A synthetic Hermitian operator of 16k even-weight terms and gates of weights 2-4 (odd included), large +// enough that fused records and replayed folds span several logical ranges. One gate sits at theta = pi/4, +// where the fused sweep recovers pre-cos values through 1/cos ~ 1.6e16. +auto big_case() -> CaseData { + CaseData data; + data.num_modes = kBig; + kernel_test::SplitMix rng{2026}; + const auto draw_indices = [&rng](size_t weight) { + VecZ idx; + while (idx.size() < weight) { + const size_t m = rng.below(2 * kBig); + if (std::ranges::find(idx, m) == idx.end()) { + idx.push_back(m); + } + } + std::ranges::sort(idx); + return idx; + }; + while (data.hamiltonian.size() < 16000) { + const auto idx = draw_indices(2 * (1 + rng.below(3))); + const double x = static_cast(rng.below(1U << 20)) / (1U << 20) - 0.5; + data.hamiltonian[idx] = x * hermitian_coefficient(indices_to_bitset(idx)); + } + data.initial_state = {0, 2, 4, 6, 8}; + constexpr size_t gates = 9; + for (size_t g = 0; g < gates; ++g) { + data.majoranas.push_back(draw_indices(2 + rng.below(3))); + data.param_inds.push_back(g); + data.gen_coeffs.push_back(1.0); + data.parameters.push_back(g == 4 ? std::numbers::pi / 4 : 0.1 + static_cast(rng.below(900)) / 1000.0); + } + // Repeated generators: rotating again by G targets the M*G terms the first rotation inserted, so both + // pictures produce hit records as well as inserts. + data.majoranas[6] = data.majoranas[2]; + data.majoranas[8] = data.majoranas[3]; + return data; +} + +auto big_prototype(const CaseData &data, bool schrodinger, int budget, MPI_Comm comm) -> MonomialPropagator { + return build_prototype( + data, + SimulatorConfig{.schrodinger_cutoff = schrodinger ? std::optional(2 * kBig) : std::nullopt, + .comm = comm}, + budget); +} + +auto bitwise_equal(const VecD &a, const VecD &b) -> bool { + return a.size() == b.size() && std::memcmp(a.data(), b.data(), a.size() * sizeof(double)) == 0; +} + +auto comm_size(MPI_Comm comm) -> int { + return mpi::size(mpi::Comm(comm)); +} + } // namespace BOOST_FIXTURE_TEST_CASE(fused_sweep_matches_graph_replay_heisenberg, ExampleDataFix) { @@ -77,3 +186,186 @@ BOOST_FIXTURE_TEST_CASE(fused_sweep_matches_graph_replay_schrodinger, ExampleDat BOOST_FIXTURE_TEST_CASE(fused_sweep_matches_graph_replay_schrodinger_atol, ExampleDataFix) { check_agreement(data, SimulatorConfig{.schrodinger_cutoff = 2 * n_modes, .atol = 1e-10}, "schrodinger atol=1e-10"); } + +// The two-pass apply (a length cap at the full width truncates nothing but disables the fused sweep) against +// graph replay, with the same tolerances: the insert snapshots must be gathered before the cosine phase. +BOOST_FIXTURE_TEST_CASE(two_pass_apply_matches_graph_replay, ExampleDataFix) { + constexpr size_t cap = 2 * n_modes; + for (const bool schrodinger : {false, true}) { + for (const int budget : {1, 4}) { + const SimulatorConfig cfg{.schrodinger_cutoff = + schrodinger ? std::optional(2 * n_modes) : std::nullopt}; + auto inplace_sim = build_prototype(data, cfg, budget); + inplace_sim.propagate(data.majoranas, data.param_inds, data.gen_coeffs, data.parameters, cap); + const double inplace = inplace_sim.expectation_value_functional(std::nullopt)(VecD{}); + auto graph_sim = build_prototype(data, cfg, budget); + graph_sim.build_graph(data.majoranas, data.param_inds, data.gen_coeffs, std::nullopt, std::nullopt, cap); + const double graph = graph_sim.expectation_value_functional(std::nullopt)(data.parameters); + BOOST_TEST_CONTEXT("schrodinger=" << schrodinger << " budget=" << budget << " inplace=" << inplace + << " graph=" << graph) { + BOOST_CHECK_SMALL(inplace - graph, kAgreeAtol); + BOOST_CHECK_SMALL(inplace - data.actual_expval, kExactAtol); + } + } + } +} + +// The same fused-versus-replay agreement on threaded one-store prototypes. This fixture is small, so the +// kernels take their serial paths here; the big-workload cases below cover the threaded ones. +BOOST_FIXTURE_TEST_CASE(fused_sweep_matches_graph_replay_threaded_prototype, ExampleDataFix) { + for (const int budget : {1, 4}) { + check_prototype_agreement(data, SimulatorConfig{}, budget, "heisenberg"); + check_prototype_agreement(data, SimulatorConfig{.atol = 1e-10}, budget, "heisenberg atol=1e-10"); + check_prototype_agreement(data, SimulatorConfig{.schrodinger_cutoff = 2 * n_modes}, budget, "schrodinger"); + } +} + +// Records from the real ContractImmediately build path, gate by gate, in both pictures and in the fused-scale +// and two-pass (length cap) modes: every destination slot has exactly one add-owner across hits, inserts and +// cross-rank halves, and the apply at budgets 2-4 matches budget 1 bitwise. Under an MPI launch +// (MPI_COMM_WORLD with several ranks) the halves are real cross-rank records. +BOOST_AUTO_TEST_CASE(fused_real_records_have_one_add_owner_and_apply_exactly) { + const auto data = big_case(); + const int ranks = comm_size(MPI_COMM_WORLD); + const auto mapped = map_params(data.parameters, data.param_inds, data.gen_coeffs, 1.0); + for (const bool schrodinger : {false, true}) { + for (const bool length_cap : {false, true}) { + BOOST_TEST_CONTEXT("schrodinger=" << schrodinger << " length_cap=" << length_cap << " ranks=" << ranks) { + auto sim = big_prototype(data, schrodinger, 1, MPI_COMM_WORLD); + // A cap at the full width truncates nothing but forces the two-pass apply. + const std::optional cap = length_cap ? std::optional(2 * kBig) : std::nullopt; + size_t duplicate_owners = 0; + size_t hits = 0; + size_t inserts = 0; + size_t halves = 0; + size_t max_apply_ranges = 0; + size_t max_gather_ranges = 0; + size_t max_gate_records = 0; + size_t max_gate_inserts = 0; + size_t nonzero_insert_snapshots = 0; + bool fused_seen = false; + bool two_pass_seen = false; + bool mismatch = false; + const size_t n = data.majoranas.size(); + for (size_t i = 0; i < n; ++i) { + // Production gate order and angles (evolve_mode_contract_immediately_ / gate_angle_). + const size_t idx = schrodinger ? i : n - 1 - i; + auto gate = BigAccess::contract_gate(sim, data.majoranas[idx], cap, mapped[idx]); + for (const auto &[slot, count] : kernel_test::add_owner_counts(gate.fc)) { + duplicate_owners += static_cast(count != 1); + } + max_gate_records = + std::max(max_gate_records, + gate.fc.hits.size() + gate.fc.inserts.size() + gate.fc.cross_half.size()); + max_gate_inserts = std::max(max_gate_inserts, gate.fc.inserts.size()); + hits += gate.fc.hits.size(); + inserts += gate.fc.inserts.size(); + halves += gate.fc.cross_half.size(); + (gate.fused_scale ? fused_seen : two_pass_seen) = true; + + const VecD before = *gate.coeffs; + for (const auto &r : gate.fc.inserts) { + nonzero_insert_snapshots += static_cast(before[r.tgt] != 0.0); + } + const auto apply = [&](int threads, kernel_test::RangeLog *log) { + auto fc = gate.fc; + VecD out = before; + if (log != nullptr) { + monoprop::detail::apply_fused_contract(fc, + out, + gate.cos, + gate.apply_angle, + schrodinger, + gate.fused_scale, + {threads}, + kernel_test::RecordingObserver{log}); + } + else { + monoprop::detail::apply_fused_contract(fc, + out, + gate.cos, + gate.apply_angle, + schrodinger, + gate.fused_scale, + {threads}); + } + return out; + }; + const VecD serial = apply(1, nullptr); + for (const int threads : {2, 3, 4}) { + kernel_test::RangeLog log; + const bool same = bitwise_equal(apply(threads, &log), serial); + mismatch = mismatch || !same; + max_apply_ranges = + std::max(max_apply_ranges, log[monoprop::detail::KernelRange::fused_apply].worker.size()); + max_gather_ranges = + std::max(max_gather_ranges, log[monoprop::detail::KernelRange::fused_gather].worker.size()); + } + *gate.coeffs = serial; + } + BOOST_TEST_MESSAGE("records: hits=" << hits << " inserts=" << inserts << " halves=" << halves + << " max apply ranges=" << max_apply_ranges + << " max gather ranges=" << max_gather_ranges + << " nonzero insert snapshots=" << nonzero_insert_snapshots); + BOOST_TEST(duplicate_owners == 0u); + BOOST_TEST(!mismatch); + // One rank sees hits and inserts only; with several ranks the routing decides how many + // rotations stay rank-local (linear routing can send all of them across), but halves appear. + if (ranks == 1) { + BOOST_TEST(hits > 0u); + BOOST_TEST(inserts > 0u); + BOOST_TEST(halves == 0u); + } + else { + BOOST_TEST(halves > 0u); + } + // The kernel split the largest gate into the expected record ranges; on one rank this workload + // is large enough for the threaded record path, not only its serial fallback. More ranks + // divide the records and may leave a rank below two ranges. + BOOST_TEST(max_apply_ranges == monoprop::detail::logical_ranges(max_gate_records, 1024)); + BOOST_TEST(max_gather_ranges + == (schrodinger ? monoprop::detail::logical_ranges(max_gate_inserts, 1024) : 0u)); + if (ranks == 1) { + BOOST_TEST(max_apply_ranges >= 2u); + BOOST_TEST(max_gather_ranges >= (schrodinger ? 2u : 0u)); + } + BOOST_TEST(fused_seen == !length_cap); + BOOST_TEST(two_pass_seen == length_cap); + + // The helper reproduced production: a serial prototype's own propagate gives the same vector. + auto reference = big_prototype(data, schrodinger, 1, MPI_COMM_WORLD); + reference.propagate(data.majoranas, data.param_inds, data.gen_coeffs, data.parameters, cap); + BOOST_TEST(bitwise_equal(BigAccess::picture_coeffs(reference), BigAccess::picture_coeffs(sim))); + } + } + } +} + +// End to end through the public API: prototypes at budgets 1 and 4 propagate (fused apply, stored-mask and +// fused-scale paths) and replay a built graph (lazy folds) to bitwise-identical coefficients and energies. +BOOST_AUTO_TEST_CASE(fused_threaded_prototype_matches_serial_prototype_exactly) { + const auto data = big_case(); + for (const bool schrodinger : {false, true}) { + for (const bool length_cap : {false, true}) { + BOOST_TEST_CONTEXT("schrodinger=" << schrodinger << " length_cap=" << length_cap) { + const std::optional cap = length_cap ? std::optional(2 * kBig) : std::nullopt; + auto serial = big_prototype(data, schrodinger, 1, MPI_COMM_WORLD); + auto threaded = big_prototype(data, schrodinger, 4, MPI_COMM_WORLD); + serial.propagate(data.majoranas, data.param_inds, data.gen_coeffs, data.parameters, cap); + threaded.propagate(data.majoranas, data.param_inds, data.gen_coeffs, data.parameters, cap); + BOOST_TEST(bitwise_equal(BigAccess::picture_coeffs(serial), BigAccess::picture_coeffs(threaded))); + BOOST_TEST(serial.expectation_value(VecD{}) == threaded.expectation_value(VecD{})); + } + } + BOOST_TEST_CONTEXT("graph replay, schrodinger=" << schrodinger) { + auto serial = big_prototype(data, schrodinger, 1, MPI_COMM_WORLD); + auto threaded = big_prototype(data, schrodinger, 4, MPI_COMM_WORLD); + serial.build_graph(data.majoranas, data.param_inds, data.gen_coeffs); + threaded.build_graph(data.majoranas, data.param_inds, data.gen_coeffs); + BOOST_TEST_MESSAGE("graph-built operator terms on this rank: " << serial.mp_op().size()); + BOOST_TEST(bitwise_equal(serial.contract_partially(data.parameters, false), + threaded.contract_partially(data.parameters, false))); + BOOST_TEST(serial.expectation_value(data.parameters) == threaded.expectation_value(data.parameters)); + } + } +} diff --git a/cpp/tests/mpi_failure_driver.cpp b/cpp/tests/mpi_failure_driver.cpp index 326b00c3..aa869ce5 100644 --- a/cpp/tests/mpi_failure_driver.cpp +++ b/cpp/tests/mpi_failure_driver.cpp @@ -24,6 +24,8 @@ // worker-throw worksharing worker throws inside a retained-functional evaluation // before-exchange a build fails in the scan, before the first query exchange // active-ticket ticket|pending a failure while a replay Ticket / construction PendingAlltoallv is posted +// active-ticket cosine-worker a worker of the threaded cosine kernel throws while the replay Ticket is +// posted // insufficient-thread-level single|funneled host-initialized MPI below the required level // wrong-thread-entry serialized|multiple a guarded entry called from a non-initializing std::thread @@ -35,6 +37,7 @@ #include #include #include +#include #include #include #include @@ -46,6 +49,7 @@ #include "monoprop/Evolution.h" #include "monoprop/MPFunctions.h" #include "monoprop/MonomialPropagator.h" +#include "monoprop/detail/evolution/CosineRecompute.h" #include "monoprop/detail/mpi/MPICompat.h" #include "monoprop/detail/mpi/OperationFailure.h" #include "monoprop/detail/parallel/Workshare.h" @@ -153,6 +157,40 @@ auto active_ticket(int rank) -> void { evolve_step(coeffs, view, 0.3, 0, MPI_COMM_WORLD, scale); } +// Throws from one logical range of a real kernel, on whichever worker owns it. +struct ThrowingRangeObserver { + auto prepare(detail::KernelRange, size_t) const noexcept -> void {} + auto visit(detail::KernelRange, size_t range) const -> void { + if (range == 2) { + throw std::runtime_error(std::format("injected failure in cosine range {} on OpenMP worker {} while the " + "replay Ticket is in flight", + range, + omp_get_thread_num())); + } + } +}; + +auto active_ticket_cosine_worker(int rank) -> void { + auto sim = make_prototype(); + build(sim); + VecD coeffs = sim.mp_op().get_operator(); + const auto view = sim.graph().replay_view(); + // A stored mask of four kernel ranges, so the kernel really opens a team; the real layer is tiny. + CosMask mask; + for (size_t w = 0; w < 4 * detail::kCosMaskRangeBlocks; ++w) { + mask.blocks.emplace_back(w * 64, ~uint64_t{0}); + mask.total_count += 64; + } + auto scratch = std::make_shared(mask.blocks.size() * 64, 1.0); + const detail::LayerCosScale scale = [rank, mask, scratch](size_t, double *, double v) { + if (rank == 0) { + detail::scale_cos_mask(scratch->data(), mask, v, {.threads = 2}, ThrowingRangeObserver{}); + } + }; + say(rank, "posting the replay exchange"); + evolve_step(coeffs, view, 0.3, 0, MPI_COMM_WORLD, scale); +} + auto active_pending(int rank, int size) -> void { const mpi::SlotWindow window{.base = 0, .count = static_cast(size)}; mpi::WindowVec> send(window); @@ -263,6 +301,9 @@ auto main(int argc, char **argv) -> int { else if (scenario == "active-ticket" && option == "ticket") { active_ticket(rank); } + else if (scenario == "active-ticket" && option == "cosine-worker") { + active_ticket_cosine_worker(rank); + } else if (scenario == "active-ticket" && option == "pending") { active_pending(rank, size); } diff --git a/cpp/tests/openmp_kernel_tests.cpp b/cpp/tests/openmp_kernel_tests.cpp new file mode 100644 index 00000000..41793f14 --- /dev/null +++ b/cpp/tests/openmp_kernel_tests.cpp @@ -0,0 +1,536 @@ +// Copyright 2026 Algorithmiq +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +// The threaded cosine and fused-apply kernels: exact agreement with their one-thread execution, and +// kernel-specific worker participation. Participation is read from a test-only range observer that each +// kernel calls at the start of each of its own logical ranges (KernelTestSupport.h); a team created +// anywhere else does not count. Lazy-fold equivalence against the materialised-fold oracle lives in +// combined_recompute_equivalence.cpp; fused records from the real build path in fused_cos_sweep_tests.cpp. + +#include + +#include + +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include "KernelTestSupport.h" +#include "monoprop/detail/evolution/CosineRecompute.h" +#include "monoprop/detail/evolution/layer_build/FusedApply.h" +#include "monoprop/detail/parallel/Options.h" + +namespace { + +using namespace monoprop; +using kernel_test::RangeLog; +using kernel_test::RecordingObserver; +using monoprop::detail::KernelRange; +using monoprop::detail::parallel::Options; + +constexpr auto kBudgets = {1, 2, 3, 4}; + +// Distinct, non-degenerate values, so a missed, extra or doubled scaling shows up bitwise. +auto distinct_values(size_t n) -> VecD { + VecD v(n); + for (size_t i = 0; i < n; ++i) { + v[i] = 1.0 + static_cast(i) * 1e-3; + } + return v; +} + +auto bitwise_equal(const VecD &a, const VecD &b) -> bool { + return a.size() == b.size() && std::memcmp(a.data(), b.data(), a.size() * sizeof(double)) == 0; +} + +// Equal up to the rounding of one multiply-add whose terms are bounded by |original| + 1 (|cos|, |sin| and every +// snapshot value are at most one here): the difference a separately compiled fused multiply-add can make. +auto equal_up_to_contraction(const VecD &a, const VecD &b, const VecD &original) -> bool { + if (a.size() != b.size() || a.size() != original.size()) { + return false; + } + constexpr double eps = std::numeric_limits::epsilon(); + for (size_t i = 0; i < a.size(); ++i) { + if (std::abs(a[i] - b[i]) > 2 * eps * (std::abs(original[i]) + 1.0)) { + return false; + } + } + return true; +} + +// The plan's mask: 4097 words, alternating bits, and only bit zero in the final word. +auto plan_mask() -> CosMask { + CosMask mask; + for (size_t w = 0; w < 4097; ++w) { + const auto bits = w == 4096 ? uint64_t{1} : uint64_t{0x5555555555555555}; + mask.blocks.emplace_back(w * 64, bits); + mask.total_count += static_cast(std::popcount(bits)); + } + return mask; +} + +// Every row below `rows` selected, one block per word, the last word partial when rows % 64 != 0. +auto full_mask(size_t rows) -> CosMask { + CosMask mask; + for (size_t base = 0; base < rows; base += 64) { + const size_t n = std::min(64, rows - base); + const uint64_t bits = n == 64 ? ~uint64_t{0} : (uint64_t{1} << n) - 1; + mask.blocks.emplace_back(base, bits); + mask.total_count += n; + } + return mask; +} + +// Selected indices of a mask, independent of the kernel under test. +auto selected(const CosMask &mask, size_t n) -> std::vector { + std::vector sel(n, 0); + for (const auto &[base, bits] : mask.blocks) { + for (size_t t = 0; t < 64; ++t) { + if ((bits >> t) & 1U) { + sel[base + t] = 1; + } + } + } + return sel; +} + +// Every budget against the one-thread kernel, and the one-thread kernel against a hand oracle that +// multiplies exactly the selected coefficients and leaves the rest untouched. +auto check_mask_budgets(const CosMask &mask, size_t n, const char *label) -> void { + constexpr double cos_val = 0.625; + const VecD original = distinct_values(n); + VecD oracle = original; + const auto sel = selected(mask, n); + for (size_t i = 0; i < n; ++i) { + if (sel[i] != 0) { + oracle[i] = original[i] * cos_val; + } + } + VecD serial = original; + monoprop::detail::scale_cos_mask(serial.data(), mask, cos_val, Options{.threads = 1}); + BOOST_TEST_CONTEXT(label) { + BOOST_TEST(bitwise_equal(serial, oracle)); + for (const int threads : kBudgets) { + VecD threaded = original; + monoprop::detail::scale_cos_mask(threaded.data(), mask, cos_val, Options{.threads = threads}); + BOOST_TEST_INFO("threads " << threads); + BOOST_TEST(bitwise_equal(threaded, serial)); + } + // More workers than ranges. + VecD many = original; + monoprop::detail::scale_cos_mask(many.data(), mask, cos_val, Options{.threads = 64}); + BOOST_TEST(bitwise_equal(many, serial)); + } +} + +// --- hand-built fused records ------------------------------------------------------------------------------- + +// A contract whose destinations are a shuffled permutation of distinct slots: hits and inserts own two slots +// each, halves one. The two-pass cosine mask covers every destination plus a padding region no record +// touches (anticommuting terms whose rotation produced no record), so it spans several mask ranges; a few +// slots are in neither. +struct HandContract { + monoprop::detail::FusedContract fc; + CosMask cos; + size_t slots = 0; +}; + +auto hand_contract(size_t n_hits, size_t n_inserts, size_t n_halves, uint64_t seed) -> HandContract { + HandContract h; + const size_t record_slots = 2 * (n_hits + n_inserts) + n_halves + 97; + constexpr size_t pad_words = 2100; + h.slots = record_slots + pad_words * 64; + std::vector perm(record_slots); + for (size_t i = 0; i < record_slots; ++i) { + perm[i] = i; + } + kernel_test::SplitMix rng{seed}; + for (size_t i = record_slots - 1; i > 0; --i) { + std::swap(perm[i], perm[rng.below(i + 1)]); + } + size_t next = 0; + const auto value = [&rng] { return static_cast(rng.below(1U << 20)) / (1U << 19) - 1.0; }; + const auto phase = [&rng] { return rng.below(2) == 0 ? int32_t{1} : int32_t{-1}; }; + for (size_t k = 0; k < n_hits; ++k) { + h.fc.hits.push_back( + {.src = perm[next], .tgt = perm[next + 1], .v_src = value(), .v_tgt = value(), .phase = phase()}); + next += 2; + } + for (size_t k = 0; k < n_inserts; ++k) { + // v_tgt stays zero: Schrödinger gathers it in the apply, Heisenberg inserts are born at zero. + h.fc.inserts.push_back({.src = perm[next], .tgt = perm[next + 1], .v_src = value(), .phase = phase()}); + next += 2; + } + for (size_t k = 0; k < n_halves; ++k) { + h.fc.cross_half.push_back( + {.local_idx = perm[next], .v_partner = value(), .phase_signed = phase(), .is_insert = rng.below(3) == 0}); + ++next; + } + std::vector selected_slots(perm.begin(), perm.begin() + static_cast(next)); + for (size_t s = record_slots; s < h.slots; s += 2) { + selected_slots.push_back(s); + } + std::ranges::sort(selected_slots); + CosineWordBuilder builder; + for (const auto s : selected_slots) { + builder.push_index(s); + } + h.cos = builder.finish(); + return h; +} + +// Read at run time, so the oracle and the kernel both take std::cos/std::sin from the math library; a +// compile-time constant could be folded with a differently rounded result. +auto rotation_angle() -> double { + static volatile double angle = 0.37; + return angle; +} + +// Independent oracle: the pre-threading serial apply_fused_contract, verbatim in its phase order, record order +// and arithmetic, over a copy of the contract. +auto reference_apply(const HandContract &h, const VecD &coeffs, bool schrodinger, bool fused_scale) -> VecD { + auto fc = h.fc; + VecD out = coeffs; + const double param = rotation_angle(); + if (schrodinger) { + for (auto &r : fc.inserts) { + r.v_tgt = out[r.tgt]; + } + } + const double cos_val = std::cos(2 * param); + const double sin_val = std::sin(2 * param); + double *const c = out.data(); + if (!fused_scale) { + for (const auto &[base, bits] : h.cos.blocks) { + for (size_t t = 0; t < 64; ++t) { + if ((bits >> t) & 1U) { + c[base + t] *= cos_val; + } + } + } + } + const size_t n_hit = fc.hits.size(); + for (size_t k = 0; k < n_hit + fc.inserts.size(); ++k) { + const bool is_insert = k >= n_hit; + const auto &r = is_insert ? fc.inserts[k - n_hit] : fc.hits[k]; + c[r.src] += sin_val * static_cast(-r.phase) * r.v_tgt; + if (fused_scale && is_insert) { + c[r.tgt] = cos_val * c[r.tgt] + sin_val * static_cast(r.phase) * r.v_src; + } + else { + c[r.tgt] += sin_val * static_cast(r.phase) * r.v_src; + } + } + for (const auto &half : fc.cross_half) { + if (fused_scale && half.is_insert) { + c[half.local_idx] = + cos_val * c[half.local_idx] + sin_val * static_cast(half.phase_signed) * half.v_partner; + } + else { + c[half.local_idx] += sin_val * static_cast(half.phase_signed) * half.v_partner; + } + } + return out; +} + +auto apply_copy(const HandContract &h, + const VecD &coeffs, + bool schrodinger, + bool fused_scale, + int threads, + const RecordingObserver *observer = nullptr) -> VecD { + auto fc = h.fc; + VecD out = coeffs; + const double param = rotation_angle(); + if (observer != nullptr) { + monoprop::detail::apply_fused_contract(fc, out, h.cos, param, schrodinger, fused_scale, {threads}, *observer); + } + else { + monoprop::detail::apply_fused_contract(fc, out, h.cos, param, schrodinger, fused_scale, {threads}); + } + return out; +} + +} // namespace + +// --- stored-mask cosine ------------------------------------------------------------------------------------ + +BOOST_AUTO_TEST_CASE(openmp_cos_mask_exact) { + const CosMask mask = plan_mask(); + VecD s(4097 * 64, 1.25), p = s; + monoprop::detail::scale_cos_mask(s.data(), mask, 0.625, {.threads = 1}); + monoprop::detail::scale_cos_mask(p.data(), mask, 0.625, {.threads = 3}); + BOOST_CHECK_EQUAL_COLLECTIONS(s.begin(), s.end(), p.begin(), p.end()); + check_mask_budgets(mask, 4097 * 64, "plan mask, 4097 words"); +} + +BOOST_AUTO_TEST_CASE(openmp_cos_mask_edge_cases) { + check_mask_budgets(CosMask{}, 128, "empty mask"); + for (const size_t tail : {63, 64, 65}) { + const size_t rows = 2048 * 64 + tail; + BOOST_TEST_CONTEXT("tail " << tail) { + check_mask_budgets(full_mask(rows), rows, "full mask with a partial tail"); + } + } + // Exactly one range (serial whatever the budget), and one block over it (two ranges). + check_mask_budgets(full_mask(1024 * 64), 1024 * 64, "1024 blocks"); + check_mask_budgets(full_mask(1024 * 64 + 1), 1024 * 64 + 1, "1025 blocks"); + // Sparse blocks: bases need not be consecutive, only ascending and distinct. + CosMask sparse; + for (size_t w = 0; w < 5000; w += 3) { + sparse.blocks.emplace_back(w * 64, uint64_t{0x8000000000000001}); + sparse.total_count += 2; + } + check_mask_budgets(sparse, 5000 * 64, "sparse blocks"); +} + +BOOST_AUTO_TEST_CASE(openmp_cos_mask_workers_participate, + *boost::unit_test::precondition(kernel_test::runtime_offers_two_workers)) { + const CosMask mask = plan_mask(); + VecD c(4097 * 64, 1.0); + RangeLog log; + monoprop::detail::scale_cos_mask(c.data(), mask, 0.5, Options{.threads = 4}, RecordingObserver{&log}); + kernel_test::check_participation(log[KernelRange::cos_mask], 5, "scale_cos_mask budget 4"); +} + +BOOST_AUTO_TEST_CASE(openmp_cos_mask_small_or_serial_work_stays_on_the_caller) { + VecD c(4097 * 64, 1.0); + RangeLog one_thread; + monoprop::detail::scale_cos_mask(c.data(), plan_mask(), 0.5, Options{.threads = 1}, RecordingObserver{&one_thread}); + kernel_test::check_serial(one_thread[KernelRange::cos_mask], 5, "budget 1"); + RangeLog tiny; + monoprop::detail::scale_cos_mask(c.data(), + full_mask(1024 * 64), + 0.5, + Options{.threads = 4}, + RecordingObserver{&tiny}); + kernel_test::check_serial(tiny[KernelRange::cos_mask], 1, "one range, budget 4"); + RangeLog empty; + monoprop::detail::scale_cos_mask(c.data(), CosMask{}, 0.5, Options{.threads = 4}, RecordingObserver{&empty}); + kernel_test::check_serial(empty[KernelRange::cos_mask], 0, "empty mask, budget 4"); +} + +// --- lazy cosine --------------------------------------------------------------------------------------------- + +BOOST_AUTO_TEST_CASE(openmp_cos_lazy_workers_participate, + *boost::unit_test::precondition(kernel_test::runtime_offers_two_workers)) { + // Three full fold blocks plus a partial fourth. + const size_t rows = 3 * monoprop::detail::kColumnBlockWords * 64 + 1000; + const auto index = kernel_test::make_synthetic_index(rows, 11); + const auto gen = kernel_test::generator({0, 3, 7}); // odd: exercises the caller-prepared row parity + const auto recipe = monoprop::detail::make_lazy_fold(index, gen, rows, Basis::Majorana); + VecD c = distinct_values(rows); + RangeLog log; + monoprop::detail::scale_cos_lazy(index, + recipe, + c.data(), + 0.5, + Options{.threads = 4}, + RecordingObserver{&log}); + kernel_test::check_participation(log[KernelRange::cos_lazy], 4, "scale_cos_lazy budget 4"); +} + +BOOST_AUTO_TEST_CASE(openmp_cos_lazy_small_or_serial_work_stays_on_the_caller) { + const size_t rows = 3 * monoprop::detail::kColumnBlockWords * 64 + 1000; + const auto index = kernel_test::make_synthetic_index(rows, 12); + const auto gen = kernel_test::generator({1, 4}); + const auto recipe = monoprop::detail::make_lazy_fold(index, gen, rows, Basis::Majorana); + VecD c = distinct_values(rows); + RangeLog one_thread; + monoprop::detail::scale_cos_lazy(index, + recipe, + c.data(), + 0.5, + Options{.threads = 1}, + RecordingObserver{&one_thread}); + kernel_test::check_serial(one_thread[KernelRange::cos_lazy], 4, "budget 1"); + // A partial scaled_count inside the first block: one range, serial at any budget. + const auto short_recipe = + monoprop::detail::make_lazy_fold(index, gen, 1000, Basis::Majorana); + RangeLog tiny; + monoprop::detail::scale_cos_lazy(index, + short_recipe, + c.data(), + 0.5, + Options{.threads = 4}, + RecordingObserver{&tiny}); + kernel_test::check_serial(tiny[KernelRange::cos_lazy], 1, "one range, budget 4"); +} + +// --- fused apply ------------------------------------------------------------------------------------------ + +BOOST_AUTO_TEST_CASE(openmp_fused_apply_exact_on_disjoint_records) { + // Several record ranges in every stream, with partial final ranges. + const auto h = hand_contract(3 * 1024 + 17, 2 * 1024 + 5, 1500, 7); + const auto owners = kernel_test::add_owner_counts(h.fc); + BOOST_TEST(std::ranges::all_of(owners, [](const auto &kv) { return kv.second == 1; })); + const VecD coeffs = distinct_values(h.slots); + for (const bool schrodinger : {false, true}) { + for (const bool fused_scale : {false, true}) { + BOOST_TEST_CONTEXT("schrodinger=" << schrodinger << " fused_scale=" << fused_scale) { + const VecD serial = apply_copy(h, coeffs, schrodinger, fused_scale, 1); + BOOST_TEST(!bitwise_equal(serial, coeffs)); + // The oracle is compiled separately, so where a slot is `cos·c + sin·φ·v` (fused-scale insert + // arms) the compiler may contract a different product into a fused multiply-add. Every other + // arm is a single multiply-add and must match bitwise. + const VecD reference = reference_apply(h, coeffs, schrodinger, fused_scale); + if (fused_scale) { + BOOST_TEST(equal_up_to_contraction(serial, reference, coeffs)); + } + else { + BOOST_TEST(bitwise_equal(serial, reference)); + } + for (const int threads : {2, 3, 4, 64}) { + BOOST_TEST_INFO("threads " << threads); + BOOST_TEST(bitwise_equal(apply_copy(h, coeffs, schrodinger, fused_scale, threads), serial)); + } + // Slots no record or mask touches keep their values. + const auto sel = selected(h.cos, h.slots); + for (size_t i = 0; i < h.slots; ++i) { + if (sel[i] == 0 && !owners.contains(i)) { + BOOST_TEST(serial[i] == coeffs[i]); + } + } + } + } + } +} + +BOOST_AUTO_TEST_CASE(openmp_fused_apply_workers_participate, + *boost::unit_test::precondition(kernel_test::runtime_offers_two_workers)) { + const auto h = hand_contract(3 * 1024 + 17, 2 * 1024 + 5, 1500, 8); + const VecD coeffs = distinct_values(h.slots); + RangeLog log; + const RecordingObserver observer{&log}; + (void)apply_copy(h, coeffs, /*schrodinger=*/true, /*fused_scale=*/false, 4, &observer); + const size_t records = h.fc.hits.size() + h.fc.inserts.size() + h.fc.cross_half.size(); + kernel_test::check_participation(log[KernelRange::fused_gather], 3, "insert gather budget 4"); + kernel_test::check_participation(log[KernelRange::cos_mask], + monoprop::detail::logical_ranges(h.cos.blocks.size(), 1024), + "two-pass cosine phase budget 4"); + kernel_test::check_participation(log[KernelRange::fused_apply], + monoprop::detail::logical_ranges(records, 1024), + "rotation records budget 4"); +} + +BOOST_AUTO_TEST_CASE(openmp_fused_apply_small_or_serial_work_stays_on_the_caller) { + const auto h = hand_contract(3 * 1024 + 17, 2 * 1024 + 5, 1500, 9); + const VecD coeffs = distinct_values(h.slots); + RangeLog one_thread; + const RecordingObserver serial_observer{&one_thread}; + (void)apply_copy(h, coeffs, true, false, 1, &serial_observer); + const size_t records = h.fc.hits.size() + h.fc.inserts.size() + h.fc.cross_half.size(); + kernel_test::check_serial(one_thread[KernelRange::fused_gather], 3, "gather budget 1"); + kernel_test::check_serial(one_thread[KernelRange::fused_apply], + monoprop::detail::logical_ranges(records, 1024), + "records budget 1"); + + // Fused-scale Heisenberg: no gather and no cosine phase at all. + RangeLog heisenberg; + const RecordingObserver heisenberg_observer{&heisenberg}; + (void)apply_copy(h, coeffs, false, true, 4, &heisenberg_observer); + BOOST_TEST(heisenberg[KernelRange::fused_gather].prepares == 0u); + BOOST_TEST(heisenberg[KernelRange::cos_mask].prepares == 0u); + + const auto tiny = hand_contract(300, 200, 100, 10); + const VecD tiny_coeffs = distinct_values(tiny.slots); + RangeLog small; + const RecordingObserver small_observer{&small}; + (void)apply_copy(tiny, tiny_coeffs, true, false, 4, &small_observer); + kernel_test::check_serial(small[KernelRange::fused_gather], 1, "tiny gather budget 4"); + kernel_test::check_serial(small[KernelRange::fused_apply], 1, "tiny records budget 4"); + + HandContract empty; + empty.slots = 16; + RangeLog none; + const RecordingObserver none_observer{&none}; + const VecD untouched = apply_copy(empty, distinct_values(16), true, false, 4, &none_observer); + BOOST_TEST(bitwise_equal(untouched, distinct_values(16))); + kernel_test::check_serial(none[KernelRange::fused_apply], 0, "empty contract budget 4"); +} + +// --- worker failures inside the kernels' own ranges -------------------------------------------------------- + +// A worker failure inside a kernel range is captured, every worker joins, and the original exception reaches +// the caller; no range runs twice. Which other ranges ran is unspecified. +BOOST_AUTO_TEST_CASE(openmp_kernel_worker_failures_join_before_rethrow) { + const auto check = [](const RangeLog &log, KernelRange kind, const char *label) { + const auto &s = log.slots[static_cast(kind)]; + BOOST_TEST_CONTEXT(label) { + BOOST_TEST(s.visits.at(2) == 1); + BOOST_TEST(std::ranges::all_of(s.visits, [](int v) { return v <= 1; })); + } + }; + for (const int threads : {1, 4}) { + BOOST_TEST_CONTEXT("threads " << threads) { + VecD c(4097 * 64, 1.0); + RangeLog mask_log; + BOOST_CHECK_THROW(monoprop::detail::scale_cos_mask(c.data(), + plan_mask(), + 0.5, + Options{.threads = threads}, + RecordingObserver{&mask_log, KernelRange::cos_mask, 2}), + std::runtime_error); + check(mask_log, KernelRange::cos_mask, "scale_cos_mask"); + + const size_t rows = 3 * monoprop::detail::kColumnBlockWords * 64 + 1000; + const auto index = kernel_test::make_synthetic_index(rows, 13); + const auto recipe = + monoprop::detail::make_lazy_fold(index, + kernel_test::generator({0, 3, 7}), + rows, + Basis::Majorana); + VecD lazy = distinct_values(rows); + RangeLog lazy_log; + BOOST_CHECK_THROW(monoprop::detail::scale_cos_lazy( + index, + recipe, + lazy.data(), + 0.5, + Options{.threads = threads}, + RecordingObserver{&lazy_log, KernelRange::cos_lazy, 2}), + std::runtime_error); + check(lazy_log, KernelRange::cos_lazy, "scale_cos_lazy"); + + const auto h = hand_contract(3 * 1024 + 17, 2 * 1024 + 5, 1500, 14); + for (const auto kind : {KernelRange::fused_gather, KernelRange::cos_mask, KernelRange::fused_apply}) { + RangeLog fused_log; + const RecordingObserver observer{&fused_log, kind, 2}; + BOOST_CHECK_THROW((void)apply_copy(h, distinct_values(h.slots), true, false, threads, &observer), + std::runtime_error); + check(fused_log, kind, "apply_fused_contract"); + } + } + } +} + +// The debug-build owner check agrees with the independent count, and rejects a slot written twice: a hit's +// target reused as a half's local slot, and a source shared by two inserts. +BOOST_AUTO_TEST_CASE(openmp_fused_add_owner_check_detects_a_shared_slot) { + auto h = hand_contract(3 * 1024 + 17, 2 * 1024 + 5, 1500, 15); + BOOST_TEST(monoprop::detail::fused_add_owners_unique(h.fc)); + BOOST_TEST(monoprop::detail::fused_add_owners_unique(monoprop::detail::FusedContract{})); + auto shared_half = h.fc; + shared_half.cross_half.back().local_idx = shared_half.hits.front().tgt; + BOOST_TEST(!monoprop::detail::fused_add_owners_unique(shared_half)); + auto shared_insert = h.fc; + shared_insert.inserts.back().src = shared_insert.inserts.front().src; + BOOST_TEST(!monoprop::detail::fused_add_owners_unique(shared_insert)); +} diff --git a/cpp/tests/openmp_runtime_tests.cpp b/cpp/tests/openmp_runtime_tests.cpp index 2d569529..7dafdc35 100644 --- a/cpp/tests/openmp_runtime_tests.cpp +++ b/cpp/tests/openmp_runtime_tests.cpp @@ -16,8 +16,8 @@ // // Budgets are captured from monoprop_NUM_THREADS when an object is constructed. Cases that need a specific // launch environment are also registered as separate fresh-process CTest entries (see CMakeLists.txt); -// the rest set the variable around one construction with ScopedEnv and restore it afterwards. Kernels are -// still serial here, so these tests check what reaches each object, never parallel execution. +// the rest set the variable around one construction with ScopedEnv and restore it afterwards. These tests +// check what reaches each object; kernel-level parallel execution is covered by openmp_kernel_tests.cpp. #include diff --git a/docs/content/docs/building.mdx b/docs/content/docs/building.mdx index 6cb5b70a..a28ed793 100644 --- a/docs/content/docs/building.mdx +++ b/docs/content/docs/building.mdx @@ -339,6 +339,26 @@ just test-cpp-tsan TSan is scoped to the concurrency surface — the partition and `ShmComm` tests — which is where its reports are worth the runtime cost. +The OpenMP kernels (`openmp_*` tests) need an OpenMP runtime that TSan understands. GCC's libgomp is +not instrumented, so TSan reports false races even on correct parallel loops; do not read its +output as evidence either way. Clang with LLVM's libomp and its Archer tool works, with the runtime's +own internal locking suppressed: + +```bash +CXX=clang++ CC=clang SKBUILD_CMAKE_BUILD_TYPE=Tsan \ +SKBUILD_CMAKE_DEFINE="monoprop_SANITIZER=tsan;monoprop_ENABLE_MPI=OFF" \ +uv sync --group workspace-test --all-extras --reinstall-package monoprop --no-cache -v + +echo 'called_from_lib:libomp.so.5' > libomp.supp +OMP_TOOL_LIBRARIES="$(clang++ -print-resource-dir)/../../libarcher.so" \ +TSAN_OPTIONS="halt_on_error=1:history_size=4:suppressions=$PWD/libomp.supp" \ + ctest --test-dir build/editable/Tsan --output-on-failure --no-tests=error -R 'openmp_' +``` + +Check a new toolchain first with a small known-racy and known-safe OpenMP program: the safe one must +stay silent and the racy one must be reported in your own frames. The Archer path varies by +distribution (on Debian and Ubuntu, `/usr/lib/llvm-/lib/libarcher.so`). + There is no ThreadSanitizer equivalent of the ASan Python leg. TSan does not support `dlopen` of instrumented libraries, so loading an instrumented `_core` into a stock interpreter crashes during TSan initialisation; exercising the diff --git a/docs/content/docs/features/parallelism.mdx b/docs/content/docs/features/parallelism.mdx index 92cf5e81..489a2c99 100644 --- a/docs/content/docs/features/parallelism.mdx +++ b/docs/content/docs/features/parallelism.mdx @@ -61,7 +61,12 @@ multi-partition runs are unchanged. (`omp_get_max_threads()`, normally `OMP_NUM_THREADS`) applies. The budget is not clamped to the cores you have; placement belongs to `OMP_PLACES`, `OMP_PROC_BIND` and your launcher. Copies and retained callables keep the budget, and a later change to the environment only affects new objects. -- **Serial kernels, for now.** The budget reaches every kernel, but no kernel runs in parallel yet. +- **Threaded kernels so far.** Two kernels use the budget: the per-gate cosine scaling (graph + replay during evaluation and contraction, and the two-pass path of in-place propagation) and the + rotation apply of in-place `propagate`. Each splits its work into fixed blocks of 1024 bitmap words + or 1024 rotation records and stays serial below two blocks, so small operators see no threads. + Results are bitwise identical at every budget. Graph construction (scan, lookup, insertion), + reductions and the gradient's endpoint work are still serial. - **Calling thread.** Every call that can use MPI (construction, operations, retained callables) must come from the thread that initialized MPI, whatever thread support MPI provides. A call from another thread prints a diagnostic and aborts that process without communicating; the launcher diff --git a/tests/test_openmp_config.py b/tests/test_openmp_config.py index 9df94aa4..6214e447 100644 --- a/tests/test_openmp_config.py +++ b/tests/test_openmp_config.py @@ -101,7 +101,8 @@ def test_prototype_accepts_valid_budgets_without_changing_results() -> None: outcome = _probe(budget, {"OMP_NUM_THREADS": "2"}, partitions=1) assert outcome["ok"], (budget, outcome) energies.append(outcome["energy"]) - # Kernels are still serial, so every budget gives the same numbers. + # Threaded kernels give bitwise-identical results at every budget (and this problem is too small + # to leave their serial paths anyway). assert all(e == energies[0] for e in energies) From a3631cb0e2c4cd6b5d944ff042e122b5be82eeed Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Roberto=20Di=20Remigio=20Eik=C3=A5s?= Date: Mon, 28 Sep 2026 18:53:03 +0000 Subject: [PATCH 17/64] =?UTF-8?q?docs(parallelism):=20=F0=9F=93=9D=20recor?= =?UTF-8?q?d=20the=20Task=204=20outcome=20and=20plan=20a=20std::fma=20eval?= =?UTF-8?q?uation?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Record what Task 4 delivered and what later tasks inherit: - the kernel range, scratch, phase and FMA-determinism contracts, and the test-only observer; - RED/GREEN, mutation, stress and sanitizer evidence, with the qualified TSan runtime; - the owner-approved replay diagnostics: one-store speedups and losses, kernel-only scaling, and the comparison with the baseline partition runtime; - where the uncommitted Task 4 artifacts live. Task 9 gains a checklist item to evaluate explicit std::fma against the current contraction-dependent arithmetic, before any numerical-policy change. Assisted-by: Pi:claude-opus-5-5 --- .../plans/2026-09-18-rank-local-openmp.md | 85 ++++++++++++++++++- 1 file changed, 84 insertions(+), 1 deletion(-) diff --git a/docs/superpowers/plans/2026-09-18-rank-local-openmp.md b/docs/superpowers/plans/2026-09-18-rank-local-openmp.md index 2ced6812..e9ad3cd8 100644 --- a/docs/superpowers/plans/2026-09-18-rank-local-openmp.md +++ b/docs/superpowers/plans/2026-09-18-rank-local-openmp.md @@ -1586,6 +1586,75 @@ BOOST_AUTO_TEST_CASE(openmp_cos_mask_exact) { persistent buffer. If replay is already bandwidth-bound with regressions, stop/profile rather than proceed on assumed speedup. +**Task 4 outcome (2026-09-28):** the owner accepted the gate, with the high-budget losses and the lack of a +`propagate` gain on record. Correctness gates pass on GCC 15.2 (R, M) and Clang 18.1.8/libomp (R) Linux. Other +platforms are pending. +- Contracts as implemented, which later tasks rely on: + - `scale_cos_lazy` runs one logical range per `kColumnBlockWords` (1024) fold words. + - Row parity is fetched on the caller for each call. + - Scratch is a `std::array` on the worker's stack, never the caller's `column_block_scratch()`. + - Tail masking stays global through `apply_fold_mask`. + - `scale_cos_mask` runs one range per 1024 stored mask blocks. It relies on ascending, distinct, word-aligned + bases. + - `apply_fused_contract` keeps its phases, each joined before the next: + 1. The insert snapshots are gathered into the existing `RotationRec::v_tgt`. + 2. The optional cosine phase runs. + 3. The records are applied in ranges of 1024. + - Every record range calls one out-of-line `[[gnu::noinline]] apply_fused_record_range`, so the compiler's + FMA contraction of `cos·c + sin·φ·v` is identical at every budget. Without it, fused-scale `propagate` differed + from Task 3 by 1 ulp. Task 9 evaluates explicit `std::fma`. + - A kernel stays serial below two ranges; the thresholds are fixed until Task 11. + - Debug builds assert `fused_add_owners_unique(fc)` on every apply. + - The kernels take a defaulted, test-only range observer (`NoRangeObserver` in production); there is no + production callback. Accumulators, index materialization and every other routine stay serial. +- Tests: + - `openmp_kernel_tests.cpp` has 11 cases, with `KernelTestSupport.h`. + - `combined_recompute_equivalence.cpp` adds two synthetic multi-block lazy-fold cases: tails, cold parity, + Majorana, Pauli, empty columns and index growth. + - `fused_cos_sweep_tests.cpp` adds four cases: real-path records with one add-owner per slot in both pictures + (cross-rank halves under MPI), exact serial/threaded prototypes, and fused and two-pass agreement with replay at + the existing tolerances. + - Two `openmp_env_kernels_*` reruns cover `OMP_THREAD_LIMIT=2` and `OMP_DYNAMIC=TRUE`. + - The failure scenario `active-ticket cosine-worker` throws from worker 1 while a Ticket is posted. + - RED: participation failed with the kernels serial, while the exactness cases already passed as preservation. + GREEN: 3–4 distinct workers observed in every kernel's own ranges, on libgomp and libomp. + - Six mutants were all killed, three of them by TSan. Stress: 800 fresh processes, no failures. + - Coefficients are bitwise equal across budgets. Six library paths are bitwise equal to Task 3. +- Final runs on fresh rebuilds: + - R: ctest 352/352; pytest 840 passed, 13 skipped. + - Clang: 352/352; 840 passed. + - ASan/UBSan: 352/352; 819 passed. + - M: ctest serial 382/382; linear R=1,2,4 and splitmix R=1-4 MPI ctest and `--with-mpi` pytest pass. + - `just test-find-package` passes on R and M. +- TSan qualifies only with Clang + libomp + Archer and one `called_from_lib:libomp.so.5` suppression. GCC's libgomp + reports false races on known-safe code, so it is unsupported (`building.mdx`). +- Replay diagnostics (owner-approved budget; 102 of 180 min used; diagnostic, not acceptance evidence): + - Setup: Task 4 R build, prototype selected, allocation read before import, one worker per core. + - One-store speedup at the best budget: + + | Case | Speedup | Budget | + | --- | --- | --- | + | random energy, Heisenberg | 5.3× | T=48 | + | random energy, Schrödinger | 24× | T=96 | + | Pauli-127 energy | 2.3× | T=16 | + | gradient | 1.44× | — | + | pared energy | 1.2× | — | + | `propagate` | none | — | + + - Losses: 7% (E1, T=96 against T=48), 9% (E4, T≥48 against T=16), 0.8% (`propagate`, T≥48 against T=1). + `OMP_WAIT_POLICY=PASSIVE` made them worse, so worker spinning is not the cause. + - Synthetic kernels scale 20–54× at T=48, so replay is limited by serial work (Tasks 5, 8, 9) and small layers, + not by the kernels. Peak memory is flat across budgets. + - Against the baseline partition runtime (90d5717) at the same T: + - One thread: within 0.95–1.07. + - T≥16: the partitions are 1.2–35× faster, the largest gaps in gradient and `propagate`. + - The baseline's peak grows with T by up to 0.38 GiB. +- Not run: `just build-docs`, and the Nix, macOS/Homebrew, devcontainer and wheel-repair routes. + - Wheel bundling of the now-used OpenMP runtime is still to be checked (Task 13). +- Artifacts: `/home/ubuntu/task4-artifacts/` on the c8a.metal-24xl host (`report.md`, `ledger.md`, + `replay-results.md`, `diag/`). They are intentionally uncommitted. The frozen campaign and workload files, the + Task 1 artifacts, and the baseline binaries and worktrees were verified unchanged. + ### Task 5: Deterministic parallel bitmap traversal and query generation **Files:** @@ -1967,9 +2036,23 @@ through `EvalRequest.parallel` under the approved numerical policy; implementati post-call-seed tests from `tests/test_circuit.py`: gradients must use the equivalent composed circuit's parameter order, including multi-monomial gates sharing one index. Cross-rank-count sums need not be bitwise equal; don't widen tolerances to pass them. +- [ ] Evaluate explicit `std::fma` for multiply-add updates whose result currently depends on compiler contraction. + Task 4 found that the fused-scale insert arms of `apply_fused_contract` (`cos·c + sin·φ·v`, two products) round + differently depending on where the compiler fuses: same source, 1-ulp differences between inlining contexts. Task + 4 pins them with one out-of-line `apply_fused_record_range`. That makes serial and threaded runs bitwise equal within + one binary, not across binaries or compilers. + - Inventory the affected expressions: fused apply, the scan's fused cosine sweep, replay endpoint and derivative + updates, and the new block partials. Single-product `c += a·b` updates already have only one possible fusion. + - Compare three options on the same inputs: the current contraction-dependent code, explicit `std::fma` with one + documented association, and contraction disabled for the affected translation units. + - Measure each option's retained-key and coefficient differences against the frozen baseline, with existing + tolerances. Also measure runtime on `-march=native` builds and on builds without FMA hardware (sanitizer, Debug, + and the wheel flags actually used), where `std::fma` becomes a software call. + - Adopt `std::fma` only with an owner-approved numerical-policy decision covering those differences and costs. Never + guard it by `__FMA__` silently, and never use it to hide a tolerance failure. Record the decision either way. **Gate:** Existing serial API guarantees remain intact; new numerical association is documented and deterministic by -team size. Global retained term keys must agree, including near cutoffs and stored zeros; matching energies/gradients +team size. The `std::fma` evaluation is recorded with its measurements and decision. Global retained term keys must agree, including near cutoffs and stored zeros; matching energies/gradients alone is insufficient. A key-set change blocks the gate. Stop and report, never widen tolerances or change truncation. ### Task 10: Cut over configuration and API semantics to one store From 29acf78a11ef9c2718bae31dab6a3343218d67a7 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Roberto=20Di=20Remigio=20Eik=C3=A5s?= Date: Mon, 28 Sep 2026 21:27:07 +0000 Subject: [PATCH 18/64] =?UTF-8?q?feat(parallel):=20=E2=9C=A8=20thread=20th?= =?UTF-8?q?e=20bitmap=20scan=20with=20a=20source-ordered=20merge?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Task 5 of the rank-local OpenMP plan. The per-gate bitmap scan that finds anticommuting terms and emits their partner queries now uses the object's thread budget. Lookup, insertion, publication and graph packing stay serial. - fused_find_and_collect prepares the inverted index and row parity on the caller, then splits the words into min(threads, ceil(words/1024)) contiguous runs of whole fold blocks (scan_ranges, scan_range_words). Range IDs depend on the word count and budget only, never on the actual team. - Each range runs fused_find_and_collect_range into its own FusedScanResult. It never calls a lazy accessor, keeps its scratch private or thread_local on the executing thread, masks only the global tail word and, in the fused sweep, scales only its own sources, once, after capturing them. - merge_scan_ranges joins the pieces in range order, stream by stream: checked totals, one reservation, the first buffer moved where it can hold the stream, drained buffers released. Self stages merge through SelfQueryStage::push. Output is identical to the serial scan at every budget. - CutoffEvaluator::parallel_safe() admits only the typed length and support cutoffs. Opaque predicates, basis-change closures and scans below two ranges keep the serial scan on the caller. - The test-only range observer moves to layer_build/Common.h, gains KernelRange::scan, and build_layer forwards a defaulted one; production passes none. Tests: - ScanTestSupport.h compares threaded scans with budget one byte for byte across windows (one rank, linear zero/non-zero shifts with non-zero bases, dense splitmix at three and four ranks), pivots, Majorana and Pauli J(G), caps, cutoffs, atol boundaries, capture, the fused sweep and cold caches. - The merge is tested on hand-built pieces: variable-length wire records, both self stages, an empty piece, buffer moves and released capacity. - Full construction at budgets 1-4 gives identical rows, row IDs, graph layers and coefficients. - Opaque and basis-change cutoffs stay serial with identical results. - A scan worker failure joins before rethrow and invalidates the propagator; the failure driver gains before-exchange scan-worker, which must abort both ranks rather than hang. - openmp_env_kernels_* rerun the scan cases under OMP_THREAD_LIMIT=2 and OMP_DYNAMIC=TRUE. Docs: the one-store prototype section lists the threaded scan and its serial cutoff fallback. Assisted-by: Pi:claude-opus-5-5 --- AGENTS.md | 7 +- cpp/monoprop/algebra/AlgebraCommon.h | 10 + .../detail/evolution/CosineRecompute.h | 29 +- .../detail/evolution/layer_build/Common.h | 29 + .../detail/evolution/layer_build/Engine.h | 47 +- .../detail/evolution/layer_build/Scan.h | 702 +++++++++++++----- cpp/tests/CMakeLists.txt | 6 +- cpp/tests/KernelTestSupport.h | 2 +- cpp/tests/PropagatorTestAccess.h | 32 + cpp/tests/README.md | 14 +- cpp/tests/ScanTestSupport.h | 517 +++++++++++++ cpp/tests/evolution_detail_tests.cpp | 172 +++++ cpp/tests/exact_upper_atol_rescue.cpp | 35 + cpp/tests/fused_cos_sweep_tests.cpp | 235 ++++++ cpp/tests/majorana_cutoff_tests.cpp | 45 ++ cpp/tests/mpi_failure_driver.cpp | 61 ++ cpp/tests/pauli_build_layer_tests.cpp | 54 ++ cpp/tests/sparse_query_tests.cpp | 193 +++++ docs/content/docs/features/parallelism.mdx | 16 +- 19 files changed, 1943 insertions(+), 263 deletions(-) create mode 100644 cpp/tests/ScanTestSupport.h diff --git a/AGENTS.md b/AGENTS.md index f123d01b..56af1576 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -72,9 +72,10 @@ just build-docs separates validation from mutation (`run_operation_`): a failure after mutation starts invalidates the object and aborts a multi-rank ordinary communicator (`mpi::operation_failed`). Catch failures inside posted `Ticket`/`PendingAlltoallv` lifetimes, never only at the outermost level. -- Threaded kernels (`scale_cos_*`, `apply_fused_contract`) take a defaulted test-only range observer; - participation tests read it (`cpp/tests/KernelTestSupport.h`). Serial and threaded runs must agree - bitwise. TSan evidence needs Clang + libomp + Archer; GCC's libgomp gives false reports (`building.mdx`). +- Threaded kernels (`scale_cos_*`, `apply_fused_contract`, the `fused_find_and_collect` scan) take a + defaulted test-only range observer; participation tests read it (`cpp/tests/KernelTestSupport.h`, + `ScanTestSupport.h`). Serial and threaded runs must agree bitwise. Scan workers only traverse frozen + storage and merge in range order; only typed cutoffs (`CutoffEvaluator::parallel_safe`) run on them. TSan evidence needs Clang + libomp + Archer; GCC's libgomp gives false reports (`building.mdx`). - `cpp/tests/mpi_failure_driver.cpp` has its own `main()`, excluded from the unit-runner glob; `ctest -L mpi_failure` runs its two-rank abort scenarios (30 s timeout; a timeout is a hang). - Slow CTest startup in MPI builds is `MPI_Init` fabric probing; see diff --git a/cpp/monoprop/algebra/AlgebraCommon.h b/cpp/monoprop/algebra/AlgebraCommon.h index 04eeadcc..7110df73 100644 --- a/cpp/monoprop/algebra/AlgebraCommon.h +++ b/cpp/monoprop/algebra/AlgebraCommon.h @@ -223,6 +223,16 @@ class CutoffEvaluator { auto support_cutoff() const -> const SupportCutoff * { return support_cutoff_; } + /*! + * \brief Whether scan workers may evaluate this cutoff concurrently. + * + * True only when the predicate is one of the typed built-in cutoffs (LengthCutoff, SupportCutoff), + * whose evaluation reads only their own immutable fields. Any other predicate is opaque, even a + * stateless lambda: the evaluator cannot inspect it, and a const std::function says nothing about what + * its target does. Opaque predicates, including the basis-change closures, keep the serial scan. + */ + auto parallel_safe() const noexcept -> bool { return length_cutoff_ != nullptr || support_cutoff_ != nullptr; } + auto operator()(const Monomial &mono) const -> bool { if (length_cutoff_ != nullptr) { return (*length_cutoff_)(mono); diff --git a/cpp/monoprop/detail/evolution/CosineRecompute.h b/cpp/monoprop/detail/evolution/CosineRecompute.h index a41b1480..a448b8be 100644 --- a/cpp/monoprop/detail/evolution/CosineRecompute.h +++ b/cpp/monoprop/detail/evolution/CosineRecompute.h @@ -35,6 +35,7 @@ #include "monoprop/algebra/Algebra.h" #include "monoprop/detail/evolution/CosineRecomputeCallbacks.h" +#include "monoprop/detail/evolution/layer_build/Common.h" #include "monoprop/detail/evolution/layer_build/Scan.h" #include "monoprop/detail/operator/InvertedIndex.h" #include "monoprop/detail/parallel/Options.h" @@ -42,37 +43,9 @@ namespace monoprop::detail { -/*! - * \brief The kinds of logical work range a threaded kernel reports to its range observer (see NoRangeObserver). - */ -enum class KernelRange { - cos_lazy, //!< scale_cos_lazy: one range per kColumnBlockWords fold words. - cos_mask, //!< scale_cos_mask: one range per kCosMaskRangeBlocks stored mask blocks. - fused_gather, //!< apply_fused_contract insert snapshots: one range per kFusedRecordRange records. - fused_apply, //!< apply_fused_contract rotations: one range per kFusedRecordRange records. -}; - -/*! - * \brief The observer production kernels use: it does nothing and compiles away. - * - * A kernel calls `prepare(kind, ranges)` once on the calling thread before any range runs, then - * `visit(kind, range)` at the start of each logical range, from whichever thread owns that range. Tests - * substitute an observer that records the worker of each range; that seam is the only reason the kernels - * take an observer, and no production caller passes one. - */ -struct NoRangeObserver { - auto prepare(KernelRange /*kind*/, size_t /*ranges*/) const noexcept -> void {} - auto visit(KernelRange /*kind*/, size_t /*range*/) const noexcept -> void {} -}; - //! Stored mask blocks (one 64-bit word each) per scale_cos_mask range: as many words as a fold block. inline constexpr size_t kCosMaskRangeBlocks = kColumnBlockWords; -//! Number of `per_range`-sized logical ranges covering `items`. -inline constexpr auto logical_ranges(size_t items, size_t per_range) -> size_t { - return (items + per_range - 1) / per_range; -} - // Reconstruct a layer's generator Monomial from the raw words stored on its LayerCore. template inline auto generator_from_words(const std::vector &gw) -> Monomial { diff --git a/cpp/monoprop/detail/evolution/layer_build/Common.h b/cpp/monoprop/detail/evolution/layer_build/Common.h index 53ebd758..1734df7b 100644 --- a/cpp/monoprop/detail/evolution/layer_build/Common.h +++ b/cpp/monoprop/detail/evolution/layer_build/Common.h @@ -28,6 +28,35 @@ namespace monoprop::detail { +/*! + * \brief The kinds of logical work range a threaded kernel reports to its range observer (see NoRangeObserver). + */ +enum class KernelRange { + cos_lazy, //!< scale_cos_lazy: one range per kColumnBlockWords fold words. + cos_mask, //!< scale_cos_mask: one range per kCosMaskRangeBlocks stored mask blocks. + fused_gather, //!< apply_fused_contract insert snapshots: one range per kFusedRecordRange records. + fused_apply, //!< apply_fused_contract rotations: one range per kFusedRecordRange records. + scan, //!< fused_find_and_collect: one range per contiguous run of whole fold blocks (scan_ranges). +}; + +/*! + * \brief The observer production kernels use: it does nothing and compiles away. + * + * A kernel calls `prepare(kind, ranges)` once on the calling thread before any range runs, then + * `visit(kind, range)` at the start of each logical range, from whichever thread owns that range. Tests + * substitute an observer that records the worker of each range; that seam is the only reason the kernels + * take an observer, and no production caller passes one. + */ +struct NoRangeObserver { + auto prepare(KernelRange /*kind*/, size_t /*ranges*/) const noexcept -> void {} + auto visit(KernelRange /*kind*/, size_t /*range*/) const noexcept -> void {} +}; + +//! Number of `per_range`-sized logical ranges covering `items`. +inline constexpr auto logical_ranges(size_t items, size_t per_range) -> size_t { + return (items + per_range - 1) / per_range; +} + // Sentinel for "no index resolved yet" in the resolve slots. Deliberately equal to // OperatorIndex::kNotFound, so one `>= size` bound check covers a miss and an unresolved slot alike. inline constexpr size_t kMissingIndex = std::numeric_limits::max(); diff --git a/cpp/monoprop/detail/evolution/layer_build/Engine.h b/cpp/monoprop/detail/evolution/layer_build/Engine.h index 00813e67..e69fd0ec 100644 --- a/cpp/monoprop/detail/evolution/layer_build/Engine.h +++ b/cpp/monoprop/detail/evolution/layer_build/Engine.h @@ -592,7 +592,9 @@ static inline auto empty_coeffs() -> const VecD & { } // Primary-path layer builder: one fused scan, then two resolve passes into the chosen sink. See LayerBuilder.h. -template +// The scan may run on workers (fused_find_and_collect); resolution, missing-ID assignment, insertion, +// publication and graph packing stay on the caller. `observer` is the scan's test seam (NoRangeObserver). +template auto build_layer(MPOperator &local_op, const Monomial &gen, const CutoffFn &cutoff_fn, @@ -609,7 +611,8 @@ auto build_layer(MPOperator &local_op, VecD *fused_scale_coeffs = nullptr, bool *fused_scale_out = nullptr, Basis basis = Basis::Majorana, - parallel::Options options = {}) -> std::shared_ptr { + parallel::Options options = {}, + const Observer &observer = {}) -> std::shared_ptr { validate_only_rotate_len_k_(only_rotate_len_k, 2 * NumModes); const size_t my_rank = static_cast(mpi::rank(comm)); const size_t R = static_cast(mpi::size(comm)); @@ -649,30 +652,40 @@ auto build_layer(MPOperator &local_op, if (!identity_gen) { double *const sweep_ptr = fused_scale ? fused_scale_coeffs->data() : nullptr; fused = with_algebra(basis, [&]() { - return fused_find_and_collect(local_op, - gen, - cut_eval, - cut_st, - coeffs, - only_rotate_len_k, - scan_window, - my_rank, - router, - gen_shift, - /*capture_values=*/use_fused, - sweep_ptr, - cos_build, - options); + return fused_find_and_collect(local_op, + gen, + cut_eval, + cut_st, + coeffs, + only_rotate_len_k, + scan_window, + my_rank, + router, + gen_shift, + /*capture_values=*/use_fused, + sweep_ptr, + cos_build, + options, + observer); }); if (fused.cos_blocks.size() == 1) { // The serial scan produces a single cosine block set — take it wholesale. cos_all = std::move(fused.cos_blocks[0]); } else { - // Cosine block sets are disjoint and ascending; concatenate in order. + // A threaded scan produces one set per word range, in range order: the ranges own disjoint + // ascending words, so concatenating in order keeps the blocks ascending and disjoint. + size_t blocks = 0; for (const auto &block : fused.cos_blocks) { + blocks += block.blocks.size(); + } + cos_all.blocks.reserve(blocks); + for (auto &block : fused.cos_blocks) { + assert(block.blocks.empty() || cos_all.blocks.empty() + || cos_all.blocks.back().first < block.blocks.front().first); cos_all.total_count += block.total_count; cos_all.blocks.insert(cos_all.blocks.end(), block.blocks.begin(), block.blocks.end()); + CosMask{}.blocks.swap(block.blocks); // release each range's copy as soon as it is appended } } fused.cos_blocks = std::vector{}; diff --git a/cpp/monoprop/detail/evolution/layer_build/Scan.h b/cpp/monoprop/detail/evolution/layer_build/Scan.h index 0bb2958b..f9451d3a 100644 --- a/cpp/monoprop/detail/evolution/layer_build/Scan.h +++ b/cpp/monoprop/detail/evolution/layer_build/Scan.h @@ -14,14 +14,19 @@ #pragma once +#include #include #include #include #include #include +#include +#include #include #include +#include #include +#include #include #include "monoprop/TypeAliases.h" @@ -38,6 +43,7 @@ #include "monoprop/detail/operator/InvertedIndex.h" #include "monoprop/detail/operator/MPOperator.h" #include "monoprop/detail/parallel/Options.h" +#include "monoprop/detail/parallel/Workshare.h" namespace monoprop::detail { @@ -225,244 +231,528 @@ struct FusedScanResult { SelfQueryStage follower_self; }; -// Classify, cut off and emit in one pass over the anticommuting terms. Queries go to the owner of -// M'=M⊕G (routing::Router; self at R==1) in ascending source-index order, so resolve and index assignment are -// deterministic. `fused_scale_coeffs` (no length cap only; must alias coeffs.data()) scales every anticommuting -// coeff in place by `fused_scale_cos`=cos(2·build_angle), so no cosine set is built and a hit's stored -// value is post-cos (resolve recovers it via 1/cos). -// -// `gen_shift` must be router.rank_shift(gen) and `op` hold only terms `my_rank` owns, so the owner of -// M⊕G is rank(M) ^ gen_shift (asserted against dest()). `window` is the plan's window for `my_rank`. +/*! + * \brief Number of logical word ranges a threaded scan of `words` inverted-index words uses. + * + * `min(options.threads, ceil(words / kColumnBlockWords))`: a function of the word count and the budget + * only, never of the team the runtime actually provides, so range IDs and the merged order are fixed. + * + * \throws std::invalid_argument if `options.threads` is not positive. + */ +inline auto scan_ranges(size_t words, parallel::Options options) -> size_t { + if (options.threads < 1) { + throw std::invalid_argument( + std::format("fused_find_and_collect: the thread budget must be positive, got {}", options.threads)); + } + return std::min(static_cast(options.threads), logical_ranges(words, kColumnBlockWords)); +} + +/*! + * \brief Words `[first, second)` of logical scan range `range` out of `ranges` over `words` words. + * + * Ranges are contiguous runs of whole kColumnBlockWords fold blocks, ascending with the range ID and + * covering every word; each is nonempty when `ranges` does not exceed the block count (scan_ranges). + */ +inline auto scan_range_words(size_t range, size_t ranges, size_t words) -> std::pair { + assert(range < ranges && ranges <= logical_ranges(words, kColumnBlockWords)); + // Rows are uint32 TermIndex values, so words <= 2^26, blocks <= 2^16 and these products stay below 2^32. + const size_t blocks = logical_ranges(words, kColumnBlockWords); + const size_t lo = (range * blocks / ranges) * kColumnBlockWords; + const size_t hi = std::min(((range + 1) * blocks / ranges) * kColumnBlockWords, words); + return {lo, hi}; +} + +/*! + * \brief Scan inverted-index words `[wlo, whi)` of `op` for terms anticommuting with `gen`, and emit their + * partner queries and cosine words into a fresh result. + * + * This is the whole serial scan restricted to one word range; fused_find_and_collect runs it once over + * every word, or once per logical range on scan workers and merges the pieces in range order. It reads only + * frozen storage and never calls a lazy accessor: `index` must be `op.inverted_index()` and `row_parity` + * its `row_parity_words()` (null unless the fold needs the odd-|G| correction), both prepared on the caller. + * The tail mask comes from the index's global last word, so only the range holding that word masks it. All + * scratch is private to the invocation or thread_local on the executing thread. With `fused_scale_coeffs` + * it writes the coefficients of anticommuting sources in its own words only, each once, after reading the + * pre-scale value its cutoff decisions and captured value use. It resolves no target, grows no store and + * calls no MPI. + * + * The remaining parameters are fused_find_and_collect's; see there. + */ template -auto fused_find_and_collect(const MPOperator &op, - const Monomial &gen, - const CutoffEvaluator &cutoff_eval, - const CutoffContext &cut_st, - const VecD &coeffs, - std::optional only_rotate_len_k, - mpi::SlotWindow window, - size_t my_rank, - const routing::Router &router, - size_t gen_shift, - bool capture_values = false, - double *fused_scale_coeffs = nullptr, - double fused_scale_cos = 1.0, - [[maybe_unused]] parallel::Options options = {}) -> FusedScanResult { - validate_only_rotate_len_k_(only_rotate_len_k, 2 * NumModes); +auto fused_find_and_collect_range(const MPOperator &op, + const Monomial &gen, + const CutoffEvaluator &cutoff_eval, + const CutoffContext &cut_st, + const VecD &coeffs, + std::optional only_rotate_len_k, + mpi::SlotWindow window, + size_t my_rank, + const routing::Router &router, + size_t gen_shift, + const InvertedIndex &index, + const uint64_t *row_parity, + size_t wlo, + size_t whi, + bool capture_values, + double *fused_scale_coeffs, + double fused_scale_cos) -> FusedScanResult { const size_t gen_pop = gen.count(); const size_t rank_count = router.flat_world(); const auto ectx = A::make_gen_context(gen); - assert(window.stop() <= rank_count && window.count != 0); FusedScanResult res; res.leader_queries.reset(window); res.leader_src.reset(window); res.follower_queries.reset(window); res.follower_src.reset(window); - // Sized even on the early returns below, so the fused engine's src_val_r access stays in bounds. if (capture_values) { res.leader_val.reset(window); res.follower_val.reset(window); } - { - // The anticommutation fold runs over G's inverted-index columns (Majorana: G; Pauli: J(G) = - // pair_swap(G), so pauli_anticommutes = parity(|M ∩ J(G)|), and Pauli never needs the odd-|G| - // correction since parity(|G ∩ J(G)|)=0). The pivot splitting each pair is a set bit of the real G - // (gen.find_first()), not J(G) — A and A⊕G differ exactly on G's bits. - const Monomial fold_gen = A::fold_generator(gen); - const bool g_odd = A::fold_needs_odd_correction(gen); - const auto gen_columns = build_even_parity_generator_columns(fold_gen); - if (gen_columns.count == 0) { - return res; + // The anticommutation fold runs over G's inverted-index columns (Majorana: G; Pauli: J(G) = + // pair_swap(G), so pauli_anticommutes = parity(|M ∩ J(G)|), and Pauli never needs the odd-|G| + // correction since parity(|G ∩ J(G)|)=0). The pivot splitting each pair is a set bit of the real G + // (gen.find_first()), not J(G) — A and A⊕G differ exactly on G's bits. + const Monomial fold_gen = A::fold_generator(gen); + const bool g_odd = A::fold_needs_odd_correction(gen); + const auto gen_columns = build_even_parity_generator_columns(fold_gen); + const std::span gen_cols(gen_columns.indices.data(), gen_columns.count); + assert(index.rows() == op.store->size() && (!g_odd || row_parity != nullptr)); + const size_t n = op.store->size(); + const size_t last_word = index.words() - 1; + const uint64_t last_word_mask = (n % 64 == 0) ? ~uint64_t{0} : ((uint64_t{1} << (n % 64)) - 1); + + auto &lq = res.leader_queries; + auto &ls = res.leader_src; + auto &lv = res.leader_val; + auto &fq = res.follower_queries; + auto &fs = res.follower_src; + auto &fv = res.follower_val; + + const OperatorIndex &ham = *op.store; + using RowPosT = typename OperatorIndex::PosT; + + // The generator's positions, once per range: the merge's second input. + std::vector gen_pos; + gen_pos.reserve(gen_pop); + for (size_t b = gen.find_first(); b < gen.size(); b = gen.find_next(b)) { + gen_pos.push_back(static_cast(b)); + } + // pbuf capacity is 2*NumModes: the partner's positions are distinct, so this always suffices. + std::vector pbuf(2 * NumModes); + + auto push = [&](const Monomial &dense, + std::span pos, + int phase, + size_t i, + double v_src, + bool is_follower) { + // Single rank: every partner is self-owned, skip the O(W) hash; multi-rank routes by owner. + // Must agree with find_rank (both go through routing::Router), or a row duplicates silently. + size_t r_prime = my_rank; + if (rank_count != 1) { + r_prime = router.dest_from_shift(dense, my_rank, gen_shift); + assert(r_prime == router.dest(dense)); // an identity, not an approximation } - const auto &inverted_index = op.inverted_index(); - const size_t word_count = inverted_index.words(); - if (word_count == 0) { - return res; + if (r_prime == my_rank) { + (is_follower ? res.follower_self : res.leader_self).push(pos, phase); } - const uint64_t *const row_parity_ptr = g_odd ? inverted_index.row_parity_words() : nullptr; - const size_t n = op.store->size(); - // The fused sweep writes fused_scale_coeffs[i] for every anticommuting i < n, so it must be the - // very array the reads come from and cover the full operator — a violation corrupts 1/cos recovery. - assert(fused_scale_coeffs == nullptr || (fused_scale_coeffs == coeffs.data() && coeffs.size() >= n)); - - const size_t last_word = word_count - 1; - const uint64_t last_word_mask = (n % 64 == 0) ? ~uint64_t{0} : ((uint64_t{1} << (n % 64)) - 1); - const std::span gen_cols(gen_columns.indices.data(), gen_columns.count); - - // The scan can be skipped only when every fold column is empty; a dense column always holds ≥1 - // posting, so any dense column makes it non-empty. - bool fold_cols_empty = true; - for (size_t ci = 0; ci < gen_columns.count; ++ci) { - const size_t c = gen_columns.indices[ci]; - if (inverted_index.column_is_dense(c)) { - fold_cols_empty = false; - } - else if (!inverted_index.sparse_column_rows(c).empty()) { - fold_cols_empty = false; - } + else { + QueryWire::push(is_follower ? fq.at_slot(r_prime) : lq.at_slot(r_prime), pos, phase); } - // Zero-postings early-out: no term touches a fold column ⇒ nothing anticommutes, so skip pass 1. - // g_odd guard is load-bearing: an odd Majorana generator anticommutes with disjoint odd-weight terms. - const bool skip_scan = !g_odd && fold_cols_empty; - - auto &lq = res.leader_queries; - auto &ls = res.leader_src; - auto &lv = res.leader_val; - auto &fq = res.follower_queries; - auto &fs = res.follower_src; - auto &fv = res.follower_val; - - const OperatorIndex &ham = *op.store; - using RowPosT = typename OperatorIndex::PosT; - - // The generator's positions, once per gate: the merge's second input. - std::vector gen_pos; - gen_pos.reserve(gen_pop); - for (size_t b = gen.find_first(); b < gen.size(); b = gen.find_next(b)) { - gen_pos.push_back(static_cast(b)); + (is_follower ? fs : ls).at_slot(r_prime).push_back(i); + if (capture_values) { + (is_follower ? fv : lv).at_slot(r_prime).push_back(v_src); } - // pbuf capacity is 2*NumModes: the partner's positions are distinct, so this always suffices. - std::vector pbuf(2 * NumModes); - - auto push = [&](const Monomial &dense, - std::span pos, - int phase, - size_t i, - double v_src, - bool is_follower) { - // Single rank: every partner is self-owned, skip the O(W) hash; multi-rank routes by owner. - // Must agree with find_rank (both go through routing::Router), or a row duplicates silently. - size_t r_prime = my_rank; - if (rank_count != 1) { - r_prime = router.dest_from_shift(dense, my_rank, gen_shift); - assert(r_prime == router.dest(dense)); // an identity, not an approximation - } - if (r_prime == my_rank) { - (is_follower ? res.follower_self : res.leader_self).push(pos, phase); - } - else { - QueryWire::push(is_follower ? fq.at_slot(r_prime) : lq.at_slot(r_prime), pos, phase); - } - (is_follower ? fs : ls).at_slot(r_prime).push_back(i); - if (capture_values) { - (is_follower ? fv : lv).at_slot(r_prime).push_back(v_src); - } - }; + }; - // The dynamic gate runs before emit_term_products, so a gate-rejected term computes no products. - // abs_c/v_src come from the caller's coeff read, not re-read. - auto emit = [&](size_t mono_pop, size_t i, double abs_c, double v_src, bool is_follower) { - if (!rotation_dynamic_gate(only_rotate_len_k, mono_pop, cut_st, abs_c)) { - return; - } - const auto p = emit_term_products(ham, - i, - ectx, - std::span(gen_pos), - std::span(pbuf)); - // Structural cutoff on the partner M⊕G, unless upper_atol rescues it (CutoffContext::is_above_upper). - const bool struct_pass = cutoff_eval.passes_with_popcount(p.new_mono, p.k); - if (!struct_pass && !cut_st.is_above_upper(abs_c)) { - return; - } - const int phase = A::emit_phase(p.phase_factor, mono_pop, gen_pop, p.overlap); - push(p.new_mono, std::span(pbuf).first(p.k), phase, i, v_src, is_follower); - }; - - // Pass 1 and pass 2 stay fused over `nz`: splitting them regressed measurably, as `nz` spills L1 - // between them. `nz` is thread_local so each partition master reuses its capacity across gates. - thread_local std::vector nz; - size_t n_anti = 0; - size_t n_foll = 0; - if (skip_scan) { - nz.clear(); // pass 1 clears it on entry; the skip must too (thread_local reuse) + // The dynamic gate runs before emit_term_products, so a gate-rejected term computes no products. + // abs_c/v_src come from the caller's coeff read, not re-read. + auto emit = [&](size_t mono_pop, size_t i, double abs_c, double v_src, bool is_follower) { + if (!rotation_dynamic_gate(only_rotate_len_k, mono_pop, cut_st, abs_c)) { + return; } - else { - even_parity_scan_pass1(inverted_index, - gen_cols, - gen.find_first(), - /*wlo=*/0, - /*whi=*/word_count, - last_word, - last_word_mask, - g_odd, - row_parity_ptr, - nz, - n_anti, - n_foll); + const auto p = + emit_term_products(ham, i, ectx, std::span(gen_pos), std::span(pbuf)); + // Structural cutoff on the partner M⊕G, unless upper_atol rescues it (CutoffContext::is_above_upper). + const bool struct_pass = cutoff_eval.passes_with_popcount(p.new_mono, p.k); + if (!struct_pass && !cut_st.is_above_upper(abs_c)) { + return; } - if (rank_count == 1) { - // A hint only; wider terms grow the buffer as needed. - const size_t pq = QueryWire::kReservePositionsPerQuery; - res.leader_self.reserve(n_anti - n_foll, pq); - ls.at_slot(my_rank).reserve(n_anti - n_foll); - res.follower_self.reserve(n_foll, pq); - fs.at_slot(my_rank).reserve(n_foll); + const int phase = A::emit_phase(p.phase_factor, mono_pop, gen_pop, p.overlap); + push(p.new_mono, std::span(pbuf).first(p.k), phase, i, v_src, is_follower); + }; + + // Pass 1 and pass 2 stay fused over `nz`: splitting them regressed measurably, as `nz` spills L1 + // between them. `nz` is thread_local, acquired here on the executing thread, so a scan worker reuses its + // own capacity across gates and never shares the caller's. + thread_local std::vector nz; + size_t n_anti = 0; + size_t n_foll = 0; + even_parity_scan_pass1(index, + gen_cols, + gen.find_first(), + wlo, + whi, + last_word, + last_word_mask, + g_odd, + row_parity, + nz, + n_anti, + n_foll); + if (rank_count == 1) { + // A hint only; wider terms grow the buffer as needed. + const size_t pq = QueryWire::kReservePositionsPerQuery; + res.leader_self.reserve(n_anti - n_foll, pq); + ls.at_slot(my_rank).reserve(n_anti - n_foll); + res.follower_self.reserve(n_foll, pq); + fs.at_slot(my_rank).reserve(n_foll); + } + auto derive_coeff = [&](size_t i) -> std::pair { + if (capture_values) { + const double v_src = (i < coeffs.size()) ? coeffs[i] : 0.0; + return {v_src, cut_st.use_coeff_checks ? std::abs(v_src) : 0.0}; } - auto derive_coeff = [&](size_t i) -> std::pair { - if (capture_values) { - const double v_src = (i < coeffs.size()) ? coeffs[i] : 0.0; - return {v_src, cut_st.use_coeff_checks ? std::abs(v_src) : 0.0}; - } - return {0.0, cut_st.abs_coeff_for(i, coeffs)}; - }; - const bool word_aligned_cos = !only_rotate_len_k.has_value(); - CosineWordBuilder cos_b; - for (const auto &w : nz) { - if (word_aligned_cos && fused_scale_coeffs != nullptr) { - // Fused cos sweep: scaling in place here is what replaces building a cosine set. - for (uint64_t m = w.overlap; m; m &= m - 1) { - const size_t tz = static_cast(std::countr_zero(m)); - const size_t i = w.base + tz; - const double v_src = fused_scale_coeffs[i]; - fused_scale_coeffs[i] = v_src * fused_scale_cos; - const double abs_c = std::abs(v_src); - if (cut_st.is_below_sin(abs_c)) { - continue; - } - const size_t mono_pop = op.store->popcount(i); - const bool is_follower = (w.foll >> tz) & 1u; - emit(mono_pop, i, abs_c, v_src, is_follower); + return {0.0, cut_st.abs_coeff_for(i, coeffs)}; + }; + const bool word_aligned_cos = !only_rotate_len_k.has_value(); + CosineWordBuilder cos_b; + for (const auto &w : nz) { + if (word_aligned_cos && fused_scale_coeffs != nullptr) { + // Fused cos sweep: scaling in place here is what replaces building a cosine set. + for (uint64_t m = w.overlap; m; m &= m - 1) { + const size_t tz = static_cast(std::countr_zero(m)); + const size_t i = w.base + tz; + const double v_src = fused_scale_coeffs[i]; + fused_scale_coeffs[i] = v_src * fused_scale_cos; + const double abs_c = std::abs(v_src); + if (cut_st.is_below_sin(abs_c)) { + continue; } + const size_t mono_pop = op.store->popcount(i); + const bool is_follower = (w.foll >> tz) & 1u; + emit(mono_pop, i, abs_c, v_src, is_follower); } - else if (word_aligned_cos) { - // No orbital gate: record the whole word in the cosine set, then per bit apply the atol - // gate before the popcount row read — deferring popcount saves random packed-row loads. - cos_b.push_word(w.base, w.overlap); - for (uint64_t m = w.overlap; m; m &= m - 1) { - const size_t tz = static_cast(std::countr_zero(m)); - const size_t i = w.base + tz; - const auto [v_src, abs_c] = derive_coeff(i); - if (cut_st.is_below_sin(abs_c)) { - continue; - } - const size_t mono_pop = op.store->popcount(i); - const bool is_follower = (w.foll >> tz) & 1u; - emit(mono_pop, i, abs_c, v_src, is_follower); + } + else if (word_aligned_cos) { + // No orbital gate: record the whole word in the cosine set, then per bit apply the atol + // gate before the popcount row read — deferring popcount saves random packed-row loads. + cos_b.push_word(w.base, w.overlap); + for (uint64_t m = w.overlap; m; m &= m - 1) { + const size_t tz = static_cast(std::countr_zero(m)); + const size_t i = w.base + tz; + const auto [v_src, abs_c] = derive_coeff(i); + if (cut_st.is_below_sin(abs_c)) { + continue; } + const size_t mono_pop = op.store->popcount(i); + const bool is_follower = (w.foll >> tz) & 1u; + emit(mono_pop, i, abs_c, v_src, is_follower); } - else { - // Orbital gate active: it needs mono_pop, and the per-index cosine push covers only - // orbital-passing terms, so the popcount row read must precede both. - for (uint64_t m = w.overlap; m; m &= m - 1) { - const size_t tz = static_cast(std::countr_zero(m)); - const size_t i = w.base + tz; - const size_t mono_pop = op.store->popcount(i); - if (mono_pop > static_cast(*only_rotate_len_k)) { - continue; - } - cos_b.push_index(i); - const auto [v_src, abs_c] = derive_coeff(i); - const bool is_follower = (w.foll >> tz) & 1u; - emit(mono_pop, i, abs_c, v_src, is_follower); + } + else { + // Orbital gate active: it needs mono_pop, and the per-index cosine push covers only + // orbital-passing terms, so the popcount row read must precede both. + for (uint64_t m = w.overlap; m; m &= m - 1) { + const size_t tz = static_cast(std::countr_zero(m)); + const size_t i = w.base + tz; + const size_t mono_pop = op.store->popcount(i); + if (mono_pop > static_cast(*only_rotate_len_k)) { + continue; } + cos_b.push_index(i); + const auto [v_src, abs_c] = derive_coeff(i); + const bool is_follower = (w.foll >> tz) & 1u; + emit(mono_pop, i, abs_c, v_src, is_follower); } } - res.cos_blocks.push_back(cos_b.finish()); } + res.cos_blocks.push_back(cos_b.finish()); return res; } +// `total + add`, refusing a sum a merged scan buffer could not hold. +inline auto checked_scan_total(size_t total, size_t add, size_t max_size, const char *what) -> size_t { + if (add > max_size || total > max_size - add) { + throw std::length_error(std::format("fused_find_and_collect: the merged {} exceed {}", what, max_size)); + } + return total + add; +} + +// Concatenates one stream across the pieces in range order into a single buffer and releases every piece's +// buffer. Moves a piece's buffer instead of copying it when it holds the whole stream or already has room +// for the total; otherwise reserves the exact total once. +template +auto merge_scan_stream(Pieces &pieces, Get &&get, const char *what) -> Vec { + constexpr size_t npos = std::numeric_limits::max(); + size_t total = 0; + size_t first = npos; + size_t nonempty = 0; + for (size_t k = 0; k < pieces.size(); ++k) { + const Vec &v = get(pieces[k]); + total = checked_scan_total(total, v.size(), v.max_size(), what); + if (!v.empty()) { + first = (first == npos) ? k : first; + ++nonempty; + } + } + Vec out; + if (first != npos) { + Vec &lead = get(pieces[first]); + size_t next = first; + if (nonempty == 1 || lead.capacity() >= total) { + out = std::move(lead); + ++next; + } + else { + out.reserve(total); + } + for (size_t k = next; k < pieces.size(); ++k) { + const Vec &v = get(pieces[k]); + out.insert(out.end(), v.begin(), v.end()); + Vec{}.swap(get(pieces[k])); // release now: clear() would keep the capacity + } + } + // Empty pieces may still hold a reservation. + for (auto &piece : pieces) { + Vec{}.swap(get(piece)); + } + return out; +} + +// Appends each piece's self-stage queries in range order through SelfQueryStage::push, which rebases the +// position offsets; only the logical queries/positions are read, never a capacity tail. +template +auto merge_self_stage(std::vector> &pieces, + SelfQueryStage FusedScanResult::*member) -> SelfQueryStage { + using Stage = SelfQueryStage; + using PosT = typename Stage::PosT; + constexpr size_t npos = std::numeric_limits::max(); + size_t queries = 0; + size_t positions = 0; + size_t first = npos; + size_t nonempty = 0; + for (size_t k = 0; k < pieces.size(); ++k) { + const Stage &st = pieces[k].*member; + queries = checked_scan_total(queries, st.size(), st.pos_off.max_size(), "self queries"); + positions = checked_scan_total(positions, st.positions(), st.pos_flat.max_size(), "self positions"); + if (st.size() != 0) { + first = (first == npos) ? k : first; + ++nonempty; + } + } + Stage out; + if (first != npos) { + Stage &lead = pieces[first].*member; + size_t next = first; + if (nonempty == 1 || (lead.pos_off.size() >= queries && lead.pos_flat.size() >= positions)) { + out = std::move(lead); + ++next; + } + else { + out.reserve(queries, 0); + // pos_flat is sized to capacity, not filled (see SelfQueryStage), so this is the exact reservation. + out.pos_flat.resize(positions); + } + for (size_t k = next; k < pieces.size(); ++k) { + const Stage &st = pieces[k].*member; + for (size_t q = 0; q < st.size(); ++q) { + out.push(std::span(st.pos_flat.data() + st.pos_off[q], st.k_of[q]), st.phase_of[q]); + } + pieces[k].*member = Stage{}; + } + assert(out.size() == queries && out.positions() == positions); + } + for (auto &piece : pieces) { + piece.*member = Stage{}; + } + return out; +} + +/*! + * \brief Merge the per-range results of one threaded scan into the result the serial scan produces. + * + * `pieces[r]` must be range `r`'s result over `window`, ranges ascending in source words. Each stream is + * concatenated in range order, so every destination's leader and follower queries, sources and values keep + * their source-order subsequence; wire records are whole words and are copied, never re-encoded. The two + * self stages merge separately through SelfQueryStage::push. The cosine sets are moved in range order and + * stay ascending and disjoint. Value streams exist only when `capture_values`. Every piece's buffers are + * released as soon as they are drained. + * + * \throws std::length_error if a merged stream would exceed its container's max_size. + */ +template +auto merge_scan_ranges(std::vector> &pieces, + mpi::SlotWindow window, + [[maybe_unused]] size_t my_rank, + bool capture_values) -> FusedScanResult { + using Result = FusedScanResult; + Result out; + const auto merge_window = [&](mpi::WindowVec Result::*member, const char *what) { + mpi::WindowVec &merged = out.*member; + merged.reset(window); + for (const auto wi : window.indices()) { + merged[wi] = merge_scan_stream(pieces, [&](Result &p) -> T & { return (p.*member)[wi]; }, what); + } + }; + merge_window(&Result::leader_queries, "leader queries"); + merge_window(&Result::leader_src, "leader sources"); + merge_window(&Result::follower_queries, "follower queries"); + merge_window(&Result::follower_src, "follower sources"); + if (capture_values) { + merge_window(&Result::leader_val, "leader values"); + merge_window(&Result::follower_val, "follower values"); + } + out.leader_self = merge_self_stage(pieces, &Result::leader_self); + out.follower_self = merge_self_stage(pieces, &Result::follower_self); + size_t masks = 0; + for (const auto &piece : pieces) { + masks = checked_scan_total(masks, piece.cos_blocks.size(), out.cos_blocks.max_size(), "cosine sets"); + } + out.cos_blocks.reserve(masks); + for (auto &piece : pieces) { + for (auto &mask : piece.cos_blocks) { + out.cos_blocks.push_back(std::move(mask)); + } + std::vector{}.swap(piece.cos_blocks); + } +#ifndef NDEBUG + if (window.contains(my_rank)) { + assert(out.leader_queries.at_slot(my_rank).empty() && out.follower_queries.at_slot(my_rank).empty()); + assert(out.leader_self.size() == out.leader_src.at_slot(my_rank).size()); + assert(out.follower_self.size() == out.follower_src.at_slot(my_rank).size()); + assert(!capture_values || out.leader_self.size() == out.leader_val.at_slot(my_rank).size()); + assert(!capture_values || out.follower_self.size() == out.follower_val.at_slot(my_rank).size()); + } + else { + assert(out.leader_self.size() == 0 && out.follower_self.size() == 0); + } +#endif + return out; +} + +// Classify, cut off and emit in one pass over the anticommuting terms. Queries go to the owner of +// M'=M⊕G (routing::Router; self at R==1) in ascending source-index order, so resolve and index assignment are +// deterministic. `fused_scale_coeffs` (no length cap only; must alias coeffs.data()) scales every anticommuting +// coeff in place by `fused_scale_cos`=cos(2·build_angle), so no cosine set is built and a hit's stored +// value is post-cos (resolve recovers it via 1/cos). +// +// `gen_shift` must be router.rank_shift(gen) and `op` hold only terms `my_rank` owns, so the owner of +// M⊕G is rank(M) ^ gen_shift (asserted against dest()). `window` is the plan's window for `my_rank`. +// +// Threading: with a parallel-safe cutoff (CutoffEvaluator::parallel_safe) and at least two logical ranges +// (scan_ranges), the words split into ranges that scan workers traverse with fused_find_and_collect_range, +// each into a private result, and merge_scan_ranges joins them in range order: the output is identical to +// the serial scan's at every budget and team size. Otherwise the serial scan runs on the caller. Lazy +// index/parity data is prepared here, on the caller, before any worker starts. `observer` is a test seam +// (see NoRangeObserver) reporting the KernelRange::scan ranges. +template +auto fused_find_and_collect(const MPOperator &op, + const Monomial &gen, + const CutoffEvaluator &cutoff_eval, + const CutoffContext &cut_st, + const VecD &coeffs, + std::optional only_rotate_len_k, + mpi::SlotWindow window, + size_t my_rank, + const routing::Router &router, + size_t gen_shift, + bool capture_values = false, + double *fused_scale_coeffs = nullptr, + double fused_scale_cos = 1.0, + parallel::Options options = {}, + const Observer &observer = {}) -> FusedScanResult { + validate_only_rotate_len_k_(only_rotate_len_k, 2 * NumModes); + (void)scan_ranges(0, options); // validates the budget before anything runs + assert(window.stop() <= router.flat_world() && window.count != 0); + + // Sized even on the early returns below, so the fused engine's src_val_r access stays in bounds. + const auto empty_result = [&] { + FusedScanResult res; + res.leader_queries.reset(window); + res.leader_src.reset(window); + res.follower_queries.reset(window); + res.follower_src.reset(window); + if (capture_values) { + res.leader_val.reset(window); + res.follower_val.reset(window); + } + return res; + }; + + const Monomial fold_gen = A::fold_generator(gen); + const bool g_odd = A::fold_needs_odd_correction(gen); + const auto gen_columns = build_even_parity_generator_columns(fold_gen); + if (gen_columns.count == 0) { + observer.prepare(KernelRange::scan, 0); + return empty_result(); + } + // Both lazy accessors may build caches, so they run here and never in a worker. + const auto &index = op.inverted_index(); + const size_t word_count = index.words(); + if (word_count == 0) { + observer.prepare(KernelRange::scan, 0); + return empty_result(); + } + const uint64_t *const row_parity = g_odd ? index.row_parity_words() : nullptr; + // The fused sweep writes fused_scale_coeffs[i] for every anticommuting i < n, so it must be the + // very array the reads come from and cover the full operator — a violation corrupts 1/cos recovery. + assert(fused_scale_coeffs == nullptr || (fused_scale_coeffs == coeffs.data() && coeffs.size() >= op.store->size())); + + // The scan can be skipped only when every fold column is empty; a dense column always holds ≥1 + // posting, so any dense column makes it non-empty. + bool fold_cols_empty = true; + for (size_t ci = 0; ci < gen_columns.count; ++ci) { + const size_t c = gen_columns.indices[ci]; + if (index.column_is_dense(c) || !index.sparse_column_rows(c).empty()) { + fold_cols_empty = false; + } + } + // Zero-postings early-out: no term touches a fold column ⇒ nothing anticommutes, so skip pass 1. + // g_odd guard is load-bearing: an odd Majorana generator anticommutes with disjoint odd-weight terms. + if (!g_odd && fold_cols_empty) { + observer.prepare(KernelRange::scan, 0); + auto res = empty_result(); + res.cos_blocks.push_back(CosMask{}); + return res; + } + + const auto scan_words = [&](size_t wlo, size_t whi) { + return fused_find_and_collect_range(op, + gen, + cutoff_eval, + cut_st, + coeffs, + only_rotate_len_k, + window, + my_rank, + router, + gen_shift, + index, + row_parity, + wlo, + whi, + capture_values, + fused_scale_coeffs, + fused_scale_cos); + }; + // Opaque cutoffs and small scans keep the serial scan: one range on the caller, no merge. + const size_t ranges = cutoff_eval.parallel_safe() ? scan_ranges(word_count, options) : 1; + observer.prepare(KernelRange::scan, ranges); + if (ranges <= 1) { + observer.visit(KernelRange::scan, 0); + return scan_words(0, word_count); + } + // One private result per logical range, never per fold block: O(ranges * window.count) metadata. + std::vector> pieces(ranges); + parallel::for_blocks(ranges, options, [&](size_t range) { + observer.visit(KernelRange::scan, range); + const auto [wlo, whi] = scan_range_words(range, ranges, word_count); + pieces[range] = scan_words(wlo, whi); + }); + return merge_scan_ranges(pieces, window, my_rank, capture_values); +} + } // namespace monoprop::detail diff --git a/cpp/tests/CMakeLists.txt b/cpp/tests/CMakeLists.txt index 4d89a5a8..e599919a 100644 --- a/cpp/tests/CMakeLists.txt +++ b/cpp/tests/CMakeLists.txt @@ -149,7 +149,7 @@ monoprop_add_launch_env_case( # participation must not depend on receiving the full budget. set( _monoprop_kernel_cases - "openmp_cos_mask_exact,openmp_cos_mask_workers_participate,openmp_cos_lazy_workers_participate,openmp_fused_apply_exact_on_disjoint_records,openmp_fused_apply_workers_participate,openmp_kernel_worker_failures_join_before_rethrow,combined_threaded_lazy_fold_matches_oracle,fused_real_records_have_one_add_owner_and_apply_exactly,fused_threaded_prototype_matches_serial_prototype_exactly" + "openmp_cos_mask_exact,openmp_cos_mask_workers_participate,openmp_cos_lazy_workers_participate,openmp_fused_apply_exact_on_disjoint_records,openmp_fused_apply_workers_participate,openmp_kernel_worker_failures_join_before_rethrow,combined_threaded_lazy_fold_matches_oracle,fused_real_records_have_one_add_owner_and_apply_exactly,fused_threaded_prototype_matches_serial_prototype_exactly,openmp_scan_ranges_match_serial_majorana,openmp_scan_workers_participate,openmp_scan_worker_failure_joins_before_rethrow,openmp_scan_prepares_cold_caches_on_the_caller,openmp_scan_pauli_ranges_match_serial,openmp_scan_threaded_construction_matches_serial_exactly,openmp_scan_construction_workers_participate" ) monoprop_add_launch_env_case( openmp_env_kernels_thread_limit_2 @@ -219,6 +219,10 @@ if(monoprop_ENABLE_MPI AND _monoprop_mpiexec) monoprop_add_failure_scenario( before_exchange before-exchange "" "injected failure in the scan" "UNGUARDED|NOT REACHED" ) + # Worker 1 of the scan's two-worker team owns word range 1; peers enter the query exchange normally. + monoprop_add_failure_scenario( + before_exchange_scan_worker before-exchange scan-worker "scan range 1 on OpenMP worker 1" "UNGUARDED|NOT REACHED" + ) monoprop_add_failure_scenario( active_ticket active-ticket ticket "injected failure while the replay Ticket" "UNGUARDED" ) diff --git a/cpp/tests/KernelTestSupport.h b/cpp/tests/KernelTestSupport.h index 1286d82c..983daf96 100644 --- a/cpp/tests/KernelTestSupport.h +++ b/cpp/tests/KernelTestSupport.h @@ -46,7 +46,7 @@ namespace kernel_test { using monoprop::detail::KernelRange; -inline constexpr size_t kKinds = 4; +inline constexpr size_t kKinds = 5; // What one kernel invocation reported for one kind of range. struct RangeSlots { diff --git a/cpp/tests/PropagatorTestAccess.h b/cpp/tests/PropagatorTestAccess.h index c3b5ed1a..936ca83e 100644 --- a/cpp/tests/PropagatorTestAccess.h +++ b/cpp/tests/PropagatorTestAccess.h @@ -23,7 +23,9 @@ #include #include "monoprop/MonomialPropagator.h" +#include "monoprop/algebra/AlgebraCommon.h" #include "monoprop/detail/evolution/layer_build/Common.h" +#include "monoprop/detail/evolution/layer_build/Engine.h" #include "monoprop/detail/parallel/Options.h" namespace monoprop::detail { @@ -83,6 +85,36 @@ struct PropagatorTestAccess { return g; } + // One graph-only layer (no coefficients, not appended to the graph) built by the real build_layer on the + // propagator's operator, budget and communicator, under the propagator's own operation guard, with a + // test observer on the scan's ranges. The operator may grow, so the object is for inspection only. + template + static auto build_layer_observed(Propagator &p, const VecZ &gate, const Observer &observer) + -> std::shared_ptr { + return p.run_operation_(true, [&](bool &mutation_started) { + const auto gen = indices_to_bitset_checked(gate, 2 * p.logical_num_modes_); + mutation_started = true; + return build_layer(p.mp_op_, + gen, + p.cutoff_fn_, + p.lower_atol_, + std::nullopt, + p.upper_atol_, + std::nullopt, + std::nullopt, + p.matched_scratch_, + p.comm_, + nullptr, + nullptr, + p.schrodinger_, + nullptr, + nullptr, + p.basis_, + p.parallel_, + observer); + }); + } + static auto partition_count(const Propagator &p) -> int { return p.partition_group_ ? p.partition_group_->partition_count() : 0; } diff --git a/cpp/tests/README.md b/cpp/tests/README.md index 312fdbea..e9aa9019 100644 --- a/cpp/tests/README.md +++ b/cpp/tests/README.md @@ -149,11 +149,23 @@ name and cannot address suite-nested cases, tests use flat from the real build path (one add-owner per slot, both pictures, cross-rank halves under MPI) in `fused_cos_sweep_tests.cpp`. `openmp_env_kernels_*` rerun them under `OMP_THREAD_LIMIT=2` and `OMP_DYNAMIC=TRUE`. +- **Threaded scan**: the `openmp_scan_*` cases compare the threaded bitmap scan + (`fused_find_and_collect` over word ranges, merged in range order) with budget one byte for byte: + every window slot's wire records, sources and values, both self stages, the cosine set and the + fused-sweep coefficients. `ScanTestSupport.h` builds a synthetic operator of four fold blocks per + rank and the scenario matrix (one rank, linear zero and non-zero shifts with non-zero window bases, + dense splitmix at three and four ranks; dense, sparse and empty pivots; Majorana and Pauli; caps, + cutoffs, atol boundaries, capture, fused sweep). They live in `evolution_detail_tests.cpp`, + `pauli_build_layer_tests.cpp`, `exact_upper_atol_rescue.cpp`, `majorana_cutoff_tests.cpp` (opaque + and basis-change cutoffs stay serial), `sparse_query_tests.cpp` (the merge itself) and + `fused_cos_sweep_tests.cpp` (full construction: rows, row IDs, graph layers and coefficients at + budgets 1-4). The scan's participation is read from its own ranges through the same observer. - **Distributed failure driver**: `mpi_failure_driver.cpp` is not part of the unit runner (the glob excludes it). The MPI build compiles it into `monoprop_mpi_failure_driver.x`, and `run_mpi_failure_scenario.cmake` runs each scenario on two ranks with a 30 s timeout, as the `mpi_failure_*` CTest entries. `active-ticket cosine-worker` throws from a worker of the threaded - cosine kernel while a replay Ticket is posted. + cosine kernel while a replay Ticket is posted; `before-exchange scan-worker` throws from a worker + of the threaded scan while the peer enters the query exchange. New `*.cpp` files are auto-discovered on the next configure, with no CMake edit needed. A file with its own `main()` must be excluded from the glob, as diff --git a/cpp/tests/ScanTestSupport.h b/cpp/tests/ScanTestSupport.h new file mode 100644 index 00000000..03c4f531 --- /dev/null +++ b/cpp/tests/ScanTestSupport.h @@ -0,0 +1,517 @@ +// Copyright 2026 Algorithmiq +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +#pragma once + +// Test-only support for the threaded bitmap scan: a synthetic operator spanning several fold blocks per rank, +// scan scenarios over windows, routers, pivots, bases and cutoffs, and an exact comparison of two scan +// results stream by stream. Every comparison is on the caller, after the scan has joined. + +#include + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include "KernelTestSupport.h" +#include "monoprop/TypeAliases.h" +#include "monoprop/algebra/Algebra.h" +#include "monoprop/algebra/AlgebraCommon.h" +#include "monoprop/detail/evolution/layer_build/Scan.h" +#include "monoprop/detail/mpi/Comm.h" +#include "monoprop/detail/mpi/MPIUtils.h" +#include "monoprop/detail/mpi/Routing.h" +#include "monoprop/detail/operator/InvertedIndex.h" +#include "monoprop/detail/operator/MPOperator.h" +#include "monoprop/detail/operator/RowAccess.h" + +namespace scan_test { + +using namespace monoprop; + +inline constexpr size_t kModes = 32; // 64 bits, so a monomial is one word +using Mono = Monomial; +using Op = detail::MPOperator; +using Result = detail::FusedScanResult; + +// Column tiers of the synthetic operator, by bit index (the generator's pivot is its lowest set bit): +inline constexpr size_t kEmptyEnd = 4; // bits [0, 4) are never set: empty fold columns +inline constexpr size_t kSparseEnd = 12; // bits [4, 12) are set with probability 1/256: sparse columns +// bits [12, 64) are set with probability 1/8: dense columns (the promotion threshold is 1/64). + +// Three full fold blocks and a one-word fourth, whose last word holds 37 rows: budgets 2-4 split it into +// 2-4 ranges, and the last range carries the global tail. +inline constexpr size_t kRowsPerRank = (3 * detail::kColumnBlockWords * 64) + 37; + +inline auto draw_term(kernel_test::SplitMix &rng) -> uint64_t { + uint64_t w = 0; + for (size_t b = kEmptyEnd; b < 64; ++b) { + const bool set = b < kSparseEnd ? rng.below(256) == 0 : rng.below(8) == 0; + w |= static_cast(set) << b; + } + return w; +} + +inline auto mono_of(uint64_t word) -> Mono { + Mono m{}; + for (uint64_t x = word; x != 0; x &= x - 1) { + m.set(static_cast(std::countr_zero(x))); + } + return m; +} + +inline auto mono_of_bits(std::initializer_list bits) -> Mono { + Mono m{}; + for (const auto b : bits) { + m.set(b); + } + return m; +} + +inline auto build_op(const std::vector &terms, Basis basis) -> Op { + Op op; + op.basis = basis; + detail::insert_absent_terms( + op, + terms.size(), + [&](size_t k) -> const Mono & { return terms[k]; }, + [&](size_t k, size_t base) { assign_row(*op.store, base + k, terms[k]); }); + return op; +} + +// Distinct terms this rank owns under `router`, `rows` of them, in draw order. Owned-only, as a propagator +// seeds each rank: the scan routes by rank(M) ^ rank_shift(G), which names the owner only for a local M. +inline auto owned_terms(const routing::Router &router, size_t my_rank, size_t rows, uint64_t seed) + -> std::vector { + kernel_test::SplitMix rng{seed}; + std::unordered_set seen; + std::vector terms; + terms.reserve(rows); + while (terms.size() < rows) { + const uint64_t w = draw_term(rng); + if (w == 0 || !seen.insert(w).second) { + continue; + } + const Mono m = mono_of(w); + if (router.flat_world() == 1 || find_rank(m, router) == my_rank) { + terms.push_back(m); + } + } + return terms; +} + +// Signed coefficients in (-1, 1), deterministic. +inline auto coefficients(size_t n, uint64_t seed) -> VecD { + kernel_test::SplitMix rng{seed}; + VecD c(n); + for (auto &v : c) { + v = (static_cast(rng.below(1U << 30)) / static_cast(1U << 29)) - 1.0; + } + return c; +} + +// One scan geometry: a router, the rank the scan runs on, and that rank's operator. +struct Geometry { + const char *label; + size_t ranks; + size_t parts; + bool linear; + size_t my_rank; +}; + +inline auto router_of(const Geometry &g) -> routing::Router { + return routing::Router::for_modes(g.ranks, g.parts, g.linear); +} + +// Operators are expensive to build at this size, so each (geometry, basis) is built once per process. +inline auto geometry_op(const Geometry &g, Basis basis) -> const Op & { + static std::map, Op> cache; + const auto key = std::tuple{g.ranks, g.parts, g.linear, g.my_rank, static_cast(basis)}; + auto it = cache.find(key); + if (it == cache.end()) { + const auto router = router_of(g); + auto op = build_op(owned_terms(router, g.my_rank, kRowsPerRank, 0x5CA11ULL + g.my_rank), basis); + (void)op.inverted_index(); + it = cache.emplace(key, std::move(op)).first; + } + return it->second; +} + +enum class Atol { + none, + lower_equal, // lower atol equal to one term's |sin|·|c|: that term is dropped (<=) + upper_equal, // upper atol equal to one term's |sin|·|c| under a tight cutoff: that term is rescued (>=) + drop_all, // lower atol above every |sin|·|c|: the scan traverses its words but emits nothing +}; + +// Everything one fused_find_and_collect call takes besides the operator and the budget. +struct Scenario { + const char *label; + Basis basis = Basis::Majorana; + Mono gen{}; + CutoffFn cutoff = detail::LengthCutoff{2 * kModes}; + std::optional cap = std::nullopt; // only_rotate_len_k + Atol atol = Atol::none; + bool capture = false; + bool fused_scale = false; // requires capture and no cap + double param = 0.3; +}; + +// A scan's outputs together with the coefficients it may have scaled. +struct Outcome { + Result result; + VecD coeffs; + mpi::SlotWindow window; + size_t my_rank = 0; +}; + +inline constexpr size_t kAtolTerm = 1234; // the term whose |sin|·|c| an atol scenario lands on exactly + +// Runs `s` on `op`, which must be the operator of geometry `g` (or one built the same way). +template +auto run_scan_on(const Op &op, + const Geometry &g, + const Scenario &s, + detail::parallel::Options options, + const Observer &observer = {}) -> Outcome { + const auto router = router_of(g); + const size_t shift = router.rank_shift(s.gen); + const mpi::PeerPlan plan{.sparse = router.is_linear(), .shift = static_cast(shift)}; + Outcome out; + out.window = plan.window(g.my_rank, router.ranks(), router.partitions()); + out.my_rank = g.my_rank; + out.coeffs = coefficients(op.size(), 0xC0EFFULL + g.my_rank); + const double abs_sin = std::abs(std::sin(2 * s.param)); + std::optional lower; + std::optional upper; + if (s.atol == Atol::lower_equal) { + lower = abs_sin * std::abs(out.coeffs[kAtolTerm]); + } + if (s.atol == Atol::upper_equal) { + upper = abs_sin * std::abs(out.coeffs[kAtolTerm]); + } + if (s.atol == Atol::drop_all) { + lower = 2.0; + } + const auto cut_st = + detail::build_majorana_evolution_cutoff_state(lower, std::cref(out.coeffs), upper, std::optional{s.param}); + const detail::CutoffEvaluator eval(s.cutoff); + double *const sweep = s.fused_scale ? out.coeffs.data() : nullptr; + const double cos_val = std::cos(2 * s.param); + out.result = with_algebra(s.basis, [&]() { + return detail::fused_find_and_collect(op, + s.gen, + eval, + cut_st, + out.coeffs, + s.cap, + out.window, + g.my_rank, + router, + shift, + s.capture, + sweep, + cos_val, + options, + observer); + }); + return out; +} + +template +auto run_scan(const Geometry &g, const Scenario &s, detail::parallel::Options options, const Observer &observer = {}) + -> Outcome { + return run_scan_on(geometry_op(g, s.basis), g, s, options, observer); +} + +// A fresh operator for `g`: its inverted index and row-parity cache have never been built. +inline auto cold_op(const Geometry &g, Basis basis) -> Op { + return build_op(owned_terms(router_of(g), g.my_rank, kRowsPerRank, 0x5CA11ULL + g.my_rank), basis); +} + +inline auto bitwise_equal(std::span a, std::span b) -> bool { + // memcmp must not see an empty vector's null data pointer, even for zero bytes. + return a.size() == b.size() && (a.empty() || std::memcmp(a.data(), b.data(), a.size() * sizeof(double)) == 0); +} + +// The concatenated cosine set, as LayerBuildEngine consumes it. +inline auto concatenated_cos(const Result &r) -> CosMask { + CosMask all; + for (const auto &m : r.cos_blocks) { + all.total_count += m.total_count; + all.blocks.insert(all.blocks.end(), m.blocks.begin(), m.blocks.end()); + } + return all; +} + +// Total queries a result emitted, over every stream. +inline auto emitted(const Result &r) -> size_t { + size_t n = 0; + for (const auto &v : r.leader_src) { + n += v.size(); + } + for (const auto &v : r.follower_src) { + n += v.size(); + } + return n; +} + +// The self stage's logical content only: never its capacity tail. +inline auto check_same_stage(const detail::SelfQueryStage &a, + const detail::SelfQueryStage &b, + const char *which) -> void { + BOOST_TEST_CONTEXT(which) { + BOOST_TEST_REQUIRE(a.size() == b.size()); + BOOST_TEST_REQUIRE(a.positions() == b.positions()); + for (size_t q = 0; q < a.size(); ++q) { + if (a.pos_off[q] != b.pos_off[q] || a.k_of[q] != b.k_of[q] || a.phase_of[q] != b.phase_of[q]) { + BOOST_TEST_REQUIRE(false, "self query " << q << " differs"); + } + } + BOOST_TEST(std::equal(a.pos_flat.begin(), + a.pos_flat.begin() + static_cast(a.positions()), + b.pos_flat.begin())); + } +} + +// Checks one result's own invariants: every populated stream carries the scan's window, value streams are +// empty without capture, self wire stays empty, and the self stages align with the self slot's sources. +inline auto check_shape(const Outcome &o, bool capture) -> void { + const Result &r = o.result; + BOOST_TEST((r.leader_queries.window() == o.window)); + BOOST_TEST((r.follower_queries.window() == o.window)); + BOOST_TEST((r.leader_src.window() == o.window)); + BOOST_TEST((r.follower_src.window() == o.window)); + if (capture) { + BOOST_TEST((r.leader_val.window() == o.window)); + BOOST_TEST((r.follower_val.window() == o.window)); + } + else { + BOOST_TEST(r.leader_val.size() == 0U); + BOOST_TEST(r.follower_val.size() == 0U); + } + if (o.window.contains(o.my_rank)) { + BOOST_TEST(r.leader_queries.at_slot(o.my_rank).empty()); + BOOST_TEST(r.follower_queries.at_slot(o.my_rank).empty()); + BOOST_TEST(r.leader_self.size() == r.leader_src.at_slot(o.my_rank).size()); + BOOST_TEST(r.follower_self.size() == r.follower_src.at_slot(o.my_rank).size()); + if (capture) { + BOOST_TEST(r.leader_self.size() == r.leader_val.at_slot(o.my_rank).size()); + BOOST_TEST(r.follower_self.size() == r.follower_val.at_slot(o.my_rank).size()); + } + } + else { + BOOST_TEST(r.leader_self.size() == 0U); + BOOST_TEST(r.follower_self.size() == 0U); + } + for (const auto wi : o.window.indices()) { + if (o.window.slot(wi) != o.my_rank) { // a remote slot: one wire record, at least one word, per source + BOOST_TEST(r.leader_queries[wi].size() >= r.leader_src[wi].size()); + BOOST_TEST(r.follower_queries[wi].size() >= r.follower_src[wi].size()); + } + if (capture) { + BOOST_TEST(r.leader_val[wi].size() == r.leader_src[wi].size()); + BOOST_TEST(r.follower_val[wi].size() == r.follower_src[wi].size()); + } + } + // Ascending and disjoint across the concatenation, which LayerBuildEngine relies on. + const auto cos = concatenated_cos(r); + size_t bits = 0; + for (size_t k = 0; k < cos.blocks.size(); ++k) { + bits += static_cast(std::popcount(cos.blocks[k].second)); + if (k != 0 && cos.blocks[k - 1].first >= cos.blocks[k].first) { + BOOST_TEST_REQUIRE(false, "cosine blocks not ascending at " << k); + } + } + BOOST_TEST(bits == cos.total_count); +} + +// Exact equality of two scans of the same inputs: every stream of every window slot, both self stages, +// the concatenated cosine set, and the (possibly scaled) coefficients. +inline auto check_same_scan(const Outcome &a, const Outcome &b, bool capture) -> void { + BOOST_TEST_REQUIRE((a.window == b.window)); + check_shape(a, capture); + check_shape(b, capture); + for (const auto wi : a.window.indices()) { + BOOST_TEST_CONTEXT("slot " << a.window.slot(wi)) { + BOOST_TEST((a.result.leader_queries[wi] == b.result.leader_queries[wi])); + BOOST_TEST((a.result.follower_queries[wi] == b.result.follower_queries[wi])); + BOOST_TEST((a.result.leader_src[wi] == b.result.leader_src[wi])); + BOOST_TEST((a.result.follower_src[wi] == b.result.follower_src[wi])); + if (capture) { + BOOST_TEST(bitwise_equal(a.result.leader_val[wi], b.result.leader_val[wi])); + BOOST_TEST(bitwise_equal(a.result.follower_val[wi], b.result.follower_val[wi])); + } + } + } + check_same_stage(a.result.leader_self, b.result.leader_self, "leader_self"); + check_same_stage(a.result.follower_self, b.result.follower_self, "follower_self"); + const auto ca = concatenated_cos(a.result); + const auto cb = concatenated_cos(b.result); + BOOST_TEST(ca.total_count == cb.total_count); + BOOST_TEST((ca.blocks == cb.blocks)); + BOOST_TEST(bitwise_equal(a.coeffs, b.coeffs)); +} + +// FNV-1a over a result's logical content, for comparing one build's scan with another's. +inline auto digest(const Outcome &o, bool capture) -> uint64_t { + uint64_t h = 1469598103934665603ULL; + const auto mix = [&h](uint64_t v) { + for (int k = 0; k < 8; ++k) { + h = (h ^ ((v >> (8 * k)) & 0xFFU)) * 1099511628211ULL; + } + }; + const auto mix_d = [&mix](double d) { mix(std::bit_cast(d)); }; + const Result &r = o.result; + for (const auto wi : o.window.indices()) { + mix(0xABCDEFULL); + for (const auto v : r.leader_queries[wi]) { + mix(v); + } + for (const auto v : r.follower_queries[wi]) { + mix(v); + } + for (const auto v : r.leader_src[wi]) { + mix(v); + } + for (const auto v : r.follower_src[wi]) { + mix(v); + } + if (capture) { + for (const auto v : r.leader_val[wi]) { + mix_d(v); + } + for (const auto v : r.follower_val[wi]) { + mix_d(v); + } + } + } + for (const auto *stage : {&r.leader_self, &r.follower_self}) { + mix(stage->size()); + for (size_t q = 0; q < stage->size(); ++q) { + mix(stage->pos_off[q]); + mix(stage->k_of[q]); + mix(static_cast(static_cast(stage->phase_of[q]))); + } + for (size_t p = 0; p < stage->positions(); ++p) { + mix(stage->pos_flat[p]); + } + } + const auto cos = concatenated_cos(r); + mix(cos.total_count); + for (const auto &[base, bits] : cos.blocks) { + mix(base); + mix(bits); + } + for (const auto c : o.coeffs) { + mix_d(c); + } + return h; +} + +enum class Expect { + output, // the scan emits queries from several ranges + empty_scanned, // the scan traverses its words but emits nothing + not_scanned, // the caller returns before any traversal (no fold columns, or all of them empty) +}; + +// Runs `s` at budget 1 and at budgets 2-4 and requires identical results. Checks on the caller, after each +// scan joined, that the threaded runs split the words into min(budget, fold blocks) ranges that ran inside +// the scan's own region, and that the serial run stayed on the caller in one range. +inline auto check_threaded_matches_serial(const Geometry &g, const Scenario &s, Expect expect) -> void { + using detail::KernelRange; + BOOST_TEST_CONTEXT(g.label << " / " << s.label) { + const size_t words = geometry_op(g, s.basis).inverted_index().words(); + const size_t blocks = detail::logical_ranges(words, detail::kColumnBlockWords); + const bool safe = detail::CutoffEvaluator(s.cutoff).parallel_safe(); + const size_t scanned = expect == Expect::not_scanned ? 0U : 1U; + + kernel_test::RangeLog serial_log; + const auto serial = run_scan(g, s, {.threads = 1}, kernel_test::RecordingObserver{&serial_log}); + kernel_test::check_serial(serial_log[KernelRange::scan], scanned, "serial scan"); + if (expect == Expect::output) { + BOOST_TEST_REQUIRE(emitted(serial.result) > 0U); + // Sources in at least two of the four budget-4 ranges, so the merge really joins pieces. + std::set ranges_hit; + const size_t k = std::min(4, blocks); + for (const auto *streams : {&serial.result.leader_src, &serial.result.follower_src}) { + for (const auto &v : *streams) { + for (const auto src : v) { + for (size_t r = 0; r < k; ++r) { + const auto [lo, hi] = detail::scan_range_words(r, k, words); + if (src >= lo * 64 && src < hi * 64) { + ranges_hit.insert(r); + } + } + } + } + } + BOOST_TEST(ranges_hit.size() >= 2U); + } + else { + BOOST_TEST(emitted(serial.result) == 0U); + } + for (const int threads : {2, 3, 4}) { + BOOST_TEST_CONTEXT("budget " << threads) { + kernel_test::RangeLog log; + const auto threaded = run_scan(g, s, {.threads = threads}, kernel_test::RecordingObserver{&log}); + check_same_scan(serial, threaded, s.capture); + const size_t expected = scanned == 0 ? 0 : (safe ? std::min(threads, blocks) : 1); + if (expected >= 2) { + kernel_test::check_participation(log[KernelRange::scan], expected, "threaded scan"); + } + else { + kernel_test::check_serial(log[KernelRange::scan], expected, "serial fallback scan"); + } + } + } + } +} + +// A generator of the requested pivot tier and parity whose rank shift under `router` is (non)zero, found by +// a deterministic search, so a scenario does not depend on the routing basis the build happens to draw. +inline auto find_generator(const routing::Router &router, + size_t pivot_lo, + size_t pivot_hi, + size_t weight, + bool want_zero_shift, + uint64_t seed) -> Mono { + kernel_test::SplitMix rng{seed}; + for (int attempt = 0; attempt < 100000; ++attempt) { + Mono g{}; + g.set(pivot_lo + rng.below(pivot_hi - pivot_lo)); + const size_t pivot = g.find_first(); + while (g.count() < weight) { + const size_t b = pivot + 1 + rng.below(64 - pivot - 1); + g.set(b); + } + if ((router.rank_shift(g) == 0) == want_zero_shift) { + return g; + } + } + BOOST_FAIL("no generator with the requested shift"); + return {}; +} + +} // namespace scan_test diff --git a/cpp/tests/evolution_detail_tests.cpp b/cpp/tests/evolution_detail_tests.cpp index a673b7ac..246c2a03 100644 --- a/cpp/tests/evolution_detail_tests.cpp +++ b/cpp/tests/evolution_detail_tests.cpp @@ -22,6 +22,7 @@ #include #include +#include "ScanTestSupport.h" #include "monoprop/TypeAliases.h" #include "monoprop/algebra/Algebra.h" #include "monoprop/detail/evolution/CutoffContext.h" @@ -223,3 +224,174 @@ BOOST_AUTO_TEST_CASE(cutoff_context_is_below_sin) { BOOST_TEST(ctx.is_below_sin(0.1)); BOOST_TEST(!ctx.is_below_sin(1.0)); } + +// --- threaded bitmap scan (fused_find_and_collect over logical word ranges) ------------------------------------ + +// Ranges are contiguous runs of whole fold blocks, nonempty, ascending with the range ID, covering every +// word, and depend only on the word count and the budget. +BOOST_AUTO_TEST_CASE(openmp_scan_ranges_partition_words_in_order) { + constexpr size_t kBlock = detail::kColumnBlockWords; + for (const size_t words : + {size_t{1}, kBlock - 1, kBlock, kBlock + 1, (3 * kBlock) + 1, 7 * kBlock, size_t{100003}}) { + for (const int threads : {1, 2, 3, 4, 7, 96}) { + const size_t ranges = detail::scan_ranges(words, {.threads = threads}); + BOOST_TEST_CONTEXT("words=" << words << " threads=" << threads) { + BOOST_TEST(ranges == std::min(threads, detail::logical_ranges(words, kBlock))); + size_t next = 0; + for (size_t r = 0; r < ranges; ++r) { + const auto [lo, hi] = detail::scan_range_words(r, ranges, words); + BOOST_TEST(lo == next); + BOOST_TEST(hi > lo); + BOOST_TEST((lo % kBlock) == 0U); + next = hi; + } + BOOST_TEST(next == words); + } + } + } +} + +// Majorana scans at budgets 2-4 against budget 1, byte for byte, across windows (one rank; linear routing +// with zero and non-zero shifts, including a two-slot window at a non-zero base; dense splitmix at four and +// three ranks), dense/sparse/empty pivots, odd generators with empty fold columns, length caps, structural +// cutoffs (including cutoff 0), lower/upper atol boundaries, scans that emit nothing, value capture and the +// fused cos sweep. only_rotate_len_k must be positive (0 is rejected before any scan). +BOOST_AUTO_TEST_CASE(openmp_scan_ranges_match_serial_majorana) { + using scan_test::Atol; + using scan_test::Expect; + using scan_test::Scenario; + const scan_test::Geometry solo{"one rank", 1, 1, false, 0}; + const auto dense_even = scan_test::mono_of_bits({14, 20, 33, 47}); + const auto sparse_even = scan_test::mono_of_bits({5, 40}); + const auto dense_odd = scan_test::mono_of_bits({13, 29, 52}); + const auto empty_odd = scan_test::mono_of_bits({0, 1, 2}); + const auto empty_even = scan_test::mono_of_bits({0, 3}); + const std::vector> single{ + {{.label = "dense pivot", .gen = dense_even}, Expect::output}, + {{.label = "sparse pivot", .gen = sparse_even}, Expect::output}, + {{.label = "odd generator", .gen = dense_odd}, Expect::output}, + {{.label = "odd generator, empty fold columns", .gen = empty_odd}, Expect::output}, + {{.label = "even generator, empty fold columns", .gen = empty_even}, Expect::not_scanned}, + {{.label = "lower atol above every term", .gen = dense_even, .atol = Atol::drop_all}, Expect::empty_scanned}, + {{.label = "lower atol above every term, fused", + .gen = dense_even, + .atol = Atol::drop_all, + .capture = true, + .fused_scale = true}, + Expect::empty_scanned}, + {{.label = "length cutoff 0", .gen = dense_even, .cutoff = detail::LengthCutoff<32>{0}}, Expect::empty_scanned}, + {{.label = "length cutoff 0, upper atol rescue", + .gen = dense_odd, + .cutoff = detail::LengthCutoff<32>{0}, + .atol = Atol::upper_equal, + .capture = true}, + Expect::output}, + {{.label = "cap 6", .gen = dense_even, .cap = 6}, Expect::output}, + {{.label = "cap 6, capture", .gen = dense_odd, .cap = 6, .capture = true}, Expect::output}, + {{.label = "length cutoff 5", .gen = dense_even, .cutoff = detail::LengthCutoff<32>{5}}, Expect::output}, + {{.label = "support cutoff 4", .gen = sparse_even, .cutoff = detail::SupportCutoff<32>{4}}, Expect::output}, + {{.label = "lower atol at a term", .gen = dense_even, .atol = Atol::lower_equal}, Expect::output}, + {{.label = "lower atol at a term, capture", .gen = dense_odd, .atol = Atol::lower_equal, .capture = true}, + Expect::output}, + {{.label = "upper atol rescue", + .gen = dense_even, + .cutoff = detail::LengthCutoff<32>{2}, + .atol = Atol::upper_equal}, + Expect::output}, + {{.label = "capture", .gen = sparse_even, .capture = true}, Expect::output}, + {{.label = "fused cos sweep", .gen = dense_even, .capture = true, .fused_scale = true}, Expect::output}, + {{.label = "fused cos sweep, odd, lower atol", + .gen = dense_odd, + .atol = Atol::lower_equal, + .capture = true, + .fused_scale = true}, + Expect::output}, + }; + for (const auto &[s, e] : single) { + scan_test::check_threaded_matches_serial(solo, s, e); + } + + const std::vector multi{ + {"linear R=4, zero shift", 4, 1, true, 2}, + {"linear R=4, non-zero shift", 4, 1, true, 0}, + {"linear R=2 S=2, non-zero shift", 2, 2, true, 1}, + {"linear R=2 S=2, zero shift", 2, 2, true, 1}, + {"splitmix R=4", 4, 1, false, 3}, + {"splitmix R=3", 3, 1, false, 1}, + }; + for (size_t gi = 0; gi < multi.size(); ++gi) { + const auto &g = multi[gi]; + const auto router = scan_test::router_of(g); + const bool want_zero = gi == 0 || gi == 3 || !g.linear; + const auto dense = g.linear ? scan_test::find_generator(router, 12, 30, 4, want_zero, 11 + gi) : dense_even; + const auto odd = g.linear ? scan_test::find_generator(router, 12, 30, 3, want_zero, 23 + gi) : dense_odd; + const auto sparse = g.linear ? scan_test::find_generator(router, 4, 12, 2, want_zero, 37 + gi) : sparse_even; + if (g.linear) { + BOOST_TEST_REQUIRE((router.rank_shift<32>(dense) == 0U) == want_zero); + } + const std::vector cases{ + {.label = "dense pivot", .gen = dense}, + {.label = "sparse pivot, capture", .gen = sparse, .capture = true}, + {.label = "odd generator, lower atol", .gen = odd, .atol = Atol::lower_equal}, + {.label = "odd generator, empty fold columns", .gen = empty_odd}, + {.label = "fused cos sweep", .gen = dense, .capture = true, .fused_scale = true}, + {.label = "cap 6, capture", .gen = odd, .cap = 6, .capture = true}, + }; + for (const auto &s : cases) { + scan_test::check_threaded_matches_serial(g, s, Expect::output); + } + } +} + +// A threaded scan as the first user of a fresh operator: the inverted index and the odd-generator row-parity +// cache are both cold, so building either lazily inside a worker would race (and TSan reports it). The scan +// must prepare both on the caller and still match a serial scan of an identical operator. +BOOST_AUTO_TEST_CASE(openmp_scan_prepares_cold_caches_on_the_caller) { + const scan_test::Geometry solo{"one rank", 1, 1, false, 0}; + for (const auto &gen : {scan_test::mono_of_bits({13, 29, 52}), scan_test::mono_of_bits({0, 1, 2})}) { + const scan_test::Scenario s{.label = "odd, cold", .gen = gen, .capture = true}; + const auto threaded_op = scan_test::cold_op(solo, Basis::Majorana); + kernel_test::RangeLog log; + const auto threaded = + scan_test::run_scan_on(threaded_op, solo, s, {.threads = 4}, kernel_test::RecordingObserver{&log}); + BOOST_TEST(log[detail::KernelRange::scan].worker.size() == 4U); + const auto serial_op = scan_test::cold_op(solo, Basis::Majorana); + const auto serial = scan_test::run_scan_on(serial_op, solo, s, {.threads = 1}); + scan_test::check_same_scan(serial, threaded, true); + } +} + +// Worker participation in the scan's own range bodies, at the budget of the ranges' count. +BOOST_AUTO_TEST_CASE(openmp_scan_workers_participate, + *boost::unit_test::precondition(kernel_test::runtime_offers_two_workers)) { + const scan_test::Geometry solo{"one rank", 1, 1, false, 0}; + const scan_test::Scenario s{.label = "dense pivot", .gen = scan_test::mono_of_bits({14, 20, 33, 47})}; + kernel_test::RangeLog log; + (void)scan_test::run_scan(solo, s, {.threads = 4}, kernel_test::RecordingObserver{&log}); + kernel_test::check_participation(log[detail::KernelRange::scan], 4, "scan"); +} + +// A worker failure inside one scan range is joined, then rethrown on the caller; the other ranges are not +// merged into anything the caller can observe. +BOOST_AUTO_TEST_CASE(openmp_scan_worker_failure_joins_before_rethrow) { + const scan_test::Geometry solo{"one rank", 1, 1, false, 0}; + for (const bool fused : {false, true}) { + const scan_test::Scenario s{.label = "failure", + .gen = scan_test::mono_of_bits({14, 20, 33, 47}), + .capture = fused, + .fused_scale = fused}; + for (const size_t failing : {size_t{0}, size_t{2}}) { + kernel_test::RangeLog log; + BOOST_CHECK_THROW( + (void)scan_test::run_scan(solo, + s, + {.threads = 3}, + kernel_test::RecordingObserver{&log, detail::KernelRange::scan, failing}), + std::runtime_error); + const auto &slots = log[detail::KernelRange::scan]; + BOOST_TEST(slots.prepares == 1U); + BOOST_TEST(slots.visits.size() == 3U); + BOOST_TEST(slots.visits[failing] == 1); + } + } +} diff --git a/cpp/tests/exact_upper_atol_rescue.cpp b/cpp/tests/exact_upper_atol_rescue.cpp index 3243d965..574c3bb6 100644 --- a/cpp/tests/exact_upper_atol_rescue.cpp +++ b/cpp/tests/exact_upper_atol_rescue.cpp @@ -18,6 +18,7 @@ #include #include +#include "ScanTestSupport.h" #include "TestUtilities.h" // upper_atol rescue: a structural cutoff of 0 rejects every non-identity partner, and upper_atol = 0 @@ -77,3 +78,37 @@ MAKE_ZERO_CUTOFF_RESCUE_TEST(zero_cutoff_upper_atol_zero_is_exact_lih, LihFixtur MAKE_ZERO_CUTOFF_RESCUE_TEST(zero_cutoff_upper_atol_zero_is_exact_lih, LihFixture, World) #undef MAKE_ZERO_CUTOFF_RESCUE_TEST + +// The atol boundaries decide per term from the pre-scale coefficient in whichever range owns it: a lower atol +// equal to a term's |sin|·|c| drops it, an upper atol equal to it rescues a partner the cutoff rejects. The +// threaded scan must make exactly the serial decisions, with and without value capture and the fused sweep. +BOOST_AUTO_TEST_CASE(openmp_scan_atol_boundaries_match_serial) { + using scan_test::Atol; + using scan_test::Expect; + const auto gen = scan_test::mono_of_bits({14, 20, 33, 47}); + const auto tight = monoprop::detail::LengthCutoff<32>{2}; + for (const scan_test::Geometry g : + {scan_test::Geometry{"one rank", 1, 1, false, 0}, scan_test::Geometry{"splitmix R=4", 4, 1, false, 1}}) { + for (const bool capture : {false, true}) { + scan_test::check_threaded_matches_serial( + g, + {.label = "lower atol at a term", .gen = gen, .atol = Atol::lower_equal, .capture = capture}, + Expect::output); + scan_test::check_threaded_matches_serial(g, + {.label = "upper atol rescue", + .gen = gen, + .cutoff = tight, + .atol = Atol::upper_equal, + .capture = capture}, + Expect::output); + } + scan_test::check_threaded_matches_serial(g, + {.label = "upper atol rescue, fused cos sweep", + .gen = gen, + .cutoff = tight, + .atol = Atol::upper_equal, + .capture = true, + .fused_scale = true}, + Expect::output); + } +} diff --git a/cpp/tests/fused_cos_sweep_tests.cpp b/cpp/tests/fused_cos_sweep_tests.cpp index fd982a79..360ee548 100644 --- a/cpp/tests/fused_cos_sweep_tests.cpp +++ b/cpp/tests/fused_cos_sweep_tests.cpp @@ -15,6 +15,7 @@ #include #include +#include #include #include #include @@ -25,6 +26,7 @@ #include "KernelTestSupport.h" #include "PropagatorTestAccess.h" +#include "ScanTestSupport.h" #include "TestUtilities.h" #include "monoprop/MPFunctions.h" #include "monoprop/detail/evolution/layer_build/FusedApply.h" @@ -369,3 +371,236 @@ BOOST_AUTO_TEST_CASE(fused_threaded_prototype_matches_serial_prototype_exactly) } } } + +// --- threaded bitmap scan: fused sweep and full construction ------------------------------------------------- + +// The fused cos sweep scales each anticommuting source exactly once, in whichever range owns it, after the +// scan captured its pre-scale value; every other coefficient stays untouched. Budgets 2-4 against the +// independently derived expectation, not only against budget 1. +BOOST_AUTO_TEST_CASE(openmp_scan_fused_sweep_scales_each_source_once) { + const scan_test::Geometry solo{"one rank", 1, 1, false, 0}; + for (const auto &gen : {scan_test::mono_of_bits({14, 20, 33, 47}), scan_test::mono_of_bits({13, 29, 52})}) { + const scan_test::Scenario plain{.label = "capture", .gen = gen, .capture = true}; + // Without a cap, the non-fused scan's cosine set is every anticommuting source. + const auto reference = scan_test::run_scan(solo, plain, {.threads = 1}); + const auto anti = scan_test::concatenated_cos(reference.result); + BOOST_TEST_REQUIRE(anti.total_count > 0U); + const double cos_val = std::cos(2 * plain.param); + VecD expect = reference.coeffs; + for (const auto &[base, bits] : anti.blocks) { + for (uint64_t m = bits; m != 0; m &= m - 1) { + const size_t i = base + static_cast(std::countr_zero(m)); + expect[i] = reference.coeffs[i] * cos_val; + } + } + scan_test::Scenario fused = plain; + fused.fused_scale = true; + for (const int threads : {1, 2, 3, 4}) { + BOOST_TEST_CONTEXT("budget " << threads) { + const auto out = scan_test::run_scan(solo, fused, {.threads = threads}); + BOOST_TEST(scan_test::bitwise_equal(out.coeffs, expect)); + BOOST_TEST(out.result.cos_blocks.size() >= 1U); + BOOST_TEST(scan_test::concatenated_cos(out.result).total_count == 0U); // the sweep replaces the set + // Captured values are the pre-scale coefficients, and the same queries are emitted. + const auto &a = reference.result.leader_val.at_slot(0); + const auto &b = out.result.leader_val.at_slot(0); + BOOST_TEST(scan_test::bitwise_equal(a, b)); + BOOST_TEST((reference.result.leader_src.at_slot(0) == out.result.leader_src.at_slot(0))); + BOOST_TEST((reference.result.follower_src.at_slot(0) == out.result.follower_src.at_slot(0))); + } + } + } +} + +namespace { + +constexpr size_t kWide = 16; // 32 Majoranas +using WideAccess = monoprop::detail::PropagatorTestAccess; + +// About 90k even-weight terms, so the scan's operator spans at least two fold blocks from the first gate and +// more as gates insert partners; gates of weights 2-4 with two repeats, and one at theta = pi/4. +auto wide_case() -> CaseData { + CaseData data; + data.num_modes = kWide; + kernel_test::SplitMix rng{0x5CA2ULL}; + const auto draw_indices = [&rng](size_t weight) { + VecZ idx; + while (idx.size() < weight) { + const size_t m = rng.below(2 * kWide); + if (std::ranges::find(idx, m) == idx.end()) { + idx.push_back(m); + } + } + std::ranges::sort(idx); + return idx; + }; + while (data.hamiltonian.size() < 90000) { + const auto idx = draw_indices(2 * (1 + rng.below(3))); + const double x = static_cast(rng.below(1U << 20)) / (1U << 20) - 0.5; + data.hamiltonian[idx] = x * hermitian_coefficient(indices_to_bitset(idx)); + } + data.initial_state = {0, 2, 4, 6, 8, 10, 12, 14}; + constexpr size_t gates = 6; + for (size_t g = 0; g < gates; ++g) { + data.majoranas.push_back(draw_indices(2 + rng.below(3))); + data.param_inds.push_back(g); + data.gen_coeffs.push_back(1.0); + data.parameters.push_back(g == 2 ? std::numbers::pi / 4 : 0.1 + static_cast(rng.below(900)) / 1000.0); + } + data.majoranas[4] = data.majoranas[1]; + return data; +} + +auto wide_prototype(const CaseData &data, bool schrodinger, int budget, MPI_Comm comm) -> MonomialPropagator { + // The Schrodinger state starts from the whole paired basis, 2^16 rows (one fold block), and grows past it. + const SimulatorConfig cfg{.schrodinger_cutoff = schrodinger ? std::optional(2 * kWide) : std::nullopt, + .comm = comm, + .atol = 1e-12}; + const kernel_test::ScopedBudget scoped(budget); + MonomialPropagator sim(data.hamiltonian, + /*cutoff=*/10, + data.initial_state, + cfg.schrodinger_cutoff, + cfg.comm, + cfg.atol, + cfg.upper_atol, + cfg.cutoff_type, + cfg.basis_change, + kWide, + Basis::Majorana, + /*partitions=*/1); + BOOST_TEST_REQUIRE(WideAccess::options(sim).threads == budget); + return sim; +} + +// Rows in row-ID order: the store's IDs are its insertion order, so equal rows mean equal IDs. +auto same_rows(const MonomialPropagator &a, const MonomialPropagator &b) -> bool { + const auto &sa = *a.mp_op().store; + const auto &sb = *b.mp_op().store; + if (sa.size() != sb.size()) { + return false; + } + for (size_t i = 0; i < sa.size(); ++i) { + if (materialize_row(sa, i) != materialize_row(sb, i)) { + return false; + } + } + return true; +} + +auto same_core(const LayerCore &a, const LayerCore &b) -> bool { + const auto &x = a.cross_rank; + const auto &y = b.cross_rank; + if (x.occupied.size() != y.occupied.size()) { + return false; + } + for (size_t k = 0; k < x.occupied.size(); ++k) { + if (x.occupied[k].slot != y.occupied[k].slot || x.occupied[k].sin_send_count != y.occupied[k].sin_send_count + || x.occupied[k].in_count != y.occupied[k].in_count) { + return false; + } + } + return x.sin_send_indices == y.sin_send_indices + && x.sin_recv_phases.uses_binary_phases == y.sin_recv_phases.uses_binary_phases + && x.sin_recv_phases.total_count == y.sin_recv_phases.total_count + && x.sin_recv_phases.phase_words == y.sin_recv_phases.phase_words + && x.sin_recv_phases.phase_values == y.sin_recv_phases.phase_values && x.world_size == y.world_size + && x.self_pos == y.self_pos && x.self_offset == y.self_offset && a.generator_words == b.generator_words + && a.scaled_count == b.scaled_count && a.param_index == b.param_index + && std::bit_cast(a.gen_coeff) == std::bit_cast(b.gen_coeff) + && a.gate_index == b.gate_index; +} + +auto same_graph(const MPGraph &a, const MPGraph &b) -> bool { + if (a.layers() != b.layers()) { + return false; + } + for (size_t l = 0; l < a.layers(); ++l) { + const auto &la = a.get_layer(l); + const auto &lb = b.get_layer(l); + if (!same_core(la.core(), lb.core()) || (la.pruned_cos() == nullptr) != (lb.pruned_cos() == nullptr)) { + return false; + } + } + return true; +} + +} // namespace + +// Full construction at fixed geometry: prototypes at budgets 2-4 build the same graph (every layer's +// endpoints, phases, slots, generator and scaled_count), the same rows in the same row-ID order and the same +// coefficients as budget 1, and propagate (fused sweep and the two-pass length-cap path) to the same rows +// and coefficients, in both pictures. Under an MPI launch each rank compares its own store and graph. +BOOST_AUTO_TEST_CASE(openmp_scan_threaded_construction_matches_serial_exactly) { + const auto data = wide_case(); + for (const bool schrodinger : {false, true}) { + auto serial_graph = wide_prototype(data, schrodinger, 1, MPI_COMM_WORLD); + serial_graph.build_graph(data.majoranas, data.param_inds, data.gen_coeffs); + BOOST_TEST_MESSAGE("schrodinger=" << schrodinger << " graph-built rows on this rank: " << serial_graph.size() + << ", layers: " << serial_graph.graph().layers()); + std::vector> serial_prop; + for (const bool cap : {false, true}) { + serial_prop.push_back(wide_prototype(data, schrodinger, 1, MPI_COMM_WORLD)); + serial_prop.back().propagate(data.majoranas, + data.param_inds, + data.gen_coeffs, + data.parameters, + cap ? std::optional(2 * kWide) : std::nullopt); + } + for (const int threads : {2, 3, 4}) { + BOOST_TEST_CONTEXT("schrodinger=" << schrodinger << " budget=" << threads) { + auto threaded = wide_prototype(data, schrodinger, threads, MPI_COMM_WORLD); + threaded.build_graph(data.majoranas, data.param_inds, data.gen_coeffs); + BOOST_TEST(same_rows(serial_graph, threaded)); + BOOST_TEST(same_graph(serial_graph.graph(), threaded.graph())); + BOOST_TEST( + bitwise_equal(WideAccess::picture_coeffs(serial_graph), WideAccess::picture_coeffs(threaded))); + BOOST_TEST(bitwise_equal(serial_graph.contract_partially(data.parameters, false), + threaded.contract_partially(data.parameters, false))); + for (const bool cap : {false, true}) { + auto prop = wide_prototype(data, schrodinger, threads, MPI_COMM_WORLD); + prop.propagate(data.majoranas, + data.param_inds, + data.gen_coeffs, + data.parameters, + cap ? std::optional(2 * kWide) : std::nullopt); + const auto &ref = serial_prop[cap ? 1 : 0]; + BOOST_TEST(same_rows(ref, prop)); + BOOST_TEST(bitwise_equal(WideAccess::picture_coeffs(ref), WideAccess::picture_coeffs(prop))); + } + } + } + } +} + +// The real build_layer on a prototype's operator splits the scan into ranges that run inside its own region. +BOOST_AUTO_TEST_CASE(openmp_scan_construction_workers_participate, + *boost::unit_test::precondition(kernel_test::runtime_offers_two_workers)) { + const auto data = wide_case(); + auto sim = wide_prototype(data, false, 4, MPI_COMM_SELF); + const size_t words = sim.mp_op().inverted_index().words(); + const size_t expected = + std::min(4, monoprop::detail::logical_ranges(words, monoprop::detail::kColumnBlockWords)); + BOOST_TEST_REQUIRE(expected >= 2U); + kernel_test::RangeLog log; + (void)WideAccess::build_layer_observed(sim, data.majoranas[0], kernel_test::RecordingObserver{&log}); + kernel_test::check_participation(log[monoprop::detail::KernelRange::scan], expected, "build_layer scan"); + BOOST_TEST(!WideAccess::is_invalid(sim)); +} + +// A scan worker's failure inside the real build path is joined and rethrown on the caller; the propagator's +// operation guard then invalidates the object, and later operations and copies reject it. +BOOST_AUTO_TEST_CASE(openmp_scan_worker_failure_invalidates_the_propagator) { + const auto data = wide_case(); + auto sim = wide_prototype(data, false, 3, MPI_COMM_SELF); + kernel_test::RangeLog log; + BOOST_CHECK_THROW((void)WideAccess::build_layer_observed( + sim, + data.majoranas[0], + kernel_test::RecordingObserver{&log, monoprop::detail::KernelRange::scan, 1}), + std::runtime_error); + BOOST_TEST(log[monoprop::detail::KernelRange::scan].visits.at(1) == 1); + BOOST_TEST(WideAccess::is_invalid(sim)); + BOOST_CHECK_THROW(sim.build_graph(data.majoranas, data.param_inds, data.gen_coeffs), InvalidPropagatorError); + BOOST_CHECK_THROW((void)WideAccess::clone(sim), InvalidPropagatorError); +} diff --git a/cpp/tests/majorana_cutoff_tests.cpp b/cpp/tests/majorana_cutoff_tests.cpp index 60c344ec..dca8740d 100644 --- a/cpp/tests/majorana_cutoff_tests.cpp +++ b/cpp/tests/majorana_cutoff_tests.cpp @@ -26,9 +26,11 @@ #include #include +#include "ScanTestSupport.h" #include "monoprop/TypeAliases.h" #include "monoprop/algebra/MajoranaAlgebra.h" #include "monoprop/core/Monomial.h" +#include "monoprop/detail/monomial_propagator/MonomialPropagatorCommon.h" using namespace monoprop; using cd = std::complex; @@ -192,6 +194,12 @@ BOOST_AUTO_TEST_CASE(majorana_cutoff_evaluator_dispatch_and_popcount) { BOOST_TEST((opaque_ev.support_cutoff() == nullptr)); BOOST_TEST(!opaque_ev.max_slot_bound().has_value()); + // Only the typed built-in cutoffs may run in scan workers. A stateless lambda is still opaque: the + // evaluator cannot see what it does, and constness of the std::function says nothing about it. + BOOST_TEST(length_ev.parallel_safe()); + BOOST_TEST(support_ev.parallel_safe()); + BOOST_TEST(!opaque_ev.parallel_safe()); + // passes_with_popcount: pc <= cutoff short-circuits to true; otherwise it equals a direct eval. Monomial unpaired; // length 4, not paired unpaired.set(0); @@ -342,3 +350,40 @@ BOOST_AUTO_TEST_CASE(majorana_cutoff_generate_paired_op_matches_the_enumeration) BOOST_TEST(listed[i] == seen[i]); } } + +// An opaque cutoff keeps the scan on the calling thread at any budget, in one range, with the same result; +// so does the basis-change cutoff, whose closure is conservatively treated as opaque. +BOOST_AUTO_TEST_CASE(openmp_scan_opaque_cutoff_stays_serial) { + using scan_test::Expect; + const scan_test::Geometry solo{"one rank", 1, 1, false, 0}; + const auto gen = scan_test::mono_of_bits({14, 20, 33, 47}); + const CutoffFn<32> opaque = [](const Monomial<32> &m) { return m.count() <= 6; }; + BOOST_TEST(!detail::CutoffEvaluator<32>(opaque).parallel_safe()); + scan_test::check_threaded_matches_serial(solo, + {.label = "opaque lambda", .gen = gen, .cutoff = opaque}, + Expect::output); + scan_test::check_threaded_matches_serial( + solo, + {.label = "opaque lambda, fused", .gen = gen, .cutoff = opaque, .capture = true, .fused_scale = true}, + Expect::output); + + // Identity basis change: the predicate is a real basis-change closure, and its decisions equal the plain + // length cutoff's, so the serial fallback must also reproduce the typed, threaded scan. + MonomialList<32> identity; + for (size_t b = 0; b < 64; ++b) { + Monomial<32> row{}; + row.set(63 - b); // change_basis reads row 2N-1-pos for bit pos + identity.push_back(row); + } + const auto changed = detail::cutoff_function_basis_change<32>(CutoffType::Length, 6, identity); + BOOST_TEST(!detail::CutoffEvaluator<32>(changed).parallel_safe()); + scan_test::check_threaded_matches_serial(solo, + {.label = "basis change", .gen = gen, .cutoff = changed}, + Expect::output); + const auto typed = scan_test::run_scan(solo, + {.label = "typed", .gen = gen, .cutoff = detail::LengthCutoff<32>{6}}, + {.threads = 4}); + const auto via_basis = + scan_test::run_scan(solo, {.label = "basis change", .gen = gen, .cutoff = changed}, {.threads = 4}); + scan_test::check_same_scan(typed, via_basis, false); +} diff --git a/cpp/tests/mpi_failure_driver.cpp b/cpp/tests/mpi_failure_driver.cpp index aa869ce5..1975d6a5 100644 --- a/cpp/tests/mpi_failure_driver.cpp +++ b/cpp/tests/mpi_failure_driver.cpp @@ -23,6 +23,8 @@ // // worker-throw worksharing worker throws inside a retained-functional evaluation // before-exchange a build fails in the scan, before the first query exchange +// before-exchange scan-worker a worker of the threaded scan throws in one of its word ranges, before +// the first query exchange the peers enter // active-ticket ticket|pending a failure while a replay Ticket / construction PendingAlltoallv is posted // active-ticket cosine-worker a worker of the threaded cosine kernel throws while the replay Ticket is // posted @@ -32,7 +34,9 @@ #include #include +#include #include +#include #include #include #include @@ -142,6 +146,60 @@ auto before_exchange(int rank) -> void { } } +// Throws from one word range of the real threaded scan, on whichever worker owns it. +struct ThrowingScanObserver { + int rank; + auto prepare(detail::KernelRange, size_t) const noexcept -> void {} + auto visit(detail::KernelRange kind, size_t range) const -> void { + if (rank == 0 && kind == detail::KernelRange::scan && range == 1) { + throw std::runtime_error( + std::format("injected failure in scan range {} on OpenMP worker {}", range, omp_get_thread_num())); + } + } +}; + +// Enough terms that each rank's store spans two fold blocks, so the scan really opens a two-worker team. +constexpr size_t kWideModes = 16; + +auto scan_worker(int rank) -> void { + OperatorDict ham; + uint64_t state = 0x5CA3ULL; + while (ham.size() < 200000) { + VecZ idx; + while (idx.size() < 6) { // weight 6: C(32, 6) ~ 9e5 distinct terms, and an imaginary Hermitian coefficient + state = (state * 6364136223846793005ULL) + 1442695040888963407ULL; + const size_t m = (state >> 33) % (2 * kWideModes); + if (std::ranges::find(idx, m) == idx.end()) { + idx.push_back(m); + } + } + std::ranges::sort(idx); + ham[idx] = std::complex{0.0, 1e-3}; + } + ::setenv("monoprop_NUM_THREADS", "2", 1); + MonomialPropagator sim(ham, + 2 * kWideModes, + VecZ{0, 1}, + std::nullopt, + MPI_COMM_WORLD, + std::nullopt, + std::nullopt, + CutoffType::Length, + std::nullopt, + kWideModes, + Basis::Majorana, + /*partitions=*/1); + say(rank, + std::format("{} local terms, {} words, budget {}", + sim.size(), + sim.mp_op().inverted_index().words(), + detail::PropagatorTestAccess::options(sim).threads)); + (void)detail::PropagatorTestAccess::build_layer_observed(sim, VecZ{0, 5}, ThrowingScanObserver{rank}); + if (rank == 0) { + say(rank, "INJECTION NOT REACHED: the scan ran in fewer than two ranges"); + } +} + auto active_ticket(int rank) -> void { auto sim = make_prototype(); build(sim); @@ -295,6 +353,9 @@ auto main(int argc, char **argv) -> int { if (scenario == "worker-throw") { worker_throw(rank); } + else if (scenario == "before-exchange" && option == "scan-worker") { + scan_worker(rank); + } else if (scenario == "before-exchange") { before_exchange(rank); } diff --git a/cpp/tests/pauli_build_layer_tests.cpp b/cpp/tests/pauli_build_layer_tests.cpp index af069fcd..5f857506 100644 --- a/cpp/tests/pauli_build_layer_tests.cpp +++ b/cpp/tests/pauli_build_layer_tests.cpp @@ -26,6 +26,7 @@ #include #include "PauliTestOracle.h" +#include "ScanTestSupport.h" #include "monoprop/MonomialPropagator.h" #include "monoprop/algebra/MajoranaAlgebra.h" #include "monoprop/algebra/PauliAlgebra.h" @@ -452,3 +453,56 @@ BOOST_AUTO_TEST_CASE(pauli_build_layer_replay_fold_consumers) { BOOST_TEST(e_contract == e_graph, boost::test_tools::tolerance(1e-9)); BOOST_TEST(e_jw == e_graph, boost::test_tools::tolerance(1e-9)); } + +// Pauli scans fold J(G) = pair_swap(G) but split on the untransformed G's lowest bit. Threaded ranges at +// budgets 2-4 must reproduce budget 1 exactly, including a J(G) that lands on empty columns while the +// pivot does not, and dense/sparse splitmix and linear windows. +BOOST_AUTO_TEST_CASE(openmp_scan_pauli_ranges_match_serial) { + using scan_test::Expect; + using scan_test::Scenario; + const auto pauli = Basis::Pauli; + // {1, 20}: J(G) = {0, 21}, so the fold reads empty column 0 while the pivot is bit 1 (also empty). + // {12, 3}: pivot 3 is empty but J(G) = {13, 2} reads dense column 13 -- the pivot is not a fold column. + const std::vector> single{ + {{.label = "dense J(G)", .basis = pauli, .gen = scan_test::mono_of_bits({14, 21, 33, 46})}, Expect::output}, + {{.label = "odd-weight G", .basis = pauli, .gen = scan_test::mono_of_bits({13, 28, 51})}, Expect::output}, + {{.label = "sparse pivot", .basis = pauli, .gen = scan_test::mono_of_bits({6, 40})}, Expect::output}, + {{.label = "J(G) on an empty column", .basis = pauli, .gen = scan_test::mono_of_bits({1, 20})}, Expect::output}, + {{.label = "pivot outside J(G)", .basis = pauli, .gen = scan_test::mono_of_bits({3, 12})}, Expect::output}, + {{.label = "empty fold columns", .basis = pauli, .gen = scan_test::mono_of_bits({0, 1})}, Expect::not_scanned}, + {{.label = "capture, cap 5", + .basis = pauli, + .gen = scan_test::mono_of_bits({14, 21}), + .cap = 5, + .capture = true}, + Expect::output}, + {{.label = "fused cos sweep", + .basis = pauli, + .gen = scan_test::mono_of_bits({13, 28, 51}), + .capture = true, + .fused_scale = true}, + Expect::output}, + {{.label = "support cutoff, lower atol", + .basis = pauli, + .gen = scan_test::mono_of_bits({14, 21, 33, 46}), + .cutoff = detail::SupportCutoff<32>{4}, + .atol = scan_test::Atol::lower_equal}, + Expect::output}, + }; + const scan_test::Geometry solo{"one rank", 1, 1, false, 0}; + for (const auto &[s, e] : single) { + scan_test::check_threaded_matches_serial(solo, s, e); + } + for (const scan_test::Geometry g : + {scan_test::Geometry{"linear R=4", 4, 1, true, 1}, scan_test::Geometry{"splitmix R=3", 3, 1, false, 2}}) { + const auto router = scan_test::router_of(g); + const auto gen = g.linear ? scan_test::find_generator(router, 12, 30, 3, false, 99) : single[1].first.gen; + scan_test::check_threaded_matches_serial(g, + {.label = "odd-weight G", .basis = pauli, .gen = gen}, + Expect::output); + scan_test::check_threaded_matches_serial( + g, + {.label = "fused cos sweep", .basis = pauli, .gen = gen, .capture = true, .fused_scale = true}, + Expect::output); + } +} diff --git a/cpp/tests/sparse_query_tests.cpp b/cpp/tests/sparse_query_tests.cpp index 0843d2e5..33ac4b10 100644 --- a/cpp/tests/sparse_query_tests.cpp +++ b/cpp/tests/sparse_query_tests.cpp @@ -17,13 +17,17 @@ #include #include +#include +#include #include #include +#include #include #include #include "monoprop/detail/evolution/layer_build/Common.h" #include "monoprop/detail/evolution/layer_build/QueryWire.h" +#include "monoprop/detail/evolution/layer_build/Scan.h" #include "dense_query_reference.h" @@ -571,3 +575,192 @@ BOOST_AUTO_TEST_CASE(sparse_record_round_trips_every_reachable_k_and_gap_width) BOOST_TEST(differential<128>(pos, 1) == 1U); } } + +// --- ordered merge of per-range scan results ------------------------------------------------------------------ + +namespace { + +constexpr size_t kMergeModes = 32; +using MergeResult = FusedScanResult; + +auto reset_streams(MergeResult &r, mpi::SlotWindow w, bool capture) -> void { + r.leader_queries.reset(w); + r.leader_src.reset(w); + r.follower_queries.reset(w); + r.follower_src.reset(w); + if (capture) { + r.leader_val.reset(w); + r.follower_val.reset(w); + } +} + +auto same_stage(const SelfQueryStage &a, const SelfQueryStage &b) -> bool { + if (a.size() != b.size() || a.positions() != b.positions()) { + return false; + } + for (size_t q = 0; q < a.size(); ++q) { + if (a.pos_off[q] != b.pos_off[q] || a.k_of[q] != b.k_of[q] || a.phase_of[q] != b.phase_of[q]) { + return false; + } + } + return std::equal(a.pos_flat.begin(), + a.pos_flat.begin() + static_cast(a.positions()), + b.pos_flat.begin()); +} + +auto bits_equal(const std::vector &a, const std::vector &b) -> bool { + return a.size() == b.size() && (a.empty() || std::memcmp(a.data(), b.data(), a.size() * sizeof(double)) == 0); +} + +} // namespace + +// Hand-built per-range pieces with variable-length wire records (escaped k included), a window at a +// non-zero base, self-owned queries on both stages, an empty middle piece and value capture on and off. +// The merge must reproduce the single stream a serial scan would have produced: each destination's +// records, sources and values in range order, each self stage rebased in range order, cosine sets in +// range order, and every piece's buffers released afterwards. +BOOST_AUTO_TEST_CASE(openmp_scan_merge_keeps_source_order_windows_and_self_stages) { + using QW = QueryWire; + const mpi::SlotWindow window{.base = 2, .count = 3}; + const size_t me = 3; + for (const bool capture : {false, true}) { + std::mt19937_64 rng(0x3E26EULL); + std::vector pieces(4); + MergeResult expect; + reset_streams(expect, window, capture); + size_t src = 0; + for (size_t p = 0; p < pieces.size(); ++p) { + auto &piece = pieces[p]; + reset_streams(piece, window, capture); + CosineWordBuilder cos; + if (p == 1) { + piece.cos_blocks.push_back(cos.finish()); // an empty range still reports its (empty) set + continue; + } + for (size_t q = 0; q < 40 + (p * 7); ++q) { + const size_t dest = window.slot(mpi::WindowIndex{rng() % window.count}); + const bool follower = (rng() & 1U) != 0; + const std::array widths{0, 1, 5, 31, 40, 64}; + const size_t k = widths[rng() % widths.size()]; + std::vector pos(64); + std::iota(pos.begin(), pos.end(), uint16_t{0}); + std::shuffle(pos.begin(), pos.end(), rng); + pos.resize(k); + std::ranges::sort(pos); + const int phase = static_cast(rng() % 3) - 1; + const double v = std::ldexp(static_cast(rng() % 1000) - 500.0, -7); + for (auto *r : {&piece, &expect}) { + if (dest == me) { + (follower ? r->follower_self : r->leader_self).push(pos, phase); + } + else { + QW::push((follower ? r->follower_queries : r->leader_queries).at_slot(dest), pos, phase); + } + (follower ? r->follower_src : r->leader_src).at_slot(dest).push_back(src); + if (capture) { + (follower ? r->follower_val : r->leader_val).at_slot(dest).push_back(v); + } + } + cos.push_index((p * 1024 * 64) + (q * 3)); + ++src; + } + piece.cos_blocks.push_back(cos.finish()); + } + std::vector expect_cos; + for (const auto &piece : pieces) { + expect_cos.push_back(piece.cos_blocks.front()); + } + + const auto merged = merge_scan_ranges(pieces, window, me, capture); + + BOOST_TEST_CONTEXT("capture=" << capture) { + BOOST_TEST((merged.leader_queries.window() == window)); + BOOST_TEST((merged.follower_src.window() == window)); + BOOST_TEST(merged.leader_val.size() == (capture ? window.count : 0U)); + BOOST_TEST(merged.follower_val.size() == (capture ? window.count : 0U)); + for (const auto wi : window.indices()) { + BOOST_TEST((merged.leader_queries[wi] == expect.leader_queries[wi])); + BOOST_TEST((merged.follower_queries[wi] == expect.follower_queries[wi])); + BOOST_TEST((merged.leader_src[wi] == expect.leader_src[wi])); + BOOST_TEST((merged.follower_src[wi] == expect.follower_src[wi])); + if (capture) { + BOOST_TEST(bits_equal(merged.leader_val[wi], expect.leader_val[wi])); + BOOST_TEST(bits_equal(merged.follower_val[wi], expect.follower_val[wi])); + } + } + // Self wire stays empty; the stages carry the self slot's queries in order. + BOOST_TEST(merged.leader_queries.at_slot(me).empty()); + BOOST_TEST(merged.follower_queries.at_slot(me).empty()); + BOOST_TEST(expect.leader_self.size() > 0U); + BOOST_TEST(expect.follower_self.size() > 0U); + BOOST_TEST(same_stage(merged.leader_self, expect.leader_self)); + BOOST_TEST(same_stage(merged.follower_self, expect.follower_self)); + BOOST_TEST(merged.leader_self.size() == merged.leader_src.at_slot(me).size()); + BOOST_TEST(merged.follower_self.size() == merged.follower_src.at_slot(me).size()); + BOOST_TEST_REQUIRE(merged.cos_blocks.size() == expect_cos.size()); + for (size_t p = 0; p < expect_cos.size(); ++p) { + BOOST_TEST((merged.cos_blocks[p].blocks == expect_cos[p].blocks)); + BOOST_TEST(merged.cos_blocks[p].total_count == expect_cos[p].total_count); + } + // Drained pieces hold no allocation: clear() alone would have kept their capacity. + size_t retained = 0; + for (const auto &piece : pieces) { + for (const auto wi : window.indices()) { + retained += piece.leader_queries[wi].capacity() + piece.follower_queries[wi].capacity(); + retained += piece.leader_src[wi].capacity() + piece.follower_src[wi].capacity(); + if (capture) { + retained += piece.leader_val[wi].capacity() + piece.follower_val[wi].capacity(); + } + } + for (const auto *stage : {&piece.leader_self, &piece.follower_self}) { + retained += stage->pos_flat.capacity() + stage->pos_off.capacity() + stage->k_of.capacity() + + stage->phase_of.capacity(); + } + retained += piece.cos_blocks.capacity(); + } + BOOST_TEST(retained == 0U); + } + } +} + +// The merge moves a buffer instead of copying it when one piece holds a whole stream, or when the first +// nonempty piece already has room for the total; otherwise it reserves the exact total once. +BOOST_AUTO_TEST_CASE(openmp_scan_merge_moves_the_first_useful_buffer) { + const mpi::SlotWindow window{.base = 0, .count = 2}; + const size_t me = 0; + std::vector pieces(3); + for (auto &piece : pieces) { + reset_streams(piece, window, false); + } + // Slot 0 sources: only piece 1 has any, so the merged stream is that very buffer. + auto &only = pieces[1].leader_src.at_slot(0); + only.reserve(100); + only.assign({4, 5, 6}); + const auto *only_data = only.data(); + // Slot 1 sources: piece 0 has room for all of them, so its buffer is kept and piece 2 appended. + auto &roomy = pieces[0].follower_src.at_slot(1); + roomy.reserve(64); + roomy.assign({1, 2}); + const auto *roomy_data = roomy.data(); + pieces[2].follower_src.at_slot(1).assign({9}); + // Slot 1 leader sources: no piece has room, so the result is one exact reservation. + pieces[0].leader_src.at_slot(1).assign({1, 2}); + pieces[0].leader_src.at_slot(1).shrink_to_fit(); + pieces[2].leader_src.at_slot(1).assign({8, 9, 10}); + // Self stage, aligned with slot 0's sources: only piece 1 has queries, so its stage is moved whole. + const std::vector pos{3, 9}; + for (int q = 0; q < 3; ++q) { + pieces[1].leader_self.push(pos, 1); + } + pieces[0].leader_self.reserve(8, 4); // capacity without content is not a query + const auto *stage_data = pieces[1].leader_self.pos_flat.data(); + + auto merged = merge_scan_ranges(pieces, window, me, false); + BOOST_TEST(merged.leader_src.at_slot(0).data() == only_data); + BOOST_TEST(merged.follower_src.at_slot(1).data() == roomy_data); + BOOST_TEST((merged.follower_src.at_slot(1) == std::vector{1, 2, 9})); + BOOST_TEST((merged.leader_src.at_slot(1) == std::vector{1, 2, 8, 9, 10})); + BOOST_TEST(merged.leader_src.at_slot(1).capacity() == 5U); + BOOST_TEST(merged.leader_self.size() == 3U); + BOOST_TEST(merged.leader_self.pos_flat.data() == stage_data); +} diff --git a/docs/content/docs/features/parallelism.mdx b/docs/content/docs/features/parallelism.mdx index 489a2c99..982a7025 100644 --- a/docs/content/docs/features/parallelism.mdx +++ b/docs/content/docs/features/parallelism.mdx @@ -61,12 +61,16 @@ multi-partition runs are unchanged. (`omp_get_max_threads()`, normally `OMP_NUM_THREADS`) applies. The budget is not clamped to the cores you have; placement belongs to `OMP_PLACES`, `OMP_PROC_BIND` and your launcher. Copies and retained callables keep the budget, and a later change to the environment only affects new objects. -- **Threaded kernels so far.** Two kernels use the budget: the per-gate cosine scaling (graph - replay during evaluation and contraction, and the two-pass path of in-place propagation) and the - rotation apply of in-place `propagate`. Each splits its work into fixed blocks of 1024 bitmap words - or 1024 rotation records and stays serial below two blocks, so small operators see no threads. - Results are bitwise identical at every budget. Graph construction (scan, lookup, insertion), - reductions and the gradient's endpoint work are still serial. +- **Threaded kernels so far.** Three kernels use the budget: the per-gate cosine scaling (graph + replay during evaluation and contraction, and the two-pass path of in-place propagation), the + rotation apply of in-place `propagate`, and the bitmap scan that finds each gate's anticommuting + terms and emits their partner queries during graph construction and in-place `propagate`. Each + splits its work into fixed blocks of 1024 bitmap words or 1024 rotation records and stays serial + below two blocks, so small operators see no threads. The scan's word ranges are merged back in + source order, so queries, new term IDs and graphs are the same at every budget, and results are + bitwise identical. The scan stays serial for a cutoff other than the built-in length or support + cutoffs, including a `basis_change` cutoff. Partner lookup, insertion, graph packing, reductions + and the gradient's endpoint work are still serial. - **Calling thread.** Every call that can use MPI (construction, operations, retained callables) must come from the thread that initialized MPI, whatever thread support MPI provides. A call from another thread prints a diagnostic and aborts that process without communicating; the launcher From 24bb70e18620a04a311d1d3169904158f1d39962 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Roberto=20Di=20Remigio=20Eik=C3=A5s?= Date: Mon, 28 Sep 2026 21:27:07 +0000 Subject: [PATCH 19/64] =?UTF-8?q?docs(parallelism):=20=F0=9F=93=9D=20recor?= =?UTF-8?q?d=20the=20Task=205=20outcome?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Record what Task 5 delivered and what later tasks inherit: - the scan range, merge, cutoff-classification and observer contracts; - RED/GREEN, mutation, stress, sanitizer and cross-revision evidence; - the owner-approved construction diagnostics: traversal and end-to-end speedups, the losses at high budgets, the growing serial merge, and the scratch and peak-memory accounting; - where the uncommitted Task 5 artifacts live. Assisted-by: Pi:claude-opus-5-5 --- .../plans/2026-09-18-rank-local-openmp.md | 53 +++++++++++++++++++ 1 file changed, 53 insertions(+) diff --git a/docs/superpowers/plans/2026-09-18-rank-local-openmp.md b/docs/superpowers/plans/2026-09-18-rank-local-openmp.md index e9ad3cd8..d4d04a4d 100644 --- a/docs/superpowers/plans/2026-09-18-rank-local-openmp.md +++ b/docs/superpowers/plans/2026-09-18-rank-local-openmp.md @@ -1753,6 +1753,59 @@ window/self regressions, not only the old serial path. GREEN: exact serial/threa graph fields at fixed geometry. Ordered query/row/graph equivalence, no callback-thread contract change, no lazy-cache races. Construction bitmap-traversal time and peak scratch are recorded separately from resolution. +**Task 5 outcome (2026-09-28):** the owner accepted the gate, with the high-budget losses and the unexplained P1 +peak growth on record. Correctness gates pass on GCC 15.2 (R, M) and Clang 18.1.8/libomp (R) Linux. Other platforms +are pending. +- Contracts as implemented, which later tasks rely on: + - `fused_find_and_collect` prepares the inverted index and row parity on the caller, then splits the words into + `scan_ranges(words, options)` = `min(threads, ceil(words/1024))` contiguous runs of whole fold blocks + (`scan_range_words`). Range IDs depend on the word count and budget only, never on the actual team. + - Each range runs `fused_find_and_collect_range` into a private `FusedScanResult`. It never calls a lazy accessor, + acquires its thread_local scratch on the executing thread, masks only the global tail word and, in fused mode, + scales only its own sources, once, after capturing the pre-scale value. + - `merge_scan_ranges` joins the pieces in range order, stream by stream: checked totals, one reservation, the + first buffer moved when it holds the stream or has room, drained buffers released (swapped, not cleared). + Self stages merge through `SelfQueryStage::push`; cosine sets stay one per range, ascending and disjoint, and + `build_layer` concatenates them. + - `CutoffEvaluator::parallel_safe()` is true only for typed `LengthCutoff`/`SupportCutoff`. Opaque predicates, + basis-change closures and scans below two ranges run the unchanged serial scan on the caller. + - The test-only observer seam (`KernelRange`, `NoRangeObserver`, `logical_ranges`) moved to + `layer_build/Common.h` and gained `KernelRange::scan`. `build_layer` forwards a defaulted observer. + - Resolution, missing-ID assignment, insertion, publication, graph packing and `scaled_count` are unchanged. +- Tests: + - `cpp/tests/ScanTestSupport.h` builds a four-fold-block synthetic operator per rank and compares every stream, + both self stages, the cosine set and the coefficients byte for byte at budgets 2-4 against budget 1. + - The `openmp_scan_*` cases span one rank, linear zero/non-zero shifts with non-zero window bases, dense splitmix + at three and four ranks, dense/sparse/empty pivots, Majorana and Pauli J(G), caps, cutoff 0, atol boundaries, + capture, the fused sweep, cold caches, the merge itself, and opaque/basis-change serial fallback. + - Full construction at budgets 1-4 gives identical rows, row IDs, graph layers and coefficients in both pictures. + - Failures: a scan worker throw joins before rethrow and invalidates the propagator; the new two-rank scenario + `before-exchange scan-worker` aborts promptly under both routings. + - RED: the tests did not compile without the helper, then failed every multi-range expectation with traversal + forced serial while exactness passed. GREEN: 2-4 distinct workers in the scan's own ranges. + - Seven mutants were all killed, one by TSan. Stress: 800 fresh processes, no failures. Serial scan digests and + Task 4's cross-revision probe are bitwise equal to Task 4 at every budget checked. +- Final runs on fresh rebuilds: R, Clang and ASan/UBSan ctest 366/366 (pytest 840, 840, 819 passed); TSan (Clang + + libomp + Archer, same single suppression) 85/85 with no reports; M serial 396/396, linear R=1,2,4 and splitmix + R=1-4 MPI ctest and `--with-mpi` pytest pass; `just test-find-package` passes on R and M. +- Construction diagnostics (owner-approved budget; 18 of 60 min used; diagnostic, not acceptance evidence): + - End-to-end speedup at the best budget: random Heisenberg `build_graph` 4.8× (T=48), random Schrödinger + `build_graph` 11.8× (T=96), Pauli-127 `build_graph` 1.55× (T=4), random Heisenberg `propagate` 5.0× (T=48; + Task 4 had no gain). + - Traversal alone scales 18-21× on the random cases and 3× on Pauli-127. Resolution, deferred insertion, + finalisation and the replay step stay serial and now bound B1 and B4. + - Losses: `propagate` 18% slower at T=96 than T=48; Pauli-127 4.5% slower at T≥48 than T=16; random Heisenberg + `build_graph` 2% slower at T=96 than T=48. + - The serial ordered merge grows with the range count, to 28-37% of traversal time at high T. It is a Task 11 + bounded-optimisation candidate. + - Memory: the threaded scan's simultaneous private+merged heap peak never exceeded the serial scan's, so the + two-pass exact fill was not needed. Process peak grows 0.5-7% at T>1; `MALLOC_ARENA_MAX=1` removes it for the + Schrödinger and Pauli cases (glibc per-thread arenas), but `propagate`'s +2.9% remains unexplained. +- Not run: `just build-docs`, and the Nix, macOS/Homebrew, devcontainer and wheel-repair routes. +- Artifacts: `/home/ubuntu/task5-artifacts/` on the c8a.metal-24xl host (`report.md`, `ledger.md`, + `diag-results.md`, `diag/`). They are intentionally uncommitted. The frozen campaign and workload files, the + Task 1 artifacts, and the baseline binaries and worktrees were verified unchanged. + ### Task 6: Parallel frozen lookup while retaining serial insertion **Files:** From 43efb1685d4bf1152d2ee7a48b0322d4e38984d9 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Roberto=20Di=20Remigio=20Eik=C3=A5s?= Date: Tue, 29 Sep 2026 11:45:21 +0000 Subject: [PATCH 20/64] =?UTF-8?q?feat(parallel):=20=E2=9C=A8=20thread=20fr?= =?UTF-8?q?ozen=20lookup=20with=20checked=20incoming=20decode?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Task 6 of the rank-local OpenMP plan, committed as the record of the one-store experiment (the owner decided afterwards to re-design and keep the sharding). Only the prototype selected by explicit partitions=1 on an ordinary communicator uses these threaded paths. - probe_frozen_positions: 256-query blocks, each one find_batch_positions call over absolute offsets into the shared position buffer with sliced counts, outputs and hashes; span contracts checked before any block runs; the store stays frozen until every worker joins. - probe_incoming_queries validates every sender stream on the caller with QueryWire::checked_extent (ternary phase, k and gap-width bounds, canonical escaped counts, whole records including the fused value word, final offset at the buffer end), checks prefixes and allocates every array once. Workers decode disjoint blocks; read_positions rejects size_t positions >= 2N before narrowing. Misses get IDs serially in sender-window/query order after a row-ID ceiling check; miss_g is vector. - resolve_incoming scatters on workers only for sinks declaring parallel_resolve (GraphSink, Heisenberg ContractSink); Schrodinger scoring and generic sinks stay serial. It runs inside guard_distributed, as the peers may be in the response round. - The self lookup probes windows of at most 4096 queries; filtering, gathering and emission stay serial and nothing is inserted between windows. Publication order and the single-writer grow/set_positions/bulk_insert_hashed/reindex sequence are unchanged; bulk_insert_hashed gains an ignored trailing parallel::Options seam. - The test-only observer gains KernelRange::{decode,incoming_probe,self_probe,scatter}. Tests: - Serial/threaded bitwise equality of probes, decoded positions, hashes, predicted and published IDs, row order, responses, sink records, marks and rebuilt inverted indices, over empty/all-hit/all-miss/ mixed inputs, 255/256/257-query boundaries, empty senders, non-zero window bases, narrow and wide positions, spilled rows, escaped counts, Plain/Fused codecs and 4096-query self windows. - Malformed streams (including the noncanonical escape 0x8037e) rejected before decode; invalid positions rejected in decode before probing; serial scatter for uncertified sinks; duplicate-key bulk insertion pinned; participation observed inside each new region; four new two-rank resolve-worker failure scenarios abort promptly. Assisted-by: Pi:claude-opus-5-5 --- AGENTS.md | 13 +- .../detail/evolution/layer_build/Common.h | 14 +- .../detail/evolution/layer_build/Engine.h | 145 +- .../detail/evolution/layer_build/QueryWire.h | 72 +- .../detail/evolution/layer_build/Resolve.h | 383 +++++- cpp/monoprop/detail/operator/OperatorIndex.h | 12 +- cpp/tests/CMakeLists.txt | 17 +- cpp/tests/KernelTestSupport.h | 57 +- cpp/tests/README.md | 16 +- cpp/tests/fused_cos_sweep_tests.cpp | 39 + cpp/tests/mpi_failure_driver.cpp | 76 ++ cpp/tests/mpi_fresh_insert_equivalence.cpp | 156 ++- cpp/tests/operator_index_tests.cpp | 95 ++ cpp/tests/sparse_query_tests.cpp | 147 ++ cpp/tests/sparse_resolve_tests.cpp | 1206 +++++++++++++++++ docs/content/docs/features/parallelism.mdx | 24 +- 16 files changed, 2328 insertions(+), 144 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index 56af1576..9ff3d5b5 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -72,10 +72,15 @@ just build-docs separates validation from mutation (`run_operation_`): a failure after mutation starts invalidates the object and aborts a multi-rank ordinary communicator (`mpi::operation_failed`). Catch failures inside posted `Ticket`/`PendingAlltoallv` lifetimes, never only at the outermost level. -- Threaded kernels (`scale_cos_*`, `apply_fused_contract`, the `fused_find_and_collect` scan) take a - defaulted test-only range observer; participation tests read it (`cpp/tests/KernelTestSupport.h`, - `ScanTestSupport.h`). Serial and threaded runs must agree bitwise. Scan workers only traverse frozen - storage and merge in range order; only typed cutoffs (`CutoffEvaluator::parallel_safe`) run on them. TSan evidence needs Clang + libomp + Archer; GCC's libgomp gives false reports (`building.mdx`). +- Threaded kernels (`scale_cos_*`, `apply_fused_contract`, the `fused_find_and_collect` scan, the resolve + phases: incoming decode, `probe_frozen_positions`, certified-sink scatter) take a defaulted test-only + range observer; participation tests read it (`cpp/tests/KernelTestSupport.h`, `ScanTestSupport.h`). + Serial and threaded runs must agree bitwise. Scan workers only traverse frozen storage and merge in range + order; only typed cutoffs (`CutoffEvaluator::parallel_safe`) run on them. Probes read a frozen store; + received query streams are validated on the caller (`QueryWire::checked_extent`) before any decode; + miss IDs, insertion (`bulk_insert_hashed`) and inverted-index maintenance stay single-writer. A sink's + `on_resolved` runs on workers only with `static constexpr bool parallel_resolve = true`. TSan evidence + needs Clang + libomp + Archer; GCC's libgomp gives false reports (`building.mdx`). - `cpp/tests/mpi_failure_driver.cpp` has its own `main()`, excluded from the unit-runner glob; `ctest -L mpi_failure` runs its two-rank abort scenarios (30 s timeout; a timeout is a hang). - Slow CTest startup in MPI builds is `MPI_Init` fabric probing; see diff --git a/cpp/monoprop/detail/evolution/layer_build/Common.h b/cpp/monoprop/detail/evolution/layer_build/Common.h index 1734df7b..3c9bfa5c 100644 --- a/cpp/monoprop/detail/evolution/layer_build/Common.h +++ b/cpp/monoprop/detail/evolution/layer_build/Common.h @@ -32,11 +32,15 @@ namespace monoprop::detail { * \brief The kinds of logical work range a threaded kernel reports to its range observer (see NoRangeObserver). */ enum class KernelRange { - cos_lazy, //!< scale_cos_lazy: one range per kColumnBlockWords fold words. - cos_mask, //!< scale_cos_mask: one range per kCosMaskRangeBlocks stored mask blocks. - fused_gather, //!< apply_fused_contract insert snapshots: one range per kFusedRecordRange records. - fused_apply, //!< apply_fused_contract rotations: one range per kFusedRecordRange records. - scan, //!< fused_find_and_collect: one range per contiguous run of whole fold blocks (scan_ranges). + cos_lazy, //!< scale_cos_lazy: one range per kColumnBlockWords fold words. + cos_mask, //!< scale_cos_mask: one range per kCosMaskRangeBlocks stored mask blocks. + fused_gather, //!< apply_fused_contract insert snapshots: one range per kFusedRecordRange records. + fused_apply, //!< apply_fused_contract rotations: one range per kFusedRecordRange records. + scan, //!< fused_find_and_collect: one range per contiguous run of whole fold blocks (scan_ranges). + decode, //!< probe_incoming_queries record decode: one range per kProbeBlockQueries queries. + incoming_probe, //!< probe_frozen_positions over incoming queries: one range per kProbeBlockQueries queries. + self_probe, //!< probe_frozen_positions over one self window: one range per kProbeBlockQueries queries. + scatter, //!< resolve_incoming on_resolved scatter: one range per kResolveScatterBlock queries. }; /*! diff --git a/cpp/monoprop/detail/evolution/layer_build/Engine.h b/cpp/monoprop/detail/evolution/layer_build/Engine.h index e69fd0ec..20e27cb9 100644 --- a/cpp/monoprop/detail/evolution/layer_build/Engine.h +++ b/cpp/monoprop/detail/evolution/layer_build/Engine.h @@ -75,6 +75,9 @@ inline auto append_inserted_endpoints(CosMask &cos_all, size_t combined_size, co template struct GraphSink { static constexpr bool wants_values = false; + // on_resolved for distinct queries may run concurrently after prepare(): each writes only its own + // acc[slot].in_entries element, which prepare() sized. Certifies nothing about the other methods. + static constexpr bool parallel_resolve = true; [[nodiscard]] auto incoming_form() const -> QueryForm { return QueryForm::Plain; } [[nodiscard]] auto querier_form() const -> QueryForm { return QueryForm::Plain; } using Response = TermIndex; @@ -196,6 +199,10 @@ struct GraphSink { template struct ContractSink { static constexpr bool wants_values = true; + // on_resolved for distinct queries may run concurrently after prepare() in Heisenberg: each writes only its + // own cross_half element and reads the pre-insert coefficients. Schrodinger scoring stays serial, see + // parallel_resolve_enabled(). Certifies nothing about the other methods (on_response_block appends). + static constexpr bool parallel_resolve = true; // This rank receives fused records, but on_response_block is handed its own plain queries_r; // reading the wrong form there decodes a neighbouring record's phase, i.e. a coefficient sign flip. [[nodiscard]] auto incoming_form() const -> QueryForm { return QueryForm::Fused; } @@ -232,6 +239,9 @@ struct ContractSink { fc.inserts[def_base_ + k] = RotationRec{src, idx, v_src, /*v_tgt=*/0.0, static_cast(phase)}; } + //! The Schrodinger fresh-insert arm scores the state through mono_at; it keeps the serial scatter. + [[nodiscard]] auto parallel_resolve_enabled() const -> bool { return !schrodinger; } + // Cross-rank (R>1). Send buffer = queries interleaved with their v_src stream into `scratch` // (combined_qv_), so one alltoallv carries query + value. auto send_buffer(mpi::WindowVec &queries, @@ -311,7 +321,8 @@ struct ContractSink { }; // Owns build_layer's machinery over a compile-time Sink policy. combined_size = the pre-layer operator size. -template +// `Observer` is the test seam of the threaded resolve phases (see NoRangeObserver). +template struct LayerBuildEngine { using RowPosT = typename OperatorIndex::PosT; @@ -351,6 +362,7 @@ struct LayerBuildEngine { Sink sink; // The owning propagator's thread budget, forwarded to the resolve kernels. parallel::Options options; + Observer observer; LayerBuildEngine(MPOperator &local_op_, mpi::Comm comm_, @@ -361,7 +373,8 @@ struct LayerBuildEngine { Sink &&sink_, mpi::PeerPlan plan_, mpi::SlotWindow window_, - parallel::Options options_ = {}) + parallel::Options options_ = {}, + const Observer &observer_ = {}) : local_op(local_op_), comm(comm_), R(R_), @@ -371,7 +384,8 @@ struct LayerBuildEngine { window(window_), plan(plan_), sink(std::move(sink_)), - options(options_) { + options(options_), + observer(observer_) { assert(window.stop() <= R && window.count != 0); queries_r.reset(window); src_idx_r.reset(window); @@ -431,7 +445,18 @@ struct LayerBuildEngine { // never reaches the handle's draining destructor while a peer is still inside the round. auto query_round = mpi::begin_alltoallv(send, comm, /*skip_self=*/false, /*known_recv_counts=*/nullptr, plan); mpi::guard_distributed(comm, [&] { query_round.wait_into(inc_q); }); - auto resp = resolve_incoming(inc_q, local_op, is_leader_pass, matched, combined_size, sink, options); + // The peers may already be inside the response round, so a failure here (a malformed stream, a + // decode/probe/scatter worker, publication) aborts rather than unwinds. + auto resp = mpi::guard_distributed(comm, [&] { + return resolve_incoming(inc_q, + local_op, + is_leader_pass, + matched, + combined_size, + sink, + options, + observer); + }); std::vector resp_recv = response_recv_counts(); mpi::WindowVec> inc_r; // Answers retrace the queries over an XOR involution, so the same plan holds. @@ -498,7 +523,8 @@ struct LayerBuildEngine { std::span(deferred_pos_flat_).subspan(m.pos_at, m.k)); sink.emit_deferred(k, base + k, m.src, m.phase, m.v_src); } - local_op.store->bulk_insert_hashed(n_miss, base, [&](size_t j) { return deferred_self_misses[j].hash; }); + local_op.store + ->bulk_insert_hashed(n_miss, base, [&](size_t j) { return deferred_self_misses[j].hash; }, options); local_op.reindex_after_growth(base, n_miss); } @@ -520,35 +546,52 @@ struct LayerBuildEngine { return counts; } - // Batched self-resolve over the index's group-prefetch find_batch; hits/misses are emitted to the sink - // in query order. `lv` is the per-query v_src array parallel to `ls` (read only when Sink::wants_values). - static constexpr size_t kResolveBatch = 64; + // Self-resolve scratch for one window of at most kSelfProbeWindow queries, reused across windows and both + // passes: gathered offsets (absolute into the stage), counts, results, hashes and the emission payload. + std::vector self_pos_off_; + std::vector self_k_of_; + std::vector self_found_; + std::vector self_hash_; + std::vector self_phase_; + std::vector self_src_; + std::vector self_val_; // ContractSink only + + // Self-resolve in bounded windows. For each window, on the caller: stable follower filtering (leaders skip + // it) and a gather into the scratch; then probe_frozen_positions (KernelRange::self_probe) on workers over + // at most 16 blocks; then, on the caller, hit/miss emission in original order. Nothing is inserted between + // windows or passes: misses keep their positions and hashes for insert_deferred_self_misses. `lv` is the + // per-query v_src array parallel to `ls` (read only when Sink::wants_values). auto resolve_range_(std::vector &ls, [[maybe_unused]] std::vector *lv, bool is_leader_pass) -> void { const size_t op_size = local_op.store->size(); - // Gathered per batch because a matched follower is skipped; offsets stay absolute into pos_flat. - std::array pos_off; - std::array k_of; - std::array hashes; - std::array phases; - std::array srcs; - std::array vals; - std::array found; const size_t hi = self_stage_.size(); + const size_t cap = std::min(hi, kSelfProbeWindow); + if (self_pos_off_.size() < cap) { + self_pos_off_.resize(cap); + self_k_of_.resize(cap); + self_found_.resize(cap); + self_hash_.resize(cap); + self_phase_.resize(cap); + self_src_.resize(cap); + if constexpr (Sink::wants_values) { + self_val_.resize(cap); + } + } + const std::span stage_pos(self_stage_.pos_flat.data(), self_stage_.positions()); size_t q = 0; while (q < hi) { size_t m = 0; - for (; q < hi && m < kResolveBatch; ++q) { + for (; q < hi && m < kSelfProbeWindow; ++q) { const size_t src = ls[q]; if (!is_leader_pass && matched.is_marked(src)) { continue; // follower already matched by a leader → not an independent rotation } - pos_off[m] = self_stage_.pos_off[q]; - k_of[m] = self_stage_.k_of[q]; - phases[m] = self_stage_.phase_of[q]; - srcs[m] = src; + self_pos_off_[m] = self_stage_.pos_off[q]; + self_k_of_[m] = self_stage_.k_of[q]; + self_phase_[m] = self_stage_.phase_of[q]; + self_src_[m] = src; if constexpr (Sink::wants_values) { - vals[m] = (*lv)[q]; + self_val_[m] = (*lv)[q]; } ++m; } @@ -556,30 +599,36 @@ struct LayerBuildEngine { break; } // The hashes come back because a miss needs one at insert, folded from these same positions. - local_op.store->find_batch_positions(std::span(self_stage_.pos_flat), - std::span(pos_off).first(m), - std::span(k_of).first(m), - std::span(found).first(m), - std::span(hashes).first(m)); + probe_frozen_positions(*local_op.store, + stage_pos, + std::span(self_pos_off_).first(m), + std::span(self_k_of_).first(m), + std::span(self_found_).first(m), + std::span(self_hash_).first(m), + options, + observer, + KernelRange::self_probe); for (size_t j = 0; j < m; ++j) { double v_src = 0.0; if constexpr (Sink::wants_values) { - v_src = vals[j]; + v_src = self_val_[j]; } + const size_t found = self_found_[j]; // kNotFound == kMissingIndex == size_t max, so one bound check covers both. - if (found[j] < op_size) { + if (found < op_size) { // Freshly inserted partners (found >= combined_size) skip the mark: combined_size bounds it. - if (is_leader_pass && found[j] < combined_size) { - matched.mark(found[j]); + if (is_leader_pass && found < combined_size) { + matched.mark(found); } - sink.self_hit(srcs[j], found[j], phases[j], v_src); + sink.self_hit(self_src_[j], found, self_phase_[j], v_src); } else { // The stage dies with this pass and the misses are flushed after both, so copy now. const size_t at = deferred_pos_flat_.size(); - const auto *const first = self_stage_.pos_flat.data() + pos_off[j]; - deferred_pos_flat_.insert(deferred_pos_flat_.end(), first, first + k_of[j]); - deferred_self_misses.push_back({at, k_of[j], hashes[j], srcs[j], phases[j], v_src}); + const auto *const first = self_stage_.pos_flat.data() + self_pos_off_[j]; + deferred_pos_flat_.insert(deferred_pos_flat_.end(), first, first + self_k_of_[j]); + deferred_self_misses.push_back( + {at, self_k_of_[j], self_hash_[j], self_src_[j], self_phase_[j], v_src}); } } } @@ -592,8 +641,9 @@ static inline auto empty_coeffs() -> const VecD & { } // Primary-path layer builder: one fused scan, then two resolve passes into the chosen sink. See LayerBuilder.h. -// The scan may run on workers (fused_find_and_collect); resolution, missing-ID assignment, insertion, -// publication and graph packing stay on the caller. `observer` is the scan's test seam (NoRangeObserver). +// The scan (fused_find_and_collect), the incoming decode and frozen probes, the self probe and a certified +// sink's scatter may run on workers; validation, missing-ID assignment, insertion, publication, inverted-index +// maintenance and graph packing stay on the caller. `observer` is the test seam of those phases (NoRangeObserver). template auto build_layer(MPOperator &local_op, const Monomial &gen, @@ -692,16 +742,17 @@ auto build_layer(MPOperator &local_op, } auto run = [&](Sink sink) -> std::shared_ptr { - LayerBuildEngine eng(local_op, - comm, - R, - my_rank, - matched_scratch, - /*combined_size=*/local_op.store->size(), - std::move(sink), - plan, - scan_window, - options); + LayerBuildEngine eng(local_op, + comm, + R, + my_rank, + matched_scratch, + /*combined_size=*/local_op.store->size(), + std::move(sink), + plan, + scan_window, + options, + observer); if (!identity_gen) { eng.run_exchange(/*is_leader_pass=*/true, std::move(fused.leader_queries), diff --git a/cpp/monoprop/detail/evolution/layer_build/QueryWire.h b/cpp/monoprop/detail/evolution/layer_build/QueryWire.h index 7f296d8a..b8568373 100644 --- a/cpp/monoprop/detail/evolution/layer_build/QueryWire.h +++ b/cpp/monoprop/detail/evolution/layer_build/QueryWire.h @@ -19,8 +19,11 @@ #include #include #include +#include #include #include +#include +#include #include #include @@ -32,6 +35,14 @@ namespace monoprop::detail { /*! @brief `Fused` records carry a trailing value word after the positions; `Plain` ones do not. */ enum class QueryForm { Plain, Fused }; +/*! @brief A received query stream that no QueryWire::push sequence could have produced: a truncated or + * noncanonical record, an out-of-range header field, or positions outside the term's width. + */ +class MalformedQueryStream : public std::invalid_argument { +public: + using std::invalid_argument::invalid_argument; +}; + /*! @brief One term's wire record for a cross-rank query: its ascending set-bit positions, * gap-coded, plus a phase. * @@ -186,6 +197,51 @@ struct QueryWire { return words_of_header(header_at(buf, off)); } + /*! @brief A validated record's position count and its words, the fused value word included. */ + struct RecordExtent { + size_t k; //!< positions the record decodes to + size_t words; //!< words from the record start to the next record + }; + + /*! @brief Validates the record at `off` in `buf` without reading past the buffer, and returns its extent. + * + * Checks, in order: `off` names a word of `buf`; the phase field is ternary; k <= kMaxPositions; gw <= + * kPosBits; the header is canonical (`h.bits == header_bits_for(h.k)`, so an escaped count is at least + * kKEscape); and the whole record, value word included for `Fused`, fits the rest of the buffer. A + * noncanonical escape would otherwise make words_of_header() undercount what read_positions() consumes. + * Positions themselves are checked by read_positions(), which decodes them. + * + * \throws MalformedQueryStream naming the failed check. + */ + [[nodiscard]] static auto checked_extent(WireView buf, QueryForm form, size_t off) -> RecordExtent { + if (off >= buf.size()) { + throw MalformedQueryStream( + std::format("QueryWire: no record at word {} of a {}-word stream", off, buf.size())); + } + const Header h = header_at(buf, off); + const auto malformed = [&](std::string_view what) { + return MalformedQueryStream(std::format("QueryWire: malformed record at word {}: {}", off, what)); + }; + if (h.phase > 1) { + throw malformed("the phase field is not ternary"); + } + if (h.k > kMaxPositions) { + throw malformed(std::format("{} positions exceed the width's {}", h.k, kMaxPositions)); + } + if (h.gw > kPosBits) { + throw malformed(std::format("gap width {} exceeds {}", h.gw, kPosBits)); + } + if (h.bits != header_bits_for(h.k)) { + throw malformed(std::format("a noncanonical escaped count {}", h.k)); + } + // Every term fits kMaxPositions <= 65535 positions of at most 16 bits, so gap_bits cannot wrap. + const size_t words = words_of_header(h) + (form == QueryForm::Fused ? 1U : 0U); + if (words > buf.size() - off) { + throw malformed(std::format("{} words needed, {} remain", words, buf.size() - off)); + } + return {h.k, words}; + } + [[nodiscard]] static auto k_at(WireView buf, size_t off) noexcept -> size_t { return header_at(buf, off).k; } [[nodiscard]] static auto phase_at(WireView buf, size_t off) noexcept -> int { return header_at(buf, off).phase; } @@ -238,18 +294,32 @@ struct QueryWire { /*! @brief Decodes the record at `off` into `out`, whose element type is deduced so the resolve * path can decode straight into the store's narrower position width. * - * `Decoded::next` names the word just past the positions, excluding any value word. + * The record must lie within `buf` (checked_extent() establishes that for received streams) and `out` + * must hold its k positions. Each cumulative position is checked below kBits as a size_t, before it is + * narrowed to the output type: that also bounds the next gap's addition, so the positions are strictly + * ascending and cannot wrap. `Decoded::next` names the word just past the positions, excluding any value + * word. + * + * \throws MalformedQueryStream if a position is outside [0, kBits); `out` is then partly written. */ template static auto read_positions(WireView buf, size_t off, Out &&out) -> Decoded { using OutT = std::ranges::range_value_t; const Header h = header_at(buf, off); Reader r{buf, off, h.bits}; + const auto check = [&](size_t pos) { + if (pos >= kBits) { + throw MalformedQueryStream( + std::format("QueryWire: record at word {} decodes position {} of a {}-bit term", off, pos, kBits)); + } + }; if (h.k != 0) { auto prev = static_cast(r.get(kPosBits)); + check(prev); out[0] = static_cast(prev); for (size_t j = 1; j < h.k; ++j) { prev += static_cast(r.get(h.gw)) + 1U; + check(prev); out[j] = static_cast(prev); } } diff --git a/cpp/monoprop/detail/evolution/layer_build/Resolve.h b/cpp/monoprop/detail/evolution/layer_build/Resolve.h index a50b6a0c..b465d8d1 100644 --- a/cpp/monoprop/detail/evolution/layer_build/Resolve.h +++ b/cpp/monoprop/detail/evolution/layer_build/Resolve.h @@ -14,10 +14,15 @@ #pragma once +#include #include +#include #include +#include +#include #include #include +#include #include #include "monoprop/TypeAliases.h" @@ -29,9 +34,78 @@ #include "monoprop/detail/operator/MPOperator.h" #include "monoprop/detail/operator/RowAccess.h" #include "monoprop/detail/parallel/Options.h" +#include "monoprop/detail/parallel/Workshare.h" namespace monoprop::detail { +//! Queries per logical block of the frozen probe and the incoming decode: a function of the query count only. +inline constexpr size_t kProbeBlockQueries = 256; +//! Queries per logical block of the incoming on_resolved scatter. +inline constexpr size_t kResolveScatterBlock = 1024; +//! Most self queries probed per window; bounds the gathered scratch (at most 16 probe blocks per window). +inline constexpr size_t kSelfProbeWindow = 4096; + +/*! + * \brief Look up `out.size()` position-list queries in a frozen store, on up to `options.threads` workers. + * + * Query `q` is `pos_flat[pos_off[q], pos_off[q] + k_of[q])`; offsets are absolute into the one shared buffer. + * The queries split into logical blocks of kProbeBlockQueries, each one OperatorIndex::find_batch_positions + * call over its slice of `pos_off`, `k_of`, `out` and `hash_out`, so the batched pipeline, the exact collision + * check and the cached hashes are the serial ones and the results are identical at every budget and team + * size. `out[q]` is the row ID or OperatorIndex::kNotFound; `hash_out`, when not empty, receives each query's + * fold hash. A single block runs on the caller. + * + * The store must stay frozen -- no row, overflow, table or index mutation -- until this returns; it returns + * only after every worker has joined. + * + * \param observer Test seam; see NoRangeObserver. \param kind The KernelRange it reports. + * \throws std::invalid_argument unless `pos_off`, `k_of` and `out` have equal lengths, `hash_out` is empty or of + * that length, and every query's span lies inside `pos_flat`; all checked before any block runs. + */ +template +auto probe_frozen_positions(const OperatorIndex &store, + std::span::PosT> pos_flat, + std::span pos_off, + std::span k_of, + std::span out, + std::span hash_out, + parallel::Options options = {}, + const Observer &observer = {}, + KernelRange kind = KernelRange::incoming_probe) -> void { + const size_t n = out.size(); + if (pos_off.size() != n || k_of.size() != n || (!hash_out.empty() && hash_out.size() != n)) { + throw std::invalid_argument(std::format("probe_frozen_positions: {} offsets, {} counts, {} outputs and {} " + "hashes; the spans must have equal lengths (hashes may be empty)", + pos_off.size(), + k_of.size(), + n, + hash_out.size())); + } + for (size_t q = 0; q < n; ++q) { + if (pos_off[q] > pos_flat.size() || k_of[q] > pos_flat.size() - pos_off[q]) { + throw std::invalid_argument(std::format("probe_frozen_positions: query {} spans [{}, +{}) outside the " + "{}-position buffer", + q, + pos_off[q], + k_of[q], + pos_flat.size())); + } + } + const size_t blocks = logical_ranges(n, kProbeBlockQueries); + observer.prepare(kind, blocks); + // Each block writes only its own slice of `out` and `hash_out`; the store and pos_flat are read-only. + parallel::for_blocks(blocks, options, [&](size_t block) { + observer.visit(kind, block); + const size_t lo = block * kProbeBlockQueries; + const size_t m = std::min(kProbeBlockQueries, n - lo); + store.find_batch_positions(pos_flat, + pos_off.subspan(lo, m), + k_of.subspan(lo, m), + out.subspan(lo, m), + hash_out.empty() ? std::span{} : hash_out.subspan(lo, m)); + }); +} + // Picture-independent probe/insert machinery shared by resolve_incoming and its callers. Each miss takes // the next index base+j in (sender,record) order, so the assignment (and multi-rank bit-exactness) cannot // drift between resolvers. Queries are source⊕G over globally-distinct sources, ⊕G injective ⇒ queries @@ -49,11 +123,14 @@ struct IncomingProbe { // g → word offset of that query inside its sender's buffer; a query ordinal names no position. DefaultInitVector off_of; DefaultInitVector idx_of; // g → resolved index (hit: < base; miss: base+j) - std::vector miss_g; // j → the g that became miss j (Phase 4 reads the key of miss_g[j]) - size_t base = 0; // op size before the miss inserts (the miss-index base) + // j → the query ordinal g that became miss j (Phase 4 reads its key). Ordinals, not TermIndex values: + // their range is the query count, not the store's. + std::vector miss_g; + size_t base = 0; // op size before the miss inserts (the miss-index base) size_t nq_total = 0; - // The queries as they arrived, flat: query g owns pos_flat[pos_off[g] .. pos_off[g] + k_of[g]). + // The queries as they arrived, flat: query g owns pos_flat[pos_off[g] .. pos_off[g] + k_of[g]). Every + // per-query array holds exactly nq_total elements, pos_flat exactly the decoded positions. DefaultInitVector pos_flat; DefaultInitVector pos_off; DefaultInitVector k_of; @@ -84,91 +161,185 @@ struct IncomingProbe { } }; -// Read-only phases 1-2 of the exchange: `form` says whether the incoming records are fused -// (ContractSink) or plain (GraphSink). The caller runs phase 3, then insert_incoming_misses. -template +namespace resolve_detail { + +// Refuses a length the vector type cannot hold, before any allocation. +template +auto check_fits(const Vec &v, size_t n, const char *what) -> void { + if (n > v.max_size()) { + throw std::length_error(std::format("probe_incoming_queries: {} {} exceed the vector limit", n, what)); + } +} + +} // namespace resolve_detail + +/*! + * \brief Read-only phases 1-2 of the exchange: validate, decode and probe the received queries, then assign + * each miss its predicted row ID. + * + * `form` says whether the records are fused (ContractSink) or plain (GraphSink). In order: + * 1. On the caller, every sender stream is walked in window order with QueryWire::checked_extent, so a + * malformed header, a noncanonical escape or a truncated record is rejected before any payload is read. + * The query and position prefixes, sender narrowing and vector limits are checked, every per-query array + * is allocated once at its exact size, and the offsets (absolute into `pos_flat`), counts and senders are + * filled in canonical sender-window/query order. + * 2. Workers decode disjoint blocks of records into their prepared position spans; read_positions rejects + * positions outside the term's width, and each record must end exactly at its prepared boundary. + * 3. After that join, probe_frozen_positions looks every query up in the frozen store, keeping its hash. + * 4. After that join, the caller checks that base + misses stays below the row-ID ceiling, then assigns miss + * j the ID base + j in ascending sender-window/query order. + * A failure in any phase leaves `op` untouched. The caller runs phase 3 (the scatter), then + * insert_incoming_misses. + * + * \param observer Test seam reporting KernelRange::decode and KernelRange::incoming_probe; see NoRangeObserver. + * \throws MalformedQueryStream for a malformed stream (from the caller before decoding, or from a decode worker + * for invalid positions, rethrown after the join and before probing). + * \throws TermIndexCeilingReached if the misses would pass the row-ID ceiling. + */ +template auto probe_incoming_queries(const mpi::WindowVec &incoming, // serialized, one VecZ per sender slot MPOperator &op, QueryForm form, - [[maybe_unused]] parallel::Options options = {}) -> IncomingProbe { + parallel::Options options = {}, + const Observer &observer = {}) -> IncomingProbe { using QW = QueryWire; using PosT = typename IncomingProbe::PosT; IncomingProbe pr; pr.window = incoming.window(); + pr.base = op.store->size(); const size_t senders = pr.window.count; + if (senders > size_t{std::numeric_limits::max()} + 1U) { + throw std::length_error( + std::format("probe_incoming_queries: {} senders exceed the uint32 sender index", senders)); + } + // 1a. Validate every stream and count its queries and positions, before any decode. + resolve_detail::check_fits(pr.goff, senders + 1, "sender offsets"); pr.goff.assign(senders + 1, 0); + size_t positions = 0; for (size_t k = 0; k < senders; ++k) { - const size_t nq = QW::count_queries(incoming[mpi::WindowIndex{k}], form); + const VecZ &buf = incoming[mpi::WindowIndex{k}]; + size_t nq = 0; + for (size_t off = 0; off < buf.size();) { + const auto ext = QW::checked_extent(buf, form, off); + if (ext.k > std::numeric_limits::max() - positions) { + throw std::length_error("probe_incoming_queries: the decoded position count overflows"); + } + positions += ext.k; + off += ext.words; // checked_extent guarantees off + words <= buf.size() + ++nq; + } + // nq <= buf.size() words, but the running sum over senders is still checked. + if (nq > std::numeric_limits::max() - pr.goff[k]) { + throw std::length_error("probe_incoming_queries: the query count overflows"); + } pr.goff[k + 1] = pr.goff[k] + nq; } pr.nq_total = pr.goff[senders]; - if (pr.nq_total == 0) { - return pr; - } + const size_t n = pr.nq_total; + + // 1b. Allocate every output once, at its exact size; workers never resize. + resolve_detail::check_fits(pr.sender_wi, n, "queries"); + resolve_detail::check_fits(pr.phase_of, n, "queries"); + resolve_detail::check_fits(pr.off_of, n, "queries"); + resolve_detail::check_fits(pr.idx_of, n, "queries"); + resolve_detail::check_fits(pr.pos_off, n, "queries"); + resolve_detail::check_fits(pr.k_of, n, "queries"); + resolve_detail::check_fits(pr.hash_of, n, "queries"); + resolve_detail::check_fits(pr.pos_flat, positions, "positions"); + pr.sender_wi.resize(n); + pr.phase_of.resize(n); + pr.off_of.resize(n); + pr.idx_of.resize(n); + pr.pos_off.resize(n); + pr.k_of.resize(n); + pr.hash_of.resize(n); + pr.pos_flat.resize(positions); - pr.sender_wi.resize(pr.nq_total); + // 1c. Canonical sender-window/query order: record offsets, counts and absolute position offsets. The headers + // were validated above, so reading them again cannot leave the buffer. + size_t pos_at = 0; for (size_t k = 0; k < senders; ++k) { - std::fill(pr.sender_wi.begin() + static_cast(pr.goff[k]), - pr.sender_wi.begin() + static_cast(pr.goff[k + 1]), - static_cast(k)); - } - - // Phase 1 (read-only): deserialize, then probe with the group-prefetch batch find. One walk per sender. - pr.phase_of.resize(pr.nq_total); - pr.off_of.resize(pr.nq_total); - pr.idx_of.resize(pr.nq_total); - pr.pos_off.resize(pr.nq_total); - pr.k_of.resize(pr.nq_total); - pr.hash_of.resize(pr.nq_total); - pr.pos_flat.clear(); - // A hint only, so this stays one allocation for the common case. - pr.pos_flat.reserve(pr.nq_total * QW::kReservePositionsPerQuery); - for (size_t si = 0; si < senders; ++si) { - const VecZ &buf = incoming[mpi::WindowIndex{si}]; + const VecZ &buf = incoming[mpi::WindowIndex{k}]; size_t off = 0; - for (size_t g = pr.goff[si]; g < pr.goff[si + 1]; ++g) { - const size_t k = QW::k_at(buf, off); - const size_t at = pr.pos_flat.size(); - pr.pos_flat.resize(at + k); // default-init grow: read_query writes every element - pr.pos_off[g] = at; - pr.k_of[g] = static_cast(k); + for (size_t g = pr.goff[k]; g < pr.goff[k + 1]; ++g) { + const auto h = QW::header_at(buf, off); + pr.sender_wi[g] = static_cast(k); pr.off_of[g] = off; - const auto d = QW::read_query(buf, form, off, std::span(pr.pos_flat).subspan(at, k)); - pr.phase_of[g] = d.phase; - off = d.next; + pr.k_of[g] = static_cast(h.k); // k <= kMaxPositions <= 65535 + pr.pos_off[g] = pos_at; + pos_at += h.k; + off += QW::words_of_header(h) + (form == QueryForm::Fused ? 1U : 0U); } } - { - const size_t op_size = op.store->size(); - // The vectors are sized to capacity, not to nq_total, so every span is trimmed explicitly. - op.store->find_batch_positions(std::span(pr.pos_flat), - std::span(pr.pos_off).first(pr.nq_total), - std::span(pr.k_of).first(pr.nq_total), - std::span(pr.idx_of).first(pr.nq_total), - std::span(pr.hash_of).first(pr.nq_total)); - for (size_t g = 0; g < pr.nq_total; ++g) { - if (pr.idx_of[g] >= op_size) { // kNotFound is size_t max → also lands here - pr.idx_of[g] = kMissingIndex; + assert(pos_at == positions); + + // 2. Decode disjoint records into their prepared spans. + const size_t blocks = logical_ranges(n, kProbeBlockQueries); + observer.prepare(KernelRange::decode, blocks); + parallel::for_blocks(blocks, options, [&](size_t block) { + observer.visit(KernelRange::decode, block); + const size_t hi = std::min(n, (block + 1) * kProbeBlockQueries); + for (size_t g = block * kProbeBlockQueries; g < hi; ++g) { + const size_t k = pr.sender_wi[g]; + const VecZ &buf = incoming[mpi::WindowIndex{k}]; + const auto d = QW::read_query(buf, + form, + pr.off_of[g], + std::span(pr.pos_flat).subspan(pr.pos_off[g], pr.k_of[g])); + const size_t boundary = (g + 1 < pr.goff[k + 1]) ? pr.off_of[g + 1] : buf.size(); + if (d.next != boundary) { + throw MalformedQueryStream( + std::format("probe_incoming_queries: query {} ends at word {}, not at its prepared boundary {}", + g, + d.next, + boundary)); } + pr.phase_of[g] = d.phase; } - } + }); - // Phase 2 ((sender,query) prefix order): each miss takes the next index base+j. - pr.base = op.store->size(); - for (size_t g = 0; g < pr.nq_total; ++g) { - if (pr.idx_of[g] == kMissingIndex) { - pr.idx_of[g] = pr.base + pr.miss_g.size(); - pr.miss_g.push_back(static_cast(g)); + // 3. Frozen probe: nothing mutates op until insert_incoming_misses. + probe_frozen_positions(*op.store, + std::span(pr.pos_flat), + std::span(pr.pos_off), + std::span(pr.k_of), + std::span(pr.idx_of), + std::span(pr.hash_of), + options, + observer, + KernelRange::incoming_probe); + + // 4. Phase 2 ((sender,query) prefix order): each miss takes the next index base+j. The row-ID limit is + // checked before any predicted ID is written. + size_t n_miss = 0; + for (size_t g = 0; g < n; ++g) { + n_miss += static_cast(pr.idx_of[g] >= pr.base); // kNotFound is size_t max, so it lands here too + } + if (n_miss > OperatorIndex::kIndexCeiling - pr.base) { + throw TermIndexCeilingReached(std::format("probe_incoming_queries: {} new terms on a store of {} would pass " + "the 2^32 TermIndex ceiling", + n_miss, + pr.base)); + } + pr.miss_g.resize(n_miss); + size_t j = 0; + for (size_t g = 0; g < n; ++g) { + if (pr.idx_of[g] >= pr.base) { + pr.idx_of[g] = pr.base + j; + pr.miss_g[j++] = g; } } return pr; } // Phase 4 (bulk insert of the distinct absent terms) into op slots [base, base+n_miss). Call after the -// caller's Phase-3 scatter, which reads pre-insert op_coeffs for hits and needs base == op.size(). +// caller's Phase-3 scatter, which reads pre-insert op_coeffs for hits and needs base == op.size(). Publication +// is single-writer: grow, set_positions, bulk_insert_hashed with the probe's cached hashes (`options` is its +// publication seam, which the packed index ignores), then the inverted-index maintenance. template -auto insert_incoming_misses(MPOperator &op, const IncomingProbe &pr) -> void { +auto insert_incoming_misses(MPOperator &op, const IncomingProbe &pr, parallel::Options options = {}) + -> void { const size_t n_miss = pr.miss_g.size(); if (n_miss == 0) { return; @@ -176,14 +347,59 @@ auto insert_incoming_misses(MPOperator &op, const IncomingProbegrow_rows_geometric(n_miss); + assert(base == pr.base); for (size_t j = 0; j < n_miss; ++j) { const size_t g = pr.miss_g[j]; op.store->set_positions(base + j, pr.positions_at(g)); } - op.store->bulk_insert_hashed(n_miss, base, [&](size_t j) { return pr.hash_of[pr.miss_g[j]]; }); + op.store->bulk_insert_hashed(n_miss, base, [&](size_t j) { return pr.hash_of[pr.miss_g[j]]; }, options); op.reindex_after_growth(base, n_miss); } +/*! + * \brief Whether the leader hits of `pr` (rows below `combined_size`) are pairwise distinct, so their matched + * marks are writes to distinct elements. + * + * Holds in production because queries are source XOR G over globally distinct sources; the parallel scatter + * relies on it and debug builds assert it. O(hits log hits); for assertions and tests only. + */ +template +auto leader_marks_unique(const IncomingProbe &pr, size_t combined_size) -> bool { + std::vector hits; + for (size_t g = 0; g < pr.nq_total; ++g) { + if (pr.idx_of[g] < combined_size) { + hits.push_back(pr.idx_of[g]); + } + } + std::ranges::sort(hits); + return std::ranges::adjacent_find(hits) == hits.end(); +} + +/*! + * \brief Whether `sink` certifies that its on_resolved calls for distinct queries may run concurrently. + * + * A sink opts in with `static constexpr bool parallel_resolve = true`, which certifies only independent + * on_resolved writes after prepare(), never its other methods. A sink may further veto it at run time with + * `parallel_resolve_enabled()`. A missing or false capability keeps the serial scatter. + */ +template +auto sink_resolves_in_parallel(const Sink &sink) -> bool { + if constexpr (requires { Sink::parallel_resolve; }) { + if constexpr (static_cast(Sink::parallel_resolve)) { + if constexpr (requires { + { sink.parallel_resolve_enabled() } -> std::convertible_to; + }) { + return sink.parallel_resolve_enabled(); + } + else { + return true; + } + } + } + (void)sink; + return false; +} + // resolve_incoming / process_responses are the picture-independent cross-rank exchange skeletons; what // each resolved query records and answers with is supplied by a compile-time sink. The two concrete sinks // (GraphSink / ContractSink) live in Engine.h, each also carrying the self-resolve + finalize policy. @@ -191,39 +407,64 @@ auto insert_incoming_misses(MPOperator &op, const IncomingProbe +// Threading: decode and probe as probe_incoming_queries; the scatter runs on workers only for a sink that +// certifies it (sink_resolves_in_parallel), otherwise on the caller; publication stays on the caller, after +// the scatter, which reads pre-insert coefficients. `observer` is the test seam (see NoRangeObserver). +template auto resolve_incoming(const mpi::WindowVec &incoming, // serialized, one VecZ per sender slot MPOperator &op, bool is_leader_pass, MatchedEpochSet &matched, size_t combined_size, // pre-layer op size: bounds the matched set Sink &sink, - parallel::Options options = {}) -> mpi::WindowVec> { + parallel::Options options = {}, + const Observer &observer = {}) -> mpi::WindowVec> { using Resp = typename Sink::Response; - const IncomingProbe pr = probe_incoming_queries(incoming, op, sink.incoming_form(), options); + const IncomingProbe pr = + probe_incoming_queries(incoming, op, sink.incoming_form(), options, observer); // The pairing is an XOR involution, so the response window is the query window. mpi::WindowVec> responses(pr.window); for (const auto wi : pr.window.indices()) { responses[wi].assign(pr.goff[wi.value + 1] - pr.goff[wi.value], Sink::init_response()); } if (pr.nq_total == 0) { + observer.prepare(KernelRange::scatter, 0); return responses; } // Phase 3 (scatter): responses + sink records + matched-follower marks. Freshly inserted partners - // (ip ≥ combined_size) skip the mark. + // (ip ≥ combined_size) skip the mark. prepare() sizes every sink container on the caller first. sink.prepare(pr, op, responses); - for (size_t g = 0; g < pr.nq_total; ++g) { - const mpi::WindowIndex s = pr.sender_index(g); - const size_t q = g - pr.goff[s.value]; - const size_t ip = pr.idx_of[g]; - responses[s][q] = sink.on_resolved(g, s, q, ip, pr, incoming); - if (is_leader_pass && ip < combined_size) { - matched.mark(ip); + const size_t n = pr.nq_total; + const size_t blocks = logical_ranges(n, kResolveScatterBlock); + const auto scatter = [&](size_t block) { + observer.visit(KernelRange::scatter, block); + const size_t hi = std::min(n, (block + 1) * kResolveScatterBlock); + for (size_t g = block * kResolveScatterBlock; g < hi; ++g) { + const mpi::WindowIndex s = pr.sender_index(g); + const size_t q = g - pr.goff[s.value]; + const size_t ip = pr.idx_of[g]; + responses[s][q] = sink.on_resolved(g, s, q, ip, pr, incoming); + if (is_leader_pass && ip < combined_size) { + matched.mark(ip); + } + } + }; + observer.prepare(KernelRange::scatter, blocks); + if (sink_resolves_in_parallel(sink)) { + // Each query writes its own response and its own sink record; a leader hit marks its own row, distinct + // across queries (leader_marks_unique), and misses (ip >= base >= combined_size) mark nothing. The + // marks are distinct uint16 elements, so no synchronization; never paper over a duplicate with atomics. + assert(!is_leader_pass || leader_marks_unique(pr, combined_size)); + parallel::for_blocks(blocks, options, scatter); + } + else { + for (size_t block = 0; block < blocks; ++block) { + scatter(block); } } - insert_incoming_misses(op, pr); + insert_incoming_misses(op, pr, options); return responses; } diff --git a/cpp/monoprop/detail/operator/OperatorIndex.h b/cpp/monoprop/detail/operator/OperatorIndex.h index 01e23cdf..b1807a4c 100644 --- a/cpp/monoprop/detail/operator/OperatorIndex.h +++ b/cpp/monoprop/detail/operator/OperatorIndex.h @@ -32,6 +32,7 @@ #include "monoprop/TypeAliases.h" #include "monoprop/core/Monomial.h" +#include "monoprop/detail/parallel/Options.h" namespace monoprop::detail { @@ -44,7 +45,8 @@ class TermIndexCeilingReached : public std::runtime_error { // those rows. Row layout: slot 0 = popcount c (or kOverflowMarker if c > inline_width_), slots 1..c = // ascending set-bit positions; stride_ is fixed for the container's life so row offsets stay stable. // inline_width_ is a free parameter -- any width is correct, over-long rows spill losslessly to overflow. -// Single-writer: one partition, one thread; parallelism is cross-partition. +// Single-writer: every mutation (growth, set, overflow, table insertion) runs on one thread. The const lookups +// (find, find_batch, find_batch_positions) may run concurrently only while nothing mutates the store. template class OperatorIndex { public: @@ -363,8 +365,14 @@ class OperatorIndex { // bulk_insert with the hashes already in hand: same precondition (n distinct rows, already written, // at consecutive indices) and the same slot assignment. `hashes[k]` must be fold_hash of the key of // row base+k -- a wrong one leaves the row unfindable, which surfaces later as a duplicate insert. + // Neither this nor bulk_insert deduplicates: two rows with one key are both indexed and find() returns + // the first (pinned by operator_index_tests). `options` is the publication seam an alternative index may + // use to publish in parallel; this packed table ignores it and always inserts serially, in order. template - auto bulk_insert_hashed(size_t n, mapped_type base, HashFn &&hash_at) -> void { + auto bulk_insert_hashed(size_t n, + mapped_type base, + HashFn &&hash_at, + [[maybe_unused]] parallel::Options options = {}) -> void { if (n == 0) { return; } diff --git a/cpp/tests/CMakeLists.txt b/cpp/tests/CMakeLists.txt index e599919a..09bdf2ef 100644 --- a/cpp/tests/CMakeLists.txt +++ b/cpp/tests/CMakeLists.txt @@ -145,11 +145,11 @@ monoprop_add_launch_env_case( openmp_propagator_work_does_not_change_affinity ENVIRONMENT "OMP_PROC_BIND=false" ) -# The threaded cosine and fused-apply kernels under runtime-limited and dynamic teams: exact results and +# The threaded cosine, fused-apply, scan and resolve kernels under runtime-limited and dynamic teams: exact results and # participation must not depend on receiving the full budget. set( _monoprop_kernel_cases - "openmp_cos_mask_exact,openmp_cos_mask_workers_participate,openmp_cos_lazy_workers_participate,openmp_fused_apply_exact_on_disjoint_records,openmp_fused_apply_workers_participate,openmp_kernel_worker_failures_join_before_rethrow,combined_threaded_lazy_fold_matches_oracle,fused_real_records_have_one_add_owner_and_apply_exactly,fused_threaded_prototype_matches_serial_prototype_exactly,openmp_scan_ranges_match_serial_majorana,openmp_scan_workers_participate,openmp_scan_worker_failure_joins_before_rethrow,openmp_scan_prepares_cold_caches_on_the_caller,openmp_scan_pauli_ranges_match_serial,openmp_scan_threaded_construction_matches_serial_exactly,openmp_scan_construction_workers_participate" + "openmp_cos_mask_exact,openmp_cos_mask_workers_participate,openmp_cos_lazy_workers_participate,openmp_fused_apply_exact_on_disjoint_records,openmp_fused_apply_workers_participate,openmp_kernel_worker_failures_join_before_rethrow,combined_threaded_lazy_fold_matches_oracle,fused_real_records_have_one_add_owner_and_apply_exactly,fused_threaded_prototype_matches_serial_prototype_exactly,openmp_scan_ranges_match_serial_majorana,openmp_scan_workers_participate,openmp_scan_worker_failure_joins_before_rethrow,openmp_scan_prepares_cold_caches_on_the_caller,openmp_scan_pauli_ranges_match_serial,openmp_scan_threaded_construction_matches_serial_exactly,openmp_scan_construction_workers_participate,openmp_probe_frozen_positions_matches_one_batch,openmp_probe_frozen_positions_workers_participate,openmp_incoming_probe_threaded_matches_serial,openmp_incoming_decode_and_probe_workers_participate,openmp_incoming_decode_worker_failure_joins_before_probing,openmp_resolve_scatter_threaded_matches_serial,openmp_resolve_scatter_workers_participate,openmp_self_windows_match_serial,openmp_mixed_self_and_remote_publication_order,openmp_self_probe_construction_workers_participate" ) monoprop_add_launch_env_case( openmp_env_kernels_thread_limit_2 @@ -223,6 +223,19 @@ if(monoprop_ENABLE_MPI AND _monoprop_mpiexec) monoprop_add_failure_scenario( before_exchange_scan_worker before-exchange scan-worker "scan range 1 on OpenMP worker 1" "UNGUARDED|NOT REACHED" ) + # Resolve workers: the self probe fails before the query exchange; the incoming decode, frozen probe and + # scatter fail after the query round while the peers enter the response round. Worker 1 owns the last range. + foreach(_phase self-probe decode incoming-probe scatter) + string( + REPLACE "-" + "_" + _phase_name + "${_phase}" + ) + monoprop_add_failure_scenario( + resolve_worker_${_phase_name} resolve-worker ${_phase} "${_phase} range [0-9]+ on OpenMP worker 1" "UNGUARDED|NOT REACHED" + ) + endforeach() monoprop_add_failure_scenario( active_ticket active-ticket ticket "injected failure while the replay Ticket" "UNGUARDED" ) diff --git a/cpp/tests/KernelTestSupport.h b/cpp/tests/KernelTestSupport.h index 983daf96..d031043b 100644 --- a/cpp/tests/KernelTestSupport.h +++ b/cpp/tests/KernelTestSupport.h @@ -46,7 +46,7 @@ namespace kernel_test { using monoprop::detail::KernelRange; -inline constexpr size_t kKinds = 5; +inline constexpr size_t kKinds = 9; // What one kernel invocation reported for one kind of range. struct RangeSlots { @@ -89,6 +89,61 @@ struct RecordingObserver { } }; +// Keeps every prepare() call, for kernels that run once per window or once per exchange pass (the self probe, +// the incoming decode/probe/scatter): prepare appends a fresh record on the calling thread, and each range's +// owner writes only its own slots of the newest record. Records are inspected after the kernel has joined. +struct PhaseLog { + struct Call { + KernelRange kind; + std::vector worker; // -1 if never visited + std::vector team; + std::vector level; + std::vector visits; + }; + std::vector calls; + + //! The calls of one kind, in order. + [[nodiscard]] auto of(KernelRange kind) const -> std::vector { + std::vector out; + for (const auto &c : calls) { + if (c.kind == kind) { + out.push_back(&c); + } + } + return out; + } +}; + +struct AccumulatingObserver { + PhaseLog *log; + KernelRange throw_kind = KernelRange::cos_lazy; + size_t throw_call = SIZE_MAX; // which call of throw_kind, counted from zero + size_t throw_range = SIZE_MAX; + + auto prepare(KernelRange kind, size_t ranges) const -> void { + log->calls.push_back({kind, + std::vector(ranges, -1), + std::vector(ranges, 0), + std::vector(ranges, 0), + std::vector(ranges, 0)}); + } + auto visit(KernelRange kind, size_t range) const -> void { + auto &c = log->calls.back(); + c.worker[range] = omp_get_thread_num(); + c.team[range] = omp_get_num_threads(); + c.level[range] = omp_get_level(); + ++c.visits[range]; + if (kind == throw_kind && range == throw_range && log->of(kind).size() == throw_call + 1) { + throw std::runtime_error("injected resolve worker failure"); + } + } +}; + +// A PhaseLog call as RangeSlots, so the participation/serial checks below apply to it. +inline auto slots_of(const PhaseLog::Call &c) -> RangeSlots { + return RangeSlots{.prepares = 1, .worker = c.worker, .team = c.team, .level = c.level, .visits = c.visits}; +} + // Evidence that one kernel's own ranges ran on several workers of a region the kernel opened. struct Participation { size_t ranges = 0; diff --git a/cpp/tests/README.md b/cpp/tests/README.md index e9aa9019..74f9badf 100644 --- a/cpp/tests/README.md +++ b/cpp/tests/README.md @@ -160,12 +160,26 @@ name and cannot address suite-nested cases, tests use flat and basis-change cutoffs stay serial), `sparse_query_tests.cpp` (the merge itself) and `fused_cos_sweep_tests.cpp` (full construction: rows, row IDs, graph layers and coefficients at budgets 1-4). The scan's participation is read from its own ranges through the same observer. +- **Threaded resolution**: `sparse_resolve_tests.cpp` compares the incoming probe (checked + preparation, decode, `probe_frozen_positions`, predicted IDs, publication) at budgets 2-8 with + budget one over empty/all-hit/all-miss/mixed inputs, 255/256/257-query boundaries, empty senders, + non-zero window bases, narrow and wide positions, spilled rows, escaped counts, Plain and Fused + records and a prebuilt inverted index. It rejects malformed streams (including the noncanonical + escape `QueryWire<128>` `0x8037e`) before any decode, invalid decoded positions before any probe, + keeps the scatter serial for sinks without `parallel_resolve` and for the Schrödinger + `ContractSink`, and checks self windows of 4096 and the publication order across leader/follower + passes. `sparse_query_tests.cpp` covers `QueryWire::checked_extent` and position validation, + `operator_index_tests.cpp` pins the duplicate-key behaviour of bulk insertion, and + `mpi_fresh_insert_equivalence.cpp` compares budgets exactly and reads participation at world size. + `PhaseLog`/`AccumulatingObserver` keep every call of a phase that runs per window or per pass. - **Distributed failure driver**: `mpi_failure_driver.cpp` is not part of the unit runner (the glob excludes it). The MPI build compiles it into `monoprop_mpi_failure_driver.x`, and `run_mpi_failure_scenario.cmake` runs each scenario on two ranks with a 30 s timeout, as the `mpi_failure_*` CTest entries. `active-ticket cosine-worker` throws from a worker of the threaded cosine kernel while a replay Ticket is posted; `before-exchange scan-worker` throws from a worker - of the threaded scan while the peer enters the query exchange. + of the threaded scan while the peer enters the query exchange. `resolve-worker self-probe` throws + from the self probe before the query exchange, and `resolve-worker decode|incoming-probe|scatter` + from the incoming phases after the query round, while the peer enters the response round. New `*.cpp` files are auto-discovered on the next configure, with no CMake edit needed. A file with its own `main()` must be excluded from the glob, as diff --git a/cpp/tests/fused_cos_sweep_tests.cpp b/cpp/tests/fused_cos_sweep_tests.cpp index 360ee548..3e3256f9 100644 --- a/cpp/tests/fused_cos_sweep_tests.cpp +++ b/cpp/tests/fused_cos_sweep_tests.cpp @@ -604,3 +604,42 @@ BOOST_AUTO_TEST_CASE(openmp_scan_worker_failure_invalidates_the_propagator) { BOOST_CHECK_THROW(sim.build_graph(data.majoranas, data.param_inds, data.gen_coeffs), InvalidPropagatorError); BOOST_CHECK_THROW((void)WideAccess::clone(sim), InvalidPropagatorError); } + +// The real build_layer on one rank resolves every partner itself: its self probe runs in bounded windows whose +// 256-query blocks run inside the probe's own region. +BOOST_AUTO_TEST_CASE(openmp_self_probe_construction_workers_participate, + *boost::unit_test::precondition(kernel_test::runtime_offers_two_workers)) { + const auto data = wide_case(); + auto sim = wide_prototype(data, false, 4, MPI_COMM_SELF); + kernel_test::PhaseLog log; + (void)WideAccess::build_layer_observed(sim, data.majoranas[0], kernel_test::AccumulatingObserver{&log}); + const auto calls = log.of(monoprop::detail::KernelRange::self_probe); + BOOST_TEST_REQUIRE(!calls.empty()); + const auto *widest = *std::ranges::max_element(calls, {}, [](const auto *c) { return c->worker.size(); }); + BOOST_TEST_REQUIRE(widest->worker.size() >= 4U); + kernel_test::check_participation(kernel_test::slots_of(*widest), widest->worker.size(), "build_layer self probe"); + for (const auto *c : calls) { + BOOST_TEST(c->worker.size() <= 16U); + } + BOOST_TEST(log.of(monoprop::detail::KernelRange::decode).empty()); // one rank: nothing crosses the wire + BOOST_TEST(!WideAccess::is_invalid(sim)); +} + +// A self-probe worker's failure inside the real build path is joined and rethrown before any self miss is +// published; the propagator's operation guard then invalidates the object. +BOOST_AUTO_TEST_CASE(openmp_self_probe_worker_failure_invalidates_the_propagator) { + const auto data = wide_case(); + auto sim = wide_prototype(data, false, 3, MPI_COMM_SELF); + kernel_test::PhaseLog log; + BOOST_CHECK_THROW((void)WideAccess::build_layer_observed( + sim, + data.majoranas[0], + kernel_test::AccumulatingObserver{&log, monoprop::detail::KernelRange::self_probe, 0, 1}), + std::runtime_error); + const auto calls = log.of(monoprop::detail::KernelRange::self_probe); + BOOST_TEST_REQUIRE(calls.size() == 1U); + BOOST_TEST(calls[0]->visits.at(1) == 1); + BOOST_TEST(WideAccess::is_invalid(sim)); + BOOST_CHECK_THROW(sim.build_graph(data.majoranas, data.param_inds, data.gen_coeffs), InvalidPropagatorError); + BOOST_CHECK_THROW((void)WideAccess::clone(sim), InvalidPropagatorError); +} diff --git a/cpp/tests/mpi_failure_driver.cpp b/cpp/tests/mpi_failure_driver.cpp index 1975d6a5..19513b99 100644 --- a/cpp/tests/mpi_failure_driver.cpp +++ b/cpp/tests/mpi_failure_driver.cpp @@ -25,6 +25,11 @@ // before-exchange a build fails in the scan, before the first query exchange // before-exchange scan-worker a worker of the threaded scan throws in one of its word ranges, before // the first query exchange the peers enter +// resolve-worker self-probe a worker of the threaded self probe throws, before the first query +// exchange the peers enter +// resolve-worker decode|incoming-probe|scatter +// a worker of the threaded incoming decode / frozen probe / scatter throws +// after the query round, while the peers enter the response round // active-ticket ticket|pending a failure while a replay Ticket / construction PendingAlltoallv is posted // active-ticket cosine-worker a worker of the threaded cosine kernel throws while the replay Ticket is // posted @@ -284,6 +289,72 @@ auto wrong_thread(int rank) -> void { } } +// Throws from the last logical range of one resolve phase on rank 0, so with a two-worker team the failure +// starts on worker 1, off the calling thread. Only the first call of that phase with two or more ranges throws. +struct ThrowingResolveObserver { + int rank; + detail::KernelRange kind; + const char *label; + std::shared_ptr ranges = std::make_shared(0); + + auto prepare(detail::KernelRange k, size_t n) const noexcept -> void { + if (k == kind && *ranges < 2) { + *ranges = n; + } + } + auto visit(detail::KernelRange k, size_t range) const -> void { + if (rank == 0 && k == kind && *ranges >= 2 && range + 1 == *ranges) { + throw std::runtime_error( + std::format("injected failure in {} range {} on OpenMP worker {}", label, range, omp_get_thread_num())); + } + } +}; + +auto resolve_worker(int rank, std::string_view phase) -> void { + OperatorDict ham; + uint64_t state = 0x5CA3ULL; + while (ham.size() < 200000) { + VecZ idx; + while (idx.size() < 6) { + state = (state * 6364136223846793005ULL) + 1442695040888963407ULL; + const size_t m = (state >> 33) % (2 * kWideModes); + if (std::ranges::find(idx, m) == idx.end()) { + idx.push_back(m); + } + } + std::ranges::sort(idx); + ham[idx] = std::complex{0.0, 1e-3}; + } + ::setenv("monoprop_NUM_THREADS", "2", 1); + MonomialPropagator sim(ham, + 2 * kWideModes, + VecZ{0, 1}, + std::nullopt, + MPI_COMM_WORLD, + std::nullopt, + std::nullopt, + CutoffType::Length, + std::nullopt, + kWideModes, + Basis::Majorana, + /*partitions=*/1); + const auto kind = phase == "self-probe" ? detail::KernelRange::self_probe + : phase == "decode" ? detail::KernelRange::decode + : phase == "scatter" ? detail::KernelRange::scatter + : detail::KernelRange::incoming_probe; + say(rank, + std::format("{} local terms, budget {}", + sim.size(), + detail::PropagatorTestAccess::options(sim).threads)); + (void)detail::PropagatorTestAccess::build_layer_observed( + sim, + VecZ{0, 5}, + ThrowingResolveObserver{rank, kind, phase.data()}); + if (rank == 0) { + say(rank, std::format("INJECTION NOT REACHED: the {} phase ran in fewer than two ranges", phase)); + } +} + auto parse_level(std::string_view name) -> std::optional { if (name == "single") { return MPI_THREAD_SINGLE; @@ -356,6 +427,11 @@ auto main(int argc, char **argv) -> int { else if (scenario == "before-exchange" && option == "scan-worker") { scan_worker(rank); } + else if (scenario == "resolve-worker" + && (option == "self-probe" || option == "decode" || option == "incoming-probe" + || option == "scatter")) { + resolve_worker(rank, option); + } else if (scenario == "before-exchange") { before_exchange(rank); } diff --git a/cpp/tests/mpi_fresh_insert_equivalence.cpp b/cpp/tests/mpi_fresh_insert_equivalence.cpp index 2973131b..0328586b 100644 --- a/cpp/tests/mpi_fresh_insert_equivalence.cpp +++ b/cpp/tests/mpi_fresh_insert_equivalence.cpp @@ -20,14 +20,20 @@ #include +#include +#include #include +#include #include #include #include +#include "KernelTestSupport.h" #include "PauliTestOracle.h" +#include "PropagatorTestAccess.h" #include "TestUtilities.h" #include "monoprop/MonomialPropagator.h" +#include "monoprop/algebra/MajoranaAlgebra.h" #include "monoprop/detail/mpi/MPICompat.h" namespace { @@ -39,7 +45,7 @@ using pauli_oracle::slots_of_string; // schrodinger_cutoff engages the picture; a low structural cutoff plus the upper_atol = 0 rescue // forces most partners to be fresh inserts, so the miss arm runs on nearly every partner. template -auto run_schrodinger_majorana(const CaseData& data, MPI_Comm comm) -> double { +auto run_schrodinger_majorana(const CaseData &data, MPI_Comm comm) -> double { MonomialPropagator sim(data.hamiltonian, /*cutoff=*/2U, data.initial_state, @@ -118,4 +124,152 @@ BOOST_AUTO_TEST_CASE(mpi_fresh_insert_schrodinger_pauli_serial_world_equiv) { BOOST_TEST(near(e_serial, e_world)); } +// ── Threaded resolution at fixed geometry (Task 6) ──────────────────────────────────────────────── +// +// The one-store prototype (explicit partitions=1 on an ordinary communicator) at launch budgets 2-4 against +// budget 1, at this launch's world size: every rank's rows in row-ID order and its coefficients bit for bit. +// The Schrodinger fresh-insert arm keeps its scatter serial (state scoring), while the probe, decode and +// publication around it are the threaded ones; Heisenberg graph builds use the parallel scatter. + +template +auto same_store(const MonomialPropagator &a, const MonomialPropagator &b) -> bool { + const auto &sa = *a.mp_op().store; + const auto &sb = *b.mp_op().store; + if (sa.size() != sb.size()) { + return false; + } + for (size_t i = 0; i < sa.size(); ++i) { + if (materialize_row(sa, i) != materialize_row(sb, i)) { + return false; + } + } + return true; +} + +auto same_bits(const VecD &a, const VecD &b) -> bool { + return a.size() == b.size() && (a.empty() || std::memcmp(a.data(), b.data(), a.size() * sizeof(double)) == 0); +} + +template +auto prototype(const OperatorDict &ham, + unsigned cutoff, + const VecZ &state, + std::optional schrodinger_cutoff, + std::optional upper_atol, + int budget, + Basis basis = Basis::Majorana, + CutoffType cutoff_type = CutoffType::Length) -> MonomialPropagator { + const kernel_test::ScopedBudget scoped(budget); + MonomialPropagator sim(ham, + cutoff, + state, + schrodinger_cutoff, + MPI_COMM_WORLD, + std::nullopt, + upper_atol, + cutoff_type, + std::nullopt, + NumModes, + basis, + /*partitions=*/1); + BOOST_TEST_REQUIRE(monoprop::detail::PropagatorTestAccess::options(sim).threads == budget); + return sim; +} + +BOOST_FIXTURE_TEST_CASE(openmp_fresh_insert_threaded_budgets_match_serial_exactly, ExampleDataFix) { + constexpr size_t N = ExampleDataFix::n_modes; + using Access = monoprop::detail::PropagatorTestAccess; + const auto run = [&](int budget, bool schrodinger) { + auto sim = prototype(data.hamiltonian, + schrodinger ? 2U : 2U * N, + data.initial_state, + schrodinger ? std::optional{4U} : std::nullopt, + std::optional{0.0}, + budget); + sim.propagate(data.majoranas, data.param_inds, data.gen_coeffs, data.parameters); + return sim; + }; + for (const bool schrodinger : {false, true}) { + auto serial = run(1, schrodinger); + const double e_serial = serial.expectation_value({}); + for (const int budget : {2, 3, 4}) { + BOOST_TEST_CONTEXT("schrodinger=" << schrodinger << " budget=" << budget + << " world=" << mpi::size(MPI_COMM_WORLD)) { + auto threaded = run(budget, schrodinger); + BOOST_TEST(same_store(serial, threaded)); + BOOST_TEST(same_bits(Access::picture_coeffs(serial), Access::picture_coeffs(threaded))); + BOOST_TEST(std::bit_cast(threaded.expectation_value({})) + == std::bit_cast(e_serial)); + } + } + } +} + +// A 16-mode operator large enough that every rank receives thousands of remote queries per pass. +constexpr size_t kWideModes = 16; + +auto wide_hamiltonian() -> OperatorDict { + OperatorDict ham; + kernel_test::SplitMix rng{0x7A5C6ULL}; + while (ham.size() < 90000) { + VecZ idx; + const size_t weight = 2 * (1 + rng.below(3)); + while (idx.size() < weight) { + const size_t m = rng.below(2 * kWideModes); + if (std::ranges::find(idx, m) == idx.end()) { + idx.push_back(m); + } + } + std::ranges::sort(idx); + const double x = static_cast(rng.below(1U << 20)) / (1U << 20) - 0.5; + ham[idx] = x * hermitian_coefficient(indices_to_bitset(idx)); + } + return ham; +} + +// Under an MPI launch the real build_layer's incoming decode, frozen probe and GraphSink scatter each run in +// their own region with several workers. Linear routing sends some generators only to self, so the case walks +// generators until one crosses the wire. Serial ctest (world 1) has no incoming queries and skips. +BOOST_AUTO_TEST_CASE(openmp_incoming_resolution_workers_participate_at_world_size, + *boost::unit_test::precondition(kernel_test::runtime_offers_two_workers)) { + if (mpi::size(MPI_COMM_WORLD) < 2) { + BOOST_TEST_MESSAGE("Skipping incoming-resolution participation (world size = 1)."); + return; + } + using Access = monoprop::detail::PropagatorTestAccess; + const auto ham = wide_hamiltonian(); + auto sim = prototype(ham, 10U, VecZ{0, 2, 4, 6}, std::nullopt, std::nullopt, 4); + const std::vector gens{{0, 5}, {1, 2}, {3, 9, 12, 20}, {4, 7}, {6, 11}, {8, 13, 17, 30}}; + bool observed = false; + for (const auto &gen : gens) { + kernel_test::PhaseLog log; + (void)Access::build_layer_observed(sim, gen, kernel_test::AccumulatingObserver{&log}); + const auto decode = log.of(monoprop::detail::KernelRange::decode); + const auto probe = log.of(monoprop::detail::KernelRange::incoming_probe); + const auto scatter = log.of(monoprop::detail::KernelRange::scatter); + // Both passes run the three phases, in the same order, whenever this generator crosses ranks. + BOOST_TEST(decode.size() == probe.size()); + BOOST_TEST(decode.size() == scatter.size()); + for (size_t pass = 0; pass < decode.size(); ++pass) { + if (decode[pass]->worker.size() >= 4 && scatter[pass]->worker.size() >= 4) { + kernel_test::check_participation(kernel_test::slots_of(*decode[pass]), + decode[pass]->worker.size(), + "decode"); + kernel_test::check_participation(kernel_test::slots_of(*probe[pass]), + probe[pass]->worker.size(), + "incoming probe"); + kernel_test::check_participation(kernel_test::slots_of(*scatter[pass]), + scatter[pass]->worker.size(), + "scatter"); + observed = true; + } + } + if (observed) { + break; + } + } + BOOST_TEST(observed); + BOOST_TEST(!Access::is_invalid(sim)); +} + } // namespace diff --git a/cpp/tests/operator_index_tests.cpp b/cpp/tests/operator_index_tests.cpp index 73a08165..1650435c 100644 --- a/cpp/tests/operator_index_tests.cpp +++ b/cpp/tests/operator_index_tests.cpp @@ -14,9 +14,11 @@ #include +#include #include #include #include +#include #include #include @@ -24,6 +26,7 @@ #include "monoprop/algebra/MajoranaAlgebra.h" // indices_to_bitset #include "monoprop/core/Monomial.h" #include "monoprop/detail/operator/OperatorIndex.h" +#include "monoprop/detail/parallel/Options.h" using namespace monoprop; using namespace monoprop::detail; @@ -222,3 +225,95 @@ BOOST_AUTO_TEST_CASE(find_batch_on_empty_store_is_all_missing) { BOOST_TEST(out[i] == Store::kNotFound); } } + +// Shared invariant 14, pinned: bulk insertion does not deduplicate. Two rows holding the same key are both +// indexed, and a lookup returns the first one inserted (it sits earlier on the probe chain). Production never +// relies on this -- query keys are distinct by construction -- but a replacement index must either keep it or +// change it as an explicit, reviewed contract change. The options overload is the Task 11 publication seam; +// the packed index ignores it and publishes serially, identically. +BOOST_AUTO_TEST_CASE(bulk_insert_duplicate_keys_are_both_indexed) { + const auto dup = bs({2, 7, 9}); + const auto other = bs({1, 4}); + const std::array keys{dup, other, dup}; + for (const int threads : {1, 4}) { + for (const bool hashed : {false, true}) { + Store s; + const size_t base = s.grow_rows_geometric(keys.size()); + for (size_t k = 0; k < keys.size(); ++k) { + s.set(base + k, keys[k]); + } + if (hashed) { + s.bulk_insert_hashed( + keys.size(), + base, + [&](size_t k) { + std::vector pos; + s.for_each_position(base + k, [&](size_t b) { pos.push_back(static_cast(b)); }); + return Store::fold_hash_positions(pos); + }, + parallel::Options{.threads = threads}); + } + else { + s.bulk_insert(keys.size(), base, [&](size_t k) -> const MSet & { return keys[k]; }); + } + BOOST_TEST_CONTEXT("threads=" << threads << " hashed=" << hashed) { + BOOST_TEST(s.size() == 3u); + size_t indexed = 0; + std::vector rows_for_dup; + s.for_each([&](const MSet &key, size_t row) { + ++indexed; + if (key == dup) { + rows_for_dup.push_back(row); + } + }); + BOOST_TEST(indexed == 3u); + std::ranges::sort(rows_for_dup); + BOOST_TEST((rows_for_dup == std::vector{0, 2})); + BOOST_REQUIRE(s.find(dup).has_value()); + BOOST_TEST(*s.find(dup) == 0u); + BOOST_TEST(*s.find(other) == 1u); + } + } + } +} + +// The frozen probe slices one shared position buffer into blocks: absolute offsets, sliced counts and outputs. +// A sliced call must give exactly what the whole call gives for the same queries, hashes included. +BOOST_AUTO_TEST_CASE(find_batch_positions_on_sliced_spans_matches_the_whole_call) { + Store s; + for (size_t i = 0; i < 120; ++i) { + const auto key = bs({i / 60, 4 + (i % 60)}); + s.push_back(key); + s.emplace(key, i); + } + std::vector flat{63, 63, 63}; // unreferenced prefix: offsets are absolute + std::vector off; + std::vector k_of; + for (size_t i = 0; i < 200; ++i) { + const auto key = (i % 3 == 0) ? bs({0, 1, 2 + (i % 20)}) : bs({(i / 60) % 2, 4 + (i % 60)}); + off.push_back(flat.size()); + uint32_t k = 0; + for (size_t b = key.find_first(); b < key.size(); b = key.find_next(b)) { + flat.push_back(static_cast(b)); + ++k; + } + k_of.push_back(k); + } + std::vector whole(off.size()); + std::vector whole_hash(off.size()); + s.find_batch_positions(flat, off, k_of, whole, whole_hash); + std::vector sliced(off.size(), 7); + std::vector sliced_hash(off.size(), 7); + for (size_t lo = 0; lo < off.size(); lo += 37) { + const size_t m = std::min(37, off.size() - lo); + s.find_batch_positions(flat, + std::span(off).subspan(lo, m), + std::span(k_of).subspan(lo, m), + std::span(sliced).subspan(lo, m), + std::span(sliced_hash).subspan(lo, m)); + } + BOOST_TEST(sliced == whole, boost::test_tools::per_element()); + BOOST_TEST(sliced_hash == whole_hash, boost::test_tools::per_element()); + BOOST_TEST(std::ranges::count(whole, Store::kNotFound) > 0); + BOOST_TEST(std::ranges::count_if(whole, [](size_t v) { return v != Store::kNotFound; }) > 0); +} diff --git a/cpp/tests/sparse_query_tests.cpp b/cpp/tests/sparse_query_tests.cpp index 33ac4b10..0a1821ef 100644 --- a/cpp/tests/sparse_query_tests.cpp +++ b/cpp/tests/sparse_query_tests.cpp @@ -22,7 +22,9 @@ #include #include #include +#include #include +#include #include #include "monoprop/detail/evolution/layer_build/Common.h" @@ -764,3 +766,148 @@ BOOST_AUTO_TEST_CASE(openmp_scan_merge_moves_the_first_useful_buffer) { BOOST_TEST(merged.leader_self.size() == 3U); BOOST_TEST(merged.leader_self.pos_flat.data() == stage_data); } + +// --- checked record boundaries and decoded-position validation (resolver input is untrusted) --------------- + +namespace { + +// A single hand-packed header word: phase field, k field (and the wide-k field when escaped), gw field. +template +auto header_word(uint64_t phase_field, uint64_t k_field, std::optional long_k, uint64_t gw) -> size_t { + using QW = QueryWire; + uint64_t w = phase_field | (k_field << QW::kPhaseBits); + size_t at = QW::kPhaseBits + QW::kKBits; + if (long_k) { + w |= *long_k << at; + at += QW::kLongKBits; + } + w |= gw << at; + return static_cast(w); +} + +template +auto expect_malformed(const VecZ &buf, QueryForm form, size_t off, const char *label) -> void { + BOOST_TEST_CONTEXT(label) { + BOOST_CHECK_THROW((void)QueryWire::checked_extent(buf, form, off), MalformedQueryStream); + } +} + +} // namespace + +// Every record push() can produce passes the check, which reports exactly the words the decoder consumes, plus +// the fused value word. Covers the whole (k, gw) surface at 128 modes, the escape boundary and full width. +BOOST_AUTO_TEST_CASE(sparse_record_checked_extent_accepts_every_canonical_record) { + const auto check = [](std::integral_constant, const std::vector &pos, int phase) { + using QW = QueryWire; + for (const auto form : {QueryForm::Plain, QueryForm::Fused}) { + VecZ buf; + const size_t words = QW::push(buf, pos, phase); + if (form == QueryForm::Fused) { + QW::push_value(buf, -0.25); + } + const auto ext = QW::checked_extent(buf, form, 0); + BOOST_TEST(ext.k == pos.size()); + BOOST_TEST(ext.words == buf.size()); + BOOST_TEST(ext.words == words + (form == QueryForm::Fused ? 1U : 0U)); + std::vector out(pos.size()); + const auto d = QW::read_query(buf, form, 0, out); + BOOST_TEST(d.next == ext.words); + BOOST_TEST(out == pos, boost::test_tools::per_element()); + } + }; + size_t cells = 0; + for (size_t k = 0; k <= 40U; ++k) { + for (size_t gw = 0; gw <= QueryWire<128>::kPosBits; ++gw) { + const auto pos = gap_shaped(k, gw, QueryWire<128>::kBits); + if (pos.size() != k || (k < 2U && gw > 0U)) { + continue; + } + check(std::integral_constant{}, pos, static_cast(k % 3U) - 1); + ++cells; + } + } + BOOST_TEST(cells == 353U); + for (const size_t k : {size_t{30}, size_t{31}, size_t{32}, size_t{200}, size_t{2048}}) { + check(std::integral_constant{}, strided(k, 0, k == 2048 ? 1 : 3, 2048), 1); + } + check(std::integral_constant{}, std::vector{0, 257, 499}, -1); +} + +// The plan's fixture: one Plain word at 128 modes with phase +1, an escaped k = 6, gw = 8 and no payload. The +// decoder would consume 20 header bits + 8 + 5*8 = 68 bits (two words), but the canonical k = 6 header is 11 +// bits, so the unchecked size calculation claims 59 bits, one word, and the decode would read past the buffer. +BOOST_AUTO_TEST_CASE(sparse_record_checked_extent_rejects_the_noncanonical_escape) { + using QW = QueryWire<128>; + const VecZ buf{size_t{0x8037e}}; + const auto h = QW::header_at(buf, 0); + BOOST_TEST(h.phase == 1); + BOOST_TEST(h.k == 6U); + BOOST_TEST(h.gw == 8U); + BOOST_TEST(h.bits == 20U); + BOOST_TEST(QW::header_bits_for(h.k) == 11U); + BOOST_TEST(QW::words_at(buf, 0) == 1U); // the undercount the check exists to catch + expect_malformed<128>(buf, QueryForm::Plain, 0, "0x8037e alone"); + // Padding that makes the decoder's read fit does not make the header canonical. + expect_malformed<128>(VecZ{size_t{0x8037e}, 0, 0}, QueryForm::Plain, 0, "0x8037e padded"); + expect_malformed<128>(VecZ{size_t{0x8037e}, 0}, QueryForm::Fused, 0, "0x8037e fused"); +} + +BOOST_AUTO_TEST_CASE(sparse_record_checked_extent_rejects_malformed_headers_and_boundaries) { + using QW8 = QueryWire<8>; // kBits 16, kPosBits 4, kLongKBits 5 + expect_malformed<8>(VecZ{}, QueryForm::Plain, 0, "offset at the end"); + expect_malformed<8>(VecZ{header_word<8>(3, 0, std::nullopt, 0)}, QueryForm::Plain, 0, "phase field 3"); + expect_malformed<8>(VecZ{header_word<8>(1, 31, 20, 0)}, QueryForm::Plain, 0, "escaped k above kMaxPositions"); + expect_malformed<8>(VecZ{header_word<8>(1, 31, 3, 0)}, QueryForm::Plain, 0, "escaped k below kKEscape"); + expect_malformed<8>(VecZ{header_word<8>(1, 2, std::nullopt, QW8::kPosBits + 1)}, QueryForm::Plain, 0, "gw"); + // A fused record without its value word, and a two-word record cut to one. + VecZ fused; + QW8::push(fused, std::vector{1, 4}, 1); + expect_malformed<8>(fused, QueryForm::Fused, 0, "fused value word missing"); + using QW = QueryWire<1024>; + VecZ wide; + const size_t words = QW::push(wide, strided(40, 0, 50, 2048), -1); + BOOST_REQUIRE(words >= 2U); + wide.pop_back(); + expect_malformed<1024>(wide, QueryForm::Plain, 0, "truncated multiword record"); + // A valid record followed by a lone partial word: the second record's header claims more than remains. + VecZ tail; + QW::push(tail, std::vector{3}, 1); + const auto first = QW::checked_extent(tail, QueryForm::Plain, 0); + tail.push_back(wide[0]); + expect_malformed<1024>(tail, QueryForm::Plain, first.words, "trailing partial record"); +} + +// read_positions checks the cumulative size_t position before narrowing it to the output type: each must stay +// below 2*NumModes, which also bounds the next addition and keeps the positions strictly ascending. +BOOST_AUTO_TEST_CASE(sparse_record_decode_rejects_positions_outside_the_width) { + // 250 modes: a 9-bit first position reaches 511 although only 0..499 exist. + { + using QW = QueryWire<250>; + VecZ buf; + QW::push(buf, std::vector{505}, 1); + std::vector out(1); + BOOST_CHECK_THROW((void)QW::read_positions(buf, 0, out), MalformedQueryStream); + VecZ ok; + QW::push(ok, std::vector{499}, 1); + BOOST_CHECK_NO_THROW((void)QW::read_positions(ok, 0, out)); + } + // 32 modes: a gap walks past position 63. + { + using QW = QueryWire<32>; + VecZ buf; + QW::push(buf, std::vector{10, 70}, -1); + std::vector out(QW::kBits); // room for any k the header could claim + BOOST_CHECK_THROW((void)QW::read_positions(buf, 0, out), MalformedQueryStream); + } + // 128 modes into the store's uint8: 257 would narrow to 1, and 300 to 44, both "valid" after narrowing. + { + using QW = QueryWire<128>; + for (const auto &pos : {std::vector{1, 257}, std::vector{100, 200, 300}}) { + VecZ buf; + QW::push(buf, pos, 1); + BOOST_REQUIRE_NO_THROW((void)QW::checked_extent(buf, QueryForm::Plain, 0)); + std::vector out(QW::kBits); + BOOST_CHECK_THROW((void)QW::read_query(buf, QueryForm::Plain, 0, out), MalformedQueryStream); + } + } +} diff --git a/cpp/tests/sparse_resolve_tests.cpp b/cpp/tests/sparse_resolve_tests.cpp index 1ce716b2..ff89702a 100644 --- a/cpp/tests/sparse_resolve_tests.cpp +++ b/cpp/tests/sparse_resolve_tests.cpp @@ -18,14 +18,24 @@ #include #include +#include #include #include +#include +#include #include #include #include +#include +#include +#include #include +#include "KernelTestSupport.h" + +#include "monoprop/algebra/MajoranaAlgebra.h" #include "monoprop/core/Monomial.h" +#include "monoprop/detail/evolution/layer_build/Engine.h" #include "monoprop/detail/evolution/layer_build/QueryWire.h" #include "monoprop/detail/evolution/layer_build/Resolve.h" #include "monoprop/detail/mpi/Comm.h" @@ -406,3 +416,1199 @@ BOOST_AUTO_TEST_CASE(sparse_resolve_finds_dense_inserted_keys) { } BOOST_TEST(genuinely_absent > 0); } + +/* ── Task 6: checked, threaded incoming resolution and bounded self probing ─────────────────────── */ + +namespace { + +using detail::KernelRange; +using kernel_test::AccumulatingObserver; +using kernel_test::PhaseLog; + +template +auto key_of(const Monomial &m) -> std::vector { + std::vector key; + for (size_t w = 0; w < Monomial::num_words(); ++w) { + key.push_back(m.word(w)); + } + return key; +} + +template +auto rows_of(const detail::MPOperator &op) -> std::vector> { + std::vector> rows; + for (size_t i = 0; i < op.store->size(); ++i) { + rows.push_back(op.store->row(i)); + } + return rows; +} + +// Every array exactly, including its length: the probe allocates each once, at the query count. +template +auto same_probe(const detail::IncomingProbe &a, const detail::IncomingProbe &b) -> bool { + const auto eq = [](const auto &x, const auto &y) { return std::ranges::equal(x, y); }; + return a.window == b.window && a.base == b.base && a.nq_total == b.nq_total && eq(a.goff, b.goff) + && eq(a.sender_wi, b.sender_wi) && eq(a.phase_of, b.phase_of) && eq(a.off_of, b.off_of) + && eq(a.idx_of, b.idx_of) && eq(a.miss_g, b.miss_g) && eq(a.pos_flat, b.pos_flat) && eq(a.pos_off, b.pos_off) + && eq(a.k_of, b.k_of) && eq(a.hash_of, b.hash_of); +} + +template +auto exact_sizes(const detail::IncomingProbe &p, size_t positions) -> bool { + const size_t n = p.nq_total; + return p.goff.size() == p.window.count + 1 && p.sender_wi.size() == n && p.phase_of.size() == n + && p.off_of.size() == n && p.idx_of.size() == n && p.pos_off.size() == n && p.k_of.size() == n + && p.hash_of.size() == n && p.pos_flat.size() == positions && p.miss_g.capacity() == p.miss_g.size(); +} + +// The rows a column holds, whichever representation it is in. +template +auto column_rows(const detail::InvertedIndex &index, size_t c) -> std::vector { + std::vector rows; + if (index.column_is_dense(c)) { + const uint64_t *w = index.dense_column_data(c); + for (size_t r = 0; r < index.rows(); ++r) { + if (((w[r >> 6] >> (r & 63U)) & 1U) != 0U) { + rows.push_back(r); + } + } + } + else { + for (const auto r : index.sparse_column_rows(c)) { + rows.push_back(r); + } + } + return rows; +} + +// An index kept up to date by reindex_after_growth against one built from scratch over the same rows. +template +auto same_inverted_index(const detail::MPOperator &grown, const detail::MPOperator &fresh) -> bool { + const auto &a = grown.inverted_index(); + const auto &b = fresh.inverted_index(); + if (a.rows() != b.rows()) { + return false; + } + for (size_t c = 0; c < 2 * N; ++c) { + if (column_rows(a, c) != column_rows(b, c)) { + return false; + } + } + return true; +} + +// Distinct keys for one resolution: `hits` drawn from the seeded store, the rest fresh, in shuffled order. +template +struct QueryPlan { + std::vector> seeds; + std::vector>> per_sender; + size_t hits = 0; + size_t misses = 0; +}; + +enum class HitMix { none, all, mixed }; + +template +auto plan_queries(uint64_t seed, const std::vector &per_sender, HitMix mix, size_t n_seed = 300) + -> QueryPlan { + std::mt19937_64 rng(seed); + QueryPlan plan; + size_t total = 0; + for (const auto n : per_sender) { + total += n; + } + // Enough stored terms for every planned hit: all of them, or one query in three. + plan.seeds = draw_distinct(rng, std::max(n_seed, mix == HitMix::all ? total : (total / 3) + 1)); + std::set> used; + for (const auto &m : plan.seeds) { + used.insert(key_of(m)); + } + size_t next_seed = 0; + size_t i = 0; + plan.per_sender.resize(per_sender.size()); + for (size_t s = 0; s < per_sender.size(); ++s) { + for (size_t q = 0; q < per_sender[s]; ++q, ++i) { + const bool hit = mix == HitMix::all || (mix == HitMix::mixed && i % 3 == 0); + if (hit) { + BOOST_REQUIRE(next_seed < plan.seeds.size()); + plan.per_sender[s].push_back(plan.seeds[next_seed++]); + ++plan.hits; + continue; + } + for (;;) { + const auto m = draw_distinct(rng, 1)[0]; + if (used.insert(key_of(m)).second) { + plan.per_sender[s].push_back(m); + ++plan.misses; + break; + } + } + } + } + return plan; +} + +// Serial reference (budget 1) against budgets 2, 3, 4 and 8 on identical fresh stores: the whole probe, the +// published rows in row-ID order, every row findable at its ID, and -- with a prebuilt inverted index -- the +// index maintained by reindex_after_growth equal to one built from scratch. +template +auto check_incoming_budgets(const char *label, + const QueryPlan &plan, + mpi::SlotWindow window, + bool fused, + bool prebuilt_index) -> void { + BOOST_TEST_CONTEXT(label << " N=" << N << " base=" << window.base << " fused=" << fused + << " prebuilt=" << prebuilt_index) { + const auto incoming = serialize(plan.per_sender, fused, window); + const auto form = fused ? detail::QueryForm::Fused : detail::QueryForm::Plain; + std::vector> expect; + std::vector expect_sender; + std::vector expect_phase; + std::vector expect_value; + for (size_t s = 0; s < plan.per_sender.size(); ++s) { + for (size_t q = 0; q < plan.per_sender[s].size(); ++q) { + expect.push_back(plan.per_sender[s][q]); + expect_sender.push_back(s); + expect_phase.push_back(((q % 2) == 0) ? 1 : -1); + expect_value.push_back(0.5 + static_cast(q)); + } + } + size_t positions = 0; + for (const auto &m : expect) { + positions += m.count(); + } + + auto serial_op = make_op(plan.seeds); + if (prebuilt_index) { + (void)serial_op.inverted_index(); + } + const auto serial = detail::probe_incoming_queries(incoming, serial_op, form, {.threads = 1}); + BOOST_TEST(exact_sizes(serial, positions)); + BOOST_REQUIRE_EQUAL(serial.nq_total, expect.size()); + BOOST_TEST(serial.base == plan.seeds.size()); + size_t hits = 0; + std::vector expect_misses; + for (size_t g = 0; g < serial.nq_total; ++g) { + BOOST_TEST((serial.mono_at(g) == expect[g])); + BOOST_TEST(serial.sender_index(g).value == expect_sender[g]); + BOOST_TEST(serial.window.slot(serial.sender_index(g)) == window.base + expect_sender[g]); + BOOST_TEST(serial.phase_of[g] == expect_phase[g]); + BOOST_TEST(serial.hash_of[g] == detail::OperatorIndex::fold_hash_positions(serial.positions_at(g))); + if (fused) { + const double v = detail::QueryWire::value_at(incoming[serial.sender_index(g)], serial.off_of[g]); + BOOST_TEST(std::bit_cast(v) == std::bit_cast(expect_value[g])); + } + const bool hit = serial.idx_of[g] < serial.base; + if (hit) { + ++hits; + BOOST_TEST((serial_op.store->row(serial.idx_of[g]) == expect[g])); + } + else { + BOOST_TEST(serial.idx_of[g] == serial.base + expect_misses.size()); + expect_misses.push_back(g); + } + } + BOOST_TEST(hits == plan.hits); + BOOST_TEST(std::ranges::equal(serial.miss_g, expect_misses)); + detail::insert_incoming_misses(serial_op, serial, {.threads = 1}); + const auto serial_rows = rows_of(serial_op); + BOOST_TEST(serial_rows.size() == plan.seeds.size() + plan.misses); + for (size_t j = 0; j < expect_misses.size(); ++j) { + BOOST_TEST((serial_rows[serial.base + j] == expect[expect_misses[j]])); + } + for (size_t i = 0; i < serial_rows.size(); ++i) { + const auto found = serial_op.store->find(serial_rows[i]); + BOOST_TEST((found.has_value() && *found == i)); + } + if (prebuilt_index) { + BOOST_TEST(same_inverted_index(serial_op, make_op(serial_rows))); + } + + for (const int threads : {2, 3, 4, 8}) { + auto op = make_op(plan.seeds); + if (prebuilt_index) { + (void)op.inverted_index(); + } + const auto pr = detail::probe_incoming_queries(incoming, op, form, {.threads = threads}); + BOOST_TEST(same_probe(serial, pr), "budget " << threads); + detail::insert_incoming_misses(op, pr, {.threads = threads}); + BOOST_TEST((rows_of(op) == serial_rows), "budget " << threads); + if (prebuilt_index) { + BOOST_TEST(same_inverted_index(op, make_op(serial_rows))); + } + } + } +} + +} // namespace + +// The plan's fixture: a window at base 4, senders 4 and 6 (5 is empty), predicted and published IDs. Repeated +// at base zero. This preserves behavior that existed before Task 6 and may pass before it. +BOOST_AUTO_TEST_CASE(openmp_incoming_miss_order) { + using namespace monoprop; + const auto a = indices_to_bitset<8>({0, 1}); + const auto b = indices_to_bitset<8>({2, 3}); + const auto c = indices_to_bitset<8>({4, 5}); + for (const size_t window_base : {size_t{4}, size_t{0}}) { + for (const auto threads : {1, 3}) { + auto op = make_op<8>(std::vector>{a}); + const mpi::SlotWindow window{.base = window_base, .count = 3}; + mpi::WindowVec incoming(window); + detail::QueryWire<8>::push(incoming.at_slot(window_base), positions_of<8>(b), +1); + detail::QueryWire<8>::push(incoming.at_slot(window_base + 2), positions_of<8>(a), -1); + detail::QueryWire<8>::push(incoming.at_slot(window_base + 2), positions_of<8>(c), +1); + const auto pr = + detail::probe_incoming_queries<8>(incoming, op, detail::QueryForm::Plain, {.threads = threads}); + BOOST_TEST(pr.window.slot(pr.sender_index(0)) == window_base); + BOOST_TEST(pr.window.slot(pr.sender_index(1)) == window_base + 2); + const std::vector offsets{0, 1, 1, 3}, ids{1, 0, 2}, misses{0, 2}; + BOOST_TEST(pr.base == 1U); + BOOST_CHECK_EQUAL_COLLECTIONS(pr.goff.begin(), pr.goff.end(), offsets.begin(), offsets.end()); + BOOST_CHECK_EQUAL_COLLECTIONS(pr.idx_of.begin(), pr.idx_of.end(), ids.begin(), ids.end()); + BOOST_CHECK_EQUAL_COLLECTIONS(pr.miss_g.begin(), pr.miss_g.end(), misses.begin(), misses.end()); + detail::insert_incoming_misses(op, pr); + BOOST_TEST(op.size() == 3U); + BOOST_REQUIRE(op.store->find(a).has_value()); + BOOST_REQUIRE(op.store->find(b).has_value()); + BOOST_REQUIRE(op.store->find(c).has_value()); + BOOST_TEST(op.store->find(a).value() == 0U); + BOOST_TEST(op.store->find(b).value() == 1U); + BOOST_TEST(op.store->find(c).value() == 2U); + } + } +} + +// miss_g holds query ordinals, which are not TermIndex values and may exceed their range. +BOOST_AUTO_TEST_CASE(openmp_incoming_miss_ordinals_are_size_t) { + static_assert(std::is_same_v::miss_g), std::vector>); + BOOST_TEST(true); +} + +// Serial against threaded over the input shapes the resolver meets: empty, all-hit, all-miss and mixed, block +// boundaries 255/256/257 and larger partial blocks, empty intervening senders, window bases zero and non-zero, +// narrow and wide positions with inline and spilled rows and multiword/escaped records, Plain and Fused codecs, +// and a prebuilt inverted index maintained across the publication. +BOOST_AUTO_TEST_CASE(openmp_incoming_probe_threaded_matches_serial) { + const mpi::SlotWindow base0{.base = 0, .count = 1}; + check_incoming_budgets<32>("empty", plan_queries<32>(1, {0}, HitMix::none), base0, false, false); + check_incoming_budgets<32>("empty senders", + plan_queries<32>(2, {0, 0, 0}, HitMix::none), + {.base = 3, .count = 3}, + true, + false); + check_incoming_budgets<32>("255 all-hit", plan_queries<32>(3, {255}, HitMix::all), base0, false, true); + check_incoming_budgets<32>("256 all-miss", plan_queries<32>(4, {256}, HitMix::none), base0, true, true); + check_incoming_budgets<32>("257 mixed", plan_queries<32>(5, {257}, HitMix::mixed), base0, false, false); + check_incoming_budgets<250>("empty intervening senders", + plan_queries<250>(6, {0, 300, 0, 413}, HitMix::mixed), + {.base = 5, .count = 4}, + false, + true); + check_incoming_budgets<250>("fused partial blocks", + plan_queries<250>(7, {1, 600, 130}, HitMix::mixed), + {.base = 2, .count = 3}, + true, + false); + check_incoming_budgets<128>("narrow store, wide wire", + plan_queries<128>(8, {513, 0}, HitMix::mixed), + {.base = 0, .count = 2}, + true, + true); + // Escaped counts: fully paired 20-mode terms carry k >= 31 on the wire. + { + auto plan = plan_queries<1024>(9, {40, 40}, HitMix::mixed, 60); + std::mt19937_64 rng(10); + for (auto &sender : plan.per_sender) { + for (size_t q = 1; q < sender.size(); q += 2) { + auto &m = sender[q]; + if (m.count() < 31U) { + auto wide = random_paired_monomial<1024>(rng, 16 + (rng() % 20)); + if (std::ranges::none_of(plan.seeds, [&](const auto &s) { return s == wide; })) { + m = wide; + } + } + } + } + // Rebuild the hit/miss counts from the store: some hits were replaced by fresh escaped terms. + auto op = make_op<1024>(plan.seeds); + plan.hits = 0; + plan.misses = 0; + size_t escaped = 0; + for (const auto &sender : plan.per_sender) { + for (const auto &m : sender) { + (op.store->find(m).has_value() ? plan.hits : plan.misses) += 1; + escaped += static_cast(m.count() >= detail::QueryWire<1024>::kKEscape); + } + } + BOOST_TEST(escaped > 0U); + check_incoming_budgets<1024>("escaped counts", plan, {.base = 1, .count = 2}, true, false); + } +} + +// Invalid headers and boundaries are rejected on the caller before any decode worker starts: nothing is +// decoded, probed or published, and the store is untouched. +BOOST_AUTO_TEST_CASE(openmp_incoming_probe_rejects_malformed_streams_before_decode) { + using QW = detail::QueryWire<128>; + std::mt19937_64 rng(11); + const auto seeds = draw_distinct<128>(rng, 20); + // Three one-word records: read as Fused, the second becomes the first's value word and the third lacks one. + VecZ good; + for (const auto &pos : {std::vector{1, 2}, std::vector{3, 9}, std::vector{0, 5}}) { + BOOST_REQUIRE(QW::push(good, pos, 1) == 1U); + } + VecZ multiword; + std::vector spread; + for (uint16_t p = 0; p < 250; p += 7) { + spread.push_back(p); + } + const size_t words = QW::push(multiword, spread, -1); + BOOST_REQUIRE(words >= 2U); + VecZ truncated = multiword; + truncated.pop_back(); + VecZ trailing = good; + trailing.push_back(multiword[0]); + VecZ fused_missing_value = good; // Fused form: every record needs a value word, and none has one + VecZ bad_phase{size_t{3}}; + struct Bad { + const char *label; + VecZ stream; + detail::QueryForm form; + }; + const std::vector cases{{"noncanonical escape 0x8037e", VecZ{size_t{0x8037e}}, detail::QueryForm::Plain}, + {"truncated multiword record", truncated, detail::QueryForm::Plain}, + {"trailing partial record", trailing, detail::QueryForm::Plain}, + {"fused stream without value words", fused_missing_value, detail::QueryForm::Fused}, + {"phase field 3", bad_phase, detail::QueryForm::Plain}}; + for (const auto &bad : cases) { + for (const int threads : {1, 3}) { + BOOST_TEST_CONTEXT(bad.label << " budget " << threads) { + auto op = make_op<128>(seeds); + const mpi::SlotWindow window{.base = 3, .count = 3}; + mpi::WindowVec incoming(window); + if (bad.form == detail::QueryForm::Plain) { + incoming.at_slot(3) = good; // a valid sender first, an empty one, then the malformed one + } + incoming.at_slot(5) = bad.stream; + PhaseLog log; + BOOST_CHECK_THROW((void)detail::probe_incoming_queries<128>(incoming, + op, + bad.form, + {.threads = threads}, + AccumulatingObserver{&log}), + detail::MalformedQueryStream); + BOOST_TEST(log.calls.empty()); // no decode, no probe + BOOST_TEST(op.size() == seeds.size()); + } + } + } +} + +// Valid headers with invalid decoded positions fail inside a decode worker; the failure joins and is rethrown +// before the probe starts or anything is published. +BOOST_AUTO_TEST_CASE(openmp_incoming_decode_rejects_invalid_positions_before_probing) { + using QW = detail::QueryWire<250>; + const auto plan = plan_queries<250>(12, {400, 300}, HitMix::mixed); + auto incoming = serialize<250>(plan.per_sender, false, {.base = 0, .count = 2}); + QW::push(incoming[mpi::WindowIndex{1}], std::vector{7, 505}, 1); // query 700, decode block 2 + for (const int threads : {1, 3}) { + BOOST_TEST_CONTEXT("budget " << threads) { + auto op = make_op<250>(plan.seeds); + PhaseLog log; + BOOST_CHECK_THROW((void)detail::probe_incoming_queries<250>(incoming, + op, + detail::QueryForm::Plain, + {.threads = threads}, + AccumulatingObserver{&log}), + detail::MalformedQueryStream); + BOOST_TEST(log.of(KernelRange::decode).size() == 1U); + BOOST_TEST(log.of(KernelRange::decode)[0]->worker.size() == 3U); + BOOST_TEST(log.of(KernelRange::incoming_probe).empty()); + BOOST_TEST(op.size() == plan.seeds.size()); + } + } +} + +// A decode worker's own failure (not a malformed input) is joined and rethrown before probing, too. +BOOST_AUTO_TEST_CASE(openmp_incoming_decode_worker_failure_joins_before_probing) { + const auto plan = plan_queries<250>(13, {600, 424}, HitMix::mixed); + const auto incoming = serialize<250>(plan.per_sender, true, {.base = 1, .count = 2}); + auto op = make_op<250>(plan.seeds); + PhaseLog log; + BOOST_CHECK_THROW((void)detail::probe_incoming_queries<250>(incoming, + op, + detail::QueryForm::Fused, + {.threads = 4}, + AccumulatingObserver{&log, KernelRange::decode, 0, 3}), + std::runtime_error); + BOOST_TEST(log.of(KernelRange::decode).size() == 1U); + BOOST_TEST(log.of(KernelRange::decode)[0]->visits.at(3) == 1); + BOOST_TEST(log.of(KernelRange::incoming_probe).empty()); + BOOST_TEST(op.size() == plan.seeds.size()); +} + +// Decode and probe each run in their own region, one logical range per 256 queries. +BOOST_AUTO_TEST_CASE(openmp_incoming_decode_and_probe_workers_participate, + *boost::unit_test::precondition(kernel_test::runtime_offers_two_workers)) { + const auto plan = plan_queries<250>(14, {500, 0, 524}, HitMix::mixed); + const auto incoming = serialize<250>(plan.per_sender, false, {.base = 0, .count = 3}); + auto op = make_op<250>(plan.seeds); + PhaseLog log; + const auto pr = detail::probe_incoming_queries<250>(incoming, + op, + detail::QueryForm::Plain, + {.threads = 4}, + AccumulatingObserver{&log}); + BOOST_TEST(pr.nq_total == 1024U); + BOOST_REQUIRE(log.of(KernelRange::decode).size() == 1U); + BOOST_REQUIRE(log.of(KernelRange::incoming_probe).size() == 1U); + kernel_test::check_participation(kernel_test::slots_of(*log.of(KernelRange::decode)[0]), 4, "decode"); + kernel_test::check_participation(kernel_test::slots_of(*log.of(KernelRange::incoming_probe)[0]), 4, "probe"); + // Budget 1 and a single block stay on the caller. + for (const auto &[threads, senders] : + std::vector>>{{1, {500, 0, 524}}, {4, {200, 0, 56}}}) { + const auto small = plan_queries<250>(15, senders, HitMix::mixed); + auto op_small = make_op<250>(small.seeds); + PhaseLog serial_log; + (void)detail::probe_incoming_queries<250>(serialize<250>(small.per_sender, false, {.base = 0, .count = 3}), + op_small, + detail::QueryForm::Plain, + {.threads = threads}, + AccumulatingObserver{&serial_log}); + const size_t blocks = detail::logical_ranges(senders[0] + senders[2], detail::kProbeBlockQueries); + kernel_test::check_serial(kernel_test::slots_of(*serial_log.of(KernelRange::decode)[0]), blocks, "decode"); + kernel_test::check_serial(kernel_test::slots_of(*serial_log.of(KernelRange::incoming_probe)[0]), + blocks, + "probe"); + } +} + +// probe_frozen_positions: 256-query blocks over absolute offsets into one shared buffer give exactly the single +// find_batch_positions call, hashes included, at every budget and with more workers than blocks. +BOOST_AUTO_TEST_CASE(openmp_probe_frozen_positions_matches_one_batch) { + const auto run = [](std::integral_constant, uint64_t seed) { + using PosT = typename detail::OperatorIndex::PosT; + std::mt19937_64 rng(seed); + const auto seeds = draw_distinct(rng, 400); + const auto op = make_op(seeds); + const auto fresh = draw_distinct(rng, 700); + for (const size_t n : {size_t{0}, size_t{1}, size_t{255}, size_t{256}, size_t{257}, size_t{1000}}) { + std::vector flat(5, PosT{1}); // unreferenced prefix + std::vector off(n); + std::vector k_of(n); + // Queries laid out in reverse, so offsets are absolute and not monotonic in the query ordinal. + for (size_t q = n; q-- > 0;) { + const auto &m = (q % 2 == 0) ? seeds[q % seeds.size()] : fresh[q % fresh.size()]; + off[q] = flat.size(); + for (const auto p : positions_of(m)) { + flat.push_back(static_cast(p)); + } + k_of[q] = static_cast(m.count()); + } + std::vector want(n); + std::vector want_hash(n); + op.store->find_batch_positions(flat, off, k_of, want, want_hash); + for (const int threads : {1, 2, 3, 4, 8}) { + std::vector out(n, 17); + std::vector hash(n, 17); + detail::probe_frozen_positions(*op.store, flat, off, k_of, out, hash, {.threads = threads}); + BOOST_TEST(out == want, "N=" << N << " n=" << n << " budget " << threads); + BOOST_TEST(hash == want_hash, "N=" << N << " n=" << n << " budget " << threads); + std::vector no_hash(n, 17); + detail::probe_frozen_positions(*op.store, flat, off, k_of, no_hash, {}, {.threads = threads}); + BOOST_TEST(no_hash == want); + } + } + }; + run(std::integral_constant{}, 16); + run(std::integral_constant{}, 17); // wide positions, spilled rows +} + +// The helper's contracts are checked on the caller, before any block runs. +BOOST_AUTO_TEST_CASE(openmp_probe_frozen_positions_rejects_bad_spans) { + using PosT = detail::OperatorIndex<32>::PosT; + const auto op = make_op<32>(std::vector>{indices_to_bitset<32>({1, 2})}); + const std::vector flat{1, 2, 3, 4}; + std::vector out(2); + std::vector hash(2); + const auto expect_throw = [&](std::vector off, std::vector k, size_t n_out, size_t n_hash) { + std::vector o(n_out); + std::vector h(n_hash); + PhaseLog log; + BOOST_CHECK_THROW(detail::probe_frozen_positions<32>(*op.store, + flat, + off, + k, + o, + h, + {.threads = 2}, + AccumulatingObserver{&log}, + KernelRange::incoming_probe), + std::invalid_argument); + BOOST_TEST(log.calls.empty()); + }; + expect_throw({0, 2}, {2}, 2, 2); // counts shorter than offsets + expect_throw({0, 2}, {2, 2}, 1, 2); // output shorter + expect_throw({0, 2}, {2, 2}, 2, 1); // hash output neither empty nor full + expect_throw({0, 3}, {2, 2}, 2, 2); // span past the buffer + expect_throw({0, std::numeric_limits::max()}, {2, 1}, 2, 2); // offset + count wraps +} + +BOOST_AUTO_TEST_CASE(openmp_probe_frozen_positions_workers_participate, + *boost::unit_test::precondition(kernel_test::runtime_offers_two_workers)) { + using PosT = detail::OperatorIndex<32>::PosT; + std::mt19937_64 rng(18); + const auto seeds = draw_distinct<32>(rng, 300); + const auto op = make_op<32>(seeds); + std::vector flat; + std::vector off; + std::vector k_of; + for (size_t q = 0; q < 1024; ++q) { + off.push_back(flat.size()); + for (const auto p : positions_of<32>(seeds[q % seeds.size()])) { + flat.push_back(static_cast(p)); + } + k_of.push_back(static_cast(seeds[q % seeds.size()].count())); + } + std::vector out(off.size()); + std::vector hash(off.size()); + PhaseLog log; + detail::probe_frozen_positions<32>(*op.store, + flat, + off, + k_of, + out, + hash, + {.threads = 4}, + AccumulatingObserver{&log}, + KernelRange::self_probe); + BOOST_REQUIRE(log.of(KernelRange::self_probe).size() == 1U); + kernel_test::check_participation(kernel_test::slots_of(log.calls[0]), 4, "frozen probe"); +} + +/* ── Phase-3 scatter: capability-gated parallel on_resolved ─────────────────────────────────────── */ + +namespace { + +// A generic sink with no parallel capability: on_resolved appends to a shared vector, so a scatter that ignored +// the missing capability would race (TSan) and scramble the order. +template +struct AppendingSink { + using Response = TermIndex; + static auto init_response() -> Response { return std::numeric_limits::max(); } + [[nodiscard]] auto incoming_form() const -> detail::QueryForm { return detail::QueryForm::Plain; } + std::vector order; + auto prepare(const detail::IncomingProbe &, + detail::MPOperator &, + const mpi::WindowVec> &) -> void {} + auto on_resolved(size_t g, + mpi::WindowIndex, + size_t, + size_t ip, + const detail::IncomingProbe &, + const mpi::WindowVec &) -> Response { + order.push_back(g); + return static_cast(ip); + } +}; + +template +struct ExplicitlySerialSink : AppendingSink { + static constexpr bool parallel_resolve = false; +}; + +struct ScatterRun { + std::vector> responses; // bit patterns compared through bit_cast + std::vector> sink_state; + std::vector marks; + std::vector> rows; + PhaseLog log; +}; + +template +auto run_scatter(const QueryPlan &plan, + mpi::SlotWindow window, + bool leader, + int threads, + Sink &sink, + detail::MPOperator &op, + Capture &&capture) -> ScatterRun { + ScatterRun run; + const auto incoming = serialize(plan.per_sender, sink.incoming_form() == detail::QueryForm::Fused, window); + detail::MatchedEpochSet matched; + const size_t combined = op.size(); + matched.begin_gate(combined); + const auto responses = detail::resolve_incoming(incoming, + op, + leader, + matched, + combined, + sink, + {.threads = threads}, + AccumulatingObserver{&run.log}); + for (const auto wi : window.indices()) { + std::vector r; + for (const auto v : responses[wi]) { + r.push_back(static_cast(v)); + } + run.responses.push_back(r); + } + for (size_t i = 0; i < combined; ++i) { + run.marks.push_back(static_cast(matched.is_marked(i))); + } + for (const auto &m : rows_of(op)) { + run.rows.push_back(key_of(m)); + } + run.sink_state = capture(sink); + return run; +} + +auto same_scatter(const ScatterRun &a, const ScatterRun &b) -> bool { + if (a.responses.size() != b.responses.size()) { + return false; + } + for (size_t s = 0; s < a.responses.size(); ++s) { + if (a.responses[s].size() != b.responses[s].size() + || !std::ranges::equal(a.responses[s], b.responses[s], [](double x, double y) { + return std::bit_cast(x) == std::bit_cast(y); + })) { + return false; + } + } + return a.sink_state == b.sink_state && a.marks == b.marks && a.rows == b.rows; +} + +template +auto graph_state(const detail::GraphSink &sink) -> std::vector> { + std::vector> out; + for (const auto &a : sink.acc) { + std::vector v; + for (const auto &e : a.in_entries) { + v.push_back(e.idx); + v.push_back(static_cast(static_cast(e.phase))); + } + out.push_back(v); + } + return out; +} + +inline auto contract_state(const detail::FusedContract &fc) -> std::vector> { + std::vector v; + for (const auto &h : fc.cross_half) { + v.push_back(h.local_idx); + v.push_back(std::bit_cast(h.v_partner)); + v.push_back(static_cast(static_cast(h.phase_signed))); + v.push_back(static_cast(h.is_insert)); + } + return {v}; +} + +} // namespace + +// GraphSink and the Heisenberg ContractSink certify parallel on_resolved; their scatter runs on workers and +// matches the serial scatter exactly: responses, sink records, leader marks and the published rows. +BOOST_AUTO_TEST_CASE(openmp_resolve_scatter_threaded_matches_serial) { + constexpr size_t N = 250; + const mpi::SlotWindow window{.base = 2, .count = 3}; + const auto plan = plan_queries(19, {1300, 0, 1250}, HitMix::mixed); + VecD coeffs; + for (size_t i = 0; i < plan.seeds.size(); ++i) { + coeffs.push_back(0.125 + (0.001 * static_cast(i))); + } + for (const bool leader : {true, false}) { + std::vector graph; + std::vector contract; + std::vector contract_swept; + for (const int threads : {1, 2, 3, 4}) { + { + auto op = make_op(plan.seeds); + detail::GraphSink sink(window.stop(), 0); + graph.push_back(run_scatter(plan, window, leader, threads, sink, op, [](const auto &s) { + return graph_state(s); + })); + } + for (const bool swept : {false, true}) { + auto op = make_op(plan.seeds); + detail::FusedContract fc; + detail::ContractSink sink{.R = window.stop(), + .my_rank = 0, + .fc = fc, + .op_coeffs = coeffs, + .fused_scale = swept, + .inv_cos = swept ? 1.25 : 1.0, + .schrodinger = false, + .basis = Basis::Majorana}; + (swept ? contract_swept : contract) + .push_back(run_scatter(plan, window, leader, threads, sink, op, [&](const auto &) { + return contract_state(fc); + })); + } + } + for (size_t t = 1; t < graph.size(); ++t) { + BOOST_TEST(same_scatter(graph[0], graph[t]), "graph leader=" << leader << " budget " << t + 1); + BOOST_TEST(same_scatter(contract[0], contract[t]), "contract leader=" << leader << " budget " << t + 1); + BOOST_TEST(same_scatter(contract_swept[0], contract_swept[t]), "swept leader=" << leader); + } + BOOST_TEST(std::ranges::count(graph[0].marks, 1) == (leader ? static_cast(plan.hits) : 0)); + } +} + +BOOST_AUTO_TEST_CASE(openmp_resolve_scatter_workers_participate, + *boost::unit_test::precondition(kernel_test::runtime_offers_two_workers)) { + constexpr size_t N = 250; + const mpi::SlotWindow window{.base = 0, .count = 2}; + const auto plan = plan_queries(20, {2048, 2048}, HitMix::mixed, 1400); + VecD coeffs(plan.seeds.size(), 0.5); + { + auto op = make_op(plan.seeds); + detail::GraphSink sink(2, 0); + const auto run = + run_scatter(plan, window, true, 4, sink, op, [](const auto &s) { return graph_state(s); }); + BOOST_REQUIRE(run.log.of(KernelRange::scatter).size() == 1U); + kernel_test::check_participation(kernel_test::slots_of(*run.log.of(KernelRange::scatter)[0]), 4, "graph"); + } + { + auto op = make_op(plan.seeds); + detail::FusedContract fc; + detail::ContractSink sink{.R = 2, + .my_rank = 0, + .fc = fc, + .op_coeffs = coeffs, + .fused_scale = false, + .inv_cos = 1.0, + .schrodinger = false, + .basis = Basis::Majorana}; + const auto run = + run_scatter(plan, window, true, 4, sink, op, [&](const auto &) { return contract_state(fc); }); + kernel_test::check_participation(kernel_test::slots_of(*run.log.of(KernelRange::scatter)[0]), 4, "contract"); + } +} + +// Sinks that do not certify parallel on_resolved keep the serial scatter at any budget: a generic sink without +// the member, one declaring it false, and the Schrodinger ContractSink (state scoring stays serial). +BOOST_AUTO_TEST_CASE(openmp_resolve_scatter_stays_serial_without_the_capability) { + constexpr size_t N = 128; + const mpi::SlotWindow window{.base = 1, .count = 2}; + const auto plan = plan_queries(21, {1500, 1100}, HitMix::mixed); + const size_t blocks = detail::logical_ranges(2600, detail::kResolveScatterBlock); + const auto order_state = [](const auto &s) { + return std::vector>{std::vector(s.order.begin(), s.order.end())}; + }; + for (const int threads : {1, 4}) { + auto op = make_op(plan.seeds); + AppendingSink generic; + const auto a = run_scatter(plan, window, true, threads, generic, op, order_state); + kernel_test::check_serial(kernel_test::slots_of(*a.log.of(KernelRange::scatter)[0]), blocks, "no member"); + std::vector in_order(2600); + std::iota(in_order.begin(), in_order.end(), 0); + BOOST_TEST(a.sink_state[0] == in_order); + + auto op2 = make_op(plan.seeds); + ExplicitlySerialSink declared; + const auto b = run_scatter(plan, window, true, threads, declared, op2, order_state); + kernel_test::check_serial(kernel_test::slots_of(*b.log.of(KernelRange::scatter)[0]), blocks, "false"); + BOOST_TEST(b.sink_state[0] == in_order); + } + // Schrodinger: fresh cross-rank misses are state-scored from their positions (paired-only mono_at). + const auto plan_s = plan_queries(22, {1400, 1200}, HitMix::mixed); + VecD coeffs(plan_s.seeds.size(), 0.25); + std::vector runs; + for (const int threads : {1, 4}) { + auto op = make_op(plan_s.seeds); + op.initial_state = {0, 3, 5, 9}; + detail::FusedContract fc; + detail::ContractSink sink{.R = 3, + .my_rank = 0, + .fc = fc, + .op_coeffs = coeffs, + .fused_scale = false, + .inv_cos = 1.0, + .schrodinger = true, + .basis = Basis::Majorana}; + runs.push_back( + run_scatter(plan_s, window, true, threads, sink, op, [&](const auto &) { return contract_state(fc); })); + kernel_test::check_serial(kernel_test::slots_of(*runs.back().log.of(KernelRange::scatter)[0]), + blocks, + "schrodinger"); + } + BOOST_TEST(same_scatter(runs[0], runs[1])); +} + +// The parallel scatter's matched marks are disjoint uint16 writes only because distinct queries resolve to +// distinct rows: queries are source XOR G over distinct sources. Pinned on generator-derived queries, with a +// duplicated query as the negative control the debug assertion would catch. +BOOST_AUTO_TEST_CASE(openmp_resolve_leader_marks_are_unique) { + constexpr size_t N = 32; + std::mt19937_64 rng(23); + const auto store_terms = draw_distinct(rng, 500); + const auto gen = indices_to_bitset({3, 17}); + // Sources whose partner is stored (hits) and others (misses): partners are pairwise distinct. + std::vector>> per_sender(2); + std::set> seen; + for (size_t i = 0; i < store_terms.size(); ++i) { + const auto query = (i % 4 == 0) ? store_terms[i] : (store_terms[i] ^ gen ^ indices_to_bitset({i % 64})); + if (seen.insert(key_of(query)).second) { + per_sender[i % 2].push_back(query); + } + } + auto op = make_op(store_terms); + const auto incoming = serialize(per_sender, false, {.base = 0, .count = 2}); + const auto pr = detail::probe_incoming_queries(incoming, op, detail::QueryForm::Plain, {.threads = 3}); + BOOST_TEST(detail::leader_marks_unique(pr, op.size())); + auto dup = per_sender; + dup[1].push_back(dup[0][0]); + const auto pr_dup = detail::probe_incoming_queries(serialize(dup, false, {.base = 0, .count = 2}), + op, + detail::QueryForm::Plain, + {.threads = 3}); + BOOST_TEST(!detail::leader_marks_unique(pr_dup, op.size())); +} + +/* ── Self resolution in bounded windows, and the publication order across passes ────────────────── */ + +namespace { + +template +using ObservedEngine = detail::LayerBuildEngine; + +template +auto stage_self(Eng &eng, const std::vector> &keys, const std::vector &srcs) -> void { + using PosT = typename Eng::RowPosT; + for (size_t q = 0; q < keys.size(); ++q) { + std::vector pos; + for (const auto p : positions_of(keys[q])) { + pos.push_back(static_cast(p)); + } + eng.self_stage_.push(pos, (q % 3 == 0) ? -1 : 1); + } + eng.src_idx_r.at_slot(eng.my_rank) = srcs; +} + +struct SelfRun { + std::vector> state; // sink entries, deferred misses, marks, rows + PhaseLog log; +}; + +// Leader then follower self passes over stages spanning several 4096-query windows, then finish(): the sink's +// hits in order, every deferred miss's positions, hash, source, phase and v_src, the leader marks, and the rows +// published after both passes. +template +auto run_self(const std::vector> &seeds, + const std::vector> &leader_keys, + const std::vector &leader_srcs, + const std::vector> &follower_keys, + const std::vector &follower_srcs, + int threads) -> SelfRun { + SelfRun run; + auto op = make_op(seeds); + detail::MatchedEpochSet matched; + const size_t combined = op.size(); + detail::FusedContract fc; + VecD coeffs; + for (size_t i = 0; i < seeds.size(); ++i) { + coeffs.push_back(1.0 / static_cast(i + 3)); + } + const mpi::SlotWindow window{.base = 0, .count = 1}; + auto body = [&](auto &eng) { + const auto values = [](const std::vector &srcs) { + std::vector v; + for (const auto s : srcs) { + v.push_back(0.5 + static_cast(s)); + } + return v; + }; + stage_self(eng, leader_keys, leader_srcs); + if constexpr (Contract) { + eng.src_val_r.reset(window); + eng.src_val_r.at_slot(0) = values(leader_srcs); + } + eng.resolve_self_queries(true); + stage_self(eng, follower_keys, follower_srcs); + if constexpr (Contract) { + eng.src_val_r.at_slot(0) = values(follower_srcs); + } + eng.resolve_self_queries(false); + std::vector deferred; + for (const auto &m : eng.deferred_self_misses) { + deferred.insert(deferred.end(), + {m.k, + m.hash, + m.src, + static_cast(static_cast(m.phase)), + std::bit_cast(m.v_src)}); + for (size_t j = 0; j < m.k; ++j) { + deferred.push_back(eng.deferred_pos_flat_[m.pos_at + j]); + } + } + run.state.push_back(deferred); + std::vector marks; + for (size_t i = 0; i < combined; ++i) { + marks.push_back(static_cast(matched.is_marked(i))); + } + run.state.push_back(marks); + (void)eng.finish(CosMask{}, nullptr); + }; + if constexpr (Contract) { + ObservedEngine> eng(op, + mpi::Comm{}, + 1, + 0, + matched, + combined, + detail::ContractSink{.R = 1, + .my_rank = 0, + .fc = fc, + .op_coeffs = coeffs, + .fused_scale = false, + .inv_cos = 1.0, + .schrodinger = false, + .basis = Basis::Majorana}, + mpi::PeerPlan{}, + window, + {.threads = threads}, + AccumulatingObserver{&run.log}); + body(eng); + std::vector recs; + for (const auto *list : {&fc.hits, &fc.inserts}) { + for (const auto &r : *list) { + recs.insert(recs.end(), + {r.src, + r.tgt, + std::bit_cast(r.v_src), + std::bit_cast(r.v_tgt), + static_cast(static_cast(r.phase))}); + } + } + run.state.push_back(recs); + } + else { + ObservedEngine> eng(op, + mpi::Comm{}, + 1, + 0, + matched, + combined, + detail::GraphSink(1, 0), + mpi::PeerPlan{}, + window, + {.threads = threads}, + AccumulatingObserver{&run.log}); + body(eng); + std::vector entries; + for (const auto *list : {&eng.sink.acc[0].in_entries, &eng.sink.acc[0].out_entries}) { + for (const auto &e : *list) { + entries.push_back(e.idx); + entries.push_back(static_cast(static_cast(e.phase))); + } + } + run.state.push_back(entries); + } + const auto rows = rows_of(op); + size_t findable = 0; + for (size_t i = 0; i < rows.size(); ++i) { + const auto found = op.store->find(rows[i]); + findable += static_cast(found.has_value() && *found == i); + run.state.push_back(key_of(rows[i])); + } + // A deferred miss published with a wrong cached hash is a correct row that no lookup can find. + BOOST_TEST(findable == rows.size()); + return run; +} + +} // namespace + +// Stages spanning two full 4096-query windows plus a short final one; followers whose source a leader matched +// are filtered before windowing. Serial and threaded self resolution agree exactly, and each window probes in +// at most 16 blocks of 256 queries. +BOOST_AUTO_TEST_CASE(openmp_self_windows_match_serial) { + constexpr size_t N = 128; + std::mt19937_64 rng(24); + const auto seeds = draw_distinct(rng, 6000); + std::set> used; + for (const auto &m : seeds) { + used.insert(key_of(m)); + } + const auto fresh = [&] { + for (;;) { + const auto m = draw_distinct(rng, 1)[0]; + if (used.insert(key_of(m)).second) { + return m; + } + } + }; + const size_t n_leader = (2 * detail::kSelfProbeWindow) + 37; + std::vector> leader_keys; + std::vector leader_srcs; + for (size_t q = 0; q < n_leader; ++q) { + leader_keys.push_back(q % 2 == 0 ? seeds[q / 2] : fresh()); + leader_srcs.push_back(q % seeds.size()); + } + // Followers: every fifth source is a row a leader hit (seeds[q/2] for even q), so it is filtered. + std::vector> follower_keys; + std::vector follower_srcs; + size_t filtered = 0; + for (size_t q = 0; q < 5000; ++q) { + const bool matched_source = q % 5 == 0; + follower_keys.push_back(q % 3 == 0 ? seeds[4200 + (q % 1700)] : fresh()); + follower_srcs.push_back(matched_source ? (q % 4000) : 5000 + (q % 900)); + filtered += static_cast(matched_source); + } + // Duplicate hit keys across the two passes would be two rotations with one target; keep them distinct. + { + std::set> keys; + for (const auto &m : leader_keys) { + BOOST_REQUIRE(keys.insert(key_of(m)).second); + } + for (const auto &m : follower_keys) { + BOOST_REQUIRE(keys.insert(key_of(m)).second); + } + } + const size_t kept_followers = follower_keys.size() - filtered; + for (const bool contract : {false, true}) { + const auto run_at = [&](int threads) { + return contract + ? run_self(seeds, leader_keys, leader_srcs, follower_keys, follower_srcs, threads) + : run_self(seeds, leader_keys, leader_srcs, follower_keys, follower_srcs, threads); + }; + const auto serial = run_at(1); + for (const int threads : {2, 3, 4}) { + const auto threaded = run_at(threads); + BOOST_TEST((serial.state == threaded.state), "contract=" << contract << " budget " << threads); + const auto calls = threaded.log.of(KernelRange::self_probe); + const size_t leader_windows = detail::logical_ranges(n_leader, detail::kSelfProbeWindow); + const size_t follower_windows = detail::logical_ranges(kept_followers, detail::kSelfProbeWindow); + BOOST_REQUIRE(calls.size() == leader_windows + follower_windows); + BOOST_TEST(calls[0]->worker.size() == 16U); + BOOST_TEST(calls[leader_windows - 1]->worker.size() == 1U); // the 37-query tail + BOOST_TEST(calls[leader_windows]->worker.size() == 16U); + BOOST_TEST( + calls.back()->worker.size() + == detail::logical_ranges(kept_followers % detail::kSelfProbeWindow, detail::kProbeBlockQueries)); + for (const auto *c : calls) { + BOOST_TEST(c->worker.size() <= detail::kSelfProbeWindow / detail::kProbeBlockQueries); + } + } + if (!!kernel_test::runtime_offers_two_workers(0)) { + const auto threaded = run_at(4); + const auto calls = threaded.log.of(KernelRange::self_probe); + kernel_test::check_participation(kernel_test::slots_of(*calls[0]), 16, "self window"); + kernel_test::check_serial(kernel_test::slots_of(*calls[2]), 1, "self tail"); + } + } +} + +// Mixed self and remote hits and misses across the leader and follower passes, as run_exchange orders them: +// remote leader misses publish in the leader round, remote follower misses in the follower round, and the +// deferred self misses (leader, then follower) only after both passes. Identical at every budget. +BOOST_AUTO_TEST_CASE(openmp_mixed_self_and_remote_publication_order) { + constexpr size_t N = 250; + std::mt19937_64 rng(25); + const auto seeds = draw_distinct(rng, 3000); + std::set> used; + for (const auto &m : seeds) { + used.insert(key_of(m)); + } + size_t next_seed = 0; + const auto pick = [&](bool hit) { + if (hit) { + return seeds[next_seed++]; + } + for (;;) { + const auto m = draw_distinct(rng, 1)[0]; + if (used.insert(key_of(m)).second) { + return m; + } + } + }; + const auto keys = [&](size_t n) { + std::vector> out; + for (size_t q = 0; q < n; ++q) { + out.push_back(pick(q % 3 == 0)); + } + return out; + }; + const auto self_leader = keys(900); + const auto remote_leader = keys(700); + const auto self_follower = keys(600); + const auto remote_follower = keys(650); + std::vector self_leader_srcs(self_leader.size()); + std::iota(self_leader_srcs.begin(), self_leader_srcs.end(), 0); + // Follower sources 0..299 are targets a leader hit? Leaders hit seeds[0..], i.e. rows 0,1,2,..; filter a few. + std::vector self_follower_srcs; + for (size_t q = 0; q < self_follower.size(); ++q) { + self_follower_srcs.push_back(q % 7 == 0 ? q / 7 : 2800 + (q % 190)); + } + const mpi::SlotWindow window{.base = 0, .count = 2}; + const auto remote = [&](const std::vector> &k) { + return serialize(std::vector>>{{}, k}, false, window); + }; + std::vector>> results; + for (const int threads : {1, 2, 3, 4}) { + auto op = make_op(seeds); + detail::MatchedEpochSet matched; + const size_t combined = op.size(); + detail::LayerBuildEngine> eng(op, + mpi::Comm{}, + 2, + 0, + matched, + combined, + detail::GraphSink(2, 0), + mpi::PeerPlan{}, + window, + {.threads = threads}); + stage_self(eng, self_leader, self_leader_srcs); + eng.resolve_self_queries(true); + const auto resp_leader = detail::resolve_incoming(remote(remote_leader), + op, + true, + matched, + combined, + eng.sink, + {.threads = threads}); + const size_t after_leader = op.size(); + stage_self(eng, self_follower, self_follower_srcs); + eng.resolve_self_queries(false); + const auto resp_follower = detail::resolve_incoming(remote(remote_follower), + op, + false, + matched, + combined, + eng.sink, + {.threads = threads}); + const size_t after_follower = op.size(); + (void)eng.finish(CosMask{}, nullptr); + + // Expected physical order: remote leader misses, remote follower misses, self leader, self follower. + std::vector> expect(seeds); + const auto misses = [&](const std::vector> &k, const std::vector *srcs) { + for (size_t q = 0; q < k.size(); ++q) { + if (srcs != nullptr && matched.is_marked((*srcs)[q])) { + continue; + } + if (std::ranges::find(seeds, k[q]) == seeds.end()) { + expect.push_back(k[q]); + } + } + }; + misses(remote_leader, nullptr); + BOOST_TEST(after_leader == expect.size()); + misses(remote_follower, nullptr); + BOOST_TEST(after_follower == expect.size()); + misses(self_leader, nullptr); + misses(self_follower, &self_follower_srcs); + BOOST_TEST((rows_of(op) == expect), "budget " << threads); + std::vector> r; + r.emplace_back(resp_leader[mpi::WindowIndex{1}].begin(), resp_leader[mpi::WindowIndex{1}].end()); + r.emplace_back(resp_follower[mpi::WindowIndex{1}].begin(), resp_follower[mpi::WindowIndex{1}].end()); + for (const auto &m : rows_of(op)) { + r.push_back(key_of(m)); + } + for (const auto &slot : graph_state(eng.sink)) { + r.push_back(slot); + } + results.push_back(r); + } + for (size_t t = 1; t < results.size(); ++t) { + BOOST_TEST((results[0] == results[t]), "budget " << t + 1); + } +} diff --git a/docs/content/docs/features/parallelism.mdx b/docs/content/docs/features/parallelism.mdx index 982a7025..0a4aab8c 100644 --- a/docs/content/docs/features/parallelism.mdx +++ b/docs/content/docs/features/parallelism.mdx @@ -61,16 +61,22 @@ multi-partition runs are unchanged. (`omp_get_max_threads()`, normally `OMP_NUM_THREADS`) applies. The budget is not clamped to the cores you have; placement belongs to `OMP_PLACES`, `OMP_PROC_BIND` and your launcher. Copies and retained callables keep the budget, and a later change to the environment only affects new objects. -- **Threaded kernels so far.** Three kernels use the budget: the per-gate cosine scaling (graph +- **Threaded kernels so far.** These kernels use the budget: the per-gate cosine scaling (graph replay during evaluation and contraction, and the two-pass path of in-place propagation), the - rotation apply of in-place `propagate`, and the bitmap scan that finds each gate's anticommuting - terms and emits their partner queries during graph construction and in-place `propagate`. Each - splits its work into fixed blocks of 1024 bitmap words or 1024 rotation records and stays serial - below two blocks, so small operators see no threads. The scan's word ranges are merged back in - source order, so queries, new term IDs and graphs are the same at every budget, and results are - bitwise identical. The scan stays serial for a cutoff other than the built-in length or support - cutoffs, including a `basis_change` cutoff. Partner lookup, insertion, graph packing, reductions - and the gradient's endpoint work are still serial. + rotation apply of in-place `propagate`, the bitmap scan that finds each gate's anticommuting + terms and emits their partner queries, and the partner lookup that follows it, during graph + construction and in-place `propagate`. Each splits its work into fixed blocks (1024 bitmap words, + 1024 rotation records, 256 looked-up partners) and stays serial below two blocks, so small + operators see no threads. The scan's word ranges are merged back in source order, so queries, new + term IDs and graphs are the same at every budget, and results are bitwise identical. The scan + stays serial for a cutoff other than the built-in length or support cutoffs, including a + `basis_change` cutoff. +- **Partner lookup.** A rank looks up the partners it owns itself in windows of up to 4096, and + checks and decodes the partner queries other ranks send it before looking them up. Both lookups + read a store that nothing changes meanwhile. New terms still get their IDs in a fixed order and + are inserted by one thread, in the same rounds as before, and a received query stream that is + truncated or not canonically encoded is rejected before it is decoded. Insertion, graph packing, + reductions and the gradient's endpoint work are still serial. - **Calling thread.** Every call that can use MPI (construction, operations, retained callables) must come from the thread that initialized MPI, whatever thread support MPI provides. A call from another thread prints a diagnostic and aborts that process without communicating; the launcher From a6a995a4352bc5a9494acc4c13df33aa61ef84ef Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Roberto=20Di=20Remigio=20Eik=C3=A5s?= Date: Tue, 29 Sep 2026 11:45:21 +0000 Subject: [PATCH 21/64] =?UTF-8?q?docs(parallelism):=20=F0=9F=93=9D=20recor?= =?UTF-8?q?d=20Task=206=20and=20the=20decision=20to=20re-design=20with=20s?= =?UTF-8?q?harding?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Record the Task 6 outcome and the post-Task-6 architecture go/no-go: - the Task 6 contracts, RED/GREEN, mutation, stress, sanitizer and cross-revision evidence; - the phase diagnostic, including the libgomp high-budget regression from small self-probe teams; - the architecture diagnostic: parity at L1, 0/16 reference cells at runtime <= 1.00 at full machine (2-62x), and no credible bounded route to parity; - the owner's decision: the one-store design is abandoned before the Task 10 cutover and the architecture is re-designed keeping the sharding. Tasks 7-13 are marked superseded. Assisted-by: Pi:claude-opus-5-5 --- .../plans/2026-09-18-rank-local-openmp.md | 86 ++++++++++++++++++- 1 file changed, 82 insertions(+), 4 deletions(-) diff --git a/docs/superpowers/plans/2026-09-18-rank-local-openmp.md b/docs/superpowers/plans/2026-09-18-rank-local-openmp.md index d4d04a4d..b09fc9be 100644 --- a/docs/superpowers/plans/2026-09-18-rank-local-openmp.md +++ b/docs/superpowers/plans/2026-09-18-rank-local-openmp.md @@ -22,6 +22,13 @@ Boost.Test, pytest. Monoprop's direct hwloc dependency is removed with the legac **Spec:** [Rank-local OpenMP design](../specs/2026-09-18-rank-local-openmp-design.md). Read both documents before execution. +**Decision after Task 6 (owner, 2026-09-29):** the one-store design is not pursued further. The post-Task-6 +architecture go/no-go showed it clearly trailing the partition baseline at full machine, with no credible bounded route +to parity (see "Architecture go/no-go outcome" after Task 6). The architecture will be re-designed and the re-design +keeps the sharding: thread-owned operator stores stay. Tasks 7–13 below describe the abandoned one-store path and are +superseded; do not execute them. Tasks 0–6 remain as the record of the experiment, and Task 1's frozen campaign and +formal baseline stay the reference for the re-design. A new spec and plan are required before further implementation. + **Status:** Planning only. Refreshed source baseline: current `origin/main` at `90d57177c2b0cd93503f88dd931d2f8dd409aa23`, inspected in rebased `refactor-parallelization` at `35f359f095fa3f1d628ca2fdc66c93fef6ea7564`. Only CONTEXT.md and these planning documents differ between those commits. @@ -144,10 +151,10 @@ Additional execution rules: | Architecture experiment | 0 | Done: evidence inconclusive for ownership; one store per rank is the working hypothesis (see Task 0 outcome) | | Baseline (conditional) | 1 | Task 0 decision, authorized baseline-only calibration, frozen inventory, fresh formal baseline evidence | | Small prototype | 2–4 | OpenMP packaging works; one-store serial and threaded replay agree | -| Shared-store construction | 5–7 | Stable queries/IDs, safety, construction scaling and scratch measured; post-Task-6 architecture go/no-go | -| Remaining evaluation kernels | 8–9 | Replay, derivative, retained closures and reductions validated | -| Integration cutover | 10–11 | Go/no-go passed before Task 10; parity at final geometry decides ownership; index A/B in Task 11 | -| Removal/release | 12–13 | Only after Task 11 confirms the single store; old runtime removed, final full matrix and parity rerun | +| Shared-store construction | 5–7 | Tasks 5–6 done; post-Task-6 go/no-go ran and failed (see its outcome); Task 7 superseded | +| Remaining evaluation kernels | 8–9 | Superseded by the post-Task-6 decision to re-design while keeping the sharding | +| Integration cutover | 10–11 | Superseded: the one-store path does not proceed to the API cutover | +| Removal/release | 12–13 | Superseded: the partition runtime is kept | One writer per checkout. Kernel and storage tasks are ordered: do not have agents concurrently edit `Engine.h`, `Evolution.cpp`, or `MonomialPropagator.inl`. Independent read-only reviews are safe. Do not batch past a failed gate. @@ -1930,6 +1937,77 @@ they cannot waive a Task 11 cell. Report per-phase time and peak memory. If the bounded Task 11 optimization explains a route to parity, stop for the owner's decision before Task 10 removes the partition API. This is the last cheap point to revert: only the additive one-store path would be dropped. +**Task 6 outcome (2026-09-29):** correctness gates pass on GCC 15.2 (R, M) and Clang 18.1.8/libomp (R) Linux; +other platforms are pending. The performance findings below are recorded, not accepted: the work ended at the +architecture decision that follows. +- Contracts as implemented: + - `probe_frozen_positions` splits position-list queries into 256-query blocks, each one `find_batch_positions` + call with absolute offsets into the shared buffer and sliced counts, outputs and hashes. It checks equal + lengths and in-bounds spans before any block runs. The store stays frozen until every worker joins. + - `probe_incoming_queries` validates every sender stream on the caller before any decode, with + `QueryWire::checked_extent`: ternary phase, k and gap-width bounds, canonical escaped counts, whole records + including the fused value word, and a final offset equal to the buffer size. It checks the prefixes and + allocates each array once at its exact size. Workers decode disjoint blocks; `read_positions` rejects any + size_t position at or above 2N before narrowing. After the frozen probe, the caller checks the row-ID ceiling + and assigns misses in sender-window/query order. `miss_g` is `vector`. Malformed input throws + `MalformedQueryStream`; the plan's `0x8037e` fixture is rejected before any payload read. + - `resolve_incoming` scatters on workers only for a sink with `static constexpr bool parallel_resolve = true` + and a true `parallel_resolve_enabled()` (`GraphSink`, Heisenberg `ContractSink`). Schrödinger scoring and + generic sinks stay serial; debug builds assert unique leader marks. It runs inside `guard_distributed`. + - The self lookup runs in windows of at most 4096 queries (at most 16 probe blocks), with serial filtering, + gathering and emission, and no insertion between windows. Publication order and the single-writer + grow → `set_positions` → `bulk_insert_hashed` → reindex sequence are unchanged. `bulk_insert_hashed` gained + a trailing `parallel::Options` that the packed index ignores. The duplicate-key behaviour is pinned by + `bulk_insert_duplicate_keys_are_both_indexed`. + - The observer seam gained `KernelRange::{decode,incoming_probe,self_probe,scatter}`, forwarded through + `build_layer` and `LayerBuildEngine`. +- Tests: + - New and extended cases in `sparse_resolve_tests`, `sparse_query_tests`, `operator_index_tests`, + `mpi_fresh_insert_equivalence` and `fused_cos_sweep_tests`. + - Four `mpi_failure_resolve_worker_*` scenarios: the self probe fails before the exchange; the decode, incoming + probe and scatter fail during the response round. All abort in about 0.4 s. + - RED: compile failure without the helper and API, then a malformed-stream test failing with the unchecked + walk, and participation failing with the regions forced serial while exactness passes. Nine mutants were + all killed. + - Serial and threaded results are bitwise equal at budgets 1–4 and 8, and at world sizes 2–4. The Task 5 + cross-revision probe is bitwise equal at budgets 1, 4 and 16. + - Final runs: R, Clang and ASan ctest 392/392; TSan (Clang + libomp + Archer, same suppression) 117/117 with + no reports; M serial 422/422, MPI splitmix 18/18, linear 17/17; `--with-mpi` pytest passes. Stress: 800 + fresh processes, no failures. +- Phase diagnostic (owner-approved; 23 of 60 min): end-to-end build at T=96 is 4.1× (B1), 8.4× (B2) and 1.5× (B4) + over T=1. Serial fractions at T=1 bound speedup at 5.9×, 15.5× and 4.4×. Hash publication and reindex take 6% of + B1's time at T=1 and 28% at T=96. Under libgomp at T≥48, Task 6 is slower than Task 5 (B1 +19%, B2 +42% at T=96) + because many 16-thread self-probe regions interleave with 96-thread ones. libomp shows no regression, and the + serial self probe restores Task 5 times. This was left unfixed. +- Artifacts: `/home/ubuntu/task6-artifacts/` (`report.md`, `ledger.md`, `design.md`, `diag-results.md`, + `arch-results.md`, `diag/`, `arch/`, `logs/`). They are intentionally uncommitted. The frozen campaign and + workload files, the Task 1 artifacts and the baseline binaries and worktrees were verified unchanged. + +**Architecture go/no-go outcome (owner decision, 2026-09-29): re-design, keeping the sharding.** +- The diagnostic (owner-approved; 150 of 180 min) covered 96 frozen cells (`tiny` and `reference`, all 16 nodes, + at mpi-1x1, mpi-1x96 and off-1x96). Each cell had 3 alternating fresh pairs through the Task 1 driver. The + prototype was selected by a recorded `partitions=1` adapter, and results were written outside the formal + evidence. + - Numerics: every cell matched the formal baseline (global keys and coefficients, energies, gradients), with + exact windows. + - L1: runtime ratios 0.94–1.04 (reference); memory at parity. + - Full machine (mpi-1x96, and MPI-off 1x96): 0 of 16 reference cells at runtime ≤ 1.00, with ratios of + 2.0–62× (gradient Schrödinger 62×, gradient Heisenberg 31×, build_graph Heisenberg 7.2×). Peaks were mostly + lower (0.5–0.96), except Hubbard (up to 1.15). +- No credible bounded route: at T=96 the prototype's parallel scan alone takes longer than the baseline's whole + operation (build_graph Heisenberg 0.465 s vs 0.399 s; Schrödinger 1.34 s vs 0.50 s). Removing every remaining + serial phase in Tasks 7–11 would therefore still leave these cells above 1.00. Evaluation would need to be + 11–62× faster. +- Decision: the one-store-per-rank design is abandoned before Task 10's API cutover, so the partition API and + runtime are unchanged. The architecture is re-designed and keeps the sharding. Tasks 7–13 are superseded. The + Task 2–6 changes stay on this branch as the record of the experiment, and the re-design's spec and plan decide + what to keep or revert. Pending that plan, carry forward: + - OpenMP is a required dependency. + - Task 3's per-object budgets and its invalidation and distributed-failure contracts apply to every + ordinary-communicator object. + - The exported functions' trailing `Options` parameter is a C++ ABI change. + - The prototype kernels are reachable only through explicit `partitions=1`. + ### Task 7: Harden rank-wide capacity and optionally parallelize append-only row fill **Files:** From f31174c65d756914d92942a1f0da24ecb2b0e101 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Roberto=20Di=20Remigio=20Eik=C3=A5s?= Date: Tue, 29 Sep 2026 19:45:26 +0200 Subject: [PATCH 22/64] =?UTF-8?q?docs(parallelism):=20=F0=9F=93=9D=20plan?= =?UTF-8?q?=20the=20sharded=20OpenMP=20replacement?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Document owner-local sharding with operation-scoped OpenMP teams and primary-thread MPI. Supersede the one-store architecture while preserving its history and frozen evidence. Keep arithmetic ordering, shard combination, reductions and FMA policy open for separately approved post-landing optimizations, individually or together. Documentation checks pass. Full prek run reports 49 pre-existing notebook Ruff findings, waived by the owner for this documentation-only push. Assisted-by: Pi:gpt-6-astra --- .../plans/2026-09-18-rank-local-openmp.md | 7 + .../plans/2026-09-29-sharded-openmp.md | 662 ++++++++++++++++++ .../2026-09-18-rank-local-openmp-design.md | 7 + .../specs/2026-09-29-sharded-openmp-design.md | 297 ++++++++ 4 files changed, 973 insertions(+) create mode 100644 docs/superpowers/plans/2026-09-29-sharded-openmp.md create mode 100644 docs/superpowers/specs/2026-09-29-sharded-openmp-design.md diff --git a/docs/superpowers/plans/2026-09-18-rank-local-openmp.md b/docs/superpowers/plans/2026-09-18-rank-local-openmp.md index b09fc9be..bb11f874 100644 --- a/docs/superpowers/plans/2026-09-18-rank-local-openmp.md +++ b/docs/superpowers/plans/2026-09-18-rank-local-openmp.md @@ -1,5 +1,12 @@ # Rank-local OpenMP Implementation Plan +> **Superseded implementation path — 2026-09-29.** Use the +> [sharded OpenMP plan](2026-09-29-sharded-openmp.md) and its +> [design](../specs/2026-09-29-sharded-openmp-design.md) for new work. Tasks 0–6, their outcomes and the failed +> architecture checkpoint are preserved below. Task 1's frozen inventory, formal baseline and evidence protocol +> remain the reference imported by the replacement plan. Tasks 7–13 and conflicting historical execution directions +> are superseded, not an authorized queue. New implementation and measurements require separate approval. + > **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or > superpowers:executing-plans to implement this plan task-by-task. Read both documents completely; full-plan execution > requires explicit approval. Task 0 now precedes all downstream work and ends with an owner architecture decision. diff --git a/docs/superpowers/plans/2026-09-29-sharded-openmp.md b/docs/superpowers/plans/2026-09-29-sharded-openmp.md new file mode 100644 index 00000000..664b7842 --- /dev/null +++ b/docs/superpowers/plans/2026-09-29-sharded-openmp.md @@ -0,0 +1,662 @@ +# Sharded OpenMP Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:executing-plans for authorized inline execution, or +> superpowers:subagent-driven-development only when the operator explicitly authorizes delegation. Read this plan and +> its spec completely. Steps use checkboxes. A task approval does not authorize the next task, measurements or +> publication. + +**Goal:** Replace the custom partition runtime with coarse-grained OpenMP execution while retaining sharded storage, +owner-local mutation and the frozen numerical/runtime/memory acceptance contracts. + +**Architecture:** One rank-level propagator owns T internal shard states, one per OpenMP worker. A single team spans +each operation; explicit local phases and published buffers replace child-propagator dispatch and in-process +communicators. MPI's initializing thread is the OpenMP primary and owns physical-rank MPI calls inside that team. + +**Tech Stack:** C++23, required OpenMP CXX, existing Boost and optional MPI, nanobind, Python 3.11+, +uv/scikit-build-core, Boost.Test and pytest. No replacement affinity/topology dependency. + +**Spec:** [Sharded OpenMP execution design](../specs/2026-09-29-sharded-openmp-design.md). + +## Status and authority + +Planning revision at `a6a995a4352bc5a9494acc4c13df33aa61ef84ef`, dated 2026-09-29. The owner approved writing the +replacement documents, not executing this plan. No implementation, build, experiment, delegation, commit or publication +is authorized by these checkboxes. Review subsequent source differences before starting; never reset to a historical +pin. + +This replaces the abandoned one-store Tasks 7–13 in the +[historical plan](2026-09-18-rank-local-openmp.md). Its Tasks 0–6 remain historical evidence, not an unexecuted queue. +Its Task 1 frozen inventory, artifacts and measurement protocol remain authoritative where imported below. New task IDs +are **S0–S8** to prevent confusion with that experiment. Task 6 is not the performance baseline. + +## Global constraints + +- C++23; GCC 14 / Clang 18 minimum; Python 3.11 minimum. +- OpenMP CXX is required, including MPI-off builds and installed consumers. MPI remains optional and OFF by default. +- Retain Linux x86_64, Linux aarch64 and macOS packaging coverage; report untested platforms as pending. +- Keep packed position-list rows, keyless indexing, sparse state, inverted indices and packed graph representation. +- Keep `TermIndex = std::uint32_t`, with its reserved sentinel, per shard. Retain checked count/allocation arithmetic. +- Preserve both bases and pictures, cutoffs, basis changes, propagation, graphs, gradients, paring, copies and updates. +- Remove the custom worker pool, in-process communicator abstractions, custom barriers and direct hwloc dependency. +- Allocation, placement and pinning belong to the launcher/OpenMP configuration. Recommend one worker per physical core. +- Numerical preservation and the frozen runtime/memory gates govern this refactor; neither is waived by a faster kernel. +- Fixed uniform P processes × T threads; T is captured at construction and is exactly the internal shard count. +- No T-consistency collectives/checks, reduced-team handling, resharding, hardware clamp or independent shard-count + knob. +- No nested kernel parallelism initially. Preserve ascending-shard combination and existing MPI reduction association. +- No arithmetic reordering, new floating-point reductions or FMA-policy change during this refactor. These options + remain open after landing, individually or together, under the explicitly separate follow-up at the end of this plan. +- Preserve immutable graph sharing and independent store copies. Raw root accessors work only at T=1. +- C++ naming/style/Doxygen, Python docstrings, generated-file restrictions and API links follow `AGENTS.md`. +- Use traversal for query-producing bitmap work, and prefix-sum for cumulative offsets. Do not call both a scan. +- Build/artifact isolation is mandatory; do not alter baseline environments, archived evidence or unrelated checkout + data. +- Before an authorized push, run `prek run --all-files` and relevant tests; previous notebook waivers do not carry + forward. + +## Evidence pins and scope + +The preserved production baseline is `90d57177c2b0cd93503f88dd931d2f8dd409aa23`. Baseline tools were subsequently +synchronized by overlay without rebuilding its binaries. At this planning checkpoint the local frozen inputs match: + +| File | SHA-256 | +| --- | --- | +| `tools/rank-local-openmp-campaign.json` | `ffde870c77a66c8ce37fb2cc07b39abef976e9d7938b7cea59f43cb407de068a` | +| `tools/rank-local-openmp-workloads.json` | `508b65f1f0eb2171fc4fbd79feba2381054c6dcdd38a614600f5b957ab814416` | +| `tools/benchmark-rank-local-openmp.py` | `2207529fb45454990dfba3ad0e1cf944ba1ae1d15385489aef4b16a365426697` | +| `CONTEXT.md` | `1eaf2d92f23f3d0f349e281363e846f7baa5235415f20117abcad40eb5249d66` | + +The recorded remote baseline index is `/home/ubuntu/task1-artifacts/formal/baseline-index.json`, hash +`82479d6ff83deb3113306ac1caa1720d25f15a400cbc89b5693f85ffad47fff5`. Its identity and raw artifacts have not been +independently reverified during this redesign. Reverify them during separately authorized preflight; do not reconstruct +or overwrite the formal baseline. Preserve the whole `/home/ubuntu/task1-artifacts/` tree and later task reports. + +The 250-cell inventory and five independent <=1.00 median gates remain: runtime, operation peak sum/max, construction +peak sum/max. Initial observations use five distinct fresh-process pairs; failing cells get the specified additional +five. Numerical validation is separate and includes global retained operators. No workload resizing, cell dropping, +geometric-mean masking, memory allowance or substitution of MPI-enabled P=1 for actual MPI-off operation. + +Task 6's full-machine failure motivates two early architecture checkpoints, S4 and S5. Neither is a formal campaign or +permission to spend the old 180-minute budget. Each experiment requires its own written scope, elapsed/per-trial/RAM +limits, allocation/provenance plan and owner approval. Budgets never roll over. Keep diagnostics and formal results +separate. A checkpoint failure stops the plan for an owner decision; it does not silently move optimization into S6. + +## Module and source map + +New private names below are proposed interfaces, not claims about existing source. Install private headers reachable +from public templates using the existing CMake header/export mechanism. Avoid a broad source reorganization. + +| Path | Responsibility | +| --- | --- | +| New `cpp/monoprop/detail/sharded/Team.h` | Operation-scoped team and checkpoint-stable exception publication; no queue, threads or transport ownership | +| New `cpp/monoprop/detail/sharded/State.h` | Stable shard addresses and owner-initialized/copyable mathematical state | +| New `cpp/monoprop/detail/sharded/Construction.h` | Rank-level gate/pass orchestration over shard-local construction phases | +| New `cpp/monoprop/detail/sharded/Evaluation.h` and `Evaluation.cpp` | Replay, reverse pass and ordered rank-local result combination | +| New `cpp/monoprop/detail/sharded/Exchange.h` and `Exchange.cpp` | Physical-MPI staging/request ownership only; no in-process collective facade | +| New `cpp/monoprop/detail/sharded/CMakeLists.txt` | Source/header registration, including exports needed by installed template consumers | +| `cpp/monoprop/detail/evolution/layer_build/{Engine,Resolve,QueryWire,Scan,FusedApply}.h` | Preserve owner-local algorithms; separate pure phases from communication; read-only input views | +| `cpp/monoprop/{Evolution,MPFunctions}.cpp` | Extract reusable local replay/derivative operations and scratch without changing formulas | +| `cpp/include/monoprop/MonomialPropagator.h`, `cpp/monoprop/detail/monomial_propagator/MonomialPropagator.inl` | Rank-level state, public operations, lifecycle, aggregates and exports | +| `cpp/monoprop/detail/mpi/{Comm,Exchange,MPICompat,MPIUtils,OperationFailure,Pairwise,Routing}.h` | Real-MPI ownership, routing cache and failure contracts; remove fake communicator dispatch at cutover | +| `cpp/monoprop/detail/mpi/MPICompat.cpp`, `cpp/monoprop/detail/EnvConfig.h` | Initialization/thread support and removal of partition configuration | +| `cpp/tests/PropagatorTestAccess.h` | Private shard inspection, actual-phase observers and failure injection | +| New `cpp/tests/sharded_{team,state,construction,evaluation,exchange}_tests.cpp` | Focused flat Boost tests for new seams | +| New `tests/test_sharded_openmp.py` | Process-isolated launch geometry, full API and exporter integration | +| `cpp/tests/CMakeLists.txt`, `justfile`, `.github/workflows/test.yml` | Separate T=1 internals and fixed-T sharded test launches | + +Existing high-risk seams to inspect before editing: + +- `Engine.h::run_exchange`, `resolve_self_queries`, `insert_deferred_self_misses`, `finish`, `build_layer`. +- `GraphSink` / `ContractSink`, `IncomingProbe`, `SelfQueryStage`, `SlotWindow` and the Task 6 checked decoder. +- `Evolution.cpp::CrossRankExchangeHandle`, `begin_flat_exchange`, `wait_flat_exchange`, the two exchange finish paths. +- `MPFunctions.cpp::prepare_evolved_operator`, `ev`, `ev_and_grad` and `eval_scratch`. +- `MonomialPropagator.inl::run_operation_`, constructors/copy, `evolved_operator_terms`, functionals and updates. +- `PartitionGroup.h`, `ShmComm.h`, `HybridComm.h`, `PartitionBarrier.h`: algorithm/order reference, not a runtime to + wrap. + +## Shared correctness contracts + +1. A store and its overflow map have one mutation owner at a time. Reserving capacity does not make concurrent + `set_positions` safe. Different stores may mutate concurrently; lazy caches belong to their owner too. +2. At fixed P/T preserve owner mapping, source/query order, per-shard row IDs and graph records. Keep different local + shards distinct from self; combine local and remote sender blocks in ascending flat-owner order. +3. Publication order is cross-owner leaders, cross-owner followers, deferred self leaders, deferred self followers. + Match followers against the completed leader pass. Keep the source boundary distinct from final `scaled_count`. +4. XOR with a fixed generator gives distinct partners for distinct sources. Preserve the existing duplicate-key behavior + rather than silently deduplicating; see invariant 14 in the historical plan and `operator_index_tests.cpp`. +5. Queries remain compact. Keep absolute position offsets, cached hashes, checked wire boundaries/position narrowing and + shard-local ID/sentinel limits. Query ordinals remain `size_t`. Do not restore a rank-wide single-store capacity + limit. +6. Send and endpoint-snapshot buffers are immutable through consumption. MPI request storage and buffer addresses do not + move after posting. Requests outlive region exit; failure handling precedes any destructor that could wait on a peer. +7. All supported workers reach the same checkpoints. Failure decisions are stable for a checkpoint generation; a fast + worker's next-phase failure cannot make a slower worker branch out of the previous phase. Empty shards participate. +8. Preserve self-pair layout (`k`, `k+count/2`), signs, cosine masks/tails, `LayerAngle`, record/restore order and + derivative `A=cos_acc(...)*sec`, `-g*(sin*(A-ep.cos_terms)-ep.sin_terms)`. Keep sparse/duplicate dot and scatter + semantics. +9. No new arithmetic association in S0–S8. Keep `kVanishingCos=0x1p-20`, `kRecordSpreadBits=53.0`, the contraction + workaround and existing tolerances. Preserve Python incremental axes and post-call seeds in both pictures/bases. +10. Validate full retained maps, including zeros/near-cutoff rows. Replicated Heisenberg identity agrees and is included + once. Duplicate nonidentity ownership or nonfinite values fail. General map comparison remains + `abs(a-b) <= 1e-9 + 1e-7*max(abs(a),abs(b))`; stricter existing tests and unrounded scalar/gradient checks remain. + +## Verification commands and task handoffs + +During authorized execution, use fresh R (MPI OFF) and M (MPI ON) environments/builds, outside baseline directories. +The following is a build/test recipe, not an instruction to execute while reviewing the plan. Set absolute `ARTIFACTS` +to the approved task directory. A stable installed Ninja avoids the expired build-isolation Ninja path in old caches. + +```bash +: "${ARTIFACTS:?Set the approved absolute task artifact directory}" +export R_ENV="$ARTIFACTS/venv-r" R_BUILD="$ARTIFACTS/build-r" +export monoprop_NUM_THREADS=1 OMP_NUM_THREADS=1 OMP_DYNAMIC=FALSE +monoprop_ENABLE_MPI=OFF UV_PROJECT_ENVIRONMENT="$R_ENV" \ + uv sync --all-extras --group workspace-test --reinstall-package monoprop --no-cache -v \ + --config-settings-package="monoprop:build-dir=$R_BUILD" \ + --config-settings-package="monoprop:cmake.define.CMAKE_MAKE_PROGRAM=$(command -v ninja)" +cmake --build "$R_BUILD" --target monoprop_unit_tests.x +ctest --test-dir "$R_BUILD" --show-only +ctest --test-dir "$R_BUILD" --output-on-failure --no-tests=error +UV_PROJECT_ENVIRONMENT="$R_ENV" uv run --no-sync python -c \ + 'import monoprop, sys; sys.exit(int(monoprop.has_mpi))' +UV_PROJECT_ENVIRONMENT="$R_ENV" uv run --no-sync pytest +``` + +For M, use distinct `M_ENV`/`M_BUILD` and `monoprop_ENABLE_MPI=ON` in the build environment; verify `has_mpi` is true. +Use the documented `just test-mpi` routes, adapted to the isolated tree when necessary. MPI-off tooling must not import +`mpi4py.MPI`. Keep `--no-sync` when running an existing build. Verify executable/extension/library paths, hashes, +mtimes, actual linkage and regenerated nonempty Boost discovery; `uv sync` alone does not prove the C++ binary was +relinked. +New sources/tests must be registered/reconfigured before claiming a missing-test RED or a passing run. + +Once sharded test registration exists, launch each supported T in a separate process; do not change T inside a test: + +```bash +monoprop_NUM_THREADS=4 OMP_NUM_THREADS=4 OMP_DYNAMIC=FALSE \ + ctest --test-dir "$R_BUILD" --output-on-failure --no-tests=error -R '^sharded_' +monoprop_NUM_THREADS=4 OMP_NUM_THREADS=4 OMP_DYNAMIC=FALSE \ + UV_PROJECT_ENVIRONMENT="$R_ENV" uv run --no-sync pytest tests/test_sharded_openmp.py -v +``` + +These commands require an allocation suitable for four workers. Ordinary low-level tests stay at T=1. A helper-level +limited-team test may remain for `for_blocks`, but it does not define a supported sharded-operation launch. +Run ASan/UBSan separately from TSan. Qualify Clang/libomp/Archer with known-safe/racy probes; unsupported libgomp TSan +is not race-free evidence. Observe actual workers inside real phases; assert only after joining, not from worker bodies. + +Every task ends with a written handoff outside the checkout: source/build identity, changed interfaces, actual RED/GREEN +commands and outcomes, preservation tests, request/memory/ordering review, pending platforms and the next authorization +boundary. A passing old fixture is preservation evidence, not an invented RED. Commits require separate authority and +repository message/trailer/signing policy; no task here contains an automatic git-write step. + +## Task S0: A fixed-team, phase-safe execution primitive + +**Deliverable:** independently tested operation-scoped team/checkpoint code, with no propagator or transport migration. + +**Files:** create `detail/sharded/Team.h`, its `CMakeLists.txt` and `cpp/tests/sharded_team_tests.cpp`; register through +`cpp/monoprop/CMakeLists.txt` and `cpp/tests/CMakeLists.txt`; clarify `detail/parallel/Options.h` documentation. +Paths under `detail/` here and below are under `cpp/monoprop/`. + +**Interfaces:** in `monoprop::detail::sharded`, introduce `TeamFailure` with a constructor taking `size_t threads`, +`record(size_t shard, std::exception_ptr error) noexcept -> void`, `checkpoint() noexcept -> bool`, and +`first_error() const noexcept -> std::exception_ptr`. One error slot per worker; `first_error` selects the lowest +failing shard after joining. Add these templates: + +```cpp +template +auto run_team(parallel::Options options, Fn &&body) -> std::exception_ptr; + +template +auto phase(TeamFailure &failure, size_t shard, Fn &&body) noexcept -> bool; +``` + +`run_team` constructs error storage before entering `omp parallel num_threads(options.threads)`, calls the noexcept +body with `(size_t shard, TeamFailure&)`, then returns the error without rethrowing. `phase` catches its local body, +records its own slot and returns the collective checkpoint decision. Every phase result is checked; false ends the +common sequence without invoking later phase bodies. No MPI, configuration check or topology query. + +- [ ] Add a real-team test and run it before implementation; the absent helper is the initial compile RED: + +```cpp +namespace parallel = monoprop::detail::parallel; +namespace sharded = monoprop::detail::sharded; + +BOOST_AUTO_TEST_CASE(sharded_team_visits_each_owner) { + const auto options = parallel::capture_thread_budget(); + auto visits = std::vector(static_cast(options.threads), 0); + const auto error = sharded::run_team(options, [&](size_t shard, sharded::TeamFailure &failure) noexcept { + sharded::phase(failure, shard, [&] { ++visits[shard]; }); + }); + BOOST_CHECK(!error); + for (const auto count : visits) { + BOOST_CHECK_EQUAL(count, 1); + } +} +``` + +- [ ] Implement a checkpoint-stable decision, not a single asynchronously changing stop flag. A valid initial shape + has owner-written `errors_` and primary-written `proceed_`: + +```cpp +auto TeamFailure::checkpoint() noexcept -> bool { +#pragma omp barrier +#pragma omp master + { + proceed_ = std::none_of(errors_.begin(), errors_.end(), [](const auto &error) { + return static_cast(error); + }); + } +#pragma omp barrier + return proceed_; +} +``` + + The next decision cannot be written until every worker reaches the next leading barrier. Integrate these boundaries + with handoff needs instead of layering an unrelated failure protocol on top. Add Qt-style declarations/member docs. + Clarify `Options` as a requested budget: the reduced-team guarantee belongs to `for_blocks`, not every consumer. + The sharded team requires exactly T; this documentation change must not add actual-team checks or weaken the generic + helper's existing behavior. +- [ ] Test T=1/2/4, empty work, nonprimary failure, simultaneous failures, primary-only work, skipped later mutation and + delayed-worker checkpoint stability. Add a mutation check that removes synchronization/actual work and makes the + corresponding tests fail; use bounded test timeouts rather than treating a hang as success. +- [ ] Run the focused R tests with `-R '^sharded_team_'`, qualified sanitizer probes and installed-header compilation. + Keep whole-body callbacks noexcept; every potentially throwing operation belongs inside a protected phase. +- [ ] Hand off the helper and barrier/lifetime argument. Do not add MPI or start an experiment in this task. + +## Task S1: Owner-initialized shard state and copy semantics + +**Deliverable:** internal state can be seeded, copied, counted and destroyed without a pool or child propagators. + +**Files:** create `detail/sharded/State.h`, `cpp/tests/sharded_state_tests.cpp`; adapt initialization/copy extraction in +`MonomialPropagator.inl`, `detail/operator/MPOperator.h` and private `PropagatorTestAccess.h` where needed. Do not +switch the public runtime yet. + +**Interfaces:** `ShardState` contains the existing `MPOperator`, `MPGraph` and `MatchedEpochSet` +state, not another `MonomialPropagator`. Use stable `std::unique_ptr` addresses. Define +`Shards = std::vector>>` and a construction helper: + +```cpp +template +auto make_shards(parallel::Options options, Initialize &&initialize) -> Shards; +``` + +The initializer consumes `size_t shard` and returns `std::unique_ptr>`; it contains only +owner-local initialization. Caller-side configuration/router/MPI setup precedes it. New state and source copies +preserve captured T. + +- [ ] Add fixtures for each basis/picture, empty owners, replicated identity, source mutation after copy, invalid-source + rejection and constructor failure. Verify the union of shard keys matches the existing partition oracle at fixed P/T. + For each owner, the test checks `router.dest(key)` against its flat owner ID. +- [ ] Implement first-touch with only the pointer vector allocated on the caller; the protected owner body allocates the + substantial state. The helper's central pattern is: + +```cpp +auto shards = Shards(static_cast(options.threads)); +const auto error = run_team(options, [&](size_t shard, TeamFailure &failure) noexcept { + phase(failure, shard, [&] { shards[shard] = initialize(shard); }); +}); +if (error) { + std::rethrow_exception(error); +} +return shards; +``` + + Caller distributed guards must still abort peers on failed distributed initialization. No requests may be owned by + this helper when it rethrows. Audit allocating `noexcept` constructors, including the current default `MPOperator` + constructor, before claiming catchable allocation-failure coverage. +- [ ] Extract seeding rather than copying full child constructors. Preserve hash ownership, streamed Schrödinger paired + basis generation, per-shard reserve and lazy-cache ownership. Reuse existing deep-store-copy and + immutable-core-sharing semantics. Preserve rank-level virtual clone/update hooks; do not recreate + `PartitionChildFactory` privately. +- [ ] Run `-R '^sharded_state_'` at supported separate launches; prove stores are distinct, owner allocation occurs in + the owner body, copied budget is unchanged and earlier independent copies survive invalidation of another instance. +- [ ] Hand off memory accounting for state, graph sharing and retained scratch. No global operator mirror or benchmark. + +## Task S2: Direct-buffer construction and graph-free propagation + +**Deliverable:** a complete P=1, T-shard construction/propagation engine with testable local phase boundaries. + +**Files:** create `detail/sharded/Construction.h`, `cpp/tests/sharded_construction_tests.cpp`; extract phases from +`detail/evolution/layer_build/Engine.h`; adapt `Resolve.h`, `QueryWire.h`, `Scan.h` and `FusedApply.h` only as needed. +Retain `sparse_query_tests.cpp`, `sparse_resolve_tests.cpp`, `operator_index_tests.cpp`, `majorana_cutoff_tests.cpp`, +`pauli_build_layer_tests.cpp` and fused tests as preservation coverage. + +**Interfaces:** retain `LayerBuildEngine`, `GraphSink` and `ContractSink`, but move blocking transport out of the owner +engine. Existing legacy calls may use an adapter while the old runtime is retained. Introduce these owner-local methods +on `LayerBuildEngine`; `Response` is `typename Sink::Response`: + +```cpp +auto prepare_exchange(bool leaders, + mpi::WindowVec &&queries, + mpi::WindowVec> &&sources, + mpi::WindowVec> &&values, + SelfQueryStage &&self_stage) -> const mpi::WindowVec &; +auto resolve_published(const mpi::WindowVec> &incoming, bool leaders) + -> mpi::WindowVec>; +auto consume_published(const mpi::WindowVec> &responses) -> void; +``` + +`prepare_exchange` performs the existing stable follower filtering/self resolution and returns the sink's outgoing +buffer. `resolve_published` performs checked decode/probe/scatter/publication within one shard. `consume_published` +preserves response order and leader marks. Existing `finish(CosMask&&, CosMask*)` remains the finalization seam. +These methods perform no MPI, barrier or nested worksharing; `SlotWindow` stays a data-layout type, not a communicator. + +- [ ] Add differential fixtures with same-shard, other-local-shard and empty destinations; mixed hits/misses; all-hit, + all-miss, fresh growth/reindex; Plain/Fused forms; both sinks; and fixed-geometry row/graph/query/response equality. + A genuine RED must show missing sharded behavior/owner participation, not merely rename passing serial assertions. +- [ ] Split `run_exchange` at the above boundaries and adapt incoming containers to immutable spans without duplicating + whole local query buffers. `QueryWire::WireView` already exists; preserve its checks and generalize the remaining + owning-container callers. Use `window.indices()`/`window.slot(wi)`/`at_slot(flat)` correctly. +- [ ] Implement leader then follower orchestration with published views. The pass order is executable structure: + +```text +owner traversal -> publish outgoing -> resolve destination-owned queries -> publish responses +-> consume responses -> complete leader marks -> repeat for followers +-> insert deferred self misses -> finalize graph/fused updates +``` + + Each arrow is a lifetime/synchronization boundary, not a new local collective API. Fixed source order must match + `ShmComm`'s historical order. Pass serial `parallel::Options{}` to reused within-shard kernels. +- [ ] Keep same-shard position stages unencoded, bounded self lookup and deferred hashes. Preserve scatter before + insertion, graph versus fused response types, Schrödinger scoring, cosine tails/pivots and paired cutoff exceptions. + Retain the malformed `QueryWire<128>` Plain fixture `VecZ{size_t{0x8037e}}` and checked overflow tests. +- [ ] Test throw sites at traversal, decode, publication and finalization. Observers record actual shard owners in those + bodies; no empty-region proxy. Test opaque callback/basis-change supported paths without assuming thread safety. +- [ ] Run `-R '^(sharded_construction_|sparse_|operator_index_|fused_|pauli_)'` plus relevant cutoff tests, verifying + the selection is nonempty and broad enough. Hand off exact fixed-geometry/global-map evidence and buffer live ranges. + +## Task S3: Snapshot-safe replay, energy, gradients and paring + +**Deliverable:** complete P=1 sharded evaluation, including reverse derivatives and retained/pared functionals, without +cross-shard live-coefficient reads or per-kernel OpenMP regions. + +**Files:** create `detail/sharded/Evaluation.h`, `Evaluation.cpp`, `cpp/tests/sharded_evaluation_tests.cpp`; extract +local functions from `Evolution.cpp`, `MPFunctions.cpp`, `detail/evolution/CosineRecompute.h` and callback plumbing. +Integrate graph views from `detail/pare/PareGraph.cpp`; preserve `combined_recompute_equivalence.cpp`, +`pare_graph_tests.cpp`, `evolution_detail_tests.cpp` and `tests/test_infinite_cutoff.py`. + +**Interfaces:** use one `EvalRequest` and `detail::CosCallbacks` per shard with this rank-level detail interface: + +```cpp +auto ev_sharded(std::span requests, + std::span callbacks, + parallel::Options options, mpi::Comm comm) -> double; +auto ev_and_grad_sharded(std::span requests, + std::span callbacks, + parallel::Options options, mpi::Comm comm) -> std::pair; +``` + +These own one team for the entire call. Initially the physical communicator has size one; S5 adds remote legs without +changing callers. Requests/callbacks are prebuilt views with T entries and no shard-count setter. Keep existing exported +single-store functions as low-level compatibility/reference paths where appropriate; new detail functions need exports +if public template instantiation calls them across the shared-library boundary. + +- [ ] Add exact small shard-pair fixtures where one owner overwrites a coefficient before another consumes its old + endpoint; only a published snapshot can pass. Cover empty owners, identity once, sparse/duplicate dot/scatter and + repeated parameter indices, then finite-difference/reference gradients and pared/unpared functionals. +- [ ] Extract pure owner-local operations from the existing replay/derivative begin/apply/finish functions. Publish only + needed endpoint values, not a full new operator mirror. Preserve this ordering: + +```text +snapshot/pack all required endpoints -> publish -> independent self/local work +-> consume peer snapshots -> cosine/record/restore phases in their original order +-> local dot/gradient contributions -> ascending-shard fold -> scalar/vector result +``` + + Keep each self pair together; no adjacent-pair reinterpretation. Forward and derivative scratch must stay valid + across phases. Reuse legitimate mathematical snapshots; account for retained TLS versus per-instance scratch. +- [ ] Implement ordered local combination. For every result position, the reference fold is: + +```cpp +auto total = 0.0; +for (size_t shard = 0; shard < contributions.size(); ++shard) { + total += contributions[shard]; +} +``` + + Add core/identity only once using existing semantics. Do not parallelize this fold with a new floating-point + association, change `std::fma` policy or remove `[[gnu::noinline]] apply_fused_record_range` without neutrality proof. +- [ ] Verify record thresholds, vanishing cosine, restoration after derivative calls, signs in both pictures/bases, + basis changes, incremental graph axes, repeated evaluation, copies, updates and functional owner lifetimes. +- [ ] Run `-R '^(sharded_evaluation_|pare_|combined_|evolution_)'`, the named Python numerical suites and qualified + sanitizers. Compare retained operators as well as energy/gradient; performance work is not part of this task. + +## Task S4: Integrated MPI-off prototype and first architecture checkpoint + +**Deliverable:** public mathematical operations use the new P=1 sharded engine in an isolated development build, with +credible end-to-end numerical/memory/runtime evidence or an explicit stop decision. + +**Files:** `MonomialPropagator.h`, `MonomialPropagator.inl`, `detail/sharded/{State,Construction,Evaluation}.h`, +`src/monoprop/bindings/binder.h`, `cpp/monoprop/CMakeLists.txt`, `cpp/tests/PropagatorTestAccess.h`; +create `tests/test_sharded_openmp.py`; adapt `tools/benchmark-rank-local-openmp.py`, +`tests/test_rank_local_openmp_benchmark.py` and +`packages/monoprop-bench-tools/src/monoprop_bench_tools/{models,preflight,bmf}.py` for candidate construction. + +**Interfaces:** connect the S1 state, S2 engine and S3 evaluation to existing root build/propagate/functional/update +entry points. A temporary compile-time `monoprop_SHARDED_OPENMP_PROTOTYPE` option selects the integrated candidate; +normal builds retain the legacy path until cutover. This is development instrumentation, not a public runtime setting +or permanent backend selector. Old constructor controls may remain in the legacy build during coexistence, but cannot +select a different shard count in the candidate. Explicit nondefault legacy controls must not silently be honored there. + +- [ ] Add separate-process T=1/2/4 Python cases through the real public API. One concrete core fixture is: + +```python +from pathlib import Path + +import pytest + +from monoprop import MajoranaPropagator +from tests.cases import load_problem + + +def test_sharded_propagation_matches_graph() -> None: + problem = load_problem(Path(__file__).parent / "data" / "random_exact.msgpack") + circuit = problem.monomial_circuit.to_circuit() + graph_sim, direct_sim = [ + MajoranaPropagator( + problem.operator, problem.monomial_circuit.initial_state, cutoff=2 * problem.n_modes + ) + for _ in range(2) + ] + graph_sim.build_graph(circuit) + direct_sim.propagate(circuit) + assert graph_sim.expectation_value(circuit) == pytest.approx( + direct_sim.expectation_value(), rel=0, abs=1e-12 + ) +``` + + This is preservation coverage; sharded ownership/participation needs its own genuine RED. Also compare decoded full + operators, unrounded energies and every gradient component. Across T, use subprocess output maps rather than changing + configuration mid-process. +- [ ] Integrate rank-level state and validity, count/memory aggregation and direct shard iteration in + `evolved_operator_terms()`. Pair each shard index with its own evolved vector; do not use root `indexing()` at T>1. + Include retained zeros, identity de-duplication and count/export failure tests before timing any result. +- [ ] Record a migration ledger for all old facade/one-store tests: kept at T=1, rewritten shard-aware, or retired with + equivalent coverage. Prove actual participation inside construction, publication, replay and derivative phases. + Audit trial selection from binary/build provenance, not `--runtime-shape openmp` or environment labels alone. +- [ ] Run integrated R correctness, installed-consumer smoke and M-with-P=1 correctness. Public P>1 candidate support is + not claimed before S5. Update the candidate builder to use the new root constructor instead of `partitions=1`; + retain the legacy partition builder for the baseline. Keep `observe`, `validate`, `compare` and the + allocation-before-import fix. Record/synchronize any approved tool overlay without rebuilding baseline binaries or + rewriting formal records. If an adapter/schema change cannot consume the frozen evidence honestly, stop for review. +- [ ] **Stop for experiment approval.** Propose matched baseline/candidate single-thread controls and actual MPI-off + full-machine reference cells spanning build_graph, propagate, energy and gradients, both bases/pictures. Use frozen + builders/parameters and operation/construction memory windows. Specify repetitions, task/per-trial/RAM caps and + first-touch/placement evidence before running; do not reuse Task 6's allowance. +- [ ] If authorized, run through the existing driver against preserved partitions, not just candidate T=1. Report every + runtime and memory cell plus full numerical validation. Single-kernel speedup is not the checkpoint. Archive results + outside the checkout; retain natural first-touch effects as architectural evidence. +- [ ] Obtain the owner's written proceed/rework/stop decision before S5. Failed parity needs a bounded proposed remedy, + not an automatic move to runtime deletion. The later frozen campaign is still mandatory after an early pass. + +## Task S5: Physical-MPI exchange and the MPI+OpenMP checkpoint + +**Deliverable:** supported P×T candidate with primary-only MPI, owner-parallel payload work and failure-safe requests. + +**Files:** create `detail/sharded/Exchange.h`, `Exchange.cpp`, `cpp/tests/sharded_exchange_tests.cpp`; extend S2/S3 +orchestration; adapt `detail/mpi/{Pairwise,Routing,MPIUtils,OperationFailure}.h`, `MPICompat.cpp`, +`cpp/tests/mpi_failure_driver.cpp`, `run_mpi_failure_scenario.cmake`, `mpi_fresh_insert_equivalence.cpp` and +`mpi_distributed_layer_equivalence.cpp`. + +**Interfaces:** a move-only physical exchange owns real-MPI request objects and stable staging buffers outside the team. +Expose physical post/wait operations to the primary and disjoint payload views to owners, not fake collective methods +on a shard. Reuse `begin_flat_exchange` / `wait_flat_exchange` and existing pending-request semantics when extracting +this code. The S2 owner-phase and S3 rank-evaluation signatures remain unchanged. + +- [ ] Add two-process tests with T=1/2/4 for mixed local/remote destinations and canonical sender ordering. Check that + local sender blocks do not precede lower-numbered remote senders. Verify response source/query alignment, empty + messages, nonzero slot windows and exact pre-update endpoint values. +- [ ] Define one staging layout with checked counts and prefix-sum offsets for `(source shard, destination shard)` + segments per physical peer. Workers write disjoint payload slices; the primary exchanges metadata/posts/waits and + publishes received extents; destination owners decode and mutate. Never serialize all payload processing on shard 0. +- [ ] Preserve construction's linear peer window and replay's actual multi-peer summaries. Warm/use communicator-agreed + `routes_pairwise` on the caller/primary, including MPI attribute accesses. Keep mode/seed agreement and remove the + partition-count agreement field rather than replacing it with T agreement. No geometry-repair fallback. +- [ ] Implement request lifetime explicitly. The owner of `pending` and every referenced buffer surrounds the team: + +```text +allocate operation frame and request owners +run_team: protected owner/primary phases, post -> independent work -> wait -> dependent work +inspect returned error while request owners are still alive +if error: invalidate after mutation; multi-rank abort, otherwise rethrow +only then permit normal request/buffer destruction +``` + + A primary post may fail after some requests became live; those requests still belong to the outer frame. Do not + return through a draining local destructor before reaching distributed failure handling. +- [ ] Adapt failure scenarios for worker throw, before-exchange throw, active-ticket throw, malformed receive and + wrong-thread entry. Supervised two-process runs must finish in failure before the 30-second timeout; timeout fails the + test. Single-process rethrow/invalidation and independent-copy survival must also pass. Test actual MPI support, not + merely the requested level. Keep SERIALIZED until S8 removes executable legacy workers. +- [ ] Run linear supported geometries and splitmix with three processes, numerical/full-map comparisons and qualified + MPI sanitizer runs. Assert MPI occurs only on the initializing primary; no arbitrary `omp single` winner may post. +- [ ] **Stop for a separate bounded measurement approval.** Propose matched MPI-only, MPI+OpenMP and single-thread + controls plus relevant full-machine cells. Use the same partition baseline and all numerical/memory surfaces. Obtain + an owner proceed/rework/stop decision before API cutover; do not claim multi-node qualification from ordinary EC2 + links. + +## Task S6: Final public surface, exports and test migration + +**Deliverable:** candidate has the complete intended interface and coverage; no test depends on a one-store facade +at T>1. + +**Files:** `MonomialPropagator.h`, `MonomialPropagator.inl`, `src/monoprop/bindings/binder.h`, +`src/monoprop/monomial_propagator.py`, `detail/EnvConfig.h`, `cpp/tests/{unit_tests,partition_equivalence_tests, +openmp_runtime_tests,fused_cos_sweep_tests,mpi_fresh_insert_equivalence,mpi_failure_driver}.cpp`, +`cpp/tests/PropagatorTestAccess.h`, `cpp/tests/link_export_probe/link_export_probe.cpp`, `cpp/tests/CMakeLists.txt`, +`justfile`, `.github/workflows/test.yml`, `tests/test_openmp_config.py`; documentation below. + +**Interfaces:** no `partitions`, `child_factory`, `PartitionChildFactory` or `monoprop_PARTITIONS` reader in the +candidate. +No replacement constructor/setter. Single-shard raw accessor rejection uses launch-time T, not a user shard selector. +Mathematical operations, virtual clone/update hooks, aggregate reporting and decoded exports retain their contracts. + +- [ ] Add compile/API tests proving old explicit partition/factory signatures are absent from the candidate, Python + construction has no new thread/shard argument, and budget parsing/capture/copy use the existing environment contract. +- [ ] Implement single-shard access with an explicit T>1 exception and a T=1 remedy. Do not fabricate raw merged + storage. Update the old `MultiPartitionUnsupported` diagnostic/type as part of the documented C++ break. +- [ ] Migrate the audit's real call sites and tests, preserving coverage. Export through shard indices/coefficient + vectors; fixed-geometry graph-layout tests inspect shards privately. Cross-geometry tests compare full retained maps. + Retain actual-participation, capacity, malformed-input, fresh-insertion, copy/clone/update and failure-lifetime tests. +- [ ] Remove the test main's `monoprop_PARTITIONS=off` assignment; ordinary C++/Python tests launch at T=1. Register + dedicated fixed-T launches without `ScopedBudget` mutation in integration tests. Do not use `OMP_THREAD_LIMIT