Skip to content

pir: update to pir-engine 3.7.0 - #234

Open
Qazi-pk wants to merge 2 commits into
cavalab:masterfrom
Qazi-pk:pir-3.6.0
Open

Qazi-pk wants to merge 2 commits into
cavalab:masterfrom
Qazi-pk:pir-3.6.0

Conversation

@Qazi-pk

@Qazi-pk Qazi-pk commented Oct 20, 2026

Copy link
Copy Markdown
Contributor

This updates the pir method to pir-engine 3.6.0, which adds optional fallback "gates" that run after the classical engine. The engine itself and its configuration are unchanged.

Changes

  • algorithms/pir/install.sh: install tag v3.6.0 (was v3.5).
  • experiment/methods/pir/regressor.py: pass gates="all" to the engine. gates is an explicit constructor argument, so it is recorded by get_params().
  • algorithms/pir/metadata.yml: describe the gates, add the software DOI, and remove an outdated benchmark figure from the description.

What the gates do (tried in this order, only when the engine's model is not already exact)

  • rational P(x)/Q(x): accepted only if it reproduces the data exactly (max relative error < 1e-6) and beats the engine
  • exponential C*exp(sum of monomials): same exactness rule
  • noise-tolerant power law C*prod x_j**(p/q): exponents snapped to simple fractions; accepted only if its training MSE is no worse than the engine's

Any error inside a gate keeps the engine's model. All thresholds are fixed in the engine.

Checks

  • ruff check, ruff format --check and scripts/typecheck.py pass locally.
  • Release: https://github.com/Qazi-pk/physics-engine/releases/tag/v3.6.0 (DOI 10.5281/zenodo.23460311), with tests and CI.
  • I ran the 119 Feynman datasets with exactly this configuration (fitted with SRBench's evaluate_model.py, scored with assess_symbolic_model.py from current master 2aabbb6, seeds 23654 and 78901). These are my own runs, not official results:
target noise v3.5 3.6.0 (23654 / 78901)
0 35 / 35 50 / 50
0.001 35 / 35 36 / 36
0.01 30 / 29 35 / 35
0.1 2 / 2 23 / 22

(symbolic solution counts out of 119; no dataset solved by v3.5 was lost at any level)

Happy to adjust anything.

🤖 Generated with Claude Code

https://claude.ai/code/session_019mo31AVugYdfXietM73iWZ

- algorithms/pir/install.sh: install tag v3.6.0 (was v3.5).
- experiment/methods/pir/regressor.py: pass gates="all" to the engine;
  gates is an explicit constructor argument so get_params() records it.
  The vanilla engine configuration is unchanged.
- algorithms/pir/metadata.yml: describe the gates; add the software DOI;
  drop the outdated Tier A figure from the description.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019mo31AVugYdfXietM73iWZ
@Qazi-pk Qazi-pk changed the title pir: update to pir-engine 3.6.0 with built-in fallback gates pir: update to pir-engine 3.7.0 Oct 25, 2026
@Qazi-pk

Qazi-pk commented Oct 25, 2026

Copy link
Copy Markdown
Contributor Author

I've updated this PR from 3.6.0 to pir-engine 3.7.0 (released today, DOI 10.5281/zenodo.23631107), so that the next benchmark run uses the current version. One new commit:

  • install.sh: tag v3.7.0.
  • regressor.py: new explicit constructor argument safeguard=True (recorded by get_params()), passed to the engine with gates; docstring updated.
  • metadata.yml: one sentence on the safeguard; software DOI updated.

What 3.7.0 adds. After the engine and the gates, an optional safeguard compares the final formula with a ridge-regression linear model and the mean. A formula with training R² ≥ 0.999 is always kept. Otherwise the linear model replaces it only if its 5-fold cross-validated R² is higher by more than 0.05, and the mean is returned if both are below 0. The rules were fixed before any benchmark run. Also: a memory cap in the exponential gate (it ran out of memory on wide datasets), and safe symbol names for column names that aren't valid identifiers (e.g. inv-nodes in 1201_BNG_breastTumor crashed the harness's simplicity() parse).

Checks. ruff check, ruff format --check (0.17.0) and scripts/typecheck.py pass locally. My own runs with SRBench's evaluate_model.py and assess_symbolic_model.py (current master, fixed scorer), not official results:

Feynman-119, seeds 23654 / 78901 v3.5 3.6.0 3.7.0
noise 0 35 / 35 50 / 50 50 / 50
noise 0.001 35 / 35 36 / 36 36 / 36
noise 0.01 30 / 29 35 / 35 35 / 35
noise 0.1 2 / 2 23 / 22 23 / 22

No dataset solved by an earlier version is lost at any noise level. On the 120 black-box datasets I could download (one seed, no tuning), median test R² goes from −0.02 (3.6.0) to 0.44 (3.7.0), with no out-of-memory failures and no dataset worse.

No rush. Happy to adjust anything.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant