Repository navigation
Conversation
`CallableEquation._lambda` was a property that re-ran `sympy.lambdify` (code generation plus `exec`) on every call, so every `predict()` and every direct `lambda_format(X)` call paid ~1-3 ms of compilation before evaluating the expression. Cache it with `functools.cached_property` and drop the cached function in `__getstate__`, since generated code cannot be pickled; it is rebuilt lazily after loading. Also remove smaller redundancies on the same paths: - `latex()` refreshed the hall of fame and then called `sympy()`, which refreshes it again. - `idx_model_selection` filtered with `DataFrame.query` on a formatted float on every `predict()`; a boolean mask is ~3x cheaper and does not round-trip the threshold through a string. - `add_export_formats` and `_search_output_to_callable_expressions` deep-copied frames they only read, and the former rebuilt the SymPy symbol list for every row. - `_run` copied `X` once just to inspect its dtype and again before `jl_numpy_array`, which already copies into Julia memory; use `np.asarray` so a copy is only made when the dtype changes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Member
|
Thanks for this! I don't see a user-facing problem this PR fixes, and it adds code to maintain. Closing for now; happy to revisit if you hit a real slowdown. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
CallableEquation._lambdawas a@propertythat calledsympy.lambdifyon every access, so everypredict()and every directequations_["lambda_format"](X)call re-generated andexec'd Python source before evaluating the expression. This caches it withfunctools.cached_property;__getstate__drops the cached function, which cannot be pickled, and it is rebuilt lazily after loading.Smaller redundancies on the same paths:
latex()calledrefresh()and thensympy(), which refreshes again, so it rebuilt every export twice.idx_model_selectionfiltered withDataFrame.query(f"loss <= {threshold}")on everypredict(). A boolean mask is about 3x cheaper and avoids round-tripping the float through a string.add_export_formatsand_search_output_to_callable_expressionsdeep-copied frames they only read, and the former rebuilt the SymPy symbol list for every row._runmade a full copy ofXjust to inspect its dtype, and another beforejl_numpy_array, which already copies into Julia memory. It now usesnp.asarray, so a copy is only made when the dtype changes.Measurements
These use a model loaded with
from_filefrom a 6-equation hall of fame with 5 features, on Windows with Julia 1.11 and SymbolicRegression 2.7.0. The machine was shared, so absolute times are noisy.equations_.iloc[-1]["lambda_format"](X)(10 rows)predict(X)(10 rows)latex()Calling
lambdifyalone on a 5-variable expression takes about 1.75 ms, against about 0.008 ms to evaluate the result.Tests
test_lambda_format_compiles_once_and_stays_picklable:lambdifyruns once across repeated calls andpredict(), and the callable still round-trips throughpickle. On masterlambdifyruns 6 times.test_latex_refreshes_once: on masterrefreshruns twice.Both new tests fail on master and pass here.
TestBest,TestLaTeXTableand the export pickling tests pass locally.This came out of a broader review of the codebase. The related PRs are independent of each other and merge cleanly together.
🤖 Generated with Claude Code