Repository navigation
Anytime pipeline on large joints: stage deadlines, sparse stationary solve, ladder fixes - #54
Merged
Merged
Conversation
…ft deadline Three stages ran far past their deadlines on 4-state survey joints with 95-144 causal-state pairs: - generator caps: one degenerate node LP ran 8+ minutes; child LPs now get the time left as a HiGHS time_limit (a cut-off LP keeps its parent's bound); - dit exact/Wyner fallbacks take no deadline; skipped above 30 pairs (every 3-state survey joint has at most 21); - memory_lift_search ran its whole first start regardless of the deadline; a start in progress now stops at it. Co-authored-by: Cursor <cursoragent@cursor.com>
_optimize and _optimize_coupled take a deadline; every L-BFGS-B objective raises once it passes and the run returns its last finished step. The pipeline's line-split stage passes its deadline and checks it between floors (one polish ran past the budget for over 10 minutes on a 144-pair 4-state joint). Rename the private exception to _DeadlineReachedError (N818). Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
… build Above 2000 states, stationary_distribution_hmm builds a sparse transition matrix from the edges and solves pi (P - I) = 0 with one equation replaced by the normalization. The dense path's matrix_rank and eig are O(n^3) and never finished on a 50k-state reverse machine in the ladder. Symbolic weights, several closed classes or a failed residual check fall back to the dense path. The ladder and try_build_bidirectional reuse the machines built by the preflight (_preflight) instead of inferring and building them twice. Co-authored-by: Cursor <cursoragent@cursor.com>
Member
Author
|
Added: a sparse stationary solve for chains with more than 2,000 states (the dense |
…xplosions A ladder build whose reverse machine exceeds max_reverse_states (off by default) is reported as an explosion so surveys use window bounds. The last 4-state survey timeout built an 8,949-state truncated reverse machine at the 50k cap, and the pipeline then spent its whole budget setting up the problem. Co-authored-by: Cursor <cursoragent@cursor.com>
…(Ubuntu CI took 31 s) Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com> # Conflicts: # sofic/generators/generative_search.py
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
These changes clear the last three 4-state survey timeouts. Each one was a single call that ignored the pipeline's deadline, or a computation that grew too fast with the size of the joint.
Stage deadlines
time_limit. A cut-off LP keeps its parent's bound, so the cap stays valid; the root LP always runs to the end. One degenerate node LP had run more than 8 minutes past a 90 s deadline.dit fallbacksstage). They had used 190–760 s of a 900 s budget on 30–70-pair joints. Every 3-state survey joint has at most 21 pairs, so the 3-state benchmark is unaffected.memory_lift_searchstops a start in progress at its deadline. Its first start used to run in full regardless (more than 14 minutes)._optimizeand_optimize_coupledtake adeadline, and the pipeline's line-split stage passes it. One polish had run more than 10 minutes over budget. The targeted search andcoupled_searchare unchanged: by design they always finish one polish (or the ε-machine seed).Builds
stationary_distribution_hmmsolves chains with more than 2,000 states sparsely. The densematrix_rankandeigaretry_build_bidirectionalreuse the machines the preflight built instead of building them twice, which roughly halves build time.max_reverse_states(off by default): a ladder build whose truncated reverse machine is larger than this is classified as an explosion. The survey driver sets it to 1,000, so such cases fall back to window bounds._DeadlineReachedError(lint rule N818).Results
80-topology 4-state sample (cpfci
survey4_v5.jsonl; ladder budget 300 s, pipeline budget 900 s, hard timeout 1500 s):[[0,1],[2,None],[2,3],[1,0]](95 pairs)[[None,1],[2,1],[1,3],[3,0]](144 pairs)[[0,1],[0,2],[3,None],[1,None]]Totals: 32 brackets (10 certified), 35 window brackets, 13 skipped, 0 timeouts (was 3). The size cap reclassifies no other case.
Test plan
memory_lift_searchstops inside a start (fake clock);ruff check,ruff format --check,ty check(no errors)