Repository navigation
Synergy benchmarks: Hourican & Quax mechanisms and planted modules - #18
Merged
Merged
Conversation
- datasets: xor_descendant, asymmetric_interaction, or_gate and xor_among, plus contemporaneous_motif(kind, law=) with CONTEMPORANEOUS_MOTIFS. The lag-0 motifs are sampled from their exact laws, and law=True returns the dit distribution for reference families. - benchmarks.synergy_validation scores parent recovery on the lagged motifs (XOR at three noise levels, XOR with a descendant, an asymmetric interaction, OR, and XOR among 12 processes). On the lag-0 motifs it scores whether contemporaneous_families returns exactly the population family. Rates come with Clopper-Pearson intervals. - benchmarks.planted_module_validation plants synthetic modules (XOR, XOR with a descendant, OR, independent bits) beside a fixed background. It reports recovered parents, exact parents and false parents. - validation/synergy.py and a validation docs section. Every check scores 9 or 10 of 10. Planted modules recovered every true parent in 10 of 10 replicates and were exact in 7 to 9.
Autoplectic
marked this pull request as ready for review
November 14, 2026 21:54
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This adds benchmarks for the mechanisms of Hourican & Quax, "Operationalising Weaker Faithfulness" (arXiv:2610.31724). It checks #16 (the pair-search budget) and #17 (lag-0 explanation families) against them.
datasets:xor_descendant,asymmetric_interaction,or_gateandxor_among, pluscontemporaneous_motif(kind, law=)withCONTEMPORANEOUS_MOTIFS. The lag-0 motifs are sampled from their exact laws.law=Truereturns the dit distribution, so reference families come fromexplanation_family(decisions_from_distribution(...)).benchmarks.synergy_validation:contemporaneous_familiesreturns exactly the population family.benchmarks.planted_module_validation: synthetic modules (XOR, XOR with a descendant, OR, independent bits) are planted beside a fixed background, following the paper's neural application. It reports recovered parents, exact parents and false parents.validation/synergy.py, a new section indocs/validation.rst, and a literature-table entry.Results
From
validation/synergy.py --planted, with 4000 samples, default settings and 10 replicates:For modules planted beside a noisy rule-90 automaton of six cells:
The XOR at flip 0.45 (0.007 bits of whole-minus-sum information) is found in every run. That signal is below the paper's 0.3-bit protection threshold.
Tests
tests/test_synergy.py: generators, motif laws, and small runs of both validations.-Ware clean.