Skip to content

Add end-to-end tests for the encoding pipeline - #12

Open
aadhavthaniga wants to merge 7 commits into
GT-LIT-Lab:mainfrom
aadhavthaniga:tests/end-to-end
Open

aadhavthaniga wants to merge 7 commits into
GT-LIT-Lab:mainfrom
aadhavthaniga:tests/end-to-end

Conversation

@aadhavthaniga

@aadhavthaniga aadhavthaniga commented Oct 12, 2026 •

Copy link
Copy Markdown

End-to-end tests for the encoding pipeline

Synthetic (tests/test_pipeline_synthetic.py, offline, ~1 min): plants a signal at FIR lags 2-3 in 10 vertices and checks recovery, the lag-specific kernel, FDR selection, FIR causality, no held-out leakage, determinism, and downsampler/FIR outputs against independent reference implementations.

Golden (tests/test_golden_lebel.py, opt-in): pins train_simple.py's wordrate results on the LeBel UTS03 assembly. Skips unless LITCODER_LEBEL_ASSEMBLY points at the file (download link in the README). Tolerances come from measured CPU/MPS/cross-library variance; see tests/fixtures/golden_lebel_uts03.json.

Code changes

  • encoding/models/folding.py: fold shuffling used the unseeded global RNG, so every run fit a different model. Now seeded; the determinism and golden tests depend on this.
  • train_simple.py: reads the assembly path from LITCODER_LEBEL_ASSEMBLY, falling back to the existing path.

Found along the way

  • Non-legacy downsamplers (average, sum, last) misread split_indices and return ~n_words rows. Marked strict xfail.
  • NaN features are silently zeroed while NaN targets pass through. Documented in a test.

Scope: the golden test uses wordrate (no model download, no downsampling); downsampling is covered by the synthetic tests.

Results: with the assembly, 29 passed, 3 xfailed. Without it, the 4 golden tests that need it skip.

@aadhavthaniga
aadhavthaniga marked this pull request as ready for review October 13, 2026 22:38

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant