Skip to content

feat: Meridian roi_calibration_period mask builder + to_code() array support - #768

Merged
igerber merged 1 commit into
mainfrom
feature/meridian-calibration-mask
Aug 15, 2026
Merged

igerber merged 1 commit into
mainfrom
feature/meridian-calibration-mask

Conversation

@igerber

@igerber igerber commented Aug 15, 2026

Copy link
Copy Markdown
Owner

Summary

  • New public meridian_calibration_mask(media_times=, media_channels=, channel=, window=) builds the boolean (n_media_times, n_media_channels) roi_calibration_period mask from the MMM's own coordinates: the experiment channel's column(s) carry exactly the window, and every other channel's column is all-True — Meridian's documented convention (channels not named in an experiment use ALL periods; an all-False column would zero that channel's aggregated calibration spend in input_data._aggregate_spend). Window is a 2-tuple of inclusive (start, end) bounds (value-ordered, pd.to_datetime-coerced against datetime coordinates, timezone mismatches fail closed both ways) or a sequence of explicit labels (fail-closed membership, named missing-label errors). Container and element gates reject strings/Mappings/scalars/MultiIndex (including tuple-label promotion)/non-1-D arrays/missing channel names.
  • MeridianROIPrior.to_code() accepts the array for roi_calibration_period, serializing it into the snippet as an np.ones prelude plus per-column-group window assignments (round-trip-exact for any accepted mask). Fail-closed acceptance: bool or 0/1-numeric only (cast, matching Google's own float np.zeros example), masked arrays rejected, all-False masks and ANY entirely-False column rejected.
  • Fix riding along: Meridian 1.7.0's ModelSpec._validate_roi_calibration_period accepts a non-None mask only for 'roi' prior types, so to_code() now fails closed for parameter="mroi_m" with any non-None roi_calibration_period — the pre-existing expression-string route emitted mroi_m snippets Meridian rejects at ModelSpec construction. The no-scope error is parameter-aware. Docs also quote Meridian's caution that the mask "is not generally recommended", pointing at full_model_window=True when the evidence transfers. Container-mode scale= additionally rejects booleans (float(True) would silently scale by one).
  • Docs/tests: REGISTRY Meridian section (three time-scope routes, mask Notes, configure-model primary source), executable api/mmm.rst mask example + autofunction section + references, api/index.rst autosummary + committed stub, both LLM guides, references.rst citation, CHANGELOG Added + Fixed, TODO mask-builder row removed (delivered); TestMeridianCalibrationMask + TestToCodeArrayMask with message-anchored coverage of the window-resolution and rejection matrices, serialization round-trips, and Google float-parity.

Methodology references (required if estimator / math changes)

  • Method name(s): No estimator/math changes — interop mask construction for Google Meridian ROI-prior calibration.
  • Paper / source link(s): Google Meridian configure-model guide, "Set the ROI calibration period" (mask shape/semantics + non-experiment-channel convention + the "not generally recommended" caution); Meridian v1.7.0 source (model/spec.py _validate_roi_calibration_period, data/input_data.py _aggregate_spend). All cited in the REGISTRY "MMM Calibration Export (interop)" section and docs/references.rst.
  • Any intentional deviations from the source (and why): None — the mask semantics, the non-experiment all-periods convention, the roi-only restriction, and the 0/1-numeric castability all mirror the pinned Meridian 1.7.0 contract; documented as REGISTRY **Note:** bullets.

Validation

  • Tests added/updated: tests/test_mmm.py (TestMeridianCalibrationMask, TestToCodeArrayMask, extended test_scope_mutually_exclusive/test_public_exports, boolean-scale rejection), tests/test_doc_snippets.py roster already executes the new api/mmm.rst example. 127 MMM tests, doc snippets, naming guard, docs IA, and agent-discoverability suites pass; make -C docs html SPHINXOPTS="-W" clean; mypy zero errors at the CI pins.
  • Backtest / simulation / notebook evidence (if applicable): N/A — deterministic mask construction; serialization round-trips are exec-verified in tests.

Security / privacy

  • Confirm no secrets/PII in this PR: Yes

@github-actions

Copy link
Copy Markdown

Overall assessment

Looks good — no unmitigated P0/P1 findings.

Executive summary

  • No estimator, weighting, variance, SE, or inference behavior changes.
  • Meridian mask semantics match the Methodology Registry and cited 1.7.0 contract.
  • Empty selections, invalid values, timezone mismatches, and all-false columns fail closed.
  • Array serialization has round-trip and parameter-interaction coverage.
  • One unrelated RDD tutorial/test removal is a P2 documentation regression.

Methodology

M1 — Documented Meridian behavior

  • Severity: P3 — informational
  • Impact: meridian_calibration_mask() and MeridianROIPrior.to_code() follow the documented (time, channel) shape, all-period convention for non-experiment channels, and roi-only restriction. Row-count/order limitations are explicitly recorded. No undocumented deviation was found.
  • Concrete fix: None required.
  • Locations: diff_diff/mmm.py:L709-L945, diff_diff/mmm.py:L1322-L1483, docs/methodology/REGISTRY.md:L6343-L6346

Code Quality

No findings.

Performance

No material findings.

Maintainability

No findings.

Tech Debt

No silent correctness debt introduced. Removing the completed mask-builder TODO is consistent with delivery.

Security

No secrets, unsafe external operations, or new execution vulnerability found. The existing expression-string route remains explicitly user-generated code.

Documentation/Tests

D1 — Unrelated RDD tutorial and regression suite removed

  • Severity: P2
  • Impact: The PR deletes the full Tutorial 28 RDD walkthrough and its 328-line numerical drift suite, reducing documentation and regression coverage unrelated to Meridian.
  • Concrete fix: Restore the notebook, drift test, tutorial index, dependency-map, and changelog entries; handle intentional removal separately with its rationale.
  • Locations: docs/tutorials/28_rdd_scholarship_illusion.ipynb:L1-L947, tests/test_t28_rdd_scholarship_illusion_drift.py:L1-L328, docs/tutorials/index.rst:L212-L231

The modified Python files parse successfully; runtime tests could not be executed because NumPy/Pandas are unavailable in the review environment.

…support

New public meridian_calibration_mask(media_times=, media_channels=, channel=,
window=) builds the boolean (n_media_times, n_media_channels) mask Meridian's
ModelSpec expects, from the MMM's own coordinates taken verbatim: the
experiment channel's column(s) are True exactly on the window and every other
channel's column is all-True (Meridian's documented convention - channels not
named in an experiment use ALL periods; an all-False column would zero that
channel's aggregated calibration spend in input_data._aggregate_spend).
Window: a 2-tuple is (start, end) inclusive bounds (value-ordered,
pd.to_datetime-coerced against datetime coordinates, tz mismatches fail
closed both ways); any other sequence is explicit labels with fail-closed
membership and named missing-label errors. Container/element gates reject
strings, Mappings, scalars, MultiIndex (including tuple-label promotion),
non-1-D arrays, and missing channel names.

MeridianROIPrior.to_code() now accepts the array for roi_calibration_period,
serialized into the snippet as an np.ones prelude plus per-column-group
window assignments (.tolist()-coerced indices; round-trip-exact for any
accepted mask). Acceptance is fail-closed: bool or 0/1-numeric only (cast to
bool, matching Google's float np.zeros example), masked arrays rejected,
all-False masks and ANY entirely-False column rejected. roi_m priors only,
per Meridian 1.7.0's ModelSpec._validate_roi_calibration_period - and
to_code() now fails closed for mroi_m with ANY non-None
roi_calibration_period (array or expression), fixing the pre-existing
expression route that emitted mroi snippets Meridian rejects at ModelSpec
construction; the no-scope error is parameter-aware. Docs quote Meridian's
own caution that the mask "is not generally recommended", pointing at
full_model_window=True when evidence transfers. Container-mode scale=
additionally rejects booleans (float(True) would silently scale by one).

Docs: REGISTRY Meridian section (three routes + mask Notes + configure-model
primary source), api/mmm.rst (executable mask example + autofunction section
+ references), api/index.rst autosummary + committed stub, both LLM guides,
references.rst citation, CHANGELOG Added + Fixed entries, TODO mask-builder
row removed (delivered). Tests: TestMeridianCalibrationMask (window
resolution, coercion successes and fail-closed matrix) and TestToCodeArrayMask
(serialization round-trips, per-column rejection, mroi rejection, Google
float-parity), all message-anchored.
@igerber
igerber force-pushed the feature/meridian-calibration-mask branch from a14b1d4 to eec7626 Compare August 15, 2026 22:13
@github-actions

Copy link
Copy Markdown

🔁 AI review rerun (requested by @igerber)

Head SHA: eec76269708abab4a80fde93cc8d4afb2b7cde7c


Overall assessment

Looks good — no unmitigated P0/P1 findings.

Executive summary

  • Prior RDD documentation/test removal is resolved.
  • No estimator, weighting, variance, SE, or inference changes.
  • Mask semantics and mroi_m restrictions match the Registry’s Meridian contract.
  • Empty windows, invalid masks, timezone mismatches, and all-false columns fail closed.
  • Array serialization and parameter interactions have strong test coverage.

Methodology

M1 — Documented Meridian caveats

  • Severity: P3 — informational
  • Impact: Mask row-count and column-order validation remains caller-owned, including the max_lag coordinate ambiguity. These limitations are explicitly documented and therefore mitigated.
  • Concrete fix: None required.
  • Locations: diff_diff/mmm.py:L737-L759, docs/methodology/REGISTRY.md:L6344-L6346

Code Quality

No findings.

Performance

No material findings.

Maintainability

No findings.

Tech Debt

No untracked correctness debt. Removing the completed mask-builder TODO is appropriate.

Security

No secrets or new unsafe execution paths found. Array serialization emits only validated integer indices and boolean assignments.

Documentation/Tests

No findings. The prior RDD regression is restored, and the new tests cover empty selections, invalid values, serialization round-trips, channel-count checks, and roi_m/mroi_m interactions.

Static compilation and git diff --check passed. Runtime tests were unavailable because NumPy, Pandas, and Pytest are not installed in the review environment.

@igerber igerber added the ready-for-ci Triggers CI test workflows label Aug 15, 2026
@igerber
igerber merged commit 10d0b83 into main Aug 15, 2026
39 of 40 checks passed
@igerber
igerber deleted the feature/meridian-calibration-mask branch August 15, 2026 23:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready-for-ci Triggers CI test workflows

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant