diff --git a/CHANGELOG.md b/CHANGELOG.md index 19d3e2a13..73706b401 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -8,6 +8,30 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] ### Added +- **Meridian `roi_calibration_period` mask builder + `to_code()` array support.** + New `meridian_calibration_mask(media_times=, media_channels=, channel=, window=)` + builds the boolean `(n_media_times, n_media_channels)` mask Meridian's + `ModelSpec` expects, from the MMM's own coordinates taken verbatim: the + experiment channel's column(s) are True exactly on the window, and every + other channel's column is all-True - Meridian's documented convention + (channels not named in an experiment use ALL periods for ROI calibration; + an all-False column would zero that channel's aggregated calibration spend + in `input_data._aggregate_spend`). + Window convention: a 2-tuple is `(start, end)` inclusive bounds (value-ordered, + `pd.to_datetime`-coerced against datetime coordinates, timezone mismatches fail + closed both ways); any other sequence is explicit labels with fail-closed + membership. `MeridianROIPrior.to_code()` now also accepts that array for + `roi_calibration_period` - a new capability, serialized into the generated + snippet as an `np.ones` prelude plus per-column-group window assignments - + alongside the pre-existing expression-string and `full_model_window=True` + routes. Array acceptance is fail-closed: bool or 0/1-numeric only (cast to + bool, matching Google's own float `np.zeros` example), masked arrays + rejected, all-False masks rejected, and ANY entirely-False column rejected + (it would zero that channel's calibration spend). Masks apply to `roi_m` + priors only. The mask's row count and column order are + not machine-checkable inside `to_code` (no time coordinate is passed and the + array carries no channel labels) - documented caveats; the builder guarantees + both when given the model's own coordinates. - **MMM exporters: container mode on the 3.9 aggregation surface.** Both `to_pymc_marketing_lift_test` and `to_meridian_roi_prior` now accept `aggregation_result=` - the pinned `AggregationResult` returned by post-fit @@ -49,6 +73,15 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 complier LATE), and CCFT 2019 covariate adjustment (precision, not identification). +### Fixed +- **`MeridianROIPrior.to_code()` emitted invalid time-scoped `mroi_m` snippets.** + Meridian 1.7.0's `ModelSpec._validate_roi_calibration_period` rejects a + non-None `roi_calibration_period` unless the media prior type is `'roi'`, so + the expression-string route (shipped with the exporters) produced `mroi_m` + snippets that fail at `ModelSpec` construction. `to_code()` now fails closed + for `parameter="mroi_m"` with any non-None `roi_calibration_period` (array or + expression), pointing at `full_model_window=True` as the mroi route. + ### Changed - **Narrative docs migrated off the deprecated fit-time `aggregate=`** (the 3.9 M-020 family; TODO "fit-time aggregate= teachings" sweep, RST half): diff --git a/TODO.md b/TODO.md index 744476378..11f400dd1 100644 --- a/TODO.md +++ b/TODO.md @@ -80,7 +80,6 @@ generic sparse-FE, QR+SVD rank-detection redundancy, `check_finite` bypass — m | Type-blind `n_bootstrap` acceptance in already-validated estimators - HAD bool (`isinstance(..., int)` passes `True`, runs as 1 replicate), dCDH bool+float (its bare `< 0` check passes both `True` and `2.5`), TROP float (`2.5` passes the `>= 2` floor), SyntheticDiD float under all three variance methods + bool/negative under jackknife (its floor check is skipped there) - align these local checks with the `utils.validate_n_bootstrap` type guard (M-081 kept them out of the sweep: it scoped to previously-UNvalidated estimators only) | `diff_diff/had.py`, `diff_diff/chaisemartin_dhaultfoeuille.py`, `diff_diff/trop.py`, `diff_diff/synthetic_did.py` | 2(d) PR-B | Quick | Low | | Evaluate adding the `BaseEstimator` param surface (get_params/set_params) to the exported classes that never had it - `PowerAnalysis`, `LinearRegression`, `BusinessReport`, `DiagnosticReport`, `TWFEWeightsResult` (a NEW public surface, deliberately out of the 2(c)-i pure-refactor scope; `LinearRegression` is the one `fit`-bearing class excluded from the contract suite's roster-completeness test). | `diff_diff/linalg.py`, `diff_diff/power.py` | mixin PR | Mid | Low | | Tighten the mypy suppressions that back the enforced-zero posture: burn down `prep_dgp`'s per-module `[index]` override (needs a None-vs-array restructure that preserves the seeded RNG stream), and evaluate re-enabling the globally disabled codes (`arg-type`, `return-value`, `var-annotated`, `assignment`) one at a time — `assignment` alone hid several real annotation drifts found during the 2026-07 triage. | `pyproject.toml` `[tool.mypy]`, `diff_diff/prep_dgp.py` | lint-CI | Mid | Low | -| MMM interop follow-up: Meridian `roi_calibration_period` mask builder - accept the MMM's time index + channel order and emit the boolean `(n_media_times, n_media_channels)` mask so `.to_code()` scopes the prior to the experiment window automatically (today the caller passes a mask expression / `full_model_window=True`). | `diff_diff/mmm.py` | mmm-interop | Quick | Low | | MMM interop PR-B: calibration tutorial notebook (fit DiD/CS -> scope -> `to_pymc_marketing_lift_test` / `to_meridian_roi_prior`) + a `llms-practitioner.txt` Step 8 pointer to the exporters as the MMM hand-off. | `docs/tutorials/`, `diff_diff/guides/llms-practitioner.txt` | mmm-interop | Mid | Low | | Tracking-file contract guard test: reject NEW active deferred-work pointers at `TODO.md` (deferred rows live in `DEFERRED.md`; allowlist for historical/past-tense prose and actionable-row pointers) and assert rows cross-linking a `docs/v4-deprecations.yaml` `M-xxx` id don't restate ledger status. Origin: tracking-split local review R2. | `tests/`, `TODO.md`, `DEFERRED.md` | tracking-split | Quick | Low | | Real-data CI canary for dataset-backed replication tests: `test_methodology_lwdid.py`'s Prop 99 / Walmart goldens skip (visibly) when loaders fall back to synthetic; add a lane or canary asserting `df.attrs["source"] == "lwdid_ssc_ancillary"` in CI so network regressions cannot silently de-gate the replication tests. Follow-on from the loader-fallback repair (#723), which made provenance explicit but deliberately did not add a network-dependent CI lane. | `tests/test_methodology_lwdid.py`, `.github/workflows/` | LWDiD validation suite | Quick | Low | diff --git a/diff_diff/__init__.py b/diff_diff/__init__.py index f73343f83..c0231157e 100644 --- a/diff_diff/__init__.py +++ b/diff_diff/__init__.py @@ -162,6 +162,7 @@ from diff_diff.lpdid_results import LPDiDResults from diff_diff.mmm import ( MeridianROIPrior, + meridian_calibration_mask, to_meridian_roi_prior, to_pymc_marketing_lift_test, ) @@ -613,6 +614,7 @@ def __getattr__(name: str) -> _Any: # MMM calibration export (interop) "to_pymc_marketing_lift_test", "to_meridian_roi_prior", + "meridian_calibration_mask", "MeridianROIPrior", # LLM guide accessor "get_llm_guide", diff --git a/diff_diff/guides/llms-full.txt b/diff_diff/guides/llms-full.txt index f19cc7f74..db56d7f53 100644 --- a/diff_diff/guides/llms-full.txt +++ b/diff_diff/guides/llms-full.txt @@ -2896,6 +2896,7 @@ parameterization, pools, and emits snippets. from diff_diff import ( ImputationDiD, SyntheticDiD, + meridian_calibration_mask, to_pymc_marketing_lift_test, to_meridian_roi_prior, ) @@ -2935,14 +2936,28 @@ prior.roi_mean, prior.roi_sd # pooled ROI moments prior.mu, prior.sigma # LogNormal params (match Google's # lognormal_dist_from_mean_std) prior.to_dict() # JSON-ready + +# Build the (n_media_times, n_media_channels) boolean mask from the MMM's own +# coordinates; window = inclusive (start, end) tuple or explicit-labels list. +# Experiment channel True on the window; every OTHER channel ALL-TRUE +# (Meridian's convention - an all-False column zeroes that channel's +# calibration spend). roi_m priors only: Meridian rejects the mask for mroi_m +# (use full_model_window=True there). to_code() serializes the array. +mask = meridian_calibration_mask( + media_times=media_times, # the model's time coordinate labels + media_channels=["search", "tv"], # InputData channel order (same list + channel="tv", # as to_code's media_channels) + window=("2024-01-15", "2024-03-04"), +) print(prior.to_code( # ready-to-paste PriorDistribution + channel="tv", # ModelSpec; roi_m/mroi_m is per-channel media_channels=["search", "tv"], # so channel scope is required (vector # prior in model channel order; other # channels keep the Meridian default), - roi_calibration_period="mask", # AND time scope (mask expr) or - # full_model_window=True; sets -)) # media_prior_type accordingly. + roi_calibration_period=mask, # AND time scope: the boolean ndarray + # above, a mask expr string, or +)) # full_model_window=True; sets + # media_prior_type accordingly. # Container route (ImputationDiD/TwoStageDiD): derive the totals from the # post-fit aggregation instead of hand-scoping. diff --git a/diff_diff/guides/llms.txt b/diff_diff/guides/llms.txt index 75b0d5227..cf89bfae7 100644 --- a/diff_diff/guides/llms.txt +++ b/diff_diff/guides/llms.txt @@ -90,7 +90,7 @@ The site is organized into 5 sections, each with a landing page: - [Honest DiD](https://diff-diff.readthedocs.io/en/stable/api/honest_did.html): Rambachan & Roth (2023) sensitivity analysis — robust CI under parallel trends violations, breakdown values - [Pre-Trends Power Analysis](https://diff-diff.readthedocs.io/en/stable/api/pretrends.html): Roth (2022) Section II.A-B no-individually-significant (NIS) box-probability pretest power + minimum detectable violation; `pretest_form='nis'` (default) implements the paper's primary form, `pretest_form='wald'` retained as paper-supported alternative (Propositions 1+3+4 all apply); linear-violation MDV in Roth's γ units when relative-time labels are threaded through `fit()`; full Σ_22 routing on non-bootstrap CallawaySantAnna and SunAbraham adapters and on admitted CS-/StackedDiD-sourced `aggregate('event_study')` containers (StackedDiD persists its ES VCV in every inference mode) - [Power Analysis](https://diff-diff.readthedocs.io/en/stable/api/power.html): Analytical and simulation-based power analysis — MDE, sample size, power curves for study design -- [MMM Calibration Export](https://diff-diff.readthedocs.io/en/stable/api/mmm.html): Assemble Marketing Mix Model calibration inputs from experiment results. Two input routes: explicit already-scoped numbers, or the pinned `AggregationResult` container from post-fit `results.aggregate('simple'|'group')` via `aggregation_result=` + `scale=` (effect = att x scale per row, in `to_dataframe()` order; `scale="auto"` reads the container's treated-obs count and is honored ONLY for ImputationDiD/TwoStageDiD fits - it acknowledges an additive-level outcome, an unweighted fit, and fully identified effects; other estimators and raw results objects fail closed). `to_pymc_marketing_lift_test(channel, x, delta_x, delta_y=, sigma=, aggregation_result=, scale=, dims=, on_wrong_sign=)` builds the PyMC-Marketing/prophetverse lift-test DataFrame with sign/zero/positivity guards. `to_meridian_roi_prior(incremental_outcome=, incremental_outcome_se=, aggregation_result=, scale=, spend, parameter="roi_m"|"mroi_m", se_widening=)` builds Google Meridian lognormal ROI priors (spend-weighted pooling, lognormal parity with `lognormal_dist_from_mean_std`, channel- and time-scoped `.to_code()` snippet setting `media_prior_type`; widen `se_widening` when pooling a group-level container's same-fit cohorts). Pure numpy/pandas; imports no MMM package; never calls `aggregate()` itself. +- [MMM Calibration Export](https://diff-diff.readthedocs.io/en/stable/api/mmm.html): Assemble Marketing Mix Model calibration inputs from experiment results. Two input routes: explicit already-scoped numbers, or the pinned `AggregationResult` container from post-fit `results.aggregate('simple'|'group')` via `aggregation_result=` + `scale=` (effect = att x scale per row, in `to_dataframe()` order; `scale="auto"` reads the container's treated-obs count and is honored ONLY for ImputationDiD/TwoStageDiD fits - it acknowledges an additive-level outcome, an unweighted fit, and fully identified effects; other estimators and raw results objects fail closed). `to_pymc_marketing_lift_test(channel, x, delta_x, delta_y=, sigma=, aggregation_result=, scale=, dims=, on_wrong_sign=)` builds the PyMC-Marketing/prophetverse lift-test DataFrame with sign/zero/positivity guards. `to_meridian_roi_prior(incremental_outcome=, incremental_outcome_se=, aggregation_result=, scale=, spend, parameter="roi_m"|"mroi_m", se_widening=)` builds Google Meridian lognormal ROI priors (spend-weighted pooling, lognormal parity with `lognormal_dist_from_mean_std`, channel- and time-scoped `.to_code()` snippet setting `media_prior_type` - time scope via `meridian_calibration_mask(media_times=, media_channels=, channel=, window=)`, which builds the boolean (n_media_times, n_media_channels) mask (window = inclusive (start, end) tuple or explicit-labels list; experiment channel True on the window, every other channel ALL-TRUE per Meridian's documented convention; roi_m priors only - Meridian rejects the mask for mroi_m) and whose array `.to_code()` serializes into the snippet, or via a mask expression string, or full_model_window=True; widen `se_widening` when pooling a group-level container's same-fit cohorts). Pure numpy/pandas; imports no MMM package; never calls `aggregate()` itself. - Conley spatial HAC SE (`vcov_type="conley"`) on cross-sectional `LinearRegression` / `compute_robust_vcov` PLUS panel `DifferenceInDifferences` / `MultiPeriodDiD` / `TwoWayFixedEffects` (with `conley_lag_cutoff=` for within-unit Bartlett temporal HAC) — Conley (1999) spatial-correlation-aware SEs with haversine/euclidean/callable distance metric and Bartlett/uniform spatial kernel; panel path uses the R `conleyreg`-form block-decomposed sandwich (within-period spatial + within-unit Bartlett serial, same-time excluded); parity vs R `conleyreg` (Düsterhöft 2021) on cross-sectional AND panel `lag_cutoff > 0` fixtures. Combining with explicit `cluster=` applies the combined spatial + cluster product kernel `K_total[i,j] = K_space · 1{c_i = c_j}` (cluster must be constant within each unit across periods on the panel path; validator-enforced). DiD takes `unit=` as a fit-time kwarg when `vcov_type="conley"` (not on `__init__`). Sparse k-d-tree fast path auto-activates for `n > 5_000` with bartlett kernel + haversine/euclidean metric ## Tutorials diff --git a/diff_diff/mmm.py b/diff_diff/mmm.py index 7599bd8e1..079d24219 100644 --- a/diff_diff/mmm.py +++ b/diff_diff/mmm.py @@ -67,7 +67,12 @@ from diff_diff.aggregation import AggregationResult from diff_diff.results_base import BaseResults -__all__ = ["MeridianROIPrior", "to_meridian_roi_prior", "to_pymc_marketing_lift_test"] +__all__ = [ + "MeridianROIPrior", + "meridian_calibration_mask", + "to_meridian_roi_prior", + "to_pymc_marketing_lift_test", +] _WRONG_SIGN_POLICIES = ("raise", "drop", "keep") _LIFT_TEST_RESERVED = frozenset({"channel", "x", "delta_x", "delta_y", "sigma"}) @@ -126,7 +131,7 @@ import tensorflow_probability as tfp from meridian.model import prior_distribution, spec -roi_prior = tfp.distributions.LogNormal({mu!r}, {sigma!r}, name="{param}") +{mask_prelude}roi_prior = tfp.distributions.LogNormal({mu!r}, {sigma!r}, name="{param}") prior = prior_distribution.PriorDistribution({param}=roi_prior) model_spec = spec.ModelSpec( prior=prior, @@ -147,7 +152,7 @@ import tensorflow_probability as tfp from meridian.model import prior_distribution, spec -mu = {mu_vector!r} +{mask_prelude}mu = {mu_vector!r} sigma = {sigma_vector!r} roi_prior = tfp.distributions.LogNormal(mu, sigma, name="{param}") prior = prior_distribution.PriorDistribution({param}=roi_prior) @@ -350,10 +355,17 @@ def _extract_aggregation_rows( ) scales = ns elif scale is not None: - scales = [ - _finite_positive("scale", v, i) - for i, v in enumerate(_broadcast("scale", scale, n_rows)) - ] + # bool is an int subclass, so float(True) == 1.0 would silently scale + # by one - a plausible typo for scale="auto" - and must fail closed. + scale_values = _broadcast("scale", scale, n_rows) + if isinstance(scale, (bool, np.bool_)) or any( + isinstance(v, (bool, np.bool_)) for v in scale_values + ): + raise ValueError( + "scale must be a number, a sequence of numbers, or the string " + "'auto'; got a boolean (did you mean scale='auto'?)" + ) + scales = [_finite_positive("scale", v, i) for i, v in enumerate(scale_values)] else: raise ValueError( f"scale is required with aggregation_result: pass a numeric " @@ -618,6 +630,35 @@ def to_pymc_marketing_lift_test( return pd.DataFrame(rows, columns=columns) +def _mask_prelude(arr: np.ndarray) -> str: + """Serialize a boolean mask into snippet statements (exact for any mask). + + Ones-based form mirroring Google's configure-model idiom: initialize + all-True, then for each group of columns sharing the same row pattern, + clear the group and set its True rows. Meridian's contract makes the + all-True base the natural one - channels without an experiment use all + periods - and every column has at least one True by the time this runs + (all-False columns are rejected upstream), so groups stay small. Position + lists go through ``.tolist()`` so plain ints are interpolated (numpy 2.x + reprs ``np.int64`` elements otherwise). Long lines for large masks are an + accepted trade-off: this is generated paste-code, not black-formatted + source. + """ + n_rows, n_cols = arr.shape + lines = [f"roi_calibration_period = np.ones(({n_rows}, {n_cols}), dtype=bool)"] + groups: Dict[Tuple[int, ...], List[int]] = {} + for col in range(n_cols): + if arr[:, col].all(): + continue + key = tuple(np.flatnonzero(arr[:, col]).tolist()) + groups.setdefault(key, []).append(col) + for rows_key, cols in groups.items(): + lines.append(f"roi_calibration_period[:, {cols!r}] = False") + lines.append(f"roi_calibration_period[np.ix_({list(rows_key)!r}, {cols!r})] = True") + body = "\n".join(lines) + return f"import numpy as np\n\n{body}\n\n" + + @dataclass(frozen=True) class ExperimentROI: """Per-experiment ROI contribution inside a :class:`MeridianROIPrior`.""" @@ -671,7 +712,7 @@ def to_code( channel: Optional[str] = None, media_channels: Optional[Sequence[str]] = None, single_channel: bool = False, - roi_calibration_period: Optional[str] = None, + roi_calibration_period: Optional[Union[str, np.ndarray]] = None, full_model_window: bool = False, ) -> str: """Ready-to-paste Meridian snippet (channel- and time-scoped; 1.7.0 pinned). @@ -691,31 +732,147 @@ def to_code( The prior's TIME scope is also required: Meridian's default ``roi_calibration_period=None`` applies the prior over all model times, but - the prior was estimated on the experiment window. Pass - ``roi_calibration_period=""`` (the boolean - ``(n_media_times, n_media_channels)`` mask for your window) or - ``full_model_window=True`` to acknowledge that the two coincide. + the prior was estimated on the experiment window. Three routes: + + - ``roi_calibration_period=`` - the + ``(n_media_times, n_media_channels)`` mask, typically built by + :func:`meridian_calibration_mask`. The array is serialized into the + snippet as a short ``np.ones`` + per-column-group assignment prelude. + Boolean dtype, or numeric containing only 0/1 (Google's own docs build + the mask with float zeros), is accepted and cast to bool; masked + arrays are rejected (fill or drop the mask explicitly). An all-False + mask is rejected, and so is ANY entirely-False column: Meridian + aggregates each channel's calibration spend through its mask column, + so an all-False column zeroes it - channels without an experiment + must use ALL periods (Google's documented convention; the builder + sets non-experiment channels all-True). Only ``roi_m`` priors can be + time-scoped: Meridian 1.7.0 rejects a non-None + ``roi_calibration_period`` unless the media prior type is ``'roi'``, + so ``parameter="mroi_m"`` priors must use ``full_model_window=True`` + (applies to the expression-string route too). Two things this method + cannot verify and the caller owns: the ROW count (no time coordinate + is passed here - the builder guarantees consistency when its + ``media_times`` matches the model's coordinates; hand-built arrays + are the caller's responsibility) and the column ORDER/identity (the + mask carries no channel labels - the ``media_channels`` given to the + builder and to this method must be the same list in the same order; + only the count is machine-checked). + - ``roi_calibration_period=""`` - a Python expression string + interpolated verbatim into the snippet (the pre-existing route). + - ``full_model_window=True`` - acknowledges that the experiment window and + the MMM window coincide. Note Meridian's own guidance: the + configure-model guide states the use of ``roi_calibration_period`` + "is not generally recommended because calibrating the ROI of a + specific time period does not necessarily improve estimation of the + overall ROI" - prefer ``full_model_window=True`` when the experiment + evidence reasonably transfers to the full window, and reserve the + mask for evidence genuinely specific to a narrower period. Snippets use the TensorFlow substrate of TensorFlow Probability; JAX-backed Meridian users should swap the import for ``tensorflow_probability.substrates.jax`` (noted in the generated code). """ if roi_calibration_period is None and not full_model_window: + if self.parameter == "roi_m": + remedy = ( + "Pass roi_calibration_period=, or full_model_window=True to acknowledge that the MMM " + "window and the experiment window coincide, or build the array " + "with meridian_calibration_mask(media_times=..., " + "media_channels=..., channel=..., window=...) and pass it here." + ) + else: + # Meridian 1.7.0 accepts roi_calibration_period only for 'roi' + # priors, so the mask routes would fail the next validation - + # recommend the one route that works for this parameter. + remedy = ( + f"Meridian 1.7.0 accepts roi_calibration_period only when the " + f"media prior type is 'roi', so a {self.parameter!r} prior has " + f"exactly one route: pass full_model_window=True to acknowledge " + f"the full-window interpretation." + ) raise ValueError( "to_code() needs the prior's time scope: Meridian's default " "roi_calibration_period=None applies the prior over ALL model times, " "but this prior was estimated on the EXPERIMENT window, and ROI " - "differs across windows under varying spend and saturation. Pass " - "roi_calibration_period=, " - "or full_model_window=True to acknowledge that the MMM window and the " - "experiment window coincide." + "differs across windows under varying spend and saturation. " + remedy ) if roi_calibration_period is not None and full_model_window: raise ValueError( "pass either roi_calibration_period or full_model_window=True, not both" ) - if roi_calibration_period is not None: + if roi_calibration_period is not None and self.parameter != "roi_m": + raise ValueError( + f"Meridian 1.7.0 rejects a non-None roi_calibration_period unless " + f"the media prior type is 'roi' " + f"(ModelSpec._validate_roi_calibration_period), so a " + f"{self.parameter!r} prior cannot be time-scoped via this argument " + f"- pass full_model_window=True to acknowledge the full-window " + f"interpretation instead" + ) + mask_prelude = "" + mask_arr: Optional[np.ndarray] = None + if roi_calibration_period is None: + calibration_period = "None" + elif isinstance(roi_calibration_period, np.ndarray): + if isinstance(roi_calibration_period, np.ma.MaskedArray): + raise TypeError( + "roi_calibration_period masked arrays are not accepted (np.asarray " + "would silently drop the mask, turning masked cells into " + "calibration values); fill or drop the mask explicitly" + ) + arr = np.asarray(roi_calibration_period) + if arr.ndim != 2: + raise ValueError( + f"roi_calibration_period array must be 2-D with shape " + f"(n_media_times, n_media_channels); got shape {arr.shape}" + ) + if arr.size == 0: + raise ValueError( + f"roi_calibration_period array must be non-empty; got shape " f"{arr.shape}" + ) + if arr.dtype != np.bool_: + if np.issubdtype(arr.dtype, np.number): + if not np.isin(arr, (0, 1)).all(): + raise ValueError( + "roi_calibration_period array must be boolean, or numeric " + "containing only 0/1 (Google's example builds it with " + "np.zeros); it contains values other than 0/1" + ) + arr = arr.astype(bool) + else: + raise ValueError( + f"roi_calibration_period array must be boolean, or numeric " + f"containing only 0/1 (Google's example builds it with " + f"np.zeros); got dtype {arr.dtype}" + ) + if not arr.any(): + raise ValueError( + "roi_calibration_period array is all False, which disables ROI " + "calibration at every time and silently discards the experiment " + "prior's time scope; build the mask with " + "meridian_calibration_mask(...) for your experiment window, or " + "pass full_model_window=True" + ) + # Meridian aggregates each channel's calibration spend through its + # mask column (input_data._aggregate_spend einsum), so an all-False + # column zeroes that channel's calibration spend - Google's own + # example sets channels without an experiment to ALL periods. + for col in range(arr.shape[1]): + if not arr[:, col].any(): + raise ValueError( + f"roi_calibration_period mask column {col} is entirely " + f"False, which zeroes that channel's aggregated calibration " + f"spend in Meridian; channels without an experiment must " + f"use ALL periods (Google's documented convention) - build " + f"the mask with meridian_calibration_mask(...), which sets " + f"non-experiment channels all-True" + ) + mask_arr = arr + mask_prelude = _mask_prelude(arr) + calibration_period = "roi_calibration_period" + elif isinstance(roi_calibration_period, str): try: ast.parse(roi_calibration_period, mode="eval") except SyntaxError as exc: @@ -725,12 +882,18 @@ def to_code( f"interpolated verbatim into the snippet); got " f"{roi_calibration_period!r}, which does not parse: {exc.msg}" ) from exc + calibration_period = roi_calibration_period + else: + raise TypeError( + f"roi_calibration_period must be a Python expression string or a " + f"boolean numpy array of shape (n_media_times, n_media_channels); " + f"got {type(roi_calibration_period).__name__}" + ) window_note = ( "Experiment window == full model window (acknowledged via full_model_window=True)." if full_model_window else "Mask restricting the prior to the experiment window." ) - calibration_period = "None" if full_model_window else roi_calibration_period prior_type = "roi" if self.parameter == "roi_m" else "mroi" if media_channels is not None: if single_channel: @@ -743,6 +906,12 @@ def to_code( f"channel must name the experiment channel within media_channels; " f"got channel={channel!r}, media_channels={channels!r}" ) + if mask_arr is not None and mask_arr.shape[1] != len(channels): + raise ValueError( + f"roi_calibration_period mask has {mask_arr.shape[1]} channel " + f"column(s) but media_channels has {len(channels)} channel(s); " + f"the mask's columns must align to media_channels order" + ) default_mu, default_sigma = _MERIDIAN_PARAM_DEFAULTS[self.parameter] mu_vector = [self.mu if c == channel else default_mu for c in channels] sigma_vector = [self.sigma if c == channel else default_sigma for c in channels] @@ -757,8 +926,15 @@ def to_code( window_note=window_note, calibration_period=calibration_period, prior_type=prior_type, + mask_prelude=mask_prelude, ) if single_channel: + if mask_arr is not None and mask_arr.shape[1] != 1: + raise ValueError( + f"single_channel=True but the roi_calibration_period mask has " + f"{mask_arr.shape[1]} channel columns; a single-channel model's " + f"mask must have exactly 1 column" + ) return _MERIDIAN_SINGLE_CHANNEL_TEMPLATE.format( mu=self.mu, sigma=self.sigma, @@ -766,6 +942,7 @@ def to_code( window_note=window_note, calibration_period=calibration_period, prior_type=prior_type, + mask_prelude=mask_prelude, ) raise ValueError( "to_code() needs explicit channel scope: a scalar prior broadcasts to " @@ -1018,3 +1195,289 @@ def to_meridian_roi_prior( parameter=parameter, per_experiment=per_experiment, ) + + +def _validate_label_sequence(name: str, value: Any) -> List[Any]: + """Container-type gate shared by the mask builder's sequence parameters. + + Accepts list/tuple/1-D ndarray/Series/Index (deliberately wider than + ``_is_sequence``, which rejects ``pd.Index`` - the natural type for a + model's coordinates). Wrong TYPES raise TypeError; ``pd.MultiIndex`` is + rejected here because ``pd.isna`` raises raw ``NotImplementedError`` on it + downstream. + """ + if isinstance(value, (str, bytes)) or isinstance(value, Mapping): + raise TypeError( + f"{name} must be a sequence of labels (list, tuple, ndarray, Series, " + f"or Index); got {type(value).__name__}" + ) + if isinstance(value, pd.MultiIndex): + raise TypeError( + f"{name} must be a flat sequence of labels; got a MultiIndex " + f"(Meridian coordinates are flat labels)" + ) + if isinstance(value, np.ndarray): + if value.ndim != 1: + raise TypeError( + f"{name} must be a 1-D sequence of labels; got a " f"{value.ndim}-D array" + ) + elif not isinstance(value, (list, tuple, pd.Series, pd.Index)): + raise TypeError( + f"{name} must be a sequence of labels (list, tuple, ndarray, Series, " + f"or Index); got {type(value).__name__}" + ) + return list(value) + + +def _coerce_window_value(value: Any, tz: Any) -> Any: + """Coerce one window bound/label against datetime media_times (fail-closed tz).""" + try: + coerced = pd.to_datetime(value) + except (ValueError, TypeError) as exc: + raise ValueError( + f"window value {value!r} could not be coerced to a datetime to match " + f"datetime media_times" + ) from exc + if tz is not None and coerced.tzinfo is None: + raise ValueError( + f"window value {value!r} is timezone-naive but media_times is " + f"timezone-aware ({tz}); pass tz-aware values or explicit labels" + ) + if tz is None and coerced.tzinfo is not None: + raise ValueError( + f"window value {value!r} is timezone-aware but media_times is " + f"timezone-naive; pass naive values or explicit labels" + ) + return coerced + + +def _window_bounds_selection(times_index: pd.Index, start: Any, end: Any) -> np.ndarray: + """Row selection for a (start, end) inclusive-bounds window.""" + for bound_name, bound in (("start", start), ("end", end)): + if isinstance(bound, (np.ndarray, list, tuple, pd.Series, pd.Index)): + raise TypeError( + f"window bounds must be scalar labels; got a " + f"{type(bound).__name__} for window {bound_name}" + ) + if pd.isna(bound): + raise ValueError( + f"window bounds must not be missing (None/NaN/NaT/pd.NA); got " + f"{bound!r} for window {bound_name}" + ) + if pd.api.types.is_datetime64_any_dtype(times_index.dtype): + tz = getattr(times_index, "tz", None) + start = _coerce_window_value(start, tz) + end = _coerce_window_value(end, tz) + try: + reversed_bounds = bool(start > end) + except TypeError: + reversed_bounds = False # unorderable -> the comparison funnel below raises + if reversed_bounds: + raise ValueError( + f"window start {start!r} is after window end {end!r}; bounds are " + f"inclusive (start, end)" + ) + try: + sel = np.asarray((times_index >= start) & (times_index <= end), dtype=bool) + except TypeError as exc: + raise ValueError( + f"window bounds ({start!r}, {end!r}) cannot be order-compared against " + f"media_times labels (mixed or mismatched types); pass window as a " + f"list of explicit time labels instead" + ) from exc + if not sel.any(): + raise ValueError( + f"window ({start!r}, {end!r}) selects no media_times labels; " + f"media_times runs {times_index[0]!r} .. {times_index[-1]!r} (bounds " + f"are inclusive and compared by value)" + ) + return sel + + +def _window_labels_selection(times_index: pd.Index, labels: List[Any]) -> np.ndarray: + """Row selection for an explicit-labels window (fail-closed membership).""" + if not labels: + raise ValueError("window must contain at least one time label") + for lab in labels: + # A missing label would fail the membership check anyway (a None + # coerces to NaT, never present in a complete coordinate index), but + # the message would then show the coerced NaT - name the actual input. + if not isinstance(lab, (np.ndarray, list, tuple, pd.Series, pd.Index)) and pd.isna(lab): + raise ValueError( + f"window labels must not be missing (None/NaN/NaT/pd.NA); got " f"{lab!r}" + ) + if pd.api.types.is_datetime64_any_dtype(times_index.dtype): + tz = getattr(times_index, "tz", None) + labels = [_coerce_window_value(lab, tz) for lab in labels] + missing = [lab for lab in labels if lab not in times_index] + if missing: + raise ValueError( + f"window label(s) {missing!r} not in media_times; labels are matched " + f"exactly (after pd.to_datetime coercion when media_times is " + f"datetime-like) - check formatting and timezone" + ) + return np.asarray(times_index.isin(labels), dtype=bool) + + +def meridian_calibration_mask( + *, + media_times: Sequence[Any], + media_channels: Sequence[str], + channel: Union[str, Sequence[str]], + window: Union[Tuple[Any, Any], Sequence[Any]], +) -> np.ndarray: + """Build Meridian's boolean ``roi_calibration_period`` mask for an experiment. + + Returns a ``(len(media_times), len(media_channels))`` bool array: the + experiment channel's column(s) are True exactly on the selected window, + and every OTHER channel's column is all-True - Meridian's documented + convention ("any media channels not specified ... will utilize all + available periods for ROI calibration", configure-model guide); an + all-False column would zero that channel's aggregated calibration spend. + Suitable to pass straight to Meridian's + ``spec.ModelSpec(roi_calibration_period=...)``, or to + :meth:`MeridianROIPrior.to_code`, which serializes it into the generated + snippet. Valid for ``roi_m`` priors only: Meridian 1.7.0 rejects a + non-None ``roi_calibration_period`` unless the media prior type is + ``'roi'``. + + **Window convention:** a 2-TUPLE is ``(start, end)`` INCLUSIVE bounds; any + OTHER sequence (list, ndarray, Series, Index) is a set of explicit time + labels. To select exactly two labels, pass a list ``[a, b]``, not a tuple. + + Parameters + ---------- + media_times : list, tuple, ndarray, Series, or Index + The MMM's time coordinate labels, taken VERBATIM in model order (str + dates, datetimes, periods, or ints; unique, no missing labels; these + five container types are the accepted forms - materialize e.g. a + ``range`` with ``list(...)``). + Meridian's ``n_media_times`` can exceed ``len(data.time)`` when + ``max_lag > 0`` adds lagged leading periods, yet Google's own + configure-model example builds the mask with ``len(data.time)`` rows - + this builder does not resolve that nuance: pass whichever coordinate + list your ``ModelSpec`` expects, and the mask gets one row per entry. + String labels order-compare lexicographically under a bounds window - + right for zero-padded ISO dates (``'2021-11-01'``), wrong for + ``'1/2/2021'``-style formats (use explicit labels or datetime labels + there). + media_channels : list, tuple, ndarray, Series, or Index of str + Channel names in the Meridian ``InputData`` media-channel order (the + same contract as ``to_code(media_channels=)``); unique, non-empty. The + mask's columns are positioned by this order - keep the SAME list, in + the same order, for the builder and for ``to_code``. + channel : str or sequence of str + The experiment channel(s) whose columns carry the window - here a + sequence means "the set of mask columns to mark for THIS experiment" + (unlike ``to_pymc_marketing_lift_test(channel=...)``, where a sequence + is one channel per experiment row; the builder has no per-row axis). + Every name must be in ``media_channels``. + window : tuple or sequence + Either ``(start, end)`` inclusive bounds - selected by order comparison + against ``media_times``, with ``pd.to_datetime`` coercion of the values + when ``media_times`` is datetime-like (timezone mismatches fail closed + in both directions) - or a sequence of explicit time labels, every one + of which must be present in ``media_times``. Must select at least one + time. Selection is by label VALUE at its position, so an unsorted + ``media_times`` is well-defined. + + Returns + ------- + np.ndarray + Boolean, shape ``(len(media_times), len(media_channels))``. + + Raises + ------ + TypeError + On wrong-typed inputs (string/Mapping/scalar/MultiIndex/non-1-D-array + containers, a scalar or Mapping ``window``, an array-valued window + bound). + ValueError + On empty inputs, duplicate or missing labels, unknown channels, + malformed or empty-selection windows, and unorderable or timezone- + mismatched bounds. + """ + times = _validate_label_sequence("media_times", media_times) + if not times: + raise ValueError("media_times must be non-empty") + times_index = pd.Index(times) + if isinstance(times_index, pd.MultiIndex): + raise TypeError( + "media_times contains tuple-valued labels, which construct a " + "MultiIndex; Meridian time coordinates are flat labels" + ) + if pd.isna(times_index).any(): + raise ValueError( + "media_times contains missing label(s) (NaN/NaT); Meridian time " + "coordinates are complete, and a missing label would silently " + "un-select its row" + ) + if times_index.has_duplicates: + dupes = times_index[times_index.duplicated()].unique().tolist() + raise ValueError( + f"media_times contains duplicate label(s): {dupes!r}; time labels " + f"must be unique to map labels to mask rows" + ) + channels = _validate_label_sequence("media_channels", media_channels) + if not channels: + raise ValueError("media_channels must be non-empty") + # Missing elements fail closed with a named error: a pd.NA element would + # otherwise raise a raw ambiguous-truth TypeError inside the duplicate + # check, and a None would silently become a mask column. + if any(pd.isna(c) for c in channels): + raise ValueError("media_channels must not contain missing names (None/NaN/pd.NA)") + dup_channels = sorted({c for i, c in enumerate(channels) if c in channels[:i]}) + if dup_channels: + raise ValueError(f"media_channels contains duplicate channel(s): {dup_channels!r}") + if isinstance(channel, str): + channel_list = [channel] + else: + channel_list = _validate_label_sequence("channel", channel) + if not channel_list: + raise ValueError("channel must name at least one experiment channel") + if any(pd.isna(c) for c in channel_list): + raise ValueError("channel must not contain missing names (None/NaN/pd.NA)") + dup_exp = sorted({c for i, c in enumerate(channel_list) if c in channel_list[:i]}) + if dup_exp: + raise ValueError(f"channel contains duplicate name(s): {dup_exp!r}") + missing_channels = [c for c in channel_list if c not in channels] + if missing_channels: + raise ValueError( + f"channel(s) {missing_channels!r} not in media_channels {channels!r}; " + f"the mask's columns are positioned by media_channels order" + ) + if isinstance(window, tuple): + if len(window) != 2: + raise ValueError( + "window given as a tuple must be exactly (start, end); to select " + "explicit time labels pass a list" + ) + sel = _window_bounds_selection(times_index, window[0], window[1]) + elif isinstance(window, (str, bytes)) or isinstance(window, Mapping): + raise TypeError( + f"window must be a (start, end) tuple (inclusive bounds) or a " + f"sequence of explicit time labels; got {type(window).__name__}" + ) + elif isinstance(window, np.ndarray) and window.ndim != 1: + raise TypeError( + f"window must be a (start, end) tuple (inclusive bounds) or a 1-D " + f"sequence of explicit time labels; got a {window.ndim}-D array" + ) + elif isinstance(window, (list, np.ndarray, pd.Series, pd.Index)): + sel = _window_labels_selection(times_index, list(window)) + else: + raise TypeError( + f"window must be a (start, end) tuple (inclusive bounds) or a " + f"sequence of explicit time labels; got {type(window).__name__}" + ) + # Meridian's documented convention (configure-model guide): channels not + # named in the experiment use ALL periods for ROI calibration - an + # all-False column would zero that channel's aggregated calibration spend + # (input_data._aggregate_spend). So: all-True base, experiment columns + # cleared, then their window rows set. + mask = np.ones((len(times), len(channels)), dtype=bool) + cols = [channels.index(c) for c in channel_list] + mask[:, cols] = False + rows = np.flatnonzero(sel) + mask[np.ix_(rows, cols)] = True + return mask diff --git a/docs/api/_autosummary/diff_diff.meridian_calibration_mask.rst b/docs/api/_autosummary/diff_diff.meridian_calibration_mask.rst new file mode 100644 index 000000000..22e0bf9d6 --- /dev/null +++ b/docs/api/_autosummary/diff_diff.meridian_calibration_mask.rst @@ -0,0 +1,7 @@ +diff\_diff.meridian\_calibration\_mask +======================================= + +.. currentmodule:: diff_diff + +.. autofunction:: meridian_calibration_mask + :no-index: diff --git a/docs/api/index.rst b/docs/api/index.rst index 13ddf823e..e872813db 100644 --- a/docs/api/index.rst +++ b/docs/api/index.rst @@ -262,6 +262,7 @@ Convert experiment results into Marketing Mix Model calibration inputs diff_diff.to_pymc_marketing_lift_test diff_diff.to_meridian_roi_prior + diff_diff.meridian_calibration_mask diff_diff.MeridianROIPrior Boundary Local-Linear Estimators diff --git a/docs/api/mmm.rst b/docs/api/mmm.rst index 34b1b0550..c3646356f 100644 --- a/docs/api/mmm.rst +++ b/docs/api/mmm.rst @@ -152,10 +152,50 @@ Example ) print(prior.roi_mean, prior.roi_sd) - # Channel- and time-scoped snippet: roi_m is per-channel, and the prior was - # estimated on the experiment window. +Scoping the prior to the experiment window +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +``to_code()`` requires the prior's time scope. Build the boolean +``(n_media_times, n_media_channels)`` mask with +:func:`~diff_diff.meridian_calibration_mask` and pass it directly - the array is +serialized into the generated snippet (a string expression or +``full_model_window=True`` also work). Note Meridian's own guidance: its +configure-model guide states that ``roi_calibration_period`` "is not generally +recommended because calibrating the ROI of a specific time period does not +necessarily improve estimation of the overall ROI" - prefer +``full_model_window=True`` when the experiment evidence reasonably transfers to +the full window, and reserve the mask for evidence genuinely specific to a +narrower period: + +.. code-block:: python + + import pandas as pd + + from diff_diff import meridian_calibration_mask, to_meridian_roi_prior + + prior = to_meridian_roi_prior( + incremental_outcome=180_000.0, + incremental_outcome_se=45_000.0, + spend=200_000.0, + ) + + # The MMM's own coordinates: time labels in model order, channels in + # InputData order. window=(start, end) is inclusive; pass a list for + # explicit (possibly non-contiguous) time labels instead. + media_times = pd.date_range('2023-09-04', periods=52, freq='W-MON') + mask = meridian_calibration_mask( + media_times=media_times, + media_channels=['search', 'tv'], + channel='tv', + window=('2024-01-15', '2024-03-04'), + ) print(prior.to_code(channel='tv', media_channels=['search', 'tv'], - roi_calibration_period='experiment_window_mask')) + roi_calibration_period=mask)) + # The snippet rebuilds the same mask (all-True base; the experiment + # channel's column carries only the window - other channels keep ALL + # periods, Meridian's documented convention) and passes it to + # ModelSpec(roi_calibration_period=...). roi_m priors only: Meridian + # rejects the mask for mroi_m (use full_model_window=True there). Deriving totals from a fitted aggregation ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -181,6 +221,13 @@ those containers' ``n`` does not count treated unit-periods (CallawaySantAnna's ) # incremental_outcome == att * 132.0, and its SE == se * 132.0. +meridian_calibration_mask +------------------------- + +Build the boolean ``roi_calibration_period`` mask from the MMM's own coordinates. + +.. autofunction:: diff_diff.meridian_calibration_mask + MeridianROIPrior ---------------- @@ -200,5 +247,8 @@ References https://developers.google.com/meridian/docs/advanced-modeling/set-custom-priors-past-experiments - Google Meridian, ROI/mROI/contribution parameterizations: https://developers.google.com/meridian/docs/advanced-modeling/roi-mroi-contribution-parameterizations +- Google Meridian, "Set the ROI calibration period" (the + ``roi_calibration_period`` mask shape/semantics contract): + https://developers.google.com/meridian/docs/user-guide/configure-model - Zhou, G., Choe, Y., & Hetrakul, C. (2023). Calibrated MMM better predicts true ROAS. Meta Marketing Science. diff --git a/docs/methodology/REGISTRY.md b/docs/methodology/REGISTRY.md index 1379f5c7b..c843ef96d 100644 --- a/docs/methodology/REGISTRY.md +++ b/docs/methodology/REGISTRY.md @@ -6315,7 +6315,7 @@ estimator-focused: ## MMM Calibration Export (interop) -**Primary sources:** [PyMC-Marketing lift-test calibration](https://www.pymc-marketing.io/en/stable/notebooks/mmm/mmm_lift_test.html) (`MMM.add_lift_test_measurements` schema); [Google Meridian, "Set custom prior distributions using past experiments"](https://developers.google.com/meridian/docs/advanced-modeling/set-custom-priors-past-experiments) and the closed-form lognormal conversion in `meridian/model/prior_distribution.py` (`lognormal_dist_from_mean_std`, Meridian 1.7.0); the roi_m/mroi_m estimand definitions in [Meridian's ROI parameterization docs](https://developers.google.com/meridian/docs/advanced-modeling/roi-mroi-contribution-parameterizations). +**Primary sources:** [PyMC-Marketing lift-test calibration](https://www.pymc-marketing.io/en/stable/notebooks/mmm/mmm_lift_test.html) (`MMM.add_lift_test_measurements` schema); [Google Meridian, "Set custom prior distributions using past experiments"](https://developers.google.com/meridian/docs/advanced-modeling/set-custom-priors-past-experiments) and the closed-form lognormal conversion in `meridian/model/prior_distribution.py` (`lognormal_dist_from_mean_std`, Meridian 1.7.0); the roi_m/mroi_m estimand definitions in [Meridian's ROI parameterization docs](https://developers.google.com/meridian/docs/advanced-modeling/roi-mroi-contribution-parameterizations); the `roi_calibration_period` mask shape/semantics contract in [Meridian's "Set the ROI calibration period"](https://developers.google.com/meridian/docs/user-guide/configure-model) (configure-model guide). **Module:** `diff_diff/mmm.py` @@ -6340,7 +6340,10 @@ estimator-focused: - Lognormal conversion matches Meridian's `lognormal_dist_from_mean_std` exactly: `sigma = sqrt(log1p((s/m)^2))`, `mu = ln(m) - log1p((s/m)^2)/2` (`log1p` keeps a ~1e-8 relative SE from rounding `sigma` to 0). - A non-positive pooled ROI mean raises (lognormal positivity), pointing to pooling, wider priors, or Meridian's contribution/coefficient parameterizations. `spend`, `incremental_outcome_se`, and `se_widening` must be finite and positive; `incremental_outcome` finite. - **Note:** Pooled `roi_sd = sqrt(sum((w_i * sd_i)^2))` treats experiments as independent (no covariance term). Ignoring covariance can misstate the pooled sd - it is anti-conservative when the net weighted covariance is positive, which experiments sharing control units or overlapping windows typically induce - and the direction cannot be determined from marginal SEs alone; the docstring instructs users to widen via `se_widening` (a conservative heuristic, not an exact correction). -- **Channel scope in `.to_code()`**: roi_m/mroi_m have batch shape `n_media_channels` and a scalar distribution broadcasts to EVERY channel, so the snippet helper requires explicit scope - `channel=`+`media_channels=` emits a vector prior in model channel order (non-experiment channels keep the parameter's Meridian default), `single_channel=True` emits a scalar snippet marked single-channel-only, and neither raises. It also requires the prior's TIME scope (`roi_calibration_period=` or `full_model_window=True`), because Meridian's default applies an experiment-window prior over all model times; and it sets `media_prior_type="roi"`/`"mroi"` on `ModelSpec` (Meridian ignores a supplied `roi_m`/`mroi_m` unless the matching prior type is selected). Snippets use the TensorFlow substrate; a comment points JAX users at `tensorflow_probability.substrates.jax`. +- **Channel scope in `.to_code()`**: roi_m/mroi_m have batch shape `n_media_channels` and a scalar distribution broadcasts to EVERY channel, so the snippet helper requires explicit scope - `channel=`+`media_channels=` emits a vector prior in model channel order (non-experiment channels keep the parameter's Meridian default), `single_channel=True` emits a scalar snippet marked single-channel-only, and neither raises. It also requires the prior's TIME scope (`roi_calibration_period=`, `roi_calibration_period=`, or `full_model_window=True`), because Meridian's default applies an experiment-window prior over all model times; and it sets `media_prior_type="roi"`/`"mroi"` on `ModelSpec` (Meridian ignores a supplied `roi_m`/`mroi_m` unless the matching prior type is selected). Snippets use the TensorFlow substrate; a comment points JAX users at `tensorflow_probability.substrates.jax`. +- **Note (mask builder + ndarray serialization):** `meridian_calibration_mask(media_times=, media_channels=, channel=, window=)` builds the boolean `(n_media_times, n_media_channels)` mask: the experiment channel's column(s) are True exactly on the window, and every OTHER channel's column is ALL-TRUE - Meridian's documented convention ("any media channels not specified ... will utilize all available periods for ROI calibration", configure-model guide; an all-False column zeroes that channel's aggregated calibration spend via `input_data._aggregate_spend`'s einsum). Window convention: a 2-TUPLE is `(start, end)` INCLUSIVE bounds (order-compared by value, `pd.to_datetime`-coerced when `media_times` is datetime-like, timezone mismatches fail closed both ways); any OTHER sequence is explicit labels, each required present (fail-closed membership, no silent drop). `to_code()` accepts the array and serializes it into the snippet as an `np.ones` prelude plus, per group of columns sharing a row pattern, a column-clear + `np.ix_` window assignment (indices `.tolist()`-coerced). Meridian's guide adds that `roi_calibration_period` "is not generally recommended because calibrating the ROI of a specific time period does not necessarily improve estimation of the overall ROI" - the docstrings point users at `full_model_window=True` when the experiment evidence reasonably transfers, reserving the mask for genuinely period-specific evidence. +- **Note (ndarray acceptance rules):** boolean dtype, or numeric containing ONLY 0/1 (cast to bool - Google's own configure-model example builds the mask with float `np.zeros`); anything else rejected, masked arrays rejected (`np.asarray` would silently drop the mask). An ALL-FALSE mask is rejected, and so is ANY entirely-False COLUMN (it would zero that channel's aggregated calibration spend; channels without an experiment must use all periods). Only `roi_m` priors accept a mask: Meridian 1.7.0's `ModelSpec._validate_roi_calibration_period` rejects a non-None `roi_calibration_period` unless the media prior type is `'roi'`, so `to_code()` fails closed for `mroi_m` on BOTH the ndarray and expression-string routes (`full_model_window=True` remains the mroi route) - this also corrects the pre-3.10 expression route, which emitted mroi snippets Meridian rejects at `ModelSpec` construction. +- **Note (what the mask route cannot verify):** the mask's ROW count (to_code receives no time coordinate; the builder guarantees consistency when its `media_times` matches the model's coordinates - hand-built arrays are the caller's responsibility) and the column ORDER/identity (the mask carries no channel labels; the `media_channels` given to the builder and to `to_code` must be the same list in the same order - only the count is machine-checked). Coordinates are taken VERBATIM: Meridian's `n_media_times` can exceed `len(data.time)` under `max_lag > 0`, yet Google's own example builds `len(data.time)` rows; the builder does not resolve that nuance - the caller passes whichever coordinate list their `ModelSpec` expects. *Outputs:* All scaled and pooled values (`roi`, `roi_sd`, pooled moments, lognormal `mu`/`sigma`) are validated finite-and-positive after arithmetic; overflow/underflow raises rather than emitting non-finite or zero-uncertainty calibration data. diff --git a/docs/references.rst b/docs/references.rst index b67cd3c81..2db0a84f7 100644 --- a/docs/references.rst +++ b/docs/references.rst @@ -361,6 +361,10 @@ MMM Calibration Interop Documents the experiment-to-ROI-prior calibration workflow and caveats; the lognormal ``(mu, sigma)`` closed form emitted by ``to_meridian_roi_prior`` matches Meridian's ``prior_distribution.lognormal_dist_from_mean_std`` (verified against Meridian 1.7.0 source). +- **Google.** "Set the ROI Calibration Period." *Google Meridian documentation (configure-model guide)*. https://developers.google.com/meridian/docs/user-guide/configure-model + + Defines the ``roi_calibration_period`` contract ``meridian_calibration_mask`` builds against: an optional boolean array of shape ``(n_media_times, n_media_channels)``, True where a period participates in ROI calibration, time labels drawn from the model's ``time`` coordinate and channels ordered by ``data.media_channel``. + - **Zhou, G., Choe, Y., & Hetrakul, C. (2023).** "Calibrated MMM Better Predicts True ROAS." *Meta Marketing Science*. https://medium.com/@gufengzhou/calibrated-mmm-better-predicts-true-roas-d5adfc8abdc4 Empirical motivation for experiment calibration: calibrating MMMs against lift experiments substantially reduces ROAS prediction error. diff --git a/tests/test_mmm.py b/tests/test_mmm.py index 4c777aa49..55aab5007 100644 --- a/tests/test_mmm.py +++ b/tests/test_mmm.py @@ -24,6 +24,7 @@ DifferenceInDifferences, ImputationDiD, TwoStageDiD, + meridian_calibration_mask, to_meridian_roi_prior, to_pymc_marketing_lift_test, ) @@ -422,6 +423,561 @@ def test_scope_mutually_exclusive(self): single_channel=True, full_model_window=True, ) + with pytest.raises(ValueError, match="not both"): + self._prior().to_code( + single_channel=True, + roi_calibration_period=np.ones((3, 1), dtype=bool), + full_model_window=True, + ) + + +class TestMeridianCalibrationMask: + """meridian_calibration_mask: window resolution + fail-closed validation.""" + + _TIMES = pd.date_range("2024-01-01", periods=5, freq="W-MON") + _CHANNELS = ["search", "tv", "radio"] + + def _expected(self, rows, cols): + # Meridian's convention: non-experiment channels use ALL periods, so + # the expected mask is all-True except the experiment columns, which + # carry only the window rows. + expected = np.ones((5, 3), dtype=bool) + expected[:, cols] = False + expected[np.ix_(rows, cols)] = True + return expected + + def _assert_mask(self, mask, rows, cols): + assert mask.dtype == np.bool_ + assert mask.shape == (5, 3) + assert np.array_equal(mask, self._expected(rows, cols)) + + def test_bounds_window_datetime_media_times_str_bounds(self): + mask = meridian_calibration_mask( + media_times=self._TIMES, + media_channels=self._CHANNELS, + channel="tv", + window=("2024-01-15", "2024-01-29"), # inclusive both ends + ) + self._assert_mask(mask, [2, 3, 4], [1]) + + def test_bounds_window_numeric_labels(self): + mask = meridian_calibration_mask( + media_times=[1, 2, 3, 4, 5], + media_channels=self._CHANNELS, + channel="tv", + window=(2, 4), + ) + self._assert_mask(mask, [1, 2, 3], [1]) + + def test_bounds_window_iso_string_labels(self): + labels = [str(t.date()) for t in self._TIMES] + mask = meridian_calibration_mask( + media_times=labels, + media_channels=self._CHANNELS, + channel="search", + window=("2024-01-15", "2024-01-29"), + ) + self._assert_mask(mask, [2, 3, 4], [0]) + + def test_explicit_labels_window_noncontiguous_multichannel(self): + mask = meridian_calibration_mask( + media_times=[1, 2, 3, 4, 5], + media_channels=self._CHANNELS, + channel=["tv", "radio"], + window=[1, 3, 5], + ) + self._assert_mask(mask, [0, 2, 4], [1, 2]) + + def test_media_times_accepts_index_and_series(self): + from_index = meridian_calibration_mask( + media_times=pd.Index([1, 2, 3, 4, 5]), + media_channels=self._CHANNELS, + channel="tv", + window=(2, 4), + ) + from_series = meridian_calibration_mask( + media_times=pd.Series([1, 2, 3, 4, 5], index=[9, 8, 7, 6, 5]), + media_channels=self._CHANNELS, + channel="tv", + window=(2, 4), + ) + assert np.array_equal(from_index, from_series) + self._assert_mask(from_index, [1, 2, 3], [1]) + + def test_channel_string_vs_sequence_equivalent(self): + a = meridian_calibration_mask( + media_times=[1, 2, 3, 4, 5], + media_channels=self._CHANNELS, + channel="tv", + window=(1, 5), + ) + b = meridian_calibration_mask( + media_times=[1, 2, 3, 4, 5], + media_channels=self._CHANNELS, + channel=["tv"], + window=(1, 5), + ) + assert np.array_equal(a, b) + + def test_tz_aware_success(self): + times = pd.date_range("2024-01-01", periods=5, freq="W-MON", tz="UTC") + mask = meridian_calibration_mask( + media_times=times, + media_channels=self._CHANNELS, + channel="tv", + window=(pd.Timestamp("2024-01-15", tz="UTC"), pd.Timestamp("2024-01-29", tz="UTC")), + ) + self._assert_mask(mask, [2, 3, 4], [1]) + + def test_datetime_coercion_on_labels_path(self): + mask = meridian_calibration_mask( + media_times=self._TIMES, + media_channels=self._CHANNELS, + channel="tv", + window=["2024-01-01", "2024-01-29"], # str labels vs DatetimeIndex + ) + self._assert_mask(mask, [0, 4], [1]) + + def test_duplicate_window_labels_ok(self): + mask = meridian_calibration_mask( + media_times=[1, 2, 3, 4, 5], + media_channels=self._CHANNELS, + channel="tv", + window=[3, 3], + ) + self._assert_mask(mask, [2], [1]) + + def test_missing_channel_raises(self): + with pytest.raises(ValueError, match="not in media_channels"): + meridian_calibration_mask( + media_times=[1, 2], + media_channels=["search"], + channel="tv", + window=(1, 2), + ) + + def test_missing_window_label_raises(self): + with pytest.raises(ValueError, match="not in media_times"): + meridian_calibration_mask( + media_times=[1, 2], + media_channels=["tv"], + channel="tv", + window=[1, 7], + ) + + def test_empty_and_bad_media_times(self): + with pytest.raises(ValueError, match="media_times must be non-empty"): + meridian_calibration_mask( + media_times=[], media_channels=["tv"], channel="tv", window=(1, 2) + ) + with pytest.raises(ValueError, match="duplicate label"): + meridian_calibration_mask( + media_times=[1, 1, 2], media_channels=["tv"], channel="tv", window=(1, 2) + ) + with pytest.raises(ValueError, match="NaN/NaT"): + meridian_calibration_mask( + media_times=[1.0, math.nan], + media_channels=["tv"], + channel="tv", + window=(1, 2), + ) + + def test_missing_channel_names_raise(self): + # pd.NA would otherwise raise a raw ambiguous-truth TypeError in the + # duplicate check; None would silently become a mask column. + for bad in (None, math.nan, pd.NA): + with pytest.raises(ValueError, match="must not contain missing names"): + meridian_calibration_mask( + media_times=[1, 2], + media_channels=["tv", bad], + channel="tv", + window=(1, 2), + ) + with pytest.raises(ValueError, match="must not contain missing names"): + meridian_calibration_mask( + media_times=[1, 2], + media_channels=["tv", "search"], + channel=["tv", bad], + window=(1, 2), + ) + + def test_duplicate_channels_raise(self): + with pytest.raises(ValueError, match="duplicate channel"): + meridian_calibration_mask( + media_times=[1, 2], + media_channels=["tv", "tv"], + channel="tv", + window=(1, 2), + ) + with pytest.raises(ValueError, match="duplicate name"): + meridian_calibration_mask( + media_times=[1, 2], + media_channels=["tv", "search"], + channel=["tv", "tv"], + window=(1, 2), + ) + + def test_empty_channel_and_media_channels(self): + with pytest.raises(ValueError, match="media_channels must be non-empty"): + meridian_calibration_mask( + media_times=[1, 2], media_channels=[], channel="tv", window=(1, 2) + ) + with pytest.raises(ValueError, match="at least one experiment channel"): + meridian_calibration_mask( + media_times=[1, 2], media_channels=["tv"], channel=[], window=(1, 2) + ) + + def test_window_tuple_wrong_length_raises(self): + with pytest.raises(ValueError, match="exactly"): + meridian_calibration_mask( + media_times=[1, 2, 3], + media_channels=["tv"], + channel="tv", + window=(1, 2, 3), + ) + + def test_window_empty_labels_raises(self): + with pytest.raises(ValueError, match="at least one time label"): + meridian_calibration_mask( + media_times=[1, 2], media_channels=["tv"], channel="tv", window=[] + ) + + def test_reversed_bounds_raises(self): + with pytest.raises(ValueError, match="after window end"): + meridian_calibration_mask( + media_times=[1, 2, 3], + media_channels=["tv"], + channel="tv", + window=(3, 1), + ) + + def test_bounds_select_nothing_raises(self): + with pytest.raises(ValueError, match="selects no"): + meridian_calibration_mask( + media_times=[1, 2, 3], + media_channels=["tv"], + channel="tv", + window=(7, 9), + ) + + def test_unorderable_bounds_raise(self): + with pytest.raises(ValueError, match="order-compared"): + meridian_calibration_mask( + media_times=["a", "b", "c"], + media_channels=["tv"], + channel="tv", + window=(1, 2), + ) + + def test_unparseable_datetime_bound_raises(self): + with pytest.raises(ValueError, match="coerced"): + meridian_calibration_mask( + media_times=self._TIMES, + media_channels=["tv"], + channel="tv", + window=("not-a-date", "2024-01-29"), + ) + + def test_tz_mismatch_fails_closed_both_ways(self): + aware = pd.date_range("2024-01-01", periods=3, freq="W-MON", tz="UTC") + with pytest.raises(ValueError, match="timezone"): + meridian_calibration_mask( + media_times=aware, + media_channels=["tv"], + channel="tv", + window=("2024-01-01", "2024-01-15"), + ) + naive = pd.date_range("2024-01-01", periods=3, freq="W-MON") + with pytest.raises(ValueError, match="timezone"): + meridian_calibration_mask( + media_times=naive, + media_channels=["tv"], + channel="tv", + window=(pd.Timestamp("2024-01-01", tz="UTC"), pd.Timestamp("2024-01-15", tz="UTC")), + ) + + def test_missing_bound_raises(self): + for bad in (None, math.nan, pd.NaT, pd.NA): + with pytest.raises(ValueError, match="must not be missing"): + meridian_calibration_mask( + media_times=[1, 2, 3], + media_channels=["tv"], + channel="tv", + window=(bad, 2), + ) + + def test_missing_window_label_raises_named(self): + # Missing labels fail closed on BOTH coordinate kinds, naming the + # actual input (not its NaT coercion). + times = pd.date_range("2024-01-01", periods=3, freq="W-MON") + for bad in (None, math.nan, pd.NaT, pd.NA): + with pytest.raises(ValueError, match="must not be missing"): + meridian_calibration_mask( + media_times=times, + media_channels=["tv"], + channel="tv", + window=["2024-01-01", bad], + ) + with pytest.raises(ValueError, match="must not be missing"): + meridian_calibration_mask( + media_times=[1, 2, 3], + media_channels=["tv"], + channel="tv", + window=[1, None], + ) + + def test_wrong_typed_media_times(self): + multi = pd.MultiIndex.from_tuples([(1, 2), (3, 4)]) + for bad in ("abc", {"a": 1}, 5, multi, np.zeros((2, 2)), np.array(1.0)): + with pytest.raises(TypeError): + meridian_calibration_mask( + media_times=bad, media_channels=["tv"], channel="tv", window=(1, 2) + ) + # Tuple-valued labels pass the outer gate but pd.Index promotes them to + # a MultiIndex - rejected after construction. + with pytest.raises(TypeError, match="MultiIndex"): + meridian_calibration_mask( + media_times=[(1, 2), (3, 4)], + media_channels=["tv"], + channel="tv", + window=(1, 2), + ) + + def test_wrong_typed_media_channels(self): + for bad in ("abc", {"a": 1}, 5, np.zeros((2, 2))): + with pytest.raises(TypeError): + meridian_calibration_mask( + media_times=[1, 2], media_channels=bad, channel="tv", window=(1, 2) + ) + + def test_wrong_typed_channel_and_window(self): + with pytest.raises(TypeError): + meridian_calibration_mask( + media_times=[1, 2], + media_channels=["tv"], + channel={"a": 1}, + window=(1, 2), + ) + with pytest.raises(TypeError): + meridian_calibration_mask( + media_times=[1, 2], + media_channels=["tv"], + channel=np.array(1.0), + window=(1, 2), + ) + for bad_window in (5, "2024-01-01", {"a": 1}): + with pytest.raises(TypeError, match=r"\(start, end\) tuple"): + meridian_calibration_mask( + media_times=[1, 2], + media_channels=["tv"], + channel="tv", + window=bad_window, + ) + with pytest.raises(TypeError, match="must be scalar labels"): + meridian_calibration_mask( + media_times=[1, 2, 3], + media_channels=["tv"], + channel="tv", + window=(np.array([1, 2]), 3), + ) + + +class TestToCodeArrayMask: + """to_code() ndarray route: validation, serialization, round-trips.""" + + def _prior(self): + return to_meridian_roi_prior( + incremental_outcome=9_600.0, + incremental_outcome_se=2_400.0, + spend=20_000.0, + ) + + def _builder_mask(self): + return meridian_calibration_mask( + media_times=pd.date_range("2024-01-01", periods=5, freq="W-MON"), + media_channels=["search", "tv"], + channel="tv", + window=("2024-01-15", "2024-01-29"), + ) + + @staticmethod + def _exec_prelude(code, stop_marker): + ns = {} + prelude = code[code.index("import numpy as np") : code.index(stop_marker)] + exec(compile(prelude, "", "exec"), ns) + return ns["roi_calibration_period"] + + def test_array_mask_prelude_and_slot(self): + code = self._prior().to_code( + channel="tv", + media_channels=["search", "tv"], + roi_calibration_period=self._builder_mask(), + ) + assert "import numpy as np" in code + assert "np.ones((5, 2), dtype=bool)" in code + assert "roi_calibration_period[:, [1]] = False" in code + assert "np.ix_([2, 3, 4], [1])" in code + assert "roi_calibration_period=roi_calibration_period" in code + + def test_array_mask_round_trip(self): + from diff_diff.mmm import _mask_prelude + + mask = self._builder_mask() + # Helper-level round-trip. + ns = {} + exec(compile(_mask_prelude(mask), "", "exec"), ns) + assert np.array_equal(ns["roi_calibration_period"], mask) + assert ns["roi_calibration_period"].dtype == np.bool_ + assert ns["roi_calibration_period"].shape == mask.shape + # Snippet-slice round-trip (multi-channel snippet: prelude ends at mu =). + code = self._prior().to_code( + channel="tv", media_channels=["search", "tv"], roi_calibration_period=mask + ) + rebuilt = self._exec_prelude(code, "mu = ") + assert np.array_equal(rebuilt, mask) + + def test_array_mask_per_channel_windows_round_trip(self): + # Different window per channel: two column groups in the prelude, each + # cleared and re-set, must reproduce the mask exactly. + mask = np.zeros((4, 2), dtype=bool) + mask[0:2, 0] = True + mask[2:4, 1] = True + from diff_diff.mmm import _mask_prelude + + prelude = _mask_prelude(mask) + assert prelude.count("= False") == 2 # one clear per column group + code = self._prior().to_code( + channel="tv", media_channels=["search", "tv"], roi_calibration_period=mask + ) + rebuilt = self._exec_prelude(code, "mu = ") + assert np.array_equal(rebuilt, mask) + assert rebuilt.dtype == np.bool_ + + def test_array_mask_all_true_round_trip(self): + mask = np.ones((3, 2), dtype=bool) + from diff_diff.mmm import _mask_prelude + + prelude = _mask_prelude(mask) + assert "= False" not in prelude # all-True needs only the ones init + ns = {} + exec(compile(prelude, "", "exec"), ns) + assert np.array_equal(ns["roi_calibration_period"], mask) + + def test_array_mask_channel_count_mismatch(self): + mask = np.ones((3, 3), dtype=bool) + with pytest.raises(ValueError, match="channel column"): + self._prior().to_code( + channel="tv", media_channels=["search", "tv"], roi_calibration_period=mask + ) + + def test_array_mask_single_channel_column_check(self): + with pytest.raises(ValueError, match="exactly 1 column"): + self._prior().to_code( + single_channel=True, roi_calibration_period=np.ones((3, 2), dtype=bool) + ) + code = self._prior().to_code( + single_channel=True, roi_calibration_period=np.ones((3, 1), dtype=bool) + ) + rebuilt = self._exec_prelude(code, "roi_prior = ") + assert rebuilt.shape == (3, 1) + + def test_array_mask_all_false_rejected(self): + with pytest.raises(ValueError, match="all False"): + self._prior().to_code( + single_channel=True, roi_calibration_period=np.zeros((3, 1), dtype=bool) + ) + + def test_array_mask_all_false_column_rejected(self): + # Meridian aggregates each channel's calibration spend through its mask + # column, so ANY all-False column (here 'tv') is rejected even when + # other columns carry Trues - Google's convention gives non-experiment + # channels ALL periods, never none. + mask = np.zeros((3, 2), dtype=bool) + mask[:, 0] = True # 'search' all periods; 'tv' entirely False + with pytest.raises(ValueError, match="entirely False"): + self._prior().to_code( + channel="tv", media_channels=["search", "tv"], roi_calibration_period=mask + ) + # The mirror direction: every column has a True -> accepted. + mask[1, 1] = True + code = self._prior().to_code( + channel="tv", media_channels=["search", "tv"], roi_calibration_period=mask + ) + assert "roi_calibration_period=roi_calibration_period" in code + + def test_mroi_priors_cannot_be_time_scoped(self): + # Meridian 1.7.0's ModelSpec rejects roi_calibration_period unless the + # media prior type is 'roi' - both the ndarray and expression routes + # must fail closed for mroi_m; full_model_window stays valid. + mroi = to_meridian_roi_prior( + incremental_outcome=9_600.0, + incremental_outcome_se=2_400.0, + spend=20_000.0, + parameter="mroi_m", + ) + with pytest.raises(ValueError, match="unless the media prior type"): + mroi.to_code( + single_channel=True, + roi_calibration_period=np.ones((3, 1), dtype=bool), + ) + with pytest.raises(ValueError, match="unless the media prior type"): + mroi.to_code(single_channel=True, roi_calibration_period="my_mask") + code = mroi.to_code(single_channel=True, full_model_window=True) + assert "roi_calibration_period=None" in code + # The no-time-scope error is parameter-aware: for mroi_m it recommends + # ONLY the route that Meridian accepts, never the mask routes the next + # validation would reject. + with pytest.raises(ValueError, match="exactly one route"): + mroi.to_code(single_channel=True) + + def test_array_mask_bad_shape_and_values(self): + with pytest.raises(ValueError, match="2-D"): + self._prior().to_code( + single_channel=True, roi_calibration_period=np.ones(3, dtype=bool) + ) + with pytest.raises(ValueError, match="non-empty"): + self._prior().to_code( + single_channel=True, roi_calibration_period=np.ones((0, 2), dtype=bool) + ) + bad = np.zeros((3, 1)) + bad[0, 0] = 0.5 + with pytest.raises(ValueError, match="only 0/1"): + self._prior().to_code(single_channel=True, roi_calibration_period=bad) + with pytest.raises(ValueError, match="only 0/1"): + self._prior().to_code( + single_channel=True, + roi_calibration_period=np.array([["a"]], dtype=object), + ) + + def test_array_mask_float01_google_parity(self): + # Google's configure-model example builds the mask with float np.zeros. + mask = np.zeros((3, 1)) + mask[1, 0] = 1.0 + code = self._prior().to_code(single_channel=True, roi_calibration_period=mask) + rebuilt = self._exec_prelude(code, "roi_prior = ") + assert rebuilt.dtype == np.bool_ + assert np.array_equal(rebuilt, mask.astype(bool)) + + def test_array_mask_masked_array_rejected(self): + masked = np.ma.masked_array(np.ones((2, 1)), mask=[[True], [False]]) + with pytest.raises(TypeError, match="masked arrays are not accepted"): + self._prior().to_code(single_channel=True, roi_calibration_period=masked) + + def test_non_str_non_array_rejected(self): + with pytest.raises(TypeError, match="expression string or a boolean numpy array"): + self._prior().to_code(single_channel=True, roi_calibration_period=[[True, False]]) + + def test_time_scope_error_mentions_builder(self): + with pytest.raises(ValueError, match="meridian_calibration_mask"): + self._prior().to_code(single_channel=True) + + def test_empty_prelude_routes_stay_clean(self): + # Both empty-{mask_prelude} routes emit no numpy artifacts. + full_window = self._prior().to_code(single_channel=True, full_model_window=True) + str_expr = self._prior().to_code(single_channel=True, roi_calibration_period="my_mask") + for code in (full_window, str_expr): + assert "import numpy" not in code + assert "np.zeros(" not in code class TestRealisticWorkflow: @@ -676,6 +1232,18 @@ def test_missing_scale_names_both_routes(self): with pytest.raises(ValueError, match="scale is required with aggregation_result"): to_pymc_marketing_lift_test(channel="tv", x=1.0, delta_x=1.0, aggregation_result=agg) + def test_boolean_scale_rejected(self): + # float(True) == 1.0 would silently scale by one - a plausible typo + # for scale="auto" - so booleans fail closed, scalar and per-row. + agg = _make_agg() + for bad in (True, np.bool_(True), [True]): + with pytest.raises(ValueError, match="got a boolean"): + to_pymc_marketing_lift_test( + channel="tv", x=1.0, delta_x=1.0, aggregation_result=agg, scale=bad + ) + with pytest.raises(ValueError, match="got a boolean"): + to_meridian_roi_prior(aggregation_result=agg, scale=True, spend=10.0) + def test_non_auto_string_scale_rejected(self): agg = _make_agg() with pytest.raises(ValueError, match="or the string 'auto'"): @@ -1123,4 +1691,5 @@ def test_public_exports(): assert diff_diff.to_pymc_marketing_lift_test is to_pymc_marketing_lift_test assert diff_diff.to_meridian_roi_prior is to_meridian_roi_prior + assert diff_diff.meridian_calibration_mask is meridian_calibration_mask assert diff_diff.MeridianROIPrior is MeridianROIPrior