Skip to content

Improve SensitivityMSM robustness and test coverage#933

Merged
jeongyoonlee merged 3 commits into
uber:masterfrom
aman-coder03:feature/msm-followups
Jul 8, 2026
Merged

Improve SensitivityMSM robustness and test coverage#933
jeongyoonlee merged 3 commits into
uber:masterfrom
aman-coder03:feature/msm-followups

Conversation

@aman-coder03

@aman-coder03 aman-coder03 commented Jul 7, 2026

Copy link
Copy Markdown
Contributor

Proposed changes

follow ups on the review comments from #925
This addresses several non-blocking follow-up suggestions from the review of the SensitivityMSM implementation.
changes focus on improving robustness, numerical stability, and test coverage without changing the public API or the underlying MSM estimation logic

Specifically,

  • replaces the unsupported-learner blocklist with an explicit allowlist of supported learner families (S-, T-, and DR-learners), so unknown learner types fail safely instead of being treated as supported
  • expands test_SensitivityMSM to cover S-, T-, and DR-learners instead of only the T-learner
  • makes the MSM tests deterministic by seeding the synthetic data generation using RANDOM_SEED
  • clips propensity bounds before division in the MSM bound computation to avoid numerical issues when propensity estimates are extremely close to 0 or 1

The documentation wording regarding the theoretical sharpness of the bound and the minor double-fit optimization are intentionally left for future work, as discussed in the previous review

Types of changes

What types of changes does your code introduce to CausalML?
Put an x in the boxes that apply

  • Bugfix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Documentation Update (if none of the other choices apply)

Checklist

Put an x in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your code.

  • I have read the CONTRIBUTING doc
  • I have signed the CLA
  • Lint and unit tests pass locally with my changes
  • I have added tests that prove my fix is effective or that my feature works
  • I have added necessary documentation (if appropriate)
  • Any dependent changes have been merged and published in downstream modules

Further comments

If this is a relatively large or complex change, kick off the discussion by explaining why you chose the solution you did and what alternatives you considered, etc. This PR template is adopted from appium.

@jeongyoonlee jeongyoonlee left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The allowlist matches type(learner).__name__ against base-class names, so concrete S/T/DR subclasses are rejected: XGBTRegressor, MLPTRegressor, LRSRegressor, and XGBDRRegressor all hit the NotImplementedError here despite being supported. Use isinstance(learner, (BaseSLearner, BaseTLearner, BaseDRLearner)) instead of name membership — it covers subclasses and still rejects X/R and unknown types.

The test_SensitivityMSM parametrization uses only the base classes (all in the allowlist), so this isn't caught — please add a concrete subclass (e.g. XGBTRegressor) to the parametrization.

@jeongyoonlee jeongyoonlee left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@jeongyoonlee jeongyoonlee added the enhancement New feature or request label Jul 8, 2026
@jeongyoonlee jeongyoonlee merged commit ba5b640 into uber:master Jul 8, 2026
7 checks passed
@aman-coder03 aman-coder03 deleted the feature/msm-followups branch July 9, 2026 08:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants