Skip to content

fix: empty variations or short meta in a rule crashed evaluation with IndexError - #137

Merged
madhuchavva merged 1 commit into
mainfrom
fix/empty-variations-index-error
Sep 8, 2026
Merged

madhuchavva merged 1 commit into
mainfrom
fix/empty-variations-index-error

Conversation

@madhuchavva

Copy link
Copy Markdown
Contributor

Summary

_getExperimentResult raised an uncaught IndexError on two malformed payload shapes; both now degrade the way the evaluation contract expects instead of crashing feature evaluation.

Root cause

The out-of-range clamp sets variationId = 0 and then indexes unconditionally:

  • variations: [] (or contextualVariations: [] on a bandit rule) → experiment.variations[0] → IndexError at core.py:1443. Pre-existing, not a bandit regression — reproduced on v3.0.0 with a plain variations: [] rule; the CB shape is just a second door into it.
  • meta shorter than variations (e.g. 2 variations, 1 meta entry, user hashes into variation 1) → experiment.meta[1] → IndexError at core.py:1422.

Behavioral change

  • Empty variations: the clamped variation's value reads as None (the JS SDK reads undefined there), the result stays inExperiment=False, and the rule falls through to the feature's defaultValue — identical user-visible behavior to JS.
  • Short meta: the missing entry reads as absent (result key falls back to the variation id). The JS SDK crashes here too (.key off undefined), so this half is a deliberate Python-stricter divergence on invalid payloads only.

Follow-ups

  • Propose an empty-variations case for the shared cases.json corpus (JS repo) so all SDKs pin the fall-through behavior.
  • JS fix for the short-meta TypeError (same latent crash in getExperimentResult).

Test plan

  • New tests fail without the fix, pass with it: pytest tests/test_growthbook.py -k "empty_variations or empty_contextual or meta_shorter"
  • Full suite: pytest tests/ -q — 972 passed (2 known local-env test_typing[*-mypy] deselects)
  • mypy at baseline (1 pre-existing urllib3 error), pyright 0 errors, flake8 --select=E9,F63,F7,F82 clean

@greptile-apps

greptile-apps Bot commented Sep 8, 2026

Copy link
Copy Markdown

RetriggerView in GreptileConfidence Score: 5/5

The PR appears safe to merge; the malformed payloads now degrade predictably without affecting valid experiment evaluation.

@madhuchavva
madhuchavva merged commit 9af3be5 into main Sep 8, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant