Repository navigation
Conversation
|
|
❌ ErrorsYour PR has failed checks. Please review the issues below and take necessary action before merging. 🚦 13 Pipeline jobs failed
ℹ️ InfoNo other issues found (see more)❄️ No new flaky tests detected Useful? React with 👍 / 👎 This comment will be updated automatically if new data arrives.🔗 Commit SHA: 0cebbe7 | Docs | View more details | Give us feedback! |
0f79993 to
5a23492
Compare
There was a problem hiding this comment.
🟡 Changes recommended
Activation gating, scenario-aborting setup validation, and missing graph telemetry assertions remain unresolved.
3 open findings
What changed in this PR
Adds dependent-flag evaluation coverage to the FFE system-test harness, with Node activation intended for stacked PR #8050.
Changes:
- Adds dependency graphs and seven evaluation cases.
- Introduces end-to-end root-result checks and fixture loading.
- Adds optional detailed responses to the Node
/ffeendpoint.
| File | Description |
|---|---|
| utils/build/docker/nodejs/express/app.js | Supports detailed flag evaluation responses. |
| tests/parametric/test_ffe/test-case-dependent-flags.json | Defines seven dependency evaluation cases. |
| tests/parametric/test_ffe/test_dynamic_evaluation.py | Excludes dependent cases from generic discovery. |
| tests/parametric/test_ffe/flags-v1.json | Adds dependency graphs and a depth limit. |
| tests/ffe/utils/canonical.py | Loads checked-in fixtures. |
| tests/ffe/test_dependent_flags.py | Exercises dependent-flag root results. |
| tests/ffe/README.md | Documents branch-testing commands. |
🧠 Review effort: Balanced
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
5a23492 to
5fcbf37
Compare
5fcbf37 to
7b19f61
Compare
36c286e to
e36f568
Compare
Add dependent-flag UFC fixtures and validate successful and failed root evaluation results through the FFE system-test harness. Environment: Datadog workspace Co-Authored-By: Codex GPT-5 <noreply@localhost>
e36f568 to
deec34b
Compare
Resolve fixture discovery and evaluation-detail conflicts with current system-tests main. Environment: Datadog workspace Co-Authored-By: Codex GPT-5 <noreply@localhost>
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
Limit response assertions to API result fields, preserve successful error-code absence checks, and refresh dependent fixtures. Environment: Datadog workspace Co-Authored-By: Codex GPT-5 <noreply@localhost>
Remove dependency-specific case partitioning and manifest exemptions so every imported evaluation case uses the shared dynamic-evaluation path. Environment: Datadog workspace Co-Authored-By: Codex GPT-5 <noreply@localhost>
leoromanovsky
left a comment
There was a problem hiding this comment.
approving if you want to move forward here while the shared files mode is in progress
FFE reviewer selectionReviewers: @hhan2024, @sameerank, @leoromanovsky Why: Dependent Flags — The PR adds shared cross-SDK parametric system tests for evaluating prerequisite (dependent) flags. Reviewers come from the matched pod and the required specialist pods. 🤖 Automated selection. Selection detailsInitiative: project / dependent-flags Why this match: The PR adds shared cross-SDK parametric system tests for evaluating prerequisite (dependent) flags. The new dependent-flags evaluation cases cover direct and transitive prerequisite matches by variant key, ONE_OF and NOT_ONE_OF prerequisite conditions, short-circuiting, memoization, missing prerequisites, depth limits and cycles. The canonical UFC config adds the dependent flag graphs and maxDependencyDepth. The generic dynamic-evaluation test runs these cases. The rest of the canonical-snapshot migration supports this coverage. Why these owners: Eligible reviewers have already approved the current head; no additional reviewers are needed for the responsibilities those approvals cover.
Additional review: openfeature-architecture — The PR changes the shared evaluation-result contract the parametric harness expects from every SDK. The Ruby endpoint switches from fetch_*_value to fetch_*_details and now returns the OpenFeature reason, errorCode and flagMetadata. The Node endpoint now returns errorCode. The shared runner now checks reason and errorCode on every case, not just value, and treats an unexpected errorCode as a failure. Coverage: @leoromanovsky. Evidence: utils/build/docker/ruby/parametric/server.rb, utils/build/docker/nodejs/parametric/server.js, tests/parametric/test_ffe/test_dynamic_evaluation.py The PR author was excluded. Existing reviewers and teams were preserved. Debug or fix a problemWorkflow run · Bits Code session
Evidence and run details✅ Complete — reviewer coverage verified. The workflow checked pending requests or submitted reviews on the evaluated head. Generated asynchronously by the FFE reviewer-routing workflow. Changed paths: Evaluated head: Registry revision: Pinned skill source: Instructions: workflow-injected snapshot (no plugin loaded) Instruction snapshot SHA-256: |
|
Superseded by #8139 (canonical snapshot import), #8190 (separately activated dependent-flags harness), and #8260 (canonical evaluation-suites import and shared runner). The old branch duplicates fixture/import history and its pre-unification CI failures are no longer actionable. Per-SDK activation and telemetry work will be restacked on #8260. |


Motivation
Validate variant-key dependent flag evaluation results through the existing parametric FFE dynamic-evaluation harness.
Linear: FFL-1694 — [SDK][JS] Support Dependent Flags
Changes
ffe-system-test-datasnapshotTest_Feature_Flag_Dynamic_Evaluation.test_ffe_flag_evaluationvalue,reason, and optionallyerrorCode)This PR deliberately adds no dependent-flags-specific loader, test method, or manifest exemption. SDKs without dependent-flags support will fail the imported cases until case-level skip support is added separately.
Validation
./format.sh --checkTEST_LIBRARY=nodejs venv/bin/python -m pytest -S PARAMETRIC --collect-only -q tests/parametric/test_ffe/test_dynamic_evaluation.py(46 tests; dependent fixtures collected by the generic test method)Stack
FFE reviewer selection
Reviewers: @leoromanovsky, @sameerank
Why: Global fallback — The PR adds parametric test coverage for evaluating dependent (prerequisite) flags. Global reviewers were selected because no project or SME pod matched.
🤖 Automated selection.
Selection details
Initiative: global / global
Why this match: The PR adds parametric test coverage for evaluating dependent (prerequisite) flags. It imports a dependent-flags evaluation-case fixture and an updated UFC config from the canonical ffe-system-test-data snapshot, and makes the shared runner assert value, reason and errorCode for each case. That work is not Bandits V0, which explicitly assigns prerequisites to a separate dependent-flags initiative. It is not Flag evaluation on EVP, because there are no EVP telemetry or consent changes. It is not Guided and observable onboarding, because the purpose is not onboarding diagnostics. The supplied catalog has no dependent-flags project and no fallback entry, so this goes to global ownership.
The PR author was excluded. Existing reviewers and teams were preserved.
Debug or fix a problem
Workflow run · Bits Code session
dry_run=false. Unchanged completed runs are skipped. If the receipt saysneeds_reconciliationor a write outcome is uncertain, ask the workflow owner to reconcile GitHub and the receipt before retrying; do not delete the lock blindly.Evidence and run details
✅ Complete — reviewer coverage verified. The workflow checked pending requests or submitted reviews on the evaluated head.
Generated asynchronously by the FFE reviewer-routing workflow.
Changed paths:
tests/parametric/test_ffe/ffe-system-test-data/evaluation-cases/test-case-dependent-flags.json,tests/parametric/test_ffe/ffe-system-test-data/ufc-config.json,tests/parametric/test_ffe/test_dynamic_evaluation.py,tests/parametric/test_ffe/ffe-system-test-data/SOURCE.mdEvaluated head:
0cebbe7939e2c5250128ba31dd4eb4c1131598b0Registry revision:
revision:1946aa2a-530e-487a-84a7-83aff44a1089Pinned skill source:
d214cfc65ae154b97d39af5cf32caf48aaec9bf0Instructions: workflow-injected snapshot (no plugin loaded)
Instruction snapshot SHA-256:
f23b2a88c13c0da90e2dd58246a14f257af91e2195c555039bd555b52e6f3516