[vjfridge]

Motivation

Agent-backed flagevaluation tests remain disabled for SDKs that now implement the contract. This activates validated cases after the shared assertion and fixture repairs in #8093 and SDK identity coverage in #8153.

Changes

• Enable Java spring-boot 1.68.0+ and Ruby rails72 2.43.0+ for the Agent-backed evaluation tests and isolated overload-accounting scenario.
• Enable the remaining Python flask-poc cases under the existing 4.16.0 development gate. Its SDK identity case retains the released 4.16.0 gate.
• Retain other weblog exclusions and existing Agentless, Go, and Node gates. Shared assertions are unchanged.

Tracks FFL-3509. This draft is a partial activation; the remaining SDK/package and weblog work is described below.

The 12,000-evaluation stress batch has a 10-second request timeout. Ruby development CI completed it successfully in 8.7 seconds, after the previous 5-second client timeout. Other requests retain their default timeout, and the accounting assertions remain unchanged.

Validation

Linux ARM64, Agent 7.84.2, Python 3.12 runner. System-tests base 1e8cc52021dad69d870069eb417c593fa124ef3a; activation commits df81e98c3f7f7bdbb431f62c449887e2371b51d0 and 7cc51707a966fb0ebcb28c87d01491d286b212a2.

Both scenarios select only tests/ffe/test_flag_eval_evp.py. All applicable tests were forced during the inventory; normal activation runs are recorded below. The 3 / 16 deselections are tests assigned to other scenarios, not skipped failures.

SDK / weblog	Evaluation scenario	Isolated stress	Manifest action
Java 1.68.0 / spring-boot	14 passed	1 passed	Enable
Ruby 2.43.0 / rails72	14 passed	1 passed	Enable
Python 4.16.0 / flask-poc	14 passed	1 passed	Enable remaining cases
Go 2.11.0 / net-http	14 passed	1 passed	Preserve existing gates
Node 6.20.0 / express5	14 passed	1 passed	Preserve existing gates
Node 5.131.0 / express5	14 passed	1 passed	Preserve existing gates
.NET 3.57.0 snapshot + locally rebuilt final #9511 provider / poc	14 passed	1 passed	Keep disabled pending provider package availability
PHP merged-support candidate reporting 1.26.0 / php-fpm-8.2	10 passed, 4 failed	1 passed	Keep disabled pending application-context fix

Normal runs without -F also passed for Ruby and Python: 14 evaluation tests and 1 isolated accounting test each, with no skips or expected failures. A direct check of the real manifest parser confirms the Java/Ruby/Python activation gates, older-version exclusions, other-weblog exclusions, and preserved Java/Ruby Agentless exclusions.

./format.sh passed. Existing manifest tests: 32 passed. No synthetic assertion tests were added.

Reproduce with the matching weblog image and Agent 7.84.2:

./run.sh FEATURE_FLAGGING_AND_EXPERIMENTATION --library <language> tests/ffe/test_flag_eval_evp.py
./run.sh FEATURE_FLAGGING_EVP_STRESS --library <language> tests/ffe/test_flag_eval_evp.py

Add -F tests/ffe/test_flag_eval_evp.py to investigate an excluded SDK. This local matrix does not claim coverage of other weblog variants or the entire FFE suite; normal PR CI remains required.

Remaining work

• PHP: the basic payload case and three privacy cases fail because application context lacks service: weblog. libdatadog#2716 is still open. The candidate image is not a released PHP 1.27.0 artifact; confirm the containing release and rerun before activation.
• .NET: #9511 merged October 28. The latest published Datadog.FeatureFlags.OpenFeature package is 2.3.2, published October 5, before that fix. The passing candidate uses the official tracer snapshot plus a rebuilt provider from the final PR head; it is not proof that the normal NuGet build contains the fix. Wait for and validate the containing provider package before enabling these cases.
• Other weblogs / gates: expand only with validation. The Node release lines passed forced runs, but their existing manifest gate remains unchanged in this activation slice.
• Older PR cleanup: Enable EVP flagevaluation system tests for Python #7183–Enable EVP flagevaluation system tests for PHP #7187 remain open until their scopes and evidence are fully accounted for by replacement changes. This draft does not close FFL-3509.

Artifact provenance

Java reports 1.68.0+36cefc772b; both the agent and provider are released 1.68.0 artifacts. The provider embedded in the rebuilt weblog was byte-compared against the downloaded release artifact (sha256:eb732977d85b13275f1ce7f7be44585cb3227719706bebe76fc94248e5b65c84). The weblog uses current main's typed-context fixture.

.NET tracer snapshot OCI digest: sha256:bdfcee2f338747c7bb28104cd80b4febafded2fb98473360b6fb905e51db18ac. Provider source: e02e2483e8f7b84735ce0f25362cf83206fea94c, the final #9511 head, built for net8.0. The snapshot reports 3.57.0. The candidate layers these artifacts over the previously validated typed-context weblog; no custom SDK binaries are committed here.

Exact weblog image IDs and per-run captures/results are retained in /tmp/ffl3509-validation on the validation workspace. The published package boundary for .NET was checked against the NuGet registration record for 2.3.2.

[github-actions]

CODEOWNERS have been resolved as:

manifests/java.yml                                                      @DataDog/system-tests-reviewers
manifests/python.yml                                                    @DataDog/system-tests-reviewers
manifests/ruby.yml                                                      @DataDog/system-tests-reviewers
tests/ffe/README.md                                                     @DataDog/feature-flagging-and-experimentation-sdk
tests/ffe/test_flag_eval_evp.py                                         @DataDog/feature-flagging-and-experimentation-sdk

[datadog-datadog-prod-us1-2]

 

🎉 All green!

🧪 All tests passed
❄️ No new flaky tests detected

_{This comment will be updated automatically if new data arrives.
🔗 Commit SHA: e199100 | Docs | View more details | Give us feedback!}