Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
35 changes: 35 additions & 0 deletions docs/understand/scenarios/parametric.md
Original file line number Diff line number Diff line change
Expand Up @@ -308,3 +308,38 @@ def test_stuff(test_library):
[2]: https://docs.pytest.org/en/6.2.x/usage.html#specifying-tests-selecting-tests
[3]: https://github.com/roydahan/xunit-viewer
[4]: ../../utils/build/docker/cpp/parametric/CMakeLists.txt

### Dependent flags over Remote Configuration

`tests/parametric/test_ffe/test_dependent_flags_rc.py` checks the wire contract for
server SDKs using Agent RC. The supported-build test requires bit 50 on every
observed request subscribing to `FFE_FLAGS`, before any dependent configuration
is delivered and after configuration updates. It verifies prerequisite variant
keys rather than flag values, using configuration delivered exclusively by RC.
The test agent models capability-gated delivery; this does not validate production
backend targeting.

Use the [custom tracer build instructions](../../execute/binaries.md) to select
the SDK and native library under test. With an unreleased supported build, force
the positive class so a missing-feature manifest cannot turn a regression into
an expected failure:

```sh
TEST_LIBRARY=python ./run.sh PARAMETRIC \
tests/parametric/test_ffe/test_dependent_flags_rc.py::Test_FFE_Dependent_Flags_RC \
-F tests/parametric/test_ffe/test_dependent_flags_rc.py::Test_FFE_Dependent_Flags_RC
```

For a build known to lack dependent-flag support, select and force
`Test_FFE_Without_Dependent_Flags_RC` instead. It must receive and evaluate an
ordinary RC flag while advertising no bit 50. No RC traffic is a failure, not
proof that an unsupported build is safe. Python 4.16.0 is the explicit legacy
baseline in the manifest; other legacy/native combinations can use `-F`.

Enable the positive class in each SDK's manifest when its dependent-flag support
ships, alongside the evaluation suite. All SDKs retain missing-feature declarations
until a release containing dependent-flag support is confirmed. Python's
implementation is tracked in [dd-trace-py #21346](https://github.com/DataDog/dd-trace-py/pull/21346);
use the forced positive class to test that build before enabling a release threshold.
For Ruby, verify the loaded native package as well as the SDK version. Do not infer
expected support from whether the SDK happens to advertise bit 50.
2 changes: 2 additions & 0 deletions manifests/cpp.yml
Original file line number Diff line number Diff line change
Expand Up @@ -187,6 +187,8 @@ manifest:
tests/parametric/test_extract_behavior.py::Test_ExtractBehavior_Restart::test_baggage: missing_feature (Automatic baggage propagation is not implemented)
tests/parametric/test_extract_behavior.py::Test_ExtractBehavior_Restart_With_Extract_First::test_baggage: missing_feature (Automatic baggage propagation is not implemented)
tests/parametric/test_ffe/test_configuration_sources.py: irrelevant (cpp is not in scope for v1 agentless server SDK configuration-source coverage)
tests/parametric/test_ffe/test_dependent_flags_rc.py::Test_FFE_Dependent_Flags_RC: missing_feature (Dependent flag support must be enabled with the SDK release)
tests/parametric/test_ffe/test_dependent_flags_rc.py::Test_FFE_Without_Dependent_Flags_RC: irrelevant (Requires a known legacy build without dependent flag support)
tests/parametric/test_ffe/test_dynamic_evaluation.py::Test_Feature_Flag_Dynamic_Evaluation: missing_feature
tests/parametric/test_ffe/test_dynamic_evaluation.py::Test_Feature_Flag_Split_Features: incomplete_test_app (Split-feature hook capture is currently implemented only in the Java parametric client)
tests/parametric/test_ffe/test_exposure_hooks.py: incomplete_test_app (Exposure hook observations are not implemented)
Expand Down
2 changes: 2 additions & 0 deletions manifests/dotnet.yml
Original file line number Diff line number Diff line change
Expand Up @@ -1218,6 +1218,8 @@ manifest:
: # TODO: a lower version might be supported
- declaration: bug (FFL-3325)
component_version: <3.55.0
tests/parametric/test_ffe/test_dependent_flags_rc.py::Test_FFE_Dependent_Flags_RC: missing_feature (Dependent flag support must be enabled with the SDK release)
tests/parametric/test_ffe/test_dependent_flags_rc.py::Test_FFE_Without_Dependent_Flags_RC: irrelevant (Requires a known legacy build without dependent flag support)
tests/parametric/test_ffe/test_dynamic_evaluation.py::Test_Feature_Flag_Dynamic_Evaluation: '>=3.36.0' # Modified by easy win activation script
tests/parametric/test_ffe/test_dynamic_evaluation.py::Test_Feature_Flag_Dynamic_Evaluation::test_ffe_dependent_flags: missing_feature (FFL-1694)
tests/parametric/test_ffe/test_dynamic_evaluation.py::Test_Feature_Flag_Dynamic_Evaluation::test_ffe_flag_evaluation: missing_feature # Created by easy win activation script
Expand Down
2 changes: 2 additions & 0 deletions manifests/golang.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2185,6 +2185,8 @@ manifest:
tests/parametric/test_extract_behavior.py::Test_ExtractBehavior_Restart: incomplete_test_app (The parametric test app does not emit restart span links or preserve baggage)
tests/parametric/test_extract_behavior.py::Test_ExtractBehavior_Restart_With_Extract_First: missing_feature (DD_TRACE_PROPAGATION_EXTRACT_FIRST=true is unsupported; dd-trace-go panics when extraction fails)
tests/parametric/test_ffe/test_configuration_sources.py: v2.11.0-dev # Easy win for all weblogs and version 2.10.1
tests/parametric/test_ffe/test_dependent_flags_rc.py::Test_FFE_Dependent_Flags_RC: missing_feature (Dependent flag support must be enabled with the SDK release)
tests/parametric/test_ffe/test_dependent_flags_rc.py::Test_FFE_Without_Dependent_Flags_RC: irrelevant (Requires a known legacy build without dependent flag support)
tests/parametric/test_ffe/test_dynamic_evaluation.py::Test_Feature_Flag_Dynamic_Evaluation: v2.5.0-dev
tests/parametric/test_ffe/test_dynamic_evaluation.py::Test_Feature_Flag_Dynamic_Evaluation::test_ffe_dependent_flags: missing_feature (FFL-1694)
tests/parametric/test_ffe/test_dynamic_evaluation.py::Test_Feature_Flag_Split_Features: incomplete_test_app (Split-feature hook capture is currently implemented only in the Java parametric client)
Expand Down
2 changes: 2 additions & 0 deletions manifests/java.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4266,6 +4266,8 @@ manifest:
? tests/parametric/test_ffe/test_configuration_sources.py::Test_Feature_Flag_Configuration_Source_Selection::test_stable_true_preserves_default_agentless
: v1.65.0
tests/parametric/test_ffe/test_configuration_sources.py::Test_Feature_Flag_Configuration_Source_Warm_State_Preservation: v1.65.0
tests/parametric/test_ffe/test_dependent_flags_rc.py::Test_FFE_Dependent_Flags_RC: missing_feature (Dependent flag support must be enabled with the SDK release)
tests/parametric/test_ffe/test_dependent_flags_rc.py::Test_FFE_Without_Dependent_Flags_RC: irrelevant (Requires a known legacy build without dependent flag support)
tests/parametric/test_ffe/test_dynamic_evaluation.py::Test_Feature_Flag_Dynamic_Evaluation: v1.56.0
tests/parametric/test_ffe/test_dynamic_evaluation.py::Test_Feature_Flag_Dynamic_Evaluation::test_ffe_dependent_flags: missing_feature (FFL-1694)
tests/parametric/test_ffe/test_dynamic_evaluation.py::Test_Feature_Flag_Split_Features: v1.69.0-SNAPSHOT
Expand Down
2 changes: 2 additions & 0 deletions manifests/nodejs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2700,6 +2700,8 @@ manifest:
tests/parametric/test_extract_behavior.py::Test_ExtractBehavior_Restart_With_Extract_First: *ref_5_47_0
tests/parametric/test_extract_behavior.py::Test_ExtractBehavior_Restart_With_Extract_First::test_baggage: *ref_5_56_0
tests/parametric/test_ffe/test_configuration_sources.py: *ref_6_8_0
tests/parametric/test_ffe/test_dependent_flags_rc.py::Test_FFE_Dependent_Flags_RC: missing_feature (Dependent flag support must be enabled with the SDK release)
tests/parametric/test_ffe/test_dependent_flags_rc.py::Test_FFE_Without_Dependent_Flags_RC: irrelevant (Requires a known legacy build without dependent flag support)
tests/parametric/test_ffe/test_dynamic_evaluation.py::Test_Feature_Flag_Dynamic_Evaluation: *ref_5_75_0
tests/parametric/test_ffe/test_dynamic_evaluation.py::Test_Feature_Flag_Dynamic_Evaluation::test_ffe_dependent_flags: missing_feature (FFL-1694)
tests/parametric/test_ffe/test_dynamic_evaluation.py::Test_Feature_Flag_Split_Features: incomplete_test_app (Split-feature hook capture is currently implemented only in the Java parametric client)
Expand Down
2 changes: 2 additions & 0 deletions manifests/php.yml
Original file line number Diff line number Diff line change
Expand Up @@ -1649,6 +1649,8 @@ manifest:
? tests/parametric/test_ffe/test_configuration_sources.py::Test_Feature_Flag_Configuration_Source_Selection::test_stable_true_preserves_default_agentless
: missing_feature (FFL-2705 tracks PHP agentless configuration-source implementation; FFL-2731 tracks system-tests configuration-source contract)
tests/parametric/test_ffe/test_configuration_sources.py::Test_Feature_Flag_Configuration_Source_Warm_State_Preservation: missing_feature (FFL-2705 tracks PHP agentless configuration-source implementation; FFL-2731 tracks system-tests configuration-source contract)
tests/parametric/test_ffe/test_dependent_flags_rc.py::Test_FFE_Dependent_Flags_RC: missing_feature (Dependent flag support must be enabled with the SDK release)
tests/parametric/test_ffe/test_dependent_flags_rc.py::Test_FFE_Without_Dependent_Flags_RC: irrelevant (Requires a known legacy build without dependent flag support)
tests/parametric/test_ffe/test_dynamic_evaluation.py::Test_Feature_Flag_Dynamic_Evaluation: v1.21.0-dev
tests/parametric/test_ffe/test_dynamic_evaluation.py::Test_Feature_Flag_Dynamic_Evaluation::test_ffe_dependent_flags: missing_feature (FFL-1694)
tests/parametric/test_ffe/test_dynamic_evaluation.py::Test_Feature_Flag_Split_Features: incomplete_test_app (Split-feature hook capture is currently implemented only in the Java parametric client)
Expand Down
4 changes: 4 additions & 0 deletions manifests/python.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3153,6 +3153,10 @@ manifest:
tests/parametric/test_extract_behavior.py::Test_ExtractBehavior_Restart: v2.20.0
tests/parametric/test_extract_behavior.py::Test_ExtractBehavior_Restart_With_Extract_First: v2.20.0
tests/parametric/test_ffe/test_configuration_sources.py: v4.14.0-dev
tests/parametric/test_ffe/test_dependent_flags_rc.py::Test_FFE_Dependent_Flags_RC: missing_feature (Dependent flag support must be enabled with the SDK release; dd-trace-py PR 21346)
tests/parametric/test_ffe/test_dependent_flags_rc.py::Test_FFE_Without_Dependent_Flags_RC:
- declaration: irrelevant (Legacy coverage is pinned to Python 4.16.0)
component_version: '<4.16.0 || >4.16.0'
tests/parametric/test_ffe/test_dynamic_evaluation.py::Test_Feature_Flag_Dynamic_Evaluation: v4.0.0
tests/parametric/test_ffe/test_dynamic_evaluation.py::Test_Feature_Flag_Dynamic_Evaluation::test_ffe_dependent_flags: missing_feature (FFL-1694)
tests/parametric/test_ffe/test_dynamic_evaluation.py::Test_Feature_Flag_Split_Features: incomplete_test_app (Split-feature hook capture is currently implemented only in the Java parametric client)
Expand Down
2 changes: 2 additions & 0 deletions manifests/ruby.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2711,6 +2711,8 @@ manifest:
tests/parametric/test_extract_behavior.py::Test_ExtractBehavior_Restart: incomplete_test_app (The parametric test app does not emit restart span links or preserve baggage)
tests/parametric/test_extract_behavior.py::Test_ExtractBehavior_Restart_With_Extract_First: incomplete_test_app (The parametric test app does not emit restart span links or preserve baggage)
tests/parametric/test_ffe/test_configuration_sources.py: v2.46.0-dev
tests/parametric/test_ffe/test_dependent_flags_rc.py::Test_FFE_Dependent_Flags_RC: missing_feature (Dependent flag support must be enabled with the SDK release)
tests/parametric/test_ffe/test_dependent_flags_rc.py::Test_FFE_Without_Dependent_Flags_RC: irrelevant (Requires a known legacy build without dependent flag support)
tests/parametric/test_ffe/test_dynamic_evaluation.py::Test_Feature_Flag_Dynamic_Evaluation: v2.23.0
tests/parametric/test_ffe/test_dynamic_evaluation.py::Test_Feature_Flag_Dynamic_Evaluation::test_ffe_dependent_flags: missing_feature (FFL-1694)
tests/parametric/test_ffe/test_dynamic_evaluation.py::Test_Feature_Flag_Split_Features: incomplete_test_app (Split-feature hook capture is currently implemented only in the Java parametric client)
Expand Down
2 changes: 2 additions & 0 deletions manifests/rust.yml
Original file line number Diff line number Diff line change
Expand Up @@ -299,6 +299,8 @@ manifest:
tests/parametric/test_dynamic_configuration.py::Test_DynamicConfigSdkConfiguration::test_sdk_config_profiling_enabled: missing_feature (SDK_CONFIGURATION capability not yet implemented)
tests/parametric/test_extract_behavior.py: '>=0.4.1-dev'
tests/parametric/test_ffe/test_configuration_sources.py: irrelevant (rust is not in scope for v1 agentless server SDK configuration-source coverage)
tests/parametric/test_ffe/test_dependent_flags_rc.py::Test_FFE_Dependent_Flags_RC: missing_feature (Dependent flag support must be enabled with the SDK release)
tests/parametric/test_ffe/test_dependent_flags_rc.py::Test_FFE_Without_Dependent_Flags_RC: irrelevant (Requires a known legacy build without dependent flag support)
tests/parametric/test_ffe/test_dynamic_evaluation.py::Test_Feature_Flag_Dynamic_Evaluation: '>=0.5.3-dev+fd1307f' # TODO: a lower version might be supported
tests/parametric/test_ffe/test_dynamic_evaluation.py::Test_Feature_Flag_Dynamic_Evaluation::test_ffe_dependent_flags: missing_feature (FFL-1694)
tests/parametric/test_ffe/test_dynamic_evaluation.py::Test_Feature_Flag_Dynamic_Evaluation::test_ffe_flag_evaluation: missing_feature
Expand Down
123 changes: 123 additions & 0 deletions tests/parametric/test_ffe/test_dependent_flags_rc.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,123 @@
"""Dependent-flag capability and evaluation contract on the Agent RC transport.

Supported and legacy builds have separate manifest-controlled expectations. Never
infer support from the advertised bit: doing so would hide a missing-bit regression.
"""

import time
from typing import Any

from tests.parametric.conftest import APMLibrary
from tests.parametric.test_ffe.utils import (
assert_dependent_flag_capability,
deliver,
ffe_requests,
observe_polls,
start_and_deliver,
)
from utils import features, pytest, scenarios
from utils.docker_fixtures import TestAgentAPI

ENV = {
"DD_EXPERIMENTAL_FLAGGING_PROVIDER_ENABLED": "true",
"DD_FEATURE_FLAGS_ENABLED": "true",
"DD_FEATURE_FLAGS_CONFIGURATION_SOURCE": "remote_config",
"DD_REMOTE_CONFIG_POLL_INTERVAL_SECONDS": "0.2",
}


def _config(*, parent_variant: str, dependent: bool) -> dict[str, Any]:
parent = {
"key": "parent",
"enabled": True,
"variationType": "STRING",
# The selected variant key deliberately differs from its value.
"variations": {
"approved": {"key": "approved", "value": "not-the-variant-key"},
"rejected": {"key": "rejected", "value": "approved"},
},
"allocations": [
{"key": "parent-default", "splits": [{"variationKey": parent_variant, "shards": []}], "doLog": False}
],
}
flags = {"parent": parent}
if dependent:
flags["child"] = {
"key": "child",
"enabled": True,
"variationType": "STRING",
"variations": {key: {"key": key, "value": key} for key in ("matched", "unmatched")},
"allocations": [
{
"key": "prerequisite",
"rules": [
{
"conditions": [
{"flagEvaluation": {"key": "parent"}, "operator": "ONE_OF", "value": ["approved"]}
]
}
],
"splits": [{"variationKey": "matched", "shards": []}],
"doLog": False,
},
{"key": "fallback", "splits": [{"variationKey": "unmatched", "shards": []}], "doLog": False},
],
}
return {"createdAt": "2026-11-03T00:00:00Z", "format": "SERVER", "environment": {"name": "test"}, "flags": flags}


def _assert_value(test_library: APMLibrary, flag: str, value: str) -> None:
deadline = time.monotonic() + 5
while True:
result = test_library.ffe_evaluate(
flag=flag,
variation_type="STRING",
default_value="caller-default",
targeting_key="subject",
)
if result.get("value") == value and result.get("errorCode") in (None, ""):
return
assert time.monotonic() < deadline, result
time.sleep(0.1)


@scenarios.parametric
@features.feature_flags_dynamic_evaluation
class Test_FFE_Dependent_Flags_RC:
@pytest.mark.parametrize("library_env", [ENV])
def test_capability_and_gated_delivery(self, test_agent: TestAgentAPI, test_library: APMLibrary) -> None:
"""Advertise before delivery, evaluate variant keys, and retain the bit after updates."""
start_and_deliver(
test_agent,
test_library,
config_id="dependent-flags",
payload=_config(parent_variant="approved", dependent=True),
supported=True,
)
_assert_value(test_library, "child", "matched")
previous_count = len(ffe_requests(test_agent))
observe_polls(test_agent, supported=True, minimum=previous_count + 2)

assert_dependent_flag_capability(ffe_requests(test_agent), supported=True)
deliver(test_agent, config_id="dependent-flags", payload=_config(parent_variant="rejected", dependent=True))
_assert_value(test_library, "child", "unmatched")
previous_count = len(ffe_requests(test_agent))
observe_polls(test_agent, supported=True, minimum=previous_count + 2)


@scenarios.parametric
@features.feature_flags_dynamic_evaluation
class Test_FFE_Without_Dependent_Flags_RC:
@pytest.mark.parametrize("library_env", [ENV])
def test_legacy_build_does_not_advertise(self, test_agent: TestAgentAPI, test_library: APMLibrary) -> None:
"""A known legacy build receives ordinary flags and never advertises bit 50."""
start_and_deliver(
test_agent,
test_library,
config_id="ordinary-flags",
payload=_config(parent_variant="approved", dependent=False),
supported=False,
)
_assert_value(test_library, "parent", "not-the-variant-key")
previous_count = len(ffe_requests(test_agent))
observe_polls(test_agent, supported=False, minimum=previous_count + 2)
Loading
Loading