Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/mocked_backend/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ One single way to observe what is sent to the Datadog backend, whatever the setu
* **HTTPS on many hostnames**: `mocked-backend.invalid` and the intake hostnames derived from it (`trace.agent.`, `config.`, `instrumentation-telemetry-intake.`, `debugger-intake.`...) are network aliases of its container. Its certificate is generated when it starts, for all of them, signed by the mitmproxy CA (`utils/proxy/.mitmproxy`) the senders trust.
* **Remote config**: it serves the remote config backend by default (no config until a test sends one). Tests send configs with `remote_config.send_state`, which sets them on the mocked backend, then waits for the library to apply them. See [Remote config](../edit/remote-config.md).
* **Controlled responses**: `MockedBackendResponse` overrides the answer on a path, for the whole session (`MockedBackendContainer.internal_mocked_responses`) or at runtime (`client.set_mocked_responses`), like the proxy's mocked responses.
* **Feature flagging**: endpoints serving the agentless FFE configuration, with scripted responses (`client.ffe_set_response(s)`) and counters (`client.ffe_get_status`).
* **Feature flagging**: endpoints serving the agentless FFE configuration, with scripted responses (`client.ffe_set_response(s)`) and counters (`client.ffe_get_status`). The JSON configuration applies flag selection (`selected_flag_tags`, `selected_flag_key_patterns`) with the semantics of `ffe-system-test-data` `flag-selection/README.md`. `client.ffe_use_suite(name)` serves an `ffe-system-test-data` evaluation suite's UFC instead, matching tags from its `flag-tags.json`; `client.ffe_set_flag_selection_ignored_reason(...)` makes it ignore every valid, non-empty selection, as when production cannot load tags.
* **Control endpoints**: under `/system-tests/`, never recorded, used by `MockedBackendClient` from the test runner.

## Using it in a scenario
Expand Down
2 changes: 1 addition & 1 deletion docs/understand/scenarios/parametric.md
Original file line number Diff line number Diff line change
Expand Up @@ -226,7 +226,7 @@ Options use the Node helper's names (`distributionChannel`, `apiKey`, `clientTok

The shared `mocked_backend` fixture serves these protobuf responses. Import shared FFE constants and fixture helpers from `utils.mocked_backend.ffe`. On the server distribution endpoint, explicitly accepting `application/protobuf` selects protobuf, including in comma-separated `Accept` lists with a positive quality value. Wildcard-only requests and existing agentless provider tests continue to receive JSON.

Use `mocked_backend.client.ffe_set_response` or `mocked_backend.client.ffe_set_responses` with `response_format="protobuf"` to configure these responses; the default format is JSON. Each format has an independent response timeline, and controls reject response IDs unsupported by the selected format. `mocked_backend.client.ffe_reset` resets both timelines and the shared request history. Use `mocked_backend.ffe_get_requests` to inspect paths, queries, and headers without including fixture control traffic.
Use `mocked_backend.client.ffe_set_response` or `mocked_backend.client.ffe_set_responses` with `response_format="protobuf"` to configure these responses; the default format is JSON. Each format has an independent response timeline, and controls reject response IDs unsupported by the selected format. `mocked_backend.client.ffe_reset` resets both timelines and the shared request history, serves the root UFC again instead of an `ffe_use_suite` suite, and clears `ffe_set_flag_selection_ignored_reason`. Use `mocked_backend.ffe_get_requests` to inspect paths, queries, and headers without including fixture control traffic.

The protobuf response IDs `selection_ignored_empty`, `selection_ignored_tag_load_failed`, and `selection_ignored_unknown` return the full rules fixture with a `DD-Flag-Selection-Ignored` header set to an empty string, `tag_load_failed`, or `future_reason`, respectively. Use these shared responses to test selection diagnostics without replacing the rules endpoint handler. The `valid` response omits the header.

Expand Down
1 change: 1 addition & 0 deletions manifests/cpp.yml
Original file line number Diff line number Diff line change
Expand Up @@ -190,6 +190,7 @@ manifest:
tests/parametric/test_ffe/test_dynamic_evaluation.py::Test_Feature_Flag_Dynamic_Evaluation: missing_feature
tests/parametric/test_ffe/test_dynamic_evaluation.py::Test_Feature_Flag_Split_Features: incomplete_test_app (Split-feature hook capture is currently implemented only in the Java parametric client)
tests/parametric/test_ffe/test_exposure_hooks.py: incomplete_test_app (Exposure hook observations are not implemented)
tests/parametric/test_ffe/test_flag_selection.py: irrelevant (cpp is not in scope for v1 agentless server SDK configuration-source coverage)
tests/parametric/test_ffe/test_rules_configuration.py: missing_feature (Standalone rules configuration helpers are currently tested only in Node.js)
tests/parametric/test_ffe/test_serial_id_metadata.py: missing_feature
tests/parametric/test_ffe/test_serial_id_metadata.py::Test_FFE_Serial_Id_Metadata_Go: irrelevant (Legacy Go metadata key)
Expand Down
1 change: 1 addition & 0 deletions manifests/dotnet.yml
Original file line number Diff line number Diff line change
Expand Up @@ -1267,6 +1267,7 @@ manifest:
tests/parametric/test_ffe/test_dynamic_evaluation.py::Test_Feature_Flag_Dynamic_Evaluation::test_ffe_flag_evaluation: missing_feature # Created by easy win activation script
tests/parametric/test_ffe/test_dynamic_evaluation.py::Test_Feature_Flag_Split_Features: incomplete_test_app (Split-feature hook capture is currently implemented only in the Java parametric client)
tests/parametric/test_ffe/test_exposure_hooks.py: incomplete_test_app (Exposure hook observations are not implemented)
tests/parametric/test_ffe/test_flag_selection.py: missing_feature (The flag selection provider option is not implemented)
Comment thread
mark-caii marked this conversation as resolved.
Resolved
tests/parametric/test_ffe/test_rules_configuration.py: missing_feature (Standalone rules configuration helpers are currently tested only in Node.js)
tests/parametric/test_ffe/test_serial_id_metadata.py: missing_feature
tests/parametric/test_ffe/test_serial_id_metadata.py::Test_FFE_Serial_Id_Metadata_Go: irrelevant (Legacy Go metadata key)
Expand Down
1 change: 1 addition & 0 deletions manifests/golang.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2290,6 +2290,7 @@ manifest:
tests/parametric/test_ffe/test_dynamic_evaluation.py::Test_Feature_Flag_Split_Features: incomplete_test_app (Split-feature hook capture is currently implemented only in the Java parametric client)
tests/parametric/test_ffe/test_exposure_hooks.py: v2.13.0-dev # Easy win for all weblogs and version 2.12.0
tests/parametric/test_ffe/test_exposure_hooks.py::Test_FFE_ExposureHooks::test_evaluation_feature_aggregation: missing_feature (Test agent needs EVP flagevaluation capture)
tests/parametric/test_ffe/test_flag_selection.py: missing_feature (The flag selection provider option is not implemented)
tests/parametric/test_ffe/test_rules_configuration.py: missing_feature (Standalone rules configuration helpers are currently tested only in Node.js)
tests/parametric/test_ffe/test_serial_id_metadata.py: v2.10.1
tests/parametric/test_ffe/test_serial_id_metadata.py::Test_FFE_Serial_Id_Metadata:
Expand Down
1 change: 1 addition & 0 deletions manifests/java.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4281,6 +4281,7 @@ manifest:
tests/parametric/test_ffe/test_exposure_hooks.py: v1.68.0-SNAPSHOT
tests/parametric/test_ffe/test_exposure_hooks.py::Test_FFE_ExposureHooks::test_evaluation_feature_aggregation: missing_feature (Test agent needs EVP flagevaluation capture)
tests/parametric/test_ffe/test_exposure_hooks.py::Test_FFE_ExposureHooks::test_multiprovider_exposure_guard: irrelevant (Go exposure hook composition guard)
tests/parametric/test_ffe/test_flag_selection.py: missing_feature (The flag selection provider option is not implemented)
tests/parametric/test_ffe/test_rules_configuration.py: missing_feature (Standalone rules configuration helpers are currently tested only in Node.js)
tests/parametric/test_ffe/test_serial_id_metadata.py: v1.66.0
tests/parametric/test_ffe/test_serial_id_metadata.py::Test_FFE_Serial_Id_Metadata_Go: irrelevant (Legacy Go metadata key)
Expand Down
1 change: 1 addition & 0 deletions manifests/nodejs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2706,6 +2706,7 @@ manifest:
tests/parametric/test_ffe/test_dynamic_evaluation.py::Test_Feature_Flag_Dynamic_Evaluation::test_ffe_dependent_flags: missing_feature (FFL-1694)
tests/parametric/test_ffe/test_dynamic_evaluation.py::Test_Feature_Flag_Split_Features: incomplete_test_app (Split-feature hook capture is currently implemented only in the Java parametric client)
tests/parametric/test_ffe/test_exposure_hooks.py: incomplete_test_app (Exposure hook observations are not implemented)
tests/parametric/test_ffe/test_flag_selection.py: missing_feature (The flag selection provider option is not implemented)
tests/parametric/test_ffe/test_rules_configuration.py: 7.0.0-pre # Exact development version until the first containing release is confirmed.
tests/parametric/test_ffe/test_rules_configuration.py::Test_FFE_Rules_Configuration_Selection: missing_feature (FFL-3364 requires the Node flag-selection dependency update in dd-trace-js)
tests/parametric/test_ffe/test_serial_id_metadata.py: v6.18.0
Expand Down
1 change: 1 addition & 0 deletions manifests/php.yml
Original file line number Diff line number Diff line change
Expand Up @@ -1660,6 +1660,7 @@ manifest:
tests/parametric/test_ffe/test_dynamic_evaluation.py::Test_Feature_Flag_Dynamic_Evaluation::test_ffe_dependent_flags: missing_feature (FFL-1694)
tests/parametric/test_ffe/test_dynamic_evaluation.py::Test_Feature_Flag_Split_Features: incomplete_test_app (Split-feature hook capture is currently implemented only in the Java parametric client)
tests/parametric/test_ffe/test_exposure_hooks.py: incomplete_test_app (Exposure hook observations are not implemented)
tests/parametric/test_ffe/test_flag_selection.py: missing_feature (The flag selection provider option is not implemented)
tests/parametric/test_ffe/test_rules_configuration.py: missing_feature (Standalone rules configuration helpers are currently tested only in Node.js)
tests/parametric/test_ffe/test_serial_id_metadata.py: missing_feature
tests/parametric/test_ffe/test_serial_id_metadata.py::Test_FFE_Serial_Id_Metadata_Go: irrelevant (Legacy Go metadata key)
Expand Down
1 change: 1 addition & 0 deletions manifests/python.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3161,6 +3161,7 @@ manifest:
tests/parametric/test_ffe/test_dynamic_evaluation.py::Test_Feature_Flag_Dynamic_Evaluation::test_ffe_dependent_flags: missing_feature (FFL-1694)
tests/parametric/test_ffe/test_dynamic_evaluation.py::Test_Feature_Flag_Split_Features: incomplete_test_app (Split-feature hook capture is currently implemented only in the Java parametric client)
tests/parametric/test_ffe/test_exposure_hooks.py: incomplete_test_app (Exposure hook observations are not implemented)
tests/parametric/test_ffe/test_flag_selection.py: missing_feature (The flag selection provider option is not implemented)
tests/parametric/test_ffe/test_rules_configuration.py: missing_feature (Standalone rules configuration helpers are currently tested only in Node.js)
tests/parametric/test_ffe/test_serial_id_metadata.py: missing_feature
tests/parametric/test_ffe/test_serial_id_metadata.py::Test_FFE_Serial_Id_Metadata_Go: irrelevant (Legacy Go metadata key)
Expand Down
1 change: 1 addition & 0 deletions manifests/ruby.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2730,6 +2730,7 @@ manifest:
tests/parametric/test_ffe/test_dynamic_evaluation.py::Test_Feature_Flag_Dynamic_Evaluation::test_ffe_dependent_flags: missing_feature (FFL-1694)
tests/parametric/test_ffe/test_dynamic_evaluation.py::Test_Feature_Flag_Split_Features: incomplete_test_app (Split-feature hook capture is currently implemented only in the Java parametric client)
tests/parametric/test_ffe/test_exposure_hooks.py: incomplete_test_app (Exposure hook observations are not implemented)
tests/parametric/test_ffe/test_flag_selection.py: missing_feature (The flag selection provider option is not implemented)
tests/parametric/test_ffe/test_rules_configuration.py: missing_feature (Standalone rules configuration helpers are currently tested only in Node.js)
tests/parametric/test_ffe/test_serial_id_metadata.py: missing_feature
tests/parametric/test_ffe/test_serial_id_metadata.py::Test_FFE_Serial_Id_Metadata_Go: irrelevant (Legacy Go metadata key)
Expand Down
1 change: 1 addition & 0 deletions manifests/rust.yml
Original file line number Diff line number Diff line change
Expand Up @@ -304,6 +304,7 @@ manifest:
tests/parametric/test_ffe/test_dynamic_evaluation.py::Test_Feature_Flag_Dynamic_Evaluation::test_ffe_flag_evaluation: missing_feature
tests/parametric/test_ffe/test_dynamic_evaluation.py::Test_Feature_Flag_Split_Features: incomplete_test_app (Split-feature hook capture is currently implemented only in the Java parametric client)
tests/parametric/test_ffe/test_exposure_hooks.py: incomplete_test_app (Exposure hook observations are not implemented)
tests/parametric/test_ffe/test_flag_selection.py: irrelevant (rust is not in scope for v1 agentless server SDK configuration-source coverage)
tests/parametric/test_ffe/test_rules_configuration.py: missing_feature (Standalone rules configuration helpers are currently tested only in Node.js)
tests/parametric/test_ffe/test_serial_id_metadata.py: missing_feature
tests/parametric/test_ffe/test_serial_id_metadata.py::Test_FFE_Serial_Id_Metadata_Go: irrelevant (Legacy Go metadata key)
Expand Down
2 changes: 1 addition & 1 deletion tests/parametric/test_ffe/ffe-system-test-data/SOURCE.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
These files are copied from the canonical FFE fixture repository.

Canonical source: https://github.com/DataDog/ffe-system-test-data
Source commit: c8cc8da239341748dc8c30c2145c0f0b5432c958
Source commit: 64572223965cfafa8428431f61e5ed4678c1a1f0

Do not edit these fixtures directly in system-tests. Add or update shared FFE behavior in
ffe-system-test-data first, then refresh this snapshot.
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,148 @@
[
{
"description": "Selected by tag team:checkout.",
"flag": "checkout-banner",
"variationType": "BOOLEAN",
"defaultValue": false,
"targetingKey": "alice",
"attributes": {},
"providerOptions": {
"flagSelection": {
"tags": [" team:search ", "team:checkout", "team:search", ""],
"keyPatterns": ["\tops-*", "ops-*", "billing-*", ""]
}
},
"expectations": {
"configurationRequest": {
"selectedFlagTags": ["team:checkout", "team:search"],
"selectedFlagKeyPatterns": ["billing-*", "ops-*"]
}
},
"result": {
"value": true,
"reason": "STATIC"
}
},
{
"description": "Selected by tag team:checkout.",
"flag": "checkout-pricing",
"variationType": "BOOLEAN",
"defaultValue": false,
"targetingKey": "alice",
"attributes": {},
"providerOptions": {
"flagSelection": {
"tags": [" team:search ", "team:checkout", "team:search", ""],
"keyPatterns": ["\tops-*", "ops-*", "billing-*", ""]
}
},
"expectations": {
"configurationRequest": {
"selectedFlagTags": ["team:checkout", "team:search"],
"selectedFlagKeyPatterns": ["billing-*", "ops-*"]
}
},
"result": {
"value": true,
"reason": "STATIC"
}
},
{
"description": "Selected by tag team:search.",
"flag": "search-ranking",
"variationType": "BOOLEAN",
"defaultValue": false,
"targetingKey": "alice",
"attributes": {},
"providerOptions": {
"flagSelection": {
"tags": [" team:search ", "team:checkout", "team:search", ""],
"keyPatterns": ["\tops-*", "ops-*", "billing-*", ""]
}
},
"expectations": {
"configurationRequest": {
"selectedFlagTags": ["team:checkout", "team:search"],
"selectedFlagKeyPatterns": ["billing-*", "ops-*"]
}
},
"result": {
"value": true,
"reason": "STATIC"
}
},
{
"description": "Not selected: it has no selected tag and matches no key pattern, so the server omits it.",
"flag": "summer-sale",
"variationType": "BOOLEAN",
"defaultValue": false,
"targetingKey": "alice",
"attributes": {},
"providerOptions": {
"flagSelection": {
"tags": [" team:search ", "team:checkout", "team:search", ""],
"keyPatterns": ["\tops-*", "ops-*", "billing-*", ""]
}
},
"expectations": {
"configurationRequest": {
"selectedFlagTags": ["team:checkout", "team:search"],
"selectedFlagKeyPatterns": ["billing-*", "ops-*"]
}
},
"result": {
"value": false,
"reason": "ERROR",
"errorCode": "FLAG_NOT_FOUND"
}
},
{
"description": "Selected by key pattern ops-*.",
"flag": "ops-kill-switch",
"variationType": "BOOLEAN",
"defaultValue": false,
"targetingKey": "alice",
"attributes": {},
"providerOptions": {
"flagSelection": {
"tags": [" team:search ", "team:checkout", "team:search", ""],
"keyPatterns": ["\tops-*", "ops-*", "billing-*", ""]
}
},
"expectations": {
"configurationRequest": {
"selectedFlagTags": ["team:checkout", "team:search"],
"selectedFlagKeyPatterns": ["billing-*", "ops-*"]
}
},
"result": {
"value": true,
"reason": "STATIC"
}
},
{
"description": "Not selected: it has no selected tag and matches no key pattern, so the server omits it.",
"flag": "untagged-flag",
"variationType": "BOOLEAN",
"defaultValue": false,
"targetingKey": "alice",
"attributes": {},
"providerOptions": {
"flagSelection": {
"tags": [" team:search ", "team:checkout", "team:search", ""],
"keyPatterns": ["\tops-*", "ops-*", "billing-*", ""]
}
},
"expectations": {
"configurationRequest": {
"selectedFlagTags": ["team:checkout", "team:search"],
"selectedFlagKeyPatterns": ["billing-*", "ops-*"]
}
},
"result": {
"value": false,
"reason": "ERROR",
"errorCode": "FLAG_NOT_FOUND"
}
}
]
Loading
Loading