Add trace filter tests [APMSP-2763] by Eldolfin · Pull Request #6952 · DataDog/system-tests

Eldolfin · 2026-05-15T17:26:44Z

APMSP-2763
APMSP-3048

Motivation

Changes

Add system-tests coverage for the CSS V1.2 spec trace filters (filter_tags, filter_tags_regex, ignore_resources).

TODO

~~add resources/tags normalization test~~ (maybe in a follow up PR, this is an edge-edge case)

Workflow

⚠️ Create your PR as draft ⚠️
Work on you PR until the CI passes
Mark it as ready for review
- Test logic is modified? -> Get a review from RFC owner.
- Framework is modified, or non obvious usage of it -> get a review from R&P team

🚀 Once your PR is reviewed and the CI green, you can merge it!

🛟 #apm-shared-testing 🛟

Reviewer checklist

Anything but tests/ or manifests/ is modified ? I have the approval from R&P team
A docker base image is modified?
- the relevant build-XXX-image label is present
A scenario is added, removed or renamed?
- Get a review from R&P team

APMSP-2763

github-actions · 2026-05-15T17:27:16Z

CODEOWNERS have been resolved as:

tests/parametric/test_trace_filters.py                                  @DataDog/system-tests-core @DataDog/apm-sdk-capabilities
manifests/cpp.yml                                                       @DataDog/dd-trace-cpp
manifests/dotnet.yml                                                    @DataDog/apm-dotnet @DataDog/asm-dotnet
manifests/golang.yml                                                    @DataDog/dd-trace-go-guild
manifests/java.yml                                                      @DataDog/asm-java @DataDog/apm-java
manifests/nodejs.yml                                                    @DataDog/dd-trace-js
manifests/php.yml                                                       @DataDog/apm-php @DataDog/asm-php
manifests/python.yml                                                    @DataDog/apm-python @DataDog/asm-python
manifests/ruby.yml                                                      @DataDog/ruby-guild @DataDog/asm-ruby
manifests/rust.yml                                                      @DataDog/apm-rust
tests/parametric/test_library_tracestats.py                             @DataDog/system-tests-core @DataDog/apm-sdk-capabilities

… scheme (#2008) These fields were renamed in version 7.79.0 of the agent for consistency. # What does this PR do? Allow both PascalCase and the newer snake_case format for these fields. # Motivation The trace exporter was crashing in a confusing way in new system tests I wrote for another feature: DataDog/system-tests#6952. I had an older agent. # Additional Notes Anything else we should know when reviewing? # How to test the change? Describe here in detail how the change can be validated. Co-authored-by: oscar.ledauphin <oscar.ledauphin@datadoghq.com>

# What does this PR do? Implements the CSS trace-level filtering mechanism, that is applied before stats computation. # Motivation It's un unmet requirement from the spec. # Additional Notes ~Please Ctrl+F `FIXME` when reviewing, I let some questions in the code that I'd like answers for before merging~ ## TODO: - [x] unit tests - [x] system-tests DataDog/system-tests#6952 - [x] normalization before filtering - [x] regex keys (currently only regex values are checked) - [x] new spec or CSS spec RFC - [x] telemetry ? Count dropped traces with the reason - [x] trim spaces when parsing config - [x] Move implementation to trace-utils - [x] stop using v4 span but generic spans instead for php - [x] move arcswap to the trace exporter (remove update_conf) - [x] use test agent for snapshot tests ## Wont do - [ ] ~advanced resource normalization (sql, redis etc...).~ No other implementation does that - [ ] ~100% normalization edge cases (empty service field + non-normalized name + filter depending on that)~ Too specific edge case. Ignoring it for now - [ ] ~check on meta_struct too~ Not in the spec [APMSP-2763] [APMSP-2763]: https://datadoghq.atlassian.net/browse/APMSP-2763?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ Co-authored-by: oscar.ledauphin <oscar.ledauphin@datadoghq.com>

datadog-official · 2026-06-17T15:00:23Z

Tests

🎉 All green!

🧪 All tests passed
❄️ No new flaky tests detected

_{This comment will be updated automatically if new data arrives.

🔗 Commit SHA: 85ab44e | Docs | Datadog PR Page | Give us feedback!}

…ter-test

chatgpt-codex-connector · 2026-07-02T13:00:14Z

💡 Codex Review

system-tests/tests/stats/test_trace_filters.py

Line 50 in 2e3ce18

    
           assert interfaces.library.wait_for(wait_function, 30), f"Timed out waiting for request with status {status_code}"

Avoid aborting scenarios from setup waits

This helper is called from the new setup_* methods, and in this framework pytest_collection_finish invokes those setup methods before tests run and closes targets/re-raises on any setup exception (conftest.py lines 449-478). If a tracer fails to emit the control stats payload within 30s, this assertion aborts the entire scenario during setup instead of reporting only the affected test failure; the wait result should be recorded and asserted in the matching test_* method.

system-tests/tests/test_the_test/scenarios.json

Lines 5641 to 5643 in 2e3ce18

    
           "tests/stats/test_trace_filters.py::Test_Trace_Filters_Regex_Require::test_filter_tags_regex_require": [ 
        
             "TRACE_STATS_COMPUTATION_TRACE_FILTER_REGEX_REQUIRE" 
        
           ],

Add the missing trace-filter scenarios to the map

compute_scenarios_by_files builds the scenarios for a modified test file solely from the nodeids present in this JSON map (utils/scripts/compute_libraries_and_scenarios.py lines 200-207). The added entries stop after the regex-require class, so changes to tests/stats/test_trace_filters.py will not select the three new edge-case scenarios (TRACE_STATS_COMPUTATION_TRACE_FILTER_REJECT_EDGE_CASES, TRACE_STATS_COMPUTATION_TRACE_FILTER_KEY_ONLY_REJECT, and TRACE_STATS_COMPUTATION_TRACE_FILTER_REQUIRE_EDGE_CASES) even though their tests were added in the same file.

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

…ter-test

…, manifests

VianneyRuhlmann · 2026-07-06T11:17:10Z

+        """A require filter drops traces whose root span lacks the required tag."""
+        with test_library, test_library.dd_start_span(name="web.request"):
+            pass
+        assert not test_agent.traces(), "trace lacking the required tag should be dropped"


nit: why not check for stats here ?

These tests are focused on trace filters specifically, so they should not fail if CSS has a problem.

When I tried checking stats here using php it failed for the same reason as why CSS tests are disabled for php:

missing_feature (CSS v1.2.0: dd-trace-php ships CSS since v1.19.0, but the sidecar stats exporter retains buckets younger than ~20s (buffer_len=2 * bucket_size=10s) and dd_trace_synchronous_flush passes force=false, so short-lived parametric tests never observe a /v0.6/stats request)

cbeauchesne

From framework usage, all good.

I let someone familiar with the feature validate the test logic and approve the PR

feat: add trace filter tests (checked on stats)

633a7a5

APMSP-2763

Eldolfin changed the title ~~feat: add trace filter tests (checked on stats)~~ feat: add trace filter tests [APMSP-2763] May 15, 2026

Eldolfin changed the title ~~feat: add trace filter tests [APMSP-2763]~~ Add trace filter tests [APMSP-2763] May 15, 2026

This was referenced May 15, 2026

CSS v1.2.0: Phase A — parametric coverage additions #6938

Merged

feat(data-pipeline)!: CSS Trace Filters DataDog/libdatadog#1985

Merged

fix: simplify tests and explain results

6c4631b

Eldolfin mentioned this pull request May 18, 2026

fix(data-pipeline): allow old PascalCase fields in obfuscation config scheme DataDog/libdatadog#2008

Merged

add manifest

c358546

Eldolfin added 11 commits June 17, 2026 17:15

Merge remote-tracking branch 'origin/main' into oscarld/add-trace-fil…

7173a8e

…ter-test

fix: bad merge

3f0500b

test: edge cases

07511e1

feat: enable for php >=1.22.0

ca48ec8

fix: format

d0de68d

Merge remote-tracking branch 'origin/main' into oscarld/add-trace-fil…

d1e0ac2

…ter-test

fix: add new scenarios to ci

23908c8

fix: stop filtering on http.status_code

37b940b

Merge remote-tracking branch 'origin/main' into oscarld/add-trace-fil…

6db12ce

…ter-test

fix(manifest): add missing_feature to cpp_httpd/nginx

a93ba6a

Merge remote-tracking branch 'origin/main' into oscarld/add-trace-fil…

2e3ce18

…ter-test

Eldolfin marked this pull request as ready for review July 2, 2026 12:54

Eldolfin requested review from a team as code owners July 2, 2026 12:54

Eldolfin requested review from P403n1x87, brettlangdon, daniel-romano-DD, dubloom and manuel-alvarez-alvarez and removed request for a team July 2, 2026 12:54

cbeauchesne reviewed Jul 2, 2026

View reviewed changes

Comment thread utils/_context/_scenarios/__init__.py Outdated

Eldolfin added 3 commits July 2, 2026 16:28

fix: add missing scenarios in scenarios.json

941cdf9

wip change to parametric

a9a1b37

Merge remote-tracking branch 'origin/main' into oscarld/add-trace-fil…

4ca7cac

…ter-test

Eldolfin requested a review from a team as a code owner July 3, 2026 18:37

Eldolfin requested review from mabdinur and removed request for a team July 3, 2026 18:37

Eldolfin added 2 commits July 3, 2026 20:37

revert: delete end-to-end filter tests changes

a380c14

better comments & assertion message, negative cases, fix env tag test…

1a7794c

…, manifests

Eldolfin force-pushed the oscarld/add-trace-filter-test branch from 97052c0 to 1a7794c Compare July 3, 2026 18:38

Eldolfin requested review from VianneyRuhlmann and ichinaski July 6, 2026 11:08

VianneyRuhlmann approved these changes Jul 6, 2026

View reviewed changes

cbeauchesne reviewed Jul 6, 2026

View reviewed changes

Eldolfin added 3 commits July 6, 2026 14:27

fix: add missing test_trace_filter_reject_missing tests

eb8a841

fix(manifest): remove useless comments

d5ef4d0

fix: fmt

85ab44e

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Add trace filter tests [APMSP-2763]#6952

Add trace filter tests [APMSP-2763]#6952
Eldolfin wants to merge 22 commits into
mainfrom
oscarld/add-trace-filter-test

Eldolfin commented May 15, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented May 15, 2026 •

edited

Loading

Uh oh!

datadog-official Bot commented Jun 17, 2026 •

edited by datadog-datadog-prod-us1 Bot

Loading

Uh oh!

chatgpt-codex-connector Bot commented Jul 2, 2026

Uh oh!

Uh oh!

Uh oh!

VianneyRuhlmann Jul 6, 2026

Uh oh!

Eldolfin Jul 6, 2026

Uh oh!

Uh oh!

cbeauchesne left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Uh oh!

Conversation

Eldolfin commented May 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Motivation

Changes

TODO

Workflow

Reviewer checklist

Uh oh!

github-actions Bot commented May 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

datadog-official Bot commented Jun 17, 2026 • edited by datadog-datadog-prod-us1 Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

chatgpt-codex-connector Bot commented Jul 2, 2026

💡 Codex Review

Uh oh!

Uh oh!

Uh oh!

VianneyRuhlmann Jul 6, 2026

Choose a reason for hiding this comment

Uh oh!

Eldolfin Jul 6, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

cbeauchesne left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Eldolfin commented May 15, 2026 •

edited

Loading

github-actions Bot commented May 15, 2026 •

edited

Loading

datadog-official Bot commented Jun 17, 2026 •

edited by datadog-datadog-prod-us1 Bot

Loading