Skip to content

test(ffe): enable .NET agentless configuration source - #7496

Merged
gh-worker-dd-mergequeue-cf854d[bot] merged 11 commits into
mainfrom
pavlo.khrebto/FFL-2703/dotnet-agentless-configuration
Sep 28, 2026
Merged

gh-worker-dd-mergequeue-cf854d[bot] merged 11 commits into
mainfrom
pavlo.khrebto/FFL-2703/dotnet-agentless-configuration

Conversation

@pavlokhrebto

@pavlokhrebto pavlokhrebto commented Aug 12, 2026 •

Copy link
Copy Markdown
Contributor

Motivation

DataDog/dd-trace-dotnet#9017 implements the .NET agentless Feature Flagging configuration source (FFL-2703). This enables the shared configuration-source contract for .NET, following the Node.js (#7355), Java (#7300), and Python (#7411) activations.

Changes

  • Enable tests/parametric/test_ffe/test_configuration_sources.py for .NET from >=3.54.0, the first release with the agentless source.
  • Add /ffe/start and /ffe/evaluate endpoints to the .NET parametric test app (FfeTestApi.cs), mirroring the Python and Java controllers.
  • /ffe/start creates a DatadogProvider and awaits Api.Instance.SetProviderAsync(). Only the async call runs the provider's InitializeAsync, which the tracer's CallTarget instrumentation uses to start agentless/RC delivery and wait for the first configuration.
  • /ffe/evaluate evaluates a flag through the OpenFeature client (Get*DetailsAsync) and returns {value, reason, errorCode}, with errorCode converted from the ErrorType enum to UPPER_SNAKE_CASE (e.g. ProviderNotReady → PROVIDER_NOT_READY) to match the test contract.
  • Add the Datadog.FeatureFlags.OpenFeature package reference (2.3.1) to the parametric test app.

Disabled tests

Five cold-start cases are marked bug (FFL-3325) for .NET:

  • Test_Feature_Flag_Configuration_Source_Cold_Failure_And_Recovery::test_bad_to_unchanged_cold_preserves_not_ready
  • Test_Feature_Flag_Configuration_Source_Cold_Failure_And_Recovery::test_malformed_cold
  • Test_Feature_Flag_Configuration_Source_Cold_Failure_And_Recovery::test_request_timeout_cold
  • Test_Feature_Flag_Configuration_Source_Cold_Failure_And_Recovery::test_unauthorized_cold
  • Test_Feature_Flag_Configuration_Source_Selection::test_remote_config_without_rc_does_not_fallback_to_agentless

They found a real .NET bug on their first run. Before configuration arrives, the provider returned null as the flag value instead of the caller's default: {'errorCode': 'PROVIDER_NOT_READY', 'value': None} where the tests expect 'default'. The error code and reason were already correct. Go, Java, Python and Node return the default and pass these cases.

The fix is in DataDog/dd-trace-dotnet#9299. Once a tracer release includes it, the bug markers get replaced with that version.

Decisions

  • Keep released configuration-source activation independent of EVP fallback test(dotnet): enable agentless EVP exposure egress #7707.
  • Preserve the five FFL-3325 exclusions pending the release containing dd-trace-dotnet#9299.
  • Refresh this existing PR rather than creating a duplicate activation.

Validation (2026-09-23 main refresh)

Base: 8770fe3d914d8f7028c1d4177292acc5b5498a34
Candidate: 4657f3bac00edab8adbb2f5f8cefd7e4a068547f (signed; GitHub verified).

  • ./format.sh --check — passed.
  • python -m pytest -S TEST_THE_TEST tests/test_the_test/test_manifest.py tests/test_the_test/test_version.py -q --disable-warnings — 171 passed, Python 3.12, with PYTHONPATH=$PWD.
  • No fresh .NET parametric E2E run in this refresh. Current-head CI must validate the adapter with the released SDK; the manifest/version self-tests are not runtime proof.

Workflow

  1. ⚠️ Create your PR as draft ⚠️
  2. Work on you PR until the CI passes
  3. Mark it as ready for review
    • Test logic is modified? -> Get a review from RFC owner.
    • Framework is modified, or non obvious usage of it -> get a review from R&P team

🚀 Once your PR is reviewed and the CI green, you can merge it!

🛟 #apm-shared-testing 🛟

Reviewer checklist

  • Anything but tests/ or manifests/ is modified ? I have the approval from R&P team
  • A docker base image is modified?
    • the relevant build-XXX-image label is present
  • A scenario is added, removed or renamed?

@github-actions

github-actions Bot commented Aug 12, 2026 •

Copy link
Copy Markdown
Contributor

CODEOWNERS have been resolved as:

utils/build/docker/dotnet/parametric/Endpoints/FfeTestApi.cs            @DataDog/feature-flagging-and-experimentation-sdk @DataDog/system-tests-core
.github/CODEOWNERS                                                      @DataDog/system-tests-core
manifests/dotnet.yml                                                    @DataDog/system-tests-reviewers
utils/build/docker/dotnet/parametric/ApmTestApi.csproj                  @DataDog/system-tests-reviewers
utils/build/docker/dotnet/parametric/Program.cs                         @DataDog/system-tests-reviewers

@datadog-datadog-prod-us1

datadog-datadog-prod-us1 Bot commented Aug 12, 2026 •

Copy link
Copy Markdown

Tests

✅ All CI checks and tests passed.

🎉 All green!

🧪 All tests passed
❄️ No new flaky tests detected

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: ac712c1 | Docs | View more details | Give us feedback!

pavlokhrebto added a commit to DataDog/dd-trace-dotnet that referenced this pull request Aug 27, 2026
…endpoint derivation (#9040)

## Summary of changes

Adds the foundational configuration layer for agentless Feature Flags
delivery: new configuration keys, a `FeatureFlagsSettings` class with
source resolution and validation, and an `AgentlessEndpoint` struct for
deriving the CDN URL from the Datadog site or a custom base URL.

This is PR 1 of a stacked PR series implementing agentless Feature Flags
configuration delivery (FFL-2703), porting functionality from
[dd-trace-py#19331](DataDog/dd-trace-py#19331)
and
[dd-trace-java#11892](DataDog/dd-trace-java#11892).

## Reason for change

The .NET tracer currently delivers Feature Flags configuration
exclusively through the Datadog Agent's Remote Configuration. To support
agentless (CDN-backed) delivery, we need a configuration layer that:

- Selects between `agentless` (new default), `remote_config`, and
`disabled` sources
- Derives the managed CDN endpoint from `DD_SITE` or accepts a custom
`DD_FEATURE_FLAGS_CONFIGURATION_SOURCE_AGENTLESS_BASE_URL`
- Validates poll interval and request timeout settings
- Grandfathers existing `DD_EXPERIMENTAL_FLAGGING_PROVIDER_ENABLED`
users onto Remote Configuration

## Implementation details

- **`FeatureFlagsSource` enum** — `Disabled`, `Agentless`,
`RemoteConfig`
- **`FeatureFlagsSettings`** — reads all new env vars, resolves the
source with the cross-SDK precedence contract (kill switch → explicit
source → fail-closed → legacy grandfathering → default agentless),
validates poll interval (1–3600s) and request timeout (>0s)
- **`AgentlessEndpoint`** — derives
`https://ufc-server.ff-cdn.<site>/api/v2/feature-flagging/config/rules-based/server`
from the site, appends `dd_env` when configured, accepts custom
HTTP/HTTPS URLs, and never echoes URLs in error messages (credentials
safety)
- **New config keys** in `supported-configurations.yaml` + generated
`ConfigurationKeys.FeatureFlags.g.cs`:
- `DD_FEATURE_FLAGS_ENABLED` (default `true`, supersedes
`DD_EXPERIMENTAL_FLAGGING_PROVIDER_ENABLED`)
  - `DD_FEATURE_FLAGS_CONFIGURATION_SOURCE` (default `agentless`)
  - `DD_FEATURE_FLAGS_CONFIGURATION_SOURCE_AGENTLESS_BASE_URL`
-
`DD_FEATURE_FLAGS_CONFIGURATION_SOURCE_AGENTLESS_POLL_INTERVAL_SECONDS`
(default 30, type `int`)
-
`DD_FEATURE_FLAGS_CONFIGURATION_SOURCE_AGENTLESS_REQUEST_TIMEOUT_SECONDS`
(default 5, type `int`)
- `DD_EXPERIMENTAL_FLAGGING_PROVIDER_INITIALIZATION_TIMEOUT_MS` (default
10000, implementation B)
- **`[Obsolete]`** on `FlaggingProviderEnabled` with pragma suppression
in `FeatureFlagsSettings`

This PR does not wire anything to `FeatureFlagsModule` yet — that
happens in a later PR in the stack.

## Test coverage

- `FeatureFlagsSettingsTests` — 43 test cases covering source resolution
(kill switch, explicit source, legacy grandfathering, `offline`
sentinel, invalid values), blank/whitespace handling, casing
normalization, defaults, poll interval validation, request timeout
validation, initialization timeout, and base URL reading
- `AgentlessEndpointTests` — 18 test cases covering managed endpoint
derivation (site lowercasing, staging, govcloud), `dd_env` query
parameter (addition, null, escaping), custom endpoint path handling
(origin-only gets canonical path, custom path used verbatim), HTTP
acceptance for custom endpoints, invalid URL rejection, empty/whitespace
site rejection, and credentials-in-error safety

## Other details
<!-- Fixes #{issue} -->

Stacked PRs:
- PR 1 (this): Configuration keys, settings, and endpoint derivation
- PR 2: UFC parser and agentless poller
- PR 3: Module wiring and OpenFeature activation
- PR 4: Manual API activation


<!--  ⚠️ Note:

Where possible, please obtain 2 approvals prior to merging. Unless
CODEOWNERS specifies otherwise, for external teams it is typically best
to have one review from a team member, and one review from apm-dotnet.
Trivial changes do not require 2 reviews.

MergeQueue is NOT enabled in this repository. If you have write access
to the repo, the PR has 1-2 approvals (see above), and all of the
required checks have passed, you can use the Squash and Merge button to
merge the PR. If you don't have write access, or you need help, reach
out in the #apm-dotnet channel in Slack.
-->

System tests:
[DataDog/system-tests#7496](DataDog/system-tests#7496)
pavlokhrebto added a commit to DataDog/dd-trace-dotnet that referenced this pull request Aug 27, 2026
## Summary of changes

Adds the agentless delivery mechanism: a `UfcConfigurationParser` that
validates the JSON:API envelope returned by the CDN endpoint, and an
`AgentlessConfigurationSource` HTTP poller with retries, ETag caching,
gzip decompression, and shutdown safety.

This is PR 2 of a stacked PR series implementing agentless Feature Flags
configuration delivery (FFL-2703), porting functionality from
[dd-trace-py#19331](DataDog/dd-trace-py#19331)
and
[dd-trace-java#11892](DataDog/dd-trace-java#11892).
Stacked on [PR 1
#9040](#9040).

## Reason for change

PR 1 added the configuration layer (settings, endpoint derivation). This
PR adds the actual HTTP delivery mechanism that polls the CDN endpoint
for UFC flag configuration. It is self-contained: the poller and parser
are fully testable in isolation and are not yet wired to the
`FeatureFlagsModule` (that happens in PR 3).

## Implementation details

- **`UfcConfigurationParser`** — validates the JSON:API Universal Flag
Configuration response envelope, extracts `data.attributes` into
`ServerConfiguration`. Rejects raw UFC documents (without the JSON:API
wrapper) so all delivery sources agree on one wire format. Returns
descriptive errors; never throws.
- **`AgentlessConfigurationSource`** — HTTP poller with:
  - Fixed-delay polling (no overlap)
- In-tick retries (max 3) for 408/429/5xx/network failures, with
jittered backoff derived from the poll interval (`clamp(P/6, 2s, 10s)`
then `clamp(P/3, 5s, 30s)`)
- ETag / `If-None-Match` conditional polling; ETag advances only after
parse + apply succeed
  - Gzip decompression (neither transport decompresses automatically)
  - Self-tracing suppression (`x-datadog-tracing-enabled: false` header)
- API key sent only to managed (Datadog-derived) endpoints, never custom
ones
- Warn-once per failure category (authentication, http, request) to
prevent log flooding
- Shutdown safety: `CancellationTokenSource` cancels the loop; shutdown
checks after request and before `ApplyAsync` prevent use-after-dispose
- `Create()` factory returns `null` when endpoint can'"'t be built or
API key is missing
  - Idempotent `Start()` via `Interlocked.CompareExchange`

## Test coverage

- **`UfcConfigurationParserTests`** (16 tests) — valid envelope parsing,
malformed JSON rejection, invalid envelope rejection (wrong type,
missing data, raw UFC), invalid attributes rejection (missing
format/createdAt/environment/flags, wrong types), flag parsing
- **`AgentlessConfigurationSourceTests`** (12 tests) — 200 success, ETag
forwarding, 304/401/malformed handling, disposal before/after request,
retry on 500 then success, max retries on 500, no retry on 400, gzip
response, network error, idempotent start

## Other details
<!-- Fixes #{issue} -->

Stacked PRs:
- PR 1 [#9040](#9040):
Configuration keys, settings, and endpoint derivation
- PR 2 (this): UFC parser and agentless HTTP poller
- PR 3: Module wiring and OpenFeature activation
- PR 4: Manual API activation


<!--  ⚠️ Note:

Where possible, please obtain 2 approvals prior to merging. Unless
CODEOWNERS specifies otherwise, for external teams it is typically best
to have one review from a team member, and one review from apm-dotnet.
Trivial changes do not require 2 reviews.

MergeQueue is NOT enabled in this repository. If you have write access
to the repo, the PR has 1-2 approvals (see above), and all of the
required checks have passed, you can use the Squash and Merge button to
merge the PR. If you don't have write access, or you need help, reach
out in the #apm-dotnet channel in Slack.
-->

System tests:
[DataDog/system-tests#7496](DataDog/system-tests#7496)
pavlokhrebto added a commit to DataDog/dd-trace-dotnet that referenced this pull request Sep 14, 2026
…InitializeAsync (#9044)

## Summary of changes

Wires the agentless poller and RC subscription into `FeatureFlagsModule`
with source-agnostic activation, and adds `InitializeAsync` through the
OpenFeature provider lifecycle so delivery starts when application code
initializes the provider.

This is PR 3 of a stacked PR series implementing agentless Feature Flags
configuration delivery (FFL-2703), porting functionality from
[dd-trace-py#19331](DataDog/dd-trace-py#19331)
and
[dd-trace-java#11892](DataDog/dd-trace-java#11892).
Stacked on [PR 1
#9040](#9040) and [PR 2
#9042](#9042).

## Reason for change

PR 1 added the configuration layer and PR 2 added the poller and parser.
This PR connects them: the module now selects the delivery source at
construction time, starts polling only when the provider is initialized,
and waits for the first configuration before returning from
`InitializeAsync`.

## Implementation details

- **`TracerSettings`** — replaces `IsFlaggingProviderEnabled` with
`FeatureFlagsSettings FeatureFlags` property
- **`FeatureFlagsModule`** — source-agnostic constructor:
- RC subscription only when `Source == RemoteConfig` (gated, no longer
always-on)
  - `Create()` checks `settings.FeatureFlags.Enabled`
- `Activate()` starts the agentless poller or marks RC intent, with
`_deliveryStarted` tracking
- `InitializeAsync()` calls `Activate()`, waits for
`FirstConfigReceived` with timeout, skips the wait if no delivery
started
- `ApplyConfiguration()` — public method used by both agentless and RC
paths, signals `FirstConfigReceived`
- `Dispose()` now also disposes the agentless source and unsubscribes RC
- **`OpenFeatureSdkInitializeAsyncIntegration`** (new CallTarget) —
intercepts `FeatureFlagsSdk.InitializeAsync` stub and calls
`FeatureFlagsModule.InitializeAsync`
- **`DatadogProvider.InitializeAsync`** — override that delegates to
`FeatureFlagsSdk.InitializeAsync`
- **`FeatureFlagsSdk.InitializeAsync`** — no-op stub in the OpenFeature
package
- **`supported_calltargets.g.json`** — registers the new integration
- **`SpanEnrichmentIntegrationTests`** — suppresses `[Obsolete]` warning
on `FlaggingProviderEnabled`

## Test coverage

- **`FeatureFlagsModuleTests`** (9 tests) — RC config update/reset,
agentless source doesn't subscribe to RC, RC source subscribes and
advertises capability, disabled returns null, `InitializeAsync` returns
immediately when config already applied, returns when config arrives
while waiting, returns on timeout without throwing, returns on
cancellation without throwing, returns immediately when agentless source
cannot start (no API key)

## Other details
<!-- Fixes #{issue} -->

Stacked PRs:
- PR 1 [#9040](#9040):
Configuration keys, settings, and endpoint derivation
- PR 2 [#9042](#9042):
UFC parser and agentless HTTP poller
- PR 3 (this): Module wiring and OpenFeature activation
- PR 4: Manual API activation


<!--  ⚠️ Note:

Where possible, please obtain 2 approvals prior to merging. Unless
CODEOWNERS specifies otherwise, for external teams it is typically best
to have one review from a team member, and one review from apm-dotnet.
Trivial changes do not require 2 reviews.

MergeQueue is NOT enabled in this repository. If you have write access
to the repo, the PR has 1-2 approvals (see above), and all of the
required checks have passed, you can use the Squash and Merge button to
merge the PR. If you don't have write access, or you need help, reach
out in the #apm-dotnet channel in Slack.
-->

System tests:
[DataDog/system-tests#7496](DataDog/system-tests#7496)
@pavlokhrebto
pavlokhrebto marked this pull request as ready for review September 23, 2026 15:18
@pavlokhrebto
pavlokhrebto requested a review from a team as a code owner September 23, 2026 15:18
Refresh the existing configuration activation without folding in EVP fallback.

Environment: Datadog workspace
pavlokhrebto added a commit to DataDog/dd-trace-dotnet that referenced this pull request Sep 24, 2026
…ror (#9299)

## Summary of changes

- `FeatureFlagsModule.Evaluate` returns the caller's default instead of
`null` when no configuration has arrived.
- `FeatureFlagsSdk.GetResolutionDetails` returns the caller's default
for every evaluation that carries an error, and when the tracer is not
attached.
- `Datadog.FeatureFlags.OpenFeature` 2.3.1 → 2.3.2.

## Reason for change

Before configuration arrives, `GetStringValueAsync("flag", "fallback")`
returned `null` instead of `"fallback"`, with `PROVIDER_NOT_READY`. That
is the startup window every application passes through.

OpenFeature substitutes the caller's default only when a provider
throws. This provider reports errors as resolution details instead, so
the default must travel with the details. Three layers could have
supplied it, and none did:

1. **Tracer** — the module's not-ready shortcut returned `null`. It came
in with the original SDK in #7896, where the error string was `No config
loaded`. #8367 changed the string to `PROVIDER_NOT_READY` and kept the
`null`. The evaluator's own not-ready path already returned
`defaultValue`, so the two paths disagreed.
2. **Package** — `GetResolutionDetails` never received the default. It
copied whatever value the tracer returned, and returned `default!` when
the tracer is not attached. For a string flag that is `null`. It came in
with the package in #8077.
3. **OpenFeature client** — it does not step in, because the provider
does not throw.

The same gap also lost JSON defaults. On error the tracer hands back the
caller's OpenFeature `Value`, which `JsonToValue` cannot convert, so the
caller got an empty `Value`.

## How we found it

system-tests [#7496](DataDog/system-tests#7496)
enabled the Feature Flags configuration-source tests for .NET for the
first time. On the first run, the five cold-start cases failed with
`{'errorCode': 'PROVIDER_NOT_READY', 'value': None}` where they expect
the default. Those tests had been `missing_feature` for .NET, and the
unit tests on this path asserted only the error, never the value, so
nothing had covered it.

## Implementation details

The package is the layer that owns the rule: any evaluation with an
error type resolves to the caller's default. The tracer fix is still
needed on its own, because it ships with the tracer, while the package
reaches customers only when they upgrade it.

`Resolve<T>` now takes `T defaultValue` instead of `object?`. No
CallTarget instruments `Resolve`, so changing the signature is safe.

## Test coverage

`Evaluate_WhenNoConfigurationYet_ReturnsTheDefaultValue` covers string,
boolean, integer and numeric flags, and asserts the value. The package
has no unit-test project. The system-tests cold-start cases cover the
not-ready path end to end.

## Other details

Jira: [FFL-3325](https://datadoghq.atlassian.net/browse/FFL-3325)

Delivery: the tracer fix ships in the next tracer release. The package
fix needs `Datadog.FeatureFlags.OpenFeature` 2.3.2.


[FFL-3325]:
https://datadoghq.atlassian.net/browse/FFL-3325?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ

@nccatoni nccatoni left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM but you should probably get a review from someone from dotnet

@andrewlock andrewlock left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The manifest changes LGTM, I'm deffering on the API as I think that's entirely up to FFE how they wish to write it (but I would def suggest updating the package version

@gh-worker-dd-mergequeue-cf854d
gh-worker-dd-mergequeue-cf854d Bot merged commit 23eaea6 into main Sep 28, 2026
447 checks passed
@gh-worker-dd-mergequeue-cf854d
gh-worker-dd-mergequeue-cf854d Bot deleted the pavlo.khrebto/FFL-2703/dotnet-agentless-configuration branch September 28, 2026 06:52
korniltsev-grafanista added a commit to grafana/pyroscope-dotnet that referenced this pull request Oct 29, 2026
* [Version Bump] 3.53.0 (#9104)

The following files were found to be modified (as expected)

- [x] docs/CHANGELOG.md
- [x] .azure-pipelines/ultimate-pipeline.yml
- [x]
profiler/src/ProfilerEngine/Datadog.Profiler.Native.Linux/CMakeLists.txt
- [x]
profiler/src/ProfilerEngine/Datadog.Profiler.Native.Windows/Resource.rc
- [x]
profiler/src/ProfilerEngine/Datadog.Profiler.Native/dd_profiler_version.h
- [x]
profiler/src/ProfilerEngine/Datadog.Linux.ApiWrapper/CMakeLists.txt
- [x] profiler/src/ProfilerEngine/ProductVersion.props
- [x] shared/src/Datadog.Trace.ClrProfiler.Native/CMakeLists.txt
- [x] shared/src/Datadog.Trace.ClrProfiler.Native/Resource.rc
- [x] shared/src/msi-installer/WindowsInstaller.wixproj
- [x] shared/src/native-src/version.h
- [x] tracer/build/artifacts/dd-dotnet.sh
- [x] tracer/build/_build/Build.cs
- [x]
tracer/samples/AutomaticTraceIdInjection/MicrosoftExtensionsExample/MicrosoftExtensionsExample.csproj
- [x]
tracer/samples/AutomaticTraceIdInjection/Log4NetExample/Log4NetExample.csproj
- [x]
tracer/samples/AutomaticTraceIdInjection/NLog40Example/NLog40Example.csproj
- [x]
tracer/samples/AutomaticTraceIdInjection/NLog45Example/NLog45Example.csproj
- [x]
tracer/samples/AutomaticTraceIdInjection/NLog46Example/NLog46Example.csproj
- [x]
tracer/samples/AutomaticTraceIdInjection/SerilogExample/SerilogExample.csproj
- [x] tracer/samples/ConsoleApp/Alpine3.10.dockerfile
- [x] tracer/samples/ConsoleApp/Alpine3.9.dockerfile
- [x] tracer/samples/ConsoleApp/Debian.dockerfile
- [x] tracer/samples/OpenTelemetry/Debian.dockerfile
- [x] tracer/samples/WindowsContainer/Dockerfile
- [x] tracer/src/Datadog.Trace.ClrProfiler.Managed.Loader/Startup.cs
- [x] tracer/src/Datadog.Tracer.Native/CMakeLists.txt
- [x] tracer/src/Datadog.Tracer.Native/dd_profiler_constants.h
- [x] tracer/src/Datadog.Tracer.Native/Resource.rc
- [x] tracer/src/Directory.Build.props
- [x] tracer/src/Datadog.Trace/TracerConstants.cs

@DataDog/apm-dotnet

Co-authored-by: dd-octo-sts[bot] <200755185+dd-octo-sts[bot]@users.noreply.github.com>

* Fix broken unit tests in `GcMemoryLoadCalculatorTests` (#9101)

## Summary of changes

Fixes the new unit tests that only fail on master

## Reason for change

The behaviour of `Environment.SetEnvironmentVariable(.., "")` is
different on < .NET 9, which causes the new unit tests introduced in
#9505 to break only on master, where we run .NET 7 and .NET 8. This
fixes those tests

## Implementation details

These tests cases don't work on < .NET 9, because they _don't_ set a
`""` variable, they instead delete the variable:

```c#
[InlineData(PlatformKeys.DotNetGCHighMemPercent, "")]
```

In .NET 9+ these tests work, as they _do_ set the variable with the `""`
value

## Test coverage

Added a couple of extra cases just to cover other edge cases while I was
there.
I'll make sure to do a full TFM run to be sure it's fixed

## Other details

Currently causing failures on master (not PRs) so high urgency

* [Test Package Versions Bump] Updating package versions (#9100)

Updates the package versions for integration tests. See below for a
summary of the changes.

## Package Version Updates

| Package | Integration | Previous | New | Published |
|---------|-------------|----------|-----|-----------|
| AWSSDK.Core | AwsSdk | 4.0.101 | 4.0.102 | 2026-08-20 |
| AWSSDK.DynamoDBv2 | AwsDynamoDb | 4.0.103.1 | 4.0.103.4 | 2026-08-20 |
| AWSSDK.Kinesis | AwsKinesis | 4.0.100.8 | 4.0.100.10 | 2026-08-20 |
| AWSSDK.SQS | AwsSqs | 4.0.100.8 | 4.0.100.10 | 2026-08-20 |
| AWSSDK.SimpleNotificationService | AwsSns | 4.0.100.8 | 4.0.100.10 |
2026-08-20 |
| AWSSDK.EventBridge | AwsEventBridge | 4.0.100.8 | 4.0.100.10 |
2026-08-20 |
| AWSSDK.S3 | AwsS3 | 4.0.102.1 | 4.0.102.3 | 2026-08-20 |
| AWSSDK.StepFunctions | AwsStepFunctions | 4.0.100.8 | 4.0.100.10 |
2026-08-20 |
| GraphQL | GraphQL7 | 8.8.4 | 8.8.5 | 2026-08-18 |
| Google.Protobuf | Protobuf | 3.35.1 | 3.36.0 | 2026-08-20 |
| ServiceStack.Redis | ServiceStackRedis | 10.1.2 | 10.1.4 | 2026-08-18
|

## New Major Versions Available

These have a new major outside our supported range. Review whether to
widen the
integration's `MaximumVersion` and the definition's
`MaxVersionExclusive`/`SpecificVersions`,
or add a new split entry, to adopt.

| Package | Integration | Current cap | Latest available |
|---------|-------------|-------------|------------------|
| xunit.v3 | XUnitRetriesV3 | 3.2.2 | 4.0.0 |
| xunit.v3 | XUnitV3 | 3.2.2 | 4.0.0 |

## Package Version Cooldown Report

The following versions were published less than **2 days** ago and have
been ignored.
These require manual review before inclusion.

| Package | Integration | Version Kept | Ignored Version | Published |
Age (days) |

|---------|-------------|--------------|-----------------|-----------|------------|
| AWSSDK.Kinesis | AwsKinesis | 4.0.100.10 | 4.0.101 | 2026-08-21 | 1 |
| StackExchange.Redis | StackExchangeRedis | 3.1.13 | 3.1.31 |
2026-08-21 | 1 |
| log4net | log4net | 3.3.2 | 3.4.0 | 2026-08-21 | 1 |
| OpenTelemetry.Api | OpenTelemetry | 1.17.0 | 1.18.0 | 2026-08-21 | 1 |

Co-authored-by: dd-octo-sts[bot] <200755185+dd-octo-sts[bot]@users.noreply.github.com>

* Move secrets to be protected environment secrets (#8865)

## Summary of changes

- Don't pass secrets down, instead obtain directly from environment
- Only expose secret on steps that need it (Nuke step)

## Reason for change

We're moving `AZURE_DEVOPS_TOKEN` from being a repo secret to a
protected environment secret. This does some of the associated cleanup.

## Implementation details

- **Don't try to pass the secrets in to the worfklow**. AFAICT, this
technically wasn't working as we expected 😅 The "top level"
`create_normal_draft_release` workflow was trying to access and pass in
the variables, but as it wasn't running in an `environment`, this was
passing an empty string. However, this didn't break anything, because
`{{ secrets.AZURE_DEVOPS_TOKEN }}` was transparently accessing the
environment secret _anyway_. So we can just stop _trying_ to pass them
in now.
- `create-system-test-docker-base-images.yml` needs to be in the
"release" env, because it needs the azure devops token to grab the build
results from CI (avoiding rate limits)

Additionally, I duplicated the repo-secret `AZURE_DEVOPS_TOKEN` into the
existing `publish-debug-symbols-env` environment, so when we merge this,
everything should "just work".

Finally, we can make the `publish-debug-symbols-env`, "protected" so it
only runs on release branches.

## Test coverage

Nope, we can't really test any of this 🙁 Hence why I want to babysit it
the first time 😅

## Other details

I'd like to change `publish-debug-symbols-env` to be called
`release-env` (or something similar), but that means changing a whole
bunch of `sts` token stuff, across multiple repos, and it doesn't seem
worth the hassle 😅

* [Feature Flags] Send the split serial id on exposure events (#9088)

## Summary of changes

Add an optional `serial_id` field to the feature-flag exposure event,
and send it when the split carries a serial id.

## Reason for change

The exposures intake uses the serial id to find the holdout that an
allocation comes from. The UFC compiler rewrites a holdout into a usual
allocation before the tracer receives the flag configuration, so the
exposure event holds no other link back to the holdout.

The tracer already reads the serial id and uses it for APM span
enrichment. This change also puts it on the exposure event.

## Implementation details

| File | Change |
|---|---|
| `Exposure/Model/ExposureEvent.cs` | Add the optional `SerialId`
property. It goes on the wire as a top-level `serial_id`. |
| `FeatureFlagsEvaluator.cs` | Give the serial id from the split to the
exposure dispatch. |
| `Exposure/ExposureCache.cs` | Compare the serial id in the
deduplication cache. |
| `Exposure/ExposureApi.cs` | Make `SerializerSettings` internal, so
that a test can assert the wire shape. |

`ResolveVariant` gives the serial id to `DispatchExposure` directly. It
does not read the value from the flag metadata, because `ResolveVariant`
writes that metadata entry only when span enrichment is on.

The serializer includes nulls, so `ExposureEvent` gives `SerialId` a
`ShouldSerializeSerialId` method. An event without a serial id omits the
key instead of sending `null`. A `JsonProperty` named argument cannot do
this, because `AttributeTests` forbids an attribute argument whose type
is not in the core assembly, and `NullValueHandling` is an enum in
`Datadog.Trace`. `ProbeException.ShouldSerializeStackTrace` solves the
same problem the same way.

The tracer does not validate the value. The flag configuration layer
validates it.

## Test coverage

Nine new test cases in three classes:

- `ExposureEventTests` — the wire shape: `serial_id` for `340132`,
`serial_id` for `0`, and no key when the value is absent.
- `FeatureFlagsEvaluatorTests` — the dispatched event, for a split with
a serial id and for a split without one.
- `ExposureCacheTests` — deduplication when the serial id appears,
changes, disappears, and when it is `0`.

A serial id of `0` is a usual value. Serial ids start at `0` for each
organization, so the first allocation of every organization has one. The
tests pin `0` as present, not as absent.

```
dotnet test tracer/test/Datadog.Trace.Tests/Datadog.Trace.Tests.csproj -f net10.0 \
  --filter "FullyQualifiedName~Datadog.Trace.Tests.FeatureFlags"
```

378 tests pass.

## Other details

The deduplication cache now compares the serial id. A subject therefore
sends one more exposure when its serial id appears or changes while the
allocation key and the variant key stay the same. This is intended: a
configuration refresh can add or change a serial id without changing
either key, and the intake must receive the new value.

Part of the work to send the serial id from every Datadog SDK.
`DataDog/dd-trace-py#19753` and `DataDog/dd-trace-go#5214` make the same
change.

*This description was generated by Claude.*

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>

* Avoid unobserved tasks for synchronous delegate exceptions (#9071)

## Summary of changes

Prevent delegate instrumentation from creating an orphaned faulted task
when a Task-returning delegate throws synchronously.

## Reason for change

[Handler exceptions in AWS Lambda resurface as UnobservedTaskException
after upgrading 3.45.0 -> 3.51.1
   #9069 ](https://github.com/DataDog/dd-trace-dotnet/issues/9069)

AWS Lambda executes customer handlers through a generated synchronous
action. For async handlers, [AWS calls
GetAwaiter().GetResult()](https://github.com/aws/aws-lambda-dotnet/blob/91df4deb1e05d9b4042b8837ed81700e4db7224e/Libraries/src/Amazon.Lambda.RuntimeSupport/Bootstrap/InvokeDelegateBuilder.cs#L220-L243),
which can throw synchronously when the customer’s task is faulted. AWS
invokes this action [before creating the outer task with
Task.FromResult](https://github.com/aws/aws-lambda-dotnet/blob/91df4deb1e05d9b4042b8837ed81700e4db7224e/Libraries/src/Amazon.Lambda.RuntimeSupport/Bootstrap/HandlerWrapper.cs#L70-L80).

In this case, [the wrapper propagates the synchronous
exception](https://github.com/DataDog/dd-trace-dotnet/blob/ba875da17863b62331b9e4e0ec9e1a4e53ab9176/tracer/src/Datadog.Trace/Util/Delegates/DelegateInstrumentation.cs#L884-L904),
but the finally block also [creates an asynchronous continuation with a
null original
task](https://github.com/DataDog/dd-trace-dotnet/blob/ba875da17863b62331b9e4e0ec9e1a4e53ab9176/tracer/src/Datadog.Trace/Util/Delegates/DelegateInstrumentation.cs#L311-L322).
The continuation rethrows the same exception, becoming a faulted task
that cannot be returned because the original synchronous exception is
already leaving the wrapper.

That task is never observed and eventually raises
`TaskScheduler.UnobservedTaskException` during garbage collection.

The PR that likely triggered this error is the one adding support for
`Amazon.Lambda.RuntimeSupport` 2.x
https://github.com/DataDog/dd-trace-dotnet/pull/8797

## Implementation details

Only rethrow an exception from the continuation when the wrapped
delegate actually returned a task.

## Test coverage

Added a non-parallel regression test. The test fails against the
previous implementation and passes with this change.

* [Propagators] Generalize tracestate parsing (#9000)

## Summary of changes

- Use `RealOnlySpan` in W3C Tracestate parsing

## Reason for change

Isolate reusable tracestate parsing parts to later add OpenTelemetry
consistent-probability-sampling feature #8983.
There will be more overhead in the next PR since it requires parsing a
new tracestate member, and some of it is already there since this PR is
making the parsing less specific

---------

Co-authored-by: Zach Montoya <zach.montoya@datadoghq.com>

* Split system-test docker image creation to work in both protected and unprotected environments (#9094)

## Summary of changes

Update system-test docker image creation to work both on master and on
PRs

## Reason for change

#8865 moves our secrets into protected environments, which means they
can no longer be accessed from PRs. Unfortunately, the docker-image
artifact creation can be enabled on PRs by adding the
`docker_image_artifacts` label, so that PR breaks that workflow. This PR
re-enables it, with the caveat that it is subject to stricter rate
limits (which shouldn't be an issue in practice, given this is rarely
used, and the only workflow that needs this approach)

## Implementation details

- Stacked on #8865
- Extract a helper for making anonymous connections to Azure DevOps
- Move the majority of the existing workflow into the composite action
instead
- Create two different workflows
- One runs on master/hotfix/release branches, uses the protected
environment and the github secret
  - The other runs on branches and uses the anonymous workflow

## Test coverage

~This is the test 🤞~

As this depends on new workflows, I don't think we can test it till
_after_ it's merged, so :yolo:

## Other details

* Don't need to download and sign when pushing a tag (#9105)

## Summary of changes

Fix benign failures during release

## Reason for change

When a release is published, we push a tag, which triggers a gitlab
build. This triggers _Some_ workflows (OCI/lib-injection images) but we
_don't_ need to build or publish. Currently we are trying to download
from an Azure DevOps pipeline that never exists, so ultimately we fail.

## Implementation details

Don't try to download/sign nuget packages on release (tag)

## Test coverage

Can't test this until the next release, but it's hopefully pretty simple

## Other details

Copied the syntax from the `publish-artifacts` stage

* Migrate remaining integration tests to CombinatorialOrPairwiseData (#8528)

## Summary of changes

Migrates (what I think are) the remaining integration tests to use
CombinatorialOrPairwiseData with the goal of speeding up PR CI runs with
the hope of not missing regressions

## Reason for change

As we have many tests our CI takes time running them, this aims to
reduce some pressure as if a PR meets the criteria for running in
Pairwise mode we will swap from combinatorial to pairwise for all of
these integrations tests.

This will hopefully reduce times for runs that _likely_ won't modify /
impact these tests.

## Implementation details

  One commit per file with the following pattern(s):

  - Drop `GetEnabledConfig()` and the matching `[MemberData(...)]`
    -   Replace with `[CombinatorialOrPairwiseData]`
- Add Attributes : `[PackageVersionData(nameof(PackageVersions.X))]` and
`[MetadataSchemaVersionData]`

Note that some tests needed a few more changes.

## Test coverage

Less!

Same though on non-PR runs or on PR runs that end up on the thorough
testing.

## Other details
<!-- Fixes #{issue} -->

I have a planned follow up to reduce the V1 Schema tests that are run in
general (the V0/V1 split causes most tests to be doubled for no real
benefit as the move to the V1 schema has largely been replaced)

<!--  ⚠️ Note:

Where possible, please obtain 2 approvals prior to merging. Unless
CODEOWNERS specifies otherwise, for external teams it is typically best
to have one review from a team member, and one review from apm-dotnet.
Trivial changes do not require 2 reviews.

MergeQueue is NOT enabled in this repository. If you have write access
to the repo, the PR has 1-2 approvals (see above), and all of the
required checks have passed, you can use the Squash and Merge button to
merge the PR. If you don't have write access, or you need help, reach
out in the #apm-dotnet channel in Slack.
-->

* Mark DSM TransportsWorkCorrectly as flaky (#9116)

## Summary of changes

TransportsWorkCorrectly flaked, marking it as flaky.

## Reason for change

Green CI progress.

## Implementation details

Marked it as flaky

## Test coverage

## Other details
<!-- Fixes #{issue} -->
Did not investigate, marking as flaky - if it flakes with this I say we
skip it and raise it to be reworked.

#incident-57303
<!--  ⚠️ Note:

Where possible, please obtain 2 approvals prior to merging. Unless
CODEOWNERS specifies otherwise, for external teams it is typically best
to have one review from a team member, and one review from apm-dotnet.
Trivial changes do not require 2 reviews.

MergeQueue is NOT enabled in this repository. If you have write access
to the repo, the PR has 1-2 approvals (see above), and all of the
required checks have passed, you can use the Squash and Merge button to
merge the PR. If you don't have write access, or you need help, reach
out in the #apm-dotnet channel in Slack.
-->

* Mark DSM ContextPropagation as flaky (#9117)

## Summary of changes

Mark
Datadog.Trace.ClrProfiler.IntegrationTests.DataStreamsMonitoringManualApiTest.ContextPropagation
as flaky

## Reason for change

Flaked in CI, Green CI, trying to make more progress in getting a green
CI

## Implementation details

Marked as flaky, didn't investigate further.

## Test coverage

Same-ish

## Other details
<!-- Fixes #{issue} -->

#incident-57303

<!--  ⚠️ Note:

Where possible, please obtain 2 approvals prior to merging. Unless
CODEOWNERS specifies otherwise, for external teams it is typically best
to have one review from a team member, and one review from apm-dotnet.
Trivial changes do not require 2 reviews.

MergeQueue is NOT enabled in this repository. If you have write access
to the repo, the PR has 1-2 approvals (see above), and all of the
required checks have passed, you can use the Squash and Merge button to
merge the PR. If you don't have write access, or you need help, reach
out in the #apm-dotnet channel in Slack.
-->

* [CI Visibility] Fix coverage rewrites with shared framework dependencies (#9102)

## Summary of changes

- resolve coverage rewrite dependencies from runtimeconfig-declared .NET
shared frameworks
- stage and validate rewritten DLL/PDB files before replacing the
originals
- restore the original assembly and symbols if publication fails

## Reason for change

The coverage collector could not resolve assemblies that were available
only through an installed shared framework, such as
`Microsoft.Extensions.Logging.Abstractions` from
`Microsoft.AspNetCore.App`. If Mono.Cecil encountered that dependency
while writing metadata, it had already truncated the assembly under test
and left a damaged DLL on disk.

## Implementation details

- discover `framework`, `frameworks`, and `includedFrameworks` entries
from sibling `*.runtimeconfig.json` files
- derive the installed shared-framework root from
`System.Private.CoreLib` and select the highest compatible patch version
using full semantic-version ordering
- cache immutable shared-framework search directories per output
directory while keeping Cecil assemblies resolver-owned
- write DLL/PDB output to unique sibling transaction files with
non-assembly extensions, validate both by reopening them, then publish
with `File.Replace` backups and rollback
- restore both backups after partial publication failures, including
when the target assembly is missing or already replaced

## Test coverage

- `CoverageResolverTests` on net10.0 and net8.0
- `CoverageRewriteTests` on net10.0
- `Datadog.Trace.Coverage.collector` project build
- net10.0 three-project reproduction with `Microsoft.AspNetCore.App` and
an optional `LogLevel` parameter

## Other details

Windows and Linux CI provide the remaining platform coverage for
replacement and locking semantics.

Fixes #9023

* [CI Visibility] Refactor CODEOWNERS parsing and source ownership resolution (#9099)

## Summary of changes

- Refactor CODEOWNERS support into the dedicated
`Datadog.Trace.Ci.CodeOwnership` namespace.
- Separate provider-independent matching from GitHub and GitLab rule
evaluation.
- Separate CODEOWNERS discovery, repository path resolution, ownership
matching, and CI session integration.
- Fix CODEOWNERS matching to follow GitHub and GitLab semantics.
- Normalize compiler-recorded source paths so CI Visibility can resolve
repository files and their owners.
- Resolve CODEOWNERS once per test session and reuse a bounded
source-path cache across test suites.

## Reason for change

CI Visibility uses CODEOWNERS metadata to determine test ownership. The
previous parser mixed GitHub and GitLab behavior and diverged from both
providers in several cases, which could assign incorrect owners.

Compiler-recorded source paths can also contain CI-agent-specific
relative prefixes. Valid repository files could therefore appear outside
the CODEOWNERS root and lose their ownership metadata.

The implementation grew while fixing these correctness issues, so the
final structure now gives parsing, provider behavior, file discovery,
path normalization, and session lifetime separate responsibilities.

## Architecture

- `CodeOwners.cs` contains parsing entry points and the shared bounded
glob matcher.
- `CodeOwners.GitHub.cs` implements GitHub owner validation, rule
parsing, and global last-match precedence.
- `CodeOwners.GitLab.cs` implements sections, default owners, scoped
exclusions, duplicate-pattern handling, and independent section
evaluation.
- `CodeOwnersFileLocator.cs` finds the applicable file and selects the
dialect from repository metadata, CI provider metadata, or an
unambiguous provider-specific location.
- `RepositorySourcePathResolver.cs` converts compiler paths into safe
repository-relative paths.
- `CodeOwnersResolver.cs` owns the session-level CODEOWNERS instance and
bounded source-path cache.
- `SourceOwnership.cs` carries the resolved repository path, matched
owners, and pre-serialized CI tag.
- `CIEnvironmentValues` now only initializes and exposes the
session-level resolver.

## Matching and path behavior

- GitHub uses the last matching rule globally.
- GitLab evaluates sections independently, applies section defaults, and
keeps exclusions scoped to their section.
- Rooted and unrooted patterns, directory patterns, wildcards,
globstars, escaped characters, comments, case sensitivity, and Windows
separators follow provider-specific behavior.
- The glob matcher has bounded pattern size and match steps so
pathological rules cannot block the instrumented process.
- Unresolved relative compiler paths are anchored using the longest
existing suffix under the CODEOWNERS root.
- Absolute paths outside the repository, URIs, bare filenames, interior
navigation segments, and non-existent suffixes are not re-anchored.

## Allocation and lifetime considerations

- The CODEOWNERS file is discovered and parsed once when the test
session is initialized.
- Test suites reuse the same resolver and source-path decisions.
- The source-path cache disables itself after its configured limit
instead of retaining paths without bounds.
- Repository suffix inspection uses span-based segment enumeration and
allocates only the selected normalized suffix.
- GitLab allocates a set only when owners from multiple matching
sections must be combined.
- Parser-only section names, defaults, pattern keys, and rule lists are
released after parsing.
- The serialized `test.codeowners` value is created once per cached
ownership result.

## Test coverage

- Added specification-focused tests for GitHub and GitLab matching and
owner parsing.
- Added tests against this repository CODEOWNERS file and real-world
GitHub and GitLab examples.
- Added provider and file-location detection coverage for GitHub,
GitLab, self-managed GitLab, Jenkins, and Azure Pipelines inputs.
- Extended source-path fallback coverage for foreign relative prefixes,
repository-root files, duplicate separators, URIs, absolute paths,
navigation segments, and paths outside the repository.
- Added lifecycle and allocation-focused coverage for session-level
discovery, cached resolutions, parser-state release, and bounded
matching behavior.

## Other details

No public API or dependency changes.

* Expose the `TrailerSize` of an `ISpanBufferSerializer` (#9115)

## Summary of changes

Expose the `TrailerSize` the same way we do `HeaderSize`

## Reason for change

The `SpanBuffer` implementation can use this to ensure that the buffer
is sized appropriately. Without it, there's an edge case that requires
doubling the size of the buffer _after_ the traces have been written,
when we're about to flush

## Implementation details

- Add `ISpanBufferSerializer.TrailerSize`
- Implement everywhere (`0` for messagepack, JSON has constant value)
- Update `SpanBuffer` to take `TrailerSize` into account during resizes

## Test coverage

Meh, covered by the next PR

## Other details

Preparatory work for #9107

* Reduce locking in `AgentWriter` to avoid contention (#9107)

## Summary of changes

Update `AgentWriter` to reduce chance of locked buffers

## Reason for change

In #9007 (and related PRs) we updated the `AgentWriter` flush loop to
avoid simultaneous flushes. The intention was to avoid the "both buffers
locked" failure scenario we have seen in CI.

However, there were still several edge cases that could lead to the
serialization thread seeing _both_ buffers as locked _despite_ the fact
we only ever locked one at a time (during flushing).
The serialization thread can "chase" the lock from one buffer to the
other and
see `Locked` twice even though the two buffers _were never locked
simultaneously_. #9007 prevented _simultaneous_ locking, but it doesn't
prevent this scenario.

What's worse, we kind of cause it ourselves, because of how we call
`WakeFlushLoop` from the serialization thread. For example

1. Flush loop is running, and is locking the front buffer (currently
active buffer)
2. Serialization thread tries to write, but finds the buffer locked, so
it calls `SwapBuffers()` and `WakeFlushLoop`
3. Flush Loop finishes, doesn't sleep because of the trigger, and so
immediately locks the back buffer (the currently active buffer, again)
4. Serialization thread tries to write, finds the buffer locked again,
and drops the trace (two buffers locked)

There are various other factors that also make this worse:
- In `SpanBuffer.TryWrite` we call `Monitor.TryEnter` with _zero_
timeout. Even if the flush loop is locking for a tiny time (e.g. if
`TraceCount == 0`) the write will fail. In other words, we don't
_actually_ have to be flushing to report `Locked`
- `FlushBuffer` locks the _active_ buffer _at least_ once every second
(more when there's heavy load/contention/forced flushing).

There's also a latent bug where a locked-but-not-full buffer can end up
skipped and sitting in the background and not being flushed until
shutdown. I'll address that in a follow up PR

## Implementation details

The key fix I implemented in the end is to _not_ lock the buffer for the
whole duration of the flush. Instead, we use a third buffer that we
"swap in", so that we can reset the buffer, and keep the contention
period to a minimum.

_Technically_, this _should_ mean that we can now _block_ on
`SpanBuffer.TryWrite`, instead of trying to aquire the lock, and bailing
if we can't. Given that the critical section during a flush is now just
"write the header + trailer, then reset the buffer", this _should_ be
tiny. However, I was too much of a coward to go this route, and instead
opted for `Monitor.TryEnter` with a 100ms timeout. We _shouldn't_ hit it
(though I'm interested to see if we do in CI). And if we do, we decide
the best way to handle it, but I just wanted to avoid the chance of
stalling indefinitely, with the queue building up, and would rather lose
traces in that case.

One thing to note is that #9115 is required so that we don't
accidentally try to do allocation while holding the lock on the buffer,
as that could increase the duration we hold it for.

## Test coverage

Mostly just unit tests, as this is an internal refactoring. Hopefully it
reduces our instances of double-lock in CI

## Other details

Depends on https://github.com/DataDog/dd-trace-dotnet/pull/9115

* Fix lock-contention causing trace to be "stranded" (#9120)

## Summary of changes

- Always try to flush both buffers
- Skip the flush entirely if the buffer is empty
- Don't log drop stats twice

## Reason for change

Spotted some minor bugs while working on #9107. Notably

- If a buffer was _locked_ (not full), then the serialization thread
swaps the buffers, and triggers a flush. However, as the buffer is not
full, the flush thread would _not_ flush it automatically, so the trace
sits in the back buffer unflushed, until we swap buffers again
- We were trying to log the "drop" stats _twice_ per flush, in cases
where we flush both buffers. Not strictly a bug, but unnecessary.
- We were locking the buffer and swapping the array, even if there were
no traces in it
- Not strictly a bug, and this had much more potential to cause genuine
drops prior to #9107
  - Still seems like a worthwhile fix to me

## Implementation details

- Always flush both buffers, starting with the back buffer _regardless_
of whether it is full or not.
- Log the drop stats in `FlushBuffers` instead of the child
`FlushBuffer` calls
- Do a soft check on whether the buffer has stats before taking the lock
- Worse case, we get a `false` because the serialization thread is
actively writing into it, but there's naturally a race here anyway, and
we'll catch it next time around.

## Test coverage

Added a `[TestingOnly]` testing seam so that we can explicitly emulate
the scenario where the buffer was locked when we tried to write to it.
Allowed red-green testing to prove the edge case was there.

## Other details

Stacked on
- https://github.com/DataDog/dd-trace-dotnet/pull/9115
- https://github.com/DataDog/dd-trace-dotnet/pull/9107

* Consolidate and fix bugs in GitLab "download from Azure" scripts (#9103)

## Summary of changes

- Consolidates the 3 "wait for build and download artifacts from Azure"
scripts into one helper
- Fixes several minor bugs in the download logic

## Reason for change

As part of #9083, @NachoEchevarria flagged some potential improvements
to the scripts, for example a cancelled build today would cause
flakiness for the download scripts. We now also have 3 copies of
essentially the same code, and AI spotted several other bugs, all fixed
here

## Implementation details

- Extracted shared logic into a helper bash script
- Arguably we could/should make this a Nuke target given the bash
complexity, but given this all goes away completely when we move to
GitLab fully, I think the current approach is fine. It's largely
copy+paste with some generalisation.
- Fixed missing diagnostics when the loop timed out.
- Fixed an unnecessary 100s wait on each _successful_ run
- Fixed a cancelled build causing the jobs to sit waiting for artifacts
that will never run, before eventually timing out.
- Added retries for cancelled/failed builds, looking for "better"
candidate builds
- Can happen if there are multiple builds running against the same
commit (e.g. manual builds)
  - Only checks a given build once, so bounded

## Test coverage

This is the test

## Other details

Note that the `.gitlab/download-serverless-artifacts.sh` and
`.gitlab/download-single-step-artifacts.sh` have different paths they
take when running against tags (during release).
`.gitlab/download-nuget-packages-to-sign.sh` does not have this path, as
it does not need to run on tags (there may be an edge case there where
we _are_ running - need to check)

* [Test Package Versions Bump] Updating package versions (#9124)

Updates the package versions for integration tests. See below for a
summary of the changes.

## Package Version Updates

| Package | Integration | Previous | New | Published |
|---------|-------------|----------|-----|-----------|
| AWSSDK.Kinesis | AwsKinesis | 4.0.100.10 | 4.0.101 | 2026-08-21 |
| StackExchange.Redis | StackExchangeRedis | 3.1.13 | 3.1.31 |
2026-08-21 |
| log4net | log4net | 3.3.2 | 3.4.0 | 2026-08-21 |
| OpenTelemetry.Api | OpenTelemetry | 1.17.0 | 1.18.0 | 2026-08-21 |

## New Major Versions Available

These have a new major outside our supported range. Review whether to
widen the
integration's `MaximumVersion` and the definition's
`MaxVersionExclusive`/`SpecificVersions`,
or add a new split entry, to adopt.

| Package | Integration | Current cap | Latest available |
|---------|-------------|-------------|------------------|
| xunit.v3 | XUnitRetriesV3 | 3.2.2 | 4.0.0 |
| xunit.v3 | XUnitV3 | 3.2.2 | 4.0.0 |

## Package Version Cooldown Report

The following versions were published less than **2 days** ago and have
been ignored.
These require manual review before inclusion.

| Package | Integration | Version Kept | Ignored Version | Published |
Age (days) |

|---------|-------------|--------------|-----------------|-----------|------------|
| AWSSDK.Core | AwsSdk | 4.0.102 | 4.0.102.1 | 2026-08-24 | 1 |
| AWSSDK.DynamoDBv2 | AwsDynamoDb | 4.0.103.4 | 4.0.103.5 | 2026-08-24 |
1 |
| AWSSDK.Kinesis | AwsKinesis | 4.0.101 | 4.0.101.1 | 2026-08-24 | 1 |
| AWSSDK.SQS | AwsSqs | 4.0.100.10 | 4.0.100.11 | 2026-08-24 | 1 |
| AWSSDK.SimpleNotificationService | AwsSns | 4.0.100.10 | 4.0.100.11 |
2026-08-24 | 1 |
| AWSSDK.EventBridge | AwsEventBridge | 4.0.100.10 | 4.0.100.11 |
2026-08-24 | 1 |
| AWSSDK.S3 | AwsS3 | 4.0.102.3 | 4.0.102.4 | 2026-08-24 | 1 |
| AWSSDK.StepFunctions | AwsStepFunctions | 4.0.100.10 | 4.0.100.11 |
2026-08-24 | 1 |

Co-authored-by: dd-octo-sts[bot] <200755185+dd-octo-sts[bot]@users.noreply.github.com>

* Bump test logger to latest (to try to fix codeowners) (#9128)

## Summary of changes

Bump the version of the testlogger

## Reason for change

It includes the latest codeowners code

## Implementation details

Bump the numbers

## Test coverage

This is the test

## Other details

Currently we're not reporting owners correctly

* Add `new Random()` to the banned API list, to encourage using those APIs (#9129)

## Summary of changes

Add `Random` to the banned API list

## Reason for change

We have `ThreadSafeRandom` which should generally be preferred

## Implementation details

- Added `Random()` constructors to the banned API lists
- I _didn't_ add `Random.Shared` (.NET 6+ only) seeing as
`ThreadSafeRandom.Shared` just delegates to this directly, so if you're
writing .NET 6+ code, using `Random.Shared` is fine.

## Test coverage

This is the test

## Other details

Spotted as part of reviewing #9042

* Try to make it so that CODEOWNERs triggers tests (#9092)

## Summary of changes

Tests rely on CODEOWNERS so if we change it we should at least run
normal integration tests.

## Reason for change

I broke tests in https://github.com/DataDog/dd-trace-dotnet/pull/9084 as
they didn't run

## Implementation details

Just went through and expanded `.github/` 

Granted that does mean that _new_ files we add there (for some reason)
will need to be added, but all that happens is CI will just run if that
is forgotten so I think that is a reasonable tradeoff.

## Test coverage

😃 

## Other details
<!-- Fixes #{issue} -->


<!--  ⚠️ Note:

Where possible, please obtain 2 approvals prior to merging. Unless
CODEOWNERS specifies otherwise, for external teams it is typically best
to have one review from a team member, and one review from apm-dotnet.
Trivial changes do not require 2 reviews.

MergeQueue is NOT enabled in this repository. If you have write access
to the repo, the PR has 1-2 approvals (see above), and all of the
required checks have passed, you can use the Squash and Merge button to
merge the PR. If you don't have write access, or you need help, reach
out in the #apm-dotnet channel in Slack.
-->

* [CI Visibility] Keep CODEOWNERS matches inside repository (#9127)

## Summary of changes

- Resolve CODEOWNERS from the Git repository root instead of accepting a
nested file first.
- Do not assign CODEOWNERS to source paths that cannot be resolved
inside the repository.
- Resolve absolute or relative PDB source paths recorded in a different
build environment.
- Preserve the source-file fallback tag when a compiler path cannot be
resolved.

## Reason for change

The CODEOWNERS implementation added in #9099 could select a nested
CODEOWNERS file before reaching the repository root. It could also
evaluate unresolved source text against repository rules, producing
incorrect owners.

PDB paths may come from a different machine or operating system than the
test runtime. Requiring those original paths to be valid locally drops
legitimate CODEOWNERS metadata, such as Windows build paths consumed by
Linux tests.

## Implementation details

- Find the nearest Git root before selecting a CODEOWNERS location.
- Fall back to the explicit workspace or source root when Git metadata
is unavailable.
- Re-anchor compiler paths by the longest existing suffix under the
repository root, including Windows, Unix, and file URI paths.
- Evaluate rules only after the source path has been resolved inside the
CODEOWNERS repository root.
- Reject non-file URIs, embedded rooted paths after parent traversal,
and suffixes that do not exist in the repository.

## Test coverage

- CodeOwnersFallbackTests: 34 passed.
- Added coverage for nested CODEOWNERS files, unresolved or external
paths, and cross-OS PDB paths.
- Includes the Azure layout from the CI regression: D:/a/_work/1/s/....

## Other details

- Companion logger port:
https://github.com/tonyredondo/datadog-test-logger/pull/37
- No public API or dependency changes.

* feat(feature-flags): add agentless configuration keys, settings, and endpoint derivation (#9040)

## Summary of changes

Adds the foundational configuration layer for agentless Feature Flags
delivery: new configuration keys, a `FeatureFlagsSettings` class with
source resolution and validation, and an `AgentlessEndpoint` struct for
deriving the CDN URL from the Datadog site or a custom base URL.

This is PR 1 of a stacked PR series implementing agentless Feature Flags
configuration delivery (FFL-2703), porting functionality from
[dd-trace-py#19331](https://github.com/DataDog/dd-trace-py/pull/19331)
and
[dd-trace-java#11892](https://github.com/DataDog/dd-trace-java/pull/11892).

## Reason for change

The .NET tracer currently delivers Feature Flags configuration
exclusively through the Datadog Agent's Remote Configuration. To support
agentless (CDN-backed) delivery, we need a configuration layer that:

- Selects between `agentless` (new default), `remote_config`, and
`disabled` sources
- Derives the managed CDN endpoint from `DD_SITE` or accepts a custom
`DD_FEATURE_FLAGS_CONFIGURATION_SOURCE_AGENTLESS_BASE_URL`
- Validates poll interval and request timeout settings
- Grandfathers existing `DD_EXPERIMENTAL_FLAGGING_PROVIDER_ENABLED`
users onto Remote Configuration

## Implementation details

- **`FeatureFlagsSource` enum** — `Disabled`, `Agentless`,
`RemoteConfig`
- **`FeatureFlagsSettings`** — reads all new env vars, resolves the
source with the cross-SDK precedence contract (kill switch → explicit
source → fail-closed → legacy grandfathering → default agentless),
validates poll interval (1–3600s) and request timeout (>0s)
- **`AgentlessEndpoint`** — derives
`https://ufc-server.ff-cdn.<site>/api/v2/feature-flagging/config/rules-based/server`
from the site, appends `dd_env` when configured, accepts custom
HTTP/HTTPS URLs, and never echoes URLs in error messages (credentials
safety)
- **New config keys** in `supported-configurations.yaml` + generated
`ConfigurationKeys.FeatureFlags.g.cs`:
- `DD_FEATURE_FLAGS_ENABLED` (default `true`, supersedes
`DD_EXPERIMENTAL_FLAGGING_PROVIDER_ENABLED`)
  - `DD_FEATURE_FLAGS_CONFIGURATION_SOURCE` (default `agentless`)
  - `DD_FEATURE_FLAGS_CONFIGURATION_SOURCE_AGENTLESS_BASE_URL`
-
`DD_FEATURE_FLAGS_CONFIGURATION_SOURCE_AGENTLESS_POLL_INTERVAL_SECONDS`
(default 30, type `int`)
-
`DD_FEATURE_FLAGS_CONFIGURATION_SOURCE_AGENTLESS_REQUEST_TIMEOUT_SECONDS`
(default 5, type `int`)
- `DD_EXPERIMENTAL_FLAGGING_PROVIDER_INITIALIZATION_TIMEOUT_MS` (default
10000, implementation B)
- **`[Obsolete]`** on `FlaggingProviderEnabled` with pragma suppression
in `FeatureFlagsSettings`

This PR does not wire anything to `FeatureFlagsModule` yet — that
happens in a later PR in the stack.

## Test coverage

- `FeatureFlagsSettingsTests` — 43 test cases covering source resolution
(kill switch, explicit source, legacy grandfathering, `offline`
sentinel, invalid values), blank/whitespace handling, casing
normalization, defaults, poll interval validation, request timeout
validation, initialization timeout, and base URL reading
- `AgentlessEndpointTests` — 18 test cases covering managed endpoint
derivation (site lowercasing, staging, govcloud), `dd_env` query
parameter (addition, null, escaping), custom endpoint path handling
(origin-only gets canonical path, custom path used verbatim), HTTP
acceptance for custom endpoints, invalid URL rejection, empty/whitespace
site rejection, and credentials-in-error safety

## Other details
<!-- Fixes #{issue} -->

Stacked PRs:
- PR 1 (this): Configuration keys, settings, and endpoint derivation
- PR 2: UFC parser and agentless poller
- PR 3: Module wiring and OpenFeature activation
- PR 4: Manual API activation


<!--  ⚠️ Note:

Where possible, please obtain 2 approvals prior to merging. Unless
CODEOWNERS specifies otherwise, for external teams it is typically best
to have one review from a team member, and one review from apm-dotnet.
Trivial changes do not require 2 reviews.

MergeQueue is NOT enabled in this repository. If you have write access
to the repo, the PR has 1-2 approvals (see above), and all of the
required checks have passed, you can use the Squash and Merge button to
merge the PR. If you don't have write access, or you need help, reach
out in the #apm-dotnet channel in Slack.
-->

System tests:
[DataDog/system-tests#7496](https://github.com/DataDog/system-tests/pull/7496)

* Bump SLO for throughput tests (#9131)

## Summary of changes

Bumps the SLO for throughput tests

## Reason for change

They're occasionally touching the threshold

## Implementation details

Told the AI to bump them. It did what it did

## Test coverage

Meh, if this passes, it'll do

## Other details

Fixes https://datadoghq.atlassian.net/browse/SRTEE-7

* [Smoke Test Docker Image Bump] Updating docker image tags (#9134)

Updates the docker images used for smoke tests.

Co-authored-by: dd-octo-sts[bot] <200755185+dd-octo-sts[bot]@users.noreply.github.com>

* [Profiler] Fix crashes at shutdown (#9050)

* Add many timeouts and retries to build stages in Azure DevOps (#9141)

## Summary of changes

Add explicit timeouts + retries for stages that didn't have them

## Reason for change

We [recently had a
build](https://dev.azure.com/datadoghq/a51c4863-3eb4-4c5d-878a-58b41a049e4e/_build/results?buildId=207980)
where the unit tests stage failed because the _build_ part timed out.
The build was absolutely fine, there was just _Something_ that caused
the nuke call to hang, which eventually hit the stage timeout, and
caused everything to failure. With these extra timeouts, the build steps
should hit their "local" timeouts, and get retried, allowing the stage
to ultimately pass.

This only adds new retries to "build" stages, not to "test" stages. 

In some cases, also extended the stage timeout to accommodate the new
retries - it's better for a job to take a long time than for it to fail
completely.

## Implementation details

Asked the API to grab the stats for each job in each stage, focus on the
"build" jobs, and suggest new timeouts. The timeouts are intentionally
conservative (a slow, no-retry complete is almost always preferable to a
retry that forces re-doing work).

## Test coverage

This is the test, but also, we'll see how it goes

## Other details

#incident-57303

<details><summary>The stats the AI produced. Obvs, take with a pinch of
salt, but they </summary>
<p>


| Stage | Step / command | Samples | Mean | P95 | Max | Applied timeout
| Applied retry |
| ---------------------------------- |
---------------------------------------------------------------------------------------------------------------------------------------------
| ------: | ----: | ----: | ------: | --------------: | ----------------
|
| build_windows_tracer | Build tracer home | 25 | 13.50 | 14.32 | 14.61
| 30 | 1 |
| build_windows_profiler | Build profiler home | 25 | 8.33 | 9.15 | 9.38
| 20 | 1 |
| build_linux_tracer | Clean CompileManagedLoader (docker) | 50 | 0.48 |
0.55 | 0.60 | 10 | 1 |
| build_linux_tracer | BuildManagedTracerHome ExtractDebugInfoLinux
ValidateNativeTracerGlibcCompatibility (docker) | 50 | 5.16 | 5.51 |
5.52 | 20 | 1 |
| build_linux_tracer_r2r | Clean CompileManagedLoader (docker) | 25 |
0.49 | 0.77 | 0.82 | 10 | 1 |
| build_linux_tracer_r2r | BuildManagedTracerHomeR2R
ExtractDebugInfoLinux (docker) | 25 | 5.68 | 6.15 | 6.17 | 20 | 1 |
| build_linux_universal | Clean BuildNativeLoader BuildNativeWrapper
ExtractDebugInfoLinux ValidateNativeLoaderSnapshotTestsLinux (docker) |
25 | 2.29 | 2.54 | 2.64 | **10** | 1 |
| build_arm64_tracer | Clean CompileManagedLoader BuildNativeTracerHome
BuildManagedTracerHome ExtractDebugInfoLinux
ValidateNativeTracerGlibcCompatibility (docker) | 50 | 8.62 | 9.59 |
9.80 | 20 | 1 |
| build_arm64_tracer_r2r | same, R2R variant (docker) | 25 | 9.39 | 9.97
| 10.41 | 20 | 1 |
| build_arm64_profiler | Clean BuildProfilerHome ExtractDebugInfoLinux
ValidateNativeProfilerGlibcCompatibility (docker) | 50 | 5.49 | 6.21 |
6.93 | 15 | 1 |
| build_arm64_universal | Clean BuildNativeLoader BuildNativeWrapper ...
(docker) | 25 | 1.47 | 1.69 | 1.70 | **10** | 1 |
| build_macos | Build native-tracer | 25 | 10.16 | 14.48 | 16.39 | 35 |
1 |
| build_macos | Build managed tracer + native loader | 25 | 12.72 |
19.46 | 20.76 | **30** | 1 |
| package_windows | Build MSI, Tracer home, FleetInstaller, and
TelemetryForwarder | 25 | 1.38 | 1.62 | 1.66 | **10** | 1 |
| build_dd_dotnet_windows | Build dd-dotnet | 25 | 1.30 | 1.54 | 1.70 |
10 | 1 |
| build_dd_dotnet_linux | Clean BuildDdDotnet --framework net8.0
(docker) | 50 | 1.26 | 1.37 | 1.59 | 10 | 1 |
| build_dd_dotnet_linux_arm64 | Clean BuildDdDotnet --framework TFM
(docker) | 50 | 0.95 | 1.06 | 1.08 | 10 | 1 |
| debug_builds | Clean CompileManagedLoader -buildConfiguration Debug
(docker) | 50 | 0.48 | 0.60 | 0.62 | 10 | 1 |
| debug_builds | BuildNativeTracerHome -buildConfiguration Debug
(docker) | 50 | 3.75 | 4.13 | 4.51 | 15 | 1 |
| debug_builds | BuildManagedTracerHome ExtractDebugInfoLinux
-buildConfiguration Debug (docker) | 50 | 5.18 | 5.60 | 6.21 | 20 | 1 |
| debug_builds | Build tracer home in debug | 25 | 6.27 | 7.00 | 7.05 |
30 | 1 |
| debug_builds | Clean Restore CreateRequiredDirectories
CompileManagedSrc BuildRunnerTool -buildConfiguration Debug (docker) |
25 | 4.97 | 5.57 | 5.77 | 30 | 1 |
| debug_builds | Build tracer home | 25 | 8.87 | 13.42 | 15.97 | 30 | 1
|
| unit_tests_windows | Build unit tests | 225 | 6.59 | 7.02 | 7.29 |
**15** | 3 (pre-existing) |
| unit_tests_macos | Build unit tests | 200 | 9.42 | 14.13 | 17.77 | 35
| 3 (pre-existing) |
| unit_tests_linux | BuildManagedUnitTests --framework TFM
--code-coverage-enabled BOOL (docker) | 400 | 5.92 | 6.26 | 57.72 ⚠ | 15
| 3 (pre-existing) |
| unit_tests_arm64 | same (docker) | 300 | 5.49 | 6.16 | 6.69 | 15 | 3
(pre-existing) |
| build_samples | Build sample projects | 275 | 5.19 | 7.52 | 8.11 |
**15** | 1 |
| build_samples_macos | Build sample projects | 25 | 10.43 | 18.17 |
20.43 | **40** | 1 |
| integration_tests_windows | Build integration tests | 900 | 2.64 |
3.36 | 4.32 | 10 | 3 (pre-existing) |
| integration_tests_windows_debugger | Build integration tests | 1600 |
1.53 | 1.92 | 2.91 | 10 | 3 (pre-existing) |
| integration_tests_windows_iis | Build Integration Tests | 100 | 2.36 |
2.94 | 3.44 | 10 | 3 (pre-existing) |
| msi_integration_tests_windows | BuildAspNetIntegrationTests | 50 |
2.04 | 2.66 | 3.14 | 10 | 3 (pre-existing) |
| msi_integration_tests_windows | BuildIntegrationTests | 25 | 1.65 |
2.02 | 2.05 | 10 | 3 (pre-existing) |
| integration_tests_linux | BuildIntegrationTests CompileTrimmingSamples
... (docker) | 1600 | 2.74 | 3.10 | 5.52 | 15 | 3 (pre-existing) |
| integration_tests_linux | BuildIntegrationTests --framework TFM
--SampleName Lambda (docker) | 250 | 1.40 | 2.07 | 2.43 | 15 | 3
(pre-existing) |
| integration_tests_arm64 | BuildIntegrationTests CompileTrimmingSamples
... (docker) | 300 | 2.34 | 2.60 | 3.63 | 15 | 3 (pre-existing) |
| integration_tests_arm64 | BuildIntegrationTests --framework TFM
--IncludeTestsRequiringDocker BOOL --SampleName (docker) | 300 | 1.61 |
1.73 | 2.07 | 15 | 3 (pre-existing) |
| integration_tests_linux_debugger | BuildDebuggerIntegrationTests ...
(docker) | 800 | 1.80 | 2.14 | 2.70 | 15 | 3 (pre-existing) |
| integration_tests_arm64_debugger | BuildDebuggerIntegrationTests ...
(docker) | 600 | 1.50 | 1.67 | 2.07 | 15 | 1 (was missing) |
| profiler_integration_tests_linux | BuildProfilerSamples (docker) | 50
| 1.24 | 1.54 | 1.80 | 10 | 3 (pre-existing) |
| profiler_integration_tests_arm64 | BuildProfilerSamples (docker) | 50
| 0.87 | 0.95 | 0.97 | 10 | 3 (pre-existing) |
| asan_profiler_tests | BuildProfilerAsanTest -Framework net7.0 (docker)
| 25 | 18.00 | 19.05 | 19.11 | **30** | 1 (was missing) |
| asan_profiler_tests | BuildProfilerSampleForSanitiserTests (docker) |
25 | 0.52 | 0.61 | 0.65 | **10** | 1 (was missing) |
| asan_arm64_profiler_tests | BuildProfilerAsanTest (docker) | 25 |
10.21 | 11.02 | 11.38 | **30** | 1 (was missing) |
| asan_arm64_profiler_tests | BuildProfilerSampleForSanitiserTests
(docker) | 25 | 0.25 | 0.31 | 0.34 | **10** | 1 (was missing) |
| ubsan_profiler_tests | BuildProfilerUbsanTest (docker) | 25 | 18.10 |
19.20 | 19.49 | **30** | 1 (was missing) |
| ubsan_profiler_tests | BuildProfilerSampleForSanitiserTests (docker) |
25 | 0.51 | 0.62 | 0.65 | **10** | 1 (was missing) |
| ubsan_arm64_profiler_tests | BuildProfilerUbsanTest (docker) | 25 |
10.39 | 11.27 | 11.81 | **30** | 1 (was missing) |
| ubsan_arm64_profiler_tests | BuildProfilerSampleForSanitiserTests
(docker) | 25 | 0.25 | 0.31 | 0.32 | **10** | 1 (was missing) |
| dotnet_tool | Build tools and pack NuGet packages (Bundle,
AzureFunctions, Benchmark) | 25 | 8.64 | 9.26 | 9.92 | 20 | 1 (was
missing) |




</p>
</details>

* [CI Visibility] Stabilize CODEOWNERS concurrency test (#9143)

## Summary of changes

- Run the CODEOWNERS concurrency check on four dedicated threads.
- Start all workers together while keeping the existing timeout and
ownership assertions.

## Reason for change

The test used `Task.Run` and then blocked its xUnit worker with
`Task.WaitAll`. On a busy CI agent, the shared ThreadPool could not
schedule all 64 tasks before the timeout, so the test reported a
CODEOWNERS deadlock even though matching had completed normally. This
failed in [Azure build
208058](https://dev.azure.com/datadoghq/dd-trace-dotnet/_build/results?buildId=208058).

## Implementation details

`TaskCreationOptions.LongRunning` gives each of the four workers a
dedicated thread, and a start signal makes the matching calls
concurrent. The test no longer depends on ThreadPool capacity shared
with the rest of the test suite.

## Test coverage

- `CodeOwnersSpecTests`: 92 passed.
- The affected test passed 10 consecutive runs (approximately 27-30 ms
each).

## Other details

Test-only change; no production behavior or public API changes.

#incident-57303

* Move CI uploads to new subscription (#9142)

## Summary of changes

Upload/download build artifacts to new Azure subscription

## Reason for change

We are moving uploads to a new location for technical reasons. We'll
need to update all downstream consumers to pull from the new location.
We already have some build dependencies stored here, so we're just
expanding that for now (until we move to gitlab)

## Implementation details

- Dual-upload to the old and new location, until we can update all the
consumers to pull from the new place.
- Extracted common code to reusable action for now
- We can inline this again once we have all the consumers of the old
location migrated

## Test coverage

Did a run
[here](https://dev.azure.com/datadoghq/dd-trace-dotnet/_build/results?buildId=208065&view=results)
that forces the dual upload location, verified it exists

## Other details

AFAICT, this covers everything using the "old" subscription in this
repo. Next step is to update consumers to pull from the new location
instead

* feat(openfeature): support arbitrary semver parts (#9122)

## Summary
- refresh FFE fixtures from 2c0c8d55f665fe4a847fc0ded3abf59b7e2896fc
- allow any number of numeric SemVer core components
- cover six-part and longer core version comparisons

## Testing
- `dotnet test
tracer/test/Datadog.Trace.Tests/Datadog.Trace.Tests.csproj --no-restore
--framework net10.0 --filter FullyQualifiedName~SemanticVersionTests
--logger "console;verbosity=minimal"`
- `dotnet test
tracer/test/Datadog.Trace.Tests/Datadog.Trace.Tests.csproj --no-restore
--framework net10.0 --filter
FullyQualifiedName~FeatureFlagsEvaluatorTests.BundledTest --logger
"console;verbosity=minimal"`

* feat(feature-flags): add UFC parser and agentless HTTP poller (#9042)

## Summary of changes

Adds the agentless delivery mechanism: a `UfcConfigurationParser` that
validates the JSON:API envelope returned by the CDN endpoint, and an
`AgentlessConfigurationSource` HTTP poller with retries, ETag caching,
gzip decompression, and shutdown safety.

This is PR 2 of a stacked PR series implementing agentless Feature Flags
configuration delivery (FFL-2703), porting functionality from
[dd-trace-py#19331](https://github.com/DataDog/dd-trace-py/pull/19331)
and
[dd-trace-java#11892](https://github.com/DataDog/dd-trace-java/pull/11892).
Stacked on [PR 1
#9040](https://github.com/DataDog/dd-trace-dotnet/pull/9040).

## Reason for change

PR 1 added the configuration layer (settings, endpoint derivation). This
PR adds the actual HTTP delivery mechanism that polls the CDN endpoint
for UFC flag configuration. It is self-contained: the poller and parser
are fully testable in isolation and are not yet wired to the
`FeatureFlagsModule` (that happens in PR 3).

## Implementation details

- **`UfcConfigurationParser`** — validates the JSON:API Universal Flag
Configuration response envelope, extracts `data.attributes` into
`ServerConfiguration`. Rejects raw UFC documents (without the JSON:API
wrapper) so all delivery sources agree on one wire format. Returns
descriptive errors; never throws.
- **`AgentlessConfigurationSource`** — HTTP poller with:
  - Fixed-delay polling (no overlap)
- In-tick retries (max 3) for 408/429/5xx/network failures, with
jittered backoff derived from the poll interval (`clamp(P/6, 2s, 10s)`
then `clamp(P/3, 5s, 30s)`)
- ETag / `If-None-Match` conditional polling; ETag advances only after
parse + apply succeed
  - Gzip decompression (neither transport decompresses automatically)
  - Self-tracing suppression (`x-datadog-tracing-enabled: false` header)
- API key sent only to managed (Datadog-derived) endpoints, never custom
ones
- Warn-once per failure category (authentication, http, request) to
prevent log flooding
- Shutdown safety: `CancellationTokenSource` cancels the loop; shutdown
checks after request and before `ApplyAsync` prevent use-after-dispose
- `Create()` factory returns `null` when endpoint can'"'t be built or
API key is missing
  - Idempotent `Start()` via `Interlocked.CompareExchange`

## Test coverage

- **`UfcConfigurationParserTests`** (16 tests) — valid envelope parsing,
malformed JSON rejection, invalid envelope rejection (wrong type,
missing data, raw UFC), invalid attributes rejection (missing
format/createdAt/environment/flags, wrong types), flag parsing
- **`AgentlessConfigurationSourceTests`** (12 tests) — 200 success, ETag
forwarding, 304/401/malformed handling, disposal before/after request,
retry on 500 then success, max retries on 500, no retry on 400, gzip
response, network error, idempotent start

## Other details
<!-- Fixes #{issue} -->

Stacked PRs:
- PR 1 [#9040](https://github.com/DataDog/dd-trace-dotnet/pull/9040):
Configuration keys, settings, and endpoint derivation
- PR 2 (this): UFC parser and agentless HTTP poller
- PR 3: Module wiring and OpenFeature activation
- PR 4: Manual API activation


<!--  ⚠️ Note:

Where possible, please obtain 2 approvals prior to merging. Unless
CODEOWNERS specifies otherwise, for external teams it is typically best
to have one review from a team member, and one review from apm-dotnet.
Trivial changes do not require 2 reviews.

MergeQueue is NOT enabled in this repository. If you have write access
to the repo, the PR has 1-2 approvals (see above), and all of the
required checks have passed, you can use the Squash and Merge button to
merge the PR. If you don't have write access, or you need help, reach
out in the #apm-dotnet channel in Slack.
-->

System tests:
[DataDog/system-tests#7496](https://github.com/DataDog/system-tests/pull/7496)

* [Profiler] Fix flacky tests (#9126)

## Summary of changes
- retry on ARM64 when possible truncated call stacks
- don't assume all events for class histograms are received compared to
reference chains

## Reason for change
Flack tests

## Implementation details
- add retry mechanism on ARM64 + invalid frame
- remove to strict comparison between reference chain and class
histogram

## Test coverage
Corresponding tests have been updated

## Other details
<!-- Fixes #{issue} -->

#incident-57303
<!--  ⚠️ Note:

Where possible, please obtain 2 approvals prior to merging. Unless
CODEOWNERS specifies otherwise, for external teams it is typically best
to have one review from a team member, and one review from apm-dotnet.
Trivial changes do not require 2 reviews.

MergeQueue is NOT enabled in this repository. If you have write access
to the repo, the PR has 1-2 approvals (see above), and all of the
required checks have passed, you can use the Squash and Merge button to
merge the PR. If you don't have write access, or you need help, reach
out in the #apm-dotnet channel in Slack.
-->

* [Profiler] Fix start/stop crash at shutdown (#9037)

* [Profiler] ManagedCodeCache updates synchronously (#8840)

* [CI Visibility] Resolve CODEOWNERS from matching local checkout (#9148)

## Summary of changes

- Use the current local Git checkout as a CODEOWNERS fallback when the
CI workspace paths are not available in the test process.
- Trust that fallback only when the local Git commit exactly matches the
commit reported by CI.

## Reason for change

Distributed CI jobs can build and run tests on different machines and
filesystems. Our benchmark job reports a Linux CI workspace and a GitLab
mirror URL, while BenchmarkDotNet executes in a Windows checkout cloned
from GitHub. The resolver therefore cannot reach the reported workspace
and selects the GitLab CODEOWNERS layout instead of the
.github/CODEOWNERS file present in the worker checkout. The benchmark
source paths are valid, but their test events receive no owners.

## Implementation details

- Keep the GitInfo instance that is already collected during CI
environment initialization.
- Compare its commit with the CI commit once when the test session
starts.
- When they match, pass the local repository root and URL to CODEOWNERS
discovery.
- Search the local Git root only after the CI source and workspace roots
fail.
- Use the local repository URL to select the matching CODEOWNERS dialect
and location.

This adds no extra Git command and no per-test allocation or filesystem
scan. CODEOWNERS is still located and loaded once per test session.

## Test coverage

- Added a regression test for a GitLab mirror with an unavailable CI
workspace, a GitHub worker checkout, and a relocated Windows PDB source
path.
- Added a guard test proving that a local checkout at a different commit
is not trusted.
- All 179 CODEOWNERS tests pass on .NET 6 and .NET 8.

## Other details

No public API or dependency changes.

* [CI Visibility] Bump Datadog test packages to 0.0.59 (#9151)

## Summary of changes

- Bump `DatadogTestLogger` from `0.0.56` to `0.0.59`.
- Bump `DatadogTestCollector` from `0.0.56` to `0.0.59`.

## Reason for change

Consume the latest test logger release, including the CODEOWNERS
local-checkout fixes from
[datadog-test-logger#40](https://github.com/tonyredondo/datadog-test-logger/pull/40).

## Implementation details

- Update the shared tracer test package references.
- Update the profiler integration test logger reference.

## Test coverage

- Restored `Datadog.Trace.Tests` successfully and verified both packages
resolve to `0.0.59`.
- Restored `Datadog.Profiler.IntegrationTests` successfully and verified
`DatadogTestLogger` resolves to `0.0.59`.

## Other details

Dependency-only change.

* [Profiler] Fix bug in FrameStore (#9140)

* [Profiler] Fix flaky tests caused by failing`timer_create` (#8988)

* Add SDK capabilities and LP as owners in addition to tracing-dotnet (#9130)

## Summary of changes

Adds SDK Capabilities and Language Platform as owners for paths
previously owned solely by `tracing-dotnet`. Retains `tracing-dotnet` as
the review team.

## Reason for change

Green CI and other initiatives are trying to better define "owning"
teams and "reviewing" teams, this is a step toward that.

`tracing-dotnet` is intended to be a "reviewing" team.
More specific teams are intended to be "owning" teams.

We don't want this to be _super_ disruptive at the moment so trying to
implement this bit by bit, so for this case we are going from _pure_
`tracing-dotnet` to be `apm-sdk-capabilities-dotnet` &&
`apm-lang-platform-dotnet` && `tracing-dotnet`.

**Functionally**, there isn't a major change at the moment with this.

## Implementation details

Took lines that were ONLY owned by `tracing-dotnet` to be expanded into
the three.

## Test coverage

Well this may impact tests which should run now that
https://github.com/DataDog/dd-trace-dotnet/pull/9092 is merged 😄

## Other details
<!-- Fixes #{issue} -->


<!--  ⚠️ Note:

Where possible, please obtain 2 approvals prior to merging. Unless
CODEOWNERS specifies otherwise, for external teams it is typically best
to have one review from a team member, and one review from apm-dotnet.
Trivial changes do not require 2 reviews.

MergeQueue is NOT enabled in this repository. If you have write access
to the repo, the PR has 1-2 approvals (see above), and all of the
required checks have passed, you can use the Squash and Merge button to
merge the PR. If you don't have write access, or you need help, reach
out in the #apm-dotnet channel in Slack.
-->

* [Smoke Test Docker Image Bump] Updating docker image tags (#9153)

Updates the docker images used for smoke tests.

Co-authored-by: dd-octo-sts[bot] <200755185+dd-octo-sts[bot]@users.noreply.github.com>

* Increase profiler integration test timeout (#9158)

## Summary of changes

Increases profiler integration test timeout

## Reason for change

They timed out (but they all passed 🎉) but because they timed out they
failed 😢.

## Implementation details

60 -> 90

## Test coverage

nope

## Other details
<!-- Fixes #{issue} -->

#incident-57303

<!--  ⚠️ Note:

Where possible, please obtain 2 approvals prior to merging. Unless
CODEOWNERS specifies otherwise, for external teams it is typically best
to have one review from a team member, and one review from apm-dotnet.
Trivial changes do not require 2 reviews.

MergeQueue is NOT enabled in this repository. If you have write access
to the repo, the PR has 1-2 approvals (see above), and all of the
required checks have passed, you can use the Squash and Merge button to
merge the PR. If you don't have write access, or you need help, reach
out in the #apm-dotnet channel in Slack.
-->

Co-authored-by: Andrew Lock <andrew.lock@datadoghq.com>

* [Test Package Versions Bump] Updating package versions (#9152)

Updates the package versions for integration tests. See below for a
summary of the changes.

## Package Version Updates

| Package | Integration | Previous | New | Published |
|---------|-------------|----------|-----|-----------|
| Quartz | Quartz | 3.19.1 | 3.20.0 | 2026-08-27 |
| AWSSDK.Core | AwsSdk | 4.0.102 | 4.0.102.1 | 2026-08-24 |
| AWSSDK.DynamoDBv2 | AwsDynamoDb | 4.0.103.4 | 4.0.103.5 | 2026-08-24 |
| AWSSDK.Kinesis | AwsKinesis | 4.0.101 | 4.0.101.1 | 2026-08-24 |
| AWSSDK.SQS | AwsSqs | …
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants