diff --git a/dd-openfeature/README.md b/dd-openfeature/README.md index c0a59136a8f..e710c21e44f 100644 --- a/dd-openfeature/README.md +++ b/dd-openfeature/README.md @@ -52,6 +52,7 @@ Settings are read from the Datadog Java agent configuration when attached, then | `DD_FEATURE_FLAGS_CONFIGURATION_SOURCE_AGENTLESS_REQUEST_TIMEOUT_SECONDS` | `5` | CDN request timeout. | | `DD_FLAGGING_EVALUATION_COUNTS_ENABLED` | `true` | Sends aggregated flag evaluation counts. | | `DD_FEATURE_FLAGS_EXPOSURES_DATADOG_LOGGING_ENABLED` | `true` | Sends exposures to Datadog. See [Stop sending exposures to Datadog](#stop-sending-exposures-to-datadog). | +| `DD_FEATURE_FLAGS_EXPERIMENTATION_OPT_OUT_MODE` | `disabled` | Controls per-subject experimentation opt-out: `disabled`, `immediate`, or `future_experiments_only`. See [Per-subject experimentation opt-out](#per-subject-experimentation-opt-out). | | `DD_FEATURE_FLAGS_EVALUATION_TELEMETRY_PER_SUBJECT_OPT_OUT_ENABLED` | `false` | Lets the `dd_evaluation_telemetry_opt_out` context attribute opt subjects out of evaluation telemetry. See [Per-subject evaluation telemetry opt-out](#per-subject-evaluation-telemetry-opt-out). | | `DD_EXPERIMENTAL_FLAGGING_PROVIDER_SPAN_ENRICHMENT_ENABLED` | `false` | Adds flag evaluations to the local root span. Requires `dd-java-agent`. | | `DD_API_KEY`, `DD_SITE`, `DD_ENV`, `DD_SERVICE`, `DD_VERSION` | | Credentials and event context. | @@ -133,6 +134,92 @@ The provider then sends no exposures to Datadog. `ExposureHook` callbacks still This setting stops exposures only. Flag evaluation events have their own setting, `DD_FLAGGING_EVALUATION_COUNTS_ENABLED`. The Java provider does not read dd-trace-py's `DD_FFE_INTAKE_ENABLED`. +## Per-subject experimentation opt-out + +Set `DD_FEATURE_FLAGS_EXPERIMENTATION_OPT_OUT_MODE=immediate` before constructing the provider. +The default is `disabled`. Existing configuration delivery and credentials are still required. + +```java +import com.datadog.openfeature.Provider; +import dev.openfeature.sdk.MutableContext; +import dev.openfeature.sdk.OpenFeatureAPI; + +OpenFeatureAPI api = OpenFeatureAPI.getInstance(); +api.setProviderAndWait(new Provider()); +boolean enabled = api.getClient().getBooleanValue("new-checkout", false, + new MutableContext("subject-123") + .add("dd_experimentation_opt_out_at", "2026-10-19T12:00:00Z")); +``` + +In immediate mode, a valid timestamp string excludes the subject from allocations with `doLog=true`. +This includes completed-experiment winners that still log exposures; omitted, null, and false `doLog` retain Java's ordinary-allocation behavior. +Evaluation continues through ordinary feature allocations. If none match, the provider returns the application's default. +A future timestamp also takes effect immediately. It does not schedule exclusion. + +Omit the attribute for subjects who can participate. OpenFeature merges API, transaction, client, and invocation +contexts. Omitting it from an invocation does not remove an inherited value; remove it from the layer that owns it. +Setting it to null does not restore participation. + +Invalid, null, empty, or non-string values return the application's default with `INVALID_CONTEXT` and a diagnostic. +This validation runs after configuration readiness and before flag lookup, so it also applies to ordinary and missing +flags. Before configuration is ready the result remains `PROVIDER_NOT_READY`. Disabled mode ignores the attribute, +including invalid values. Supply `Instant.toString()` as a string; passing an `Instant` object is invalid. + +Java uses `DateTimeFormatter.ISO_OFFSET_DATE_TIME`, also used for allocation `startAt`, `endAt`, and activation. +It validates calendar dates, accepts up to nine fractional digits and offsets through ±18:00, and rejects leap +seconds, missing offsets, and surrounding whitespace. Its ISO syntax also accepts lowercase `t`/`z`, omitted +seconds, an empty fractional part, hour-only or second-bearing offsets, and signed extended years. This is Java's +existing grammar, not the browser SDK's timestamp grammar. + +An explicit option overrides configuration: + +```java +import com.datadog.openfeature.ExperimentationOptOutMode; + +Provider provider = new Provider(new Provider.Options() + .experimentationOptOutMode(ExperimentationOptOutMode.DISABLED)); +``` + +`DEFAULT` reads the normal SDK configuration sources, including +`-Ddd.feature.flags.experimentation.opt.out.mode=immediate`. +The provider resolves the mode at construction; later changes to options or configuration do not change it. +Unsupported values throw `IllegalArgumentException` when Feature Flags is enabled. A disabled product skips this +unused setting's validation and retains its existing initialization behavior. Passing null to the option throws +`NullPointerException`. +The deprecated `datadog.trace.api.openfeature.Provider` alias uses the same evaluator +and accepts the same `com.datadog.openfeature.ExperimentationOptOutMode` options. + +This setting is independent of evaluation telemetry opt-out. +Immediate withdrawal can change an enrolled subject's experience. Account for this change in experiment analysis. + +Use `future_experiments_only` or `ExperimentationOptOutMode.FUTURE_EXPERIMENTS_ONLY` to exclude newer experiments. +Supply the same stored `dd_experimentation_opt_out_at` timestamp with each evaluation. +For exposure-logging allocations, the provider compares `randomizationActivatedAt` with that timestamp: + +| Allocation activation | Behavior | +| --- | --- | +| Before the subject's opt-out time | Evaluate normally, including exposure logging | +| Equal to or after the opt-out time | Skip the allocation and continue ordinary delivery | +| Missing or null | Skip the allocation because its activation time is unknown | + +Ordinary allocations remain available. Missing subject attributes preserve normal evaluation. +Both enabled modes reject invalid subject timestamps. +Malformed activation metadata on any allocation, including an ordinary, unreachable, or disabled flag's allocation, +returns `PARSE_ERROR` for the affected flag only in future mode with a valid subject timestamp. +Disabled mode, immediate mode, and evaluations without a subject timestamp preserve ordinary evaluation. +Invalid regex patterns and unconvertible numeric variant values are configuration errors regardless of opt-out: +the parser rejects only the affected flag, including when the malformed allocation would be skipped. +Java's existing regex syntax and numeric coercions remain supported. +Comparisons retain nanosecond precision and account for timezone offsets. +The cutoff does not depend on the SDK clock. It is not a scheduled change or proof of prior enrollment. +An older allocation can still assign a subject for the first time. + +Enable future mode after the configuration service supplies stable activation timestamps and preserves ordinary fallback allocations. +Keep the subject's stored opt-out time stable across sessions and SDK instances. +The provider stores no consent history and emits no new event type. The opt-out attribute remains in the evaluation +context: existing exposure reports can include it, and evaluation events can include it when full evaluation data +reporting is enabled. Existing reporting controls and consent settings still apply. + ## Per-subject evaluation telemetry opt-out Applications can keep flag evaluation enabled while excluding selected subjects from evaluation diff --git a/dd-openfeature/build.gradle.kts b/dd-openfeature/build.gradle.kts index 3d9772ec925..c4898b7a5fc 100644 --- a/dd-openfeature/build.gradle.kts +++ b/dd-openfeature/build.gradle.kts @@ -110,3 +110,17 @@ val japicmpPublicApi = tasks.register("japicmpPublicApi") { tasks.named("check") { dependsOn(japicmpPublicApi) } + +// These suites read shared fixtures directly from the smoke-test resources. +tasks.withType().configureEach { + inputs.dir( + rootProject.layout.projectDirectory.dir( + "dd-smoke-tests/openfeature/src/test/resources/ffe-system-test-data" + ) + ).withPathSensitivity(PathSensitivity.RELATIVE) + inputs.dir( + rootProject.layout.projectDirectory.dir( + "dd-smoke-tests/openfeature/src/test/resources/experimentation-opt-out" + ) + ).withPathSensitivity(PathSensitivity.RELATIVE) +} diff --git a/dd-openfeature/src/jmh/java/com/datadog/openfeature/ExperimentationOptOutBenchmark.java b/dd-openfeature/src/jmh/java/com/datadog/openfeature/ExperimentationOptOutBenchmark.java new file mode 100644 index 00000000000..04bda2106da --- /dev/null +++ b/dd-openfeature/src/jmh/java/com/datadog/openfeature/ExperimentationOptOutBenchmark.java @@ -0,0 +1,117 @@ +package com.datadog.openfeature; + +import static java.util.Arrays.asList; +import static java.util.Collections.emptyList; +import static java.util.Collections.emptyMap; +import static java.util.Collections.singletonList; +import static java.util.Collections.singletonMap; +import static java.util.concurrent.TimeUnit.NANOSECONDS; +import static java.util.concurrent.TimeUnit.SECONDS; + +import com.datadog.openfeature.internal.bridge.v1.AgentBridge; +import com.datadog.openfeature.internal.config.Settings; +import com.datadog.openfeature.internal.ufc.Allocation; +import com.datadog.openfeature.internal.ufc.Flag; +import com.datadog.openfeature.internal.ufc.ServerConfiguration; +import com.datadog.openfeature.internal.ufc.Split; +import com.datadog.openfeature.internal.ufc.ValueType; +import com.datadog.openfeature.internal.ufc.Variant; +import dev.openfeature.sdk.ImmutableContext; +import dev.openfeature.sdk.LayeredEvaluationContext; +import dev.openfeature.sdk.MutableContext; +import dev.openfeature.sdk.ProviderEvaluation; +import java.util.HashMap; +import java.util.Map; +import org.openjdk.jmh.annotations.Benchmark; +import org.openjdk.jmh.annotations.BenchmarkMode; +import org.openjdk.jmh.annotations.Fork; +import org.openjdk.jmh.annotations.Measurement; +import org.openjdk.jmh.annotations.Mode; +import org.openjdk.jmh.annotations.OutputTimeUnit; +import org.openjdk.jmh.annotations.Param; +import org.openjdk.jmh.annotations.Scope; +import org.openjdk.jmh.annotations.Setup; +import org.openjdk.jmh.annotations.State; +import org.openjdk.jmh.annotations.TearDown; +import org.openjdk.jmh.annotations.Warmup; + +/** + * Measures provider resolution, including timestamp validation and ordinary allocation fallback. + */ +@State(Scope.Thread) +@Warmup(iterations = 3, time = 1, timeUnit = SECONDS) +@Measurement(iterations = 5, time = 1, timeUnit = SECONDS) +@BenchmarkMode(Mode.AverageTime) +@OutputTimeUnit(NANOSECONDS) +@Fork(1) +public class ExperimentationOptOutBenchmark { + @Param({"DISABLED", "IMMEDIATE", "FUTURE_EXPERIMENTS_ONLY"}) + public ExperimentationOptOutMode mode; + + @Param({"false", "true"}) + public boolean optedOut; + + @Param({"0", "16"}) + public int extraAttributes; + + private Provider provider; + private MutableContext context; + + @Setup + public void setUp() { + final DDEvaluator evaluator = + new DDEvaluator(() -> {}, AgentBridge.NONE, Settings.load(AgentBridge.NONE), mode); + final Map variants = new HashMap<>(); + variants.put("experiment", new Variant("experiment", false)); + variants.put("ordinary", new Variant("ordinary", true)); + final Flag flag = + new Flag( + "flag", + true, + ValueType.BOOLEAN, + variants, + asList(allocation("experiment", true), allocation("ordinary", false))); + evaluator.accept(new ServerConfiguration("", "", false, null, singletonMap("flag", flag))); + provider = + new Provider(new Provider.Options().experimentationOptOutMode(mode), evaluator, false); + context = new MutableContext("subject"); + for (int i = 0; i < extraAttributes; i++) { + context.add("attribute" + i, "value" + i); + } + if (optedOut) { + context.add("dd_experimentation_opt_out_at", "2026-10-19T12:00:00Z"); + } + final ProviderEvaluation result = evaluate(); + final boolean excluded = mode != ExperimentationOptOutMode.DISABLED && optedOut; + if (result.getErrorCode() != null + || result.getValue() != excluded + || !(excluded ? "ordinary" : "experiment") + .equals(result.getFlagMetadata().getString("allocationKey"))) { + throw new IllegalStateException("Benchmark did not resolve the expected allocation"); + } + } + + private static Allocation allocation(final String key, final boolean doLog) { + return new Allocation( + key, null, null, null, singletonList(new Split(emptyList(), key, emptyMap(), 1)), doLog); + } + + @Benchmark + public ProviderEvaluation evaluate() { + return provider.getBooleanEvaluation("flag", false, context); + } + + @Benchmark + public ProviderEvaluation evaluateFreshLayeredContext() { + return provider.getBooleanEvaluation( + "flag", + false, + new LayeredEvaluationContext( + ImmutableContext.EMPTY, ImmutableContext.EMPTY, null, context)); + } + + @TearDown + public void tearDown() { + provider.shutdown(); + } +} diff --git a/dd-openfeature/src/main/java/com/datadog/openfeature/DDEvaluator.java b/dd-openfeature/src/main/java/com/datadog/openfeature/DDEvaluator.java index 8c6ced76c65..da9f400e14b 100644 --- a/dd-openfeature/src/main/java/com/datadog/openfeature/DDEvaluator.java +++ b/dd-openfeature/src/main/java/com/datadog/openfeature/DDEvaluator.java @@ -84,12 +84,22 @@ class DDEvaluator implements Evaluator, Consumer { private volatile FeatureFlagsRuntime runtime; DDEvaluator(final Runnable configCallback, final AgentBridge bridge, final Settings settings) { + this(configCallback, bridge, settings, ExperimentationOptOutMode.DISABLED); + } + + DDEvaluator( + final Runnable configCallback, + final AgentBridge bridge, + final Settings settings, + final ExperimentationOptOutMode experimentationOptOutMode) { this.configCallback = configCallback; this.bridge = bridge; this.settings = settings; // When span enrichment is off, the __dd_* span-enrichment metadata is attached only to // evaluations of allocations logging exposures, for the exposure hook. - this.core = new FlagEvaluator(settings.isSpanEnrichmentEnabled(), this::isExposed); + this.core = + new FlagEvaluator( + settings.isSpanEnrichmentEnabled(), this::isExposed, experimentationOptOutMode); } @Override diff --git a/dd-openfeature/src/main/java/com/datadog/openfeature/ExperimentationOptOutMode.java b/dd-openfeature/src/main/java/com/datadog/openfeature/ExperimentationOptOutMode.java new file mode 100644 index 00000000000..6f22dbc858b --- /dev/null +++ b/dd-openfeature/src/main/java/com/datadog/openfeature/ExperimentationOptOutMode.java @@ -0,0 +1,25 @@ +package com.datadog.openfeature; + +/** + * Controls per-subject exclusion from experimentation. + * + * @see Provider.Options#experimentationOptOutMode(ExperimentationOptOutMode) + */ +public enum ExperimentationOptOutMode { + /** Reads DD_FEATURE_FLAGS_EXPERIMENTATION_OPT_OUT_MODE, which defaults to disabled. */ + DEFAULT, + /** Ignores dd_experimentation_opt_out_at, overriding the environment setting. */ + DISABLED, + /** + * Skips exposure-logging allocations when dd_experimentation_opt_out_at is a timestamp String + * accepted by {@link java.time.format.DateTimeFormatter#ISO_OFFSET_DATE_TIME}. Applies + * immediately, including to experiments already in progress. + */ + IMMEDIATE, + /** + * Skips exposure-logging allocations activated at or after dd_experimentation_opt_out_at. Unknown + * activation is skipped. Older allocations continue normally, including exposures. This is an + * allocation activation cutoff, not a record of prior subject enrollment. + */ + FUTURE_EXPERIMENTS_ONLY +} diff --git a/dd-openfeature/src/main/java/com/datadog/openfeature/Provider.java b/dd-openfeature/src/main/java/com/datadog/openfeature/Provider.java index 70ac3436b8c..543aa018663 100644 --- a/dd-openfeature/src/main/java/com/datadog/openfeature/Provider.java +++ b/dd-openfeature/src/main/java/com/datadog/openfeature/Provider.java @@ -1,5 +1,6 @@ package com.datadog.openfeature; +import static java.util.Objects.requireNonNull; import static java.util.concurrent.TimeUnit.SECONDS; import com.datadog.openfeature.internal.FeatureFlagsRuntime; @@ -7,6 +8,7 @@ import com.datadog.openfeature.internal.bridge.v1.AgentBridge; import com.datadog.openfeature.internal.bridge.v1.AgentBridges; import com.datadog.openfeature.internal.config.Settings; +import com.datadog.openfeature.internal.evaluation.ExperimentationOptOut; import com.datadog.openfeature.internal.flagevaluation.FlagEvaluationWriter; import dev.openfeature.sdk.ErrorCode; import dev.openfeature.sdk.EvaluationContext; @@ -23,6 +25,7 @@ import java.util.ArrayList; import java.util.Collections; import java.util.List; +import java.util.Locale; import java.util.concurrent.TimeUnit; import java.util.concurrent.atomic.AtomicReference; import org.slf4j.Logger; @@ -39,6 +42,7 @@ public class Provider extends EventProvider implements Metadata { private final Options options; private final AgentBridge bridge; private final Settings settings; + private final ExperimentationOptOutMode experimentationOptOutMode; private final AtomicReference initializationState = new AtomicReference<>(InitializationState.NOT_STARTED); private final FlagEvalMetrics flagEvalMetrics; @@ -79,6 +83,10 @@ public Provider(final Options options) { this.evaluator = evaluator; this.bridge = AgentBridges.detect(); this.settings = Settings.load(this.bridge); + this.experimentationOptOutMode = + this.settings.isEnabled() + ? ExperimentationOptOut.resolve(options.experimentationOptOutMode, this.settings) + : ExperimentationOptOutMode.DISABLED; FlagEvalMetrics metrics = null; FlagEvalMetricsHook hook = null; try { @@ -145,6 +153,10 @@ public Provider(final Options options) { } else { log.info("{} span enrichment disabled", METADATA); } + log.info( + "{} experimentation opt-out mode: {}", + METADATA, + this.experimentationOptOutMode.name().toLowerCase(Locale.ROOT)); } @Override @@ -281,7 +293,8 @@ private Evaluator buildEvaluator() { if (evaluator != null) { return evaluator; } - return new DDEvaluator(this::onConfigurationChange, this.bridge, this.settings); + return new DDEvaluator( + this::onConfigurationChange, this.bridge, this.settings, this.experimentationOptOutMode); } private FlagEvaluationWriter flagEvaluationPipeline() { @@ -376,6 +389,35 @@ public static class Options { private long timeout = 30; private TimeUnit unit = SECONDS; private Boolean evaluationTelemetryOptOutParameterEnabled; + private ExperimentationOptOutMode experimentationOptOutMode = ExperimentationOptOutMode.DEFAULT; + + /** + * Sets per-subject experimentation opt-out. {@link ExperimentationOptOutMode#DEFAULT} reads + * {@code DD_FEATURE_FLAGS_EXPERIMENTATION_OPT_OUT_MODE}; every other mode, including {@code + * DISABLED}, overrides configuration. The provider captures the mode at construction. + * Unsupported configured modes throw {@link IllegalArgumentException} unless Feature Flags is + * disabled. + * + *

Enabled modes read {@code dd_experimentation_opt_out_at} from the merged evaluation + * context. Supply a timestamp String accepted by {@link + * java.time.format.DateTimeFormatter#ISO_OFFSET_DATE_TIME}, for example {@code + * Instant.now().toString()}. An {@code Instant} object, null, or any other invalid value + * returns the application's default with {@code INVALID_CONTEXT} after configuration is ready, + * even for ordinary or missing flags. Omit the attribute to permit experimentation; disabled + * mode ignores it. + * + *

This controls experiment allocation selection, independently of evaluation telemetry + * opt-out. The attribute remains available for targeting and reporting under the existing + * exposure and evaluation-data settings. + * + * @param mode the mode; configure before constructing the provider. + * @return these options. + * @throws NullPointerException if mode is null. + */ + public Options experimentationOptOutMode(final ExperimentationOptOutMode mode) { + this.experimentationOptOutMode = requireNonNull(mode, "mode"); + return this; + } /** * Enables per-evaluation telemetry opt-out through the {@code dd_evaluation_telemetry_opt_out} diff --git a/dd-openfeature/src/main/java/com/datadog/openfeature/internal/config/Settings.java b/dd-openfeature/src/main/java/com/datadog/openfeature/internal/config/Settings.java index aa43f1ab4cf..9990e23b19e 100644 --- a/dd-openfeature/src/main/java/com/datadog/openfeature/internal/config/Settings.java +++ b/dd-openfeature/src/main/java/com/datadog/openfeature/internal/config/Settings.java @@ -38,6 +38,7 @@ public final class Settings { "feature.flags.exposures.datadog.logging.enabled"; static final String EVALUATION_TELEMETRY_PER_SUBJECT_OPT_OUT_ENABLED = "feature.flags.evaluation.telemetry.per.subject.opt.out.enabled"; + static final String EXPERIMENTATION_OPT_OUT_MODE = "feature.flags.experimentation.opt.out.mode"; static final String TRACE_DEBUG = "trace.debug"; static final String API_KEY = "api-key"; static final String SITE = "site"; @@ -175,6 +176,12 @@ public boolean isEvaluationTelemetryPerSubjectOptOutEnabled() { return Boolean.TRUE.equals(getBoolean(EVALUATION_TELEMETRY_PER_SUBJECT_OPT_OUT_ENABLED)); } + /** Returns the configured experimentation opt-out mode, disabled by default. */ + public String experimentationOptOutMode() { + final String mode = get(EXPERIMENTATION_OPT_OUT_MODE); + return mode == null ? "disabled" : mode; + } + /** * Tells whether {@code DD_TRACE_DEBUG} is on. It gates the diagnostics that log raw evaluation * contexts and values, on top of the application's logging configuration. diff --git a/dd-openfeature/src/main/java/com/datadog/openfeature/internal/evaluation/ExperimentationOptOut.java b/dd-openfeature/src/main/java/com/datadog/openfeature/internal/evaluation/ExperimentationOptOut.java new file mode 100644 index 00000000000..1423beabc91 --- /dev/null +++ b/dd-openfeature/src/main/java/com/datadog/openfeature/internal/evaluation/ExperimentationOptOut.java @@ -0,0 +1,98 @@ +package com.datadog.openfeature.internal.evaluation; + +import com.datadog.openfeature.ExperimentationOptOutMode; +import com.datadog.openfeature.internal.config.Settings; +import com.datadog.openfeature.internal.ufc.Allocation; +import dev.openfeature.sdk.EvaluationContext; +import dev.openfeature.sdk.Value; +import java.time.DateTimeException; +import java.time.Instant; +import java.time.format.DateTimeFormatter; +import java.util.List; +import java.util.Locale; +import java.util.Objects; + +/** Resolves opt-out policy, validates subject timestamps, and excludes experiment allocations. */ +public final class ExperimentationOptOut { + public static final String ATTRIBUTE = "dd_experimentation_opt_out_at"; + public static final String INVALID_TIMESTAMP = + ATTRIBUTE + + " must be an ISO offset date-time string; supply a valid timestamp to opt out," + + " or omit the attribute to permit experimentation"; + + private ExperimentationOptOut() {} + + /** Resolves the provider option before reading central configuration. */ + public static ExperimentationOptOutMode resolve( + final ExperimentationOptOutMode override, final Settings settings) { + Objects.requireNonNull(override, "experimentationOptOutMode"); + if (override != ExperimentationOptOutMode.DEFAULT) { + return override; + } + final String configured = settings.experimentationOptOutMode(); + switch (configured.trim().toLowerCase(Locale.ROOT)) { + case "disabled": + return ExperimentationOptOutMode.DISABLED; + case "immediate": + return ExperimentationOptOutMode.IMMEDIATE; + case "future_experiments_only": + return ExperimentationOptOutMode.FUTURE_EXPERIMENTS_ONLY; + default: + throw new IllegalArgumentException( + "DD_FEATURE_FLAGS_EXPERIMENTATION_OPT_OUT_MODE must be disabled, immediate, or future_experiments_only"); + } + } + + /** Parses the cutoff once per evaluation; null means no exclusion. */ + static Instant cutoff(final ExperimentationOptOutMode mode, final EvaluationContext context) { + if ((mode != ExperimentationOptOutMode.IMMEDIATE + && mode != ExperimentationOptOutMode.FUTURE_EXPERIMENTS_ONLY) + || !context.keySet().contains(ATTRIBUTE)) { + return null; + } + final Value value = context.getValue(ATTRIBUTE); + if (value != null && value.isString()) { + try { + // Use the same timestamp parser as allocation startAt/endAt. + return DateTimeFormatter.ISO_OFFSET_DATE_TIME.parse(value.asString(), Instant::from); + } catch (final DateTimeException e) { + // Do not expose the supplied context value in the diagnostic. + } + } + throw new InvalidTimestampException(); + } + + /** Distinguishes invalid subject input from exceptions thrown by a context implementation. */ + static final class InvalidTimestampException extends IllegalArgumentException { + InvalidTimestampException() { + super(INVALID_TIMESTAMP); + } + } + + /** Checks prepared activation states before selecting an allocation. */ + static boolean hasInvalidActivation(final List allocations) { + if (allocations == null) { + return false; + } + for (final Allocation allocation : allocations) { + if (allocation != null && allocation.invalidRandomizationActivation) { + return true; + } + } + return false; + } + + /** Excludes before targeting; unknown activation does not qualify as older. */ + static boolean excludes( + final ExperimentationOptOutMode mode, final Instant cutoff, final Allocation allocation) { + if (cutoff == null) { + return false; + } + if (!Boolean.TRUE.equals(allocation.doLog)) { + return false; + } + return mode == ExperimentationOptOutMode.IMMEDIATE + || allocation.randomizationActivatedAt == null + || !allocation.randomizationActivatedAt.isBefore(cutoff); + } +} diff --git a/dd-openfeature/src/main/java/com/datadog/openfeature/internal/evaluation/FlagEvaluator.java b/dd-openfeature/src/main/java/com/datadog/openfeature/internal/evaluation/FlagEvaluator.java index dd4b4956492..03cdfe3b9de 100644 --- a/dd-openfeature/src/main/java/com/datadog/openfeature/internal/evaluation/FlagEvaluator.java +++ b/dd-openfeature/src/main/java/com/datadog/openfeature/internal/evaluation/FlagEvaluator.java @@ -12,6 +12,7 @@ import static com.datadog.openfeature.internal.evaluation.MetadataKeys.SPLIT_SERIAL_ID; import static java.util.Arrays.asList; +import com.datadog.openfeature.ExperimentationOptOutMode; import com.datadog.openfeature.internal.ufc.Allocation; import com.datadog.openfeature.internal.ufc.ConditionConfiguration; import com.datadog.openfeature.internal.ufc.ConditionOperator; @@ -59,6 +60,7 @@ public final class FlagEvaluator { static final Set WARNED_FEATURE_DESTINATIONS = ConcurrentHashMap.newKeySet(); private final boolean spanEnrichmentEnabled; private final ExposureLookup exposureLookup; + private final ExperimentationOptOutMode experimentationOptOutMode; /** * Creates an evaluator. @@ -67,8 +69,16 @@ public final class FlagEvaluator { * @param exposureLookup the lookup telling whether an exposure was already recorded. */ public FlagEvaluator(final boolean spanEnrichmentEnabled, final ExposureLookup exposureLookup) { + this(spanEnrichmentEnabled, exposureLookup, ExperimentationOptOutMode.DISABLED); + } + + public FlagEvaluator( + final boolean spanEnrichmentEnabled, + final ExposureLookup exposureLookup, + final ExperimentationOptOutMode experimentationOptOutMode) { this.spanEnrichmentEnabled = spanEnrichmentEnabled; this.exposureLookup = exposureLookup; + this.experimentationOptOutMode = experimentationOptOutMode; } public ProviderEvaluation evaluate( @@ -93,6 +103,20 @@ public ProviderEvaluation evaluate( return error(defaultValue, ErrorCode.INVALID_CONTEXT, null, observeFullEvaluationData); } + final Instant experimentationCutoff; + try { + experimentationCutoff = ExperimentationOptOut.cutoff(experimentationOptOutMode, context); + } catch (final ExperimentationOptOut.InvalidTimestampException e) { + // This diagnostic is fixed SDK text. It contains no customer context values. + return ProviderEvaluation.builder() + .value(defaultValue) + .reason(Reason.ERROR.name()) + .errorCode(ErrorCode.INVALID_CONTEXT) + .errorMessage(ExperimentationOptOut.INVALID_TIMESTAMP) + .flagMetadata(consentMetadata(observeFullEvaluationData)) + .build(); + } + final Flag flag = config.flags.get(key); if (flag == null) { if (config.invalidFlags != null && config.invalidFlags.containsKey(key)) { @@ -105,6 +129,16 @@ public ProviderEvaluation evaluate( return error(defaultValue, ErrorCode.FLAG_NOT_FOUND, null, observeFullEvaluationData); } + if (experimentationCutoff != null + && experimentationOptOutMode == ExperimentationOptOutMode.FUTURE_EXPERIMENTS_ONLY + && ExperimentationOptOut.hasInvalidActivation(flag.allocations)) { + return error( + defaultValue, + ErrorCode.PARSE_ERROR, + "invalid randomizationActivatedAt for flag " + key, + observeFullEvaluationData); + } + if (!flag.enabled) { return ProviderEvaluation.builder() .value(defaultValue) @@ -126,6 +160,11 @@ public ProviderEvaluation evaluate( final String targetingKey = context.getTargetingKey(); for (final Allocation allocation : flag.allocations) { + if (experimentationCutoff != null + && ExperimentationOptOut.excludes( + experimentationOptOutMode, experimentationCutoff, allocation)) { + continue; + } if (!isAllocationActive(allocation, now)) { continue; } @@ -275,9 +314,9 @@ private static boolean evaluateCondition( switch (condition.operator) { case MATCHES: - return matchesRegex(attributeValue, condition.value); + return matchesRegex(attributeValue, condition); case NOT_MATCHES: - return !matchesRegex(attributeValue, condition.value); + return !matchesRegex(attributeValue, condition); case ONE_OF: return isOneOf(attributeValue, condition.value); case NOT_ONE_OF: @@ -307,23 +346,17 @@ private static boolean evaluateCondition( } } - private static boolean matchesRegex(final Object attributeValue, final Object conditionValue) { - // PatternSyntaxException is intentionally not caught here so it propagates to evaluate(), - // which maps it to ErrorCode.PARSE_ERROR. - final Pattern pattern = Pattern.compile(normalizeRegex(String.valueOf(conditionValue))); + private static boolean matchesRegex( + final Object attributeValue, final ConditionConfiguration condition) { + // Parsed configurations cache the validated pattern. Directly constructed configurations + // retain the same syntax and error handling. + final Pattern pattern = + condition.regex != null + ? condition.regex + : ConditionConfiguration.compileRegex(condition.value); return pattern.matcher(String.valueOf(attributeValue)).find(); } - private static String normalizeRegex(final String regex) { - return regex - .replace("[:alnum:]", "\\p{Alnum}") - .replace("[:alpha:]", "\\p{Alpha}") - .replace("[:digit:]", "\\p{Digit}") - .replace("[:lower:]", "\\p{Lower}") - .replace("[:upper:]", "\\p{Upper}") - .replace("[:space:]", "\\p{Space}"); - } - private static boolean isOneOf(final Object attributeValue, final Object conditionValue) { if (!(conditionValue instanceof Iterable)) { return false; diff --git a/dd-openfeature/src/main/java/com/datadog/openfeature/internal/ufc/Allocation.java b/dd-openfeature/src/main/java/com/datadog/openfeature/internal/ufc/Allocation.java index b37a676571a..b26b7da0448 100644 --- a/dd-openfeature/src/main/java/com/datadog/openfeature/internal/ufc/Allocation.java +++ b/dd-openfeature/src/main/java/com/datadog/openfeature/internal/ufc/Allocation.java @@ -11,6 +11,12 @@ public final class Allocation { public final List splits; public final Boolean doLog; + /** First activation of this allocation's randomization; null means unknown or malformed. */ + public final Instant randomizationActivatedAt; + + /** Distinguishes malformed activation from missing/null for future-mode evaluation errors. */ + public final boolean invalidRandomizationActivation; + public Allocation( final String key, final List rules, @@ -18,11 +24,36 @@ public Allocation( final Instant endAt, final List splits, final Boolean doLog) { + this(key, rules, startAt, endAt, splits, doLog, null); + } + + public Allocation( + final String key, + final List rules, + final Instant startAt, + final Instant endAt, + final List splits, + final Boolean doLog, + final Instant randomizationActivatedAt) { + this(key, rules, startAt, endAt, splits, doLog, randomizationActivatedAt, false); + } + + public Allocation( + final String key, + final List rules, + final Instant startAt, + final Instant endAt, + final List splits, + final Boolean doLog, + final Instant randomizationActivatedAt, + final boolean invalidRandomizationActivation) { this.key = key; this.rules = rules; this.startAt = startAt; this.endAt = endAt; this.splits = splits; this.doLog = doLog; + this.randomizationActivatedAt = randomizationActivatedAt; + this.invalidRandomizationActivation = invalidRandomizationActivation; } } diff --git a/dd-openfeature/src/main/java/com/datadog/openfeature/internal/ufc/ConditionConfiguration.java b/dd-openfeature/src/main/java/com/datadog/openfeature/internal/ufc/ConditionConfiguration.java index c5d79cd67e1..484d126cecf 100644 --- a/dd-openfeature/src/main/java/com/datadog/openfeature/internal/ufc/ConditionConfiguration.java +++ b/dd-openfeature/src/main/java/com/datadog/openfeature/internal/ufc/ConditionConfiguration.java @@ -1,5 +1,7 @@ package com.datadog.openfeature.internal.ufc; +import java.util.regex.Pattern; + public final class ConditionConfiguration { public final ConditionOperator operator; public final String attribute; @@ -9,10 +11,28 @@ public final class ConditionConfiguration { // (not from JSON) when the operator is a SEMVER_* operator. public transient ParsedSemver semverComparand; + // The validated MATCHES/NOT_MATCHES pattern, prepared when the configuration is parsed. + public transient Pattern regex; + public ConditionConfiguration( final ConditionOperator operator, final String attribute, final Object value) { this.operator = operator; this.attribute = attribute; this.value = value; } + + /** Compiles the Java pattern after translating the supported POSIX character classes. */ + public static Pattern compileRegex(final Object value) { + return Pattern.compile(normalizeRegex(String.valueOf(value))); + } + + private static String normalizeRegex(final String regex) { + return regex + .replace("[:alnum:]", "\\p{Alnum}") + .replace("[:alpha:]", "\\p{Alpha}") + .replace("[:digit:]", "\\p{Digit}") + .replace("[:lower:]", "\\p{Lower}") + .replace("[:upper:]", "\\p{Upper}") + .replace("[:space:]", "\\p{Space}"); + } } diff --git a/dd-openfeature/src/main/java/com/datadog/openfeature/internal/ufc/UniversalFlagConfigParser.java b/dd-openfeature/src/main/java/com/datadog/openfeature/internal/ufc/UniversalFlagConfigParser.java index 2721be5bfef..d6fb0e435f4 100644 --- a/dd-openfeature/src/main/java/com/datadog/openfeature/internal/ufc/UniversalFlagConfigParser.java +++ b/dd-openfeature/src/main/java/com/datadog/openfeature/internal/ufc/UniversalFlagConfigParser.java @@ -235,13 +235,17 @@ private static Allocation toAllocation(@Nullable final Object value) { return null; } final Map json = asObject(value, "allocation"); + final Object activation = json.get("randomizationActivatedAt"); + final Instant parsedActivation = activation instanceof String ? toInstant(activation) : null; return new Allocation( asString(json.get("key"), "key"), toList(json.get("rules"), "rules", UniversalFlagConfigParser::toRule), toInstant(json.get("startAt")), toInstant(json.get("endAt")), toList(json.get("splits"), "splits", UniversalFlagConfigParser::toSplit), - lenientBoolean(json.get("doLog"))); + lenientBoolean(json.get("doLog")), + parsedActivation, + activation != null && parsedActivation == null); } @Nullable @@ -486,8 +490,9 @@ private static Instant toInstant(@Nullable final Object value) { } } - /** Validates the required nested UFC fields and SemVer comparands for a flag. */ + /** Validates the required nested UFC fields and condition operands for a flag. */ private static void validateFlag(final String flagKey, final Flag flag) { + validateNumericVariants(flagKey, flag); if (flag.allocations == null) { return; } @@ -523,6 +528,24 @@ private static void validateFlag(final String flagKey, final Flag flag) { validateAndCacheSemverComparands(flagKey, flag); } + /** Preserve Java's numeric coercion, but reject values that cannot be mapped at all. */ + private static void validateNumericVariants(final String flagKey, final Flag flag) { + if (flag.variations == null + || (flag.variationType != ValueType.INTEGER && flag.variationType != ValueType.NUMERIC)) { + return; + } + for (final Variant variant : flag.variations.values()) { + if (variant == null || variant.value == null || variant.value instanceof Number) { + continue; + } + try { + Double.parseDouble(String.valueOf(variant.value)); + } catch (final NumberFormatException e) { + throw new IllegalArgumentException("flag \"" + flagKey + "\" has a non-numeric variant"); + } + } + } + private static void validateConditionOperands(final String flagKey, final Allocation allocation) { if (allocation.rules == null) { return; @@ -536,6 +559,11 @@ private static void validateConditionOperands(final String flagKey, final Alloca continue; } switch (condition.operator) { + case MATCHES: + case NOT_MATCHES: + // Configuration validity is independent of targeting and experimentation opt-out. + condition.regex = ConditionConfiguration.compileRegex(condition.value); + break; case LT: case LTE: case GT: diff --git a/dd-openfeature/src/main/java/datadog/trace/api/openfeature/Provider.java b/dd-openfeature/src/main/java/datadog/trace/api/openfeature/Provider.java index ba8a1f9968f..14a739423ed 100644 --- a/dd-openfeature/src/main/java/datadog/trace/api/openfeature/Provider.java +++ b/dd-openfeature/src/main/java/datadog/trace/api/openfeature/Provider.java @@ -39,6 +39,13 @@ public Provider(final Options options) { */ @Deprecated public static class Options extends com.datadog.openfeature.Provider.Options { + @Override + public Options experimentationOptOutMode( + final com.datadog.openfeature.ExperimentationOptOutMode mode) { + super.experimentationOptOutMode(mode); + return this; + } + @Override public Options enableEvaluationTelemetryOptOutParameter(final boolean enabled) { super.enableEvaluationTelemetryOptOutParameter(enabled); diff --git a/dd-openfeature/src/test/java/com/datadog/openfeature/CanonicalFixtureTest.java b/dd-openfeature/src/test/java/com/datadog/openfeature/CanonicalFixtureTest.java index 9dd40371799..c89e1e4ab0d 100644 --- a/dd-openfeature/src/test/java/com/datadog/openfeature/CanonicalFixtureTest.java +++ b/dd-openfeature/src/test/java/com/datadog/openfeature/CanonicalFixtureTest.java @@ -4,11 +4,15 @@ import static org.hamcrest.CoreMatchers.equalTo; import static org.hamcrest.MatcherAssert.assertThat; import static org.hamcrest.Matchers.greaterThan; +import static org.junit.jupiter.api.Assertions.assertEquals; +import static org.junit.jupiter.api.Assertions.assertFalse; +import static org.junit.jupiter.api.Assertions.assertNull; import static org.mockito.Mockito.mock; import com.datadog.openfeature.internal.JsonReading; import com.datadog.openfeature.internal.bridge.v1.AgentBridge; import com.datadog.openfeature.internal.config.TestSettings; +import com.datadog.openfeature.internal.evaluation.ExperimentationOptOut; import com.datadog.openfeature.internal.ufc.UniversalFlagConfigParser; import dev.openfeature.sdk.ErrorCode; import dev.openfeature.sdk.EvaluationContext; @@ -44,17 +48,69 @@ void canonicalFixturesArePresent() throws IOException { @MethodSource("canonicalTestCases") @ParameterizedTest(name = "{0}") void evaluateCanonicalFixture(final FixtureCase testCase) throws IOException { + evaluateCanonicalFixture(testCase, ExperimentationOptOutMode.DISABLED); + } + + @MethodSource("immediateOptOutTestCases") + @ParameterizedTest(name = "immediate: {0}") + void evaluateImmediateOptOutFixture(final FixtureCase testCase) throws IOException { + evaluateCanonicalFixture(testCase, ExperimentationOptOutMode.IMMEDIATE); + } + + @MethodSource("futureOptOutTestCases") + @ParameterizedTest(name = "future: {0}") + void evaluateFutureOptOutFixture(final FixtureCase testCase) throws IOException { + evaluateCanonicalFixture(testCase, ExperimentationOptOutMode.FUTURE_EXPERIMENTS_ONLY); + } + + static List futureOptOutTestCases() throws IOException { + return optOutTestCases("experimentationOptOutFutureExperimentsOnly", 44); + } + + static List immediateOptOutTestCases() throws IOException { + return optOutTestCases("experimentationOptOutImmediate", 28); + } + + @SuppressWarnings("unchecked") + private static List optOutTestCases(final String capability, final int expectedCount) + throws IOException { + final List cases = new ArrayList<>(); + for (final FixtureCase testCase : canonicalTestCases()) { + final Map capabilities = + (Map) testCase.json.get("capabilities"); + if (capabilities != null && capabilities.containsKey(capability)) { + final Map json = new java.util.LinkedHashMap<>(testCase.json); + json.put("result", capabilities.get(capability)); + cases.add(new FixtureCase(testCase.fileName, testCase.index, json)); + } + } + assertEquals(expectedCount, cases.size(), "The opt-out capability fixtures must execute"); + return cases; + } + + private void evaluateCanonicalFixture( + final FixtureCase testCase, final ExperimentationOptOutMode mode) throws IOException { final DDEvaluator evaluator = - new DDEvaluator(mock(Runnable.class), AgentBridge.NONE, TestSettings.of()); + new DDEvaluator( + mock(Runnable.class), + AgentBridge.NONE, + TestSettings.of("experimental.flagging.provider.span.enrichment.enabled", "true"), + mode); evaluator.accept( UniversalFlagConfigParser.parse( Files.readAllBytes(fixtureRoot().resolve("ufc-config.json")))); final Class targetType = targetType(testCase.variationType); - final Object defaultValue = DDEvaluator.mapValue(targetType, testCase.defaultValue); - final Object expectedValue = DDEvaluator.mapValue(targetType, testCase.resultValue); - final ProviderEvaluation details = - evaluate(evaluator, targetType, testCase.flag, defaultValue, context(testCase)); + final Object defaultValue = fixtureValue(targetType, testCase.defaultValue); + final Object expectedValue = fixtureValue(targetType, testCase.resultValue); + final Provider provider = + new Provider(new Provider.Options().experimentationOptOutMode(mode), evaluator, false); + final ProviderEvaluation details; + try { + details = evaluate(provider, targetType, testCase.flag, defaultValue, context(testCase)); + } finally { + provider.shutdown(); + } assertThat(details.getValue(), equalTo(expectedValue)); assertThat(details.getReason(), equalTo(testCase.resultReason)); @@ -63,6 +119,22 @@ void evaluateCanonicalFixture(final FixtureCase testCase) throws IOException { } if (testCase.resultErrorCode != null) { assertThat(details.getErrorCode(), equalTo(ErrorCode.valueOf(testCase.resultErrorCode))); + if ("INVALID_CONTEXT".equals(testCase.resultErrorCode)) { + assertEquals(ExperimentationOptOut.INVALID_TIMESTAMP, details.getErrorMessage()); + } + } else if (!"ERROR".equals(testCase.resultReason)) { + assertNull(details.getErrorCode()); + } + final Map metadata = details.getFlagMetadata().asUnmodifiableMap(); + if (testCase.checksSerialId) { + if (testCase.resultSerialId == null) { + assertFalse(metadata.containsKey("__dd_split_serial_id")); + } else { + assertEquals(testCase.resultSerialId.intValue(), metadata.get("__dd_split_serial_id")); + } + } + if (testCase.resultDoLog != null) { + assertEquals(testCase.resultDoLog, metadata.get("__dd_do_log")); } if (testCase.resultAllocationKey != null) { assertThat( @@ -71,14 +143,41 @@ void evaluateCanonicalFixture(final FixtureCase testCase) throws IOException { } } - @SuppressWarnings({"rawtypes", "unchecked"}) + private static Object fixtureValue(final Class type, final Object value) { + if (value == null) { + return null; + } + if (type == Value.class) { + return Value.objectToValue(value); + } + if (type == Integer.class) { + return ((Number) value).intValue(); + } + if (type == Double.class) { + return ((Number) value).doubleValue(); + } + return value; + } + private static ProviderEvaluation evaluate( - final DDEvaluator evaluator, + final Provider provider, final Class targetType, final String flag, final Object defaultValue, final EvaluationContext context) { - return evaluator.evaluate((Class) targetType, flag, defaultValue, context); + if (targetType == Boolean.class) { + return provider.getBooleanEvaluation(flag, (Boolean) defaultValue, context); + } + if (targetType == Integer.class) { + return provider.getIntegerEvaluation(flag, (Integer) defaultValue, context); + } + if (targetType == Double.class) { + return provider.getDoubleEvaluation(flag, (Double) defaultValue, context); + } + if (targetType == String.class) { + return provider.getStringEvaluation(flag, (String) defaultValue, context); + } + return provider.getObjectEvaluation(flag, (Value) defaultValue, context); } @SuppressWarnings("unchecked") @@ -164,6 +263,7 @@ private static Object normalize(final Object value) { } static final class FixtureCase { + final Map json; final String fileName; final int index; final Map attributes; @@ -176,9 +276,13 @@ static final class FixtureCase { final String resultErrorCode; final String resultVariant; final String resultAllocationKey; + final boolean checksSerialId; + final Number resultSerialId; + final Boolean resultDoLog; @SuppressWarnings("unchecked") FixtureCase(final String fileName, final int index, final Map json) { + this.json = json; this.fileName = fileName; this.index = index; final Object attributes = normalize(json.get("attributes")); @@ -188,6 +292,9 @@ static final class FixtureCase { this.targetingKey = (String) json.get("targetingKey"); this.variationType = (String) json.get("variationType"); final Map result = (Map) json.get("result"); + this.checksSerialId = result.containsKey("serialId"); + this.resultSerialId = (Number) result.get("serialId"); + this.resultDoLog = (Boolean) result.get("doLog"); this.resultValue = normalize(result.get("value")); this.resultReason = (String) result.get("reason"); this.resultErrorCode = (String) result.get("errorCode"); diff --git a/dd-openfeature/src/test/java/com/datadog/openfeature/EvaluationTelemetryOptOutTest.java b/dd-openfeature/src/test/java/com/datadog/openfeature/EvaluationTelemetryOptOutTest.java index b85b8dff282..ee9109c5a80 100644 --- a/dd-openfeature/src/test/java/com/datadog/openfeature/EvaluationTelemetryOptOutTest.java +++ b/dd-openfeature/src/test/java/com/datadog/openfeature/EvaluationTelemetryOptOutTest.java @@ -75,6 +75,7 @@ import org.junit.jupiter.params.provider.MethodSource; import org.junit.jupiter.params.provider.NullSource; import org.junit.jupiter.params.provider.ValueSource; +import org.tabletest.junit.TableTest; /** * Exercises real OpenFeature resolution and the provider telemetry hooks, capturing each output @@ -599,6 +600,99 @@ void missingConfigurationKeepsTheDefaultWithoutEmittingTelemetry() throws Except assertEquals(1, this.customerEvaluations.size()); } + @TableTest({ + "scenario | mode | exclude experimentation | suppress telemetry", + "neither | DISABLED | false | false ", + "telemetry only | DISABLED | false | true ", + "experiments only | IMMEDIATE | true | false ", + "both | IMMEDIATE | true | true ", + "retained future | FUTURE_EXPERIMENTS_ONLY | false | false ", + "retained private | FUTURE_EXPERIMENTS_ONLY | false | true " + }) + void experimentationAndEvaluationTelemetryControlsRemainIndependent( + final ExperimentationOptOutMode mode, + final boolean excludeExperimentation, + final boolean suppressTelemetry) + throws Exception { + this.evaluator = + new CapturingEvaluator( + experimentationConfiguration("2026-10-19T11:59:59.999999999Z"), mode); + this.api.setProviderAndWait( + provider( + new Provider.Options() + .experimentationOptOutMode(mode) + .enableEvaluationTelemetryOptOutParameter(suppressTelemetry), + this.evaluator)); + final FlagEvaluationDetails details = + this.api + .getClient() + .getIntegerDetails( + "flag", + 0, + new MutableContext("subject-1") + .add(PARAMETER, true) + .add("dd_experimentation_opt_out_at", "2026-10-19T12:00:00Z")); + + assertNull(details.getErrorCode()); + assertEquals(excludeExperimentation ? 2 : 1, details.getValue()); + assertEquals(excludeExperimentation ? 0 : 1, this.exposures.size()); + assertTelemetryCounts(suppressTelemetry ? 0 : 1); + assertEquals(1, this.customerEvaluations.size()); + } + + @TableTest({ + "mode | activation | excluded", + "IMMEDIATE | 2026-10-19T11:59:59Z | true ", + "FUTURE_EXPERIMENTS_ONLY | 2026-10-19T12:00:00Z | true ", + "FUTURE_EXPERIMENTS_ONLY | 2026-10-19T11:59:59Z | false " + }) + void optOutDoesNotConsumeOrClearExposureDeduplication( + final ExperimentationOptOutMode mode, final String activation, final boolean excluded) + throws Exception { + this.evaluator = new CapturingEvaluator(experimentationConfiguration(activation), mode); + this.api.setProviderAndWait( + provider(new Provider.Options().experimentationOptOutMode(mode), this.evaluator)); + final Client client = this.api.getClient(); + final MutableContext optedOut = + new MutableContext("subject").add("dd_experimentation_opt_out_at", "2026-10-19T12:00:00Z"); + final MutableContext participating = new MutableContext("subject"); + assertEquals(excluded ? 2 : 1, client.getIntegerValue("flag", 0, optedOut)); + assertEquals(excluded ? 0 : 1, this.exposures.size()); + assertEquals(1, client.getIntegerValue("flag", 0, participating)); + assertEquals(1, this.exposures.size(), "exclusion must not consume the first exposure"); + assertEquals(excluded ? 2 : 1, client.getIntegerValue("flag", 0, optedOut)); + assertEquals(1, client.getIntegerValue("flag", 0, participating)); + assertEquals(1, this.exposures.size(), "exclusion must not clear the previous exposure"); + } + + private static ServerConfiguration experimentationConfiguration(final String activation) { + final ServerConfiguration configuration = defaultConfiguration(); + final Flag flag = configuration.flags.get("flag"); + final Allocation original = flag.allocations.get(0); + final Allocation experiment = + new Allocation( + original.key, + original.rules, + original.startAt, + original.endAt, + original.splits, + original.doLog, + java.time.Instant.parse(activation)); + final Allocation ordinary = + new Allocation( + "ordinary", + null, + null, + null, + singletonList(new Split(emptyList(), "off", emptyMap(), 8)), + false); + final Map variants = new HashMap<>(flag.variations); + variants.put("off", new Variant("off", 2)); + final Flag withFallback = + new Flag("flag", true, ValueType.INTEGER, variants, asList(experiment, ordinary)); + return new ServerConfiguration("", "", false, null, singletonMap("flag", withFallback)); + } + private void start() throws Exception { start(new Provider.Options().enableEvaluationTelemetryOptOutParameter(true)); } @@ -709,10 +803,16 @@ private void assertTelemetryCounts(final int expected) { */ private final class CapturingEvaluator extends DDEvaluator { CapturingEvaluator(final ServerConfiguration configuration) { + this(configuration, ExperimentationOptOutMode.DISABLED); + } + + CapturingEvaluator( + final ServerConfiguration configuration, final ExperimentationOptOutMode mode) { super( () -> {}, AgentBridge.NONE, - TestSettings.of("experimental.flagging.provider.span.enrichment.enabled", "true")); + TestSettings.of("experimental.flagging.provider.span.enrichment.enabled", "true"), + mode); accept(configuration); } diff --git a/dd-openfeature/src/test/java/com/datadog/openfeature/ExperimentationOptOutProviderTest.java b/dd-openfeature/src/test/java/com/datadog/openfeature/ExperimentationOptOutProviderTest.java new file mode 100644 index 00000000000..46e33390e78 --- /dev/null +++ b/dd-openfeature/src/test/java/com/datadog/openfeature/ExperimentationOptOutProviderTest.java @@ -0,0 +1,496 @@ +package com.datadog.openfeature; + +import static java.nio.charset.StandardCharsets.UTF_8; +import static java.util.concurrent.TimeUnit.SECONDS; +import static org.junit.jupiter.api.Assertions.assertEquals; +import static org.junit.jupiter.api.Assertions.assertNull; +import static org.junit.jupiter.api.Assertions.assertThrows; +import static org.junit.jupiter.api.Assertions.assertTrue; +import static org.mockito.Mockito.mock; +import static org.mockito.Mockito.when; + +import com.datadog.openfeature.internal.JsonReading; +import com.datadog.openfeature.internal.LocalHttpServer; +import com.datadog.openfeature.internal.bridge.v1.AgentBridge; +import com.datadog.openfeature.internal.config.TestSettings; +import com.datadog.openfeature.internal.ufc.UniversalFlagConfigParser; +import dev.openfeature.sdk.ErrorCode; +import dev.openfeature.sdk.EvaluationContext; +import dev.openfeature.sdk.MutableContext; +import dev.openfeature.sdk.ProviderEvaluation; +import dev.openfeature.sdk.exceptions.FatalError; +import java.nio.file.Files; +import java.nio.file.Path; +import java.nio.file.Paths; +import java.util.HashMap; +import java.util.HashSet; +import java.util.List; +import java.util.Locale; +import java.util.Map; +import java.util.Set; +import java.util.stream.Stream; +import org.junit.jupiter.api.Test; +import org.junit.jupiter.params.ParameterizedTest; +import org.junit.jupiter.params.provider.Arguments; +import org.junit.jupiter.params.provider.MethodSource; +import org.junit.jupiter.params.provider.ValueSource; +import org.tabletest.junit.TableTest; + +/** Exercises experimentation opt-out through the production parser and evaluator. */ +class ExperimentationOptOutProviderTest { + private static final String ATTRIBUTE = "dd_experimentation_opt_out_at"; + private static final String CUTOFF = "2026-10-19T12:00:00Z"; + + @SuppressWarnings("unchecked") + static Stream policyCases() throws Exception { + final byte[] bytes = Files.readAllBytes(fixture("cases.json")); + final List> cases = (List>) JsonReading.read(bytes); + assertEquals(90, cases.size()); + final Set ids = new HashSet<>(); + for (final Map testCase : cases) { + assertTrue(ids.add(testCase.get("id")), "duplicate policy case"); + } + return cases.stream().map(testCase -> Arguments.of(testCase.get("id"), testCase)); + } + + @ParameterizedTest(name = "{0}") + @MethodSource("policyCases") + @SuppressWarnings("unchecked") + void policy(final String id, final Map testCase) throws Exception { + final Map args = (Map) testCase.get("args"); + final Map expected = (Map) testCase.get("expected"); + final String mode = (String) args.get("mode"); + final ProviderEvaluation result = + evaluate( + configuration( + activation((String) args.get("activation")), args.get("doLog").toString(), false), + mode, + (String) args.get("attribute"), + "target"); + final String error = (String) expected.get("error"); + if (!"NONE".equals(error)) { + assertEquals(ErrorCode.valueOf(error), result.getErrorCode(), id); + assertEquals("default", result.getValue(), id); + } else { + assertNull(result.getErrorCode(), id); + final String allocation = + Boolean.TRUE.equals(expected.get("exclude")) ? "ordinary" : "experiment"; + assertEquals(allocation, result.getValue(), id); + assertEquals(allocation, result.getFlagMetadata().getString("allocationKey"), id); + } + } + + @ParameterizedTest + @ValueSource(strings = {"immediate", "future_experiments_only"}) + void missingAndNullDoLogPreserveOrdinaryAllocation(final String mode) throws Exception { + for (final String doLog : new String[] {null, "null", "false"}) { + final ProviderEvaluation result = + evaluate(configuration(null, doLog, false), mode, "valid", "target"); + assertNull(result.getErrorCode()); + assertEquals("experiment", result.getValue()); + assertEquals("experiment", result.getFlagMetadata().getString("allocationKey")); + } + } + + @Test + void unreachableInvalidActivationFailsOnlyOptedOutFutureMode() throws Exception { + final String json = configuration("\"bad\"", "true", true); + for (final String mode : new String[] {"disabled", "immediate", "future_experiments_only"}) { + for (final String attribute : new String[] {"absent", "valid"}) { + final ProviderEvaluation result = evaluate(json, mode, attribute, "target"); + if ("future_experiments_only".equals(mode) && "valid".equals(attribute)) { + assertEquals(ErrorCode.PARSE_ERROR, result.getErrorCode()); + assertEquals("default", result.getValue()); + } else { + assertNull(result.getErrorCode()); + assertEquals("ordinary", result.getValue()); + } + final ProviderEvaluation neighbor = evaluate(json, mode, attribute, "neighbor"); + assertNull(neighbor.getErrorCode()); + assertEquals("ordinary", neighbor.getValue()); + } + } + } + + @Test + void invalidSubjectPrecedesMissingFlagButNotReadiness() throws Exception { + final ProviderEvaluation missing = + evaluate( + configuration(null, "true", false), "future_experiments_only", "invalid", "missing"); + assertEquals(ErrorCode.INVALID_CONTEXT, missing.getErrorCode()); + final ProviderEvaluation notReady = + evaluate(null, "future_experiments_only", "invalid", "missing"); + assertEquals(ErrorCode.PROVIDER_NOT_READY, notReady.getErrorCode()); + } + + @TableTest({ + "Scenario | deprecatedAlias | mode | configured | expectedValue", + "explicit False DISABLED | false | DISABLED | unsupported | experiment ", + "configured False DISABLED | false | DEFAULT | disabled | experiment ", + "explicit False IMMEDIATE | false | IMMEDIATE | unsupported | ordinary ", + "configured False IMMEDIATE | false | DEFAULT | immediate | ordinary ", + "explicit False FUTURE_EXPERIMENTS_ONLY | false | FUTURE_EXPERIMENTS_ONLY | unsupported | ordinary ", + "configured False FUTURE_EXPERIMENTS_ONLY | false | DEFAULT | future_experiments_only | ordinary ", + "explicit True DISABLED | true | DISABLED | unsupported | experiment ", + "configured True DISABLED | true | DEFAULT | disabled | experiment ", + "explicit True IMMEDIATE | true | IMMEDIATE | unsupported | ordinary ", + "configured True IMMEDIATE | true | DEFAULT | immediate | ordinary ", + "explicit True FUTURE_EXPERIMENTS_ONLY | true | FUTURE_EXPERIMENTS_ONLY | unsupported | ordinary ", + "configured True FUTURE_EXPERIMENTS_ONLY | true | DEFAULT | future_experiments_only | ordinary " + }) + @SuppressWarnings("deprecation") + void publicProvidersApplyOptOutAfterHttpConfiguration( + final boolean deprecatedAlias, + final ExperimentationOptOutMode mode, + final String configured, + final String expectedValue) + throws Exception { + try (final LocalHttpServer server = new LocalHttpServer()) { + server.otherwise( + 200, + "{\"data\":{\"type\":\"universal-flag-configuration\",\"attributes\":" + + configuration(activation("equal"), "true", false) + + "}}"); + final Map previousProperties = new HashMap<>(); + final Map properties = + Map.of( + "dd.feature.flags.enabled", "true", + "dd.feature.flags.configuration.source", "agentless", + "dd.feature.flags.configuration.source.agentless.base.url", + server.uri("/").toString(), + "dd.feature.flags.experimentation.opt.out.mode", configured, + "dd.api.key", ""); + properties.forEach( + (key, value) -> previousProperties.put(key, System.setProperty(key, value))); + try { + final Provider.Options options = + deprecatedAlias + ? new datadog.trace.api.openfeature.Provider.Options() + : new Provider.Options(); + options.initTimeout(5, SECONDS).experimentationOptOutMode(mode); + final Provider provider = + deprecatedAlias + ? new datadog.trace.api.openfeature.Provider( + (datadog.trace.api.openfeature.Provider.Options) options) + : new Provider(options); + // The policy is resolved at construction, before initialization or later option changes. + options.experimentationOptOutMode(ExperimentationOptOutMode.DEFAULT); + System.setProperty( + "dd.feature.flags.experimentation.opt.out.mode", "unsupported-after-construction"); + final MutableContext context = new MutableContext("subject").add(ATTRIBUTE, CUTOFF); + try { + provider.initialize(context); + final ProviderEvaluation result = + provider.getStringEvaluation("target", "default", context); + assertNull(result.getErrorCode()); + assertEquals(expectedValue, result.getValue()); + assertEquals(expectedValue, result.getFlagMetadata().getString("allocationKey")); + assertTrue(server.requests().size() > 0); + } finally { + provider.shutdown(); + } + } finally { + previousProperties.forEach( + (key, value) -> { + if (value == null) { + System.clearProperty(key); + } else { + System.setProperty(key, value); + } + }); + } + } + } + + @ParameterizedTest + @ValueSource(booleans = {false, true}) + @SuppressWarnings("deprecation") + void disabledProductDoesNotValidateAnUnusedMode(final boolean alias) { + final String enabledKey = "dd.feature.flags.enabled"; + final String modeKey = "dd.feature.flags.experimentation.opt.out.mode"; + final String previousEnabled = System.setProperty(enabledKey, "false"); + final String previousMode = System.setProperty(modeKey, "unsupported"); + try { + final Provider provider = + alias ? new datadog.trace.api.openfeature.Provider() : new Provider(); + try { + // Disabling Feature Flags keeps the existing initialization failure, not a mode error. + final FatalError error = + assertThrows( + FatalError.class, () -> provider.initialize(new MutableContext("subject"))); + assertEquals( + "Failed to initialize Datadog Feature Flags provider: Feature Flags is disabled", + error.getMessage()); + } finally { + provider.shutdown(); + } + } finally { + restoreProperty(enabledKey, previousEnabled); + restoreProperty(modeKey, previousMode); + } + } + + @ParameterizedTest + @ValueSource(strings = {"IMMEDIATE", "FUTURE_EXPERIMENTS_ONLY"}) + void excludedAllocationDoesNotEvaluateSubjectDependentNumericRules(final String mode) + throws Exception { + final DDEvaluator evaluator = + evaluator(withRule(configuration(activation("equal"), "true", false), "GT", "10"), mode); + final MutableContext context = new MutableContext("subject").add("attribute", "not-a-number"); + assertEquals( + ErrorCode.TYPE_MISMATCH, + evaluator.evaluate(String.class, "target", "default", context).getErrorCode()); + context.add(ATTRIBUTE, CUTOFF); + final ProviderEvaluation result = + evaluator.evaluate(String.class, "target", "default", context); + assertNull(result.getErrorCode()); + assertEquals("ordinary", result.getValue()); + } + + @Test + void configurationRefreshReplacesActivationAndRecoversFromInvalidMetadata() throws Exception { + final DDEvaluator evaluator = evaluator(null, "FUTURE_EXPERIMENTS_ONLY"); + final MutableContext context = new MutableContext("subject").add(ATTRIBUTE, CUTOFF); + for (final String activation : new String[] {"before", "equal", "invalid", "before"}) { + evaluator.accept( + UniversalFlagConfigParser.parse( + configuration(activation(activation), "true", false).getBytes(UTF_8))); + final ProviderEvaluation result = + evaluator.evaluate(String.class, "target", "default", context); + if ("invalid".equals(activation)) { + assertEquals(ErrorCode.PARSE_ERROR, result.getErrorCode()); + assertEquals("default", result.getValue()); + } else { + assertNull(result.getErrorCode()); + assertEquals("before".equals(activation) ? "experiment" : "ordinary", result.getValue()); + } + assertEquals( + "ordinary", evaluator.evaluate(String.class, "neighbor", "default", context).getValue()); + } + } + + @ParameterizedTest + @ValueSource(booleans = {false, true}) + void invalidSubjectRetainsEvaluationDataConsent(final boolean consent) throws Exception { + final String json = + configuration(null, "true", false) + .replaceFirst("\\{", "{\"observeFullEvaluationData\":" + consent + ","); + final ProviderEvaluation result = evaluate(json, "immediate", "invalid", "target"); + assertEquals(ErrorCode.INVALID_CONTEXT, result.getErrorCode()); + assertEquals("default", result.getValue()); + assertEquals(consent, result.getFlagMetadata().getBoolean("observe_full_evaluation_data")); + } + + @Test + void missingAllocationsRetainGeneralErrorInFutureMode() throws Exception { + final ProviderEvaluation result = + evaluate( + "{\"flags\":{\"target\":{\"enabled\":true,\"variationType\":\"STRING\"}}}", + "future_experiments_only", + "valid", + "target"); + assertEquals(ErrorCode.GENERAL, result.getErrorCode()); + assertEquals("default", result.getValue()); + } + + @ParameterizedTest + @ValueSource(strings = {"keySet", "getValue"}) + void contextImplementationFailureIsNotAnInvalidTimestamp(final String method) throws Exception { + final EvaluationContext context = mock(EvaluationContext.class); + when(context.keySet()).thenReturn(Set.of(ATTRIBUTE)); + if ("keySet".equals(method)) { + when(context.keySet()).thenThrow(new IllegalArgumentException("context failure")); + } else { + when(context.getValue(ATTRIBUTE)).thenThrow(new IllegalArgumentException("context failure")); + } + final ProviderEvaluation result = + evaluator(configuration(null, "true", false), "IMMEDIATE") + .evaluate(String.class, "target", "default", context); + assertEquals(ErrorCode.GENERAL, result.getErrorCode()); + assertEquals("default", result.getValue()); + } + + @TableTest({ + "operator | position ", + "MATCHES | excluded ", + "MATCHES | unreachable", + "MATCHES | expired ", + "MATCHES | disabled ", + "NOT_MATCHES | excluded ", + "NOT_MATCHES | unreachable", + "NOT_MATCHES | expired ", + "NOT_MATCHES | disabled " + }) + void malformedRegexFailsOnlyItsFlagRegardlessOfSelection( + final String operator, final String position) throws Exception { + String json = + withRule( + configuration(activation("equal"), "true", "unreachable".equals(position)), + operator, + "\"*@datadoghq.com\""); + if ("expired".equals(position)) { + json = + json.replace( + "\"key\":\"experiment\",\"rules\"", + "\"key\":\"experiment\",\"endAt\":\"2000-01-01T00:00:00Z\",\"rules\""); + } + if ("disabled".equals(position)) { + json = json.replaceFirst("\"enabled\":true", "\"enabled\":false"); + } + for (final String mode : new String[] {"DISABLED", "IMMEDIATE", "FUTURE_EXPERIMENTS_ONLY"}) { + final DDEvaluator evaluator = evaluator(json, mode); + for (final boolean optedOut : new boolean[] {false, true}) { + for (final boolean hasAttribute : new boolean[] {false, true}) { + final MutableContext context = new MutableContext("subject"); + if (optedOut) context.add(ATTRIBUTE, CUTOFF); + if (hasAttribute) context.add("attribute", "user@example.com"); + final ProviderEvaluation result = + evaluator.evaluate(String.class, "target", "default", context); + assertEquals(ErrorCode.PARSE_ERROR, result.getErrorCode()); + assertEquals("default", result.getValue()); + final ProviderEvaluation neighbor = + evaluator.evaluate(String.class, "neighbor", "default", context); + assertNull(neighbor.getErrorCode()); + assertEquals("ordinary", neighbor.getValue()); + } + } + assertEquals( + "DISABLED".equals(mode) ? ErrorCode.PARSE_ERROR : ErrorCode.INVALID_CONTEXT, + evaluator + .evaluate( + String.class, + "target", + "default", + new MutableContext("subject").add(ATTRIBUTE, "bad")) + .getErrorCode()); + } + } + + @TableTest({ + "type | value | valid", + "INTEGER | 42 | true ", + "INTEGER | '\"42\"' | true ", + "INTEGER | '\"invalid\"' | false", + "INTEGER | true | false", + "NUMERIC | 4.2 | true ", + "NUMERIC | '\"4.2\"' | true ", + "NUMERIC | '\"invalid\"' | false", + "NUMERIC | '{}' | false" + }) + void numericVariantValidityIsIndependentOfOptOut( + final String type, final String value, final boolean valid) throws Exception { + // Only the experiment variant in the target flag changes; the neighboring flag remains valid. + final String original = configuration(activation("equal"), "true", false); + final int neighbor = original.indexOf(",\"neighbor\":"); + final String target = + original + .substring(0, neighbor) + .replace("\"variationType\":\"STRING\"", "\"variationType\":\"" + type + "\"") + .replace("\"value\":\"experiment\"", "\"value\":" + value) + .replace("\"value\":\"ordinary\"", "\"value\":1"); + for (final String mode : new String[] {"IMMEDIATE", "FUTURE_EXPERIMENTS_ONLY"}) { + final DDEvaluator evaluator = evaluator(target + original.substring(neighbor), mode); + final MutableContext context = new MutableContext("subject").add(ATTRIBUTE, CUTOFF); + final ProviderEvaluation result = + "INTEGER".equals(type) + ? evaluator.evaluate(Integer.class, "target", 0, context) + : evaluator.evaluate(Double.class, "target", 0.0, context); + assertEquals(valid ? null : ErrorCode.PARSE_ERROR, result.getErrorCode()); + assertEquals(valid ? 1.0 : 0.0, ((Number) result.getValue()).doubleValue()); + assertEquals( + "ordinary", evaluator.evaluate(String.class, "neighbor", "default", context).getValue()); + } + } + + private static DDEvaluator evaluator(final String json, final String mode) throws Exception { + final DDEvaluator evaluator = + new DDEvaluator( + () -> {}, AgentBridge.NONE, TestSettings.of(), ExperimentationOptOutMode.valueOf(mode)); + if (json != null) evaluator.accept(UniversalFlagConfigParser.parse(json.getBytes(UTF_8))); + return evaluator; + } + + private static String withRule(final String json, final String operator, final String operand) { + return json.replace( + "\"key\":\"experiment\",\"doLog\"", + "\"key\":\"experiment\",\"rules\":[{\"conditions\":[{\"attribute\":\"attribute\",\"operator\":\"" + + operator + + "\",\"value\":" + + operand + + "}]}],\"doLog\""); + } + + private static void restoreProperty(final String key, final String previous) { + if (previous == null) System.clearProperty(key); + else System.setProperty(key, previous); + } + + private static ProviderEvaluation evaluate( + final String json, final String configuredMode, final String attribute, final String flag) + throws Exception { + final ExperimentationOptOutMode mode = + ExperimentationOptOutMode.valueOf(configuredMode.toUpperCase(Locale.ROOT)); + final DDEvaluator evaluator = + new DDEvaluator(() -> {}, AgentBridge.NONE, TestSettings.of(), mode); + if (json != null) { + evaluator.accept(UniversalFlagConfigParser.parse(json.getBytes(UTF_8))); + } + final MutableContext context = new MutableContext("subject"); + if (!"absent".equals(attribute)) { + context.add(ATTRIBUTE, "valid".equals(attribute) ? CUTOFF : "bad"); + } + return evaluator.evaluate(String.class, flag, "default", context); + } + + private static String activation(final String state) { + switch (state) { + case "missing": + return null; + case "invalid": + return "\"bad\""; + case "before": + return "\"2026-10-19T11:59:59.999999999Z\""; + case "equal": + return "\"2026-10-19T12:00:00Z\""; + case "after": + return "\"2026-10-19T12:00:00.000000001Z\""; + default: + throw new IllegalArgumentException(state); + } + } + + private static String configuration( + final String activation, final String doLog, final boolean unreachable) { + final String ordinary = + "{\"key\":\"ordinary\",\"doLog\":false,\"splits\":[{\"variationKey\":\"ordinary\",\"shards\":[]}]}"; + final String experiment = + "{\"key\":\"experiment\"" + + (doLog == null ? "" : ",\"doLog\":" + doLog) + + (activation == null ? "" : ",\"randomizationActivatedAt\":" + activation) + + ",\"splits\":[{\"variationKey\":\"experiment\",\"shards\":[]}]}"; + final String prefix = + "{\"enabled\":true,\"variationType\":\"STRING\",\"variations\":{\"experiment\":{\"key\":\"experiment\",\"value\":\"experiment\"},\"ordinary\":{\"key\":\"ordinary\",\"value\":\"ordinary\"}},\"allocations\":["; + return "{\"flags\":{\"target\":" + + prefix + + (unreachable ? ordinary + "," + experiment : experiment + "," + ordinary) + + "]},\"neighbor\":" + + prefix + + ordinary + + "]}}}"; + } + + private static Path fixture(final String name) { + Path directory = Paths.get("").toAbsolutePath(); + while (directory != null) { + final Path candidate = + directory + .resolve("dd-smoke-tests/openfeature/src/test/resources/experimentation-opt-out") + .resolve(name); + if (Files.isRegularFile(candidate)) { + return candidate; + } + directory = directory.getParent(); + } + throw new IllegalStateException("Missing experimentation opt-out fixture " + name); + } +} diff --git a/dd-openfeature/src/test/java/com/datadog/openfeature/internal/config/SettingsTest.java b/dd-openfeature/src/test/java/com/datadog/openfeature/internal/config/SettingsTest.java index a2476b5bae7..6fc2d6c6d0f 100644 --- a/dd-openfeature/src/test/java/com/datadog/openfeature/internal/config/SettingsTest.java +++ b/dd-openfeature/src/test/java/com/datadog/openfeature/internal/config/SettingsTest.java @@ -89,6 +89,32 @@ void appliesDefaults() { assertFalse(settings.isSpanEnrichmentEnabled()); assertTrue(settings.isEvaluationCountsEnabled()); assertTrue(settings.isExposuresDatadogLoggingEnabled()); + assertEquals("disabled", settings.experimentationOptOutMode()); + } + + @TableTest({ + "scenario | agent | property | environment | expected ", + "default | | | | disabled ", + "blank defaults | ' ' | '' | ' ' | disabled ", + "blank property | | ' ' | immediate | immediate ", + "blank agent | ' ' | disabled | immediate | disabled ", + "environment enables | | | immediate | immediate ", + "property wins | | disabled | immediate | disabled ", + "agent wins | immediate | disabled | disabled | immediate ", + "invalid is preserved | | unsupported | immediate | unsupported" + }) + void resolvesExperimentationOptOutSource( + final String agent, final String property, final String environment, final String expected) { + final String key = Settings.EXPERIMENTATION_OPT_OUT_MODE; + final Map bridge = new HashMap<>(); + final Map properties = new HashMap<>(); + final Map variables = new HashMap<>(); + putIfNotNull(bridge, key, agent); + putIfNotNull(properties, "dd." + key, property); + putIfNotNull(variables, "DD_FEATURE_FLAGS_EXPERIMENTATION_OPT_OUT_MODE", environment); + final Settings settings = + new Settings(new SettingAgentBridge(bridge), properties::get, variables::get); + assertEquals(expected, settings.experimentationOptOutMode()); } @TableTest({ diff --git a/dd-openfeature/src/test/java/com/datadog/openfeature/internal/evaluation/ExperimentationOptOutTest.java b/dd-openfeature/src/test/java/com/datadog/openfeature/internal/evaluation/ExperimentationOptOutTest.java new file mode 100644 index 00000000000..5f1fe133cf3 --- /dev/null +++ b/dd-openfeature/src/test/java/com/datadog/openfeature/internal/evaluation/ExperimentationOptOutTest.java @@ -0,0 +1,237 @@ +package com.datadog.openfeature.internal.evaluation; + +import static java.nio.charset.StandardCharsets.UTF_8; +import static org.junit.jupiter.api.Assertions.assertEquals; +import static org.junit.jupiter.api.Assertions.assertFalse; +import static org.junit.jupiter.api.Assertions.assertNotNull; +import static org.junit.jupiter.api.Assertions.assertNull; +import static org.junit.jupiter.api.Assertions.assertThrows; + +import com.datadog.openfeature.ExperimentationOptOutMode; +import com.datadog.openfeature.Provider; +import com.datadog.openfeature.internal.bridge.v1.AgentBridge; +import com.datadog.openfeature.internal.config.Settings; +import com.datadog.openfeature.internal.ufc.Allocation; +import com.datadog.openfeature.internal.ufc.UniversalFlagConfigParser; +import dev.openfeature.sdk.MutableContext; +import dev.openfeature.sdk.Value; +import java.time.Instant; +import java.time.format.DateTimeFormatter; +import java.util.Collections; +import java.util.stream.Stream; +import org.junit.jupiter.api.Test; +import org.junit.jupiter.params.ParameterizedTest; +import org.junit.jupiter.params.provider.Arguments; +import org.junit.jupiter.params.provider.MethodSource; +import org.junit.jupiter.params.provider.ValueSource; +import org.tabletest.junit.TableTest; + +class ExperimentationOptOutTest { + private static final String PROPERTY = "dd.feature.flags.experimentation.opt.out.mode"; + + @ParameterizedTest + @ValueSource( + strings = { + "disabled", + "immediate", + " IMMEDIATE ", + "future_experiments_only", + " FUTURE_EXPERIMENTS_ONLY ", + "unsupported" + }) + void providerConfigurationPrecedence(final String configured) { + final String previous = System.getProperty(PROPERTY); + System.setProperty(PROPERTY, configured); + try { + for (final ExperimentationOptOutMode override : + new ExperimentationOptOutMode[] { + ExperimentationOptOutMode.DISABLED, + ExperimentationOptOutMode.IMMEDIATE, + ExperimentationOptOutMode.FUTURE_EXPERIMENTS_ONLY + }) { + assertEquals( + override, ExperimentationOptOut.resolve(override, Settings.load(AgentBridge.NONE))); + new Provider(new Provider.Options().experimentationOptOutMode(override)).shutdown(); + } + if ("unsupported".equals(configured)) { + assertThrows(IllegalArgumentException.class, Provider::new); + } else { + final ExperimentationOptOutMode expected = + configured.trim().equalsIgnoreCase("immediate") + ? ExperimentationOptOutMode.IMMEDIATE + : configured.trim().equalsIgnoreCase("future_experiments_only") + ? ExperimentationOptOutMode.FUTURE_EXPERIMENTS_ONLY + : ExperimentationOptOutMode.DISABLED; + assertEquals( + expected, + ExperimentationOptOut.resolve( + ExperimentationOptOutMode.DEFAULT, Settings.load(AgentBridge.NONE))); + new Provider().shutdown(); + } + } finally { + if (previous == null) { + System.clearProperty(PROPERTY); + } else { + System.setProperty(PROPERTY, previous); + } + } + } + + @ParameterizedTest + @ValueSource( + strings = { + "2026-10-19T15:30:00Z", + "2026-10-19T15:30:00.123456789Z", + "2026-10-19T15:30:00+05:30", + "2026-10-19T15:30:00-04:00", + "9999-12-31T23:59:59Z", + "2024-02-29T00:00:00Z" + }) + void validTimestampProvidesCutoff(final String timestamp) { + assertNotNull( + ExperimentationOptOut.cutoff( + ExperimentationOptOutMode.IMMEDIATE, + new MutableContext().add(ExperimentationOptOut.ATTRIBUTE, timestamp))); + } + + static Stream invalidValues() { + return Stream.of( + new Value(), + new Value(""), + new Value("not a timestamp"), + new Value("2026-10-19"), + new Value("2026-10-19T15:30:00"), + new Value("2026-02-30T00:00:00Z"), + new Value("2026-10-19T24:00:00Z"), + new Value("2026-10-19T15:30:60Z"), + new Value("2026-10-19T15:30:00+99:00"), + new Value(" 2026-10-19T15:30:00Z"), + new Value(Instant.parse("2026-10-19T12:00:00Z")), + new Value(true), + new Value(42)) + .map(Arguments::of); + } + + @ParameterizedTest + @MethodSource("invalidValues") + void invalidValuesAreRejectedOnlyWhenEnabled(final Value value) { + final MutableContext context = + new MutableContext(Collections.singletonMap(ExperimentationOptOut.ATTRIBUTE, value)); + for (final ExperimentationOptOutMode mode : + new ExperimentationOptOutMode[] { + ExperimentationOptOutMode.IMMEDIATE, ExperimentationOptOutMode.FUTURE_EXPERIMENTS_ONLY + }) { + final IllegalArgumentException error = + assertThrows( + IllegalArgumentException.class, () -> ExperimentationOptOut.cutoff(mode, context)); + assertEquals( + "dd_experimentation_opt_out_at must be an ISO offset date-time string;" + + " supply a valid timestamp to opt out, or omit the attribute to permit experimentation", + error.getMessage()); + } + assertNull(ExperimentationOptOut.cutoff(ExperimentationOptOutMode.DISABLED, context)); + } + + @Test + void missingAttributePreservesParticipation() { + for (final ExperimentationOptOutMode mode : ExperimentationOptOutMode.values()) { + assertNull(ExperimentationOptOut.cutoff(mode, new MutableContext())); + // An inactive cutoff must return without inspecting the allocation. + assertFalse(ExperimentationOptOut.excludes(mode, null, null)); + } + } + + @TableTest({ + "timestamp | accepted", + "2026-10-19T15:30:00Z | true ", + "2026-10-19T15:30:00.123456789Z | true ", + "2026-10-19T15:30:00.123456789123Z | false ", + "2026-10-19T15:30:00+05:30 | true ", + "2026-10-19T15:30:00+18:00 | true ", + "2026-10-19T15:30:00+18:01 | false ", + "2026-10-19T15:30:00+24:00 | false ", + "2026-10-19T15:30:00+00:60 | false ", + "2026-10-19T15:30:60Z | false ", + "2026-02-30T00:00:00Z | false ", + "2024-02-29T00:00:00Z | true ", + "2026-10-19t15:30:00z | true ", + "2026-10-19T5:30:00Z | false ", + "2026-10-19 | false ", + "2026-10-19T15:30:00 | false ", + "2026-10-19T15:30Z | true ", + "2026-10-19T15:30:00.Z | true ", + "2026-10-19T15:30:00+05 | true ", + "2026-10-19T15:30:00+05:30:15 | true ", + "+10000-10-19T15:30:00Z | true ", + "' 2026-10-19T15:30:00Z' | false ", + "2026-10-19T24:00:00Z | false " + }) + void timestampDialectHasIndependentExpectations(final String timestamp, final boolean accepted) + throws Exception { + final MutableContext context = + new MutableContext().add(ExperimentationOptOut.ATTRIBUTE, timestamp); + if (accepted) { + assertNotNull(ExperimentationOptOut.cutoff(ExperimentationOptOutMode.IMMEDIATE, context)); + } else { + assertThrows( + ExperimentationOptOut.InvalidTimestampException.class, + () -> ExperimentationOptOut.cutoff(ExperimentationOptOutMode.IMMEDIATE, context)); + } + final String json = + "{\"flags\":{\"flag\":{\"allocations\":[{\"randomizationActivatedAt\":\"" + + timestamp + + "\"}]}}}"; + final Allocation allocation = + UniversalFlagConfigParser.parse(json.getBytes(UTF_8)).flags.get("flag").allocations.get(0); + assertEquals(!accepted, allocation.invalidRandomizationActivation); + assertEquals(accepted, allocation.randomizationActivatedAt != null); + } + + @Test + void rawNullEntryIsPresentAndInvalid() { + final MutableContext context = + new MutableContext(Collections.singletonMap(ExperimentationOptOut.ATTRIBUTE, null)); + assertThrows( + ExperimentationOptOut.InvalidTimestampException.class, + () -> ExperimentationOptOut.cutoff(ExperimentationOptOutMode.IMMEDIATE, context)); + } + + @TableTest({ + "mode | doLog | activation | excluded", + "IMMEDIATE | true | 2026-10-19T11:59:59.999999999Z | true ", + "FUTURE_EXPERIMENTS_ONLY | true | 2026-10-19T11:59:59.999999999Z | false ", + "FUTURE_EXPERIMENTS_ONLY | true | 2026-10-19T12:00:00Z | true ", + "FUTURE_EXPERIMENTS_ONLY | true | 2026-10-19T12:00:00.000000001Z | true ", + "FUTURE_EXPERIMENTS_ONLY | true | 2026-10-19T08:00:00-04:00 | true ", + "FUTURE_EXPERIMENTS_ONLY | true | 2026-10-19T07:59:59.999999999-04:00 | false ", + "FUTURE_EXPERIMENTS_ONLY | true | | true ", + "FUTURE_EXPERIMENTS_ONLY | false | | false ", + "IMMEDIATE | false | | false " + }) + void appliesActivationCutoff( + final ExperimentationOptOutMode mode, + final boolean doLog, + final String activation, + final boolean excluded) { + final Allocation allocation = + new Allocation( + "allocation", + null, + null, + null, + Collections.emptyList(), + doLog, + activation == null + ? null + : DateTimeFormatter.ISO_OFFSET_DATE_TIME.parse(activation, Instant::from)); + assertEquals( + excluded, + ExperimentationOptOut.excludes(mode, Instant.parse("2026-10-19T12:00:00Z"), allocation)); + } + + @Test + void nullProgrammaticModeIsRejected() { + assertThrows( + NullPointerException.class, () -> new Provider.Options().experimentationOptOutMode(null)); + } +} diff --git a/dd-openfeature/src/test/java/com/datadog/openfeature/internal/ufc/RandomizationActivationTest.java b/dd-openfeature/src/test/java/com/datadog/openfeature/internal/ufc/RandomizationActivationTest.java new file mode 100644 index 00000000000..c7e82046f52 --- /dev/null +++ b/dd-openfeature/src/test/java/com/datadog/openfeature/internal/ufc/RandomizationActivationTest.java @@ -0,0 +1,56 @@ +package com.datadog.openfeature.internal.ufc; + +import static java.nio.charset.StandardCharsets.UTF_8; +import static org.junit.jupiter.api.Assertions.assertEquals; +import static org.junit.jupiter.api.Assertions.assertFalse; +import static org.junit.jupiter.api.Assertions.assertNull; +import static org.junit.jupiter.api.Assertions.assertTrue; + +import java.time.Instant; +import org.junit.jupiter.params.ParameterizedTest; +import org.junit.jupiter.params.provider.NullSource; +import org.junit.jupiter.params.provider.ValueSource; + +class RandomizationActivationTest { + @ParameterizedTest + @ValueSource(strings = {"true", "42", "[]", "{}", "\"bad\"", "\"\"", "\"2026-02-30T12:00:00Z\""}) + void preservesMalformedActivationForEvaluationPolicy(final String activation) throws Exception { + final ServerConfiguration config = parse(activation); + assertTrue(config.flags.containsKey("target")); + assertTrue(config.flags.get("target").allocations.get(0).invalidRandomizationActivation); + assertNull(config.flags.get("target").allocations.get(0).randomizationActivatedAt); + assertTrue(config.flags.containsKey("neighbor")); + } + + @ParameterizedTest + @NullSource + @ValueSource(strings = {"null"}) + void acceptsUnknownActivation(final String activation) throws Exception { + final ServerConfiguration config = parse(activation); + assertTrue(config.invalidFlags.isEmpty()); + assertNull(config.flags.get("target").allocations.get(0).randomizationActivatedAt); + assertFalse(config.flags.get("target").allocations.get(0).invalidRandomizationActivation); + } + + @ParameterizedTest + @ValueSource( + strings = {"\"2026-10-19T12:00:00.123456789Z\"", "\"2026-10-19T08:00:00.123456789-04:00\""}) + void retainsPrecisionAndOffsets(final String activation) throws Exception { + final ServerConfiguration config = parse(activation); + assertTrue(config.invalidFlags.isEmpty()); + assertEquals( + Instant.parse("2026-10-19T12:00:00.123456789Z"), + config.flags.get("target").allocations.get(0).randomizationActivatedAt); + } + + private static ServerConfiguration parse(final String activation) throws Exception { + final String json = + "{\"flags\":{" + + "\"target\":{\"key\":\"target\",\"enabled\":true,\"variationType\":\"STRING\"," + + "\"variations\":{},\"allocations\":[{\"key\":\"experiment\",\"doLog\":true" + + (activation == null ? "" : ",\"randomizationActivatedAt\":" + activation) + + "}]},\"neighbor\":{\"key\":\"neighbor\",\"enabled\":true,\"variationType\":\"STRING\"," + + "\"variations\":{},\"allocations\":[]}}}"; + return UniversalFlagConfigParser.parse(json.getBytes(UTF_8)); + } +} diff --git a/dd-openfeature/src/test/java/datadog/trace/api/openfeature/ProviderTest.java b/dd-openfeature/src/test/java/datadog/trace/api/openfeature/ProviderTest.java index f7b06ea9033..7ddca1bf1aa 100644 --- a/dd-openfeature/src/test/java/datadog/trace/api/openfeature/ProviderTest.java +++ b/dd-openfeature/src/test/java/datadog/trace/api/openfeature/ProviderTest.java @@ -39,7 +39,10 @@ void aliasOptionsKeepTheirType() { final Provider.Options options = new Provider.Options(); final Provider.Options configured = - options.initTimeout(5, SECONDS).enableEvaluationTelemetryOptOutParameter(true); + options + .initTimeout(5, SECONDS) + .enableEvaluationTelemetryOptOutParameter(true) + .experimentationOptOutMode(com.datadog.openfeature.ExperimentationOptOutMode.IMMEDIATE); assertSame(options, configured); assertEquals(5, configured.getTimeout()); diff --git a/dd-smoke-tests/openfeature/src/test/groovy/datadog/smoketest/springboot/AbstractOpenFeatureSmokeTest.groovy b/dd-smoke-tests/openfeature/src/test/groovy/datadog/smoketest/springboot/AbstractOpenFeatureSmokeTest.groovy index 147c6fa2fde..870a1adb419 100644 --- a/dd-smoke-tests/openfeature/src/test/groovy/datadog/smoketest/springboot/AbstractOpenFeatureSmokeTest.groovy +++ b/dd-smoke-tests/openfeature/src/test/groovy/datadog/smoketest/springboot/AbstractOpenFeatureSmokeTest.groovy @@ -74,34 +74,19 @@ abstract class AbstractOpenFeatureSmokeTest extends AbstractServerSmokeTest { assert !testCases.isEmpty() when: - final results = testCases.collect { - testCase -> + final results = testCases.collect { testCase -> final response = evaluate(testCase) final responseBody = new JsonSlurper().parse(response.body().byteStream()) return [testCase: testCase, response: response, body: responseBody] } - final expectedExposures = uniqueExpectedExposures(results) + final expectedExposures = buildExpectedExposures(results) then: results.every { it.response.code() == 200 } !expectedExposures.isEmpty() - new PollingConditions(timeout: 10).eventually { - final requests = evpProxyMessages*.getV2() as List> - final events = requests*.exposures.flatten() - assert events.size() == expectedExposures.size() - expectedExposures.each { - expected -> - assert events.find { - event -> - event.flag.key == expected.flag && - event.allocation.key == expected.allocation && - event.variant.key == expected.variant && - event.subject.id == expected.targetingKey - } != null : "Unable to find exposure ${expected}" - } - } + assertExposures(expectedExposures) } @Unroll("test open feature evaluation - #testCase.fileName[#testCase.index] - flag=#testCase.flag") @@ -166,9 +151,25 @@ abstract class AbstractOpenFeatureSmokeTest extends AbstractServerSmokeTest { return result } - protected List> uniqueExpectedExposures(final List> results) { + protected void assertExposures(final List> expectedExposures) { + new PollingConditions(timeout: 10).eventually { + final requests = evpProxyMessages*.getV2() as List> + final events = requests*.exposures.flatten().collect { + event -> + [flag: event.flag.key, allocation: event.allocation.key, variant: event.variant.key, + targetingKey: event.subject.id, serialId: event.serial_id] + } + assert events.countBy { + it + } == expectedExposures.countBy { + it + } + } + } + + protected List> buildExpectedExposures(final List> results) { final expected = [] - final seen = [] as Set + final previousBySubject = [:] results.each { result -> final testCase = result.testCase as Map final body = result.body as Map @@ -183,10 +184,12 @@ abstract class AbstractOpenFeatureSmokeTest extends AbstractServerSmokeTest { flag: flag, allocation: allocation, variant: variant, - targetingKey: testCase.targetingKey + targetingKey: testCase.targetingKey, + serialId: body.flagMetadata?.__dd_split_serial_id ] - final key = "${exposure.flag}\u0000${exposure.targetingKey}\u0000${exposure.allocation}\u0000${exposure.variant}" - if (seen.add(key)) { + // The cache stores the last assignment per flag and subject. A -> B -> A emits three events. + final key = [flag, testCase.targetingKey] + if (previousBySubject.put(key, exposure) != exposure) { expected.add(exposure) } } diff --git a/dd-smoke-tests/openfeature/src/test/groovy/datadog/smoketest/springboot/OpenFeatureProviderSmokeTest.groovy b/dd-smoke-tests/openfeature/src/test/groovy/datadog/smoketest/springboot/OpenFeatureProviderSmokeTest.groovy index d0808be6aea..134b4409093 100644 --- a/dd-smoke-tests/openfeature/src/test/groovy/datadog/smoketest/springboot/OpenFeatureProviderSmokeTest.groovy +++ b/dd-smoke-tests/openfeature/src/test/groovy/datadog/smoketest/springboot/OpenFeatureProviderSmokeTest.groovy @@ -82,6 +82,34 @@ class OpenFeatureProviderSmokeTest extends AbstractOpenFeatureSmokeTest { response.close() } + void 'test exposures retain allocation changes and suppress repeats'() { + setup: + publishConfiguration() + final subject = 'exposure-cache-smoke-test' + final standard = [flag: 'serial-id-targeting-flag', allocation: 'default-allocation', + variant: 'standard', targetingKey: subject, serialId: 200] + final vip = [flag: 'serial-id-targeting-flag', allocation: 'vip-allocation', + variant: 'vip', targetingKey: subject, serialId: 201] + + when: + final results = [[:], [:], [segment: 'vip'], [segment: 'vip'], [:], [:]].collect { attributes -> + final testCase = [flag: 'serial-id-targeting-flag', variationType: 'STRING', + defaultValue: 'fallback', targetingKey: subject, attributes: attributes] + final response = evaluate(testCase) + try { + assert response.code() == 200 + return [testCase: testCase, body: new JsonSlurper().parse(response.body().byteStream())] + } finally { + response.close() + } + } + final expectedExposures = buildExpectedExposures(results) + + then: + expectedExposures == [standard, vip, standard] + assertExposures(expectedExposures) + } + private static Set decodeProducts(final Map request) { return request.client.products.collect { Product.valueOf(it) } } diff --git a/dd-smoke-tests/openfeature/src/test/resources/experimentation-opt-out/README.md b/dd-smoke-tests/openfeature/src/test/resources/experimentation-opt-out/README.md new file mode 100644 index 00000000000..61803d5861d --- /dev/null +++ b/dd-smoke-tests/openfeature/src/test/resources/experimentation-opt-out/README.md @@ -0,0 +1,25 @@ +# Experimentation opt-out test cases + +`cases.json` covers 90 combinations of opt-out mode, subject timestamp, allocation activation, +and exposure logging. `ExperimentationOptOutProviderTest` runs each case through the production +parser and evaluator and checks the selected allocation or evaluation error. + +The source is [`browser-proposal.jsonl`](https://github.com/DataDog/ffe-system-test-data/blob/64ea4a312f2167c7d48c7df642ae1e5d0a9327de/contracts/experimentation-opt-out/corpus/browser-proposal.jsonl) +at `DataDog/ffe-system-test-data` revision `64ea4a312f2167c7d48c7df642ae1e5d0a9327de`. +Its SHA-256 is `4c4628ae30217bd2ead2ef34144f43e0163f9916ff0c8accae0847391dff64a6`. + +Extraction preserves source order, selects only `kind == "policy"`, removes `kind` and +`decision_ids`, and strips the `policy/` prefix from each ID. No inputs or expected results +are changed. Java's approved adoption scope includes these policy rows; the browser timestamp, +transport, and pending-state rows do not define Java behavior. Native timestamp expectations +live in `ExperimentationOptOutTest`. + +To verify both the pinned source bytes and equality of all extracted rows, save the source +file from that revision and run from this directory: + +```sh +python3 verify.py /path/to/browser-proposal.jsonl +``` + +The provider Gradle test tasks declare both this directory and the canonical fixture directory +as inputs, so fixture-only changes invalidate their cached results. diff --git a/dd-smoke-tests/openfeature/src/test/resources/experimentation-opt-out/cases.json b/dd-smoke-tests/openfeature/src/test/resources/experimentation-opt-out/cases.json new file mode 100644 index 00000000000..c9d8feb3b0f --- /dev/null +++ b/dd-smoke-tests/openfeature/src/test/resources/experimentation-opt-out/cases.json @@ -0,0 +1,1172 @@ +[ + { + "args": { + "activation": "missing", + "attribute": "absent", + "doLog": false, + "mode": "disabled" + }, + "expected": { + "error": "NONE", + "exclude": false + }, + "id": "disabled/absent/missing/0" + }, + { + "args": { + "activation": "missing", + "attribute": "absent", + "doLog": true, + "mode": "disabled" + }, + "expected": { + "error": "NONE", + "exclude": false + }, + "id": "disabled/absent/missing/1" + }, + { + "args": { + "activation": "invalid", + "attribute": "absent", + "doLog": false, + "mode": "disabled" + }, + "expected": { + "error": "NONE", + "exclude": false + }, + "id": "disabled/absent/invalid/0" + }, + { + "args": { + "activation": "invalid", + "attribute": "absent", + "doLog": true, + "mode": "disabled" + }, + "expected": { + "error": "NONE", + "exclude": false + }, + "id": "disabled/absent/invalid/1" + }, + { + "args": { + "activation": "before", + "attribute": "absent", + "doLog": false, + "mode": "disabled" + }, + "expected": { + "error": "NONE", + "exclude": false + }, + "id": "disabled/absent/before/0" + }, + { + "args": { + "activation": "before", + "attribute": "absent", + "doLog": true, + "mode": "disabled" + }, + "expected": { + "error": "NONE", + "exclude": false + }, + "id": "disabled/absent/before/1" + }, + { + "args": { + "activation": "equal", + "attribute": "absent", + "doLog": false, + "mode": "disabled" + }, + "expected": { + "error": "NONE", + "exclude": false + }, + "id": "disabled/absent/equal/0" + }, + { + "args": { + "activation": "equal", + "attribute": "absent", + "doLog": true, + "mode": "disabled" + }, + "expected": { + "error": "NONE", + "exclude": false + }, + "id": "disabled/absent/equal/1" + }, + { + "args": { + "activation": "after", + "attribute": "absent", + "doLog": false, + "mode": "disabled" + }, + "expected": { + "error": "NONE", + "exclude": false + }, + "id": "disabled/absent/after/0" + }, + { + "args": { + "activation": "after", + "attribute": "absent", + "doLog": true, + "mode": "disabled" + }, + "expected": { + "error": "NONE", + "exclude": false + }, + "id": "disabled/absent/after/1" + }, + { + "args": { + "activation": "missing", + "attribute": "valid", + "doLog": false, + "mode": "disabled" + }, + "expected": { + "error": "NONE", + "exclude": false + }, + "id": "disabled/valid/missing/0" + }, + { + "args": { + "activation": "missing", + "attribute": "valid", + "doLog": true, + "mode": "disabled" + }, + "expected": { + "error": "NONE", + "exclude": false + }, + "id": "disabled/valid/missing/1" + }, + { + "args": { + "activation": "invalid", + "attribute": "valid", + "doLog": false, + "mode": "disabled" + }, + "expected": { + "error": "NONE", + "exclude": false + }, + "id": "disabled/valid/invalid/0" + }, + { + "args": { + "activation": "invalid", + "attribute": "valid", + "doLog": true, + "mode": "disabled" + }, + "expected": { + "error": "NONE", + "exclude": false + }, + "id": "disabled/valid/invalid/1" + }, + { + "args": { + "activation": "before", + "attribute": "valid", + "doLog": false, + "mode": "disabled" + }, + "expected": { + "error": "NONE", + "exclude": false + }, + "id": "disabled/valid/before/0" + }, + { + "args": { + "activation": "before", + "attribute": "valid", + "doLog": true, + "mode": "disabled" + }, + "expected": { + "error": "NONE", + "exclude": false + }, + "id": "disabled/valid/before/1" + }, + { + "args": { + "activation": "equal", + "attribute": "valid", + "doLog": false, + "mode": "disabled" + }, + "expected": { + "error": "NONE", + "exclude": false + }, + "id": "disabled/valid/equal/0" + }, + { + "args": { + "activation": "equal", + "attribute": "valid", + "doLog": true, + "mode": "disabled" + }, + "expected": { + "error": "NONE", + "exclude": false + }, + "id": "disabled/valid/equal/1" + }, + { + "args": { + "activation": "after", + "attribute": "valid", + "doLog": false, + "mode": "disabled" + }, + "expected": { + "error": "NONE", + "exclude": false + }, + "id": "disabled/valid/after/0" + }, + { + "args": { + "activation": "after", + "attribute": "valid", + "doLog": true, + "mode": "disabled" + }, + "expected": { + "error": "NONE", + "exclude": false + }, + "id": "disabled/valid/after/1" + }, + { + "args": { + "activation": "missing", + "attribute": "invalid", + "doLog": false, + "mode": "disabled" + }, + "expected": { + "error": "NONE", + "exclude": false + }, + "id": "disabled/invalid/missing/0" + }, + { + "args": { + "activation": "missing", + "attribute": "invalid", + "doLog": true, + "mode": "disabled" + }, + "expected": { + "error": "NONE", + "exclude": false + }, + "id": "disabled/invalid/missing/1" + }, + { + "args": { + "activation": "invalid", + "attribute": "invalid", + "doLog": false, + "mode": "disabled" + }, + "expected": { + "error": "NONE", + "exclude": false + }, + "id": "disabled/invalid/invalid/0" + }, + { + "args": { + "activation": "invalid", + "attribute": "invalid", + "doLog": true, + "mode": "disabled" + }, + "expected": { + "error": "NONE", + "exclude": false + }, + "id": "disabled/invalid/invalid/1" + }, + { + "args": { + "activation": "before", + "attribute": "invalid", + "doLog": false, + "mode": "disabled" + }, + "expected": { + "error": "NONE", + "exclude": false + }, + "id": "disabled/invalid/before/0" + }, + { + "args": { + "activation": "before", + "attribute": "invalid", + "doLog": true, + "mode": "disabled" + }, + "expected": { + "error": "NONE", + "exclude": false + }, + "id": "disabled/invalid/before/1" + }, + { + "args": { + "activation": "equal", + "attribute": "invalid", + "doLog": false, + "mode": "disabled" + }, + "expected": { + "error": "NONE", + "exclude": false + }, + "id": "disabled/invalid/equal/0" + }, + { + "args": { + "activation": "equal", + "attribute": "invalid", + "doLog": true, + "mode": "disabled" + }, + "expected": { + "error": "NONE", + "exclude": false + }, + "id": "disabled/invalid/equal/1" + }, + { + "args": { + "activation": "after", + "attribute": "invalid", + "doLog": false, + "mode": "disabled" + }, + "expected": { + "error": "NONE", + "exclude": false + }, + "id": "disabled/invalid/after/0" + }, + { + "args": { + "activation": "after", + "attribute": "invalid", + "doLog": true, + "mode": "disabled" + }, + "expected": { + "error": "NONE", + "exclude": false + }, + "id": "disabled/invalid/after/1" + }, + { + "args": { + "activation": "missing", + "attribute": "absent", + "doLog": false, + "mode": "immediate" + }, + "expected": { + "error": "NONE", + "exclude": false + }, + "id": "immediate/absent/missing/0" + }, + { + "args": { + "activation": "missing", + "attribute": "absent", + "doLog": true, + "mode": "immediate" + }, + "expected": { + "error": "NONE", + "exclude": false + }, + "id": "immediate/absent/missing/1" + }, + { + "args": { + "activation": "invalid", + "attribute": "absent", + "doLog": false, + "mode": "immediate" + }, + "expected": { + "error": "NONE", + "exclude": false + }, + "id": "immediate/absent/invalid/0" + }, + { + "args": { + "activation": "invalid", + "attribute": "absent", + "doLog": true, + "mode": "immediate" + }, + "expected": { + "error": "NONE", + "exclude": false + }, + "id": "immediate/absent/invalid/1" + }, + { + "args": { + "activation": "before", + "attribute": "absent", + "doLog": false, + "mode": "immediate" + }, + "expected": { + "error": "NONE", + "exclude": false + }, + "id": "immediate/absent/before/0" + }, + { + "args": { + "activation": "before", + "attribute": "absent", + "doLog": true, + "mode": "immediate" + }, + "expected": { + "error": "NONE", + "exclude": false + }, + "id": "immediate/absent/before/1" + }, + { + "args": { + "activation": "equal", + "attribute": "absent", + "doLog": false, + "mode": "immediate" + }, + "expected": { + "error": "NONE", + "exclude": false + }, + "id": "immediate/absent/equal/0" + }, + { + "args": { + "activation": "equal", + "attribute": "absent", + "doLog": true, + "mode": "immediate" + }, + "expected": { + "error": "NONE", + "exclude": false + }, + "id": "immediate/absent/equal/1" + }, + { + "args": { + "activation": "after", + "attribute": "absent", + "doLog": false, + "mode": "immediate" + }, + "expected": { + "error": "NONE", + "exclude": false + }, + "id": "immediate/absent/after/0" + }, + { + "args": { + "activation": "after", + "attribute": "absent", + "doLog": true, + "mode": "immediate" + }, + "expected": { + "error": "NONE", + "exclude": false + }, + "id": "immediate/absent/after/1" + }, + { + "args": { + "activation": "missing", + "attribute": "valid", + "doLog": false, + "mode": "immediate" + }, + "expected": { + "error": "NONE", + "exclude": false + }, + "id": "immediate/valid/missing/0" + }, + { + "args": { + "activation": "missing", + "attribute": "valid", + "doLog": true, + "mode": "immediate" + }, + "expected": { + "error": "NONE", + "exclude": true + }, + "id": "immediate/valid/missing/1" + }, + { + "args": { + "activation": "invalid", + "attribute": "valid", + "doLog": false, + "mode": "immediate" + }, + "expected": { + "error": "NONE", + "exclude": false + }, + "id": "immediate/valid/invalid/0" + }, + { + "args": { + "activation": "invalid", + "attribute": "valid", + "doLog": true, + "mode": "immediate" + }, + "expected": { + "error": "NONE", + "exclude": true + }, + "id": "immediate/valid/invalid/1" + }, + { + "args": { + "activation": "before", + "attribute": "valid", + "doLog": false, + "mode": "immediate" + }, + "expected": { + "error": "NONE", + "exclude": false + }, + "id": "immediate/valid/before/0" + }, + { + "args": { + "activation": "before", + "attribute": "valid", + "doLog": true, + "mode": "immediate" + }, + "expected": { + "error": "NONE", + "exclude": true + }, + "id": "immediate/valid/before/1" + }, + { + "args": { + "activation": "equal", + "attribute": "valid", + "doLog": false, + "mode": "immediate" + }, + "expected": { + "error": "NONE", + "exclude": false + }, + "id": "immediate/valid/equal/0" + }, + { + "args": { + "activation": "equal", + "attribute": "valid", + "doLog": true, + "mode": "immediate" + }, + "expected": { + "error": "NONE", + "exclude": true + }, + "id": "immediate/valid/equal/1" + }, + { + "args": { + "activation": "after", + "attribute": "valid", + "doLog": false, + "mode": "immediate" + }, + "expected": { + "error": "NONE", + "exclude": false + }, + "id": "immediate/valid/after/0" + }, + { + "args": { + "activation": "after", + "attribute": "valid", + "doLog": true, + "mode": "immediate" + }, + "expected": { + "error": "NONE", + "exclude": true + }, + "id": "immediate/valid/after/1" + }, + { + "args": { + "activation": "missing", + "attribute": "invalid", + "doLog": false, + "mode": "immediate" + }, + "expected": { + "error": "INVALID_CONTEXT", + "exclude": false + }, + "id": "immediate/invalid/missing/0" + }, + { + "args": { + "activation": "missing", + "attribute": "invalid", + "doLog": true, + "mode": "immediate" + }, + "expected": { + "error": "INVALID_CONTEXT", + "exclude": false + }, + "id": "immediate/invalid/missing/1" + }, + { + "args": { + "activation": "invalid", + "attribute": "invalid", + "doLog": false, + "mode": "immediate" + }, + "expected": { + "error": "INVALID_CONTEXT", + "exclude": false + }, + "id": "immediate/invalid/invalid/0" + }, + { + "args": { + "activation": "invalid", + "attribute": "invalid", + "doLog": true, + "mode": "immediate" + }, + "expected": { + "error": "INVALID_CONTEXT", + "exclude": false + }, + "id": "immediate/invalid/invalid/1" + }, + { + "args": { + "activation": "before", + "attribute": "invalid", + "doLog": false, + "mode": "immediate" + }, + "expected": { + "error": "INVALID_CONTEXT", + "exclude": false + }, + "id": "immediate/invalid/before/0" + }, + { + "args": { + "activation": "before", + "attribute": "invalid", + "doLog": true, + "mode": "immediate" + }, + "expected": { + "error": "INVALID_CONTEXT", + "exclude": false + }, + "id": "immediate/invalid/before/1" + }, + { + "args": { + "activation": "equal", + "attribute": "invalid", + "doLog": false, + "mode": "immediate" + }, + "expected": { + "error": "INVALID_CONTEXT", + "exclude": false + }, + "id": "immediate/invalid/equal/0" + }, + { + "args": { + "activation": "equal", + "attribute": "invalid", + "doLog": true, + "mode": "immediate" + }, + "expected": { + "error": "INVALID_CONTEXT", + "exclude": false + }, + "id": "immediate/invalid/equal/1" + }, + { + "args": { + "activation": "after", + "attribute": "invalid", + "doLog": false, + "mode": "immediate" + }, + "expected": { + "error": "INVALID_CONTEXT", + "exclude": false + }, + "id": "immediate/invalid/after/0" + }, + { + "args": { + "activation": "after", + "attribute": "invalid", + "doLog": true, + "mode": "immediate" + }, + "expected": { + "error": "INVALID_CONTEXT", + "exclude": false + }, + "id": "immediate/invalid/after/1" + }, + { + "args": { + "activation": "missing", + "attribute": "absent", + "doLog": false, + "mode": "future_experiments_only" + }, + "expected": { + "error": "NONE", + "exclude": false + }, + "id": "future_experiments_only/absent/missing/0" + }, + { + "args": { + "activation": "missing", + "attribute": "absent", + "doLog": true, + "mode": "future_experiments_only" + }, + "expected": { + "error": "NONE", + "exclude": false + }, + "id": "future_experiments_only/absent/missing/1" + }, + { + "args": { + "activation": "invalid", + "attribute": "absent", + "doLog": false, + "mode": "future_experiments_only" + }, + "expected": { + "error": "NONE", + "exclude": false + }, + "id": "future_experiments_only/absent/invalid/0" + }, + { + "args": { + "activation": "invalid", + "attribute": "absent", + "doLog": true, + "mode": "future_experiments_only" + }, + "expected": { + "error": "NONE", + "exclude": false + }, + "id": "future_experiments_only/absent/invalid/1" + }, + { + "args": { + "activation": "before", + "attribute": "absent", + "doLog": false, + "mode": "future_experiments_only" + }, + "expected": { + "error": "NONE", + "exclude": false + }, + "id": "future_experiments_only/absent/before/0" + }, + { + "args": { + "activation": "before", + "attribute": "absent", + "doLog": true, + "mode": "future_experiments_only" + }, + "expected": { + "error": "NONE", + "exclude": false + }, + "id": "future_experiments_only/absent/before/1" + }, + { + "args": { + "activation": "equal", + "attribute": "absent", + "doLog": false, + "mode": "future_experiments_only" + }, + "expected": { + "error": "NONE", + "exclude": false + }, + "id": "future_experiments_only/absent/equal/0" + }, + { + "args": { + "activation": "equal", + "attribute": "absent", + "doLog": true, + "mode": "future_experiments_only" + }, + "expected": { + "error": "NONE", + "exclude": false + }, + "id": "future_experiments_only/absent/equal/1" + }, + { + "args": { + "activation": "after", + "attribute": "absent", + "doLog": false, + "mode": "future_experiments_only" + }, + "expected": { + "error": "NONE", + "exclude": false + }, + "id": "future_experiments_only/absent/after/0" + }, + { + "args": { + "activation": "after", + "attribute": "absent", + "doLog": true, + "mode": "future_experiments_only" + }, + "expected": { + "error": "NONE", + "exclude": false + }, + "id": "future_experiments_only/absent/after/1" + }, + { + "args": { + "activation": "missing", + "attribute": "valid", + "doLog": false, + "mode": "future_experiments_only" + }, + "expected": { + "error": "NONE", + "exclude": false + }, + "id": "future_experiments_only/valid/missing/0" + }, + { + "args": { + "activation": "missing", + "attribute": "valid", + "doLog": true, + "mode": "future_experiments_only" + }, + "expected": { + "error": "NONE", + "exclude": true + }, + "id": "future_experiments_only/valid/missing/1" + }, + { + "args": { + "activation": "invalid", + "attribute": "valid", + "doLog": false, + "mode": "future_experiments_only" + }, + "expected": { + "error": "PARSE_ERROR", + "exclude": false + }, + "id": "future_experiments_only/valid/invalid/0" + }, + { + "args": { + "activation": "invalid", + "attribute": "valid", + "doLog": true, + "mode": "future_experiments_only" + }, + "expected": { + "error": "PARSE_ERROR", + "exclude": false + }, + "id": "future_experiments_only/valid/invalid/1" + }, + { + "args": { + "activation": "before", + "attribute": "valid", + "doLog": false, + "mode": "future_experiments_only" + }, + "expected": { + "error": "NONE", + "exclude": false + }, + "id": "future_experiments_only/valid/before/0" + }, + { + "args": { + "activation": "before", + "attribute": "valid", + "doLog": true, + "mode": "future_experiments_only" + }, + "expected": { + "error": "NONE", + "exclude": false + }, + "id": "future_experiments_only/valid/before/1" + }, + { + "args": { + "activation": "equal", + "attribute": "valid", + "doLog": false, + "mode": "future_experiments_only" + }, + "expected": { + "error": "NONE", + "exclude": false + }, + "id": "future_experiments_only/valid/equal/0" + }, + { + "args": { + "activation": "equal", + "attribute": "valid", + "doLog": true, + "mode": "future_experiments_only" + }, + "expected": { + "error": "NONE", + "exclude": true + }, + "id": "future_experiments_only/valid/equal/1" + }, + { + "args": { + "activation": "after", + "attribute": "valid", + "doLog": false, + "mode": "future_experiments_only" + }, + "expected": { + "error": "NONE", + "exclude": false + }, + "id": "future_experiments_only/valid/after/0" + }, + { + "args": { + "activation": "after", + "attribute": "valid", + "doLog": true, + "mode": "future_experiments_only" + }, + "expected": { + "error": "NONE", + "exclude": true + }, + "id": "future_experiments_only/valid/after/1" + }, + { + "args": { + "activation": "missing", + "attribute": "invalid", + "doLog": false, + "mode": "future_experiments_only" + }, + "expected": { + "error": "INVALID_CONTEXT", + "exclude": false + }, + "id": "future_experiments_only/invalid/missing/0" + }, + { + "args": { + "activation": "missing", + "attribute": "invalid", + "doLog": true, + "mode": "future_experiments_only" + }, + "expected": { + "error": "INVALID_CONTEXT", + "exclude": false + }, + "id": "future_experiments_only/invalid/missing/1" + }, + { + "args": { + "activation": "invalid", + "attribute": "invalid", + "doLog": false, + "mode": "future_experiments_only" + }, + "expected": { + "error": "INVALID_CONTEXT", + "exclude": false + }, + "id": "future_experiments_only/invalid/invalid/0" + }, + { + "args": { + "activation": "invalid", + "attribute": "invalid", + "doLog": true, + "mode": "future_experiments_only" + }, + "expected": { + "error": "INVALID_CONTEXT", + "exclude": false + }, + "id": "future_experiments_only/invalid/invalid/1" + }, + { + "args": { + "activation": "before", + "attribute": "invalid", + "doLog": false, + "mode": "future_experiments_only" + }, + "expected": { + "error": "INVALID_CONTEXT", + "exclude": false + }, + "id": "future_experiments_only/invalid/before/0" + }, + { + "args": { + "activation": "before", + "attribute": "invalid", + "doLog": true, + "mode": "future_experiments_only" + }, + "expected": { + "error": "INVALID_CONTEXT", + "exclude": false + }, + "id": "future_experiments_only/invalid/before/1" + }, + { + "args": { + "activation": "equal", + "attribute": "invalid", + "doLog": false, + "mode": "future_experiments_only" + }, + "expected": { + "error": "INVALID_CONTEXT", + "exclude": false + }, + "id": "future_experiments_only/invalid/equal/0" + }, + { + "args": { + "activation": "equal", + "attribute": "invalid", + "doLog": true, + "mode": "future_experiments_only" + }, + "expected": { + "error": "INVALID_CONTEXT", + "exclude": false + }, + "id": "future_experiments_only/invalid/equal/1" + }, + { + "args": { + "activation": "after", + "attribute": "invalid", + "doLog": false, + "mode": "future_experiments_only" + }, + "expected": { + "error": "INVALID_CONTEXT", + "exclude": false + }, + "id": "future_experiments_only/invalid/after/0" + }, + { + "args": { + "activation": "after", + "attribute": "invalid", + "doLog": true, + "mode": "future_experiments_only" + }, + "expected": { + "error": "INVALID_CONTEXT", + "exclude": false + }, + "id": "future_experiments_only/invalid/after/1" + } +] diff --git a/dd-smoke-tests/openfeature/src/test/resources/experimentation-opt-out/verify.py b/dd-smoke-tests/openfeature/src/test/resources/experimentation-opt-out/verify.py new file mode 100644 index 00000000000..fff06c3d360 --- /dev/null +++ b/dd-smoke-tests/openfeature/src/test/resources/experimentation-opt-out/verify.py @@ -0,0 +1,37 @@ +#!/usr/bin/env python3 +"""Verify the native policy fixture against the pinned shared contract corpus.""" +import argparse +import hashlib +import json +from pathlib import Path + +SOURCE_SHA256 = "4c4628ae30217bd2ead2ef34144f43e0163f9916ff0c8accae0847391dff64a6" + + +def main(): + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument("source", type=Path, help="pinned browser-proposal.jsonl") + args = parser.parse_args() + source = args.source.read_bytes() + if hashlib.sha256(source).hexdigest() != SOURCE_SHA256: + parser.error("source corpus does not match the pinned SHA-256") + rows = [json.loads(line) for line in source.splitlines()] + expected = [ + { + key: value[len("policy/"):] if key == "id" else value + for key, value in row.items() + if key not in ("kind", "decision_ids") + } + for row in rows + if row["kind"] == "policy" + ] + actual = json.loads(Path(__file__).with_name("cases.json").read_text()) + if len(expected) != 90 or len({row["id"] for row in expected}) != 90: + parser.error("expected 90 unique policy cases") + if actual != expected: + parser.error("cases.json differs from the pinned policy extraction") + print("Verified all 90 policy cases against the pinned source") + + +if __name__ == "__main__": + main() diff --git a/dd-smoke-tests/openfeature/src/test/resources/ffe-system-test-data b/dd-smoke-tests/openfeature/src/test/resources/ffe-system-test-data index 0c467473aa8..8c4d2410aa6 160000 --- a/dd-smoke-tests/openfeature/src/test/resources/ffe-system-test-data +++ b/dd-smoke-tests/openfeature/src/test/resources/ffe-system-test-data @@ -1 +1 @@ -Subproject commit 0c467473aa8dc1f1cbfc0ef9a60d5c1c7a8826e6 +Subproject commit 8c4d2410aa643aa6b97c8547ae5fe358c46b7f6f diff --git a/dd-trace-api/src/main/java/datadog/trace/api/config/FeatureFlaggingConfig.java b/dd-trace-api/src/main/java/datadog/trace/api/config/FeatureFlaggingConfig.java index a2e6a5378f4..a14f4adbf5c 100644 --- a/dd-trace-api/src/main/java/datadog/trace/api/config/FeatureFlaggingConfig.java +++ b/dd-trace-api/src/main/java/datadog/trace/api/config/FeatureFlaggingConfig.java @@ -28,5 +28,13 @@ public final class FeatureFlaggingConfig { public static final String FEATURE_FLAGS_EVALUATION_TELEMETRY_PER_SUBJECT_OPT_OUT_ENABLED = "feature.flags.evaluation.telemetry.per.subject.opt.out.enabled"; + /** + * Lets the {@code dd_experimentation_opt_out_at} evaluation context attribute opt subjects out of + * experimentation. Maps to {@code DD_FEATURE_FLAGS_EXPERIMENTATION_OPT_OUT_MODE}. Accepted + * values: {@code disabled} (default), {@code immediate}, and {@code future_experiments_only}. + */ + public static final String FEATURE_FLAGS_EXPERIMENTATION_OPT_OUT_MODE = + "feature.flags.experimentation.opt.out.mode"; + private FeatureFlaggingConfig() {} } diff --git a/metadata/supported-configurations.json b/metadata/supported-configurations.json index 4b6663c7fe0..4b71411c15d 100644 --- a/metadata/supported-configurations.json +++ b/metadata/supported-configurations.json @@ -1612,6 +1612,14 @@ "aliases": [] } ], + "DD_FEATURE_FLAGS_EXPERIMENTATION_OPT_OUT_MODE": [ + { + "version": "A", + "type": "string", + "default": "disabled", + "aliases": [] + } + ], "DD_FEATURE_FLAGS_EXPOSURES_DATADOG_LOGGING_ENABLED": [ { "version": "A",