From 8895efccb4dd53fbee2e9684381e83c17f9382fe Mon Sep 17 00:00:00 2001 From: Simon Schrottner Date: Mon, 2 Nov 2026 20:40:31 +0100 Subject: [PATCH 1/5] feat: the flagd scenarios as Provider TCK extensions Adds tck-extensions/, the gherkin/ scenarios rewritten in the Provider TCK's step vocabulary, so every flagd provider adopting the TCK runs the same files instead of porting its own copy. Java ports them first, in open-feature/java-sdk-contrib#1951. gherkin/ is unchanged. The scenario-size limits go up by one step: the TCK form sets the reconnect options as explicit steps where gherkin/ folds them into its provider step. Signed-off-by: Simon Schrottner --- .gherkin-lintrc | 4 +- tck-extensions/README.md | 17 + tck-extensions/connection.feature | 118 +++++ tck-extensions/context-enrichment.feature | 73 +++ tck-extensions/disabled.feature | 39 ++ tck-extensions/evaluation.feature | 76 ++++ tck-extensions/events.feature | 56 +++ tck-extensions/fractional.feature | 525 ++++++++++++++++++++++ tck-extensions/metadata.feature | 58 +++ tck-extensions/rpc-caching.feature | 68 +++ tck-extensions/selector.feature | 27 ++ tck-extensions/sync-payload.feature | 40 ++ tck-extensions/targeting.feature | 222 +++++++++ 13 files changed, 1321 insertions(+), 2 deletions(-) create mode 100644 tck-extensions/README.md create mode 100644 tck-extensions/connection.feature create mode 100644 tck-extensions/context-enrichment.feature create mode 100644 tck-extensions/disabled.feature create mode 100644 tck-extensions/evaluation.feature create mode 100644 tck-extensions/events.feature create mode 100644 tck-extensions/fractional.feature create mode 100644 tck-extensions/metadata.feature create mode 100644 tck-extensions/rpc-caching.feature create mode 100644 tck-extensions/selector.feature create mode 100644 tck-extensions/sync-payload.feature create mode 100644 tck-extensions/targeting.feature diff --git a/.gherkin-lintrc b/.gherkin-lintrc index 36b2d0c..5679dc5 100644 --- a/.gherkin-lintrc +++ b/.gherkin-lintrc @@ -49,8 +49,8 @@ "on", { "steps-length": { - "Background": 3, - "Scenario": 16 + "Background": 4, + "Scenario": 17 } } ] diff --git a/tck-extensions/README.md b/tck-extensions/README.md new file mode 100644 index 0000000..f0bf22c --- /dev/null +++ b/tck-extensions/README.md @@ -0,0 +1,17 @@ +# TCK extensions + +The scenarios in `gherkin/`, rewritten as extensions for the +[OpenFeature Provider TCK](https://github.com/open-feature/spec/issues/417). A flagd provider that +adopts the TCK runs these alongside the TCK's own scenarios, instead of running `gherkin/` through a +harness of its own. + +They ask only what the TCK's canonical scenarios do not. Each file's header names its source feature, +the scenarios it leaves out, and why; most of those the TCK already asks. + +They are written in the TCK's step vocabulary (`a stable provider`, `a context with the following +keys:`), plus flagd-specific steps an adopter supplies as extension glue: provider options +(`an option "" of type "" with value ""`) and connection control. Feature tags +name the resolvers a scenario applies to (`@rpc`, `@in-process`, `@file`), as in `gherkin/`. + +`gherkin/` stays as it is for SDKs that still run it directly. A scenario added there should be +added here too, or listed as not ported in the matching file's header. diff --git a/tck-extensions/connection.feature b/tck-extensions/connection.feature new file mode 100644 index 0000000..e769fa1 --- /dev/null +++ b/tck-extensions/connection.feature @@ -0,0 +1,118 @@ +@ext-connection +Feature: TCK extension: flagd provider disconnect and reconnect + + # Ported from gherkin/connection.feature. + # + # Not ported: the "stable" row of the Connection outline, which every canonical evaluation + # scenario asks, and the @unixsocket rows, which need a socket shared between host and container + # that the TCK stack does not mount. The testbed's "Provider unavailable" is kept although the + # canonical lifecycle.feature asks the same, because it holds flagd to a 3000ms bound where the + # canonical scenario allows 10000ms. + # + # The outlines over provider flavours are spelled out, one scenario per flavour, because a flavour + # here is a set of steps rather than a value. Reconnect timing is the testbed's "stable flagd + # provider", as in events.feature. + # + # @targetURI and @forbidden connect through the envoy sidecar in the testbed's + # test-harness/docker-compose.yaml. + + @rpc @in-process @file @customCert + Scenario: Connection with a custom certificate + Given an option "tls" of type "Boolean" with value "true" + And an option "certPath" of type "String" with value "test-harness/ssl/custom-root-cert.crt" + And a stable provider with "ssl" configuration + And a Boolean-flag with key "boolean-flag" and a default value "false" + When the flag was evaluated with details + Then the resolved details value should be "true" + + @rpc @in-process @file @reconnect + Scenario: Provider reconnection + Given an option "retryGracePeriod" of type "Integer" with value "5" + And an option "retryBackoffMaxMs" of type "Integer" with value "500" + And a stable provider + And a ready event handler + And a error event handler + When a ready event was fired + When the connection is lost + Then the error event handler should have been executed + When the connection is restored + Then the ready event handler should have been executed + + @rpc @in-process @file @reconnect @customCert + Scenario: Provider reconnection with a custom certificate + Given an option "tls" of type "Boolean" with value "true" + And an option "certPath" of type "String" with value "test-harness/ssl/custom-root-cert.crt" + And an option "retryGracePeriod" of type "Integer" with value "5" + And an option "retryBackoffMaxMs" of type "Integer" with value "500" + And a stable provider with "ssl" configuration + And a ready event handler + And a error event handler + When a ready event was fired + When the connection is lost + Then the error event handler should have been executed + When the connection is restored + Then the ready event handler should have been executed + + @rpc @in-process @file @unavailable + Scenario: Provider unavailable + Given an option "deadlineMs" of type "Integer" with value "1000" + And a unavailable provider + And a error event handler + Then the error event handler should have been executed within 3000ms + + # A forbidden response from flagd results in a fatal client state. "a unavailable provider" is the + # TCK's flavour that does not wait for readiness; the step before it points that provider at envoy. + @rpc @in-process @forbidden + Scenario: Provider forbidden + Given an option "fatalStatusCodes" of type "StringList" with value "PERMISSION_DENIED" + And the provider connects to envoy's forbidden listener + And a unavailable provider + And a error event handler + Then the error event handler should have been executed within 5000ms + And the client should be in fatal state + + @targetURI @rpc + Scenario: Connection via TargetUri rpc + Given a target URI "envoy://localhost:/rpc.service" routed through envoy + And a stable provider + And a Boolean-flag with key "boolean-flag" and a default value "false" + When the flag was evaluated with details + Then the resolved details value should be "true" + + @targetURI @in-process + Scenario: Connection via TargetUri in-process + Given a target URI "envoy://localhost:/sync.service" routed through envoy + And a stable provider + And a Boolean-flag with key "boolean-flag" and a default value "false" + When the flag was evaluated with details + Then the resolved details value should be "true" + + @targetURI @in-process @reconnect + Scenario: Re-Connection via TargetUri in-process + Given a target URI "envoy://localhost:/sync.service" routed through envoy + And an option "retryGracePeriod" of type "Integer" with value "5" + And an option "retryBackoffMaxMs" of type "Integer" with value "500" + And a stable provider + And a ready event handler + And a error event handler + When a ready event was fired + When the connection is lost + Then the error event handler should have been executed + When the connection is restored + Then the ready event handler should have been executed + + @targetURI @in-process @reconnect @grace + Scenario: Re-Connection via TargetUri in-process with grace period + Given a target URI "envoy://localhost:/sync.service" routed through envoy + And an option "retryGracePeriod" of type "Integer" with value "5" + And an option "retryBackoffMaxMs" of type "Integer" with value "500" + And a stable provider + And a ready event handler + And a stale event handler + And a error event handler + When a ready event was fired + When the connection is lost + Then the stale event handler should have been executed + Then the error event handler should have been executed + When the connection is restored + Then the ready event handler should have been executed diff --git a/tck-extensions/context-enrichment.feature b/tck-extensions/context-enrichment.feature new file mode 100644 index 0000000..5e4a147 --- /dev/null +++ b/tck-extensions/context-enrichment.feature @@ -0,0 +1,73 @@ +@ext-metadata @contextEnrichment +Feature: TCK extension: Context Enrichment + + # Ported from gherkin/contextEnrichment.feature. The testbed's Background "a stable + # flagd provider" moves into each scenario as the TCK's "a stable provider", because the @grace + # scenarios record options that have to come before it. + # + # The @grace scenarios restore the testbed provider's retry timing (grace period 5s, backoff + # 500ms capped at 500ms) over the adoption's (grace 2s, backoff capped at the 5000ms default), so + # the provider is back within the event timeout once the connection is restored. The RPC one also + # restores the testbed's 1000ms deadline: at the adoption's 15000ms an evaluation issued while + # flagd is down could outlast the outage and succeed, which is not what it asks. + # + # The testbed's @caching is renamed @flagd-caching: the TCK reserves @caching and refuses a + # scenario that carries it. + + @in-process @rpc + Scenario: Use enriched context + Given a stable provider + And a String-flag with key "flagd-context-aware" and a default value "not" + When the flag was evaluated with details + Then the resolved details value should be "INTERNAL" + + @grace @in-process + Scenario: Use enriched context on connection error for IN-PROCESS + Given an option "retryGracePeriod" of type "Integer" with value "5" + And an option "retryBackoffMs" of type "Integer" with value "500" + And an option "retryBackoffMaxMs" of type "Integer" with value "500" + And a stable provider + And a String-flag with key "flagd-context-aware" and a default value "not" + And a stale event handler + And a ready event handler + When the flag was evaluated with details + Then the resolved details value should be "INTERNAL" + When the connection is lost + And a stale event was fired + When the flag was evaluated with details + Then the resolved details value should be "INTERNAL" + When the connection is restored + And a ready event was fired + + @grace @rpc + Scenario: Use enriched context on connection error for RPC + Given an option "deadlineMs" of type "Integer" with value "1000" + And an option "retryGracePeriod" of type "Integer" with value "5" + And an option "retryBackoffMs" of type "Integer" with value "500" + And an option "retryBackoffMaxMs" of type "Integer" with value "500" + And a stable provider + And a String-flag with key "flagd-context-aware" and a default value "not" + And a stale event handler + And a ready event handler + When the flag was evaluated with details + Then the resolved details value should be "INTERNAL" + When the connection is lost + And a stale event was fired + When the flag was evaluated with details + Then the resolved details value should be "not" + When the connection is restored + And a ready event was fired + + @rpc @flagd-caching + Scenario: Use enriched context on RPC connection will not cache the value + Given a stable provider + And a String-flag with key "flagd-context-aware" and a default value "not" + And a change event handler + And a ready event handler + When the flag was modified + And a change event was fired + And the flag was evaluated with details + Then the reason should be "TARGETING_MATCH" + # ensure that we do not cache a "TARGETING_MATCH", we should only cache evaluation with a "STATIC" reason + When the flag was evaluated with details + Then the reason should be "TARGETING_MATCH" diff --git a/tck-extensions/disabled.feature b/tck-extensions/disabled.feature new file mode 100644 index 0000000..71ba385 --- /dev/null +++ b/tck-extensions/disabled.feature @@ -0,0 +1,39 @@ +@ext-evaluation @disabled @in-process @rpc +Feature: TCK extension: Disabled flag evaluation + + # Ported from gherkin/disabled.feature. A flag with state=DISABLED resolves + # successfully with reason=DISABLED; the evaluator substitutes the caller-provided default value + # and omits the variant. Relates to: https://github.com/open-feature/flagd/issues/1965 + # + # The TCK's @disabled-flags scenarios ask the value for the four scalar flags and the reason for + # disabled-boolean-flag only, so the boolean row is left out here: the remaining rows add the + # DISABLED reason for each type and the structured flag. + + Scenario Outline: Evaluating a disabled flag returns reason DISABLED and code default + Given an option "cache" of type "CacheType" with value "disabled" + And a stable provider + And a -flag with key "" and a default value "" + When the flag was evaluated with details + Then the resolved details value should be "" + And the reason should be "DISABLED" + + Examples: + | key | type | default | + | disabled-string-flag | String | bye | + | disabled-integer-flag | Integer | 1 | + | disabled-float-flag | Float | 0.1 | + | disabled-object-flag | Object | {} | + + Scenario Outline: Flag disabled in one flag set, enabled in another + Given an option "cache" of type "CacheType" with value "disabled" + And an option "selector" of type "String" with value "" + And a stable provider + And a Boolean-flag with key "cross-flagset-flag" and a default value "false" + When the flag was evaluated with details + Then the resolved details value should be "" + And the reason should be "" + + Examples: + | selector | resolved_value | reason | + | flags/allFlags.json | false | DISABLED | + | rawflags/selector-flags.json | true | STATIC | diff --git a/tck-extensions/evaluation.feature b/tck-extensions/evaluation.feature new file mode 100644 index 0000000..690bee8 --- /dev/null +++ b/tck-extensions/evaluation.feature @@ -0,0 +1,76 @@ +@ext-evaluation @rpc @in-process @file +Feature: TCK extension: flagd evaluations + + # Ported from gherkin/evaluation.feature. Asks what the TCK's evaluation.feature + # and reason.feature do not: the reason flagd reports for a zero value, a float zero value, + # zero values served by a targeting rule, and flags with no default variant. + # + # Not ported: "Resolve values" (the TCK's evaluation.feature asks it), the @deprecated + # @no-default outline, which @no-default-variant replaces, and the two @precision scenarios, + # "Resolves numeric values without loss of precision" and "Resolves an integer beyond 32 bits + # without loss of precision", which the TCK's own precision scenarios ask. + + Background: + Given an option "cache" of type "CacheType" with value "disabled" + And a stable provider + + Scenario Outline: Resolves zero value + Given a -flag with key "" and a default value "" + When the flag was evaluated with details + Then the resolved details value should be "" + And the reason should be "STATIC" + + Examples: + | key | type | default | resolved_value | + | boolean-zero-flag | Boolean | true | false | + | string-zero-flag | String | hi | | + | integer-zero-flag | Integer | 1 | 0 | + | float-zero-flag | Float | 0.1 | 0.0 | + + @targeting + Scenario Outline: Resolves zero value with targeting + Given a -flag with key "" and a default value "" + And a context containing a key "email", with type "String" and with value "ballmer@macrosoft.com" + When the flag was evaluated with details + Then the resolved details value should be "" + And the reason should be "TARGETING_MATCH" + + Examples: + | key | type | default | resolved_value | + | boolean-targeted-zero-flag | Boolean | true | false | + | string-targeted-zero-flag | String | hi | | + | integer-targeted-zero-flag | Integer | 1 | 0 | + | float-targeted-zero-flag | Float | 0.1 | 0.0 | + + @targeting + Scenario Outline: Resolves zero value with targeting using default + Given a -flag with key "" and a default value "" + And a context containing a key "email", with type "String" and with value "ballmer@none.com" + When the flag was evaluated with details + Then the resolved details value should be "" + And the reason should be "DEFAULT" + + Examples: + | key | type | default | resolved_value | + | boolean-targeted-zero-flag | Boolean | true | false | + | string-targeted-zero-flag | String | hi | | + | integer-targeted-zero-flag | Integer | 1 | 0 | + | float-targeted-zero-flag | Float | 0.1 | 0.0 | + + @no-default-variant + Scenario Outline: Resolves flag with no default variant correctly + Given a -flag with key "" and a default value "" + And a context containing a key "email", with type "String" and with value "" + When the flag was evaluated with details + Then the resolved details value should be "" + And the reason should be "" + + Examples: + | key | type | email | code_default | resolved_value | reason | + | null-default-flag | Boolean | | true | true | DEFAULT | + | null-default-flag | Boolean | | false | false | DEFAULT | + | undefined-default-flag | Integer | | 100 | 100 | DEFAULT | + | no-default-flag-null-targeting-variant | String | wozniak@orange.com | Inventor | Inventor | DEFAULT | + | no-default-flag-null-targeting-variant | String | wozniak@orange.com | Founder | Founder | DEFAULT | + | no-default-flag-null-targeting-variant | String | jobs@orange.com | CEO | CEO | TARGETING_MATCH | + | no-default-flag-undefined-targeting-variant | String | wozniak@orange.com | Retired | Retired | DEFAULT | diff --git a/tck-extensions/events.feature b/tck-extensions/events.feature new file mode 100644 index 0000000..702c218 --- /dev/null +++ b/tck-extensions/events.feature @@ -0,0 +1,56 @@ +@ext-connection @rpc @in-process @file @events +Feature: TCK extension: flagd provider state changes + + # Ported from gherkin/events.feature. Not ported: "Flag change event", which the + # canonical events.feature asks in "A configuration change is signalled and applied", together + # with the changed evaluation it leaves out. + # + # These chains ask what the retry grace period decides, and they ask it by sequence rather than by + # clock: stale -> ready when the connection returns inside the grace period, stale -> error -> ready + # when the error assertion has first waited the grace period out. The testbed timed the outage + # instead (the launchpad's /restart), which made the same question depend on a 1s or 8s wall clock. + # + # Each "should have been executed" claims the record it matched and leaves the rest, so the next + # assertion waits for a newer event of its own kind while an event of another kind stays visible + # to whatever asks about it later. + # + # Timing is the testbed's "stable flagd provider" except for the deadline, which stays at the + # adoption's 15000ms for the in-process sync (see AbstractResolverTest): a 5s grace period, and a + # retry backoff capped at 500ms so reconnecting does not wait out an exponential backoff. + + Background: + Given an option "retryGracePeriod" of type "Integer" with value "5" + And an option "retryBackoffMaxMs" of type "Integer" with value "500" + And a stable provider + + Scenario: Provider events chain ready -> error -> ready + Given a ready event handler + And a error event handler + Then the ready event handler should have been executed + When the connection is lost + Then the error event handler should have been executed + When the connection is restored + Then the ready event handler should have been executed + + @grace + Scenario: Provider events chain ready -> stale -> error -> ready + Given a ready event handler + And a error event handler + And a stale event handler + Then the ready event handler should have been executed + When the connection is lost + Then the stale event handler should have been executed + Then the error event handler should have been executed + When the connection is restored + Then the ready event handler should have been executed + + @grace + Scenario: Provider events chain ready -> stale -> ready + Given a ready event handler + And a error event handler + And a stale event handler + Then the ready event handler should have been executed + When the connection is lost + Then the stale event handler should have been executed + When the connection is restored + Then the ready event handler should have been executed with no error event before it diff --git a/tck-extensions/fractional.feature b/tck-extensions/fractional.feature new file mode 100644 index 0000000..7114839 --- /dev/null +++ b/tck-extensions/fractional.feature @@ -0,0 +1,525 @@ +@ext-fractional @rpc @in-process @file +Feature: TCK extension: Fractional targeting + + # Ported from the gherkin/targeting.feature: every @fractional scenario, plus the + # fractional @operator-errors ones. The TCK's own @targeting scenarios do not cover the + # fractional operator. + # + # @fractional-vN tags select the bucketing version. The suites run v3, which the Java in-process + # evaluator and the testbed's flagd implement, and exclude @fractional-v2 with @TckExcludeTags. + # Untagged scenarios must hold for every version. + # The @fractional-v1 blocks are left out: no backend here implements v1 bucketing. + # + # Where the testbed's context values are Objects or integers outside 32 bits, the TCK's context + # step cannot convert them, so those outlines use the table step with the testbed's conversion. + + Background: + Given an option "cache" of type "CacheType" with value "disabled" + And a stable provider + + @fractional + Scenario Outline: Fractional operator + Given a String-flag with key "fractional-flag" and a default value "fallback" + And a context containing a nested property with outer key "user" and inner key "name", with value "" + When the flag was evaluated with details + Then the resolved details value should be "" + + @fractional-v2 + Examples: v2 + | name | value | + | jack | hearts | + | queen | spades | + | ten | clubs | + | nine | diamonds | + | 3 | clubs | + + @fractional-v3 + Examples: v3 + | name | value | + | jack | diamonds | + | queen | diamonds | + | ten | clubs | + | nine | clubs | + | 3 | spades | + + @fractional + Scenario Outline: Fractional operator shorthand + Given a String-flag with key "fractional-flag-shorthand" and a default value "fallback" + And a context containing a targeting key with value "" + When the flag was evaluated with details + Then the resolved details value should be "" + + @fractional-v2 + Examples: v2 + | targeting key | value | + | jon@company.com | heads | + | jane@company.com | tails | + + @fractional-v3 + Examples: v3 + | targeting key | value | + | jon@company.com | tails | + | jane@company.com | tails | + + @fractional @fractional-v2 @fractional-single-entry + Scenario: Fractional operator with single entry always resolves to the only variant + Given a String-flag with key "fractional-single-entry-flag" and a default value "fallback" + And a context containing a targeting key with value "some-targeting-key" + When the flag was evaluated with details + Then the resolved details value should be "single" + + @fractional + Scenario Outline: Fractional operator with shared seed + Given a String-flag with key "fractional-flag-A-shared-seed" and a default value "fallback" + And a context containing a nested property with outer key "user" and inner key "name", with value "" + When the flag was evaluated with details + Then the resolved details value should be "" + + @fractional-v2 + Examples: v2 + | name | value | + | seven | hearts | + | eight | diamonds | + | nine | clubs | + | two | spades | + + @fractional-v3 + Examples: v3 + | name | value | + | seven | hearts | + | eight | hearts | + | nine | diamonds | + | two | diamonds | + + @fractional + Scenario Outline: Second fractional operator with shared seed + Given a String-flag with key "fractional-flag-B-shared-seed" and a default value "fallback" + And a context containing a nested property with outer key "user" and inner key "name", with value "" + When the flag was evaluated with details + Then the resolved details value should be "" + + @fractional-v2 + Examples: v2 + | name | value | + | seven | ace-of-hearts | + | eight | ace-of-diamonds | + | nine | ace-of-clubs | + | two | ace-of-spades | + + @fractional-v3 + Examples: v3 + | name | value | + | seven | ace-of-hearts | + | eight | ace-of-hearts | + | nine | ace-of-diamonds | + | two | ace-of-diamonds | + + # Hash edge-case vectors — keys chosen by brute-force search so their + # MurmurHash3-x86-32 (seed=0) falls at the six critical boundary values. + # All keys are exact 6-char MurmurHash3-x86-32 (seed=0) preimages found by exhaustive search. + # ejOoVL → hash=0 EXACT → bv(100)=0 → lower + # bY9fO- → hash=1 EXACT → bv(100)=0 → lower + # SI7p- → hash=2147483647 EXACT i32::MAX → bv(100)=49 → lower + # 6LvT0 → hash=2147483648 EXACT i32::MIN u32 → bv(100)=50 → upper + # ceQdGm → hash=4294967295 EXACT u32::MAX → bv(100)=99 → upper + @fractional @fractional-v2 + Scenario Outline: Fractional operator hash edge cases + Given a String-flag with key "fractional-hash-edge-flag" and a default value "fallback" + And a context containing a targeting key with value "" + When the flag was evaluated with details + Then the resolved details value should be "" + Examples: + | key | value | + | ejOoVL | lower | + | bY9fO- | lower | + | SI7p- | lower | + | 6LvT0 | upper | + | ceQdGm | upper | + + # Nested JSON Logic expressions as bucket variant names. + # Requires providers to support the @fractional-nested feature. + # Use -t "not @fractional-nested" to exclude during transition. + @fractional @fractional-nested + Scenario Outline: Fractional operator with nested if expression as variant name + # fractional-nested-if-flag: seed=targetingKey, bucket0=[if(tier=="premium","premium","standard"),50], bucket1=["standard",50] + # jon@company.com bv(100)=36 → bucket0; user1 bv(100)=76 → bucket1 + Given a String-flag with key "fractional-nested-if-flag" and a default value "fallback" + And a context containing a targeting key with value "" + And a context containing a key "tier", with type "String" and with value "" + When the flag was evaluated with details + Then the resolved details value should be "" + Examples: + | targetingKey | tier | value | + | jon@company.com | premium | premium | + | jon@company.com | basic | standard | + | user1 | premium | standard | + | user1 | basic | standard | + + @fractional @fractional-nested + Scenario Outline: Fractional operator with nested var expression as variant name + # fractional-nested-var-flag: seed=targetingKey, bucket0=[var("color"),50], bucket1=["blue",50] + # jon@company.com bv(100)=36 → bucket0 (resolves var "color"); user1 bv(100)=76 → bucket1 ("blue") + Given a String-flag with key "fractional-nested-var-flag" and a default value "fallback" + And a context containing a targeting key with value "" + And a context containing a key "color", with type "String" and with value "" + When the flag was evaluated with details + Then the resolved details value should be "" + Examples: + | targetingKey | color | value | + | jon@company.com | red | red | + | jon@company.com | green | green | + | user1 | red | blue | + | jon@company.com | yellow | fallback | + | jon@company.com | | fallback | + + @fractional @fractional-nested + Scenario Outline: Fractional operator with nested if expression as weight + # fractional-nested-weight-flag: seed=targetingKey, bucket0=["red",if(tier=="premium",100,0)], bucket1=["blue",10] + Given a String-flag with key "fractional-nested-weight-flag" and a default value "fallback" + And a context containing a targeting key with value "" + And a context containing a key "tier", with type "String" and with value "" + When the flag was evaluated with details + Then the resolved details value should be "" + Examples: + | targetingKey | tier | value | + | jon@company.com | premium | red | + | jon@company.com | basic | blue | + | user1 | premium | red | + | user1 | basic | blue | + + @fractional @fractional-nested + Scenario: Fractional as condition + Given a String-flag with key "fractional-as-condition-flag" and a default value "zero" + And a context containing a targeting key with value "some-targeting-key" + When the flag was evaluated with details + Then the resolved details value should be "hundreds" + + @fractional @fractional-nested + Scenario: Fractional as condition evaluates false path + Given a String-flag with key "fractional-as-condition-false-flag" and a default value "zero" + And a context containing a targeting key with value "some-targeting-key" + When the flag was evaluated with details + Then the resolved details value should be "ones" + + # fractional composes like any other operator + @fractional @fractional-nested @fractional-v3 + Scenario Outline: Fractional with null bucket as whole rule + Given a String-flag with key "fractional-null-bucket-flag" and a default value "code-default" + And a context containing a targeting key with value "" + When the flag was evaluated with details + Then the resolved details value should be "" + And the reason should be "" + And the error-code should be "" + Examples: + | targetingKey | value | reason | + | user2 | fallback | DEFAULT | + | user1 | a | TARGETING_MATCH | + + @fractional @fractional-nested @fractional-v3 + Scenario Outline: Fractional with null bucket as if condition + Given a String-flag with key "fractional-null-bucket-condition-flag" and a default value "code-default" + And a context containing a targeting key with value "" + When the flag was evaluated with details + Then the resolved details value should be "" + And the reason should be "TARGETING_MATCH" + Examples: + | targetingKey | value | + | user1 | else | + | user2 | then | + + @fractional @fractional-nested @fractional-v3 + Scenario Outline: Fractional with boolean buckets as whole rule + Given a String-flag with key "fractional-bool-bucket-flag" and a default value "code-default" + And a context containing a targeting key with value "" + When the flag was evaluated with details + Then the resolved details value should be "" + And the variant should be "" + And the reason should be "TARGETING_MATCH" + Examples: + | targetingKey | value | variant | + | user1 | on | true | + | user2 | off | false | + + @fractional @fractional-nested @fractional-v3 + Scenario Outline: Fractional result consumed by another operator + Given a String-flag with key "fractional-cat-flag" and a default value "code-default" + And a context containing a targeting key with value "" + When the flag was evaluated with details + Then the resolved details value should be "" + And the reason should be "TARGETING_MATCH" + Examples: + | targetingKey | value | + | user2 | big | + | user1 | dig | + + @fractional @fractional-nested @fractional-v3 + Scenario Outline: Fractional as bucketing input of another fractional + Given a String-flag with key "fractional-of-fractional-flag" and a default value "code-default" + And a context containing a targeting key with value "" + When the flag was evaluated with details + Then the resolved details value should be "" + And the reason should be "TARGETING_MATCH" + Examples: + | targetingKey | value | + | user2 | red | + | user3 | red | + | user1 | blue | + + @operator-errors + Scenario: fractional operator with missing bucket key falls back to default variant + Given a String-flag with key "fractional-null-bucket-key-flag" and a default value "wrong" + When the flag was evaluated with details + Then the resolved details value should be "fallback" + And the reason should be "DEFAULT" + + @operator-errors @fractional + Scenario: fractional with all-zero bucket weights falls back to default variant + Given a String-flag with key "fractional-zero-weights-flag" and a default value "wrong" + And a context containing a targeting key with value "any-user" + When the flag was evaluated with details + Then the resolved details value should be "fallback" + And the reason should be "DEFAULT" + + @operator-errors @fractional + Scenario: fractional negative bucket weight is clamped to zero + # ["one", -50] is treated as ["one", 0]; "two" gets 100% of the weight + Given a String-flag with key "fractional-negative-weight-flag" and a default value "wrong" + And a context containing a targeting key with value "any-user" + When the flag was evaluated with details + Then the resolved details value should be "two" + + @fractional @fractional-v3 + Scenario Outline: Fractional operator with basic types + Given a String-flag with key "fractional-basic-flag" and a default value "fallback" + And a context containing a key "hashing_input", with type "" and with value "" + When the flag was evaluated with details + Then the resolved details value should be "" + + Examples: v3 + | hashing_input | type | value | + | true | Boolean | bucket5 | + | false | Boolean | bucket1 | + | user1 | String | bucket22 | + | user2 | String | bucket16 | + | 123 | Integer | bucket20 | + | 456 | Integer | bucket12 | + | 1.23 | Float | bucket14 | + | 4.56 | Float | bucket18 | + | 123 | String | bucket22 | + | true | String | bucket24 | + | false | String | bucket1 | + | null | String | bucket8 | + | 1.23 | String | bucket4 | + | 0 | Integer | bucket8 | + + @fractional @fractional-v3 + Scenario Outline: Fractional operator with float mapping + Given a String-flag with key "fractional-basic-flag" and a default value "fallback" + And a context with the following keys: + | key | type | value | + | hashing_input | | | + When the flag was evaluated with details + Then the resolved details value should be "" + + Examples: v3 + | hashing_input | type | value | + | 1.0 | Float | bucket23 | + | 1 | Integer | bucket23 | + | 1.0000000000000001 | Float | bucket23 | + | -2.0 | Float | bucket12 | + | -2 | Integer | bucket12 | + | 9007199254740992.0 | Float | bucket10 | + | 9007199254740992 | Integer | bucket10 | + + @fractional @fractional-v3 + Scenario Outline: Fractional operator with zero values + Given a String-flag with key "fractional-basic-flag" and a default value "fallback" + And a context containing a key "hashing_input", with type "" and with value "" + When the flag was evaluated with details + Then the resolved details value should be "" + + Examples: v3 + | hashing_input | type | value | + | 0.0 | Float | bucket8 | + | -0.0 | Float | bucket8 | + | 0 | Integer | bucket8 | + + @fractional @fractional-v3 + Scenario Outline: Fractional operator with integer limits + Given a String-flag with key "fractional-basic-flag" and a default value "fallback" + And a context with the following keys: + | key | type | value | + | hashing_input | Integer | | + When the flag was evaluated with details + Then the resolved details value should be "" + + Examples: v3 + | hashing_input | value | + | 2147483647 | bucket10 | + | 2147483648 | bucket25 | + | -2147483648 | bucket8 | + | -2147483649 | bucket18 | + | 9007199254740991 | bucket2 | + | 23 | bucket21 | + | 24 | bucket11 | + | 255 | bucket13 | + | 256 | bucket13 | + | 65535 | bucket4 | + | 65536 | bucket11 | + | 4294967295 | bucket14 | + | 4294967296 | bucket19 | + | -24 | bucket4 | + | -25 | bucket16 | + + @fractional @fractional-v3 + Scenario Outline: Fractional operator with floats limits + Given a String-flag with key "fractional-basic-flag" and a default value "fallback" + And a context containing a key "hashing_input", with type "Float" and with value "" + When the flag was evaluated with details + Then the resolved details value should be "" + + Examples: v3 + | hashing_input | value | + | 1e20 | bucket19 | + | -1e20 | bucket25 | + | 65504.0 | bucket16 | + | -65504.0 | bucket9 | + | 0.00006103515625 | bucket22 | + | 0.000000059604644775390625 | bucket23 | + | 65505.0 | bucket17 | + | -65505.0 | bucket17 | + | 3.4028234663852886e+38 | bucket10 | + | -3.4028234663852886e+38 | bucket3 | + | 1.1754943508222875e-38 | bucket24 | + | 3.5e+38 | bucket1 | + | -3.5e+38 | bucket6 | + | 1.7976931348623157e+308 | bucket17 | + | -1.7976931348623157e+308 | bucket9 | + | 2.2250738585072014e-308 | bucket11 | + | 4.9406564584124654e-324 | bucket2 | + + @fractional @fractional-v3 + Scenario Outline: Fractional operator with map key ordering + Given a String-flag with key "fractional-basic-flag" and a default value "fallback" + And a context with the following keys: + | key | type | value | + | hashing_input | Object | | + When the flag was evaluated with details + Then the resolved details value should be "" + + Examples: v3 + | hashing_input | value | + | {"a": 1, "b": 2} | bucket23 | + | {"b": 2, "a": 1} | bucket23 | + | {"a": 1, "b": 2, "c": 3} | bucket18 | + | {"c": 3, "a": 1, "b": 2} | bucket18 | + | {"z": 1, "aa": 2} | bucket21 | + | {"aa": 2, "z": 1} | bucket21 | + | {"a": {"b": 1}, "b": 2} | bucket9 | + | {"b": 2, "a": {"b": 1}} | bucket9 | + | {"a": true, "bbb": 1, "c": "text", "dddd": 2.5} | bucket25 | + | {"c": "text", "bbb": 1, "a": true, "dddd": 2.5} | bucket25 | + | {"aaa": 1, "ÿ": 2} | bucket12 | + | {"ÿ": 2, "aaa": 1} | bucket12 | + | {"": 2, "abc": 1} | bucket25 | + | {"key": -0.0} | bucket10 | + | {"key": 0} | bucket10 | + | {"key": 0.0} | bucket10 | + | {"b": -0.0, "a": {"b": 1}, "c": 0.0} | bucket8 | + | {"c": 0.0, "b": -0.0, "a": {"b": 1}} | bucket8 | + | {"café": "façade", "b": {"résumé": {"ÿ": true}}} | bucket22 | + + @fractional @fractional-v3 + Scenario Outline: Fractional operator with advanced structures + Given a String-flag with key "fractional-basic-flag" and a default value "fallback" + And a context with the following keys: + | key | type | value | + | hashing_input | Object | | + When the flag was evaluated with details + Then the resolved details value should be "" + + Examples: v3 + | hashing_input | value | + | {} | bucket3 | + | {"a": {} } | bucket20 | + | {"b": 2, "a": {} } | bucket20 | + | {"a": {}, "b": {}} | bucket21 | + | {"a": {}, "b": []} | bucket7 | + | {"c": 0.0, "a": []} | bucket18 | + | {"c": 0.0, "b": -0.0, "a": {"b": []}} | bucket9 | + | {"a": [] } | bucket12 | + | {"a": [1, [], 3], "b": null} | bucket3 | + | {"a": ["a", "b", {}], "b": [1, null, 2]} | bucket23 | + | {"a": [1, "two", true, [], null]} | bucket23 | + | {"a": [false, 2.5, ""], "b": [], "c": [{}], "c": null} | bucket15 | + | {"x": [{"a": 1}, {"b": 2}, {"b": null}]} | bucket3 | + | {"x": [{"a": {"b": []}}] } | bucket16 | + | {"x": [{"a": {"b": null}}] } | bucket6 | + | {"x": [{"a": {"b": {}}}] } | bucket2 | + | {"": "", "": [], "": {}, "": null} | bucket25 | + | {"": {"": ""}, "": ["", {"": ""}], "": {"": {"": ""}}} | bucket11 | + | {"a": [[[[[]]]]] } | bucket15 | + | {"a": [[[[null]]]] } | bucket7 | + | {"a": {"b": {"c": {"d": {}}} } } | bucket21 | + | {"a": {"b": {"c": {"d": null}} } } | bucket19 | + + @fractional @fractional-v3 + Scenario Outline: Fractional operator with string length boundaries + Given a String-flag with key "fractional-basic-flag" and a default value "fallback" + And a context containing a key "hashing_input", with type "String" and with value "" + When the flag was evaluated with details + Then the resolved details value should be "" + + Examples: v3 + | hashing_input | value | + | | bucket4 | + | a | bucket6 | + | 12345678901234567890123 | bucket4 | + | 123456789012345678901234 | bucket14 | + | aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa | bucket18 | + | aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa | bucket7 | + + @fractional @fractional-v3 + Scenario Outline: Valid well-formed UTF-8 String consistency across different languages + Given a String-flag with key "fractional-basic-flag" and a default value "fallback" + And a context containing a key "hashing_input", with type "String" and with value "" + When the flag was evaluated with details + Then the resolved details value should be "" + + Examples: v3 + | hashing_input | value | + | あいうえおかき | bucket17 | + | あいうえおかきく | bucket16 | + | ééééééééééé | bucket16 | + | éééééééééééé | bucket19 | + | café façade résumé café | bucket9 | + | A\u0000B | bucket15 | + | e\u0301 | bucket7 | + | \uD83D\uDE00 | bucket23 | + + @fractional @fractional-v3 + Scenario Outline: Fractional operator using implicit targeting key + Given a String-flag with key "fractional-flag-shorthand" and a default value "draw" + And a context containing a targeting key with value "" + When the flag was evaluated with details + Then the resolved details value should be "" + + Examples: v3 + | targeting_key | value | + | user-1 | heads | + | user-2 | tails | + | user-3 | heads | + + @fractional @fractional-v3 + Scenario Outline: Fractional operator invalid implicit inputs + Given a String-flag with key "fractional-flag-shorthand" and a default value "fallback" + And a context containing a key "", with type "" and with value "" + When the flag was evaluated with details + Then the resolved details value should be "draw" + Then the reason should be "DEFAULT" + + Examples: v3 + | key | type | value | + | irrelevant | String | none | + | targetingKey | Integer | 12345 | + | targetingKey | Boolean | true | diff --git a/tck-extensions/metadata.feature b/tck-extensions/metadata.feature new file mode 100644 index 0000000..92b7ddf --- /dev/null +++ b/tck-extensions/metadata.feature @@ -0,0 +1,58 @@ +@ext-metadata @in-process @file @metadata +Feature: TCK extension: flag and flag set metadata + + # Ported from gherkin/metadata.feature. The testbed's "a stable flagd provider" + # becomes the TCK's "a stable provider", and its "a metadata flagd provider" becomes "a stable + # provider with "metadata" configuration". The two scenarios a file resolver cannot serve name a + # selector flag file and carry @metadata-provider. + + Scenario: Returns metadata + Given a stable provider + And a Boolean-flag with key "metadata-flag" and a default value "true" + When the flag was evaluated with details + Then the resolved metadata should contain + | key | metadata_type | value | + | string | String | a | + | integer | Integer | 1 | + | float | Float | 1.2 | + | boolean | Boolean | true | + + @metadata-provider + Scenario: Returns flag set metadata + Given an option "selector" of type "String" with value "rawflags/selector-flag-combined-metadata.json" + And a stable provider with "metadata" configuration + And a Boolean-flag with key "only-set-metadata-flag" and a default value "true" + When the flag was evaluated with details + Then the resolved metadata should contain + | key | metadata_type | value | + | string | String | b | + | integer | Integer | 2 | + | float | Float | 2.2 | + | boolean | Boolean | false | + | flag-set-string | String | c | + | flag-set-integer | Integer | 3 | + | flag-set-float | Float | 3.2 | + | flag-set-boolean | Boolean | false | + + @metadata-provider + Scenario: Flag metadata overwrites flag set metadata + Given an option "selector" of type "String" with value "rawflags/selector-flag-combined-metadata.json" + And a stable provider with "metadata" configuration + And a Boolean-flag with key "combined-metadata-flag" and a default value "true" + When the flag was evaluated with details + Then the resolved metadata should contain + | key | metadata_type | value | + | string | String | a | + | integer | Integer | 1 | + | float | Float | 1.2 | + | boolean | Boolean | true | + | flag-set-string | String | c | + | flag-set-integer | Integer | 3 | + | flag-set-float | Float | 3.2 | + | flag-set-boolean | Boolean | false | + + Scenario: Returns no metadata + Given a Boolean-flag with key "boolean-flag" and a default value "true" + And a stable provider + When the flag was evaluated with details + Then the resolved metadata is empty diff --git a/tck-extensions/rpc-caching.feature b/tck-extensions/rpc-caching.feature new file mode 100644 index 0000000..d6e6e2c --- /dev/null +++ b/tck-extensions/rpc-caching.feature @@ -0,0 +1,68 @@ +@ext-connection @rpc @flagd-caching +Feature: TCK extension: Flag evaluation with caching + + # Ported from gherkin/rpc-caching.feature (RPC only). The testbed's @caching tag is + # a reserved TCK capability, which the TCK refuses on any scenario, so this feature carries + # @flagd-caching instead. + # + # Reconnect timing is the testbed's "stable flagd provider", as in events.feature. + + Background: + Given an option "cache" of type "CacheType" with value "lru" + And an option "retryGracePeriod" of type "Integer" with value "5" + And an option "retryBackoffMaxMs" of type "Integer" with value "500" + And a stable provider + + Scenario Outline: Resolves details with caching + Given a -flag with key "" and a default value "" + When the flag was evaluated with details + Then the resolved details value should be "" + And the variant should be "" + And the reason should be "STATIC" + When the flag was evaluated with details + Then the resolved details value should be "" + And the variant should be "" + And the reason should be "CACHED" + + Examples: + | key | type | default | resolved_variant | resolved_value | + | boolean-flag | Boolean | false | on | true | + | string-flag | String | bye | greeting | hi | + | integer-flag | Integer | 1 | ten | 10 | + | float-flag | Float | 0.1 | half | 0.5 | + | object-flag | Object | {} | template | {"showImages": true, "title": "Check out these pics!", "imagesPerPage": 100.0 } | + + Scenario: Flag change event with caching + Given a String-flag with key "changing-flag" and a default value "false" + And a change event handler + When the flag was modified + And a change event was fired + And the flag was evaluated with details + Then the reason should be "STATIC" + When the flag was evaluated with details + Then the reason should be "CACHED" + When the flag was modified + And a change event was fired + And the flag was evaluated with details + Then the reason should be "STATIC" + When the flag was evaluated with details + Then the reason should be "CACHED" + + Scenario: Stale and Error stage + Given a ready event handler + And a stale event handler + And a error event handler + And a Boolean-flag with key "boolean-flag" and a default value "false" + When the flag was evaluated with details + Then the reason should be "STATIC" + When the flag was evaluated with details + Then the reason should be "CACHED" + When the connection is lost + And a stale event was fired + And the flag was evaluated with details + Then the reason should be "CACHED" + When a error event was fired + And the connection is restored + And a ready event was fired + And the flag was evaluated with details + Then the reason should be "STATIC" diff --git a/tck-extensions/selector.feature b/tck-extensions/selector.feature new file mode 100644 index 0000000..6a1508a --- /dev/null +++ b/tck-extensions/selector.feature @@ -0,0 +1,27 @@ +@ext-evaluation @in-process +Feature: TCK extension: flagd selector + + # Ported from gherkin/selector.feature. The launchpad's default configuration + # serves both flags/allFlags.json and rawflags/selector-flags.json, so the selector narrows the + # sync to the second source. + + Background: + Given an option "selector" of type "String" with value "rawflags/selector-flags.json" + And a stable provider + + Scenario Outline: Flags not found + Given a -flag with key "" and a default value "" + When the flag was evaluated with details + Then the reason should be "ERROR" + + Examples: + | key | type | default | + | boolean-flag | Boolean | false | + | string-flag | String | bye | + | integer-flag | Integer | 1 | + | float-flag | Float | 0.1 | + + Scenario: Resolve values + Given a String-flag with key "selector-flag" and a default value "foo" + When the flag was evaluated with details + Then the reason should be "STATIC" diff --git a/tck-extensions/sync-payload.feature b/tck-extensions/sync-payload.feature new file mode 100644 index 0000000..e29468d --- /dev/null +++ b/tck-extensions/sync-payload.feature @@ -0,0 +1,40 @@ +@ext-metadata @in-process @sync-payload +Feature: TCK extension: Disabled Sync Metadata + + # Ported from gherkin/sync-payload.feature. The testbed's "a syncpayload flagd + # provider" becomes "a stable provider with "sync-payload" configuration", moved from the + # Background into each scenario so the @grace one can record options first. + # + # The @grace scenario restores the testbed provider's retry timing (grace period 5s, backoff + # 500ms capped at 500ms) over the adoption's (grace 2s, backoff capped at the 5000ms default), so + # the provider is back within the event timeout once the connection is restored. + + Scenario: Use enriched context + Given a stable provider with "sync-payload" configuration + And a String-flag with key "flagd-context-aware" and a default value "not" + When the flag was evaluated with details + Then the resolved details value should be "INTERNAL" + + @grace + Scenario: Use enriched context on connection error for IN-PROCESS + Given an option "retryGracePeriod" of type "Integer" with value "5" + And an option "retryBackoffMs" of type "Integer" with value "500" + And an option "retryBackoffMaxMs" of type "Integer" with value "500" + And a stable provider with "sync-payload" configuration + And a String-flag with key "flagd-context-aware" and a default value "not" + And a stale event handler + And a ready event handler + When the flag was evaluated with details + Then the resolved details value should be "INTERNAL" + When the connection is lost + And a stale event was fired + When the flag was evaluated with details + Then the resolved details value should be "INTERNAL" + When the connection is restored + And a ready event was fired + + Scenario: Resolve value independent of context + Given a stable provider with "sync-payload" configuration + And a Boolean-flag with key "boolean-flag" and a default value "false" + When the flag was evaluated with details + Then the resolved details value should be "true" diff --git a/tck-extensions/targeting.feature b/tck-extensions/targeting.feature new file mode 100644 index 0000000..1d47525 --- /dev/null +++ b/tck-extensions/targeting.feature @@ -0,0 +1,222 @@ +@ext-targeting @rpc @in-process @file +Feature: TCK extension: Targeting rules + + # Ported from gherkin/targeting.feature: the json-evaluation scenarios the Provider + # TCK does not ask. The @fractional scenarios live in their own extension feature. + # + # Not ported: + # - "Targeting by targeting key": the TCK's evaluation.feature (@targeting) asserts both values + # and reason.feature asserts TARGETING_MATCH and DEFAULT for the same flag and keys. + # - "fractional operator with missing bucket key": a fractional scenario, despite its tags. + # + # Deviations from the testbed's wording: + # - "a stable flagd provider" collides with the TCK's "a {} provider", so the provider step is the + # TCK's "a stable provider". + # - "Time-based operations" sets the context with "a context with the following keys:", because the + # TCK's context step parses Integer as 32 bits and 4133980802 does not fit. + + Background: + Given an option "cache" of type "CacheType" with value "disabled" + And a stable provider + + # evaluator refs + Scenario Outline: Evaluator reuse + Given a String-flag with key "" and a default value "fallback" + And a context containing a key "email", with type "String" and with value "ballmer@macrosoft.com" + When the flag was evaluated with details + Then the resolved details value should be "" + Examples: + | key | value | + | some-email-targeted-flag | hi | + | some-other-email-targeted-flag | yes | + + @evaluator-refs @evaluator-refs-whitespace + Scenario Outline: Evaluator $ref resolves regardless of whitespace around the colon + Given a String-flag with key "" and a default value "fallback" + And a context containing a key "email", with type "String" and with value "" + When the flag was evaluated with details + Then the resolved details value should be "" + And the reason should be "TARGETING_MATCH" + Examples: + | key | email | value | + | ref-whitespace-compact-flag | ballmer@macrosoft.com | hi | + | ref-whitespace-space-after-colon-flag | ballmer@macrosoft.com | hi | + | ref-whitespace-space-around-colon-flag | ballmer@macrosoft.com | hi | + | ref-whitespace-compact-flag | user@example.com | bye | + | ref-whitespace-space-after-colon-flag | user@example.com | bye | + | ref-whitespace-space-around-colon-flag | user@example.com | bye | + + @evaluator-refs @non-existent-evaluator-ref + Scenario: Ref to nonexistent evaluator yields parse error + Given a String-flag with key "ref-to-nonexistent-evaluator-flag" and a default value "fallback" + When the flag was evaluated with details + Then the resolved details value should be "fallback" + And the error-code should be "PARSE_ERROR" + + @string + Scenario Outline: Substring operators + Given a String-flag with key "starts-ends-flag" and a default value "fallback" + And a context containing a key "id", with type "String" and with value "" + When the flag was evaluated with details + Then the resolved details value should be "" + Examples: + | id | value | + | abcdef | prefix | + | uvwxyz | postfix | + | abcxyz | prefix | + | lmnopq | none | + | 3 | none | + + @semver + Scenario Outline: Semantic version operator numeric comparison + Given a String-flag with key "equal-greater-lesser-version-flag" and a default value "fallback" + And a context containing a key "version", with type "String" and with value "" + When the flag was evaluated with details + Then the resolved details value should be "" + Examples: + | version | value | + | 2.0.0 | equal | + | 2.1.0 | greater | + | 1.9.0 | lesser | + | 2.0.0-alpha | lesser | + | 2.0.0.0 | none | + + @semver + Scenario Outline: Semantic version operator semantic comparison + Given a String-flag with key "major-minor-version-flag" and a default value "fallback" + And a context containing a key "version", with type "String" and with value "" + When the flag was evaluated with details + Then the resolved details value should be "" + Examples: + | version | value | + | 3.0.1 | minor | + | 3.1.0 | major | + | 4.0.0 | none | + + @semver @semver-edge-cases @semver-v-prefix + Scenario Outline: sem_ver v-prefix handling + Given a String-flag with key "semver-v-prefix-flag" and a default value "fallback" + And a context containing a key "version", with type "String" and with value "" + When the flag was evaluated with details + Then the resolved details value should be "" + Examples: + | version | value | + | 1.0.0 | match | + | v1.0.0 | match | + | V1.0.0 | match | + | 2.0.0 | no-match | + + @semver @semver-edge-cases @semver-partial-version + Scenario Outline: sem_ver partial version string handling + Given a String-flag with key "semver-partial-version-flag" and a default value "fallback" + And a context containing a key "version", with type "" and with value "" + When the flag was evaluated with details + Then the resolved details value should be "" + Examples: + | version | type | value | + | 1.5.0 | String | match | + | 1.0.0 | String | match | + | 1.0 | String | match | + | 1 | String | match | + | 1 | Integer | match | + | 1 | Float | match | + | 1.2 | Float | match | + | 2.0.0 | String | no-match | + + @semver @semver-edge-cases @semver-numeric-context + Scenario Outline: sem_ver numeric context value coercion + Given a String-flag with key "semver-numeric-context-flag" and a default value "fallback" + And a context containing a key "version", with type "" and with value "" + When the flag was evaluated with details + Then the resolved details value should be "" + Examples: + | version | type | value | + | 1.2 | Float | match | + | 1.1 | Float | no-match | + | 2 | Integer | match | + | 1 | Integer | no-match | + | 1.2 | String | match | + + @semver @semver-edge-cases @semver-build-metadata + Scenario Outline: sem_ver build metadata ignored + Given a String-flag with key "semver-build-metadata-flag" and a default value "fallback" + And a context containing a key "version", with type "String" and with value "" + When the flag was evaluated with details + Then the resolved details value should be "" + Examples: + | version | value | + | 1.0.0 | match | + | 1.0.0+other | match | + | 2.0.0 | no-match | + + Scenario Outline: Time-based operations + Given a Integer-flag with key "timestamp-flag" and a default value "0" + And a context with the following keys: + | key | type | value | + | time | Integer |