diff --git a/CHANGELOG.md b/CHANGELOG.md index 8930e51..e271c29 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,31 @@ All notable changes to tagged releases are documented here. ## Unreleased +- Add a DRAFT-RFC PROTOTYPE of the specification's RFC 0008 (Draft), bounded collection quantifiers, + behind the new `judgment-pack experimental evaluate --rfc0008-quantifiers` flag (ADR-0009, under + ADR-0007's experimental umbrella). The flag admits three condition operators — `exists`, `every`, + and `uniform` — with the RFC's element re-rooting, its pinned empty-array values, its + aggregate-depth bound of two, and a candidate work-accounting model the RFC itself leaves open. + Every successful evaluation payload produced this way carries a new output member, + `draftPrototype`, naming the operators used and stating that a pack using one is NOT valid under + any published JPS version; a refusal carries no such member, because a grammar or work-limit + failure is reported through the ordinary operational-error envelope. Four diagnostic codes are + minted, `codeStability: "provisional"` like every other: `JPS-EVALUATION-RFC0008-GRAMMAR`, + `JPS-EVALUATION-RFC0008-DEPTH`, `JPS-EVALUATION-RFC0008-SHAPE`, and + `JPS-RESOURCE-EVALUATION-WORK-LIMIT`. **No conformance claim and no validation behavior changes**: + `spec validate` is untouched and still rejects every pack using an operator, the evaluator without + the flag refuses one for the same reason, the MCP surface does not expose the flag, and everything + the draft grammar does not add is still held to full document conformance through the pack's Core + projection. The operators belong to an open proposal that may never be accepted. +- Make §7.4 equality total in the EXPERIMENTAL evaluator. JSON numbers are compared as normalized + tokens — sign, significant digits, adjusted exponent — rather than through arbitrary-precision + arithmetic, so `1e3`, `1000`, and `1.0e3` are one value, `-0` equals `0`, and a pair no arithmetic + type can hold (`1e999999999` against `2e999999999`) compares `false` instead of degrading to + `unknown`. `equals`, `not-equals`, and `in` therefore always decide, and `uniform` needs no arm + beyond RFC 0008's five clauses: an evaluator's inability to represent an admitted value is a + resource condition, not a semantics two implementations would have to agree on. Ordered comparison + is unchanged — still defined only over §2.2 decimal strings — and `spec validate`, the conformance + corpus, and the exit classes are untouched. - Add the MCP `prompts` capability (ADR-0008): three non-normative authoring-method prompts — `author_pack`, `test_pack`, `fix_pack` — served as static, versioned text that the client's model executes with the client's key. The server still calls no model, holds no key, and opens no diff --git a/README.md b/README.md index c3bccbe..12e5f51 100644 --- a/README.md +++ b/README.md @@ -20,6 +20,15 @@ defines no evaluator conformance class. Its one evaluation surface is explicitly specification's §§7–8 experiment per [ADR-0007](docs/adr/0007-experimental-evaluator.md), claims no conformance, and may change or be removed without compatibility promise. +That command carries one further opt-in, `--rfc0008-quantifiers`, which is a **draft-RFC +prototype** per [ADR-0009](docs/adr/0009-draft-rfc-quantifier-prototype.md): it admits the +collection quantifiers `exists`, `every`, and `uniform` proposed by the specification's RFC 0008 +(Draft). Those operators belong to no published JPS version. A pack using one is **not valid** under +JPS `0.1.0-draft`, `spec validate` rejects it, the evaluator without the flag refuses it, and every +successful evaluation payload produced under the flag says so in band through a `draftPrototype` +member — a refusal is an operational error and carries none. The flag is CLI only; the MCP tool does +not expose it. + The command binary is `judgment-pack`; release archives also ship a `jpack` short alias for the same program. @@ -33,6 +42,7 @@ judgment-pack spec schema judgment-pack spec examples [name] judgment-pack mcp judgment-pack experimental evaluate (EXPERIMENTAL; no conformance claim) +judgment-pack experimental evaluate --rfc0008-quantifiers (DRAFT-RFC PROTOTYPE) ``` The namespace is `judgment-pack spec`, not `judgment-pack jps`. JPS remains the name of the diff --git a/docs/adr/0009-draft-rfc-quantifier-prototype.md b/docs/adr/0009-draft-rfc-quantifier-prototype.md new file mode 100644 index 0000000..0863545 --- /dev/null +++ b/docs/adr/0009-draft-rfc-quantifier-prototype.md @@ -0,0 +1,148 @@ +--- +status: proposed # proposed | accepted | deprecated | superseded by NNNN +date: 2026-07-27 +deciders: Brian Jin +--- + +# Prototype the specification's RFC 0008 quantifiers behind an opt-in flag on the experimental evaluator + +## Context and problem statement + +The specification published RFC 0008 (Draft), bounded collection quantifiers, and names this +runtime's experimental evaluator as the natural prototype bed. The RFC cannot advance without +implementation experience: its Implementation section asks for two independent implementations, and +its own Specification section leaves the limit-accounting model undefined and calls producing one an +acceptance precondition. Nothing can be learned about a grammar nobody has run. But the operators +belong to no published JPS version — a pack using one is structurally non-conforming under +`0.1.0-draft`, whose `$defs/condition` is a closed `oneOf` — so admitting them anywhere near the +default path would mean this runtime evaluating packs the same runtime's validator rejects. + +## Decision drivers + +- Generate RFC 0008's implementation evidence and, in particular, a concrete candidate for the + accounting model it leaves open, including whether the model is even usable. +- Never contaminate the conformance surface: `spec validate` behavior, claims, and exit classes stay + byte-identical, and no pack becomes valid that was not valid before. +- Make the prototype impossible to reach by accident and impossible to mistake for a standard, in + both output formats and in the artifact a machine reads. +- Hold everything the draft grammar does not add to full document conformance anyway, so the opt-in + buys three operators and not a validation hole. + +## Considered options + +- **A. An opt-in flag on the existing experimental evaluator**, with an in-band prototype marker. +- **B. A separate experimental binary, branch, or repository.** +- **C. Wait for RFC 0008 to be accepted, or for its accounting model to be written.** +- **D. Admit the operators unconditionally in the experimental evaluator.** + +## Decision outcome + +Chosen option: **A**. C is circular in the same way ADR-0007's option C was, and doubly so here: the +accounting model is a precondition for acceptance and can only be argued from an implementation. D +would make a non-conformant pack evaluable on the surface's default path, which the whole ADR-0007 +guardrail structure exists to prevent. B fragments distribution for no isolation gain — the +guardrail is the flag and the marker, not a second artifact. + +Settled constraints: + +- **Surface:** CLI `judgment-pack experimental evaluate --rfc0008-quantifiers` only. The MCP tool + does not expose the flag; a prototype grammar is not something an agent should reach through a + tool description. +- **Semantics:** RFC 0008's Specification section as written — the current condition root and its + restoration per level, `uniform`'s `at` rooted in each member, the empty-array values (`exists` + false, `every` true) as pinned choices, `uniform`'s five ordered clauses with clause 3 before + clause 4, and short-circuiting on the dominant value only, never on `unknown`. The five clauses + need no sixth arm, and this record no longer proposes one. An earlier revision produced `unknown` + for two *resolved* `at` values whose §7.4 equality the arithmetic could not settle, and proposed + folding that case into clause 4 as an RFC amendment; the proposal is withdrawn, because §7.4 + equality is *total* over carrier-valid JSON once it is decided the right way. Numbers are compared + as normalized tokens — sign, significant digits, adjusted exponent — which settles `1e999999999` + against `2e999999999` in twenty bytes rather than the gigabyte materializing them would need, and + makes `1e3 == 1000 == 1.0e3` and `-0 == 0` fall out of the same rule. An implementation's + inability to represent an admitted value is a resource condition, reported as an error; it is not + a semantics, and pinning one would have made a disposition depend on the evaluator's number type + rather than on the documents — the Python prototype, comparing the same pair by value, already + answered `false` where this runtime answered `unknown`. Ordering is untouched: it remains defined + only over §2.2 decimal strings, where "incomparable values produce unknown" still applies. +- **Grammar:** the aggregate-depth bound of two, structural rather than syntactic, enforced by a + depth-indexed check whose declarative twin is committed as a testdata schema artifact. That check + owns aggregate shape and depth and nothing else; the Core validity of a `where` is delegated to + the untouched `0.1.0-draft` validator run over the pack's Core projection. +- **Limits:** a candidate accounting model, invented here because RFC 0008 defines none and stated + in full in the code — a work unit, a preflight charge complete before any element of a condition + tree is evaluated and invariant under any permutation of the elements, ragged nesting charged as + Σᵢ|Bᵢ|, Boolean branches a short-circuiting evaluator never reaches charged anyway, deep equality + charged by the size of both values compared, `uniform` charged per member *plus a reread term*, + siblings additive. That term is `(n-1) × max` over the `n` resolved `at`-values, and it is what + the per-member charge alone missed: §7.4 equality is total, so one representative absorbs the + members it equals and there are `n-1` comparisons rather than `n²/2`, but the representative is + itself one of the values and each of those comparisons can read the whole of it — one long token + among short equal ones was work proportional to `long × members` priced at `long + members`. The + maximum is taken over the *set* of resolved values rather than over the elected representative, so + the term bounds the pass whichever member is elected and is the same number under any permutation. + The evaluator spends less than the bound, canonicalizing each value's number tokens once so that + repeated comparisons compare canonical forms, but the charge is the bound. A unit is + byte-sensitive wherever the processing it stands for is: a pointer costs its path's bytes to + compile — charged before the scan runs, and once per distinct authored pointer, because the + compiled form is then cached — plus one step per reference token and those tokens' bytes per + resolution, so a pointer of a hundred empty reference tokens costs a hundred traversals rather + than one; a scalar costs its token length, and `evidence-present` costs the bytes of the + requirement id it looks up, so a long path, a long operand, or a long identifier cannot buy + unbounded work for one flat unit. Both *sides* of a comparison are priced, which is why the + preflight resolves a `fact`'s pointer and not only an aggregate's: the operand is the half the + pack's author wrote, and charging it alone left the half + the facts document supplies free — a two-byte operand compared against a megabyte of runtime JSON, + once per element. A `fact` therefore costs its pointer plus the value that pointer selected plus + the authored operand, and `in` charges the selected value once per candidate. This is the third + candidate model, and each predecessor was retired by a demonstrated attack — the flat pointer + charge, then the unpriced selected value — rather than by taste; the third has since been repaired + twice under the same discipline, once for the unpriced repeated representative comparison above + and once for the flat per-resolution pointer step. The charge is *not* duplication-invariant: a + duplicated element is one more element and is charged like one. Only the condition's value is + duplication-invariant, and only while both inputs fit the limits, which is + what RFC 0008's result-invariance wording says. Exhaustion is an explicit evaluation error + (`JPS-RESOURCE-EVALUATION-WORK-LIMIT`), never a disposition. The budget is also this runtime's §10 + collection-size limit, which RFC 0008 raises to a MUST: the bound is derived, not a second knob. +- **Labeling:** every successful evaluation payload carries a `draftPrototype` member naming the + operators used and denying validity under the published `specVersion`, and both output formats say + the same thing — including for a pack that used no draft operator, which the marker reports as + unchanged rather than invalid. A refusal is not such a payload: a grammar, projection, or + work-limit failure is reported through the ordinary operational-error envelope, which carries no + `draftPrototype` member and names the draft codes instead. +- **What does not change:** `spec validate`, the conformance corpus, the exit classes, the MCP + surface, and the evaluator without the flag — with one exception, which the CHANGELOG records as + its own entry: the §7.4 equality decision above applies unconditionally, flag or no flag, so an + `equals`, `not-equals`, or `in` over a pair of numbers no arithmetic type can hold now decides + where the experimental evaluator previously produced `unknown`. Everything else the flag buys is + behind the flag. No conformance claim of any kind is made or implied; Core §3.4 forbids one under + `0.1.0-draft` whatever is implemented. + +This is a decision under [0007](0007-experimental-evaluator.md)'s umbrella, not an amendment to it: +the surface, the labeling discipline, and the no-claim posture are ADR-0007's, and this record adds +one opt-in inside them. + +### Consequences + +- Good, because RFC 0008 gains its first implementation, a concrete accounting model to argue with, + and executable conformance rows for the corpus its Conformance section describes. +- Good, because the equivalence check the RFC asks for — re-encoding the three census facts a bare + quantifier reaches (`A6:/reservation/anySegmentCancelledByAirline`, + `R3:/modification/allNewItemsAvailable`, and `R5:/request/allNewItemsAvailable`) and confirming the + dispositions match their prepared-boolean twins — runs as a committed test rather than as a claim, + with a distinct pair of pack fixtures per fact. +- Bad, because a runtime that refuses a pack in one command and evaluates it in another is a + confusing thing to explain; mitigated by the flag being explicit, the marker being unavoidable, + and the refusal without the flag being unchanged. +- Bad, because the accounting model is invented rather than specified, so another implementation + will disagree with it. That disagreement is the evidence, not a defect. +- Revisit when RFC 0008 is accepted, rejected, or superseded, or when a `specVersion` publishing the + operators exists — at which point this stops being a prototype and becomes ordinary validation. + +## More information + +Semantics source: the specification's +[RFC 0008 (Draft)](https://github.com/Judgment-Pack/judgment-pack-spec/blob/main/rfcs/0008-bounded-collection-quantifiers.md), +its dependency [RFC 0006 (Draft)](https://github.com/Judgment-Pack/judgment-pack-spec/blob/main/rfcs/0006-evaluator-conformance.md), +and JPS Core §§7–8 and §10. Implementation: `internal/evaluation/rfc0008.go` (the grammar gate and +the Core projection), `internal/evaluation/quantifier.go` (the operators and the accounting model), +and their testdata fixtures. Follows [0007](0007-experimental-evaluator.md). diff --git a/docs/adr/README.md b/docs/adr/README.md index 6dd6056..e13690c 100644 --- a/docs/adr/README.md +++ b/docs/adr/README.md @@ -100,3 +100,4 @@ authority, and following it confers no conformance status on anything. | [0006](0006-authoring-lifecycle-in-the-client.md) | The authoring lifecycle lives in the client; the runtime is a stateless oracle | accepted | | [0007](0007-experimental-evaluator.md) | Ship an experimental evaluator behind an explicit experimental surface | accepted | | [0008](0008-mcp-prompts-authoring-method.md) | Serve authoring method as MCP prompts; the intelligence stays in the client | accepted | +| [0009](0009-draft-rfc-quantifier-prototype.md) | Prototype spec RFC 0008 quantifiers behind an opt-in experimental flag | proposed | diff --git a/docs/architecture.md b/docs/architecture.md index 8c035a1..8fe5427 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -52,7 +52,7 @@ evaluation exists only behind the explicitly experimental surface of - `internal/fssecure` opens selected local files defensively and enforces bounded regular-file reads. - `internal/result` defines machine output version 1 and exit classes. - `internal/describe` composes the machine descriptions the CLI and MCP server share, so neither drifts. -- `internal/evaluation` implements the EXPERIMENTAL, non-conformance-claiming JPS §§7–8 evaluator (ADR-0007). +- `internal/evaluation` implements the EXPERIMENTAL, non-conformance-claiming JPS §§7–8 evaluator (ADR-0007), and behind a further CLI opt-in the draft-RFC prototype of the specification's RFC 0008 collection quantifiers (ADR-0009), whose packs no published JPS version accepts. - `internal/cli` owns commands, streams, and human/JSON rendering. - `internal/mcp` adapts the offline core onto Model Context Protocol tools over stdio. - `tools/sync-spec-artifacts` is an explicit maintainer-only snapshot importer. diff --git a/internal/cli/app.go b/internal/cli/app.go index 8226958..7526a1b 100644 --- a/internal/cli/app.go +++ b/internal/cli/app.go @@ -114,10 +114,11 @@ func (a *App) evaluateCommand() *cobra.Command { factsPath := "" evidencePath := "" supported := []string{} + quantifiers := false command := &cobra.Command{ Use: "evaluate ", Short: "EXPERIMENTAL: apply the JPS §§7-8 experiment to one pack; no conformance claim", - Long: "Apply the experimental JPS Core §§7-8 resolution model, as pinned by the specification's RFC 0006 (Draft), to one conformant pack and one facts document. The result is a disposition, not a conformance claim, an authorization, or an executed action; producing any disposition exits 0.", + Long: "Apply the experimental JPS Core §§7-8 resolution model, as pinned by the specification's RFC 0006 (Draft), to one conformant pack and one facts document. The result is a disposition, not a conformance claim, an authorization, or an executed action; producing any disposition exits 0. With --rfc0008-quantifiers the condition grammar of the specification's RFC 0008 (Draft) is admitted as a prototype; such a pack is not valid under any published JPS version and every evaluation payload produced this way says so in band.", Args: cobra.ExactArgs(1), RunE: func(_ *cobra.Command, args []string) error { if err := validateFormat(format); err != nil { @@ -156,7 +157,11 @@ func (a *App) evaluateCommand() *cobra.Command { } } evaluator := evaluation.NewEngine(a.engine) - output, failure := evaluator.Evaluate(pack, facts, evidence, supported, "experimental evaluate") + output, failure := evaluator.EvaluateWith(pack, facts, evidence, evaluation.Options{ + Command: "experimental evaluate", + SupportedExtensions: supported, + RFC0008Quantifiers: quantifiers, + }) if failure != nil { return a.operational("experimental evaluate", format, failure.ExitCode, failure.Code, failure.Message) } @@ -170,6 +175,7 @@ func (a *App) evaluateCommand() *cobra.Command { command.Flags().StringVar(&factsPath, "facts", factsPath, "JSON facts document: a file path, or - for standard input (required)") command.Flags().StringVar(&evidencePath, "evidence", evidencePath, "optional tri-state evidence availability: {\"\": \"present\"|\"absent\"|\"unknown\"}") command.Flags().StringArrayVar(&supported, "supported-extension", supported, "extension name this consumer supports (repeatable)") + command.Flags().BoolVar(&quantifiers, "rfc0008-quantifiers", quantifiers, "DRAFT-RFC PROTOTYPE: admit the spec's RFC 0008 (Draft) collection quantifiers -- exists, every, uniform -- in conditions. A pack using them is NOT valid under any published JPS version; spec validate rejects it, and every successful evaluation payload produced this way is labeled a draft-RFC prototype (a refusal is reported as an operational error and carries no such label)") return command } diff --git a/internal/cli/app_test.go b/internal/cli/app_test.go index 97ac9a9..451ef79 100644 --- a/internal/cli/app_test.go +++ b/internal/cli/app_test.go @@ -474,3 +474,114 @@ func TestExperimentalEvaluateCommand(t *testing.T) { } assertDiagnosticCode(t, stdout, "JPS-INVOCATION-FACTS") } + +// The draft RFC 0008 grammar is reachable only through the opt-in flag on +// experimental evaluate. spec validate is untouched and still rejects a pack +// using the operators; without the flag the evaluator refuses it for the same +// reason; with the flag it evaluates, and the output says in band that the pack +// is not valid under the published specification. +func TestExperimentalEvaluateRFC0008QuantifierFlag(t *testing.T) { + pack := filepath.Join("..", "evaluation", "testdata", "rfc0008", "airline-cancellation-quantifier.json") + facts := filepath.Join(t.TempDir(), "facts.json") + document := `{"reservation":{"segments":[{"cancelledByAirline":false},{"cancelledByAirline":true}]}}` + if err := os.WriteFile(facts, []byte(document), 0o644); err != nil { + t.Fatal(err) + } + + // spec validate is untouched: the pack is structurally non-conforming. + code, stdout, stderr := runTest(t, []string{"spec", "validate", pack, "--format", "json"}, "") + if code != result.ExitInvalid || stderr != "" { + t.Fatalf("spec validate must still reject a draft-operator pack: exit=%d stderr=%q", code, stderr) + } + assertDiagnosticCode(t, stdout, "JPS-STRUCTURE-CONDITION-SHAPE") + + // Without the flag the evaluator refuses it exactly as it does today. + code, stdout, _ = runTest(t, []string{"experimental", "evaluate", pack, "--facts", facts, "--format", "json"}, "") + if code != result.ExitInvalid { + t.Fatalf("without the flag the pack must be refused, got exit=%d", code) + } + assertDiagnosticCode(t, stdout, "JPS-EVALUATION-PACK-NOT-CONFORMANT") + + // With the flag it evaluates, and the marker travels with the result. + code, stdout, stderr = runTest(t, []string{"experimental", "evaluate", pack, "--facts", facts, "--rfc0008-quantifiers", "--format", "json"}, "") + if code != result.ExitSuccess || stderr != "" { + t.Fatalf("exit=%d stderr=%q stdout=%q", code, stderr, stdout) + } + var output struct { + SpecVersion string `json:"specVersion"` + DraftPrototype *struct { + RFC string `json:"rfc"` + Status string `json:"status"` + Operators []string `json:"operators"` + PackValidUnderSpecVersion bool `json:"packValidUnderSpecVersion"` + Note string `json:"note"` + } `json:"draftPrototype"` + Disposition struct { + Kind string `json:"kind"` + OutcomeID string `json:"outcomeId"` + } `json:"disposition"` + } + if err := json.Unmarshal([]byte(stdout), &output); err != nil { + t.Fatal(err) + } + if output.Disposition.Kind != "outcome" || output.Disposition.OutcomeID != "free-cancellation" { + t.Fatalf("disposition = %+v", output.Disposition) + } + if output.DraftPrototype == nil { + t.Fatal("a draft-grammar evaluation must carry the prototype marker") + } + if output.DraftPrototype.RFC != "0008" || output.DraftPrototype.Status != "draft-rfc-prototype" { + t.Fatalf("marker = %+v", output.DraftPrototype) + } + if !reflect.DeepEqual(output.DraftPrototype.Operators, []string{"exists"}) { + t.Fatalf("marker must name the operators used: %v", output.DraftPrototype.Operators) + } + if output.DraftPrototype.PackValidUnderSpecVersion || !strings.Contains(output.DraftPrototype.Note, "NOT valid") { + t.Fatalf("the marker must deny validity under %s: %+v", output.SpecVersion, output.DraftPrototype) + } + + // The human surface carries the same warning, above the disposition. + code, stdout, stderr = runTest(t, []string{"experimental", "evaluate", pack, "--facts", facts, "--rfc0008-quantifiers"}, "") + if code != result.ExitSuccess || stderr != "" { + t.Fatalf("exit=%d stderr=%q", code, stderr) + } + marker := strings.Index(stdout, "DRAFT-RFC PROTOTYPE") + disposition := strings.Index(stdout, "disposition:") + if marker < 0 || disposition < 0 || marker > disposition { + t.Fatalf("the prototype marker must precede the disposition: %q", stdout) + } + if !strings.Contains(stdout, "NOT valid under JPS 0.1.0-draft") { + t.Fatalf("the human marker must name the specification version it is not valid under: %q", stdout) + } + + // The flag on a pack that uses no draft operator changes nothing about it, + // and the human marker says so rather than accusing a valid pack. The JSON + // marker of the same run reports packValidUnderSpecVersion: true, so the two + // surfaces must not contradict each other. + corePack := filepath.Join("..", "evaluation", "testdata", "rfc0008", "airline-cancellation-prepared.json") + coreFacts := filepath.Join(t.TempDir(), "facts.json") + if err := os.WriteFile(coreFacts, []byte(`{"reservation":{"anySegmentCancelledByAirline":true}}`), 0o644); err != nil { + t.Fatal(err) + } + code, stdout, stderr = runTest(t, []string{"experimental", "evaluate", corePack, "--facts", coreFacts, "--rfc0008-quantifiers"}, "") + if code != result.ExitSuccess || stderr != "" { + t.Fatalf("exit=%d stderr=%q", code, stderr) + } + if !strings.Contains(stdout, "DRAFT-RFC PROTOTYPE") || strings.Contains(stdout, "NOT valid") { + t.Fatalf("a pack using no draft operator must not be called invalid: %q", stdout) + } + if !strings.Contains(stdout, "uses no draft operator and remains a plain JPS 0.1.0-draft pack") { + t.Fatalf("the human marker must say the pack is unchanged by the flag: %q", stdout) + } + code, stdout, stderr = runTest(t, []string{"experimental", "evaluate", corePack, "--facts", coreFacts, "--rfc0008-quantifiers", "--format", "json"}, "") + if code != result.ExitSuccess || stderr != "" { + t.Fatalf("exit=%d stderr=%q", code, stderr) + } + output.DraftPrototype = nil + if err := json.Unmarshal([]byte(stdout), &output); err != nil { + t.Fatal(err) + } + if output.DraftPrototype == nil || !output.DraftPrototype.PackValidUnderSpecVersion || len(output.DraftPrototype.Operators) != 0 { + t.Fatalf("the JSON marker must agree with the human one: %+v", output.DraftPrototype) + } +} diff --git a/internal/cli/render.go b/internal/cli/render.go index 3977e25..b40270b 100644 --- a/internal/cli/render.go +++ b/internal/cli/render.go @@ -137,6 +137,18 @@ func (a *App) renderEvaluation(format string, output result.Evaluation) error { return a.writeJSON(output) } fmt.Fprintln(a.out, "EXPERIMENTAL evaluation (no conformance claim; JPS 0.1.0-draft defines no evaluator conformance)") + if prototype := output.DraftPrototype; prototype != nil { + // The two wordings mirror the JSON marker's own: a pack that used no + // draft operator is a plain pack the published validator accepts, and + // saying otherwise would contradict the same run's JSON output. + if prototype.PackValidUnderSpecVersion { + fmt.Fprintf(a.out, "DRAFT-RFC PROTOTYPE: RFC %s grammar enabled; this pack uses no draft operator and remains a plain JPS %s pack\n", + display.Sanitize(prototype.RFC), display.Sanitize(output.SpecVersion)) + } else { + fmt.Fprintf(a.out, "DRAFT-RFC PROTOTYPE: RFC %s operators %s; this pack is NOT valid under JPS %s and spec validate rejects it\n", + display.Sanitize(prototype.RFC), display.Sanitize(operatorList(prototype.Operators)), display.Sanitize(output.SpecVersion)) + } + } switch output.Disposition.Kind { case "outcome": fmt.Fprintf(a.out, "disposition: outcome %s\n", display.Sanitize(output.Disposition.OutcomeID)) @@ -176,6 +188,16 @@ func (a *App) renderEvaluation(format string, output result.Evaluation) error { return nil } +// operatorList names the draft operators a pack actually used. The empty case +// never reaches the marker line — a pack that used none gets the other wording +// above — but it is spelled out rather than left to render as an empty list. +func operatorList(operators []string) string { + if len(operators) == 0 { + return "(none used)" + } + return strings.Join(operators, ", ") +} + func joinNonEmpty(values ...string) string { output := []string{} for _, value := range values { diff --git a/internal/evaluation/condition.go b/internal/evaluation/condition.go index 9fc8a43..86b96bc 100644 --- a/internal/evaluation/condition.go +++ b/internal/evaluation/condition.go @@ -35,11 +35,33 @@ func triFromBool(value bool) tri { return triFalse } -// evalCondition applies §7's experimental interpretation, as pinned by spec -// RFC 0006, to one condition node. The pack has full document conformance, so -// node shapes are schema-guaranteed; anything unexpected still degrades to -// unknown ("unsupported operators or operand shapes ... produce unknown"). -func evalCondition(node any, facts any, evidence map[string]tri) tri { +// evaluator interprets condition trees. It carries the evidence tri-state and, +// when the caller opted into the draft RFC 0008 grammar, the work budget that +// grammar makes load-bearing. One evaluator serves one evaluation, so the +// budget is charged across the whole pack rather than per condition. +type evaluator struct { + evidence map[string]tri + // quantifiers admits the draft RFC 0008 aggregates and turns on the work + // accounting they require. Without it nothing about §7 changes. + quantifiers bool + budget int + charged int + exceeded bool + // pointers memoizes the compiled form of every authored pointer this + // evaluation has resolved, keyed by the pointer text as the pack wrote it. + // Compiling scans the path's bytes; the cache is what makes a per-element + // resolution cost its steps rather than a fresh scan, and the accounting + // model on evaluate charges each scan exactly once to match. + pointers map[string]compiledPointer +} + +// condition evaluates one node against the current condition root: the runtime +// facts document at the top level and, under draft RFC 0008, the selected +// element inside a quantifier's where. The pack has full document conformance, +// so node shapes are schema-guaranteed; anything unexpected still degrades to +// unknown ("unsupported operators or operand shapes ... produce unknown"), +// which is also what an aggregate op does when the caller did not opt in. +func (e *evaluator) condition(node any, root any) tri { condition, ok := node.(map[string]any) if !ok { return triUnknown @@ -51,11 +73,11 @@ func evalCondition(node any, facts any, evidence map[string]tri) tri { } return triUnknown case "all": - return evalAll(condition["conditions"], facts, evidence) + return e.all(condition["conditions"], root) case "any": - return evalAny(condition["conditions"], facts, evidence) + return e.any(condition["conditions"], root) case "not": - switch evalCondition(condition["condition"], facts, evidence) { + switch e.condition(condition["condition"], root) { case triTrue: return triFalse case triFalse: @@ -64,32 +86,42 @@ func evalCondition(node any, facts any, evidence map[string]tri) tri { return triUnknown } case "fact": - return evalFact(condition, facts) + return e.evalFact(condition, root) case "evidence-present": name, ok := condition["evidenceRequirement"].(string) if !ok { return triUnknown } - presence, declared := evidence[name] + presence, declared := e.evidence[name] if !declared { return triUnknown } return presence + case "exists", "every": + if !e.quantifiers { + return triUnknown + } + return e.quantify(condition, root) + case "uniform": + if !e.quantifiers { + return triUnknown + } + return e.uniform(condition, root) default: return triUnknown } } -// evalAll is strong three-valued conjunction (§7.1): false if any child is +// all is strong three-valued conjunction (§7.1): false if any child is // false, true if every child is true, unknown otherwise. -func evalAll(children any, facts any, evidence map[string]tri) tri { +func (e *evaluator) all(children any, root any) tri { items, ok := children.([]any) if !ok { return triUnknown } sawUnknown := false for _, item := range items { - switch evalCondition(item, facts, evidence) { + switch e.condition(item, root) { case triFalse: return triFalse case triUnknown: @@ -102,16 +134,16 @@ func evalAll(children any, facts any, evidence map[string]tri) tri { return triTrue } -// evalAny is strong three-valued disjunction (§7.2): true if any child is +// any is strong three-valued disjunction (§7.2): true if any child is // true, false if every child is false, unknown otherwise. -func evalAny(children any, facts any, evidence map[string]tri) tri { +func (e *evaluator) any(children any, root any) tri { items, ok := children.([]any) if !ok { return triUnknown } sawUnknown := false for _, item := range items { - switch evalCondition(item, facts, evidence) { + switch e.condition(item, root) { case triTrue: return triTrue case triUnknown: @@ -124,53 +156,38 @@ func evalAny(children any, facts any, evidence map[string]tri) tri { return triFalse } -// evalFact selects a value from the facts document by RFC 6901 pointer and -// applies the declared operator (§7.4). A pointer that does not resolve, an -// unsupported operand shape, or an incomparable ordered pair produces unknown. -func evalFact(condition map[string]any, facts any) tri { +// evalFact selects a value from the current condition root by RFC 6901 pointer +// and applies the declared operator (§7.4). The root is the facts document +// everywhere except inside a draft RFC 0008 where, which is the whole of that +// RFC's amendment to §7.4. A pointer that does not resolve, an unsupported +// operand shape, or an incomparable ordered pair produces unknown. +func (e *evaluator) evalFact(condition map[string]any, root any) tri { path, ok := condition["path"].(string) if !ok { return triUnknown } - value, resolved := resolvePointer(facts, path) + value, resolved := e.resolve(root, path) if !resolved { return triUnknown } operand := condition["value"] switch condition["operator"] { case "equals": - equal, determinable := jsonEqual(value, operand) - if !determinable { - return triUnknown - } - return triFromBool(equal) + return triFromBool(jsonEqual(value, operand)) case "not-equals": - // §7.4: not-equals is the Boolean inverse of equals only when equality - // can be determined. - equal, determinable := jsonEqual(value, operand) - if !determinable { - return triUnknown - } - return triFromBool(!equal) + // §7.4: equality is total over carrier-valid JSON, so not-equals is + // simply its Boolean inverse. + return triFromBool(!jsonEqual(value, operand)) case "in": items, ok := operand.([]any) if !ok { return triUnknown } - sawUndeterminable := false for _, item := range items { - equal, determinable := jsonEqual(value, item) - if !determinable { - sawUndeterminable = true - continue - } - if equal { + if jsonEqual(value, item) { return triTrue } } - if sawUndeterminable { - return triUnknown - } return triFalse case "greater-than", "greater-than-or-equal", "less-than", "less-than-or-equal": comparison, comparable := decimalCompare(value, operand) @@ -192,20 +209,51 @@ func evalFact(condition map[string]any, facts any) tri { } } -// resolvePointer resolves an RFC 6901 JSON Pointer against a decoded JSON -// document. The empty string selects the root. It reports false for any -// pointer that does not resolve, including an invalid array traversal and the -// "-" past-the-end token. -func resolvePointer(document any, pointer string) (any, bool) { +// compiledPointer is an authored RFC 6901 pointer with its bytes already read: +// the escape-decoded reference tokens, in order, and the number of bytes those +// tokens carry. Compiling is the part of a resolution whose cost is the path's +// length; walking the compiled tokens against a document is the part whose cost +// is the tokens'. Separating them is what lets one scan be charged once and +// each of many per-element resolutions be charged for what it actually does. +type compiledPointer struct { + tokens []string + // bytes is the total length of the decoded tokens: what one resolution + // hashes or compares against the document it walks. + bytes int + // rooted reports a syntactically usable pointer: the empty string, or one + // beginning with "/". Anything else resolves against nothing. + rooted bool +} + +// compilePointer scans one authored pointer. This is the byte-length-sensitive +// step: it reads every byte of the path, splits it, and decodes the ~1 and ~0 +// escapes. The accounting model charges it before it runs. +func compilePointer(pointer string) compiledPointer { if pointer == "" { - return document, true + return compiledPointer{rooted: true} } if !strings.HasPrefix(pointer, "/") { + return compiledPointer{} + } + raw := strings.Split(pointer[1:], "/") + compiled := compiledPointer{tokens: make([]string, 0, len(raw)), rooted: true} + for _, token := range raw { + token = strings.ReplaceAll(strings.ReplaceAll(token, "~1", "/"), "~0", "~") + compiled.tokens = append(compiled.tokens, token) + compiled.bytes += len(token) + } + return compiled +} + +// resolve walks a compiled pointer against a decoded JSON document. The empty +// pointer selects the root. It reports false for any pointer that does not +// resolve, including an invalid array traversal and the "-" past-the-end token. +func (p compiledPointer) resolve(document any) (any, bool) { + if !p.rooted { return nil, false } current := document - for _, token := range strings.Split(pointer[1:], "/") { - token = strings.ReplaceAll(strings.ReplaceAll(token, "~1", "/"), "~0", "~") + for _, token := range p.tokens { switch container := current.(type) { case map[string]any: value, ok := container[token] @@ -226,6 +274,35 @@ func resolvePointer(document any, pointer string) (any, bool) { return current, true } +// resolve resolves one authored pointer against a document through the +// evaluator's compiled-pointer cache, so an authored path is scanned once per +// evaluation however many elements it is later resolved against. It charges +// nothing: under the draft grammar every resolution it performs was already +// reserved by the preflight, and without the draft grammar there is no budget. +func (e *evaluator) resolve(document any, pointer string) (any, bool) { + return e.compiled(pointer).resolve(document) +} + +// compiled returns the cached compilation of an authored pointer, scanning it +// if this evaluation has not seen it before. +func (e *evaluator) compiled(pointer string) compiledPointer { + if compiled, scanned := e.pointers[pointer]; scanned { + return compiled + } + compiled := compilePointer(pointer) + if e.pointers == nil { + e.pointers = map[string]compiledPointer{} + } + e.pointers[pointer] = compiled + return compiled +} + +// resolvePointer resolves an RFC 6901 JSON Pointer against a decoded JSON +// document without a cache, for callers that resolve one pointer once. +func resolvePointer(document any, pointer string) (any, bool) { + return compilePointer(pointer).resolve(document) +} + // arrayIndex parses an RFC 6901 array-reference token: decimal digits with no // leading zero (other than "0" itself), in range. "-" refers past the end and // never resolves. @@ -250,82 +327,221 @@ func arrayIndex(token string, length int) (int, bool) { // jsonEqual is §7.4's type-preserving JSON equality: no coercion between // types; numbers compare by mathematical value; arrays in order; objects by -// member name and value regardless of order. The second result reports -// whether equality could be determined at all: a syntactically valid JSON -// number the arithmetic cannot represent (for example an exponent beyond -// big.Rat's range) makes the comparison undeterminable rather than silently -// false — §7.4: "incomparable values ... produce unknown". -func jsonEqual(a, b any) (bool, bool) { +// member name and value regardless of order. +// +// It is total over carrier-valid JSON. Every pair of decoded values is equal +// or unequal, and there is no third answer: numbers are decided on their +// tokens (numbersEqual), which needs no arithmetic type big enough to hold +// them, so an evaluator's inability to represent a value never becomes a +// semantics that another evaluator would have to reproduce. §7.4's +// "incomparable values ... produce unknown" governs the ordered comparisons +// (decimalCompare), which are defined only over §2.2 decimal strings and are +// untouched by this. +func jsonEqual(a, b any) bool { switch left := a.(type) { case nil: - return b == nil, true + return b == nil case bool: right, ok := b.(bool) - return ok && left == right, true + return ok && left == right case string: right, ok := b.(string) - return ok && left == right, true + return ok && left == right case json.Number: right, ok := b.(json.Number) - if !ok { - return false, true - } - if left.String() == right.String() { - // Identical tokens are equal regardless of representability. - return true, true - } - leftRat, leftOK := new(big.Rat).SetString(left.String()) - rightRat, rightOK := new(big.Rat).SetString(right.String()) - if !leftOK || !rightOK { - return false, false - } - return leftRat.Cmp(rightRat) == 0, true + return ok && numbersEqual(left, right) case []any: right, ok := b.([]any) if !ok || len(left) != len(right) { - return false, true + return false } - sawUndeterminable := false - for i := range left { - equal, determinable := jsonEqual(left[i], right[i]) - if !determinable { - sawUndeterminable = true - continue - } - if !equal { - return false, true + for index := range left { + if !jsonEqual(left[index], right[index]) { + return false } } - if sawUndeterminable { - return false, false - } - return true, true + return true case map[string]any: right, ok := b.(map[string]any) if !ok || len(left) != len(right) { - return false, true + return false } - sawUndeterminable := false for key, value := range left { other, present := right[key] - if !present { - return false, true - } - equal, determinable := jsonEqual(value, other) - if !determinable { - sawUndeterminable = true - continue - } - if !equal { - return false, true + if !present || !jsonEqual(value, other) { + return false } } - if sawUndeterminable { - return false, false + return true + default: + return false + } +} + +// numbersEqual decides §7.4 equality between two JSON number tokens by +// mathematical value, in one pass over each token and without materializing +// either value. Two tokens denote the same number exactly when their normal +// forms agree — same sign, same significant digits, same adjusted exponent — +// so 1e3, 1000, and 1.0e3 are one value and -0 is 0. +// +// Identical tokens are settled on the strings, before either is normalized. +// That short-circuit is what makes a canonicalized value (canonicalNumbers) +// cheap to compare repeatedly: equal numbers reach it as byte-identical tokens, +// so a caller that compares one value against many pays one normalization per +// value rather than one per comparison. +// +// Deciding it this way is what makes equality total. Materializing the values +// (big.Rat) answers the same question only for the values the machine can +// hold: 1e999999999 against 2e999999999 would need a gigabyte of digits to +// settle by arithmetic and settles in twenty bytes on the tokens. The earlier +// arithmetic-based path reported such a pair as undeterminable and let uniform +// and fact produce unknown from it, which made a disposition depend on the +// evaluator's number type rather than on the documents. Inability is a +// resource condition, not a semantics. +func numbersEqual(left, right json.Number) bool { + if left.String() == right.String() { + return true + } + return normalizeNumber(left.String()).equals(normalizeNumber(right.String())) +} + +// numberNormalForm is one JSON number token reduced to the unique triple that +// identifies its value: the sign, the significant digits with every leading +// and trailing zero stripped, and the power of ten those digits are scaled by. +// Zero has no significant digits, which is what makes -0, 0, and 0.0e10 one +// value whatever their signs and exponents. The exponent is arbitrary-precision +// because the token's is — but it is the exponent that is materialized, never +// the value, so its cost is the token's length and not the number's magnitude. +// +// malformed marks a token outside the JSON number grammar. The carrier cannot +// produce one; a hand-built json.Number can, and it compares equal to nothing +// but an identical token. +type numberNormalForm struct { + negative bool + digits string + exponent *big.Int + malformed bool +} + +func (n numberNormalForm) equals(other numberNormalForm) bool { + if n.malformed || other.malformed { + return false + } + if n.digits == "" || other.digits == "" { + // Zero, which has no sign and no exponent to disagree about. + return n.digits == other.digits + } + return n.negative == other.negative && n.digits == other.digits && n.exponent.Cmp(other.exponent) == 0 +} + +// normalizeNumber computes that normal form in one pass over the token: split +// off the sign, the exponent, and the fraction; concatenate the integer and +// fraction digits into the significand; charge the exponent one power of ten +// for every fraction digit it absorbed; then strip the leading zeros, which +// carry no value, and the trailing zeros, each of which is one more power of +// ten. +func normalizeNumber(token string) numberNormalForm { + form := numberNormalForm{exponent: new(big.Int)} + rest := token + if strings.HasPrefix(rest, "-") { + form.negative, rest = true, rest[1:] + } + mantissa, exponent := rest, "" + if index := strings.IndexAny(rest, "eE"); index >= 0 { + mantissa, exponent = rest[:index], rest[index+1:] + } + integer, fraction := mantissa, "" + if index := strings.IndexByte(mantissa, '.'); index >= 0 { + integer, fraction = mantissa[:index], mantissa[index+1:] + } + if exponent != "" { + if _, ok := form.exponent.SetString(exponent, 10); !ok { + return numberNormalForm{malformed: true} + } + } + digits := integer + fraction + for index := 0; index < len(digits); index++ { + if digits[index] < '0' || digits[index] > '9' { + return numberNormalForm{malformed: true} + } + } + form.exponent.Sub(form.exponent, big.NewInt(int64(len(fraction)))) + leading := 0 + for leading < len(digits) && digits[leading] == '0' { + leading++ + } + digits = digits[leading:] + significant := len(digits) + for significant > 0 && digits[significant-1] == '0' { + significant-- + } + form.exponent.Add(form.exponent, big.NewInt(int64(len(digits)-significant))) + form.digits = digits[:significant] + return form +} + +// token writes a normal form back out as the one JSON number token that denotes +// its value: "0" for zero, the significant digits alone when they need no +// scaling, and the digits followed by "e" and the power of ten otherwise. It is +// injective on normal forms — two tokens denote the same number exactly when +// their canonical tokens are byte-identical — and normalizing a canonical token +// reproduces the same form, so canonicalizing twice changes nothing. Its length +// is within a small constant of the token it came from: the digits are a subset +// of the original's and the exponent is at most the original's exponent digits +// plus the fraction's length. +func (n numberNormalForm) token() string { + if n.digits == "" { + return "0" // Zero, which has neither a sign nor an exponent. + } + text := n.digits + if n.negative { + text = "-" + text + } + if n.exponent.Sign() == 0 { + return text + } + return text + "e" + n.exponent.String() +} + +// canonicalNumbers deep-copies a value with every number token replaced by the +// canonical token of its normal form, leaving every other leaf as it is. It +// costs one normalization per number, once, and it is what stops a value that +// is compared many times from being normalized once per comparison: afterwards +// two equal numbers are byte-identical tokens, which numbersEqual settles on +// the strings before it normalizes anything. +// +// §7.4 equality is unchanged by it, in both directions: canonicalization maps +// equal values to identical tokens and unequal values to different ones, so a +// comparison between canonical forms answers exactly what a comparison between +// the original tokens answers. uniform is the caller — it compares one elected +// representative against every other member, which is the shape that turns a +// per-comparison normalization into a reread of the longest token per member. +// +// A token outside the JSON number grammar has no normal form and is returned +// untouched, so it goes on comparing equal to an identical token and to nothing +// else. +func canonicalNumbers(value any) any { + switch typed := value.(type) { + case []any: + items := make([]any, len(typed)) + for index, item := range typed { + items[index] = canonicalNumbers(item) + } + return items + case map[string]any: + members := make(map[string]any, len(typed)) + for name, item := range typed { + members[name] = canonicalNumbers(item) + } + return members + case json.Number: + form := normalizeNumber(typed.String()) + if form.malformed { + return typed } - return true, true + return json.Number(form.token()) default: - return false, true + return value } } diff --git a/internal/evaluation/condition_test.go b/internal/evaluation/condition_test.go index 97a07cf..9214abe 100644 --- a/internal/evaluation/condition_test.go +++ b/internal/evaluation/condition_test.go @@ -6,6 +6,13 @@ import ( "testing" ) +// evalCondition applies §7's experimental interpretation, as pinned by spec +// RFC 0006, to one condition node, with no draft grammar and no work budget: +// the surface every row in this file is written against. +func evalCondition(node any, facts any, evidence map[string]tri) tri { + return (&evaluator{evidence: evidence}).condition(node, facts) +} + func TestDecimalCompare(t *testing.T) { cases := []struct { fact, operand any @@ -68,34 +75,58 @@ func TestResolvePointer(t *testing.T) { } } +// §7.4 equality, including its totality: every pair below is equal or unequal, +// and no pair is anything else. The rows carrying numbers no machine word or +// big.Rat can hold are the point — they are decided on the tokens, so the +// answer is a property of the two numbers rather than of the arithmetic that +// happened to be available. func TestJSONEqualIsTypePreserving(t *testing.T) { cases := []struct { - a, b any - want bool - determinable bool + a, b any + want bool }{ - {json.Number("5.0"), json.Number("5"), true, true}, // mathematical value - {json.Number("5"), "5", false, true}, // no coercion across types - {"5", json.Number("5"), false, true}, - {true, "true", false, true}, - {nil, nil, true, true}, - {[]any{json.Number("1"), "x"}, []any{json.Number("1.0"), "x"}, true, true}, - {[]any{"x", "y"}, []any{"y", "x"}, false, true}, // arrays compare in order - {map[string]any{"a": json.Number("1"), "b": "z"}, map[string]any{"b": "z", "a": json.Number("1.00")}, true, true}, - {map[string]any{"a": "1"}, map[string]any{"a": "1", "b": "2"}, false, true}, - // A representable-vs-unrepresentable pair is undeterminable, never a - // silent false (spec §7.4: incomparable values produce unknown)... - {json.Number("1e999999999"), json.Number("2e999999999"), false, false}, - {[]any{json.Number("1e999999999")}, []any{json.Number("2e999999999")}, false, false}, - // ...but identical tokens are equal regardless of representability. - {json.Number("1e999999999"), json.Number("1e999999999"), true, true}, + {json.Number("5.0"), json.Number("5"), true}, // mathematical value + {json.Number("5"), "5", false}, // no coercion across types + {"5", json.Number("5"), false}, + {true, "true", false}, + {nil, nil, true}, + {[]any{json.Number("1"), "x"}, []any{json.Number("1.0"), "x"}, true}, + {[]any{"x", "y"}, []any{"y", "x"}, false}, // arrays compare in order + {map[string]any{"a": json.Number("1"), "b": "z"}, map[string]any{"b": "z", "a": json.Number("1.00")}, true}, + {map[string]any{"a": "1"}, map[string]any{"a": "1", "b": "2"}, false}, + // Normal forms: an exponent, a shifted point, and a signed zero are + // notations, not values. + {json.Number("1e3"), json.Number("1000"), true}, + {json.Number("1000"), json.Number("1.0e3"), true}, + {json.Number("-0"), json.Number("0"), true}, + {json.Number("0.0e10"), json.Number("-0.000"), true}, + {json.Number("1.23e4"), json.Number("12300"), true}, + {json.Number("1E+3"), json.Number("1000"), true}, + {json.Number("1e-2"), json.Number("0.01"), true}, + {json.Number("0e999999999"), json.Number("-0"), true}, + {json.Number("-1.5"), json.Number("1.5"), false}, + {json.Number("1e3"), json.Number("1e4"), false}, + {json.Number("1e-2"), json.Number("0.1"), false}, + // Numbers past every arithmetic type still compare exactly, in both + // directions: unequal ones are unequal... + {json.Number("1e999999999"), json.Number("2e999999999"), false}, + {json.Number("1e999999999"), json.Number("1e999999998"), false}, + {[]any{json.Number("1e999999999")}, []any{json.Number("2e999999999")}, false}, + // ...and equal ones are equal, whether the tokens match or not. + {json.Number("1e999999999"), json.Number("1e999999999"), true}, + {json.Number("1e999999999"), json.Number("1.0e999999999"), true}, + {json.Number("10e999999998"), json.Number("1e999999999"), true}, + {json.Number("-1e999999999"), json.Number("1e999999999"), false}, // A definite mismatch elsewhere still decides the comparison. - {[]any{json.Number("1e999999999"), "x"}, []any{json.Number("2e999999999"), "y"}, false, true}, + {[]any{json.Number("1e999999999"), "x"}, []any{json.Number("2e999999999"), "y"}, false}, } for _, testCase := range cases { - got, determinable := jsonEqual(testCase.a, testCase.b) - if determinable != testCase.determinable || (determinable && got != testCase.want) { - t.Errorf("jsonEqual(%v, %v) = (%v, %v), want (%v, %v)", testCase.a, testCase.b, got, determinable, testCase.want, testCase.determinable) + if got := jsonEqual(testCase.a, testCase.b); got != testCase.want { + t.Errorf("jsonEqual(%v, %v) = %v, want %v", testCase.a, testCase.b, got, testCase.want) + } + // Equality is symmetric, which a normal-form comparison must not lose. + if got := jsonEqual(testCase.b, testCase.a); got != testCase.want { + t.Errorf("jsonEqual(%v, %v) = %v, want %v", testCase.b, testCase.a, got, testCase.want) } } } @@ -131,6 +162,28 @@ func TestThreeValuedOperators(t *testing.T) { } } +// The draft RFC 0008 aggregates are inert without the opt-in. No published +// surface reaches this state — the validator rejects a pack carrying an +// aggregate before evaluation begins — so the rows are written against the +// evaluator directly, which is the only caller that could ever bypass the gate. +func TestAggregatesAreUnknownWithoutTheOptIn(t *testing.T) { + facts := map[string]any{"list": []any{map[string]any{"ok": true}}} + predicate := map[string]any{"op": "fact", "path": "/ok", "operator": "equals", "value": true} + cases := []struct { + name string + node map[string]any + }{ + {"exists", map[string]any{"op": "exists", "path": "/list", "where": predicate}}, + {"every", map[string]any{"op": "every", "path": "/list", "where": predicate}}, + {"uniform", map[string]any{"op": "uniform", "path": "/list", "at": "/ok"}}, + } + for _, testCase := range cases { + if got := evalCondition(testCase.node, facts, map[string]tri{}); got != triUnknown { + t.Errorf("%s without the opt-in = %v, want unknown", testCase.name, got) + } + } +} + // Resolver edges exercised directly, without the validation gate, so states no // conformant single pack can combine are still pinned. func TestResolverEdges(t *testing.T) { @@ -146,7 +199,7 @@ func TestResolverEdges(t *testing.T) { pack := base() pack["rules"] = []any{map[string]any{"id": "r", "when": map[string]any{"op": "literal", "value": true}, "outcome": "a", "onUnknown": "ignore"}} pack["exceptions"] = []any{map[string]any{"id": "x", "when": map[string]any{"op": "literal", "value": true}, "effect": "escalate", "onUnknown": "ignore"}} - disposition, _ := resolve(pack, map[string]any{}, map[string]tri{}) + disposition, _, _ := resolve(pack, map[string]any{}, &evaluator{evidence: map[string]tri{}}) if disposition.Kind != "unresolved" || !reflect.DeepEqual(disposition.Reasons, []string{"exception-escalation"}) { t.Fatalf("direct escalation must be unresolved: %+v", disposition) } @@ -161,7 +214,7 @@ func TestResolverEdges(t *testing.T) { map[string]any{"id": "x1", "when": map[string]any{"op": "literal", "value": true}, "effect": "force-outcome", "outcome": "a", "onUnknown": "ignore"}, map[string]any{"id": "x2", "when": map[string]any{"op": "literal", "value": true}, "effect": "force-outcome", "outcome": "b", "onUnknown": "ignore"}, } - disposition, _ := resolve(pack, map[string]any{}, map[string]tri{}) + disposition, _, _ := resolve(pack, map[string]any{}, &evaluator{evidence: map[string]tri{}}) if disposition.Kind != "unresolved" || !reflect.DeepEqual(disposition.Reasons, []string{"conflict"}) { t.Fatalf("incompatible forced outcomes must conflict: %+v", disposition) } @@ -172,7 +225,7 @@ func TestResolverEdges(t *testing.T) { pack["rules"] = []any{map[string]any{"id": "r", "when": map[string]any{"op": "literal", "value": true}, "outcome": "a", "onUnknown": "ignore"}} pack["exceptions"] = []any{map[string]any{"id": "x", "when": map[string]any{"op": "literal", "value": true}, "effect": "suppress-rule", "targetRule": "r", "onUnknown": "ignore"}} pack["fallbackOutcome"] = "b" - disposition, trace := resolve(pack, map[string]any{}, map[string]tri{}) + disposition, trace, _ := resolve(pack, map[string]any{}, &evaluator{evidence: map[string]tri{}}) if disposition.Kind != "outcome" || disposition.OutcomeID != "b" { t.Fatalf("with the only rule suppressed the fallback applies: %+v", disposition) } @@ -190,7 +243,7 @@ func TestResolverEdges(t *testing.T) { t.Run("no-match-without-fallback", func(t *testing.T) { pack := base() pack["rules"] = []any{map[string]any{"id": "r", "when": map[string]any{"op": "literal", "value": false}, "outcome": "a", "onUnknown": "ignore"}} - disposition, _ := resolve(pack, map[string]any{}, map[string]tri{}) + disposition, _, _ := resolve(pack, map[string]any{}, &evaluator{evidence: map[string]tri{}}) if disposition.Kind != "unresolved" || !reflect.DeepEqual(disposition.Reasons, []string{"no-match"}) { t.Fatalf("no rule and no fallback is no-match: %+v", disposition) } @@ -200,7 +253,7 @@ func TestResolverEdges(t *testing.T) { pack := base() pack["rules"] = []any{map[string]any{"id": "r", "when": map[string]any{"op": "fact", "path": "/missing", "operator": "equals", "value": "x"}, "outcome": "a", "onUnknown": "ignore"}} pack["fallbackOutcome"] = "b" - disposition, trace := resolve(pack, map[string]any{}, map[string]tri{}) + disposition, trace, _ := resolve(pack, map[string]any{}, &evaluator{evidence: map[string]tri{}}) if disposition.Kind != "outcome" || disposition.OutcomeID != "b" { t.Fatalf("an ignored unknown must not block the fallback: %+v", disposition) } @@ -219,7 +272,7 @@ func TestResolverEdges(t *testing.T) { pack := base() pack["rules"] = []any{map[string]any{"id": "r", "when": map[string]any{"op": "fact", "path": "/missing", "operator": "equals", "value": "x"}, "outcome": "a", "onUnknown": "escalate"}} pack["fallbackOutcome"] = "b" - disposition, _ := resolve(pack, map[string]any{}, map[string]tri{}) + disposition, _, _ := resolve(pack, map[string]any{}, &evaluator{evidence: map[string]tri{}}) if disposition.Kind != "unresolved" || !reflect.DeepEqual(disposition.Reasons, []string{"unknown"}) { t.Fatalf("an escalating unknown blocks both candidate and fallback: %+v", disposition) } @@ -232,7 +285,7 @@ func TestResolverEdges(t *testing.T) { map[string]any{"id": "r2", "when": map[string]any{"op": "literal", "value": true}, "outcome": "b", "onUnknown": "ignore"}, map[string]any{"id": "r3", "when": map[string]any{"op": "fact", "path": "/missing", "operator": "equals", "value": "x"}, "outcome": "a", "onUnknown": "escalate"}, } - disposition, _ := resolve(pack, map[string]any{}, map[string]tri{}) + disposition, _, _ := resolve(pack, map[string]any{}, &evaluator{evidence: map[string]tri{}}) if !reflect.DeepEqual(disposition.Reasons, []string{"conflict", "unknown"}) { t.Fatalf("neither unknown nor conflict is discarded: %+v", disposition) } @@ -255,10 +308,12 @@ func TestEvidencePresentTriState(t *testing.T) { } } -// Ordered comparison at the condition level: decimal strings decide, JSON -// numbers and non-grammar strings degrade to unknown, and undeterminable -// number equality degrades to unknown rather than false. -func TestOrderedAndUndeterminableConditions(t *testing.T) { +// The two comparison families at the condition level, and the line between +// them. Ordered comparison is defined only over §2.2 decimal strings, so a JSON +// number or a non-grammar string degrades to unknown; equality is total, so +// every equals, not-equals, and in below decides — including over numbers no +// arithmetic type can hold, which produce false or true and never unknown. +func TestOrderedAndEqualityConditions(t *testing.T) { facts := map[string]any{ "amount": "150", "count": json.Number("150"), @@ -272,10 +327,12 @@ func TestOrderedAndUndeterminableConditions(t *testing.T) { {"decimal-string-greater", map[string]any{"op": "fact", "path": "/amount", "operator": "greater-than", "value": "100"}, triTrue}, {"decimal-string-not-less", map[string]any{"op": "fact", "path": "/amount", "operator": "less-than", "value": "100"}, triFalse}, {"json-number-ordered-unknown", map[string]any{"op": "fact", "path": "/count", "operator": "greater-than", "value": "100"}, triUnknown}, - {"huge-number-equals-unknown", map[string]any{"op": "fact", "path": "/huge", "operator": "equals", "value": json.Number("2e999999999")}, triUnknown}, - {"huge-number-not-equals-unknown", map[string]any{"op": "fact", "path": "/huge", "operator": "not-equals", "value": json.Number("2e999999999")}, triUnknown}, + {"huge-number-equals-false", map[string]any{"op": "fact", "path": "/huge", "operator": "equals", "value": json.Number("2e999999999")}, triFalse}, + {"huge-number-not-equals-true", map[string]any{"op": "fact", "path": "/huge", "operator": "not-equals", "value": json.Number("2e999999999")}, triTrue}, {"huge-identical-token-equal", map[string]any{"op": "fact", "path": "/huge", "operator": "equals", "value": json.Number("1e999999999")}, triTrue}, - {"in-with-undeterminable-member", map[string]any{"op": "fact", "path": "/huge", "operator": "in", "value": []any{json.Number("2e999999999"), "x"}}, triUnknown}, + {"huge-equal-under-a-different-token", map[string]any{"op": "fact", "path": "/huge", "operator": "equals", "value": json.Number("1.0e999999999")}, triTrue}, + {"in-over-huge-tokens-decides-false", map[string]any{"op": "fact", "path": "/huge", "operator": "in", "value": []any{json.Number("2e999999999"), "x"}}, triFalse}, + {"in-over-huge-tokens-decides-true", map[string]any{"op": "fact", "path": "/huge", "operator": "in", "value": []any{json.Number("2e999999999"), json.Number("10e999999998")}}, triTrue}, } for _, testCase := range cases { if got := evalCondition(testCase.node, facts, map[string]tri{}); got != testCase.want { @@ -303,7 +360,7 @@ func TestHandoffTriggerMatching(t *testing.T) { "triggers": []any{"conflict"}, "target": map[string]any{"kind": "human-role", "name": "Reviewer"}, } - disposition, _ := resolve(pack, map[string]any{}, map[string]tri{}) + disposition, _, _ := resolve(pack, map[string]any{}, &evaluator{evidence: map[string]tri{}}) if !reflect.DeepEqual(disposition.Reasons, []string{"no-match"}) { t.Fatalf("expected no-match: %+v", disposition) } @@ -318,7 +375,7 @@ func TestHandoffTriggerMatching(t *testing.T) { "triggers": []any{"no-match"}, "target": map[string]any{"kind": "team", "name": "Intake board"}, } - disposition, _ := resolve(pack, map[string]any{}, map[string]tri{}) + disposition, _, _ := resolve(pack, map[string]any{}, &evaluator{evidence: map[string]tri{}}) if disposition.Handoff.State != "requested" || disposition.Handoff.Target == nil || disposition.Handoff.Target.Kind != "team" || disposition.Handoff.Target.Name != "Intake board" { t.Fatalf("a matching trigger must echo the declared target exactly: %+v", disposition.Handoff) @@ -334,7 +391,7 @@ func TestHandoffTriggerMatching(t *testing.T) { "triggers": []any{"conflict"}, // exception-escalation is deliberately absent "target": map[string]any{"kind": "human-role", "name": "Escalation desk"}, } - disposition, _ := resolve(pack, map[string]any{}, map[string]tri{}) + disposition, _, _ := resolve(pack, map[string]any{}, &evaluator{evidence: map[string]tri{}}) if disposition.Handoff.State != "requested" || disposition.Handoff.Target == nil || disposition.Handoff.Target.Name != "Escalation desk" { t.Fatalf("a direct request uses the configured target regardless of the trigger list: %+v", disposition.Handoff) } @@ -346,7 +403,7 @@ func TestHandoffTriggerMatching(t *testing.T) { map[string]any{"id": "force", "when": map[string]any{"op": "literal", "value": true}, "effect": "force-outcome", "outcome": "b", "onUnknown": "ignore"}, map[string]any{"id": "esc", "when": map[string]any{"op": "literal", "value": true}, "effect": "escalate", "onUnknown": "ignore"}, } - disposition, _ := resolve(pack, map[string]any{}, map[string]tri{}) + disposition, _, _ := resolve(pack, map[string]any{}, &evaluator{evidence: map[string]tri{}}) if disposition.Kind != "unresolved" || !reflect.DeepEqual(disposition.Reasons, []string{"exception-escalation"}) { t.Fatalf("a direct escalation takes precedence over a compatible forced outcome: %+v", disposition) } diff --git a/internal/evaluation/engine.go b/internal/evaluation/engine.go index 6e44e7c..910bceb 100644 --- a/internal/evaluation/engine.go +++ b/internal/evaluation/engine.go @@ -30,6 +30,24 @@ type Failure struct { ExitCode int } +// Options carries one evaluation's caller opt-ins. The zero value is the +// published surface: Core conditions only, no draft grammar, no work budget. +type Options struct { + // Command names the reporting surface, exactly as in the describe package. + Command string + // SupportedExtensions is the consumer's extension capability set. + SupportedExtensions []string + // RFC0008Quantifiers admits the draft RFC 0008 aggregates (exists, every, + // uniform). A pack using one is not valid under any published JPS version, + // so every evaluation made under this opt-in is labeled a draft-RFC + // prototype in its output. + RFC0008Quantifiers bool + // WorkBudget overrides DefaultWorkBudget. Zero or negative selects + // DefaultWorkBudget. It applies only under the draft grammar, which is what + // makes an evaluation-work limit load-bearing. + WorkBudget int +} + // Engine evaluates conformant packs. It wraps the validation engine so a pack // is always checked to full document conformance before a single condition is // interpreted. @@ -45,6 +63,11 @@ func NewEngine(validator *validation.Engine) *Engine { // applies the §§7–8 experiment to the facts and evidence inputs. command names // the reporting surface, exactly as in the describe package. func (e *Engine) Evaluate(pack, facts, evidence []byte, supported []string, command string) (result.Evaluation, *Failure) { + return e.EvaluateWith(pack, facts, evidence, Options{Command: command, SupportedExtensions: supported}) +} + +// EvaluateWith is Evaluate with the caller's opt-ins stated explicitly. +func (e *Engine) EvaluateWith(pack, facts, evidence []byte, options Options) (result.Evaluation, *Failure) { // One byte-limit boundary for every surface: the MCP wire must refuse the // same oversized input the CLI's bounded reads refuse. for name, data := range map[string][]byte{"pack": pack, "facts": facts, "evidence": evidence} { @@ -56,26 +79,10 @@ func (e *Engine) Evaluate(pack, facts, evidence []byte, supported []string, comm } } } - validated, operational := e.validator.Validate(pack, validation.Options{ - Through: "semantic", - SupportedExtensions: supported, - Limits: carrier.DefaultLimits(), - }) - if operational != nil { - return result.Evaluation{}, &Failure{Code: operational.Code, Message: operational.Message, ExitCode: operational.ExitCode} - } - if validated.Status != "valid" { - return result.Evaluation{}, packNotConformant(validated) - } - - packDocument, failure := decodeInput(pack, "pack") + validated, packRoot, failure := e.conformance(pack, options) if failure != nil { return result.Evaluation{}, failure } - packRoot, ok := packDocument.(map[string]any) - if !ok { - return result.Evaluation{}, &Failure{Code: "JPS-EVALUATION-INTERNAL", Message: "The validated pack did not decode to a JSON object.", ExitCode: result.ExitInternal} - } factsDocument, failure := decodeInput(facts, "facts") if failure != nil { @@ -87,14 +94,25 @@ func (e *Engine) Evaluate(pack, facts, evidence []byte, supported []string, comm return result.Evaluation{}, failure } - disposition, trace := resolve(packRoot, factsDocument, presence) + budget := options.WorkBudget + if budget <= 0 { + budget = DefaultWorkBudget + } + disposition, trace, failure := resolve(packRoot, factsDocument, &evaluator{ + evidence: presence, + quantifiers: options.RFC0008Quantifiers, + budget: budget, + }) + if failure != nil { + return result.Evaluation{}, failure + } if trace == nil { trace = []result.TraceEntry{} } evaluation := result.Evaluation{ OutputVersion: result.OutputVersion, Tool: result.CurrentTool(), - Command: command, + Command: options.Command, Status: "evaluated", Experimental: true, ConformanceClaim: result.EvaluationClaim, @@ -102,6 +120,9 @@ func (e *Engine) Evaluate(pack, facts, evidence []byte, supported []string, comm Disposition: disposition, Trace: trace, } + if options.RFC0008Quantifiers { + evaluation.DraftPrototype = draftPrototype(packRoot, validated.SpecVersion) + } if set, err := artifacts.Load(validated.SpecVersion); err == nil { evaluation.Artifact = &result.Artifact{ SpecVersion: validated.SpecVersion, @@ -112,6 +133,36 @@ func (e *Engine) Evaluate(pack, facts, evidence []byte, supported []string, comm return evaluation, nil } +// conformance establishes the pack's conformance status and returns its +// decoded root. Without the draft opt-in this is the published path unchanged: +// full document conformance against the bundled schema, or a refusal. Under the +// opt-in it is the RFC 0008 gate, which no pack reaches by accident. +func (e *Engine) conformance(pack []byte, options Options) (result.Validation, map[string]any, *Failure) { + if options.RFC0008Quantifiers { + return e.rfc0008Conformance(pack, options) + } + validated, operational := e.validator.Validate(pack, validation.Options{ + Through: "semantic", + SupportedExtensions: options.SupportedExtensions, + Limits: carrier.DefaultLimits(), + }) + if operational != nil { + return result.Validation{}, nil, &Failure{Code: operational.Code, Message: operational.Message, ExitCode: operational.ExitCode} + } + if validated.Status != "valid" { + return result.Validation{}, nil, packNotConformant(validated) + } + document, failure := decodeInput(pack, "pack") + if failure != nil { + return result.Validation{}, nil, failure + } + packRoot, ok := document.(map[string]any) + if !ok { + return result.Validation{}, nil, &Failure{Code: "JPS-EVALUATION-INTERNAL", Message: "The validated pack did not decode to a JSON object.", ExitCode: result.ExitInternal} + } + return validated, packRoot, nil +} + // packNotConformant refuses a pack that did not reach full document // conformance, naming the first diagnostic so the refusal is self-sufficient. func packNotConformant(validated result.Validation) *Failure { diff --git a/internal/evaluation/quantifier.go b/internal/evaluation/quantifier.go new file mode 100644 index 0000000..d10ded5 --- /dev/null +++ b/internal/evaluation/quantifier.go @@ -0,0 +1,497 @@ +package evaluation + +import "encoding/json" + +// Draft RFC 0008 (bounded collection quantifiers) semantics and their work +// accounting. Nothing here is reachable unless the caller opts in: the +// operators belong to no published JPS version, a pack using them is not valid +// under 0.1.0-draft, and spec validate rejects it unchanged. + +// DefaultWorkBudget is this runtime's §10 evaluation-work limit for one +// evaluation under the draft grammar, in the work units charged below. It is a +// runtime choice, not a portable one: RFC 0008 leaves the common limit open, so +// two evaluators may pick different numbers and an above-limit input is not +// portable between them. The number is set so the RFC's own attack sketch — an +// aggregate over 10³ elements each carrying an aggregate over 10³ — is refused +// with room to spare: with the short paths and small values that sketch implies +// it charges about 6·10⁶ units against this budget of 100,000, a factor of +// about 60, while every collection a hand-authored pack plausibly quantifies +// over fits. +// +// It is also this runtime's §10 collection-size limit, which RFC 0008's uplift +// raises to a MUST alongside the evaluation-work one. No second knob states it, +// because the work budget already implies it: an aggregate whose where costs c +// units admits at most (budget - aggregate cost)/c elements, so the cheapest +// possible where (a one-unit literal) bounds one aggregate at just under +// 100,000 elements and an ordinary fact predicate over a short pointer, a +// boolean selected and a boolean authored — six units — bounds it at about +// 16,000. The document that carries those elements +// is bounded independently by the carrier layer's byte limit +// (carrier.HardMaxBytes) and nesting depth (carrier.DefaultLimits), which apply +// to the facts input before any condition is charged. +const DefaultWorkBudget = 100_000 + +// Work units. One unit is one elementary step of bounded cost: visiting one +// condition node, taking one step of a pointer resolution, or reading one byte +// of a path, an object member name, or a scalar token that a scan or a +// comparison has to look at. +// +// The byte-proportional half is load-bearing rather than pedantic. A pointer +// resolution scans, splits, and unescapes its whole path; a member lookup +// hashes the member name, including an evidence requirement's; numeric equality +// normalizes whole tokens and decimal ordering parses them. Charging a flat +// unit for any of those would let one condition node buy unbounded processing, +// and the budget would bound the shape of the work rather than its amount. +const ( + unitNode = 1 + // unitPointerScan is the fixed part of compiling one authored pointer; its + // variable part is the path's byte length. + unitPointerScan = 1 + // unitPointerStep is one step of a resolution of an already compiled + // pointer: one traversal per reference token, and one for a pointer with no + // tokens, which still selects the root. The variable part of a resolution + // is the bytes of those tokens. + unitPointerStep = 1 + // unitValue is the fixed part of one JSON node of a value; a scalar's + // variable part is its token length and an object member's is its name. + unitValue = 1 +) + +// quantify evaluates an RFC 0008 exists or every. The aggregate path resolves +// against the current condition root; where is then evaluated once per element +// with that element as the root, and the per-element results are combined by +// §7.2 (exists) and §7.1 (every) over a child list whose length is fixed at +// runtime. An unresolved or non-array path is unknown, mirroring §7.4. The +// empty array is the RFC's pinned choice, not an inheritance: false for exists +// and true for every, the identity elements of finite disjunction and +// conjunction. Short-circuiting is permitted on the dominant value only — true +// for exists, false for every — and never on unknown. +func (e *evaluator) quantify(node map[string]any, root any) tri { + elements, resolved := e.selectArray(node, root) + if !resolved { + return triUnknown + } + // dominant decides on sight; vacuous is both the empty-array value and the + // value an all-non-dominant array takes. + dominant, vacuous := triTrue, triFalse + if node["op"] == "every" { + dominant, vacuous = triFalse, triTrue + } + sawUnknown := false + for _, element := range elements { + switch e.condition(node["where"], element) { + case dominant: + return dominant + case triUnknown: + sawUnknown = true + } + } + if sawUnknown { + return triUnknown + } + return vacuous +} + +// uniform evaluates an RFC 0008 uniform by its five ordered clauses, the first +// that applies deciding: (1) an unresolved or non-array path is unknown; (2) an +// empty array is true; (3) any two members whose at-values both resolve and are +// unequal under §7.4 equality make it false, because a known counterexample +// dominates missing data; (4) otherwise an at that fails to resolve in any +// member — including a singleton — is unknown; (5) otherwise true. +// +// The five clauses are the whole of it, and they need no sixth arm. §7.4 +// equality is total over carrier-valid JSON — jsonEqual decides numbers on +// their tokens rather than on an arithmetic type that has to hold them — so +// there is no resolved pair whose comparison is "indeterminate" and no third +// thing for a clause to say about one. An implementation that cannot compare +// two admitted values has a resource problem, which is an error, not a +// disposition; it is not a semantics, and pinning one would have made the +// result depend on the evaluator's number type instead of on the documents. +// +// One representative therefore suffices for clause 3: total equality is +// transitive, so every member equal to the first stands for it and the first +// member that differs is a counterexample against all of them, in whatever +// order the members arrive. Clause 3 still dominates clause 4 — a member whose +// at is missing is skipped rather than answered, and the unknown it earns is +// only reached when the whole pass found no counterexample. +// +// The representative is compared against every other resolved member, so each +// value is canonicalized (canonicalNumbers) once as it is resolved rather than +// normalized inside each comparison. Both are the same equality; the difference +// is that one long token among many short equal ones is then read once instead +// of once per member. The accounting model reserves the comparisons either way +// — see the reread term on evaluate — but a bound is not a reason to spend it. +func (e *evaluator) uniform(node map[string]any, root any) tri { + at, ok := node["at"].(string) + if !ok { + return triUnknown + } + members, resolved := e.selectArray(node, root) + if !resolved { + return triUnknown // clause 1 + } + if len(members) == 0 { + return triTrue // clause 2 + } + var representative any + elected, missing := false, false + for _, member := range members { + value, resolved := e.resolve(member, at) + if !resolved { + missing = true + continue + } + value = canonicalNumbers(value) + if !elected { + representative, elected = value, true + continue + } + if !jsonEqual(representative, value) { + return triFalse // clause 3 + } + } + if missing { + return triUnknown // clause 4 + } + return triTrue // clause 5 +} + +// selectArray resolves an aggregate's path against the current condition root +// and reports its elements. The second result is false when the path does not +// resolve or does not select a JSON array, which every aggregate reads as +// unknown. An empty path selects the current root, so an array-valued element +// can be quantified over directly. +func (e *evaluator) selectArray(node map[string]any, root any) ([]any, bool) { + path, ok := node["path"].(string) + if !ok { + return nil, false + } + selected, resolved := e.resolve(root, path) + if !resolved { + return nil, false + } + elements, isArray := selected.([]any) + if !isArray { + return nil, false + } + return elements, true +} + +// evaluate charges one whole condition tree and only then evaluates it. +// +// The accounting model, stated in full, is the candidate RFC 0008 leaves open. +// It is the third candidate, and each predecessor was broken by a demonstrated +// attack rather than retired on taste. The first charged a flat unit per node, +// and a megabyte-long authored pointer resolved once per element bought +// unbounded scanning for two units apiece. The second made authored bytes +// count — paths and operands — and left the runtime values a condition +// actually reads unpriced, so a two-byte operand compared against a megabyte +// of facts, once per element, was again free. This one prices both sides: +// every authored operand and every selected value. +// +// - Work unit — one visited condition node, one step of a pointer +// resolution, or one byte of a path, an object member name, or a scalar +// token that a scan or a comparison has to read. Units are byte-sensitive +// wherever the processing they stand for is: charging one flat unit for a +// path, a string, or a numeric token would bound the number of nodes an +// evaluation touches while leaving the processing per node unbounded, which +// is not a limit at all. +// - Preflight — charge walks the tree without evaluating any predicate, but +// it does resolve pointers, including a fact's. An aggregate's cost is its +// element count and a comparison's cost is the size of the value its +// pointer selected; neither has an honest cheaper approximation, and a +// model that priced only what the author wrote would price only the half +// of a comparison an attacker does not control. Every resolution is charged +// before it is performed, so a pointer that fails to resolve still costs +// what it took to look it up, an unresolved or non-array aggregate path +// costs that lookup and nothing more, an unresolved fact path costs its +// lookup and its authored operand, and a path too long to afford is refused +// with its bytes still unread. Measuring a resolved value is the one step +// that runs before its own charge is known — its size cannot be charged +// until it is known — and it is bounded anyway: the charge lands +// immediately after, so the preflight stops within one selected value of +// the budget. A charged resolution is performed twice, once to price it and +// once to evaluate it, and a charged value is walked twice for the same +// reason; the budget bounds the work up to that constant factor of two, +// which is what a work unit is for. +// - Per pointer — resolving an authored pointer costs two things. Compiling +// it — reading its bytes, splitting it, decoding its escapes — costs +// 1 + len(path) and is charged, and performed, at most once per distinct +// authored pointer in one evaluation, because the compiled form is cached +// and reused. Each resolution of the compiled form then costs +// max(1, tokens) + Σ len(token): one step per reference token, plus the +// token bytes that step hashes or compares against the document. The steps +// are counted rather than folded into the bytes, because a reference +// token can be empty and still cost a lookup — "/a////b" is five traversals +// over two bytes of token — and the floor of one is the resolution that +// walks no token at all, which still selects the root. A path a where +// resolves once per element is therefore scanned once and stepped per +// element, and the charge says exactly that. +// - Per value — a scalar costs 1 plus its token length: a string its +// characters, a number its digits, and a boolean or null nothing beyond the +// 1. A container costs 1 plus its members, and an object member also costs +// its name's bytes, because comparing objects looks names up. This is what +// a deep equality or an in over that value can cost at most, since §7.4 +// equality descends only where both sides have the same shape. +// - Per node — literal costs 1; evidence-present costs 1 plus the bytes of +// the requirement id, which every one of its lookups hashes and which a +// where inside an aggregate looks up once per element; not and all/any cost +// 1 plus their children, every branch included, so a branch a +// short-circuiting evaluator never reaches is still charged; fact costs 1 +// plus its pointer, plus the size of the value that pointer selected, plus +// the size of its authored operand. Both sides, because a comparison reads +// both and only one of them is authored — and each by its bytes, so a long +// decimal or numeric token is not pretended to be one unit on either side. +// in charges the selected value once per candidate, since each candidate +// can be compared against the whole of it, and the candidate array's own +// size once. +// - Per aggregate — exists and every cost 1 plus their path resolution plus +// the sum of their where's cost over the elements actually present, so a +// ragged nesting costs Σᵢ|Bᵢ| and never |A|×|B|; uniform, which has no +// where, costs 1 plus its path resolution plus two terms over its members. +// The first is per member: one resolution of at and the size of the value +// that resolution selected, which pays for the single pass that reads each +// value once. The second is the reread term, (n-1)×max over the n resolved +// values, which pays for the comparisons. §7.4 equality is total, so one +// representative absorbs every member it equals and the comparison count +// stays linear — but the representative is one of the values and it is +// compared against each of the other n-1, and any of those comparisons can +// read the whole of it, so the sum of the values is not a bound on the pass +// and the earlier "one pass is the whole cost" claim was wrong: one long +// token among short equal ones was work proportional to long×members priced +// at long+members. The term is a maximum over the resolved values rather +// than the elected representative's size precisely so that it does not +// depend on which member is elected, and therefore not on the order the +// members arrive in; it bounds the pass whichever one is. The evaluator +// then spends less than the bound — it canonicalizes each value once, so a +// comparison compares canonical forms — but the charge is the bound. +// - Composition — sibling aggregates add, since the charge is a sum over the +// tree; the budget accumulates across every condition of one evaluation. +// A condition §8 never reaches is never charged: a suppressed rule, every +// rule after a step-6 forced outcome, and everything after a false or +// unknown applicability. The total is therefore a property of the §8 path +// rather than of pack and facts alone — but §8 fixes that path, so it is +// the same path, and the same total, for every evaluator that follows it. +// +// The charge is a sum over the elements present and over the distinct authored +// pointers the tree names, and neither depends on the order elements arrive in, +// so it is identical under any permutation of them. It is not identical under +// duplication: a duplicated element is one more element, and it is charged like +// any other, which may be what crosses the limit. Only the condition's value is +// duplication-invariant, and only while both inputs remain within the limits — +// which is exactly what RFC 0008's result-invariance wording says. +// +// Each condition tree's charge is complete before the first predicate of that +// tree runs. Short-circuiting can therefore only reduce actual work: it can +// never change whether the limit was exceeded. Exceeding it is an explicit +// evaluation error, never a disposition — the caller must refuse rather than +// report — so this returns unknown only to unwind, and the caller checks +// exceeded before reading the value. +func (e *evaluator) evaluate(node any, root any) tri { + if e.quantifiers { + e.preflight(node, root) + if e.exceeded { + return triUnknown + } + } + return e.condition(node, root) +} + +// charge adds units to the running total and reports whether the budget still +// holds. Preflight stops at the first overrun, which bounds the preflight's own +// work by the budget without changing the verdict: the total is a monotone sum, +// so a prefix exceeds the budget exactly when the whole tree does. +func (e *evaluator) charge(units int) bool { + e.charged += units + if e.charged > e.budget { + e.exceeded = true + } + return !e.exceeded +} + +// chargePointer reserves everything one resolution of an authored pointer can +// cost and reports whether the budget still holds. The scan that compiles the +// path is charged before a byte of it is read and is charged once per distinct +// authored pointer, because the compiled form is then cached for the rest of +// the evaluation; the resolution itself is charged per call, at one step per +// reference token plus the bytes that step reads. Reserving before resolving is +// the whole point: an authored path long enough to be expensive to scan is +// refused by the budget rather than scanned and then billed for. +// +// The step count is the token count and not a flat one, because a resolution +// loops once per token whatever the tokens weigh: a pointer of a hundred empty +// reference tokens ("////…") carries no token bytes at all and still performs a +// hundred map traversals on every one of its resolutions. Charging the bytes +// alone priced that at one unit per resolution. The floor of one is the empty +// pointer, whose single "resolution" selects the root. +func (e *evaluator) chargePointer(path string) bool { + compiled, scanned := e.pointers[path] + if !scanned { + if !e.charge(unitPointerScan + len(path)) { + return false + } + compiled = e.compiled(path) + } + steps := len(compiled.tokens) + if steps < 1 { + steps = 1 + } + return e.charge(steps*unitPointerStep + compiled.bytes) +} + +// preflight charges one condition tree under the model documented on evaluate. +func (e *evaluator) preflight(node any, root any) { + if e.exceeded { + return + } + condition, ok := node.(map[string]any) + if !ok { + e.charge(unitNode) + return + } + switch condition["op"] { + case "not": + if e.charge(unitNode) { + e.preflight(condition["condition"], root) + } + case "all", "any": + if !e.charge(unitNode) { + return + } + for _, child := range asArray(condition["conditions"]) { + e.preflight(child, root) + if e.exceeded { + return + } + } + case "fact": + if !e.charge(unitNode) { + return + } + path, _ := condition["path"].(string) + if !e.chargePointer(path) { + return + } + // The resolution the pointer charge just paid for is performed here, so + // the value the comparison will actually read can be priced. Charging + // the authored operand alone would price the half of the comparison the + // author wrote and leave the half the facts document supplies free, + // which is the same defect the flat pointer charge had. + selected := 0 + if value, resolved := e.resolve(root, path); resolved { + selected = valueUnits(value) + } + // in compares the selected value against each candidate in turn, and + // each of those comparisons can read all of it. + comparisons := 1 + if condition["operator"] == "in" { + comparisons = len(asArray(condition["value"])) + } + e.charge(comparisons*selected + valueUnits(condition["value"])) + case "evidence-present": + // The lookup hashes the requirement id, and a where does it once per + // element, so the id's bytes are charged like any other name a lookup + // reads. + name, _ := condition["evidenceRequirement"].(string) + e.charge(unitNode + len(name)) + case "exists", "every": + if !e.charge(unitNode) { + return + } + path, _ := condition["path"].(string) + if !e.chargePointer(path) { + return + } + elements, _ := e.selectArray(condition, root) + for _, element := range elements { + e.preflight(condition["where"], element) + if e.exceeded { + return + } + } + case "uniform": + if !e.charge(unitNode) { + return + } + path, _ := condition["path"].(string) + if !e.chargePointer(path) { + return + } + at, _ := condition["at"].(string) + members, _ := e.selectArray(condition, root) + // Two terms. The first is one resolution of at per member plus the size + // of the value it selected: the single pass that reads each value once. + // The second is the reread term below, which prices the comparisons. + resolvedValues, largest := 0, 0 + for _, member := range members { + // The member's at is reserved before it is resolved, exactly as the + // aggregate's own path was. + if !e.chargePointer(at) { + return + } + value, resolved := e.resolve(member, at) + if !resolved { + continue + } + units := valueUnits(value) + resolvedValues++ + if units > largest { + largest = units + } + if !e.charge(units) { + return + } + } + // §7.4 equality is total, so one elected representative absorbs every + // member equal to it and there are n-1 comparisons rather than n²/2 — + // but the representative is a value like any other, and each of those + // comparisons can read the whole of it. Charging one reread of the + // largest resolved value per comparison bounds the pass whichever + // member is elected, which is also what keeps the term independent of + // the members' order: the count and the maximum are properties of the + // set. Both are unchanged by any permutation, so the total is. + if resolvedValues > 1 { + e.charge((resolvedValues - 1) * largest) + } + default: + // literal and any operand shape §7 does not define. + e.charge(unitNode) + } +} + +// valueUnits is the size of a value in work units: a container costs one plus +// its members, an object member costs its name's bytes as well, and a scalar +// costs one plus the length of its JSON token — a string's characters, a +// number's digits, and nothing beyond the one for a boolean or null. It is what +// a deep equality or an in over that value can cost at most, since §7.4 +// equality descends only where both sides have the same shape. +// +// The token length is not decoration. §7.4 equality on numbers normalizes both +// tokens byte by byte and RFC 0006's ordered comparison parses two decimal +// strings, so a long value compared once per element is work proportional to +// its length times the element count; charging one unit per scalar would have +// priced that at one unit per element. It applies to whichever side is long: +// the operand the author wrote and the value the pointer selected are charged +// by the same function for the same reason. +func valueUnits(value any) int { + switch typed := value.(type) { + case []any: + units := unitValue + for _, item := range typed { + units += valueUnits(item) + } + return units + case map[string]any: + units := unitValue + for name, item := range typed { + units += len(name) + valueUnits(item) + } + return units + case string: + return unitValue + len(typed) + case json.Number: + return unitValue + len(typed.String()) + default: + return unitValue + } +} diff --git a/internal/evaluation/quantifier_test.go b/internal/evaluation/quantifier_test.go new file mode 100644 index 0000000..04916df --- /dev/null +++ b/internal/evaluation/quantifier_test.go @@ -0,0 +1,988 @@ +package evaluation + +import ( + "fmt" + "math" + "reflect" + "strings" + "testing" + "time" + + "github.com/Judgment-Pack/judgment-pack-runtime/internal/carrier" +) + +// documentOf decodes one JSON document exactly as the engine decodes its +// inputs, so numbers arrive as json.Number and every row below reads as the +// document an author or a producer would actually supply. +func documentOf(t *testing.T, document string) any { + t.Helper() + value, failure := carrier.Decode([]byte(document), carrier.DefaultLimits()) + if failure != nil { + t.Fatalf("undecodable document %s: %s", document, failure.Diagnostic.Message) + } + return value +} + +func conditionOf(t *testing.T, document string) map[string]any { + t.Helper() + condition, ok := documentOf(t, document).(map[string]any) + if !ok { + t.Fatalf("condition must be a JSON object: %s", document) + } + return condition +} + +// draftEval evaluates one condition under the draft RFC 0008 grammar with the +// default budget, which every semantic row in this file fits. A row that +// exhausts the budget is a bug in the row: exhaustion is an error, not a value. +func draftEval(t *testing.T, condition, facts string, evidence map[string]tri) tri { + t.Helper() + evaluator := &evaluator{evidence: evidence, quantifiers: true, budget: DefaultWorkBudget} + verdict := evaluator.evaluate(conditionOf(t, condition), documentOf(t, facts)) + if evaluator.exceeded { + t.Fatalf("row exhausted the work budget after %d units", evaluator.charged) + } + return verdict +} + +// The RFC's positive and negative rows, its boundary rows, and its scoping +// rows, one table entry each. Where a row is pinned per operator the operator +// is named in the row rather than inferred. +func TestRFC0008QuantifierConformanceRows(t *testing.T) { + const segments = `{"reservation":{"segments":[{"cancelledByAirline":false},{"cancelledByAirline":true},{"cancelledByAirline":false}]}}` + const allTrue = `{"reservation":{"segments":[{"cancelledByAirline":true},{"cancelledByAirline":true}]}}` + const allFalse = `{"reservation":{"segments":[{"cancelledByAirline":false},{"cancelledByAirline":false}]}}` + const oneFalse = `{"reservation":{"segments":[{"cancelledByAirline":true},{"cancelledByAirline":false}]}}` + const empty = `{"reservation":{"segments":[]}}` + const cancelled = `{"op":"fact","path":"/cancelledByAirline","operator":"equals","value":true}` + + existsOver := func(path, where string) string { + return fmt.Sprintf(`{"op":"exists","path":%q,"where":%s}`, path, where) + } + everyOver := func(path, where string) string { + return fmt.Sprintf(`{"op":"every","path":%q,"where":%s}`, path, where) + } + + cases := []struct { + name string + condition string + facts string + want tri + }{ + // Positive and negative. + {"exists-one-of-three-matches", existsOver("/reservation/segments", cancelled), segments, triTrue}, + {"every-all-match", everyOver("/reservation/segments", cancelled), allTrue, triTrue}, + {"exists-non-empty-all-false", existsOver("/reservation/segments", cancelled), allFalse, triFalse}, + {"every-exactly-one-false", everyOver("/reservation/segments", cancelled), oneFalse, triFalse}, + {"exists-where-names-absent-member", existsOver("/reservation/segments", + `{"op":"fact","path":"/noSuchMember","operator":"equals","value":true}`), allTrue, triUnknown}, + {"every-where-names-absent-member", everyOver("/reservation/segments", + `{"op":"fact","path":"/noSuchMember","operator":"equals","value":true}`), allTrue, triUnknown}, + + // Boundary: the empty array is pinned for both operators. + {"exists-empty-array-is-false", existsOver("/reservation/segments", cancelled), empty, triFalse}, + {"every-empty-array-is-true", everyOver("/reservation/segments", cancelled), empty, triTrue}, + + // Boundary: unknown dominance in all four directions. + {"exists-unknown-and-none-true", existsOver("/list", cancelled), + `{"list":[{"cancelledByAirline":false},{}]}`, triUnknown}, + {"exists-unknown-and-one-true", existsOver("/list", cancelled), + `{"list":[{},{"cancelledByAirline":true}]}`, triTrue}, + {"every-unknown-and-rest-true", everyOver("/list", cancelled), + `{"list":[{"cancelledByAirline":true},{}]}`, triUnknown}, + {"every-unknown-and-one-false", everyOver("/list", cancelled), + `{"list":[{},{"cancelledByAirline":false}]}`, triFalse}, + + // Ragged arrays, one row per operator and per position of the dominant + // element, exactly as the RFC tabulates them. + {"ragged-exists-false-false-missing", existsOver("/list", cancelled), + `{"list":[{"cancelledByAirline":false},{"cancelledByAirline":false},{}]}`, triUnknown}, + {"ragged-exists-missing-true-false", existsOver("/list", cancelled), + `{"list":[{},{"cancelledByAirline":true},{"cancelledByAirline":false}]}`, triTrue}, + {"ragged-every-true-true-missing", everyOver("/list", cancelled), + `{"list":[{"cancelledByAirline":true},{"cancelledByAirline":true},{}]}`, triUnknown}, + {"ragged-every-missing-false-true", everyOver("/list", cancelled), + `{"list":[{},{"cancelledByAirline":false},{"cancelledByAirline":true}]}`, triFalse}, + + // Singleton with a predicate. + {"singleton-true-exists", existsOver("/list", `{"op":"fact","path":"/ok","operator":"equals","value":true}`), + `{"list":[{"ok":true}]}`, triTrue}, + {"singleton-true-every", everyOver("/list", `{"op":"fact","path":"/ok","operator":"equals","value":true}`), + `{"list":[{"ok":true}]}`, triTrue}, + {"singleton-false-exists", existsOver("/list", `{"op":"fact","path":"/ok","operator":"equals","value":true}`), + `{"list":[{"ok":false}]}`, triFalse}, + {"singleton-false-every", everyOver("/list", `{"op":"fact","path":"/ok","operator":"equals","value":true}`), + `{"list":[{"ok":false}]}`, triFalse}, + {"singleton-missing-pointer-exists", existsOver("/list", `{"op":"fact","path":"/ok","operator":"equals","value":true}`), + `{"list":[{}]}`, triUnknown}, + {"singleton-missing-pointer-every", everyOver("/list", `{"op":"fact","path":"/ok","operator":"equals","value":true}`), + `{"list":[{}]}`, triUnknown}, + + // Non-array values at path, one row per JSON type, plus the unresolved + // pointer. Each is unknown, mirroring §7.4. + {"non-array-object", everyOver("/value", cancelled), `{"value":{"a":1}}`, triUnknown}, + {"non-array-string", everyOver("/value", cancelled), `{"value":"segments"}`, triUnknown}, + {"non-array-number", everyOver("/value", cancelled), `{"value":3}`, triUnknown}, + {"non-array-null", everyOver("/value", cancelled), `{"value":null}`, triUnknown}, + {"non-array-true", everyOver("/value", cancelled), `{"value":true}`, triUnknown}, + {"unresolved-path-exists", existsOver("/missing", cancelled), `{"value":[]}`, triUnknown}, + {"unresolved-path-every", everyOver("/missing", cancelled), `{"value":[]}`, triUnknown}, + + // Scope and re-rooting. + {"empty-pointer-over-scalar-elements-true", + everyOver("/list", `{"op":"fact","path":"","operator":"equals","value":"gold"}`), + `{"list":["gold","gold"]}`, triTrue}, + {"empty-pointer-over-scalar-elements-false", + everyOver("/list", `{"op":"fact","path":"","operator":"equals","value":"gold"}`), + `{"list":["gold","silver"]}`, triFalse}, + {"empty-aggregate-path-selects-current-root", + existsOver("", `{"op":"fact","path":"","operator":"equals","value":"gold"}`), + `["silver","gold"]`, triTrue}, + {"collision-inner-root-wins", + existsOver("/items", `{"op":"fact","path":"/status","operator":"equals","value":"inner"}`), + `{"status":"outer","items":[{"status":"inner"}]}`, triTrue}, + {"collision-outer-value-is-not-visible", + existsOver("/items", `{"op":"fact","path":"/status","operator":"equals","value":"outer"}`), + `{"status":"outer","items":[{"status":"inner"}]}`, triFalse}, + {"outer-pointer-intent-is-read-element-relative", + existsOver("/list", `{"op":"fact","path":"/threshold","operator":"equals","value":"5"}`), + `{"threshold":"5","list":[{"amount":"7"}]}`, triUnknown}, + {"nesting-one-level-true", + existsOver("/rows", everyOver("/cells", `{"op":"fact","path":"/ok","operator":"equals","value":true}`)), + `{"rows":[{"cells":[{"ok":false}]},{"cells":[{"ok":true},{"ok":true}]}]}`, triTrue}, + {"nested-inner-path-resolves-only-at-facts-root", + existsOver("/rows", existsOver("/other", `{"op":"fact","path":"/ok","operator":"equals","value":true}`)), + `{"other":[{"ok":true}],"rows":[{"cells":[]}]}`, triUnknown}, + {"inner-where-re-roots-per-level", + existsOver("/rows", existsOver("/cells", `{"op":"fact","path":"/tag","operator":"equals","value":"t"}`)), + `{"rows":[{"tag":"t","cells":[{"z":1}]}]}`, triUnknown}, + } + + for _, testCase := range cases { + t.Run(testCase.name, func(t *testing.T) { + if got := draftEval(t, testCase.condition, testCase.facts, map[string]tri{}); got != testCase.want { + t.Fatalf("%s = %v, want %v", testCase.name, got, testCase.want) + } + }) + } +} + +// evidence-present reads no facts document, so it is element-invariant, and an +// empty array overrides it in both directions: every is true and exists is +// false whether the evidence is present or absent. +func TestRFC0008EmptyArrayOverridesElementInvariantEvidence(t *testing.T) { + const where = `{"op":"evidence-present","evidenceRequirement":"intake-form"}` + cases := []struct { + name string + condition string + presence tri + want tri + }{ + {"every-empty-evidence-present", fmt.Sprintf(`{"op":"every","path":"/list","where":%s}`, where), triTrue, triTrue}, + {"every-empty-evidence-absent", fmt.Sprintf(`{"op":"every","path":"/list","where":%s}`, where), triFalse, triTrue}, + {"exists-empty-evidence-present", fmt.Sprintf(`{"op":"exists","path":"/list","where":%s}`, where), triTrue, triFalse}, + {"exists-empty-evidence-absent", fmt.Sprintf(`{"op":"exists","path":"/list","where":%s}`, where), triFalse, triFalse}, + // Non-empty, for contrast: with elements present the element-invariant + // value decides both operators. + {"every-non-empty-evidence-absent", fmt.Sprintf(`{"op":"every","path":"/two","where":%s}`, where), triFalse, triFalse}, + {"exists-non-empty-evidence-present", fmt.Sprintf(`{"op":"exists","path":"/two","where":%s}`, where), triTrue, triTrue}, + } + for _, testCase := range cases { + t.Run(testCase.name, func(t *testing.T) { + evidence := map[string]tri{"intake-form": testCase.presence} + if got := draftEval(t, testCase.condition, `{"list":[],"two":[{},{}]}`, evidence); got != testCase.want { + t.Fatalf("%s = %v, want %v", testCase.name, got, testCase.want) + } + }) + } +} + +// Element order carries no meaning: within the limits, a permutation or a +// duplicated element cannot change the result. Permitted short-circuiting on +// the dominant value must not make it change either. This is a claim about the +// condition's value only; the work charge is invariant under permutation but +// not under duplication, which +// TestRFC0008DuplicationRaisesTheChargeAndKeepsTheValue pins separately. +func TestRFC0008PermutationAndDuplicateInvariance(t *testing.T) { + const where = `{"op":"fact","path":"/ok","operator":"equals","value":true}` + elements := []string{`{"ok":true}`, `{"ok":false}`, `{}`} + + // The labels name which element leads, not which value dominates: the + // dominant value is true for exists and false for every, so one arrangement + // cannot be "dominant-first" for both operators, and every position is + // covered for each of them either way. + arrangements := map[string][]int{ + "true-first": {0, 1, 2}, + "false-first": {1, 0, 2}, + "missing-first": {2, 0, 1}, + "reversed": {2, 1, 0}, + "duplicated": {1, 0, 2, 0, 1}, + } + for _, op := range []string{"exists", "every"} { + want := tri(-1) + for _, name := range []string{"true-first", "false-first", "missing-first", "reversed", "duplicated"} { + items := make([]string, 0, len(arrangements[name])) + for _, index := range arrangements[name] { + items = append(items, elements[index]) + } + facts := fmt.Sprintf(`{"list":[%s]}`, strings.Join(items, ",")) + got := draftEval(t, fmt.Sprintf(`{"op":%q,"path":"/list","where":%s}`, op, where), facts, map[string]tri{}) + if want == tri(-1) { + want = got + continue + } + if got != want { + t.Fatalf("%s over %s = %v, want %v (the true-first order's value)", op, name, got, want) + } + } + // The invariant value is itself pinned, so an evaluator cannot satisfy + // this test by being uniformly wrong. + expected := triTrue + if op == "every" { + expected = triFalse + } + if want != expected { + t.Fatalf("%s over one true, one false, and one missing pointer = %v, want %v", op, want, expected) + } + } +} + +// uniform's five clauses, applied in order, one row each, plus the §7.4 +// equality rows the RFC pins and a permutation of every one of them. +// +// Every row is conformance to the five clauses as published; none pins an +// extension of them. The rows over numbers no arithmetic type can hold are +// where that used to be in doubt: they are ordinary clause 3 and clause 5 rows, +// because §7.4 equality is total and decides them on the tokens. See the note +// on uniform. +func TestRFC0008UniformConformanceRows(t *testing.T) { + cases := []struct { + name string + condition string + facts string + want tri + }{ + {"clause-1-unresolved-path", `{"op":"uniform","path":"/missing","at":"/cabin"}`, `{"list":[]}`, triUnknown}, + {"clause-1-non-array-path", `{"op":"uniform","path":"/value","at":"/cabin"}`, `{"value":{"cabin":"economy"}}`, triUnknown}, + {"clause-2-empty-array", `{"op":"uniform","path":"/list","at":"/cabin"}`, `{"list":[]}`, triTrue}, + {"clause-3-unequal", `{"op":"uniform","path":"/list","at":"/cabin"}`, + `{"list":[{"cabin":"economy"},{"cabin":"business"}]}`, triFalse}, + {"clause-3-beats-clause-4", `{"op":"uniform","path":"/list","at":"/cabin"}`, + `{"list":[{"cabin":1},{"cabin":2},{}]}`, triFalse}, + + // Numbers past every arithmetic type, under the same two clauses as any + // other value. Equality is decided on the tokens, so an at-value of + // 1e999999999 is a counterexample against 2e999999999 exactly as 1 is + // against 2, and 1e999999999 confirms 1.0e999999999 exactly as 1000 + // confirms 1e3. No row here is unknown: unknown is what a missing at + // earns (clause 4), not what an expensive number earns. + {"clause-3-huge-token-against-small-ones", `{"op":"uniform","path":"/list","at":"/cabin"}`, + `{"list":[{"cabin":1e999999999},{"cabin":1},{"cabin":2}]}`, triFalse}, + {"clause-3-huge-token-differs-from-its-equal-neighbours", `{"op":"uniform","path":"/list","at":"/cabin"}`, + `{"list":[{"cabin":1e999999999},{"cabin":1},{"cabin":1}]}`, triFalse}, + {"clause-3-distinct-huge-tokens-are-unequal", `{"op":"uniform","path":"/list","at":"/cabin"}`, + `{"list":[{"cabin":1e999999999},{"cabin":2e999999999}]}`, triFalse}, + {"clause-3-huge-token-inside-a-composite-at-value", `{"op":"uniform","path":"/list","at":"/seats"}`, + `{"list":[{"seats":[1,1e999999999]},{"seats":[2,1e999999999]}]}`, triFalse}, + {"clause-5-equal-huge-tokens-written-differently", `{"op":"uniform","path":"/list","at":"/cabin"}`, + `{"list":[{"cabin":1e999999999},{"cabin":1.0e999999999},{"cabin":10e999999998}]}`, triTrue}, + {"clause-5-equal-numbers-written-differently", `{"op":"uniform","path":"/list","at":"/cabin"}`, + `{"list":[{"cabin":1000},{"cabin":1e3},{"cabin":1.0e3}]}`, triTrue}, + {"clause-5-signed-zero-is-one-value", `{"op":"uniform","path":"/list","at":"/cabin"}`, + `{"list":[{"cabin":-0},{"cabin":0},{"cabin":0.0e10}]}`, triTrue}, + + {"clause-4-missing-at-among-equals", `{"op":"uniform","path":"/list","at":"/cabin"}`, + `{"list":[{"cabin":"economy"},{"cabin":"economy"},{}]}`, triUnknown}, + {"clause-4-singleton-missing-at-is-not-true", `{"op":"uniform","path":"/list","at":"/cabin"}`, + `{"list":[{}]}`, triUnknown}, + {"clause-5-all-equal", `{"op":"uniform","path":"/list","at":"/cabin"}`, + `{"list":[{"cabin":"economy"},{"cabin":"economy"}]}`, triTrue}, + {"singleton-with-at-present", `{"op":"uniform","path":"/list","at":"/cabin"}`, + `{"list":[{"cabin":"economy"}]}`, triTrue}, + {"at-is-member-relative-not-root-relative", `{"op":"uniform","path":"/list","at":"/cabin"}`, + `{"cabin":"economy","list":[{},{}]}`, triUnknown}, + + // The empty at selects the whole member and compares members to each + // other under §7.4 recursive equality. + {"empty-at-equal-members", `{"op":"uniform","path":"/list","at":""}`, + `{"list":[{"a":1,"b":[2]},{"b":[2],"a":1.0}]}`, triTrue}, + {"empty-at-unequal-members", `{"op":"uniform","path":"/list","at":""}`, + `{"list":[{"a":1},{"a":2}]}`, triFalse}, + + // §7.4 equality, exactly as it behaves for a fact operand. + {"null-equals-null", `{"op":"uniform","path":"/list","at":"/cabin"}`, + `{"list":[{"cabin":null},{"cabin":null}]}`, triTrue}, + {"array-at-in-order", `{"op":"uniform","path":"/list","at":"/seats"}`, + `{"list":[{"seats":[1,2]},{"seats":[1,2]}]}`, triTrue}, + {"array-at-out-of-order", `{"op":"uniform","path":"/list","at":"/seats"}`, + `{"list":[{"seats":[1,2]},{"seats":[2,1]}]}`, triFalse}, + {"object-at-disregards-member-order", `{"op":"uniform","path":"/list","at":"/fare"}`, + `{"list":[{"fare":{"code":"Y","refundable":true}},{"fare":{"refundable":true,"code":"Y"}}]}`, triTrue}, + } + + for _, testCase := range cases { + t.Run(testCase.name, func(t *testing.T) { + got := draftEval(t, testCase.condition, testCase.facts, map[string]tri{}) + if got != testCase.want { + t.Fatalf("%s = %v, want %v", testCase.name, got, testCase.want) + } + // Permutation of the elements in every row above: identical result. + if permuted, ok := reversedElements(t, testCase.facts); ok { + if got := draftEval(t, testCase.condition, permuted, map[string]tri{}); got != testCase.want { + t.Fatalf("%s permuted = %v, want %v", testCase.name, got, testCase.want) + } + } + }) + } +} + +// reversedElements rewrites a facts document's /list array in reverse order, +// which is the permutation every uniform row is also run under. It reports +// false for a document carrying no such array. +func reversedElements(t *testing.T, facts string) (string, bool) { + t.Helper() + document, ok := documentOf(t, facts).(map[string]any) + if !ok { + return "", false + } + elements, ok := document["list"].([]any) + if !ok || len(elements) < 2 { + return "", false + } + prefix := strings.Index(facts, "[") + suffix := strings.LastIndex(facts, "]") + if prefix < 0 || suffix < prefix { + return "", false + } + items := splitTopLevel(facts[prefix+1 : suffix]) + if len(items) != len(elements) { + return "", false + } + for left, right := 0, len(items)-1; left < right; left, right = left+1, right-1 { + items[left], items[right] = items[right], items[left] + } + return facts[:prefix+1] + strings.Join(items, ",") + facts[suffix:], true +} + +// splitTopLevel splits a JSON array body on the commas that separate its own +// elements, ignoring commas nested inside members. +func splitTopLevel(body string) []string { + items := []string{} + depth, start, inString, escaped := 0, 0, false, false + for index, character := range body { + switch { + case escaped: + escaped = false + case character == '\\' && inString: + escaped = true + case character == '"': + inString = !inString + case inString: + case character == '{' || character == '[': + depth++ + case character == '}' || character == ']': + depth-- + case character == ',' && depth == 0: + items = append(items, body[start:index]) + start = index + 1 + } + } + return append(items, body[start:]) +} + +// chargeOf runs the preflight alone, with a budget no test row can reach, and +// reports the units charged. It never evaluates a predicate. +func chargeOf(t *testing.T, condition, facts string) int { + t.Helper() + evaluator := &evaluator{quantifiers: true, budget: math.MaxInt} + evaluator.preflight(conditionOf(t, condition), documentOf(t, facts)) + return evaluator.charged +} + +// The charge is a sum over the elements present and over the distinct authored +// pointers the tree names, and neither summand depends on the order elements +// arrive in, so the charge is identical under any permutation of them — the +// property that makes short-circuiting safe, since it cannot change whether the +// limit was exceeded. The pointer cache does not weaken it: whichever element +// happens to be preflighted first pays the one scan, and the total is the same. +func TestRFC0008WorkChargeIsOrderIndependent(t *testing.T) { + const condition = `{"op":"exists","path":"/list","where":{"op":"fact","path":"/ok","operator":"equals","value":true}}` + arrangements := []string{ + `{"list":[{"ok":true},{"ok":false},{}]}`, + `{"list":[{},{"ok":false},{"ok":true}]}`, + `{"list":[{"ok":false},{"ok":true},{}]}`, + } + want := chargeOf(t, condition, arrangements[0]) + for _, facts := range arrangements[1:] { + if got := chargeOf(t, condition, facts); got != want { + t.Fatalf("charge for %s = %d, want %d", facts, got, want) + } + } +} + +// Duplication is not permutation, and the accounting model does not claim it +// is. A duplicated element is one more element and is charged like any other, +// so the charge strictly increases; what survives duplication is the condition's +// value, and only while both inputs remain within the limits, which is what RFC +// 0008's result-invariance wording actually says. +func TestRFC0008DuplicationRaisesTheChargeAndKeepsTheValue(t *testing.T) { + const where = `{"op":"fact","path":"/ok","operator":"equals","value":true}` + const condition = `{"op":"every","path":"/list","where":` + where + `}` + const once = `{"list":[{"ok":true},{"ok":false}]}` + const twice = `{"list":[{"ok":true},{"ok":false},{"ok":false},{"ok":true}]}` + + onceCharge := chargeOf(t, condition, once) + twiceCharge := chargeOf(t, condition, twice) + // Each duplicate costs a fact node, one step of the already-compiled "/ok", + // the boolean it selects, and the boolean operand: six units apiece, twelve + // for the two. + if twiceCharge-onceCharge != 12 { + t.Fatalf("duplicating two elements raised the charge from %d to %d; the model says it must rise by 12", onceCharge, twiceCharge) + } + if twiceCharge <= onceCharge { + t.Fatalf("duplication must raise the charge: %d then %d", onceCharge, twiceCharge) + } + + // Both fit the budget, so the value is unchanged — the invariance the RFC + // states, and the only one duplication has. + first := draftEval(t, condition, once, map[string]tri{}) + second := draftEval(t, condition, twice, map[string]tri{}) + if first != second || first != triFalse { + t.Fatalf("duplication changed the value: %v then %v, want false", first, second) + } + + // And the increase is not cosmetic: a budget the un-duplicated input fits + // refuses the duplicated one, which is precisely why the charge cannot be + // called duplication-invariant. + engine := newTestEngine(t) + pack := draftPack(condition, "escalate") + if _, failure := engine.EvaluateWith(pack, []byte(once), nil, draftOptions(onceCharge)); failure != nil { + t.Fatalf("a budget equal to the charge must admit the un-duplicated input: %+v", failure) + } + _, failure := engine.EvaluateWith(pack, []byte(twice), nil, draftOptions(onceCharge)) + if failure == nil || failure.Code != "JPS-RESOURCE-EVALUATION-WORK-LIMIT" { + t.Fatalf("the same budget must refuse the duplicated input: %+v", failure) + } +} + +// The accounting model's per-shape obligations, each as one row with the exact +// charge the model produces. Stating the numbers rather than an inequality is +// the point: an accounting model that cannot be recomputed by hand is not a +// model. +// +// Two quantities recur, so they are stated once here rather than in every row. +// A pointer costs 1+len(path) the first time this evaluation compiles it and +// max(1,tokens)+Σlen(token) for each resolution — one traversal per reference +// token, empty ones included — so the single-token "/ok" costs 4+3 = 7 on first +// use and 3 on every later one, "/list" costs 6+5 = 11 then 5, "/value" and +// "/cabin" cost 7+6 = 13 then 6, "/rows" costs 6+5 = 11 then 5, "/cells" costs +// 7+6 = 13 then 6, "/fare" costs 6+5 = 11 then 5, and "/missing" costs 9+8 = +// 17. A scalar value costs 1 plus its token length, so true costs 1, "Y" costs +// 2, and the number 1 costs 2. +// +// A fact charges both sides of its comparison: the operand its author wrote and +// the value its pointer selected. Every row over a resolving fact therefore +// carries a summand the facts document supplies, and a row whose pointer does +// not resolve carries none. +func TestRFC0008WorkChargeModel(t *testing.T) { + const elementPredicate = `{"op":"fact","path":"/ok","operator":"equals","value":true}` + cases := []struct { + name string + condition string + facts string + want int + why string + }{ + { + name: "fact-node-and-pointer", + condition: elementPredicate, + facts: `{"ok":true}`, + want: 10, + why: "one node, the compile and one resolution of \"/ok\" (7), one unit for the boolean it selected, and one for the boolean operand", + }, + { + name: "pointer-costs-one-step-per-reference-token-including-empty-ones", + condition: `{"op":"fact","path":"/a////b","operator":"equals","value":true}`, + facts: `{"a":{"":{"":{"":{"b":true}}}}}`, + want: 18, + why: "one node, 8 to compile the seven-byte path, and 7 to resolve it — five reference tokens, three of them empty, is five map traversals over two bytes of token — plus one unit for the boolean selected and one for the operand; " + + "charging a flat step would have priced those five traversals at three units", + }, + { + name: "fact-charges-the-selected-value-as-well-as-the-operand", + condition: `{"op":"fact","path":"/value","operator":"equals","value":"gold"}`, + facts: `{"value":"platinum"}`, + want: 28, + why: "one node, 13 for \"/value\", 9 for the eight-character string the pointer selected, and 5 for the four-character operand; the runtime side is the larger one here and is priced like the authored side", + }, + { + name: "fact-whose-pointer-does-not-resolve-charges-no-selected-value", + condition: `{"op":"fact","path":"/value","operator":"equals","value":"gold"}`, + facts: `{"other":"platinum"}`, + want: 19, + why: "one node, 13 for the lookup that failed, and 5 for the operand; nothing was selected, so nothing is charged for it", + }, + { + name: "deep-equality-charged-by-both-sides", + condition: `{"op":"fact","path":"/value","operator":"equals","value":{"a":[1,2,3]}}`, + facts: `{"value":{"a":[1,2,3]}}`, + want: 32, + why: "one node, 13 for \"/value\", and 9 twice — for the selected value and for the operand — where 9 is the object (1) plus its member name (1) plus the array (1) plus three two-unit number tokens", + }, + { + name: "in-charges-the-selected-value-once-per-candidate", + condition: `{"op":"fact","path":"/value","operator":"in","value":["a","b","c","d"]}`, + facts: `{"value":"a"}`, + want: 31, + why: "one node, 13 for \"/value\", 9 for the operand (the array plus four one-character strings at two units each), and the two-unit selected value once per candidate: 8", + }, + { + name: "evidence-present-charges-its-requirement-id", + condition: `{"op":"evidence-present","evidenceRequirement":"intake-form"}`, + facts: `{}`, + want: 12, + why: "one node plus the eleven bytes of the id every lookup hashes", + }, + { + name: "unresolved-aggregate-path-costs-its-lookup", + condition: `{"op":"exists","path":"/missing","where":` + elementPredicate + `}`, + facts: `{"list":[]}`, + want: 18, + why: "the node and the compile-and-resolve of the pointer that failed to resolve (17), and nothing more", + }, + { + name: "non-array-aggregate-path-costs-its-lookup", + condition: `{"op":"exists","path":"/value","where":` + elementPredicate + `}`, + facts: `{"value":"not an array"}`, + want: 14, + why: "the node and \"/value\" (13), with no elements to charge for", + }, + { + name: "empty-array-costs-only-the-aggregate", + condition: `{"op":"every","path":"/list","where":` + elementPredicate + `}`, + facts: `{"list":[]}`, + want: 12, + why: "the node and \"/list\" (11); the vacuous value costs no predicate", + }, + { + name: "per-element-predicate", + condition: `{"op":"exists","path":"/list","where":` + elementPredicate + `}`, + facts: `{"list":[{"ok":true},{"ok":false},{}]}`, + want: 33, + why: "12 for the aggregate, 10 for the first element, 6 for the second, and 5 for the third, whose \"/ok\" selects nothing: \"/ok\" is scanned once and stepped three times", + }, + { + name: "boolean-subtree-charges-branches-never-reached", + condition: `{"op":"any","conditions":[{"op":"literal","value":true},{"op":"exists","path":"/list","where":` + elementPredicate + `}]}`, + facts: `{"list":[{"ok":true},{"ok":false},{}]}`, + want: 35, + why: "the any node, the literal that short-circuits it, and the whole 33-unit aggregate the evaluator never reaches", + }, + { + name: "sibling-aggregates-add", + condition: `{"op":"all","conditions":[` + + `{"op":"exists","path":"/list","where":` + elementPredicate + `},` + + `{"op":"every","path":"/list","where":` + elementPredicate + `}]}`, + facts: `{"list":[{"ok":true},{"ok":false},{}]}`, + want: 57, + why: "the all node, the first aggregate at 33, and the second at 23 because both its pointers are already compiled; no single product bounds them", + }, + { + name: "ragged-nesting-sums-inner-lengths", + condition: `{"op":"exists","path":"/rows","where":{"op":"every","path":"/cells","where":` + elementPredicate + `}}`, + facts: `{"rows":[{"cells":[{"ok":true}]},{"cells":[{"ok":true},{"ok":true}]},{"cells":[{"ok":true},{"ok":true},{"ok":true}]}]}`, + want: 80, + why: "12 for the outer aggregate, 14 then 7 then 7 for the three inner ones, 10 for the first cell, and 6 for each of the other five", + }, + { + name: "uniform-charges-members-and-selected-values", + condition: `{"op":"uniform","path":"/list","at":"/fare"}`, + facts: `{"list":[{"fare":{"code":"Y"}},{"fare":{"code":"Y"}},{}]}`, + want: 54, + why: "12 for the aggregate, 11 then 5 then 5 for the three resolutions of \"/fare\", 7 for each of the two objects that resolved (the object, its four-byte member name, and the two-unit string), " + + "and a reread term of 7: two values resolved, so the representative is compared once, and that comparison can read all 7 units of the larger of them — the third member resolves nothing and joins neither term", + }, + { + name: "uniform-charges-a-reread-of-the-largest-value-per-comparison", + condition: `{"op":"uniform","path":"/list","at":"/cabin"}`, + facts: `{"list":[{"cabin":1},{"cabin":2},{"cabin":3}]}`, + want: 47, + why: "12 for the aggregate, 13 then 6 then 6 for \"/cabin\", two units for each one-digit number, and (3-1)×2 = 4 for the two comparisons the representative takes part in", + }, + { + name: "uniform-charges-a-long-numeric-token-by-its-bytes-and-by-its-rereads", + condition: `{"op":"uniform","path":"/list","at":"/cabin"}`, + facts: `{"list":[{"cabin":1e999999999},{"cabin":2},{"cabin":3}]}`, + want: 77, + why: "the 47 above with the first value's eleven-character token costing 12 rather than 2 (ten extra units in the per-member term) and the reread term rising from (3-1)×2 to (3-1)×12 (twenty more): " + + "the long token is read once per member however the members are ordered, because the maximum is taken over the set and not over whichever member happens to be elected", + }, + } + + for _, testCase := range cases { + t.Run(testCase.name, func(t *testing.T) { + if got := chargeOf(t, testCase.condition, testCase.facts); got != testCase.want { + t.Fatalf("charge = %d, want %d (%s)", got, testCase.want, testCase.why) + } + }) + } +} + +// The adversarial pointer row. A depth-two aggregate over tens of thousands of +// elements whose inner path is a valid megabyte-long pointer used to cost two +// units per element and scan a megabyte per element anyway: 60,002 units +// charged against a 100,000-unit budget for tens of gigabytes of processing, +// which is the whole defect. The charge is now byte-sensitive and is reserved +// before the scan, so the budget refuses the input with the path's bytes still +// unread. +// +// Both halves are asserted, because they fail differently. Under a flat pointer +// charge the input is not refused at all, which the code assertion catches; the +// wall clock then catches a charge that is byte-sensitive but levied after the +// scan rather than before it. The clock bound is deliberately loose — this +// refusal is milliseconds' work and the unbounded version is tens of seconds. +func TestRFC0008LongUnresolvedPointerIsRefusedBeforeItIsScanned(t *testing.T) { + engine := newTestEngine(t) + // One byte under the carrier's per-string limit, so the pack still decodes + // and the refusal is the work limit rather than a resource limit on input. + innerPath := "/" + strings.Repeat("a", 1<<20-1) + condition := fmt.Sprintf( + `{"op":"exists","path":"/list","where":{"op":"exists","path":%q,"where":{"op":"fact","path":"/ok","operator":"equals","value":true}}}`, + innerPath) + const count = 60_000 + elements := make([]string, 0, count) + for index := 0; index < count; index++ { + elements = append(elements, `{}`) + } + facts := []byte(fmt.Sprintf(`{"list":[%s]}`, strings.Join(elements, ","))) + + start := time.Now() + _, failure := engine.EvaluateWith(draftPack(condition, "escalate"), facts, nil, draftOptions(0)) + elapsed := time.Since(start) + + if failure == nil || failure.Code != "JPS-RESOURCE-EVALUATION-WORK-LIMIT" { + t.Fatalf("a megabyte-long pointer over %d elements must be refused by the work limit: %+v", count, failure) + } + if elapsed > 10*time.Second { + t.Fatalf("the refusal took %s; the budget must stop this before the path is scanned, not after", elapsed) + } + + // The same shape with a pointer short enough to afford is not refused, so + // the row above turns on the path's length and not on the element count. + short := `{"op":"exists","path":"/list","where":{"op":"exists","path":"/a","where":{"op":"fact","path":"/ok","operator":"equals","value":true}}}` + if _, failure := engine.EvaluateWith(draftPack(short, "escalate"), []byte(`{"list":[{},{}]}`), nil, draftOptions(0)); failure != nil { + t.Fatalf("the same shape over a short pointer must still evaluate: %+v", failure) + } +} + +// The adversarial operand row. A long decimal operand is parsed once per +// element by RFC 0006's ordered comparison, so it costs its token length per +// element rather than one unit per element, and a token long enough to make +// that expensive is refused rather than run. +func TestRFC0008LongDecimalOperandIsChargedByItsLength(t *testing.T) { + const elements = 200 + items := make([]string, 0, elements) + for index := 0; index < elements; index++ { + items = append(items, `{"amount":"7"}`) + } + facts := fmt.Sprintf(`{"list":[%s]}`, strings.Join(items, ",")) + condition := func(operand string) string { + return fmt.Sprintf( + `{"op":"exists","path":"/list","where":{"op":"fact","path":"/amount","operator":"greater-than","value":%q}}`, + operand) + } + const longOperand = 1_000 + short := "5" + long := strings.Repeat("9", longOperand) + + shortCharge := chargeOf(t, condition(short), facts) + longCharge := chargeOf(t, condition(long), facts) + // Every element pays the operand's bytes, so the difference is the token + // lengths' difference times the element count and nothing else. + if want := elements * (len(long) - len(short)); longCharge-shortCharge != want { + t.Fatalf("charges %d and %d differ by %d; a per-element operand must cost its length per element (%d)", + shortCharge, longCharge, longCharge-shortCharge, want) + } + + // And the difference is load-bearing: the short operand fits the default + // budget and the long one does not, so the 200,000 decimal parses the long + // one would force are refused rather than performed. + engine := newTestEngine(t) + if _, failure := engine.EvaluateWith(draftPack(condition(short), "escalate"), []byte(facts), nil, draftOptions(0)); failure != nil { + t.Fatalf("a one-digit operand over 200 elements must evaluate: %+v", failure) + } + _, failure := engine.EvaluateWith(draftPack(condition(long), "escalate"), []byte(facts), nil, draftOptions(0)) + if failure == nil || failure.Code != "JPS-RESOURCE-EVALUATION-WORK-LIMIT" { + t.Fatalf("a thousand-digit operand over the same 200 elements must be refused: %+v", failure) + } +} + +// The adversarial selected-value row, and the second defect of the same class +// as the long pointer. An authored operand says nothing about the value it is +// compared against: a two-byte operand can be compared, once per element, +// against as much runtime JSON as the carrier admits, and a model that charged +// only what the author wrote priced the one half of the comparison an attacker +// does not supply. Both shapes below are charged by their bytes, per element, +// and a facts document whose selected values are long enough to be expensive is +// refused under the same budget that admits the short one. +func TestRFC0008LongSelectedValueIsChargedByItsLength(t *testing.T) { + engine := newTestEngine(t) + const elements = 200 + const longWidth = 1_000 + shapes := []struct { + name string + element func(width int) string + operand string + }{ + {"string", func(width int) string { return fmt.Sprintf(`{"blob":%q}`, strings.Repeat("a", width)) }, `"a"`}, + {"number", func(width int) string { return fmt.Sprintf(`{"blob":%s}`, strings.Repeat("9", width)) }, `9`}, + } + for _, shape := range shapes { + t.Run(shape.name, func(t *testing.T) { + facts := func(width int) string { + items := make([]string, 0, elements) + for index := 0; index < elements; index++ { + items = append(items, shape.element(width)) + } + return fmt.Sprintf(`{"list":[%s]}`, strings.Join(items, ",")) + } + condition := fmt.Sprintf( + `{"op":"exists","path":"/list","where":{"op":"fact","path":"/blob","operator":"equals","value":%s}}`, + shape.operand) + short, long := facts(1), facts(longWidth) + + // The operand is one byte wide in both runs, so the difference is + // the selected values' difference and nothing else. + shortCharge := chargeOf(t, condition, short) + longCharge := chargeOf(t, condition, long) + if want := elements * (longWidth - 1); longCharge-shortCharge != want { + t.Fatalf("charges %d and %d differ by %d; a selected value must cost its length per element (%d)", + shortCharge, longCharge, longCharge-shortCharge, want) + } + + // And the difference decides admission: the 200 one-byte + // comparisons run, the 200 thousand-byte ones are refused. + pack := draftPack(condition, "escalate") + if _, failure := engine.EvaluateWith(pack, []byte(short), nil, draftOptions(0)); failure != nil { + t.Fatalf("one-byte selected values over 200 elements must evaluate: %+v", failure) + } + _, failure := engine.EvaluateWith(pack, []byte(long), nil, draftOptions(0)) + if failure == nil || failure.Code != "JPS-RESOURCE-EVALUATION-WORK-LIMIT" { + t.Fatalf("thousand-byte selected values over the same 200 elements must be refused: %+v", failure) + } + }) + } +} + +// The adversarial evidence-requirement row. evidence-present was charged as one +// flat node however long the identifier it looks up, so a where could hash a +// long id once per element for one unit per element — the flat-unit defect +// again, in the one node shape that had kept it. The id's bytes are now charged +// like any other name a lookup reads. +func TestRFC0008LongEvidenceRequirementIdIsChargedByItsLength(t *testing.T) { + engine := newTestEngine(t) + const elements = 200 + items := make([]string, 0, elements) + for index := 0; index < elements; index++ { + items = append(items, `{}`) + } + facts := fmt.Sprintf(`{"list":[%s]}`, strings.Join(items, ",")) + condition := func(id string) string { + return fmt.Sprintf( + `{"op":"exists","path":"/list","where":{"op":"evidence-present","evidenceRequirement":%q}}`, id) + } + // Both are localIds the schema admits; only their lengths differ. + short, long := "e", strings.Repeat("e", 1_000) + + shortCharge := chargeOf(t, condition(short), facts) + longCharge := chargeOf(t, condition(long), facts) + if want := elements * (len(long) - len(short)); longCharge-shortCharge != want { + t.Fatalf("charges %d and %d differ by %d; a requirement id looked up per element must cost its length per element (%d)", + shortCharge, longCharge, longCharge-shortCharge, want) + } + + // The pack declares the requirement, so both runs are conformant packs that + // differ only in the identifier's length — and the long one is refused. + pack := func(id string) []byte { + return []byte(fmt.Sprintf(`{ + "specVersion": "0.1.0-draft", + "id": "https://example.invalid/judgment-packs/rfc0008-evidence-id", + "version": "0.1.0", + "title": "Synthetic draft RFC 0008 evidence-identifier row", + "description": "Invented content for specification testing; it authorizes nothing.", + "decision": { + "intent": "Exercise the work charge of an evidence-present inside a where.", + "question": "Was the identifier charged by its length?" + }, + "outcomes": [ + {"id": "held", "label": "Held"}, + {"id": "did-not-hold", "label": "Did not hold"} + ], + "evidenceRequirements": [ + {"id": %q, "description": "Declared so the reference resolves.", "required": false} + ], + "rules": [ + { + "id": "the-rule", + "description": "Looks the requirement up once per element.", + "when": %s, + "outcome": "held", + "onUnknown": "ignore" + } + ], + "fallbackOutcome": "did-not-hold" +}`, id, condition(id))) + } + if _, failure := engine.EvaluateWith(pack(short), []byte(facts), nil, draftOptions(0)); failure != nil { + t.Fatalf("a one-character requirement id over 200 elements must evaluate: %+v", failure) + } + _, failure := engine.EvaluateWith(pack(long), []byte(facts), nil, draftOptions(0)) + if failure == nil || failure.Code != "JPS-RESOURCE-EVALUATION-WORK-LIMIT" { + t.Fatalf("a thousand-character requirement id over the same 200 elements must be refused: %+v", failure) + } +} + +// The adversarial empty-token pointer row. A resolution loops once per +// reference token and a reference token may be empty, so "////…" performs one +// map traversal per token over no token bytes at all: a charge of one flat step +// plus the token bytes priced a hundred traversals at one unit — on every +// resolution, and a where resolves once per element. The step count is now the +// token count, so the charge is what the loop does. +func TestRFC0008EmptyReferenceTokensAreChargedPerStep(t *testing.T) { + const elements = 1_000 + items := make([]string, 0, elements) + for index := 0; index < elements; index++ { + items = append(items, `{}`) + } + facts := fmt.Sprintf(`{"list":[%s]}`, strings.Join(items, ",")) + condition := func(path string) string { + return fmt.Sprintf( + `{"op":"exists","path":"/list","where":{"op":"fact","path":%q,"operator":"equals","value":true}}`, path) + } + // A hundred empty reference tokens against one, both carrying zero token + // bytes: the pointers differ in traversals and in nothing else. + const tokens = 100 + one, hundred := "/", strings.Repeat("/", tokens) + + oneCharge := chargeOf(t, condition(one), facts) + hundredCharge := chargeOf(t, condition(hundred), facts) + // 99 further steps on each of the 1,000 resolutions, and 99 further path + // bytes on the single compile: 3,014 units against 102,113. + if want := (tokens - 1) * (elements + 1); hundredCharge-oneCharge != want { + t.Fatalf("charges %d and %d differ by %d; each further reference token must cost a step per resolution (%d)", + oneCharge, hundredCharge, hundredCharge-oneCharge, want) + } + + // And the difference decides admission: the one-token pointer evaluates and + // the hundred-token one is refused rather than walked 100,000 times. + engine := newTestEngine(t) + if _, failure := engine.EvaluateWith(draftPack(condition(one), "escalate"), []byte(facts), nil, draftOptions(0)); failure != nil { + t.Fatalf("a one-token pointer over %d elements must evaluate: %+v", elements, failure) + } + _, failure := engine.EvaluateWith(draftPack(condition(hundred), "escalate"), []byte(facts), nil, draftOptions(0)) + if failure == nil || failure.Code != "JPS-RESOURCE-EVALUATION-WORK-LIMIT" { + t.Fatalf("a hundred-token pointer over the same %d elements must be refused: %+v", elements, failure) + } +} + +// The adversarial uniform row, and the fourth defect of the class the long +// pointer, the unpriced selected value, and the flat pointer step belong to. +// uniform elects one representative and compares it against every other member, +// so the representative is read once per member: one 80,001-byte token among +// 999 short tokens of the same value is work proportional to long×members, and +// the per-member charge alone came to long+members — 94,013 units against the +// default 100,000, admitted, with one permutation taking milliseconds and the +// other hundreds of times longer. The reread term prices those comparisons at +// (n-1)×max, so the input is refused, in either order and with the same error. +func TestRFC0008UniformRepeatedRepresentativeComparisonIsCharged(t *testing.T) { + engine := newTestEngine(t) + const condition = `{"op":"uniform","path":"/list","at":"/cabin"}` + pack := draftPack(condition, "escalate") + // The long token and the short one denote the same value, so no clause-3 + // counterexample can cut the pass short: every member is compared, and + // every comparison is against the long representative in one order or the + // short-then-long representative in the other. + facts := func(long, short string, copies int, longFirst bool) string { + items := make([]string, 0, copies+1) + for index := 0; index < copies; index++ { + items = append(items, fmt.Sprintf(`{"cabin":%s}`, short)) + } + if longFirst { + items = append([]string{fmt.Sprintf(`{"cabin":%s}`, long)}, items...) + } else { + items = append(items, fmt.Sprintf(`{"cabin":%s}`, long)) + } + return fmt.Sprintf(`{"list":[%s]}`, strings.Join(items, ",")) + } + + // 10^80000 written out in full, against 999 copies of 1e80000. + huge := "1" + strings.Repeat("0", 80_000) + first := facts(huge, "1e80000", 999, true) + last := facts(huge, "1e80000", 999, false) + if firstCharge, lastCharge := chargeOf(t, condition, first), chargeOf(t, condition, last); firstCharge != lastCharge { + t.Fatalf("the two permutations charge %d and %d; the charge must not depend on the order", firstCharge, lastCharge) + } + _, firstFailure := engine.EvaluateWith(pack, []byte(first), nil, draftOptions(0)) + _, lastFailure := engine.EvaluateWith(pack, []byte(last), nil, draftOptions(0)) + if firstFailure == nil || lastFailure == nil { + t.Fatalf("both permutations must be refused: %+v / %+v", firstFailure, lastFailure) + } + if firstFailure.Code != "JPS-RESOURCE-EVALUATION-WORK-LIMIT" { + t.Fatalf("exhaustion must be the work limit: %+v", firstFailure) + } + if !reflect.DeepEqual(firstFailure, lastFailure) { + t.Fatalf("the two permutations must produce the same error:\n first=%+v\n last=%+v", firstFailure, lastFailure) + } + + // The same shape small enough to afford: identical charges and identical + // dispositions in both orders, so the refusal above turns on the size and + // not on the permutation. 12 for the aggregate, 13 then nine 6s for + // "/cabin", 302 for the 301-byte token and 6 for each of the nine + // five-byte ones, and a reread term of (10-1)×302. + small := "1" + strings.Repeat("0", 300) + smallFirst := facts(small, "1e300", 9, true) + smallLast := facts(small, "1e300", 9, false) + firstCharge, lastCharge := chargeOf(t, condition, smallFirst), chargeOf(t, condition, smallLast) + if want := 12 + 13 + 9*6 + 302 + 9*6 + 9*302; firstCharge != want || lastCharge != want { + t.Fatalf("charges %d and %d, want %d in both orders", firstCharge, lastCharge, want) + } + firstOutput, firstFailure := engine.EvaluateWith(pack, []byte(smallFirst), nil, draftOptions(0)) + lastOutput, lastFailure := engine.EvaluateWith(pack, []byte(smallLast), nil, draftOptions(0)) + if firstFailure != nil || lastFailure != nil { + t.Fatalf("the affordable variant must evaluate in both orders: %+v / %+v", firstFailure, lastFailure) + } + if !reflect.DeepEqual(firstOutput.Disposition, lastOutput.Disposition) { + t.Fatalf("permuted members must produce one disposition: %+v / %+v", firstOutput.Disposition, lastOutput.Disposition) + } + // 10^300 and 1e300 are one value, so uniform is true and the rule fires. + if firstOutput.Disposition.Kind != "outcome" || firstOutput.Disposition.OutcomeID != "held" { + t.Fatalf("disposition = %+v; the members are equal, so uniform holds", firstOutput.Disposition) + } +} + +// A ragged outer array costs the sum of its inner lengths, never the product of +// the outer length with any single inner length — the row the RFC states as +// Σᵢ|Bᵢ| rather than |A|×|B|. +func TestRFC0008RaggedNestingIsNotAProduct(t *testing.T) { + const condition = `{"op":"exists","path":"/rows","where":{"op":"every","path":"/cells","where":{"op":"fact","path":"/ok","operator":"equals","value":true}}}` + cell := `{"ok":true}` + row := func(cells int) string { + items := make([]string, 0, cells) + for index := 0; index < cells; index++ { + items = append(items, cell) + } + return fmt.Sprintf(`{"cells":[%s]}`, strings.Join(items, ",")) + } + ragged := fmt.Sprintf(`{"rows":[%s,%s,%s]}`, row(1), row(2), row(3)) + rectangular := fmt.Sprintf(`{"rows":[%s,%s,%s]}`, row(3), row(3), row(3)) + + raggedCharge := chargeOf(t, condition, ragged) + rectangularCharge := chargeOf(t, condition, rectangular) + // Σ|Bᵢ| = 6 cells against |A|×max|B| = 9: the ragged charge must be the + // sum, which is exactly three cells cheaper. A cell after the first costs + // six units — its fact node, one step of the already-compiled "/ok", the + // boolean it selects, and the boolean operand — so the difference is 18. + if rectangularCharge-raggedCharge != 18 { + t.Fatalf("ragged %d and rectangular %d must differ by the three absent cells (18 units)", raggedCharge, rectangularCharge) + } +} diff --git a/internal/evaluation/resolve.go b/internal/evaluation/resolve.go index 943bf4b..0b89b31 100644 --- a/internal/evaluation/resolve.go +++ b/internal/evaluation/resolve.go @@ -19,34 +19,41 @@ const ( // resolver walks §8 over one conformant pack. It is a pure function of its // inputs: the decoded pack, the decoded facts document, and the tri-state -// presence of every declared evidence requirement. +// presence of every declared evidence requirement, which the evaluator carries +// along with the running work charge of the draft RFC 0008 opt-in. type resolver struct { - pack map[string]any - facts any - evidence map[string]tri + pack map[string]any + facts any + eval *evaluator reasons map[string]bool directEscalation bool trace []result.TraceEntry } -// resolve produces the RFC 0006 disposition and the informative trace. -func resolve(pack map[string]any, facts any, evidence map[string]tri) (result.Disposition, []result.TraceEntry) { - r := &resolver{pack: pack, facts: facts, evidence: evidence, reasons: map[string]bool{}} +// resolve produces the RFC 0006 disposition and the informative trace. The +// third result is an evaluation error rather than a disposition: exhausting the +// draft RFC 0008 work budget interrupts §8 wherever it happens, and no partial +// disposition is reported. +func resolve(pack map[string]any, facts any, eval *evaluator) (result.Disposition, []result.TraceEntry, *Failure) { + r := &resolver{pack: pack, facts: facts, eval: eval, reasons: map[string]bool{}} // Step 1: omitted applicability is the literal value true. False is a // terminal not-applicable result; unknown produces unresolved and stops. applicability := triTrue if condition, present := pack["applicability"]; present { - applicability = evalCondition(condition, facts, evidence) + applicability = r.eval.evaluate(condition, facts) + if r.eval.exceeded { + return result.Disposition{}, r.trace, workLimitFailure(r.eval) + } } switch applicability { case triFalse: r.reasons[reasonNotApplicable] = true - return r.disposition("not-applicable", ""), r.trace + return r.disposition("not-applicable", ""), r.trace, nil case triUnknown: r.reasons[reasonUnknown] = true - return r.disposition("unresolved", ""), r.trace + return r.disposition("unresolved", ""), r.trace, nil } // Step 2, as pinned by RFC 0006: missing-required-evidence iff any @@ -62,7 +69,7 @@ func resolve(pack map[string]any, facts any, evidence map[string]tri) (result.Di continue } id, _ := requirement["id"].(string) - switch r.evidence[id] { + switch r.eval.evidence[id] { case triFalse: requiredFalse = true case triUnknown: @@ -87,7 +94,10 @@ func resolve(pack map[string]any, facts any, evidence map[string]tri) (result.Di } id, _ := exception["id"].(string) effect, _ := exception["effect"].(string) - verdict := evalCondition(exception["when"], facts, evidence) + verdict := r.eval.evaluate(exception["when"], facts) + if r.eval.exceeded { + return result.Disposition{}, r.trace, workLimitFailure(r.eval) + } entry := result.TraceEntry{Stage: "exception", ID: id, Condition: verdict.String()} switch verdict { case triUnknown: @@ -124,7 +134,7 @@ func resolve(pack map[string]any, facts any, evidence map[string]tri) (result.Di r.reasons[reasonConflict] = true } if len(r.reasons) > 0 { - return r.disposition("unresolved", ""), r.trace + return r.disposition("unresolved", ""), r.trace, nil } // Step 6: one compatible forced outcome, no blocking state: produce it @@ -137,7 +147,7 @@ func resolve(pack map[string]any, facts any, evidence map[string]tri) (result.Di r.trace = append(r.trace, result.TraceEntry{Stage: "rule", ID: id, Condition: "not-evaluated", Skipped: true}) } } - return r.disposition("outcome", outcome), r.trace + return r.disposition("outcome", outcome), r.trace, nil } } @@ -155,7 +165,10 @@ func resolve(pack map[string]any, facts any, evidence map[string]tri) (result.Di r.trace = append(r.trace, result.TraceEntry{Stage: "rule", ID: id, Condition: "not-evaluated", Suppressed: true}) continue } - verdict := evalCondition(rule["when"], facts, evidence) + verdict := r.eval.evaluate(rule["when"], facts) + if r.eval.exceeded { + return result.Disposition{}, r.trace, workLimitFailure(r.eval) + } entry := result.TraceEntry{Stage: "rule", ID: id, Condition: verdict.String()} switch verdict { case triTrue: @@ -176,23 +189,23 @@ func resolve(pack map[string]any, facts any, evidence map[string]tri) (result.Di r.reasons[reasonConflict] = true } if len(r.reasons) > 0 { - return r.disposition("unresolved", ""), r.trace + return r.disposition("unresolved", ""), r.trace, nil } // Step 9: one distinct outcome and no blocking reason: produce it. if len(candidates) == 1 { for outcome := range candidates { - return r.disposition("outcome", outcome), r.trace + return r.disposition("outcome", outcome), r.trace, nil } } // Step 10: no true rule contributed an outcome. Use the fallback when // declared; otherwise unresolved with reason no-match. if fallback, ok := r.pack["fallbackOutcome"].(string); ok { - return r.disposition("outcome", fallback), r.trace + return r.disposition("outcome", fallback), r.trace, nil } r.reasons[reasonNoMatch] = true - return r.disposition("unresolved", ""), r.trace + return r.disposition("unresolved", ""), r.trace, nil } // disposition assembles the RFC 0006 result: sorted deduplicated reasons and diff --git a/internal/evaluation/rfc0008.go b/internal/evaluation/rfc0008.go new file mode 100644 index 0000000..c566cd0 --- /dev/null +++ b/internal/evaluation/rfc0008.go @@ -0,0 +1,429 @@ +package evaluation + +import ( + "encoding/json" + "fmt" + "regexp" + "sort" + + "github.com/Judgment-Pack/judgment-pack-runtime/internal/carrier" + "github.com/Judgment-Pack/judgment-pack-runtime/internal/result" + "github.com/Judgment-Pack/judgment-pack-runtime/internal/validation" +) + +// The draft RFC 0008 grammar gate. The RFC's operators belong to no published +// JPS version, so no bundled schema describes them and spec validate rejects +// every pack that uses one — which is exactly the behavior this file preserves. +// Under the opt-in, a pack is admitted in two independent steps with a division +// of labour worth stating exactly, because neither step checks the whole +// condition. The Go gate here owns what the draft grammar adds and nothing +// else: aggregate shape and the aggregate-depth bound, checked against the same +// depth-indexed construction the testdata schema artifact states declaratively +// for those branches. The Core validity of everything inside a where — its op, +// its operand shapes, its evidence references — is delegated to the untouched +// 0.1.0-draft validator over the pack's Core projection, which is why a where +// carrying an op §7 does not define reaches evaluation through neither step but +// is still refused. + +// rfc0008Conformance admits a pack that may use the draft operators. The pack's +// real bytes still pass the carrier layer, the draft grammar is checked against +// the RFC's shape and depth bound, and the pack's Core projection is then put +// through the untouched validator, so everything the draft grammar does not add +// is held to full document conformance. What is not claimed, and what the +// output marker says out loud, is that the pack is valid: it is not, under any +// published JPS version, and spec validate rejects it. +func (e *Engine) rfc0008Conformance(pack []byte, options Options) (result.Validation, map[string]any, *Failure) { + carrierOnly, operational := e.validator.Validate(pack, validation.Options{ + Through: "carrier", + Limits: carrier.DefaultLimits(), + }) + if operational != nil { + return result.Validation{}, nil, &Failure{Code: operational.Code, Message: operational.Message, ExitCode: operational.ExitCode} + } + if carrierOnly.Status != "valid" { + return result.Validation{}, nil, packNotConformant(carrierOnly) + } + document, failure := decodeInput(pack, "pack") + if failure != nil { + return result.Validation{}, nil, failure + } + packRoot, isObject := document.(map[string]any) + if !isObject { + // The bundled validator already reports this precisely; borrow it rather + // than inventing a second message for the same document. + validated, operational := e.validator.Validate(pack, validation.Options{Through: "semantic", Limits: carrier.DefaultLimits()}) + if operational != nil { + return result.Validation{}, nil, &Failure{Code: operational.Code, Message: operational.Message, ExitCode: operational.ExitCode} + } + return result.Validation{}, nil, packNotConformant(validated) + } + if diagnostics := rfc0008Diagnostics(packRoot); len(diagnostics) > 0 { + return result.Validation{}, nil, draftGrammarFailure(diagnostics) + } + projected, err := json.Marshal(projectPack(packRoot)) + if err != nil { + return result.Validation{}, nil, &Failure{Code: "JPS-EVALUATION-INTERNAL", Message: "The pack's Core projection could not be re-encoded for validation.", ExitCode: result.ExitInternal} + } + validated, operational := e.validator.Validate(projected, validation.Options{ + Through: "semantic", + SupportedExtensions: options.SupportedExtensions, + Limits: carrier.DefaultLimits(), + }) + if operational != nil { + return result.Validation{}, nil, &Failure{Code: operational.Code, Message: operational.Message, ExitCode: operational.ExitCode} + } + if validated.Status != "valid" { + return result.Validation{}, nil, projectionNotConformant(validated) + } + return validated, packRoot, nil +} + +// projectionNotConformant refuses a pack whose Core projection did not reach +// full document conformance. It is packNotConformant's finding with two +// corrections the projection forces. The instance path names a location in the +// projection, not in the author's pack: every exists and every is replaced by +// its where and every uniform by a true literal, so a diagnostic inside a where +// is reported with the intervening /where segments elided. And spec validate +// cannot reproduce the finding for a pack using a draft operator — it rejects +// that pack at the aggregate itself — so the refusal does not send the author +// there for a report that will say something else. +func projectionNotConformant(validated result.Validation) *Failure { + exitCode := result.ExitInvalid + if validated.Status == "unsupported" { + exitCode = result.ExitUnsupported + } + detail := "" + if len(validated.Diagnostics) > 0 { + first := validated.Diagnostics[0] + location := first.InstancePath + if location == "" { + location = "" + } + detail = fmt.Sprintf(" First diagnostic: %s at %s: %s", first.Code, location, first.Message) + } + return &Failure{ + Code: "JPS-EVALUATION-PACK-NOT-CONFORMANT", + Message: fmt.Sprintf( + "The pack is not evaluated because the document conformance status of its Core projection is %q; evaluation requires a fully conformant pack.%s The instance path is relative to that projection, in which each exists and every is replaced by its where and each uniform by a true literal, so a location inside a where is named with the /where segments elided.", + validated.Status, detail), + ExitCode: exitCode, + } +} + +// draftGrammarFailure refuses a pack whose draft-operator use is malformed or +// too deep, naming the first diagnostic so the refusal is self-sufficient. A +// depth-three aggregate is refused here, before any condition is evaluated. +func draftGrammarFailure(diagnostics []result.Diagnostic) *Failure { + first := diagnostics[0] + location := first.InstancePath + if location == "" { + location = "" + } + return &Failure{ + Code: "JPS-EVALUATION-RFC0008-GRAMMAR", + Message: fmt.Sprintf( + "The pack is not evaluated because it does not satisfy the draft RFC 0008 grammar. First diagnostic: %s at %s: %s", + first.Code, location, first.Message), + ExitCode: result.ExitInvalid, + } +} + +// workLimitFailure is the explicit evaluation error RFC 0008 requires when the +// budget is exhausted. It is an error, never a disposition: the charge is +// complete before any element is evaluated, so an exhausted budget means no +// disposition was ever entitled to exist. +func workLimitFailure(e *evaluator) *Failure { + return &Failure{ + Code: "JPS-RESOURCE-EVALUATION-WORK-LIMIT", + Message: fmt.Sprintf( + "The evaluation exceeds this runtime's draft RFC 0008 evaluation-work limit of %d units. The charge is computed before any element is evaluated and does not depend on element order, so no disposition is produced.", + e.budget), + ExitCode: result.ExitIO, + } +} + +// draftPrototype builds the in-band marker every draft-grammar evaluation +// carries. It states the one thing a reader must not miss: this pack is not +// valid under the published specification, and the operators it uses are a +// prototype of an open proposal that may never be accepted. +func draftPrototype(packRoot map[string]any, specVersion string) *result.DraftPrototype { + operators := rfc0008Operators(packRoot) + note := fmt.Sprintf( + "Draft RFC 0008 (bounded collection quantifiers) prototype. The operators are a draft-RFC prototype, not part of JPS %s; a pack using one is NOT valid under it and spec validate rejects it. Nothing here claims conformance of any kind.", + specVersion) + if len(operators) == 0 { + note = fmt.Sprintf( + "Draft RFC 0008 (bounded collection quantifiers) prototype was enabled, but this pack uses no draft operator, so it remains a plain JPS %s pack.", + specVersion) + } + return &result.DraftPrototype{ + RFC: "0008", + Status: "draft-rfc-prototype", + Operators: operators, + PackValidUnderSpecVersion: len(operators) == 0, + Note: note, + } +} + +// aggregateOps are RFC 0008's aggregates. uniform counts as one because it +// traverses a collection exactly as the quantifiers do. +var aggregateOps = map[string]bool{"exists": true, "every": true, "uniform": true} + +// maxAggregateDepth is RFC 0008's bound: a top-level aggregate may contain +// aggregates in its where, and those may contain none. Depth is structural, so +// all/any/not wrappers inside a where do not launder it. +const maxAggregateDepth = 2 + +// pointerPattern is the schema's fact.path grammar, reused unchanged for the +// draft operators' path and at members. +var pointerPattern = regexp.MustCompile(`^(?:/(?:[^~/]|~0|~1)*)*$`) + +// rfc0008Diagnostics reports the draft grammar's structural errors over one +// decoded pack: an aggregate whose shape does not match the RFC, and an +// aggregate past the maximum depth. Depth is counted from each condition root, +// which is where a condition tree begins in a pack. +func rfc0008Diagnostics(root map[string]any) []result.Diagnostic { + diagnostics := []result.Diagnostic{} + for _, site := range conditionSites(root) { + collectAggregateDiagnostics(site.node, site.location, maxAggregateDepth, &diagnostics) + } + return diagnostics +} + +// conditionSite is one authored condition tree and its location in the pack. +type conditionSite struct { + node any + location []string +} + +// conditionSites enumerates every position §7 allows a condition: the pack's +// applicability, each rule's when, and each exception's when. +func conditionSites(root map[string]any) []conditionSite { + sites := []conditionSite{} + if condition, present := root["applicability"]; present { + sites = append(sites, conditionSite{node: condition, location: []string{"applicability"}}) + } + for _, collection := range []string{"rules", "exceptions"} { + for index, entry := range asArray(root[collection]) { + object, ok := entry.(map[string]any) + if !ok { + continue + } + if condition, present := object["when"]; present { + sites = append(sites, conditionSite{node: condition, location: []string{collection, fmt.Sprint(index), "when"}}) + } + } + } + return sites +} + +// collectAggregateDiagnostics walks one condition tree with the remaining +// aggregate depth in hand. all, any, and not recurse within their own tier; +// an aggregate consumes one tier and its where is checked in the next one, so +// the tier a node is checked in does not depend on how it was wrapped. +func collectAggregateDiagnostics(node any, location []string, remaining int, diagnostics *[]result.Diagnostic) { + condition, ok := node.(map[string]any) + if !ok { + return + } + op, _ := condition["op"].(string) + switch { + case op == "all" || op == "any": + for index, child := range asArray(condition["conditions"]) { + collectAggregateDiagnostics(child, appendPath(location, "conditions", fmt.Sprint(index)), remaining, diagnostics) + } + case op == "not": + collectAggregateDiagnostics(condition["condition"], appendPath(location, "condition"), remaining, diagnostics) + case aggregateOps[op]: + if remaining < 1 { + *diagnostics = append(*diagnostics, result.ErrorDiagnostic( + "JPS-EVALUATION-RFC0008-DEPTH", + "structural", + carrier.Pointer(location), + fmt.Sprintf("Draft RFC 0008 allows a maximum aggregate depth of %d; this %q is deeper. Wrapping an aggregate in all, any, or not does not reduce its depth.", maxAggregateDepth, op), + )) + return + } + collectShapeDiagnostics(condition, op, location, diagnostics) + if op == "uniform" { + return + } + collectAggregateDiagnostics(condition["where"], appendPath(location, "where"), remaining-1, diagnostics) + } +} + +// collectShapeDiagnostics checks one aggregate against the RFC's shape: the +// required members, no others, and RFC 6901 pointers in path (and in at, for +// uniform, whose empty value is admitted and selects the whole member). +func collectShapeDiagnostics(condition map[string]any, op string, location []string, diagnostics *[]result.Diagnostic) { + required := []string{"op", "path", "where"} + if op == "uniform" { + required = []string{"op", "path", "at"} + } + allowed := map[string]bool{} + for _, member := range required { + allowed[member] = true + } + shape := func(parts []string, message string) { + *diagnostics = append(*diagnostics, result.ErrorDiagnostic( + "JPS-EVALUATION-RFC0008-SHAPE", "structural", carrier.Pointer(parts), message)) + } + for _, member := range required { + if _, present := condition[member]; !present { + shape(appendPath(location, member), fmt.Sprintf("Draft RFC 0008 requires the %q member on a %q condition.", member, op)) + } + } + for _, member := range sortedMembers(condition) { + if !allowed[member] { + shape(appendPath(location, member), fmt.Sprintf("Member is not allowed on a draft RFC 0008 %q condition.", op)) + } + } + for _, member := range []string{"path", "at"} { + if !allowed[member] { + continue + } + value, present := condition[member] + if !present { + continue + } + text, isString := value.(string) + if !isString || !pointerPattern.MatchString(text) { + shape(appendPath(location, member), fmt.Sprintf("Draft RFC 0008 %q must be a JSON Pointer string.", member)) + } + } + if op != "uniform" { + if where, present := condition["where"]; present { + if _, isObject := where.(map[string]any); !isObject { + shape(appendPath(location, "where"), "Draft RFC 0008 where must be a condition object.") + } + } + } +} + +// rfc0008Operators lists, sorted, the draft operators a pack actually uses. An +// empty result means the pack is expressible in Core and the opt-in changed +// nothing about it. +func rfc0008Operators(root map[string]any) []string { + used := map[string]bool{} + for _, site := range conditionSites(root) { + collectOperators(site.node, used) + } + operators := make([]string, 0, len(used)) + for op := range used { + operators = append(operators, op) + } + sort.Strings(operators) + return operators +} + +func collectOperators(node any, used map[string]bool) { + condition, ok := node.(map[string]any) + if !ok { + return + } + op, _ := condition["op"].(string) + switch { + case op == "all" || op == "any": + for _, child := range asArray(condition["conditions"]) { + collectOperators(child, used) + } + case op == "not": + collectOperators(condition["condition"], used) + case aggregateOps[op]: + used[op] = true + collectOperators(condition["where"], used) + } +} + +// projectPack produces the pack's Core projection: the same document with every +// aggregate replaced by a condition 0.1.0-draft defines. A quantifier is +// replaced by its own where, so the inner condition is still checked — its +// operand shapes structurally and its evidence-present references semantically, +// which is RFC 0008's §3.3 amendment obtained for free rather than +// reimplemented. uniform has no where and is replaced by a true literal. +// +// The projection is a validation instrument only: it does not mean the same +// thing as the pack, and it is never evaluated. Evaluation always reads the +// original document, which is left untouched. +func projectPack(root map[string]any) map[string]any { + projected := copyObject(root) + if condition, present := root["applicability"]; present { + projected["applicability"] = projectCondition(condition) + } + for _, collection := range []string{"rules", "exceptions"} { + entries, present := root[collection].([]any) + if !present { + continue + } + copied := make([]any, 0, len(entries)) + for _, entry := range entries { + object, ok := entry.(map[string]any) + if !ok { + copied = append(copied, entry) + continue + } + item := copyObject(object) + if condition, present := object["when"]; present { + item["when"] = projectCondition(condition) + } + copied = append(copied, item) + } + projected[collection] = copied + } + return projected +} + +// projectCondition rewrites one condition tree into Core. +func projectCondition(node any) any { + condition, ok := node.(map[string]any) + if !ok { + return node + } + op, _ := condition["op"].(string) + switch { + case op == "all" || op == "any": + children, present := condition["conditions"].([]any) + if !present { + return condition + } + copied := make([]any, 0, len(children)) + for _, child := range children { + copied = append(copied, projectCondition(child)) + } + projected := copyObject(condition) + projected["conditions"] = copied + return projected + case op == "not": + projected := copyObject(condition) + projected["condition"] = projectCondition(condition["condition"]) + return projected + case op == "exists" || op == "every": + return projectCondition(condition["where"]) + case op == "uniform": + return map[string]any{"op": "literal", "value": true} + default: + return condition + } +} + +func copyObject(object map[string]any) map[string]any { + copied := make(map[string]any, len(object)) + for key, value := range object { + copied[key] = value + } + return copied +} + +func sortedMembers(object map[string]any) []string { + members := make([]string, 0, len(object)) + for member := range object { + members = append(members, member) + } + sort.Strings(members) + return members +} + +func appendPath(location []string, parts ...string) []string { + output := append([]string(nil), location...) + return append(output, parts...) +} diff --git a/internal/evaluation/rfc0008_test.go b/internal/evaluation/rfc0008_test.go new file mode 100644 index 0000000..e9f18f6 --- /dev/null +++ b/internal/evaluation/rfc0008_test.go @@ -0,0 +1,920 @@ +package evaluation + +import ( + "fmt" + "os" + "path/filepath" + "reflect" + "strings" + "testing" + + "github.com/Judgment-Pack/judgment-pack-runtime/internal/carrier" + "github.com/Judgment-Pack/judgment-pack-runtime/internal/result" + "github.com/Judgment-Pack/judgment-pack-runtime/internal/validation" +) + +// draftPack builds one minimal conformant-apart-from-the-draft-grammar pack +// whose single rule carries the given condition, so a pack-level row states +// only the thing it is about. +func draftPack(when, onUnknown string) []byte { + return []byte(fmt.Sprintf(`{ + "specVersion": "0.1.0-draft", + "id": "https://example.invalid/judgment-packs/rfc0008-row", + "version": "0.1.0", + "title": "Synthetic draft RFC 0008 row", + "description": "Invented content for specification testing; it authorizes nothing.", + "decision": { + "intent": "Exercise one draft RFC 0008 condition against one facts document.", + "question": "Does the condition hold?" + }, + "outcomes": [ + {"id": "held", "label": "Held"}, + {"id": "did-not-hold", "label": "Did not hold"} + ], + "rules": [ + { + "id": "the-rule", + "description": "The one rule under test.", + "when": %s, + "outcome": "held", + "onUnknown": %q + } + ], + "fallbackOutcome": "did-not-hold" +}`, when, onUnknown)) +} + +func draftOptions(budget int) Options { + return Options{Command: "test", RFC0008Quantifiers: true, WorkBudget: budget} +} + +// The depth-indexed schema construction, as a declarative artifact, and the +// runtime's own grammar check must agree over what the draft grammar adds: +// aggregate shape and aggregate depth. Every row below is one of those, and is +// run twice — once through the testdata JSON-Schema tiers and once through the +// Go check that gates evaluation. The agreement is scoped to that: the Go check +// never looks inside a where for Core validity, which the Core projection owns +// instead (TestRFC0008CoreProjectionStillValidatesInsideWhere), so a row whose +// where is Core-invalid but whose aggregates are well-shaped belongs there and +// not here. +func TestRFC0008DepthIndexedGrammar(t *testing.T) { + schemaBytes, err := os.ReadFile(filepath.Join("testdata", "rfc0008-condition-schema.json")) + if err != nil { + t.Fatal(err) + } + schema, err := validation.CompileSchema(schemaBytes, "urn:judgmentpack:rfc0008:condition:draft") + if err != nil { + t.Fatal(err) + } + + const predicate = `{"op":"fact","path":"/ok","operator":"equals","value":true}` + const depthOneExists = `{"op":"exists","path":"/cells","where":` + predicate + `}` + const depthOneUniform = `{"op":"uniform","path":"/cells","at":"/cabin"}` + + cases := []struct { + name string + condition string + valid bool + }{ + {"core-condition-unchanged", predicate, true}, + {"top-level-exists", `{"op":"exists","path":"/rows","where":` + predicate + `}`, true}, + {"top-level-uniform", `{"op":"uniform","path":"/rows","at":"/cabin"}`, true}, + {"depth-two-nesting-of-both-operators", + `{"op":"exists","path":"/rows","where":{"op":"every","path":"/cells","where":` + predicate + `}}`, true}, + {"sibling-aggregates-at-depth-two", + `{"op":"exists","path":"/rows","where":{"op":"all","conditions":[` + depthOneExists + `,` + depthOneExists + `]}}`, true}, + {"aggregate-reached-through-all-inside-a-where", + `{"op":"exists","path":"/rows","where":{"op":"all","conditions":[` + depthOneExists + `]}}`, true}, + {"aggregate-reached-through-not-inside-a-where", + `{"op":"exists","path":"/rows","where":{"op":"not","condition":` + depthOneExists + `}}`, true}, + {"uniform-at-depth-two", + `{"op":"exists","path":"/rows","where":` + depthOneUniform + `}`, true}, + {"uniform-at-depth-two-through-a-wrapper", + `{"op":"exists","path":"/rows","where":{"op":"any","conditions":[` + depthOneUniform + `]}}`, true}, + + {"depth-three-nesting", `{"op":"exists","path":"/rows","where":{"op":"exists","path":"/cells","where":` + depthOneExists + `}}`, false}, + {"all-wrapper-does-not-launder-depth", + `{"op":"exists","path":"/rows","where":{"op":"all","conditions":[{"op":"exists","path":"/cells","where":` + depthOneExists + `}]}}`, false}, + {"not-wrapper-does-not-launder-depth", + `{"op":"exists","path":"/rows","where":{"op":"not","condition":{"op":"exists","path":"/cells","where":` + depthOneExists + `}}}`, false}, + {"uniform-inside-an-inner-where-is-depth-three", + `{"op":"exists","path":"/rows","where":{"op":"every","path":"/cells","where":` + depthOneUniform + `}}`, false}, + + {"exists-missing-where", `{"op":"exists","path":"/rows"}`, false}, + {"exists-missing-path", `{"op":"exists","where":` + predicate + `}`, false}, + {"exists-with-an-extra-member", `{"op":"exists","path":"/rows","at":"/ok","where":` + predicate + `}`, false}, + {"exists-with-a-non-pointer-path", `{"op":"exists","path":"rows","where":` + predicate + `}`, false}, + {"uniform-missing-at", `{"op":"uniform","path":"/rows"}`, false}, + {"uniform-with-a-where", `{"op":"uniform","path":"/rows","at":"/cabin","where":` + predicate + `}`, false}, + {"uniform-with-a-non-pointer-at", `{"op":"uniform","path":"/rows","at":"cabin"}`, false}, + } + + for _, testCase := range cases { + t.Run(testCase.name, func(t *testing.T) { + condition := documentOf(t, testCase.condition) + schemaErr := schema.Validate(condition) + if testCase.valid && schemaErr != nil { + t.Fatalf("the depth-indexed schema rejected a valid condition: %v", schemaErr) + } + if !testCase.valid && schemaErr == nil { + t.Fatal("the depth-indexed schema accepted an invalid condition") + } + + root, ok := documentOf(t, string(draftPack(testCase.condition, "escalate"))).(map[string]any) + if !ok { + t.Fatal("the row's pack must decode to an object") + } + diagnostics := rfc0008Diagnostics(root) + if testCase.valid && len(diagnostics) > 0 { + t.Fatalf("the runtime grammar check rejected a valid condition: %+v", diagnostics) + } + if !testCase.valid && len(diagnostics) == 0 { + t.Fatal("the runtime grammar check accepted a condition the schema rejects") + } + }) + } +} + +// The RFC's structural conformance rows are corpus rows, and a depth-three +// aggregate is refused before any condition is evaluated, with the offending +// location named. +func TestRFC0008StructuralRefusals(t *testing.T) { + engine := newTestEngine(t) + const predicate = `{"op":"fact","path":"/ok","operator":"equals","value":true}` + + cases := []struct { + name string + condition string + wantCode string + wantPath string + }{ + { + name: "depth-three-through-an-all-wrapper", + condition: `{"op":"exists","path":"/rows","where":{"op":"all","conditions":[{"op":"exists","path":"/cells","where":{"op":"exists","path":"/deeper","where":` + predicate + `}}]}}`, + wantCode: "JPS-EVALUATION-RFC0008-DEPTH", + wantPath: "/rules/0/when/where/conditions/0/where", + }, + { + name: "depth-three-through-a-not-wrapper", + condition: `{"op":"exists","path":"/rows","where":{"op":"not","condition":{"op":"exists","path":"/cells","where":{"op":"uniform","path":"/deeper","at":"/cabin"}}}}`, + wantCode: "JPS-EVALUATION-RFC0008-DEPTH", + wantPath: "/rules/0/when/where/condition/where", + }, + { + name: "missing-where", + condition: `{"op":"exists","path":"/rows"}`, + wantCode: "JPS-EVALUATION-RFC0008-SHAPE", + wantPath: "/rules/0/when/where", + }, + { + name: "member-not-allowed", + condition: `{"op":"uniform","path":"/rows","at":"/cabin","extra":true}`, + wantCode: "JPS-EVALUATION-RFC0008-SHAPE", + wantPath: "/rules/0/when/extra", + }, + { + name: "path-is-not-a-pointer", + condition: `{"op":"every","path":"rows","where":` + predicate + `}`, + wantCode: "JPS-EVALUATION-RFC0008-SHAPE", + wantPath: "/rules/0/when/path", + }, + } + + for _, testCase := range cases { + t.Run(testCase.name, func(t *testing.T) { + _, failure := engine.EvaluateWith(draftPack(testCase.condition, "escalate"), []byte(`{}`), nil, draftOptions(0)) + if failure == nil { + t.Fatal("a structurally invalid draft condition must be refused, never evaluated") + } + if failure.Code != "JPS-EVALUATION-RFC0008-GRAMMAR" || failure.ExitCode != result.ExitInvalid { + t.Fatalf("failure = %+v", failure) + } + if !strings.Contains(failure.Message, testCase.wantCode) || !strings.Contains(failure.Message, testCase.wantPath) { + t.Fatalf("the refusal must name %s at %s: %q", testCase.wantCode, testCase.wantPath, failure.Message) + } + }) + } +} + +// Without the opt-in a pack using the draft operators fails exactly as it does +// today — the bundled schema's closed condition union rejects it — and with the +// opt-in the same pack evaluates, carrying a marker that says so. +func TestRFC0008OptInGate(t *testing.T) { + engine := newTestEngine(t) + pack := draftPack(`{"op":"exists","path":"/list","where":{"op":"fact","path":"/ok","operator":"equals","value":true}}`, "escalate") + facts := []byte(`{"list":[{"ok":true}]}`) + + _, failure := engine.Evaluate(pack, facts, nil, nil, "test") + if failure == nil || failure.Code != "JPS-EVALUATION-PACK-NOT-CONFORMANT" || failure.ExitCode != result.ExitInvalid { + t.Fatalf("without the opt-in the pack must be refused as non-conformant: %+v", failure) + } + if !strings.Contains(failure.Message, "JPS-STRUCTURE-CONDITION-SHAPE") { + t.Fatalf("the refusal must be the ordinary structural one: %q", failure.Message) + } + + output, failure := engine.EvaluateWith(pack, facts, nil, draftOptions(0)) + if failure != nil { + t.Fatalf("with the opt-in the pack must evaluate: %s", failure.Message) + } + if output.Disposition.Kind != "outcome" || output.Disposition.OutcomeID != "held" { + t.Fatalf("disposition = %+v", output.Disposition) + } + marker := output.DraftPrototype + if marker == nil { + t.Fatal("a draft-grammar evaluation must carry the prototype marker") + } + if marker.RFC != "0008" || marker.Status != "draft-rfc-prototype" { + t.Fatalf("marker = %+v", marker) + } + if !reflect.DeepEqual(marker.Operators, []string{"exists"}) { + t.Fatalf("marker must name the operators used: %+v", marker.Operators) + } + if marker.PackValidUnderSpecVersion { + t.Fatal("a pack using a draft operator is not valid under the published specification") + } + if !strings.Contains(marker.Note, "NOT valid") || !strings.Contains(marker.Note, output.SpecVersion) { + t.Fatalf("the marker note must say the pack is not valid under the named version: %q", marker.Note) + } + + // The opt-in changes nothing for a pack that uses no draft operator, and the + // marker says that too rather than implying an accusation. + core := draftPack(`{"op":"fact","path":"/ok","operator":"equals","value":true}`, "escalate") + output, failure = engine.EvaluateWith(core, []byte(`{"ok":true}`), nil, draftOptions(0)) + if failure != nil { + t.Fatalf("a Core pack must still evaluate under the opt-in: %s", failure.Message) + } + if output.DraftPrototype == nil || len(output.DraftPrototype.Operators) != 0 || !output.DraftPrototype.PackValidUnderSpecVersion { + t.Fatalf("marker = %+v", output.DraftPrototype) + } +} + +// Aggregates are structurally valid anywhere a condition is allowed, so the +// gate, the projection, and the evaluator must reach all three positions: the +// pack's applicability, an exception's when, and a rule's when. +func TestRFC0008AggregatesInEveryConditionPosition(t *testing.T) { + engine := newTestEngine(t) + pack := []byte(`{ + "specVersion": "0.1.0-draft", + "id": "https://example.invalid/judgment-packs/rfc0008-positions", + "version": "0.1.0", + "title": "Synthetic draft RFC 0008 condition positions", + "description": "Invented content for specification testing; it authorizes nothing.", + "decision": { + "intent": "Exercise a draft aggregate in every position a condition may appear.", + "question": "Which position decided?" + }, + "outcomes": [ + {"id": "held", "label": "Held"}, + {"id": "forced", "label": "Forced"} + ], + "applicability": {"op": "every", "path": "/segments", "where": {"op": "fact", "path": "/inScope", "operator": "equals", "value": true}}, + "rules": [ + { + "id": "the-rule", + "description": "Fires when any segment is flagged.", + "when": {"op": "exists", "path": "/segments", "where": {"op": "fact", "path": "/flagged", "operator": "equals", "value": true}}, + "outcome": "held", + "onUnknown": "escalate" + } + ], + "exceptions": [ + { + "id": "the-exception", + "description": "Forces an outcome when the cabin is uniform.", + "when": {"op": "uniform", "path": "/segments", "at": "/cabin"}, + "effect": "force-outcome", + "outcome": "forced", + "onUnknown": "ignore" + } + ], + "fallbackOutcome": "held" +}`) + + cases := []struct { + name string + facts string + wantKind string + wantOutcome string + }{ + { + name: "applicability-aggregate-decides", + facts: `{"segments":[{"inScope":true,"cabin":"economy"},{"inScope":false,"cabin":"economy"}]}`, + wantKind: "not-applicable", + }, + { + name: "exception-aggregate-forces", + facts: `{"segments":[{"inScope":true,"cabin":"economy","flagged":false},{"inScope":true,"cabin":"economy","flagged":false}]}`, + wantKind: "outcome", + wantOutcome: "forced", + }, + { + name: "rule-aggregate-decides", + facts: `{"segments":[{"inScope":true,"cabin":"economy","flagged":true},{"inScope":true,"cabin":"business","flagged":false}]}`, + wantKind: "outcome", + wantOutcome: "held", + }, + } + for _, testCase := range cases { + t.Run(testCase.name, func(t *testing.T) { + output, failure := engine.EvaluateWith(pack, []byte(testCase.facts), nil, draftOptions(0)) + if failure != nil { + t.Fatalf("evaluation failed: %s", failure.Message) + } + if output.Disposition.Kind != testCase.wantKind || output.Disposition.OutcomeID != testCase.wantOutcome { + t.Fatalf("disposition = %+v", output.Disposition) + } + if !reflect.DeepEqual(output.DraftPrototype.Operators, []string{"every", "exists", "uniform"}) { + t.Fatalf("the marker must name every draft operator used: %v", output.DraftPrototype.Operators) + } + }) + } +} + +// The Core projection is a validation instrument, not a hole: everything the +// draft grammar does not add is still held to full document conformance, +// including the conditions inside a where — a Core-invalid op among them, which +// the Go grammar gate does not check at all. RFC 0008's §3.3 amendment — the +// condition-walking semantic bullets must recurse into where — is what the +// evidence-present row pins. +// +// The reported instance path is the projected one, with the elided /where +// segments missing, and each row pins it rather than leaving the offset latent: +// the projection replaces each exists and every by its where, so a finding two +// levels inside a where surfaces at the aggregate's own location. +func TestRFC0008CoreProjectionStillValidatesInsideWhere(t *testing.T) { + engine := newTestEngine(t) + cases := []struct { + name string + condition string + wantCode string + wantPath string + original string + }{ + { + name: "evidence-present-inside-where-must-resolve", + condition: `{"op":"exists","path":"/list","where":{"op":"evidence-present","evidenceRequirement":"no-such-requirement"}}`, + wantCode: "JPS-SEMANTIC-UNRESOLVED-EVIDENCE", + wantPath: "/rules/0/when/evidenceRequirement", + original: "/rules/0/when/where/evidenceRequirement", + }, + { + name: "operand-shape-inside-where-is-structural", + condition: `{"op":"exists","path":"/list","where":{"op":"fact","path":"/ok","operator":"in","value":"not-an-array"}}`, + wantCode: "JPS-STRUCTURE-IN-OPERAND", + wantPath: "/rules/0/when/value", + original: "/rules/0/when/where/value", + }, + { + name: "ordered-operand-inside-a-nested-where", + condition: `{"op":"exists","path":"/rows","where":{"op":"every","path":"/cells","where":{"op":"fact","path":"/amount","operator":"greater-than","value":5}}}`, + wantCode: "JPS-STRUCTURE-DECIMAL-OPERAND", + wantPath: "/rules/0/when/value", + original: "/rules/0/when/where/where/value", + }, + { + // The Go gate owns aggregate shape and depth only; an op §7 does not + // define is Core's business, and the projection is what refuses it. + name: "core-invalid-op-inside-where", + condition: `{"op":"exists","path":"/rows","where":{"op":"bogus"}}`, + wantCode: "JPS-STRUCTURE-CONDITION-SHAPE", + wantPath: "/rules/0/when", + original: "/rules/0/when/where", + }, + } + for _, testCase := range cases { + t.Run(testCase.name, func(t *testing.T) { + pack := draftPack(testCase.condition, "escalate") + root, ok := documentOf(t, string(pack)).(map[string]any) + if !ok { + t.Fatal("the row's pack must decode to an object") + } + // The Go grammar gate reports nothing here: every row's aggregate is + // well-shaped and within depth, and what is wrong sits inside the where. + if diagnostics := rfc0008Diagnostics(root); len(diagnostics) > 0 { + t.Fatalf("the draft grammar gate owns shape and depth only: %+v", diagnostics) + } + _, failure := engine.EvaluateWith(pack, []byte(`{"list":[]}`), nil, draftOptions(0)) + if failure == nil || failure.Code != "JPS-EVALUATION-PACK-NOT-CONFORMANT" { + t.Fatalf("failure = %+v", failure) + } + if !strings.Contains(failure.Message, testCase.wantCode) { + t.Fatalf("the refusal must name %s: %q", testCase.wantCode, failure.Message) + } + if !strings.Contains(failure.Message, " at "+testCase.wantPath+":") { + t.Fatalf("the refusal must report the projected path %s: %q", testCase.wantPath, failure.Message) + } + if strings.Contains(failure.Message, testCase.original) { + t.Fatalf("the projected path is not the authored one (%s): %q", testCase.original, failure.Message) + } + // Because it is not the authored path, the refusal has to say so, and + // must not send the author to spec validate, which for this pack + // reports the aggregate itself and never this finding. + if !strings.Contains(failure.Message, "relative to that projection") { + t.Fatalf("the refusal must qualify the projected path: %q", failure.Message) + } + if strings.Contains(failure.Message, "Run spec validate") { + t.Fatalf("a draft-operator pack must not be sent to spec validate for this report: %q", failure.Message) + } + }) + } +} + +// The adversarial row: a facts document sized past the limit yields an explicit +// resource error rather than true, false, or unresolved — and it yields the +// same error whether the dominant element sits first or last, which is what +// order-independent precharged accounting buys. Under a budget the same inputs +// fit, both permutations produce the same disposition, so the permutation is +// not doing the work. +func TestRFC0008WorkLimitIsOrderIndependent(t *testing.T) { + engine := newTestEngine(t) + pack := draftPack(`{"op":"exists","path":"/list","where":{"op":"fact","path":"/ok","operator":"equals","value":true}}`, "escalate") + + elements := make([]string, 0, 64) + for index := 0; index < 63; index++ { + elements = append(elements, `{"ok":false}`) + } + dominantFirst := []byte(fmt.Sprintf(`{"list":[%s]}`, strings.Join(append([]string{`{"ok":true}`}, elements...), ","))) + dominantLast := []byte(fmt.Sprintf(`{"list":[%s]}`, strings.Join(append(append([]string{}, elements...), `{"ok":true}`), ","))) + + // Over budget: the dominant element that lets an evaluator stop at index 0 + // must not buy it a disposition the other permutation cannot have. + first, firstFailure := engine.EvaluateWith(pack, dominantFirst, nil, draftOptions(64)) + last, lastFailure := engine.EvaluateWith(pack, dominantLast, nil, draftOptions(64)) + if firstFailure == nil || lastFailure == nil { + t.Fatalf("both permutations must be refused: %+v / %+v", firstFailure, lastFailure) + } + if !reflect.DeepEqual(firstFailure, lastFailure) { + t.Fatalf("the two permutations must produce the same error:\n first=%+v\n last=%+v", firstFailure, lastFailure) + } + if firstFailure.Code != "JPS-RESOURCE-EVALUATION-WORK-LIMIT" || firstFailure.ExitCode != result.ExitIO { + t.Fatalf("exhaustion must be an explicit resource error: %+v", firstFailure) + } + if first.Disposition.Kind != "" || last.Disposition.Kind != "" { + t.Fatalf("exhaustion is an error, never a disposition: %+v / %+v", first.Disposition, last.Disposition) + } + + // Within budget: identical dispositions, so the rows above turn on the limit + // and not on the permutation. + first, firstFailure = engine.EvaluateWith(pack, dominantFirst, nil, draftOptions(0)) + last, lastFailure = engine.EvaluateWith(pack, dominantLast, nil, draftOptions(0)) + if firstFailure != nil || lastFailure != nil { + t.Fatalf("within budget both permutations must evaluate: %+v / %+v", firstFailure, lastFailure) + } + if !reflect.DeepEqual(first.Disposition, last.Disposition) || first.Disposition.OutcomeID != "held" { + t.Fatalf("permuted inputs must produce one disposition: %+v / %+v", first.Disposition, last.Disposition) + } +} + +// The budget is charged across the whole evaluation, so sibling aggregates in +// different conditions add, and the error names the configured limit. +func TestRFC0008WorkLimitIsConfigurable(t *testing.T) { + engine := newTestEngine(t) + pack := draftPack(`{"op":"every","path":"/list","where":{"op":"fact","path":"/ok","operator":"equals","value":true}}`, "escalate") + facts := []byte(`{"list":[{"ok":true},{"ok":true},{"ok":true}]}`) + + // The charge is 12 for the aggregate — its node, the six-unit compile of + // "/list" and the five-unit walk — then 10 for the first element, whose + // "/ok" is compiled there and whose boolean is charged on both sides of the + // comparison, and 6 for each of the other two: 34 exactly. + if _, failure := engine.EvaluateWith(pack, facts, nil, draftOptions(34)); failure != nil { + t.Fatalf("a budget equal to the charge must not trip: %+v", failure) + } + _, failure := engine.EvaluateWith(pack, facts, nil, draftOptions(33)) + if failure == nil || failure.Code != "JPS-RESOURCE-EVALUATION-WORK-LIMIT" { + t.Fatalf("a budget one unit short must trip: %+v", failure) + } + if !strings.Contains(failure.Message, "33") { + t.Fatalf("the error must name the configured limit: %q", failure.Message) + } + + // The default budget is generous enough for a collection a pack plausibly + // quantifies over, and small enough to refuse the RFC's attack sketch. + if DefaultWorkBudget < 10_000 { + t.Fatalf("DefaultWorkBudget = %d is too small to be usable", DefaultWorkBudget) + } + if DefaultWorkBudget > 1_000_000 { + t.Fatalf("DefaultWorkBudget = %d does not bound the RFC's 10^6 attack sketch", DefaultWorkBudget) + } +} + +// A collection of numbers no arithmetic type can hold is ordinary work, and the +// charge says so. This row used to be the quadratic one: an arithmetic-based +// equality could not compare such values, every member became another +// representative of uniform's clause 3, and the preflight had to reserve one +// comparison pass per member — 2,204,711 units for the 400 members below, which +// the default budget refused. Deciding equality on the tokens removes both the +// quadratic and the unknown: one representative absorbs the members, the charge +// is a sum over them plus one reread of the largest value per comparison, and +// the collection produces a disposition. +func TestRFC0008HugeNumericTokensAreLinearAndDecided(t *testing.T) { + engine := newTestEngine(t) + pack := draftPack(`{"op":"uniform","path":"/list","at":"/cabin"}`, "escalate") + // Distinct values, each a fourteen-byte token — 1001e999999999 and its + // siblings — so every member costs the same and the arithmetic below is by + // hand: six units to step the compiled "/cabin", fifteen for the token, and + // fifteen for the one comparison the member takes part in. + facts := func(members int) string { + items := make([]string, 0, members) + for index := 1; index <= members; index++ { + items = append(items, fmt.Sprintf(`{"cabin":1%03de999999999}`, index)) + } + return fmt.Sprintf(`{"list":[%s]}`, strings.Join(items, ",")) + } + const condition = `{"op":"uniform","path":"/list","at":"/cabin"}` + + // Linear, with no quadratic term hiding in it: doubling the members adds + // exactly one per-member charge and one reread per added member. The reread + // term is (n-1) times the largest resolved value, and every value here is + // the same fifteen units, so each further member adds six to step "/cabin", + // fifteen for its own token, and fifteen for the comparison it takes part + // in — a constant per member, not a term that grows with the collection. + small, large := chargeOf(t, condition, facts(200)), chargeOf(t, condition, facts(400)) + if want := 200 * (6 + 15 + 15); large-small != want { + t.Fatalf("charges %d and %d differ by %d; 200 further members must cost %d", small, large, large-small, want) + } + + // 14,404 units for the 400 members, against the default 100,000: the + // collection is evaluated rather than refused, and it decides — the tokens + // are pairwise unequal, so clause 3 makes the rule false and the pack falls + // back rather than escalating an unknown. + output, failure := engine.EvaluateWith(pack, []byte(facts(400)), nil, draftOptions(0)) + if failure != nil { + t.Fatalf("400 members at 36 units apiece must fit the default budget: %+v", failure) + } + if output.Disposition.Kind != "outcome" || output.Disposition.OutcomeID != "did-not-hold" { + t.Fatalf("disposition = %+v; distinct huge tokens are unequal, not unknown", output.Disposition) + } + + // The same members written as one repeated value are equal, which is the + // other half of determinacy: uniform is true rather than unknown. + equal := make([]string, 0, 400) + for index := 0; index < 400; index++ { + equal = append(equal, `{"cabin":1.0e999999999}`) + } + output, failure = engine.EvaluateWith(pack, []byte(fmt.Sprintf(`{"list":[%s]}`, strings.Join(equal, ","))), nil, draftOptions(0)) + if failure != nil { + t.Fatalf("400 equal members must fit the default budget too: %+v", failure) + } + if output.Disposition.Kind != "outcome" || output.Disposition.OutcomeID != "held" { + t.Fatalf("disposition = %+v; equal huge tokens are equal, not unknown", output.Disposition) + } +} + +// A condition §8 never reaches is never charged, so the total is a property of +// the §8 path rather than of pack and facts alone. A suppressed rule is the +// clearest case: its aggregate costs nothing, and the same pack with the +// suppression withheld exhausts the same budget on the same facts. §8 fixes the +// path, so this is deterministic across evaluators rather than a portability +// hole — but it is the code's behavior, so it is pinned rather than described. +func TestRFC0008SuppressedRuleIsNeverCharged(t *testing.T) { + engine := newTestEngine(t) + pack := func(suppressWhen string) []byte { + return []byte(fmt.Sprintf(`{ + "specVersion": "0.1.0-draft", + "id": "https://example.invalid/judgment-packs/rfc0008-suppression", + "version": "0.1.0", + "title": "Synthetic draft RFC 0008 suppression row", + "description": "Invented content for specification testing; it authorizes nothing.", + "decision": { + "intent": "Exercise the work charge of a rule §8 removes before evaluating it.", + "question": "Was the expensive rule charged?" + }, + "outcomes": [ + {"id": "held", "label": "Held"}, + {"id": "did-not-hold", "label": "Did not hold"} + ], + "rules": [ + { + "id": "the-expensive-rule", + "description": "Quantifies over every element of a long collection.", + "when": {"op": "exists", "path": "/list", "where": {"op": "fact", "path": "/ok", "operator": "equals", "value": true}}, + "outcome": "held", + "onUnknown": "escalate" + } + ], + "exceptions": [ + { + "id": "the-suppression", + "description": "Removes the expensive rule before it is evaluated.", + "when": %s, + "effect": "suppress-rule", + "targetRule": "the-expensive-rule", + "onUnknown": "ignore" + } + ], + "fallbackOutcome": "did-not-hold" +}`, suppressWhen)) + } + + elements := make([]string, 0, 20) + for index := 0; index < 20; index++ { + elements = append(elements, `{"ok":true}`) + } + facts := []byte(fmt.Sprintf(`{"list":[%s]}`, strings.Join(elements, ","))) + + // The exception's literal costs one unit; the rule's aggregate would cost + // 136. A budget of one unit therefore admits the whole evaluation exactly + // when the suppressed rule is never charged. + output, failure := engine.EvaluateWith(pack(`{"op":"literal","value":true}`), facts, nil, draftOptions(1)) + if failure != nil { + t.Fatalf("a suppressed rule must cost nothing: %+v", failure) + } + if output.Disposition.Kind != "outcome" || output.Disposition.OutcomeID != "did-not-hold" { + t.Fatalf("disposition = %+v", output.Disposition) + } + + // With the suppression withheld the rule is reached and the same budget is + // exhausted, so the row above turns on §8's path and not on the numbers. + _, failure = engine.EvaluateWith(pack(`{"op":"literal","value":false}`), facts, nil, draftOptions(1)) + if failure == nil || failure.Code != "JPS-RESOURCE-EVALUATION-WORK-LIMIT" { + t.Fatalf("an evaluated rule must be charged: %+v", failure) + } +} + +// The two ragged unknown rows run twice each, once under onUnknown: ignore and +// once under escalate, so the divergence is visible in the disposition. +func TestRFC0008RaggedUnknownUnderBothPolicies(t *testing.T) { + engine := newTestEngine(t) + cases := []struct { + name string + condition string + facts string + onUnknown string + wantKind string + wantOutcome string + wantReasons []string + }{ + { + name: "exists-ragged-unknown-ignored", + condition: `{"op":"exists","path":"/list","where":{"op":"fact","path":"/ok","operator":"equals","value":true}}`, + facts: `{"list":[{"ok":false},{"ok":false},{}]}`, + onUnknown: "ignore", + wantKind: "outcome", + wantOutcome: "did-not-hold", + wantReasons: []string{}, + }, + { + name: "exists-ragged-unknown-escalated", + condition: `{"op":"exists","path":"/list","where":{"op":"fact","path":"/ok","operator":"equals","value":true}}`, + facts: `{"list":[{"ok":false},{"ok":false},{}]}`, + onUnknown: "escalate", + wantKind: "unresolved", + wantReasons: []string{"unknown"}, + }, + { + name: "every-ragged-unknown-ignored", + condition: `{"op":"every","path":"/list","where":{"op":"fact","path":"/ok","operator":"equals","value":true}}`, + facts: `{"list":[{"ok":true},{"ok":true},{}]}`, + onUnknown: "ignore", + wantKind: "outcome", + wantOutcome: "did-not-hold", + wantReasons: []string{}, + }, + { + name: "every-ragged-unknown-escalated", + condition: `{"op":"every","path":"/list","where":{"op":"fact","path":"/ok","operator":"equals","value":true}}`, + facts: `{"list":[{"ok":true},{"ok":true},{}]}`, + onUnknown: "escalate", + wantKind: "unresolved", + wantReasons: []string{"unknown"}, + }, + } + for _, testCase := range cases { + t.Run(testCase.name, func(t *testing.T) { + output, failure := engine.EvaluateWith(draftPack(testCase.condition, testCase.onUnknown), []byte(testCase.facts), nil, draftOptions(0)) + if failure != nil { + t.Fatalf("evaluation failed: %s", failure.Message) + } + if output.Disposition.Kind != testCase.wantKind || output.Disposition.OutcomeID != testCase.wantOutcome { + t.Fatalf("disposition = %+v", output.Disposition) + } + if !reflect.DeepEqual(output.Disposition.Reasons, testCase.wantReasons) { + t.Fatalf("reasons = %v, want %v", output.Disposition.Reasons, testCase.wantReasons) + } + }) + } +} + +// The equivalence check the RFC asks any implementation to run, on all three +// census facts a quantifier actually reaches, each as its own pair of packs: +// A6:/reservation/anySegmentCancelledByAirline as an exists over segments +// carrying a cancelledByAirline boolean, R3:/modification/allNewItemsAvailable +// as an every over the modification's items, and +// R5:/request/allNewItemsAvailable as an every over the request's new items. +// R3 and R5 are separate rooms with separate facts, so they are separate +// artifacts here rather than one pack read twice. A prepared-boolean pack and +// its quantifier twin must produce identical dispositions from the facts each is +// given. +func TestRFC0008EquivalenceWithPreparedBooleanPacks(t *testing.T) { + engine := newTestEngine(t) + read := func(name string) []byte { + t.Helper() + data, err := os.ReadFile(filepath.Join("testdata", "rfc0008", name)) + if err != nil { + t.Fatal(err) + } + return data + } + airlinePrepared := read("airline-cancellation-prepared.json") + airlineQuantifier := read("airline-cancellation-quantifier.json") + itemsPrepared := read("item-availability-prepared.json") + itemsQuantifier := read("item-availability-quantifier.json") + exchangePrepared := read("exchange-item-availability-prepared.json") + exchangeQuantifier := read("exchange-item-availability-quantifier.json") + + cases := []struct { + name string + prepared []byte + preparedFacts string + quantifier []byte + quantifierFacts string + wantKind string + wantOutcome string + }{ + { + name: "a6-one-segment-cancelled-by-airline", + prepared: airlinePrepared, + preparedFacts: `{"reservation":{"anySegmentCancelledByAirline":true}}`, + quantifier: airlineQuantifier, + quantifierFacts: `{"reservation":{"segments":[{"cancelledByAirline":false},{"cancelledByAirline":true}]}}`, + wantKind: "outcome", + wantOutcome: "free-cancellation", + }, + { + name: "a6-no-segment-cancelled-by-airline", + prepared: airlinePrepared, + preparedFacts: `{"reservation":{"anySegmentCancelledByAirline":false}}`, + quantifier: airlineQuantifier, + quantifierFacts: `{"reservation":{"segments":[{"cancelledByAirline":false},{"cancelledByAirline":false}]}}`, + wantKind: "outcome", + wantOutcome: "standard-rules", + }, + { + name: "a6-no-segments-at-all", + prepared: airlinePrepared, + preparedFacts: `{"reservation":{"anySegmentCancelledByAirline":false}}`, + quantifier: airlineQuantifier, + quantifierFacts: `{"reservation":{"segments":[]}}`, + wantKind: "outcome", + wantOutcome: "standard-rules", + }, + { + name: "a6-fact-absent-on-both-sides", + prepared: airlinePrepared, + preparedFacts: `{"reservation":{}}`, + quantifier: airlineQuantifier, + quantifierFacts: `{"reservation":{}}`, + wantKind: "unresolved", + }, + { + name: "a6-one-segment-missing-the-member", + prepared: airlinePrepared, + preparedFacts: `{"reservation":{}}`, + quantifier: airlineQuantifier, + quantifierFacts: `{"reservation":{"segments":[{"cancelledByAirline":false},{}]}}`, + wantKind: "unresolved", + }, + { + name: "r3-every-item-available", + prepared: itemsPrepared, + preparedFacts: `{"modification":{"allNewItemsAvailable":true}}`, + quantifier: itemsQuantifier, + quantifierFacts: `{"modification":{"items":[{"availability":"available"},{"availability":"available"}]}}`, + wantKind: "outcome", + wantOutcome: "modification-allowed", + }, + { + name: "r3-one-item-unavailable", + prepared: itemsPrepared, + preparedFacts: `{"modification":{"allNewItemsAvailable":false}}`, + quantifier: itemsQuantifier, + quantifierFacts: `{"modification":{"items":[{"availability":"available"},{"availability":"out-of-stock"}]}}`, + wantKind: "outcome", + wantOutcome: "modification-refused", + }, + { + // Vacuous truth, agreed on both sides: a producer computing + // allNewItemsAvailable over an empty request also reports true. The + // equivalence holds only because the two artifacts agree about the + // empty case, which is exactly the authoring footgun the RFC records. + name: "r3-no-items-requested", + prepared: itemsPrepared, + preparedFacts: `{"modification":{"allNewItemsAvailable":true}}`, + quantifier: itemsQuantifier, + quantifierFacts: `{"modification":{"items":[]}}`, + wantKind: "outcome", + wantOutcome: "modification-allowed", + }, + { + name: "r3-one-item-missing-availability", + prepared: itemsPrepared, + preparedFacts: `{"modification":{}}`, + quantifier: itemsQuantifier, + quantifierFacts: `{"modification":{"items":[{"availability":"available"},{}]}}`, + wantKind: "unresolved", + }, + { + name: "r5-every-new-item-available", + prepared: exchangePrepared, + preparedFacts: `{"request":{"allNewItemsAvailable":true}}`, + quantifier: exchangeQuantifier, + quantifierFacts: `{"request":{"newItems":[{"availability":"available"},{"availability":"available"}]}}`, + wantKind: "outcome", + wantOutcome: "exchange-allowed", + }, + { + name: "r5-one-new-item-unavailable", + prepared: exchangePrepared, + preparedFacts: `{"request":{"allNewItemsAvailable":false}}`, + quantifier: exchangeQuantifier, + quantifierFacts: `{"request":{"newItems":[{"availability":"discontinued"},{"availability":"available"}]}}`, + wantKind: "outcome", + wantOutcome: "exchange-refused", + }, + { + // The same vacuous-truth agreement R3 needs, in the other room: a + // producer computing allNewItemsAvailable over an empty exchange + // request also reports true. + name: "r5-no-new-items-requested", + prepared: exchangePrepared, + preparedFacts: `{"request":{"allNewItemsAvailable":true}}`, + quantifier: exchangeQuantifier, + quantifierFacts: `{"request":{"newItems":[]}}`, + wantKind: "outcome", + wantOutcome: "exchange-allowed", + }, + { + // R5's own residue records what the prepared boolean loses here: one + // item without an availability makes the whole boolean unresolved, + // and the quantifier is no better at saying which item it was. + name: "r5-one-new-item-missing-availability", + prepared: exchangePrepared, + preparedFacts: `{"request":{}}`, + quantifier: exchangeQuantifier, + quantifierFacts: `{"request":{"newItems":[{"availability":"available"},{}]}}`, + wantKind: "unresolved", + }, + { + name: "r5-fact-absent-on-both-sides", + prepared: exchangePrepared, + preparedFacts: `{"request":{}}`, + quantifier: exchangeQuantifier, + quantifierFacts: `{"request":{}}`, + wantKind: "unresolved", + }, + } + + for _, testCase := range cases { + t.Run(testCase.name, func(t *testing.T) { + prepared, failure := engine.Evaluate(testCase.prepared, []byte(testCase.preparedFacts), nil, nil, "test") + if failure != nil { + t.Fatalf("the prepared-boolean pack must evaluate without the opt-in: %s", failure.Message) + } + quantifier, failure := engine.EvaluateWith(testCase.quantifier, []byte(testCase.quantifierFacts), nil, draftOptions(0)) + if failure != nil { + t.Fatalf("the quantifier pack must evaluate under the opt-in: %s", failure.Message) + } + if !reflect.DeepEqual(prepared.Disposition, quantifier.Disposition) { + t.Fatalf("dispositions differ:\n prepared=%+v\n quantifier=%+v", prepared.Disposition, quantifier.Disposition) + } + if prepared.Disposition.Kind != testCase.wantKind || prepared.Disposition.OutcomeID != testCase.wantOutcome { + t.Fatalf("disposition = %+v, want kind %q outcome %q", prepared.Disposition, testCase.wantKind, testCase.wantOutcome) + } + // Only the quantifier twin is a draft-RFC prototype, and only it is + // refused by the published grammar. + if prepared.DraftPrototype != nil { + t.Fatal("a Core pack evaluated without the opt-in must carry no prototype marker") + } + if quantifier.DraftPrototype == nil || quantifier.DraftPrototype.PackValidUnderSpecVersion { + t.Fatalf("marker = %+v", quantifier.DraftPrototype) + } + }) + } + + // The quantifier twins are exactly the packs the published validator + // rejects; the prepared twins are exactly the packs it accepts. + validator, err := validation.NewEngine() + if err != nil { + t.Fatal(err) + } + for name, pack := range map[string][]byte{ + "airline-cancellation-quantifier": airlineQuantifier, + "item-availability-quantifier": itemsQuantifier, + "exchange-item-availability-quantifier": exchangeQuantifier, + } { + output, operational := validator.Validate(pack, validation.Options{Through: "semantic", Limits: carrier.DefaultLimits()}) + if operational != nil || output.Status != "invalid" { + t.Fatalf("%s must be invalid under the published schema: %+v", name, output.Status) + } + } + for name, pack := range map[string][]byte{ + "airline-cancellation-prepared": airlinePrepared, + "item-availability-prepared": itemsPrepared, + "exchange-item-availability-prepared": exchangePrepared, + } { + output, operational := validator.Validate(pack, validation.Options{Through: "semantic", Limits: carrier.DefaultLimits()}) + if operational != nil || output.Status != "valid" { + t.Fatalf("%s must be valid under the published schema: %+v", name, output.Status) + } + } +} diff --git a/internal/evaluation/testdata/README.md b/internal/evaluation/testdata/README.md index 0c4f740..cdaf99d 100644 --- a/internal/evaluation/testdata/README.md +++ b/internal/evaluation/testdata/README.md @@ -4,3 +4,23 @@ data-request-intake-triage.json is copied verbatim from Judgment-Pack/judgment-pack-spec examples/ (commit 839488c). It is the pack that spec RFC 0006's appendix instances walk; the evaluation tests execute those instances against this engine. + +rfc0008-condition-schema.json is the depth-indexed condition grammar of the +specification's RFC 0008 (Draft), written out as the RFC describes it: three +condition definitions by remaining aggregate depth, with `all`/`any`/`not` +recursing within their own tier and the depth-zero tier carrying no aggregate +branch, so aggregate depth three is unrepresentable rather than merely +forbidden. Its Core branches are the bundled JPS 0.1.0-draft schema's +`$defs/condition` branches, re-targeted per tier. It is a prototype artifact of +an open proposal: it is not a JPS schema, and no document validated against it +is valid under any published JPS version. + +rfc0008/ holds the equivalence-check fixtures RFC 0008's Conformance section +asks any implementation to run, on the two census shapes a bare quantifier +reaches. Each pair is one prepared-boolean pack and one quantifier twin over +the same policy: airline-cancellation-* re-encodes +`A6:/reservation/anySegmentCancelledByAirline`, and item-availability-* +re-encodes the availability conjunct of `R3:/modification/allNewItemsAvailable` +(and its `R5` twin). The prepared packs are valid JPS 0.1.0-draft documents; +the quantifier packs are deliberately not, and the tests assert both facts. All +content is invented for specification testing and authorizes nothing. diff --git a/internal/evaluation/testdata/rfc0008-condition-schema.json b/internal/evaluation/testdata/rfc0008-condition-schema.json new file mode 100644 index 0000000..f9152b0 --- /dev/null +++ b/internal/evaluation/testdata/rfc0008-condition-schema.json @@ -0,0 +1,566 @@ +{ + "$schema": "https://json-schema.org/draft/2020-12/schema", + "$id": "urn:judgmentpack:rfc0008:condition:draft", + "title": "Draft RFC 0008 condition grammar (prototype, not a JPS artifact)", + "$comment": "Depth-indexed condition definitions for the specification's RFC 0008 (Draft). The Core branches are copied verbatim from the bundled JPS 0.1.0-draft schema's $defs/condition and re-targeted per tier; the exists/every and uniform branches are the RFC's. Three tiers by remaining aggregate depth enforce the maximum aggregate depth of two without a per-implementation depth counter. This is a test artifact of an open proposal: no pack validated against it is valid under any published JPS version.", + "$ref": "#/$defs/condition", + "$defs": { + "factPath": { + "type": "string", + "pattern": "^(?:/(?:[^~/]|~0|~1)*)*$" + }, + "localId": { + "type": "string", + "pattern": "^[a-z][a-z0-9]*(?:-[a-z0-9]+)*$" + }, + "decimalString": { + "type": "string", + "pattern": "^-?(?:0|[1-9][0-9]*)(?:\\.[0-9]+)?$" + }, + "condition": { + "$comment": "Remaining aggregate depth 2. all, any, and not recurse within this tier, so wrapping an aggregate cannot launder its depth; an aggregate takes its where from the next tier down.", + "oneOf": [ + { + "type": "object", + "additionalProperties": false, + "required": [ + "op", + "value" + ], + "properties": { + "op": { + "const": "literal" + }, + "value": { + "type": "boolean" + } + } + }, + { + "type": "object", + "additionalProperties": false, + "required": [ + "op", + "conditions" + ], + "properties": { + "op": { + "enum": [ + "all", + "any" + ] + }, + "conditions": { + "type": "array", + "minItems": 1, + "items": { + "$ref": "#/$defs/condition" + } + } + } + }, + { + "type": "object", + "additionalProperties": false, + "required": [ + "op", + "condition" + ], + "properties": { + "op": { + "const": "not" + }, + "condition": { + "$ref": "#/$defs/condition" + } + } + }, + { + "type": "object", + "additionalProperties": false, + "required": [ + "op", + "path", + "operator", + "value" + ], + "properties": { + "op": { + "const": "fact" + }, + "path": { + "type": "string", + "pattern": "^(?:/(?:[^~/]|~0|~1)*)*$" + }, + "operator": { + "enum": [ + "equals", + "not-equals", + "greater-than", + "greater-than-or-equal", + "less-than", + "less-than-or-equal", + "in" + ] + }, + "value": true + }, + "allOf": [ + { + "if": { + "properties": { + "operator": { + "enum": [ + "greater-than", + "greater-than-or-equal", + "less-than", + "less-than-or-equal" + ] + } + }, + "required": [ + "operator" + ] + }, + "then": { + "properties": { + "value": { + "$ref": "#/$defs/decimalString" + } + } + } + }, + { + "if": { + "properties": { + "operator": { + "const": "in" + } + }, + "required": [ + "operator" + ] + }, + "then": { + "properties": { + "value": { + "type": "array", + "minItems": 1 + } + } + } + } + ] + }, + { + "type": "object", + "additionalProperties": false, + "required": [ + "op", + "evidenceRequirement" + ], + "properties": { + "op": { + "const": "evidence-present" + }, + "evidenceRequirement": { + "$ref": "#/$defs/localId" + } + } + }, + { + "type": "object", + "additionalProperties": false, + "required": [ + "op", + "path", + "where" + ], + "properties": { + "op": { + "enum": [ + "exists", + "every" + ] + }, + "path": { + "$ref": "#/$defs/factPath" + }, + "where": { + "$ref": "#/$defs/conditionDepth1" + } + } + }, + { + "type": "object", + "additionalProperties": false, + "required": [ + "op", + "path", + "at" + ], + "properties": { + "op": { + "const": "uniform" + }, + "path": { + "$ref": "#/$defs/factPath" + }, + "at": { + "$ref": "#/$defs/factPath" + } + } + } + ] + }, + "conditionDepth1": { + "$comment": "Remaining aggregate depth 1. An aggregate here is at depth two, which is the bound; its where comes from the depth-zero tier, which carries no aggregate branch.", + "oneOf": [ + { + "type": "object", + "additionalProperties": false, + "required": [ + "op", + "value" + ], + "properties": { + "op": { + "const": "literal" + }, + "value": { + "type": "boolean" + } + } + }, + { + "type": "object", + "additionalProperties": false, + "required": [ + "op", + "conditions" + ], + "properties": { + "op": { + "enum": [ + "all", + "any" + ] + }, + "conditions": { + "type": "array", + "minItems": 1, + "items": { + "$ref": "#/$defs/conditionDepth1" + } + } + } + }, + { + "type": "object", + "additionalProperties": false, + "required": [ + "op", + "condition" + ], + "properties": { + "op": { + "const": "not" + }, + "condition": { + "$ref": "#/$defs/conditionDepth1" + } + } + }, + { + "type": "object", + "additionalProperties": false, + "required": [ + "op", + "path", + "operator", + "value" + ], + "properties": { + "op": { + "const": "fact" + }, + "path": { + "type": "string", + "pattern": "^(?:/(?:[^~/]|~0|~1)*)*$" + }, + "operator": { + "enum": [ + "equals", + "not-equals", + "greater-than", + "greater-than-or-equal", + "less-than", + "less-than-or-equal", + "in" + ] + }, + "value": true + }, + "allOf": [ + { + "if": { + "properties": { + "operator": { + "enum": [ + "greater-than", + "greater-than-or-equal", + "less-than", + "less-than-or-equal" + ] + } + }, + "required": [ + "operator" + ] + }, + "then": { + "properties": { + "value": { + "$ref": "#/$defs/decimalString" + } + } + } + }, + { + "if": { + "properties": { + "operator": { + "const": "in" + } + }, + "required": [ + "operator" + ] + }, + "then": { + "properties": { + "value": { + "type": "array", + "minItems": 1 + } + } + } + } + ] + }, + { + "type": "object", + "additionalProperties": false, + "required": [ + "op", + "evidenceRequirement" + ], + "properties": { + "op": { + "const": "evidence-present" + }, + "evidenceRequirement": { + "$ref": "#/$defs/localId" + } + } + }, + { + "type": "object", + "additionalProperties": false, + "required": [ + "op", + "path", + "where" + ], + "properties": { + "op": { + "enum": [ + "exists", + "every" + ] + }, + "path": { + "$ref": "#/$defs/factPath" + }, + "where": { + "$ref": "#/$defs/conditionDepth0" + } + } + }, + { + "type": "object", + "additionalProperties": false, + "required": [ + "op", + "path", + "at" + ], + "properties": { + "op": { + "const": "uniform" + }, + "path": { + "$ref": "#/$defs/factPath" + }, + "at": { + "$ref": "#/$defs/factPath" + } + } + } + ] + }, + "conditionDepth0": { + "$comment": "Remaining aggregate depth 0: Core conditions only. No aggregate branch exists here at all, so depth three is unrepresentable rather than merely forbidden.", + "oneOf": [ + { + "type": "object", + "additionalProperties": false, + "required": [ + "op", + "value" + ], + "properties": { + "op": { + "const": "literal" + }, + "value": { + "type": "boolean" + } + } + }, + { + "type": "object", + "additionalProperties": false, + "required": [ + "op", + "conditions" + ], + "properties": { + "op": { + "enum": [ + "all", + "any" + ] + }, + "conditions": { + "type": "array", + "minItems": 1, + "items": { + "$ref": "#/$defs/conditionDepth0" + } + } + } + }, + { + "type": "object", + "additionalProperties": false, + "required": [ + "op", + "condition" + ], + "properties": { + "op": { + "const": "not" + }, + "condition": { + "$ref": "#/$defs/conditionDepth0" + } + } + }, + { + "type": "object", + "additionalProperties": false, + "required": [ + "op", + "path", + "operator", + "value" + ], + "properties": { + "op": { + "const": "fact" + }, + "path": { + "type": "string", + "pattern": "^(?:/(?:[^~/]|~0|~1)*)*$" + }, + "operator": { + "enum": [ + "equals", + "not-equals", + "greater-than", + "greater-than-or-equal", + "less-than", + "less-than-or-equal", + "in" + ] + }, + "value": true + }, + "allOf": [ + { + "if": { + "properties": { + "operator": { + "enum": [ + "greater-than", + "greater-than-or-equal", + "less-than", + "less-than-or-equal" + ] + } + }, + "required": [ + "operator" + ] + }, + "then": { + "properties": { + "value": { + "$ref": "#/$defs/decimalString" + } + } + } + }, + { + "if": { + "properties": { + "operator": { + "const": "in" + } + }, + "required": [ + "operator" + ] + }, + "then": { + "properties": { + "value": { + "type": "array", + "minItems": 1 + } + } + } + } + ] + }, + { + "type": "object", + "additionalProperties": false, + "required": [ + "op", + "evidenceRequirement" + ], + "properties": { + "op": { + "const": "evidence-present" + }, + "evidenceRequirement": { + "$ref": "#/$defs/localId" + } + } + } + ] + } + } +} diff --git a/internal/evaluation/testdata/rfc0008/airline-cancellation-prepared.json b/internal/evaluation/testdata/rfc0008/airline-cancellation-prepared.json new file mode 100644 index 0000000..f3f90fc --- /dev/null +++ b/internal/evaluation/testdata/rfc0008/airline-cancellation-prepared.json @@ -0,0 +1,38 @@ +{ + "specVersion": "0.1.0-draft", + "id": "https://example.invalid/judgment-packs/airline-cancellation-prepared", + "version": "0.1.0", + "title": "Synthetic airline cancellation triage (prepared boolean)", + "description": "Synthetic re-encoding of census fact A6:/reservation/anySegmentCancelledByAirline. Invented content for specification testing; it authorizes nothing. This twin reads the prepared boolean a producer supplies.", + "decision": { + "intent": "Decide whether an airline-caused cancellation entitles the reservation to a free cancellation.", + "question": "Was any segment of this reservation cancelled by the airline?" + }, + "outcomes": [ + { + "id": "free-cancellation", + "label": "Free cancellation", + "description": "The reservation may be cancelled at no charge." + }, + { + "id": "standard-rules", + "label": "Standard cancellation rules", + "description": "The ordinary fare rules decide the charge." + } + ], + "rules": [ + { + "id": "airline-cancelled-segment", + "description": "A segment cancelled by the airline entitles the reservation to a free cancellation.", + "when": { + "op": "fact", + "path": "/reservation/anySegmentCancelledByAirline", + "operator": "equals", + "value": true + }, + "outcome": "free-cancellation", + "onUnknown": "escalate" + } + ], + "fallbackOutcome": "standard-rules" +} \ No newline at end of file diff --git a/internal/evaluation/testdata/rfc0008/airline-cancellation-quantifier.json b/internal/evaluation/testdata/rfc0008/airline-cancellation-quantifier.json new file mode 100644 index 0000000..0e81907 --- /dev/null +++ b/internal/evaluation/testdata/rfc0008/airline-cancellation-quantifier.json @@ -0,0 +1,42 @@ +{ + "specVersion": "0.1.0-draft", + "id": "https://example.invalid/judgment-packs/airline-cancellation-quantifier", + "version": "0.1.0", + "title": "Synthetic airline cancellation triage (draft RFC 0008 quantifier)", + "description": "Synthetic re-encoding of census fact A6:/reservation/anySegmentCancelledByAirline. Invented content for specification testing; it authorizes nothing. This twin quantifies over the per-segment records instead of reading a prepared boolean, and is NOT valid under JPS 0.1.0-draft.", + "decision": { + "intent": "Decide whether an airline-caused cancellation entitles the reservation to a free cancellation.", + "question": "Was any segment of this reservation cancelled by the airline?" + }, + "outcomes": [ + { + "id": "free-cancellation", + "label": "Free cancellation", + "description": "The reservation may be cancelled at no charge." + }, + { + "id": "standard-rules", + "label": "Standard cancellation rules", + "description": "The ordinary fare rules decide the charge." + } + ], + "rules": [ + { + "id": "airline-cancelled-segment", + "description": "A segment cancelled by the airline entitles the reservation to a free cancellation.", + "when": { + "op": "exists", + "path": "/reservation/segments", + "where": { + "op": "fact", + "path": "/cancelledByAirline", + "operator": "equals", + "value": true + } + }, + "outcome": "free-cancellation", + "onUnknown": "escalate" + } + ], + "fallbackOutcome": "standard-rules" +} \ No newline at end of file diff --git a/internal/evaluation/testdata/rfc0008/exchange-item-availability-prepared.json b/internal/evaluation/testdata/rfc0008/exchange-item-availability-prepared.json new file mode 100644 index 0000000..0b8e59d --- /dev/null +++ b/internal/evaluation/testdata/rfc0008/exchange-item-availability-prepared.json @@ -0,0 +1,38 @@ +{ + "specVersion": "0.1.0-draft", + "id": "https://example.invalid/judgment-packs/exchange-item-availability-prepared", + "version": "0.1.0", + "title": "Synthetic exchange item availability check (prepared boolean)", + "description": "Synthetic re-encoding of census fact R5:/request/allNewItemsAvailable, scoped to the availability conjunct a quantifier reaches. Invented content for specification testing; it authorizes nothing. This twin reads the prepared boolean a producer supplies.", + "decision": { + "intent": "Decide whether a requested item exchange may proceed on availability grounds alone.", + "question": "Is every requested new item available?" + }, + "outcomes": [ + { + "id": "exchange-allowed", + "label": "Exchange allowed", + "description": "Every requested new item is available." + }, + { + "id": "exchange-refused", + "label": "Exchange refused", + "description": "At least one requested new item is not available." + } + ], + "rules": [ + { + "id": "all-new-items-available", + "description": "Every requested new item is available in the catalog.", + "when": { + "op": "fact", + "path": "/request/allNewItemsAvailable", + "operator": "equals", + "value": true + }, + "outcome": "exchange-allowed", + "onUnknown": "escalate" + } + ], + "fallbackOutcome": "exchange-refused" +} diff --git a/internal/evaluation/testdata/rfc0008/exchange-item-availability-quantifier.json b/internal/evaluation/testdata/rfc0008/exchange-item-availability-quantifier.json new file mode 100644 index 0000000..d923a85 --- /dev/null +++ b/internal/evaluation/testdata/rfc0008/exchange-item-availability-quantifier.json @@ -0,0 +1,42 @@ +{ + "specVersion": "0.1.0-draft", + "id": "https://example.invalid/judgment-packs/exchange-item-availability-quantifier", + "version": "0.1.0", + "title": "Synthetic exchange item availability check (draft RFC 0008 quantifier)", + "description": "Synthetic re-encoding of census fact R5:/request/allNewItemsAvailable, scoped to the availability conjunct a quantifier reaches. Invented content for specification testing; it authorizes nothing. This twin quantifies over the requested new items instead of reading a prepared boolean, and is NOT valid under JPS 0.1.0-draft.", + "decision": { + "intent": "Decide whether a requested item exchange may proceed on availability grounds alone.", + "question": "Is every requested new item available?" + }, + "outcomes": [ + { + "id": "exchange-allowed", + "label": "Exchange allowed", + "description": "Every requested new item is available." + }, + { + "id": "exchange-refused", + "label": "Exchange refused", + "description": "At least one requested new item is not available." + } + ], + "rules": [ + { + "id": "all-new-items-available", + "description": "Every requested new item is available in the catalog.", + "when": { + "op": "every", + "path": "/request/newItems", + "where": { + "op": "fact", + "path": "/availability", + "operator": "equals", + "value": "available" + } + }, + "outcome": "exchange-allowed", + "onUnknown": "escalate" + } + ], + "fallbackOutcome": "exchange-refused" +} diff --git a/internal/evaluation/testdata/rfc0008/item-availability-prepared.json b/internal/evaluation/testdata/rfc0008/item-availability-prepared.json new file mode 100644 index 0000000..df0b209 --- /dev/null +++ b/internal/evaluation/testdata/rfc0008/item-availability-prepared.json @@ -0,0 +1,38 @@ +{ + "specVersion": "0.1.0-draft", + "id": "https://example.invalid/judgment-packs/item-availability-prepared", + "version": "0.1.0", + "title": "Synthetic item availability check (prepared boolean)", + "description": "Synthetic re-encoding of census fact R3:/modification/allNewItemsAvailable, scoped to the availability conjunct a quantifier reaches. Invented content for specification testing; it authorizes nothing. This twin reads the prepared boolean a producer supplies.", + "decision": { + "intent": "Decide whether a requested item modification may proceed on availability grounds alone.", + "question": "Is every requested new item available?" + }, + "outcomes": [ + { + "id": "modification-allowed", + "label": "Modification allowed", + "description": "Every requested new item is available." + }, + { + "id": "modification-refused", + "label": "Modification refused", + "description": "At least one requested new item is not available." + } + ], + "rules": [ + { + "id": "all-new-items-available", + "description": "Every requested new item is available in the catalog.", + "when": { + "op": "fact", + "path": "/modification/allNewItemsAvailable", + "operator": "equals", + "value": true + }, + "outcome": "modification-allowed", + "onUnknown": "escalate" + } + ], + "fallbackOutcome": "modification-refused" +} \ No newline at end of file diff --git a/internal/evaluation/testdata/rfc0008/item-availability-quantifier.json b/internal/evaluation/testdata/rfc0008/item-availability-quantifier.json new file mode 100644 index 0000000..3879714 --- /dev/null +++ b/internal/evaluation/testdata/rfc0008/item-availability-quantifier.json @@ -0,0 +1,42 @@ +{ + "specVersion": "0.1.0-draft", + "id": "https://example.invalid/judgment-packs/item-availability-quantifier", + "version": "0.1.0", + "title": "Synthetic item availability check (draft RFC 0008 quantifier)", + "description": "Synthetic re-encoding of census fact R3:/modification/allNewItemsAvailable, scoped to the availability conjunct a quantifier reaches. Invented content for specification testing; it authorizes nothing. This twin quantifies over the requested items instead of reading a prepared boolean, and is NOT valid under JPS 0.1.0-draft.", + "decision": { + "intent": "Decide whether a requested item modification may proceed on availability grounds alone.", + "question": "Is every requested new item available?" + }, + "outcomes": [ + { + "id": "modification-allowed", + "label": "Modification allowed", + "description": "Every requested new item is available." + }, + { + "id": "modification-refused", + "label": "Modification refused", + "description": "At least one requested new item is not available." + } + ], + "rules": [ + { + "id": "all-new-items-available", + "description": "Every requested new item is available in the catalog.", + "when": { + "op": "every", + "path": "/modification/items", + "where": { + "op": "fact", + "path": "/availability", + "operator": "equals", + "value": "available" + } + }, + "outcome": "modification-allowed", + "onUnknown": "escalate" + } + ], + "fallbackOutcome": "modification-refused" +} \ No newline at end of file diff --git a/internal/result/result.go b/internal/result/result.go index 0fd987e..63ab660 100644 --- a/internal/result/result.go +++ b/internal/result/result.go @@ -245,21 +245,36 @@ type TraceEntry struct { Skipped bool `json:"skipped,omitempty"` } +// DraftPrototype marks an evaluation that ran under a draft-RFC grammar +// extension rather than a published JPS version. It is present exactly when the +// caller opted into such a grammar, and it says in band what the operators are +// and that the pack carrying them is not valid under the specification the rest +// of the payload names. A consumer that ignores it is reading a disposition +// produced by operators no JPS version defines. +type DraftPrototype struct { + RFC string `json:"rfc"` + Status string `json:"status"` + Operators []string `json:"operators"` + PackValidUnderSpecVersion bool `json:"packValidUnderSpecVersion"` + Note string `json:"note"` +} + // Evaluation is the experimental-evaluation envelope. Experimental and // ConformanceClaim are always set so no consumer can read the payload as a // standard: this surface may change or be removed without compatibility // promise (ADR-0007). type Evaluation struct { - OutputVersion string `json:"outputVersion"` - Tool Tool `json:"tool"` - Command string `json:"command"` - Status string `json:"status"` - Experimental bool `json:"experimental"` - ConformanceClaim string `json:"conformanceClaim"` - SpecVersion string `json:"specVersion"` - Disposition Disposition `json:"disposition"` - Trace []TraceEntry `json:"trace"` - Artifact *Artifact `json:"artifact,omitempty"` + OutputVersion string `json:"outputVersion"` + Tool Tool `json:"tool"` + Command string `json:"command"` + Status string `json:"status"` + Experimental bool `json:"experimental"` + ConformanceClaim string `json:"conformanceClaim"` + SpecVersion string `json:"specVersion"` + DraftPrototype *DraftPrototype `json:"draftPrototype,omitempty"` + Disposition Disposition `json:"disposition"` + Trace []TraceEntry `json:"trace"` + Artifact *Artifact `json:"artifact,omitempty"` } type OperationalError struct {