feat: verification — inline guardrails + async judge (Phase 5)#7
Merged
Conversation
Add the in-path correctness layer: deterministic guardrails that flag (or block) a bad response before it returns, plus an async, sampled local-Ollama judge whose 1-5 verdict is attached to the trace out of band. - verify/policy.ts: PII (email, Luhn-checked cards, SSN, phone, IPv4 with octet validation, API-key tokens) + a configurable content blocklist + refusal detection. Returns category codes only, never the matched value. - verify/schema.ts: a small deterministic JSON-Schema subset validator. - verify/guardrails.ts: JSON-validity + schema match (when JSON is requested) + policy/PII, aggregated to pass/flag/block. Fails closed — a check that throws blocks, never passes. - verify/judge.ts: local-Ollama judge; the response is wrapped as delimited, untrusted DATA so it can't talk the judge into a pass; verdict parsed defensively (clamped 1-5). - verify/verifier.ts: orchestrates sync inspect() + fire-and-forget sampled scheduleJudge() (attaches via store.attachVerdict) + drain() for shutdown. - verify/regression.ts + GET /regression: group judge scores by (promptFingerprint, model) to compare a prompt's quality across models. - Traces gain guardrailStatus/guardrailViolations/judgeScore/judgeReason/ judgeError/promptFingerprint (trace.ts, both stores + attachVerdict, query API). New GuardrailBlockedError (422). Config: VERIFY_ENABLED/ GUARDRAILS_BLOCK/JUDGE_ENABLED/JUDGE_MODEL/JUDGE_SAMPLE_RATE + a guardrails block in sentinel.config.json. Inline guardrails are non-streaming only; streams are judged from buffered output. No verifier configured = unchanged. Docs: README "Verification", .env.example, config example, SR-005 (ticks prompt-injection/judge-integrity boxes 2.1-2.3). 190 tests, 98% coverage; pnpm verify green.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Phase 5 — Verification: Guardrails + Judge (C6)
Adds the in-path correctness layer that is Sentinel's wedge vs LiteLLM/Helicone/Langfuse: it can flag or block a bad response before it returns, and score sampled responses out of band.
What's new
verify/policy.ts): email, Luhn-checked credit cards, SSN, phone, IPv4 (octet-validated), API-key-like tokens, a configurable content blocklist, and refusal detection. Aggregated topass | flag | block. A check that throws fails closed (blocks), never passes. Traces store violation category codes only (e.g.pii.email) — never the matched value.verify/judge.tscalls a local Ollama model; the response is wrapped behind delimiters as untrusted DATA so it can't talk the judge into a pass, and the{score 1–5, reason}verdict is parsed defensively. It runs out of band (never adds response latency); a judge failure is recorded as "unscored", never a pass.GET /regressiongroups judge scores by(prompt, model)so you can compare a prompt's quality across models/versions.guardrailStatus,guardrailViolations,judgeScore,judgeReason,judgeError,promptFingerprint, queryable viaGET /traces?guardrailStatus=block/?judgeScoreMax=2/?promptFingerprint=…. The async judge attaches via a newstore.attachVerdict(spanId, …)after the span is exported.Decisions (locked with maintainer)
GUARDRAILS_BLOCK=trueto return 422; guardrail execution errors always fail closed./regressioncomputed from the trace store — no second persistence layer).Architecture notes
SimpleSpanProcessorexport, then updates the row viaattachVerdict.VERIFY_ENABLED/JUDGE_ENABLED, both off by default). New modules live underverify/; theProviderRegistry/handler contracts are untouched.Security — SR-005 (area 2)
Ticks prompt-injection / judge-integrity boxes 2.1–2.3: content treated as data not instructions; judge prompt isolates the response and degrades to "unscored"; guardrail policy comes from env/config only (request content can't disable it); traces never persist the matched PII/secret.
Tests / verification
pnpm verifygreen. 190 tests, 98% coverage (≥90% gate). New unit suites for policy, schema, guardrails, judge (prompt isolation + defensive parse), verifier (sampling, attach, fail-open, drain) and regression; server integration tests cover guardrail flag (trace) / block (422), judge verdict + "unscored" on failure, streamed-response judging, and the/regressionendpoint.