From cfbccec6469d6b78a34e97f96c6146d8e8afcdac Mon Sep 17 00:00:00 2001 From: Jaixii Date: Sun, 4 Oct 2026 02:47:32 -0400 Subject: [PATCH 01/22] Align managed Jev clients with individual access and fixed advisory operations --- .claude-plugin/skill-assets.sha256 | 2 +- README.md | 6 +- docs/CONFIGURATION.md | 25 +- docs/HOSTED_PLANS.md | 74 +- docs/MCP_TOOLS.md | 26 +- docs/RELEASE_1_7_9.md | 5 +- .../offline-fixtures-v135.json | 696 ++++++++++++++++++ .../offline-fixtures-v135.json.sha256 | 1 + engraphis/backends/jev_decision.py | 7 +- engraphis/backends/jev_query_planner.py | 1 + engraphis/backends/jev_transport.py | 84 ++- engraphis/mcp_server.py | 7 +- skills/engraphis-memory/references/TOOLS.md | 29 +- tests/test_jev_configuration.py | 12 +- tests/test_jev_origin_binding.py | 13 +- tests/test_jev_transport.py | 194 ++++- tests/test_mcp_jev_payloads.py | 165 ++++- tests/test_mcp_server.py | 2 +- tests/test_smart_mcp_gateway.py | 48 +- 19 files changed, 1333 insertions(+), 64 deletions(-) create mode 100644 docs/benchmark-evidence/offline-fixtures-v135.json create mode 100644 docs/benchmark-evidence/offline-fixtures-v135.json.sha256 diff --git a/.claude-plugin/skill-assets.sha256 b/.claude-plugin/skill-assets.sha256 index 1878e1c3..91fd5ed1 100644 --- a/.claude-plugin/skill-assets.sha256 +++ b/.claude-plugin/skill-assets.sha256 @@ -3,4 +3,4 @@ c5d0c26f28c9ee14092f9deaf24c98dd8bef49d971fef2b7a537ffb1ab9f2887 .claude-plugin aeee7a94671ceb306fe2d24c5acc9f2d96ad8a8e7410536566799eea6265f080 skills/engraphis-memory/SKILL.md 055655db84af07561d002f0c69744313d8413c39f3e873f941f0fa0b1e76dc66 skills/engraphis-memory/references/CONVENTIONS.md 9d090a03f5b3f36a34d91f66b72c3844591f6915755ac3a6c6ba5f1b16977de5 skills/engraphis-memory/references/SCOPING.md -3e669356f9cf2cc60d70ccdd4dd1ebdca47f91c30b0c9196f4016a0887d2c395 skills/engraphis-memory/references/TOOLS.md +5827b5300649aba4d46f954dce560d098c3d0c5e82133d4b77e85a254f11c9a8 skills/engraphis-memory/references/TOOLS.md diff --git a/README.md b/README.md index 549036e7..a84d896b 100644 --- a/README.md +++ b/README.md @@ -64,11 +64,13 @@ The payload figure is a serialized JSON-shape estimate, not an MCP transport mea ## Optional Jev assistance -In the local dashboard, choose **Review with Jev** on a memory to check an evidence claim or compare two selected memories for a possible contradiction. Remote review sends only the selected, bounded excerpts and claim after per-call consent and classification; secret-classified memories are blocked. Agent workflows can also use the existing MCP decision tool for command and completion reviews. On Smart MCP, setting `allow_remote=true` and a `public` or `internal` classification on `engraphis_recall_context` opts that call into Jev route selection and remote processing. Classic MCP recall also requires `planning="auto"` and `jev_assisted=true`. For route planning, Jev receives the original query and bounded deterministic routes, not recalled memory bodies, and cannot change scope, time, type, or trust filters. Local behavior is the default. Jev does not modify memories, change grounded-recall requirements, authorize command execution, or certify task completion. Uncertain, malformed, or failed requests are labeled and fall back to deterministic behavior. Direct BYOK use is separate and may incur provider charges. +In the local dashboard, choose **Review with Jev** on a memory to check an evidence claim or compare two selected memories for a possible contradiction. Remote review sends only the selected, bounded excerpts and claim after per-call consent and classification; secret-classified memories are blocked. Managed Jev supports four Engraphis advisory workflows: command review, contradiction classification, evidence support, and completion evidence. Each requires its concrete context and fixed questions; arbitrary `custom` questions and `query_planning` are rejected before managed credential refresh or network requests. Local behavior is the default. Jev does not modify memories, change grounded-recall requirements, authorize command execution, or certify task completion. Uncertain, malformed, or failed requests are labeled and fall back to deterministic behavior. Direct BYOK use is an explicit, separate choice and may incur provider charges; managed and `auto` never silently switch to it. + +Experimental Jev recall route selection is available only with explicit BYOK. On Smart MCP, set `allow_remote=true` and a `public` or `internal` classification on `engraphis_recall_context`; Classic recall also requires `planning="auto"` and `jev_assisted=true`. The provider receives the original query and bounded deterministic routes, not recalled memory bodies, and cannot change scope, time, type, or trust filters. Managed route-planning requests retain deterministic local order with a visible unsupported-operation fallback. Jev-assisted recall planning is experimental. In an exploratory comparison using 40 public synthetic tasks, Jev selected a route on all 40 calls but did not change nDCG@5, Recall@5, or answer-token coverage. This fixture does not establish a benefit on held-out user workloads, so no retrieval-quality improvement is claimed. -Managed Jev is currently `not_yet_available` pending release acceptance. When released and enabled, Pro and Team will include Jev use at no additional charge within a finite allowance; Team usage is pooled. No fixed public quota is advertised. Once available, the account portal will show the current allowance, usage, and availability. See [hosted plans and Jev details](https://github.com/Coding-Dev-Tools/engraphis/blob/fee9d0c150c250632d8e0c0ee86c1325c9e1ee78/docs/HOSTED_PLANS.md#included-system-1-decision-engine-jev). +Managed Jev is currently `not_yet_available` pending release acceptance. When released and enabled, each paid Pro user and each paid Team named seat, including viewers, receives **100 evaluated questions per rolling hour, 1,000 per rolling five hours, and 2,000 per rolling 24 hours** at no additional managed charge and without a personal provider key. Active legitimate trial/test entitlements receive the same limits. All three caps apply independently to the individual; there is no monthly or Team pool. A command review evaluates two questions and consumes two uses. The unchanged production fleet guard of **100 questions/day** conflicts with this allowance at launch and can block requests earlier. Provider terms, live acceptance, and quality evaluation remain release gates; synthetic fixtures do not establish model accuracy. See [hosted plans and Jev details](https://github.com/Coding-Dev-Tools/engraphis/blob/fee9d0c150c250632d8e0c0ee86c1325c9e1ee78/docs/HOSTED_PLANS.md#included-system-1-decision-engine-jev). ## Guides diff --git a/docs/CONFIGURATION.md b/docs/CONFIGURATION.md index d7bfbb18..53c35b4e 100644 --- a/docs/CONFIGURATION.md +++ b/docs/CONFIGURATION.md @@ -44,7 +44,7 @@ control-plane, relay, compute, and worker implementations are private services. | `ENGRAPHIS_LLM_MODEL` | `gpt-4o-mini` | Model name (provider-specific) | | `ENGRAPHIS_LLM_API_KEY` | Not set | API key for chat/synthesis, `llm` / `llm_structured` extraction, and structured consolidation | | `ENGRAPHIS_LLM_BASE_URL` | Not set | Base URL for openrouter / custom OpenAI-compatible endpoints | -| `ENGRAPHIS_DECISION_BACKEND` | `none` | `none` or `local` keeps advisory decisions local; `managed` uses the saved Cloud session and included allowance; `auto` selects managed when configured and never switches to BYOK; explicit `byok` uses a personal TypeSafe key. Legacy `typesafe`, `jev`, and `system1` mean BYOK. Remote calls also require per-call consent. | +| `ENGRAPHIS_DECISION_BACKEND` | `none` | `none` or `local` keeps advisory decisions local; `managed` uses the saved Cloud session and included individual allowance for four concrete workflows; `auto` selects managed when configured and never switches to BYOK; explicit `byok` uses a personal TypeSafe key and may incur provider charges. Managed custom questions and query planning are unsupported. Legacy `typesafe`, `jev`, and `system1` mean BYOK. Remote calls also require per-call consent. | | `ENGRAPHIS_DECISION_MODEL` | `jev-1.13.0` | Pinned model accepted by the Jev transport; other model identifiers are rejected. | | `TYPESAFE_API_KEY` | Not set | Personal credential for explicit BYOK decisions; `JEV_API_KEY` is a fallback alias. Managed decisions use the saved Cloud session instead. | | `TYPESAFE_BASE_URL` | `https://api.typesafe.ai` | Direct BYOK provider origin; does not change the managed session's bound Cloud control origin. | @@ -62,6 +62,29 @@ control-plane, relay, compute, and worker implementations are private services. | `ENGRAPHIS_CLOUD_ACCESS_TOKEN` | Not set | Optional short-lived access token for ephemeral jobs | | `ENGRAPHIS_MANAGED_COMPUTE_CONSENT` | *(unset)* | Deny-only operator override: `0` pauses readable managed processing. A truthy value cannot grant approval. Each workspace requires explicit confirmation in Manage → Settings; encrypted sync is separate | +Managed Jev requires a current paid Pro user or paid Team named seat, including viewers, or an +active legitimate trial/test entitlement. Each individual receives 100 evaluated questions per +rolling hour, 1,000 per rolling five hours, and 2,000 per rolling 24 hours. All three caps apply; +there is no monthly allowance or Team pool, no extra managed charge, and no personal provider +key requirement. Each batch question is one use; a command review evaluates two questions. +These are service-enforced limits, not client configuration overrides. The unchanged production +fleet guard of 100 questions/day remains a launch conflict and can reject requests earlier. +Configuration does not enable the service or satisfy provider terms, live acceptance, or quality +gates. See [hosted plans](HOSTED_PLANS.md#included-system-1-decision-engine-jev). + +The four managed workflows require a command, a pair of facts, query/evidence, or goal/output +context and their fixed question schemas. Arbitrary `custom` and `query_planning` payloads are +rejected before credential refresh or network requests. Experimental recall route planning +requires explicit BYOK; managed failure never silently selects a personal key. Remote consent +and `public` or `internal` classification remain mandatory, and `offline_mode=true` prevents +remote requests. Viewers can use direct Classic advisory decisions; generic Smart stateful +execution still requires admin and the private Team tool catalog is unchanged. + +Managed transport callers can supply `request_key` for explicit retry deduplication. A duplicate +admitted key returns 409 without an additional provider call or usage increment; there is no +stored answer replay or automatic retry. Admitted errors remain counted. This parameter is not +an environment setting or an MCP/dashboard field. + The optional cross-encoder reranker is model- and hardware-dependent. Treat its quality and latency as deployment-specific until a versioned model identity, exact configuration, and reproducible evaluation artifact are available for the comparison being reported. diff --git a/docs/HOSTED_PLANS.md b/docs/HOSTED_PLANS.md index e9b5a187..e87945a4 100644 --- a/docs/HOSTED_PLANS.md +++ b/docs/HOSTED_PLANS.md @@ -16,7 +16,7 @@ implementations are not part of this repository. | Local dashboard, memory engine, and MCP tools | Yes | Yes | Yes | | Local version history, graph, and manual consolidation | Yes | Yes | Yes | | Local workspace export | Yes | Yes | Yes | -| Advisory Jev decisions | Local heuristics; optional BYOK | Included managed allowance after service acceptance | Included pooled allowance after service acceptance | +| Advisory Jev decisions | Local heuristics; optional BYOK | Included individual allowance after service acceptance | Included allowance per named seat after service acceptance | | Hosted Cloud Sync, Analytics, and managed automation | | Yes | Yes | | Private account and billing support | | Yes | Yes | | Hosted multi-user dashboard, roles, seats, and audit export | | | Yes | @@ -26,15 +26,36 @@ Start or manage a hosted subscription in the [Engraphis account portal](https:// ## Included System 1 Decision Engine (Jev) -After release acceptance and service enablement, Pro and Team include managed Jev decisions at -no additional charge within a finite allowance. Team usage is pooled across the organization. -There is no fixed public quota. After acceptance, the account portal reports current allowance, -usage, reset time, pooling, and availability. Requests that may have reached the provider remain -counted, including failed or interrupted requests, and no overage charge is applied. A shared -daily provider-protection limit can pause service before a plan allowance is exhausted. The -service is currently `not_yet_available` pending release acceptance; client configuration does -not enable it. No latency, accuracy, or cost-saving guarantee is established by configuration or -a successful health check. +After release acceptance and service enablement, every legitimate paid Pro user and every +paid Team named seat, including a viewer seat, includes managed Jev at no additional charge +and without a personal provider API key. Active legitimate trial/test entitlements receive +the same limits. Each individual has all three rolling caps: + +| Rolling window | Maximum evaluated questions per individual | +|---|---| +| 1 hour | 100 | +| 5 hours | 1,000 | +| 24 hours | 2,000 | + +Each evaluated question counts as one use, including questions in a batch. The command +workflow evaluates two questions and consumes two uses; the other three workflows each +evaluate one. All three caps are enforced, including the five-hour cap. Usage follows the +individual member rather than a device or seat allocation: reassigning a seat does not reset +that member's recent usage. There is no monthly allowance, organization pool, or unlimited +access. Revocation or expiry prevents further managed access. The account portal reports +individual window usage and availability; capacity returns as admissions age out of each +window rather than at a calendar reset. + +Admission reserves question usage atomically before a provider request. Admitted failed, +interrupted, or subsequently revoked requests remain counted; requests rejected before +admission do not consume an individual use. No overage charge is applied. The production fleet guard remains +**100 questions/day** across the service. That guard is an unresolved launch conflict with +the individual allowances and can pause service before a user's caps are reached. +The service is currently `not_yet_available` pending release acceptance; client configuration +does not enable it. Launch also requires provider-terms review, live acceptance and quality +evaluation. Deterministic fixtures validate integration and fallback behavior, not model +accuracy; no latency, accuracy, or cost-saving guarantee follows from configuration or a +successful health check. The managed transport and MCP decision route require client **1.7.9 or newer**. The published 1.7.8 client has an experimental adapter but does not provide this route. @@ -54,6 +75,39 @@ content is rejected. This is permission for the supplied text only, not a standi to upload memory. `offline_mode=true` always prevents remote requests. Known secret patterns are filtered before credential refresh; this does not guarantee arbitrary prose is secret-free. +Managed access is limited to the following concrete Engraphis workflows. The client and +service validate the required context and fixed question schema, rather than trusting a +purpose label alone: + +| Workflow | Required context | Fixed evaluation | +|---|---|---| +| `guard_command` | Command text | Destructive-loss/secret-leak safety and operation category | +| `classify_contradiction` | Existing memory and candidate fact | Supersedes, reinforces, or orthogonal | +| `verify_support` | Query and supplied evidence | Whether the evidence directly supports the query | +| `verify_completion` | Goal and supplied output; optional recent actions | Whether the evidence establishes the goal | + +The context remains caller-supplied: schema validation cannot establish its truth or provenance. +Managed `custom` and `query_planning` requests fail closed before credential refresh or network +requests. Experimental recall route planning and custom remote questions require explicit BYOK; +managed failure never selects BYOK automatically. Classic/direct advisory decisions are available +to authorized viewers without granting memory writes or administration. The generic Smart +`engraphis_execute_action` gateway still requires admin because it can execute stateful actions; +the private hosted Team tool catalog is unchanged. + +The managed transport accepts an optional opaque `request_key` (16–64 ASCII letters, digits, +underscores or hyphens) and generates a new key when it is omitted. A caller can preserve the +key for an explicit retry after a lost reply. A duplicate admitted key for the same organization +and member receives a 409 outcome without another provider request or question increment, +including when the supplied content changes. No answer is stored for replay and the transport +does not automatically retry. A new key is a new admission and can consume another use. +`request_key` is a transport parameter, not an MCP or dashboard field. + +Existing clients can continue sending the legacy managed envelope for these four canonical +workflows and fixed questions; the service also accepts its typed operation envelope. Legacy +requests without a key retain their original single-call behavior and lack retry deduplication. +Legacy arbitrary custom payloads are no longer admitted by managed access; choose local behavior +or explicit BYOK if that separate capability is intended. + The model is pinned to `jev-1.13.0`. Choice/score confidence is provider-supplied; Noul confidence is explicitly labelled derived decisiveness, not measured calibration. Uncertain, malformed, unavailable and fallback results remain distinguishable. Local heuristic confidence diff --git a/docs/MCP_TOOLS.md b/docs/MCP_TOOLS.md index c5615e10..df5ea7eb 100644 --- a/docs/MCP_TOOLS.md +++ b/docs/MCP_TOOLS.md @@ -39,7 +39,8 @@ namesakes; advanced controls are discoverable rather than routine: `full`. It preserves the same selected text and whitespace; it does not apply an additional summary or promise extra token savings. Source IDs remain in `sources`. -Jev-assisted recall planning is opt-in. On the Smart context tool, set `allow_remote=true` and +Jev-assisted recall planning requires explicit BYOK and is opt-in. On the Smart context tool, +set `allow_remote=true` and `data_classification="public"` or `"internal"`; that call both enables bounded route selection and gives per-call consent for remote processing. The default stays deterministic and local. Classic recall tools additionally require `planning="auto"` and `jev_assisted=true`. @@ -47,6 +48,9 @@ The provider receives the query and bounded local routes, not recalled memory bo change scope, time, type, or trust filters; uncertainty and failures retain deterministic route order and appear in `planning_advisory`. Leaving the Jev controls at their defaults keeps local deterministic behavior. +Managed access does not admit `query_planning`: it fails closed before credential refresh or +network requests, retaining deterministic route order with a visible fallback. Neither `managed` +nor `auto` silently switches to BYOK. No user profile choice or tool switching is required. The dashboard `/mcp` endpoint and @@ -201,6 +205,26 @@ Decision inputs must be nonblank for the selected kind: `guard_command` needs `s and `query`; `verify_completion` needs `state` and `goal`, with optional `recent_actions`. `custom` accepts either `state` or `question`. Missing required input returns `invalid_request` before backend lookup, with unknown/null conclusions and no remote allowance consumed. +Managed access admits only the four concrete workflows above with their fixed questions and +required context. A purpose label cannot authorize arbitrary question schemas. Managed `custom` +requests return `managed_operation_unsupported` without a credential refresh or network request; +custom remote questions require explicit BYOK. Local custom fallback remains available. +For managed usage, `guard_command` counts as two evaluated questions and the other three +workflows count as one each. Every paid Pro individual and paid Team named seat, including +viewers, and active legitimate trial/test entitlements has independent rolling caps of +100 questions/1 hour, 1,000/5 hours, and 2,000/24 hours. All three apply; usage is neither monthly +nor pooled. No extra managed charge or personal provider key is required. +Admitted errors remain counted. Managed transport callers can preserve an explicit `request_key` +for a retry; duplicate admitted keys return 409 without a second provider call or usage increment. +No answer is stored for replay, and no automatic retry occurs. The MCP tool does not expose a +`request_key` parameter. + +The direct Classic `engraphis_decide` tool permits authorized viewers and remains advisory. +It does not grant memory writes or administration. Smart discovery still classifies the action +as stateful because it can consume allowance, so `engraphis_execute_action` retains its admin +requirement. The private hosted Team tool catalog is unchanged. Managed service release +acceptance and live quality evaluation remain required; the unchanged production fleet guard of +100 questions/day can block requests before the individual caps and remains a launch conflict. Command decisions always return `allow_auto=false` and `escalate_to_user=true`, including successful remote answers. Provider probability and category are advice, not shell authorization. Local command labels are coarse: only one simple inspection command, without chaining, pipes, diff --git a/docs/RELEASE_1_7_9.md b/docs/RELEASE_1_7_9.md index 7d01c5f5..9de3339b 100644 --- a/docs/RELEASE_1_7_9.md +++ b/docs/RELEASE_1_7_9.md @@ -36,8 +36,9 @@ does not ship the managed transport or the registered MCP decision tool. `offline_mode=true` prevents remote execution. Pattern filtering cannot detect every secret in arbitrary prose. -The service enforces current membership, entitlement, monthly allowance and a -separate fleet cap. A local success, fallback, health response or configured +The service enforces current membership, entitlement, and individual rolling limits +of 100 questions per hour, 1,000 per five hours, and 2,000 per 24 hours. The separate +100-question/day fleet cap remains a launch conflict. A local success, fallback, health response or configured backend does not establish a successful managed provider request. The account portal reports availability and usage. Jev advice does not authorize actions or replace deterministic checks. diff --git a/docs/benchmark-evidence/offline-fixtures-v135.json b/docs/benchmark-evidence/offline-fixtures-v135.json new file mode 100644 index 00000000..9e512a7e --- /dev/null +++ b/docs/benchmark-evidence/offline-fixtures-v135.json @@ -0,0 +1,696 @@ +{ + "environment": { + "embedding": "deterministic", + "numpy": "2.5.3", + "platform": "win32", + "python": "3.12.14", + "vector_backend": "numpy" + }, + "generated_on": "2026-10-04", + "privacy": { + "contains_answers": false, + "contains_customer_data": false, + "contains_per_record_fingerprints": false, + "contains_prompts": false, + "contains_raw_questions": false + }, + "runs": [ + { + "boundary": "Deterministic offline retrieval fixture; normalized-character token estimator; not external QA or provider billing.", + "command": "python -m eval.chunking_eval --dataset eval/datasets/longdoc.jsonl --k 5", + "config_digest": "c1c8196aa7e1568ef3844a9fb2d76b87f342c39108e32d6ad144b885a76143b8", + "config_digest_method": "sha256(UTF-8 exact command)", + "id": "offline-chunking", + "result": { + "chunked": { + "max_stored_tokens": 59, + "mean_context_tokens": 214.3, + "mean_evidence_tokens": 42.4, + "memories": 24, + "recall_at_k": 1.0 + }, + "context_reduction_pct": 71.1, + "documents": 6, + "k": 5, + "questions": 18, + "token_counter": "engraphis.chars4.v1", + "whole": { + "max_stored_tokens": 213, + "mean_context_tokens": 740.3, + "mean_evidence_tokens": 162.2, + "memories": 6, + "recall_at_k": 1.0 + } + } + }, + { + "boundary": "Deterministic offline CodeMem fixture; serialized JSON-shape payload proxies, not MCP transport responses, provider billing, or latency claims.", + "command": "python -m eval.performance --dataset eval/datasets/codemem.jsonl --k 5 --iterations 10 --json", + "config_digest": "bbe4aca81e58d4830e50a8fc7729a1d15b71d97a6299bccd79432b7f119677d7", + "config_digest_method": "sha256(UTF-8 exact command)", + "id": "offline-performance", + "result": { + "answer_token_recall": 1.0, + "compact_serialized_payload_tokens": 11138, + "dataset_cases": 14, + "full_serialized_payload_tokens": 24590, + "hit_at_k": 1.0, + "k": 5, + "max_context_tokens": 108, + "mean_context_tokens": 85.38, + "memories": 44, + "packed_quality": { + "answer_token_recall": 1.0, + "hit_at_k": 1.0, + "recall_at_k": 1.0, + "sample_count": 26 + }, + "payload_boundary": { + "kind": "serialized_json_shape_proxy", + "mcp_envelope_serialized": false, + "token_counter": "engraphis.regex.v1", + "transport_measured": false + }, + "quality_scope": { + "packed": "packed_quality fields score only chunks admitted to reader context", + "retrieved": "legacy quality fields score all candidate chunks returned before context packing" + }, + "questions": 26, + "recall_at_k": 1.0, + "saved_serialized_payload_tokens": 13452, + "serialized_payload_savings_ratio": 0.5471, + "timed_recalls": 260, + "token_budget": 1500, + "token_counter": "engraphis.regex.v1" + } + }, + { + "boundary": "Deterministic offline support/abstention fixture; not a frontier-model answer-quality score.", + "command": "python -m eval.grounded", + "config_digest": "590442e51e3642c10489165759919dc86ffac62c182937330c153e7f8d5fc26f", + "config_digest_method": "sha256(UTF-8 exact command)", + "id": "offline-grounded", + "result": { + "abstained": 6, + "answerable": 5, + "decision_accuracy": 1.0, + "grounded": 5, + "off_topic": 6, + "quarantine_hits": 1, + "quarantined": 1 + } + } + ], + "schema": "engraphis-public-offline-fixtures/v1", + "suite": { + "digest": "cf6d3551ad6c8bcae12b509512849fd33788db45e5fc5a1db7a3fcaae01c370b", + "digest_method": "sha256(canonical compact JSON mapping each sorted path to its file SHA-256)", + "files": { + "engraphis/__init__.py": "f24241e8da2ede9ccba54ce4bf7531a800b55c29b2469e5de93bd423cffbf0be", + "engraphis/ai_context.py": "4dfd5d39eb95d05c591d1981e9e855eced73232f53efd9fe9966e0e08957d030", + "engraphis/app.py": "44d68ad8c0ff46baed01978c9b04e40b8be5031609d5a69e205b7f05da3777fb", + "engraphis/backends/__init__.py": "a9f22b9278362904166614081f1df78469d453601b298ce4e8afdba8a3722b25", + "engraphis/backends/codegraph.py": "83e723a91068d23694092fbe00157fcb2597061d453eb36f955bf55bc5b4d35e", + "engraphis/backends/embedder_api.py": "56a6bceea4f757325dcea987b0339e41d3875634b1ae5103f0537a358cf5878b", + "engraphis/backends/embedder_deterministic.py": "ec8b23de7e7e8273416125f5876ca96f55e4ae7881841bae55783ab0ba9130ad", + "engraphis/backends/embedder_st.py": "e1c20fd980e07060387e3f9fa37fe02a916959fc8abce4e6067a62699c726de1", + "engraphis/backends/encrypted_db.py": "25f6c1480d296a88f317213700a8b3c81e2732399bfac464b0d893465b25e846", + "engraphis/backends/extractor.py": "f2e3455ab7f14caee1d5b5c4ef071e498e90118f1b0ddcb8510c969583b0fc57", + "engraphis/backends/graph_extractor.py": "88561efa0d3fabc447a0a005b10e36261379d46218cf62d905e6928cd2fda676", + "engraphis/backends/jev_decision.py": "fdab1ae9a7ac3bc19632f23a58e53ecce97b6c1df2d4edd1fef06d4e59800888", + "engraphis/backends/jev_query_planner.py": "8c9b2859ef59868559a6b14a854a3280ed0a4f7aedd2fd4e5699d661e45e948e", + "engraphis/backends/jev_transport.py": "b55b37e08813b0419602a55445846cf838c4a0987b147001274dd46e161466be", + "engraphis/backends/model_source.py": "8c3c7681f95214a2bbabd8de222e5ee11f42fe13402d27365654ae75fb363d4e", + "engraphis/backends/postgres_schema.py": "8468578c3add701d30d5eaa36d768ded2375d116e55f1836e09f6107a07a267e", + "engraphis/backends/query_planner.py": "bbdd77afc9b5523421b85b2ae63c8da7f5a7b777265450e0d21708a83e7bb23c", + "engraphis/backends/reranker.py": "747761d6cbfa421388974bcfd98d844f92391d80f4bf6a4feca00b0c7a6908ca", + "engraphis/backends/resources.py": "47cc867c3aecc8bd95fa284bc5bb04715f3339c19a0a11512973ef6171c95944", + "engraphis/backends/retention.py": "381d9371e3951d762f8b55eb54711de5697642acb39de99a714f246c059ecbd0", + "engraphis/backends/sync_folder.py": "e4f70a92a17f6a365910670df041e6e3ca421d44ada2827917cd66b4dc067bfa", + "engraphis/backends/sync_relay.py": "b8b9ad265453aba17ba7c27a355e12a793469b3e44cb217943c6fad9382a3006", + "engraphis/backends/vector_numpy.py": "c598831bea547824cfe08844816fa79857d3617cb0631238f95dad955a424f72", + "engraphis/backends/vector_sqlitevec.py": "6148e14ceaacc19239b64c642a3fba0e98797c78cec356210157afadf475a08b", + "engraphis/build_info.py": "624c22471e56d4c4047160808c4245488292af611564d1a63ca437605bbb414f", + "engraphis/classic_assets/__init__.py": "a7c1d52b285e3faa670ce231814b5758754aa0fbd05e1428e74c20c3ec51a4f1", + "engraphis/cloud_authz.py": "e80500579cb3a1d5fbf30814dc94e3e3967e50b311e8ed2fa56afcc13f7eb565", + "engraphis/cloud_features.py": "90e876f8993d99f01760cd01ae9fb63140e3f65036706e8fb0c8224560416330", + "engraphis/cloud_session.py": "cc6ab27c6af69b5c676e721ab06c520730e205f5bbffe27818650edfb71d4b63", + "engraphis/commercial.py": "184f312066a9e682e51a0abeff042f1c0e8eed2d47470157b23930b5a17633aa", + "engraphis/config.py": "9af2072734f0e0fb409ce3cd422d5a4cb9a4854c9a92d2e03d7e680e119a8fb9", + "engraphis/core/__init__.py": "dd5143729c3939237f04636f437032b1f2d3a5f7d82c91bbc2a5a283c3f0ebaa", + "engraphis/core/adaptive_context.py": "cc5ce48109bb0d5230a5b2b8424b829c853feec5b5d5b82596413f2279b0c9f9", + "engraphis/core/browsing.py": "cfae752d52ef51b17c4ffbe44dde62d5b0e1ed0ca34ec0e1788f3991d94bf05f", + "engraphis/core/codegraph_export.py": "4641074258d7b23498f92dd45053a0fbb111863eaad2001c08e5e3c2dc2fd54f", + "engraphis/core/conflicts.py": "28530be25a4af0bffd8f609b965b33ca7f93199a70887789b2148fda8a61a486", + "engraphis/core/consolidate.py": "f66eedfe08319a64b761261ccf1c99b163eba564d6d4060534bf8c575aecf232", + "engraphis/core/context.py": "8cc15746e4de88bce7d34db8c11dfaa545f17305788e2167a286179171717fb9", + "engraphis/core/diagnostics.py": "5ba449bdf5087d8ccb5808379596e695c8f405da497372e99f2bac509f96e8e4", + "engraphis/core/documents.py": "84385db39ba44e06b58b4b26dbf954228ff4abed7230f28a7280166fa6457861", + "engraphis/core/engine.py": "6a7f03b074caacb654be354b3cd10b158219bfa43c82271178b8dd50dbbddac2", + "engraphis/core/evidence.py": "97912b52a3d22de909218f83c09c92572ed695d7b3c2117e5330271c6d84a7bd", + "engraphis/core/fsutil.py": "6db770fa8bd3e1a57dfa70eb8e8bc46d48c2dc53ead1b0b43eeecf085ef58cfc", + "engraphis/core/graph_layers.py": "64d74ab01c77119f6343ba6f1d6a84f9653f1a5d34d47ce7966f3ac31b29d2ea", + "engraphis/core/graph_policy.py": "ec5b373d01adb2de87df31d9f543130018e9a73faaaed239a184f14a32646615", + "engraphis/core/graph_scene.py": "5bfb4dadd90aff6be89dad0384aabec533c04c9a5225d3e9d43a468152569d75", + "engraphis/core/graphrank.py": "1279a58396104d3f906bfd5ec75b32efedfefe52201467bf19d80be3517017a5", + "engraphis/core/grounded.py": "ba38c3a47d188c4f936531de10df8698f9593c7ce14446f54959b092eb637787", + "engraphis/core/ids.py": "e47eeadfaf560bc7638e1879fc491fa82d981cd42b9e4b05ef87033b2d7dfa60", + "engraphis/core/interfaces.py": "05d65c1919f3c23592f48e5f2820bbe857923fb65f5146727636eba51941271c", + "engraphis/core/mutations.py": "dbb46a97686994e1652b2698e50c3309ff428d7258d6e8e53cbad7bb55b459aa", + "engraphis/core/obsidian.py": "991267c153cb7c4c40f7fe8f50aa71688892e250aaaf383b22c9e5910dc263b7", + "engraphis/core/poisoning.py": "5bc67169ee8032f3777f2d3969dcf71821437bd473ce4f917c4b41a50e845fb6", + "engraphis/core/query_planner.py": "249062d67392ab7c203cc71e9040e99bee91bf570604e90949149a93cb652120", + "engraphis/core/read_snapshots.py": "be08e63a88bd38ed91d61b28657a65201c38994856db2798b209a73151dd202a", + "engraphis/core/recall.py": "f4652b7eadb310ce19a8f9418674061848bfdf847b0b73054b8f5d08f87eee1a", + "engraphis/core/relocation.py": "86bbd292b374539b480dddc1a9ac19292fbc17ff9fa8e1fe6ff69c91ee7c8bb9", + "engraphis/core/resolve.py": "f01a6f55e44320ab04b97e516342f20155668863b2fc4765305e066d87586524", + "engraphis/core/retention_policy.py": "864c03bdb6e743cd0002c706de471e920f1fa1f1a9918ab343ef4c2042b47429", + "engraphis/core/retrieval_policy.py": "d169eb442115bd06c1e1da776fa6c1849e0795e658edd732a8d6821326b46b03", + "engraphis/core/savings.py": "cfbcfc7e476f4e28028555cd519696e23099f6210cf0b733225832aeaa0bc7dc", + "engraphis/core/schema.py": "ac273d3f0383995be815bbd866f2a36ea1398f30833aed57b8d4459afc87096b", + "engraphis/core/scoring.py": "f5b6ac291edf0968b3de83cb1951a97d5bfd8a8d079199cb2ef95bba0884c89a", + "engraphis/core/secrets.py": "a4835ba06e2616156528df6365ca1aba6cba0c97cdf834a709d2797a371099d2", + "engraphis/core/store.py": "0ba21d11b9a61b105f743aac3850f12638cc51b425ce76fb8d0c1eb27d540647", + "engraphis/core/sync.py": "69f75b50fdb1ec9352f92460c89efeec8d72ca642bd10beb29474a8c62b4f58a", + "engraphis/core/textutil.py": "acd65031729fa5d91d09527b8eb52518b83cfce77a55e6b7ae2d94686e35c1a6", + "engraphis/core/user_model.py": "3147ec8ee7cfd855783f639874b63f331cdc822bd8bd9298cfb326dd18666026", + "engraphis/core/vector_repair.py": "a8c1812de4e3ed288eda3e54a505e136d6ebd3ec296788bcbb8804b11e13cfc8", + "engraphis/core/vector_search.py": "75800052e573e9af01c6fd098eaf8647c3c45cd7eb2601dae05d42359e19b234", + "engraphis/dashboard_app.py": "9305c9c72c43027b2b79c0ce559d398fc232929968d6ea03c37065e3227c8d51", + "engraphis/dashboard_assets/__init__.py": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", + "engraphis/device_connect.py": "c8cd0a22e9fd2d92a65bd74047cc3a8b7159bfc9a297cb639c1c7f1699f8802e", + "engraphis/document_import.py": "94fa0ca340ad0ebd060b143f46657798a81c440a11b82aa46fefa83fb65295dc", + "engraphis/engines/__init__.py": "111232af583889195c5f5a60298e32484348608e81d2bdd822fd3ecd2a33c1e4", + "engraphis/engines/embedder.py": "998b65dd566966bb6581fd9f09cdf46c58a3923ef5df3073ccdaa541153a3581", + "engraphis/engines/ingest.py": "1a5d4b52c13e533864329f9fff11c0f6ebc299ff7093a9d9275e5c9626a39ce2", + "engraphis/engines/intelligence.py": "b561589b98deb98271f104dfec6276aeba13c4815e627259be5f8769f2e7ad82", + "engraphis/engines/recall.py": "f979580d065599c07acbc3add59f71e52d9a186a2104b0e257daa7959ea88d26", + "engraphis/engines/reweight.py": "91ec5815f5d356a7068c36133d24405334450f361f902f99275bed3ccebffd49", + "engraphis/engines/thoughts.py": "4adb9c8a9bcfe736cb42fff9b5ce24631da6ec473d178e83f1c176fde3b8b814", + "engraphis/factory.py": "06735a5acfc784fd1514ae42b24f4a58d041f6eeb040f8610184b458682fe030", + "engraphis/graphdata.py": "f5dc93395f52203c66deaeec8bc58d5c1f990a35a535151e0b9a84cecf7d8430", + "engraphis/hosted_client.py": "5b56b0d2384cf24ecb409090af15f09e437ec8ab5a98bed931384daff46b4f48", + "engraphis/http_deadline.py": "8f41de086f36ab39d4e6461c6284c776e3b123afee845824e22f071ca128c918", + "engraphis/http_security.py": "596981e96741fd47064d03db409605bbb435f062c0c6ef69b040aacdd8763a20", + "engraphis/inspector/__init__.py": "720cac28b8a6019d0a0c53809d5905b7d6eb9d4ecbac767505904c6ec3c39071", + "engraphis/inspector/app.py": "8aefe8a935397dde3c6466e691a7e6a7203ebe42e07aa00c332b733465e59e04", + "engraphis/licensing.py": "7e73c28b0e1c3536e2080a129af614f838d2cae3ec3a48a3d8ed7e5153c9e935", + "engraphis/llm/__init__.py": "f3096d2ddd652b99e6fc0b4c5a8786d9bbaba41b259df0b44ce93b57bde3a8b1", + "engraphis/llm/client.py": "0c84000033d3f85b0e0161f100c271c35b591e1480701455ec65c7d3f111ec42", + "engraphis/local_auth.py": "b0ad3a1926d417a2aaa6c44a7dbf6e51f575290ddfa27875b78a03db176623c3", + "engraphis/logging_setup.py": "f7d2edc756458a852e0401453e9c71b785aba08fa7859aacbdc23fb30dc7982a", + "engraphis/managed_processing.py": "6d33cdfd10800d9552fcfae3c2b071d2b39fe10fb69d1a8e5d9ed6026882197e", + "engraphis/mcp_classic_cli.py": "778122b121a1c654b8f85810fad05d9c8d7799cf13903d7dc274938b51008da0", + "engraphis/mcp_cli.py": "ed2d997438d727180842dc5fb3f6776f5a5d97974f690ee7229264be1ff61957", + "engraphis/mcp_http_cli.py": "5a04bcaae4531a6ea116a827bb65c5df3bd6d07ea336f65ff3ed45034972c396", + "engraphis/mcp_server.py": "c168fcd4850bbab65404e8af7c0af428475218fe4ec830712cc5def7df06c794", + "engraphis/models.py": "6e76e97db0aca3805c6f582ea78cc6e1c0ccd2665e16fd8ab81eb91b94141b51", + "engraphis/netutil.py": "2e0f8a9095f6f31dcb5b96d323f023b9214369d59e1c488125a7d443a55972ff", + "engraphis/observability.py": "a3a6945bf33a0d8e216da56ca7efec0031b82dc64a66cff36f0b2161f5db4981", + "engraphis/obsidian_import.py": "c7cf4b5e993afce3ffb45f01637804719cb9e7cd2461ffe629c055be5975721f", + "engraphis/private_state.py": "7485570efcaee8a1dc00b64ebfdc23178517235ae17609aa78db8bdda7e45fe3", + "engraphis/read_only_api.py": "aeea03282cfdc02074535b3ec6618b5baab0d26725be434911b777db6e7da158", + "engraphis/redirector.py": "5ba964b81f09008c9369180cb49d247519000763274aabc5e046215b12fa641d", + "engraphis/routes/__init__.py": "f0d59080212cfa0d9b50877bca28e5832d1ae917899500d52c68b1c8821231af", + "engraphis/routes/memory.py": "9ca066e1762eeeaafd9790ed57d730efcaceb57742dbe0c830ad8764500ff9e5", + "engraphis/routes/v2_api.py": "b43701f59c56458a7198f543463506aa8dddec18dba8c3fc0b51f232156af1c7", + "engraphis/routes/vault.py": "1a7eee7c1a7c7aa11091042756aa2739a3eb963647020f40584a6a38da1988af", + "engraphis/service.py": "11c509c871b4de4d1617c1eaf1b75d5548716347eb68a12e330d986bdaec9805", + "engraphis/service_context.py": "3de9289f49a977cdc206285ac42a9953a104eb1d1b1bc8ea77febd75c7cd82ab", + "engraphis/static/__init__.py": "1fff4c4e2554e7f5fcf3eace269feba09827524917193a4dd65df95bae64ad1f", + "engraphis/stores/__init__.py": "48ee4326c8f28eecf46f558b7aea21adb779226a7038299017ab90196d5d84be", + "engraphis/stores/graph.py": "ebf603b54cf8450e7c9a7319bd05db2f39fcda8491f5969d6af3e8da61571bc2", + "engraphis/stores/ledger.py": "df5cbb30d977decc0a9c3a2365c9c115cfbb5fe951bae48446436661c80e5d30", + "engraphis/stores/vaults.py": "2c986129b9d1e7aab33e18a3b9a278eb5ad2236895587f69b5816a5d96796bc3", + "engraphis/stores/vectors.py": "45a1baca381fc647548cc89424eb36d5853f7562c275b191b35b99339a190b7d", + "engraphis/update_check.py": "ffbf5ef682fb15177915073ebc0b5eab4dee9092bbf0f58d36a8a54c6605ccf4", + "eval/__init__.py": "639f0c6d9d6aac8ff6dc605a34a0a301058905cc53bff4eaed5912247f0e7c56", + "eval/ablation.py": "16f159dee75d2f96cc42f230c2403fa19ea0bda7091c4823660da553463a194a", + "eval/adversarial_memory_security.py": "35dd8d981bcbad50e9815465be420b05b62a9dea28a78eb6cc1e09ec51c320c4", + "eval/agent_benchmarks.py": "8e88943e45a1b2083b366cbb0329326ef918b119a2366ed395c404c2b7ef98b0", + "eval/benchmark.py": "b71832affdf87d23bc1db7b522a669a7888571b3989b03a235c70f5da9952bd6", + "eval/benchmark_analysis.py": "1200791425d029695fee0b7da1f188a8962f337aa31eb4217d50e4503641d2b3", + "eval/benchmark_campaign.py": "033dbebfb47df5fcd3a6588b29ea4ab5fd44d4387ae45334379278ab30f3e57f", + "eval/campaign_adapters.py": "2bdb51de0dcdfceb1be2a649d84541780ac4d1fc0f9b7585004d87490073213e", + "eval/campaign_api.py": "323be4e1d9520e8047ab54c9cbb158013785402b06ee4e93e55227971c54bf95", + "eval/campaign_candidate.py": "88541840d16c3b7368586ed0ee3566054214b1f594826b746b28d0e407cb812a", + "eval/campaign_continuation.py": "7f77a20ac8f85cd96729bba0cf45872c98a3421af005bd735434b7b416dd7e1f", + "eval/campaign_ledger.py": "ba3079ba541cb1f70929a76f147d2bc5e8b264103a1cd7faad97685e0ecc7790", + "eval/campaign_oracle.py": "9ea5479786efd9caa2b59e612e891a93dd34b500a2b36aa20bcf749fb6d4d48a", + "eval/campaign_storage.py": "ebcd1f4aeceaa5ff9304494c64dd31fe31637ceb107d8cc39fa292e3b14055d4", + "eval/capacity_matrix.py": "8c25bd97754c1d7a8468a687042a3c87cc9c0299cf01c19b80dacfec2be4e552", + "eval/chunking_eval.py": "a16544353940c0a8c40cea3b9932d3399b35ea5994b809b78f5dbe4a952c467f", + "eval/code_agent_ab.py": "d98bba6b77700ff6bf86ff0d2cf518a67e5a2676ef51bbddabbb2ff26e1f3aaa", + "eval/code_arm.py": "d211166fce1b8a4173848e1617873b7e84aadeff43746effe7483c54bbdb6f1d", + "eval/codex_oauth.py": "f235fca482de4201d1850bbfb583765ac5f1a057b4ad7e2d504d036bfc03f391", + "eval/coding_acceptance.py": "4b39cbcb60d7fba503cca597399cf9d04ad9567a43cdb950015ccf552e6a2773", + "eval/coding_corpus.py": "7b5205e8544578fe99d9d9cf6cfc34e40e5d1d049238d502ff7bc6f3c14948f5", + "eval/consolidation_ranking.py": "917b578d4e0bcb929bf1a1a37611acf7716a520c076abf4ade0d8d12a1c455de", + "eval/context_economy.py": "709ac7cc866855f96d7717ab2bea12e8b0d3a140d15fb978a2a929ad085931f2", + "eval/context_efficiency_guardrails.py": "22afd1a6fe17219e74701dc587ec35f569a5bf22bea270944525f51746723f14", + "eval/datasets/codemem.jsonl": "341313023c22850a2e14f02742b571ad1deca824f886a1654a59541304c01f3c", + "eval/datasets/coding_memory_v1/oracles/atlas-green--code_relationships.py": "969aa71235e0ae3cb764b9ee12b789184cedb5fea6fcc0d86e5dcb2c0498fa56", + "eval/datasets/coding_memory_v1/oracles/atlas-green--condition_values.py": "648702646fb62990168e98fba0dc6a27be128876548fa7035a59d71aee9da02d", + "eval/datasets/coding_memory_v1/oracles/atlas-green--corrections.py": "c7132c7fe39b7afbdf11065037369c0f3c38a22e69d8742e8afddfd5785d2e61", + "eval/datasets/coding_memory_v1/oracles/atlas-green--long_documents.py": "65dffb25944cb612872655f12db16af73f7a095e7abcf6a7cc5d31dc91f083b1", + "eval/datasets/coding_memory_v1/oracles/atlas-green--multilingual.py": "ff49874a691bfc30c082695ee6d3c89b9f2e1e5eecf086026110770d8e1b0452", + "eval/datasets/coding_memory_v1/oracles/atlas-green--paraphrases.py": "dbdcf5d7abf136aa5816df8465bfde842284aef5ef002e955cee05ae69d574ca", + "eval/datasets/coding_memory_v1/oracles/atlas-green--poisoning.py": "58b8ca4a37f55c4d112640414dc82620e32e643ab0aa539260884a5ecc50740f", + "eval/datasets/coding_memory_v1/oracles/atlas-green--scope_boundaries.py": "c1e92b10b40004c95abd3dfbada1a38fe0ce56300ecddca8360b51a1e6671458", + "eval/datasets/coding_memory_v1/oracles/atlas-green--temporal_history.py": "a64e4e5a29ce1040cf53e94571491325347f443d72a93f05bcf87c2878665952", + "eval/datasets/coding_memory_v1/oracles/atlas-green--unsupported_questions.py": "c8b415052ea7320e1df3c5475a676d4dd24602c45dbf1868fba6880846e029ee", + "eval/datasets/coding_memory_v1/oracles/atlas-north--code_relationships.py": "bb4305d15a81b4b4ef80b374acb59fee6fc32b213b101dd4b173ac4144d8570d", + "eval/datasets/coding_memory_v1/oracles/atlas-north--condition_values.py": "4bc5b1aff48ef6ad5ece1bd915f6949f72d365bb4815c4d82515b438bc644112", + "eval/datasets/coding_memory_v1/oracles/atlas-north--corrections.py": "7851502a9bc842f840bee1001e42140ee1df08be2730b563b1b197285d14742c", + "eval/datasets/coding_memory_v1/oracles/atlas-north--long_documents.py": "6204123eded20fc0c7aa59b312f8b5bd29aa88a720f7318e32b3c6a17651b57b", + "eval/datasets/coding_memory_v1/oracles/atlas-north--multilingual.py": "5c989b34fdad1e076b61ffb1f7a1371dc635f44a537abc743b765d5f6009004e", + "eval/datasets/coding_memory_v1/oracles/atlas-north--paraphrases.py": "fd189285d0cde9991dea461af9cd18b4203656a69da1298494fccf972ed8a58b", + "eval/datasets/coding_memory_v1/oracles/atlas-north--poisoning.py": "66305d69419736bdbb11a896543933604d898cd5394d94eedda8b92e632a09ac", + "eval/datasets/coding_memory_v1/oracles/atlas-north--scope_boundaries.py": "d9fbbe3924b5178f32f1937485af3b8db242a0e3a08c7bf41a3d2a57f34a8879", + "eval/datasets/coding_memory_v1/oracles/atlas-north--temporal_history.py": "7e7be0d6e558a71d3baad14a8812a6c998ad25e3f52c997a301333e0711f820e", + "eval/datasets/coding_memory_v1/oracles/atlas-north--unsupported_questions.py": "91f574dd678bf244a1fa6f9708e945259bb8ff8e2eb213426be5c3b187be3c3b", + "eval/datasets/coding_memory_v1/oracles/atlas-violet--code_relationships.py": "d0998e13db38d1fae9be6d254431b4fe592f89eb03ee25705f4c70985cd48c89", + "eval/datasets/coding_memory_v1/oracles/atlas-violet--condition_values.py": "f97610aee190dca03b01aa48ffe4aa3f75e099a59fdd867db98f743f4a376729", + "eval/datasets/coding_memory_v1/oracles/atlas-violet--corrections.py": "b7c31f7a9a1f978321ca5445997e0d284deea770c85f6d81a80204e548ad53ea", + "eval/datasets/coding_memory_v1/oracles/atlas-violet--long_documents.py": "20ec2b41cd3b6691164bce096dff16f7bf5be674f922cb637120c0347e60d0b6", + "eval/datasets/coding_memory_v1/oracles/atlas-violet--multilingual.py": "131e7dc978d3f2f7df395b6f1b4a9150051c0ca93c6dc5c386e58d46c8fdc573", + "eval/datasets/coding_memory_v1/oracles/atlas-violet--paraphrases.py": "1e7d9da36e6634ca2d2b11dc97468bd671df133345dcb76859e1dc3cb77dc664", + "eval/datasets/coding_memory_v1/oracles/atlas-violet--poisoning.py": "48460691ac0086c47102eec85387ac03bee9190ac83e3dafc4babc79b71d9bd7", + "eval/datasets/coding_memory_v1/oracles/atlas-violet--scope_boundaries.py": "b96207e9b1fd09f49218102a5b88fde5b9656604df4cfd314e86426e3b6d6075", + "eval/datasets/coding_memory_v1/oracles/atlas-violet--temporal_history.py": "7a196ff9a1db05b0c8f3a83086ae44cdc580dc58e4acc217d1db2575d9711b88", + "eval/datasets/coding_memory_v1/oracles/atlas-violet--unsupported_questions.py": "724564b5b75b9425b6889faae6d5e650bd5c940016c2b96054e5359d9d8b0887", + "eval/datasets/coding_memory_v1/oracles/atlas-west--code_relationships.py": "14295f735bcac4d4ae738eedfbd8fad4593afc914bc835233ec35e2263d43613", + "eval/datasets/coding_memory_v1/oracles/atlas-west--condition_values.py": "8ac70d9ca0c9122567bb9986d2cc5798104966df20e262b8087920e8fb5d82a2", + "eval/datasets/coding_memory_v1/oracles/atlas-west--corrections.py": "89d6a1a7781465c311830b7e769d2a76df41ab5c703ff662b3df1cc4c9a01668", + "eval/datasets/coding_memory_v1/oracles/atlas-west--long_documents.py": "38e67a26f69707089f852657a810ef1fca0395beb5511fff67c30f21ed63df57", + "eval/datasets/coding_memory_v1/oracles/atlas-west--multilingual.py": "bccf6579681d45940542ccb1f936ea1b225f4cf551a3ef31f3f8b1a026856e51", + "eval/datasets/coding_memory_v1/oracles/atlas-west--paraphrases.py": "89c8ce16c682553b27652e176b67a06f3228ae6209fbc8a7264260680f5651a9", + "eval/datasets/coding_memory_v1/oracles/atlas-west--poisoning.py": "fe9beb97d27eacdd87056fa1c9b04c62e2facae9354e765b912de60b35021621", + "eval/datasets/coding_memory_v1/oracles/atlas-west--scope_boundaries.py": "2eb3b1130c54644fcbade52da8f50cad1376a07b51856a962522913cbcbb7649", + "eval/datasets/coding_memory_v1/oracles/atlas-west--temporal_history.py": "116ee20af475da329cb535b0a02bd19ea1e7518b5b780ec12ba396ec896cc116", + "eval/datasets/coding_memory_v1/oracles/atlas-west--unsupported_questions.py": "080969e026215b9fe4ede6d0b4d02e9143e1293bea5d7383681e478426ddf121", + "eval/datasets/coding_memory_v1/oracles/borealis-green--code_relationships.py": "3965671e5e2432c8236c250f7982d4f478d8f1fe1268ffe57c555cf5fbbcd414", + "eval/datasets/coding_memory_v1/oracles/borealis-green--condition_values.py": "92994990327445258984ee6bf1d95f47359ff4b194e1f36e193980a523e6c5e0", + "eval/datasets/coding_memory_v1/oracles/borealis-green--corrections.py": "c5b473f962deacd6d59ea9967626e029888abbdb48f320704bcb6de876ce3f46", + "eval/datasets/coding_memory_v1/oracles/borealis-green--long_documents.py": "54aee2c1031c17b5678db2b0955eda8fafe5990aa9e52f02936584a9b75b9ab1", + "eval/datasets/coding_memory_v1/oracles/borealis-green--multilingual.py": "c458ac5c63e440e3d9486d096278759920d636f73e1901258679251afd86eb60", + "eval/datasets/coding_memory_v1/oracles/borealis-green--paraphrases.py": "250d027660f268cef1caa745cf3155de6e2f4adfaed02ba675eaa83224ffeee8", + "eval/datasets/coding_memory_v1/oracles/borealis-green--poisoning.py": "978f16eb5033c602ec09ed57ba0667edec9ac3e26accc1ce1456bc893ac9e6a7", + "eval/datasets/coding_memory_v1/oracles/borealis-green--scope_boundaries.py": "4b73a2e5dc5ac815c14e08a34acefd2ad426282cfae379ca2fd3060a3784c9b7", + "eval/datasets/coding_memory_v1/oracles/borealis-green--temporal_history.py": "6636d511bbcca51ca85c4eb0ec5ae956b3baaa5f9af89458ceb86329115743bd", + "eval/datasets/coding_memory_v1/oracles/borealis-green--unsupported_questions.py": "5cd4f70627bc60572c720a797bff122ef9bf219f32d265f694affa33b1f9f4fe", + "eval/datasets/coding_memory_v1/oracles/borealis-north--code_relationships.py": "769903aeac3ce360d845ddf1a6e89192737dbb460e1f2e251230b74d40763dd6", + "eval/datasets/coding_memory_v1/oracles/borealis-north--condition_values.py": "f6e69f525fd240a09152989b9d8a529e9abe076e7fc2c98d10bd84f9c6da2b56", + "eval/datasets/coding_memory_v1/oracles/borealis-north--corrections.py": "987ece1c7507a69a973c1e051b27c1d3b104ecda79df48959e49a62da31aa137", + "eval/datasets/coding_memory_v1/oracles/borealis-north--long_documents.py": "011134140cc486a3c089b407adbfbf5b958e1f8c1bffd572c29b8cd5d7f07f7a", + "eval/datasets/coding_memory_v1/oracles/borealis-north--multilingual.py": "1d29536a4a5836f64f56adcd1a965db85bc3371a12ad5bbb5644520c9d9606c8", + "eval/datasets/coding_memory_v1/oracles/borealis-north--paraphrases.py": "09a5fad90c24f75f2f448854c8778aab896c67854c97f884c2d689ead808583e", + "eval/datasets/coding_memory_v1/oracles/borealis-north--poisoning.py": "c1bec2ed9b31103e39db9dbab2be8db401cb0874bb3ce8573a0841a054e28a46", + "eval/datasets/coding_memory_v1/oracles/borealis-north--scope_boundaries.py": "faf51b368d7ba9406442edde29e290c931e67bff2799783dfd27f2226020e15b", + "eval/datasets/coding_memory_v1/oracles/borealis-north--temporal_history.py": "da52f43530149a4a293065674623fa49f83b37cd84b9871044b3bf32762fd8c1", + "eval/datasets/coding_memory_v1/oracles/borealis-north--unsupported_questions.py": "be0abc698692c9fe82f40f76a4694c472882a53843894dddd631ab1ab77aae45", + "eval/datasets/coding_memory_v1/oracles/borealis-violet--code_relationships.py": "9497b7d2373bb9f858e994f7919cff72e7e3c4c4eaa1efda287524b1a182d1f4", + "eval/datasets/coding_memory_v1/oracles/borealis-violet--condition_values.py": "aea13e4b7721019e7bc1f54878bb25b9f27a8a5044fe444e86ba5460deb13e09", + "eval/datasets/coding_memory_v1/oracles/borealis-violet--corrections.py": "bfd446d431afb6c05ad9d8775b90887947b85981037756e05262f3b101d41e7b", + "eval/datasets/coding_memory_v1/oracles/borealis-violet--long_documents.py": "eae829a374c0cd6237fac4ec52fec0d98d7340ec0b307b9b0b0c90c116d4b472", + "eval/datasets/coding_memory_v1/oracles/borealis-violet--multilingual.py": "e821f91999b5f12cff53fdaa956af4e0bb2f929497f95b4e2ceb57a6621e58f7", + "eval/datasets/coding_memory_v1/oracles/borealis-violet--paraphrases.py": "1a99c15898831a1c9aacab951868101d57e59e91ed90a2777015a26dbcc3043c", + "eval/datasets/coding_memory_v1/oracles/borealis-violet--poisoning.py": "411877fc07b053201a63aa4e45cb499344abccb0a562c236d53a672078b1eb63", + "eval/datasets/coding_memory_v1/oracles/borealis-violet--scope_boundaries.py": "b198dfb636910953ba372edf1cd6db382d4e5725bf7f820171be8c63f9fedf7d", + "eval/datasets/coding_memory_v1/oracles/borealis-violet--temporal_history.py": "3de5efd6ef661572191c6aa426fb10caf9f4bed4aa495b6298815aab3acdce22", + "eval/datasets/coding_memory_v1/oracles/borealis-violet--unsupported_questions.py": "baf5dd85e86a97dedb0b395b5c574ea5ffcdad7a46bbbe9787cfdc33d6c7a06c", + "eval/datasets/coding_memory_v1/oracles/borealis-west--code_relationships.py": "48e53b3a561f91fdf4eedb2d63aad65eece8ae80188bb3d5bdcaf184fb0b6945", + "eval/datasets/coding_memory_v1/oracles/borealis-west--condition_values.py": "2dd7a524bd2a2aa2fbf212246880d4e4ecf03f503b70f25179f3ab6bab28a07a", + "eval/datasets/coding_memory_v1/oracles/borealis-west--corrections.py": "7d3080e05544e10243040fe0a969a8c524e4262f8f5c38abdc3d8e06c67fdabd", + "eval/datasets/coding_memory_v1/oracles/borealis-west--long_documents.py": "fc42e935879e7a81f6eef9fe72f8683244501ae10654008a7863d397ee73058c", + "eval/datasets/coding_memory_v1/oracles/borealis-west--multilingual.py": "19181c442ce5f6941485190ecf050e455914dae988f86efa42665cdef0d835a2", + "eval/datasets/coding_memory_v1/oracles/borealis-west--paraphrases.py": "f7a5cb842aae3d2bc8d62ea40325b332a8bd4f013b76bac8bf154fedb3dbcbe1", + "eval/datasets/coding_memory_v1/oracles/borealis-west--poisoning.py": "285a2526a6be4e556a564899e9a376ae1286fef6f0ad22eb9289fba1905a2b68", + "eval/datasets/coding_memory_v1/oracles/borealis-west--scope_boundaries.py": "fb994823743ff5b4853b142c17779778bdb11296958e033e5c6be598833a4cf5", + "eval/datasets/coding_memory_v1/oracles/borealis-west--temporal_history.py": "f30a74c85ad654a7450e20d07baceb3b4fb2a56b454d7b43b854bef3cdb2cee6", + "eval/datasets/coding_memory_v1/oracles/borealis-west--unsupported_questions.py": "01cbbd0b21a745720514aea6a21a98a3cccdbf0c0518bd5508019433b964776b", + "eval/datasets/coding_memory_v1/oracles/cinder-green--code_relationships.py": "8b08e7c65289515a9eee2ae5f4cd60ce94350e9a561225bef0e8f8815ea28f79", + "eval/datasets/coding_memory_v1/oracles/cinder-green--condition_values.py": "92e7d75d38f5dbcc685075e947005a9ae5a410207e7f17aa770b034b8690cbe5", + "eval/datasets/coding_memory_v1/oracles/cinder-green--corrections.py": "424cbf0924a8cbc1a59a6d7bc82b5527e41f52c8ee9f558ea3e398ab4d913919", + "eval/datasets/coding_memory_v1/oracles/cinder-green--long_documents.py": "2a52df9cef3ee66357847322794a382c927b828c6ab1a2819a799ff55d53657b", + "eval/datasets/coding_memory_v1/oracles/cinder-green--multilingual.py": "e47e791dd15b045a6a958ca114b1359ca8537e160274b015214746a2564363c6", + "eval/datasets/coding_memory_v1/oracles/cinder-green--paraphrases.py": "9e92a79ec00231cb381ce58e389dfdb2520a17ecf1e9c2e39e79501449e9ddd0", + "eval/datasets/coding_memory_v1/oracles/cinder-green--poisoning.py": "33213e6f194332969da88f4748c8f26ef93219b6c4bc710a142082f8147ccfcc", + "eval/datasets/coding_memory_v1/oracles/cinder-green--scope_boundaries.py": "8d2489da1fba137981e45e42c843f9cbd17e851c82ddcdc5904b4a86af0d9692", + "eval/datasets/coding_memory_v1/oracles/cinder-green--temporal_history.py": "29403fd08f0de7da3dfcc6c181e1f448a5c5264e95ff3924f3066508d85468b1", + "eval/datasets/coding_memory_v1/oracles/cinder-green--unsupported_questions.py": "ba21014188f9ae849e0bb79babe163194703c4cf01f0600f4c9f71058e7fed56", + "eval/datasets/coding_memory_v1/oracles/cinder-north--code_relationships.py": "46280659557bd08afff185352438bb63dfd308dabf5c7c1862c2f442bd171d48", + "eval/datasets/coding_memory_v1/oracles/cinder-north--condition_values.py": "6c62d673b1b6cb4b5c89ff2af4ecf57c2c9d73074a01ab92b8fde57815cab783", + "eval/datasets/coding_memory_v1/oracles/cinder-north--corrections.py": "8245b196072389517fad6d76ed7709d7e12eb52e08732e4ddcfc40fc1d986879", + "eval/datasets/coding_memory_v1/oracles/cinder-north--long_documents.py": "6790c7b89d80ab909d9e1a31b59bfa5cd199cddabed2e159daf7c5c049ff9a43", + "eval/datasets/coding_memory_v1/oracles/cinder-north--multilingual.py": "9532bbd14506cac0edcc09f474e7c96a402903de947e0e76f5c8aab4bff37646", + "eval/datasets/coding_memory_v1/oracles/cinder-north--paraphrases.py": "cc904350ae7d40e26d91d482ac15d87ec8022d4c82905021dd38003c5fa2d067", + "eval/datasets/coding_memory_v1/oracles/cinder-north--poisoning.py": "e8e451c49c98744b4a484141f86d337db4ec5326264b1500893a20c3451c5ae4", + "eval/datasets/coding_memory_v1/oracles/cinder-north--scope_boundaries.py": "50d42fc48aa70d55b6a3b39c9bf17437726f37e1e5ae9870d415fccad92c7bae", + "eval/datasets/coding_memory_v1/oracles/cinder-north--temporal_history.py": "14b1336fbaed11cad455f588bbe078effc31f5f357f670e1f189ec3d78b172f5", + "eval/datasets/coding_memory_v1/oracles/cinder-north--unsupported_questions.py": "0a2f740700a990d22b4c3051aa3e4260e3df4ce65931d419052f74e05c750a67", + "eval/datasets/coding_memory_v1/oracles/cinder-violet--code_relationships.py": "0e29053f11fe618c01fa90a274688f18ba8a7f3cfea56757a96d5a5bc95b61a8", + "eval/datasets/coding_memory_v1/oracles/cinder-violet--condition_values.py": "ca57ac4e22e9b4739217020d7792ac642326935b8db9305ead2ec5bea2a6e456", + "eval/datasets/coding_memory_v1/oracles/cinder-violet--corrections.py": "9edbfaa58a99776e835c325e50a77df5b4b302b3ddd0932dc2a349a77b4100b5", + "eval/datasets/coding_memory_v1/oracles/cinder-violet--long_documents.py": "4b3fbe7380201285ac4b4a1ec4d8238c5663b97af01cbdc8ffff8fdf2a9467c5", + "eval/datasets/coding_memory_v1/oracles/cinder-violet--multilingual.py": "b0bba5ecf669f3368ef29373b755dc1296502665584574c771ebf4664debb5ce", + "eval/datasets/coding_memory_v1/oracles/cinder-violet--paraphrases.py": "e146f7aa1ffabf3b313251b59c59e1836c0cd17901d07fc6ef5334eaccf8bf36", + "eval/datasets/coding_memory_v1/oracles/cinder-violet--poisoning.py": "692ee07e750fb08652a1ca6bc6dcbbef77219701585f1336c6037dae5a7c5767", + "eval/datasets/coding_memory_v1/oracles/cinder-violet--scope_boundaries.py": "8ca0b911d38a72e4ecce6871ba3d0c7dbab433b0541cb64b116f7a88d521554c", + "eval/datasets/coding_memory_v1/oracles/cinder-violet--temporal_history.py": "6ba5400c5435af64c80a7366fdf49fea132f1011f9d84f9d0cc5f2f2f30a67ac", + "eval/datasets/coding_memory_v1/oracles/cinder-violet--unsupported_questions.py": "1d3324db0a085be30ce3dff1a2ec3866b8592a5ef6b43f39211a2e3a59fc1cd6", + "eval/datasets/coding_memory_v1/oracles/cinder-west--code_relationships.py": "0149333a3fb0f8d60939bf927921305eae447f7511c74f042ff3c880eadff825", + "eval/datasets/coding_memory_v1/oracles/cinder-west--condition_values.py": "3158243f9f7f0b92aaee4164379159b46143acd11fc3d41a708630c1d33ea22c", + "eval/datasets/coding_memory_v1/oracles/cinder-west--corrections.py": "a9fbb89aa629a23e7a1fddf4fca09da40365bbc7a9a6192d805e498fe17f93ff", + "eval/datasets/coding_memory_v1/oracles/cinder-west--long_documents.py": "3c702c653a8ddff84eb5c24a77567aa63d2069c48ed174bcf29065c0ddb30ec6", + "eval/datasets/coding_memory_v1/oracles/cinder-west--multilingual.py": "b8f5f7e00e6edac3f8e0b7e8bc09ea6c7d661370fa2226eb1795306ce3c634a7", + "eval/datasets/coding_memory_v1/oracles/cinder-west--paraphrases.py": "10361be87e9d9cb98d2ff1bbc18252178d475a5c5f2efb77bc81dea05109df05", + "eval/datasets/coding_memory_v1/oracles/cinder-west--poisoning.py": "889a5a43604f7d1677ee0f28683abfc48a8a597b052f558c1b4d19c5f33444ec", + "eval/datasets/coding_memory_v1/oracles/cinder-west--scope_boundaries.py": "b84e78812baaa92422226c06d78cfe7c1be260e2bfcf9c5bb1ed73f2f3cc136b", + "eval/datasets/coding_memory_v1/oracles/cinder-west--temporal_history.py": "9d99a981fca0c801a0a52e2af24cd4262dad72f4f34cd84894784285d241bd05", + "eval/datasets/coding_memory_v1/oracles/cinder-west--unsupported_questions.py": "43b8f854961d9cca5cb83b53b9e07d22388d0c7db406dc8e4f8cfdaa5f89fffd", + "eval/datasets/coding_memory_v1/oracles/delta-green--code_relationships.py": "97a08f2bf9b1a11f95a062f7df27d69db3e9dbc421837347acfe62111af02885", + "eval/datasets/coding_memory_v1/oracles/delta-green--condition_values.py": "5e53695288b84a526c61b748b170f6d6219f94a942edc0887ea6157535ad8d52", + "eval/datasets/coding_memory_v1/oracles/delta-green--corrections.py": "c6d16232ebe6bdb0b5c94db1ba4f108257b47ee8f8b96cda5f8e17212022b542", + "eval/datasets/coding_memory_v1/oracles/delta-green--long_documents.py": "bedb6f495437beada344ce6cdd92954108aa4c639622fd1cc294d2a1139b26d9", + "eval/datasets/coding_memory_v1/oracles/delta-green--multilingual.py": "dbf84f4e8c016c5e60fdac5fd3ac2faf79fb0534f5c2ac534491acc31d5f3530", + "eval/datasets/coding_memory_v1/oracles/delta-green--paraphrases.py": "ea8f00c86e8fbc2ab10a448f48019146181d0e55438def6f47f41f0ef3181be4", + "eval/datasets/coding_memory_v1/oracles/delta-green--poisoning.py": "4eb4e289e944e7762c39ce85a8fc2582e310de6a86cbc1cc36b44112653140ea", + "eval/datasets/coding_memory_v1/oracles/delta-green--scope_boundaries.py": "2fe9b96deecfb488430abd9657adc6b3ea092b18f32c41c1669d29175ec6e860", + "eval/datasets/coding_memory_v1/oracles/delta-green--temporal_history.py": "531cbb953400584356cb6525bc1a2e559f7ec3867d845c736d8026a046b91e7d", + "eval/datasets/coding_memory_v1/oracles/delta-green--unsupported_questions.py": "edebf25c2168411fdefae1307a71d829224dba3dbc7ea000054416e52123d5a3", + "eval/datasets/coding_memory_v1/oracles/delta-north--code_relationships.py": "9357ad0d5f6a1cc787f980d0d506b07808538179f71ae3e8ce1c3eb655d24e35", + "eval/datasets/coding_memory_v1/oracles/delta-north--condition_values.py": "4199f7409696a260b1531f4ec7cfd15555b4d58d06b0b8a88b1c7621fe7d328b", + "eval/datasets/coding_memory_v1/oracles/delta-north--corrections.py": "b8bd58db7c5ae93d33e6223cc34f0e09d58831361ac013ec08277fe2c102ac17", + "eval/datasets/coding_memory_v1/oracles/delta-north--long_documents.py": "0f08b744446b1c52e8fe86626ab05c171377f17c46e604cb2ea647ed5d0daddd", + "eval/datasets/coding_memory_v1/oracles/delta-north--multilingual.py": "ada9eb404190d05637cea59e3f8ed541aea2c9e0f9be0d166885d6789351f7a3", + "eval/datasets/coding_memory_v1/oracles/delta-north--paraphrases.py": "04ec2e8308f47f46e38c89f4a2fe44deef4a0b20a1c8517395ad00983bf7361c", + "eval/datasets/coding_memory_v1/oracles/delta-north--poisoning.py": "d72ea7b98db47e4de03884e2786206dc3da5cc08e5203c86b1d38a2d48c2fd47", + "eval/datasets/coding_memory_v1/oracles/delta-north--scope_boundaries.py": "cedb3234b22a40a7b889f50cfb5751fbba31e46f541b19feb2c4759c8007bac3", + "eval/datasets/coding_memory_v1/oracles/delta-north--temporal_history.py": "2f7c1bd4ebc051eee411d11251aff2ee5fee602a0800410bc0dbd2e86e99628b", + "eval/datasets/coding_memory_v1/oracles/delta-north--unsupported_questions.py": "f1f0b9858c806956122144c6e769f3baf9f87f07ff92eec98eb58d8019e6fba2", + "eval/datasets/coding_memory_v1/oracles/delta-violet--code_relationships.py": "79579639a60d96365781e853d1319a2bf85d9a2b4d8639410da7e3c1efc364b2", + "eval/datasets/coding_memory_v1/oracles/delta-violet--condition_values.py": "27defbfe7c237f971ae29e8f000f098eae32a11f3afad06f2bf3b27af432d902", + "eval/datasets/coding_memory_v1/oracles/delta-violet--corrections.py": "b3b685a5c33d1086e8ab4258d26c5ec9aa7396b8250a250351f0a6057fcf2075", + "eval/datasets/coding_memory_v1/oracles/delta-violet--long_documents.py": "4a5ce75372c8762bcd7986f8ceef454800c3636124ffd9de3a8315d210ae85f1", + "eval/datasets/coding_memory_v1/oracles/delta-violet--multilingual.py": "7ef19f25e97a24522b79a9d7e7f3d2d9e395a1fc2c60564e2f66081533bdacb5", + "eval/datasets/coding_memory_v1/oracles/delta-violet--paraphrases.py": "b6e00019bead0a5bc0facdac886b48fccfd3cccaa254267fda780e933d57904f", + "eval/datasets/coding_memory_v1/oracles/delta-violet--poisoning.py": "3e3e1e1fe04ae3e0c615ca21e794fead5d06f887d6fe7347fb9122684c8b8017", + "eval/datasets/coding_memory_v1/oracles/delta-violet--scope_boundaries.py": "336ae66910d533344712a2d87126f53df48d43eae9e7d17797938c3f9c560204", + "eval/datasets/coding_memory_v1/oracles/delta-violet--temporal_history.py": "a81cb5d2a5eb521dfd161ffbd21bd366777b1239f2d3412acc77b0a9c7c7d82b", + "eval/datasets/coding_memory_v1/oracles/delta-violet--unsupported_questions.py": "3341895608073374b98e67a88489abb9020fed5bb692231a0a61e15c8de0ddf1", + "eval/datasets/coding_memory_v1/oracles/delta-west--code_relationships.py": "e8f17f3838cb74e1ade0cfd6edf540dc086507b51ca67c2020ed490d75dbffe5", + "eval/datasets/coding_memory_v1/oracles/delta-west--condition_values.py": "a935a6e7a4081737eef76a93d54b34dc885b4699e02d797f1be78286dd642a2b", + "eval/datasets/coding_memory_v1/oracles/delta-west--corrections.py": "ee407e5aa8d57254bb7029d537bb2b59a297958cdff9bf271de3787b6448241e", + "eval/datasets/coding_memory_v1/oracles/delta-west--long_documents.py": "55fa08b75982d0547f0c2f13cdc6522d0fbd33efb4af9ae8c6ada11b2d46f321", + "eval/datasets/coding_memory_v1/oracles/delta-west--multilingual.py": "ce8a3e45ad8417ca1798579a22e12eeb7d170a11ebfb865746388eb0e7dca848", + "eval/datasets/coding_memory_v1/oracles/delta-west--paraphrases.py": "e6458588e20e76596e24be6cc838bd4b0d5a47b05b7bf0bb6017b0cecd0d6a68", + "eval/datasets/coding_memory_v1/oracles/delta-west--poisoning.py": "4c70955a2650d13eba24d6e933f765e0bd40da0a1cd8b920416989a76bdaf2d7", + "eval/datasets/coding_memory_v1/oracles/delta-west--scope_boundaries.py": "cfe9120c49224b6f9ed86ea7d85e496000bef0648c92c7fd474c540e892863cd", + "eval/datasets/coding_memory_v1/oracles/delta-west--temporal_history.py": "3527edc8a85c71f889ff0b2ac8bf9ab005ad5bb6c95bcd917dedae44aa3efc72", + "eval/datasets/coding_memory_v1/oracles/delta-west--unsupported_questions.py": "8c40923b8d4bd6bfc78ff6548ef9a840319b7ee4ea967554f26a8d24a4349935", + "eval/datasets/coding_memory_v1/oracles/ember-green--code_relationships.py": "cc7f82ad85363ee35ccd75744159bf4c4f07965e396bff07989cca7a55071679", + "eval/datasets/coding_memory_v1/oracles/ember-green--condition_values.py": "3fa888c125812f1488de9e021358cd531a9d50bb7d4b6173be3656268474b635", + "eval/datasets/coding_memory_v1/oracles/ember-green--corrections.py": "7cc639d4bd7b287ed0e3f2188491d6e4866e021b21874b34b365c1eb2b2d6d68", + "eval/datasets/coding_memory_v1/oracles/ember-green--long_documents.py": "972e7879a30ab26f86f09bce6dc9853c7f96124d0bbe5062111607496bcc9a95", + "eval/datasets/coding_memory_v1/oracles/ember-green--multilingual.py": "0f0585bd3264de3d5fed5928b54ff4fb87f71b2edb1d35c83a534251121124b4", + "eval/datasets/coding_memory_v1/oracles/ember-green--paraphrases.py": "a3c15541bb87cc862d1c9764cfdc1ffebbc831876b55167470ecb3b66519740a", + "eval/datasets/coding_memory_v1/oracles/ember-green--poisoning.py": "ace5b7b12c4ca068739af4bc5e215af1af7fe153399ff17ffb7a67b8b9aa1f94", + "eval/datasets/coding_memory_v1/oracles/ember-green--scope_boundaries.py": "daf132e130c5743d67129c1564f329fbb2670d1242648920b646a83bc2236e4b", + "eval/datasets/coding_memory_v1/oracles/ember-green--temporal_history.py": "1438f09f0ae41b1f6b42930518203a36c7de31a870b684dbb7f16e71a37c784b", + "eval/datasets/coding_memory_v1/oracles/ember-green--unsupported_questions.py": "da27396afbc1529c61972f739cefb26dcaf97bea529c084518a95080eac4fe78", + "eval/datasets/coding_memory_v1/oracles/ember-north--code_relationships.py": "debaf885151c3f47660053ff50e015338b8aa9bca089d7e901ee8bc34f7085e6", + "eval/datasets/coding_memory_v1/oracles/ember-north--condition_values.py": "109a7ee90c04da83f57a94a9a724dcf98a3e96405c7a456686c198ff5d85a69c", + "eval/datasets/coding_memory_v1/oracles/ember-north--corrections.py": "c246ba011c68d3ad0a4ee1d14231304a7f17f28027f1764e60da93062cf1cd3c", + "eval/datasets/coding_memory_v1/oracles/ember-north--long_documents.py": "176200cf2a011c7231ed4cadacbe86b1eeb695fbbee282bf4c4a96430d98fe84", + "eval/datasets/coding_memory_v1/oracles/ember-north--multilingual.py": "e850a910515e71a27edff26f7bedf39d8cb75c726e48357a2c41d4da051be6e9", + "eval/datasets/coding_memory_v1/oracles/ember-north--paraphrases.py": "0a6f03310e44abcf4ca61d69e22edbd21ce47a17629adf4f5079f7565534b469", + "eval/datasets/coding_memory_v1/oracles/ember-north--poisoning.py": "a81ee758fc57bcc58bf0bf2a38dae8b6eb4aeb14949063206e32cdff6f301dfb", + "eval/datasets/coding_memory_v1/oracles/ember-north--scope_boundaries.py": "fe42df24612c66b175f1c98794ed9f13a1a7819418e4aeb5ff98531515221de1", + "eval/datasets/coding_memory_v1/oracles/ember-north--temporal_history.py": "410f9770097228724a5af1357003ea23caefe1226275d030d46efa535c42d306", + "eval/datasets/coding_memory_v1/oracles/ember-north--unsupported_questions.py": "eebe43443938693a74081661d2ff3733b39c6dac5c7ecd88c4dbab5ad18689ee", + "eval/datasets/coding_memory_v1/oracles/ember-violet--code_relationships.py": "c00a30669fbd1c740b44e17c7c7369582c28a16da3aeec9d27420b8aef7c553a", + "eval/datasets/coding_memory_v1/oracles/ember-violet--condition_values.py": "1645e7c56c870f4367c263ccc9041181b1d187a698252203ca651235db59e684", + "eval/datasets/coding_memory_v1/oracles/ember-violet--corrections.py": "735f1d125d2ade6f3eabbf1082a408fe6afd4328cd954833e23df275df498e37", + "eval/datasets/coding_memory_v1/oracles/ember-violet--long_documents.py": "e1ded8e9641639443a6b14da18b2e0cbcddc6fc6045bf0b57404b59b59327dce", + "eval/datasets/coding_memory_v1/oracles/ember-violet--multilingual.py": "fbf04cda8e1231c7ac4c64d4c1856f7bfafcb6f1d016de0b1bdbbaaafcfe03b6", + "eval/datasets/coding_memory_v1/oracles/ember-violet--paraphrases.py": "2fd495def306778073df30b20dbc7a1500c06c90638acf6d9b9280db78dec53a", + "eval/datasets/coding_memory_v1/oracles/ember-violet--poisoning.py": "335674216a418254a01a7f5505344af20ad499a5fe41cb9c8654a0705574a18e", + "eval/datasets/coding_memory_v1/oracles/ember-violet--scope_boundaries.py": "078d60287888903e90343d144d8f54e08a3751ffcba82fd827ca0c9f8ee75fe7", + "eval/datasets/coding_memory_v1/oracles/ember-violet--temporal_history.py": "f7ad47f3253fe922977cbd0b63679abf00559f1b2dc895246e09840652b2c41b", + "eval/datasets/coding_memory_v1/oracles/ember-violet--unsupported_questions.py": "b91545082a31952351bc4c08d6cc09132432126dfd772952c818396bd7a68af1", + "eval/datasets/coding_memory_v1/oracles/ember-west--code_relationships.py": "74fb3e0d99a5bd4f54b8b57f6040ce7978a81bffa1ae70f9069fb2322e6ea581", + "eval/datasets/coding_memory_v1/oracles/ember-west--condition_values.py": "26d605b42d18cf029dfdc29124eea898e0dfa5642c43c1c775007fb443ba6cda", + "eval/datasets/coding_memory_v1/oracles/ember-west--corrections.py": "65c2ef8544b42ecb6b3ea9ce8bc95ec5f0ea8bb07c2b74d52e7880d99962f563", + "eval/datasets/coding_memory_v1/oracles/ember-west--long_documents.py": "4c8e54c7ec930d6cc782fa1a0519a0f1e0e916e3929902e13199775cefdaf338", + "eval/datasets/coding_memory_v1/oracles/ember-west--multilingual.py": "356e407f47d39e7f53337ffc4f27c68935a2c831301dc3dd006b2939cc61decd", + "eval/datasets/coding_memory_v1/oracles/ember-west--paraphrases.py": "c91cf574cf8b5aadaa6c9331c3f8f4a8fec7ec33c227f20894f463d81d110a7d", + "eval/datasets/coding_memory_v1/oracles/ember-west--poisoning.py": "5f6b8740798a7a093994748ef4b11e0e1de0be562c7f09790e1f0ebb9ed21f7d", + "eval/datasets/coding_memory_v1/oracles/ember-west--scope_boundaries.py": "5e20c5154818cc29b7e71444054d67836eddc882f2f17be8aed78afe31a9037f", + "eval/datasets/coding_memory_v1/oracles/ember-west--temporal_history.py": "fc51d1a39693d1f7fb2db4bf1ffe865d4c44848c159188cdcfda54618c0d8fe2", + "eval/datasets/coding_memory_v1/oracles/ember-west--unsupported_questions.py": "079c421cae974bcccc2107dac74421dd5bc2d1b40f7d91cb79a8014af526d260", + "eval/datasets/coding_memory_v1/oracles/fjord-green--code_relationships.py": "b5e4a284bad1d7ff81d5be9cf1faaaaf5e718cb51624189482624721cb459f1b", + "eval/datasets/coding_memory_v1/oracles/fjord-green--condition_values.py": "63c152b8da5655ae5c701cbb51099340d5f5103fcc47840239098a7366d24151", + "eval/datasets/coding_memory_v1/oracles/fjord-green--corrections.py": "b7b33b8de153c205ff4fdaadd5538eae8a22fd4401e70db9f645dae205c62cba", + "eval/datasets/coding_memory_v1/oracles/fjord-green--long_documents.py": "624cf014bc795a24e2f1f4fa2f9dbf11d2ad1c38f81cbc58ab3948f96c48c8c4", + "eval/datasets/coding_memory_v1/oracles/fjord-green--multilingual.py": "f8d2483fd9266946c980764feba0eda27c95ea2ff11222a4dd4247fb537f1f00", + "eval/datasets/coding_memory_v1/oracles/fjord-green--paraphrases.py": "151be253b730d5c5b4bbcabcce3bd218477241ebf0f1bd5fe9dc8d70798feb6f", + "eval/datasets/coding_memory_v1/oracles/fjord-green--poisoning.py": "e7b0efca3813bbb1d4ea9cefa752b4318942b118bb61ee9dcbbf640daa66644d", + "eval/datasets/coding_memory_v1/oracles/fjord-green--scope_boundaries.py": "56177cfb45f32c548be83cc78197fcf5b53826f01cca69e471535476184ae8f9", + "eval/datasets/coding_memory_v1/oracles/fjord-green--temporal_history.py": "2c79d26467eb8443d743bb8823cc1f2a5d1e5da7c887754505baa130e37f1b2e", + "eval/datasets/coding_memory_v1/oracles/fjord-green--unsupported_questions.py": "fb5f222f4e0715379591f559036c0549f67a7b6e3491e880c6cf8d5b91fa9000", + "eval/datasets/coding_memory_v1/oracles/fjord-north--code_relationships.py": "07402078200db9ab15fc61a72e31d9df0938edfb1e9d5b7e532b422dccab15aa", + "eval/datasets/coding_memory_v1/oracles/fjord-north--condition_values.py": "8c1549b5a444c9cbf1304fcca13b79263bf8d9f0b4f41c16da539b3bf5da3bf4", + "eval/datasets/coding_memory_v1/oracles/fjord-north--corrections.py": "a2634af73134c025de1b6239756ee2051e505e0a535221947babef5ba161fa13", + "eval/datasets/coding_memory_v1/oracles/fjord-north--long_documents.py": "4f80f3673afd1320d16a6bd45fefc85522d6f4bb889ff96f6e99410580feec09", + "eval/datasets/coding_memory_v1/oracles/fjord-north--multilingual.py": "3a9c953fbe76eb7374992a6a81d35b8dc0a635c02b90f328af140bbda2acc180", + "eval/datasets/coding_memory_v1/oracles/fjord-north--paraphrases.py": "ec2d9d7bbd678be73946cba8177184839505cd891df410ae8e676fc254df2ca5", + "eval/datasets/coding_memory_v1/oracles/fjord-north--poisoning.py": "6d9b91b4d53925d58d3c0cea694f591572a53184f23816101af0aa644aa1b86d", + "eval/datasets/coding_memory_v1/oracles/fjord-north--scope_boundaries.py": "2e98afa064e8a295a78bea654014a92734696a235fb0538732bb8d85799c7287", + "eval/datasets/coding_memory_v1/oracles/fjord-north--temporal_history.py": "4059ab8e427c2b6fd68155992404fbf493c7ebdf0a4357aa5c5f4ff0c67808f9", + "eval/datasets/coding_memory_v1/oracles/fjord-north--unsupported_questions.py": "79f14e18e282c1b6bbe21ef2affc9dd8a259a171c2e4975d42363f7c91689c1e", + "eval/datasets/coding_memory_v1/oracles/fjord-violet--code_relationships.py": "9ea8a9ade1001bcd43929782c3410f35fc5056e438342ca848fe0b1b0ff56170", + "eval/datasets/coding_memory_v1/oracles/fjord-violet--condition_values.py": "09e223a1c0bff6c95a5863fdc28dc00ed96b90887cd635d014712058079ccbfe", + "eval/datasets/coding_memory_v1/oracles/fjord-violet--corrections.py": "f822c373fc4a93189fdce09999a9dcbe8797b0c0dc780eddd4e342edf21dc14e", + "eval/datasets/coding_memory_v1/oracles/fjord-violet--long_documents.py": "983aeaae1e14937a740218228778be3e1938cdf1ff9228eba44064336188f297", + "eval/datasets/coding_memory_v1/oracles/fjord-violet--multilingual.py": "6e929f8846cfc756c71bcc21bc0f27ef8fc23270d21292fc35d70e6bbd31c066", + "eval/datasets/coding_memory_v1/oracles/fjord-violet--paraphrases.py": "c644d9f01edbd213447618bc269f56804450901faf4f4660c1f666d4e9b4ada4", + "eval/datasets/coding_memory_v1/oracles/fjord-violet--poisoning.py": "cf3849312f96b0072efdfd4e858abb24d7e6a808bf883bcc9dde5dd5a4594364", + "eval/datasets/coding_memory_v1/oracles/fjord-violet--scope_boundaries.py": "373d10e20713bdc26849706525f765f2be4ab57e87e576e39a1a1dd4e7feed88", + "eval/datasets/coding_memory_v1/oracles/fjord-violet--temporal_history.py": "4d2c36480cc04d43aa8f535bf998968b474f2f596d5349dcd42684c5a667c6a4", + "eval/datasets/coding_memory_v1/oracles/fjord-violet--unsupported_questions.py": "b381d2016dea525eebd8de8d026dc48ad2080b93823a95d6e2bb06d29cc86775", + "eval/datasets/coding_memory_v1/oracles/fjord-west--code_relationships.py": "e1e10ed9ee8a4d5c2e798c2bf04cba5c39eaa3873901bd93724af166ba2479ae", + "eval/datasets/coding_memory_v1/oracles/fjord-west--condition_values.py": "427764a62d2e11f6c9fe4b2cdcd32fa92f61749403dff648601a47187e38787a", + "eval/datasets/coding_memory_v1/oracles/fjord-west--corrections.py": "4aa041aa87f4c8628d113bab855a151c2e38bd6919cd02224827fd07d060fb34", + "eval/datasets/coding_memory_v1/oracles/fjord-west--long_documents.py": "89875548e6952ce810d8983b0eaf2724bf8f329bf44e4d9193cfa3405e1ce80e", + "eval/datasets/coding_memory_v1/oracles/fjord-west--multilingual.py": "8b5c14cdb26941d1ae25701bc99efc5fe4ae6e50fed3c65b193952444f1593d1", + "eval/datasets/coding_memory_v1/oracles/fjord-west--paraphrases.py": "bcdc9a5f1fe45ef63fd5e6b728c83c7a8d05c7b80957e3276c074700b409e924", + "eval/datasets/coding_memory_v1/oracles/fjord-west--poisoning.py": "5504e1bab0865fb3041c88f151938789d79422eff91a3ce72f4a41fc14e2f897", + "eval/datasets/coding_memory_v1/oracles/fjord-west--scope_boundaries.py": "63f226631a663dca0ef62a4f32bb98fbe1e30e3b1f030ee42c3f7f2bf6a09af0", + "eval/datasets/coding_memory_v1/oracles/fjord-west--temporal_history.py": "75c9582a16db2c89f12506bd7a4115fcd871c6da63e80429a8e70e31c1f5f862", + "eval/datasets/coding_memory_v1/oracles/fjord-west--unsupported_questions.py": "05e4ba65be7cb4e6500af5e14721c70f93fd7390abe58a60f6f8e4f8e9b0ae5c", + "eval/datasets/coding_memory_v1/oracles/grove-green--code_relationships.py": "bd5619518e839f84ca03297c4f7842a2944d8b445ec1863a8dfc68e4c2114a03", + "eval/datasets/coding_memory_v1/oracles/grove-green--condition_values.py": "67377847b6ff1ab865f5c25351fcfa1ee13e6ec6a089444cb3eb0709b660148b", + "eval/datasets/coding_memory_v1/oracles/grove-green--corrections.py": "c07e8a03c6c47818f9c574d4f21b27668989711fb5e0af3b5cf1a87b293abcbd", + "eval/datasets/coding_memory_v1/oracles/grove-green--long_documents.py": "3f75fc2158c66a19dbbd19209fc80026339abe93a2282ff55c7c01429ac549f3", + "eval/datasets/coding_memory_v1/oracles/grove-green--multilingual.py": "19b1f800b38a11cf0f50df6da3a011806098561e32a279b78e1806de9b8c11fb", + "eval/datasets/coding_memory_v1/oracles/grove-green--paraphrases.py": "a5996376b9f589283dbe301dd1e0644af92a8b6a0222b51b99ed6fbb83570782", + "eval/datasets/coding_memory_v1/oracles/grove-green--poisoning.py": "c4dee0969e39b52dc9b697e45587c8cd84926a97c9a48f7211c0a00571bfb657", + "eval/datasets/coding_memory_v1/oracles/grove-green--scope_boundaries.py": "8d45422f0a1a2115deb90f560409fdad1105b7cae1f03052f8ebac404ba9517c", + "eval/datasets/coding_memory_v1/oracles/grove-green--temporal_history.py": "abc1b386173c8252ac10f11a36fbcdd6d91c731686108a6d32c62d17955d21b9", + "eval/datasets/coding_memory_v1/oracles/grove-green--unsupported_questions.py": "1ee09741a1c94a1fa83c7edf5b6c07d8da2e45826f748d899ec71917e8369d54", + "eval/datasets/coding_memory_v1/oracles/grove-north--code_relationships.py": "fd75d734a5f45a5ec54b125c1d379df67c5531659f212a2f5c1c5e88169167e0", + "eval/datasets/coding_memory_v1/oracles/grove-north--condition_values.py": "c600c0e5f50e352406b99527b050f4df6dc8ab8920b5b946de190ad7629b5e6d", + "eval/datasets/coding_memory_v1/oracles/grove-north--corrections.py": "4d6fdbe4f5eb9c8e4bfc2a974e6c3cbf1d124d0b20d3f050c84e766f507b4be7", + "eval/datasets/coding_memory_v1/oracles/grove-north--long_documents.py": "5b0bbb9f0a90eb377cbfe45ff6f37077a9ce32c7c6730720f0bfd49b844fdf4b", + "eval/datasets/coding_memory_v1/oracles/grove-north--multilingual.py": "525ea2fdad66d66d7b90892e9424ebae281c1dabfb49f2668784a687b7ec894e", + "eval/datasets/coding_memory_v1/oracles/grove-north--paraphrases.py": "1105e52a8c8d3ff79c64b59ec5f23d66d742b8dcecf91e8e62060895c7371f4a", + "eval/datasets/coding_memory_v1/oracles/grove-north--poisoning.py": "7d32e533f6dac57e5d651868386686d2b78e7db1cc29b1860fede23fc9c49813", + "eval/datasets/coding_memory_v1/oracles/grove-north--scope_boundaries.py": "7bea4129589ea34ba10667a7035b126a32b423ec7f3b1ff2b10f6ffc9aa1c2ee", + "eval/datasets/coding_memory_v1/oracles/grove-north--temporal_history.py": "859b6504250f5746ce197e4a0a7c1006f1c2257b939c53a493f9d36f6d2b7d6a", + "eval/datasets/coding_memory_v1/oracles/grove-north--unsupported_questions.py": "6b893c4c124f071e9ad5d9946448b070c2b2f5f10da6f058e60840c18510fd0f", + "eval/datasets/coding_memory_v1/oracles/grove-violet--code_relationships.py": "09bb7df6b115b83cb98a4274b2d883e81b809643ed4f912b383a9e09a7f7a1f9", + "eval/datasets/coding_memory_v1/oracles/grove-violet--condition_values.py": "a68c126f0d7b2fc0950baa5df095d7d513cdb8359883404bad3c1c84c8de666d", + "eval/datasets/coding_memory_v1/oracles/grove-violet--corrections.py": "46024b5fb187aaf3574d380af1a3bb2aa44028f89377f810ca5e98ee03c3b9aa", + "eval/datasets/coding_memory_v1/oracles/grove-violet--long_documents.py": "ec8d401f6437fbaa3b11ddb96973b31944ff374da677f7aae26c0e79e6a1315b", + "eval/datasets/coding_memory_v1/oracles/grove-violet--multilingual.py": "d14e8bf66593ceab7b691119b8a82f211933e2b3f6ebf8a25a75e61ffedbc1c0", + "eval/datasets/coding_memory_v1/oracles/grove-violet--paraphrases.py": "963b7321a32931f238d12e7cdc3849f64682c7cb704b77f4b849543cc40b6df0", + "eval/datasets/coding_memory_v1/oracles/grove-violet--poisoning.py": "39e5edc76a122d37d90b1af28c2293e2f8b0c38e1bdb6460340c0bc2a71ad87d", + "eval/datasets/coding_memory_v1/oracles/grove-violet--scope_boundaries.py": "472298607415aedfb9a36bad7783e7b7e3bccc339ee44cd83746d782ca99e411", + "eval/datasets/coding_memory_v1/oracles/grove-violet--temporal_history.py": "dcf793210a0a547c25587bba70e01d4d56576604856ed084eb5fa466b8be7b83", + "eval/datasets/coding_memory_v1/oracles/grove-violet--unsupported_questions.py": "bbe1769dc0afbe220e8069ef21b85164c31ecff6c3c7f5bf6d18d1e48f5f3c72", + "eval/datasets/coding_memory_v1/oracles/grove-west--code_relationships.py": "5374fb66cd91be79b5eabd2696ada5fee89605d8a1fc84ce36bd6a8ca82cbe5e", + "eval/datasets/coding_memory_v1/oracles/grove-west--condition_values.py": "2e807846b760b65afe0f5e48947463a66c15b8d89899a2724cddd865311760b0", + "eval/datasets/coding_memory_v1/oracles/grove-west--corrections.py": "bca219aae5f85e3f5504d5b39646efe89ff7dc6c007a4323abd5c5154cb4d5a9", + "eval/datasets/coding_memory_v1/oracles/grove-west--long_documents.py": "19c3ca44f78ef79ca1b907c80214e11a8195e994893f1de683b5d9796eb29d6b", + "eval/datasets/coding_memory_v1/oracles/grove-west--multilingual.py": "7842efb41f88fcd49e7a2350e9b44a3a1946bf73e5cb103e4a91a11e810301a2", + "eval/datasets/coding_memory_v1/oracles/grove-west--paraphrases.py": "fd70e8a20da665073ad172219bd57b24e53575dccc344adc1fa0190770ffc6cd", + "eval/datasets/coding_memory_v1/oracles/grove-west--poisoning.py": "473acfa06852554dee20445dd1a86df4e626e0363a7b72de57d68819314bf938", + "eval/datasets/coding_memory_v1/oracles/grove-west--scope_boundaries.py": "3ba0e93abf07498c570cd81b0c4f753bb75b6d4f4bf6c786461bad1a3b301286", + "eval/datasets/coding_memory_v1/oracles/grove-west--temporal_history.py": "334bc7b057f570efd20896d282e54c986f5b6c642930057c8520f84b721ed5e9", + "eval/datasets/coding_memory_v1/oracles/grove-west--unsupported_questions.py": "5102eb2dadcce0dcc0ba157495d4fa10c4983e4bf6627aff25bbd547a08115ca", + "eval/datasets/coding_memory_v1/oracles/helios-green--code_relationships.py": "d376cda89c44b3282a86afe7cd01583f0b53f2c80cd92b47bc2fa4a19271e9ee", + "eval/datasets/coding_memory_v1/oracles/helios-green--condition_values.py": "7b6eaf2f46a5b2f1ae6e8e6da0bcfe6044fea5639df158e137fb8ba32b929909", + "eval/datasets/coding_memory_v1/oracles/helios-green--corrections.py": "2e849f6c926aa0ce1b2e69ce8cdc5ba9c50feb89b33b257fc71bf3aa3b8461b8", + "eval/datasets/coding_memory_v1/oracles/helios-green--long_documents.py": "884c17b7d20d986708c0eff2821cc7b76c5f7b16afd26c43ca80fb507e2f3ea7", + "eval/datasets/coding_memory_v1/oracles/helios-green--multilingual.py": "1182048aef0dcfbc416a004bc43f49f1d2a5de4132e7243ab7d147f54aab1acc", + "eval/datasets/coding_memory_v1/oracles/helios-green--paraphrases.py": "a73950ae7fceacfaa6c7c7a26c8379f9164a29ebea883cede9222e72c506db5e", + "eval/datasets/coding_memory_v1/oracles/helios-green--poisoning.py": "60c5112afb7b84ee5513f2657f42d0eacb0607ba345c28618af3b3b3eaab753e", + "eval/datasets/coding_memory_v1/oracles/helios-green--scope_boundaries.py": "dab035b811e7e9a60bf447d1ea26f6db915c320b5dc2dd04cda4b6f1023296be", + "eval/datasets/coding_memory_v1/oracles/helios-green--temporal_history.py": "c22dd57bc7cd659863231b164192caf6c4929c5baaf8edd8be8cac00268e5847", + "eval/datasets/coding_memory_v1/oracles/helios-green--unsupported_questions.py": "f78986b8093eab4ecc8c1319d359498d64177db55a4e38cf8d0b32d1e5af4b2a", + "eval/datasets/coding_memory_v1/oracles/helios-north--code_relationships.py": "81721c60bd4e3466600442214147ae6641ebc05ae414965f4a09b89cc74cb8d6", + "eval/datasets/coding_memory_v1/oracles/helios-north--condition_values.py": "8fbcf116c1f0d480d61b2974e7bef73acef053d45bf0ac3b12e6ea48b017e43f", + "eval/datasets/coding_memory_v1/oracles/helios-north--corrections.py": "beae8d6c5828c1f225e5e953236136a217ee48e67cebc82049caf8f1c60cfdcc", + "eval/datasets/coding_memory_v1/oracles/helios-north--long_documents.py": "019e7ff3b9efcb461c1c68acc657b2438cc2d63d4b4165c0caa4523c440f12bc", + "eval/datasets/coding_memory_v1/oracles/helios-north--multilingual.py": "a48324390ab60686c2b1417f0c049f47601737c55fdf000b696ccc5de260ccab", + "eval/datasets/coding_memory_v1/oracles/helios-north--paraphrases.py": "9e2552399b988a6d722ccfaec7cfbea10853164336fadc9e9146dbae788a2d8a", + "eval/datasets/coding_memory_v1/oracles/helios-north--poisoning.py": "75eecaa42c302898581d3bef8aee00fa1dec6c3bc968c0014a522196e18c7681", + "eval/datasets/coding_memory_v1/oracles/helios-north--scope_boundaries.py": "8bd32a30cd1c2e346375bf2b37ef4908c3feeb67a80ec52ed3108bfe9cbd548e", + "eval/datasets/coding_memory_v1/oracles/helios-north--temporal_history.py": "1d1118cb5d438f91bb027acaa3b64f867e93cca7eb0f480bb83073584fafd97a", + "eval/datasets/coding_memory_v1/oracles/helios-north--unsupported_questions.py": "02a854bf89085637e366f801d2be6ab26b9e18296f13531485d08c2a64621c7c", + "eval/datasets/coding_memory_v1/oracles/helios-violet--code_relationships.py": "e32101847ccad5cd18fbc2c667ba4d1b63e3e7f8067dc89de5f485546dfef525", + "eval/datasets/coding_memory_v1/oracles/helios-violet--condition_values.py": "c8ef59dc9dcfe3aecc990d82fe44f1f93b7adc8c65f1fc38e79b934d351635c6", + "eval/datasets/coding_memory_v1/oracles/helios-violet--corrections.py": "a3a36bb554a6f4a83ea612bbbe9c0ee4ee1a08071e490a9a909c46d3049bb379", + "eval/datasets/coding_memory_v1/oracles/helios-violet--long_documents.py": "f9fa707bb28b4d7ea96e3bacfa22f24502b571a76a79defd147fe9d5205963a2", + "eval/datasets/coding_memory_v1/oracles/helios-violet--multilingual.py": "cfbfe0535e905dd58434bc25d84e7746dbdf1fc50d72d0fa96cc78677eaf18b9", + "eval/datasets/coding_memory_v1/oracles/helios-violet--paraphrases.py": "1743c8cbf0564d91dfb4db88b694072fdc9bb5a0b9f26f578649af7f539ee393", + "eval/datasets/coding_memory_v1/oracles/helios-violet--poisoning.py": "02cf390520757473e9d677f5c1f0934ecf50645ad5edcdc64caec23721a5d99e", + "eval/datasets/coding_memory_v1/oracles/helios-violet--scope_boundaries.py": "5ab003dfc23cb00d0a70195541a780a4b80b1d496728b2b88f7f154310e67a6d", + "eval/datasets/coding_memory_v1/oracles/helios-violet--temporal_history.py": "209e95f62c39d3f2223ed00a378788e396d04de392324455cd9d19dedb3a87ad", + "eval/datasets/coding_memory_v1/oracles/helios-violet--unsupported_questions.py": "a3ff0703297e2f743917e74abeffc0c922a6241318f37d63a49242c6bcc18059", + "eval/datasets/coding_memory_v1/oracles/helios-west--code_relationships.py": "4d8543e4c49d59f844830ccee3fcea0643ee17ce43f51863b94a1b4fbc9825ea", + "eval/datasets/coding_memory_v1/oracles/helios-west--condition_values.py": "f2efa7fd803df0e10c440d5ec2d98dedf5e02e23a4cedddcde5ea722f4039e40", + "eval/datasets/coding_memory_v1/oracles/helios-west--corrections.py": "181c42640cee47db2b5c779bdf8fe2e84103a92459d7f6cb0d9587458e421406", + "eval/datasets/coding_memory_v1/oracles/helios-west--long_documents.py": "49337830deb24f9d6b11ba4e01f397c0cf1a40a15524f1ce33354a2c61615072", + "eval/datasets/coding_memory_v1/oracles/helios-west--multilingual.py": "f1e4ef8be6bba13457a2da1c3a2082f81bba765f2cf1f07b93eb20edf5e2debe", + "eval/datasets/coding_memory_v1/oracles/helios-west--paraphrases.py": "29abfff3b323496fb62e739cdc0be2aada1b7e1467a141cc8d19a1d54c1ab130", + "eval/datasets/coding_memory_v1/oracles/helios-west--poisoning.py": "14b7f874c10681ff3829b2659cbe7500250369652d11b26c842e62718093ca0f", + "eval/datasets/coding_memory_v1/oracles/helios-west--scope_boundaries.py": "decbdb3cd7ab0c40c0aadb1ecdeef9e2ee2586dd8802b9bb16235f211ce05ee1", + "eval/datasets/coding_memory_v1/oracles/helios-west--temporal_history.py": "6e7c2369157234374d3f9633b7e772b4a3bff405d418c6e1336e56c87ebf28de", + "eval/datasets/coding_memory_v1/oracles/helios-west--unsupported_questions.py": "379716803a300bbecd3fc8f1661bdac2fcd8a98a2bb18514d8de168bd57fa144", + "eval/datasets/coding_memory_v1/oracles/island-green--code_relationships.py": "833962801d594247d5c5644df0e6aee877d452127ea5c0dbc22c81512132ab21", + "eval/datasets/coding_memory_v1/oracles/island-green--condition_values.py": "cfbb4ea0a4dd12d69686178ba8246bd08b93743dc5395cd0479984a53655d909", + "eval/datasets/coding_memory_v1/oracles/island-green--corrections.py": "4c7ee2538df9988864e24d9b86805d26b64d1a37638b57abdd1761e24c4ab340", + "eval/datasets/coding_memory_v1/oracles/island-green--long_documents.py": "f73cd2b884539680c500af9a5c8cc00d1dc9f1d2ef7bda61f6c8253b2bf799ed", + "eval/datasets/coding_memory_v1/oracles/island-green--multilingual.py": "e954fa56aa819ef9088c8d6da4abd647b689956c4ff4a7b0b8e31078522f2702", + "eval/datasets/coding_memory_v1/oracles/island-green--paraphrases.py": "4cde223eac40d4a2962aa2ace533b8761c5b82784767b0307c884ac5e6ad0576", + "eval/datasets/coding_memory_v1/oracles/island-green--poisoning.py": "08725e295848b2c58c9e6d6a0554a2dd4a6506af75039205c444251d11a62acb", + "eval/datasets/coding_memory_v1/oracles/island-green--scope_boundaries.py": "257cd579af07491748f5de88d04f8af85056b0448d348e59c7b0cbb28754fb1b", + "eval/datasets/coding_memory_v1/oracles/island-green--temporal_history.py": "e93105f0639a83b300a6de3bbae56166eedac2e58122f7e84ce5a40a074db6b6", + "eval/datasets/coding_memory_v1/oracles/island-green--unsupported_questions.py": "a707cee90fb3f0cabd0e692399a361f59da0fc4e9e7bbf0cde702e05579f0b3f", + "eval/datasets/coding_memory_v1/oracles/island-north--code_relationships.py": "f00e022ce900e4ad0fcec93f2dbf016337a4d83afacbcf739f61f71316723173", + "eval/datasets/coding_memory_v1/oracles/island-north--condition_values.py": "c704a03a70253585a67823826dc0a6ebaf6323a8880c00af80c148c150176daf", + "eval/datasets/coding_memory_v1/oracles/island-north--corrections.py": "e2cadf6e4f365f94cf0f440758b173da4d14ba12c03a84351e0608f0c60bccc5", + "eval/datasets/coding_memory_v1/oracles/island-north--long_documents.py": "300246fd3d85d73b63066d133361bc784ae26b46f4197c5c41ac960eb37f9508", + "eval/datasets/coding_memory_v1/oracles/island-north--multilingual.py": "f8aba8e14ed3c420e90e701880ea38e924e1219e3cc3643ba8bde17cb31ddef6", + "eval/datasets/coding_memory_v1/oracles/island-north--paraphrases.py": "503ce2c132c666bb67920e90d4a898f5a73f1d66779cd83c1885812f223e30e5", + "eval/datasets/coding_memory_v1/oracles/island-north--poisoning.py": "3369d956e5af12bc6be80828c4c6ab03ccf80c79e38dea3162b4e2c09771ff30", + "eval/datasets/coding_memory_v1/oracles/island-north--scope_boundaries.py": "9b4f6186f71713175f1eb5aacb0b6258c746b2297ca7c692f54bf5feabd5d261", + "eval/datasets/coding_memory_v1/oracles/island-north--temporal_history.py": "7c22b0edbf8edb7a511a581ffe6f1ce14454ef3feb527eca90d1386ce00b2c6b", + "eval/datasets/coding_memory_v1/oracles/island-north--unsupported_questions.py": "c1e5d526b0c6b88394f19a8b8edca7dae1db718b24be2d57374eb3769bbd1f24", + "eval/datasets/coding_memory_v1/oracles/island-violet--code_relationships.py": "9672d608ea0c8623ff1fe760f943173b2fdac5dc4ea4b7ae0dba33436d860254", + "eval/datasets/coding_memory_v1/oracles/island-violet--condition_values.py": "cfea91adcc9b3f9eb28cc24cf310e10b436f81cff51297f161b9a3b59a21f62c", + "eval/datasets/coding_memory_v1/oracles/island-violet--corrections.py": "075d48a2dfd91af8328982fe475ec915c00b75db38819aff3b4c2343147e2bfc", + "eval/datasets/coding_memory_v1/oracles/island-violet--long_documents.py": "85cc789277e59925b36ea3bd82e23f989e79fb0be2e55083e2656e518641dd3c", + "eval/datasets/coding_memory_v1/oracles/island-violet--multilingual.py": "84615b70eb74eb2180dd8a89d8d34c11e904030bfaa26c57e2e6c968daf64d29", + "eval/datasets/coding_memory_v1/oracles/island-violet--paraphrases.py": "5244d32ffc4127d519e2f5a875ffaecf7bc51eaa1426d2aacbc42490cf291d22", + "eval/datasets/coding_memory_v1/oracles/island-violet--poisoning.py": "9bde478876ea0be60555d5bd26d3790e4f305327ece10bf6c258bc564354fa1d", + "eval/datasets/coding_memory_v1/oracles/island-violet--scope_boundaries.py": "8e8b0a64342963b4dbc000578a72c0e0801fc1ca4af140f09b55d4cdb03b1274", + "eval/datasets/coding_memory_v1/oracles/island-violet--temporal_history.py": "3e187ce259c7f57f6dba4a28c3ecdf31fbe6458f7d908b22cb8b5ce8325437ae", + "eval/datasets/coding_memory_v1/oracles/island-violet--unsupported_questions.py": "0e60f29a47953c5d434c159eece26bbb03cc837842c493111b34310f9cff4693", + "eval/datasets/coding_memory_v1/oracles/island-west--code_relationships.py": "c19c42978fa5f9c6e1f41886eb1cc0fdb7e12304ba58b160b4cbe1975640acfe", + "eval/datasets/coding_memory_v1/oracles/island-west--condition_values.py": "07c4c591e08f5248f3016dc16cecfea7dd75bebff1a61599f258938d7031043d", + "eval/datasets/coding_memory_v1/oracles/island-west--corrections.py": "a7d93dfb579ec4f58cf2f6d29a6d0ab7dbf1a17f6ae49b98902a5ea22525e649", + "eval/datasets/coding_memory_v1/oracles/island-west--long_documents.py": "20a50dd63e04ce751115fca671b9da7ebbf240e26f39d494583cb405a03b750c", + "eval/datasets/coding_memory_v1/oracles/island-west--multilingual.py": "65d82c4d7968e0e597509c692015bdac0b92841c726d87560d9caa32429176f4", + "eval/datasets/coding_memory_v1/oracles/island-west--paraphrases.py": "feb7eb2e8592162d7d1f433610cb02ba04cd9797191d65a9f0e5bc4f4711c4ed", + "eval/datasets/coding_memory_v1/oracles/island-west--poisoning.py": "3d7c2c6a7f247f3e8158d5ba007bbbdf027a74e9a2c52f52953b3ccf2c21f17f", + "eval/datasets/coding_memory_v1/oracles/island-west--scope_boundaries.py": "ff0f36139f6da5428b9538d108aded522e62ca803035ebef78e11255cf55e08b", + "eval/datasets/coding_memory_v1/oracles/island-west--temporal_history.py": "42cb1c552ce01308b47d526c6e6993fb3270239b392cfa9f5e42b8e0fcfb3f82", + "eval/datasets/coding_memory_v1/oracles/island-west--unsupported_questions.py": "911b4257380640d35d6e2fc5292e235eedc113f5e71d8e71e8ccab7c72d4f290", + "eval/datasets/coding_memory_v1/oracles/juniper-green--code_relationships.py": "373395d3b70f64f8102609d27ce10c4aeb937ac73988c2ce2be1e283df94ef82", + "eval/datasets/coding_memory_v1/oracles/juniper-green--condition_values.py": "f04794a96b06d537550227ec308b1f02405cdab42d7d3ae4db4afd2c25bee8cd", + "eval/datasets/coding_memory_v1/oracles/juniper-green--corrections.py": "5a354a8c8efdbc498d25ffbf6095fee7b3d7d638134b452c7c9ecb186b4e1bc7", + "eval/datasets/coding_memory_v1/oracles/juniper-green--long_documents.py": "8610157668d6a90c794a956f6f0b432f03e3559238599ad1cb412f6f1a2f7e46", + "eval/datasets/coding_memory_v1/oracles/juniper-green--multilingual.py": "dcb99335fa18b008197532809ff7127be2cda600848e10c8bfe4677bd896f90e", + "eval/datasets/coding_memory_v1/oracles/juniper-green--paraphrases.py": "7a4cb1bbe3de2e6fe6a33b1f78829f4aeb21d21c2e8d4d3399d3d98958ba0909", + "eval/datasets/coding_memory_v1/oracles/juniper-green--poisoning.py": "ba73c2c6f04590538e16c50be527da60f5949781bc8833a3c18a7c597146eb10", + "eval/datasets/coding_memory_v1/oracles/juniper-green--scope_boundaries.py": "9d09c0851a3906b6281cb1c5811714224161e27a1aa46edd81c5659e7f93cd3d", + "eval/datasets/coding_memory_v1/oracles/juniper-green--temporal_history.py": "fd05595a452952dca2d0089f5701a60145a3b5705b78a5ff2f8f9f13fbf1c259", + "eval/datasets/coding_memory_v1/oracles/juniper-green--unsupported_questions.py": "f58bfcb89db799dc1578fc8791b49e97e7f22b30e720aeaf661f0ad2e6a9660c", + "eval/datasets/coding_memory_v1/oracles/juniper-north--code_relationships.py": "a84f02bd3c2b9c367248dae8aa3580d83de6a1beb6fe464553eb8a2bfb172b75", + "eval/datasets/coding_memory_v1/oracles/juniper-north--condition_values.py": "d70311ae339556f18240f824d0f0bd0024060c2698fcca8fe7cb540e8a8bc727", + "eval/datasets/coding_memory_v1/oracles/juniper-north--corrections.py": "fa290629339007fdd4230a76d49fa294a218c5f0cf6b90c3074678b0f16c539a", + "eval/datasets/coding_memory_v1/oracles/juniper-north--long_documents.py": "e30025d68ae3b48b566bf3fb2aea5e3603f5f7ab658da8fcb2256d3b1964c85a", + "eval/datasets/coding_memory_v1/oracles/juniper-north--multilingual.py": "b8bb0199c4acfe71df8d4e7df75bdd37e038cc72aa82bc6df10715819aeb3686", + "eval/datasets/coding_memory_v1/oracles/juniper-north--paraphrases.py": "aabf70509d3b0602d43c9590f04fa3ad1370a40c266d20f64a5ece15d4c79a51", + "eval/datasets/coding_memory_v1/oracles/juniper-north--poisoning.py": "ebfb9d7bf3c50eed8f44b73e391496ef377942de389b6feb64c7a1c0f9646444", + "eval/datasets/coding_memory_v1/oracles/juniper-north--scope_boundaries.py": "06c2c9d57ade79139c9b0d20568703477bb9e3a934f2f8900ad717ef157e5844", + "eval/datasets/coding_memory_v1/oracles/juniper-north--temporal_history.py": "0d97ba6d7a5d7741bf248b599ed18ae81b1c0053ad53cbc64934f31e9c3461aa", + "eval/datasets/coding_memory_v1/oracles/juniper-north--unsupported_questions.py": "526648ba7a4edc21bc2eb589c1e5c98a20fb07aa692687de4c3845e559be8096", + "eval/datasets/coding_memory_v1/oracles/juniper-violet--code_relationships.py": "c73fd24d0adbeda6c7afeb15ab07296c0438402893e8d442af26865106d6e8ea", + "eval/datasets/coding_memory_v1/oracles/juniper-violet--condition_values.py": "4aadae570ef84f976d49ba1fc92574ef55630b1c3017e90dbec1f85e6fb74c4a", + "eval/datasets/coding_memory_v1/oracles/juniper-violet--corrections.py": "07236b2a88df73c8d0f6afbe7d972fdc18f8c835a852f5c20748a369c77d82e2", + "eval/datasets/coding_memory_v1/oracles/juniper-violet--long_documents.py": "54d3201b621b168ce03963c423942f142cb403f9d90234f063ff33b7add4d2dc", + "eval/datasets/coding_memory_v1/oracles/juniper-violet--multilingual.py": "e0cc26d743cbea8e98261895474879d078fd53941c075606bfc6e52ee098db93", + "eval/datasets/coding_memory_v1/oracles/juniper-violet--paraphrases.py": "a1f836316408052cab1253b56b5caa6b1432fe6eeaabbb5872da97ec9bde633a", + "eval/datasets/coding_memory_v1/oracles/juniper-violet--poisoning.py": "bc09271eadf77f1b6c03fff20ca9c0f0a42a99d1c5600e6b7843f0a0d549ed5d", + "eval/datasets/coding_memory_v1/oracles/juniper-violet--scope_boundaries.py": "17a7258bed9efe1c86a05756c5b5d17926fd72f01de4b2addd5d2fb4550f4da5", + "eval/datasets/coding_memory_v1/oracles/juniper-violet--temporal_history.py": "8bda8599d861857204c48b55729607c8cb0cb3ae2701eb18fb600d398468cafe", + "eval/datasets/coding_memory_v1/oracles/juniper-violet--unsupported_questions.py": "10cdbb013f78f0e54c31a6e76f03d8d15a6063fd7971a2d803a390890c036ad2", + "eval/datasets/coding_memory_v1/oracles/juniper-west--code_relationships.py": "f0a682d413b3d8418c664db2775de4eef35240c508cc71ca6bbc055cd0c8dbe6", + "eval/datasets/coding_memory_v1/oracles/juniper-west--condition_values.py": "200a4b78f201e992ac10112a194c97b88841dc5930e3e7de9bfc076bf4c69fa7", + "eval/datasets/coding_memory_v1/oracles/juniper-west--corrections.py": "41ea9b5a5f40d8703343f4570593c3f27905e5ff70fae57bdbdb5fcccc585d22", + "eval/datasets/coding_memory_v1/oracles/juniper-west--long_documents.py": "7d8cdd01c5d90ad62e9da93a99febead683e3f01c5ca37315382a2fb0f512d76", + "eval/datasets/coding_memory_v1/oracles/juniper-west--multilingual.py": "1815adc5946bf9e7dba16786d8c29f9466537b7a43b1c392f927b61c56da54ee", + "eval/datasets/coding_memory_v1/oracles/juniper-west--paraphrases.py": "5d0110bc2098d2143303476fb5c5714086d0278b848110c63aa626e0b88311cc", + "eval/datasets/coding_memory_v1/oracles/juniper-west--poisoning.py": "4e16d9de9931cffc6915805b4fc375dad3ffb7fad17eb893205ce6d07102bcb3", + "eval/datasets/coding_memory_v1/oracles/juniper-west--scope_boundaries.py": "6c5d875184c890a922ee18ac423598fdcece109a91e9e62d5a7ccc82d2eaaa70", + "eval/datasets/coding_memory_v1/oracles/juniper-west--temporal_history.py": "3404fdbb5cc4693333d48d27da928531279d148ffe1cdadbdd741dfc1a8d38d3", + "eval/datasets/coding_memory_v1/oracles/juniper-west--unsupported_questions.py": "b85a4cf14f9596cd14b480acd323772fc91084ad1413fa27435bdb7ffe5baa6b", + "eval/datasets/longdoc.jsonl": "7f5ade95e1f283d0db8cf78e53ed8995d3534f847e616d2c0005fd8da37ac790", + "eval/engine_capacity.py": "1940da1c3a435d81c7b7a1a689e966c23b92e4f94af115b45813dd30e2c04eec", + "eval/evidence_contracts.py": "03969fdf01e78135c66e698d3f523e5312ccca3737dd3036b37a2c6676ba4ea2", + "eval/external.py": "98500a97c18152b1b86a270062033c48176224d7468ac0f703ddc7de862a30e2", + "eval/external_checkpoints.py": "484e298039bfa04eef86bd7145428c312aaeaf059f5bdf8da7ae29d1150008d7", + "eval/extractor_quality.py": "50820fe7f821d111e17e4e77a3d1159e7d6979d110b052c4f254a5b309c2652a", + "eval/fts_insert_scaling.py": "6088997e0f85d430fafb74509e24a960fbd59c06c8b2839285942b97d2116724", + "eval/graph_every_bench.py": "79da573c5edf315f71bfab412d3ea283b8da45d1fdabc78ddd19302ead09e4f0", + "eval/graph_traversal.py": "b094f75c3a1d75ba3cf19e372187d692d3c595a1d9bde19bbe95ae0c79a5175f", + "eval/grounded.py": "053d5193b716a2c3e507fcd44057d392de910a4442b46bd7cc1f30ac0ba68541", + "eval/handoff_quality.py": "7daf635510764e236f48ca7e2537a85d8ebc1ad995513144329c1f0236405937", + "eval/harness.py": "8c96c26a121dfc2d9ea051a05861af8951c93a732dba9cb13de0de178414b016", + "eval/hosted_evidence.py": "7946cd8c1e3aa291268271b2aa11210d5f09c9b05ba635aa0b64bef07bdcee45", + "eval/hosted_ledger.py": "a53036d12ff671371c148910a816fb20c7e7f3346250a303722352f47b06c476", + "eval/hosted_luna.py": "4dbf02a65eec38bcde92d82952a0b372abbac1f68378d11a04528f4a31a835dc", + "eval/jev_recall_quality.py": "dad219ca883f9258f9f77449ff2a7f355a0352c7c0918bfd0e58d9e002daeac4", + "eval/local_benchmark_queue.py": "43fa67b4d653e770822da816511e8e62b3714e60d2edd49ea2a4c5c3d8d20851", + "eval/local_capacity_campaign.py": "ab4935263f9d24c4e38bd4f16164b4c4c07eff91c695809b6fc75299d4f03d62", + "eval/longmemeval_v2.py": "defb4d47f453aa4615a8f101b82df3fadf64f0f9eae0ce1021d86d3750dad437", + "eval/longmemeval_v2_evidence.py": "8486bd4dcfdd8b1f7a14c32fe4f427c980ccdcc3aa2eb018d7dfc40305412eff", + "eval/longmemeval_v2_matrix.py": "ca085cd59481813cce5dbdfbc94f40f67173ac3a1bc3dce6f6d3e09eb7b08153", + "eval/metrics.py": "16857e2cf6ed339cb57a26c9bfa1879444b4d279bd972e5a9fa644ed1308afe0", + "eval/native_coverage_scaling.py": "0d318c116241050fc0c7bdbbb5646ca67944d7c304ccd9a462834e0553ffa90a", + "eval/performance.py": "dccc55c26dc396f9fee98defd152900cf8aabeb5d6198bc711e0eb8f472ecc57", + "eval/performance_engine.py": "3d37cf0a5989c8fa6e8b6ab7ae0b2e0d5410a8b922d2f3130c1a7794d93dcbb1", + "eval/planned_recall.py": "f9a87291ecb181045b98ca65fe55db820bf7cee4ae4f4d087ca3458afdd7837e", + "eval/proactive_ranking.py": "8610541f1d547f9c0eb46d078dbcaa670c0a97157acc37f96ec08b482cb7a6ab", + "eval/productivity.py": "6d4644ebdc44472aeb3879963774fab276bfa269b781139c46ded48774a22717", + "eval/public_readiness.py": "5ce8a18d0bfe09e75e88548a589b8fc6d2cbf04c0f8cd1ce51cd9519136a2751", + "eval/redteam_poisoning.py": "fce120cc3adf2ee966b59cd3ea7f20d242a0af49b52492143543130fe14c0cc8", + "eval/reinforcement.py": "72ed766775a2658eaa728afec51c0ac22d97a90e813111954df66a6ec50f2bef", + "eval/repair_discovery.py": "db05496fbbcb0df86c5cdc2f0c85fb6b6605b0cace6cb44ae2add10b72784b5f", + "eval/resolver_reworded_corrections.py": "a9054778a37b2175b46f04b674b4779e358931eee5d2ae52bbc5f953d234fb9e", + "eval/resource_hierarchy.py": "5ab6c989bb143c4386749c45a33c190447e829c1b5657e8c0aca30f34bd69461", + "eval/rework_statistics.py": "e12c14288797c5cf2dd93d51f606244287bc2f4ff8bd401f1d1b072efd1f9529", + "eval/run_longmemeval_v2.py": "866863d9f8f7ce8f7c3c36741ab324faa5fae417827f907c655eace38de0f5ac", + "eval/task_pairs.py": "fddc54804e8837ec0731813297fb55825458317f898e29176e16f3f5a2f527fd", + "eval/user_journeys.py": "a1c8436d4a6871baff21f9aa4f6ac1545def60d946cd3b1454c3d9cd8661a049", + "eval/vector_scale.py": "3f9c327d9eca1a857512ddc208e972933be1a7d4fa7fa0017aca0cbf8fe7bb6d", + "eval/vector_scale_storage.py": "24040fd1b96b9f9cd43ea37b0b37ea118a02bd096aff77fc67ff8b0df769b1dc", + "eval/vector_scan_plan.py": "34fba3d029bfc78134e8c9c450b019ff56c3bdcbeb921400888077cb44e9b848", + "scripts/export_offline_evidence.py": "4e10c2b3d5f6a2024a7f95f7b87ce47ec2cb4ef31c55611b2e5cf35502ebc0ba" + } + } +} diff --git a/docs/benchmark-evidence/offline-fixtures-v135.json.sha256 b/docs/benchmark-evidence/offline-fixtures-v135.json.sha256 new file mode 100644 index 00000000..180d0ce4 --- /dev/null +++ b/docs/benchmark-evidence/offline-fixtures-v135.json.sha256 @@ -0,0 +1 @@ +8bba1a3cd99d9659bc8a2b1df08c9edfade5053f112b7438a10f09d93dc03133 offline-fixtures-v135.json diff --git a/engraphis/backends/jev_decision.py b/engraphis/backends/jev_decision.py index 30d5370c..42d8daa7 100644 --- a/engraphis/backends/jev_decision.py +++ b/engraphis/backends/jev_decision.py @@ -156,7 +156,7 @@ def _evaluate( purpose: str, data_classification: str, timeout_s: Optional[float] = None, ) -> tuple[Optional[DecisionBatch], str, Optional[str]]: - from engraphis.backends.jev_transport import contains_sensitive_content + from engraphis.backends.jev_transport import DecisionClientError, contains_sensitive_content if allow_remote is not True: return None, "fallback", "remote_not_authorized" @@ -215,6 +215,11 @@ def _evaluate( return None, "fallback", "provider_fallback" batch = cast(DecisionBatch, response) return batch, "decision", None + except DecisionClientError as exc: + # Preserve the local eligibility denial without exposing provider details. + reason = ("managed_operation_unsupported" if exc.code == "managed_operation_unsupported" + else "remote_unavailable") + return None, "fallback", reason except Exception: # Provider exceptions may contain request text or credentials. Do not log them. return None, "fallback", "remote_unavailable" diff --git a/engraphis/backends/jev_query_planner.py b/engraphis/backends/jev_query_planner.py index a93f0e36..6940fe95 100644 --- a/engraphis/backends/jev_query_planner.py +++ b/engraphis/backends/jev_query_planner.py @@ -163,5 +163,6 @@ def _fallback_code(reason: Optional[str]) -> str: "provider_fallback": "jev_provider_fallback", "remote_unavailable": "jev_remote_unavailable", "malformed_response": "jev_malformed_response", + "managed_operation_unsupported": "jev_managed_operation_unsupported", } return allowed.get(reason or "", "jev_fallback") diff --git a/engraphis/backends/jev_transport.py b/engraphis/backends/jev_transport.py index b20d83d8..b3b79fcc 100644 --- a/engraphis/backends/jev_transport.py +++ b/engraphis/backends/jev_transport.py @@ -16,6 +16,7 @@ from dataclasses import dataclass, field from typing import TYPE_CHECKING, Dict, Optional, Sequence from urllib.parse import urlsplit +from uuid import uuid4 from engraphis.http_deadline import ( deadline_handlers as _deadline_handlers, @@ -222,6 +223,72 @@ def _timeout(value: float) -> float: return float(value) +def _validate_managed_context( + state: str, questions: Sequence[DecisionQuestion], purpose: str, +) -> None: + """Match Cloud's fixed advisory questions and concrete context schemas. + + Known MCP and direct-adapter wire forms remain compatible. A purpose label + cannot turn arbitrary questions into an included managed operation. These + checks do not establish the provenance or truth of supplied plaintext. + """ + expected = { + "guard_command": ( + ("is_safe", "Is this command free of destructive data loss or secret leakage?", + "noul", ()), + ("category", "Categorize this operation", "choice", + ("read_only", "state_change", "destructive_or_leak")), + ), + "classify_contradiction": ( + ("verdict", "Classify the relationship between the facts.", "choice", + ("contradicts_and_supersedes", "reinforces", "orthogonal")), + ), + "verify_support": ( + ("has_support", "Does this evidence directly support answering the query?", + "noul", ()), + ), + "verify_completion": ( + ("is_complete", "Does the supplied evidence establish the task goal?", "noul", ()), + ), + }.get(purpose) + if expected is None: + raise DecisionClientError("managed_operation_unsupported") + actual = tuple((q.id, q.prompt, q.kind, tuple(q.options)) for q in questions) + fields = [(state, 16000)] + if purpose in {"classify_contradiction", "verify_support"}: + if purpose == "classify_contradiction": + prefix, delimiter = "EXISTING FACT: ", "\nNEW CANDIDATE FACT: " + direct_prompt = "Classify the relationship between the candidate and existing fact." + direct_delimiter = "\n\nNEW CANDIDATE FACT:\n" + first_limit = 16000 + else: + prefix, delimiter = "QUERY: ", "\nEVIDENCE: " + direct_prompt = "Does the evidence directly support answering the query?" + direct_delimiter = "\n\nEVIDENCE:\n" + first_limit = 4096 + first = expected[0] + direct = ((first[0], direct_prompt, first[2], first[3]),) + if actual == direct: + expected, delimiter = direct, direct_delimiter + if not state.startswith(prefix) or state.count(delimiter) != 1: + raise DecisionClientError("invalid_request") + before, after = state[len(prefix):].split(delimiter) + fields = [(before, first_limit), (after, 16000)] + elif purpose == "verify_completion": + if (not state.startswith("GOAL: ") or state.count("\nACTIONS: ") != 1 + or state.count("\nOUTPUT: ") != 1): + raise DecisionClientError("invalid_request") + goal, remaining = state[len("GOAL: "):].split("\nACTIONS: ") + if "\nOUTPUT: " not in remaining: + raise DecisionClientError("invalid_request") + actions, output = remaining.split("\nOUTPUT: ") + if len(actions) > 8192: + raise DecisionClientError("invalid_request") + fields = [(goal, 4096), (output, 16000)] + if actual != expected or any(not value.strip() or len(value) > limit for value, limit in fields): + raise DecisionClientError("invalid_request") + + def _read_response(response, deadline: float) -> bytes: raw = _read_deadline_response(response, deadline, max_bytes=MAX_RESPONSE_BYTES) if len(raw) > MAX_RESPONSE_BYTES: @@ -328,11 +395,26 @@ def allow_fallback(self) -> bool: def evaluate(self, state: str, questions: Sequence[DecisionQuestion], *, model: str, allow_remote: bool = False, purpose: str = "custom", data_classification: str = "internal", - timeout_s: Optional[float] = None) -> CloudDecisionBatch: + timeout_s: Optional[float] = None, + request_key: Optional[str] = None) -> CloudDecisionBatch: + """Evaluate once; callers can reuse an explicit key after a lost reply. + + Cloud charges evaluated questions and rejects duplicate keys without a + second provider call. This client never retries automatically. + """ effective_timeout = min(self.timeout_s, _timeout(timeout_s)) if timeout_s is not None else self.timeout_s deadline = time.monotonic() + effective_timeout payload = _request_payload(state, questions, model, allow_remote=allow_remote, purpose=purpose, data_classification=data_classification) + _validate_managed_context(state, questions, purpose) + if request_key is not None and ( + not isinstance(request_key, str) + or not re.fullmatch(r"[A-Za-z0-9_-]{16,64}", request_key) + ): + raise DecisionClientError("invalid_request") + payload["request_key"] = request_key if request_key is not None else uuid4().hex + if len(json.dumps(payload, ensure_ascii=False).encode()) > MAX_REQUEST_BYTES: + raise DecisionClientError("invalid_request") from engraphis import cloud_session from engraphis.hosted_client import validate_cloud_base_url try: diff --git a/engraphis/mcp_server.py b/engraphis/mcp_server.py index e72121eb..d5a8d53d 100644 --- a/engraphis/mcp_server.py +++ b/engraphis/mcp_server.py @@ -454,11 +454,14 @@ def minimum_role(tool_name: str) -> str: dynamic role, discovered reads stay viewer-accessible while the generic stateful executor fails closed to admin. Local stdio has no role boundary and retains the owner's full capability; routine remote member writes remain available through the - dedicated session and remember tools. Optional remote decisions may consume account - allowance, so their direct tool uses the default member requirement too. + dedicated session and remember tools. Direct advisory decisions are viewer-accessible + so an entitled viewer can use their individual allowance. They still consume quota + and remain stateful for Smart discovery and execution. """ if tool_name in _SMART_GATEWAY_ROLES: return _SMART_GATEWAY_ROLES[tool_name] + if tool_name == "engraphis_decide": + return "viewer" if tool_name in _ADMIN_TOOLS: return "admin" if tool_name in _READ_ONLY_TOOLS: diff --git a/skills/engraphis-memory/references/TOOLS.md b/skills/engraphis-memory/references/TOOLS.md index 03bbbe24..e3bf49b2 100644 --- a/skills/engraphis-memory/references/TOOLS.md +++ b/skills/engraphis-memory/references/TOOLS.md @@ -130,7 +130,8 @@ bodies already represented in `context`. original query plus at most two planner routes, with strict per-route and cumulative bounds. - `jev_assisted (bool, false)`: opt in to Jev prioritizing a route generated by the local deterministic planner. Requires `planning="auto"`; scope, time, type, and trust filters remain - unchanged. + unchanged. Remote route selection requires explicit BYOK; managed `query_planning` fails closed + before credential refresh or network requests and keeps deterministic order with a visible fallback. - `allow_remote (bool, false)` and `data_classification (str, None)`: per-call consent for a Jev request; remote use requires `true` plus `public` or `internal`. The provider receives the query and bounded routes, not retrieved memory bodies. Uncertain or failed requests use deterministic @@ -181,7 +182,9 @@ It is the full-response compatibility surface; prefer `engraphis_recall_context` - `diagnostics (bool, false)`: include `retrieval_trace` with raw/normalized/fusion/rerank data. - `planning (str, "off")`: `off` preserves single-query recall; `auto` enables bounded planning. - `jev_assisted (bool, false)`: opt in to Jev prioritizing one alternate route from the local - deterministic planner. Requires `planning="auto"`; Jev cannot change retrieval filters. + deterministic planner. Requires `planning="auto"` and explicit BYOK; Jev cannot change retrieval + filters. Managed `query_planning` fails closed before credential refresh or network requests and + keeps deterministic order with a visible fallback. - `allow_remote (bool, false)` and `data_classification (str, None)`: per-call consent for a Jev request; remote use requires `true` plus `public` or `internal`. The provider receives the query and bounded routes, not recalled memories. Uncertain or failed requests keep deterministic order. @@ -536,6 +539,8 @@ Smart `engraphis_recall_context` accepts only `query`, `workspace`, `repo`, `ses `allow_remote` and `data_classification`; advanced planning/profile controls are discoverable rather than routine. Setting `allow_remote=true` with a `public` or `internal` classification both opts in to bounded route selection and grants consent for that call only. Defaults stay deterministic/local. +Remote route selection requires explicit BYOK. Managed and `auto` do not admit `query_planning` +or silently select BYOK; unsupported managed planning retains deterministic local order. ### `engraphis_session` Start or resume a session, or end it with a next-session handoff. @@ -696,6 +701,26 @@ Managed availability and allowance require service verification. No latency, acc savings guarantee follows from configuration. Smart discovery uses `engraphis_execute_action` because a remote request may consume allowance. +Managed access admits only `guard_command`, `classify_contradiction`, `verify_support`, and +`verify_completion`, with the concrete context below and fixed question schemas. A purpose label +does not authorize arbitrary questions. Managed `custom` and `query_planning` fail closed before +credential refresh or network requests; custom remote questions and experimental route selection +require explicit BYOK. Neither managed nor `auto` silently switches to a personal provider key. +Every paid Pro user and paid Team named seat, including viewers, and active legitimate trial/test +entitlements receives 100 evaluated questions per rolling hour, 1,000 per rolling five hours, +and 2,000 per rolling 24 hours. All three caps apply to the individual; there is no monthly or +Team pool and no extra managed charge or personal provider key requirement. Command review +evaluates two questions; the other three workflows evaluate one each. Admitted failures remain +counted. The unchanged production fleet guard of 100 questions/day remains a launch conflict. +Release acceptance, provider terms, and live quality evaluation remain gates; synthetic fixtures +do not demonstrate model accuracy. + +Authorized viewers can use direct Classic advisory decisions without gaining memory writes or +administration. The generic Smart stateful executor still requires admin, and the private hosted +Team tool catalog is unchanged. The managed transport accepts an optional `request_key` for an +explicit retry: duplicate admitted keys return 409 without a second provider call or use increment. +There is no stored answer replay or automatic retry; the MCP tool does not expose this parameter. + - `kind (str, "guard_command")`: one of `'guard_command'`, `'classify_contradiction'`, `'verify_support'`, `'verify_completion'`, or `'custom'`. - `state (str, "")`: nonblank shell command, candidate fact, or evidence text; required except when `custom` supplies `question`. - `query (str, "")`: nonblank query required for support verification. diff --git a/tests/test_jev_configuration.py b/tests/test_jev_configuration.py index f7a37ba6..1a103fc5 100644 --- a/tests/test_jev_configuration.py +++ b/tests/test_jev_configuration.py @@ -105,7 +105,7 @@ def open_request(request, timeout): requests.append(request) assert 0 < timeout <= client.timeout_s response = io.BytesIO(json.dumps({ - "model": transport.MODEL, "is_fallback": False, "decisions": {"q": { + "model": transport.MODEL, "is_fallback": False, "decisions": {"has_support": { "type": "noul", "probability": 0.9, "confidence": 0.8, "confidence_source": "derived_decisiveness", }}, @@ -117,13 +117,15 @@ def open_request(request, timeout): monkeypatch.setattr(socket, "getaddrinfo", resolve) monkeypatch.setattr(hosted_client, "build_pinned_https_opener", lambda *handlers: SimpleNamespace(open=open_request)) - batch = client.evaluate("Synthetic evidence", [DecisionQuestion("q", "Assess", "noul")], - model=transport.MODEL, allow_remote=True) - assert batch.get_noul("q").probability == 0.9 + state = "QUERY: Which evidence supports this fact?\nEVIDENCE: Synthetic evidence" + batch = client.evaluate(state, [DecisionQuestion( + "has_support", "Does this evidence directly support answering the query?", "noul", + )], model=transport.MODEL, allow_remote=True, purpose="verify_support") + assert batch.get_noul("has_support").probability == 0.9 assert len(requests) == 1 and resolutions assert requests[0].full_url == control + "/v1/jev/decide" assert requests[0].get_header("Authorization") == "Bearer synthetic-direct-access" - assert json.loads(requests[0].data)["state"] == "Synthetic evidence" + assert json.loads(requests[0].data)["state"] == state assert direct_credentials == [] diff --git a/tests/test_jev_origin_binding.py b/tests/test_jev_origin_binding.py index e03714e3..1765d170 100644 --- a/tests/test_jev_origin_binding.py +++ b/tests/test_jev_origin_binding.py @@ -53,7 +53,7 @@ def post(url, token, payload, timeout_s, *, deadline): assert token == f"synthetic-access-{len(calls['refresh'])}" assert deadline > transport.time.monotonic() calls["decision"].append(url) - return {"model": transport.MODEL, "is_fallback": False, "decisions": {"q": { + return {"model": transport.MODEL, "is_fallback": False, "decisions": {"has_support": { "type": "noul", "probability": 0.9, "confidence": 0.8, "confidence_source": "derived_decisiveness", }}} @@ -66,8 +66,9 @@ def post(url, token, payload, timeout_s, *, deadline): def _evaluate(client): - return client.evaluate("Synthetic evidence", [DecisionQuestion("q", "Assess", "noul")], - model=transport.MODEL, allow_remote=True) + return client.evaluate("QUERY: Which evidence supports this fact?\nEVIDENCE: Synthetic evidence", [ + DecisionQuestion("has_support", "Does this evidence directly support answering the query?", "noul"), + ], model=transport.MODEL, allow_remote=True, purpose="verify_support") @pytest.mark.parametrize("source", ["environment", "saved"]) @@ -93,7 +94,7 @@ def dns(host, *args, **kwargs): monkeypatch.setattr(socket, "getaddrinfo", dns) assert client.is_configured for _ in range(2): - assert _evaluate(client).get_noul("q").probability == 0.9 + assert _evaluate(client).get_noul("has_support").probability == 0.9 assert "unavailable-compute.example.test" not in resolved saved = cloud_session._load() assert saved["compute_url"] == compute @@ -136,7 +137,7 @@ def test_first_managed_decision_uses_canonical_bootstrap_and_reuses_rotation( ): client, calls = bootstrap(raw, canonical, source=source) assert client.is_configured - assert _evaluate(client).get_noul("q").probability == 0.9 + assert _evaluate(client).get_noul("has_support").probability == 0.9 saved = cloud_session._load() assert saved["control_url"] == canonical assert saved["refresh_credential"] == "synthetic-rotated-1" @@ -144,7 +145,7 @@ def test_first_managed_decision_uses_canonical_bootstrap_and_reuses_rotation( # Persisted family binding wins over an environment endpoint replacement. monkeypatch.setenv("ENGRAPHIS_CLOUD_CONTROL_URL", "https://unused.example.invalid") - assert _evaluate(client).get_noul("q").probability == 0.9 + assert _evaluate(client).get_noul("has_support").probability == 0.9 assert calls["refresh"] == [(canonical, "synthetic-bootstrap"), (canonical, "synthetic-rotated-1")] assert calls["decision"] == [canonical + "/v1/jev/decide"] * 2 diff --git a/tests/test_jev_transport.py b/tests/test_jev_transport.py index 031ff2dd..5b991026 100644 --- a/tests/test_jev_transport.py +++ b/tests/test_jev_transport.py @@ -15,6 +15,13 @@ def _question(kind="noul"): () if kind == "noul" else ("no", "yes")) +def _support_question(): + return DecisionQuestion("has_support", "Does this evidence directly support answering the query?", "noul") + + +SUPPORT_STATE = "QUERY: Which evidence supports this fact?\nEVIDENCE: A synthetic statement" + + def _normalized(probability=0.9): return {"model": transport.MODEL, "is_fallback": False, "decisions": {"q": { "type": "noul", "probability": probability, "confidence": abs(2*probability-1), @@ -41,7 +48,9 @@ def opener(*handlers): def open_request(request, timeout): calls.append(("request", request, timeout)) - response = io.BytesIO(json.dumps(_normalized()).encode()) + body = _normalized() + body["decisions"]["has_support"] = body["decisions"].pop("q") + response = io.BytesIO(json.dumps(body).encode()) response.status = 200 response.headers = {"Content-Type": "application/json"} return response @@ -54,8 +63,9 @@ def open_request(request, timeout): def test_constructor_and_configuration_are_network_free_and_managed_refresh_is_bound(managed): client = transport.create_cloud_decision_client() assert client.is_configured and managed == [] - batch = client.evaluate("A synthetic statement", [_question()], model=transport.MODEL, - allow_remote=True, purpose="verify_support", data_classification="public") + batch = client.evaluate(SUPPORT_STATE, [_support_question()], model=transport.MODEL, + allow_remote=True, purpose="verify_support", data_classification="public", + request_key="constructor_key_0001") assert managed[0][:2] == ("refresh", None) assert managed[0][2]["require_compute"] is False assert 0 < managed[0][2]["deadline"] - transport.time.monotonic() <= client.timeout_s @@ -64,11 +74,12 @@ def test_constructor_and_configuration_are_network_free_and_managed_refresh_is_b assert request.get_header("Authorization") == "Bearer synthetic-access-token" assert 0 < timeout <= 15 assert json.loads(request.data) == { - "model": transport.MODEL, "state": "A synthetic statement", "questions": [_question().to_dict()], + "model": transport.MODEL, "state": SUPPORT_STATE, "questions": [_support_question().to_dict()], "allow_remote": True, "purpose": "verify_support", "data_classification": "public", + "request_key": "constructor_key_0001", } - assert batch.get_noul("q").probability == 0.9 - assert batch.get_noul("q").confidence_source == "derived_decisiveness" + assert batch.get_noul("has_support").probability == 0.9 + assert batch.get_noul("has_support").confidence_source == "derived_decisiveness" @pytest.mark.parametrize("kwargs", ( @@ -106,7 +117,8 @@ def test_credential_origin_change_fails_without_using_token(managed, monkeypatch monkeypatch.setattr(cloud_session, "credential_bound_control_url", lambda: next(values)) with pytest.raises(transport.DecisionClientError, match="session_changed"): transport.create_cloud_decision_client().evaluate( - "Synthetic", [_question()], model=transport.MODEL, allow_remote=True, + SUPPORT_STATE, [_support_question()], model=transport.MODEL, + allow_remote=True, purpose="verify_support", ) assert len(managed) == 1 and managed[0][0] == "refresh" @@ -251,7 +263,8 @@ def test_https_loopback_managed_requests_disable_ambient_proxies(managed, monkey monkeypatch.setenv("HTTPS_PROXY", "http://proxy.invalid:8080") monkeypatch.setattr(cloud_session, "credential_bound_control_url", lambda: "https://localhost:8443") transport.create_cloud_decision_client().evaluate( - "Synthetic", [_question()], model=transport.MODEL, allow_remote=True, + SUPPORT_STATE, [_support_question()], model=transport.MODEL, + allow_remote=True, purpose="verify_support", ) handlers = next(value[1] for value in managed if value[0] == "handlers") assert any(isinstance(handler, urllib.request.ProxyHandler) and handler.proxies == {} @@ -270,7 +283,9 @@ def do_POST(self): requests.append((self.path, self.headers["Authorization"], json.loads( self.rfile.read(int(self.headers["Content-Length"])), ))) - body = json.dumps(_normalized()).encode() + normalized = _normalized() + normalized["decisions"]["has_support"] = normalized["decisions"].pop("q") + body = json.dumps(normalized).encode() self.send_response(200) self.send_header("Content-Type", "application/json") self.send_header("Content-Length", str(len(body))) @@ -312,14 +327,14 @@ def refresh(control_url, credential, workspace_id, token_subject, *, deadline): monkeypatch.setenv("ENGRAPHIS_CLOUD_CONTROL_URL", "http://other.invalid") client = transport.create_cloud_decision_client(timeout_s=2) assert client.is_configured - batch = client.evaluate("Synthetic local evidence", [_question()], model=transport.MODEL, - allow_remote=True, data_classification="public") - assert batch.get_noul("q").probability == 0.9 + batch = client.evaluate(SUPPORT_STATE, [_support_question()], model=transport.MODEL, + allow_remote=True, purpose="verify_support", data_classification="public") + assert batch.get_noul("has_support").probability == 0.9 assert len(requests) == 1 path, authorization, body = requests[0] assert path == "/v1/jev/decide" assert authorization == "Bearer synthetic-access" - assert body["state"] == "Synthetic local evidence" + assert body["state"] == SUPPORT_STATE assert cloud_session.credential_bound_control_url() == control assert cloud_session._load()["refresh_credential"] == "synthetic-rotated" finally: @@ -376,3 +391,156 @@ def read1(self, size=-1): transport._post_json("https://synthetic.invalid/v1/jev/decide", "synthetic", {}, 2.0) assert timeouts == [2.0, 1.25, 0.5] assert response.closed + + +def _managed_cases(): + contradiction = DecisionQuestion( + "verdict", "Classify the relationship between the facts.", "choice", + ("contradicts_and_supersedes", "reinforces", "orthogonal"), + ) + return ( + ("guard_command", "git status", [ + DecisionQuestion("is_safe", "Is this command free of destructive data loss or secret leakage?", "noul"), + DecisionQuestion("category", "Categorize this operation", "choice", + ("read_only", "state_change", "destructive_or_leak")), + ]), + ("classify_contradiction", "EXISTING FACT: Port 8000\nNEW CANDIDATE FACT: Port 9000", [contradiction]), + ("classify_contradiction", "EXISTING FACT: Port 8000\n\nNEW CANDIDATE FACT:\nPort 9000", [ + DecisionQuestion(contradiction.id, + "Classify the relationship between the candidate and existing fact.", + contradiction.kind, contradiction.options), + ]), + ("verify_support", SUPPORT_STATE, [_support_question()]), + ("verify_support", "QUERY: Which port?\n\nEVIDENCE:\nPort 8000", [ + DecisionQuestion("has_support", "Does the evidence directly support answering the query?", "noul"), + ]), + ("verify_completion", "GOAL: Fix the port\nACTIONS: \nOUTPUT: Port is corrected", [ + DecisionQuestion("is_complete", "Does the supplied evidence establish the task goal?", "noul"), + ]), + ) + + +@pytest.mark.parametrize("purpose,state,questions", _managed_cases()) +def test_managed_fixed_workflows_and_direct_adapter_variants_keep_legacy_wire_shape( + managed, monkeypatch, purpose, state, questions, +): + posted = [] + + def post(url, token, payload, timeout_s, *, deadline): + posted.append(payload) + return {"model": transport.MODEL, "is_fallback": True} + + monkeypatch.setattr(transport, "_post_json", post) + batch = transport.create_cloud_decision_client().evaluate( + state, questions, model=transport.MODEL, purpose=purpose, allow_remote=True, + request_key="compatible_request_0001", + ) + assert batch.is_fallback + assert posted == [{ + "model": transport.MODEL, "state": state, "questions": [q.to_dict() for q in questions], + "allow_remote": True, "purpose": purpose, "data_classification": "internal", + "request_key": "compatible_request_0001", + }] + assert len(managed) == 1 and managed[0][0] == "refresh" + + +@pytest.mark.parametrize("purpose,code", [ + ("custom", "managed_operation_unsupported"), ("query_planning", "invalid_request"), +]) +def test_arbitrary_managed_operations_are_rejected_before_refresh(managed, purpose, code): + with pytest.raises(transport.DecisionClientError, match="^" + code + "$"): + transport.create_cloud_decision_client().evaluate( + SUPPORT_STATE, [_support_question()], model=transport.MODEL, + purpose=purpose, allow_remote=True, + ) + assert managed == [] + + +@pytest.mark.parametrize("purpose,state,questions", ( + ("verify_support", SUPPORT_STATE, [DecisionQuestion("has_support", "Answer an arbitrary prompt", "noul")]), + ("verify_support", "Ordinary arbitrary context", [_support_question()]), + ("verify_support", "QUERY: \nEVIDENCE: Evidence", [_support_question()]), + ("verify_support", "QUERY: Query\nEVIDENCE: ", [_support_question()]), + ("verify_support", "QUERY: Query\nEVIDENCE: A\nEVIDENCE: B", [_support_question()]), + ("verify_support", "QUERY: " + "q" * 4097 + "\nEVIDENCE: Evidence", [_support_question()]), + ("verify_support", SUPPORT_STATE, [_support_question(), DecisionQuestion("other", "Other", "noul")]), + ("classify_contradiction", "EXISTING FACT: \nNEW CANDIDATE FACT: Candidate", _managed_cases()[1][2]), + ("classify_contradiction", "EXISTING FACT: Existing\nNEW CANDIDATE FACT: ", _managed_cases()[1][2]), + ("classify_contradiction", "EXISTING FACT: Existing\n\nNEW CANDIDATE FACT:\nCandidate", _managed_cases()[1][2]), + ("verify_completion", "GOAL: Goal\nOUTPUT: Output", _managed_cases()[-1][2]), + ("verify_completion", "GOAL: Goal\nACTIONS: Done\nOUTPUT: ", _managed_cases()[-1][2]), + ("verify_completion", "GOAL: " + "g" * 4097 + "\nACTIONS: \nOUTPUT: Output", _managed_cases()[-1][2]), + ("verify_completion", "GOAL: Goal\nACTIONS: " + "a" * 8193 + "\nOUTPUT: Output", _managed_cases()[-1][2]), + ("verify_completion", "GOAL: Goal\nOUTPUT: Output\nACTIONS: Done", _managed_cases()[-1][2]), + ("guard_command", "git status", [_support_question()]), +)) +def test_managed_purpose_laundering_incomplete_context_and_batches_fail_preflight( + managed, purpose, state, questions, +): + with pytest.raises(transport.DecisionClientError, match="^invalid_request$"): + transport.create_cloud_decision_client().evaluate( + state, questions, model=transport.MODEL, purpose=purpose, allow_remote=True, + ) + assert managed == [] + + +@pytest.mark.parametrize("key", ["", "short", "a" * 65, "a" * 15, "a" * 16 + " ", + "é" * 16, "path/request_key", True, 123]) +def test_invalid_request_key_fails_before_refresh(managed, key): + with pytest.raises(transport.DecisionClientError, match="^invalid_request$"): + transport.create_cloud_decision_client().evaluate( + SUPPORT_STATE, [_support_question()], model=transport.MODEL, + purpose="verify_support", allow_remote=True, request_key=key, + ) + assert managed == [] + + +@pytest.mark.parametrize("key", ["a" * 16, "Z0_-" * 16]) +def test_request_key_length_boundaries_are_sent_verbatim(managed, key): + transport.create_cloud_decision_client().evaluate( + SUPPORT_STATE, [_support_question()], model=transport.MODEL, + purpose="verify_support", allow_remote=True, request_key=key, + ) + assert json.loads(managed[-1][1].data)["request_key"] == key + + +def test_omitted_request_key_creates_distinct_opaque_keys_for_distinct_calls(managed): + client = transport.create_cloud_decision_client() + for _ in range(2): + client.evaluate(SUPPORT_STATE, [_support_question()], model=transport.MODEL, + purpose="verify_support", allow_remote=True) + keys = [json.loads(call[1].data)["request_key"] for call in managed if call[0] == "request"] + assert len(keys) == len(set(keys)) == 2 + assert all(len(key) == 32 and set(key) <= set("0123456789abcdef") for key in keys) + + +def test_lost_reply_has_no_automatic_retry_and_explicit_caller_retries_keep_key(managed, monkeypatch): + posted = [] + + def unavailable(url, token, payload, timeout_s, *, deadline): + posted.append((payload["request_key"], timeout_s)) + raise transport.DecisionClientError("remote_timeout") + + monkeypatch.setattr(transport, "_post_json", unavailable) + client = transport.create_cloud_decision_client(timeout_s=10) + for attempt in range(2): + with pytest.raises(transport.DecisionClientError, match="^remote_timeout$"): + client.evaluate(SUPPORT_STATE, [_support_question()], model=transport.MODEL, + purpose="verify_support", allow_remote=True, + request_key="lost_reply_retry_0001", timeout_s=2) + assert len(posted) == attempt + 1 + assert posted == [("lost_reply_retry_0001", 2.0)] * 2 + assert sum(call[0] == "refresh" for call in managed) == 2 + + +def test_request_key_bytes_are_included_in_preflight_size_bound(managed, monkeypatch): + payload = transport._request_payload(SUPPORT_STATE, [_support_question()], transport.MODEL, + allow_remote=True, purpose="verify_support", + data_classification="internal") + monkeypatch.setattr(transport, "MAX_REQUEST_BYTES", len(json.dumps(payload, ensure_ascii=False).encode()) + 1) + with pytest.raises(transport.DecisionClientError, match="^invalid_request$"): + transport.create_cloud_decision_client().evaluate( + SUPPORT_STATE, [_support_question()], model=transport.MODEL, + purpose="verify_support", allow_remote=True, + ) + assert managed == [] diff --git a/tests/test_mcp_jev_payloads.py b/tests/test_mcp_jev_payloads.py index 1506d3cd..f366d73e 100644 --- a/tests/test_mcp_jev_payloads.py +++ b/tests/test_mcp_jev_payloads.py @@ -62,6 +62,7 @@ def post(url, token, payload, timeout_s, **kwargs): monkeypatch.setattr(cloud_session, "access_for_workspace", access) monkeypatch.setattr(jev_transport, "_post_json", post) service = MemoryService.create(":memory:") + calls["service"] = service monkeypatch.setattr(mcp_server, "_service", service) try: yield calls @@ -107,25 +108,23 @@ def test_custom_question_only_uses_real_client_validation( if options is not None: arguments["options"] = options result = dispatch(arguments) + if wire_client["backend"] == "managed": + assert result["is_fallback"] is True + assert result["fallback_reason"] == "managed_operation_unsupported" + assert result["advisory_only"] is True + assert wire_client["http"] == wire_client["refresh"] == [] + return assert result["is_fallback"] is False assert result["decision_status"] == "decision" assert len(wire_client["http"]) == 1 payload = wire_client["http"][0] state = state_args.get("state", "") assert payload["state"] == (state if state.strip() else question) - if wire_client["backend"] == "managed": - assert len(wire_client["refresh"]) == 1 - expected = {"id": "custom", "prompt": question, "type": "choice" if options else "noul"} - if options: - expected["options"] = options - assert payload["questions"] == [expected] - assert payload["purpose"] == "custom" - else: - assert wire_client["refresh"] == [] - expected = {"type": "choice" if options else "noul", "instructions": question} - if options: - expected["criteria"] = {item: item for item in options} - assert payload["questions"] == {"custom": expected} + assert wire_client["refresh"] == [] + expected = {"type": "choice" if options else "noul", "instructions": question} + if options: + expected["criteria"] = {item: item for item in options} + assert payload["questions"] == {"custom": expected} if options: assert result["selected"] == "yes" else: @@ -176,12 +175,16 @@ def test_blank_optional_question_uses_default_with_meaningful_state( if options: arguments["options"] = options result = dispatch(arguments) + if wire_client["backend"] == "managed": + assert result["is_fallback"] is True + assert result["fallback_reason"] == "managed_operation_unsupported" + assert wire_client["http"] == wire_client["refresh"] == [] + return assert result["is_fallback"] is False assert len(wire_client["http"]) == 1 payload = wire_client["http"][0] assert payload["state"] == arguments["state"] - prompt = (payload["questions"][0]["prompt"] if wire_client["backend"] == "managed" - else payload["questions"]["custom"]["instructions"]) + prompt = payload["questions"]["custom"]["instructions"] assert prompt == "Evaluate state" @@ -213,6 +216,11 @@ def test_custom_question_schema_matches_real_client_limit( arguments = {"kind": "custom", "question": "q" * length, "allow_remote": True} if length == 1024: result = dispatch(arguments) + if wire_client["backend"] == "managed": + assert result["is_fallback"] is True + assert result["fallback_reason"] == "managed_operation_unsupported" + assert wire_client["http"] == wire_client["refresh"] == [] + return assert result["is_fallback"] is False assert len(wire_client["http"]) == 1 assert wire_client["http"][0]["state"] == arguments["question"] @@ -304,3 +312,130 @@ def test_sensitive_raw_mcp_fields_never_refresh_or_send( assert wire_client["http"] == wire_client["refresh"] == [] assert "DB_PASSWORD" not in json.dumps(result) + caplog.text assert "synthetic" not in json.dumps(result) + caplog.text + + +def _memory_snapshot(service): + """Capture contents, validity, reinforcement and graph state, excluding receipts.""" + return { + table: tuple(tuple(row) for row in service.store.conn.execute( + f"SELECT * FROM {table} ORDER BY rowid", + )) + for table in ("memories", "edges", "mem_links") + } + + +@pytest.mark.parametrize("kind,question_count", [ + ("guard_command", 2), + ("classify_contradiction", 1), + ("verify_support", 1), + ("verify_completion", 1), +]) +def test_scoped_memory_advisories_keep_offline_baseline_and_minimal_wire_calls( + wire_client, dispatch, kind, question_count, +): + """Check useful integration behavior; synthetic replies do not measure model quality.""" + from engraphis.core.interfaces import Scope + + service = wire_client["service"] + workspace_id = service.store.get_or_create_workspace("jev-fixture") + memory_id = service.engine.remember( + "Application database is SQLite.", workspace_id=workspace_id, + scope=Scope.WORKSPACE, title="Application database", + ) + service.engine.remember( + "UNSELECTED-MEMORY-FIXTURE: release owner is Morgan.", + workspace_id=workspace_id, scope=Scope.WORKSPACE, title="Release owner", + ) + evidence = service.store.get_memory(memory_id).content + arguments = { + "guard_command": {"state": "git status"}, + "classify_contradiction": { + "state": "Application database is now Postgres.", "existing_content": evidence, + }, + "verify_support": {"state": evidence, "query": "Which application database?"}, + "verify_completion": { + "state": "The fixture completed with zero failures.", + "goal": "Run the fixture", "recent_actions": "Fixture passed", + }, + }[kind] + before = _memory_snapshot(service) + + baseline = dispatch({"kind": kind, **arguments}) + assert baseline["decision_status"] == "local_fallback" + assert baseline["fallback_reason"] == "remote_not_authorized" + assert baseline["confidence"] is None + assert baseline["confidence_source"] == "unmeasured_heuristic" + assert baseline["advisory_only"] is True + assert wire_client["refresh"] == wire_client["http"] == [] + assert _memory_snapshot(service) == before + + decision = dispatch({"kind": kind, **arguments, "allow_remote": True, + "data_classification": "public"}) + assert decision["decision_status"] == "decision" + assert decision["advisory_only"] is True + assert len(wire_client["http"]) == 1 + assert len(wire_client["refresh"]) == (wire_client["backend"] == "managed") + payload = wire_client["http"][0] + assert len(payload["questions"]) == question_count + assert "UNSELECTED-MEMORY-FIXTURE" not in payload["state"] + assert _memory_snapshot(service) == before + if kind == "guard_command": + assert decision["allow_auto"] is False + assert decision["escalate_to_user"] is True + + +def test_command_advice_never_executes_even_with_synthetic_positive_answer( + wire_client, dispatch, tmp_path, monkeypatch, +): + import subprocess + + marker = tmp_path / "command-must-not-execute.txt" + command = ('python -c "from pathlib import Path; ' + f"Path({str(marker.as_posix())!r}).write_text('executed')\"") + + def forbidden(*args, **kwargs): + pytest.fail("an advisory decision attempted shell execution") + + monkeypatch.setattr(subprocess, "run", forbidden) + result = dispatch({"kind": "guard_command", "state": command, "allow_remote": True}) + assert result["is_fallback"] is False + assert result["safety_probability"] == 0.9 + assert result["advisory_only"] is True + assert result["allow_auto"] is False and result["escalate_to_user"] is True + assert len(wire_client["http"]) == 1 + assert not marker.exists() + + +def test_managed_query_route_choice_preserves_deterministic_plan_without_network(wire_client): + from engraphis.backends.jev_decision import JevDecisionBackend + from engraphis.backends.jev_query_planner import JevAssistedQueryPlanner + from engraphis.core.interfaces import PlannedQuery, RetrievalPlan + + class FixturePlanner: + def plan(self, query, **kwargs): + return RetrievalPlan(( + PlannedQuery(query, 1, "balanced"), + PlannedQuery("CACHE.get()", 2, "lexical"), + PlannedQuery("cache restart path", 3, "graph"), + )) + + client = (jev_transport.EngraphisCloudDecisionClient() + if wire_client["backend"] == "managed" + else jev_transport.TypeSafeDecisionClient()) + planner = JevAssistedQueryPlanner( + JevDecisionBackend(client=client, model=jev_transport.MODEL), FixturePlanner(), + ) + original = " Why does CACHE.get() fail after restart? " + baseline = planner.plan_with_jev(original, allow_remote=False) + assert baseline.reason_codes[-1] == "jev_remote_consent_required" + assert wire_client["refresh"] == wire_client["http"] == [] + result = planner.plan_with_jev(original, allow_remote=True, data_classification="public") + assert result.queries[0].text == original + if wire_client["backend"] == "managed": + assert result.reason_codes[-1] == "jev_managed_operation_unsupported" + assert result.queries == baseline.queries + assert wire_client["refresh"] == wire_client["http"] == [] + else: + assert result.reason_codes[-1] == "jev_route_selected" + assert len(wire_client["http"]) == 1 + assert wire_client["refresh"] == [] diff --git a/tests/test_mcp_server.py b/tests/test_mcp_server.py index 81864d9e..376324b0 100644 --- a/tests/test_mcp_server.py +++ b/tests/test_mcp_server.py @@ -1781,7 +1781,7 @@ def test_mcp_decide_tool_registration_and_offline_guardrails(monkeypatch): # 1. Registration assert "engraphis_decide" in classic_mcp._tool_manager._tools - assert minimum_role("engraphis_decide") == "member" + assert minimum_role("engraphis_decide") == "viewer" annotations = classic_mcp._tool_manager._tools["engraphis_decide"].annotations assert annotations.readOnlyHint is False assert annotations.idempotentHint is False diff --git a/tests/test_smart_mcp_gateway.py b/tests/test_smart_mcp_gateway.py index b7c31f8b..8123c05c 100644 --- a/tests/test_smart_mcp_gateway.py +++ b/tests/test_smart_mcp_gateway.py @@ -401,9 +401,20 @@ def test_gateway_context_usage_counts_authoritative_receipt_once(monkeypatch): ("engraphis_discover_actions", "viewer"), ("engraphis_execute_read", "viewer"), ("engraphis_execute_action", "admin"), - ("engraphis_decide", "member"), + ("engraphis_decide", "viewer"), + ("engraphis_stats", "viewer"), ("engraphis_remember", "member"), + ("engraphis_remember_many", "member"), + ("engraphis_correct", "member"), + ("engraphis_retire", "member"), + ("engraphis_secure_erase", "member"), + ("engraphis_session", "member"), + ("engraphis_update_memory", "member"), + ("engraphis_future_unknown_tool", "member"), ("engraphis_consolidate", "admin"), + ("engraphis_index_repo", "admin"), + ("engraphis_ingest_postgres_schema", "admin"), + ("engraphis_link_symbol", "admin"), ]) def test_smart_gateway_roles_fail_closed_at_the_outer_auth_boundary( monkeypatch, tool_name, required_role, @@ -413,6 +424,41 @@ def test_smart_gateway_roles_fail_closed_at_the_outer_auth_boundary( assert server.minimum_role(tool_name) == required_role +@pytest.mark.parametrize("offline_mode", [False, True]) +def test_viewer_decision_access_does_not_bypass_smart_read_boundary(monkeypatch, offline_mode): + server = _memory_server(monkeypatch) + action = _payload(server.engraphis_discover_actions( + task="guard command safety", intent="write", + ))["actions"][0] + assert action["canonical_action"] == "decide" + assert action["side_effect"] == "write" + assert server.minimum_role("engraphis_decide") == "viewer" + assert server.minimum_role("engraphis_execute_action") == "admin" + assert server.ACTION_SPECS["decide"].annotations["readOnlyHint"] is False + assert server.ACTION_SPECS["decide"].annotations["idempotentHint"] is False + before = server._service.store.conn.execute( + "SELECT COUNT(*) FROM operation_receipts" + ).fetchone()[0] + + def forbidden(*args, **kwargs): + pytest.fail("the read executor dispatched a quota-consuming decision") + + monkeypatch.setattr(server, "_run_action", forbidden) + response = server.engraphis_execute_read( + capability_id=action["capability_id"], schema_digest=action["schema_digest"], + arguments={"kind": "guard_command", "state": "git status", "allow_remote": True, + "offline_mode": offline_mode}, + ) + code, message, retryable = _error_envelope(response) + assert code == "E_VALIDATION" + assert "action_requires_execute_action" in message + assert retryable is False + after = server._service.store.conn.execute( + "SELECT COUNT(*) FROM operation_receipts" + ).fetchone()[0] + assert after == before + + @pytest.mark.parametrize(("task", "executor_name"), [ ("Show memory store statistics.", "engraphis_execute_read"), ("Record a deployment event.", "engraphis_execute_action"), From e584e098a7e851eddbfc628a35897ed5583a22be Mon Sep 17 00:00:00 2001 From: Jaixii Date: Sun, 4 Oct 2026 03:57:56 -0400 Subject: [PATCH 02/22] Use supported managed context in credential deadline regressions --- tests/test_cloud_session_deadline.py | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/tests/test_cloud_session_deadline.py b/tests/test_cloud_session_deadline.py index bc05be49..d1ce1f2f 100644 --- a/tests/test_cloud_session_deadline.py +++ b/tests/test_cloud_session_deadline.py @@ -51,8 +51,10 @@ def saved_session(monkeypatch, tmp_path): def _evaluate(timeout=0.1): return jev_transport.create_cloud_decision_client(timeout_s=timeout).evaluate( - "Synthetic local evidence.", [DecisionQuestion("q", "Is this supported?", "noul")], - model=jev_transport.MODEL, allow_remote=True, data_classification="public", + "QUERY: Which port?\nEVIDENCE: The selected memory says port 443.", + [DecisionQuestion("has_support", "Does this evidence directly support answering the query?", "noul")], + model=jev_transport.MODEL, allow_remote=True, purpose="verify_support", + data_classification="public", ) @@ -66,7 +68,7 @@ def _rotation(): def _decision(): - return {"model": jev_transport.MODEL, "is_fallback": False, "decisions": {"q": { + return {"model": jev_transport.MODEL, "is_fallback": False, "decisions": {"has_support": { "type": "noul", "probability": 0.9, "confidence": 0.8, "confidence_source": "derived_decisiveness", }}} @@ -137,7 +139,7 @@ def open_request(request, timeout): monkeypatch.setattr(hosted_client, "validate_cloud_base_url", lambda value: value) monkeypatch.setattr(hosted_client, "build_pinned_https_opener", lambda *handlers: SimpleNamespace(open=open_request)) - assert _evaluate(5).get_noul("q").probability == 0.9 + assert _evaluate(5).get_noul("has_support").probability == 0.9 assert calls == [("refresh", 105.0), ("decision", 2.0)] assert cloud_session._load()["refresh_credential"] == "synthetic-rotated" From 66920e48eb12444876d6e08e937c7a12e02ad4c5 Mon Sep 17 00:00:00 2001 From: Jaixii Date: Sun, 4 Oct 2026 04:01:52 -0400 Subject: [PATCH 03/22] Refresh source-bound offline evidence after current Jev workflow integration --- BENCHMARKS.md | 10 +- README.md | 2 +- .../offline-fixtures-v137.json | 696 ++++++++++++++++++ .../offline-fixtures-v137.json.sha256 | 1 + docs/images/context-efficiency.png | Bin 95758 -> 96080 bytes docs/images/context-efficiency.svg | 4 +- .../images/evidence-backed-agent-examples.png | Bin 157202 -> 157846 bytes .../images/evidence-backed-agent-examples.svg | 4 +- tests/test_benchmark_evidence.py | 4 +- tests/test_cloud_session_deadline.py | 2 +- 10 files changed, 710 insertions(+), 13 deletions(-) create mode 100644 docs/benchmark-evidence/offline-fixtures-v137.json create mode 100644 docs/benchmark-evidence/offline-fixtures-v137.json.sha256 diff --git a/BENCHMARKS.md b/BENCHMARKS.md index cb1d60bc..80f85b8f 100644 --- a/BENCHMARKS.md +++ b/BENCHMARKS.md @@ -94,14 +94,14 @@ interpretation and do not count as additional benchmark-quality gains. ### Public numeric evidence registry Every exact public aggregate retained below comes from the checked-in, public-safe -[`offline-fixtures-v136.json`](docs/benchmark-evidence/offline-fixtures-v136.json) artifact. Its +[`offline-fixtures-v137.json`](docs/benchmark-evidence/offline-fixtures-v137.json) artifact. Its SHA-256 is -`1b894b37574eed06e56feec10840744c36b98813a36ef4c7cc38b9957856e1ef`, also recorded in the +`ead6889f9050aa88a916055ea805a546aa179b6cc9a5edf05291e3419089ea4e`, also recorded in the adjacent `.sha256` file. The artifact contains no raw questions, answers, prompts, customer data, or per-record content fingerprints. The fixture-suite digest is -`dd0b15d17063bbe96da38a3468651d2e9642f5338b9024917b61fc9d94497034`. The artifact defines +`c9ca07b242550702677cfca35a1c66efc628caa7e289649bc5277eaf3986ca54`. The artifact defines the digest algorithm and records the SHA-256 of every suite and dataset file. Each evidence ID also binds its exact command through `sha256(UTF-8 exact command)`: @@ -126,13 +126,13 @@ grounded checks in separate panels; provider billing and MCP transport are not m the SVG and matching PNG with: ```bash -python scripts/render_benchmark_report.py --report docs/benchmark-evidence/offline-fixtures-v136.json --output docs/images/context-efficiency.svg --png-output docs/images/context-efficiency.png +python scripts/render_benchmark_report.py --report docs/benchmark-evidence/offline-fixtures-v137.json --output docs/images/context-efficiency.svg --png-output docs/images/context-efficiency.png ``` The companion examples are also generated from that artifact with: ```bash -python -m scripts.render_benchmark_examples --report docs/benchmark-evidence/offline-fixtures-v136.json --output docs/images/evidence-backed-agent-examples.svg --png-output docs/images/evidence-backed-agent-examples.png +python -m scripts.render_benchmark_examples --report docs/benchmark-evidence/offline-fixtures-v137.json --output docs/images/evidence-backed-agent-examples.svg --png-output docs/images/evidence-backed-agent-examples.png ``` The historical-to-executable mapping is in [`docs/BENCHMARK_CHANGE_COVERAGE.md`](docs/BENCHMARK_CHANGE_COVERAGE.md). diff --git a/README.md b/README.md index a84d896b..39eb9542 100644 --- a/README.md +++ b/README.md @@ -49,7 +49,7 @@ Connect a coding agent over MCP with the [agent setup guide](https://github.com/ The current registered artifact contains three deterministic offline fixture runs. It separates context size, retrieval quality, and grounded decision checks; these small fixtures do not establish general task performance.

- Three registered offline fixtures: structure-aware chunking reduced mean retrieved context from 740.3 to 214.3 tokens per question (71.1%, 18 questions); the serialized JSON-shape proxy fell from 24,590 to 11,138 tokens across 26 payload samples and 260 recalls. Candidate and packed retrieval quality (Recall@5, Hit@5, and answer-token recall) are shown separately (each 1.000). Grounded checks show 5/5 answerable queries grounded and 6/6 abstention queries rejected, including a 1/1 quarantined-evidence probe; 11/11 decisions were correct. MCP transport and provider billing were not measured. Artifact SHA-256 prefix 1b894b37574e; see the benchmark guide for the full checksum. + Three registered offline fixtures: structure-aware chunking reduced mean retrieved context from 740.3 to 214.3 tokens per question (71.1%, 18 questions); the serialized JSON-shape proxy fell from 24,590 to 11,138 tokens across 26 payload samples and 260 recalls. Candidate and packed retrieval quality (Recall@5, Hit@5, and answer-token recall) are shown separately (each 1.000). Grounded checks show 5/5 answerable queries grounded and 6/6 abstention queries rejected, including a 1/1 quarantined-evidence probe; 11/11 decisions were correct. MCP transport and provider billing were not measured. Artifact SHA-256 prefix ead6889f9050; see the benchmark guide for the full checksum.
Three offline fixtures separate context reduction, candidate and packed retrieval quality, and grounded behavior. The chart shows the artifact checksum prefix; see the benchmark guide for the full checksum and reproduction steps.

diff --git a/docs/benchmark-evidence/offline-fixtures-v137.json b/docs/benchmark-evidence/offline-fixtures-v137.json new file mode 100644 index 00000000..b01b6480 --- /dev/null +++ b/docs/benchmark-evidence/offline-fixtures-v137.json @@ -0,0 +1,696 @@ +{ + "environment": { + "embedding": "deterministic", + "numpy": "2.5.3", + "platform": "win32", + "python": "3.12.14", + "vector_backend": "numpy" + }, + "generated_on": "2026-10-04", + "privacy": { + "contains_answers": false, + "contains_customer_data": false, + "contains_per_record_fingerprints": false, + "contains_prompts": false, + "contains_raw_questions": false + }, + "runs": [ + { + "boundary": "Deterministic offline retrieval fixture; normalized-character token estimator; not external QA or provider billing.", + "command": "python -m eval.chunking_eval --dataset eval/datasets/longdoc.jsonl --k 5", + "config_digest": "c1c8196aa7e1568ef3844a9fb2d76b87f342c39108e32d6ad144b885a76143b8", + "config_digest_method": "sha256(UTF-8 exact command)", + "id": "offline-chunking", + "result": { + "chunked": { + "max_stored_tokens": 59, + "mean_context_tokens": 214.3, + "mean_evidence_tokens": 42.4, + "memories": 24, + "recall_at_k": 1.0 + }, + "context_reduction_pct": 71.1, + "documents": 6, + "k": 5, + "questions": 18, + "token_counter": "engraphis.chars4.v1", + "whole": { + "max_stored_tokens": 213, + "mean_context_tokens": 740.3, + "mean_evidence_tokens": 162.2, + "memories": 6, + "recall_at_k": 1.0 + } + } + }, + { + "boundary": "Deterministic offline CodeMem fixture; serialized JSON-shape payload proxies, not MCP transport responses, provider billing, or latency claims.", + "command": "python -m eval.performance --dataset eval/datasets/codemem.jsonl --k 5 --iterations 10 --json", + "config_digest": "bbe4aca81e58d4830e50a8fc7729a1d15b71d97a6299bccd79432b7f119677d7", + "config_digest_method": "sha256(UTF-8 exact command)", + "id": "offline-performance", + "result": { + "answer_token_recall": 1.0, + "compact_serialized_payload_tokens": 11138, + "dataset_cases": 14, + "full_serialized_payload_tokens": 24590, + "hit_at_k": 1.0, + "k": 5, + "max_context_tokens": 108, + "mean_context_tokens": 85.38, + "memories": 44, + "packed_quality": { + "answer_token_recall": 1.0, + "hit_at_k": 1.0, + "recall_at_k": 1.0, + "sample_count": 26 + }, + "payload_boundary": { + "kind": "serialized_json_shape_proxy", + "mcp_envelope_serialized": false, + "token_counter": "engraphis.regex.v1", + "transport_measured": false + }, + "quality_scope": { + "packed": "packed_quality fields score only chunks admitted to reader context", + "retrieved": "legacy quality fields score all candidate chunks returned before context packing" + }, + "questions": 26, + "recall_at_k": 1.0, + "saved_serialized_payload_tokens": 13452, + "serialized_payload_savings_ratio": 0.5471, + "timed_recalls": 260, + "token_budget": 1500, + "token_counter": "engraphis.regex.v1" + } + }, + { + "boundary": "Deterministic offline support/abstention fixture; not a frontier-model answer-quality score.", + "command": "python -m eval.grounded", + "config_digest": "590442e51e3642c10489165759919dc86ffac62c182937330c153e7f8d5fc26f", + "config_digest_method": "sha256(UTF-8 exact command)", + "id": "offline-grounded", + "result": { + "abstained": 6, + "answerable": 5, + "decision_accuracy": 1.0, + "grounded": 5, + "off_topic": 6, + "quarantine_hits": 1, + "quarantined": 1 + } + } + ], + "schema": "engraphis-public-offline-fixtures/v1", + "suite": { + "digest": "c9ca07b242550702677cfca35a1c66efc628caa7e289649bc5277eaf3986ca54", + "digest_method": "sha256(canonical compact JSON mapping each sorted path to its file SHA-256)", + "files": { + "engraphis/__init__.py": "f24241e8da2ede9ccba54ce4bf7531a800b55c29b2469e5de93bd423cffbf0be", + "engraphis/ai_context.py": "4dfd5d39eb95d05c591d1981e9e855eced73232f53efd9fe9966e0e08957d030", + "engraphis/app.py": "44d68ad8c0ff46baed01978c9b04e40b8be5031609d5a69e205b7f05da3777fb", + "engraphis/backends/__init__.py": "a9f22b9278362904166614081f1df78469d453601b298ce4e8afdba8a3722b25", + "engraphis/backends/codegraph.py": "83e723a91068d23694092fbe00157fcb2597061d453eb36f955bf55bc5b4d35e", + "engraphis/backends/embedder_api.py": "56a6bceea4f757325dcea987b0339e41d3875634b1ae5103f0537a358cf5878b", + "engraphis/backends/embedder_deterministic.py": "ec8b23de7e7e8273416125f5876ca96f55e4ae7881841bae55783ab0ba9130ad", + "engraphis/backends/embedder_st.py": "e1c20fd980e07060387e3f9fa37fe02a916959fc8abce4e6067a62699c726de1", + "engraphis/backends/encrypted_db.py": "25f6c1480d296a88f317213700a8b3c81e2732399bfac464b0d893465b25e846", + "engraphis/backends/extractor.py": "f2e3455ab7f14caee1d5b5c4ef071e498e90118f1b0ddcb8510c969583b0fc57", + "engraphis/backends/graph_extractor.py": "88561efa0d3fabc447a0a005b10e36261379d46218cf62d905e6928cd2fda676", + "engraphis/backends/jev_decision.py": "fdab1ae9a7ac3bc19632f23a58e53ecce97b6c1df2d4edd1fef06d4e59800888", + "engraphis/backends/jev_query_planner.py": "8c9b2859ef59868559a6b14a854a3280ed0a4f7aedd2fd4e5699d661e45e948e", + "engraphis/backends/jev_transport.py": "b55b37e08813b0419602a55445846cf838c4a0987b147001274dd46e161466be", + "engraphis/backends/model_source.py": "8c3c7681f95214a2bbabd8de222e5ee11f42fe13402d27365654ae75fb363d4e", + "engraphis/backends/postgres_schema.py": "8468578c3add701d30d5eaa36d768ded2375d116e55f1836e09f6107a07a267e", + "engraphis/backends/query_planner.py": "bbdd77afc9b5523421b85b2ae63c8da7f5a7b777265450e0d21708a83e7bb23c", + "engraphis/backends/reranker.py": "747761d6cbfa421388974bcfd98d844f92391d80f4bf6a4feca00b0c7a6908ca", + "engraphis/backends/resources.py": "47cc867c3aecc8bd95fa284bc5bb04715f3339c19a0a11512973ef6171c95944", + "engraphis/backends/retention.py": "381d9371e3951d762f8b55eb54711de5697642acb39de99a714f246c059ecbd0", + "engraphis/backends/sync_folder.py": "e4f70a92a17f6a365910670df041e6e3ca421d44ada2827917cd66b4dc067bfa", + "engraphis/backends/sync_relay.py": "b8b9ad265453aba17ba7c27a355e12a793469b3e44cb217943c6fad9382a3006", + "engraphis/backends/vector_numpy.py": "c598831bea547824cfe08844816fa79857d3617cb0631238f95dad955a424f72", + "engraphis/backends/vector_sqlitevec.py": "6148e14ceaacc19239b64c642a3fba0e98797c78cec356210157afadf475a08b", + "engraphis/build_info.py": "624c22471e56d4c4047160808c4245488292af611564d1a63ca437605bbb414f", + "engraphis/classic_assets/__init__.py": "a7c1d52b285e3faa670ce231814b5758754aa0fbd05e1428e74c20c3ec51a4f1", + "engraphis/cloud_authz.py": "e80500579cb3a1d5fbf30814dc94e3e3967e50b311e8ed2fa56afcc13f7eb565", + "engraphis/cloud_features.py": "90e876f8993d99f01760cd01ae9fb63140e3f65036706e8fb0c8224560416330", + "engraphis/cloud_session.py": "cc6ab27c6af69b5c676e721ab06c520730e205f5bbffe27818650edfb71d4b63", + "engraphis/commercial.py": "184f312066a9e682e51a0abeff042f1c0e8eed2d47470157b23930b5a17633aa", + "engraphis/config.py": "9af2072734f0e0fb409ce3cd422d5a4cb9a4854c9a92d2e03d7e680e119a8fb9", + "engraphis/core/__init__.py": "dd5143729c3939237f04636f437032b1f2d3a5f7d82c91bbc2a5a283c3f0ebaa", + "engraphis/core/adaptive_context.py": "cc5ce48109bb0d5230a5b2b8424b829c853feec5b5d5b82596413f2279b0c9f9", + "engraphis/core/browsing.py": "cfae752d52ef51b17c4ffbe44dde62d5b0e1ed0ca34ec0e1788f3991d94bf05f", + "engraphis/core/codegraph_export.py": "4641074258d7b23498f92dd45053a0fbb111863eaad2001c08e5e3c2dc2fd54f", + "engraphis/core/conflicts.py": "28530be25a4af0bffd8f609b965b33ca7f93199a70887789b2148fda8a61a486", + "engraphis/core/consolidate.py": "f66eedfe08319a64b761261ccf1c99b163eba564d6d4060534bf8c575aecf232", + "engraphis/core/context.py": "8cc15746e4de88bce7d34db8c11dfaa545f17305788e2167a286179171717fb9", + "engraphis/core/diagnostics.py": "5ba449bdf5087d8ccb5808379596e695c8f405da497372e99f2bac509f96e8e4", + "engraphis/core/documents.py": "84385db39ba44e06b58b4b26dbf954228ff4abed7230f28a7280166fa6457861", + "engraphis/core/engine.py": "600e21318b95f78830210457657bbcd0d916d5d118ab736a375206fb1fb44ebf", + "engraphis/core/evidence.py": "97912b52a3d22de909218f83c09c92572ed695d7b3c2117e5330271c6d84a7bd", + "engraphis/core/fsutil.py": "6db770fa8bd3e1a57dfa70eb8e8bc46d48c2dc53ead1b0b43eeecf085ef58cfc", + "engraphis/core/graph_layers.py": "64d74ab01c77119f6343ba6f1d6a84f9653f1a5d34d47ce7966f3ac31b29d2ea", + "engraphis/core/graph_policy.py": "ec5b373d01adb2de87df31d9f543130018e9a73faaaed239a184f14a32646615", + "engraphis/core/graph_scene.py": "5bfb4dadd90aff6be89dad0384aabec533c04c9a5225d3e9d43a468152569d75", + "engraphis/core/graphrank.py": "1279a58396104d3f906bfd5ec75b32efedfefe52201467bf19d80be3517017a5", + "engraphis/core/grounded.py": "1fdbd9522c541b7e9aef05b4af245fc741503f761c70dd1193b9d4e68b610c1b", + "engraphis/core/ids.py": "e47eeadfaf560bc7638e1879fc491fa82d981cd42b9e4b05ef87033b2d7dfa60", + "engraphis/core/interfaces.py": "05d65c1919f3c23592f48e5f2820bbe857923fb65f5146727636eba51941271c", + "engraphis/core/mutations.py": "dbb46a97686994e1652b2698e50c3309ff428d7258d6e8e53cbad7bb55b459aa", + "engraphis/core/obsidian.py": "991267c153cb7c4c40f7fe8f50aa71688892e250aaaf383b22c9e5910dc263b7", + "engraphis/core/poisoning.py": "5bc67169ee8032f3777f2d3969dcf71821437bd473ce4f917c4b41a50e845fb6", + "engraphis/core/query_planner.py": "249062d67392ab7c203cc71e9040e99bee91bf570604e90949149a93cb652120", + "engraphis/core/read_snapshots.py": "be08e63a88bd38ed91d61b28657a65201c38994856db2798b209a73151dd202a", + "engraphis/core/recall.py": "f4652b7eadb310ce19a8f9418674061848bfdf847b0b73054b8f5d08f87eee1a", + "engraphis/core/relocation.py": "86bbd292b374539b480dddc1a9ac19292fbc17ff9fa8e1fe6ff69c91ee7c8bb9", + "engraphis/core/resolve.py": "f01a6f55e44320ab04b97e516342f20155668863b2fc4765305e066d87586524", + "engraphis/core/retention_policy.py": "864c03bdb6e743cd0002c706de471e920f1fa1f1a9918ab343ef4c2042b47429", + "engraphis/core/retrieval_policy.py": "d169eb442115bd06c1e1da776fa6c1849e0795e658edd732a8d6821326b46b03", + "engraphis/core/savings.py": "cfbcfc7e476f4e28028555cd519696e23099f6210cf0b733225832aeaa0bc7dc", + "engraphis/core/schema.py": "ac273d3f0383995be815bbd866f2a36ea1398f30833aed57b8d4459afc87096b", + "engraphis/core/scoring.py": "f5b6ac291edf0968b3de83cb1951a97d5bfd8a8d079199cb2ef95bba0884c89a", + "engraphis/core/secrets.py": "a4835ba06e2616156528df6365ca1aba6cba0c97cdf834a709d2797a371099d2", + "engraphis/core/store.py": "0ba21d11b9a61b105f743aac3850f12638cc51b425ce76fb8d0c1eb27d540647", + "engraphis/core/sync.py": "69f75b50fdb1ec9352f92460c89efeec8d72ca642bd10beb29474a8c62b4f58a", + "engraphis/core/textutil.py": "acd65031729fa5d91d09527b8eb52518b83cfce77a55e6b7ae2d94686e35c1a6", + "engraphis/core/user_model.py": "3147ec8ee7cfd855783f639874b63f331cdc822bd8bd9298cfb326dd18666026", + "engraphis/core/vector_repair.py": "a8c1812de4e3ed288eda3e54a505e136d6ebd3ec296788bcbb8804b11e13cfc8", + "engraphis/core/vector_search.py": "75800052e573e9af01c6fd098eaf8647c3c45cd7eb2601dae05d42359e19b234", + "engraphis/dashboard_app.py": "9305c9c72c43027b2b79c0ce559d398fc232929968d6ea03c37065e3227c8d51", + "engraphis/dashboard_assets/__init__.py": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", + "engraphis/device_connect.py": "c8cd0a22e9fd2d92a65bd74047cc3a8b7159bfc9a297cb639c1c7f1699f8802e", + "engraphis/document_import.py": "94fa0ca340ad0ebd060b143f46657798a81c440a11b82aa46fefa83fb65295dc", + "engraphis/engines/__init__.py": "111232af583889195c5f5a60298e32484348608e81d2bdd822fd3ecd2a33c1e4", + "engraphis/engines/embedder.py": "998b65dd566966bb6581fd9f09cdf46c58a3923ef5df3073ccdaa541153a3581", + "engraphis/engines/ingest.py": "1a5d4b52c13e533864329f9fff11c0f6ebc299ff7093a9d9275e5c9626a39ce2", + "engraphis/engines/intelligence.py": "b561589b98deb98271f104dfec6276aeba13c4815e627259be5f8769f2e7ad82", + "engraphis/engines/recall.py": "f979580d065599c07acbc3add59f71e52d9a186a2104b0e257daa7959ea88d26", + "engraphis/engines/reweight.py": "91ec5815f5d356a7068c36133d24405334450f361f902f99275bed3ccebffd49", + "engraphis/engines/thoughts.py": "4adb9c8a9bcfe736cb42fff9b5ce24631da6ec473d178e83f1c176fde3b8b814", + "engraphis/factory.py": "06735a5acfc784fd1514ae42b24f4a58d041f6eeb040f8610184b458682fe030", + "engraphis/graphdata.py": "f5dc93395f52203c66deaeec8bc58d5c1f990a35a535151e0b9a84cecf7d8430", + "engraphis/hosted_client.py": "5b56b0d2384cf24ecb409090af15f09e437ec8ab5a98bed931384daff46b4f48", + "engraphis/http_deadline.py": "8f41de086f36ab39d4e6461c6284c776e3b123afee845824e22f071ca128c918", + "engraphis/http_security.py": "596981e96741fd47064d03db409605bbb435f062c0c6ef69b040aacdd8763a20", + "engraphis/inspector/__init__.py": "720cac28b8a6019d0a0c53809d5905b7d6eb9d4ecbac767505904c6ec3c39071", + "engraphis/inspector/app.py": "8aefe8a935397dde3c6466e691a7e6a7203ebe42e07aa00c332b733465e59e04", + "engraphis/licensing.py": "7e73c28b0e1c3536e2080a129af614f838d2cae3ec3a48a3d8ed7e5153c9e935", + "engraphis/llm/__init__.py": "f3096d2ddd652b99e6fc0b4c5a8786d9bbaba41b259df0b44ce93b57bde3a8b1", + "engraphis/llm/client.py": "0c84000033d3f85b0e0161f100c271c35b591e1480701455ec65c7d3f111ec42", + "engraphis/local_auth.py": "b0ad3a1926d417a2aaa6c44a7dbf6e51f575290ddfa27875b78a03db176623c3", + "engraphis/logging_setup.py": "f7d2edc756458a852e0401453e9c71b785aba08fa7859aacbdc23fb30dc7982a", + "engraphis/managed_processing.py": "6d33cdfd10800d9552fcfae3c2b071d2b39fe10fb69d1a8e5d9ed6026882197e", + "engraphis/mcp_classic_cli.py": "778122b121a1c654b8f85810fad05d9c8d7799cf13903d7dc274938b51008da0", + "engraphis/mcp_cli.py": "ed2d997438d727180842dc5fb3f6776f5a5d97974f690ee7229264be1ff61957", + "engraphis/mcp_http_cli.py": "5a04bcaae4531a6ea116a827bb65c5df3bd6d07ea336f65ff3ed45034972c396", + "engraphis/mcp_server.py": "c168fcd4850bbab65404e8af7c0af428475218fe4ec830712cc5def7df06c794", + "engraphis/models.py": "6e76e97db0aca3805c6f582ea78cc6e1c0ccd2665e16fd8ab81eb91b94141b51", + "engraphis/netutil.py": "2e0f8a9095f6f31dcb5b96d323f023b9214369d59e1c488125a7d443a55972ff", + "engraphis/observability.py": "a3a6945bf33a0d8e216da56ca7efec0031b82dc64a66cff36f0b2161f5db4981", + "engraphis/obsidian_import.py": "c7cf4b5e993afce3ffb45f01637804719cb9e7cd2461ffe629c055be5975721f", + "engraphis/private_state.py": "7485570efcaee8a1dc00b64ebfdc23178517235ae17609aa78db8bdda7e45fe3", + "engraphis/read_only_api.py": "aeea03282cfdc02074535b3ec6618b5baab0d26725be434911b777db6e7da158", + "engraphis/redirector.py": "5ba964b81f09008c9369180cb49d247519000763274aabc5e046215b12fa641d", + "engraphis/routes/__init__.py": "f0d59080212cfa0d9b50877bca28e5832d1ae917899500d52c68b1c8821231af", + "engraphis/routes/memory.py": "9ca066e1762eeeaafd9790ed57d730efcaceb57742dbe0c830ad8764500ff9e5", + "engraphis/routes/v2_api.py": "a027b71b9eea52b302abf95dbe4374305d4f6c368350999dc2fa88255c65a09d", + "engraphis/routes/vault.py": "1a7eee7c1a7c7aa11091042756aa2739a3eb963647020f40584a6a38da1988af", + "engraphis/service.py": "6df47bb264f4729a41cf84cd2778a483598c5da6dfd907792c6565a95acacb5f", + "engraphis/service_context.py": "3de9289f49a977cdc206285ac42a9953a104eb1d1b1bc8ea77febd75c7cd82ab", + "engraphis/static/__init__.py": "1fff4c4e2554e7f5fcf3eace269feba09827524917193a4dd65df95bae64ad1f", + "engraphis/stores/__init__.py": "48ee4326c8f28eecf46f558b7aea21adb779226a7038299017ab90196d5d84be", + "engraphis/stores/graph.py": "ebf603b54cf8450e7c9a7319bd05db2f39fcda8491f5969d6af3e8da61571bc2", + "engraphis/stores/ledger.py": "df5cbb30d977decc0a9c3a2365c9c115cfbb5fe951bae48446436661c80e5d30", + "engraphis/stores/vaults.py": "2c986129b9d1e7aab33e18a3b9a278eb5ad2236895587f69b5816a5d96796bc3", + "engraphis/stores/vectors.py": "45a1baca381fc647548cc89424eb36d5853f7562c275b191b35b99339a190b7d", + "engraphis/update_check.py": "ffbf5ef682fb15177915073ebc0b5eab4dee9092bbf0f58d36a8a54c6605ccf4", + "eval/__init__.py": "639f0c6d9d6aac8ff6dc605a34a0a301058905cc53bff4eaed5912247f0e7c56", + "eval/ablation.py": "16f159dee75d2f96cc42f230c2403fa19ea0bda7091c4823660da553463a194a", + "eval/adversarial_memory_security.py": "35dd8d981bcbad50e9815465be420b05b62a9dea28a78eb6cc1e09ec51c320c4", + "eval/agent_benchmarks.py": "8e88943e45a1b2083b366cbb0329326ef918b119a2366ed395c404c2b7ef98b0", + "eval/benchmark.py": "b71832affdf87d23bc1db7b522a669a7888571b3989b03a235c70f5da9952bd6", + "eval/benchmark_analysis.py": "1200791425d029695fee0b7da1f188a8962f337aa31eb4217d50e4503641d2b3", + "eval/benchmark_campaign.py": "033dbebfb47df5fcd3a6588b29ea4ab5fd44d4387ae45334379278ab30f3e57f", + "eval/campaign_adapters.py": "2bdb51de0dcdfceb1be2a649d84541780ac4d1fc0f9b7585004d87490073213e", + "eval/campaign_api.py": "323be4e1d9520e8047ab54c9cbb158013785402b06ee4e93e55227971c54bf95", + "eval/campaign_candidate.py": "88541840d16c3b7368586ed0ee3566054214b1f594826b746b28d0e407cb812a", + "eval/campaign_continuation.py": "7f77a20ac8f85cd96729bba0cf45872c98a3421af005bd735434b7b416dd7e1f", + "eval/campaign_ledger.py": "ba3079ba541cb1f70929a76f147d2bc5e8b264103a1cd7faad97685e0ecc7790", + "eval/campaign_oracle.py": "9ea5479786efd9caa2b59e612e891a93dd34b500a2b36aa20bcf749fb6d4d48a", + "eval/campaign_storage.py": "ebcd1f4aeceaa5ff9304494c64dd31fe31637ceb107d8cc39fa292e3b14055d4", + "eval/capacity_matrix.py": "8c25bd97754c1d7a8468a687042a3c87cc9c0299cf01c19b80dacfec2be4e552", + "eval/chunking_eval.py": "a16544353940c0a8c40cea3b9932d3399b35ea5994b809b78f5dbe4a952c467f", + "eval/code_agent_ab.py": "d98bba6b77700ff6bf86ff0d2cf518a67e5a2676ef51bbddabbb2ff26e1f3aaa", + "eval/code_arm.py": "d211166fce1b8a4173848e1617873b7e84aadeff43746effe7483c54bbdb6f1d", + "eval/codex_oauth.py": "f235fca482de4201d1850bbfb583765ac5f1a057b4ad7e2d504d036bfc03f391", + "eval/coding_acceptance.py": "4b39cbcb60d7fba503cca597399cf9d04ad9567a43cdb950015ccf552e6a2773", + "eval/coding_corpus.py": "7b5205e8544578fe99d9d9cf6cfc34e40e5d1d049238d502ff7bc6f3c14948f5", + "eval/consolidation_ranking.py": "917b578d4e0bcb929bf1a1a37611acf7716a520c076abf4ade0d8d12a1c455de", + "eval/context_economy.py": "709ac7cc866855f96d7717ab2bea12e8b0d3a140d15fb978a2a929ad085931f2", + "eval/context_efficiency_guardrails.py": "22afd1a6fe17219e74701dc587ec35f569a5bf22bea270944525f51746723f14", + "eval/datasets/codemem.jsonl": "341313023c22850a2e14f02742b571ad1deca824f886a1654a59541304c01f3c", + "eval/datasets/coding_memory_v1/oracles/atlas-green--code_relationships.py": "969aa71235e0ae3cb764b9ee12b789184cedb5fea6fcc0d86e5dcb2c0498fa56", + "eval/datasets/coding_memory_v1/oracles/atlas-green--condition_values.py": "648702646fb62990168e98fba0dc6a27be128876548fa7035a59d71aee9da02d", + "eval/datasets/coding_memory_v1/oracles/atlas-green--corrections.py": "c7132c7fe39b7afbdf11065037369c0f3c38a22e69d8742e8afddfd5785d2e61", + "eval/datasets/coding_memory_v1/oracles/atlas-green--long_documents.py": "65dffb25944cb612872655f12db16af73f7a095e7abcf6a7cc5d31dc91f083b1", + "eval/datasets/coding_memory_v1/oracles/atlas-green--multilingual.py": "ff49874a691bfc30c082695ee6d3c89b9f2e1e5eecf086026110770d8e1b0452", + "eval/datasets/coding_memory_v1/oracles/atlas-green--paraphrases.py": "dbdcf5d7abf136aa5816df8465bfde842284aef5ef002e955cee05ae69d574ca", + "eval/datasets/coding_memory_v1/oracles/atlas-green--poisoning.py": "58b8ca4a37f55c4d112640414dc82620e32e643ab0aa539260884a5ecc50740f", + "eval/datasets/coding_memory_v1/oracles/atlas-green--scope_boundaries.py": "c1e92b10b40004c95abd3dfbada1a38fe0ce56300ecddca8360b51a1e6671458", + "eval/datasets/coding_memory_v1/oracles/atlas-green--temporal_history.py": "a64e4e5a29ce1040cf53e94571491325347f443d72a93f05bcf87c2878665952", + "eval/datasets/coding_memory_v1/oracles/atlas-green--unsupported_questions.py": "c8b415052ea7320e1df3c5475a676d4dd24602c45dbf1868fba6880846e029ee", + "eval/datasets/coding_memory_v1/oracles/atlas-north--code_relationships.py": "bb4305d15a81b4b4ef80b374acb59fee6fc32b213b101dd4b173ac4144d8570d", + "eval/datasets/coding_memory_v1/oracles/atlas-north--condition_values.py": "4bc5b1aff48ef6ad5ece1bd915f6949f72d365bb4815c4d82515b438bc644112", + "eval/datasets/coding_memory_v1/oracles/atlas-north--corrections.py": "7851502a9bc842f840bee1001e42140ee1df08be2730b563b1b197285d14742c", + "eval/datasets/coding_memory_v1/oracles/atlas-north--long_documents.py": "6204123eded20fc0c7aa59b312f8b5bd29aa88a720f7318e32b3c6a17651b57b", + "eval/datasets/coding_memory_v1/oracles/atlas-north--multilingual.py": "5c989b34fdad1e076b61ffb1f7a1371dc635f44a537abc743b765d5f6009004e", + "eval/datasets/coding_memory_v1/oracles/atlas-north--paraphrases.py": "fd189285d0cde9991dea461af9cd18b4203656a69da1298494fccf972ed8a58b", + "eval/datasets/coding_memory_v1/oracles/atlas-north--poisoning.py": "66305d69419736bdbb11a896543933604d898cd5394d94eedda8b92e632a09ac", + "eval/datasets/coding_memory_v1/oracles/atlas-north--scope_boundaries.py": "d9fbbe3924b5178f32f1937485af3b8db242a0e3a08c7bf41a3d2a57f34a8879", + "eval/datasets/coding_memory_v1/oracles/atlas-north--temporal_history.py": "7e7be0d6e558a71d3baad14a8812a6c998ad25e3f52c997a301333e0711f820e", + "eval/datasets/coding_memory_v1/oracles/atlas-north--unsupported_questions.py": "91f574dd678bf244a1fa6f9708e945259bb8ff8e2eb213426be5c3b187be3c3b", + "eval/datasets/coding_memory_v1/oracles/atlas-violet--code_relationships.py": "d0998e13db38d1fae9be6d254431b4fe592f89eb03ee25705f4c70985cd48c89", + "eval/datasets/coding_memory_v1/oracles/atlas-violet--condition_values.py": "f97610aee190dca03b01aa48ffe4aa3f75e099a59fdd867db98f743f4a376729", + "eval/datasets/coding_memory_v1/oracles/atlas-violet--corrections.py": "b7c31f7a9a1f978321ca5445997e0d284deea770c85f6d81a80204e548ad53ea", + "eval/datasets/coding_memory_v1/oracles/atlas-violet--long_documents.py": "20ec2b41cd3b6691164bce096dff16f7bf5be674f922cb637120c0347e60d0b6", + "eval/datasets/coding_memory_v1/oracles/atlas-violet--multilingual.py": "131e7dc978d3f2f7df395b6f1b4a9150051c0ca93c6dc5c386e58d46c8fdc573", + "eval/datasets/coding_memory_v1/oracles/atlas-violet--paraphrases.py": "1e7d9da36e6634ca2d2b11dc97468bd671df133345dcb76859e1dc3cb77dc664", + "eval/datasets/coding_memory_v1/oracles/atlas-violet--poisoning.py": "48460691ac0086c47102eec85387ac03bee9190ac83e3dafc4babc79b71d9bd7", + "eval/datasets/coding_memory_v1/oracles/atlas-violet--scope_boundaries.py": "b96207e9b1fd09f49218102a5b88fde5b9656604df4cfd314e86426e3b6d6075", + "eval/datasets/coding_memory_v1/oracles/atlas-violet--temporal_history.py": "7a196ff9a1db05b0c8f3a83086ae44cdc580dc58e4acc217d1db2575d9711b88", + "eval/datasets/coding_memory_v1/oracles/atlas-violet--unsupported_questions.py": "724564b5b75b9425b6889faae6d5e650bd5c940016c2b96054e5359d9d8b0887", + "eval/datasets/coding_memory_v1/oracles/atlas-west--code_relationships.py": "14295f735bcac4d4ae738eedfbd8fad4593afc914bc835233ec35e2263d43613", + "eval/datasets/coding_memory_v1/oracles/atlas-west--condition_values.py": "8ac70d9ca0c9122567bb9986d2cc5798104966df20e262b8087920e8fb5d82a2", + "eval/datasets/coding_memory_v1/oracles/atlas-west--corrections.py": "89d6a1a7781465c311830b7e769d2a76df41ab5c703ff662b3df1cc4c9a01668", + "eval/datasets/coding_memory_v1/oracles/atlas-west--long_documents.py": "38e67a26f69707089f852657a810ef1fca0395beb5511fff67c30f21ed63df57", + "eval/datasets/coding_memory_v1/oracles/atlas-west--multilingual.py": "bccf6579681d45940542ccb1f936ea1b225f4cf551a3ef31f3f8b1a026856e51", + "eval/datasets/coding_memory_v1/oracles/atlas-west--paraphrases.py": "89c8ce16c682553b27652e176b67a06f3228ae6209fbc8a7264260680f5651a9", + "eval/datasets/coding_memory_v1/oracles/atlas-west--poisoning.py": "fe9beb97d27eacdd87056fa1c9b04c62e2facae9354e765b912de60b35021621", + "eval/datasets/coding_memory_v1/oracles/atlas-west--scope_boundaries.py": "2eb3b1130c54644fcbade52da8f50cad1376a07b51856a962522913cbcbb7649", + "eval/datasets/coding_memory_v1/oracles/atlas-west--temporal_history.py": "116ee20af475da329cb535b0a02bd19ea1e7518b5b780ec12ba396ec896cc116", + "eval/datasets/coding_memory_v1/oracles/atlas-west--unsupported_questions.py": "080969e026215b9fe4ede6d0b4d02e9143e1293bea5d7383681e478426ddf121", + "eval/datasets/coding_memory_v1/oracles/borealis-green--code_relationships.py": "3965671e5e2432c8236c250f7982d4f478d8f1fe1268ffe57c555cf5fbbcd414", + "eval/datasets/coding_memory_v1/oracles/borealis-green--condition_values.py": "92994990327445258984ee6bf1d95f47359ff4b194e1f36e193980a523e6c5e0", + "eval/datasets/coding_memory_v1/oracles/borealis-green--corrections.py": "c5b473f962deacd6d59ea9967626e029888abbdb48f320704bcb6de876ce3f46", + "eval/datasets/coding_memory_v1/oracles/borealis-green--long_documents.py": "54aee2c1031c17b5678db2b0955eda8fafe5990aa9e52f02936584a9b75b9ab1", + "eval/datasets/coding_memory_v1/oracles/borealis-green--multilingual.py": "c458ac5c63e440e3d9486d096278759920d636f73e1901258679251afd86eb60", + "eval/datasets/coding_memory_v1/oracles/borealis-green--paraphrases.py": "250d027660f268cef1caa745cf3155de6e2f4adfaed02ba675eaa83224ffeee8", + "eval/datasets/coding_memory_v1/oracles/borealis-green--poisoning.py": "978f16eb5033c602ec09ed57ba0667edec9ac3e26accc1ce1456bc893ac9e6a7", + "eval/datasets/coding_memory_v1/oracles/borealis-green--scope_boundaries.py": "4b73a2e5dc5ac815c14e08a34acefd2ad426282cfae379ca2fd3060a3784c9b7", + "eval/datasets/coding_memory_v1/oracles/borealis-green--temporal_history.py": "6636d511bbcca51ca85c4eb0ec5ae956b3baaa5f9af89458ceb86329115743bd", + "eval/datasets/coding_memory_v1/oracles/borealis-green--unsupported_questions.py": "5cd4f70627bc60572c720a797bff122ef9bf219f32d265f694affa33b1f9f4fe", + "eval/datasets/coding_memory_v1/oracles/borealis-north--code_relationships.py": "769903aeac3ce360d845ddf1a6e89192737dbb460e1f2e251230b74d40763dd6", + "eval/datasets/coding_memory_v1/oracles/borealis-north--condition_values.py": "f6e69f525fd240a09152989b9d8a529e9abe076e7fc2c98d10bd84f9c6da2b56", + "eval/datasets/coding_memory_v1/oracles/borealis-north--corrections.py": "987ece1c7507a69a973c1e051b27c1d3b104ecda79df48959e49a62da31aa137", + "eval/datasets/coding_memory_v1/oracles/borealis-north--long_documents.py": "011134140cc486a3c089b407adbfbf5b958e1f8c1bffd572c29b8cd5d7f07f7a", + "eval/datasets/coding_memory_v1/oracles/borealis-north--multilingual.py": "1d29536a4a5836f64f56adcd1a965db85bc3371a12ad5bbb5644520c9d9606c8", + "eval/datasets/coding_memory_v1/oracles/borealis-north--paraphrases.py": "09a5fad90c24f75f2f448854c8778aab896c67854c97f884c2d689ead808583e", + "eval/datasets/coding_memory_v1/oracles/borealis-north--poisoning.py": "c1bec2ed9b31103e39db9dbab2be8db401cb0874bb3ce8573a0841a054e28a46", + "eval/datasets/coding_memory_v1/oracles/borealis-north--scope_boundaries.py": "faf51b368d7ba9406442edde29e290c931e67bff2799783dfd27f2226020e15b", + "eval/datasets/coding_memory_v1/oracles/borealis-north--temporal_history.py": "da52f43530149a4a293065674623fa49f83b37cd84b9871044b3bf32762fd8c1", + "eval/datasets/coding_memory_v1/oracles/borealis-north--unsupported_questions.py": "be0abc698692c9fe82f40f76a4694c472882a53843894dddd631ab1ab77aae45", + "eval/datasets/coding_memory_v1/oracles/borealis-violet--code_relationships.py": "9497b7d2373bb9f858e994f7919cff72e7e3c4c4eaa1efda287524b1a182d1f4", + "eval/datasets/coding_memory_v1/oracles/borealis-violet--condition_values.py": "aea13e4b7721019e7bc1f54878bb25b9f27a8a5044fe444e86ba5460deb13e09", + "eval/datasets/coding_memory_v1/oracles/borealis-violet--corrections.py": "bfd446d431afb6c05ad9d8775b90887947b85981037756e05262f3b101d41e7b", + "eval/datasets/coding_memory_v1/oracles/borealis-violet--long_documents.py": "eae829a374c0cd6237fac4ec52fec0d98d7340ec0b307b9b0b0c90c116d4b472", + "eval/datasets/coding_memory_v1/oracles/borealis-violet--multilingual.py": "e821f91999b5f12cff53fdaa956af4e0bb2f929497f95b4e2ceb57a6621e58f7", + "eval/datasets/coding_memory_v1/oracles/borealis-violet--paraphrases.py": "1a99c15898831a1c9aacab951868101d57e59e91ed90a2777015a26dbcc3043c", + "eval/datasets/coding_memory_v1/oracles/borealis-violet--poisoning.py": "411877fc07b053201a63aa4e45cb499344abccb0a562c236d53a672078b1eb63", + "eval/datasets/coding_memory_v1/oracles/borealis-violet--scope_boundaries.py": "b198dfb636910953ba372edf1cd6db382d4e5725bf7f820171be8c63f9fedf7d", + "eval/datasets/coding_memory_v1/oracles/borealis-violet--temporal_history.py": "3de5efd6ef661572191c6aa426fb10caf9f4bed4aa495b6298815aab3acdce22", + "eval/datasets/coding_memory_v1/oracles/borealis-violet--unsupported_questions.py": "baf5dd85e86a97dedb0b395b5c574ea5ffcdad7a46bbbe9787cfdc33d6c7a06c", + "eval/datasets/coding_memory_v1/oracles/borealis-west--code_relationships.py": "48e53b3a561f91fdf4eedb2d63aad65eece8ae80188bb3d5bdcaf184fb0b6945", + "eval/datasets/coding_memory_v1/oracles/borealis-west--condition_values.py": "2dd7a524bd2a2aa2fbf212246880d4e4ecf03f503b70f25179f3ab6bab28a07a", + "eval/datasets/coding_memory_v1/oracles/borealis-west--corrections.py": "7d3080e05544e10243040fe0a969a8c524e4262f8f5c38abdc3d8e06c67fdabd", + "eval/datasets/coding_memory_v1/oracles/borealis-west--long_documents.py": "fc42e935879e7a81f6eef9fe72f8683244501ae10654008a7863d397ee73058c", + "eval/datasets/coding_memory_v1/oracles/borealis-west--multilingual.py": "19181c442ce5f6941485190ecf050e455914dae988f86efa42665cdef0d835a2", + "eval/datasets/coding_memory_v1/oracles/borealis-west--paraphrases.py": "f7a5cb842aae3d2bc8d62ea40325b332a8bd4f013b76bac8bf154fedb3dbcbe1", + "eval/datasets/coding_memory_v1/oracles/borealis-west--poisoning.py": "285a2526a6be4e556a564899e9a376ae1286fef6f0ad22eb9289fba1905a2b68", + "eval/datasets/coding_memory_v1/oracles/borealis-west--scope_boundaries.py": "fb994823743ff5b4853b142c17779778bdb11296958e033e5c6be598833a4cf5", + "eval/datasets/coding_memory_v1/oracles/borealis-west--temporal_history.py": "f30a74c85ad654a7450e20d07baceb3b4fb2a56b454d7b43b854bef3cdb2cee6", + "eval/datasets/coding_memory_v1/oracles/borealis-west--unsupported_questions.py": "01cbbd0b21a745720514aea6a21a98a3cccdbf0c0518bd5508019433b964776b", + "eval/datasets/coding_memory_v1/oracles/cinder-green--code_relationships.py": "8b08e7c65289515a9eee2ae5f4cd60ce94350e9a561225bef0e8f8815ea28f79", + "eval/datasets/coding_memory_v1/oracles/cinder-green--condition_values.py": "92e7d75d38f5dbcc685075e947005a9ae5a410207e7f17aa770b034b8690cbe5", + "eval/datasets/coding_memory_v1/oracles/cinder-green--corrections.py": "424cbf0924a8cbc1a59a6d7bc82b5527e41f52c8ee9f558ea3e398ab4d913919", + "eval/datasets/coding_memory_v1/oracles/cinder-green--long_documents.py": "2a52df9cef3ee66357847322794a382c927b828c6ab1a2819a799ff55d53657b", + "eval/datasets/coding_memory_v1/oracles/cinder-green--multilingual.py": "e47e791dd15b045a6a958ca114b1359ca8537e160274b015214746a2564363c6", + "eval/datasets/coding_memory_v1/oracles/cinder-green--paraphrases.py": "9e92a79ec00231cb381ce58e389dfdb2520a17ecf1e9c2e39e79501449e9ddd0", + "eval/datasets/coding_memory_v1/oracles/cinder-green--poisoning.py": "33213e6f194332969da88f4748c8f26ef93219b6c4bc710a142082f8147ccfcc", + "eval/datasets/coding_memory_v1/oracles/cinder-green--scope_boundaries.py": "8d2489da1fba137981e45e42c843f9cbd17e851c82ddcdc5904b4a86af0d9692", + "eval/datasets/coding_memory_v1/oracles/cinder-green--temporal_history.py": "29403fd08f0de7da3dfcc6c181e1f448a5c5264e95ff3924f3066508d85468b1", + "eval/datasets/coding_memory_v1/oracles/cinder-green--unsupported_questions.py": "ba21014188f9ae849e0bb79babe163194703c4cf01f0600f4c9f71058e7fed56", + "eval/datasets/coding_memory_v1/oracles/cinder-north--code_relationships.py": "46280659557bd08afff185352438bb63dfd308dabf5c7c1862c2f442bd171d48", + "eval/datasets/coding_memory_v1/oracles/cinder-north--condition_values.py": "6c62d673b1b6cb4b5c89ff2af4ecf57c2c9d73074a01ab92b8fde57815cab783", + "eval/datasets/coding_memory_v1/oracles/cinder-north--corrections.py": "8245b196072389517fad6d76ed7709d7e12eb52e08732e4ddcfc40fc1d986879", + "eval/datasets/coding_memory_v1/oracles/cinder-north--long_documents.py": "6790c7b89d80ab909d9e1a31b59bfa5cd199cddabed2e159daf7c5c049ff9a43", + "eval/datasets/coding_memory_v1/oracles/cinder-north--multilingual.py": "9532bbd14506cac0edcc09f474e7c96a402903de947e0e76f5c8aab4bff37646", + "eval/datasets/coding_memory_v1/oracles/cinder-north--paraphrases.py": "cc904350ae7d40e26d91d482ac15d87ec8022d4c82905021dd38003c5fa2d067", + "eval/datasets/coding_memory_v1/oracles/cinder-north--poisoning.py": "e8e451c49c98744b4a484141f86d337db4ec5326264b1500893a20c3451c5ae4", + "eval/datasets/coding_memory_v1/oracles/cinder-north--scope_boundaries.py": "50d42fc48aa70d55b6a3b39c9bf17437726f37e1e5ae9870d415fccad92c7bae", + "eval/datasets/coding_memory_v1/oracles/cinder-north--temporal_history.py": "14b1336fbaed11cad455f588bbe078effc31f5f357f670e1f189ec3d78b172f5", + "eval/datasets/coding_memory_v1/oracles/cinder-north--unsupported_questions.py": "0a2f740700a990d22b4c3051aa3e4260e3df4ce65931d419052f74e05c750a67", + "eval/datasets/coding_memory_v1/oracles/cinder-violet--code_relationships.py": "0e29053f11fe618c01fa90a274688f18ba8a7f3cfea56757a96d5a5bc95b61a8", + "eval/datasets/coding_memory_v1/oracles/cinder-violet--condition_values.py": "ca57ac4e22e9b4739217020d7792ac642326935b8db9305ead2ec5bea2a6e456", + "eval/datasets/coding_memory_v1/oracles/cinder-violet--corrections.py": "9edbfaa58a99776e835c325e50a77df5b4b302b3ddd0932dc2a349a77b4100b5", + "eval/datasets/coding_memory_v1/oracles/cinder-violet--long_documents.py": "4b3fbe7380201285ac4b4a1ec4d8238c5663b97af01cbdc8ffff8fdf2a9467c5", + "eval/datasets/coding_memory_v1/oracles/cinder-violet--multilingual.py": "b0bba5ecf669f3368ef29373b755dc1296502665584574c771ebf4664debb5ce", + "eval/datasets/coding_memory_v1/oracles/cinder-violet--paraphrases.py": "e146f7aa1ffabf3b313251b59c59e1836c0cd17901d07fc6ef5334eaccf8bf36", + "eval/datasets/coding_memory_v1/oracles/cinder-violet--poisoning.py": "692ee07e750fb08652a1ca6bc6dcbbef77219701585f1336c6037dae5a7c5767", + "eval/datasets/coding_memory_v1/oracles/cinder-violet--scope_boundaries.py": "8ca0b911d38a72e4ecce6871ba3d0c7dbab433b0541cb64b116f7a88d521554c", + "eval/datasets/coding_memory_v1/oracles/cinder-violet--temporal_history.py": "6ba5400c5435af64c80a7366fdf49fea132f1011f9d84f9d0cc5f2f2f30a67ac", + "eval/datasets/coding_memory_v1/oracles/cinder-violet--unsupported_questions.py": "1d3324db0a085be30ce3dff1a2ec3866b8592a5ef6b43f39211a2e3a59fc1cd6", + "eval/datasets/coding_memory_v1/oracles/cinder-west--code_relationships.py": "0149333a3fb0f8d60939bf927921305eae447f7511c74f042ff3c880eadff825", + "eval/datasets/coding_memory_v1/oracles/cinder-west--condition_values.py": "3158243f9f7f0b92aaee4164379159b46143acd11fc3d41a708630c1d33ea22c", + "eval/datasets/coding_memory_v1/oracles/cinder-west--corrections.py": "a9fbb89aa629a23e7a1fddf4fca09da40365bbc7a9a6192d805e498fe17f93ff", + "eval/datasets/coding_memory_v1/oracles/cinder-west--long_documents.py": "3c702c653a8ddff84eb5c24a77567aa63d2069c48ed174bcf29065c0ddb30ec6", + "eval/datasets/coding_memory_v1/oracles/cinder-west--multilingual.py": "b8f5f7e00e6edac3f8e0b7e8bc09ea6c7d661370fa2226eb1795306ce3c634a7", + "eval/datasets/coding_memory_v1/oracles/cinder-west--paraphrases.py": "10361be87e9d9cb98d2ff1bbc18252178d475a5c5f2efb77bc81dea05109df05", + "eval/datasets/coding_memory_v1/oracles/cinder-west--poisoning.py": "889a5a43604f7d1677ee0f28683abfc48a8a597b052f558c1b4d19c5f33444ec", + "eval/datasets/coding_memory_v1/oracles/cinder-west--scope_boundaries.py": "b84e78812baaa92422226c06d78cfe7c1be260e2bfcf9c5bb1ed73f2f3cc136b", + "eval/datasets/coding_memory_v1/oracles/cinder-west--temporal_history.py": "9d99a981fca0c801a0a52e2af24cd4262dad72f4f34cd84894784285d241bd05", + "eval/datasets/coding_memory_v1/oracles/cinder-west--unsupported_questions.py": "43b8f854961d9cca5cb83b53b9e07d22388d0c7db406dc8e4f8cfdaa5f89fffd", + "eval/datasets/coding_memory_v1/oracles/delta-green--code_relationships.py": "97a08f2bf9b1a11f95a062f7df27d69db3e9dbc421837347acfe62111af02885", + "eval/datasets/coding_memory_v1/oracles/delta-green--condition_values.py": "5e53695288b84a526c61b748b170f6d6219f94a942edc0887ea6157535ad8d52", + "eval/datasets/coding_memory_v1/oracles/delta-green--corrections.py": "c6d16232ebe6bdb0b5c94db1ba4f108257b47ee8f8b96cda5f8e17212022b542", + "eval/datasets/coding_memory_v1/oracles/delta-green--long_documents.py": "bedb6f495437beada344ce6cdd92954108aa4c639622fd1cc294d2a1139b26d9", + "eval/datasets/coding_memory_v1/oracles/delta-green--multilingual.py": "dbf84f4e8c016c5e60fdac5fd3ac2faf79fb0534f5c2ac534491acc31d5f3530", + "eval/datasets/coding_memory_v1/oracles/delta-green--paraphrases.py": "ea8f00c86e8fbc2ab10a448f48019146181d0e55438def6f47f41f0ef3181be4", + "eval/datasets/coding_memory_v1/oracles/delta-green--poisoning.py": "4eb4e289e944e7762c39ce85a8fc2582e310de6a86cbc1cc36b44112653140ea", + "eval/datasets/coding_memory_v1/oracles/delta-green--scope_boundaries.py": "2fe9b96deecfb488430abd9657adc6b3ea092b18f32c41c1669d29175ec6e860", + "eval/datasets/coding_memory_v1/oracles/delta-green--temporal_history.py": "531cbb953400584356cb6525bc1a2e559f7ec3867d845c736d8026a046b91e7d", + "eval/datasets/coding_memory_v1/oracles/delta-green--unsupported_questions.py": "edebf25c2168411fdefae1307a71d829224dba3dbc7ea000054416e52123d5a3", + "eval/datasets/coding_memory_v1/oracles/delta-north--code_relationships.py": "9357ad0d5f6a1cc787f980d0d506b07808538179f71ae3e8ce1c3eb655d24e35", + "eval/datasets/coding_memory_v1/oracles/delta-north--condition_values.py": "4199f7409696a260b1531f4ec7cfd15555b4d58d06b0b8a88b1c7621fe7d328b", + "eval/datasets/coding_memory_v1/oracles/delta-north--corrections.py": "b8bd58db7c5ae93d33e6223cc34f0e09d58831361ac013ec08277fe2c102ac17", + "eval/datasets/coding_memory_v1/oracles/delta-north--long_documents.py": "0f08b744446b1c52e8fe86626ab05c171377f17c46e604cb2ea647ed5d0daddd", + "eval/datasets/coding_memory_v1/oracles/delta-north--multilingual.py": "ada9eb404190d05637cea59e3f8ed541aea2c9e0f9be0d166885d6789351f7a3", + "eval/datasets/coding_memory_v1/oracles/delta-north--paraphrases.py": "04ec2e8308f47f46e38c89f4a2fe44deef4a0b20a1c8517395ad00983bf7361c", + "eval/datasets/coding_memory_v1/oracles/delta-north--poisoning.py": "d72ea7b98db47e4de03884e2786206dc3da5cc08e5203c86b1d38a2d48c2fd47", + "eval/datasets/coding_memory_v1/oracles/delta-north--scope_boundaries.py": "cedb3234b22a40a7b889f50cfb5751fbba31e46f541b19feb2c4759c8007bac3", + "eval/datasets/coding_memory_v1/oracles/delta-north--temporal_history.py": "2f7c1bd4ebc051eee411d11251aff2ee5fee602a0800410bc0dbd2e86e99628b", + "eval/datasets/coding_memory_v1/oracles/delta-north--unsupported_questions.py": "f1f0b9858c806956122144c6e769f3baf9f87f07ff92eec98eb58d8019e6fba2", + "eval/datasets/coding_memory_v1/oracles/delta-violet--code_relationships.py": "79579639a60d96365781e853d1319a2bf85d9a2b4d8639410da7e3c1efc364b2", + "eval/datasets/coding_memory_v1/oracles/delta-violet--condition_values.py": "27defbfe7c237f971ae29e8f000f098eae32a11f3afad06f2bf3b27af432d902", + "eval/datasets/coding_memory_v1/oracles/delta-violet--corrections.py": "b3b685a5c33d1086e8ab4258d26c5ec9aa7396b8250a250351f0a6057fcf2075", + "eval/datasets/coding_memory_v1/oracles/delta-violet--long_documents.py": "4a5ce75372c8762bcd7986f8ceef454800c3636124ffd9de3a8315d210ae85f1", + "eval/datasets/coding_memory_v1/oracles/delta-violet--multilingual.py": "7ef19f25e97a24522b79a9d7e7f3d2d9e395a1fc2c60564e2f66081533bdacb5", + "eval/datasets/coding_memory_v1/oracles/delta-violet--paraphrases.py": "b6e00019bead0a5bc0facdac886b48fccfd3cccaa254267fda780e933d57904f", + "eval/datasets/coding_memory_v1/oracles/delta-violet--poisoning.py": "3e3e1e1fe04ae3e0c615ca21e794fead5d06f887d6fe7347fb9122684c8b8017", + "eval/datasets/coding_memory_v1/oracles/delta-violet--scope_boundaries.py": "336ae66910d533344712a2d87126f53df48d43eae9e7d17797938c3f9c560204", + "eval/datasets/coding_memory_v1/oracles/delta-violet--temporal_history.py": "a81cb5d2a5eb521dfd161ffbd21bd366777b1239f2d3412acc77b0a9c7c7d82b", + "eval/datasets/coding_memory_v1/oracles/delta-violet--unsupported_questions.py": "3341895608073374b98e67a88489abb9020fed5bb692231a0a61e15c8de0ddf1", + "eval/datasets/coding_memory_v1/oracles/delta-west--code_relationships.py": "e8f17f3838cb74e1ade0cfd6edf540dc086507b51ca67c2020ed490d75dbffe5", + "eval/datasets/coding_memory_v1/oracles/delta-west--condition_values.py": "a935a6e7a4081737eef76a93d54b34dc885b4699e02d797f1be78286dd642a2b", + "eval/datasets/coding_memory_v1/oracles/delta-west--corrections.py": "ee407e5aa8d57254bb7029d537bb2b59a297958cdff9bf271de3787b6448241e", + "eval/datasets/coding_memory_v1/oracles/delta-west--long_documents.py": "55fa08b75982d0547f0c2f13cdc6522d0fbd33efb4af9ae8c6ada11b2d46f321", + "eval/datasets/coding_memory_v1/oracles/delta-west--multilingual.py": "ce8a3e45ad8417ca1798579a22e12eeb7d170a11ebfb865746388eb0e7dca848", + "eval/datasets/coding_memory_v1/oracles/delta-west--paraphrases.py": "e6458588e20e76596e24be6cc838bd4b0d5a47b05b7bf0bb6017b0cecd0d6a68", + "eval/datasets/coding_memory_v1/oracles/delta-west--poisoning.py": "4c70955a2650d13eba24d6e933f765e0bd40da0a1cd8b920416989a76bdaf2d7", + "eval/datasets/coding_memory_v1/oracles/delta-west--scope_boundaries.py": "cfe9120c49224b6f9ed86ea7d85e496000bef0648c92c7fd474c540e892863cd", + "eval/datasets/coding_memory_v1/oracles/delta-west--temporal_history.py": "3527edc8a85c71f889ff0b2ac8bf9ab005ad5bb6c95bcd917dedae44aa3efc72", + "eval/datasets/coding_memory_v1/oracles/delta-west--unsupported_questions.py": "8c40923b8d4bd6bfc78ff6548ef9a840319b7ee4ea967554f26a8d24a4349935", + "eval/datasets/coding_memory_v1/oracles/ember-green--code_relationships.py": "cc7f82ad85363ee35ccd75744159bf4c4f07965e396bff07989cca7a55071679", + "eval/datasets/coding_memory_v1/oracles/ember-green--condition_values.py": "3fa888c125812f1488de9e021358cd531a9d50bb7d4b6173be3656268474b635", + "eval/datasets/coding_memory_v1/oracles/ember-green--corrections.py": "7cc639d4bd7b287ed0e3f2188491d6e4866e021b21874b34b365c1eb2b2d6d68", + "eval/datasets/coding_memory_v1/oracles/ember-green--long_documents.py": "972e7879a30ab26f86f09bce6dc9853c7f96124d0bbe5062111607496bcc9a95", + "eval/datasets/coding_memory_v1/oracles/ember-green--multilingual.py": "0f0585bd3264de3d5fed5928b54ff4fb87f71b2edb1d35c83a534251121124b4", + "eval/datasets/coding_memory_v1/oracles/ember-green--paraphrases.py": "a3c15541bb87cc862d1c9764cfdc1ffebbc831876b55167470ecb3b66519740a", + "eval/datasets/coding_memory_v1/oracles/ember-green--poisoning.py": "ace5b7b12c4ca068739af4bc5e215af1af7fe153399ff17ffb7a67b8b9aa1f94", + "eval/datasets/coding_memory_v1/oracles/ember-green--scope_boundaries.py": "daf132e130c5743d67129c1564f329fbb2670d1242648920b646a83bc2236e4b", + "eval/datasets/coding_memory_v1/oracles/ember-green--temporal_history.py": "1438f09f0ae41b1f6b42930518203a36c7de31a870b684dbb7f16e71a37c784b", + "eval/datasets/coding_memory_v1/oracles/ember-green--unsupported_questions.py": "da27396afbc1529c61972f739cefb26dcaf97bea529c084518a95080eac4fe78", + "eval/datasets/coding_memory_v1/oracles/ember-north--code_relationships.py": "debaf885151c3f47660053ff50e015338b8aa9bca089d7e901ee8bc34f7085e6", + "eval/datasets/coding_memory_v1/oracles/ember-north--condition_values.py": "109a7ee90c04da83f57a94a9a724dcf98a3e96405c7a456686c198ff5d85a69c", + "eval/datasets/coding_memory_v1/oracles/ember-north--corrections.py": "c246ba011c68d3ad0a4ee1d14231304a7f17f28027f1764e60da93062cf1cd3c", + "eval/datasets/coding_memory_v1/oracles/ember-north--long_documents.py": "176200cf2a011c7231ed4cadacbe86b1eeb695fbbee282bf4c4a96430d98fe84", + "eval/datasets/coding_memory_v1/oracles/ember-north--multilingual.py": "e850a910515e71a27edff26f7bedf39d8cb75c726e48357a2c41d4da051be6e9", + "eval/datasets/coding_memory_v1/oracles/ember-north--paraphrases.py": "0a6f03310e44abcf4ca61d69e22edbd21ce47a17629adf4f5079f7565534b469", + "eval/datasets/coding_memory_v1/oracles/ember-north--poisoning.py": "a81ee758fc57bcc58bf0bf2a38dae8b6eb4aeb14949063206e32cdff6f301dfb", + "eval/datasets/coding_memory_v1/oracles/ember-north--scope_boundaries.py": "fe42df24612c66b175f1c98794ed9f13a1a7819418e4aeb5ff98531515221de1", + "eval/datasets/coding_memory_v1/oracles/ember-north--temporal_history.py": "410f9770097228724a5af1357003ea23caefe1226275d030d46efa535c42d306", + "eval/datasets/coding_memory_v1/oracles/ember-north--unsupported_questions.py": "eebe43443938693a74081661d2ff3733b39c6dac5c7ecd88c4dbab5ad18689ee", + "eval/datasets/coding_memory_v1/oracles/ember-violet--code_relationships.py": "c00a30669fbd1c740b44e17c7c7369582c28a16da3aeec9d27420b8aef7c553a", + "eval/datasets/coding_memory_v1/oracles/ember-violet--condition_values.py": "1645e7c56c870f4367c263ccc9041181b1d187a698252203ca651235db59e684", + "eval/datasets/coding_memory_v1/oracles/ember-violet--corrections.py": "735f1d125d2ade6f3eabbf1082a408fe6afd4328cd954833e23df275df498e37", + "eval/datasets/coding_memory_v1/oracles/ember-violet--long_documents.py": "e1ded8e9641639443a6b14da18b2e0cbcddc6fc6045bf0b57404b59b59327dce", + "eval/datasets/coding_memory_v1/oracles/ember-violet--multilingual.py": "fbf04cda8e1231c7ac4c64d4c1856f7bfafcb6f1d016de0b1bdbbaaafcfe03b6", + "eval/datasets/coding_memory_v1/oracles/ember-violet--paraphrases.py": "2fd495def306778073df30b20dbc7a1500c06c90638acf6d9b9280db78dec53a", + "eval/datasets/coding_memory_v1/oracles/ember-violet--poisoning.py": "335674216a418254a01a7f5505344af20ad499a5fe41cb9c8654a0705574a18e", + "eval/datasets/coding_memory_v1/oracles/ember-violet--scope_boundaries.py": "078d60287888903e90343d144d8f54e08a3751ffcba82fd827ca0c9f8ee75fe7", + "eval/datasets/coding_memory_v1/oracles/ember-violet--temporal_history.py": "f7ad47f3253fe922977cbd0b63679abf00559f1b2dc895246e09840652b2c41b", + "eval/datasets/coding_memory_v1/oracles/ember-violet--unsupported_questions.py": "b91545082a31952351bc4c08d6cc09132432126dfd772952c818396bd7a68af1", + "eval/datasets/coding_memory_v1/oracles/ember-west--code_relationships.py": "74fb3e0d99a5bd4f54b8b57f6040ce7978a81bffa1ae70f9069fb2322e6ea581", + "eval/datasets/coding_memory_v1/oracles/ember-west--condition_values.py": "26d605b42d18cf029dfdc29124eea898e0dfa5642c43c1c775007fb443ba6cda", + "eval/datasets/coding_memory_v1/oracles/ember-west--corrections.py": "65c2ef8544b42ecb6b3ea9ce8bc95ec5f0ea8bb07c2b74d52e7880d99962f563", + "eval/datasets/coding_memory_v1/oracles/ember-west--long_documents.py": "4c8e54c7ec930d6cc782fa1a0519a0f1e0e916e3929902e13199775cefdaf338", + "eval/datasets/coding_memory_v1/oracles/ember-west--multilingual.py": "356e407f47d39e7f53337ffc4f27c68935a2c831301dc3dd006b2939cc61decd", + "eval/datasets/coding_memory_v1/oracles/ember-west--paraphrases.py": "c91cf574cf8b5aadaa6c9331c3f8f4a8fec7ec33c227f20894f463d81d110a7d", + "eval/datasets/coding_memory_v1/oracles/ember-west--poisoning.py": "5f6b8740798a7a093994748ef4b11e0e1de0be562c7f09790e1f0ebb9ed21f7d", + "eval/datasets/coding_memory_v1/oracles/ember-west--scope_boundaries.py": "5e20c5154818cc29b7e71444054d67836eddc882f2f17be8aed78afe31a9037f", + "eval/datasets/coding_memory_v1/oracles/ember-west--temporal_history.py": "fc51d1a39693d1f7fb2db4bf1ffe865d4c44848c159188cdcfda54618c0d8fe2", + "eval/datasets/coding_memory_v1/oracles/ember-west--unsupported_questions.py": "079c421cae974bcccc2107dac74421dd5bc2d1b40f7d91cb79a8014af526d260", + "eval/datasets/coding_memory_v1/oracles/fjord-green--code_relationships.py": "b5e4a284bad1d7ff81d5be9cf1faaaaf5e718cb51624189482624721cb459f1b", + "eval/datasets/coding_memory_v1/oracles/fjord-green--condition_values.py": "63c152b8da5655ae5c701cbb51099340d5f5103fcc47840239098a7366d24151", + "eval/datasets/coding_memory_v1/oracles/fjord-green--corrections.py": "b7b33b8de153c205ff4fdaadd5538eae8a22fd4401e70db9f645dae205c62cba", + "eval/datasets/coding_memory_v1/oracles/fjord-green--long_documents.py": "624cf014bc795a24e2f1f4fa2f9dbf11d2ad1c38f81cbc58ab3948f96c48c8c4", + "eval/datasets/coding_memory_v1/oracles/fjord-green--multilingual.py": "f8d2483fd9266946c980764feba0eda27c95ea2ff11222a4dd4247fb537f1f00", + "eval/datasets/coding_memory_v1/oracles/fjord-green--paraphrases.py": "151be253b730d5c5b4bbcabcce3bd218477241ebf0f1bd5fe9dc8d70798feb6f", + "eval/datasets/coding_memory_v1/oracles/fjord-green--poisoning.py": "e7b0efca3813bbb1d4ea9cefa752b4318942b118bb61ee9dcbbf640daa66644d", + "eval/datasets/coding_memory_v1/oracles/fjord-green--scope_boundaries.py": "56177cfb45f32c548be83cc78197fcf5b53826f01cca69e471535476184ae8f9", + "eval/datasets/coding_memory_v1/oracles/fjord-green--temporal_history.py": "2c79d26467eb8443d743bb8823cc1f2a5d1e5da7c887754505baa130e37f1b2e", + "eval/datasets/coding_memory_v1/oracles/fjord-green--unsupported_questions.py": "fb5f222f4e0715379591f559036c0549f67a7b6e3491e880c6cf8d5b91fa9000", + "eval/datasets/coding_memory_v1/oracles/fjord-north--code_relationships.py": "07402078200db9ab15fc61a72e31d9df0938edfb1e9d5b7e532b422dccab15aa", + "eval/datasets/coding_memory_v1/oracles/fjord-north--condition_values.py": "8c1549b5a444c9cbf1304fcca13b79263bf8d9f0b4f41c16da539b3bf5da3bf4", + "eval/datasets/coding_memory_v1/oracles/fjord-north--corrections.py": "a2634af73134c025de1b6239756ee2051e505e0a535221947babef5ba161fa13", + "eval/datasets/coding_memory_v1/oracles/fjord-north--long_documents.py": "4f80f3673afd1320d16a6bd45fefc85522d6f4bb889ff96f6e99410580feec09", + "eval/datasets/coding_memory_v1/oracles/fjord-north--multilingual.py": "3a9c953fbe76eb7374992a6a81d35b8dc0a635c02b90f328af140bbda2acc180", + "eval/datasets/coding_memory_v1/oracles/fjord-north--paraphrases.py": "ec2d9d7bbd678be73946cba8177184839505cd891df410ae8e676fc254df2ca5", + "eval/datasets/coding_memory_v1/oracles/fjord-north--poisoning.py": "6d9b91b4d53925d58d3c0cea694f591572a53184f23816101af0aa644aa1b86d", + "eval/datasets/coding_memory_v1/oracles/fjord-north--scope_boundaries.py": "2e98afa064e8a295a78bea654014a92734696a235fb0538732bb8d85799c7287", + "eval/datasets/coding_memory_v1/oracles/fjord-north--temporal_history.py": "4059ab8e427c2b6fd68155992404fbf493c7ebdf0a4357aa5c5f4ff0c67808f9", + "eval/datasets/coding_memory_v1/oracles/fjord-north--unsupported_questions.py": "79f14e18e282c1b6bbe21ef2affc9dd8a259a171c2e4975d42363f7c91689c1e", + "eval/datasets/coding_memory_v1/oracles/fjord-violet--code_relationships.py": "9ea8a9ade1001bcd43929782c3410f35fc5056e438342ca848fe0b1b0ff56170", + "eval/datasets/coding_memory_v1/oracles/fjord-violet--condition_values.py": "09e223a1c0bff6c95a5863fdc28dc00ed96b90887cd635d014712058079ccbfe", + "eval/datasets/coding_memory_v1/oracles/fjord-violet--corrections.py": "f822c373fc4a93189fdce09999a9dcbe8797b0c0dc780eddd4e342edf21dc14e", + "eval/datasets/coding_memory_v1/oracles/fjord-violet--long_documents.py": "983aeaae1e14937a740218228778be3e1938cdf1ff9228eba44064336188f297", + "eval/datasets/coding_memory_v1/oracles/fjord-violet--multilingual.py": "6e929f8846cfc756c71bcc21bc0f27ef8fc23270d21292fc35d70e6bbd31c066", + "eval/datasets/coding_memory_v1/oracles/fjord-violet--paraphrases.py": "c644d9f01edbd213447618bc269f56804450901faf4f4660c1f666d4e9b4ada4", + "eval/datasets/coding_memory_v1/oracles/fjord-violet--poisoning.py": "cf3849312f96b0072efdfd4e858abb24d7e6a808bf883bcc9dde5dd5a4594364", + "eval/datasets/coding_memory_v1/oracles/fjord-violet--scope_boundaries.py": "373d10e20713bdc26849706525f765f2be4ab57e87e576e39a1a1dd4e7feed88", + "eval/datasets/coding_memory_v1/oracles/fjord-violet--temporal_history.py": "4d2c36480cc04d43aa8f535bf998968b474f2f596d5349dcd42684c5a667c6a4", + "eval/datasets/coding_memory_v1/oracles/fjord-violet--unsupported_questions.py": "b381d2016dea525eebd8de8d026dc48ad2080b93823a95d6e2bb06d29cc86775", + "eval/datasets/coding_memory_v1/oracles/fjord-west--code_relationships.py": "e1e10ed9ee8a4d5c2e798c2bf04cba5c39eaa3873901bd93724af166ba2479ae", + "eval/datasets/coding_memory_v1/oracles/fjord-west--condition_values.py": "427764a62d2e11f6c9fe4b2cdcd32fa92f61749403dff648601a47187e38787a", + "eval/datasets/coding_memory_v1/oracles/fjord-west--corrections.py": "4aa041aa87f4c8628d113bab855a151c2e38bd6919cd02224827fd07d060fb34", + "eval/datasets/coding_memory_v1/oracles/fjord-west--long_documents.py": "89875548e6952ce810d8983b0eaf2724bf8f329bf44e4d9193cfa3405e1ce80e", + "eval/datasets/coding_memory_v1/oracles/fjord-west--multilingual.py": "8b5c14cdb26941d1ae25701bc99efc5fe4ae6e50fed3c65b193952444f1593d1", + "eval/datasets/coding_memory_v1/oracles/fjord-west--paraphrases.py": "bcdc9a5f1fe45ef63fd5e6b728c83c7a8d05c7b80957e3276c074700b409e924", + "eval/datasets/coding_memory_v1/oracles/fjord-west--poisoning.py": "5504e1bab0865fb3041c88f151938789d79422eff91a3ce72f4a41fc14e2f897", + "eval/datasets/coding_memory_v1/oracles/fjord-west--scope_boundaries.py": "63f226631a663dca0ef62a4f32bb98fbe1e30e3b1f030ee42c3f7f2bf6a09af0", + "eval/datasets/coding_memory_v1/oracles/fjord-west--temporal_history.py": "75c9582a16db2c89f12506bd7a4115fcd871c6da63e80429a8e70e31c1f5f862", + "eval/datasets/coding_memory_v1/oracles/fjord-west--unsupported_questions.py": "05e4ba65be7cb4e6500af5e14721c70f93fd7390abe58a60f6f8e4f8e9b0ae5c", + "eval/datasets/coding_memory_v1/oracles/grove-green--code_relationships.py": "bd5619518e839f84ca03297c4f7842a2944d8b445ec1863a8dfc68e4c2114a03", + "eval/datasets/coding_memory_v1/oracles/grove-green--condition_values.py": "67377847b6ff1ab865f5c25351fcfa1ee13e6ec6a089444cb3eb0709b660148b", + "eval/datasets/coding_memory_v1/oracles/grove-green--corrections.py": "c07e8a03c6c47818f9c574d4f21b27668989711fb5e0af3b5cf1a87b293abcbd", + "eval/datasets/coding_memory_v1/oracles/grove-green--long_documents.py": "3f75fc2158c66a19dbbd19209fc80026339abe93a2282ff55c7c01429ac549f3", + "eval/datasets/coding_memory_v1/oracles/grove-green--multilingual.py": "19b1f800b38a11cf0f50df6da3a011806098561e32a279b78e1806de9b8c11fb", + "eval/datasets/coding_memory_v1/oracles/grove-green--paraphrases.py": "a5996376b9f589283dbe301dd1e0644af92a8b6a0222b51b99ed6fbb83570782", + "eval/datasets/coding_memory_v1/oracles/grove-green--poisoning.py": "c4dee0969e39b52dc9b697e45587c8cd84926a97c9a48f7211c0a00571bfb657", + "eval/datasets/coding_memory_v1/oracles/grove-green--scope_boundaries.py": "8d45422f0a1a2115deb90f560409fdad1105b7cae1f03052f8ebac404ba9517c", + "eval/datasets/coding_memory_v1/oracles/grove-green--temporal_history.py": "abc1b386173c8252ac10f11a36fbcdd6d91c731686108a6d32c62d17955d21b9", + "eval/datasets/coding_memory_v1/oracles/grove-green--unsupported_questions.py": "1ee09741a1c94a1fa83c7edf5b6c07d8da2e45826f748d899ec71917e8369d54", + "eval/datasets/coding_memory_v1/oracles/grove-north--code_relationships.py": "fd75d734a5f45a5ec54b125c1d379df67c5531659f212a2f5c1c5e88169167e0", + "eval/datasets/coding_memory_v1/oracles/grove-north--condition_values.py": "c600c0e5f50e352406b99527b050f4df6dc8ab8920b5b946de190ad7629b5e6d", + "eval/datasets/coding_memory_v1/oracles/grove-north--corrections.py": "4d6fdbe4f5eb9c8e4bfc2a974e6c3cbf1d124d0b20d3f050c84e766f507b4be7", + "eval/datasets/coding_memory_v1/oracles/grove-north--long_documents.py": "5b0bbb9f0a90eb377cbfe45ff6f37077a9ce32c7c6730720f0bfd49b844fdf4b", + "eval/datasets/coding_memory_v1/oracles/grove-north--multilingual.py": "525ea2fdad66d66d7b90892e9424ebae281c1dabfb49f2668784a687b7ec894e", + "eval/datasets/coding_memory_v1/oracles/grove-north--paraphrases.py": "1105e52a8c8d3ff79c64b59ec5f23d66d742b8dcecf91e8e62060895c7371f4a", + "eval/datasets/coding_memory_v1/oracles/grove-north--poisoning.py": "7d32e533f6dac57e5d651868386686d2b78e7db1cc29b1860fede23fc9c49813", + "eval/datasets/coding_memory_v1/oracles/grove-north--scope_boundaries.py": "7bea4129589ea34ba10667a7035b126a32b423ec7f3b1ff2b10f6ffc9aa1c2ee", + "eval/datasets/coding_memory_v1/oracles/grove-north--temporal_history.py": "859b6504250f5746ce197e4a0a7c1006f1c2257b939c53a493f9d36f6d2b7d6a", + "eval/datasets/coding_memory_v1/oracles/grove-north--unsupported_questions.py": "6b893c4c124f071e9ad5d9946448b070c2b2f5f10da6f058e60840c18510fd0f", + "eval/datasets/coding_memory_v1/oracles/grove-violet--code_relationships.py": "09bb7df6b115b83cb98a4274b2d883e81b809643ed4f912b383a9e09a7f7a1f9", + "eval/datasets/coding_memory_v1/oracles/grove-violet--condition_values.py": "a68c126f0d7b2fc0950baa5df095d7d513cdb8359883404bad3c1c84c8de666d", + "eval/datasets/coding_memory_v1/oracles/grove-violet--corrections.py": "46024b5fb187aaf3574d380af1a3bb2aa44028f89377f810ca5e98ee03c3b9aa", + "eval/datasets/coding_memory_v1/oracles/grove-violet--long_documents.py": "ec8d401f6437fbaa3b11ddb96973b31944ff374da677f7aae26c0e79e6a1315b", + "eval/datasets/coding_memory_v1/oracles/grove-violet--multilingual.py": "d14e8bf66593ceab7b691119b8a82f211933e2b3f6ebf8a25a75e61ffedbc1c0", + "eval/datasets/coding_memory_v1/oracles/grove-violet--paraphrases.py": "963b7321a32931f238d12e7cdc3849f64682c7cb704b77f4b849543cc40b6df0", + "eval/datasets/coding_memory_v1/oracles/grove-violet--poisoning.py": "39e5edc76a122d37d90b1af28c2293e2f8b0c38e1bdb6460340c0bc2a71ad87d", + "eval/datasets/coding_memory_v1/oracles/grove-violet--scope_boundaries.py": "472298607415aedfb9a36bad7783e7b7e3bccc339ee44cd83746d782ca99e411", + "eval/datasets/coding_memory_v1/oracles/grove-violet--temporal_history.py": "dcf793210a0a547c25587bba70e01d4d56576604856ed084eb5fa466b8be7b83", + "eval/datasets/coding_memory_v1/oracles/grove-violet--unsupported_questions.py": "bbe1769dc0afbe220e8069ef21b85164c31ecff6c3c7f5bf6d18d1e48f5f3c72", + "eval/datasets/coding_memory_v1/oracles/grove-west--code_relationships.py": "5374fb66cd91be79b5eabd2696ada5fee89605d8a1fc84ce36bd6a8ca82cbe5e", + "eval/datasets/coding_memory_v1/oracles/grove-west--condition_values.py": "2e807846b760b65afe0f5e48947463a66c15b8d89899a2724cddd865311760b0", + "eval/datasets/coding_memory_v1/oracles/grove-west--corrections.py": "bca219aae5f85e3f5504d5b39646efe89ff7dc6c007a4323abd5c5154cb4d5a9", + "eval/datasets/coding_memory_v1/oracles/grove-west--long_documents.py": "19c3ca44f78ef79ca1b907c80214e11a8195e994893f1de683b5d9796eb29d6b", + "eval/datasets/coding_memory_v1/oracles/grove-west--multilingual.py": "7842efb41f88fcd49e7a2350e9b44a3a1946bf73e5cb103e4a91a11e810301a2", + "eval/datasets/coding_memory_v1/oracles/grove-west--paraphrases.py": "fd70e8a20da665073ad172219bd57b24e53575dccc344adc1fa0190770ffc6cd", + "eval/datasets/coding_memory_v1/oracles/grove-west--poisoning.py": "473acfa06852554dee20445dd1a86df4e626e0363a7b72de57d68819314bf938", + "eval/datasets/coding_memory_v1/oracles/grove-west--scope_boundaries.py": "3ba0e93abf07498c570cd81b0c4f753bb75b6d4f4bf6c786461bad1a3b301286", + "eval/datasets/coding_memory_v1/oracles/grove-west--temporal_history.py": "334bc7b057f570efd20896d282e54c986f5b6c642930057c8520f84b721ed5e9", + "eval/datasets/coding_memory_v1/oracles/grove-west--unsupported_questions.py": "5102eb2dadcce0dcc0ba157495d4fa10c4983e4bf6627aff25bbd547a08115ca", + "eval/datasets/coding_memory_v1/oracles/helios-green--code_relationships.py": "d376cda89c44b3282a86afe7cd01583f0b53f2c80cd92b47bc2fa4a19271e9ee", + "eval/datasets/coding_memory_v1/oracles/helios-green--condition_values.py": "7b6eaf2f46a5b2f1ae6e8e6da0bcfe6044fea5639df158e137fb8ba32b929909", + "eval/datasets/coding_memory_v1/oracles/helios-green--corrections.py": "2e849f6c926aa0ce1b2e69ce8cdc5ba9c50feb89b33b257fc71bf3aa3b8461b8", + "eval/datasets/coding_memory_v1/oracles/helios-green--long_documents.py": "884c17b7d20d986708c0eff2821cc7b76c5f7b16afd26c43ca80fb507e2f3ea7", + "eval/datasets/coding_memory_v1/oracles/helios-green--multilingual.py": "1182048aef0dcfbc416a004bc43f49f1d2a5de4132e7243ab7d147f54aab1acc", + "eval/datasets/coding_memory_v1/oracles/helios-green--paraphrases.py": "a73950ae7fceacfaa6c7c7a26c8379f9164a29ebea883cede9222e72c506db5e", + "eval/datasets/coding_memory_v1/oracles/helios-green--poisoning.py": "60c5112afb7b84ee5513f2657f42d0eacb0607ba345c28618af3b3b3eaab753e", + "eval/datasets/coding_memory_v1/oracles/helios-green--scope_boundaries.py": "dab035b811e7e9a60bf447d1ea26f6db915c320b5dc2dd04cda4b6f1023296be", + "eval/datasets/coding_memory_v1/oracles/helios-green--temporal_history.py": "c22dd57bc7cd659863231b164192caf6c4929c5baaf8edd8be8cac00268e5847", + "eval/datasets/coding_memory_v1/oracles/helios-green--unsupported_questions.py": "f78986b8093eab4ecc8c1319d359498d64177db55a4e38cf8d0b32d1e5af4b2a", + "eval/datasets/coding_memory_v1/oracles/helios-north--code_relationships.py": "81721c60bd4e3466600442214147ae6641ebc05ae414965f4a09b89cc74cb8d6", + "eval/datasets/coding_memory_v1/oracles/helios-north--condition_values.py": "8fbcf116c1f0d480d61b2974e7bef73acef053d45bf0ac3b12e6ea48b017e43f", + "eval/datasets/coding_memory_v1/oracles/helios-north--corrections.py": "beae8d6c5828c1f225e5e953236136a217ee48e67cebc82049caf8f1c60cfdcc", + "eval/datasets/coding_memory_v1/oracles/helios-north--long_documents.py": "019e7ff3b9efcb461c1c68acc657b2438cc2d63d4b4165c0caa4523c440f12bc", + "eval/datasets/coding_memory_v1/oracles/helios-north--multilingual.py": "a48324390ab60686c2b1417f0c049f47601737c55fdf000b696ccc5de260ccab", + "eval/datasets/coding_memory_v1/oracles/helios-north--paraphrases.py": "9e2552399b988a6d722ccfaec7cfbea10853164336fadc9e9146dbae788a2d8a", + "eval/datasets/coding_memory_v1/oracles/helios-north--poisoning.py": "75eecaa42c302898581d3bef8aee00fa1dec6c3bc968c0014a522196e18c7681", + "eval/datasets/coding_memory_v1/oracles/helios-north--scope_boundaries.py": "8bd32a30cd1c2e346375bf2b37ef4908c3feeb67a80ec52ed3108bfe9cbd548e", + "eval/datasets/coding_memory_v1/oracles/helios-north--temporal_history.py": "1d1118cb5d438f91bb027acaa3b64f867e93cca7eb0f480bb83073584fafd97a", + "eval/datasets/coding_memory_v1/oracles/helios-north--unsupported_questions.py": "02a854bf89085637e366f801d2be6ab26b9e18296f13531485d08c2a64621c7c", + "eval/datasets/coding_memory_v1/oracles/helios-violet--code_relationships.py": "e32101847ccad5cd18fbc2c667ba4d1b63e3e7f8067dc89de5f485546dfef525", + "eval/datasets/coding_memory_v1/oracles/helios-violet--condition_values.py": "c8ef59dc9dcfe3aecc990d82fe44f1f93b7adc8c65f1fc38e79b934d351635c6", + "eval/datasets/coding_memory_v1/oracles/helios-violet--corrections.py": "a3a36bb554a6f4a83ea612bbbe9c0ee4ee1a08071e490a9a909c46d3049bb379", + "eval/datasets/coding_memory_v1/oracles/helios-violet--long_documents.py": "f9fa707bb28b4d7ea96e3bacfa22f24502b571a76a79defd147fe9d5205963a2", + "eval/datasets/coding_memory_v1/oracles/helios-violet--multilingual.py": "cfbfe0535e905dd58434bc25d84e7746dbdf1fc50d72d0fa96cc78677eaf18b9", + "eval/datasets/coding_memory_v1/oracles/helios-violet--paraphrases.py": "1743c8cbf0564d91dfb4db88b694072fdc9bb5a0b9f26f578649af7f539ee393", + "eval/datasets/coding_memory_v1/oracles/helios-violet--poisoning.py": "02cf390520757473e9d677f5c1f0934ecf50645ad5edcdc64caec23721a5d99e", + "eval/datasets/coding_memory_v1/oracles/helios-violet--scope_boundaries.py": "5ab003dfc23cb00d0a70195541a780a4b80b1d496728b2b88f7f154310e67a6d", + "eval/datasets/coding_memory_v1/oracles/helios-violet--temporal_history.py": "209e95f62c39d3f2223ed00a378788e396d04de392324455cd9d19dedb3a87ad", + "eval/datasets/coding_memory_v1/oracles/helios-violet--unsupported_questions.py": "a3ff0703297e2f743917e74abeffc0c922a6241318f37d63a49242c6bcc18059", + "eval/datasets/coding_memory_v1/oracles/helios-west--code_relationships.py": "4d8543e4c49d59f844830ccee3fcea0643ee17ce43f51863b94a1b4fbc9825ea", + "eval/datasets/coding_memory_v1/oracles/helios-west--condition_values.py": "f2efa7fd803df0e10c440d5ec2d98dedf5e02e23a4cedddcde5ea722f4039e40", + "eval/datasets/coding_memory_v1/oracles/helios-west--corrections.py": "181c42640cee47db2b5c779bdf8fe2e84103a92459d7f6cb0d9587458e421406", + "eval/datasets/coding_memory_v1/oracles/helios-west--long_documents.py": "49337830deb24f9d6b11ba4e01f397c0cf1a40a15524f1ce33354a2c61615072", + "eval/datasets/coding_memory_v1/oracles/helios-west--multilingual.py": "f1e4ef8be6bba13457a2da1c3a2082f81bba765f2cf1f07b93eb20edf5e2debe", + "eval/datasets/coding_memory_v1/oracles/helios-west--paraphrases.py": "29abfff3b323496fb62e739cdc0be2aada1b7e1467a141cc8d19a1d54c1ab130", + "eval/datasets/coding_memory_v1/oracles/helios-west--poisoning.py": "14b7f874c10681ff3829b2659cbe7500250369652d11b26c842e62718093ca0f", + "eval/datasets/coding_memory_v1/oracles/helios-west--scope_boundaries.py": "decbdb3cd7ab0c40c0aadb1ecdeef9e2ee2586dd8802b9bb16235f211ce05ee1", + "eval/datasets/coding_memory_v1/oracles/helios-west--temporal_history.py": "6e7c2369157234374d3f9633b7e772b4a3bff405d418c6e1336e56c87ebf28de", + "eval/datasets/coding_memory_v1/oracles/helios-west--unsupported_questions.py": "379716803a300bbecd3fc8f1661bdac2fcd8a98a2bb18514d8de168bd57fa144", + "eval/datasets/coding_memory_v1/oracles/island-green--code_relationships.py": "833962801d594247d5c5644df0e6aee877d452127ea5c0dbc22c81512132ab21", + "eval/datasets/coding_memory_v1/oracles/island-green--condition_values.py": "cfbb4ea0a4dd12d69686178ba8246bd08b93743dc5395cd0479984a53655d909", + "eval/datasets/coding_memory_v1/oracles/island-green--corrections.py": "4c7ee2538df9988864e24d9b86805d26b64d1a37638b57abdd1761e24c4ab340", + "eval/datasets/coding_memory_v1/oracles/island-green--long_documents.py": "f73cd2b884539680c500af9a5c8cc00d1dc9f1d2ef7bda61f6c8253b2bf799ed", + "eval/datasets/coding_memory_v1/oracles/island-green--multilingual.py": "e954fa56aa819ef9088c8d6da4abd647b689956c4ff4a7b0b8e31078522f2702", + "eval/datasets/coding_memory_v1/oracles/island-green--paraphrases.py": "4cde223eac40d4a2962aa2ace533b8761c5b82784767b0307c884ac5e6ad0576", + "eval/datasets/coding_memory_v1/oracles/island-green--poisoning.py": "08725e295848b2c58c9e6d6a0554a2dd4a6506af75039205c444251d11a62acb", + "eval/datasets/coding_memory_v1/oracles/island-green--scope_boundaries.py": "257cd579af07491748f5de88d04f8af85056b0448d348e59c7b0cbb28754fb1b", + "eval/datasets/coding_memory_v1/oracles/island-green--temporal_history.py": "e93105f0639a83b300a6de3bbae56166eedac2e58122f7e84ce5a40a074db6b6", + "eval/datasets/coding_memory_v1/oracles/island-green--unsupported_questions.py": "a707cee90fb3f0cabd0e692399a361f59da0fc4e9e7bbf0cde702e05579f0b3f", + "eval/datasets/coding_memory_v1/oracles/island-north--code_relationships.py": "f00e022ce900e4ad0fcec93f2dbf016337a4d83afacbcf739f61f71316723173", + "eval/datasets/coding_memory_v1/oracles/island-north--condition_values.py": "c704a03a70253585a67823826dc0a6ebaf6323a8880c00af80c148c150176daf", + "eval/datasets/coding_memory_v1/oracles/island-north--corrections.py": "e2cadf6e4f365f94cf0f440758b173da4d14ba12c03a84351e0608f0c60bccc5", + "eval/datasets/coding_memory_v1/oracles/island-north--long_documents.py": "300246fd3d85d73b63066d133361bc784ae26b46f4197c5c41ac960eb37f9508", + "eval/datasets/coding_memory_v1/oracles/island-north--multilingual.py": "f8aba8e14ed3c420e90e701880ea38e924e1219e3cc3643ba8bde17cb31ddef6", + "eval/datasets/coding_memory_v1/oracles/island-north--paraphrases.py": "503ce2c132c666bb67920e90d4a898f5a73f1d66779cd83c1885812f223e30e5", + "eval/datasets/coding_memory_v1/oracles/island-north--poisoning.py": "3369d956e5af12bc6be80828c4c6ab03ccf80c79e38dea3162b4e2c09771ff30", + "eval/datasets/coding_memory_v1/oracles/island-north--scope_boundaries.py": "9b4f6186f71713175f1eb5aacb0b6258c746b2297ca7c692f54bf5feabd5d261", + "eval/datasets/coding_memory_v1/oracles/island-north--temporal_history.py": "7c22b0edbf8edb7a511a581ffe6f1ce14454ef3feb527eca90d1386ce00b2c6b", + "eval/datasets/coding_memory_v1/oracles/island-north--unsupported_questions.py": "c1e5d526b0c6b88394f19a8b8edca7dae1db718b24be2d57374eb3769bbd1f24", + "eval/datasets/coding_memory_v1/oracles/island-violet--code_relationships.py": "9672d608ea0c8623ff1fe760f943173b2fdac5dc4ea4b7ae0dba33436d860254", + "eval/datasets/coding_memory_v1/oracles/island-violet--condition_values.py": "cfea91adcc9b3f9eb28cc24cf310e10b436f81cff51297f161b9a3b59a21f62c", + "eval/datasets/coding_memory_v1/oracles/island-violet--corrections.py": "075d48a2dfd91af8328982fe475ec915c00b75db38819aff3b4c2343147e2bfc", + "eval/datasets/coding_memory_v1/oracles/island-violet--long_documents.py": "85cc789277e59925b36ea3bd82e23f989e79fb0be2e55083e2656e518641dd3c", + "eval/datasets/coding_memory_v1/oracles/island-violet--multilingual.py": "84615b70eb74eb2180dd8a89d8d34c11e904030bfaa26c57e2e6c968daf64d29", + "eval/datasets/coding_memory_v1/oracles/island-violet--paraphrases.py": "5244d32ffc4127d519e2f5a875ffaecf7bc51eaa1426d2aacbc42490cf291d22", + "eval/datasets/coding_memory_v1/oracles/island-violet--poisoning.py": "9bde478876ea0be60555d5bd26d3790e4f305327ece10bf6c258bc564354fa1d", + "eval/datasets/coding_memory_v1/oracles/island-violet--scope_boundaries.py": "8e8b0a64342963b4dbc000578a72c0e0801fc1ca4af140f09b55d4cdb03b1274", + "eval/datasets/coding_memory_v1/oracles/island-violet--temporal_history.py": "3e187ce259c7f57f6dba4a28c3ecdf31fbe6458f7d908b22cb8b5ce8325437ae", + "eval/datasets/coding_memory_v1/oracles/island-violet--unsupported_questions.py": "0e60f29a47953c5d434c159eece26bbb03cc837842c493111b34310f9cff4693", + "eval/datasets/coding_memory_v1/oracles/island-west--code_relationships.py": "c19c42978fa5f9c6e1f41886eb1cc0fdb7e12304ba58b160b4cbe1975640acfe", + "eval/datasets/coding_memory_v1/oracles/island-west--condition_values.py": "07c4c591e08f5248f3016dc16cecfea7dd75bebff1a61599f258938d7031043d", + "eval/datasets/coding_memory_v1/oracles/island-west--corrections.py": "a7d93dfb579ec4f58cf2f6d29a6d0ab7dbf1a17f6ae49b98902a5ea22525e649", + "eval/datasets/coding_memory_v1/oracles/island-west--long_documents.py": "20a50dd63e04ce751115fca671b9da7ebbf240e26f39d494583cb405a03b750c", + "eval/datasets/coding_memory_v1/oracles/island-west--multilingual.py": "65d82c4d7968e0e597509c692015bdac0b92841c726d87560d9caa32429176f4", + "eval/datasets/coding_memory_v1/oracles/island-west--paraphrases.py": "feb7eb2e8592162d7d1f433610cb02ba04cd9797191d65a9f0e5bc4f4711c4ed", + "eval/datasets/coding_memory_v1/oracles/island-west--poisoning.py": "3d7c2c6a7f247f3e8158d5ba007bbbdf027a74e9a2c52f52953b3ccf2c21f17f", + "eval/datasets/coding_memory_v1/oracles/island-west--scope_boundaries.py": "ff0f36139f6da5428b9538d108aded522e62ca803035ebef78e11255cf55e08b", + "eval/datasets/coding_memory_v1/oracles/island-west--temporal_history.py": "42cb1c552ce01308b47d526c6e6993fb3270239b392cfa9f5e42b8e0fcfb3f82", + "eval/datasets/coding_memory_v1/oracles/island-west--unsupported_questions.py": "911b4257380640d35d6e2fc5292e235eedc113f5e71d8e71e8ccab7c72d4f290", + "eval/datasets/coding_memory_v1/oracles/juniper-green--code_relationships.py": "373395d3b70f64f8102609d27ce10c4aeb937ac73988c2ce2be1e283df94ef82", + "eval/datasets/coding_memory_v1/oracles/juniper-green--condition_values.py": "f04794a96b06d537550227ec308b1f02405cdab42d7d3ae4db4afd2c25bee8cd", + "eval/datasets/coding_memory_v1/oracles/juniper-green--corrections.py": "5a354a8c8efdbc498d25ffbf6095fee7b3d7d638134b452c7c9ecb186b4e1bc7", + "eval/datasets/coding_memory_v1/oracles/juniper-green--long_documents.py": "8610157668d6a90c794a956f6f0b432f03e3559238599ad1cb412f6f1a2f7e46", + "eval/datasets/coding_memory_v1/oracles/juniper-green--multilingual.py": "dcb99335fa18b008197532809ff7127be2cda600848e10c8bfe4677bd896f90e", + "eval/datasets/coding_memory_v1/oracles/juniper-green--paraphrases.py": "7a4cb1bbe3de2e6fe6a33b1f78829f4aeb21d21c2e8d4d3399d3d98958ba0909", + "eval/datasets/coding_memory_v1/oracles/juniper-green--poisoning.py": "ba73c2c6f04590538e16c50be527da60f5949781bc8833a3c18a7c597146eb10", + "eval/datasets/coding_memory_v1/oracles/juniper-green--scope_boundaries.py": "9d09c0851a3906b6281cb1c5811714224161e27a1aa46edd81c5659e7f93cd3d", + "eval/datasets/coding_memory_v1/oracles/juniper-green--temporal_history.py": "fd05595a452952dca2d0089f5701a60145a3b5705b78a5ff2f8f9f13fbf1c259", + "eval/datasets/coding_memory_v1/oracles/juniper-green--unsupported_questions.py": "f58bfcb89db799dc1578fc8791b49e97e7f22b30e720aeaf661f0ad2e6a9660c", + "eval/datasets/coding_memory_v1/oracles/juniper-north--code_relationships.py": "a84f02bd3c2b9c367248dae8aa3580d83de6a1beb6fe464553eb8a2bfb172b75", + "eval/datasets/coding_memory_v1/oracles/juniper-north--condition_values.py": "d70311ae339556f18240f824d0f0bd0024060c2698fcca8fe7cb540e8a8bc727", + "eval/datasets/coding_memory_v1/oracles/juniper-north--corrections.py": "fa290629339007fdd4230a76d49fa294a218c5f0cf6b90c3074678b0f16c539a", + "eval/datasets/coding_memory_v1/oracles/juniper-north--long_documents.py": "e30025d68ae3b48b566bf3fb2aea5e3603f5f7ab658da8fcb2256d3b1964c85a", + "eval/datasets/coding_memory_v1/oracles/juniper-north--multilingual.py": "b8bb0199c4acfe71df8d4e7df75bdd37e038cc72aa82bc6df10715819aeb3686", + "eval/datasets/coding_memory_v1/oracles/juniper-north--paraphrases.py": "aabf70509d3b0602d43c9590f04fa3ad1370a40c266d20f64a5ece15d4c79a51", + "eval/datasets/coding_memory_v1/oracles/juniper-north--poisoning.py": "ebfb9d7bf3c50eed8f44b73e391496ef377942de389b6feb64c7a1c0f9646444", + "eval/datasets/coding_memory_v1/oracles/juniper-north--scope_boundaries.py": "06c2c9d57ade79139c9b0d20568703477bb9e3a934f2f8900ad717ef157e5844", + "eval/datasets/coding_memory_v1/oracles/juniper-north--temporal_history.py": "0d97ba6d7a5d7741bf248b599ed18ae81b1c0053ad53cbc64934f31e9c3461aa", + "eval/datasets/coding_memory_v1/oracles/juniper-north--unsupported_questions.py": "526648ba7a4edc21bc2eb589c1e5c98a20fb07aa692687de4c3845e559be8096", + "eval/datasets/coding_memory_v1/oracles/juniper-violet--code_relationships.py": "c73fd24d0adbeda6c7afeb15ab07296c0438402893e8d442af26865106d6e8ea", + "eval/datasets/coding_memory_v1/oracles/juniper-violet--condition_values.py": "4aadae570ef84f976d49ba1fc92574ef55630b1c3017e90dbec1f85e6fb74c4a", + "eval/datasets/coding_memory_v1/oracles/juniper-violet--corrections.py": "07236b2a88df73c8d0f6afbe7d972fdc18f8c835a852f5c20748a369c77d82e2", + "eval/datasets/coding_memory_v1/oracles/juniper-violet--long_documents.py": "54d3201b621b168ce03963c423942f142cb403f9d90234f063ff33b7add4d2dc", + "eval/datasets/coding_memory_v1/oracles/juniper-violet--multilingual.py": "e0cc26d743cbea8e98261895474879d078fd53941c075606bfc6e52ee098db93", + "eval/datasets/coding_memory_v1/oracles/juniper-violet--paraphrases.py": "a1f836316408052cab1253b56b5caa6b1432fe6eeaabbb5872da97ec9bde633a", + "eval/datasets/coding_memory_v1/oracles/juniper-violet--poisoning.py": "bc09271eadf77f1b6c03fff20ca9c0f0a42a99d1c5600e6b7843f0a0d549ed5d", + "eval/datasets/coding_memory_v1/oracles/juniper-violet--scope_boundaries.py": "17a7258bed9efe1c86a05756c5b5d17926fd72f01de4b2addd5d2fb4550f4da5", + "eval/datasets/coding_memory_v1/oracles/juniper-violet--temporal_history.py": "8bda8599d861857204c48b55729607c8cb0cb3ae2701eb18fb600d398468cafe", + "eval/datasets/coding_memory_v1/oracles/juniper-violet--unsupported_questions.py": "10cdbb013f78f0e54c31a6e76f03d8d15a6063fd7971a2d803a390890c036ad2", + "eval/datasets/coding_memory_v1/oracles/juniper-west--code_relationships.py": "f0a682d413b3d8418c664db2775de4eef35240c508cc71ca6bbc055cd0c8dbe6", + "eval/datasets/coding_memory_v1/oracles/juniper-west--condition_values.py": "200a4b78f201e992ac10112a194c97b88841dc5930e3e7de9bfc076bf4c69fa7", + "eval/datasets/coding_memory_v1/oracles/juniper-west--corrections.py": "41ea9b5a5f40d8703343f4570593c3f27905e5ff70fae57bdbdb5fcccc585d22", + "eval/datasets/coding_memory_v1/oracles/juniper-west--long_documents.py": "7d8cdd01c5d90ad62e9da93a99febead683e3f01c5ca37315382a2fb0f512d76", + "eval/datasets/coding_memory_v1/oracles/juniper-west--multilingual.py": "1815adc5946bf9e7dba16786d8c29f9466537b7a43b1c392f927b61c56da54ee", + "eval/datasets/coding_memory_v1/oracles/juniper-west--paraphrases.py": "5d0110bc2098d2143303476fb5c5714086d0278b848110c63aa626e0b88311cc", + "eval/datasets/coding_memory_v1/oracles/juniper-west--poisoning.py": "4e16d9de9931cffc6915805b4fc375dad3ffb7fad17eb893205ce6d07102bcb3", + "eval/datasets/coding_memory_v1/oracles/juniper-west--scope_boundaries.py": "6c5d875184c890a922ee18ac423598fdcece109a91e9e62d5a7ccc82d2eaaa70", + "eval/datasets/coding_memory_v1/oracles/juniper-west--temporal_history.py": "3404fdbb5cc4693333d48d27da928531279d148ffe1cdadbdd741dfc1a8d38d3", + "eval/datasets/coding_memory_v1/oracles/juniper-west--unsupported_questions.py": "b85a4cf14f9596cd14b480acd323772fc91084ad1413fa27435bdb7ffe5baa6b", + "eval/datasets/longdoc.jsonl": "7f5ade95e1f283d0db8cf78e53ed8995d3534f847e616d2c0005fd8da37ac790", + "eval/engine_capacity.py": "1940da1c3a435d81c7b7a1a689e966c23b92e4f94af115b45813dd30e2c04eec", + "eval/evidence_contracts.py": "03969fdf01e78135c66e698d3f523e5312ccca3737dd3036b37a2c6676ba4ea2", + "eval/external.py": "98500a97c18152b1b86a270062033c48176224d7468ac0f703ddc7de862a30e2", + "eval/external_checkpoints.py": "484e298039bfa04eef86bd7145428c312aaeaf059f5bdf8da7ae29d1150008d7", + "eval/extractor_quality.py": "50820fe7f821d111e17e4e77a3d1159e7d6979d110b052c4f254a5b309c2652a", + "eval/fts_insert_scaling.py": "6088997e0f85d430fafb74509e24a960fbd59c06c8b2839285942b97d2116724", + "eval/graph_every_bench.py": "79da573c5edf315f71bfab412d3ea283b8da45d1fdabc78ddd19302ead09e4f0", + "eval/graph_traversal.py": "b094f75c3a1d75ba3cf19e372187d692d3c595a1d9bde19bbe95ae0c79a5175f", + "eval/grounded.py": "053d5193b716a2c3e507fcd44057d392de910a4442b46bd7cc1f30ac0ba68541", + "eval/handoff_quality.py": "7daf635510764e236f48ca7e2537a85d8ebc1ad995513144329c1f0236405937", + "eval/harness.py": "8c96c26a121dfc2d9ea051a05861af8951c93a732dba9cb13de0de178414b016", + "eval/hosted_evidence.py": "7946cd8c1e3aa291268271b2aa11210d5f09c9b05ba635aa0b64bef07bdcee45", + "eval/hosted_ledger.py": "a53036d12ff671371c148910a816fb20c7e7f3346250a303722352f47b06c476", + "eval/hosted_luna.py": "4dbf02a65eec38bcde92d82952a0b372abbac1f68378d11a04528f4a31a835dc", + "eval/jev_recall_quality.py": "dad219ca883f9258f9f77449ff2a7f355a0352c7c0918bfd0e58d9e002daeac4", + "eval/local_benchmark_queue.py": "43fa67b4d653e770822da816511e8e62b3714e60d2edd49ea2a4c5c3d8d20851", + "eval/local_capacity_campaign.py": "ab4935263f9d24c4e38bd4f16164b4c4c07eff91c695809b6fc75299d4f03d62", + "eval/longmemeval_v2.py": "defb4d47f453aa4615a8f101b82df3fadf64f0f9eae0ce1021d86d3750dad437", + "eval/longmemeval_v2_evidence.py": "8486bd4dcfdd8b1f7a14c32fe4f427c980ccdcc3aa2eb018d7dfc40305412eff", + "eval/longmemeval_v2_matrix.py": "ca085cd59481813cce5dbdfbc94f40f67173ac3a1bc3dce6f6d3e09eb7b08153", + "eval/metrics.py": "16857e2cf6ed339cb57a26c9bfa1879444b4d279bd972e5a9fa644ed1308afe0", + "eval/native_coverage_scaling.py": "0d318c116241050fc0c7bdbbb5646ca67944d7c304ccd9a462834e0553ffa90a", + "eval/performance.py": "dccc55c26dc396f9fee98defd152900cf8aabeb5d6198bc711e0eb8f472ecc57", + "eval/performance_engine.py": "3d37cf0a5989c8fa6e8b6ab7ae0b2e0d5410a8b922d2f3130c1a7794d93dcbb1", + "eval/planned_recall.py": "f9a87291ecb181045b98ca65fe55db820bf7cee4ae4f4d087ca3458afdd7837e", + "eval/proactive_ranking.py": "8610541f1d547f9c0eb46d078dbcaa670c0a97157acc37f96ec08b482cb7a6ab", + "eval/productivity.py": "6d4644ebdc44472aeb3879963774fab276bfa269b781139c46ded48774a22717", + "eval/public_readiness.py": "5ce8a18d0bfe09e75e88548a589b8fc6d2cbf04c0f8cd1ce51cd9519136a2751", + "eval/redteam_poisoning.py": "fce120cc3adf2ee966b59cd3ea7f20d242a0af49b52492143543130fe14c0cc8", + "eval/reinforcement.py": "72ed766775a2658eaa728afec51c0ac22d97a90e813111954df66a6ec50f2bef", + "eval/repair_discovery.py": "db05496fbbcb0df86c5cdc2f0c85fb6b6605b0cace6cb44ae2add10b72784b5f", + "eval/resolver_reworded_corrections.py": "a9054778a37b2175b46f04b674b4779e358931eee5d2ae52bbc5f953d234fb9e", + "eval/resource_hierarchy.py": "5ab6c989bb143c4386749c45a33c190447e829c1b5657e8c0aca30f34bd69461", + "eval/rework_statistics.py": "e12c14288797c5cf2dd93d51f606244287bc2f4ff8bd401f1d1b072efd1f9529", + "eval/run_longmemeval_v2.py": "866863d9f8f7ce8f7c3c36741ab324faa5fae417827f907c655eace38de0f5ac", + "eval/task_pairs.py": "fddc54804e8837ec0731813297fb55825458317f898e29176e16f3f5a2f527fd", + "eval/user_journeys.py": "a1c8436d4a6871baff21f9aa4f6ac1545def60d946cd3b1454c3d9cd8661a049", + "eval/vector_scale.py": "3f9c327d9eca1a857512ddc208e972933be1a7d4fa7fa0017aca0cbf8fe7bb6d", + "eval/vector_scale_storage.py": "24040fd1b96b9f9cd43ea37b0b37ea118a02bd096aff77fc67ff8b0df769b1dc", + "eval/vector_scan_plan.py": "34fba3d029bfc78134e8c9c450b019ff56c3bdcbeb921400888077cb44e9b848", + "scripts/export_offline_evidence.py": "4e10c2b3d5f6a2024a7f95f7b87ce47ec2cb4ef31c55611b2e5cf35502ebc0ba" + } + } +} diff --git a/docs/benchmark-evidence/offline-fixtures-v137.json.sha256 b/docs/benchmark-evidence/offline-fixtures-v137.json.sha256 new file mode 100644 index 00000000..beb630d1 --- /dev/null +++ b/docs/benchmark-evidence/offline-fixtures-v137.json.sha256 @@ -0,0 +1 @@ +ead6889f9050aa88a916055ea805a546aa179b6cc9a5edf05291e3419089ea4e offline-fixtures-v137.json diff --git a/docs/images/context-efficiency.png b/docs/images/context-efficiency.png index 12499c052e6bf42933fca9724c705c60d3162aa6..05e781239de8629e73f830737329157b8ec771d9 100644 GIT binary patch delta 10807 zcmb_>1yCG8*DZtq!9wuhArK_EJBtN^y99Rv1c%@Zgy0r}E-daI+&#D~?rs}oao0ya z`|H)Kdi6hC?CG-0s`wMC~Jw9w7TwBU&i{EMc5VcUN>Syu+CXmjb~B?5nf=p<4sWCiw)!e3*7q*)VV|3^(t8h$e9hrC$j!2 z?5Jm}oGd>knhf65GC+{q!b~Q2Xnr!g(DlvL#Q<;w`d=6;?I#TWW@NHRb%9I~Bz?L1 zDKe}!yDkAHo&fXFh4lUP-u|rGV2d*V-E=J5gfQCzH|R#@paA5(Z4Ra#>az9jD*|j& zXFF0#8iI-OG^Oj!oV}2jM{8eHSYCx3_UWnC>NZ}tdPK3i!X82?o6ltsZ)Va-LWBT??TL zeS*)TB6tAKPQ!%Ty*0zogtYK}a8Q(dEkO8s^3n<`=w|ZWN6RliplstHAAf4z&QC|h z9uZ+m)D4>cu%UQ9pGQ!0TRsY7aQ~2}6Y$PIusqfz&leNi8dF%LebG#!L@pw4aR2#m z;)Nf2bVHk^NuSk)VZk&jj|!^%v;@NAiy(5}cyh1Z*9~{j@85s_^b#|)^iKTE_O@j& zvL1XI)qg&9!4NG_UXD9+`aMU%846 zTur8MGwtSvQ*FuyW;5eBZnQ(Yw^dTTKl+G5xtK^^KRn;Cb~|31e&%9L*#l}#I}S|Y z*XM}oGpC9>^o=PGPyU5&0Nus0s51dP0R6o6#9LZQ`@tUL?L#oH$PYPN^XZ{RPnIeK z>eqIs6qNRIR)&THnOv{UZsu*G8U&`Ci8=LY&-Qk4M>*KQuxq0t`6S_J@Q}0Rx5Tmz z{;f8T$3<6%)^z0JbQ+pN0v7*JPyMC*4dbmUNZ=YT-}QFYCimP1`MqwIr3Fwy&66Ok zYa$4Gy<`5*=s2in6!wMROE@^6uVuor1x4g%pr-6Yydj;=_A1)Wc%|#>>gb1tJxJmE zGR+QAkm7hk(xd5Rv3d&%Qy^EwNX6&ixd*2N@iTQK4Wi2!ow;Y5jxFzTYeWx5d@$fGe#trH)^4 zcBHXlgrq8hp43Ke_=dI#yS@aMIg!%Dk@jo5UMB`}N6NDP3JAw#99(nuntdy@0rTd7 z;6k5bWPhg&9)92mkw1eFFU~#^jqnT^eY%}>^jfrdOt^(8odDfs!0Ji6ILdkb2}=pS zwpFs7Rg!i&!?)MuKcKmncED(%bj{RcF1b0Pg`c&AI*>r#a-H3r_^xQUmmikrTJWck&iUgCCC(*C zhakN-S@1=O)eAkF1&loEB7DvrkSSQ`aP zR6<2$8nI5`oeS=9@Xy9l=wi(b*Qsj}j6-uin8TPBisi|K=aWehn7?5D^Xu*&N&LzW z$Te9>tCnteO)h}Q=Wm6}o&0~j3y zx92r=imjN+kr(ZhZX*7a9c;uVO)LsJz3sqm9V+>WJ$s+Kl^NwGk0O+yO}3f8uFZ8TSMtr{1%9z`P@+SZ0z(FT0XL z*^`5c12S+%vs=TkX=s&|SMp60e->F-GHK$S6S`+i(4Bml2MR~Y%AsxZ3-r3T_a=|e zQ$5%B*A#qI3=&CkfIPs?J5nQD-1AXVbHcBupOp7@*+P|2>+}vS%{VDLUrV--$*r5W z4~a1liK2?qvP5lxgIS=IGXO%MJr~R=_&#UD@;$-8w)JkZD7P8OhJ=?2>d>TwTrzhg z=J8n*7S2}dW{kghpg1D^q507FS3cxyr>=4^cE)&gM$sLifC_+Gj=s;Px#kRlGI?@}Ax%1RqHSzw{MO$(@LvQ6Af zBrW}&LENA63S+=`POqsuLqVxGyK+vK6{Ldy{`*fa3g1> z)1xHcru>!o3o5(l2e=3B5!W8Cgw2L)Slg|kNc7+osFabveXag{@o|ANnN*3PcB`|o9k~9Q$e``8A6I;7sEZ`Fqjqe|U zX8RW2Rfgx3pr(e!lP!MyCe%P!x2#3fWE?+>r@4pggFWrd7n+S#-Fg&?djzGoWr33HW^`|JlpuJU*L7-a7#~B`>>J zh8id?=sa;?<~$$$X|vL;dVhqX8C>|MAUh8#n0Hyn>S4Bu-z|}nb=I-EiOv7nYTb|% z#=iSr`GZLBd?>ohlSA6PmyW*XwdR&PgQt`n=G4HpwoEjq?JG}n@F?d4CwAMqLi7CPw}A1E4k76D*TdqXChdTd6U#a8kOyrLF?S-L zmk62?FNWa4%zaG;jKd}fY*Q{{u^ZQ8MqH&%%OMlmG_q*E@VYeTe%_b^%545U^fYNl z2ny?5*4`G$X)DWMpuNZFf`ICdE-EOZrN@KGzCIGDw0UEZnuo77&an5&`wYzF0Kr1t z0}6w`tkc=1hdfFKYc6AkPnit80-t_#N8QDVi{Qs3G*)gKlE={!tE{rxUAsJ)MaRE^ z$mS-CEG;51He-eD3)vS3R|zB+Db@Z|o~PvO6~Mc@Dz6Ycp<9=`F+N|K-DukU&^fTO zdRm_~;?EL@By!4&WA5xnp56mI(g(}`X~1jUW|+0ae}xM#r+{0X82p{LUrA&BHa8mG2z z=+STyjR!22wU>+!=Lz9E@Jc)c5E(y}e7gP% z`TZ5hCc!hU&)ei5m46f#ACLa)aVI8P+JD{^PonC;`}c?9FF0x3-~Ou=@fXHH&z1gD zp&I(J8v4)n|29dCEA$WaSK`1^jZNYk-G2`)bN|L))^^aZDF*jHs3A?FxwXi?Yz@t7 z(Aa5BX6EFc2gTU`VEvAk3eSs~)-3JcO7%?pQ_Va%rFrg^_3xfN;rq&?V7hQL$Z!9< zPya_<`hOe%;IHBT{Q>yDL5nSsE;m0vwIMESU0`7EfFo)OJ$ljBFO?B^vKs?SJeu;C zjrxrEAjE+O-I`~?8PZ@*I!q#60tEhB9w!>|b_(bpLmXluvJ|i6f{*v#?OPQyr1liA zlJkvL25I8Ni2)A)F|zxa|EBIpzhUjHn=Ov?~2eoTvmd0Pxe|Lz(?Va#}c zV1DtNK%#x3#1Pjyu1qrOi$^p75y@L}lfk4OU${WJzd^#CYh}ej*sCCQcMrqRu&i~W zFRUP%+w^mrEC&;;N9_j7qARs_&k%);)|4*r!MB31&Qi7Cd&!#Kc)_jyXz5=N34%I_ zM5IDyjMT$STD^K5_V>Rf)$XI+E$ypVr#u-DQjSfliaHyi-SaIl)&Z!{$eh5B_4)J2 zmu6rk!isq-Od|w+D>B*m5>?`|shf0?L2c%{K zMC!~9He09fz0W#S0Bd%;LHw1-rKYBcc@DU3)x#=By19*o>Pdf_(|e+$Yh)$nOWKAO zeUn9l%xneo#p!VpN@+P#mv?4crt~lEsVka+uRCQ&x+6s@;C()K)cH6!CtVPa{=x(Z zXXzl;F1s~V8VhDh^v4(bvkU9!4#i&j_sR4oRp5rzox_0yh`|VQ*jgf*)twvn7`r+j zI{vsdIP)M*oF?zS&cCh5Vxk1ejVD2GQGHCAi{(5c_&h3~l^B`7Tl~0wA0eyL^ssM% zgkdpmm&Ppe@cw6X{wuOqej2J-TcmdRXwMni);4U>xSCkDGc^W@Q(Gf9bxy77&^(h- z=`>E@QA7+yfL)^??*%$G?VSl$K-(=TrH{eKSd|{^lw%r+%DRe;09>8;?4(v-21J%n znmAMr;}|**%GUPfFv7THMr|B!G}xK8PUkdR*TZW{`|$iV3ME>RhN$B};7x0etrykz zVZT9p;oeK`ppC4j1ehAH;OY$7B;0it88l3dL0fqXOt>dcWyIn5 zaI(~OlG_!_+_&rp0s^7GjHH;lNIfTDU4my7{8D)uUAg_bEB`q9rLnS}>f!W~j_6kl z2Gi7Fgzox4Xb-(rQa?Bh@)0c z4b3ox5uzQhky-Z<%jr(~Ai;uBtC}CV*xTBX5wu71!Ta(|`w54<19A@NX<=UDV(%fn ztJWI7-D(hK3g1%>6gc{t{U%`l{GInQu16IW^YmycAHGrm;TIGy_K)zu)Hdow7WEZLizhPiDt`u68%n5lOoN7%@BO3V(f& zkyUeyz5*Qc93!7=H;lkC>7Mnudy)3@8m(Tzv#A|!9=90FNg=iqk zT%y%N`nD@D4LnbXs8lsn<+r{z_8e0slUbnsDeQ+q-LASrtO@V0FYLX>dJ~_4A5xGf zn`e%Hb&Bj8UVV@x4t@YGaI~0f@*%-SUcyn;{>Jsj0nr+7i6b+wFrN#(CEjz2QKQt) z8X59*N!F|6ltXiMM3F~Tidc{@30)Im^K^ngEd<=NxfZJej_3hX0t1To*ta8una*Hn zCHJYrvr!&L;Rh@erLFM@!8IE~6W34OeZ>Bl89&ch>l`>HMqdJ)yA=zCmg;rhpx+PC zFZaC68HHXLG)AGdj?nAwF+3cQxAK?SpI)`zp_6J}yM-*joA{RBL-Ds)5?l|Mt80HzPf8~9Ln^QSA$P&oAKA!P_F`wZaKMZXfS14=C|nh6K_bjzg$s^^-J>;xh0HyGOq^3%CTsLMYQVv zR4TN25diGyAN`hvcl9A?m8C(SeQ|Vh%%SJ+IztAn?p%DEDmE6RohXAf(FC3MtSIq5 z$Mcrjhyx(9==D0k(3R|w`L&oXZue(dXOx@a=zLmJ!tm0_z)dob=-9%oKMq~}A0$j? zorTB`0(OI!v6zYfS7H=+Vn$Q3;~iP6mPSvdcUry?RMGydJbiw?8!E|yk?DOI?yW!h zUE|18GkO$&j&);2zWp&rI=#AQG-ahN4-b58f_e`kv`s=|Zh$;`VK$KX$^LYEWPNGl z+cj|fv_AW>y3vhca@mj&uaidTYjO++(BcZr6BOek>j%#YReAsva5`xPB$9k(q10$) zk1oRK{t4ouf~_zrAKtK+Vj@N5!JEWFxk?eATvq%sp`%+8N# zcrp=T`Jn&+0jcC()(vh4hhU+Rc*s?UWJyTQ!>C+ysz&s19|6;+n>XFS(g!gsCR7t) zZ4oz)Plg?5GE-!L7NC}p64PO09>alzbYx2DF~AK2 zmD6g8m3YHh9pv2;_ft-9th8gsl=a;QKP7q*EhV6;sk}v~6rXE)J64NWax_lmGiO`Y zh%T^H`?Z1S_U@DriB0n&`^Ax4=9`YYBqX1d!dvCjmjTC4th5)AE2_B>Mjb))%_9eI z>V*FC$mfPCQ6@JxsKoKJ(+hmxMGuAd0XSB&l>O>)D4tB8keKj6L?~w7mq(2K;k^gt za)5e;YIO~SgF^^?z^K7&0;oG&0n5v2v*a(|z!$#-XbJ8}=`i7{*h*flbLY=0uv+B} zAK&;Z>YgdB-AUsW%S=2*d;aV#hYZR8x0K$7l&K`@?scvhw*u!PDgshKOKy!++b1b%z z^qTQyt#5j8sZn>y3zS^Lc1vUY^9t6JuN5!jd&MdA8vSg5r7mtx_)hCy-(_QF^sg^F zCPBG?nY-w$8u8+X@O~%9P{zjmW2Xf<8MIZLAp3i(i$uSMWzuOXB{pcA9}@cl&`?&< z`M73VJ^ae#pcTpUeLyIUrg1{-igN49`7>$j<`C_sC^V5!$bjvh@N6z@wUlkfbmWf^ zobB{-h?KG~SHf!bk!`F?ZT50^P`!W-x5g9qL;9Ho{V4R;!`dNa~pgXgs{LfJMUTh~fDp4PY@ixPt;`s!>Thu4Sa8o>_%-)h5~GS|O_ zA-2jTwe6!x5!q2U$V8%Je<7)@a@J)5)w7uO4@kcTKhP4BsNHUc)U{y)0m3{Lp0d69 z{IxfLAZQ9<06P<9*GKAKv~;FeV?q>0-MG$hd#3fsp{GCfdF_?((OXVBgZ1o<>)Gc* zBllv+WZ`q2k?ecm$5|4RW%t+!6fbY0vaWJ7RKXS2ZeRz65Trf40*Zv~_G8m~pD}3K zRf=g5E1lG!qtpQ~E$O<(?9{jQ`JsY1zZdiC*pJgl!X6w$Wl>1BiYe97k_Iyl4UAW| z)`rp!tX(j9b}zfAe#i{+ydq8*e!#df-C_Gmu6D?{HH+$Hh3+->*$)5jCYrKtJfV>0 zoMVU-ws^aWCZi{mAciDr{dOkn&Jo!Ou`(JrJ-h9Ozz4V&6hS&l3bI`K)H*QEt4CVd z`mXANK`%8UgDhDUv|jyq%A}8+D{Vm4%SNf~JRnnP0X6%6Lzy%8ePaQCkV>vAeDRyS zJ;s|oHp^3MKN8t6TVi8J)qZbP)@36{KWxiz^drnOn)@?pE>fTDhxNT?pNbhUs;I3e zqFu3`%pH)DVBvN@p&MTlxW*lXsv)3&{DEkBbzynvlcgqu6`hh=CL$Nzwe_CNeQ(cc z2cq(j8@welT!!B_c~>)oKALm`egZiq-ah1xwXK2Nt8z$CRB0)TG<;PcK&uCBF+lwv^HRDMryutmK zl@N8vArM5LV*gQ5Hm5$Xu4SYRHh#>ti*O=hG2_+V4tDo&U#+z!C8r9K{DLr4oxh+b z2sDH1?BDZ7oV2|0Z{FU*bo^GVG3rcwW*pY<6r;4;twZtR>}@MeI@JT!WL~rNF(1|` zR@Tv?uHMXNMCMYs#QM;Ro9N1b(Nwojj%F>wf8?2QgZqn%9X=&q{4G;*qxHKi<+GDv zCQ#SS#KieN;T|+~Jh}s~WjYxkQ^jtTAE}8f2}=?)@gzB~p{TDHSJeIFcG_7}9pc{J zFnCxxRr`lzfdw~|Y_2lk=W70^)>WKuYHf6_ImtyQQaz1c;zbS8WOdZ=ioCXx7skIo zdU+$6y!dh}uQvQk!b&gK85(8xvYBZy4hjfDP_Sg4+=9F-w~jXkK1K%-a>ztO zbCENy1W$@2Uh}{9G+t>5 zKAf_S$EH5U2?9wb%m~~+lea+SL*jkSFA}$&4gINK9(Yy9cX&_qOiJ))j7Zd~A0uQg zSu}u<1-=yBYoEvAWF*uN)D%K|-SG0yiNj!bAkKotaWoaV?m*tFTjVYP^~15UsPL4i zt2wb^tfk7y^&nZJDmGHDQNGpgBpcG_QWjnHN3rD*We3ZJhz&^=CDihH3~JVy#}y zanAmJl-*2^VCKcR)%&!UUGVSB%hPoH0LE#Tb}<$&8*fzQWBCmz6|u=bYBDi#RHy?( z3-^yBp4;c~dt=VD-{lspO>R9C8F09o6Z5;32RcfFvrfw)rYs&2nd?MgQ`_S5Yv{my z&~%f#nj;;T4JHyGSS{eocB7YDxxJeS*iwdB4C%Gb(6;oiR0VBN2!UmaH8Za47}HbX zB23GL+>HgQnBf#gm*J;jxR`YjO7s_HA*k5mY%>q_;|sOSt5 zi$WUXp3J05EKz@#O>Mba@H{R}J|s=n_5|5ZSpklv{3(f1ArNL^-{;KvG(Ai^ma$<9H4lW2qdaD^;r6!?7Yq0>eMo^41kE6g=#RE-4WbJCsf82v)Ooa;GR>U;a!(;SJBF=9d z?Dqzc4Ld=^8dlKJYYzCO<`>^WSvC@>924rFnWlR4fOCyTB5t?W1O<Q<%~K7Q#u1o~XWRjcq7mfUMUa68MqgsUTG>#;FuaZv za24=|4}39Ou&yV)TS9Jda3UNv5#I;NNB2~{N!h#qRhrd|85CvM=Ma3r`B`V+avJCn zM~5D6wfAG9G}X4iG{Y75hTd2rKmA@qJl2oN>hX%1c5g2ONryegdgQ$ther!QRTp55 ztFs%8c69vr$QhmNLb&NC;hW%=Pj$*HTr!)JL93*%!le6UDuaClsg`VC%>XoH4^Ucg%18&^FNOJ$U|rtI`*_eRo88v3#GHrM z723d;<4@IrHKd8`CFvXQ?`KYP28&m$!=MgVr)rZ(=T8!b@s2=($Rpdpv@ANbrDF}_ zrc96+b*1xsEK-Pj45e7pXYxS2?nvz&!%7;h`<#!vZ>*Hq0iCzwk;=Bv5pY<-gDJSf z)c_qH@i3u<3s;#W*(2ogr;`ddFl<}1NFrZ3)b^cG)4_h3KW`Ff-ybeXwY?fWdY)ri zpoHP71$DH58FaZXS&zSjTmH#rt@!F6Hl55h0yHnJYcE%-*~k2HVGd9Cm{q10A$-XUvjb!6*T;wJC$*-oeIATAZw^hL-oZA2%~-fa~bBh z#%Dpk)IA!UX0%1y#$kK)mJ-$KlKStD#!7P@O%7Hm5jK zN=jRRpoafOCabzy9Tw}lJ9V+9@p+x!cQib(xRyhsG@At`_tHB)4*p37m6{NW#-2J7Q#=T0?i-b!PKxd}PV&(f?^k z`+0AaYuhUCazTy;dl=3`8I8_C zm_aG)<04g0*&Vk{m&@c9jYEsYN7h_?g( zNI;Lx;(VVCUl=HC&rhJ#@V;+n!8a@?-?X@}DAy;h+vV=5NyL!D8`gPG6ltd4JTQLS zt9mhPob9XxYe4nk?RAN}48QI3{eo4@^Z2bXLa6~lEU9~t+5;?F6?k{b zmz(fT#AxsyTCqoBIg1OnAbl$Aq`k3U?{*dYWB`_SX62-HypaR6+fB++=$JtR+mchd zenmE*g?Fb~4i-p1gs=X5+&-P?{-eCccTSve>)TsaE7R5uibO<8eZ9JR);n|vZF|a& z+Mpi)6uo?z&Xi1qZN~W?!CMB!_uc&KA1J{4lZ7+ZGh1@K5*LZ=!e^p{>-pVo{Wg*> z6*f-j1U+#8J{*upB>p=(Hd;?SoBX2DOWdZuiu?R`4kvij!v7_p9amYF!TM9dY?}Wt z_;0bE9{1l-*#7tW67c^=&y(GFESRjttk-4um#+7OS}&(=ts6TyA3!HiqF6A+^K`n= zsQj1E2cmttkF*HqN?dhCBuG4RMMVBvpuFZhenfoiaCiKKf@_8FbjW;Alq?fB`u0Bn DuDu^W delta 10478 zcmchcWmH^Sx27RjuwV%iEF6La2u^Sh5L|;>Ah^4;kw7365P}3J1b27W!V9lJp()%8 zE1XNdbMF1d=)OHh-yhxeZ`*v=TD9Ie_ny!A0Y=OoMo=xPjWWPi%&K~E!C$Aw_9f`u z3+A*hK_ODkV2r*MQggGP@>X1tF!$!s+$ope7b;%Oe)&9OzZ{rGVcXC>S>dS2NX9Q* zwhLcL8GaCYTF8AhBF+9(+zM!+_Gv^hr>-M?XtJ>Enl=|SHy8X44l~`#6S>Kx9@F(Y zxa>OW8x>^}k);EeXy+4eRf{&g7u>W0tRNXYSyp!-Tzm24D5GB_Y>!(-9!ibW1RVO%v173hGNRVP4niwQV!k2_KKYDZ)H>4xjw$BLi)L_?&JGw;IeB5 z4oV|sG_MXmpxiK_ySZ*g^)|>>Z;^3<1XE<26IanM{bqq0d=!S5I7p?LKBpj7b*CFw zlsTrGk9z9iwdt1>;HmRacwieMpLj-M>YM&^kAU&hPmCKJyr0ga% zkX5|Ea3Pg0qIbmpH;CI+1K1vHXEn$kC)<8KjB zRL#VLKTX(7l{84VqZRxu#lP$}02>)b_{`}A+?#`d95%a8A|CF(`LB^PB^_|hE{8nt zDrEY7?5|8Ylxc6s{ppCFnYL z>@os)OdI*PoX9-kyCAN!d>wzIDX2c)u{=IG@T4Ik??St^=iJN9x}iF%-kzZKF-@BE z&2C5jg2gKWj#x|YZ+UQCaUXU|7ZCNJlf#P!Nl4|E@>|N~=Tw*IB+tE$my*m*p973y z?J9ww1CP;j$wWEPvzz_@1lwT-DMNxHi+dynAWi^=wxrX+SH75Q)s6& z%;C()3RRZ7`_4gZA$6ju4TPr0*?ZFjC02AOJ`}QSS0S&Tq5om=v3?|~(=embn~KL^ z!|@ZlL9{{t{n6O=@QNovl(wk6?rN3*mgT%hKt~`!*gX!Zobl?#a#l{|40gva6B1!S zICdX_*qf(XzU_~QwqXB6C=~&N*MlxRaPWGjda|&LBldl{V3jd_M?r)NDjh2FJsziD zhIw8!SqUQ9UH&@TvpH6@EAw30zmYl`5ba*lm+t&|cglrh^_d5+5Nk){{Hq=e^QmhS z`}1f_Hb@`mf>ARQ3!zoCOI6GsI?4hd8!ZDk3a7j;YipmWxW#C6PR{k0=C9ftUOtLM zScHM?H^eI+JCXUpQ^|`yqBO?#>11`rO}qMrTNi7nxSmx#7KweTCV1>9G6@&_u+0vO zcBbm9W!m6bOB1i!ym`DacDm)i`{a93?^s%2V%xpq?-@O|!*t<;Pn!!SZVfQNMGSVS zx^QST@>Ps~DO4&_3g#<4)oY)R_rYF`XmhFkQnEmdZ+5EGM*WbO2@!7Gu-e9s_lN)F$QqsUu~iER2xewv~k zVk*vBwUJuXlYPRV9g>%M#`)AhqoKBZVgEMd4>o9^$xIw35YZF8())>jS75~1OJt7hiAAe#F~#A7<`9!w$xc{W0VE)Lb(CJzvF@AP>b+@ z>4rs6dTyNS%g-g9Gu40Xg*dv)Z}EP6l6y9vndesFlLXUR_9(5U_2`yLp*+)Lx{~I> zq=lU1?sL}i*>?OTdT1=`i#oF~4gVx3!8y7KXSSb3#ebS9|I??4mi^;7j;>dgD;3E< z={JPwz_@XTrBN5aDwYsVg*aMl64{-w+q3_Gje{hl`}$LgL`v+VrBI_E z)D+;TQlNl#$F9UVL+JAtRg;a*K1*_uCvxV_IeGQO_wqpeyBlZR?L?s?uP%8%YzHHf zvx8W!_B{`=JArs@!z1d2klCb_7~{{z=CRax-Y6E%z5XIKUT*e3LR8v_-Mfmag7OU# z49WOx-;*-k9Y5ED)>R0dW}N8+H|>8vQGLApT* zbE}0Vf6!(2Nn&&6DYelQg)qhkf%@dRC1i@iVyxStM{QBL&}^2_hwKr+ww(;jMfc$6 zQ+Gw0_oc2+Ea$)ZspV1SsbT3H*NlIDZMwjDZ46KpvT$^J4Lv%THnjR&+m^BuLZ5zR zO>053)Tn!TQ8#SC8w)Z^PkXM$V{geO)<@RpA5i3}ZlgB!4xac%YphEtQ>6T)?CRKz zim{h?tMuTEujaaBDw}pMrUpmLI4Ux8Z(_HlZ7y3wS}_2BpM_ucEqG-Lp>0H$7^D97 zQ!{Wz#AQdv(e}7w9P9B4ub}ou2kb3L|ALjtD&p75*s?Cb6>aA5tj+@}u{~1QbmB6o zvQDfPpC${RQJj8*#zM;|Bd3v;AzQ;9`ZKDGS8HZGh8X-6E5cb+F0cyUhk0oIn<{4y z*3EqX_pk3T=3b$ZxbH-zHPDV)go(1heUt*U-#yc2$F8}JDLrS=*AW-Vuz#H^qJWEr zonc{^17=K|*7hSL{2eW|4MHw}DGuimnNQlrqUjtJ9?IoNH1_UQs($p2zNm8951Kcj zk*jZNQE~UjQ|;zS!!JS2*hK4h32|{6(Hhu=2Ok%5Ii)Epk6i4)Tw(grkJOq*&2E4# zoG-Wrj%)zwRvljgKna0flZ^Y~S&fgv+{vQATy7)Ks;eEGaP-AdwLD_E4hu%`oPq-G zDJU}4!mwcItp|q3T^z_2JrRayro6lN+Oy4LcCrb|#`>=!VcM=z`*Y^X^?U5L;Dl6) zfm<7xYoDPwt)H0KhlvGm^H&6*PS)riDw)4q-u>LeQkqb{!QBbZ~-<(c#FrgYk}m-7(W`= zh`Tvck@-YN|C*QJPJ5CJr2jc7uy4p&e+?%?d(-dH{(10crLP0Ng&eT-qZ01M7*$R| z>dM1DFdO$d(IB}zxW4%9lK|%s%O55Lv_*<97y_P8K>E3i3{*T-P^on)ucESCI4u|^ z>>7y69!-VajvP1{J$gmP&vh6ka9SsyYo+tsijO4mBLgE*wKvbMnSPPHu&RX|qkBsF zQb34c)P3mtsOiNaDv2z>D^(FU^Akk8&Y`ibW!ub))5+YhAgQ7Aw_5gxdIpdWga=%l2z4S|G;k+wuF zQFvg!(%D}fFtY?kj}Yb3s2$zh3NXEx4%o%Z>Xt{9%M~-{%WvRvMTqo-$7p~YZ$B4~ zD-N0Vhc$dVGlpOYme>?xU30B^$A>E;lPDqIQr|=;TN!@{pG*o9GZ>h=7or)~ zss4=tk7p=Nii_#)75!BDt21@bVLF%kWr^=ML>t~%#utn2d7TLRF*-hBG9en100ie? ze3tn{c*+)K^#H(HQYB_IEG1mKG0BgZ7QetN`D@y}PW=Q-eR2)U;^uaZUJb_Q3O9AW z^4n{uKdK0WlPmKT=44roeq``<5S+{Eu=N+xyg!DvjXfuA+$$NYZ%Fvsa_FRuoS8fy zsdlE~=@32fP|EW9Iz^KCC)CzZjO&%ZAocg6_e{o2fY03X1HW_0o~ouC6)oCL_l=5V zq&-#5^RYN@MtoXTR$Kbh(*THvsL;JqPkDp2v`HPbZ@JXM_k=(I`WJD7r%k~RMg|%9x}o%roCMX0lIV_N*XZzJ?AwtAPbG@j|A`_gy^BC zSA8fHe-s~%;w>H4B1@SV|Le?$VDx`qrepiM68WF^(tW}p79slQLApoxg?vx&nyCfvYx6RUn=Lh4s3{H@-?D-_!H4GU;QorXZ{g~@sgoN zqwP#~|Lq9>X9N77E(h?hMfvx7{J&7s^E8K6?u7AyW)}|9lUIE>2s(}o7=4yX@vpc6 zn|*DX%x%A#H_U12d{!ncwBh-`MxPwk6?e?I}lB-EiF!W?$p;59+Xde$xRc2a2E2RPx5e%~4+mf4G(`vWLav1T9`zc{`^H zdlb|=zbNYqH*@Xh*!VtzeIj*h^qK)|vN>F1u_avZkc!+v^S`kasS8-=y51iXNzNJf zUKMsB@=~P&kB96tW*r(Gp=2<=K%1x zs4Mq8nfj}8zF?jGA@YA|)<#$0gCX!ZH!upEip_Ov9(mu|tMQv9q;-)#; zm99$cFywNG*hHDq?sY!+tS45cVmCLf5CE%B*+pbRgI0~8Num`5fsb=<7n06-oV((? zcp{8>K~YX^0|)c<2Tsa{V39*5F<>1i>i#URDgwDFve;9otXDS&@^mQLdAw|6vQhC< zl=qG!rvnhnt?`N@e7j6t&VDM%V&qQ zudBZ$6ANQq2hYXAUf$EbS{j;sF!zz8Jd*=M9HF7wJT; zCl1C=k6QuueY;SZbe~(FF+3$lr}Ww^nAu7Q({nlAeP!Odb38F`F3VO{J*;!}nI~!^ zDWOc*Abj(8QmGCwM~)3CM+W#}IO@8$>{#FH6SVe$qPkZR&{_PwRnK}XGf>xWOJQ@xvdVtb19U@QGSxfI2y9|d{`no5@ux*_33 z6N&rgkeHsN!{5G3U`SHY#g$VF*C|U#%4LZRZ|g177zN}|h3I|=^0Qdrr~&z^ZZ->W zr3~*`N3X_|TZt6C`=aHl-;JQf^OoaT5)C6c%*q+;_B@?`#J;9Yd~kr7Opw_qWVk*VrGoI?h@I1audxOYPvKik@p0;HL$@e?%i`ALiDjpZ?sNK#7>(Js zP7g2l{Fgh0*&HMfDY}!@PaU1(BHQsCuVcNj4sqnPf1{Xtoe#L;EaJ^NObZ%up)##O6tBeUwH9i5DPK_R?2@ zGxT7KD`ddFiE{T8ElNnP+-tqdxhkN_JXfkgcyVD;(BzV#f-mZ0In#u;4*S- z5tQP3xT9)N7`m||Q*pipsRc|#A{~*6FW>7vZ~LHez(Vay{nYa&ob+lc0NSOs8{n`h zKjLReiaIu)&>p&Ytts4KMJ{SBb!p!dTW~}hWjCR2 z`zpWIn?Wp4;h0088aY>ILbG}yNb1&H?zIFnP#6$DeFNqxnSe)@v?L`(Z2AaI(w=$M zt(o- z1AGp+J&&YZMie*azYcziJ(Gyb=1uCu`MF3?wJGs>hnaF}=ht%HssxD)0?0?=3!8Ko z_#N+C$s6+c%p9fFPUy#V7q!tkXP5gm!NOjDH1rS=7J4J_5qRU4ujPDN$99@i;Ca2b zGQgBP-WZRpf0_fm;6Wffl-TA5`U<3Opmo5bTP^qG#78DAkeYYUjHf#NKn=%I+Ov~_ zZ%B-hbPdQv10Icb zFeY%{6TZ!}p_LB*8OVpa)KbKZcGH945FNyVdYJcql9E?6Ckm^h`=mq}%8tZMy}>hA z-d{NEE!qOjpn$q<@FatQgQrXenQm%nN8B)CHf0~0FBLk1h>F`(q4!UtG9$bAU4$_h z>Fw2{KWsE#Q^e16%?p4`JB;=M*7@|oVmqrEeiFMBjhjO%o^!>vfbwT0Ne{@%?JqTKMjsbZi$4f!`xW03?6jd<$^Byk@(LGPjOd z?xkG!onDG+0_Of=zx5#d&KnW z_AZBqr>z-DYHV>VXy>?}Z87}{yb{V{s3-AMgsI-+t)w!IKBc|ksK4h4>ZBm3x!62; zHIL+$bF_Y9C~)$oA5~c^(6KQNOE=|+hPBA2eio0jVH-$Uyl zESrOG#V;RCRk#!A|H2q$DBNbwN`3oY=(9n|0y)>FiEg7e<~ntTGD_>?v^B-0IA}B~ z8Nbx`ksQZmr|YeEO)F$u5Vb@6(p1ZD;nKkH`v(*U5J00(Eo=ux2yhu2=V(Oeq7&!^ z4KW6k)-{w~&`6+mhV{H$r)*xeFu4m?MQOEKnMxQS^9hgHooTHYbnwR^&|xkQpB+I8 zQd~S5)|Oh&7LNmFz7Exa;A^89qwhw=2diUW{@}|*o6UcCELj!z#o>#{FnkHLXX{Tu zwL;WDeZ)ut>jK%Ds+k^_SeMwxEdLHzblgUQjt_m`MmO3It`>4gB2Dg{GMJB}XVIW8 zBXOIrpYr{m=9QHR+|FROw5IS|DV(Wj;?X?ZziPd4)dEat$p(Z-s4|&jwWd1>$x)`A zMi1dMaZeE8k1H|cUGB3rBlX%9=T$c@MF87c#*FV{uAgz|nv{@dsLv7Dg-pA*?mb&8 z8}5@+yOD_Fpr~+4(>{5>S2idEtG&xg8v;nhWgn*#+%9_LW4~z4r5#n;9A{5~o=Wb{ z`H+p;9z$y!#-DV-2_u*{(E)WWk9wS+q>jKPS%t+7bhc5Z4cL&yH%bc!K1t-)6~H@H z#k|(8S@q3j*8^)(Wt1@5>JtakBYXe0HiThQVmYBS%Q?!OIbu+Jq>H;p%P+v-(4C!n zVSp0){`sFOIV%a|KNwNwV?M8|QzeJ|GZ{Qwyc{k#hc-~Kib|J|9+)w#PyHjO%}pWw z?9Lt7F*O>Z;kJVX{S7jJ2a=KaCIGpyEu>=dnzfsKU<4skRErp76Yn~of?n=n!XRbb zL0u%F=i&a`$PT5uyt7hJs*J5C>1gy)qn|{eoDuT0FQ*#x($~ZqW0MkImWjZUQ1bB(W|oMl&VZ;%k*^`CqzJU2sW>8vZCWPbB7-%12jsclBGwiays@ z7|V09keB=1Bf;g$p1uG1GPhGVcA!zgci3&Qy%_hoQF^V1M{@9%}U&3Wu_Wm(elp4v=|%@ zNoyP5KZj)GZfrlmHNh1tuXG;%N-g?Zp(nGu57yn+FDz@$$E%D=oeL zHdwj-jzVl#UY2ifBa~X+aAwZ;yP;ek^uEvURr^J3n)JvOQ_@$lVIb~>0DCw~v`)fN+Vq1y4!&EGWhCZtY)`Y1N`*FN0 zR%7;Sn~hQkZAOs$2^X}V{(yQcWi;6MLHlSD7y05>`p)b`yuLNdfq#3RK7VNLf4t9G zy80e;cuEbmQFxj$@u&s$P_<-_w^j?u58^}t*uSd_3WWOs&!#S4M1v01&n4y|qt12@ z#H7|LV%YRlCv99q!Z`m+)kKZ6I_d*&=u;`+x@FTTEI@q0Ld_N{L z!r=}U*D*QC=SMks@;=eGqM2Iesh_h~6(7$&)lkP$>&*KtAsZ%?BM~|8M~oUneR%er zI8PtD&?OSo-ysyQ@p^m`H0#4lzc3w+l)YF{3y5@-FIqTCtxeL)WO_F^MIjekdU%6$ zkrMR4=BOE6la~z0xA;bOR!IS(P?rD#9ihPnT&{GhPYMP&d7dd`BmVlR{WP`uQM}BD z#m8wx@-jn3Uo?_iMnm&TjsUGyZXI!zB0}kDb}$#`lE|IpuK!TK*EKEERiQ7m2%`q+N`?0WXaje z7}3}pulyNX;rUcs>%PqF)hKMhvySgIwZqxv8l(NDt}TzWsM7g+3#&bzSUUe!EV3H6 zU>sW(Q3_EPych*(;OgM9)sUjSw?brnqMdP0<1S;u)PEwfDgwcO2$=|qP2#f+XJ~I9 z$oZfxB)`tp$>~-0+KA&hHRMB~9GE3Q=v0V5yz!N~fDFLR&p6`CxKsUVLUrTdT|6I$ zkd}_xChZ`z+yftaQ6Ia%`0PY(R-}a(NTW3yuMy9X%I=HuU_FyclMOT{N0nUl)}dm8 zgi}_Z&8*`QME26~v9H>t(ELJ-b)FVHv*p8)Sbr0R$YEt>Z=Fy#;=+V%4_0kX-TwUv1oUSZ)z-WUBd~E)?uB|BHM_qhVWxes+&{vQHhgCsJwA~ zk_CrIL~&cvx2X!3qtexCA*=gcZ;|TE0!CO4r&#&9eZ`UHYYLOBm+qH~7ygMdE?si| z_4h;@i`TTJ++qrqu-{_PmW{};R-|&*QP2yP6F{7=$xTzm2DKOcGG5p6E-Bui*Hub1 z{lkIm>gRi%E=1G^S0q(C8WZbjYQExaN?j&VJNf1y-T6A#LLS);W}KBtQ2}Pv zlQY4k49;us4xhXoPSl@IG$*InlC(X;#~X!O#8sLz)f^ZMejx)%L!2%qa?gM@21ahn zL0Vmsl3E)-nF0dNy9twOMk!UzK<@99;d5VO2H{#ZId#8z<-Ag~^3`LjY~kkNujH^@ z{PljI>EqpQud5;kFqx!pMQFf82D6T=Vk)kq`lPL#B+nj5g`H%d*b~N5c;>&Up6O&( zd!&t#qF-#Nafc}|HedFuq|<#>?PT4tnXowfFy+Dqd=*8JF8pkGvaW(~E?wHzlU+c6 zScb^T3(RvLyy?T;Um0Y%S(qPl7M~eyuH7R@49Q)cg4^Ik>)3X3rK2qGKV6%zhSDhM zUgD4l?S|EbS(eR2ksfO(=y^iCi`#{)UH7=kLJ2Fm%1eOug1yyl^Af!#gca|)|H$P6 zp^yU75?N69zf!JN70z2~F81FGaYcB3$pcNCM-F}4 z`B(md9YDk(Zcm18)*mOC1x)WqxJ1qON^2Sw$@_f{2tZd^`1z*F{x8D|k@8rgfrn3U zbhd!7t(Izm`-yW(?~BmJ0DWdfRZGJ?%vmL!Pz3f5x%13~&cRp^=-*hVS2Z&)j;X=2 zOZOk<@_<}ung78-|3CQ0{}DbtG!?`i<7(yKtJ_hLySKRh8$!9HJzyrw%9ruqBWeY8lFho8KR%G+`& H(~$oFImeJc diff --git a/docs/images/context-efficiency.svg b/docs/images/context-efficiency.svg index cdcef00b..fef5e29c 100644 --- a/docs/images/context-efficiency.svg +++ b/docs/images/context-efficiency.svg @@ -1,6 +1,6 @@ Offline context, retrieval, and grounded benchmark results -Artifact-driven offline benchmark report with context efficiency, retrieval quality, and grounded behavior reported separately. Structure-aware chunking reports 740.3 to 214.3 retrieved tokens per question. The JSON-shape payload proxy reports 24,590 full versus 11,138 compact tokens; MCP transport was not measured; provider billing was not measured. Retrieved candidate quality is Recall@5 1.000, Hit@5 1.000, and answer-token recall 1.000 across 26 questions. Packed context quality is Recall@5 1.000, Hit@5 1.000, and answer-token recall 1.000 across 26 questions. Grounded checks score 5 of 5 answerable queries grounded and 6 of 6 abstention queries rejected, including 1 of 1 quarantined-evidence checks. Decision accuracy is 1.000 across 11 decisions. Source artifact SHA-256 1b894b37574eed06e56feec10840744c36b98813a36ef4c7cc38b9957856e1ef. +Artifact-driven offline benchmark report with context efficiency, retrieval quality, and grounded behavior reported separately. Structure-aware chunking reports 740.3 to 214.3 retrieved tokens per question. The JSON-shape payload proxy reports 24,590 full versus 11,138 compact tokens; MCP transport was not measured; provider billing was not measured. Retrieved candidate quality is Recall@5 1.000, Hit@5 1.000, and answer-token recall 1.000 across 26 questions. Packed context quality is Recall@5 1.000, Hit@5 1.000, and answer-token recall 1.000 across 26 questions. Grounded checks score 5 of 5 answerable queries grounded and 6 of 6 abstention queries rejected, including 1 of 1 quarantined-evidence checks. Decision accuracy is 1.000 across 11 decisions. Source artifact SHA-256 ead6889f9050aa88a916055ea805a546aa179b6cc9a5edf05291e3419089ea4e. @@ -65,6 +65,6 @@ Quarantined-evidence probe 1 / 1 abstained Decision accuracy 1.000 (11 / 11); the quarantine probe is included in the abstention total. -SOURCE SHA-256 1b894b37574e +SOURCE SHA-256 ead6889f9050 JSON proxy only; MCP transport not measured; provider billing not measured. diff --git a/docs/images/evidence-backed-agent-examples.png b/docs/images/evidence-backed-agent-examples.png index 4116fb4bbda835dd205efb5e7cbea724b2d40b7f..2354af3f301aa87da617c9e64ed58667035f18fd 100644 GIT binary patch delta 24363 zcma&NWk6Ni7B)<%903Vwq`SLQMClMIX=#w|W}&E*u<1>ANOzZXvkB?WEghTAcjGyp zd+zbX2UZpQ#KQ;x=a zOI+E}*qYHkxr)L$U7La`7OTPcHT|cvg}Urj;KKPnwgv5e*8cKJm+{86n^*=mRtXOR ziRZX49()SBy^TXO`(ju-*t48;20Q$z@_nGryEzr&_S0Knk^1yVVnuFYuCDfYJJ%~T zOCE+%wO|XBCMxcIMT9u!_*8C(7zK;@#*JBC8slSQM^tP=cga;6my^p&P(=FUHT1R) z0tX}&bLBoIjrpB*@nz~cK_&ZuKJh_zgMPzqG2WDyFJBIQ@z_Rl-!rJoHdw@(59H+M zIfP+2y8UA4rBG?|%gPWlUV9?2Zr z_%5jMn&0Hfg<*Zgb`lsXY;wALDDJRzgPo)_?NAMZ&(jg>yKu`M+`>p95=2Y}}nYS{l#CA2s>^VF=3L8lt)M*s67okqXV?T>01);&h+o#{*#ujcJ zLvTyZL%KGb_$bLe`kSo`CoC%MzB2_saEw~OCyZpx6aJvVW~@M7m%dCRX5ZXI|1lO_ck zF+d7%e6*XzUBefBxHFH5l$NXoZeSaCnWh#M#5Q9J1S-TPG!8@3nG|<#fz7j_pt{8H zbXAMj?JykpvL7j#)2LMb#59H2w@uqKkqHvCX-~|5h!0Ot8h2*sf4AdRwChX!4+YPWC>OHihwqT+Qqk83?PrN5ZGMDk_3dWLBmXxp&}j1}rHF0bvrvlTMd zxDF++P?c+$6e=VF&O84A_CWO!U18xuB0})FFIm?+RBp>w<;4_qJ0ZEDqx+a!xcxfF zsf8n?N@h*&qZC&NVanaZ6b0733y+qI{;0cU916POO&a@#0aAMTGRs#u$|-K;#0~~K z1uZ#iDK8y&`g43~$`(%Yem$1==m>R)W3vLsyKFbQ`%DLJzY1{ER$HWLoFN=A{9S0W z6TVm<>9#v8Cjf&E@GPoUqI>X4`Sy$Zh|je}GaooHjJ?(!WbT7@4(?ajp9z8XYcQ<3 zn!pR+clEr+xL2r%-?c^&U}HZn&3q_c#zy-A0!;Pr2dpr| zCqhE#a+Q{V^GjM{I`6&1UAQ{}xsx&^;3wIJCOom%pXFY@r1?ISG|)Cxfk>WsA`&oUL&Ae;C*#E0 zLQh47^jztdNC`h>S*E&mk6OHfeQfC~Xi@4?_i0-PfUQ{fZCB9zjsjAShf>XLxvl#~ z$t3nltAE(3u5s|8?rx;zq7ozZu3Qko@rYWfnuWU5FcE9HEatgEWk`DQ17tkL! z>V>YC;P&lQY-{#m)EDvMwB=w~o*qH9J@Y3Cm1%}o48)Z?3jU=VSQCvHVXxuw`3cog zKs6cy#HWr5_&D>MhwXMxeh%g)I9`Y9norbIRbW!rx@3J_iKz3M+47;&ZB2B$*snl*Gtus`yNg|0 zlEMNSaXOO$G*nb?*m)e2Boz{Eb=_|9(6>iHFi7aPQ$7}#4{MbY{!LaE=j~ho@og>f z5or7b8}8T-%rGn^x7tW<0=@3dZ^8m&*Vpx88J@2cb14M92bKbO{J*9uIZ@s52tGuu zi2M5H&C9JG4@KhPEC*z5KAWD0Hl)r$_QL`%`-e5w4uTXSLAl{6K+;@B)LM#=7ThGd z9FQA!Srm+zmr{f#IyN2ZFamh&)_wCpc|pqCpgMyAGxtxzB__E+RLYVxWP5@N{??gO#VmPkzUg+TmB&M942Y5KDp21T= zgG^#QxmFbx!GRCHZYYP0A%r{#?PDIsRVl;R7LUJT8T~==*UMl_ETpFxX>WoaCQvcq zJ4z|$`VE7>Vj?}mNmKFo^}m(aq4z@LkvWezdBIKCNCddQhr%M?EX+}TV)-6RRTYc{ ze}Xgotr1BP8xQR;3jT?D)Zd?A{{AGKDXStuNgXVJfkc8Sk$Hs5^Gb~+i}-#VTCgD| z5-vd{Pk?sPqqj5`YWg2?0)s!RaVL*E=lg+!v5=nqJ##p54ty9fi@#^a{5^A?sZhb@ zniLof{yP6hsdYSy;BRkb{6@7D>3Mhqdl@AvER5l;*U)q{l9(f5np# z3n*VRW#IYoAyNvakl;Iv`fxkb6N5OA5j7FV{!HV=z zU2i&M!J-)OC%@OVV{KEnBteM|ls%OU#>7G*u(}5hJvjfZjHnFMbHq6x#Q|4X^ZaDjJ+U74cL}L1B~8h@P*&oj2*_O9fe2 zZl|T+tH0MnRKa|VFL}$5rBeyqf`dee``gnk2cg8`O*N1)qifoKnE%0R0LysiAF z@nsf5ld@yI8D+QyuYYY3+~01MxeKL^=R|cHGoJhj4(@L+0u@Iu3VD*PpUUTz{PF_$ z?S*yhkc5rUG)wH8MTY+!$emLh+5V%><%T5LV!v(tZvdA$2%%JNnyd=Q`J~hQoj>%S z&|_n(j$T_j5aYg!_`TQ}&KSC5vSqmcx#3lW!1*r!c<^_C!XK!vf-!KBNU{GlM)B*a zM5W+t@DnT~BH(vS1ddLZF!3kv;CIiXaBW{=!IS;(zm89AA}ua@^QLM3nFaObH-iG>BL6)FVr5qt;C=3fCc|xr17lzz5n}y~ zJ=}=Cw$%}EI1Uo&Z`_TdM}5_4MWAXYpQ&7=9y8J99OIVZB&s(iC{j`7WDEXS;rUMW zJLq3-?r6HSIfWT9{+k{DLi3+so)*w8A6_WXM+oH_5&X{F`h#?TB2VN%{O>RGEvt;l z4!r7Q1B+uI;o<#rN{Y73oTxbeU_CH4782esQ16q$J*dyWNGsCk=`gwe*9^tMkZ(jO zXWoKDO2(U>dn(f0|a>4mT}H z^nc9%D?`HPAJPDi;3MIIF@=!+!27*XRD`Zvs_%z*0Q$>2@qeyXJt`8bmFj?Oxm z{%`pgKKd4y$a^8#rEkOFmM2K~w!aS;xHnvq_uhQxkmL(dd;UAWB>sQ%3mi>$?$7sV zl5(3?1QPtOgb$x^fbbqpvw^rlDmHK_J`(QlfO-)<i`11GHaCUkD; z@P!mHfDwg}aDab+u5NG-dbFlNg_gC%{|2SxlI-471s{l68PER(l)|U*S3B;LSQdO2 zMJ=gcpHAa=&^JdC@VwYd0yR&waI`GM8F%qTA=4@jv&)CJQs}BnC9DW!v z&D$(pW7T2h=j)p+Vk_57jy>)<7?!Fv0ct_@mp9yl(iqYcBV}(jfJ<%tr1~#B2vmf8 z%bd1$1Zy`<-uP$qMo5>y`L81rzH(ZOB2Tt25wb9nTeabCWWe2chNI@KBG%1~QFhUf z>AqoSC1~=%!GO*Vj{H>2@8ok!E_Ppl-6OAo4i8#a(<%jgd6oNeO%Q0}!YElT0!l`a zT9zqNiv=?osf>L8A7WdW&};MAr1vqSH$|D44J^F5H%E(E z>{BdQAL27f%jMt^3)6Y{dCjY3K%n)^Dz^OHi9QB2rR}?fMa3WW`Yun-(0668q*BXcOP*rVYvG3qja)Ix&=O18R|KlsRZt6!^+-31 zLSK5%NE{!*@x})Sf|Vcq*>?giG20W#ycsoSI;vK%qsU?d8wt2bEGR!$zGc# zQCJSX!7rQ}Y>nz1{`NSTeYZalyXmGQzubID?e;4PwHrAze&M@L%=l}ycYe%aDmTUJ z#n0GT38bNdWTr5%L752@dsG=8nUcq5Hs$96@5(9Zj4F?tO*CRHbtXzjZ6 zGYH>?l9Ltm0@$uJnWqidG$&4qglqw=ypv->ids~0%C&E)k3 z)RNi&ResQ}?;Z_o`#Or7nHTE{x)V}g)#O^n3<;-_;}4n6B%o`0Ct?Y2ICORLv>G&f zicm9|J=k9)SdPaDO&BijkC_u)@idw_C!{er!E@p^2F9bt3L2|P1I=?{`m@U~EPoX3 z!J;JET5h|m&s;2dVBO zboJJixL6C?&uP&Yv9E2YxY=*h2E=WusG34>gA=(U-1H!OAtYR3Wrob|uthHc{WXe8 z7nDcyPUL`_?hb08L-3DU7nZkI2B)>A^6lvMBIz-*uDI-) zUWv!jOltK(*`x^8syP`0ISAvJ%v-o+OHGy{6o;lrug`qp0((1|T>q;7+w$$pEy{{p zJ_F#s8iyz69b9-GDjZUpW;qGFPn28Iodn4MQ~r4XJtoK9Af}eX)QzF?qtA8H=ZjnH z9==v0nJKlYi9>GZBJ*PblLtqWL1d6%CLxbcwXev-OJSaLB0i+a-h2e8(-+IS7}@pO z@46VnHm((nHQRe-=G!Xy!LWAO5+}j<{#HF^kCTR?j_WOnt`o~mkFp;-*>;ouFr@YTZs7IP9emYZn3XPF)*ZT3d(^S++WD0Yfh!4E)A zEOhLvScwEa->Vgy+^^$jgD~hR@~c`jR61PsdvE;_EUjl;qg(X}+Rr*@Yl)SX0=Hh> zjedy&aURTQwzXn6a?#}n58?qwL5WuY2KS26649mVb@Y*Fi22?3`l)3FGQE*zbS{1f zv>oIY1wCBQ8@PpGhutn?e{uQat=m9$c!zRsg~^#ijj`)S--hxcwc$R)tx&zNp>K_0 zc=?ZlWgbrwPzO)mLV6a^Z(ty@q-Fl9}=8- zEPZ7@-EZh#NaNVXvf5~p_s*Tb+uvG46!R$fczwPhuRnzk>FJg~$^;Zb&*7irNOckm(q5IeGIWi4f8U z-nrvDJw=^wkzL6JoP2nMdGIpv1uC8<=FXoz-*A6Gp@!4ly zN$g^d$+LNhu<>~Hh0OMRA-U7(2{Gs?ke4S`!VtnB8l*0RzS-@+mc~%iHJ5tS-#opo zuIrqDfIxvD`%*&f_Q_N6eM3g-%u$pTw~JSU`zjCCwx8n zmM;>Ru5B7#U4JNyBg=A;p^jUYEU(`HhKMuP)snTIfW+GiH>2F``jSmUji~);`=f5w zwDo$;`=wz`#Hjk-9h*Wm0yg=tlFS}4D|J^NXncmf7jtj|V;g)PT#M#QLLm91KN|2M z88NJUZSd#`y>chX+Y`#MSn{HYbWO!|*!M0V(qjpI$ld#O^EA{X+=&5saZE}~N4)rj zZ=-rF`Rl}+pdaifOEnjsQgD(bkN&RTKy!zfY<|<^mFs{>9{;a8ax@rZ@EsYXewsn2 z>m<*T4WxS8+ysdCDfD?+lV8G!b41l!1mqiV+zd*rVlZ*eF^m|Wks(4B;!5#eVqCu9!&p4=ql%(S`ZFdB& z!35#s(2dIixku*=iF~=WY*|wFV+U5c{d~MR{Y$>OW!{v+h&?nXf4ol4Mg*FO8ZJE5 zr8ZybNCzy4s3Zgkd+dkO7Skn4tR&dU?sEHKG*{#JBDPB{_AZePI;`!jR@B_7Py9ST zM36mR>C-03v9tA1#Be~n?G@IMM@nc*xv^LMy6?8p$-Y)qM@!&O`tq2lvqEx4qh)#R zjRrmRbt8aN($bV5T5OsA4ggT|oE^Fbm3@e&z21s!rRy5U8Z>R6D5snGd}A#C>_DmwgRgFbM0x;0 zN^iw_Nl^QEt@`#^pcAX|`z&%=Qg5gVsQ|LQJ&u8A=O%WIK+`6Et~#Lapsg>F#CI7q zW>zB*>KromxAqkS;yU_&5m{R_@^R`C^Es;#ZznO)_q6pl?h*hw6mqs(wV z?cZA-NmkM@nw-e{f%dBohT`oDfaAQ#Hqw8V7nlV+jqKmc2qGK6nUuoXLii)y6r@AQ z{Vhw>G(65b?c=WN@-4mX@)16zpZ{=AiU`;3WqCv1*LfO%M%`W^`O47_*oW?Bm9nxm z92ZO=?=Qzw-iP;E?W2PM>ZX2T^X`?mn&mg-XLUE(t`8*1iIh@%x`skB46)HkyfhyU zmdprzAJQ{*d#vj|%OG`Xp_Ccv6ZW&8c6qmM=S!mdRvilA9LD#hmd6e9iL|?qp0N91 zhIJ(^0=U50ORzO(;vMbyseUf;9wak9dhqnTa&(t=M~0;dnW4}eS0$Si#D|{95pF4&~(^OL$jDEW3HL=AjbhFYucnF&eB_q-uA}v zHUu`HxHt)$`Hy`NF$0%c?J@OlOAQ9ytXHXRZUWNNv+lFId!W*v!tTo{wpDL0nIjvz zz(0cghtWz*U+$YV8*Jq^6IcE0n^&Hbb4Jiik-r#n^7U+NfnFcKeh7<_j-nAPW04M{ zEwwS|El5%XKMjk+9!S_5nAI}Qm?L(7B=tOsFTxGp7?h|1LPCMW47$msez zK3>bbB#pnc?)u3<#)}o#=)s^cvKlT!XJw&Jf?#s=(4F3P$ecD`2~mSq3oZD0!KFy} zD&-X^GCo`X|r=UjEfwD%$4eYvNI7$2GTbFAYf|(QT3||2l&nd4k z%_Kk*S!7D=R_|XPFUjpUSdh#I?jb%aD5q0ZHzhLzELSsmRYV)vK2>jZI{~}UlbltP z+Mb*(Krz6dtg63>V`dEJFxAcS7hhy$8T3?&5W%(ItUY#!H_;k zogPc~pG=lbPmd94@6F~|G_EW7nAv^t`2B)2s*N&zN$uy)pOMztOLKEg&r3>%EsBG* zU%q&uh}ji&c71$s&NQUF2chP6t*|ktyt(`oiI@7e`lShIs@(Kk1UCu*JBIK3&UN2K zo*mwNQ1E5qfb5bqi-k6>@>E9(WRJ{yphSWMPEN3Mx0!-ejb5)rcVgG9^EVi%+!C1+ z=;J9isBeFBju{HHeHkMhfJ}4uv-*aR9O8`zTFDP-p&B`nesYDGgNVmtV)boKWVklN zgTf!TW-qhEy)vx+Q_m~FGRwn9oO+B|oUBC7_lGw?Wu^`R68S0NwHTb(1GlZpD&}4) zOtbuAy`nrd4mqjMp^_@Eqam$DAs5}y-k}CcB~qhx&4!;2vKKk3&yyy<9sza>So_7c|h7PDJ^v=k2S(59x@f>>q+g?|v*jTBXtMt5U4vp)O zhGr)*YWft@ZoedVKDzm~&$;t^WR`e?;{+bf-93tC%LBB9B1`*P@>NjIJ-n}O>)NB$ zihda1h!glRhdZ~aw=Ya!?C$j-$csGP5Hp^V6?tY*jlYvZ#N7}y!pC9QgI<1R!oB>; z*{wkpBH;{_FqDM+>?D~wcv#U4zxRFE`T9u4UMy7GK3Ox#@zR;T7}X(x^jLuCdWX;B z-1>oL)duMkAD&Lpntkg^PqfaPS*dq+>qv7OAKqh>W~PJOE3G7H9%Mguu0ErGDs!>} zuUwwPD;GFXm3?rYlcVaBWsU!f_LLPZIhRuW33%@^DIh^!ds8P)og2R2tW0^K z2E4JfrY_mjs*a}ei)b@188m-o8Atl5dhhuUy!kphV!m6Shj&kgJ=9Da3rb@U_~i0q z&#LgrI!pDXwHn26IAnAigF6cB+;mnr1eicI(9hS;QMaAM?KF*jkKD6i& z;?B;wi|Q$gUm6Et9v5Rxri!C*=jUjHffP;WT%QfDg{z!udEMARqq$>QvPD_FOV`b( zUJ;UA+%7WHjc@0bGfTzJxJCUOwzjF2O4P7&UE}-XuuBE%wnnQmKYo1iy@1mssd#1- zT)YuTuPK?gHf`EmtxWmQlsQ*p8$)K*8INGS@JNVPJy6;VpD4zfwh(foK&~Eh4g4_x zzb9*p-i0oF`irVURqi}i$h~|S^5V}nO_vbfB99rvS$A}m-Pd+2_mg%L5Lzee++H=q zxna$$WP1T6Z@l}y*ihO0b;+?A6-x3@NmmigAax*1Q*})L(0R0@TR-8|(`l{x2L=oT zuC8pNlWe0lc^OG-Use3*?11ZC}y>l-JYvlht1 zC_hpH=hXYxmgu1QX|TRt0?wPv)w4mTM$GbAVazjDC}`6>W9i*E?zS_17xDjynV9U=vz9}jyG@uh`?bJ`8h1|g zUb+UK_3$LWO5ah0RBiVWGa8P#9bFbcI1kIx(YGgKFw$LMEwSo1Joz?Bk0z*;N}4W7 z*hTrGXRlm<^=rSbJl@^;+epfb7Fm5oC4W&J93}(pk{32VmnMSb!jxIW=~EO;8fKq* z6Sx*H5lO{&Hzu%8(@b0ndM`AtL@beF_FS^XvbU0vx^>ve1_u6iL#gfnT~XWA%S;a*iXUhlk|HY&7$~OS(lKpXAnh`0e7glY5{icTXF}XU3w);#Ly@x z?C@&`UZd8L;JYL$mv-+3ZPgL;H|jjA`BP|e5QxkrCOc$)9lu6zAYmgG5@xGA+fe8+ z#+P9_o%8i2KD%uC@OOY03W1{Sf3YsAqRVga61i=!d}Vuh?n+s5Gkdn9%_eSULtiz_ zDt>E6WlZLF%kAPKt6JOUzySO#W;*y=`0})g$z<;}JHe zsl5#rPCbEXj^1VK{MM?x!=O^Y2H>woHa3iti?v z9}gTIR!mIlLnjEoCG1HCJZ*47bf8b}zHT4&5UKbv$((4v+q2yt>xLF;cAhp!!{Vkd z!0Q+lf7Zu5P%x^mGnVI%%%!JEq)f+k)%ns)^HL0Dg#R>s_0&#=6M@>4WKm)TDllD~gi#zk8qQ99KSdHfutM(m1uyr%wXZ2xaSI}F1zwnY{=W{pVMDg`i z^82+Im;p)y(RdV`N-)M8mVV1%u&|_3-*NnD&_`XEE@s=QagoUU+69bfG_* zFLJitZ6O}ci1TO|@%u<v!bU92=6H|2(appy?|EO;bK^rSy-8?`v_e3&kw zsT~hQ4)FiZ$~l!O-p)3&E+U=;8= z?MSv~FG27!uVKSj?5mq_Ix+hy;mh&Bv1B-9R<9xaR=;0h5EiUDu0gE0Hh+i~=;VVZ zZ1%Jz)ZzB*v;QSd)dsWXDt#(BcxYAY41ySd9N-nzkdSaj+BU4n{WfXK`nyS&Pp`a# z!KHh)f3e$~oIy=qpM$R#m8|Qa`6Ga_-yAk|BsWb@#&PhoN0BjMFVCj_F&+=_I zja?3GlV8cYeMIMK_R?0D47kd5qEHupe42_wV?K}fkJBm6HhzWjjO>;5D>v(^hA?w! z>4ux}mXg7m^_+l5PNuz|<)sPsyjDEfEY_f`gBc!c5et>471K-w>$iQZRqM5ZQz>G^ zQlgjKjXJ&afZ_#>w@|WlxANKufw<|7ip7s_*3_2L?^|ee19YTdGCOs?P6H$9{Ghsd zJhg%wJ$}Eb;G@xz*r}g>Up4TmmcAIAF#G5$E^Mr`H>%|u+d6yu)O{XrN;{Qh2`{`Nug$k7O{Lp?EBUjM$*E3fU8LF{1W zA%Cwv{9qf9`l~R9_vWvOmJ}E?7eu2A=GdZhi*$QNBSXfPRYbVf&qTbsvz=`hvW{2s z;f;-CFN~FGanTMNJYADV07W75Su;mU*^ThX=*foE^6sWfx7~UF*rfVgN8YDQNxPK9 zBQIZ|8_(HF$5l79O=oPEz~9}qZ=YzpV0sBoVJV9ZIl1Y|LG34LUrQU$IM;n_NcB!~ z(A02F$08wp#IX!=<0_Sxe?e+{wGGg-D-*S{U-PskbfH7X8C#Zty_8)+_QJJqB%k=K z6l-oJJq-J(*AooGFeVR24Z2{j@02R7CTfp$#do_c%F}INqA7be_9! zR8=Pvo)?rgUIqYi#Bt-;WS!JFNvm@|VxqIFxYn`1Bm=H@{2e39LQ*G;y#)%fEOXd% zP=CQ0t{u!4=ro)ff#$PIzm;brUnx9OvVFK||6nPl;DDh^gG>C$3AeaQ?Imi>hAa*8 z)s-(BD6{)D9f3R&>C%cwV&`>q9R)x*W9*FRi@GAs6cR zZPKEVWZ?<(3gxUNnakiqyM4f()s3*zE*C^m9fp|~{SO%&T&3{F9Z3@6oNx8lxjH{p zxaIF%&li#t7%jI(S_9o5X8O{d+8i2ZhQXoa;11Jdnbthra>ZudUZGG_gqe?L*WEcF zW7N;z>ZCO1J)wN=a7u>HHlu#(T@*M`rSqQsH4pG_Iu>vn&Re+hkJhx}&vNlzOTZT- z^SUZck$EX@yWjY*@x;4vA1qWq0^>qiCstR-?-bykuX`h(AYM@Nsvl+_zUg_J*1A`c z1N*7jY%mLN3Xwuq%3?DAA&i5!;&4-zhPT>WB+w**zolU0YhzVfJg>4|M4*#;;9x&v%iPaMg_!$JhipI+&>A*293ic+&Jjb3RXf-XdVbnJRF3yec2s5iO{Lrl+4w(i z>-U14J?p0t6)c7D5WLG1F1Tw7dm?V;AVQx?H3=d9%%AC_QKOGeIs?|MW5M!RrdT=*5Ibp=jN!UF(m%GvgbeLkH4bq{wD@AZ(@az2*m#(2gyE& zgKHLev;VrV!xg4~>y&=Ul>QQs!pT|rWlnnDg!={e58%B}%xk_s7v}!W|36*d{hQzvVI)%A-vZD3I40XJvm2A0{&T(2|0%Hm z9|$9nq5VU01SS$eB76EEMvmpTCIl5E`nF{g><2d;_?J`;j%FEw6@PN{;GZ01nN}yt zma{V1yTiFomb*iN&tLq9&9HSnCjTG{v`(3AdKug?Q$`JNwM+D znw`v{yXL(*;j7^;La<=D&EtLFThahD^m`U~=_-J`P$vrEVeWgm6)Zs$5!HZ7hZLPX zvDg8ff$}VpVE9fEcV$@J529;Z_6Bv=;k+0NQ<;M;?dpzoogf`u7;e5;FsMk)eE2A= zG*(y0o)8nW&JV~LVhtxc!`t_b?SZR2cu?(>cjj$ZAK~ukbbH;+eRrGg6lfE2I;+5R zltiw%;d}oA8rOO~v0LkHQ?p+J42q6GV{Q3;E`&u|Xxh2|*t=Nn%PRpylqf?joerz6 zPnUglhZ*cSOMI^L)8;(f*UhI#!VgMX4nakqJML9Q0AWq4S|kc#_w?ltX&Ml+Zt_sEBKKO~VW zlPxim^Ex`z5$CLm zTELw6SUz#VbB=TYfko`f=ut{ z%KV9V++rpwqYkJ~5d7QbsI60kLp)@qOzlpoKifD=58d(HQ_qGMU|c$G^O+ZXxofHq z-X*PAj1uPzPQ8fTrtM1SDFH%!QsAxWm28g&0o^5$iHsdlmXxwj0^|+C1*lQ+up<-} zr9MW{n)X@zqm4=Fe&0RcJ=`d8c}V=&tz-uCPF_Eh%Vv`-tbS;aL14&DUVLqkcc>Ph zhQ(-WPLD)H3;p?sEVOn|mO=1BmyhN0%YJ!?-oE&g+@@091B<%S+iH%EGN2-x>*Bb5bsPe5+MJ81`G*U zV|UOc%d?4#-IubWMk9Rn_6WgWG(^l~qndgRFD9wmb-~ky)=RFR-5)vNdfdo#6&oR@ zOBf5P@-WYmJFgy(Cr*vVLX1`#^|I>UrwJ^UF<(ecg(}-i2Is)Q0U40jhRtUJvbuP}-eaRkq z;R8K|c1;_8ll@y9yW4dNI@JN0pNj?S4g?liufl8Q#Kw>!&eV;=YnPn(lF-|7vb*3> z2K+P32DROU7^D6^H!-_5&FD%kQxVhWJ$`0mA3uhe!;&@!lzPm)t`Viqe!j^ZBn}du z3P&z7&a@;0KVQiaD8D?;teUk#b@lC^hUV>Z z3}&0(r(mi8#{dOf93(bGm&fgMm(+VXRkW+iN4Ri&u8@Nm#R+rl*8E` zFCvuSv;u|A%{8t0-ib0w7H6)4f~>_K^MKZ;!M2$wSLg4JPzw8HO|C#SIOEhI=jZ6? zX;4W}l~jYZ*IS3g=%h)(y+Y&~~vjKp<`t<}hm(cz(1ytHIEnm*c)dBk@&TO_+h zN1w9paDheEIrX@`t}CMFI0)^+U=Oq-BI1Vu1tyo$bmwxRE5~{R$%lA4pi6--@GfTj z-swp%Y2)p+Ce3JUxMq%hX zL$xNQ#s{Mvo!$W71UpC9X?Rno;_s%;A8)fafA@6YCE~5MNJZWO*5d7%K-tagu=x)8 zrfLBzafh8^njbzop*!;~+SBPM!tMnBo7x7a=>iOLQHhWPETGNEQwXfLjXM~F{Y&9$Q z^)ovrB2;uj0kDLFYvxPJr7JU7qZgjM!`*Pqz)E`(v5TlZ%+`jOM@vAd90iMlDVMj; zyc*7^Ovi0i*Axa7sO-3)zfI_g^k>}Re5!S@hP0A2vYNc z;imKY0jDa*aBN>Xp5PT}dD4Wm=V@V$;h*M~V^SPrrHTm0y9Ey|UyHdN{$OtquF>js zM|d6GRj!qKxK9pJwK#uETvC4cjb-730A`6PX=~jN^Zf`@dEq_brABf_>wVNN(K^2s zDQsP@#TzntaYsiKZn6205wVXw)bV6)bA( z2b@WXU0aJbj4BxlG{!=l3L+MYiRw9vI!4{s5&cPeuQQ6bHuE72^$>Zw+a7ETBCO3GQ=r#;cmg>5$ptkZ zCdCYO?mVX1>|SnsjWTbfH~Vva=8GZl&OZ2~cS+gKI&W-P{+c`|iU!c;)t}ry>cK8N zdAL#5=Hv5HO_V_F&jJ(Z?-gQn^Nn3FA* z1h2?z;9J|Xmt=TmKWVN}uFP@0&ALR_{ks#})fO`O#c8;X7!jlQ=`w!#3u+(`eUN5w zRkxrN^D$xK4IhBdkYU6rX9>UIFsihGC&zp||8*O=aE2h5Q)O&=MIiUkFOO}C9IiB$ zXMb-b)oKq}r9VoW6=!EVarjWk@gel_qso2R)S>j_X1)ZDa;*kK{ym_3-e$_U@pO`) zFXcmYLeG#w+IT1MvIbdZ^}7Vr3Ra%ntrp!B0MSo`Tt+=N!{2d-{}^DUpnQ&Lfcc&dx zEMiGpp;_O(EI=fUnBcT^h3UQpZxo%cF*owCe;9}it=u^tWr9E==fsK_ z13BDT86^awSyJ6ZpbrotcByrr*$-2eSIvr{PM@dGM(QHmYiEnrlCy{KTW^5|_VGuA zx#x_g!u#IQON~8x+vnC>g7&$k86?WX@u`VDa;*Cn*fd9?b$|Z48=-VUia2~w&`wBxGUK4 z3|Lh@x=uTC>#F6B)Hw8rL@e@cZQ`$S;T;mOY9M5AE!b5!n-MFIm=2lS$Y@G*U$yH` zCb( zIt8^s^i%h>V_B1YF+~TrOPbX#cQPCGf@qM#zS+JUi7Tax0h!9^Xg1IN6gC)bPU&P0 z>5I)SaG$Qj6}uQ?R?taRl*!#D0(7(<66S56zfpoA`U0V4AZ63Oe1#V?`khm=4Qe*zB&NOF zVvfsIk-7{fV~u@VV-%*-nuQ|xasxd&b@_938r7JOai78Wax8o=Q}SII+-|*b@6fqv z8v6%9c0FC&@YoL@m^FXbhP?E=+GnwH!&~(g|8YS88;@=>MS1*4*Zl-< z2yIxU>F1(I3f}B$5czoMTLXZ_n=69Iu9!OXb#Y@6I{}i38EmXh(L&9e^6YLg`S>xr zR=E{>0l$fZjSf_&!qu9^=aHHUoQG6S4OxXt!L@DUb9DKDLOBW$d;n<8*WEFaayKs6 zdgqRA)#nZ!TY4s*FN;mss;BtWp%8P8i{V~zMAz8JQ&p&S{jL2xC(~j(jL8GD0^@Fv&uyCEXJ6~OrL0!e?Fx2PL+HP5I zX%~J9Kc3M=H?-+WeZ=WRTtxMVZw3}J)PtO^t`nULqCnyM7q~O=yCS9d+WrG-0GNtRJZ;`P2G!U+y?;t zM8jb39e*t~#=fAS*#js2xBVfili?G-6smOS6p@+URX?uWpS?+_HfOj&M`dgdUlehp zZ)%s1{W{yf6{>aIGVRpLPwm1wwn3{3zo*!R(4nK;*an*QHp2sIaA`!nxvR9?+krJo z=7E2zXP_{RL!GO8czM=ChQC3Bv|u^tuO1xE-8Zl1G}_F3dbm&M1K@MlX@hGt+y+}%FnDl8JNvk#xL61l6K>Twa({F&t+R&)KOtV}UU zJn-jW1)%EH(8LwFz=_Qw8YwIH{!wj$k||_Pu=TQTpNpT;-*j=u^rz?oBvZ~1BIftv z|1@$HZc%mJcW4BZl$4SZMY>z0Mp`-tL_ip10BL~>h;&Fx3?N8@bPFRPBi+&v+HC_Xphb+`aZ%=j^@iJ=??o!jtK8a?5Um?BzRn8`wVG$yU;G+Cl_2tQvbg^5*Q_xLn{ru8scu{q@TZ#<@mTcRV4{dwGQ`Q@}H2 zB1o%jdLDzUU7HQ?tF6$?!1rs$=h>S*Y4`NvWK-LJ`a*)wKckO}8MfGChZ&Yc-?5;b z`>>Jak6a82sX+}#Mrt|QY%RKm>TuRbS!dE|b{MNoCVLR6M1C=w5Q$Bi8GYm*YxT=N zwPNl#*L&Ni9_F^$t-mc+b*73R3OwJRa+CB!*Gzbl5RINxyBi9I%c0go6AvmM^~iR5 zN+nFCevREF^_s7>WXEUmPeS2L^B;v{1lIh!Z%o&y3c@eZuLtsP%b{B-hhifResMCF zi;xViH<<|>*k8$3p|*xfa>LV7F0Qxdyp8lu8^7)E_L1KsyZb?dY>EvZ4m5zAT-wal z#$TnEy>@UA29JNt<7@<*mUm6oY;140RBpV^WRGp}c8cfRucT+QRkD64Fk-1%K&FF> zv@)c^)uCIv60rNl-%g@Dw9$t-C$(MU+1>S9=n#?{IFEyswVUBB&qjsjjW{AY3J-)b z%@@3L_Kk|@sz#i}n4Fg90ETsPJTpl(a9ns$`7`c^qN5T?u%*oYt;T}O^(*xO)umMh zN&~gK$@yw1uj-G)PdWF0#2Znmo8L?7IhTGY#rex@9MJ-J#EcK~y4I^`t|vgNf~F7oCI55fO7H$XU@nu@E#{7O_&cebJ5XD0NE$VmaKbRm>p`K zBoKEWglw)gn(G=`jBYdW&^Hd`x9u}zCY5-FD>v1kBQ$?PZ$DG_PDaVjeyd8^4Ee}m z&DDQK<0|d3_NVTJ$L4E%n~G{ghaxf(&FPt___AnCdj^5xVZ zRLc1T6A0bT=zSXsX|?D}hph;l*quPrO8oQ^iyO7am3T9L134*-?mc4LD>DcCxb6aN zKdb%1U>(S-G{hbDoc2gn$!7LN@|}wBE zdzu)B8Ow@&eN8Hhn{!_wkXqqgV0EogF^ddQVR1D8rXyEoKlVKeHS{ho4gV~aX{%8% z^OFRZo<-P;Mf<6e&26$fMwNP#h_mY5+|&z0i(lWd`FYJcJ$EwOE;@jliVYkSrkqH);^ELC1K<^GvB=nh-+Zsk4(My z80lAoi^V29@+8_id0h_$NyNLh7(5QVbLE7XOnaYiNAX?W;%{IkO&Ji;))GRR$Ec7l z!F|fstq^K2M`5l-Vb9hH)NF<9j74D5l~2uq6p{zi9n06JBDb&Tq%V6@I~~NxhqAuq zem_Ct)vZ+8m}jmjQSyD6l1E2G&z7v^pGe=Moa(3HRY+(?*RSs|!(Y_elsBt1nr}E- z32+4jxa100468JiZ-+TcIw?20g-6b9Tkf$01y*lwJ;oEN?Y;6!?VURj3{W#ed2oFL z0=%fp{DYM?W8u(F^RKA$=7Q4jPTm8tY=ZN*)JQnj&Ok?Mt}L`d&ob+CwSjdJmP*Ff z38L!_^JMuGz-;Qfh^kF33Yz6CIA0sDzbpcc`a~vhB`O*{5w*%EY<)Aar%=um8Crgs znvqaOYbf1zpr<`Op;Du5@HV8X9JLRKw(LUHr@f%|$ucV?ip!e3*mcFQ*MmM`9-*3hU4*kW`fA8%$_d@HLkmg+N$8Fzy|90MCk)! zIXkU!4IY35w)kst85E4`c}$q08iJH>&0CJsyVBe(oNms#;3VXac`;qMi2KujAlzqL z3bh5S7$)!UtJn&TBez=K%43+1@#87(XBUPb{4Q>f?%sQny-FDRsBK@#4mte{qV2Zj zt4v=!GhtAL$k(K;xo3CXY7!-+U_T(HgS)*|_z^g5?L1^Qs>tVEo+uf4C38eaFn}Gt zAN6}DazYm~bR(Qk77SX0wW54gl-sp)7)W*OWcr%V>1O4q?-U&~CDW=9+SapCd=Srg6{G9& z8%8yYHt(V@9?!&PZNn*&+^gF{mrL|HX_Uq{hv^R*ar0I(!!|Yc4Zk|fV<-^%Pz)eZ zTPLRrmC~2;wL>hm;4GhSj5!ygGqkpEY5JPcc7BgXjWHjNdEpU|Vw1pqjJtx2F4AK5 zkQ!1CUfr^~Lf5h@a%)e?TkMdp-C=n8;O+^sSN|a`a3;Rh#k71^BYHN!C@%4{YT?u_ zrv)o}(Ya~i8Z3$v(4}B8NkC}1B>{*r3m2wm9poO&&Ymd=fmXw;+Iq3v*MxS_xr@6e z#~;J!dz|wQMP5yG$IVjf_deMcA?{p2C+?oN+iadhjRUT4nTF#? z41YN$g}*aD2{=3fcP20UHkBuv+6IrTgw-#nt=TfSb!51z;WO&Ixr8mH^5u7Xm4gyYjTNFoqr`sMrMw$R0Imnafv0^LG!Nh^E*M zCk^!acNQFN>(Rv&LeVyK_%~Nszp}&!&2f3#9oo*j_?m^YL!a>&5?Oa$2+MkXSGa=)I(^kh3`;z{g_2h zK*&zO<(k1a^JeOvgHsdXvqAO_5LRk}>H?G#aX_32Y(~b2k?;!wBP^DnYFp)ldYSQ$ z^HV)Ij9e-x6!^r2E;Yca{(V#XwsehcGunq`l=>yb$gx#h1(hzk$!8N^f{6X-iBfX8b*^9o1d zi-dp5c|p40kX-86q`{puhsnzl4aV~nao`;)ZVsMRp6Grw)w-%%h)49hkl4@4a?#_@4wRVI4vXyrOKJ5# zxT3S1TFm`$rxcU(nO{KtvG}C(jOFbD;!0ic@tGT|t}e&W^(JNfgk4>(pnOx_yONJW>JGj^gb@*`eY0^O#!DB>TE8O^Fr;m3u2C-vG3-pMv}5M!EGw3SB)7L}ODh(;N7Id7Y5 zP<(Y)vEX8EDG_0vQf{r`Bc324K$_-xhVOdk#})?$#LJKZm1bg-6ElkRr6WjLtM)E~&JD0rBS&Te;@ z^M1BJ*xukhc-rpqN zdV94f3wCzO;70~m<@`%IT)tmfeEQ)Q7YN%rm0vVih)Yvv#x2}{;Q`hsb}f@19iDmQ zr33>h``raL!gtgy*J`|KM#0e>q1}(|gFe~61Q;T`j}IT3P;$M28Sr2YG!wN%Z6sKa zjd8R06LlP&AgM$=+_iC3;mKf!56mvs&-diKgGSKAHJcd9Ov&i+(?OPD|7E;)@uRd) zL{zlkxsCdtRu{r2w~C^&({c`k%qr02lCF{q&5fz;2b}o%F9n5c zKu5&ZeIHpfcpn!uEeaZ3%P)=4>`#)8e?+VBJN1zbWEFqTX~lcwkNch8xYb91?%i=f zW*fJC)LwbP*9(%{Wul1zOBu2{AP|8a7`VB*udUqQ@~)Qn12u*on%u%N>ZT)I`59NCd`wAigq|5M$? zRSb1`a7F{wHi0w~QL-FkBQ5wCTZ^zPh8TmZ!< z&}E3AgBK+vy?7hR=tnGP?D+^isGcDSWwAHFA621!Ae5v#IuZh?3Qj#r`QUS)7E#Pg zDa#5^Y@47*1b&`hv1Jatr1gJu&!|USXJSM)M5|Pt%WOBpIL7GdQ3H=)NHM}Ah2VF1 z!#Q<`82ne3{(9rfW3~q#(x-C5u|N$7qj|)7>!o-!GMIU`!*7}3*iriE$>*3zoL+Fx z7T=J=)48r#<5XF~mYIE3jw%Gk3$o=M{F$9h+oFBAiMsTvEh^KK>y*<}+_@&X*UI{1 zCavo4FyR6}TH{ul*qJWf6nZFU@zAqV<g|joo{qYS8?np4b!K7Fwz9_Sg~h_{Lz@+U6J_f$Duccgq?v!r+E4%Bn2dpGj9p>u zo#4Q^Bjj%J*)3V{*_O#G-9wCvO7tt&k{Pe#T}fc#)q!VcgScDT#qsDGAK9dmmc`m^0;2$N z6$FWojoT%-Z!6amTZ|D67JP_NDNWsJFzxogy$SPZ!Y^|*dlyr+H4d<4Y#!v4teHDY z*DGe`HW&#dpW1gl?~UVoGVQDL))*r`janA=Cy$-8W>+M|{hp9HCZOr_^g-_0Y`2dF zWWs?*LVZU-!%=ehsH!r}V(mm4CBrh%NKgt#_G6`G<3as)U5=rnN~vMWwwVfYwY1 z8)AkC1YS6bHS$|*+W?gszL%sLtuiY)vhXi)Y(YL1v0p+=Y7Q$AYvnlMg?sno2YA6= zowY+Ebw)B=q0;NB!3{y53&>hKkq_q+IMaj46lxN#HdbE-y#Foq+@eL}{==_fpR3g< zn1&gom-l`Cxw){Z0u!=I#~*3&apCYNX2l(Bz6L0C7M0lH=J9)B^KRB$aQ5`~1PMn} z?I~wQ-+NyskSTbvZ)90$;v0p@vaOEK{KFIQTkQH*Pavh6lQnoxbwsWugQSAYPr0Y0 z9<2C2B`)!g;t1Vy40nrHr)E+g(Itc|S>NSL&`=;;ys3XeeI)2CS>5qSn)Pgf3n5>-*e0}69_)X!6j=OxGll-oQ;>Z-ot=I+&wEqaW z%Giq^ti|dZ9;^RVUGx@fuuA-Yg~g?1Bk+x1l?_AuaJw}a_-o&VIdk*lNmC#khJFVA z+Z)ZY-1hl4TaupW`0p{q{mBECcYPS|Z6;~_vv2aB7y*3$s34`~iWLJd6{eN=Els^vS@1|r= zY`B$)!neCi(1q%7qQAR5DJmO(4&!2+W0PSB?xyC(Jvp(un%4v~&nK8S4?8H9O#jXw z_#Zj`6MVcX@Jqfx9Dp;BdEWefI`O5u9KO1Ft9icWKN0YMF3=$A*2Ek_I;4+Xqa~^I z+ig8b^b-45);28Z&vTUcFEsON5C!16U@);?AZ@-h~ZJ=oSd9 Mq^bBt!7}Lo0JXt-xBvhE delta 23662 zcmafacOcdK|9`052+1BHd&^#jlWaoB$|^G>>)7MGE0w*+F%GB9j0njlJ3FInGLO9u zj>Ez4=-%$<-tYSP3-9;q^?L60dc2bduf_LXd;SXV6Mn%}00b)B+gLcFxq1&jv>5MZ z^A+8v&g?h&ik^oPD?iY^%tx)z%>VqXliEX_FbtR?6`zcj2i2NC8L+ru#vL}&^ zwNIY=r0Az)GgXuX#ea@^bSV}h0!x?AP`kH0ARcJL$_lU^-a&^ffj-ao&6l23#n?M< zMB&Zo3fp4~G@_4b740Fl>ZZsu6fn04e+So?49tsBQc`M5#t}#Q8gXv9JDM-JHc-IV zan(1ZVWdmo&i+Ujk($y@%h#05uNr=XQ@+k?RmDJZvyYL;rleYYc{zLwx{P}eX5TF zfP;CL?6WWGBL(xAu+UI7hx~FqZtO-^yuH+%vlJCcIx%^l)7F>!YFhak6Fy%b_cC;j zL4S_UZ&ykc3C2v1h02yjvJ_{g>EIYOs67AJywAbqYwsoBl#GocH0pe_TMCva`0gWx znF!!d80cUtp^PcRJ*ieBgeknhv?%SNj%*l8$fbu$x*t<^0v-lO6DwN23cjgDwn;duGh zvBsN`@dn0jBgoN49gnqe;bSv_;56;;A3+7NJj}>=?=kL!3cILv5A`~K_++=ju!@&M z;Sw6>ESJ7^$@Yc49A=bq{CN7Mbfa5BY7O7vQb${v( z?9;+{F7yWo%MYDg1NA!%y|J5%s>EjE9H2*tODER!;a!!jCs6rvM20k>)Mmc&i0JW6 zPA_Jyki5a98P#WXkM+<<+Yxh#I2q{^jniW7R=bgB42l`cNIkAB9wo64FW4w?cKr^^ zs7jIPQYG48FTcT7HOAS{e}`*H>3WoKh!(IEDT) zGd*{&HK}whwE1MH3(dHbUrmAv`F0M!^&wfr_;x;q9cPJDMj-%*vDf8>=jAPp0#Rut-&<|`kX{5WzYUjM3>cgfo=9nJSA0BRKH zhj}q-N=iB2IORUvw}m&pCCj&RnfDxqI39;OQi*5WvpIhfj1akA4wY`-t=C^%*-9jAhOc?+wCER;NwScKJ%Z%*eCe2NA=dTG4g?$%M!S7B(=5e z?+UVddw3fw(tjt*_d6OlbMppF1)tJuRcAZ;XiPpiD^5sI$p{8F@80K@QptE}iCfjr zD0}^5sPQY#;xsqjEFP~l$wklB<^EjMBhR0n4Ohj3j4PAVv~RzC9$vts(B77q$?9!Y z5l*(dI>DdD%Bql!8J-vW!g@PYD*LY9>K2~v)eqh`1PW1P6f{Pa|#shYr)9L@o6ajI)J}T zN2X}?p>i61+X$?Y9G{j#gtzpuzEb-={EHPy|5-uydqocZYH^_aZhE5$~pYK@mD%E z`$@6?Y(Vl`(C}u@&w>AbSB)g6!Zuy|Z#GDNv*CCrLtqs4gbe={S*%{zFM9P_SDjPy z5@5BY_zaYX3Xe8oP3_vbmue4jTCzr_ucBg2{HZ~Bc{GPH(0%kT2HcmLZg`4jua zXFSUv@VAHm`9P1i?3carC-;m0@GLiGY_$MeCVztN$Gn{zpgV&Ilg$)Z=ge(_8-*e`0@o;q?Fb z|J;W)t2xHuf7&_lTfW>GJr7s&60CUsFAw@<1X-S>?_0FE>e}5;GB?uXFXOVn?;qiJ zr~wC);gH9?TTFVd)3m*vq;MtC-#==P8&9)kYObiH%5jLNw~gSW!13FLoOtB^yR%*# zRlIZ50E?r*zjfoc!RpEoKD6+yPlMf{#AkH*?SPSbGH=6QxiQ^C4N~aV)#EoRnUCTv8Cst$IS6rT$G z&xhZy#iVIl3G#aXJIHbLA^sKQG?336XEbvRnCh=Lqt)cf?x zV0UEjsU2{P-c!(}BE~UFxv}$pm0eu%wcxM7NRe((pN31MdAZUf|HumbN~??y-nV4u zCBXguNr)RR+WrrO+`EM%WU*KG&*LcmAy?15i%ni-!am|0mg=`?Lx-f_ie9W?_xy`^ z=YJ6Y{h{xOXZ%n7vWPGqIpvKuJE~Hp?6 z6#0s$D=FRAguHKec&_|U$xYnENiHX_k|Vu6`LE>uzc?~M;j;@8x(z4GO`Mhgxve@1 z=1Plyh4x?j(qHWRX(Fx9iuk{Y==e zf~WwnKP$TP^;4rTW)Vm4^7RPXErfjH(Tz=B+98^zj*5ti49icOojJQfVS;FL`#YJR zy<%VM=T+=HM-{!tTSx)DHRxl~lXhG~4_pUJGa4v%TA8Hs`FZaViChY`**PMl9pP*{ zN%y#l=CZ;+mJIw=JmCDQm@>0gJm``h5Ug$e!I@g~LeEjxcpLA;u$i0~qgPEqX9Gza zqW7wW5+vbDN*fxe{6BSc{)P-!Qe8MtEY*cR%}oRn+Vy_vsNb?Q@{}*-&`)JYl5u`m z++IvssaNOs!aq|!&MjE@LR5R!$D<5HP52dn3yp~z@472uw&fZ%+o{|aJ6$mYaD$Dn zK2YShiOcb>OiB-^lZG?2SUIU$PlvgF>?^2|{Fegd!V*bwX3!{RZ^fOUCCo^X}a3_G0nsqzb4RW z#Heq_L16TSaNSH~n0qbTYvokA@30i>np&M$_AqP6DRyJvsaltmJccb@IOZ$R#NgeK@!B>#lR^Ii(&yFv6-E}G70=+xRaof%H`9jKaH1E4QIyFseiStqjfZ`3K0 zD;S=0yQE7cGdwzScG~vH40I>y?{)0_*o6k%p6UDCF|*!@lFjEnpDOHSLFOOSpN#v* zi+b?fxuib_pICdSX5X~MKx0o9=x=gcHe`|m zRFa9(Y^)6s*5cdGWXw9ADVDO9Bq0>`j4a(pg_jQn;m6(flg++%j|$9qQxR{@XCP1! zHGd=m8=#1HiF|s?W-a5^u6!{54=fVVtM0ABpVetz@2wg#kD#`5SqjF?mh)`hsCnJL zEzL>~>Gy*T0cv1G;2)#|yYGKM-r)mn4k-w!hBfkhqtiBr4?6I?fVP77a49b`D>(e* zQTwr+D%@*mFR+9GO3~Eh%k1j^{b85r>zj5uKfxvTXl0UWgfMwe%5l>zMjp=_Ga_kc zK9N83BFylFwc|5ChLagV2S`V;bD>Y~7rpPl&Irwzh45_C88m_sa*d_yTwMUR)1Ik% z4|Y(G6eVlBl0vA_$(FVyd42xyaOv_8&P=-70~gbocN;Nt6-~vwTRJ(^BI~ZGN*Jyz zzp#4k3#(t-NbJ*FzliEN-VX!kx?0KsUn??Heq&$xX(m~K2)C5?bLPgg#C%D15$wCV z>O;nm*2_7kgD=Td$a10S@A#tH9(RyN z^wsp*D#!j7X|;(6?IUm3A5cey4@zZceL6J^xY3wg~ZqQmxWFh2Nj=e z2uPG{b_d6hv3@33&@DKv4846ewX`uq9NRn*MOf2}v^t7@#ctGkjZxTq)qKirPxPw% ziN=I%B8^Jt!;dvL)tIh9EbUcV-0KyAc*`NE&0I0U7*~8CRg)`)f;YTSekTo^i~bEh zd=+Qj&x3Vtz$QB^l;)MJ6{*4cr~b%ZqEzg4yqNHL{kDZciWa3~UZO z*>~luI`PODYdz7QwKJ(b$2ef$D7}&Mq#0F_y+kMsxMzfo#j452Hu1!@&D{z^Gd8}o zk>ISW`X1RRr%BIK6UMx^zp)q(a@!UF!y0&;QV*(b-R=%yaqC;?=?NVcEs;7XwSpfX zwK)V{nHS^!!@XO}?91g+zwd8#~gwV7&G>sfjY8oM(Ie*imIC%s9L+nJ~@4n`YD z-;NpF#`1=}CuCuprpXOO6al`#BRb4(Up}ZpIPM8qEh<|n3C*R~GZH_mY}6&I<0 zx+U9Q#w00;=5^vDor{oPnKMs}`igZ2cuTBNF?c;~r+T0(XGH=Ls7ZZ~ z5OOLeJ4Kk-oJoN9*~B2V$hYiaYgf8Ee;6G^GC)kC-1sQZmEbE!eqnXFsf3WE|0&b{;hfF+7h^E)fo02_(?BNc{>oLfBysK15W( z(yQZ3MT>w(tD4AU7b!mkmGm{@l(LeM(;&fuz>RUYJ%1-^V(_E}^nh%*Q`pGKDeE{XU>%N$?1aE(<7|Jh#E@+FqaCC$Qe`3NeCUfh@O&udR?u{mEnAX!I#Inv zW)nxM4Vp&4Dil0=I*}2GZ9JLxXE%~Nk`m3=S&UMbgo%Ym@lyi=S|qVf;v6vt7u#_k z?NH>)Gr2!9jgo4hE0nrrG4M$Q`5km~S@3oletH*=ixi&JA5GPSr~Ox?zGfBVGHB0) zkDZ6oJ-nZ)r}zhQH^{tEVZy}xN3(;Q0Q;u$LcWnIs2C4V}w7jgWt-WP!yOu zHseBABrb&M0h(uBmiim?Ow+42%HF*;1>(O-iJ@>EY-}Af_qH<0s&^2m9YECk2oQ&- zOOS*$oVuKih{n6AlB+_oX-2cg+s@n>W8LLiqc?{IrnU`l4Nm(VH5Or>Rn$`2I4^ui zMN~9<-4^m$@c%J8^uhu<6ZBPwpkxTJPU?hBzw zonW8WZSi=g5KoA}6S_Rt#*%u3l=X^pgOo!S-&?iy5Xip zjHw$|%MTWH!CUK*sttZEI%}^^M^eB$UYD#BxSW6DU%i}9iNq!@ry-JQ26S}WfKGAi zE+bXE;Keq|y}I@q`mD|)Z$)*AP3+8WJBcrwgQ$}^B$};6s8z^YVu!J>*3Ej`RozWM z$I*s~odS#0CBZc>J_3m2Hiu$Wc%FaTDjz~hK!8o#3BS!DLlyqUug&4D4!rz}Q&&y* z#ObDrH>IQky+>S~1tXWbCHfjr8VG+hm#4ar@&Y@%($kO}+cUevfuNDJVpQrNvzpp$ z;*h~768S7#dIafZdDaCh-VET9I|VKq26y_^EWbGS?I);v+vLQ=O%=lc0@F0vtwkPi zAQUqLQP%HupuIa^V_rFh)v(t4z{SXsJADlQ_5jE{;{SI$gw^7AW z)TG@39^ss>Ij!tgVSHUd!}j^X~&eF3aUuv@T09$mpMLiRto+k zBmL&`4Dws1+0=Uhm|2iOjJ%hm-TH={zOHtkOUj;MxAUqO2E&^dxJ2xa0j1^pXup|6ciw8BFf6=f zdH$QKdk^gZs2(nRf>2%XrtvuKV@ugVX4*;+$&l1%G;awzdEiwYY_GWYUd?~$*DFXq z?ZVTw{xPp@)}MpQ|9EgPs(#rWt*W%41%jw{I`W@VwF@jvdCjw6v3ztB4Poc7tM5xc z+R=DSDmHNe5}|wpxzA9U2&RoqIWSJH*v0Sc>RN(bajojkQ%lgKiq-k$P=@Z_)&BYn zX_=1f7xs@Yi#sIv1x;O6@xvNnH<($hcRfoHQEBDzo?z2mtk_=EaP7n=K_r$s0I2M ztlGH#=4n`pxxo3enO7GRPuv)WC?;XcvmD>H{Lg_6&cr024jlL#=AUzHn$tvT05-OM zVxsZXZUY3w-BXcqnd&kb8kW`IDz>JBpydY<(PlSm;L{~e1*?Rl_uweMyP1Q|!!>NV znM8375sC9)OV<`#_f3uI=DBK?H$7=3_9mov@#@e_FZ%`5XAvetBcU6;wv9f6QZhFu z>3%%4(aH<&o*CGh?@-mWk~uN^19&zVk$*PWgVa*}x@Y`>2eP^z`RLK1;LuRTThAQc zcZDY?=wZ*hSM1(Xl2*8rwvPnc8p_SokPn-28neV_BN_k8X2ujdcffr z6ppZO6c3-7vAj623Wnl6Vt#z zy|*2spW&kjpX%Hjs2ak|4{7DUm8!dzdp=Vy*f_d*s$Ah>4wlSE$ZWbx-j^aL(QLZh z5*B(1J)zOT12**;IS5z}nEV_Uo3e@IGFbGg&mE}y43TH!isJ}*0ayBwV>W=v2W>Gq zkoQp0_%?M)0m7+w7$8QgVZt*@PTK2+3Nmjso=%9V(Ug)EFQ0lfEtD}q ziE{({(!PCqDI@@%Aze#lJcj1IgCWg%e!~44u-uVYI89Bx?r@1zj&N`Hy2h0YJ=QR&`@=lqsBgHXHky&PB7;{7!MX<0zvWEuBcmX-DMt@IvS zNv)}#KSHp!$*3Bkj^@7m856INnH!oXB7#J!`_&Pn+ba(+r?$+j6tk)*I8gXaPgD2V zso5r(>)oy;6#m|+{L6ypSZsVLqu09csn8|d*h2r2;o;mKSudhI*`&y{bK#|(NhibD zYelAoRcP}m@$t{i0Ti9|73UM-sF4Jo_OhtjEX^*x@JlN6%ncxfZTl1Ni_SDfQ+7~* z`UlpDZkBKr#49pGHKuU`VBnzYIdOENZ2eh6Xi_c1a7WGJh^ouz`kwLp9kK41D)ejY zS`0r5%UsbosoW_j8M+=li}t@Br8K&K~lxG}$bTXiVcdzzct7!Kp0r z^YC}S$}1U4n}*?N91byJ2fwTU$E6we0NIkaf2&DfLTJPnRdHmG{aYO zcaBX4!1|oXB2f1G`aO5im^y(yU}9_c+m42s4#!|s48&I!OgVdh@$IQ}Qgu>}1G{ef z=;!D%4&e_Y(;^)uHlHblnrZqmNEukB6U9>5LtSmhuJ+-1Nf-NdGAdd1QjVfaY+f`_ zW-Y4mw3Yb`ChL=uYaK z0KYX73yh}#IOmFw19)fn3#dM0k!3~+Y2-_&0+N5#tXP87K$jc;SDP_$I(S@Uhf5?+TKrM8(~J|X$`SA1pu0So0gqXN{ms8hD@5sQPWWRv6xYx^@><%#w;1BPEgzNM z{}6h(qLj49P&>54; zKO@-Pe@q=)^yS4f9EpEtwBn<614Y56lOOTK!*^X;?#<{3hwGqQWid>ofZE6e+Piauw|W6odc6HhDZ!tTGzQDn zvfnoM;_b%YQEl%I&EsY}m$~QFx-~4)${H7&R|Mu)hsO`jg=)VnxWr`hrwb#fm^v%n zj=73=k9xdZ1so_$_6#P%U5$M{y_cdMp3&xC&`yi?L{&39MxQUEpc>H?&AKVApRb%R z0P7o|!@T09*3p^AYZKOYrJ7E)>gLh3JeE82_yZ8L?pGmPcN}r0M$`!dB)%x%L`@ux zta&vv$3HYBT+fp1(m0?Y92w8~WSG>`VSCW78$>&mn_^|PYQcDXUEUAEGZ{^d*S`xr z58nPN+!x65=a2}$n$-{ z?lM#?+H?w0324-AI0MeWOpwN zv^PE&$5#gvbA6;LAvB&C*W04(KuKG@jKu`bPsV=uoZRNQ7uiWuzn`iKYBc@QbB+Sn zC3nZ9rmG3zs`B&nFDrx@=YZDP#I51NZJiuWlieyzTJE34xt=etcF%fhhR9!OlDgDa zmGI3pT~9KddP&oKzx4rjX&<`fFX(wGBLFgd#Wc04$In(k(vN&VzU=+JH@%mwTGNn@ zi-#yFr!eX@yWKIyq=)asSL`Zp-tdn%=neaNtPVV4T}8LD7;)vqKyxhCG8v(!!8m6Z2rEUX7#O8Zu)swzKGIlWLKN?0a0uK^z51Hu1rlUVU+v>&W8uW9n(lR9SG z?la!}?zz{LW8je46EpakIiUm1LH}hiZoR~z?^xM{_IX@mCLmWgvn@g2iO%v-X0~|9 zNx0b`_KJP9jMjwTA9Dr!u$OL7>dgOY=}iAg-F4uO_ug(x#(sTvFakMxS>2bPpwvt` ze%-3c{Z5#P5WC+M7_d{LYz z0O8IY|MQp^y03JGYfR6P=xwdu@B#BE1PE#Yis1r*2^mQlGqtuf948=`2uAxKKdG(2 zOUT&a8Q=(9Lne_5)~}-X_frZPk)JriAC3mv#eVjrCgT^d5BZMuTELzekym-+g-BrLCTZN&2`?uBU_{gLtJ<9(JCZN-<+IE&%x1hPj zIKSh6R>$7s^{#|&H2T)PDo#8>{SamM9>pf16F*_CNq2X?bMSR1fQoEpq-2k#nW6p8 z;k)&kYTf6fQvF$zUr*=;$06&a6Vq;dtuhk)3tA*$F|tjO075#n?o4AM$?Id3RjAN< z^aqI`Un`s9PRrAi$v~%)2IN&ih-I~>=i@d-BXFwCor|TcbiHerbGz2P_ zs-GVTLg`g90z`ANPqy+v@$xDCK?|I2vlYxAk6B*5PI6{nxO)J5710P~w_X?^O0kY! z*#iA^H+oQc2#n?n?bQz@)sWaTT}h~*^>T7RI0Yv zf4h}LxOvLb3z^UDZ$`d)kdf>zoS2E1=5{~=qZuv5nH~)SnyE|H^d)iNFqXj(F<;)* z#hKk($5F)T_=TW1Q$B8dm`lXrE=Mw}z>ml?Dx66Gc=IC_FWqaqxwM7kPC?ns&=)jj z?!1kq!d&SD-=p-iOCss$6s-X;1g>1QUOQAPGwJim?wexks(A+zrXSSL;6@g6BC$S! zjc#_g4eFPAN;6upC>_##?3Jy{&!+0lQ6AaqrPO(YW_6pZZv~u(kJ){z1AXWRu-Zol zLDK=S{_jpr=X=4bjYPNI-*@6V1o^Es8=qg8ZMnY#S!3|2>x8r=!=G&AsQS+5&APb^ zziFp2xa_&hrUKtD9l`@m|2(^_UF+igm`d;2u(0 zb*K^C`o%oW)v?;e>K1^$GG}4?7}b`eO1n@x{_|;HB^mguzU1i}2t$qF3mN)vg{`?$ zhdaSjfQbnWQ8W28Y!|3Wh2Bb|*xDJh+)hswao}zu{&v}kQSucH*J<2j?Wd)UKhV0; zf4C0CM>i1e_2!i2rag&Xkq4pC?o-2ca?jGlxaDck@`&^G*m8h@(cr;{Q)!vO9&5Vh z(4JE61=7!UX5w4Wj(ry$J|D)ZU0{dYqOMBQ9fe#u?qfGNDM`UyAqBpoVcyUg^^S_@ zB*9%$xlCt5t77Kbxq2KP8>(c}c0~I`L#&PTs7JeV%C5^Jd(TAM$frZ*eV42=+zlCGFux2 zg=JZ~o)0X!doq1REgTP5W+2UmlQl~kQ+WU&)O4yxvpm7Tsj#BhH0&*u9@nkSsK@M7 z^J#A_$H;2Oz$o(HYqU9J;r&di6oFEZqYYEA%(PdrS?p6r(a^4i5Z@q`@2(>_D(PDmxsC@Li!&r}gdtBCRjc{HoJRvO z04@r#w>ck9V(I=4LWQOMyZDO2Vgyky|Au+omY+Vg74YF?mGF+VEv@f8se@Z8lhAf9 z@=n)S2T7bL5k(~q(4i;nMm<7=adYYi^G-*x;s3tXXL-#fwSPZ9E$=?{FSuD$U!gqE zmhV*YG+eFB#sckoMk+AxyHmyx<}`n4f&FAOs6We`qtu0Byq=SEXhaM!G<$<&mi+m4 z;VpP~tn$NqGRj-i%`XQ&Cxa~c%54vlCy1&44W7u^xeQCw7FQqV|0fdUre5FE@Ay(1 z$9rz9z;LGa{<)s2YOUG)+lSeOEmqaD_kBtpCM*whJN>{qGuqT;f}egoMXgSuYtBbM zM{X+BiTXO{KK&P70mqrxr&>3yZ!Lee7#9GqkR#7$)LP^``jiAA=GrzjF^-OE*srE5 zOXyU~!z@h28f&e&(HbD_dnT22B8(g347??h{;%Y}O5XC6Nl5Ja8#cDy&q*%b$|^&) zh-vbkUqAmaKZzi7XXlE!H#*QlFaGXXXUTQOv4oOmYojafKg9YF&ELX{I!;p~C=7S! zx}W9Tl4%yg^q*w9iT&J2{!mlQwS91XgIsIagW;V*-aqh#IEVmcrfBwsA(WAdD;vW; zQsYxmUqC5{!VCW#Bp`bSGqm63+^2>$li*WP{YI+cZY9;`q){WDL{zpFzo0*VL)Cr_ zq;&dphk3>l`+fm>{>GQ#06jb)aUHGv$GR{f8GIVx9}t+FosoP>R0} z7G@UDDAf43zW)P>%)Lqc8@$kBt?ejdko;GHCn-_nFOI`G+*kh#u0Z)0MDfp&$<&Bv z;q||`q4+0|iAZPjZJxd|3{Mt^mHQ7Yj6l!Vkw!cN|4&}*|HaD)?p#dv|IW*AyqMZt zLBojW9!|dh!e$N+F*^PeoB50Dg3}RC4}I8kS$rC?e;sCw6Wl*C$}cGC{{U`+^~mBg zQU1oJUSK;~tj!OdWO&kH@?`kb#Q)$7#stD=qI#96!T$?X`D?@|XWX><+YW$#*+ChN zlVnC>-{0m-xb7Ln=IpzzIHLu+>vH>*IIdw1a#@_gLk`kDx3OSG-@ex7cE10vEuc|y zS5c);`_1Rm6tPC3!Avlp+TyHEkV-tBKq*}h*fpFNGP}*e#8UN5!M6uhvqh#zJ5}~> z{R&V}kJ$;7pq+7zV3vzJSujhN`W&H*qAq9W2)T#L6k*I?t7WaiJQICpCR0%(vc)_* z@};lFDSYJy1DK&R_q|(?SY)=%VJ7#>k8j`%J_NwtSShX+J?}TUckdC?YbbPU3D;Ko zLV08(_k+*>!rIz@FlrsKvb@I z=LCXwKU&xxBmI!lZb_Z<#Mc=tMq2LH?Mcmq1M7&A-Ki!yiqe1-?2W2VI=7O3?#fm|<*<|bJ>eJxjnL!%@M*?gNjF@klsvr(!KI~*NF;+i zWH9K=V?un(iP-8W!3f~TT?lS+^xayVaHQaBtbaT3rA%?y44n~5#B|oPUbhR6`c%8OBEFw$Y`ID|d;3&_y zY`j79pygK-0Y}&muLChCa-!>__H(y5tYh!k=*68`$N7KJFeV6-Fqyp3R7*F z(lOS5Fka!&t#cgN1*wt|+QX+@F7sKy2RMO`6s-1@SufcuHWIodZZVu9p@{=ul zJ3kCj;DuN``f~gU6#mvXGi6@4iB1!_cQJqI^~rn{2fTU~F>9CR1ntK$%>!M|;@PTs zp8}a0J*GorNtkz92*L4Nz=Z1H;4OZ*JWLsyP7Kv~Ras%b8;$U_t+t#GAN5E0_GuO$ zvBT^4Au(t~%TBS#A2Z0`*X-%7>#2U6%bC1EXLdVXd?<{EeKZ6fm5>76mb zB0CLQUfLM4`)<_rDab?JuSSsNlz9)|E@^1wzyzQZu37lR`EJ^t(o4XfcepG1LT&gD zZX!`5QyGjWyQ)K1_upk8*hHf@tp)+G)!bn{ZwB4n?qnN}7j|;zUuWM~UIL%SG~L(y zxsq(7x4m*ValQKKBoKkjT+yj}$!>Swb;Viz z{KczYd8Misi-7BoI6JAQKBsb#%wuW2%C0-3OHChxCP|4F1aBAP*LCftaAkV+QMlGF zhZ#tE&mU8VjK!zC#`CN_aA(?_j9uVE?v8cVjmM>EXg6(bc=su51)Rk-)h%b8rzhWy zb`lJ4_8L`7U9k8{2DT?Bzp8Z{iDOl39J^f9d%xRi0Z0zMe%%e0>sr5d(#AJPRjQy6 z4zr&I%^67g1)8?ka9wo5hh0_q=9t5Pfxx1EL%ZsANey)#V_z+LbG=^Q{~=}@G;s+??ncqb>FD^BEIhE#ql4c%lz;tmN_e) zm;m^aorhVOva=Jui<88}YGWJylJ(0~U@bnmCK7sZoC3$7RVUZ*(PVhGSoC^d(ue2a za82oxue&LVTxY_`{-#J20P*qQ&%KHI{RFJ zf0Sxl`QB$F-x|$fUV`P3T#E>O7jSZXm(s{=BEFrF^aw=V$(=!(9sh{fOKGf>7puC%n-CnXg$6u4OyVgYz~`0eyQ*YTM}nuQk}i z<%XSWMWWjGl-SK(YLia4Ligw1UhS`XG?di(B#8OD5d)MMj{ZiI^t(yhy4eFk# z2BWx-V!;`8Bv0oY8#BgwBgwq2K1H#w-SzG=$IA>KGTEDu?U%K0NgFCd-({+=$%d#G zLXLNOiG#vpvVY!fL;<9U)O}BN9TokGGf%E`y$SXvcyp&ZCN$bbtU*yQXEnyyPLDb9zGfo_(AG*AQR4&}%yWR1Ep~3FeP0ZxBlh_#H*{a8IFY zD_GuxVBn5prISPV-l6FW+GQ8ZR#WB=c1|7Yk$S@XLg=0d%j=u0rr+qpQ6t<^9lQ9? z4V5Yd#`VT)9snupTQz2`5_pY;fu;P*CsvJcj*w%_BJTF?&5_Ji}| zGujVl@!X{bZ*pIfaXS##9U+_`34A_IpviodvJ741hjrQ|ji&sO?-w#FY}H%YCb0+B zx$fCVA?wZXEsO3M>#f+W#)+VCxTW@`8-twjd5M`3+Ka%V$4!>PB&V9;;J`WIt&JYm z&h?{~YHUeoHC7PawT%0e@v_v@aV-C+T}IE_ejkD6@N7JL)pn0jgv(n^3PQ2!SAz*c zPd=SQ#w_cuP?REiqF@TzLpSpp*X#8=1L&HA`)#K|Yj*o<@wUVS$d?MfqUY6AB#XI@^mXO zU?9nL4!YOpm2bHgGJ|t`>)(!lfhcTYbEypQlbg=x`rX?= z+Znd(73fTykI!y=i`2A!gcmZ_Fp7b7U^+dx?|k_>aA+L9{)Y)6UYf^VXO-{%+_gym zqbr%|FeUHjO={j^VR^u8D0mlL2h|_CgtG4GM48Mzder%Kzz&6_!v`7vd{G@c|7(|Vt5KfNjz`Sj9Vm3hNMO8nP!&e*dXmr}=7%>uq!x7PKj zNwCsMmHVX9REQ+wiutYF2_neY*?}ym>FvIVo-8#s{UkrA^BwS!+l3B7g*?GdsbRun z^shhNl~pktt*zFLtMO@qIOM)uVC<5F_cRB|B51Ju23}7OV1Yrjj2WeKB~mfS2DqBl z7MHB>#Nm?-uiWiOHA|TU9z+s9xuNg;rS;{0o6k7sugLGUe;qI`vA|B)kSr>T0|L6j z_)Uy(mE3?lV7rSq(7?E$MtYYoYqGlWdtR<+$#}@u!Uv6vHiJDCNAGVmfF9mS$0JX!z#ub7!zgRi2i_lt%QB4v9!QGmuR4qUBpjg()ro0%H_5_ zj401t;g=z+i7QI4ouRlx!~2np%%HEz!>Q-1)!axG26*o)Ed|yLs9h`#NI%Imzb9fp z_-s*NpNLRO$xh-M>&xPpJjiVIwMgi`B{rZ8QO8SK{N#tdtd!m#58C1~ocs=q>INmscSN-wG$y*rJc7ijFyEGv(ZB&%KqR@1k=`i7%vRi*-@pS6060$5Ze)cLBW z$olYS^oOa5u9MR+VvszHsC*oIcHP1pMkvOTb^ESFx`f?>%aog%R+@%mGcvMobFDFv>ye01H69It{oo686 z#UOAfs?((B-w4O-*;AJOSb5rerNPy1{*`GpfBnNpQmAKns*O~PPbTB5{0fLPkJP`9 ztyEz|m2e-xmW_ike42CQmX%c(1BKs%5Bauy({evT;i?!LoafAa88)RpeFtz4z@<%Kv1IA6J}A}7h zF6SC?iXf9zY3UfE;~Jxvc)pbM6{|^pnd^SNI$M0X@u3!IIfpl$Sid8}WM|Ndzr40KomM>VI>7gp zWF8LHY+v#Y$R?12e@#QYE9e9XuMXn;-1Dkt;PN##8-t9Xk(pyO9E13~f z3s5B0hxe8;4`)20>$aexQe_b0b#nwBa(3SyhuJg6&N8sUH6iq>P#&-R6)IEmEUmP7+SRDVEKm1NPx?g$1rXt{9*;758O_==tpF%eI$x`{;CHLX4fe zNXp9$XL@-fkDS@HwJFnl96G%1JbTzp*z1HPn{34;js$_CoJWG$9sj zV=w|;sMi4$M~*N&fm#TJY{1YCgVZYv+OYl(Qvk%y#uRP`=}v&C^UUPG9DaXmM#;pZxoK*lllu1*9!uvW z;pg^gd4iC6&IkMfIUQqIE?x1*!kFL;;`qv-(DkfE;rr0cCkaPS3hFW<7oR!vUc!z4 z$hiDZ`K=cWy~$R#gJ`1N;Uo}CWM3Ny=cN~Smh1Q^vWZ03cDdKc|# zYEA$@3o=mtzLWOm*_$q*(%WoLNM>h7!kp6eY+;AniXndxx>36u`P_CqB@#Kb`^XUd zNe67FY=`^&q~vH@MDVmztL=@C)&7FwdszrMgQ)iWQ}5eBg1xpF&QYpH3iaL~a1xe! z+HWz>TWSXYI%)5w&jhUu_WIRSZ&pr+o&+*qE3vm~Q%bHEbXkP-(#0(W3jU~KM$K{P zh;+;O;()ku&Ubwojm)=B?Pp=60g(U+A4UcnZfv!j8~@lQKKUA~W=oByQNPa%nP%E` zdv7RSfW^r|oO|{xw+S?^B!iVdRL~HF@v^OT ztz}p{zYL(2Z#uj!T^r)kmT@Hs)IqH|BoUmn z*D_bj{R9!sBI$NBk1Vw4x#Es^LbB-6^ecwOtS2`4J8OVoP1G*TE=U&nC?xRRMrjn` zX~Y+S5mRPjV8GEfK ztKtGC*%Q1w<6n%59-p|9dw9_!K3RM_st^^Xn>P0xL}Z)>QU6(kzv{8o`JpHLk?rcu z)_xgH(+NO+C(%E1{#HNQ!O>4ZK68^n(f_AiH?~?i*!jccbzWrS_%#y z!X3xb+8$yDcVfY~0eQTl-*@_3a*c5?6TQiV#%M79ovBB8WmDYbxt(*2mGuG)m9!g& zX9rt~awSKor9*%Bi`w*7h`X7Q@)VjL$Et1BDND~M91AQQZrn=@&XlPfR5%l#D*O#fMg*ps2@OeqHXdifzH z$m~oMIfKr?Vu{4LbYef&8&59rXy+#+Wxr%UW-P#rYO;4uCoJG|pvibPJd9}JF+KfeQH#Cl9 z2Ga*Q5L!QJhXOC++<9vnxL+o=5zPSy)|(MSterGP-c?J~x`*OOpq+ueRERK+xr|Mg zX0BiZs)8{=-vU}tH-y$r%*5_;C7SfhUfN@gN8^Hm2F|>9Cc?rovj(wLip#y2y2r#F zsX84Lw|`rc**FaP+Pg*GV%#2GQYnDW5B&AlOfoy-!L`S7#Qg0Dr85~pAVb1a;KpCI ztx{||6IOx>=9evrG`0r`>OQ!TfL%y;Tg7UP;GBJ7$Yw#ZGjg<3obY{_tuKu?{gSba z_a{Qd#b5>O_I5(i``Qhal9nORSfpJAmoQT*`U-S@Be8#;AsC(x+dkDbT%93Mmme=T z_)jZ2HpyPr{)5#)W*&Hx692Y?Jep}|v5)U?AF^fRY72L4UGDTb^r%F_rm8o~-q*h) zV#@N^x1C-bzFb*YL&7okIDYa)=v4 zeKL130UYbmZHLOUIjpGRL{1X@&E2}2ZTFrVk1trV6{C!Pw_N%CD~m^yf$h%3v4 zsFj5-2`zgGK+)JhL|##$uIRX)aQH%Y^}fD}O8=4fseEf=r#|0P-?7O2PM_=~^6=&< z{;#~C`A0%jvM6OOp{TtQRjO%&Hl>wBo*^-8&X|)t%);VonhPv1{-?K@#R1gOm#D$6 z;&MjU-;56+uKO`HE_qIgWGn16cJjZe7b7V{ia=3-UZJu3i)YbO@DS?qL^w$@51O)e zpq>{KIL3*FjRrO(y^b~%!PaW6?}e!JEMQmj>5OF=yhO91w>zG74!N^n0s`q?bty9t;=`iBeycBVnf+E4<~OvRXF>j&CE;sRV5r_q9?!2QmJrydzVRm zC`}Q-Qc)3>AvAa8C|)f-<6Olz9XjS&)5PwLezUT3%s7B+?R1 z^}sfZfz!V9oB4Uln{@RT$_11bdx*8PMyNu&;{d-0}ZYlADdU| zG+<*@;kcIAdG^kiwzq-JE9tX4Cab4jU|?EH5Gj5q?iOotkgZT#@gb$<#}wK6_d%jb zXWs;0kB;1tooa|yO1ffW!?`iDFH17(hrdqC7wE_HC6nuThzfu%Um?<&)})z+hZJ56 zE3NnLw>j7YYdsy1-rz461IOSs3YarwSSegXO$&%DvIWxLB{RKk8^8mB)vs5ETmWkT z3bv7V<9o&C-Bf>~i3z(*zFYi!I{nu4m%_B5yN|`#=Z>q9&K;8(W|42x>2*7b$#|X2_O3Wx52~UMxtDtaio!19{9RZap{p1wQlPMn0(P zAH!<69|1!Tq)RE4cHD<|1l)&#CjXPps(73gIc?Tro=y|XghH~9!{8H_aDc{r%kMow z1+r_2U^?!)@=&q`R2ZYs5#8%PF5Lp^-few^8`Qga8GH`5lpKNq9+v7?wB2%HUo!P`^*p#o(e=IVHg>qhEU9$u8 z&iOQfO}%T^(>D=WTR5W0aK4a?=B5?x@pd;c5#lI&cZy`-IEDff2Np^GkUSIbG+3wC zjif*)pP`wl>N6eWo-WK{$g6U)Y-=;tkbcenkr_zmbXMH=bB$OTFiWqg6@u%aBFvvS zP{!%<;nru59VXKZBsw2AyDFlwq>qM*%2!Mtx$6~1emO$>U&yn<#Ec<#CHEghS@Y4 zu=GUzWnIe&c1v^ew|}N-V^hzAHJ@PLp;=@WSv66XBYjo{7L{xgWj;%hlo->4fK#x> z1W%r3984n{r&IRtkI&b{OS77M)-^z=$Y0zhr}Ne}gbt?~>bi4OaqgZG#o4}5e@kS{ zr_*Kg1txLz`ozbR7amhg`nb4)BpZUiOyH}_OJW9laMy;s##{AWTBDcO1_@7L zL90~(_h>;bbN5yXLGLO9!*I1*;BG5%&oY9K9Bl7en6qyz$F%uM&gLKt@I-;X%u6Y_ z?&nm*6fqk=d?2CX(#u85*Aey6xUDT}YB<^V!|2qOt6C>nb>JH2`0Yv0!N;HR1!cJ7 z03$l6Wk_>z^QYk{BSy%8u{zS4x`P$pWEvB8()=+%wo-z_@okV+4hy>MUU$*v>J>?U z;}$!XQibr&%ef7o&--#mfpJpqgt=MbLOSA$N%*$cnAe-9&Fw0TE)szg$Gy^$E@`7i zx|?*t+!eZxzG27qy4wl+i(zWToQ&!oO-43}$e_!VXD4^>#StbF^BeqI&A6~0Wz z)r0y7!eXeDO6c;!f|;gJ?jqc!!AvMELgs<^QI_-88XrVd#!s} ztWM`uJ8--h#*omKlZxxGmT&nmFsw$~_1g5B(K3WN`@1gNs`6&9^zj=ZR&_B^7{q1F zp!%q~WYtYYLOQk&XlL{fSv=Nzngm{$1C@`VxH%^c;F z>AO?|pPA{JvJBlmXoSS}axp~=ZoGyE(x?lfE#_Vg*4lR--dMeF=dIPM;D-!=i+MbU z2bx%vyC71rRI#4>qonyf@mq|-M@MJELDSb1!g7Fu%4%p&>Fqzi2mD063VVxvQe@~e z*591qJHTQU*fYA~0Ua%$bP8A#BP7lEwQ5fPxlvDWy#F{+B-i6Wb0NNz&To3%J|pw5 zFrNL>sWbd*D2PN2yk!;uL40idK3UV450y}G@A>GurO}+#M(}!R%2*qWX!i{iKB@km z47?0^n4N@+0*!8#?A6ToT!-Q$IN9SDTkACBiZEi(1qPggVD1d|A!`z+j`~n7v-XQf zV%PGf+Y-J|Z&|UDF7azs!kTu!D0t2rmG&hep?%M@&2eqmOCJeEF@zPQ3hKpt@S)Uh zh+|ax3GWpE0|{zq6uxHBjrDGi5v4ZJ5T3=#}=d%JNI zY+d+NH{AV0AxTx7a)hh75Wn2$b53Dya z48FH%#Znw&qT@Tc9p%#&MG;_(dcBaDAp#LJy$@F5f-xv%=ywJ?Z#{W*Zs5^O=c{O! zn&p>SnK))8ZzaZkH@wQo`}Bd&QV(%QF!HGs7Z~ya)`v}|$y9eICd<_@3Hjv~c56)= zVrT4M2tmeGXBzhM#98I=0CTI?+QMJ!Ku3hC-x7KKkAxBq%Wg8EIM&R@tj8noRm$yH z>1Vj&iOT@*;4F4-)8pLBiwW&kK4ZCEhL6be{@Tja7wg`H>fO4X($S&nwknCbWfM$1 zj(20xPK852h!8H~2RXI7i`GGLtB!Uh*1OZvq^`ZbcPvPco&h8ze=Pm#*;gCk!b>p? z3OmiIVf&XxRRjr4I;Sn?$1mIkYn&`Bh%ed2SE{ORRm8%$MRwZzN7s60Jf}1k52&yU zL}X;r`N`Sv2E90nNft3k)Z@000rB5XL7i+7=@ry!vbmEbYUp;^;oyAk5zuFYt@p5A zXnZgdZT@RmX+$IL+qlY8xENRbSdB%*R)3(~U`88xWI)*6kLn!#oYcmTR`Y9r7H zo}yWp(RzXJo0Q`}v7w-3Yz@gu+U(@7n7FH}${lknBT&yLUxB@dja7vYxQBUgUpVJK z8S10db1-2%Z%NjRha9)*C0NZu3eR_9z+YQ2jq9P^2Y!7v7hn8^yPJ%dutG#=Xy~0> zhX1sP&;GuBNkY}&OT_Rb-)nWUwWbUYJt?gVeGPivA-u%rKij zmtO51N|9fl{xW(n|NOM~-rw{wsYS_iiXMG}fl^1hoDsL3L^GKVK$j?z_wH9DTOjKY z5BseCZNgVQ8k+w$;pOI(&UvvLPw3y+CoVJ4IY{w_=Dzqq@hjDrY8`^#{z=r4PhTpn zfh)2+i%OyqDfjW^`)5J%Hy&6&#e`|norn# zXP~=y?VlmPkGSJ5K=y(52Y=!q{g=PNS9F!Vymawz^N(0oN88WQmJ5t_Q6EC-3TkvO z#Rc>@aIsg3UoZMU_W}RyKapGi_>b&B{d3ou^929ibnrhb+ZI4@K%tffx7JHV|L9ZuG7gjYv-Iyzbb=j?gfpNhUa2&-A}R zw3bW$cY*}}KaBwL^j{uI%E?*rcFNGI`zXpz_qstIc}{lx{_JP}C7W~A{Js8P^`;A7 zstCQ-^CGxVv*>>KQjD9ZrTjLa6E5(ZJ`lfq?7{E)04};Qa|llyAHJsr_5$X&6&D4Q z1r>AK_x)XG-C>nF`7i@J)s1eOJTsHEe+lPz6@WkUv%dji3KhCa#5Qz^uUf}YzHAQJmmqT~ZB z+?qR98b(Mdgf&Rhm75IWpMLoEi~gzB{Pf>q{>>(CD_NOW(U9PihUb7gOT3GBtXIJI dsfDvEz;VtJsSFDh$@!(J`jFy&#XXCc{{t#=PFMf{ diff --git a/docs/images/evidence-backed-agent-examples.svg b/docs/images/evidence-backed-agent-examples.svg index 0f1cf74c..177c94cf 100644 --- a/docs/images/evidence-backed-agent-examples.svg +++ b/docs/images/evidence-backed-agent-examples.svg @@ -1,6 +1,6 @@ Three evidence-backed Engraphis agent behaviors - A three-card summary of deterministic offline fixtures. Focused context returns 740.3 to 214.3 tokens while retaining Recall at 5 of 1.000. A grounded answer returns support for 5/5 answerable questions. An unsupported question safely abstains for 6/6 off-topic questions. Reproduce with eval.chunking_eval and eval.grounded. Exact commands and config digests are registered in BENCHMARKS.md. Public-safe artifact SHA-256: 1b894b37574eed06e56feec10840744c36b98813a36ef4c7cc38b9957856e1ef. + A three-card summary of deterministic offline fixtures. Focused context returns 740.3 to 214.3 tokens while retaining Recall at 5 of 1.000. A grounded answer returns support for 5/5 answerable questions. An unsupported question safely abstains for 6/6 off-topic questions. Reproduce with eval.chunking_eval and eval.grounded. Exact commands and config digests are registered in BENCHMARKS.md. Public-safe artifact SHA-256: ead6889f9050aa88a916055ea805a546aa179b6cc9a5edf05291e3419089ea4e. @@ -47,5 +47,5 @@ Reproduce: eval.chunking_eval + eval.grounded - SHA256 1b894b37574eed06e56feec10840744c36b98813a36ef4c7cc38b9957856e1ef + SHA256 ead6889f9050aa88a916055ea805a546aa179b6cc9a5edf05291e3419089ea4e diff --git a/tests/test_benchmark_evidence.py b/tests/test_benchmark_evidence.py index fbb838fd..82f7aed8 100644 --- a/tests/test_benchmark_evidence.py +++ b/tests/test_benchmark_evidence.py @@ -33,8 +33,8 @@ ROOT = Path(__file__).resolve().parents[1] -PUBLIC_OFFLINE_ARTIFACT = "offline-fixtures-v136.json" -PUBLIC_OFFLINE_SHA = "1b894b37574eed06e56feec10840744c36b98813a36ef4c7cc38b9957856e1ef" +PUBLIC_OFFLINE_ARTIFACT = "offline-fixtures-v137.json" +PUBLIC_OFFLINE_SHA = "ead6889f9050aa88a916055ea805a546aa179b6cc9a5edf05291e3419089ea4e" @pytest.fixture(scope="module") diff --git a/tests/test_cloud_session_deadline.py b/tests/test_cloud_session_deadline.py index d1ce1f2f..276b69d1 100644 --- a/tests/test_cloud_session_deadline.py +++ b/tests/test_cloud_session_deadline.py @@ -491,7 +491,7 @@ def resolve(host, port, *args, **kwargs): try: started = time.monotonic() if phase == "complete": - assert _evaluate(2).get_noul("q").probability == 0.9 + assert _evaluate(2).get_noul("has_support").probability == 0.9 assert [entry[0] for entry in requests] == ["/v1/tokens/refresh", "/v1/jev/decide"] assert cloud_session._load()["refresh_credential"] == "synthetic-rotated" else: From ded0675c3dac0ff8d56b2cd6b14f901610a6517e Mon Sep 17 00:00:00 2001 From: Jaixii Date: Sun, 4 Oct 2026 04:03:11 -0400 Subject: [PATCH 04/22] Pin Jev documentation and charts to the current reviewed snapshot --- README.md | 36 ++++++++++++------------- tests/test_commercial_hardening.py | 2 +- tests/test_documentation_contracts.py | 8 +++--- tests/test_pro_cta.py | 2 +- tests/test_provider_docs.py | 4 +-- tests/test_release_infrastructure.py | 12 ++++----- tests/test_setup_plugin_distribution.py | 6 ++--- tests/test_skill_package.py | 2 +- 8 files changed, 36 insertions(+), 36 deletions(-) diff --git a/README.md b/README.md index 39eb9542..8fcdf561 100644 --- a/README.md +++ b/README.md @@ -1,14 +1,14 @@ # Engraphis [![PyPI version](https://img.shields.io/pypi/v/engraphis.svg)](https://pypi.org/project/engraphis/) -[![License](https://img.shields.io/badge/license-Apache--2.0-green.svg)](https://github.com/Coding-Dev-Tools/engraphis/blob/fee9d0c150c250632d8e0c0ee86c1325c9e1ee78/LICENSE) +[![License](https://img.shields.io/badge/license-Apache--2.0-green.svg)](https://github.com/Coding-Dev-Tools/engraphis/blob/66920e48eb12444876d6e08e937c7a12e02ad4c5/LICENSE) **Persistent, local-first memory for AI agents.** Engraphis stores scoped project knowledge, retrieves relevant evidence across vector, lexical, graph, and code search, and returns bounded context with sources an agent can inspect. The local engine uses SQLite and works offline. It keeps changes over time instead of silently replacing facts, and grounded recall cites retrieved memories or abstains when evidence is weak.

- Engraphis local knowledge graph showing relationships between remembered entities + Engraphis local knowledge graph showing relationships between remembered entities
Explore memories and their relationships in the local dashboard.

@@ -35,7 +35,7 @@ hit = memory.recall("Why did we change auth?", workspace="acme", repo="api") print(hit["context"]) ``` -Connect a coding agent over MCP with the [agent setup guide](https://github.com/Coding-Dev-Tools/engraphis/blob/fee9d0c150c250632d8e0c0ee86c1325c9e1ee78/docs/AGENT_CONNECT.md). +Connect a coding agent over MCP with the [agent setup guide](https://github.com/Coding-Dev-Tools/engraphis/blob/66920e48eb12444876d6e08e937c7a12e02ad4c5/docs/AGENT_CONNECT.md). ## What it provides @@ -49,7 +49,7 @@ Connect a coding agent over MCP with the [agent setup guide](https://github.com/ The current registered artifact contains three deterministic offline fixture runs. It separates context size, retrieval quality, and grounded decision checks; these small fixtures do not establish general task performance.

- Three registered offline fixtures: structure-aware chunking reduced mean retrieved context from 740.3 to 214.3 tokens per question (71.1%, 18 questions); the serialized JSON-shape proxy fell from 24,590 to 11,138 tokens across 26 payload samples and 260 recalls. Candidate and packed retrieval quality (Recall@5, Hit@5, and answer-token recall) are shown separately (each 1.000). Grounded checks show 5/5 answerable queries grounded and 6/6 abstention queries rejected, including a 1/1 quarantined-evidence probe; 11/11 decisions were correct. MCP transport and provider billing were not measured. Artifact SHA-256 prefix ead6889f9050; see the benchmark guide for the full checksum. + Three registered offline fixtures: structure-aware chunking reduced mean retrieved context from 740.3 to 214.3 tokens per question (71.1%, 18 questions); the serialized JSON-shape proxy fell from 24,590 to 11,138 tokens across 26 payload samples and 260 recalls. Candidate and packed retrieval quality (Recall@5, Hit@5, and answer-token recall) are shown separately (each 1.000). Grounded checks show 5/5 answerable queries grounded and 6/6 abstention queries rejected, including a 1/1 quarantined-evidence probe; 11/11 decisions were correct. MCP transport and provider billing were not measured. Artifact SHA-256 prefix ead6889f9050; see the benchmark guide for the full checksum.
Three offline fixtures separate context reduction, candidate and packed retrieval quality, and grounded behavior. The chart shows the artifact checksum prefix; see the benchmark guide for the full checksum and reproduction steps.

@@ -60,7 +60,7 @@ The current registered artifact contains three deterministic offline fixture run | Recall payload proxy | JSON-shape proxy: 24,590 → 11,138 tokens (54.71% lower, 26 samples; 260 timed recalls) | Candidate and packed Recall@5, Hit@5, and answer-token recall are each 1.000 | | Grounded decisions | 5/5 answerable queries grounded; 6/6 abstention queries rejected, including 1/1 quarantined-evidence check | 11/11 decisions correct | -The payload figure is a serialized JSON-shape estimate, not an MCP transport measurement or provider billing total. See the [Benchmark methodology](https://github.com/Coding-Dev-Tools/engraphis/blob/27ab1b3baab791d9f6957171cd6d21b09f39eed7/BENCHMARKS.md) for artifact identity, counting methods, reproduction commands, external-evaluation boundaries, and limitations. +The payload figure is a serialized JSON-shape estimate, not an MCP transport measurement or provider billing total. See the [Benchmark methodology](https://github.com/Coding-Dev-Tools/engraphis/blob/66920e48eb12444876d6e08e937c7a12e02ad4c5/BENCHMARKS.md) for artifact identity, counting methods, reproduction commands, external-evaluation boundaries, and limitations. ## Optional Jev assistance @@ -70,22 +70,22 @@ Experimental Jev recall route selection is available only with explicit BYOK. On Jev-assisted recall planning is experimental. In an exploratory comparison using 40 public synthetic tasks, Jev selected a route on all 40 calls but did not change nDCG@5, Recall@5, or answer-token coverage. This fixture does not establish a benefit on held-out user workloads, so no retrieval-quality improvement is claimed. -Managed Jev is currently `not_yet_available` pending release acceptance. When released and enabled, each paid Pro user and each paid Team named seat, including viewers, receives **100 evaluated questions per rolling hour, 1,000 per rolling five hours, and 2,000 per rolling 24 hours** at no additional managed charge and without a personal provider key. Active legitimate trial/test entitlements receive the same limits. All three caps apply independently to the individual; there is no monthly or Team pool. A command review evaluates two questions and consumes two uses. The unchanged production fleet guard of **100 questions/day** conflicts with this allowance at launch and can block requests earlier. Provider terms, live acceptance, and quality evaluation remain release gates; synthetic fixtures do not establish model accuracy. See [hosted plans and Jev details](https://github.com/Coding-Dev-Tools/engraphis/blob/fee9d0c150c250632d8e0c0ee86c1325c9e1ee78/docs/HOSTED_PLANS.md#included-system-1-decision-engine-jev). +Managed Jev is currently `not_yet_available` pending release acceptance. When released and enabled, each paid Pro user and each paid Team named seat, including viewers, receives **100 evaluated questions per rolling hour, 1,000 per rolling five hours, and 2,000 per rolling 24 hours** at no additional managed charge and without a personal provider key. Active legitimate trial/test entitlements receive the same limits. All three caps apply independently to the individual; there is no monthly or Team pool. A command review evaluates two questions and consumes two uses. The unchanged production fleet guard of **100 questions/day** conflicts with this allowance at launch and can block requests earlier. Provider terms, live acceptance, and quality evaluation remain release gates; synthetic fixtures do not establish model accuracy. See [hosted plans and Jev details](https://github.com/Coding-Dev-Tools/engraphis/blob/66920e48eb12444876d6e08e937c7a12e02ad4c5/docs/HOSTED_PLANS.md#included-system-1-decision-engine-jev). ## Guides -- [Configuration reference](https://github.com/Coding-Dev-Tools/engraphis/blob/fee9d0c150c250632d8e0c0ee86c1325c9e1ee78/docs/CONFIGURATION.md) -- [MCP tool reference](https://github.com/Coding-Dev-Tools/engraphis/blob/fee9d0c150c250632d8e0c0ee86c1325c9e1ee78/docs/MCP_TOOLS.md) -- [Agent and LLM provider setup](https://github.com/Coding-Dev-Tools/engraphis/blob/fee9d0c150c250632d8e0c0ee86c1325c9e1ee78/docs/LLM_PROVIDERS.md) -- [Architecture and query planning](https://github.com/Coding-Dev-Tools/engraphis/blob/fee9d0c150c250632d8e0c0ee86c1325c9e1ee78/docs/ARCHITECTURE_V3.md#query-planning) -- [Docker deployment](https://github.com/Coding-Dev-Tools/engraphis/blob/fee9d0c150c250632d8e0c0ee86c1325c9e1ee78/docs/DOCKER.md) -- [Document import](https://github.com/Coding-Dev-Tools/engraphis/blob/fee9d0c150c250632d8e0c0ee86c1325c9e1ee78/docs/DOCUMENT_IMPORT.md) -- [Cloud Sync](https://github.com/Coding-Dev-Tools/engraphis/blob/fee9d0c150c250632d8e0c0ee86c1325c9e1ee78/docs/SYNC.md) -- [Pi extension](https://github.com/Coding-Dev-Tools/engraphis/blob/fee9d0c150c250632d8e0c0ee86c1325c9e1ee78/integrations/pi/README.md) -- [Memory write review](https://github.com/Coding-Dev-Tools/engraphis/blob/fee9d0c150c250632d8e0c0ee86c1325c9e1ee78/docs/WRITE_REVIEW.md) -- [Security policy](https://github.com/Coding-Dev-Tools/engraphis/blob/fee9d0c150c250632d8e0c0ee86c1325c9e1ee78/SECURITY.md) -- [Hosted plans and licensing](https://github.com/Coding-Dev-Tools/engraphis/blob/fee9d0c150c250632d8e0c0ee86c1325c9e1ee78/docs/HOSTED_PLANS.md) +- [Configuration reference](https://github.com/Coding-Dev-Tools/engraphis/blob/66920e48eb12444876d6e08e937c7a12e02ad4c5/docs/CONFIGURATION.md) +- [MCP tool reference](https://github.com/Coding-Dev-Tools/engraphis/blob/66920e48eb12444876d6e08e937c7a12e02ad4c5/docs/MCP_TOOLS.md) +- [Agent and LLM provider setup](https://github.com/Coding-Dev-Tools/engraphis/blob/66920e48eb12444876d6e08e937c7a12e02ad4c5/docs/LLM_PROVIDERS.md) +- [Architecture and query planning](https://github.com/Coding-Dev-Tools/engraphis/blob/66920e48eb12444876d6e08e937c7a12e02ad4c5/docs/ARCHITECTURE_V3.md#query-planning) +- [Docker deployment](https://github.com/Coding-Dev-Tools/engraphis/blob/66920e48eb12444876d6e08e937c7a12e02ad4c5/docs/DOCKER.md) +- [Document import](https://github.com/Coding-Dev-Tools/engraphis/blob/66920e48eb12444876d6e08e937c7a12e02ad4c5/docs/DOCUMENT_IMPORT.md) +- [Cloud Sync](https://github.com/Coding-Dev-Tools/engraphis/blob/66920e48eb12444876d6e08e937c7a12e02ad4c5/docs/SYNC.md) +- [Pi extension](https://github.com/Coding-Dev-Tools/engraphis/blob/66920e48eb12444876d6e08e937c7a12e02ad4c5/integrations/pi/README.md) +- [Memory write review](https://github.com/Coding-Dev-Tools/engraphis/blob/66920e48eb12444876d6e08e937c7a12e02ad4c5/docs/WRITE_REVIEW.md) +- [Security policy](https://github.com/Coding-Dev-Tools/engraphis/blob/66920e48eb12444876d6e08e937c7a12e02ad4c5/SECURITY.md) +- [Hosted plans and licensing](https://github.com/Coding-Dev-Tools/engraphis/blob/66920e48eb12444876d6e08e937c7a12e02ad4c5/docs/HOSTED_PLANS.md) ## License -Engraphis is licensed under Apache-2.0. The license does not grant trademark rights. See [LICENSE](https://github.com/Coding-Dev-Tools/engraphis/blob/fee9d0c150c250632d8e0c0ee86c1325c9e1ee78/LICENSE), [NOTICE](https://github.com/Coding-Dev-Tools/engraphis/blob/fee9d0c150c250632d8e0c0ee86c1325c9e1ee78/NOTICE), and the [licensing guide](https://github.com/Coding-Dev-Tools/engraphis/blob/fee9d0c150c250632d8e0c0ee86c1325c9e1ee78/docs/LICENSING.md). The hosted control plane and managed services are private services. +Engraphis is licensed under Apache-2.0. The license does not grant trademark rights. See [LICENSE](https://github.com/Coding-Dev-Tools/engraphis/blob/66920e48eb12444876d6e08e937c7a12e02ad4c5/LICENSE), [NOTICE](https://github.com/Coding-Dev-Tools/engraphis/blob/66920e48eb12444876d6e08e937c7a12e02ad4c5/NOTICE), and the [licensing guide](https://github.com/Coding-Dev-Tools/engraphis/blob/66920e48eb12444876d6e08e937c7a12e02ad4c5/docs/LICENSING.md). The hosted control plane and managed services are private services. diff --git a/tests/test_commercial_hardening.py b/tests/test_commercial_hardening.py index 0b09e78a..574c4b3a 100644 --- a/tests/test_commercial_hardening.py +++ b/tests/test_commercial_hardening.py @@ -275,7 +275,7 @@ def test_the_published_prices_match_the_manifest_where_pricing_is_documented() - monthly = "$%d" % manifest["plans"][plan]["monthly_usd"] annual = "$%d" % manifest["plans"][plan]["annual_usd"] assert ( - "[Hosted plans and licensing](https://github.com/Coding-Dev-Tools/engraphis/blob/fee9d0c150c250632d8e0c0ee86c1325c9e1ee78/" + "[Hosted plans and licensing](https://github.com/Coding-Dev-Tools/engraphis/blob/66920e48eb12444876d6e08e937c7a12e02ad4c5/" "docs/HOSTED_PLANS.md)" in readme ) assert monthly in hosted_plans and annual in hosted_plans, plan diff --git a/tests/test_documentation_contracts.py b/tests/test_documentation_contracts.py index fd11c5a7..c98f0570 100644 --- a/tests/test_documentation_contracts.py +++ b/tests/test_documentation_contracts.py @@ -13,7 +13,7 @@ ROOT = Path(__file__).resolve().parents[1] -README_BENCHMARK_PIN = "27ab1b3baab791d9f6957171cd6d21b09f39eed7" +README_BENCHMARK_PIN = "66920e48eb12444876d6e08e937c7a12e02ad4c5" def _read(path: str) -> str: @@ -41,12 +41,12 @@ def test_readme_targets_resolve_in_the_repository() -> None: if parsed.scheme in {"http", "https"}: if parsed.netloc == "github.com": prefixes = ( - "/Coding-Dev-Tools/engraphis/blob/fee9d0c150c250632d8e0c0ee86c1325c9e1ee78/", + "/Coding-Dev-Tools/engraphis/blob/66920e48eb12444876d6e08e937c7a12e02ad4c5/", f"/Coding-Dev-Tools/engraphis/blob/{README_BENCHMARK_PIN}/", ) elif parsed.netloc == "raw.githubusercontent.com": prefixes = ( - "/Coding-Dev-Tools/engraphis/fee9d0c150c250632d8e0c0ee86c1325c9e1ee78/", + "/Coding-Dev-Tools/engraphis/66920e48eb12444876d6e08e937c7a12e02ad4c5/", f"/Coding-Dev-Tools/engraphis/{README_BENCHMARK_PIN}/", ) else: @@ -281,7 +281,7 @@ def test_configuration_and_recovery_guidance_matches_public_contracts() -> None: sync = _read("docs/SYNC.md") assert ( - "[Configuration reference](https://github.com/Coding-Dev-Tools/engraphis/blob/fee9d0c150c250632d8e0c0ee86c1325c9e1ee78/" + "[Configuration reference](https://github.com/Coding-Dev-Tools/engraphis/blob/66920e48eb12444876d6e08e937c7a12e02ad4c5/" "docs/CONFIGURATION.md)" in readme ) for document in (configuration, security, connect, providers, sync): diff --git a/tests/test_pro_cta.py b/tests/test_pro_cta.py index 18c776d1..c9c43585 100644 --- a/tests/test_pro_cta.py +++ b/tests/test_pro_cta.py @@ -102,7 +102,7 @@ def test_public_pro_ctas_use_documentation_attribution(): hosted_plans = (ROOT / "docs" / "HOSTED_PLANS.md").read_text(encoding="utf-8") assert ( - "[Hosted plans and licensing](https://github.com/Coding-Dev-Tools/engraphis/blob/fee9d0c150c250632d8e0c0ee86c1325c9e1ee78/" + "[Hosted plans and licensing](https://github.com/Coding-Dev-Tools/engraphis/blob/66920e48eb12444876d6e08e937c7a12e02ad4c5/" "docs/HOSTED_PLANS.md)" in readme ) assert "utm_medium=docs" in hosted_plans diff --git a/tests/test_provider_docs.py b/tests/test_provider_docs.py index a88fac90..91abfd32 100644 --- a/tests/test_provider_docs.py +++ b/tests/test_provider_docs.py @@ -99,11 +99,11 @@ def test_readme_and_env_example_link_to_the_provider_guides(): readme = _read("README.md") provider_guide = _read("docs/LLM_PROVIDERS.md") assert ( - "[Agent and LLM provider setup](https://github.com/Coding-Dev-Tools/engraphis/blob/fee9d0c150c250632d8e0c0ee86c1325c9e1ee78/" + "[Agent and LLM provider setup](https://github.com/Coding-Dev-Tools/engraphis/blob/66920e48eb12444876d6e08e937c7a12e02ad4c5/" "docs/LLM_PROVIDERS.md)" in readme ) assert ( - "[agent setup guide](https://github.com/Coding-Dev-Tools/engraphis/blob/fee9d0c150c250632d8e0c0ee86c1325c9e1ee78/" + "[agent setup guide](https://github.com/Coding-Dev-Tools/engraphis/blob/66920e48eb12444876d6e08e937c7a12e02ad4c5/" "docs/AGENT_CONNECT.md)" in readme ) assert "Ollama" in provider_guide diff --git a/tests/test_release_infrastructure.py b/tests/test_release_infrastructure.py index 35b510c1..b1934d63 100644 --- a/tests/test_release_infrastructure.py +++ b/tests/test_release_infrastructure.py @@ -95,7 +95,7 @@ def test_all_public_launchers_converge_on_the_v2_service(): assert '"url": "http://:8700/mcp/"' in docker_docs assert '".[server,mcp,documents,cloud-sync]"' in dockerfile assert ( - "[Docker deployment](https://github.com/Coding-Dev-Tools/engraphis/blob/fee9d0c150c250632d8e0c0ee86c1325c9e1ee78/" + "[Docker deployment](https://github.com/Coding-Dev-Tools/engraphis/blob/66920e48eb12444876d6e08e937c7a12e02ad4c5/" "docs/DOCKER.md)" in readme ) @@ -125,7 +125,7 @@ def test_advanced_query_planning_stays_in_architecture_docs(): guidance = "`planning=\"auto\"` keeps the original query" assert ( - "[Architecture and query planning](https://github.com/Coding-Dev-Tools/engraphis/blob/fee9d0c150c250632d8e0c0ee86c1325c9e1ee78/" + "[Architecture and query planning](https://github.com/Coding-Dev-Tools/engraphis/blob/66920e48eb12444876d6e08e937c7a12e02ad4c5/" "docs/ARCHITECTURE_V3.md#query-planning)" in readme ) @@ -140,7 +140,7 @@ def test_pi_and_public_write_review_details_stay_in_supporting_docs(): review_guide = _text("docs/WRITE_REVIEW.md") assert ( - "[Pi extension](https://github.com/Coding-Dev-Tools/engraphis/blob/fee9d0c150c250632d8e0c0ee86c1325c9e1ee78/" + "[Pi extension](https://github.com/Coding-Dev-Tools/engraphis/blob/66920e48eb12444876d6e08e937c7a12e02ad4c5/" "integrations/pi/README.md)" in readme ) @@ -172,7 +172,7 @@ def test_compose_keeps_container_safety_defaults_and_has_an_explicit_port_overri assert '"0.0.0.0:${ENGRAPHIS_COMPOSE_PORT:-8700}:${ENGRAPHIS_COMPOSE_PORT:-8700}"' in lan_compose assert "ENGRAPHIS_API_TOKEN: ${ENGRAPHIS_API_TOKEN:?Set a strong ENGRAPHIS_API_TOKEN for LAN use}" in lan_compose assert ( - "[Docker deployment](https://github.com/Coding-Dev-Tools/engraphis/blob/fee9d0c150c250632d8e0c0ee86c1325c9e1ee78/" + "[Docker deployment](https://github.com/Coding-Dev-Tools/engraphis/blob/66920e48eb12444876d6e08e937c7a12e02ad4c5/" "docs/DOCKER.md)" in readme ) @@ -559,7 +559,7 @@ def test_public_capability_and_support_docs_match_the_shipped_tree(): assert "engraphis_recall_context" in readme mcp_reference = _text("docs/MCP_TOOLS.md") assert ( - "[MCP tool reference](https://github.com/Coding-Dev-Tools/engraphis/blob/fee9d0c150c250632d8e0c0ee86c1325c9e1ee78/" + "[MCP tool reference](https://github.com/Coding-Dev-Tools/engraphis/blob/66920e48eb12444876d6e08e937c7a12e02ad4c5/" "docs/MCP_TOOLS.md)" in readme ) assert "`engraphis_check_update`" in mcp_reference @@ -626,7 +626,7 @@ def test_public_capability_and_support_docs_match_the_shipped_tree(): in readme ) assert ( - 'src="https://raw.githubusercontent.com/Coding-Dev-Tools/engraphis/fee9d0c150c250632d8e0c0ee86c1325c9e1ee78/' + 'src="https://raw.githubusercontent.com/Coding-Dev-Tools/engraphis/66920e48eb12444876d6e08e937c7a12e02ad4c5/' 'docs/images/knowledge-graph.png"' in readme ) assert re.search( diff --git a/tests/test_setup_plugin_distribution.py b/tests/test_setup_plugin_distribution.py index 297a2bdb..3c09a7f5 100644 --- a/tests/test_setup_plugin_distribution.py +++ b/tests/test_setup_plugin_distribution.py @@ -8,9 +8,9 @@ ROOT = Path(__file__).resolve().parents[1] REPOSITORY = "Coding-Dev-Tools/engraphis" -README_BENCHMARK_PIN = "27ab1b3baab791d9f6957171cd6d21b09f39eed7" +README_BENCHMARK_PIN = "66920e48eb12444876d6e08e937c7a12e02ad4c5" README_LINK_PINS = ( - "fee9d0c150c250632d8e0c0ee86c1325c9e1ee78", + "66920e48eb12444876d6e08e937c7a12e02ad4c5", README_BENCHMARK_PIN, ) @@ -80,7 +80,7 @@ def test_pypi_readme_has_only_absolute_repository_assets_and_links() -> None: assert local.exists(), f"{target} maps to missing repository path {local.relative_to(ROOT)}" assert ( - f"https://raw.githubusercontent.com/{REPOSITORY}/fee9d0c150c250632d8e0c0ee86c1325c9e1ee78/" + f"https://raw.githubusercontent.com/{REPOSITORY}/66920e48eb12444876d6e08e937c7a12e02ad4c5/" "docs/images/knowledge-graph.png" ) in targets assert ( diff --git a/tests/test_skill_package.py b/tests/test_skill_package.py index 5e3761fe..3027ae38 100644 --- a/tests/test_skill_package.py +++ b/tests/test_skill_package.py @@ -49,7 +49,7 @@ def test_portable_tool_reference_matches_registered_runtime_schemas() -> None: kilo = (ROOT / "docs" / "KILO_CODE_INTEGRATION.md").read_text(encoding="utf-8") mcp_reference = (ROOT / "docs" / "MCP_TOOLS.md").read_text(encoding="utf-8") assert ( - "[MCP tool reference](https://github.com/Coding-Dev-Tools/engraphis/blob/fee9d0c150c250632d8e0c0ee86c1325c9e1ee78/" + "[MCP tool reference](https://github.com/Coding-Dev-Tools/engraphis/blob/66920e48eb12444876d6e08e937c7a12e02ad4c5/" "docs/MCP_TOOLS.md)" in readme ) assert "39 direct tools" in mcp_reference From 2cd9d75a471b56e00c0b82ac967709d8586bce08 Mon Sep 17 00:00:00 2001 From: Jaixii Date: Sun, 4 Oct 2026 04:26:08 -0400 Subject: [PATCH 05/22] Pin public Jev documentation to the reconciled parent snapshot --- README.md | 36 ++++++++++++------------- tests/test_commercial_hardening.py | 2 +- tests/test_documentation_contracts.py | 8 +++--- tests/test_pro_cta.py | 2 +- tests/test_provider_docs.py | 4 +-- tests/test_release_infrastructure.py | 12 ++++----- tests/test_setup_plugin_distribution.py | 6 ++--- tests/test_skill_package.py | 2 +- 8 files changed, 36 insertions(+), 36 deletions(-) diff --git a/README.md b/README.md index 5816a891..4286e3c0 100644 --- a/README.md +++ b/README.md @@ -1,14 +1,14 @@ # Engraphis [![PyPI version](https://img.shields.io/pypi/v/engraphis.svg)](https://pypi.org/project/engraphis/) -[![License](https://img.shields.io/badge/license-Apache--2.0-green.svg)](https://github.com/Coding-Dev-Tools/engraphis/blob/66920e48eb12444876d6e08e937c7a12e02ad4c5/LICENSE) +[![License](https://img.shields.io/badge/license-Apache--2.0-green.svg)](https://github.com/Coding-Dev-Tools/engraphis/blob/9ede220ba8553f2894bf5722a0bd83ea6f4a9bf6/LICENSE) **Persistent, local-first memory for AI agents.** Engraphis stores scoped project knowledge, retrieves relevant evidence across vector, lexical, graph, and code search, and returns bounded context with sources an agent can inspect. The local engine uses SQLite and works offline. It keeps changes over time instead of silently replacing facts, and grounded recall cites retrieved memories or abstains when evidence is weak.

- Engraphis local knowledge graph showing relationships between remembered entities + Engraphis local knowledge graph showing relationships between remembered entities
Explore memories and their relationships in the local dashboard.

@@ -35,7 +35,7 @@ hit = memory.recall("Why did we change auth?", workspace="acme", repo="api") print(hit["context"]) ``` -Connect a coding agent over MCP with the [agent setup guide](https://github.com/Coding-Dev-Tools/engraphis/blob/66920e48eb12444876d6e08e937c7a12e02ad4c5/docs/AGENT_CONNECT.md). +Connect a coding agent over MCP with the [agent setup guide](https://github.com/Coding-Dev-Tools/engraphis/blob/9ede220ba8553f2894bf5722a0bd83ea6f4a9bf6/docs/AGENT_CONNECT.md). ## What it provides @@ -49,7 +49,7 @@ Connect a coding agent over MCP with the [agent setup guide](https://github.com/ The current registered artifact contains three deterministic offline fixture runs. It separates context size, retrieval quality, and grounded decision checks; these small fixtures do not establish general task performance.

- Three registered offline fixtures: structure-aware chunking reduced mean retrieved context from 740.3 to 214.3 tokens per question (71.1%, 18 questions); the serialized JSON-shape proxy fell from 24,590 to 11,138 tokens across 26 payload samples and 260 recalls. Candidate and packed retrieval quality (Recall@5, Hit@5, and answer-token recall) are shown separately (each 1.000). Grounded checks show 5/5 answerable queries grounded and 6/6 abstention queries rejected, including a 1/1 quarantined-evidence probe; 11/11 decisions were correct. MCP transport and provider billing were not measured. Artifact SHA-256 prefix ead6889f9050; see the benchmark guide for the full checksum. + Three registered offline fixtures: structure-aware chunking reduced mean retrieved context from 740.3 to 214.3 tokens per question (71.1%, 18 questions); the serialized JSON-shape proxy fell from 24,590 to 11,138 tokens across 26 payload samples and 260 recalls. Candidate and packed retrieval quality (Recall@5, Hit@5, and answer-token recall) are shown separately (each 1.000). Grounded checks show 5/5 answerable queries grounded and 6/6 abstention queries rejected, including a 1/1 quarantined-evidence probe; 11/11 decisions were correct. MCP transport and provider billing were not measured. Artifact SHA-256 prefix ead6889f9050; see the benchmark guide for the full checksum.
Three offline fixtures separate context reduction, candidate and packed retrieval quality, and grounded behavior. The chart shows the artifact checksum prefix; see the benchmark guide for the full checksum and reproduction steps.

@@ -60,7 +60,7 @@ The current registered artifact contains three deterministic offline fixture run | Recall payload proxy | JSON-shape proxy: 24,590 → 11,138 tokens (54.71% lower, 26 samples; 260 timed recalls) | Candidate and packed Recall@5, Hit@5, and answer-token recall are each 1.000 | | Grounded decisions | 5/5 answerable queries grounded; 6/6 abstention queries rejected, including 1/1 quarantined-evidence check | 11/11 decisions correct | -The payload figure is a serialized JSON-shape estimate, not an MCP transport measurement or provider billing total. See the [Benchmark methodology](https://github.com/Coding-Dev-Tools/engraphis/blob/66920e48eb12444876d6e08e937c7a12e02ad4c5/BENCHMARKS.md) for artifact identity, counting methods, reproduction commands, external-evaluation boundaries, and limitations. +The payload figure is a serialized JSON-shape estimate, not an MCP transport measurement or provider billing total. See the [Benchmark methodology](https://github.com/Coding-Dev-Tools/engraphis/blob/9ede220ba8553f2894bf5722a0bd83ea6f4a9bf6/BENCHMARKS.md) for artifact identity, counting methods, reproduction commands, external-evaluation boundaries, and limitations. ## Optional Jev assistance @@ -70,22 +70,22 @@ Experimental Jev recall route selection is available only with explicit BYOK. On Jev-assisted recall planning is experimental. In an exploratory comparison using 40 public synthetic tasks, Jev selected a route on all 40 calls but did not change nDCG@5, Recall@5, or answer-token coverage. This fixture does not establish a benefit on held-out user workloads, so no retrieval-quality improvement is claimed. -Managed Jev is currently `not_yet_available` pending release acceptance and capacity qualification. When released and enabled, each paid Pro user and each paid Team named seat, including viewers, receives **100 evaluated questions per rolling hour, 1,000 per rolling five hours, and 2,000 per rolling 24 hours** at no additional managed charge and without a personal provider key. Active legitimate trial/test entitlements receive the same limits. The account portal reports each member's rolling usage and next release times; admitted failures remain counted and no overage is charged. Direct BYOK is separate and may incur provider charges. All three caps apply independently to the individual; there is no monthly or Team pool. A command review evaluates two questions and consumes two uses. The unchanged production fleet guard of **100 questions/day** conflicts with this allowance at launch and can block requests earlier. Provider terms, live acceptance, and quality evaluation remain release gates; synthetic fixtures do not establish model accuracy. See [hosted plans and Jev details](https://github.com/Coding-Dev-Tools/engraphis/blob/66920e48eb12444876d6e08e937c7a12e02ad4c5/docs/HOSTED_PLANS.md#included-system-1-decision-engine-jev). +Managed Jev is currently `not_yet_available` pending release acceptance and capacity qualification. When released and enabled, each paid Pro user and each paid Team named seat, including viewers, receives **100 evaluated questions per rolling hour, 1,000 per rolling five hours, and 2,000 per rolling 24 hours** at no additional managed charge and without a personal provider key. Active legitimate trial/test entitlements receive the same limits. The account portal reports each member's rolling usage and next release times; admitted failures remain counted and no overage is charged. Direct BYOK is separate and may incur provider charges. All three caps apply independently to the individual; there is no monthly or Team pool. A command review evaluates two questions and consumes two uses. The unchanged production fleet guard of **100 questions/day** conflicts with this allowance at launch and can block requests earlier. Provider terms, live acceptance, and quality evaluation remain release gates; synthetic fixtures do not establish model accuracy. See [hosted plans and Jev details](https://github.com/Coding-Dev-Tools/engraphis/blob/9ede220ba8553f2894bf5722a0bd83ea6f4a9bf6/docs/HOSTED_PLANS.md#included-system-1-decision-engine-jev). ## Guides -- [Configuration reference](https://github.com/Coding-Dev-Tools/engraphis/blob/66920e48eb12444876d6e08e937c7a12e02ad4c5/docs/CONFIGURATION.md) -- [MCP tool reference](https://github.com/Coding-Dev-Tools/engraphis/blob/66920e48eb12444876d6e08e937c7a12e02ad4c5/docs/MCP_TOOLS.md) -- [Agent and LLM provider setup](https://github.com/Coding-Dev-Tools/engraphis/blob/66920e48eb12444876d6e08e937c7a12e02ad4c5/docs/LLM_PROVIDERS.md) -- [Architecture and query planning](https://github.com/Coding-Dev-Tools/engraphis/blob/66920e48eb12444876d6e08e937c7a12e02ad4c5/docs/ARCHITECTURE_V3.md#query-planning) -- [Docker deployment](https://github.com/Coding-Dev-Tools/engraphis/blob/66920e48eb12444876d6e08e937c7a12e02ad4c5/docs/DOCKER.md) -- [Document import](https://github.com/Coding-Dev-Tools/engraphis/blob/66920e48eb12444876d6e08e937c7a12e02ad4c5/docs/DOCUMENT_IMPORT.md) -- [Cloud Sync](https://github.com/Coding-Dev-Tools/engraphis/blob/66920e48eb12444876d6e08e937c7a12e02ad4c5/docs/SYNC.md) -- [Pi extension](https://github.com/Coding-Dev-Tools/engraphis/blob/66920e48eb12444876d6e08e937c7a12e02ad4c5/integrations/pi/README.md) -- [Memory write review](https://github.com/Coding-Dev-Tools/engraphis/blob/66920e48eb12444876d6e08e937c7a12e02ad4c5/docs/WRITE_REVIEW.md) -- [Security policy](https://github.com/Coding-Dev-Tools/engraphis/blob/66920e48eb12444876d6e08e937c7a12e02ad4c5/SECURITY.md) -- [Hosted plans and licensing](https://github.com/Coding-Dev-Tools/engraphis/blob/66920e48eb12444876d6e08e937c7a12e02ad4c5/docs/HOSTED_PLANS.md) +- [Configuration reference](https://github.com/Coding-Dev-Tools/engraphis/blob/9ede220ba8553f2894bf5722a0bd83ea6f4a9bf6/docs/CONFIGURATION.md) +- [MCP tool reference](https://github.com/Coding-Dev-Tools/engraphis/blob/9ede220ba8553f2894bf5722a0bd83ea6f4a9bf6/docs/MCP_TOOLS.md) +- [Agent and LLM provider setup](https://github.com/Coding-Dev-Tools/engraphis/blob/9ede220ba8553f2894bf5722a0bd83ea6f4a9bf6/docs/LLM_PROVIDERS.md) +- [Architecture and query planning](https://github.com/Coding-Dev-Tools/engraphis/blob/9ede220ba8553f2894bf5722a0bd83ea6f4a9bf6/docs/ARCHITECTURE_V3.md#query-planning) +- [Docker deployment](https://github.com/Coding-Dev-Tools/engraphis/blob/9ede220ba8553f2894bf5722a0bd83ea6f4a9bf6/docs/DOCKER.md) +- [Document import](https://github.com/Coding-Dev-Tools/engraphis/blob/9ede220ba8553f2894bf5722a0bd83ea6f4a9bf6/docs/DOCUMENT_IMPORT.md) +- [Cloud Sync](https://github.com/Coding-Dev-Tools/engraphis/blob/9ede220ba8553f2894bf5722a0bd83ea6f4a9bf6/docs/SYNC.md) +- [Pi extension](https://github.com/Coding-Dev-Tools/engraphis/blob/9ede220ba8553f2894bf5722a0bd83ea6f4a9bf6/integrations/pi/README.md) +- [Memory write review](https://github.com/Coding-Dev-Tools/engraphis/blob/9ede220ba8553f2894bf5722a0bd83ea6f4a9bf6/docs/WRITE_REVIEW.md) +- [Security policy](https://github.com/Coding-Dev-Tools/engraphis/blob/9ede220ba8553f2894bf5722a0bd83ea6f4a9bf6/SECURITY.md) +- [Hosted plans and licensing](https://github.com/Coding-Dev-Tools/engraphis/blob/9ede220ba8553f2894bf5722a0bd83ea6f4a9bf6/docs/HOSTED_PLANS.md) ## License -Engraphis is licensed under Apache-2.0. The license does not grant trademark rights. See [LICENSE](https://github.com/Coding-Dev-Tools/engraphis/blob/66920e48eb12444876d6e08e937c7a12e02ad4c5/LICENSE), [NOTICE](https://github.com/Coding-Dev-Tools/engraphis/blob/66920e48eb12444876d6e08e937c7a12e02ad4c5/NOTICE), and the [licensing guide](https://github.com/Coding-Dev-Tools/engraphis/blob/66920e48eb12444876d6e08e937c7a12e02ad4c5/docs/LICENSING.md). The hosted control plane and managed services are private services. +Engraphis is licensed under Apache-2.0. The license does not grant trademark rights. See [LICENSE](https://github.com/Coding-Dev-Tools/engraphis/blob/9ede220ba8553f2894bf5722a0bd83ea6f4a9bf6/LICENSE), [NOTICE](https://github.com/Coding-Dev-Tools/engraphis/blob/9ede220ba8553f2894bf5722a0bd83ea6f4a9bf6/NOTICE), and the [licensing guide](https://github.com/Coding-Dev-Tools/engraphis/blob/9ede220ba8553f2894bf5722a0bd83ea6f4a9bf6/docs/LICENSING.md). The hosted control plane and managed services are private services. diff --git a/tests/test_commercial_hardening.py b/tests/test_commercial_hardening.py index 574c4b3a..2be80dc4 100644 --- a/tests/test_commercial_hardening.py +++ b/tests/test_commercial_hardening.py @@ -275,7 +275,7 @@ def test_the_published_prices_match_the_manifest_where_pricing_is_documented() - monthly = "$%d" % manifest["plans"][plan]["monthly_usd"] annual = "$%d" % manifest["plans"][plan]["annual_usd"] assert ( - "[Hosted plans and licensing](https://github.com/Coding-Dev-Tools/engraphis/blob/66920e48eb12444876d6e08e937c7a12e02ad4c5/" + "[Hosted plans and licensing](https://github.com/Coding-Dev-Tools/engraphis/blob/9ede220ba8553f2894bf5722a0bd83ea6f4a9bf6/" "docs/HOSTED_PLANS.md)" in readme ) assert monthly in hosted_plans and annual in hosted_plans, plan diff --git a/tests/test_documentation_contracts.py b/tests/test_documentation_contracts.py index b331fc69..da8e75d1 100644 --- a/tests/test_documentation_contracts.py +++ b/tests/test_documentation_contracts.py @@ -13,7 +13,7 @@ ROOT = Path(__file__).resolve().parents[1] -README_BENCHMARK_PIN = "66920e48eb12444876d6e08e937c7a12e02ad4c5" +README_BENCHMARK_PIN = "9ede220ba8553f2894bf5722a0bd83ea6f4a9bf6" README_HOSTED_PLANS_PIN = README_BENCHMARK_PIN @@ -42,13 +42,13 @@ def test_readme_targets_resolve_in_the_repository() -> None: if parsed.scheme in {"http", "https"}: if parsed.netloc == "github.com": prefixes = ( - "/Coding-Dev-Tools/engraphis/blob/66920e48eb12444876d6e08e937c7a12e02ad4c5/", + "/Coding-Dev-Tools/engraphis/blob/9ede220ba8553f2894bf5722a0bd83ea6f4a9bf6/", f"/Coding-Dev-Tools/engraphis/blob/{README_BENCHMARK_PIN}/", f"/Coding-Dev-Tools/engraphis/blob/{README_HOSTED_PLANS_PIN}/", ) elif parsed.netloc == "raw.githubusercontent.com": prefixes = ( - "/Coding-Dev-Tools/engraphis/66920e48eb12444876d6e08e937c7a12e02ad4c5/", + "/Coding-Dev-Tools/engraphis/9ede220ba8553f2894bf5722a0bd83ea6f4a9bf6/", f"/Coding-Dev-Tools/engraphis/{README_BENCHMARK_PIN}/", ) else: @@ -283,7 +283,7 @@ def test_configuration_and_recovery_guidance_matches_public_contracts() -> None: sync = _read("docs/SYNC.md") assert ( - "[Configuration reference](https://github.com/Coding-Dev-Tools/engraphis/blob/66920e48eb12444876d6e08e937c7a12e02ad4c5/" + "[Configuration reference](https://github.com/Coding-Dev-Tools/engraphis/blob/9ede220ba8553f2894bf5722a0bd83ea6f4a9bf6/" "docs/CONFIGURATION.md)" in readme ) for document in (configuration, security, connect, providers, sync): diff --git a/tests/test_pro_cta.py b/tests/test_pro_cta.py index c9c43585..a92e5746 100644 --- a/tests/test_pro_cta.py +++ b/tests/test_pro_cta.py @@ -102,7 +102,7 @@ def test_public_pro_ctas_use_documentation_attribution(): hosted_plans = (ROOT / "docs" / "HOSTED_PLANS.md").read_text(encoding="utf-8") assert ( - "[Hosted plans and licensing](https://github.com/Coding-Dev-Tools/engraphis/blob/66920e48eb12444876d6e08e937c7a12e02ad4c5/" + "[Hosted plans and licensing](https://github.com/Coding-Dev-Tools/engraphis/blob/9ede220ba8553f2894bf5722a0bd83ea6f4a9bf6/" "docs/HOSTED_PLANS.md)" in readme ) assert "utm_medium=docs" in hosted_plans diff --git a/tests/test_provider_docs.py b/tests/test_provider_docs.py index 91abfd32..e7c51406 100644 --- a/tests/test_provider_docs.py +++ b/tests/test_provider_docs.py @@ -99,11 +99,11 @@ def test_readme_and_env_example_link_to_the_provider_guides(): readme = _read("README.md") provider_guide = _read("docs/LLM_PROVIDERS.md") assert ( - "[Agent and LLM provider setup](https://github.com/Coding-Dev-Tools/engraphis/blob/66920e48eb12444876d6e08e937c7a12e02ad4c5/" + "[Agent and LLM provider setup](https://github.com/Coding-Dev-Tools/engraphis/blob/9ede220ba8553f2894bf5722a0bd83ea6f4a9bf6/" "docs/LLM_PROVIDERS.md)" in readme ) assert ( - "[agent setup guide](https://github.com/Coding-Dev-Tools/engraphis/blob/66920e48eb12444876d6e08e937c7a12e02ad4c5/" + "[agent setup guide](https://github.com/Coding-Dev-Tools/engraphis/blob/9ede220ba8553f2894bf5722a0bd83ea6f4a9bf6/" "docs/AGENT_CONNECT.md)" in readme ) assert "Ollama" in provider_guide diff --git a/tests/test_release_infrastructure.py b/tests/test_release_infrastructure.py index b1934d63..95ba34b3 100644 --- a/tests/test_release_infrastructure.py +++ b/tests/test_release_infrastructure.py @@ -95,7 +95,7 @@ def test_all_public_launchers_converge_on_the_v2_service(): assert '"url": "http://:8700/mcp/"' in docker_docs assert '".[server,mcp,documents,cloud-sync]"' in dockerfile assert ( - "[Docker deployment](https://github.com/Coding-Dev-Tools/engraphis/blob/66920e48eb12444876d6e08e937c7a12e02ad4c5/" + "[Docker deployment](https://github.com/Coding-Dev-Tools/engraphis/blob/9ede220ba8553f2894bf5722a0bd83ea6f4a9bf6/" "docs/DOCKER.md)" in readme ) @@ -125,7 +125,7 @@ def test_advanced_query_planning_stays_in_architecture_docs(): guidance = "`planning=\"auto\"` keeps the original query" assert ( - "[Architecture and query planning](https://github.com/Coding-Dev-Tools/engraphis/blob/66920e48eb12444876d6e08e937c7a12e02ad4c5/" + "[Architecture and query planning](https://github.com/Coding-Dev-Tools/engraphis/blob/9ede220ba8553f2894bf5722a0bd83ea6f4a9bf6/" "docs/ARCHITECTURE_V3.md#query-planning)" in readme ) @@ -140,7 +140,7 @@ def test_pi_and_public_write_review_details_stay_in_supporting_docs(): review_guide = _text("docs/WRITE_REVIEW.md") assert ( - "[Pi extension](https://github.com/Coding-Dev-Tools/engraphis/blob/66920e48eb12444876d6e08e937c7a12e02ad4c5/" + "[Pi extension](https://github.com/Coding-Dev-Tools/engraphis/blob/9ede220ba8553f2894bf5722a0bd83ea6f4a9bf6/" "integrations/pi/README.md)" in readme ) @@ -172,7 +172,7 @@ def test_compose_keeps_container_safety_defaults_and_has_an_explicit_port_overri assert '"0.0.0.0:${ENGRAPHIS_COMPOSE_PORT:-8700}:${ENGRAPHIS_COMPOSE_PORT:-8700}"' in lan_compose assert "ENGRAPHIS_API_TOKEN: ${ENGRAPHIS_API_TOKEN:?Set a strong ENGRAPHIS_API_TOKEN for LAN use}" in lan_compose assert ( - "[Docker deployment](https://github.com/Coding-Dev-Tools/engraphis/blob/66920e48eb12444876d6e08e937c7a12e02ad4c5/" + "[Docker deployment](https://github.com/Coding-Dev-Tools/engraphis/blob/9ede220ba8553f2894bf5722a0bd83ea6f4a9bf6/" "docs/DOCKER.md)" in readme ) @@ -559,7 +559,7 @@ def test_public_capability_and_support_docs_match_the_shipped_tree(): assert "engraphis_recall_context" in readme mcp_reference = _text("docs/MCP_TOOLS.md") assert ( - "[MCP tool reference](https://github.com/Coding-Dev-Tools/engraphis/blob/66920e48eb12444876d6e08e937c7a12e02ad4c5/" + "[MCP tool reference](https://github.com/Coding-Dev-Tools/engraphis/blob/9ede220ba8553f2894bf5722a0bd83ea6f4a9bf6/" "docs/MCP_TOOLS.md)" in readme ) assert "`engraphis_check_update`" in mcp_reference @@ -626,7 +626,7 @@ def test_public_capability_and_support_docs_match_the_shipped_tree(): in readme ) assert ( - 'src="https://raw.githubusercontent.com/Coding-Dev-Tools/engraphis/66920e48eb12444876d6e08e937c7a12e02ad4c5/' + 'src="https://raw.githubusercontent.com/Coding-Dev-Tools/engraphis/9ede220ba8553f2894bf5722a0bd83ea6f4a9bf6/' 'docs/images/knowledge-graph.png"' in readme ) assert re.search( diff --git a/tests/test_setup_plugin_distribution.py b/tests/test_setup_plugin_distribution.py index 3c09a7f5..9a5fb21e 100644 --- a/tests/test_setup_plugin_distribution.py +++ b/tests/test_setup_plugin_distribution.py @@ -8,9 +8,9 @@ ROOT = Path(__file__).resolve().parents[1] REPOSITORY = "Coding-Dev-Tools/engraphis" -README_BENCHMARK_PIN = "66920e48eb12444876d6e08e937c7a12e02ad4c5" +README_BENCHMARK_PIN = "9ede220ba8553f2894bf5722a0bd83ea6f4a9bf6" README_LINK_PINS = ( - "66920e48eb12444876d6e08e937c7a12e02ad4c5", + "9ede220ba8553f2894bf5722a0bd83ea6f4a9bf6", README_BENCHMARK_PIN, ) @@ -80,7 +80,7 @@ def test_pypi_readme_has_only_absolute_repository_assets_and_links() -> None: assert local.exists(), f"{target} maps to missing repository path {local.relative_to(ROOT)}" assert ( - f"https://raw.githubusercontent.com/{REPOSITORY}/66920e48eb12444876d6e08e937c7a12e02ad4c5/" + f"https://raw.githubusercontent.com/{REPOSITORY}/9ede220ba8553f2894bf5722a0bd83ea6f4a9bf6/" "docs/images/knowledge-graph.png" ) in targets assert ( diff --git a/tests/test_skill_package.py b/tests/test_skill_package.py index 3027ae38..8a79897b 100644 --- a/tests/test_skill_package.py +++ b/tests/test_skill_package.py @@ -49,7 +49,7 @@ def test_portable_tool_reference_matches_registered_runtime_schemas() -> None: kilo = (ROOT / "docs" / "KILO_CODE_INTEGRATION.md").read_text(encoding="utf-8") mcp_reference = (ROOT / "docs" / "MCP_TOOLS.md").read_text(encoding="utf-8") assert ( - "[MCP tool reference](https://github.com/Coding-Dev-Tools/engraphis/blob/66920e48eb12444876d6e08e937c7a12e02ad4c5/" + "[MCP tool reference](https://github.com/Coding-Dev-Tools/engraphis/blob/9ede220ba8553f2894bf5722a0bd83ea6f4a9bf6/" "docs/MCP_TOOLS.md)" in readme ) assert "39 direct tools" in mcp_reference From 4646aa004fda3d60580734eba95f91615c911c5f Mon Sep 17 00:00:00 2001 From: Jaixii Date: Sun, 4 Oct 2026 04:59:47 -0400 Subject: [PATCH 06/22] Pin current combined Jev implementation and evidence in public documentation --- README.md | 36 ++++++++++++------------- tests/test_commercial_hardening.py | 2 +- tests/test_documentation_contracts.py | 8 +++--- tests/test_pro_cta.py | 2 +- tests/test_provider_docs.py | 4 +-- tests/test_release_infrastructure.py | 12 ++++----- tests/test_setup_plugin_distribution.py | 6 ++--- tests/test_skill_package.py | 2 +- 8 files changed, 36 insertions(+), 36 deletions(-) diff --git a/README.md b/README.md index 62b05adc..01fea885 100644 --- a/README.md +++ b/README.md @@ -1,14 +1,14 @@ # Engraphis [![PyPI version](https://img.shields.io/pypi/v/engraphis.svg)](https://pypi.org/project/engraphis/) -[![License](https://img.shields.io/badge/license-Apache--2.0-green.svg)](https://github.com/Coding-Dev-Tools/engraphis/blob/9ede220ba8553f2894bf5722a0bd83ea6f4a9bf6/LICENSE) +[![License](https://img.shields.io/badge/license-Apache--2.0-green.svg)](https://github.com/Coding-Dev-Tools/engraphis/blob/12d2e405ed4bd78917a3e5eef2cf0d3a7f1561b2/LICENSE) **Persistent, local-first memory for AI agents.** Engraphis stores scoped project knowledge, retrieves relevant evidence across vector, lexical, graph, and code search, and returns bounded context with sources an agent can inspect. The local engine uses SQLite and works offline. It keeps changes over time instead of silently replacing facts, and grounded recall cites retrieved memories or abstains when evidence is weak.

- Engraphis local knowledge graph showing relationships between remembered entities + Engraphis local knowledge graph showing relationships between remembered entities
Explore memories and their relationships in the local dashboard.

@@ -35,7 +35,7 @@ hit = memory.recall("Why did we change auth?", workspace="acme", repo="api") print(hit["context"]) ``` -Connect a coding agent over MCP with the [agent setup guide](https://github.com/Coding-Dev-Tools/engraphis/blob/9ede220ba8553f2894bf5722a0bd83ea6f4a9bf6/docs/AGENT_CONNECT.md). +Connect a coding agent over MCP with the [agent setup guide](https://github.com/Coding-Dev-Tools/engraphis/blob/12d2e405ed4bd78917a3e5eef2cf0d3a7f1561b2/docs/AGENT_CONNECT.md). ## What it provides @@ -49,7 +49,7 @@ Connect a coding agent over MCP with the [agent setup guide](https://github.com/ The current registered artifact contains three deterministic offline fixture runs. It separates context size, retrieval quality, and grounded decision checks; these small fixtures do not establish general task performance.

- Three registered offline fixtures: structure-aware chunking reduced mean retrieved context from 740.3 to 214.3 tokens per question (71.1%, 18 questions); the serialized JSON-shape proxy fell from 24,590 to 11,138 tokens across 26 payload samples and 260 recalls. Candidate and packed retrieval quality (Recall@5, Hit@5, and answer-token recall) are shown separately (each 1.000). Grounded checks show 5/5 answerable queries grounded and 6/6 abstention queries rejected, including a 1/1 quarantined-evidence probe; 11/11 decisions were correct. MCP transport and provider billing were not measured. Artifact SHA-256 prefix 94205036f156; see the benchmark guide for the full checksum. + Three registered offline fixtures: structure-aware chunking reduced mean retrieved context from 740.3 to 214.3 tokens per question (71.1%, 18 questions); the serialized JSON-shape proxy fell from 24,590 to 11,138 tokens across 26 payload samples and 260 recalls. Candidate and packed retrieval quality (Recall@5, Hit@5, and answer-token recall) are shown separately (each 1.000). Grounded checks show 5/5 answerable queries grounded and 6/6 abstention queries rejected, including a 1/1 quarantined-evidence probe; 11/11 decisions were correct. MCP transport and provider billing were not measured. Artifact SHA-256 prefix 94205036f156; see the benchmark guide for the full checksum.
Three offline fixtures separate context reduction, candidate and packed retrieval quality, and grounded behavior. The chart shows the artifact checksum prefix; see the benchmark guide for the full checksum and reproduction steps.

@@ -60,7 +60,7 @@ The current registered artifact contains three deterministic offline fixture run | Recall payload proxy | JSON-shape proxy: 24,590 → 11,138 tokens (54.71% lower, 26 samples; 260 timed recalls) | Candidate and packed Recall@5, Hit@5, and answer-token recall are each 1.000 | | Grounded decisions | 5/5 answerable queries grounded; 6/6 abstention queries rejected, including 1/1 quarantined-evidence check | 11/11 decisions correct | -The payload figure is a serialized JSON-shape estimate, not an MCP transport measurement or provider billing total. See the [Benchmark methodology](https://github.com/Coding-Dev-Tools/engraphis/blob/9ede220ba8553f2894bf5722a0bd83ea6f4a9bf6/BENCHMARKS.md) for artifact identity, counting methods, reproduction commands, external-evaluation boundaries, and limitations. +The payload figure is a serialized JSON-shape estimate, not an MCP transport measurement or provider billing total. See the [Benchmark methodology](https://github.com/Coding-Dev-Tools/engraphis/blob/12d2e405ed4bd78917a3e5eef2cf0d3a7f1561b2/BENCHMARKS.md) for artifact identity, counting methods, reproduction commands, external-evaluation boundaries, and limitations. ## Optional Jev assistance @@ -70,22 +70,22 @@ Experimental Jev recall route selection is available only with explicit BYOK. On Jev-assisted recall planning is experimental. In an exploratory comparison using 40 public synthetic tasks, Jev selected a route on all 40 calls but did not change nDCG@5, Recall@5, or answer-token coverage. This fixture does not establish a benefit on held-out user workloads, so no retrieval-quality improvement is claimed. -Managed Jev is currently `not_yet_available` pending release acceptance and capacity qualification. When released and enabled, each paid Pro user and each paid Team named seat, including viewers, receives **100 evaluated questions per rolling hour, 1,000 per rolling five hours, and 2,000 per rolling 24 hours** at no additional managed charge and without a personal provider key. Active legitimate trial/test entitlements receive the same limits. The account portal reports each member's rolling usage and next release times; admitted failures remain counted and no overage is charged. Direct BYOK is separate and may incur provider charges. All three caps apply independently to the individual; there is no monthly or Team pool. A command review evaluates two questions and consumes two uses. The unchanged production fleet guard of **100 questions/day** conflicts with this allowance at launch and can block requests earlier. Provider terms, live acceptance, and quality evaluation remain release gates; synthetic fixtures do not establish model accuracy. See [hosted plans and Jev details](https://github.com/Coding-Dev-Tools/engraphis/blob/9ede220ba8553f2894bf5722a0bd83ea6f4a9bf6/docs/HOSTED_PLANS.md#included-system-1-decision-engine-jev). +Managed Jev is currently `not_yet_available` pending release acceptance and capacity qualification. When released and enabled, each paid Pro user and each paid Team named seat, including viewers, receives **100 evaluated questions per rolling hour, 1,000 per rolling five hours, and 2,000 per rolling 24 hours** at no additional managed charge and without a personal provider key. Active legitimate trial/test entitlements receive the same limits. The account portal reports each member's rolling usage and next release times; admitted failures remain counted and no overage is charged. Direct BYOK is separate and may incur provider charges. All three caps apply independently to the individual; there is no monthly or Team pool. A command review evaluates two questions and consumes two uses. The unchanged production fleet guard of **100 questions/day** conflicts with this allowance at launch and can block requests earlier. Provider terms, live acceptance, and quality evaluation remain release gates; synthetic fixtures do not establish model accuracy. See [hosted plans and Jev details](https://github.com/Coding-Dev-Tools/engraphis/blob/12d2e405ed4bd78917a3e5eef2cf0d3a7f1561b2/docs/HOSTED_PLANS.md#included-system-1-decision-engine-jev). ## Guides -- [Configuration reference](https://github.com/Coding-Dev-Tools/engraphis/blob/9ede220ba8553f2894bf5722a0bd83ea6f4a9bf6/docs/CONFIGURATION.md) -- [MCP tool reference](https://github.com/Coding-Dev-Tools/engraphis/blob/9ede220ba8553f2894bf5722a0bd83ea6f4a9bf6/docs/MCP_TOOLS.md) -- [Agent and LLM provider setup](https://github.com/Coding-Dev-Tools/engraphis/blob/9ede220ba8553f2894bf5722a0bd83ea6f4a9bf6/docs/LLM_PROVIDERS.md) -- [Architecture and query planning](https://github.com/Coding-Dev-Tools/engraphis/blob/9ede220ba8553f2894bf5722a0bd83ea6f4a9bf6/docs/ARCHITECTURE_V3.md#query-planning) -- [Docker deployment](https://github.com/Coding-Dev-Tools/engraphis/blob/9ede220ba8553f2894bf5722a0bd83ea6f4a9bf6/docs/DOCKER.md) -- [Document import](https://github.com/Coding-Dev-Tools/engraphis/blob/9ede220ba8553f2894bf5722a0bd83ea6f4a9bf6/docs/DOCUMENT_IMPORT.md) -- [Cloud Sync](https://github.com/Coding-Dev-Tools/engraphis/blob/9ede220ba8553f2894bf5722a0bd83ea6f4a9bf6/docs/SYNC.md) -- [Pi extension](https://github.com/Coding-Dev-Tools/engraphis/blob/9ede220ba8553f2894bf5722a0bd83ea6f4a9bf6/integrations/pi/README.md) -- [Memory write review](https://github.com/Coding-Dev-Tools/engraphis/blob/9ede220ba8553f2894bf5722a0bd83ea6f4a9bf6/docs/WRITE_REVIEW.md) -- [Security policy](https://github.com/Coding-Dev-Tools/engraphis/blob/9ede220ba8553f2894bf5722a0bd83ea6f4a9bf6/SECURITY.md) -- [Hosted plans and licensing](https://github.com/Coding-Dev-Tools/engraphis/blob/9ede220ba8553f2894bf5722a0bd83ea6f4a9bf6/docs/HOSTED_PLANS.md) +- [Configuration reference](https://github.com/Coding-Dev-Tools/engraphis/blob/12d2e405ed4bd78917a3e5eef2cf0d3a7f1561b2/docs/CONFIGURATION.md) +- [MCP tool reference](https://github.com/Coding-Dev-Tools/engraphis/blob/12d2e405ed4bd78917a3e5eef2cf0d3a7f1561b2/docs/MCP_TOOLS.md) +- [Agent and LLM provider setup](https://github.com/Coding-Dev-Tools/engraphis/blob/12d2e405ed4bd78917a3e5eef2cf0d3a7f1561b2/docs/LLM_PROVIDERS.md) +- [Architecture and query planning](https://github.com/Coding-Dev-Tools/engraphis/blob/12d2e405ed4bd78917a3e5eef2cf0d3a7f1561b2/docs/ARCHITECTURE_V3.md#query-planning) +- [Docker deployment](https://github.com/Coding-Dev-Tools/engraphis/blob/12d2e405ed4bd78917a3e5eef2cf0d3a7f1561b2/docs/DOCKER.md) +- [Document import](https://github.com/Coding-Dev-Tools/engraphis/blob/12d2e405ed4bd78917a3e5eef2cf0d3a7f1561b2/docs/DOCUMENT_IMPORT.md) +- [Cloud Sync](https://github.com/Coding-Dev-Tools/engraphis/blob/12d2e405ed4bd78917a3e5eef2cf0d3a7f1561b2/docs/SYNC.md) +- [Pi extension](https://github.com/Coding-Dev-Tools/engraphis/blob/12d2e405ed4bd78917a3e5eef2cf0d3a7f1561b2/integrations/pi/README.md) +- [Memory write review](https://github.com/Coding-Dev-Tools/engraphis/blob/12d2e405ed4bd78917a3e5eef2cf0d3a7f1561b2/docs/WRITE_REVIEW.md) +- [Security policy](https://github.com/Coding-Dev-Tools/engraphis/blob/12d2e405ed4bd78917a3e5eef2cf0d3a7f1561b2/SECURITY.md) +- [Hosted plans and licensing](https://github.com/Coding-Dev-Tools/engraphis/blob/12d2e405ed4bd78917a3e5eef2cf0d3a7f1561b2/docs/HOSTED_PLANS.md) ## License -Engraphis is licensed under Apache-2.0. The license does not grant trademark rights. See [LICENSE](https://github.com/Coding-Dev-Tools/engraphis/blob/9ede220ba8553f2894bf5722a0bd83ea6f4a9bf6/LICENSE), [NOTICE](https://github.com/Coding-Dev-Tools/engraphis/blob/9ede220ba8553f2894bf5722a0bd83ea6f4a9bf6/NOTICE), and the [licensing guide](https://github.com/Coding-Dev-Tools/engraphis/blob/9ede220ba8553f2894bf5722a0bd83ea6f4a9bf6/docs/LICENSING.md). The hosted control plane and managed services are private services. +Engraphis is licensed under Apache-2.0. The license does not grant trademark rights. See [LICENSE](https://github.com/Coding-Dev-Tools/engraphis/blob/12d2e405ed4bd78917a3e5eef2cf0d3a7f1561b2/LICENSE), [NOTICE](https://github.com/Coding-Dev-Tools/engraphis/blob/12d2e405ed4bd78917a3e5eef2cf0d3a7f1561b2/NOTICE), and the [licensing guide](https://github.com/Coding-Dev-Tools/engraphis/blob/12d2e405ed4bd78917a3e5eef2cf0d3a7f1561b2/docs/LICENSING.md). The hosted control plane and managed services are private services. diff --git a/tests/test_commercial_hardening.py b/tests/test_commercial_hardening.py index 2be80dc4..6fe9b0b6 100644 --- a/tests/test_commercial_hardening.py +++ b/tests/test_commercial_hardening.py @@ -275,7 +275,7 @@ def test_the_published_prices_match_the_manifest_where_pricing_is_documented() - monthly = "$%d" % manifest["plans"][plan]["monthly_usd"] annual = "$%d" % manifest["plans"][plan]["annual_usd"] assert ( - "[Hosted plans and licensing](https://github.com/Coding-Dev-Tools/engraphis/blob/9ede220ba8553f2894bf5722a0bd83ea6f4a9bf6/" + "[Hosted plans and licensing](https://github.com/Coding-Dev-Tools/engraphis/blob/12d2e405ed4bd78917a3e5eef2cf0d3a7f1561b2/" "docs/HOSTED_PLANS.md)" in readme ) assert monthly in hosted_plans and annual in hosted_plans, plan diff --git a/tests/test_documentation_contracts.py b/tests/test_documentation_contracts.py index da8e75d1..ddfdec92 100644 --- a/tests/test_documentation_contracts.py +++ b/tests/test_documentation_contracts.py @@ -13,7 +13,7 @@ ROOT = Path(__file__).resolve().parents[1] -README_BENCHMARK_PIN = "9ede220ba8553f2894bf5722a0bd83ea6f4a9bf6" +README_BENCHMARK_PIN = "12d2e405ed4bd78917a3e5eef2cf0d3a7f1561b2" README_HOSTED_PLANS_PIN = README_BENCHMARK_PIN @@ -42,13 +42,13 @@ def test_readme_targets_resolve_in_the_repository() -> None: if parsed.scheme in {"http", "https"}: if parsed.netloc == "github.com": prefixes = ( - "/Coding-Dev-Tools/engraphis/blob/9ede220ba8553f2894bf5722a0bd83ea6f4a9bf6/", + "/Coding-Dev-Tools/engraphis/blob/12d2e405ed4bd78917a3e5eef2cf0d3a7f1561b2/", f"/Coding-Dev-Tools/engraphis/blob/{README_BENCHMARK_PIN}/", f"/Coding-Dev-Tools/engraphis/blob/{README_HOSTED_PLANS_PIN}/", ) elif parsed.netloc == "raw.githubusercontent.com": prefixes = ( - "/Coding-Dev-Tools/engraphis/9ede220ba8553f2894bf5722a0bd83ea6f4a9bf6/", + "/Coding-Dev-Tools/engraphis/12d2e405ed4bd78917a3e5eef2cf0d3a7f1561b2/", f"/Coding-Dev-Tools/engraphis/{README_BENCHMARK_PIN}/", ) else: @@ -283,7 +283,7 @@ def test_configuration_and_recovery_guidance_matches_public_contracts() -> None: sync = _read("docs/SYNC.md") assert ( - "[Configuration reference](https://github.com/Coding-Dev-Tools/engraphis/blob/9ede220ba8553f2894bf5722a0bd83ea6f4a9bf6/" + "[Configuration reference](https://github.com/Coding-Dev-Tools/engraphis/blob/12d2e405ed4bd78917a3e5eef2cf0d3a7f1561b2/" "docs/CONFIGURATION.md)" in readme ) for document in (configuration, security, connect, providers, sync): diff --git a/tests/test_pro_cta.py b/tests/test_pro_cta.py index a92e5746..14374280 100644 --- a/tests/test_pro_cta.py +++ b/tests/test_pro_cta.py @@ -102,7 +102,7 @@ def test_public_pro_ctas_use_documentation_attribution(): hosted_plans = (ROOT / "docs" / "HOSTED_PLANS.md").read_text(encoding="utf-8") assert ( - "[Hosted plans and licensing](https://github.com/Coding-Dev-Tools/engraphis/blob/9ede220ba8553f2894bf5722a0bd83ea6f4a9bf6/" + "[Hosted plans and licensing](https://github.com/Coding-Dev-Tools/engraphis/blob/12d2e405ed4bd78917a3e5eef2cf0d3a7f1561b2/" "docs/HOSTED_PLANS.md)" in readme ) assert "utm_medium=docs" in hosted_plans diff --git a/tests/test_provider_docs.py b/tests/test_provider_docs.py index e7c51406..080fa514 100644 --- a/tests/test_provider_docs.py +++ b/tests/test_provider_docs.py @@ -99,11 +99,11 @@ def test_readme_and_env_example_link_to_the_provider_guides(): readme = _read("README.md") provider_guide = _read("docs/LLM_PROVIDERS.md") assert ( - "[Agent and LLM provider setup](https://github.com/Coding-Dev-Tools/engraphis/blob/9ede220ba8553f2894bf5722a0bd83ea6f4a9bf6/" + "[Agent and LLM provider setup](https://github.com/Coding-Dev-Tools/engraphis/blob/12d2e405ed4bd78917a3e5eef2cf0d3a7f1561b2/" "docs/LLM_PROVIDERS.md)" in readme ) assert ( - "[agent setup guide](https://github.com/Coding-Dev-Tools/engraphis/blob/9ede220ba8553f2894bf5722a0bd83ea6f4a9bf6/" + "[agent setup guide](https://github.com/Coding-Dev-Tools/engraphis/blob/12d2e405ed4bd78917a3e5eef2cf0d3a7f1561b2/" "docs/AGENT_CONNECT.md)" in readme ) assert "Ollama" in provider_guide diff --git a/tests/test_release_infrastructure.py b/tests/test_release_infrastructure.py index 95ba34b3..f02f63d9 100644 --- a/tests/test_release_infrastructure.py +++ b/tests/test_release_infrastructure.py @@ -95,7 +95,7 @@ def test_all_public_launchers_converge_on_the_v2_service(): assert '"url": "http://:8700/mcp/"' in docker_docs assert '".[server,mcp,documents,cloud-sync]"' in dockerfile assert ( - "[Docker deployment](https://github.com/Coding-Dev-Tools/engraphis/blob/9ede220ba8553f2894bf5722a0bd83ea6f4a9bf6/" + "[Docker deployment](https://github.com/Coding-Dev-Tools/engraphis/blob/12d2e405ed4bd78917a3e5eef2cf0d3a7f1561b2/" "docs/DOCKER.md)" in readme ) @@ -125,7 +125,7 @@ def test_advanced_query_planning_stays_in_architecture_docs(): guidance = "`planning=\"auto\"` keeps the original query" assert ( - "[Architecture and query planning](https://github.com/Coding-Dev-Tools/engraphis/blob/9ede220ba8553f2894bf5722a0bd83ea6f4a9bf6/" + "[Architecture and query planning](https://github.com/Coding-Dev-Tools/engraphis/blob/12d2e405ed4bd78917a3e5eef2cf0d3a7f1561b2/" "docs/ARCHITECTURE_V3.md#query-planning)" in readme ) @@ -140,7 +140,7 @@ def test_pi_and_public_write_review_details_stay_in_supporting_docs(): review_guide = _text("docs/WRITE_REVIEW.md") assert ( - "[Pi extension](https://github.com/Coding-Dev-Tools/engraphis/blob/9ede220ba8553f2894bf5722a0bd83ea6f4a9bf6/" + "[Pi extension](https://github.com/Coding-Dev-Tools/engraphis/blob/12d2e405ed4bd78917a3e5eef2cf0d3a7f1561b2/" "integrations/pi/README.md)" in readme ) @@ -172,7 +172,7 @@ def test_compose_keeps_container_safety_defaults_and_has_an_explicit_port_overri assert '"0.0.0.0:${ENGRAPHIS_COMPOSE_PORT:-8700}:${ENGRAPHIS_COMPOSE_PORT:-8700}"' in lan_compose assert "ENGRAPHIS_API_TOKEN: ${ENGRAPHIS_API_TOKEN:?Set a strong ENGRAPHIS_API_TOKEN for LAN use}" in lan_compose assert ( - "[Docker deployment](https://github.com/Coding-Dev-Tools/engraphis/blob/9ede220ba8553f2894bf5722a0bd83ea6f4a9bf6/" + "[Docker deployment](https://github.com/Coding-Dev-Tools/engraphis/blob/12d2e405ed4bd78917a3e5eef2cf0d3a7f1561b2/" "docs/DOCKER.md)" in readme ) @@ -559,7 +559,7 @@ def test_public_capability_and_support_docs_match_the_shipped_tree(): assert "engraphis_recall_context" in readme mcp_reference = _text("docs/MCP_TOOLS.md") assert ( - "[MCP tool reference](https://github.com/Coding-Dev-Tools/engraphis/blob/9ede220ba8553f2894bf5722a0bd83ea6f4a9bf6/" + "[MCP tool reference](https://github.com/Coding-Dev-Tools/engraphis/blob/12d2e405ed4bd78917a3e5eef2cf0d3a7f1561b2/" "docs/MCP_TOOLS.md)" in readme ) assert "`engraphis_check_update`" in mcp_reference @@ -626,7 +626,7 @@ def test_public_capability_and_support_docs_match_the_shipped_tree(): in readme ) assert ( - 'src="https://raw.githubusercontent.com/Coding-Dev-Tools/engraphis/9ede220ba8553f2894bf5722a0bd83ea6f4a9bf6/' + 'src="https://raw.githubusercontent.com/Coding-Dev-Tools/engraphis/12d2e405ed4bd78917a3e5eef2cf0d3a7f1561b2/' 'docs/images/knowledge-graph.png"' in readme ) assert re.search( diff --git a/tests/test_setup_plugin_distribution.py b/tests/test_setup_plugin_distribution.py index 9a5fb21e..ed3e29cc 100644 --- a/tests/test_setup_plugin_distribution.py +++ b/tests/test_setup_plugin_distribution.py @@ -8,9 +8,9 @@ ROOT = Path(__file__).resolve().parents[1] REPOSITORY = "Coding-Dev-Tools/engraphis" -README_BENCHMARK_PIN = "9ede220ba8553f2894bf5722a0bd83ea6f4a9bf6" +README_BENCHMARK_PIN = "12d2e405ed4bd78917a3e5eef2cf0d3a7f1561b2" README_LINK_PINS = ( - "9ede220ba8553f2894bf5722a0bd83ea6f4a9bf6", + "12d2e405ed4bd78917a3e5eef2cf0d3a7f1561b2", README_BENCHMARK_PIN, ) @@ -80,7 +80,7 @@ def test_pypi_readme_has_only_absolute_repository_assets_and_links() -> None: assert local.exists(), f"{target} maps to missing repository path {local.relative_to(ROOT)}" assert ( - f"https://raw.githubusercontent.com/{REPOSITORY}/9ede220ba8553f2894bf5722a0bd83ea6f4a9bf6/" + f"https://raw.githubusercontent.com/{REPOSITORY}/12d2e405ed4bd78917a3e5eef2cf0d3a7f1561b2/" "docs/images/knowledge-graph.png" ) in targets assert ( diff --git a/tests/test_skill_package.py b/tests/test_skill_package.py index 8a79897b..ae804d90 100644 --- a/tests/test_skill_package.py +++ b/tests/test_skill_package.py @@ -49,7 +49,7 @@ def test_portable_tool_reference_matches_registered_runtime_schemas() -> None: kilo = (ROOT / "docs" / "KILO_CODE_INTEGRATION.md").read_text(encoding="utf-8") mcp_reference = (ROOT / "docs" / "MCP_TOOLS.md").read_text(encoding="utf-8") assert ( - "[MCP tool reference](https://github.com/Coding-Dev-Tools/engraphis/blob/9ede220ba8553f2894bf5722a0bd83ea6f4a9bf6/" + "[MCP tool reference](https://github.com/Coding-Dev-Tools/engraphis/blob/12d2e405ed4bd78917a3e5eef2cf0d3a7f1561b2/" "docs/MCP_TOOLS.md)" in readme ) assert "39 direct tools" in mcp_reference From 2b860073cd8b762e36e5c17423af52071f209df7 Mon Sep 17 00:00:00 2001 From: Jaixii Date: Sun, 4 Oct 2026 06:46:50 -0400 Subject: [PATCH 07/22] Pin public Jev requirements and charts to the combined snapshot --- README.md | 36 ++++++++++++------------- tests/test_documentation_contracts.py | 10 +++---- tests/test_setup_plugin_distribution.py | 6 ++--- 3 files changed, 26 insertions(+), 26 deletions(-) diff --git a/README.md b/README.md index e10f9fa7..93a5a5e7 100644 --- a/README.md +++ b/README.md @@ -1,14 +1,14 @@ # Engraphis [![PyPI version](https://img.shields.io/pypi/v/engraphis.svg)](https://pypi.org/project/engraphis/) -[![License](https://img.shields.io/badge/license-Apache--2.0-green.svg)](https://github.com/Coding-Dev-Tools/engraphis/blob/12d2e405ed4bd78917a3e5eef2cf0d3a7f1561b2/LICENSE) +[![License](https://img.shields.io/badge/license-Apache--2.0-green.svg)](https://github.com/Coding-Dev-Tools/engraphis/blob/54eae9fc5105ab491a9aa9dc2bfeba6109baf63b/LICENSE) **Persistent, local-first memory for AI agents.** Engraphis stores scoped project knowledge, retrieves relevant evidence across vector, lexical, graph, and code search, and returns bounded context with sources an agent can inspect. The local engine uses SQLite and works offline. It keeps changes over time instead of silently replacing facts, and grounded recall cites retrieved memories or abstains when evidence is weak.

- Engraphis local knowledge graph showing relationships between remembered entities + Engraphis local knowledge graph showing relationships between remembered entities
Explore memories and their relationships in the local dashboard.

@@ -35,7 +35,7 @@ hit = memory.recall("Why did we change auth?", workspace="acme", repo="api") print(hit["context"]) ``` -Connect a coding agent over MCP with the [agent setup guide](https://github.com/Coding-Dev-Tools/engraphis/blob/12d2e405ed4bd78917a3e5eef2cf0d3a7f1561b2/docs/AGENT_CONNECT.md). +Connect a coding agent over MCP with the [agent setup guide](https://github.com/Coding-Dev-Tools/engraphis/blob/54eae9fc5105ab491a9aa9dc2bfeba6109baf63b/docs/AGENT_CONNECT.md). ## What it provides @@ -49,7 +49,7 @@ Connect a coding agent over MCP with the [agent setup guide](https://github.com/ The current registered artifact contains three deterministic offline fixture runs. It separates context size, retrieval quality, and grounded decision checks; these small fixtures do not establish general task performance.

- Three registered offline fixtures: structure-aware chunking reduced mean retrieved context from 740.3 to 214.3 tokens per question (71.1%, 18 questions); the serialized JSON-shape proxy fell from 24,590 to 11,138 tokens across 26 payload samples and 260 recalls. Candidate and packed retrieval quality (Recall@5, Hit@5, and answer-token recall) are shown separately (each 1.000). Grounded checks show 5/5 answerable queries grounded and 6/6 abstention queries rejected, including a 1/1 quarantined-evidence probe; 11/11 decisions were correct. MCP transport and provider billing were not measured. Artifact SHA-256 prefix 9449d94b7e80; see the benchmark guide for the full checksum. + Three registered offline fixtures: structure-aware chunking reduced mean retrieved context from 740.3 to 214.3 tokens per question (71.1%, 18 questions); the serialized JSON-shape proxy fell from 24,590 to 11,138 tokens across 26 payload samples and 260 recalls. Candidate and packed retrieval quality (Recall@5, Hit@5, and answer-token recall) are shown separately (each 1.000). Grounded checks show 5/5 answerable queries grounded and 6/6 abstention queries rejected, including a 1/1 quarantined-evidence probe; 11/11 decisions were correct. MCP transport and provider billing were not measured. Artifact SHA-256 prefix 9449d94b7e80; see the benchmark guide for the full checksum.
Three offline fixtures separate context reduction, candidate and packed retrieval quality, and grounded behavior. The chart shows the artifact checksum prefix; see the benchmark guide for the full checksum and reproduction steps.

@@ -60,7 +60,7 @@ The current registered artifact contains three deterministic offline fixture run | Recall payload proxy | JSON-shape proxy: 24,590 → 11,138 tokens (54.71% lower, 26 samples; 260 timed recalls) | Candidate and packed Recall@5, Hit@5, and answer-token recall are each 1.000 | | Grounded decisions | 5/5 answerable queries grounded; 6/6 abstention queries rejected, including 1/1 quarantined-evidence check | 11/11 decisions correct | -The payload figure is a serialized JSON-shape estimate, not an MCP transport measurement or provider billing total. See the [Benchmark methodology](https://github.com/Coding-Dev-Tools/engraphis/blob/12d2e405ed4bd78917a3e5eef2cf0d3a7f1561b2/BENCHMARKS.md) for artifact identity, counting methods, reproduction commands, external-evaluation boundaries, and limitations. +The payload figure is a serialized JSON-shape estimate, not an MCP transport measurement or provider billing total. See the [Benchmark methodology](https://github.com/Coding-Dev-Tools/engraphis/blob/54eae9fc5105ab491a9aa9dc2bfeba6109baf63b/BENCHMARKS.md) for artifact identity, counting methods, reproduction commands, external-evaluation boundaries, and limitations. ## Optional Jev assistance @@ -70,22 +70,22 @@ Experimental Jev recall route selection is available only with explicit BYOK. On Jev-assisted recall planning is experimental. In an exploratory comparison using 40 public synthetic tasks, Jev selected a route on all 40 calls but did not change nDCG@5, Recall@5, or answer-token coverage. This fixture does not establish a benefit on held-out user workloads, so no retrieval-quality improvement is claimed. -Managed Jev is currently `not_yet_available` pending release acceptance and capacity qualification. When released and enabled, each paid Pro user and each paid Team named seat, including viewers, receives **100 evaluated questions per rolling hour, 1,000 per rolling five hours, and 2,000 per rolling 24 hours** at no additional managed charge and without a personal provider key. Active legitimate trial/test entitlements receive the same limits. The account portal reports each member's rolling usage and next release times; admitted failures remain counted and no overage is charged. Direct BYOK is separate and may incur provider charges. All three caps apply independently to the individual; there is no monthly or Team pool. A command review evaluates two questions and consumes two uses. The unchanged production fleet guard of **100 questions/day** conflicts with this allowance at launch and can block requests earlier. Provider terms, live acceptance, and quality evaluation remain release gates; synthetic fixtures do not establish model accuracy. See [hosted plans and Jev details](https://github.com/Coding-Dev-Tools/engraphis/blob/12d2e405ed4bd78917a3e5eef2cf0d3a7f1561b2/docs/HOSTED_PLANS.md#included-system-1-decision-engine-jev). +Managed Jev is currently `not_yet_available` pending release acceptance and capacity qualification. When released and enabled, each paid Pro user and each paid Team named seat, including viewers, receives **100 evaluated questions per rolling hour, 1,000 per rolling five hours, and 2,000 per rolling 24 hours** at no additional managed charge and without a personal provider key. Active legitimate trial/test entitlements receive the same limits. The account portal reports each member's rolling usage and next release times; admitted failures remain counted and no overage is charged. Direct BYOK is separate and may incur provider charges. All three caps apply independently to the individual; there is no monthly or Team pool. A command review evaluates two questions and consumes two uses. The unchanged production fleet guard of **100 questions/day** conflicts with this allowance at launch and can block requests earlier. Provider terms, live acceptance, and quality evaluation remain release gates; synthetic fixtures do not establish model accuracy. See [hosted plans and Jev details](https://github.com/Coding-Dev-Tools/engraphis/blob/54eae9fc5105ab491a9aa9dc2bfeba6109baf63b/docs/HOSTED_PLANS.md#included-system-1-decision-engine-jev). ## Guides -- [Configuration reference](https://github.com/Coding-Dev-Tools/engraphis/blob/12d2e405ed4bd78917a3e5eef2cf0d3a7f1561b2/docs/CONFIGURATION.md) -- [MCP tool reference](https://github.com/Coding-Dev-Tools/engraphis/blob/12d2e405ed4bd78917a3e5eef2cf0d3a7f1561b2/docs/MCP_TOOLS.md) -- [Agent and LLM provider setup](https://github.com/Coding-Dev-Tools/engraphis/blob/12d2e405ed4bd78917a3e5eef2cf0d3a7f1561b2/docs/LLM_PROVIDERS.md) -- [Architecture and query planning](https://github.com/Coding-Dev-Tools/engraphis/blob/12d2e405ed4bd78917a3e5eef2cf0d3a7f1561b2/docs/ARCHITECTURE_V3.md#query-planning) -- [Docker deployment](https://github.com/Coding-Dev-Tools/engraphis/blob/12d2e405ed4bd78917a3e5eef2cf0d3a7f1561b2/docs/DOCKER.md) -- [Document import](https://github.com/Coding-Dev-Tools/engraphis/blob/12d2e405ed4bd78917a3e5eef2cf0d3a7f1561b2/docs/DOCUMENT_IMPORT.md) -- [Cloud Sync](https://github.com/Coding-Dev-Tools/engraphis/blob/12d2e405ed4bd78917a3e5eef2cf0d3a7f1561b2/docs/SYNC.md) -- [Pi extension](https://github.com/Coding-Dev-Tools/engraphis/blob/12d2e405ed4bd78917a3e5eef2cf0d3a7f1561b2/integrations/pi/README.md) -- [Memory write review](https://github.com/Coding-Dev-Tools/engraphis/blob/12d2e405ed4bd78917a3e5eef2cf0d3a7f1561b2/docs/WRITE_REVIEW.md) -- [Security policy](https://github.com/Coding-Dev-Tools/engraphis/blob/12d2e405ed4bd78917a3e5eef2cf0d3a7f1561b2/SECURITY.md) -- [Hosted plans and licensing](https://github.com/Coding-Dev-Tools/engraphis/blob/12d2e405ed4bd78917a3e5eef2cf0d3a7f1561b2/docs/HOSTED_PLANS.md) +- [Configuration reference](https://github.com/Coding-Dev-Tools/engraphis/blob/54eae9fc5105ab491a9aa9dc2bfeba6109baf63b/docs/CONFIGURATION.md) +- [MCP tool reference](https://github.com/Coding-Dev-Tools/engraphis/blob/54eae9fc5105ab491a9aa9dc2bfeba6109baf63b/docs/MCP_TOOLS.md) +- [Agent and LLM provider setup](https://github.com/Coding-Dev-Tools/engraphis/blob/54eae9fc5105ab491a9aa9dc2bfeba6109baf63b/docs/LLM_PROVIDERS.md) +- [Architecture and query planning](https://github.com/Coding-Dev-Tools/engraphis/blob/54eae9fc5105ab491a9aa9dc2bfeba6109baf63b/docs/ARCHITECTURE_V3.md#query-planning) +- [Docker deployment](https://github.com/Coding-Dev-Tools/engraphis/blob/54eae9fc5105ab491a9aa9dc2bfeba6109baf63b/docs/DOCKER.md) +- [Document import](https://github.com/Coding-Dev-Tools/engraphis/blob/54eae9fc5105ab491a9aa9dc2bfeba6109baf63b/docs/DOCUMENT_IMPORT.md) +- [Cloud Sync](https://github.com/Coding-Dev-Tools/engraphis/blob/54eae9fc5105ab491a9aa9dc2bfeba6109baf63b/docs/SYNC.md) +- [Pi extension](https://github.com/Coding-Dev-Tools/engraphis/blob/54eae9fc5105ab491a9aa9dc2bfeba6109baf63b/integrations/pi/README.md) +- [Memory write review](https://github.com/Coding-Dev-Tools/engraphis/blob/54eae9fc5105ab491a9aa9dc2bfeba6109baf63b/docs/WRITE_REVIEW.md) +- [Security policy](https://github.com/Coding-Dev-Tools/engraphis/blob/54eae9fc5105ab491a9aa9dc2bfeba6109baf63b/SECURITY.md) +- [Hosted plans and licensing](https://github.com/Coding-Dev-Tools/engraphis/blob/54eae9fc5105ab491a9aa9dc2bfeba6109baf63b/docs/HOSTED_PLANS.md) ## License -Engraphis is licensed under Apache-2.0. The license does not grant trademark rights. See [LICENSE](https://github.com/Coding-Dev-Tools/engraphis/blob/12d2e405ed4bd78917a3e5eef2cf0d3a7f1561b2/LICENSE), [NOTICE](https://github.com/Coding-Dev-Tools/engraphis/blob/12d2e405ed4bd78917a3e5eef2cf0d3a7f1561b2/NOTICE), and the [licensing guide](https://github.com/Coding-Dev-Tools/engraphis/blob/12d2e405ed4bd78917a3e5eef2cf0d3a7f1561b2/docs/LICENSING.md). The hosted control plane and managed services are private services. +Engraphis is licensed under Apache-2.0. The license does not grant trademark rights. See [LICENSE](https://github.com/Coding-Dev-Tools/engraphis/blob/54eae9fc5105ab491a9aa9dc2bfeba6109baf63b/LICENSE), [NOTICE](https://github.com/Coding-Dev-Tools/engraphis/blob/54eae9fc5105ab491a9aa9dc2bfeba6109baf63b/NOTICE), and the [licensing guide](https://github.com/Coding-Dev-Tools/engraphis/blob/54eae9fc5105ab491a9aa9dc2bfeba6109baf63b/docs/LICENSING.md). The hosted control plane and managed services are private services. diff --git a/tests/test_documentation_contracts.py b/tests/test_documentation_contracts.py index bfb3ea7e..521c0465 100644 --- a/tests/test_documentation_contracts.py +++ b/tests/test_documentation_contracts.py @@ -13,8 +13,8 @@ ROOT = Path(__file__).resolve().parents[1] -README_BENCHMARK_PIN = "12d2e405ed4bd78917a3e5eef2cf0d3a7f1561b2" -README_HOSTED_PLANS_PIN = "12d2e405ed4bd78917a3e5eef2cf0d3a7f1561b2" +README_BENCHMARK_PIN = "54eae9fc5105ab491a9aa9dc2bfeba6109baf63b" +README_HOSTED_PLANS_PIN = "54eae9fc5105ab491a9aa9dc2bfeba6109baf63b" def _read(path: str) -> str: @@ -42,13 +42,13 @@ def test_readme_targets_resolve_in_the_repository() -> None: if parsed.scheme in {"http", "https"}: if parsed.netloc == "github.com": prefixes = ( - "/Coding-Dev-Tools/engraphis/blob/12d2e405ed4bd78917a3e5eef2cf0d3a7f1561b2/", + "/Coding-Dev-Tools/engraphis/blob/54eae9fc5105ab491a9aa9dc2bfeba6109baf63b/", f"/Coding-Dev-Tools/engraphis/blob/{README_BENCHMARK_PIN}/", f"/Coding-Dev-Tools/engraphis/blob/{README_HOSTED_PLANS_PIN}/", ) elif parsed.netloc == "raw.githubusercontent.com": prefixes = ( - "/Coding-Dev-Tools/engraphis/12d2e405ed4bd78917a3e5eef2cf0d3a7f1561b2/", + "/Coding-Dev-Tools/engraphis/54eae9fc5105ab491a9aa9dc2bfeba6109baf63b/", f"/Coding-Dev-Tools/engraphis/{README_BENCHMARK_PIN}/", ) else: @@ -283,7 +283,7 @@ def test_configuration_and_recovery_guidance_matches_public_contracts() -> None: sync = _read("docs/SYNC.md") assert ( - "[Configuration reference](https://github.com/Coding-Dev-Tools/engraphis/blob/12d2e405ed4bd78917a3e5eef2cf0d3a7f1561b2/" + "[Configuration reference](https://github.com/Coding-Dev-Tools/engraphis/blob/54eae9fc5105ab491a9aa9dc2bfeba6109baf63b/" "docs/CONFIGURATION.md)" in readme ) for document in (configuration, security, connect, providers, sync): diff --git a/tests/test_setup_plugin_distribution.py b/tests/test_setup_plugin_distribution.py index ed3e29cc..2e73b33d 100644 --- a/tests/test_setup_plugin_distribution.py +++ b/tests/test_setup_plugin_distribution.py @@ -8,9 +8,9 @@ ROOT = Path(__file__).resolve().parents[1] REPOSITORY = "Coding-Dev-Tools/engraphis" -README_BENCHMARK_PIN = "12d2e405ed4bd78917a3e5eef2cf0d3a7f1561b2" +README_BENCHMARK_PIN = "54eae9fc5105ab491a9aa9dc2bfeba6109baf63b" README_LINK_PINS = ( - "12d2e405ed4bd78917a3e5eef2cf0d3a7f1561b2", + "54eae9fc5105ab491a9aa9dc2bfeba6109baf63b", README_BENCHMARK_PIN, ) @@ -80,7 +80,7 @@ def test_pypi_readme_has_only_absolute_repository_assets_and_links() -> None: assert local.exists(), f"{target} maps to missing repository path {local.relative_to(ROOT)}" assert ( - f"https://raw.githubusercontent.com/{REPOSITORY}/12d2e405ed4bd78917a3e5eef2cf0d3a7f1561b2/" + f"https://raw.githubusercontent.com/{REPOSITORY}/54eae9fc5105ab491a9aa9dc2bfeba6109baf63b/" "docs/images/knowledge-graph.png" ) in targets assert ( From b00096c26a134f99b12d162225ca3fd410656e1a Mon Sep 17 00:00:00 2001 From: Jaixii Date: Sun, 4 Oct 2026 06:50:31 -0400 Subject: [PATCH 08/22] Align all public documentation checks with the combined snapshot --- tests/test_commercial_hardening.py | 2 +- tests/test_pro_cta.py | 2 +- tests/test_provider_docs.py | 4 ++-- tests/test_release_infrastructure.py | 12 ++++++------ tests/test_skill_package.py | 2 +- 5 files changed, 11 insertions(+), 11 deletions(-) diff --git a/tests/test_commercial_hardening.py b/tests/test_commercial_hardening.py index 6fe9b0b6..85283be7 100644 --- a/tests/test_commercial_hardening.py +++ b/tests/test_commercial_hardening.py @@ -275,7 +275,7 @@ def test_the_published_prices_match_the_manifest_where_pricing_is_documented() - monthly = "$%d" % manifest["plans"][plan]["monthly_usd"] annual = "$%d" % manifest["plans"][plan]["annual_usd"] assert ( - "[Hosted plans and licensing](https://github.com/Coding-Dev-Tools/engraphis/blob/12d2e405ed4bd78917a3e5eef2cf0d3a7f1561b2/" + "[Hosted plans and licensing](https://github.com/Coding-Dev-Tools/engraphis/blob/54eae9fc5105ab491a9aa9dc2bfeba6109baf63b/" "docs/HOSTED_PLANS.md)" in readme ) assert monthly in hosted_plans and annual in hosted_plans, plan diff --git a/tests/test_pro_cta.py b/tests/test_pro_cta.py index 14374280..48981296 100644 --- a/tests/test_pro_cta.py +++ b/tests/test_pro_cta.py @@ -102,7 +102,7 @@ def test_public_pro_ctas_use_documentation_attribution(): hosted_plans = (ROOT / "docs" / "HOSTED_PLANS.md").read_text(encoding="utf-8") assert ( - "[Hosted plans and licensing](https://github.com/Coding-Dev-Tools/engraphis/blob/12d2e405ed4bd78917a3e5eef2cf0d3a7f1561b2/" + "[Hosted plans and licensing](https://github.com/Coding-Dev-Tools/engraphis/blob/54eae9fc5105ab491a9aa9dc2bfeba6109baf63b/" "docs/HOSTED_PLANS.md)" in readme ) assert "utm_medium=docs" in hosted_plans diff --git a/tests/test_provider_docs.py b/tests/test_provider_docs.py index 080fa514..51ae7ebb 100644 --- a/tests/test_provider_docs.py +++ b/tests/test_provider_docs.py @@ -99,11 +99,11 @@ def test_readme_and_env_example_link_to_the_provider_guides(): readme = _read("README.md") provider_guide = _read("docs/LLM_PROVIDERS.md") assert ( - "[Agent and LLM provider setup](https://github.com/Coding-Dev-Tools/engraphis/blob/12d2e405ed4bd78917a3e5eef2cf0d3a7f1561b2/" + "[Agent and LLM provider setup](https://github.com/Coding-Dev-Tools/engraphis/blob/54eae9fc5105ab491a9aa9dc2bfeba6109baf63b/" "docs/LLM_PROVIDERS.md)" in readme ) assert ( - "[agent setup guide](https://github.com/Coding-Dev-Tools/engraphis/blob/12d2e405ed4bd78917a3e5eef2cf0d3a7f1561b2/" + "[agent setup guide](https://github.com/Coding-Dev-Tools/engraphis/blob/54eae9fc5105ab491a9aa9dc2bfeba6109baf63b/" "docs/AGENT_CONNECT.md)" in readme ) assert "Ollama" in provider_guide diff --git a/tests/test_release_infrastructure.py b/tests/test_release_infrastructure.py index f02f63d9..3cacd424 100644 --- a/tests/test_release_infrastructure.py +++ b/tests/test_release_infrastructure.py @@ -95,7 +95,7 @@ def test_all_public_launchers_converge_on_the_v2_service(): assert '"url": "http://:8700/mcp/"' in docker_docs assert '".[server,mcp,documents,cloud-sync]"' in dockerfile assert ( - "[Docker deployment](https://github.com/Coding-Dev-Tools/engraphis/blob/12d2e405ed4bd78917a3e5eef2cf0d3a7f1561b2/" + "[Docker deployment](https://github.com/Coding-Dev-Tools/engraphis/blob/54eae9fc5105ab491a9aa9dc2bfeba6109baf63b/" "docs/DOCKER.md)" in readme ) @@ -125,7 +125,7 @@ def test_advanced_query_planning_stays_in_architecture_docs(): guidance = "`planning=\"auto\"` keeps the original query" assert ( - "[Architecture and query planning](https://github.com/Coding-Dev-Tools/engraphis/blob/12d2e405ed4bd78917a3e5eef2cf0d3a7f1561b2/" + "[Architecture and query planning](https://github.com/Coding-Dev-Tools/engraphis/blob/54eae9fc5105ab491a9aa9dc2bfeba6109baf63b/" "docs/ARCHITECTURE_V3.md#query-planning)" in readme ) @@ -140,7 +140,7 @@ def test_pi_and_public_write_review_details_stay_in_supporting_docs(): review_guide = _text("docs/WRITE_REVIEW.md") assert ( - "[Pi extension](https://github.com/Coding-Dev-Tools/engraphis/blob/12d2e405ed4bd78917a3e5eef2cf0d3a7f1561b2/" + "[Pi extension](https://github.com/Coding-Dev-Tools/engraphis/blob/54eae9fc5105ab491a9aa9dc2bfeba6109baf63b/" "integrations/pi/README.md)" in readme ) @@ -172,7 +172,7 @@ def test_compose_keeps_container_safety_defaults_and_has_an_explicit_port_overri assert '"0.0.0.0:${ENGRAPHIS_COMPOSE_PORT:-8700}:${ENGRAPHIS_COMPOSE_PORT:-8700}"' in lan_compose assert "ENGRAPHIS_API_TOKEN: ${ENGRAPHIS_API_TOKEN:?Set a strong ENGRAPHIS_API_TOKEN for LAN use}" in lan_compose assert ( - "[Docker deployment](https://github.com/Coding-Dev-Tools/engraphis/blob/12d2e405ed4bd78917a3e5eef2cf0d3a7f1561b2/" + "[Docker deployment](https://github.com/Coding-Dev-Tools/engraphis/blob/54eae9fc5105ab491a9aa9dc2bfeba6109baf63b/" "docs/DOCKER.md)" in readme ) @@ -559,7 +559,7 @@ def test_public_capability_and_support_docs_match_the_shipped_tree(): assert "engraphis_recall_context" in readme mcp_reference = _text("docs/MCP_TOOLS.md") assert ( - "[MCP tool reference](https://github.com/Coding-Dev-Tools/engraphis/blob/12d2e405ed4bd78917a3e5eef2cf0d3a7f1561b2/" + "[MCP tool reference](https://github.com/Coding-Dev-Tools/engraphis/blob/54eae9fc5105ab491a9aa9dc2bfeba6109baf63b/" "docs/MCP_TOOLS.md)" in readme ) assert "`engraphis_check_update`" in mcp_reference @@ -626,7 +626,7 @@ def test_public_capability_and_support_docs_match_the_shipped_tree(): in readme ) assert ( - 'src="https://raw.githubusercontent.com/Coding-Dev-Tools/engraphis/12d2e405ed4bd78917a3e5eef2cf0d3a7f1561b2/' + 'src="https://raw.githubusercontent.com/Coding-Dev-Tools/engraphis/54eae9fc5105ab491a9aa9dc2bfeba6109baf63b/' 'docs/images/knowledge-graph.png"' in readme ) assert re.search( diff --git a/tests/test_skill_package.py b/tests/test_skill_package.py index ae804d90..8581ef42 100644 --- a/tests/test_skill_package.py +++ b/tests/test_skill_package.py @@ -49,7 +49,7 @@ def test_portable_tool_reference_matches_registered_runtime_schemas() -> None: kilo = (ROOT / "docs" / "KILO_CODE_INTEGRATION.md").read_text(encoding="utf-8") mcp_reference = (ROOT / "docs" / "MCP_TOOLS.md").read_text(encoding="utf-8") assert ( - "[MCP tool reference](https://github.com/Coding-Dev-Tools/engraphis/blob/12d2e405ed4bd78917a3e5eef2cf0d3a7f1561b2/" + "[MCP tool reference](https://github.com/Coding-Dev-Tools/engraphis/blob/54eae9fc5105ab491a9aa9dc2bfeba6109baf63b/" "docs/MCP_TOOLS.md)" in readme ) assert "39 direct tools" in mcp_reference From 92fb4514aec8ff639c821b7c0ca69f353766dbd1 Mon Sep 17 00:00:00 2001 From: Jaixii Date: Sun, 4 Oct 2026 10:47:46 -0400 Subject: [PATCH 09/22] Fix pinned README links for current release evidence --- README.md | 8 ++++---- tests/test_documentation_contracts.py | 4 ++-- tests/test_provider_docs.py | 4 ++-- tests/test_release_infrastructure.py | 12 ++++++------ tests/test_setup_plugin_distribution.py | 4 ++-- tests/test_skill_package.py | 2 +- 6 files changed, 17 insertions(+), 17 deletions(-) diff --git a/README.md b/README.md index a2bb94bf..1ae1ced4 100644 --- a/README.md +++ b/README.md @@ -49,7 +49,7 @@ Connect a coding agent over MCP with the [agent setup guide](https://github.com/ The current registered artifact contains three deterministic offline fixture runs. It separates context size, retrieval quality, and grounded decision checks; these small fixtures do not establish general task performance.

- Three registered offline fixtures: structure-aware chunking reduced mean retrieved context from 740.3 to 214.3 tokens per question (71.1%, 18 questions); the serialized JSON-shape proxy fell from 24,590 to 11,138 tokens across 26 payload samples and 260 recalls. Candidate and packed retrieval quality (Recall@5, Hit@5, and answer-token recall) are shown separately (each 1.000). Grounded checks show 5/5 answerable queries grounded and 6/6 abstention queries rejected, including a 1/1 quarantined-evidence probe; 11/11 decisions were correct. MCP transport and provider billing were not measured. Artifact SHA-256 prefix 170642b51e55; see the benchmark guide for the full checksum. + Three registered offline fixtures: structure-aware chunking reduced mean retrieved context from 740.3 to 214.3 tokens per question (71.1%, 18 questions); the serialized JSON-shape proxy fell from 24,590 to 11,138 tokens across 26 payload samples and 260 recalls. Candidate and packed retrieval quality (Recall@5, Hit@5, and answer-token recall) are shown separately (each 1.000). Grounded checks show 5/5 answerable queries grounded and 6/6 abstention queries rejected, including a 1/1 quarantined-evidence probe; 11/11 decisions were correct. MCP transport and provider billing were not measured. Artifact SHA-256 prefix 170642b51e55; see the benchmark guide for the full checksum.
Three offline fixtures separate context reduction, candidate and packed retrieval quality, and grounded behavior. The chart shows the artifact checksum prefix; see the benchmark guide for the full checksum and reproduction steps.

@@ -60,7 +60,7 @@ The current registered artifact contains three deterministic offline fixture run | Recall payload proxy | JSON-shape proxy: 24,590 → 11,138 tokens (54.71% lower, 26 samples; 260 timed recalls) | Candidate and packed Recall@5, Hit@5, and answer-token recall are each 1.000 | | Grounded decisions | 5/5 answerable queries grounded; 6/6 abstention queries rejected, including 1/1 quarantined-evidence check | 11/11 decisions correct | -The payload figure is a serialized JSON-shape estimate, not an MCP transport measurement or provider billing total. See the [Benchmark methodology](https://github.com/Coding-Dev-Tools/engraphis/blob/0bd6a8e8f5804a664d9850c57b159d73012ce59b/BENCHMARKS.md) for artifact identity, counting methods, reproduction commands, external-evaluation boundaries, and limitations. +The payload figure is a serialized JSON-shape estimate, not an MCP transport measurement or provider billing total. See the [Benchmark methodology](https://github.com/Coding-Dev-Tools/engraphis/blob/c5827974d2ba211f49ff442640baa373723cf112/BENCHMARKS.md) for artifact identity, counting methods, reproduction commands, external-evaluation boundaries, and limitations. ## Optional Jev assistance @@ -70,7 +70,7 @@ Recall route selection is experimental and BYOK-only. Smart MCP opts in through Jev-assisted recall planning is experimental. In an exploratory comparison using 40 public synthetic tasks, Jev selected a route on all 40 calls but did not change nDCG@5, Recall@5, or answer-token coverage. This fixture does not establish a benefit on held-out user workloads, so no retrieval-quality improvement is claimed. -Managed Jev remains `not_yet_available` pending release acceptance and capacity qualification. Under the user-approved 2026-10-04 allowance, each eligible Pro member and each eligible Team seat receives its own 100 questions per rolling hour, 1,000 per rolling five hours, and 2,000 per rolling 24 hours; Team use is not pooled. Each evaluated question uses one unit, so command review uses two. Paid viewers and eligible active trial/test entitlements receive the same limits, with no extra customer charge or customer provider key. The account portal reports each member's limits and usage. The unchanged 100-questions/day production fleet guard conflicts with the per-person allowance and remains a launch blocker. See [hosted plans and Jev details](docs/HOSTED_PLANS.md#included-system-1-decision-engine-jev). +Managed Jev remains `not_yet_available` pending release acceptance and capacity qualification. Under the user-approved 2026-10-04 allowance, each eligible Pro member and each eligible Team seat receives its own 100 questions per rolling hour, 1,000 per rolling five hours, and 2,000 per rolling 24 hours; Team use is not pooled. Each evaluated question uses one unit, so command review uses two. Paid viewers and eligible active trial/test entitlements receive the same limits, with no extra customer charge or customer provider key. The account portal reports each member's limits and usage. The unchanged 100-questions/day production fleet guard conflicts with the per-person allowance and remains a launch blocker. See [hosted plans and Jev details](https://github.com/Coding-Dev-Tools/engraphis/blob/c5827974d2ba211f49ff442640baa373723cf112/docs/HOSTED_PLANS.md#included-system-1-decision-engine-jev). ## Guides @@ -84,7 +84,7 @@ Managed Jev remains `not_yet_available` pending release acceptance and capacity - [Pi extension](https://github.com/Coding-Dev-Tools/engraphis/blob/fee9d0c150c250632d8e0c0ee86c1325c9e1ee78/integrations/pi/README.md) - [Memory write review](https://github.com/Coding-Dev-Tools/engraphis/blob/fee9d0c150c250632d8e0c0ee86c1325c9e1ee78/docs/WRITE_REVIEW.md) - [Security policy](https://github.com/Coding-Dev-Tools/engraphis/blob/fee9d0c150c250632d8e0c0ee86c1325c9e1ee78/SECURITY.md) -- [Hosted plans and licensing](https://github.com/Coding-Dev-Tools/engraphis/blob/e440bf6ba0ff600648fdac6eb53dd28d6d80df24/docs/HOSTED_PLANS.md) +- [Hosted plans and licensing](https://github.com/Coding-Dev-Tools/engraphis/blob/c5827974d2ba211f49ff442640baa373723cf112/docs/HOSTED_PLANS.md) ## License diff --git a/tests/test_documentation_contracts.py b/tests/test_documentation_contracts.py index abf22d71..ea0470a3 100644 --- a/tests/test_documentation_contracts.py +++ b/tests/test_documentation_contracts.py @@ -13,8 +13,8 @@ ROOT = Path(__file__).resolve().parents[1] -README_BENCHMARK_PIN = "0bd6a8e8f5804a664d9850c57b159d73012ce59b" -README_HOSTED_PLANS_PIN = "e440bf6ba0ff600648fdac6eb53dd28d6d80df24" +README_BENCHMARK_PIN = "c5827974d2ba211f49ff442640baa373723cf112" +README_HOSTED_PLANS_PIN = "c5827974d2ba211f49ff442640baa373723cf112" def _read(path: str) -> str: diff --git a/tests/test_provider_docs.py b/tests/test_provider_docs.py index 51ae7ebb..a88fac90 100644 --- a/tests/test_provider_docs.py +++ b/tests/test_provider_docs.py @@ -99,11 +99,11 @@ def test_readme_and_env_example_link_to_the_provider_guides(): readme = _read("README.md") provider_guide = _read("docs/LLM_PROVIDERS.md") assert ( - "[Agent and LLM provider setup](https://github.com/Coding-Dev-Tools/engraphis/blob/54eae9fc5105ab491a9aa9dc2bfeba6109baf63b/" + "[Agent and LLM provider setup](https://github.com/Coding-Dev-Tools/engraphis/blob/fee9d0c150c250632d8e0c0ee86c1325c9e1ee78/" "docs/LLM_PROVIDERS.md)" in readme ) assert ( - "[agent setup guide](https://github.com/Coding-Dev-Tools/engraphis/blob/54eae9fc5105ab491a9aa9dc2bfeba6109baf63b/" + "[agent setup guide](https://github.com/Coding-Dev-Tools/engraphis/blob/fee9d0c150c250632d8e0c0ee86c1325c9e1ee78/" "docs/AGENT_CONNECT.md)" in readme ) assert "Ollama" in provider_guide diff --git a/tests/test_release_infrastructure.py b/tests/test_release_infrastructure.py index 3cacd424..f8d4c186 100644 --- a/tests/test_release_infrastructure.py +++ b/tests/test_release_infrastructure.py @@ -95,7 +95,7 @@ def test_all_public_launchers_converge_on_the_v2_service(): assert '"url": "http://:8700/mcp/"' in docker_docs assert '".[server,mcp,documents,cloud-sync]"' in dockerfile assert ( - "[Docker deployment](https://github.com/Coding-Dev-Tools/engraphis/blob/54eae9fc5105ab491a9aa9dc2bfeba6109baf63b/" + "[Docker deployment](https://github.com/Coding-Dev-Tools/engraphis/blob/fee9d0c150c250632d8e0c0ee86c1325c9e1ee78/" "docs/DOCKER.md)" in readme ) @@ -125,7 +125,7 @@ def test_advanced_query_planning_stays_in_architecture_docs(): guidance = "`planning=\"auto\"` keeps the original query" assert ( - "[Architecture and query planning](https://github.com/Coding-Dev-Tools/engraphis/blob/54eae9fc5105ab491a9aa9dc2bfeba6109baf63b/" + "[Architecture and query planning](https://github.com/Coding-Dev-Tools/engraphis/blob/fee9d0c150c250632d8e0c0ee86c1325c9e1ee78/" "docs/ARCHITECTURE_V3.md#query-planning)" in readme ) @@ -140,7 +140,7 @@ def test_pi_and_public_write_review_details_stay_in_supporting_docs(): review_guide = _text("docs/WRITE_REVIEW.md") assert ( - "[Pi extension](https://github.com/Coding-Dev-Tools/engraphis/blob/54eae9fc5105ab491a9aa9dc2bfeba6109baf63b/" + "[Pi extension](https://github.com/Coding-Dev-Tools/engraphis/blob/fee9d0c150c250632d8e0c0ee86c1325c9e1ee78/" "integrations/pi/README.md)" in readme ) @@ -172,7 +172,7 @@ def test_compose_keeps_container_safety_defaults_and_has_an_explicit_port_overri assert '"0.0.0.0:${ENGRAPHIS_COMPOSE_PORT:-8700}:${ENGRAPHIS_COMPOSE_PORT:-8700}"' in lan_compose assert "ENGRAPHIS_API_TOKEN: ${ENGRAPHIS_API_TOKEN:?Set a strong ENGRAPHIS_API_TOKEN for LAN use}" in lan_compose assert ( - "[Docker deployment](https://github.com/Coding-Dev-Tools/engraphis/blob/54eae9fc5105ab491a9aa9dc2bfeba6109baf63b/" + "[Docker deployment](https://github.com/Coding-Dev-Tools/engraphis/blob/fee9d0c150c250632d8e0c0ee86c1325c9e1ee78/" "docs/DOCKER.md)" in readme ) @@ -559,7 +559,7 @@ def test_public_capability_and_support_docs_match_the_shipped_tree(): assert "engraphis_recall_context" in readme mcp_reference = _text("docs/MCP_TOOLS.md") assert ( - "[MCP tool reference](https://github.com/Coding-Dev-Tools/engraphis/blob/54eae9fc5105ab491a9aa9dc2bfeba6109baf63b/" + "[MCP tool reference](https://github.com/Coding-Dev-Tools/engraphis/blob/fee9d0c150c250632d8e0c0ee86c1325c9e1ee78/" "docs/MCP_TOOLS.md)" in readme ) assert "`engraphis_check_update`" in mcp_reference @@ -626,7 +626,7 @@ def test_public_capability_and_support_docs_match_the_shipped_tree(): in readme ) assert ( - 'src="https://raw.githubusercontent.com/Coding-Dev-Tools/engraphis/54eae9fc5105ab491a9aa9dc2bfeba6109baf63b/' + 'src="https://raw.githubusercontent.com/Coding-Dev-Tools/engraphis/fee9d0c150c250632d8e0c0ee86c1325c9e1ee78/' 'docs/images/knowledge-graph.png"' in readme ) assert re.search( diff --git a/tests/test_setup_plugin_distribution.py b/tests/test_setup_plugin_distribution.py index 693d04bd..4cd7e59d 100644 --- a/tests/test_setup_plugin_distribution.py +++ b/tests/test_setup_plugin_distribution.py @@ -8,10 +8,10 @@ ROOT = Path(__file__).resolve().parents[1] REPOSITORY = "Coding-Dev-Tools/engraphis" -README_BENCHMARK_PIN = "0bd6a8e8f5804a664d9850c57b159d73012ce59b" +README_BENCHMARK_PIN = "c5827974d2ba211f49ff442640baa373723cf112" README_LINK_PINS = ( "fee9d0c150c250632d8e0c0ee86c1325c9e1ee78", - "e440bf6ba0ff600648fdac6eb53dd28d6d80df24", + "c5827974d2ba211f49ff442640baa373723cf112", README_BENCHMARK_PIN, ) diff --git a/tests/test_skill_package.py b/tests/test_skill_package.py index 8581ef42..5e3761fe 100644 --- a/tests/test_skill_package.py +++ b/tests/test_skill_package.py @@ -49,7 +49,7 @@ def test_portable_tool_reference_matches_registered_runtime_schemas() -> None: kilo = (ROOT / "docs" / "KILO_CODE_INTEGRATION.md").read_text(encoding="utf-8") mcp_reference = (ROOT / "docs" / "MCP_TOOLS.md").read_text(encoding="utf-8") assert ( - "[MCP tool reference](https://github.com/Coding-Dev-Tools/engraphis/blob/54eae9fc5105ab491a9aa9dc2bfeba6109baf63b/" + "[MCP tool reference](https://github.com/Coding-Dev-Tools/engraphis/blob/fee9d0c150c250632d8e0c0ee86c1325c9e1ee78/" "docs/MCP_TOOLS.md)" in readme ) assert "39 direct tools" in mcp_reference From 6a93da125e34daa8ca33d250efbf5f49895a5ad0 Mon Sep 17 00:00:00 2001 From: Jaixii Date: Sun, 4 Oct 2026 10:56:46 -0400 Subject: [PATCH 10/22] Update pinned hosted plan links in pricing tests --- tests/test_commercial_hardening.py | 2 +- tests/test_pro_cta.py | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/tests/test_commercial_hardening.py b/tests/test_commercial_hardening.py index be6632c9..e403de41 100644 --- a/tests/test_commercial_hardening.py +++ b/tests/test_commercial_hardening.py @@ -275,7 +275,7 @@ def test_the_published_prices_match_the_manifest_where_pricing_is_documented() - monthly = "$%d" % manifest["plans"][plan]["monthly_usd"] annual = "$%d" % manifest["plans"][plan]["annual_usd"] assert ( - "[Hosted plans and licensing](https://github.com/Coding-Dev-Tools/engraphis/blob/e440bf6ba0ff600648fdac6eb53dd28d6d80df24/" + "[Hosted plans and licensing](https://github.com/Coding-Dev-Tools/engraphis/blob/c5827974d2ba211f49ff442640baa373723cf112/" "docs/HOSTED_PLANS.md)" in readme ) assert monthly in hosted_plans and annual in hosted_plans, plan diff --git a/tests/test_pro_cta.py b/tests/test_pro_cta.py index 226d12f9..b6248246 100644 --- a/tests/test_pro_cta.py +++ b/tests/test_pro_cta.py @@ -102,7 +102,7 @@ def test_public_pro_ctas_use_documentation_attribution(): hosted_plans = (ROOT / "docs" / "HOSTED_PLANS.md").read_text(encoding="utf-8") assert ( - "[Hosted plans and licensing](https://github.com/Coding-Dev-Tools/engraphis/blob/e440bf6ba0ff600648fdac6eb53dd28d6d80df24/" + "[Hosted plans and licensing](https://github.com/Coding-Dev-Tools/engraphis/blob/c5827974d2ba211f49ff442640baa373723cf112/" "docs/HOSTED_PLANS.md)" in readme ) assert "utm_medium=docs" in hosted_plans From 616aa27012000401182c84e07334b231e74e6a57 Mon Sep 17 00:00:00 2001 From: Coding-Dev-Tools Date: Sun, 4 Oct 2026 11:23:12 -0400 Subject: [PATCH 11/22] docs: pin README links to the refreshed Jev snapshot --- README.md | 36 ++++++++++++------------- tests/test_commercial_hardening.py | 2 +- tests/test_documentation_contracts.py | 10 +++---- tests/test_pro_cta.py | 2 +- tests/test_provider_docs.py | 4 +-- tests/test_release_infrastructure.py | 12 ++++----- tests/test_setup_plugin_distribution.py | 2 +- tests/test_skill_package.py | 2 +- 8 files changed, 35 insertions(+), 35 deletions(-) diff --git a/README.md b/README.md index e2df18f5..3b083aac 100644 --- a/README.md +++ b/README.md @@ -1,14 +1,14 @@ # Engraphis [![PyPI version](https://img.shields.io/pypi/v/engraphis.svg)](https://pypi.org/project/engraphis/) -[![License](https://img.shields.io/badge/license-Apache--2.0-green.svg)](https://github.com/Coding-Dev-Tools/engraphis/blob/3e593bbf4ccaf7166b7429e8a9e1508476eaa846/LICENSE) +[![License](https://img.shields.io/badge/license-Apache--2.0-green.svg)](https://github.com/Coding-Dev-Tools/engraphis/blob/719e1712c3f59d3fd6836d10c2c67b18317cf1ce/LICENSE) **Persistent, local-first memory for AI agents.** Engraphis stores scoped project knowledge, retrieves relevant evidence across vector, lexical, graph, and code search, and returns bounded context with sources an agent can inspect. The local engine uses SQLite and works offline. It keeps changes over time instead of silently replacing facts, and grounded recall cites retrieved memories or abstains when evidence is weak.

- Engraphis local knowledge graph showing relationships between remembered entities + Engraphis local knowledge graph showing relationships between remembered entities
Explore memories and their relationships in the local dashboard.

@@ -35,7 +35,7 @@ hit = memory.recall("Why did we change auth?", workspace="acme", repo="api") print(hit["context"]) ``` -Connect a coding agent over MCP with the [agent setup guide](https://github.com/Coding-Dev-Tools/engraphis/blob/3e593bbf4ccaf7166b7429e8a9e1508476eaa846/docs/AGENT_CONNECT.md). +Connect a coding agent over MCP with the [agent setup guide](https://github.com/Coding-Dev-Tools/engraphis/blob/719e1712c3f59d3fd6836d10c2c67b18317cf1ce/docs/AGENT_CONNECT.md). ## What it provides @@ -49,7 +49,7 @@ Connect a coding agent over MCP with the [agent setup guide](https://github.com/ The current registered artifact contains three deterministic offline fixture runs. It separates context size, retrieval quality, and grounded decision checks; these small fixtures do not establish general task performance.

- Three registered offline fixtures: structure-aware chunking reduced mean retrieved context from 740.3 to 214.3 tokens per question (71.1%, 18 questions); the serialized JSON-shape proxy fell from 24,590 to 11,138 tokens across 26 payload samples and 260 recalls. Candidate and packed retrieval quality (Recall@5, Hit@5, and answer-token recall) are shown separately (each 1.000). Grounded checks show 5/5 answerable queries grounded and 6/6 abstention queries rejected, including a 1/1 quarantined-evidence probe; 11/11 decisions were correct. MCP transport and provider billing were not measured. Artifact SHA-256 prefix 08ed1a6451af; see the benchmark guide for the full checksum. + Three registered offline fixtures: structure-aware chunking reduced mean retrieved context from 740.3 to 214.3 tokens per question (71.1%, 18 questions); the serialized JSON-shape proxy fell from 24,590 to 11,138 tokens across 26 payload samples and 260 recalls. Candidate and packed retrieval quality (Recall@5, Hit@5, and answer-token recall) are shown separately (each 1.000). Grounded checks show 5/5 answerable queries grounded and 6/6 abstention queries rejected, including a 1/1 quarantined-evidence probe; 11/11 decisions were correct. MCP transport and provider billing were not measured. Artifact SHA-256 prefix 08ed1a6451af; see the benchmark guide for the full checksum.
Three offline fixtures separate context reduction, candidate and packed retrieval quality, and grounded behavior. The chart shows the artifact checksum prefix; see the benchmark guide for the full checksum and reproduction steps.

@@ -60,7 +60,7 @@ The current registered artifact contains three deterministic offline fixture run | Recall payload proxy | JSON-shape proxy: 24,590 → 11,138 tokens (54.71% lower, 26 samples; 260 timed recalls) | Candidate and packed Recall@5, Hit@5, and answer-token recall are each 1.000 | | Grounded decisions | 5/5 answerable queries grounded; 6/6 abstention queries rejected, including 1/1 quarantined-evidence check | 11/11 decisions correct | -The payload figure is a serialized JSON-shape estimate, not an MCP transport measurement or provider billing total. See the [Benchmark methodology](https://github.com/Coding-Dev-Tools/engraphis/blob/3e593bbf4ccaf7166b7429e8a9e1508476eaa846/BENCHMARKS.md) for artifact identity, counting methods, reproduction commands, external-evaluation boundaries, and limitations. +The payload figure is a serialized JSON-shape estimate, not an MCP transport measurement or provider billing total. See the [Benchmark methodology](https://github.com/Coding-Dev-Tools/engraphis/blob/719e1712c3f59d3fd6836d10c2c67b18317cf1ce/BENCHMARKS.md) for artifact identity, counting methods, reproduction commands, external-evaluation boundaries, and limitations. ## Optional Jev assistance @@ -70,22 +70,22 @@ Recall route selection is experimental and BYOK-only. Smart MCP opts in through Jev-assisted recall planning is experimental. In an exploratory comparison using 40 public synthetic tasks, Jev selected a route on all 40 calls but did not change nDCG@5, Recall@5, or answer-token coverage. This fixture does not establish a benefit on held-out user workloads, so no retrieval-quality improvement is claimed. -Managed Jev is currently `not_yet_available` pending release acceptance and capacity qualification. After enablement, Pro and Team include fixed command, completion, evidence-support, and contradiction decisions at no additional charge within a finite allowance. Team usage shares one pool sized by licensed seat count. The account portal reports availability and usage; no overage is charged, and service protection may pause requests. Subscribers do not need a provider key for managed use. See [hosted plans and Jev details](https://github.com/Coding-Dev-Tools/engraphis/blob/3e593bbf4ccaf7166b7429e8a9e1508476eaa846/docs/HOSTED_PLANS.md#included-system-1-decision-engine-jev). +Managed Jev is currently `not_yet_available` pending release acceptance and capacity qualification. After enablement, Pro and Team include fixed command, completion, evidence-support, and contradiction decisions at no additional charge within a finite allowance. Team usage shares one pool sized by licensed seat count. The account portal reports availability and usage; no overage is charged, and service protection may pause requests. Subscribers do not need a provider key for managed use. See [hosted plans and Jev details](https://github.com/Coding-Dev-Tools/engraphis/blob/719e1712c3f59d3fd6836d10c2c67b18317cf1ce/docs/HOSTED_PLANS.md#included-system-1-decision-engine-jev). ## Guides -- [Configuration reference](https://github.com/Coding-Dev-Tools/engraphis/blob/3e593bbf4ccaf7166b7429e8a9e1508476eaa846/docs/CONFIGURATION.md) -- [MCP tool reference](https://github.com/Coding-Dev-Tools/engraphis/blob/3e593bbf4ccaf7166b7429e8a9e1508476eaa846/docs/MCP_TOOLS.md) -- [Agent and LLM provider setup](https://github.com/Coding-Dev-Tools/engraphis/blob/3e593bbf4ccaf7166b7429e8a9e1508476eaa846/docs/LLM_PROVIDERS.md) -- [Architecture and query planning](https://github.com/Coding-Dev-Tools/engraphis/blob/3e593bbf4ccaf7166b7429e8a9e1508476eaa846/docs/ARCHITECTURE_V3.md#query-planning) -- [Docker deployment](https://github.com/Coding-Dev-Tools/engraphis/blob/3e593bbf4ccaf7166b7429e8a9e1508476eaa846/docs/DOCKER.md) -- [Document import](https://github.com/Coding-Dev-Tools/engraphis/blob/3e593bbf4ccaf7166b7429e8a9e1508476eaa846/docs/DOCUMENT_IMPORT.md) -- [Cloud Sync](https://github.com/Coding-Dev-Tools/engraphis/blob/3e593bbf4ccaf7166b7429e8a9e1508476eaa846/docs/SYNC.md) -- [Pi extension](https://github.com/Coding-Dev-Tools/engraphis/blob/3e593bbf4ccaf7166b7429e8a9e1508476eaa846/integrations/pi/README.md) -- [Memory write review](https://github.com/Coding-Dev-Tools/engraphis/blob/3e593bbf4ccaf7166b7429e8a9e1508476eaa846/docs/WRITE_REVIEW.md) -- [Security policy](https://github.com/Coding-Dev-Tools/engraphis/blob/3e593bbf4ccaf7166b7429e8a9e1508476eaa846/SECURITY.md) -- [Hosted plans and licensing](https://github.com/Coding-Dev-Tools/engraphis/blob/3e593bbf4ccaf7166b7429e8a9e1508476eaa846/docs/HOSTED_PLANS.md) +- [Configuration reference](https://github.com/Coding-Dev-Tools/engraphis/blob/719e1712c3f59d3fd6836d10c2c67b18317cf1ce/docs/CONFIGURATION.md) +- [MCP tool reference](https://github.com/Coding-Dev-Tools/engraphis/blob/719e1712c3f59d3fd6836d10c2c67b18317cf1ce/docs/MCP_TOOLS.md) +- [Agent and LLM provider setup](https://github.com/Coding-Dev-Tools/engraphis/blob/719e1712c3f59d3fd6836d10c2c67b18317cf1ce/docs/LLM_PROVIDERS.md) +- [Architecture and query planning](https://github.com/Coding-Dev-Tools/engraphis/blob/719e1712c3f59d3fd6836d10c2c67b18317cf1ce/docs/ARCHITECTURE_V3.md#query-planning) +- [Docker deployment](https://github.com/Coding-Dev-Tools/engraphis/blob/719e1712c3f59d3fd6836d10c2c67b18317cf1ce/docs/DOCKER.md) +- [Document import](https://github.com/Coding-Dev-Tools/engraphis/blob/719e1712c3f59d3fd6836d10c2c67b18317cf1ce/docs/DOCUMENT_IMPORT.md) +- [Cloud Sync](https://github.com/Coding-Dev-Tools/engraphis/blob/719e1712c3f59d3fd6836d10c2c67b18317cf1ce/docs/SYNC.md) +- [Pi extension](https://github.com/Coding-Dev-Tools/engraphis/blob/719e1712c3f59d3fd6836d10c2c67b18317cf1ce/integrations/pi/README.md) +- [Memory write review](https://github.com/Coding-Dev-Tools/engraphis/blob/719e1712c3f59d3fd6836d10c2c67b18317cf1ce/docs/WRITE_REVIEW.md) +- [Security policy](https://github.com/Coding-Dev-Tools/engraphis/blob/719e1712c3f59d3fd6836d10c2c67b18317cf1ce/SECURITY.md) +- [Hosted plans and licensing](https://github.com/Coding-Dev-Tools/engraphis/blob/719e1712c3f59d3fd6836d10c2c67b18317cf1ce/docs/HOSTED_PLANS.md) ## License -Engraphis is licensed under Apache-2.0. The license does not grant trademark rights. See [LICENSE](https://github.com/Coding-Dev-Tools/engraphis/blob/3e593bbf4ccaf7166b7429e8a9e1508476eaa846/LICENSE), [NOTICE](https://github.com/Coding-Dev-Tools/engraphis/blob/3e593bbf4ccaf7166b7429e8a9e1508476eaa846/NOTICE), and the [licensing guide](https://github.com/Coding-Dev-Tools/engraphis/blob/3e593bbf4ccaf7166b7429e8a9e1508476eaa846/docs/LICENSING.md). The hosted control plane and managed services are private services. +Engraphis is licensed under Apache-2.0. The license does not grant trademark rights. See [LICENSE](https://github.com/Coding-Dev-Tools/engraphis/blob/719e1712c3f59d3fd6836d10c2c67b18317cf1ce/LICENSE), [NOTICE](https://github.com/Coding-Dev-Tools/engraphis/blob/719e1712c3f59d3fd6836d10c2c67b18317cf1ce/NOTICE), and the [licensing guide](https://github.com/Coding-Dev-Tools/engraphis/blob/719e1712c3f59d3fd6836d10c2c67b18317cf1ce/docs/LICENSING.md). The hosted control plane and managed services are private services. diff --git a/tests/test_commercial_hardening.py b/tests/test_commercial_hardening.py index 7bfbfb07..5d7baaff 100644 --- a/tests/test_commercial_hardening.py +++ b/tests/test_commercial_hardening.py @@ -275,7 +275,7 @@ def test_the_published_prices_match_the_manifest_where_pricing_is_documented() - monthly = "$%d" % manifest["plans"][plan]["monthly_usd"] annual = "$%d" % manifest["plans"][plan]["annual_usd"] assert ( - "[Hosted plans and licensing](https://github.com/Coding-Dev-Tools/engraphis/blob/3e593bbf4ccaf7166b7429e8a9e1508476eaa846/" + "[Hosted plans and licensing](https://github.com/Coding-Dev-Tools/engraphis/blob/719e1712c3f59d3fd6836d10c2c67b18317cf1ce/" "docs/HOSTED_PLANS.md)" in readme ) assert monthly in hosted_plans and annual in hosted_plans, plan diff --git a/tests/test_documentation_contracts.py b/tests/test_documentation_contracts.py index 0d1666bc..e5e1d467 100644 --- a/tests/test_documentation_contracts.py +++ b/tests/test_documentation_contracts.py @@ -13,8 +13,8 @@ ROOT = Path(__file__).resolve().parents[1] -README_BENCHMARK_PIN = "3e593bbf4ccaf7166b7429e8a9e1508476eaa846" -README_HOSTED_PLANS_PIN = "3e593bbf4ccaf7166b7429e8a9e1508476eaa846" +README_BENCHMARK_PIN = "719e1712c3f59d3fd6836d10c2c67b18317cf1ce" +README_HOSTED_PLANS_PIN = "719e1712c3f59d3fd6836d10c2c67b18317cf1ce" def _read(path: str) -> str: @@ -42,13 +42,13 @@ def test_readme_targets_resolve_in_the_repository() -> None: if parsed.scheme in {"http", "https"}: if parsed.netloc == "github.com": prefixes = ( - "/Coding-Dev-Tools/engraphis/blob/3e593bbf4ccaf7166b7429e8a9e1508476eaa846/", + "/Coding-Dev-Tools/engraphis/blob/719e1712c3f59d3fd6836d10c2c67b18317cf1ce/", f"/Coding-Dev-Tools/engraphis/blob/{README_BENCHMARK_PIN}/", f"/Coding-Dev-Tools/engraphis/blob/{README_HOSTED_PLANS_PIN}/", ) elif parsed.netloc == "raw.githubusercontent.com": prefixes = ( - "/Coding-Dev-Tools/engraphis/3e593bbf4ccaf7166b7429e8a9e1508476eaa846/", + "/Coding-Dev-Tools/engraphis/719e1712c3f59d3fd6836d10c2c67b18317cf1ce/", f"/Coding-Dev-Tools/engraphis/{README_BENCHMARK_PIN}/", ) else: @@ -283,7 +283,7 @@ def test_configuration_and_recovery_guidance_matches_public_contracts() -> None: sync = _read("docs/SYNC.md") assert ( - "[Configuration reference](https://github.com/Coding-Dev-Tools/engraphis/blob/3e593bbf4ccaf7166b7429e8a9e1508476eaa846/" + "[Configuration reference](https://github.com/Coding-Dev-Tools/engraphis/blob/719e1712c3f59d3fd6836d10c2c67b18317cf1ce/" "docs/CONFIGURATION.md)" in readme ) for document in (configuration, security, connect, providers, sync): diff --git a/tests/test_pro_cta.py b/tests/test_pro_cta.py index aa51abeb..292aed99 100644 --- a/tests/test_pro_cta.py +++ b/tests/test_pro_cta.py @@ -102,7 +102,7 @@ def test_public_pro_ctas_use_documentation_attribution(): hosted_plans = (ROOT / "docs" / "HOSTED_PLANS.md").read_text(encoding="utf-8") assert ( - "[Hosted plans and licensing](https://github.com/Coding-Dev-Tools/engraphis/blob/3e593bbf4ccaf7166b7429e8a9e1508476eaa846/" + "[Hosted plans and licensing](https://github.com/Coding-Dev-Tools/engraphis/blob/719e1712c3f59d3fd6836d10c2c67b18317cf1ce/" "docs/HOSTED_PLANS.md)" in readme ) assert "utm_medium=docs" in hosted_plans diff --git a/tests/test_provider_docs.py b/tests/test_provider_docs.py index c4c38bf8..63ce2a5a 100644 --- a/tests/test_provider_docs.py +++ b/tests/test_provider_docs.py @@ -99,11 +99,11 @@ def test_readme_and_env_example_link_to_the_provider_guides(): readme = _read("README.md") provider_guide = _read("docs/LLM_PROVIDERS.md") assert ( - "[Agent and LLM provider setup](https://github.com/Coding-Dev-Tools/engraphis/blob/3e593bbf4ccaf7166b7429e8a9e1508476eaa846/" + "[Agent and LLM provider setup](https://github.com/Coding-Dev-Tools/engraphis/blob/719e1712c3f59d3fd6836d10c2c67b18317cf1ce/" "docs/LLM_PROVIDERS.md)" in readme ) assert ( - "[agent setup guide](https://github.com/Coding-Dev-Tools/engraphis/blob/3e593bbf4ccaf7166b7429e8a9e1508476eaa846/" + "[agent setup guide](https://github.com/Coding-Dev-Tools/engraphis/blob/719e1712c3f59d3fd6836d10c2c67b18317cf1ce/" "docs/AGENT_CONNECT.md)" in readme ) assert "Ollama" in provider_guide diff --git a/tests/test_release_infrastructure.py b/tests/test_release_infrastructure.py index 546e1998..71e6a523 100644 --- a/tests/test_release_infrastructure.py +++ b/tests/test_release_infrastructure.py @@ -95,7 +95,7 @@ def test_all_public_launchers_converge_on_the_v2_service(): assert '"url": "http://:8700/mcp/"' in docker_docs assert '".[server,mcp,documents,cloud-sync]"' in dockerfile assert ( - "[Docker deployment](https://github.com/Coding-Dev-Tools/engraphis/blob/3e593bbf4ccaf7166b7429e8a9e1508476eaa846/" + "[Docker deployment](https://github.com/Coding-Dev-Tools/engraphis/blob/719e1712c3f59d3fd6836d10c2c67b18317cf1ce/" "docs/DOCKER.md)" in readme ) @@ -125,7 +125,7 @@ def test_advanced_query_planning_stays_in_architecture_docs(): guidance = "`planning=\"auto\"` keeps the original query" assert ( - "[Architecture and query planning](https://github.com/Coding-Dev-Tools/engraphis/blob/3e593bbf4ccaf7166b7429e8a9e1508476eaa846/" + "[Architecture and query planning](https://github.com/Coding-Dev-Tools/engraphis/blob/719e1712c3f59d3fd6836d10c2c67b18317cf1ce/" "docs/ARCHITECTURE_V3.md#query-planning)" in readme ) @@ -140,7 +140,7 @@ def test_pi_and_public_write_review_details_stay_in_supporting_docs(): review_guide = _text("docs/WRITE_REVIEW.md") assert ( - "[Pi extension](https://github.com/Coding-Dev-Tools/engraphis/blob/3e593bbf4ccaf7166b7429e8a9e1508476eaa846/" + "[Pi extension](https://github.com/Coding-Dev-Tools/engraphis/blob/719e1712c3f59d3fd6836d10c2c67b18317cf1ce/" "integrations/pi/README.md)" in readme ) @@ -172,7 +172,7 @@ def test_compose_keeps_container_safety_defaults_and_has_an_explicit_port_overri assert '"0.0.0.0:${ENGRAPHIS_COMPOSE_PORT:-8700}:${ENGRAPHIS_COMPOSE_PORT:-8700}"' in lan_compose assert "ENGRAPHIS_API_TOKEN: ${ENGRAPHIS_API_TOKEN:?Set a strong ENGRAPHIS_API_TOKEN for LAN use}" in lan_compose assert ( - "[Docker deployment](https://github.com/Coding-Dev-Tools/engraphis/blob/3e593bbf4ccaf7166b7429e8a9e1508476eaa846/" + "[Docker deployment](https://github.com/Coding-Dev-Tools/engraphis/blob/719e1712c3f59d3fd6836d10c2c67b18317cf1ce/" "docs/DOCKER.md)" in readme ) @@ -559,7 +559,7 @@ def test_public_capability_and_support_docs_match_the_shipped_tree(): assert "engraphis_recall_context" in readme mcp_reference = _text("docs/MCP_TOOLS.md") assert ( - "[MCP tool reference](https://github.com/Coding-Dev-Tools/engraphis/blob/3e593bbf4ccaf7166b7429e8a9e1508476eaa846/" + "[MCP tool reference](https://github.com/Coding-Dev-Tools/engraphis/blob/719e1712c3f59d3fd6836d10c2c67b18317cf1ce/" "docs/MCP_TOOLS.md)" in readme ) assert "`engraphis_check_update`" in mcp_reference @@ -626,7 +626,7 @@ def test_public_capability_and_support_docs_match_the_shipped_tree(): in readme ) assert ( - 'src="https://raw.githubusercontent.com/Coding-Dev-Tools/engraphis/3e593bbf4ccaf7166b7429e8a9e1508476eaa846/' + 'src="https://raw.githubusercontent.com/Coding-Dev-Tools/engraphis/719e1712c3f59d3fd6836d10c2c67b18317cf1ce/' 'docs/images/knowledge-graph.png"' in readme ) assert re.search( diff --git a/tests/test_setup_plugin_distribution.py b/tests/test_setup_plugin_distribution.py index 7a45a949..663f5a3e 100644 --- a/tests/test_setup_plugin_distribution.py +++ b/tests/test_setup_plugin_distribution.py @@ -8,7 +8,7 @@ ROOT = Path(__file__).resolve().parents[1] REPOSITORY = "Coding-Dev-Tools/engraphis" -README_BENCHMARK_PIN = "3e593bbf4ccaf7166b7429e8a9e1508476eaa846" +README_BENCHMARK_PIN = "719e1712c3f59d3fd6836d10c2c67b18317cf1ce" README_LINK_PINS = (README_BENCHMARK_PIN,) diff --git a/tests/test_skill_package.py b/tests/test_skill_package.py index 0e82ef46..49e00eb6 100644 --- a/tests/test_skill_package.py +++ b/tests/test_skill_package.py @@ -49,7 +49,7 @@ def test_portable_tool_reference_matches_registered_runtime_schemas() -> None: kilo = (ROOT / "docs" / "KILO_CODE_INTEGRATION.md").read_text(encoding="utf-8") mcp_reference = (ROOT / "docs" / "MCP_TOOLS.md").read_text(encoding="utf-8") assert ( - "[MCP tool reference](https://github.com/Coding-Dev-Tools/engraphis/blob/3e593bbf4ccaf7166b7429e8a9e1508476eaa846/" + "[MCP tool reference](https://github.com/Coding-Dev-Tools/engraphis/blob/719e1712c3f59d3fd6836d10c2c67b18317cf1ce/" "docs/MCP_TOOLS.md)" in readme ) assert "39 direct tools" in mcp_reference From 667ebea5e12efe4d4a1edaa42a18c5e34af19381 Mon Sep 17 00:00:00 2001 From: Jaixii Date: Sun, 4 Oct 2026 12:50:52 -0400 Subject: [PATCH 12/22] docs: publish individual managed Jev limits --- .claude-plugin/skill-assets.sha256 | 2 +- README.md | 19 ++++++- docs/CONFIGURATION.md | 56 +++++++++++------- docs/HOSTED_PLANS.md | 34 ++++++----- docs/MCP_TOOLS.md | 31 +++++++--- docs/RELEASE_1_7_9.md | 27 +++++++-- skills/engraphis-memory/references/TOOLS.md | 26 ++++++--- tests/test_documentation_contracts.py | 63 +++++++++++++++------ 8 files changed, 185 insertions(+), 73 deletions(-) diff --git a/.claude-plugin/skill-assets.sha256 b/.claude-plugin/skill-assets.sha256 index 16583373..bb1e6b90 100644 --- a/.claude-plugin/skill-assets.sha256 +++ b/.claude-plugin/skill-assets.sha256 @@ -3,4 +3,4 @@ c5d0c26f28c9ee14092f9deaf24c98dd8bef49d971fef2b7a537ffb1ab9f2887 .claude-plugin aeee7a94671ceb306fe2d24c5acc9f2d96ad8a8e7410536566799eea6265f080 skills/engraphis-memory/SKILL.md 055655db84af07561d002f0c69744313d8413c39f3e873f941f0fa0b1e76dc66 skills/engraphis-memory/references/CONVENTIONS.md 9d090a03f5b3f36a34d91f66b72c3844591f6915755ac3a6c6ba5f1b16977de5 skills/engraphis-memory/references/SCOPING.md -5fcca0d293c02047666cea8f598fc4fc85c7d8fc38d2eb35aa8de244f2a774be skills/engraphis-memory/references/TOOLS.md +f037030b439f5846b156107d106e3aa157785e576dcd131234a2fb07eb181668 skills/engraphis-memory/references/TOOLS.md diff --git a/README.md b/README.md index 3b083aac..93cdc6ba 100644 --- a/README.md +++ b/README.md @@ -70,7 +70,24 @@ Recall route selection is experimental and BYOK-only. Smart MCP opts in through Jev-assisted recall planning is experimental. In an exploratory comparison using 40 public synthetic tasks, Jev selected a route on all 40 calls but did not change nDCG@5, Recall@5, or answer-token coverage. This fixture does not establish a benefit on held-out user workloads, so no retrieval-quality improvement is claimed. -Managed Jev is currently `not_yet_available` pending release acceptance and capacity qualification. After enablement, Pro and Team include fixed command, completion, evidence-support, and contradiction decisions at no additional charge within a finite allowance. Team usage shares one pool sized by licensed seat count. The account portal reports availability and usage; no overage is charged, and service protection may pause requests. Subscribers do not need a provider key for managed use. See [hosted plans and Jev details](https://github.com/Coding-Dev-Tools/engraphis/blob/719e1712c3f59d3fd6836d10c2c67b18317cf1ce/docs/HOSTED_PLANS.md#included-system-1-decision-engine-jev). +Managed Jev is currently `not_yet_available` pending release acceptance and +service-capacity qualification; client configuration does not enable it. After +enablement, every legitimate Pro user and each eligible Team named seat, including paid +viewers, with an active paid, trial, or test entitlement receives managed Jev at no +additional customer charge and without a personal provider key. + +Each individual receives all three independent rolling limits: **100 evaluated questions per rolling hour, 1,000 per rolling five hours, and 2,000 per rolling 24 hours**. Usage is per +person, not pooled across a Team and not monthly. Each evaluated question counts once; +if a batch is evaluated, every question counts, and command review evaluates two +questions. Admitted attempts that fail or are interrupted remain counted. No overage is +charged. The account portal reports each member's remaining usage across all three +windows; usage returns as earlier questions leave each rolling window. + +The existing production fleet guard remains 100 questions per day across the service. +That limit conflicts with these per-person rolling caps and may pause or reject requests +earlier. Resolve capacity and the fleet guard before launch. The supported decisions are +advisory; current synthetic fixtures do not demonstrate Jev accuracy or cost savings. +See [hosted plans and Jev details](docs/HOSTED_PLANS.md#included-system-1-decision-engine-jev). ## Guides diff --git a/docs/CONFIGURATION.md b/docs/CONFIGURATION.md index f13cf5f0..1f0199ee 100644 --- a/docs/CONFIGURATION.md +++ b/docs/CONFIGURATION.md @@ -44,7 +44,7 @@ control-plane, relay, compute, and worker implementations are private services. | `ENGRAPHIS_LLM_MODEL` | `gpt-4o-mini` | Model name (provider-specific) | | `ENGRAPHIS_LLM_API_KEY` | Not set | API key for chat/synthesis, `llm` / `llm_structured` extraction, and structured consolidation | | `ENGRAPHIS_LLM_BASE_URL` | Not set | Base URL for openrouter / custom OpenAI-compatible endpoints | -| `ENGRAPHIS_DECISION_BACKEND` | `none` | `none` or `local` keeps advisory decisions local; `managed` uses the saved Cloud session and included finite allowance for four concrete workflows after service acceptance; Team usage is pooled by licensed seat count. `auto` selects managed when configured and never switches to BYOK; explicit `byok` uses a personal TypeSafe key and may incur provider charges. Managed custom questions and query planning are unsupported. Legacy `typesafe`, `jev`, and `system1` mean BYOK. Remote calls also require per-call consent. | +| `ENGRAPHIS_DECISION_BACKEND` | `none` | `none` or `local` keeps advisory decisions local; `managed` uses the saved Cloud session and an included per-person rolling allowance after service acceptance; `auto` selects managed when configured and never switches to BYOK; explicit `byok` uses a personal TypeSafe key and may incur provider charges. Managed custom questions and query planning are unsupported. Legacy `typesafe`, `jev`, and `system1` mean BYOK. Remote calls also require per-call consent. | | `ENGRAPHIS_DECISION_MODEL` | `jev-1.13.0` | Pinned model accepted by the Jev transport; other model identifiers are rejected. | | `TYPESAFE_API_KEY` | Not set | Personal credential for explicit BYOK decisions; `JEV_API_KEY` is a fallback alias. Managed decisions use the saved Cloud session instead. | | `TYPESAFE_BASE_URL` | `https://api.typesafe.ai` | Direct BYOK provider origin; does not change the managed session's bound Cloud control origin. | @@ -62,29 +62,41 @@ control-plane, relay, compute, and worker implementations are private services. | `ENGRAPHIS_CLOUD_ACCESS_TOKEN` | Not set | Optional short-lived access token for ephemeral jobs | | `ENGRAPHIS_MANAGED_COMPUTE_CONSENT` | *(unset)* | Deny-only operator override: `0` pauses readable managed processing. A truthy value cannot grant approval. Each workspace requires explicit confirmation in Manage → Settings; encrypted sync is separate | -Managed Jev is currently `not_yet_available` pending release acceptance and service-capacity -qualification. After enablement, paid Pro users and paid Team named seats (including viewers) can -use four fixed advisory workflows at no additional charge within a finite allowance. Pro usage is -owner-scoped; Team questions share one organization pool sized by licensed seat count. The account -portal reports current availability and usage. No fixed quota is published, overage is not charged, -and service-protection limits may pause requests. A command review evaluates two questions; each -of the other three workflows evaluates one. Subscribers do not need a provider key for managed -use. These rules are service-enforced, not client configuration overrides. Setting a backend does -not enable the service or satisfy provider terms, release acceptance, capacity, or quality gates. -See [hosted plans](HOSTED_PLANS.md#included-system-1-decision-engine-jev). +Managed Jev is currently `not_yet_available` pending release acceptance and +service-capacity qualification; client configuration does not enable it. After +enablement, every legitimate Pro user and each eligible Team named seat, including paid +viewers, with an active paid, trial, or test entitlement receives managed Jev at no +additional customer charge and without a personal provider key. -The four managed workflows require a command, a pair of facts, query/evidence, or goal/output -context and their fixed question schemas. Arbitrary `custom` and `query_planning` payloads are -rejected before credential refresh or network requests. Experimental recall route planning -requires explicit BYOK; managed failure never silently selects a personal key. Remote consent -and `public` or `internal` classification remain mandatory, and `offline_mode=true` prevents -remote requests. Viewers can use direct Classic advisory decisions; generic Smart stateful -execution still requires admin and the private Team tool catalog is unchanged. +Each individual receives all three independent rolling limits: **100 evaluated questions per rolling hour, 1,000 per rolling five hours, and 2,000 per rolling 24 hours**. Usage is per +person, not pooled across a Team and not monthly. Each evaluated question counts once; +if a batch is evaluated, every question counts. A `guard_command` review evaluates two +questions, and each other supported workflow evaluates one. Admitted attempts that fail +or are interrupted remain counted. No overage is charged. The account portal reports +that member's remaining usage across all three windows; usage returns as earlier +questions leave each rolling window. -Managed transport callers can supply `request_key` for explicit retry deduplication. A duplicate -admitted key returns 409 without an additional provider call or usage increment; there is no -stored answer replay or automatic retry. Admitted errors remain counted. This parameter is not -an environment setting or an MCP/dashboard field. +The existing production fleet guard remains 100 questions per day across the service. It +conflicts with the per-person rolling caps and may pause or reject requests earlier, so +resolve capacity and the fleet guard before launch. No latency, accuracy, or cost-saving +guarantee is established by configuration or a successful health check. These rules are +service-enforced, not client configuration overrides. Selecting a backend does not +enable the service or satisfy release, provider-terms, capacity, or quality gates. See +[hosted plans](HOSTED_PLANS.md#included-system-1-decision-engine-jev). + +The four managed workflows require a command, a pair of facts, query/evidence, or +goal/output context and their fixed question schemas. Arbitrary `custom` and +`query_planning` payloads are rejected before credential refresh or network requests. +Experimental recall route planning requires explicit BYOK; managed failure never +silently selects a personal key. Remote consent and `public` or `internal` +classification remain mandatory, and `offline_mode=true` prevents remote requests. +Viewers can use direct Classic advisory decisions; generic Smart stateful execution +still requires admin and the private Team tool catalog is unchanged. + +Managed transport callers can supply `request_key` for explicit retry deduplication. A +duplicate admitted key returns 409 without an additional provider call or usage +increment; there is no stored answer replay or automatic retry. Admitted errors remain +counted. This parameter is not an environment setting or an MCP/dashboard field. The optional cross-encoder reranker is model- and hardware-dependent. Treat its quality and latency as deployment-specific until a versioned model identity, exact configuration, and diff --git a/docs/HOSTED_PLANS.md b/docs/HOSTED_PLANS.md index c7edc63e..cb17bb67 100644 --- a/docs/HOSTED_PLANS.md +++ b/docs/HOSTED_PLANS.md @@ -16,7 +16,7 @@ implementations are not part of this repository. | Local dashboard, memory engine, and MCP tools | Yes | Yes | Yes | | Local version history, graph, and manual consolidation | Yes | Yes | Yes | | Local workspace export | Yes | Yes | Yes | -| Advisory Jev decisions | Local heuristics; optional BYOK | Included managed decisions after service acceptance | Included managed decisions with a finite pool shared across named seats after service acceptance | +| Advisory Jev decisions | Local heuristics; optional BYOK | Included managed decisions per eligible individual after service acceptance | Included managed decisions per eligible named seat after service acceptance; no shared pool | | Hosted Cloud Sync, Analytics, and managed automation | | Yes | Yes | | Private account and billing support | | Yes | Yes | | Hosted multi-user dashboard, roles, seats, and audit export | | | Yes | @@ -26,18 +26,26 @@ Start or manage a hosted subscription in the [Engraphis account portal](https:// ## Included System 1 Decision Engine (Jev) -After release acceptance and service enablement, each Pro subscription owner and each active -Team named seat has managed Jev decisions at no additional charge, within a finite rolling -allowance. Team usage shares one pool sized by its licensed seat count. The account portal reports current -availability and usage. Some admitted requests may count if -they fail or are interrupted; no overage charge is applied. A service protection limit may pause -requests before the included allowance is exhausted. - -Managed Jev is currently `not_yet_available` pending release acceptance and service capacity -qualification; client configuration does not enable it. Subscribers do not need a personal -provider key for managed decisions. Direct BYOK is separate and may incur provider charges. No -latency, accuracy, or cost-saving guarantee is established by configuration or a successful -health check. +After release acceptance and service enablement, every legitimate Pro user and each +eligible Team named seat, including paid viewers, with an active paid, trial, or test +entitlement receives managed Jev at no additional customer charge and without a personal +provider key. + +Each individual receives all three independent rolling limits: **100 evaluated questions per rolling hour, 1,000 per rolling five hours, and 2,000 per rolling 24 hours**. All three +limits apply. Usage is per person, not pooled across a Team and not monthly. Each +evaluated question counts once; if a batch is evaluated, every question counts. A +`guard_command` review evaluates two questions, and each other supported workflow +evaluates one. Admitted attempts that fail or are interrupted remain counted. No overage +is charged. The account portal reports the authenticated member's remaining use across +all three windows; usage returns as earlier questions leave each rolling window. + +Managed Jev is currently `not_yet_available` pending release acceptance and +service-capacity qualification; client configuration does not enable it. The existing +production fleet guard remains 100 questions per day across the service. It conflicts +with these individual caps and may pause or reject requests earlier, so resolve capacity +and the fleet guard before launch. Direct BYOK is separate and may incur provider +charges. Configuration or a successful health check does not establish latency, +accuracy, or cost savings. The managed transport and MCP decision route require client **1.7.9 or newer**. The published 1.7.8 client has an experimental adapter but does not provide this route. diff --git a/docs/MCP_TOOLS.md b/docs/MCP_TOOLS.md index 55bb5b46..b7787768 100644 --- a/docs/MCP_TOOLS.md +++ b/docs/MCP_TOOLS.md @@ -209,13 +209,30 @@ Managed access admits only the four concrete workflows above with their fixed qu required context. A purpose label cannot authorize arbitrary question schemas. Managed `custom` requests return `managed_operation_unsupported` without a credential refresh or network request; custom remote questions require explicit BYOK. Local custom fallback remains available. -Managed Jev remains `not_yet_available` until release acceptance and service-capacity -qualification. After enablement, Pro usage is owner-scoped and Team questions share one -organization pool sized by licensed seat count, at no additional charge within a finite allowance. -The account portal reports availability and usage; no fixed quota is published, no overage is -charged, and service-protection limits may pause requests. `guard_command` counts as two evaluated -questions; each other workflow counts as one. No personal provider key is required for managed -use. +Managed Jev is currently `not_yet_available` pending release acceptance and +service-capacity qualification; client configuration does not enable it. After +enablement, every legitimate Pro user and each eligible Team named seat, including paid +viewers, with an active paid, trial, or test entitlement receives managed Jev at no +additional customer charge and without a personal provider key. + +Each individual receives all three independent rolling limits: **100 evaluated questions per rolling hour, 1,000 per rolling five hours, and 2,000 per rolling 24 hours**. Usage is per +person, not pooled across a Team and not monthly. Each evaluated question counts once; +if a batch is evaluated, every question counts. A `guard_command` review evaluates two +questions, and each other supported workflow evaluates one. Admitted attempts that fail +or are interrupted remain counted. No overage is charged. The account portal reports +that member's remaining usage across all three windows; usage returns as earlier +questions leave each rolling window. + +The existing production fleet guard remains 100 questions per day across the service. It +conflicts with the per-person rolling caps and may pause or reject requests earlier, so +resolve capacity and the fleet guard before launch. No latency, accuracy, or cost-saving +guarantee is established by configuration or a successful health check. + +Managed access admits only `guard_command`, `classify_contradiction`, `verify_support`, +and `verify_completion`, with fixed question schemas and required context. Managed +`custom` questions and `query_planning` fail closed before credential refresh or network +requests; explicit BYOK and local custom heuristics remain separate choices. + Admitted errors remain counted. Managed transport callers can preserve an explicit `request_key` for a retry; duplicate admitted keys return 409 without a second provider call or usage increment. No answer is stored for replay, and no automatic retry occurs. The MCP tool does not expose a diff --git a/docs/RELEASE_1_7_9.md b/docs/RELEASE_1_7_9.md index 28af430b..0c9e82a7 100644 --- a/docs/RELEASE_1_7_9.md +++ b/docs/RELEASE_1_7_9.md @@ -36,12 +36,27 @@ does not ship the managed transport or the registered MCP decision tool. `offline_mode=true` prevents remote execution. Pattern filtering cannot detect every secret in arbitrary prose. -Managed Jev remains `not_yet_available` until release acceptance and service-capacity -qualification pass. After enablement, Pro usage is owner-scoped and Team questions share a finite -organization pool sized by licensed seat count. The account portal reports availability and usage; -fixed quotas are not published and overage is not charged. A local success, fallback, health -response, or configured backend does not establish a successful managed provider request. Jev -advice does not authorize actions or replace deterministic checks. +Managed Jev is currently `not_yet_available` pending release acceptance and +service-capacity qualification; client configuration does not enable it. After +enablement, every legitimate Pro user and each eligible Team named seat, including paid +viewers, with an active paid, trial, or test entitlement receives managed Jev at no +additional customer charge and without a personal provider key. + +Each individual receives all three independent rolling limits: **100 evaluated questions per rolling hour, 1,000 per rolling five hours, and 2,000 per rolling 24 hours**. Usage is per +person, not pooled across a Team and not monthly. Each evaluated question counts once; +if a batch is evaluated, every question counts. A `guard_command` review evaluates two +questions, and each other supported workflow evaluates one. Admitted attempts that fail +or are interrupted remain counted. No overage is charged. The account portal reports +that member's remaining usage across all three windows; usage returns as earlier +questions leave each rolling window. + +The existing production fleet guard remains 100 questions per day across the service. It +conflicts with the per-person rolling caps and may pause or reject requests earlier, so +resolve capacity and the fleet guard before launch. No latency, accuracy, or cost-saving +guarantee is established by configuration or a successful health check. A successful +local fallback, health response, or configured backend does not establish a successful +managed provider request. Jev advice does not authorize actions or replace deterministic +checks. ## Qualification and publication sequence diff --git a/skills/engraphis-memory/references/TOOLS.md b/skills/engraphis-memory/references/TOOLS.md index 003a0f9a..f79a480b 100644 --- a/skills/engraphis-memory/references/TOOLS.md +++ b/skills/engraphis-memory/references/TOOLS.md @@ -697,13 +697,25 @@ Returns `{enabled, current, latest, update_available, url, notice}`. Advisory command, contradiction, support, completion, or custom checks. The default backend is local. Selecting `managed` uses the saved Engraphis Cloud session when configured; `auto` selects only managed access, and `byok` explicitly selects a personal TypeSafe key. -Managed Jev remains `not_yet_available` until release acceptance and service-capacity qualification -pass. After enablement, Pro usage is owner-scoped and Team questions share a finite organization -pool sized by licensed seat count at no additional charge. The account portal reports current -availability and usage; fixed quotas are not published and overage is not charged. Service -protection may pause requests. No latency, accuracy, or savings guarantee follows from -configuration. Smart discovery uses `engraphis_execute_action` because a remote request may -consume allowance. +Managed Jev is currently `not_yet_available` pending release acceptance and +service-capacity qualification; client configuration does not enable it. After +enablement, every legitimate Pro user and each eligible Team named seat, including paid +viewers, with an active paid, trial, or test entitlement receives managed Jev at no +additional customer charge and without a personal provider key. + +Each individual receives all three independent rolling limits: **100 evaluated questions per rolling hour, 1,000 per rolling five hours, and 2,000 per rolling 24 hours**. Usage is per +person, not pooled across a Team and not monthly. Each evaluated question counts once; +if a batch is evaluated, every question counts. A `guard_command` review evaluates two +questions, and each other supported workflow evaluates one. Admitted attempts that fail +or are interrupted remain counted. No overage is charged. The account portal reports +that member's remaining usage across all three windows; usage returns as earlier +questions leave each rolling window. + +The existing production fleet guard remains 100 questions per day across the service. It +conflicts with the per-person rolling caps and may pause or reject requests earlier, so +resolve capacity and the fleet guard before launch. No latency, accuracy, or cost-saving +guarantee is established by configuration or a successful health check. Smart discovery +uses `engraphis_execute_action` because a remote request may consume allowance. Managed access admits only `guard_command`, `classify_contradiction`, `verify_support`, and `verify_completion`, with the concrete context below and fixed question schemas. A purpose label diff --git a/tests/test_documentation_contracts.py b/tests/test_documentation_contracts.py index e5e1d467..9af4b8d3 100644 --- a/tests/test_documentation_contracts.py +++ b/tests/test_documentation_contracts.py @@ -14,7 +14,6 @@ ROOT = Path(__file__).resolve().parents[1] README_BENCHMARK_PIN = "719e1712c3f59d3fd6836d10c2c67b18317cf1ce" -README_HOSTED_PLANS_PIN = "719e1712c3f59d3fd6836d10c2c67b18317cf1ce" def _read(path: str) -> str: @@ -44,7 +43,6 @@ def test_readme_targets_resolve_in_the_repository() -> None: prefixes = ( "/Coding-Dev-Tools/engraphis/blob/719e1712c3f59d3fd6836d10c2c67b18317cf1ce/", f"/Coding-Dev-Tools/engraphis/blob/{README_BENCHMARK_PIN}/", - f"/Coding-Dev-Tools/engraphis/blob/{README_HOSTED_PLANS_PIN}/", ) elif parsed.netloc == "raw.githubusercontent.com": prefixes = ( @@ -361,22 +359,55 @@ def test_consolidation_docs_expose_only_live_public_options() -> None: assert "supersede_sources" not in document assert "supersede-sources" not in document - assert "docs/HOSTED_PLANS.md" in readme + assert "docs/HOSTED_PLANS.md#included-system-1-decision-engine-jev" in readme hosted_plan = " ".join(_read("docs/HOSTED_PLANS.md").split()) - for public_copy in (readme, hosted_plan): - assert "100 evaluated questions" not in public_copy - assert "1,000 per rolling" not in public_copy - assert "2,000 per rolling" not in public_copy - assert "finite rolling allowance" in hosted_plan - assert "Team usage shares one pool sized by its licensed seat count" in hosted_plan - assert "current availability and usage" in hosted_plan - assert "currently `not_yet_available` pending release acceptance" in hosted_plan + configuration = " ".join(_read("docs/CONFIGURATION.md").split()) + mcp_tools = " ".join(_read("docs/MCP_TOOLS.md").split()) + release = " ".join(_read("docs/RELEASE_1_7_9.md").split()) + normalized_tools = " ".join(tools.split()) + public_documents = (readme, hosted_plan, configuration, mcp_tools, release, normalized_tools) + public_copy = " ".join(" ".join(public_documents).split()) + for document in public_documents: + normalized_document = " ".join(document.split()).lower() + for limit in ( + "100 evaluated questions per rolling hour", + "1,000 per rolling five hours", + "2,000 per rolling 24 hours", + ): + assert limit in normalized_document + for invariant in ( + "not pooled across a team", + "not monthly", + "if a batch is evaluated, every question counts", + "admitted attempts that fail or are interrupted remain counted", + "the existing production fleet guard remains 100 questions per day", + "may pause or reject requests earlier", + ): + assert invariant in normalized_document + for obsolete in ( + "team usage shares one pool", + "team questions share one organization pool", + "fixed quotas are not published", + "no fixed quota is published", + "finite rolling allowance", + "monthly allowance", + ): + assert obsolete not in normalized_document + for entitlement in ( + "every legitimate pro user", + "eligible team named seat", + "paid viewers", + "trial, or test entitlement", + "no additional customer charge", + "without a personal provider key", + ): + assert entitlement in public_copy.lower() + assert "managed jev is currently" in public_copy.lower() + assert "Recall route selection is experimental and BYOK-only" in readme + assert "no retrieval-quality improvement is claimed" in readme assert "Managed Jev accepts only the fixed command-review, completion-review" in hosted_plan assert "experimental BYOK planner can reorder bounded deterministic query routes" in hosted_plan assert "no retrieval-quality improvement" in hosted_plan - assert "Team usage shares one pool sized by licensed seat count" in readme - assert "Recall route selection is experimental and BYOK-only" in readme - assert "no retrieval-quality improvement is claimed" in readme - assert "Managed Jev is currently `not_yet_available`" in readme - normalized_tools = " ".join(tools.split()) + assert "Managed `custom` questions" in mcp_tools + assert "query_planning" in normalized_tools assert "`profiles (bool, false)`; `structured (bool, false)`." in normalized_tools From 6ab916117031a8411e6fb70c3e9b5a8f7dd8fc52 Mon Sep 17 00:00:00 2001 From: Jaixii Date: Sun, 4 Oct 2026 13:50:41 -0400 Subject: [PATCH 13/22] chore: export fresh managed Jev benchmark evidence --- .github/workflows/export-offline-evidence.yml | 48 +++++++++++++++++++ 1 file changed, 48 insertions(+) create mode 100644 .github/workflows/export-offline-evidence.yml diff --git a/.github/workflows/export-offline-evidence.yml b/.github/workflows/export-offline-evidence.yml new file mode 100644 index 00000000..d1a6e216 --- /dev/null +++ b/.github/workflows/export-offline-evidence.yml @@ -0,0 +1,48 @@ +name: Temporary offline evidence export +on: + pull_request: + types: [opened, synchronize, reopened] + paths: + - ".github/workflows/export-offline-evidence.yml" + - "engraphis/**" + - "eval/**" + - "scripts/export_offline_evidence.py" +permissions: + contents: read +jobs: + export: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7 + with: + ref: ${{ github.event.pull_request.head.sha }} + - uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7 + with: + python-version: "3.11" + - run: python -m pip install numpy + - run: python -m scripts.export_offline_evidence --output docs/benchmark-evidence/offline-fixtures-v152.json + - run: python scripts/render_benchmark_report.py --report docs/benchmark-evidence/offline-fixtures-v152.json --output /tmp/context-efficiency.svg + - run: python -m scripts.render_benchmark_examples --report docs/benchmark-evidence/offline-fixtures-v152.json --output /tmp/evidence-backed-agent-examples.svg + - name: Show generated public artifacts + run: | + echo "BEGIN_OFFLINE_FIXTURES_JSON" + cat docs/benchmark-evidence/offline-fixtures-v152.json + echo "END_OFFLINE_FIXTURES_JSON" + echo "BEGIN_OFFLINE_FIXTURES_SHA" + cat docs/benchmark-evidence/offline-fixtures-v152.json.sha256 + echo "END_OFFLINE_FIXTURES_SHA" + echo "BEGIN_CONTEXT_EFFICIENCY_SVG" + cat /tmp/context-efficiency.svg + echo "END_CONTEXT_EFFICIENCY_SVG" + echo "BEGIN_EVIDENCE_BACKED_EXAMPLES_SVG" + cat /tmp/evidence-backed-agent-examples.svg + echo "END_EVIDENCE_BACKED_EXAMPLES_SVG" + - uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7 + with: + name: offline-fixtures-v152 + path: | + docs/benchmark-evidence/offline-fixtures-v152.json + docs/benchmark-evidence/offline-fixtures-v152.json.sha256 + /tmp/context-efficiency.svg + /tmp/evidence-backed-agent-examples.svg + retention-days: 1 From e69af55deea6370fabc3e3edf260a93381cc897b Mon Sep 17 00:00:00 2001 From: Jaixii Date: Sun, 4 Oct 2026 13:55:11 -0400 Subject: [PATCH 14/22] chore: regenerate managed Jev benchmark image exports --- .github/workflows/export-offline-evidence.yml | 14 +++++++++++--- 1 file changed, 11 insertions(+), 3 deletions(-) diff --git a/.github/workflows/export-offline-evidence.yml b/.github/workflows/export-offline-evidence.yml index d1a6e216..47404d6f 100644 --- a/.github/workflows/export-offline-evidence.yml +++ b/.github/workflows/export-offline-evidence.yml @@ -19,10 +19,10 @@ jobs: - uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7 with: python-version: "3.11" - - run: python -m pip install numpy + - run: python -m pip install numpy cairosvg - run: python -m scripts.export_offline_evidence --output docs/benchmark-evidence/offline-fixtures-v152.json - - run: python scripts/render_benchmark_report.py --report docs/benchmark-evidence/offline-fixtures-v152.json --output /tmp/context-efficiency.svg - - run: python -m scripts.render_benchmark_examples --report docs/benchmark-evidence/offline-fixtures-v152.json --output /tmp/evidence-backed-agent-examples.svg + - run: python scripts/render_benchmark_report.py --report docs/benchmark-evidence/offline-fixtures-v152.json --output /tmp/context-efficiency.svg --png-output /tmp/context-efficiency.png + - run: python -m scripts.render_benchmark_examples --report docs/benchmark-evidence/offline-fixtures-v152.json --output /tmp/evidence-backed-agent-examples.svg --png-output /tmp/evidence-backed-agent-examples.png - name: Show generated public artifacts run: | echo "BEGIN_OFFLINE_FIXTURES_JSON" @@ -37,6 +37,12 @@ jobs: echo "BEGIN_EVIDENCE_BACKED_EXAMPLES_SVG" cat /tmp/evidence-backed-agent-examples.svg echo "END_EVIDENCE_BACKED_EXAMPLES_SVG" + echo "BEGIN_CONTEXT_EFFICIENCY_PNG_BASE64" + base64 -w 76 /tmp/context-efficiency.png + echo "END_CONTEXT_EFFICIENCY_PNG_BASE64" + echo "BEGIN_EVIDENCE_BACKED_EXAMPLES_PNG_BASE64" + base64 -w 76 /tmp/evidence-backed-agent-examples.png + echo "END_EVIDENCE_BACKED_EXAMPLES_PNG_BASE64" - uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7 with: name: offline-fixtures-v152 @@ -45,4 +51,6 @@ jobs: docs/benchmark-evidence/offline-fixtures-v152.json.sha256 /tmp/context-efficiency.svg /tmp/evidence-backed-agent-examples.svg + /tmp/context-efficiency.png + /tmp/evidence-backed-agent-examples.png retention-days: 1 From 2a1957825206aa4e768a9bd69999b15959b7bb4b Mon Sep 17 00:00:00 2001 From: Jaixii Date: Sun, 4 Oct 2026 14:02:19 -0400 Subject: [PATCH 15/22] chore: run exact-head benchmark export on branch pushes --- .github/workflows/export-offline-evidence.yml | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/.github/workflows/export-offline-evidence.yml b/.github/workflows/export-offline-evidence.yml index 47404d6f..206ba38f 100644 --- a/.github/workflows/export-offline-evidence.yml +++ b/.github/workflows/export-offline-evidence.yml @@ -1,5 +1,13 @@ name: Temporary offline evidence export on: + push: + branches: + - codex/jev-individual-access-20261004 + paths: + - ".github/workflows/export-offline-evidence.yml" + - "engraphis/**" + - "eval/**" + - "scripts/export_offline_evidence.py" pull_request: types: [opened, synchronize, reopened] paths: From 96a9024d473bc0ce22f73d183a033fb2afabbf6b Mon Sep 17 00:00:00 2001 From: Jaixii Date: Sun, 4 Oct 2026 14:06:47 -0400 Subject: [PATCH 16/22] chore: remove superseded draft benchmark artifact --- .../offline-fixtures-v149.json | 696 ------------------ 1 file changed, 696 deletions(-) delete mode 100644 docs/benchmark-evidence/offline-fixtures-v149.json diff --git a/docs/benchmark-evidence/offline-fixtures-v149.json b/docs/benchmark-evidence/offline-fixtures-v149.json deleted file mode 100644 index 8b72785d..00000000 --- a/docs/benchmark-evidence/offline-fixtures-v149.json +++ /dev/null @@ -1,696 +0,0 @@ -{ - "environment": { - "embedding": "deterministic", - "numpy": "2.5.3", - "platform": "win32", - "python": "3.12.14", - "vector_backend": "numpy" - }, - "generated_on": "2026-10-04", - "privacy": { - "contains_answers": false, - "contains_customer_data": false, - "contains_per_record_fingerprints": false, - "contains_prompts": false, - "contains_raw_questions": false - }, - "runs": [ - { - "boundary": "Deterministic offline retrieval fixture; normalized-character token estimator; not external QA or provider billing.", - "command": "python -m eval.chunking_eval --dataset eval/datasets/longdoc.jsonl --k 5", - "config_digest": "c1c8196aa7e1568ef3844a9fb2d76b87f342c39108e32d6ad144b885a76143b8", - "config_digest_method": "sha256(UTF-8 exact command)", - "id": "offline-chunking", - "result": { - "chunked": { - "max_stored_tokens": 59, - "mean_context_tokens": 214.3, - "mean_evidence_tokens": 42.4, - "memories": 24, - "recall_at_k": 1.0 - }, - "context_reduction_pct": 71.1, - "documents": 6, - "k": 5, - "questions": 18, - "token_counter": "engraphis.chars4.v1", - "whole": { - "max_stored_tokens": 213, - "mean_context_tokens": 740.3, - "mean_evidence_tokens": 162.2, - "memories": 6, - "recall_at_k": 1.0 - } - } - }, - { - "boundary": "Deterministic offline CodeMem fixture; serialized JSON-shape payload proxies, not MCP transport responses, provider billing, or latency claims.", - "command": "python -m eval.performance --dataset eval/datasets/codemem.jsonl --k 5 --iterations 10 --json", - "config_digest": "bbe4aca81e58d4830e50a8fc7729a1d15b71d97a6299bccd79432b7f119677d7", - "config_digest_method": "sha256(UTF-8 exact command)", - "id": "offline-performance", - "result": { - "answer_token_recall": 1.0, - "compact_serialized_payload_tokens": 11138, - "dataset_cases": 14, - "full_serialized_payload_tokens": 24590, - "hit_at_k": 1.0, - "k": 5, - "max_context_tokens": 108, - "mean_context_tokens": 85.38, - "memories": 44, - "packed_quality": { - "answer_token_recall": 1.0, - "hit_at_k": 1.0, - "recall_at_k": 1.0, - "sample_count": 26 - }, - "payload_boundary": { - "kind": "serialized_json_shape_proxy", - "mcp_envelope_serialized": false, - "token_counter": "engraphis.regex.v1", - "transport_measured": false - }, - "quality_scope": { - "packed": "packed_quality fields score only chunks admitted to reader context", - "retrieved": "legacy quality fields score all candidate chunks returned before context packing" - }, - "questions": 26, - "recall_at_k": 1.0, - "saved_serialized_payload_tokens": 13452, - "serialized_payload_savings_ratio": 0.5471, - "timed_recalls": 260, - "token_budget": 1500, - "token_counter": "engraphis.regex.v1" - } - }, - { - "boundary": "Deterministic offline support/abstention fixture; not a frontier-model answer-quality score.", - "command": "python -m eval.grounded", - "config_digest": "590442e51e3642c10489165759919dc86ffac62c182937330c153e7f8d5fc26f", - "config_digest_method": "sha256(UTF-8 exact command)", - "id": "offline-grounded", - "result": { - "abstained": 6, - "answerable": 5, - "decision_accuracy": 1.0, - "grounded": 5, - "off_topic": 6, - "quarantine_hits": 1, - "quarantined": 1 - } - } - ], - "schema": "engraphis-public-offline-fixtures/v1", - "suite": { - "digest": "01a40e7f2d192b78b37edf9e941bb20d2f857a83bf2d882d83e1e028be4d4aa7", - "digest_method": "sha256(canonical compact JSON mapping each sorted path to its file SHA-256)", - "files": { - "engraphis/__init__.py": "f24241e8da2ede9ccba54ce4bf7531a800b55c29b2469e5de93bd423cffbf0be", - "engraphis/ai_context.py": "4dfd5d39eb95d05c591d1981e9e855eced73232f53efd9fe9966e0e08957d030", - "engraphis/app.py": "44d68ad8c0ff46baed01978c9b04e40b8be5031609d5a69e205b7f05da3777fb", - "engraphis/backends/__init__.py": "a9f22b9278362904166614081f1df78469d453601b298ce4e8afdba8a3722b25", - "engraphis/backends/codegraph.py": "83e723a91068d23694092fbe00157fcb2597061d453eb36f955bf55bc5b4d35e", - "engraphis/backends/embedder_api.py": "56a6bceea4f757325dcea987b0339e41d3875634b1ae5103f0537a358cf5878b", - "engraphis/backends/embedder_deterministic.py": "ec8b23de7e7e8273416125f5876ca96f55e4ae7881841bae55783ab0ba9130ad", - "engraphis/backends/embedder_st.py": "e1c20fd980e07060387e3f9fa37fe02a916959fc8abce4e6067a62699c726de1", - "engraphis/backends/encrypted_db.py": "25f6c1480d296a88f317213700a8b3c81e2732399bfac464b0d893465b25e846", - "engraphis/backends/extractor.py": "f2e3455ab7f14caee1d5b5c4ef071e498e90118f1b0ddcb8510c969583b0fc57", - "engraphis/backends/graph_extractor.py": "88561efa0d3fabc447a0a005b10e36261379d46218cf62d905e6928cd2fda676", - "engraphis/backends/jev_decision.py": "5be22a463ee7737185dab89141e2da3f126d58a9c238d7c3ad07727b7722937d", - "engraphis/backends/jev_query_planner.py": "4e80d7e801b253ed4854549e3efb6f1605ed97caee94d6859ba20eecb7444260", - "engraphis/backends/jev_transport.py": "b55b37e08813b0419602a55445846cf838c4a0987b147001274dd46e161466be", - "engraphis/backends/model_source.py": "8c3c7681f95214a2bbabd8de222e5ee11f42fe13402d27365654ae75fb363d4e", - "engraphis/backends/postgres_schema.py": "8468578c3add701d30d5eaa36d768ded2375d116e55f1836e09f6107a07a267e", - "engraphis/backends/query_planner.py": "bbdd77afc9b5523421b85b2ae63c8da7f5a7b777265450e0d21708a83e7bb23c", - "engraphis/backends/reranker.py": "747761d6cbfa421388974bcfd98d844f92391d80f4bf6a4feca00b0c7a6908ca", - "engraphis/backends/resources.py": "47cc867c3aecc8bd95fa284bc5bb04715f3339c19a0a11512973ef6171c95944", - "engraphis/backends/retention.py": "381d9371e3951d762f8b55eb54711de5697642acb39de99a714f246c059ecbd0", - "engraphis/backends/sync_folder.py": "e4f70a92a17f6a365910670df041e6e3ca421d44ada2827917cd66b4dc067bfa", - "engraphis/backends/sync_relay.py": "b8b9ad265453aba17ba7c27a355e12a793469b3e44cb217943c6fad9382a3006", - "engraphis/backends/vector_numpy.py": "c598831bea547824cfe08844816fa79857d3617cb0631238f95dad955a424f72", - "engraphis/backends/vector_sqlitevec.py": "6148e14ceaacc19239b64c642a3fba0e98797c78cec356210157afadf475a08b", - "engraphis/build_info.py": "624c22471e56d4c4047160808c4245488292af611564d1a63ca437605bbb414f", - "engraphis/classic_assets/__init__.py": "a7c1d52b285e3faa670ce231814b5758754aa0fbd05e1428e74c20c3ec51a4f1", - "engraphis/cloud_authz.py": "e80500579cb3a1d5fbf30814dc94e3e3967e50b311e8ed2fa56afcc13f7eb565", - "engraphis/cloud_features.py": "90e876f8993d99f01760cd01ae9fb63140e3f65036706e8fb0c8224560416330", - "engraphis/cloud_session.py": "cc6ab27c6af69b5c676e721ab06c520730e205f5bbffe27818650edfb71d4b63", - "engraphis/commercial.py": "184f312066a9e682e51a0abeff042f1c0e8eed2d47470157b23930b5a17633aa", - "engraphis/config.py": "9af2072734f0e0fb409ce3cd422d5a4cb9a4854c9a92d2e03d7e680e119a8fb9", - "engraphis/core/__init__.py": "dd5143729c3939237f04636f437032b1f2d3a5f7d82c91bbc2a5a283c3f0ebaa", - "engraphis/core/adaptive_context.py": "cc5ce48109bb0d5230a5b2b8424b829c853feec5b5d5b82596413f2279b0c9f9", - "engraphis/core/browsing.py": "cfae752d52ef51b17c4ffbe44dde62d5b0e1ed0ca34ec0e1788f3991d94bf05f", - "engraphis/core/codegraph_export.py": "4641074258d7b23498f92dd45053a0fbb111863eaad2001c08e5e3c2dc2fd54f", - "engraphis/core/conflicts.py": "28530be25a4af0bffd8f609b965b33ca7f93199a70887789b2148fda8a61a486", - "engraphis/core/consolidate.py": "f66eedfe08319a64b761261ccf1c99b163eba564d6d4060534bf8c575aecf232", - "engraphis/core/context.py": "8cc15746e4de88bce7d34db8c11dfaa545f17305788e2167a286179171717fb9", - "engraphis/core/diagnostics.py": "5ba449bdf5087d8ccb5808379596e695c8f405da497372e99f2bac509f96e8e4", - "engraphis/core/documents.py": "84385db39ba44e06b58b4b26dbf954228ff4abed7230f28a7280166fa6457861", - "engraphis/core/engine.py": "600e21318b95f78830210457657bbcd0d916d5d118ab736a375206fb1fb44ebf", - "engraphis/core/evidence.py": "97912b52a3d22de909218f83c09c92572ed695d7b3c2117e5330271c6d84a7bd", - "engraphis/core/fsutil.py": "6db770fa8bd3e1a57dfa70eb8e8bc46d48c2dc53ead1b0b43eeecf085ef58cfc", - "engraphis/core/graph_layers.py": "64d74ab01c77119f6343ba6f1d6a84f9653f1a5d34d47ce7966f3ac31b29d2ea", - "engraphis/core/graph_policy.py": "ec5b373d01adb2de87df31d9f543130018e9a73faaaed239a184f14a32646615", - "engraphis/core/graph_scene.py": "5bfb4dadd90aff6be89dad0384aabec533c04c9a5225d3e9d43a468152569d75", - "engraphis/core/graphrank.py": "1279a58396104d3f906bfd5ec75b32efedfefe52201467bf19d80be3517017a5", - "engraphis/core/grounded.py": "0d6b60ef16f3cd65b7032bdb60ba853dac364d0df58a74c932ecf12aec98e4b7", - "engraphis/core/ids.py": "e47eeadfaf560bc7638e1879fc491fa82d981cd42b9e4b05ef87033b2d7dfa60", - "engraphis/core/interfaces.py": "05d65c1919f3c23592f48e5f2820bbe857923fb65f5146727636eba51941271c", - "engraphis/core/mutations.py": "dbb46a97686994e1652b2698e50c3309ff428d7258d6e8e53cbad7bb55b459aa", - "engraphis/core/obsidian.py": "991267c153cb7c4c40f7fe8f50aa71688892e250aaaf383b22c9e5910dc263b7", - "engraphis/core/poisoning.py": "5bc67169ee8032f3777f2d3969dcf71821437bd473ce4f917c4b41a50e845fb6", - "engraphis/core/query_planner.py": "249062d67392ab7c203cc71e9040e99bee91bf570604e90949149a93cb652120", - "engraphis/core/read_snapshots.py": "be08e63a88bd38ed91d61b28657a65201c38994856db2798b209a73151dd202a", - "engraphis/core/recall.py": "92e3e8a01101c21b8b1179abb8d5357e33f808f190b84d83a405cda92d28b04d", - "engraphis/core/relocation.py": "86bbd292b374539b480dddc1a9ac19292fbc17ff9fa8e1fe6ff69c91ee7c8bb9", - "engraphis/core/resolve.py": "f01a6f55e44320ab04b97e516342f20155668863b2fc4765305e066d87586524", - "engraphis/core/retention_policy.py": "864c03bdb6e743cd0002c706de471e920f1fa1f1a9918ab343ef4c2042b47429", - "engraphis/core/retrieval_policy.py": "d169eb442115bd06c1e1da776fa6c1849e0795e658edd732a8d6821326b46b03", - "engraphis/core/savings.py": "cfbcfc7e476f4e28028555cd519696e23099f6210cf0b733225832aeaa0bc7dc", - "engraphis/core/schema.py": "ac273d3f0383995be815bbd866f2a36ea1398f30833aed57b8d4459afc87096b", - "engraphis/core/scoring.py": "f5b6ac291edf0968b3de83cb1951a97d5bfd8a8d079199cb2ef95bba0884c89a", - "engraphis/core/secrets.py": "a4835ba06e2616156528df6365ca1aba6cba0c97cdf834a709d2797a371099d2", - "engraphis/core/store.py": "0ba21d11b9a61b105f743aac3850f12638cc51b425ce76fb8d0c1eb27d540647", - "engraphis/core/sync.py": "69f75b50fdb1ec9352f92460c89efeec8d72ca642bd10beb29474a8c62b4f58a", - "engraphis/core/textutil.py": "acd65031729fa5d91d09527b8eb52518b83cfce77a55e6b7ae2d94686e35c1a6", - "engraphis/core/user_model.py": "3147ec8ee7cfd855783f639874b63f331cdc822bd8bd9298cfb326dd18666026", - "engraphis/core/vector_repair.py": "a8c1812de4e3ed288eda3e54a505e136d6ebd3ec296788bcbb8804b11e13cfc8", - "engraphis/core/vector_search.py": "75800052e573e9af01c6fd098eaf8647c3c45cd7eb2601dae05d42359e19b234", - "engraphis/dashboard_app.py": "9305c9c72c43027b2b79c0ce559d398fc232929968d6ea03c37065e3227c8d51", - "engraphis/dashboard_assets/__init__.py": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", - "engraphis/device_connect.py": "c8cd0a22e9fd2d92a65bd74047cc3a8b7159bfc9a297cb639c1c7f1699f8802e", - "engraphis/document_import.py": "94fa0ca340ad0ebd060b143f46657798a81c440a11b82aa46fefa83fb65295dc", - "engraphis/engines/__init__.py": "111232af583889195c5f5a60298e32484348608e81d2bdd822fd3ecd2a33c1e4", - "engraphis/engines/embedder.py": "998b65dd566966bb6581fd9f09cdf46c58a3923ef5df3073ccdaa541153a3581", - "engraphis/engines/ingest.py": "1a5d4b52c13e533864329f9fff11c0f6ebc299ff7093a9d9275e5c9626a39ce2", - "engraphis/engines/intelligence.py": "b561589b98deb98271f104dfec6276aeba13c4815e627259be5f8769f2e7ad82", - "engraphis/engines/recall.py": "f979580d065599c07acbc3add59f71e52d9a186a2104b0e257daa7959ea88d26", - "engraphis/engines/reweight.py": "91ec5815f5d356a7068c36133d24405334450f361f902f99275bed3ccebffd49", - "engraphis/engines/thoughts.py": "4adb9c8a9bcfe736cb42fff9b5ce24631da6ec473d178e83f1c176fde3b8b814", - "engraphis/factory.py": "06735a5acfc784fd1514ae42b24f4a58d041f6eeb040f8610184b458682fe030", - "engraphis/graphdata.py": "f5dc93395f52203c66deaeec8bc58d5c1f990a35a535151e0b9a84cecf7d8430", - "engraphis/hosted_client.py": "5b56b0d2384cf24ecb409090af15f09e437ec8ab5a98bed931384daff46b4f48", - "engraphis/http_deadline.py": "8f41de086f36ab39d4e6461c6284c776e3b123afee845824e22f071ca128c918", - "engraphis/http_security.py": "596981e96741fd47064d03db409605bbb435f062c0c6ef69b040aacdd8763a20", - "engraphis/inspector/__init__.py": "720cac28b8a6019d0a0c53809d5905b7d6eb9d4ecbac767505904c6ec3c39071", - "engraphis/inspector/app.py": "8aefe8a935397dde3c6466e691a7e6a7203ebe42e07aa00c332b733465e59e04", - "engraphis/licensing.py": "7e73c28b0e1c3536e2080a129af614f838d2cae3ec3a48a3d8ed7e5153c9e935", - "engraphis/llm/__init__.py": "f3096d2ddd652b99e6fc0b4c5a8786d9bbaba41b259df0b44ce93b57bde3a8b1", - "engraphis/llm/client.py": "0c84000033d3f85b0e0161f100c271c35b591e1480701455ec65c7d3f111ec42", - "engraphis/local_auth.py": "b0ad3a1926d417a2aaa6c44a7dbf6e51f575290ddfa27875b78a03db176623c3", - "engraphis/logging_setup.py": "f7d2edc756458a852e0401453e9c71b785aba08fa7859aacbdc23fb30dc7982a", - "engraphis/managed_processing.py": "6d33cdfd10800d9552fcfae3c2b071d2b39fe10fb69d1a8e5d9ed6026882197e", - "engraphis/mcp_classic_cli.py": "778122b121a1c654b8f85810fad05d9c8d7799cf13903d7dc274938b51008da0", - "engraphis/mcp_cli.py": "ed2d997438d727180842dc5fb3f6776f5a5d97974f690ee7229264be1ff61957", - "engraphis/mcp_http_cli.py": "5a04bcaae4531a6ea116a827bb65c5df3bd6d07ea336f65ff3ed45034972c396", - "engraphis/mcp_server.py": "c168fcd4850bbab65404e8af7c0af428475218fe4ec830712cc5def7df06c794", - "engraphis/models.py": "6e76e97db0aca3805c6f582ea78cc6e1c0ccd2665e16fd8ab81eb91b94141b51", - "engraphis/netutil.py": "2e0f8a9095f6f31dcb5b96d323f023b9214369d59e1c488125a7d443a55972ff", - "engraphis/observability.py": "a3a6945bf33a0d8e216da56ca7efec0031b82dc64a66cff36f0b2161f5db4981", - "engraphis/obsidian_import.py": "c7cf4b5e993afce3ffb45f01637804719cb9e7cd2461ffe629c055be5975721f", - "engraphis/private_state.py": "7485570efcaee8a1dc00b64ebfdc23178517235ae17609aa78db8bdda7e45fe3", - "engraphis/read_only_api.py": "aeea03282cfdc02074535b3ec6618b5baab0d26725be434911b777db6e7da158", - "engraphis/redirector.py": "5ba964b81f09008c9369180cb49d247519000763274aabc5e046215b12fa641d", - "engraphis/routes/__init__.py": "f0d59080212cfa0d9b50877bca28e5832d1ae917899500d52c68b1c8821231af", - "engraphis/routes/memory.py": "9ca066e1762eeeaafd9790ed57d730efcaceb57742dbe0c830ad8764500ff9e5", - "engraphis/routes/v2_api.py": "bb78559a7b11c226fc1a23440b674c156fdf3b56dad7ac4be2f71e30fbf87dec", - "engraphis/routes/vault.py": "1a7eee7c1a7c7aa11091042756aa2739a3eb963647020f40584a6a38da1988af", - "engraphis/service.py": "e4a4d3e8b3938b44e864d7e29fa8c9989b6305e8fb30c69fbbb968077a634d19", - "engraphis/service_context.py": "3de9289f49a977cdc206285ac42a9953a104eb1d1b1bc8ea77febd75c7cd82ab", - "engraphis/static/__init__.py": "1fff4c4e2554e7f5fcf3eace269feba09827524917193a4dd65df95bae64ad1f", - "engraphis/stores/__init__.py": "48ee4326c8f28eecf46f558b7aea21adb779226a7038299017ab90196d5d84be", - "engraphis/stores/graph.py": "ebf603b54cf8450e7c9a7319bd05db2f39fcda8491f5969d6af3e8da61571bc2", - "engraphis/stores/ledger.py": "df5cbb30d977decc0a9c3a2365c9c115cfbb5fe951bae48446436661c80e5d30", - "engraphis/stores/vaults.py": "2c986129b9d1e7aab33e18a3b9a278eb5ad2236895587f69b5816a5d96796bc3", - "engraphis/stores/vectors.py": "45a1baca381fc647548cc89424eb36d5853f7562c275b191b35b99339a190b7d", - "engraphis/update_check.py": "ffbf5ef682fb15177915073ebc0b5eab4dee9092bbf0f58d36a8a54c6605ccf4", - "eval/__init__.py": "639f0c6d9d6aac8ff6dc605a34a0a301058905cc53bff4eaed5912247f0e7c56", - "eval/ablation.py": "16f159dee75d2f96cc42f230c2403fa19ea0bda7091c4823660da553463a194a", - "eval/adversarial_memory_security.py": "35dd8d981bcbad50e9815465be420b05b62a9dea28a78eb6cc1e09ec51c320c4", - "eval/agent_benchmarks.py": "8e88943e45a1b2083b366cbb0329326ef918b119a2366ed395c404c2b7ef98b0", - "eval/benchmark.py": "b71832affdf87d23bc1db7b522a669a7888571b3989b03a235c70f5da9952bd6", - "eval/benchmark_analysis.py": "1200791425d029695fee0b7da1f188a8962f337aa31eb4217d50e4503641d2b3", - "eval/benchmark_campaign.py": "033dbebfb47df5fcd3a6588b29ea4ab5fd44d4387ae45334379278ab30f3e57f", - "eval/campaign_adapters.py": "2bdb51de0dcdfceb1be2a649d84541780ac4d1fc0f9b7585004d87490073213e", - "eval/campaign_api.py": "323be4e1d9520e8047ab54c9cbb158013785402b06ee4e93e55227971c54bf95", - "eval/campaign_candidate.py": "88541840d16c3b7368586ed0ee3566054214b1f594826b746b28d0e407cb812a", - "eval/campaign_continuation.py": "7f77a20ac8f85cd96729bba0cf45872c98a3421af005bd735434b7b416dd7e1f", - "eval/campaign_ledger.py": "ba3079ba541cb1f70929a76f147d2bc5e8b264103a1cd7faad97685e0ecc7790", - "eval/campaign_oracle.py": "9ea5479786efd9caa2b59e612e891a93dd34b500a2b36aa20bcf749fb6d4d48a", - "eval/campaign_storage.py": "ebcd1f4aeceaa5ff9304494c64dd31fe31637ceb107d8cc39fa292e3b14055d4", - "eval/capacity_matrix.py": "8c25bd97754c1d7a8468a687042a3c87cc9c0299cf01c19b80dacfec2be4e552", - "eval/chunking_eval.py": "a16544353940c0a8c40cea3b9932d3399b35ea5994b809b78f5dbe4a952c467f", - "eval/code_agent_ab.py": "d98bba6b77700ff6bf86ff0d2cf518a67e5a2676ef51bbddabbb2ff26e1f3aaa", - "eval/code_arm.py": "d211166fce1b8a4173848e1617873b7e84aadeff43746effe7483c54bbdb6f1d", - "eval/codex_oauth.py": "f235fca482de4201d1850bbfb583765ac5f1a057b4ad7e2d504d036bfc03f391", - "eval/coding_acceptance.py": "4b39cbcb60d7fba503cca597399cf9d04ad9567a43cdb950015ccf552e6a2773", - "eval/coding_corpus.py": "7b5205e8544578fe99d9d9cf6cfc34e40e5d1d049238d502ff7bc6f3c14948f5", - "eval/consolidation_ranking.py": "917b578d4e0bcb929bf1a1a37611acf7716a520c076abf4ade0d8d12a1c455de", - "eval/context_economy.py": "709ac7cc866855f96d7717ab2bea12e8b0d3a140d15fb978a2a929ad085931f2", - "eval/context_efficiency_guardrails.py": "22afd1a6fe17219e74701dc587ec35f569a5bf22bea270944525f51746723f14", - "eval/datasets/codemem.jsonl": "341313023c22850a2e14f02742b571ad1deca824f886a1654a59541304c01f3c", - "eval/datasets/coding_memory_v1/oracles/atlas-green--code_relationships.py": "969aa71235e0ae3cb764b9ee12b789184cedb5fea6fcc0d86e5dcb2c0498fa56", - "eval/datasets/coding_memory_v1/oracles/atlas-green--condition_values.py": "648702646fb62990168e98fba0dc6a27be128876548fa7035a59d71aee9da02d", - "eval/datasets/coding_memory_v1/oracles/atlas-green--corrections.py": "c7132c7fe39b7afbdf11065037369c0f3c38a22e69d8742e8afddfd5785d2e61", - "eval/datasets/coding_memory_v1/oracles/atlas-green--long_documents.py": "65dffb25944cb612872655f12db16af73f7a095e7abcf6a7cc5d31dc91f083b1", - "eval/datasets/coding_memory_v1/oracles/atlas-green--multilingual.py": "ff49874a691bfc30c082695ee6d3c89b9f2e1e5eecf086026110770d8e1b0452", - "eval/datasets/coding_memory_v1/oracles/atlas-green--paraphrases.py": "dbdcf5d7abf136aa5816df8465bfde842284aef5ef002e955cee05ae69d574ca", - "eval/datasets/coding_memory_v1/oracles/atlas-green--poisoning.py": "58b8ca4a37f55c4d112640414dc82620e32e643ab0aa539260884a5ecc50740f", - "eval/datasets/coding_memory_v1/oracles/atlas-green--scope_boundaries.py": "c1e92b10b40004c95abd3dfbada1a38fe0ce56300ecddca8360b51a1e6671458", - "eval/datasets/coding_memory_v1/oracles/atlas-green--temporal_history.py": "a64e4e5a29ce1040cf53e94571491325347f443d72a93f05bcf87c2878665952", - "eval/datasets/coding_memory_v1/oracles/atlas-green--unsupported_questions.py": "c8b415052ea7320e1df3c5475a676d4dd24602c45dbf1868fba6880846e029ee", - "eval/datasets/coding_memory_v1/oracles/atlas-north--code_relationships.py": "bb4305d15a81b4b4ef80b374acb59fee6fc32b213b101dd4b173ac4144d8570d", - "eval/datasets/coding_memory_v1/oracles/atlas-north--condition_values.py": "4bc5b1aff48ef6ad5ece1bd915f6949f72d365bb4815c4d82515b438bc644112", - "eval/datasets/coding_memory_v1/oracles/atlas-north--corrections.py": "7851502a9bc842f840bee1001e42140ee1df08be2730b563b1b197285d14742c", - "eval/datasets/coding_memory_v1/oracles/atlas-north--long_documents.py": "6204123eded20fc0c7aa59b312f8b5bd29aa88a720f7318e32b3c6a17651b57b", - "eval/datasets/coding_memory_v1/oracles/atlas-north--multilingual.py": "5c989b34fdad1e076b61ffb1f7a1371dc635f44a537abc743b765d5f6009004e", - "eval/datasets/coding_memory_v1/oracles/atlas-north--paraphrases.py": "fd189285d0cde9991dea461af9cd18b4203656a69da1298494fccf972ed8a58b", - "eval/datasets/coding_memory_v1/oracles/atlas-north--poisoning.py": "66305d69419736bdbb11a896543933604d898cd5394d94eedda8b92e632a09ac", - "eval/datasets/coding_memory_v1/oracles/atlas-north--scope_boundaries.py": "d9fbbe3924b5178f32f1937485af3b8db242a0e3a08c7bf41a3d2a57f34a8879", - "eval/datasets/coding_memory_v1/oracles/atlas-north--temporal_history.py": "7e7be0d6e558a71d3baad14a8812a6c998ad25e3f52c997a301333e0711f820e", - "eval/datasets/coding_memory_v1/oracles/atlas-north--unsupported_questions.py": "91f574dd678bf244a1fa6f9708e945259bb8ff8e2eb213426be5c3b187be3c3b", - "eval/datasets/coding_memory_v1/oracles/atlas-violet--code_relationships.py": "d0998e13db38d1fae9be6d254431b4fe592f89eb03ee25705f4c70985cd48c89", - "eval/datasets/coding_memory_v1/oracles/atlas-violet--condition_values.py": "f97610aee190dca03b01aa48ffe4aa3f75e099a59fdd867db98f743f4a376729", - "eval/datasets/coding_memory_v1/oracles/atlas-violet--corrections.py": "b7c31f7a9a1f978321ca5445997e0d284deea770c85f6d81a80204e548ad53ea", - "eval/datasets/coding_memory_v1/oracles/atlas-violet--long_documents.py": "20ec2b41cd3b6691164bce096dff16f7bf5be674f922cb637120c0347e60d0b6", - "eval/datasets/coding_memory_v1/oracles/atlas-violet--multilingual.py": "131e7dc978d3f2f7df395b6f1b4a9150051c0ca93c6dc5c386e58d46c8fdc573", - "eval/datasets/coding_memory_v1/oracles/atlas-violet--paraphrases.py": "1e7d9da36e6634ca2d2b11dc97468bd671df133345dcb76859e1dc3cb77dc664", - "eval/datasets/coding_memory_v1/oracles/atlas-violet--poisoning.py": "48460691ac0086c47102eec85387ac03bee9190ac83e3dafc4babc79b71d9bd7", - "eval/datasets/coding_memory_v1/oracles/atlas-violet--scope_boundaries.py": "b96207e9b1fd09f49218102a5b88fde5b9656604df4cfd314e86426e3b6d6075", - "eval/datasets/coding_memory_v1/oracles/atlas-violet--temporal_history.py": "7a196ff9a1db05b0c8f3a83086ae44cdc580dc58e4acc217d1db2575d9711b88", - "eval/datasets/coding_memory_v1/oracles/atlas-violet--unsupported_questions.py": "724564b5b75b9425b6889faae6d5e650bd5c940016c2b96054e5359d9d8b0887", - "eval/datasets/coding_memory_v1/oracles/atlas-west--code_relationships.py": "14295f735bcac4d4ae738eedfbd8fad4593afc914bc835233ec35e2263d43613", - "eval/datasets/coding_memory_v1/oracles/atlas-west--condition_values.py": "8ac70d9ca0c9122567bb9986d2cc5798104966df20e262b8087920e8fb5d82a2", - "eval/datasets/coding_memory_v1/oracles/atlas-west--corrections.py": "89d6a1a7781465c311830b7e769d2a76df41ab5c703ff662b3df1cc4c9a01668", - "eval/datasets/coding_memory_v1/oracles/atlas-west--long_documents.py": "38e67a26f69707089f852657a810ef1fca0395beb5511fff67c30f21ed63df57", - "eval/datasets/coding_memory_v1/oracles/atlas-west--multilingual.py": "bccf6579681d45940542ccb1f936ea1b225f4cf551a3ef31f3f8b1a026856e51", - "eval/datasets/coding_memory_v1/oracles/atlas-west--paraphrases.py": "89c8ce16c682553b27652e176b67a06f3228ae6209fbc8a7264260680f5651a9", - "eval/datasets/coding_memory_v1/oracles/atlas-west--poisoning.py": "fe9beb97d27eacdd87056fa1c9b04c62e2facae9354e765b912de60b35021621", - "eval/datasets/coding_memory_v1/oracles/atlas-west--scope_boundaries.py": "2eb3b1130c54644fcbade52da8f50cad1376a07b51856a962522913cbcbb7649", - "eval/datasets/coding_memory_v1/oracles/atlas-west--temporal_history.py": "116ee20af475da329cb535b0a02bd19ea1e7518b5b780ec12ba396ec896cc116", - "eval/datasets/coding_memory_v1/oracles/atlas-west--unsupported_questions.py": "080969e026215b9fe4ede6d0b4d02e9143e1293bea5d7383681e478426ddf121", - "eval/datasets/coding_memory_v1/oracles/borealis-green--code_relationships.py": "3965671e5e2432c8236c250f7982d4f478d8f1fe1268ffe57c555cf5fbbcd414", - "eval/datasets/coding_memory_v1/oracles/borealis-green--condition_values.py": "92994990327445258984ee6bf1d95f47359ff4b194e1f36e193980a523e6c5e0", - "eval/datasets/coding_memory_v1/oracles/borealis-green--corrections.py": "c5b473f962deacd6d59ea9967626e029888abbdb48f320704bcb6de876ce3f46", - "eval/datasets/coding_memory_v1/oracles/borealis-green--long_documents.py": "54aee2c1031c17b5678db2b0955eda8fafe5990aa9e52f02936584a9b75b9ab1", - "eval/datasets/coding_memory_v1/oracles/borealis-green--multilingual.py": "c458ac5c63e440e3d9486d096278759920d636f73e1901258679251afd86eb60", - "eval/datasets/coding_memory_v1/oracles/borealis-green--paraphrases.py": "250d027660f268cef1caa745cf3155de6e2f4adfaed02ba675eaa83224ffeee8", - "eval/datasets/coding_memory_v1/oracles/borealis-green--poisoning.py": "978f16eb5033c602ec09ed57ba0667edec9ac3e26accc1ce1456bc893ac9e6a7", - "eval/datasets/coding_memory_v1/oracles/borealis-green--scope_boundaries.py": "4b73a2e5dc5ac815c14e08a34acefd2ad426282cfae379ca2fd3060a3784c9b7", - "eval/datasets/coding_memory_v1/oracles/borealis-green--temporal_history.py": "6636d511bbcca51ca85c4eb0ec5ae956b3baaa5f9af89458ceb86329115743bd", - "eval/datasets/coding_memory_v1/oracles/borealis-green--unsupported_questions.py": "5cd4f70627bc60572c720a797bff122ef9bf219f32d265f694affa33b1f9f4fe", - "eval/datasets/coding_memory_v1/oracles/borealis-north--code_relationships.py": "769903aeac3ce360d845ddf1a6e89192737dbb460e1f2e251230b74d40763dd6", - "eval/datasets/coding_memory_v1/oracles/borealis-north--condition_values.py": "f6e69f525fd240a09152989b9d8a529e9abe076e7fc2c98d10bd84f9c6da2b56", - "eval/datasets/coding_memory_v1/oracles/borealis-north--corrections.py": "987ece1c7507a69a973c1e051b27c1d3b104ecda79df48959e49a62da31aa137", - "eval/datasets/coding_memory_v1/oracles/borealis-north--long_documents.py": "011134140cc486a3c089b407adbfbf5b958e1f8c1bffd572c29b8cd5d7f07f7a", - "eval/datasets/coding_memory_v1/oracles/borealis-north--multilingual.py": "1d29536a4a5836f64f56adcd1a965db85bc3371a12ad5bbb5644520c9d9606c8", - "eval/datasets/coding_memory_v1/oracles/borealis-north--paraphrases.py": "09a5fad90c24f75f2f448854c8778aab896c67854c97f884c2d689ead808583e", - "eval/datasets/coding_memory_v1/oracles/borealis-north--poisoning.py": "c1bec2ed9b31103e39db9dbab2be8db401cb0874bb3ce8573a0841a054e28a46", - "eval/datasets/coding_memory_v1/oracles/borealis-north--scope_boundaries.py": "faf51b368d7ba9406442edde29e290c931e67bff2799783dfd27f2226020e15b", - "eval/datasets/coding_memory_v1/oracles/borealis-north--temporal_history.py": "da52f43530149a4a293065674623fa49f83b37cd84b9871044b3bf32762fd8c1", - "eval/datasets/coding_memory_v1/oracles/borealis-north--unsupported_questions.py": "be0abc698692c9fe82f40f76a4694c472882a53843894dddd631ab1ab77aae45", - "eval/datasets/coding_memory_v1/oracles/borealis-violet--code_relationships.py": "9497b7d2373bb9f858e994f7919cff72e7e3c4c4eaa1efda287524b1a182d1f4", - "eval/datasets/coding_memory_v1/oracles/borealis-violet--condition_values.py": "aea13e4b7721019e7bc1f54878bb25b9f27a8a5044fe444e86ba5460deb13e09", - "eval/datasets/coding_memory_v1/oracles/borealis-violet--corrections.py": "bfd446d431afb6c05ad9d8775b90887947b85981037756e05262f3b101d41e7b", - "eval/datasets/coding_memory_v1/oracles/borealis-violet--long_documents.py": "eae829a374c0cd6237fac4ec52fec0d98d7340ec0b307b9b0b0c90c116d4b472", - "eval/datasets/coding_memory_v1/oracles/borealis-violet--multilingual.py": "e821f91999b5f12cff53fdaa956af4e0bb2f929497f95b4e2ceb57a6621e58f7", - "eval/datasets/coding_memory_v1/oracles/borealis-violet--paraphrases.py": "1a99c15898831a1c9aacab951868101d57e59e91ed90a2777015a26dbcc3043c", - "eval/datasets/coding_memory_v1/oracles/borealis-violet--poisoning.py": "411877fc07b053201a63aa4e45cb499344abccb0a562c236d53a672078b1eb63", - "eval/datasets/coding_memory_v1/oracles/borealis-violet--scope_boundaries.py": "b198dfb636910953ba372edf1cd6db382d4e5725bf7f820171be8c63f9fedf7d", - "eval/datasets/coding_memory_v1/oracles/borealis-violet--temporal_history.py": "3de5efd6ef661572191c6aa426fb10caf9f4bed4aa495b6298815aab3acdce22", - "eval/datasets/coding_memory_v1/oracles/borealis-violet--unsupported_questions.py": "baf5dd85e86a97dedb0b395b5c574ea5ffcdad7a46bbbe9787cfdc33d6c7a06c", - "eval/datasets/coding_memory_v1/oracles/borealis-west--code_relationships.py": "48e53b3a561f91fdf4eedb2d63aad65eece8ae80188bb3d5bdcaf184fb0b6945", - "eval/datasets/coding_memory_v1/oracles/borealis-west--condition_values.py": "2dd7a524bd2a2aa2fbf212246880d4e4ecf03f503b70f25179f3ab6bab28a07a", - "eval/datasets/coding_memory_v1/oracles/borealis-west--corrections.py": "7d3080e05544e10243040fe0a969a8c524e4262f8f5c38abdc3d8e06c67fdabd", - "eval/datasets/coding_memory_v1/oracles/borealis-west--long_documents.py": "fc42e935879e7a81f6eef9fe72f8683244501ae10654008a7863d397ee73058c", - "eval/datasets/coding_memory_v1/oracles/borealis-west--multilingual.py": "19181c442ce5f6941485190ecf050e455914dae988f86efa42665cdef0d835a2", - "eval/datasets/coding_memory_v1/oracles/borealis-west--paraphrases.py": "f7a5cb842aae3d2bc8d62ea40325b332a8bd4f013b76bac8bf154fedb3dbcbe1", - "eval/datasets/coding_memory_v1/oracles/borealis-west--poisoning.py": "285a2526a6be4e556a564899e9a376ae1286fef6f0ad22eb9289fba1905a2b68", - "eval/datasets/coding_memory_v1/oracles/borealis-west--scope_boundaries.py": "fb994823743ff5b4853b142c17779778bdb11296958e033e5c6be598833a4cf5", - "eval/datasets/coding_memory_v1/oracles/borealis-west--temporal_history.py": "f30a74c85ad654a7450e20d07baceb3b4fb2a56b454d7b43b854bef3cdb2cee6", - "eval/datasets/coding_memory_v1/oracles/borealis-west--unsupported_questions.py": "01cbbd0b21a745720514aea6a21a98a3cccdbf0c0518bd5508019433b964776b", - "eval/datasets/coding_memory_v1/oracles/cinder-green--code_relationships.py": "8b08e7c65289515a9eee2ae5f4cd60ce94350e9a561225bef0e8f8815ea28f79", - "eval/datasets/coding_memory_v1/oracles/cinder-green--condition_values.py": "92e7d75d38f5dbcc685075e947005a9ae5a410207e7f17aa770b034b8690cbe5", - "eval/datasets/coding_memory_v1/oracles/cinder-green--corrections.py": "424cbf0924a8cbc1a59a6d7bc82b5527e41f52c8ee9f558ea3e398ab4d913919", - "eval/datasets/coding_memory_v1/oracles/cinder-green--long_documents.py": "2a52df9cef3ee66357847322794a382c927b828c6ab1a2819a799ff55d53657b", - "eval/datasets/coding_memory_v1/oracles/cinder-green--multilingual.py": "e47e791dd15b045a6a958ca114b1359ca8537e160274b015214746a2564363c6", - "eval/datasets/coding_memory_v1/oracles/cinder-green--paraphrases.py": "9e92a79ec00231cb381ce58e389dfdb2520a17ecf1e9c2e39e79501449e9ddd0", - "eval/datasets/coding_memory_v1/oracles/cinder-green--poisoning.py": "33213e6f194332969da88f4748c8f26ef93219b6c4bc710a142082f8147ccfcc", - "eval/datasets/coding_memory_v1/oracles/cinder-green--scope_boundaries.py": "8d2489da1fba137981e45e42c843f9cbd17e851c82ddcdc5904b4a86af0d9692", - "eval/datasets/coding_memory_v1/oracles/cinder-green--temporal_history.py": "29403fd08f0de7da3dfcc6c181e1f448a5c5264e95ff3924f3066508d85468b1", - "eval/datasets/coding_memory_v1/oracles/cinder-green--unsupported_questions.py": "ba21014188f9ae849e0bb79babe163194703c4cf01f0600f4c9f71058e7fed56", - "eval/datasets/coding_memory_v1/oracles/cinder-north--code_relationships.py": "46280659557bd08afff185352438bb63dfd308dabf5c7c1862c2f442bd171d48", - "eval/datasets/coding_memory_v1/oracles/cinder-north--condition_values.py": "6c62d673b1b6cb4b5c89ff2af4ecf57c2c9d73074a01ab92b8fde57815cab783", - "eval/datasets/coding_memory_v1/oracles/cinder-north--corrections.py": "8245b196072389517fad6d76ed7709d7e12eb52e08732e4ddcfc40fc1d986879", - "eval/datasets/coding_memory_v1/oracles/cinder-north--long_documents.py": "6790c7b89d80ab909d9e1a31b59bfa5cd199cddabed2e159daf7c5c049ff9a43", - "eval/datasets/coding_memory_v1/oracles/cinder-north--multilingual.py": "9532bbd14506cac0edcc09f474e7c96a402903de947e0e76f5c8aab4bff37646", - "eval/datasets/coding_memory_v1/oracles/cinder-north--paraphrases.py": "cc904350ae7d40e26d91d482ac15d87ec8022d4c82905021dd38003c5fa2d067", - "eval/datasets/coding_memory_v1/oracles/cinder-north--poisoning.py": "e8e451c49c98744b4a484141f86d337db4ec5326264b1500893a20c3451c5ae4", - "eval/datasets/coding_memory_v1/oracles/cinder-north--scope_boundaries.py": "50d42fc48aa70d55b6a3b39c9bf17437726f37e1e5ae9870d415fccad92c7bae", - "eval/datasets/coding_memory_v1/oracles/cinder-north--temporal_history.py": "14b1336fbaed11cad455f588bbe078effc31f5f357f670e1f189ec3d78b172f5", - "eval/datasets/coding_memory_v1/oracles/cinder-north--unsupported_questions.py": "0a2f740700a990d22b4c3051aa3e4260e3df4ce65931d419052f74e05c750a67", - "eval/datasets/coding_memory_v1/oracles/cinder-violet--code_relationships.py": "0e29053f11fe618c01fa90a274688f18ba8a7f3cfea56757a96d5a5bc95b61a8", - "eval/datasets/coding_memory_v1/oracles/cinder-violet--condition_values.py": "ca57ac4e22e9b4739217020d7792ac642326935b8db9305ead2ec5bea2a6e456", - "eval/datasets/coding_memory_v1/oracles/cinder-violet--corrections.py": "9edbfaa58a99776e835c325e50a77df5b4b302b3ddd0932dc2a349a77b4100b5", - "eval/datasets/coding_memory_v1/oracles/cinder-violet--long_documents.py": "4b3fbe7380201285ac4b4a1ec4d8238c5663b97af01cbdc8ffff8fdf2a9467c5", - "eval/datasets/coding_memory_v1/oracles/cinder-violet--multilingual.py": "b0bba5ecf669f3368ef29373b755dc1296502665584574c771ebf4664debb5ce", - "eval/datasets/coding_memory_v1/oracles/cinder-violet--paraphrases.py": "e146f7aa1ffabf3b313251b59c59e1836c0cd17901d07fc6ef5334eaccf8bf36", - "eval/datasets/coding_memory_v1/oracles/cinder-violet--poisoning.py": "692ee07e750fb08652a1ca6bc6dcbbef77219701585f1336c6037dae5a7c5767", - "eval/datasets/coding_memory_v1/oracles/cinder-violet--scope_boundaries.py": "8ca0b911d38a72e4ecce6871ba3d0c7dbab433b0541cb64b116f7a88d521554c", - "eval/datasets/coding_memory_v1/oracles/cinder-violet--temporal_history.py": "6ba5400c5435af64c80a7366fdf49fea132f1011f9d84f9d0cc5f2f2f30a67ac", - "eval/datasets/coding_memory_v1/oracles/cinder-violet--unsupported_questions.py": "1d3324db0a085be30ce3dff1a2ec3866b8592a5ef6b43f39211a2e3a59fc1cd6", - "eval/datasets/coding_memory_v1/oracles/cinder-west--code_relationships.py": "0149333a3fb0f8d60939bf927921305eae447f7511c74f042ff3c880eadff825", - "eval/datasets/coding_memory_v1/oracles/cinder-west--condition_values.py": "3158243f9f7f0b92aaee4164379159b46143acd11fc3d41a708630c1d33ea22c", - "eval/datasets/coding_memory_v1/oracles/cinder-west--corrections.py": "a9fbb89aa629a23e7a1fddf4fca09da40365bbc7a9a6192d805e498fe17f93ff", - "eval/datasets/coding_memory_v1/oracles/cinder-west--long_documents.py": "3c702c653a8ddff84eb5c24a77567aa63d2069c48ed174bcf29065c0ddb30ec6", - "eval/datasets/coding_memory_v1/oracles/cinder-west--multilingual.py": "b8f5f7e00e6edac3f8e0b7e8bc09ea6c7d661370fa2226eb1795306ce3c634a7", - "eval/datasets/coding_memory_v1/oracles/cinder-west--paraphrases.py": "10361be87e9d9cb98d2ff1bbc18252178d475a5c5f2efb77bc81dea05109df05", - "eval/datasets/coding_memory_v1/oracles/cinder-west--poisoning.py": "889a5a43604f7d1677ee0f28683abfc48a8a597b052f558c1b4d19c5f33444ec", - "eval/datasets/coding_memory_v1/oracles/cinder-west--scope_boundaries.py": "b84e78812baaa92422226c06d78cfe7c1be260e2bfcf9c5bb1ed73f2f3cc136b", - "eval/datasets/coding_memory_v1/oracles/cinder-west--temporal_history.py": "9d99a981fca0c801a0a52e2af24cd4262dad72f4f34cd84894784285d241bd05", - "eval/datasets/coding_memory_v1/oracles/cinder-west--unsupported_questions.py": "43b8f854961d9cca5cb83b53b9e07d22388d0c7db406dc8e4f8cfdaa5f89fffd", - "eval/datasets/coding_memory_v1/oracles/delta-green--code_relationships.py": "97a08f2bf9b1a11f95a062f7df27d69db3e9dbc421837347acfe62111af02885", - "eval/datasets/coding_memory_v1/oracles/delta-green--condition_values.py": "5e53695288b84a526c61b748b170f6d6219f94a942edc0887ea6157535ad8d52", - "eval/datasets/coding_memory_v1/oracles/delta-green--corrections.py": "c6d16232ebe6bdb0b5c94db1ba4f108257b47ee8f8b96cda5f8e17212022b542", - "eval/datasets/coding_memory_v1/oracles/delta-green--long_documents.py": "bedb6f495437beada344ce6cdd92954108aa4c639622fd1cc294d2a1139b26d9", - "eval/datasets/coding_memory_v1/oracles/delta-green--multilingual.py": "dbf84f4e8c016c5e60fdac5fd3ac2faf79fb0534f5c2ac534491acc31d5f3530", - "eval/datasets/coding_memory_v1/oracles/delta-green--paraphrases.py": "ea8f00c86e8fbc2ab10a448f48019146181d0e55438def6f47f41f0ef3181be4", - "eval/datasets/coding_memory_v1/oracles/delta-green--poisoning.py": "4eb4e289e944e7762c39ce85a8fc2582e310de6a86cbc1cc36b44112653140ea", - "eval/datasets/coding_memory_v1/oracles/delta-green--scope_boundaries.py": "2fe9b96deecfb488430abd9657adc6b3ea092b18f32c41c1669d29175ec6e860", - "eval/datasets/coding_memory_v1/oracles/delta-green--temporal_history.py": "531cbb953400584356cb6525bc1a2e559f7ec3867d845c736d8026a046b91e7d", - "eval/datasets/coding_memory_v1/oracles/delta-green--unsupported_questions.py": "edebf25c2168411fdefae1307a71d829224dba3dbc7ea000054416e52123d5a3", - "eval/datasets/coding_memory_v1/oracles/delta-north--code_relationships.py": "9357ad0d5f6a1cc787f980d0d506b07808538179f71ae3e8ce1c3eb655d24e35", - "eval/datasets/coding_memory_v1/oracles/delta-north--condition_values.py": "4199f7409696a260b1531f4ec7cfd15555b4d58d06b0b8a88b1c7621fe7d328b", - "eval/datasets/coding_memory_v1/oracles/delta-north--corrections.py": "b8bd58db7c5ae93d33e6223cc34f0e09d58831361ac013ec08277fe2c102ac17", - "eval/datasets/coding_memory_v1/oracles/delta-north--long_documents.py": "0f08b744446b1c52e8fe86626ab05c171377f17c46e604cb2ea647ed5d0daddd", - "eval/datasets/coding_memory_v1/oracles/delta-north--multilingual.py": "ada9eb404190d05637cea59e3f8ed541aea2c9e0f9be0d166885d6789351f7a3", - "eval/datasets/coding_memory_v1/oracles/delta-north--paraphrases.py": "04ec2e8308f47f46e38c89f4a2fe44deef4a0b20a1c8517395ad00983bf7361c", - "eval/datasets/coding_memory_v1/oracles/delta-north--poisoning.py": "d72ea7b98db47e4de03884e2786206dc3da5cc08e5203c86b1d38a2d48c2fd47", - "eval/datasets/coding_memory_v1/oracles/delta-north--scope_boundaries.py": "cedb3234b22a40a7b889f50cfb5751fbba31e46f541b19feb2c4759c8007bac3", - "eval/datasets/coding_memory_v1/oracles/delta-north--temporal_history.py": "2f7c1bd4ebc051eee411d11251aff2ee5fee602a0800410bc0dbd2e86e99628b", - "eval/datasets/coding_memory_v1/oracles/delta-north--unsupported_questions.py": "f1f0b9858c806956122144c6e769f3baf9f87f07ff92eec98eb58d8019e6fba2", - "eval/datasets/coding_memory_v1/oracles/delta-violet--code_relationships.py": "79579639a60d96365781e853d1319a2bf85d9a2b4d8639410da7e3c1efc364b2", - "eval/datasets/coding_memory_v1/oracles/delta-violet--condition_values.py": "27defbfe7c237f971ae29e8f000f098eae32a11f3afad06f2bf3b27af432d902", - "eval/datasets/coding_memory_v1/oracles/delta-violet--corrections.py": "b3b685a5c33d1086e8ab4258d26c5ec9aa7396b8250a250351f0a6057fcf2075", - "eval/datasets/coding_memory_v1/oracles/delta-violet--long_documents.py": "4a5ce75372c8762bcd7986f8ceef454800c3636124ffd9de3a8315d210ae85f1", - "eval/datasets/coding_memory_v1/oracles/delta-violet--multilingual.py": "7ef19f25e97a24522b79a9d7e7f3d2d9e395a1fc2c60564e2f66081533bdacb5", - "eval/datasets/coding_memory_v1/oracles/delta-violet--paraphrases.py": "b6e00019bead0a5bc0facdac886b48fccfd3cccaa254267fda780e933d57904f", - "eval/datasets/coding_memory_v1/oracles/delta-violet--poisoning.py": "3e3e1e1fe04ae3e0c615ca21e794fead5d06f887d6fe7347fb9122684c8b8017", - "eval/datasets/coding_memory_v1/oracles/delta-violet--scope_boundaries.py": "336ae66910d533344712a2d87126f53df48d43eae9e7d17797938c3f9c560204", - "eval/datasets/coding_memory_v1/oracles/delta-violet--temporal_history.py": "a81cb5d2a5eb521dfd161ffbd21bd366777b1239f2d3412acc77b0a9c7c7d82b", - "eval/datasets/coding_memory_v1/oracles/delta-violet--unsupported_questions.py": "3341895608073374b98e67a88489abb9020fed5bb692231a0a61e15c8de0ddf1", - "eval/datasets/coding_memory_v1/oracles/delta-west--code_relationships.py": "e8f17f3838cb74e1ade0cfd6edf540dc086507b51ca67c2020ed490d75dbffe5", - "eval/datasets/coding_memory_v1/oracles/delta-west--condition_values.py": "a935a6e7a4081737eef76a93d54b34dc885b4699e02d797f1be78286dd642a2b", - "eval/datasets/coding_memory_v1/oracles/delta-west--corrections.py": "ee407e5aa8d57254bb7029d537bb2b59a297958cdff9bf271de3787b6448241e", - "eval/datasets/coding_memory_v1/oracles/delta-west--long_documents.py": "55fa08b75982d0547f0c2f13cdc6522d0fbd33efb4af9ae8c6ada11b2d46f321", - "eval/datasets/coding_memory_v1/oracles/delta-west--multilingual.py": "ce8a3e45ad8417ca1798579a22e12eeb7d170a11ebfb865746388eb0e7dca848", - "eval/datasets/coding_memory_v1/oracles/delta-west--paraphrases.py": "e6458588e20e76596e24be6cc838bd4b0d5a47b05b7bf0bb6017b0cecd0d6a68", - "eval/datasets/coding_memory_v1/oracles/delta-west--poisoning.py": "4c70955a2650d13eba24d6e933f765e0bd40da0a1cd8b920416989a76bdaf2d7", - "eval/datasets/coding_memory_v1/oracles/delta-west--scope_boundaries.py": "cfe9120c49224b6f9ed86ea7d85e496000bef0648c92c7fd474c540e892863cd", - "eval/datasets/coding_memory_v1/oracles/delta-west--temporal_history.py": "3527edc8a85c71f889ff0b2ac8bf9ab005ad5bb6c95bcd917dedae44aa3efc72", - "eval/datasets/coding_memory_v1/oracles/delta-west--unsupported_questions.py": "8c40923b8d4bd6bfc78ff6548ef9a840319b7ee4ea967554f26a8d24a4349935", - "eval/datasets/coding_memory_v1/oracles/ember-green--code_relationships.py": "cc7f82ad85363ee35ccd75744159bf4c4f07965e396bff07989cca7a55071679", - "eval/datasets/coding_memory_v1/oracles/ember-green--condition_values.py": "3fa888c125812f1488de9e021358cd531a9d50bb7d4b6173be3656268474b635", - "eval/datasets/coding_memory_v1/oracles/ember-green--corrections.py": "7cc639d4bd7b287ed0e3f2188491d6e4866e021b21874b34b365c1eb2b2d6d68", - "eval/datasets/coding_memory_v1/oracles/ember-green--long_documents.py": "972e7879a30ab26f86f09bce6dc9853c7f96124d0bbe5062111607496bcc9a95", - "eval/datasets/coding_memory_v1/oracles/ember-green--multilingual.py": "0f0585bd3264de3d5fed5928b54ff4fb87f71b2edb1d35c83a534251121124b4", - "eval/datasets/coding_memory_v1/oracles/ember-green--paraphrases.py": "a3c15541bb87cc862d1c9764cfdc1ffebbc831876b55167470ecb3b66519740a", - "eval/datasets/coding_memory_v1/oracles/ember-green--poisoning.py": "ace5b7b12c4ca068739af4bc5e215af1af7fe153399ff17ffb7a67b8b9aa1f94", - "eval/datasets/coding_memory_v1/oracles/ember-green--scope_boundaries.py": "daf132e130c5743d67129c1564f329fbb2670d1242648920b646a83bc2236e4b", - "eval/datasets/coding_memory_v1/oracles/ember-green--temporal_history.py": "1438f09f0ae41b1f6b42930518203a36c7de31a870b684dbb7f16e71a37c784b", - "eval/datasets/coding_memory_v1/oracles/ember-green--unsupported_questions.py": "da27396afbc1529c61972f739cefb26dcaf97bea529c084518a95080eac4fe78", - "eval/datasets/coding_memory_v1/oracles/ember-north--code_relationships.py": "debaf885151c3f47660053ff50e015338b8aa9bca089d7e901ee8bc34f7085e6", - "eval/datasets/coding_memory_v1/oracles/ember-north--condition_values.py": "109a7ee90c04da83f57a94a9a724dcf98a3e96405c7a456686c198ff5d85a69c", - "eval/datasets/coding_memory_v1/oracles/ember-north--corrections.py": "c246ba011c68d3ad0a4ee1d14231304a7f17f28027f1764e60da93062cf1cd3c", - "eval/datasets/coding_memory_v1/oracles/ember-north--long_documents.py": "176200cf2a011c7231ed4cadacbe86b1eeb695fbbee282bf4c4a96430d98fe84", - "eval/datasets/coding_memory_v1/oracles/ember-north--multilingual.py": "e850a910515e71a27edff26f7bedf39d8cb75c726e48357a2c41d4da051be6e9", - "eval/datasets/coding_memory_v1/oracles/ember-north--paraphrases.py": "0a6f03310e44abcf4ca61d69e22edbd21ce47a17629adf4f5079f7565534b469", - "eval/datasets/coding_memory_v1/oracles/ember-north--poisoning.py": "a81ee758fc57bcc58bf0bf2a38dae8b6eb4aeb14949063206e32cdff6f301dfb", - "eval/datasets/coding_memory_v1/oracles/ember-north--scope_boundaries.py": "fe42df24612c66b175f1c98794ed9f13a1a7819418e4aeb5ff98531515221de1", - "eval/datasets/coding_memory_v1/oracles/ember-north--temporal_history.py": "410f9770097228724a5af1357003ea23caefe1226275d030d46efa535c42d306", - "eval/datasets/coding_memory_v1/oracles/ember-north--unsupported_questions.py": "eebe43443938693a74081661d2ff3733b39c6dac5c7ecd88c4dbab5ad18689ee", - "eval/datasets/coding_memory_v1/oracles/ember-violet--code_relationships.py": "c00a30669fbd1c740b44e17c7c7369582c28a16da3aeec9d27420b8aef7c553a", - "eval/datasets/coding_memory_v1/oracles/ember-violet--condition_values.py": "1645e7c56c870f4367c263ccc9041181b1d187a698252203ca651235db59e684", - "eval/datasets/coding_memory_v1/oracles/ember-violet--corrections.py": "735f1d125d2ade6f3eabbf1082a408fe6afd4328cd954833e23df275df498e37", - "eval/datasets/coding_memory_v1/oracles/ember-violet--long_documents.py": "e1ded8e9641639443a6b14da18b2e0cbcddc6fc6045bf0b57404b59b59327dce", - "eval/datasets/coding_memory_v1/oracles/ember-violet--multilingual.py": "fbf04cda8e1231c7ac4c64d4c1856f7bfafcb6f1d016de0b1bdbbaaafcfe03b6", - "eval/datasets/coding_memory_v1/oracles/ember-violet--paraphrases.py": "2fd495def306778073df30b20dbc7a1500c06c90638acf6d9b9280db78dec53a", - "eval/datasets/coding_memory_v1/oracles/ember-violet--poisoning.py": "335674216a418254a01a7f5505344af20ad499a5fe41cb9c8654a0705574a18e", - "eval/datasets/coding_memory_v1/oracles/ember-violet--scope_boundaries.py": "078d60287888903e90343d144d8f54e08a3751ffcba82fd827ca0c9f8ee75fe7", - "eval/datasets/coding_memory_v1/oracles/ember-violet--temporal_history.py": "f7ad47f3253fe922977cbd0b63679abf00559f1b2dc895246e09840652b2c41b", - "eval/datasets/coding_memory_v1/oracles/ember-violet--unsupported_questions.py": "b91545082a31952351bc4c08d6cc09132432126dfd772952c818396bd7a68af1", - "eval/datasets/coding_memory_v1/oracles/ember-west--code_relationships.py": "74fb3e0d99a5bd4f54b8b57f6040ce7978a81bffa1ae70f9069fb2322e6ea581", - "eval/datasets/coding_memory_v1/oracles/ember-west--condition_values.py": "26d605b42d18cf029dfdc29124eea898e0dfa5642c43c1c775007fb443ba6cda", - "eval/datasets/coding_memory_v1/oracles/ember-west--corrections.py": "65c2ef8544b42ecb6b3ea9ce8bc95ec5f0ea8bb07c2b74d52e7880d99962f563", - "eval/datasets/coding_memory_v1/oracles/ember-west--long_documents.py": "4c8e54c7ec930d6cc782fa1a0519a0f1e0e916e3929902e13199775cefdaf338", - "eval/datasets/coding_memory_v1/oracles/ember-west--multilingual.py": "356e407f47d39e7f53337ffc4f27c68935a2c831301dc3dd006b2939cc61decd", - "eval/datasets/coding_memory_v1/oracles/ember-west--paraphrases.py": "c91cf574cf8b5aadaa6c9331c3f8f4a8fec7ec33c227f20894f463d81d110a7d", - "eval/datasets/coding_memory_v1/oracles/ember-west--poisoning.py": "5f6b8740798a7a093994748ef4b11e0e1de0be562c7f09790e1f0ebb9ed21f7d", - "eval/datasets/coding_memory_v1/oracles/ember-west--scope_boundaries.py": "5e20c5154818cc29b7e71444054d67836eddc882f2f17be8aed78afe31a9037f", - "eval/datasets/coding_memory_v1/oracles/ember-west--temporal_history.py": "fc51d1a39693d1f7fb2db4bf1ffe865d4c44848c159188cdcfda54618c0d8fe2", - "eval/datasets/coding_memory_v1/oracles/ember-west--unsupported_questions.py": "079c421cae974bcccc2107dac74421dd5bc2d1b40f7d91cb79a8014af526d260", - "eval/datasets/coding_memory_v1/oracles/fjord-green--code_relationships.py": "b5e4a284bad1d7ff81d5be9cf1faaaaf5e718cb51624189482624721cb459f1b", - "eval/datasets/coding_memory_v1/oracles/fjord-green--condition_values.py": "63c152b8da5655ae5c701cbb51099340d5f5103fcc47840239098a7366d24151", - "eval/datasets/coding_memory_v1/oracles/fjord-green--corrections.py": "b7b33b8de153c205ff4fdaadd5538eae8a22fd4401e70db9f645dae205c62cba", - "eval/datasets/coding_memory_v1/oracles/fjord-green--long_documents.py": "624cf014bc795a24e2f1f4fa2f9dbf11d2ad1c38f81cbc58ab3948f96c48c8c4", - "eval/datasets/coding_memory_v1/oracles/fjord-green--multilingual.py": "f8d2483fd9266946c980764feba0eda27c95ea2ff11222a4dd4247fb537f1f00", - "eval/datasets/coding_memory_v1/oracles/fjord-green--paraphrases.py": "151be253b730d5c5b4bbcabcce3bd218477241ebf0f1bd5fe9dc8d70798feb6f", - "eval/datasets/coding_memory_v1/oracles/fjord-green--poisoning.py": "e7b0efca3813bbb1d4ea9cefa752b4318942b118bb61ee9dcbbf640daa66644d", - "eval/datasets/coding_memory_v1/oracles/fjord-green--scope_boundaries.py": "56177cfb45f32c548be83cc78197fcf5b53826f01cca69e471535476184ae8f9", - "eval/datasets/coding_memory_v1/oracles/fjord-green--temporal_history.py": "2c79d26467eb8443d743bb8823cc1f2a5d1e5da7c887754505baa130e37f1b2e", - "eval/datasets/coding_memory_v1/oracles/fjord-green--unsupported_questions.py": "fb5f222f4e0715379591f559036c0549f67a7b6e3491e880c6cf8d5b91fa9000", - "eval/datasets/coding_memory_v1/oracles/fjord-north--code_relationships.py": "07402078200db9ab15fc61a72e31d9df0938edfb1e9d5b7e532b422dccab15aa", - "eval/datasets/coding_memory_v1/oracles/fjord-north--condition_values.py": "8c1549b5a444c9cbf1304fcca13b79263bf8d9f0b4f41c16da539b3bf5da3bf4", - "eval/datasets/coding_memory_v1/oracles/fjord-north--corrections.py": "a2634af73134c025de1b6239756ee2051e505e0a535221947babef5ba161fa13", - "eval/datasets/coding_memory_v1/oracles/fjord-north--long_documents.py": "4f80f3673afd1320d16a6bd45fefc85522d6f4bb889ff96f6e99410580feec09", - "eval/datasets/coding_memory_v1/oracles/fjord-north--multilingual.py": "3a9c953fbe76eb7374992a6a81d35b8dc0a635c02b90f328af140bbda2acc180", - "eval/datasets/coding_memory_v1/oracles/fjord-north--paraphrases.py": "ec2d9d7bbd678be73946cba8177184839505cd891df410ae8e676fc254df2ca5", - "eval/datasets/coding_memory_v1/oracles/fjord-north--poisoning.py": "6d9b91b4d53925d58d3c0cea694f591572a53184f23816101af0aa644aa1b86d", - "eval/datasets/coding_memory_v1/oracles/fjord-north--scope_boundaries.py": "2e98afa064e8a295a78bea654014a92734696a235fb0538732bb8d85799c7287", - "eval/datasets/coding_memory_v1/oracles/fjord-north--temporal_history.py": "4059ab8e427c2b6fd68155992404fbf493c7ebdf0a4357aa5c5f4ff0c67808f9", - "eval/datasets/coding_memory_v1/oracles/fjord-north--unsupported_questions.py": "79f14e18e282c1b6bbe21ef2affc9dd8a259a171c2e4975d42363f7c91689c1e", - "eval/datasets/coding_memory_v1/oracles/fjord-violet--code_relationships.py": "9ea8a9ade1001bcd43929782c3410f35fc5056e438342ca848fe0b1b0ff56170", - "eval/datasets/coding_memory_v1/oracles/fjord-violet--condition_values.py": "09e223a1c0bff6c95a5863fdc28dc00ed96b90887cd635d014712058079ccbfe", - "eval/datasets/coding_memory_v1/oracles/fjord-violet--corrections.py": "f822c373fc4a93189fdce09999a9dcbe8797b0c0dc780eddd4e342edf21dc14e", - "eval/datasets/coding_memory_v1/oracles/fjord-violet--long_documents.py": "983aeaae1e14937a740218228778be3e1938cdf1ff9228eba44064336188f297", - "eval/datasets/coding_memory_v1/oracles/fjord-violet--multilingual.py": "6e929f8846cfc756c71bcc21bc0f27ef8fc23270d21292fc35d70e6bbd31c066", - "eval/datasets/coding_memory_v1/oracles/fjord-violet--paraphrases.py": "c644d9f01edbd213447618bc269f56804450901faf4f4660c1f666d4e9b4ada4", - "eval/datasets/coding_memory_v1/oracles/fjord-violet--poisoning.py": "cf3849312f96b0072efdfd4e858abb24d7e6a808bf883bcc9dde5dd5a4594364", - "eval/datasets/coding_memory_v1/oracles/fjord-violet--scope_boundaries.py": "373d10e20713bdc26849706525f765f2be4ab57e87e576e39a1a1dd4e7feed88", - "eval/datasets/coding_memory_v1/oracles/fjord-violet--temporal_history.py": "4d2c36480cc04d43aa8f535bf998968b474f2f596d5349dcd42684c5a667c6a4", - "eval/datasets/coding_memory_v1/oracles/fjord-violet--unsupported_questions.py": "b381d2016dea525eebd8de8d026dc48ad2080b93823a95d6e2bb06d29cc86775", - "eval/datasets/coding_memory_v1/oracles/fjord-west--code_relationships.py": "e1e10ed9ee8a4d5c2e798c2bf04cba5c39eaa3873901bd93724af166ba2479ae", - "eval/datasets/coding_memory_v1/oracles/fjord-west--condition_values.py": "427764a62d2e11f6c9fe4b2cdcd32fa92f61749403dff648601a47187e38787a", - "eval/datasets/coding_memory_v1/oracles/fjord-west--corrections.py": "4aa041aa87f4c8628d113bab855a151c2e38bd6919cd02224827fd07d060fb34", - "eval/datasets/coding_memory_v1/oracles/fjord-west--long_documents.py": "89875548e6952ce810d8983b0eaf2724bf8f329bf44e4d9193cfa3405e1ce80e", - "eval/datasets/coding_memory_v1/oracles/fjord-west--multilingual.py": "8b5c14cdb26941d1ae25701bc99efc5fe4ae6e50fed3c65b193952444f1593d1", - "eval/datasets/coding_memory_v1/oracles/fjord-west--paraphrases.py": "bcdc9a5f1fe45ef63fd5e6b728c83c7a8d05c7b80957e3276c074700b409e924", - "eval/datasets/coding_memory_v1/oracles/fjord-west--poisoning.py": "5504e1bab0865fb3041c88f151938789d79422eff91a3ce72f4a41fc14e2f897", - "eval/datasets/coding_memory_v1/oracles/fjord-west--scope_boundaries.py": "63f226631a663dca0ef62a4f32bb98fbe1e30e3b1f030ee42c3f7f2bf6a09af0", - "eval/datasets/coding_memory_v1/oracles/fjord-west--temporal_history.py": "75c9582a16db2c89f12506bd7a4115fcd871c6da63e80429a8e70e31c1f5f862", - "eval/datasets/coding_memory_v1/oracles/fjord-west--unsupported_questions.py": "05e4ba65be7cb4e6500af5e14721c70f93fd7390abe58a60f6f8e4f8e9b0ae5c", - "eval/datasets/coding_memory_v1/oracles/grove-green--code_relationships.py": "bd5619518e839f84ca03297c4f7842a2944d8b445ec1863a8dfc68e4c2114a03", - "eval/datasets/coding_memory_v1/oracles/grove-green--condition_values.py": "67377847b6ff1ab865f5c25351fcfa1ee13e6ec6a089444cb3eb0709b660148b", - "eval/datasets/coding_memory_v1/oracles/grove-green--corrections.py": "c07e8a03c6c47818f9c574d4f21b27668989711fb5e0af3b5cf1a87b293abcbd", - "eval/datasets/coding_memory_v1/oracles/grove-green--long_documents.py": "3f75fc2158c66a19dbbd19209fc80026339abe93a2282ff55c7c01429ac549f3", - "eval/datasets/coding_memory_v1/oracles/grove-green--multilingual.py": "19b1f800b38a11cf0f50df6da3a011806098561e32a279b78e1806de9b8c11fb", - "eval/datasets/coding_memory_v1/oracles/grove-green--paraphrases.py": "a5996376b9f589283dbe301dd1e0644af92a8b6a0222b51b99ed6fbb83570782", - "eval/datasets/coding_memory_v1/oracles/grove-green--poisoning.py": "c4dee0969e39b52dc9b697e45587c8cd84926a97c9a48f7211c0a00571bfb657", - "eval/datasets/coding_memory_v1/oracles/grove-green--scope_boundaries.py": "8d45422f0a1a2115deb90f560409fdad1105b7cae1f03052f8ebac404ba9517c", - "eval/datasets/coding_memory_v1/oracles/grove-green--temporal_history.py": "abc1b386173c8252ac10f11a36fbcdd6d91c731686108a6d32c62d17955d21b9", - "eval/datasets/coding_memory_v1/oracles/grove-green--unsupported_questions.py": "1ee09741a1c94a1fa83c7edf5b6c07d8da2e45826f748d899ec71917e8369d54", - "eval/datasets/coding_memory_v1/oracles/grove-north--code_relationships.py": "fd75d734a5f45a5ec54b125c1d379df67c5531659f212a2f5c1c5e88169167e0", - "eval/datasets/coding_memory_v1/oracles/grove-north--condition_values.py": "c600c0e5f50e352406b99527b050f4df6dc8ab8920b5b946de190ad7629b5e6d", - "eval/datasets/coding_memory_v1/oracles/grove-north--corrections.py": "4d6fdbe4f5eb9c8e4bfc2a974e6c3cbf1d124d0b20d3f050c84e766f507b4be7", - "eval/datasets/coding_memory_v1/oracles/grove-north--long_documents.py": "5b0bbb9f0a90eb377cbfe45ff6f37077a9ce32c7c6730720f0bfd49b844fdf4b", - "eval/datasets/coding_memory_v1/oracles/grove-north--multilingual.py": "525ea2fdad66d66d7b90892e9424ebae281c1dabfb49f2668784a687b7ec894e", - "eval/datasets/coding_memory_v1/oracles/grove-north--paraphrases.py": "1105e52a8c8d3ff79c64b59ec5f23d66d742b8dcecf91e8e62060895c7371f4a", - "eval/datasets/coding_memory_v1/oracles/grove-north--poisoning.py": "7d32e533f6dac57e5d651868386686d2b78e7db1cc29b1860fede23fc9c49813", - "eval/datasets/coding_memory_v1/oracles/grove-north--scope_boundaries.py": "7bea4129589ea34ba10667a7035b126a32b423ec7f3b1ff2b10f6ffc9aa1c2ee", - "eval/datasets/coding_memory_v1/oracles/grove-north--temporal_history.py": "859b6504250f5746ce197e4a0a7c1006f1c2257b939c53a493f9d36f6d2b7d6a", - "eval/datasets/coding_memory_v1/oracles/grove-north--unsupported_questions.py": "6b893c4c124f071e9ad5d9946448b070c2b2f5f10da6f058e60840c18510fd0f", - "eval/datasets/coding_memory_v1/oracles/grove-violet--code_relationships.py": "09bb7df6b115b83cb98a4274b2d883e81b809643ed4f912b383a9e09a7f7a1f9", - "eval/datasets/coding_memory_v1/oracles/grove-violet--condition_values.py": "a68c126f0d7b2fc0950baa5df095d7d513cdb8359883404bad3c1c84c8de666d", - "eval/datasets/coding_memory_v1/oracles/grove-violet--corrections.py": "46024b5fb187aaf3574d380af1a3bb2aa44028f89377f810ca5e98ee03c3b9aa", - "eval/datasets/coding_memory_v1/oracles/grove-violet--long_documents.py": "ec8d401f6437fbaa3b11ddb96973b31944ff374da677f7aae26c0e79e6a1315b", - "eval/datasets/coding_memory_v1/oracles/grove-violet--multilingual.py": "d14e8bf66593ceab7b691119b8a82f211933e2b3f6ebf8a25a75e61ffedbc1c0", - "eval/datasets/coding_memory_v1/oracles/grove-violet--paraphrases.py": "963b7321a32931f238d12e7cdc3849f64682c7cb704b77f4b849543cc40b6df0", - "eval/datasets/coding_memory_v1/oracles/grove-violet--poisoning.py": "39e5edc76a122d37d90b1af28c2293e2f8b0c38e1bdb6460340c0bc2a71ad87d", - "eval/datasets/coding_memory_v1/oracles/grove-violet--scope_boundaries.py": "472298607415aedfb9a36bad7783e7b7e3bccc339ee44cd83746d782ca99e411", - "eval/datasets/coding_memory_v1/oracles/grove-violet--temporal_history.py": "dcf793210a0a547c25587bba70e01d4d56576604856ed084eb5fa466b8be7b83", - "eval/datasets/coding_memory_v1/oracles/grove-violet--unsupported_questions.py": "bbe1769dc0afbe220e8069ef21b85164c31ecff6c3c7f5bf6d18d1e48f5f3c72", - "eval/datasets/coding_memory_v1/oracles/grove-west--code_relationships.py": "5374fb66cd91be79b5eabd2696ada5fee89605d8a1fc84ce36bd6a8ca82cbe5e", - "eval/datasets/coding_memory_v1/oracles/grove-west--condition_values.py": "2e807846b760b65afe0f5e48947463a66c15b8d89899a2724cddd865311760b0", - "eval/datasets/coding_memory_v1/oracles/grove-west--corrections.py": "bca219aae5f85e3f5504d5b39646efe89ff7dc6c007a4323abd5c5154cb4d5a9", - "eval/datasets/coding_memory_v1/oracles/grove-west--long_documents.py": "19c3ca44f78ef79ca1b907c80214e11a8195e994893f1de683b5d9796eb29d6b", - "eval/datasets/coding_memory_v1/oracles/grove-west--multilingual.py": "7842efb41f88fcd49e7a2350e9b44a3a1946bf73e5cb103e4a91a11e810301a2", - "eval/datasets/coding_memory_v1/oracles/grove-west--paraphrases.py": "fd70e8a20da665073ad172219bd57b24e53575dccc344adc1fa0190770ffc6cd", - "eval/datasets/coding_memory_v1/oracles/grove-west--poisoning.py": "473acfa06852554dee20445dd1a86df4e626e0363a7b72de57d68819314bf938", - "eval/datasets/coding_memory_v1/oracles/grove-west--scope_boundaries.py": "3ba0e93abf07498c570cd81b0c4f753bb75b6d4f4bf6c786461bad1a3b301286", - "eval/datasets/coding_memory_v1/oracles/grove-west--temporal_history.py": "334bc7b057f570efd20896d282e54c986f5b6c642930057c8520f84b721ed5e9", - "eval/datasets/coding_memory_v1/oracles/grove-west--unsupported_questions.py": "5102eb2dadcce0dcc0ba157495d4fa10c4983e4bf6627aff25bbd547a08115ca", - "eval/datasets/coding_memory_v1/oracles/helios-green--code_relationships.py": "d376cda89c44b3282a86afe7cd01583f0b53f2c80cd92b47bc2fa4a19271e9ee", - "eval/datasets/coding_memory_v1/oracles/helios-green--condition_values.py": "7b6eaf2f46a5b2f1ae6e8e6da0bcfe6044fea5639df158e137fb8ba32b929909", - "eval/datasets/coding_memory_v1/oracles/helios-green--corrections.py": "2e849f6c926aa0ce1b2e69ce8cdc5ba9c50feb89b33b257fc71bf3aa3b8461b8", - "eval/datasets/coding_memory_v1/oracles/helios-green--long_documents.py": "884c17b7d20d986708c0eff2821cc7b76c5f7b16afd26c43ca80fb507e2f3ea7", - "eval/datasets/coding_memory_v1/oracles/helios-green--multilingual.py": "1182048aef0dcfbc416a004bc43f49f1d2a5de4132e7243ab7d147f54aab1acc", - "eval/datasets/coding_memory_v1/oracles/helios-green--paraphrases.py": "a73950ae7fceacfaa6c7c7a26c8379f9164a29ebea883cede9222e72c506db5e", - "eval/datasets/coding_memory_v1/oracles/helios-green--poisoning.py": "60c5112afb7b84ee5513f2657f42d0eacb0607ba345c28618af3b3b3eaab753e", - "eval/datasets/coding_memory_v1/oracles/helios-green--scope_boundaries.py": "dab035b811e7e9a60bf447d1ea26f6db915c320b5dc2dd04cda4b6f1023296be", - "eval/datasets/coding_memory_v1/oracles/helios-green--temporal_history.py": "c22dd57bc7cd659863231b164192caf6c4929c5baaf8edd8be8cac00268e5847", - "eval/datasets/coding_memory_v1/oracles/helios-green--unsupported_questions.py": "f78986b8093eab4ecc8c1319d359498d64177db55a4e38cf8d0b32d1e5af4b2a", - "eval/datasets/coding_memory_v1/oracles/helios-north--code_relationships.py": "81721c60bd4e3466600442214147ae6641ebc05ae414965f4a09b89cc74cb8d6", - "eval/datasets/coding_memory_v1/oracles/helios-north--condition_values.py": "8fbcf116c1f0d480d61b2974e7bef73acef053d45bf0ac3b12e6ea48b017e43f", - "eval/datasets/coding_memory_v1/oracles/helios-north--corrections.py": "beae8d6c5828c1f225e5e953236136a217ee48e67cebc82049caf8f1c60cfdcc", - "eval/datasets/coding_memory_v1/oracles/helios-north--long_documents.py": "019e7ff3b9efcb461c1c68acc657b2438cc2d63d4b4165c0caa4523c440f12bc", - "eval/datasets/coding_memory_v1/oracles/helios-north--multilingual.py": "a48324390ab60686c2b1417f0c049f47601737c55fdf000b696ccc5de260ccab", - "eval/datasets/coding_memory_v1/oracles/helios-north--paraphrases.py": "9e2552399b988a6d722ccfaec7cfbea10853164336fadc9e9146dbae788a2d8a", - "eval/datasets/coding_memory_v1/oracles/helios-north--poisoning.py": "75eecaa42c302898581d3bef8aee00fa1dec6c3bc968c0014a522196e18c7681", - "eval/datasets/coding_memory_v1/oracles/helios-north--scope_boundaries.py": "8bd32a30cd1c2e346375bf2b37ef4908c3feeb67a80ec52ed3108bfe9cbd548e", - "eval/datasets/coding_memory_v1/oracles/helios-north--temporal_history.py": "1d1118cb5d438f91bb027acaa3b64f867e93cca7eb0f480bb83073584fafd97a", - "eval/datasets/coding_memory_v1/oracles/helios-north--unsupported_questions.py": "02a854bf89085637e366f801d2be6ab26b9e18296f13531485d08c2a64621c7c", - "eval/datasets/coding_memory_v1/oracles/helios-violet--code_relationships.py": "e32101847ccad5cd18fbc2c667ba4d1b63e3e7f8067dc89de5f485546dfef525", - "eval/datasets/coding_memory_v1/oracles/helios-violet--condition_values.py": "c8ef59dc9dcfe3aecc990d82fe44f1f93b7adc8c65f1fc38e79b934d351635c6", - "eval/datasets/coding_memory_v1/oracles/helios-violet--corrections.py": "a3a36bb554a6f4a83ea612bbbe9c0ee4ee1a08071e490a9a909c46d3049bb379", - "eval/datasets/coding_memory_v1/oracles/helios-violet--long_documents.py": "f9fa707bb28b4d7ea96e3bacfa22f24502b571a76a79defd147fe9d5205963a2", - "eval/datasets/coding_memory_v1/oracles/helios-violet--multilingual.py": "cfbfe0535e905dd58434bc25d84e7746dbdf1fc50d72d0fa96cc78677eaf18b9", - "eval/datasets/coding_memory_v1/oracles/helios-violet--paraphrases.py": "1743c8cbf0564d91dfb4db88b694072fdc9bb5a0b9f26f578649af7f539ee393", - "eval/datasets/coding_memory_v1/oracles/helios-violet--poisoning.py": "02cf390520757473e9d677f5c1f0934ecf50645ad5edcdc64caec23721a5d99e", - "eval/datasets/coding_memory_v1/oracles/helios-violet--scope_boundaries.py": "5ab003dfc23cb00d0a70195541a780a4b80b1d496728b2b88f7f154310e67a6d", - "eval/datasets/coding_memory_v1/oracles/helios-violet--temporal_history.py": "209e95f62c39d3f2223ed00a378788e396d04de392324455cd9d19dedb3a87ad", - "eval/datasets/coding_memory_v1/oracles/helios-violet--unsupported_questions.py": "a3ff0703297e2f743917e74abeffc0c922a6241318f37d63a49242c6bcc18059", - "eval/datasets/coding_memory_v1/oracles/helios-west--code_relationships.py": "4d8543e4c49d59f844830ccee3fcea0643ee17ce43f51863b94a1b4fbc9825ea", - "eval/datasets/coding_memory_v1/oracles/helios-west--condition_values.py": "f2efa7fd803df0e10c440d5ec2d98dedf5e02e23a4cedddcde5ea722f4039e40", - "eval/datasets/coding_memory_v1/oracles/helios-west--corrections.py": "181c42640cee47db2b5c779bdf8fe2e84103a92459d7f6cb0d9587458e421406", - "eval/datasets/coding_memory_v1/oracles/helios-west--long_documents.py": "49337830deb24f9d6b11ba4e01f397c0cf1a40a15524f1ce33354a2c61615072", - "eval/datasets/coding_memory_v1/oracles/helios-west--multilingual.py": "f1e4ef8be6bba13457a2da1c3a2082f81bba765f2cf1f07b93eb20edf5e2debe", - "eval/datasets/coding_memory_v1/oracles/helios-west--paraphrases.py": "29abfff3b323496fb62e739cdc0be2aada1b7e1467a141cc8d19a1d54c1ab130", - "eval/datasets/coding_memory_v1/oracles/helios-west--poisoning.py": "14b7f874c10681ff3829b2659cbe7500250369652d11b26c842e62718093ca0f", - "eval/datasets/coding_memory_v1/oracles/helios-west--scope_boundaries.py": "decbdb3cd7ab0c40c0aadb1ecdeef9e2ee2586dd8802b9bb16235f211ce05ee1", - "eval/datasets/coding_memory_v1/oracles/helios-west--temporal_history.py": "6e7c2369157234374d3f9633b7e772b4a3bff405d418c6e1336e56c87ebf28de", - "eval/datasets/coding_memory_v1/oracles/helios-west--unsupported_questions.py": "379716803a300bbecd3fc8f1661bdac2fcd8a98a2bb18514d8de168bd57fa144", - "eval/datasets/coding_memory_v1/oracles/island-green--code_relationships.py": "833962801d594247d5c5644df0e6aee877d452127ea5c0dbc22c81512132ab21", - "eval/datasets/coding_memory_v1/oracles/island-green--condition_values.py": "cfbb4ea0a4dd12d69686178ba8246bd08b93743dc5395cd0479984a53655d909", - "eval/datasets/coding_memory_v1/oracles/island-green--corrections.py": "4c7ee2538df9988864e24d9b86805d26b64d1a37638b57abdd1761e24c4ab340", - "eval/datasets/coding_memory_v1/oracles/island-green--long_documents.py": "f73cd2b884539680c500af9a5c8cc00d1dc9f1d2ef7bda61f6c8253b2bf799ed", - "eval/datasets/coding_memory_v1/oracles/island-green--multilingual.py": "e954fa56aa819ef9088c8d6da4abd647b689956c4ff4a7b0b8e31078522f2702", - "eval/datasets/coding_memory_v1/oracles/island-green--paraphrases.py": "4cde223eac40d4a2962aa2ace533b8761c5b82784767b0307c884ac5e6ad0576", - "eval/datasets/coding_memory_v1/oracles/island-green--poisoning.py": "08725e295848b2c58c9e6d6a0554a2dd4a6506af75039205c444251d11a62acb", - "eval/datasets/coding_memory_v1/oracles/island-green--scope_boundaries.py": "257cd579af07491748f5de88d04f8af85056b0448d348e59c7b0cbb28754fb1b", - "eval/datasets/coding_memory_v1/oracles/island-green--temporal_history.py": "e93105f0639a83b300a6de3bbae56166eedac2e58122f7e84ce5a40a074db6b6", - "eval/datasets/coding_memory_v1/oracles/island-green--unsupported_questions.py": "a707cee90fb3f0cabd0e692399a361f59da0fc4e9e7bbf0cde702e05579f0b3f", - "eval/datasets/coding_memory_v1/oracles/island-north--code_relationships.py": "f00e022ce900e4ad0fcec93f2dbf016337a4d83afacbcf739f61f71316723173", - "eval/datasets/coding_memory_v1/oracles/island-north--condition_values.py": "c704a03a70253585a67823826dc0a6ebaf6323a8880c00af80c148c150176daf", - "eval/datasets/coding_memory_v1/oracles/island-north--corrections.py": "e2cadf6e4f365f94cf0f440758b173da4d14ba12c03a84351e0608f0c60bccc5", - "eval/datasets/coding_memory_v1/oracles/island-north--long_documents.py": "300246fd3d85d73b63066d133361bc784ae26b46f4197c5c41ac960eb37f9508", - "eval/datasets/coding_memory_v1/oracles/island-north--multilingual.py": "f8aba8e14ed3c420e90e701880ea38e924e1219e3cc3643ba8bde17cb31ddef6", - "eval/datasets/coding_memory_v1/oracles/island-north--paraphrases.py": "503ce2c132c666bb67920e90d4a898f5a73f1d66779cd83c1885812f223e30e5", - "eval/datasets/coding_memory_v1/oracles/island-north--poisoning.py": "3369d956e5af12bc6be80828c4c6ab03ccf80c79e38dea3162b4e2c09771ff30", - "eval/datasets/coding_memory_v1/oracles/island-north--scope_boundaries.py": "9b4f6186f71713175f1eb5aacb0b6258c746b2297ca7c692f54bf5feabd5d261", - "eval/datasets/coding_memory_v1/oracles/island-north--temporal_history.py": "7c22b0edbf8edb7a511a581ffe6f1ce14454ef3feb527eca90d1386ce00b2c6b", - "eval/datasets/coding_memory_v1/oracles/island-north--unsupported_questions.py": "c1e5d526b0c6b88394f19a8b8edca7dae1db718b24be2d57374eb3769bbd1f24", - "eval/datasets/coding_memory_v1/oracles/island-violet--code_relationships.py": "9672d608ea0c8623ff1fe760f943173b2fdac5dc4ea4b7ae0dba33436d860254", - "eval/datasets/coding_memory_v1/oracles/island-violet--condition_values.py": "cfea91adcc9b3f9eb28cc24cf310e10b436f81cff51297f161b9a3b59a21f62c", - "eval/datasets/coding_memory_v1/oracles/island-violet--corrections.py": "075d48a2dfd91af8328982fe475ec915c00b75db38819aff3b4c2343147e2bfc", - "eval/datasets/coding_memory_v1/oracles/island-violet--long_documents.py": "85cc789277e59925b36ea3bd82e23f989e79fb0be2e55083e2656e518641dd3c", - "eval/datasets/coding_memory_v1/oracles/island-violet--multilingual.py": "84615b70eb74eb2180dd8a89d8d34c11e904030bfaa26c57e2e6c968daf64d29", - "eval/datasets/coding_memory_v1/oracles/island-violet--paraphrases.py": "5244d32ffc4127d519e2f5a875ffaecf7bc51eaa1426d2aacbc42490cf291d22", - "eval/datasets/coding_memory_v1/oracles/island-violet--poisoning.py": "9bde478876ea0be60555d5bd26d3790e4f305327ece10bf6c258bc564354fa1d", - "eval/datasets/coding_memory_v1/oracles/island-violet--scope_boundaries.py": "8e8b0a64342963b4dbc000578a72c0e0801fc1ca4af140f09b55d4cdb03b1274", - "eval/datasets/coding_memory_v1/oracles/island-violet--temporal_history.py": "3e187ce259c7f57f6dba4a28c3ecdf31fbe6458f7d908b22cb8b5ce8325437ae", - "eval/datasets/coding_memory_v1/oracles/island-violet--unsupported_questions.py": "0e60f29a47953c5d434c159eece26bbb03cc837842c493111b34310f9cff4693", - "eval/datasets/coding_memory_v1/oracles/island-west--code_relationships.py": "c19c42978fa5f9c6e1f41886eb1cc0fdb7e12304ba58b160b4cbe1975640acfe", - "eval/datasets/coding_memory_v1/oracles/island-west--condition_values.py": "07c4c591e08f5248f3016dc16cecfea7dd75bebff1a61599f258938d7031043d", - "eval/datasets/coding_memory_v1/oracles/island-west--corrections.py": "a7d93dfb579ec4f58cf2f6d29a6d0ab7dbf1a17f6ae49b98902a5ea22525e649", - "eval/datasets/coding_memory_v1/oracles/island-west--long_documents.py": "20a50dd63e04ce751115fca671b9da7ebbf240e26f39d494583cb405a03b750c", - "eval/datasets/coding_memory_v1/oracles/island-west--multilingual.py": "65d82c4d7968e0e597509c692015bdac0b92841c726d87560d9caa32429176f4", - "eval/datasets/coding_memory_v1/oracles/island-west--paraphrases.py": "feb7eb2e8592162d7d1f433610cb02ba04cd9797191d65a9f0e5bc4f4711c4ed", - "eval/datasets/coding_memory_v1/oracles/island-west--poisoning.py": "3d7c2c6a7f247f3e8158d5ba007bbbdf027a74e9a2c52f52953b3ccf2c21f17f", - "eval/datasets/coding_memory_v1/oracles/island-west--scope_boundaries.py": "ff0f36139f6da5428b9538d108aded522e62ca803035ebef78e11255cf55e08b", - "eval/datasets/coding_memory_v1/oracles/island-west--temporal_history.py": "42cb1c552ce01308b47d526c6e6993fb3270239b392cfa9f5e42b8e0fcfb3f82", - "eval/datasets/coding_memory_v1/oracles/island-west--unsupported_questions.py": "911b4257380640d35d6e2fc5292e235eedc113f5e71d8e71e8ccab7c72d4f290", - "eval/datasets/coding_memory_v1/oracles/juniper-green--code_relationships.py": "373395d3b70f64f8102609d27ce10c4aeb937ac73988c2ce2be1e283df94ef82", - "eval/datasets/coding_memory_v1/oracles/juniper-green--condition_values.py": "f04794a96b06d537550227ec308b1f02405cdab42d7d3ae4db4afd2c25bee8cd", - "eval/datasets/coding_memory_v1/oracles/juniper-green--corrections.py": "5a354a8c8efdbc498d25ffbf6095fee7b3d7d638134b452c7c9ecb186b4e1bc7", - "eval/datasets/coding_memory_v1/oracles/juniper-green--long_documents.py": "8610157668d6a90c794a956f6f0b432f03e3559238599ad1cb412f6f1a2f7e46", - "eval/datasets/coding_memory_v1/oracles/juniper-green--multilingual.py": "dcb99335fa18b008197532809ff7127be2cda600848e10c8bfe4677bd896f90e", - "eval/datasets/coding_memory_v1/oracles/juniper-green--paraphrases.py": "7a4cb1bbe3de2e6fe6a33b1f78829f4aeb21d21c2e8d4d3399d3d98958ba0909", - "eval/datasets/coding_memory_v1/oracles/juniper-green--poisoning.py": "ba73c2c6f04590538e16c50be527da60f5949781bc8833a3c18a7c597146eb10", - "eval/datasets/coding_memory_v1/oracles/juniper-green--scope_boundaries.py": "9d09c0851a3906b6281cb1c5811714224161e27a1aa46edd81c5659e7f93cd3d", - "eval/datasets/coding_memory_v1/oracles/juniper-green--temporal_history.py": "fd05595a452952dca2d0089f5701a60145a3b5705b78a5ff2f8f9f13fbf1c259", - "eval/datasets/coding_memory_v1/oracles/juniper-green--unsupported_questions.py": "f58bfcb89db799dc1578fc8791b49e97e7f22b30e720aeaf661f0ad2e6a9660c", - "eval/datasets/coding_memory_v1/oracles/juniper-north--code_relationships.py": "a84f02bd3c2b9c367248dae8aa3580d83de6a1beb6fe464553eb8a2bfb172b75", - "eval/datasets/coding_memory_v1/oracles/juniper-north--condition_values.py": "d70311ae339556f18240f824d0f0bd0024060c2698fcca8fe7cb540e8a8bc727", - "eval/datasets/coding_memory_v1/oracles/juniper-north--corrections.py": "fa290629339007fdd4230a76d49fa294a218c5f0cf6b90c3074678b0f16c539a", - "eval/datasets/coding_memory_v1/oracles/juniper-north--long_documents.py": "e30025d68ae3b48b566bf3fb2aea5e3603f5f7ab658da8fcb2256d3b1964c85a", - "eval/datasets/coding_memory_v1/oracles/juniper-north--multilingual.py": "b8bb0199c4acfe71df8d4e7df75bdd37e038cc72aa82bc6df10715819aeb3686", - "eval/datasets/coding_memory_v1/oracles/juniper-north--paraphrases.py": "aabf70509d3b0602d43c9590f04fa3ad1370a40c266d20f64a5ece15d4c79a51", - "eval/datasets/coding_memory_v1/oracles/juniper-north--poisoning.py": "ebfb9d7bf3c50eed8f44b73e391496ef377942de389b6feb64c7a1c0f9646444", - "eval/datasets/coding_memory_v1/oracles/juniper-north--scope_boundaries.py": "06c2c9d57ade79139c9b0d20568703477bb9e3a934f2f8900ad717ef157e5844", - "eval/datasets/coding_memory_v1/oracles/juniper-north--temporal_history.py": "0d97ba6d7a5d7741bf248b599ed18ae81b1c0053ad53cbc64934f31e9c3461aa", - "eval/datasets/coding_memory_v1/oracles/juniper-north--unsupported_questions.py": "526648ba7a4edc21bc2eb589c1e5c98a20fb07aa692687de4c3845e559be8096", - "eval/datasets/coding_memory_v1/oracles/juniper-violet--code_relationships.py": "c73fd24d0adbeda6c7afeb15ab07296c0438402893e8d442af26865106d6e8ea", - "eval/datasets/coding_memory_v1/oracles/juniper-violet--condition_values.py": "4aadae570ef84f976d49ba1fc92574ef55630b1c3017e90dbec1f85e6fb74c4a", - "eval/datasets/coding_memory_v1/oracles/juniper-violet--corrections.py": "07236b2a88df73c8d0f6afbe7d972fdc18f8c835a852f5c20748a369c77d82e2", - "eval/datasets/coding_memory_v1/oracles/juniper-violet--long_documents.py": "54d3201b621b168ce03963c423942f142cb403f9d90234f063ff33b7add4d2dc", - "eval/datasets/coding_memory_v1/oracles/juniper-violet--multilingual.py": "e0cc26d743cbea8e98261895474879d078fd53941c075606bfc6e52ee098db93", - "eval/datasets/coding_memory_v1/oracles/juniper-violet--paraphrases.py": "a1f836316408052cab1253b56b5caa6b1432fe6eeaabbb5872da97ec9bde633a", - "eval/datasets/coding_memory_v1/oracles/juniper-violet--poisoning.py": "bc09271eadf77f1b6c03fff20ca9c0f0a42a99d1c5600e6b7843f0a0d549ed5d", - "eval/datasets/coding_memory_v1/oracles/juniper-violet--scope_boundaries.py": "17a7258bed9efe1c86a05756c5b5d17926fd72f01de4b2addd5d2fb4550f4da5", - "eval/datasets/coding_memory_v1/oracles/juniper-violet--temporal_history.py": "8bda8599d861857204c48b55729607c8cb0cb3ae2701eb18fb600d398468cafe", - "eval/datasets/coding_memory_v1/oracles/juniper-violet--unsupported_questions.py": "10cdbb013f78f0e54c31a6e76f03d8d15a6063fd7971a2d803a390890c036ad2", - "eval/datasets/coding_memory_v1/oracles/juniper-west--code_relationships.py": "f0a682d413b3d8418c664db2775de4eef35240c508cc71ca6bbc055cd0c8dbe6", - "eval/datasets/coding_memory_v1/oracles/juniper-west--condition_values.py": "200a4b78f201e992ac10112a194c97b88841dc5930e3e7de9bfc076bf4c69fa7", - "eval/datasets/coding_memory_v1/oracles/juniper-west--corrections.py": "41ea9b5a5f40d8703343f4570593c3f27905e5ff70fae57bdbdb5fcccc585d22", - "eval/datasets/coding_memory_v1/oracles/juniper-west--long_documents.py": "7d8cdd01c5d90ad62e9da93a99febead683e3f01c5ca37315382a2fb0f512d76", - "eval/datasets/coding_memory_v1/oracles/juniper-west--multilingual.py": "1815adc5946bf9e7dba16786d8c29f9466537b7a43b1c392f927b61c56da54ee", - "eval/datasets/coding_memory_v1/oracles/juniper-west--paraphrases.py": "5d0110bc2098d2143303476fb5c5714086d0278b848110c63aa626e0b88311cc", - "eval/datasets/coding_memory_v1/oracles/juniper-west--poisoning.py": "4e16d9de9931cffc6915805b4fc375dad3ffb7fad17eb893205ce6d07102bcb3", - "eval/datasets/coding_memory_v1/oracles/juniper-west--scope_boundaries.py": "6c5d875184c890a922ee18ac423598fdcece109a91e9e62d5a7ccc82d2eaaa70", - "eval/datasets/coding_memory_v1/oracles/juniper-west--temporal_history.py": "3404fdbb5cc4693333d48d27da928531279d148ffe1cdadbdd741dfc1a8d38d3", - "eval/datasets/coding_memory_v1/oracles/juniper-west--unsupported_questions.py": "b85a4cf14f9596cd14b480acd323772fc91084ad1413fa27435bdb7ffe5baa6b", - "eval/datasets/longdoc.jsonl": "7f5ade95e1f283d0db8cf78e53ed8995d3534f847e616d2c0005fd8da37ac790", - "eval/engine_capacity.py": "1940da1c3a435d81c7b7a1a689e966c23b92e4f94af115b45813dd30e2c04eec", - "eval/evidence_contracts.py": "03969fdf01e78135c66e698d3f523e5312ccca3737dd3036b37a2c6676ba4ea2", - "eval/external.py": "98500a97c18152b1b86a270062033c48176224d7468ac0f703ddc7de862a30e2", - "eval/external_checkpoints.py": "484e298039bfa04eef86bd7145428c312aaeaf059f5bdf8da7ae29d1150008d7", - "eval/extractor_quality.py": "50820fe7f821d111e17e4e77a3d1159e7d6979d110b052c4f254a5b309c2652a", - "eval/fts_insert_scaling.py": "6088997e0f85d430fafb74509e24a960fbd59c06c8b2839285942b97d2116724", - "eval/graph_every_bench.py": "79da573c5edf315f71bfab412d3ea283b8da45d1fdabc78ddd19302ead09e4f0", - "eval/graph_traversal.py": "b094f75c3a1d75ba3cf19e372187d692d3c595a1d9bde19bbe95ae0c79a5175f", - "eval/grounded.py": "053d5193b716a2c3e507fcd44057d392de910a4442b46bd7cc1f30ac0ba68541", - "eval/handoff_quality.py": "7daf635510764e236f48ca7e2537a85d8ebc1ad995513144329c1f0236405937", - "eval/harness.py": "8c96c26a121dfc2d9ea051a05861af8951c93a732dba9cb13de0de178414b016", - "eval/hosted_evidence.py": "7946cd8c1e3aa291268271b2aa11210d5f09c9b05ba635aa0b64bef07bdcee45", - "eval/hosted_ledger.py": "a53036d12ff671371c148910a816fb20c7e7f3346250a303722352f47b06c476", - "eval/hosted_luna.py": "4dbf02a65eec38bcde92d82952a0b372abbac1f68378d11a04528f4a31a835dc", - "eval/jev_recall_quality.py": "3c31080767c2bc0301f1085c837eaa60ac6e1f5ae8eeb6ff908e573a630510cd", - "eval/local_benchmark_queue.py": "43fa67b4d653e770822da816511e8e62b3714e60d2edd49ea2a4c5c3d8d20851", - "eval/local_capacity_campaign.py": "ab4935263f9d24c4e38bd4f16164b4c4c07eff91c695809b6fc75299d4f03d62", - "eval/longmemeval_v2.py": "defb4d47f453aa4615a8f101b82df3fadf64f0f9eae0ce1021d86d3750dad437", - "eval/longmemeval_v2_evidence.py": "8486bd4dcfdd8b1f7a14c32fe4f427c980ccdcc3aa2eb018d7dfc40305412eff", - "eval/longmemeval_v2_matrix.py": "ca085cd59481813cce5dbdfbc94f40f67173ac3a1bc3dce6f6d3e09eb7b08153", - "eval/metrics.py": "16857e2cf6ed339cb57a26c9bfa1879444b4d279bd972e5a9fa644ed1308afe0", - "eval/native_coverage_scaling.py": "0d318c116241050fc0c7bdbbb5646ca67944d7c304ccd9a462834e0553ffa90a", - "eval/performance.py": "dccc55c26dc396f9fee98defd152900cf8aabeb5d6198bc711e0eb8f472ecc57", - "eval/performance_engine.py": "3d37cf0a5989c8fa6e8b6ab7ae0b2e0d5410a8b922d2f3130c1a7794d93dcbb1", - "eval/planned_recall.py": "f9a87291ecb181045b98ca65fe55db820bf7cee4ae4f4d087ca3458afdd7837e", - "eval/proactive_ranking.py": "8610541f1d547f9c0eb46d078dbcaa670c0a97157acc37f96ec08b482cb7a6ab", - "eval/productivity.py": "6d4644ebdc44472aeb3879963774fab276bfa269b781139c46ded48774a22717", - "eval/public_readiness.py": "5ce8a18d0bfe09e75e88548a589b8fc6d2cbf04c0f8cd1ce51cd9519136a2751", - "eval/redteam_poisoning.py": "fce120cc3adf2ee966b59cd3ea7f20d242a0af49b52492143543130fe14c0cc8", - "eval/reinforcement.py": "72ed766775a2658eaa728afec51c0ac22d97a90e813111954df66a6ec50f2bef", - "eval/repair_discovery.py": "db05496fbbcb0df86c5cdc2f0c85fb6b6605b0cace6cb44ae2add10b72784b5f", - "eval/resolver_reworded_corrections.py": "a9054778a37b2175b46f04b674b4779e358931eee5d2ae52bbc5f953d234fb9e", - "eval/resource_hierarchy.py": "5ab6c989bb143c4386749c45a33c190447e829c1b5657e8c0aca30f34bd69461", - "eval/rework_statistics.py": "e12c14288797c5cf2dd93d51f606244287bc2f4ff8bd401f1d1b072efd1f9529", - "eval/run_longmemeval_v2.py": "866863d9f8f7ce8f7c3c36741ab324faa5fae417827f907c655eace38de0f5ac", - "eval/task_pairs.py": "fddc54804e8837ec0731813297fb55825458317f898e29176e16f3f5a2f527fd", - "eval/user_journeys.py": "a1c8436d4a6871baff21f9aa4f6ac1545def60d946cd3b1454c3d9cd8661a049", - "eval/vector_scale.py": "3f9c327d9eca1a857512ddc208e972933be1a7d4fa7fa0017aca0cbf8fe7bb6d", - "eval/vector_scale_storage.py": "24040fd1b96b9f9cd43ea37b0b37ea118a02bd096aff77fc67ff8b0df769b1dc", - "eval/vector_scan_plan.py": "34fba3d029bfc78134e8c9c450b019ff56c3bdcbeb921400888077cb44e9b848", - "scripts/export_offline_evidence.py": "4e10c2b3d5f6a2024a7f95f7b87ce47ec2cb4ef31c55611b2e5cf35502ebc0ba" - } - } -} From 7df48fd258092dfe110edf5cd355bce9a01d3090 Mon Sep 17 00:00:00 2001 From: Jaixii Date: Sun, 4 Oct 2026 14:06:53 -0400 Subject: [PATCH 17/22] chore: remove superseded draft benchmark artifact --- docs/benchmark-evidence/offline-fixtures-v149.json.sha256 | 1 - 1 file changed, 1 deletion(-) delete mode 100644 docs/benchmark-evidence/offline-fixtures-v149.json.sha256 diff --git a/docs/benchmark-evidence/offline-fixtures-v149.json.sha256 b/docs/benchmark-evidence/offline-fixtures-v149.json.sha256 deleted file mode 100644 index f26d510f..00000000 --- a/docs/benchmark-evidence/offline-fixtures-v149.json.sha256 +++ /dev/null @@ -1 +0,0 @@ -170642b51e554dab603e2de887eef78d2b3ac1de20db07d1e1a49598fb55a59b offline-fixtures-v149.json From 6332018aebbf53383bdb2e3a4c0b7adb92d3d2d6 Mon Sep 17 00:00:00 2001 From: Jaixii Date: Sun, 4 Oct 2026 14:07:11 -0400 Subject: [PATCH 18/22] docs: sync benchmark contract with parent stack --- README.md | 34 +++++++++++++++++----------------- 1 file changed, 17 insertions(+), 17 deletions(-) diff --git a/README.md b/README.md index f579b4f2..6e34df5f 100644 --- a/README.md +++ b/README.md @@ -1,14 +1,14 @@ # Engraphis [![PyPI version](https://img.shields.io/pypi/v/engraphis.svg)](https://pypi.org/project/engraphis/) -[![License](https://img.shields.io/badge/license-Apache--2.0-green.svg)](https://github.com/Coding-Dev-Tools/engraphis/blob/719e1712c3f59d3fd6836d10c2c67b18317cf1ce/LICENSE) +[![License](https://img.shields.io/badge/license-Apache--2.0-green.svg)](https://github.com/Coding-Dev-Tools/engraphis/blob/fee9d0c150c250632d8e0c0ee86c1325c9e1ee78/LICENSE) **Persistent, local-first memory for AI agents.** Engraphis stores scoped project knowledge, retrieves relevant evidence across vector, lexical, graph, and code search, and returns bounded context with sources an agent can inspect. The local engine uses SQLite and works offline. It keeps changes over time instead of silently replacing facts, and grounded recall cites retrieved memories or abstains when evidence is weak.

- Engraphis local knowledge graph showing relationships between remembered entities + Engraphis local knowledge graph showing relationships between remembered entities
Explore memories and their relationships in the local dashboard.

@@ -35,7 +35,7 @@ hit = memory.recall("Why did we change auth?", workspace="acme", repo="api") print(hit["context"]) ``` -Connect a coding agent over MCP with the [agent setup guide](https://github.com/Coding-Dev-Tools/engraphis/blob/719e1712c3f59d3fd6836d10c2c67b18317cf1ce/docs/AGENT_CONNECT.md). +Connect a coding agent over MCP with the [agent setup guide](https://github.com/Coding-Dev-Tools/engraphis/blob/fee9d0c150c250632d8e0c0ee86c1325c9e1ee78/docs/AGENT_CONNECT.md). ## What it provides @@ -49,7 +49,7 @@ Connect a coding agent over MCP with the [agent setup guide](https://github.com/ The current registered artifact contains three deterministic offline fixture runs. It separates context size, retrieval quality, and grounded decision checks; these small fixtures do not establish general task performance.

- Three registered offline fixtures: structure-aware chunking reduced mean retrieved context from 740.3 to 214.3 tokens per question (71.1%, 18 questions); the serialized JSON-shape proxy fell from 24,590 to 11,138 tokens across 26 payload samples and 260 recalls. Candidate and packed retrieval quality (Recall@5, Hit@5, and answer-token recall) are shown separately (each 1.000). Grounded checks show 5/5 answerable queries grounded and 6/6 abstention queries rejected, including a 1/1 quarantined-evidence probe; 11/11 decisions were correct. MCP transport and provider billing were not measured. Artifact SHA-256 prefix 08ed1a6451af; see the benchmark guide for the full checksum. + Three registered offline fixtures: structure-aware chunking reduced mean retrieved context from 740.3 to 214.3 tokens per question (71.1%, 18 questions); the serialized JSON-shape proxy fell from 24,590 to 11,138 tokens across 26 payload samples and 260 recalls. Candidate and packed retrieval quality (Recall@5, Hit@5, and answer-token recall) are shown separately (each 1.000). Grounded checks show 5/5 answerable queries grounded and 6/6 abstention queries rejected, including a 1/1 quarantined-evidence probe; 11/11 decisions were correct. MCP transport and provider billing were not measured. Artifact SHA-256 prefix d5d36c55c430; see the benchmark guide for the full checksum.
Three offline fixtures separate context reduction, candidate and packed retrieval quality, and grounded behavior. The chart shows the artifact checksum prefix; see the benchmark guide for the full checksum and reproduction steps.

@@ -60,7 +60,7 @@ The current registered artifact contains three deterministic offline fixture run | Recall payload proxy | JSON-shape proxy: 24,590 → 11,138 tokens (54.71% lower, 26 samples; 260 timed recalls) | Candidate and packed Recall@5, Hit@5, and answer-token recall are each 1.000 | | Grounded decisions | 5/5 answerable queries grounded; 6/6 abstention queries rejected, including 1/1 quarantined-evidence check | 11/11 decisions correct | -The payload figure is a serialized JSON-shape estimate, not an MCP transport measurement or provider billing total. See the [Benchmark methodology](https://github.com/Coding-Dev-Tools/engraphis/blob/719e1712c3f59d3fd6836d10c2c67b18317cf1ce/BENCHMARKS.md) for artifact identity, counting methods, reproduction commands, external-evaluation boundaries, and limitations. +The payload figure is a serialized JSON-shape estimate, not an MCP transport measurement or provider billing total. See the [Benchmark methodology](https://github.com/Coding-Dev-Tools/engraphis/blob/35037b63f372486f66c28257e29146f74c5654f3/BENCHMARKS.md) for artifact identity, counting methods, reproduction commands, external-evaluation boundaries, and limitations. ## Optional Jev assistance @@ -91,18 +91,18 @@ See [hosted plans and Jev details](https://github.com/Coding-Dev-Tools/engraphis ## Guides -- [Configuration reference](https://github.com/Coding-Dev-Tools/engraphis/blob/719e1712c3f59d3fd6836d10c2c67b18317cf1ce/docs/CONFIGURATION.md) -- [MCP tool reference](https://github.com/Coding-Dev-Tools/engraphis/blob/719e1712c3f59d3fd6836d10c2c67b18317cf1ce/docs/MCP_TOOLS.md) -- [Agent and LLM provider setup](https://github.com/Coding-Dev-Tools/engraphis/blob/719e1712c3f59d3fd6836d10c2c67b18317cf1ce/docs/LLM_PROVIDERS.md) -- [Architecture and query planning](https://github.com/Coding-Dev-Tools/engraphis/blob/719e1712c3f59d3fd6836d10c2c67b18317cf1ce/docs/ARCHITECTURE_V3.md#query-planning) -- [Docker deployment](https://github.com/Coding-Dev-Tools/engraphis/blob/719e1712c3f59d3fd6836d10c2c67b18317cf1ce/docs/DOCKER.md) -- [Document import](https://github.com/Coding-Dev-Tools/engraphis/blob/719e1712c3f59d3fd6836d10c2c67b18317cf1ce/docs/DOCUMENT_IMPORT.md) -- [Cloud Sync](https://github.com/Coding-Dev-Tools/engraphis/blob/719e1712c3f59d3fd6836d10c2c67b18317cf1ce/docs/SYNC.md) -- [Pi extension](https://github.com/Coding-Dev-Tools/engraphis/blob/719e1712c3f59d3fd6836d10c2c67b18317cf1ce/integrations/pi/README.md) -- [Memory write review](https://github.com/Coding-Dev-Tools/engraphis/blob/719e1712c3f59d3fd6836d10c2c67b18317cf1ce/docs/WRITE_REVIEW.md) -- [Security policy](https://github.com/Coding-Dev-Tools/engraphis/blob/719e1712c3f59d3fd6836d10c2c67b18317cf1ce/SECURITY.md) -- [Hosted plans and licensing](https://github.com/Coding-Dev-Tools/engraphis/blob/719e1712c3f59d3fd6836d10c2c67b18317cf1ce/docs/HOSTED_PLANS.md) +- [Configuration reference](https://github.com/Coding-Dev-Tools/engraphis/blob/fee9d0c150c250632d8e0c0ee86c1325c9e1ee78/docs/CONFIGURATION.md) +- [MCP tool reference](https://github.com/Coding-Dev-Tools/engraphis/blob/fee9d0c150c250632d8e0c0ee86c1325c9e1ee78/docs/MCP_TOOLS.md) +- [Agent and LLM provider setup](https://github.com/Coding-Dev-Tools/engraphis/blob/fee9d0c150c250632d8e0c0ee86c1325c9e1ee78/docs/LLM_PROVIDERS.md) +- [Architecture and query planning](https://github.com/Coding-Dev-Tools/engraphis/blob/fee9d0c150c250632d8e0c0ee86c1325c9e1ee78/docs/ARCHITECTURE_V3.md#query-planning) +- [Docker deployment](https://github.com/Coding-Dev-Tools/engraphis/blob/fee9d0c150c250632d8e0c0ee86c1325c9e1ee78/docs/DOCKER.md) +- [Document import](https://github.com/Coding-Dev-Tools/engraphis/blob/fee9d0c150c250632d8e0c0ee86c1325c9e1ee78/docs/DOCUMENT_IMPORT.md) +- [Cloud Sync](https://github.com/Coding-Dev-Tools/engraphis/blob/fee9d0c150c250632d8e0c0ee86c1325c9e1ee78/docs/SYNC.md) +- [Pi extension](https://github.com/Coding-Dev-Tools/engraphis/blob/fee9d0c150c250632d8e0c0ee86c1325c9e1ee78/integrations/pi/README.md) +- [Memory write review](https://github.com/Coding-Dev-Tools/engraphis/blob/fee9d0c150c250632d8e0c0ee86c1325c9e1ee78/docs/WRITE_REVIEW.md) +- [Security policy](https://github.com/Coding-Dev-Tools/engraphis/blob/fee9d0c150c250632d8e0c0ee86c1325c9e1ee78/SECURITY.md) +- [Hosted plans and licensing](https://github.com/Coding-Dev-Tools/engraphis/blob/e440bf6ba0ff600648fdac6eb53dd28d6d80df24/docs/HOSTED_PLANS.md) ## License -Engraphis is licensed under Apache-2.0. The license does not grant trademark rights. See [LICENSE](https://github.com/Coding-Dev-Tools/engraphis/blob/719e1712c3f59d3fd6836d10c2c67b18317cf1ce/LICENSE), [NOTICE](https://github.com/Coding-Dev-Tools/engraphis/blob/719e1712c3f59d3fd6836d10c2c67b18317cf1ce/NOTICE), and the [licensing guide](https://github.com/Coding-Dev-Tools/engraphis/blob/719e1712c3f59d3fd6836d10c2c67b18317cf1ce/docs/LICENSING.md). The hosted control plane and managed services are private services. +Engraphis is licensed under Apache-2.0. The license does not grant trademark rights. See [LICENSE](https://github.com/Coding-Dev-Tools/engraphis/blob/fee9d0c150c250632d8e0c0ee86c1325c9e1ee78/LICENSE), [NOTICE](https://github.com/Coding-Dev-Tools/engraphis/blob/fee9d0c150c250632d8e0c0ee86c1325c9e1ee78/NOTICE), and the [licensing guide](https://github.com/Coding-Dev-Tools/engraphis/blob/fee9d0c150c250632d8e0c0ee86c1325c9e1ee78/docs/LICENSING.md). The hosted control plane and managed services are private services. From 8338a4c2e991180f324e2ee2147ab2ef9f928efc Mon Sep 17 00:00:00 2001 From: Jaixii Date: Sun, 4 Oct 2026 14:07:17 -0400 Subject: [PATCH 19/22] docs: sync benchmark contract with parent stack --- BENCHMARKS.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/BENCHMARKS.md b/BENCHMARKS.md index f012f89d..d39910d1 100644 --- a/BENCHMARKS.md +++ b/BENCHMARKS.md @@ -94,14 +94,14 @@ interpretation and do not count as additional benchmark-quality gains. ### Public numeric evidence registry Every exact public aggregate retained below comes from the checked-in, public-safe -[`offline-fixtures-v151.json`](docs/benchmark-evidence/offline-fixtures-v151.json) artifact. Its +[`offline-fixtures-v149.json`](docs/benchmark-evidence/offline-fixtures-v149.json) artifact. Its SHA-256 is -`08ed1a6451af926cde1f5ba4287556a738e08fe568c4ec6abf39758bb60d5bc5`, also recorded in the +`d5d36c55c4303d77b161137521dd31f77f39b7a0c9e2fed3ddb63e303b12cc6d`, also recorded in the adjacent `.sha256` file. The artifact contains no raw questions, answers, prompts, customer data, or per-record content fingerprints. The fixture-suite digest is -`034720202d3f805aa2edcd26c8522d745d62a72b1da3379b73d7f329dfd9de8a`. The artifact defines +`b0caf9b03349d20da8e30fc9d75b415ed498f0670e9973fc0196d1327c08ba6e`. The artifact defines the digest algorithm and records the SHA-256 of every suite and dataset file. Each evidence ID also binds its exact command through `sha256(UTF-8 exact command)`: @@ -126,13 +126,13 @@ grounded checks in separate panels; provider billing and MCP transport are not m the SVG and matching PNG with: ```bash -python scripts/render_benchmark_report.py --report docs/benchmark-evidence/offline-fixtures-v151.json --output docs/images/context-efficiency.svg --png-output docs/images/context-efficiency.png +python scripts/render_benchmark_report.py --report docs/benchmark-evidence/offline-fixtures-v149.json --output docs/images/context-efficiency.svg --png-output docs/images/context-efficiency.png ``` The companion examples are also generated from that artifact with: ```bash -python -m scripts.render_benchmark_examples --report docs/benchmark-evidence/offline-fixtures-v151.json --output docs/images/evidence-backed-agent-examples.svg --png-output docs/images/evidence-backed-agent-examples.png +python -m scripts.render_benchmark_examples --report docs/benchmark-evidence/offline-fixtures-v149.json --output docs/images/evidence-backed-agent-examples.svg --png-output docs/images/evidence-backed-agent-examples.png ``` The historical-to-executable mapping is in [`docs/BENCHMARK_CHANGE_COVERAGE.md`](docs/BENCHMARK_CHANGE_COVERAGE.md). From 4622281a5d1aeb8fe9073eaf202196993e8e8e86 Mon Sep 17 00:00:00 2001 From: Jaixii Date: Sun, 4 Oct 2026 14:07:22 -0400 Subject: [PATCH 20/22] docs: sync benchmark contract with parent stack --- tests/test_benchmark_evidence.py | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/tests/test_benchmark_evidence.py b/tests/test_benchmark_evidence.py index 9122f5f4..63a3e4ce 100644 --- a/tests/test_benchmark_evidence.py +++ b/tests/test_benchmark_evidence.py @@ -33,8 +33,8 @@ ROOT = Path(__file__).resolve().parents[1] -PUBLIC_OFFLINE_ARTIFACT = "offline-fixtures-v151.json" -PUBLIC_OFFLINE_SHA = "08ed1a6451af926cde1f5ba4287556a738e08fe568c4ec6abf39758bb60d5bc5" +PUBLIC_OFFLINE_ARTIFACT = "offline-fixtures-v149.json" +PUBLIC_OFFLINE_SHA = "d5d36c55c4303d77b161137521dd31f77f39b7a0c9e2fed3ddb63e303b12cc6d" @pytest.fixture(scope="module") From 34982ea4d9a1b978a4df66dd40abc567fc9bf8e7 Mon Sep 17 00:00:00 2001 From: Jaixii Date: Sun, 4 Oct 2026 14:07:28 -0400 Subject: [PATCH 21/22] docs: sync benchmark contract with parent stack --- docs/images/context-efficiency.svg | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/images/context-efficiency.svg b/docs/images/context-efficiency.svg index a6db92e5..5286d17b 100644 --- a/docs/images/context-efficiency.svg +++ b/docs/images/context-efficiency.svg @@ -1,6 +1,6 @@ Offline context, retrieval, and grounded benchmark results -Artifact-driven offline benchmark report with context efficiency, retrieval quality, and grounded behavior reported separately. Structure-aware chunking reports 740.3 to 214.3 retrieved tokens per question. The JSON-shape payload proxy reports 24,590 full versus 11,138 compact tokens; MCP transport was not measured; provider billing was not measured. Retrieved candidate quality is Recall@5 1.000, Hit@5 1.000, and answer-token recall 1.000 across 26 questions. Packed context quality is Recall@5 1.000, Hit@5 1.000, and answer-token recall 1.000 across 26 questions. Grounded checks score 5 of 5 answerable queries grounded and 6 of 6 abstention queries rejected, including 1 of 1 quarantined-evidence checks. Decision accuracy is 1.000 across 11 decisions. Source artifact SHA-256 08ed1a6451af926cde1f5ba4287556a738e08fe568c4ec6abf39758bb60d5bc5. +Artifact-driven offline benchmark report with context efficiency, retrieval quality, and grounded behavior reported separately. Structure-aware chunking reports 740.3 to 214.3 retrieved tokens per question. The JSON-shape payload proxy reports 24,590 full versus 11,138 compact tokens; MCP transport was not measured; provider billing was not measured. Retrieved candidate quality is Recall@5 1.000, Hit@5 1.000, and answer-token recall 1.000 across 26 questions. Packed context quality is Recall@5 1.000, Hit@5 1.000, and answer-token recall 1.000 across 26 questions. Grounded checks score 5 of 5 answerable queries grounded and 6 of 6 abstention queries rejected, including 1 of 1 quarantined-evidence checks. Decision accuracy is 1.000 across 11 decisions. Source artifact SHA-256 d5d36c55c4303d77b161137521dd31f77f39b7a0c9e2fed3ddb63e303b12cc6d. @@ -65,6 +65,6 @@ Quarantined-evidence probe 1 / 1 abstained Decision accuracy 1.000 (11 / 11); the quarantine probe is included in the abstention total. -SOURCE SHA-256 08ed1a6451af +SOURCE SHA-256 d5d36c55c430 JSON proxy only; MCP transport not measured; provider billing not measured. - + \ No newline at end of file From 6defd55934712a08926838b301b9f940134d994b Mon Sep 17 00:00:00 2001 From: Jaixii Date: Sun, 4 Oct 2026 14:07:33 -0400 Subject: [PATCH 22/22] docs: sync benchmark contract with parent stack --- docs/images/evidence-backed-agent-examples.svg | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/images/evidence-backed-agent-examples.svg b/docs/images/evidence-backed-agent-examples.svg index 6522687f..d98f103e 100644 --- a/docs/images/evidence-backed-agent-examples.svg +++ b/docs/images/evidence-backed-agent-examples.svg @@ -1,6 +1,6 @@ Three evidence-backed Engraphis agent behaviors - A three-card summary of deterministic offline fixtures. Focused context returns 740.3 to 214.3 tokens while retaining Recall at 5 of 1.000. A grounded answer returns support for 5/5 answerable questions. An unsupported question safely abstains for 6/6 off-topic questions. Reproduce with eval.chunking_eval and eval.grounded. Exact commands and config digests are registered in BENCHMARKS.md. Public-safe artifact SHA-256: 08ed1a6451af926cde1f5ba4287556a738e08fe568c4ec6abf39758bb60d5bc5. + A three-card summary of deterministic offline fixtures. Focused context returns 740.3 to 214.3 tokens while retaining Recall at 5 of 1.000. A grounded answer returns support for 5/5 answerable questions. An unsupported question safely abstains for 6/6 off-topic questions. Reproduce with eval.chunking_eval and eval.grounded. Exact commands and config digests are registered in BENCHMARKS.md. Public-safe artifact SHA-256: d5d36c55c4303d77b161137521dd31f77f39b7a0c9e2fed3ddb63e303b12cc6d. @@ -47,5 +47,5 @@ Reproduce: eval.chunking_eval + eval.grounded - SHA256 08ed1a6451af926cde1f5ba4287556a738e08fe568c4ec6abf39758bb60d5bc5 - + SHA256 d5d36c55c4303d77b161137521dd31f77f39b7a0c9e2fed3ddb63e303b12cc6d + \ No newline at end of file