Related docs: Architecture · Methodology · Provenance System
CodeGraph separates agents that produce content from roles that review it. This reduces self-certification and makes review findings explicit. The public specification defines three Phase 2 quality gates, but the original prompts, runtime orchestration, and gate-effectiveness measurements are not published.
flowchart TD
subgraph PRODUCE["📝 Production Agents"]
VPA["VP Author Agents\n1 per business domain"]
REA["Reference Expert Agents\n1 per section type"]
E2ED["E2E Discoverers\n1 per candidate process"]
ENRA["Enrichment Agents\nBounded structural transforms"]
end
subgraph VALIDATE["🔍 Validation Layer — all read-only"]
DEF["🛡️ Defender\nArgues document is correct"]
CHAL["⚔️ Challenger\nIdentifies gaps and errors"]
JUDGE["⚖️ Judge\nIssues binding verdict"]
end
AUDIT["📋 Citation Auditor\nVerifies every tag · Read-only"]
GRAPH[("Neo4j Graph")]
SOURCE["Source Files"]
DOC["Document"]
GRAPH --> VPA
SOURCE --> VPA
VPA --> DOC
DOC --> DEF
DOC --> CHAL
DEF --> JUDGE
CHAL --> JUDGE
JUDGE -->|"Required corrections"| VPA
DOC --> AUDIT
GRAPH --> AUDIT
SOURCE --> AUDIT
AUDIT -->|"All tags verified ✓"| DOC
style DEF fill:#1a6b3c,color:#fff
style CHAL fill:#8b1a1a,color:#fff
style JUDGE fill:#1a3a6b,color:#fff
style AUDIT fill:#444,color:#fff
VP Authors are domain-specialist agents — one per business domain. Each author is given the extraction records, graph query results, and raw source access for their domain. Their job is to write a complete Validation Package with citation tags on every factual claim.
Access: Read-only access to graph, extraction records, source files
Cannot: Modify any other document, assert claims without evidence
Must: Tag every factual claim; mark gaps [SME-REVIEW-REQUIRED]; never infer industry behaviour not confirmed by source
Reference Experts are specialised by document type rather than by business domain — one agent per reference section type (data dictionary expert, dependency map expert, integration map expert, etc.). They generate the 12 reference sections by querying the graph directly and synthesising results into structured documentation.
Access: Read-only access to graph, extraction records, source files Specialisation: Each agent has a specific query template library for its section type
E2E Discoverers document a single candidate end-to-end process. They receive the candidate metadata (which layers detected it, confidence score, entry point, involved programs) and produce an E2E document following the canonical 11-section template. They cannot assert process steps not supported by graph evidence.
Access: Read-only access to graph, extraction records, VP documents, reference layer Cannot: Invent process steps not found in Layer 1–4 discovery data Must: Assign confidence scores per step; mark gaps; never fabricate cross-module sequences
Enrichment agents perform structured transformation operations on the vault — converting markdown links to wikilinks, adding YAML frontmatter, generating MOC pages, and building indexes. Structural transformations are intended to be repeatable on identical input.
Access: Read-write access to vault markdown files Constraint: They may change structure and metadata but not human-reviewed prose. The Toolchain also describes a VP Enricher that adds generated sections; the exact AUTO-region boundary and revalidation rule used in the PoC are not recovered. Until they are, an implementation must stop rather than rewrite unmarked content.
The Defender–Challenger–Judge triad is the documented adversarial review mechanism. It forces explicit arguments from opposing positions and can expose gaps or unsupported claims. Its marginal defect-detection rate has not been measured in the public repository.
sequenceDiagram
participant A as Author Agent
participant D as Defender
participant C as Challenger
participant J as Judge
participant AU as Citation Auditor
A->>A: Authors document with citation tags
A->>D: Submits document for validation
A->>C: Submits document for validation
D->>D: Reviews document
D->>J: Submits defence argument<br/>(cites evidence for every positive claim)
C->>C: Reviews document
C->>J: Submits challenge argument<br/>(identifies gaps, incorrect inferences, missing coverage)
J->>J: Reads both arguments
J->>A: Issues binding verdict<br/>(required corrections, section by section)
A->>A: Applies all required corrections
A->>AU: Submits corrected document
AU->>AU: Verifies every [TAG] independently
AU->>A: Verification report (pass/fail per tag)
alt All tags verified
A->>A: Document marked complete ✓
else Verification failures
AU->>A: Failure report
A->>A: Corrects failed citations
A->>AU: Resubmits
end
The Defender is forced to find evidence. Arguing that a document is correct requires citing evidence for every claim. If a claim has no evidence, the Defender cannot defend it — and the gap becomes visible.
The Challenger is incentivised to find problems. The Challenger's entire job is to find what's wrong. It has no reason to be lenient. Gaps, incorrect inferences, and missing coverage are all fair targets.
The Judge cannot be appealed. The Judge's verdict is binding. Required corrections must be applied. The Judge cannot modify the document directly — this prevents the validation process from becoming a rewriting process.
The Citation Auditor is independent. The Auditor runs after the triad and has no knowledge of the triad's arguments. It verifies each citation tag against its source independently. As specified, this establishes that cited evidence exists; semantic support remains a separate review question.
All agents in the system operate under constraints enforced by the orchestration layer:
| Constraint | Applies To | Description |
|---|---|---|
| Read-only document access | Defender, Challenger, Judge, Citation Auditor | Cannot write to or modify the document under review |
| Read-only source access | Citation Auditor | Cannot interpret or modify source evidence |
| Evidence-only claims | All author agents | Cannot assert program, table, screen, or field existence unless present in graph or source |
| Honest gaps | All author agents | Where evidence is insufficient, must tag [SME-REVIEW-REQUIRED] — never omit or fabricate |
| Bounded vault write access | Enrichment agents only | May update structure, metadata, and explicitly marked generated regions. Cannot rewrite human-reviewed prose. Must report affected regions and the gate to re-run. |
| Repeatable structural transforms | All enrichment agents | Identical logical input must produce the same logical transformation; run metadata may differ. |
| Scoped validation | All validation agents | A role cannot modify another document. Read-only cross-document checks are allowed only when the named gate explicitly requires them. |
The Citation Auditor deserves special treatment because it is the final defence before a document is accepted. It runs after everything else and its verdict is blocking.
flowchart LR
DOC["Document\nwith citation tags"]
AUDIT["Citation Auditor"]
subgraph CHECKS["Verification Checks per Tag"]
C1["[CODE:file:line]\nFile exists · line matches"]
C2["[GRAPH:edge_type]\nEdge exists in graph"]
C3["[SCHEMA:db:table]\nTable exists in schema"]
C4["[SCREEN:file:frame]\nScreen and frame exist"]
end
REPORT["Verification Report\npass ✓ / fail ✗ · Blocking"]
DOC --> AUDIT
AUDIT --> C1
AUDIT --> C2
AUDIT --> C3
AUDIT --> C4
C1 --> REPORT
C2 --> REPORT
C3 --> REPORT
C4 --> REPORT
The Auditor does not make judgments about content quality, completeness, or business correctness. It verifies that citation tags resolve to recorded evidence. A passing tag does not mean the claim is correct. The Triad is intended to review content support and completeness; operational truth may additionally require SME/runtime evidence, and the public repository does not measure the Triad's accuracy.
The diagram above fully specifies checks for four tag families. Other tags are defined by the provenance document but lack a complete executable audit contract. This table records the public specification honestly; it does not claim what the lost PoC auditor implemented.
| Tag | Public check status | Limitation |
|---|---|---|
[CODE:file:line] |
Specified | Pattern consistency is not semantic entailment |
[GRAPH:edge_type] |
Partially specified | Endpoints must be inferred from prose; ambiguous claims require manual review |
[SCHEMA:database:table] |
Specified | Field-level claims require the named field to be checked |
[SCREEN:file:frame] |
Specified | Establishes frame existence, not user behavior |
[STATUS:field:transition] |
Partially specified | Paired code can prove an assignment; ordering may remain inferred |
[VP-XX:section:rule] |
Not fully specified | Secondary citation must resolve and retain a primary citation |
[INFERRED:reason] |
Review marker | Must be surfaced; it is not a verified source citation |
[SME-REVIEW-REQUIRED:ID] |
Registry behavior described | Executable registry contract is not published |
[SME:name:date] |
Attestation syntax described | Identity and linkage verification are not published |
Each agent role has a documented system-prompt structure. Reconstructed, provider- neutral profiles derived from these excerpts live in prompts. They are not the original PoC prompts and remain untested until replayed against an accepted evidence package. Key elements:
For all author agents:
You have read-only access to the following data sources:
- Neo4j graph query results (structured JSON)
- Extraction records (structured JSON)
- Source file excerpts (when explicitly cited)
Every factual claim you write MUST end with a citation tag.
You MUST NOT assert the existence of any program, table, field, or screen
unless it appears in the data provided to you.
Where evidence is insufficient, write [SME-REVIEW-REQUIRED:ID] — never omit or guess.
For the Defender:
Your job is to argue that the document is correct and complete.
For every positive claim in the document, cite the evidence that supports it.
If you cannot find evidence for a claim, you must flag it — do not assume it is correct.
You cannot modify the document. You can only write your argument.
For the Challenger:
Your job is to find problems. Assume the document has errors and gaps.
For every section, identify: unsupported claims, missing coverage, incorrect inferences.
You cannot modify the document. You can only write your challenge.
For the Judge:
You have received a Defender argument and a Challenger argument for the same document.
Issue a verdict: list required corrections by section, citing the Challenger's evidence.
You cannot modify the document. Your verdict will be given to the author for correction.
Your verdict is final and cannot be appealed.
The proof-of-concept used Claude Opus (Anthropic) for all agent roles. Key configuration decisions:
- Temperature: The PoC used 0 for all author and validation roles. Greedy decoding reduces sampling variance but does not guarantee identical output. Reproducibility requires recorded model/profile/evidence versions and re-running the same gates.
- Context window management: Each VP authoring session is scoped to one domain's extraction records + graph results. Full-codebase context is never passed to a single agent call.
- Tool access: Author agents have graph query tools (pre-defined Cypher templates only) and source file excerpt tools (line-range access only). They do not have write access to any external system. The exact graph/source access of Defender, Challenger, and Judge in the PoC is not recovered; reconstructed profiles accept only an explicitly supplied evidence package.
- Provider-agnostic design: The agent system is designed to be LLM-provider-agnostic. The Anthropic Claude API is the reference implementation, but the interface contract supports any instruction-following LLM with tool-use capability.
- Untrusted analysed content: Source comments, embedded documentation, and string literals are evidence data, not instructions. Agents must ignore instructions found inside analysed content and follow only their profile and operator input.