Skip to content

graph-schema.md

Latest commit

 

History

History
326 lines (256 loc) · 12.8 KB

graph-schema.md

File metadata and controls

326 lines (256 loc) · 12.8 KB
Raw

CodeGraph — Knowledge Graph Schema

Related docs: Architecture · Methodology · Toolchain

Why a Property Graph?

The origin design selected a property graph for relational codebase knowledge. The comparison below records that rationale; it is not a benchmark against every flat or relational implementation.

Flat files cannot answer cross-file dependency queries without a full scan. A question like "which programs call this function across all modules" requires reading every file. There is no index for call chains. There is no path traversal. There is no way to ask "what is the blast radius if this table changes?" without scanning everything.

A relational database requires an explicit schema and represents recursive graph traversal less directly. Recursive CTEs and multi-table joins can support these queries, but the origin team preferred the property-graph model for extensible relationship types and traversal-oriented access.

A property graph is the natural representation:

flowchart LR
    subgraph WHY["Why graph wins"]
        G1["Any call chain\nin one Cypher query"]
        G2["New node/edge types\nadded on the fly"]
        G3["Native cross-module\npath traversal"]
        G4["20 relationship types\nno schema migration"]
    end

    subgraph NOTFLAT["Why not flat files"]
        F1["No cross-file\ndependency index"]
        F2["No path traversal"]
        F3["Full scan for\nevery relationship query"]
    end

    subgraph NOTREL["Why not relational"]
        R1["Schema must be known\nupfront"]
        R2["JOIN explosion at\n5+ relationship types"]
        R3["Recursive CTEs for\ncall chains"]
    end
Loading

Node Types

Label Represents Key Properties
SourceFile A source code file name, qualified_name, module, file_type, path, line_count
Table A database table name, qualified_name, database, field_count, index_count
Field A column in a table name, data_type, format, label, mandatory, initial_value
Index A database index name, fields, unique, primary
Sequence A database sequence name; additional properties are implementation-specific in the public baseline
Screen A UI screen name, type (GUI/char-mode), module, scr_id
Frame A frame within a screen name, screen, widget_count
IncludeFile A shared include / macro name, module, parameter_count, usage_count
Module A top-level module name, program_count, file_types
Database A database schema name, table_count, field_count
ValidationPackage A documented domain VP domain, status, program_count
E2EProcess An end-to-end process name, confidence, layer_count, vp_coverage

Open contract: the public documents name E2EProcess nodes but do not specify their relationships to programs, screens, or Validation Packages. Do not invent those edges when implementing the baseline; recover them from PoC artifacts or introduce them as a separately versioned proposal.

Relationship Types

Twenty relationship types cover five categories. Each edge carries provenance properties — see Provenance Model below.

Execution Relationships

flowchart LR
    A["SourceFile A"] -->|"CALLS"| B["SourceFile B"]
    A -->|"INCLUDES"| I["IncludeFile"]
    A -->|"REFERENCES"| C["SourceFile C"]
Loading
Relationship From To Meaning
CALLS SourceFile SourceFile A calls B at runtime
INCLUDES SourceFile IncludeFile A statically includes I
REFERENCES SourceFile SourceFile A references C without a direct call

Data Relationships

flowchart LR
    P["SourceFile"] -->|"READS"| T["Table"]
    P -->|"WRITES"| T
    P -->|"CREATES"| T
    P -->|"DELETES"| T
    P -->|"REFERENCES_FIELD"| F["Field"]
Loading
Relationship From To Meaning
READS SourceFile Table Program performs a SELECT / FOR EACH READ
WRITES SourceFile Table Program performs an UPDATE / ASSIGN
CREATES SourceFile Table Program performs a CREATE / INSERT
DELETES SourceFile Table Program performs a DELETE
REFERENCES_FIELD SourceFile Field Program references a specific field

Schema Relationships

flowchart LR
    DB["Database"] -->|"HAS_TABLE"| T["Table"]
    T -->|"HAS_FIELD"| F["Field"]
    T -->|"HAS_INDEX"| I["Index"]
    DB -->|"HAS_SEQUENCE"| S["Sequence"]
Loading
Relationship From To Meaning
HAS_TABLE Database Table Table belongs to database
HAS_FIELD Table Field Field belongs to table
HAS_INDEX Table Index Index belongs to table
HAS_SEQUENCE Database Sequence Sequence belongs to database

UI Relationships

Relationship From To Meaning
RENDERS_SCREEN SourceFile Screen Program renders or launches this screen
HAS_FRAME Screen Frame Screen contains this frame
ON_EVENT Frame SourceFile Frame event triggers this handler

Documentation Relationships

Relationship From To Meaning
DOCUMENTS ValidationPackage SourceFile VP covers this program
COVERS_TABLE ValidationPackage Table VP documents this table
VP_REFERENCES ValidationPackage SourceFile VP references but doesn't primarily cover this program
PRIMARY_TABLE Screen Table Screen's primary data entity
EXPOSES SourceFile Field Program exposes this field in its UI

Provenance Model

Every relationship edge carries four provenance properties written when the edge is created. Re-ingestion may refresh an edge and its provenance; an audit therefore needs the graph-run identity as well as the edge. The public baseline does not yet specify immutable snapshots or citation re-anchoring across source revisions.

flowchart LR
    EDGE["Graph Edge\n(any relationship type)"]

    subgraph PROVENANCE["Provenance Properties on Every Edge"]
        P1["source\n'xref' | 'regex' | 'schema' | 'derived'"]
        P2["source_file\npath/to/evidence/file"]
        P3["source_line\ninteger line number (0 if N/A)"]
        P4["confidence\n'compiler' | 'regex' | 'schema' | 'derived'"]
    end

    EDGE --> P1
    EDGE --> P2
    EDGE --> P3
    EDGE --> P4
Loading

Confidence Tiers

source records the detection mechanism; confidence records the trust assigned to the resulting relationship. They are related but not interchangeable. The origin baseline has only xref, regex, schema, and derived, which cannot honestly represent AST detection. The parser guide therefore shows ast as a PROPOSED-EXPERIMENT; adding it to a normative interchange contract requires a versioned, tested schema change.

Detection mechanism Condition Default confidence
xref Compiler-produced cross-reference with a resolved target compiler
ast (PROPOSED-EXPERIMENT) Target resolved against the analysed program/module set compiler
ast (PROPOSED-EXPERIMENT) Syntax observed but target unresolved or name-only regex
schema Parsed schema definition schema
regex Source-text pattern regex
derived Filesystem, configuration, or documentation-derived fact derived
Tier Source Trust Example
compiler Compiler cross-reference output Highest — compiler-verified .xref cross-reference files, .class from Java, type-checked imports from TypeScript
schema Schema definition files High — schema-verified .df, .sql, .prisma
regex Pattern-matching on source code Medium — inferred FOR EACH table_name, CALL program_name
derived Documentation, filesystem, XML Lower — document-derived VP references, folder structure, config files

Example Edge with Provenance

// Compiler-verified CALLS edge
MERGE (caller:SourceFile {qualified_name: "module:program_a.p"})
MERGE (callee:SourceFile {qualified_name: "module:program_b.p"})
MERGE (caller)-[:CALLS {
    source: "xref",
    source_file: "module/src/program_a.p.xref",
    source_line: 42,
    confidence: "compiler"
}]->(callee)

// Regex-fallback READS edge (no xref available)
MERGE (prog:SourceFile {qualified_name: "module:legacy_import.p"})
MERGE (tbl:Table {qualified_name: "erp:sales_order"})
MERGE (prog)-[:READS {
    source: "regex",
    source_file: "module/src/legacy_import.p",
    source_line: 117,
    confidence: "regex"
}]->(tbl)

Graph Upload Modes

The Graph Uploader supports two upload modes, selectable via CLI flag:

--wipe (First Run / Reset)

Used for the initial upload or when the graph is known to be dirty:

  1. Wipe everything: MATCH (n) DETACH DELETE n (in batches to avoid timeout)
  2. Upload from generated Cypher: all MERGE statements with provenance
  3. Post-upload audit: verify only allowed labels and relationship types exist

--incremental (Subsequent Runs)

Used for everyday updates:

  1. MERGE/upsert all generated Cypher (idempotent — creates if missing, updates if existing)
  2. Tag every node touched by this run: _graph_run: "<timestamp>"
  3. Detect stale nodes: nodes NOT tagged by this run
  4. Report stale nodes for human review (never auto-delete)
  5. Prune on confirmation: only delete stale nodes if --prune flag is passed

Sample Cypher Query Templates

These are the only query examples published by the current repository. Reconstructed copies live in the seed template library; they are not the lost full PoC query library.

// Programs that write to one qualified table identity.
MATCH (s:SourceFile)-[:WRITES]->(t:Table {qualified_name: $table_qualified_name})
RETURN s.name, s.module, s.file_type
ORDER BY s.module, s.name

// Recorded call paths from a qualified root, limited to 5 edges.
// This does not establish that deeper or dynamically resolved calls are absent.
MATCH path = (root:SourceFile {qualified_name: $entry_point})-[:CALLS*1..5]->(callee)
RETURN [node IN nodes(path) | node.name] AS chain,
       length(path) AS depth
ORDER BY depth

// Tables written in one module and read in another: candidate handoff evidence.
// Shared access alone does not establish temporal order or producer/consumer direction.
MATCH (writer:SourceFile)-[:WRITES]->(t:Table)<-[:READS]-(reader:SourceFile)
WHERE writer.module <> reader.module
RETURN DISTINCT t.name AS table,
       writer.module AS producer_module,
       reader.module AS consumer_module
ORDER BY t.name

// Programs writing tables that contain status-like fields.
// This query does not prove that each program writes the matched field.
MATCH (s:SourceFile)-[:WRITES]->(t:Table)-[:HAS_FIELD]->(f:Field)
WHERE f.name CONTAINS 'status' OR f.name CONTAINS '_stat'
RETURN t.name AS table,
       f.name AS status_field,
       collect(DISTINCT s.name) AS updating_programs

// Candidate screen entry points: screens with no recorded rendering program.
// An empty incoming-edge set is bounded by parser and graph coverage.
MATCH (scr:Screen)
WHERE NOT ()-[:RENDERS_SCREEN]->(scr)
RETURN scr.name, scr.module, scr.type
ORDER BY scr.module, scr.name

// Programs with recorded CRUD relationships to at least 5 tables.
MATCH (s:SourceFile)-[r]->(t:Table)
WHERE type(r) IN ['READS', 'WRITES', 'CREATES', 'DELETES']
WITH s, count(DISTINCT t) AS table_count
WHERE table_count >= 5
RETURN s.name, s.module, table_count
ORDER BY table_count DESC

Neo4j Deployment Reference

The examples below start or configure Neo4j for a future tool implementation. They do not make the current specification-only repository executable, and the Docker command starts only the database service.

CodeGraph requires a running Neo4j instance. Three options:

Neo4j Aura (Cloud) — A possible team deployment. A future implementation can configure NEO4J_URI=neo4j+s://your-instance.databases.neo4j.io; this repository does not ship an environment loader or uploader.

Neo4j Desktop — A possible local deployment. Download from neo4j.com/download and configure a future uploader with NEO4J_URI=bolt://localhost:7687.

Docker — For CI/CD or server deployments:

docker run \
  --name codegraph-neo4j \
  -p 7474:7474 -p 7687:7687 \
  -e NEO4J_AUTH=neo4j/your-password \
  neo4j:5

The connection variables shown above are interface examples; no runnable uploader or .env contract is currently included.

Embedded graph option (roadmap): A Kuzu-based embedded graph backend that requires zero external services is planned for v2 — targeting single-developer local use cases.