Explore Lore MCP server as knowledge backend for hierarchical context

[itdove]

Problem

DevAIFlow's hierarchical context system (ENTERPRISE.md, ORGANIZATION.md, TEAM.md, USER.md) is read in full every session start. This causes:

1. Token waste — full file reads every session, even when only a fraction is relevant
2. No persistence — knowledge derived mid-session is lost when the session ends
3. No search — agent must read entire files to find relevant context
4. No conflict resolution — if enterprise and team contexts contradict, no detection or priority mechanism beyond read order
5. No cross-repo sharing — context is local to the DevAIFlow config directory
6. context_files.py is already deprecated in favor of skills, but skills have the same read-everything problem

Proposed Solution

Integrate Lore as the knowledge backend for DevAIFlow's hierarchical context.

Lore is an MCP server providing persistent, cross-team memory for AI coding agents with:

• N-level hierarchy with priority resolution (maps to enterprise/org/team/user)
• SQLite + FTS5 persistent store with cheap LLM synthesis for retrieval
• Git-backed shared knowledge with PR review for writes
• Conflict detection with bidirectional links and lock mechanism
• Auto-capture from session transcripts at session end
• Cross-repo knowledge spanning repositories

What Lore replaces

Component	Replace with Lore?	Notes
Context files (ENTERPRISE.md etc)	Yes	Lore levels 1-4 replace these — query instead of full file read
Config hierarchy (enterprise.json etc)	No	Structured config (JSON) for field defaults, JIRA mappings, model provider enforcement — not knowledge
Skills hierarchy (01-enterprise/ etc)	No	Executable instructions, not knowledge entries
Session knowledge re-derivation	Yes	Core Lore value — persistent store eliminates re-reading

Integration points

1. Session start — daf-workflow skill calls query_knowledge() instead of reading 4 markdown files
2. Mid-session — agent stores discoveries via Lore MCP tools at appropriate hierarchy level
3. Session end — daf complete triggers Lore auto-capture from session transcript
4. Config — DevAIFlow maps its hierarchy levels to Lore levels:
   • Level 1 → Enterprise (git repo, PR-reviewed)
   • Level 2 → Organization (git repo, PR-reviewed)
   • Level 3 → Team (git repo, PR-reviewed)
   • Level 4 → User (local SQLite, immediate writes)

Migration path

1. Phase 1: Add Lore as optional MCP dependency — if available, use it; if not, fall back to current .md files
2. Phase 2: Migrate existing context files to Lore entries via daf config import-to-lore
3. Phase 3: Deprecate .md context files, Lore becomes primary knowledge backend
4. Phase 4: Enable auto-capture from session transcripts in daf complete

Out of Scope

• Replacing DevAIFlow's JSON config hierarchy (enterprise.json, organization.json, team.json, config.json)
• Replacing the skills system
• Lore development itself (tracked in itdove/lore repo)

Acceptance Criteria

• Document mapping between DevAIFlow hierarchy levels and Lore levels
• Prototype session start using query_knowledge() instead of reading context files
• Verify Lore MCP server works alongside DevAIFlow's existing MCP setup
• Define fallback behavior when Lore is not available
• Identify what context file content migrates to Lore vs stays as config/skills
• Evaluate token savings from query-based retrieval vs full file reads

References

• Lore repo: https://github.com/itdove/lore
• Current context_files.py: devflow/utils/context_files.py (already deprecated)
• Current hierarchy skills: ~/.config/devaiflow/.claude/skills/01-enterprise/ through 04-user/
• Config hierarchy: devflow/config/loader.py