From 257a295e0bd5516c16aaac48fbeaee58b4c5e710 Mon Sep 17 00:00:00 2001 From: Eran Kuris Date: Mon, 13 Jul 2026 17:23:50 +0300 Subject: [PATCH 01/17] docs: add architecture diagram Co-Authored-By: Claude Opus 4.6 (1M context) --- docs/architecture.md | 140 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 140 insertions(+) create mode 100644 docs/architecture.md diff --git a/docs/architecture.md b/docs/architecture.md new file mode 100644 index 00000000..7245d491 --- /dev/null +++ b/docs/architecture.md @@ -0,0 +1,140 @@ +# Forge Architecture + +## System Overview + +``` +┌─────────────────────────────────────────────────────────────────────────┐ +│ External Systems │ +│ │ +│ ┌──────────┐ ┌──────────┐ ┌───────────┐ ┌────────────────┐ │ +│ │ Jira │ │ GitHub │ │ Deep │ │ Langfuse │ │ +│ │ │ │ │ │ Agents │ │ (Observability)│ │ +│ └────┬─────┘ └────┬─────┘ └───────────┘ └────────────────┘ │ +│ │ │ │ +└────────┼───────────────┼────────────────────────────────────────────────┘ + │ webhooks │ webhooks + ▼ ▼ +┌─────────────────────────────────────────────────────────────────────────┐ +│ FastAPI Server (:8000) │ +│ │ +│ POST /api/v1/webhooks/jira POST /api/v1/webhooks/github │ +│ GET /api/v1/health GET /api/v1/metrics │ +│ │ +└────────────────────────────┬────────────────────────────────────────────┘ + │ enqueue events + ▼ +┌─────────────────────────────────────────────────────────────────────────┐ +│ Redis Streams ("forge:events") │ +│ │ +│ Consumer group: forge-workers State: AsyncRedisSaver │ +│ Per-ticket thread IDs (AISOS-123) LangGraph checkpointing │ +│ │ +└────────────────────────────┬────────────────────────────────────────────┘ + │ dequeue + dispatch + ▼ +┌─────────────────────────────────────────────────────────────────────────┐ +│ LangGraph Orchestrator │ +│ │ +│ Orchestrator.resume(event) → StateGraph(WorkflowState) │ +│ │ +│ ┌─────────────────────────────────────────────────────────────────┐ │ +│ │ Workflow Nodes │ │ +│ │ │ │ +│ │ CLASSIFY ─┬─► [Bug] TRIAGE → RCA_OPTIONS → RCA_GATE → RCA │ │ +│ │ │ │ │ │ +│ │ └─► [Feature] ──────────────────────────┤ │ │ +│ │ ▼ │ │ +│ │ PRD → PRD_GATE ─► SPEC → SPEC_GATE │ │ +│ │ │ │ │ +│ │ ▼ │ │ +│ │ PLAN → PLAN_GATE ─► TASKS → TASK_GATE │ │ +│ │ │ │ │ +│ │ ▼ │ │ +│ │ IMPLEMENT ─► PR_CREATE ─► CI_GATE ◄─┐ │ │ +│ │ │ │ │ │ +│ │ CI_EVALUATOR┤ CI_FIX (retry) │ │ +│ │ │ │ │ │ +│ │ ▼ │ │ │ +│ │ HUMAN_REVIEW_GATE───┘ │ │ +│ │ │ │ │ +│ │ MERGE → DONE │ │ +│ │ │ │ +│ │ ⏸ Gates = human checkpoints (skipped with forge:yolo label) │ │ +│ └─────────────────────────────────────────────────────────────────┘ │ +│ │ +└───────────────┬─────────────────────────────┬───────────────────────────┘ + │ │ + ▼ ▼ +┌──────────────────────────┐ ┌──────────────────────────────────────────┐ +│ Claude LLM │ │ Podman Container (ephemeral) │ +│ │ │ │ +│ Anthropic API │ │ ┌─────────────────────────────────┐ │ +│ or Vertex AI │ │ │ Deep Agents + MCP tools │ │ +│ │ │ │ .forge/task.json │ │ +│ Used by: classify, │ │ │ workspace mounted │ │ +│ triage, RCA, PRD, │ │ │ commits changes locally │ │ +│ spec, plan, CI eval │ │ └─────────────────────────────────┘ │ +│ │ │ │ +└──────────────────────────┘ └──────────────────────────────────────────┘ +``` + +## Ticket Lifecycle + +``` +Jira ticket created + labeled "forge:managed" + │ + ▼ +CLASSIFY ── Claude determines: bug or feature? + │ + ├── Bug ────► TRIAGE → RCA_OPTIONS → ⏸ RCA_OPTION_GATE → RCA_DEEP_DIVE + │ (user picks ">option N") + │ + └── Feature ─────────────────────────────────────┐ + │ + ◄─────────────────────────────────────────────────┘ + │ + ▼ +PRD ── generate requirements ── ⏸ PRD_GATE (Jira or proposals PR) + │ + ▼ +SPEC ── generate tech design ── ⏸ SPEC_GATE + │ + ▼ +PLAN ── generate impl plan ──── ⏸ PLAN_GATE + │ + ▼ +TASKS ── break into work items ─ ⏸ TASK_GATE + │ + ▼ +IMPLEMENT ── Podman container + Deep Agents executes code + │ + ▼ +PR_CREATE ── open GitHub PR + │ + ▼ +CI_GATE ── wait for checks ──► CI_EVALUATOR + │ │ + │ ┌── pass ────────┤ + │ │ └── fail ──► ATTEMPT_CI_FIX ─┐ + │ │ (retry ≤N) │ + │ │ ◄────────────────────────────┘ + ▼ ▼ +HUMAN_REVIEW_GATE ── ⏸ wait for PR approval + │ + ▼ +MERGE ── merge PR ──► DONE ✓ + +⏸ = human checkpoint (auto-approved when forge:yolo label is set) +``` + +## Data Flow Summary + +``` +Inbound events: Jira/GitHub webhooks → FastAPI → Redis Streams +State persistence: Redis (LangGraph AsyncRedisSaver, keyed by ticket) +LLM calls: Orchestrator nodes → Claude (Anthropic API / Vertex AI) +Code execution: IMPLEMENT node → Podman container → Deep Agents +Outbound actions: Jira (comments, labels, transitions) + GitHub (PRs, branches, reviews) +Observability: Langfuse (LLM traces, workflow spans, costs) +``` From 4530c3a9719b809da063c4d1c4e13251c6d3b871 Mon Sep 17 00:00:00 2001 From: Eran Kuris Date: Mon, 13 Jul 2026 19:10:10 +0300 Subject: [PATCH 02/17] docs: fix architecture diagram inaccuracies Verified against actual source code and corrected: - Remove fictional CLASSIFY node (routing is done by WorkflowRouter) - Show Feature and Bug as separate graphs with correct node names - Fix Redis stream names (forge:events:jira, forge:events:github) - Fix metrics endpoint path (/metrics, not /api/v1/metrics) - Fix container task path (/task.json, not .forge/task.json) - Add Gemini as a supported LLM backend - Add missing nodes: local_review, update_documentation, setup/teardown_workspace, task_router, completion/aggregation Co-Authored-By: Claude Opus 4.6 (1M context) --- docs/architecture.md | 187 ++++++++++++++++++++++++++++--------------- 1 file changed, 124 insertions(+), 63 deletions(-) diff --git a/docs/architecture.md b/docs/architecture.md index 7245d491..149f94b4 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -18,122 +18,183 @@ │ FastAPI Server (:8000) │ │ │ │ POST /api/v1/webhooks/jira POST /api/v1/webhooks/github │ -│ GET /api/v1/health GET /api/v1/metrics │ +│ GET /api/v1/health GET /metrics │ │ │ └────────────────────────────┬────────────────────────────────────────────┘ │ enqueue events ▼ ┌─────────────────────────────────────────────────────────────────────────┐ -│ Redis Streams ("forge:events") │ +│ Redis │ │ │ -│ Consumer group: forge-workers State: AsyncRedisSaver │ -│ Per-ticket thread IDs (AISOS-123) LangGraph checkpointing │ +│ Streams: forge:events:jira State: AsyncRedisSaver │ +│ forge:events:github LangGraph checkpointing │ +│ Consumer group: forge-workers Per-ticket thread IDs (AISOS-123) │ │ │ └────────────────────────────┬────────────────────────────────────────────┘ │ dequeue + dispatch ▼ ┌─────────────────────────────────────────────────────────────────────────┐ -│ LangGraph Orchestrator │ +│ WorkflowRouter │ │ │ -│ Orchestrator.resume(event) → StateGraph(WorkflowState) │ +│ Selects graph based on Jira issue type: │ +│ Feature/Story → FeatureWorkflow (StateGraph) │ +│ Bug → BugWorkflow (StateGraph) │ │ │ -│ ┌─────────────────────────────────────────────────────────────────┐ │ -│ │ Workflow Nodes │ │ -│ │ │ │ -│ │ CLASSIFY ─┬─► [Bug] TRIAGE → RCA_OPTIONS → RCA_GATE → RCA │ │ -│ │ │ │ │ │ -│ │ └─► [Feature] ──────────────────────────┤ │ │ -│ │ ▼ │ │ -│ │ PRD → PRD_GATE ─► SPEC → SPEC_GATE │ │ -│ │ │ │ │ -│ │ ▼ │ │ -│ │ PLAN → PLAN_GATE ─► TASKS → TASK_GATE │ │ -│ │ │ │ │ -│ │ ▼ │ │ -│ │ IMPLEMENT ─► PR_CREATE ─► CI_GATE ◄─┐ │ │ -│ │ │ │ │ │ -│ │ CI_EVALUATOR┤ CI_FIX (retry) │ │ -│ │ │ │ │ │ -│ │ ▼ │ │ │ -│ │ HUMAN_REVIEW_GATE───┘ │ │ -│ │ │ │ │ -│ │ MERGE → DONE │ │ -│ │ │ │ -│ │ ⏸ Gates = human checkpoints (skipped with forge:yolo label) │ │ -│ └─────────────────────────────────────────────────────────────────┘ │ +│ ┌──────────────────────────────┐ ┌────────────────────────────────┐ │ +│ │ Feature Workflow │ │ Bug Workflow │ │ +│ │ │ │ │ │ +│ │ generate_prd │ │ triage_check │ │ +│ │ → ⏸ prd_approval_gate │ │ → ⏸ triage_gate │ │ +│ │ generate_spec │ │ analyze_bug ◄─┐ │ │ +│ │ → ⏸ spec_approval_gate │ │ → reflect_rca ─┘ (≤3x) │ │ +│ │ decompose_epics │ │ → ⏸ rca_option_gate │ │ +│ │ → ⏸ plan_approval_gate │ │ plan_bug_fix │ │ +│ │ generate_tasks │ │ → ⏸ plan_approval_gate │ │ +│ │ → ⏸ task_approval_gate │ │ decompose_plan │ │ +│ │ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ │ │ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ │ │ +│ │ task_router (per-repo) │ │ Shared implementation path: │ │ +│ │ setup_workspace │ │ (same nodes as Feature) │ │ +│ │ implement_task │ │ │ │ +│ │ local_review │ │ Post-merge: │ │ +│ │ update_documentation │ │ post_merge_summary → END │ │ +│ │ create_pr │ │ │ │ +│ │ teardown_workspace │ └────────────────────────────────┘ │ +│ │ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ │ │ +│ │ wait_for_ci_gate │ │ +│ │ ci_evaluator ◄── ci_fix │ │ +│ │ ⏸ human_review_gate │ │ +│ │ complete_tasks │ │ +│ │ aggregate_epic_status │ │ +│ │ aggregate_feature_status │ │ +│ │ → END │ │ +│ └──────────────────────────────┘ │ +│ │ +│ ⏸ = human checkpoint (auto-approved with forge:yolo label) │ │ │ └───────────────┬─────────────────────────────┬───────────────────────────┘ │ │ ▼ ▼ ┌──────────────────────────┐ ┌──────────────────────────────────────────┐ -│ Claude LLM │ │ Podman Container (ephemeral) │ -│ │ │ │ -│ Anthropic API │ │ ┌─────────────────────────────────┐ │ -│ or Vertex AI │ │ │ Deep Agents + MCP tools │ │ -│ │ │ │ .forge/task.json │ │ -│ Used by: classify, │ │ │ workspace mounted │ │ -│ triage, RCA, PRD, │ │ │ commits changes locally │ │ -│ spec, plan, CI eval │ │ └─────────────────────────────────┘ │ +│ LLM Backends │ │ Podman Container (ephemeral) │ │ │ │ │ +│ Anthropic API (Claude) │ │ ┌─────────────────────────────────┐ │ +│ Vertex AI (Claude) │ │ │ Deep Agents + MCP tools │ │ +│ Vertex AI (Gemini) │ │ │ /task.json (task description) │ │ +│ │ │ │ /workspace (repo mounted) │ │ +│ Used by: PRD, spec, │ │ │ commits changes locally │ │ +│ plan, RCA, triage, │ │ └─────────────────────────────────┘ │ +│ code review, CI eval │ │ │ └──────────────────────────┘ └──────────────────────────────────────────┘ ``` -## Ticket Lifecycle +## Feature Ticket Lifecycle ``` -Jira ticket created + labeled "forge:managed" +Jira ticket created + labeled "forge:managed" (type: Feature or Story) + │ + ▼ +generate_prd ── AI drafts Product Requirements Document + │ + ▼ +⏸ prd_approval_gate ── wait for approval (Jira comment or proposals PR) + │ + ▼ +generate_spec ── AI drafts Technical Specification + │ + ▼ +⏸ spec_approval_gate │ ▼ -CLASSIFY ── Claude determines: bug or feature? +decompose_epics ── AI breaks spec into Jira Epics │ - ├── Bug ────► TRIAGE → RCA_OPTIONS → ⏸ RCA_OPTION_GATE → RCA_DEEP_DIVE - │ (user picks ">option N") + ▼ +⏸ plan_approval_gate + │ + ▼ +generate_tasks ── AI creates implementation Tasks under Epics + │ + ▼ +⏸ task_approval_gate │ - └── Feature ─────────────────────────────────────┐ - │ - ◄─────────────────────────────────────────────────┘ + ▼ +task_router ── group tasks by target repo + │ + ▼ (per repo) +setup_workspace ── clone repo, create branch │ ▼ -PRD ── generate requirements ── ⏸ PRD_GATE (Jira or proposals PR) +implement_task ── Podman container + Deep Agents writes code │ ▼ -SPEC ── generate tech design ── ⏸ SPEC_GATE +local_review ── AI reviews diff, fixes issues (≤2 passes) │ ▼ -PLAN ── generate impl plan ──── ⏸ PLAN_GATE +update_documentation ── update stale docs │ ▼ -TASKS ── break into work items ─ ⏸ TASK_GATE +create_pr ── open GitHub PR │ ▼ -IMPLEMENT ── Podman container + Deep Agents executes code +teardown_workspace ── cleanup (loop back if more repos) │ ▼ -PR_CREATE ── open GitHub PR +wait_for_ci_gate ── wait for CI checks │ ▼ -CI_GATE ── wait for checks ──► CI_EVALUATOR - │ │ - │ ┌── pass ────────┤ - │ │ └── fail ──► ATTEMPT_CI_FIX ─┐ - │ │ (retry ≤N) │ - │ │ ◄────────────────────────────┘ - ▼ ▼ -HUMAN_REVIEW_GATE ── ⏸ wait for PR approval +ci_evaluator ─┬─ pass ──────────────────────────┐ + └─ fail → attempt_ci_fix (≤5x) ───┘ │ ▼ -MERGE ── merge PR ──► DONE ✓ +⏸ human_review_gate ── wait for PR approval + │ (changes_requested → implement_review → CI loop) + ▼ +complete_tasks → aggregate_epic_status → aggregate_feature_status → END ⏸ = human checkpoint (auto-approved when forge:yolo label is set) ``` +## Bug Ticket Lifecycle + +``` +Jira ticket created + labeled "forge:managed" (type: Bug) + │ + ▼ +triage_check ── validate bug report has required fields + │ + ▼ +⏸ triage_gate ── wait for reporter update if incomplete + │ + ▼ +analyze_bug ◄──┐ + │ │ reflect_rca (up to 3 reflection cycles) + └────────────┘ + │ + ▼ +⏸ rca_option_gate ── present fix options, user selects ">option N" + │ + ▼ +plan_bug_fix ── AI generates fix plan + │ + ▼ +⏸ plan_approval_gate + │ + ▼ +decompose_plan ── break into tasks + │ + ▼ +setup_workspace → implement → local_review → create_pr → CI → review + │ + ▼ +post_merge_summary → END +``` + ## Data Flow Summary ``` Inbound events: Jira/GitHub webhooks → FastAPI → Redis Streams State persistence: Redis (LangGraph AsyncRedisSaver, keyed by ticket) -LLM calls: Orchestrator nodes → Claude (Anthropic API / Vertex AI) -Code execution: IMPLEMENT node → Podman container → Deep Agents +LLM calls: Orchestrator nodes → Claude/Gemini (Anthropic API / Vertex AI) +Code execution: implement_task → Podman container → Deep Agents Outbound actions: Jira (comments, labels, transitions) GitHub (PRs, branches, reviews) Observability: Langfuse (LLM traces, workflow spans, costs) From a38f15c4525679b5f2681bc9c5f75ec22d105963 Mon Sep 17 00:00:00 2001 From: Eran Kuris Date: Mon, 13 Jul 2026 21:13:15 +0300 Subject: [PATCH 03/17] =?UTF-8?q?docs:=20fix=20diagram=20alignment=20?= =?UTF-8?q?=E2=80=94=20all=20box=20lines=20at=20width=2075?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-Authored-By: Claude Opus 4.6 (1M context) --- docs/architecture.md | 311 ++++++++++++++++++++++--------------------- 1 file changed, 158 insertions(+), 153 deletions(-) diff --git a/docs/architecture.md b/docs/architecture.md index 149f94b4..a6469fc2 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -6,195 +6,200 @@ ┌─────────────────────────────────────────────────────────────────────────┐ │ External Systems │ │ │ -│ ┌──────────┐ ┌──────────┐ ┌───────────┐ ┌────────────────┐ │ -│ │ Jira │ │ GitHub │ │ Deep │ │ Langfuse │ │ -│ │ │ │ │ │ Agents │ │ (Observability)│ │ -│ └────┬─────┘ └────┬─────┘ └───────────┘ └────────────────┘ │ -│ │ │ │ -└────────┼───────────────┼────────────────────────────────────────────────┘ - │ webhooks │ webhooks - ▼ ▼ +│ ┌──────────┐ ┌──────────┐ ┌─────────────┐ ┌──────────────────┐ │ +│ │ Jira │ │ GitHub │ │ Deep Agents │ │ Langfuse │ │ +│ │ │ │ │ │ │ │ (Observability) │ │ +│ └────┬─────┘ └────┬─────┘ └─────────────┘ └──────────────────┘ │ +│ │ │ │ +└────────┼──────────────┼─────────────────────────────────────────────────┘ + │ webhooks │ webhooks + v v ┌─────────────────────────────────────────────────────────────────────────┐ -│ FastAPI Server (:8000) │ +│ FastAPI Server (:8000) │ │ │ -│ POST /api/v1/webhooks/jira POST /api/v1/webhooks/github │ -│ GET /api/v1/health GET /metrics │ +│ POST /api/v1/webhooks/jira POST /api/v1/webhooks/github │ +│ GET /api/v1/health GET /metrics │ │ │ └────────────────────────────┬────────────────────────────────────────────┘ │ enqueue events - ▼ + v ┌─────────────────────────────────────────────────────────────────────────┐ -│ Redis │ +│ Redis │ │ │ -│ Streams: forge:events:jira State: AsyncRedisSaver │ -│ forge:events:github LangGraph checkpointing │ -│ Consumer group: forge-workers Per-ticket thread IDs (AISOS-123) │ +│ Streams: forge:events:jira State: AsyncRedisSaver │ +│ forge:events:github LangGraph checkpointing │ +│ Consumer group: forge-workers Per-ticket thread IDs (AISOS-123) │ │ │ └────────────────────────────┬────────────────────────────────────────────┘ │ dequeue + dispatch - ▼ + v ┌─────────────────────────────────────────────────────────────────────────┐ -│ WorkflowRouter │ +│ WorkflowRouter │ │ │ -│ Selects graph based on Jira issue type: │ -│ Feature/Story → FeatureWorkflow (StateGraph) │ -│ Bug → BugWorkflow (StateGraph) │ +│ Routes by Jira issue type: │ +│ Feature/Story --> FeatureWorkflow (StateGraph) │ +│ Bug --> BugWorkflow (StateGraph) │ │ │ -│ ┌──────────────────────────────┐ ┌────────────────────────────────┐ │ -│ │ Feature Workflow │ │ Bug Workflow │ │ -│ │ │ │ │ │ -│ │ generate_prd │ │ triage_check │ │ -│ │ → ⏸ prd_approval_gate │ │ → ⏸ triage_gate │ │ -│ │ generate_spec │ │ analyze_bug ◄─┐ │ │ -│ │ → ⏸ spec_approval_gate │ │ → reflect_rca ─┘ (≤3x) │ │ -│ │ decompose_epics │ │ → ⏸ rca_option_gate │ │ -│ │ → ⏸ plan_approval_gate │ │ plan_bug_fix │ │ -│ │ generate_tasks │ │ → ⏸ plan_approval_gate │ │ -│ │ → ⏸ task_approval_gate │ │ decompose_plan │ │ -│ │ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ │ │ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ │ │ -│ │ task_router (per-repo) │ │ Shared implementation path: │ │ -│ │ setup_workspace │ │ (same nodes as Feature) │ │ -│ │ implement_task │ │ │ │ -│ │ local_review │ │ Post-merge: │ │ -│ │ update_documentation │ │ post_merge_summary → END │ │ -│ │ create_pr │ │ │ │ -│ │ teardown_workspace │ └────────────────────────────────┘ │ -│ │ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ │ │ -│ │ wait_for_ci_gate │ │ -│ │ ci_evaluator ◄── ci_fix │ │ -│ │ ⏸ human_review_gate │ │ -│ │ complete_tasks │ │ -│ │ aggregate_epic_status │ │ -│ │ aggregate_feature_status │ │ -│ │ → END │ │ -│ └──────────────────────────────┘ │ +│ ┌──────────────────────────────┐ ┌──────────────────────────────┐ │ +│ │ Feature Workflow │ │ Bug Workflow │ │ +│ │ │ │ │ │ +│ │ generate_prd │ │ triage_check │ │ +│ │ >> prd_approval_gate │ │ >> triage_gate │ │ +│ │ generate_spec │ │ analyze_bug │ │ +│ │ >> spec_approval_gate │ │ <> reflect_rca (up to 3x) │ │ +│ │ decompose_epics │ │ >> rca_option_gate │ │ +│ │ >> plan_approval_gate │ │ plan_bug_fix │ │ +│ │ generate_tasks │ │ >> plan_approval_gate │ │ +│ │ >> task_approval_gate │ │ decompose_plan │ │ +│ │──────────────────────────────│ │──────────────────────────────│ │ +│ │ task_router (per-repo) │ │ Shared impl path: │ │ +│ │ setup_workspace │ │ (same nodes as Feature) │ │ +│ │ implement_task │ │ │ │ +│ │ local_review │ │ Post-merge: │ │ +│ │ update_documentation │ │ post_merge_summary --> END│ │ +│ │ create_pr │ │ │ │ +│ │ teardown_workspace │ └──────────────────────────────┘ │ +│ │──────────────────────────────│ │ +│ │ wait_for_ci_gate │ │ +│ │ ci_evaluator <-- ci_fix │ │ +│ │ >> human_review_gate │ │ +│ │ complete_tasks │ │ +│ │ aggregate_epic_status │ │ +│ │ aggregate_feature_status │ │ +│ │ --> END │ │ +│ └──────────────────────────────┘ │ │ │ -│ ⏸ = human checkpoint (auto-approved with forge:yolo label) │ +│ >> = human checkpoint (auto-approved with forge:yolo label) │ │ │ └───────────────┬─────────────────────────────┬───────────────────────────┘ │ │ - ▼ ▼ -┌──────────────────────────┐ ┌──────────────────────────────────────────┐ -│ LLM Backends │ │ Podman Container (ephemeral) │ -│ │ │ │ -│ Anthropic API (Claude) │ │ ┌─────────────────────────────────┐ │ -│ Vertex AI (Claude) │ │ │ Deep Agents + MCP tools │ │ -│ Vertex AI (Gemini) │ │ │ /task.json (task description) │ │ -│ │ │ │ /workspace (repo mounted) │ │ -│ Used by: PRD, spec, │ │ │ commits changes locally │ │ -│ plan, RCA, triage, │ │ └─────────────────────────────────┘ │ -│ code review, CI eval │ │ │ -└──────────────────────────┘ └──────────────────────────────────────────┘ + v v +┌────────────────────────────┐ ┌────────────────────────────────────────┐ +│ LLM Backends │ │ Podman Container (ephemeral) │ +│ │ │ │ +│ Anthropic API (Claude) │ │ ┌──────────────────────────────────┐ │ +│ Vertex AI (Claude) │ │ │ Deep Agents + MCP tools │ │ +│ Vertex AI (Gemini) │ │ │ /task.json (task description) │ │ +│ │ │ │ /workspace (repo mounted) │ │ +│ Used by: PRD, spec, │ │ │ commits changes locally │ │ +│ plan, RCA, triage, │ │ └──────────────────────────────────┘ │ +│ code review, CI eval │ │ │ +└────────────────────────────┘ └────────────────────────────────────────┘ ``` ## Feature Ticket Lifecycle ``` Jira ticket created + labeled "forge:managed" (type: Feature or Story) - │ - ▼ -generate_prd ── AI drafts Product Requirements Document - │ - ▼ -⏸ prd_approval_gate ── wait for approval (Jira comment or proposals PR) - │ - ▼ -generate_spec ── AI drafts Technical Specification - │ - ▼ -⏸ spec_approval_gate - │ - ▼ -decompose_epics ── AI breaks spec into Jira Epics - │ - ▼ -⏸ plan_approval_gate - │ - ▼ -generate_tasks ── AI creates implementation Tasks under Epics - │ - ▼ -⏸ task_approval_gate - │ - ▼ -task_router ── group tasks by target repo - │ - ▼ (per repo) -setup_workspace ── clone repo, create branch - │ - ▼ -implement_task ── Podman container + Deep Agents writes code - │ - ▼ -local_review ── AI reviews diff, fixes issues (≤2 passes) - │ - ▼ -update_documentation ── update stale docs - │ - ▼ -create_pr ── open GitHub PR - │ - ▼ -teardown_workspace ── cleanup (loop back if more repos) - │ - ▼ -wait_for_ci_gate ── wait for CI checks - │ - ▼ -ci_evaluator ─┬─ pass ──────────────────────────┐ - └─ fail → attempt_ci_fix (≤5x) ───┘ - │ - ▼ -⏸ human_review_gate ── wait for PR approval - │ (changes_requested → implement_review → CI loop) - ▼ -complete_tasks → aggregate_epic_status → aggregate_feature_status → END + | + v +generate_prd -- AI drafts Product Requirements Document + | + v +>> prd_approval_gate -- wait for approval (Jira comment or proposals PR) + | + v +generate_spec -- AI drafts Technical Specification + | + v +>> spec_approval_gate + | + v +decompose_epics -- AI breaks spec into Jira Epics + | + v +>> plan_approval_gate + | + v +generate_tasks -- AI creates implementation Tasks under Epics + | + v +>> task_approval_gate + | + v +task_router -- group tasks by target repo + | + v (per repo) +setup_workspace -- clone repo, create branch + | + v +implement_task -- Podman container + Deep Agents writes code + | + v +local_review -- AI reviews diff, fixes issues (up to 2 passes) + | + v +update_documentation -- update stale docs + | + v +create_pr -- open GitHub PR + | + v +teardown_workspace -- cleanup (loop back if more repos) + | + v +wait_for_ci_gate -- wait for CI checks + | + v +ci_evaluator --+-- pass ------+ + +-- fail ---+ | + | | + attempt_ci_fix | + (up to 5x) | + | | + +-----------+ | + | | + v v +>> human_review_gate -- wait for PR approval + | (changes_requested --> implement_review --> CI loop) + v +complete_tasks --> aggregate_epic_status --> aggregate_feature_status --> END -⏸ = human checkpoint (auto-approved when forge:yolo label is set) +>> = human checkpoint (auto-approved when forge:yolo label is set) ``` ## Bug Ticket Lifecycle ``` Jira ticket created + labeled "forge:managed" (type: Bug) - │ - ▼ -triage_check ── validate bug report has required fields - │ - ▼ -⏸ triage_gate ── wait for reporter update if incomplete - │ - ▼ -analyze_bug ◄──┐ - │ │ reflect_rca (up to 3 reflection cycles) - └────────────┘ - │ - ▼ -⏸ rca_option_gate ── present fix options, user selects ">option N" - │ - ▼ -plan_bug_fix ── AI generates fix plan - │ - ▼ -⏸ plan_approval_gate - │ - ▼ -decompose_plan ── break into tasks - │ - ▼ -setup_workspace → implement → local_review → create_pr → CI → review - │ - ▼ -post_merge_summary → END + | + v +triage_check -- validate bug report has required fields + | + v +>> triage_gate -- wait for reporter update if incomplete + | + v +analyze_bug <--+ + | | + +--- reflect_rca (up to 3 reflection cycles) + | + v +>> rca_option_gate -- present fix options, user selects ">option N" + | + v +plan_bug_fix -- AI generates fix plan + | + v +>> plan_approval_gate + | + v +decompose_plan -- break into tasks + | + v +setup_workspace --> implement --> local_review --> create_pr --> CI --> review + | + v +post_merge_summary --> END ``` ## Data Flow Summary ``` -Inbound events: Jira/GitHub webhooks → FastAPI → Redis Streams +Inbound events: Jira/GitHub webhooks --> FastAPI --> Redis Streams State persistence: Redis (LangGraph AsyncRedisSaver, keyed by ticket) -LLM calls: Orchestrator nodes → Claude/Gemini (Anthropic API / Vertex AI) -Code execution: implement_task → Podman container → Deep Agents +LLM calls: Orchestrator nodes --> Claude/Gemini (Anthropic / Vertex AI) +Code execution: implement_task --> Podman container --> Deep Agents Outbound actions: Jira (comments, labels, transitions) GitHub (PRs, branches, reviews) Observability: Langfuse (LLM traces, workflow spans, costs) From 9ce53bf2fea639ac30807f7fc698efc284d53e07 Mon Sep 17 00:00:00 2001 From: Eran Kuris Date: Tue, 14 Jul 2026 11:33:57 +0300 Subject: [PATCH 04/17] docs: address review feedback on architecture diagram - Remove Deep Agents from External Systems (it's a library, not a service) - Show LLM Backends connected to both orchestrator and container - Clarify Deep Agents runs inside Podman containers - Add bidirectional LLM call arrows Co-Authored-By: Claude Opus 4.6 (1M context) --- docs/architecture.md | 26 ++++++++++++++------------ 1 file changed, 14 insertions(+), 12 deletions(-) diff --git a/docs/architecture.md b/docs/architecture.md index a6469fc2..4cb46a32 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -6,10 +6,9 @@ ┌─────────────────────────────────────────────────────────────────────────┐ │ External Systems │ │ │ -│ ┌──────────┐ ┌──────────┐ ┌─────────────┐ ┌──────────────────┐ │ -│ │ Jira │ │ GitHub │ │ Deep Agents │ │ Langfuse │ │ -│ │ │ │ │ │ │ │ (Observability) │ │ -│ └────┬─────┘ └────┬─────┘ └─────────────┘ └──────────────────┘ │ +│ ┌──────────┐ ┌──────────┐ ┌──────────────────────────────────┐ │ +│ │ Jira │ │ GitHub │ │ Langfuse (Observability) │ │ +│ └────┬─────┘ └────┬─────┘ └──────────────────────────────────┘ │ │ │ │ │ └────────┼──────────────┼─────────────────────────────────────────────────┘ │ webhooks │ webhooks @@ -71,19 +70,21 @@ │ │ │ >> = human checkpoint (auto-approved with forge:yolo label) │ │ │ -└───────────────┬─────────────────────────────┬───────────────────────────┘ - │ │ - v v +└────────────────────────────┬────────────────────────────────────────────┘ + │ + ┌──────────────┼──────────────┐ + │ LLM calls │ │ LLM calls + v v v ┌────────────────────────────┐ ┌────────────────────────────────────────┐ │ LLM Backends │ │ Podman Container (ephemeral) │ │ │ │ │ │ Anthropic API (Claude) │ │ ┌──────────────────────────────────┐ │ -│ Vertex AI (Claude) │ │ │ Deep Agents + MCP tools │ │ +│ Vertex AI (Claude) │ │ │ Deep Agents (library) + MCP │ │ │ Vertex AI (Gemini) │ │ │ /task.json (task description) │ │ │ │ │ │ /workspace (repo mounted) │ │ -│ Used by: PRD, spec, │ │ │ commits changes locally │ │ -│ plan, RCA, triage, │ │ └──────────────────────────────────┘ │ -│ code review, CI eval │ │ │ +│ Called by orchestrator │ │ │ commits changes locally │ │ +│ nodes and container │ │ └──────────────────────────────────┘ │ +│ agents (bidirectional) │ │ │ └────────────────────────────┘ └────────────────────────────────────────┘ ``` @@ -199,7 +200,8 @@ post_merge_summary --> END Inbound events: Jira/GitHub webhooks --> FastAPI --> Redis Streams State persistence: Redis (LangGraph AsyncRedisSaver, keyed by ticket) LLM calls: Orchestrator nodes --> Claude/Gemini (Anthropic / Vertex AI) -Code execution: implement_task --> Podman container --> Deep Agents + Container agents --> Claude/Gemini (same backends) +Code execution: implement_task --> Podman container --> Deep Agents (library) Outbound actions: Jira (comments, labels, transitions) GitHub (PRs, branches, reviews) Observability: Langfuse (LLM traces, workflow spans, costs) From 537c3d91de821720c25fdf1795c784bc2c6fdced Mon Sep 17 00:00:00 2001 From: Eran Kuris Date: Tue, 14 Jul 2026 12:06:26 +0300 Subject: [PATCH 05/17] cosmetic changes Co-authored-by: Lukas Svaty --- docs/architecture.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/docs/architecture.md b/docs/architecture.md index 4cb46a32..8f6c9469 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -71,17 +71,17 @@ │ >> = human checkpoint (auto-approved with forge:yolo label) │ │ │ └────────────────────────────┬────────────────────────────────────────────┘ - │ + v ┌──────────────┼──────────────┐ - │ LLM calls │ │ LLM calls - v v v + │ LLM calls │ LLM calls + v v ┌────────────────────────────┐ ┌────────────────────────────────────────┐ │ LLM Backends │ │ Podman Container (ephemeral) │ │ │ │ │ │ Anthropic API (Claude) │ │ ┌──────────────────────────────────┐ │ │ Vertex AI (Claude) │ │ │ Deep Agents (library) + MCP │ │ │ Vertex AI (Gemini) │ │ │ /task.json (task description) │ │ -│ │ │ │ /workspace (repo mounted) │ │ +│ │<->│ │ /workspace (repo mounted) │ │ │ Called by orchestrator │ │ │ commits changes locally │ │ │ nodes and container │ │ └──────────────────────────────────┘ │ │ agents (bidirectional) │ │ │ From 2150db90a1fb897df9ca05bb7e76753fac41fb42 Mon Sep 17 00:00:00 2001 From: Eran Kuris Date: Thu, 16 Jul 2026 17:12:39 +0300 Subject: [PATCH 06/17] Fix architecture diagram per PR review feedback - Convert all diagrams from ASCII art to mermaid (eshulman2) - Move Deep Agents inside Podman Container, not External Systems (StLuke) - Add bidirectional LLM arrows to both Router and Container (StLuke) - Add TaskTakeoverWorkflow (Task/Epic) to system overview and lifecycle (danchild) - Expand Bug workflow implementation path with correct node names - Data Flow Summary as markdown list instead of code block Co-Authored-By: Claude Opus 4.6 (1M context) --- docs/architecture.md | 336 +++++++++++++++++++------------------------ 1 file changed, 145 insertions(+), 191 deletions(-) diff --git a/docs/architecture.md b/docs/architecture.md index 8f6c9469..c5c03059 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -2,207 +2,161 @@ ## System Overview -``` -┌─────────────────────────────────────────────────────────────────────────┐ -│ External Systems │ -│ │ -│ ┌──────────┐ ┌──────────┐ ┌──────────────────────────────────┐ │ -│ │ Jira │ │ GitHub │ │ Langfuse (Observability) │ │ -│ └────┬─────┘ └────┬─────┘ └──────────────────────────────────┘ │ -│ │ │ │ -└────────┼──────────────┼─────────────────────────────────────────────────┘ - │ webhooks │ webhooks - v v -┌─────────────────────────────────────────────────────────────────────────┐ -│ FastAPI Server (:8000) │ -│ │ -│ POST /api/v1/webhooks/jira POST /api/v1/webhooks/github │ -│ GET /api/v1/health GET /metrics │ -│ │ -└────────────────────────────┬────────────────────────────────────────────┘ - │ enqueue events - v -┌─────────────────────────────────────────────────────────────────────────┐ -│ Redis │ -│ │ -│ Streams: forge:events:jira State: AsyncRedisSaver │ -│ forge:events:github LangGraph checkpointing │ -│ Consumer group: forge-workers Per-ticket thread IDs (AISOS-123) │ -│ │ -└────────────────────────────┬────────────────────────────────────────────┘ - │ dequeue + dispatch - v -┌─────────────────────────────────────────────────────────────────────────┐ -│ WorkflowRouter │ -│ │ -│ Routes by Jira issue type: │ -│ Feature/Story --> FeatureWorkflow (StateGraph) │ -│ Bug --> BugWorkflow (StateGraph) │ -│ │ -│ ┌──────────────────────────────┐ ┌──────────────────────────────┐ │ -│ │ Feature Workflow │ │ Bug Workflow │ │ -│ │ │ │ │ │ -│ │ generate_prd │ │ triage_check │ │ -│ │ >> prd_approval_gate │ │ >> triage_gate │ │ -│ │ generate_spec │ │ analyze_bug │ │ -│ │ >> spec_approval_gate │ │ <> reflect_rca (up to 3x) │ │ -│ │ decompose_epics │ │ >> rca_option_gate │ │ -│ │ >> plan_approval_gate │ │ plan_bug_fix │ │ -│ │ generate_tasks │ │ >> plan_approval_gate │ │ -│ │ >> task_approval_gate │ │ decompose_plan │ │ -│ │──────────────────────────────│ │──────────────────────────────│ │ -│ │ task_router (per-repo) │ │ Shared impl path: │ │ -│ │ setup_workspace │ │ (same nodes as Feature) │ │ -│ │ implement_task │ │ │ │ -│ │ local_review │ │ Post-merge: │ │ -│ │ update_documentation │ │ post_merge_summary --> END│ │ -│ │ create_pr │ │ │ │ -│ │ teardown_workspace │ └──────────────────────────────┘ │ -│ │──────────────────────────────│ │ -│ │ wait_for_ci_gate │ │ -│ │ ci_evaluator <-- ci_fix │ │ -│ │ >> human_review_gate │ │ -│ │ complete_tasks │ │ -│ │ aggregate_epic_status │ │ -│ │ aggregate_feature_status │ │ -│ │ --> END │ │ -│ └──────────────────────────────┘ │ -│ │ -│ >> = human checkpoint (auto-approved with forge:yolo label) │ -│ │ -└────────────────────────────┬────────────────────────────────────────────┘ - v - ┌──────────────┼──────────────┐ - │ LLM calls │ LLM calls - v v -┌────────────────────────────┐ ┌────────────────────────────────────────┐ -│ LLM Backends │ │ Podman Container (ephemeral) │ -│ │ │ │ -│ Anthropic API (Claude) │ │ ┌──────────────────────────────────┐ │ -│ Vertex AI (Claude) │ │ │ Deep Agents (library) + MCP │ │ -│ Vertex AI (Gemini) │ │ │ /task.json (task description) │ │ -│ │<->│ │ /workspace (repo mounted) │ │ -│ Called by orchestrator │ │ │ commits changes locally │ │ -│ nodes and container │ │ └──────────────────────────────────┘ │ -│ agents (bidirectional) │ │ │ -└────────────────────────────┘ └────────────────────────────────────────┘ +```mermaid +flowchart TD + subgraph External["External Systems"] + Jira + GitHub + Langfuse["Langfuse (Observability)"] + end + + subgraph API["FastAPI Server (:8000)"] + JiraWH["POST /webhooks/jira"] + GitHubWH["POST /webhooks/github"] + end + + subgraph Queue["Redis"] + Streams["Streams: forge:events:jira\nforge:events:github"] + State["AsyncRedisSaver\nLangGraph checkpointing"] + end + + subgraph Router["WorkflowRouter"] + Route{"Route by\nissue type"} + Feature["FeatureWorkflow\n(Feature/Story)"] + Bug["BugWorkflow\n(Bug)"] + Task["TaskTakeoverWorkflow\n(Task/Epic)"] + end + + subgraph Container["Podman Container (ephemeral)"] + Agent["Deep Agents + MCP\n/workspace (repo mounted)"] + end + + LLM["LLM Backends\nAnthropic API (Claude)\nVertex AI (Claude/Gemini)"] + + Jira -- webhooks --> JiraWH + GitHub -- webhooks --> GitHubWH + JiraWH --> Streams + GitHubWH --> Streams + Streams --> Route + Route --> Feature + Route --> Bug + Route --> Task + Feature --> Container + Bug --> Container + Task --> Container + Router <--> LLM + Container <--> LLM + Router --> Jira + Router --> GitHub + Router --> Langfuse ``` ## Feature Ticket Lifecycle -``` -Jira ticket created + labeled "forge:managed" (type: Feature or Story) - | - v -generate_prd -- AI drafts Product Requirements Document - | - v ->> prd_approval_gate -- wait for approval (Jira comment or proposals PR) - | - v -generate_spec -- AI drafts Technical Specification - | - v ->> spec_approval_gate - | - v -decompose_epics -- AI breaks spec into Jira Epics - | - v ->> plan_approval_gate - | - v -generate_tasks -- AI creates implementation Tasks under Epics - | - v ->> task_approval_gate - | - v -task_router -- group tasks by target repo - | - v (per repo) -setup_workspace -- clone repo, create branch - | - v -implement_task -- Podman container + Deep Agents writes code - | - v -local_review -- AI reviews diff, fixes issues (up to 2 passes) - | - v -update_documentation -- update stale docs - | - v -create_pr -- open GitHub PR - | - v -teardown_workspace -- cleanup (loop back if more repos) - | - v -wait_for_ci_gate -- wait for CI checks - | - v -ci_evaluator --+-- pass ------+ - +-- fail ---+ | - | | - attempt_ci_fix | - (up to 5x) | - | | - +-----------+ | - | | - v v ->> human_review_gate -- wait for PR approval - | (changes_requested --> implement_review --> CI loop) - v -complete_tasks --> aggregate_epic_status --> aggregate_feature_status --> END - ->> = human checkpoint (auto-approved when forge:yolo label is set) +```mermaid +flowchart TD + A["Jira ticket created\nforge:managed (Feature/Story)"] --> B[generate_prd] + B --> C[">> prd_approval_gate"] + C --> D[generate_spec] + D --> E[">> spec_approval_gate"] + E --> F[decompose_epics] + F --> G[">> plan_approval_gate"] + G --> H[generate_tasks] + H --> I[">> task_approval_gate"] + I --> J["task_router (per-repo)"] + + J --> K[setup_workspace] + K --> L["implement_task\n(Podman + Deep Agents)"] + L --> M[local_review] + M -->|"needs work (up to 2x)"| L + M --> N[update_documentation] + N --> O[create_pr] + O --> P["teardown_workspace"] + P -->|more repos| K + + P --> Q[wait_for_ci_gate] + Q --> R{ci_evaluator} + R -->|fail| S["attempt_ci_fix\n(up to 5x)"] + S --> Q + R -->|pass| T[">> human_review_gate"] + T -->|changes_requested| U[implement_review] + U --> Q + T -->|approved| V["complete_tasks\naggregate_epic_status\naggregate_feature_status\nEND"] ``` +`>>` = human checkpoint (auto-approved when `forge:yolo` label is set) + ## Bug Ticket Lifecycle +```mermaid +flowchart TD + A["Jira ticket created\nforge:managed (Bug)"] --> B[triage_check] + B -->|missing fields| C[">> triage_gate"] + C --> B + B -->|sufficient| D[analyze_bug] + D --> E["reflect_rca\n(up to 3 cycles)"] + E --> D + D --> F[">> rca_option_gate\nuser selects >option N"] + F --> G[plan_bug_fix] + G --> H[">> plan_approval_gate"] + H --> I[decompose_plan] + + I --> J[setup_workspace] + J --> K["implement_bug_fix\n(Podman + Deep Agents)"] + K --> L[local_review] + L -->|"needs work"| K + L --> M[update_documentation] + M --> N[create_pr] + N --> O[teardown_workspace] + O -->|more repos| J + + O --> P[wait_for_ci_gate] + P --> Q{ci_evaluator} + Q -->|fail| R["attempt_ci_fix"] + R --> P + Q -->|pass| S[">> human_review_gate"] + S -->|changes_requested| T[implement_review] + T --> P + S -->|merged| U[post_merge_summary] + U --> V[END] ``` -Jira ticket created + labeled "forge:managed" (type: Bug) - | - v -triage_check -- validate bug report has required fields - | - v ->> triage_gate -- wait for reporter update if incomplete - | - v -analyze_bug <--+ - | | - +--- reflect_rca (up to 3 reflection cycles) - | - v ->> rca_option_gate -- present fix options, user selects ">option N" - | - v -plan_bug_fix -- AI generates fix plan - | - v ->> plan_approval_gate - | - v -decompose_plan -- break into tasks - | - v -setup_workspace --> implement --> local_review --> create_pr --> CI --> review - | - v -post_merge_summary --> END + +## Task Ticket Lifecycle + +```mermaid +flowchart TD + A["Jira ticket created\nforge:managed (Task/Epic)"] --> B[triage_check] + B -->|missing context| C[">> triage_gate\nforge:task-triage-pending"] + C --> B + B -->|sufficient| D[generate_plan] + D --> E[">> task_plan_approval_gate\nforge:plan-pending"] + E -->|"? question"| F[answer_question] + F --> E + E -->|"! feedback"| D + E -->|approved| G[setup_workspace] + + G --> H["execute_task_changes\n(Podman + Deep Agents)"] + H --> I[qualitative_review] + I -->|"needs work (up to 2x)"| H + I -->|adequate| J[create_pr] + J --> K[teardown_workspace] + K -->|more repos| G + + K --> L[wait_for_ci_gate] + L --> M{ci_evaluator} + M -->|fail| N["attempt_ci_fix"] + N --> L + M -->|pass| O[">> human_review_gate"] + O -->|changes_requested| P[implement_review] + P --> L + O -->|merged| Q[complete_task_takeover] ``` ## Data Flow Summary -``` -Inbound events: Jira/GitHub webhooks --> FastAPI --> Redis Streams -State persistence: Redis (LangGraph AsyncRedisSaver, keyed by ticket) -LLM calls: Orchestrator nodes --> Claude/Gemini (Anthropic / Vertex AI) - Container agents --> Claude/Gemini (same backends) -Code execution: implement_task --> Podman container --> Deep Agents (library) -Outbound actions: Jira (comments, labels, transitions) - GitHub (PRs, branches, reviews) -Observability: Langfuse (LLM traces, workflow spans, costs) -``` +- **Inbound events:** Jira/GitHub webhooks --> FastAPI --> Redis Streams +- **State persistence:** Redis (LangGraph AsyncRedisSaver, keyed by ticket) +- **LLM calls:** Orchestrator nodes and container agents --> Claude/Gemini (Anthropic / Vertex AI), bidirectional +- **Code execution:** implement_task --> Podman container --> Deep Agents (library) +- **Outbound actions:** Jira (comments, labels, transitions), GitHub (PRs, branches, reviews) +- **Observability:** Langfuse (LLM traces, workflow spans, costs) From d47e2e4b75efdb7efcd419dfd5eb2b9040d8685d Mon Sep 17 00:00:00 2001 From: Eran Kuris Date: Sun, 19 Jul 2026 11:58:36 +0300 Subject: [PATCH 07/17] Add verbal explanations to architecture doc for newcomers - Add project intro paragraph explaining what Forge does - Add component descriptions for each layer in System Overview - Add context paragraphs before each workflow lifecycle diagram - All descriptions verified against current codebase Co-Authored-By: Claude Opus 4.6 (1M context) --- docs/architecture.md | 23 +++++++++++++++++++++++ 1 file changed, 23 insertions(+) diff --git a/docs/architecture.md b/docs/architecture.md index c5c03059..f175bddd 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -1,7 +1,18 @@ # Forge Architecture +Forge is an AI-powered SDLC orchestrator. It listens for Jira and GitHub events, routes them through LangGraph workflows, and drives implementation end-to-end — from requirements through code generation, CI repair, and human review. This page describes the major components and how they connect. + ## System Overview +Forge is built from five layers that form a pipeline from external events to code changes. + +- **External Systems** — Jira (ticket lifecycle), GitHub (PRs, CI, code review), and Langfuse (observability and cost tracking). Forge receives webhooks from Jira and GitHub and writes back to both throughout the workflow. +- **FastAPI Server** — A lightweight API layer that validates incoming webhooks and enqueues them as events. It also exposes health and Prometheus metrics endpoints. +- **Redis** — Serves two roles: an event bus (Redis Streams with consumer groups for reliable delivery) and a state store (LangGraph's `AsyncRedisSaver` checkpoints workflow state per ticket so workflows survive restarts). +- **WorkflowRouter** — The orchestration core. Incoming events are dispatched to one of three LangGraph `StateGraph` workflows based on Jira issue type: **FeatureWorkflow** (Feature/Story), **BugWorkflow** (Bug), or **TaskTakeoverWorkflow** (Task/Epic). Each workflow is a graph of nodes connected by conditional edges, with human approval gates at key decision points. +- **Podman Container** — Implementation runs in ephemeral rootless containers. Each container mounts the target repository, receives a task description, and uses Deep Agents (an AI coding library) with MCP tool access to make changes and commit them locally. The orchestrator handles pushing and PR creation after the container exits. +- **LLM Backends** — Claude and Gemini models are called bidirectionally: orchestrator nodes call them for planning and review, and container agents call them for code generation. Forge supports Anthropic's direct API and Vertex AI, selected by configuration. + ```mermaid flowchart TD subgraph External["External Systems"] @@ -53,6 +64,10 @@ flowchart TD ## Feature Ticket Lifecycle +The Feature workflow handles the largest scope of work. It takes a Jira Feature or Story from a one-line description through a full planning pipeline — PRD, technical spec, epic decomposition, and task breakdown — before any code is written. Each planning stage produces an artifact posted to Jira (or as a GitHub PR in the proposals repo) and pauses at a human approval gate. Reviewers can approve, request revisions with a `!` comment, or ask questions with `?` without advancing the workflow. + +Once all planning is approved, Forge groups tasks by target repository and implements them in parallel. Each task runs in its own Podman container. After implementation, Forge reviews the diff, updates documentation, and opens a PR. If CI fails, Forge analyzes the failure and attempts automated fixes (up to 5 times). When CI passes, the workflow pauses for human PR review. Review feedback triggers another implementation-CI cycle. After merge, Forge aggregates status up through tasks, epics, and the parent feature. + ```mermaid flowchart TD A["Jira ticket created\nforge:managed (Feature/Story)"] --> B[generate_prd] @@ -88,6 +103,10 @@ flowchart TD ## Bug Ticket Lifecycle +The Bug workflow starts with triage — Forge checks whether the ticket has enough context to investigate. If information is missing, it pauses and asks the reporter to fill in the gaps. Once the report is sufficient, Forge performs root cause analysis with up to three reflection cycles to refine its understanding. It then presents numbered fix options and waits for the user to select one with a `>option N` comment. + +After the user selects a fix approach, Forge generates a fix plan, pauses for approval, and decomposes it into implementation tasks. From there the workflow shares the same implementation path as the Feature workflow: container execution, local review, PR creation, CI repair loop, and human review. After the PR is merged, Forge posts a summary of the fix back to the Jira ticket. + ```mermaid flowchart TD A["Jira ticket created\nforge:managed (Bug)"] --> B[triage_check] @@ -123,6 +142,10 @@ flowchart TD ## Task Ticket Lifecycle +The Task workflow is the shortest path from ticket to PR. It handles standalone Jira Tasks and Epics that are already scoped enough to implement directly — no PRD, spec, or epic decomposition needed. Forge triages the ticket for sufficient context, generates an implementation plan, and pauses for approval. At the approval gate, reviewers can ask questions (`?`), request revisions (`!`), or approve to proceed. + +After approval, implementation follows the same container-based execution as the other workflows: Forge sets up a workspace, runs the changes in a Podman container with Deep Agents, reviews the output for quality (up to 2 retries), and opens a PR. The CI repair loop and human review gate work identically to the Feature and Bug workflows. + ```mermaid flowchart TD A["Jira ticket created\nforge:managed (Task/Epic)"] --> B[triage_check] From 4384012aa14fbcb750c78c812cd40f8c0d454cc0 Mon Sep 17 00:00:00 2001 From: Eran Kuris Date: Sun, 19 Jul 2026 12:53:57 +0300 Subject: [PATCH 08/17] Clean up formatting: remove em dashes, use colons for component labels Co-Authored-By: Claude Opus 4.6 (1M context) --- docs/architecture.md | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/docs/architecture.md b/docs/architecture.md index f175bddd..f9e5b862 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -1,17 +1,17 @@ # Forge Architecture -Forge is an AI-powered SDLC orchestrator. It listens for Jira and GitHub events, routes them through LangGraph workflows, and drives implementation end-to-end — from requirements through code generation, CI repair, and human review. This page describes the major components and how they connect. +Forge is an AI-powered SDLC orchestrator. It listens for Jira and GitHub events, routes them through LangGraph workflows, and drives implementation end-to-end, from requirements through code generation, CI repair, and human review. This page describes the major components and how they connect. ## System Overview Forge is built from five layers that form a pipeline from external events to code changes. -- **External Systems** — Jira (ticket lifecycle), GitHub (PRs, CI, code review), and Langfuse (observability and cost tracking). Forge receives webhooks from Jira and GitHub and writes back to both throughout the workflow. -- **FastAPI Server** — A lightweight API layer that validates incoming webhooks and enqueues them as events. It also exposes health and Prometheus metrics endpoints. -- **Redis** — Serves two roles: an event bus (Redis Streams with consumer groups for reliable delivery) and a state store (LangGraph's `AsyncRedisSaver` checkpoints workflow state per ticket so workflows survive restarts). -- **WorkflowRouter** — The orchestration core. Incoming events are dispatched to one of three LangGraph `StateGraph` workflows based on Jira issue type: **FeatureWorkflow** (Feature/Story), **BugWorkflow** (Bug), or **TaskTakeoverWorkflow** (Task/Epic). Each workflow is a graph of nodes connected by conditional edges, with human approval gates at key decision points. -- **Podman Container** — Implementation runs in ephemeral rootless containers. Each container mounts the target repository, receives a task description, and uses Deep Agents (an AI coding library) with MCP tool access to make changes and commit them locally. The orchestrator handles pushing and PR creation after the container exits. -- **LLM Backends** — Claude and Gemini models are called bidirectionally: orchestrator nodes call them for planning and review, and container agents call them for code generation. Forge supports Anthropic's direct API and Vertex AI, selected by configuration. +- **External Systems:** Jira (ticket lifecycle), GitHub (PRs, CI, code review), and Langfuse (observability and cost tracking). Forge receives webhooks from Jira and GitHub and writes back to both throughout the workflow. +- **FastAPI Server:** A lightweight API layer that validates incoming webhooks and enqueues them as events. It also exposes health and Prometheus metrics endpoints. +- **Redis:** Serves two roles: an event bus (Redis Streams with consumer groups for reliable delivery) and a state store (LangGraph's `AsyncRedisSaver` checkpoints workflow state per ticket so workflows survive restarts). +- **WorkflowRouter:** The orchestration core. Incoming events are dispatched to one of three LangGraph `StateGraph` workflows based on Jira issue type: **FeatureWorkflow** (Feature/Story), **BugWorkflow** (Bug), or **TaskTakeoverWorkflow** (Task/Epic). Each workflow is a graph of nodes connected by conditional edges, with human approval gates at key decision points. +- **Podman Container:** Implementation runs in ephemeral rootless containers. Each container mounts the target repository, receives a task description, and uses Deep Agents (an AI coding library) with MCP tool access to make changes and commit them locally. The orchestrator handles pushing and PR creation after the container exits. +- **LLM Backends:** Claude and Gemini models are called bidirectionally. Orchestrator nodes call them for planning and review, and container agents call them for code generation. Forge supports Anthropic's direct API and Vertex AI, selected by configuration. ```mermaid flowchart TD @@ -64,7 +64,7 @@ flowchart TD ## Feature Ticket Lifecycle -The Feature workflow handles the largest scope of work. It takes a Jira Feature or Story from a one-line description through a full planning pipeline — PRD, technical spec, epic decomposition, and task breakdown — before any code is written. Each planning stage produces an artifact posted to Jira (or as a GitHub PR in the proposals repo) and pauses at a human approval gate. Reviewers can approve, request revisions with a `!` comment, or ask questions with `?` without advancing the workflow. +The Feature workflow handles the largest scope of work. It takes a Jira Feature or Story from a one-line description through a full planning pipeline (PRD, technical spec, epic decomposition, and task breakdown) before any code is written. Each planning stage produces an artifact posted to Jira (or as a GitHub PR in the proposals repo) and pauses at a human approval gate. Reviewers can approve, request revisions with a `!` comment, or ask questions with `?` without advancing the workflow. Once all planning is approved, Forge groups tasks by target repository and implements them in parallel. Each task runs in its own Podman container. After implementation, Forge reviews the diff, updates documentation, and opens a PR. If CI fails, Forge analyzes the failure and attempts automated fixes (up to 5 times). When CI passes, the workflow pauses for human PR review. Review feedback triggers another implementation-CI cycle. After merge, Forge aggregates status up through tasks, epics, and the parent feature. @@ -103,7 +103,7 @@ flowchart TD ## Bug Ticket Lifecycle -The Bug workflow starts with triage — Forge checks whether the ticket has enough context to investigate. If information is missing, it pauses and asks the reporter to fill in the gaps. Once the report is sufficient, Forge performs root cause analysis with up to three reflection cycles to refine its understanding. It then presents numbered fix options and waits for the user to select one with a `>option N` comment. +The Bug workflow starts with triage. Forge checks whether the ticket has enough context to investigate. If information is missing, it pauses and asks the reporter to fill in the gaps. Once the report is sufficient, Forge performs root cause analysis with up to three reflection cycles to refine its understanding. It then presents numbered fix options and waits for the user to select one with a `>option N` comment. After the user selects a fix approach, Forge generates a fix plan, pauses for approval, and decomposes it into implementation tasks. From there the workflow shares the same implementation path as the Feature workflow: container execution, local review, PR creation, CI repair loop, and human review. After the PR is merged, Forge posts a summary of the fix back to the Jira ticket. @@ -142,7 +142,7 @@ flowchart TD ## Task Ticket Lifecycle -The Task workflow is the shortest path from ticket to PR. It handles standalone Jira Tasks and Epics that are already scoped enough to implement directly — no PRD, spec, or epic decomposition needed. Forge triages the ticket for sufficient context, generates an implementation plan, and pauses for approval. At the approval gate, reviewers can ask questions (`?`), request revisions (`!`), or approve to proceed. +The Task workflow is the shortest path from ticket to PR. It handles standalone Jira Tasks and Epics that are already scoped enough to implement directly, without PRD, spec, or epic decomposition. Forge triages the ticket for sufficient context, generates an implementation plan, and pauses for approval. At the approval gate, reviewers can ask questions (`?`), request revisions (`!`), or approve to proceed. After approval, implementation follows the same container-based execution as the other workflows: Forge sets up a workspace, runs the changes in a Podman container with Deep Agents, reviews the output for quality (up to 2 retries), and opens a PR. The CI repair loop and human review gate work identically to the Feature and Bug workflows. From 59e6b5870a908e9eaef74e43f4a5ef8b0bc2e5af Mon Sep 17 00:00:00 2001 From: Eran Kuris Date: Sun, 19 Jul 2026 14:03:58 +0300 Subject: [PATCH 09/17] Remove inline code formatting from prose paragraphs Co-Authored-By: Claude Opus 4.6 (1M context) --- docs/architecture.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/docs/architecture.md b/docs/architecture.md index f9e5b862..7a670976 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -8,8 +8,8 @@ Forge is built from five layers that form a pipeline from external events to cod - **External Systems:** Jira (ticket lifecycle), GitHub (PRs, CI, code review), and Langfuse (observability and cost tracking). Forge receives webhooks from Jira and GitHub and writes back to both throughout the workflow. - **FastAPI Server:** A lightweight API layer that validates incoming webhooks and enqueues them as events. It also exposes health and Prometheus metrics endpoints. -- **Redis:** Serves two roles: an event bus (Redis Streams with consumer groups for reliable delivery) and a state store (LangGraph's `AsyncRedisSaver` checkpoints workflow state per ticket so workflows survive restarts). -- **WorkflowRouter:** The orchestration core. Incoming events are dispatched to one of three LangGraph `StateGraph` workflows based on Jira issue type: **FeatureWorkflow** (Feature/Story), **BugWorkflow** (Bug), or **TaskTakeoverWorkflow** (Task/Epic). Each workflow is a graph of nodes connected by conditional edges, with human approval gates at key decision points. +- **Redis:** Serves two roles: an event bus (Redis Streams with consumer groups for reliable delivery) and a state store (LangGraph's AsyncRedisSaver checkpoints workflow state per ticket so workflows survive restarts). +- **WorkflowRouter:** The orchestration core. Incoming events are dispatched to one of three LangGraph StateGraph workflows based on Jira issue type: **FeatureWorkflow** (Feature/Story), **BugWorkflow** (Bug), or **TaskTakeoverWorkflow** (Task/Epic). Each workflow is a graph of nodes connected by conditional edges, with human approval gates at key decision points. - **Podman Container:** Implementation runs in ephemeral rootless containers. Each container mounts the target repository, receives a task description, and uses Deep Agents (an AI coding library) with MCP tool access to make changes and commit them locally. The orchestrator handles pushing and PR creation after the container exits. - **LLM Backends:** Claude and Gemini models are called bidirectionally. Orchestrator nodes call them for planning and review, and container agents call them for code generation. Forge supports Anthropic's direct API and Vertex AI, selected by configuration. @@ -64,7 +64,7 @@ flowchart TD ## Feature Ticket Lifecycle -The Feature workflow handles the largest scope of work. It takes a Jira Feature or Story from a one-line description through a full planning pipeline (PRD, technical spec, epic decomposition, and task breakdown) before any code is written. Each planning stage produces an artifact posted to Jira (or as a GitHub PR in the proposals repo) and pauses at a human approval gate. Reviewers can approve, request revisions with a `!` comment, or ask questions with `?` without advancing the workflow. +The Feature workflow handles the largest scope of work. It takes a Jira Feature or Story from a one-line description through a full planning pipeline (PRD, technical spec, epic decomposition, and task breakdown) before any code is written. Each planning stage produces an artifact posted to Jira (or as a GitHub PR in the proposals repo) and pauses at a human approval gate. Reviewers can approve, request revisions with a "!" comment, or ask questions with "?" without advancing the workflow. Once all planning is approved, Forge groups tasks by target repository and implements them in parallel. Each task runs in its own Podman container. After implementation, Forge reviews the diff, updates documentation, and opens a PR. If CI fails, Forge analyzes the failure and attempts automated fixes (up to 5 times). When CI passes, the workflow pauses for human PR review. Review feedback triggers another implementation-CI cycle. After merge, Forge aggregates status up through tasks, epics, and the parent feature. @@ -99,11 +99,11 @@ flowchart TD T -->|approved| V["complete_tasks\naggregate_epic_status\naggregate_feature_status\nEND"] ``` -`>>` = human checkpoint (auto-approved when `forge:yolo` label is set) +">>" = human checkpoint (auto-approved when forge:yolo label is set) ## Bug Ticket Lifecycle -The Bug workflow starts with triage. Forge checks whether the ticket has enough context to investigate. If information is missing, it pauses and asks the reporter to fill in the gaps. Once the report is sufficient, Forge performs root cause analysis with up to three reflection cycles to refine its understanding. It then presents numbered fix options and waits for the user to select one with a `>option N` comment. +The Bug workflow starts with triage. Forge checks whether the ticket has enough context to investigate. If information is missing, it pauses and asks the reporter to fill in the gaps. Once the report is sufficient, Forge performs root cause analysis with up to three reflection cycles to refine its understanding. It then presents numbered fix options and waits for the user to select one with a ">option N" comment. After the user selects a fix approach, Forge generates a fix plan, pauses for approval, and decomposes it into implementation tasks. From there the workflow shares the same implementation path as the Feature workflow: container execution, local review, PR creation, CI repair loop, and human review. After the PR is merged, Forge posts a summary of the fix back to the Jira ticket. @@ -142,7 +142,7 @@ flowchart TD ## Task Ticket Lifecycle -The Task workflow is the shortest path from ticket to PR. It handles standalone Jira Tasks and Epics that are already scoped enough to implement directly, without PRD, spec, or epic decomposition. Forge triages the ticket for sufficient context, generates an implementation plan, and pauses for approval. At the approval gate, reviewers can ask questions (`?`), request revisions (`!`), or approve to proceed. +The Task workflow is the shortest path from ticket to PR. It handles standalone Jira Tasks and Epics that are already scoped enough to implement directly, without PRD, spec, or epic decomposition. Forge triages the ticket for sufficient context, generates an implementation plan, and pauses for approval. At the approval gate, reviewers can ask questions ("?"), request revisions ("!"), or approve to proceed. After approval, implementation follows the same container-based execution as the other workflows: Forge sets up a workspace, runs the changes in a Podman container with Deep Agents, reviews the output for quality (up to 2 retries), and opens a PR. The CI repair loop and human review gate work identically to the Feature and Bug workflows. From 481a927876863c87374d82a21dd14b201920d782 Mon Sep 17 00:00:00 2001 From: Eran Kuris Date: Sun, 19 Jul 2026 14:23:23 +0300 Subject: [PATCH 10/17] Fix layer count: five to six in System Overview Co-Authored-By: Claude Opus 4.6 (1M context) --- docs/architecture.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/architecture.md b/docs/architecture.md index 7a670976..b161cadf 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -4,7 +4,7 @@ Forge is an AI-powered SDLC orchestrator. It listens for Jira and GitHub events, ## System Overview -Forge is built from five layers that form a pipeline from external events to code changes. +Forge is built from six layers that form a pipeline from external events to code changes. - **External Systems:** Jira (ticket lifecycle), GitHub (PRs, CI, code review), and Langfuse (observability and cost tracking). Forge receives webhooks from Jira and GitHub and writes back to both throughout the workflow. - **FastAPI Server:** A lightweight API layer that validates incoming webhooks and enqueues them as events. It also exposes health and Prometheus metrics endpoints. From 8c3a528447689fe64e0cc8d50a527523dccbe9ac Mon Sep 17 00:00:00 2001 From: Eran Kuris Date: Sun, 19 Jul 2026 20:14:51 +0300 Subject: [PATCH 11/17] Address PR review feedback on architecture diagram 1. Feature: change human_review_gate edge label from 'approved' to 'merged' 2. Bug: fix RCA flow to show conditional decision at reflect_rca instead of fork 3. Task: rename qualitative_review to run_qualitative_review (actual node name) 4. Fix checkpoint legend from HTML-escaped text to inline code 5. System overview: rename to Gateway/Workers to show gateway->redis->workers flow Co-Authored-By: Claude Opus 4.6 (1M context) --- docs/architecture.md | 34 +++++++++++++++++----------------- 1 file changed, 17 insertions(+), 17 deletions(-) diff --git a/docs/architecture.md b/docs/architecture.md index b161cadf..f5919a31 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -21,7 +21,7 @@ flowchart TD Langfuse["Langfuse (Observability)"] end - subgraph API["FastAPI Server (:8000)"] + subgraph Gateway["FastAPI Gateway (:8000)"] JiraWH["POST /webhooks/jira"] GitHubWH["POST /webhooks/github"] end @@ -31,8 +31,8 @@ flowchart TD State["AsyncRedisSaver\nLangGraph checkpointing"] end - subgraph Router["WorkflowRouter"] - Route{"Route by\nissue type"} + subgraph Workers["Worker Processes (consumer group: forge-workers)"] + Router{"WorkflowRouter\nroute by issue type"} Feature["FeatureWorkflow\n(Feature/Story)"] Bug["BugWorkflow\n(Bug)"] Task["TaskTakeoverWorkflow\n(Task/Epic)"] @@ -48,18 +48,18 @@ flowchart TD GitHub -- webhooks --> GitHubWH JiraWH --> Streams GitHubWH --> Streams - Streams --> Route - Route --> Feature - Route --> Bug - Route --> Task + Streams --> Router + Router --> Feature + Router --> Bug + Router --> Task Feature --> Container Bug --> Container Task --> Container - Router <--> LLM + Workers <--> LLM Container <--> LLM - Router --> Jira - Router --> GitHub - Router --> Langfuse + Workers --> Jira + Workers --> GitHub + Workers --> Langfuse ``` ## Feature Ticket Lifecycle @@ -96,10 +96,10 @@ flowchart TD R -->|pass| T[">> human_review_gate"] T -->|changes_requested| U[implement_review] U --> Q - T -->|approved| V["complete_tasks\naggregate_epic_status\naggregate_feature_status\nEND"] + T -->|merged| V["complete_tasks\naggregate_epic_status\naggregate_feature_status\nEND"] ``` -">>" = human checkpoint (auto-approved when forge:yolo label is set) +`>>` = human checkpoint (auto-approved when forge:yolo label is set) ## Bug Ticket Lifecycle @@ -113,9 +113,9 @@ flowchart TD B -->|missing fields| C[">> triage_gate"] C --> B B -->|sufficient| D[analyze_bug] - D --> E["reflect_rca\n(up to 3 cycles)"] - E --> D - D --> F[">> rca_option_gate\nuser selects >option N"] + D --> E{reflect_rca} + E -->|"needs refinement (up to 3x)"| D + E -->|"analysis complete"| F[">> rca_option_gate\nuser selects >option N"] F --> G[plan_bug_fix] G --> H[">> plan_approval_gate"] H --> I[decompose_plan] @@ -159,7 +159,7 @@ flowchart TD E -->|approved| G[setup_workspace] G --> H["execute_task_changes\n(Podman + Deep Agents)"] - H --> I[qualitative_review] + H --> I[run_qualitative_review] I -->|"needs work (up to 2x)"| H I -->|adequate| J[create_pr] J --> K[teardown_workspace] From 2b77843783996b755c7a370dde3b94b3e15a59e2 Mon Sep 17 00:00:00 2001 From: Eran Kuris Date: Sun, 19 Jul 2026 20:33:22 +0300 Subject: [PATCH 12/17] Fix review findings: label mismatch, legend placement, docs nav - Rename 'FastAPI Server' to 'FastAPI Gateway' in prose to match diagram - Move >> legend to end of page (applies to all three workflow diagrams) - Add architecture.md to zensical.toml nav under Developer Guide Co-Authored-By: Claude Opus 4.6 (1M context) --- docs/architecture.md | 6 +++--- zensical.toml | 1 + 2 files changed, 4 insertions(+), 3 deletions(-) diff --git a/docs/architecture.md b/docs/architecture.md index f5919a31..142c29ef 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -7,7 +7,7 @@ Forge is an AI-powered SDLC orchestrator. It listens for Jira and GitHub events, Forge is built from six layers that form a pipeline from external events to code changes. - **External Systems:** Jira (ticket lifecycle), GitHub (PRs, CI, code review), and Langfuse (observability and cost tracking). Forge receives webhooks from Jira and GitHub and writes back to both throughout the workflow. -- **FastAPI Server:** A lightweight API layer that validates incoming webhooks and enqueues them as events. It also exposes health and Prometheus metrics endpoints. +- **FastAPI Gateway:** A lightweight API layer that validates incoming webhooks and enqueues them as events. It also exposes health and Prometheus metrics endpoints. - **Redis:** Serves two roles: an event bus (Redis Streams with consumer groups for reliable delivery) and a state store (LangGraph's AsyncRedisSaver checkpoints workflow state per ticket so workflows survive restarts). - **WorkflowRouter:** The orchestration core. Incoming events are dispatched to one of three LangGraph StateGraph workflows based on Jira issue type: **FeatureWorkflow** (Feature/Story), **BugWorkflow** (Bug), or **TaskTakeoverWorkflow** (Task/Epic). Each workflow is a graph of nodes connected by conditional edges, with human approval gates at key decision points. - **Podman Container:** Implementation runs in ephemeral rootless containers. Each container mounts the target repository, receives a task description, and uses Deep Agents (an AI coding library) with MCP tool access to make changes and commit them locally. The orchestrator handles pushing and PR creation after the container exits. @@ -99,8 +99,6 @@ flowchart TD T -->|merged| V["complete_tasks\naggregate_epic_status\naggregate_feature_status\nEND"] ``` -`>>` = human checkpoint (auto-approved when forge:yolo label is set) - ## Bug Ticket Lifecycle The Bug workflow starts with triage. Forge checks whether the ticket has enough context to investigate. If information is missing, it pauses and asks the reporter to fill in the gaps. Once the report is sufficient, Forge performs root cause analysis with up to three reflection cycles to refine its understanding. It then presents numbered fix options and waits for the user to select one with a ">option N" comment. @@ -175,6 +173,8 @@ flowchart TD O -->|merged| Q[complete_task_takeover] ``` +In all workflow diagrams above, `>>` marks a human checkpoint (auto-approved when the forge:yolo label is set). + ## Data Flow Summary - **Inbound events:** Jira/GitHub webhooks --> FastAPI --> Redis Streams diff --git a/zensical.toml b/zensical.toml index 692a4acb..40db45b0 100644 --- a/zensical.toml +++ b/zensical.toml @@ -18,6 +18,7 @@ nav = [ ]}, {"Developer Guide" = [ {"Overview" = "developer-guide.md"}, + {"Architecture" = "architecture.md"}, {"Local Setup" = "dev/setup.md"}, {"Testing" = "dev/testing.md"}, {"Contributing" = "dev/contributing.md"}, From b2777349a5fb0f2688ddc53a852e697291c05562 Mon Sep 17 00:00:00 2001 From: Eran Kuris Date: Sun, 19 Jul 2026 20:40:18 +0300 Subject: [PATCH 13/17] Fix two factual errors found in code review 1. Feature diagram: local_review retries itself (self-loop), not implement_task. The actual graph routes local_review -> local_review or -> update_documentation. 2. Webhook paths: add missing /api/v1 prefix to match actual routes. Co-Authored-By: Claude Opus 4.6 (1M context) --- docs/architecture.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/architecture.md b/docs/architecture.md index 142c29ef..38b655fa 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -22,8 +22,8 @@ flowchart TD end subgraph Gateway["FastAPI Gateway (:8000)"] - JiraWH["POST /webhooks/jira"] - GitHubWH["POST /webhooks/github"] + JiraWH["POST /api/v1/webhooks/jira"] + GitHubWH["POST /api/v1/webhooks/github"] end subgraph Queue["Redis"] @@ -83,7 +83,7 @@ flowchart TD J --> K[setup_workspace] K --> L["implement_task\n(Podman + Deep Agents)"] L --> M[local_review] - M -->|"needs work (up to 2x)"| L + M -->|"mechanical fixes (up to 2x)"| M M --> N[update_documentation] N --> O[create_pr] O --> P["teardown_workspace"] From 2596a07f43daff14112790ff6e685c82615f28c9 Mon Sep 17 00:00:00 2001 From: Eran Kuris Date: Mon, 20 Jul 2026 13:03:22 +0300 Subject: [PATCH 14/17] Add high-level pipeline diagram before detailed system overview Co-Authored-By: Claude Opus 4.6 (1M context) --- docs/architecture.md | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/docs/architecture.md b/docs/architecture.md index 38b655fa..035c8898 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -2,6 +2,21 @@ Forge is an AI-powered SDLC orchestrator. It listens for Jira and GitHub events, routes them through LangGraph workflows, and drives implementation end-to-end, from requirements through code generation, CI repair, and human review. This page describes the major components and how they connect. +## High-Level Pipeline + +At its core, Forge is a four-stage pipeline. External events flow in through an API gateway, queue in Redis, get processed by worker processes running LangGraph workflows, and produce code changes inside ephemeral containers. + +```mermaid +flowchart LR + A["Jira / GitHub\n(webhooks)"] --> B["FastAPI\nGateway"] + B --> C["Redis\n(Streams + State)"] + C --> D["Workers\n(LangGraph)"] + D --> E["Podman\nContainers"] + D <--> F["LLM\n(Claude / Gemini)"] + E <--> F + D --> A +``` + ## System Overview Forge is built from six layers that form a pipeline from external events to code changes. From 4aa367eee499b6f6b6e2b73290c257a02b0b216e Mon Sep 17 00:00:00 2001 From: Eran Kuris Date: Mon, 20 Jul 2026 13:17:46 +0300 Subject: [PATCH 15/17] Rewrite high-level architecture section with accurate component descriptions Co-Authored-By: Claude Opus 4.6 (1M context) --- docs/architecture.md | 12 +++++++++--- 1 file changed, 9 insertions(+), 3 deletions(-) diff --git a/docs/architecture.md b/docs/architecture.md index 035c8898..2b8bda99 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -2,13 +2,19 @@ Forge is an AI-powered SDLC orchestrator. It listens for Jira and GitHub events, routes them through LangGraph workflows, and drives implementation end-to-end, from requirements through code generation, CI repair, and human review. This page describes the major components and how they connect. -## High-Level Pipeline +## High-Level Architecture -At its core, Forge is a four-stage pipeline. External events flow in through an API gateway, queue in Redis, get processed by worker processes running LangGraph workflows, and produce code changes inside ephemeral containers. +Forge consists of three main services: the Gateway, the Worker, and Redis. + +- **Gateway (FastAPI)** accepts Jira and GitHub webhooks, validates signatures, deduplicates events, and publishes them to Redis Streams. It performs no workflow logic — its only job is ingestion. It also serves health, readiness, and Prometheus metrics endpoints. +- **Worker** consumes events from Redis Streams via consumer groups, resolves the target LangGraph workflow (Feature, Bug, or Task Takeover) based on ticket type, and drives execution through planning, implementation, CI repair, and human review stages. Implementation runs inside ephemeral Podman containers using Deep Agents. Multiple workers can run concurrently — Redis consumer groups distribute events across them, and each message is delivered to exactly one worker. +- **Redis** ties the system together. It serves as the event bus (Streams with consumer groups for reliable delivery), the workflow state store (LangGraph AsyncRedisSaver checkpoints per ticket for pause/resume and crash recovery), the retry queue (sorted set with exponential backoff), and supporting indexes (PR-to-ticket mapping, webhook deduplication). + +Because the Gateway and Workers communicate only through Redis, they can be deployed and scaled independently. ```mermaid flowchart LR - A["Jira / GitHub\n(webhooks)"] --> B["FastAPI\nGateway"] + A["Jira / GitHub\n(webhooks)"] --> B["Gateway\n(FastAPI)"] B --> C["Redis\n(Streams + State)"] C --> D["Workers\n(LangGraph)"] D --> E["Podman\nContainers"] From f26b5a15047ef7e0b5f63c5f74c34e9ab7a5cce7 Mon Sep 17 00:00:00 2001 From: Eran Kuris Date: Mon, 20 Jul 2026 14:24:38 +0300 Subject: [PATCH 16/17] Address review feedback: legend placement, happy-path note, configurable CI limit - Move >> legend before the first diagram so readers see it before hitting >> prd_approval_gate - Add note that diagrams show the primary flow only (error-handling, revision loops omitted) - Change CI fix limit from "up to 5 times" to configurable with default - Task diagram already includes implement_review (added in prior commit) Co-Authored-By: Claude Opus 4.6 (1M context) --- docs/architecture.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/docs/architecture.md b/docs/architecture.md index 2b8bda99..e33cc090 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -12,6 +12,8 @@ Forge consists of three main services: the Gateway, the Worker, and Redis. Because the Gateway and Workers communicate only through Redis, they can be deployed and scaled independently. +In the workflow diagrams below, `>>` marks a human checkpoint where the workflow pauses for approval, revision, or questions. These gates are auto-approved when the `forge:yolo` label is set. Diagrams show the primary flow only — error-handling, revision loops, and question-answering nodes are omitted for clarity. + ```mermaid flowchart LR A["Jira / GitHub\n(webhooks)"] --> B["Gateway\n(FastAPI)"] @@ -87,7 +89,7 @@ flowchart TD The Feature workflow handles the largest scope of work. It takes a Jira Feature or Story from a one-line description through a full planning pipeline (PRD, technical spec, epic decomposition, and task breakdown) before any code is written. Each planning stage produces an artifact posted to Jira (or as a GitHub PR in the proposals repo) and pauses at a human approval gate. Reviewers can approve, request revisions with a "!" comment, or ask questions with "?" without advancing the workflow. -Once all planning is approved, Forge groups tasks by target repository and implements them in parallel. Each task runs in its own Podman container. After implementation, Forge reviews the diff, updates documentation, and opens a PR. If CI fails, Forge analyzes the failure and attempts automated fixes (up to 5 times). When CI passes, the workflow pauses for human PR review. Review feedback triggers another implementation-CI cycle. After merge, Forge aggregates status up through tasks, epics, and the parent feature. +Once all planning is approved, Forge groups tasks by target repository and implements them in parallel. Each task runs in its own Podman container. After implementation, Forge reviews the diff, updates documentation, and opens a PR. If CI fails, Forge analyzes the failure and attempts automated fixes (up to 5 times by default, configurable via `CI_FIX_MAX_RETRIES`). When CI passes, the workflow pauses for human PR review. Review feedback triggers another implementation-CI cycle. After merge, Forge aggregates status up through tasks, epics, and the parent feature. ```mermaid flowchart TD @@ -112,7 +114,7 @@ flowchart TD P --> Q[wait_for_ci_gate] Q --> R{ci_evaluator} - R -->|fail| S["attempt_ci_fix\n(up to 5x)"] + R -->|fail| S["attempt_ci_fix\n(default 5x)"] S --> Q R -->|pass| T[">> human_review_gate"] T -->|changes_requested| U[implement_review] @@ -194,8 +196,6 @@ flowchart TD O -->|merged| Q[complete_task_takeover] ``` -In all workflow diagrams above, `>>` marks a human checkpoint (auto-approved when the forge:yolo label is set). - ## Data Flow Summary - **Inbound events:** Jira/GitHub webhooks --> FastAPI --> Redis Streams From 8f44f519b66cbaa3ad316753732e5e765a0cd980 Mon Sep 17 00:00:00 2001 From: Eran Kuris Date: Mon, 20 Jul 2026 14:46:30 +0300 Subject: [PATCH 17/17] Cosmetic fixes Cosmetic fixes --- docs/architecture.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/docs/architecture.md b/docs/architecture.md index e33cc090..7f6a6999 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -6,13 +6,13 @@ Forge is an AI-powered SDLC orchestrator. It listens for Jira and GitHub events, Forge consists of three main services: the Gateway, the Worker, and Redis. -- **Gateway (FastAPI)** accepts Jira and GitHub webhooks, validates signatures, deduplicates events, and publishes them to Redis Streams. It performs no workflow logic — its only job is ingestion. It also serves health, readiness, and Prometheus metrics endpoints. -- **Worker** consumes events from Redis Streams via consumer groups, resolves the target LangGraph workflow (Feature, Bug, or Task Takeover) based on ticket type, and drives execution through planning, implementation, CI repair, and human review stages. Implementation runs inside ephemeral Podman containers using Deep Agents. Multiple workers can run concurrently — Redis consumer groups distribute events across them, and each message is delivered to exactly one worker. +- **Gateway (FastAPI)** accepts Jira and GitHub webhooks, validates signatures, deduplicates events, and publishes them to Redis Streams. It performs no workflow logic; its only job is ingestion. It also serves health, readiness, and Prometheus metrics endpoints. +- **Worker** consumes events from Redis Streams via consumer groups, resolves the target LangGraph workflow (Feature, Bug, or Task Takeover) based on ticket type, and drives execution through planning, implementation, CI repair, and human review stages. Implementation runs inside ephemeral Podman containers using Deep Agents. Multiple workers can run concurrently, Redis consumer groups distribute events across them, and each message is delivered to exactly one worker. - **Redis** ties the system together. It serves as the event bus (Streams with consumer groups for reliable delivery), the workflow state store (LangGraph AsyncRedisSaver checkpoints per ticket for pause/resume and crash recovery), the retry queue (sorted set with exponential backoff), and supporting indexes (PR-to-ticket mapping, webhook deduplication). Because the Gateway and Workers communicate only through Redis, they can be deployed and scaled independently. -In the workflow diagrams below, `>>` marks a human checkpoint where the workflow pauses for approval, revision, or questions. These gates are auto-approved when the `forge:yolo` label is set. Diagrams show the primary flow only — error-handling, revision loops, and question-answering nodes are omitted for clarity. +In the workflow diagrams below, `>>` marks a human checkpoint where the workflow pauses for approval, revision, or questions. These gates are auto-approved when the `forge:yolo` label is set. Diagrams show the primary flow only; error-handling, revision loops, and question-answering nodes are omitted for clarity. ```mermaid flowchart LR @@ -126,7 +126,7 @@ flowchart TD The Bug workflow starts with triage. Forge checks whether the ticket has enough context to investigate. If information is missing, it pauses and asks the reporter to fill in the gaps. Once the report is sufficient, Forge performs root cause analysis with up to three reflection cycles to refine its understanding. It then presents numbered fix options and waits for the user to select one with a ">option N" comment. -After the user selects a fix approach, Forge generates a fix plan, pauses for approval, and decomposes it into implementation tasks. From there the workflow shares the same implementation path as the Feature workflow: container execution, local review, PR creation, CI repair loop, and human review. After the PR is merged, Forge posts a summary of the fix back to the Jira ticket. +After the user selects a fix approach, Forge generates a fix plan, pauses for approval, and decomposes it into implementation tasks. From there, the workflow shares the same implementation path as the Feature workflow: container execution, local review, PR creation, CI repair loop, and human review. After the PR is merged, Forge posts a summary of the fix back to the Jira ticket. ```mermaid flowchart TD @@ -163,7 +163,7 @@ flowchart TD ## Task Ticket Lifecycle -The Task workflow is the shortest path from ticket to PR. It handles standalone Jira Tasks and Epics that are already scoped enough to implement directly, without PRD, spec, or epic decomposition. Forge triages the ticket for sufficient context, generates an implementation plan, and pauses for approval. At the approval gate, reviewers can ask questions ("?"), request revisions ("!"), or approve to proceed. +The Task workflow is the shortest path from ticket to PR. It handles standalone Jira Tasks and Epics that are already scoped enough to implement directly, without PRD, spec, or epic decomposition. Forge triages the ticket for sufficient context, generates an implementation plan, and pauses for approval. At the approval gate, reviewers can ask questions ("?"), Request revisions ("!"), or approve to proceed. After approval, implementation follows the same container-based execution as the other workflows: Forge sets up a workspace, runs the changes in a Podman container with Deep Agents, reviews the output for quality (up to 2 retries), and opens a PR. The CI repair loop and human review gate work identically to the Feature and Bug workflows.