Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
87 changes: 87 additions & 0 deletions .github/copilot-instructions.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,87 @@
# Copilot instructions for codeweb

Purpose
- Give future Copilot sessions concise, repo-specific guidance: how to build/test/lint, the big-picture architecture, and non-obvious conventions.

Quick commands
- Build (default features):
- cargo build
- Build with features:
- cargo build --features serve
- cargo build --features mcp
- cargo build --features full
- Tests:
- Run all tests: cargo test
- Run tests with feature: cargo test --features serve
- Run a single unit test by name: cargo test <test_name>
- Run a single integration test file: cargo test --test <integration_test_name>
- Lint & format checks (CI uses these):
- cargo fmt -- --check
- cargo clippy --features full -- -D warnings
- cargo fmt (to autoformat)

CI notes
- CI runs clippy and tests with --features full. Some long/host-bound tests are skipped in CI via -- --skip test_path_mapping_applied --skip test_serve_.
- Follow the project's Definition of Done: verify builds/tests/clippy/format under relevant feature combos (especially --features full when touching feature-gated code).

High-level architecture (big picture)
- Purpose: build a semantic directed graph linking Java methods, MyBatis mappers, SQL, and stored procedures/tables.
- Layers:
- CLI (src/main.rs) — clap commands (init, analyze, trace, export, serve, mcp, tui)
- Parser layer (src/parser/*) — ogsql-parser for SQL; tree-sitter-java for Java; ibatis XML loader for mappers; JSP preprocessing when jsp feature enabled
- Graph model (src/graph/*) — GraphStore, builder, resolver, traversal and declarative QuerySpec
- Export/import (src/export, src/import) — DOT/JSON/Mermaid and CGEF import/merge
- Runtime surfaces:
- TUI (feature: tui)
- HTTP server + Browser UI (feature: serve)
- MCP server (feature: mcp) — exposes MCP tools for LLM clients
- Incremental analysis: file fingerprinting (blake3) used to avoid re-parsing unchanged files; GraphStore serialized with bincode.
- Exports: DOT, JSON, Mermaid. Imports: CGEF JSON.

Key repo-specific conventions and mapping rules
- Feature flags matter: default = [cli, tui, jsp]. Use `--features full` when making changes that touch multiple gated areas.
- Java <-> Mapper <-> SQL mapping rules (important for trace tasks):
- Java interface FQN == mapper namespace
- Java method name == mapper statement id
- Calls like sqlSession.selectList("namespace.id") map to a MappedStatement
- JSP feature: jsp preprocesses JSP into synthetic Java and extracts SQL via ogsql-parser's Java extraction; JDBC escape `{call ...}` may require post-processing
- Node types & tags: procedures (proc/ proc*), functions (func/func*), table/table*, mapper, method, jsp/jspsql, etc. Many commands accept node-type filters (e.g., --type proc)
- Incremental test/dev workflow:
- Run targeted unit tests during development: cargo test <test_name>
- For integration tests use: cargo test --test <name>
- CI may skip long tests — be aware when reproducing CI locally (remove --skip args to run everything)
- Code quality gates:
- No any/anyhow in libraries; prefer thiserror for error types
- Use cargo fmt and cargo clippy with -D warnings before PR
- Git workflow: never push directly to main; branch naming: feature/<desc> or fix/<desc> (kebab-case); create PRs and ensure CI passes
- Commit trailer: include Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> when a Copilot-made commit is created (project policy enforced by tooling in this workspace)

Where to look for deeper context
- README.md, CONTRIBUTION.md, AGENTS.md, docs/DeveloperGuide.md, docs/getting-started.md, docs/serve-api-guide.md
- parser/ and graph/ folders for how relationships are extracted and represented

Existing AI-agent configs
- AGENTS.md exists and documents phased goals and conventions for automated agents (useful for MCP/tool integration). No CLAUDE.md or other assistant-specific config files were found.

MCP server integration snippet (from README)
- Claude Desktop example:
{
"mcpServers": {
"codeweb": {
"command": "/path/to/codeweb",
"args": ["mcp", "--project", "/path/to/your/project"]
}
}
}

Notes for Copilot sessions
- When reasoning about call chains prefer starting from parser outputs (parser/*) and GraphStore APIs rather than searching ad-hoc across the repo.
- When editing or adding feature-gated code, run cargo build/test/clippy with --features full locally before opening PR.
- Use exported node types and mapping rules to resolve cross-language edges (Java → mapper → SQL → proc)

Files used to compose this guidance
- README.md, CONTRIBUTION.md, AGENTS.md, Cargo.toml, .github/workflows/ci.yml

---

If you'd like, configure MCP servers (Claude/Desktop, Cursor, VS Code Copilot Chat) for this repo now — say which server(s) to add and Copilot will prepare suggested config snippets and a short setup checklist.
23 changes: 23 additions & 0 deletions .github/hooks/workmux-status/hooks.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
{
"version": 1,
"hooks": {
"userPromptSubmitted": [
{
"type": "command",
"bash": "workmux set-window-status working"
}
],
"postToolUse": [
{
"type": "command",
"bash": "workmux set-window-status working"
}
],
"agentStop": [
{
"type": "command",
"bash": "workmux set-window-status done"
}
]
}
}
119 changes: 119 additions & 0 deletions .sisyphus/plans/fix-analyze-stale-store-version.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,119 @@
# Fix: analyze 不校验 store 版本 → 升级二进制后 stale 缓存死循环

## 问题(来自真实用户场景)

用户升级 codeweb 二进制(STORE_VERSION 7→8, PR #148)后,在旧项目目录执行:

```
codeweb analyze → "Up to date. 136 files, 0 nodes, 0 edges." (不重建)
codeweb stats → error: unsupported cache version 7, expected 8 — run `codeweb analyze` to regenerate
```

死循环:错误信息让用户跑 analyze,但 analyze 因指纹未变而拒绝重建。

## 根因

`src/project/mod.rs:143-172` — `analyze()` 的 up-to-date 判定只比较**文件指纹 vs manifest 边车**(`load_manifest_only` → `compute_changes`),从不触碰 store 文件本身。而 STORE_VERSION 升级作废的是 store(`src/graph/store.rs:22` `STORE_VERSION: u32 = 8`),不是 manifest 边车(边无版本头,`FileRecord` 布局 v7↔v8 未变,旧边车反序列化成功)→ `changes.is_empty()` 为真 → 提前返回,v7 store 原样保留。

次生问题:up-to-date 提前返回时 `self.store` 为 `None`,报告里 `nodes/edges` 来自 `unwrap_or(0)`(mod.rs:158-167),`print_analyze_report`(main.rs:3758-3764)于是永远打印 `0 nodes, 0 edges` —— 与 store 实际内容无关,误导用户。

## 修复设计(两个 TDD 循环)

### 循环 1(核心):analyze 在 up-to-date 判定前校验 store 版本,不匹配 → 强制 full rebuild

**行为断言(Red 测试)**:`analyze_rebuilds_when_store_version_stale`(`src/project/mod.rs` `mod tests`,行 579 处已有测试模块,可在模块内直接构造 `Project { root, config, store: None }`,config 用 `ProjectConfig::load(toml_str)` 从字符串解析)

测试步骤:
1. `tempfile::TempDir` 建项目目录,写入一个 `.sql` 文件(内容任意,`parse_file` 对 tokenizer 失败才返回 Err,普通文本也会记录哈希 → 断言不依赖 SQL 解析成功)
2. `proj.analyze()` → 断言 `report.is_full_build == true`(建立基线,manifest 边车写入)
3. 用 v7 布局字节覆盖 `store.bincode`:`STORE_MAGIC + 7u32.to_le_bytes() + [0u8; 8]`(模板照抄既有测试 `load_bincode_rejects_previous_layout_version`, store.rs:2324-2343);manifest 边车不动
4. 再跑 `proj.analyze()` → 断言 `report.is_up_to_date == false` 且 `report.is_full_build == true`
5. 断言磁盘上 `store.bincode` 头部版本 == 当前 `STORE_VERSION`(自愈验证)

**最小实现(Green)**:

1. `src/graph/store.rs` 新增轻量头探测(不做全量反序列化):
```rust
/// Peek at the on-disk store's format version WITHOUT deserializing the payload.
/// Returns `None` for missing files and legacy headerless (pre-#110) stores;
/// callers treat `None` as stale.
pub fn peek_version(path: &Path) -> Option<u32> {
use std::io::Read;
let mut file = std::fs::File::open(path).ok()?;
let mut header = [0u8; 13];
file.read_exact(&mut header).ok()?;
if header[..9] != STORE_MAGIC {
return None;
}
Some(u32::from_le_bytes([header[9], header[10], header[11], header[12]]))
}
```

2. `src/project/mod.rs` `Project` 新增私有方法:
```rust
/// True when the on-disk store exists and its format version matches
/// STORE_VERSION. Mismatch / missing / legacy headerless → stale.
fn store_is_current(&self) -> bool {
let store_path = self.store_path();
if !store_path.exists() { return false; }
match self.config.store.format {
config::StoreFormat::Bincode => {
GraphStore::peek_version(&store_path).is_some_and(|v| v == STORE_VERSION)
}
config::StoreFormat::Json => {
// JSON 无 13 字节头;全量加载后查 version 字段(JSON 格式为非默认 opt-in,
// 且 load_json 自带版本门禁,mismatch 即 Err → false)
GraphStore::load_json(&store_path).map(|s| s.version == STORE_VERSION).unwrap_or(false)
}
}
}
```

3. `analyze()` 修改一行判定(mod.rs:148):
```rust
let is_up_to_date = changes.is_empty() && self.store_is_current();
```
STORE_VERSION 不变(保持 8);修复后 analyze 自愈:mismatch → 走既有 full build 路径 → `save_bincode` 覆写为 v8 + 重写边车。

**store.rs 单测(随循环 1 一起,锚定 peek 行为)**:
- `peek_version_returns_header_version`:写 `STORE_MAGIC + STORE_VERSION.to_le_bytes()` → `Some(STORE_VERSION)`
- `peek_version_none_for_legacy_headerless`:纯 bincode 字节(模板 store.rs:2346-2361)→ `None`
- `peek_version_none_for_missing_file`:不存在路径 → `None`

### 循环 2(次生):up-to-date 输出不再显示假 0/0 计数

**行为断言(Red 测试)**:把 main.rs:3760-3764 的行文案构造提为纯函数并测试:
```rust
fn format_up_to_date_line(report: &project::AnalyzeReport) -> String {
// store 未加载时(analyze 快路径)报告中的 nodes/edges 恒为 0,
// 打印出来是误导 —— 只报文件数。
format!("Up to date. {} files.", report.files_scanned)
}
```
测试:`up_to_date_line_reports_files_without_zero_counts` —— 构造 `AnalyzeReport { nodes: 0, edges: 0, files_scanned: 136, .. }`,断言输出为 `"Up to date. 136 files."` 且不含 `"0 nodes"`。
(注:核心 bug 修复后 up-to-date 快路径只在 store 版本匹配时可达,加载 store 换真计数会牺牲快路径性能;只报文件数是零成本且诚实的折中。`print_analyze_report` 改调该函数。)

## 明确不做(Out of scope)

- 不改 `STORE_VERSION`(保持 8,无需再 bump:修复不改变 v8 布局)
- 不动 `diff` 命令(只展示文件差异,不读 store)
- 不做 store 迁移/升级(full rebuild 即自愈,符合既有"拒绝+重建"设计,store.rs:1172-1179 错误信息语义不变)
- 不修 explore 报告提到的其他静默失败(loader 丢文件等,与本 bug 无关)

## 验证门禁(AGENTS.md 规定)

```bash
cargo test --features <default> <新测试名> # 循环内单测
cargo fmt --all -- --check
cargo clippy --features full -- -D warnings
cargo test --features full -- --skip test_path_mapping_applied --skip test_serve_
```

注:`test_serve_*` / `test_path_mapping_applied` 为 CI 既有环境跳过项,失败与本次无关。

## 风险与边界

- store 存在但边车缺失:`load_manifest_only` 返回空 → `is_full_build=true` → 本来就 full rebuild,行为不变
- v7 边车可被 v8 二进制正常反序列化(FileRecord 未变)→ 指纹判定仍为"无变化",由 store 版本校验兜底触发重建 —— 两道检查互补
- 性能:bincode 快路径只读 13 字节;Json 格式全量加载但属非默认 opt-in
- `is_some_and`:Rust 1.70+ 稳定 API,仓库无 rust-toolchain.toml、CI 用 stable,可用
32 changes: 32 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -262,6 +262,9 @@ Add to `claude_desktop_config.json`:

| Tool | Description |
|------|-------------|
| `codeweb_init` | Create `codeweb.toml` + `.codeweb/` in the served directory (no analysis) |
| `codeweb_analyze` | Build/refresh the graph and hot-swap it in (no restart needed) |
| `codeweb_diff` | List files changed since the last analysis |
| `codeweb_stats` | Project statistics (node/edge/file counts by type) |
| `codeweb_nodes` | List nodes with search, type filter, pagination |
| `codeweb_node_detail` | Node properties, callers, and callees by ID |
Expand All @@ -271,6 +274,20 @@ Add to `claude_desktop_config.json`:
| `codeweb_column_analysis` | Aggregate column-analysis for a procedure/package |
| `codeweb_lineage` | Table-level and column-level lineage analysis |

### Lifecycle (init / analyze / diff)

The server starts even when the directory has no `codeweb.toml` yet: queries then
return `status: "uninitialized"` instead of the process exiting.

1. `codeweb_init` — writes `codeweb.toml` + `.codeweb/` under the served directory.
It never analyzes on its own.
2. `codeweb_analyze` — builds or incrementally refreshes the graph and hot-swaps it
into the running server, so later queries see it without a restart.
3. `codeweb_diff` — lists files changed since the last analysis.

Reads may point at any directory (`analysis.paths`), but every write is confined to
the served directory: a tampered `store.path` that escapes it is rejected.

## Project Structure

```
Expand Down Expand Up @@ -607,6 +624,9 @@ codeweb mcp --project /path/to/your/project

| 工具 | 说明 |
|------|------|
| `codeweb_init` | 在服务目录创建 `codeweb.toml` + `.codeweb/`(不触发分析) |
| `codeweb_analyze` | 构建/刷新图谱并热替换到内存(无需重启) |
| `codeweb_diff` | 列出相对上次分析变更的文件 |
| `codeweb_stats` | 项目统计(各类型节点/边/文件数量) |
| `codeweb_nodes` | 节点列表(搜索、类型过滤、分页) |
| `codeweb_node_detail` | 节点详情:属性 + 上游调用方 + 下游被调用方 |
Expand All @@ -616,6 +636,18 @@ codeweb mcp --project /path/to/your/project
| `codeweb_column_analysis` | 按过程/包聚合列级分析结果 |
| `codeweb_lineage` | 表级与列级血缘分析 |

### 生命周期工具(init / analyze / diff)

即使目录里还没有 `codeweb.toml`,服务也会正常启动:查询返回
`status: "uninitialized"`,而不是进程直接退出。

1. `codeweb_init` —— 在服务目录写入 `codeweb.toml` + `.codeweb/`,**不会**自动分析。
2. `codeweb_analyze` —— 构建或增量刷新图谱,并热替换到运行中的服务,后续查询无需重启即可看到。
3. `codeweb_diff` —— 列出相对上次分析变更的文件。

读取可以指向任意目录(`analysis.paths`),但所有写入都被限制在服务目录内:
`store.path` 若被改成逃逸出该目录会被拒绝。

## 项目结构

```
Expand Down
12 changes: 12 additions & 0 deletions docs/DeveloperGuide.md
Original file line number Diff line number Diff line change
Expand Up @@ -354,6 +354,9 @@ codeweb 提供四种 MCP/外部集成方式:

| 工具 | 参数 | 说明 |
|------|------|------|
| `codeweb_init` | `name`, `paths` | 在服务目录创建 `codeweb.toml` + `.codeweb/`(不触发分析) |
| `codeweb_analyze` | 无 | 构建/刷新图谱并热替换内存快照(`spawn_blocking`) |
| `codeweb_diff` | 无 | 列出相对上次分析变更的文件 |
| `codeweb_stats` | 无 | 项目统计(各类节点/边/文件数量) |
| `codeweb_nodes` | `search`, `node_type`, `limit`, `offset` | 节点列表(搜索、类型过滤、分页) |
| `codeweb_node_detail` | `id` (usize) | 节点详情:属性 + callers + callees |
Expand Down Expand Up @@ -382,6 +385,15 @@ codeweb 提供四种 MCP/外部集成方式:
- `src/mcp/server.rs` — 服务入口(加载 GraphStore → 启动 tokio runtime → stdio 传输)
- 复用 `GraphStore` 的全部索引和查询能力,与 HTTP API 共享后端

**状态模型(issue #171):**

- `McpState` 持有 `Arc<Inner>`:`permitted_root`(唯一可写目录)、`Mutex<Option<Project>>`(生命周期工具)、`RwLock<GraphSnapshot>`(查询工具)。
- 查询工具读取 `Arc<GraphStore>` 快照后立即释放锁,长查询不会阻塞 `codeweb_analyze`。
- `codeweb_analyze` 在 `tokio::task::spawn_blocking` 中运行 `Project::analyze`(CPU 密集、同步),完成后把新 store 换入快照。
- 未初始化目录不再导致进程退出:查询返回 `status: "uninitialized"`,引导调用 `codeweb_init`。
- 写守卫 `confine_to_root` 做词法归一化后校验路径在 `permitted_root` 内;`store.path` 逃逸时 analyze 直接返回错误。
- stdout 只用于 JSON-RPC:进度条与报告一律走 stderr,且 MCP 不调用 CLI 的 `print_analyze_report`。

### 程序化 API 示例

```rust
Expand Down
Loading
Loading