feat: 列级分析查询面与造数入口 — columns/predicates/transform/跨表键 (#165-#169) - #170
Conversation
fix #169, fix #168) - HardFilter 增加 transform(FilterTransform,serde "fn");白名单 {substr,substring,nvl,trim,upper,lower},FunctionCall+SpecialFunction 双变体;六个比较操作符统一支持 - transform 序列化采用 is_human_readable() 分支手写实现(bincode 固定布局 / JSON 省略 None) - JoinConditionSource 新增 RecordField 变体;column_source 记录字段解析抽取为 record_field_source/resolve_record_field 复用;等值一侧为已解析记录字段时产出跨表 JoinCondition - 诊断类型补 Hash derive(方案A 并集合并前置) Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent) Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
- PredicateExtractor 走整个 PlBlock 保留分支结构(extract_body_sql 摊平前收集) - 置信度规则:记录字段/裸列→high;SELECT INTO 主表变量→medium;维表变量→low + param_table_hint;函数/动态 SQL→low 保留 origin,绝不静默 high - 条件转换复用 #169 column_transform_of 与 #168 记录字段解析;PredicateClause.transform 沿用 HardFilter 的 is_human_readable 序列化模式 - 过程内 SELECT INTO 变量源追踪;SELECT INTO 目标/变量数不匹配时 parse_log 告警;游标 WHERE HardFilter 不混入谓词列表 Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent) Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
- v10: merge_table_access_edges 诊断字段并集(#165 方案A)+ transform 预留(#169) - v11: procedure_predicates 侧表(HashMap<NodeKey, Vec<PlPredicate>>,serde default,merge 时并集) - v12: PredicateClause.transform(bincode 布局变更) - 版本拒绝基线测试不变;版本戳字面量测试同步至 12 Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent) Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
- merge_table_access_edges 对 join_conditions/hard_filters/enum_mappings/select_into/insert_columns/update_columns/column_refs 做保序 HashSet 并集,alias_map 首见优先;column_mappings/read_tables 语义不变(方案A,修复同过程多语句同表时诊断字段「保留第一条」丢失) - 过程构建期走整个 PlBlock 收集谓词,procedure_predicates 经 RoutineId::normalized() 以与 NodeKey::from_node 一致的小写键写入(修复大小写不匹配查找 miss) - project/mod.rs 接线 ctx.procedure_predicates → GraphStore Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent) Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
column_analysis_of_routine/column_analysis_of_package 扫入边+出边逐字段去重聚合,--table 过滤;AggregatedColumnAnalysis 字段与 ColumnAnalysis 1:1(schema_version 1),供 CLI/MCP/HTTP 三面共享,不另造 schema Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent) Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
- columns --procedure|--package [--table] --format json:旧 store(<v10) 软提示重跑 analyze;unknown 过程非零退出不静默
- predicates --procedure --format json:旧 store(<v12) 软提示;输出 {schema_version, procedure, predicates}
- regress_columns / regress_predicates 端到端回归(含大写过程名大小写无关解析、记录字段跨表键、param_table_hint)
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)
Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
…165) - MCP 新增 codeweb_column_analysis / codeweb_lineage(与 CLI JSON 字段 1:1,空图守卫与错误措辞对齐既有工具);工具清单测试 6→8 - HTTP 新增 GET /api/v1/columns、GET /api/v1/lineage(400 非法输入 / 404 未命中,同 serde 结构直出) - lineage 目标解析抽取为 graph::lineage::parse_lineage_target 纯函数(6 个单测),cmd_lineage 行为不变 Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent) Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
- README 中英 CLI/HTTP/MCP 三表补齐(示例与 --help 逐字一致) - user-guide §6 新增 lineage/columns/predicates 小节;DeveloperGuide 增加 ColumnAnalysis 字段表(含 transform/RecordField)与 mock 造数消费场景 - getting-started(_zh) 增加可照跑的列级血缘示例;serve-api-guide 补 /columns 与 /lineage 端点(真实 JSON 输出) Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent) Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
| impl Visitor for PredicateExtractor<'_> { | ||
| fn visit_pl_statement(&mut self, stmt: &PlStatement) -> VisitorResult { | ||
| match stmt { | ||
| PlStatement::If(spanned) => self.push_condition( |
There was a problem hiding this comment.
[bug] PredicateExtractor only records PlIfStmt.condition and each searched-CASE when.condition. walk_pl_statement walks elsifs[].condition via walk_expr, but this visitor does not implement visit_expr, so ELSIF predicates are never collected. Simple CASE (CASE r.x WHEN '1', where PlCaseStmt.expression is Some and when.condition is just the WHEN value) is pushed as a bare literal and degrades to Low/table_predicate: null instead of r.x = '1'. The plan explicitly lists elsifs: Vec<PlElsif> and PlCaseStmt.expression; both are untested. Enterprise PL/SQL that branches on ELSIF or simple CASE will under-report constraints to mock/codegen consumers.
Suggestion: In the PlStatement::If arm, also push_condition every spanned.elsifs entry (same PredicateKind::If, elsif span/line). In the Case arm, when spanned.expression is Some, synthesize expression = when.condition (or a dedicated comparison Expr) before push_condition. Add fixtures for IF … ELSIF … and CASE rec.col WHEN '1'.
| return Some(ConditionResolution::Direct(vec![resolved])); | ||
| } | ||
| } | ||
| let name = expr_name(expr)?.to_lowercase(); |
There was a problem hiding this comment.
[bug] After the record-field resolved_clause miss, condition_operand does expr_name(expr)?. expr_name only matches ColumnRef/PlVariable, so any FunctionCall/SpecialFunction returns None and bails out before (a) the SELECT INTO var_sources lookup and (b) the sole-cursor-table naked-column fallback that would have attached transform. Consequences: IF substr(stock_kind,1,2) = '05' (naked column, one cursor table — the documented 裸列→high path) and IF substr(v_kind,1,2) = '05' (SELECT INTO var) both become Low with no table_predicate, even though the same substr on r.stock_kind is High. The Derived arm also hard-codes transform: None, so even a future inner-name lookup would drop the #169 descriptor. Only the record-field transform path is tested.
Suggestion: Do not ? on expr_name(expr). Prefer, in order: record-field resolve (already done); var_sources lookup using expr_name(expr) or the inner transform target name; then the names.len()==1 && tables.len()==1 fallback. Thread transform through the Derived PredicateClause as well. Add tests for naked-column substr(...) and substr(v_into_var, …).
| .ok_or_else(|| error::CodeWebError::ExportError { | ||
| message: format!("No PL predicates found for '{}'", procedure), | ||
| })?; | ||
| let procedure_name = display |
There was a problem hiding this comment.
[bug] cmd_predicates sets procedure from NodeKey::from_node(...).to_string() after split_once(':'). For a packaged routine that display key is proc:pkg.prc, so JSON procedure becomes pkg.prc. column_analysis_of_routine instead uses id.name (prc) plus a separate package field. Predicates JSON has no package field. Mock consumers joining columns and predicates on procedure will miss packaged routines. Tests only cover standalone procedures, so this never fails in CI.
Suggestion: Use the same identity as columns: id.name plus package: id.package (and schema if you want it). Do not parse it back out of the NodeKey display string. Add an end-to-end packaged-procedure predicates test that asserts procedure/package match codeweb columns.
| &name, | ||
| crate::graph::search::MatchMode::Substring, | ||
| false, | ||
| false, |
There was a problem hiding this comment.
[suggestion] cmd_columns / cmd_predicates match ResolveResult::Ambiguous and return a non-zero error, but they call resolve_single_node(..., all_matches=false, fail_on_multiple=false). That helper returns Single (first hit) on multiple substring matches, so the Ambiguous arm is unreachable. MCP resolve_node and HTTP resolve_node use the same flags, so they also silently pick the first match (MCP then reports "No nodes matching" only for Empty). For a machine-readable mock entry point this can emit another routine's filters without failing.
Suggestion: Pass fail_on_multiple=true (4th argument) in CLI/MCP/HTTP, and make MCP/HTTP distinguish empty vs ambiguous. Add a two-proc substring fixture that expects a non-zero / 4xx response.
| .cloned() | ||
| .filter(|predicates| !predicates.is_empty()) | ||
| .ok_or_else(|| error::CodeWebError::ExportError { | ||
| message: format!("No PL predicates found for '{}'", procedure), |
There was a problem hiding this comment.
[suggestion] A procedure that exists but has no IF/CASE (or whose predicates were never stored because collect_block_predicates skips empty vecs) exits non-zero with No PL predicates found. codeweb columns returns empty arrays for a routine with no diagnostics. Empty branch constraints are a valid mock input (predicates: []); treating them as an error makes “unknown name” and “no branches” look similar to automations (they do differ in the message, but both are hard failures).
Suggestion: On a resolved procedure with no stored predicates, print {schema_version, procedure, predicates: []} and exit 0; reserve non-zero for unresolved/ambiguous names.
| } | ||
| } | ||
|
|
||
| /// #168: an equi-comparison where exactly one side is a `%ROWTYPE` record field |
There was a problem hiding this comment.
[suggestion] Several new comments restate control flow or embed review/issue history instead of a short WHY. The extract_record_field_join block restates the match arms and the no-guess policy already visible in the code; parse_lineage_target (src/graph/lineage.rs:1916) narrates why cmd_lineage was left duplicated. Same pattern on collect_diagnostics and HardFilter's serialize impl (the bincode-vs-JSON constraint there is a useful WHY, but it is longer than needed).
Suggestion: Keep one sentence on the non-obvious constraint (bincode field count; RecordField vs alias lookup). Drop the issue-number / review-finding / “intentionally left untouched” narration; if CLI lineage parsing should stay in sync, share parse_lineage_target instead of documenting the fork.
- If 臂遍历 elsifs 逐条产出谓词;简单 CASE(expression: Some)合成 expression = WHEN 值比较,不再把裸字面量降级为 Low - condition_operand 移除 expr_name(expr)? 提前断链:记录字段 → var_sources(expr_name 或 transform 目标列名)→ sole-table fallback 顺序保持;裸列 substr 与 SELECT INTO 变量 substr 均正确解析 - Derived 臂携带 transform(删除硬编码 None);PL 变量的 transform 提取打通 Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent) Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
…/F5) - predicates JSON 的 procedure 取 RoutineId 裸名并新增 package 字段,与 columns 可 join(此前包内过程为 pkg.prc) - columns/predicates/MCP/HTTP 四处新查询面 fail_on_multiple=true:多匹配非零/400/error JSON 显式失败,消灭静默取首个(MCP 区分 Empty 与 Ambiguous 文案;HTTP 歧义 400、未命中 404) - resolved 无分支过程返回 predicates: [] exit 0,非零保留给未解析/歧义 - cmd_lineage 改用共享 parse_lineage_target(行为不变,regress_lineage 套件守护);文档同步 Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent) Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
仅注释:保留非显性约束(bincode 固定字段数等)并压缩篇幅,删除控制流复述与评审/issue 编号叙事 Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent) Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
六项评审发现的核实记录与 F1-F6 修复方案 Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent) Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
|
六条评审意见已全部修复(计划经 Momus 审核),提交
门禁:build/clippy/fmt 干净,39 测试二进制 0 失败(757 单测)。无既有测试断言改动; |
| &name, | ||
| crate::graph::search::MatchMode::Substring, | ||
| false, | ||
| false, |
There was a problem hiding this comment.
[bug] Previous F4 asked every new query surface to pass fail_on_multiple=true so ResolveResult::Ambiguous is reachable. The follow-up flipped the procedure arm of cmd_columns, cmd_predicates, MCP codeweb_column_analysis, and HTTP GET /api/v1/columns, but the package arm of cmd_columns still calls resolve_single_node(..., false, false). With fail_on_multiple=false, multiple substring hits return Single(first) rather than Ambiguous, so the _ => "Ambiguous match" arm below it is dead and codeweb columns --package pkg_order silently succeeds when both pkg_order and pkg_order_header exist. MCP/HTTP package resolution already uses true, so the CLI is now the inconsistent surface. The new columns_ambiguous_substring_fails_explicitly test only covers --procedure.
Suggestion: Pass true on this call (same as the procedure arm), and add a --package sibling of columns_ambiguous_substring_fails_explicitly so the Ambiguous path cannot regress independently.
There was a problem hiding this comment.
已修复(2028d0b):--package 臂 fail_on_multiple 翻转为 true,与 procedure 臂及 MCP/HTTP 对齐;新增 columns_ambiguous_package_fails_explicitly(Red:双包 substring 静默 exit 0 → Green:非零 + stderr 歧义提示),防此路径独立回退。门禁全绿(39 测试二进制 0 失败)。
…llow-up) package 臂 resolve_single_node 补上 fail_on_multiple=true(此前仍静默取首个匹配,Ambiguous 臂不可达);新增 columns_ambiguous_package_fails_explicitly 防回归 Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent) Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
概述
打通「列级分析 → 造数/mock 机器可读入口」主线,一次性落地 #165–#169 五个 issue。实施计划见
docs/plans/2026-09-08-issue-165-169-column-analysis-surface.md(Momus 两轮审核通过,决策 D1–D6 全部锁定)。Closes #165, closes #166, closes #167, closes #168, closes #169
主要变更
#169 函数包裹列的字面量过滤 → HardFilter transform
HardFilter.transform: Option<FilterTransform>(serde"fn"),白名单{substr, substring→substr, nvl, trim, upper, lower},FunctionCall + SpecialFunction 双变体(关键字语法SUBSTR(x FROM 1 FOR 2)不再漏)substr(col,1,v_len)等含变量实参自动排除is_human_readable()分支手写实现——skip_serializing_if会破坏 bincode 固定布局(已被regress_mark8/20 失败暴露后修复),后续 bincode 持久化结构沿用此模式#168 %ROWTYPE 记录字段 → 跨表等值键
JoinConditionSource新增RecordField变体;WHERE/JOIN ON 中一侧为已解析记录字段时产出跨表JoinCondition(如par_sys_purchase.security_id ↔ mid_yjqs_detail.security_id)column_source的三段解析规则抽取为resolve_record_field复用;plain join / 记录-vs-参数 / 未注册记录三组负例锁死#167 PL IF/CASE 条件 → 表列谓词
PredicateExtractor(走完整 PlBlock,分支结构在extract_body_sql摊平前收集)param_table_hint;函数/动态 SQL→low 保留 origin,绝不静默 high)PredicateClause.transform携带 feat: 函数包裹列的字面量过滤纳入 HardFilter(substr/nvl/trim) #169 同款变换描述;游标 WHERE HardFilter 不混入谓词列表procedure_predicates侧表;CLIcodeweb predicates#165 按过程导出 ColumnAnalysis(CLI + MCP + HTTP 三面同 schema)
merge_table_access_edges对全部诊断字段做保序并集(此前「保留第一条」导致同过程多语句同表时 hard_filters/join_conditions 丢失)codeweb columns --procedure|--package [--table] --format json;后端graph::columns三面共享,字段与ColumnAnalysis1:1codeweb_column_analysis/codeweb_lineage(6→8 工具);HTTP 新增GET /api/v1/columns、GET /api/v1/lineagegraph::lineage::parse_lineage_target纯函数(cmd_lineage行为不变)#166 文档
ColumnAnalysis字段表(含 transform/RecordField)与 mock 造数场景、getting-started(_zh) 可照跑示例、serve-api-guide 双端点——全部示例由真实--help/真实 JSON 输出生成,QA 清单 5/5STORE_VERSION9 → 12(v10 诊断并集+transform、v11 谓词侧表、v12 PredicateClause.transform)。旧 store 会被版本门禁拒绝并提示codeweb analyze重建;增量 analyze 场景经store_is_current()自动重建RoutineId::normalized()小写化(与NodeKey::from_node查找路径一致)——修复 review 发现的大写过程名查找 miss验证
新增 ~45 个测试(extractor 单测 15、predicates 单测 11、store round-trip、lineage 目标解析 6、集成 regress_columns/regress_predicates/mcp/serve)。TDD Red→Green 全程留痕(含 kill-switch 式修复验证)。
已披露的合规偏差
tests/regress_issue_159_sequence_informed.rs版本戳字面量 9→12(跟踪 STORE_VERSION 的机械同步,语义断言不变;版本拒绝基线测试未动)tests/mcp_test.rs工具清单断言 6→8(新增工具的必要同步,plan-sanctioned)"proc:P"→"proc:p"(跟随正确的归一化修复)已知限制
LineageConfig(不读取项目codeweb.toml [lineage]覆盖项);与 CLI 默认行为一致🤖 Generated with Claude Code