|
| 1 | +--- |
| 2 | +id: DR-087 |
| 3 | +slug: cross_artifact_tui_fuzzy_search_with_relational_weighting |
| 4 | +name: Design Revision - Cross-artifact TUI fuzzy search with relational weighting |
| 5 | +created: '2026-03-10' |
| 6 | +updated: '2026-03-10' |
| 7 | +status: draft |
| 8 | +kind: design_revision |
| 9 | +aliases: [] |
| 10 | +owners: [] |
| 11 | +relations: |
| 12 | +- type: implements |
| 13 | + target: DE-087 |
| 14 | +delta_ref: DE-087 |
| 15 | +source_context: |
| 16 | +- type: improvement |
| 17 | + id: IMPR-011 |
| 18 | + summary: TUI polish, navigation, and relational display |
| 19 | +code_impacts: |
| 20 | +- path: supekku/tui/widgets/search_overlay.py |
| 21 | + change: new |
| 22 | + summary: Modal overlay widget with fuzzy search across all artifact types |
| 23 | +- path: supekku/tui/search/scorer.py |
| 24 | + change: new |
| 25 | + summary: Pure-function search scorer with weighted field matching |
| 26 | +- path: supekku/tui/search/index.py |
| 27 | + change: new |
| 28 | + summary: Self-contained SearchEntry builder — instantiates own registries |
| 29 | +- path: supekku/tui/app.py |
| 30 | + change: modify |
| 31 | + summary: Rebind / to global search overlay, add Ctrl+F for per-type search |
| 32 | +- path: supekku/tui/browser.py |
| 33 | + change: modify |
| 34 | + summary: Mount search overlay, handle selection → navigate_to_artifact |
| 35 | +verification_alignment: |
| 36 | +- id: VT-087-001 |
| 37 | + kind: VT |
| 38 | + impact: new |
| 39 | + summary: Search scorer weights and ranking correctness |
| 40 | +- id: VT-087-002 |
| 41 | + kind: VT |
| 42 | + impact: new |
| 43 | + summary: SearchEntry index builder covers all artifact types and fields |
| 44 | +- id: VT-087-003 |
| 45 | + kind: VT |
| 46 | + impact: new |
| 47 | + summary: For equivalent match quality, own-ID matches rank above relation-target matches |
| 48 | +- id: VA-087-001 |
| 49 | + kind: VA |
| 50 | + impact: new |
| 51 | + summary: Manual TUI walkthrough — overlay opens, searches, navigates |
| 52 | +design_decisions: |
| 53 | +- id: DEC-087-01 |
| 54 | + summary: Separate SearchEntry index rather than enriching ArtifactEntry |
| 55 | + status: accepted |
| 56 | +- id: DEC-087-02 |
| 57 | + summary: Tiered scoring weights — own-ID 1.0, title 0.7, relation targets 0.5, attributes 0.4 |
| 58 | + status: accepted |
| 59 | +- id: DEC-087-03 |
| 60 | + summary: Modal overlay UX (fzf-style) rather than inline or separate screen |
| 61 | + status: accepted |
| 62 | +- id: DEC-087-04 |
| 63 | + summary: "/" for global search, Ctrl+F for per-type inline search |
| 64 | + status: accepted |
| 65 | +- id: DEC-087-05 |
| 66 | + summary: Self-contained index builder — instantiates own registries, no ArtifactSnapshot coupling |
| 67 | + status: accepted |
| 68 | +open_questions: [] |
| 69 | +--- |
| 70 | + |
| 71 | +# DR-087 – Cross-artifact TUI fuzzy search with relational weighting |
| 72 | + |
| 73 | +## 1. Executive Summary |
| 74 | +- **Delta**: [DE-087](./DE-087.md) |
| 75 | +- **Status**: draft |
| 76 | +- **Last Updated**: 2026-03-10 |
| 77 | +- **Synopsis**: Add a modal fuzzy-search overlay to the TUI that searches across all artifact types, including YAML frontmatter attributes and relation targets, with weighted scoring that ranks own IDs above referenced IDs. |
| 78 | + |
| 79 | +## 2. Problem & Constraints |
| 80 | + |
| 81 | +- **Current Behaviour**: The TUI fuzzy search (`ArtifactList._filtered_entries()`) operates within a single artifact type at a time, matching only on `id`, `title`, and `status` via `textual.fuzzy.Matcher`. Discovering an artifact requires first selecting the correct type tab, then searching. YAML attributes (tags, kind, category) and relation targets are not searchable. |
| 82 | +- **Drivers / Inputs**: |
| 83 | + - IMPR-011 — TUI polish, navigation, and relational display |
| 84 | + - DE-085 — relation query layer providing `collect_references()` and `ReferenceHit` |
| 85 | +- **Constraints / Guardrails**: |
| 86 | + - ADR-002: No backlinks in frontmatter — relation targets are forward-only, computed at runtime |
| 87 | + - ADR-009: Standard registry API — cross-registry access via `collect()` |
| 88 | + - POL-001: Reuse existing `textual.fuzzy.Matcher`, registry factories, relation query layer |
| 89 | + - POL-002: Named constants for score weights and field names |
| 90 | +- **Out of Scope**: |
| 91 | + - Relational navigation (selecting a relation to jump to its target) — future IMPR-011 work |
| 92 | + - Full-text content search (searching inside markdown body) |
| 93 | + - CLI (non-TUI) cross-type search |
| 94 | + - Embedded sub-entity display (DR/IP/phase inline) |
| 95 | + |
| 96 | +## 3. Architecture Intent |
| 97 | + |
| 98 | +- **Target Outcomes**: |
| 99 | + - A single keybinding (`/`) opens a modal overlay that fuzzy-searches across all artifact types simultaneously |
| 100 | + - YAML frontmatter attributes (tags, kind, category, c4_level, slug) are included in the match surface |
| 101 | + - Own-ID matches rank above relation-target matches for equivalent match quality |
| 102 | + - Selecting a result navigates to the artifact in the browser (reuses `BrowserScreen.navigate_to_artifact()`) |
| 103 | +- **Guiding Principles**: |
| 104 | + - **SRP**: Search scoring is a pure-function module, not a widget concern. The overlay is a thin UI shell over the scorer. |
| 105 | + - **Self-contained search**: The index builder instantiates its own registries (DEC-087-05). No coupling to ArtifactSnapshot's internal state. Reuses registry factory code (POL-001), not shared mutable state. |
| 106 | + - **ArtifactEntry stays clean**: The view model is not enriched with search-specific data. A separate `SearchEntry` carries the searchable surface. |
| 107 | + - **"Find the thing" posture**: The scoring model privileges locating a specific artifact over finding artifacts related to a query. The CLI `--related-to` flag serves the latter use case. |
| 108 | + |
| 109 | +## 4. Code Impact Summary |
| 110 | + |
| 111 | +| Path | Current State | Target State | |
| 112 | +| --- | --- | --- | |
| 113 | +| `supekku/tui/search/__init__.py` | Does not exist | New package | |
| 114 | +| `supekku/tui/search/scorer.py` | Does not exist | Pure-function scorer: `score_entry(query, search_entry) -> float` | |
| 115 | +| `supekku/tui/search/index.py` | Does not exist | `build_search_index(root) -> list[SearchEntry]` — self-contained | |
| 116 | +| `supekku/tui/widgets/search_overlay.py` | Does not exist | Modal overlay: input + results DataTable | |
| 117 | +| `supekku/tui/app.py` | `/` bound to `action_focus_search` | `/` bound to `action_global_search`; `Ctrl+F` bound to `action_focus_search` | |
| 118 | +| `supekku/tui/browser.py` | No overlay awareness | Mounts search overlay; handles result selection | |
| 119 | + |
| 120 | +`supekku/scripts/lib/core/artifact_view.py` is **not modified**. The search index builder reuses `_REGISTRY_FACTORIES` and `adapt_record` from `artifact_view` but does not touch `ArtifactSnapshot`. |
| 121 | + |
| 122 | +## 5. Verification Alignment |
| 123 | + |
| 124 | +| Verification | Impact | Notes | |
| 125 | +| --- | --- | --- | |
| 126 | +| VT-087-001 | new | Scorer unit tests: weight application, field priority, edge cases | |
| 127 | +| VT-087-002 | new | Index builder: all artifact types produce SearchEntry with correct fields | |
| 128 | +| VT-087-003 | new | Ranking: for equivalent fuzzy match quality, own-ID outranks relation-target | |
| 129 | +| VA-087-001 | new | Manual TUI walkthrough: overlay lifecycle, search, navigation | |
| 130 | + |
| 131 | +## 6. Supporting Context |
| 132 | + |
| 133 | +- **Related Deltas**: DE-085 (relation query layer — completed), DE-053 (TUI MVP) |
| 134 | +- **Related Specs**: PROD-010 (CLI UX), PROD-015 (spec taxonomy and navigation) |
| 135 | +- **Backlog**: IMPR-011 (TUI polish — this delta partially addresses the "relational display" item) |
| 136 | + |
| 137 | +## 7. Design Decisions & Trade-offs |
| 138 | + |
| 139 | +### DEC-087-01 — Separate SearchEntry index |
| 140 | + |
| 141 | +**Decision**: Build a `SearchEntry` dataclass in the search module rather than enriching `ArtifactEntry`. |
| 142 | + |
| 143 | +**Rationale**: `ArtifactEntry` is a view model for the artifact list panel. Adding `attrs`, `relation_targets`, etc. would couple it to search concerns that most consumers don't need. A separate index keeps SRP clean and lets the search surface evolve independently. |
| 144 | + |
| 145 | +**Data shape**: |
| 146 | + |
| 147 | +```python |
| 148 | +@dataclass(frozen=True) |
| 149 | +class SearchEntry: |
| 150 | + entry: ArtifactEntry |
| 151 | + searchable_fields: dict[str, str] # field_name -> text value |
| 152 | + relation_targets: tuple[str, ...] # referenced artifact IDs |
| 153 | +``` |
| 154 | + |
| 155 | +`searchable_fields` is populated from the registry record's frontmatter attributes: |
| 156 | +- `id`, `title`/`name`, `status` (from ArtifactEntry) |
| 157 | +- `kind`, `slug`, `category`, `c4_level` (from raw record via `getattr`) |
| 158 | +- `tags` — per-tag entries (`tag.0`, `tag.1`, etc.) for correct per-tag scoring |
| 159 | + |
| 160 | +**Known fragility**: The index builder uses `getattr` on heterogeneous record types to extract frontmatter attributes. If a record model renames an attribute (e.g. `tags` → `labels`), the builder silently produces no entries for that field. When ADR-009 eventually introduces a Protocol for registry records, the builder should adopt it. |
| 161 | + |
| 162 | +**Index construction**: `build_search_index(root)` instantiates registries via `_REGISTRY_FACTORIES`, calls `collect()` on each, extracts frontmatter attributes via `getattr`, and calls `collect_references()` on each record for relation targets. Built lazily when the overlay opens; not cached across open/close cycles. |
| 163 | + |
| 164 | +### DEC-087-02 — Tiered scoring weights |
| 165 | + |
| 166 | +**Decision**: Score = `max(weight × fuzzy_score)` across all fields and relation targets. |
| 167 | + |
| 168 | +```python |
| 169 | +WEIGHT_OWN_ID = 1.0 |
| 170 | +WEIGHT_TITLE = 0.7 |
| 171 | +WEIGHT_RELATION_TARGET = 0.5 |
| 172 | +WEIGHT_ATTRIBUTE = 0.4 |
| 173 | +``` |
| 174 | + |
| 175 | +**Rationale**: Typing "DE-08" should surface DE-08x artifacts at the top. Relation targets score high enough (0.5) that a perfect relation match (0.5) beats a weak own-ID match (~0.3), but loses to a moderate one (~0.6). This balances "find the thing" with "find things that reference the thing." |
| 176 | + |
| 177 | +**UX posture**: The scoring model deliberately privileges "find the thing" over "find referrers." The CLI `--related-to` flag already serves the latter use case. This is a known limitation of single-box search, not a bug. |
| 178 | + |
| 179 | +**Per-target scoring**: Each relation target ID is scored individually; the best match is used. This avoids inflating scores from concatenated target strings. |
| 180 | + |
| 181 | +### DEC-087-03 — Modal overlay UX |
| 182 | + |
| 183 | +**Decision**: The global search is a modal overlay (Textual `ModalScreen` or equivalent), not an inline replacement of the artifact list or a separate screen. |
| 184 | + |
| 185 | +**Rationale**: Preserves browser state underneath. Familiar fzf/Ctrl+P pattern. Clean focus lifecycle — open, type, select or dismiss. |
| 186 | + |
| 187 | +**Behaviour**: |
| 188 | +- `/` opens the overlay with an empty search input and focus |
| 189 | +- Typing filters results in real-time via Textual's reactive input handling |
| 190 | +- Results show: `[Type] ID Title` with type badge styled per `ArtifactGroup` |
| 191 | +- Up/Down navigate results; Enter selects → overlay closes, browser navigates to artifact |
| 192 | +- Escape dismisses without action |
| 193 | + |
| 194 | +### DEC-087-04 — Keybinding reassignment |
| 195 | + |
| 196 | +**Decision**: `/` → global search overlay. `Ctrl+F` → per-type inline search (existing `_SearchInput`). |
| 197 | + |
| 198 | +**Rationale**: Cross-type search will be the 90% use case. It deserves the single-keystroke binding. Per-type search is a refinement within a known type — `Ctrl+F` is a reasonable secondary binding. |
| 199 | + |
| 200 | +**Migration**: `action_focus_search` binding changes from `slash` to `ctrl+f`. New `action_global_search` binding on `slash`. |
| 201 | + |
| 202 | +### DEC-087-05 — Self-contained index builder |
| 203 | + |
| 204 | +**Decision**: The search index builder instantiates its own registries via `_REGISTRY_FACTORIES` from `artifact_view.py`. It does not read from or modify `ArtifactSnapshot`. |
| 205 | + |
| 206 | +**Rationale**: The adversarial review (point 1) identified that storing `raw_records` in `ArtifactSnapshot` creates a hidden sync invariant — `entries` and `raw_records` must stay in lockstep, and the snapshot becomes a search concern. Registry instantiation is cheap (frontmatter scan), the overlay is opened infrequently, and full decoupling means the search module has no reverse dependencies into the snapshot's internal structure. |
| 207 | + |
| 208 | +**Reuse posture**: The builder reuses `_REGISTRY_FACTORIES` and `adapt_record` (code reuse per POL-001) but does not share mutable state with the snapshot. This is the right form of reuse — shared code, not shared data. |
| 209 | + |
| 210 | +```python |
| 211 | +def build_search_index(*, root: Path) -> list[SearchEntry]: |
| 212 | + """Build search index from fresh registry instances. |
| 213 | +
|
| 214 | + Self-contained — no dependency on ArtifactSnapshot. |
| 215 | + """ |
| 216 | + entries: list[SearchEntry] = [] |
| 217 | + for art_type in ArtifactType: |
| 218 | + registry = _REGISTRY_FACTORIES[art_type](root) |
| 219 | + try: |
| 220 | + records = registry.collect() |
| 221 | + except Exception: |
| 222 | + continue |
| 223 | + for record_id, record in records.items(): |
| 224 | + ae = adapt_record(record, art_type) |
| 225 | + fields = _extract_searchable_fields(ae, record) |
| 226 | + targets = _extract_relation_targets(record) |
| 227 | + entries.append(SearchEntry( |
| 228 | + entry=ae, |
| 229 | + searchable_fields=fields, |
| 230 | + relation_targets=targets, |
| 231 | + )) |
| 232 | + return entries |
| 233 | +``` |
| 234 | + |
| 235 | +## 8. Open Questions |
| 236 | + |
| 237 | +None — resolved during design triage and adversarial review. |
| 238 | + |
| 239 | +## 9. Rollout & Operational Notes |
| 240 | + |
| 241 | +- **Non-breaking**: Per-type search continues to work on `Ctrl+F`. |
| 242 | +- **No data migration**: Search index is ephemeral, built on each overlay open. |
| 243 | +- **Performance**: Registry instantiation + `collect()` + `collect_references()` for ~500 artifacts is well under 100ms on modern hardware. No debounce needed at current scale — Textual's reactive input handling is sufficient. If performance becomes a concern, debounce can be added with real measurements. |
0 commit comments