Desktop: the Browse modals match a search word by word, ranked like the model's search (F5)

[Broccolito]

Summary

#242 fixed QA finding F5 for the model-facing marketplace search. The desktop app's own BAAM search had the same defect in TypeScript: skillMatches / extensionMatches in ui/desktop/src/components/baam/registry.ts checked whether the whole query was a substring of a single field.

This PR ports crates/biorouter/src/catalog_search.rs (landed on main via #242 as marketplace/search.rs; moved, and given the word-boundary rule, by #266) to TypeScript as ui/desktop/src/components/baam/search.ts, and both Browse modals use it:

• the query is split at whitespace and punctuation, lowercased and de-duplicated
• an entry matching any term is a hit
• terms under 3 characters match whole words only
• filler words are dropped unless nothing else is left
• a plural falls back to its singular
• ranking: the query as written first — its words, in that order, as whole words — then terms matched, then field weight × match quality, then registry order

scoreEntry is the pure per-entry function: it returns a score (0 means no match) and the matchedTerms. rankSkills / rankExtensions replace the old matchers. Their only callers were BrowseSkillsModal, BrowseExtensionsModal and one test mock, and all three are updated.

Measured before → after

In the real BrowseSkillsModal, rendered with seven rows copied verbatim from landing/registry.json, on the unfixed code:

query	before	after
R scripting ggplot visualization	0 rows	5: ggplot-visualization, r-scripting, data-visualization, python-scripting, clinical-biostatistics
ggplot	2	exactly the same 2
r-scripting	0 rows (the id was never searched)	r-scripting first

On the full registry (129 skills, 37 extensions): the phrase goes 0 → 11 (the same count #242 reports for the model-facing search), r-scripting 0 → 9, and the extension query SPOKE knowledge graph 0 → 3.

Behaviour changes, each deliberate

• A search shows one ranked list headed "Matches (n)". Browsing with no query keeps the category headings in registry order. Under the headings, a Core skill matching one word of the query would sit above a Biomedical skill matching all of them, so the ranking would exist but not be visible. The category filter chips still apply.
• The id is searched (and an extension's extension_name), as in the Rust catalog.
• The license is no longer searched. All 166 registry entries are Apache-2.0, so the field never told entries apart. Under word matching it also made PACS list all 129 skills, because the singular pac is inside apache. The Rust catalog never searched it, and dropping it gives exact parity (below). apache still finds the 49 skills that carry it as a tag or keyword.
• A query of only spaces now browses. Before, it searched for the spaces and showed "No skills match your search."
• Malformed entries are handled. Browsing reads no field at all. A search skips missing fields where it used to throw during render, because isRegistryDocument only checks that each entry is an object.

Parity with the Rust matcher

⚠ This was measured before #266, against the file at marketplace/search.rs; the realignment section below re-measures the one rule that has changed since. I compiled the real search.rs (byte-identical to main's at the time) into a small std-only Rust harness and diffed it against the TypeScript functions on the full registry. The query set was 1,345 skill and 336 extension queries: hand-written phrases, every id, name, tag and keyword, and description openings, including filler, plurals, short terms, punctuation and non-ASCII text. rankSkills / rankExtensions return the same entries, matched terms and order: 0 mismatches over 11,920 compared hits.

To check that the harness can fail, I mutated the TypeScript. Raising the short-term threshold by one produced 82 mismatches, and deleting a single filler word produced 13.

The only allowed divergence is tie order. Rust breaks ties by id (a BTreeMap), while the modal breaks them by the registry document's order, which is also the order it browses in.

Tests

• search.test.ts (43 tests):
  • the Rust file's own cases, ported, including its six written_in boundary cases
  • the score packing: more terms beat better placement, and the query as written beats the same words scattered in weightier fields
  • the three QA queries
  • filler words, short terms, plurals and Unicode
  • robustness against malformed entries
  • the searched field set and weights of each catalog, one field at a time
  • that the license is not searched
• BrowseSkillsModal.test.tsx (+8): the QA queries typed into the real modal, the grouped browse view against the ranked "Matches" view, a whitespace-only query, the category filter during a search, and the empty state.
• BrowseExtensionsModal.test.tsx (+1): the ranked list on screen. The extensionMatches mock is gone; that suite now runs the real search.
• I mutation-tested every rule and each piece of modal wiring. Two first-draft gaps were caught this way and closed: the whole-query tier being demoted to "one more term", and keywords being dropped from the searched fields.

Realigned onto #266 (catalog_search.rs)

#266 changes two things under this port, and the header's own sentence — "A change to a rule below is a change to both files" — is what made that findable. Both are answered here, one commit each:

1. The file moved. crates/biorouter/src/marketplace/search.rs → crates/biorouter/src/catalog_search.rs, MarketplaceSearch/MarketplaceSearchHit → CatalogSearch/CatalogSearchHit. search.ts and search.test.ts now cite the real path and names. The sentence is kept word for word.

2. The whole-query rank rule changed, and search.ts:231 still had the substring test it replaced. A query now counts as written only where it starts and ends at a word boundary (written_in), the same edges the three-character term rule already enforced.

Measured, before → after

Same harness, the real search.ts bundled twice — this branch's previous commit and the new one — ranking the live landing/registry.json (129 skills + 37 extensions), 38 queries × both catalogs = 76 searches, full ordered hit lists compared. Control run (old vs old): 0 of 76 changed, so a 0 means something.

entries that START being returned:   0
entries that STOP being returned:  198
searches whose ranked list changes:  7 of 76
every changed search has all terms under 3 chars: true

catalog	query	before	after	leading entries unchanged	before top	after top
skills	R	125	8	8 of 8	empirical-paper-submission-rr	empirical-paper-submission-rr
skills	dy	6	0	—	financial-accounting-econometrics	—
skills	s p	6	1	0 of 1	research-evaluation-venues	frontend-design
skills	ml	9	3	3 of 3	scientific-machine-learning	scientific-machine-learning
skills	AI	24	2	2 of 2	clinical-ai-modeling	clinical-ai-modeling
extensions	R	37	1	1 of 1	codegraphagent	codegraphagent
extensions	s p	7	1	0 of 1	spokeagent	playwrightagent

That is #266's blast radius reproduced from the TypeScript side, drop for drop: 0 start / 198 stop / 7 of 76, every one a query whose every term is short, and R 125→8 with its whole surviving list in the same order. Every survivor list is a prefix of its before-list except the two s p searches — #266's "only two searches change their first result, both for s p". The other 33 queries (ggplot, complexheatmap, python, rna, knowledge graph, differential expression, R scripting ggplot visualization, r-scripting, SPOKE knowledge graph, c++, ucsf, …) return the identical list in the identical order in both catalogs.

Fail-before, on the rule this replaces: a one-letter R returned 125 of 129 skills, 117 of them matching no term at all, and ranked empirical-paper-submission-rr above r-scripting.

The word boundary, and why it is not \b

writtenIn is a transliteration of written_in, not a regular expression, and that is measured rather than assumed. JavaScript defines \b over [A-Za-z0-9_] alone, so a \b-built test disagrees with Rust on 4 of the 11 cases now asserted:

text	phrase	ours (= Rust)	\b
c++ code	++	true	false — an edge that is not a letter or digit is a boundary itself
snake_case	case	true	false — _ is a word character to \b, not to Rust
naïve	naï	false	true — ï is a boundary to \b, a letter to Rust
𝐚rna	rna	false	true — an astral letter read as two UTF-16 halves

The scan compares code points, like Rust's char_indices: read as UTF-16 units, the trailing half of an astral letter is a lone surrogate matching no letter class, so the phrase would be read as written where Rust refuses it. It also tries every position rather than the first occurrence, because a refused occurrence can overlap an accepted one (a a in ba a a). WORD_CHAR is spelled as the explicit complement of the WORD_BREAK that words() already splits on, so a term and the query around it can never be held to different edges.

Tests

Three existing tests change — the same three #266 changed in Rust — and no modal test moves (15 + 12 keep passing):

• still finds a verbatim occurrence: its dy half asserted the leak as a feature, exactly as Rust's a_verbatim_occurrence_is_still_a_hit did. Replaced by three tests: a one-letter query as a whole word, a phrase found only inside longer words, and dy/s p both empty.
• ranks a verbatim match above every term match, even one matching more terms: the premise is now unreachable. A phrase written in a field has every one of its own words in that field as a whole word, so a written entry always matches every term. Re-pitted against placement instead (prose vs two names), with catalog_search.rs's own tidy code → [styler, code-tidy] ranking beside it.
• R alone over the fixture: 6 hits, 3 matching no term → the 3 skills that are about R.

Added: a direct writtenIn describe carrying Rust's six written_in cases plus the four \b divergences. writtenIn is exported for it, the way Rust asserts them from inside the module.

Mutation check. Reverting the body to return text.includes(phrase) and changing nothing else fails 6 of 43 tests, both boundary tests among them. The one new test that survives is the score/packing one — correctly: it pins the tier arithmetic, not the boundary rule.

Follow-up, deliberately not in this PR

#266 found the same whole-phrase filter a third and fourth time, both renderer TypeScript with no shared code: ui/desktop/src/components/skills/SkillsView.tsx:72-86 (Settings → Skills) and ui/desktop/src/components/bottom_menu/BottomMenuSkillSelection.tsx:84-101 (the composer picker). Pointing them at this branch's search.ts is the right move, and it does not fit yet:

• They list rows, not skills. SkillCatalogEntry is a union — a single skill, or a bundle row whose searchable text is displayName, name and its member skill names. The Rust matcher has no bundle row: it ranks individual skills and carries the bundle as a Label field on each member (skills_extension.rs::search_fields). A member's name is therefore a Name field of a different entry there, and weighting it on the bundle row would be a TypeScript-only rule — in the one file whose whole value is that every rule is shared, with no Rust row to diff it against.
• SkillsView discards the ranking. It re-groups its filtered entries by provenance (Biorouter / project / per-extension / other) and renders them grouped, so the ranking would be thrown away unless that view switches to one ranked list under a query, the way BrowseSkillsModal does here. That is a layout decision with its own tests, not a port.
• fix(skills): rank an installed-skill search by the words of the query, not all of them (F5) #266 is unmerged, so the weights to port can still move; fix(skills): rank an installed-skill search by the words of the query, not all of them (F5) #266's own body files this as a follow-up "once it lands".

Verification

• npx vitest run src/components/baam/: 84 passed (80 before the realignment)
• npm run lint:check: exit 0 (re-run after the realignment)
• npm run format:check / npx prettier --check on both changed files: exit 0
• npm run test:run: 5,061 passed, 1 skipped, 0 failed, but the run exits 1. Every test passes; the afterAll teardown (browser.close() on a real Playwright Chromium) in src/utils/artifactCdnAssets.browser.test.ts hangs past 30 s. It fails the same way when run alone and on pristine main (6455bc2), so it is not caused by this branch. With that one file excluded: 445/445 files, exit 0. I've flagged it as a separate task.
• I did not check this in the running Electron app. The modal tests render the real component in jsdom.

🤖 Generated with Claude Code