Skip to content

[docs][material-icons] Prototype semantic icon search with static embeddings - #49205

Closed
Janpot wants to merge 18 commits into
mui:masterfrom
Janpot:docs/icon-search-potion
Closed

Janpot wants to merge 18 commits into
mui:masterfrom
Janpot:docs/icon-search-potion

Conversation

@Janpot

@Janpot Janpot commented Sep 24, 2026 •

Copy link
Copy Markdown
Member

Prototype for ranking icon search results by meaning instead of by keyword match order. Builds on #49204, which adds generated keywords and visual descriptions for every icon to synonyms.js.

Keyword search finds icons but orders them poorly: "bell" puts ConveyorBelt first, "car" puts Card icons next to cars, and searches with no keyword hit ("rubbish", "puppy") return nothing. This uses the potion-base-8M static embedding model to score how close each icon is to the query and orders the results by that.

How it works:

  • A static embedding model is a lookup table with one vector per word, so the browser needs no model and no tokenizer. docs/scripts/buildIconSearchIndex.mjs precomputes a vector for every single-token word in the model's vocabulary and one vector per icon (its name and synonyms), and writes them to docs/public/static/material-icons/.
  • The page fetches that index the first time someone searches, so the page itself does not get bigger. Until it has loaded, or if it fails to load, results are in the current keyword order.
  • The query vector is the sum of its words' vectors. Results are grouped as: name matches, then keyword matches, then the 60 nearest icons without a keyword match. Each group is sorted by similarity.

Notes for reviewers:

  • The index is about 1.6 MB compressed. Keeping 64 of the model's 256 dimensions ranked as well as more dimensions on our test queries.
  • Words that are not a single token in the model ("cog", "dustbin") have no vector, because a vector built from word pieces is noise. Those still work through keyword matching.
  • On a hand-written set of test queries, the right icon was ranked first for 27 of 40 meaning queries (18 today) and 13 of 22 appearance queries (5 today). The queries were written by us, so treat that as a rough signal.
  • Appearance queries ranked better (18 of 22 first) when the index was built from the descriptions directly instead of from synonyms.js, since the synonyms script sorts words and drops words contained in longer ones.
  • The index has to be rebuilt with pnpm docs:mdicons:search-index when synonyms.js changes. The icon-descriptions skill does that as part of its workflow.
  • Known weak spot: "three dots" does not rank MoreVert near the top.

@code-infra-dashboard

code-infra-dashboard Bot commented Sep 24, 2026 •

Copy link
Copy Markdown

Deploy preview

Bundle size

Bundle Parsed size Gzip size
@mui/material 0B(0.00%) 0B(0.00%)
@mui/lab 0B(0.00%) 0B(0.00%)
@mui/private-theming 0B(0.00%) 0B(0.00%)
@mui/system 0B(0.00%) 0B(0.00%)
@mui/utils 0B(0.00%) 0B(0.00%)

Details of bundle changes


Check out the code infra dashboard for more information about this PR.

@github-actions github-actions Bot added the PR: out-of-date The pull request has merge conflicts and can't be merged. label Sep 24, 2026
@Janpot Janpot closed this Sep 24, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

PR: out-of-date The pull request has merge conflicts and can't be merged.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant