Repository navigation
feat(hsearch)!: literal queries, surface DB failures, always back up before migration - #80
Merged
Merged
Conversation
hybrid_search treated an error from has_embeddings() as "no embeddings" and silently fell back to FTS-only results. The vec_bookmarks table is always created at startup, so an error here is a real database fault (lock, corruption) and must reach the user. The info command keeps its fallback on purpose: it is display-only and should not hide the remaining statistics.
The pre-migration backup was skipped when the bookmark count query reported zero rows, and also when that query failed. A locked or corrupted database was therefore treated as empty, and the migration ran without a backup. Drop the row count check and keep only the 16 KB size gate. Backing up a large database that has no bookmarks costs nothing; skipping the backup of a real one can lose data.
On zero hits hsearch --json wrote a colored message to stderr and nothing to stdout, so JSON consumers received invalid input. Fall through to the JSON branch instead, matching search --json.
hsearch passed the raw query to FTS5, so ordinary technical terms such as c++, node.js, foo-bar or title:x failed with syntax errors. The whole command aborted even though the semantic branch could have answered. Quote every whitespace-separated term for the FTS branch. The semantic branch still embeds the original text. FTS5 power syntax stays available in search, where it belongs: in hybrid mode the same text is also embedded, so operators such as "NOT oauth" would pull semantic results towards oauth. BREAKING CHANGE: hsearch no longer interprets FTS5 syntax (AND/OR/NOT, phrases, column filters, prefix*). Use search for these queries.
sysid
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
hsearchand the pre-migration backup turned database failures into "nothingfound" / "nothing to back up". This PR makes those failures visible, always backs
up a real database before migrating, and makes
hsearchtreat its query as plaintext so technical terms stop breaking it.
Supersedes the valid parts of #79 with a minimal implementation.
Changes
fix(hsearch): propagate vector store failureshas_embeddings().unwrap_or(false)— a locked or corrupt DB silently degraded hybrid search to FTS-onlyvec_bookmarksalways exists after startup, so an error is a real faultfix(db): always back up databases before migratingCOUNT(*)returned 0 or failed — a broken DB was migrated without a backupfix(hsearch): print [] for --json when nothing matches[], consistent withsearch --jsonfeat(hsearch)!: match query terms literallyc++,node.js,foo-bar,title:x,ANDaborted the whole search with syntax errorshsearchno longer interprets FTS5 syntax —AND/OR/NOT,"phrases",column filters and
prefix*are matched as literal text. Usesearchfor those.Rationale: in hybrid mode the same text is also embedded, so operators work against
the semantic branch (
auth NOT oauthpulls semantic results towards oauth).Before / after:
Requires a minor version bump.
Testing
All new tests were written first and verified failing on
main:Ok([]))hsearch --jsonwith zero hits prints[](CLI test)fts_query()quoting incl. embedded"; service-level search forc++,node.js,foo-bar,AND,tips:finds the bookmarkmake testgreen.Docs
hsearch --help: query is matched literallyskill/bkmr-memory/SKILL.md: no quoting needed forhsearchSearch-and-Discovery.mdupdated separately (to be published with the release)Follow-up
Error-prefix duplication and blank-query handling: #81