[syarihu]

What

• Remove the Laya integration added in Integrate Laya-MLX for semantic duplicate detection, keyword selection, and reranking #37 and Show Laya status in lk stats and MCP get_stats #39. Measured on real knowledge bases, none of its features beat the heuristics they sit on top of.
• Keep the Japanese handling that turned out to work without a model: Japanese terms in auto-extracted keywords, and splitting a pasted Japanese sentence query into searchable terms.

Why

Measured on personal knowledge bases (lk's own plus a couple of others), with 28 Japanese sentence queries and 28 English keyword queries written against their entries:

• Duplicate check: noul scored unrelated pairs 0.97-0.99, the same as paraphrases. Other question forms did not separate them either (AUC 0.54 / 0.63 for old vs. new versions of the same entry against distinct but similar entries).
• Auto category: 16/58 correct, below always answering the most common category (24/58).
• Reranking: top-1 hits went from 24/25 to 18/25 on Japanese sentence queries, and from 23/28 to 24/28 on English keyword queries.
• Keyword selection: the gain came from adding kanji compounds as candidates. Picking them by frequency did about as well as picking them with Laya.

Changes

• Remove src/laya.rs, scripts/laya_server.py, the [laya] config, LK_LAYA / LK_NO_LAYA, reranking and semantic_score, auto category, and the Laya status in lk stats / MCP get_stats.
  • Files that were only changed for Laya are restored to their state before Integrate Laya-MLX for semantic duplicate detection, keyword selection, and reranking #37. An existing [laya] section in a config file is ignored like any unknown key.
  • Kept from Integrate Laya-MLX for semantic duplicate detection, keyword selection, and reranking #37: project-scope results win ties in search, and MCP add_knowledge auto-extracts keywords when none are given.
• src/keywords.rs:
  • extract_keywords adds up to 3 Japanese terms (mostly kanji compounds such as 排他制御) on top of the 8 ASCII / katakana keywords. Against curated keywords, recall of the Japanese ones went from 0.18 to 0.45, and overall precision from 0.53 to 0.44.
  • Add query_terms: a whitespace-separated token containing hiragana is split into the ASCII words, katakana words, and kanji compounds in it. Tokens without hiragana are kept as typed.
• src/db.rs: search_entries runs the query through query_terms, so both the CLI and MCP search get it.
  • Japanese sentence queries: top-1 hits went from 3/28 to 25/28, and queries with no results from 18 to 0. English keyword queries are unchanged.
• src/main.rs: the default --threshold of lk keywords --regen follows the new cap (8 -> 11).
• README.md: drop the Laya docs and describe the keyword and query handling.

Tests

• cargo test: 237 unit / 136 integration passed (1 pre-existing ignored).
• cargo clippy --all-targets and cargo fmt --check: clean.
• Added test_query_terms_*, test_extract_keywords_adds_japanese_terms, test_extract_keywords_caps_japanese_terms, test_extract_keywords_english_only_is_unchanged, and test_search_with_japanese_sentence_query.

Summary by CodeRabbit

• New Features
  • Japanese sentence queries are now split into searchable terms, improving results for unspaced Japanese searches.
  • Automatic keyword extraction can include up to 8 English or katakana keywords and 3 Japanese keywords.
• Changes
  • Search results now use lexical matching and project and update-time ordering; semantic reranking and semantic scores are no longer available.
  • Adding an entry without a category leaves it uncategorized, and keyword extraction no longer uses Laya.
  • Laya status has been removed from statistics.

[coderabbitai]

Navigate logical layers of code changes, visualize relationships, and explore their blast radius.

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Advanced

Run ID: 540d27d1-c9e3-4c41-8e06-f4729fd453d7

📥 Commits

Reviewing files that changed from the base of the PR and between 4114502 and 7fb2c28.

📒 Files selected for processing (15)

• .gitignore
• README.md
• scripts/laya_server.py
• src/cmd/add.rs
• src/cmd/search.rs
• src/cmd/stats.rs
• src/config.rs
• src/db.rs
• src/keywords.rs
• src/laya.rs
• src/main.rs
• src/mcp.rs
• src/similarity.rs
• src/util.rs
• tests/integration_test.rs

💤 Files with no reviewable changes (6)

• src/cmd/stats.rs
• scripts/laya_server.py
• .gitignore
• src/cmd/search.rs
• src/similarity.rs
• src/laya.rs

Included review availability: Your plan provides up to 10 included reviews per hour; 9 remain after this review.

---

📝 Walkthrough

Walkthrough

The changes remove Laya configuration, daemon support, and command and MCP integrations. They add Japanese query-term expansion and keyword handling. Search results retain lexical ordering.

Changes

Laya removal and Japanese search

Layer / File(s)	Summary
Remove Laya configuration and runtime .gitignore, README.md, scripts/laya_server.py, src/config.rs, src/laya.rs, src/main.rs, src/similarity.rs, src/util.rs	Removes Laya configuration, client and server code, semantic-duplicate refinement, and related documentation and tests. Config parsing now matches keys directly.
Remove Laya from command and MCP paths README.md, src/cmd/*, src/mcp.rs, tests/integration_test.rs	CLI and MCP add, search, and stats paths no longer connect to Laya or report its status. Search uses lexical ordering, project priority, and update time before truncation. The integration-test helper no longer sets LK_NO_LAYA by default.
Expand Japanese search terms and keywords README.md, src/db.rs, src/keywords.rs, src/main.rs, tests/integration_test.rs	Search expands Japanese sentence queries into eligible ASCII words, katakana terms, and normalized kanji compounds. Automatic keyword extraction uses up to eight base keywords and adds up to three Japanese terms. Tests cover query expansion and Japanese sentence search.

Priority: ⬇️ Low

Estimated code review effort: 3 (Moderate) | ~25 minutes

Change: Feature

Sequence Diagram(s)

sequenceDiagram
  participant SearchCommand
  participant search_entries
  participant query_terms
  participant SearchPaths
  SearchCommand->>search_entries: Submit query
  search_entries->>query_terms: Expand Japanese sentence terms
  query_terms-->>search_entries: Return expanded terms
  search_entries->>SearchPaths: Use expanded text for FTS, keyword, and LIKE queries
  SearchPaths-->>SearchCommand: Return matching entries

Loading

Merge Risk: ⚪ Minimal · up to 7fb2c

This change removes the optional Laya integration and adds model-free Japanese sentence search and keyword extraction. Japanese sentence queries are split into terms, and short kanji terms are still found through the existing fallback matching. English keyword queries are unchanged. Old Laya settings in config files are ignored without breaking loading. No blocking issues were identified.

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly summarizes the two main changes: removing Laya integration and adding Japanese keyword and query handling.
Docstring Coverage	✅ Passed	Docstring coverage is 80.00% which is sufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 35 functions across 8 files. (1 skipped: 1 …
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

📝 Generate docstrings

• Commit to this branch
• Create a new PR

🧪 Generate unit tests (beta)

• Commit to this branch
• Create a new PR

---

_{Comment @coderabbitai help to get the list of available commands.}