Suzume is a lightweight Japanese tokenizer for browsers and native apps. It is not a full morphological analyzer like MeCab: its primary goal is to split text into useful units for search, display, and application code. Unlike a boundary-only tokenizer, it also returns part-of-speech tags and lemmas.
Reach for it when you need to:
- Tokenize Japanese anywhere — use the same tokenizer in a browser, serverless function, Python process, or C/C++ application.
- Extract search terms — generate keyword tags with POS filtering, lemmas, and duplicate removal.
- Add your vocabulary — load application-specific words through a user dictionary.
📖 Documentation · 🧪 Live Demo · Getting Started
MeCab is a morphological analyzer designed for detailed, dictionary-based analysis. Suzume is a tokenizer designed around practical token boundaries. It uses character patterns and compact rules, keeping compounds and quantities together when that produces a more useful search unit.
Input: 経済成長 3人
MeCab: 経済 / 成長 3 / 人
Suzume: 経済成長 3人
Suzume still provides POS tagging and lemmatization, so applications can normalize inflected verbs and adjectives without adopting MeCab-style output. The outputs are intentionally different rather than drop-in compatible. See Differences from MeCab for examples, trade-offs, and known constraints.
npm install @libraz/suzume # JavaScript / TypeScript
pip install suzume # PythonFor C/C++ installation, native builds, user dictionaries, and all runtime options, see the documentation.
import { Suzume } from '@libraz/suzume'
const suzume = await Suzume.create()
const tokens = suzume.analyze('すもももももももものうち')
const tags = suzume.generateTags('東京の公園に行きました')
suzume.destroy() // optional immediate cleanupfrom suzume import Suzume
with Suzume() as sz:
tokens = sz.analyze("すもももももももものうち")
tags = sz.generate_tags("東京の公園に行きました")