Semantic vector search over source-code repositories. Smritikosh parses source files with tree-sitter, identifies meaningful definitions, turns them into chunks, embeds those chunks, and stores the vectors in DuckDB.
FileSource FileRouter
(lists + reads) (extension -> language + strategy)
| |
+--------> discovery <+ select once, per file
|
| 3 gates: excluded dirs -> gitignore -> router
v
SourceFile path, language, content,
| has_tags_scm, strategy
v
[ file unchanged? ] --yes--> skip file file_hashes
| no
v
parser text -> tree-sitter AST
v
extractor AST -> definition captures (tags.scm)
v
chunker captures -> chunks (ast|section|regex)
v
[ chunk text seen? ] --yes--> reuse vector content_hash
| no
v
embedder chunk text -> vector
v
DuckDB nodes, vectors, file_hashes, memo_cache
Routing happens once, in discovery, and is carried on the SourceFile. Later
stages read language, has_tags_scm, and strategy off that object rather
than re-deriving them from the path.
Two skip checks make re-indexing cheap. A file whose SHA-256 is unchanged is skipped whole. A chunk is keyed by the hash of its own text, so editing one function in a file re-embeds only that function's chunk.
The extractor uses packaged tags.scm queries for Python, Java, Kotlin,
TypeScript, JavaScript, Markdown, and JSON. TOML bypasses extraction and uses
regex chunking. JSON is indexed except for known noise — lockfiles, minified
bundles, and fixture directories.
smritikosh/indexing— file router, discovery, parser, extractor, chunker, and pipeline orchestration.smritikosh/engine— memoization, tracking, batching, and concurrent pipeline helpers.smritikosh/ports— file-source, storage, vector-store, and embedder contracts.smritikosh/adapters— local-filesystem, DuckDB, and sentence-transformer implementations of those contracts.smritikosh/queries— packaged tree-sitter tag queries.
The extractor stays under indexing: it transforms internal pipeline data
(ParsedFile -> list[Capture]) rather than adapting an external system.
pip install smritikoshOr with uv, to get the CLI on your PATH without managing a virtualenv:
uv tool install smritikoshBoth uv and Poetry are supported. uv is recommended for local development — it resolves and installs the full dependency tree roughly 5× faster than Poetry, thanks to a Rust-based resolver and parallel downloads. (Measured on this project: uv locked 94 packages in ~46 s vs Poetry's ~3.5 min.)
# Install uv: https://docs.astral.sh/uv/getting-started/installation/
uv sync # install deps + project into .venv
uv sync --no-install-project # deps only (skip the editable install)# Requires Poetry >= 2.3
poetry install # install deps + project into .venv
poetry install --no-root # deps only# uv
uv run pytest
# Poetry / activated venv
make testBoth tools create a .venv in the project root and share the same Makefile
targets. Pass TOOL=uv or TOOL=poetry to override the default:
make install TOOL=uv
make lock TOOL=uv # regenerates uv.lock
make test # always uses .venv/bin/python directlyLock files are not committed. Smritikosh is a library, so installs resolve from
the version ranges in pyproject.toml; uv.lock and poetry.lock are local
artifacts that each tool regenerates on demand.
The explore commands expose the indexed repository to agents without reading
the source tree or taking a DuckDB write lock:
smritikosh explore tools
smritikosh explore info --db-path smritikosh.duckdb
smritikosh explore search \
"where is access control enforced" \
"authorization decision logic" \
--exclude-path 'tests/%' \
--db-path smritikosh.duckdb
smritikosh explore paths permissions \
--db-path smritikosh.duckdb
smritikosh explore chunks src/auth/permissions.py \
--db-path smritikosh.duckdb
smritikosh explore chunks src/auth/permissions.py \
--start-line 40 --end-line 52 \
--db-path smritikosh.duckdb
smritikosh explore text is_allowed \
--path src/auth/permissions.py \
--db-path smritikosh.duckdbThe two calls that answer most questions are search, which reports where the
answer lives, and chunks PATH --start-line N --end-line M, which prints that
source with line numbers. Without a line range chunks outlines what a file
defines rather than printing it; --full restores the whole-chunk dump, on
both commands.
Results come back as TOON — a tabular array that declares its columns once and then streams one row per hit:
[2]{path,start_line,end_line,score,chunk_kind,symbol}:
src/auth/permissions.py,59,81,0.434,class,PermissionChecker
src/auth/policy.py,176,193,0.436,method,evaluate
That costs roughly half of the equivalent JSON. --prose switches any command
to human-readable output. Source lines are exempt: TOON must quote any value
containing a colon, so a table of code lines costs more than the numbered text
it would replace, and chunks with a line range always prints numbered text.
Multiple semantic queries are embedded in one model call. Results are deduplicated and ranked by each chunk's best score across those queries; a hit that mostly repeats the lines of a better-scoring one is dropped. Path, text, and chunk commands do not load the embedding model.
explore tools returns a machine-readable command manifest and recommended
workflow. An agent instruction can therefore stay short:
Use the Smritikosh exploration CLI instead of Grep for code discovery.
Run `uv run smritikosh explore tools` to discover its commands.
See CONTRIBUTING.md for commit and PR conventions.
Built by Shantanu Vashishtha and Sarvesh Sawant.
Alpha. The pipeline runs end to end — index builds an index, search and
explore query it, and re-indexing is incremental at both the file and the
chunk level. Embedding runs locally on CPU through ONNX Runtime, so the only
network access is a one-time model download.
The CLI surface may still change before 1.0.