Speed up the local code verbs: prefilter refs, cache map's parse (#83)

[tarekziade]

Closes #83.

Parsing is the whole cost of the offline verbs — on huggingface/transformers @ d9890f6 (4,883 claimed files) reading them all is 0.6 s and parsing them is 16.4 s — so both levers skip parses and nothing else.

Lever 1 — refs uses the #45 prefilter

A file whose bytes lack the identifier cannot reference it. survey.py has had this since #45; refs takes a single symbol, so the argument carries over unchanged. The filter moved to walk.needle so both call sites share one.

	before	after	identical?
refs use_kernels (9/4883 files)	10.7 s	0.6 s	byte-for-byte
refs compute_default_rope_parameters (198)	~8 s	1.4 s	byte-for-byte
refs forward (1648)	10.9 s	7.1 s	byte-for-byte
refs config (3182)	9.7 s	9.3 s	byte-for-byte

The candidate counts reproduce the issue's table exactly. Two things the ordering protects: the refs-tier check still runs on every claimed file, so unsupported/complete cannot go quietly wrong; and a prefiltered file still counts as searched, because it was searched (#37).

needle now declines to filter — parses everything — when a name has no ASCII trailing identifier run. Falling back to the whole string made Vec<T> its own needle, which is in no file, so every file would be rejected and the verb would answer "nothing found" confidently.

Lever 2 — a .relore/ parse cache, for map only

map has no query symbol, so visiting everything is the verb. It memoises instead: 18.5 s → 1.0 s, and it is the only writer, because it is the one verb that already computes both halves of an entry.

The key is the content, and this cost a redesign

The issue proposed keying on git ls-files -s and reparsing only what git diff-files calls dirty. That was built and measured at 0.6 s instead of 1.0 s — and it is wrong. git's "clean" is a comparison of stat data, not of content. Reproduced on a throwaway repo:

$ git update-index --assume-unchanged m.py      # documented, widely used
$ printf 'def afterr():\n    pass\n' > m.py
$ git diff-files --name-only                    # (empty — git says clean)
$ relore map | tail -1
  before   function  ...  ./m.py:1               # ← the cache lying
$ RELORE_NO_CACHE=1 relore map | tail -1
  afterr   function  ...  ./m.py:1               # ← the tree

That is precisely the staleness hazard walk.py refused a cache over, wearing git's name. So every file is read and hashed on every call and only the parse is skipped. Reading is affordable because it was never the cost. git is still consulted for one thing that cannot be wrong: where the cache file lives.

Also: the provider is half the key (a pip uninstall tree-sitter-python moves every .py to ctags — a different parse of identical bytes); no reference positions are stored (map ranks on counts, refs recomputes every line it prints); defs is untouched, since the same code runs in relored against historical blobs where a working-tree cache is actively wrong; the directory writes a .gitignore of * so it never appears in your git status; and RELORE_NO_CACHE removes the path entirely.

refs does not read the cache. It was built, benchmarked and removed — it cost up to 1.3 s and never won. Once rule 1 took away read-skipping, all a cache can save is a parse, which is exactly what the prefilter already does for a single named symbol.

The benchmark — benchmarks/probes/code_lens.py

Client-side, beside page_cost.py rather than in bench/ (which is §10's server-side retrieval set). Both halves, because either speedup could be made faster by returning less.

  relore refs use_kernels        0.73s    2.3 KB     48
  git grep -n use_kernels        0.13s   18.4 KB    120
  grep -rn use_kernels          17.80s   92.6 KB    635
  relore map (cold)             18.91s    5.7 KB     42
  relore map (warm)              1.00s    5.7 KB     42
  relore map (no cache)         18.45s    5.7 KB     42

accuracy -- relore's .py locations against git grep's
  use_kernels            45 of 66     subset  (12 wider identifier, 9 prose, 0 unexplained)
  forward              8627 of 15355  subset  (4445 wider identifier, 2283 prose, 0 unexplained)
  config              77732 of 121697 subset  (35210 wider identifier, 8755 prose, 0 unexplained; 7 outside the walk)

identity -- cached output against RELORE_NO_CACHE:  all identical

Zero unexplained on every symbol, and use_kernels reproduces the issue's 45-of-66 with its 12 + 9 split exactly. Every grep line the lens omits is accounted for by one of three independently-established reasons: outside the walk (taken from walk.source_files itself, not guessed), a longer identifier containing the substring, or prose — decided by tokenize, the stdlib's own lexer, deliberately a different implementation from the tree-sitter grammar under test. A per-line regex could not see that a docstring's fourth line is inside a docstring and reported 867 of them as losses.

Anything unexplained, any location the lens reports that grep cannot see, or any cached/uncached divergence is printed in full and exits non-zero.

Notes for review

• walk.py's "Nothing is cached" paragraph is rewritten rather than deleted: "fast enough" was measured and was false; "staleness hazard" was true and survives as the constraint that produced rule 1.
• No version bump. Nothing on the wire, in a renderer or in a CLI flag changed, and these verbs never touch the daemon — so a bump would oblige a deploy for a local-only change. Happy to add one if you read RELORE_NO_CACHE as a client-visible surface.
• 1,222 tests pass on both dialects (+41 new), ruff clean.

🤖 Generated with Claude Code

[tarekziade]

Added a commit making the .relore/ convention binding in AGENTS.md rather than implicit in one module: one directory per tree, the daemon's copy on the PVC (a pod-local path evaporates on rollout and leaves a cache that looks like it works), the three rules this PR paid for, and the verbs that must stay exempt — status and inflight, whose freshness is the answer, plus defs for the separate reason that the same code runs in relored against historical blobs.

Filed the follow-on work rather than growing this PR:

• Caching by design: one .relore/ directory, and two layers that do not exist yet #85 — the convention plus the two layers that do not exist yet, with the five questions layer 2 has to settle first (what the validator is computed from, the version in the key, what must never be cached, the scoping cost of a tree-local directory, and the untrusted envelope crossing onto disk).
• The GitHub client makes no conditional requests: no ETag, no 304, so every re-read spends rate-limit budget #86 — relore/github/ has no conditional requests at all: no ETag, no 304. Aimed at rate-limit budget rather than latency, since a 304 is free against the limit and the limit is why the daemon exists.
• poll re-fetches every candidate thread in full, including ones it already holds unchanged #87 — poll calls fetch_thread unconditionally for every candidate, so the 60 s OVERLAP re-fetches threads it already holds. The module's "re-processing is free" is true in CPU and not on the wire.
• why re-runs the same git log -S pickaxes on every call, and two of three UI samples are the five-pass case #88 — why re-runs up to five git log -S pickaxes per call, and two of the web UI's three sample buttons are the degenerate licence-header case. Notes that fixing the candidate-word selection may be the better fix than caching it.
• Commented on copies/symbol take ~20s on transformers against a 30s client timeout, and a read timeout reports as "cannot reach" #43 — copies/symbol have had the prefilter since symbol takes ~20s warm and does not cache — 30–45× every other verb, against a 30s client timeout #45, so its ~20 s predates it and wants re-measuring; and the cache in this PR fits the daemon's long-lived clone better than it fits a developer's checkout, which symbol takes ~20s warm and does not cache — 30–45× every other verb, against a 30s client timeout #45 left open when it closed with the prefilter instead.