Skip to content

Code grep and symbol mentions retain source coordinates - #1107

Merged
zlareb1 merged 3 commits into
mainfrom
fix/code-grep-source-spans
Nov 21, 2026
Merged

zlareb1 merged 3 commits into
mainfrom
fix/code-grep-source-spans

Conversation

@zlareb1

@zlareb1 zlareb1 commented Nov 21, 2026 •

Copy link
Copy Markdown
Member

Related: #1080

Changelog category

  • Bug Fix (user-visible misbehavior in a release)

Changelog entry

Code grep and symbol mentions report physical source line numbers when indexed chunks contain copied documentation or continuation headers. Results disclose partial coverage when compacted or incomplete chunks cannot establish exact coordinates.

Why

Chunk presentation text can prepend documentation that already appears inside a function. Counting every stored text line from the chunk's source start shifts literal hits and message matches, and can invent hits in synthetic text. Agents then read the wrong source location.

Use the retained-source mapping that definition bodies already use for literal grep, message matching and symbol mentions. This keeps their coordinates consistent and counts physical LF boundaries rather than Unicode separators inside source strings.

Scope and risk

The read paths share a source-line helper. Chunk queries carry kind so compacted class skeletons are omitted rather than assigned guessed coordinates. Grep discloses partial coverage; symbol mentions disclose truncation. Coverage validation runs off the event loop and obeys the line-scan size bound, including large retained chunks. A negative result from an omitted fragment is not proof of absence.

Existing unit fixtures with missing or inconsistent declared source spans are corrected to represent complete source rows. Their behavioral assertions remain, with scope boundaries corrected where the source spans were inconsistent. Regression tests cover copied documentation, continuation headers, Unicode separators, compacted classes and stored rows read through both SQL paths. Continuation pages and message fallbacks preserve warnings from every read leg. Unversioned cursor pools are rerun rather than replaying cached synthetic coordinates; callers are told to restart pagination for complete results. The analyzer statement corpus is regenerated.

Compatibility, rollout and rollback

  • Wire: no operations, fields or defaults are added or removed. Existing partial-coverage and truncation fields disclose unmappable chunks. Signed cursors and stored match layouts are unchanged; additive pool metadata is ignored by older readers. Current readers disclose a rerun for pools with another source mapping.
  • Operator steps: none. This is a read-side fix; extraction, extractor stamps, schemas and stored rows are unchanged. No reindex is required.
  • Rollback: revert the change. It leaves no migrations or generated data behind.

Tests

  • Focused Linux grep, symbol, observability, message and cursor tests: 355 passed.
  • Focused grep, symbol-body and documentation checks: 434 passed.
  • Source-coordinate integration regression against private ClickHouse: 1 passed.
  • Analyzer corpus against private ClickHouse: 12 passed.
  • An actual previous-build cursor reader accepts pools written with the additive metadata; current-reader regressions cover old and unsupported pool mappings.
  • Documentation and public-copy checks: 275 passed.
  • make format, make lint and make typecheck: passed.
  • Linux unit, coverage, benchmark, integration, mixed-version and image smoke gates: passed. Trigger acceptance is still running.
  • Fresh independent review of the final head: no findings.

AI assistance in the code

  • AI-generated: Codex wrote the fix and regressions, checked the affected read paths and validated returned coordinates against source files. Human review is pending.

  • I can explain every line of this diff and how it interacts with the rest of Loom.

Checklist

  • Formatting, lint, type checks, unit tests and applicable integration checks pass.
  • Generated analyzer statements are regenerated rather than edited by hand.
  • No migration, settings, plugin or frontend behavior changes.
  • Architecture documentation describes partial source coverage.
  • Public text passes the prose and public-copy checks.
  • Commits are signed off.

@vercel

vercel Bot commented Nov 21, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
loom-docs Ready Ready Preview Nov 21, 2026 11:18am UTC

Request Review

@zlareb1
zlareb1 force-pushed the fix/code-grep-source-spans branch 2 times, most recently from 83fe5a3 to 00b8d70 Compare November 21, 2026 09:30
Use retained source lines for literal hits, message matches and symbol mentions so copied documentation and continuation headers cannot shift file locations. Disclose unmappable fragments through existing coverage fields.

Signed-off-by: Lareb Zafar <lareb.zafar@clickhouse.com>
Carry partial coverage across both message fallback legs and cached pages. Rerun cursor pools without the current line mapping so cached synthetic positions are not returned.

Signed-off-by: Lareb Zafar <lareb.zafar@clickhouse.com>
Do not parse chunks beyond the existing line-scan bound when collecting coverage warnings. Keep that validation off the event loop for both literal and message reads.

Signed-off-by: Lareb Zafar <lareb.zafar@clickhouse.com>
@zlareb1
zlareb1 force-pushed the fix/code-grep-source-spans branch from 00b8d70 to 8a0dee0 Compare November 21, 2026 11:17
@zlareb1
zlareb1 merged commit 9b960bb into main Nov 21, 2026
56 of 57 checks passed
@zlareb1
zlareb1 deleted the fix/code-grep-source-spans branch November 21, 2026 12:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant