Skip to content

Our method as a draw.io diagram, with the work per record against embeddings - #149

Merged
mvsbm merged 1 commit into
mainfrom
diagram-our-method
Nov 17, 2026
Merged

mvsbm merged 1 commit into
mainfrom
diagram-our-method

Conversation

@mvsbm

@mvsbm mvsbm commented Nov 17, 2026 •

Copy link
Copy Markdown
Owner

What and why

Adds docs/diagrams/our-method.drawio and a rendered docs/diagrams/our-method.png, to explain our method next to an embedding-based approach.

Sections:

  1. The simple example: 10 balls, one tag, points on each line.
  2. The actual pipeline, one row per step: the data as tables, what the step does, its cost.
  3. Why we won't miss pairs: which tag still matches for each kind of difference, and what keeps a good pair from being dropped.
  4. How a customer with 3 or more records is found.
  5. Number of comparisons against full pairwise.
  6. Work per record, ours and embeddings (example model: EmbeddingGemma 2), filled in only as far as known; the missing numbers are listed as discussion points.

Open in draw.io with math typesetting on (Extras → Mathematical Typesetting) for the formulas.

Checklist (CONTRIBUTING.md)

  • Reviewed before merging: each section was rendered and looked at, and reviewed by the owner in previews
  • Unhappy paths tested at the real size: does not apply, no code changes (two files under docs/diagrams)
  • Every new limit names itself (shortcut: …), and every store that grows says what bounds it: does not apply, no code changes
  • Docs and diagrams describe the code as it is: rules, tag kinds, levels and steps taken from config/final.toml and er/run.py; numbers from runs on generated files

🤖 Generated with Claude Code

https://claude.ai/code/session_012o8JNBKSY1aREPbMQDG8g2

…line step by step, why it doesn't miss pairs, groups of 3+, and the work per record against embeddings

docs/diagrams/our-method.drawio (math typesetting on) and a rendered our-method.png. The work comparison fills in only what is known and lists the numbers still needed as discussion points.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012o8JNBKSY1aREPbMQDG8g2
@mvsbm
mvsbm merged commit b7c6277 into main Nov 17, 2026
6 of 7 checks passed
@mvsbm
mvsbm deleted the diagram-our-method branch November 17, 2026 05:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant