Repository navigation
Concepts pages: architecture, identity map, type safety, backends and performance #718
Description
Parent
What to build
Stale facts: architecture has a dated implementation-status note, links into docs/plans/, and an example (lambda t: t.active) that raises; backends says Postgres stores dict/list as JSON (it is jsonb) and has no time row; performance says the identity map lives for the connection and eager loading is on the roadmap (include() exists); type safety claims field-name completion inside query lambdas that does not happen. Every claim is checked against the code, headings become sentence-case noun phrases, and about 100 ADR numbers, codenames and test-file names give way to links where the reasoning helps.
Pages: concepts/architecture.md, concepts/identity-map.md, concepts/type-safety.md, concepts/backends.md, concepts/performance.md
Acceptance criteria
- Every fact on the page matches the code at the time of the PR; each High finding in the audit rows below is fixed, and each Medium/Low finding is fixed or answered in the PR body with the reason it was left.
- Each page is one kind (tutorial, how-to, reference or explanation) and opens with one sentence saying what it covers, then an example on the first screen (writing-for-humans rules 1–2).
- Each page uses one running example, declared once near the top and reused; each section shows only what changed (rule 3).
- Every code block has a lead-in sentence ending in a colon and a result sentence after it, and shows its result: printed output, or the asserted value inside the embedded snippet range (rules 7–8). Snippet
end:markers sit below the assert or print that holds the result. - Every copyable block either shows the
async with engines.session():wrapper or sits under a sentence that says it runs inside one and links to Connections › Sessions; the page's first block shows the wrapper. - Every inline Python block that does real work runs under the harness (
{test=run prelude=<name>}) or moves into an example script; a block left compile-only says why in its fence marker. - Every lambda passed to a query method names its parameter after the model in lowercase singular (AGENTS.md I-8);
python -m tests._docs_checksreports no violation for the owned files. - Every error check in an owned example fails when the expected error is not raised (
try/except/else: raise AssertionError, or the harness helper). - Field-declaring examples show both declaration styles as tabs, each backed by a runnable script (AGENTS.md I-7), and the twins use the one Annotated idiom recorded in the docs conventions.
- Headings are short sentence-case noun phrases naming the feature (rule 5); how-to titles name the goal (rule 24); no minimising words (rule 17); admonitions follow rule 20.
- No ADR numbers, issue numbers, codenames, dated or release-bound notes, or contributor-only references in page text; version markers are self-contained blocks at the end of a section.
- Reference material that has a home in Reference is linked, not repeated; each topic ends with a pointer to its reference entry (rule 4).
Files owned
- Modify:
docs/pages/concepts/architecture.md,docs/pages/concepts/identity-map.md,docs/pages/concepts/type-safety.md,docs/pages/concepts/backends.md,docs/pages/concepts/performance.md
Interfaces
- Consumes: docs conventions (AGENTS.md ticket (AGENTS.md: rewrite I-8 for the lambda-only query API, add a docstring invariant, record the docs conventions #703))
- Consumes: harness (harness ticket (Docs harness: run inline blocks with a prelude, run docstring examples, and check lambda names and error checks #704))
- Produces: nothing other tickets rely on
Test scenarios
- Happy path: the strict site build succeeds and every snippet range on the owned pages renders without leading indentation.
- Happy path: every owned example script runs to completion under
tests/test_docs_examples.py, and every inline block marked{test=run ...}runs. - Error path: deleting the
raisean owned example's error check expects makes that example's test fail. - Edge case:
python -m tests._docs_checksover the owned files exits 0; renaming one lambda parameter totmakes it exit 1 naming the file and line.
Verify
uv run --no-sync zensical build --clean --strict
uv run --no-sync pytest tests/test_docs_examples.py -q --no-cov -p no:cacheprovider -n 8
uv run --no-sync python -m tests._docs_checks docs/pages/conceptsBlocked by
- Docs site config: Zensical features, Markdown extensions, API reference options and a strict build #701, AGENTS.md: rewrite I-8 for the lambda-only query API, add a docstring invariant, record the docs conventions #703, Docs harness: run inline blocks with a prelude, run docstring examples, and check lambda names and error checks #704
Audit findings
Rows from the docs audit this ticket covers (file, line, rule, fix)
docs/pages/concepts/architecture.md
- Kind (rule 1): explanation (with reference-ish IR detail)
- Running example (rule 3):
User(introduced at l.169, after three diagrams that useUseralready) - Verdict: The layer story and trade-offs are good. It carries a dated, now-false "implementation status" note, a link into
docs/plans/, and an example predicate that raises at runtime. C
| Sev | Line | Rule | Finding | Suggested fix |
|---|---|---|---|---|
| H | 43–50 | facts | The "Implementation status (2026-06-26)" note says the runtime DDL path "re-derives its own SchemaIR" and "src/schema.rs uses its own canonical type system rather than the shared ferro-ddl-lowering crate". AGENTS.md I-1 says the opposite today: one planner (ferro_migrate::plan_from_ir), the create pass is ferro_migrate::render_create_table, and types come from ferro_ddl_lowering::resolve_column_storage. It also contradicts this page's own l.209 ("all derive from the Python-compiled SchemaIR modelset"). |
Delete the note. If the history matters, it belongs in an ADR, not a concepts page. |
| H | 114 | facts, 24 | The SchemaIR bullet repeats the stale "re-derives its own SchemaIR" claim and links "Phase 8.5" to docs/plans/2026-06-19-001-ir-first-roadmap.md. That is a contributor planning file; the repo's agent rules say plans live in Issues. |
Drop the parenthetical and the link. |
| H | 107 | facts, I-8 | User.where(lambda t: t.active) raises TypeError: where() predicate callable must return QueryNode, got FieldProxy (verified). |
User.where(lambda user: user.active == True).order_by(lambda user: user.age, "desc").limit(10). |
| H | 169–193 | I-7 | The tabbed User model is inline, and no test compile-checks it. tests/test_docs_examples.py globs only docs/examples/*.py, so the I-7 requirement that "inline blocks … are compile-checked" is not met. The same applies to identity-map.md. |
Move the model into docs/examples/concepts_user.py and _annotated.py, and include it with --8<--. |
| M | 107, 142, 158 | I-8 | lambda t: in three predicate examples, including the sequence diagram (User.where(lambda t: t.age > 18)). |
lambda user:. |
| M | 5–7 | 2 | "## Overview" heading, then a sentence about two pipelines. The first example (a mermaid chart of internals) is not something the user wrote. | Lead with the user-facing anchor (I-11): the 6-line User model and await User.where(...).all(), then "here is what happens in each layer". |
| M | 206–207 | 20, facts | "migrate_updates=True (0.11.0)" and "(0.11.0)" are inline version markers. "Runtime updates are planned from SchemaIR diffing (ferro-migrate)" restates the schema guide. |
Drop the inline versions (or move them to a version note), and link to Auto-migrate instead of restating the flags. |
| M | 209 | 1, 15 | One 110-word paragraph names three test files and ferro-ddl-lowering tokens. That is contributor detail on a user page. |
Cut it to "Every door plans with the same planner, so they agree byte for byte; see Schema management." Leave the test-file names to docs/solutions/. |
| M | 99, 131, 165, 211, 236 | 5 | Title Case headings: "The Layers", "Life of a Query", "Model Registration", "Async Architecture", "See Also". | Use sentence case ("Life of a query"). "The Layers" → "Layers". |
| L | 87–97 | 7 | A text block of seven "X owns Y" lines has no lead-in or result sentence. |
Make it a two-column table (layer, owns) with a lead-in. |
| L | 118, 223 | 3 | "The GIL is released while Rust waits" is stated three times (l.118, 145, 223) and again in performance.md. | State it once and link. |
docs/pages/concepts/identity-map.md
- Kind: explanation with how-to sections (evict, opt out, tests)
- Running example:
User(id, username, email), consistent throughout. Good. - Verdict: Strong page: a clear guarantee, runnable-looking examples, and assertions as output. It has some minor history and heading issues, and its inline examples are untested. B
| Sev | Line | Rule | Finding | Suggested fix |
|---|---|---|---|---|
| H | 27–91 | I-7, 9 | The tabbed example is inline and untested (see the architecture row). | Back it with docs/examples/identity_map.py and _annotated.py. |
| M | 5, 23, 95, 116, 141, 156, 193, 205 | 5 | Headings are Title Case. Some are slogans or questions in form: "What It Is", "Why It Matters", "No Session, No Identity", "The Guarantee: Refresh-on-Load". | "Weak-reference map", "Refresh on load", "Sessionless operations", "What enters the map", "Eviction", "Turning the map off", "Identity map in tests". |
| M | 191 | 1, 2 | "this is unchanged from before weak-valued maps and refresh-on-load existed" is history the reader does not need. | Delete the clause. |
| M | 136 | 8 | await User.get(created.id) is shown as "Raises RuntimeError" in a comment, without the message. |
Show the exception line: RuntimeError: No database route for this operation. Open a session (…) (the text from state.py:47). |
| L | 93 | 14 | "Merging old and new field values was considered and rejected" uses the passive voice. | "Ferro does not merge old and new values: a merge policy is how stale-read bugs survive." |
| L | 177 | facts | evict_instance("User", str(user.id)) matches the signature (pk: str). |
OK. The str() cast deserves a note ("pk is the primary key as a string"). |
docs/pages/concepts/type-safety.md
- Kind: explanation
- Running example:
User(username, age), thenAccountfor validators - Verdict: The overview is good, but one IDE claim is stale (lambda proxies give no field completion) and two predicates break I-8. B−
| Sev | Line | Rule | Finding | Suggested fix |
|---|---|---|---|---|
| H | 64 | facts | "Field names complete when you type User. inside a query expression." Predicates are lambda-only, and the proxy's attributes are FieldProxy[Any] through __getattr__, so the IDE offers no field names on user. inside a lambda. The page's own l.102 and query-typing.md say so. |
Delete the bullet, or replace it with: "Inside a lambda predicate, attribute names are not completed yet; a typo raises AttributeError at build time (see Typed Query Predicates)." |
| M | 51–53 | I-8 | lambda t: ×3 |
lambda user:. |
| M | 35 | 8 | # age: Input should be a valid integer is not what print(e) prints. Pydantic prints a multi-line message ("1 validation error for User\nage\n Input should be a valid integer, unable to parse string as an integer …"). |
Show the real first lines. |
| M | 102 | 15, 3 | The Limits paragraph uses user.age >= "eighteen" but the earlier examples use t. PEP 827 is explained here and twice more in query-typing.md. |
Keep one sentence and link to query-typing.md#…; drop the PEP detail here. |
| L | 5, 38, 58, 68, 74, 100 | 5 | Title Case headings ("Pydantic at the Core", "The Rust Boundary"). | Sentence case: "Pydantic models", "Rust boundary". |
| L | 70 | 1 | src/ferro/_core.pyi is a repo path, so readers who installed the wheel do not have src/. |
"ferro/_core.pyi". |
docs/pages/concepts/backends.md
- Kind: mixed: reference (type table), explanation, how-to (troubleshooting)
- Running example: none
- Verdict: Useful page, but the type table is wrong for JSON on Postgres, omits
time, and the raw SQL link goes to the wrong guide. C
| Sev | Line | Rule | Finding | Suggested fix |
|---|---|---|---|---|
| H | 40 | facts | "dict / list → PostgreSQL JSON". Derived JSON storage is jsonb on Postgres (ADR-0005, canonical_from_parts l.381–388 in crates/ferro-ddl-lowering/src/lib.rs); db_type="json" is the opt-out. |
"jsonb (db_type="json" for plain json)". |
| H | 32–43 | facts | The table omits datetime.time (time on both dialects, canonical_from_parts l.379). int → "INTEGER/BIGINT" is ambiguous: the derived type is integer, and bigint comes only from db_type. float → "double precision" is correct. |
Add a time row, write integer (use db_type="bigint" for 64-bit), and generate the table from ferro_ddl_lowering or the db_type parity test so it cannot drift again. Link to the db_type token list in api/fields.md. |
| H | 51 | 24 | "raw SQL" points at the queries guide; the raw SQL guide is ../guide/raw-sql.md. The same mismatch appears at l.107 ("Queries — the query builder and raw SQL escape hatch"). |
Link ../guide/raw-sql.md. |
| M | 85–101 | 1 | The troubleshooting how-to sits inside a concepts page. | Move it to the FAQ or a how-to, or keep it with a Tip linking out. |
| M | 25–26 | 20 | The Tip admonition holds feature documentation (ferro_search_path). Rule 20 says a Tip is for a pointer elsewhere. "a private ferro_search_path URL parameter" ("private" is unclear). |
Make it a short section "PostgreSQL schema isolation" with a URL example: postgresql://…/app?ferro_search_path=test_42. |
| M | 3, 30, 45 | 17, 18 | "the practical difference is mostly on the PostgreSQL side"; "In practice SQLite's type affinity makes declared-type drift mostly cosmetic" (l.81) uses hedges without a condition. | State the conditions: "SQLite stores any value in any column, so a declared-type mismatch does not reject writes." |
| L | 21 | 16 | "Ferro's test matrix runs against Supabase-managed databases" is a claim to verify. | Confirm it in CI config, or drop it. |
| L | 1, 5, 28, 47, 85 | 5 | Title Case headings; the page title "Backends" against nav "Database Backends". | "Database backends", "Type mapping", "Backend differences", "Troubleshooting". |
docs/pages/concepts/performance.md
- Kind: explanation and how-to ("Getting the Most Out of Ferro")
- Running example:
User(but withemailsometimes required, sometimes omitted) - Verdict: Honest framing, but two claims are stale and contradict other pages: the identity map caching and eager loading on the roadmap. C
| Sev | Line | Rule | Finding | Suggested fix |
|---|---|---|---|---|
| H | 80 | facts | "Repeated fetches of the same row return the cached instance rather than re-hydrating… every hydrated instance stays cached for the connection's lifetime, so long-running jobs… should paginate and evict". identity-map.md (l.3, 18, 21, 25) says the opposite: the map holds weak references, every fetch runs a query and refreshes in place, entries die with the instance, and the map is session-scoped, not connection-lifetime. | "The identity map holds weak references and is scoped to a session, so it does not grow a long job's memory; it saves object identity, not queries. identity_map=False drops the dedup work for ETL-style reads." |
| H | 82 | facts | "Eager loading (prefetch_related-style) is on the roadmap; until then, restructure hot paths…". The example post.author is a forward FK, which include(lambda post: post.author) populates in the same statement today (roadmap.md l.9; api/queries.md l.13). Only BackRef/M2M population is future work. |
"Use Post.where(...).include(lambda post: post.author) to load authors in the same query. Reverse and M2M collections are not populated yet (roadmap); batch them by id with in_()." |
| M | 52, 58, 76 | I-8 | lambda t: ×3. |
lambda user:. |
| M | 39–43, 91 | 9, 3 | User.create(username=…) without email, while l.13 builds User(username=…, email=…). One of the two is not valid for the same model. No model is shown. |
Show the User model once (from a docs/examples file) and keep the calls consistent. |
| M | 9, 19, 27 | 16 | "largely disappears", "far less time", "in the same neighborhood": unmeasured hedges on a page that says it publishes no numbers. | Say what moves to Rust, and leave magnitude to the reader's benchmark. |
| M | 86–100 | 8 | The benchmark has no shown output. | Add a sample output line, labeled as a sample from one machine, or say "prints two timings". |
| L | 5, 23, 33, 84 | 5 | "Where It Doesn't" and "Getting the Most Out of Ferro" are not noun phrases; Title Case. | "Where Rust helps", "Where it does not help", "Tuning checklist", "Benchmarking". |
| L | 38, 42 | 13 | "Slow:"/"Fast:" comments without SQL. | Show "N × INSERT" against one "INSERT … VALUES (…), (…)" (rule 13). |
Activity
0x054 commentedon Oct 19, 2026
Status: verify-passed
Worker: worker-718
Branch:
sdd/718-conceptsPR: #746
Verify: passed 02ebf8c
Scoped re-review by reviewer-718: Spec PASS, Quality PASS.
Both Important findings ADDRESSED (type-safety.md:3, performance.md:130); both Minor fixed.
The new
InterfaceErrortext in backends.md matches a fresh run. Base unchanged since the rebase (b0e5348), so no trial merge was needed.docs(concepts): concept pages checked against the code, one running e…
0x054 commentedon Oct 19, 2026
Merged in #746 into
feat/docs-rewrite