Activity

1. dougc95

   added

   enhancementNew feature or requestNew feature or request

   on Oct 12, 2026
2. dougc95

   self-assigned this

   on Oct 27, 2026
3. dougc95 commented on Oct 28, 2026

   

   dougc95

   on Oct 28, 2026

   ContributorAuthor

   More actions

   Ceiling measured. Decision record below for maintainer review; nothing is implemented yet.

   • Measured with a new ignored testcontainers harness (v1_history_copy_ceiling, in the test-only PR test(mongodb): measure the v1 history copy's transaction ceiling #2252) on mongo:7.0:
     • At a 2 GB WiredTiger cache, offloaded document shape, one lone transaction holds 69,200 Bundle-create entries without the version-1 resource_history copy and 34,300 with it.
     • That is 2.017x at insert and at least about 2.13x at commit, and about 2x at every cache size from 0.25 to 2 GB.
     • On the native shape after mongodb: write search_index rows after commit for transaction Bundles of 4,000+ entries, so they fit WiredTiger's per-transaction cache limit #1975 the gain is 1.495x.
     • These are raw inserts of fixture-shaped documents. A real process_transaction control (resource + history only) aborted 3.7% later than the raw model.
   • Recommendation: option (c) in its partial form, for transaction Bundles of at least N entries (default 4,000):
     • v1 history is written after commit, through its own ledger, a live step and a repair modelled on mongodb: write search_index rows after commit for transaction Bundles of 4,000+ entries, so they fit WiredTiger's per-transaction cache limit #1975's;
     • a marker-gated instance-level read fallback;
     • a mandatory "store v1 before overwrite" rule, with the upsert issued first.
   • Implementation waits for a maintainer to agree to this record, in particular open question 1. Under (c) partial, type and system history, their counts and the dashboard lag a large Bundle's creates until the post-commit step or the repair runs. Requiring no window at all means (b).

   ---

   #1976 decision record: the version-1 history copy in MongoDB transaction Bundles

   Status: proposed. A maintainer needs to agree to it before implementation starts. This record changes no product code. It adds only an ignored measurement harness (section 2).

   Code references are at origin/main 6f62064, because the issue's line numbers are out of date. Unless a path says otherwise, files are under crates/persistence/src/backends/mongodb/.

   Labels: [M] means measured, or read from code or logs. [I] means inferred.

   Summary

   • Measured ceiling [M]. The harness does raw insert_many calls of fixture-shaped documents in one lone transaction on mongo:7.0, in the offloaded document shape (mongodb-elasticsearch). The only real-path calibration covers resource + history (section 2.1). A resource-only transaction holds this many more entries than resource + history:
     • 2.017x at insert time, 2 GB cache: 69,200 vs 34,300 entries, 3 runs each.
     • At least about 2.13x at commit, 2 GB: resource-only committed at 68,700; resource + history committed at 31,900 and failed at 32,200 [M/I].
     • About 2.0x at every cache size from 0.25 to 2 GB. The entries ratio is 2.006-2.023 (3 runs per variant at 0.5-2 GB, 1 run at 0.25 GB), and 1.997-2.032 across all estimators and invocations.
   • Offloaded leg. The 87.0 MB / 49,027-entry Bundle needs about 1.70x. The projected margin is roughly 5-10% (section 2.4), so the change is worth doing. The benchmark is the acceptance check [I].
   • Native leg after mongodb: write search_index rows after commit for transaction Bundles of 4,000+ entries, so they fit WiredTiger's per-transaction cache limit #1975 (native_deferred shape): the gain is 1.495x [M]. That helps, but on its own it is not enough for mongodb 1000/1000 [I] (section 2.4).
   • Recommendation: option (c) in its partial form, for transaction Bundles at or above an entry threshold:
     • v1 history is written after commit, through a ledger, a live step and a repair modelled on mongodb: write search_index rows after commit for transaction Bundles of 4,000+ entries, so they fit WiredTiger's per-transaction cache limit #1975's, with a ledger collection of its own;
     • a cheap, marker-gated read fallback serves the instance level: vread, list_versions, instance history and its count;
     • a mandatory "store v1 before overwrite" rule applies to every overwriter. The v1 upsert is issued before the update that clears the marker.
   • The maintainer's key decision:
     • Instance-level reads stay identical to today at all times.
     • Type history, system history, their counts and the dashboard figures are not identical while the post-commit step runs, or after a crash until the repair runs.
     • During that window the resource is already readable and searchable. A _since poller that moves past the commit misses those v1 entries permanently, because they carry the create's timestamp.
     • If that is not acceptable, choose (b) instead (section 4).

   1. Context

   What is in the transaction today [M]:

   • create_resource_in_bundle_transaction (storage.rs:5519) builds the resources document and the version-1 resource_history document from the same payload (:5552-5580), with created_at = last_updated = now (:5546).
   • The history document travels as StagedCreate.history (bundle_creates.rs:84-85).
   • send_create_group sends it as a second ordered insert_many per chunk of 100, right after the resources one (bundle_creates.rs:294, :333-348).
   • The chunk byte cap counts the payload twice (bundle_creates.rs:108).
   • PUT-create in a Bundle uses the same function.

   Changes since the issue was filed:

   • fix(persistence): write a large MongoDB Bundle's search rows after commit #2030 (Refs mongodb: write search_index rows after commit for transaction Bundles of 4,000+ entries, so they fit WiredTiger's per-transaction cache limit #1975) moved search rows out of the transaction for native Bundles of 4,000 entries or more. It added:
     • a resource marker (search_rows_pending);
     • a ledger collection (search_index_pending);
     • a post-commit step that finishes before the Bundle response (storage.rs:4800-4823);
     • a repair task.
   • perf(mongodb): stage transaction Bundle creates and send them in chunks #2144 (Refs mongodb: transaction Bundle creates spend 3 of 6–7 round trips per entry on guaranteed no-ops — drop them and batch inserts per chunk #1977) chunks the creates.

   Latest benchmark: run 39657850695 (main @ e0bd5cb, 2026-10-21, 2 GB cache). These figures come from that run as summarized for this workflow. Its logs were not re-read for this record [M].

   Leg	Bundles OK	Failing Bundles	Abort position
   mongodb-elasticsearch	996/1000	The 4 largest: 33,562 / 35,593 / 43,966 / 49,027 entries	Entry 28,728-31,763
   mongodb (native)	989/1000	11	Entry 19,215-21,100

   • On the offloaded leg, the transaction holds only the resource and its history copy, so that leg is this issue's target.
   • Why the native deferred path caps lower is being investigated under mongodb: write search_index rows after commit for transaction Bundles of 4,000+ entries, so they fit WiredTiger's per-transaction cache limit #1975.

   From the issue (run 37986575337) [M]:

   • Offloaded aborts fall at entries 29.9k-33.0k, at 51.1-56.4 MB of cumulative resource JSON.
   • The four failing Bundles carry 56.0, 63.6, 82.5 and 87.0 MB.

   Gain needed, measured against the conservative 51.1 MB abort [I]:

   Failing Bundle (MB of resource JSON)	Gain needed
   56.0	1.10x
   63.6	1.25x
   82.5	1.61x
   87.0 (49,027 entries)	1.70x

   Premise correction. The issue assumed both collections have the same index count (4 + _id each). At HEAD they do not [M]:

   Collection	Index keys per offloaded document
   resources	6: _id, idx_resources_identity, idx_resources_type_scan, idx_resources_updated, idx_resources_live_tenant, idx_resources_missing_params
   resource_history	5 (schema.rs:388-426)

   • idx_resources_missing_params is not partial, so it also indexes the absent field as null (missing_index_builder.rs:28-42).
   • idx_resources_missing_params_fp is partial and skips these documents (:47-58).
   • MissingIndexBuilder runs whenever missing_metadata is on, whether or not search is offloaded (backend.rs:1323-1343) [M]. The benchmark's offloaded leg therefore very likely has the index [I]. This was not checked on the benchmark host.

   2. Measurement

   2.1 Method

   Harness.

   • File: crates/persistence/tests/mongodb/v1_history_ceiling.rs.
   • Test: mongodb_tests::v1_history_ceiling::v1_history_copy_ceiling, #[ignore]d.
   • Added on branch docs/1976-mongodb-v1-history-decision, with a 5-line module registration in tests/mongodb_tests.rs (4 lines plus a blank separator). No src/ change.
   • Run command: cargo test -p helios-persistence --features mongodb --test mongodb_tests v1_history_copy_ceiling -- --ignored --nocapture --test-threads=1.
   • The HFS_TEST_V1_CEILING_* settings are documented in the file.

   Database setup.

   • Each invocation starts its own mongo:7.0 single-member replica set (7.0.43). Flags: --wiredTigerCacheSizeGB=<N>, --oplogSize 1024, transactionLifetimeLimitSeconds=900 [M].
   • Document templates:
     • One real 267-entry transaction Bundle runs through process_transaction. It is the US Core fixture crates/hfs/tests/inferno/uscore_bundle_patient_85.json.
     • Every resources document and its v1 resource_history document are read back.
     • Each pair is asserted to be version "1" with identical data.
     • These are the exact shapes HFS writes: on average 1,341 B of compact JSON per entry and 1,662 B of BSON per document. Offloaded resources documents have no missing_params [M].
   • The measurement database is created through MongoBackend::new + initialize() with Inline index builds, so the real indexes exist. Its V1_CEILING_SETUP line lists them [M].

   Each run [M]:

   • fsync, then a 5 s settle.
   • One session transaction, insert_many in chunks of 100 (the production chunk size), UUIDv7 ids, a fresh tenant, inserting until the server aborts.
   • Two variants:
     • resource: the resources insert only.
     • resource_history: the resources insert, then the history insert, per chunk, in production order.
   • 3 repeats per variant, alternating which variant runs first.

   Additional runs [M]:

   • Commit probe: bisects the largest entry count that commits.
   • Control: one real lone Bundle of 148 fixture copies through process_transaction.
   • native_deferred shape: native documents with missing_params, missing_params_fp and the search_rows_pending marker. This is the native leg after mongodb: write search_index rows after commit for transaction Bundles of 4,000+ entries, so they fit WiredTiger's per-transaction cache limit #1975.
     • A deferred Bundle create still writes missing_params on the resources document inside the transaction (storage.rs:5552, :2549-2556).
     • Deferral only skips the search rows (:6847-6865) [M].

   Limits:

   • The harness does raw inserts: no search rows, no ledger, no Bundle-path CPU.
   • It cycles one fixture at about 1.34 KB of JSON per entry. The failing Synthea Bundles average about 1.77 KB per entry.
   • The only real-path check (the control) covers the resource + history shape. A resource-only control is impossible, because the backend has no switch to skip the v1 write. So the resource-only half of the ratio has no real-path calibration.
   • The host was loaded: load average 5-16 at run starts, with sibling builds running. Even so, the offloaded spread across repeats was 0.6% or less [M].
   • docs in V1_CEILING_RUN also counts the resources half of a chunk whose history insert failed (for example entries=34300 docs=68700). No figure in this record uses docs [M].

   2.2 Results [M]

   Cache	Shape	Variant	Runs	Entries median (min-max)	JSON MB median	Dirty B/entry	End codes	Gain: entries / json / dirty
   0.25 GB	offloaded	resource	1	8,700	11.68	3,174	388 x1	2.023 / 2.027 / 2.030
   0.25 GB	offloaded	resource_history	1	4,300	5.76	6,444	388 x1
   0.5 GB	offloaded	resource	3	17,300 (17,300-17,300)	23.21	3,121	388 x3	2.012 / 2.012 / 2.017
   0.5 GB	offloaded	resource_history	3	8,600 (8,600-8,600)	11.54	6,297	388 x3
   1 GB	offloaded	resource	3	34,500 (34,500-34,600)	46.27	3,116	388 x3	2.006 / 2.006 / 2.032
   1 GB	offloaded	resource_history	3	17,200 (17,100-17,200)	23.07	6,333	388 x3
   2 GB	offloaded	resource	3	69,200 (69,100-69,200)	92.80	3,103	388 x3	2.017 / 2.017 / 2.017
   2 GB	offloaded	resource_history	3	34,300 (34,300-34,400)	46.00	6,261	388 x3
   2 GB (commit-probe invocation)	offloaded	resource	1	68,700	92.13	3,140	388 x1	1.997 / 1.997 / 2.001
   2 GB (commit-probe invocation)	offloaded	resource_history	1	34,400	46.13	6,283	388 x1
   2 GB (control invocation)	offloaded	resource_history	1	34,300	46.00	6,276	388 x1
   2 GB	native_deferred	resource	3	32,300 (31,700-32,300)	43.31	6,664	388 x3	1.495 / 1.495 / 1.491
   2 GB	native_deferred	resource_history	3	21,600 (21,600-21,600)	28.97	9,933	388 x3

   How to read the table:

   • Dirty B/entry is the median of tracked dirty bytes in the cache at the last acknowledged chunk, divided by the entry count.
   • Gain shows the harness's entries_ratio, json_bytes_ratio and dirty_per_entry_ratio.
   • The 0.25 GB row is a single run per variant.
   • Every run ended with code 388 (too_large). No run ended with 112 (eviction), 365 or any other code.

   Commit ceiling at 2 GB, offloaded (v1c-2-commit.log) [M]:

   Variant	Probes	Largest that committed	Smallest that failed	Failure
   resource_history	8	31,900	32,200	388 at phase=commit (also at 34,400)
   resource	1	68,700	none found	none

   • The resource + history commit ceiling lies in [31,900, 32,200).
   • For resource, the single probe at the scan ceiling committed. Its commit ceiling is therefore only bounded: at least 68,700, and at most about the 69,200 scan median.
   • The gain at commit is therefore at least 68,700 / 32,200 = 2.13x, and at most about 69,200 / 31,900 = 2.17x [I].
   • Both bounds rest on single probes, run at load averages of 7.5-12.6.

   Calibration at 2 GB, offloaded (v1c-2-control.log) [M]:

   • A real lone Bundle of 39,516 entries (148 fixture copies) went through process_transaction.
   • It failed with TooLargeForCache at entry 35,584.
   • The raw resource_history scan in the same invocation stopped at 34,300.

   2.3 Reading

   • The ceiling is a dirty-bytes budget.

     • At the last good chunk of every run, tracked dirty bytes were about 10% of the cache [M]:

       Cache	Tracked dirty bytes at the last good chunk
       0.25 GB	27.6 MB
       0.5 GB	53.7-54.7 MB
       1 GB	106.9-109.0 MB
       2 GB	214.1-216.1 MB (2 GiB x 10% = 214.7 MB)

     • That matches the issue's reading of eviction_updates_trigger [I].

     • In the offloaded shape, the history copy costs about 3.16 KB of dirty bytes per entry (6,261 - 3,103), and the resources document costs about 3.10 KB. Each is about half the budget [I, derived].

   • The gain does not depend on cache size. Ceilings double with each doubling of the cache. The entries ratio stays at 2.006-2.023, and at 1.997-2.032 across all estimators and invocations [M].
   • The raw model tracks the real Bundle path.
     • The real Bundle got 3.7% further than the raw scan (35,584 vs 34,300 entries), so the raw figure is slightly conservative [M]. This calibration exists for the resource + history shape only.
     • In entries, the raw resource_history ceiling (34.3k at insert, 31.9k-32.2k at commit) is close to the benchmark's offloaded aborts. Those were 28.7k-31.8k in run 39657850695 and 29.9k-33.0k in run 37986575337.
     • In bytes it is lower (46.0 MB vs 51.1-56.4 MB), because the fixture's documents are smaller and per-document overhead weighs more.
     • The gain is a ratio of two variants over the same documents, so it should carry over to the benchmark corpus [I].
   • The commit itself lowers the resource+history ceiling by about 6.1-7.3% [M/I]. That is the [31,900, 32,200) bracket against the 34,300-34,400 scan. It fits mongodb: a TransactionTooLargeForCache (388) at commit answers 500 transient instead of too-costly #1991's point that the commit's own writes count. The resource-only commit penalty was not characterized, because its one probe committed at 0.7% below the scan median.

   2.4 Applying the decision rule [I]

   Offloaded leg.

   • The measured gain is 2.017x at insert and at least about 2.13x at commit. It clears all four thresholds: 1.10, 1.25, 1.61 and 1.70x.
   • The real implementation also adds cost to the transaction:
     • a marker field per document, about 60 B of BSON, not indexed;
     • ledger refs: about 1-3% of the 2 GB budget for 49k entries, depending on type and id length.
     • The expected net gain is therefore about 1.9-1.95x.
   • Projected, applying the 7.3% commit penalty to both variants as a conservative assumption:
     • Entries, from run 39657850695's lowest offloaded abort, 28,728:
       • x 2.017: 57.9k at insert, at least about 53.7k at commit, 9.6% above 49,027;
       • x 1.95 net: at least about 51.9k at commit, about 6% above 49,027.
     • Bytes, from run 37986575337's lowest abort, 51.1 MB:
       • x 2.017: at least about 95.5 MB at commit;
       • x 1.95 net: at least about 92.4 MB at commit;
       • both against 87.0 MB.
   • Host load moves aborts by about 1.3k entries between runs (fhir-benchmark: the comparison matrix marks host contention only on postgres crud; legs run on a saturated host look clean #1953). The projection gives a margin of roughly 5-10%, not a guarantee. The acceptance check remains the benchmark.
   • If the margin proves too thin, the next lever on this leg is the resources document's own index keys: idx_resources_missing_params covers offloaded documents too (section 1, follow-up).

   Native leg after #1975.

   • The gain is 1.495x [M]. Applied to native aborts at 19,215-21,100, the ceiling becomes about 28.7k-31.5k entries. That is still below the 33,562-49,027-entry Bundles.
   • The native resources document costs 6,664 dirty bytes per entry, against 3,103 offloaded [M]. Its extra fields (missing_params as a multikey array in idx_resources_missing_params, missing_params_fp, and the marker) are the inferred reason. No run varied them alone.
   • The raw native_deferred resource_history ceiling (21,600) sits near the benchmark's native aborts (19.2k-21.1k).
     • That is consistent with the native cap coming mostly from the write set itself.
     • It does not exclude other in-transaction writes. The same raw model overstates the offloaded benchmark's abort entries by 8-19%. The harness also writes no ledger, no inline rows for ifNoneExist target types and no Bundle-path work.
     • It is offered to mongodb: write search_index rows after commit for transaction Bundles of 4,000+ entries, so they fit WiredTiger's per-transaction cache limit #1975 as a hypothesis, not a finding.
   • So this issue is necessary but not sufficient for native 1000/1000.

   Worth doing: yes. For the target leg the gain is about 2x, not well under 2x.

   3. Options

   What every option needs

   A writer rule [M from code read]:

   • Once v1 is no longer written in the transaction, resources.data is the only copy of v1 until the first overwrite.
   • Unlike search rows, v1 cannot be rebuilt from current state.
   • So every option needs this rule. If the pre-image carries a "v1 not yet in resource_history" marker, the writer:
     1. first upserts the v1 row built from that pre-image: a full replacement keyed on the unique idx_history_identity, not $setOnInsert (section 6.1);
     2. then issues its usual version-guarded update, which also $unsets the marker.

   Why the upsert comes first. The non-Bundle overwriters do not always run in a transaction, even on a replica set [M]:

   • begin_best_effort_multi_write_session (storage.rs:1500-1523) sets session = None and writes without a transaction whenever start_session, hello or start_transaction fails.
   • update_attempt (:2801), soft_delete_attempt (:3144) and restore_attempt (:3367) use it.
   • In that mode, update_attempt updates resources (:2882-2888) and only afterwards inserts history (:2939-2946), with no atomicity.
   • If the v1 write came after the marker-clearing update, or in the same update, a failure between the two steps would lose v1 for good [I]. The marker would be cleared, data overwritten, and v1 never stored.
   • Upsert-first is idempotent, so it is safe both inside and outside a transaction.
   • The alternative is to fail with a retryable error when the pre-image is marked and no transaction is active (open question 9).

   The two invariants this gives [I]:

   • Marker set implies that data, version_id, created_at and fhir_version are still v1's, because every writer clears the marker in the same update that overwrites them.
   • Marker cleared implies that the v1 row is stored, because every writer stores v1 before, or atomically with, clearing the marker.

   Where the rule applies. Every writer that changes data, version_id or last_updated:

   • update_attempt (storage.rs:2789)
   • soft_delete_attempt (:3132)
   • restore_attempt (:3355)
   • write_next_version_in_bundle_transaction (:5757)
   • delete_resource_in_bundle_transaction (:5873), which is also reached from delete_with_match_resource_in_bundle_transaction (:6008)
   • bulk ingest's update path (bulk_ingest.rs:673, :751-790)

   Other resources writers [M]. A grep of update_*, replace_*, find_one_and_* and delete_* calls under backends/mongodb finds only:

   • purge, purge_all and purge_tenant_data;
   • metadata-only rewrites that never touch data or version_id:
     • mongodb: write search_index rows after commit for transaction Bundles of 4,000+ entries, so they fit WiredTiger's per-transaction cache limit #1975's deferred_resource_update (deferred_index.rs:283);
     • the reindex pipeline's missing_params updates (reindex_pipeline.rs:690-722).

   Why a marker. It turns the rule into a field check on a pre-image every writer already reads, and readers never need an anti-join.

   Comparison

   	(a) post-commit write	(b) synthesized v1	(c) full hybrid	(c) partial (recommended)
   Inserted in the transaction per create	1 doc + ledger (1 doc per 5,000 entries)	1 doc + 2 partial-index keys	1 doc + ledger	1 doc + ledger
   v1 at instance level right after commit	404 until the post-commit step	yes	yes	yes (fallback)
   v1 in type/system history and dashboard right after commit	no, until the post-commit step	yes	yes	no, until the post-commit step
   Reader changes	none	12 functions, 2 new partial indexes, explain-plan tests reworked	12 functions + 2 indexes	4 instance-level functions
   Post-commit work and repair	yes	none	yes	yes
   Writer rule	required	required	required	required
   Storage at rest	as today	saves the v1 copy of never-updated resources	as today	as today
   Shares #1975's mechanism	yes	no	yes	yes

   The 12 readers under (b) [M]. These are the history and dashboard readers of resource_history in storage.rs. purge also counts the collection (:7003-7020); section 5.7 covers it.

   • vread (:4098)
   • list_versions (:4221)
   • history_instance and its count (:4258, :4324)
   • history_type and its count (:4347, :4440)
   • history_system and its count (:4461, :4541)
   • count_deltas_by_bucket (:1954)
   • count_deltas_by_type_and_bucket (:2035)
   • activity_histogram (:2134)
   • latest_write_marker (:2300)

   No reader exists outside storage.rs. bulk_ingest.rs and bundle_creates.rs only write to the collection, and schema.rs only creates its indexes [M].

   Why (b) needs new indexes:

   • Type and system history must stay bounded index walks. For that they need resources indexes in history order, restricted to marked documents. Examples: partial {tenant_id, resource_type, last_updated:-1, id:-1} and {tenant_id, last_updated:-1, resource_type:-1, id:-1}, filtered on the marker existing.
   • Without them:
     • type history walks idx_resources_type_scan and discards unmarked documents;
     • system history has no resources index in (last_updated, resource_type, id) order, so it needs a blocking sort.
   • Those two keys are added inside the transaction for every Bundle-created document. That gives back part of the 2x. How much was not measured [I].

   Rejected sub-variants [I]:

   • A server-side $merge copy from resources into resource_history. $merge cannot run in a multi-document transaction, so it cannot be serialized against purge (section 6.2). It could bring back a purged v1 row.
   • Writing v1 before the transaction. That leaves orphan history rows on abort, and purge treats those rows as an existing resource (storage.rs:7015-7020).

   Scope

   • The plain create (create_attempt, storage.rs:2628), batch Bundles, small transaction Bundles and bulk ingest have no transaction-size ceiling to relieve. Moving them only adds windows.
   • Recommended scope: transaction Bundles at or above an entry threshold.
     • It needs its own gate, which stays on when search is offloaded. mongodb: write search_index rows after commit for transaction Bundles of 4,000+ entries, so they fit WiredTiger's per-transaction cache limit #1975's gate returns None when search is offloaded (backend.rs:1704-1708).
     • Default 4,000, the same as mongodb: write search_index rows after commit for transaction Bundles of 4,000+ entries, so they fit WiredTiger's per-transaction cache limit #1975's admission room. That is a trade-off [I]:
       • Bundles of 4,000 entries up to the ceiling commit today and would move into the lagging model for no benefit.
       • But the ceilings scale with the cache: resource + history scan ceilings were 4.3k / 8.6k / 17.2k / 34.3k at 0.25 / 0.5 / 1 / 2 GB [M].
       • The 388 trigger is a cache-wide dirty budget, so concurrent Bundles share it [I].
       • A default near the 2 GB lone ceiling (about 30k) would leave smaller caches and concurrent imports failing.
       • 4,000 trades lag exposure for coverage. Open question 3 asks the maintainer to choose.
   • Everything else stays byte-for-byte as today, so the mixed model is confined to large Bundle imports.

   Other backends are not expected to follow. SQLite and PostgreSQL have no comparable per-transaction cache ceiling, and S3 is untouched [I].

   4. Recommendation

   Option (c), partial form, scoped to transaction Bundles of at least N entries (default 4,000, see section 3).

   1. In the transaction.
      • Insert only the resources document.
      • Mark it, for example history_v1_pending: <bundle_id>. Keep this separate from search_rows_pending so the two rewriters stay independent.
      • Insert the ledger refs last, as mongodb: write search_index rows after commit for transaction Bundles of 4,000+ entries, so they fit WiredTiger's per-transaction cache limit #1975 does (storage.rs:5040-5066).
      • Drop StagedCreate.history for marked creates, and count their payload once in the byte cap (bundle_creates.rs:108).
   2. Ledger: a collection of its own, for example history_v1_pending.
      • It uses the same document shape as search_index_pending (ledger_documents, deferred_index.rs:217), with its not_before index. Schema init creates it, so no collection is created inside a transaction.
      • Every ref in it is a POST-created, marked resource.
      • The search ledger cannot simply carry an owes flag per document, because what a ref owes differs per ref [M]:
        • Types in inline_index_types (deferred_index.rs:100-112, the ifNoneExist targets) keep their search rows inline and get no search marker, yet they would owe v1.
        • Bundle update and delete entries are recorded for search (marker at storage.rs:5797 and :5942) but owe no v1.
      • rewrite_deferred_batch (deferred_index.rs:470-480) and deferred_resource_update (:283-300) select and clear only search_rows_pending.
      • An older build's repair deletes a ledger document once a pass over it succeeds (deferred_index.rs:955-958), whether or not any marker was found. A history-only document in search_index_pending would be claimed by an old build, treated as finished and deleted, which destroys the debt [I]. A separate collection keeps old builds away from it.
      • The alternative is in open question 4.
   3. Live post-commit step before the Bundle response.
      • It runs on the Bundle's causally consistent session and is bounded by the Bundle deadline, like write_deferred_search_rows (deferred_index.rs:751, called at storage.rs:4813).
      • In small transactions it reads the referenced resources documents that still carry history_v1_pending, upserts their v1 rows, and $unsets only that marker.
      • On native it can share the per-batch transaction that writes the search rows of a resource that owes both. That is an optimization, not a requirement [I].
      • The submitting client never sees a window, unless the deadline cuts the step short.
   4. Repair on both legs.
      • Today repair_deferred_search_index returns early when search is offloaded (deferred_index.rs:888-891).
      • HFS spawns the repair only from start_mongodb (crates/hfs/src/main.rs:1436), not from start_mongodb_elasticsearch (main.rs:4094).
      • The history repair must run on both legs, reusing mongodb: write search_index rows after commit for transaction Bundles of 4,000+ entries, so they fit WiredTiger's per-transaction cache limit #1975's claim, lease, minimum age and failure bookkeeping.
   5. Writer rule in every overwriter listed in section 3: v1 upsert first, then the marker-clearing update.
      • Every v1 writer, including the deferred one, writes a full-replacement upsert built from the marked pre-image. It is never a plain insert, and never $setOnInsert.
      • Section 6.1 explains why.
   6. Instance-level read fallback, used only when history returns nothing:
      • Trigger: vread(.., "1") misses, or the history query of list_versions, history_instance or history_instance_count returns no rows.
      • Read resources by identity: one find_one on idx_resources_identity.
      • If it is marked, synthesize the v1 row from it. For instance history, the synthesized row is added to rows before the existing sort, the version-cursor retain (parse_cursor_version, storage.rs:1096) and the has_more/truncate logic. It is filtered by _since, _before and include_deleted as apply_history_params_filter would filter it.
      • If it is not marked, re-run the history read once. If the marker was cleared between the two reads, the v1 row was stored first (section 3 invariants).
      • Then return what that gives, as today.
      • Doing both reads in one snapshot session is an equivalent alternative.
   7. Documented lag. Type history, system history, their counts and the dashboard figures may lag a large Bundle's creates:
      • while the live step runs (for the submitter, that is before the response);
      • for other clients after a crash, cancel or deadline cut, until the repair runs. That is at least max(60 s, Bundle deadline) plus up to one interval while HFS runs (main.rs:2391-2393, :2411).

   Why (c) partial rather than (b) [I]

   • Readers and plans stay as they are. It leaves the type and system history code untouched, along with their bounded-walk plan tests and the dashboard queries.
   • It keeps nearly the whole measured gain. It adds no index keys inside the transaction.
   • It reuses mongodb: write search_index rows after commit for transaction Bundles of 4,000+ entries, so they fit WiredTiger's per-transaction cache limit #1975's machinery: the ledger shape, claim/lease and repair.
   • Its cost is a new kind of window, not a longer copy of today's.
     • A Bundle's rows take their last_updated when each entry is processed (storage.rs:5546) and become visible only at commit (:5082). The harness's large lone transactions ran 80-124 s.
     • Today the rows are invisible everywhere until commit. A _since poller that misses them was not looking at a visible resource.
     • Under (c) partial, after commit the resource is readable, searchable and _lastUpdated-filterable, but absent from type and system _history and their counts until the step or the repair runs.
     • A poller that moves its _since mark past the commit therefore loses those v1 entries permanently, not just late, because they arrive with older create-time stamps.
     • The maintainer's choice between (c) partial and (b) rests on whether that is acceptable (open question 1).

   Why not the other forms

   • (a) alone: _history/1 would return 404 for a resource that GET already returns. That is the most visible inconsistency, and closing it costs four small marker-gated fallbacks.
   • Full (c): it carries both costs, and it costs more than (b).

   When to choose (b) instead

   Choose (b) if the maintainer holds the issue's "identical to today" requirement literally, with no window at all for type history, system history and the dashboard. In that case, measure (b)'s extra in-transaction index keys first, by adding the two partial indexes to the harness.

   5. Semantics under the recommendation (points 1-8)

   1. vread of version 1

   Today [M]:

   • vread (storage.rs:4098) does a find_one on history by (tenant_id, resource_type, id, version_id) (:4109), served by the unique idx_history_identity (schema.rs:388-401).
   • meta.lastUpdated comes from the row's last_updated (into_stored_resource, :1221).
   • The v1 row never changes after the create.

   Under the recommendation:

   • After the post-commit step (normally before the Bundle response), the stored v1 row is identical to today's:
     • same data;
     • version_id "1";
     • last_updated equal to the create's created_at (:5546, :5557; later writers keep created_at through extract_created_at, :1009);
     • same fhir_version.
   • Before the step, a history miss for "1" falls back to the marked resources document and synthesizes the same row. While the marker is set, data, version_id, created_at and fhir_version are still v1's (section 3 invariants) [I].
   • If the step commits between the two reads, the resources document is no longer marked, and the fallback's one history re-read finds the stored row. _history/1 does not return 404 for an existing resource [I].
   • After a later update, delete or restore, the writer rule has already stored v1 from the pre-image, so the body and meta are exact.

   2. _history at instance, type and system level

   Today [M]:

   • Instance (:4258):
     • sorts in Rust;
     • drops rows at or above the version cursor (rows.retain, using parse_cursor_version, :1096);
     • pages on that cursor;
     • count at :4324.
   • Type (:4347) and system (:4461):
     • sort on the server over idx_history_type_updated and idx_history_system_updated (:1191, :1197; schema.rs:403-426);
     • fetch count+1 (:1204);
     • page with keyset $or cursors (:1164, :1175);
     • count at :4440 and :4541.
   • _since ($gte), _before ($lt) and include_deleted are applied in apply_history_params_filter (:1042-1058).

   Under the recommendation:

   • Instance level and its count are identical at all times [I].
     • A marked document has version "1" and no other history rows.
     • When the history query is empty, the fallback contributes exactly the synthesized v1 (count 1).
     • That row goes through the same _since, _before and include_deleted filtering, the same version-cursor retain and the same has_more logic as a stored row. A stale or client-supplied cursor of 1 or more still gives an empty page, as today.
     • The read-order race is closed by the history re-read (section 4, item 6).
   • Type and system level are unchanged in code, and identical once the step has run: same rows, last_updated, indexes and plans. These tests stay as they are:
     • mongodb_history_type_does_not_read_rows_outside_the_page
     • mongodb_history_type_paging_is_ordered_and_complete
     • mongodb_history_system_does_not_read_rows_outside_the_page
     • mongodb_history_type_plan_is_a_bounded_index_walk
   • During the window, the v1 entries are missing from type and system history and their counts. They carry the create's timestamp, so a pager mid-traversal or a _since poller during the window can miss them permanently [I]. Section 4 explains how this differs from today.
   • include_deleted is unaffected, because a delete clears the marker.

   3. Later writes (update, conditional update, patch, delete, conditional delete, restore), in and out of Bundles

   Today [M]:

   • Each writer reads the current document, does a CAS on version_id, and inserts only the new version's row:

     Writer	History write
     update_attempt	:2924-2947
     soft_delete_attempt	:3268-3292
     restore_attempt	:3520-3546
     Bundle update / restore	:5828-5843
     Bundle delete	:5969-5985

   • next_version (:995) increments the version.

   • If-Match compares against resources.version_id (update_with_match :4127, delete_with_match :4158).

   • These writers are reached by:

     • conditional update and delete (search_impl.rs:1487, :1527);
     • patch (crates/persistence/src/core/storage.rs:1698);
     • Bundle PATCH (storage.rs:5435).

   Under the recommendation:

   • The writer rule (section 3) makes v1 durable before any overwrite.
   • Versions 2, 3, ... and If-Match behave as today, because resources.version_id is not touched.
   • A create followed by an update of the same id in one Bundle works:
     • creates are flushed before any non-POST entry (stages_its_create, :156);
     • so the pre-image is readable, and v1 goes into that transaction.
   • Transaction Bundles require a replica set or sharded topology (storage.rs:1460-1486), so marked documents exist only there [M].
   • Even there, update_attempt, soft_delete_attempt and restore_attempt can run without a transaction through the fallback in begin_best_effort_multi_write_session (:1500-1523) [M]. That is why the rule issues the v1 upsert before the marker-clearing update.
   • A write whose version-guarded update then fails leaves a correct v1 row behind, because any writer holding a marked pre-image writes the same v1 [I].

   4. $export and bulk paths

   • $export reads only resources, filtered on last_updated (bulk_export.rs:50-59, used at :242, :269 and :396). It is unaffected [M].
   • Bulk-submit ingest has no transaction and no ceiling to relieve.
     • It keeps writing v1 history as today (write_history, bulk_ingest.rs:891-911).
     • Its update path can overwrite a resource that a large Bundle created, so it gets the writer rule: upsert v1 first, then the version-guarded update that $unsets the marker.
     • Its batch pre-read (ExistingResource, bulk_ingest.rs:136-142) keeps version_id, content (a serde Value converted from the BSON data), created_at, fhir_version and is_deleted. It keeps neither the marker nor the raw BSON data [M].
     • The rule needs both added. The v1 upsert should use the raw data document, as the Bundle delete path does, so that the v1 body stays exact.
     • The pre-read is taken at batch start and can be stale. That is safe only because a marked pre-image's v1 content never changes and the upsert is idempotent [I].
   • Bulk-submit rollback goes through delete and update (bulk_submit.rs:1653, :1670), so point 3 covers it.

   5. Crash or abort between commit and the history write

   • Abort before commit: the resource, its marker and the ledger refs roll back together. Nothing is left behind.
   • After commit (crash, cancellation or deadline cut):
     • The ledger documents survive, and the repair writes the missing rows.
     • No committed resource is left without v1 permanently, as long as some HFS instance runs the repair.
     • GET [type]/[id]/_history/1 does not return 404 for an existing resource, because of the fallback and its re-read.
     • The type, system and dashboard lag is bounded by the repair cadence. This record proposes accepting that lag.
   • Repair turned off. HFS_MONGODB_DEFERRED_INDEX_REPAIR_INTERVAL=0 turns the repair off today. With this change, that would also leave history lagging indefinitely (see open question 7).
   • Section 6 has the details.

   6. Races

   See section 6.2.

   7. Other readers of resource_history

   • These are unchanged in code [M]:
     • count_deltas_by_bucket (:1954)
     • count_deltas_by_type_and_bucket (:2035)
     • activity_histogram (:2134)
     • latest_write_marker (:2300)
   • A large Bundle's creates are counted once their v1 rows exist, in the bucket of their create time [I].
   • The lag does not always heal on its own [I]:
     • latest_write_marker (:2300-2350) returns the newest last_updated and a count of recent writes capped at 10,000 (RECENT_CAP). The REST dashboard uses it as its change detector (crates/rest/src/dashboard.rs, module doc around lines 80-100).
     • Late v1 rows carry create-time stamps. If any later write exists, latest does not move. Once at least 10,000 recent writes exist, recent_writes does not move either.
     • A dashboard that did not record those writes itself (a sibling instance) then gets no signal, and picks the rows up only at its own reconcile or reseed.
     • resource_count_series (crates/rest/src/dashboard.rs:339-377) takes the total from resources (count_by_types) and subtracts history deltas. While the Bundle's v1 rows are missing, every point before the bucket of the Bundle's create time is raised by the number of missing rows. Points from that bucket on are exact.
   • Purge's "only in history" check (:7015-7020) also checks resources, so it is unaffected [M].
   • list_versions gets the fallback.
   • count_by_day (:1889) reads resources and is unaffected.

   8. Existing data and tenant isolation

   See section 7.

   6. Repair, crash model and races

   6.1 Repair and crash model

   The post-commit writer, per batch, runs one transaction that:

   1. reads the referenced resources documents that still carry history_v1_pending;
   2. upserts their v1 rows as full replacements built from those documents;
   3. $unsets only history_v1_pending, with a filter that still requires it.

   A ledger document is deleted once all its refs are done.

   The repair reuses #1975's machinery, pointed at the history ledger and enabled on both legs:

   • claim and lease (claim_ledger_chunk, deferred_index.rs:1056);
   • a minimum age of max(60 s, Bundle deadline);
   • a bounded number of ledger documents per pass (DEFERRED_INDEX_REPAIR_CHUNKS_PER_PASS);
   • failure bookkeeping.

   Why every v1 write is a full-replacement upsert, never a plain insert or $setOnInsert [I]:

   • Not a plain insert. "Marker set implies no v1 row" does not hold. Upsert-first writers leave the state "v1 stored, marker still set" between their two steps, or after a crash between them: bulk ingest, and update, delete and restore when running without a transaction. A plain insert_many in the deferred transaction would then hit 11000 and abort on every retry.
   • Not $setOnInsert. purge (storage.rs:7026-7058) deletes resources and then resource_history in two separate non-transactional calls, and its comment at :7008-7010 treats "exists only in history" as a supported state. So a stale v1 row from an earlier incarnation of the same id can exist.
     • Today a Bundle create of such an id fails loudly, because the v1 insert_many hits the unique idx_history_identity (bundle_creates.rs:339-348).
     • $setOnInsert would silently keep the stale row, and vread(.., "1") would return the earlier incarnation's body.
   • Why replacement is safe. While the marker is set, the resources document's v1 content is authoritative (section 3 invariants). Every writer that holds a marked pre-image therefore writes the same row, so replacing it is idempotent. A marked create of an id with a stale v1 row ends with the new v1, and the stale v2+ rows of the earlier incarnation remain, as today.

   Readers stay correct in the "v1 stored, marker still set" state: the vread fallback runs only on a history miss, and the instance fallback only on an empty result, so a stored v1 is never listed twice [I].

   6.2 Races [I, from the code paths cited]

   An update before the deferred write:

   • The update stores v1 first, then clears the marker with its own update.
   • The deferred writer's filter then finds no marker and writes nothing.
   • If the two run concurrently in transactions, both write the same resources document. One gets WriteConflict and retries from the winner's state. This is the same argument as index_deferred_batch_once (deferred_index.rs:430).
   • Both write identical v1 content, keyed on the unique idx_history_identity, so v1 stays single and exact. If two upserts race and one ends in 11000, the server retries an upsert whose filter is an equality match on the unique index's fields (MongoDB 4.2+).
   • Overwriters are already serialized by the CAS plus WriteConflict (test mongodb_concurrent_updates_from_the_same_version_admit_one).

   Purge (storage.rs:6993) is non-transactional. It deletes from resources, then from history (:7026-7058). purge_all (:7061) does the same. Against the deferred writer:

   Order of events	Result
   Purge's delete committed before the deferred transaction started	The writer finds nothing.
   Purge's delete committed after the writer's snapshot	The writer's marker $unset gets WriteConflict, and its history upsert rolls back.
   The writer committed first	Purge's later history delete removes the row.

   So the deferred write must run in a transaction that updates the resources document. A non-transactional or $merge copy is not allowed.

   Upsert-first writers without a transaction. These are bulk ingest, plus update, delete and restore in the session fallback. If a purge lands between their v1 upsert and their update, they can leave an orphan v1 row. Bulk ingest's existing non-transactional history write already has this kind of hazard. The rule widens it slightly.

   purge_tenant_data (storage.rs:2428-2449) deletes resource_history before resources and before the ledger.

   • A repair pass that commits between those deletes would leave an orphan v1 row for a purged tenant.
   • Fix: delete the ledger first, then resources, then the derived collections, or have the repair skip tenants that are being deleted. The new history ledger must also be covered.
   • The same order already exposes mongodb: write search_index rows after commit for transaction Bundles of 4,000+ entries, so they fit WiredTiger's per-transaction cache limit #1975's search rows today. search_index is deleted first, so a deferred search pass that commits before resources is deleted can leave orphan rows (mongodb: purge_tenant_data deletes search rows before resources, so a concurrent deferred search-index batch can leave rows for a purged tenant #2259).

   7. Migration and tenant isolation

   No data migration.

   • Existing documents carry no marker and behave exactly as today.
   • Only new large Bundle creates are marked.

   Ledger options:

   • Recommended: a collection of its own (section 4, item 2). It needs:
     • a SCHEMA_VERSION bump (15 today, schema.rs:42; migrations run through migrate_schema_async, :85-101);
     • coverage in purge_tenant_data.
   • Alternative: history-only documents in search_index_pending with an owes field, with homogeneous refs per document. This needs no schema bump. But older builds' repair would claim and delete those documents (section 4, item 2), and both rewriters' filters and updates would have to become marker-specific.
   • Either way there is no new index on resources, so no index keys are added inside the transaction.

   Tenant isolation:

   • Ledger documents carry tenant_id (ledger_documents, deferred_index.rs:217). Every deferred-write and repair query takes it from there.
   • The fallback lookup uses idx_resources_identity, which leads with tenant_id.
   • No query is added without a tenant_id equality.

   Downgrade and rolling upgrade (a new risk):

   • A build that does not know the marker would overwrite a marked resource without storing v1, so v1 would be lost. It would also never write pending v1 rows.
   • With the separate collection, an older build at least leaves the history ledger alone. With owes in search_index_pending, an older native build's repair would delete history-only documents as finished.
   • Before a downgrade, drain: run the repair until no history work is left, and say so in the release notes.
   • In a multi-instance deployment, finish the upgrade before large Bundles are accepted. No mixed versions should run while marked documents exist.

   8. Tests the implementation PR must include

   All are MongoDB testcontainers tests. They need test controls:

   • a threshold knob, so that small Bundles take the new path;
   • a test hook that skips the live post-commit step, to open the window deterministically;
   • a hook that pauses a fallback read between its history read and its resources read.

   1. Reads after a large Bundle create, with the step run and with it skipped.
      • Covers:
        • vread v1 (body, meta.versionId, meta.lastUpdated);
        • _history at instance, type and system level with _since, _before and paging, including an instance-level version cursor of 1 or more on a marked resource;
        • counts and include_deleted.
      • Each is compared with a below-threshold reference Bundle.
      • With the step skipped:
        • instance level is identical;
        • type and system level match the reference after the repair.
   2. The same after each later write, applied while the marker is still set.
      • Writes: update, conditional update, patch, delete, conditional delete and restore.
      • v1 is exact, versions run 2, 3, ..., and If-Match conflicts are unchanged.
      • Also: a create followed by an update of the same id in one Bundle.
      • Also with the session fallback forced (transaction_active == false), and with a failure injected between the v1 upsert and the update. v1 survives, the marker is still set, and a retry or the repair completes.
   3. Bulk-ingest update of a marked resource stores a byte-exact v1 from the raw data. Also simulate a crash between its two steps, then run the repair.
   4. Purge.
      • purge and purge_all on a marked resource, then the repair: no history row comes back.
      • purge_tenant_data removes the history ledger, and a repair racing it leaves nothing.
      • A stale v1 row left by an interrupted purge: a marked create of that id, then the step. vread(.., "1") returns the new body.
   5. Simulated failure between commit and the history write.
      • Skip the step, then run the repair, on the offloaded leg too.
      • v1 is written exactly once, the marker and ledger are cleared, and a second pass is a no-op.
   6. Concurrent deferred write and update of the same resource: exactly one v1 row, holding the v1 body.
   7. Read-order race. With the fallback paused between its two reads, let the step commit. vread v1, instance history, list_versions and the count still return v1.
   8. Ledger contents.
      • A type in inline_index_types (an ifNoneExist target) owes v1.
      • Bundle update and delete entries owe none.
      • On native, a resource deferred for both has each marker cleared independently.
   9. Dashboard and activity.
      • After the step or the repair, count_deltas_by_bucket, count_deltas_by_type_and_bucket, activity_histogram and latest_write_marker equal the reference.
      • During the window, the effects documented in section 5.7 on latest_write_marker and on the resource_count_series baseline are asserted.
   10. Tenant isolation. Two tenants with the same ids: the repair and the fallback never cross tenants.
   11. One document per entry.
       • A unit test: a marked StagedCreate carries no history document and counts its payload once.
       • Benchmark acceptance:
         • mongostat insert opcounters show about 1 document per entry-write, plus the ledger;
         • the 49,027-entry Bundle commits on mongodb-elasticsearch at 2 GB, or the new abort position is reported;
         • that leg reaches 1000/1000.
   12. No regressions. The existing versioning and history tests pass unchanged on every backend, including the four plan and paging tests listed above. SQLite, PostgreSQL and S3 are untouched.
   13. Re-measure on the PR. Run v1_history_copy_ceiling at 2 GB with COMMIT_PROBE=on and CONTROL_COPIES, with the control run against the new path.

   9. Open questions for the maintainer

   1. Lag or no lag? Accept the window in section 4, item 7, which means (c) partial? In that window a resource is readable and searchable but missing from type and system history, and a _since poller that moves past the commit loses those v1 entries permanently. Or require strict identity with no window, which means (b)?
   2. Timing of the live step. Run it before the Bundle response, as recommended? Or detach it to keep Bundle latency down? The step writes one history document and clears one marker per entry. Its duration was not measured.
   3. Threshold. Share mongodb: write search_index rows after commit for transaction Bundles of 4,000+ entries, so they fit WiredTiger's per-transaction cache limit #1975's 4,000 (recommended for coverage of small caches and concurrent Bundles, at the cost of exposing 4k-30k-entry Bundles to the lag)? Or set it higher, or derive it from the cache size? And what knob name, for example HFS_MONGODB_DEFERRED_HISTORY_MIN_ENTRIES?
   4. Ledger shape. A separate collection (recommended), or history-only owes documents in search_index_pending?
   5. Scope. Keep the plain create, batch Bundles, small transaction Bundles and bulk ingest as they are today (recommended)?
   6. Downgrade policy for marked documents (section 7).
   7. Disabling the repair. Should HFS_MONGODB_DEFERRED_INDEX_REPAIR_INTERVAL=0 still turn off the repair once it also owes history? If it stays allowed, it should at least log a warning.
   8. Native follow-up. On native, the resources document's own cost (missing_params multikey and missing_params_fp) is the larger share of what is left in the transaction: 6,664 vs 3,103 dirty bytes per entry. Should it get its own issue, linked to mongodb: write search_index rows after commit for transaction Bundles of 4,000+ entries, so they fit WiredTiger's per-transaction cache limit #1975 and MongoDB: WriteConflict aborts exhaust the 3-attempt bundle retry under 20 concurrent imports — 477/1000 benchmark bundles fail with 503 #1776? (Since answered for creates: fix(persistence): write deferred Bundle creates' missing_params after commit #2256 removes it from the transaction and measures the deferred ceiling at 98-99% of the offloaded one. Updates and deletes are on perf(mongodb): a deferred transaction Bundle extracts search rows twice, in the transaction and again after the commit #2035.)
   9. Writer rule without a transaction. Upsert first (recommended)? Or fail with a retryable error when the pre-image is marked and no transaction is active? Separately, should the silent non-transactional fallback on replica sets go away (mongodb: single-resource writes on a replica set silently run without a transaction when start_session, hello or start_transaction fails #2258)?

   10. What was not run

   Not run:

   • No product code was written or tested. Sections 3-7 come from reading the code at 6f62064; none of those semantics was exercised.
   • No fhir-benchmark run, and no mongostat.
   • No Synthea corpus, and no 49,027-entry Bundle through the real backend.
   • The native shape without the marker.
   • The commit probe and the control for native_deferred.
   • A resource-only control: impossible, because the backend has no switch to skip the v1 write.
   • clippy --all-targets.

   Caveats:

   • The resource-only commit ceiling rests on a single probe. The resource + history commit bracket also comes from one bisection.
   • The 0.25 GB point is one run per variant.
   • The host was loaded during every run.

   Gate that was run, all exit 0 [M]:

   • cargo fmt -- --check on the two test files.
   • cargo clippy -p helios-persistence --features mongodb --test mongodb_tests with the repo allow-list: 0 warnings.
   • cargo test -p helios-persistence --features mongodb --test mongodb_tests --no-run.

   Raw harness lines (verbatim)

   # v1c-smoke.log (index list as created by initialize())
   V1_CEILING_SETUP wt_cache_gb=0.25 cache_max_bytes=268435456 mongod=7.0.43 shape=offloaded templates=267 avg_json_bytes=1341 avg_resource_doc_bson_bytes=1662 avg_history_doc_bson_bytes=1662 chunk=100 max_entries=300000 repeats=1 settle_secs=5 resource_fields=tenant_id,resource_type,id,version_id,data,created_at,last_updated,is_deleted,deleted_at,fhir_version resources_indexes=_id_[_id] | idx_resources_identity[tenant_id,resource_type,id;unique] | idx_resources_type_scan[tenant_id,resource_type,is_deleted,last_updated,id] | idx_resources_updated[tenant_id,last_updated] | idx_resources_live_tenant[is_deleted,tenant_id] | idx_resources_missing_params[tenant_id,resource_type,is_deleted,missing_params,last_updated,id] | idx_resources_missing_params_fp[tenant_id,resource_type,is_deleted,missing_params_fp;partial={"missing_params":{"$exists":true}}] history_indexes=_id_[_id] | idx_history_identity[tenant_id,resource_type,id,version_id;unique] | idx_history_resource_updated[tenant_id,resource_type,id,last_updated] | idx_history_type_updated[tenant_id,resource_type,last_updated,id] | idx_history_system_updated[tenant_id,last_updated,resource_type,id]
   
   # v1c-2.log
   V1_CEILING_SUMMARY wt_cache_gb=2 shape=offloaded variant=resource runs=3 entries_median=69200 entries_min=69100 entries_max=69200 spread_pct=0.1 json_mb_median=92.80 json_mb_min=92.67 json_mb_max=92.80 dirty_bytes_per_entry_median=3103 outcomes=too_large:3 codes=388:3
   V1_CEILING_SUMMARY wt_cache_gb=2 shape=offloaded variant=resource_history runs=3 entries_median=34300 entries_min=34300 entries_max=34400 spread_pct=0.3 json_mb_median=46.00 json_mb_min=46.00 json_mb_max=46.13 dirty_bytes_per_entry_median=6261 outcomes=too_large:3 codes=388:3
   V1_CEILING_GAIN wt_cache_gb=2 shape=offloaded entries_ratio=2.017 json_bytes_ratio=2.017 dirty_per_entry_ratio=2.017 dirty_at_half_ratio=2.016 dirty_at_min_ratio=2.010 ref_half=17100 ref_min=34300 censored=false
   
   # v1c-2-commit.log
   V1_CEILING_COMMIT_RESULT wt_cache_gb=2 shape=offloaded variant=resource max_committed_entries=68700 min_failed_entries=na start=68700
   V1_CEILING_COMMIT_RESULT wt_cache_gb=2 shape=offloaded variant=resource_history max_committed_entries=31900 min_failed_entries=32200 start=34400
   
   # v1c-2-control.log
   V1_CEILING_CONTROL wt_cache_gb=2 shape=offloaded copies=148 entries=39516 outcome=too_large failed_entry=35584 secs=105.3 loadavg=7.76
   
   # v1c-2-native-deferred.log
   V1_CEILING_SUMMARY wt_cache_gb=2 shape=native_deferred variant=resource runs=3 entries_median=32300 entries_min=31700 entries_max=32300 spread_pct=1.9 json_mb_median=43.31 json_mb_min=42.52 json_mb_max=43.31 dirty_bytes_per_entry_median=6664 outcomes=too_large:3 codes=388:3
   V1_CEILING_SUMMARY wt_cache_gb=2 shape=native_deferred variant=resource_history runs=3 entries_median=21600 entries_min=21600 entries_max=21600 spread_pct=0.0 json_mb_median=28.97 json_mb_min=28.97 json_mb_max=28.97 dirty_bytes_per_entry_median=9933 outcomes=too_large:3 codes=388:3
   V1_CEILING_GAIN wt_cache_gb=2 shape=native_deferred entries_ratio=1.495 json_bytes_ratio=1.495 dirty_per_entry_ratio=1.491 dirty_at_half_ratio=1.494 dirty_at_min_ratio=1.485 ref_half=10800 ref_min=21600 censored=false
   
4. sandhums

   added a commit that references this issue on Oct 28, 2026

   test(mongodb): measure the v1 history copy's transaction ceiling

   ...

   2e11ae4