Skip to content

Keep live pages when filling in lazy memory - #410

Open
Max042004 wants to merge 1 commit into
sysprog21:mainfrom
Max042004:materialize-keep-valid
Open

Max042004 wants to merge 1 commit into
sysprog21:mainfrom
Max042004:materialize-keep-valid

Conversation

@Max042004

@Max042004 Max042004 commented Oct 4, 2026 •

Copy link
Copy Markdown
Collaborator

guest_materialize_lazy fills in the part of a MAP_NORESERVE region that lies in the faulting 2 MiB block. Three things it did wrong:

It zeroed every page of that part, including pages already mapped. A mapping placed next to one the guest has written joins the same region, so the first touch of the new mapping wiped the old one.

It zeroed after publishing the descriptors, so a sibling vCPU could store through a new descriptor before the zeroing landed and lose the store.

It mapped a PROT_NONE region readable, so touching a PROT_NONE reservation read zeros instead of raising SIGSEGV.

Zero only the pages that had no mapping, before any descriptor is published, refuse a region with no access bits, and treat a fault on a page another vCPU has already filled in as a stale translation.


Summary by cubic

Fixes guest_materialize_lazy so the first touch of MAP_NORESERVE memory no longer wipes neighbor data, loses concurrent stores, grants access to PROT_NONE reservations, or lets a sibling vCPU reach a hole in a partly covered block.

  • Zeroes only pages without an existing mapping, before any descriptor is published.
  • Returns an error for regions with no access bits instead of making them readable.
  • Treats a fault on a page another vCPU already filled in as a stale translation, with an I-cache invalidation hint for executable regions, and reports it as SEGV_ACCERR.
  • Fills the L3 table of a partly covered block before the L2 slot points at it, so no vCPU sees a hole or guard during the trim.
  • Adds test-noreserve-materialize, a stress test that races a writer against the fill, checks PROT_NONE behavior, and verifies a hole past the mapping still faults.

Written for commit cfca490. Summary will update on new commits.

View guided diff

cubic-dev-ai[bot]

This comment was marked as resolved.

@Max042004
Max042004 force-pushed the materialize-keep-valid branch from 291d375 to 112d505 Compare October 5, 2026 09:35
cubic-dev-ai[bot]

This comment was marked as resolved.

@Max042004
Max042004 force-pushed the materialize-keep-valid branch from 112d505 to 7114c26 Compare October 5, 2026 13:06
cubic-dev-ai[bot]

This comment was marked as resolved.

@Max042004
Max042004 force-pushed the materialize-keep-valid branch from 7114c26 to 31e6fe0 Compare October 5, 2026 13:33
Comment thread src/core/guest.c Outdated
Comment thread tests/test-noreserve-materialize.c
guest_materialize_lazy fills in the part of a MAP_NORESERVE region
that lies in the faulting 2 MiB block. Four things it did wrong:

It zeroed every page of that part, including pages already mapped.
A mapping placed next to one the guest has written joins the same
region, so the first touch of the new mapping wiped the old one.

It zeroed after publishing the descriptors, so a sibling vCPU could
store through a new descriptor before the zeroing landed and lose the
store.

It mapped a PROT_NONE region readable, so touching a PROT_NONE
reservation read zeros instead of raising SIGSEGV.

For a block the region only partly covers, it published a whole 2 MiB
block and trimmed it afterwards, so a sibling vCPU could reach a guard
or hole in that block during the trim.

Zero only the pages that had no mapping, before any descriptor is
published; fill the L3 table of a partly covered block before the L2
slot points at it; refuse a region with no access bits; and leave a
fault on a page another vCPU has already filled in to the bounded
stale-TLB retry, which reissues the TLBI. A translation fault on a page
that is mapped by the time it is reported, but without the access it
needs, is SEGV_ACCERR, as Linux reports it.
@Max042004
Max042004 force-pushed the materialize-keep-valid branch from 31e6fe0 to cfca490 Compare October 12, 2026 15:46

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1 issue found across 4 files (changes from recent commits).

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="src/syscall/proc.c">

<violation number="1" location="src/syscall/proc.c:4047">
P2: A tracked `PROT_NONE` reservation has no PTE, so this check returns false and reports `SEGV_MAPERR`; consult region metadata to report `SEGV_ACCERR` for that mapped, inaccessible range.</violation>
</file>

Tip: Review your code locally with the cubic CLI to iterate faster.

View guided diff | Re-trigger cubic

Comment thread src/syscall/proc.c
bool mapped_now = false;
if (fsc_type == 0x01) {
mmap_lock_acquire();
mapped_now = guest_ptr_avail(g, far_addr, NULL, 0) != NULL;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: A tracked PROT_NONE reservation has no PTE, so this check returns false and reports SEGV_MAPERR; consult region metadata to report SEGV_ACCERR for that mapped, inaccessible range.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At src/syscall/proc.c, line 4047:

<comment>A tracked `PROT_NONE` reservation has no PTE, so this check returns false and reports `SEGV_MAPERR`; consult region metadata to report `SEGV_ACCERR` for that mapped, inaccessible range.</comment>

<file context>
@@ -4036,9 +4036,19 @@ static bool vcpu_handle_el0_fault(guest_t *g,
+    bool mapped_now = false;
+    if (fsc_type == 0x01) {
+        mmap_lock_acquire();
+        mapped_now = guest_ptr_avail(g, far_addr, NULL, 0) != NULL;
+        mmap_lock_release();
+    }
</file context>

Comment thread tests/test-matrix.sh
test_rc "$runner" "test-madvise" 0 "$bindir/test-madvise"

printf "\nMAP_NORESERVE first touch\n"
test_rc "$runner" "test-noreserve-materialize" 0 \

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The test is registered here only, not in tests/manifest.txt, so tests/driver.sh never picks it up. That leaves it out of make check and out of the sanitizer lanes, which select manifest sections, so the vCPU race this change fixes is never run under TSAN. test-madvise just above is in both lists; add test-noreserve-materialize to the manifest the same way.

Comment thread src/core/guest.c
*/
uint64_t fault_page = fault_offset & ~(PAGE_SIZE - 1);
if (gva_resolve_perm(g, g->ipa_base + fault_page, NULL, 0, UINT64_MAX))
return -1;

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Declining here sends an ordinary race down the stale-TLB branch of vcpu_handle_el0_fault. Two threads that first-touch the same page are enough: the one that waited on mmap_lock gets -1, guest_ptr_avail then grants the access, and the handler prints the live PT grants access (stale TLB) warning at the default log level, once per thread. That message is meant to flag a TLBI that did not take, so it now fires on correct behavior. Return a distinct value for "already mapped" so the caller can keep the bounded retry but skip the warning.

Comment thread src/core/guest.c
* and leave the rest of the block unmapped. The L3 table is complete before the
* release store publishes it, so no vCPU ever sees a page outside [start, end).
*/
static int materialize_partial_block(guest_t *g,

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The description does not mention materialize_partial_block, and it fixes a fourth defect of its own: the old path published a full 2 MiB block, split it, then invalidated the pages outside the region, so a sibling vCPU could read or write a guard or hole in between. Add it to the description next to the other three, since only the generated summary names it.

Comment thread src/syscall/proc.c
mapped_now = guest_ptr_avail(g, far_addr, NULL, 0) != NULL;
mmap_lock_release();
}
int si_code = (fsc_type == 0x03 || mapped_now) ? LINUX_SEGV_ACCERR

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This si_code change is guest-visible and the description does not state it: a translation fault on a page the live page table now maps is reported as SEGV_ACCERR instead of SEGV_MAPERR. Say so in the description, with the case that needs it (a write racing the first read of a PROT_READ mapping).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants