allocator: refactor for stabilisation by nia-e · Pull Request #157428 · rust-lang/rust

nia-e · 2026-06-04T11:47:58Z

Adds my current proposal per the doc in #156882 and follow-up Zulip conversations (notably for dyn-compat) unstably.

r? libs

nia-e · 2026-06-04T14:08:19Z

Note that the no-panic bounds introduced close #156490 and #155746. However, if we want to relax them in the future, we may need to adjust our collection types to be more resilient.

clarfonthey · 2026-06-04T19:19:49Z

@rustbot author

(mostly so you can more clearly signal when you think things are ready; I've commented here already so I'll see any additional changes for review as they're made)

maxdexh · 2026-07-26T19:26:04Z

For the record, LinkedList is also sensitive to alloc.clone() unwinding. Not filing an issue because I assume this is getting merged soon (unless you want me to file an issue 👀)

nia-e · 2026-07-26T19:27:29Z

please do file issues for everything distinct! we need to keep track of what's outstanding as we go and stabilise things

RalfJung · 2026-07-27T06:54:16Z

Other than the opsem stuff (which I haven't been following at all and therefore don't know the state of), and btree stuff (which is a later problem), I don't think there are other soundness issues. I'll probably have a few more looks at it before stabilization though.

Which opsem stuff? There should be none left since we delegated all opsem magic to a hypothetical future NativeAllocator wrapper.

And there's the unleaking problem but not much we can do about that, that just is fundamentally a tricky footgun.

clarfonthey · 2026-07-27T11:30:32Z

(Have been mostly just waiting for the discussion to settle before doing a proper review, and poking at random simple changes to the code where they don't require much thinking to verify they're correct.)

maxdexh · 2026-07-28T17:31:36Z

@@ -74,7 +74,8 @@ impl fmt::Display for AllocError {
 /// * Moving, subtyping, unsize-coercing, or trait-upcasting an allocator does not change
 ///   what the allocator is equivalent to.
 /// * Copying or cloning allocator results in an allocator that's


Do we require copies to be equivalent? If so, this relies on the orphan rule for Copy (which might be fine since Copy is special), but I'm confused because the AllocatorClone docs do not talk about requirements for copies.

View changes since the review

I thought there was wording to the effect of "if Copy could be implemented for an AllocatorClone type, that must also obey its semantics", but if not I will add it. ty!

What do you mean by "could"? Can we rely on Drop + Copy being disallowed for soundness? I think we didn't discuss this part enough 😅

yk what we can do btree later raaaaagh pain and suffering *unleaks your box* whats thiSegmentation fault (core dumped) straight up testing it. and by it, well, let's just say. "my allocator" linked and listed the unnecesary bits? gone. reduced to atoms

rustbot · 2026-08-04T18:10:45Z

This PR was rebased onto a different main commit. Here's a range-diff highlighting what actually changed.

Rebasing is a normal part of keeping PRs up to date, so no action is needed—this note is just to help reviewers.

nia-e · 2026-08-04T18:11:06Z

(apologies for the squash-and-rebase, nothing meaningful to review should have changed between force-pushes)

clarfonthey · 2026-08-06T04:54:08Z

-        if self.len() > source.len() {
-            self.split_off(source.len());
-        }
-        for (elem, source_elem) in self.iter_mut().zip(&mut source_iter) {
+        for elem in self.iter_mut() {
+            let Some(source_elem) = source_iter.next() else {
+                break;
+            };
            elem.clone_from(source_elem);
        }
+        while self.len() > source.len() {
+            self.pop_back();
+        }


Unreasonably upset at this code just because of how limited the LinkedList API is. Should be doable, but isn't… :(

View changes since the review

samesies qwq

clarfonthey · 2026-08-06T04:55:52Z

    /// This function is mainly useful for data that lives for the remainder of the program's life,
    /// i.e., memory that is meant to leak. Reconstructing ("unleaking") a `Box` from the mutable
    /// reference returned here (e.g. via [`Box::from_raw`]) is a grey area (meaning it is possible
    /// under specific circumstances but many seemingly harmless ways of doing it are undefined
    /// behavior) and should be avoided. If the memory should eventually be freed, prefer to use
    /// [`Box::into_raw`] or [`Box::into_non_null`] instead.
    ///
+    /// However, "unleaking" as per the above via [`Box::from_raw_in`] is only sound
+    /// for the global allocator.
+    ///


These two paragraphs combined read quite weird. I think you probably want to just separate the idea of "unleaking" into a second paragraph, starting with the no-unleak case and then clarifying unleaking only works for Global.

View changes since the review

It seems like this is an attempt to partially spell out what "specific circumstances" make unleaking sound. If that's what we want to do we should actually spell it out IMO:

Unleaking is allowed if

A is Global, and

the pointer passed to from_raw was derived from the reference returned by this method in a way that all intermediate references on the derivation path are &mut T (no shared references, no references to other types).

that's better wording, yes. i'll adapt that, ty <3

Oh and it may be worth reminding the reader that if a function takes an &mut argument, freeing that while the function runs is UB. So it'd be okay to to Box::from_raw inside such a function but not to drop that box.

clarfonthey · 2026-08-06T04:58:02Z

 #[cfg(not(no_global_oom_handling))]
 #[stable(feature = "box_slice_clone", since = "1.3.0")]
-impl Clone for Box<str> {
+impl<A: Allocator + Clone> Clone for Box<str, A> {


Would just comment here: not AllocatorClone since the new allocation doesn't have to be related to the original.

View changes since the review

happy to add that, sure thing

clarfonthey · 2026-08-06T04:59:20Z

+#[unstable(feature = "allocator_api", issue = "32838")]
+#[must_use]
+#[inline]
+pub unsafe fn from_boxed_utf8_unchecked_in<A: crate::alloc::Allocator>(


I still don't like this method, because I don't think we should use the _in naming when the allocator is part of a box, rather than as an extra parameter.

I assume there are problems with parameterising from_boxed_utf8_unchecked normally?

View changes since the review

I still don't like this method, because I don't think we should use the _in naming when the allocator is part of a box, rather than as an extra parameter.

Fair, happy to rename it and/or make it private, we just need the functionality for this clone impl.

I assume there are problems with parameterising from_boxed_utf8_unchecked normally?

Would be a breaking change to a stable api :c

i guess i should explicitly ask - what would be preferred here? i'm fine with any option as long as I can get the semantics of this function

I guess I would prefer just this being either private or a method on Box for now. I just don't want it public to imply this is the method we want; maybe with an edition redirect we could update the old method.

clarfonthey · 2026-08-06T05:01:29Z

 /// This ensures that pointer arithmetic within the allocation
 /// (for example, `ptr.add(len)`) cannot overflow the address space.
 ///
+/// None of the allocating or deallocating methods may unwind. This restriction


Even via handle_alloc_error? I guess that's not going to be stabilised at the same time?

View changes since the review

handle_alloc_error is called by users of the allocator, not by the allocator itself.

clarfonthey · 2026-08-06T05:03:57Z

Sign in to view

-///   be lifted in the future, but currently a panic from any of these
-///   functions may lead to memory unsafety.
-///
-/// * You must not rely on allocations actually happening, even if there are explicit


Is this no longer the case?

View changes since the review

Per conversations with opsem, the language magic (e.g. allocations being elided regardless of possible side effects) will be moved to its own wrapper type NativeAllocator<A: StaticAllocator> as it requires manually (i.e. within the Allocator impl of said wrapper) invoking LLVM intrinsics. so, Allocator can just be a non-magical normal trait

Darksonn · 2026-08-06T09:31:50Z

This PR seems to make multiple unrelated changes. Is it possible to get an overview of all the changes it's making? Or even better, perhaps we could split out this into multiple PRs so that we can merge the things we agree on now, without blocking them on the things still being discussed?

nia-e · 2026-08-06T09:52:56Z

I'm unsure how i'd go about splitting this up, since many of the miscellaneous library changes are necessary for the API to be sound. (ofc, if you have an idea that'd lighten the review burden, i'm happy to do it and i think @clarfonthey would be relieved too)

In short, the changes made are:

split out requirements from Allocator into the 2 new traits StaticAllocator and AllocatorClone, as not all allocators we want to support can obey those semantics
- StaticAllocator is required for pinning and acting as a global allocator, effectively being a promise that this allocator will never free memory without an explicit deallocate call (and not from e.g. dropping)
- AllocatorClone promises semantics about what cloning an allocator does, namely that it requires the clone not to unwind and that the new allocator is interchangeable with the old one. this is
update std APIs to use these new trait bounds where necessary for soundness
implement said traits for std pointer types where doing so is sound, to facilitate using methods bound by them even if the new traits take a longer while to be proposed for stabilisation than the main one (i guess this could be split out?)

i'm quite happy with the fact that (beyond adding the ban on unwinding out of an allocating method or drop) none of this tightens requirements on Allocator implementors at all.

maxdexh · 2026-08-06T12:48:17Z

Also, unlike the old allocator api, we think this is sound (modulo some details in the docs we haven't discussed fully) :D

rustbot assigned clarfonthey Jun 4, 2026

rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. T-libs Relevant to the library team, which will review and decide on the PR/issue. labels Jun 4, 2026

nia-e added the A-allocators Area: Custom and system allocators label Jun 4, 2026

Urgau reviewed Jun 4, 2026

View reviewed changes

Comment thread library/core/src/alloc/mod.rs Outdated

This comment has been minimized.

Sign in to view

This comment was marked as resolved.

Sign in to view

This comment has been minimized.

Sign in to view

clarfonthey reviewed Jun 4, 2026

View reviewed changes

Comment thread library/alloc/src/str.rs Outdated

clarfonthey reviewed Jun 4, 2026

View reviewed changes

Comment thread library/core/src/alloc/mod.rs Outdated

clarfonthey reviewed Jun 4, 2026

View reviewed changes

Comment thread library/core/src/alloc/mod.rs Outdated

clarfonthey reviewed Jun 4, 2026

View reviewed changes

Comment thread library/core/src/alloc/mod.rs Outdated

clarfonthey reviewed Jun 4, 2026

View reviewed changes

Comment thread library/core/src/alloc/mod.rs Outdated

clarfonthey reviewed Jun 4, 2026

View reviewed changes

Comment thread library/core/src/alloc/mod.rs Outdated

clarfonthey reviewed Jun 4, 2026

View reviewed changes

Comment thread library/core/src/alloc/mod.rs Outdated

clarfonthey reviewed Jun 4, 2026

View reviewed changes

Comment thread library/core/src/alloc/mod.rs Outdated

clarfonthey reviewed Jun 4, 2026

View reviewed changes

Comment thread library/core/src/alloc/mod.rs Outdated

clarfonthey reviewed Jun 4, 2026

View reviewed changes

Comment thread library/core/src/alloc/mod.rs Outdated

clarfonthey reviewed Jun 4, 2026

View reviewed changes

Comment thread library/core/src/alloc/mod.rs Outdated

rustbot added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Jun 4, 2026

nia-e commented Jun 4, 2026

View reviewed changes

Comment thread library/core/src/alloc/mod.rs

This comment has been minimized.

Sign in to view

theemathas mentioned this pull request Jun 5, 2026

Add more safety requirements for Allocator impls #156544

Closed

This comment was marked as off-topic.

Sign in to view

maxdexh mentioned this pull request Jul 26, 2026

LinkedList::split_off breaks length when allocator clone panics #159982

Open

clarfonthey reviewed Jul 26, 2026

View reviewed changes

Comment thread library/alloc/src/collections/linked_list.rs Outdated

maxdexh reviewed Jul 28, 2026

View reviewed changes

Comment thread library/core/src/alloc/mod.rs Outdated

maxdexh reviewed Jul 28, 2026

View reviewed changes

Comment thread library/core/src/alloc/mod.rs Outdated

maxdexh reviewed Jul 28, 2026

View reviewed changes

Comment thread library/core/src/alloc/mod.rs Outdated

maxdexh reviewed Jul 28, 2026

View reviewed changes

Comment thread library/core/src/alloc/mod.rs

This comment has been minimized.

Sign in to view

RalfJung mentioned this pull request Aug 3, 2026

Writing an invalid value through a reference the caller can never use again rust-lang/unsafe-code-guidelines#618

Closed

nia-e force-pushed the allocator-refactor branch from a41958f to acb148f Compare August 4, 2026 18:10

This comment has been minimized.

Sign in to view

ditch now-irrelevant test

5e024df

clarfonthey reviewed Aug 6, 2026

View reviewed changes

Comment thread library/core/src/alloc/mod.rs

clarfonthey reviewed Aug 6, 2026

View reviewed changes

Uh oh!

Uh oh!

Conversation

nia-e commented Jun 4, 2026 • edited by rustbot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

This comment has been minimized.

This comment was marked as resolved.

Uh oh!

This comment has been minimized.

nia-e commented Jun 4, 2026

Uh oh!

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

clarfonthey commented Jun 4, 2026

Uh oh!

Uh oh!

This comment has been minimized.

maxdexh commented Jul 26, 2026

Uh oh!

nia-e commented Jul 26, 2026

Uh oh!

This comment was marked as off-topic.

Uh oh!

RalfJung commented Jul 27, 2026

Uh oh!

clarfonthey commented Jul 27, 2026

Uh oh!

Uh oh!

Uh oh!

maxdexh Jul 28, 2026 • edited by rustbot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

This comment has been minimized.

rustbot commented Aug 4, 2026

Uh oh!

nia-e commented Aug 4, 2026

Uh oh!

This comment has been minimized.

clarfonthey Aug 6, 2026 • edited by rustbot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

clarfonthey Aug 6, 2026 • edited by rustbot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

RalfJung Aug 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

clarfonthey Aug 6, 2026 • edited by rustbot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

nia-e commented Jun 4, 2026 •

edited by rustbot

Loading

maxdexh Jul 28, 2026 •

edited by rustbot

Loading

clarfonthey Aug 6, 2026 •

edited by rustbot

Loading

clarfonthey Aug 6, 2026 •

edited by rustbot

Loading

RalfJung Aug 6, 2026 •

edited

Loading

clarfonthey Aug 6, 2026 •

edited by rustbot

Loading

clarfonthey Aug 6, 2026 •

edited by rustbot

Loading

clarfonthey Aug 6, 2026 •

edited by rustbot

Loading

clarfonthey Aug 6, 2026 •

edited by rustbot

Loading

nia-e commented Aug 6, 2026 •

edited

Loading

maxdexh commented Aug 6, 2026 •

edited

Loading