Skip to content

Commit 11b0122

Browse files
corona10x42005e1f
andcommitted
gh-158592: Skip reallocation when shrinking a small list
Lists with at most 16 allocated slots keep their array when they shrink; reallocating to reclaim a few dozen bytes (often to the very same capacity) is not worth the alloc+copy+free. Also make `del list[i]` go through list_resize() like list.pop() does, so it releases memory for large lists. Co-authored-by: Ilya Egorov <0x42005e1f@gmail.com>
1 parent e580c88 commit 11b0122

2 files changed

Lines changed: 14 additions & 1 deletion

File tree

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
Skip the array reallocation when shrinking a small :class:`list`. Patch by
2+
Donghee Na.

‎Objects/listobject.c‎

Lines changed: 12 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -91,6 +91,8 @@ ensure_shared_on_resize(PyListObject *self)
9191
#endif
9292
}
9393

94+
#define LIST_SMALL_ALLOCATED 16
95+
9496
/* Ensure ob_item has room for at least newsize elements, and set
9597
* ob_size to newsize. If newsize > ob_size on entry, the content
9698
* of the new slots at exit is undefined heap trash; it's the caller's
@@ -136,6 +138,15 @@ list_resize(PyListObject *self, Py_ssize_t newsize)
136138
if (newsize == 0)
137139
new_allocated = 0;
138140

141+
// gh-158602: when shrinking, do not reallocate the array of a small list.
142+
if (newsize < allocated) {
143+
if (allocated <= LIST_SMALL_ALLOCATED) {
144+
Py_SET_SIZE(self, newsize);
145+
return 0;
146+
}
147+
assert(new_allocated < (size_t)allocated);
148+
}
149+
139150
ensure_shared_on_resize(self);
140151

141152
#ifdef Py_GIL_DISABLED
@@ -1148,7 +1159,7 @@ list_ass_item_lock_held(PyListObject *a, Py_ssize_t i, PyObject *v)
11481159
for (Py_ssize_t idx = i; idx < size - 1; idx++) {
11491160
FT_ATOMIC_STORE_PTR_RELEASE(a->ob_item[idx], a->ob_item[idx + 1]);
11501161
}
1151-
Py_SET_SIZE(a, size - 1);
1162+
list_resize(a, size - 1); // NB: shrinking a list can't fail
11521163
}
11531164
else {
11541165
FT_ATOMIC_STORE_PTR_RELEASE(a->ob_item[i], Py_NewRef(v));

0 commit comments

Comments
 (0)