Repository navigation
Prefer posix_spawn over fork/exec on every Unix platform - #365
Merged
Merged
Conversation
jakepetroules
marked this pull request as draft
August 13, 2026 07:20
jakepetroules
marked this pull request as ready for review
August 13, 2026 08:44
jakepetroules
force-pushed
the
prefer-posix-spawn
branch
2 times, most recently
from
August 14, 2026 06:11
3977fb6 to
2c436e8
Compare
iCharlesHu
reviewed
Sep 16, 2026
jakepetroules
force-pushed
the
prefer-posix-spawn
branch
from
October 6, 2026 22:07
2c436e8 to
e8c33aa
Compare
iCharlesHu
approved these changes
Oct 7, 2026
This was referenced Oct 7, 2026
Closed
Subprocess used posix_spawn only on Darwin and fork/exec everywhere else, in two near-duplicate implementations that each owned the candidate-path loop, the transient-failure retry, the file-descriptor bookkeeping and the error mapping. Both now live in one shared implementation in Subprocess+Spawn.swift, which prefers posix_spawn and falls back to fork/exec -- or, on Darwin, a pre-fork plus posix_spawn(POSIX_SPAWN_SETEXEC), so Darwin still never reaches a raw execve -- only for configurations posix_spawn cannot express. Five rules decide, against capabilities probed once per process at runtime rather than at compile time, since a binary built against one C library is expected to run against another: 1. userID, groupID or supplementaryGroups is set. No platform has a posix_spawn attribute for setuid, setgid or setgroups. 2. The C library offers no way to close every inherited descriptor. Closing them by enumerating this process's own table would leak any descriptor another thread opens between the enumeration and the spawn, so that is not an acceptable substitute. 3. createSession without POSIX_SPAWN_SETSID (FreeBSD, OpenBSD). 4. createSession together with processGroupID. glibc calls setsid then setpgid, Bionic the reverse, where setsid then fails EPERM because the child already leads a process group. The fallback path controls the order explicitly and reproduces the previous behavior. 5. workingDirectory without an fchdir file action, or without a way to open a descriptor for it (OpenBSD, Android below API 34). Both halves are tested because they are established differently -- one by dlsym at runtime, one by which of O_SEARCH/O_PATH the headers define -- and a working directory silently ignored would be the worst available outcome. Linux with glibc 2.34 or newer, Android 13 or newer and FreeBSD 13.1 or newer therefore stop forking for the common case: posix_spawn on glibc clones with CLONE_VM and CLONE_VFORK, so there is no page-table copy and no fork() in a multithreaded parent. musl, older glibc, older Android and OpenBSD keep fork/exec. On Darwin, createSession no longer forks, since POSIX_SPAWN_SETSID works there. File actions are appended in the order the child must run them, which is load-bearing in three places. The fchdir action goes first, because the descriptor it names can itself be 0, 1 or 2 -- it is opened after the pipes, so it takes the lowest free number, and a caller whose standard descriptors are closed leaves low numbers free -- and a dup2 onto that number would replace the directory before the child reached it. Parent-end closes precede the dup2s, or a close of a parent end that happens to occupy 0, 1 or 2 would close the child's standard stream instead. The close-all action goes last, after everything that still needs a descriptor. The child's signal dispositions are reset through POSIX_SPAWN_SETSIGDEF with every signal except SIGKILL and SIGSTOP. Excluding those two is required, not tidiness: Bionic resets each member of that set unconditionally, and Linux rejects sigaction() on them with EINVAL even for SIG_DFL, whereupon Bionic _exit(127)s a child that posix_spawn still reports as launched. Neither signal can be caught or ignored, so neither can have a disposition to reset anywhere. Swift no longer names posix_spawn_file_actions_t or posix_spawnattr_t. glibc and musl typedef structs, Darwin and the BSDs typedef pointers, and Bionic annotates its mutators _Nonnull, so no single Swift spelling compiles everywhere; the C shim wraps every entry point behind an opaque handle. The public PlatformSpawnAttributes and PlatformSpawnFileActions name the form each platform's C library accepts, and an unavailable function calls the library's mutators through them so a mistake there is a build failure rather than a public API nobody can use. Darwin declares neither: its preSpawnProcessConfigurator predates them and keeps its inout signature, so code that configures the spawn on Darwin and elsewhere still has to branch. Those typealiases have to be public, and on every platform whose C library declares the two types as structs Swift resolves them through the shim rather than through the libc overlay -- even where that overlay is imported publicly. The shim is therefore imported publicly off Darwin, which is also why OpenBSD already needed that import; its FIXME asking why is now answered. A working directory is now applied through an fchdir file action on a descriptor opened in the parent, which replaces the TOCTOU-prone guess about whether a spawn failure meant a missing executable or a missing directory. The open flags come from the shim: chdir needs only execute permission, so O_RDONLY would reject a directory that is a perfectly legal working directory. Only a real O_SEARCH (Darwin, FreeBSD) checks that permission, though -- the kernel zeroes acc_mode for O_PATH, so on glibc, musl and Bionic a directory with mode 000 opens successfully and the child's fchdir then fails EACCES, which is one of the errnos that means "try the next candidate path" and so would surface as executableNotFound. Searchability is therefore checked explicitly, against the returned descriptor rather than the path, so the object checked is the object fchdir will use. Whether that check can ask about effective IDs is another C library question answered in C: AT_EACCESS where it works, 0 on Bionic, which rejects every nonzero faccessat flag and has no set-uid programs for the flag to matter to. Where no search-only open exists the directory is validated without being opened. posix_spawn_file_actions_addfchdir_np is referenced under TARGET_OS_OSX rather than TARGET_OS_MAC, which is every Apple platform: the SDK marks it __API_UNAVAILABLE(ios, tvos, watchos, visionos), and that is a hard error rather than a deprecation warning. Mac Catalyst is included in the exclusion, since it reports as iOS even though it runs on macOS. Leaving the capability unset there routes a working directory through rule 5 to the fallback path. Also observable: working-directory errors carry the errno that actually occurred, and on FreeBSD a process descriptor is available only on the fork/exec path until posix_spawnattr_setprocdescp_np can be required. Both FreeBSD 15.1 follow-ups are recorded as FIXMEs. A setpgid or setsid failure is reported where the C library applies it, as part of posix_spawn, and still silently ignored on the fork/exec path, which is documented on processGroupID. A test-only task-local forces the fallback path, so every spawn-sensitive test runs under both paths on every platform instead of macOS never exercising fork and modern glibc never exercising fork/exec. Which path a spawn is expected to take is derived from the same rule the implementation consults rather than from that override alone, because where posix_spawn cannot express the request -- glibc below 2.34 and Android below API 34 have no close-all mechanism -- the unforced case legitimately takes the fallback too. Fixes swiftlang#172
It is new public API on Linux, Android and the BSDs, and only best-effort there: the fork/exec fallback never builds posix_spawn attributes, so the closure is silently skipped whenever a configuration needs that path. It should go through API review on its own rather than ride along with this change. This also removes the PlatformSpawnAttributes and PlatformSpawnFileActions typealiases, the compile-time check that backed them, and the public import of _SubprocessCShims they required off Darwin. Darwin's configurator is unchanged and is still always called, since Darwin's fallback path applies the same attributes via POSIX_SPAWN_SETEXEC.
It existed only so tests could assert which spawn path a configuration took. Those tests now check end-to-end behavior instead: each still runs under both paths through forceFallbackPathOverride, so the fallback path keeps coverage on platforms whose posix_spawn would otherwise always be used. The routing rules themselves stay covered by the requiresFallbackSpawnPath tests that pass explicit capabilities.
Switch over closeAllViaSpawn instead of testing for each case separately, so the invariant that posix_spawn is never reached without one (requiresFallbackSpawnPath rule 2) is stated and asserted at the point that depends on it. In a release build that invariant breaking fails the spawn rather than leaking every inherited descriptor.
This package does not support iOS, tvOS, watchOS or visionOS, so there is no need to keep the symbol out of their builds.
Like SpawnPathTally, it existed only for tests. The tests that ran under both paths now run once, on whichever path the host selects; the routing rules stay covered by the requiresFallbackSpawnPath tests that pass explicit capabilities.
jakepetroules
force-pushed
the
prefer-posix-spawn
branch
from
October 7, 2026 06:18
e8c33aa to
5e459a9
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Subprocess used
posix_spawnonly on Darwin andfork/execeverywhere else, in two near-duplicate implementations that each owned the candidate-path loop, the transient-failure retry, the file-descriptor bookkeeping and the error mapping.Both now live in one shared implementation in
Subprocess+Spawn.swift, which prefersposix_spawnand falls back tofork/exec— or, on Darwin, a pre-forkplusposix_spawn(POSIX_SPAWN_SETEXEC), so Darwin still never reaches a rawexecve— only for configurationsposix_spawncannot express.When the fallback path is still required
Capabilities are probed once per process at runtime, not at compile time, since a binary built against one C library is expected to run against another.
userID,groupIDorsupplementaryGroupsis set. No platform has aposix_spawnattribute forsetuid,setgidorsetgroups.createSessionwithoutPOSIX_SPAWN_SETSID(FreeBSD, OpenBSD).createSessiontogether withprocessGroupID. glibc callssetsidthensetpgid, Bionic the reverse, wheresetsidthen failsEPERMbecause the child already leads a process group. The fallback path controls the order explicitly and reproduces the previous behavior.workingDirectorywithout anfchdirfile action, or without a way to open a descriptor for it (OpenBSD, Android below API 34). Both halves are tested because they are established differently — one bydlsymat runtime, one by which ofO_SEARCH/O_PATHthe headers define — and a working directory silently ignored would be the worst available outcome.Effect
Linux with glibc 2.34+, Android 13+ and FreeBSD 13.1+ stop forking for the common case:
posix_spawnon glibc clones withCLONE_VMandCLONE_VFORK, so there is no page-table copy and nofork()in a multithreaded parent. musl, older glibc, older Android and OpenBSD keepfork/exec. On Darwin,createSessionno longer forks, sincePOSIX_SPAWN_SETSIDworks there.Why the C shim grew
Swift no longer names
posix_spawn_file_actions_torposix_spawnattr_t. glibc and musl typedef structs, Darwin and the BSDs typedef pointers, and Bionic annotates its mutators_Nonnull, so no single Swift spelling compiles everywhere; the shim wraps every entry point behind an opaque handle. The handles are rebound to Darwin's own types only at the one point that needs them, the existing Darwin-onlypreSpawnProcessConfigurator, whose signature and behavior are unchanged. No configurator is added on other platforms: there it could only be best-effort, since thefork/execfallback never buildsposix_spawnattributes, and it would be new API needing its own review.Working directory
A working directory is now applied through an
fchdirfile action on a descriptor opened in the parent, which replaces the TOCTOU-prone guess about whether a spawn failure meant a missing executable or a missing directory. The open flags come from the shim:chdirneeds only execute permission, soO_RDONLYwould reject a directory that is a perfectly legal working directory.Only a real
O_SEARCH(Darwin, FreeBSD) actually checks that permission, though. The kernel zeroesacc_modeforO_PATH, so on glibc, musl and Bionic a directory with mode000opens successfully and the child'sfchdirthen failsEACCES— one of the errnos meaning "try the next candidate path", so it would surface asexecutableNotFound, the very misdiagnosis this replaces. Searchability is therefore checked explicitly, against the returned descriptor rather than the path, so the object checked is the objectfchdirwill use and no TOCTOU window is added. Whether that check can ask about effective IDs is another C library question answered in C:AT_EACCESSwhere it works,0on Bionic, which rejects every nonzerofaccessatflag and has no set-uid programs for the flag to matter to. Where no search-only open exists the directory is validated without being opened.Using a descriptor rather than a path-based
addchdiris also groundwork: a future API could let callers specify the working directory as a file descriptor, for those who want to name an exact directory object rather than a path, and reuse this same path under the hood.Ordering that is load-bearing
File actions are appended in the order the child must run them, which matters in three places. The
fchdiraction goes first, because the descriptor it names can itself be 0, 1 or 2 — it is opened after the pipes, so it takes the lowest free number, and a caller whose standard descriptors are closed leaves low numbers free — and adup2onto that number would replace the directory before the child reached it. Parent-end closes precede thedup2s, or a close of a parent end that happens to occupy 0, 1 or 2 would close the child's standard stream instead. The close-all action goes last, after everything that still needs a descriptor.The child's signal dispositions are reset through
POSIX_SPAWN_SETSIGDEFwith every signal exceptSIGKILLandSIGSTOP. Excluding those two is required rather than tidiness: Bionic resets each member of that set unconditionally, and Linux rejectssigaction()on them withEINVALeven forSIG_DFL, whereupon Bionic_exit(127)s a child thatposix_spawnstill reports as successfully launched. Neither signal can be caught or ignored, so neither can have a disposition needing a reset on any platform.Other observable changes
A
setpgidorsetsidfailure is reported rather than silently ignored where the C library applies it, as part ofposix_spawn; the fork/exec path still discards both results, which is now documented onprocessGroupID. Working-directory errors carry the errno that actually occurred, and on FreeBSD a process descriptor is available only on thefork/execpath untilposix_spawnattr_setprocdescp_npcan be required. Both FreeBSD 15.1 follow-ups are recorded as FIXMEs.Testing
Tests check end-to-end behavior on whichever path the host selects; nothing in the library exists solely for tests. The path-selection rules are covered by tests of
requiresFallbackSpawnPaththat pass explicit capability sets, so every rule is exercised regardless of the host's C library.CI is what validates this change: its entire behavioral effect lands on platforms that cannot be exercised on a development Mac. The matrix covers glibc across five distros on 6.2 and nightly, the 4.18 and 5.14 kernels under a modern userspace, Android on NDK r27d and r29, FreeBSD, musl static-SDK builds, Windows, macOS, and the CMake build that does not glob sources. OpenBSD, where every spawn takes the fallback path, is not in CI (it is a manually tested platform), but the full test suite passed there on an earlier revision of this PR via a temporary QEMU job, since removed. In CI the fallback path runs for the configurations that require it, such as
createSessioncombined withprocessGroupID.Found but not fixed here
A review of this change turned up three pre-existing defects, each of which behaves the same before and after it. They are out of scope here (this PR is already large, and the fixes touch the
fork/execpath) and are tracked separately:supplementaryGroups: []retains the parent's supplementary groups instead of dropping themfork/execfallback leaks every descriptor into the child when/dev/fdcannot be openedFixes #172