One binary for glibc and musl distros - #76
Open
unxed wants to merge 26 commits into
Open
Conversation
Binaries importing goffi are dynamically linked even with CGO_ENABLED=0 and
-ldflags "-extldflags '-static'". The cause is //go:cgo_import_dynamic: the Go
linker emits PT_INTERP and DT_NEEDED as soon as any package in the build
carries one, under internal linking too. -extldflags is inert in such builds
because no external linker ever runs, so the flag is accepted and ignored.
goffi emits those directives from three packages, and all three must go before
the interpreter disappears:
internal/dl dlopen, dlsym, dlerror, dlclose libdl.so.2
internal/syscall __errno_location libc.so.6
internal/fakecgo malloc, pthread_*, sigaltstack libc.so.6, libpthread.so.0
Add a goffi_static build tag that compiles all of them out. The directives move
into dl_{linux,darwin,freebsd}_dynamic.go so the RTLD_* constants stay
available in both modes; the dlopen wrappers, errno trampolines, callback
dispatcher and the fakecgo import are gated behind !goffi_static and replaced
by stubs.
There is no way to keep FFI in a static binary on Linux: dlopen is a service of
the dynamic loader, which a static executable has no way to reach. So the tag
disables FFI rather than pretending. The API keeps its shape, LoadLibrary,
GetSymbol and CallFunction return an error wrapping the new ffi.ErrStaticBuild,
and ffi.Available() reports the mode as a compile-time constant so callers can
branch to a pure-Go fallback and have the linker drop the dead path.
The callback dispatcher needs separate handling: it enters Go through
crosscall2, supplied by internal/fakecgo. Without that package the symbol is an
unresolved relocation, and the library currently links only because dead code
elimination removes it. NewCallback panics in a static build instead.
The tag is a no-op on Windows and Android, which resolve symbols through
kernel32 and Bionic and are dynamically linked by construction. Available()
stays true there, so a build matrix can pass the tag on every target without
special-casing mobile.
ffi/static_link_test.go builds a real consumer binary for linux/amd64 and
linux/arm64 and inspects it with debug/elf; the test binary itself cannot be
used, since the harness always links the dynamic build. scripts/check-static.sh
and a CI job wrap that together with the cross-platform compile checks.
Refs: unxed/f4#693
A default goffi binary cannot start on Alpine: PT_INTERP names the glibc
loader (which musl systems do not have), and the cgo_import_dynamic
directives name libdl.so.2 / libc.so.6 / libpthread.so.0, none of which
musl ships -- its whole POSIX surface lives in one arch-named object,
libc.musl-<arch>.so.1. Both facts are baked into the ELF at link time,
so the libc flavor is a build-time choice.
The goffi_musl tag selects musl flavors of the three directive groups
(internal/dl, internal/syscall, internal/fakecgo) and bakes the musl
loader path into PT_INTERP via //go:cgo_dynamic_linker. That directive
is restricted to cgo-generated code, so musl builds pass
-gcflags=github.com/go-webgpu/goffi/internal/dl=-std -- a deliberately
loud failure mode: forgetting the flag is a compile error naming the
directive, not a binary that dies at startup with a confusing ENOENT.
One symbol is dropped from the musl set: pthread_get_stacksize_np is a
Darwin-only API that glibc's lazy PLT silently tolerates but musl's
immediate binding would fatally reject at load time. Its trampoline is
only reachable from the Darwin thread-entry path, so the linker
dead-code-eliminates it on Linux. The fakecgo musl files are produced
by gen.go from the same symbol tables as the glibc ones.
Verification:
- TestMuslDirectiveParity pins glibc/musl symbol-set parity (including
the one intentional exclusion), per-arch SONAMEs and the interpreter
directive, so the flavors cannot drift apart silently.
- TestMuslLinkArtifacts builds linux/{amd64,arm64} probes and asserts
PT_INTERP and DT_NEEDED with debug/elf.
- cmd/musl-probe runs the full machinery against a real musl libc:
dlopen/dlsym, float and integer calls, errno capture through
__errno_location, qsort with a Go callback (crosscall2), and a
64-goroutine hammer forcing the runtime to create OS threads through
fakecgo's pthread imports. scripts/check-musl.sh executes it inside
an Alpine userland (docker, or a sha256-pinned minirootfs via chroot,
or the musl loader invoked directly) and is wired into CI.
Verified on Alpine 3.24.1: all probe checks pass on first run; glibc
and goffi_static test suites unchanged. Pre-existing arm64 vet warnings
in ffi/callback_arm64.go (present on master) are out of scope.
Tag interplay: goffi_static wins over goffi_musl; the tag is inert off
Linux. See docs/MUSL.md.
A callback frame is filled in by the native caller: the arguments arrive as raw
machine words in registers and stack slots. Turning one of those words back into
a pointer is what checkptr objects to, and since -race implies -d=checkptr, any
race-detector run that reaches a callback died on the spot:
fatal error: checkptr: pointer arithmetic result points to invalid allocation
runtime.checkptrArithmetic
github.com/go-webgpu/goffi/ffi.callbackWrap callback_arm64.go:189
Every one of the eight sites is the same case, and it is the benign one: the
memory belongs to the caller, so the Go collector neither owns nor moves it, and
the conversion is correct. The compiler simply cannot recover that from an
integer. So the conversion stays and moves into one helper, pointerFromNative,
marked //go:nocheckptr with the reason written next to it. There is no site here
of the other kind -- a Go pointer this package itself laundered through a
uintptr -- and if one ever appears it needs the opposite treatment, staying
pointer-typed so the collector can keep it alive, not a pragma.
amd64 was already working around this by reading the slot as
*(*unsafe.Pointer)(unsafe.Pointer(&frame[pos])), which sidesteps the check by
never naming a uintptr. That worked, but it made the two architectures read
differently for no reason a reader could see, and it left the arm64 side to
crash. Both now say the same thing.
The //nolint:govet,gosec comments that sat on the call sites move to the helper
with them. Worth noting they were never what kept checkptr quiet: those silence
linters, and checkptr is a runtime check.
The ffi package now passes go test -race. What still fails there is unrelated
and unchanged: TestExecuteCaptureRegistersSimple in internal/arch/arm64 fails
with and without -race on the branch point, and three more in that package fail
under -race both before and after.
fix: route callback pointer arguments through one nocheckptr helper
Snapshot of in-progress work. DOES NOT COMPILE YET (mid-refactor).
Design (validated on real glibc and musl):
One portable CGO-free binary that does live in-process FFI on both libcs via
(1) empty-SONAME cgo_import_dynamic imports -> no DT_NEEDED, no libc pinning
(2) stripped PT_INTERP -> kernel loads it on any distro
(3) early re-exec through the host's own loader with the host libc --preload
(done at the top of x_cgo_init, before any libc symbol is touched)
This mirrors static-everywhere's 'launch with the host's own loader, found at
runtime' doctrine and needs no in-process ELF loader (no SoLo).
In this snapshot:
- Retag glibc/musl dl/errno/fakecgo import files as !goffi_universal
- internal/dl/dl_universal.go empty-SONAME dlopen/dlsym/dlerror/dlclose
- internal/syscall/errno_universal.go empty-SONAME __errno_location
- internal/fakecgo/symbols_universal.go empty-SONAME malloc/pthread/... (musl set)
- internal/fakecgo/reexec_syscall_amd64.s raw libc-free syscall primitive
Pending (see conversation notes): reexec logic + arm64 stub + per-arch tables,
noop shim + x_cgo_init call site, cmd/goffi-strip-interp, scripts/build-universal.sh,
internal/loader + ffi.HostLoader/HostLibC/LibcKind, cmd/goffi-audit, cmd/universal-probe,
tests, both-libc CI, purego-coexistence CI (nofakecgo), docs/PROFILE_U.md + attribution.
Delta over the first WIP commit. Everything here compiles under CGO_ENABLED=0
in all modes (default / goffi_universal / goffi_static; goffi_musl with its
usual -gcflags). The universal runtime path is NOT working yet: a TLS-before-
setup blocker is diagnosed and documented, with the fix designed but not landed.
New:
- internal/fakecgo/reexec_universal_linux.go
Full libc-free early re-exec: mmap scratch, read /proc/self/{environ,
cmdline}, resolve host loader (glibc-first, musl fallback), build argv/
envp, execve(<loader> --preload <soname> /proc/self/exe ...). Uses only
raw syscalls; pointer-global stores avoided (uintptr) to dodge GC write
barriers this early. Carries a KNOWN-ISSUE note: on the first (kernel-
direct) launch %fs/TLS is not set up (rt0_go delegates TLS to _cgo_init),
so the compiler's post-ABI0-call "MOVQ FS:-8, R14" g-reload faults before
execve. Fix in progress: a per-arch setupUniversalTLS asm shim.
- internal/fakecgo/reexec_noop.go no-op bridge for non-universal linux
- cmd/universal-probe/main.go runtime FFI probe (host libc auto-detect)
- cmd/goffi-strip-interp/main.go PT_INTERP -> PT_NULL post-link tool
- scripts/build-universal.sh build + strip-interp helper
Changed:
- internal/fakecgo/go_linux_{amd64,arm64}.go
call maybeReexecUniversal() at the very top of x_cgo_init (before malloc)
Diagnosis captured (asm_amd64.s rt0_go: "JZ needtls" only when _cgo_init==nil)
so the remaining work is well-scoped.
Self-contained continuation document so another contributor (or a smaller
model) can pick the work up without the original conversation:
- the task and its constraints (single portable CGO-free binary running FFI
on both glibc and musl; no purego conflict; both-libc CI; branch delivery);
- the validated design (empty-SONAME imports -> no DT_NEEDED, strip PT_INTERP,
early re-exec through the host loader with --preload), with the empirical
evidence on glibc and musl;
- THE current blocker and its designed fix: on the first kernel-direct launch
%fs/TLS is not set up (rt0_go delegates TLS to _cgo_init), so the compiler's
post-ABI0-call g-reload faults before execve; fix = a per-arch
setupUniversalTLS asm shim called first in x_cgo_init;
- a file-by-file account of what already exists and compiles;
- the ordered remaining work (loader package + public API, goffi-audit,
purego coexistence job, tests, both-libc CI, docs/attribution);
- build/test commands, honesty notes, and delivery/push instructions;
- a "START HERE" pointer to the immediate next action.
Docs-only; no code changes.
On the first, kernel-direct launch of a universal binary there is no host
ld.so, so the thread pointer (%fs on amd64, TPIDR_EL0 on arm64) is unset:
rt0_go delegates TLS setup to _cgo_init (the "JZ needtls" branch is taken
only when _cgo_init is nil), and the compiler's post-ABI0-call g-reload
(MOVQ FS:-8, R14) then faults before the re-exec bridge can run.
setupUniversalTLS points the thread pointer at a scratch mmap page, but only
when it is not yet set up; on the re-executed launch the host loader has
configured real TLS and the shim is a no-op. It is called as the very first
statement of x_cgo_init, before maybeReexecUniversal, so every subsequent
ABI0-call g-reload reads mapped memory. The fake g it exposes is never
dereferenced before execve.
- setup_universal_tls.go prototype + amd64 scratch slot (universal)
- setup_universal_tls_amd64.s arch_prctl(GET_FS/SET_FS) + mmap
- setup_universal_tls_arm64.s MRS/MSR TPIDR_EL0 + mmap (cross-asm only)
- setup_universal_tls_noop.go no-op for default/goffi_musl
- go_linux_{amd64,arm64}.go call setupUniversalTLS() first in x_cgo_init
Verified: with this shim the first launch now sets up TLS and re-execs
through the host loader (previously it segfaulted at the first g-reload).
musl is end-to-end green (the re-executed no-interp binary binds the
empty-SONAME symbols under the musl loader and passes the full FFI probe).
Still open (tracked in docs/PROFILE_U_PLAN.md): glibc's ld.so does not bind
the empty-SONAME symbols of a PT_INTERP-stripped binary, so the glibc
re-exec path is not yet green; musl's loader does. Also the universal probe
looks for sqrt in libc, which holds on musl but not glibc (sqrt is in libm).
… on both libcs
The empty-SONAME + stripped-PT_INTERP + re-exec design relies on the host
loader binding the undefined symbols under --preload. Empirically the two
loaders differ: musl's ld.so binds a no-PT_INTERP main object directly, but
glibc's ld.so only binds a re-exec'd main object that carries a PT_INTERP.
So the bridge now branches on the detected libc:
- musl -> re-exec the on-disk binary as-is (musl binds it).
- glibc -> copy /proc/self/exe into a memfd with the PT_INTERP header
restored (only the program header's p_type was cleared when the
interp was stripped; the .interp string is intact, so restoring
it is a single field write), then re-exec the loader against
/proc/self/fd/<memfd>. The memfd is created without MFD_CLOEXEC
(it must survive execve) and prefers MFD_EXEC with a fallback to
flags=0 on pre-6.3 kernels.
New raw-syscall helpers, all //go:nosplit and heap-free (this runs before the
Go scheduler): restoreInterpToMemfd, copyExeToMemfd, patchInterpInBuf,
procFdPath, plus memfd_create in the amd64/arm64 syscall tables.
The universal probe's floating-point check moves from sqrt (which glibc keeps
in libm, so it will not resolve from the libc handle) to atof (in libc on both
glibc and musl).
Verified end-to-end with ONE stripped binary, autonomous re-exec (no manual
--preload):
glibc host -> UNIVERSAL-PROBE-OK
musl (Alpine, /proc mounted, chroot) -> UNIVERSAL-PROBE-OK
Both bind libc, load it via dlopen, call atof/strlen/getpid, run a Go
comparator through qsort, and hammer 32 OS threads through pthread_create.
ELF contract holds: no PT_INTERP, no DT_NEEDED. Existing tests still pass.
arm64 remains cross-compile-verified only.
Also updates docs/PROFILE_U_PLAN.md to mark the TLS shim and the glibc-binding
problem resolved and to repoint START HERE at the remaining work.
…Kind
The auditable "Profile U" loader table, as a plain-Go package usable from any
build mode (not just the universal, nosplit re-exec bridge). For each targeted
architecture (linux/amd64, linux/arm64) it records the host dynamic loader path
and libc SONAME for glibc and musl, and Detect() probes the running host --
glibc first, then musl -- exactly as the re-exec bridge chooses. Unsupported
architectures report KindUnknown.
Public API on the ffi package:
- ffi.HostLoader() string -- path to the host ld.so (or "")
- ffi.HostLibC() string -- host libc SONAME (or "")
- ffi.LibcKind() string -- "glibc" / "musl" / "unknown"
Tests:
- internal/loader: Kind.String and Detect consistency.
- internal/fakecgo (goffi_universal): assert the bridge's reexec_table_*.go
constants match internal/loader's table, so the two descriptions of the
host cannot drift apart.
Verified: compiles in default/universal/static/musl and on arm64; on a glibc
host the API returns loader=/lib64/ld-linux-x86-64.so.2 libc=libc.so.6
kind=glibc.
cmd/goffi-audit opens a binary with debug/elf and asserts the Profile U ELF contract: no PT_INTERP program header and no DT_NEEDED entries. Non-zero exit on failure; accepts multiple paths. The portable equivalent of static-everywhere's onebin profile check, and what scripts/build-universal.sh points at. ffi/universal_link_test.go builds a CGO-free -tags goffi_universal binary and asserts it has no DT_NEEDED (the empty-SONAME imports must not pull in a specific libc). The interp strip is a separate build step covered by goffi-audit, so it is not asserted here. Guarded by testing.Short(). Verified: audit passes a stripped universal binary and fails a default glibc build (PT_INTERP + libc.so.6/libdl.so.2/libpthread.so.0); the link test passes.
.github/workflows/universal.yml:
- build: build every mode (default/universal/static/musl), vet, unit tests
(incl. the goffi_universal sync test), then build+strip the universal probe
and assert the Profile U contract with cmd/goffi-audit; upload the binary.
- run-glibc: download that SAME binary and run it under debian:stable-slim
and ubuntu:24.04, expecting UNIVERSAL-PROBE-OK.
- run-musl: run the SAME binary under alpine:latest (musl), expecting
UNIVERSAL-PROBE-OK.
- purego-coexistence: build and run a CGO-free program importing both goffi
and purego with -tags nofakecgo.
This is the "one binary, both libc" guarantee wired into CI. YAML validated
locally; the actual Actions run cannot be exercised in the dev sandbox, so the
first real run may need minor tweaks (runner images, action versions).
…/solo)
- docs/PROFILE_U.md: how to build a universal binary, how the empty-SONAME /
stripped-PT_INTERP / re-exec mechanism works, the public HostLoader/
HostLibC/LibcKind API, limitations, and attribution.
- NOTICE: credit unxed/static-everywhere for the Profile U concept and note
pg83/solo as the (intentionally not vendored) in-process loader.
- docs/MUSL.md and README.md: short pointers to the universal build.
The universal workflow ran a bare 'go vet ./...', which trips the unsafeptr analyzer on ffi/callback_pointer.go — the deliberate //go:nocheckptr helper that reconstructs a pointer from a native callback register. ci.yml never hit this because it runs vet through golangci-lint, whose .golangci.yml excludes govet on exactly those paths. Pass -unsafeptr=false so the bare vet enforces the same contract, and pin go-version to 1.26.x as ci.yml does (the goffi_musl build's -gcflags workaround is calibrated against that toolchain).
The hand-off plan still opened with "work in progress, the universal runtime path does not work yet" and pointed START HERE at \xc2\xa75.2, although \xc2\xa75.2-\xc2\xa75.7 (loader package, audit tool, tests, CI, docs) have all landed. Mark them DONE, demote \xc2\xa73 to a historical record of the fixed TLS blocker, and repoint START HERE at what actually remains: a green CI run (the unit tests on this branch have never executed \xe2\x80\x94 vet failed first), real-hardware arm64 verification, and release notes. Also document why -tags nofakecgo is not available in universal mode (it drops the re-exec bridge with fakecgo) and that unxed/pureffi is therefore the way to satisfy a purego-API dependency here \xe2\x80\x94 it carries no fakecgo of its own and needs no changes for this branch.
Clears all 12 issues reported by golangci-lint 2.13.2 on the profile_u
branch, without changing any observable behaviour.
errcheck (6):
- goffi-audit, musl-probe, universal-probe: wrap the deferred read-only
Close()/FreeLibrary() calls whose errors are genuinely uninteresting in
`func(){ _ = ... }()`.
- goffi-strip-interp: the read-only elf.File Close is discarded with
`_ =`, while the read-write file handle now reports its Close error via
a named return without clobbering an earlier WriteAt/ReadFull error, so
a failed flush on close is no longer silently dropped.
- universal-probe thread-hammer goroutine: the CallFunction result is
intentionally discarded, now made explicit with `_, _ =`.
govet shadow (4):
- musl-probe: the three CallFunction checks in main() reused the existing
outer err via `=` instead of shadowing it with `:=`.
- static_link_test: the inner DynString err is renamed to derr so it no
longer shadows the elf.Open err.
staticcheck SA1019 (2):
- static_link_test / musl_link_test: runtime.GOROOT (deprecated in Go
1.24) is replaced by a shared goToolPath() helper that queries
`go env GOROOT` and falls back to PATH. The now-unused runtime import is
dropped from musl_link_test.
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
Three independent CI failures on profile_u, none related to the earlier lint fix (Lint stays green). Android arm64 (Check Android arm64): the gate regenerates the fakecgo sources from internal/fakecgo/gen.go and diffs them against the committed files. symbols_linux.go carried a hand-added `&& !goffi_universal` build constraint (required so it does not collide with symbols_universal.go in a universal build), but the generator never emitted it, so the check reported the file as stale. Reverting the file is not an option — it would break the universal build — so the generator is taught to emit the tag for the linux symbols file. The two musl symbol files had the same latent drift (the gate does not check them, but regenerating would have silently dropped their `!goffi_universal` tag and broken the universal build), so they are fixed in the same place. No generated file changes: regeneration now reproduces every committed file byte-for-byte under Go 1.26.5. Cross-Compile (examples build): two masked failures. - examples/simple and examples/struct declared `go 1.25`, which is lower than the goffi module's `go 1.25.0` they depend on via replace, so `go build` demanded `go mod tidy`. Both are bumped to `go 1.25.0`. - With that resolved, examples/struct then failed because structlib.c sits in the package directory and the package does not use cgo, so `go build` rejected the stray C source. The file is compiled with gcc and loaded at run time, not linked into the package, so it is moved to a csrc/ subdirectory (out of the package file set) and embedded via //go:embed; buildLib writes it to its temp dir before compiling. This also drops the fragile os.Args[0]/CWD source lookup, so the example runs even when the binary is relocated. Codecov patch coverage: add ffi/hostinfo_test.go covering the three public host-inspection helpers (HostLoader/HostLibC/LibcKind), which were the new, trivially testable uncovered lines. The static-build guard in call.go is a compile-time-false branch outside goffi_static and is left uncovered by design. Codecov patch is informational and not a merge gate regardless.
The Android gate got past the fakecgo staleness check and the tests, then
failed at `go vet ./...` with:
ffi/callback_pointer.go:14:9: possible misuse of unsafe.Pointer
pointerFromNative reconstructs a pointer from a native callback register or
stack slot: the native caller owns the memory, so the uintptr->unsafe.Pointer
conversion is intentional and carries a //go:nocheckptr contract. The
function is already annotated with //nolint:govet,gosec, but that only
suppresses golangci-lint; a bare `go vet` honors no such comment, and unlike
`go test` (which runs a reduced vet subset) it runs the full analyzer set,
including unsafeptr. That is why Lint and the Test jobs stay green while this
explicit vet does not.
universal.yml already runs `go vet -unsafeptr=false ./...` for exactly this
reason, documenting it as the bare-vet equivalent of the golangci-lint govet
exclusion on these FFI paths. This applies the same flag to the two vet
invocations in check-android-arm64.sh so the Android gate matches. Only the
unsafeptr analyzer is disabled; every other vet check still runs.
Raises coverage of the files Codecov flagged on the PR (ffi 90.8% -> 94.8% of statements), adding tests only; no production code changes. callback.go (callbackWrap 88.5% -> 100%): the existing callback tests never pass pointer, unsafe.Pointer, or stack-spilled arguments, nor a pointer return. New tests drive callbackWrap directly with a hand-built System V AMD64 argument frame (the same technique callback_struct_args_test.go uses) to exercise: a typed pointer and an unsafe.Pointer in an integer register and spilled to the stack, a bool spilled to the stack, and a pointer return value. Guarded to linux/darwin/freebsd + amd64 + !goffi_static, matching the frame layout these paths assume. call.go (executeFunction 66.7% -> 83.3%): add a test for the guard that returns ErrUnsupportedArchitecture when no architecture caller is registered, by swapping arch.Registry.Caller to nil and restoring it (ffi tests run sequentially, so no concurrent call observes the nil). The one remaining uncovered line is the `if static.Enabled` guard returning ErrStaticBuild: static.Enabled is a compile-time-false constant outside goffi_static, so that branch is unreachable in the CGO_ENABLED=0 build Codecov measures, and can only be covered by a static-tagged build. Verified with -race and under GOOS=android GOARCH=arm64 (the amd64-only callback test is correctly excluded there); golangci-lint clean.
The target branch's protection requires an "Android arm64 (Go 1.25.12)"
status check, but this branch's matrix only produced the 1.26.5 and 1.26.x
jobs, so that required check was never reported and the PR could not merge.
Add 1.25.12 to the android-cross matrix so the job (named
"Android arm64 (Go ${{ matrix.go }})") produces the expected check, and add
go1.25.12 to check-android-arm64.sh's audited-toolchain allowlist so the
version tripwire admits it.
The allowlist exists because this port depends on the Go runtime's Android
arm64 startup ABI, which must be re-audited per Go version. That audit was
done for 1.25.12: the _cgo_init callsite in runtime/asm_arm64.s, the
tls_g slot-2 offset in tls_arm64.s, and the TLS_SLOT_APP / API-level probe
in runtime/cgo/gcc_android.c are byte-for-byte the same invariants the gate
already checks for 1.26.x; the generated fakecgo sources regenerate
identically; the cross-compiled android/arm64 objects pass the trampoline,
TLS-guard, and dlerror-capture objdump checks; and `go vet` and the test
build succeed under 1.25.12. The tripwire still rejects unaudited versions.
A universal binary that cannot find a host loader had nothing to re-exec
through, so it never bound a libc -- and then went on to call one. The bridge
printed "no known host dynamic loader found; FFI unavailable" and returned,
x_cgo_init continued straight into malloc() and the pthread_attr_* trio, and
the process died on an unbound symbol before main. The message was accurate
about FFI and wrong about everything else: nothing was available, including
the parts of the program that never wanted FFI.
That is a real class of host -- a scratch container, a distribution that keeps
ld.so somewhere else -- and it is the one thing a goffi_static build could do
that universal mode could not, which is awkward when universal mode is meant
to replace it.
So the bridge now reports its outcome instead of only its failure. Its body
becomes reexecUniversal, returning whether this process has a libc: true only
on the guard-variable path (we already came back through the host loader),
false on every path that leaves the empty-SONAME imports unbound. When it is
false, maybeReexecUniversal records that in the new internal/hostlibc package,
which is deliberately dependency-free because it is written from x_cgo_init,
before the runtime is up, where only a plain store to a package-level variable
is safe.
x_cgo_init then returns early rather than touching libc, and clears
runtime.iscgo on the way out. That is what makes the process survivable: the
runtime read _cgo_init long before this and took the branch that delegates TLS
setup to us (setupUniversalTLS already did it, onto a scratch page), but every
later decision -- pthread_create versus clone(2) for a new M, g in TLS versus
the g register on arm64, how signal handlers are installed -- is made by
reading iscgo at the point of use. False from here on means the runtime never
reaches for the libc that is not there and threads it starts set up their own
TLS, exactly as in a CGO_ENABLED=0 build. g.stacklo keeps the bounds rt0_go
computed; the pthread_attr_getstacksize refinement is what a non-cgo binary
does without anyway.
The FFI surface then has to answer for the state rather than fault in it:
- internal/dl's Dlopen, Dlsym and Dlclose return hostlibc.ErrMissing.
- ffi.CallFunction's guard sits next to the existing goffi_static one.
- ffi.Available() reports false, which is the honest answer and now a
run-time one: the same binary has full FFI on any host with a glibc or
musl loader, so this cannot be decided from build tags.
- ffi.ErrNoHostLibc exposes the sentinel, mirroring ErrStaticBuild.
Verified on linux/amd64 with a rootfs containing no loader at all: goffi's own
universal-probe, a 64-thread allocate-and-GC stress binary, and a large real
application (unxed/f4) all started, ran and exited 0 there, where every one of
them took SIGSEGV before this change; LoadLibrary returns the wrapped sentinel
and Available() is false. Unchanged on hosts that do have a loader -- the
probe is still green end to end on glibc and on musl (Alpine), for the amd64
and arm64 binaries alike. gofmt, go vet -unsafeptr=false and the test suite
pass in the default, universal and static modes.
…kes them A universal binary reaches libc by re-execing itself through the host loader, and that execve costs the process its own identity. Afterwards /proc/self/exe names the loader, so os.Executable answers with /usr/lib/<triplet>/ld-linux-*.so.2; argv[0] names the image the loader was handed, which on glibc is a memfd copy and names no file at all. Nothing in the process knows better, because the only moment at which both facts are still true is inside the bridge, just before it calls execve. That is a real loss rather than a curiosity. unxed/f4 hit both halves of it: its updater takes filepath.Dir(os.Executable()) as the install directory and its writeFileSafe escalates through sudo when a write is refused, so an update from a universal build unpacks the release into the system library directory next to the loader -- while the binary the user actually runs is untouched. Portable-mode detection looks for ini files in the same wrong place. So the bridge now writes down what it is about to take: GOFFI_UNIVERSAL_EXE=<pid>:<readlink /proc/self/exe> GOFFI_UNIVERSAL_ARGV0=<pid>:<original argv[0]> Both are pid-tagged. The environment is inherited by every child, and a child that read GOFFI_UNIVERSAL_EXE as being about itself would get its parent's binary -- exactly the class of wrong answer this exists to remove. execve keeps the pid, so the re-executed process still matches, and a child (a new pid) is told nothing rather than something false. ffi.Executable and ffi.Argv0 read them back, falling through to os.Executable and os.Args[0] wherever there was no re-exec, so callers do not have to know which build they are in. Both values are recorded only when they are real: an unreadable /proc/self/exe or an empty /proc/self/cmdline leaves the variable out, and a missing variable is a better answer than a guessed one. Everything here still runs before libc exists, so taggedEnv formats into the same mmap staging buffer the rest of the bridge uses, with no allocation and no libc; getpid is the only syscall added, and it is in both architecture tables. Also documents in PROFILE_U.md what argv[0] and /proc/self/exe really hold after the re-exec -- the old note said argv[0] became "the resolved executable path", which is true only on musl -- and how to start another copy of a universal binary, since the inherited guard makes the obvious exec.Command(os.Args[0]) fatal. Verified on glibc/amd64 with a stripped universal binary: ffi.Executable() returns the on-disk path where os.Executable() returns the loader, ffi.Argv0() returns the invocation name, and a child process correctly declines the inherited record. Cross-builds clean for arm64.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Inspired by https://github.com/unxed/static-everywhere Profile U