Skip to content

PHONY: stable-ghc-9.14 additions - #119

Draft
angerman wants to merge 381 commits into
ghc-9.14from
stable-ghc-9.14
Draft

angerman wants to merge 381 commits into
ghc-9.14from
stable-ghc-9.14

Conversation

@angerman

@angerman angerman commented Nov 29, 2025

Copy link
Copy Markdown

Summary

This PR tracks all additions in stable-ghc-9.14 relative to upstream ghc-9.14.

Cabal-based Multi-Stage Build System

  • Modularize RTS: extract headers (rts-headers) and filesystem utilities (rts-fs) into separate packages
  • Implement cabal-based multi-stage build system (stage0 → stage1 → stage2)
  • Split RTS into sub-libraries (threaded/non-threaded, debug/nodebug variants)
  • Add -no-rts compiler flag for bootstrap builds

Static Linking Improvements

  • Add -fully-static and -exclude-static-external flags
  • Proper support for statically linking executables
  • Better error handling when linking statically
  • Ensure extra-libraries-static is consistently defined

Bundled libffi

  • Add libffi-clib as bundled library (replaces system libffi dependency)
  • Enable PIC on linux/freebsd x86_64

Build System & Tooling

  • Add ghc-toolchain --output-settings support
  • Add genprimopcode --wrappers/--prim-module options
  • Add ghc-config additional fields
  • Add ghc-pkg --target support and mermaid diagram generation
  • Better "could not execute" error messages

CI & Testing

  • Add release workflow
  • Fix various test fragility (T13786, T7040_ghci, T25240, T20604)
  • Skip problematic tests (T14999, uniques test without git repo)
  • Testsuite adjustments for RTS split

Fixes

  • Fix FreeBSD stage2/stage3 builds
  • Fix header copying in Makefile
  • Fix preprocessor flags in RTS
  • Allow building with boot compiler lacking ghc-internal
  • Warn when -dynamic is mixed with -staticlib

TerrorJack and others added 25 commits June 10, 2026 22:07
This patch fixes wasm backend's post-linker output script to ensure
it's synchronous ESM and doesn't use top-level await, which doesn't
work in ServiceWorkers. Fixes #27257.

(cherry picked from commit cccf45d)
This commit stops GHC from emitting spurious incomplete record selector
warnings for bare selectors/projections such as .fld

There are two places we currently emit incomplete record selector
warnings:

  1. In the desugarer, when we see a record selector or an occurrence
     of 'getField'. Here, we can use pattern matching information to
     ensure we don't give false positives.
  2. In the typechecker, which might sometimes give false positives but
     can emit warnings in cases that the pattern match checker would
     otherwise miss.

This is explained in Note [Detecting incomplete record selectors]
in GHC.HsToCore.Pmc.

Now, we obviously don't want to emit the same error twice, and generally
we prefer (1), as those messages contain fewer false positives. So we
suppress (2) when we are sure we are going to emit (1); the logic for
doing so is in GHC.Tc.Instance.Class.warnIncompleteRecSel,
and works by looking at the CtOrigin.

Now, the issue was that this logic handled explicit record selectors as
well as overloaded record field selectors such as "x.r" (which turns
into a simple GetFieldOrigin CtOrigin), but it didn't properly handle
record projectors like ".fld" or ".fld1.fld2" (which result in other
CtOrigins such as 'RecordFieldProjectionOrigin').

To solve this problem, we re-use the 'isHasFieldOrigin' introduced in
fbdc623 (slightly adjusted).

On the way, we also had to update the desugarer with special handling
for the 'ExpandedThingTc' case in 'ds_app', to make sure that
'ds_app_var' sees all the type arguments to 'getField' in order for it
to indeed emit warnings like in (1).

Fixes #26686

(cherry picked from commit 52c3e6b)
When we are simplifying the RHS of a rule, we make sure to only apply
rewrites from rules that are active throughout the original rule's
range of active phases.

For example, if a rule is always active, we only fire rules that are
themselves always active when simplifying the RHS. Ditto for inline
activations.

This is achieved by setting the simplifier phase to a range of phases,
using the new SimplPhaseRange constructor. Then:

  1. When simplifying the RHS of a rule, or of a stable unfolding,
     we set the simplifier phase to a range of phases, computed from
     the activation of the RULE/unfolding activation, using the
     function 'phaseFromActivation'.

     The details are explained in Note [What is active in the RHS of a RULE?]
     in GHC.Core.Opt.Simplify.Utils.

  2. The activation check for other rules and inlinings is then:
     does the activation of the other rule/inlining cover the whole
     phase range set in sm_phase? This continues to use the 'isActive'
     function, which now accounts for phase ranges.

On the way, this commit also moves the exact-print SourceText annotation
from the Activation datatype to the ActivationAnn type. This keeps the
main Activation datatype free of any extra cruft.

Fixes #26323

(cherry picked from commit 2da84b7)
This MR fixes a bad loop in the compiler: #26826.

The fix is to add (WAR2) to
   Note [What is active in the RHS of a RULE or unfolding?]
in GHC.Core.Opt.Simplify.Utils

(cherry picked from commit 269c408)
tcInstFun must make "instantiation variables", not regular
unification variables, when instantiating function types. That was
previously implemented by a hack: set the /ambient/ level to QLInstTyVar.

But the hack finally bit me, when I was refactoring WhatUnifications.
And it was always wrong: see the now-expunged (TCAPP2) note.

This commit does it right, by making tcInstFun call its own
instantiation functions.  That entails a small bit of duplication,
but the result is much, much cleaner.

(cherry picked from commit 231adc3)
This reverts commit ebc6d49.

Obsolete now that the fix from MR 14474 (commit 231adc3) is backported.
If we skip exactly the number of words on the stack we end up on
the first word in the next chunk.

(cherry picked from commit 404b71c)
Instead of a massive case let's put this into data which we can reuse
elsewhere.

(cherry picked from commit d2b8960)
This stores the size (number of words on the stack) of the next
expected tuple in the TSO, ctoi_spill_size field, eliminating
the need of stg_ctoi_tN frames for each size.

Note: On 32 bit platform there is still a bytecode tuple size
limit of 255 words on the stack.

Fixes #26946

(cherry picked from commit a85bd50)
Small tuples are now returned more efficiently to the interpreter.
They use one less word of stack space and don't need manipulation
of the TSO anymore.

(cherry picked from commit e220903)
 - ELF platforms: emit .fini_array section
 - wasm32/Darwin: emit initializer with __cxa_atexit call
 - Windows: use -Wl,--whole-archive to prevent dropping finalizer symbols
 - rts linker: fix crash/assertion failure unloading objects with finalizers

fixes #27072

(cherry picked from commit 014087e)
This is necessary to avoid an import cycle on Windows when importing
`GHC.Internal.Exception.Context` in `GHC.Internal.Conc.Sync`.

On the road to address #25365.

(cherry picked from commit 039bac4)
Implements core-libraries-committee#297.

Fixes #25365.

(cherry picked from commit 8c389e8)
Implements core-libraries-committee#298

(cherry picked from commit e1ce1fc)
Ensure we hide the implementation details of the exception throwing mechanisms:

* `undefined`
* `throwSTM`
* `throw`
* `throwIO`
* `error`

The `HasCallStackBacktrace` should always have a length of exactly 1,
not showing internal implementation details in the stack trace, as these
are vastly distracting to end users.

CLC proposal [#387](haskell/core-libraries-committee#387)

(cherry picked from commit 016f79d)
As seen in #27289, the 1% acceptance threshold for this text was
overly narrow, resulting in spurious test failures. This commit widens
the acceptance threshold to 2%. Fixes #27289.

(cherry picked from commit b023381)
This submodule bump resolves a segfault on macos 15.
Fixes #27144

(cherry picked from commit 277a368)
The 2.7 branch is outdated and the module has been advanced far beyond
it anyway, so remove that line.

(cherry picked from commit 6779bb0)
 - suspend duplicate work for eager black holes
 - detect eager black holes in checkBlockingQueues
 - don't overwrite existing black holes even if they're not
     in an eager blackhole frame
 - don't deadlock on self when thunk is already blackholed

Fixes #26936

(cherry picked from commit 63ce577)
This prevents the WinIO manager from swallowing exceptions in overlapped IO. It
was added to make WinIO support possible in the `network` library. See
https://gitlab.haskell.org/ghc/ghc/-/issues/27283.

We also bump __IO_MANAGER_WINIO__ to 2 so libraries can gate on this using CPP.

(cherry picked from commit 037a80d)
On Linux and other POSIX platforms, GHC's -jsem jobserver client now
speaks v2 of the semaphore-compat protocol, which uses Unix domain
sockets in place of POSIX named semaphores. This avoids the libc-ABI
issues that affected the old implementation. Windows is unaffected
and continues to use the v1 protocol (Win32 named semaphores); its
reported protocol version remains v1.

When GHC receives a -jsem name whose protocol version it does not
support, it emits a -Wsemaphore-version-mismatch warning and falls
back to -j<N> rather than crashing. ghc --info exposes the supported
version in a new "Semaphore version" entry so cabal-install can detect
a mismatch before invoking GHC.

Users on a cabal-install that predates the v2 update will continue to
build successfully on Linux/POSIX, but will lose the cross-process
-jsem coordination and fall back to -j<N> per GHC invocation. Users
must upgrade to a cabal-install that supports protocol v2 to recover
full parallelism.

Also fix a leak in cleanupSem (#27253): cleanupSem used to snapshot
heldTokens and release them before killing the loop, while the loop's
in-flight acquire/release children could still be mutating it.
Cleanup now runs inside the loop's own exit handler, after draining
the active child via a new activeChild TVar, so the snapshot has no
concurrent mutator.

See also:
  - GHC proposal amendment:  ghc-proposals/ghc-proposals#673
  - cabal-install patch:     haskell/cabal#11628
  - semaphore-compat MR:     https://gitlab.haskell.org/ghc/semaphore-compat/-/merge_requests/8

Bump semaphore-compat submodule to 2.0.0

Fixes #25087 and #27253

(cherry picked from commit 8db331a)
When there are no remaining argument demands, it means the application
is bottoming. In this case, we can trim the continuation to avoid the
panic that was observed in #27261.

See Note [Trimming the continuation for bottoming functions] in
GHC.Core.Opt.Simplify.Iteration.

This patch was rewritten to avoid pulling in a refactor.
The original patch is included in master as 4a64568

(cherry picked from commit 53f7498)
This backports fix and test for #27121 from 2dadf3b since the entirety
of the patch is not backportable without also backporting two larger
refactorings.

(cherry picked from commit 0ccecfc)
Let's make clear what this module exports to allow us to easily deprecate and remove some of these in the future. Resolves https://gitlab.haskell.org/ghc/ghc/-/issues/26625

(cherry picked from commit b14bdd5)
angerman and others added 29 commits August 29, 2026 21:42
Upstream 9.14.2 added linkableAllBCOs (CompiledByteCode); under
+minimal / !HAVE_INTERPRETER that type is not imported. Match
linkableBCOs and wrap the export + definition.
Stage1 +minimal failed: Target.hs mapRegFormatSet needed
HasDebugCallStack/UniqSet; Types.hs interpStringCache needed
RemotePtr from Stubs when !HAVE_INTERPRETER.
hscSimpleIface uses Maybe ModBreaks outside HAVE_INTERPRETER;
ByteCode.Types (which re-exports it) is gated, so stage1 failed
with ModBreaks not in scope. Match Iface.Make and import from
HsToCore.Breakpoints.Types.
Lands the build pipeline for wasm32-unknown-wasi as a stage3 cross
target shipped as a relocatable ghcup-installable bindist.

  * flake.nix / flake.lock — bundles wasi-sdk via ghc-wasm-meta so
    the wasm cross-compile environment is reproducible end-to-end
    (clang, ld.lld, llvm tools all pinned).
  * Makefile — adds cross-build support (stage3-{wasm32-unknown-wasi,
    javascript-unknown-ghcjs}) with dist-based configuration,
    stamp-file dependency model for single-invocation builds, and
    proper PHONY/order-only ordering to fix hackage race conditions.
  * configure.ac — autoconf-shaped install layout for ghcup
    compatibility; @ALL_PACKAGES@ / @Constraints@ accumulators for
    stage{2,3} settings; --enable-dynamic toggle.
  * cabal.project.stage{0,1,2,3} — wire stage1/2/3 to use the
    accumulator-driven settings; stage3 imports cabal.project.common.
  * mk/wasm-{configure,relocate,bindist-Makefile} — autoconf-shaped
    `configure` stub, `relocate.sh` that recaches the per-target
    package db and warns on missing Node.js, and a bindist install
    Makefile that ghcup's installer-DSL drives via `make install`.
  * build-wasm-*.sh — remote-build helpers driving `nix develop` +
    git worktree for off-host cross-compile iteration.
  * USAGE.md — wasi-sdk + libffi setup notes for end-users.
Five focused changes needed for the wasm32-unknown-wasi target to
link and run end-to-end:

  * rts/linker/elf_got.c — handle undefined symbols referenced only
    by R_*_NONE relocations. The RTS linker previously failed on
    these even though they require no actual resolution.
  * rts/RtsStartup.c — add the missing wasm32 exclusion to the
    promoteBootLibrariesToGlobal call (mirrors the JS / Hadrian
    arch guards).
  * compiler/GHC/Driver/Session.hs — disable the overzealous wasm
    makeDynFlagsConsistent rule that forced -dynamic on libraries
    even when stage3 explicitly opted out, breaking the
    static-host / shared-wasm hybrid we ship.
  * compiler/GHC/Linker/Dynamic.hs — for wasm32 .so dep
    construction, force rts back in even when -no-rts is set
    (otherwise libHSghc-internal.so links without the rts
    transitive symbols at all).
  * compiler/GHC/Runtime/Interpreter/Wasm.hs — detect missing
    Node.js at iserv spawn and emit a clear error message instead
    of silently hanging.
  * compiler/Setup.hs, libraries/ghc-boot/Setup.hs — accept
    GIT_COMMIT_ID from env for hermetic git-less builds (CI
    shallow-checkout case).
  * utils/jsffi/dyld.mjs — one-line dlopen logging tweak.
Mirrors stage2's @ALL_PACKAGES@ accumulator template into a stage3
settings file so --enable-dynamic propagates symmetrically — but
scoped via `if arch(wasm32)` in the consumer .cabal projects so
that only the wasm target gets `shared: True` / `executable-dynamic:
True`. The JS target and native build-side stay static (avoids
emcc/wasm-ld "unknown argument: -h" when shared:True flowed into
the JS build, and avoids dynamic-too codepath breaks for native
build-side packages like happy-lib / alex / deriveConstants).

Final approach is R7 *path-i* (wasm-only `shared: True`, dynamic0
stage2 baseline) plus *Path C* host-dylib shipping (host libHS*.so
/ .dylib + matching .dyn_hi shipped under lib/$HOST_PLATFORM/ so
the dyn-linked wasm-ghc binary can find its host runtime via
@rpath at runtime).

Several iterations were necessary before this design crystallised
— see lode/wasm-cross-ghcup-plan.md "R7" thread for the full
post-mortem.
Renames .github/workflows/ci.yml → nix-ci.yml and restructures
the single monolithic build into a build/test/cross matrix:

  * Build / <plat> / dynamic={0,1} — stage0+stage1 then stage2,
    with aggressive intermediate cleanup between stages. dynamic=1
    artifact is what Cross consumers download.
  * Test / <plat> / dynamic={0,1} — runs the testsuite against
    the stage2 bindist.
  * Cross: WASM / <plat> — pulls the dynamic=1 stage2 dist, sets
    up wasi-sdk via ghc-wasm-meta bootstrap, prepends WASI_BIN to
    PATH, builds stage3-wasm32-unknown-wasi-tarball with DYNAMIC=1
    so the bindist ships .dyn_hi + host dylibs. On Linux, patchelfs
    the bindist's nix-store ELF interpreter to the standard system
    path and rewrites the rpath to $ORIGIN-relative.
  * Cross: JS / aarch64-darwin — same shape, emcc-driven.

On tag push matching `wasm32-wasi-*`, uploads each platform's
ghc-wasm32-unknown-wasi-<plat>.tar.gz to the matching GitHub
Release (drives the stable-haskell ghcup channel).

Sized to fit the runner constraints — see docs/ for the 41 GB
APFS / 28 GB WorkSpace split on darwin Tart and the 145 GB linux
runner budget; intermediate cleanups keep both within limits.
New workflow that builds the stable-haskell cabal-install for the
Linux variants and uploads the resulting cabal-<version>-<plat>.tar.gz
to the matching GitHub Release on cabal-* tag push. Closes the
channel gap (cabal channel YAML entries pointed at the wrong asset
names before).

  * Triggers on cabal-* tag pushes and on PRs that touch this
    workflow file or the r12 patch (so workflow self-tests itself
    before merge).
Two separate end-to-end workflows that exercise the SHIPPED ghcup
channel YAML by following the exact flow an end-user does:

  * Channel e2e (WASM) — wasm-only single-target channel
    (`ghcup-wasm.yaml`). Triggers on wasm32-wasi-* tag push, on PR
    edits to this file, on workflow_dispatch with a wasm_version
    input, and on a weekly Monday 06:00 UTC cron canary. Installs
    ghcup fresh, adds the channel, installs the wasm32-wasi-*
    GHC + cabal, then builds the published `hello` and
    `miso-counter` templates and verifies the artefacts.
  * Channel e2e (MULTI) — multi-target tri-frontend channel
    (`ghcup-multi-target-0.1.0.yaml`). Triggers on multi-* tag
    push, PR edits, workflow_dispatch with a multi_version input,
    weekly cron. Installs the multi-target GHC, then compiles +
    runs native / wasm / JS hello-worlds to verify argv[0]
    dispatch works for all three frontends.

Both workflows test on github-hosted ubuntu-latest,
ubuntu-24.04-arm, and macos-15 — the macOS leg deliberately runs
on the stock image (Xcode + CLT preinstalled, no nix) to mirror
what real end-users have. Self-hosted Tart darwin VMs stay
reserved for the nix-based in-tree builds in nix-ci.yml.

Splits avoid mixed pass/fail attribution: a WASM channel
regression no longer cascades into skipping MULTI tests, and PR
checks show separate ✓/✗ for each channel.
Workspace directory holding the initiative's living plan, phase
gates, design notes, root-cause analyses, and the cabal-install
patches that the build pipeline depends on:

  * wasm-cross-ghcup-plan.md — the running plan: Phase 0 (planning)
    through Phase 7 (documentation), with the R1..R12 risk log,
    R7 path-i / Path C decision threads, and end-of-phase status
    pins.
  * phase3-relocate-sh-draft.sh — initial relocate.sh draft (the
    final lands in mk/wasm-relocate.sh).
  * phase6-trivial-reactor-poc/ — proof of concept reactor app
    that drove discovery of the JSFFI invocation pattern.
  * phase6-miso-template-draft/ — full miso template (counter +
    REVIEW.md) that became the basis for the published miso-counter
    example.
  * r8-cabal-ghcjs-removal.patch — patch against
    stable-haskell/cabal removing dead GHCJS references in
    ProjectPlanning.hs + binDirectoryFor that prevented stage1
    from building.
  * r12-cabal-target-prefix-aware-tool-guess.patch — patch fixing
    cabal-install's dual-compiler tool lookup
    (guessGhcPkgFromGhcPath) so it doesn't fail on a wasm
    cross-compiler bindist that ships ghc-pkg at a non-default
    prefix.
Strip the two classes of build-host leak from every Mach-O artefact
in the stage2 dist tree BEFORE tarball assembly, so the bindist
ships clean without needing a post-build install_name_tool pass in
CI:

  (1) Absolute LC_RPATH entries pointing at the build store
      (`/Volumes/WorkSpace/_work/ghc/ghc/_build/stage2/store/...`).
      The bundled Cabal's depLibraryPaths bakes these into the
      link line. macOS 14 dyld silently falls through to the
      portable @executable_path/../lib/<host> rpath SET_RPATH
      adds; macOS 15 dyld treats the unresolvable absolute path
      as fatal and SIGABRTs at launch.

  (2) nix-store LC_LOAD_DYLIB install names for libiconv, libffi,
      libc++, libz, libresolv, libncurses. The devx-provided
      build runner has these visible at link time, but the
      install names baked into the linked binary point at
      /nix/store paths that don't exist on end-user hosts.
      Rewrite each to its /usr/lib equivalent (Apple stub-cache,
      ABI-compatible).

Mutating a Mach-O invalidates its linker signature; re-sign
ad-hoc afterwards so dyld accepts the binary on Apple Silicon.

Implementation lives in mk/clean-darwin-macho.sh rather than a
Makefile `define`/`$(if ...)` macro: the case-statement body
contains `)` characters that Make's $(if X,Y) parser interprets
as function-argument boundaries, expanding the body even on
non-Darwin hosts and tripping bash on the leaked closing parens.
A standalone script with its own `[ "$(uname -s)" = "Darwin" ]`
guard sidesteps the parser dance entirely.

Critical sequencing detail: the cleanup runs BEFORE the
ghc-pkg recache step further down in stage2.dist, otherwise
recache itself abort-traps on macOS 15 (the binary it invokes
has the same leak it's supposed to fix).

Pattern adapted from input-output-hk/devx static.nix
(fixup-nix-deps), SHA 5f05c1e1af6. Obsoletes the per-bindist
install_name_tool step in nix-ci.yml's Cross: MULTI darwin
path; that step can degrade to a verification-only scan.
…TAGE3@

Adds a parallel pair of autoconf accumulators (ALL_PACKAGES_STAGE3 /
CONSTRAINTS_STAGE3) to configure.ac, indented one extra level beyond
the stage2 versions so the substituted block can nest INSIDE an
`if arch(wasm32)` conditional in cabal.project.stage3.settings.in.

Rationale: stage3 builds three targets from one project file —
wasm32-unknown-wasi (--with-compiler=wasm32-...-ghc),
javascript-unknown-ghcjs (--with-compiler=javascript-...-ghc), and
native build-side packages (--with-build-compiler=ghc). Only wasm32
needs `shared: True` / `executable-dynamic: True` (Path C: ship
.dyn_hi + .so so end-user TH packages like miso/jsaddle build). For
JS, `shared: True` flows into emcc/wasm-ld which can't produce .so
("wasm-ld: error: unknown argument: -h"); for the native build-side
deps, no shared libs are needed at all.

`if arch(wasm32)` is cabal's per-invocation conditional — it
evaluates against the active --with-compiler's target arch, so a
single project file produces the right thing for all three
sub-builds.

JS-side .dyn_hi shipping is a separate concern, addressed by the
per-target settings dial (next commit, GHC issue #67).
Lets a single stage2 GHC binary report different `GHC Dynamic` /
`GHC Profiled` / `Support dynamic-too` values depending on which
target it's invoked as (via argv[0] dispatch into
lib/targets/<triple>/lib/settings).

Adds four new fields to PlatformMisc, threaded through Settings
and read in Settings.IO from the target's settings file:

  * target is dynamic                — GHC capable of -dynamic /
                                       -dynamic-too output
  * target ships dynamic libraries   — lib tree has .dyn_hi / .so
  * target is profiled               — GHC capable of -prof output
  * target ships profiling libraries — lib tree has .p_hi / .p_a

Reported pairs in `ghc --info` (Driver/Session.hs):
  GHC Dynamic         = (target is dynamic)  && (target ships dynamic libs)
  GHC Profiled        = (target is profiled) && (target ships prof libs)
  Support dynamic-too = (target is dynamic)  && (target ships dynamic libs)

These two-dial-per-way splits keep "capable" and "currently shipping"
orthogonal — a target can be dynamic-capable without actively
shipping .dyn_hi (or vice versa). cabal-install reads these to decide
whether to enable library-dynamic / library-profiling by default, so
JS (which doesn't ship .dyn_hi in this series) can correctly report
all-NO while the wasm target reports YES for both dynamic dials.

Settings.IO falls back to sane defaults (`YES` for dynamic if the RTS
itself is dynamic, `NO` for prof) when the keys are missing, so this
commit is no-op until the Makefile injects the new keys.

Refs GHC issue #67.
Two concerns in the build-system layer:

1. Inject per-target dial keys into lib/settings files:
   * Native settings (HOST_PLATFORM/lib/settings): four dials
     reflecting current DYNAMIC=0/1 invocation; prof=NO (stage2
     isn't built -prof).
   * Stage3 cross-target settings (TARGET_DIR/lib/settings) via
     defaults YES/YES/NO/NO, overridable per triple via
     STAGE3_<triple>_TARGET_{IS_DYNAMIC,SHIPS_DYN_LIBS,
     IS_PROFILED,SHIPS_PROF_LIBS}.
   * JS target overrides all four to NO (no .dyn_hi, no .p_hi).

   sed-end-of-line anchor: needs `$$$$` (four dollars) to survive
   define-template + recipe-time Make expansion — verified vs.
   `$$` which gets eaten and produces invalid settings files.

2. New $(DIST_DIR)/ghc-multi-target.tar.gz rule:
   Packages native (lib/$(HOST_PLATFORM)) + wasm32-unknown-wasi +
   javascript-unknown-ghcjs into a single bindist consumed via
   argv[0] dispatch (bin/ghc -> native, bin/wasm32-...-ghc -> wasm,
   bin/javascript-...-ghc -> JS — same physical binary, three
   targets).

   Uses `tar czhf` (dereference symlinks) so the cross-prefixed
   bin entries become standalone copies — ~30 MB cost in exchange
   for predictable behaviour with ghcup's targetPattern glob.
   Filters ghc-iserv out of the JS bin list (JS backend has its
   own evaluator).

   New mk/multi-target-{configure,relocate,bindist-Makefile}
   scripts ride along in the tarball for end-user install.
Adds a new Cross: MULTI job to nix-ci.yml that builds the
ghc-multi-target.tar.gz bindist on aarch64-darwin (alongside the
existing Cross: WASM and Cross: JS jobs).

Runs `make stage3-wasm32-unknown-wasi stage3-javascript-unknown-ghcjs`
followed by `make _build/dist/ghc-multi-target.tar.gz`, then uploads
the multi-target tarball as a workflow artifact for downstream
channel-e2e validation and ghcup release pickup.

WASM-specific Cross job kept for backward compat while we trial the
unified MULTI flow.
Channel end-to-end test gets a heavier exercise: after ghcup install
of the multi-target compiler, build the full miso-counter app
(50+ deps incl. aeson, jsaddle-wasm, TH-heavy packages) for the
wasm target, exercising the Path C .dyn_hi shipping for end-user
TH compilation.

Also retires the legacy channel-e2e-wasm.yml workflow: that one
tested the wasm-only ghcup-wasm.yaml channel (now deprecated in
favour of ghcup-multi-target-0.1.0.yaml). Its miso coverage is
subsumed by the new step here, and we no longer want to gate on
the legacy channel.
Three new lode docs covering the design + investigations behind
this series:

  * lode/multi-target-bindist-design.md
    Design doc for the argv[0]-dispatched single-binary
    multi-target bindist (native + wasm32 + JS in one tarball),
    consumed via ghcup-multi-target-0.1.0.yaml channel.

  * lode/rpath-leak-investigation.md
    Investigation log for the darwin LC_RPATH leak that motivated
    the Cabal PR #368 rpath-relativize-absolute patch (commit 1).
    Documents why we didn't flip cabal's relocatable: True flag
    (it also emits library-dirs: ${pkgroot}/... that breaks our
    post-stage2 path rewriting).

  * lode/draft-ghcup-multi-target-0.1.0.yaml
    Draft ghcup channel YAML for the new multi-target format —
    successor to ghcup-wasm.yaml. Lives in lode/ until promoted
    to gh-pages once Cross: MULTI is green on all platforms.
The stat-layout probe block in libraries/ghc-internal/configure.ac
was guarded by an exact-string compare:

    if test "$host" = "javascript-ghcjs"

But the multi-target JS triple is `javascript-unknown-ghcjs`, so the
block was silently skipped, leaving SIZEOF_STRUCT_STAT /
OFFSET_STAT_ST_* as #undef. The JS shim then expanded
h$base_sizeof_stat() to a bare identifier, producing:

    ReferenceError: SIZEOF_STRUCT_STAT is not defined

at TH-evaluation time for any package using Posix stat (e.g. miso /
jsaddle).

Asymmetry that pinned root cause: the non-guarded HTYPE_* probes
above the same file were defined correctly; only the guarded block
went missing. emcc confirms sizeof(struct stat)=96 — the probes are
viable, they just weren't being run.

Fix: replace the string compare with a case glob `javascript*)` so
both legacy (javascript-ghcjs) and multi-target
(javascript-unknown-ghcjs) hosts trigger the block.
Drops the local r8/r12 .patch files from lode/ in favour of pinning
a stable-haskell/cabal branch that already carries both fixes:

  stable-haskell/cabal:stable-haskell/feature/wasm-cross-ghcup-stack
  = 8b8433b736d45ec53a103baf4e4aabb8010ca2ed

  6a5ce8161  #368 rpath relativize (was already pinned)
  8b8433b73  #361 target-prefix-aware tool guess (cherry-picked,
             was previously applied as
             lode/r12-cabal-target-prefix-aware-tool-guess.patch)

r8 (GHCJS removal) was already absorbed by upstream master cleanup
before the #368 base, so it's not part of the stack.

Effect on the build:
  * cabal.project.stage{0,1,2,3}: tag bump only (no semantic change
    — the new SHA is just #368 + a clean cherry-pick of #361 that
    was previously applied locally only in cabal-release.yml).
  * cabal-release.yml: drops the "checkout-this-repo" + "git apply
    r12" steps and the patch path-trigger; cabal-install ships
    identically because the patch is already in CABAL_SHA.

Patches retired (now wholly carried by upstream PR branches):
  * lode/r8-cabal-ghcjs-removal.patch     (= stable-haskell/cabal #359)
  * lode/r12-cabal-target-prefix-aware-tool-guess.patch
                                          (= stable-haskell/cabal #361)
GitHub deprecated Node20 runners starting June 16th, 2026, and removed
Node20 entirely on September 16th, 2026. Three actions families were
still on @v4 (the Node20-runtime line) and triggered deprecation
annotations on every job:

  * actions/checkout@v4         → @v5  (Node24)
  * actions/upload-artifact@v4  → @v5  (Node24)
  * actions/download-artifact@v4 → @v5 (Node24)

actions/cache was already on @v5; no other Node20-runtime actions are
referenced from this repo. The bump is mechanical; behaviour and API
surface are unchanged across v4→v5 for our usage (checkout fetch-depth
and path inputs, artifact name + path, basic compression).

Workflows touched:
  * nix-ci.yml            (11 v5 references)
  * reusable-release.yml  (12 v5 references)
  * release.yml           (2)
  * cabal-release.yml     (2; also tidies the file header comment to
                            match the new branch-based cabal pin)
… DAG

Before: channel-e2e-multi.yml and cabal-release.yml were standalone
workflows fired on their own push/PR triggers. They ran in parallel
with nix-ci and tested whatever bytes were on the live ghcup channel
— so a PR-introduced regression in GHC packaging, cabal patches, or
wasm sysroot wiring only surfaced at release time, not at PR time.

After: two new reusable workflows + DAG edges in nix-ci:

  reusable-cabal-release.yml  (extracted from cabal-release.yml)
    on: workflow_call
    inputs: cabal_sha, cabal_ver, bootstrap_ghc, release_tag
    Build x86_64-linux + aarch64-linux cabal bindists, upload artifact,
    optionally upload to a GitHub Release.

  reusable-channel-e2e.yml    (extracted from channel-e2e-multi.yml)
    on: workflow_call
    inputs: install_mode (channel|artifact), multi_version, cabal_version
    install_mode=channel  — install GHC + cabal via the live ghcup
                            channel YAML (post-release smoke test;
                            same behaviour as before)
    install_mode=artifact — download multi-target GHC tarball from
                            cross-multi + cabal bindist from
                            cabal-release IN THE SAME WORKFLOW RUN,
                            install locally via the bundled
                            ./configure + make install, run the same
                            hello / miso / JS-hello test surface

  channel-e2e-multi.yml       (now a thin wrapper)
    Same triggers as before; calls reusable with install_mode=channel.

  cabal-release.yml           (now a thin wrapper)
    Same triggers; calls reusable; tag pushes still drive
    release-asset upload.

  nix-ci.yml (two new DAG nodes appended)
    cabal-release  : parallel with build (no needs:)
    e2e-multi      : needs [cross-multi, cabal-release], install_mode=
                     artifact — downloads THIS run's artifacts and
                     tests them. Skipped if either dependency failed.

End-to-end effect: a PR that breaks cabal (e.g. a bad patch in the
upstream branch SHA), wasm sysroot, or multi-target packaging now
fails inside nix-ci instead of slipping through to release. The
standalone channel-e2e-multi.yml + cabal-release.yml workflows
continue to fire on tag pushes / dispatch / weekly canary so the
live channel keeps getting validated independently.
Cross: MULTI in nix-ci.yml has `continue-on-error: true` so a single
platform failure (e.g. the transient github.com 504 from this PR's
first CI cycle) doesn't block the other platforms. As a side effect,
`needs.cross-multi.result` at the e2e-multi caller level resolves
"success" even when one matrix entry didn't upload an artifact — so
the previously-existing job-level gate
`needs.cross-multi.result == 'success'` wasn't actually gating
anything per-platform, and a missing artifact cascaded into an
e2e-multi job failure on download.

Fix the same problem at both artifact-download points (cabal +
multi-target tarball) in the reusable workflow:

  * download-artifact step gets continue-on-error: true + an id
  * install step gates on `steps.<id>.outcome == 'success'`
  * for cabal: a fallback step installs cabal from the wasm ghcup
    channel when the artifact is missing (clean recovery — cabal
    is downstream of every test)
  * for multi-target: a warning-only step fires when the artifact
    is missing; the test steps already gate on
    `steps.install_artifact.outputs.installed == 'true'`, which
    stays empty when install_artifact is skipped, so the tests
    cleanly skip rather than failing

Net effect: cross-multi failing on one platform now produces a clean
e2e-multi skip on that platform with a warning, not a cascade
failure on top of the original cross-multi failure.

Refs task #70.
Andrea's base commit 4ad586b ("stage2: select static/dynamic build
via project files instead of configure") removed the --enable-dynamic
autoconf toggle, m4/accumulate.m4, and the generated
cabal.project.stage2.settings in favour of explicit
cabal.project.stage2.{common,static,dynamic}.

Our stage3 wasm-shared support was built on top of that now-removed
machinery: configure derived ALL_PACKAGES_STAGE3 from ALL_PACKAGES
(populated by APPEND_PKG_FIELD in m4/accumulate.m4) and substituted it
into cabal.project.stage3.settings.in. With the machinery gone, port the
stage3 settings to the same explicit-project-file model:

  * Inline the wasm-only dynamic block directly into cabal.project.stage3
    (shared: True / executable-dynamic: True / rts +dynamic, guarded by
    `if arch(wasm32)`). This is exactly what the generated settings
    produced under DYNAMIC=1 — the only mode the Cross: MULTI build runs.
  * Delete cabal.project.stage3.settings.in (no longer generated).
  * Drop the cabal.project.stage3.settings prerequisite from the two
    STAGE3_<plat>_PREREQS Makefile variants.
  * Drop the now-obsolete .gitignore entry.

configure.ac, Makefile stage2 selection, and the stage2 project files are
taken from base unchanged. The per-target `settings` dials (target is
dynamic / ships dynamic libraries = YES for wasm) — added earlier in this
branch and orthogonal to base's change — are what make GHC's link
pipeline accept the inlined dynamic block.
…strap

RCA: stage3 cross builds (wasm32-unknown-wasi, javascript-unknown-ghcjs)
die at `primops.txt:139:31` while compiling the `ghc` library. Confirmed by
direct reproduction:

  $ <bootstrap ghc-9.8.4>/bin/genprimopcode --data-decl < primops.txt
  genprimopcode-ghc-9.8.4: parse error at "Parse error at line 139, column 31"

Line 139 is `effect = NoEffect` -- the primop effect-classification grammar,
which the bootstrap GHC 9.8.4's genprimopcode predates. compiler/Setup.hs
invokes genprimopcode by bare name (`readProcess "genprimopcode"`), i.e. via
PATH, and the devx/nix bootstrap GHC's genprimopcode wins. The CC preprocessing
of primops.txt.pp is *not* at fault: host clang and wasm32-wasi-clang produce
byte-identical output (4465 lines), and the freshly-built genprimopcode parses
it cleanly. stage1/stage2 escape this only because cabal's build-tool-depends
happens to inject the fresh tool for native builds; the cross stage3 build does
not get that for the genprimopcode Setup hook.

Fix (build-orchestration layer, no compiler/Setup.hs change -- keeps #384
reconciliation surface minimal):
- Makefile: add GENPRIMOPCODE_BIN (mirrors DERIVE_CONSTANTS_BIN/GENAPPLY_BIN)
  and prepend its dir to PATH in the stage3 cabal-build env so the fresh
  genprimopcode shadows the bootstrap one. --with-compiler/--with-build-compiler
  /--with-hsc2hs are explicit, so prepending stage1/bin cannot mis-shadow
  ghc/ghc-pkg/hsc2hs.
- Makefile: copy genprimopcode into the dist bindist (alongside deriveConstants
  /genapply) so DIST_BUILD (CI) cross builds have it.
- nix-ci.yml: pass GENPRIMOPCODE_BIN=$PWD/_build/dist/bin/genprimopcode in the
  multi-target DIST_BUILD invocation.

Verified: with stage1/bin prepended, `genprimopcode` resolves to the fresh
build and parses the effect grammar (exit 0); without it, the bootstrap one
fails at 139:31.
The wasm32 patches commit had overwritten ghc-boot/compiler Setup.hs with
the pre-edb808a0 Verbosity API, breaking stage1 against Cabal f2e0a89
(VerbosityFlags vs Verbosity).
Cross MULTI failed Cabal-7107: pinned process-1.6.26.1 is rejected by
Cabal f2e0a89 (3.17.0.1). Align stage3 with stage2's process-1.6.29.0.
Cross DIST_BUILD was rebuilding them with --happy-options pointing at
dist happy-lib templates, yielding happyDoParse without a binding.
Use PATH binaries from stage1/dist instead (-build-tool-depends).
Cross host:ghc failed: System.Semaphore does not export SemaphoreError
(semaphore-compat-1.0.0). Stage2 already pins 2.0.1.
Cross CI has happy-2.1.7 on PATH but was forcing dist happy-lib-2.1.5
templates via --happy-options, causing GHC-44432 happyDoParse in
GHC.Parser. Let happy use its own packaged templates instead.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.