Skip to content

AArch64 DWARF unwind + macOS os_signpost integration - #176

Open
angerman wants to merge 36 commits into
stable-ghc-9.14from
feat/aarch64-dwarf-signpost
Open

AArch64 DWARF unwind + macOS os_signpost integration#176
angerman wants to merge 36 commits into
stable-ghc-9.14from
feat/aarch64-dwarf-signpost

Conversation

@angerman

@angerman angerman commented Mar 10, 2026

Copy link
Copy Markdown

Summary

Two independent, complementary features that make GHC-compiled Haskell code visible to standard profiling tools on macOS:

Part 1: AArch64 DWARF Unwind Support (GHC #19913)

The AArch64 NCG was silently discarding CmmUnwind nodes (return nilOL), producing no DWARF unwind information. This made lldb bt, Instruments, and Samply unable to unwind through Haskell frames on Apple Silicon.

  • Add UNWIND pseudo-instruction to AArch64 Instr data type (mirrors X86)
  • Implement CmmUnwind → UNWIND conversion in stmtToInstrs
  • Add addSpUnwindings to emit UNWIND after DELTA (tracks SP changes)
  • Add extractUnwindPoints and wire into NcgImpl (was const [])
  • Add UNWIND pretty-printing to Ppr.hs

Part 2: macOS os_signpost Integration

The RTS had no os_signpost support, making GC pauses, thread events, and user events invisible in Apple Instruments.

  • New rts/Signpost.{h,c} with os_signpost API wrappers
  • GC interval signposts (begin/end pairs, per-capability tracking)
  • Thread lifecycle signposts (create/run/stop)
  • User event forwarding (traceEvent#/traceMarker#)
  • Zero overhead when Instruments is not attached (os_signpost_enabled() gate)
  • Empty macros on non-Darwin (zero overhead)
  • Events appear in Instruments "Points of Interest" lane by default

Files Changed

Compiler (AArch64 DWARF):

  • compiler/GHC/CmmToAsm/AArch64/Instr.hs — UNWIND constructor + pattern matches
  • compiler/GHC/CmmToAsm/AArch64/CodeGen.hs — CmmUnwind handler, addSpUnwindings, extractUnwindPoints
  • compiler/GHC/CmmToAsm/AArch64/Ppr.hs — UNWIND pretty-printing
  • compiler/GHC/CmmToAsm/AArch64.hs — Wire extractUnwindPoints into NcgImpl

RTS (os_signpost):

  • rts/Signpost.h — Header with Darwin functions / non-Darwin empty macros
  • rts/Signpost.c — os_signpost implementation
  • rts/RtsStartup.c — initSignposts/freeSignposts lifecycle
  • rts/Stats.c — GC begin/end signpost calls
  • rts/Trace.h — Thread event signpost calls
  • rts/Trace.c — User event signpost calls
  • rts/rts.cabal — Add Signpost.c to build

Test plan

  • Build GHC with changes on AArch64 (Apple Silicon)
  • Compile test program with -g and verify dwarfdump --debug-frame shows FDE entries
  • Run under lldb and verify bt shows Haskell frames
  • Build GHC on macOS and run allocation-heavy program under Instruments
  • Verify GC intervals appear in "Points of Interest" lane
  • Verify traceEvent/traceMarker events appear as signpost events
  • Verify non-Darwin builds compile without warnings (empty macros)

@angerman
angerman force-pushed the feat/aarch64-dwarf-signpost branch from b203c9e to eeb6c01 Compare March 12, 2026 05:27
angerman and others added 15 commits August 30, 2026 11:13
@angerman @cursoragent
ci(release): detect cygwin/mingw make-hosts, not just msys
3f6e354 
Hosted Windows runners' make has switched MSYS2→Cygwin (MAKE_HOST
x86_64-pc-cygwin); msys-only guards dropped CC/CXX overrides and broke
stage1 with "g++ not found". Match msys/cygwin/mingw32.

Co-authored-by: Cursor <cursoragent@cursor.com>
@angerman @cursoragent
ci(release/freebsd): build the native bindist only (no JS cross)
38d8c28 
FreeBSD self-hosted runner has no JS toolchain; skip javascript bindist
and haskell-toolchain.tar.gz. Ship native ghc/cabal/tests only.

Co-authored-by: Cursor <cursoragent@cursor.com>
@angerman @cursoragent
build(stage1): disable libffi raw API for Rocky Linux 8 gcc
c4d2027 
Rocky 8 promotes ffi_java_raw_* deprecations to errors in libffi-clib's
java_raw_api.c. Force -optc-DFFI_NO_RAW_API=1 so that file compiles out;
GHC does not use the Java raw API.

Co-authored-by: Cursor <cursoragent@cursor.com>
@angerman @cursoragent
fix(stage1): pin time-1.14 from source for Windows host:Cabal
fad0526 
Post-split host:Cabal pulls installed time-1.12.2 which forces installed
Win32-2.13.4.0, conflicting with the Win32-2.14.2.1 pin (Cabal-7107 on
Windows release CI). Build time-1.14 from source instead (in range for hpc).

Co-authored-by: Cursor <cursoragent@cursor.com>
@angerman @cursoragent
ci(release/rocky): ship native glibc bindist only (skip emscripten/JS)
7c6b260 
Rocky stage3-javascript fails with Cabal-4123 (alex not on PATH) after the
hermetic stage1 layout. JS/multi-target coverage stays on Nix CI Cross:MULTI;
Rocky keeps validating the native glibc release path (incl. libffi FFI_NO_RAW).

Co-authored-by: Cursor <cursoragent@cursor.com>
@angerman @cursoragent
ci(release): fix emscripten YAML anchor after Rocky skip
692cc6c 
Move &emscripten onto the macOS x86 job so *emscripten on aarch64 still
resolves; Rocky continues to skip emsdk (native glibc only).

Co-authored-by: Cursor <cursoragent@cursor.com>
@angerman @cursoragent
ci(release): default CC for bindist testsuite makefile_tests
630c5bf 
Mac aarch64 release tests failed T27072 with `/bin/sh: : command not
found` because $(CC) was empty. Bindist test jobs only unpack GHC; set
CC to clang/cc/gcc when unset before running the packaged testsuite.

Co-authored-by: Cursor <cursoragent@cursor.com>
@angerman @cursoragent
ci(release): fix empty CC export; skip emsdk on Mac x86
9439e09 
T27072 still failed because `CC=` (empty) overrides make's default cc.
Only export CC when a real compiler path is found (/usr/bin/clang first).

Mac x86 hit the same stage3-javascript alex Cabal-4123 as Rocky — drop
emsdk there too so it ships a native bindist; JS stays on Nix CI.

Co-authored-by: Cursor <cursoragent@cursor.com>
@angerman @cursoragent
ci(release): pass CC/TEST_CC on make cmdline for T27072
0711561 
Nested makefile_test makes only see env/MAKEFLAGS, so an empty CC still
broke Mac aarch64. Resolve an absolute compiler, export CC and TEST_CC,
pass both on the make command line, and teach T27072 to prefer TEST_CC.

Co-authored-by: Cursor <cursoragent@cursor.com>
@angerman @cursoragent
fix(testsuite): resolve CC inside makefile_test for T27072
d4f691b 
Top-level bindist CC=/usr/bin/clang still left nested makefile_test
makes with an empty $(CC). Pass CC/TEST_CC on the make cmdline, sanitize
empty CC in the driver env, and resolve the compiler in T27072's recipe.

Co-authored-by: Cursor <cursoragent@cursor.com>
@angerman @cursoragent
fix(T27072): do not echo the resolved CC onto test stdout
d174096 
The debug line made CI fail T27072 with [bad stdout]. Keep the silent
shell fallback only.

Co-authored-by: Cursor <cursoragent@cursor.com>
@angerman @cursoragent
fix(release/testsuite): pass TEST_HC into nested makefile_tests
4dc4974 
Mac/Windows BaR still failed T27072 with `/bin/sh: : command not found`
after the CC fix: nested makes saw an empty $(TEST_HC) because those jobs
only set THREADS. Export/default TEST_HC in the bindist test script, export
it from test.mk, and pass it on the makefile_test make cmdline.

Co-authored-by: Cursor <cursoragent@cursor.com>
@angerman @cursoragent
fix(testsuite): strip quote_path quotes before passing TEST_HC
6d2fcc5 
makefile_test forwarded config.compiler into TEST_HC=, but that value
already includes make's quote_path quotes. Nested makes then searched for
ghc" and ~676 CI tests failed. Strip surrounding quotes; stop exporting
TEST_HC_OPTS.

Co-authored-by: Cursor <cursoragent@cursor.com>
@angerman @cursoragent
test(T26537): fragile + longer compile timeout on Darwin
d34d980 
Mac aarch64 BaR passed T27072 after the TEST_HC/CC fixes; the only
remaining unexpected failure was T26537 timing out at -O2 -fregs-graph.

Co-authored-by: Cursor <cursoragent@cursor.com>
@angerman @cursoragent
test(static001): ignore ranlib stderr on Darwin
fad9f27 
Mac x86 BaR failed static001 with Xcode ranlib "has no symbols" noise
after T27072/T26537 were otherwise green. Ignore stderr for this
darwin-only makefile_test.

Co-authored-by: Cursor <cursoragent@cursor.com>
@angerman
angerman force-pushed the feat/aarch64-dwarf-signpost branch from 108cdd1 to 4521a53 Compare August 31, 2026 04:28
angerman and others added 8 commits August 31, 2026 17:27
@angerman @cursoragent
ci(release/freebsd): cap cabal --jobs=1 to avoid OOM on Cabal-syntax
c667c7a 
FreeBSD self-hosted runner SIGKILLs Cabal-syntax under default parallel
cabal install (Cabal-7125). Cap CABAL_ARGS for the FreeBSD bindist build.

Co-authored-by: Cursor <cursoragent@cursor.com>
@angerman @cursoragent
ci(release/freebsd): also pass --ghc-options=-j1 for Cabal-syntax OOM
e21ce12 
--jobs=1 alone still OOMs: GHC compiles Cabal-syntax modules in parallel.

Co-authored-by: Cursor <cursoragent@cursor.com>
@angerman @cursoragent
ci(release/freebsd): use ghcup cabal instead of building cabal-install
73a4c78 
Even --jobs=1 --ghc-options=-j1 OOMs on cabal-install after Cabal-syntax.
Ship/use the ghcup cabal binary so FreeBSD can get past stable-cabal.

Co-authored-by: Cursor <cursoragent@cursor.com>
@angerman @cursoragent
ci(release/freebsd): build stable-cabal with -O0 -j1, then reuse it
7657917 
ghcup cabal 3.14 lacks --with-build-compiler. Building cabal-install still
OOMs at -j1; add -O0 for stable-cabal only, then USE_SYSTEM_CABAL so later
targets do not rebuild it.

Co-authored-by: Cursor <cursoragent@cursor.com>
@angerman @cursoragent
ci(release/freebsd): use ghcup cabal 3.18 bindist (skip source build)
d41e6d5 
Building cabal-install OOMs hard enough to kill the FreeBSD runner even at
-O0 -j1. Install ghcup's FreeBSD cabal 3.18.1.0 (has --with-build-compiler)
and ship that binary instead.

Co-authored-by: Cursor <cursoragent@cursor.com>
@angerman
AArch64: implement DWARF unwind support (GHC #19913)
83f9fbe 
The AArch64 native code generator was silently discarding CmmUnwind
nodes (`return nilOL`), making DWARF-based profiling and debugging
impossible on Apple Silicon and ARM64 Linux.

This commit adds full DWARF unwind support, mirroring the existing
X86 implementation:

  - Add UNWIND pseudo-instruction to AArch64.Instr
  - Convert CmmUnwind nodes to UNWIND instructions in CodeGen
  - Emit UNWIND after DELTA via addSpUnwindings for SP tracking
  - Wire extractUnwindPoints into the AArch64 NcgImpl record
  - Pretty-print UNWIND as a label + comment in Ppr

With this change, `ghc -g` on AArch64 produces .debug_frame entries,
enabling `lldb` backtraces, `dwarfdump --debug-frame`, and sampler-
based profilers (Instruments, Samply) to unwind through Haskell code.
@angerman
RTS: add os_signpost integration for macOS Instruments profiling
5c6a22f 
GHC-compiled programs are invisible to Apple Instruments because the
RTS emits no os_signpost events. This makes it hard to correlate GC
pauses and thread scheduling with system-level activity on macOS.

Add a new Signpost.c/Signpost.h module that bridges RTS events to the
os_signpost API, using OS_LOG_CATEGORY_POINTS_OF_INTEREST so events
appear in Instruments by default without a custom .instrpkg:

  - GC intervals: begin/end pairs tracked per-capability with unique
    signpost IDs, emitting generation, bytes copied, and slop
  - Thread lifecycle: create/run/stop as point events with cap and tid
  - User events: traceEvent#/traceMarker# forwarded as signposts

All functions gate on os_signpost_enabled() so the overhead when
Instruments is not attached is near zero (a single branch on the
log handle's signpost-enabled flag).

On non-Darwin platforms, all functions compile to empty macros.

Integration points:
  - Stats.c: GC begin/end with full statistics
  - Trace.h/Trace.c: thread and user event forwarding
  - RtsStartup.c: init after initScheduler, free before endTracing
@angerman
AArch64: enable DWARF unwind on ELF, fix register mappings
ab54f4c 
The AArch64 NCG was explicitly excluded from DWARF debug info
generation despite having full UNWIND pseudo-instruction support.
This meant all computed unwind data was silently discarded.

Changes:

- Remove ArchAArch64 exclusion from ncgDwarfEnabled, enabling
  .debug_frame output on AArch64 ELF (Linux)

- Define REG_MachSp as r31 in arm64.h so the C stack pointer maps
  to DWARF register 31 (SP) instead of 0 (x0). Without this, all
  addSpUnwindings output incorrectly described x0 changes.

- Add DW_CFA_same_value for AArch64 SP (register 31) in the CIE
  initial instructions. This prevents the DWARF unwinder from
  incorrectly setting SP = CFA (which is the STG Sp on x20).

- Fix UNWIND Note reference to point to GHC.CmmToAsm (where the
  Note actually lives) instead of X86/Instr.hs.
angerman and others added 10 commits September 15, 2026 11:10
@angerman
AArch64: add verifyBasicBlock to NCG
46f763c 
Add basic block structure validation to the AArch64 code generator,
mirroring the existing X86 implementation. This catches NCG bugs
where non-control-flow instructions appear after block-terminating
jumps, which would violate the basic block invariant.

BL (Branch and Link) is exempted from the block-end check since
it is a call that returns to the caller, not a block terminator.
Only active when debugIsOn (debug builds).
@angerman
Enable DWARF debug info on MachO/darwin targets
f1c84e9 
DWARF generation was gated on osElfTarget, excluding all MachO
targets despite the DWARF assembly output code already handling
MachO section directives (__DWARF,__debug_*), darwin-specific
alignment (.align as log2), and section offsets.

Changes:

- Replace osElfTarget gate with osDwarfTarget that accepts both
  ELF and MachO, enabling -g on macOS/darwin

- Make DWARF section labels (dwarfInfoLabel, etc.) platform-aware:
  use "L" prefix on darwin (MachO convention) instead of hardcoded
  ".L" (ELF convention), via asmTempLabelPrefix

With this change, ghc -g on AArch64-darwin produces .debug_frame
entries visible to dsymutil, lldb, and Instruments.
@angerman
AArch64: add CFI directives to StgRun for DWARF unwinding
84a5d25 
Add DWARF Call Frame Information directives to the AArch64 StgRun
function so debuggers and profilers can unwind through the Haskell↔C
boundary. Without CFI, tools like lldb, gdb, and perf cannot produce
backtraces that cross from Haskell into C code.

The CFA is anchored at x29+16 (the frame pointer saved by the first
stp), and all callee-saved registers (x19-x28, x16-x17, x29-x30,
d8-d15) are annotated with their save locations on the C stack.

Enable ENABLE_UNWINDING on AArch64-darwin in addition to Linux.
The original restriction (#15207) was about x86_64 GCC/Clang
assembler incompatibilities that do not apply to AArch64 where
both Linux and darwin use Clang-compatible assemblers.
@angerman @cursoragent
fix: resolve cherry-pick conflict markers in DWARF/Ppr imports
7756b00 
Co-authored-by: Cursor <cursoragent@cursor.com>
@angerman
Fix DWARF Constants.hs: inline local label prefix instead of importing
5249cf9 
asmTempLabelPrefix is not exported from GHC.Cmm.CLabel. Use a local
dwarfLocalLabel helper that implements the same logic: "L" on darwin,
".L" on ELF targets.
@angerman
AArch64: emit DWARF proc_end and block_end labels
2bc3e71 
The assembler was reporting 'local symbol LcXX_proc_end not defined'
because AArch64/Ppr.hs never emitted _proc_end labels that DWARF
.debug_info and .debug_frame reference for procedure address ranges.

Add pprProcEndLabel and pprBlockEndLabel helpers (matching the X86
pattern) and emit them:
- At the end of each basic block (since blocks may become standalone
  top-level blocks after branch-chain elimination)
- At the end of each procedure in pprNatCmmDecl (both with and
  without info tables)

This fixes 14 test failures on aarch64-darwin with DWARF enabled.
@angerman @cursoragent
Revert MachO DWARF enablement: ELF-only for now
8cd31d8 
The MachO assembler cannot handle relocations against local symbols
(L-prefixed labels on darwin) in DWARF debug sections, producing:

    error: unsupported relocation of local symbol 'Lc134_die'.
    Must have non-local symbol earlier in section.

This is a fundamental MachO assembler limitation that requires either
non-local DWARF labels or section anchor symbols to resolve.

Revert to ELF-only DWARF debug sections for now. The MachO-related
infrastructure (section directives, local label prefix support, CFI
directives in StgRun) is kept in place for future MachO DWARF work.

CFI directives (.cfi_*) in StgCRun.c remain enabled on AArch64-darwin
as they produce .eh_frame entries that the system tools handle fine.

Co-authored-by: Cursor <cursoragent@cursor.com>
@angerman
Signpost: handle dynamic capability growth; Ppr: use label helper
a111917 
Add signpostsAddCapabilities() to resize per-capability signpost ID
arrays when setNumCapabilities grows the number of capabilities at
runtime. Without this, new capabilities' GC intervals would not be
tracked in Instruments (graceful degradation via bounds check, but
data loss).

Follows the pattern of tracingAddCapabilities() and
storageAddCapabilities() in Schedule.c.

Also use pprBlockEndLabel helper consistently in AArch64/Ppr.hs
instead of manually constructing the label.
@angerman
MachO DWARF: add section anchors for darwin assembler compatibility
c3b6211 
On MachO, the assembler cannot create relocations against temporary
symbols (L-prefixed) in DWARF debug sections unless there is a
non-temporary symbol earlier in the section to serve as the relocation
base. Without such an anchor, the assembler fails with:

  error: unsupported relocation of local symbol 'Lfoo'.
  Must have non-local symbol earlier in section.

This was preventing DWARF debug info (-g) from working on macOS/darwin.

The fix emits a linker-private anchor symbol (l_ prefix) at the start
of each DWARF section (.debug_info, .debug_abbrev, .debug_line,
.debug_frame, .debug_aranges). The l_ prefix gives us:
  - A symbol table entry (assembler can create relocations against it)
  - Local binding (no duplicate symbol errors across compilation units)

Also fixes the label ordering in .debug_info where the section label
was emitted BEFORE the section directive (placing it in the wrong
section).

With these anchors in place, DWARF is re-enabled on MachO targets.

Additional changes:
- Gate ncgComputeUnwinding to DWARF-capable targets (ELF + MachO)
  to avoid wasting work on platforms that cannot emit debug info
- Document info table alignment decision in AArch64/Ppr.hs
@angerman @cursoragent
build: omit --with-build-compiler for stock (USE_SYSTEM_CABAL) cabal
a23fb26 
FreeBSD CI cannot build stable-haskell cabal without OOM'ing, so it uses
ghcup cabal. That binary rejects --with-build-compiler (stable-haskell
only). Native-only FreeBSD bindists do not need dual-compiler, so skip
the flag when USE_SYSTEM_CABAL=1.

Co-authored-by: Cursor <cursoragent@cursor.com>
@angerman
angerman force-pushed the feat/aarch64-dwarf-signpost branch from 4521a53 to a23fb26 Compare September 15, 2026 02:11
@angerman

angerman commented Sep 15, 2026

Copy link
Copy Markdown
Author

Rebased onto #188 tip and fixed the FreeBSD cabal failure:

Tip: a23fb265d6. Watching CI.

angerman and others added 2 commits September 15, 2026 12:13
@angerman @cursoragent
fix: probe cabal for --with-build-compiler (not USE_SYSTEM_CABAL)
cdb0836 
CI sets USE_SYSTEM_CABAL=1 for stage2 to reuse the already-built
stable-haskell cabal; keying the flag off that env var dropped
--with-build-compiler and broke stage2 dependency solving everywhere.
Probe the cabal binary instead.

Co-authored-by: Cursor <cursoragent@cursor.com>
@angerman @cursoragent
ci(release/freebsd): clarify cabal probe vs USE_SYSTEM_CABAL
7d527b9 
Co-authored-by: Cursor <cursoragent@cursor.com>
@angerman

angerman commented Sep 15, 2026

Copy link
Copy Markdown
Author

Fix push cdb0836415: previous change keyed off USE_SYSTEM_CABAL, but CI uses that flag for stage2 with the already-built stable-haskell cabal — which still needs --with-build-compiler. Now the Makefile probes the cabal binary instead. Re-running CI.

@angerman @cursoragent
ci(release/freebsd): install Node.js before actions/checkout
db9e027 
The self-hosted FreeBSD act runner sometimes has no node on PATH, so
checkout@v5 fails immediately. Install node via pkg first.

Co-authored-by: Cursor <cursoragent@cursor.com>
@angerman

angerman commented Sep 15, 2026

Copy link
Copy Markdown
Author

Another fix — FreeBSD failed again, but this time at actions/checkout@v5 with Cannot find: node in PATH (runner env, not the Makefile). Added a pre-checkout pkg install node step on FreeBSD build+test jobs.

Tip now includes:

  1. ci: fix release.yml — Cabal pin + Windows/RockyLinux/FreeBSD platform builds #188 platform stack
  2. cabal --with-build-compiler capability probe (fixes stage2 CI)
  3. FreeBSD node bootstrap for act runner

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@luite luite Awaiting requested review from luite
@doyougnu doyougnu Awaiting requested review from doyougnu
@hsyl20 hsyl20 Awaiting requested review from hsyl20

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@angerman