Skip to content

m3front: define the static link on both arms of a closure call; cm3: pass debug/optimize to the C back end (fixes #1243) - #1244

Open
mikan-cerebras wants to merge 2 commits into
modula3:masterfrom
mikan-cerebras:fix-undefined-static-link
Open

mikan-cerebras wants to merge 2 commits into
modula3:masterfrom
mikan-cerebras:fix-undefined-static-link

Conversation

@mikan-cerebras

@mikan-cerebras mikan-cerebras commented Oct 28, 2026 •

Copy link
Copy Markdown

GenClosureCall emits Pop_static_link only inside the arm taken when the procedure value is a closure, but the indirect call after the join reads the static link on both arms. When the value is a top-level procedure the static link is therefore read having never been written.

m3-sys/m3front/src/types/UserProc.m3, before:

CG.If_closure (p_temp, CG.No_label, skip, CG.Maybe);
CG.Push (p_temp);
CG.Closure_frame ();
CG.Pop_static_link ();      (* the only write -- inside the branch *)
CG.Push (p_temp);
CG.Closure_proc ();
CG.Store_temp (p_temp);
CG.Set_label (skip);        (* join *)
CG.Push (p_temp);
...
CG.Gen_Call_indirect (result, cc);   (* reads the static link on both arms *)

The value really is a don't-care for a top-level procedure, since the callee ignores it. But don't care and undefined are not the same thing to an optimizer.

What goes wrong

With the C back end and clang -O2, the never-written local becomes undef:

%20 = phi ptr [ undef, %3 ], [ %15, %13 ]

undef means "any value I like", so the two arms become interchangeable and SimplifyCFGPass folds the branch, keeping the CL_marker comparison only as an assumption:

%12 = icmp eq i64 %11, -1
call void @llvm.assume(i1 %12)     ; <- was a branch
; CL_proc / CL_frame loads now unconditional

After that, every suitably aligned non-NIL procedure value takes the closure path, loads CL_proc and CL_frame out of the target function's own instruction stream, and branches to the result. The resulting pc decodes to a real instruction — in our case stp x29, x30, [sp, #0x16], the prologue of the very function that should have been called.

Bisected with -opt-bisect-limit: pass #132, SimplifyCFGPass.

Why it is intermittent

The "not 8-aligned, so not a closure" pre-check survives the fold, so whether a program crashes depends on where the linker puts the target procedure. Four variants of one test differing only in a record's field count:

int fields target proc address & 7 result
1 0x100001244 4 clean
2 0x100001288 0 crash
4 0x1000012f0 0 crash
18 0x1000015bc 4 clean

Adding an unrelated field can move a program between the two.

Why the fix is here and not in the C back end

M3C.m3's loadBeforeStore analysis would normally emit ={0} for a read-before-write temp, but it is branch-insensitive by design — its own comment says "This knowingly imagines the IR runs in order, without branches around." Patching it would fix one back end. M3CG_LLVM and M3x86 have no equivalent mitigation at all, and the undefined read originates in the front end regardless.

Note also that Pop_static_link spills the stack top to a fresh temporary on each call, so simply calling it again before the branch does not work — it allocates a second temp and the first store is dead. The fix uses one temp, written on both arms, popped once after the join.

Testing

ARM64_DARWIN, C back end, Apple clang.

  • A 27-line reproducer that passes a procedure as a parameter crashes at -O2 before and prints the correct answer after, with the target procedure forced to 8-byte alignment so the failing arm is the one exercised.
  • Nested procedures still get the correct static link. A closure reading a captured local returns identical values under patched -O2 and stock -O1 (1407, 407), so the environment is still delivered correctly — this was the obvious regression risk.
  • A ~20k-line numerical model builds at -O2 and reproduces its reference output bit-for-bit (99,999 rows, matching SHA-256), and a checkpoint-restart that segfaulted at -O2 before now completes.
  • -O1 behaviour is unchanged before and after.

Related, but not this bug

The CL_marker test itself dereferences a function pointer as data, which is undefined in C (a function is not an object, so there is nothing there to read). That is worth addressing separately, but it is not what is happening here: under opaque pointers LLVM cannot tell the pointer designates a function, and a hand transcription of the dispatch sequence into plain C does not reproduce at any optimization level. The undef is the whole trigger.


Second commit: pass the unit's debug/optimize flags to the C back end

Added after the above, because it is what makes -O2 reachable for the C back end in the first place — and therefore what exposes the bug fixed in the first commit.

Builder.m3 hardcoded TRUE, FALSE for debug, optimize at the DoRunC call site:

IF ok AND DoRunC THEN
  ok := RunCC (s, CCodeName, u.object, TRUE, FALSE, s.include_path_empty);
END;

so with the C back end, m3_optimize in an m3makefile and -O on the command line were silently ignored — every build was -O0 with debug forced on. Every other back end on the same path already passes u.debug, u.optimize, as do the other two RunCC call sites in the same file. This fixes #1243.

Measured on ARM64_DARWIN with a config carrying the current Unix.common:

flag seen by compile_c args to cc Callee_m.o
before optimize= -g -O0 f4e0ea4a…
after optimize=TRUE -g -O2 ec98460b…

_m3main.cpp is unaffected — it goes through a separate call at the bottom of the file that passes debug := TRUE, optimize := FALSE deliberately, and still reports optimize= after the change.

Please take these together, or this one second

Landing this commit alone turns on -O2 for the C back end and will produce programs that call procedure parameters through a word read out of their own instruction stream, on whichever procedures the linker happens to place at a multiple of 8. The static-link fix is what makes -O2 safe here. With both commits applied, the same test program compiles at -O2 and produces correct results for both top-level and nested procedures.

Happy to split this into two PRs if you'd prefer to review them separately — the ordering constraint is the only reason they are together.

GenClosureCall emitted Pop_static_link only inside the arm taken when
the procedure value is a closure, while the indirect call after the
join reads the static link on both arms.  For a top-level procedure
the static link was therefore read having never been written.

The value is genuinely a don't-care there -- the callee ignores it --
but "don't care" and "undefined" are not the same thing to an
optimizer.  With the C back end and clang -O2 the never-written local
becomes undef, which lets SimplifyCFG treat the two arms as
interchangeable and fold the closure test away: the CL_marker
comparison is demoted to llvm.assume, and every suitably aligned
non-NIL procedure value then takes the closure path.  It loads
CL_proc and CL_frame out of the target function's own instruction
stream and branches to the result.

Whether a given program crashes depends on where the linker places
the target procedure, because the "not 8-aligned, so not a closure"
pre-check survives the fold.  A procedure landing at 4 mod 8 escapes
through it and is called correctly; one landing at 0 mod 8 is called
through its own code.  Adding an unrelated field to a record can move
a program between the two.

Pop_static_link spills the stack top to a fresh temporary each time
it is called, so calling it a second time before the branch does not
help -- it allocates a second temp and the first store is dead.  This
uses one temp, written on both arms, popped once after the join.

Fixed in the front end rather than in the C back end's
loadBeforeStore analysis.  That analysis is branch-insensitive by
design ("This knowingly imagines the IR runs in order, without
branches around") and so does not mark this temp, but more
importantly M3CG_LLVM and M3x86 have no equivalent mitigation at all,
and the undefined read is the front end's to begin with.

Tested on ARM64_DARWIN, C back end:

  - a 27-line reproducer passing a procedure as a parameter crashes
    at -O2 before, prints the right answer after, with the target
    procedure forced to 8-byte alignment so the arm is the failing one
  - nested procedures still receive the correct static link: a closure
    reading a captured local returns identical values under patched
    -O2 and stock -O1
  - a 20k-line numerical model builds at -O2 and reproduces its
    reference output bit-for-bit, and a checkpoint-restart that
    segfaulted at -O2 before now completes
  - -O1 was unaffected throughout, before and after

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Builder.m3 hardcoded TRUE, FALSE for debug and optimize at the DoRunC
call site, so with the C back end m3_optimize in an m3makefile and -O
on the command line were silently ignored: every build was compiled
-O0 with debug forced on, whatever the m3makefile asked for.

Every other back end on the same path already passes u.debug and
u.optimize, as do the other two RunCC call sites in the same file.
Only this one did not.

The flag does reach compile_c once this is fixed, measured on
ARM64_DARWIN with a config carrying the current Unix.common:

  before:  optimize=      args=-g -O0
  after:   optimize=TRUE  args=-g -O2

and the resulting objects differ, as expected.

_m3main.cpp is unaffected: it is compiled through a separate call at
the bottom of this file that passes debug := TRUE, optimize := FALSE
deliberately, and still reports optimize= after this change.

ORDERING NOTE: this change is what exposes the undefined static link
fixed in the preceding commit.  Taking this one alone turns on -O2
for the C back end and will produce programs that call procedure
parameters through a word read out of their own instruction stream,
depending on where the linker places the target.  The two belong
together, or this one belongs after.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@mikan-cerebras mikan-cerebras changed the title m3front: define the static link on both arms of a closure call m3front: define the static link on both arms of a closure call; cm3: pass debug/optimize to the C back end (fixes #1243) Oct 28, 2026
@VictorMiasnikov

Copy link
Copy Markdown
Contributor

@mikan-cerebras

Copy link
Copy Markdown
Author

This is not https://github.com/modula3/cm3/actions/runs/40608654077 runned?

sorry I don't know how this works. Do I need to log in with my maintainer credentials and approve something? Can you approve it?

@VictorMiasnikov

VictorMiasnikov commented Oct 28, 2026 via email

Copy link
Copy Markdown
Contributor

@VictorMiasnikov

Copy link
Copy Markdown
Contributor

This is not https://github.com/modula3/cm3/actions/runs/40608654077 runned?

sorry I don't know how this works. Do I need to log in with my maintainer credentials and approve something? Can you approve it?

I can't

You can. Yes: first logon, and...

I don't know details -(

@VictorMiasnikov

VictorMiasnikov commented Oct 28, 2026 •

Copy link
Copy Markdown
Contributor

( Empty msg. May will be deleted later by my)

@VictorMiasnikov

Copy link
Copy Markdown
Contributor

(
I see:
https://github.com/modula3/cm3/actions/runs/40608654077
It have been run.

Thank!
)

Have you seen this?

AMD64_LINUX / build-and-test (llvm)
failed 10 hours ago in 3m 39s

  . . .

buildship m3-libs/m3core m3-libs/libm3 m3-libs/set m3-libs/sysutils m3-sys/m3middle m3-sys/m3objfile m3-sys/m3linker m3-sys/m3back m3-sys/m3cggen m3-sys/m3quake m3-sys/m3front m3-sys/m3cgcat m3-tools/m3bundle m3-sys/mklib m3-sys/cm3

cd /home/runner/work/cm3/cm3/m3-libs/m3core 

/home/runner/work/cm3/install/bin/cm3 -build -DBUILD_DIR=AMD64_LINUX -DROOT=/home/runner/work/cm3/cm3 -DTARGET=AMD64_LINUX -DM3_BACKEND_MODE=StAloneLlvmObj -g0
--- building in AMD64_LINUX ---

ignoring ../src/m3overrides

clang -I../src -S -emit-llvm -o../src/proc.ll ../src/Csupport/hand.c
; ModuleID = '../src/Csupport/hand.c'
source_filename = "../src/Csupport/hand.c"
target datalayout = "e-m:e-p270:32:32-p271:32:32-p272:64:64-i64:64-i128:128-f80:128-n8:16:32:64-S128"
target triple = "x86_64-pc-linux-gnu"

; Function Attrs: noinline nounwind optnone uwtable
define dso_local i64 @m3_dnew source -> compiling RTHooks.i3
new source -> compiling RT0.i3
new source -> compiling RuntimeError.i3
new source -> compiling WordRep.i3

   . . .


new source -> compiling FloatModeC.c
new source -> compiling DatePosixC.c
new source -> compiling TimePosixC.c
***  PARALLEL BACK-END (LLVM) BUILD, M3_PARALLEL_BACK = 20


***
*** runtime error:
***    <*ASSERT*> failed: LLVM.GetInstructionOpcode(loadInst) = LLVM.Opcode.LLVMStore 
***    file "../src/M3CG_LLVM.m3", line 5323
***

Aborted (core dumped)
llc: error: llc: RTLinker.mb: error: Could not open input file: No such file or directory


***
*** runtime error:
***    <*ASSERT*> failed: LLVM.GetInstructionOpcode(loadInst) = LLVM.Opcode.LLVMStore 
***    file "../src/M3CG_LLVM.m3", line 5323
***

Aborted (core dumped)
llc: error: llc: RTutils.mb: error: Could not open input file: No such file or directory


***
*** runtime error:
***    <*ASSERT*> failed: LLVM.GetInstructionOpcode(loadInst) = LLVM.Opcode.LLVMStore 
***    file "../src/M3CG_LLVM.m3", line 5323
***

Aborted (core dumped)
llc: error: llc: RTExFrame.mb: error: Could not open input file: No such file or directory


***
*** runtime error:
***    <*ASSERT*> failed: LLVM.GetInstructionOpcode(loadInst) = LLVM.Opcode.LLVMStore 
***    file "../src/M3CG_LLVM.m3", line 5323
***

Aborted (core dumped)
llc: error: llc: Text.mb: error: Could not open input file: No such file or directory
new exporters -> recompiling RTHooks.i3
new exporters -> recompiling RTAllocCnts.i3
new exporters -> recompiling RTHeapRep.i3

   . . .

new exporters -> recompiling TimePosix.i3
new exporters -> recompiling DatePosix.i3
new exporters -> recompiling Text.i3
 -> archiving libm3core.a
ar: RTLinker_m.o: No such file or directory
  make_lib => 256
librarian failed building: m3core
Fatal Error: package build failed

@VictorMiasnikov

VictorMiasnikov commented Oct 29, 2026 •

Copy link
Copy Markdown
Contributor

P.S. Can You fix this:
h--t----t--p---s---://gi--t--hub.com/modula3/cm3/i---s------s---ue---s---/1238
?

The files scripts/version

CM3VERSION d5.11.9
CM3VERSIONNUM 051109
CM3LASTCHANGED 2022-02-05
and

scripts/version.quake

CM3VERSION = "d5.11.9"
CM3VERSIONNUM = "051109"
CM3LASTCHANGED = "2022-02-05"
refer to a older release than the latest one.

cm3 --version shows, from a fresh local build, cloned from modula3/cm3 @ master:

➜  cm3.git git:(master) cm3 --version
Critical Mass Modula-3 version d5.11.9

@demoitem

Copy link
Copy Markdown
Contributor

It is best if Rodney fixes this problem. In the llvm backend there is some tenuous code to remove
instructions in the call invoke direct procedure based on knowledge of the emission of code from the
front end. In fact a comment recommends review of the front end in this case of a closure call.
Mika has in effect fixed the front end and now we have to fix any backends that had work arounds for
this situation. Its best to leave this to an expert. I'm not sure how it impacts the integrated backend
but the gcc backend seems unaffected.

@VictorMiasnikov

Copy link
Copy Markdown
Contributor

It is best if Rodney fixes this problem. In the llvm backend there is some tenuous code to remove instructions in the call invoke direct procedure based on knowledge of the emission of code from the front end. In fact a comment recommends review of the front end in this case of a closure call. Mika has in effect fixed the front end and now we have to fix any backends that had work arounds for this situation. Its best to leave this to an expert. I'm not sure how it impacts the integrated backend but the gcc backend seems unaffected.

Ok

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

C backend has code generation optimization disabled

3 participants