Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 26 additions & 0 deletions changelog.d/8095-elf-object-source-identity.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
`cargo-test` is green again on Linux. Four `perry-codegen` tests
(`rs4gc_canonicalizes_construction_time_folds_before_root_liveness` and the
three `native_emit` construction-path tests) assert that textual and native
construction emit byte-identical output. The generated code had already
converged; the objects differed only in the name LLVM records for the module.

Nothing set `source_filename`, so LLVM fell back to whatever path reached the
assembler: the textual pipeline writes each module to a per-call temp file, so
its recorded name carried a random nonce (`perry_llvm_<nonce>.ll`), while
native construction recorded its in-memory module id (`perry_native_module`).
ELF stores that name as an `STT_FILE` symbol, so the two paths could never
agree and neither was reproducible run to run. Mach-O records no such symbol,
which is why all four passed on a macOS host and failed only on the Linux
runner.

All three module-header sites (`to_ir`, `skeleton_ir`, and the per-codegen-unit
prologue) now emit an explicit `source_filename`, so the recorded name is the
same constant on both paths and independent of the temp path. Emitted objects
no longer embed a random temp filename, so they are reproducible across runs.

The three `native_emit` tests only ever ran against the host triple, so on a
macOS machine they exercised Mach-O exclusively — the one object format that
does not record this name. `native_and_text_arms_agree_on_an_elf_target` now
pins the comparison to `x86_64-unknown-linux-gnu` and asserts the bytes start
with `\x7fELF` before comparing, so it cannot pass by testing the wrong format;
it is sabotage-tested against the un-fixed emission.
10 changes: 8 additions & 2 deletions crates/perry-codegen/src/inprocess.rs
Original file line number Diff line number Diff line change
Expand Up @@ -519,8 +519,14 @@ mod tests {
optimize_and_emit_module(&module, &target, &["-O3".into(), "-S".into()], true)
.expect("fixture emits assembly")
};
let text = emit(&text_ir, "constant_fold_text");
let folded = emit(&folded_ir, "constant_fold_native");
// Both arms must be emitted under the SAME module name. The name
// becomes the module id, and on ELF the assembler writes it into the
// object as a `.file` directive — so two differently-named arms differ
// by that one line no matter how perfectly the code itself converged.
// Mach-O records no such directive, which is why naming them apart only
// ever failed on Linux (#8087).
let text = emit(&text_ir, "constant_fold_order");
let folded = emit(&folded_ir, "constant_fold_order");
assert_eq!(
text, folded,
"construction-time constant folding must converge before RS4GC assigns root liveness"
Expand Down
64 changes: 64 additions & 0 deletions crates/perry-codegen/src/module.rs
Original file line number Diff line number Diff line change
Expand Up @@ -349,6 +349,18 @@ pub struct LlModule {
fp_flags: FpFlags,
}

/// The `source_filename` every Perry-emitted module records.
///
/// Without it, LLVM records whatever path the caller handed the assembler. The
/// textual pipeline writes each module to a per-call temp file
/// (`perry_llvm_<nonce>.ll`), so the recorded name carried a random nonce,
/// while native construction recorded its in-memory module id instead. ELF
/// stores that name as an `STT_FILE` symbol, so the two construction paths
/// could never produce byte-identical objects and neither was reproducible
/// across runs. Mach-O records no such symbol, which is why this was invisible
/// on macOS hosts and only ever failed on Linux (#8087).
pub(crate) const MODULE_SOURCE_NAME: &str = "perry_module";

impl LlModule {
pub(crate) fn declaration_lines(&self) -> impl Iterator<Item = (&str, &str)> {
self.declarations
Expand Down Expand Up @@ -680,6 +692,7 @@ impl LlModule {
pub(crate) fn skeleton_ir(&self) -> String {
let mut ir = String::new();
ir.push_str("; Generated by perry-codegen\n");
ir.push_str(&format!("source_filename = \"{MODULE_SOURCE_NAME}\"\n"));
ir.push_str(&format!("target triple = \"{}\"\n\n", self.target_triple));
if crate::codegen::helpers::native_stack_roots_enabled()
&& self.target_triple.contains("apple")
Expand Down Expand Up @@ -720,6 +733,7 @@ impl LlModule {
pub fn to_ir(&self) -> String {
let mut ir = String::new();
ir.push_str("; Generated by perry-codegen\n");
ir.push_str(&format!("source_filename = \"{MODULE_SOURCE_NAME}\"\n"));
ir.push_str(&format!("target triple = \"{}\"\n\n", self.target_triple));
if crate::codegen::helpers::native_stack_roots_enabled()
&& self.target_triple.contains("apple")
Expand Down Expand Up @@ -1030,6 +1044,7 @@ impl LlModule {
let defined: HashSet<&str> = bucket.iter().map(|f| f.name.as_str()).collect();
let mut pre = String::new();
pre.push_str("; Generated by perry-codegen (codegen unit)\n");
pre.push_str(&format!("source_filename = \"{MODULE_SOURCE_NAME}\"\n"));
pre.push_str(&format!("target triple = \"{}\"\n\n", self.target_triple));
if crate::codegen::helpers::native_stack_roots_enabled()
&& self.target_triple.contains("apple")
Expand Down Expand Up @@ -1470,6 +1485,55 @@ mod tests {
);
}

#[test]
fn every_module_header_declares_the_same_source_filename() {
// #8087: the recorded source name is what ELF stores as the object's
// `STT_FILE` symbol. If a header site omits it, LLVM substitutes the
// path that reached the assembler — a per-call temp name on the textual
// path, the in-memory module id on the native one — and the two
// construction paths can no longer produce byte-identical objects.
// Mach-O records no such symbol, so a macOS-only check of this would be
// vacuous; asserting on the emitted TEXT keeps it host-independent.
let declaration = format!("source_filename = \"{MODULE_SOURCE_NAME}\"");

let mut m = LlModule::new("x86_64-unknown-linux-gnu");
for name in ["first", "second"] {
let f = m.define_function(name, I32, vec![]);
f.create_block("entry").ret(I32, "0");
}

assert!(
m.to_ir().contains(&declaration),
"to_ir must declare the source filename:\n{}",
m.to_ir()
);

// A real split: every unit is compiled separately, so every unit
// prologue needs the declaration, not just the first.
let units = m.render_codegen_units(2);
assert_eq!(units.len(), 2, "fixture must exercise a real split");
for (i, unit) in units.iter().enumerate() {
assert!(
unit.contains(&declaration),
"codegen unit {i} must declare the source filename:\n{unit}"
);
}
}

#[cfg(feature = "llvm-inprocess")]
#[test]
fn skeleton_ir_declares_the_same_source_filename_as_to_ir() {
// The native path parses `skeleton_ir`; the textual path compiles
// `to_ir`. They must record the same name or #8087 returns.
let mut m = LlModule::new("x86_64-unknown-linux-gnu");
let f = m.define_function("only", I32, vec![]);
f.create_block("entry").ret(I32, "0");

let declaration = format!("source_filename = \"{MODULE_SOURCE_NAME}\"");
assert!(m.skeleton_ir().contains(&declaration));
assert!(m.to_ir().contains(&declaration));
}

#[test]
fn render_codegen_units_single_unit_matches_to_ir() {
let mut m = LlModule::new("arm64-apple-macosx15.0.0");
Expand Down
41 changes: 40 additions & 1 deletion crates/perry-codegen/src/native_emit.rs
Original file line number Diff line number Diff line change
Expand Up @@ -594,7 +594,14 @@ mod tests {
use crate::types::{I1, I32, I64, PTR, VOID};

fn precise_root_fixture(extra_plain_function: bool) -> LlModule {
let mut module = LlModule::new(crate::codegen::default_target_triple());
precise_root_fixture_for(
&crate::codegen::default_target_triple(),
extra_plain_function,
)
}

fn precise_root_fixture_for(triple: &str, extra_plain_function: bool) -> LlModule {
let mut module = LlModule::new(triple);
module.declare_function_with_ret_attrs("js_shadow_frame_enter", PTR, &[I32], "nonnull");
module.declare_function("js_shadow_frame_pop", VOID, &[I64]);
module.declare_function("js_shadow_slot_bind", VOID, &[I32, PTR]);
Expand Down Expand Up @@ -785,6 +792,38 @@ mod tests {
);
}

/// #8087: the same construction-path comparison, pinned to an **ELF**
/// target rather than the host's.
///
/// The three sibling tests above ran only against the host triple, so on a
/// macOS developer machine they exercised Mach-O exclusively — and Mach-O
/// records no `STT_FILE` symbol. That is precisely why a module-name
/// difference that made all of them fail on the Linux runner was invisible
/// locally for two days. Naming the object format explicitly keeps this
/// check honest on every host.
#[test]
fn native_and_text_arms_agree_on_an_elf_target() {
const ELF_TRIPLE: &str = "x86_64-unknown-linux-gnu";
let _native = crate::codegen::helpers::NativeRootsPin::native();
let module = precise_root_fixture_for(ELF_TRIPLE, false);

let text = crate::linker::compile_ll_to_object(&module.to_ir(), Some(ELF_TRIPLE))
.expect("trusted text arm emits an ELF object");
let native = compile_module_native(&module, Some(ELF_TRIPLE), "native_root_elf_fixture")
.expect("direct native arm emits an ELF object");

assert_eq!(
&text[..4],
b"\x7fELF",
"fixture must actually produce ELF, or this test proves nothing"
);
assert_eq!(
native, text,
"native and text construction must emit byte-identical ELF objects; \
a difference here is a recorded-name or lowering divergence (#8087)"
);
}

#[test]
fn split_native_construction_propagates_shadow_backend_to_workers() {
let _shadow = crate::codegen::helpers::NativeRootsPin::shadow();
Expand Down
Loading