Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
42 changes: 39 additions & 3 deletions .github/workflows/ffi-full-integration.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
env:
FORCE_JAVASCRIPT_ACTIONS_TO_NODE24: true
permissions:
actions: read

Check warning on line 8 in .github/workflows/ffi-full-integration.yml

View check run for this annotation

SonarQubeCloud / SonarCloud Code Analysis

Move this read permission from workflow level to job level.

See more on https://sonarcloud.io/project/issues?id=hyperpolymath_bulletproof-core&issues=AaHg3tq4JT4klX7eUBO-&open=AaHg3tq4JT4klX7eUBO-&pullRequest=236
contents: read
on:
push:
Expand All @@ -16,6 +16,10 @@
- 'proven-ffi.ipkg'
- 'scripts/build-refc.sh'
- '.github/workflows/ffi-full-integration.yml'
- 'bindings/elixir/native/proven_nif/src/lib.rs'
- 'scripts/check-native-nif-exports.sh'
- 'tests/check-native-nif-exports.sh'
- 'tests/check-refc-staging.sh'
pull_request:
branches: [main]
paths:
Expand All @@ -24,11 +28,32 @@
- 'proven-ffi.ipkg'
- 'scripts/build-refc.sh'
- '.github/workflows/ffi-full-integration.yml'
- 'bindings/elixir/native/proven_nif/src/lib.rs'
- 'scripts/check-native-nif-exports.sh'
- 'tests/check-native-nif-exports.sh'
- 'tests/check-refc-staging.sh'
workflow_dispatch:
concurrency:
group: ffi-full-integration-${{ github.ref }}
cancel-in-progress: true
jobs:
nif-abi-symbol-controls:
name: Native ABI symbol validator (planted controls only)
runs-on: ubuntu-24.04
timeout-minutes: 5
steps:
- name: Checkout proven
uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11 # v4
with:
persist-credentials: false
- name: Execute real positive and negative controls
# This proves the validator detects missing exports. It does NOT
# imply any real libproven artifact was built or validated.
run: bash tests/check-native-nif-exports.sh
- name: RefC staging regression controls
# Planted empty/stale compiler outputs must not become fake evidence.
run: bash tests/check-refc-staging.sh

idris-refc-zig:
name: Idris -> RefC -> Zig
runs-on: ubuntu-latest
Expand All @@ -50,10 +75,11 @@
set -euo pipefail
if [ -z "$LANGUAGE_BRIDGES_TOKEN" ]; then
echo "available=false" >> "$GITHUB_OUTPUT"
echo "::notice::LANGUAGE_BRIDGES_TOKEN is not configured; skipping private full FFI integration."
else
echo "available=true" >> "$GITHUB_OUTPUT"
echo "::error title=Full FFI not run::LANGUAGE_BRIDGES_TOKEN is absent; genuine Idris2/RefC/Zig integration cannot be verified."
echo "PROVEN_FFI_CLASS=not_run:missing_private_dependency"
exit 2
fi
echo "available=true" >> "$GITHUB_OUTPUT"
- name: Checkout language-bridges
if: steps.bridge-token.outputs.available == 'true'
uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11 # v4
Expand Down Expand Up @@ -126,3 +152,13 @@
-Didris-c-support="$C_SUPPORT" \
-Didris-support-lib="$SUPPORT_LIB"
ls -la zig-out/lib/libproven.*
- name: Verify the genuine built artifact exports every Rustler NIF symbol
if: steps.bridge-token.outputs.available == 'true'
run: |
set -euo pipefail
artifact="$GITHUB_WORKSPACE/ffi/zig/zig-out/lib/libproven.a"
if [ ! -f "$artifact" ]; then
echo "::error::Missing actual libproven.a, not a successful integration."
exit 1
fi
bash scripts/check-native-nif-exports.sh "$artifact"
80 changes: 80 additions & 0 deletions docs/ffi/NATIVE-NIF-ABI-CONTRACT.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,80 @@
// SPDX-License-Identifier: CC-BY-SA-4.0
// SPDX-FileCopyrightText: 2026 Jonathan D.A. Jewell
= Native libproven / Rustler ABI boundary

This repository currently has a *broken consumer boundary*, not a missing
search directory alone. `bindings/elixir/native/proven_nif/src/lib.rs`
declares 66 `proven_*` C ABI functions. The current
`ffi/zig/src/main.zig` exports 17 `proven_*` functions and does not
define most of the required functions, such as `proven_math_div`.
The shared name `proven_init` also has incompatible return types:
Rust declares `i32` while Zig returns an optional `*Handle`.
These are source observations, not a verified inventory of any
generated Idris2 RefC C output.

== Actual Idris2 RefC package scope

On 2026-11-15 an isolated checkout ran `idris2 --build proven-ffi.ipkg`
and `idris2 --codegen refc --build proven-ffi.ipkg` using Idris2
0.8.0. Both compiled successfully. The package currently lists
`Proven.Core`, `Proven.SafeUrl.Parser`,
`Proven.FFI.SafeUrl` and `Proven.FFI.Main`; compilation also loads
the imported SafeUrl/Query modules. `Proven.FFI.Main` references
*SafeUrl only*. This package does not presently include an Idris2 FFI
implementation for the Rustler NIF's math, crypto, string and other
API families. A successfully generated `proven-ffi.c` is **not** a
complete 66-function Idris-backed ABI.

The existing `scripts/build-refc.sh` also previously scanned
`build/exec` and its parent `build` (double-counting the same
source), and could report success even when the compiler produced no
C. It now requires at least one *fresh* generated C source, deduplicates
the scan, and replaces stale staged sources. The five fake-compiler
controls in `tests/check-refc-staging.sh` exercise empty, stale,
fresh and failed generation, while an independent live Idris2/RefC
build has confirmed one generated source file. Neither fixture
testing nor one generated C source proves complete ABI parity.

== A mandatory precondition, not a substitute for proof

Run `bash scripts/check-native-nif-exports.sh /path/to/libproven.so`
*after* the authentic Idris2 -> RefC -> Zig library build.
For a static library, pass `libproven.a` instead.

The checker extracts required C symbols from the Rust `extern "C"` block
and confirms that the compiled library *defines* all of them. Missing
names cause exit 1; missing tools/inputs or unreadable artifacts cause
exit 2 (NOT RUN); only a complete symbol list gives exit 0.

The script expressly **does not verify ABI types, memory ownership,
verified Idris2 provenance, dynamic dependencies, or behavioural
semantics**. Its positive controls are fixture libraries, not
`libproven` evidence. Type/layout compatibility must additionally be
checked against an authoritative C header, with compiled calls and
negative fixtures, before declaring the full interface safe.
Actual runtime behaviour must be exercised from Elixir and Rust.

== Required upstream repairs

. Resolve the currently private `nextgen-languages/language-bridges`
dependency with correctly scoped credentials or a reviewed vendor pin.
. Produce genuine Idris2 RefC code, with pinned compiler/runtime inputs,
and include the required ABI implementation in the Zig linker.
. Agree whether the consumer links a static or shared `libproven`.
The Zig build currently emits a static target while Rust explicitly
requests `dylib=proven`.
. Repair all 66 function contracts, including `proven_init` return
type and correct ownership of returned buffers.
. Check exported symbols on *the exact artifact to be consumed*; run
typed calls against it and the real Elixir/Rustler integration.
. Publish immutable, attestable artifact provenance and a pinned,
reproducible consumer configuration. Never silently substitute
`build_standalone.zig`, which builds a narrower pure Zig ABI.

== Positive and negative gate controls

`bash tests/check-native-nif-exports.sh` runs six isolated cases:
a defined shared and static fixture, deliberately missing symbols,
missing input, an empty contract, and a corrupt binary. These tests
establish only that the name-presence gate fails closed; they do
not certify this repository's actual `libproven` ABI.
63 changes: 30 additions & 33 deletions scripts/build-refc.sh
Original file line number Diff line number Diff line change
Expand Up @@ -121,6 +121,11 @@

mkdir -p "$REFC_OUTPUT_DIR"

# A successful compiler exit is not enough: the C must be produced by THIS
# invocation. A prior run's generated file cannot serve as fresh evidence.
REFC_BUILD_STAMP=$(mktemp "$PROVEN_ROOT/build/.refc-start.XXXXXXXX")
trap 'rm -f "$REFC_BUILD_STAMP"' EXIT

# ---------------------------------------------------------------------------
# Step 1: Compile Idris2 to C via RefC backend
# ---------------------------------------------------------------------------
Expand Down Expand Up @@ -150,51 +155,43 @@
EXEC_DIR="$PROVEN_ROOT/build/exec"
BUILD_DIR="$PROVEN_ROOT/build"

# Find all generated .c files from the RefC compilation
# Only include RefC output created by the current compiler invocation.
# Do not also scan BUILD_DIR: it contains EXEC_DIR AND the previously staged
# build/refc directory, which made the old implementation double-count outputs
# and accept stale C as though Idris2 had generated it again.
GENERATED_C_FILES=()
GENERATED_H_FILES=()

# RefC puts generated C code in the output directory
for dir in "$EXEC_DIR" "$BUILD_DIR"; do
if [ -d "$dir" ]; then
while IFS= read -r -d '' f; do
GENERATED_C_FILES+=("$f")
done < <(find "$dir" -name "*.c" -print0 2>/dev/null)

while IFS= read -r -d '' f; do
GENERATED_H_FILES+=("$f")
done < <(find "$dir" -name "*.h" -print0 2>/dev/null)
fi
done

if [ ${#GENERATED_C_FILES[@]} -eq 0 ]; then
log "WARNING: No .c files found in build directory."
log "The RefC backend may have placed output in a different location."
log "Checking alternative locations..."
if [ -d "$EXEC_DIR" ]; then

Check failure on line 165 in scripts/build-refc.sh

View check run for this annotation

SonarQubeCloud / SonarCloud Code Analysis

Use '[[' instead of '[' for conditional tests. The '[[' construct is safer and more feature-rich.

See more on https://sonarcloud.io/project/issues?id=hyperpolymath_bulletproof-core&issues=AaHg_SaDtvBvsV68V9zq&open=AaHg_SaDtvBvsV68V9zq&pullRequest=236
while IFS= read -r -d '' f; do
GENERATED_C_FILES+=("$f")
done < <(find "$EXEC_DIR" -type f -name '*.c' -newer "$REFC_BUILD_STAMP" -print0 2>/dev/null)
Comment thread
hyperpolymath marked this conversation as resolved.
Resolved
Comment thread
hyperpolymath marked this conversation as resolved.
Resolved
while IFS= read -r -d '' f; do
GENERATED_H_FILES+=("$f")
done < <(find "$EXEC_DIR" -type f -name '*.h' -newer "$REFC_BUILD_STAMP" -print0 2>/dev/null)
fi

# Check if RefC created a single output binary instead of .c files
if [ -f "$EXEC_DIR/proven-ffi" ]; then
log "Found compiled binary at $EXEC_DIR/proven-ffi"
log "Note: RefC may have compiled directly. Re-run with --codegen refc --cg-opt --no-compile for .c output."
fi
else
log "Found ${#GENERATED_C_FILES[@]} generated C file(s):"
for f in "${GENERATED_C_FILES[@]}"; do
verbose " $(realpath --relative-to="$PROVEN_ROOT" "$f")"
done
if [ "${#GENERATED_C_FILES[@]}" -eq 0 ]; then

Check failure on line 174 in scripts/build-refc.sh

View check run for this annotation

SonarQubeCloud / SonarCloud Code Analysis

Use '[[' instead of '[' for conditional tests. The '[[' construct is safer and more feature-rich.

See more on https://sonarcloud.io/project/issues?id=hyperpolymath_bulletproof-core&issues=AaHg_SaDtvBvsV68V9zr&open=AaHg_SaDtvBvsV68V9zr&pullRequest=236
die "RefC generated no fresh C source; cannot claim a build artifact. No prior staged output may be reused."
fi

if [ ${#GENERATED_H_FILES[@]} -gt 0 ]; then
log "Found ${#GENERATED_H_FILES[@]} generated header file(s):"
for f in "${GENERATED_H_FILES[@]}"; do
verbose " $(realpath --relative-to="$PROVEN_ROOT" "$f")"
done
log "Found ${#GENERATED_C_FILES[@]} fresh generated C file(s):"
for f in "${GENERATED_C_FILES[@]}"; do
verbose " $(realpath --relative-to="$PROVEN_ROOT" "$f")"
done

if [ "${#GENERATED_H_FILES[@]}" -gt 0 ]; then

Check failure on line 183 in scripts/build-refc.sh

View check run for this annotation

SonarQubeCloud / SonarCloud Code Analysis

Use '[[' instead of '[' for conditional tests. The '[[' construct is safer and more feature-rich.

See more on https://sonarcloud.io/project/issues?id=hyperpolymath_bulletproof-core&issues=AaHg_SaDtvBvsV68V9zs&open=AaHg_SaDtvBvsV68V9zs&pullRequest=236
log "Found ${#GENERATED_H_FILES[@]} fresh generated header file(s):"
fi

# ---------------------------------------------------------------------------
# Step 3: Copy/symlink generated files to build/refc/ for Zig consumption
# ---------------------------------------------------------------------------

# Remove stale staged C/H only after verifying fresh RefC output exists.
# This directory is owned by this build script and must represent one run.
find "$REFC_OUTPUT_DIR" -maxdepth 1 -type f \( -name '*.c' -o -name '*.h' \) -delete

# Copy generated C files into a clean staging directory that Zig can reference
for f in "${GENERATED_C_FILES[@]}"; do
cp -v "$f" "$REFC_OUTPUT_DIR/" 2>/dev/null || cp "$f" "$REFC_OUTPUT_DIR/"
Expand Down
94 changes: 94 additions & 0 deletions scripts/check-native-nif-exports.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,94 @@
#!/usr/bin/env bash
# SPDX-License-Identifier: MPL-2.0
# SPDX-FileCopyrightText: 2026 Jonathan D.A. Jewell
#
# Verify the *defined* exported symbols in a compiled libproven artifact
# cover the Rustler consumer's declared C ABI. This is a necessary, not
# sufficient, ABI condition: it does not establish compatible signatures,
# ownership, verified Idris provenance, or correct behaviour.
#
# Usage: bash scripts/check-native-nif-exports.sh LIBPROVEN [NIF_LIB_RS]
# Exit 0: all required names are defined; 1: missing symbols; 2: NOT RUN.
set -euo pipefail

if [[ $# -lt 1 || $# -gt 2 ]]; then
echo 'PROVEN_NIF_ABI_CLASS=not_run:usage' >&2
echo 'usage: check-native-nif-exports.sh LIBPROVEN [NIF_LIB_RS]' >&2
exit 2
fi

root=$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)
lib=$1
nif=${2:-"$root/bindings/elixir/native/proven_nif/src/lib.rs"}

if [[ ! -f $lib || ! -r $lib || ! -f $nif || ! -r $nif ]]; then
echo 'PROVEN_NIF_ABI_CLASS=not_run:missing_input' >&2
exit 2
fi
for cmd in awk comm nm sort wc; do
command -v "$cmd" >/dev/null 2>&1 || {
echo "PROVEN_NIF_ABI_CLASS=not_run:missing_tool:$cmd" >&2
exit 2
}
done

tmp=$(mktemp -d)
trap 'rm -rf -- "$tmp"' EXIT

# Scan only the Rust extern-C block. Do not count Rust NIF wrappers or
# incidental function names elsewhere in the file.
awk '
/^[[:space:]]*extern "C"[[:space:]]*\{/ { in_extern=1; next }
in_extern && /^[[:space:]]*\}/ { in_extern=0 }
in_extern && /^[[:space:]]*((pub(\([^)]*\))?|safe|unsafe)[[:space:]]+)*fn[[:space:]]+proven_[A-Za-z0-9_]+[[:space:]]*\(/ {
line=$0
sub(/^[[:space:]]*((pub(\([^)]*\))?|safe|unsafe)[[:space:]]+)*fn[[:space:]]+/, "", line)
sub(/[[:space:]]*\(.*/, "", line)
print line
}
' "$nif" | LC_ALL=C sort -u > "$tmp/required"

if [[ ! -s $tmp/required ]]; then
echo 'PROVEN_NIF_ABI_CLASS=not_run:empty_contract' >&2
exit 2
fi

case $lib in
*.a) nm -g --defined-only "$lib" > "$tmp/nm" 2>"$tmp/nm-error" || {
echo 'PROVEN_NIF_ABI_CLASS=not_run:unreadable_archive' >&2
cat "$tmp/nm-error" >&2
exit 2
} ;;
*.so|*.so.*) nm -D --defined-only "$lib" > "$tmp/nm" 2>"$tmp/nm-error" || {
echo 'PROVEN_NIF_ABI_CLASS=not_run:unreadable_shared_lib' >&2
cat "$tmp/nm-error" >&2
exit 2
} ;;
*)
echo 'PROVEN_NIF_ABI_CLASS=not_run:unsupported_artifact' >&2
exit 2 ;;
esac

# Use only defined symbols; never mistake an undefined (U) reference for
# evidence that a function exists. nm --defined-only excludes those.
# Default-version definitions satisfy unversioned consumers; single-@ aliases do not.
awk '{
symbol=$NF
sub(/@@[^@[:space:]]+$/, "", symbol)
if (symbol ~ /^proven_[A-Za-z0-9_]+$/) print symbol
}' "$tmp/nm" |
LC_ALL=C sort -u > "$tmp/defined"

LC_ALL=C comm -23 "$tmp/required" "$tmp/defined" > "$tmp/missing"

required_count=$(wc -l < "$tmp/required")
defined_count=$(wc -l < "$tmp/defined")
missing_count=$(wc -l < "$tmp/missing")
echo "NIF ABI symbols: required=$required_count artifact_defined=$defined_count missing=$missing_count"
if [[ -s $tmp/missing ]]; then
echo 'PROVEN_NIF_ABI_CLASS=missing_symbols' >&2
sed 's/^/ missing C ABI symbol: /' "$tmp/missing" >&2
exit 1
fi
echo 'PROVEN_NIF_ABI_CLASS=symbols_present_only'
echo 'NOTE: symbol coverage is NOT ABI type conformance or formal proof.'
Loading
Loading