Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
68 changes: 35 additions & 33 deletions docs/BINARY_SIZE.md
Original file line number Diff line number Diff line change
@@ -1,49 +1,51 @@
# Binary size monitoring and admission

`tests/baseline_size.txt` records 419135 bytes, the canonical Ubuntu x86_64
GCC `.text` measurement from eligible successful main ancestor
`c6e263d492828d1208fa11005c18d19b05342233`, rather than current main or PR #2032.
The successful
[main workflow run 36673112247](https://github.com/semantic-reasoning/wirelog/actions/runs/36673112247)
produced `wirelog-size-monitor-ubuntu-latest` artifact `11079113586`.
Its source, run, artifact, profile, library digests and measurement are pinned
in `tests/baseline_size.provenance.json`. The unchanged verifier checks those
identities and independently rebuilds that source to reproduce its profile and
`.text` size through the normal `trusted-ci-artifact` policy.
## Pre-stable policy mode

The report records `configure`, `build` and `test` each as success, with status
`within-budget` and a 5084-byte delta against the previous 414051-byte baseline.
The previous baseline measured main ancestor `863e011e`; subsequent changes
on main contribute to the new ancestor measurement. After #2040 found no
coherent reduction sufficient to admit PR #2032, the maintainer authorized
this measured reset. It grants fresh headroom rather than claiming a size
reduction, and does not change TDD eligibility. The fixed 5120-byte allowance
is unchanged, so the resulting ceiling is 424255 bytes. A rebuilt library may
have different non-`.text` bytes due to LTO metadata.
Until the user explicitly declares a stable version, the 5120-byte allowance
and resulting `.text` ceiling are reference values. PR CI reports a valid
overage as `over-budget` but exits successfully. Measurement failures,
production profile mismatches, malformed provenance, and tested-merge identity
failures remain blocking. The comparison reads the policy mode from the event
base; older bases without `tests/size_policy_mode.txt` use advisory mode. A
future switch to `enforced` requires the user's stable-version declaration and
a separate reviewed policy change.

Main's measurement at any given tip is a moving figure and is deliberately
not tracked here. Read it from the most recent concluded `main` run's
`wirelog-size-monitor-ubuntu-latest` artifact, and compute headroom as 424255
minus that measurement. Recording an ancestor stays valid as main advances:
the verifier requires that source to be an ancestor of the event base. The
artifact must remain unexpired, and its source must still reproduce with the
recorded toolchain profile.
## Current baseline

`tests/baseline_size.txt` records 428046 bytes, measured from the production
library for PR #2037 head `2a41e98f818082f72b3782af1e187a91b185164f` on the
canonical Ubuntu x86_64 GCC profile. The base was measured at 421283 bytes;
the head exceeded the previous 419135-byte reference by 8911 bytes. The exact
run `37110844170`, job `111168520922`, tested merge, profile, and job-log digest
are pinned in `tests/baseline_size.provenance.json`. The verifier reproduces
the source profile and `.text` measurement and limits this one-time exception
to the listed repair paths. The reset records the measured PR head as the
reference; it does not claim a size reduction. A rebuilt library may have
different non-`.text` bytes due to LTO metadata.

Main's measurement at any given tip is a moving figure. Read it from the most
recent concluded `main` run's `wirelog-size-monitor-ubuntu-latest` artifact.
Main-branch measurements remain useful for tracking size changes even while
the PR ceiling is advisory.

Historically, the 408989-byte baseline at `13d9244a` rose to 414051 bytes at
`863e011e`, consuming 5062 bytes of the same fixed allowance. This reset
replaces that ancestor measurement with the authenticated 419135-byte figure.
replaces that ancestor measurement with the PR #2037 measurement above.

The production limit remains 5120 bytes. PR CI compares the production shared
library from the exact `pull_request.base.sha` tree with the library from the
The production reference allowance remains 5120 bytes. PR CI compares the
production shared library from the exact `pull_request.base.sha` tree with the library from the
tested merge SHA. It verifies that the tested SHA is the merge commit and that
its first parent is the event base. Both libraries are configured and built on
the same runner with `-Dtests=true -DmbedTLS=disabled`, and the resolved Meson
options, compiler/linker identity and version, target, platform, and effective
wirelog build commands must match. A profile mismatch is an error. The
candidate's baseline file is used only after its main CI provenance is verified.
candidate baseline is used only after its CI evidence or exact reviewed-PR
record is verified.

Normally the head may be at most baseline + 5120 bytes. If the measured base
already exceeds that ceiling, the head may not exceed the measured base. A
The reference allowance normally places the head at baseline + 5120 bytes. If
the measured base already exceeds that ceiling, the reference size is the
measured base. In enforced mode, a head above that allowed size fails. A
docs-only change receives no special exemption; equal measured binaries pass
because they add no size to the base.

Expand Down Expand Up @@ -77,7 +79,7 @@ grant additional budget only when the verified measurement comes from an eligibl
main revision distinct from the candidate. If artifact access, toolchain
reproduction, or provenance validation fails, the update is rejected. No
workflow writes or commits baseline changes. The only reviewed-PR exceptions
are the exact, one-time records for PRs #1959 and #1961 embedded in the verifier;
are the exact, one-time records for PRs #1959, #1961, and #2037 embedded in the verifier;
each is restricted to its recorded measurement and repair paths. They grant no
general PR-based baseline eligibility and retain the same 5120-byte allowance.

Expand Down
12 changes: 8 additions & 4 deletions docs/THREADING.md
Original file line number Diff line number Diff line change
Expand Up @@ -465,7 +465,7 @@ measured by `bench/bench_intern.c`; baselines are in `docs/INTERN_PERF.md`
21 + 4 + 5 + 19 + 1 + 1 + 1 + 37 + 5 + 7 + 3 = **104 atomic call sites**
before the source-access contract below.

### 5.13 `wirelog/columnar/source_access.h` — relation source gate (20 rows)
### 5.13 `wirelog/columnar/source_access.h` — relation source gate (21 rows)

This header-only gate protects relation descriptors and canonical source storage.
It is linked into the production relation lifecycle and its gate state is zero-initialized.
Expand Down Expand Up @@ -509,11 +509,12 @@ those slots and arena allocations are quiescent.
| `source_access.h:wl_columnar_source_access_cohort_restore#2` | `cohort->source->state` | `atomic_compare_exchange_weak_explicit` | release/relaxed | Republish the source reader count after all descriptors are restored |
| `source_access.h:wl_columnar_source_access_reader_release` | `gate->state` | `atomic_load_explicit` | acquire | Observe the active gate before releasing this reader |
| `source_access.h:wl_columnar_source_access_writer_acquire` | `gate->state` | `atomic_compare_exchange_weak_explicit` | acquire/relaxed | Linearize exclusive writer admission and retry spurious failure |
| `source_access.h:wl_columnar_source_access_writer_validate` | `gate->state` | `atomic_load_explicit` | acquire | Validate that the address-bound, same-thread token still owns the exact writer sentinel before authorizing mutation without consuming the token |
| `source_access.h:wl_columnar_source_access_writer_release` | `gate->state` | `atomic_load_explicit` | acquire | Validate the writer state before terminal publication |
| `source_access.h:wl_columnar_source_access_writer_release#2` | `gate->state` | `atomic_compare_exchange_weak_explicit` | release/relaxed | Publish writer payload completion and retry spurious failure |
| `source_access.h:wl_columnar_source_access_writer_move` | `src->owner->state` | `atomic_load_explicit` | acquire | Confirm that the source token still holds WRITER before moving its address-bound ownership to another token |

### 5.14 `wirelog/columnar/relation.c` and `session.c` — alias ownership and pool promotion (25 rows)
### 5.14 `wirelog/columnar/relation.c` and `session.c` — alias ownership and pool promotion (28 rows)

The canonical owner's flattened alias count uses `wl_atomic_u64` because a
quiesced worker can retire its alias while unrelated readers still hold the
Expand All @@ -531,6 +532,9 @@ concurrent alias removals cannot underflow the count.
| `relation.c:col_rel_storage_alias_borrow_acquire#2` | `storage_alias_borrows` | `atomic_compare_exchange_weak_explicit` | release/relaxed | Publish one new alias borrow without wrapping; retry with the observed value after a lost race |
| `relation.c:col_rel_storage_alias_borrow_release` | `storage_alias_borrows` | `atomic_load_explicit` | acquire | Read the candidate count before refusing underflow or attempting retirement |
| `relation.c:col_rel_storage_alias_borrow_release#2` | `storage_alias_borrows` | `atomic_compare_exchange_weak_explicit` | acq_rel/acquire | Retire one borrow atomically while pairing peer-visible descriptor publication with removal |
| `relation.c:col_rel_mutation_set_unwind` | `init->relation->storage_alias_borrows` | `atomic_store_explicit` | relaxed | Restore the exact provisional legacy borrow snapshot under the descriptor writer; its later release publishes rollback |
| `relation.c:col_rel_mutation_set_acquire` | `relation->storage_alias_borrows` | `atomic_store_explicit` | relaxed | Initialize provisional legacy ownership under descriptor exclusion; finish publishes commit or restores the snapshot before descriptor release |
| `relation.c:col_rel_storage_alias_release_locked` | `owner->storage_alias_borrows` | `atomic_fetch_sub_explicit` | acq_rel | Retire exactly one validated alias borrow as the final old-owner access; release publishes descriptor rebinding and acquire pairs with published borrow accounting |
| `relation.c:col_rel_storage_alias_release` | `alias->storage_alias_borrows` | `atomic_store_explicit` | relaxed | Clear child-borrow metadata while the alias descriptor is exclusively held |
| `relation.c:col_rel_destroy_checked` | `r->storage_alias_borrows` | `atomic_store_explicit` | relaxed | Leave an inert pool tombstone with no live alias borrows |
| `relation.c:col_rel_destroy_checked#2` | `r->source_access.state` | `atomic_store_explicit` | release | Keep the retired pool slot closed until allocator reset or reuse |
Expand All @@ -551,7 +555,7 @@ concurrent alias removals cannot underflow the count.
| `session.c:session_pool_rel_promote#2` | `src->retained_reservation.owner_bits` | `atomic_load_explicit` | acquire | Promote a committed reservation only when the pool slot still owns it |
| `session.c:session_pool_rel_promote#3` | `src->storage_alias_borrows` | `atomic_store_explicit` | relaxed | Leave the closed pool tombstone with no child aliases |

104 + 20 + 25 = **149 atomic call sites**.
104 + 21 + 28 = **153 atomic call sites**.

The `#N` suffix counts all atomic sites in a symbol, regardless of operation;
the first site remains unsuffixed. `scripts/ci/check-threading-doc.sh` uses
Expand Down Expand Up @@ -676,7 +680,7 @@ the committed token after publication and before a growth transaction.
| `eval_dedup.c:wl_columnar_eval_dedup_test_fail_next_growth_alloc` | test-only fault flag | `atomic_store_explicit` | release | Arm one allocation refusal before a test invokes dedup growth; excluded from the production library |
| `eval_dedup.c:wl_columnar_eval_dedup_set_grow` | test-only fault flag | `atomic_exchange_explicit` | acquire-release | Consume the one-shot fault safely when test workers grow dedup tables; excluded from the production library |

The complete source audit now contains **210 atomic call sites**.
The complete source audit now contains **214 atomic call sites**.

---

Expand Down
19 changes: 17 additions & 2 deletions scripts/ci/check-text-size.sh
Original file line number Diff line number Diff line change
Expand Up @@ -10,8 +10,11 @@ json_out=
source_sha=${SOURCE_SHA:-unknown}
profile_file=${SIZE_PROFILE_FILE:-}
measure_only=no
mode_file="$repo_root/tests/size_policy_mode.txt"
mode=advisory
mode_explicit=no

usage() { echo "usage: $0 <library> [--baseline-file FILE] [--json FILE] [--source-sha SHA] [--profile FILE]" >&2; exit 2; }
usage() { echo "usage: $0 <library> [--baseline-file FILE] [--json FILE] [--source-sha SHA] [--profile FILE] [--mode advisory|enforced] [--mode-file FILE]" >&2; exit 2; }
[ "$#" -ge 1 ] || usage
library=$1; shift
while [ "$#" -gt 0 ]; do
Expand All @@ -20,13 +23,21 @@ while [ "$#" -gt 0 ]; do
--json) [ "$#" -ge 2 ] || usage; json_out=$2; shift 2 ;;
--source-sha) [ "$#" -ge 2 ] || usage; source_sha=$2; shift 2 ;;
--profile) [ "$#" -ge 2 ] || usage; profile_file=$2; shift 2 ;;
--mode) [ "$#" -ge 2 ] || usage; mode=$2; mode_explicit=yes; shift 2 ;;
--mode-file) [ "$#" -ge 2 ] || usage; mode_file=$2; shift 2 ;;
--measure-only) measure_only=yes; shift ;;
*) usage ;;
esac
done
fail() { printf 'error: %s\n' "$1" >&2; exit 2; }
[ -f "$library" ] || fail "library not found: $library"
[ "$measure_only" = yes ] || [ -f "$baseline_file" ] || fail "baseline file not found: $baseline_file"
if [ "$measure_only" != yes ]; then
if [ "$mode_explicit" = no ] && [ -f "$mode_file" ]; then
mode=$(cat "$mode_file") || fail "size policy mode file unreadable: $mode_file"
fi
case "$mode" in advisory|enforced) ;; *) fail "invalid size policy mode: $mode" ;; esac
fi

case $(uname -s) in
Linux) raw=$(size --format=sysv "$library") || fail "size failed for $library"; section=.text ;;
Expand Down Expand Up @@ -65,8 +76,12 @@ fi
if [ "$measure_only" = yes ]; then
exit 0
fi
if [ "$status" = over-budget ]; then
if [ "$status" = over-budget ] && [ "$mode" = enforced ]; then
printf '\nFAIL: .text growth (%+d) exceeds %s-byte budget\n' "$delta" "$threshold" >&2
exit 1
fi
if [ "$status" = over-budget ]; then
printf '\nADVISORY: .text growth (%+d) exceeds %s-byte reference budget\n' "$delta" "$threshold"
exit 0
fi
printf '\nPASS: .text delta within budget\n'
2 changes: 1 addition & 1 deletion scripts/ci/check-threading-doc.sh
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ fi
rows="$tmp_dir/rows"
sed -nE 's/^\| `([^`]+:[A-Za-z_][A-Za-z0-9_]*(#[0-9]+)?)` \| [^|]* \| `([^`]*)` \|.*/\1\t\3/p' "$doc" >"$rows"
row_count=$(wc -l <"$rows")
expected_rows="${WIRELOG_THREADING_EXPECTED_ROWS:-210}"
expected_rows="${WIRELOG_THREADING_EXPECTED_ROWS:-214}"
[ "$row_count" -eq "$expected_rows" ] || {
echo "check-threading-doc: FAIL: expected $expected_rows audit rows, found $row_count" >&2
exit 1
Expand Down
16 changes: 15 additions & 1 deletion scripts/ci/run-size-comparison.sh
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,12 @@
set -eu
script_dir=$(CDPATH= cd -- "$(dirname -- "$0")" && pwd -P)
repo_root=$(CDPATH= cd -- "$script_dir/../.." && pwd -P)
case "${TMPDIR:-}" in ''|/tmp|/tmp/*|/dev/shm|/dev/shm/*)
TMPDIR="${HOME:?HOME must be set}/.tmp"
export TMPDIR
;;
esac
mkdir -p "$TMPDIR" || { printf 'size comparison setup error: cannot create TMPDIR %s\n' "$TMPDIR" >&2; exit 2; }
head_build=${1:-}
base_sha=${2:-}
head_sha=${3:-}
Expand Down Expand Up @@ -35,6 +41,14 @@ baseline=$(git show "$base_sha:tests/baseline_size.txt") || die "base-owned base
printf '%s\n' "$baseline" >"$tmp/base-baseline.txt"
case "$baseline" in ''|*[!0-9]*) die "base-owned baseline is invalid" ;; esac
policy_baseline=$baseline
# Until the user declares a stable version, the size allowance is advisory.
# Older event bases lack the policy file and inherit that pre-stable mode.
policy_mode=advisory
if git cat-file -e "$base_sha:tests/size_policy_mode.txt" 2>/dev/null; then
policy_mode=$(git show "$base_sha:tests/size_policy_mode.txt") \
|| die "base-owned size policy mode is unreadable"
fi
case "$policy_mode" in advisory|enforced) ;; *) die "base-owned size policy mode is invalid" ;; esac

git show "$head_sha:tests/baseline_size.provenance.json" >"$tmp/head-baseline.provenance.json" 2>/dev/null \
|| die "candidate baseline provenance sidecar is missing"
Expand Down Expand Up @@ -67,4 +81,4 @@ base_profile=$(python3 -c 'import json,sys; d=json.load(open(sys.argv[1])); d.po
head_profile=$(python3 -c 'import json,sys; d=json.load(open(sys.argv[1])); d.pop("source_sha",None); import hashlib; print(hashlib.sha256(json.dumps(d,sort_keys=True,separators=(",",":")).encode()).hexdigest())' "$tmp/head-profile.json")
python3 "$script_dir/text-size-policy.py" --base-size "$base_bytes" --head-size "$head_bytes" \
--baseline "$policy_baseline" --base-profile "$base_profile" --head-profile "$head_profile" \
--base-sha "$base_sha" --head-sha "$head_sha" --output "$report"
--base-sha "$base_sha" --head-sha "$head_sha" --mode "$policy_mode" --output "$report"
28 changes: 19 additions & 9 deletions scripts/ci/test-early-size-gate.sh
Original file line number Diff line number Diff line change
Expand Up @@ -11,12 +11,9 @@
# the workflow would leave the behavioural assertions green -- the
# silent-downgrade shape this gate exists to prevent.
#
# 2. Behaviour -- the real gate (scripts/ci/check-text-size.sh) FAILs on
# an intentionally oversize library and PASSes on a small one, against
# the committed baseline. A fixture that only ever supplied a passing
# library would pin nothing: the negative control must drive the gate's
# own fail path, which is what #1573 moved earlier in the job so a size
# regression fails within minutes instead of ~24.
# 2. Behaviour -- the real gate reports an intentionally oversize library
# as advisory in pre-stable mode and fails it in explicitly enforced
# mode. A small library passes in both modes.
#
# The wiring half runs FIRST and needs only awk + the workflow file; the
# behavioural half needs cc/size and platform-specific size tools, and is the
Expand Down Expand Up @@ -186,7 +183,9 @@ if ! command -v cc >/dev/null 2>&1 || ! command -v size >/dev/null 2>&1 \
exit 77
fi

tmp=$(mktemp -d "${TMPDIR:-/tmp}/wirelog-early-size.XXXXXX")
case "${TMPDIR:-}" in ''|/tmp|/tmp/*|/dev/shm|/dev/shm/*) TMPDIR="${HOME:?HOME must be set}/.tmp"; export TMPDIR ;; esac
mkdir -p "$TMPDIR"
tmp=$(mktemp -d "$TMPDIR/wirelog-early-size.XXXXXX")
trap 'rm -rf "$tmp"' EXIT

# Negative control: size it relative to the committed baseline so it remains
Expand Down Expand Up @@ -240,12 +239,23 @@ else
printf 'test-early-size-gate: generated fixture is not above the budget (%s <= %s)\n' "$actual_size" "$budget_limit" >&2
failures=$((failures + 1))
fi
negative_control() {
advisory_control() {
local st=0
"$gate" "$big_so" >/dev/null 2>&1 || st=$?
[ "$st" = 0 ]
}
assert 'oversize library is advisory before stable declaration' advisory_control
enforced_control() {
local st=0
"$gate" "$big_so" --mode enforced >/dev/null 2>&1 || st=$?
[ "$st" = 1 ]
}
assert 'oversize library FAILs the size gate (negative control)' negative_control
assert 'oversize library fails in enforced mode' enforced_control
advisory_override() {
printf 'enforced\n' >"$tmp/enforced-mode.txt"
"$gate" "$big_so" --mode-file "$tmp/enforced-mode.txt" --mode advisory >/dev/null 2>&1
}
assert 'explicit advisory mode overrides the default mode file' advisory_override
fi

# Positive control: a small library must PASS against the same baseline.
Expand Down
Loading
Loading