Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 24 additions & 1 deletion .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -161,6 +161,8 @@ jobs:

steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false # nothing here pushes

# The required "SonarCloud Code Analysis" check is the GitHub App's
# automatic analysis -- nothing in this repository configures or runs it,
Expand Down Expand Up @@ -304,6 +306,15 @@ jobs:
renv-${{ runner.os }}-R${{ env.R_APT_VERSION }}-${{ hashFiles('renv.lock') }}-
renv-${{ runner.os }}-R${{ env.R_APT_VERSION }}-

# SHA-512 of the cache contents before and after the install, so the save below
# is skipped when this run added nothing (the key is unique per run, so an
# unchanged cache would otherwise be uploaded again for no benefit).
- name: Hash the renv cache before the install
id: renv_before
run: |
sudo mkdir -p /opt/renv-cache
echo "hash=$(sudo find /opt/renv-cache -type f -print0 | LC_ALL=C sort -z | sudo xargs -0 -r sha512sum | sha512sum | cut -d" " -f1)" >> "$GITHUB_OUTPUT"

# R_CLI_DYNAMIC and TERM make renv print progress one line at a time in the log.
- name: Install R packages
timeout-minutes: 25
Expand Down Expand Up @@ -331,8 +342,14 @@ jobs:
if: always()
run: sudo chmod -R a+rX /opt/renv-cache || true

- name: Save the renv cache
- name: Hash the renv cache after the install
id: renv_after
if: always()
run: |
echo "hash=$(sudo find /opt/renv-cache -type f -print0 | LC_ALL=C sort -z | sudo xargs -0 -r sha512sum | sha512sum | cut -d" " -f1)" >> "$GITHUB_OUTPUT"

- name: Save the renv cache
if: always() && steps.renv_after.outputs.hash != steps.renv_before.outputs.hash
uses: actions/cache/save@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
with:
path: /opt/renv-cache
Expand Down Expand Up @@ -580,6 +597,8 @@ jobs:
timeout-minutes: 60
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false # nothing here pushes

- name: Set up Julia
timeout-minutes: 10
Expand Down Expand Up @@ -739,6 +758,8 @@ jobs:
AGDA_IMAGE: debian:trixie-20250929-slim@sha256:1caf1c703c8f7e15dcf2e7769b35000c764e6f50e4d7401c355fb0248f3ddfdb
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false # nothing here pushes

# A pass is memoised on its inputs: the proofs, the checker pins and the
# invocation. Pull requests reuse a pass whose inputs are unchanged, which
Expand Down Expand Up @@ -869,6 +890,8 @@ jobs:
timeout-minutes: 30
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false # nothing here pushes
- name: Install proof tools
timeout-minutes: 10
run: |
Expand Down
54 changes: 54 additions & 0 deletions .github/workflows/julia-canary.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
# SPDX-License-Identifier: MPL-2.0
# SPDX-FileCopyrightText: 2026 Jonathan D.A. Jewell (hyperpolymath) <j.d.a.jewell@open.ac.uk>
#
name: Julia canary

# Weekly check that the dependency set still resolves and installs when
# Julia's package resolver is free to pick the newest compatible versions. The
# committed Manifest.toml pins CI; this run deliberately ignores it, so drift
# shows up here on a schedule instead of surprising the next pull request.
# It is not a required check and changes no ruleset.

on:
schedule:
- cron: '17 5 * * 1'
workflow_dispatch:

permissions:
contents: read

concurrency:
group: julia-canary
cancel-in-progress: true

jobs:
canary:
name: Julia dependency canary
runs-on: ubuntu-24.04
timeout-minutes: 30
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false # nothing here pushes

- name: Set up Julia
timeout-minutes: 10
run: bash scripts/install-ci-julia.sh

- name: Resolve without the Manifest
timeout-minutes: 10
run: |
rm -f Manifest.toml
julia --project=. -e 'import Pkg; Pkg.instantiate()'

- name: Report what moved
run: |
{
echo '### Julia canary'
echo
echo 'Newest compatible versions resolved without Manifest.toml. Outdated packages:'
echo
echo '```'
julia --project=. -e 'import Pkg; Pkg.status(outdated=true)' 2>&1
echo '```'
} >> "$GITHUB_STEP_SUMMARY"
41 changes: 41 additions & 0 deletions docs/maintenance/ci-speed-upstream-patch.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
# CI speed work: what it changed and how to take it upstream

This records the CI speed, fail-early and diagnostics work merged on the fork in
PR #328, with the follow-up in this change. It is a note for whoever carries the
work to `JoshuaJewell/MetaManifold-WebUI`; nothing here has been pushed or posted there.

## What changed

- Precompiled Julia caches keyed on the Julia pin, the R apt pin and `Manifest.toml`.
- Fail-early gates, per-step timing and a classifier for failure causes.
- An Agda pass memo: a pull request reuses an earlier pass when the proofs, the
checker pins and the invocation are unchanged.
- All eleven Python files replaced by GNU `bc`, `awk` and Julia. No Python tests
or scripts remain.
- The renv cache save is skipped when a SHA-512 of the cache contents is the same
before and after the R install.
- `persist-credentials: false` on the four `ci.yml` checkouts that were missing it.
- A weekly, non-required Julia dependency canary (`julia-canary.yml`).

## Measured, cold cache (job timings)

| Step | Before | After |
| --- | --- | --- |
| Julia tests job | 27m58s | 23m38s |
| Instantiate | 3m55s | 31s |
| Validated-statistics step | 2m37s | 49s |
| Full-suite "Run tests" step | 17m52s | 13m28s |
| Statistics standards job | 17m49s | 8m39s |
| Mutation campaign | 11m25s | 5m13s |

"Rebuild RCall" is still about 5m04s because `Pkg.build` precompiles.

Warm-cache timings: not measured yet. They are added below once a second run on
the same cache key has finished.

## Taking it upstream

A patch against upstream `89be534` applied cleanly in a scratch clone. Generate it
from the merge commit `c20ae05` and this change, apply it on a branch of the
upstream repository, and let upstream CI run. No ruleset, required check or job
name changes, and no matrix is added.
Loading