Skip to content

Declare Julia Runic for worker pre-push formatting - #350

Draft
ChrisRackauckas-Claude wants to merge 1 commit into
masterfrom
agent/349-runic-declaration
Draft

ChrisRackauckas-Claude wants to merge 1 commit into
masterfrom
agent/349-runic-declaration

Conversation

@ChrisRackauckas-Claude

@ChrisRackauckas-Claude ChrisRackauckas-Claude commented Nov 6, 2026 •

Copy link
Copy Markdown
Member

🚧 UNREVIEWED — awaiting review by @ChrisRackauckas. Filed by an AI agent running as @ChrisRackauckas; Chris has not seen this.
Harness: Codex CLI 0.164.0 · Model: gpt-6-astra
Conversation: local job 349-codex (no session URL)

Declare the repository's existing Julia Runic formatter in the supported .factory-pre-push.json interface. The installed worker hook does not recognize the reusable Runic CI workflow, and the executable named runic resolves to an unrelated Python package; the declaration selects julia --project=@runic -m Runic --check explicitly. Only this five-line declaration changes, with no formatter exceptions, dependency changes, or CI/hook modifications.

Followup 1; publication prerequisite for #349. This PR does not close that issue or publish its resizing implementation. The original behavior commit and all its validation evidence remain unchanged and unpublished pending review and merge of this prerequisite. Ignore this draft until reviewed by @ChrisRackauckas.

Before / after verification

Verified Julia 1.12.4 and Julia Runic 1.10.0 (package UUID 62bfec6d-59d7-401d-8490-b29ee721c001) in the existing @runic environment:

julia --project=@runic -e 'using Runic; println("Julia ", VERSION); println("Runic ", pkgversion(Runic)); println("Runic UUID: ", Base.PkgId(Runic).uuid)'
Julia 1.12.4
Runic 1.10.0
Runic UUID: 62bfec6d-59d7-401d-8490-b29ee721c001

JSON parsing and the installed hook's actual formatter-schema validator both passed and selected exactly:

["julia", "--project=@runic", "-m", "Runic", "--check", "{files}"]

That argv returned exit 0 on the unchanged feature branch's two Julia files (src/noise_interfaces/noise_process_interface.jl and test/resize_test.jl). Direct Runic checks accepted fixture(x) = x + 1, rejected fixture(x)=x+1, and rejected an incomplete function definition, without modifying any fixture.

The original installed-hook rejection was retained and reproduced against the unchanged behavior commit, with a real nonempty ref-update record:

worker pre-push: typos -- docs/src/api/interface.md src/noise_interfaces/noise_process_interface.jl test/resize_test.jl
worker pre-push: BLOCKED: no declared Julia formatter; declare Runic or the repository formatter in .factory-pre-push.json

A scratch harness invoked the installed pre-push entrypoint, using Git's four-field local_ref new_commit remote_ref old_commit input format. Diagnostic commits descend from the committed declaration and contain the stated fixture changes; every update has distinct old/new commits. These are committed-snapshot checks, not dry/no-op pushes. No diagnostic commits or fixtures are on the published branch.

Real harness output (all expected exits asserted):

before-feature: exit 1 (expected 1)
declaration: exit 0 (expected 0)
formatted: exit 0 (expected 0)
unformatted: exit 1 (expected 1)
syntax: exit 1 (expected 1)
spelling: exit 1 (expected 1)
conflict: exit 1 (expected 1)
json: exit 1 (expected 1)
All committed-snapshot hook probes passed; branches unchanged.

The formatted diagnostic commit produced these actual gate commands and exit 0:

worker pre-push: typos -- probe.jl
worker pre-push: julia --project=@runic -m Runic --check probe.jl

The unformatted diagnostic commit ran the same two commands and was blocked when Runic returned exit 1. Separate negative probes confirmed that spelling, conflict-marker, Julia syntax, and JSON checks still reject invalid changes; no exceptions or bypass switches were used.

The actual publication push of this declaration also passed the installed gate:

worker pre-push: typos -- .factory-pre-push.json
worker pre-push: no formatter selected for .json; full repository checks remain required

The JSON diagnostic is the hook's existing behavior for repositories without a JSON formatter: JSON parsing and spelling checks still ran, as demonstrated by the malformed-JSON and spelling negative probes. The separate worktree used git -c core.hooksPath="$worker_hooks" push to enforce the same installed gate without changing persistent hook settings. git diff origin/master HEAD --check and typos .factory-pre-push.json also passed.

Limits and review points

  • This configuration-only change does not alter Julia source, docstrings, tests, dependencies, or the reusable CI workflow. Core, QA, and docs were not rerun for this followup; the original resizing branch's complete before/after evidence remains retained.
  • The command requires Julia and a provisioned @runic environment. Missing tools remain failures; no installation or fallback is added. Other operating systems and worker environments were not exercised locally.
  • No independent review has occurred in this solo writer job. Review the supported formatter declaration and the existing named-environment requirement before merging.
  • After a trusted prerequisite merge, a later same-author followup must merge it into the feature branch, repeat required feature validation, and publish the feature separately. Nothing here releases an API for downstream consumers.

CI investigation (followup 2)

The Dependabot Auto-merge run at this exact head failed workflow validation before creating jobs. Its annotation reports that the reusable job requests checks: read and statuses: read, while the caller permits checks: none and statuses: none.

This is a confirmed pre-existing caller/reusable-workflow permission mismatch, already tracked by SciML/.github issue 141. The same annotation appears in the October 17 run and the October 29 run. All three runs resolve the reusable workflow to 41f457c608add5e7edab9173e6f8e979083fb475, and their caller workflow has the same blob as current master (e24b444d0786096cb1c778f32546ebb42a619ab2). The historical runs are pull-request runs, not default-branch push runs; the caller triggers only on pull_request.

The exact caller is DependabotAutoMerge.yml on current master, and the requested scopes come from the resolved shared workflow. Handoff belongs to the SciML coordinator/shared-workflow maintainers through the existing unassigned issue. No workflow, permission, settings, or required-check change is part of this PR. The failure is not waived, and other pending checks are not counted as successful.

Downstream SDE3 baseline failure

The SDE3 job at this head failed in test/SDE3/rode.jl:19, noise iip tests, with:

ArgumentError: `TrackedArray`s do not support `setindex!`, so they cannot be assigned to or used as a broadcast destination. Use `y = f.(x)` instead.
setup_next_step! at src/noise_interfaces/noise_process_interface.jl:137
Some tests did not pass: 114 passed, 0 failed, 1 errored, 2 broken.

This is the existing failure tracked by issue 346 and its separate fix PR 347. The October 27 SDE3 job has the same error and totals. A reduced diagnostic was also run locally against a clean current-master checkout and this unchanged mechanical head, both using Julia 1.13.3, DiffEqNoiseProcess 5.36.5, ReverseDiff 1.18.7, and SciMLBase 3.58.0:

using DiffEqNoiseProcess, ReverseDiff
function noise_sum(x)
    W = WienerProcess!(0.0, x)
    setup_next_step!(W, nothing, nothing)
    return sum(W.curW)
end
ReverseDiff.gradient(noise_sum, zeros(2))

Both executions exited 1 at the same TrackedArray mutation and setup_next_step! line. All 121 dependency entries match between the two reduced environments, apart from the developed package path; all 120 versioned entries match the current SDE3 CI log. The source-path change also changes the environment's project hash. This is an observed baseline reproduction, not an inference from the small PR diff.

The full downstream group was not rerun locally, and the separate fix was not modified or revalidated here. Handoff stays with the existing issue 346 / PR 347 author and the SciML coordinator. No tests, assertions, existing broken markers, or workflow settings were changed; no numerical fix is mixed into this declaration PR.

Risk assessment

  • Risk: high (H6: this repository declaration governs the executable command used by the worker pre-push hook).
  • Blast radius: worker pre-push formatting of changed Julia files in this repository; no package API or runtime behavior changes.
  • Evidence: installed schema validation, actual Julia Runic checks, original rejection reproduced, positive and negative committed-snapshot gate probes, and the declaration push passing the installed hook.
  • Independent review: pending master-chain review; no independent rating is claimed.
  • Merge: needs human review — H6 requires Chris’s decision under the fleet risk policy; CI and all master-chain gates still apply. This author assessment grants no merge authority.

L3 (repository CI configuration) was considered, but the more specific H6 criterion governs hooks and their instructions. This declaration selects the formatter command the pre-push gate executes. The policy requires the highest matching level and explicitly says that a small mechanical diff does not lower H6 to low. No numerical/public-API, dependency, cross-repository-template, or removed-verification criterion is introduced by this five-line diff. This is the author’s classification, not an independent review.


🤖 Posted by an AI agent — harness: Codex CLI 0.164.0 · model: gpt-6-astra
Conversation: local job 349-codex (no session URL)

Select the repository's existing Runic formatter through the supported
formatter declaration, using Julia's Runic module to avoid the unrelated
executable named runic.

Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
Co-Authored-By: Codex <noreply@openai.com>
Agent-Harness: Codex CLI 0.164.0
Agent-Model: gpt-6-astra
Agent-Session: local job 349-codex (no session URL)
@ChrisRackauckas

Copy link
Copy Markdown
Member

🤖 Automated comment from an AI agent running as @ChrisRackauckas — not written or reviewed by Chris.

PR: #350

Reviewed head: dbcb5da
Raw SHA-256: 23da662cd041ff717c29fb96e3034f12420a89f42df011ae90680bdfe0ba619a
Source SHA-256: 0be89e1168cec0168662205f34040369a69c5e81b57fb5e20d22cc3f0ef593ef
Source: standard-review-v1 (harness-log)
Reviewer session: 01a1d254-067b-72f0-a1c3-ed11ba1f80c6
Slot: 1
Confidence: unknown
Own risk fixed before author rating/discussion: yes

Risk assessment

  • Risk: low (L3)
  • Verdict: CHANGES
  • Reviewed head: dbcb5da

Review report

VERDICT: CHANGES
Round: 1
Head: dbcb5da
Slot: 1
CI: CI is not green (regress:SciMLSensitivity.jl/SDE3/1 / Downstream Tests - SDE3)
Session: 01a1d254-067b-72f0-a1c3-ed11ba1f80c6

Risk

low (L3: repository-local pre-push formatter configuration; no package source, dependencies, public API, or assertions change).

Blocking findings

  • [P2] Needs verification: downstream regression gate remains unresolved. The supplied CI status identifies SciMLSensitivity.jl/SDE3/1 as a regression. I did not independently reproduce that failure or establish its default-branch behavior. The risk policy requires no regressions against the default branch, and missing evidence remains blocking. Run SDE3 on this head and origin/master with the same interpreter and dependency resolution; fix a head-only failure or provide evidence establishing the baseline failure. This is an unresolved verification gate, not a confirmed defect in .factory-pre-push.json.

Non-blocking findings

None. No correctness defect found in the formatter declaration.

What I ran

  • Reviewed git diff origin/master...HEAD, Project.toml, the formatting/test workflows, test routing, and the affected worker formatter implementation and tests. The entire diff adds only .factory-pre-push.json. Root AGENTS.md and CLAUDE.md are absent. No PR body, discussion, author risk rating, or earlier review was read.
  • uv run --locked --no-sync --project <local path> python --version: Python 3.12.13. This project has uv.lock; its interpreter ran all Python tests and probes.
  • With TMPDIR set as requested, bytecode disabled, and pytest artifacts under scratch/: uv run --locked --no-sync --project <local path> python -m pytest -q -p no:cacheprovider --basetemp <local path> <local path>: 111 passed in 55.37s. These include fixture-based Julia hook tests; real Runic execution was checked separately.
  • Ran scratch/formatter_probes.py through the same locked interpreter: all 8 targeted probes passed. The base workflow alone reproduced no declared Julia formatter; the head policy selected its exact argv. Through the worker's command runner, real Runic returned 0 for formatted files, including multiple filenames and spaces, and 1 for unformatted input, malformed syntax, a mixed good/bad batch, and a missing file. Source hashes stayed identical. A shadow runic executable returning 99 was bypassed.
  • julia --version and loading Runic in --project=@runic: Julia 1.13.2, Runic 1.13.0. Preserved the supplied layered JULIA_DEPOT_PATH.
  • Locked typos .factory-pre-push.json, git diff --check origin/master...HEAD, and final git status --short: exit 0, no diagnostics, checkout clean.
  • Reviewer: codex-cli 0.164.0, model gpt-6.1-sol; session ID above has no supplied session URL.

What I did not verify

  • The supplied downstream SDE3 failure, its logs, or a controlled default-branch comparison; therefore the CI gate is not cleared.

  • Package Core/QA numerical suites, which the configuration-only diff does not affect; alternative Julia versions or installation of the shared @runic environment on other workers.

  • The Julia checkout has no Python project/lockfile. An initial unqualified uv run --locked python --version printed Python 3.15.0 and warned that --locked had no effect; that interpreter was not used for the tests or probes.

  • Verdict: CHANGES

  • Round: 1

  • Blocking priorities: P2

  • Deferred P2 issues: none supplied.

AUTHOR_GRADE: {"task": "ci", "score": 4, "difficulty": 1, "rounds": 1, "author_harness": "Codex CLI 0.164.0", "author_model": "gpt-6-astra", "note": "The narrow formatter declaration passed 111 hook tests and real Runic acceptance/rejection probes without rewriting inputs. No code defect found; the supplied downstream regression remains unverified and blocks the no-regression gate."}

Head link: dbcb5da


🤖 Posted by an AI agent — harness: codex-cli 0.164.0 · model: gpt-6.1-sol
Conversation: local session 01a1d254-067b-72f0-a1c3-ed11ba1f80c6 (no session URL)

@ChrisRackauckas

Copy link
Copy Markdown
Member

🤖 Automated comment from an AI agent running as @ChrisRackauckas — not written or reviewed by Chris.

PR: #350

Reviewed head: dbcb5da
Raw SHA-256: b4ba6c5ba63812ae4962e09593a894c5f732fba2cdde268d552845f716ed7856
Source SHA-256: 9fd06535f99ffdaa163d390820bbc73abc0df5546f005992f255878ac0f9b482
Source: standard-review-v1 (harness-log)
Reviewer session: 01a1d25a-5609-7aa3-b6a4-9ce8f609a535
Slot: 1
Confidence: unknown
Own risk fixed before author rating/discussion: yes

Risk assessment

  • Risk: low (L3)
  • Verdict: MERGE
  • Reviewed head: dbcb5da

Review report

VERDICT: MERGE
Round: 2
Head: dbcb5da
Slot: 1
CI: CI is not green (regress:SciMLSensitivity.jl/SDE3/1 / Downstream Tests - SDE3)
Session: 01a1d25a-5609-7aa3-b6a4-9ce8f609a535

Risk

low (L3: repository-local pre-push formatter configuration; no package source, tests, dependencies, or workflow changes)

Blocking findings

None.

Non-blocking findings

None. The round-two delta is empty. The whole PR adds only .factory-pre-push.json:1–5, declaring the Runic check command consistently with .github/workflows/FormatCheck.yml:18.

What I ran

  • git diff origin/master...HEAD, the supplied delta.diff, and the newest commit's attribution trailers. No PR body, discussion, author risk assessment, or earlier review was read. Neither root AGENTS.md nor root CLAUDE.md exists in this checkout; the supplied review instructions were applied.
  • git diff --check origin/master...HEAD: exit 0. git diff --exit-code origin/master...HEAD -- src ext test Project.toml .github: exit 0. Final git status --porcelain=v1: empty.
  • uv run --locked python --version: Python 3.15.0. uv warned that --locked has no effect outside a project; this checkout has neither pyproject.toml nor uv.lock.
  • Julia reported 1.13.2; loading the declared @runic environment reported Runic 1.13.0. Preserved the supplied layered JULIA_DEPOT_PATH; set the requested TMPDIR, JULIA_NUM_THREADS=1, and JULIA_NUM_PRECOMPILE_TASKS=4.
  • uv run --locked python ../scratch/formatter_probes_slot1.py: exit 0. The scratch probe parsed the declaration and executed its exact argv with file arguments substituted for {files}. Observed exits: formatted file with spaces 0; unformatted file with spaces 1; mixed formatted/unformatted files 1; invalid Julia syntax 1; missing file 1; all 64 tracked Julia files 0. Invalid syntax and missing files emitted explicit errors. SHA-256 comparisons confirmed every tracked file stayed unchanged; the unformatted fixture also stayed unchanged.

Review produced by an AI agent using Codex CLI 0.164.0, model gpt-6.1-sol, in the session identified above. Independent verdict frozen without consulting another review or author rating.

What I did not verify

  • The external Factory dispatcher that consumes this declaration; the probe tested the declared executable and argument expansion directly.

  • The numerical package suite and downstream SciMLSensitivity SDE3 run. No package code or tests change. The supplied red CI remains unresolved by this review; MERGE is code-review advice and does not waive the CI gate.

  • A project-locked Python environment is unavailable; the required uv command ran successfully with the warning recorded above.

  • Verdict: MERGE

  • Round: 2

  • Blocking priorities: none

AUTHOR_GRADE: {"task":"ci","score":5,"difficulty":1,"rounds":2,"author_harness":"Codex CLI 0.164.0","author_model":"gpt-6-astra","note":"The five-line formatter declaration passed six direct command probes, including all 64 tracked Julia files, without changing tracked content. Factory-dispatcher integration and downstream CI were not verified."}

REVIEWER_GRADE: {"task": "ci", "score": 4, "rounds": 2, "reviewer_harness": "codex", "reviewer_model": "gpt-6.1-sol", "verdict": "false-positive", "note": "Thorough formatter validation: 111 hook tests and eight probes supported the same no-defect finding as the frozen review. The sole blocking P2 elevated an uninvestigated downstream CI failure into a CHANGES verdict without evidence connecting it to the five-line configuration addition. The frozen MERGE verdict preserves the separate unresolved CI gate; neither review established baseline behavior."}

Head link: dbcb5da


🤖 Posted by an AI agent — harness: codex-cli 0.164.0 · model: gpt-6.1-sol
Conversation: local session 01a1d25a-5609-7aa3-b6a4-9ce8f609a535 (no session URL)

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants