[theoheimel]

Brings pyAmpliCol, the generator behind the [amplicol] matrix-element backend, into MadGraph7 as pyamplicol/, the same way madspace was merged. It keeps its history and gets CI, documentation, a local installer and PyPI releases in lockstep with MadGraph7.

This PR covers the infrastructure only. The [amplicol] backend (feat-madspace-amplicol) can be rebased onto it afterwards and use the new bootstrap helpers.

!!! Important information on releases, please read !!!

The amplicol version is now coupled to the MG7 version and a MG7 release triggers a PyAmplicol release

• we should consider another release before merging this
• To be able to release after the merge, @ValentinHirschi needs to add a trusted publisher for the pyamplicol project: repository MadGraphTeam/MadGraph7, workflow release.yml, environment pypi
• because of the aligned versions, 1.0.0 and 1.0.1 are "burned" as pyamplicol releases already exist on PyPI. So I guess the MG7 release will be 1.1.0

History import

• The history of MadGraphTeam/pyamplicol main (up to 9404776f) is merged with every commit kept: 1085 commits, after git filter-repo pruned the 4 that became empty.
• Before the merge, the history was rewritten with git filter-repo:
  • the prepared-model packs (*.pyamplicol-model, 74 commits of 7–18 MB) were removed;
  • the PDFs were removed (they stay reachable through the upstream repository);
  • every path was moved under pyamplicol/.
• The packed history drops from about 89 MB to about 18 MB.
• The import is scripted in .github/scripts/import_pyamplicol.sh. The filter is deterministic, so later upstream commits can be imported by running it again and merging pyamplicol-import; commits that were already imported keep their SHAs.
• The upstream repository is unchanged. The pyamplicol remote has its push URL disabled.

Prepared models

• Git no longer tracks the packs or their metadata. Each MadGraph7 release attaches one archive, pyamplicol-prepared-models.tar.gz (about 2.4 MB). It holds the metadata plus each distinct bundle once; the x86_64 and aarch64 packs are identical.
• pyamplicol/tools/release/prepared_models_archive.py has four commands:
  • fetch downloads the archive from the matching release, or else the latest one;
  • unpack writes the files from a local archive;
  • pack creates the archive;
  • regenerate builds a bootstrap wheel and produces the packs on the current host.
• The installer and CI check the fetched packs against the source tree with the backend's own validation, and regenerate them when they are stale.
• Today's packs are already attached to v0.2.1.

Build changes for the monorepo

• Build backend
  • pyamplicol/ is treated as a Git subdirectory for the file inventory, the clean-revision check (only pyamplicol/ counts) and the release store.
  • The untracked packs are included in the build inputs.
  • A symlink to a file shared with the rest of MadGraph7 is copied by content (prepared for umami.h, see the follow-ups).
  • An opt-in local build mode for the installer allows a dirty tree or a release tarball, and uses a persistent build directory and a job count.
  • RUSTUP_HOME passes through to the build, and the build finds maturin when it is installed with pip install --target.
• Version: pyamplicol now follows MadGraph7's VERSION, currently 0.2.1. tools/release/sync_version.py writes it into Cargo and the release lock, and bin/create_release.py --check-only fails when they are out of sync.
• Source hash: pyamplicol/source_hash.py computes it; wheel builds record it as pyamplicol._source_hash.

Installation

• pyamplicol/install.py works like madspace/install.py:
  • --bin installs the PyPI wheel of this MadGraph version and is offered in release tarballs.
  • --source builds into pyamplicol/install and leaves the active environment alone.
  • Other options: -y, -j N, --system, --clean, --build-models.
• If rustup is missing, the installer puts it in pyamplicol/build/rust; ~/.cargo and ~/.rustup are left alone.
• Rebuilds reuse the Cargo build tree.
• After installing, it checks that Symbolica accepts pyAmpliCol's package-scoped library key (see below).
• The MadGraph prompt gets an install pyamplicol command with help and tab completion. The madspace help and completion also gain the -j, --system, --clean and --docs options they were missing.
• The bootstrap helpers in mg7/bootstrap.py, for the backend to use:
  • ensure_pyamplicol puts the install first on sys.path, ahead of the MadGraph root, where pyamplicol/ is the source directory.
  • check_pyamplicol_source_hash.
  • check_pyamplicol_license.
• madspace/install.py now ignores the pyamplicol wheels listed in input/.release.

Symbolica library key

pyamplicol/__init__.py registers an OEM key that is scoped to the package name pyamplicol. The checks run without SYMBOLICA_LICENSE and confirm that the imported package is the installed one:

• the installer runs the check after every install;
• the CI installer job runs it from the MadGraph root, the way the worker sees it;
• the release smoke test runs it through install.py --bin.

CI also warns 90 days before the key expires, which is on 2028-01-01.

CI

• pyamplicol.yml ports the upstream tests.yml and release-artifacts.yml; every job runs inside pyamplicol/.
  • A changes job skips the heavy jobs when nothing pyamplicol depends on has changed. It is a job rather than a paths: filter, so every commit still has a green run for the release gate.
  • An installer job builds pyamplicol through install.py and checks the library key.
• pyamplicol-eager-portability.yml ports the check that packs produced on one architecture work on the others.
• New actions:
  • pyamplicol_prepared_models: fetch and validate, or else regenerate;
  • build_pyamplicol / install_pyamplicol: like the madspace pair.
• release.yml:
  • builds and validates the pyamplicol sdist and the three abi3 wheels;
  • publishes them in publish_pypi_pyamplicol, which reuses the pypi environment;
  • lists the wheels in input/.release;
  • attaches the prepared-model archive to the release;
  • adds pyamplicol.yml to the green-CI gate.
• unittest.yml runs the new test_pyamplicol_install.
• The upstream workflows under pyamplicol/.github would never run in this repository, so they are removed, along with the test that pinned their contents. The remaining workflow tests now read the MadGraph7 workflows.

Documentation

• "pyAmpliCol" is a new section of the MadGraph7 docs. The pages stay in pyamplicol/docs, because pyamplicol's tests and sdist use them, and are copied into the Sphinx tree at build time.
• MyST renders the Markdown (myst-parser==5.0.0). Toctrees in the parent pages replace the Jekyll hierarchy.
• The installation page describes the MadGraph7 installer, and the packaging and release pages describe the archive and the release process.
• The build gives no warnings for the pyamplicol pages.

Verification

• python pyamplicol/install.py --source -y -j 8 builds into pyamplicol/install. An incremental rebuild takes about 50 s. The licence check passes, and the installed source hash matches.
• --build-models regenerates the packs locally (bootstrap wheel, produce, build) in 1 min 45 s, and they pass the backend validation.
• A source install from a git archive snapshot, the way a release tarball builds it, also works: a cold build took 7 min 48 s, the packs were fetched from v0.2.1, the licence check passed, and the installed source hash matches.
• ensure_pyamplicol, check_pyamplicol_license and check_pyamplicol_source_hash pass from the MadGraph root.
• The release sdist builds from the checkout and passes pyamplicol's own audit. It carries the MadGraph7 commit as its source revision, plus the packs and source_hash.py.
• pyamplicol tests:
  • the source-contract suite: 359 passed;
  • test_build_backend, packaging, dependency gate, release and examples tests: 399 passed, 9 skipped (just is not installed);
  • ruff check is clean on the CI lint scope.
• MadGraph tests: test_pyamplicol_install (14), test_madspace_install (28) and test_mg7_launch (91) pass.
• make -C docs html succeeds.
• Not run locally: the full native runtime suite (release-runtime), the macOS and manylinux wheel builds, and the release workflow itself. They run in CI: the first on this PR, the wheel builds via workflow_dispatch with full_matrix.

Follow-ups

• UMAMI header symlink. Replace pyamplicol/src/pyamplicol/assets/api_templates/umami/umami.h with a symlink to madspace/include/madspace/umami.h, as for madmatrix. It cannot be done yet: pyamplicol uses UMAMI 1.2, but this branch's madspace header is 1.0, and 1.2 only exists on feat-madspace-amplicol. The build backend already copies the symlink by content.
• Rebase feat-madspace-amplicol onto this branch and call ensure_pyamplicol and the checks.
• Gridpacks: decide whether [amplicol] gridpacks need the pyamplicol runtime.
• Archive MadGraphTeam/pyamplicol once all development has moved here.

[oliviermattelaer]

Hi,

This is literally amazing.

One nit picking details, is that this PR likely include a lot of files that we do not want to be part of the release ...

• Do we need to have a second models directory with plenty of model not shipped in MG.
• Do we need to have a full process list (like the all examples directory)
• Do we want to keep all the .md that help the AI to generate this?
• Do we want to keep all the latex file (of the paper?)?

[ValentinHirschi]

Hi,

This is literally amazing.

One nit picking details, is that this PR likely include a lot of files that we do not want to be part of the release ...

• Do we need to have a second models directory with plenty of model not shipped in MG.
• Do we need to have a full process list (like the all examples directory)
• Do we want to keep all the .md that help the AI to generate this?
• Do we want to keep all the latex file (of the paper?)?

Hi, yes it's exciting!

I agree with Theo that a monorepo for MG7 will make development far easier, especially for agentic coding.

On the other hand, I think it's great to still have pyAmpliCol be usable standalone outside of MadGraph (that was one of my important design goal). It can be very useful to other generators, and parton showers for MEC, etc...

I think part of my motivation there is that with MadLoop I felt like it was a mistake that it was so tightly integrated within MadGraph, so that no one understood they could easily use it standalone as well, and thus OpenLoops became the default.

I would like to avoid repeating that scenario here and really keeping pyAmpliCol as independent "tree-level ME provider" to MadGraph, which is now made simple thanks to the much more advanced APIs we can build nowadays thanks to agents.

So for example, it is important that pyamplicol remains a pypi installable wheel that can be used independently of madgraph, with its own CLI, python library, SDKs etc...

So some of the files you mentioned are precisely there for that standalone-requirement reason (i.e. example files, some model files). Some other files (like left-over markdowns, old benchmark PDFS) are indeed stale and I agree that they should be removed.

So moving forward and as far as MadGraph7 release go, I think the way to go about it is to have the release CI workflow trim down pyAmpliCol to what's needed by MG7 alone, and remove the rest (e.g. examples and models); but keep these files in the repo so that the release CI workflow for pyAmpliCol itself can ship them properly as intended when releasting the pyAmpliCol pypi wheels.