Repository navigation
build(nix): publish LLVM/MLIR 23 via cachix instead of per-user source builds - #45
Merged
Merged
Conversation
…e builds nixpkgs has no llvmPackages_23, so this flake builds two LLVM trees from source (~1h, llvm23 1.2GiB + mlir23 774MiB). Every nix run user would pay that, so: - flake.nix nixConfig: add comamoca-lython.cachix.org as an extra substituter with its trusted public key, so first-time users get a one-time accept prompt and pull the prebuilt closure. - .github/workflows/nix-cache.yml: rebuild and push llvm23/mlir23/ lython on pushes touching the flake or compiler sources (CACHIX_AUTH_TOKEN secret), keeping the cache fresh. - .gitignore: result*, .direnv/.
|
Important Review skippedAuto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the ⚙️ Run configuration
You can disable this status message by setting the Use the checkbox below for a quick retry:
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Flake inputs are fetched lazily, but flake-parts' mkFlake forces its `imports`, so evaluating packages.default dragged flake-parts, nixpkgs-lib, systems, treefmt-nix and git-hooks-nix into every end-user fetch. Hand-rolled outputs keep the tooling inputs referenced only from checks/formatter, which `nix run`/`nix build` never touch; the nixpkgs follows collapse the lock from three nixpkgs revisions to one (11 nodes -> 6). `nix run` also could not execute at all: it looked for bin/lython (the pname) while CMake installs bin/lyc, so meta.mainProgram = "lyc" now points it at the real binary. Verified with forged-narHash probes: packages.default succeeds with all four dev inputs poisoned, while checks.* still fail (they do fetch them). nix flake check, nix develop -c true and `nix run . -- parse examples/hello.py` all pass; nix fmt fixed the trailing-newline diff that kept treefmt/pre-commit checks red.
nixpkgs-unstable now ships LLVM/MLIR 23 (previously only 21), so the
custom nix/llvm23.nix and nix/mlir23.nix (which built the upstream
release tarball to get a shared MLIR CMake target) are redundant.
- flake.nix: drop the llvm23Overlay and the llvm23/mlir23 package
outputs; pull pkgs.llvmPackages_23.{llvm,mlir,tblgen} directly.
- build-lython.nix: take llvm / mlir / tblgen as arguments; pass
LLVM_TABLEGEN_EXE and MLIR_TABLEGEN_EXE because nixpkgs's mlir
derivation does not install mlir-tblgen into its `out`.
- delete nix/llvm23.nix and nix/mlir23.nix (no longer referenced).
- nix-cache.yml: drop .#llvm23 and .#mlir23 from the Cachix push set;
nixpkgs already provides them as binary cache hits.
cmake's find_program for mlir-tblgen (Lython's CMakeLists.txt:283) was failing inside `nix develop` because the nixpkgs mlir derivation does not install mlir-tblgen into its `out`. The build-lython.nix side already worked around this by passing the tablegen path explicitly; the devShell side was missing the same fix. Lython's devShell workflow (`cmake -B build && cmake --build build` inside `nix develop`) now works again, which is required to build the test suite (`LYTHON_BUILD_TESTS=ON`).
`nix develop` users building lython with `cmake -B build-test` (or any other suffixed build dir) would otherwise accidentally stage the hundreds of generated files the next time they ran `git add -A`. The existing `build/` entry only matched an exact name; `build-*/` catches `build-test`, `build-debug`, `build-asan`, etc.
LLVM/MLIR is now sourced from nixpkgs's llvmPackages_23, which is already on the public binary cache. The Cachix push workflow no longer needs to hold a self-built LLVM/MLIR (those derivations are gone), and lython's own source build is cheap enough that a private cache adds little value. - .github/workflows/nix-cache.yml: deleted (Cachix push no longer needed) - flake.nix: drop git-hooks-nix input and the pre-commit check (the URL is github:cachix/git-hooks.nix, the only remaining cachix reference; the pre-commit hooks depended on it) - flake.lock: regenerated without git-hooks-nix and its transitive inputs (flake-compat, gitignore)
Comamoca
force-pushed
the
feat/nix-flake-cachix
branch
from
October 12, 2026 06:52
494b310 to
50f6ee3
Compare
t3tra-dev
added a commit
that referenced
this pull request
Oct 17, 2026
`nix run`, `nix develop` and `nix flake check` for aarch64-darwin, aarch64-linux and x86_64-linux, against nixpkgs' llvmPackages_23. Hydra builds it and cache.nixos.org serves it for all three, so no LLVM is built on a user's machine and no other binary cache is trusted. lyc links it as shared libraries out of the store, as it links Homebrew's or apt's; nothing is bundled. Measured: the package builds in about 2 minutes on either platform, and the ctest suite (bench aside) passes in the sandbox -- 839 of 839 on aarch64-darwin and aarch64-linux. Taken in part from PR #45 (Comamoca): the flake layout, the systems, the .envrc and .gitignore entries, `mainProgram`, tests off in the package. Not taken: building LLVM and MLIR from source with a personal Cachix in nixConfig -- the lock predated LLVM 23 in nixpkgs, and that is now cached upstream -- nor treefmt/git-hooks inputs, nor AGENTS.md. Differences from that PR, and why: - llvmPackages_23.stdenv, so Linux builds with the clang that matches the LLVM, as every other build of this tree does. - lyc is wrapped with that clang and lld on PATH: AOT and WASI links run them, and a wrapped clang is what finds this system's libc. - googletest comes from nixpkgs' source through FETCHCONTENT_SOURCE_DIR_GOOGLETEST, so the suite runs in the sandbox as `checks`. - x86_64-darwin is left out: nixpkgs no longer builds it. Building against it found four defects that Homebrew and apt hide, each fixed in its own commit before this one (GCC's pattern names, ld64's local personality, aarch64's type-table width, the macOS version), and .github/workflows/nix.yml keeps all three systems building and tested.
t3tra-dev
added a commit
that referenced
this pull request
Oct 17, 2026
Owner
|
Thank you for the updates.
This is all thanks to you bringing in the Nix pipeline. Thank you very much. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds Nix support to lython.
https://nixos.org/
Nix is a purely functional package manager characterized by its ability to perform idempotent builds.
This PR adds flake.nix, a file used to build lython using Nix, and the lock file flake.lock.
Motivation
lython is built with C++ and depends on LLVM.
These two have a relatively high barrier to environment setup, and it also takes time to set up a development environment to run it.
Supporting Nix lowers the difficulty of environment setup, making it easier for users interested in lython to run it easily and participate in its development.
Changes
Added flake.nix, which defines the build and development environment for lython.
Additionally, a sub-flake structure is used to prevent inputs for development environment use from being downloaded when running nix run.
Special Notes
LLVM23 has not yet been released on nixpkgs, and as of now (2026/10/9), no build cache exists on hydra.
Therefore, running this flake will start building lython locally, including a build of LLVM23.
Since this is an extremely heavy process, I have published the cache generated when I built lython using Nix on cachix.
Since it is not a build from nixpkgs, use it at your own risk, but if you want to reduce the build load even a little or build faster, please feel free to use it together.
https://app.cachix.org/cache/comamoca-lython