chore: ci optimisations - #318
Conversation
Signed-off-by: GHkrishna <krishna@parity.io>
Signed-off-by: GHkrishna <krishna@parity.io>
Signed-off-by: GHkrishna <krishna@parity.io>
Signed-off-by: GHkrishna <krishna@parity.io>
CI Summary
Labelsother, ci |
re-gius
left a comment
There was a problem hiding this comment.
The Foundry cache saves all of cache/, and Forge also stores failed fuzz and invariant examples there (cache/fuzz/, cache/invariant/, cache/test-failures). The next forge test replays those examples before starting a new run, so a saved failure keeps failing CI after the bug is fixed. foundry.toml already leaves corpus persistence off for this reason.
The gas report is how one of those examples gets uploaded. forge test is piped to tee with no pipefail, so a failing fuzz test still leaves the job green, and actions/cache saves on success. The test workflow restores any cache named foundry-build-Linux-…, which also matches the gas-report keys.
Delete those three directories before forge test, and again as the last step of each job that saves this cache, so a replay is neither executed nor uploaded.
Signed-off-by: GHkrishna <krishna@parity.io>
re-gius
left a comment
There was a problem hiding this comment.
Thanks for the fix, now I have a final comment.
Coverage and the test jobs share one cache key, foundry-build--. Coverage ends with forge coverage --ir-minimum, which recompiles in a lighter mode and overwrites cache/ and out/. GitHub keeps the first upload for a key. If the test job fails and does not save, the green coverage job stores that lighter build. The next test run restores it, recompiles because the compiler settings differ, and cannot replace the stored copy.
Give coverage its own prefix so only the next coverage run restores it
Signed-off-by: GHkrishna <krishna@parity.io>
|
Ohh nice catch, missed it. Though I've now changed the key for coverage. |
re-gius
left a comment
There was a problem hiding this comment.
Approving with one optimization comment left
|
Notice that some of the CI checks you suggest marking as required, like "Run Solidity Tests / Contract Tests (Unit + Fuzz)", do not run for every PR, so we should either run them every time or we cannot required them. We can do this in a follow up PR |
Signed-off-by: GHkrishna <krishna@parity.io>
Yess |
Description
This PR fixes three problems with our GitHub Actions setup.
First, when you pushed several commits to an open PR, some checks kept running for every old commit instead of stopping and only running for the latest one. This was happening because four workflows (Format and Lint, PR Title and Labels, Release Metadata, and the Genesis Extractor Test) were missing a concurrency setting that every other workflow already had. This PR adds that setting so a new push now cancels the still running check from the previous commit
Second, our slowest checks (the test suite, the gas report, and the coverage report) were doing a full cold Foundry build every single time, even when nothing in the contracts had changed. This PR adds a cache for Foundry's build output so unchanged code does not need to be recompiled from scratch on every run.
While setting up that cache, we found a timing bug: the cache key was being calculated before bun install runs, but bun install is what actually re-downloads and patches our OpenZeppelin dependencies. So the cache key was based on the wrong version of those files. This is fixed by moving the cache step to run after bun install, so the key always reflects what actually gets built.
Lastly, we noticed the Gas Report workflow was starting and stopping a Docker container (the local RPC node used for fork tests) on every run, even though the gas report command explicitly skips fork tests. That container was never being used, so it has been removed. This saves time on every single gas report run.
Type
Scope
Related Issues
This came out of a general review of our CI setup.
Fixes
Checklist
Code
forge buildpassesforge testpassesTesting
Security
selfdestructordelegatecallDocumentation
Breaking Changes
Breaking changes:
No
How to test
This PR changes workflow behavior, so the best way to test it is to run it.
Notes
While reviewing this, a couple of smaller ideas came up that are intentionally not part of this PR, flagging them for later:
None of our workflows set a timeout-minutes limit on any job, so a stuck run could sit there for hours (GitHub's default) and hold up its concurrency group. Worth adding reasonable limits later.
The Foundry cache key does not account for a future bump of the pinned Foundry version in .github/actions/setup-foundry/action.yml. Worth adding that file's hash to the cache key whenever that version is next bumped.
Below checks should be marked required in the repo setting:
Following not req: