Repository navigation
Add workload through leveraging OCI images to run on CI - #438
henrybear327 wants to merge 2 commits into
Conversation
henrybear327
assigned
Max042004,
henrybear327,
jserv and
sunxiaoguang and unassigned
henrybear327
![cubic-dev-ai[bot]](https://avatars.githubusercontent.com/in/1082092?s=60&v=4){kind=small}
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
scripts/ci/oci-runtime.sh runs elfuse on a real runtime per key: python, node, go, jvm, c, and redis, each driving a workload from scripts/ci/workloads and requiring its sentinel. The node and redis lanes serve from one guest to the host or to a second guest. The oci-runtime job builds crane and umoci from source as oci-smoke does, then runs each key on the self-hosted macOS runner, with a store per key and runner, since the runners on one host share $HOME. Like oci-smoke, it pulls the library images through mirror.gcr.io. The helpers oci-smoke.sh defined move to scripts/ci/oci-lib.sh, which both lanes source. wait_for there probes once more than the oci-smoke.sh copy did, so it fails after the full timeout instead of half a second short of it.
docs/oci-images.md lists what each oci-runtime key covers, and docs/usage.md gives the local command.
0ad18e6 to
e3e97d3
Compare
| set -e | ||
|
|
||
| # Async preemption can deliver a signal while the vCPU is in the EL1 MRS path. | ||
| export GOCACHE=/tmp/elfuse-go-cache GOTOOLCHAIN=local GODEBUG=asyncpreemptoff=1 |
There was a problem hiding this comment.
GODEBUG=asyncpreemptoff=1 hides an elfuse bug instead of exercising it: the comment admits that a signal delivered while the vCPU is in the EL1 MRS path breaks the guest, and nothing else tracks that. Every Go guest runs with async preemption on by default, so this lane stays green while that failure ships. As a follow-up, please file an issue with a repro and link it here, so the override can be dropped once it is fixed.
| exit 1 | ||
| fi | ||
|
|
||
| remaining=$(gofmt -l . | wc -l | tr -d ' ') |
There was a problem hiding this comment.
This runs under POSIX sh, which has no pipefail, so set -e only sees the exit status of tr. If gofmt -l crashes before printing anything, remaining is 0 and the lane prints its success marker. The listed count above has the same gap. Capture the list first so a gofmt failure stops the script:
| remaining=$(gofmt -l . | wc -l | tr -d ' ') | |
| gofmt -l . > "$d/list" | |
| remaining=$(wc -l < "$d/list" | tr -d ' ') |
|
|
||
| local i=0 reqs=100 body | ||
| while [ "$i" -lt "$reqs" ]; do | ||
| body="$(curl -fsS "http://127.0.0.1:$port/")" \ |
There was a problem hiding this comment.
curl has no deadline, so a guest that accepts the connection and then stops responding holds the self-hosted runner until the job timeout. The /quit request below can hang the same way before await_server_exit gets to start its 10 second check. Add --connect-timeout 5 --max-time 10 to both calls.
| # Redis treats port 0 as disabled. | ||
| local port="" try=0 | ||
| while [ "$try" -lt 10 ]; do | ||
| port=$((20000 + RANDOM % 20000)) |
There was a problem hiding this comment.
There is a race between the nc -z probe and redis binding the port. Runners on this host share loopback, so another lane can take the port in between, and that shows up as "server guest exited before readiness" with nothing pointing at the real cause. As a follow-up, retry start_server with a fresh port when the guest exits before it is ready.
| [ -n "$port" ] || fail "no free loopback port for redis after 10 probes" | ||
|
|
||
| # Make BGSAVE the only background save. | ||
| start_server 'Ready to accept connections' \ |
There was a problem hiding this comment.
The clean at the start of the lane leaves the store empty, so this run pulls, unpacks and seals redis:7-alpine inside the 60 second readiness window that start_server allows. A slow pull from the mirror then fails as a server timeout. There is a second effect: during the pull, $guest is still the Python wrapper and not elfuse, so on_exit kills only the wrapper and leaves its crane, umoci or hdiutil child running against the shared store. Pull the image first, so the window measures only server startup and $guest is always elfuse:
| start_server 'Ready to accept connections' \ | |
| "$BIN" pull "$MIRROR/redis:7-alpine" > /dev/null | |
| start_server 'Ready to accept connections' \ |
| exit 2 | ||
| } | ||
|
|
||
| "$BIN" clean > /dev/null |
There was a problem hiding this comment.
clean deletes the images, refs, run and tmp directories, so the per-key, per-runner store set in build.yml never keeps anything. Every push downloads and seals gcc:14, eclipse-temurin:21, golang:1.23-alpine and the rest again, on six parallel jobs. That costs time and makes every run depend on the registry. oci-smoke.sh already tests clean, so this lane could drop both calls (here and at the end) and keep the cache. Pinning the images by digest would also keep the image under test fixed between runs.
|
|
||
|
|
||
| def main(): | ||
| # Plain rootfs runs can retain files from an earlier invocation. |
There was a problem hiding this comment.
This comment is not accurate under elfuse-oci run. cmd_run attaches every run with its own -shadow under run/<pid>, and oci-smoke.sh asserts that each run gets its own shadow, so nothing carries over from an earlier invocation. Either drop the comment or say the cleanup is only there for runs outside elfuse-oci.
4 participants
Based on the workload used in #224 and the newly merged OCI image capability, python, node, go, jvm, c, and redis images are pulled and run on CI for all PR, making sure that elfuse is able to execute common workload normally after changes.
Summary by cubic
Adds
oci-runtimeCI lanes that run elfuse against real runtimes (python, node, go, jvm, c, redis) on OCI images, so common workloads are exercised on every PR and push. The smoke helper functions move to a sharedoci-lib.sh, and each lane is documented indocs/oci-images.mdanddocs/usage.md.The python lane covers threaded SQLite writers, filesystem walks, subprocesses, and Unix sockets longer than the host's
sun_path. The node and redis lanes expose server guests to the host or a second guest, and the go lane setsGODEBUG=asyncpreemptoff=1so async preemption cannot inject signals into the vCPU path.Written for commit e3e97d3. Summary will update on new commits.