Repository navigation
Upload test runs, build_report on runs (phase 3) - #122
Merged
Merged
Conversation
A case slug never starts with a dot, and attachment and part names keep ASCII [A-Za-z0-9._-] only (str.isalnum() let other letters through). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
--embeddedci-upload (ini embeddedci_upload, EMBEDDEDCI_UPLOAD=1) uploads the recorded run at session end: POST /api/runs, then junit.xml, the traces of failed and error cases with their attachments, and logs/. Credentials: BENCHPOD_API_KEY, else the benchpod login session, else GitHub Actions OIDC. A failed upload is a warning; under benchpod ci the runner uploads instead. In GitHub Actions a session whose tests use build_report records its run and uploads it with inputs.dut_firmware.build_id set to the build the fixture created, using the reporter's credentials. The build status call stays for older servers. BuildReporter gains api_base and session_token. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A fake runs API: only failed traces and their attachments go up, the upload order, a refused upload, the plugin option and env, a failed upload as a warning, no upload under benchpod ci, build_report linking the run to its build, and trace/attachment names against the server's rule. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
One pytest step over e2e/ and examples/ (the make e2e tiers), dut: none: the DUT tier skips without --benchpod-firmware. Mentioned in CONTRIBUTING and the e2e README. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
BP0026 on a lease joined through BENCHPOD_LEASE asks the run to stop; the plugin exits the session after the test that saw it (exit code 2, "run stopped by <by>") and still writes parts/<step>.json with the cases so far and health lease_lost/valid/reasons. The SDK's own lease keeps going on without it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
wardviaene
force-pushed
the
ci-runs-upload
branch
from
November 19, 2026 05:26
7349a59 to
41eb1de
Compare
Merged
wardviaene
added a commit
that referenced
this pull request
Nov 19, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
--embeddedci-upload(also theembeddedci_uploadini option andEMBEDDEDCI_UPLOAD=1) uploads the recorded run at session end:POST /api/runswithrun.json, thenPOST /api/runs/{id}/files?name=forjunit.xml, the traces of failed and error cases with the attachments they point to, andlogs/. The uploadedrun.jsonsays so (keep.traces: failed,trace: ""for cases kept local); the folder on disk is unchanged. Without--embeddedci-run-dirit records into./embeddedci-runs/first. Credentials:BENCHPOD_API_KEY, else thebenchpod loginsession, else GitHub Actions OIDC (mint_session_token). The run URL is printed; a failed upload is a warning and never changes the result. Underbenchpod ci(EMBEDDEDCI_RUN_STEP) the plugin never uploads.embeddedci.benchpod.runs.upload.upload_run(folder, api_base=None, token=None, build_id=None)uploads a folder later.build_reporton runs: in GitHub Actions, a session whose collected tests usebuild_reportstarts recording by itself; at the end the run is uploaded withinputs.dut_firmware.build_idset to the build the fixture created (the first one), with the reporter's own session token. The/api/cloud/builds/{id}/statuscall stays, so older servers keep the pass/fail.BuildReporter.api_baseandBuildReporter.session_tokenare new (additive).[A-Za-z0-9._-]only, so every file passes the server's name rule.packages/embeddedci/tests/benchpod-ci.yaml: the SDK's hardware tests (e2e/andexamples/, themake e2etiers) as one pytest step,dut: none(the DUT tier skips without--benchpod-firmware). Mentioned in CONTRIBUTING and the e2e README.Tests
tests/test_runs_upload.pyagainst an in-process fake runs API: only failed traces and their attachments go up, upload order and auth header, a refused upload, the option and the env var, a failed upload is a warning, no upload underbenchpod ci,build_reportlinking the run to its build (localrun.jsontoo), nothing recorded outside GitHub Actions, trace/attachment names (long, unicode, leading dot) against the server's rule, and the repo'sbenchpod-ci.yaml. SDK (1089 passed, bundled-copy check against firmware main), MCP and scripts suites pass.🤖 Generated with Claude Code