Repository navigation
Conversation
|
Merge-order note: this and #118 are siblings off #114, and both insert a block into |
0dd98b0 to
af2010b
Compare
|
Rebased onto #116 → #114. #113 is closed and this branch carried its two commits; #116 repairs the same two things on master, so they are dropped here. Stack is now #116 → #114 → this. 39/39 locally. The |
srpatcha
left a comment
There was a problem hiding this comment.
Review — eos#115 "fix(pkg): refuse a source download that cannot be verified"
head: af2010b author: Kartikey1306 ci: pass
Verdict: The tarball half of the fix is correct, well-tested and reproduces as
claimed — 5/5 offline, and the guard fires before the network is touched. But the
identical defect is still live on the git path: eos_fetch_source() accepts an
expected_hash, ignores it entirely for git URLs, and returns EOS_OK. I confirmed
that with a wrong digest. Because is_git_url() is a bare .git substring test, that
path also captures URLs the guard was meant to cover.
Reviewing commit af2010b only, per the PR body. 753cb10 is #114 and d58bac4 is
#116; both are reviewed separately in this run.
Findings
| # | Severity | File:line | Finding | Recommended fix |
|---|---|---|---|---|
| 1 | High | pkg/src/fetch.c:219,235-236 |
The guard is if (is_tarball(url) && ...), so a git URL with no digest passes it, and a git URL with a digest also passes it — then fetch_git() (fetch.c:143-153) never looks at expected_hash. A caller that supplies a digest for a git source gets EOS_OK and the content, with nothing compared. This is verbatim the property the PR body identifies as the defect: "the caller cannot tell the two apart, because both return EOS_OK." |
Either verify the git path (clone, then check git rev-parse HEAD against expected_hash when one is given) or refuse a digest you will not honour: if (is_git_url(url) && expected_hash && expected_hash[0]) return EOS_ERR_INVALID; — silently discarding it is the worse of the two. |
| 2 | Medium | pkg/src/fetch.c:124-128 |
is_git_url() is strstr(url, ".git") != NULL, an unanchored substring test. It matches any host under *.github.io — including this organisation's own embeddedos-org.github.io. Because line 235 tests it before line 237's is_tarball(), a correctly-digested tarball on GitHub Pages is routed to git clone and can never be fetched. |
Anchor the test on the URL path suffix and scheme: match a trailing .git, git://, git@ or ssh://, not .git anywhere in the string. |
| 3 | Medium | pkg/src/fetch.c:219-224 vs fetch.c:130-134 |
The guard's rationale — "Git URLs are exempt: a clone carries its own object hashes" — does not support the exemption. fetch_git() runs git clone --depth 1 "$url" with no pinned commit, tag or signature. Git's object hashes attest the internal integrity of whatever the remote chose to serve; they do not attest that it is the intended revision. The git path is an unpinned fetch, not a verified one. |
State the real position in the comment, and pin the revision. expected_hash is the natural place to carry the commit id. |
| 4 | Low | tests/test_pkg_fetch.c:52 |
PASS("every archive form this fetcher extracts is covered") and the PR body's matching table row overstate coverage. The suite exercises .tar.gz, .tar.xz and .tgz; eos_fetch_source() also extracts .tar.bz2 and .zip (fetch.c:266-271). The guard does cover both, because is_tarball() matches .tar and .zip — the code is right, the claim is wider than the test. |
Add .tar.bz2 and .zip cases, or narrow the assertion text to the three forms actually asserted. |
| 5 | Low | pkg/src/fetch.c:288-291 |
The trailing else branch (EOS_WARN("Assuming tarball for URL: %s") → fetch_tarball()) is now reachable only for URLs where is_tarball() is false, and fetch_tarball() extracts only when the local basename carries an archive suffix — which is a substring of the URL, so is_tarball() would have been true. The branch can therefore download a file and then return EOS_ERR_INVALID, but never succeed. It also duplicates the inline download/extract block and ignores expected_hash. |
Delete fetch_tarball() and replace the else with EOS_ERR_INVALID before any download, so an unrecognised URL is refused rather than fetched-then-rejected. |
Findings 1 and 2, reproduced against this PR's head. libeos_pkg.a built from
refs/pull/115/head, driven directly, with a local git repo as the remote so the test
is offline:
$ ./drv "$FAKE_REMOTE/.git" /tmp/out/a \
"0000000000000000000000000000000000000000000000000000000000000000"
Cloning into '/tmp/out/a'... done.
RESULT=0 (EOS_OK=0 EOS_ERR_CHECKSUM=9 EOS_ERR_FETCH=8)
$ ls /tmp/out/a
f.txt
An all-zero digest — the same all-zero-key shape #99/#101 hardened eos_pkg against —
returns EOS_OK and delivers the content.
$ ./drv "https://embeddedos-org.github.io/dist/src.tar.gz" /tmp/out/b "abc123"
[INFO] Fetching (git): https://embeddedos-org.github.io/dist/src.tar.gz
fatal: repository '.../src.tar.gz/' not found
RESULT=8
The part that does work, verified:
$ cmake -B build/host -DEOS_BUILD_TESTS=ON && cmake --build build/host --target test_pkg_fetch
[100%] Built target test_pkg_fetch
$ ./build/host/tests/test_pkg_fetch
[PASS] a tarball with no expected hash is refused
[PASS] an empty expected hash is refused, not treated as absent
[PASS] every archive form this fetcher extracts is covered
[PASS] an unsafe URL still fails as unsafe, not as a checksum error
[PASS] no URL is still a no-op rather than an error
5/5 tests passed
No network access occurred — every case returns before the wget/curl call, which is
the property the suite was designed around and it holds.
Severity note per .ai/reviewer.md: finding 1 is High, not Critical. eos_fetch_source()
fetches build-time source, not a signed package or a boot image, and the eSec/eBoot
verification paths are not involved. It is not rounded below High because a supplied
digest is discarded without a word, which is the failure mode the PR exists to remove.
Architecture conformance
Conforms. Master design §21 places eos in Tier 1 - Foundation, and pkg/ is
package-fetch machinery inside the owning repo rather than a new repository — correct
under §21.1, which requires a stable interface, independent lifecycle, named maintainers
and multiple consumers before a subsystem splits out. No include, link or manifest edge
points up a tier; pkg/src/fetch.c reaches only eos/error.h and eos/package.h, and
the new test target links eos_pkg eos_core, both Tier 1. §5.1 holds.
§14.1 is the section the findings bear on. The guard moves the tarball path from "no
digest means no verification" toward fail-closed, which is the direction §14.1 asks for.
Findings 1 and 3 are where it stops short: the git path stays fail-open, and --depth 1
with no ref is not a verified fetch under §28.1's Security row ("threat model,
tests/review and implementation status").
The PR body also records two structural observations that are worth a maintainer's
attention and that I confirmed independently: services/pkg/eos_pkg.c (the ed25519
verification path) is in no CMakeLists.txt, while the eos_pkg library that is
actually built is pkg/src/package.c and pkg/src/fetch.c — my build log above shows
exactly those two objects and nothing else. A component whose verification code is not
compiled cannot claim §28 Implemented. That belongs in its own issue, not here.
Proposed changes
The tarball guard should land as it is. Findings 1-3 are one follow-up:
--- a/pkg/src/fetch.c
+++ b/pkg/src/fetch.c
@@ static int is_git_url(const char *url) {
- return (strstr(url, ".git") != NULL ||
- strncmp(url, "git://", 6) == 0 ||
- strncmp(url, "git@", 4) == 0);
+ size_t n = strlen(url);
+ if (n >= 4 && strcmp(url + n - 4, ".git") == 0) return 1;
+ if (n >= 5 && strcmp(url + n - 5, ".git/") == 0) return 1;
+ return (strncmp(url, "git://", 6) == 0 ||
+ strncmp(url, "git@", 4) == 0 ||
+ strncmp(url, "ssh://", 6) == 0);
}
@@ EosResult eos_fetch_source(...)
if (is_tarball(url) && !(expected_hash && expected_hash[0])) {
...
}
+ /* A digest we will not check is worse than no digest: the caller believes
+ * the fetch was verified. fetch_git() clones an unpinned HEAD, so there is
+ * nothing here to compare a SHA-256 against. */
+ if (is_git_url(url) && expected_hash && expected_hash[0]) {
+ EOS_ERROR("Refusing to fetch %s: a git source cannot be verified "
+ "against a SHA-256. Pin a commit instead.", url);
+ return EOS_ERR_INVALID;
+ }Order: finding 2 first — anchoring is_git_url() changes which URLs reach which branch,
so it has to settle before the git guard is written against it. Then finding 1. Extend
tests/test_pkg_fetch.c with a *.github.io tarball case and a git-URL-with-digest
case; both fail against the current head, which is the evidence to quote.
Findings 4 and 5 are cleanups and can ride along.
Merge-order note from the existing comments still stands and I did not re-check it: this
and #118 add blocks at the same tests/CMakeLists.txt anchor, so the second to land
conflicts there.
Not checked
- I did not exercise the tarball download-and-extract path end to end. Every test case,
mine and the PR's, returns before the network call. Whetherwget/curl→ SHA-256 →
tarworks against a real archive is untested here. - I did not audit
is_url_safe(). It rejects;|><$()'but not backslash, newline or&, and bothfetch_git()and the inline path build shell command strings passed tosystem()`. That is pre-existing and outside this commit, but it is the reason a URL
classification bug is worth more than cosmetic attention. - Finding 5 rests on reading
is_tarball()againstfetch_tarball()'s suffix tests. I
did not construct a URL that reaches theelsebranch and extracts, because I believe
none exists; I did not prove that exhaustively. - I did not run the full 39-test suite the author reports, only
test_pkg_fetch. The
39/39claim in the second comment is unverified by me. - CI is green at this head (
Build & Test (Linux x86_64),Host Tests (x86_64),
Static Analysis,CodeQL,QEMU Boot Testall pass). I read the check list, not
the logs. - The
services/pkg/eos_pkg.corphan and the third private SHA-256 implementation in
pkg/src/fetch.care recorded, not assessed. Neither is changed by this commit.
Automated architecture review of af2010b0b813 — scheduled, model claude-opus-5, checked against the EmbeddedOS Master Design v2.0. Advisory only: this reviewer never approves, requests changes, or merges. Reply here to discuss or push back — a wrong finding is a bug worth reporting.
Answers the review on embeddedos-org#115. The tarball half of this PR was correct; the identical defect was still live one branch over. Finding 1 (High) -- the guard read `if (is_tarball(url) && ...)`, so a git URL passed it with a digest and without one, and fetch_git() then never looked at expected_hash. A caller that pinned a revision got EOS_OK and the content with nothing compared -- verbatim the property this PR's own body names as the defect: "the caller cannot tell the two apart, because both return EOS_OK." fetch_git() now takes the pin and compares it against the commit the clone actually landed on (`git -C <dest> rev-parse HEAD`), failing EOS_ERR_CHECKSUM on a mismatch or if HEAD cannot be read. Finding 3 (Medium) -- the exemption's stated reason was "a clone carries its own object hashes". That is true and does not support the exemption: object hashes prove the transfer was not corrupted, not that the content is the content that was reviewed. `git clone --depth 1` takes whatever the default branch points at when it runs, so an unpinned git source is exactly the moving input this function exists to refuse. The pre-network guard now covers git as well, naming "commit" rather than "SHA-256" in the message. Behaviour change worth stating plainly: a git source with no pinned commit is now refused where it used to be cloned. No recipe in this repository uses a git source, so nothing in-tree changes, but an out-of-tree recipe relying on an unpinned clone will need a commit added. Finding 2 (Medium) -- is_git_url() was `strstr(url, ".git") != NULL`, an unanchored substring test. It matched any host under *.github.io, including this organisation's own pages, and any archive whose path contained ".git" -- https://example.com/.github/release.tar.gz took the git branch and skipped the tarball digest check entirely. Now anchored: ".git" at end of string or before /, # or ?, plus the git:// and git@ schemes. Finding 4 (Low) -- "every archive form this fetcher extracts is covered" named three of four; .zip was missing. Added, and the claim narrowed to what it asserts. Verified: ctest 39/39 PASS test_pkg_fetch 7/7 PASS the new tests against the pre-fix fetcher: the git cases FAIL, so they discriminate rather than restate the fix no .git recipe in-tree (grep over *.yaml/*.yml) so the stricter git rule breaks nothing here Refs embeddedos-org#115
…removed `CI — eos` and `Python Unit Tests` are red on master (769191a), and both fail for one reason: these suites exist in tests/ but no add_executable() in tests/CMakeLists.txt builds them, so they never run: ['test_crypto_ed25519_loworder.c'] 3aa8644 (embeddedos-org#93, "remove duplicate test targets from tests/CMakeLists.txt") removed all four lines that build and register test_crypto_ed25519_loworder. It was not a duplicate — there was exactly one registration before that commit, and it took it. What stopped running is the suite that guards embeddedos-org#101: Ed25519 rejecting low-order public keys. A low-order key makes every term of the verification equation collapse to the identity regardless of the message, so a signature of all zeros verifies against any content at all. The fix is still in services/crypto/src/ed25519_verify.c and still correct; nothing has been checking it since embeddedos-org#93 merged, and nothing would have noticed if a later change had removed it too. Restored verbatim from 3aa8644^, in its original position before test_crypto_sha512. No other file changes. The guard that caught this is test_cmake_test_registration.py, added recently. It did exactly its job — this is the first thing it found. Verified: test_cmake_test_registration.py -> 5 passed (1 failed on master) pytest tests/ -> 15 passed (14 passed 1 failed on master) test_crypto_ed25519_loworder -> 5/5 tests passed ctest -> 39/39 passed (master builds 38)
eos_fetch_source() compared the archive against its expected digest only
when one was supplied:
if (expected_hash && expected_hash[0]) { ...compare... }
so calling it without a digest downloaded the archive and extracted it
having compared it against nothing -- and returned EOS_OK, which is what a
verified fetch also returns. The caller cannot tell the two apart, and the
argument is an ordinary optional parameter of a public function declared
in pkg/include/eos/package.h.
Refuse instead, before the network is touched. Git URLs stay exempt: a
clone carries its own object hashes.
ebuild made the same change to PackageFetcher.fetch() for the same reason
(embeddedos-org/ebuild#67) -- omitting one field bought an unverified
download there too.
tests/test_pkg_fetch.c is new; the function had no tests. Every case is
offline by construction, because returning before the network is touched
is the property under test: each archive form this fetcher extracts is
refused without a digest, an empty digest is refused rather than treated
as absent, an unsafe URL still fails as unsafe rather than as a checksum
error, and an empty URL is still the no-op it was.
39/39. Against the unfixed fetcher the suite fails on its first assertion.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Answers the review on embeddedos-org#115. The tarball half of this PR was correct; the identical defect was still live one branch over. Finding 1 (High) -- the guard read `if (is_tarball(url) && ...)`, so a git URL passed it with a digest and without one, and fetch_git() then never looked at expected_hash. A caller that pinned a revision got EOS_OK and the content with nothing compared -- verbatim the property this PR's own body names as the defect: "the caller cannot tell the two apart, because both return EOS_OK." fetch_git() now takes the pin and compares it against the commit the clone actually landed on (`git -C <dest> rev-parse HEAD`), failing EOS_ERR_CHECKSUM on a mismatch or if HEAD cannot be read. Finding 3 (Medium) -- the exemption's stated reason was "a clone carries its own object hashes". That is true and does not support the exemption: object hashes prove the transfer was not corrupted, not that the content is the content that was reviewed. `git clone --depth 1` takes whatever the default branch points at when it runs, so an unpinned git source is exactly the moving input this function exists to refuse. The pre-network guard now covers git as well, naming "commit" rather than "SHA-256" in the message. Behaviour change worth stating plainly: a git source with no pinned commit is now refused where it used to be cloned. No recipe in this repository uses a git source, so nothing in-tree changes, but an out-of-tree recipe relying on an unpinned clone will need a commit added. Finding 2 (Medium) -- is_git_url() was `strstr(url, ".git") != NULL`, an unanchored substring test. It matched any host under *.github.io, including this organisation's own pages, and any archive whose path contained ".git" -- https://example.com/.github/release.tar.gz took the git branch and skipped the tarball digest check entirely. Now anchored: ".git" at end of string or before /, # or ?, plus the git:// and git@ schemes. Finding 4 (Low) -- "every archive form this fetcher extracts is covered" named three of four; .zip was missing. Added, and the claim narrowed to what it asserts. Verified: ctest 39/39 PASS test_pkg_fetch 7/7 PASS the new tests against the pre-fix fetcher: the git cases FAIL, so they discriminate rather than restate the fix no .git recipe in-tree (grep over *.yaml/*.yml) so the stricter git rule breaks nothing here Refs embeddedos-org#115
53884e4 to
7626eec
Compare
srpatcha
left a comment
There was a problem hiding this comment.
Review — eos#115 "fix(pkg): refuse a source download that cannot be verified"
head: 7626eec author: Kartikey1306 ci: 2 required checks failing (pre-existing — see below)
Verdict: The central change is right and overdue — refusing a fetch with no digest, and anchoring is_git_url() so .github.io stops taking the git branch, both hold up and are well tested. But while verifying the new git_head_commit() path I found that pkg/src/fetch.c will execute an arbitrary command supplied in a source URL and return EOS_OK, which I reproduced. That is pre-existing on master, not introduced here; it is in this review because this PR is the one that adds a test asserting the URL guard works. Separately, the new commit pin cannot express a pin: git clone --depth 1 can only ever land on the current branch tip, so pinning to any earlier commit fails permanently. Also reproduced.
Findings
| # | Severity | File:line | Finding | Recommended fix |
|---|---|---|---|---|
| 1 | Critical | pkg/src/fetch.c:156-161, :195 |
Command injection through the source URL, with a success return. is_url_safe() blocks ; | > < $ ( ) \ 'and **does not block", `, &, or whitespace**, while every command is assembled as "…\"%s\"…" and handed to system(). A " closes the quoting; && then chains. Reproduced against this head through the public eos_fetch_source(): a URL of the form file://…/upstream.git" "…/x" && touch /tmp/ar/INJECT_MARKER && git clone "file://…/upstream.git created the marker and the call returned EOS_OK (0) — so the caller sees a verified fetch while an attacker-chosen command has run. master has the identical is_url_safe() at pkg/src/fetch.c:136-141, so this is not introduced by this PR. It belongs in this review because the PR adds PASS("an unsafe URL still fails as unsafe, not as a checksum error"), which exercises ; only and now stands as evidence that the guard holds. The same blocklist gates fetch_tarball()'s wget/curl, its tar/unzip extraction, and fetch_git()'s git clone. Source URLs come from package recipes, which is externally supplied data (§11). |
The real fix is to stop building a command line: posix_spawnp/execvp with an argument vector, _wspawnvp on Windows, no shell at all — then quoting stops being a security property. That is bigger than this PR and I am not asking for it here. What this PR should not do is leave a test in the tree implying the current guard is sufficient. At minimum, retitle that case to say it covers ; only, and open the follow-up. If a blocklist is kept in the interim it must reject ", \, &, and all whitespace — but a blocklist that has already missed four characters is not the thing to build confidence on. |
| 2 | High | pkg/src/fetch.c:164-181 |
dest_dir is validated nowhere — is_url_safe() is applied to url only — and this PR adds a second shell sink for it: git_head_commit() interpolates dest into git -C "%s" rev-parse HEAD and runs it through POPEN. Reproduced: a dest_dir containing " ; touch … ; git -C " created the marker. The pre-existing system() in fetch_git() has the same exposure, so this widens rather than creates it. |
Same fix as finding 1. If git_head_commit() stays shell-based in the meantime, it should at least refuse a dest containing anything outside a conservative path character set, and say so. |
| 3 | High | pkg/src/fetch.c:191-215 |
The commit pin cannot pin. fetch_git() runs git clone --depth 1 "<url>" "<dest>", which checks out whatever the default branch points at now, and then compares that against expected_commit. So the only value of expected_commit that can ever succeed is the current branch tip — which is precisely the moving target the guard's own comment says it exists to refuse. The moment upstream lands one commit, every fetch of that package fails with EOS_ERR_CHECKSUM and stays failing. Proven twice: a local two-commit repository shows --depth 1 landing on the tip, and end-to-end through eos_fetch_source(), pinning the tip returns EOS_OK while pinning the immediately preceding commit returns 9 with Commit mismatch. Failing closed is the right direction, so this is not a hole — it is a feature that does not work, with no test covering a pin that is not the tip. |
Fetch the pinned object rather than the branch: git init, git remote add origin <url>, git fetch --depth 1 origin <commit>, git checkout --detach FETCH_HEAD — then keep the rev-parse HEAD comparison as the belt-and-braces check it should be. Add the missing test: two commits, pin the older one, expect success. That test fails against the current implementation, which is what makes it worth having. |
| 4 | Medium | PR body | The body is a revision behind and contradicts the diff in its central claim. It says "Git URLs stay exempt — a clone carries its own object hashes" and shows a fix gated on is_tarball(url). The diff does the opposite: the guard covers every URL, and the code comment at fetch.c:276-291 explicitly rebuts that sentence ("object hashes prove the transfer was not corrupted, not that the content is the content that was reviewed"). The entire git_head_commit() / commit-pin mechanism — the largest and riskiest part of the change, and the subject of finding 3 — appears nowhere in the body. "5/5 in this suite, 39/39 repo-wide" is wrong in both halves: the suite runs 7 and the repo runs 40. The "Noted while here" section is stale too: services/pkg/eos_pkg.c is built on master (CMakeLists.txt:152 → services/pkg/CMakeLists.txt:8, landed with #120), and the all-zero eos_pkg_public_key now survives only as a comment at eos_pkg.c:97. |
Rewrite it around what the diff does. The git pin deserves the most space, because it is the part a reviewer most needs to think about and the part currently described as not happening. |
| 5 | Low | pkg/src/fetch.c:135-146 |
expected_commit is never checked for shape. A recipe that pins a git source with a SHA-256 archive digest — an easy confusion, since the parameter is named expected_hash at every call site and the header documents it as a hash — gets "Commit mismatch … expected <64 hex> … got <40 hex>", which describes a moved branch rather than the actual mistake. §9.2 asks for actionable diagnostics. |
Before comparing, check the pin is 40 or 64 lowercase-or-uppercase hex characters, and say "that is not a commit object name" when it is not. Three lines, and it turns the most likely user error into a message that names it. |
| 6 | Low | pkg/src/fetch.c:131-146 |
The anchoring is a clear improvement, but .git immediately before a query still wins: https://host/path?ref=v1.git is classified as a git repository. The ? case in the next == '?' branch is meant for repo.git?x=y, and it also catches .git occurring inside the query itself. |
Anchor on the path component only — stop the scan at the first ? or # before searching. Marginal; worth it only if you are touching the function anyway. |
| 7 | Low | CI | Two required checks are red: Cross-compile ARM64 kernel and Full-stack integration summary. Not caused by this PR — the same pair fails on all four open eos heads in this run, including #117, whose whole diff is .gitignore plus deleted build artifacts. Named because the brief requires it. |
Nothing here. |
Architecture conformance
Conforms. eos is Tier 1 Foundation (§21) and pkg/ is a core service in the same repository; the diff adds no include, link line or manifest entry that points up a tier. tests/test_pkg_fetch.c links eos_pkg and eos_core, both at or below its own level. §5.1's dependency direction is untouched.
On substance the change is doing what §14.1 and .ai/security.md ask. "Fail closed" is the named failure mode this project keeps hitting, and "a verification step that cannot run must fail, not pass" is exactly the if (expected_hash && expected_hash[0]) shape being removed. §28's evidence policy is the other half: a function that returns EOS_OK for both a verified and an unverified fetch makes "Validated" unreachable for anything built on it, because the caller has no way to tell which happened.
Findings 1 and 2 are where the same section is not yet satisfied. .ai/security.md says "every externally reachable parser is attack surface" and lists package distribution among the paths needing a documented threat model; §14.1 requires one for "boot, update, network and package distribution". A fetcher that shell-executes its input is what an unwritten threat model looks like. The pattern is not local to this file — git grep finds system() or popen() in 18 .c files on origin/master, including all eleven build backends, and ebuild has shell=True across layers/eosuite/gui/apps/. I have appended a proposal to .ai/autoreview/proposals/2026-09.md for the rule the design is missing; it is not this PR's job to fix any of it.
Proposed changes
In merge order:
- Finding 3 first, because it changes the shape of
fetch_git(): fetch the pinned object instead of the branch tip, and add the two-commit test that pins the older one. Without it the pin is decorative. - Finding 5, three lines, while
fetch_git()is open. - Finding 1 — do not fix the injection here, but change
PASS("an unsafe URL still fails as unsafe…")to say what it actually covers, and file the follow-up for argv-based execution. A test that reads as "unsafe URLs are handled" when four shell metacharacters get through is the kind of green tick this PR is otherwise removing from the tree. - Finding 4, rewrite the body.
Findings 1 and 2 do not block this PR — they are master's, and this PR does not make them worse except by adding one more sink for an already-unvalidated dest. Finding 3 does block, in the sense that merging it ships a pinning mechanism that breaks every git source on the next upstream commit.
Not attempted: a fix PR
I did not open one for findings 1 or 2, deliberately. The correct fix is to replace system()/popen() with argument-vector execution, which is a refactor and outside what an unattended agent may open. The available small fix — widening the blocklist — is a partial fix that would read as a solution, and a blocklist that has already missed ", \, & and whitespace is not the mechanism to add confidence to. Reporting it with a reproduction is the more useful thing I can do here.
Not checked
- Not run: the two failing required checks.
Cross-compile ARM64 kernelneeds a cross toolchain andFull-stack integration summaryneeds the integration harness. I did not reproduce either and did not determine the root cause on master. - Not verified: anything on Windows. The
_popen/_pclose/_stricmpmappings and thecurlbranch offetch_tarball()were read, not run, and the injection reproduction is POSIX/bin/sh.cmd.exequoting differs and I make no claim about it either way. - Not verified: whether any real recipe in this organisation currently pins a git source, and therefore how much finding 3 breaks in practice. The exposure is to whoever pins one next.
- Not verified: that
expected_hash's existing tarball comparison is correct. I exercised the refusal path, which returns before the network; I did not download and verify a real archive, socompute_file_sha256()and the comparison itself are untested by me. - Not checked: the other 17
.cfiles that callsystem()/popen(). I grepped for them to establish that the pattern is not local; I did not audit any of them. - Not checked: callers of
eos_fetch_source(). The new refusal is a behaviour change for anyone passingNULL, and I did not enumerate who does.
Evidence
Read-only git archive extractions of origin/master (feee272) and this head; the eos checkout was clean and untouched. Injection probes ran in a private temp directory against a local file:// repository — no network, and the only side effect was a zero-byte marker file under /tmp.
this head cmake -B build/test -DCMAKE_BUILD_TYPE=Release -DEOS_BUILD_TESTS=ON \
-DEOS_PRODUCT=vbox_test && cmake --build -> OK
ctest --no-tests=error -> 40/40 passed
./tests/test_pkg_fetch -> 7/7 tests passed
origin/master same configuration, ctest -> 38/38 passed
finding 3 — the pin can only ever be the tip:
local repo, two empty commits on main
pinned-old : 52757c8f… branch tip : 7d952435…
git clone --depth 1 file://… -> HEAD = 7d952435… (the tip)
through eos_fetch_source(), same repo:
pin = tip -> INFO Verified git commit: 7bab32c5… RESULT=0
pin = previous -> ERROR Commit mismatch …
expected 95a70226… got 7bab32c5… RESULT=9
finding 1 — injection through the URL, with a success return:
is_url_safe() blocklist: ; | > < $ ( ) ` ' — missing: " \ & whitespace
url = file://…/upstream.git" "…/x" && touch /tmp/ar/INJECT_MARKER && git clone "file://…/upstream.git
-> /tmp/ar/INJECT_MARKER created
-> RESULT=0 (EOS_OK — indistinguishable from a verified fetch)
finding 2 — injection through dest_dir, via the popen this PR adds:
dest = …/d2" ; touch /tmp/ar/INJECT_MARKER2 ; git -C "…/d2
-> /tmp/ar/INJECT_MARKER2 created
-> RESULT=8
finding 4 — body versus tree:
body: "Git URLs stay exempt" vs fetch.c:276-291, which rebuts it in a comment
body: "5/5 … 39/39" vs 7/7 and 40/40
body: "services/pkg/eos_pkg.c is in no CMakeLists.txt"
vs CMakeLists.txt:152 add_subdirectory(services/pkg)
services/pkg/CMakeLists.txt:8 eos_pkg.c (landed with #120)
body: "still carries static const uint8_t eos_pkg_public_key[32] = {0}"
vs eos_pkg.c:97, which is a comment
Automated architecture review of 7626eec04c9b — scheduled, model claude-opus-5, checked against the EmbeddedOS Master Design v2.0. Advisory only: this reviewer never approves, requests changes, or merges. Reply here to discuss or push back — a wrong finding is a bug worth reporting.
Every open eos pull request is currently red on `EoS Full-Stack Simulation`, and none of them caused it. The kernel job checks out embeddedos-org/eBoot at `ref: master`, and eBoot master does not compile: include/eos_image.h:135 and :142 static-assert offsetof(eos_image_header_t, reserved) for a member that became tlv_len + tlv_hash, and core/ed25519_verify.c redefines point_is_identity -- two pairs of PRs that each merged clean and broke the build together. eBoot embeddedos-org#94 repairs it and waits on review; until it lands, no change to any eos PR can make this job pass, and after it lands the next broken eBoot master does the same thing again. A floating `ref: master` makes every eos PR's CI depend on the moment-to- moment state of another repository's default branch. Pinning the two compiled-against checkouts makes the job a function of this repository plus two recorded SHAs -- red means the PR broke something, green means it did not, and a cross-repo bump is a reviewable one-line diff with CI on it. EBOOT_COMMIT a172a6d6e1e6 the newest eBoot master ancestor that compiles; found by walking master back and building each candidate. Its child 8a015b2 already carries the ed25519 redefinition. EBUILD_COMMIT e5d8052f3e2c ebuild's current master, frozen as-is. Both of its checkouts feed steps that are commented out (eFab does not exist), so this pin only removes the float. The middleware matrix (10 repos) still floats; it runs those repos' own tests rather than compiling eos against them, and 10 more pins deserve their own decision. When eBoot embeddedos-org#94 merges, bump EBOOT_COMMIT to the merge SHA. Verified before pushing, master vs pin under the job's own configuration (-DEBLDR_BOARD=qemu_arm64 -DEBLDR_VERIFY_STAGE1=OFF -DCMAKE_BUILD_TYPE=Release, cross-compile mode): eBoot@master fails at the eos_image.h asserts, the exact error in the job log; eBoot@a172a6d builds every target, rc=0. Both pinned SHAs verified reachable from their repos' master. Workflow parses; the three `ref:` edits and the env block are the whole diff. This PR's own simulation run is the live test: it uses this branch's workflow and should go green while every master-based PR stays red. Carried onto this branch from embeddedos-org#135 (001128d) so this PR's own simulation job runs against a buildable eBoot: each pull request executes its own copy of the workflow, so the fix greens a PR only once its branch contains it. Byte-identical to embeddedos-org#135's commit -- git drops it as already-upstream the moment embeddedos-org#135 merges. The pin itself is embeddedos-org#135's to review.
srpatcha
left a comment
There was a problem hiding this comment.
Review — eos#115 "fix(pkg): refuse a source download that cannot be verified"
head: 4375466 author: Kartikey1306 ci: 22 pass, QEMU ARM64 simulation (13 TCs) and Cross-compile ARM Cortex-M4 pending, 0 failing
Verdict: The only change since the 7626eec0 review is 4375466, and it is a byte-identical copy of #135's cross-repo pin — the fourth open PR now carrying it. pkg/src/fetch.c is untouched, so every finding from that review is still open, including one rated Critical, and there has been no reply on the thread since 2026-09-01. I re-ran the reproductions against this exact head rather than carrying them forward on trust; all three still fire. Build is clean, ctest --no-tests=error 40/40, test_pkg_fetch 7/7.
Findings
| # | Severity | File:line | Finding | Recommended fix |
|---|---|---|---|---|
| 1 | Medium | .github/workflows/eos-simulation.yml:11-20, :41, :48, :288 |
4375466 ("ci: pin the eBoot and ebuild checkouts instead of floating on master") duplicates #135, and also #114's f43d860 and #116's 2a32e3a. Verified: git diff of this file between this head and each of those three heads is empty in all three comparisons — same env: block, same comment, same three ref: substitutions. Four open PRs, one CI change, none of the four bodies mentioning it. Identical hunks merge cleanly, so nothing breaks; the cost is that a cross-repository dependency pin is being carried through a PR about package-fetch verification, where nobody reviewing the title will look for it. |
Drop 4375466. #135 is scoped to it and has the staleness-guard commit (f34edb3) these copies lack. Its substance is reviewed there and is not re-raised here. |
Carried forward from 7626eec0 — unresolved, re-verified at this head
pkg/src/fetch.c is byte-identical between 7626eec0 and this head, so none of these are new. They are restated with their severities intact because a Critical must not drop out of the record between heads, and because each was re-reproduced today rather than assumed.
| # | Severity | File:line | Status at 43754663 |
|---|---|---|---|
| C1 | Critical | pkg/src/fetch.c:157-162, :192-194 |
Still reproduces, still returns success. is_url_safe() blocks ; | > < $ ( ) \ 'and still does not block", `, & or whitespace, while fetch_git() assembles git clone --depth 1 "%s" "%s" and hands it to system(). Probed through the public eos_fetch_source() against a local file:// repository: a URL closing the quote and chaining with && created a marker file and the call returned EOS_OK (0) — the caller cannot distinguish it from a verified fetch. Pre-existing on master; in this review because this PR adds PASS("an unsafe URL still fails as unsafe, not as a checksum error"), which exercises ; only and now reads as evidence the guard holds. |
| C2 | High | pkg/src/fetch.c:165-171 |
Still reproduces. dest_dir is validated nowhere and this PR's git_head_commit() interpolates it into git -C "%s" rev-parse HEAD under POPEN. A dest_dir containing " ; touch … ; git -C " created a second marker (RESULT=8). This is the one place the PR widens an existing exposure rather than inheriting it. |
| C3 | High | pkg/src/fetch.c:192-205 |
Still reproduces. The pin can only ever be the branch tip: git clone --depth 1 checks out whatever the default branch points at now, and the comparison then rejects anything else. Against a two-commit local repository at this head — pin the tip → Verified git commit, RESULT=0; pin HEAD~1 → The branch has moved since this revision was pinned, RESULT=9. So the feature fails permanently for every package the moment upstream lands one commit, and no test covers a pin that is not the tip. Failing closed is the right direction, so this is a feature that does not work rather than a hole — but merging it ships that. |
| C4 | Medium | PR body | Unchanged, and now also silent about 4375466. Still says "Git URLs stay exempt — a clone carries its own object hashes", which the code comments at fetch.c:197-200 and :284-290 explicitly rebut; still reports "5/5 … 39/39" against an actual 7/7 and 40/40; still says services/pkg/eos_pkg.c "is in no CMakeLists.txt" when CMakeLists.txt:152 → services/pkg/CMakeLists.txt:8 builds it. |
| C5 | Low | pkg/src/fetch.c:291-296, fetch_git() |
Unchanged. expected_commit is never checked for shape, so pinning a git source with a 64-hex archive digest reports "Commit mismatch" — a moved branch — rather than naming the actual mistake. §9.2 asks for actionable diagnostics. |
| C6 | Low | pkg/src/fetch.c:137-149 (is_git_url) |
Unchanged. .git immediately before a query still classifies as git: https://host/path?ref=v1.git. |
The full reasoning and the recommended fixes for C1–C6 are in the review posted against 7626eec0; I have not rewritten them here, because nothing about them has changed and repeating them at length makes the thread harder to act on rather than easier.
Architecture conformance
Conforms, unchanged from 7626eec0. eos is Tier 1 — Foundation (§21); pkg/ is a core service in the same repository and tests/test_pkg_fetch.c links eos_pkg and eos_core, both at or below its own level, so nothing points up a tier (§5.1). The pin in 4375466 is a CI-time dependency onto two other Tier-1 repositories, so it points sideways; §5.1 governs runtime dependencies only, and that gap plus this exact SHA are already recorded in .ai/autoreview/proposals/2026-09.md.
The substance still does what §14.1 and .ai/security.md ask of it. "Fail closed" is the named failure mode this project keeps hitting and if (expected_hash && expected_hash[0]) was exactly the shape it warns about; §28's evidence policy is the other half, since a function returning EOS_OK for both a verified and an unverified fetch makes "Validated" unreachable for anything built on it. C1 and C2 are where the same section is still not satisfied: §14.1 requires a documented threat model for package distribution, and a fetcher that shell-executes its input is what an unwritten one looks like. The proposal for the missing design rule is already in .ai/autoreview/proposals/2026-09.md and is not re-appended.
Proposed changes
Unchanged from the previous review, in merge order: C3 first (fetch the pinned object — git init / git remote add / git fetch --depth 1 origin <commit> / git checkout --detach FETCH_HEAD — plus the two-commit test that pins the older one, which fails against the current implementation); then C5; then retitle the unsafe-URL test case to say it covers ; only and file the argv-execution follow-up for C1/C2; then the body. Plus, new here: drop 4375466.
C1 and C2 do not block this PR — they are master's, and this PR worsens them only by adding one more sink for an already-unvalidated dest. C3 does block, in the sense that merging it ships a pinning mechanism that breaks every git source on the next upstream commit.
Not attempted: a fix PR
Still deliberately not opened for C1 or C2. The correct fix is replacing system()/popen() with argument-vector execution, which is a refactor and outside what an unattended agent may open. The available small fix — widening the blocklist — would read as a solution, and a blocklist that has already missed four characters is not the mechanism to add confidence to. C3 is small but is a behaviour change to a fetch path with no test I can write offline that proves the replacement clones correctly from a real remote, so it does not meet the "provable by running something" bar either.
Not checked
QEMU ARM64 simulation (13 TCs)andCross-compile ARM Cortex-M4were pending when this bundle was built, so I cannot report their outcome; nothing was failing.Cross-compile ARM64 kernel, red on this PR's previous head, is green here — which is what4375466buys and why it keeps being copied.Cross-compile ARM Cortex-M4is declared by two workflows (build.yml:74,ci.yml:117), so that name covers two check runs on everyeosPR and its state is not answerable from the name alone.- Windows: NOT VERIFIED. The
_popen/_pclose/_stricmpmappings and thecurlbranch were read, not run. The reproductions above are POSIX/bin/sh;cmd.exequoting differs and I make no claim either way. - The tarball download path was not exercised end to end. My probe's non-
.gitURL returnedEOS_ERR_FETCHbefore any comparison, socompute_file_sha256()and the archive comparison remain untested by me. Only the refusal path and the git path were driven. - Not verified: whether any real recipe in this organisation pins a git source, and therefore how much C3 breaks in practice.
- Not checked: the other 17
.cfiles onorigin/masterthat callsystem()/popen(). Grepped only to establish the pattern is not local to this file. - Not checked: callers of
eos_fetch_source(). The refusal is a behaviour change for anyone passingNULLand I did not enumerate who does. - The pinned eBoot tree was not built.
a172a6dexists and is an ancestor of eBoot'sorigin/master; I did not compile eos's kernel job against it.
Evidence
Read-only git archive extraction of 43754663; the eos checkout was clean and was not modified. The injection probes ran in a private /tmp directory against a local file:// repository — no network, and the only side effects were two zero-byte marker files under that directory.
this head cmake -B build/test -DCMAKE_BUILD_TYPE=Release -DEOS_BUILD_TESTS=ON \
-DEOS_PRODUCT=vbox_test && cmake --build -> OK
ctest --no-tests=error -> 40/40 passed
./tests/test_pkg_fetch -> 7/7 tests passed
pkg/src/fetch.c identical to 7626eec0:
git diff --stat 7626eec0 43754663
-> .github/workflows/eos-simulation.yml | 12 +++, 3 --- (only)
finding 1 (the pin, duplicated):
git diff 43754663 <#114 f43d860> -- .github/workflows/eos-simulation.yml -> empty
git diff 43754663 <#116 2a32e3a> -- .github/workflows/eos-simulation.yml -> empty
git diff 43754663 <#135 f34edb3> -- .github/workflows/eos-simulation.yml -> empty
C1, through the public eos_fetch_source(), at this head:
is_url_safe() @ fetch.c:160 strpbrk(url, ";|><$()`'") — no " \ & whitespace
url = file://…/upstream.git" "…/cx" && touch …/INJECT_MARKER && git clone "file://…/upstream.git
-> INJECT_MARKER created
-> INFO Verified git commit: 88bab0b1…
-> RESULT=0 (EOS_OK — indistinguishable from a verified fetch)
C2, through dest_dir into git_head_commit()'s POPEN:
dest = …/d" ; touch …/INJECT_MARKER2 ; git -C "…/d
-> INJECT_MARKER2 created
-> RESULT=8
C3, the pin can only be the tip (two-commit local repo):
pin = 88bab0b1 (tip) -> INFO Verified git commit: 88bab0b1… RESULT=0
pin = c1441fad (HEAD~1) -> "The branch has moved since this revision
was pinned." got 88bab0b1… RESULT=9
Automated architecture review of 43754663a738 — scheduled, model claude-opus-5, checked against the EmbeddedOS Master Design v2.0. Advisory only: this reviewer never approves, requests changes, or merges. Reply here to discuss or push back — a wrong finding is a bug worth reporting.
Nine PRs were merged into master within minutes on 09-08, each on the base it was written against. Master has not compiled since, and the Python guards that would have named the rest never ran because the C build failed first. Compile (every C job, CodeQL, and the ARM64 kernel in the simulation): - services/linux/src/linux_security.c: embeddedos-org#119 and embeddedos-org#132 each added an #else to the same #ifndef _WIN32 in eos_busybox_install_to_rootfs(), so master has "#else after #else". The embeddedos-org#132 arm (`(void)bb;`) is the one removed -- the embeddedos-org#119 arm already uses bb and reports the unsupported platform. Guards from embeddedos-org#121 / embeddedos-org#93 that later merges walked back: - ci.yml: embeddedos-org#132 added windows-test after embeddedos-org#121's gate; the gate did not wait for it, so "CI Gate" could be green with the MSVC leg red. - test_ci_gate.py: embeddedos-org#129 gave eosim-sanity.yml and simulation-test.yml a path-filtered pull_request trigger (they test their own edits); a path-filtered check cannot be required, so both are recorded in NOT_REQUIRED with the book-build.yml reason. - tests/CMakeLists.txt: test_linux_security_paths (embeddedos-org#119) and test_pkg_fetch (embeddedos-org#115) had no add_executable(). embeddedos-org#115's replay replaced embeddedos-org#119's block with its own, and embeddedos-org#118's replay replaced that; two suites compiled against nothing. Both registered again. 41 -> 43 suites. - tests/test_kernel.c: four tests from embeddedos-org#130 and embeddedos-org#131 were defined and never called -- their RUN() lines did not survive the replay of main(). And the one that was not bookkeeping: - kernel/src/task.c: embeddedos-org#130 was merged after embeddedos-org#131 from a base that predates it, and its copy of task.c replaced embeddedos-org#131's. embeddedos-org#131 had also flattened 574 CRLF line endings, so its 1166-line diff hid a 39/21 change and the replay took embeddedos-org#130's side wholesale. Master kept embeddedos-org#131's tests and lost its kernel: the idle task could be deleted and suspended, a half-initialised TCB was published to the scheduler before its stack existed, and eos_schedule() pointed g_current_sp at the outgoing task. test_idle_task_is_permanent fails on master the moment it is called. embeddedos-org#131's task.c diff re-applied on top of embeddedos-org#130's; the result is embeddedos-org#131's file plus embeddedos-org#130's wake_armed hunk and nothing else (verified by diff against 3a00bd9). Verified locally (macOS, clang): Release build clean, 43/43 ctest; the README default configuration builds; 47/47 pytest. Not in this PR: the nightly "Upstream drift" job builds eBoot at master and eBoot master is broken separately (its own repair PR); bump EBOOT_COMMIT in eos-simulation.yml once that lands.
Answers the review on embeddedos-org#115. The tarball half of this PR was correct; the identical defect was still live one branch over. Finding 1 (High) -- the guard read `if (is_tarball(url) && ...)`, so a git URL passed it with a digest and without one, and fetch_git() then never looked at expected_hash. A caller that pinned a revision got EOS_OK and the content with nothing compared -- verbatim the property this PR's own body names as the defect: "the caller cannot tell the two apart, because both return EOS_OK." fetch_git() now takes the pin and compares it against the commit the clone actually landed on (`git -C <dest> rev-parse HEAD`), failing EOS_ERR_CHECKSUM on a mismatch or if HEAD cannot be read. Finding 3 (Medium) -- the exemption's stated reason was "a clone carries its own object hashes". That is true and does not support the exemption: object hashes prove the transfer was not corrupted, not that the content is the content that was reviewed. `git clone --depth 1` takes whatever the default branch points at when it runs, so an unpinned git source is exactly the moving input this function exists to refuse. The pre-network guard now covers git as well, naming "commit" rather than "SHA-256" in the message. Behaviour change worth stating plainly: a git source with no pinned commit is now refused where it used to be cloned. No recipe in this repository uses a git source, so nothing in-tree changes, but an out-of-tree recipe relying on an unpinned clone will need a commit added. Finding 2 (Medium) -- is_git_url() was `strstr(url, ".git") != NULL`, an unanchored substring test. It matched any host under *.github.io, including this organisation's own pages, and any archive whose path contained ".git" -- https://example.com/.github/release.tar.gz took the git branch and skipped the tarball digest check entirely. Now anchored: ".git" at end of string or before /, # or ?, plus the git:// and git@ schemes. Finding 4 (Low) -- "every archive form this fetcher extracts is covered" named three of four; .zip was missing. Added, and the claim narrowed to what it asserts. Verified: ctest 39/39 PASS test_pkg_fetch 7/7 PASS the new tests against the pre-fix fetcher: the git cases FAIL, so they discriminate rather than restate the fix no .git recipe in-tree (grep over *.yaml/*.yml) so the stricter git rule breaks nothing here Refs embeddedos-org#115
An omitted argument buys an unverified download
eos_fetch_source()compared the archive against its expected digest only when one was supplied:So calling it without a digest downloads the archive and extracts it having compared it against nothing — and returns
EOS_OK, which is exactly what a verified fetch returns. The caller cannot tell the two apart.expected_hashis an ordinary optional parameter of a public function declared inpkg/include/eos/package.h, so this is one omitted argument away at every call site.Precedent in this organisation
ebuild made the same change to
PackageFetcher.fetch()for the same reason — embeddedos-org/ebuild#67, "No checksum meant no verification… omitting one field silently bought an unverified download." This is that defect in the C fetcher.Fix
Refuse before the network is touched:
Git URLs stay exempt — a clone carries its own object hashes.
Tests
tests/test_pkg_fetch.cis new; the function had no tests at all. Every case is offline by construction, because returning before the network is touched is the property under test:.tar.gz/.tar.xz/.tgzwith no digest""as the digest;rm -rf)EOS_ERR_FETCH, so the new guard did not widen into rejecting everythingEOS_OKno-op it wasVerified in both directions. Against the unfixed fetcher:
With the fix:
5/5in this suite, 39/39 repo-wide.Noted while here, not changed
services/pkg/eos_pkg.c(964 lines, with the full ed25519 verification path) is in noCMakeLists.txt. The package library that is built —pkg/src/package.c, 179 lines — performs no signature verification at all. The orphan also still carriesstatic const uint8_t eos_pkg_public_key[32] = {0}, the all-zero key fix(crypto): reject Ed25519 public keys outside the prime-order subgroup #99/fix(crypto): reject Ed25519 low-order public keys #101 hardened the verifier against.pkg/src/fetch.ccarries a third private SHA-256 implementation alongsideservices/crypto/src/sha256.c.Both are structural calls for a maintainer rather than something to change in a fetch fix.