Repository navigation
Conversation
|
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
srpatcha
left a comment
There was a problem hiding this comment.
Verified — and the framing is right
recipes/*.yaml is the tool's supply chain, and treating a recipe as a pin rather than a hint is the correct model. Merged onto current master locally: 223 passed.
The defect that matters most
archive_path = self._download(recipe)
if recipe.checksum: # ← absent: download and extract, unverified
self._verify_checksum(archive_path, recipe.checksum)Omitting one optional field silently bought an unverified download. That is a fail-open default on the one code path where failing open means executing someone else's bytes.
Worse, tests/ebuild/test_package_fetcher.py carried test_empty_checksum_skips_verification — the insecure behaviour was pinned by a test as though it were intended, so anyone who fixed it would have looked like they were breaking something. Replacing that test with one asserting the refusal is the part of this PR I would least want dropped.
Checked the new gate reads the way it should:
Refusing to fetch <name> v<version>: the recipe carries no checksum, so there
is nothing to verify the download against.
Keeping checksum optional on PackageRecipe while making it mandatory at fetch() is the right seam. A recipe also models packages that are never downloaded, and forcing it at the dataclass punished twenty resolver tests that build recipes in memory. The requirement belongs at the boundary where bytes actually arrive.
Independently confirmed one of the digests
I did not re-download all five, but I checked the FreeRTOS claim against the API, since "matches none of the release's assets" is a strong statement:
$ gh api repos/FreeRTOS/FreeRTOS-Kernel/releases/tags/V11.1.0
FreeRTOS-KernelV11.1.0.zip 3324125 bytes
Exactly one asset, exactly the size and name you record, and the recipe URL points at it. That matches your account precisely.
sha256:placeholder passing validate() is the enabling bug behind all of it — a checksum field that accepts any string is not a checksum field. Requiring the shape when the value is present closes it.
Also right
Rejecting plaintext http:// outright rather than warning. A pinned digest over plaintext is still better than nothing, but there is no reason to accept the downgrade when every URL here is available over TLS.
Verification
Merged onto origin/master; pytest 223 passed; FreeRTOS release assets checked via the GitHub API. Not re-verified: the littlefs, lwip, mbedtls and zlib digests — I am taking those on your stated method of downloading twice and comparing.
73e4f43 to
e29f03c
Compare
|
Rebased onto current The PR is unchanged in substance: 8 files, the four corrected digests plus the two gates. I re-confirmed the defect is still live on master:
Thank you for independently checking the FreeRTOS release assets. That was the claim most worth not taking on trust, since "matches none of the release's assets" is either true or I had made an error, with nothing in between. |
9fe2f57 to
c2408b4
Compare
|
Now stacked on #66 — merge #66 first, which matches the merge order you already published. Why: enabling the Windows leg (via the ci.yml fix) made this PR's checks run — the unescaped drive-letter colon, i.e. exactly the
|
7721287 to
9191fe3
Compare
targz_bytes() built its .tar.gz with default timestamps, so two calls
with the same marker returned different bytes -- and different SHA-256 --
whenever they landed on opposite sides of a second boundary:
a = sha256(targz_bytes("x")); sleep(1.1)
b = sha256(targz_bytes("x"))
a == b -> False
Every checksum in this module is derived from the helper and then has to
match the bytes fake_download hands out later, so the suite was a coin
flip on timing. It held on fast Linux runners and failed on Windows.
Two timestamps are involved and both default to "now": the tar member
header and the gzip header. Writing the tar uncompressed with mtime=0 and
compressing it through GzipFile(mtime=0) pins both.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
28063de to
6af6881
Compare
eos_fetch_source() compared the archive against its expected digest only
when one was supplied:
if (expected_hash && expected_hash[0]) { ...compare... }
so calling it without a digest downloaded the archive and extracted it
having compared it against nothing -- and returned EOS_OK, which is what a
verified fetch also returns. The caller cannot tell the two apart, and the
argument is an ordinary optional parameter of a public function declared
in pkg/include/eos/package.h.
Refuse instead, before the network is touched. Git URLs stay exempt: a
clone carries its own object hashes.
ebuild made the same change to PackageFetcher.fetch() for the same reason
(embeddedos-org/ebuild#67) -- omitting one field bought an unverified
download there too.
tests/test_pkg_fetch.c is new; the function had no tests. Every case is
offline by construction, because returning before the network is touched
is the property under test: each archive form this fetcher extracts is
refused without a digest, an empty digest is refused rather than treated
as absent, an unsafe URL still fails as unsafe rather than as a checksum
error, and an empty URL is still the no-op it was.
39/39. Against the unfixed fetcher the suite fails on its first assertion.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
eos_fetch_source() compared the archive against its expected digest only
when one was supplied:
if (expected_hash && expected_hash[0]) { ...compare... }
so calling it without a digest downloaded the archive and extracted it
having compared it against nothing -- and returned EOS_OK, which is what a
verified fetch also returns. The caller cannot tell the two apart, and the
argument is an ordinary optional parameter of a public function declared
in pkg/include/eos/package.h.
Refuse instead, before the network is touched. Git URLs stay exempt: a
clone carries its own object hashes.
ebuild made the same change to PackageFetcher.fetch() for the same reason
(embeddedos-org/ebuild#67) -- omitting one field bought an unverified
download there too.
tests/test_pkg_fetch.c is new; the function had no tests. Every case is
offline by construction, because returning before the network is touched
is the property under test: each archive form this fetcher extracts is
refused without a digest, an empty digest is refused rather than treated
as absent, an unsafe URL still fails as unsafe rather than as a checksum
error, and an empty URL is still the no-op it was.
39/39. Against the unfixed fetcher the suite fails on its first assertion.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(build): restore the low-order Ed25519 suite that #93 removed `CI — eos` and `Python Unit Tests` are red on master (769191a), and both fail for one reason: these suites exist in tests/ but no add_executable() in tests/CMakeLists.txt builds them, so they never run: ['test_crypto_ed25519_loworder.c'] 3aa8644 (#93, "remove duplicate test targets from tests/CMakeLists.txt") removed all four lines that build and register test_crypto_ed25519_loworder. It was not a duplicate — there was exactly one registration before that commit, and it took it. What stopped running is the suite that guards #101: Ed25519 rejecting low-order public keys. A low-order key makes every term of the verification equation collapse to the identity regardless of the message, so a signature of all zeros verifies against any content at all. The fix is still in services/crypto/src/ed25519_verify.c and still correct; nothing has been checking it since #93 merged, and nothing would have noticed if a later change had removed it too. Restored verbatim from 3aa8644^, in its original position before test_crypto_sha512. No other file changes. The guard that caught this is test_cmake_test_registration.py, added recently. It did exactly its job — this is the first thing it found. Verified: test_cmake_test_registration.py -> 5 passed (1 failed on master) pytest tests/ -> 15 passed (14 passed 1 failed on master) test_crypto_ed25519_loworder -> 5/5 tests passed ctest -> 39/39 passed (master builds 38) * fix(pkg): refuse a source download that cannot be verified eos_fetch_source() compared the archive against its expected digest only when one was supplied: if (expected_hash && expected_hash[0]) { ...compare... } so calling it without a digest downloaded the archive and extracted it having compared it against nothing -- and returned EOS_OK, which is what a verified fetch also returns. The caller cannot tell the two apart, and the argument is an ordinary optional parameter of a public function declared in pkg/include/eos/package.h. Refuse instead, before the network is touched. Git URLs stay exempt: a clone carries its own object hashes. ebuild made the same change to PackageFetcher.fetch() for the same reason (embeddedos-org/ebuild#67) -- omitting one field bought an unverified download there too. tests/test_pkg_fetch.c is new; the function had no tests. Every case is offline by construction, because returning before the network is touched is the property under test: each archive form this fetcher extracts is refused without a digest, an empty digest is refused rather than treated as absent, an unsafe URL still fails as unsafe rather than as a checksum error, and an empty URL is still the no-op it was. 39/39. Against the unfixed fetcher the suite fails on its first assertion. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(pkg): honour the digest on the git path too, and anchor is_git_url Answers the review on #115. The tarball half of this PR was correct; the identical defect was still live one branch over. Finding 1 (High) -- the guard read `if (is_tarball(url) && ...)`, so a git URL passed it with a digest and without one, and fetch_git() then never looked at expected_hash. A caller that pinned a revision got EOS_OK and the content with nothing compared -- verbatim the property this PR's own body names as the defect: "the caller cannot tell the two apart, because both return EOS_OK." fetch_git() now takes the pin and compares it against the commit the clone actually landed on (`git -C <dest> rev-parse HEAD`), failing EOS_ERR_CHECKSUM on a mismatch or if HEAD cannot be read. Finding 3 (Medium) -- the exemption's stated reason was "a clone carries its own object hashes". That is true and does not support the exemption: object hashes prove the transfer was not corrupted, not that the content is the content that was reviewed. `git clone --depth 1` takes whatever the default branch points at when it runs, so an unpinned git source is exactly the moving input this function exists to refuse. The pre-network guard now covers git as well, naming "commit" rather than "SHA-256" in the message. Behaviour change worth stating plainly: a git source with no pinned commit is now refused where it used to be cloned. No recipe in this repository uses a git source, so nothing in-tree changes, but an out-of-tree recipe relying on an unpinned clone will need a commit added. Finding 2 (Medium) -- is_git_url() was `strstr(url, ".git") != NULL`, an unanchored substring test. It matched any host under *.github.io, including this organisation's own pages, and any archive whose path contained ".git" -- https://example.com/.github/release.tar.gz took the git branch and skipped the tarball digest check entirely. Now anchored: ".git" at end of string or before /, # or ?, plus the git:// and git@ schemes. Finding 4 (Low) -- "every archive form this fetcher extracts is covered" named three of four; .zip was missing. Added, and the claim narrowed to what it asserts. Verified: ctest 39/39 PASS test_pkg_fetch 7/7 PASS the new tests against the pre-fix fetcher: the git cases FAIL, so they discriminate rather than restate the fix no .git recipe in-tree (grep over *.yaml/*.yml) so the stricter git rule breaks nothing here Refs #115 * ci: pin the eBoot and ebuild checkouts instead of floating on master Every open eos pull request is currently red on `EoS Full-Stack Simulation`, and none of them caused it. The kernel job checks out embeddedos-org/eBoot at `ref: master`, and eBoot master does not compile: include/eos_image.h:135 and :142 static-assert offsetof(eos_image_header_t, reserved) for a member that became tlv_len + tlv_hash, and core/ed25519_verify.c redefines point_is_identity -- two pairs of PRs that each merged clean and broke the build together. eBoot #94 repairs it and waits on review; until it lands, no change to any eos PR can make this job pass, and after it lands the next broken eBoot master does the same thing again. A floating `ref: master` makes every eos PR's CI depend on the moment-to- moment state of another repository's default branch. Pinning the two compiled-against checkouts makes the job a function of this repository plus two recorded SHAs -- red means the PR broke something, green means it did not, and a cross-repo bump is a reviewable one-line diff with CI on it. EBOOT_COMMIT a172a6d6e1e6 the newest eBoot master ancestor that compiles; found by walking master back and building each candidate. Its child 8a015b2 already carries the ed25519 redefinition. EBUILD_COMMIT e5d8052f3e2c ebuild's current master, frozen as-is. Both of its checkouts feed steps that are commented out (eFab does not exist), so this pin only removes the float. The middleware matrix (10 repos) still floats; it runs those repos' own tests rather than compiling eos against them, and 10 more pins deserve their own decision. When eBoot #94 merges, bump EBOOT_COMMIT to the merge SHA. Verified before pushing, master vs pin under the job's own configuration (-DEBLDR_BOARD=qemu_arm64 -DEBLDR_VERIFY_STAGE1=OFF -DCMAKE_BUILD_TYPE=Release, cross-compile mode): eBoot@master fails at the eos_image.h asserts, the exact error in the job log; eBoot@a172a6d builds every target, rc=0. Both pinned SHAs verified reachable from their repos' master. Workflow parses; the three `ref:` edits and the env block are the whole diff. This PR's own simulation run is the live test: it uses this branch's workflow and should go green while every master-based PR stays red. Carried onto this branch from #135 (001128d) so this PR's own simulation job runs against a buildable eBoot: each pull request executes its own copy of the workflow, so the fix greens a PR only once its branch contains it. Byte-identical to #135's commit -- git drops it as already-upstream the moment #135 merges. The pin itself is #135's to review. --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> Co-authored-by: Srikanth Patchava <srikanth.patchava@outlook.com>
eos_fetch_source() compared the archive against its expected digest only
when one was supplied:
if (expected_hash && expected_hash[0]) { ...compare... }
so calling it without a digest downloaded the archive and extracted it
having compared it against nothing -- and returned EOS_OK, which is what a
verified fetch also returns. The caller cannot tell the two apart, and the
argument is an ordinary optional parameter of a public function declared
in pkg/include/eos/package.h.
Refuse instead, before the network is touched. Git URLs stay exempt: a
clone carries its own object hashes.
ebuild made the same change to PackageFetcher.fetch() for the same reason
(embeddedos-org/ebuild#67) -- omitting one field bought an unverified
download there too.
tests/test_pkg_fetch.c is new; the function had no tests. Every case is
offline by construction, because returning before the network is touched
is the property under test: each archive form this fetcher extracts is
refused without a digest, an empty digest is refused rather than treated
as absent, an unsafe URL still fails as unsafe rather than as a checksum
error, and an empty URL is still the no-op it was.
39/39. Against the unfixed fetcher the suite fails on its first assertion.
Rebased onto current master and reduced to one change. The supply-chain content this PR opened with — the four corrected pins, the checksum-format validation, and
fetch()refusing to download without a digest — is now on master, applied there directly rather than by merging this branch. All of it verified identical, so it is dropped from here.What remains is the one piece master does not have.
The fixture archive is not reproducible
targz_bytes()builds its.tar.gzwith default timestamps, so two calls with the same marker return different bytes:Every checksum in this module is derived from the helper and then has to match the bytes
fake_downloadhands out later, so the suite is a coin flip on timing — it holds on fast Linux runners and fails on Windows.Two timestamps default to "now": the tar member header and the gzip header. Writing the tar uncompressed with
info.mtime = 0and compressing throughGzipFile(mtime=0)pins both.16/16 in this module pass, and the digest is now stable across a second boundary.