exfatprogs: serve modified Source0 with upstream tests/ stripped#17206
Open
PawelWMS wants to merge 1 commit into
Open
exfatprogs: serve modified Source0 with upstream tests/ stripped#17206PawelWMS wants to merge 1 commit into
PawelWMS wants to merge 1 commit into
Conversation
PawelWMS
force-pushed
the
pawelwi/exfatprogs-strip-tests
branch
4 times, most recently
from
7522b34 to
89c7384
Compare
Contributor
There was a problem hiding this comment.
Pull request overview
This PR switches exfatprogs to use a repacked Source0 tarball with the upstream tests/ directory removed, to avoid failures in the automated package-signing malware scan while keeping Azure Linux package behavior unchanged (no %check, and tests/ isn’t shipped).
Changes:
- Add a dedicated
exfatprogs.comp.tomlthat pointsSource0at a modified tarball stored in lookaside storage. - Add a deterministic
modify_source.shhelper to striptests/and repack with reproducible tar/xz settings. - Regenerate rendered packaging artifacts (
specs/.../sources,specs/.../*.spec) and refreshlocks/exfatprogs.lock.
Reviewed changes
Copilot reviewed 5 out of 6 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| base/comps/exfatprogs/exfatprogs.comp.toml | Defines a replacement Source0 tarball via source-files for exfatprogs. |
| base/comps/exfatprogs/modify_source.sh | Provides deterministic download/verify/strip/repack instructions for the modified tarball. |
| base/comps/components.toml | Removes the inline exfatprogs component entry (component now defined via included .comp.toml). |
| specs/e/exfatprogs/sources | Updates the lookaside sources manifest to the modified tarball SHA-512. |
| specs/e/exfatprogs/exfatprogs.spec | Re-renders the spec (autorelease/autochangelog update only). |
| locks/exfatprogs.lock | Updates the lock input fingerprint after config changes. |
PawelWMS
force-pushed
the
pawelwi/exfatprogs-strip-tests
branch
2 times, most recently
from
de24bb3 to
01f71ac
Compare
The upstream `exfatprogs-1.3.1.tar.xz` ships a `tests/` tree
containing 19 deliberately-corrupted exFAT filesystem images (one
`exfat.img.tar.xz` per corruption scenario: `bad_bitmap`,
`bad_dentries`, `bs_bad_csum`, `loop_chain`, etc.). Those malformed
images are the root cause: the FS-aware deep scanner in the
automated package-signing pipeline unwraps the outer xz/tar/xz
layers fine, then walks the resulting raw exFAT metadata (FAT
cluster chains, directory entries, allocation bitmap) looking for
embedded executables and enters runaway behaviour on the crafted
cycles / self-referential structures -- the per-artifact scan
either spins until its timeout fires or trips a "potential bomb"
abort, marking the SRPM unscannable. The failure is shape-driven,
not resource-bound: it persists on hosts with ample storage / CPU.
The double `.tar.xz` indirection hides the malformed images from
simple compression-ratio heuristics so the deep scanner is reached.
The rest of `tests/` -- the `tests/upcase_table/` shell fixtures
and the `test_fsck.sh` driver -- is not itself problematic. It is
just the shell harness that drives `fsck.exfat` over those images,
and it becomes unused once the images are removed, so we strip the
entire `tests/` directory in one shot rather than carving around it.
Stripping `tests/` is functionally inert in Azure Linux:
* `tests/` is `EXTRA_DIST`-only in upstream Makefile.am (not in
SUBDIRS), so it is never built.
* The Azure Linux spec has no `%check` block.
* `%files` does not reference `tests/`, so the tree never ships
in any binary RPM.
Changes
-------
1. `base/comps/exfatprogs/exfatprogs.comp.toml` -- new dedicated
component file with a single `[[components.exfatprogs.source-files]]`
block:
* `filename = "exfatprogs-1.3.1.tar.xz"` (matches upstream
exactly so the spec's Source0 line does not need editing).
* `hash = "<sha-512 of repacked tarball>"`.
* `origin.uri` points at the lookaside `repo` container under
the `pkgs_modified/` prefix.
* `replace-upstream = true` + `replace-reason = "..."` swap the
same-named upstream entry in the Fedora `sources` manifest
in place; no `file-remove` overlay needed.
2. `base/comps/exfatprogs/modify_source.sh` -- deterministic strip-
and-repack helper. Downloads upstream, verifies its SHA-512,
removes `tests/`, repacks with
`LC_ALL=C tar --sort=name --owner=0 --group=0 --numeric-owner
--mtime=@1577836800 --format=gnu | xz -T 1 -9`, prints the new
SHA-512 plus a ready-to-paste `az storage blob upload` command.
Output lives at `base/build/work/scratch/exfatprogs/` (covered
by the repo's top-level .gitignore via `build/`); no per-
component `.gitignore` needed.
3. `base/comps/components.toml` -- inline `[components.exfatprogs]`
replaced with a comment pointer to the new dedicated file.
4. `specs/e/exfatprogs/sources` -- regenerated by
`azldev -q comp render -p exfatprogs`. Now carries
the modified-tarball SHA-512.
5. `specs/e/exfatprogs/exfatprogs.spec` -- regenerated; the
autorelease counter advances (no functional change beyond the
automatic changelog entry).
6. `locks/exfatprogs.lock` -- refreshed input-fingerprint via
`azldev -q comp update -p exfatprogs`.
Validation
----------
- Render: clean (`STATUS: ok`).
- Lock: refreshed.
- Modified-tarball SHA-512 produced deterministically; re-running
`modify_source.sh` against the same upstream tarball yields a
byte-identical output.
- Modified tarball uploaded to the lookaside blob at the path
the comp.toml `origin.uri` references.
PawelWMS
force-pushed
the
pawelwi/exfatprogs-strip-tests
branch
from
01f71ac to
4155ca6
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Koji build
Summary
The upstream
exfatprogs-1.3.1.tar.xzships atests/tree containing 19 deliberately-corrupted exFAT filesystem images (each packaged astests/<scenario>/exfat.img.tar.xz) plus a shell-driventests/upcase_table/fixture set and atest_fsck.shdriver. The malformed images defeat the decompressor used by the automated package-signing pipeline (failure persists even on hosts with ample storage / CPU — this is an archive-shape issue, not a resource bound), which blocks signing of the SRPM.Stripping
tests/is functionally inert in Azure Linux:tests/isEXTRA_DIST-only in upstreamMakefile.am(not inSUBDIRS), so it is never built.%checkblock.%filesdoes not referencetests/, so the tree never ships in any binary RPM.Changes
base/comps/exfatprogs/exfatprogs.comp.toml(new) — dedicated component file with a single[[components.exfatprogs.source-files]]block pointing at the repacked tarball in the lookasiderepocontainer under thepkgs_modified/prefix. Uses the newreplace-upstream = true/replace-reason = "..."mechanism (azure-linux-dev-tools PR Fix kernel aarch64 package build break due to missing CONFIG_IMA_KEXEC #171) to swap the same-named upstream entry in the Fedorasourcesmanifest in place.base/comps/exfatprogs/modify_source.sh(new) — deterministic strip-and-repack helper. Downloads upstream, verifies its SHA-512, removestests/, repacks withLC_ALL=C tar --sort=name --owner=0 --group=0 --numeric-owner --mtime=@1577836800 --format=gnu | xz -T 1 -9, prints the new SHA-512 plus a ready-to-pasteaz storage blob uploadcommand. Output lives atbase/build/work/scratch/exfatprogs/(covered by the repo's top-level.gitignoreviabuild/); no per-component.gitignoreneeded.base/comps/components.toml— inline[components.exfatprogs]removed.specs/e/exfatprogs/sources— regenerated byazldev -q comp render -p exfatprogs. Carries the modified-tarball SHA-512.specs/e/exfatprogs/exfatprogs.spec— regenerated; the autorelease counter advances (no functional change beyond the automatic changelog entry).locks/exfatprogs.lock— refreshed input-fingerprint viaazldev -q comp update -p exfatprogs.Validation
STATUS: ok).modify_source.shagainst the same upstream tarball yields a byte-identical output.origin.urireferences.