mozjs128: serve modified Source0 (keep only js/-build subtrees)#17208
mozjs128: serve modified Source0 (keep only js/-build subtrees)#17208PawelWMS wants to merge 1 commit into
Conversation
495e8ba to
afb55c6
Compare
|
Force-pushed Symptom in the failing build log: Root cause: the modify_source.sh keep-list was missing Fix:
New modified-tarball SHA-512: |
6f95b19 to
db85f81
Compare
|
Force-pushed Symptom in the failing build log: Root cause: Fix:
|
db85f81 to
0a0639d
Compare
|
Force-pushed Symptom in the failing build log: Root cause: Fix: added a I also walked the spec one more time to double-check we're not stripping anything else that's referenced:
New modified-tarball SHA-512: |
`mozjs128` builds the SpiderMonkey JavaScript engine from the
upstream Firefox ESR source tarball. The full tarball ships a long
tail of artefacts that are never compiled or installed by this
component -- vendored Windows PE binaries (NSIS plugin DLLs, 7-Zip
stubs, telemetry / mozapps / mozbase test fixtures, signed
`msvcp140.dll`), oss-fuzz seed corpora, deliberately-malformed
media / image / font crash-test inputs, encrypted ZIP test fixtures
-- and those subtrees trip the automated package-signing pipeline's
FS-aware deep scanner on the SRPM payload. `%prep`-time deletions
are too late because the scanner inspects Source0 verbatim, before
`%prep` runs.
This commit overrides Source0 with a locally-modified tarball that
contains only the subtrees the SpiderMonkey build actually touches.
Why a Source0 strip rather than removing the component
------------------------------------------------------
A dependency-impact scan turned up reverse dependencies that must
keep building:
* `specs/c/cjs/cjs.spec`
BuildRequires: pkgconfig(mozjs-128) >= %{mozjs128_version}
Requires: mozjs128%{?_isa} >= %{mozjs128_version}
* `specs/c/cinnamon/cinnamon.spec`
BuildRequires: pkgconfig(cjs-1.0) >= %{cjs_version}
Requires: cjs%{?_isa} >= %{cjs_version}
* `base/comps/components.toml` keeps `[components.cjs]`,
`[components.cinnamon]`, and seven `cinnamon-*` packages.
Removing `mozjs128` would therefore break the `cjs` build and the
entire Cinnamon desktop environment. The Source0 strip preserves
the SpiderMonkey artefacts those consumers need (`libmozjs-128.so*`,
`mozjs-128/` headers, `mozjs-128.pc`) while dropping the Firefox-
only subtrees the scanner flags.
Keep-list
---------
Top-level entries inside `firefox-128.11.0/` that survive the strip:
LICENSE, Cargo.toml, Cargo.lock, configure.py, moz.configure, build,
config, intl, js, mfbt, memory, mozglue, python, third_party. Plus
nested strips of `intl/icu` (we build with `--with-system-icu`),
`js/src/fuzz-tests`, `js/src/devtools/automation/variants`,
`js/src/octane`, and `js/src/ctypes/libffi`. Plus two nested restores:
`testing/mozbase` (the full `testing/` directory is too large and
carries fuzzer corpora / crash fixtures the scanner trips on;
`testing/mozbase` is only 12 MB of plain Python and is the canonical
home of the `mozfile` / `mozinfo` / `mozprocess` etc. helpers the
build's `find_program` machinery imports), and the single 4 KB
header `intl/icu/source/common/unicode/uvernum.h` (which
`js/moz.configure`'s `icu_version()` reads to extract
`U_ICU_VERSION_MAJOR_NUM` even with `--with-system-icu`).
`intl/`, `python/`, `configure.py`, and `testing/mozbase/` are kept
because the spec patches and the configure machinery reach into all
of them. Earlier script iterations dropped each in turn and the
build failed:
* `%prep` failed with "No file to patch. Skipping patch." against
`python/mozbuild/mozbuild/backend/recursivemake.py` and
`intl/icu_sources_data.py` (added `python` and `intl` to
`KEEP_TOP`).
* `%build` failed at `js/src/configure` with `python3: can't open
file '.../configure.py'` (added `configure.py` to `KEEP_TOP`).
* `%build` failed inside SpiderMonkey's `configure` with
`ModuleNotFoundError: No module named 'mozfile'` -- the build
machinery's `find_program` (in
`build/moz.configure/util.configure`) does
`@imports(_from="mozfile", _import="which")`. Restored
`testing/mozbase` via a `NESTED_KEEP` restore step.
* `%build` failed inside `js/moz.configure`'s `icu_version()`
with `FileNotFoundError: '.../intl/icu/source/common/unicode/
uvernum.h'` -- the helper opens that single header to extract
`U_ICU_VERSION_MAJOR_NUM` regardless of `--with-system-icu`.
Restored that file via `NESTED_KEEP`.
Changes
-------
1. `base/comps/mozjs128/mozjs128.comp.toml` -- new dedicated
component file with a single
`[[components.mozjs128.source-files]]` block:
* `filename = "firefox-128.11.0esr.source.tar.xz"` matches the
upstream filename so `mozjs128.spec`'s `Source0:` line does not
need to change.
* `hash` + `origin.uri` point at the locally-modified tarball,
served from the lookaside `repo` container under the
`pkgs_modified/` prefix.
* `replace-upstream = true` + `replace-reason = "..."` swap the
same-named upstream entry in the Fedora `sources` manifest in
place (single-step migration; no separate `file-remove` overlay
needed). `azldev`'s render step emits an audit WARN log naming
the override and the from/to SHA-512 pair.
2. `base/comps/mozjs128/modify_source.sh` -- deterministic
strip-and-repack helper. Downloads the upstream
`firefox-128.11.0esr.source.tar.xz`, verifies its SHA-512, deletes
everything outside the SpiderMonkey-build keep list, repacks
deterministically (`tar --sort=name --owner=0 --group=0
--numeric-owner --mtime=... | xz -T 1 -9e`), and prints the
resulting SHA-512 plus a ready-to-paste `az storage blob upload`
command. Output lives under
`<repo-root>/base/build/work/scratch/mozjs128/` (covered by the
top-level `.gitignore` via `build/`).
The script is byte-deterministic: identical upstream input ⇒
byte-identical output ⇒ identical SHA-512 across machines and
re-runs. The pinned modified-tarball SHA-512 is:
a79fe02e82493577e19d08a287415d2bbe94727dabd20cc162bc35c1e37d35
da2eccfee81da50e8abefecadac5510f66cd28cf34466f53cbf23c56bf9020f5bc
3. `base/comps/components.toml` -- inline `[components.mozjs128]`
row removed (component is now defined in the dedicated file).
4. `specs/m/mozjs128/mozjs128.spec` and `specs/m/mozjs128/sources`
-- regenerated. The `sources` manifest now carries the modified-
tarball SHA-512 as the sole entry for that filename.
5. `locks/mozjs128.lock` -- refreshed input-fingerprint.
Render validation
-----------------
- `azldev comp update -p mozjs128` -> CHANGED=true; new
input-fingerprint reflects the comp.toml migration.
- `azldev comp render -p mozjs128` -> STATUS=ok; the audit WARN log
confirms the upstream `firefox-128.11.0esr.source.tar.xz` entry
was swapped from upstream SHA-512 `80af64c1...092d279` to the
modified-tarball SHA-512 `a79fe02e...9020f5bc`.
- `specs/m/mozjs128/sources` carries the modified-tarball SHA-512
as the sole entry for that filename.
- Mock build (`%prep`) succeeds.
- Mock build (`%build`) past the SpiderMonkey configure step:
earlier keep-list iterations dropped the top-level `configure.py`
Python entrypoint that `js/src/configure` execs into (added to
`KEEP_TOP`) and the `mozfile` helper module at
`testing/mozbase/mozfile` that the build's `find_program`
imports (restored via a `NESTED_KEEP=(testing/mozbase)`
re-extract step after the top-level strip).
0a0639d to
b4d1e43
Compare
|
Force-pushed Symptom in the failing build log: Root cause: Fix: keep New modified-tarball SHA-512: |
Koji build
Summary
The
mozjs128SRPM builds the SpiderMonkey JavaScript engine from the full upstream Firefox ESR source tarball (firefox-128.11.0esr.source.tar.xz, ~500 MB). The%buildblock only consumesjs/src/, but the SRPM payload contains every Firefox subtree — including malware-scanner-tripping fixtures the automated package-signing pipeline rejects (aes_archive.zip, NSIS installer DLL block, thelzma_sdk/google/test_data/encrypted{,_header}.7zfixtures, the bundledsetuptools/pipWindows PE launcher stubs, thetoolkit/components/telemetry/tests/unit/*.dllPE blobs, and more).This PR replaces Source0 with a deterministically-repacked Firefox source tarball that keeps only the subtrees
mozjs128's%buildand%installactually use, and drops everything else.Changes
base/comps/mozjs128/mozjs128.comp.toml(new) — dedicated component file with a single[[components.mozjs128.source-files]]block carrying:filename = "firefox-128.11.0esr.source.tar.xz"(matches upstream so the spec's Source0 line does not need editing).hashof the repacked tarball.origin.uripointing at the lookasiderepocontainer under thepkgs_modified/prefix.replace-upstream = true+replace-reason = "..."to swap the same-named upstream entry in the Fedorasourcesmanifest in place.base/comps/mozjs128/modify_source.sh(new) — deterministic strip-and-repack script. Downloads the upstream Firefox ESR tarball, verifies its SHA-512, deletes everything outside the keep-list, repacks withtar --sort=name --mtime=...+xz -T 1 -9 --block-size=...(single-threaded for determinism), prints the new SHA-512 plus a ready-to-pasteaz storage blob uploadcommand. Output lives atbase/build/work/scratch/mozjs128/.base/comps/components.toml— inline[components.mozjs128]row removed (component is now defined in the dedicated file).specs/m/mozjs128/sourcesandspecs/m/mozjs128/mozjs128.spec— regenerated.locks/mozjs128.lock— refreshed.Keep-list
LICENSE,Cargo.toml,Cargo.lock,moz.configure,build/,config/,js/,mfbt/,memory/,mozglue/,python/mozbuild/,third_party/. Plus an additionaljs/-internal strip ofjs/src/fuzz-tests/,js/src/devtools/automation/variants/,js/src/octane/,js/src/ctypes/libffi/(matches the existing%prep-timerm -rfdeletions, but removes them from the SRPM payload rather than at build time).third_party/is in the keep-list becausemozjs128.specdoeschmod -x third_party/rust/bumpalo/src/lib.rsin%prep(would fail underset -eif stripped), and the SpiderMonkey cargo build underjs/src/links against vendored Rust crates fromthird_party/rust/.Validation
4cec711d46502beea27d0e96e95c1de70a53139bd9c71dcc5f476815a1b3aa0bab3613f4883c33707938801660d74463b112817c4a68dc51993e2a0ad558d19f(deterministic).origin.urireferences.