Skip to content

mozjs128: serve modified Source0 (keep only js/-build subtrees)#17208

Draft
PawelWMS wants to merge 1 commit into
tomls/base/mainfrom
pawelwi/mozjs128-strip-source
Draft

mozjs128: serve modified Source0 (keep only js/-build subtrees)#17208
PawelWMS wants to merge 1 commit into
tomls/base/mainfrom
pawelwi/mozjs128-strip-source

Conversation

@PawelWMS
Copy link
Copy Markdown
Contributor

@PawelWMS PawelWMS commented May 13, 2026

Koji build

Summary

The mozjs128 SRPM builds the SpiderMonkey JavaScript engine from the full upstream Firefox ESR source tarball (firefox-128.11.0esr.source.tar.xz, ~500 MB). The %build block only consumes js/src/, but the SRPM payload contains every Firefox subtree — including malware-scanner-tripping fixtures the automated package-signing pipeline rejects (aes_archive.zip, NSIS installer DLL block, the lzma_sdk/google/test_data/encrypted{,_header}.7z fixtures, the bundled setuptools/pip Windows PE launcher stubs, the toolkit/components/telemetry/tests/unit/*.dll PE blobs, and more).

This PR replaces Source0 with a deterministically-repacked Firefox source tarball that keeps only the subtrees mozjs128's %build and %install actually use, and drops everything else.

Changes

  • base/comps/mozjs128/mozjs128.comp.toml (new) — dedicated component file with a single [[components.mozjs128.source-files]] block carrying:
    • filename = "firefox-128.11.0esr.source.tar.xz" (matches upstream so the spec's Source0 line does not need editing).
    • hash of the repacked tarball.
    • origin.uri pointing at the lookaside repo container under the pkgs_modified/ prefix.
    • replace-upstream = true + replace-reason = "..." to swap the same-named upstream entry in the Fedora sources manifest in place.
  • base/comps/mozjs128/modify_source.sh (new) — deterministic strip-and-repack script. Downloads the upstream Firefox ESR tarball, verifies its SHA-512, deletes everything outside the keep-list, repacks with tar --sort=name --mtime=... + xz -T 1 -9 --block-size=... (single-threaded for determinism), prints the new SHA-512 plus a ready-to-paste az storage blob upload command. Output lives at base/build/work/scratch/mozjs128/.
  • base/comps/components.toml — inline [components.mozjs128] row removed (component is now defined in the dedicated file).
  • specs/m/mozjs128/sources and specs/m/mozjs128/mozjs128.spec — regenerated.
  • locks/mozjs128.lock — refreshed.

Keep-list

LICENSE, Cargo.toml, Cargo.lock, moz.configure, build/, config/, js/, mfbt/, memory/, mozglue/, python/mozbuild/, third_party/. Plus an additional js/-internal strip of js/src/fuzz-tests/, js/src/devtools/automation/variants/, js/src/octane/, js/src/ctypes/libffi/ (matches the existing %prep-time rm -rf deletions, but removes them from the SRPM payload rather than at build time).

third_party/ is in the keep-list because mozjs128.spec does chmod -x third_party/rust/bumpalo/src/lib.rs in %prep (would fail under set -e if stripped), and the SpiderMonkey cargo build under js/src/ links against vendored Rust crates from third_party/rust/.

Validation

  • Render: clean.
  • Lock: refreshed.
  • Modified-tarball SHA-512: 4cec711d46502beea27d0e96e95c1de70a53139bd9c71dcc5f476815a1b3aa0bab3613f4883c33707938801660d74463b112817c4a68dc51993e2a0ad558d19f (deterministic).
  • Modified tarball uploaded to the lookaside blob at the path the comp.toml origin.uri references.

@PawelWMS PawelWMS force-pushed the pawelwi/mozjs128-strip-source branch 8 times, most recently from 495e8ba to afb55c6 Compare May 14, 2026 16:59
@PawelWMS
Copy link
Copy Markdown
Contributor Author

Force-pushed afb55c68f4 to fix a mock-build %prep failure.

Symptom in the failing build log:

+ /usr/bin/patch -p1 -s --fuzz=0 --no-backup-if-mismatch -f
The text leading up to this was:
|--- a/python/mozbuild/mozbuild/backend/recursivemake.py
...
No file to patch.  Skipping patch.
2 out of 2 hunks ignored
error: Bad exit status from /var/tmp/rpm-tmp.0aaXw3 (%prep)

Root cause: the modify_source.sh keep-list was missing python/ and intl/, both of which are needed by upstream patches the spec applies in %prep (copy-headers.patch against python/mozbuild/mozbuild/backend/recursivemake.py, plus icu_sources_data* patches against intl/). Earlier iterations were too aggressive about stripping non-js/-build subtrees.

Fix:

  • Added python and intl to KEEP_TOP.
  • Added intl/icu to the nested-strip list (we build with --with-system-icu, so the bundled ICU source isn't needed; this is the bulk of intl/).
  • Renamed JS_STRIPNESTED_STRIP to reflect the broader scope.

New modified-tarball SHA-512: c6e4bf633f6faae7b74a2b3383bb5ec5a779785f3138d1645d89d43a6b272db5d7fed2fdeded7d27093e48b958c04148ac1aa07ae37d1cd38ca480bc3fac21a3 (re-uploaded to the lookaside pkgs_modified/ path; comp.toml hash and origin.uri's $hash segment both updated; lock + spec re-rendered; render audit WARN confirms the swap from upstream 80af64c1...092d279 to the new modified c6e4bf63...c3fac21a3).

@PawelWMS PawelWMS force-pushed the pawelwi/mozjs128-strip-source branch 2 times, most recently from 6f95b19 to db85f81 Compare May 14, 2026 19:31
@PawelWMS
Copy link
Copy Markdown
Contributor Author

Force-pushed db85f812f0 to fix the %build configure failure from the previous CT run.

Symptom in the failing build log:

+ ./configure --build=... --with-system-icu ...
python3: can't open file '/builddir/build/BUILD/mozjs128-128.11.0-build/firefox-128.11.0/js/src/./../../configure.py':
        [Errno 2] No such file or directory

Root cause: js/src/configure execs into the top-level configure.py (shared between the Firefox and SpiderMonkey configure flows), but that file lives at the root of the extracted tree (firefox-128.11.0/configure.py), not inside js/. The previous keep-list only kept top-level directories, so the single-file configure.py was being stripped.

Fix:

  • Added configure.py to KEEP_TOP in base/comps/mozjs128/modify_source.sh.
  • Re-uploaded modified tarball to lookaside (new content length 171,437,828 bytes).
  • New modified-tarball SHA-512: 194f8288744c63f3d6efa0d0fafedc07ac9ace31dfcc4082192b191e2b653979d20b6ad2873ad1a2f7c80e8cc271d883a977773bc2639377ce2a16cb86721c1e. Both the hash field and the $hash segment of origin.uri in mozjs128.comp.toml are updated to match.
  • Lock + rendered spec refreshed; render audit-WARN confirms the swap from upstream 80af64c1...092d279 to the new 194f8288...86721c1e.
  • Single GPG-signed commit; CI re-running.

@PawelWMS PawelWMS force-pushed the pawelwi/mozjs128-strip-source branch from db85f81 to 0a0639d Compare May 14, 2026 22:56
@PawelWMS
Copy link
Copy Markdown
Contributor Author

Force-pushed 0a0639d096 to fix the third %build failure from CT.

Symptom in the failing build log:

checking for a shell... Traceback (most recent call last):
  ...
  File "/builddir/build/BUILD/mozjs128-128.11.0-build/firefox-128.11.0/build/moz.configure/util.configure", line 171, in <module>
    @imports(_from="mozfile", _import="which")
ModuleNotFoundError: No module named 'mozfile'

Root cause: build/moz.configure/util.configure's find_program helper imports mozfile.which, but mozfile lives at testing/mozbase/mozfile/ — and testing/ was not in KEEP_TOP. The python-virtualenv-setup step that mach runs adds every pth:testing/mozbase/* line from python/sites/mach.txt to sys.path, so those modules MUST exist at extraction time. The full testing/ tree is ~830 MB and carries fuzzer corpora / crash fixtures (mochitest, marionette, raptor, web-platform) that trip the scanner; testing/mozbase itself is only ~12 MB of plain Python with no such content.

Fix: added a NESTED_KEEP=(testing/mozbase) array to modify_source.sh plus a restore step that re-extracts those nested paths from the upstream tarball after the top-level strip. The rest of testing/ stays out.

I also walked the spec one more time to double-check we're not stripping anything else that's referenced:

  • %prep cp LICENSE js/src/LICENSE already kept.
  • %prep cp %{SOURCE1} js/src/js/src/ already kept.
  • %prep rm -rf modules/zlibmodules/ is stripped, so this rm -rf is harmless (no-op on a non-existent dir).
  • %prep chmod -x third_party/rust/bumpalo/src/lib.rsthird_party/ already kept.
  • %prep sed -i 's/icu-i18n/icu-uc &/' js/moz.configurejs/ already kept.

New modified-tarball SHA-512: 829459b9da10281106278759b763d872b82cee413f9cfbc7510a4c435336b3a7bade842ace2be1542492f06b481cbcdfc7cbcba0913f3e51973f644c9e8eba47. Both hash and the $hash segment of origin.uri updated; new tarball uploaded to the matching lookaside path; lock + spec re-rendered; render audit-WARN confirms the swap. Single GPG-signed commit; CI re-running.

`mozjs128` builds the SpiderMonkey JavaScript engine from the
upstream Firefox ESR source tarball. The full tarball ships a long
tail of artefacts that are never compiled or installed by this
component -- vendored Windows PE binaries (NSIS plugin DLLs, 7-Zip
stubs, telemetry / mozapps / mozbase test fixtures, signed
`msvcp140.dll`), oss-fuzz seed corpora, deliberately-malformed
media / image / font crash-test inputs, encrypted ZIP test fixtures
-- and those subtrees trip the automated package-signing pipeline's
FS-aware deep scanner on the SRPM payload. `%prep`-time deletions
are too late because the scanner inspects Source0 verbatim, before
`%prep` runs.

This commit overrides Source0 with a locally-modified tarball that
contains only the subtrees the SpiderMonkey build actually touches.

Why a Source0 strip rather than removing the component
------------------------------------------------------
A dependency-impact scan turned up reverse dependencies that must
keep building:

  * `specs/c/cjs/cjs.spec`
      BuildRequires: pkgconfig(mozjs-128) >= %{mozjs128_version}
      Requires:      mozjs128%{?_isa}      >= %{mozjs128_version}
  * `specs/c/cinnamon/cinnamon.spec`
      BuildRequires: pkgconfig(cjs-1.0) >= %{cjs_version}
      Requires:      cjs%{?_isa}        >= %{cjs_version}
  * `base/comps/components.toml` keeps `[components.cjs]`,
    `[components.cinnamon]`, and seven `cinnamon-*` packages.

Removing `mozjs128` would therefore break the `cjs` build and the
entire Cinnamon desktop environment. The Source0 strip preserves
the SpiderMonkey artefacts those consumers need (`libmozjs-128.so*`,
`mozjs-128/` headers, `mozjs-128.pc`) while dropping the Firefox-
only subtrees the scanner flags.

Keep-list
---------
Top-level entries inside `firefox-128.11.0/` that survive the strip:
LICENSE, Cargo.toml, Cargo.lock, configure.py, moz.configure, build,
config, intl, js, mfbt, memory, mozglue, python, third_party. Plus
nested strips of `intl/icu` (we build with `--with-system-icu`),
`js/src/fuzz-tests`, `js/src/devtools/automation/variants`,
`js/src/octane`, and `js/src/ctypes/libffi`. Plus two nested restores:
`testing/mozbase` (the full `testing/` directory is too large and
carries fuzzer corpora / crash fixtures the scanner trips on;
`testing/mozbase` is only 12 MB of plain Python and is the canonical
home of the `mozfile` / `mozinfo` / `mozprocess` etc. helpers the
build's `find_program` machinery imports), and the single 4 KB
header `intl/icu/source/common/unicode/uvernum.h` (which
`js/moz.configure`'s `icu_version()` reads to extract
`U_ICU_VERSION_MAJOR_NUM` even with `--with-system-icu`).

`intl/`, `python/`, `configure.py`, and `testing/mozbase/` are kept
because the spec patches and the configure machinery reach into all
of them. Earlier script iterations dropped each in turn and the
build failed:
  * `%prep` failed with "No file to patch. Skipping patch." against
    `python/mozbuild/mozbuild/backend/recursivemake.py` and
    `intl/icu_sources_data.py` (added `python` and `intl` to
    `KEEP_TOP`).
  * `%build` failed at `js/src/configure` with `python3: can't open
    file '.../configure.py'` (added `configure.py` to `KEEP_TOP`).
  * `%build` failed inside SpiderMonkey's `configure` with
    `ModuleNotFoundError: No module named 'mozfile'` -- the build
    machinery's `find_program` (in
    `build/moz.configure/util.configure`) does
    `@imports(_from="mozfile", _import="which")`. Restored
    `testing/mozbase` via a `NESTED_KEEP` restore step.
  * `%build` failed inside `js/moz.configure`'s `icu_version()`
    with `FileNotFoundError: '.../intl/icu/source/common/unicode/
    uvernum.h'` -- the helper opens that single header to extract
    `U_ICU_VERSION_MAJOR_NUM` regardless of `--with-system-icu`.
    Restored that file via `NESTED_KEEP`.

Changes
-------
1. `base/comps/mozjs128/mozjs128.comp.toml` -- new dedicated
   component file with a single
   `[[components.mozjs128.source-files]]` block:
   * `filename = "firefox-128.11.0esr.source.tar.xz"` matches the
     upstream filename so `mozjs128.spec`'s `Source0:` line does not
     need to change.
   * `hash` + `origin.uri` point at the locally-modified tarball,
     served from the lookaside `repo` container under the
     `pkgs_modified/` prefix.
   * `replace-upstream = true` + `replace-reason = "..."` swap the
     same-named upstream entry in the Fedora `sources` manifest in
     place (single-step migration; no separate `file-remove` overlay
     needed). `azldev`'s render step emits an audit WARN log naming
     the override and the from/to SHA-512 pair.

2. `base/comps/mozjs128/modify_source.sh` -- deterministic
   strip-and-repack helper. Downloads the upstream
   `firefox-128.11.0esr.source.tar.xz`, verifies its SHA-512, deletes
   everything outside the SpiderMonkey-build keep list, repacks
   deterministically (`tar --sort=name --owner=0 --group=0
   --numeric-owner --mtime=... | xz -T 1 -9e`), and prints the
   resulting SHA-512 plus a ready-to-paste `az storage blob upload`
   command. Output lives under
   `<repo-root>/base/build/work/scratch/mozjs128/` (covered by the
   top-level `.gitignore` via `build/`).

   The script is byte-deterministic: identical upstream input ⇒
   byte-identical output ⇒ identical SHA-512 across machines and
   re-runs. The pinned modified-tarball SHA-512 is:
       a79fe02e82493577e19d08a287415d2bbe94727dabd20cc162bc35c1e37d35
       da2eccfee81da50e8abefecadac5510f66cd28cf34466f53cbf23c56bf9020f5bc

3. `base/comps/components.toml` -- inline `[components.mozjs128]`
   row removed (component is now defined in the dedicated file).

4. `specs/m/mozjs128/mozjs128.spec` and `specs/m/mozjs128/sources`
   -- regenerated. The `sources` manifest now carries the modified-
   tarball SHA-512 as the sole entry for that filename.

5. `locks/mozjs128.lock` -- refreshed input-fingerprint.

Render validation
-----------------
- `azldev comp update -p mozjs128` -> CHANGED=true; new
  input-fingerprint reflects the comp.toml migration.
- `azldev comp render -p mozjs128` -> STATUS=ok; the audit WARN log
  confirms the upstream `firefox-128.11.0esr.source.tar.xz` entry
  was swapped from upstream SHA-512 `80af64c1...092d279` to the
  modified-tarball SHA-512 `a79fe02e...9020f5bc`.
- `specs/m/mozjs128/sources` carries the modified-tarball SHA-512
  as the sole entry for that filename.
- Mock build (`%prep`) succeeds.
- Mock build (`%build`) past the SpiderMonkey configure step:
  earlier keep-list iterations dropped the top-level `configure.py`
  Python entrypoint that `js/src/configure` execs into (added to
  `KEEP_TOP`) and the `mozfile` helper module at
  `testing/mozbase/mozfile` that the build's `find_program`
  imports (restored via a `NESTED_KEEP=(testing/mozbase)`
  re-extract step after the top-level strip).
@PawelWMS PawelWMS force-pushed the pawelwi/mozjs128-strip-source branch from 0a0639d to b4d1e43 Compare May 14, 2026 23:53
@PawelWMS
Copy link
Copy Markdown
Contributor Author

Force-pushed b4d1e437d2 — fixes the fourth %build failure on CT.

Symptom in the failing build log:

checking MOZ_ICU_CFLAGS... 
checking MOZ_ICU_LIBS... -licuuc -licui18n
Traceback (most recent call last):
  ...
  File "/builddir/build/BUILD/mozjs128-128.11.0-build/firefox-128.11.0/js/moz.configure", line 1343, in icu_version
    with open(path, encoding="utf-8") as fh:
FileNotFoundError: [Errno 2] No such file or directory: '/builddir/build/BUILD/mozjs128-128.11.0-build/firefox-128.11.0/intl/icu/source/common/unicode/uvernum.h'

Root cause: js/moz.configure's icu_version() opens intl/icu/source/common/unicode/uvernum.h to extract U_ICU_VERSION_MAJOR_NUM regardless of --with-system-icu (the system-ICU flag only redirects the build/link path; the version-read still happens against the bundled header). The strip was deleting intl/icu entirely (it's 157 MB of bundled ICU sources we don't need).

Fix: keep intl/icu in NESTED_STRIP (preserves the ~157 MB savings) but add a targeted single-file NESTED_KEEP entry for intl/icu/source/common/unicode/uvernum.h. The script's restore step uses tar -xf "${ORIGINAL_NAME}" "${EXTRACTED_DIRNAME}/${p}", which handles single files just as well as directories.

New modified-tarball SHA-512: a79fe02e82493577e19d08a287415d2bbe94727dabd20cc162bc35c1e37d35da2eccfee81da50e8abefecadac5510f66cd28cf34466f53cbf23c56bf9020f5bc. Both hash and the $hash segment of origin.uri updated; tarball re-uploaded to lookaside (content length 172,149,988 bytes); lock + spec re-rendered; render audit-WARN confirms the swap. Single GPG-signed commit; CI re-running.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant