Skip to content

fix(zephyr): take module sources and settings from the west build - #46

Open
Frauschi wants to merge 1 commit into
masterfrom
fix/zephyr-sources-ifdef
Open

Frauschi wants to merge 1 commit into
masterfrom
fix/zephyr-sources-ifdef

Conversation

@Frauschi

Copy link
Copy Markdown
Member

Problem

zephyr_sbom.py found sources with a regex over zephyr_library_sources(...). It ignored zephyr_library_sources_ifdef() and Kconfig if() blocks, parsed commented-out calls and skipped unknown variables silently, so a wolfCert SBOM listed the EST and SCEP clients whether or not they were built. wolfssl's module could not be read at all, because its source list is built from FIPS-boundary lists, a foreach and IN_LIST checks.

Changes

  • The frontend reads the west build's compile_commands.json (--compdb, --root <module>). The module's sources are the entries under --root whose object lands in <build>/modules/<name>/; the name comes from zephyr/module.yml, else the directory name (--module-name overrides). Application files inside the module tree, such as wolfCert's test app, are left out because they compile into app.dir.
  • Reading the CMakeLists without a build is removed. A .config only exists in a configured build, which already has compile_commands.json, and a static reader cannot follow list(APPEND), include() and the like. No product runs the frontend today.
  • --user-settings <module>/zephyr/user_settings.h records the module's configuration: its -D flags plus the macros the settings header defines. The build's own compiler and target flags preprocess it with and without those inputs, and only the difference is kept, so compiler built-ins and Kconfig symbols drop out. The -D flags of all the module's C files are merged; files that disagree on a macro, or a header that adds no macros (for example because a copy of it is already force-included), stop the run. --hostcc switches to a host compiler.
  • The licence, supplier, component-type and --dep-* flags are forwarded to the driver, and --options-h can replace --source-only.
  • Docstring and both READMEs describe the routes: the in-module CMake helper (called after the last zephyr_library_sources()), else this frontend.

Testing

  • tests/test_zephyr_sbom.py (26 tests) covers the module name, the compile_commands.json selection, the settings capture in both compiler modes, the error paths and flag forwarding; selftest.yml runs it.
  • Checked on real west builds (Zephyr 4.4.1): wolfCert's unit smoke test on qemu_x86 with EST, with SCEP and with CONFIG_WOLFSSL_DEBUG, and wolfssl's wolfssl_test sample on qemu_x86 and on nucleo_h743zi with CONFIG_WOLFCRYPT_ASM. The source sets matched the compiled sets every time (wolfssl 101 files, 109 with the Cortex-M assembly; wolfCert 14). The settings capture matched one taken by hand with the target compiler, and records DEBUG_WOLFSSL when set. The SBOMs validate, contain no host paths and are byte-reproducible.

Notes

  • Zephyr-wide -D flags such as KERNEL and K_HEAP_MEM_POOL_SIZE appear in the recorded configuration. Filtering them would also drop module flags set with zephyr_compile_definitions().
  • Only the module's settings header is captured, not what settings.h derives from it.

@Frauschi Frauschi self-assigned this Oct 20, 2026
Copilot AI balanced review requested due to automatic review settings October 20, 2026 12:33
@Frauschi Frauschi assigned wolfSSL-Bot and unassigned Frauschi Oct 20, 2026
@Frauschi
Frauschi requested review from sameehj and removed request for Copilot October 20, 2026 12:36
@Frauschi
Frauschi force-pushed the fix/zephyr-sources-ifdef branch from cc605bd to 9a79baa Compare October 22, 2026 10:53
Copilot AI balanced review requested due to automatic review settings October 22, 2026 10:53

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Source selection and command handling can reject valid build entries, hide configuration conflicts, or fail settings capture.

3 open issues
What changed in this PR

This PR replaces static CMake parsing in the Zephyr SBOM frontend with compilation-database extraction so module inventories reflect the configured west build.

Changes:

  • Selects module sources from build outputs while excluding application sources.
  • Adds settings-header capture and forwards configuration, licensing, and dependency options.
  • Adds frontend tests, CI execution, and updated integration guidance.
File Description
tests/​test_zephyr_sbom.py Adds source-selection, settings-capture, error-path, and forwarding tests.
share/​README.md Documents Zephyr integration routes and settings capture.
share/​frontends/​zephyr_sbom.py Replaces static parsing with build-derived source and configuration capture.
README.md Updates the Zephyr frontend description.
.github/​workflows/​selftest.yml Adds Zephyr tests to CI checks and execution.

Review effort: Balanced

Comment thread share/frontends/zephyr_sbom.py
Comment thread share/frontends/zephyr_sbom.py Outdated
Comment thread share/frontends/zephyr_sbom.py
@Frauschi
Frauschi force-pushed the fix/zephyr-sources-ifdef branch 2 times, most recently from c49f08b to c61d8d8 Compare October 22, 2026 13:11

@sameehj sameehj left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, just rebase please so we can merge. Thank you Tobias

zephyr_sbom.py found sources with a regex over zephyr_library_sources(...).
It missed zephyr_library_sources_ifdef(), listed sources inside if(CONFIG_X)
blocks whatever Kconfig said, parsed commented-out calls, and silently
skipped any ${VAR} other than the three module-dir variables. For a module
like wolfCert that meant an SBOM listing the EST and SCEP clients whether or
not they were built. wolfssl's module could not be read at all: its source
list comes from FIPS-boundary lists, a foreach and IN_LIST checks.

The frontend now reads the compile_commands.json CMake writes when west
configures the build (--compdb, with --root set to the module). The module's
sources are the entries under --root whose object Zephyr placed in the
module's library directory, <build>/modules/<name>/. The name comes from
zephyr/module.yml, else the module directory name, and --module-name
overrides it. This follows Kconfig and whatever CMake logic the module uses,
and it leaves out application files inside the module tree, such as
wolfCert's test app, because those compile into app.dir. Library sources
outside --root are reported and excluded. It needs the "output" field CMake
3.20 and later write.

Reading the CMakeLists without a build is gone. A .config only exists in a
configured build, which already has compile_commands.json, and without one
any Kconfig-gated source had to be refused. A static reader also cannot
follow list(APPEND), include() or similar, and returned wrong lists for them.
No product runs the frontend today; wolfCert and wolfTPM only ship it.

--user-settings records the module's configuration: the -D flags of its
compile command plus the macros its settings header defines. It runs the
compile command's own compiler and target flags with -dM -E twice, once
without the -D flags and the header and once with -D flags and
-include of the header, and keeps what changed. Compiler built-ins and the
Kconfig symbols from autoconf.h cancel out. The build's compiler is used
because the host one answers target checks wrongly: wolfssl's Zephyr
user_settings.h tests __SIZEOF_INT128__ under CONFIG_WOLFCRYPT_ASM, and an
aarch64 host recorded HAVE___UINT128_T for a Cortex-M7 build. --hostcc
switches to a host compiler with only the -D, -U, -I, -iquote and autoconf.h
flags. Zephyr-wide -D flags such as KERNEL are recorded as well; filtering
them out would also drop module flags set with zephyr_compile_definitions().
The -D and -U flags of every C compile command in the module are merged,
including a file built by two targets; a macro that one command defines and
another undefines or defines differently stops the run. A settings header
the command already force-includes is taken out of the first run so its
macros still show, and a header that adds no macros at all, for example
because a copy of it is force-included, stops the run.
Response files are refused, and "command" strings are split with the
Windows rules on Windows.

Checked on west builds (Zephyr 4.4.1) of wolfCert's unit smoke test on
qemu_x86 with EST, with SCEP and with CONFIG_WOLFSSL_DEBUG, and of wolfssl's
wolfssl_test sample on qemu_x86 and on nucleo_h743zi with
CONFIG_WOLFCRYPT_ASM. The source sets matched the compiled sets every time
(wolfssl 101 files, 109 with the Cortex-M assembly; wolfCert 14). The
settings capture equalled one taken by hand with the target compiler and
recorded DEBUG_WOLFSSL when set. The SBOMs validate, contain no host paths
and are byte-reproducible.

The frontend also forwards --license-override, --license-text, --supplier,
--component-type, --dep-wolfssl, --dep-wolfcrypt, --dep-openssl and
--dep-version, and accepts --options-h in place of --source-only. Before,
these SBOMs fell back to the inferred licence (GPL-3.0-only for a verbatim
GPLv3 file) and recorded no wolfSSL dependency.

The docstring and both READMEs describe the CMake helper, called with
TARGETS ${ZEPHYR_CURRENT_LIBRARY} after the module's last
zephyr_library_sources(), as the accurate route and this frontend as the one
without a hook.

New tests/test_zephyr_sbom.py covers the module name, the
compile_commands.json selection, the settings capture in both compiler
modes, the error paths and flag forwarding through a stub driver;
selftest.yml runs it.
@Frauschi
Frauschi force-pushed the fix/zephyr-sources-ifdef branch from c61d8d8 to 953f2ad Compare October 22, 2026 14:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants