[Frauschi]

Problem

A tier-E SBOM hashes the files listed as compile_commands.json entries. Some sources are compiled by being #included from another .c file, and those never appear as entries. wolfSSL has 29 of them, among them src/ssl_api_*.c, src/x509.c and, with inlining on, wolfcrypt/src/misc.c. Editing one of those changed neither the source-set hash nor the SBOM.

Changes

• compdb_sbom.py --included-sources reruns each selected C, C++ and .S compile command as <compiler> -M <flags> <source>. It adds every file the compiler read that has a source extension, lies under --root (else under the common directory of the listed sources), passes --include/--exclude and is not already listed. Headers stay out.

• Every entry for a file is rerun, since a second target can compile it with other -D flags. Files are matched by device and inode, so a case-variant #include or a hardlink is not listed twice. An included file whose basename clashes with another source is named in the error.

• The rerun keeps only the flags that affect preprocessing:

  • -D, -U, the -I family, -include, -imacros, --sysroot, --specs, -std, -m, -O;
  • -Wp, with its -M and -o parts removed;
  • the -f flags that set predefined macros.

  Every other -f flag is dropped, because several of them write files. Options that take a separate argument are kept or dropped as a pair. Preprocessor options passed through -Xclang or -Xpreprocessor (-D, -I, -include, -include-pch) are forwarded as they are, so a CMake precompiled header still applies.

• Response files are expanded, including nested ones. A missing one is an error. A ccache/sccache/distcc/icecc launcher is skipped. A universal build is scanned once per -arch, with that architecture's -Xarch_ flags, and the results are merged.

• Assembler entries are not rerun: GNU as reads -M as --mri. A -M run whose output does not list its own source stops the run, so a compiler that gives -M another meaning cannot pass silently. A compiler that fails stops the run.

• The scan lives in the new share/frontends/included_sources.py, which is imported only when the option is given, so the Zephyr frontend can reuse it.

Why opt-in

The scan executes the compilers that compile_commands.json names, and the frontend otherwise treats that file as untrusted input. The TI, IAR and CCRX compilers that compdb also serves have no -M. Without the flag, nothing changes.

Testing

• tests/test_included_sources.py has 34 tests covering the command rewrite, the -M output parser and the frontend end to end; selftest.yml runs it. They pass on macOS (Apple clang) and on Linux (gcc 13), where the case-variant test is skipped.

• On a CMake build of wolfSSL master the scan adds exactly the 29 included files to the 38 library sources in src/ and wolfcrypt/src/, in about 1.5 seconds. For the whole compile database, including examples and tests, it goes from 170 to 199 sources.

• Checked with real compilers:

  • a CMake + Ninja build with clang precompiled headers;
  • gcc-16 with -Wp,-o;
  • arm-none-eabi-gcc with --specs and .S files;
  • arm-none-eabi-as;
  • Apple clang with -undefined, -framework and SARIF output flags;
  • an Apple clang universal build (-arch x86_64 -arch arm64) whose source includes a different .c file per architecture, where both are listed.

  None of the reruns writes a file.

Notes

• Windows needs follow-up work. compdb's command tokenizer and the response-file reader use POSIX shlex, which mangles backslash paths, and some Windows shares report a zero inode. The Zephyr frontend in fix(zephyr): take module sources and settings from the west build #46 has a Windows-aware command splitter that compdb can share once fix(zephyr): take module sources and settings from the west build #46 lands.
• A Zephyr hook for --included-sources will follow fix(zephyr): take module sources and settings from the west build #46.

[Copilot]

🟡 Changes recommended

Dependency replay can silently omit compiled sources by discarding preprocessing inputs and additional architecture configurations.

3 open issues

• Preserve forwarded preprocessing inputs during dependency scanning · New
• Preserve -fabi-version in dependency scanning · New
• Scan and merge dependencies for every target architecture · New

What changed in this PR

Adds opt-in discovery of source files compiled through #include, extending the compilation-database frontend’s SBOM source coverage.

Changes:

• Adds compiler dependency scanning with command filtering and response-file expansion.
• Integrates source filtering, deduplication, and collision reporting.
• Adds documentation, tests, and CI coverage.

File	Description
tests/​test_included_sources.py	Tests command rewriting, dependency parsing, and frontend integration.
share/​README.md	Documents the new scanner module.
share/​frontends/​included_sources.py	Implements dependency scanning and command rewriting.
share/​frontends/​compdb_sbom.py	Adds --included-sources and merges discovered sources.
.github/​workflows/​selftest.yml	Includes scanner checks and tests in CI.

_{Review effort: Balanced}