Repository navigation
doc: publish the nvx crate's module diagrams and keep them current - #930
Merged
Pedro Henrique Penna (ppenna) merged 6 commits intoOct 27, 2026
Merged
Conversation
The nvx crate holds more than 200 modules across the host, the guest, and the code that they share, and nothing showed how they fit together. module_diagrams.py draws them with cargo-modules and Graphviz. It exports the modules of the crate's library, with its cli, testing, and guest features, for x86_64-unknown-linux-musl, the guests' target, and for x86_64-pc-windows-msvc, and merges the two exports into one set of diagrams: the top-level modules, with what each holds and how many modules it contains; the dependencies between top-level modules, counted by the pairs of their modules in which one uses the other; and the hierarchy of each top-level module that contains modules. A module that only one platform builds, such as the guest module or the host's sys module on Linux, names that platform, while the color of a box shows the part of the crate that holds the module and its border the module's visibility. It writes the diagrams, as SVG images, with the Markdown page that shows them, to build/modules, or to the directory that --output names. It replaces the PowerShell driver, scripts/module-diagrams.ps1, and the interactive atlas, scripts/module_atlas.py, of the wip/module-diagrams branch. A Markdown page of SVG images shows on GitHub, where the atlas's HTML page shows only as source, so the documentation can publish it; the page keeps the atlas's diagrams, colors, and descriptions. The script reads its targets from the build constants, which own the target triples, and refuses an export that holds anything but modules, their containment, and their uses, so that a change in cargo-modules' output cannot draw wrong diagrams silently. It replaces an output directory only when the directory holds nothing but the page and the diagrams of an earlier run, whose first line marks the page, and never writes through a symbolic link. When cargo-modules is missing, it names the release to install, 0.27.0, which builds with the Rust release that rust-toolchain.toml pins. Unit tests cover how it reads exports, written here in cargo-modules' DOT form, how it merges the platforms' modules, the graphs and the page that it draws, how it replaces an earlier run's output, and the commands that a drawing runs, with a stand-in for cargo-modules and Graphviz. validate-nvx runs them on Linux and Windows. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
doc/modules publishes the module diagrams of the nvx crate that scripts/module_diagrams.py draws: the overview of its top-level modules, the dependencies between them, and the hierarchies of the eight top-level modules that contain modules, as SVG images on a Markdown page that GitHub renders. The project README lists the page beside the project structure, and the crate's README and the project structure link to it. They were drawn on Ubuntu 24.04 with cargo-modules 0.27.0, built with the Rust release that rust-toolchain.toml pins, and Graphviz 2.43.0, which measures their labels with Liberation Sans, the first font that they name. Drawing them again there writes the same bytes, while other Graphviz releases and fonts lay them out differently. A test checks that doc/modules holds the page, whose first line marks it as the script's, and exactly the diagrams that the page shows, each an SVG document, so that no diagram that the page stopped showing lingers and none that it shows is missing. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The module diagrams that doc/modules publishes go stale as soon as the nvx crate changes. The module diagrams workflow draws them again after every push to dev that changes their inputs, nvx-rs/, rust-toolchain.toml, the script, the build constants that name the crate's targets, or the workflow, as the merge of such a pull request does, and commits them to dev when they change. Its draw job runs on ubuntu-24.04 and installs Graphviz and Liberation Sans, the first font that the diagrams name, so that the Graphviz release and the font that lay them out stay those that drew doc/modules: other releases and fonts lay them out differently. It builds cargo-modules 0.27.0 with the Rust release that rust-toolchain.toml pins, into a cache that only dev pushes save. It checks out without credentials the newest commit of the branch that a push or a dispatch names, rather than the commit that started the run, draws the diagrams into doc/modules, summarizes the files that they change, uploads them as an artifact, and names the commit that it drew in its outputs. The publish job runs only for dev. It checks out the newest dev with the deploy key with which CI persists the performance history, since deploy keys may push to dev past its rules, replaces doc/modules with the artifact, and, when that changes anything, commits it as github-actions[bot] with a message that names the drawn commit and skips CI, and pushes it, retrying on the newest dev when dev moved. It builds and runs nothing of the commit, refuses an artifact that holds anything but the page and SVG files, or a malformed drawn commit, and commits nothing outside doc/modules. Runs on dev wait for one another, and a newer push replaces the run that waits; since each run draws the newest dev, whichever run commits last commits the diagrams of the newest dev. A ready pull request that changes the inputs runs the draw job alone, so that a change that breaks the drawing fails before it merges, and its summary and artifact show the diagrams that its merge will commit. The workflow is not part of the required status check. The page now says that CI keeps the diagrams current, and links to the new Module diagrams section of doc/ci.md, which describes the workflow and how to draw the diagrams locally. Tests check the workflow's triggers, credentials, concurrency, and pins, and the handoff of the drawn commit and its diagrams to the publish job, and, on POSIX hosts, run the publish step against a local repository: it commits changed diagrams, commits nothing for unchanged ones, retries a rejected push on the newest dev, and refuses a malformed artifact or drawn commit. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Contributor
There was a problem hiding this comment.
🟡 Changes recommended
Incorrect platform filtering mislabels modules, and publication-only reruns can overwrite newer diagrams with stale artifacts.
2 open issues
What changed in this PR
Publishes the nvx crate’s module diagrams as GitHub-rendered documentation and adds automation to refresh them when their inputs change.
Changes:
- Adds a Python generator that merges Linux and Windows module exports into SVG diagrams and a Markdown page.
- Publishes the diagrams and links them from existing documentation.
- Adds drawing and publication jobs, plus generator and workflow tests.
| File | Description |
|---|---|
scripts/test_module_diagrams.py |
Tests generation, published files, and workflow behavior. |
scripts/module_diagrams.py |
Generates cross-platform diagrams and their page. |
README.md |
Links to the module diagrams. |
nvx-rs/README.md |
Links crate readers to the diagrams. |
doc/project-structure.md |
Documents the generator, workflow, and output directory. |
doc/modules/README.md |
Presents the diagrams and explains their scope. |
doc/modules/overview.svg |
Shows top-level modules. |
doc/modules/nvx-testing.svg |
Shows the testing module hierarchy. |
doc/modules/nvx-protocol.svg |
Shows the protocol module hierarchy. |
doc/modules/nvx-net.svg |
Shows the network module hierarchy. |
doc/modules/nvx-gpt.svg |
Shows the GPT module hierarchy. |
doc/modules/nvx-exec.svg |
Shows the execution module hierarchy. |
doc/modules/nvx-activation.svg |
Shows the activation module hierarchy. |
doc/modules/dependencies.svg |
Shows dependencies between top-level modules. |
doc/ci.md |
Documents diagram generation and publication. |
.github/workflows/module-diagrams.yml |
Draws diagrams and publishes updates to dev. |
.github/actions/validate-nvx/action.yml |
Adds diagram tests to Linux and Windows validation. |
Review effort: Balanced
Unless it analyzes with cfg(test), cargo-modules drops as a test module every module whose cfg holds when test holds and every other condition fails: has_test_cfg, in its analyzer.rs, folds the whole condition with test true and every other condition false, so that cfg(not(windows)) reads as a test. The pinned 0.27.0, 0.28.0, and its main branch all do. The Linux export therefore lacked nvx::host::sandbox::scratch::imp, which builds under cfg(not(windows)), and the host diagram labeled the module of that path that Windows builds "Windows only", though both platforms build one. The script now also exports each target with cfg(test), and reads the cfg conditions of each module from cargo-modules' structure. A module whose conditions, or those of a module that contains it, include test builds only for tests. Every other module of the analysis with cfg(test) builds outside tests, so the script restores those that the plain analysis lacks, with their uses in the analysis with cfg(test). It stops at a condition that depends on test in another way, such as any(test, unix) or not(test), whose modules it cannot tell apart, at a module that only the plain analysis holds, and at a structure that it cannot read or that names other modules than the analysis. For the crate as it is, it restores scratch::imp alone, with its four uses, which are also the only uses that the analysis with cfg(test) adds between modules that build outside tests; Windows lacks nothing. Drawn again, the host diagram no longer labels scratch::imp, and the other diagrams and the page stay the same. Tests cover the reading of the structure, the restoration and its refusals, and a drawing in which each platform builds a module of the same path and cargo-modules drops Linux's. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Rerunning a run's failed jobs reruns its publish job alone, which reuses the run's drawing and the commit that it drew. When a newer run had published the diagrams of inputs that a newer commit changed, such a rerun put the older diagrams back on dev, in a commit whose message skips CI, so that nothing drew the newer ones again until the inputs changed once more. On every attempt, once it fetches dev, the publish job now compares the inputs at the drawn commit and on the newest dev, as the paths that start the workflow name them, which a test keeps equal to the job's. When dev changed them, it publishes nothing and says why, since the push that changed them started a run that publishes their diagrams; when dev changed other files only, it publishes the drawing as before. A drawn commit that the repository lacks fails the comparison, and the job. Tests run the publish step against a local repository: a rerun after a newer commit changed an input leaves dev untouched, the drawing still lands after a newer commit that changes no input, and an unknown drawn commit fails. doc/ci.md describes the comparison. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
cargo-modules drops every module whose cfg holds when test holds and every other condition fails, such as one under cfg(not(windows)), unless it analyzes with cfg(test), so module_diagrams.py restores those modules from its analysis with cfg(test). But cargo-modules' filter (filter.rs) does not drop the uses of a module that it drops: it moves them, and the uses of the modules that use it, to the nearest module that contains it and that the analysis keeps, and drops those that this turns into a module's use of itself. restore() added the restored modules' own uses to all the plain uses, so a use that only a restored module made counted twice: once as that module's, and once as the use that cargo-modules had moved to the module that contains it. The dependencies diagram, which counts the pairs of modules in which one uses the other across top-level modules, would show 2 where 1 is right. restore() now projects each use of the analysis with cfg(test) between modules that build outside tests: it maps each end to itself, or to the nearest module that contains it and that the plain analysis keeps. It keeps the use if its projection is a plain use, or a module's use of itself, which filter.rs drops, and marks that plain use as moved. The uses are those, with the plain uses that no projection lands on, such as those of an item under cfg(not(test)). A moved plain use stays only if the analysis with cfg(test) holds it too, as when the module that contains a restored module makes the same use itself. The uses that only tests make stay out, as before. Where a use that involves a restored module lands on a plain use, the two analyses cannot tell whether only tests make the first, or whether only an item under cfg(not(test)) makes the second; the docstring says so. The published diagrams do not change. The only restored module, nvx::host::sandbox::scratch::imp on Linux, uses the module that contains it, a use that filter.rs drops rather than moves, and nvx::error, nvx::host::seal and nvx::host::sandbox::mount::request, which that module uses itself. So the analysis with cfg(test) holds each moved use, and both versions of restore() give the same uses on Linux and on Windows, as a redraw of doc/modules confirms. A regression test has the plain analysis hold host's use of protocol, which only the restored module imp makes, and checks that the use counts once, as imp's, while the use of error, which host and imp both make, counts twice. It fails with the previous union. Two more tests pin that the plain uses that no projection explains stay, and that the uses that only tests make between modules that the plain analysis keeps stay out. doc/ci.md now says that the script restores those modules with their own uses in place of the ones that the plain analysis moves. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Pedro Henrique Penna (ppenna)
deleted the
agents/rebase-module-diagrams-for-docs
branch
October 27, 2026 19:35
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Publishes diagrams of the
nvxcrate's modules in the documentation, and has CI draw them again after every merge todevthat changes the crate. It supersedes thewip/module-diagramsbranch, rebased on the currentdev: its PowerShell driver and interactive HTML atlas, which GitHub shows only as source, become a cross-platform Python generator whose Markdown page of SVG images GitHub renders.What changed
The first three commits each pass the checks below on their own, and the last three address the review:
scripts: draw the module diagrams of the nvx crate:scripts/module_diagrams.pyexports the modules of the crate's library, with itscli,testing, andguestfeatures, with cargo-modules forx86_64-unknown-linux-muslandx86_64-pc-windows-msvc, and merges the two exports. It draws, with Graphviz, the top-level modules, the dependencies between them, and the hierarchy of each top-level module, and marks the modules that only one platform builds. It writes the SVG diagrams and their page tobuild/modulesunless--outputnames another directory, and refuses exports that hold anything but modules and their uses. validate-nvx runs its unit tests.doc: publish the module diagrams of the nvx crate:doc/modulespublishes the diagrams, drawn on Ubuntu 24.04 with cargo-modules 0.27.0 and Graphviz 2.43.0 with Liberation Sans. The README, the crate's README, and the project structure link to them, and a test checks that the directory holds exactly the page and the diagrams that it shows.ci: draw the module diagrams again after every merge to dev:module-diagrams.ymlruns for pushes todev, ready pull requests, and dispatches that change the crate, the Rust pin, the script, the build constants, or the workflow. Its credential-freedrawjob draws the newest commit and uploads the diagrams. Fordev, itspublishjob commits them when they change, in a commit whose message skips CI, as the performance baselines' commits do. The new Module diagrams section ofdoc/ci.mddocuments it.scripts: restore the modules that cargo-modules mistakes for tests: unless it analyzes withcfg(test), cargo-modules (0.27.0, 0.28.0, andmain) drops as a test module every module whose cfg folds true withtesttrue and every other condition false, such as#[cfg(not(windows))] mod impinhost::sandbox::scratch, which the host diagram therefore labeled Windows only. The script now also reads the analysis withcfg(test)and each module's cfg conditions, restores the modules that build outside tests, and fails on a cfg that depends ontestotherwise.ci: publish no drawing of inputs that dev has since changed: a rerun of thepublishjob alone reuses its run's drawing, so the job publishes it only whiledevholds the inputs that it was drawn from, and leaves newer inputs to the run that their push started.scripts: count each use of a restored module once: cargo-modules' filter moves the uses of a module that it drops to the nearest module that contains it and that it keeps, so adding a restored module's uses to the plain ones counted a use that only the restored module makes twice. The script now projects each use of the analysis withcfg(test)onto the modules that the plain analysis keeps, keeps the uses that land on a plain use or on a module itself, and drops the moved plain uses that the analysis withcfg(test)does not hold. The published diagrams do not change.Notes for reviewers
publishjob pushes withPERFORMANCE_PERSIST_DEPLOY_KEY, the only deploy key that may push todevpast its ruleset. It builds and runs nothing of the commit, refuses an artifact that holds anything but the page and SVG files, and commits nothing outsidedoc/modules. A dedicated key would only change the secret's name.drawjob runs onubuntu-24.04and installs Liberation Sans, as the environment that drew the committed diagrams did.rust-toolchain.tomlpins; 0.28.0 needs Rust 1.98.Required status check. A ready pull request that changes its inputs runs thedrawjob alone, so a change that breaks the drawing fails before it merges.activationusinghost: cargo-modules counts the trait implementations of a type toward the type's module, andhost::template::key(nvx-rs: derive the compatibility keys of templates #879) implementsFrom<KeyIdentity>foractivation::Identity.activationimports nothing fromhost.devhas since gained the template store (nvx-rs: add the template store #881), which adds modules tohost. Thedrawjob of this pull request, which draws its merge, reports that it changesREADME.md,overview.svg,dependencies.svg, andnvx-host.svgindoc/modules, and thepublishjob commits them after the merge, as it will for every later change of the crate.Validation
ruff checkandruff format --checkofscriptsand.github/specula, Pyright strict for the Linux and Windows platforms,check_markdown_links.py, the module's tests, and the tests that require validate-nvx to run every test module, the build constants to own the target triples, and no file to repeat a tool pin.test_adversarial.py,test_hosts.py, thenvx.pyhelp commands, and the link check. The publish step's tests run against a local bare repository, including a rejected push and a rerun after newer inputs. The regression tests of the three review fixes fail without their fix.test_setup_azure.py, which run the Linux setup scripts withbash, fail on this host, wherebashis WSL's, and fail the same way ondev; the Windows CI job runs them with Git Bash.scripts/module_diagrams.py --output doc/modulesin a container that mirrors thedrawjob reproducesdoc/modulesbyte for byte, run after run, and the first CI run of thedrawjob drew the same bytes.drawjob on a GitHub runner. Thepublishjob runs only after the merge. The change selects no hypervisor suite.