Repository navigation
Speed up the docs build; refactor the board support matrix code - #11921
Merged
Merged
Conversation
Parallel reading and writing gives identical HTML and LaTeX output and saves a few seconds per build. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Six scripts in tools/ import it, each by adding docs/ to sys.path. In tools/, they can import it directly. conf.py adds tools/ to sys.path. ci_set_matrix.py: keep treating the file as docs-only: it triggers the docs build, but not board builds or the Zephyr tests. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
collect_board_data() gathers everything about each board once, as JSON types. support_matrix_by_board() keeps its signature but now only formats that data, collecting it first if no board_data is passed. zephyr-cp boards have no mpconfigboard.h or .mk, so they now get empty MCU and flash values instead of raising FileNotFoundError when add_chips is set. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The NVM size in board stub docstrings has been wrong for many boards: - SAMD51 boards showed 256, the SAMD21 value. The regex took the first define in atmel-samd's mpconfigport.h, ignoring #ifdefs. - A board-level CIRCUITPY_INTERNAL_NVM_SIZE in mpconfigboard.h was never used: the code searched the port's file a second time instead. The value is a C macro, so only the C preprocessor can compute it. Stub builds don't have the port toolchains for that. The line was only informational, so drop it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
board_stub_builder.py collects the board data once, uses it for both the support matrix and the board names, and saves it as circuitpython-stubs/board_data.json for conf.py to reuse. The names now come from the same parsing the docs use, which cuts at the closing quote. lolin_c3_mini and lolin_c3_pico stubs no longer include the trailing "// from Wemos MP" comment in the board name. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Before, the docs build collected the board data twice: once inside make stubs and again here. Each collection runs make once per board (about 15 seconds). A fresh HTML build drops from about 54 to 42 seconds. If the saved file is missing, support_matrix_by_board() still collects the data itself. The old comment said the stubs must be built before the matrix. The matrix no longer reads the stubs; now the matrix needs the saved data. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
write_csvs() iterated over the dict itself, so unpacking each key into (key, values) raised ValueError before any CSV was written. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The port was read by splitting the " - port: ..." line on every "-", so atmel-samd became "atmel". `circuitpython_setboard -l` then showed the wrong heading, and filtering by "atmel-samd" found nothing. Split on the first ":" instead. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Under docs/, shared_bindings_matrix.py matched ^docs/ in the tests and mpy-cross ignore patterns. Its move to tools/ lost that. Add it to both patterns so it is still treated as docs-only. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Changes under tools/board_stubs/ matched no rule, so CI built every board and skipped the docs build. That code only builds the board stubs, which the docs build makes. Build the docs for it, and skip the board builds, the Zephyr tests, the unix tests and mpy-cross. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The PDF build needs LaTeX and rsvg-convert, which were listed only in the CI workflow. Document the Debian/Ubuntu packages and make latexpdf. Setting SPHINXOPTS replaces the default -W --keep-going -j auto, so the example now includes them. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
pdflatex's first pass prints a warning for every cross-reference, about 3,300 lines, because nothing is resolved until a later pass. -silent sends the pdflatex output to the .log file only. latexmk still reports references left undefined after the last pass, and still fails on LaTeX errors. Use "make latexpdf LATEXMKOPTS=" to see the full output. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Code by Claude, reviewed by @dhalbert.
A fresh
make htmlgoes from 57 seconds to 42 seconds on a 20-CPU Linux machine. The HTML output is unchanged. The docs build scripts were refactored to remove duplicate work and for clarity.This also speeds up the GitHub CI docs build and the Read the Docs builds. This PR's Read the Docs build took 264 seconds instead of about 392 seconds for other recent PRs, because its Sphinx step went from about 315 to 186 seconds. Read the Docs doesn't use
-j auto(and we can't make it do that), so that saving comes entirely from collecting the board data once.Parallel Sphinx builds
make html,make latexpdfand the CI HTML build now use-j auto. Confirmed that the output is still identical after the change.Board data collected twice per docs build
board_stub_builder.py, run bymake stubs, and again inconf.py. Each run invokesmakeonce per board, about 15 seconds.make stubsnow saves the board data ascircuitpython-stubs/board_data.json, andconf.pyreuses it. If the file is missing,conf.pycollects the data itself. The data file is not included in the stubs package.Support matrix code
shared_bindings_matrix.pywas moved totools/, so othertools/scripts don't have to adddocs/to their path.collect_board_data()now gathers each board's data once.support_matrix_by_board()keeps its signature and formats that data. Its output for the existing callers is unchanged.FileNotFoundErrorwhenadd_chipsis set. They don't record those yet.Board stubs
Wrong NVM size: the NVM size in each board stub's docstring was wrong for many boards. SAMD51 boards showed the SAMD21 value (256 instead of 8192), and a board-level
CIRCUITPY_INTERNAL_NVM_SIZEwas ignored. The value is a C macro, which only the C preprocessor can compute.Fix for now: removed the NVM size line. Board names now come from the same parsing the docs use. This fixes a glitch where
lolin_c3_miniandlolin_c3_picohad a trailing// from Wemos MPcomment in the name.We can add back the (correct!) NVM size info later, maybe by moving the definition to
.mkfrom.h.Smaller fixes
tools/board_summaries.pyiterated over a dict instead of.items()and raisedValueErrorbefore writing any CSV.circuitpython_setboard -lshowed hyphenated ports as their first word, such asatmelforatmel-samd, so filtering byatmel-samdfound nothing.CI
tools/board_stubs/matched no rule inci_set_matrix.py, so they built every board and skipped the docs build.tools/shared_bindings_matrix.pyneeds the same docs-only treatment it had underdocs/.PDF build
pdflatex's first pass prints a useless warning for every cross-reference, about 3,300 lines; later passes resolve the cross-references. The PDF prerequisites were listed only in the CI workflow.make latexpdfrunslatexmkwith-silent. Warnings left after the last pass, and errors, are still reported.make latexpdf LATEXMKOPTS=shows the full output.docs/README.mdwas also updated: it now lists the packages the PDF build needs, and notes that settingSPHINXOPTSreplaces the defaults.