Skip to content

Make the local documentation build work, and document it - #36

Open
hakonhagland wants to merge 5 commits into
OPM:masterfrom
hakonhagland:local-doc-build
Open

hakonhagland wants to merge 5 commits into
OPM:masterfrom
hakonhagland:local-doc-build

Conversation

@hakonhagland

@hakonhagland hakonhagland commented Sep 24, 2026 •

Copy link
Copy Markdown
Collaborator

Updated after Copilot's review: commits 4 and 5 added, and the statement about release branches under commit 1 corrected.

Following the repository's own instructions for building the documentation locally fails on any branch that is not a release branch. opmdoc-download-files writes the docstring JSON files to a directory (python in the root) that docs/conf.py does not read (it reads python/master-tmp), so the build stops on a file the user has just downloaded:

Exception occurred:
FileNotFoundError: [Errno 2] No such file or directory:
'.../python/master-tmp/docstrings_common.json'

Download the files where the build looks for them (commit 1)

  • docs/conf.py picks the directory by branch name: release branches read the snapshots committed under python/, every other branch reads python/master-tmp/.
  • opmdoc-download-files always wrote the JSON files to python/, and wrote dune.module to the repository root — That matches neither layout: on most branches the build reads python/master-tmp/, and on a release branch python/ holds the committed release snapshot, which the download overwrote (fixed in commit 4).
  • docstrings_dir() now applies the same branch rule conf.py does, and creates python/master-tmp when it is missing. For release branches that rule is wrong for writing, and commit 4 replaces it. That directory is gitignored, so it is absent on a fresh clone.
  • All three files now land in the same directory, which is also what the CI workflow does for the published build.

Add tests for the destination (commit 2)

  • Parameterized over a release branch, master and an ordinary feature branch, asserting the directory each one resolves to. Commit 4 replaces the release-branch case.
  • One test for the fresh-clone case, where python/master-tmp does not exist yet and has to be created.

Give the README the steps (commit 3)

The section previously read, in full, "Follow the commands in .github/workflows/python_sphinx_docs.yml for your local setup!" That workflow interleaves the local build with sphinx-versioned's multi-branch machinery, the gh-pages deployment and CI-only setup, and does not indicate which parts a person building locally actually needs.

It is replaced with the four commands that do the job, plus the two things that cannot be worked out from the workflow at all:

  • sphinx-versioned builds from git history, not from the working tree, so changes have to be committed before the build will see them. The Makefile says so in a comment; the README did not.
  • The docstring JSON files come from opm-common and opm-simulators, not from this repository, which is why a download step exists at all.

It also mentions make view-docs and opmdoc-view-doc, which already existed but were only reachable through a link to another README, and notes that the built pages open straight from disk — no web server needed, since the generated HTML references its stylesheet by relative path.

Refuse to download on a release branch (commit 4)

  • Copilot's review pointed out that the README instructions could overwrite release-branch inputs. Reproduced: in a worktree of release-2026.04, the download replaced the committed python/docstrings_common.json, python/docstrings_simulators.json and python/dune.module with master's copies and left them modified. The code before this PR already overwrote the two JSON files there; commit 1 added dune.module.
  • A release branch needs nothing downloaded, since its snapshot is taken from the release's own sources. The command now stops with an explanation and exit status 1 before making any request, and docstrings_dir() only returns python/master-tmp.
  • Tests: the release-branch case of the directory test is replaced by a test that main() on a release branch makes no request, leaves the snapshot unchanged and creates no python/master-tmp. A new test checks that on master all three files are written to python/master-tmp/.
  • README step 2 now says the download is not for release branches.

Use opmdoc-view-doc in the README (commit 5)

  • make view-docs runs xdg-open, which exists on Linux only, so the documented step failed on macOS (from Copilot's review). Step 3 now uses opmdoc-view-doc, which opens the same page through click.launch on any platform and defaults to the current branch.

Verification

Reproduced the failure and confirmed the fix end to end: removed python/master-tmp to stand in for a fresh clone, ran opmdoc-download-files, then built. Before commit 1 that sequence raised the FileNotFoundError above and produced no simulators.html. After it, the three files land in python/master-tmp/, the build reports build succeeded, and simulators.html contains the generated API documentation. The full test suite passes.

After commit 4, the same release-branch worktree check exits with status 1 and the explanation, git status stays clean, and no python/master-tmp is created. The full test suite passes: 6 tests.

docs/conf.py picks the directory to read docstrings_common.json and
docstrings_simulators.json from by branch name: release branches use the
snapshots committed under python/, every other branch uses
python/master-tmp/. opmdoc-download-files always wrote to python/, and
wrote dune.module to the repository root, which matches neither layout.

So on any branch other than a release branch, the documented sequence

  opmdoc-download-files
  make docs

fails on a file the user has just downloaded:

  Exception occurred:
  FileNotFoundError: [Errno 2] No such file or directory:
  '.../python/master-tmp/docstrings_common.json'

Pick the destination the same way conf.py does, and create
python/master-tmp when it does not exist yet, as on a fresh clone. All
three files now go to the same directory, which is also what the
workflow does for the published build.
Checks that docstrings_dir() follows the same branch rule as
docs/conf.py, and that it creates python/master-tmp when it is missing,
which is the case on a fresh clone since that directory is gitignored.
The section pointed at .github/workflows/python_sphinx_docs.yml, which
interleaves the local build with the multi-branch machinery, the
gh-pages deployment and the CI-only setup, and does not say which parts
someone building locally needs.

Replace it with the four commands that do the job, and state the two
things that are not discoverable from the workflow: that sphinx-versioned
builds from git history rather than the working tree, so changes have to
be committed first, and that the docstring JSON files come from
opm-common and opm-simulators rather than from this repository.

Also mention make view-docs and opmdoc-view-doc, which already exist but
were only reachable through a link to another README, and note that the
built pages open straight from disk without a web server.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

README instructions may overwrite release-branch inputs and need correction before approval.

Get a fresh assessment by requesting another Copilot review.

Review effort: Lite
Findings: 1 Low severity

Open (1)
What changed in this PR

This PR fixes local documentation builds by aligning downloaded assets with branch-specific Sphinx paths and documenting the complete workflow.

Changes:

  • Added branch-aware download destinations and tests.
  • Creates python/master-tmp when needed.
  • Expanded README setup, build, and viewing instructions.
File Summary
README.md Documents the local workflow; requires updates for release-branch safety, portability, and virtualenv usage.
python/​sphinx_docs/​tests/​test_download_files.py Tests branch-specific destinations and fresh-clone directory creation.
python/​sphinx_docs/​src/​opm_python_docs/​download_files.py Aligns downloaded files with the paths expected by the documentation build.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread README.md Outdated
The previous commits made opmdoc-download-files write where the build
reads, following docs/conf.py's branch rule: python/master-tmp/ on most
branches, python/ on release branches. That rule is right for reading
but wrong for writing. On a release branch, python/ holds the committed
snapshot of docstrings_common.json, docstrings_simulators.json and
dune.module, taken from the release's own sources. Downloading master's
copies there replaces that snapshot and leaves the files modified in
the working tree, ready to be committed by mistake.

Reproduced in a worktree of release-2026.04: the command wrote all
three files and git status showed them modified. The code before this
branch already overwrote the two JSON files there; the previous commits
added dune.module.

A release branch needs nothing downloaded, so the command now stops
with an explanation and a non-zero exit before any request is made.
docstrings_dir() therefore only returns python/master-tmp. The tests
check both paths through main(): three files written to
python/master-tmp/ on master, and on a release branch no request, no
change to the snapshot, and no python/master-tmp created. The README
says the download step is not for release branches.
The README told users to open the result with make view-docs, but that
Makefile target runs xdg-open, which exists on Linux only; on macOS the
documented step fails.

opmdoc-view-doc opens the same page, python/sphinx_docs/docs/_build/
<branch>/index.html, through click.launch, which uses the platform's own
opener. Without --branch it opens the current branch, as make view-docs
does, so it replaces that step directly. The README already mentioned it
for opening other branches.
@hakonhagland

Copy link
Copy Markdown
Collaborator Author

Copilot's overview also flagged that the README instructions "may overwrite release-branch inputs", without filing it as a finding. It is right, and it is the more important of the two points. I reproduced it: on a release branch the download writes into python/, replacing the committed release snapshots (docstrings_common.json, docstrings_simulators.json and dune.module) with master's copies, and leaves them modified in the working tree. The old script already overwrote the two JSON files there, and my change added dune.module — and the README now presents the download as a step for every branch. Release branches should not download anything, since their snapshots are taken from the release's own sources. In f02529c the downloader now refuses to run on a release branch and writes nothing, with a test, and the README says the step is for non-release branches. I have also corrected the sentence in the description that called python/ the correct destination for release branches.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🔵 Needs a closer look

The README’s documented build prerequisites omit make, causing the workflow to fail in environments without it.

Review effort: Lite
Findings: None

Resolved since last review (1)

@hakonhagland

Copy link
Copy Markdown
Collaborator Author

[copilot]: The README’s documented build prerequisites omit make, causing the workflow to fail in environments without it.

I'd rather leave this out. make is standard on Linux and comes with the command line tools on macOS (the same package that provides git there), and the people building these docs work on OPM, which is itself built with CMake and make. If it is missing, the step fails right away with make: command not found, which explains itself. Listing it next to Python and Poetry would add clutter for very little benefit.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants