Skip to content

Per-page rendering and reproducible output for library consumers - #16

Merged
ewels merged 3 commits into
mainfrom
claude/nf-docs-library-consumer-gaps-v585qf
Aug 9, 2026
Merged

ewels merged 3 commits into
mainfrom
claude/nf-docs-library-consumer-gaps-v585qf

Conversation

@ewels

@ewels ewels commented Aug 9, 2026

Copy link
Copy Markdown
Owner

Two gaps blocked a downstream consumer — they generate docs in a build hook, currently by shelling out to nf-docs, and want to switch to the library API from #14. Their acceptance criterion was deleting two workarounds: a temp-folder round-trip, and code that strips the generated footer.

1. Per-page markdown as strings

render() returns one combined document, and the per-page Markdown was only reachable via render_to_directory(), which writes files. So consuming pages in memory meant generating into a temporary directory and reading them back.

BaseRenderer.render_pages(pipeline) -> dict[str, str] returns the same content keyed by filename, with nf_docs.render_pages() alongside render() on the facade:

pages = nf_docs.render_pages(pipeline, "markdown")

for filename, content in pages.items():
    my_site.add_page(f"api/{filename}", content)

render_to_directory() moved into the base class and now just writes what render_pages() returns, which drops the duplicated write loop from four of the five renderers. render_pages() is the abstract method in its place.

TableRenderer keeps its own render_to_directory(). It reads the existing README.md and, if it finds the BEGIN_NF_DOCS / END_NF_DOCS markers, injects between them, honouring {{ section }} template tags. That output depends on what's already in the destination, so it isn't expressible as dict[str, str]. Its render_pages() returns the standalone marker-wrapped form — the same bytes render_to_directory() writes into an empty directory — and a test asserts the two genuinely diverge when a marker README already exists.

Keys are conditional. Markdown always returns index.md and inputs.md, and adds config.md, workflows.md, processes.md and functions.md only when the pipeline has the corresponding entries. That's six possible pages, not the five originally reported. The docs say plainly not to assume a fixed set.

render_pages() defaults to html like render() and generate(), even though markdown is the format it exists for — a facade where one function silently defaults differently is a trap.

2. Reproducible output

The request was footer=False, but that only covers markdown and table. Every place generation metadata is injected:

Format Timestamp? Where
markdown yes get_markdown_footer()
table yes get_markdown_footer()
json yes, microsecond resolution data["generated_by"]
html yes get_generation_timestamp()
yaml no already reproducible

So footer=False would leave them stripping output for anything but markdown. Instead, BaseRenderer takes include_generation_info: bool = True, covering every format. Being a renderer option, it reaches render(), render_pages() and generate() through the existing **renderer_kwargs with no plumbing.

It's keyword-only, so it can't collide with the positional indent / use_tailwind / default_flow_style.

HTML keeps its Documentation generated by nf-docs v0.4.0 line and the Nextflow attribution — both identical on every build from a given install — and drops only the timestamp beside them.

Verification

The tests assert the actual requirement: render the same pipeline twice, per format, and compare bytes. Worth noting, because a naive version of that test passes no matter what the flag does — the suite finishes inside a single wall-clock second, so the timestamps match by accident. A stub clock in conftest.py advances a second per call, and a guard test asserts output does vary with generation info on.

Also verified independently outside pytest with real sleep between renders: reproducible with the flag off, varying with it on, for every format except YAML.

425 tests pass. ruff check, ruff format --check, ty check and prettier clean. The html.html edit adds no class names, and a real Tailwind rebuild leaves tailwind.css byte-identical.

Notes for review

  • BaseRenderer's abstract surface changed. render_to_directory() → render_pages(). The five built-in renderers are unaffected, but a subclass outside this repo that implemented only render_to_directory() now needs a render_pages() too. Called out under Changed in the CHANGELOG.
  • HTML still isn't fully reproducible, and this flag can't fix it. When a pipeline has a GitHub repository, get_repository_avatar_data_url() fetches the org avatar over the network and inlines it at render time, so a failed request or a changed avatar changes the page. That's content rather than generation metadata, so it's out of scope here — but since the stated requirement is byte-identical builds, it's flagged in a docs warning rather than left to be discovered.
  • CLI unchanged. No --no-generation-info flag; this was scoped to the library API. Easy to add if you want it.

🤖 Generated with Claude Code

https://claude.ai/code/session_01922CMQ76XwtfSr5ZgYJ8rj


Generated by Claude Code

claude added 3 commits August 9, 2026 10:35
Two gaps blocked a downstream consumer generating docs in a build hook
from switching off the nf-docs subprocess.

Per-page output in memory. render() returns one combined document, and
the per-page Markdown was only reachable via render_to_directory(), so
callers had to write to a temporary folder and read the files back.
BaseRenderer.render_pages() returns {filename: content} instead, with
nf_docs.render_pages() alongside render() on the facade.

render_to_directory() moves into the base class and just writes what
render_pages() returns, which drops the duplicated write loop from four
of the five renderers. render_pages() is the abstract method now.
TableRenderer keeps its own render_to_directory(): it injects into an
existing README.md between the doc markers, honouring {{ section }}
tags, and that result depends on what is already on disk. Its
render_pages() returns the standalone marker-wrapped form.

Which keys appear depends on the pipeline - Markdown emits config.md,
workflows.md, processes.md and functions.md only when non-empty - so
the docs tell consumers not to assume a fixed set.

Reproducible output. The request was footer=False, but that only covers
markdown and table; json embeds a microsecond timestamp under
generated_by and html renders one into the page footer, so anything but
markdown still needed stripping. BaseRenderer gains
include_generation_info=True instead, covering every format, and it
reaches render(), render_pages() and generate() through the existing
**renderer_kwargs. YAML never carried generation metadata and is
unaffected.

The tests assert the actual requirement: render the same pipeline
twice, per format, and compare bytes. A stub clock advances a second
per call, since the suite otherwise finishes inside one wall-clock
second and the comparison would hold whatever the flag did - the guard
test asserts output does vary with generation info on.

Verified: 434 tests, ruff check, ruff format --check, ty check.
The html.html edit changes no class names, and a Tailwind rebuild
leaves tailwind.css byte-identical.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01922CMQ76XwtfSr5ZgYJ8rj
include_generation_info=False removed the whole "Documentation generated
by nf-docs vX on <timestamp>" footer row. The version is the same on
every build from a given install, so only the timestamp needs to go for
the output to be reproducible.

The row now renders unconditionally and just the timestamp span is
guarded, which also means generation_info is always passed to the
template — so the Nextflow attribution goes back to reading its URL from
there rather than needing its own variable.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01922CMQ76XwtfSr5ZgYJ8rj
Cleanup only — no behaviour change. All four review angles agreed the
production code was a net simplification; these are the leftovers.

- The AdvancingClock stub and its fixture were copied verbatim into both
  test files. Moved to conftest.py, so the four hand-rolled
  patch("nf_docs.generation_info.datetime", ...) calls in test_api.py
  become the shared advancing_clock fixture.
- Dropped the ten parametrized facade-level byte-identity cases in
  test_api.py. render()/render_pages() are pass-throughs, so they
  couldn't fail for any reason the renderer-level cases wouldn't already
  catch. One test that the flag reaches the renderer replaces them; the
  generate() and single-file cases stay, since those are distinct paths.
- render_pages(..., "html") in test_all_formats now passes
  use_tailwind=False like its siblings, so it stops shelling out to the
  Tailwind CLI just to check the dict is non-empty.
- Dropped the per-file parent mkdir in the base render_to_directory. The
  output directory is created once just above and no renderer nests its
  output, which the render_pages contract now says explicitly.
- The "table injection can't be expressed as render_pages" caveat was
  written out four times and already drifting. It stays in
  TableRenderer.render_pages and the docs page; the other two now point
  at it.

425 tests pass (down from 434 with the redundant cases gone), ruff and
ty clean.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01922CMQ76XwtfSr5ZgYJ8rj
@ewels
ewels merged commit 3f194df into main Aug 9, 2026
8 checks passed
@ewels
ewels deleted the claude/nf-docs-library-consumer-gaps-v585qf branch August 9, 2026 15:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants