UCI Regulations → Markdown

Tracks the UCI Cycling Regulations as searchable, diffable Markdown.

📖 Live site: https://jdanton.github.io/ucirules/ — the Markdown published as a searchable ProperDocs site on GitHub Pages.

The PDFs are published on UCI's CDN and change over time. sync.py pulls the current set, detects what changed since the last run, and regenerates only the affected Markdown — so git diff after a sync shows exactly which regulations were amended. It can also publish the Markdown as a browsable ProperDocs site (see Usage).

Unofficial mirror. The PDFs on uci.org remain the authoritative source; this site is for search and change-tracking.

How it works

A single pass in sync.py:

1. Fetch the regulations page and extract every PDF URL (they're embedded in the page's JSON, hosted on Contentful's CDN).
2. Download all PDFs into a temporary directory (removed when the run ends — PDFs are never kept on disk).
3. Checksum each PDF (SHA-256) and compare against docs/.manifest.json from the previous run to classify every document as added / changed / removed / unchanged. A download that keeps failing after retries is skipped with a warning — its Markdown and manifest entry carry forward and the next run picks it up — so one unavailable asset can't abort the sync or look like a withdrawn document.
4. Convert only the added and changed PDFs to Markdown (via pymupdf4llm), extracting any figures/diagrams to docs/images/. Markdown and images for removed PDFs are deleted, and the manifest is rewritten.
5. Build site (optional) — with --docs, render the Markdown into a documentation-style static site using ProperDocs (a MkDocs-style generator). The site uses the Read the Docs theme, configured in properdocs.yml; the build output lands in site/ and can be deployed to GitHub Pages, or anywhere else.
6. Deploy to GitHub Pages (optional) — with --ghdeploy, build and push the site to the gh-pages branch, served at https://jdanton.github.io/ucirules/.

The manifest is keyed by source URL — each document's stable identity — so files can be renamed freely without desyncing change detection. Each Markdown file also carries a small YAML front-matter block (source_pdf, source_url, source_sha256) for traceability.

File naming

Markdown files are named after their content: a descriptive slug derived from the document's title, joined by __ to the original source filename (kept for uniqueness and traceability):

11-JO-20260301-E.pdf  ->  part-11-olympic-games__11-JO-20260301-E.md

The numbered Regulation Parts (and their amendments) are detected from the document title — or the source code when the doc just opens with "MEMORANDUM" — and named from a stable table (PART_TITLES in sync.py), so all of e.g. Part 2's documents share the part-02-road-races__… prefix and sort together. Other documents (policies, lists, protocols, qualification systems) take a slug from their title heading. When the slug would just repeat the source name, the prefix is dropped (e.g. preliminary-provisions.md).

Images

Figures are rendered to PNGs under docs/images/, named <document>.pdf-<page>-<index>.png, and referenced from the Markdown with relative links (![](images/...)). Because each image is namespaced by its source document, the per-document figures are easy to find, and clear_images removes a document's old figures before re-rendering so nothing is orphaned.

Usage

# One-time setup
python3 -m venv .venv
./.venv/bin/pip install -r requirements.txt

# Sync (run anytime to pull the latest and regenerate changed docs)
./.venv/bin/python sync.py

# Preview the delta without writing anything
./.venv/bin/python sync.py --dry-run

# Rebuild everything from scratch, ignoring the manifest
./.venv/bin/python sync.py --force

# Refresh page titles + grouped nav (after docs are added/removed)
./.venv/bin/python build_nav.py

# Generate docs-style static site pages which can be deployed
./.venv/bin/python sync.py --docs

# Deploy to github pages branch
./.venv/bin/python sync.py --ghdeploy

# Rebuild the site from the existing Markdown, without re-downloading the PDFs
./.venv/bin/python sync.py --docs --no-sync

Page titles

The site's page titles and sidebar come from titles.yml, a {filename: title} mapping generated by build_nav.py. It scans docs/, derives a clean title for each document (numbered Parts get Part N — <name> (in force/amendments <date>); others use their heading or a tidied source name), and writes a nav: into properdocs.yml grouped by Part. Re-runs preserve existing entries, so hand-edited titles survive; documents that resist auto-detection are fixed via the OVERRIDES table in build_nav.py. Run build_nav.py before --docs/--ghdeploy whenever the set of documents changes.

Options: --out DIR (default ./docs), --dry-run, --force, --docs, --ghdeploy, --no-sync.

--docs / --ghdeploy first run the sync, then build (and deploy) the site — add --no-sync to build straight from the committed Markdown with no network access. The build step needs properdocs and its Read the Docs theme installed (both in requirements.txt); the built site lands in site/ (gitignored).

pymupdf4llm is pinned to an exact version. Its output shifts between releases, and change detection diffs the generated Markdown, so an unpinned converter rewrites documents UCI never touched. Note that pymupdf4llm 2.0 is an empty placeholder — upstream moved the code into pymupdf 2.x, which needs a code change here, plus a --force re-conversion of the whole tree.

Change-control workflow

./.venv/bin/python sync.py
git add docs/
git commit -m "Sync UCI regulations $(date +%F)"

The commit diff is a human-readable record of what UCI changed. Run on a schedule (cron / CI) to keep a continuous history.

Scheduled sync (GitHub Actions)

.github/workflows/monthly-sync.yml runs the whole flow on the 1st of each month (and on demand): it installs the dependencies, runs sync.py, regenerates the titles/nav with build_nav.py, and — only if something changed — commits the new Markdown to the default branch. The site is then redeployed to gh-pages on every run, changes or not, so it tracks the default branch: code changes and hand-written pages move the site without UCI touching a PDF. No external infrastructure or secrets are required; it uses the built-in GITHUB_TOKEN (the workflow grants itself contents: write).

# Trigger a run by hand (e.g. to test it) instead of waiting for the schedule:
gh workflow run monthly-sync.yml
gh run watch

The automated run handles the mechanical sync + deploy. Each sync that finds changes also inserts a dated entry at the top of What changed listing the documents it added, updated or removed, flagged "summary not yet written". The plain-English summary reads the amendment text, so write it by hand: replace that note with the article-by-article changes.

Layout

Path	Tracked	Description
sync.py	✅	Download → checksum-diff → convert; optionally build/deploy the site
docs/	✅	Generated Markdown, one file per regulation
docs/images/	✅	Figures/diagrams extracted from the PDFs (PNG)
docs/.manifest.json	✅	Source URL → {name, stem, SHA-256} (drives delta detection)
.github/workflows/monthly-sync.yml	✅	Scheduled monthly sync + deploy (GitHub Actions)
clean_md.py	✅	Strips printed TOCs / page furniture from converted Markdown
reclean_docs.py	✅	One-off: re-clean existing docs/ in place
build_nav.py	✅	Scans docs/ → clean page titles + grouped nav
titles.yml	✅	{filename: title} mapping (editable; drives the nav)
properdocs.yml	✅	ProperDocs site config + generated nav:
site/	❌	Built static site (output of --docs); rebuild any time
gh-pages branch	—	Published site served at https://jdanton.github.io/ucirules/ (pushed by --ghdeploy)
.venv/	❌	Local virtualenv
PDFs	❌	Pulled to a temp dir per run; never committed

Notes

• These are text-based PDFs, so extraction needs no OCR. If image-based/scanned PDFs are ever added, swap pymupdf4llm for an OCR-capable tool (e.g. marker or docling).
• Tables of contents render as Markdown tables with dot-leaders — faithful to the source but a little noisy; can be post-processed if desired.