[docs] Refactor documentation

.codacy.yml

@@ -1,7 +1,7 @@
 ---
 # Codacy configuration file
 # https://docs.codacy.com/repositories-configure/codacy-configuration-file/
-# Analyze only doctr/ and references/ — everything else is excluded.
+# Analyze only doctr/ and references/ - everything else is excluded.
 
 exclude_paths:
   - ".github/**"

.gitignore

@@ -70,6 +70,8 @@ instance/
 
 # Sphinx documentation
 docs/_build/
+docs/build/
+docs/source/models/_generated/
 
 # PyBuilder
 target/

.pre-commit-config.yaml

@@ -16,7 +16,7 @@ repos:
       - id: no-commit-to-branch
         args: ['--branch', 'main']
   - repo: https://github.com/astral-sh/ruff-pre-commit
-    rev: v0.17.1
+    rev: v0.17.3
     hooks:
       - id: ruff
         args: [ --fix ]

README.md

@@ -25,7 +25,7 @@ What you can expect from this repository:
 ### Getting your pretrained model
 
 End-to-End OCR is achieved in docTR using a two-stage approach: text detection (localizing words), then text recognition (identify all characters in the word).
-As such, you can select the architecture used for [text detection](https://mindee.github.io/doctr/latest/modules/models.html#doctr-models-detection), and the one for [text recognition](https://mindee.github.io/doctr/latest//modules/models.html#doctr-models-recognition) from the list of available implementations.
+As such, you can select the architecture used for [text detection](https://mindee.github.io/doctr/latest/models/detection.html), and the one for [text recognition](https://mindee.github.io/doctr/latest/models/recognition.html) from the list of available implementations.
 
 ```python
 from doctr.models import ocr_predictor
@@ -119,7 +119,7 @@ plt.show()
 ![Synthesis sample](https://github.com/mindee/doctr/raw/main/docs/images/synthesized_sample.png)
 
 The `ocr_predictor` returns a `Document` object with a nested structure (with `Page`, `Block`, `Line`, `Word`, `Artefact`).
-To get a better understanding of our document model, check our [documentation](https://mindee.github.io/doctr/modules/io.html#document-structure):
+To get a better understanding of our document model, check our [documentation](https://mindee.github.io/doctr/latest/getting_started/concepts.html#the-document-tree):
 
 You can also export them as a nested dict, more appropriate for JSON format:
 

docs/README.md

@@ -1,13 +1,85 @@
-# Contribute to Documentation
+# docTR documentation
 
-Please have a look at our [contribution guide](../CONTRIBUTING.md) to see how to install
-the development environment and how to generate the documentation.
+The source of [mindee.github.io/doctr](https://mindee.github.io/doctr), built with [Sphinx](https://www.sphinx-doc.org),
+the [PyData theme](https://pydata-sphinx-theme.readthedocs.io), [MyST Markdown](https://myst-parser.readthedocs.io)
+and [sphinx-design](https://sphinx-design.readthedocs.io).
 
-To install only the `docs` environment, you can do:
+## Build it locally
 
 ```bash
-# Make sure you are at the root of the repository before executing these commands
-python -m pip install --upgrade pip
-pip install -e .[viz,html]
-pip install -e .[docs]
+# From the root of the repository
+pip install -e ".[viz,html,docs]"
+make docs-single-version
 ```
+
+Then open `docs/_build/index.html` in your browser. Sphinx only rebuilds the pages that changed: delete
+`docs/_build` to force a full rebuild. For a live preview while you write, `pip install sphinx-autobuild` and run
+`sphinx-autobuild docs/source docs/_build`.
+
+`make docs` builds all the deployed versions (see `build.sh`), as the CI does.
+
+## Structure
+
+| Folder | Section | Content |
+| --- | --- | --- |
+| `source/index.md` | Landing page | Hero, features and entry points |
+| `source/getting_started/` | Get started | Installation, quickstart, core concepts |
+| `source/user_guide/` | User guide | One page per feature, task-oriented |
+| `source/models/` | Model zoo | Architectures, benchmarks and vocabularies |
+| `source/training/` | Training | Datasets, training scripts, custom and shared models |
+| `source/deployment/` | Deployment | Performance, ONNX, Docker, web API, configuration |
+| `source/api/` | API reference | Generated from the docstrings with `autodoc` |
+| `source/community/` | Community | Help, tools, resources, contributing and changelog |
+| `source/_static/` | | CSS (`css/doctr.css`), JS, fonts, images and the version switcher (`switcher.json`) |
+| `source/_templates/` | | Footer components |
+| `source/_includes/` | | Snippets included in several pages (pipeline diagram) |
+
+Each section has an `index.md` with an overview and the `toctree` of its pages. The top navigation bar lists the
+sections of the `toctree` of `source/index.md`.
+
+## Writing pages
+
+Pages are written in [MyST Markdown](https://myst-parser.readthedocs.io/en/latest/syntax/typography.html), a superset
+of CommonMark that supports all Sphinx features:
+
+````markdown
+# Page title
+
+A first paragraph that says what the page is about: it is displayed as a lead.
+
+## Section
+
+Link to another page: [](../user_guide/exporting.md), or to a section: [](exporting.md#figures).
+Link to the API: {func}`~doctr.models.ocr_predictor`, {class}`~doctr.io.Document`.
+
+```python
+from doctr.models import ocr_predictor
+```
+
+:::{tip}
+Admonitions: note, tip, important, warning, seealso, dropdown...
+:::
+
+::::{tab-set}
+:::{tab-item} pip
+...
+:::
+::::
+````
+
+Guidelines:
+
+- **Show, then explain**: start sections with a short runnable snippet, then describe the options.
+- **Keep snippets runnable**: complete imports, realistic file names, and real outputs. The outputs shown in the
+  guides come from `source/_static/images/examples/invoice.pdf`.
+- **One topic per page**, and link to the related pages instead of repeating them.
+- **Cards** (`grid-item-card`) are for navigation, **tabs** for alternatives (e.g. CPU / GPU), **dropdowns** for
+  details most readers can skip.
+- Document new public functions and classes in the docstrings (Google style), and add them to the matching page of
+  `source/api/`.
+
+## Releasing a new version
+
+1. Add the release to `source/changelog.md`.
+2. Add it to `source/_static/switcher.json` and move the `(stable)` label and `"preferred": true` to it.
+3. In `build.sh`, build the previous stable release into its own folder and the new one at the root.

docs/build.sh

@@ -1,40 +1,56 @@
+# Build the multi-version documentation deployed on https://mindee.github.io/doctr
+#
+#   build/          -> latest stable release
+#   build/latest/   -> main branch
+#   build/vX.Y.Z/   -> previous releases
+#
+# Every version is rendered with the configuration, theme and static files of the main branch, so that they all share
+# the same look and version switcher (`source/_static/switcher.json`, to update at each release).
+#
+# deploy_doc <commit> <folder> <version>
+#   commit:  commit to build ("" for the current one)
+#   folder:  output folder in build/ ("latest", "vX.Y.Z", or "" for the stable version at the root)
+#   version: version shown as selected in the version switcher (must match an entry of switcher.json)
 function deploy_doc(){
-    if [ ! -z "$1" ]
-    then
-        git checkout $1
+    local commit=$1 folder=$2 version=$3
+    if [ -n "$commit" ]; then
+        git checkout "$commit"
     fi
-    COMMIT=$(git rev-parse --short HEAD)
-    echo "Creating doc at commit" $COMMIT "and pushing to folder $2"
+    echo "Creating doc at commit $(git rev-parse --short HEAD) (version: $version)"
     pip install -U ..
-    if [ ! -z "$2" ]
-    then
-        if [ "$2" == "latest" ]; then
-            echo "Pushing main"
-            sphinx-build source _build -a && mkdir build && mkdir build/$2 && cp -a _build/* build/$2/
-        elif [ -d build/$2 ]; then
-            echo "Directory" $2 "already exists"
-        else
-            echo "Pushing version" $2
-            cp -r _static source/ && cp _conf.py source/conf.py
-            sphinx-build source _build -a
-            mkdir build/$2 && cp -a _build/* build/$2/ && git checkout source/ && git clean -f source/
-        fi
+
+    # Older commits are rendered with the current configuration and assets
+    local legacy=0
+    if [ -n "$commit" ]; then
+        legacy=$([ -f source/index.md ] && echo 0 || echo 1)
+        rm -rf source/_static source/_templates
+        cp -r _static _templates source/ && cp _conf.py source/conf.py
+    fi
+
+    local out=build
+    if [ -n "$folder" ]; then
+        out=build/$folder
+    fi
+    if [ -d "$out" ] && [ -n "$folder" ]; then
+        echo "Directory $out already exists"
     else
-        echo "Pushing stable"
-        cp -r _static source/ && cp _conf.py source/conf.py
-        sphinx-build source build -a && git checkout source/ && git clean -f source/
+        DOCTR_DOCS_VERSION=$version DOCTR_DOCS_LEGACY=$legacy sphinx-build source _build -a
+        mkdir -p "$out" && cp -a _build/. "$out/" && rm -rf _build
+    fi
+
+    if [ -n "$commit" ]; then
+        git checkout source/ && git clean -fd source/
     fi
 }
 
 # You can find the commit for each tag on https://github.com/mindee/doctr/tags
 if [ -d build ]; then rm -Rf build; fi
-cp -r source/_static .
-cp source/conf.py _conf.py
+cp -r source/_static source/_templates . && cp source/conf.py _conf.py
 git fetch --all --tags --unshallow
-deploy_doc "" latest
-deploy_doc "1c9ce92" v0.11.0
-deploy_doc "97d4006" v0.12.0
-deploy_doc "7dabbe1" v1.0.0
-deploy_doc "6541a6e" v1.0.1
-deploy_doc "2d8e244" # v1.1.0 Latest stable release
-rm -rf _build _static _conf.py
+deploy_doc "" latest latest
+deploy_doc "1c9ce92" v0.11.0 v0.11.0
+deploy_doc "97d4006" v0.12.0 v0.12.0
+deploy_doc "7dabbe1" v1.0.0 v1.0.0
+deploy_doc "6541a6e" v1.0.1 v1.0.1
+deploy_doc "2d8e244" "" v1.1.0  # Latest stable release
+rm -rf _build _static _templates _conf.py

docs/source/_includes/pipeline.html

@@ -0,0 +1,67 @@
+<div class="doctr-diagram" role="img" aria-label="docTR pipeline: your files go through text detection and text recognition, are assembled by the document builder into a Document. Optional stages: orientation handling, layout analysis and table structure recognition.">
+<svg viewBox="0 0 1000 300" xmlns="http://www.w3.org/2000/svg">
+  <defs>
+    <marker id="doctr-ah" viewBox="0 0 10 10" refX="8" refY="5" markerWidth="7" markerHeight="7" orient="auto-start-reverse">
+      <path class="arrow-head" d="M 0 0 L 10 5 L 0 10 z"/>
+    </marker>
+  </defs>
+  <!-- Main pipeline -->
+  <rect class="box" x="10" y="30" width="130" height="92" rx="12"/>
+  <text class="title" x="75" y="62" text-anchor="middle">Your files</text>
+  <text class="sub" x="75" y="84" text-anchor="middle">PDF · images</text>
+  <text class="sub" x="75" y="101" text-anchor="middle">web pages</text>
+
+  <rect class="box core" x="180" y="30" width="165" height="92" rx="12"/>
+  <text class="title" x="262" y="60" text-anchor="middle">Text detection</text>
+  <text class="sub" x="262" y="82" text-anchor="middle">Localize every word</text>
+  <text class="mono" x="262" y="104" text-anchor="middle">det_arch</text>
+
+  <rect class="box core" x="385" y="30" width="165" height="92" rx="12"/>
+  <text class="title" x="467" y="60" text-anchor="middle">Text recognition</text>
+  <text class="sub" x="467" y="82" text-anchor="middle">Read each word crop</text>
+  <text class="mono" x="467" y="104" text-anchor="middle">reco_arch</text>
+
+  <rect class="box" x="590" y="30" width="165" height="92" rx="12"/>
+  <text class="title" x="672" y="60" text-anchor="middle">Document builder</text>
+  <text class="sub" x="672" y="82" text-anchor="middle">Lines, blocks and</text>
+  <text class="sub" x="672" y="99" text-anchor="middle">reading order</text>
+
+  <rect class="box out" x="795" y="30" width="195" height="92" rx="12"/>
+  <text class="title on-dark" x="892" y="60" text-anchor="middle">Document</text>
+  <text class="sub on-dark" x="892" y="82" text-anchor="middle">JSON · text · Markdown</text>
+  <text class="sub on-dark" x="892" y="99" text-anchor="middle">HTML · AsciiDoc · hOCR</text>
+
+  <line class="arrow" x1="140" y1="76" x2="176" y2="76" marker-end="url(#doctr-ah)"/>
+  <line class="arrow" x1="345" y1="76" x2="381" y2="76" marker-end="url(#doctr-ah)"/>
+  <line class="arrow" x1="550" y1="76" x2="586" y2="76" marker-end="url(#doctr-ah)"/>
+  <line class="arrow" x1="755" y1="76" x2="791" y2="76" marker-end="url(#doctr-ah)"/>
+
+  <!-- Optional stages -->
+  <rect class="box opt" x="180" y="182" width="370" height="80" rx="12"/>
+  <text class="title" x="365" y="210" text-anchor="middle">Orientation handling</text>
+  <text class="sub" x="365" y="230" text-anchor="middle">Straighten rotated pages and word crops</text>
+  <text class="mono" x="365" y="249" text-anchor="middle">assume_straight_pages=False</text>
+
+  <rect class="box opt" x="590" y="182" width="165" height="80" rx="12"/>
+  <text class="title" x="672" y="210" text-anchor="middle">Layout analysis</text>
+  <text class="sub" x="672" y="230" text-anchor="middle">Titles, tables, figures…</text>
+  <text class="mono" x="672" y="249" text-anchor="middle">detect_layout=True</text>
+
+  <rect class="box opt" x="795" y="182" width="195" height="80" rx="12"/>
+  <text class="title" x="892" y="210" text-anchor="middle">Table structure</text>
+  <text class="sub" x="892" y="230" text-anchor="middle">Rows, columns and cells</text>
+  <text class="mono" x="892" y="249" text-anchor="middle">detect_tables=True</text>
+
+  <line class="arrow" x1="262" y1="182" x2="262" y2="126" marker-end="url(#doctr-ah)" stroke-dasharray="4 4"/>
+  <line class="arrow" x1="467" y1="182" x2="467" y2="126" marker-end="url(#doctr-ah)" stroke-dasharray="4 4"/>
+  <line class="arrow" x1="672" y1="182" x2="672" y2="126" marker-end="url(#doctr-ah)" stroke-dasharray="4 4"/>
+  <line class="arrow" x1="892" y1="182" x2="892" y2="126" marker-end="url(#doctr-ah)" stroke-dasharray="4 4"/>
+  <line class="arrow" x1="755" y1="222" x2="791" y2="222" marker-end="url(#doctr-ah)" stroke-dasharray="4 4"/>
+
+  <!-- Legend -->
+  <line class="arrow" x1="10" y1="200" x2="40" y2="200"/>
+  <text class="legend" x="48" y="204">always runs</text>
+  <line class="arrow" x1="10" y1="226" x2="40" y2="226" stroke-dasharray="4 4"/>
+  <text class="legend" x="48" y="230">optional</text>
+</svg>
+</div>