Bundle and minify static CSS/JS with django-compressor

Containerfile

@@ -25,6 +25,10 @@ COPY . .
 
 ENV PATH="/app/.venv/bin:$PATH"
 
+# Release id (e.g. git short SHA); selects this image's django-compressor offline manifest.
+ARG METRON_RELEASE=dev
+ENV METRON_RELEASE=$METRON_RELEASE
+
 # Compile translation catalogs into the image. Uses django-admin (not
 # manage.py) so it doesn't require settings/env vars (DB, SECRET_KEY, etc.)
 # to be configured at build time. gettext (msgfmt) is only needed here, so

DEPLOYMENT.md

@@ -38,6 +38,9 @@ with Quadlet (systemd-managed containers).
   - [Manual backup](#manual-backup)
   - [Automated backups with a systemd timer](#automated-backups-with-a-systemd-timer)
   - [Restoring from a backup](#restoring-from-a-backup)
+- [Upgrading PostgreSQL](#upgrading-postgresql)
+  - [Minor version updates](#minor-version-updates)
+  - [Major version upgrade (16 → 18)](#major-version-upgrade-16--18)
 - [Enable persistent journals](#enable-persistent-journals)
 - [Fail2ban](#fail2ban)
   - [Install](#install)
@@ -322,7 +325,8 @@ systemctl --user list-timers certbot-renew.timer
 
 ```bash
 cd ~/metron
-podman build -t localhost/metron:latest .
+podman build --build-arg METRON_RELEASE=$(git rev-parse --short HEAD) \
+    -t localhost/metron:latest .
 ```
 
 ---
@@ -411,11 +415,21 @@ systemctl --user start metron-nginx
 > ```bash
 > podman exec metron-web python manage.py migrate
 > podman exec metron-web python manage.py collectstatic --no-input
+> podman exec metron-web python manage.py compress --force
+> systemctl --user restart metron-web
 > ```
 >
 > When restoring from an existing droplet the schema and static files are
 > already in place — `migrate` is a no-op and static files live on
-> DigitalOcean Spaces, so `collectstatic` is not needed either.
+> DigitalOcean Spaces, so `collectstatic` is not needed either. Run
+> `compress` anyway: the image's release manifest
+> (`static/CACHE/manifest-<METRON_RELEASE>.json`) won't be in the bucket yet.
+>
+> `compress` builds the minified CSS/JS bundles for every `{% compress %}`
+> block in the templates (django-compressor, offline mode) and uploads them
+> plus `static/CACHE/manifest-<METRON_RELEASE>.json` to Spaces. Each gunicorn worker caches
+> the manifest in memory, so restart `metron-web` after compressing. A page
+> whose bundle is missing from the manifest fails with `OfflineGenerationError`.
 
 The container services start automatically on boot without needing
 `systemctl enable` — the `WantedBy=default.target` line in each `.container`
@@ -541,14 +555,33 @@ git pull
 cp ~/metron/.quadlet/* ~/.config/containers/systemd/
 systemctl --user daemon-reload
 
-podman build -t localhost/metron:latest .
+podman build --build-arg METRON_RELEASE=$(git rev-parse --short HEAD) \
+    -t localhost/metron:latest .
+
+# Re-run collectstatic only if static assets changed:
+# podman run --rm --network host --env-file ~/.config/containers/metron.env \
+#     localhost/metron:latest python manage.py collectstatic --no-input
+
+# Build the CSS/JS bundles and this release's manifest from the *new* image before
+# restarting. Each release gets its own manifest-<sha>.json, so containers still running
+# the previous release keep reading their own manifest.
+podman run --rm --network host --env-file ~/.config/containers/metron.env \
+    localhost/metron:latest python manage.py compress --force
+
 systemctl --user restart metron-web metron-anubis
 
 podman exec metron-web python manage.py migrate
-# Re-run collectstatic only if static assets changed:
-# podman exec metron-web python manage.py collectstatic --no-input
 ```
 
+> **Why `compress` every deploy:** production uses django-compressor in offline
+> mode. If a template's `{% compress %}` block or any bundled CSS/JS file changes
+> and `compress` isn't re-run, those pages fail with `OfflineGenerationError`.
+> It is cheap to run, so run it on every deploy. The manifest name includes the
+> `METRON_RELEASE` build arg, so always pass `--build-arg METRON_RELEASE=...` when
+> building; rolling back to an earlier image reuses that image's manifest, which
+> is still in the bucket. (The "Invalid template
+> django_filters/rest_framework/crispy_form.html" warning it prints is harmless.)
+
 ---
 
 ## History cleanup
@@ -795,6 +828,171 @@ systemctl --user start metron-web
 
 ---
 
+## Upgrading PostgreSQL
+
+### Minor version updates
+
+Minor releases (e.g. 16.4 → 16.10) never change the on-disk format, so updating
+is just a pull and restart:
+
+```bash
+podman pull docker.io/library/postgres:16
+systemctl --user restart metron-postgres
+podman exec metron-postgres postgres --version
+```
+
+### Major version upgrade (16 → 18)
+
+A new major version uses a different data-directory format, and `postgres:18`
+will not start on a volume initialized by 16, so changing the `Image=` tag alone
+does not work. The procedure below dumps the database, starts 18 on a **new**
+volume, and restores into it. The old volume is left in place as the rollback.
+Restoring from a dump also rebuilds every index, which avoids the collation
+problems that can come with a newer glibc in the new image.
+
+Two things are different from a normal restore:
+
+1. **The 18 image moved its data directory.** `PGDATA` is now
+   `/var/lib/postgresql/18/docker`, and the volume must be mounted at
+   `/var/lib/postgresql`, not `/var/lib/postgresql/data`.
+2. **The `unaccent` change must be redone before restoring.** Migration
+   `0050` runs `ALTER FUNCTION unaccent(text) IMMUTABLE`. `unaccent(text)`
+   belongs to the `unaccent` extension, and `pg_dump` does not save changes
+   made to extension members. A fresh database therefore gets the stock
+   STABLE `unaccent()`, and the five `*_name_unaccent_trgm_idx` indexes fail
+   with *"functions in index expression must be marked IMMUTABLE"*. `pg_restore`
+   logs the error and keeps going, so the site comes up with slow name
+   searches and nothing obviously broken. Step 6 below handles this.
+
+Expect downtime from step 2 until step 9. Rehearse the restore locally against
+a production dump first: it shows how long the restore takes and catches problems
+before you're working on production.
+
+**1. Record a baseline** to compare against after the restore:
+
+```bash
+podman exec metron-postgres psql -U {db_username} -d metron -Atc \
+  "SELECT count(*) FROM pg_indexes WHERE schemaname = 'public';"
+podman exec metron-postgres psql -U {db_username} -d metron -Atc \
+  "SELECT count(*) FROM comicsdb_issue;"
+```
+
+**2. Stop everything that writes to the database**, including the timers:
+
+```bash
+systemctl --user stop metron-nginx metron-anubis metron-web
+systemctl --user stop metron-backup.timer metron-history-cleanup.timer \
+    metron-opencollective-sync.timer
+```
+
+**3. Take the final dump.** Run the backup script first so there is also an
+off-site copy in Spaces:
+
+```bash
+bash ~/metron/scripts/backup.sh
+podman exec metron-postgres pg_dump -U {db_username} -Fc metron -f /tmp/final.dump
+podman cp metron-postgres:/tmp/final.dump ~/backups/metron-pg16-final.dump
+```
+
+**4. Stop Postgres 16.** Do not remove its volume (`systemd-metron-postgres`):
+it is the rollback.
+
+```bash
+systemctl --user stop metron-postgres
+```
+
+**5. Point the quadlet at 18 and a new volume.** Create the volume unit:
+
+```bash
+printf '[Volume]\n' > ~/.config/containers/systemd/metron-postgres18.volume
+```
+
+Then edit `~/.config/containers/systemd/metron-postgres.container` and change
+these two lines:
+
+```ini
+Image=docker.io/library/postgres:18
+Volume=metron-postgres18.volume:/var/lib/postgresql
+```
+
+Make the same change in the repo's `.quadlet/` files, but merge it only when you
+do the upgrade. Otherwise the next routine deploy (`cp ~/metron/.quadlet/* ...`)
+would switch Postgres over by accident.
+
+```bash
+systemctl --user daemon-reload
+systemctl --user start metron-postgres     # initializes the empty volume
+podman exec metron-postgres postgres --version
+```
+
+**6. Prepare the empty database before restoring.** The dump uses
+`CREATE EXTENSION IF NOT EXISTS`, so creating the extensions first doesn't
+conflict with the restore:
+
+```bash
+podman exec metron-postgres psql -U {db_username} -d metron -c "
+  CREATE EXTENSION IF NOT EXISTS unaccent;
+  CREATE EXTENSION IF NOT EXISTS pg_trgm;
+  ALTER FUNCTION unaccent(text) IMMUTABLE;"
+```
+
+**7. Restore, then rebuild planner statistics.** A dump does not carry
+statistics, and queries will be slow until `ANALYZE` has run:
+
+```bash
+podman cp ~/backups/metron-pg16-final.dump metron-postgres:/tmp/final.dump
+podman exec metron-postgres pg_restore \
+  --username {db_username} \
+  --dbname metron \
+  --no-owner \
+  --role {db_username} \
+  /tmp/final.dump 2>&1 | tee ~/pg18-restore.log
+
+grep -i error ~/pg18-restore.log            # should print nothing
+podman exec metron-postgres vacuumdb -U {db_username} -d metron --analyze-in-stages
+podman exec metron-postgres rm /tmp/final.dump
+```
+
+**8. Verify** that both counts match the baseline from step 1, and that the
+`unaccent` indexes exist:
+
+```bash
+podman exec metron-postgres psql -U {db_username} -d metron -Atc \
+  "SELECT count(*) FROM pg_indexes WHERE schemaname = 'public';"
+podman exec metron-postgres psql -U {db_username} -d metron -Atc \
+  "SELECT count(*) FROM comicsdb_issue;"
+podman exec metron-postgres psql -U {db_username} -d metron -Atc \
+  "SELECT indexname FROM pg_indexes WHERE indexname LIKE '%unaccent_trgm_idx';"
+```
+
+**9. Bring the site back** and restart the timers:
+
+```bash
+systemctl --user start metron-web metron-anubis metron-nginx
+systemctl --user start metron-backup.timer metron-history-cleanup.timer \
+    metron-opencollective-sync.timer
+```
+
+Check that the site loads, that a name search with an accented name works
+(e.g. "Pokémon"), and that `journalctl --user -u metron-web -n 50` is free of
+database errors. Run the backup service once (`systemctl --user start
+metron-backup`) to confirm backups work against 18.
+
+**Rollback:** stop `metron-postgres`, put the old `Image=` and `Volume=` lines
+back (`postgres:16`, `metron-postgres.volume:/var/lib/postgresql/data`), run
+`systemctl --user daemon-reload`, and start it again. Anything written after the
+cutover is lost, so decide early.
+
+**Cleanup:** after a week or two without problems, remove the old volume and
+image:
+
+```bash
+podman volume rm systemd-metron-postgres
+podman rmi docker.io/library/postgres:16
+```
+
+---
+
 ## Enable persistent journals
 
 On CentOS, journal logs are not persisted across reboots by default. Run this

comicsdb/templates/comicsdb/home.html

@@ -1,8 +1,10 @@
 {% extends parent_template|default:"base.html" %}
-{% load i18n static %}
+{% load i18n static compress %}
 {% load cache %}
 {% block headcss %}
-    <link rel="stylesheet" href="{% static 'site/css/home/index.css' %}">
+    {% compress css %}
+        <link rel="stylesheet" href="{% static 'site/css/home/index.css' %}">
+    {% endcompress %}
 {% endblock %}
 {% block sitedesc %}
     {% trans "Metron is a community-based comic book database with a public REST API. Search and discover comic books with comprehensive metadata." as home_description %}

comicsdb/templates/comicsdb/issue_form.html

@@ -1,15 +1,19 @@
 {% extends parent_template|default:"base.html" %}
-{% load i18n static %}
+{% load i18n static compress %}
 {% load bulma_tags %}
 {% block title %}
     {% trans "Issue Form" %}
 {% endblock title %}
 {% block headcss %}
-    <link rel="stylesheet"
-          href="{% static 'site/css/bulma-calendar.min.css' %}">
-    {# Sticky bar, section nav, grouped fields #}
-    <link rel="stylesheet"
-          href="{% static 'site/css/issue-form-usability.css' %}">
+    {% compress css %}
+        <link rel="stylesheet"
+              href="{% static 'site/css/bulma-calendar.min.css' %}">
+    {% endcompress %}
+    {% compress css %}
+        {# Sticky bar, section nav, grouped fields #}
+        <link rel="stylesheet"
+              href="{% static 'site/css/issue-form-usability.css' %}">
+    {% endcompress %}
 {% endblock %}
 {% block content %}
     <form method="post" enctype="multipart/form-data" id="issueForm" novalidate>
@@ -143,7 +147,13 @@ <h2 class="title is-5">{% trans "Attribution" %}</h2>
     {% endif %}
 {% endblock %}
 {% block js %}
-    <script src="{% static 'extensions/bulma-calendar/dist/js/bulma-calendar.min.js' %}"></script>
+    {% compress js %}
+        <script src="{% static 'extensions/bulma-calendar/dist/js/bulma-calendar.min.js' %}"></script>
+    {% endcompress %}
+    {% compress js %}
+        {# Usability behaviour (section nav active-state); independent of the inline script below #}
+        <script src="{% static 'site/js/issue-form-usability.js' %}"></script>
+    {% endcompress %}
     <script>
 document.addEventListener('DOMContentLoaded', function() {
     'use strict';
@@ -250,6 +260,4 @@ <h2 class="title is-5">{% trans "Attribution" %}</h2>
 }
 
     </script>
-    {# Usability behaviour — section nav active-state #}
-    <script src="{% static 'site/js/issue-form-usability.js' %}"></script>
 {% endblock %}

comicsdb/templates/comicsdb/model_with_attribution_form.html

@@ -1,16 +1,20 @@
 {% extends parent_template|default:"base.html" %}
-{% load i18n static %}
+{% load i18n static compress %}
 {% load bulma_tags %}
 {% block title %}
     {% trans "Form" as default_title %}
     {{ title|default:default_title }}
 {% endblock title %}
 {% block headcss %}
-    <link rel="stylesheet"
-          href="{% static 'site/css/bulma-calendar.min.css' %}">
-    {# usability styles — shared with issue_form.html #}
-    <link rel="stylesheet"
-          href="{% static 'site/css/issue-form-usability.css' %}">
+    {% compress css %}
+        <link rel="stylesheet"
+              href="{% static 'site/css/bulma-calendar.min.css' %}">
+    {% endcompress %}
+    {% compress css %}
+        {# usability styles — shared with issue_form.html #}
+        <link rel="stylesheet"
+              href="{% static 'site/css/issue-form-usability.css' %}">
+    {% endcompress %}
 {% endblock %}
 {% block content %}
     {% trans "Form" as default_title %}
@@ -53,7 +57,13 @@ <h2 class="title is-5">{% trans "Attribution" %}</h2>
     </form>
 {% endblock %}
 {% block js %}
-    <script src="{% static 'extensions/bulma-calendar/dist/js/bulma-calendar.min.js' %}"></script>
+    {% compress js %}
+        <script src="{% static 'extensions/bulma-calendar/dist/js/bulma-calendar.min.js' %}"></script>
+    {% endcompress %}
+    {% compress js %}
+        {# Usability behaviour (section nav active-state); independent of the inline script below #}
+        <script src="{% static 'site/js/issue-form-usability.js' %}"></script>
+    {% endcompress %}
     <script>
 document.addEventListener('DOMContentLoaded', function() {
     'use strict';
@@ -110,6 +120,4 @@ <h2 class="title is-5">{% trans "Attribution" %}</h2>
 }
 
     </script>
-    {# Tier 3 usability behaviour — shared with issue_form.html (section nav; role chips no-op here) #}
-    <script src="{% static 'site/js/issue-form-usability.js' %}"></script>
 {% endblock %}

comicsdb/templates/comicsdb/partials/issue_filter.html

@@ -1,4 +1,4 @@
-{% load i18n range_tags static %}
+{% load i18n range_tags static compress %}
 {% comment %}
     USABILITY CHANGE: the ~14 advanced filter fields are now organised
     under labelled sub-sections (Series / Issue Numbers / Dates / Codes & IDs)
@@ -446,8 +446,12 @@ <h4 class="title is-6 has-text-grey mt-4 mb-3">{% trans "Description" %}</h4>
         </div>
     </form>
 </div>
-<link rel="stylesheet" href="{% static 'site/css/bulma-calendar.min.css' %}">
-<script src="{% static 'extensions/bulma-calendar/dist/js/bulma-calendar.min.js' %}"></script>
+{% compress css %}
+    <link rel="stylesheet" href="{% static 'site/css/bulma-calendar.min.css' %}">
+{% endcompress %}
+{% compress js %}
+    <script src="{% static 'extensions/bulma-calendar/dist/js/bulma-calendar.min.js' %}"></script>
+{% endcompress %}
 <script>
 document.addEventListener('DOMContentLoaded', function() {
     'use strict';