-
Notifications
You must be signed in to change notification settings - Fork 1
Add Dokploy deployment configuration #51
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,18 @@ | ||
| .git | ||
| .github | ||
| .env | ||
| .env.* | ||
| **/.env | ||
| **/.env.* | ||
| venv* | ||
| .venv* | ||
| **/__pycache__ | ||
| *.pyc | ||
| db | ||
| *.sqlite3 | ||
| **/*.sqlite3 | ||
| files | ||
| static | ||
| testing/bck | ||
| node_modules | ||
| docs/_build |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,27 @@ | ||
| FROM python:3.13-slim-trixie@sha256:9d2e5553305c7c7b0097999bb17187c69b921ccd6bc9d40e4bb5ebe652c00285 AS app | ||
| ENV PYTHONUNBUFFERED=1 PYTHONDONTWRITEBYTECODE=1 DEBUG=0 \ | ||
| ALLOWED_HOSTS=testing.nc3.lu PKGVER=GitHub-0-gmain | ||
| WORKDIR /app | ||
| RUN apt-get update && apt-get install -y --no-install-recommends \ | ||
| build-essential libffi-dev libxml2-dev libxslt1-dev \ | ||
| libpango-1.0-0 libpangoft2-1.0-0 libcairo2 libgdk-pixbuf-2.0-0 \ | ||
| shared-mime-info fonts-dejavu-core iputils-ping nmap bind9-dnsutils whois \ | ||
| openssl ca-certificates \ | ||
| && rm -rf /var/lib/apt/lists/* | ||
| COPY requirements.txt deploy/gunicorn-requirement.txt /tmp/ | ||
| RUN pip install --no-cache-dir --require-hashes -r /tmp/requirements.txt \ | ||
| && pip install --no-cache-dir --require-hashes --no-deps -r /tmp/gunicorn-requirement.txt | ||
| COPY . /app/ | ||
| COPY deploy/start.sh /app/start.sh | ||
| RUN chmod 755 /app/start.sh \ | ||
| && mkdir -p /app/db /app/files \ | ||
| && mkdir -p /run/testingplatform && chown www-data:www-data /run/testingplatform \ | ||
| && DEBUG=1 python manage.py collectstatic --noinput \ | ||
| && test ! -e /app/db/db.sqlite3 | ||
| ENV DJANGO_SETTINGS_MODULE=deploy.settings | ||
| USER www-data | ||
| ENTRYPOINT ["/app/start.sh"] | ||
|
|
||
| FROM nginx:stable-alpine@sha256:dc5069ad14f19660b141b21236140b91656bf89bbc3e2417c70ae650cd66104c AS proxy | ||
| COPY --from=app /app/static/ /srv/testing-static/ | ||
| COPY deploy/nginx.conf /etc/nginx/conf.d/default.conf |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,51 @@ | ||
| # Dokploy deployment | ||
|
|
||
| Production is built from `NC3-LU/TestingPlatform`, branch `main`, using | ||
| `deploy/compose.yml` on `testingplatformprodvm2`. Configure the existing NC3 GitHub | ||
| provider in Dokploy and enable automatic deployment for pushes to `main`. | ||
|
|
||
| Set a private `SECRET_KEY` in Dokploy. Keep it stable across deployments. | ||
| Set `TESTING_DATA_ROOT` in Dokploy to an existing absolute host directory containing | ||
| `db/db.sqlite3` and `files/`. These directories must be readable and writable by | ||
| UID/GID 33 (`www-data`). Keep this directory outside the Git checkout. Bind mounts | ||
| refuse missing host paths, startup refuses a missing database, and SQLite opens in | ||
| `mode=rw` so a missing database cannot be silently recreated. | ||
|
|
||
| **Never delete the original database, database copies, uploads, or backups.** | ||
| The migration uses a separate, verified SQLite backup and a separate uploads copy. | ||
| Application rebuilds and container replacements reuse the same persistent paths. | ||
| Do not use `down -v`, volume pruning, database resets, or flush commands. | ||
| Database migrations are intentionally not run on startup. Schema changes need a | ||
| reviewed migration and a verified backup before the corresponding code is deployed. | ||
|
|
||
| Traefik on the Dokploy remote terminates HTTPS and manages certificate renewal. | ||
| In the Compose service's Dokploy Domains settings, route `testing.nc3.lu`, path | ||
| `/`, to service `proxy`, port `80`, with HTTPS and the `letsencrypt` resolver. | ||
| The nginx container joins `dokploy-network`, serves the built static assets and | ||
| forwards Django requests over the private `runtime` volume's Unix socket. | ||
| Its loopback-only `127.0.0.1:18080` mapping supports local health checks. | ||
|
|
||
| The Django container retains host networking for IPv6 network tests and local | ||
| SMTP access, but Gunicorn has no TCP listener. Only the web and nginx containers | ||
| mount the socket volume. Traefik sets `X-Forwarded-Proto`, nginx preserves it, | ||
| and the deployment settings recognize public HTTPS for generated links and CSRF. | ||
| Apache and its former mod_wsgi application are stopped and disabled at cutover; | ||
| neither is part of the production request path afterward. | ||
|
|
||
| Mail settings (`EMAIL_HOST`, `EMAIL_PORT`, `EMAIL_USE_TLS`, `EMAIL_HOST_USER`, | ||
| `EMAIL_HOST_PASSWORD`, `DEFAULT_FROM_EMAIL`) are supplied privately in Dokploy. | ||
| The original service had no active Django Q worker. This deployment preserves that | ||
| state; enabling a worker requires reviewing the existing scheduled tasks first. | ||
|
|
||
| Validate changes with: | ||
|
|
||
| ```sh | ||
| DJANGO_SETTINGS_MODULE=deploy.settings python -m unittest discover -s deploy/tests -v | ||
| docker compose -f deploy/compose.yml config --quiet | ||
| ``` | ||
|
|
||
| Before cutover, test on a separate database/uploads copy, check database integrity | ||
| and table counts, exercise non-mutating application routes, and compare uploads. | ||
| After stopping legacy writes, create a fresh final backup and production copy. | ||
| Keep the original database and source for recovery. If the new application has | ||
| accepted writes, recovery must preserve those newer writes before switching back. |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,77 @@ | ||
| services: | ||
| web: | ||
| build: | ||
| context: .. | ||
| dockerfile: deploy/Dockerfile | ||
| target: app | ||
| # Preserve the existing host IPv6 routing used by network tests. | ||
| # Gunicorn serves a private socket; Traefik provides public HTTPS. | ||
| network_mode: host | ||
| restart: unless-stopped | ||
| init: true | ||
| stop_grace_period: 340s | ||
| environment: | ||
| DEBUG: "0" | ||
| SECRET_KEY: ${SECRET_KEY:?Set the production Django signing key in Dokploy} | ||
| ALLOWED_HOSTS: testing.nc3.lu,localhost | ||
| EMAIL_HOST: ${EMAIL_HOST:-localhost} | ||
| EMAIL_PORT: ${EMAIL_PORT:-25} | ||
| EMAIL_USE_TLS: ${EMAIL_USE_TLS:-0} | ||
| EMAIL_HOST_USER: ${EMAIL_HOST_USER:-} | ||
| EMAIL_HOST_PASSWORD: ${EMAIL_HOST_PASSWORD:-} | ||
| DEFAULT_FROM_EMAIL: ${DEFAULT_FROM_EMAIL:-webmaster@localhost} | ||
| volumes: | ||
| - type: bind | ||
| source: ${TESTING_DATA_ROOT:?Set TESTING_DATA_ROOT to the prepared database and uploads directory}/db | ||
| target: /app/db | ||
| bind: | ||
| create_host_path: false | ||
| - type: bind | ||
| source: ${TESTING_DATA_ROOT:?Set TESTING_DATA_ROOT to the prepared database and uploads directory}/files | ||
| target: /app/files | ||
| bind: | ||
| create_host_path: false | ||
| - runtime:/run/testingplatform | ||
| healthcheck: | ||
| test: ["CMD", "python", "-c", "import socket; s=socket.socket(socket.AF_UNIX); s.settimeout(10); s.connect('/run/testingplatform/gunicorn.sock'); s.sendall(b'GET / HTTP/1.0\\r\\nHost: testing.nc3.lu\\r\\n\\r\\n'); assert b' 200 ' in s.recv(64)"] | ||
| interval: 30s | ||
| timeout: 15s | ||
| retries: 3 | ||
| start_period: 30s | ||
| logging: | ||
| driver: json-file | ||
| options: | ||
| max-size: 10m | ||
| max-file: "3" | ||
| proxy: | ||
| build: | ||
| context: .. | ||
| dockerfile: deploy/Dockerfile | ||
| target: proxy | ||
| networks: | ||
| - dokploy-network | ||
| ports: | ||
| - "127.0.0.1:18080:80" | ||
| volumes: | ||
| - runtime:/run/testingplatform:ro | ||
| restart: unless-stopped | ||
| depends_on: | ||
| web: | ||
| condition: service_healthy | ||
| healthcheck: | ||
| test: ["CMD", "wget", "-q", "--spider", "--header=Host: testing.nc3.lu", "http://127.0.0.1/"] | ||
| interval: 30s | ||
| timeout: 15s | ||
| retries: 3 | ||
| logging: | ||
| driver: json-file | ||
| options: | ||
| max-size: 10m | ||
| max-file: "3" | ||
|
|
||
| networks: | ||
| dokploy-network: | ||
| external: true | ||
|
|
||
| volumes: | ||
| runtime: | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| gunicorn==26.2.0 --hash=sha256:bd249d0b3f7972f7432f0a6b6ff3b3ee2d129f70cd1ff6c09a9dd9e29a2b88e3 |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,26 @@ | ||
| map $http_x_forwarded_proto $original_proto { | ||
| default $scheme; | ||
| https https; | ||
| } | ||
| upstream testingplatform_django { | ||
| server unix:/run/testingplatform/gunicorn.sock; | ||
| } | ||
| server { | ||
| listen 80; | ||
| server_name testing.nc3.lu; | ||
| client_max_body_size 100m; | ||
| location /static/ { | ||
| alias /srv/testing-static/; | ||
| access_log off; | ||
| expires 7d; | ||
| } | ||
| location / { | ||
| proxy_pass http://testingplatform_django; | ||
| proxy_set_header Host $host; | ||
| proxy_set_header X-Real-IP $remote_addr; | ||
| proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; | ||
| proxy_set_header X-Forwarded-Proto $original_proto; | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Configure Django to trust the forwarded HTTPS scheme When HTTPS terminates at a reverse proxy, Django needs the forwarded scheme to generate HTTPS absolute URLs, including password-reset links. Configure |
||
| proxy_read_timeout 340s; | ||
| proxy_connect_timeout 10s; | ||
| } | ||
| } | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,17 @@ | ||
| """Settings for the existing-database deployment managed by Dokploy.""" | ||
|
|
||
| from testing_platform.settings import * # noqa: F403 | ||
| from testing_platform.settings import BASE_DIR, DATABASES | ||
|
|
||
| # Traefik sets the scheme at the public edge; nginx preserves it on the private | ||
| # socket. The application has no public HTTP listener. | ||
| SECURE_PROXY_SSL_HEADER = ("HTTP_X_FORWARDED_PROTO", "https") | ||
|
|
||
| # SQLite must open an existing database, never silently create an empty one. | ||
| DATABASES = { | ||
| "default": { | ||
| **DATABASES["default"], | ||
| "NAME": (BASE_DIR / "db" / "db.sqlite3").as_uri() + "?mode=rw", | ||
| "OPTIONS": {"uri": True}, | ||
| } | ||
| } |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,11 @@ | ||
| #!/bin/sh | ||
| set -eu | ||
| cd "$(dirname "$0")" | ||
| if [ ! -s db/db.sqlite3 ]; then | ||
| echo 'Refusing to start without the existing database copy at db/db.sqlite3' >&2 | ||
| exit 78 | ||
| fi | ||
| exec python -m gunicorn testing_platform.wsgi:application \ | ||
| --bind unix:/run/testingplatform/gunicorn.sock --umask 0111 \ | ||
| --workers 1 --worker-class gthread --threads 2 \ | ||
| --timeout 330 --graceful-timeout 330 --access-logfile - --error-logfile - |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,33 @@ | ||
| import unittest | ||
|
|
||
| import django | ||
| from django.test import RequestFactory, override_settings | ||
|
|
||
| from deploy import settings as deployment_settings | ||
|
|
||
| django.setup() | ||
|
|
||
|
|
||
| class ProxySchemeTests(unittest.TestCase): | ||
| def setUp(self): | ||
| self.enterContext( | ||
| override_settings( | ||
| ALLOWED_HOSTS=["testing.nc3.lu"], | ||
| SECURE_PROXY_SSL_HEADER=getattr( | ||
| deployment_settings, "SECURE_PROXY_SSL_HEADER", None | ||
| ), | ||
| ) | ||
| ) | ||
|
|
||
| def test_public_https_generates_https_links_through_the_internal_http_proxy(self): | ||
| request = RequestFactory().get( | ||
| "/login/", | ||
| HTTP_HOST="testing.nc3.lu", | ||
| HTTP_X_FORWARDED_PROTO="https", | ||
| ) | ||
| self.assertTrue(request.is_secure()) | ||
| self.assertEqual(request.build_absolute_uri(), "https://testing.nc3.lu/login/") | ||
|
|
||
| def test_plain_internal_http_is_not_treated_as_https(self): | ||
| request = RequestFactory().get("/", HTTP_HOST="testing.nc3.lu") | ||
| self.assertFalse(request.is_secure()) |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,51 @@ | ||
| from pathlib import Path | ||
| import os, shutil, sqlite3, subprocess, tempfile, unittest | ||
|
|
||
|
|
||
| class StartGuardTests(unittest.TestCase): | ||
| def test_missing_database_refuses_start_without_creating_it(self): | ||
| script = Path(__file__).resolve().parents[1] / "start.sh" | ||
| self.assertTrue(script.exists(), "Startup guard must exist before deploying") | ||
| with tempfile.TemporaryDirectory(prefix="tp-start-check-") as d: | ||
| p = Path(d) | ||
| shutil.copy2(script, p / script.name) | ||
| r = subprocess.run( | ||
| ["bash", str(p / script.name)], capture_output=True, text=True | ||
| ) | ||
| self.assertNotEqual(r.returncode, 0) | ||
| self.assertIn("existing database", r.stderr) | ||
| self.assertFalse((p / "db/db.sqlite3").exists()) | ||
|
|
||
| def test_present_database_is_retained_and_start_does_not_run_migrations(self): | ||
| script = Path(__file__).resolve().parents[1] / "start.sh" | ||
| self.assertTrue(script.exists(), "Startup guard must exist before deploying") | ||
| with tempfile.TemporaryDirectory(prefix="tp-start-check-") as d: | ||
| p = Path(d) | ||
| shutil.copy2(script, p / script.name) | ||
| (p / "db").mkdir() | ||
| db = p / "db/db.sqlite3" | ||
| with sqlite3.connect(db) as c: | ||
| c.execute("CREATE TABLE retained(id INTEGER)") | ||
| before = db.read_bytes() | ||
| (p / "bin").mkdir(parents=True) | ||
| fake = p / "bin/python" | ||
| fake.write_text('#!/bin/sh\nprintf "%s\\n" "$@"\n') | ||
| fake.chmod(0o700) | ||
| r = subprocess.run( | ||
| ["bash", str(p / script.name)], | ||
| capture_output=True, | ||
| text=True, | ||
| env={ | ||
| **os.environ, | ||
| "PATH": str(p / "bin") + os.pathsep + os.environ["PATH"], | ||
| }, | ||
| ) | ||
| self.assertEqual(r.returncode, 0, r.stderr) | ||
| self.assertIn("gunicorn", r.stdout) | ||
| self.assertIn("unix:/run/testingplatform/gunicorn.sock", r.stdout) | ||
| self.assertNotIn("migrate", r.stdout) | ||
| self.assertEqual(db.read_bytes(), before) | ||
|
|
||
|
|
||
| if __name__ == "__main__": | ||
| unittest.main() |
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Forward required API settings into the container
The Compose service only injects variables listed in its environment configuration. If an enabled integration requires
DMARC_API_KEY, omitting it leaves the empty settings default and authenticated uploads can return HTTP 401. Pass required integration settings through the runtime environment and document the supported variables.