Skip to content

Setup resonate worker in ECS - #4824

Merged
marcoacierno merged 4 commits into
resonatefrom
resonate-ecs-setup
Oct 30, 2026
Merged

marcoacierno merged 4 commits into
resonatefrom
resonate-ecs-setup

Conversation

@marcoacierno

Copy link
Copy Markdown
Member
  • Build the Resonate client in one expression
  • Run Resonate and its worker on the ECS cluster

What

ToDo

@vercel

vercel Bot commented Sep 20, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
pycon Ready Ready Preview Oct 30, 2026 8:00pm UTC

@claude

claude Bot commented Sep 20, 2026 •

Copy link
Copy Markdown
Contributor

Adds a Resonate server (ECS service backed by a resonate Postgres database on the existing RDS instance) and a resonate_worker service running the backend's manage.py resonate_worker, plus a security-group opening for port 8001 and a RESONATE_URL env var shared by all backend tasks.

  • Swapped names in cluster/security.tf: in_resonate is an egress rule and out_resonate is an ingress rule, and the comment on out_resonate is attached to the wrong one. Both are needed, but the names and comment should match their types.
  • Plaintext DB credentials in the task definition: resonate/task.tf puts the full postgres://user:password@... URL in the container environment, so it shows up in the task definition and the console. Use secrets (SSM/Secrets Manager) if the other services do, or at least mark the value sensitive.
  • Database must exist beforehand: the resonate DB is not created anywhere. The service will crash-loop on first apply, and deployment_minimum_healthy_percent = 0 makes that visible only as downtime. Document the step or automate it.
  • Placeholder healthcheck on the worker: echo 1 always succeeds, so ECS never detects a dead worker. Check the worker process instead, or drop the healthCheck.
  • Resonate server has no auth and the image tag is pinned by version only: the server listens on 0.0.0.0:8001 with no auth, and the SG rule allows any member of the server SG to reach it. That is acceptable only if nothing public shares that SG. Check this.
  • Minor: systemControls with somaxconn and the fixed memoryReservation = 10 on non-prod were copied from the web task. The worker doesn't need the first, and 10 MB is unrealistically low for a Python process that holds an open connection.
  • Comment style: several new comments in the PR explain rationale at 3–4 lines. Trim them to the one or two lines the repo guidelines call for.

No tests are included, which is expected for Terraform-only changes.

@codecov

codecov Bot commented Sep 20, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 0% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 92.47%. Comparing base (bd31258) to head (1765b37).

Additional details and impacted files
@@             Coverage Diff              @@
##           resonate    #4824      +/-   ##
============================================
+ Coverage     92.43%   92.47%   +0.04%     
============================================
  Files           358      358              
  Lines         11272    11267       -5     
  Branches        911      909       -2     
============================================
  Hits          10419    10419              
+ Misses          742      737       -5     
  Partials        111      111              
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

The durable workflows had nowhere to run outside a laptop. This adds both
halves: a Resonate server task on the cluster, next to redis and clamav,
and a `backend-resonate-worker` service running `manage.py
resonate_worker` off the backend image.

The server keeps its promises on the database instance the backend
already uses, in a database of its own. Terraform cannot create that
database (the instance is private), so a short container does it with
`createdb` before the server starts, on every deploy, doing nothing after
the first; Resonate creates its own schema from there.

Port 8001 is reachable only from the cluster's own security group, the
way redis and clamav are, and `RESONATE_URL` goes to every backend task:
the worker executes workflows, the admin starts them.
The resonate module was the only one in applications.tf splitting its
inputs into two groups.
Review follow-ups on the ECS setup.

`createdb ... || true` exited zero whatever happened, so the server's
SUCCESS dependency on it only promised that the container had run: a
wrong password or an unreachable instance was swallowed, the server
started anyway, and the only evidence sat in a log stream nobody would
think to open. Falling back to a `select 1` against the database instead
makes the container exit zero if and only if the database is there, and
stops it logging `already exists` on every task start.

The health check also had no start period and three retries, giving the
server ~30 seconds to come up -- the same 30 seconds it spends waiting on
the database pool before giving up, and before creating its schema on
first boot. It now gets the room docker-compose already gave it.
The bootstrap container is gone: the `resonate` database has to exist on
the instance before this deploys.
@marcoacierno
marcoacierno merged commit c3a68b4 into main Oct 30, 2026
8 checks passed
@marcoacierno
marcoacierno deleted the resonate-ecs-setup branch October 30, 2026 20:09

This branch was successfully deployed

1 active deployment
Preview — d088eb1d Deployed Oct 30, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant