[srikoushik]

Fixes #1446

Summary

Adds a "Checks" workflow that runs on every pull request into dev and main. It fails only on problems the pull request adds: new lint findings of a bug-catching kind, a newly broken manage.py check, or newly failing tests. Existing problems on dev don't block anyone, and there's no list of known failures to maintain.

Size: 3 new files (workflow plus two small scripts); no application code changes.

Problem

Nothing checks pull requests today. dev already has 38 failing test entries and 137 ruff findings of the kinds below, so a new problem is hard to see by eye. #1419 and #1420 show the cost: a repeated dict key (ruff F601) and then a regression an existing test caught, both merged unnoticed (#1432).

Root cause

No CI for pull requests: the only workflows publish the Docker image and run Copilot's review.

Fix

.github/workflows/checks.yml, two jobs, each comparing the PR with its target branch:

Job	What it runs	Fails when
No new ruff findings	ruff 0.17.3, rules E9,F63,F7,F82,F601,F811 (syntax errors, repeated dict keys, redefinitions, undefined names), notebooks excluded	A (file, rule) pair has more findings than on the target branch
Django check and no new test failures	A conda environment built from the PR's own installation/environment.yml (mamba-org/setup-micromamba, cached); a postgres:16 service; migrations generated the way database-init.sh does; then manage.py check and the suite on both branches	check passes on the target branch but not on the PR, or a test fails only on the PR

Scripts:

• .github/scripts/no_new_ruff_findings.py
• .github/scripts/no_new_test_failures.sh

Settings:

• All required variables get placeholder values, plus a throwaway Fernet key. No secrets are used.
• CELERY_TASK_ALWAYS_EAGER=True: CI has no broker or workers. With eager mode off, 76 tests wait on broker retries, which took the suite from 15 s to about 8 minutes.

Why build the environment rather than use the published image: the image is only rebuilt when environment.yml changes on main. Building from the PR's environment.yml tests exactly what each branch declares, so a missing dependency shows up straight away. It's cached, so later runs take about 30 s to set up.

Tests

New checks; no new unit tests. Trialled on the fork (fplaunchpad/core-stack-backend), with a base branch identical to dev @ bd8fbcd0, and locally with deliberate breakages:

Case	Expected	Result
This PR alone (fork run 41790733139)	pass, warning that dev itself can't run its suite (see below)	✅ lint 13 s, tests 140 s
This PR plus the dependency fix below (fork run 41790728760)	full comparison	✅ lint 14 s, tests 171 s (environment 33 s cached; each suite 15 s; 232 tests, 36 failing on both sides)
Locally: dev vs dev	pass	✅ no new failures (38 already failing)
Locally: a repeated dict key added (#1419's mistake)	fail, naming the line	✅ layer_generation_in_order.py:302: F601
Locally: a new undefined name	fail, naming the line	✅ F821 Undefined name
Locally: parse_bool broken	fail, naming the test	✅ ParseBoolTests.test_values
Locally: manage.py check broken	fail	✅
Locally: suite can't start on the PR only	fail loudly, not "0 failing"	✅

Verification

Check	Result
actionlint on the workflow	✅ clean
shellcheck on the shell script	✅ clean
black, flake8 on the Python script	✅ clean
bandit on the Python script	2 low (B404/B603, subprocess), a false positive: fixed argument list, no shell, the only input is the ruff path
Action versions	actions/checkout@v6, mamba-org/setup-micromamba@v3, taken from the current official docs

Evidence

What CI found on dev today: the app can't start from environment.yml

Target branch:
manage.py check: FAILED
ModuleNotFoundError: No module named 'pyogrio'
Warning: the test suite can't run on the target branch either, so this isn't counted against the pull request

pyogrio, rasterstats and s2sphere are imported by modules loaded at startup (swb3_local.py, farm_boundaries/fetch_raw.py), but none is in environment.yml, on dev, staging or main. Fresh installs from environment.yml (the Docker image, install.sh) therefore can't start. A fix adding them is ready on a branch, and I'll raise it as a separate issue and PR. Until it lands, the test job passes with this warning; once it lands, the full comparison runs.

With the dependency fix: the full comparison

Target branch:
manage.py check: ok
Ran 232 tests in 15.190s
FAILED (failures=9, errors=27)
Pull request:
manage.py check: ok
Ran 232 tests in 15.321s
FAILED (failures=9, errors=27)
No new failing tests (36 already failing on the target branch).

Security and performance

• Security:
  • permissions: contents: read; no secrets.
  • Runs on pull_request, so PRs from forks get a read-only token. First-time contributors' runs need a maintainer's approval, as GitHub sets by default.
  • Third-party actions are the official actions/checkout and mamba-org/setup-micromamba.
• Performance:
  • Lint takes about 15 s.
  • Tests take about 3 minutes with a warm cache. The first run builds the environment, about 2 minutes extra.
  • No effect on the application.

Risk and rollback

• Risk: low.
  • It only reports a status. Whether it blocks merging is a branch-protection setting for the maintainers.
  • Known limit: a break that only affects tests already failing on the target branch can't be detected by comparison. Fixing the existing failures makes the check stronger.
  • Known limit: findings are counted per file and rule, so fixing one finding and adding another of the same kind in the same file in one PR goes unnoticed.
• Rollback: delete the workflow file, or revert this single commit.

Related

• Issue [Feature]: Check pull requests for new lint findings and new test failures (CI) #1446; [Bug]: Local ZOI pipeline has no regeneration datasets since #1420 (bulk --regenerate-local fails) #1432 is the regression that motivated it.
• Not included: the environment.yml dependency fix (separate PR); fixing the 36 existing failures.

Reviewer checklist

• The workflow runs on this PR and both jobs pass (the test job with the dependency warning)
• Decide later whether to make the checks required in branch protection

🤖 Generated with Claude Code

[srikoushik]

@amanodt could you review this when you get a chance? It adds PR checks that fail only on what a PR adds (new ruff findings of bug-catching kinds, a newly broken manage.py check, newly failing tests), so nothing on dev today blocks anyone. It is trialled on the fork (lint ~15 s, tests ~3 min). Trialling it also found that dev/staging/main cannot start in an environment built from environment.yml: pyogrio, rasterstats and s2sphere are imported at startup but not listed. That fix is ready separately. Proposal: #1446.

[srikoushik]

Closing for now; I'll come back to this later.