docs(DEPLOY-004): Postgres worked guide + pool_pre_ping for server DBs

[DoRmAmMu1997]

Ticket scope (DEPLOY-004 — Postgres worked guide, a previously-declined item now requested)

Goal: close the gap the June review flagged and the register deferred — docs/operations.md covered Compose-with-Postgres, Render, and backups, but the core self-hosted path ("I have a machine and want shared Postgres history") and the data migration ("I already have months of SQLite history") were left entirely to the reader.

In scope

• Worked end-to-end example in the Database section of docs/operations.md: provision (container or package-server route, least-privilege role — the app needs database ownership only, no superuser), DATABASE_URL, schema via the automatic startup migration or manual python -m alembic upgrade head, and a concrete verification checklist (alembic_version, \dt, Scan history view, Admin health, a scan_runs spot query).
• SQLite → Postgres data-migration recipe for existing history, with the three traps an operator actually hits called out explicitly:
  1. copy in Base.metadata.sorted_tables order (parents before FK children) into a schema Alembic built — and never copy alembic_version;
  2. pin the session to UTC before inserting (SQLite stored naive-UTC timestamps; Postgres would otherwise reinterpret them in local time);
  3. bump every serial sequence past the copied max ids afterward — otherwise the first new scan dies on a duplicate primary key.
     Plus: stop writers first, verify counts before flipping, keep the SQLite file as the rollback copy.
• Pool guidance: why the defaults are ample for this app's short-transaction profile; size the database's max_connections for instances, not vice-versa; PgBouncer session pooling as the conservative default.
• Code rider — backend/storage/database.py: _make_engine now sets pool_pre_ping=True for any non-SQLite URL. Managed Postgres proxies / PgBouncer / cloud NAT idle-kill TCP connections, so the first scan after a quiet stretch inherited a dead pooled connection ("server closed the connection unexpectedly"). One lightweight round-trip per checkout buys transparent reconnection. SQLite engine arguments are byte-identical, locked by a new test next to the existing pragma tests (the Postgres side is asserted on a lazily-built engine — no server needed, the pinned psycopg 3 driver only has to import).

Out of scope (deliberate)

• No changes to the Compose, Render, backup, or credential-rotation sections — they were already adequate.
• No new dependencies: psycopg 3 was already pinned; constraints.txt/pyproject.toml diff vs main is empty.
• The CI command strings and scheduler/database contract lines asserted verbatim by tests/test_supply_chain_policy.py are untouched — that test passing is the proof.
• Note: the DEPLOY-004 ID also tags the production settings validation shipped earlier (see app.py's main() docstring); this PR extends the same deployment epic rather than opening a new ID mid-wave.

This is ticket 7 of a 10-ticket second improvement wave (TEST-007 #98, REF-003 #99, AI-006 #100, IPO-006 #101, TEST-006 #102, PERF-002 #103, DEPLOY-004, QUAL-006, QUAL-007, DOC-003), each shipped as its own PR off main.

Gates (all green, local, Python 3.13)

Gate	Result
pytest -q --cov=backend --cov=screeners --cov=ui --cov-fail-under=87	1,387 passed, 1 skipped; coverage 88.13%
tests/test_supply_chain_policy.py (verbatim doc-contract lock)	passing, untouched
pre_commit validate-config / compileall / ruff / mypy (119 files) / bandit	clean
pip_audit -r constraints.txt	no known vulnerabilities

Review

/code-review: Approve — the rider is minimal and branch-isolated (the else arm only adds a kwarg; the SQLite arm is untouched, and the new test asserts pool._pre_ping on both engine flavors); the migration recipe was checked against the ORM metadata (sorted_tables ordering, serial-sequence reset, alembic_version exclusion) and the repository's _utc naive-timestamp handling. /security-review: no findings — the doc examples use placeholder credentials, the recipe requires no superuser, and pool_pre_ping adds no new connection surface (same URL, same driver).

🤖 Generated with Claude Code

[DoRmAmMu1997]

Self-review (posted as COMMENT — the bot runs as the PR author and cannot APPROVE). Verdict: Approve.

1. Rider isolation — the diff to _make_engine is one else arm adding pool_pre_ping=True; the SQLite branch (connect_args, pragma listener) is character-identical. The new test asserts pool._pre_ping is False for SQLite and is True for a postgresql+psycopg:// engine — built lazily, so no server is needed and the pinned psycopg 3 driver only has to import. All 8 storage-database tests pass.
2. Migration recipe verified against the code, not written from memory — Base.metadata.sorted_tables yields FK parents first (scan_results→scan_runs, IPO children→ipo_issues); alembic_version is not in Base.metadata, so the loop cannot copy it even if an operator forgets the warning; the UTC session pin matches the repository's _utc note that SQLite persists naive-UTC while Postgres preserves offsets; the serial-sequence reset guards with pg_get_serial_sequence IS NOT NULL so PK-less or non-serial tables are skipped cleanly.
3. Verbatim doc contracts intact — tests/test_supply_chain_policy.py asserts CRON_TZ=Asia/Kolkata, the host-timezone note, and the `psycopg[binary]` string in operations.md; the insert is purely additive between the existing schema paragraph and the Docker section, and the policy test passes.
4. No dependency motion — psycopg 3 was already pinned for postgresql+psycopg:// URLs; constraints/pyproject diff vs main is empty.
5. ID reuse flagged, not hidden — DEPLOY-004 also tags the earlier production-settings validation; the PR body notes this extends the same deployment epic.
6. Docs render sanity — new subsections nest under the existing ## Database heading (three ### blocks), fenced code blocks are bash/sql/python/env-tagged, and no Mermaid was touched.

Gates: 1,387 passed + 1 skipped, coverage 88.13% (floor 87); ruff/mypy(119)/bandit/pip-audit/pre-commit/compileall clean. Security review: no findings (placeholder credentials only; least-privilege provisioning; no new connection surface).

[DoRmAmMu1997]

Codex review plan before fixes:

• Replace password-bearing shell examples with a protected env-file/password-prompt workflow.
• Explain percent-encoding for reserved password characters without putting credentials in shell history or process arguments.
• Correct audit_log to the real audit_logs table and lock the documentation contract in tests.
• Keep the existing pool_pre_ping implementation unchanged.

I’ll update the guide/tests, run the full docs/runtime/security/Docker gates, and push the follow-up to this branch.

[DoRmAmMu1997]

Codex review findings for DEPLOY-004. The runtime pool_pre_ping change is sound; the operational examples need credential-safe handling and one schema-name correction.

[DoRmAmMu1997]

Completion verdict

Complete — no remaining reportable findings. This is a completion comment, not an approval.

Commit 2528eaf added protected environment-file and password-prompt guidance, URL-encoding guidance, the audit_logs correction, Alembic encoded-password safety, Docker/README examples, and policy tests. pool_pre_ping remained unchanged. Verification completed with 1,390 tests passed, 1 skipped, 87.89% coverage, and green hosted CI.

A formal diff-security review found no remaining reportable findings.