Skip to content

[codex] Add 508 members newsletter sync#332

Open
michaelmwu wants to merge 18 commits into
mainfrom
michaelmwu/add-users-to-brevo-list
Open

[codex] Add 508 members newsletter sync#332
michaelmwu wants to merge 18 commits into
mainfrom
michaelmwu/add-users-to-brevo-list

Conversation

@michaelmwu

@michaelmwu michaelmwu commented Jun 9, 2026

Copy link
Copy Markdown
Member

Summary

  • add shared Brevo, Keila, and Migadu newsletter-sync helpers
  • keep /create-mailbox, /create-user-accounts, and agent account provisioning on direct best-effort newsletter sync for mailbox plus backup email
  • add weekly Migadu -> 508 members newsletter reconciliation that skips configured system mailboxes, CRM-blocked contacts, and provider-suppressed contacts
  • add worker job, API scheduler, and admin dashboard manual trigger for 508 members newsletter sync
  • support Brevo list lookup by BREVO_508_MEMBERS_NEWSLETTER_LIST_NAME with production override BREVO_508_MEMBERS_NEWSLETTER_LIST_ID=4
  • support Keila project contacts using contact data audiences: ["508_members"] for segmenting

Validation

  • uv run pytest tests/ -q
  • uv run pytest tests/unit/test_brevo_client.py tests/unit/test_keila_client.py tests/unit/test_newsletter_sync.py tests/unit/test_migadu_create_mailbox.py tests/unit/test_crm_create_sso_user.py tests/unit/test_shared_settings.py tests/unit/test_agent_gateway.py tests/unit/test_backend_api.py
  • ./scripts/lint.sh
  • ./scripts/mypy.sh
  • pre-commit hook: ruff, ruff format, mypy

Frontend verification note

  • bun run lint passes through ./scripts/lint.sh
  • bun run typecheck is blocked in this workspace because apps/admin_dashboard/node_modules has TypeScript 5.9.3 and is missing local typescript/@types/bun, while the lockfile expects TypeScript 6.0.3 and @types/bun
  • bun run test is blocked because local vitest is missing from node_modules

Notes

  • Production should set BREVO_508_MEMBERS_NEWSLETTER_LIST_ID=4; non-prod can omit it and rely on list-name lookup or leave Brevo disabled.
  • Keila does not expose Brevo-style contact lists in the documented API; this sync writes contact data for a 508_members audience so Keila segments can target it.

Summary by CodeRabbit

  • New Features
    • Added newsletter contact sync via Brevo and Keila, subscribing both the 508 mailbox email and its backup email to the configured audience/list.
    • Introduced an admin dashboard “Sync newsletters” action with dry-run support and job/queue feedback.
    • Added optional periodic syncing with configurable interval and excluded mailboxes.
    • Updated mailbox provisioning flows to surface newsletter subscription success/warning status without blocking creation.
  • Documentation
    • Expanded environment/configuration docs for newsletter sync settings and dashboard field lock behavior when env values are set.

Copilot AI review requested due to automatic review settings June 9, 2026 17:37
@coderabbitai

coderabbitai Bot commented Jun 9, 2026

Copy link
Copy Markdown

Review Change Stack

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: c9c56d89-0c26-46a3-831a-222c33d779bb

📥 Commits

Reviewing files that changed from the base of the PR and between 54304cf and 9786246.

📒 Files selected for processing (26)
  • .env.example
  • ENVIRONMENT.md
  • apps/admin_dashboard/src/main.tsx
  • apps/api/src/five08/backend/api.py
  • apps/api/src/five08/backend/static/dashboard/.vite/manifest.json
  • apps/api/src/five08/backend/static/dashboard/assets/index-DLefnMNV.js
  • apps/api/src/five08/backend/static/dashboard/index.html
  • apps/worker/src/five08/worker/jobs.py
  • docs/configuration.md
  • packages/shared/src/five08/agent/tools.py
  • packages/shared/src/five08/clients/brevo.py
  • packages/shared/src/five08/clients/contact_email.py
  • packages/shared/src/five08/clients/keila.py
  • packages/shared/src/five08/clients/migadu.py
  • packages/shared/src/five08/newsletter_sync.py
  • packages/shared/src/five08/redaction.py
  • packages/shared/src/five08/runtime_config.py
  • packages/shared/src/five08/settings.py
  • tests/unit/test_agent_gateway.py
  • tests/unit/test_backend_api.py
  • tests/unit/test_brevo_client.py
  • tests/unit/test_keila_client.py
  • tests/unit/test_migadu_client.py
  • tests/unit/test_newsletter_sync.py
  • tests/unit/test_runtime_config.py
  • tests/unit/test_worker_newsletter_sync.py
✅ Files skipped from review due to trivial changes (3)
  • packages/shared/src/five08/clients/contact_email.py
  • apps/api/src/five08/backend/static/dashboard/.vite/manifest.json
  • ENVIRONMENT.md
🚧 Files skipped from review as they are similar to previous changes (13)
  • apps/api/src/five08/backend/static/dashboard/index.html
  • docs/configuration.md
  • tests/unit/test_worker_newsletter_sync.py
  • packages/shared/src/five08/clients/migadu.py
  • apps/worker/src/five08/worker/jobs.py
  • tests/unit/test_backend_api.py
  • packages/shared/src/five08/runtime_config.py
  • tests/unit/test_agent_gateway.py
  • apps/admin_dashboard/src/main.tsx
  • tests/unit/test_newsletter_sync.py
  • tests/unit/test_runtime_config.py
  • packages/shared/src/five08/agent/tools.py
  • packages/shared/src/five08/newsletter_sync.py

📝 Walkthrough

Walkthrough

Adds Brevo and Keila newsletter provider clients, a NewsletterSyncProcessor with CRM-aware orchestration, settings and environment configuration, dashboard API trigger with periodic scheduling, worker job with email redaction, mailbox provisioning integration across Discord and tooling flows, admin dashboard action UI, and comprehensive unit test coverage.

Changes

Newsletter Sync End-to-End Implementation

Layer / File(s) Summary
Configuration, env examples, and settings
.env.example, ENVIRONMENT.md, docs/configuration.md, packages/shared/src/five08/runtime_config.py, packages/shared/src/five08/settings.py, tests/unit/test_shared_settings.py, tests/unit/test_runtime_config.py
Adds newsletter-related env examples (.env.example), runtime-config definitions with restart-required flags, SharedSettings fields for Brevo/Keila credentials/endpoints/timeouts, and a brevo_508_members_newsletter_list_id field validator. Documentation describes env-locking of dashboard fields and sync behavior including CRM-matching and mailbox exclusions.
Brevo, Keila, and Migadu client primitives
packages/shared/src/five08/clients/brevo.py, packages/shared/src/five08/clients/keila.py, packages/shared/src/five08/clients/migadu.py, packages/shared/src/five08/clients/contact_email.py, packages/shared/src/five08/redaction.py, tests/unit/test_brevo_client.py, tests/unit/test_keila_client.py, tests/unit/test_migadu_client.py
Implements Brevo client with contact-list subscription and list-name lookup; Keila client with contact upsert; Migadu mailbox listing. All clients include redacted error reporting via _response_body_excerpt() and email validation via normalize_provider_contact_email(). Tests validate request/response handling, URL encoding, 404/204 responses, timeouts, and error redaction.
Newsletter sync orchestration and providers
packages/shared/src/five08/newsletter_sync.py, tests/unit/test_newsletter_sync.py
Adds NewsletterProvider protocol, BrevoNewsletterProvider and KeilaNewsletterProvider implementations with provider-side suppression rules, NewsletterSyncProcessor.sync_508_members() with Migadu mailbox scanning/CRM gating, sync_newsletter_contacts() for direct email sync, build_newsletter_providers(), and format_newsletter_sync_warning(). Unit tests cover exclusions, suppression, CRM-blocking, provider failures, list-name caching, and deduplication.
Agent runtime mailbox provisioning
packages/shared/src/five08/agent/tools.py, tests/unit/test_agent_gateway.py
Extends ToolRuntimeConfig with Brevo/Keila fields, maps settings into config, adds _add_emails_to_newsletter() helper, and propagates newsletter_error/newsletter_subscribed through _create_migadu_mailbox() and result shapes. Tests check credential wiring, subscription calls, suppression handling, and runtime_config_factory injection.
Discord bot Migadu and CRM flows
apps/discord_bot/src/five08/discord_bot/cogs/migadu.py, apps/discord_bot/src/five08/discord_bot/cogs/crm.py, apps/discord_bot/README.md, tests/unit/test_migadu_create_mailbox.py, tests/unit/test_crm_create_sso_user.py
Calls best-effort newsletter sync in mailbox creation and CRM provisioning flows; extends MailboxCreationOutcome and MailboxProvisioningResult with newsletter_error field; updates audit metadata to include newsletter_subscribed and error; appends newsletter status to user-facing messages; documents optional sync in README. Tests assert helper invocation and message content.
API endpoint, scheduler, and worker job
apps/api/src/five08/backend/api.py, apps/worker/src/five08/worker/jobs.py, tests/unit/test_backend_api.py, tests/unit/test_worker_newsletter_sync.py
Adds POST /dashboard/api/sync/newsletters with dry-run support and audit event recording; scheduler task that runs at configured interval using idempotency keys; _enqueue_newsletter_sync_job() helper with distinct key generation for scheduler vs manual triggers; registers sync_508_members_newsletters_job() worker job that calls processor and masks emails. Tests verify endpoint response codes, idempotency keys, audit fields, and email masking.
Admin dashboard UI and static assets
apps/admin_dashboard/src/main.tsx, apps/api/src/five08/backend/static/dashboard/index.html, apps/api/src/five08/backend/static/dashboard/.vite/manifest.json
Adds syncNewsletters() handler in dashboard App, newsletter configuration group, People view button wired to trigger sync with loading state and toast feedback (including returned job_id), and updates hashed JavaScript bundle references.

Sequence Diagram

sequenceDiagram
  participant AdminUI as Admin Dashboard
  participant API as apps/api
  participant Queue as Job Queue
  participant Worker as apps/worker
  participant Processor as NewsletterSyncProcessor
  participant Brevo as Brevo API
  participant Keila as Keila API

  AdminUI->>API: POST /dashboard/api/sync/newsletters
  API->>Queue: enqueue sync_508_members_newsletters_job
  API->>Audit: record newsletter.508_members_sync event
  Queue->>Worker: deliver job by name
  Worker->>Processor: NewsletterSyncProcessor.sync_508_members()
  Processor->>Brevo: add_contact_to_list / list lookup
  Processor->>Keila: upsert_active_contact
  Processor-->>Worker: result (emails masked)
  Worker-->>API: audit/event already recorded at enqueue
Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~45 minutes

Possibly related PRs

  • 508-dev/508-workflows#207: Prior changes to the Migadu mailbox creation flow that are extended here with best-effort newsletter subscription integration.
  • 508-dev/508-workflows#115: Overlapping modifications to the Discord MigaduCog mailbox provisioning flow that add the newsletter sync feature.

Poem

🐰 I hopped through configs, keys in tow,
Brevo and Keila now learn who to know.
Mailboxes whispered, backup emails too,
Best-effort syncs—a gentle woo!
Scheduler ticks and dashboards glow,
Cheers from a rabbit—small, not slow.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch michaelmwu/add-users-to-brevo-list

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 3c1183a9be

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread packages/shared/src/five08/settings.py Outdated
brevo_api_key: str | None = None
brevo_api_base_url: str = "https://api.brevo.com/v3"
brevo_api_timeout_seconds: float = 20.0
brevo_newsletter_list_id: int | None = Field(default=None, ge=1)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Handle blank Brevo list IDs before integer validation

When .env is copied from the updated .env.example, BREVO_NEWSLETTER_LIST_ID= is present but blank; pydantic-settings does not coerce an empty string to None for an int | None, so constructing SharedSettings fails before the optional Brevo sync can simply remain disabled. The existing optional numeric Docuseal setting has a mode="before" normalizer for this case; this new field needs the same treatment (or equivalent env_ignore_empty) so local/dev deployments with the documented blank value still start.

Useful? React with 👍 / 👎.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds best-effort Brevo newsletter subscription during mailbox/account provisioning across the agent tools and Discord bot flows, with shared configuration and a shared Brevo API client.

Changes:

  • Introduces a shared BrevoClient plus settings/runtime-config wiring for BREVO_API_KEY and BREVO_NEWSLETTER_LIST_ID.
  • Subscribes both the 508 mailbox email and backup email to a Brevo list during /create-mailbox and /create-user-accounts (non-blocking, warning surfaced).
  • Extends unit tests and docs/env examples to cover the new integration.

Reviewed changes

Copilot reviewed 13 out of 13 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
packages/shared/src/five08/clients/brevo.py Adds shared Brevo API wrapper for create-or-update contact + list subscription.
packages/shared/src/five08/settings.py Exposes Brevo settings in shared BaseSettings.
packages/shared/src/five08/agent/tools.py Adds Brevo runtime config + newsletter subscription in mailbox provisioning tool results.
apps/discord_bot/src/five08/discord_bot/cogs/migadu.py Subscribes created mailbox + backup email to Brevo after Migadu provisioning; surfaces result in embeds/audit metadata.
apps/discord_bot/src/five08/discord_bot/cogs/crm.py Subscribes mailbox + backup email to Brevo during user-account provisioning; surfaces result in messages/audit metadata.
tests/unit/test_brevo_client.py Adds unit tests for BrevoClient request payload and error handling.
tests/unit/test_shared_settings.py Extends shared settings/runtime config test coverage for Brevo fields.
tests/unit/test_migadu_create_mailbox.py Asserts newsletter sync hook is invoked in mailbox command flow.
tests/unit/test_crm_create_sso_user.py Asserts newsletter sync hook is invoked and success message includes newsletter status.
tests/unit/test_agent_gateway.py Adds Brevo fake + assertions for agent tool subscription behavior and result metadata.
ENVIRONMENT.md Documents optional Brevo newsletter sync env vars and behavior.
apps/discord_bot/README.md Documents Brevo newsletter sync behavior for /create-mailbox.
.env.example Adds Brevo env var placeholders.
Comments suppressed due to low confidence (1)

apps/discord_bot/src/five08/discord_bot/cogs/crm.py:8632

  • exc.newsletter_error is interpolated into a Discord message inside backticks without sanitization/truncation. If the underlying Brevo error contains backticks/control characters or is long (e.g., includes an HTTP body), this can break formatting or exceed Discord limits. Sanitize/truncate the newsletter error before building newsletter_line.
            self._audit_command_safe(
                interaction=interaction,
                action="crm.create_user_accounts",
                result="error",
                metadata={

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread packages/shared/src/five08/settings.py Outdated
Comment on lines +82 to +85
brevo_api_key: str | None = None
brevo_api_base_url: str = "https://api.brevo.com/v3"
brevo_api_timeout_seconds: float = 20.0
brevo_newsletter_list_id: int | None = Field(default=None, ge=1)
Comment on lines +246 to +252
logger.warning(
"Failed to add %s to Brevo newsletter list: %s",
normalized_email,
exc,
)
errors.append(f"{normalized_email}: {exc}")
return "; ".join(errors) if errors else None
Comment on lines +3855 to +3862
except (BrevoAPIError, ValueError) as exc:
logger.warning(
"Failed to add %s to Brevo newsletter list: %s",
normalized_email,
exc,
)
errors.append(f"{normalized_email}: {exc}")
return "; ".join(errors) if errors else None
assert runtime_config.brevo_api_key == "brevo-key"
assert runtime_config.brevo_newsletter_list_id == 4


@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 6780a019e4

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +227 to +228
if client is None:
return "BREVO_API_KEY is not configured."

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Treat missing Brevo config as disabled

When BREVO_API_KEY is intentionally unset, the documented behavior is that newsletter sync is optional, but this return value is propagated as newsletter_error, so successful /create-mailbox and /create-user-accounts flows display “Brevo subscription failed: BREVO_API_KEY is not configured” and audit newsletter_subscribed=false. In deployments that do not configure Brevo, every mailbox/account creation now looks partially failed even though sync should simply be skipped unless the key is configured.

Useful? React with 👍 / 👎.

Copilot AI review requested due to automatic review settings June 10, 2026 06:01
@michaelmwu michaelmwu changed the title [codex] Add Brevo newsletter sync for mailbox provisioning [codex] Add 508 members newsletter sync Jun 10, 2026

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: e6dce19025

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +251 to +252
except EspoAPIError:
return None

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Fail closed when CRM block lookup errors

When EspoCRM is configured but list_contacts raises, this returns None, and the caller treats that the same as “no blocked CRM contact” because _is_crm_blocked(None) is false. During a transient CRM outage or bad CRM credentials, the weekly newsletter sync will therefore subscribe every non-excluded Migadu mailbox/recovery email, including contacts that would otherwise be skipped as inactive, rejected, or blocked; the lookup error should skip the mailbox or abort the sync instead of failing open.

Useful? React with 👍 / 👎.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 22 out of 22 changed files in this pull request and generated 5 comments.

Comment thread apps/discord_bot/src/five08/discord_bot/cogs/migadu.py
Comment on lines +8603 to +8607
newsletter_line = (
f"\nNewsletter: subscription warning: `{exc.newsletter_error}`"
if exc.newsletter_error
else "\nNewsletter: added mailbox and backup email."
)
Comment on lines +8768 to +8771
message_lines.append(
"Newsletter: subscription warning: "
f"`{result.mailbox.newsletter_error}`"
)
Comment thread .env.example Outdated
Comment thread ENVIRONMENT.md

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 224a385aba

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +116 to +122
if existing is not None and (
bool(existing.get("emailBlacklisted"))
or bool(existing.get("smsBlacklisted"))
or str(existing.get("status") or "").strip().casefold()
in PROVIDER_SUPPRESSED_STATUSES
):
return "skipped_provider_suppressed"

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Skip Brevo list-level unsubscribes

When a Brevo contact has unsubscribed from only the 508 members list, Brevo returns that list ID in listUnsubscribed while emailBlacklisted remains false. This suppression check does not inspect listUnsubscribed, so the code falls through and calls add_contact_to_list, which can re-add or repeatedly attempt to re-add contacts who explicitly opted out of this list, contrary to the new “does not re-add provider-suppressed contacts” behavior.

Useful? React with 👍 / 👎.

Copilot AI review requested due to automatic review settings June 10, 2026 15:13

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 29 out of 31 changed files in this pull request and generated 3 comments.

Comment on lines +18 to +23
def _response_body_excerpt(body: object) -> str:
"""Return a bounded response-body excerpt for persisted/logged errors."""
text = " ".join(str(body or "").split())
if len(text) <= ERROR_BODY_MAX_LENGTH:
return text
return f"{text[:ERROR_BODY_MAX_LENGTH]}..."
Comment on lines +19 to +24
def _response_body_excerpt(body: object) -> str:
"""Return a bounded response-body excerpt for persisted/logged errors."""
text = " ".join(str(body or "").split())
if len(text) <= ERROR_BODY_MAX_LENGTH:
return text
return f"{text[:ERROR_BODY_MAX_LENGTH]}..."
Comment on lines +18 to +23
def _response_body_excerpt(body: object) -> str:
"""Return a bounded response-body excerpt for persisted/logged errors."""
text = " ".join(str(body or "").split())
if len(text) <= ERROR_BODY_MAX_LENGTH:
return text
return f"{text[:ERROR_BODY_MAX_LENGTH]}..."

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 30 out of 31 changed files in this pull request and generated 3 comments.

Comment on lines +433 to +439
if failed and isinstance(failures, list) and failures:
detail = "; ".join(
f"{item.get('email')}: {item.get('error')}"
for item in failures[:3]
if isinstance(item, dict)
)
messages.append(f"{provider_name} failed for {failed} contact(s): {detail}")
Comment on lines +6684 to +6687
"job_type": "sync_508_members_newsletters_job",
"reason": "dashboard",
"idempotency_key_pattern": "newsletter-sync:508-members:dashboard:<timestamp>",
},
Comment on lines 3 to 7
import base64
import logging
import re
from datetime import datetime, timezone
from email import message_from_bytes

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: a4252e5e38

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread apps/worker/src/five08/worker/jobs.py Outdated


DOCUSEAL_COMPLETED_AT_UTC_FORMAT = "%Y-%m-%d %H:%M:%S"
EMAIL_PATTERN = re.compile(r"(?<![\w.+-])[\w.+-]+@[\w.-]+\.[A-Za-z]{2,}(?![\w.-])")

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Mask percent-encoded emails in newsletter job errors

When a Brevo or Keila per-contact lookup fails at the request layer, requests can include the URL path in the exception (for example /contacts/jane%40example.com); the new worker-side masking only matches raw @ email strings, so these encoded mailbox addresses can still be persisted in newsletter job results via provider failures[].error. Fresh evidence since the earlier raw-email masking comment is that provider lookup URLs encode the contact email before the exception reaches this regex, so the current pattern will not redact it.

Useful? React with 👍 / 👎.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 30 out of 31 changed files in this pull request and generated 5 comments.

Comment on lines +49 to +51
normalized_email = email.strip().lower()
if not normalized_email or normalized_email.count("@") != 1:
raise ValueError("Brevo contact email must be a full email address.")
Comment on lines +97 to +99
normalized_email = email.strip().lower()
if not normalized_email or normalized_email.count("@") != 1:
raise ValueError("Brevo contact email must be a full email address.")
Comment on lines +45 to +47
normalized_email = email.strip().lower()
if not normalized_email or normalized_email.count("@") != 1:
raise ValueError("Keila contact email must be a full email address.")
Comment on lines +66 to +68
normalized_email = email.strip().lower()
if not normalized_email or normalized_email.count("@") != 1:
raise ValueError("Keila contact email must be a full email address.")
Comment on lines +38 to +39
def ensure_contact(self, contact: NewsletterContact) -> str:
"""Ensure contact exists, returning added/updated/already/skipped."""

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 19a831cec4

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +98 to +100
payload["data"] = {
**(existing_data if isinstance(existing_data, dict) else {}),
**(data or {}),

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Preserve existing Keila audiences when tagging

When an existing Keila contact already has data.audiences for other segments, this merge still lets the newsletter payload from KeilaNewsletterProvider overwrite that list with only ['508_members']; fresh evidence after the earlier data-preservation concern is that the current spread order preserves top-level keys but replaces the nested audience array. A member sync would silently drop the contact's other Keila audience tags, so union the existing and new audiences before PATCHing.

Useful? React with 👍 / 👎.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@apps/api/src/five08/backend/api.py`:
- Around line 6243-6299: The current implementation sends the email via
send_onboarding_email_message before calling
_mark_dashboard_onboarding_email_sent to record the send, creating a race
condition where duplicate emails can be sent if the function is retried between
a successful SMTP send and the marker update. Refactor this by moving the marker
reservation or update to occur before the SMTP send operation. Check if the
marker already exists at the beginning of the logic after validation, and only
proceed with send_onboarding_email_message if the marker reservation succeeds.
This makes the operation idempotent so retries after a successful SMTP send will
be safely skipped.
- Around line 6372-6387: The audit metadata being passed to
_audit_dashboard_onboarding_email() contains raw email addresses in the
recipient_email and reply_to_email fields, which violates PII safety practices.
Either mask these email addresses using the existing mask_email() helper
function before including them in the metadata dictionary, or omit these fields
entirely and rely on the contact_id field for traceability. Choose the approach
that best balances debugging needs with privacy requirements.

In `@packages/shared/src/five08/settings.py`:
- Line 90: The newsletter_sync_excluded_mailboxes field in settings.py is
initialized with an empty string default, which means no mailboxes are excluded
by default. This is unsafe because downstream sync logic directly consumes this
value to determine which mailboxes to skip, potentially exposing system and
service mailbox addresses to external providers. Replace the empty string
default with a safe default value that includes common system or service
mailboxes that should be excluded from newsletter synchronization.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: c9c56d89-0c26-46a3-831a-222c33d779bb

📥 Commits

Reviewing files that changed from the base of the PR and between 54304cf and 9786246.

📒 Files selected for processing (26)
  • .env.example
  • ENVIRONMENT.md
  • apps/admin_dashboard/src/main.tsx
  • apps/api/src/five08/backend/api.py
  • apps/api/src/five08/backend/static/dashboard/.vite/manifest.json
  • apps/api/src/five08/backend/static/dashboard/assets/index-DLefnMNV.js
  • apps/api/src/five08/backend/static/dashboard/index.html
  • apps/worker/src/five08/worker/jobs.py
  • docs/configuration.md
  • packages/shared/src/five08/agent/tools.py
  • packages/shared/src/five08/clients/brevo.py
  • packages/shared/src/five08/clients/contact_email.py
  • packages/shared/src/five08/clients/keila.py
  • packages/shared/src/five08/clients/migadu.py
  • packages/shared/src/five08/newsletter_sync.py
  • packages/shared/src/five08/redaction.py
  • packages/shared/src/five08/runtime_config.py
  • packages/shared/src/five08/settings.py
  • tests/unit/test_agent_gateway.py
  • tests/unit/test_backend_api.py
  • tests/unit/test_brevo_client.py
  • tests/unit/test_keila_client.py
  • tests/unit/test_migadu_client.py
  • tests/unit/test_newsletter_sync.py
  • tests/unit/test_runtime_config.py
  • tests/unit/test_worker_newsletter_sync.py
✅ Files skipped from review due to trivial changes (3)
  • packages/shared/src/five08/clients/contact_email.py
  • apps/api/src/five08/backend/static/dashboard/.vite/manifest.json
  • ENVIRONMENT.md
🚧 Files skipped from review as they are similar to previous changes (13)
  • apps/api/src/five08/backend/static/dashboard/index.html
  • docs/configuration.md
  • tests/unit/test_worker_newsletter_sync.py
  • packages/shared/src/five08/clients/migadu.py
  • apps/worker/src/five08/worker/jobs.py
  • tests/unit/test_backend_api.py
  • packages/shared/src/five08/runtime_config.py
  • tests/unit/test_agent_gateway.py
  • apps/admin_dashboard/src/main.tsx
  • tests/unit/test_newsletter_sync.py
  • tests/unit/test_runtime_config.py
  • packages/shared/src/five08/agent/tools.py
  • packages/shared/src/five08/newsletter_sync.py

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Caution

Inline review comments failed to post. This is likely due to GitHub's internal server error or limits when posting large numbers of comments. If you are seeing this consistently it is likely a permissions issue. Please check "Moderation" -> "Code review limits" under your organization settings.

Actionable comments posted: 3

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@apps/api/src/five08/backend/api.py`:
- Around line 6243-6299: The current implementation sends the email via
send_onboarding_email_message before calling
_mark_dashboard_onboarding_email_sent to record the send, creating a race
condition where duplicate emails can be sent if the function is retried between
a successful SMTP send and the marker update. Refactor this by moving the marker
reservation or update to occur before the SMTP send operation. Check if the
marker already exists at the beginning of the logic after validation, and only
proceed with send_onboarding_email_message if the marker reservation succeeds.
This makes the operation idempotent so retries after a successful SMTP send will
be safely skipped.
- Around line 6372-6387: The audit metadata being passed to
_audit_dashboard_onboarding_email() contains raw email addresses in the
recipient_email and reply_to_email fields, which violates PII safety practices.
Either mask these email addresses using the existing mask_email() helper
function before including them in the metadata dictionary, or omit these fields
entirely and rely on the contact_id field for traceability. Choose the approach
that best balances debugging needs with privacy requirements.

In `@packages/shared/src/five08/settings.py`:
- Line 90: The newsletter_sync_excluded_mailboxes field in settings.py is
initialized with an empty string default, which means no mailboxes are excluded
by default. This is unsafe because downstream sync logic directly consumes this
value to determine which mailboxes to skip, potentially exposing system and
service mailbox addresses to external providers. Replace the empty string
default with a safe default value that includes common system or service
mailboxes that should be excluded from newsletter synchronization.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: c9c56d89-0c26-46a3-831a-222c33d779bb

📥 Commits

Reviewing files that changed from the base of the PR and between 54304cf and 9786246.

📒 Files selected for processing (26)
  • .env.example
  • ENVIRONMENT.md
  • apps/admin_dashboard/src/main.tsx
  • apps/api/src/five08/backend/api.py
  • apps/api/src/five08/backend/static/dashboard/.vite/manifest.json
  • apps/api/src/five08/backend/static/dashboard/assets/index-DLefnMNV.js
  • apps/api/src/five08/backend/static/dashboard/index.html
  • apps/worker/src/five08/worker/jobs.py
  • docs/configuration.md
  • packages/shared/src/five08/agent/tools.py
  • packages/shared/src/five08/clients/brevo.py
  • packages/shared/src/five08/clients/contact_email.py
  • packages/shared/src/five08/clients/keila.py
  • packages/shared/src/five08/clients/migadu.py
  • packages/shared/src/five08/newsletter_sync.py
  • packages/shared/src/five08/redaction.py
  • packages/shared/src/five08/runtime_config.py
  • packages/shared/src/five08/settings.py
  • tests/unit/test_agent_gateway.py
  • tests/unit/test_backend_api.py
  • tests/unit/test_brevo_client.py
  • tests/unit/test_keila_client.py
  • tests/unit/test_migadu_client.py
  • tests/unit/test_newsletter_sync.py
  • tests/unit/test_runtime_config.py
  • tests/unit/test_worker_newsletter_sync.py
✅ Files skipped from review due to trivial changes (3)
  • packages/shared/src/five08/clients/contact_email.py
  • apps/api/src/five08/backend/static/dashboard/.vite/manifest.json
  • ENVIRONMENT.md
🚧 Files skipped from review as they are similar to previous changes (13)
  • apps/api/src/five08/backend/static/dashboard/index.html
  • docs/configuration.md
  • tests/unit/test_worker_newsletter_sync.py
  • packages/shared/src/five08/clients/migadu.py
  • apps/worker/src/five08/worker/jobs.py
  • tests/unit/test_backend_api.py
  • packages/shared/src/five08/runtime_config.py
  • tests/unit/test_agent_gateway.py
  • apps/admin_dashboard/src/main.tsx
  • tests/unit/test_newsletter_sync.py
  • tests/unit/test_runtime_config.py
  • packages/shared/src/five08/agent/tools.py
  • packages/shared/src/five08/newsletter_sync.py
🛑 Comments failed to post (3)
apps/api/src/five08/backend/api.py (2)

6243-6299: ⚠️ Potential issue | 🟠 Major | 🏗️ Heavy lift

Make the SMTP send idempotent before calling the provider.

Line 6285 performs the non-idempotent email send before any durable “already sent” reservation. A double-click, retry, or crash between SMTP success and the marker update can send duplicate onboarding emails. Check/reserve the send marker before SMTP, or route this through an idempotent queued job/outbox so retries become no-ops.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@apps/api/src/five08/backend/api.py` around lines 6243 - 6299, The current
implementation sends the email via send_onboarding_email_message before calling
_mark_dashboard_onboarding_email_sent to record the send, creating a race
condition where duplicate emails can be sent if the function is retried between
a successful SMTP send and the marker update. Refactor this by moving the marker
reservation or update to occur before the SMTP send operation. Check if the
marker already exists at the beginning of the logic after validation, and only
proceed with send_onboarding_email_message if the marker reservation succeeds.
This makes the operation idempotent so retries after a successful SMTP send will
be safely skipped.

6372-6387: ⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Do not persist raw email addresses in audit metadata.

This success audit stores recipient_email and reply_to_email verbatim. Use the existing mask_email() helper, or omit these fields and rely on contact_id for traceability.

Proposed PII-safe audit metadata
         metadata={
             **metadata,
-            "candidate_name": candidate_name,
-            "recipient_email": recipient_email,
-            "reply_to_email": reply_to_email,
-            "sender_display_name": sender_display_name,
-            "signature_name": signature_name,
+            "recipient_email_masked": mask_email(recipient_email),
+            "reply_to_email_masked": mask_email(reply_to_email),
             "onboarding_status": onboarding_status,
             "marker_status": marker_status,
             "marker_error": marker_error,
         },
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

    await _audit_dashboard_onboarding_email(
        session,
        result=AuditResult.SUCCESS,
        contact_id=contact_id.strip(),
        metadata={
            **metadata,
            "recipient_email_masked": mask_email(recipient_email),
            "reply_to_email_masked": mask_email(reply_to_email),
            "onboarding_status": onboarding_status,
            "marker_status": marker_status,
            "marker_error": marker_error,
        },
    )
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@apps/api/src/five08/backend/api.py` around lines 6372 - 6387, The audit
metadata being passed to _audit_dashboard_onboarding_email() contains raw email
addresses in the recipient_email and reply_to_email fields, which violates PII
safety practices. Either mask these email addresses using the existing
mask_email() helper function before including them in the metadata dictionary,
or omit these fields entirely and rely on the contact_id field for traceability.
Choose the approach that best balances debugging needs with privacy
requirements.
packages/shared/src/five08/settings.py (1)

90-90: ⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Use a safe default for excluded newsletter mailboxes.

Line 90 sets newsletter_sync_excluded_mailboxes to empty, and downstream sync logic consumes this value directly to decide which mailboxes to skip. That can send system/service mailbox addresses to external providers when exclusions are not explicitly configured.

🔧 Proposed fix
-    newsletter_sync_excluded_mailboxes: str = ""
+    newsletter_sync_excluded_mailboxes: str = "system,service-account"
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

    newsletter_sync_excluded_mailboxes: str = "system,service-account"
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@packages/shared/src/five08/settings.py` at line 90, The
newsletter_sync_excluded_mailboxes field in settings.py is initialized with an
empty string default, which means no mailboxes are excluded by default. This is
unsafe because downstream sync logic directly consumes this value to determine
which mailboxes to skip, potentially exposing system and service mailbox
addresses to external providers. Replace the empty string default with a safe
default value that includes common system or service mailboxes that should be
excluded from newsletter synchronization.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants