Skip to content

test(personas): e2e harness + pure-LLM rewrites (13/13 passing) - #14

Merged
sebtsang merged 4 commits into
mainfrom
test/persona-e2e-harness
May 10, 2026
Merged

sebtsang merged 4 commits into
mainfrom
test/persona-e2e-harness

Conversation

@sebtsang

@sebtsang sebtsang commented May 9, 2026

Copy link
Copy Markdown
Owner

Summary

  • Adds scripts/_test-personas.ts — drives all 13 personas end-to-end via dev-only app/api/test-persona/route.ts, reporting timing + 1-line preview per persona. 13/13 passing in 195s.
  • Rewrites 9 stub prompts as pure-LLM Pattern B synthesizers (post-approval dispatch already lives in the dashboard handler — no need for mid-loop tool coupling): activation, brief-writer, crm-logger, feedback-tagger, linear-filer, pipeline-reporter, scheduler, slack-digest, theme-synthesizer.
  • Empties Composio scopes for those 9 personas in lib/tools/scopes.ts and removes the invalid nurture CTA from lib/personas/prompts/strategist.md (CallToActionSchema enforces book_call | free_trial | demo_video).

Test plan

  • pnpm tsx scripts/_test-personas.ts — 13/13 pass
  • researcher (Pattern B fetch + synth)
  • qualifier, strategist, writer, scheduler (sales chain)
  • brief-writer, activation, crm-logger
  • pipeline-reporter, slack-digest (revops)
  • feedback-tagger, theme-synthesizer, linear-filer (insight)

Pass matrix

Persona Time Output sample
researcher 14.8s EnrichedLead from fetchBundle
qualifier 30.7s tier=cold fit=20 intent=10
strategist 6.3s cta=demo_video
writer 73.7s drafts subject+body
scheduler 5.0s meeting at +3 days
brief-writer 13.9s 5 talking points + sentinel notion url
activation 5.1s email channel, ≤60-char subject
crm-logger 6.8s action=created
pipeline-reporter 9.3s summary + integer-only metrics
slack-digest 5.1s channel=#gtm + summaryBlocks
feedback-tagger 8.3s sentiment + theme[]
theme-synthesizer 12.2s sentinel notion themes url
linear-filer 3.8s sentinel linear issue url

🤖 Generated with Claude Code

sebtsang and others added 2 commits May 9, 2026 19:20
Adds scripts/_test-personas.ts harness that drives all 13 personas
end-to-end through a dev-only POST /api/test-persona endpoint. Inserts
a parent workflow_runs row for FK satisfaction, builds synthetic
previousOutputs chains per persona, reports timing + 1-line preview,
exits non-zero on any failure. 13/13 passing.

Rewrites these prompts as pure-LLM Pattern B synthesizers (post-approval
dispatch lives in the dashboard handler, not the LLM loop): activation,
brief-writer, crm-logger, feedback-tagger, linear-filer, pipeline-reporter,
scheduler, slack-digest, theme-synthesizer. Each emits structured JSON
matching its Zod schema with sentinel URLs/IDs the dashboard rewrites.

scopes.ts: empties Composio scopes for the 9 rewritten personas (they're
no longer tool-coupled). Strategist: removes "nurture" from CTA enum to
match CallToActionSchema (book_call | free_trial | demo_video).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Merges three RevOps specialists (crm-logger / pipeline-reporter /
slack-digest) into one `revenue-operations` synth that emits a
composite envelope: per-lead CRM updates + pipeline summary + Slack
digest. Same for the three Insight specialists (feedback-tagger /
theme-synthesizer / linear-filer) → one `insights` synth that emits
tagged feedback + themes + issues to file in one envelope.

Total persona count: 13 → 9 specialists. Each merged persona has one
prompt, one approval surface, and one output schema; the dashboard's
post-approval handler still does the actual external dispatch
(HubSpot / Slack / Notion / Linear / GitHub).

Touch points: PersonaId union (types.ts, schemas.ts), registry config
+ composite Zod schemas, scopes, persona-meta (labels/icons/order/
roles), revops + insight managers (now delegate to a single
specialist), conductor enum, workflows approval rules, mocks +
mock-driver, CLAUDE.md roster, test-persona route allowlist.

Test harness rewritten to invoke the two merged personas with realistic
synthetic input. 9/9 personas pass end-to-end in 182s, with insights
producing 5 tagged rows / 4 themes / 2 file-issue actions and
revenue-operations producing crmUpdates + slack envelope from
upstream sales fanout.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
sebtsang and others added 2 commits May 9, 2026 20:08
…ersonas

refactor(personas): collapse 6→2 — revenue-operations + insights
User rolled back the 13→9 collapse. Restores the original specialist
roster: crm-logger, pipeline-reporter, slack-digest (RevOps) and
feedback-tagger, theme-synthesizer, linear-filer (Insight). Their
Pattern B prompt rewrites (committed in 4beed79 / PR #14) remain
in place — only the consolidation is reverted.

This reverts merge commit 5e1c3f4 (PR #15).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@sebtsang
sebtsang merged commit 70931bb into main May 10, 2026
1 check passed
@sebtsang
sebtsang deleted the test/persona-e2e-harness branch May 10, 2026 00:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant