Skip to content

perf(workspace): index and batch direct-message conversation summaries - #885

Draft
zomux wants to merge 2 commits into
developfrom
codex/conversation-history-query
Draft

zomux wants to merge 2 commits into
developfrom
codex/conversation-history-query

Conversation

@zomux

@zomux zomux commented Oct 28, 2026 •

Copy link
Copy Markdown
Contributor

Conversation listing scans general workspace message history to aggregate DMs, then issues one preview and one unread query per returned pair. Under production load this endpoint reached roughly 21-second p95 before the preceding membership fix and remained the slowest read afterward.

Add a partial covering index on (network_id, source, target, timestamp DESC, id DESC) INCLUDE (visibility) for visibility = 'direct' AND type = 'workspace.message.posted'. Aggregate exact lifetime counts from direct-message index entries, and batch preview and unread queries so up to 100 conversations use at most three event queries instead of up to 201. Latest-event timestamp ties now resolve deterministically by descending ID. Preserve identity filtering before the limit, legacy unlinked DMs, stored read markers, unread cutoffs, preferences and workspace isolation.

Validation:

  • 119 tests passed, including synthetic read-only PostgreSQL and SQLite semantics; API query-count coverage for 1, 10 and 100 conversations; existing people messaging, privacy, read-state, access and personalization tests.
  • Isolated PostgreSQL benchmark: 250,000 channel messages and 5,000 direct messages, returning 20 conversations with read markers. Three signed-in-viewer runs using the final covering index: before 30.11–34.65 ms / 41 event queries; after 6.54–9.73 ms / 3 queries, with identical results. These are local query-batch timings, not production HTTP timings.
  • Maximum-page benchmark (100 conversations, same isolated dataset): 85.73–91.04 ms / 201 queries before; 19.63–35.23 ms / 3 queries after with the signed-in privacy filter and final index, identical results.
  • EXPLAIN ANALYZE after indexing used an index-only scan of the 5,000 DM entries, with zero heap fetches. Signed-in-viewer aggregate execution was 2.371 ms; including visibility avoids heap fetches for the privacy OR predicate.
  • Actual migration upgrade, repeat application and downgrade verified on isolated PostgreSQL.

Production rollout requires the user's explicit schema-change approval under workspace/AGENTS.md. Do not promote this draft yet. Migration 086 creates the additive index CONCURRENTLY with a five-second lock timeout and verifies index validity. Existing application versions remain compatible; no messages or columns are rewritten. The normal deployment coordinator takes and verifies a database backup before migration, then rolls the two backend nodes. Index construction still reads the events table and consumes I/O/CPU; it may temporarily increase latency. Production statistics currently estimate 13.2 million events, a 5.3 GB heap, and 1.8% direct traffic. Actual build duration/size are not yet measured.

After approval: promote the tested merge through promote-release.yml, verify index validity and the production query plan, then compare authenticated probes and request timing samples after rollout. On failure, leave the old application serving; inspect any invalid index and retry only after cleanup. Application rollback can leave the additive index in place; removing it uses DROP INDEX CONCURRENTLY after approval.

@vercel

vercel Bot commented Oct 28, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
openagents-workspace Ready Ready Preview Oct 28, 2026 8:36am UTC

Request Review

This branch was successfully deployed

1 active deployment
Preview — 995a35a5 Deployed Oct 28, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant