Skip to content

perf: add idempotent MongoDB index-creation script#550

Open
xpoes123 wants to merge 1 commit into
qbreader:mainfrom
xpoes123:perf/create-indexes-script
Open

perf: add idempotent MongoDB index-creation script#550
xpoes123 wants to merge 1 commit into
qbreader:mainfrom
xpoes123:perf/create-indexes-script

Conversation

@xpoes123

@xpoes123 xpoes123 commented Jun 30, 2026

Copy link
Copy Markdown
Contributor

The application defines no MongoDB indexes, so several hot query paths fall back to full collection scans. This adds an idempotent operator script (database/create-indexes.js, invoked via npm run create-indexes) that ensures the indexes the existing queries already filter and sort on.

Background

Query paths such as get-query, get-random-tossups, get-packet, and the leaderboard/user-stats aggregations operate against collections with no supporting indexes. The script reuses the existing collection handles from database/qbreader/collections.js and database/account-info/collections.js — no second client or connection is opened.

Changes

  • database/create-indexes.js — new idempotent script; calls createIndex for each entry in a documented table and logs the confirmed index name per call. createIndex is a no-op when an equivalent index already exists.
  • package.json — adds "create-indexes": "node database/create-indexes.js" script.

Indexes created and the queries they serve:

qbreader.tossups / qbreader.bonuses

  • { 'set.name': -1, 'packet.number': 1, number: 1 } — get-query result sort; get-set and get-pg-lookup filters and sorts.
  • { 'set.year': 1 } — year-range $match in get-random-tossups / get-random-bonuses, get-frequency-list, and get-query year filters.
  • { 'packet._id': 1, number: 1 } — get-packet (filter by packet._id, sort by number) and the packet-metadata-list $lookup.

qbreader.packets

  • { 'set.name': 1, number: 1 } — get-packet findOne, get-num-packets countDocuments, get-packet-list find.
  • { 'set._id': 1 } — packet-metadata-list $match.

qbreader.sets

  • { name: 1 } — get-set-id and rename-set findOne({ name }).

account-info.tossup-stars / account-info.bonus-stars

  • { user_id: 1 } — every star operation (get-ids, is-starred, star, unstar, clear) filters on user_id.

account-info.per-tossup-data / account-info.per-bonus-data

  • { 'data.user_id': 1 } — leaderboard grouping and user-stats match.
  • { set_id: 1 } — user-stats set filter.

Risk & testing

This is purely additive: a btree index never changes query results, it only speeds up filters and sorts. The unanchored $regex substring search is unaffected. createIndex is idempotent and modern MongoDB builds indexes non-blocking, so the script is safe to re-run. Verified with node --check and semistandard.

The script has not been run against the database in this PR. An operator must execute npm run create-indexes against the production database after merging.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@xpoes123 xpoes123 changed the title perf: add idempotent index-creation script perf: add idempotent MongoDB index-creation script Jun 30, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant