perf: add idempotent MongoDB index-creation script#550
Open
xpoes123 wants to merge 1 commit into
Open
Conversation
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The application defines no MongoDB indexes, so several hot query paths fall back to full collection scans. This adds an idempotent operator script (
database/create-indexes.js, invoked vianpm run create-indexes) that ensures the indexes the existing queries already filter and sort on.Background
Query paths such as get-query, get-random-tossups, get-packet, and the leaderboard/user-stats aggregations operate against collections with no supporting indexes. The script reuses the existing collection handles from
database/qbreader/collections.jsanddatabase/account-info/collections.js— no second client or connection is opened.Changes
database/create-indexes.js— new idempotent script; callscreateIndexfor each entry in a documented table and logs the confirmed index name per call.createIndexis a no-op when an equivalent index already exists.package.json— adds"create-indexes": "node database/create-indexes.js"script.Indexes created and the queries they serve:
qbreader.tossups / qbreader.bonuses
{ 'set.name': -1, 'packet.number': 1, number: 1 }— get-query result sort; get-set and get-pg-lookup filters and sorts.{ 'set.year': 1 }— year-range$matchin get-random-tossups / get-random-bonuses, get-frequency-list, and get-query year filters.{ 'packet._id': 1, number: 1 }— get-packet (filter bypacket._id, sort bynumber) and the packet-metadata-list$lookup.qbreader.packets
{ 'set.name': 1, number: 1 }— get-packetfindOne, get-num-packetscountDocuments, get-packet-listfind.{ 'set._id': 1 }— packet-metadata-list$match.qbreader.sets
{ name: 1 }— get-set-id and rename-setfindOne({ name }).account-info.tossup-stars / account-info.bonus-stars
{ user_id: 1 }— every star operation (get-ids, is-starred, star, unstar, clear) filters onuser_id.account-info.per-tossup-data / account-info.per-bonus-data
{ 'data.user_id': 1 }— leaderboard grouping and user-stats match.{ set_id: 1 }— user-stats set filter.Risk & testing
This is purely additive: a btree index never changes query results, it only speeds up filters and sorts. The unanchored
$regexsubstring search is unaffected.createIndexis idempotent and modern MongoDB builds indexes non-blocking, so the script is safe to re-run. Verified withnode --checkandsemistandard.The script has not been run against the database in this PR. An operator must execute
npm run create-indexesagainst the production database after merging.