Indexer skips archived realms; archive cancels in-flight indexing (CS-11668)#5361
Draft
lukemelia wants to merge 2 commits into
Draft
Indexer skips archived realms; archive cancels in-flight indexing (CS-11668)#5361lukemelia wants to merge 2 commits into
lukemelia wants to merge 2 commits into
Conversation
The full-reindex sweep source (getFullReindexRealmUrls) excludes any
realm with realm_metadata.archived_at IS NOT NULL, so the Grafana
/_full-reindex endpoint and the post-deployment hook never enqueue
from-scratch-index jobs for archived realms. fetchAllRealmsWithOwners
stays unchanged: full-reindex uses it as a lookup (URL → username), not
as a source, and the unarchive handler legitimately needs to reach a
just-archived row to enqueue its restore reindex.
handle-archive-realm calls cancelAllJobsInConcurrencyGroup on
`indexing:${realmURL}` after archiveRealm. This mirrors the realm-level
cancel-jobs endpoint (Realm.handleCancelJobsRequest): in-flight
from-scratch / incremental-index jobs are marked rejected, the pending
queue is dropped, reservations are released, and a NOTIFY jobs_finished
fans out to peer replicas. The unarchive flow rebuilds boxel_index from
disk via the existing full-reindex enqueue, so any partial work left
behind by an in-flight cancellation is discarded on restore.
Catalog / public realms are unaffected because the archive endpoint
rejects them with 422 — they can never carry archived_at.
Tests:
- full-reindex-test: getFullReindexRealmUrls returns only active realms
across two registered realms with one archived; an unarchived realm
reappears on the next call.
- archive-realm-test: a pending from-scratch-index job for the realm is
marked rejected once the archive endpoint runs.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Contributor
Host Test Results 1 files 1 suites 2h 13m 55s ⏱️ Results for commit 7951fbf. Realm Server Test Results 1 files ±0 1 suites ±0 10m 51s ⏱️ +27s Results for commit 7951fbf. ± Comparison against earlier commit c95faa9. |
lukemelia
changed the base branch from
cs-11665-enumeration-omits-archived-realms-from-_realm-auth-realm
to
main
…-archived-realms-in-the-full-index-sweep-stop # Conflicts: # packages/realm-server/tests/server-endpoints/archive-realm-test.ts
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Archived realms shouldn't consume indexing. Two changes:
getFullReindexRealmUrlsexcludes realms withrealm_metadata.archived_at IS NOT NULL, so the Grafana/_full-reindexendpoint and the post-deployment hook never enqueue from-scratch jobs for archived realms.archiveRealmwrites the flag,handle-archive-realmcallscancelAllJobsInConcurrencyGroup(dbAdapter, 'indexing:' + realmURL). In-flight and pendingfrom-scratch-index/incremental-indexjobs are marked rejected, reservations are released, and aNOTIFY jobs_finishedfans out to peer replicas. Mirrors the realm-level cancel-jobs endpoint (Realm.handleCancelJobsRequest). Partial work left behind by an in-flight cancellation is discarded by the unarchive-side full reindex.What this PR doesn't change:
fetchAllRealmsWithOwners—full-reindex.tsuses it as a lookup (URL → username), not as a source. The source list comes fromgetFullReindexRealmUrls(system sweep) or from the unarchive handler (which legitimately needs to reach a just-archived row).archived_atand the filter is a no-op for them.Tests
full-reindex-test:getFullReindexRealmUrlsreturns only active realms across two registered realms with one archived; an unarchived realm reappears on the next call.archive-realm-test: a pendingfrom-scratch-indexjob for the realm is marked rejected once the archive endpoint runs.Test plan
pnpm testinpackages/realm-server(full-reindex-test,server-endpoints/archive-realm-test) — requires Docker for the test PG; will verify in CI.