Skip to content

Check-in reloads inactive groups every 10 minutes #7142

Open

Description

@zackdutra

Description

Next-gen check-in loads every group attached to every location under a kiosk into GroupCache, then keeps only the active groups in the kiosk's areas. Groups whose type is not under a check-in configuration get a hard-coded 10 minute cache lifespan. So every inactive, archived or non check-in group sitting on a campus location is reloaded, with its attributes, on the first check-in call after each 10 minute expiry, at every kiosk.

The cost grows with stale data nobody can see. At one of our campuses about 300 old groups (small groups, missions teams, vehicle placement groups, mostly inactive for years) were still attached to locations. Each rebuild ran about 760 database queries and took 1.3 seconds, enough to push family load times over our alert threshold every Sunday morning.

Numbers from our production (Rock 18.7, from Rock's OpenTelemetry spans):

Queries Duration
Kiosk rebuild before removing the stale group locations 757 1.3 s
Same kiosks after removing them 71 to 107 113 to 177 ms

We worked around it with a data view to workflow job that removes group locations from groups inactive for 90 days with no recent attendance. That works, but it seems like something check-in should not need.

Actual Behavior

CheckInDirector (KioskStatus path, around line 278) and OpportunityCollection.Create (around line 134) call GroupCache.Get on every group from GroupLocationCache.AllForLocationId before filtering on IsActive, IsArchived and GroupTypeId.

GroupCache.Lifespan (GroupCache.cs line 51/52) is 10 minutes when GroupType.GetCheckInConfigurationType() is null, and it is not configurable.

Together, the first KioskStatus, FamilyMembers or AttendeeOpportunities call after each 10 minute window reloads every non check-in group on the kiosk's locations. Calls in between run about 15 queries. Same code in 18.7.0, 19.7.0 and the current v20 and v21 branches.

Expected Behavior

Groups that check-in is going to discard (inactive, archived, or not in one of the kiosk's areas) should not cost a cache load.

One possible fix is to filter on group type and active state before loading the group, for example by carrying GroupTypeId and IsActive on GroupLocationCache, or by querying only group locations whose group type is in the kiosk's areas.

Steps to Reproduce

  • Attach a few hundred groups whose type is not under a check-in configuration (inactive is fine) to locations under a next-gen check-in kiosk.
  • Turn on Rock's observability (or watch SQL) and load the kiosk.
  • Watch /api/v2/checkin/KioskStatus, FamilyMembers or AttendeeOpportunities.
  • The first call after each 10 minute window runs hundreds of queries; calls in between run about 15.

Issue Confirmation

  • Perform a search on the Github Issues to see if your bug is already reported.
  • Reproduced the problem on a fresh install or on the demo site.

Rock Version

18.7.0

Client Culture Setting

en-US

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions