Repository navigation
fix(mailer): retry the configured template after a default fallback - #2911
Open
mansur-akhmadi wants to merge 1 commit into
Open
mansur-akhmadi wants to merge 1 commit into
mansur-akhmadi wants to merge 1 commit into
Conversation
When the first load of a template fails and there is no previous entry, the cache stores the default template. Cache.get treated that entry like any other and served it until TemplateMaxAge (10m by default), so a short outage of the template endpoint at the wrong moment meant default emails for the next ten minutes. The background reloader already skips the max age check for default entries and retries them after TemplateRetryInterval. Do the same in get, which is the only path when template reloading is disabled (the default).
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What kind of change does this PR introduce?
Bug fix. Fixes #2910
What is the current behavior?
If the first fetch of a template fails and nothing is cached yet, the cache stores the default template.
Cache.gettreats that entry like a normal one and serves it untilTemplateMaxAge(10m) runs out.TemplateRetryInterval(10s) only kicks in after that. With template reloading disabled (the default) one failed fetch at a bad moment means ten minutes of default emails.The background reloader already handles this:
reloadAtskips the max age check whenent.defis set.What is the new behavior?
Cache.getskips the max age check for default fallback entries too, so the configured template is retried afterTemplateRetryInterval. Nothing else changes:TemplateMaxAge;defstays false on the copy);How this change was tested
New
TestCacheGetininternal/mailer/templatemailer/template_test.gouses a fake clock and anhttptestserver that counts requests. It covers the cases above, plus the background reloader path.DefaultFallbackIsRetriedAfterRetryIntervalfails on master (no retry happens, request count stays at 1) and passes with the fix.go test -race -count=20 ./internal/mailer/...,go vet ./...,make static secandmake check-formatpass locally.Additional context
The tradeoff: if the template endpoint is down for a long time, mail requests for that type now try the fetch every
TemplateRetryIntervalrather than everyTemplateMaxAge. That's the same retry behaviour expired custom templates already get, and what the background reloader already does for the fallback.AI disclosure
AI assistance was used for the investigation and for drafting the test. I reviewed every line and reproduced the fail-before / pass-after locally.