Improve performance of snowflake ids in cron jobs - #64944
Open
CarlSchwan wants to merge 3 commits into
Open
CarlSchwan wants to merge 3 commits into
CarlSchwan wants to merge 3 commits into
Conversation
ISequence::nextId() takes the server id, the seconds and the milliseconds, but the generator passed the seconds, the milliseconds and the server id, so the sequences were keyed on the wrong values. Assisted-by: ClaudeCode:claude-opus-5-5 Signed-off-by: Carl Schwan <carl@carlschwan.eu>
Store one slot per second of the TTL window and per millisecond in each lock file, so generating an id only reads and writes 8 bytes instead of decoding, filtering and rewriting a JSON map of the last 30 seconds. The fsync is not needed as the files only coordinate processes running at the same time. Generating an id went from 660µs to 22µs with the temporary directory on disk, and from 24µs to 11µs on tmpfs. Assisted-by: ClaudeCode:claude-opus-5-5 Signed-off-by: Carl Schwan <carl@carlschwan.eu>
CarlSchwan
requested review from
Altahrim,
come-nc,
icewind1991 and
leftybournes
and removed request for
a team
September 30, 2026 15:39
Member
Author
|
/backport to stable35 |
Member
Author
|
/backport to stable34 |
Member
Author
|
/backport to stable33 |
Member
|
A description of the algorithm would be nice |
Altahrim
approved these changes
Oct 1, 2026
This allow to simplify a bit the code, also add a small description of the algorithm. Assisted-by: ClaudeCode:claude-opus-5-5 Signed-off-by: Carl Schwan <carl@carlschwan.eu>
CarlSchwan
force-pushed
the
worktree-snowflake-file-sequence
branch
from
October 1, 2026 08:18
7b9de62 to
e69b8ed
Compare
Member
Author
|
Ci failure is unrelated and will be fixed with #64975 |
artonge
approved these changes
Oct 1, 2026
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Improve greatly speed of preview migration job as this is generating a lot of snowflake ids
Checklist
3. to review, feature component)stable32)AI (if applicable)