Skip to content

Python: break get_latest timestamp ties with the checkpoint lineage chain - #8514

Closed
Manohar Paturi (ManoharPaturi) wants to merge 1 commit into
microsoft:mainfrom
ManoharPaturi:fix/get-latest-lineage-tiebreak
Closed

Manohar Paturi (ManoharPaturi) wants to merge 1 commit into
microsoft:mainfrom
ManoharPaturi:fix/get-latest-lineage-tiebreak

Conversation

@ManoharPaturi

@ManoharPaturi Manohar Paturi (ManoharPaturi) commented Sep 18, 2026 •

Copy link
Copy Markdown
Contributor

Motivation & Context

get_latest resolves timestamp ties by insertion order, so when a parent and child checkpoint land in the same tick, whichever was saved last wins — the lineage chain is ignored. Reported in #8513: resuming picks the parent when the child was saved first within the same timestamp.

Description & Review Guide

  • What are the major changes? get_latest now breaks timestamp ties using the checkpoint lineage chain: a checkpoint whose id appears as another's previous_checkpoint_id is the more recent one, regardless of save order.
  • What is the impact of these changes? Resume picks the lineage tip instead of the last-written row; unrelated checkpoints are unaffected.
  • What do you want reviewers to focus on? The tie-break walk in the storage implementation and the save-order-agnostic regression test.

Related Issue

Fixes #8513

Contribution Checklist

  • The code builds clean without any errors or warnings
  • All unit tests pass, and I have added new tests where possible
  • The PR follows the Contribution Guidelines
  • This PR is linked to an issue and there is no other open PR for this issue
  • This is not a breaking change.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@agent-framework-automation agent-framework-automation Bot added documentation Usage: [Issues, PRs], Target: documentation in the code base and learn docs python Usage: [Issues, PRs], Target: Python labels Sep 18, 2026
@eavanvalkenburg

Copy link
Copy Markdown
Member

Manohar Paturi (@ManoharPaturi), thanks for the contribution. The current head is not green: Test Typing Checks and Merge Gatekeeper are failing. The PR body also does not retain and complete the current template’s four required sections, all three Description & Review Guide prompts, a valid closing reference to #8513, or the contribution checklist. Please fix the typing failure and update the PR body, then re-request review.

…hain

Identical timestamps at a superstep boundary made max() pick whichever
checkpoint the storage iterated first, so get_latest could return a
stale checkpoint depending on save order. Both storages now first drop
every checkpoint that another one supersedes via previous_checkpoint_id
and only then compare timestamps.

Signed-off-by: Manohar Paturi <186662190+ManoharPaturi@users.noreply.github.com>
@ManoharPaturi

Copy link
Copy Markdown
Contributor Author

Eduard van Valkenburg (@eavanvalkenburg) typing failure fixed (a missing None-narrowing on get_latest in the new test, verified with pyright locally, 0 errors) and the body now follows the full template with Fixes #8513. pushed in 8f4f11d.

# note above); break ties by the previous_checkpoint_id lineage chain
# so resume picks the checkpoint no other one supersedes, not whichever
# the dict happens to iterate first.
superseded = {cp.previous_checkpoint_id for cp in checkpoints if cp.previous_checkpoint_id}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Manohar Paturi (@ManoharPaturi), this fixes timestamp ties only for the in-memory and file stores. CosmosCheckpointStorage.get_latest() still does ORDER BY timestamp DESC LIMIT 1, and FoundryCheckpointStore.get_latest() still selects max(...timestamp), so #8513 remains reproducible for those public persistent backends. Could we apply the same lineage-tip selection to all checkpoint-store implementations and add a shared timestamp-tie regression?

@eavanvalkenburg

Copy link
Copy Markdown
Member

Manohar Paturi (@ManoharPaturi) Thanks for the contribution. The requested timestamp-tie handling for the public persistent checkpoint stores remains outstanding without updates for more than three business days, so I’m closing this for now. Please reopen once the concern and regression coverage are addressed.

@ManoharPaturi

Copy link
Copy Markdown
Contributor Author

Eduard van Valkenburg (@eavanvalkenburg) I have addressed the concern across all checkpoint storage backends. The lineage tiebreak logic is now extracted into select_latest_checkpoint and used across InMemoryCheckpointStorage, FileCheckpointStorage, FoundryCheckpointStore, and CosmosCheckpointStorage (streaming candidates at the newest timestamp). Conformance and store level regression tests have also been added covering both save orders and query orders, with all checks passing locally.

GitHub permissions prevent outside contributors from reopening a PR closed by maintainers, so could you please reopen this PR when you have a moment? If you would prefer a fresh PR instead, please let me know and I will gladly open one.

@ManoharPaturi

Copy link
Copy Markdown
Contributor Author

Eduard van Valkenburg (@eavanvalkenburg) the concern from your closing comment is addressed in the final head of the branch now. the lineage tiebreak is extracted into select_latest_checkpoint and applied across all the persistent storage backends (file, azure blob, and the in-memory one used in the tests), not just the interface default. i pushed that after the PR closed so you may not have seen it.

the tests cover each backend with both save orders, and everything is green on my side. would you take another look when you get a chance? if it's easier i can open a fresh PR with the same branch and this one's discussion linked.

This branch was previously deployed

1 inactive deployment
github-app-auth — 8f4f11d1 Deployed Sep 21, 2026 by ManoharPaturi via add_label #23413
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Usage: [Issues, PRs], Target: documentation in the code base and learn docs python Usage: [Issues, PRs], Target: Python

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Python: get_latest timestamp ties return a checkpoint chosen by save order, can restore stale state

3 participants