Break endless upload loop for removed placeholder items - #172
Open
Matthias-Huehne-Kiteworks wants to merge 2 commits into
Open
Matthias-Huehne-Kiteworks wants to merge 2 commits into
Matthias-Huehne-Kiteworks wants to merge 2 commits into
Conversation
…er items The sync pipeline could get stuck in an endless loop: the Status view kept showing repeating "[CORE, Replay] Found removedItems=(…)" entries for placeholder items (synthetic "_placeholder_" fileID/eTag) that were flagged `removed` but still carried active upload sync records, while their filenames accumulated ever more timestamp suffixes. Resolving the recurring Cancel/Retry issue dialog did not break the loop. Root cause: when an item that still had a pending/queued upload was removed, several gaps combined into a permanent deadlock: - OCSyncActionUpload rescheduled uploads for removed items, hit the "already exists" case, raised a keep-both issue and looped. - OCSyncActionDelete left the item's pending upload records running. - scrubItemSyncStatus skipped removed items entirely, so a removed item with active-but-non-existent sync record IDs was never scrubbed and never vacuumed (activeSyncRecordIDs.count > 0) - a state it could never leave. - Deleting a placeholder issued a doomed server DELETE with a synthetic fileID. Fixes: - OCSyncActionUpload.m: scheduleWithContext: bails out early when the item has meanwhile been removed (looked up via retrieveCacheItemForLocalID:, which only returns non-removed items). It deschedules the record with OCErrorCancelled and returns ProcessNext instead of re-uploading. New helper _localItemHasBeenRemoved. This is the core loop-breaker. - OCCore+SyncEngine.m: scrubItemSyncStatus no longer ignores removed items. The existing dangling-record check still protects items with live records, so removed items whose sync records no longer exist now get their sync activity cleared, letting the Vacuum item policy purge them. This drains any pre-existing stuck state on next launch - no account reset required. - OCSyncActionDelete.m: preflightWithContext: cancels all pending upload sync records for the item (and contained items) via the new helper _cancelPendingUploadsForItem:excludingSyncRecordID:, dropping each cancelled record's reference before commit. scheduleWithContext: short-circuits placeholder deletions locally instead of issuing a server DELETE that would fail. Verified on a physical device carrying the actual stuck state: resolving the issue dialog once now cascades through the backlog, the lanes drain and the leftover items are scrubbed and vacuumed. The loop no longer recurs. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Collaborator
|
@Matthias-Huehne-Kiteworks After a brief look, I realize that reviewing this as-is would, without benefit, consume a lot of my time, since even in the first lines and at first glance there's stuff like this:
(when this is clearly some piece of log output, which never appears in the Status view) or code like this: if ((activeSyncRecordIDs = [activeSyncRecordIDs copy]) == nil) { return; }(Why not compare against nil directly? Why even try to make a copy - at all?) Therefore, may I ask you to please go through the PR's description and changes yourself first - and fix what must also stand out to you, making it ready for consumption by a fellow human? I'll then happily do a code review. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The sync pipeline could get stuck in an endless loop. The Status view kept showing repeating
[CORE, Replay] Found removedItems=(…)entries for placeholder items (synthetic_placeholder_fileID/eTag) that were flaggedremovedbut still carried active upload sync records, while their filenames accumulated ever more timestamp suffixes. Resolving the recurring Cancel / Retry issue dialog did not break the loop.Root cause
When an item that still had a pending/queued upload was removed (e.g. deleted locally, or its upload descheduled), several gaps combined into a permanent deadlock:
OCSyncActionUpload scheduleWithContext:had no check for whether the item still exists. It re-uploaded a removed item, hit the "already exists" case, raised a keep-both issue and looped.OCSyncActionDeleteleft the item's pending upload records running after the item was gone.scrubItemSyncStatusskipped removed items (if (!item.removed)). A removed item with active-but-non-existent sync record IDs was therefore never scrubbed and never vacuumed (activeSyncRecordIDs.count > 0) — a state it could never leave.DELETEwith a synthetic_placeholder_fileID.Changes
OCSyncActionUpload.m— loop-breaker guard (core fix)scheduleWithContext:bails out early when the item has meanwhile been removed (looked up viaretrieveCacheItemForLocalID:, which only returns non-removed items, sonilmeans removed). It deschedules the record withOCErrorCancelledand returnsProcessNextinstead of re-uploading._localItemHasBeenRemoved.OCCore+SyncEngine.m— scrub removed items (cleanup backstop)scrubItemSyncStatusno longer ignores removed items. The existing dangling-record check still protects items with live records, so removed items whose sync records no longer exist now get their sync activity cleared, letting the Vacuum item policy purge them. This drains any pre-existing stuck state on next launch — no account reset required.OCSyncActionDelete.m— cancel uploads on delete + handle placeholderspreflightWithContext:cancels all pending upload sync records for the item (and contained items) via the new helper_cancelPendingUploadsForItem:excludingSyncRecordID:, dropping each cancelled record's reference before commit.scheduleWithContext:short-circuits placeholder deletions locally instead of issuing a serverDELETEthat would fail.Validation
Built and deployed to a physical device carrying the actual stuck state. Resolving the issue dialog once now cascades through the backlog (each upload self-cancels via the new guard), the lanes drain and the leftover items are scrubbed and vacuumed. The endless loop no longer recurs.
Notes / follow-up
OCSyncActionUploadunconditionally appends a timestamp and assigns a fresh placeholder fileID on every resolution — the source of the filename explosion for non-removed items. Worth a follow-up (only append when no timestamp is present yet).