Skip to content

feat: implement lazy virtual drive metadata synchronization - #470

Open
egalvis27 wants to merge 3 commits into
feat/prepare-database-for-best-syncfrom
feat/add-lazy-metadata-loading
Open

egalvis27 wants to merge 3 commits into
feat/prepare-database-for-best-syncfrom
feat/add-lazy-metadata-loading

Conversation

@egalvis27

Copy link
Copy Markdown

What is Changed / Added


Lazy directory hydration

  • Added LazyVirtualDriveHydrator to resolve virtual-drive paths and hydrate directory children on demand.
  • readDirectory resolves a directory, refreshes its children when the cached state is stale, and returns local folder/file entries.
  • ensurePathLoaded hydrates a path's parent directory before retrying file lookups.
  • Added per-directory in-flight request deduplication, keyed by folderId:statusScope, so concurrent FUSE operations share the same remote hydration request.

Remote directory persistence

  • Added hydrateDirectoryChildren, which:

    • Fetches folder content from the Drive API.
    • Filters non-existing or removed folders/files.
    • Maps API DTOs into remote sync entities.
    • Persists folders and files sequentially through SQLite batch operations.
    • Removes stale local descendants before storing the refreshed children.
    • Invalidates cached directory state for deleted descendant folders.
    • Recreates local Folder and File entities and updates the corresponding repositories.
    • Marks directory state as loaded on success or errored on failure.
  • Added dedicated remote file/folder mappers with direct tests for:

    • Existing status assignment.
    • Nullable fileId normalization.
    • File-size conversion from API string values.

FUSE operation integration

  • Updated opendir to use lazy hydration before listing local directory entries.
  • Updated getAttributes to hydrate the parent directory and retry local file/folder lookup when an entry is not initially present.
  • Updated open to hydrate and retry file lookup before returning ENOENT.
  • Preserved FUSE-specific errors from the hydration layer instead of converting them into generic I/O errors.

Why


The Virtual Drive must remain usable while the remote account contains a very large number of files and folders.

Previously, FUSE operations depended on the local catalog being fully synchronized before users could browse directories or access files. This change allows the application to fetch and persist only the directory children required by the current filesystem operation.

The directory-state cache avoids repeatedly fetching recently loaded folders, while the in-flight request registry prevents duplicate API calls when multiple FUSE requests target the same stale directory concurrently. Stale-child cleanup and directory-state invalidation keep the local cache coherent after remote changes.

@coderabbitai

coderabbitai Bot commented Sep 30, 2026 •

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration
  • Configuration used: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 0758f5cc-1c3b-4cf6-bc06-ce24e93d4e72

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@egalvis27
egalvis27 force-pushed the feat/prepare-database-for-best-sync branch from 28267c9 to 28117b0 Compare October 9, 2026 20:23
@egalvis27
egalvis27 force-pushed the feat/add-lazy-metadata-loading branch from 4ae6839 to 425bca8 Compare October 9, 2026 20:28
@sonarqubecloud

sonarqubecloud Bot commented Oct 9, 2026

Copy link
Copy Markdown

Quality Gate Failed Quality Gate failed

Failed conditions
66.1% Coverage on New Code (required ≥ 80%)

See analysis details on SonarQube Cloud

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant