Skip to content

Add read-only upload history and process reports - #830

Open
rtwojcik wants to merge 1 commit into
mapillary:mainfrom
rtwojcik:upload-history-process-report
Open

rtwojcik wants to merge 1 commit into
mapillary:mainfrom
rtwojcik:upload-history-process-report

Conversation

@rtwojcik

Copy link
Copy Markdown

Summary

  • Add an unauthenticated check_upload_history command that uses the same
    candidate discovery, ordering, and checksums as upload without network or
    history writes.
  • Return ordered already_uploaded_filenames for each candidate, including
    image matches found across historical sequence boundaries; preserve the
    existing exact-sequence behavior for images, videos, and ZIP files.
  • Tolerate missing, corrupt, non-UTF-8, and malformed history records while
    scanning stored image descriptions.
  • Add optional process --process_report_path version 1 JSON output for
    recognized and unsupported files. Existing processing behavior is unchanged
    when the option is omitted.
  • Document both contracts and add unit and CLI integration coverage.

Motivation

Mapillary Desktop Uploader needs to explain which files were uploaded before
the user starts an upload and which selected files are unsupported. Today the
upload-time history listener can only silently skip whole candidates, and
process silently ignores unsupported inputs.

Compatibility and privacy

  • The new history command is read-only and does not require authentication.
  • Existing upload-time history checks remain the final race-condition guard.
  • already_uploaded remains an all-members result; the new ordered filename
    list adds per-file detail without weakening exact candidate checks.
  • Process reports are opt-in, written only to the requested local path, and
    contain absolute source filenames.
  • Directory reports ignore hidden/system entries and common GPS/metadata
    sidecars; explicit unsupported regular files are always reported.

Test plan

  • pytest -q tests — 719 passed, 52 skipped
  • ruff check mapillary_tools
  • ruff format --check mapillary_tools tests
  • usort diff mapillary_tools
  • mypy mapillary_tools tests/cli
  • Built both ARM64 PyInstaller layouts with script/build_osx
  • Ran the packaged folder-mode history/process-report integration smoke — 4
    passed
  • Verified both new options in packaged CLI help

The sandbox blocks execution of the one-file PyInstaller binary at its System V
semaphore initialization step. The folder-mode packaged executable runs and
passes the functional smoke; both generated binaries are ARM64 and pass local
code-signature verification.

Summary:
- add an unauthenticated check_upload_history command that shares upload candidate discovery and checksum semantics
- report ordered per-file image matches across sequence boundaries while tolerating malformed local history
- add an optional versioned process report for unsupported inputs without changing default processing behavior
- document both JSON contracts and cover them with unit, integration, and read-only parity tests

Test Plan:
- pytest -q tests (719 passed, 52 skipped)
- ruff check mapillary_tools
- ruff format --check mapillary_tools tests
- usort diff mapillary_tools
- mypy mapillary_tools tests/cli
- PyInstaller ARM64 onefile and folder builds
- packaged folder-mode history/process-report integration smoke (4 passed)
@meta-cla

meta-cla Bot commented Sep 18, 2026

Copy link
Copy Markdown

Hi @rtwojcik!

Thank you for your pull request and welcome to our community.

Action Required

In order to merge any pull request (code, docs, etc.), we require contributors to sign our Contributor License Agreement, and we don't seem to have one on file for you.

Process

In order for us to review and merge your suggested changes, please sign at https://code.facebook.com/cla. If you are contributing on behalf of someone else (eg your employer), the individual CLA may not be sufficient and your employer may need to sign the corporate CLA.

Once the CLA is signed, our tooling will perform checks and validations. Afterwards, the pull request will be tagged with CLA signed. The tagging process may take up to 1 hour after signing. Please give it that time before contacting us about it.

If you have received this in error or have any questions, please contact us at cla@meta.com. Thanks!

@meta-cla meta-cla Bot added the cla signed label Sep 18, 2026
@meta-cla

meta-cla Bot commented Sep 18, 2026

Copy link
Copy Markdown

Thank you for signing our Contributor License Agreement. We can now accept your code for this (and any) Meta Open Source project. Thanks!

@rtwojcik
rtwojcik requested a review from caglarpir September 18, 2026 12:21
@rtwojcik
rtwojcik marked this pull request as ready for review September 18, 2026 12:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant