Skip to content

Federated UploadPartCopy cannot reliably return the remote computed part checksum #64

Description

@Vonng

Problem

The legacy etcd federation branch of CopyObjectPartHandler forwards copied bytes to the remote destination by calling ordinary PutObjectPart.

After #46, a checksum-enabled remote multipart upload can accept an UploadPart request that omits a client checksum and persist a server-computed part checksum. However, AWS-compatible UploadPart responses omit checksum headers when the request did not provide one, while UploadPartCopy must return the computed checksum in CopyPartResult.

The proxy node can therefore receive a successful remote PutObjectPart response without the checksum it needs for its client-visible UploadPartCopy response. A pre-#46 remote can still reject the forwarded part entirely.

Relevant code:

  • cmd/object-multipart-handlers.go: isRemoteCopyRequired branch
  • minio-go Core.PutObjectPart response parsing
  • remote ListObjectParts as the only standard way to recover a persisted checksum

Relationship to #46

#46 will intentionally cover the normal local object-layer paths and establish these shared semantics:

  • server computation happens over logical plaintext before transformations;
  • UploadPart omits a computed checksum from the response when the client omitted it;
  • UploadPartCopy returns the computed checksum;
  • ListParts exposes the persisted checksum.

This federation issue must preserve those semantics. It must not solve the proxy problem by making every external UploadPart response return a server-computed checksum.

Proposed direction

For deployments where legacy federation remains supported:

  1. Prefer checksum fields already present in the remote PutObjectPart result.
  2. If all are empty, query remote ListObjectParts after the upload.
  3. Accept the listed checksum only when part number and canonical ETag match the just-uploaded result; otherwise report a conflict rather than returning another writers checksum.
  4. Populate CopyPartResult from the verified value.
  5. Document that all participating deployments must contain the UploadPart requires a per-part checksum header that AWS S3 computes server-side #46 upload-side fix.

An internal protocol extension may be considered only if the extra ListParts round trip is proven material. It would need explicit rolling-upgrade behavior and must remain invisible to external S3 clients.

If etcd federation is no longer a supported product surface, resolve this issue through an explicit deprecation decision instead of silently claiming complete UploadPartCopy compatibility.

Acceptance criteria

  • A federation test harness or reproducible two-deployment probe.
  • Checksum-enabled FULL_OBJECT and COMPOSITE UploadPartCopy responses include the correct checksum.
  • Concurrent overwrite of the same part number cannot produce an ETag/checksum pair from different uploads.
  • Mixed-version behavior is documented.
  • Ordinary external UploadPart response semantics remain AWS-compatible.

This issue is deliberately separate from #46 and should reuse its response and checksum metadata conventions.

Metadata

Metadata

Assignees

Labels

enhancementNew feature or request

Projects

Status
Done

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions