Skip to content

HDDS-16672. Support RFC 2047 encoding for S3 user metadata - #11387

Open
F64116045 wants to merge 3 commits into
apache:masterfrom
F64116045:HDDS-16672
Open

F64116045 wants to merge 3 commits into
apache:masterfrom
F64116045:HDDS-16672

Conversation

@F64116045

@F64116045 F64116045 commented Oct 2, 2026 •

Copy link
Copy Markdown
Contributor

What changes were proposed in this pull request?

The ceph/s3-tests test_object_set_get_unicode_metadata exposed that S3 Gateway does not handle non-ASCII user metadata correctly.

Investigation showed that S3 uses RFC 2047 for non-ASCII x-amz-meta-* values. Ozone currently stores encoded values as ordinary text and does not encode them when returning response headers.

This patch decodes RFC 2047 Q/B values before storing metadata and encodes non-ASCII values in GET and HEAD responses.

References:

What is the link to the Apache JIRA?

https://issues.apache.org/jira/browse/HDDS-16672

How was this patch tested?

CI: https://github.com/F64116045/ozone/actions/runs/37118609154

Uploaded an object with this RFC 2047 encoded café metadata:

x-amz-meta-meta1: =?UTF-8?B?Y2Fmw6k=?=

The same request was tested against Ozone before and after this patch, and against S3.

B and Q are two RFC 2047 representations of the same café value.

Ozone before Ozone after S3
Stored value Encoded header text café Not observable
HEAD response B-encoded café Q-encoded café Q-encoded café

@github-actions github-actions Bot added the s3 S3 Gateway label Oct 2, 2026
Signed-off-by: F64116045 <f64116045@gs.ncku.edu.tw>
@F64116045 F64116045 changed the title HDDS-16672. Preserve Unicode values in S3 user metadata HDDS-16672. Support Unicode values in S3 user metadata Oct 2, 2026
@F64116045 F64116045 changed the title HDDS-16672. Support Unicode values in S3 user metadata HDDS-16672. Support RFC 2047 encoding for S3 user metadata Oct 2, 2026
Signed-off-by: F64116045 <f64116045@gs.ncku.edu.tw>
@F64116045

Copy link
Copy Markdown
Contributor Author

The original ceph/s3-tests case still fails because it sends raw non-ASCII metadata instead of RFC 2047. S3 does not round-trip that input either.

@F64116045
F64116045 marked this pull request as ready for review October 3, 2026 14:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

s3 S3 Gateway

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant