Skip to content

HDDS-4333. [OEP] Support append operation - #11434

Draft
smengcl wants to merge 2 commits into
apache:masterfrom
smengcl:HDDS-4333-append-oep
Draft

smengcl wants to merge 2 commits into
apache:masterfrom
smengcl:HDDS-4333-append-oep

Conversation

@smengcl

@smengcl smengcl commented Oct 7, 2026

Copy link
Copy Markdown
Contributor

Generated-by: Codex (GPT-6 Astra), Claude Code (Fable 5.1)

What changes were proposed in this pull request?

Add an Ozone Enhancement Proposal for FileSystem.append() support in OzoneFS (ofs:// and o3fs://), as hadoop-hdds/docs/content/design/append.md plus four diagrams. This is a design doc for discussion only. No code changes.

Summary of the proposal:

  • Reopen a closed file for a single appender. Existing blocks stay immutable and appended bytes go to newly allocated blocks, so no old block or closed container is reopened.
  • Ownership is an append session recorded on the committed and open records in the existing OM tables, with a renewable lease, fencing of stale sessions, and explicit retry and recovery behavior. No new append table and no general block reference counting.
  • RATIS and RATIS Streaming keep their existing hflush/hsync behavior. EC append publishes on close only.
  • FSO is supported. LEGACY is included if its adaptations stay small. OBJECT_STORE buckets reject filesystem append.
  • Snapshots keep reading their captured lengths, and deletion reclaims only the blocks that no snapshot references.
  • Gated by an OM layout feature and ozone.om.append.enabled, disabled by default.

Out of scope: truncate, concat, random writes, concurrent appenders, native S3 or object append, EC hflush/hsync.

The doc also covers rename and other namespace operations during an open append, quota accounting, performance considerations, alternatives, the implementation sequence, and a validation plan. It builds on the earlier design in #1515, which was closed without merging.

What is the link to the Apache JIRA

https://issues.apache.org/jira/browse/HDDS-4333

How was this patch tested?

Docs only, no tests.

🤖 Generated with Claude Code

Copilot AI balanced review requested due to automatic review settings October 7, 2026 08:02

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot wasn't able to review any files in this pull request. Check if the Files changed in this pull request are included in default exclusions.

@smengcl smengcl added the design label Oct 7, 2026
@smengcl smengcl self-assigned this Oct 7, 2026
@github-actions github-actions Bot added the documentation Improvements or additions to documentation label Oct 7, 2026
@smengcl smengcl changed the title HDDS-4333. Add OEP for append support in Ozone HDDS-4333. [OEP] Support append operation Oct 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

design documentation Improvements or additions to documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants