Skip to content

Docs: Add a File Format API page - #18587

Open
lisancao wants to merge 1 commit into
apache:mainfrom
lisancao:docs-file-format-api-page
Open

lisancao wants to merge 1 commit into
apache:mainfrom
lisancao:docs-file-format-api-page

Conversation

@lisancao

Copy link
Copy Markdown
Contributor

The File Format API shipped in 1.11.0 but the only documentation is the announcement blog post; the org.apache.iceberg.formats package has no page under the versioned docs. This adds docs/docs/file-format-api.md under API, next to FileIO: what a FormatModel and the FormatModelRegistry are, the read and write builders, engine schemas, the built-in object models, how an engine registers its own model, and what adding a new file format currently involves. The examples are taken from GenericReader, RegistryBasedFileWriterFactory and GenericFormatModels so they match the code. Related: #12225.

Test plan: make lint passes; built the dev site with mkdocs serve -f mkdocs-dev.yml and reviewed the rendered page and its nav entry.


AI Disclosure

  • Model: Claude Fable 5.1
  • Platform/Tool: Claude Code
  • Human Oversight: Reviewed the rendered page on a local docs build and checked the examples against the 1.12.0 source.
  • Prompt Summary: Draft a docs page for the File Format API from the Javadoc, the format model sources and the announcement post; the structure, prose and examples were generated, then checked against the 1.12.0 source and a local site build.

The File Format API shipped in 1.11.0 but only the announcement blog post
describes it. Add a page under API, next to FileIO, covering FormatModel and
the FormatModelRegistry, the read and write builders, engine schemas, the
built-in object models, how an engine registers its own model, and what
adding a new file format currently involves.

Generated-by: Claude Code
@github-actions github-actions Bot added the docs label Oct 19, 2026
@lisancao

Copy link
Copy Markdown
Contributor Author

@pvary if you have the availability to review this, I would be very grateful :)

@pvary

pvary commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor

@lisancao Thanks for the PR! I’d personally prefer to see FileFormat opened up and made configurable. Currently, the available formats are restricted by the FileFormat enum. If the community agrees to open this up to users, I think this new page would fit in very nicely as part of that change. Until that this is just Iceberg java internal stuff.

@lisancao

Copy link
Copy Markdown
Contributor Author

@lisancao Thanks for the PR! I’d personally prefer to see FileFormat opened up and made configurable. Currently, the available formats are restricted by the FileFormat enum. If the community agrees to open this up to users, I think this new page would fit in very nicely as part of that change. Until that this is just Iceberg java internal stuff.

Yes, totally makes sense since this is more of a reference doc at this point- would love to see more options opened for users to plug in their own formats and readers in the future. Let's revisit once that's sorted, otherwise it will be a bit premature to merge this in now.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants