Repository navigation
Conversation
The File Format API shipped in 1.11.0 but only the announcement blog post describes it. Add a page under API, next to FileIO, covering FormatModel and the FormatModelRegistry, the read and write builders, engine schemas, the built-in object models, how an engine registers its own model, and what adding a new file format currently involves. Generated-by: Claude Code
|
@pvary if you have the availability to review this, I would be very grateful :) |
|
@lisancao Thanks for the PR! I’d personally prefer to see FileFormat opened up and made configurable. Currently, the available formats are restricted by the FileFormat enum. If the community agrees to open this up to users, I think this new page would fit in very nicely as part of that change. Until that this is just Iceberg java internal stuff. |
Yes, totally makes sense since this is more of a reference doc at this point- would love to see more options opened for users to plug in their own formats and readers in the future. Let's revisit once that's sorted, otherwise it will be a bit premature to merge this in now. |
The File Format API shipped in 1.11.0 but the only documentation is the announcement blog post; the
org.apache.iceberg.formatspackage has no page under the versioned docs. This addsdocs/docs/file-format-api.mdunder API, next to FileIO: what aFormatModeland theFormatModelRegistryare, the read and write builders, engine schemas, the built-in object models, how an engine registers its own model, and what adding a new file format currently involves. The examples are taken fromGenericReader,RegistryBasedFileWriterFactoryandGenericFormatModelsso they match the code. Related: #12225.Test plan:
make lintpasses; built the dev site withmkdocs serve -f mkdocs-dev.ymland reviewed the rendered page and its nav entry.AI Disclosure