From 55010d6ebfdce90333aff2b41528ae56222cd59b Mon Sep 17 00:00:00 2001 From: Divjot Arora Date: Thu, 6 Aug 2026 10:18:19 +0000 Subject: [PATCH 1/3] Specify handling for unrecognized logical types --- LogicalTypes.md | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/LogicalTypes.md b/LogicalTypes.md index fae874c0..eab68de0 100644 --- a/LogicalTypes.md +++ b/LogicalTypes.md @@ -40,6 +40,8 @@ according to well defined conversion rules. ### Compatibility +#### ConvertedType + The Thrift definition of the metadata has two fields for logical types: `ConvertedType` and `LogicalType`. `ConvertedType` is an enum of all available annotations. Since Thrift enums can't have additional type parameters, it is cumbersome to define additional type parameters, like decimal scale and precision @@ -54,6 +56,13 @@ in case `LogicalType` annotations are not present. Parquet writers must always w `LogicalType` annotations where applicable, but must also write the corresponding `ConvertedType` annotations (if any) to maintain compatibility with old readers. +#### Unsupported Logical Types + +When reading a file written by a newer writer that contains an unrecognized logical type or an +unrecognized logical/physical type combination on a column, readers should ignore both the logical +type annotation and statistics for that column. Only the physical type information should be used +to process the column's data. + Compatibility considerations are mentioned for each annotation in the corresponding section. ## String Types From e2061ed69e2f006bb67fbc2f6d2404ec43179334 Mon Sep 17 00:00:00 2001 From: Divjot Arora Date: Mon, 31 Aug 2026 23:06:34 +0000 Subject: [PATCH 2/3] update CONTRIBUTING.md --- CONTRIBUTING.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index f9fdf21a..e193f6f1 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -126,8 +126,9 @@ For the purposes of this discussion we classify features into the following buck the feature enabled can be read under an older version of the format, but some metadata might be missing or performance might be suboptimal. Simply phrased, forward compatible means all data can be read back in an older - version of the format. New logical types are considered forward - compatible despite the loss of semantic meaning. + version of the format. New logical types and new combinations of existing + logical/physical types are considered forward compatible despite the loss + of semantic meaning. 3. Forward incompatible. A file written under a newer version of the format with the feature enabled cannot be read under an older version of the format (e.g. From 16d5deba359856942ee278031ab9ee719eb7ce7a Mon Sep 17 00:00:00 2001 From: Divjot Arora Date: Wed, 2 Sep 2026 16:24:38 +0000 Subject: [PATCH 3/3] address wordign --- LogicalTypes.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/LogicalTypes.md b/LogicalTypes.md index eab68de0..95969290 100644 --- a/LogicalTypes.md +++ b/LogicalTypes.md @@ -60,7 +60,7 @@ in case `LogicalType` annotations are not present. Parquet writers must always w When reading a file written by a newer writer that contains an unrecognized logical type or an unrecognized logical/physical type combination on a column, readers should ignore both the logical -type annotation and statistics for that column. Only the physical type information should be used +type annotation and column order for that column. Only the physical type information should be used to process the column's data. Compatibility considerations are mentioned for each annotation in the corresponding section.