Skip to content

GH-3760: Clean up hadoop README.md #3761

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
ggershinsky wants to merge 3 commits into
base: master
Choose a base branch
from
+17 −23
Open

GH-3760: Clean up hadoop README.md #3761

Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
b466ff7
add spaces
ggershinsky Sep 2, 2026
43443b7
rm duplicate, fix additional parameters
ggershinsky Sep 2, 2026
005ac3a
2 more
ggershinsky Sep 2, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
40 changes: 17 additions & 23 deletions parquet-hadoop/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -176,7 +176,7 @@ If the frequency is low, the performance will be better.
---

**Property:** `parquet.page.value.count.threshold`
**Description:** The value count threshold within a Parquet page used on each page check.
**Description:** The value count threshold within a Parquet page used on each page check.
**Default value:** `Integer.MAX_VALUE / 2`

---
Expand Down Expand Up @@ -222,15 +222,15 @@ conf.set("parquet.bloom.filter.enabled#column.path", false);
If it is true, the bloom filter will be generated with the optimal bit size
according to the number of real data distinct values. If it is false, it will not take effect.
Note that the maximum bytes of the bloom filter will not exceed `parquet.bloom.filter.max.bytes` configuration (if it is
set too small, the generated bloom filter will not be efficient).
set too small, the generated bloom filter will not be efficient).
**Default value:** `false`

---

**Property:** `parquet.bloom.filter.candidates.number`
**Description:** The number of candidate bloom filters written at the same time.
When `parquet.bloom.filter.adaptive.enabled` is true, multiple candidate bloom filters will be inserted
at the same time, finally a bloom filter with the optimal bit size will be selected and written to the file.
at the same time, finally a bloom filter with the optimal bit size will be selected and written to the file.
**Default value:** `5`

---
Expand Down Expand Up @@ -268,13 +268,13 @@ conf.set("parquet.bloom.filter.fpp#column.path", 0.02)

**Property:** `parquet.decrypt.off-heap.buffer.enabled`
**Description:** Whether to use direct buffers to decrypt encrypted files. This should be set to
true if the reader is using a `DirectByteBufferAllocator`
true if the reader is using a `DirectByteBufferAllocator`
**Default value:** `false`

---

**Property:** `parquet.block.row.count.limit`
**Description:** The maximum number of rows per row group.
**Property:** `parquet.block.row.count.limit`
**Description:** The maximum number of rows per row group.
**Default value:** `2147483647` (Integer.MAX_VALUE)

---
Expand All @@ -288,12 +288,6 @@ true if the reader is using a `DirectByteBufferAllocator`
**Description:** Whether to write out page level checksums.
**Default value:** `true`

---

**Property:** `parquet.crypto.factory.class`
**Description:** Class implementing EncryptionPropertiesFactory.
**Default value:** None. If not set, the file won't be encrypted by a crypto factory.

## Class: ParquetInputFormat

**Property:** `parquet.read.support.class`
Expand Down Expand Up @@ -406,7 +400,7 @@ ParquetInputFormat to materialize records. It should be a the descendant class o
## Class: HadoopReadOptions

**Property:** `parquet.crypto.factory.class`
**Description:** Class implementing DecryptionPropertiesFactory.
**Description:** Class implementing DecryptionPropertiesFactory.
**Default value:** None. If not set, the file won't be decrypted by a crypto factory.

## Class: PropertiesDrivenCryptoFactory
Expand Down Expand Up @@ -469,12 +463,12 @@ If `false`, write files in encrypted footer mode, that fully encrypts the footer

---

**Property:** `parquet.encryption.kms.enable.url.read`
**Property:** `parquet.encryption.kms.enable.url.read`
**Description:** If a KMS URL is set by writers, it will be stored in the key material. However, by default, it will not be
given to readers because the storage is untrusted. Readers that need the URL should set the KMS URL property.
If they cannot do so, they can enable retrieving the KMS URL from the stored key material by setting this parameter
to `true`. KMS client implementations must validate the URL value and use authentication to prevent key material
tampering attacks that could, for example, result in a KMS access token being sent to a malicious URL endpoint.
tampering attacks that could, for example, result in a KMS access token being sent to a malicious URL endpoint.
**Default value:** `false`

---
Expand Down Expand Up @@ -520,17 +514,17 @@ If `false`, key material is stored in separate new files, created in the same fo
**Property:** `parquet.hadoop.vectored.io.enabled`
**Description:** Flag to enable use of the FileSystem Vector IO API on Hadoop releases which support the feature.
If `true` then an attempt will be made to dynamically load the relevant classes;
if not found then the library will use the classic non-vectored reads: it is safe to enable this option on older releases.
if not found then the library will use the classic non-vectored reads: it is safe to enable this option on older releases.
**Default value:** `true`

---

**Property:** `parquet.column.statistics.enabled`
**Property:** `parquet.column.statistics.enabled`
**Description:** Whether to enable column statistics collection.
If `true`, statistics will be collected for all columns unless explicitly disabled for specific columns.
If `false`, statistics will be disabled for all columns regardless of column-specific settings.
It is possible to enable or disable statistics for specific columns by appending `#` followed by the column path.
**Default value:** `true`
It is possible to enable or disable statistics for specific columns by appending `#` followed by the column path.
**Default value:** `true`
**Example:**
```java
// Enable statistics for all columns
Expand All @@ -542,12 +536,12 @@ conf.set("parquet.column.statistics.enabled#column.path", false);

---

**Property:** `parquet.size.statistics.enabled`
**Description:** Whether to enable size statistics collection.
**Property:** `parquet.size.statistics.enabled`
**Description:** Whether to enable size statistics collection.
If `true`, size statistics will be collected for all columns unless explicitly disabled for specific columns.
If `false`, size statistics will be disabled for all columns regardless of column-specific settings.
It is possible to enable or disable size statistics for specific columns by appending `#` followed by the column path.
**Default value:** `true`
It is possible to enable or disable size statistics for specific columns by appending `#` followed by the column path.
**Default value:** `true`
**Example:**
```java
// Enable size statistics for all columns
Expand Down
Loading