[jiangxt2]

Rationale for this change

ORCFile.read_stripe() materializes an entire stripe as one record batch. Dataset already scans ORC in batches; this adds native batching with explicit stripe selection to ORCFile, supporting incremental consumption and early termination.

Closes #54921.

What changes are included in this PR?

• Add ORCFile.iter_batches(batch_size=65536, stripes=None, columns=None) with column projection.
• Add stateless C++ GetStripeReader overloads that decode directly into batches without changing the reader's Seek position.
• Handle zero-column row counts and iterator lifetimes, with API documentation and regression tests.

Are these changes tested?

Validated on macOS arm64 with matching Release C++/Python builds, Python 3.13.7 and ORC 2.2.1.

cpp/build/dev/release/arrow-orc-adapter-test

44 passed; one existing timezone-database test restricted to ORC < 2.0 was skipped.

From python/:

python -m pytest -q -ra --enable-orc --enable-dataset --enable-pandas pyarrow/tests/test_orc.py pyarrow/tests/test_dataset.py::test_orc_format pyarrow/tests/test_dataset.py::test_orc_scan_options pyarrow/tests/test_dataset.py::test_orc_writer_not_implemented_for_dataset

238 passed with NumPy 2.5.4. In an isolated environment without NumPy, the iter_batches tests and test_without_numpy.py had 220 passes; only the marked NumPy integer-input test was skipped. Coverage includes five codecs, stripe order and duplicates, batch limits, nested/zero-column projections, invalid arguments, interleaved iterators and read failures. Both environments cover owner lifetime, early close and resource release for whole-file and selected-stripe iteration.

The C++ memory regression uses four int64 columns and batches of 256 rows. For single stripes of 262144 and 1048576 rows, whole-stripe Arrow allocation peaks were 16.125 and 64.5 MiB; streaming peaked at 16.25 KiB for both sizes. NONE/ZLIB range tests verify that consuming the first batch and closing early does not read unselected stripe data.

Changed-file format/lint checks and git diff --check passed.

Are there any user-facing changes?

Yes. The new API yields RecordBatch objects. stripes=None reads all stripes; explicit sequences preserve order and duplicates, and empty sequences yield no batches. columns=[] preserves row counts. Closing the iterator releases its reader without closing the input; returned batches remain valid.

batch_size limits output rows, not total process memory. ORC buffers and dictionaries require additional memory, and small batches can reduce throughput. Reading the same stripes does not inherently reduce I/O.

Was AI used for this PR?

PR code and description written by:

• Human
• AI

Reviewed before submission by:

• Human
• AI
• Not reviewed

The human contributor directed the design and reviewed the proposal and PR text. OpenAI Codex assisted with code, tests, documentation and static review.

• GitHub Issue: [C++][Python][ORC] Support batch iteration over selected stripes #54921

[github-actions]

⚠️ GitHub issue #54921 has been automatically assigned in GitHub to PR creator.