tracking: improve Python binding I/O performance

[Xuanwo]

Summary

Several Python binding paths perform avoidable full-payload copies, add an extra storage request, or cross the Tokio-to-asyncio boundary once per result. Blocking APIs also keep the CPython GIL while waiting for storage I/O. These costs affect large-object throughput, async listing, remote-read latency, and Python thread concurrency.

Source references are against 7ca2c0261.

Work items

Fix sized readline

• Fix File.readline(size) and add regression coverage. The current implementation starts with size zero bytes, appends the line, and then truncates using the number of appended bytes. For example, reading b"abc\ndef" with readline(4) returns b"\x00\x00\x00\x00". (implementation, existing coverage)

Reduce payload copies

• Return Python bytes from Operator.read and AsyncOperator.read with one full-payload copy. The current path copies opendal::Buffer into Vec<u8>, creates a temporary Python buffer object, and copies it again into Python bytes. (sync, async, conversion)
• Avoid allocating and zero-initializing the complete requested size before File.read(size). Avoid repeated vector growth for read() when the length is already known. Preserve partial-read and EOF behavior. (sync, async)
• Let immutable Python bytes enter Operator.write and AsyncOperator.write without a binding-owned payload copy. Keep the Python object alive until the storage operation completes; mutable inputs must remain isolated. (sync, async)
• Remove the initial Python-bytes-to-Vec copy in AsyncFile.write. Preserve the existing small-write coalescing behavior so the change does not increase storage requests or file syscalls. (binding, writer buffer)

Reduce async listing overhead

• Avoid one Tokio-to-asyncio round trip per AsyncLister entry. Fetch or prefetch entries in batches and serve buffered entries without another cross-thread event-loop wakeup. Preserve streaming and bounded-memory behavior. (implementation)

Avoid an extra request when opening readers

• Do not require stat/HEAD before a normal sequential open(path, "rb") read. The current file adapter resolves an unbounded range by fetching the object length before reading. Preserve bounded-range reads and seek behavior. (binding, range resolution)

Release the GIL during blocking I/O

• Release the CPython GIL while blocking Operator methods wait for storage I/O. Ensure borrowed Python inputs remain alive and immutable for the complete operation. Extend the same behavior to File and blocking-list operations when their state can be transferred safely.

#[pymodule(gil_used = false)] declares support for free-threaded Python, but it does not release the GIL for blocking calls on regular CPython. The binding currently has no Python::detach or allow_threads path.

Evidence

A release build on CPython 3.14.5 with the GIL enabled used the memory backend to isolate binding overhead. These results indicate where copies and runtime crossings dominate; they do not predict remote-storage throughput.

• Reading an 8 MiB object took 195.4 us with Operator.read and 94.5 us with File.readinto using a preallocated buffer.
• Reading the same 8 MiB object took 233.7 us with AsyncOperator.read and 1,047.3 us through AsyncFile.read.
• Listing 2,000 in-memory entries took 0.613 us per entry synchronously and 40.14 us per entry asynchronously.
• The async list produced 2,002 event-loop wakeups: one for lister creation, one per entry, and one for completion.

Validation

• Add focused benchmarks that keep Operator and event-loop setup outside the measured operation. The current benchmark includes both in every timed run and combines multiple object sizes into one result. (current benchmark)
• Measure payload copies and allocation volume for changed read and write paths.
• Measure runtime crossings and event-loop wakeups for async listing.
• Verify request count against a real HTTP object store for the reader-open change.
• Verify blocking calls allow another Python thread to make progress.
• Run affected tests and benchmarks on regular CPython, the Python 3.11 abi3 wheel, and free-threaded Python 3.14.

Completion criteria

• Whole-object reads create Python bytes with one full-payload copy.
• Immutable Python bytes do not require a binding-owned payload copy before whole-object writes.
• Async listing materially reduces runtime crossings and event-loop wakeups per entry without collecting an unbounded result set.
• Sequential file reads do not add stat/HEAD unless the requested operation needs the object length.
• Blocking storage I/O does not hold the GIL.
• File cursor, partial-read, buffering, close/durability, cancellation, and free-threaded safety behavior remain unchanged.