Repository navigation
Fix #1459: Call check_signals in the scan to support signal handlers - #10101
pamod-madubashana wants to merge 4 commits into
Conversation
|
I think this needs some testing? |
|
Added vortex-python/test/test_iter_signals.py covering all four check_signals sites: next(), read_all() (pending-signal early-abort plus a mid-scan SIGINT timing test), and the to_arrow record-batch reader. Each test queues a real SIGINT against a counting Python iterator and asserts the scan surfaces it promptly without consuming every chunk first. Note I could not run the suite locally (no built extension / flatc in this environment), so CI verification would be appreciated. |
417c9e7 to
31f2719
Compare
|
This PR has been marked as stale because it has been open for 14 days with no activity. Please comment or remove the stale label if you wish to keep it active, otherwise it will be closed in 7 days |
…l handlers Signed-off-by: Pamod Madubashana <contact@pamod.is-a.dev>
Queue a real SIGINT and assert next(), read_all(), and the to_arrow record-batch reader surface it promptly while aborting early, plus a mid-scan interrupt test proving read_all does not run to completion. Signed-off-by: Pamod Madubashana <contact@pamod.is-a.dev>
6e268ac to
08fe8f7
Compare
|
Rebased onto latest |
Fixes #1459
What changed
Honours Python signal handlers (e.g. Ctrl+C /
KeyboardInterrupt) while scanning from Python by callingpy.check_signals()at iteration boundaries invortex-python/src/iter/:PyArrayIterator::__next__now checks signals with the GIL held before and after the detached chunk fetch, so eachnext()raises promptly if a signal arrived.read_allnow pulls chunks one at a time, checking signals before each chunk (plus a final check), instead of running the whole scan inside a singledetachwith no signal checks.to_arrowRecordBatchReaderclosure checks signals viaPython::attachbefore converting each chunk, sopyarrowreaders are interruptible too.PythonArrayIterator::nextnow implements the signal check its comment always promised, mapping aPyErrinto aVortexErrorsince onlyVortexResultis available there.Why
Long scans run inside
py.detach(...), which releases the GIL, so Python signal handlers never run and Ctrl+C appears to hang until the scan finishes. Checking signals wherever we already hold the GIL at chunk boundaries lets the interpreter raiseKeyboardInterruptbetween chunks without changing any scan semantics.Verification
cargo fmt -p vortex-python -- --check: clean (only stable-channel warnings about nightly-only options, no diff).cargo clippy -p vortex-python --all-targets -- -D warnings: could not compile in this environment —vortex-array's build script fails withfailed to run flatc: No such file or directory (os error 2), a pre-existing missing FlatBuffers compiler unrelated to this change. For the same reason no Python test requiring the built extension could be run.