IO hooks PoC by bukka · Pull Request #23997 · php/php-src

bukka · 2026-09-29T17:05:48Z

This is a proof of concept for IO hooks and contains working implementation for Polling API additions, IO Hooks and IO Ring API as defined in the following RFC's:

Polling API additions: https://wiki.php.net/rfc/poll_api_additions
IO Hooks: https://wiki.php.net/rfc/io_hooks
IO Ring: https://wiki.php.net/rfc/io_ring

It was important to design and implement all 3 together to make sure all need functionality is covered. The detailed living design can be seen at https://gist.github.com/bukka/87359261a4bfaa572ce43c93c0554b55 (it is updated at date of creation of this PR but as this is draft it might get slightly out of sync but the implementation should always cover what is in the actual RFC's).

It should be noted that this also depends on #22538 that is pre-requisite for this and integrated to this PoC. Once merged, this will get rebased.

This is not supposed to be reviewes as it is and it will stay as a draft. Instead smaller PR's will be introduced for the actual review. This is more as a reference and CI check.

Introduce function stream_set_hook(callable $hook). The given hook is called just before performing a read or write operation on any stream, and must have the following signature: function (/*resource*/ $fd, StreamOperation $operation).

Closing a stream resource frees the stream itself, so doing that in a hook will result in UAFs in some parent function. Possible solutions: * Deny fclose() during hook invokation * Never access streams after stream operations, or use the resource to check whether the stream was closed * Do not free the php_stream itself in fclose(). Replace ->ops with always-fail handlers, mark as eof.

The stream hook is now a php_pollfd_for() replacement. Stream ops typically call php_pollfd_for() before any blocking operations, to implement timeouts. We can hook here to delegate polling to user-space. TODO: * php_pollfd_for() is not called where there is no timeout. Ensure that we call it when a hook is installed. * Prevent concurrent stream ops (lock / serialize) * Timeouts should be handled by the hook

Rationale: * We can't ensure consistency of internal structures when a stream is accessed concurrency, at least for the same direction. * It may be possible to accept concurrent accesses from different directions (read and write) * Valid use-cases for concurrent accesses in the same direction are unknown. This would require synchronization from the concumer.

* Introduce StreamPollWeakHandle, which holds its stream weakly * Context stops watching any Handle whose stream is collected * Introduce Context::onWatcherRemoved(?callable $callback). Callback is invoked when a weak handle is removed automatically. * Generalized to Curl's SocketHandle -> SocketWeakHandle

* Mark OS sockets as non-blocking when creating a stream (without affecting the stream's blocking status) * Perform I/O optimistically before polling * Poll on EAGAIN only This should be faster (less polling), and makes operations compatible with edge-triggering.

The op layer carries absolute monotonic deadlines. The header is copied verbatim from php#22538 so the file merges cleanly when that PR lands; the branch does not take its signal dispatch.

Every blocking point is now expressed as a php_io_op executed through php_io_run(): a provider registered with php_io_hooks_register() gets run(), add(), remove() and dtor(), and with no provider the core waits itself on a poll queue (main/hooks/io_queue_poll.c), a Poll context plus a deadline heap that executes Poll, Timer and Any ops and answers readiness for descriptor ops. Timer ops on the synchronous path use nanosleep so the sleep family keeps its resolution. The poll and sleep hooks of the draft are gone; FG(io_hooks) is now an opaque state pointer and FG(io_queue) the core's queue. The concurrent access guard is renamed PHP_STREAM_FLAG_IN_USE and set by the wrappers for the duration of an operation.

xp_socket.c, the accept path in network.c, xp_ssl.c and the sleep family use php_io_poll() and php_io_sleep() with a deadline computed once per stream op. The openssl transport no longer toggles the descriptor's blocking mode and puts accepted sockets in non-blocking mode like xp_socket.c does. curl_exec() waits with one Any op per iteration, a Poll member per socket and a Timer member for libcurl's timeout, keeps its multi handle on the easy handle so connections are reused, and reads the transfer result whenever the transfer ends. Userland gets Io\Operation and its Poll, Timer and Any subclasses, Io\Completion, Io\CompletionStatus, Io\OperationQueue, Io\Poll\OperationQueue over the C poll queue, and Io\Hooks\Hooks with set_hooks(), get_hooks() and is_active().

scheduler.inc is the minimal queue driven provider; the hook tests use run() and completions, and new tests cover the sleep family and a provider that completes Timer and Any operations itself.

The context gets a deadline heap: php_poll_timer_add(), _modify() and _remove() register one-shot or periodic timers, the nearest one bounds php_poll_wait(), and due timers are reported first as events with PHP_POLL_TIMER, with their slots reserved before the backend is called so a level-triggered descriptor cannot starve them. A fired one-shot timer stays registered and disarmed until it is modified or removed. PHP_POLL_PRI maps to POLLPRI and EPOLLPRI; backends declare whether they report it and add() and modify() refuse it with NOSUPPORT where they do not. The poll() backend keeps a fired one-shot registration disarmed instead of dropping it, so modify() re-arms it as on epoll.

…ority Io\Poll\WeakHandle is the type of handles that hold their resource weakly, implemented by StreamPollWeakHandle and the curl socket handle. Io\Poll\TimerHandle is a deadline in the context, one-shot or periodic, watched with Event::Timer and re-armed by modifyEvents(); its watchers live in a table of their own since they have no descriptor. Io\Poll\NotifyHandle is an eventfd, or a pipe where there is none, watched with Event::Notify and raised by notify() or php_poll_notify(). Backend::supportsPriority() says whether Event::Priority is reported. The C poll queue uses the context's timers for Timer ops and op deadlines instead of a heap of its own, and Io\Operation\Timer::getHandle() returns a TimerHandle for the remaining time so a provider on a plain Context can watch it like any other handle.

Covers timers alone and beside descriptors, the poll() backend's disarmed one-shot registrations, notify handles, priority support, the WeakHandle type, and the section 6.3 ContextProvider driving sockets and sleeps through a plain Context with TimerHandle.

… queue php_io_op_persistent() creates a Poll op owned by the core and kept on the handle, the same php_io_op for every run that asks for the same (handle, events) pair, and calls the provider's add hook once; php_io_op_persistent_release() and php_io_handle_release_ops() call the remove hook and free it. The op holds a reference on its handle, and a per-request list lets a provider replacement mark every op unregistered so the next provider sees them as new. An Any takes its members by pointer, since a persistent member is the handle's op. The poll queue keeps one registration per descriptor and multiplexes interest over it: its armed events are the union of the submitted ops' events, re-armed at submit and withdrawn at completion or cancel, and add() and remove() retain and drop it around a persistent op's life. php_poll_handle_invalidate() is the entry point for the resource going away; the stream free path calls it and then releases the handle's ops.

The socket callback only keeps the table: it creates the handle and the entry on the first report, attaches the entry with curl_multi_assign(), records the wanted direction, and on CURL_POLL_REMOVE invalidates the handle inside the callback, before libcurl closes the socket, and moves the entry to a removed list. The close-socket callback is gone. The provider is called from the loop, after curl_multi_socket_action() returned, in a reconcile step: removed entries release their ops, new or changed directions get a persistent Poll op for the new mask. Each iteration waits with an Any of the table's persistent ops plus a Timer member, on the no-hooks path too, so the select() path is gone. An exception a curl callback left pending no longer stops the transfer.

The upload of section 5.12 to a server that reads slower than curl sends, which stalls with an edge-triggered watcher and must complete with level readiness at arm time, and a provider that checks curl's sockets arrive as persistent Poll operations bracketed by add() and remove() with a stable object across runs. The handle leak test warms up once, since the first transfer creates the request's IO state.

php_poll_wait() restarts on EINTR with the remaining time, in one place for the userland Context and the queues: an io_uring in the same process delivers its task work by interrupting a sleeping syscall, so a wait on a Poll context beside a ring sees EINTR without any signal. php_io_poll_notify_handle_create_external() makes an Io\Poll\NotifyHandle over a descriptor someone else owns and clears, which is how a ring's notification descriptor is exposed; notify() is unavailable on it.

* upstream/master: (99 commits) NEWS Fix property hook escape analysis causing misoptimization Fix __isset escape analysis causing misoptimization Fix memory leak when closing a statement on a killed connection Reset field_count for OK packet (php#23890) Fix phpGH-23986: Clear the realpath cache in the child after pcntl_fork() (php#23987) ext/standard: Remove redundant if-branch in rot13 (php#23795) ext/pdo: Throw a ValueError from bindColumn() for an unknown column (php#23835) Zend: rename zend_object* parameter to "this_ptr" for zend_call_* functions (php#23989) ext/pdo: Release driver options after bindParam and bindColumn tests: Raise test stack for stream error depth limit under MSan (php#23985) Zend: Remove zend_atomic.[ch] abstraction (php#23927) fibers: fix phpGH-23921 (Fibers start with error_reporting = 0 when the error_reporting INI directive is not set) ext/pdo: Keep statement class when ATTR_STATEMENT_CLASS is rejected Verify bundled sources using CI - Opcache JIT IR (php#20179) zend_portability: Simplify definition of `ZEND_NORETURN` (php#23908) Fix phpGH-23758: PDO_Firebird returns null for empty BLOBs (php#23763) Remove redundant parentheses in session tests (php#23609) Update IR (php#23861) Fix OSS-Fuzz #552682112: assertion failure wrt zp_arg_must_be_sent_by_ref() (php#23760) ... # Conflicts: # ext/openssl/xp_ssl.c

…y the core, portable signal tests The poll queue's fire read the descriptor record after completing its last request, which drops a record no registration retains (ASAN, the JIT run of run-tests.php itself). The edge interest is read before the requests complete. HAVE_SIGTIMEDWAIT came from pcntl's config.m4 only, so a build without pcntl took the sigwait() fallbacks in the core and left the kqueue pending scan unused under -Werror; configure checks it now and the scan is kqueue's only. x32: the sigtimedwait deadline compared a 32-bit value with the 64-bit bound. macOS: fdatasync() is a symbol without a declaration, as plain_wrapper.c has it. FreeBSD: the tests carried Linux's errno and signal numbers (PCNTL_EAGAIN and the names now), and the signal handle test waited with a zero timeout for a kick that FreeBSD delivers from a kernel task; it waits bounded.

…s, 32-bit and timing in tests The ring read the request a cancel's own cqe names, which the target's completion may have freed already (ASAN, the orphan and destroy tests); the tag is checked before the record is touched. FreeBSD drops NOTE_TRIGGER given together with the EV_ADD of a new user note, so the kick that announces a pending signal never fired there; the note is added and triggered in two kevents. 32-bit: no pid exceeds an int, and hrtime() is a float. The proc_open wait test bounds the overlapped waits by one child's own time, since an ASAN build starts php slowly.

TimWolla

I'm aware this is not intended for review, but I had taken a look anyway and have some general remarks (one about the Time\Duration API that is for me to solve). Generally: If you feel that something is missing from that API to be conveniently used, please let me know so that we can solve that 😃

TimWolla · 2026-09-29T19:12:50Z

+static void php_io_poll_ns_to_duration(zval *rv, zend_hrtime_t ns)
+{
+	zval arg;
+	ZVAL_LONG(&arg, (zend_long) MIN(ns, (zend_hrtime_t) ZEND_LONG_MAX));
+	zend_call_method_with_1_params(NULL, php_date_ce_time_duration, NULL, "fromnanoseconds", rv, &arg);
+}


Note: We should export create_duration() instead of forcing other users to go through the userland API.

Yeah that would be useful.

* upstream/master: ext/bcmath: Clear the sign of BcMath\Number results that truncate to zero Use C11 atomics for epoll_pwait2_available Fix phpGH-23980: ZEND_ASSERT violation @ ZEND_INCLUDE_OR_EVAL (include/eval run with a pending exception) Fix phpGH-23842: skipLazyInitialization() copies unresolved constant defaults Fall back to epoll_wait when epoll_pwait2 is unavailable at runtime (php#23825) Fall back to epoll_wait when epoll_pwait2 is unavailable at runtime (php#23825) openssl: Fix memory leak by doing early salt validation ext/libxml: Keep SimpleXML children alive across reconstruction Fix phpGH-23741: pdo_dblib use-after-free of statement error state # Conflicts: # main/poll/poll_backend_epoll.c

The classes with a constructor are final, not serializable and refuse clone, and reflection refuses newInstanceWithoutConstructor() for a final internal class with its own create_object, so no method can see an object the constructor did not finish.

devnexen · 2026-09-30T04:50:17Z

+	if (!o->active) {
+		return recv(sock->bsd_socket, buf, len, flags);
+	}
+	php_deadline dl = php_socket_op_deadline(sock, SO_RCVTIMEO, flags);


question: is it possible to optimise this case, caching the result ?

The mask followed the object: the constructor blocked the set and the cleanup unblocked it, discarding pending signals nobody handled with a SIG_IGN round trip. It now follows the registrations, through added and removed callbacks in php_poll_handle_ops: the first context that watches the handle blocks its signals and the last removal unblocks them, a context destroyed with the handle in it counting as a removal. Before the unblock the signals it unblocks are taken into the handle's record, so a delivery that arrived while watched is kept for getDelivered() and nothing is discarded. A handle in no context does nothing.

ior now reports a WAITPID child with WNOWAIT and leaves collecting it to the caller, so the process handle does the same: its fired callback reads the exit through php_poll_process_exit_probe(), waitid(WEXITED | WNOHANG | WNOWAIT) with the waitpid() status rebuilt from siginfo, and treats any other report as still running, since macOS's waitid() reports a stopped child whatever the options ask. Nothing collects a child behind its creator's back any more, so the registry of collected children goes: FG(io_reaped), its lookups in php_io_waitpid() and in pcntl_waitpid() and pcntl_wait() with $rusage, the pgid kept by the handle and the ring, and the pid part of php_io_child_forget(). php_io_waitpid() collects a child the ring reported with waitpid(WNOHANG) for that pid and resubmits when another waiter took it first; an orphaned WaitPid leaves the child waitable.

* upstream/master: (34 commits) Remove the mysqlnd_reverse_api feature (php#21409) ext/intl: Fix grapheme_strstr() and grapheme_stristr() match offsets after supplementary characters (php#24012) pdo: refactor pdo_stmt_construct() to use zend_object* rather than zval* (php#24019) Zend: Skip frameless registration for temporary modules ext/curl/config.m4: include string.h for test using strncmp() ext/soap/tests/bugs/bug62900.phpt: make use of TEST_PHP_ vars sapi/cli/tests/php_cli_server_ipv6_error_message.phpt: use TEST_PHP_ vars Zend/tests: organize some tests with sub directories (11) (php#22731) zend_hrtime: remove zend_hrtime_posix_clock_id (php#24016) Zend/lazy_objects: add const qualifiers Zend: use zend_get_gc_buffer_add_fcc for lazy objects Zend: use uint32_t type for lazy_properties_count reflection: mark reflection_class_new_lazy as static always inline reflection: use ZEND_FCC_INITIALIZED macro for lazy objects Zend/lazy_objects: drop variable which is used once Zend: use zend_object* parameter type instead of zval* for closure API (php#24015) Fix HashTable UAF when rebound from a parameter __toString() Fix phpGH-22051: report errors from SQLite3Result reset and finalize Update IR (php#24013) Fix array_map optimization with non-literal function or non-literal args (php#23254) ...

The child exited after sleep 0.1 while the timed signal wait ran out after 100 ms, so on a slow runner the two lines swapped. The child now exits well after the timeout.

Bytes in the stream buffer are committed, bounced ones mark the stream so later reads fail. A provider's cancel() keeps the stream frozen like orphan().

With an exception pending the core runs nothing more for the op and a finished read leaves its bytes to the stream. The userland bridge keeps a queue's completion across an exception.

Orphans hold a stream's resource only when it has one and also cover a handle's descriptor, so a Socket stays busy until its op settled. Stream free refuses a busy stream unless called from the resource destructor. A connect retry drops the previous socket's registrations. No provider run starts with an exception pending. A readiness Any with an Unsupported member runs synchronously, and the ring refuses priority polls. Accept takes fd 0, provider_data is reset on re-add, and more Windows errno values are mapped.

mysqli_close(), mysqli_stmt_close() and a reconnect would free the connection under the parked call; they throw while its stream is busy.

arnaud-lb and others added 30 commits September 23, 2026 19:10

Stream hook

44ee92b

Introduce function stream_set_hook(callable $hook). The given hook is called just before performing a read or write operation on any stream, and must have the following signature: function (/*resource*/ $fd, StreamOperation $operation).

stream_socket_accept()

7c3be3b

Prevent closing stream during hook invokation

05d9e50

Cleanup

46212d4

Prepare API for curl: Watch multiple handlers at once

da9868a

Timeout support and simplify poll() again

061aa6c

curl_exec() support

9ae88c2

Use zend_enum_get_case_by_id

c3845ce

Update test scheduler to use EdgeTriggered mode

764bb3c

Rename poll_multi() -> pollMulti()

d2b7fe6

Hooks::sleep()

b50d15a

Internal API

789a484

Update pollMulti() signature

6f8e015

Adapt test scheduler to Context::wait() taking Time\Duration

dba9496

Borrow php_deadline.h and ZEND_HRTIME_T_MAX from the EINTR PR

d3c11f7

The op layer carries absolute monotonic deadlines. The header is copied verbatim from php#22538 so the file merges cleanly when that PR lands; the branch does not take its signal dispatch.

Adapt the hook tests to the operation model

d6e1870

scheduler.inc is the minimal queue driven provider; the hook tests use run() and completions, and new tests cover the sleep family and a provider that completes Timer and Any operations itself.

github-actions Bot added Extension: standard Extension: xmlreader Extension: xmlwriter Category: CI ABI break labels Sep 29, 2026

bukka added STF: Streams Category: IO Category: Streams Category: Filesystem Category: DNS labels Sep 29, 2026

bukka added 5 commits September 29, 2026 19:31

Follow master's removal of zend_atomic: atomic_load() on vm_interrupt

5e13f55

static_assert() where master dropped ZEND_STATIC_ASSERT

0ef0e5a

TimWolla reviewed Sep 29, 2026

View reviewed changes

bukka added 3 commits September 29, 2026 22:10

Address feedback about ZEND_THIS, named_struct and more

31f8eca

devnexen reviewed Sep 30, 2026

View reviewed changes

bukka added 8 commits September 30, 2026 18:27

Fix the output order race in process-ops-context.phpt

8a6cf8f

The child exited after sleep 0.1 while the timed signal wait ran out after 100 ms, so on a slow runner the two lines swapped. The child now exits well after the timeout.

Keep the bytes of a cancelled or orphaned ring read

f258640

Bytes in the stream buffer are committed, bounced ones mark the stream so later reads fail. A provider's cancel() keeps the stream frozen like orphan().

Keep the bytes of a read cancelled after its completion

d620a52

With an exception pending the core runs nothing more for the op and a finished read leaves its bytes to the stream. The userland bridge keeps a queue's completion across an exception.

Refuse closing a MySQL connection another fiber is suspended in

b05a2fd

mysqli_close(), mysqli_stmt_close() and a reconnect would free the connection under the parked call; they throw while its stream is busy.

github-actions Bot added Extension: mysqli Extension: mysqlnd labels Sep 30, 2026

Navigation Menu

IO hooks PoC - #23997

IO hooks PoC#23997
bukka wants to merge 173 commits into
php:masterfrom
bukka:io_hooks_poc

bukka commented Sep 29, 2026

Uh oh!

TimWolla left a comment

Uh oh!

TimWolla Sep 29, 2026

Uh oh!

bukka Sep 29, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

devnexen Sep 30, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

bukka commented Sep 29, 2026

Uh oh!

TimWolla left a comment

Choose a reason for hiding this comment

Uh oh!

TimWolla Sep 29, 2026

Choose a reason for hiding this comment

Uh oh!

bukka Sep 29, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

devnexen Sep 30, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants