Skip to content

aio: sleep as iowait while io_getevents() waits for completions - #2483

Open
satchlj wants to merge 1 commit into
integration/aurorafrom
satchlj/aio-iowait-20261119
Open

satchlj wants to merge 1 commit into
integration/aurorafrom
satchlj/aio-iowait-20261119

Conversation

@satchlj

@satchlj satchlj commented Nov 19, 2026 •

Copy link
Copy Markdown
Member

Use io_schedule() in io_getevents() only when the shared AIO context has a request in flight. The counter tracks requests until completion is posted or submission fails. A thread parked on an idle context uses ordinary schedule().

Evidence

Earlier J775c baseline on the default governor, 4 KiB O_DIRECT random reads at QD1, fio pinned to an S core.

ioengine IOPS Mean latency
libaio 11,146 87.4 µs
pvsync2 16,553 60.2 µs
posixaio 16,103 61.9 µs
libaio with the cluster held at top clock 16,885 59.0 µs

A later kernel with several storage-path changes measured libaio QD1 at 52.6–52.8 µs, against 56.1 µs on macOS. The isolated effect of this change remains unmeasured.

Testing

  • ARM64 fs/aio.o built with W=1.
  • The revised head is untested on hardware. Compare pinned libaio QD1 against the same base, and check iowait accounting for an idle io_getevents() loop.

Assisted-by: LLM (OpenAI:GPT-6.2 Sol)

@RyanTheTide

Copy link
Copy Markdown
Contributor

Blockers: 2.

  1. The commit message carries Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>. Drop it and keep only the human Signed-off-by. The PR body should end with Assisted-by: LLM (Claude:Opus 5.5), not the "Generated with Claude Code" footer.
  2. fs/aio.c:1420 sets in_iowait on every sleep in read_events(), whether or not the context has requests in flight. That makes iowait accounting report a thread parked in io_getevents() on an idle context (a common event-loop pattern with long timeouts) as waiting on I/O. It also gives that CPU the boost on every wake, timer or not. io_uring, which the comment cites, had the same problem and gates it: it sets in_iowait only when the task has I/O pending (current_pending_io()). Gate this the same way on in-flight requests for this kioctx. An approximate count is enough for a boost hint, for example ctx->nr_events - 1 - atomic_read(&ctx->reqs_available) minus the per-cpu slack. Alternatively, show why the unconditional form is acceptable. The comment's "waiting for I/O it submitted" is also not guaranteed: a kioctx is shared by all threads of the process, so the waiter may have submitted nothing.

There is no measurement of this commit on its own yet. The table is the pre-change baseline, so the gain is still unproven.

HW row: J775c on integration/aurora plus this commit only, against the same tip without it. fio 4 KiB O_DIRECT randread QD1, libaio, pinned to an S core: IOPS and mean latency. Also /proc/stat iowait for an idle io_getevents() loop with no I/O queued. 0 AVC, 0 failed units, 0 new kernel errors.

Assisted-by: LLM (Claude:Opus 5.5)

A task waiting in io_getevents() for AIO completions sleeps in plain
schedule(), so it never wakes with in_iowait set and schedutil's iowait
boost never applies to it. Synchronous direct I/O waits in io_schedule()
and io_uring sets in_iowait while the waiter has requests in flight
("turns out to be important for low QD IO"), so a libaio loop is the one
common low queue depth path the boost misses.

On a T6050 (M5 Max) the difference is large because an idle cluster
drops to 300 MHz, and a task that sleeps most of the time keeps the
utilization of its CPU too low for schedutil to raise the clock. 4 KiB
O_DIRECT random reads at queue depth 1 (fio 3.43, ext4 on the internal
ANS SSD), the fio job pinned to an S core at the default governor:

  ioengine         IOPS    mean latency
  libaio          11,146   87.4 us
  pvsync2         16,553   60.2 us   (io_schedule(): boosted)
  posixaio        16,103   61.9 us   (glibc helpers in pread(): boosted)
  libaio, cluster
  at top clock    16,885   59.0 us

Use io_schedule() in read_events() when the context has a request in
flight. Count accepted request slots until completion is posted or
submission fails. This also covers I/O submitted by another thread
sharing the context. An idle context uses schedule() so a parked event
loop incurs no iowait accounting or iowait boost.

Signed-off-by: Satya Benson <satya@benson.earth>
@satchlj
satchlj force-pushed the satchlj/aio-iowait-20261119 branch from f9ce6eb to d7e25fe Compare November 21, 2026 12:33
@satchlj

satchlj commented Nov 21, 2026

Copy link
Copy Markdown
Member Author

Codex GPT-6.2 Sol on @satchlj's behalf

Removed the co-author trailer and updated the PR footer. read_events() now uses io_schedule() only when the kioctx has an outstanding request. The count is decremented on completion or failed submission. An idle event loop uses schedule(), and the comment now accounts for requests submitted by another thread sharing the context.

ARM64 fs/aio.o built with W=1. The revised head is untested on hardware. The isolated QD1 and idle-loop iowait comparisons and regression checks are pending.

@satchlj

satchlj commented Nov 21, 2026 •

Copy link
Copy Markdown
Member Author

Codex GPT-6.2 Sol on @satchlj's behalf

J775c A/B on exact integration parent 26294e82e9c8 and exact #2483 head d7e25fedf0e4. Two 27.0 Aura one-shots used the same carrier DTB and byte-identical trial config with PCI SDHCI disabled. The only source delta was fs/aio.c.

fio 3.43 ran 4 KiB O_DIRECT randread, libaio QD1, one job on live Everest CPU13, with the same fully allocated 16 GiB ext4 file and seed. Three independent 20 s runs per boot with 3 s ramp. Both arms measured 100% at depth 1 and zero fio errors.

Metric Exact parent PR head
Median IOPS (three-run range) 13,378 (13,339–13,385) 14,997 (14,983–15,002)
Median mean end-to-end latency 74.095 µs 66.484 µs
Median mean completion latency 65.694 µs 63.993 µs

Observed +12.10% IOPS and −10.27% mean end-to-end latency. Starting SoC die readings were 44.1–45.1 °C on parent and 41.5–41.6 °C on head. The parent ran first and there was one boot per arm.

The empty-context control submitted no I/O. Each of three invocations per kernel returned 100 empty timeouts from io_getevents() (100 ms timeout). CPU13 /proc/stat iowait delta was 0/0/0 on both kernels, also 0/0/0 for the three pinned sleep controls. In the fio windows, including ramp, it was 0/0/0 ticks on the parent and 2155/2152/2146 on the head.

Both boots had 0 AVC denials, the same one failed unit (ac-power-mode.service), and the same five boot errors (Gen4 SPMI, AOP and DCP). The head added 0 failed units and 0 boot errors versus its parent. Both warm handbacks verified default Aura, both ESPs offline, healthy root and Thunderbolt peer connectivity.

Scope follow-up: the exact A/B base 26294e82e9c87a0162a7973c5e23c599ceaea22f was 370 commits behind integration/aurora a84637dca209 on Nov 21. This change merges cleanly in a changed-file text check against the newer tip. That combined tree is untested on hardware here. Both arms used the carrier DTB and the same trial differences from installed Aura: PCI SDHCI off, arm64.nomte, console=tty0, and console=ttyDC0.

@RyanTheTide

Copy link
Copy Markdown
Contributor

Re-review of d7e25fe: Blockers: 1.

From our last comment:

  • Co-Authored-By trailer: resolved, but see 1 below.
  • Unconditional iowait: resolved. read_events() uses io_schedule() only while reqs_in_flight is non-zero (fs/aio.c:1423). The count is balanced: it is incremented once per slot in aio_get_req() (:1125) and decremented on the posted completion in aio_complete() (:1214) or on the submit error path that destroys the request (:2132). Cancel and poll requests also end in aio_complete(). The comment now covers requests submitted by another thread.
  • Isolated measurement: resolved. The J775c A/B on 26294e8, with only fs/aio.c differing, gives +12.1% IOPS and -10.3% mean latency at libaio QD1. The idle io_getevents() loop shows 0 iowait ticks on both kernels, and failed units and boot errors match the parent.
  1. The commit message now ends with Assisted-by: LLM (OpenAI:GPT-6.2 Sol). Keep that line in the PR body only; the commit should carry just the human Signed-off-by.

Nit: the commit message's table and the PR body are still the pre-change baseline, and the body says the change is unmeasured. Replace them with the parent-vs-head A/B and the idle-loop iowait result.

Assisted-by: LLM (Claude:Opus 5.5)

@DjDeveloperr DjDeveloperr left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pre-PR checklist review of d7e25fedf0e422cb2f4fd57faa0a6c80a14ac1f0 (§1–§6): 0 code/policy/attribution/naming/CI blockers; holding on default-cmdline §5 proof and PR-body update.

Code Audit (fs/aio.c)

  • Counter balance verified across all paths: ctx->reqs_in_flight is initialized to 0 in ioctx_alloc() (:853), incremented once per accepted slot in aio_get_req() (:1125, called only from io_submit_one() :2117), and decremented once either in aio_complete() (:1214, reached via iocb_put() for sync/async read/write, fsync, poll, and cancel paths) or on the io_submit_one() non-zero err destruction path (:2132). Every __io_submit_one() opcode (PREAD/PWRITE/PREADV/PWRITEV, FSYNC/FDSYNC, POLL, and invalid opcodes) is balanced.
  • Wait loop & wake ordering (read_events(), :1405-1428): prepare_to_wait_event() sets TASK_INTERRUPTIBLE before aio_read_events(), and when nr_got == ret, io_schedule() is used only if atomic_read(&ctx->reqs_in_flight) is non-zero (schedule() otherwise). aio_complete() orders atomic_dec() -> smp_wmb() -> ring->tail -> smp_mb() -> wake_up_process(), so there is no missed-wakeup window.
  • Non-blocking observations:
    • reqs_in_flight shares a cacheline with reqs_available (fs/aio.c:144-148), which aio_get_req() otherwise avoids touching per-I/O via per-CPU batching. On high-QD multi-core AIO this adds a shared-cacheline atomic bounce per submit/complete (acceptable for correctness since aio_complete() already takes completion_lock, though worth keeping in mind).
    • IOCB_CMD_POLL also increments reqs_in_flight, matching io_uring's current_pending_io() trade-off.

Checklist Summary

  • §1 No model gating — PASS: Generic fs/aio.c change, no chip/SoC/board/fw gates.
  • §2 Production-ready, no hacks — PASS (code): No new cmdline/module params, sleeps, or hacks in the change itself. (See §5 below regarding re-proving on the default command line.)
  • §3 Safety — PASS.
  • §4 Clean-room & attribution — PASS: Verified directly on git commit d7e25fedf0e422cb2f4fd57faa0a6c80a14ac1f0: author and committer are Satya Benson <satya@benson.earth> with a single human Signed-off-by: Satya Benson <satya@benson.earth> and no Co-Authored-By: or Assisted-by: trailer in the commit message (contrary to the earlier comment). The PR body ends with Assisted-by:.
  • §5 Evidence — HOLD (needs default-cmdline confirmation & PR body refresh):
    1. The isolated J775c A/B on 26294e82e9c8 vs d7e25fedf0e4 (+12.10% IOPS, -10.27% mean latency, 0/0/0 idle-context iowait ticks, 0 AVC, 0 new failed units, 0 new boot errors) is strong, but both arms used non-default trial boot differences (arm64.nomte, console=tty0, console=ttyDC0, PCI SDHCI disabled, carrier DTB) on a 518-commit-behind base (26294e82e9c8). Per §2/§5 ("Remove them and re-prove on the default command line before the PR" + 0 SELinux denials, 0 failed units, 0 new kernel errors), please run a default-cmdline confirmation row (without arm64.nomte / trial SDHCI-off overrides) and note the regression counts.
    2. The PR body still shows only the pre-change baseline table and states "The isolated effect of this change remains unmeasured" / "The revised head is untested on hardware" — please update it with the actual A/B and idle-loop results.
  • §6 Naming — PASS.
  • CI — PASS: host-tests green on d7e25fedf0e4.

Assisted-by: LLM (Gemini 4 Neon)

@satchlj

satchlj commented Nov 22, 2026 •

Copy link
Copy Markdown
Member Author

Claude Opus 5.5 on @satchlj's behalf.

This is the #2483 / #2754 pairing on lobata (J775c, t6050 max): two A-B-B-A palindromes of one-shot boots, run on 2026-11-22. The same text is on #2483 and #2754.

Builds

Decision rule, written before round 1

Accept the narrowed #2754 if, in both orders:

Round 1: with, narrow, without, without, narrow, with

Arm (boot) QD32 MB/s QD32 W 4 KiB O_DSYNC W O_DSYNC floor
with (1) 12323 31.71 14.22 0x24, 21 of 36 samples
narrow (2) 12114 29.46 12.62 0
without (3) 11237 23.01 12.66 0
without (4) 11109 21.80 12.58 0
narrow (5) 12226 30.05 12.73 0
with (6) 12354 31.90 13.53 0x12, 16 of 36 samples

Narrow is −1.4 % on QD32 against "with" and +0.05 W on O_DSYNC against "without", so it passes the rule. The 4 % threshold, though, was fitted to these two workloads, which is why round 2 was run.

Round 2: held-out workloads, predictions written first

Order: narrow, without, aio-revert, aio-revert, without, narrow.

Arm (boot) QD32 MB/s QD1 4 KiB random read: IOPS / W / floor-raised samples 4 concurrent 4 KiB O_DSYNC writers: W / floor-raised samples
narrow (1) 12203 14678 / 16.15 / 5 of 20 12.69 / 5 of 30
without (2) 11254 13249 / 15.53 / 0 12.55 / 0
aio-revert (3) 11626 13473 / 13.28 / 0 12.57 / 1
aio-revert (4) 11666 12264 / 13.15 / 2 12.53 / 3
without (5) 11360 13951 / 15.41 / 0 12.49 / 2
narrow (6) 12188 14109 / 16.01 / 1 12.83 / 9
  • QD1 random read. Predicted: the floor is raised. Held in part: a few samples. Narrow is +3.5 % IOPS and +0.6 W against "without".
  • Four concurrent writers. Predicted: the floor is not raised. Failed: narrow raised it in 5 and 9 of 30 samples, at +0.24 W.
  • aio-revert. Predicted: QD32 returns to the stock tip's 12.0-12.2 GB/s, measured earlier and not in this palindrome. Failed: it reached 11.63 and 11.67 GB/s, against 11.25 and 11.36 for "without".
  • Also run: a buffered-write held-out, which is invalid because it finished before its sampling window. I also logged a per-window busy distribution. Its 100 ms windows count in 10 ms ticks, too coarse to resolve a 4 % threshold.

What this shows

Raw rows are in lobata's session evidence: claude-narrow2754-* and claude-r2-*.

Assisted-by: LLM (Anthropic:Claude Opus 5.5)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants