[pytorchbot]

What is wrong today

On Linux, the default ExecuTorch thread pool can start one thread per CPU in the machine, even when the process may only use a few, as under taskset or in a container with a CPU set. The extra threads compete for those CPUs and made a small XNNPACK model 1.4x slower.

$ taskset -c 0-3 python -c "from executorch.extension.pybindings import portable_lib as m; print(m._threadpool_get_thread_count())"
168   # before, on an x86 machine with 168 CPUs
4     # after

Why it happens

The pool size comes from get_num_performant_cores() or, in Buck builds, the logical core count. Neither looks at the affinity mask. On x86 the first returns 0 (unknown), so with standard glibc pthreadpool_create(0) counts every online CPU and skips the 63-thread cap. On ARM servers with one core type, it counts every core.

What this change does

On Linux other than Android, the default pool never starts more threads than the CPUs in the affinity mask, and uses that number when the core count is unknown. get_num_performant_cores() applies the same limit, so runners that use it follow the mask too. In CMake builds, x86 Linux hosts with more than 63 allowed CPUs now get 63 threads by default, as ARM servers did, and the FAQ shows how to set the count in C++. Android and other systems keep their old sizing, because Android moves apps between CPU sets after the pool is sized.

The new get_num_allowed_cpus() counts the CPUs in the mask on Linux, even past 1024. The /sys probes now run only on ARM Linux. With an unknown core count, the Gemma 4 runner uses the default pool size, not hardware_concurrency(), still capped at 8. Reading ARM core IDs no longer throws, as it did on AmpereOne.

Tests

• A new test limits a process to one CPU before the pool exists, and checks for one thread and, where known, one performance core. It runs on Linux other than Android and fails without this change.
• ParallelAdd and ParallelReduce now pass when only one CPU is allowed.
• Ran the threadpool tests on Linux x86_64 and aarch64 (ASAN and UBSAN on aarch64), and compiled the sources for macOS and iOS.
• Pool size with taskset -c 0-3: 168 to 4 on x86, 14 to 4 on aarch64.
• On a busy machine, that model took 70 instead of 98 ms with 4 CPUs. With no limit, an LLM-style prefill took 94 ms with 63 threads and 109 with 168, or 98 and 102 ms when less busy. No idle machine with over 63 CPUs was free.

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/24449

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

✅ You can merge normally! (1 Unrelated Failure)

As of commit f3803b8 with merge base 4db778e ():

BROKEN TRUNK - The following job failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

• pull / nxp-mcuxpresso-test / linux-job (gh) (trunk failure)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1

This comment was automatically generated by Dr. CI and updates every 15 minutes.