Skip to content

Commit 2fcfe59

Browse files
zhidao suhtejun
authored andcommitted
sched_ext: Use WRITE_ONCE() for the write side of scx_enable helper pointer
scx_enable() uses double-checked locking to lazily initialize a static kthread_worker pointer. The fast path reads helper locklessly: if (!READ_ONCE(helper)) { // lockless read -- no helper_mutex The write side initializes helper under helper_mutex, but previously used a plain assignment: helper = kthread_run_worker(0, "scx_enable_helper"); ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ plain write -- KCSAN data race with READ_ONCE() above Since READ_ONCE() on the fast path and the plain write on the initialization path access the same variable without a common lock, they constitute a data race. KCSAN requires that all sides of a lock-free access use READ_ONCE()/WRITE_ONCE() consistently. Use a temporary variable to stage the result of kthread_run_worker(), and only WRITE_ONCE() into helper after confirming the pointer is valid. This avoids a window where a concurrent caller on the fast path could observe an ERR pointer via READ_ONCE(helper) before the error check completes. Fixes: b06ccba ("sched_ext: Fix starvation of scx_enable() under fair-class saturation") Signed-off-by: zhidao su <suzhidao@xiaomi.com> Acked-by: Andrea Righi <arighi@nvidia.com> Signed-off-by: Tejun Heo <tj@kernel.org>
1 parent 57ccf5c commit 2fcfe59

1 file changed

Lines changed: 5 additions & 4 deletions

File tree

kernel/sched/ext.c

Lines changed: 5 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -5258,13 +5258,14 @@ static int scx_enable(struct sched_ext_ops *ops, struct bpf_link *link)
52585258
if (!READ_ONCE(helper)) {
52595259
mutex_lock(&helper_mutex);
52605260
if (!helper) {
5261-
helper = kthread_run_worker(0, "scx_enable_helper");
5262-
if (IS_ERR_OR_NULL(helper)) {
5263-
helper = NULL;
5261+
struct kthread_worker *w =
5262+
kthread_run_worker(0, "scx_enable_helper");
5263+
if (IS_ERR_OR_NULL(w)) {
52645264
mutex_unlock(&helper_mutex);
52655265
return -ENOMEM;
52665266
}
5267-
sched_set_fifo(helper->task);
5267+
sched_set_fifo(w->task);
5268+
WRITE_ONCE(helper, w);
52685269
}
52695270
mutex_unlock(&helper_mutex);
52705271
}

0 commit comments

Comments
 (0)