Skip to content

Commit cf549ed

Browse files
Walk VM: keep precise scalars out of the loop state so the callbacks converge
From 7.2.7 the verifier simulates every one of the VM's nested bpf_loop iterations in full, 8 x 16 x 8 of them at ~830 instructions each, and hits the 1M limit. A callback loop converges only when its entry state matches an earlier one, and a precise scalar in that state blocks it. Two leaks: `step` stored the probe-read size into st->wrote, and a size argument is always marked precise; and walk_node's `wrote == 0` test is predictable on a range the verifier knows excludes zero, which marks the operand precise back through the step loop's entry. Keep the count as a separate opaque copy of arg, never the register that went to the helper, and test it after a round trip through the event, where a load from ring buffer memory is an unknown the verifier cannot predict a branch on.
1 parent 9a69758 commit cf549ed

1 file changed

Lines changed: 18 additions & 2 deletions

File tree

‎bpf/walk/walk.bpf.c‎

Lines changed: 18 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -240,13 +240,22 @@ static long step(__u32 i, void *pst)
240240
else
241241
cur = next;
242242
} else if (code == WOP_READ) {
243+
/* The count kept in `wrote` is a second copy of arg, opaque to
244+
* clang so it is never the register handed to the helper as a
245+
* size: the verifier marks a size argument precise, and a
246+
* precise scalar left in the state blocks the callback-loop
247+
* convergence that keeps this VM verifiable (7.2.7's bpf_loop
248+
* precision fixes made that bite: 850k instructions in `step`
249+
* alone against 14k for the whole object before them). */
250+
__u32 w = arg;
251+
barrier_var(w);
243252
__u32 sz = arg & (ENTRY_CAP - 1);
244253
if (sz == 0)
245254
sz = 1;
246255
if (bpf_probe_read_kernel(st->out, sz, (void *) cur))
247256
st->live = 0;
248257
else
249-
st->wrote = sz;
258+
st->wrote = w & (ENTRY_CAP - 1);
250259
} else if (code == WOP_STR) {
251260
long r = bpf_probe_read_kernel_str(st->out, ENTRY_CAP, (void *) cur);
252261
if (r < 0)
@@ -294,9 +303,16 @@ static long walk_node(__u32 it, void *pst)
294303
__u32 idx = ENTRY_IDX(st->f, it);
295304
__u8 *out = &st->e->edata[idx * ENTRY_CAP];
296305
run_section(st, p->body, p->n_body, st->node, out);
297-
if (!st->live || st->wrote == 0)
306+
if (!st->live)
298307
return 1;
308+
/* Test the count after a round trip through the event, not from
309+
* the state: on a range the verifier can see excludes zero this
310+
* branch is predictable, and a predictable branch marks its
311+
* operand precise all the way back into the step loop's entry
312+
* state. A load from ring buffer memory is unknown, so it is not. */
299313
st->e->elen[idx] = st->wrote;
314+
if (*(volatile __u32 *) &st->e->elen[idx] == 0)
315+
return 1;
300316
st->e->fcount[st->f & (MAX_FIELDS - 1)] = it + 1;
301317
st->e->ok |= (1u << (st->f & (MAX_FIELDS - 1)));
302318
if (p->next_off == 0)

0 commit comments

Comments
 (0)