Repository navigation
Commit d8c5201
committed
fix(run-engine): make ck vtime pass 2 discover unregistered variants
At the moment the flag is flipped, :ckVtime is empty and every already-queued
variant is unregistered. The first dequeue has an empty pass 1, so pass 2 serves
and registers up to actualMaxCount variants. From the next call on, pass 1 can
serve one message per registered variant and actualMaxCount is often small, so
pass 1 fills the batch off that cohort alone. Pass 2 was gated on
dequeuedCount < actualMaxCount, so it never ran again, and the rest of the
backlog stayed invisible until a registered variant fully drained or an enqueue
or nack happened to land on it. A key that gets no further work has no other
route into the fair order. Same reachability shape 1a6d1a5 fixed for
registered-but-unservable variants, applied to the unregistered cohort. It also
covers a mixed deploy, where an instance with the flag still off enqueues
through the non-vtime command, and a :ckVtime that expired while ckIndex lived.
Pass 2 now always runs. When the batch is already full it registers the variants
pass 1 could not see, at the floor, instead of serving them, so the next call's
pass 1 leads with them. Serving is still capped at actualMaxCount, so no serve
happens that the old gate would have refused, and the fairness scenarios are
byte-identical: ckSkew, ckTrickle, ckSybil, ckBalanced, ckManyKeys, ckHeavyIdle
and ckStalledNewcomer all report the same numbers as before.
Op cost is one extra fixed op per call. The pass-1 window read doubles as a free
membership set, so nothing is registered twice, and the registrations are
collected into a single variadic ZADD NX rather than one call each. Measured on
the op-count budget test: 11.62 ops per dequeue over the flag-off path, against
10.90 before. The budget comment now counts 8 fixed ops rather than 7.
Devin's suggestion of reserving a batch slot for pass 2 does not fix it. Pass 2
walks ckIndex in age order, so its one reserved slot always lands on a variant
that is already registered and never reaches the cohort that is not. Measured
against the new tests: identical to no fix at all, 12, 16 and 12 calls.
Reported by Devin on #4367.1 parent b932442 commit d8c5201
3 files changed
Lines changed: 178 additions & 12 deletions
File tree
- internal-packages/run-engine/src/run-queue
- tests
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
5164 | 5164 | | |
5165 | 5165 | | |
5166 | 5166 | | |
| 5167 | + | |
| 5168 | + | |
| 5169 | + | |
5167 | 5170 | | |
5168 | 5171 | | |
5169 | 5172 | | |
| |||
5322 | 5325 | | |
5323 | 5326 | | |
5324 | 5327 | | |
| 5328 | + | |
| 5329 | + | |
| 5330 | + | |
| 5331 | + | |
| 5332 | + | |
| 5333 | + | |
| 5334 | + | |
| 5335 | + | |
5325 | 5336 | | |
5326 | 5337 | | |
5327 | 5338 | | |
5328 | 5339 | | |
5329 | 5340 | | |
5330 | 5341 | | |
5331 | | - | |
5332 | | - | |
5333 | | - | |
5334 | | - | |
5335 | | - | |
5336 | | - | |
5337 | | - | |
5338 | | - | |
| 5342 | + | |
| 5343 | + | |
| 5344 | + | |
| 5345 | + | |
| 5346 | + | |
| 5347 | + | |
| 5348 | + | |
| 5349 | + | |
| 5350 | + | |
| 5351 | + | |
| 5352 | + | |
| 5353 | + | |
| 5354 | + | |
| 5355 | + | |
5339 | 5356 | | |
| 5357 | + | |
| 5358 | + | |
| 5359 | + | |
| 5360 | + | |
| 5361 | + | |
| 5362 | + | |
| 5363 | + | |
| 5364 | + | |
5340 | 5365 | | |
5341 | 5366 | | |
5342 | 5367 | | |
| 5368 | + | |
| 5369 | + | |
| 5370 | + | |
5343 | 5371 | | |
5344 | 5372 | | |
5345 | 5373 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1273 | 1273 | | |
1274 | 1274 | | |
1275 | 1275 | | |
| 1276 | + | |
| 1277 | + | |
| 1278 | + | |
| 1279 | + | |
| 1280 | + | |
| 1281 | + | |
| 1282 | + | |
| 1283 | + | |
| 1284 | + | |
| 1285 | + | |
| 1286 | + | |
| 1287 | + | |
| 1288 | + | |
| 1289 | + | |
| 1290 | + | |
| 1291 | + | |
| 1292 | + | |
| 1293 | + | |
| 1294 | + | |
| 1295 | + | |
| 1296 | + | |
| 1297 | + | |
| 1298 | + | |
| 1299 | + | |
| 1300 | + | |
| 1301 | + | |
| 1302 | + | |
| 1303 | + | |
| 1304 | + | |
| 1305 | + | |
| 1306 | + | |
| 1307 | + | |
| 1308 | + | |
| 1309 | + | |
| 1310 | + | |
| 1311 | + | |
| 1312 | + | |
| 1313 | + | |
| 1314 | + | |
| 1315 | + | |
| 1316 | + | |
| 1317 | + | |
| 1318 | + | |
| 1319 | + | |
| 1320 | + | |
| 1321 | + | |
| 1322 | + | |
| 1323 | + | |
| 1324 | + | |
| 1325 | + | |
| 1326 | + | |
| 1327 | + | |
| 1328 | + | |
| 1329 | + | |
| 1330 | + | |
| 1331 | + | |
| 1332 | + | |
| 1333 | + | |
| 1334 | + | |
| 1335 | + | |
| 1336 | + | |
| 1337 | + | |
| 1338 | + | |
| 1339 | + | |
| 1340 | + | |
| 1341 | + | |
| 1342 | + | |
| 1343 | + | |
| 1344 | + | |
| 1345 | + | |
| 1346 | + | |
| 1347 | + | |
| 1348 | + | |
| 1349 | + | |
| 1350 | + | |
| 1351 | + | |
| 1352 | + | |
| 1353 | + | |
| 1354 | + | |
| 1355 | + | |
| 1356 | + | |
| 1357 | + | |
| 1358 | + | |
| 1359 | + | |
| 1360 | + | |
| 1361 | + | |
| 1362 | + | |
| 1363 | + | |
| 1364 | + | |
| 1365 | + | |
| 1366 | + | |
| 1367 | + | |
| 1368 | + | |
| 1369 | + | |
| 1370 | + | |
| 1371 | + | |
| 1372 | + | |
| 1373 | + | |
| 1374 | + | |
| 1375 | + | |
| 1376 | + | |
| 1377 | + | |
| 1378 | + | |
| 1379 | + | |
| 1380 | + | |
| 1381 | + | |
| 1382 | + | |
| 1383 | + | |
| 1384 | + | |
| 1385 | + | |
| 1386 | + | |
| 1387 | + | |
| 1388 | + | |
| 1389 | + | |
| 1390 | + | |
| 1391 | + | |
| 1392 | + | |
| 1393 | + | |
| 1394 | + | |
| 1395 | + | |
| 1396 | + | |
| 1397 | + | |
| 1398 | + | |
| 1399 | + | |
| 1400 | + | |
| 1401 | + | |
| 1402 | + | |
| 1403 | + | |
| 1404 | + | |
| 1405 | + | |
| 1406 | + | |
| 1407 | + | |
| 1408 | + | |
| 1409 | + | |
| 1410 | + | |
| 1411 | + | |
1276 | 1412 | | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
346 | 346 | | |
347 | 347 | | |
348 | 348 | | |
349 | | - | |
350 | | - | |
351 | | - | |
352 | | - | |
| 349 | + | |
| 350 | + | |
| 351 | + | |
| 352 | + | |
| 353 | + | |
| 354 | + | |
353 | 355 | | |
354 | 356 | | |
355 | 357 | | |
| |||
0 commit comments