pipe_wait's sched_yield spin storms the scheduler for the full 0.5-3ms of
each in-flight expert read; behind COLI_PIPE_BLOCK=1 it parks on a condvar
instead (~5us wake, no lost-wakeup: workers store ready with release
BEFORE taking the mutex to broadcast, and the waiter re-checks under the
lock). Default OFF = byte-identical spin. Pthread pool only: the URING
backend has no waiter spin to replace.
Setting PIPE_WORKERS>0 in the env without PIPE now implies PIPE=1 with a
stderr note: sizing the pool declares the intent to use it (a full
benchmark campaign ran with PIPE_WORKERS=16 and the pipe silently off).
The implication fires only when the platform default left the pipe off
(no-op on _WIN32 where PIPE defaults to 1), only on a positive value
(PIPE_WORKERS=0/empty does not enable a clamped 1-worker pipe), and an
explicit PIPE=0 always wins. The rule lives in pipe_workers_imply_pipe()
so the table is unit-testable.
tests/test_pipe_block.c (in TEST_BINS, all platforms): pins the
implication table, and drives the pool through 200 generations under each
waiter against an on-disk expert fixture — spin as control, condvar arm
alternating parked and fast-path waits — verifying identical slot bytes.
Measured on the spin side (2x5090 + Gen5 NVMe, GLM-5.2 744B int4, CPU
decode): 1.98 -> 2.16 tok/s at 192 tokens (expert-disk service 44.6s ->
33.5s). On Metal/GPU decode an M5 Pro A/B (PR thread) measured no change,
consistent with the mechanism: the win is freeing the core the spinner
was stealing from the CPU matmul team.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
ebootheee
dc196633f1