url/colibri
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

pipe: blocking pipe_wait (COLI_PIPE_BLOCK=1) + PIPE_WORKERS implies PIPE=1

Browse Source 
pipe_wait's sched_yield spin storms the scheduler for the full 0.5-3ms of
each in-flight expert read; behind COLI_PIPE_BLOCK=1 it parks on a condvar
instead (~5us wake, no lost-wakeup: workers store ready with release
BEFORE taking the mutex to broadcast, and the waiter re-checks under the
lock). Default OFF = byte-identical spin. Pthread pool only: the URING
backend has no waiter spin to replace.

Setting PIPE_WORKERS>0 in the env without PIPE now implies PIPE=1 with a
stderr note: sizing the pool declares the intent to use it (a full
benchmark campaign ran with PIPE_WORKERS=16 and the pipe silently off).
The implication fires only when the platform default left the pipe off
(no-op on _WIN32 where PIPE defaults to 1), only on a positive value
(PIPE_WORKERS=0/empty does not enable a clamped 1-worker pipe), and an
explicit PIPE=0 always wins. The rule lives in pipe_workers_imply_pipe()
so the table is unit-testable.

tests/test_pipe_block.c (in TEST_BINS, all platforms): pins the
implication table, and drives the pool through 200 generations under each
waiter against an on-disk expert fixture — spin as control, condvar arm
alternating parked and fast-path waits — verifying identical slot bytes.

Measured on the spin side (2x5090 + Gen5 NVMe, GLM-5.2 744B int4, CPU
decode): 1.98 -> 2.16 tok/s at 192 tokens (expert-disk service 44.6s ->
33.5s). On Metal/GPU decode an M5 Pro A/B (PR thread) measured no change,
consistent with the mechanism: the win is freeing the core the spinner
was stealing from the CPU matmul team.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
ebootheee
2026-07-16 23:20:16 -06:00
parent d45cfa268a
commit dc196633f1
3 changed files with 212 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files
c/Makefile
+4 -1
View File
@@ -166,7 +166,7 @@ else
PYTHON ?= python3
endif
CUDA_OBJ =
TEST_BINS = tests/test_json$(EXE) tests/test_st$(EXE) tests/test_tier$(EXE) tests/test_grammar$(EXE) tests/test_schema_gbnf$(EXE) tests/test_decode_batch$(EXE) tests/test_idot$(EXE) tests/test_i4_grouped$(EXE) tests/test_stops$(EXE) tests/test_kv_alloc$(EXE) tests/test_i4_acc512$(EXE) tests/test_compat_direct$(EXE)
TEST_BINS = tests/test_json$(EXE) tests/test_st$(EXE) tests/test_tier$(EXE) tests/test_grammar$(EXE) tests/test_schema_gbnf$(EXE) tests/test_decode_batch$(EXE) tests/test_idot$(EXE) tests/test_i4_grouped$(EXE) tests/test_stops$(EXE) tests/test_kv_alloc$(EXE) tests/test_i4_acc512$(EXE) tests/test_compat_direct$(EXE) tests/test_pipe_block$(EXE)
ifneq (,$(LINUX))
TEST_BINS += tests/test_uring$(EXE)
endif
@@ -329,6 +329,9 @@ tests/test_compat_direct$(EXE): tests/test_compat_direct.c compat.h
tests/test_uring$(EXE): tests/test_uring.c glm.c st.h uring.h json.h tok.h tok_unicode.h compat.h grammar.h tier.h
$(CC) $(CFLAGS) $< -o $@ $(LDFLAGS)
tests/test_pipe_block$(EXE): tests/test_pipe_block.c glm.c st.h uring.h json.h tok.h tok_unicode.h compat.h grammar.h tier.h
$(CC) $(CFLAGS) $< -o $@ $(LDFLAGS)
test-c: $(TEST_BINS)
$(PYTHON) tools/run_tests.py $(TEST_BINS)