omp: cap LLVM libomp idle spin — a parked engine must not burn 3000% CPU (#341)

The hot-thread tuning sets OMP_WAIT_POLICY=active and caps libgomp's spin
with GOMP_SPINCOUNT=200000. LLVM libomp reads NEITHER: under active policy it
sets KMP_BLOCKTIME=infinite, so once the answer ends and the engine parks on
stdin, the whole OMP team spins forever — ~100% per thread, the 3000% figure
reported on FreeBSD 15.1 in #341 (clang/libomp is the default toolchain
neighborhood there; the same applies to macOS builds).

KMP_BLOCKTIME=200 keeps the team hot for 200 ms after each parallel region —
plenty to bridge back-to-back per-expert matmuls — and then sleeps it.
setenv with overwrite=0, so a user-provided KMP_BLOCKTIME still wins, and
libgomp builds ignore the variable entirely: on GCC toolchains this commit
is a no-op by construction.

Not reproduced locally (this box is gcc/libgomp and idles clean); shipped as
the mechanism-matching candidate fix with a request on the issue for the
reporter to verify on dev and to name their OpenMP runtime (ldd | grep omp).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
JustVugg
2026-07-17 13:53:20 +02:00
parent 617ae51631
commit d5327e2252
+8
View File
@@ -6000,6 +6000,14 @@ int main(int argc, char **argv){
!getenv("COLI_CUDA") && !getenv("COLI_METAL")){
setenv("OMP_WAIT_POLICY","active",0); /* keep the team hot across the tiny per-expert matmul regions */
setenv("GOMP_SPINCOUNT","200000",0); /* spin briefly, then yield so long disk waits don't burn a core */
/* LLVM libomp (clang builds: FreeBSD cc, macOS, some Linux setups) does not
* read GOMP_*: with OMP_WAIT_POLICY=active it sets KMP_BLOCKTIME=infinite,
* so the idle team SPINS FOREVER once generation ends a serve-mode engine
* parked on stdin burns ~100% x nthreads (#341, measured 3000% on FreeBSD).
* 200 ms of blocktime keeps the team hot across back-to-back expert matmuls
* and lets it sleep at the prompt. libgomp ignores KMP_*; overwrite=0 keeps
* the user's own setting authoritative. */
setenv("KMP_BLOCKTIME","200",0);
setenv("OMP_PROC_BIND","close",0); /* pack the team onto adjacent cores for cache locality */
setenv("OMP_DYNAMIC","FALSE",0); /* fixed team size: no per-region thread-count churn */
setenv("COLI_OMP_TUNED","1",1);