The gateway's tool-calling path had unit coverage (parse_tool_calls,
render_chat) but nothing exercised the real subprocess wire protocol or the
HTTP surface a coding client actually hits. #401 reports plain-text replies
where tool_calls were expected; every documented path checks out, so pin the
whole path down with a mock engine speaking SUBMIT/DATA/DONE and assert:
- non-stream: tool_calls populated, finish_reason tool_calls, no raw markers
- stream: markers suppressed across 20-way chunk splits, tool_calls delta
- tool-result round trip: <|observation|><tool_response> rendering, text reply
- no tools: plain text untouched
Also emit a stderr diagnosis when tools are declared and tool-call markers
are present in the reply but the strict parse matches nothing (typically
quantization-mangled output) pointing at COLI_TOOL_SALVAGE=1 -- the likely
field condition behind #401.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
fix(test): make test_stops build on Windows (mkdtemp compat shim)
Conflict with #366's getenv_utf8 in compat.h resolved by keeping both blocks.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Both coli's cuda_binary() and doctor.py's cuda_linkage() detect CUDA support
by running `ldd` on the engine binary and looking for a linked libcudart —
but that's Linux-only (cuda_binary() checks sys.platform != "linux", and
cuda_linkage() checks os.name != "posix", both short-circuiting to False on
win32). Windows CUDA_DLL=1 builds never link libcudart at all: glm.exe loads
coli_cuda.dll dynamically via LoadLibrary at startup (backend_loader.c), so
there's no import-table entry for ldd/dumpbin to find in the first place.
The practical effect: `coli doctor` always reported "NVIDIA GPU detected but
the engine is CPU-only" on Windows, and `coli run/chat/serve --gpu ...`
always hard-exited with "--gpu needs the CUDA build" — even on a correctly
built CUDA_DLL=1 binary with coli_cuda.dll sitting right next to glm.exe.
Fix: on win32, detect a COLI_CUDA build by scanning glm.exe for the marker
string "[CUDA] mode: routed experts", which only exists in code compiled
under #ifdef COLI_CUDA (see glm.c's cuda init block), then confirm
coli_cuda.dll actually sits next to the binary — mirroring the Linux
"linked but missing" distinction. Linux/macOS detection is unchanged.
Verified on Windows 11 with mingw-w64 GCC 16.1.0 + CUDA 13.2 + RTX 5080:
`coli doctor` now reports "CUDA engine and devices are available", and
`coli run --gpu 0 --vram 8` populates VRAM (confirmed via the engine's own
"[CUDA] resident set: N tensors, X GB VRAM" runtime log) instead of exiting.
- c/version.py: single source of truth (__version__ = "1.0.0")
- coli: reads version.py, banner shows dynamic version, --version flag
- .github/workflows/release.yml: tag push triggers cross-platform build
(Linux x86_64, macOS ARM64, Windows x86_64) and creates a GitHub
Release with packaged binaries + changelog notes
- CHANGELOG.md: v1.0.0 baseline documenting all shipped features
To cut a release:
1. bump c/version.py
2. add a CHANGELOG section
3. git tag v1.0.0 && git push --tags
Per-row int4 quantization of the MTP head's eh_proj [D,2D] zeroes its ENTIRE
embedding half: the tensor's two column halves differ in scale by ~20-30x per
row (embedding-half absmax ~0.05, hidden-half ~1.5 on GLM-5.2), so a single
per-row scale (absmax/7) puts every embedding-half weight below half a
quantization step and np.rint rounds it to exact zero (packed bytes 0x88).
The draft head then cannot see the input token and MTP acceptance collapses
to ~0% — the mechanism behind issue #8's measurement (int4: 0-4%; int8:
39-59%), and the reason --mtp already defaults to --ebits 8.
This showed up in the wild: a published pre-converted container had its MTP
shards made with an explicit --ebits 4 and drafts were pure garbage on every
backend (deterministic, data-side; verified byte-for-byte by reproducing the
published bytes with quant_int4 on the official BF16 rows).
- tools/repair_mtp_int8.py: repairs such a container IN PLACE — finds the MTP
layer's per-row-int4 dense tensors (eh_proj, q/kv/o projections, shared
experts; routed experts untouched), re-downloads only those from the FP8
source repo (~355 MB of HTTP range reads, no torch, no token), requantizes
at int8 with quant_int8's exact math, and rewrites the shards atomically
with *.bak-int4 backups. --dry-run to inspect. The engine picks up int8
automatically (qt_from_disk detects format by blob size).
Validated on GLM-5.2 744B: MTP acceptance 0% -> 100% (greedy), 1.11 -> 3.20
tokens/forward, decode 0.18 -> 0.30 tok/s (Metal, 4-bit KV).
- convert_fp8_to_int4.py: print a warning when --mtp is combined with
--ebits <8 and per-row scales (the exact footgun above); --group-size 128
remains a valid int4 alternative since group scales give the embedding
half its own scale.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Two gaps in the local path, both raised in #383:
- The main --indir pass copied only config.json while the download path
copies four files; without tokenizer.json the converted container can't
run chat/serve. Copy the same four, print what was copied, and warn
when the source is missing any (tokenizer.json called out explicitly).
- Local passes restarted from shard 0 on every rerun. out-NNNNN names
count EMITTED shards, not input indexes (shards with no relevant
tensors emit nothing), so the download path's exists-check can't be
mirrored directly: a sidecar manifest per prefix records
input -> output (or empty) plus the conversion parameters, written
atomically after each shard. Rerun skips what matches and refuses a
parameter mismatch on the same outdir instead of mixing containers
(the #355 failure mode).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
coli run hardcoded the nothink template (<|assistant|><think></think>),
so THINK=1 had no effect in one-shot runs while serve mode honors it
(glm.c serve template picks <think> vs <think></think> from the same
env var). Pick the template the same way here.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
When --indir contains model.safetensors.index.json, the head/indexer
passes convert only the shards that actually hold the requested tensors
(3 instead of scanning 141 for --mtp — every empty scan still opens a
5 GB shard). Prints the selection in the [PLAN] block. Without the index
the full scan is unchanged.
Output is byte-identical to the unfiltered path (sha256-verified on the
GLM-5.2 FP8 head shards, with a decoy shard proving the filter excludes
rather than reorders).
Co-Authored-By: Claude <noreply@anthropic.com>
Two operator-surprise fixes:
openai_server.py (#382, LordMZTE): a request without max_tokens defaulted to
min(256, limit) — so `coli serve --ngen 32768` still cut every answer at 256
tokens for clients that omit the field. The operator's configured budget IS the
default now; generation still ends at EOS, so it's a cap, not a target.
convert_fp8_to_int4.py (#383, bokiko): the resolved conversion plan (mode,
source, ebits/io/x bits, grouped-vs-per-row) prints as a [PLAN] line before any
work. The two traps in #383 — --mtp defaulting ebits to 8 with the grouped
branch silently gated off, and --mtp appearing to ignore --indir — are already
defused by the #355 fix (the --mtp pass emits ONLY head tensors on the local
path), but a 3.5-hour job must show its plan at second 1, not in a post-hoc
size sanity check. bokiko's exact invocation now prints:
[PLAN] mode: MTP head only | source: local ./fp8 | experts 8-bit, ... |
PER-ROW (grouped branch needs bits<=4; ebits=8 disables it)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
`coli serve` runs the engine with SERVE_BATCH=1 (openai_server.py), which
selects run_serve_mux, which sets g_draft=0 — speculation isn't ragged-safe
across the multi-slot batch. But the "[MTP] active: native speculative decoding
(draft=N)" line is printed in main() BEFORE the serve path is chosen, so
`coli serve DRAFT=8` announced draft=8 and then silently disabled it.
@LordMZTE reported the misleading message.
The load line and the [MTP] stderr line now detect the mux case (SERVE +
SERVE_BATCH) and report it truthfully: "MTP DISABLED (multiplexed serve)" with
draft=0, plus a one-line explanation that single-client interactive use
(`coli chat`, which spawns run_serve without SERVE_BATCH) keeps MTP. No more
draft=8 claim on a path that runs draft=0.
This is the honesty half of #358. The feature half — MTP inside the HTTP
server for a single client — is a real enhancement but needs engine work: the
mux decode kernel would have to run the speculative path when exactly one KV
slot is active (S=1 is not actually ragged), and it needs a server round-trip
test. Tracked separately; the message no longer lies in the meantime.
Reported-by: LordMZTE <#358>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The expert lookup counted a hit identically whether the pinned hot-store or the
LRU ecache served it — both Phase C branches bumped the single `m->hits`. With a
warm pin profile the pin tier absorbs most hot experts, so the ecache could be
serving anywhere from ~0% to most hits and the logs couldn't tell which. That
made every cache-policy question unanswerable, including #223's "at what cap
does the eviction policy start to win?" (a flat A/B can mean "pin absorbed
everything" or "genuine floor" and the lumped counter can't distinguish them).
Two counters `hit_pin`/`hit_ecache` bumped at the two lookup branches (the
existing `m->hits++` stays, so all existing math is unchanged), snapshotted in
ProfBase and reset alongside `hits`. Surfaced in the human-readable summaries:
decode: expert hit rate 31.3% (pin 22.1% + lru 9.2%)
[PROF]: hit 31.3% (X pin + Y lru / Z load)
tiny: hit rate 88.1% (0 pin + 74 lru / 10 miss)
The serve-mux STAT protocol line is untouched (openai_server.py parses it
positionally).
Invariant hit_pin + hit_ecache == hits holds by construction — exactly two
sites bump hits and each also bumps one split counter, nothing else mutates
hits. Verified numerically on the tiny oracle: 0 pin + 74 lru = 74 hits, 88.1%.
Zero cost: two increments on paths that already increment.
Reported-by: KingIcyCreamProjects <#336>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The local (`--indir`) branch of main() ignored --mtp and --indexer entirely:
it always called convert_shard() without keep_mtp/keep_idx and always wrote
`out-NNNNN.safetensors`. So the documented two-pass workflow — convert the main
model into an outdir, then run `--mtp` into the SAME outdir for the head — did
the opposite on the local path: with --mtp defaulting ebits to 8 and keep_mtp
staying False, it silently re-converted the whole model to per-row int8 and
overwrote the finished fmt=4 container shard by shard, printing nothing wrong.
@mohamedmastouri2000-boop lost 137 of 141 freshly-converted g64 shards to this
while building the public container for #298/#326.
The --indir branch now mirrors the download path: keep_mtp=a.mtp /
keep_idx=a.indexer passed through, output named out-mtp-/out-idx-/out- by mode,
empty shards skipped (an MTP pass emits only shards containing layer n_layers),
and config/tokenizer copied only on the main pass (the head/idx passes land in
an already-complete outdir).
Verified with a synthetic 2-layer + MTP-shard model: after the main pass, a
second --mtp pass into the same outdir leaves out-00000's md5 BYTE-IDENTICAL
and writes out-mtp-00000 alongside it. The bug would have changed that md5.
Reported-by: mohamedmastouri2000-boop <#355>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A single NaN or +Inf logit silently broke the default sampling path. +Inf
became `mx`, then `expf((Inf-mx))`/`expf((NaN-mx))` is NaN, the softmax sum went
NaN, every probability went NaN — and dist_sample's fallback loop
`if(g_pbuf[i]>0)` is false for NaN at every index, so it returned 0. Every
subsequent token: 0. No error, no warning. @KingIcyCreamProjects found it.
dist_build now takes `mx` over finite logits only, gives a non-finite logit
probability 0, and when the distribution is unusable (no finite logit, or a
non-finite/zero sum) collapses to a delta on the finite argmax and warns ONCE
on stderr — degraded, but a valid token and a visible cause, never a silent
stream of zeros. The finite argmax uses the index found during the mx pass
(robust even when lo[0] itself is NaN, where argmax_v would wrongly return 0).
tests/test_sample_nan.c: healthy logits still sample correctly; NaN/+Inf
injected at lo[0], the middle, and the last position all pick the finite
argmax; an all-non-finite vocab leaves no NaN in the buffer and doesn't crash.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@KingIcyCreamProjects caught a real artifact on the 9950X3D (#357 thread): the
nk=8192 cell reported ~75x, far above the real ~13-40x algorithm. Root cause was
two compounding bench bugs, both verified against the code:
1. ONE frozen input per cell. partial_select_desc's median-of-three pivot is
fully deterministic (no RNG), and bench_dsa_select froze the input then ran
2000 reps on that identical array -- so all 2000 reps hit the exact same
pivot sequence. A single lucky input spiked one nk row (stable across re-runs
because it's deterministic, not because it's real).
2. brng was a static global, initialized once and NEVER reset between cells.
So each (shape,nk) cell's input depended on every prior cell's brand() draws
-- reordering nks[] or adding a shape silently shifted all later inputs.
Fix:
- brng_seed() reseeds per (shape, nk, seed) so every cell is reproducible and
independent of cell ordering.
- Report the MEDIAN of N_SEEDS=11 independent inputs, each itself a median over
N_REPEAT=2000 timing reps. A lucky pivot now moves one of 11 samples, not the
reported number.
Measured on Intel Core Ultra 9 185H (this fix, median of 11 seeds):
realistic@8192 10.97x (was 13.57x single-seed on this box)
uniform@8192 9.39x (was 7.48x)
plateau@8192 16.11x (plateau ignores the RNG, unchanged as expected)
band: 6-26x across all cells, no anomalous spikes.
Correctness is unaffected (test_dsa_select, 129 cases, unchanged). This is a bench
fidelity fix only -- no engine code touched.
An all-grouped container (kv_b_proj at fmt=4) generates one correct token
and then EOS: qt_addrow and qt_matvec_rows handle fmt 0/1/2 and fall
through to the int2 decoder, so grouped-int4 kv_b was unpacked as 2-bit
pairs under a per-row scale that does not exist in the [O,ng] layout.
Prefill (S>4, reconstruction) is unaffected, which made the failure look
like an EOS bug rather than an attention bug.
Same class as #298 (CUDA absorb kernels missing fmt=4), CPU side. Existing
containers escape it because the recommended mixed-precision recipe keeps
kv_b at int8 (#237).
Adds per-group branches mirroring matmul_i4_grouped semantics. fmt 0/1/2/3
paths are untouched.
Co-Authored-By: Claude <noreply@anthropic.com>
Per review: the collapse starts one line before the sum — seeding
mx=lo[0] means a NaN at index 0 makes mx NaN, every (lo[i]-mx) NaN, and
the softmax is doomed at the max-finding, not the normalize. Seed mx
from -INFINITY and skip NaNs (x==x), mirroring the argmax_v change; if
nothing finite survives, fall back to mx=0 and let the post-sum guard
decide. The isfinite(s) guard is now the second line of defense rather
than the only one.
Clean logits take a byte-identical path (the extra x==x compare is
noise next to V expf calls). test_logit_nan gains the NaN-at-index-0
and all-NaN dist_build cases; test_topp's 123-case sweep still passes
on this tree, confirming no interaction with the #354 heap select.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
On the default serve path (TEMP>0, 0<NUCLEUS<1) a single NaN or +Inf in
the logits — a bad streamed expert tile, or an fp overflow in the matmul
at a low-RAM eviction boundary — poisoned softmax: g_pbuf became all-NaN,
dist_sample never satisfied cum>=u, and the fallback returned token 0. The
engine then emitted an unbroken run of token 0 with NO error. The greedy
path was equally blind: argmax_v started bv=lo[0] and `lo[i]>NaN` is always
false, so a NaN at index 0 pinned the argmax to 0.
- argmax_v: skip NaN (x==x) and seed from -inf, so it returns the max
finite/+Inf entry instead of being NaN-pinned to 0. Covers greedy decode
and the speculative-verify argmax path.
- dist_build: after the softmax sum, if s is non-finite or <=0, collapse
g_pbuf to a one-hot over the finite argmax and warn once, instead of
dividing every entry into NaN. Covers the nucleus and verify paths.
Both are O(1)/free on the happy path (one branch after the existing loop;
one extra comparison inside the existing argmax loop). Degrade + diagnose,
never silently corrupt.
test_logit_nan (wired into TEST_BINS): asserts argmax_v skips NaN/picks
+Inf, dist_build yields a finite normalized one-hot on the max finite
logit, dist_sample emits that token (not 0), and clean logits still give a
valid distribution. Fails on stock dev, passes with this change.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
coli passes the chat prompt to glm.exe through the PROMPT/COLI_PROMPT
environment variable. On Windows, plain getenv() is populated by the CRT
from the ANSI-codepage view of the environment block, not UTF-8 — so any
non-ASCII prompt text (Cyrillic, CJK, ...) is silently mangled before the
byte-level tokenizer ever sees it, even though the parent process (coli's
Python subprocess call) sets the value correctly via the wide env block.
Add compat_getenv_utf8() in compat.h: reads the variable through
GetEnvironmentVariableW and converts straight to UTF-8, bypassing the ANSI
codepage entirely. No-op passthrough to getenv() on non-Windows platforms.
coli_user_prompt() in glm.c now uses it for both COLI_PROMPT and PROMPT.
Verified: tiny-oracle self-test still 32/32 after rebuild, and a real run
against the full GLM-5.2-int4 model with a Cyrillic prompt now round-trips
correctly end to end (input echoed intact, coherent Cyrillic output),
where it previously produced replacement-character garbage on both input
and output.
compat_pread mapped every non-EOF ReadFile failure to EIO, so the field report
was always the same three words: "Input/output error". #307 then burned three
rounds of guessing across three people — storage? contention? alignment? —
because the actual GetLastError code never appeared anywhere.
compat_pread now stashes the code in a per-thread slot and pread_full appends
it to the failure line:
pread qs: Input/output error (off 780592, 0/8192 bytes, WinErr=1450)
That one number is the difference between "insufficient system resources"
(1450 -> memory pressure), "sharing violation" (32 -> AV interference),
"device not ready" (21 -> the T: drive itself) and a dozen other distinct
diagnoses. Diagnostic only: no behavior change, no retry policy — that
discussion lives in #361 and should be settled AFTER the first report tells
us which error we are actually retrying.
Linux path untouched byte-for-byte; the Windows compile is certified by the
check.yml MSYS2 job.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The DSA lightning indexer selects the top-index_topk (2048) context keys to
attend to by finding the threshold = keep-th largest attention score. It
previously full-qsorted all nk scores per layer per token (O(nk log nk)) just
to read one pivot value, then scanned the original array in position order to
build the kept set.
Replace the qsort with partial_select_desc (Hoare quickselect, median-of-three,
descending): O(nk) average to partition the keep largest into a[0..k), then the
threshold is min of that block. The two position-order scans (>thr then ==thr)
are UNCHANGED, so the kept-position set is bit-identical -- a stronger contract
than #335's sampling heap (which was multiset-only because the heap was unstable
and changed accumulation order). The quickselect pivot IS by definition the
keep-th largest, so the new threshold equals old tmp[keep-1] exactly.
Measured (bench_dsa_select, keep=2048, median of 2000 reps):
nk=2049: 119us -> 5.7us (21x)
nk=8192: 626us -> 43us (15x)
nk=32768: 2.8ms -> 0.28ms (10x)
nk=65536: 6.6ms -> 0.47ms (14x)
The gap widens with context (linear vs n-log-n). DSA only fires past index_topk,
so this is precisely the long-conversation regime where decode latency matters.
Adds test_dsa_select (in TEST_BINS): 129 cases asserting element-wise identical
kept-set vs an independent qsort reference across shapes (random, peaked,
sorted, reverse-sorted, tie-plateau, all-equal) and edges (keep==1, keep==nk).
Also directly checks the partition invariant.
Adds bench_dsa_select (on-demand, NOT a gate): reproduces the table above.
test_topp proves correctness; bench_topp measures the latency claim. It re-implements
the OLD dist_build (full-vocab qsort) inline on a private buffer and times it against
the REAL new dist_build over identical inputs in one process: V=151936, temp=0.7,
3 shapes (realistic / uniform / plateau) x 4 nuc values (0.5/0.9/0.95/0.99), 2000
timed reps each, median reported. Deliberately NOT in TEST_BINS -- it's a microbench,
not a gate. Build on demand: make tests/bench_topp && ./tests/bench_topp
dist_build() sorted the entire 151936-entry vocab by probability (qsort) on every
sampled token whenever 0 < g_nuc < 1 — the serve default — and again per draft
position under rejection sampling. Measured cost: 5.6-8.0 ms/call; the actual work
is finding the few-hundred-token head whose cumulative mass reaches g_nuc.
Replace the full qsort + linear scan with a max-heap partial select:
- Floyd heapify g_pidx over V by descending g_pbuf prob (O(V), cache-friendly)
- pop winners to the array's high end until cum >= g_nuc (k * O(log V))
- the remaining heap prefix IS the tail -> zero it, renormalize the head
Winners land in g_pidx[out..V-1] in descending order, so s2 accumulates in the
same order as before -> head is unchanged on tie-free shapes (ties were already
unspecified under the unstable qsort). All four dist_build/dist_sample contract
properties hold: g_pbuf stays id-indexed, g_pidx stays internal, the tail is
fully zeroed, the head renormalizes to 1.
No API change, no caller change, no new globals.
c/tests/test_topp.c (new): drives the REAL dist_build via the include-glm.c
pattern against an independent double-precision reimplementation of the OLD
algorithm. 123 cases: 6 sizes (1..1519) x 5 shapes (uniform/peaked/geometric/
plateau/sharptail) x 4 nuc values, plus the g_nuc>=1 guard-off paths and V=1.
Tie-free shapes compare head values within 1e-6 rel (float vs double renorm
noise); tie shapes compare value-multisets. No scratch files -> builds clean on
Windows MinGW without the unmerged mkdtemp shim (#352).
test_stops.c uses POSIX mkdtemp() to make a scratch dir in the CWD, but
MinGW-w64 does not declare mkdtemp, so the test failed to compile on the
Windows job - and only there:
tests/test_stops.c:73:9: error: implicit declaration of function 'mkdtemp';
did you mean 'mktemp'? [-Wimplicit-function-declaration]
make: *** [Makefile:318: tests/test_stops.exe] Error 1
That halted `make test` at the C-test stage on Windows (test_uring is
correctly Linux-only, so test_stops was the only blocker).
Added a compat_mkdtemp shim to compat.h, following the file's existing
convention (every platform difference lives there; the .c stays clean):
_mktemp fills the trailing X's in place (same contract as mkdtemp), then
_mkdir creates the directory. Also added <direct.h> for _mkdir. On Linux
compat.h is a complete no-op, so POSIX mkdtemp is untouched there.
Verified: test_stops builds clean on MinGW and all 5 sub-cases pass:
tokenizer special-flag parsing, config/generation_config eos union,
no-generation_config fallback, both-configs-mutilated tokenizer sweep,
and the T=NULL validation path.
A clean 'make glm' on MinGW emitted exactly two warnings, both real:
1. compat.h:240 - ignoring pragma comment [-Wunknown-pragmas]
#pragma comment(lib, "psapi.lib") is an MSVC directive; MinGW/GCC
warns about it. Guarded with ifdef _MSC_VER - MinGW links psapi via
-lpsapi (already in the Makefile), MSVC keeps the pragma.
2. glm.c:1210 - g_numa_nodes defined but not used [-Wunused-variable]
g_numa_nodes is only read/written inside ifdef __linux__ blocks, so on
every non-Linux build it is a static that is never used. Moved the
definition under the same __linux__ guard; nothing references it off-Linux.
Verified: rm -f *.o glm.exe && make glm -> 0 warnings, 0 errors.
The hot-thread tuning sets OMP_WAIT_POLICY=active and caps libgomp's spin
with GOMP_SPINCOUNT=200000. LLVM libomp reads NEITHER: under active policy it
sets KMP_BLOCKTIME=infinite, so once the answer ends and the engine parks on
stdin, the whole OMP team spins forever — ~100% per thread, the 3000% figure
reported on FreeBSD 15.1 in #341 (clang/libomp is the default toolchain
neighborhood there; the same applies to macOS builds).
KMP_BLOCKTIME=200 keeps the team hot for 200 ms after each parallel region —
plenty to bridge back-to-back per-expert matmuls — and then sleeps it.
setenv with overwrite=0, so a user-provided KMP_BLOCKTIME still wins, and
libgomp builds ignore the variable entirely: on GCC toolchains this commit
is a no-op by construction.
Not reproduced locally (this box is gcc/libgomp and idles clean); shipped as
the mechanism-matching candidate fix with a request on the issue for the
reporter to verify on dev and to name their OpenMP runtime (ldd | grep omp).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
JustVugg