fix(coli): honour explicit COLI_CUDA=0 over the Windows bare-run auto-enable

Before this, a Windows user setting COLI_CUDA=0 for a CPU baseline silently got a ~12.6 GB VRAM expert tier anyway: the bare-run auto-enable ran before any env check, so every Windows 'CPU' benchmark row taken this way was actually GPU-assisted (found while building the 12-cell resource matrix in #467). Setting COLI_GPU=none instead collided with auto-enable's COLI_GPUS ('use COLI_GPU or COLI_GPUS, not both', exit 2). Now an explicit COLI_CUDA=0 suppresses auto-enable and clears stale device/sizing vars, same as --gpu none, which stays the canonical hard off-switch. Docs: COLI_CUDA row updated; DIRECT row + README get the measured, drive-dependent O_DIRECT guidance (+34 decode on real NVMe w/ DRAM cache; neutral-to-negative on QLC/DRAM-less).
This commit is contained in:
Mohamed Mastouri
2026-07-21 01:44:48 +03:00
parent 68ac9ff696
commit f939191404
3 changed files with 22 additions and 3 deletions
+7
View File
@@ -148,6 +148,13 @@ on-device across layers so the CPU expert loop runs uninterrupted; on Apple
Silicon an experimental [Metal backend](docs/metal.md) does the batched expert
math on the unified-memory GPU.
> **On real NVMe, measure `DIRECT=1`.** O_DIRECT bypasses the page cache and is
> often a large win on drives with DRAM cache and bandwidth headroom (+34%
> decode measured with `PIPE=1` on a Blackwell/Windows box; 4.25→9.69 GB/s in
> iobench on a GB10) — but it is drive-dependent: QLC/DRAM-less or virtualised
> disks can be neutral to negative. Try it first; keep what your hardware
> rewards.
### Faithful model, compressed state
The forward pass is validated **token-exact against a `transformers` oracle**