fix(coli): honour explicit COLI_CUDA=0 over the Windows bare-run auto-enable

Before this, a Windows user setting COLI_CUDA=0 for a CPU baseline silently got a ~12.6 GB VRAM expert tier anyway: the bare-run auto-enable ran before any env check, so every Windows 'CPU' benchmark row taken this way was actually GPU-assisted (found while building the 12-cell resource matrix in #467). Setting COLI_GPU=none instead collided with auto-enable's COLI_GPUS ('use COLI_GPU or COLI_GPUS, not both', exit 2). Now an explicit COLI_CUDA=0 suppresses auto-enable and clears stale device/sizing vars, same as --gpu none, which stays the canonical hard off-switch. Docs: COLI_CUDA row updated; DIRECT row + README get the measured, drive-dependent O_DIRECT guidance (+34 decode on real NVMe w/ DRAM cache; neutral-to-negative on QLC/DRAM-less).
This commit is contained in:
Mohamed Mastouri
2026-07-21 01:44:48 +03:00
parent 68ac9ff696
commit f939191404
3 changed files with 22 additions and 3 deletions
+2 -2
View File
@@ -41,7 +41,7 @@ Format: `VAR` — default — effect.
| `PIPE` | `0` (off) | Overlap expert disk-load with matmul via I/O worker threads. Byte-identical output; reorders I/O. `PIPE=1` opts in. |
| `PIPE_WORKERS` | `8` | Number of pthread loaders when `PIPE=1`, or the io-wq worker maximum per ring when `URING=1` (capped at 64). Tune to SSD queue depth and available cores. |
| `URING` | `0` (off) | Linux-only queued expert I/O. `URING=1` implies `PIPE=1`, forces cold reads through io-wq (`IOSQE_ASYNC`), replaces blocking loader pthreads and spin waits with batched SQEs/CQEs, and batches `PILOT_REAL` loads on a separate ring. Use `DIRECT=1` for cold NVMe to avoid page-cache copy/readahead limits. Fails clearly if the kernel denies io_uring; incompatible with `COLI_MMAP=1`. |
| `DIRECT` | `0` (off) | Use `O_DIRECT`/unbuffered reads for expert slabs. Helps sustained NVMe; keeps the zero-copy GPU path. |
| `DIRECT` | `0` (off) | Use `O_DIRECT`/unbuffered reads for expert slabs. **Drive-dependent — measure it on your hardware.** On real NVMe with DRAM cache and headroom it is often a large win (measured +34% decode with `PIPE=1` on a Blackwell/Windows box, and 4.25→9.69 GB/s in iobench on a GB10); on QLC/DRAM-less drives or slow/virtualised disks it can be neutral to negative. Helps sustained NVMe; keeps the zero-copy GPU path. |
| `COLI_NO_OMP_TUNE` | off | **Kill-switch** for the OpenMP hot-thread tuning (`OMP_WAIT_POLICY=active` spin + proc-bind). Set `=1` when the CPU is mostly waiting on the GPU (Metal) so spin doesn't steal the shared power budget. |
| `COLI_NUMA` | auto in generated plans on multi-socket Linux; otherwise off | `COLI_NUMA=1` selectively interleaves large expert and dense slabs across NUMA nodes via `mbind` (raw syscall, no libnuma). Helps multi-socket hosts (+740% expert matmul); silent no-op on single-node or non-Linux. Explicit `COLI_NUMA=0` overrides the generated plan. |
| `MLOCK` | `-1` (auto: on for macOS) | Wire the streamed expert cache into physical RAM (`mlock`) to dodge the memory compressor. `0` off, `1` force. |
@@ -91,7 +91,7 @@ Per-drive byte counts are reported in a `MIRROR:` stats line. Combine with `DIRE
| Variable | Default | Effect |
|---|---|---|
| `COLI_CUDA` | off | Enable the CUDA backend. Requires a CUDA build. |
| `COLI_CUDA` | off | Enable the CUDA backend. Requires a CUDA build. An explicit `COLI_CUDA=0` disables it **and suppresses the Windows bare-run auto-enable** (before this, Windows "CPU" runs with `COLI_CUDA=0` silently got a VRAM expert tier). The CLI flag `--gpu none` is the canonical hard off-switch on every platform. |
| `COLI_GPU` / `COLI_GPUS` | unset | Device selection (`auto`, `none`, or a list like `0,1`). Requires `COLI_CUDA=1`. |
| `CUDA_DENSE` | `0` | Place dense (non-expert) matmuls on the GPU. |
| `CUDA_EXPERT_GB` | `0` | VRAM budget (GB) for caching experts on the GPU. |