On Windows a bare 'coli chat' (no --gpu/--vram/--auto-tier) ALWAYS ran
CPU-only, even on a CUDA build with a GPU present. Two defects:
1. cuda_binary() returned False on Windows. It detects CUDA by running
'ldd glm | grep libcudart', which is Linux-only (no ldd on win32) and
meaningless anyway because the Windows engine links cudart only inside a
runtime-loaded coli_cuda.dll, not as a libcudart symbol in glm.exe. So
the --gpu/--vram/--auto-tier gates (which call cuda_binary()) never opened.
2. Even with detection fixed, bare 'coli chat' set no CUDA env: env_for's
else-branch only enables CUDA when --gpu/--vram is passed. Nothing
auto-enabled the GPU.
Now: cuda_binary() on non-Linux returns True iff coli_cuda.dll exists next
to glm.exe — the exact file backend_loader.c loads from the engine's own
directory, so its presence is a faithful, cheap, DLL-hijack-safe proxy for a
CUDA-capable build. And env_for, scoped to win32 (Linux keeps its working
explicit-flag UX), auto-enables CUDA when a bare chat detects a CUDA build
plus a GPU via nvidia-smi, sizing the expert-tier VRAM budget from real free
VRAM via the existing build_plan/environment_for_plan machinery (same as
--auto-tier, no guessed budget). If nvidia-smi is missing it falls back to
CPU with a clear warning; --gpu none still forces CPU; explicit --vram/--gpu
still win. CUDA_DENSE stays an explicit opt-in (matches --auto-tier).
Verified on a Windows + RTX 5070 Ti box: bare 'coli chat --model <g64>' now
prints '[GPU] auto-enabled CUDA ... 13.0 GB expert tier' and emits
COLI_CUDA=1 / COLI_GPUS=0 / CUDA_EXPERT_GB=13.044 (was: all unset, CPU-only).
Tests: 4 new cases (auto-enable, nvidia-smi-missing fallback, CPU-build
silent, Linux-unchanged) plus the 4 existing default-I/O tests guarded to
mock cuda_binary() so they stay host-independent. Full python suite green
(env_defaults 8, resource_plan 10, doctor 8, makefile_platform 3, cli_output 3).
Out of scope: doctor.cuda_linkage is also POSIX-only and mis-reports on
Windows — separate follow-up.
On Windows the engine self-exec OMP tuning never runs (Linux/FreeBSD-only)
and posix_fadvise readahead is a compat.h no-op, so a stock Windows run
leaves large measured wins on the table. The launcher now setdefaults, on
win32 only, each independently overridable by setting the variable:
- OMP_WAIT_POLICY=active, GOMP_SPINCOUNT=200000, OMP_DYNAMIC=FALSE,
OMP_NUM_THREADS=<physical cores> (parity with the glm.c self-exec block;
COLI_NO_OMP_TUNE disables exactly this block, presence-based like the
engine). OMP_PROC_BIND/OMP_PLACES deliberately omitted and also removed
from environment_for_plan on win32: MinGW libgomp has no affinity support
("Affinity not supported on this configuration").
- DIRECT=1: unbuffered expert reads. Measured on a 9950X3D + Samsung 9100
PRO Gen5 + Win11: iobench 10.68 GB/s O_DIRECT vs 9.03 buffered (warm);
end-to-end REPLAY 0.48 -> 1.02 tok/s. Matches #162 (1.47x same class).
- PIPE=1: load/matmul overlap, byte-identical output; +8% on top of DIRECT
(PIPE_WORKERS untouched at 8 - 4/8/16 swept flat on Gen5).
- PILOT_REAL=1: real cross-layer prefetch, the only working prefetch on
Windows; +11% and expert hit rate +19 points.
Full ladder methodology and numbers: 96-token greedy REPLAY, one lever per
step, medians of 3-4 runs (see the fork tuning doc referenced in the PR).
tests/test_env_defaults.py covers the defaults, explicit-override-wins,
the kill-switch scope, and the non-win32 no-op.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
woolcoxm