url/colibri
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Merge pull request #363 from woolcoxm/fix/win32-coli-chat-cuda-autoenable

Browse Source 
win32: auto-enable the GPU in bare coli chat
This commit is contained in:
Vincenzo Fornaro
2026-07-20 17:39:09 +02:00
committed by GitHub
parent b347092e5c 0b05f6a76b
commit f8e0612f26
3 changed files with 130 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/ENVIRONMENT.md
+9
View File
@@ -105,6 +105,15 @@ Format: `VAR` — default — effect.
| `COLI_CUDA_SHARED_W4A16_MIN_ROWS` | `32` | Min row count to engage the shared-MLP W4A16 kernel. |
| `COLI_METAL_UNTRACKED` | off (Metal only) | `=1` sets `MTLResourceHazardTrackingModeUntracked` on Metal buffers (reduces hazard-tracking overhead). |
> **Windows note.** On Windows, a bare `coli chat` / `coli run` / `coli serve`
> (no `--gpu`/`--vram`/`--auto-tier`) **auto-enables the GPU** when it detects a
> CUDA build (`coli_cuda.dll` next to the engine) and at least one GPU via
> `nvidia-smi`. The expert-tier VRAM budget is then sized automatically from the
> card's free VRAM (same computation as `--auto-tier`). If `nvidia-smi` is not on
> `PATH` the run falls back to CPU with a warning — pass `--vram N` (or add
> `nvidia-smi` to `PATH`) to enable CUDA in that case. `--gpu none` forces
> CPU-only. (Linux/macOS behaviour is unchanged: pass a flag to enable CUDA.)
---
## Advanced / experimental / debug