KingIcyCreamProjects
1a243cfc3e
On Windows the engine self-exec OMP tuning never runs (Linux/FreeBSD-only)
and posix_fadvise readahead is a compat.h no-op, so a stock Windows run
leaves large measured wins on the table. The launcher now setdefaults, on
win32 only, each independently overridable by setting the variable:
- OMP_WAIT_POLICY=active, GOMP_SPINCOUNT=200000, OMP_DYNAMIC=FALSE,
OMP_NUM_THREADS=<physical cores> (parity with the glm.c self-exec block;
COLI_NO_OMP_TUNE disables exactly this block, presence-based like the
engine). OMP_PROC_BIND/OMP_PLACES deliberately omitted and also removed
from environment_for_plan on win32: MinGW libgomp has no affinity support
("Affinity not supported on this configuration").
- DIRECT=1: unbuffered expert reads. Measured on a 9950X3D + Samsung 9100
PRO Gen5 + Win11: iobench 10.68 GB/s O_DIRECT vs 9.03 buffered (warm);
end-to-end REPLAY 0.48 -> 1.02 tok/s. Matches #162 (1.47x same class).
- PIPE=1: load/matmul overlap, byte-identical output; +8% on top of DIRECT
(PIPE_WORKERS untouched at 8 - 4/8/16 swept flat on Gen5).
- PILOT_REAL=1: real cross-layer prefetch, the only working prefetch on
Windows; +11% and expert hit rate +19 points.
Full ladder methodology and numbers: 96-token greedy REPLAY, one lever per
step, medians of 3-4 runs (see the fork tuning doc referenced in the PR).
tests/test_env_defaults.py covers the defaults, explicit-override-wins,
the kill-switch scope, and the non-win32 no-op.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
69 lines
2.5 KiB
Python
69 lines
2.5 KiB
Python
"""env_for: default Windows misurati (DIRECT/PIPE/PILOT_REAL + blocco OMP).
|
|
|
|
Carica `coli` come modulo (ha la guardia __main__) e verifica il contratto:
|
|
- win32: i tre default I/O e il blocco OMP sono setdefault
|
|
- un override esplicito dell'utente vince sempre
|
|
- COLI_NO_OMP_TUNE spegne SOLO il blocco OMP, non i default I/O
|
|
- non-win32: env_for non tocca nulla di tutto questo
|
|
"""
|
|
import importlib.machinery
|
|
import importlib.util
|
|
import os
|
|
import sys
|
|
import types
|
|
import unittest
|
|
from pathlib import Path
|
|
from unittest import mock
|
|
|
|
HERE = Path(__file__).resolve().parent.parent
|
|
sys.path.insert(0, str(HERE))
|
|
|
|
_loader = importlib.machinery.SourceFileLoader("coli_cli", str(HERE / "coli"))
|
|
_spec = importlib.util.spec_from_loader("coli_cli", _loader)
|
|
coli = importlib.util.module_from_spec(_spec)
|
|
_loader.exec_module(coli)
|
|
|
|
|
|
def args(**over):
|
|
base = dict(model="X", policy="quality", ram=0, ngen=0, topp=0, topk=0,
|
|
temp=None, repin=0, ctx=0, auto_tier=False, gpu=None, vram=0)
|
|
base.update(over)
|
|
return types.SimpleNamespace(**base)
|
|
|
|
|
|
class EnvDefaultsTest(unittest.TestCase):
|
|
def env_for_with(self, environ, platform):
|
|
with mock.patch.dict(os.environ, environ, clear=True), \
|
|
mock.patch.object(sys, "platform", platform):
|
|
return coli.env_for(args())
|
|
|
|
def test_win32_sets_measured_defaults(self):
|
|
e = self.env_for_with({}, "win32")
|
|
self.assertEqual(e["DIRECT"], "1")
|
|
self.assertEqual(e["PIPE"], "1")
|
|
self.assertEqual(e["PILOT_REAL"], "1")
|
|
self.assertEqual(e["OMP_WAIT_POLICY"], "active")
|
|
self.assertNotIn("OMP_PROC_BIND", e) # MinGW libgomp: niente affinity
|
|
|
|
def test_explicit_override_wins(self):
|
|
e = self.env_for_with({"DIRECT": "0", "PIPE": "0"}, "win32")
|
|
self.assertEqual(e["DIRECT"], "0")
|
|
self.assertEqual(e["PIPE"], "0")
|
|
self.assertEqual(e["PILOT_REAL"], "1") # non overridden -> default
|
|
|
|
def test_kill_switch_scope_is_omp_only(self):
|
|
e = self.env_for_with({"COLI_NO_OMP_TUNE": "1"}, "win32")
|
|
self.assertNotIn("OMP_WAIT_POLICY", e)
|
|
self.assertNotIn("OMP_NUM_THREADS", e)
|
|
self.assertEqual(e["DIRECT"], "1") # i default I/O restano attivi
|
|
self.assertEqual(e["PIPE"], "1")
|
|
|
|
def test_non_windows_untouched(self):
|
|
e = self.env_for_with({}, "linux")
|
|
for k in ("DIRECT", "PIPE", "PILOT_REAL", "OMP_WAIT_POLICY"):
|
|
self.assertNotIn(k, e)
|
|
|
|
|
|
if __name__ == "__main__":
|
|
unittest.main()
|