url/colibri
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Windows: direct I/O (FILE_FLAG_NO_BUFFERING, 1.47x decode) + VirtualLock pin wiring + select_ctx device cache (#162)

Browse Source 
* win: direct I/O via FILE_FLAG_NO_BUFFERING + compat_fsize + VirtualLock primitives

compat_open_direct() gives Windows the O_DIRECT twin fd st.h already uses
on Linux/macOS: FILE_FLAG_NO_BUFFERING, same 4K-alignment contract as
O_DIRECT (the engine's DIRECT=1 path already aligns offset/len and slabs
are posix_memalign'd).

Measured on GLM-5.2 744B int4, Ryzen 9 9950X3D / 126 GB / PCIe4 NVMe
(5.8 GB/s at the engine's 19MBx8T pattern), Windows 11, MinGW GCC 16.1,
32-token greedy runs at --topp 0.7, 40 GB pin, current dev HEAD:
  buffered:  0.38 tok/s (expert-disk dominates)
  DIRECT=1:  0.56 tok/s (1.47x) — byte-identical greedy output vs buffered

compat_fsize() (GetFileSizeEx): CRT lseek(SEEK_END) returns -1 on
NO_BUFFERING fds (measured on UCRT); iobench uses it and gains a
NO_BUFFERING branch so disk numbers are comparable across platforms.

compat_mlock/compat_munlock: VirtualLock with working-set growth (bare
VirtualLock caps at the default working-set minimum, a few hundred KB).
Wired into the engine in the next commit.

tests/test_compat_direct.c covers the alignment contract, data integrity,
fsize on both fd kinds; skips cleanly off Windows.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* win: wire VirtualLock into mem_wire, munlock pairing in expert_host_release

MLOCK=1 was a silent no-op on Windows: pinned experts could be paged out
by working-set trimming under memory pressure. mem_wire now uses
compat_mlock (VirtualLock + working-set growth); expert_host_release
unlocks before freeing, mirroring the POSIX branch.

Validated: 39.6 GB pin wired in 17s on a 126 GB machine, zero failures;
TF oracle 32/32 with MLOCK=1.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* cuda: thread-local current-device cache in select_ctx

cudaSetDevice on every call is expensive when the serial expert loop
alternates devices. Measured on RTX 5090 + RTX 4090 (Windows, DLL
backend, pre-#68 dispatch): expert-matmul 14.3s -> 25.4s per 32 tokens
going from 1 to 2 devices, entirely per-call context switching. The
current device is per-thread in the CUDA runtime, so a thread_local
cache skips redundant switches; multi-GPU expert serving becomes
positive-scaling instead of negative.

Kernel suite passes on sm_120 + sm_89; TF oracle 32/32 dual-GPU.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: olorin <io@zyphyr.co>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Tom Olorin
2026-07-14 07:12:15 -07:00
committed by GitHub
parent ec0aadf91a
commit d47b095875
7 changed files with 138 additions and 4 deletions
Download Patch File Download Diff File Expand all files Collapse all files
c/compat.h
+49
View File
@@ -57,6 +57,7 @@ static inline int compat_open_direct(const char *path){
* _read/_lseeki64 which are racy AND
* corrupt 0x0A bytes in binary files).
* posix_fadvise -> no-op (advisory only; macOS already no-ops DONTNEED).
* mlock -> compat_mlock (VirtualLock + crescita working set).
* posix_memalign->_aligned_malloc(free must be compat_aligned_free).
* rename -> compat_rename (MoveFileEx MOVEFILE_REPLACE_EXISTING;
* CRT rename fails EEXIST if dest exists,
@@ -135,6 +136,25 @@ static inline ssize_t compat_pread(int fd, void *buf, size_t n, off_t off){
}
#define pread(fd,buf,n,off) compat_pread(fd,buf,n,off)
/* --- mlock -> VirtualLock con crescita del working set ---
* VirtualLock fallisce oltre il working set MINIMO del processo (default ~qualche
* centinaio di KB): prima si allarga il working set di len + margine, poi si blocca.
* Best effort come mlock su Linux: -1 su fallimento, il chiamante decide (pin_wire
* lo tratta come non-fatale). SeIncreaseWorkingSetPrivilege e' concesso agli utenti
* standard di default. */
static inline int compat_mlock(const void *addr, size_t len){
HANDLE p = GetCurrentProcess();
SIZE_T mn = 0, mx = 0;
if(GetProcessWorkingSetSize(p, &mn, &mx)){
SIZE_T need = len + (SIZE_T)(1u<<20);
SetProcessWorkingSetSize(p, mn + need, mx + need); /* best effort */
}
return VirtualLock((LPVOID)addr, len) ? 0 : -1;
}
static inline int compat_munlock(const void *addr, size_t len){
return VirtualUnlock((LPVOID)addr, len) ? 0 : -1;
}
/* --- posix_memalign -> _aligned_malloc ---
* ATTN: memoria allocata con _aligned_malloc DEVE essere liberata con
* _aligned_free, NON con free(). Vedi compat_aligned_free sotto.
@@ -215,6 +235,35 @@ static inline ssize_t compat_getline(char **lineptr, size_t *n, FILE *stream){
}
#define getline(lineptr,n,stream) compat_getline(lineptr,n,stream)
/* --- O_DIRECT -> FILE_FLAG_NO_BUFFERING ---
* Apre il fd "gemello" senza cache del file system, come il twin O_DIRECT di
* st.h su Linux e F_NOCACHE su macOS. Stesso contratto: offset, lunghezza e
* buffer del chiamante devono essere allineati a 4K (gli slab expert usano
* posix_memalign(4096) e il percorso DIRECT=1 del motore allinea gia' offset
* e len); richieste non allineate falliscono con -1, mai dati corrotti.
* Il fd si usa con la normale pread() (compat_pread -> ReadFile+OVERLAPPED). */
static inline int compat_open_direct(const char *path){
HANDLE h = CreateFileA(path, GENERIC_READ,
FILE_SHARE_READ|FILE_SHARE_WRITE|FILE_SHARE_DELETE,
NULL, OPEN_EXISTING, FILE_FLAG_NO_BUFFERING, NULL);
if(h == INVALID_HANDLE_VALUE) return -1;
int fd = _open_osfhandle((intptr_t)h, _O_RDONLY|_O_BINARY);
if(fd < 0){ CloseHandle(h); return -1; }
return fd;
}
/* --- dimensione file da fd: GetFileSizeEx ---
* La lseek(SEEK_END) del CRT ritorna -1 sui fd NO_BUFFERING (misurato su
* UCRT): la dimensione si chiede direttamente al kernel. Funziona su
* qualsiasi fd (buffered o direct). -1 su errore. */
static inline off_t compat_fsize(int fd){
intptr_t osfh = _get_osfhandle(fd);
if(osfh == -1 || osfh == -2) return -1;
LARGE_INTEGER li;
if(!GetFileSizeEx((HANDLE)osfh, &li)) return -1;
return (off_t)li.QuadPart;
}
/* --- setenv -> SetEnvironmentVariableA (POSIX setenv assente su Windows) --- */
static inline int compat_setenv(const char *name, const char *value, int overwrite){
if(!overwrite && getenv(name)) return 0;