Windows: direct I/O (FILE_FLAG_NO_BUFFERING, 1.47x decode) + VirtualLock pin wiring + select_ctx device cache (#162)
* win: direct I/O via FILE_FLAG_NO_BUFFERING + compat_fsize + VirtualLock primitives compat_open_direct() gives Windows the O_DIRECT twin fd st.h already uses on Linux/macOS: FILE_FLAG_NO_BUFFERING, same 4K-alignment contract as O_DIRECT (the engine's DIRECT=1 path already aligns offset/len and slabs are posix_memalign'd). Measured on GLM-5.2 744B int4, Ryzen 9 9950X3D / 126 GB / PCIe4 NVMe (5.8 GB/s at the engine's 19MBx8T pattern), Windows 11, MinGW GCC 16.1, 32-token greedy runs at --topp 0.7, 40 GB pin, current dev HEAD: buffered: 0.38 tok/s (expert-disk dominates) DIRECT=1: 0.56 tok/s (1.47x) — byte-identical greedy output vs buffered compat_fsize() (GetFileSizeEx): CRT lseek(SEEK_END) returns -1 on NO_BUFFERING fds (measured on UCRT); iobench uses it and gains a NO_BUFFERING branch so disk numbers are comparable across platforms. compat_mlock/compat_munlock: VirtualLock with working-set growth (bare VirtualLock caps at the default working-set minimum, a few hundred KB). Wired into the engine in the next commit. tests/test_compat_direct.c covers the alignment contract, data integrity, fsize on both fd kinds; skips cleanly off Windows. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * win: wire VirtualLock into mem_wire, munlock pairing in expert_host_release MLOCK=1 was a silent no-op on Windows: pinned experts could be paged out by working-set trimming under memory pressure. mem_wire now uses compat_mlock (VirtualLock + working-set growth); expert_host_release unlocks before freeing, mirroring the POSIX branch. Validated: 39.6 GB pin wired in 17s on a 126 GB machine, zero failures; TF oracle 32/32 with MLOCK=1. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * cuda: thread-local current-device cache in select_ctx cudaSetDevice on every call is expensive when the serial expert loop alternates devices. Measured on RTX 5090 + RTX 4090 (Windows, DLL backend, pre-#68 dispatch): expert-matmul 14.3s -> 25.4s per 32 tokens going from 1 to 2 devices, entirely per-call context switching. The current device is per-thread in the CUDA runtime, so a thread_local cache skips redundant switches; multi-GPU expert serving becomes positive-scaling instead of negative. Kernel suite passes on sm_120 + sm_89; TF oracle 32/32 dual-GPU. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: olorin <io@zyphyr.co> Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
@@ -27,6 +27,10 @@ int main(int argc,char**argv){
|
||||
int fd=open(argv[1],O_RDONLY|(direct?O_DIRECT:0));
|
||||
if(fd<0 && direct){ fprintf(stderr,"O_DIRECT is unavailable (%s); using buffered I/O\n",strerror(errno));
|
||||
direct=0; fd=open(argv[1],O_RDONLY); }
|
||||
#elif defined(_WIN32)
|
||||
int fd = direct ? compat_open_direct(argv[1]) : open(argv[1],COMPAT_O_RDONLY);
|
||||
if(fd<0 && direct){ fprintf(stderr,"NO_BUFFERING is unavailable; using buffered I/O\n");
|
||||
direct=0; fd=open(argv[1],COMPAT_O_RDONLY); }
|
||||
#else
|
||||
int fd=open(argv[1],O_RDONLY); /* macOS: F_NOCACHE ~ O_DIRECT */
|
||||
#ifdef __APPLE__
|
||||
@@ -36,7 +40,11 @@ int main(int argc,char**argv){
|
||||
#endif
|
||||
#endif
|
||||
if(fd<0){perror("open");return 1;}
|
||||
#ifdef _WIN32
|
||||
off_t sz=compat_fsize(fd); /* CRT lseek(SEEK_END) ritorna -1 sui fd NO_BUFFERING */
|
||||
#else
|
||||
off_t sz=lseek(fd,0,SEEK_END);
|
||||
#endif
|
||||
if(sz<blk*2){fprintf(stderr,"file is too small\n");return 1;}
|
||||
/* offset random pre-generati (stessi per ogni configurazione: srand fisso).
|
||||
* 30 bit di rand combinati: su Windows RAND_MAX=32767 e un singolo rand()*4096
|
||||
|
||||
Reference in New Issue
Block a user