windows: PIPE default ON + compat_fadvise WILLNEED cache-warmer (pread path)
Cut Windows decode disk I/O from 2.06s/tok to 1.70s/tok (budget=4), meeting the <=2s/tok target. Two changes on the pread expert-load path: 1. compat.h: replace the posix_fadvise no-op with a real WILLNEED cache-warmer (overlapped ReadFile into a scratch buffer -> populates the standby page cache so the later synchronous pread faults from RAM). Re-arms the existing expert_prefetch/PILOT/next-block prefetch chain on Windows. DONTNEED stays a no-op (matches macOS; Windows standby-list trimming self-regulates). Measured: hit rate 16.4% -> 27.6%. 2. glm.c: flip PIPE (async expert-load thread pool) from default OFF to default ON on Windows. Dispatches expert pread onto worker threads so loads overlap the matmul, instead of blocking serial load-then-compute. PIPE=0 opts out. Measured: expert-disk 65.9s -> 54.3s (-18%). Also adds compat_fadvise assertions to tests/test_compat_direct.c (data integrity after cache-warmer, safe no-op on bad fd / non-WILLNEED). mmap (CreateFileMapping/MapViewOfFile) was implemented and tested at length but reverted: it regressed on Windows (RSS bloat from touched mapped pages collapsed the expert cache via ullAvailPhys — a fundamental Windows-vs-Linux difference). Full findings + the dead-end analysis recorded in issue_diskio.md.
This commit is contained in:
@@ -1622,7 +1622,10 @@ static int expert_load(Model *m, int layer, int eid, ESlot *s, int fatal){
|
||||
* condvar exist ONLY to park/wake idle workers, never for correctness. Gated
|
||||
* behind PIPE=1; OFF => the original blocking-load + serial-matmul path runs
|
||||
* byte-identically. */
|
||||
static int g_pipe=0; /* PIPE=1: async expert-load pipeline (default OFF) */
|
||||
static int g_pipe=0; /* PIPE=1: async expert-load pipeline. Default ON for Windows
|
||||
* (parsed in main: getenv("PIPE")?:1 on _WIN32, :0 elsewhere).
|
||||
* Keeps expert pread off the forward-pass thread so loads overlap
|
||||
* the matmul. PIPE=0 opts back into the blocking serial path. */
|
||||
static int g_pipe_nw=8; /* PIPE_WORKERS=n: I/O worker threads (disk-parallel reads) */
|
||||
typedef struct {
|
||||
_Atomic uint64_t cur; /* (gen<<8)|index; gen main-only, index 0..njobs (≤64) */
|
||||
@@ -5078,7 +5081,13 @@ int main(int argc, char **argv){
|
||||
g_pilot_k = getenv("PILOT_K")?atoi(getenv("PILOT_K")):(g_pilot_real?6:8);
|
||||
if(g_pilot_k<1) g_pilot_k=1;
|
||||
g_disk_split = getenv("DISK_SPLIT")?atoi(getenv("DISK_SPLIT")):0; /* 1 = split dei disk load nelle stats */
|
||||
g_pipe = getenv("PIPE")?atoi(getenv("PIPE")):0; /* default OFF: overlap expert load ‖ matmul (byte-identical; reorders I/O). PIPE=1 opts in */
|
||||
g_pipe = getenv("PIPE")?atoi(getenv("PIPE")):
|
||||
#ifdef _WIN32
|
||||
1 /* default ON: overlap expert load ‖ matmul (byte-identical; reorders I/O). PIPE=0 opts out */
|
||||
#else
|
||||
0
|
||||
#endif
|
||||
;
|
||||
g_pipe_nw = getenv("PIPE_WORKERS")?atoi(getenv("PIPE_WORKERS")):8; /* I/O worker threads */
|
||||
if(g_pipe_nw<1) g_pipe_nw=1;
|
||||
g_direct = getenv("DIRECT")?atoi(getenv("DIRECT")):0;
|
||||
|
||||
Reference in New Issue
Block a user