A downloaded (supply-chain) model file was fully trusted by the loader. Three
memory-safety holes, all reachable by pointing the engine at a crafted shard —
demonstrated crashing on pre-fix, now rejected fail-closed:
st.h (safetensors):
- header length `hlen` (u64 from the file) was unbounded before malloc(hlen+1):
a crafted value overflows (malloc(0) then hdr[hlen]=0 OOB) or forces a giant
allocation. Now bounded to the file size and a 512 MB cap; malloc NULL-checked.
- json_get() returns NULL for missing/mistyped fields, but dtype/data_offsets/
shape were dereferenced blind (off->kids[0]) — a header omitting data_offsets
SIGSEGV'd (verified). Now type/arity-checked before use.
- data_offsets [a0,b0] were trusted: b0<a0 gave a negative nbytes -> malloc((size_t))
giant and an oversized memcpy into the caller's buffer in st_read_f32 (heap
overflow); off could point outside the file. Now validated 0<=a0<=b0 and
data_start+b0<=filesize.
json.h: j_parse_val recursed with no depth limit -> stack overflow on nested
input like [[[[...]]]]. Added J_MAX_DEPTH=1024 (headers are ~3 deep); wide-but-
flat objects like the GLM header are unaffected (depth is decremented per return).
eval_glm.py: tempfile.mktemp() -> mkstemp() — closes the TOCTOU/symlink race on
a shared tmp dir (CWE-377).
Network path (openai_server.py + serve SUBMIT parser) audited separately and is
already sound: hmac.compare_digest auth, MAX_BODY cap, resolve()+relative_to
traversal guard, list-form subprocess, bounded/validated SUBMIT header. All 62
tests pass; valid GLM/OLMoE shards load unchanged.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* win: direct I/O via FILE_FLAG_NO_BUFFERING + compat_fsize + VirtualLock primitives
compat_open_direct() gives Windows the O_DIRECT twin fd st.h already uses
on Linux/macOS: FILE_FLAG_NO_BUFFERING, same 4K-alignment contract as
O_DIRECT (the engine's DIRECT=1 path already aligns offset/len and slabs
are posix_memalign'd).
Measured on GLM-5.2 744B int4, Ryzen 9 9950X3D / 126 GB / PCIe4 NVMe
(5.8 GB/s at the engine's 19MBx8T pattern), Windows 11, MinGW GCC 16.1,
32-token greedy runs at --topp 0.7, 40 GB pin, current dev HEAD:
buffered: 0.38 tok/s (expert-disk dominates)
DIRECT=1: 0.56 tok/s (1.47x) — byte-identical greedy output vs buffered
compat_fsize() (GetFileSizeEx): CRT lseek(SEEK_END) returns -1 on
NO_BUFFERING fds (measured on UCRT); iobench uses it and gains a
NO_BUFFERING branch so disk numbers are comparable across platforms.
compat_mlock/compat_munlock: VirtualLock with working-set growth (bare
VirtualLock caps at the default working-set minimum, a few hundred KB).
Wired into the engine in the next commit.
tests/test_compat_direct.c covers the alignment contract, data integrity,
fsize on both fd kinds; skips cleanly off Windows.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* win: wire VirtualLock into mem_wire, munlock pairing in expert_host_release
MLOCK=1 was a silent no-op on Windows: pinned experts could be paged out
by working-set trimming under memory pressure. mem_wire now uses
compat_mlock (VirtualLock + working-set growth); expert_host_release
unlocks before freeing, mirroring the POSIX branch.
Validated: 39.6 GB pin wired in 17s on a 126 GB machine, zero failures;
TF oracle 32/32 with MLOCK=1.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* cuda: thread-local current-device cache in select_ctx
cudaSetDevice on every call is expensive when the serial expert loop
alternates devices. Measured on RTX 5090 + RTX 4090 (Windows, DLL
backend, pre-#68 dispatch): expert-matmul 14.3s -> 25.4s per 32 tokens
going from 1 to 2 devices, entirely per-call context switching. The
current device is per-thread in the CUDA runtime, so a thread_local
cache skips redundant switches; multi-GPU expert serving becomes
positive-scaling instead of negative.
Kernel suite passes on sm_120 + sm_89; TF oracle 32/32 dual-GPU.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: olorin <io@zyphyr.co>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
JustVugg