#421 committed the compiled tests/test_st_pread (mode 644, non-executable).
On a fresh CI checkout make sees it as up-to-date and skips rebuilding, so
run_tests.py hits 'Permission denied' exec'ing a non-executable file — the
linux/macos 'make check' failure. Remove the binary and gitignore it next to
the existing test_st_mirror rule.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A second (read-only) copy of the model on another drive is registered as a
per-shard read replica; expert loads are split between the drives by a
deterministic (layer,eid) hash weighted by COLI_DISK_WEIGHTS=<primary>,<mirror>
or, by default, by a startup bandwidth probe using the engine's own access
pattern (parallel ~19 MB reads, O_DIRECT). Cold decode is disk-bound
(~11 GB/token), so two NVMe queues add up.
- st.h: mirror accepted per file only if size + safetensors header are
byte-identical to the primary (identical data_offsets by construction, so
every pread is valid on either copy); partial mirrors work (smaller second
SSD holding only some shards); the mirror is never written — .coli_usage /
.coli_kv stay on the primary.
- glm.c: routing covers the coalesced slab pread, O_DIRECT, mmap/Metal and
scale reads, plus the OMP-parallel pin/autopin warmup (streams from both
drives). Deterministic routing keeps readahead/PILOT WILLNEED on the same
drive as the demand read and avoids caching an expert twice. A mirror read
error falls back to the primary (one warning, no crash). Per-drive bytes
are reported in a MIRROR: stats line.
- tests: test_st_mirror covers validation, read equality on both replicas,
and the rejection paths (divergent header, size mismatch, missing file).
Measured on 2x NVMe (GLM-5.2 int4, greedy, DRAFT=0, DIRECT=1, cold-ish
cache): 0.42 -> 0.57 tok/s (+36%), expert-disk service 15.2s -> 10.3s,
byte-identical output; probe chose a 48/52 split.
JustVugg