Mohamed Mastouri 0d2fb6f8a2 feat(st): COLI_MODEL_DIRS - split model shards across N drives, no duplication ...

Each extra directory holds a DISTINCT subset of the .safetensors shards (search path, dedup by basename; first-listed dir wins). Demand preads hit whichever drive holds the shard, so concurrent expert loads parallelise across drives and combined capacity is used - a 400 GB container fits across two smaller drives that individually cannot hold it, which COLI_MODEL_MIRROR (a full second copy) cannot do. Composable with the mirror: st_mirror_init matches per-shard by basename against the merged index. st_init stays as a back-compat wrapper over st_init_multi. Verified on RTX 5080 / Windows: 72+70 shards across two NVMes, coherent output, [SPLIT] startup log, decode parity with single-drive at full RAM (0.92 vs 0.89-0.90).

2026-07-21 01:42:04 +03:00

..

experiments

docs: GLM-5.2 6x RTX 5090 full-residency experiment — 6.84 tok/s, honest dead-ends log (#94)

2026-07-12 22:06:51 +02:00

media

docs/media: refreshed dashboard shots (full-residency 4 tok/s, disk 0), new Atlas galaxy, phone + sidebar shots

2026-07-17 00:29:50 +08:00

api.md

docs(api): tell coding-CLI users about the prefill cost before they hit it (#373)

2026-07-18 02:33:23 +02:00

benchmarks.md

plan NUMA interleave on multi-socket Linux

2026-07-18 18:38:40 +08:00

CACHE_ROUTE.md

routing: opt-in CACHE_ROUTE max-rank prefer (pin∪LRU), default off — 2.4->3.33 tok/s on GB10 (#199)

2026-07-14 22:08:28 +02:00

cuda.md

README: restructure around the technical path — parameters and full results move to docs/, four new figures

2026-07-16 23:24:37 +08:00

ENVIRONMENT.md

feat(st): COLI_MODEL_DIRS - split model shards across N drives, no duplication

2026-07-21 01:42:04 +03:00

grammar-draft.md

review: docs section, 1 MiB grammar pre-check, negative tests, measured compile overhead

2026-07-20 17:27:33 +01:00

MAINTAINING-DOCS.md

docs: environment variables + CLI settings reference (#151)

2026-07-14 14:10:59 +02:00

METAL-M5MAX-PERF-REPORT.md

docs: Metal M5 Max perf report — OMP hot-team spin (#77) throttles Apple GPU, NO_OMP+PIPE recovers to 2.24 tok/s (#116)

2026-07-13 20:47:19 +02:00

metal.md

README: restructure around the technical path — parameters and full results move to docs/, four new figures

2026-07-16 23:24:37 +08:00

quickstart.md

docs: explain the prebuilt Windows binary — what the .exe is and how to run it (#450)

2026-07-20 17:23:21 +02:00

serve_protocol.md

docs: serve protocol reference — mux wire format, telemetry lines, HTTP surface (#310)

2026-07-16 21:25:45 +08:00

SETTINGS.md

docs: environment variables + CLI settings reference (#151)

2026-07-14 14:10:59 +02:00

tuning.md

README: restructure around the technical path — parameters and full results move to docs/, four new figures

2026-07-16 23:24:37 +08:00

windows.md

docs: collapse WINDOWS.md/windows.md case collision into one file (#371)

2026-07-18 01:15:48 +02:00