- load_expert_w: detect I8/U8 tensors and read them raw with their .qs scales.
Before this, pointing SNAP at a convert_olmoe.py container crashed with
SIGBUS (st_read_f32 walks I8 data as 2-byte elements). Container misses now
cost half the I/O and skip quantize_rows entirely: 2.08 -> 3.69 tok/s on the
miss-heavy 16-slot ref.json run (M5). Raw bf16 checkpoints keep working.
- matmul_q: Q8_0-style integer path on ARM (per-16 activation blocks, sdot on
dotprod CPUs, vmull fallback). Same IDOT family as glm.c, same semantics:
IDOT=0 stays byte-exact vs the oracle (12/12); default integer path can flip
an argmax tie (documented in glm.c README). End-to-end on M5: 4.5 -> 12 tok/s
warm-cache decode.
- rss_gb: ru_maxrss is bytes on macOS, KB on Linux. RSS lines were reading
'2029 GB' on Macs.
- convert_olmoe.py: snapshot_download(local_files_only=True) raises
LocalEntryNotFoundError when the repo is not cached; the download fallback
was unreachable.
Measured on M5 MacBook (10 cores, 24 GB, macOS 26.5), OLMoE-1B-7B-0125:
ref.json greedy 12/12 with IDOT=0 on both container and raw paths.
Co-authored-by: x <x@Mac.fritz.box>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Louie
ec0aadf91a