Oleksandr Kuvshynov 5c84b25645 fix: old x86 Mac scalar -> vnni ...

Older macs still need -march, not -mcpu.

Before this fix, no ymm (or zmm, vnni, etc.).

Standalone repro:

```
% clang -O3  -mcpu=native glm.c -o /tmp/glm
clang: error: unsupported option '-mcpu=' for target 'x86_64-apple-darwin25.4.0'
% clang -O3  -mcpu=native glm.c -o /tmp/glm -lm
% otool -tv /tmp/glm | grep -c ymm
0
```

I'm not entirely sure why `-lm` would ignore `-mcpu=` rather than error, but either way we don't get vector instructions:

colibri itself:

```
% ulimit -l unlimited; OMP_NUM_THREADS=16 RAM_GB=700 PIN=auto PIN_GB=all PIN_FILL=1 ./coli run --ram 700 --model ~/projects/llms/glm5p2-i4 --ngen 256 "Explain the difference between concurrency and parallelism"
...
== GLM C engine (glm_moe_dsa), cache=8 experts/layer | experts@8-bit dense@8-bit | idot: scalar ==
```

After the fix, we'll get it:

```
% clang -O3  -march=native glm.c -o /tmp/glm -lm
% otool -tv /tmp/glm | grep -c ymm
2621
```

colibri run:
```
% ulimit -l unlimited; OMP_NUM_THREADS=16 RAM_GB=700 PIN=auto PIN_GB=all PIN_FILL=1 ./coli run --ram 700 --model ~/projects/llms/glm5p2-i4 --ngen 256 "Explain the difference between concurrency and parallelism"
...
== GLM C engine (glm_moe_dsa), cache=8 experts/layer | experts@8-bit dense@8-bit | idot: avx512-vnni ==
```

2026-07-19 16:40:09 -04:00

..

scripts

Translate user-facing runtime output to English, machine prefixes preserved, + CLI output test (#67, #85)

2026-07-12 13:38:40 +02:00

tests

fix: update test_logit_nan to include colibri.c instead of glm.c

2026-07-19 21:18:33 +08:00

tools

Merge pull request #362 from EgonRuiter/prefetcher-v3

2026-07-19 12:40:43 +02:00

.gitignore

Fix Windows cuda-dll build: quote NVCC, gate MSVC warnings, add config stamp (#306)

2026-07-16 15:27:08 +03:00

backend_cuda.cu

cuda: keep ragged KV resident across decode steps

2026-07-18 18:34:46 +08:00

backend_cuda.h

cuda: keep ragged KV resident across decode steps

2026-07-18 18:34:46 +08:00

backend_loader.c

cuda: keep ragged KV resident across decode steps

2026-07-18 18:34:46 +08:00

backend_metal.h

Metal backend (Apple Silicon): batched experts + fused attention on GPU, unified-memory zero-copy, gated behind COLI_METAL — 2.06 tok/s M5 Max (#72, #87, #103)

2026-07-13 08:57:10 +02:00

backend_metal.mm

Metal backend (Apple Silicon): batched experts + fused attention on GPU, unified-memory zero-copy, gated behind COLI_METAL — 2.06 tok/s M5 Max (#72, #87, #103)

2026-07-13 08:57:10 +02:00

build_cuda.bat

windows-dev: CUDA build script, expert budget + grouped quant research

2026-07-15 02:32:12 -04:00

coli

refactor: split glm.c into colibri.c + 4 header modules

2026-07-19 21:12:04 +08:00

colibri.c

fix: remove duplicate argmax_v — use NaN-safe version from sample.h

2026-07-19 21:15:54 +08:00

compat.h

Merge pull request #352 from woolcoxm/fix/test-stops-windows

2026-07-19 10:42:41 +02:00

decode_batch.h

Unify continuous batching + heterogeneous runtime: decode batching, physical-core planning, disjoint VRAM/RAM placement, topp-policy warning (CPU-validated, CUDA on 6x5090) (#68)

2026-07-13 14:30:36 +02:00

doctor.py

Fix CUDA detection on Windows: cuda_binary()/cuda_linkage() always returned False

2026-07-18 23:18:16 +02:00

download_fp8.py

Windows-dev: grouped quantization + mixed precision + expert budget + download tool

2026-07-15 02:32:12 -04:00

grammar.h

Translate user-facing runtime output to English, machine prefixes preserved, + CLI output test (#67, #85)

2026-07-12 13:38:40 +02:00

iobench.c

Windows: direct I/O (FILE_FLAG_NO_BUFFERING, 1.47x decode) + VirtualLock pin wiring + select_ctx device cache (#162)

2026-07-14 16:12:15 +02:00

json.h

security: harden safetensors/JSON parsers against malicious model files

2026-07-15 08:13:20 +02:00

kv_persist.h

refactor: split glm.c into colibri.c + 4 header modules

2026-07-19 21:12:04 +08:00

Makefile

fix: old x86 Mac scalar -> vnni

2026-07-19 16:40:09 -04:00

olmoe.c

Merge pull request #362 from EgonRuiter/prefetcher-v3

2026-07-19 12:40:43 +02:00

openai_server.py

serve: end-to-end tool-calling regression test + unparsed-marker diagnosis (#401)

2026-07-19 10:56:00 +02:00

quant.h

refactor: split glm.c into colibri.c + 4 header modules

2026-07-19 21:12:04 +08:00

ref_glm.json

colibrì: pure-C GLM-5.2 (744B MoE) engine with disk-streamed experts

2026-07-05 20:57:25 +02:00

ref_olmoe_real.json

feat(prefetch): implement prefetcher v2.1 with lookahead-2, hot pinning, adaptive cache, RMSNorm scaling, and routing EMA

2026-07-15 20:45:37 +02:00

resource_plan.py

plan NUMA interleave on multi-socket Linux

2026-07-18 18:38:40 +08:00

sample.h

fix: remove duplicate argmax_v — use NaN-safe version from sample.h

2026-07-19 21:15:54 +08:00

schema_gbnf.h

SCHEMA=<file.json>: JSON-Schema -> GBNF compiler for grammar-forced drafts (#148)

2026-07-14 16:31:09 +02:00

setup.sh

refactor: split glm.c into colibri.c + 4 header modules

2026-07-19 21:12:04 +08:00

st.h

security: reject malformed model tensors at the untrusted-mirror boundary

2026-07-19 13:06:35 +02:00

telemetry.h

refactor: split glm.c into colibri.c + 4 header modules

2026-07-19 21:12:04 +08:00

tier.h

Unify continuous batching + heterogeneous runtime: decode batching, physical-core planning, disjoint VRAM/RAM placement, topp-policy warning (CPU-validated, CUDA on 6x5090) (#68)

2026-07-13 14:30:36 +02:00

tok_unicode.h

Translate user-facing runtime output to English, machine prefixes preserved, + CLI output test (#67, #85)

2026-07-12 13:38:40 +02:00

tok.h

Don't trust a converted checkpoint's config for the stop set

2026-07-16 17:59:34 +02:00

uring.h

use async and fix workers

2026-07-15 12:42:13 +02:00

version.py

release: version infrastructure + GitHub Release workflow

2026-07-19 05:01:41 +08:00

warmup.ps1

Windows native port: serve-mode pipe fix + RAM detection + POSIX guards, AVX-VNNI kernel, gated CUDA DLL (#131, fixes #123)

2026-07-13 20:54:30 +02:00