url/colibri
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix: old x86 Mac scalar -> vnni

Browse Source 
Older macs still need -march, not -mcpu.

Before this fix, no ymm (or zmm, vnni, etc.).

Standalone repro:

```
% clang -O3  -mcpu=native glm.c -o /tmp/glm
clang: error: unsupported option '-mcpu=' for target 'x86_64-apple-darwin25.4.0'
% clang -O3  -mcpu=native glm.c -o /tmp/glm -lm
% otool -tv /tmp/glm | grep -c ymm
0
```

I'm not entirely sure why `-lm` would ignore `-mcpu=` rather than error, but either way we don't get vector instructions:

colibri itself:

```
% ulimit -l unlimited; OMP_NUM_THREADS=16 RAM_GB=700 PIN=auto PIN_GB=all PIN_FILL=1 ./coli run --ram 700 --model ~/projects/llms/glm5p2-i4 --ngen 256 "Explain the difference between concurrency and parallelism"
...
== GLM C engine (glm_moe_dsa), cache=8 experts/layer | experts@8-bit dense@8-bit | idot: scalar ==
```

After the fix, we'll get it:

```
% clang -O3  -march=native glm.c -o /tmp/glm -lm
% otool -tv /tmp/glm | grep -c ymm
2621
```

colibri run:
```
% ulimit -l unlimited; OMP_NUM_THREADS=16 RAM_GB=700 PIN=auto PIN_GB=all PIN_FILL=1 ./coli run --ram 700 --model ~/projects/llms/glm5p2-i4 --ngen 256 "Explain the difference between concurrency and parallelism"
...
== GLM C engine (glm_moe_dsa), cache=8 experts/layer | experts@8-bit dense@8-bit | idot: avx512-vnni ==
```
This commit is contained in:
Oleksandr Kuvshynov
2026-07-19 16:06:49 -04:00
parent 61004dcb84
commit 5c84b25645
1 changed files with 7 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files
c/Makefile
+7 -1
View File
@@ -58,9 +58,14 @@ CFLAGS = -O3 $(OMPC) -Wall -Wextra -Wno-unused-parameter -Wno-misleading-indent
# Opt-in: ARCH=native appends -mcpu=native (arm64 clang uses -mcpu, not -march), # Opt-in: ARCH=native appends -mcpu=native (arm64 clang uses -mcpu, not -march),
# which unlocks the i8mm SMMLA int8/int4 dot kernels in colibri.c. ARCH unset -> # which unlocks the i8mm SMMLA int8/int4 dot kernels in colibri.c. ARCH unset ->
# no -mcpu, default build byte-identical. Apple clang knows apple-m4 / native. # no -mcpu, default build byte-identical. Apple clang knows apple-m4 / native.
# For older X86_64 Macs (for example, Mac Pro 2019) we need to use -march
ifneq ($(ARCH),) ifneq ($(ARCH),)
ifneq (,$(X86_64))
CFLAGS += -march=$(ARCH)
else
CFLAGS += -mcpu=$(ARCH) CFLAGS += -mcpu=$(ARCH)
endif endif
endif
LDFLAGS = -lm $(OMPL) LDFLAGS = -lm $(OMPL)
EXE = EXE =
else ifneq ($(IS_WIN),) else ifneq ($(IS_WIN),)
@@ -273,8 +278,9 @@ olmoe$(EXE): olmoe.c st.h json.h compat.h
# Use a baseline that matches the compiler target. macOS already targets a # Use a baseline that matches the compiler target. macOS already targets a
# portable baseline when ARCH is empty; forcing the x86 value there breaks # portable baseline when ARCH is empty; forcing the x86 value there breaks
# Apple Silicon. Unknown targets use native rather than an invalid x86 flag. # Apple Silicon. Unknown targets use native rather than an invalid x86 flag.
# Intel Macs need -march for vector instructions
ifneq (,$(DARWIN)) ifneq (,$(DARWIN))
PORTABLE_ARCH = PORTABLE_ARCH = $(if $(X86_64),x86-64-v3,)
else ifneq (,$(AARCH64)) else ifneq (,$(AARCH64))
PORTABLE_ARCH = armv8-a PORTABLE_ARCH = armv8-a
else ifneq (,$(PPC64)) else ifneq (,$(PPC64))