colibri

url/colibri

Fork 0

Commit Graph

Author	SHA1	Message	Date
Louie	ec0aadf91a	olmoe: int8 container support (fixes SIGBUS), NEON int8 dot kernels (2.7x on ARM), macOS RSS fix, converter fallback (#187 ) - load_expert_w: detect I8/U8 tensors and read them raw with their .qs scales. Before this, pointing SNAP at a convert_olmoe.py container crashed with SIGBUS (st_read_f32 walks I8 data as 2-byte elements). Container misses now cost half the I/O and skip quantize_rows entirely: 2.08 -> 3.69 tok/s on the miss-heavy 16-slot ref.json run (M5). Raw bf16 checkpoints keep working. - matmul_q: Q8_0-style integer path on ARM (per-16 activation blocks, sdot on dotprod CPUs, vmull fallback). Same IDOT family as glm.c, same semantics: IDOT=0 stays byte-exact vs the oracle (12/12); default integer path can flip an argmax tie (documented in glm.c README). End-to-end on M5: 4.5 -> 12 tok/s warm-cache decode. - rss_gb: ru_maxrss is bytes on macOS, KB on Linux. RSS lines were reading '2029 GB' on Macs. - convert_olmoe.py: snapshot_download(local_files_only=True) raises LocalEntryNotFoundError when the repo is not cached; the download fallback was unreachable. Measured on M5 MacBook (10 cores, 24 GB, macOS 26.5), OLMoE-1B-7B-0125: ref.json greedy 12/12 with IDOT=0 on both container and raw paths. Co-authored-by: x <x@Mac.fritz.box> Co-authored-by: Claude Fable 5 <noreply@anthropic.com>	2026-07-14 15:59:49 +02:00
nalepy	89d95fc73b	Windows 11 native port, phase 1: MinGW-w64 static build, compat shims, setup + docs (#40 ) * fix: Windows port audit fixes — _FILE_OFFSET_BITS guard, O_BINARY st.h, getrusage peak, oracle diagnostic, setup.sh wmic Audit remediation (all MEDIUM issues fixed): - compat.h: compile-time #error if _FILE_OFFSET_BITS < 64 on _WIN32 - compat.h: COMPAT_O_RDONLY macro (O_RDONLY\|O_BINARY on Windows, belt-and-braces) - st.h: use COMPAT_O_RDONLY in both open() call sites (plan §1 row 1) - compat.h: getrusage shim now uses PeakWorkingSetSize (ru_maxrss = peak, not current) - glm.c: oracle mismatch diagnostic — prints position/expected/got on TF failures - setup.sh: replace deprecated wmic with /proc/meminfo (MSYS2 provides it) - .gitignore: .exe, glm_tiny/, olmoe_hf/, olmoe_i4/ LOW issues addressed: - _FILE_OFFSET_BITS guard prevents silent 32-bit off_t wrap at >4GB offsets - coli Windows venv path (Scripts/python.exe) fixed earlier - posix_fadvise do{}while(0) kept intentionally — no caller checks return code Verification: oracle 32/32, 27/27 Python tests, rename EEXIST, >4GB pread offset. docs: add Windows 11 native port section to README - Toolchain: MinGW-w64 (winlibs or MSYS2), GCC 16.1 tested - Build instructions: make glm.exe, tiny oracle verification - Runtime: SNAP=..., coli chat, coli serve all work - Status: Phase 1 complete (compiles, correct, static-linked) - Update platform requirements to include Windows 11 natively	2026-07-11 12:59:49 +02:00

Author

SHA1

Message

Date

Louie

ec0aadf91a

olmoe: int8 container support (fixes SIGBUS), NEON int8 dot kernels (2.7x on ARM), macOS RSS fix, converter fallback (#187 )

- load_expert_w: detect I8/U8 tensors and read them raw with their .qs scales.
  Before this, pointing SNAP at a convert_olmoe.py container crashed with
  SIGBUS (st_read_f32 walks I8 data as 2-byte elements). Container misses now
  cost half the I/O and skip quantize_rows entirely: 2.08 -> 3.69 tok/s on the
  miss-heavy 16-slot ref.json run (M5). Raw bf16 checkpoints keep working.
- matmul_q: Q8_0-style integer path on ARM (per-16 activation blocks, sdot on
  dotprod CPUs, vmull fallback). Same IDOT family as glm.c, same semantics:
  IDOT=0 stays byte-exact vs the oracle (12/12); default integer path can flip
  an argmax tie (documented in glm.c README). End-to-end on M5: 4.5 -> 12 tok/s
  warm-cache decode.
- rss_gb: ru_maxrss is bytes on macOS, KB on Linux. RSS lines were reading
  '2029 GB' on Macs.
- convert_olmoe.py: snapshot_download(local_files_only=True) raises
  LocalEntryNotFoundError when the repo is not cached; the download fallback
  was unreachable.

Measured on M5 MacBook (10 cores, 24 GB, macOS 26.5), OLMoE-1B-7B-0125:
ref.json greedy 12/12 with IDOT=0 on both container and raw paths.

Co-authored-by: x <x@Mac.fritz.box>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>

2026-07-14 15:59:49 +02:00

nalepy

89d95fc73b

Windows 11 native port, phase 1: MinGW-w64 static build, compat shims, setup + docs (#40 )

* fix: Windows port audit fixes — _FILE_OFFSET_BITS guard, O_BINARY st.h, getrusage peak, oracle diagnostic, setup.sh wmic

Audit remediation (all MEDIUM issues fixed):
- compat.h: compile-time #error if _FILE_OFFSET_BITS < 64 on _WIN32
- compat.h: COMPAT_O_RDONLY macro (O_RDONLY|O_BINARY on Windows, belt-and-braces)
- st.h: use COMPAT_O_RDONLY in both open() call sites (plan §1 row 1)
- compat.h: getrusage shim now uses PeakWorkingSetSize (ru_maxrss = peak, not current)
- glm.c: oracle mismatch diagnostic — prints position/expected/got on TF failures
- setup.sh: replace deprecated wmic with /proc/meminfo (MSYS2 provides it)
- .gitignore: *.exe, glm_tiny/, olmoe_hf/, olmoe_i4/

LOW issues addressed:
- _FILE_OFFSET_BITS guard prevents silent 32-bit off_t wrap at >4GB offsets
- coli Windows venv path (Scripts/python.exe) fixed earlier
- posix_fadvise do{}while(0) kept intentionally — no caller checks return code

Verification: oracle 32/32, 27/27 Python tests, rename EEXIST, >4GB pread offset.

* docs: add Windows 11 native port section to README

- Toolchain: MinGW-w64 (winlibs or MSYS2), GCC 16.1 tested
- Build instructions: make glm.exe, tiny oracle verification
- Runtime: SNAP=..., coli chat, coli serve all work
- Status: Phase 1 complete (compiles, correct, static-linked)
- Update platform requirements to include Windows 11 natively

2026-07-11 12:59:49 +02:00

2 Commits