woolcoxm 37c96ee07a bench: add bench_topp -- head-to-head old-qsort vs new-heap timing on V=151936 (#335) ...

test_topp proves correctness; bench_topp measures the latency claim. It re-implements
the OLD dist_build (full-vocab qsort) inline on a private buffer and times it against
the REAL new dist_build over identical inputs in one process: V=151936, temp=0.7,
3 shapes (realistic / uniform / plateau) x 4 nuc values (0.5/0.9/0.95/0.99), 2000
timed reps each, median reported. Deliberately NOT in TEST_BINS -- it's a microbench,
not a gate. Build on demand: make tests/bench_topp && ./tests/bench_topp

2026-07-17 09:22:26 -04:00

..

scripts

Translate user-facing runtime output to English, machine prefixes preserved, + CLI output test (#67, #85)

2026-07-12 13:38:40 +02:00

tests

bench: add bench_topp -- head-to-head old-qsort vs new-heap timing on V=151936 (#335)

2026-07-17 09:22:26 -04:00

tools

Merge pull request #322 from woolcoxm/fix/oracle-transformers-pin

2026-07-16 19:46:20 +02:00

.gitignore

Fix Windows cuda-dll build: quote NVCC, gate MSVC warnings, add config stamp (#306)

2026-07-16 15:27:08 +03:00

backend_cuda.cu

GPU resident pipeline: batch CUDA attention, head-sharded kv_b, prefill expert groups, W4A16 mixed dispatch (#111)

2026-07-14 18:18:05 +02:00

backend_cuda.h

cuda(win): export + load the #111 pipeline ABI — 24 missing DLLEXPORT decorations + loader wrappers, unbreaks CUDA_DLL=1 (#201)

2026-07-14 18:43:56 +02:00

backend_loader.c

security(win): load coli_cuda.dll by absolute path, never from the CWD (DLL hijack)

2026-07-15 08:16:20 +02:00

backend_metal.h

Metal backend (Apple Silicon): batched experts + fused attention on GPU, unified-memory zero-copy, gated behind COLI_METAL — 2.06 tok/s M5 Max (#72, #87, #103)

2026-07-13 08:57:10 +02:00

backend_metal.mm

Metal backend (Apple Silicon): batched experts + fused attention on GPU, unified-memory zero-copy, gated behind COLI_METAL — 2.06 tok/s M5 Max (#72, #87, #103)

2026-07-13 08:57:10 +02:00

build_cuda.bat

windows-dev: CUDA build script, expert budget + grouped quant research

2026-07-15 02:32:12 -04:00

coli

coli: serve pidfile + coli stop — one command to shut engine and server down

2026-07-17 13:27:02 +02:00

compat.h

windows: PIPE default ON + compat_fadvise WILLNEED cache-warmer (pread path)

2026-07-15 06:14:15 -04:00

decode_batch.h

Unify continuous batching + heterogeneous runtime: decode batching, physical-core planning, disjoint VRAM/RAM placement, topp-policy warning (CPU-validated, CUDA on 6x5090) (#68)

2026-07-13 14:30:36 +02:00

doctor.py

config: fix UTF-8 decode crash + null config-value TypeError on Windows (doctor/resource_plan) (#184)

2026-07-14 14:17:42 +02:00

download_fp8.py

Windows-dev: grouped quantization + mixed precision + expert budget + download tool

2026-07-15 02:32:12 -04:00

glm.c

sampling: partial top-p select in dist_build — O(V) heapify + k pops, not O(V log V) qsort (#335)

2026-07-17 09:01:23 -04:00

grammar.h

Translate user-facing runtime output to English, machine prefixes preserved, + CLI output test (#67, #85)

2026-07-12 13:38:40 +02:00

iobench.c

Windows: direct I/O (FILE_FLAG_NO_BUFFERING, 1.47x decode) + VirtualLock pin wiring + select_ctx device cache (#162)

2026-07-14 16:12:15 +02:00

json.h

security: harden safetensors/JSON parsers against malicious model files

2026-07-15 08:13:20 +02:00

Makefile

bench: add bench_topp -- head-to-head old-qsort vs new-heap timing on V=151936 (#335)

2026-07-17 09:22:26 -04:00

olmoe.c

olmoe: PPL=1 teacher-forced perplexity mode (loss meter for throughput experiments)

2026-07-16 13:10:50 -03:00

openai_server.py

Profiling page: per-turn phase timings, live in the web dashboard

2026-07-16 08:56:12 -04:00

ref_glm.json

colibrì: pure-C GLM-5.2 (744B MoE) engine with disk-streamed experts

2026-07-05 20:57:25 +02:00

resource_plan.py

coli: measured Windows launcher defaults — OMP tuning parity + DIRECT/PIPE/PILOT_REAL

2026-07-15 12:06:48 -05:00

schema_gbnf.h

SCHEMA=<file.json>: JSON-Schema -> GBNF compiler for grammar-forced drafts (#148)

2026-07-14 16:31:09 +02:00

setup.sh

Translate user-facing runtime output to English, machine prefixes preserved, + CLI output test (#67, #85)

2026-07-12 13:38:40 +02:00

st.h

security: harden safetensors/JSON parsers against malicious model files

2026-07-15 08:13:20 +02:00

tier.h

Unify continuous batching + heterogeneous runtime: decode batching, physical-core planning, disjoint VRAM/RAM placement, topp-policy warning (CPU-validated, CUDA on 6x5090) (#68)

2026-07-13 14:30:36 +02:00

tok_unicode.h

Translate user-facing runtime output to English, machine prefixes preserved, + CLI output test (#67, #85)

2026-07-12 13:38:40 +02:00

tok.h

Don't trust a converted checkpoint's config for the stop set

2026-07-16 17:59:34 +02:00

uring.h

use async and fix workers

2026-07-15 12:42:13 +02:00

warmup.ps1

Windows native port: serve-mode pipe fix + RAM detection + POSIX guards, AVX-VNNI kernel, gated CUDA DLL (#131, fixes #123)

2026-07-13 20:54:30 +02:00