Vincenzo Fornaro e16ec68168 Merge #469: COLI_MODEL_DIRS — split model shards across N drives (capacity aggregation) ...

Search-path of extra directories, each holding a distinct subset of the .safetensors shards (no duplication). Each shard lives on one drive; demand preads hit whichever drive holds it, so concurrent expert loads parallelise across drives and combined capacity is used — a 400 GB container fits across two 250 GB drives the mirror (#421) can't. st_init kept as a back-compat wrapper. Verified locally: clean build, no regression on the default path (token-exact tiny models unchanged), and the split path itself validated token-exact (shard on a separate dir, metadata in primary -> 32/32 vs oracle). Composable with the #421 mirror. CI 8/8. Thanks @mohamedmastouri2000-boop.

2026-07-21 00:57:16 +02:00

..

scripts

Translate user-facing runtime output to English, machine prefixes preserved, + CLI output test (#67, #85)

2026-07-12 13:38:40 +02:00

tests

fix(ci): remove accidentally-committed test_st_pread binary

2026-07-20 21:58:15 +02:00

tools

Merge #458: fmt=5 index codec tools — deployable bytes for the E8/IQ3 container

2026-07-20 21:07:39 +02:00

.gitignore

fix(ci): remove accidentally-committed test_st_pread binary

2026-07-20 21:58:15 +02:00

backend_cuda.cu

fix(rebase): adapt to dev's split tensor_upload/_g API + fault-injection hook

2026-07-20 13:15:19 -04:00

backend_cuda.h

fix(rebase): adapt to dev's split tensor_upload/_g API + fault-injection hook

2026-07-20 13:15:19 -04:00

backend_loader.c

fix(rebase): adapt to dev's split tensor_upload/_g API + fault-injection hook

2026-07-20 13:15:19 -04:00

backend_metal.h

Merge pull request #426 from rgbkrk/quod/metal-residency-set-dev

2026-07-20 18:22:25 +02:00

backend_metal.mm

Merge pull request #426 from rgbkrk/quod/metal-residency-set-dev

2026-07-20 18:22:25 +02:00

build_cuda.bat

windows-dev: CUDA build script, expert budget + grouped quant research

2026-07-15 02:32:12 -04:00

coli

fix(coli): honour explicit COLI_CUDA=0 over the Windows bare-run auto-enable

2026-07-21 01:44:48 +03:00

colibri.c

feat(st): COLI_MODEL_DIRS - split model shards across N drives, no duplication

2026-07-21 01:42:04 +03:00

compat.h

Merge pull request #352 from woolcoxm/fix/test-stops-windows

2026-07-19 10:42:41 +02:00

decode_batch.h

serve stage 2: per-request grammars end-to-end — response_format -> SUBMIT -> grammar-forced drafts in the multiplexed server

2026-07-20 17:27:33 +01:00

doctor.py

Fix CUDA detection on Windows: cuda_binary()/cuda_linkage() always returned False

2026-07-18 23:18:16 +02:00

download_fp8.py

Windows-dev: grouped quantization + mixed precision + expert budget + download tool

2026-07-15 02:32:12 -04:00

grammar.h

Translate user-facing runtime output to English, machine prefixes preserved, + CLI output test (#67, #85)

2026-07-12 13:38:40 +02:00

iobench.c

Windows: direct I/O (FILE_FLAG_NO_BUFFERING, 1.47x decode) + VirtualLock pin wiring + select_ctx device cache (#162)

2026-07-14 16:12:15 +02:00

json.h

security: harden safetensors/JSON parsers against malicious model files

2026-07-15 08:13:20 +02:00

kv_persist.h

refactor: split glm.c into colibri.c + 4 header modules

2026-07-19 21:12:04 +08:00

Makefile

Merge dev into dual-ssd-mirror: resolve #298/#192-era conflicts

2026-07-20 21:24:39 +02:00

olmoe.c

Merge pull request #362 from EgonRuiter/prefetcher-v3

2026-07-19 12:40:43 +02:00

openai_server.py

review: docs section, 1 MiB grammar pre-check, negative tests, measured compile overhead

2026-07-20 17:27:33 +01:00

quant.h

int3-g64 (fmt=5): per-group-scale 3-bit weight format — engine, converter, tests

2026-07-20 17:31:05 +01:00

ref_glm.json

colibrì: pure-C GLM-5.2 (744B MoE) engine with disk-streamed experts

2026-07-05 20:57:25 +02:00

ref_olmoe_real.json

feat(prefetch): implement prefetcher v2.1 with lookahead-2, hot pinning, adaptive cache, RMSNorm scaling, and routing EMA

2026-07-15 20:45:37 +02:00

resource_plan.py

fix(resource_plan): take last two lscpu fields, not [1]/[2] (#332 review)

2026-07-20 10:56:54 -04:00

sample.h

Merge remote-tracking branch 'origin/dev' into pr437

2026-07-20 17:04:18 +02:00

schema_gbnf.h

SCHEMA=<file.json>: JSON-Schema -> GBNF compiler for grammar-forced drafts (#148)

2026-07-14 16:31:09 +02:00

setup.sh

refactor: split glm.c into colibri.c + 4 header modules

2026-07-19 21:12:04 +08:00

st.h

feat(st): COLI_MODEL_DIRS - split model shards across N drives, no duplication

2026-07-21 01:42:04 +03:00

telemetry.h

refactor: split glm.c into colibri.c + 4 header modules

2026-07-19 21:12:04 +08:00

tier.h

Unify continuous batching + heterogeneous runtime: decode batching, physical-core planning, disjoint VRAM/RAM placement, topp-policy warning (CPU-validated, CUDA on 6x5090) (#68)

2026-07-13 14:30:36 +02:00

tok_unicode_o200k.h

tok: o200k pre-tokenizer support, auto-detected from tokenizer.json

2026-07-16 15:14:50 -04:00

tok_unicode.h

Translate user-facing runtime output to English, machine prefixes preserved, + CLI output test (#67, #85)

2026-07-12 13:38:40 +02:00

tok.h

tok: o200k pre-tokenizer support, auto-detected from tokenizer.json

2026-07-16 15:14:50 -04:00

uring.h

use async and fix workers

2026-07-15 12:42:13 +02:00

version.py

release: version infrastructure + GitHub Release workflow

2026-07-19 05:01:41 +08:00

warmup.ps1

Windows native port: serve-mode pipe fix + RAM detection + POSIX guards, AVX-VNNI kernel, gated CUDA DLL (#131, fixes #123)

2026-07-13 20:54:30 +02:00