colibri

Files

T

Marquez 62419af188 routing: opt-in CACHE_ROUTE max-rank prefer (pin∪LRU), default off — 2.4->3.33 tok/s on GB10 (#199 )

Paper-style cache-aware MoE selection (arXiv:2412.00099 max-rank):
keep true top-J always; fill remaining K slots preferring experts already
resident in pin∪LRU within top-M. Default OFF so stock full top-K is
unchanged.

Env: CACHE_ROUTE, ROUTE_J/M/P/ALPHA, ROUTE_AGREE (auto-on with CACHE_ROUTE).
Telemetry: swap%/route_swaps/route_slots, route_agree, route_kl on footer
and serve STAT. Complementary to PILOT (prefetch vs selection change).

Routing-only PR for clean A/B vs PILOT / #119; no CUDA/fuse stack.
See docs/CACHE_ROUTE.md. Closes nothing; for #161 discussion.

Co-authored-by: Vincent Marquez <vincentmarquez405@gmail.com>

2026-07-14 22:08:28 +02:00

experiments

docs: GLM-5.2 6x RTX 5090 full-residency experiment — 6.84 tok/s, honest dead-ends log (#94 )

2026-07-12 22:06:51 +02:00

media

README: dashboard screenshots + contents index + Web dashboard section (#202 )

2026-07-14 18:45:42 +02:00

CACHE_ROUTE.md

routing: opt-in CACHE_ROUTE max-rank prefer (pin∪LRU), default off — 2.4->3.33 tok/s on GB10 (#199 )

2026-07-14 22:08:28 +02:00

ENVIRONMENT.md

routing: opt-in CACHE_ROUTE max-rank prefer (pin∪LRU), default off — 2.4->3.33 tok/s on GB10 (#199 )