diff --git a/README.md b/README.md index 4109d5f..86c3009 100644 --- a/README.md +++ b/README.md @@ -17,6 +17,32 @@ $ ./coli chat ◆ Ciao! 😊 Come posso aiutarti oggi? ``` + +## See it running + +

+ colibrì web dashboard — live metrics, hardware panel, expert tiers +

+

The web dashboard (./coli web): a 744B model answering at 4+ tok/s end-to-end on 6× RTX 5090 — +with live token metrics, the hardware panel, and the VRAM/RAM/disk expert tiers.

+ +

+ the Brain page — 19,456 experts as a live cortex +

+

The Brain page: all 19,456 experts as a living cortex — colour is the storage tier, +brightness is routing heat, and every expert routed in a turn flashes white. Hovering shows the expert's +measured topic affinity.

+ +## Contents + +- [The idea](#the-idea) +- [See it running](#see-it-running) +- [What's implemented](#whats-implemented) +- [Honest numbers](#honest-numbers-wsl2-12-cores-25-gb-ram-nvme-via-vhdx) +- [Download the model](#download-the-model) +- [Web dashboard](#web-dashboard) +- [Got a better machine?](#got-a-better-machine-try-it--heres-what-to-expect) + ## The idea A 744B Mixture-of-Experts model activates only ~40B parameters per token — and only ~11 GB of those change from token to token (the routed experts). So: @@ -501,6 +527,23 @@ thrashing. Persistent `.coli_usage` remains the long-term signal and is not deca **Conversations reopen warm** (`.coli_kv`, since 2026-07-10): `coli chat` persists the compressed MLA KV-cache to disk after every turn (~182 KB/token, appended incrementally, crash-safe). Close the chat, reopen it tomorrow — the model still remembers the whole conversation and **zero re-prefill happens**: validated byte-identical to an uninterrupted session. `:reset` clears it, `KVSAVE=0` disables it. +## Web dashboard + +One command serves the OpenAI-compatible API **and** the web console on the same port, then opens your browser when the engine is ready: + +```bash +cd web && npm install && npm run build # once +./coli web --model +``` + +What you get: + +- **Chat** with live metrics: a flashing token counter while generating, then tok/s, time-to-first-token, prompt→completion counts and queue wait; +- **Runtime panel**: your hardware (CPU, GPUs + VRAM, RAM, cores), the scheduler, and the live expert-tier bar — how many of the 19,456 experts sit in VRAM / RAM / disk right now; +- **Brain**: the whole model as a 76×256 cortex, one cell per expert. Colour = tier, brightness = routing heat, and the experts routed in each turn flash white and decay — you watch the model think. Hover any cell for its tier, heat and [measured topic affinity](https://github.com/JustVugg/colibri/issues/175) (specialists for code, Chinese, math, law… live in layers 11–22). + +The dashboard talks to the engine over two tiny protocol lines (`TIERS`, `EMAP`/`HITS`) and plain JSON endpoints — nothing heavier than the engine itself. + ## Got a better machine? Try it — here's what to expect colibrì was built on deliberately humble hardware (12 cores, 25 GB RAM, an older DRAM-less NVMe behind a WSL2 VHDX that measured ~1 GB/s random on *this* drive — note WSL2 VHDX is not inherently slow: a community 5090 box measured 10.5 GB/s O_DIRECT through one, [#101](https://github.com/JustVugg/colibri/issues/101)). **Every one of those constraints is a knob your machine can turn up.** The engine needs: Linux (or WSL2), macOS, or **Windows 11 natively (MinGW-w64)**; gcc with OpenMP, AVX2, ≥16 GB RAM, and the ~370 GB int4 model on a local NVMe (ext4/NTFS — never a network/9p mount). diff --git a/docs/media/colibri-brain.png b/docs/media/colibri-brain.png new file mode 100644 index 0000000..0cd6c90 Binary files /dev/null and b/docs/media/colibri-brain.png differ diff --git a/docs/media/colibri-dashboard.png b/docs/media/colibri-dashboard.png new file mode 100644 index 0000000..123fd3e Binary files /dev/null and b/docs/media/colibri-dashboard.png differ