README: dashboard screenshots + contents index + Web dashboard section (#202)
- 'See it running' right after the intro: the live dashboard (744B answering at 4+ tok/s end-to-end with metrics and tier bar) and the Brain cortex (19,456 experts, tier colour, routing heat, per-turn firing) — three seconds to understand what colibri is. - A short table of contents for the long read below. - 'Web dashboard' section: coli web one-liner, what each panel shows, link to the Expert Atlas (#175). Existing content untouched.
This commit is contained in:
@@ -17,6 +17,32 @@ $ ./coli chat
|
||||
◆ Ciao! 😊 Come posso aiutarti oggi?
|
||||
```
|
||||
|
||||
|
||||
## See it running
|
||||
|
||||
<p align="center">
|
||||
<img src="docs/media/colibri-dashboard.png" width="900" alt="colibrì web dashboard — live metrics, hardware panel, expert tiers">
|
||||
</p>
|
||||
<p align="center"><em>The web dashboard (<code>./coli web</code>): a 744B model answering at 4+ tok/s end-to-end on 6× RTX 5090 —
|
||||
with live token metrics, the hardware panel, and the VRAM/RAM/disk expert tiers.</em></p>
|
||||
|
||||
<p align="center">
|
||||
<img src="docs/media/colibri-brain.png" width="900" alt="the Brain page — 19,456 experts as a live cortex">
|
||||
</p>
|
||||
<p align="center"><em>The <strong>Brain</strong> page: all 19,456 experts as a living cortex — colour is the storage tier,
|
||||
brightness is routing heat, and every expert routed in a turn flashes white. Hovering shows the expert's
|
||||
<a href="https://github.com/JustVugg/colibri/issues/175">measured topic affinity</a>.</em></p>
|
||||
|
||||
## Contents
|
||||
|
||||
- [The idea](#the-idea)
|
||||
- [See it running](#see-it-running)
|
||||
- [What's implemented](#whats-implemented)
|
||||
- [Honest numbers](#honest-numbers-wsl2-12-cores-25-gb-ram-nvme-via-vhdx)
|
||||
- [Download the model](#download-the-model)
|
||||
- [Web dashboard](#web-dashboard)
|
||||
- [Got a better machine?](#got-a-better-machine-try-it--heres-what-to-expect)
|
||||
|
||||
## The idea
|
||||
|
||||
A 744B Mixture-of-Experts model activates only ~40B parameters per token — and only ~11 GB of those change from token to token (the routed experts). So:
|
||||
@@ -501,6 +527,23 @@ thrashing. Persistent `.coli_usage` remains the long-term signal and is not deca
|
||||
|
||||
**Conversations reopen warm** (`.coli_kv`, since 2026-07-10): `coli chat` persists the compressed MLA KV-cache to disk after every turn (~182 KB/token, appended incrementally, crash-safe). Close the chat, reopen it tomorrow — the model still remembers the whole conversation and **zero re-prefill happens**: validated byte-identical to an uninterrupted session. `:reset` clears it, `KVSAVE=0` disables it.
|
||||
|
||||
## Web dashboard
|
||||
|
||||
One command serves the OpenAI-compatible API **and** the web console on the same port, then opens your browser when the engine is ready:
|
||||
|
||||
```bash
|
||||
cd web && npm install && npm run build # once
|
||||
./coli web --model <model-dir>
|
||||
```
|
||||
|
||||
What you get:
|
||||
|
||||
- **Chat** with live metrics: a flashing token counter while generating, then tok/s, time-to-first-token, prompt→completion counts and queue wait;
|
||||
- **Runtime panel**: your hardware (CPU, GPUs + VRAM, RAM, cores), the scheduler, and the live expert-tier bar — how many of the 19,456 experts sit in VRAM / RAM / disk right now;
|
||||
- **Brain**: the whole model as a 76×256 cortex, one cell per expert. Colour = tier, brightness = routing heat, and the experts routed in each turn flash white and decay — you watch the model think. Hover any cell for its tier, heat and [measured topic affinity](https://github.com/JustVugg/colibri/issues/175) (specialists for code, Chinese, math, law… live in layers 11–22).
|
||||
|
||||
The dashboard talks to the engine over two tiny protocol lines (`TIERS`, `EMAP`/`HITS`) and plain JSON endpoints — nothing heavier than the engine itself.
|
||||
|
||||
## Got a better machine? Try it — here's what to expect
|
||||
|
||||
colibrì was built on deliberately humble hardware (12 cores, 25 GB RAM, an older DRAM-less NVMe behind a WSL2 VHDX that measured ~1 GB/s random on *this* drive — note WSL2 VHDX is not inherently slow: a community 5090 box measured 10.5 GB/s O_DIRECT through one, [#101](https://github.com/JustVugg/colibri/issues/101)). **Every one of those constraints is a knob your machine can turn up.** The engine needs: Linux (or WSL2), macOS, or **Windows 11 natively (MinGW-w64)**; gcc with OpenMP, AVX2, ≥16 GB RAM, and the ~370 GB int4 model on a local NVMe (ext4/NTFS — never a network/9p mount).
|
||||
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 421 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 545 KiB |
Reference in New Issue
Block a user