Merge dev into dual-ssd-mirror: resolve #298/#192-era conflicts

Combined resolution: mir_pread/st_prefetch_rep (this PR) now carry dev's
DISK-CLASS accounting unwind (dc_wall_exit) and O_DIRECT prefetch skip
(g_direct); direct-path keeps dc_direct=1 plus the mirror read counters.
Verified: clean build, token-exact tiny models unchanged, test_st_mirror
and test_st_pread pass.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
JustVugg
2026-07-20 21:24:39 +02:00
60 changed files with 8507 additions and 346 deletions
+55
View File
@@ -3,6 +3,12 @@
</p>
<p align="center">
<a href="https://justvugg.github.io/colibri"><img src="https://img.shields.io/badge/website-justvugg.github.io%2Fcolibri-1f6feb" alt="Website"></a>
<a href="https://github.com/JustVugg/colibri/releases"><img src="https://img.shields.io/github/v/release/JustVugg/colibri?color=2ea043" alt="Latest release"></a>
</p>
<p align="center">
<a href="https://justvugg.github.io/colibri"><b>Website</b></a> ·
English · <a href="README.zh-CN.md">简体中文</a> · <a href="README.zh-TW.md">繁體中文</a> · <a href="README.it.md">Italiano</a>
</p>
@@ -44,6 +50,16 @@ brightness is routing heat, and every expert routed in a turn flashes white. Hov
as a 3-D galaxy — 13,260 characterised experts, 1,041 replicated specialists clustering by topic
(poetry, law, Chinese, SQL…). Position is measured routing affinity, not a learned embedding. Drag to spin.</em></p>
## The vision
Frontier models should not be sealed inside datacenters. colibrì exists so that
**anyone curious enough can open one up**: run a 744B-parameter mind on hardware
you already own, watch every expert fire in real time, and change the code that
does it. Not renting intelligence behind an API — *holding* it: probing it,
measuring it, improving it. Every optimisation in this project started with
someone measuring something on their own machine; the engine is deliberately
small enough that the next one can come from you.
## The idea
A 744B Mixture-of-Experts model activates only ~40B parameters per token — and
@@ -61,6 +77,18 @@ So the model doesn't need to *fit* in fast memory — it needs to be **placed**:
at int4) live **on disk** (~370 GB) and are **streamed on demand**, with a
per-layer LRU cache, a learned pinned hot-store, and an optional VRAM tier.
Think of the core algorithm as **a JIT, but for weights**. A compiler JIT never
compiles the whole program — it watches what actually runs and compiles the hot
paths, just in time. colibrì makes the same bet about a 744B parameter space:
parameters are not resident state to be held, they are **data to be staged**
across a heterogeneous storage hierarchy (VRAM / RAM / NVMe), exactly when the
router proves they are needed. Measured routing heat decides which experts earn
which tier, the router runs a layer ahead so prefetch hides the staging latency,
and — like a JIT — the engine learns your workload: the more you run, the hotter
the right experts get. It works because routing has measurable structure (see
the [expert atlas](https://github.com/JustVugg/colibri/issues/175)) — and
structure is cacheable.
The engine is a single C file (`c/glm.c`) plus small headers. No BLAS, no Python
at runtime, no GPU required.
@@ -203,6 +231,13 @@ COLI_MODEL=/nvme/glm52_i4 ./coli doctor # read-only readiness check
The engine at runtime is pure C — python is only used by the one-time converter
and the optional API gateway.
**On Windows?** You don't need to build. Download the
`colibri-<version>-windows-x86_64.zip` from
[Releases](https://github.com/JustVugg/colibri/releases), unzip it, rename
`colibri-*-windows-x86_64.exe``glm.exe` (so the `coli` launcher finds the
engine), install [Python 3](https://www.python.org/downloads/), then run
`coli chat`. Full walkthrough in the [Quick Start guide](docs/quickstart.md#windows).
Prefer a `coli` command on your PATH? From a checkout, `pip install -e .`
registers it (the engine itself still lives in `c/` — this is an editable
install from the clone, not a standalone wheel).
@@ -220,6 +255,18 @@ install from the clone, not a standalone wheel).
| Grammar-forced drafts (structured output) | [docs/grammar-draft.md](docs/grammar-draft.md) |
| Environment variable inventory | [docs/ENVIRONMENT.md](docs/ENVIRONMENT.md) |
## What's next
- **Algorithmic research is active.** The current hierarchy is LRU + a learned
pin set; the next step is under way — smarter placement and scheduling,
overlap of CPU and GPU expert execution, and routing-aware speculation.
Everything lands the way this project always works: measured, reviewed, and
merged in the open.
- **More open models.** The tiering algorithm is model-agnostic: any MoE with
routed experts can be staged the same way. GLM-5.2 and OLMoE run today;
support for more open-weight families — **Kimi K2** (Moonshot AI),
**Qwen3 MoE** (Alibaba), **MiniMax** — is on the roadmap.
## Supporting the project
colibrì started as a one-person project on a 12-core laptop with 25 GB of RAM;
@@ -260,6 +307,14 @@ The hummingbird weighs a few grams, hovers in place, and visits a thousand
flowers a day. This engine keeps a 744-billion-parameter giant alive on
hummingbird rations: 25 GB of RAM, twelve CPU cores, and a lot of disk patience.
## Acknowledgements
colibrì is an engine; the minds it runs are a gift. Thank you to the teams
releasing frontier-class weights in the open — **Z.ai** (GLM), **Moonshot AI**
(Kimi), **Alibaba Qwen**, **MiniMax**, and **Allen AI** (OLMoE) — and to every
contributor who benchmarked, bisected, replicated an atlas run, or sent a patch.
This project is proof of what open weights make possible.
## License
Apache 2.0. GLM-5.2 weights are released by Z.ai under MIT.