From c6801451bdf1a9ad8b88043ac2963b80b8bca286 Mon Sep 17 00:00:00 2001 From: JustVugg Date: Mon, 20 Jul 2026 17:43:46 +0200 Subject: [PATCH] site: bring the official website + Pages deploy workflow to main (#424) The site.yml workflow deploys on push to main, so the site only goes live once site/ is on main. Bring site/index.html + icons + the workflow so a GitHub Pages deploy (Settings -> Pages -> Source: GitHub Actions) publishes it at justvugg.github.io/colibri. Refs #424 Co-Authored-By: Claude Fable 5 --- .github/workflows/site.yml | 36 ++ site/colibri-icon.svg | 50 +++ site/colibri.svg | 55 +++ site/index.html | 698 +++++++++++++++++++++++++++++++++++++ 4 files changed, 839 insertions(+) create mode 100644 .github/workflows/site.yml create mode 100644 site/colibri-icon.svg create mode 100644 site/colibri.svg create mode 100644 site/index.html diff --git a/.github/workflows/site.yml b/.github/workflows/site.yml new file mode 100644 index 0000000..160cd61 --- /dev/null +++ b/.github/workflows/site.yml @@ -0,0 +1,36 @@ +name: Deploy website + +# Publishes site/ to GitHub Pages. One-time repo setup: +# Settings → Pages → Build and deployment → Source: "GitHub Actions". +# Custom domain later: add site/CNAME with the bare domain, point DNS +# (A/AAAA to GitHub Pages IPs or CNAME to .github.io), done. + +on: + push: + branches: [main] + paths: ['site/**', '.github/workflows/site.yml'] + workflow_dispatch: + +permissions: + contents: read + pages: write + id-token: write + +concurrency: + group: pages + cancel-in-progress: true + +jobs: + deploy: + runs-on: ubuntu-latest + environment: + name: github-pages + url: ${{ steps.deployment.outputs.page_url }} + steps: + - uses: actions/checkout@v4 + - uses: actions/configure-pages@v5 + - uses: actions/upload-pages-artifact@v3 + with: + path: site + - id: deployment + uses: actions/deploy-pages@v4 diff --git a/site/colibri-icon.svg b/site/colibri-icon.svg new file mode 100644 index 0000000..76838b9 --- /dev/null +++ b/site/colibri-icon.svg @@ -0,0 +1,50 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/site/colibri.svg b/site/colibri.svg new file mode 100644 index 0000000..4aed0bb --- /dev/null +++ b/site/colibri.svg @@ -0,0 +1,55 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + colibrì + tiny engine, immense model + GLM-5.2 · 744B MoE · int4 · streaming CPU + diff --git a/site/index.html b/site/index.html new file mode 100644 index 0000000..dd6dac1 --- /dev/null +++ b/site/index.html @@ -0,0 +1,698 @@ + + + + + +colibrì — tiny engine, immense model + + + + + + + + + + + + +
+ +
tiny engine · immense model
+

colibrì

+
a 744-billion-parameter mind, on your desk
+

Run GLM-5.2 — 744B Mixture-of-Experts — on hardware you already own. + Pure C, zero dependencies, experts staged just-in-time across VRAM, RAM and disk. + The galaxy behind this text is real: the model's measured expert atlas.

+
+ ★ Star on GitHub + Quick start → +
+
+
744B
parameters
+
19,456
experts
+
0
dependencies
+
25 GB
min RAM
+
9.2
tok/s measured
+
+
scroll
+
+ +
+ +
+
the vision
+

Frontier models should not be sealed inside datacenters.
+ Anyone curious enough should be able to open one up —
+ run it, watch every expert fire, and make it better.

+
+
⚡

Run it

+

A 744B frontier-class MoE answering on a 25 GB machine — the proven floor — up to + 9+ tok/s on a workstation. No cluster, no cloud bill, no API key.

+
🔬

Study it

+

The engine is a microscope: live routing telemetry, per-expert heat, and the first + published expert atlas of a 700B-class model — measured on machines like yours.

+
🛠

Improve it

+

One C file, readable in an afternoon. Every optimisation this project found came from + someone measuring something — the next one can come from you.

+
+
+ +
+
01 — the algorithm
+

A JIT, but for weights

+
+
+

A compiler JIT never compiles the whole program — it watches what actually runs + and compiles the hot paths, just in time. colibrì applies the same bet to a + 744B parameter space: a token activates only ~40B parameters, and just + ~11 GB of those change from token to token — the routed experts.

+
So parameters are not resident state to be held — they are + data to be staged, exactly when the router proves they are needed.
+

The engine keeps the dense weights resident and treats the 19,456 experts as a + heterogeneous storage hierarchy: measured routing heat decides which experts earn + VRAM, which earn pinned RAM, and which stream from NVMe. The router runs a layer + ahead, so prefetch hides the staging latency — and because routing has + measurable structure (the atlas below), the hierarchy keeps getting the answer right. + Like a JIT, it learns your workload: the more you run, the hotter the right experts get.

+

Everything is validated token-exact against the reference transformers + implementation — speed never buys drift.

+
⚗ Active research: the next algorithmic step is under way — + smarter placement and scheduling beyond LRU + pin, overlap of CPU and GPU expert execution, + and routing-aware speculation. Measured, reviewed, and merged in the open.
+
+
+
+ VRAMhottest experts · grouped GPU matmuls · + six 5090s hold the entire routed set — disk reads hit zero
+
↑ promote by measured heat ↓ demote
+
+ RAMwarm set · pinned & wired, learned from + your usage · AVX-512/NEON int4 kernels + LRU behind it
+
↑ promote ↓ demote
+
+ NVMecold tail · streamed on demand with io_uring, + prefetched a layer ahead — how 744B fits a 25 GB machine at all
+
token → router picks 8 of 256 experts × 75 layers → + gather across tiers → int4 matmul → next token
+
+
+
+ +
+
02 — the engine, live
+

Watch it think

+

A replay of the engine at work, paced to measured decode speeds from real + community hardware. The grid is all 19,456 experts (colour = storage tier, brightness = + routing heat, routed = white flash); the galaxy below it is the measured atlas — + watch different questions light different regions of the mind.

+
+
+
+
+ coli chat —
+
+
+
+
+ the brain — 75 MoE layers × 256 experts + MTP
+
+ +
+
0.0 tok/s
decode
+
—
ttft (measured)
+
—
resident hit
+
+
+ VRAM tier + RAM tier + disk tier + routed now +
+
+
+
+
+ the atlas, live — measured specialists routed by the current topic
+
+ +
routing: —
+
+
+
+

Simulation replays a fixed transcript at each profile's measured decode rate + (#82, #389, + #387, #273); + “—” means that machine's report did not publish the number — nothing here is invented. + Atlas flashes sample real measured affinity (atlas v1) for the active topic.

+
+ +
+
+
03 — the atlas
+

Measured, not assumed

+

Every point below is a real measured expert from the published + canonical atlas v1: + 721 canonical specialists confirmed across three independent kernel families + (AVX-512 VNNI · NEON · AVX2, engine-pinned, replication-gated) plus 637 gate-sensitive ones. + Position is the measured 10-topic affinity vector — a specialist sits at its topic's anchor, + a generalist drifts to the centre. Colour = top topic, large points = canonical. Drag to spin.

+
+
+ +
+
drag to spin · scroll past to release
+
+
+
+ +
+
04 — the ladder
+

Same engine, any hardware

+

Measured decode on real community machines — the hardware only changes + where the experts live. Every row links to its public report.

+ + + + + + + + + + + + + + + + + + + + +
hardwaredecodewhere the experts live
6× RTX 5090 · full residency + selective NUMA (#82)9.0–9.2 tok/s
all in VRAM+RAM · disk 0
2× Xeon Gold 6430 · 1 TB DDR5 · CPU-only (#389)5.42 tok/s
every expert pinned in RAM
DGX Spark GB10 · 121 GB unified (#161)3.33 tok/s
unified memory + CACHE_ROUTE
MacBook Pro M5 Max · 128 GB · Metal (#387)2.0 tok/s
Metal tier + warm pin
Ryzen AI Max+ 395 · 128 GB (#200)1.83 tok/s
hot set pinned in RAM
Ryzen 9 9950X3D2 · RTX 5090 · Gen5 NVMe (#120)1.23 tok/s
28 GB VRAM tier + 11.5 GB/s disk
single RTX 5070 Ti laptop-class (#273)1.07 tok/s
GPU-resident pipeline
25 GB dev box · cold — the proven floor0.05–0.1 tok/s
streamed from NVMe
+

…and 15+ more community machines, from an M4 Pro Mac Mini to a 430 GB EPYC server. + Full tables, methodology and quality ablations: docs/benchmarks.md

+
+ +
+
05 — the models
+

One engine, more minds coming

+

The tiering algorithm is model-agnostic: any Mixture-of-Experts with routed + experts can be staged the same way. GLM-5.2 is live today; support for more open-weight + families is on the roadmap.

+
+
live

GLM-5.2

Z.ai · 744B MoE
+

The flagship target: int4 container, token-exact vs reference, full atlas published.

+
live

OLMoE

Allen AI · 7B MoE
+

The small research workhorse — quantization A/Bs and quality ablations run here first.

+
planned

Kimi K2

Moonshot AI · 1T MoE
+

The next scale step: a trillion-parameter mind on the same tiered engine.

+
planned

Qwen3 MoE

Alibaba Qwen
+

The most widely-deployed open family — broad hardware coverage meets broad adoption.

+
planned

MiniMax

MiniMax · MoE
+

Long-context specialists — a different routing profile for the atlas to map.

+
+
+

Thank you to the teams whose open weights make this project possible — + Z.ai (GLM), Moonshot AI (Kimi), Alibaba Qwen, MiniMax, + Allen AI (OLMoE), and everyone releasing frontier-class models in the open. + An engine is nothing without a mind to run.

+
+
+ +
+
06 — join in
+

Come build the microscope

+

Every optimisation in this project started with someone measuring something on + their own machine — a disk swap, a NUMA experiment, an atlas replication on a Mac Mini. + You don't need a datacenter to move frontier-model research. That is the whole point.

+
+

▸ run a benchmark

+

Any hardware is a datapoint — the community table grows one issue at a time.

 +

▸ pick an issue

+

Kernels, schedulers, the atlas, the dashboard — measured problems, waiting.

 +

▸ read the engine

+

One C file plus headers, zero dependencies. Readable in an afternoon — really.

 +
+
+ +
+ +
+ 🐦 colibrì — MIT license + GitHub + API + Tuning + the hummingbird: tiny, fast, precise. +
+ + + +