diff --git a/.github/workflows/site.yml b/.github/workflows/site.yml new file mode 100644 index 0000000..160cd61 --- /dev/null +++ b/.github/workflows/site.yml @@ -0,0 +1,36 @@ +name: Deploy website + +# Publishes site/ to GitHub Pages. One-time repo setup: +# Settings → Pages → Build and deployment → Source: "GitHub Actions". +# Custom domain later: add site/CNAME with the bare domain, point DNS +# (A/AAAA to GitHub Pages IPs or CNAME to .github.io), done. + +on: + push: + branches: [main] + paths: ['site/**', '.github/workflows/site.yml'] + workflow_dispatch: + +permissions: + contents: read + pages: write + id-token: write + +concurrency: + group: pages + cancel-in-progress: true + +jobs: + deploy: + runs-on: ubuntu-latest + environment: + name: github-pages + url: ${{ steps.deployment.outputs.page_url }} + steps: + - uses: actions/checkout@v4 + - uses: actions/configure-pages@v5 + - uses: actions/upload-pages-artifact@v3 + with: + path: site + - id: deployment + uses: actions/deploy-pages@v4 diff --git a/site/colibri-icon.svg b/site/colibri-icon.svg new file mode 100644 index 0000000..76838b9 --- /dev/null +++ b/site/colibri-icon.svg @@ -0,0 +1,50 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/site/colibri.svg b/site/colibri.svg new file mode 100644 index 0000000..4aed0bb --- /dev/null +++ b/site/colibri.svg @@ -0,0 +1,55 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + colibrì + tiny engine, immense model + GLM-5.2 · 744B MoE · int4 · streaming CPU + diff --git a/site/index.html b/site/index.html new file mode 100644 index 0000000..3972690 --- /dev/null +++ b/site/index.html @@ -0,0 +1,456 @@ + + + + + +colibrì — tiny engine, immense model + + + + + + + + + + + + + +
+
+ +

tiny engine, immense model

+

Run GLM-5.2 — a 744-billion-parameter Mixture-of-Experts — on your own machine. + Pure C, zero dependencies, experts streamed from disk. From a 25 GB dev box to a + six-GPU workstation: same engine, same container, the hardware only changes where the experts live.

+
+ ★ GitHub + Quick start + Benchmarks +
+
+
744B
parameters
+
19,456
experts
+
~11 GB
routed / token
+
0
dependencies
+
25 GB
min RAM
+
+
+ +
+

# watch it think

+

A replay of the engine at work, paced to measured decode speeds from real hardware. + Every token routes through 8 experts in each of 75 MoE layers — the grid on the right is all + 19,456 experts: colour is the storage tier, brightness is routing heat, and every expert + a token touches flashes white.

+
+
+
+
+ coli chat —
+
+
+
+
+ the brain — 75 MoE layers × 256 experts + MTP
+
+ +
+
0.0 tok/s
decode
+
— s
ttft (measured)
+
— %
resident hit
+
+
+ VRAM tier + RAM tier + disk tier + routed now +
+
+
+
+

Simulation replays a fixed transcript at each profile's measured decode rate; TTFT shown is the measured value, not a live wait.

+
+ +
+
+

# the expert atlas

+

Routing affinity is measured, not assumed. Characterised experts cluster by what + they actually fire on — poetry, law, Chinese, SQL, code, mathematics… Position below is measured + routing affinity, not a learned embedding. Drag to spin.

+
+
+ +
+
drag to spin · scroll past to release
+
+
+ +
+

# how it fits

+

A 744B MoE activates only ~40B parameters per token — and only ~11 GB of those + change from token to token (the routed experts). colibrì keeps the dense weights resident and tiers the + 19,456 experts (~19 MB each, int4) across three levels by measured heat:

+
+

VRAM

+
hottest experts · grouped GPU matmuls
+

With CUDA or Metal, the hottest experts live on-device and are batched into grouped kernels. + Six RTX 5090s hold the entire routed set: disk reads drop to zero.

+

RAM

+
warm experts · pinned & wired
+

The warm set is pinned in RAM (PIN_GB), guided by + recorded usage stats, and served by AVX-512/NEON int4 kernels with an LRU cache behind it.

+

disk

+
cold tail · streamed on demand
+

Everything else streams from NVMe with io_uring, prefetched a layer ahead by the router. + This is how 744B fits a 25 GB machine at all — the proven floor.

+
+
token → router picks 8 of 256 experts per layer → engine gathers them from + VRAM / RAM / disk → int4 matmuls → next token. Output is validated token-exact against the + reference transformers implementation.
+
+ +
+

# the ladder

+

Same engine, same int4 container — measured decode on real machines, + from the proven floor to full residency.

+ + + + + + + + + + + + +
hardwaredecodewhere the experts live
6× RTX 5090 · full residency5.8–6.8 tok/s
all in VRAM+RAM · disk 0
128 GB CPU-only desktop~1.8 tok/s
hot set pinned in RAM
single RTX 5070 Ti laptop-class1.07 tok/s
GPU-resident pipeline
25 GB dev box · cold0.05–0.1 tok/s
streamed from NVMe
+

Full tables, methodology and quality ablations: docs/benchmarks.md

+
+
+ + + + + +