Merge pull request #424 from ZacharyZcR/feat/website
site: official website — animated demo, expert brain, 3-D atlas + GitHub Pages deploy
This commit is contained in:
Vincenzo Fornaro
@@ -0,0 +1,36 @@
|
||||
name: Deploy website
|
||||
|
||||
# Publishes site/ to GitHub Pages. One-time repo setup:
|
||||
# Settings → Pages → Build and deployment → Source: "GitHub Actions".
|
||||
# Custom domain later: add site/CNAME with the bare domain, point DNS
|
||||
# (A/AAAA to GitHub Pages IPs or CNAME to <org>.github.io), done.
|
||||
|
||||
on:
|
||||
push:
|
||||
branches: [main]
|
||||
paths: ['site/**', '.github/workflows/site.yml']
|
||||
workflow_dispatch:
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
pages: write
|
||||
id-token: write
|
||||
|
||||
concurrency:
|
||||
group: pages
|
||||
cancel-in-progress: true
|
||||
|
||||
jobs:
|
||||
deploy:
|
||||
runs-on: ubuntu-latest
|
||||
environment:
|
||||
name: github-pages
|
||||
url: ${{ steps.deployment.outputs.page_url }}
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- uses: actions/configure-pages@v5
|
||||
- uses: actions/upload-pages-artifact@v3
|
||||
with:
|
||||
path: site
|
||||
- id: deployment
|
||||
uses: actions/deploy-pages@v4
|
||||
@@ -44,6 +44,16 @@ brightness is routing heat, and every expert routed in a turn flashes white. Hov
|
||||
as a 3-D galaxy — 13,260 characterised experts, 1,041 replicated specialists clustering by topic
|
||||
(poetry, law, Chinese, SQL…). Position is measured routing affinity, not a learned embedding. Drag to spin.</em></p>
|
||||
|
||||
## The vision
|
||||
|
||||
Frontier models should not be sealed inside datacenters. colibrì exists so that
|
||||
**anyone curious enough can open one up**: run a 744B-parameter mind on hardware
|
||||
you already own, watch every expert fire in real time, and change the code that
|
||||
does it. Not renting intelligence behind an API — *holding* it: probing it,
|
||||
measuring it, improving it. Every optimisation in this project started with
|
||||
someone measuring something on their own machine; the engine is deliberately
|
||||
small enough that the next one can come from you.
|
||||
|
||||
## The idea
|
||||
|
||||
A 744B Mixture-of-Experts model activates only ~40B parameters per token — and
|
||||
@@ -61,6 +71,18 @@ So the model doesn't need to *fit* in fast memory — it needs to be **placed**:
|
||||
at int4) live **on disk** (~370 GB) and are **streamed on demand**, with a
|
||||
per-layer LRU cache, a learned pinned hot-store, and an optional VRAM tier.
|
||||
|
||||
Think of the core algorithm as **a JIT, but for weights**. A compiler JIT never
|
||||
compiles the whole program — it watches what actually runs and compiles the hot
|
||||
paths, just in time. colibrì makes the same bet about a 744B parameter space:
|
||||
parameters are not resident state to be held, they are **data to be staged**
|
||||
across a heterogeneous storage hierarchy (VRAM / RAM / NVMe), exactly when the
|
||||
router proves they are needed. Measured routing heat decides which experts earn
|
||||
which tier, the router runs a layer ahead so prefetch hides the staging latency,
|
||||
and — like a JIT — the engine learns your workload: the more you run, the hotter
|
||||
the right experts get. It works because routing has measurable structure (see
|
||||
the [expert atlas](https://github.com/JustVugg/colibri/issues/175)) — and
|
||||
structure is cacheable.
|
||||
|
||||
The engine is a single C file (`c/glm.c`) plus small headers. No BLAS, no Python
|
||||
at runtime, no GPU required.
|
||||
|
||||
@@ -211,6 +233,18 @@ install from the clone, not a standalone wheel).
|
||||
| Grammar-forced drafts (structured output) | [docs/grammar-draft.md](docs/grammar-draft.md) |
|
||||
| Environment variable inventory | [docs/ENVIRONMENT.md](docs/ENVIRONMENT.md) |
|
||||
|
||||
## What's next
|
||||
|
||||
- **Algorithmic research is active.** The current hierarchy is LRU + a learned
|
||||
pin set; the next step is under way — smarter placement and scheduling,
|
||||
overlap of CPU and GPU expert execution, and routing-aware speculation.
|
||||
Everything lands the way this project always works: measured, reviewed, and
|
||||
merged in the open.
|
||||
- **More open models.** The tiering algorithm is model-agnostic: any MoE with
|
||||
routed experts can be staged the same way. GLM-5.2 and OLMoE run today;
|
||||
support for more open-weight families — **Kimi K2** (Moonshot AI),
|
||||
**Qwen3 MoE** (Alibaba), **MiniMax** — is on the roadmap.
|
||||
|
||||
## Supporting the project
|
||||
|
||||
colibrì started as a one-person project on a 12-core laptop with 25 GB of RAM;
|
||||
@@ -251,6 +285,14 @@ The hummingbird weighs a few grams, hovers in place, and visits a thousand
|
||||
flowers a day. This engine keeps a 744-billion-parameter giant alive on
|
||||
hummingbird rations: 25 GB of RAM, twelve CPU cores, and a lot of disk patience.
|
||||
|
||||
## Acknowledgements
|
||||
|
||||
colibrì is an engine; the minds it runs are a gift. Thank you to the teams
|
||||
releasing frontier-class weights in the open — **Z.ai** (GLM), **Moonshot AI**
|
||||
(Kimi), **Alibaba Qwen**, **MiniMax**, and **Allen AI** (OLMoE) — and to every
|
||||
contributor who benchmarked, bisected, replicated an atlas run, or sent a patch.
|
||||
This project is proof of what open weights make possible.
|
||||
|
||||
## License
|
||||
|
||||
Apache 2.0. GLM-5.2 weights are released by Z.ai under MIT.
|
||||
|
||||
@@ -0,0 +1,50 @@
|
||||
<svg xmlns="http://www.w3.org/2000/svg" width="512" height="512" viewBox="0 0 168 168">
|
||||
<rect width="168" height="168" rx="36" fill="#080b0d"/>
|
||||
<g transform="translate(7 21)" shape-rendering="crispEdges">
|
||||
<rect x="56" y="0" width="14" height="14" fill="#d75fd7"/>
|
||||
<rect x="70" y="0" width="14" height="14" fill="#d75fd7"/>
|
||||
<rect x="84" y="0" width="14" height="14" fill="#d75fd7"/>
|
||||
<rect x="42" y="14" width="14" height="14" fill="#d75fd7"/>
|
||||
<rect x="56" y="14" width="14" height="14" fill="#d75fd7"/>
|
||||
<rect x="70" y="14" width="14" height="14" fill="#d75fd7"/>
|
||||
<rect x="84" y="14" width="14" height="14" fill="#d75fd7"/>
|
||||
<rect x="98" y="14" width="14" height="14" fill="#d75fd7"/>
|
||||
<rect x="140" y="14" width="14" height="14" fill="#5fd7d7"/>
|
||||
<rect x="56" y="28" width="14" height="14" fill="#d75fd7"/>
|
||||
<rect x="70" y="28" width="14" height="14" fill="#d75fd7"/>
|
||||
<rect x="84" y="28" width="14" height="14" fill="#d75fd7"/>
|
||||
<rect x="98" y="28" width="14" height="14" fill="#d75fd7"/>
|
||||
<rect x="126" y="28" width="14" height="14" fill="#5fd7d7"/>
|
||||
<rect x="140" y="28" width="14" height="14" fill="#5fd7d7"/>
|
||||
<rect x="0" y="42" width="14" height="14" fill="#ff8700"/>
|
||||
<rect x="14" y="42" width="14" height="14" fill="#ff8700"/>
|
||||
<rect x="28" y="42" width="14" height="14" fill="#ff8700"/>
|
||||
<rect x="42" y="42" width="14" height="14" fill="#ff8700"/>
|
||||
<rect x="56" y="42" width="14" height="14" fill="#00afaf"/>
|
||||
<rect x="70" y="42" width="14" height="14" fill="#00afaf"/>
|
||||
<rect x="84" y="42" width="14" height="14" fill="#fff"/>
|
||||
<rect x="98" y="42" width="14" height="14" fill="#00afaf"/>
|
||||
<rect x="112" y="42" width="14" height="14" fill="#5fd7d7"/>
|
||||
<rect x="126" y="42" width="14" height="14" fill="#5fd7d7"/>
|
||||
<rect x="56" y="56" width="14" height="14" fill="#00afaf"/>
|
||||
<rect x="70" y="56" width="14" height="14" fill="#00afaf"/>
|
||||
<rect x="84" y="56" width="14" height="14" fill="#00afaf"/>
|
||||
<rect x="98" y="56" width="14" height="14" fill="#00afaf"/>
|
||||
<rect x="112" y="56" width="14" height="14" fill="#00afaf"/>
|
||||
<rect x="126" y="56" width="14" height="14" fill="#5fd7d7"/>
|
||||
<rect x="140" y="56" width="14" height="14" fill="#5fd7d7"/>
|
||||
<rect x="70" y="70" width="14" height="14" fill="#00afaf"/>
|
||||
<rect x="84" y="70" width="14" height="14" fill="#00afaf"/>
|
||||
<rect x="98" y="70" width="14" height="14" fill="#00afaf"/>
|
||||
<rect x="112" y="70" width="14" height="14" fill="#00afaf"/>
|
||||
<rect x="126" y="70" width="14" height="14" fill="#5fd7d7"/>
|
||||
<rect x="140" y="70" width="14" height="14" fill="#5fd7d7"/>
|
||||
<rect x="84" y="84" width="14" height="14" fill="#00afaf"/>
|
||||
<rect x="98" y="84" width="14" height="14" fill="#00afaf"/>
|
||||
<rect x="112" y="84" width="14" height="14" fill="#5fd7d7"/>
|
||||
<rect x="126" y="84" width="14" height="14" fill="#5fd7d7"/>
|
||||
<rect x="98" y="98" width="14" height="14" fill="#00afaf"/>
|
||||
<rect x="112" y="98" width="14" height="14" fill="#5fd7d7"/>
|
||||
<rect x="112" y="112" width="14" height="14" fill="#5fd7d7"/>
|
||||
</g>
|
||||
</svg>
|
||||
|
After Width: | Height: | Size: 3.0 KiB |
@@ -0,0 +1,55 @@
|
||||
<svg xmlns="http://www.w3.org/2000/svg" width="620" height="140" viewBox="0 0 620 140">
|
||||
<g shape-rendering="crispEdges">
|
||||
<rect x="56" y="0" width="14" height="14" fill="#d75fd7"/>
|
||||
<rect x="70" y="0" width="14" height="14" fill="#d75fd7"/>
|
||||
<rect x="84" y="0" width="14" height="14" fill="#d75fd7"/>
|
||||
<rect x="42" y="14" width="14" height="14" fill="#d75fd7"/>
|
||||
<rect x="56" y="14" width="14" height="14" fill="#d75fd7"/>
|
||||
<rect x="70" y="14" width="14" height="14" fill="#d75fd7"/>
|
||||
<rect x="84" y="14" width="14" height="14" fill="#d75fd7"/>
|
||||
<rect x="98" y="14" width="14" height="14" fill="#d75fd7"/>
|
||||
<rect x="140" y="14" width="14" height="14" fill="#5fd7d7"/>
|
||||
<rect x="56" y="28" width="14" height="14" fill="#d75fd7"/>
|
||||
<rect x="70" y="28" width="14" height="14" fill="#d75fd7"/>
|
||||
<rect x="84" y="28" width="14" height="14" fill="#d75fd7"/>
|
||||
<rect x="98" y="28" width="14" height="14" fill="#d75fd7"/>
|
||||
<rect x="126" y="28" width="14" height="14" fill="#5fd7d7"/>
|
||||
<rect x="140" y="28" width="14" height="14" fill="#5fd7d7"/>
|
||||
<rect x="0" y="42" width="14" height="14" fill="#ff8700"/>
|
||||
<rect x="14" y="42" width="14" height="14" fill="#ff8700"/>
|
||||
<rect x="28" y="42" width="14" height="14" fill="#ff8700"/>
|
||||
<rect x="42" y="42" width="14" height="14" fill="#ff8700"/>
|
||||
<rect x="56" y="42" width="14" height="14" fill="#00afaf"/>
|
||||
<rect x="70" y="42" width="14" height="14" fill="#00afaf"/>
|
||||
<rect x="84" y="42" width="14" height="14" fill="#ffffff"/>
|
||||
<rect x="98" y="42" width="14" height="14" fill="#00afaf"/>
|
||||
<rect x="112" y="42" width="14" height="14" fill="#5fd7d7"/>
|
||||
<rect x="126" y="42" width="14" height="14" fill="#5fd7d7"/>
|
||||
<rect x="56" y="56" width="14" height="14" fill="#00afaf"/>
|
||||
<rect x="70" y="56" width="14" height="14" fill="#00afaf"/>
|
||||
<rect x="84" y="56" width="14" height="14" fill="#00afaf"/>
|
||||
<rect x="98" y="56" width="14" height="14" fill="#00afaf"/>
|
||||
<rect x="112" y="56" width="14" height="14" fill="#00afaf"/>
|
||||
<rect x="126" y="56" width="14" height="14" fill="#5fd7d7"/>
|
||||
<rect x="140" y="56" width="14" height="14" fill="#5fd7d7"/>
|
||||
<rect x="70" y="70" width="14" height="14" fill="#00afaf"/>
|
||||
<rect x="84" y="70" width="14" height="14" fill="#00afaf"/>
|
||||
<rect x="98" y="70" width="14" height="14" fill="#00afaf"/>
|
||||
<rect x="112" y="70" width="14" height="14" fill="#00afaf"/>
|
||||
<rect x="126" y="70" width="14" height="14" fill="#5fd7d7"/>
|
||||
<rect x="140" y="70" width="14" height="14" fill="#5fd7d7"/>
|
||||
<rect x="84" y="84" width="14" height="14" fill="#00afaf"/>
|
||||
<rect x="98" y="84" width="14" height="14" fill="#00afaf"/>
|
||||
<rect x="112" y="84" width="14" height="14" fill="#5fd7d7"/>
|
||||
<rect x="126" y="84" width="14" height="14" fill="#5fd7d7"/>
|
||||
<rect x="98" y="98" width="14" height="14" fill="#00afaf"/>
|
||||
<rect x="112" y="98" width="14" height="14" fill="#5fd7d7"/>
|
||||
<rect x="112" y="112" width="14" height="14" fill="#5fd7d7"/>
|
||||
</g>
|
||||
<text x="252" y="62" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace"
|
||||
font-size="52" font-weight="bold" fill="#00afaf">colibrì</text>
|
||||
<text x="252" y="94" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace"
|
||||
font-size="19" fill="#808080" font-style="italic">tiny engine, immense model</text>
|
||||
<text x="252" y="122" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace"
|
||||
font-size="15" fill="#9a9a9a">GLM-5.2 · 744B MoE · int4 · streaming CPU</text>
|
||||
</svg>
|
||||
|
After Width: | Height: | Size: 3.5 KiB |
+698
File diff suppressed because one or more lines are too long
Reference in New Issue
Block a user