Merge branch 'p394' into trialsafe
This commit is contained in:
JustVugg
@@ -0,0 +1,99 @@
|
||||
name: Release
|
||||
|
||||
on:
|
||||
push:
|
||||
tags: ['v*']
|
||||
|
||||
permissions:
|
||||
contents: write
|
||||
|
||||
jobs:
|
||||
build:
|
||||
strategy:
|
||||
matrix:
|
||||
include:
|
||||
- os: ubuntu-latest
|
||||
name: linux-x86_64
|
||||
ext: ""
|
||||
make_args: "ARCH=x86-64-v3"
|
||||
- os: macos-latest
|
||||
name: macos-arm64
|
||||
ext: ""
|
||||
make_args: ""
|
||||
- os: windows-latest
|
||||
name: windows-x86_64
|
||||
ext: ".exe"
|
||||
make_args: ""
|
||||
shell: msys2
|
||||
runs-on: ${{ matrix.os }}
|
||||
defaults:
|
||||
run:
|
||||
shell: ${{ matrix.shell || 'bash' }}
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
- if: matrix.shell == 'msys2'
|
||||
uses: msys2/setup-msys2@v2
|
||||
with:
|
||||
msystem: UCRT64
|
||||
update: false
|
||||
install: >-
|
||||
make
|
||||
mingw-w64-ucrt-x86_64-gcc
|
||||
|
||||
- if: matrix.os == 'macos-latest'
|
||||
run: brew install libomp
|
||||
|
||||
- name: Build engine
|
||||
run: |
|
||||
cd c
|
||||
make glm ${{ matrix.make_args }}
|
||||
ls -lh glm${{ matrix.ext }}
|
||||
|
||||
- name: Package
|
||||
run: |
|
||||
TAG=${GITHUB_REF#refs/tags/}
|
||||
mkdir -p dist
|
||||
cp c/glm${{ matrix.ext }} dist/colibri-${TAG}-${{ matrix.name }}${{ matrix.ext }}
|
||||
cp c/coli dist/
|
||||
cp c/version.py dist/
|
||||
cp c/openai_server.py dist/
|
||||
cp c/resource_plan.py dist/
|
||||
cp c/doctor.py dist/
|
||||
cp LICENSE dist/
|
||||
cd dist
|
||||
if [ "${{ matrix.ext }}" = ".exe" ]; then
|
||||
7z a colibri-${TAG}-${{ matrix.name }}.zip *
|
||||
else
|
||||
tar czf colibri-${TAG}-${{ matrix.name }}.tar.gz *
|
||||
fi
|
||||
|
||||
- uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: colibri-${{ matrix.name }}
|
||||
path: dist/colibri-*.*
|
||||
|
||||
release:
|
||||
needs: build
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
- uses: actions/download-artifact@v4
|
||||
with:
|
||||
path: artifacts
|
||||
merge-multiple: true
|
||||
|
||||
- name: Create GitHub Release
|
||||
env:
|
||||
GH_TOKEN: ${{ github.token }}
|
||||
run: |
|
||||
TAG=${GITHUB_REF#refs/tags/}
|
||||
# Extract the latest version section from CHANGELOG
|
||||
NOTES=$(awk "/^## \\[${TAG#v}\\]/{found=1;next} /^## \\[/{if(found)exit} found{print}" CHANGELOG.md)
|
||||
if [ -z "$NOTES" ]; then
|
||||
NOTES="Release ${TAG}"
|
||||
fi
|
||||
gh release create "$TAG" artifacts/* \
|
||||
--title "colibrì ${TAG}" \
|
||||
--notes "$NOTES"
|
||||
@@ -0,0 +1,53 @@
|
||||
# Changelog
|
||||
|
||||
All notable changes to colibrì are documented here.
|
||||
Format follows [Keep a Changelog](https://keepachangelog.com/).
|
||||
|
||||
## [1.0.0] — 2026-07-19
|
||||
|
||||
First tagged release. The engine has been running in production since late June
|
||||
2026; this tag marks the baseline for semantic versioning going forward.
|
||||
|
||||
### Highlights
|
||||
|
||||
- **GLM-5.2 (744B MoE)** runs on ~25 GB RAM in pure C, streaming experts from disk
|
||||
- **Three-tier placement**: VRAM (hot) / RAM (warm) / NVMe (cold), with a learning
|
||||
cache that pins your workload's hottest experts automatically
|
||||
- **CUDA backend**: multi-GPU expert tier, dense tensor distribution, batched
|
||||
ragged attention, resident pipeline (`COLI_CUDA_PIPE=2`)
|
||||
- **Metal backend** (Apple Silicon): batched expert SwiGLU + fused decode attention
|
||||
on unified memory GPU
|
||||
- **MTP speculation**: native GLM-5.2 draft heads, grammar-forced drafts, kernel-
|
||||
pinned verification (`SPEC_PIN=1`)
|
||||
- **OpenAI-compatible API**: `coli serve` with SSE streaming, KV slots, bounded
|
||||
queue, web dashboard (`coli web`)
|
||||
- **Web UI**: chat with live metrics, expert cortex brain page, profiling breakdown,
|
||||
expert atlas 3-D galaxy
|
||||
- **Cross-platform**: Linux, macOS, Windows 11 (native MinGW), PowerPC; CI on all three
|
||||
- **Auto-tune**: `coli plan --auto-tier` classifies the bottleneck and derives
|
||||
MTP/PIPE/NUMA/PIN settings with explanations
|
||||
|
||||
### Engine
|
||||
|
||||
- Token-exact validation against `transformers` oracle (teacher-forcing 32/32)
|
||||
- Compressed MLA KV cache (576 floats/token, 57× smaller), persisted across
|
||||
restarts (`.coli_kv`, zero re-prefill)
|
||||
- DSA sparse attention (lightning indexer), faithfully implemented
|
||||
- Router-lookahead prefetch (`PILOT=1`, 71.6% predictive)
|
||||
- Async expert I/O pool (`PIPE=1`), io_uring batching (`URING=1`)
|
||||
- NUMA-aware expert placement (`COLI_NUMA=1`, +13–40% on multi-socket)
|
||||
- AVX2 / AVX-512 / AVX-VNNI / ARM NEON / NEON-i8mm / POWER VSX kernels
|
||||
- int4 / int8 / int2 / grouped-int4 (fmt=4) quantization formats
|
||||
|
||||
### Tools
|
||||
|
||||
- `coli convert` — FP8→int4 one-shard-at-a-time converter
|
||||
- `coli doctor` — read-only setup diagnostics
|
||||
- `coli plan` — resource planner with auto-tune prescription
|
||||
- `coli bench` — MMLU / HellaSwag / ARC quality benchmarks
|
||||
- Expert atlas (`tools/analyze.py --web`) — measured topic affinity for 19,456 experts
|
||||
|
||||
### Community
|
||||
|
||||
- 30+ hardware datapoints in the benchmark tracker
|
||||
- Contributions from 20+ authors across engine, docs, tooling, and ports
|
||||
@@ -40,6 +40,8 @@ if sys.platform == "win32":
|
||||
except (AttributeError, OSError): pass
|
||||
|
||||
HERE = os.path.dirname(os.path.abspath(__file__))
|
||||
sys.path.insert(0, HERE)
|
||||
from version import __version__ as _version
|
||||
|
||||
# Run-in-place (source checkout, "cd c && ./coli ..."): the engine, the
|
||||
# support modules (resource_plan.py, doctor.py, openai_server.py) and
|
||||
@@ -115,7 +117,7 @@ def sprite_lines():
|
||||
def banner(sub=""):
|
||||
sp=sprite_lines()
|
||||
txt=[
|
||||
f"{C.teal}{C.b}colibrì{C.r} {C.dim}v1.0{C.r}",
|
||||
f"{C.teal}{C.b}colibrì{C.r} {C.dim}v{_version}{C.r}",
|
||||
f"{C.dim}tiny engine, immense model{C.r}",
|
||||
f"{C.gray}GLM-5.2 · 744B MoE · int4 · streaming CPU{C.r}",
|
||||
f"{C.dgray}{sub}{C.r}" if sub else "",
|
||||
@@ -890,6 +892,7 @@ def main():
|
||||
common.add_argument("--topp", type=float, default=0); common.add_argument("--topk", type=int, default=0)
|
||||
common.add_argument("--temp", type=float, default=None) # temperatura token (0=greedy, default 1.0+nucleus .95)
|
||||
ap=argparse.ArgumentParser(prog="coli", parents=[common], description="colibrì — run GLM-5.2 locally")
|
||||
ap.add_argument("--version", action="version", version=f"colibrì {_version}")
|
||||
sub=ap.add_subparsers(dest="cmd")
|
||||
sub.add_parser("build", parents=[common]); sub.add_parser("info", parents=[common])
|
||||
pp=sub.add_parser("plan",parents=[common])
|
||||
|
||||
@@ -0,0 +1,3 @@
|
||||
"""Single source of truth for the colibrì version number."""
|
||||
|
||||
__version__ = "1.0.0"
|
||||
Reference in New Issue
Block a user