7 Commits

Author SHA1 Message Date
JustVugg ca39e5333f packaging: single version source + honest editable-install semantics (on top of #396)
colibri/_version.py now reads c/version.py (#394's single source of truth --
coli --version, the release workflow, and pip metadata can no longer drift),
with an importlib.metadata fallback for the installed-wheel case where c/ is
not on disk. README documents that pip install -e . is the supported form:
the engine lives in c/ and is not packaged into a standalone wheel.

Verified in a clean venv: pip install -e . -> colibri.__version__ == 1.0.0
read from c/version.py, coli entrypoint on PATH and functional.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-19 12:38:11 +02:00
JustVugg 741d46ba25 Merge branch 'pr396' into p396-packaging 2026-07-19 12:36:18 +02:00
JustVugg 3cd2674f68 release: fix the Windows job shell — 'msys2 {0}', not 'msys2' (+ inherit PATH for 7z)
GitHub Actions shells are format strings; the bare 'msys2' string made the
v1.0.0 tag build fail before its first step ('Invalid shell option'). Same
invocation ci.yml already uses. path-type: inherit so the Package step can
reach the runner's 7z.exe.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-19 12:29:33 +02:00
Vincenzo Fornaro e25aeecc4c Merge pull request #410 from JustVugg/p403-rss-guard
glm: measured-RSS guard — the RAM budget enforces itself at the safe point (#403)
2026-07-19 11:29:11 +02:00
Vincenzo Fornaro 2f8fefd701 Merge pull request #408 from JustVugg/p401-tools-e2e
serve: end-to-end tool-calling regression test + unparsed-marker diagnosis (#401)
2026-07-19 11:06:14 +02:00
JustVugg c90e2cc438 glm: measured-RSS guard — the RAM budget enforces itself at the safe point (#403)
cap_for_ram's projection is an estimate: on the GB10 (#403) long generations
overshot it by ~40 GB (projected 74.4, real 115.6) and the kernel killed the
engine three times. Run D of the issue proves a low cap CONTAINS the growth;
this guard does that automatically, keyed on MEASURED RSS instead of the
projection.

At the repin safe point (no moe in flight), every ~16 emitted tokens: if RSS
exceeds the resolved budget (RAM_GB/auto, or an explicit RSS_GUARD_GB
ceiling), free the least-used LRU expert slabs in place and lower ecap so the
cache cannot regrow. Slabs are >128 KB so glibc returns the pages to the
kernel immediately -- RSS actually drops.

Eviction never compacts the array: with PILOT_REAL the pilot worker holds
pointers into ecache[] across its preads, so the slot stays in place with
eid=-1/used=0 (first candidate for reuse); reserved slots (eid<0) are never
touched and victim selection happens under g_pilot_mx. resident_bytes is left
alone: LRU slots are never accounted there (only pin + dense).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-19 11:02:31 +02:00
ZacharyZcR d86a6b93ad packaging: pyproject.toml + editorconfig + clang-format
Python packaging:
- pyproject.toml: pip install colibri-engine (editable dev install works)
- colibri/ package with __version__, CLI entry point delegating to c/coli
- Optional dependency groups: [convert] (numpy, huggingface_hub),
  [oracle] (torch, transformers, safetensors), [bench] (tokenizers, datasets)

Code style:
- .editorconfig: consistent indent/charset across all file types
- .clang-format: LLVM-based, 120 col, matches existing engine style

Usage:
  pip install -e .              # dev install (CLI + serve, no heavy deps)
  pip install -e .[convert]     # adds converter dependencies
  pip install -e .[oracle]      # adds torch/transformers for oracle validation
2026-07-19 06:10:14 +08:00
9 changed files with 218 additions and 2 deletions
+10
View File
@@ -0,0 +1,10 @@
BasedOnStyle: LLVM
IndentWidth: 4
ColumnLimit: 120
AllowShortFunctionsOnASingleLine: All
AllowShortIfStatementsOnASingleLine: AllIfsAndElse
AllowShortLoopsOnASingleLine: true
BreakBeforeBraces: Attach
PointerAlignment: Right
SpaceAfterCStyleCast: false
SortIncludes: false
+24
View File
@@ -0,0 +1,24 @@
root = true
[*]
charset = utf-8
end_of_line = lf
insert_final_newline = true
trim_trailing_whitespace = true
indent_style = space
indent_size = 4
[*.{c,h,cu}]
indent_size = 4
[*.{ts,tsx,js,json,css}]
indent_size = 2
[Makefile]
indent_style = tab
[*.yml]
indent_size = 2
[*.md]
trim_trailing_whitespace = false
+7 -2
View File
@@ -24,7 +24,9 @@ jobs:
name: windows-x86_64
ext: ".exe"
make_args: ""
shell: msys2
# GitHub shells are format strings: 'msys2 {0}', never bare 'msys2'
# (the v1.0.0 tag build failed on exactly this). Same as ci.yml.
shell: "msys2 {0}"
runs-on: ${{ matrix.os }}
defaults:
run:
@@ -32,11 +34,14 @@ jobs:
steps:
- uses: actions/checkout@v4
- if: matrix.shell == 'msys2'
- if: matrix.os == 'windows-latest'
uses: msys2/setup-msys2@v2
with:
msystem: UCRT64
update: false
# inherit: the Package step calls the runner's 7z.exe, which lives on
# the Windows PATH; the default minimal path would not see it.
path-type: inherit
install: >-
make
mingw-w64-ucrt-x86_64-gcc
+4
View File
@@ -183,6 +183,10 @@ COLI_MODEL=/nvme/glm52_i4 ./coli doctor # read-only readiness check
The engine at runtime is pure C — python is only used by the one-time converter
and the optional API gateway.
Prefer a `coli` command on your PATH? From a checkout, `pip install -e .`
registers it (the engine itself still lives in `c/` — this is an editable
install from the clone, not a standalone wheel).
### 3. Go deeper
| topic | doc |
+62
View File
@@ -4924,7 +4924,68 @@ static int repin_pick(Model *m, RepinCand *out, int maxc){
}
return nb;
}
/* ---- RSS GUARD (#403) -----------------------------------------------------
* La proiezione di cap_for_ram e' una STIMA: sul GB10 (#403) le generazioni
* lunghe l'hanno sforata di ~40 GB (proiettato 74.4, reale 115.6 -> 3 kill del
* kernel). La run D dell'issue prova che un cap piu' basso CONTIENE la crescita:
* questa guardia lo fa da sola, sull'RSS MISURATO invece che sul proiettato.
* Al safe point (stessa sede di repin: nessun moe in volo), ogni ~16 token
* emessi: se l'RSS supera il budget, svuota gli slot LRU meno usati e abbassa
* ecap perche' non ricrescano. Gli slab sono >128KB (mmap'd da glibc): la free
* restituisce le pagine al kernel subito, quindi l'RSS scende davvero.
* Lo slot NON viene compattato via: resta al suo posto con eid=-1/used=0 (primo
* candidato al riuso), perche' con PILOT_REAL il worker tiene puntatori dentro
* ecache[] durante i suoi pread e uno spostamento li invaliderebbe; per lo
* stesso motivo gli slot eid<0 (riservati/in caricamento) non si toccano e la
* selezione avviene sotto g_pilot_mx. resident_bytes resta invariato: gli slot
* LRU non sono mai contati li' (solo pin e densa).
* EN: evict = free the slab in place (eid=-1, used=0, never compact: PILOT_REAL
* EN: holds pointers into ecache[] across its preads), skip eid<0 reservations,
* EN: select under g_pilot_mx. RSS_GUARD_GB=<gb> forces an explicit ceiling. */
static double g_ram_budget_gb=0; /* budget risolto, scritto da cap_for_ram */
static uint64_t g_rssg_last=0;
static void rss_guard(Model *m){
double lim = getenv("RSS_GUARD_GB") ? atof(getenv("RSS_GUARD_GB")) : g_ram_budget_gb;
if(lim<=0) return;
if(m->n_emit - g_rssg_last < 16) return;
g_rssg_last = m->n_emit;
double rss=rss_gb();
if(rss <= lim*1.02+0.3) return; /* tolleranza: 2% + 300MB */
Cfg *c=&m->c;
int64_t need=(int64_t)((rss-lim)*1e9), freed=0; int dropped=0;
for(int pass=0; pass<8 && freed<need; pass++){
for(int l=0; l<=c->n_layers && freed<need; l++){
if(!m->ecache || !m->ecache[l]) continue;
pthread_mutex_lock(&g_pilot_mx);
int nn=m->ecn[l], lru=-1;
for(int z=0;z<nn;z++){ /* solo slot pubblicati e con slab */
ESlot *cand=&m->ecache[l][z];
if(cand->eid<0 || !cand->slab) continue;
if(lru<0 || cand->used<m->ecache[l][lru].used) lru=z;
}
if(lru<0){ pthread_mutex_unlock(&g_pilot_mx); continue; }
ESlot *s=&m->ecache[l][lru];
s->eid=-1; /* nascosto: nessun hit/evict altrui */
pthread_mutex_unlock(&g_pilot_mx);
int64_t sb=s->slab_cap + s->fslab_cap*4;
#ifdef COLI_METAL
if(s->slab && g_metal_enabled) coli_metal_unregister(s->slab);
#endif
compat_aligned_free(s->slab); free(s->fslab);
s->slab=NULL; s->fslab=NULL; s->slab_cap=s->fslab_cap=0;
QT *q[3]={&s->g,&s->u,&s->d};
for(int k=0;k<3;k++){ q[k]->qf=NULL; q[k]->q8=NULL; q[k]->q4=NULL; q[k]->s=NULL; }
s->used=0; /* primo candidato al riuso */
freed += sb; dropped++;
}
if(m->ecap>2) m->ecap--; /* il tetto scende: niente ricrescita */
}
if(dropped)
fprintf(stderr,"[RAM-GUARD] RSS %.1f GB over the %.1f GB budget (#403): "
"dropped %d cached experts, cap -> %d\n", rss, lim, dropped, m->ecap);
}
static void repin_pass_limit(Model *m,int limit){
rss_guard(m); /* #403: il budget si fa rispettare sull'RSS MISURATO */
if(g_repin<=0) return;
if(m->n_emit - g_last_repin < (uint64_t)g_repin) return;
g_last_repin = m->n_emit;
@@ -6024,6 +6085,7 @@ static void cap_for_ram(Model *m, double ram_gb, int ebits, int max_ctx){
if(auto_b){ ram_gb = g_mem_avail_boot*0.88; /* misurata PRIMA del load: il residente gia'
* allocato viene sottratto sotto, non due volte */
if(ram_gb<4){ fprintf(stderr,"[RAM] MemAvailable is unreadable or too low; assuming 8 GB\n"); ram_gb=8; } }
g_ram_budget_gb = ram_gb; /* #403: la RSS-guard usa il budget RISOLTO */
/* slack ONESTO, non forfettario (l'OOM del 2026-07-04 veniva da qui):
* ws[64] slab del working-set (si materializzano TUTTI nel prefill batch-union),
* KV cache a max_ctx, kvb_all della ricostruzione k/v in attention,
+5
View File
@@ -0,0 +1,5 @@
"""colibrì — tiny engine, immense model."""
from colibri._version import __version__
__all__ = ["__version__"]
+23
View File
@@ -0,0 +1,23 @@
"""Version accessor for the pip package.
The single source of truth is c/version.py (#394): coli --version and the
GitHub Release workflow read it, so the pip metadata must read the SAME file
instead of carrying a second literal that would drift on the first bump.
From a checkout (the supported install: `pip install -e .`) the file is read
directly. From an installed wheel c/ is not on disk, so fall back to the
package metadata that setuptools baked at build time from that same file.
"""
from pathlib import Path
try:
_ns = {}
exec((Path(__file__).resolve().parent.parent / "c" / "version.py").read_text(), _ns)
__version__ = _ns["__version__"]
except OSError:
from importlib.metadata import PackageNotFoundError, version
try:
__version__ = version("colibri-engine")
except PackageNotFoundError:
__version__ = "0.0.0+unknown"
+30
View File
@@ -0,0 +1,30 @@
"""Entry point for `coli` when installed via pip.
Delegates to the original c/coli script which handles all subcommands.
This wrapper exists so `pip install colibri-engine` creates a `coli` console
script that works without the user having to add c/ to PATH manually.
"""
import os
import sys
import runpy
def main():
here = os.path.dirname(os.path.abspath(__file__))
engine_dir = os.path.join(os.path.dirname(here), "c")
coli_script = os.path.join(engine_dir, "coli")
if not os.path.exists(coli_script):
sys.exit(
"colibri engine directory not found.\n"
"Install from source: git clone + pip install -e ."
)
sys.path.insert(0, engine_dir)
sys.argv[0] = coli_script
runpy.run_path(coli_script, run_name="__main__")
if __name__ == "__main__":
main()
+53
View File
@@ -0,0 +1,53 @@
[build-system]
requires = ["setuptools>=68.0"]
build-backend = "setuptools.build_meta"
[project]
name = "colibri-engine"
dynamic = ["version"]
description = "Tiny engine, immense model — run GLM-5.2 (744B MoE) locally"
readme = "README.md"
license = "Apache-2.0"
requires-python = ">=3.10"
authors = [
{name = "JustVugg"},
]
classifiers = [
"Development Status :: 4 - Beta",
"Environment :: Console",
"Intended Audience :: Science/Research",
"Operating System :: POSIX :: Linux",
"Operating System :: MacOS",
"Operating System :: Microsoft :: Windows",
"Programming Language :: Python :: 3",
"Topic :: Scientific/Engineering :: Artificial Intelligence",
]
[project.optional-dependencies]
convert = [
"numpy",
"huggingface_hub",
]
oracle = [
"torch>=2.0",
"transformers>=4.40",
"safetensors",
]
bench = [
"tokenizers",
"datasets",
]
[project.scripts]
coli = "colibri.cli:main"
[project.urls]
Homepage = "https://github.com/JustVugg/colibri"
Issues = "https://github.com/JustVugg/colibri/issues"
[tool.setuptools.dynamic]
version = {attr = "colibri._version.__version__"}
[tool.setuptools.packages.find]
where = ["."]
include = ["colibri*"]