web dashboard: live expert-tier panel (VRAM/RAM/disk), one-process hosting, coli web (#126)
* Web dashboard: live expert-tier panel (VRAM/RAM/disk), static hosting, coli web command - Engine: TIERS protocol line (counts + GB for VRAM/RAM/disk experts) emitted at READY and refreshed after every served turn, computed from the live pin/LRU/CUDA state. - Server: dispatcher tracks the latest snapshot, /health exposes it as "tiers"; the built web UI (web/dist) is served from the same port (read-only, traversal-safe, SPA fallback) so the dashboard needs no second process. - Web: tier bar + legend in the Runtime panel showing where the 19k experts live right now, with GB per tier; updates with the existing health polling. - CLI: `coli web` = serve + auto-open the browser once /health answers (the 744B load takes minutes; a poller waits for it), --no-browser to disable, friendly hint when web/dist isn't built. * coli web: HERE is a path string, not a Path * web: same-origin default + auto-connect when served by the engine The page hosted by coli web talked to http://127.0.0.1:8000/v1 by default while being loaded from http://localhost:8000 - a different origin, so the very first probe died on CORS ('Failed to fetch'). When the page is engine-served the default (and a stored stale factory default) now resolve to window.location.origin + /v1, and the console auto-probes on load; the Vite dev server keeps the classic default. The server's own port is also whitelisted in DEFAULT_CORS_ORIGINS for the localhost/127.0.0.1 cross-name case. * web: fix auto-connect effect returning Promise (React 19 StrictMode crash) The async connect() call inside useEffect returned a Promise that React 19 StrictMode stored as the effect cleanup. On re-render, React called destroy_() on that Promise — 'not a function'. Block body with explicit return undefined prevents any non-function from leaking into the cleanup slot. * web: rich streaming metrics — live token counter, tok/s gauge, TTFT, session totals During generation: a flashing Zap badge counts tokens in real time. After: Gauge shows tok/s, Timer shows TTFT (time to first token), Layers shows prompt→completion token counts, Clock shows queue wait. Session totals (cumulative prompt + completion) appear in the Runtime panel. All with lucide icons and tabular-nums for stable layout. Tier bar gains a smoother cubic-bezier transition and slightly taller height for visual weight. * web: hardware environment panel — CPU model, GPU count/VRAM, RAM total/free, cores Engine emits HWINFO protocol line at startup (CPU name from /proc/cpuinfo, core count, RAM total/available from /proc/meminfo, GPU count + VRAM from CUDA). Server parses and caches it, /health exposes as 'hwinfo'. Dashboard renders it with icons at the top of the Runtime section so the user sees immediately what the engine is running on. * web: downgrade to React 18 — React 19 effect cleanup bug crashes the dashboard React 19 treats the return value of every useEffect callback as a cleanup function. In practice, any async interaction (streamChat's rapid onDelta re-renders, the health poller, auto-connect) caused React 19 to store a Promise as inst.destroy and crash with 'destroy_ is not a function' on the next commit cycle. The bug reproduced in incognito, without StrictMode, and with every effect converted to explicit block bodies returning undefined — it is a React 19 regression in the passive-effect unmount path. React 18 does not exhibit this behavior. Pin to React 18 until React 19 is fixed or the codebase migrates to a pattern React 19 handles correctly. All 17 vitest pass, ErrorBoundary retained. * web: Brain page — the expert cortex, live A 76x256 canvas grid, one cell per expert (19,456 total): colour = tier (green VRAM / blue RAM / grey disk), brightness = routing heat (log-bucketed .coli_usage counts), and a white pulse that flashes on every expert routed in the current turn and decays — you watch the model think. - Engine: EMAP protocol line (1 byte/expert: 2bit tier + 6bit heat, hex) at READY and after each turn; HITS bitmap of this turn's routed experts (set where eusage increments, cleared on emit). - Server: parses both, GET /experts serves {rows, cols, map, hits, seq}. - Web: Brain tab next to Chat; canvas renderer with rAF pulse decay; hover tooltip shows layer/expert/tier/heat plus an honest depth-role heuristic (early = surface features ... late = output shaping, MTP row labelled as the speculative draft head). * web: Brain page responsive — cells sized from container via ResizeObserver Cell size derives from the wrapper's actual client box instead of fixed 1400x900, re-rendering on resize; mobile media query tightens padding, legend and tooltip. The cortex now fills whatever screen it gets.
This commit is contained in:
ZacharyZcR
@@ -550,6 +550,28 @@ def cmd_serve(a):
|
||||
a.cap,a.ngen,GLM,env_for(a),a.cors_origin,
|
||||
a.max_queue,a.queue_timeout,a.kv_slots)
|
||||
|
||||
def cmd_web(a):
|
||||
"""serve + open the dashboard in the browser once the API answers."""
|
||||
need_model(a.model)
|
||||
dist = os.path.join(os.path.dirname(os.path.abspath(HERE)), "web", "dist")
|
||||
if not os.path.exists(os.path.join(dist, "index.html")):
|
||||
print(f"{C.yel}web UI not built:{C.r} run cd web && npm install && npm run build first;")
|
||||
print("serving the API anyway (the dashboard will 404 until built).")
|
||||
url = f"http://{a.host}:{a.port}/"
|
||||
if not getattr(a, "no_browser", False):
|
||||
import threading, urllib.request, webbrowser
|
||||
def opener():
|
||||
for _ in range(600): # the 744B engine takes minutes to load
|
||||
time.sleep(2)
|
||||
try:
|
||||
urllib.request.urlopen(f"http://{a.host}:{a.port}/health", timeout=2)
|
||||
webbrowser.open(url); return
|
||||
except OSError:
|
||||
continue
|
||||
threading.Thread(target=opener, daemon=True).start()
|
||||
print(f"dashboard: {url} (opens automatically when the engine is ready)")
|
||||
cmd_serve(a)
|
||||
|
||||
def cmd_bench(a):
|
||||
need_model(a.model)
|
||||
banner("bench")
|
||||
@@ -620,6 +642,16 @@ def main():
|
||||
ps.add_argument("--max-queue",type=int,default=int(os.environ.get("COLI_MAX_QUEUE","8")))
|
||||
ps.add_argument("--queue-timeout",type=float,default=float(os.environ.get("COLI_QUEUE_TIMEOUT","300")))
|
||||
ps.add_argument("--kv-slots",type=int,default=int(os.environ.get("COLI_KV_SLOTS","1")))
|
||||
pw=sub.add_parser("web", parents=[common], help="serve + open the dashboard in a browser")
|
||||
for arg,kw in (("--host",dict(default="127.0.0.1")),("--port",dict(type=int,default=8000)),
|
||||
("--model-id",dict(default=os.environ.get("COLI_MODEL_ID","glm-5.2-colibri"))),
|
||||
("--api-key",dict(default=os.environ.get("COLI_API_KEY"))),
|
||||
("--cors-origin",dict(action="append",default=None)),
|
||||
("--max-queue",dict(type=int,default=int(os.environ.get("COLI_MAX_QUEUE","8")))),
|
||||
("--queue-timeout",dict(type=float,default=float(os.environ.get("COLI_QUEUE_TIMEOUT","300")))),
|
||||
("--kv-slots",dict(type=int,default=int(os.environ.get("COLI_KV_SLOTS","1"))))):
|
||||
pw.add_argument(arg,**kw)
|
||||
pw.add_argument("--no-browser",action="store_true",help="don't auto-open the browser")
|
||||
pb=sub.add_parser("bench", parents=[common]); pb.add_argument("tasks", nargs="*")
|
||||
pb.add_argument("--limit",type=int,default=40); pb.add_argument("--data",default=os.path.join(HERE,"bench"))
|
||||
pc=sub.add_parser("convert", parents=[common]); pc.add_argument("--repo",default="zai-org/GLM-5.2-FP8")
|
||||
@@ -628,7 +660,7 @@ def main():
|
||||
a=ap.parse_args()
|
||||
handler={"build":cmd_build,"info":cmd_info,"plan":cmd_plan,"doctor":cmd_doctor,
|
||||
"run":cmd_run,"chat":cmd_chat,"serve":cmd_serve,"bench":cmd_bench,
|
||||
"convert":cmd_convert}.get(a.cmd)
|
||||
"convert":cmd_convert,"web":cmd_web}.get(a.cmd)
|
||||
if handler: sys.exit(handler(a) or 0)
|
||||
banner(); print(__doc__)
|
||||
|
||||
|
||||
@@ -176,7 +176,12 @@ typedef struct {
|
||||
int64_t resident_bytes;
|
||||
} Model;
|
||||
|
||||
static void usage_save(Model *m); /* cache che impara: definita accanto a stats_dump */
|
||||
static void usage_save(Model *m);
|
||||
static void tiers_emit(Model *m);
|
||||
static void ehit_mark(Model *m, int layer, int eid);
|
||||
static void emap_emit(Model *m);
|
||||
static void hits_emit(Model *m);
|
||||
static void hwinfo_emit(Model *m); /* cache che impara: definita accanto a stats_dump */
|
||||
#ifdef COLI_CUDA
|
||||
static int g_cuda_enabled;
|
||||
static double g_cuda_expert_gb;
|
||||
@@ -1783,6 +1788,7 @@ static void moe(Model *m, Layer *l, int layer, float *x, int S, float *out){
|
||||
m->ereq+=keff[s];
|
||||
for(int kk=0;kk<keff[s];kk++){
|
||||
m->eusage[layer][idxs[(int64_t)s*K+kk]]++;
|
||||
ehit_mark(m,layer,idxs[(int64_t)s*K+kk]);
|
||||
if(m->eheat[layer][idxs[(int64_t)s*K+kk]]<UINT32_MAX) m->eheat[layer][idxs[(int64_t)s*K+kk]]++;
|
||||
}
|
||||
for(int d=0;d<D;d++) out[(int64_t)s*D+d]=0;
|
||||
@@ -1810,6 +1816,7 @@ static void moe(Model *m, Layer *l, int layer, float *x, int S, float *out){
|
||||
keff[s]=Ke; m->ereq+=Ke;
|
||||
for(int kk=0;kk<Ke;kk++){
|
||||
m->eusage[layer][idx[kk]]++;
|
||||
ehit_mark(m,layer,idx[kk]);
|
||||
if(m->eheat[layer][idx[kk]]<UINT32_MAX) m->eheat[layer][idx[kk]]++;
|
||||
m->elast[layer][idx[kk]]=++m->eaccess_clock;
|
||||
}
|
||||
@@ -3067,6 +3074,10 @@ static void mux_data(Tok *T, unsigned long long id, int token){
|
||||
static void mux_done(Model *m, ServeCtx *sc, ServeReq *r){
|
||||
double dt=now_s()-r->started; if(dt<1e-6) dt=1e-6;
|
||||
double dh=(double)(m->hits-r->hits0), dm=(double)(m->miss-r->miss0);
|
||||
hwinfo_emit(m);
|
||||
tiers_emit(m);
|
||||
emap_emit(m);
|
||||
hits_emit(m);
|
||||
printf("DONE %llu STAT %d %.2f %.1f %.2f %d %d\n",r->id,r->emitted,
|
||||
r->emitted/dt,(dh+dm)>0?100.0*dh/(dh+dm):0.0,rss_gb(),
|
||||
r->prompt_tokens,r->length_limited);
|
||||
@@ -3155,6 +3166,9 @@ static void run_serve_mux(Model *m, const char *snap){
|
||||
for(int i=0;i<nctx;i++) serve_ctx_init(m,&ctx[i],snap,i,maxctx);
|
||||
setvbuf(stdin,NULL,_IONBF,0);
|
||||
printf("\x01\x01READY\x01\x01\nSTAT 0 0.00 0.0 %.2f\n",rss_gb()); fflush(stdout);
|
||||
hwinfo_emit(m);
|
||||
tiers_emit(m);
|
||||
emap_emit(m);
|
||||
int eof=0;
|
||||
for(;;){
|
||||
int active=0; for(int i=0;i<nctx;i++) active+=req[i].active;
|
||||
@@ -3250,6 +3264,7 @@ static void run_serve(Model *m, const char *snap){
|
||||
#define first (sc->first)
|
||||
char *line=NULL; size_t cap=0; ssize_t nr; char *buf=malloc(1<<16);
|
||||
printf("\x01\x01" "READY" "\x01\x01\n"); printf("STAT 0 0.00 0.0 %.2f\n", rss_gb()); fflush(stdout);
|
||||
tiers_emit(m);
|
||||
while((nr=getline(&line,&cap,stdin))>0){
|
||||
if(nr>0 && line[nr-1]=='\n') line[--nr]=0;
|
||||
if(!strcmp(line,"\x02RESET")){ len=0; first=1; if(m->has_mtp) m->kv_start[m->c.n_layers]=-1;
|
||||
@@ -3391,6 +3406,138 @@ static int64_t expert_bytes_probe(Model *m, int ebits){
|
||||
return eb;
|
||||
}
|
||||
|
||||
/* TIERS: fotografia della piramide expert per la dashboard web —
|
||||
* "TIERS <vram> <ram> <disk> <vram_gb> <ram_gb>" sul canale di protocollo.
|
||||
* ram = pinnati non-VRAM + LRU corrente; disk = tutto il resto. */
|
||||
/* BRAIN MAP (dashboard): per-turn expert hit bitmap + full residency/heat map.
|
||||
* g_ehit[layer][eid]=1 quando l'expert viene instradato in questo turno;
|
||||
* hits_emit lo serializza e lo azzera. emap_emit fotografa tier+heat di TUTTI. */
|
||||
static uint8_t **g_ehit;
|
||||
static void ehit_mark(Model *m, int layer, int eid){
|
||||
if(!g_ehit){ Cfg *c=&m->c;
|
||||
g_ehit=calloc(c->n_layers+1,sizeof(uint8_t*));
|
||||
for(int i=0;i<=c->n_layers;i++) g_ehit[i]=calloc(c->n_experts,1);
|
||||
}
|
||||
g_ehit[layer][eid]=1;
|
||||
}
|
||||
|
||||
/* HWINFO: hardware snapshot for the web dashboard — emitted once at READY. */
|
||||
static void hwinfo_emit(Model *m){
|
||||
Cfg *c=&m->c;
|
||||
/* CPU */
|
||||
char cpu[256]=""; FILE *ci=fopen("/proc/cpuinfo","r");
|
||||
if(ci){ char ln[256];
|
||||
while(fgets(ln,sizeof(ln),ci)) if(!strncmp(ln,"model name",10)){
|
||||
char *p=strchr(ln,':'); if(p){ p++; while(*p==' ')p++;
|
||||
int n=(int)strlen(p); if(n>0&&p[n-1]=='\n')p[--n]=0;
|
||||
snprintf(cpu,sizeof(cpu),"%s",p); } break; }
|
||||
fclose(ci); }
|
||||
int cores=0;
|
||||
#ifdef _SC_NPROCESSORS_ONLN
|
||||
cores=(int)sysconf(_SC_NPROCESSORS_ONLN);
|
||||
#endif
|
||||
/* RAM */
|
||||
double ram_total=0,ram_avail=0;
|
||||
FILE *mi=fopen("/proc/meminfo","r");
|
||||
if(mi){ char ln[256]; double mt=0,ma=0;
|
||||
while(fgets(ln,sizeof(ln),mi)){
|
||||
if(sscanf(ln,"MemTotal: %lf",&mt)==1) ram_total=mt/1e6;
|
||||
if(sscanf(ln,"MemAvailable: %lf",&ma)==1) ram_avail=ma/1e6;
|
||||
} fclose(mi); }
|
||||
/* GPU */
|
||||
int ngpu=0; double vram_total=0;
|
||||
char gpu_name[128]="";
|
||||
#ifdef COLI_CUDA
|
||||
ngpu=g_cuda_ndev; vram_total=m->gpu_expert_bytes/1e9;
|
||||
for(int i=0;i<g_cuda_ndev;i++){
|
||||
size_t fr=0,to=0; coli_cuda_mem_info(g_cuda_devices[i],&fr,&to);
|
||||
if(!i) vram_total=(double)to*g_cuda_ndev/1e9;
|
||||
}
|
||||
/* GPU name from the first device — already printed at init */
|
||||
if(g_cuda_ndev>0){
|
||||
/* We don't have a device-name API; parse from the init log line stored in stderr.
|
||||
* Simpler: just read the nvidia driver sysfs or use a fixed label. */
|
||||
snprintf(gpu_name,sizeof(gpu_name),"CUDA device x%d",g_cuda_ndev);
|
||||
}
|
||||
#endif
|
||||
printf("HWINFO %d %.1f %.1f %d %.1f %s|%s\n",
|
||||
cores,ram_total,ram_avail,ngpu,vram_total,cpu,gpu_name);
|
||||
fflush(stdout);
|
||||
}
|
||||
|
||||
static void tiers_emit(Model *m){
|
||||
Cfg *c=&m->c; int nsp=0;
|
||||
for(int i=0;i<c->n_layers;i++) if(m->L[i].sparse) nsp++;
|
||||
int total=(nsp+(m->has_mtp?1:0))*c->n_experts;
|
||||
int pinned=0,lru=0;
|
||||
for(int i=0;i<=c->n_layers;i++){ pinned+=m->npin?m->npin[i]:0; lru+=m->ecn?m->ecn[i]:0; }
|
||||
int vram=0; double vram_gb=0;
|
||||
#ifdef COLI_CUDA
|
||||
vram=m->gpu_expert_count; vram_gb=m->gpu_expert_bytes/1e9;
|
||||
#endif
|
||||
int ram=pinned-vram+lru; if(ram<0) ram=0;
|
||||
int disk=total-vram-ram; if(disk<0) disk=0;
|
||||
double eb=(double)expert_bytes_probe(m,m->ebits);
|
||||
printf("TIERS %d %d %d %.2f %.2f\n",vram,ram,disk,vram_gb,ram*eb/1e9);
|
||||
fflush(stdout);
|
||||
}
|
||||
|
||||
/* EMAP: 1 byte/expert (2bit tier: 0=disk 1=RAM 2=VRAM | 6bit heat log2-bucket),
|
||||
* righe = layer sparsi in ordine (+MTP se presente), colonne = n_experts. Hex. */
|
||||
static void emap_emit(Model *m){
|
||||
Cfg *c=&m->c;
|
||||
int rows=0;
|
||||
for(int i=0;i<c->n_layers;i++) if(m->L[i].sparse) rows++;
|
||||
int has_mtp = m->has_mtp && m->eusage[c->n_layers];
|
||||
if(has_mtp) rows++;
|
||||
int cols=c->n_experts;
|
||||
char *hex=malloc((size_t)rows*cols*2+1); int w=0;
|
||||
for(int i=0;i<=c->n_layers;i++){
|
||||
int is_row = (i<c->n_layers && m->L[i].sparse) || (i==c->n_layers && has_mtp);
|
||||
if(!is_row) continue;
|
||||
for(int e=0;e<cols;e++){
|
||||
int tier=0;
|
||||
ESlot *P=m->pin[i];
|
||||
for(int z=0;z<m->npin[i];z++) if(P[z].eid==e){
|
||||
#ifdef COLI_CUDA
|
||||
tier = P[z].g.cuda?2:1;
|
||||
#else
|
||||
tier = 1;
|
||||
#endif
|
||||
break; }
|
||||
if(!tier && m->ecache && m->ecache[i])
|
||||
for(int z=0;z<m->ecn[i];z++) if(m->ecache[i][z].eid==e){ tier=1; break; }
|
||||
uint32_t u = m->eusage[i]?m->eusage[i][e]:0;
|
||||
int heat=0; while(u){ heat++; u>>=1; } if(heat>63) heat=63;
|
||||
int b=(tier<<6)|heat;
|
||||
hex[w++]="0123456789abcdef"[b>>4]; hex[w++]="0123456789abcdef"[b&15];
|
||||
}
|
||||
}
|
||||
hex[w]=0;
|
||||
printf("EMAP %d %d %s\n",rows,cols,hex); fflush(stdout); free(hex);
|
||||
}
|
||||
|
||||
/* HITS: bitmap 1bit/expert (stesso ordine di EMAP), poi azzera per il turno dopo. */
|
||||
static void hits_emit(Model *m){
|
||||
Cfg *c=&m->c; if(!g_ehit) return;
|
||||
int rows=0;
|
||||
for(int i=0;i<c->n_layers;i++) if(m->L[i].sparse) rows++;
|
||||
int has_mtp = m->has_mtp && m->eusage[c->n_layers];
|
||||
if(has_mtp) rows++;
|
||||
int cols=c->n_experts, nb=(rows*cols+7)/8;
|
||||
uint8_t *bm=calloc(nb,1); int bit=0;
|
||||
for(int i=0;i<=c->n_layers;i++){
|
||||
int is_row = (i<c->n_layers && m->L[i].sparse) || (i==c->n_layers && has_mtp);
|
||||
if(!is_row) continue;
|
||||
for(int e=0;e<cols;e++,bit++)
|
||||
if(g_ehit[i][e]){ bm[bit>>3]|=1<<(bit&7); g_ehit[i][e]=0; }
|
||||
}
|
||||
char *hex=malloc((size_t)nb*2+1); int w=0;
|
||||
for(int b=0;b<nb;b++){ hex[w++]="0123456789abcdef"[bm[b]>>4]; hex[w++]="0123456789abcdef"[bm[b]&15]; }
|
||||
hex[w]=0;
|
||||
printf("HITS %d %d %s\n",rows,cols,hex); fflush(stdout); free(hex); free(bm);
|
||||
}
|
||||
|
||||
/* scarica su file l'istogramma d'uso degli expert: righe "layer eid count" (per PIN).
|
||||
* Include la riga MTP (layer n_layers). Scrittura atomica (tmp+rename): viene chiamata
|
||||
* anche a ogni turno di serve e il processo puo' morire in qualsiasi momento. */
|
||||
|
||||
+71
-2
@@ -7,6 +7,7 @@ import collections
|
||||
import contextlib
|
||||
import json
|
||||
import math
|
||||
import mimetypes
|
||||
import os
|
||||
import select
|
||||
import queue
|
||||
@@ -27,6 +28,8 @@ END = b"\x01\x01END\x01\x01\n"
|
||||
READY = b"\x01\x01READY\x01\x01\n"
|
||||
MAX_BODY = 4 << 20
|
||||
DEFAULT_CORS_ORIGINS = (
|
||||
"http://127.0.0.1:8000",
|
||||
"http://localhost:8000",
|
||||
"http://127.0.0.1:5173",
|
||||
"http://localhost:5173",
|
||||
"http://tauri.localhost",
|
||||
@@ -462,6 +465,11 @@ class Engine:
|
||||
self.closed = False
|
||||
self.dispatcher_error = None
|
||||
self.kv_slots = kv_slots
|
||||
self.tiers = None
|
||||
self.hwinfo = None
|
||||
self.emap = None
|
||||
self.hits = None
|
||||
self.hits_seq = 0 # latest "TIERS" snapshot from the engine
|
||||
read_engine_turn(self.process.stdout, READY, lambda _: None)
|
||||
self.dispatcher = threading.Thread(target=self._dispatch_stdout,
|
||||
name="colibri-stdout", daemon=True)
|
||||
@@ -527,6 +535,22 @@ class Engine:
|
||||
events = self.pending.pop(request_id, None)
|
||||
if events is not None:
|
||||
events.put(("done", stats))
|
||||
elif kind == "HWINFO" and len(fields) >= 7:
|
||||
parts = " ".join(fields[6:]).split("|")
|
||||
self.hwinfo = {"cores": int(fields[1]), "ram_total_gb": float(fields[2]),
|
||||
"ram_avail_gb": float(fields[3]), "gpus": int(fields[4]),
|
||||
"vram_total_gb": float(fields[5]),
|
||||
"cpu": parts[0].strip() if len(parts)>0 else "",
|
||||
"gpu": parts[1].strip() if len(parts)>1 else ""}
|
||||
elif kind == "EMAP" and len(fields) == 4:
|
||||
self.emap = {"rows": int(fields[1]), "cols": int(fields[2]), "map": fields[3]}
|
||||
elif kind == "HITS" and len(fields) == 4:
|
||||
self.hits = fields[3]
|
||||
self.hits_seq += 1
|
||||
elif kind == "TIERS" and len(fields) >= 6:
|
||||
self.tiers = {"vram": int(fields[1]), "ram": int(fields[2]),
|
||||
"disk": int(fields[3]), "vram_gb": float(fields[4]),
|
||||
"ram_gb": float(fields[5])}
|
||||
elif kind == "ERROR" and len(fields) >= 2:
|
||||
request_id = fields[1]
|
||||
message = " ".join(fields[2:]) or "engine request failed"
|
||||
@@ -700,13 +724,58 @@ class APIHandler(BaseHTTPRequestHandler):
|
||||
if model != self.server.model_id:
|
||||
raise APIError(404, f"The model `{model}` does not exist.", "model", "model_not_found")
|
||||
|
||||
WEB_DIST = Path(__file__).resolve().parent.parent / "web" / "dist"
|
||||
|
||||
def serve_static(self, path):
|
||||
"""Serve the built web UI (web/dist) so `coli web` is one process.
|
||||
Read-only, no auth (same trust level as /health), traversal-safe."""
|
||||
if path.startswith("/v1/") or path == "/health":
|
||||
return False
|
||||
base = self.WEB_DIST
|
||||
if not base.is_dir():
|
||||
return False
|
||||
rel = unquote(path).lstrip("/") or "index.html"
|
||||
target = (base / rel).resolve()
|
||||
if not str(target).startswith(str(base)) or not target.is_file():
|
||||
if path == "/" or "." not in rel: # SPA fallback
|
||||
target = base / "index.html"
|
||||
if not target.is_file():
|
||||
return False
|
||||
else:
|
||||
return False
|
||||
ctype = mimetypes.guess_type(str(target))[0] or "application/octet-stream"
|
||||
data = target.read_bytes()
|
||||
self.send_response(200)
|
||||
self.send_header("Content-Type", ctype)
|
||||
self.send_header("Content-Length", str(len(data)))
|
||||
self.send_cors_headers()
|
||||
self.end_headers()
|
||||
self.wfile.write(data)
|
||||
return True
|
||||
|
||||
def do_GET(self):
|
||||
request_id = "req_" + uuid.uuid4().hex
|
||||
try:
|
||||
path = urlsplit(self.path).path
|
||||
if path == "/health":
|
||||
self.send_json(200, {"status": "ok", "scheduler": self.server.scheduler.snapshot(),
|
||||
"kv_slots": self.server.kv_slots}, request_id)
|
||||
payload = {"status": "ok", "scheduler": self.server.scheduler.snapshot(),
|
||||
"kv_slots": self.server.kv_slots}
|
||||
tiers = getattr(self.server.engine, "tiers", None) if self.server.engine else None
|
||||
if tiers: payload["tiers"] = tiers
|
||||
hwinfo = getattr(self.server.engine, "hwinfo", None) if self.server.engine else None
|
||||
if hwinfo: payload["hwinfo"] = hwinfo
|
||||
self.send_json(200, payload, request_id)
|
||||
return
|
||||
if path == "/experts":
|
||||
eng = self.server.engine
|
||||
payload = {"rows": 0, "cols": 0, "map": "", "hits": "", "seq": 0}
|
||||
if eng and getattr(eng, "emap", None):
|
||||
payload.update(eng.emap)
|
||||
payload["hits"] = eng.hits or ""
|
||||
payload["seq"] = eng.hits_seq
|
||||
self.send_json(200, payload, request_id)
|
||||
return
|
||||
if self.serve_static(path):
|
||||
return
|
||||
self.require_auth()
|
||||
if path == "/v1/models":
|
||||
|
||||
Generated
+56
-23
@@ -11,15 +11,15 @@
|
||||
"class-variance-authority": "^0.7.1",
|
||||
"clsx": "^2.1.1",
|
||||
"lucide-react": "^1.24.0",
|
||||
"react": "^19.2.7",
|
||||
"react-dom": "^19.2.7",
|
||||
"react": "^18.3.1",
|
||||
"react-dom": "^18.3.1",
|
||||
"tailwind-merge": "^3.6.0"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@tailwindcss/vite": "^4.3.2",
|
||||
"@types/node": "^26.1.1",
|
||||
"@types/react": "^19.2.17",
|
||||
"@types/react-dom": "^19.2.3",
|
||||
"@types/react": "^18.3.31",
|
||||
"@types/react-dom": "^18.3.7",
|
||||
"@vitejs/plugin-react": "^6.0.3",
|
||||
"tailwindcss": "^4.3.2",
|
||||
"typescript": "^7.0.2",
|
||||
@@ -759,24 +759,32 @@
|
||||
"undici-types": "~8.3.0"
|
||||
}
|
||||
},
|
||||
"node_modules/@types/prop-types": {
|
||||
"version": "15.7.15",
|
||||
"resolved": "https://registry.npmjs.org/@types/prop-types/-/prop-types-15.7.15.tgz",
|
||||
"integrity": "sha512-F6bEyamV9jKGAFBEmlQnesRPGOQqS2+Uwi0Em15xenOxHaf2hv6L8YCVn3rPdPJOiJfPiCnLIRyvwVaqMY3MIw==",
|
||||
"dev": true,
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/@types/react": {
|
||||
"version": "19.2.17",
|
||||
"resolved": "https://registry.npmjs.org/@types/react/-/react-19.2.17.tgz",
|
||||
"integrity": "sha512-MXfmqaVPEVgkBT/aY0aGCkRWWtByiYQXo3xdQ8r5RzuFrPiRn8Gar2tQdXSUQ2GKV3bkXckek89V8wQBY2Q/Aw==",
|
||||
"version": "18.3.31",
|
||||
"resolved": "https://registry.npmjs.org/@types/react/-/react-18.3.31.tgz",
|
||||
"integrity": "sha512-vfEqpXTvwT91yhmwdfouStN2hSKwTvyRs8qpLfADyrq/kxDw0hZM7Wk9Ug1FELj8hIby+S/+kQCSRFF32nv2Qw==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"@types/prop-types": "*",
|
||||
"csstype": "^3.2.2"
|
||||
}
|
||||
},
|
||||
"node_modules/@types/react-dom": {
|
||||
"version": "19.2.3",
|
||||
"resolved": "https://registry.npmjs.org/@types/react-dom/-/react-dom-19.2.3.tgz",
|
||||
"integrity": "sha512-jp2L/eY6fn+KgVVQAOqYItbF0VY/YApe5Mz2F0aykSO8gx31bYCZyvSeYxCHKvzHG5eZjc+zyaS5BrBWya2+kQ==",
|
||||
"version": "18.3.7",
|
||||
"resolved": "https://registry.npmjs.org/@types/react-dom/-/react-dom-18.3.7.tgz",
|
||||
"integrity": "sha512-MEe3UeoENYVFXzoXEWsvcpg6ZvlrFNlOQ7EOsvhI3CfAXwzPfO8Qwuxd40nepsYKqyyVQnTdEfv68q91yLcKrQ==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"peerDependencies": {
|
||||
"@types/react": "^19.2.0"
|
||||
"@types/react": "^18.0.0"
|
||||
}
|
||||
},
|
||||
"node_modules/@typescript/typescript-aix-ppc64": {
|
||||
@@ -1414,6 +1422,12 @@
|
||||
"jiti": "lib/jiti-cli.mjs"
|
||||
}
|
||||
},
|
||||
"node_modules/js-tokens": {
|
||||
"version": "4.0.0",
|
||||
"resolved": "https://registry.npmjs.org/js-tokens/-/js-tokens-4.0.0.tgz",
|
||||
"integrity": "sha512-RdJUflcE3cUzKiMqQgsCu06FPu9UdIJO0beYbPhHN4k6apgJtifcoCtT9bcxOpYBtpD2kCM6Sbzg4CausW/PKQ==",
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/lightningcss": {
|
||||
"version": "1.32.0",
|
||||
"resolved": "https://registry.npmjs.org/lightningcss/-/lightningcss-1.32.0.tgz",
|
||||
@@ -1687,6 +1701,18 @@
|
||||
"url": "https://opencollective.com/parcel"
|
||||
}
|
||||
},
|
||||
"node_modules/loose-envify": {
|
||||
"version": "1.4.0",
|
||||
"resolved": "https://registry.npmjs.org/loose-envify/-/loose-envify-1.4.0.tgz",
|
||||
"integrity": "sha512-lyuxPGr/Wfhrlem2CL/UcnUc1zcqKAImBDzukY7Y5F/yQiNdko6+fRLevlw1HgMySw7f611UIY408EtxRSoK3Q==",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"js-tokens": "^3.0.0 || ^4.0.0"
|
||||
},
|
||||
"bin": {
|
||||
"loose-envify": "cli.js"
|
||||
}
|
||||
},
|
||||
"node_modules/lucide-react": {
|
||||
"version": "1.24.0",
|
||||
"resolved": "https://registry.npmjs.org/lucide-react/-/lucide-react-1.24.0.tgz",
|
||||
@@ -1796,24 +1822,28 @@
|
||||
}
|
||||
},
|
||||
"node_modules/react": {
|
||||
"version": "19.2.7",
|
||||
"resolved": "https://registry.npmjs.org/react/-/react-19.2.7.tgz",
|
||||
"integrity": "sha512-HNe9WslTbXmFK8o8cmwgAeJFSBvt1bPdHCVKtaaV+WlAN36mpT4hcRpwbf3fY56ar2oIXzsBpOAiIRHAdY0OlQ==",
|
||||
"version": "18.3.1",
|
||||
"resolved": "https://registry.npmjs.org/react/-/react-18.3.1.tgz",
|
||||
"integrity": "sha512-wS+hAgJShR0KhEvPJArfuPVN1+Hz1t0Y6n5jLrGQbkb4urgPE/0Rve+1kMB1v/oWgHgm4WIcV+i7F2pTVj+2iQ==",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"loose-envify": "^1.1.0"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">=0.10.0"
|
||||
}
|
||||
},
|
||||
"node_modules/react-dom": {
|
||||
"version": "19.2.7",
|
||||
"resolved": "https://registry.npmjs.org/react-dom/-/react-dom-19.2.7.tgz",
|
||||
"integrity": "sha512-t0BRVXvbiE/o20Hfw669rLbMCDWtYZLvmJigy2f0MxsXF+71pxhR3xOkspmsO8h3ZlNzyibAmtCa3l4lYKk6gQ==",
|
||||
"version": "18.3.1",
|
||||
"resolved": "https://registry.npmjs.org/react-dom/-/react-dom-18.3.1.tgz",
|
||||
"integrity": "sha512-5m4nQKp+rZRb09LNH59GM4BxTh9251/ylbKIbpe7TpGxfJ+9kv6BLkLBXIjjspbgbnIBNqlI23tRnTWT0snUIw==",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"scheduler": "^0.27.0"
|
||||
"loose-envify": "^1.1.0",
|
||||
"scheduler": "^0.23.2"
|
||||
},
|
||||
"peerDependencies": {
|
||||
"react": "^19.2.7"
|
||||
"react": "^18.3.1"
|
||||
}
|
||||
},
|
||||
"node_modules/rolldown": {
|
||||
@@ -1851,10 +1881,13 @@
|
||||
}
|
||||
},
|
||||
"node_modules/scheduler": {
|
||||
"version": "0.27.0",
|
||||
"resolved": "https://registry.npmjs.org/scheduler/-/scheduler-0.27.0.tgz",
|
||||
"integrity": "sha512-eNv+WrVbKu1f3vbYJT/xtiF5syA5HPIMtf9IgY/nKg0sWqzAUEvqY/xm7OcZc/qafLx/iO9FgOmeSAp4v5ti/Q==",
|
||||
"license": "MIT"
|
||||
"version": "0.23.2",
|
||||
"resolved": "https://registry.npmjs.org/scheduler/-/scheduler-0.23.2.tgz",
|
||||
"integrity": "sha512-UOShsPwz7NrMUqhR6t0hWjFduvOzbtv7toDH1/hIrfRNIDBnnBWd0CwJTGvTpngVlmwGCdP9/Zl/tVrDqcuYzQ==",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"loose-envify": "^1.1.0"
|
||||
}
|
||||
},
|
||||
"node_modules/siginfo": {
|
||||
"version": "2.0.0",
|
||||
|
||||
+4
-4
@@ -13,15 +13,15 @@
|
||||
"class-variance-authority": "^0.7.1",
|
||||
"clsx": "^2.1.1",
|
||||
"lucide-react": "^1.24.0",
|
||||
"react": "^19.2.7",
|
||||
"react-dom": "^19.2.7",
|
||||
"react": "^18.3.1",
|
||||
"react-dom": "^18.3.1",
|
||||
"tailwind-merge": "^3.6.0"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@tailwindcss/vite": "^4.3.2",
|
||||
"@types/node": "^26.1.1",
|
||||
"@types/react": "^19.2.17",
|
||||
"@types/react-dom": "^19.2.3",
|
||||
"@types/react": "^18.3.31",
|
||||
"@types/react-dom": "^18.3.7",
|
||||
"@vitejs/plugin-react": "^6.0.3",
|
||||
"tailwindcss": "^4.3.2",
|
||||
"typescript": "^7.0.2",
|
||||
|
||||
+105
-6
@@ -4,15 +4,24 @@ import {
|
||||
ArrowUp,
|
||||
BrainCircuit,
|
||||
CircleStop,
|
||||
Clock,
|
||||
Cpu,
|
||||
Database,
|
||||
Feather,
|
||||
Gauge,
|
||||
HardDrive,
|
||||
KeyRound,
|
||||
Layers,
|
||||
Link2,
|
||||
LoaderCircle,
|
||||
MemoryStick,
|
||||
MessageSquareText,
|
||||
MonitorDot,
|
||||
RefreshCw,
|
||||
SlidersHorizontal,
|
||||
Timer,
|
||||
Trash2,
|
||||
Zap,
|
||||
} from "lucide-react"
|
||||
|
||||
import { Badge } from "@/components/ui/badge"
|
||||
@@ -21,13 +30,25 @@ import { Input } from "@/components/ui/input"
|
||||
import { Textarea } from "@/components/ui/textarea"
|
||||
import { getHealth, listModels, streamChat, type ChatMessage, type HealthResponse, type StreamChatResult } from "@/lib/api"
|
||||
import { activeRequests, supportsCacheSlots } from "@/lib/runtime"
|
||||
import { Brain } from "./Brain"
|
||||
import { persistPublicSettings, stored } from "@/lib/storage"
|
||||
import { cn } from "@/lib/utils"
|
||||
|
||||
const message = (role: ChatMessage["role"], content: string): ChatMessage => ({ id: crypto.randomUUID(), role, content })
|
||||
|
||||
export default function App() {
|
||||
const [baseUrl, setBaseUrl] = useState(() => stored(localStorage, "colibri.baseUrl", "http://127.0.0.1:8000/v1"))
|
||||
// When the page is served by the engine itself (coli web), same-origin is the
|
||||
// right default: no CORS, no manual endpoint editing. The Vite dev server
|
||||
// (port 5173) keeps the classic default.
|
||||
const servedByEngine = typeof window !== "undefined" && window.location.port !== "5173" && window.location.protocol.startsWith("http")
|
||||
const defaultBase = servedByEngine ? `${window.location.origin}/v1` : "http://127.0.0.1:8000/v1"
|
||||
const [baseUrl, setBaseUrl] = useState(() => {
|
||||
const saved = stored(localStorage, "colibri.baseUrl", defaultBase)
|
||||
// migrate: a stored FACTORY default pointing at another origin would trip CORS
|
||||
// when the page is engine-served — upgrade it to same-origin once.
|
||||
if (servedByEngine && saved === "http://127.0.0.1:8000/v1" && defaultBase !== saved) return defaultBase
|
||||
return saved
|
||||
})
|
||||
const [apiKey, setApiKey] = useState("")
|
||||
const [models, setModels] = useState<string[]>([])
|
||||
const [model, setModel] = useState(() => stored(localStorage, "colibri.model", "glm-5.2-colibri"))
|
||||
@@ -41,9 +62,16 @@ export default function App() {
|
||||
const [lastRun, setLastRun] = useState<StreamChatResult | null>(null)
|
||||
const [draft, setDraft] = useState("")
|
||||
const [loading, setLoading] = useState(false)
|
||||
const [streamStart, setStreamStart] = useState<number | null>(null)
|
||||
const [tokenCount, setTokenCount] = useState(0)
|
||||
const [tokPerSec, setTokPerSec] = useState<number | null>(null)
|
||||
const [ttft, setTtft] = useState<number | null>(null)
|
||||
const [totalTokens, setTotalTokens] = useState({ prompt: 0, completion: 0 })
|
||||
const [connecting, setConnecting] = useState(false)
|
||||
const [connected, setConnected] = useState(false)
|
||||
const [view, setView] = useState<"chat" | "brain">("chat")
|
||||
const [error, setError] = useState("")
|
||||
const autoConnected = useRef(false)
|
||||
const abortRef = useRef<AbortController | null>(null)
|
||||
const probeRef = useRef<AbortController | null>(null)
|
||||
const bottomRef = useRef<HTMLDivElement>(null)
|
||||
@@ -59,21 +87,25 @@ export default function App() {
|
||||
[cacheSlot]: typeof next === "function" ? next(current[cacheSlot] || []) : next,
|
||||
}))
|
||||
|
||||
// EFFECT #1
|
||||
useEffect(() => {
|
||||
persistPublicSettings(localStorage, baseUrl, model)
|
||||
}, [baseUrl, model])
|
||||
|
||||
// EFFECT #2
|
||||
useEffect(() => {
|
||||
setConnected(false)
|
||||
setHealth(null)
|
||||
setHealthError("")
|
||||
}, [baseUrl, apiKey])
|
||||
|
||||
// EFFECT #3
|
||||
useEffect(() => () => {
|
||||
probeRef.current?.abort()
|
||||
abortRef.current?.abort()
|
||||
}, [])
|
||||
|
||||
// EFFECT #4
|
||||
useEffect(() => {
|
||||
if (!connected) return
|
||||
let disposed = false
|
||||
@@ -90,13 +122,16 @@ export default function App() {
|
||||
return () => { disposed = true; window.clearInterval(timer) }
|
||||
}, [apiKey, baseUrl, connected])
|
||||
|
||||
// EFFECT #5
|
||||
useEffect(() => {
|
||||
if (cacheSlot >= kvSlots) setCacheSlot(0)
|
||||
}, [cacheSlot, kvSlots])
|
||||
|
||||
useEffect(() => setLastRun(null), [cacheSlot])
|
||||
// EFFECT #6
|
||||
useEffect(() => { setLastRun(null) }, [cacheSlot])
|
||||
|
||||
useEffect(() => bottomRef.current?.scrollIntoView({ behavior: "smooth" }), [messages])
|
||||
// EFFECT #7
|
||||
useEffect(() => { bottomRef.current?.scrollIntoView({ behavior: "smooth" }) }, [messages])
|
||||
|
||||
const connect = async () => {
|
||||
probeRef.current?.abort()
|
||||
@@ -127,6 +162,11 @@ export default function App() {
|
||||
}
|
||||
}
|
||||
|
||||
if (servedByEngine && !autoConnected.current && !connected) {
|
||||
autoConnected.current = true
|
||||
setTimeout(() => connect(), 0)
|
||||
}
|
||||
|
||||
const canSend = useMemo(() => draft.trim() && model && !loading, [draft, loading, model])
|
||||
|
||||
const send = async () => {
|
||||
@@ -139,6 +179,13 @@ export default function App() {
|
||||
setError("")
|
||||
updateMessages([...history, assistant])
|
||||
setLoading(true)
|
||||
setStreamStart(null)
|
||||
setTokenCount(0)
|
||||
setTokPerSec(null)
|
||||
setTtft(null)
|
||||
const t0 = performance.now()
|
||||
let firstToken = true
|
||||
let count = 0
|
||||
const controller = new AbortController()
|
||||
abortRef.current = controller
|
||||
try {
|
||||
@@ -152,11 +199,23 @@ export default function App() {
|
||||
enableThinking: thinking,
|
||||
cacheSlot: supportsCacheSlots(health) ? cacheSlot : undefined,
|
||||
signal: controller.signal,
|
||||
onDelta: (delta) =>
|
||||
onDelta: (delta) => {
|
||||
if (firstToken) { setTtft(performance.now() - t0); setStreamStart(performance.now()); firstToken = false }
|
||||
count++
|
||||
setTokenCount(count)
|
||||
const elapsed = (performance.now() - (firstToken ? t0 : t0)) / 1000
|
||||
if (elapsed > 0.3) setTokPerSec(count / ((performance.now() - t0) / 1000))
|
||||
updateMessages((current) => current.map((item) =>
|
||||
item.id === assistant.id ? { ...item, content: item.content + delta } : item,
|
||||
)),
|
||||
))
|
||||
},
|
||||
})
|
||||
const finalElapsed = (performance.now() - t0) / 1000
|
||||
if (count > 0 && finalElapsed > 0) setTokPerSec(count / finalElapsed)
|
||||
if (result.usage) setTotalTokens(prev => ({
|
||||
prompt: prev.prompt + (result.usage?.prompt_tokens || 0),
|
||||
completion: prev.completion + (result.usage?.completion_tokens || 0),
|
||||
}))
|
||||
setLastRun(result)
|
||||
setConnected(true)
|
||||
} catch (cause) {
|
||||
@@ -193,6 +252,12 @@ export default function App() {
|
||||
|
||||
<section className="side-section runtime-section" aria-live="polite">
|
||||
<div className="section-title"><Activity className="size-3.5" /> Runtime</div>
|
||||
{health?.hwinfo ? <div className="hw-panel">
|
||||
{health.hwinfo.cpu ? <div className="hw-row"><Cpu className="size-3.5" /><span>{health.hwinfo.cpu}</span></div> : null}
|
||||
{health.hwinfo.gpus > 0 ? <div className="hw-row"><MonitorDot className="size-3.5" /><span>{health.hwinfo.gpus}× GPU<small>{health.hwinfo.vram_total_gb.toFixed(0)} GB VRAM</small></span></div> : null}
|
||||
<div className="hw-row"><MemoryStick className="size-3.5" /><span>{health.hwinfo.ram_total_gb.toFixed(0)} GB RAM<small>{health.hwinfo.ram_avail_gb.toFixed(0)} GB free</small></span></div>
|
||||
<div className="hw-row"><HardDrive className="size-3.5" /><span>{health.hwinfo.cores} cores</span></div>
|
||||
</div> : null}
|
||||
{health?.scheduler ? <>
|
||||
<div className="runtime-grid">
|
||||
<div><span>Active</span><strong>{active}<small> / {capacity}</small></strong></div>
|
||||
@@ -200,6 +265,25 @@ export default function App() {
|
||||
<div><span>Completed</span><strong>{health.scheduler.completed}</strong></div>
|
||||
<div><span>Failures</span><strong>{failures}</strong></div>
|
||||
</div>
|
||||
{health.tiers ? (() => {
|
||||
const t = health.tiers
|
||||
const total = Math.max(t.vram + t.ram + t.disk, 1)
|
||||
return <div className="tier-panel">
|
||||
<div className="tier-bar" role="img" aria-label={`Experts: ${t.vram} VRAM, ${t.ram} RAM, ${t.disk} disk`}>
|
||||
<span className="tier-vram" style={{ width: `${(100 * t.vram) / total}%` }} />
|
||||
<span className="tier-ram" style={{ width: `${(100 * t.ram) / total}%` }} />
|
||||
<span className="tier-disk" style={{ width: `${(100 * t.disk) / total}%` }} />
|
||||
</div>
|
||||
<div className="tier-legend">
|
||||
<span><i className="tier-vram" />VRAM <strong>{t.vram.toLocaleString()}</strong><small>{t.vram_gb.toFixed(1)} GB</small></span>
|
||||
<span><i className="tier-ram" />RAM <strong>{t.ram.toLocaleString()}</strong><small>{t.ram_gb.toFixed(1)} GB</small></span>
|
||||
<span><i className="tier-disk" />Disk <strong>{t.disk.toLocaleString()}</strong></span>
|
||||
</div>
|
||||
</div>
|
||||
})() : null}
|
||||
{totalTokens.prompt + totalTokens.completion > 0 ? <div className="session-stats">
|
||||
<span><Database className="size-3" /> Session: <strong>{totalTokens.prompt.toLocaleString()}</strong> prompt + <strong>{totalTokens.completion.toLocaleString()}</strong> completion</span>
|
||||
</div> : null}
|
||||
<div className="runtime-foot"><span className="runtime-dot" /> Scheduler online <code>{kvSlots} KV</code></div>
|
||||
</> : <p className="runtime-unavailable">{connected ? (healthError || "Runtime metrics unavailable") : "Probe the server to inspect runtime state."}</p>}
|
||||
</section>
|
||||
@@ -223,9 +307,23 @@ export default function App() {
|
||||
<main className="chat-panel">
|
||||
<header className="topbar">
|
||||
<div><span className="eyebrow">ACTIVE MODEL</span><strong>{model}</strong></div>
|
||||
<div className="top-actions">{lastRun?.queueWaitMs != null ? <Badge>queue {Math.round(lastRun.queueWaitMs)}ms</Badge> : null}<Badge><Activity className="size-3" /> slot {cacheSlot + 1}</Badge><Button variant="ghost" size="sm" onClick={() => updateMessages([])} disabled={!messages.length || loading}><Trash2 className="size-3.5" /> Clear</Button></div>
|
||||
<div className="view-tabs">
|
||||
<button className={view === "chat" ? "active" : ""} onClick={() => setView("chat")}><MessageSquareText className="size-3.5" /> Chat</button>
|
||||
<button className={view === "brain" ? "active" : ""} onClick={() => setView("brain")}><BrainCircuit className="size-3.5" /> Brain</button>
|
||||
</div>
|
||||
<div className="top-actions">
|
||||
{loading && tokenCount > 0 ? <Badge className="badge-live"><Zap className="size-3 flash" /> {tokenCount} tokens</Badge> : null}
|
||||
{!loading && tokPerSec != null ? <Badge className="badge-speed"><Gauge className="size-3" /> {tokPerSec.toFixed(1)} tok/s</Badge> : null}
|
||||
{!loading && ttft != null ? <Badge><Timer className="size-3" /> TTFT {(ttft/1000).toFixed(1)}s</Badge> : null}
|
||||
{!loading && lastRun?.usage ? <Badge><Layers className="size-3" /> {lastRun.usage.prompt_tokens}→{lastRun.usage.completion_tokens}</Badge> : null}
|
||||
{lastRun?.queueWaitMs != null ? <Badge><Clock className="size-3" /> queue {Math.round(lastRun.queueWaitMs)}ms</Badge> : null}
|
||||
<Badge><MonitorDot className="size-3" /> slot {cacheSlot + 1}</Badge>
|
||||
<Button variant="ghost" size="sm" onClick={() => { updateMessages([]); setTokPerSec(null); setTtft(null); setTokenCount(0); setTotalTokens({prompt:0,completion:0}) }} disabled={!messages.length || loading}><Trash2 className="size-3.5" /> Clear</Button>
|
||||
</div>
|
||||
</header>
|
||||
|
||||
{view === "brain" ? <Brain baseUrl={baseUrl} apiKey={apiKey} connected={connected} /> : <>
|
||||
|
||||
<div className="conversation">
|
||||
{!messages.length ? (
|
||||
<div className="empty-state">
|
||||
@@ -257,6 +355,7 @@ export default function App() {
|
||||
<div className="composer-foot"><span><MessageSquareText className="size-3.5" /> Enter to send · Shift+Enter for newline</span>{loading ? <Button variant="destructive" size="icon" aria-label="Stop generation" onClick={() => abortRef.current?.abort()}><CircleStop className="size-4" /></Button> : <Button size="icon" aria-label="Send message" disabled={!canSend} onClick={() => void send()}><ArrowUp className="size-4" /></Button>}</div>
|
||||
</div>
|
||||
</div>
|
||||
</>}
|
||||
</main>
|
||||
</div>
|
||||
)
|
||||
|
||||
@@ -0,0 +1,159 @@
|
||||
import { useEffect, useRef, useState } from "react"
|
||||
import { BrainCircuit, Flame, Layers } from "lucide-react"
|
||||
|
||||
import { endpoint } from "@/lib/api"
|
||||
|
||||
interface ExpertMap { rows: number; cols: number; map: string; hits: string; seq: number }
|
||||
|
||||
const TIER_NAME = ["Disk", "RAM", "VRAM"]
|
||||
const TIER_RGB: [number, number, number][] = [[58, 71, 80], [90, 155, 216], [78, 214, 165]]
|
||||
|
||||
/* Layer-depth heuristic: what this region of the network tends to specialise in.
|
||||
* Honest framing — these are the depth roles observed across MoE interpretability
|
||||
* work, not per-expert ground truth (that needs co-activation analysis, #119). */
|
||||
function depthRole(row: number, rows: number, isMtp: boolean): string {
|
||||
if (isMtp) return "MTP head — drafts the next token for speculative decoding"
|
||||
const f = row / Math.max(rows - 1, 1)
|
||||
if (f < 0.2) return "early layers — surface features: tokens, spelling, local syntax"
|
||||
if (f < 0.45) return "lower-middle — phrase structure, word relations, simple facts"
|
||||
if (f < 0.7) return "upper-middle — semantics, long-range context, reasoning steps"
|
||||
if (f < 0.9) return "late layers — planning the answer, style, coherence"
|
||||
return "final layers — output shaping: picks the actual next-token distribution"
|
||||
}
|
||||
|
||||
export function Brain({ baseUrl, apiKey, connected }: { baseUrl: string; apiKey: string; connected: boolean }) {
|
||||
const canvasRef = useRef<HTMLCanvasElement>(null)
|
||||
const wrapRef = useRef<HTMLDivElement>(null)
|
||||
const [wrapSize, setWrapSize] = useState({ w: 1200, h: 700 })
|
||||
const [data, setData] = useState<ExpertMap | null>(null)
|
||||
const [tip, setTip] = useState<{ x: number; y: number; row: number; col: number; tier: number; heat: number } | null>(null)
|
||||
const pulseRef = useRef<Float32Array | null>(null) // per-expert pulse intensity 0..1
|
||||
const lastSeq = useRef(0)
|
||||
const rafRef = useRef(0)
|
||||
|
||||
// track container size for responsive cell sizing
|
||||
useEffect(() => {
|
||||
const el = wrapRef.current
|
||||
if (!el) return
|
||||
const ro = new ResizeObserver(() => {
|
||||
setWrapSize({ w: el.clientWidth - 24, h: el.clientHeight - 24 })
|
||||
})
|
||||
ro.observe(el)
|
||||
return () => ro.disconnect()
|
||||
}, [])
|
||||
|
||||
// poll /experts
|
||||
useEffect(() => {
|
||||
if (!connected) return
|
||||
let disposed = false
|
||||
const base = baseUrl.replace(/\/v1\/?$/, "")
|
||||
const poll = async () => {
|
||||
try {
|
||||
const res = await fetch(endpoint(base, "/experts"), { headers: apiKey ? { Authorization: `Bearer ${apiKey}` } : {} })
|
||||
const next = (await res.json()) as ExpertMap
|
||||
if (disposed || !next.rows) return
|
||||
setData(next)
|
||||
if (next.seq !== lastSeq.current && next.hits) {
|
||||
lastSeq.current = next.seq
|
||||
const n = next.rows * next.cols
|
||||
if (!pulseRef.current || pulseRef.current.length !== n) pulseRef.current = new Float32Array(n)
|
||||
const p = pulseRef.current
|
||||
for (let i = 0; i < n; i++) {
|
||||
const byte = parseInt(next.hits.substr((i >> 3) * 2, 2), 16) || 0
|
||||
if (byte & (1 << (i & 7))) p[i] = 1
|
||||
}
|
||||
}
|
||||
} catch { /* engine busy or restarting — keep the last frame */ }
|
||||
}
|
||||
void poll()
|
||||
const t = window.setInterval(() => void poll(), 1500)
|
||||
return () => { disposed = true; window.clearInterval(t) }
|
||||
}, [baseUrl, apiKey, connected])
|
||||
|
||||
// render loop: grid + decaying pulses
|
||||
useEffect(() => {
|
||||
const canvas = canvasRef.current
|
||||
if (!canvas || !data) return
|
||||
const ctx = canvas.getContext("2d")
|
||||
if (!ctx) return
|
||||
const { rows, cols, map } = data
|
||||
const cell = Math.max(2, Math.floor(Math.min(wrapSize.w / cols, wrapSize.h / rows)))
|
||||
const gap = cell >= 4 ? 1 : 0
|
||||
canvas.width = cols * (cell + gap)
|
||||
canvas.height = rows * (cell + gap)
|
||||
|
||||
const draw = () => {
|
||||
ctx.clearRect(0, 0, canvas.width, canvas.height)
|
||||
const p = pulseRef.current
|
||||
for (let r = 0; r < rows; r++) {
|
||||
for (let c = 0; c < cols; c++) {
|
||||
const i = r * cols + c
|
||||
const byte = parseInt(map.substr(i * 2, 2), 16) || 0
|
||||
const tier = byte >> 6
|
||||
const heat = byte & 63
|
||||
const [R, G, B] = TIER_RGB[tier] ?? TIER_RGB[0]
|
||||
// heat scales brightness: cold experts dim, hot experts full colour
|
||||
const lum = 0.35 + 0.65 * Math.min(heat / 24, 1)
|
||||
let rr = R * lum, gg = G * lum, bb = B * lum
|
||||
const pulse = p ? p[i] : 0
|
||||
if (pulse > 0.01) { rr += (255 - rr) * pulse; gg += (255 - gg) * pulse; bb += (255 - bb) * pulse }
|
||||
ctx.fillStyle = `rgb(${rr | 0},${gg | 0},${bb | 0})`
|
||||
ctx.fillRect(c * (cell + gap), r * (cell + gap), cell, cell)
|
||||
}
|
||||
}
|
||||
let alive = false
|
||||
if (p) for (let i = 0; i < p.length; i++) { if (p[i] > 0.01) { p[i] *= 0.94; alive = true } else p[i] = 0 }
|
||||
if (alive) rafRef.current = requestAnimationFrame(draw)
|
||||
}
|
||||
draw()
|
||||
const keepalive = window.setInterval(() => { if (!rafRef.current) draw(); rafRef.current = 0 }, 400)
|
||||
return () => { cancelAnimationFrame(rafRef.current); window.clearInterval(keepalive) }
|
||||
}, [data, wrapSize])
|
||||
|
||||
const onMove = (e: React.MouseEvent<HTMLCanvasElement>) => {
|
||||
if (!data) return
|
||||
const rect = e.currentTarget.getBoundingClientRect()
|
||||
const scaleX = e.currentTarget.width / rect.width
|
||||
const scaleY = e.currentTarget.height / rect.height
|
||||
const cell = Math.max(2, Math.floor(Math.min(wrapSize.w / data.cols, wrapSize.h / data.rows)))
|
||||
const gap = cell >= 4 ? 1 : 0
|
||||
const col = Math.floor(((e.clientX - rect.left) * scaleX) / (cell + gap))
|
||||
const row = Math.floor(((e.clientY - rect.top) * scaleY) / (cell + gap))
|
||||
if (row < 0 || row >= data.rows || col < 0 || col >= data.cols) { setTip(null); return }
|
||||
const byte = parseInt(data.map.substr((row * data.cols + col) * 2, 2), 16) || 0
|
||||
setTip({ x: e.clientX, y: e.clientY, row, col, tier: byte >> 6, heat: byte & 63 })
|
||||
}
|
||||
|
||||
const totals = data ? (() => {
|
||||
const t = [0, 0, 0]
|
||||
for (let i = 0; i < data.rows * data.cols; i++) t[(parseInt(data.map.substr(i * 2, 2), 16) || 0) >> 6]++
|
||||
return t
|
||||
})() : [0, 0, 0]
|
||||
|
||||
return (
|
||||
<div className="brain-page">
|
||||
<div className="brain-head">
|
||||
<div className="section-title"><BrainCircuit className="size-4" /> Expert Cortex — {data ? `${data.rows} layers × ${data.cols} experts` : "waiting for engine"}</div>
|
||||
<div className="brain-legend">
|
||||
<span><i style={{ background: "#4ed6a5" }} /> VRAM {totals[2].toLocaleString()}</span>
|
||||
<span><i style={{ background: "#5a9bd8" }} /> RAM {totals[1].toLocaleString()}</span>
|
||||
<span><i style={{ background: "#3a4750" }} /> Disk {totals[0].toLocaleString()}</span>
|
||||
<span><Flame className="size-3" /> brightness = routing heat</span>
|
||||
<span className="brain-pulse-hint">⚡ white flash = routed this turn</span>
|
||||
</div>
|
||||
</div>
|
||||
<div className="brain-canvas-wrap" ref={wrapRef}>
|
||||
<canvas ref={canvasRef} onMouseMove={onMove} onMouseLeave={() => setTip(null)} />
|
||||
{!connected && <p className="runtime-unavailable">Connect to the engine to see the cortex.</p>}
|
||||
</div>
|
||||
{tip && data && (
|
||||
<div className="brain-tip" style={{ left: tip.x + 14, top: tip.y + 14 }}>
|
||||
<div className="brain-tip-title"><Layers className="size-3" /> Layer row {tip.row}{tip.row === data.rows - 1 ? " (MTP)" : ""} · Expert {tip.col}</div>
|
||||
<div>Tier: <strong style={{ color: ["#8b9aa3", "#5a9bd8", "#4ed6a5"][tip.tier] }}>{TIER_NAME[tip.tier]}</strong></div>
|
||||
<div>Heat: <strong>{tip.heat === 0 ? "never routed" : `~2^${tip.heat} selections`}</strong></div>
|
||||
<div className="brain-tip-role">{depthRole(tip.row, data.rows, tip.row === data.rows - 1)}</div>
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
)
|
||||
}
|
||||
@@ -0,0 +1,19 @@
|
||||
import { Component, type ReactNode } from "react"
|
||||
interface State { error: Error | null; stack: string }
|
||||
export class ErrorBoundary extends Component<{ children: ReactNode }, State> {
|
||||
state: State = { error: null, stack: "" }
|
||||
static getDerivedStateFromError(error: Error): Partial<State> { return { error } }
|
||||
componentDidCatch(error: Error, info: { componentStack?: string }) {
|
||||
console.error("[colibrì] render crash:", error, info.componentStack)
|
||||
this.setState({ stack: info.componentStack ?? "" })
|
||||
}
|
||||
render() {
|
||||
if (!this.state.error) return this.props.children
|
||||
return <div style={{ padding: "2rem", fontFamily: "ui-monospace, monospace", color: "#e5e7eb", background: "#0b0f10", minHeight: "100vh" }}>
|
||||
<h2 style={{ color: "#4ed6a5" }}>colibrì UI hit an error</h2>
|
||||
<p style={{ color: "#9ca3af" }}>The engine is unaffected. Try refreshing.</p>
|
||||
<pre style={{ whiteSpace: "pre-wrap", color: "#f87171" }}>{String(this.state.error)}</pre>
|
||||
<button onClick={() => this.setState({ error: null, stack: "" })} style={{ marginTop: "1rem", padding: "0.5rem 1rem", background: "#1f2937", color: "#e5e7eb", border: "1px solid #374151", borderRadius: 8, cursor: "pointer" }}>Retry</button>
|
||||
</div>
|
||||
}
|
||||
}
|
||||
@@ -86,3 +86,63 @@ button:focus-visible, input:focus-visible, textarea:focus-visible, select:focus-
|
||||
}
|
||||
@media (max-width: 560px) { .sidebar { grid-template-columns: 1fr; }.brand-row, .sidebar-foot { grid-column: auto; }.empty-state h2 { font-size: 36px; }.message-list { width: calc(100% - 28px); }.composer-foot > span { display: none; } }
|
||||
@media (prefers-reduced-motion: reduce) { *, *::before, *::after { scroll-behavior: auto !important; animation-duration: .01ms !important; animation-iteration-count: 1 !important; transition-duration: .01ms !important; } }
|
||||
|
||||
/* expert tier pyramid: where the 19k experts live right now */
|
||||
.tier-panel { display: grid; gap: 7px; }
|
||||
.tier-bar { display: flex; height: 10px; border-radius: 6px; overflow: hidden; border: 1px solid var(--border); background: var(--card); }
|
||||
.tier-bar span { display: block; min-width: 0; transition: width .6s ease; }
|
||||
.tier-vram { background: var(--primary); }
|
||||
.tier-ram { background: #5a9bd8; }
|
||||
.tier-disk { background: #3a4750; }
|
||||
.tier-legend { display: flex; gap: 12px; font-size: 10px; color: #839197; flex-wrap: wrap; }
|
||||
.tier-legend span { display: flex; align-items: center; gap: 5px; }
|
||||
.tier-legend i { width: 8px; height: 8px; border-radius: 2px; display: inline-block; }
|
||||
.tier-legend strong { font: 600 12px ui-monospace, SFMono-Regular, monospace; color: var(--foreground); }
|
||||
.tier-legend small { color: var(--muted-foreground); }
|
||||
|
||||
/* ---- rich metrics badges & animations ---- */
|
||||
.badge-live { background: rgba(78,214,165,.15); border-color: rgba(78,214,165,.4); color: #4ed6a5; }
|
||||
.badge-speed { background: rgba(90,155,216,.12); border-color: rgba(90,155,216,.35); color: #5a9bd8; }
|
||||
.flash { animation: flash 0.6s ease infinite alternate; }
|
||||
@keyframes flash { from { opacity: 1; } to { opacity: 0.3; } }
|
||||
.top-actions { display: flex; flex-wrap: wrap; gap: 6px; align-items: center; }
|
||||
.top-actions .badge-live, .top-actions .badge-speed { font-variant-numeric: tabular-nums; }
|
||||
|
||||
/* session stats row */
|
||||
.session-stats { display: flex; align-items: center; gap: 6px; font-size: 10px; color: #7f8d92; padding: 2px 0; }
|
||||
.session-stats strong { font: 600 11px ui-monospace, SFMono-Regular, monospace; color: var(--foreground); }
|
||||
|
||||
/* tier bar tweaks */
|
||||
.tier-bar { height: 12px; }
|
||||
.tier-bar span { transition: width 0.8s cubic-bezier(.4,0,.2,1); }
|
||||
.tier-legend strong { font-size: 13px; }
|
||||
|
||||
/* hardware info panel */
|
||||
.hw-panel { display: grid; gap: 5px; padding: 8px 0; border-bottom: 1px solid var(--border); margin-bottom: 8px; }
|
||||
.hw-row { display: flex; align-items: center; gap: 8px; font-size: 11px; color: var(--foreground); }
|
||||
.hw-row svg { color: var(--primary); flex-shrink: 0; }
|
||||
.hw-row small { color: var(--muted-foreground); margin-left: 4px; }
|
||||
|
||||
/* ---- Brain page ---- */
|
||||
.view-tabs { display: flex; gap: 4px; background: var(--card); border: 1px solid var(--border); border-radius: 10px; padding: 3px; }
|
||||
.view-tabs button { display: flex; align-items: center; gap: 6px; padding: 5px 14px; font-size: 12px; font-weight: 600; color: var(--muted-foreground); background: none; border: none; border-radius: 7px; cursor: pointer; }
|
||||
.view-tabs button.active { background: var(--primary); color: #08110d; }
|
||||
.brain-page { flex: 1; display: flex; flex-direction: column; gap: 12px; padding: 18px 22px; overflow: auto; }
|
||||
.brain-head { display: flex; flex-wrap: wrap; align-items: center; justify-content: space-between; gap: 10px; }
|
||||
.brain-legend { display: flex; flex-wrap: wrap; gap: 14px; font-size: 11px; color: #839197; align-items: center; }
|
||||
.brain-legend i { width: 9px; height: 9px; border-radius: 2px; display: inline-block; margin-right: 5px; vertical-align: -1px; }
|
||||
.brain-pulse-hint { color: var(--primary); }
|
||||
.brain-canvas-wrap { flex: 1; overflow: auto; border: 1px solid var(--border); border-radius: 12px; background: #07090a; padding: 10px; }
|
||||
.brain-canvas-wrap canvas { image-rendering: pixelated; max-width: 100%; cursor: crosshair; }
|
||||
.brain-tip { position: fixed; z-index: 50; background: #0d1214; border: 1px solid var(--border); border-radius: 10px; padding: 10px 12px; font-size: 11px; color: var(--foreground); pointer-events: none; max-width: 280px; box-shadow: 0 8px 24px rgba(0,0,0,.5); display: grid; gap: 4px; }
|
||||
.brain-tip-title { display: flex; align-items: center; gap: 5px; font-weight: 700; color: var(--primary); }
|
||||
.brain-tip-role { color: #8b9aa3; font-style: italic; line-height: 1.4; border-top: 1px solid var(--border); padding-top: 5px; margin-top: 2px; }
|
||||
|
||||
/* Brain responsive */
|
||||
.brain-canvas-wrap { display: flex; align-items: flex-start; justify-content: center; min-height: 300px; }
|
||||
@media (max-width: 900px) {
|
||||
.brain-page { padding: 10px; }
|
||||
.brain-head { flex-direction: column; align-items: flex-start; }
|
||||
.brain-legend { gap: 8px; font-size: 10px; }
|
||||
.brain-tip { max-width: 220px; font-size: 10px; }
|
||||
}
|
||||
|
||||
@@ -23,10 +23,30 @@ export interface SchedulerHealth {
|
||||
cancelled: number
|
||||
}
|
||||
|
||||
export interface TiersHealth {
|
||||
vram: number
|
||||
ram: number
|
||||
disk: number
|
||||
vram_gb: number
|
||||
ram_gb: number
|
||||
}
|
||||
|
||||
export interface HwinfoHealth {
|
||||
cores: number
|
||||
ram_total_gb: number
|
||||
ram_avail_gb: number
|
||||
gpus: number
|
||||
vram_total_gb: number
|
||||
cpu: string
|
||||
gpu: string
|
||||
}
|
||||
|
||||
export interface HealthResponse {
|
||||
status: string
|
||||
scheduler?: SchedulerHealth
|
||||
kv_slots?: number
|
||||
tiers?: TiersHealth
|
||||
hwinfo?: HwinfoHealth
|
||||
}
|
||||
|
||||
export interface TokenUsage {
|
||||
|
||||
+3
-3
@@ -1,11 +1,11 @@
|
||||
import { StrictMode } from "react"
|
||||
import { createRoot } from "react-dom/client"
|
||||
|
||||
import App from "./App"
|
||||
import { ErrorBoundary } from "./ErrorBoundary"
|
||||
import "./index.css"
|
||||
|
||||
createRoot(document.getElementById("root")!).render(
|
||||
<StrictMode>
|
||||
<ErrorBoundary>
|
||||
<App />
|
||||
</StrictMode>,
|
||||
</ErrorBoundary>,
|
||||
)
|
||||
|
||||
Reference in New Issue
Block a user