web dashboard: live expert-tier panel (VRAM/RAM/disk), one-process hosting, coli web (#126)
* Web dashboard: live expert-tier panel (VRAM/RAM/disk), static hosting, coli web command - Engine: TIERS protocol line (counts + GB for VRAM/RAM/disk experts) emitted at READY and refreshed after every served turn, computed from the live pin/LRU/CUDA state. - Server: dispatcher tracks the latest snapshot, /health exposes it as "tiers"; the built web UI (web/dist) is served from the same port (read-only, traversal-safe, SPA fallback) so the dashboard needs no second process. - Web: tier bar + legend in the Runtime panel showing where the 19k experts live right now, with GB per tier; updates with the existing health polling. - CLI: `coli web` = serve + auto-open the browser once /health answers (the 744B load takes minutes; a poller waits for it), --no-browser to disable, friendly hint when web/dist isn't built. * coli web: HERE is a path string, not a Path * web: same-origin default + auto-connect when served by the engine The page hosted by coli web talked to http://127.0.0.1:8000/v1 by default while being loaded from http://localhost:8000 - a different origin, so the very first probe died on CORS ('Failed to fetch'). When the page is engine-served the default (and a stored stale factory default) now resolve to window.location.origin + /v1, and the console auto-probes on load; the Vite dev server keeps the classic default. The server's own port is also whitelisted in DEFAULT_CORS_ORIGINS for the localhost/127.0.0.1 cross-name case. * web: fix auto-connect effect returning Promise (React 19 StrictMode crash) The async connect() call inside useEffect returned a Promise that React 19 StrictMode stored as the effect cleanup. On re-render, React called destroy_() on that Promise — 'not a function'. Block body with explicit return undefined prevents any non-function from leaking into the cleanup slot. * web: rich streaming metrics — live token counter, tok/s gauge, TTFT, session totals During generation: a flashing Zap badge counts tokens in real time. After: Gauge shows tok/s, Timer shows TTFT (time to first token), Layers shows prompt→completion token counts, Clock shows queue wait. Session totals (cumulative prompt + completion) appear in the Runtime panel. All with lucide icons and tabular-nums for stable layout. Tier bar gains a smoother cubic-bezier transition and slightly taller height for visual weight. * web: hardware environment panel — CPU model, GPU count/VRAM, RAM total/free, cores Engine emits HWINFO protocol line at startup (CPU name from /proc/cpuinfo, core count, RAM total/available from /proc/meminfo, GPU count + VRAM from CUDA). Server parses and caches it, /health exposes as 'hwinfo'. Dashboard renders it with icons at the top of the Runtime section so the user sees immediately what the engine is running on. * web: downgrade to React 18 — React 19 effect cleanup bug crashes the dashboard React 19 treats the return value of every useEffect callback as a cleanup function. In practice, any async interaction (streamChat's rapid onDelta re-renders, the health poller, auto-connect) caused React 19 to store a Promise as inst.destroy and crash with 'destroy_ is not a function' on the next commit cycle. The bug reproduced in incognito, without StrictMode, and with every effect converted to explicit block bodies returning undefined — it is a React 19 regression in the passive-effect unmount path. React 18 does not exhibit this behavior. Pin to React 18 until React 19 is fixed or the codebase migrates to a pattern React 19 handles correctly. All 17 vitest pass, ErrorBoundary retained. * web: Brain page — the expert cortex, live A 76x256 canvas grid, one cell per expert (19,456 total): colour = tier (green VRAM / blue RAM / grey disk), brightness = routing heat (log-bucketed .coli_usage counts), and a white pulse that flashes on every expert routed in the current turn and decays — you watch the model think. - Engine: EMAP protocol line (1 byte/expert: 2bit tier + 6bit heat, hex) at READY and after each turn; HITS bitmap of this turn's routed experts (set where eusage increments, cleared on emit). - Server: parses both, GET /experts serves {rows, cols, map, hits, seq}. - Web: Brain tab next to Chat; canvas renderer with rAF pulse decay; hover tooltip shows layer/expert/tier/heat plus an honest depth-role heuristic (early = surface features ... late = output shaping, MTP row labelled as the speculative draft head). * web: Brain page responsive — cells sized from container via ResizeObserver Cell size derives from the wrapper's actual client box instead of fixed 1400x900, re-rendering on resize; mobile media query tightens padding, legend and tooltip. The cortex now fills whatever screen it gets.
This commit is contained in:
ZacharyZcR
+71
-2
@@ -7,6 +7,7 @@ import collections
|
||||
import contextlib
|
||||
import json
|
||||
import math
|
||||
import mimetypes
|
||||
import os
|
||||
import select
|
||||
import queue
|
||||
@@ -27,6 +28,8 @@ END = b"\x01\x01END\x01\x01\n"
|
||||
READY = b"\x01\x01READY\x01\x01\n"
|
||||
MAX_BODY = 4 << 20
|
||||
DEFAULT_CORS_ORIGINS = (
|
||||
"http://127.0.0.1:8000",
|
||||
"http://localhost:8000",
|
||||
"http://127.0.0.1:5173",
|
||||
"http://localhost:5173",
|
||||
"http://tauri.localhost",
|
||||
@@ -462,6 +465,11 @@ class Engine:
|
||||
self.closed = False
|
||||
self.dispatcher_error = None
|
||||
self.kv_slots = kv_slots
|
||||
self.tiers = None
|
||||
self.hwinfo = None
|
||||
self.emap = None
|
||||
self.hits = None
|
||||
self.hits_seq = 0 # latest "TIERS" snapshot from the engine
|
||||
read_engine_turn(self.process.stdout, READY, lambda _: None)
|
||||
self.dispatcher = threading.Thread(target=self._dispatch_stdout,
|
||||
name="colibri-stdout", daemon=True)
|
||||
@@ -527,6 +535,22 @@ class Engine:
|
||||
events = self.pending.pop(request_id, None)
|
||||
if events is not None:
|
||||
events.put(("done", stats))
|
||||
elif kind == "HWINFO" and len(fields) >= 7:
|
||||
parts = " ".join(fields[6:]).split("|")
|
||||
self.hwinfo = {"cores": int(fields[1]), "ram_total_gb": float(fields[2]),
|
||||
"ram_avail_gb": float(fields[3]), "gpus": int(fields[4]),
|
||||
"vram_total_gb": float(fields[5]),
|
||||
"cpu": parts[0].strip() if len(parts)>0 else "",
|
||||
"gpu": parts[1].strip() if len(parts)>1 else ""}
|
||||
elif kind == "EMAP" and len(fields) == 4:
|
||||
self.emap = {"rows": int(fields[1]), "cols": int(fields[2]), "map": fields[3]}
|
||||
elif kind == "HITS" and len(fields) == 4:
|
||||
self.hits = fields[3]
|
||||
self.hits_seq += 1
|
||||
elif kind == "TIERS" and len(fields) >= 6:
|
||||
self.tiers = {"vram": int(fields[1]), "ram": int(fields[2]),
|
||||
"disk": int(fields[3]), "vram_gb": float(fields[4]),
|
||||
"ram_gb": float(fields[5])}
|
||||
elif kind == "ERROR" and len(fields) >= 2:
|
||||
request_id = fields[1]
|
||||
message = " ".join(fields[2:]) or "engine request failed"
|
||||
@@ -700,13 +724,58 @@ class APIHandler(BaseHTTPRequestHandler):
|
||||
if model != self.server.model_id:
|
||||
raise APIError(404, f"The model `{model}` does not exist.", "model", "model_not_found")
|
||||
|
||||
WEB_DIST = Path(__file__).resolve().parent.parent / "web" / "dist"
|
||||
|
||||
def serve_static(self, path):
|
||||
"""Serve the built web UI (web/dist) so `coli web` is one process.
|
||||
Read-only, no auth (same trust level as /health), traversal-safe."""
|
||||
if path.startswith("/v1/") or path == "/health":
|
||||
return False
|
||||
base = self.WEB_DIST
|
||||
if not base.is_dir():
|
||||
return False
|
||||
rel = unquote(path).lstrip("/") or "index.html"
|
||||
target = (base / rel).resolve()
|
||||
if not str(target).startswith(str(base)) or not target.is_file():
|
||||
if path == "/" or "." not in rel: # SPA fallback
|
||||
target = base / "index.html"
|
||||
if not target.is_file():
|
||||
return False
|
||||
else:
|
||||
return False
|
||||
ctype = mimetypes.guess_type(str(target))[0] or "application/octet-stream"
|
||||
data = target.read_bytes()
|
||||
self.send_response(200)
|
||||
self.send_header("Content-Type", ctype)
|
||||
self.send_header("Content-Length", str(len(data)))
|
||||
self.send_cors_headers()
|
||||
self.end_headers()
|
||||
self.wfile.write(data)
|
||||
return True
|
||||
|
||||
def do_GET(self):
|
||||
request_id = "req_" + uuid.uuid4().hex
|
||||
try:
|
||||
path = urlsplit(self.path).path
|
||||
if path == "/health":
|
||||
self.send_json(200, {"status": "ok", "scheduler": self.server.scheduler.snapshot(),
|
||||
"kv_slots": self.server.kv_slots}, request_id)
|
||||
payload = {"status": "ok", "scheduler": self.server.scheduler.snapshot(),
|
||||
"kv_slots": self.server.kv_slots}
|
||||
tiers = getattr(self.server.engine, "tiers", None) if self.server.engine else None
|
||||
if tiers: payload["tiers"] = tiers
|
||||
hwinfo = getattr(self.server.engine, "hwinfo", None) if self.server.engine else None
|
||||
if hwinfo: payload["hwinfo"] = hwinfo
|
||||
self.send_json(200, payload, request_id)
|
||||
return
|
||||
if path == "/experts":
|
||||
eng = self.server.engine
|
||||
payload = {"rows": 0, "cols": 0, "map": "", "hits": "", "seq": 0}
|
||||
if eng and getattr(eng, "emap", None):
|
||||
payload.update(eng.emap)
|
||||
payload["hits"] = eng.hits or ""
|
||||
payload["seq"] = eng.hits_seq
|
||||
self.send_json(200, payload, request_id)
|
||||
return
|
||||
if self.serve_static(path):
|
||||
return
|
||||
self.require_auth()
|
||||
if path == "/v1/models":
|
||||
|
||||
Reference in New Issue
Block a user