web dashboard: live expert-tier panel (VRAM/RAM/disk), one-process hosting, coli web (#126)
* Web dashboard: live expert-tier panel (VRAM/RAM/disk), static hosting, coli web command - Engine: TIERS protocol line (counts + GB for VRAM/RAM/disk experts) emitted at READY and refreshed after every served turn, computed from the live pin/LRU/CUDA state. - Server: dispatcher tracks the latest snapshot, /health exposes it as "tiers"; the built web UI (web/dist) is served from the same port (read-only, traversal-safe, SPA fallback) so the dashboard needs no second process. - Web: tier bar + legend in the Runtime panel showing where the 19k experts live right now, with GB per tier; updates with the existing health polling. - CLI: `coli web` = serve + auto-open the browser once /health answers (the 744B load takes minutes; a poller waits for it), --no-browser to disable, friendly hint when web/dist isn't built. * coli web: HERE is a path string, not a Path * web: same-origin default + auto-connect when served by the engine The page hosted by coli web talked to http://127.0.0.1:8000/v1 by default while being loaded from http://localhost:8000 - a different origin, so the very first probe died on CORS ('Failed to fetch'). When the page is engine-served the default (and a stored stale factory default) now resolve to window.location.origin + /v1, and the console auto-probes on load; the Vite dev server keeps the classic default. The server's own port is also whitelisted in DEFAULT_CORS_ORIGINS for the localhost/127.0.0.1 cross-name case. * web: fix auto-connect effect returning Promise (React 19 StrictMode crash) The async connect() call inside useEffect returned a Promise that React 19 StrictMode stored as the effect cleanup. On re-render, React called destroy_() on that Promise — 'not a function'. Block body with explicit return undefined prevents any non-function from leaking into the cleanup slot. * web: rich streaming metrics — live token counter, tok/s gauge, TTFT, session totals During generation: a flashing Zap badge counts tokens in real time. After: Gauge shows tok/s, Timer shows TTFT (time to first token), Layers shows prompt→completion token counts, Clock shows queue wait. Session totals (cumulative prompt + completion) appear in the Runtime panel. All with lucide icons and tabular-nums for stable layout. Tier bar gains a smoother cubic-bezier transition and slightly taller height for visual weight. * web: hardware environment panel — CPU model, GPU count/VRAM, RAM total/free, cores Engine emits HWINFO protocol line at startup (CPU name from /proc/cpuinfo, core count, RAM total/available from /proc/meminfo, GPU count + VRAM from CUDA). Server parses and caches it, /health exposes as 'hwinfo'. Dashboard renders it with icons at the top of the Runtime section so the user sees immediately what the engine is running on. * web: downgrade to React 18 — React 19 effect cleanup bug crashes the dashboard React 19 treats the return value of every useEffect callback as a cleanup function. In practice, any async interaction (streamChat's rapid onDelta re-renders, the health poller, auto-connect) caused React 19 to store a Promise as inst.destroy and crash with 'destroy_ is not a function' on the next commit cycle. The bug reproduced in incognito, without StrictMode, and with every effect converted to explicit block bodies returning undefined — it is a React 19 regression in the passive-effect unmount path. React 18 does not exhibit this behavior. Pin to React 18 until React 19 is fixed or the codebase migrates to a pattern React 19 handles correctly. All 17 vitest pass, ErrorBoundary retained. * web: Brain page — the expert cortex, live A 76x256 canvas grid, one cell per expert (19,456 total): colour = tier (green VRAM / blue RAM / grey disk), brightness = routing heat (log-bucketed .coli_usage counts), and a white pulse that flashes on every expert routed in the current turn and decays — you watch the model think. - Engine: EMAP protocol line (1 byte/expert: 2bit tier + 6bit heat, hex) at READY and after each turn; HITS bitmap of this turn's routed experts (set where eusage increments, cleared on emit). - Server: parses both, GET /experts serves {rows, cols, map, hits, seq}. - Web: Brain tab next to Chat; canvas renderer with rAF pulse decay; hover tooltip shows layer/expert/tier/heat plus an honest depth-role heuristic (early = surface features ... late = output shaping, MTP row labelled as the speculative draft head). * web: Brain page responsive — cells sized from container via ResizeObserver Cell size derives from the wrapper's actual client box instead of fixed 1400x900, re-rendering on resize; mobile media query tightens padding, legend and tooltip. The cortex now fills whatever screen it gets.
This commit is contained in:
ZacharyZcR
@@ -0,0 +1,159 @@
|
||||
import { useEffect, useRef, useState } from "react"
|
||||
import { BrainCircuit, Flame, Layers } from "lucide-react"
|
||||
|
||||
import { endpoint } from "@/lib/api"
|
||||
|
||||
interface ExpertMap { rows: number; cols: number; map: string; hits: string; seq: number }
|
||||
|
||||
const TIER_NAME = ["Disk", "RAM", "VRAM"]
|
||||
const TIER_RGB: [number, number, number][] = [[58, 71, 80], [90, 155, 216], [78, 214, 165]]
|
||||
|
||||
/* Layer-depth heuristic: what this region of the network tends to specialise in.
|
||||
* Honest framing — these are the depth roles observed across MoE interpretability
|
||||
* work, not per-expert ground truth (that needs co-activation analysis, #119). */
|
||||
function depthRole(row: number, rows: number, isMtp: boolean): string {
|
||||
if (isMtp) return "MTP head — drafts the next token for speculative decoding"
|
||||
const f = row / Math.max(rows - 1, 1)
|
||||
if (f < 0.2) return "early layers — surface features: tokens, spelling, local syntax"
|
||||
if (f < 0.45) return "lower-middle — phrase structure, word relations, simple facts"
|
||||
if (f < 0.7) return "upper-middle — semantics, long-range context, reasoning steps"
|
||||
if (f < 0.9) return "late layers — planning the answer, style, coherence"
|
||||
return "final layers — output shaping: picks the actual next-token distribution"
|
||||
}
|
||||
|
||||
export function Brain({ baseUrl, apiKey, connected }: { baseUrl: string; apiKey: string; connected: boolean }) {
|
||||
const canvasRef = useRef<HTMLCanvasElement>(null)
|
||||
const wrapRef = useRef<HTMLDivElement>(null)
|
||||
const [wrapSize, setWrapSize] = useState({ w: 1200, h: 700 })
|
||||
const [data, setData] = useState<ExpertMap | null>(null)
|
||||
const [tip, setTip] = useState<{ x: number; y: number; row: number; col: number; tier: number; heat: number } | null>(null)
|
||||
const pulseRef = useRef<Float32Array | null>(null) // per-expert pulse intensity 0..1
|
||||
const lastSeq = useRef(0)
|
||||
const rafRef = useRef(0)
|
||||
|
||||
// track container size for responsive cell sizing
|
||||
useEffect(() => {
|
||||
const el = wrapRef.current
|
||||
if (!el) return
|
||||
const ro = new ResizeObserver(() => {
|
||||
setWrapSize({ w: el.clientWidth - 24, h: el.clientHeight - 24 })
|
||||
})
|
||||
ro.observe(el)
|
||||
return () => ro.disconnect()
|
||||
}, [])
|
||||
|
||||
// poll /experts
|
||||
useEffect(() => {
|
||||
if (!connected) return
|
||||
let disposed = false
|
||||
const base = baseUrl.replace(/\/v1\/?$/, "")
|
||||
const poll = async () => {
|
||||
try {
|
||||
const res = await fetch(endpoint(base, "/experts"), { headers: apiKey ? { Authorization: `Bearer ${apiKey}` } : {} })
|
||||
const next = (await res.json()) as ExpertMap
|
||||
if (disposed || !next.rows) return
|
||||
setData(next)
|
||||
if (next.seq !== lastSeq.current && next.hits) {
|
||||
lastSeq.current = next.seq
|
||||
const n = next.rows * next.cols
|
||||
if (!pulseRef.current || pulseRef.current.length !== n) pulseRef.current = new Float32Array(n)
|
||||
const p = pulseRef.current
|
||||
for (let i = 0; i < n; i++) {
|
||||
const byte = parseInt(next.hits.substr((i >> 3) * 2, 2), 16) || 0
|
||||
if (byte & (1 << (i & 7))) p[i] = 1
|
||||
}
|
||||
}
|
||||
} catch { /* engine busy or restarting — keep the last frame */ }
|
||||
}
|
||||
void poll()
|
||||
const t = window.setInterval(() => void poll(), 1500)
|
||||
return () => { disposed = true; window.clearInterval(t) }
|
||||
}, [baseUrl, apiKey, connected])
|
||||
|
||||
// render loop: grid + decaying pulses
|
||||
useEffect(() => {
|
||||
const canvas = canvasRef.current
|
||||
if (!canvas || !data) return
|
||||
const ctx = canvas.getContext("2d")
|
||||
if (!ctx) return
|
||||
const { rows, cols, map } = data
|
||||
const cell = Math.max(2, Math.floor(Math.min(wrapSize.w / cols, wrapSize.h / rows)))
|
||||
const gap = cell >= 4 ? 1 : 0
|
||||
canvas.width = cols * (cell + gap)
|
||||
canvas.height = rows * (cell + gap)
|
||||
|
||||
const draw = () => {
|
||||
ctx.clearRect(0, 0, canvas.width, canvas.height)
|
||||
const p = pulseRef.current
|
||||
for (let r = 0; r < rows; r++) {
|
||||
for (let c = 0; c < cols; c++) {
|
||||
const i = r * cols + c
|
||||
const byte = parseInt(map.substr(i * 2, 2), 16) || 0
|
||||
const tier = byte >> 6
|
||||
const heat = byte & 63
|
||||
const [R, G, B] = TIER_RGB[tier] ?? TIER_RGB[0]
|
||||
// heat scales brightness: cold experts dim, hot experts full colour
|
||||
const lum = 0.35 + 0.65 * Math.min(heat / 24, 1)
|
||||
let rr = R * lum, gg = G * lum, bb = B * lum
|
||||
const pulse = p ? p[i] : 0
|
||||
if (pulse > 0.01) { rr += (255 - rr) * pulse; gg += (255 - gg) * pulse; bb += (255 - bb) * pulse }
|
||||
ctx.fillStyle = `rgb(${rr | 0},${gg | 0},${bb | 0})`
|
||||
ctx.fillRect(c * (cell + gap), r * (cell + gap), cell, cell)
|
||||
}
|
||||
}
|
||||
let alive = false
|
||||
if (p) for (let i = 0; i < p.length; i++) { if (p[i] > 0.01) { p[i] *= 0.94; alive = true } else p[i] = 0 }
|
||||
if (alive) rafRef.current = requestAnimationFrame(draw)
|
||||
}
|
||||
draw()
|
||||
const keepalive = window.setInterval(() => { if (!rafRef.current) draw(); rafRef.current = 0 }, 400)
|
||||
return () => { cancelAnimationFrame(rafRef.current); window.clearInterval(keepalive) }
|
||||
}, [data, wrapSize])
|
||||
|
||||
const onMove = (e: React.MouseEvent<HTMLCanvasElement>) => {
|
||||
if (!data) return
|
||||
const rect = e.currentTarget.getBoundingClientRect()
|
||||
const scaleX = e.currentTarget.width / rect.width
|
||||
const scaleY = e.currentTarget.height / rect.height
|
||||
const cell = Math.max(2, Math.floor(Math.min(wrapSize.w / data.cols, wrapSize.h / data.rows)))
|
||||
const gap = cell >= 4 ? 1 : 0
|
||||
const col = Math.floor(((e.clientX - rect.left) * scaleX) / (cell + gap))
|
||||
const row = Math.floor(((e.clientY - rect.top) * scaleY) / (cell + gap))
|
||||
if (row < 0 || row >= data.rows || col < 0 || col >= data.cols) { setTip(null); return }
|
||||
const byte = parseInt(data.map.substr((row * data.cols + col) * 2, 2), 16) || 0
|
||||
setTip({ x: e.clientX, y: e.clientY, row, col, tier: byte >> 6, heat: byte & 63 })
|
||||
}
|
||||
|
||||
const totals = data ? (() => {
|
||||
const t = [0, 0, 0]
|
||||
for (let i = 0; i < data.rows * data.cols; i++) t[(parseInt(data.map.substr(i * 2, 2), 16) || 0) >> 6]++
|
||||
return t
|
||||
})() : [0, 0, 0]
|
||||
|
||||
return (
|
||||
<div className="brain-page">
|
||||
<div className="brain-head">
|
||||
<div className="section-title"><BrainCircuit className="size-4" /> Expert Cortex — {data ? `${data.rows} layers × ${data.cols} experts` : "waiting for engine"}</div>
|
||||
<div className="brain-legend">
|
||||
<span><i style={{ background: "#4ed6a5" }} /> VRAM {totals[2].toLocaleString()}</span>
|
||||
<span><i style={{ background: "#5a9bd8" }} /> RAM {totals[1].toLocaleString()}</span>
|
||||
<span><i style={{ background: "#3a4750" }} /> Disk {totals[0].toLocaleString()}</span>
|
||||
<span><Flame className="size-3" /> brightness = routing heat</span>
|
||||
<span className="brain-pulse-hint">⚡ white flash = routed this turn</span>
|
||||
</div>
|
||||
</div>
|
||||
<div className="brain-canvas-wrap" ref={wrapRef}>
|
||||
<canvas ref={canvasRef} onMouseMove={onMove} onMouseLeave={() => setTip(null)} />
|
||||
{!connected && <p className="runtime-unavailable">Connect to the engine to see the cortex.</p>}
|
||||
</div>
|
||||
{tip && data && (
|
||||
<div className="brain-tip" style={{ left: tip.x + 14, top: tip.y + 14 }}>
|
||||
<div className="brain-tip-title"><Layers className="size-3" /> Layer row {tip.row}{tip.row === data.rows - 1 ? " (MTP)" : ""} · Expert {tip.col}</div>
|
||||
<div>Tier: <strong style={{ color: ["#8b9aa3", "#5a9bd8", "#4ed6a5"][tip.tier] }}>{TIER_NAME[tip.tier]}</strong></div>
|
||||
<div>Heat: <strong>{tip.heat === 0 ? "never routed" : `~2^${tip.heat} selections`}</strong></div>
|
||||
<div className="brain-tip-role">{depthRole(tip.row, data.rows, tip.row === data.rows - 1)}</div>
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
)
|
||||
}
|
||||
Reference in New Issue
Block a user