web dashboard: live expert-tier panel (VRAM/RAM/disk), one-process hosting, coli web (#126)
* Web dashboard: live expert-tier panel (VRAM/RAM/disk), static hosting, coli web command - Engine: TIERS protocol line (counts + GB for VRAM/RAM/disk experts) emitted at READY and refreshed after every served turn, computed from the live pin/LRU/CUDA state. - Server: dispatcher tracks the latest snapshot, /health exposes it as "tiers"; the built web UI (web/dist) is served from the same port (read-only, traversal-safe, SPA fallback) so the dashboard needs no second process. - Web: tier bar + legend in the Runtime panel showing where the 19k experts live right now, with GB per tier; updates with the existing health polling. - CLI: `coli web` = serve + auto-open the browser once /health answers (the 744B load takes minutes; a poller waits for it), --no-browser to disable, friendly hint when web/dist isn't built. * coli web: HERE is a path string, not a Path * web: same-origin default + auto-connect when served by the engine The page hosted by coli web talked to http://127.0.0.1:8000/v1 by default while being loaded from http://localhost:8000 - a different origin, so the very first probe died on CORS ('Failed to fetch'). When the page is engine-served the default (and a stored stale factory default) now resolve to window.location.origin + /v1, and the console auto-probes on load; the Vite dev server keeps the classic default. The server's own port is also whitelisted in DEFAULT_CORS_ORIGINS for the localhost/127.0.0.1 cross-name case. * web: fix auto-connect effect returning Promise (React 19 StrictMode crash) The async connect() call inside useEffect returned a Promise that React 19 StrictMode stored as the effect cleanup. On re-render, React called destroy_() on that Promise — 'not a function'. Block body with explicit return undefined prevents any non-function from leaking into the cleanup slot. * web: rich streaming metrics — live token counter, tok/s gauge, TTFT, session totals During generation: a flashing Zap badge counts tokens in real time. After: Gauge shows tok/s, Timer shows TTFT (time to first token), Layers shows prompt→completion token counts, Clock shows queue wait. Session totals (cumulative prompt + completion) appear in the Runtime panel. All with lucide icons and tabular-nums for stable layout. Tier bar gains a smoother cubic-bezier transition and slightly taller height for visual weight. * web: hardware environment panel — CPU model, GPU count/VRAM, RAM total/free, cores Engine emits HWINFO protocol line at startup (CPU name from /proc/cpuinfo, core count, RAM total/available from /proc/meminfo, GPU count + VRAM from CUDA). Server parses and caches it, /health exposes as 'hwinfo'. Dashboard renders it with icons at the top of the Runtime section so the user sees immediately what the engine is running on. * web: downgrade to React 18 — React 19 effect cleanup bug crashes the dashboard React 19 treats the return value of every useEffect callback as a cleanup function. In practice, any async interaction (streamChat's rapid onDelta re-renders, the health poller, auto-connect) caused React 19 to store a Promise as inst.destroy and crash with 'destroy_ is not a function' on the next commit cycle. The bug reproduced in incognito, without StrictMode, and with every effect converted to explicit block bodies returning undefined — it is a React 19 regression in the passive-effect unmount path. React 18 does not exhibit this behavior. Pin to React 18 until React 19 is fixed or the codebase migrates to a pattern React 19 handles correctly. All 17 vitest pass, ErrorBoundary retained. * web: Brain page — the expert cortex, live A 76x256 canvas grid, one cell per expert (19,456 total): colour = tier (green VRAM / blue RAM / grey disk), brightness = routing heat (log-bucketed .coli_usage counts), and a white pulse that flashes on every expert routed in the current turn and decays — you watch the model think. - Engine: EMAP protocol line (1 byte/expert: 2bit tier + 6bit heat, hex) at READY and after each turn; HITS bitmap of this turn's routed experts (set where eusage increments, cleared on emit). - Server: parses both, GET /experts serves {rows, cols, map, hits, seq}. - Web: Brain tab next to Chat; canvas renderer with rAF pulse decay; hover tooltip shows layer/expert/tier/heat plus an honest depth-role heuristic (early = surface features ... late = output shaping, MTP row labelled as the speculative draft head). * web: Brain page responsive — cells sized from container via ResizeObserver Cell size derives from the wrapper's actual client box instead of fixed 1400x900, re-rendering on resize; mobile media query tightens padding, legend and tooltip. The cortex now fills whatever screen it gets.
This commit is contained in:
ZacharyZcR
+105
-6
@@ -4,15 +4,24 @@ import {
|
||||
ArrowUp,
|
||||
BrainCircuit,
|
||||
CircleStop,
|
||||
Clock,
|
||||
Cpu,
|
||||
Database,
|
||||
Feather,
|
||||
Gauge,
|
||||
HardDrive,
|
||||
KeyRound,
|
||||
Layers,
|
||||
Link2,
|
||||
LoaderCircle,
|
||||
MemoryStick,
|
||||
MessageSquareText,
|
||||
MonitorDot,
|
||||
RefreshCw,
|
||||
SlidersHorizontal,
|
||||
Timer,
|
||||
Trash2,
|
||||
Zap,
|
||||
} from "lucide-react"
|
||||
|
||||
import { Badge } from "@/components/ui/badge"
|
||||
@@ -21,13 +30,25 @@ import { Input } from "@/components/ui/input"
|
||||
import { Textarea } from "@/components/ui/textarea"
|
||||
import { getHealth, listModels, streamChat, type ChatMessage, type HealthResponse, type StreamChatResult } from "@/lib/api"
|
||||
import { activeRequests, supportsCacheSlots } from "@/lib/runtime"
|
||||
import { Brain } from "./Brain"
|
||||
import { persistPublicSettings, stored } from "@/lib/storage"
|
||||
import { cn } from "@/lib/utils"
|
||||
|
||||
const message = (role: ChatMessage["role"], content: string): ChatMessage => ({ id: crypto.randomUUID(), role, content })
|
||||
|
||||
export default function App() {
|
||||
const [baseUrl, setBaseUrl] = useState(() => stored(localStorage, "colibri.baseUrl", "http://127.0.0.1:8000/v1"))
|
||||
// When the page is served by the engine itself (coli web), same-origin is the
|
||||
// right default: no CORS, no manual endpoint editing. The Vite dev server
|
||||
// (port 5173) keeps the classic default.
|
||||
const servedByEngine = typeof window !== "undefined" && window.location.port !== "5173" && window.location.protocol.startsWith("http")
|
||||
const defaultBase = servedByEngine ? `${window.location.origin}/v1` : "http://127.0.0.1:8000/v1"
|
||||
const [baseUrl, setBaseUrl] = useState(() => {
|
||||
const saved = stored(localStorage, "colibri.baseUrl", defaultBase)
|
||||
// migrate: a stored FACTORY default pointing at another origin would trip CORS
|
||||
// when the page is engine-served — upgrade it to same-origin once.
|
||||
if (servedByEngine && saved === "http://127.0.0.1:8000/v1" && defaultBase !== saved) return defaultBase
|
||||
return saved
|
||||
})
|
||||
const [apiKey, setApiKey] = useState("")
|
||||
const [models, setModels] = useState<string[]>([])
|
||||
const [model, setModel] = useState(() => stored(localStorage, "colibri.model", "glm-5.2-colibri"))
|
||||
@@ -41,9 +62,16 @@ export default function App() {
|
||||
const [lastRun, setLastRun] = useState<StreamChatResult | null>(null)
|
||||
const [draft, setDraft] = useState("")
|
||||
const [loading, setLoading] = useState(false)
|
||||
const [streamStart, setStreamStart] = useState<number | null>(null)
|
||||
const [tokenCount, setTokenCount] = useState(0)
|
||||
const [tokPerSec, setTokPerSec] = useState<number | null>(null)
|
||||
const [ttft, setTtft] = useState<number | null>(null)
|
||||
const [totalTokens, setTotalTokens] = useState({ prompt: 0, completion: 0 })
|
||||
const [connecting, setConnecting] = useState(false)
|
||||
const [connected, setConnected] = useState(false)
|
||||
const [view, setView] = useState<"chat" | "brain">("chat")
|
||||
const [error, setError] = useState("")
|
||||
const autoConnected = useRef(false)
|
||||
const abortRef = useRef<AbortController | null>(null)
|
||||
const probeRef = useRef<AbortController | null>(null)
|
||||
const bottomRef = useRef<HTMLDivElement>(null)
|
||||
@@ -59,21 +87,25 @@ export default function App() {
|
||||
[cacheSlot]: typeof next === "function" ? next(current[cacheSlot] || []) : next,
|
||||
}))
|
||||
|
||||
// EFFECT #1
|
||||
useEffect(() => {
|
||||
persistPublicSettings(localStorage, baseUrl, model)
|
||||
}, [baseUrl, model])
|
||||
|
||||
// EFFECT #2
|
||||
useEffect(() => {
|
||||
setConnected(false)
|
||||
setHealth(null)
|
||||
setHealthError("")
|
||||
}, [baseUrl, apiKey])
|
||||
|
||||
// EFFECT #3
|
||||
useEffect(() => () => {
|
||||
probeRef.current?.abort()
|
||||
abortRef.current?.abort()
|
||||
}, [])
|
||||
|
||||
// EFFECT #4
|
||||
useEffect(() => {
|
||||
if (!connected) return
|
||||
let disposed = false
|
||||
@@ -90,13 +122,16 @@ export default function App() {
|
||||
return () => { disposed = true; window.clearInterval(timer) }
|
||||
}, [apiKey, baseUrl, connected])
|
||||
|
||||
// EFFECT #5
|
||||
useEffect(() => {
|
||||
if (cacheSlot >= kvSlots) setCacheSlot(0)
|
||||
}, [cacheSlot, kvSlots])
|
||||
|
||||
useEffect(() => setLastRun(null), [cacheSlot])
|
||||
// EFFECT #6
|
||||
useEffect(() => { setLastRun(null) }, [cacheSlot])
|
||||
|
||||
useEffect(() => bottomRef.current?.scrollIntoView({ behavior: "smooth" }), [messages])
|
||||
// EFFECT #7
|
||||
useEffect(() => { bottomRef.current?.scrollIntoView({ behavior: "smooth" }) }, [messages])
|
||||
|
||||
const connect = async () => {
|
||||
probeRef.current?.abort()
|
||||
@@ -127,6 +162,11 @@ export default function App() {
|
||||
}
|
||||
}
|
||||
|
||||
if (servedByEngine && !autoConnected.current && !connected) {
|
||||
autoConnected.current = true
|
||||
setTimeout(() => connect(), 0)
|
||||
}
|
||||
|
||||
const canSend = useMemo(() => draft.trim() && model && !loading, [draft, loading, model])
|
||||
|
||||
const send = async () => {
|
||||
@@ -139,6 +179,13 @@ export default function App() {
|
||||
setError("")
|
||||
updateMessages([...history, assistant])
|
||||
setLoading(true)
|
||||
setStreamStart(null)
|
||||
setTokenCount(0)
|
||||
setTokPerSec(null)
|
||||
setTtft(null)
|
||||
const t0 = performance.now()
|
||||
let firstToken = true
|
||||
let count = 0
|
||||
const controller = new AbortController()
|
||||
abortRef.current = controller
|
||||
try {
|
||||
@@ -152,11 +199,23 @@ export default function App() {
|
||||
enableThinking: thinking,
|
||||
cacheSlot: supportsCacheSlots(health) ? cacheSlot : undefined,
|
||||
signal: controller.signal,
|
||||
onDelta: (delta) =>
|
||||
onDelta: (delta) => {
|
||||
if (firstToken) { setTtft(performance.now() - t0); setStreamStart(performance.now()); firstToken = false }
|
||||
count++
|
||||
setTokenCount(count)
|
||||
const elapsed = (performance.now() - (firstToken ? t0 : t0)) / 1000
|
||||
if (elapsed > 0.3) setTokPerSec(count / ((performance.now() - t0) / 1000))
|
||||
updateMessages((current) => current.map((item) =>
|
||||
item.id === assistant.id ? { ...item, content: item.content + delta } : item,
|
||||
)),
|
||||
))
|
||||
},
|
||||
})
|
||||
const finalElapsed = (performance.now() - t0) / 1000
|
||||
if (count > 0 && finalElapsed > 0) setTokPerSec(count / finalElapsed)
|
||||
if (result.usage) setTotalTokens(prev => ({
|
||||
prompt: prev.prompt + (result.usage?.prompt_tokens || 0),
|
||||
completion: prev.completion + (result.usage?.completion_tokens || 0),
|
||||
}))
|
||||
setLastRun(result)
|
||||
setConnected(true)
|
||||
} catch (cause) {
|
||||
@@ -193,6 +252,12 @@ export default function App() {
|
||||
|
||||
<section className="side-section runtime-section" aria-live="polite">
|
||||
<div className="section-title"><Activity className="size-3.5" /> Runtime</div>
|
||||
{health?.hwinfo ? <div className="hw-panel">
|
||||
{health.hwinfo.cpu ? <div className="hw-row"><Cpu className="size-3.5" /><span>{health.hwinfo.cpu}</span></div> : null}
|
||||
{health.hwinfo.gpus > 0 ? <div className="hw-row"><MonitorDot className="size-3.5" /><span>{health.hwinfo.gpus}× GPU<small>{health.hwinfo.vram_total_gb.toFixed(0)} GB VRAM</small></span></div> : null}
|
||||
<div className="hw-row"><MemoryStick className="size-3.5" /><span>{health.hwinfo.ram_total_gb.toFixed(0)} GB RAM<small>{health.hwinfo.ram_avail_gb.toFixed(0)} GB free</small></span></div>
|
||||
<div className="hw-row"><HardDrive className="size-3.5" /><span>{health.hwinfo.cores} cores</span></div>
|
||||
</div> : null}
|
||||
{health?.scheduler ? <>
|
||||
<div className="runtime-grid">
|
||||
<div><span>Active</span><strong>{active}<small> / {capacity}</small></strong></div>
|
||||
@@ -200,6 +265,25 @@ export default function App() {
|
||||
<div><span>Completed</span><strong>{health.scheduler.completed}</strong></div>
|
||||
<div><span>Failures</span><strong>{failures}</strong></div>
|
||||
</div>
|
||||
{health.tiers ? (() => {
|
||||
const t = health.tiers
|
||||
const total = Math.max(t.vram + t.ram + t.disk, 1)
|
||||
return <div className="tier-panel">
|
||||
<div className="tier-bar" role="img" aria-label={`Experts: ${t.vram} VRAM, ${t.ram} RAM, ${t.disk} disk`}>
|
||||
<span className="tier-vram" style={{ width: `${(100 * t.vram) / total}%` }} />
|
||||
<span className="tier-ram" style={{ width: `${(100 * t.ram) / total}%` }} />
|
||||
<span className="tier-disk" style={{ width: `${(100 * t.disk) / total}%` }} />
|
||||
</div>
|
||||
<div className="tier-legend">
|
||||
<span><i className="tier-vram" />VRAM <strong>{t.vram.toLocaleString()}</strong><small>{t.vram_gb.toFixed(1)} GB</small></span>
|
||||
<span><i className="tier-ram" />RAM <strong>{t.ram.toLocaleString()}</strong><small>{t.ram_gb.toFixed(1)} GB</small></span>
|
||||
<span><i className="tier-disk" />Disk <strong>{t.disk.toLocaleString()}</strong></span>
|
||||
</div>
|
||||
</div>
|
||||
})() : null}
|
||||
{totalTokens.prompt + totalTokens.completion > 0 ? <div className="session-stats">
|
||||
<span><Database className="size-3" /> Session: <strong>{totalTokens.prompt.toLocaleString()}</strong> prompt + <strong>{totalTokens.completion.toLocaleString()}</strong> completion</span>
|
||||
</div> : null}
|
||||
<div className="runtime-foot"><span className="runtime-dot" /> Scheduler online <code>{kvSlots} KV</code></div>
|
||||
</> : <p className="runtime-unavailable">{connected ? (healthError || "Runtime metrics unavailable") : "Probe the server to inspect runtime state."}</p>}
|
||||
</section>
|
||||
@@ -223,9 +307,23 @@ export default function App() {
|
||||
<main className="chat-panel">
|
||||
<header className="topbar">
|
||||
<div><span className="eyebrow">ACTIVE MODEL</span><strong>{model}</strong></div>
|
||||
<div className="top-actions">{lastRun?.queueWaitMs != null ? <Badge>queue {Math.round(lastRun.queueWaitMs)}ms</Badge> : null}<Badge><Activity className="size-3" /> slot {cacheSlot + 1}</Badge><Button variant="ghost" size="sm" onClick={() => updateMessages([])} disabled={!messages.length || loading}><Trash2 className="size-3.5" /> Clear</Button></div>
|
||||
<div className="view-tabs">
|
||||
<button className={view === "chat" ? "active" : ""} onClick={() => setView("chat")}><MessageSquareText className="size-3.5" /> Chat</button>
|
||||
<button className={view === "brain" ? "active" : ""} onClick={() => setView("brain")}><BrainCircuit className="size-3.5" /> Brain</button>
|
||||
</div>
|
||||
<div className="top-actions">
|
||||
{loading && tokenCount > 0 ? <Badge className="badge-live"><Zap className="size-3 flash" /> {tokenCount} tokens</Badge> : null}
|
||||
{!loading && tokPerSec != null ? <Badge className="badge-speed"><Gauge className="size-3" /> {tokPerSec.toFixed(1)} tok/s</Badge> : null}
|
||||
{!loading && ttft != null ? <Badge><Timer className="size-3" /> TTFT {(ttft/1000).toFixed(1)}s</Badge> : null}
|
||||
{!loading && lastRun?.usage ? <Badge><Layers className="size-3" /> {lastRun.usage.prompt_tokens}→{lastRun.usage.completion_tokens}</Badge> : null}
|
||||
{lastRun?.queueWaitMs != null ? <Badge><Clock className="size-3" /> queue {Math.round(lastRun.queueWaitMs)}ms</Badge> : null}
|
||||
<Badge><MonitorDot className="size-3" /> slot {cacheSlot + 1}</Badge>
|
||||
<Button variant="ghost" size="sm" onClick={() => { updateMessages([]); setTokPerSec(null); setTtft(null); setTokenCount(0); setTotalTokens({prompt:0,completion:0}) }} disabled={!messages.length || loading}><Trash2 className="size-3.5" /> Clear</Button>
|
||||
</div>
|
||||
</header>
|
||||
|
||||
{view === "brain" ? <Brain baseUrl={baseUrl} apiKey={apiKey} connected={connected} /> : <>
|
||||
|
||||
<div className="conversation">
|
||||
{!messages.length ? (
|
||||
<div className="empty-state">
|
||||
@@ -257,6 +355,7 @@ export default function App() {
|
||||
<div className="composer-foot"><span><MessageSquareText className="size-3.5" /> Enter to send · Shift+Enter for newline</span>{loading ? <Button variant="destructive" size="icon" aria-label="Stop generation" onClick={() => abortRef.current?.abort()}><CircleStop className="size-4" /></Button> : <Button size="icon" aria-label="Send message" disabled={!canSend} onClick={() => void send()}><ArrowUp className="size-4" /></Button>}</div>
|
||||
</div>
|
||||
</div>
|
||||
</>}
|
||||
</main>
|
||||
</div>
|
||||
)
|
||||
|
||||
Reference in New Issue
Block a user