JustVugg 8a1dccae3b docs(api): tell coding-CLI users about the prefill cost before they hit it (#373) ...

yurivict connected crush per the walkthrough and got 'thought for 1h8m and did
nothing' — which is not a hang: agent CLIs send a 10-20k-token system preamble,
and prefill on the CPU-streaming path runs at a few tok/s (attention-bound,
#153). An hour of silent prefill looks exactly like a dead server. The note now
spells out the arithmetic, the curl smoke-test that separates slow-but-working
from broken, and honest guidance on what agent workloads are (not) viable.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

2026-07-18 02:33:23 +02:00

..

experiments

docs: GLM-5.2 6x RTX 5090 full-residency experiment — 6.84 tok/s, honest dead-ends log (#94)

2026-07-12 22:06:51 +02:00

media

docs/media: refreshed dashboard shots (full-residency 4 tok/s, disk 0), new Atlas galaxy, phone + sidebar shots

2026-07-17 00:29:50 +08:00

api.md

docs(api): tell coding-CLI users about the prefill cost before they hit it (#373)

2026-07-18 02:33:23 +02:00

benchmarks.md

README: restructure around the technical path — parameters and full results move to docs/, four new figures

2026-07-16 23:24:37 +08:00

CACHE_ROUTE.md

routing: opt-in CACHE_ROUTE max-rank prefer (pin∪LRU), default off — 2.4->3.33 tok/s on GB10 (#199)

2026-07-14 22:08:28 +02:00

cuda.md

README: restructure around the technical path — parameters and full results move to docs/, four new figures

2026-07-16 23:24:37 +08:00

ENVIRONMENT.md

docs(env): refresh ENVIRONMENT.md — document all 34 drifted env vars

2026-07-17 08:28:41 -04:00

grammar-draft.md

docs: canonical grammar-forced-drafts reference page + README link (#170, fixes #146)

2026-07-14 13:49:44 +02:00

MAINTAINING-DOCS.md

docs: environment variables + CLI settings reference (#151)

2026-07-14 14:10:59 +02:00

METAL-M5MAX-PERF-REPORT.md

docs: Metal M5 Max perf report — OMP hot-team spin (#77) throttles Apple GPU, NO_OMP+PIPE recovers to 2.24 tok/s (#116)

2026-07-13 20:47:19 +02:00

metal.md

README: restructure around the technical path — parameters and full results move to docs/, four new figures

2026-07-16 23:24:37 +08:00

serve_protocol.md

docs: serve protocol reference — mux wire format, telemetry lines, HTTP surface (#310)

2026-07-16 21:25:45 +08:00

SETTINGS.md

docs: environment variables + CLI settings reference (#151)

2026-07-14 14:10:59 +02:00

tuning.md

README: restructure around the technical path — parameters and full results move to docs/, four new figures

2026-07-16 23:24:37 +08:00

windows.md

docs: collapse WINDOWS.md/windows.md case collision into one file (#371)

2026-07-18 01:15:48 +02:00