yurivict connected crush per the walkthrough and got 'thought for 1h8m and did
nothing' — which is not a hang: agent CLIs send a 10-20k-token system preamble,
and prefill on the CPU-streaming path runs at a few tok/s (attention-bound,
#153). An hour of silent prefill looks exactly like a dead server. The note now
spells out the arithmetic, the curl smoke-test that separates slow-but-working
from broken, and honest guidance on what agent workloads are (not) viable.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Documents connecting OpenAI-compatible coding CLIs to `coli serve`.
Covers the common snag reported in #373 — clients like crush refuse to
start without an API key even though the local endpoint needs none — by
showing that any dummy key works (Colibri only enforces COLI_API_KEY if
set). Concrete recipes for aider and crush, a curl smoke test, the
generic base-URL/model/key pattern for other tools, and an honest
tok/s-latency caveat for the streaming path.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
JustVugg