From 73e0e5ec82e98ef40bcfab5abd42c6776e6a381e Mon Sep 17 00:00:00 2001
From: KingIcyCreamProjects <safereesekapellen@gmail.com>
Date: Fri, 17 Jul 2026 16:24:33 -0500
Subject: [PATCH] docs(api): add "Connect a coding CLI or editor" recipe (#373)
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Documents connecting OpenAI-compatible coding CLIs to `coli serve`.
Covers the common snag reported in #373 — clients like crush refuse to
start without an API key even though the local endpoint needs none — by
showing that any dummy key works (Colibri only enforces COLI_API_KEY if
set). Concrete recipes for aider and crush, a curl smoke test, the
generic base-URL/model/key pattern for other tools, and an honest
tok/s-latency caveat for the streaming path.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---
 docs/api.md | 61 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 61 insertions(+)

diff --git a/docs/api.md b/docs/api.md
index d3b09f1..cb29eb2 100644
--- a/docs/api.md
+++ b/docs/api.md
@@ -49,6 +49,67 @@ errors before streaming headers are sent. `GET /health` exposes
 active/queued/completed/rejected counters, and successful generation responses
 include `x-colibri-queue-wait-ms`.
 
+## Connect a coding CLI or editor
+
+The API is OpenAI-compatible, so most coding CLIs and editor extensions work by
+pointing them at Colibri as an *OpenAI-compatible* provider. Three settings:
+
+- **Base URL** — `http://localhost:8000/v1`
+- **Model** — `glm-5.2-colibri` (or whatever you pass to `--model-id`)
+- **API key** — any non-empty string, e.g. `local`
+
+Colibri needs **no** API key by default, but many clients refuse to start without
+one — give them any dummy value. The key is only enforced if you set `COLI_API_KEY`.
+
+Smoke-test the endpoint first (no key needed unless you set one):
+
+```bash
+curl http://127.0.0.1:8000/v1/chat/completions \
+  -H 'Content-Type: application/json' \
+  -d '{"model":"glm-5.2-colibri","messages":[{"role":"user","content":"hi"}]}'
+```
+
+**aider**
+
+```bash
+export OPENAI_API_BASE=http://localhost:8000/v1
+export OPENAI_API_KEY=local
+aider --model openai/glm-5.2-colibri     # the openai/ prefix routes to OPENAI_API_BASE
+```
+
+**crush** — add a provider to `crush.json` (`~/.config/crush/crush.json`, or
+`%USERPROFILE%\AppData\Local\crush\crush.json` on Windows):
+
+```json
+{
+  "$schema": "https://charm.land/crush.json",
+  "providers": {
+    "colibri": {
+      "name": "Colibri",
+      "type": "openai-compat",
+      "base_url": "http://localhost:8000/v1/",
+      "api_key": "local",
+      "models": [
+        { "name": "GLM-5.2 (Colibri)", "id": "glm-5.2-colibri",
+          "context_window": 131072, "default_max_tokens": 1024 }
+      ]
+    }
+  }
+}
+```
+
+The `"api_key": "local"` dummy is what satisfies clients that demand a key.
+`context_window` is only the client's budget display — set it to whatever your
+KV configuration actually allows.
+
+**Continue, Cline / Roo, `llm`, the OpenAI SDKs, …** — set the provider's base
+URL to `http://localhost:8000/v1`, the model to `glm-5.2-colibri`, and any dummy
+key (`OPENAI_API_KEY` / `OPENAI_BASE_URL` for env-based tools).
+
+> On the CPU-streaming path a large model decodes at roughly 1 tok/s, so
+> interactive agent loops will feel slow — it connects and works, but the latency
+> is very different from a hosted model.
+
 ## Isolated KV contexts
 
 `coli serve --kv-slots N` allocates up to 16 independent sequence contexts.