serve: #401 diagnostics + optional single-system-block tool rendering (experiment)

Two changes to crack #401 (real coding-CLI clients get plain-text replies where
tool_calls are expected), both aimed at producing evidence and an A/B, neither
changing default behavior:

- parse_tool_calls now separates the two failure modes it could not before:
  markers-present-but-unparsed (mangled int4 output) vs NO markers at all (the
  model never attempted a tool call -- a prompt/behavior issue). COLI_DEBUG
  dumps the raw model reply so we can SEE which one the real scenario hits.

- render_chat gains COLI_TOOL_SYS_MERGE=1: folds the tool declaration into the
  client's system turn (one <|system|>) instead of emitting a separate tool
  system block before it (two consecutive <|system|> turns). Hypothesis: a
  coding CLI's large system prompt as a second system turn buries the tool
  instructions and GLM never emits <tool_call>. Default keeps the historical
  two-block behavior; this is here to A/B against the real model.

44/44 openai tests green (default path byte-identical). Not for merge until
validated against the real GLM-5.2 (needs the disk free).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

README.md

@@ -151,6 +151,10 @@ scale-granularity/rotation ablations live in
 
 ## Get started
 
+> **New here?** The [Quick Start guide](docs/quickstart.md) walks through
+> install → build → model → first chat step by step for Linux, Windows, and
+> macOS, with copy-paste commands and no assumed background.
+
 ### 1. Get the model
 
 A pre-converted **GLM-5.2 int4** container is on Hugging Face — **use the

c/openai_server.py

@@ -271,13 +271,23 @@ def parse_tool_calls(reply, tools=None):
                 salvaged.append(name)
         calls.append({"id": "call_" + uuid.uuid4().hex[:24], "type": "function",
                       "function": {"name": name, "arguments": json.dumps(args, ensure_ascii=False)}})
-    if tools and not calls and re.search(r"</?tool_call>|</?arg_key>|</?arg_value>", reply):
+    if tools and not calls:
-        # Diagnosi per la #401: il client ha dichiarato i tools e il modello ha PROVATO la
+        # Diagnosi #401: distingui i due modi di fallire. (a) marker presenti ma il parse
-        # sintassi, ma il parse rigoroso non ha agganciato nulla (tipico output int4 storpiato).
+        # rigoroso non aggancia -> output storpiato (int4). (b) nessun marker -> il modello
-        # EN: #401 field diagnosis: tools were declared and the model attempted the syntax,
+        # NON ha nemmeno provato a chiamare i tool (prompt/behavior, non parsing). Sapere QUALE
-        # EN: but the strict parse matched nothing (typically quantization-mangled output).
+        # dei due e' il nodo del report: senza distinguerli si tira a indovinare.
-        sys.stderr.write("[api] tools declared and tool-call markers present, but no call "
+        # EN: #401: separate the two failure modes -- markers-but-unparsed (mangled int4 output)
-                         "parsed -- output may be quantization-mangled; try COLI_TOOL_SALVAGE=1\n")
+        # EN: vs no-markers-at-all (the model never attempted a tool call: a prompt/behavior
+        # EN: issue, not a parsing one). COLI_DEBUG dumps the raw reply so we can SEE which.
+        if re.search(r"</?tool_call>|</?arg_key>|</?arg_value>", reply):
+            sys.stderr.write("[api] #401: tools declared, tool-call markers present but nothing "
+                             "parsed -- output likely quantization-mangled; try COLI_TOOL_SALVAGE=1\n")
+        else:
+            sys.stderr.write("[api] #401: tools declared but the reply has NO tool-call markers -- "
+                             "the model answered in plain text (prompt/behavior, not parsing)\n")
+        if os.environ.get("COLI_DEBUG"):
+            head = reply[:800].replace("\n", "\\n")
+            sys.stderr.write("[api] #401 raw model reply (first 800 chars): %s\n" % head)
         sys.stderr.flush()
     text = _BOX_RE.sub("", reply)
     if THINK_CLOSE in text:
@@ -310,32 +320,47 @@ def render_chat(messages, enable_thinking=False, reasoning_effort=None, tools=No
                      if ((t.get("function", t) if isinstance(t, dict) else {}).get("name") == forced)]
     elif tool_choice == "none":
         tools = None                              # the client forbade tools: do not offer them
+    # AUTHORITATIVE GLM-5.2 tool-declaration block (byte-matches chat_template.jinja): the
+    # `# Tools` + <tools></tools> XML structure is what the model was trained on. A made-up
+    # preamble makes it hallucinate other frameworks' syntax (e.g. `end_action`).
+    tool_block = ""
     if tools:
-        # AUTHORITATIVE GLM-5.2 tool-declaration block (byte-matches chat_template.jinja): the
+        parts = ["\n# Tools\n\nYou may call one or more functions to assist with the user "
-        # `# Tools` + <tools></tools> XML structure is what the model was trained on. A made-up
+                 "query.\n\nYou are provided with function signatures within <tools></tools> "
-        # preamble makes it hallucinate other frameworks' syntax (e.g. `end_action`).
+                 "XML tags:\n<tools>\n"]
-        prompt.append("<|system|>\n# Tools\n\nYou may call one or more functions to assist with the "
-                      "user query.\n\nYou are provided with function signatures within <tools></tools> "
-                      "XML tags:\n<tools>\n")
         for tool in tools:
             fn = tool.get("function", tool) if isinstance(tool, dict) else {}
             clean = {k: v for k, v in fn.items() if k not in ("defer_loading", "strict")}
-            prompt.append(json.dumps(clean, ensure_ascii=False) + "\n")
+            parts.append(json.dumps(clean, ensure_ascii=False) + "\n")
-        prompt.append("</tools>\n\nFor each function call, output the function name and arguments "
+        parts.append("</tools>\n\nFor each function call, output the function name and arguments "
-                      "within the following XML format:\n<tool_call>{function-name}"
+                     "within the following XML format:\n<tool_call>{function-name}"
-                      "<arg_key>{arg-key-1}</arg_key><arg_value>{arg-value-1}</arg_value>"
+                     "<arg_key>{arg-key-1}</arg_key><arg_value>{arg-value-1}</arg_value>"
-                      "<arg_key>{arg-key-2}</arg_key><arg_value>{arg-value-2}</arg_value>...</tool_call>")
+                     "<arg_key>{arg-key-2}</arg_key><arg_value>{arg-value-2}</arg_value>...</tool_call>")
         if forced:
-            prompt.append(f"\n\nYou must call the function `{forced}`. Do not answer directly.")
+            parts.append(f"\n\nYou must call the function `{forced}`. Do not answer directly.")
         elif tool_choice == "required":
-            prompt.append("\n\nYou must call one of the functions above. Do not answer directly.")
+            parts.append("\n\nYou must call one of the functions above. Do not answer directly.")
+        tool_block = "".join(parts)
+    # #401 experiment: a coding CLI sends its own big system prompt, so the default emits TWO
+    # consecutive <|system|> turns (tools, then the client's) -- GLM may prioritize the later
+    # one and never see the tool instructions. COLI_TOOL_SYS_MERGE=1 folds the tool block into
+    # the client's system turn instead (one <|system|>, tools last), to A/B which one actually
+    # makes the model emit tool calls. Default keeps the historical two-block behavior.
+    merge_tools = bool(tool_block) and os.environ.get("COLI_TOOL_SYS_MERGE") == "1"
+    if tool_block and not merge_tools:
+        prompt.append("<|system|>" + tool_block)
+    tools_pending = tool_block if merge_tools else ""
     prev_tool = False
     for index, message in enumerate(messages):
         if not isinstance(message, dict):
             raise APIError(400, "Each message must be an object.", f"messages.{index}")
         role = message.get("role")
         if role in ("system", "developer"):
-            prompt.append(f"<|system|>{content_text(message.get('content'), f'messages.{index}.content')}")
+            body = content_text(message.get('content'), f'messages.{index}.content')
+            if tools_pending:                       # merge mode: fold tools into this system turn
+                body = body + "\n" + tools_pending
+                tools_pending = ""
+            prompt.append(f"<|system|>{body}")
         elif role == "user":
             prompt.append(f"<|user|>{content_text(message.get('content'), f'messages.{index}.content')}")
         elif role == "assistant":
@@ -365,6 +390,8 @@ def render_chat(messages, enable_thinking=False, reasoning_effort=None, tools=No
             raise APIError(400, f"Unsupported message role: {role!r}.",
                            f"messages.{index}.role", "unsupported_role")
         prev_tool = (role == "tool")
+    if tools_pending:                               # merge mode but no client system turn existed
+        prompt.append("<|system|>" + tools_pending)
     prompt.append("<|assistant|><think>" if enable_thinking else
                   "<|assistant|><think></think>")
     return "".join(prompt)

docs/quickstart.md

@@ -0,0 +1,164 @@
+# Quick Start — from zero to a running model
+
+A step-by-step guide for first-time users on **Linux**, **Windows**, and **macOS**.
+No prior experience with C, CUDA, or model conversion is assumed. If you get
+stuck, `./coli doctor` (below) tells you exactly what's missing.
+
+> **What you're setting up:** colibrì runs a very large Mixture-of-Experts model
+> (e.g. GLM-5.2, 744B parameters) on a normal machine by streaming the model's
+> experts from disk instead of needing them all in RAM. The engine is a single
+> C program; Python is only used once, to prepare the model files.
+
+---
+
+## 0. What you need first (prerequisites)
+
+| | Minimum | Recommended |
+|---|---|---|
+| **RAM** | ~16 GB | 24 GB+ |
+| **Free disk** | ~380 GB for the int4 model | a fast NVMe SSD (streaming speed = your token speed) |
+| **OS** | Linux, Windows 10/11, or macOS | any |
+| **Tools** | a C compiler + `make` + `git` + `python3` | — |
+
+You do **not** need a GPU. A GPU only helps if you have one; the engine runs
+CPU-only by default.
+
+---
+
+## 1. Install the build tools
+
+### Linux (Ubuntu / Debian)
+
+```bash
+sudo apt update
+sudo apt install -y build-essential git python3
+```
+
+`build-essential` gives you `gcc`, `make`, and OpenMP (libgomp) — everything the
+engine needs.
+
+### Windows
+
+You have two options.
+
+**Option A — download a prebuilt binary (no compiler needed).**
+Grab the latest `colibri-<version>-windows-x86_64.zip` from the
+[Releases page](https://github.com/JustVugg/colibri/releases), unzip it, and
+skip to [step 3](#3-get-the-model). Python 3 (from
+[python.org](https://www.python.org/downloads/)) is still needed if you want to
+convert a model yourself.
+
+**Option B — build from source with MSYS2.**
+Install [MSYS2](https://www.msys2.org/), open the **UCRT64** shell, and run:
+
+```bash
+pacman -S --needed mingw-w64-ucrt-x86_64-gcc make git python
+```
+
+### macOS
+
+```bash
+xcode-select --install          # C compiler (clang)
+brew install libomp git python  # OpenMP for multithreading
+```
+
+---
+
+## 2. Get the code and build the engine
+
+```bash
+git clone https://github.com/JustVugg/colibri.git
+cd colibri/c
+./setup.sh
+```
+
+`setup.sh` checks your compiler and OpenMP, builds the engine, and runs a tiny
+self-test. When it prints:
+
+```
+engine self-test: 32/32  (expected 32/32)
+```
+
+the engine is working correctly. (On Windows Option A you already have the
+binary — you can skip this step.)
+
+---
+
+## 3. Get the model
+
+You have two paths.
+
+### Easiest — download a ready-made int4 container
+
+A pre-converted **GLM-5.2 int4** model is on Hugging Face. **Use the version
+with the int8 MTP heads** (the plain int4 heads disable speculative decoding —
+see [#8](https://github.com/JustVugg/colibri/issues/8)):
+
+**https://huggingface.co/mateogrgic/GLM-5.2-colibri-int4-with-int8-mtp**
+
+Download it into a folder on a fast disk, e.g. `/nvme/glm52_i4` (Linux/macOS) or
+`D:\glm52_i4` (Windows). It is about **372 GB**, so make sure you have the space.
+
+### Or convert it yourself from the FP8 source
+
+One resumable command downloads and converts the model shard by shard, so it
+never needs the full ~756 GB on disk at once:
+
+```bash
+./coli convert --model /nvme/glm52_i4
+```
+
+This step uses Python and runs only once. Safe to interrupt and re-run — it
+resumes where it left off.
+
+---
+
+## 4. Run it
+
+Point `COLI_MODEL` at the folder from step 3 and start chatting:
+
+```bash
+# Linux / macOS
+COLI_MODEL=/nvme/glm52_i4 ./coli chat
+
+# Windows (UCRT64 shell)
+COLI_MODEL=/d/glm52_i4 ./coli chat
+```
+
+Useful first commands:
+
+```bash
+COLI_MODEL=/nvme/glm52_i4 ./coli doctor   # read-only check: is everything ready?
+COLI_MODEL=/nvme/glm52_i4 ./coli plan     # shows where the model will live (RAM/disk/GPU)
+COLI_MODEL=/nvme/glm52_i4 ./coli chat --topp 0.85   # faster: reads less from disk, same quality
+```
+
+> **Tip:** `--topp 0.85` is worth adding on a disk-bound machine — it reads
+> fewer expert bytes per token with no quality loss, which directly means more
+> tokens per second.
+
+---
+
+## 5. What to expect
+
+- **First launch loads the resident weights** (~10 GB) — this takes a moment.
+- **Speed depends on your disk.** The experts stream from storage, so a fast
+  NVMe SSD is the single biggest factor in tokens/second. On a slow or shared
+  disk, generation can be well under 1 token/second — that's expected, and it's
+  the honest cost of running a 744B model on a small machine.
+- **It's still the full model.** Placement only changes speed, never the model's
+  answers or precision.
+
+If something doesn't work, run `./coli doctor` — it reports exactly what's
+missing (compiler, model files, permissions) and how to fix it.
+
+---
+
+## Where to go next
+
+| Topic | Doc |
+|---|---|
+| Windows native build (and CUDA DLL) | [docs/windows.md](windows.md) |
+| Tuning: cache, prefetch, speculation | [docs/tuning.md](tuning.md) |
+| OpenAI-compatible API + web dashboard | [docs/api.md](api.md) |
+| Every environment variable | [docs/ENVIRONMENT.md](ENVIRONMENT.md) |