serve: MTP status message tells the truth in multiplexed mode (#358)

`coli serve` runs the engine with SERVE_BATCH=1 (openai_server.py), which
selects run_serve_mux, which sets g_draft=0 — speculation isn't ragged-safe
across the multi-slot batch. But the "[MTP] active: native speculative decoding
(draft=N)" line is printed in main() BEFORE the serve path is chosen, so
`coli serve DRAFT=8` announced draft=8 and then silently disabled it.
@LordMZTE reported the misleading message.

The load line and the [MTP] stderr line now detect the mux case (SERVE +
SERVE_BATCH) and report it truthfully: "MTP DISABLED (multiplexed serve)" with
draft=0, plus a one-line explanation that single-client interactive use
(`coli chat`, which spawns run_serve without SERVE_BATCH) keeps MTP. No more
draft=8 claim on a path that runs draft=0.

This is the honesty half of #358. The feature half — MTP inside the HTTP
server for a single client — is a real enhancement but needs engine work: the
mux decode kernel would have to run the speculative path when exactly one KV
slot is active (S=1 is not actually ragged), and it needs a server round-trip
test. Tracked separately; the message no longer lies in the meantime.

Reported-by: LordMZTE <#358>

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
JustVugg
2026-07-18 01:57:17 +02:00
parent 679c0742b2
commit b09273f73d
+13 -2
View File
@@ -6386,11 +6386,22 @@ int main(int argc, char **argv){
#endif #endif
} }
if(getenv("DSA_TOPK")) m.c.index_topk=atoi(getenv("DSA_TOPK")); /* override per test */ if(getenv("DSA_TOPK")) m.c.index_topk=atoi(getenv("DSA_TOPK")); /* override per test */
/* Il path MUX (SERVE_BATCH=1, cioe' `coli serve`) forza g_draft=0 sotto —
* la speculazione non e' ragged-safe nel batch multi-slot. Segnalarlo QUI,
* altrimenti "MTP active (draft=8)" mentirebbe: il messaggio e' stampato
* prima della scelta del path (run_serve_mux, sotto), e con DRAFT=8 diceva
* "active" per poi disabilitarlo in silenzio (#358, LordMZTE). */
int mux_will_disable_mtp = getenv("SERVE") && getenv("SERVE_BATCH") && atoi(getenv("SERVE_BATCH"));
int eff_draft = mux_will_disable_mtp ? 0 : g_draft;
printf("loaded in %.2fs | resident dense: %.2f MB | layers=%d experts=%d | MTP %s (draft=%d)\n", printf("loaded in %.2fs | resident dense: %.2f MB | layers=%d experts=%d | MTP %s (draft=%d)\n",
now_s()-t0, m.resident_bytes/(1024.0*1024.0), m.c.n_layers, m.c.n_experts, now_s()-t0, m.resident_bytes/(1024.0*1024.0), m.c.n_layers, m.c.n_experts,
m.has_mtp?"ACTIVE":"absent", g_draft); m.has_mtp?(mux_will_disable_mtp?"DISABLED (multiplexed serve)":"ACTIVE"):"absent", eff_draft);
/* anche su stderr: e' il canale che le UI (coli) mostrano all'utente */ /* anche su stderr: e' il canale che le UI (coli) mostrano all'utente */
fprintf(stderr,"[MTP] %s (draft=%d)\n", m.has_mtp?"active: native speculative decoding":"absent", g_draft); if(mux_will_disable_mtp && m.has_mtp)
fprintf(stderr,"[MTP] disabled in multiplexed serve (SERVE_BATCH=1): speculation is not "
"ragged-safe across KV slots. Single-client interactive use (`coli chat`) keeps MTP.\n");
else
fprintf(stderr,"[MTP] %s (draft=%d)\n", m.has_mtp?"active: native speculative decoding":"absent", eff_draft);
#ifdef __linux__ #ifdef __linux__
{ /* Only warn for a GENUINE 9p mount (WSL Windows drives, magic 0x01021997), where { /* Only warn for a GENUINE 9p mount (WSL Windows drives, magic 0x01021997), where
* fadvise is a no-op. The old check was `snap` starting with "/mnt/", which * fadvise is a no-op. The old check was `snap` starting with "/mnt/", which