Addresses the #192 review: server usage documented in docs/grammar-draft.md
(incl. back-compat statement for the additive SUBMIT field and the #100-class
near-tie caveat); gateway pre-checks grammar payloads at 1 MiB (matching the
engine's gbytes bound); negative tests for non-dict response_format, empty and
oversized grammars, plus an explicit test that malformed GBNF passes the
gateway by design (engine fail-soft, draft-source semantics). Measured compile
overhead: 7.8 us/request typical schema, 17.9 us at the 32-level nesting cap.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The OpenAI gateway previously 400'd every response_format; the mux engine path
ran with speculation disabled entirely. Now:
- openai_server.py: response_format {"type":"json_object"} (generic ws-tolerant
JSON grammar), {"type":"json_schema"} (schema forwarded as-is, compiled
engine-side by schema_gbnf.h), and a raw-GBNF {"type":"gbnf"} extension.
Draft-source semantics throughout: a schema the engine cannot compile costs
the speedup, never the request and never the output.
- SUBMIT protocol: optional 7th field gbytes; grammar text appended to the
payload after the prompt. 6-field headers unchanged (back-compatible).
- Engine: per-slot GrDraft (grammar_setup_text/grammar_teardown split out of
the env-driven setup); walkers fed on every emitted token. Grammar-forced
drafting in run_serve_mux for greedy requests: a drafting slot leaves the
shared batch for one forward and runs the proven single-sequence verify path
(kv_bind + step_all) — the same primitives prefill already uses per
submission — then rejoins; rejected drafts' KV entries are overwritten by the
next forward exactly like the existing prefix-truncation path. Sampling
requests never draft (verification under sampling needs rejection resampling;
out of scope).
Tests: 7-field SUBMIT parse cases; response_format->grammar plumbing incl.
fail cases; test doubles updated; generic JSON grammar parse+walk validated
against grammar.h. make test-c green; python suite green except the known
environmental memory_available failure (#150 fixes it).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The engine already tracks where each turn's wall time goes (expert-disk
service, I/O wait, expert matmul, attention, lm_head) — it just only spoke
at exit or under PROF=1. Stream it instead:
- glm.c: mux serve emits a per-turn "PROF" protocol line next to TIERS/HITS
(window deltas per request, same convention as the STAT hit%); the phase
window base is now always captured (a few loads per request).
- openai_server.py: parses PROF into a 120-turn rolling window and serves it
at /profile (read-only, same trust level as /health).
- web: new Profiling tab — stat tiles (tok/s, wall, tokens/forward, disk
service), wall-time composition bars for the last turn and the window,
per-turn throughput and stacked phase columns with hover readouts, and a
table of recent turns. Disk service is shown apart from the stack: it
overlaps with compute, so only the I/O wait the compute thread felt counts
inside wall time. Phase colours are a CVD-validated set with gaps + legend
+ table so identity never rides on colour alone.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WhTmF8yvEBgSkUKSVfZF7P
opencode / the ai-sdk OpenAI-compatible client sends a large default max_tokens
(> the server's --max-tokens cap, 1024 by default), and generation_options
returned 400 "must be an integer between 1 and 1024" — even for a trivial
"hello". OpenAI-compatible servers clamp to their own ceiling rather than
reject. Now max_tokens > limit is clamped to limit; only non-int / non-positive
values are a hard error. Test updated to assert the clamp + keep the <1 reject.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Two OpenAI-compat tool-calling bugs found against the real GLM-5.2 (dnnspaul): (1) string-typed args coerced to numbers — declared schema type now decides, string kept verbatim, bool rejected as number, schema-less params keep permissive decoding; (2) tool_choice was ignored — none/auto/required/{function} now honored, invalid returns 400. Python-only (openai_server.py + tests), engine untouched. 36/36 tests pass (verified independently in a clean worktree).
FABIOTESS