url/colibri
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

server: clamp max_tokens to the server cap instead of rejecting it (fixes #260)

Browse Source 
opencode / the ai-sdk OpenAI-compatible client sends a large default max_tokens
(> the server's --max-tokens cap, 1024 by default), and generation_options
returned 400 "must be an integer between 1 and 1024" — even for a trivial
"hello". OpenAI-compatible servers clamp to their own ceiling rather than
reject. Now max_tokens > limit is clamped to limit; only non-int / non-positive
values are a hard error. Test updated to assert the clamp + keep the <1 reject.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
JustVugg
2026-07-15 14:56:01 +02:00
parent 5d4c3aa11b
commit da260c38cb
2 changed files with 11 additions and 3 deletions
Download Patch File Download Diff File Expand all files Collapse all files
c/tests/test_openai_server.py
+6 -1
View File
@@ -72,8 +72,13 @@ class TemplateTest(unittest.TestCase):
def test_validates_generation_limits(self):
self.assertEqual(generation_options({"max_tokens": 4, "temperature": 0, "top_p": 1}, 8),
(4, 0.0, 1.0))
# max_tokens above the server cap is clamped, not rejected (#260): OpenAI
# clients default to large values; erroring breaks them.
self.assertEqual(generation_options({"max_tokens": 9, "temperature": 0, "top_p": 1}, 8),
(8, 0.0, 1.0))
# non-positive / non-int max_tokens is still a hard error
with self.assertRaises(APIError):
generation_options({"max_tokens": 9}, 8)
generation_options({"max_tokens": 0}, 8)
with self.assertRaises(APIError):
generation_options({"temperature": math.nan}, 8)
with self.assertRaises(APIError):