fix(model): forward Qwen timeout and only set enable_thinking when true
Two bugs made local vLLM targets score acc=0.000: the router did not forward 'timeout' to the Qwen backend (so runs used the 300s default), and qwen_backend always injected chat_template_kwargs.enable_thinking, which non-Qwen vLLM servers reject or answer with <think> output and no <answer> tag. Forward timeout and only set the field when enabled. Closes #28 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -142,6 +142,7 @@ def chat_target(
|
||||
retries=retries,
|
||||
stage=stage,
|
||||
reasoning_effort=reasoning_effort,
|
||||
timeout=timeout,
|
||||
)
|
||||
if get_target_backend() == "minimax_chat":
|
||||
return _minimax.chat_target(
|
||||
@@ -249,6 +250,7 @@ def chat_target_messages(
|
||||
tools=tools,
|
||||
tool_choice=tool_choice,
|
||||
return_message=return_message,
|
||||
timeout=timeout,
|
||||
)
|
||||
if get_target_backend() == "minimax_chat":
|
||||
return _minimax.chat_target_messages(
|
||||
|
||||
@@ -191,7 +191,8 @@ def _chat_messages_impl(
|
||||
"messages": _json_safe(messages),
|
||||
"max_tokens": min(max_completion_tokens, config.max_tokens),
|
||||
}
|
||||
payload["chat_template_kwargs"] = {"enable_thinking": config.enable_thinking}
|
||||
if config.enable_thinking:
|
||||
payload["chat_template_kwargs"] = {"enable_thinking": True}
|
||||
if config.temperature is not None:
|
||||
payload["temperature"] = config.temperature
|
||||
if tools:
|
||||
|
||||
Reference in New Issue
Block a user