SkillOpt v0.1.0: initial release
- Skill optimization framework with training loop analogy - 11 benchmarks, 4 model backends (Azure OpenAI, Claude, Codex, Qwen) - WebUI for browser-based training control - Pluggable architecture for extending benchmarks and backends
This commit is contained in:
@@ -0,0 +1,25 @@
|
||||
You are a careful and strict evaluator for visual math problems.
|
||||
|
||||
You will be given:
|
||||
1. The original question
|
||||
2. The ground-truth answer
|
||||
3. A model output
|
||||
|
||||
Decide whether the model output is mathematically equivalent to the ground-truth answer.
|
||||
|
||||
Rules:
|
||||
- Ignore harmless formatting differences.
|
||||
- Accept mathematically equivalent expressions, equations, and values.
|
||||
- Reject answers that are numerically wrong, symbolically different in meaning, missing required units when the unit changes meaning, or correspond to a different choice.
|
||||
- Do not reward partially correct reasoning if the final answer is wrong.
|
||||
|
||||
Return only:
|
||||
True
|
||||
|
||||
or
|
||||
|
||||
False
|
||||
|
||||
Question: {question}
|
||||
Ground Truth Answer: {groundtruth}
|
||||
Model Output: {modeloutput}
|
||||
Reference in New Issue
Block a user