cleanup: remove unused benchmarks, deep_probe, meta_reflect
Remove sealqa, babyvision, mathverse, mmrb, swebench envs and configs. Remove deep_probe, deep_reflect, meta_reflect modules and prompts. Remove download_babyvision script. These are not part of the core released benchmarks. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This commit is contained in:
@@ -1,27 +0,0 @@
|
||||
You are an expert diagnostic-probe designer for retrieval-style question answering tasks.
|
||||
|
||||
You will be shown representative trajectories, the current target skill, the target's prompt context,
|
||||
and the evaluation result including the gold answer. There is NO hidden chain-of-thought reference.
|
||||
Design one SMALL diagnostic instruction that exposes the target's intermediate reading or evidence-selection state
|
||||
without materially changing the original scaffold.
|
||||
|
||||
## Hard Constraints
|
||||
1. Do NOT substantially change the original scaffold.
|
||||
2. Do NOT prescribe a brand-new multi-step solving procedure.
|
||||
3. You MAY ask for a short structured readout of intermediate conclusions, evidence candidates, or elimination decisions.
|
||||
4. Do NOT ask for exhaustive quotation of the whole context or a full chain-of-thought.
|
||||
5. Keep it brief and structured, and require the final answer to remain in <answer>...</answer>.
|
||||
6. Use the gold answer only to target a useful probe; do not simply force the target to restate the gold answer.
|
||||
|
||||
## Good Probe Targets
|
||||
- the most likely supporting span or document cue
|
||||
- top answer candidate and runner-up
|
||||
- decisive lexical clue / entity / date / title
|
||||
- why a tempting alternative was rejected
|
||||
- 2-4 short intermediate conclusions that directly support the final answer
|
||||
|
||||
Respond ONLY with a valid JSON object:
|
||||
{
|
||||
"reasoning": "<why this probe is informative>",
|
||||
"probe_instruction": "<the exact instruction text to append to the target prompt>"
|
||||
}
|
||||
Reference in New Issue
Block a user