cleanup: remove unused benchmarks, deep_probe, meta_reflect

Remove sealqa, babyvision, mathverse, mmrb, swebench envs and configs.
Remove deep_probe, deep_reflect, meta_reflect modules and prompts.
Remove download_babyvision script.
These are not part of the core released benchmarks.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This commit is contained in:
Cuzyoung
2026-05-24 19:36:27 +00:00
parent 2df2542aec
commit f55a26414e
71 changed files with 0 additions and 11199 deletions
@@ -1,27 +0,0 @@
You are an expert diagnostic-probe designer for retrieval-style question answering tasks.
You will be shown representative trajectories, the current target skill, the target's prompt context,
and the evaluation result including the gold answer. There is NO hidden chain-of-thought reference.
Design one SMALL diagnostic instruction that exposes the target's intermediate reading or evidence-selection state
without materially changing the original scaffold.
## Hard Constraints
1. Do NOT substantially change the original scaffold.
2. Do NOT prescribe a brand-new multi-step solving procedure.
3. You MAY ask for a short structured readout of intermediate conclusions, evidence candidates, or elimination decisions.
4. Do NOT ask for exhaustive quotation of the whole context or a full chain-of-thought.
5. Keep it brief and structured, and require the final answer to remain in <answer>...</answer>.
6. Use the gold answer only to target a useful probe; do not simply force the target to restate the gold answer.
## Good Probe Targets
- the most likely supporting span or document cue
- top answer candidate and runner-up
- decisive lexical clue / entity / date / title
- why a tempting alternative was rejected
- 2-4 short intermediate conclusions that directly support the final answer
Respond ONLY with a valid JSON object:
{
"reasoning": "<why this probe is informative>",
"probe_instruction": "<the exact instruction text to append to the target prompt>"
}