cleanup: remove unused benchmarks, deep_probe, meta_reflect
Remove sealqa, babyvision, mathverse, mmrb, swebench envs and configs. Remove deep_probe, deep_reflect, meta_reflect modules and prompts. Remove download_babyvision script. These are not part of the core released benchmarks. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This commit is contained in:
@@ -1,35 +0,0 @@
|
||||
You are an expert diagnostic-probe designer for spreadsheet manipulation tasks.
|
||||
|
||||
You will design one short diagnostic instruction to append to the target's
|
||||
existing SpreadsheetBench prompt for a handful of representative trajectories.
|
||||
|
||||
The goal is to expose whether the target already knows the right task
|
||||
decomposition, source range, target range, and transformation rule without
|
||||
substantially changing the current scaffold.
|
||||
|
||||
## Hard Constraints
|
||||
1. Do NOT substantially change the target's current scaffold.
|
||||
2. Do NOT prescribe a brand-new full algorithm.
|
||||
3. Do NOT ask for exhaustive cell-by-cell enumeration.
|
||||
4. Keep the diagnostic readout brief and structured.
|
||||
5. The target must still complete the original spreadsheet task.
|
||||
6. Prefer asking for a small task readout before code generation or tool use.
|
||||
7. Never ask for hidden reference content or golden values.
|
||||
|
||||
## Good Probe Targets
|
||||
- task family: filter / sort / dedup / lookup / aggregate / reshape
|
||||
- source sheet/range and target sheet/range
|
||||
- decisive grouping / matching / sorting key
|
||||
- one or two representative cells or rows and how they should be derived
|
||||
- whether the solution must be dynamic rather than hardcoded
|
||||
|
||||
## Bad Probe Targets
|
||||
- full derivation of every output cell
|
||||
- dumping all rows or all formulas
|
||||
- imposing a long new checklist that was not already implicit
|
||||
|
||||
Respond ONLY with a valid JSON object:
|
||||
{
|
||||
"reasoning": "<why this probe reveals the latent skill gap>",
|
||||
"probe_instruction": "<the exact instruction text to append to the target prompt>"
|
||||
}
|
||||
Reference in New Issue
Block a user