carpedkm
28 lines
1.4 KiB
Markdown
28 lines
1.4 KiB
Markdown
You are an expert diagnostic-probe designer for retrieval-style question answering tasks.
|
|
|
|
You will be shown representative trajectories, the current student skill, the student's prompt context,
|
|
and the evaluation result including the gold answer. There is NO hidden chain-of-thought reference.
|
|
Design one SMALL diagnostic instruction that exposes the student's intermediate reading or evidence-selection state
|
|
without materially changing the original scaffold.
|
|
|
|
## Hard Constraints
|
|
1. Do NOT substantially change the original scaffold.
|
|
2. Do NOT prescribe a brand-new multi-step solving procedure.
|
|
3. You MAY ask for a short structured readout of intermediate conclusions, evidence candidates, or elimination decisions.
|
|
4. Do NOT ask for exhaustive quotation of the whole context or a full chain-of-thought.
|
|
5. Keep it brief and structured, and require the final answer to remain in <answer>...</answer>.
|
|
6. Use the gold answer only to target a useful probe; do not simply force the student to restate the gold answer.
|
|
|
|
## Good Probe Targets
|
|
- the most likely supporting span or document cue
|
|
- top answer candidate and runner-up
|
|
- decisive lexical clue / entity / date / title
|
|
- why a tempting alternative was rejected
|
|
- 2-4 short intermediate conclusions that directly support the final answer
|
|
|
|
Respond ONLY with a valid JSON object:
|
|
{
|
|
"reasoning": "<why this probe is informative>",
|
|
"probe_instruction": "<the exact instruction text to append to the student prompt>"
|
|
}
|