benchmark_report.md now 7/7 direct + 4/4 transfer, all 0->1.00:
- Claude Sonnet->Haiku: all 4 seeds (brief-writer, advisor, thorough-analyst,
quick-answerer) 0->1.00
- Codex self-optimized: brief-writer, advisor, quick-answerer 0->1.00
- quick-answerer uses the real ./search tool loop on both runtimes.
This matches gbrain's own "4/4 skills 0->1.00" headline, extended to a second
runtime (Codex) and to cross-model/cross-runtime transfer.
Co-Authored-By: Claude Opus 4 <noreply@anthropic.com>
quick-answerer (judge: tool_called=search) reaches 0.00 -> 1.00 with Sonnet
optimizer -> Haiku target: the optimizer wrote an OVERRIDE of the "never use
tools" instruction and the Haiku target genuinely invoked the ./search shim.
All 4 gbrain skillopt-v1 seeds now at 0->1.00, matching gbrain's own headline.
Co-Authored-By: Claude Opus 4 <noreply@anthropic.com>
Strong-optimizer/weak-target (Sonnet -> Haiku), fully isolated:
brief-writer, advisor, thorough-analyst all 0.00 -> 1.00 on held-out.
thorough-analyst shows 2-night convergence (0.33 -> 1.00). Codex self-optimized
brief-writer also 0 -> 1.00.
Key finding answering the optimizer/target-split request: the OPTIMIZER MODEL is
decisive — weak Haiku-as-optimizer is flaky (0 or 1.0 across runs), strong
Sonnet-as-optimizer reliably hits 1.0 on every seed. Raw logs under docs/sleep/raw/.
Co-Authored-By: Claude Opus 4 <noreply@anthropic.com>
Yifan Yang