chore(release): v0.2.0

Highlights since v0.1.0:
- feat: SkillOpt-Sleep engine — nightly offline self-evolution
  (harvest -> mine -> replay -> consolidate behind a validation gate),
  with multi-objective reward, experience replay + dream rollouts,
  slow-update long-term memory, and secret redaction in cycle diagnostics.
  Shipped as the `skillopt-sleep` CLI.
- feat: cross-tool backends & plugin shells — Claude, Codex (+Desktop
  harvest), Copilot, Devin, and OpenClaw.
- feat: SearchQA split materialization + rollout fail-fast.
- fix: Windows robustness for claude/codex backends, hardened JSON
  fallback, Qwen timeout/thinking gating, Codex failure surfacing.

Packaging:
- Bump pyproject / skillopt / skillopt_sleep to 0.2.0.
- Restore skillopt_webui to the packaged wheel.

See CHANGELOG.md for the full changelog and contributor acknowledgements.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

.gitignore

@@ -22,6 +22,8 @@ data/*
 outputs/
 logs/
 external/
+# SkillOpt-Sleep runtime state (staging proposals, config, diagnostics, cron logs)
+.skillopt-sleep/
 
 /BabyVision/
 /MMRB/

CHANGELOG.md

@@ -0,0 +1,100 @@
+# Changelog
+
+All notable changes to SkillOpt are documented here. This project adheres to
+[Semantic Versioning](https://semver.org/) and the format is based on
+[Keep a Changelog](https://keepachangelog.com/).
+
+## [0.2.0] — 2026-07-02
+
+The headline of this release is **SkillOpt-Sleep**: a nightly offline
+self-evolution engine that harvests a coding agent's real session
+transcripts, mines recurring tasks, replays them offline, and consolidates
+short-term experience into long-term memory and skills — all behind the same
+held-out validation gate that keeps SkillOpt training honest. It ships as a
+decoupled top-level package (`skillopt_sleep/`, zero dependency on the
+research code) and as the new `skillopt-sleep` CLI.
+
+### Added
+- **SkillOpt-Sleep engine** — nightly offline self-evolution cycle
+  (harvest → mine → replay → consolidate) behind a validation gate, exposed
+  as the `skillopt-sleep` console script and `python -m skillopt_sleep`.
+  - Multi-objective reward (accuracy / tokens / latency) with user preferences.
+  - Multi-rollout contrastive reflection under a token/time budget.
+  - Experience replay + controllable dream rollouts (opt-in).
+  - Slow-update long-term memory field (runs even with the gate off).
+  - 3-way train/val/test split with `gate_mode on|off`.
+  - Verifier-discipline validation gate, with a stress-test suite
+    (thanks @Tanmay9223, #87).
+- **Cross-tool backends & plugin shells** for Claude Code, Codex, Copilot,
+  Devin, and OpenClaw:
+  - Codex Desktop transcript harvesting, skill-first Codex integration, and a
+    reviewed task-file flow (thanks @Kirchberg, #48, #49, #60).
+  - GitHub Copilot backend (`CopilotCliBackend`) + research-engine MCP plugin
+    (thanks @Dongbumlee, #50).
+  - Devin plugin: MCP server + ATIF-v1.7 harvest (thanks @xerxes-y, #88).
+  - OpenClaw shell for SkillOpt-Sleep (thanks @Elzlxx, #59).
+- **SearchQA** split materialization helper and fail-fast on systemic rollout
+  failures, with a `searchqa` install extra (thanks @summerview1997,
+  #63, #64, #65).
+- WebUI environment loading and backend preflight (thanks @summerview1997, #63).
+
+### Changed
+- Decoupled the Sleep engine into a standalone top-level `skillopt_sleep/`
+  package with zero dependency on the research code.
+- Made `EnvAdapter.reflect` a shared default so reflect kwargs are no longer
+  dropped (thanks @imshunsuke, #44).
+- English-only pass across the engine, plugins, and docs.
+
+### Fixed
+- Windows robustness for the Claude/Codex backends, plus a hardened JSON
+  fallback path (thanks @Yif-Yang, #79).
+- Reject prose pseudo-JSON wrapped in single quotes/backticks (#82).
+- Surface Codex auth/model/version failures instead of silently scoring 0
+  (thanks @dmmdea, #92).
+- Redact secrets before persisting cycle diagnostics.
+- Configure the `qwen_chat`/`minimax` backends so local LLM endpoints work
+  (thanks @imrehg, #85).
+- Forward the Qwen target timeout and gate `enable_thinking` for vLLM targets
+  (thanks @mvanhorn, #40).
+- Make `--bare` conditional on `ANTHROPIC_API_KEY` (#68), add a
+  `SKILLOPT_SLEEP_PYTHON` override with a lookback-hours first-run fallback
+  (#74), and fix ALFWorld gamefile paths relative to `ALFWORLD_DATA`.
+
+### Packaging
+- Bump `skillopt`, `skillopt.__version__`, and `skillopt_sleep.__version__`
+  to `0.2.0`.
+- Restore `skillopt_webui` to the built wheel (it was dropped when the
+  `packages.find` include list was made explicit).
+- Add the `searchqa` extra and include `json_repair` in the `claude`, `qwen`,
+  and `all` extras.
+
+### Acknowledgements 🙏
+v0.2.0 landed thanks to our community contributors — thank you!
+
+- @Kirchberg — Codex Desktop harvesting, skill-first Codex integration,
+  reviewed task-file flow (#48, #49, #60)
+- @Dongbumlee — GitHub Copilot backend + research-engine MCP plugin (#50)
+- @summerview1997 — SearchQA materialization, rollout fail-fast, WebUI
+  preflight (#63, #64, #65)
+- @xerxes-y — Devin plugin: MCP server + ATIF-v1.7 harvest (#88)
+- @Elzlxx — OpenClaw shell for SkillOpt-Sleep (#59)
+- @imshunsuke — shared `EnvAdapter.reflect` default + docs fixes (#43, #44)
+- @mvanhorn — Qwen timeout forwarding + `enable_thinking` gating (#40)
+- @dmmdea — surface Codex auth/model/version failures (#92)
+- @Tanmay9223 — verifier-discipline stress test (#87)
+- @imrehg — `configure_qwen_chat` for local LLM endpoints (#85)
+- @samuelgoofus-boop — community contributions
+
+Special thanks to @Yif-Yang for driving the SkillOpt-Sleep engine.
+
+**Full changelog:** https://github.com/microsoft/SkillOpt/compare/v0.1.0...v0.2.0
+
+## [0.1.0] — 2026-06-02
+
+Initial public release: the full training loop (rollout → reflect →
+aggregate → select → update → evaluate), multi-backend support
+(OpenAI / Azure / Claude / Qwen / MiniMax), six built-in benchmarks, and the
+WebUI dashboard.
+
+[0.2.0]: https://github.com/microsoft/SkillOpt/releases/tag/v0.2.0
+[0.1.0]: https://github.com/microsoft/SkillOpt/releases/tag/v0.1.0

PUBLISHING.md

@@ -1,81 +0,0 @@
-# Publishing SkillOpt-Sleep — how people install and use it
-
-This is the open-source SkillOpt-Sleep tool: a nightly offline "sleep cycle" for
-local coding agents, shipped as plugins for **Claude Code**, **Codex**, and
-**Copilot**. One engine ([`skillopt_sleep/`](skillopt_sleep)), three thin shells
-([`plugins/`](plugins)), decoupled from the research code.
-
-## How end users install it
-
-### Claude Code
-
-The Claude Code plugin ships a marketplace manifest at
-`plugins/claude-code/.claude-plugin/marketplace.json`.
-
-```text
-# inside Claude Code:
-/plugin marketplace add microsoft/SkillOpt
-/plugin install skillopt-sleep
-/sleep status
-```
-
-(`/plugin marketplace add <owner>/<repo>` reads the marketplace manifest from the
-repo; the entry points at `plugins/claude-code`.)
-
-### Codex
-
-```bash
-git clone https://github.com/microsoft/SkillOpt.git
-cd SkillOpt
-bash plugins/codex/install.sh           # installs /sleep prompt + skill
-export SKILLOPT_SLEEP_REPO="$(pwd)"      # so the runner is found anywhere
-# then, in Codex:  /sleep status
-```
-
-### Copilot
-
-```bash
-git clone https://github.com/microsoft/SkillOpt.git
-# register the MCP server with your Copilot config (see plugins/copilot/README.md
-# and plugins/copilot/mcp-config.example.json), pointing SKILLOPT_SLEEP_REPO at
-# the clone. Then ask Copilot to "run the sleep cycle".
-```
-
-Requirements for all three: Python ≥ 3.10, and the corresponding agent CLI on
-PATH. The default backend is `mock` (no API spend); `--backend claude|codex`
-uses the user's own budget.
-
-## Wider distribution (optional, maintainer steps)
-
-1. **GitHub Release.** Tag the milestone so users can pin a version:
-   ```bash
-   gh release create sleep-v0.1.0 --title "SkillOpt-Sleep v0.1.0" \
-     --notes "Nightly offline self-evolution plugins for Claude Code, Codex, Copilot."
-   ```
-
-2. **Official Claude Code plugin marketplace.** To appear in the public
-   directory, open a PR adding a `marketplace.json` entry to
-   [`anthropics/claude-code` / the official marketplace repo], pointing at
-   `microsoft/SkillOpt` subdir `plugins/claude-code`. Users could then
-   `/plugin install skillopt-sleep@<official-marketplace>`.
-
-3. **PyPI (optional).** `skillopt_sleep` is a standalone package
-   (`pyproject.toml` lists it). A `pip install skillopt-sleep` distribution would
-   let users run `python -m skillopt_sleep ...` without cloning. Build with
-   `python -m build` and publish with `twine`.
-
-4. **README News.** The main [`README.md`](README.md) already announces the
-   release and links to [`plugins/`](plugins) and
-   [`docs/sleep/FINAL_REPORT.md`](docs/sleep/FINAL_REPORT.md).
-
-## Verifying a release works
-
-```bash
-# deterministic, no API key:
-python -m skillopt_sleep.experiments.run_experiment --persona researcher --assert-improves
-# the unit suite:
-python -m unittest tests.test_sleep_engine
-# the MCP server (Copilot):
-printf '%s\n' '{"jsonrpc":"2.0","id":1,"method":"tools/list"}' \
-  | SKILLOPT_SLEEP_REPO="$(pwd)" python3 plugins/copilot/mcp_server.py
-```

README.md

@@ -4,12 +4,18 @@
 
 [![Project Page](https://img.shields.io/badge/Project%20Page-SkillOpt-8dbb3c)](https://microsoft.github.io/SkillOpt/) [![Paper](https://img.shields.io/badge/Paper-arXiv-b31b1b)](https://arxiv.org/abs/2605.23904) [![Project Video](https://img.shields.io/badge/Project%20Video-Watch%20Demo-ff0000)](https://youtu.be/JUBMDTCiM0M) [![PyPI](https://img.shields.io/badge/PyPI-skillopt-green.svg)](https://pypi.org/project/skillopt/) [![Python 3.10+](https://img.shields.io/badge/Python-3.10%2B-blue.svg)](https://www.python.org/) [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE)
 
-> 📖 **For installation, data preparation, training/eval commands, the full configuration reference, and framework internals, see the [Documentation & Reproduction Guide](docs/guideline.html)** — view it [rendered online](https://htmlpreview.github.io/?https://github.com/microsoft/SkillOpt/blob/main/docs/guideline.html) or via [GitHub Pages](https://microsoft.github.io/SkillOpt/docs/guideline.html).
+<p align="center">
+  <a href="https://trendshift.io/repositories/38498?utm_source=trendshift-badge&utm_medium=badge&utm_campaign=badge-trendshift-38498" target="_blank" rel="noopener noreferrer"><img src="https://trendshift.io/api/badge/trendshift/repositories/38498/daily?language=Python" alt="microsoft%2FSkillOpt | Trendshift" width="250" height="55"/></a>
+  <a href="https://trendshift.io/repositories/38498?utm_source=trendshift-badge&utm_medium=badge&utm_campaign=badge-trendshift-38498" target="_blank" rel="noopener noreferrer"><img src="https://trendshift.io/api/badge/trendshift/repositories/38498/weekly?language=Python" alt="microsoft%2FSkillOpt | Trendshift" width="250" height="55"/></a>
+</p>
+
+> 📖 **For installation, data preparation, training/eval commands, the full configuration reference, and framework internals, see the [Documentation & Reproduction Guide](https://microsoft.github.io/SkillOpt/docs/guideline.html)** (rendered on GitHub Pages).
 
 ---
 
 ## News 🔥🔥🔥
-- **[2026-06-08]** 😴 **SkillOpt-Sleep is here — plugins for Claude Code, Codex, and Copilot.** Give your local coding agent a nightly *sleep cycle*: it reviews your past sessions offline, replays your recurring tasks, and consolidates validated long-term memory + skills behind a held-out gate, so it gets better the more you use it. Validated on the public [gbrain-evals](https://github.com/garrytan/gbrain-evals) `skillopt-v1` benchmark with **real Claude and Codex** (deficient skills 0.00 → 1.00 on held-out, all 4 seeds). It's an **open-source tool decoupled from the paper code**. See [`plugins/`](plugins/) and the [SkillOpt-Sleep section](#-skillopt-sleep--the-deployment-time-companion) below.
+- **[2026-07-02]** 🚀 **SkillOpt [v0.2.0](https://github.com/microsoft/SkillOpt/releases/tag/v0.2.0) is out on [PyPI](https://pypi.org/project/skillopt/)!** Headline feature: **SkillOpt-Sleep**, a nightly offline self-evolution engine (harvest → mine → replay → consolidate, all behind a held-out validation gate) with multi-objective reward, experience replay + dream rollouts, and long-term memory — now shipped as the `skillopt-sleep` CLI. This release also adds cross-tool backends and plugin shells for **Claude, Codex, Copilot, Devin, and OpenClaw**, SearchQA split materialization, Windows robustness, and hardened JSON parsing. See the [release notes](https://github.com/microsoft/SkillOpt/releases/tag/v0.2.0) for the full changelog and contributor acknowledgements.
+- **[2026-06-15]** 😴 **SkillOpt-Sleep (preview)** — a nightly offline self-evolution companion for local coding agents (Claude Code / Codex / Copilot): review past sessions, replay recurring tasks, and consolidate validated skills behind a held-out gate. See **[`docs/sleep/README.md`](docs/sleep/README.md)** for what it is, how to use it, and results.
 - **[2026-06-03]** 🎉 **[gbrain](https://github.com/garrytan/gbrain), [gbrain-evals](https://github.com/garrytan/gbrain-evals/blob/main/docs/benchmarks/2026-06-03-skillopt.md), and [darwin-skill](https://github.com/alchaincyf/darwin-skill) have all integrated SkillOpt.**
 - **[2026-06-02]** 🎉 **SkillOpt [v0.1.0](https://github.com/microsoft/SkillOpt/releases/tag/v0.1.0) is now available on [PyPI](https://pypi.org/project/skillopt/)!** Install with `pip install skillopt`. This initial release includes the full training loop (rollout → reflect → aggregate → select → update → evaluate), multi-backend support (OpenAI / Azure / Claude / Qwen / MiniMax), six built-in benchmarks, and WebUI dashboard.
 
@@ -53,51 +59,6 @@ https://github.com/user-attachments/assets/eb12d3bc-371c-467f-904d-91b61f339ed7
 
 ---
 
-## 😴 SkillOpt-Sleep — the deployment-time companion
-
-SkillOpt (above) trains a skill offline on a benchmark. **SkillOpt-Sleep**
-applies the same discipline to *your own daily usage*: it gives a local coding
-agent a nightly **sleep cycle** that reviews your past sessions, replays your
-recurring tasks on your own API budget, and consolidates what it learns into
-**validated** long-term memory and skills — behind a held-out gate, staged for
-your review. The agent gets better the more you use it, with no weight training.
-
-It synthesizes **SkillOpt** (validation-gated bounded text edits), **Claude
-Dreams** (offline consolidation; review-then-adopt), and the **agent sleep**
-idea (short-term experience → long-term competence). One "night":
-
-```
-harvest session transcripts → mine recurring tasks → replay offline
-   → consolidate (reflect → bounded edit → GATE on real held-out tasks)
-   → stage proposal → (you) adopt
-```
-
-**Plugins for three agents** (one engine, three thin shells — see [`plugins/`](plugins/)):
-
-| Platform | Folder | Install |
-|---|---|---|
-| **Claude Code** | [`plugins/claude-code`](plugins/claude-code) | `/plugin marketplace add ./plugins/claude-code` → `/sleep` |
-| **Codex** | [`plugins/codex`](plugins/codex) | `bash plugins/codex/install.sh` → `/sleep` |
-| **Copilot** | [`plugins/copilot`](plugins/copilot) | register `plugins/copilot/mcp_server.py` as an MCP server |
-
-**Validated on real models.** On the public
-[gbrain-evals](https://github.com/garrytan/gbrain-evals) `skillopt-v1` benchmark,
-deficient skills go **0.00 → 1.00** on held-out sets with **both Claude and
-Codex** (all 4 seeds, including a real tool-use loop), cross-model transfer is
-positive, and the gate blocks regressions
-([full results](docs/sleep/FINAL_REPORT.md)).
-
-> **Open-source tool, decoupled from the research.** The engine lives in the
-> top-level [`skillopt_sleep/`](skillopt_sleep) package with **zero dependency**
-> on the paper's `skillopt/` experiment code (the validation gate is vendored).
-> Controls — optional gate, multi-rollout contrastive reflection, token/time
-> budget, multi-objective reward, user preferences, optimizer/target split — are
-> documented in [`docs/sleep/CONTROLLABLE_DREAMING.md`](docs/sleep/CONTROLLABLE_DREAMING.md).
-
-Deterministic proof (no API key): `python -m skillopt_sleep.experiments.run_experiment --persona researcher --assert-improves`.
-
----
-
 ## Extensibility & WebUI
 
 ### Adding a new backend
@@ -137,13 +98,10 @@ python -m skillopt_webui.app
 ## Citation
 
 ```bibtex
-@misc{yang2026skilloptexecutivestrategyselfevolving,
+@article{yang2026skillopt,
-      title={SkillOpt: Executive Strategy for Self-Evolving Agent Skills}, 
+  title={Skillopt: Executive strategy for self-evolving agent skills},
-      author={Yifan Yang and Ziyang Gong and Weiquan Huang and Qihao Yang and Ziwei Zhou and Zisu Huang and Yan Li and Xuemei Gao and Qi Dai and Bei Liu and Kai Qiu and Yuqing Yang and Dongdong Chen and Xue Yang and Chong Luo},
+  author={Yang, Yifan and Gong, Ziyang and Huang, Weiquan and Yang, Qihao and Zhou, Ziwei and Huang, Zisu and Li, Yan and Gao, Xuemei and Dai, Qi and Liu, Bei and others},
-      year={2026},
+  journal={arXiv preprint arXiv:2605.23904},
-      eprint={2605.23904},
+  year={2026}
-      archivePrefix={arXiv},
-      primaryClass={cs.AI},
-      url={https://arxiv.org/abs/2605.23904}
 }
 ```

data/README.md

@@ -138,6 +138,20 @@ ALFWorld:
 `searchqa_id_split/` is an ID-only manifest. Each released `id` exactly matches
 the `key` field in `lucadiliello/searchqa`.
 
+To materialize the runnable SearchQA split used by
+`configs/searchqa/default.yaml`, install the optional dependency and run:
+
+```bash
+python -m pip install 'skillopt[searchqa]'
+python scripts/materialize_searchqa.py
+```
+
+This writes full examples to:
+
+```text
+data/searchqa_split
+```
+
 Materialized examples must include the fields consumed by the SearchQA
 environment, including:
 

docs/guide/new-benchmark.md

@@ -36,7 +36,7 @@ touch skillopt/envs/docfaithful/__init__.py
 
 ## Step 2 — Implement the data loader
 
-`skillopt/envs/docfaithful/loader.py`:
+`skillopt/envs/docfaithful/dataloader.py`:
 
 ```python
 from __future__ import annotations
@@ -161,13 +161,10 @@ Two design points worth flagging:
 ```python
 from __future__ import annotations
 
-import os
-
 from skillopt.datasets.base import BatchSpec
 from skillopt.envs.base import EnvAdapter
-from skillopt.envs.docfaithful.loader import DocFaithfulDataLoader
+from skillopt.envs.docfaithful.dataloader import DocFaithfulDataLoader
 from skillopt.envs.docfaithful.rollout import run_batch
-from skillopt.gradient.reflect import run_minibatch_reflect
 
 
 class DocFaithfulAdapter(EnvAdapter):
@@ -234,7 +231,7 @@ class DocFaithfulAdapter(EnvAdapter):
         )
         return self.build_env_from_batch(batch, **kwargs)
 
-    # ── The two real action methods ─────────────────────────────────────
+    # ── The rollout method (reflect is inherited) ───────────────────────
 
     def rollout(self, env_manager, skill_content: str,
                 out_dir: str, **kwargs) -> list[dict]:
@@ -247,27 +244,9 @@ class DocFaithfulAdapter(EnvAdapter):
             max_completion_tokens=self.max_completion_tokens,
         )
 
-    def reflect(self, results: list[dict], skill_content: str,
+    # reflect() is inherited from EnvAdapter — it delegates to
-                out_dir: str, **kwargs) -> list[dict | None]:
+    # run_minibatch_reflect with your analyst_error_* / analyst_success_*
-        return run_minibatch_reflect(
+    # prompts. Override it only if you need custom reflection logic.
-            results=results,
-            skill_content=skill_content,
-            prediction_dir=kwargs.get(
-                "prediction_dir", os.path.join(out_dir, "predictions")
-            ),
-            patches_dir=kwargs.get(
-                "patches_dir", os.path.join(out_dir, "patches")
-            ),
-            workers=self.analyst_workers,
-            failure_only=self.failure_only,
-            minibatch_size=self.minibatch_size,
-            edit_budget=self.edit_budget,
-            random_seed=kwargs.get("random_seed"),
-            error_system=self.get_error_minibatch_prompt(),
-            success_system=self.get_success_minibatch_prompt(),
-            step_buffer_context=kwargs.get("step_buffer_context", ""),
-            update_mode=getattr(self, "_cfg", {}).get("skill_update_mode", "patch"),
-        )
 
     def get_task_types(self) -> list[str]:
         seen: list[str] = []
@@ -373,9 +352,8 @@ If you get `ValueError: Unknown environment 'docfaithful'. Available: [...]`,
 you forgot Step 5.
 
 If you get `TypeError: Can't instantiate abstract class DocFaithfulAdapter`,
-you forgot to implement one of the five abstract methods on `EnvAdapter`:
+you forgot to implement one of the four abstract methods on `EnvAdapter`:
-`build_train_env`, `build_eval_env`, `rollout`, `reflect`,
+`build_train_env`, `build_eval_env`, `rollout`, `get_task_types`.
-`get_task_types`.
 
 ## Tips
 

docs/guideline.html

@@ -288,6 +288,12 @@
       <a href="#cli">CLI scripts</a>
       <a href="#webui">WebUI</a>
     </div>
+    <div class="group">
+      <div class="glabel"><span class="num">9</span> SkillOpt-Sleep</div>
+      <a href="#sleep">Deployment companion</a>
+      <a href="#sleep-plugins">Plugins (3 agents)</a>
+      <a href="#sleep-replay">Experience replay (opt-in)</a>
+    </div>
   </nav>
 
   <!-- ───────────── MAIN CONTENT ───────────── -->
@@ -917,6 +923,57 @@ PY</code></pre>
           <tr><td><code>--share</code></td><td class="def">off</td><td>Create a public Gradio share link.</td></tr>
         </tbody>
       </table></div>
+    </section>
+
+    <section id="sleep">
+      <h2>9.1 SkillOpt-Sleep — the deployment-time companion (preview) <a class="anchor" href="#sleep">#</a></h2>
+      <p><strong>SkillOpt-Sleep</strong> applies SkillOpt's discipline to your own daily usage. It gives a
+      local coding agent a nightly <em>sleep cycle</em> that reviews your past sessions, replays your
+      recurring tasks on your own API budget, and consolidates what it learns into <strong>validated</strong>
+      long-term memory and skills — behind a held-out gate, staged for your review. The agent gets better
+      the more you use it, with no weight training and zero inference-time overhead. It is an early
+      <strong>preview</strong> we are actively iterating on; interfaces and defaults may change.</p>
+      <p>One "night":</p>
+<pre><code>harvest Claude Code / Codex transcripts &rarr; mine recurring tasks &rarr; replay offline
+   &rarr; consolidate (reflect &rarr; bounded edit &rarr; GATE on real held-out tasks)
+   &rarr; stage proposal &rarr; (you) adopt</code></pre>
+      <p>The engine lives in the top-level <code>skillopt_sleep/</code> package with <strong>zero dependency</strong>
+      on the paper's <code>skillopt/</code> experiment code (the validation gate is vendored). Deterministic
+      proof, no API key required:
+      <code>python -m skillopt_sleep.experiments.run_experiment --persona researcher --assert-improves</code>.</p>
+
+      <h2 id="sleep-plugins">9.2 Plugins (three agents) <a class="anchor" href="#sleep-plugins">#</a></h2>
+      <p>One engine, thin per-agent shells (see <a href="https://github.com/microsoft/SkillOpt/tree/main/plugins"><code>plugins/</code></a>):</p>
+      <div class="table-wrap"><table>
+        <thead><tr><th>Platform</th><th>Folder</th><th>Install</th></tr></thead>
+        <tbody>
+          <tr><td>Claude Code</td><td><code>plugins/claude-code</code></td><td><code>/plugin marketplace add ./plugins/claude-code</code> &rarr; <code>/skillopt-sleep</code></td></tr>
+          <tr><td>Codex</td><td><code>plugins/codex</code></td><td><code>bash plugins/codex/install.sh</code> &rarr; <code>skillopt-sleep</code> skill</td></tr>
+          <tr><td>Copilot</td><td><code>plugins/copilot</code></td><td>register <code>plugins/copilot/mcp_server.py</code> as an MCP server</td></tr>
+        </tbody>
+      </table></div>
+      <p>Transcript source and replay backend are separate knobs: <code>--source claude</code> for Claude Code
+      transcripts, <code>--source codex</code> for Codex Desktop archived sessions under
+      <code>~/.codex/archived_sessions</code>, and <code>--backend codex</code> only when you want the
+      replay/optimizer to spend Codex budget.</p>
+
+      <h2 id="sleep-replay">9.3 Experience replay &amp; dream rollouts (opt-in) <a class="anchor" href="#sleep-replay">#</a></h2>
+      <p>Two consolidation mechanisms, both default <strong>off</strong> (so behavior is unchanged unless
+      enabled). They strengthen the nightly update when your tasks have a clean correctness signal; the
+      validation gate still governs what ships.</p>
+      <div class="table-wrap"><table>
+        <thead><tr><th>Config knob</th><th>Default</th><th>Effect</th></tr></thead>
+        <tbody>
+          <tr><td><code>dream_rollouts</code></td><td class="def">1</td><td>Run each task K times and learn from the good-vs-bad contrast (contrastive reflection).</td></tr>
+          <tr><td><code>recall_k</code></td><td class="def">0</td><td>Associative recall — pull the K most-similar past tasks (from a persisted archive) into tonight's dream.</td></tr>
+          <tr><td><code>dream_factor</code></td><td class="def">0</td><td>Add N lightweight synthetic variants of each task.</td></tr>
+        </tbody>
+      </table></div>
+      <p>On a clean-signal benchmark the gain scales with recall depth (deployment protocol: 5 nights &times;
+      10 new real tasks/night, full held-out test, GPT-5.5, gated): <code>recall_k=10</code> &rarr; +3.1 pts,
+      <code>recall_k=20</code> &rarr; +4.5 pts, full-history replay reference &rarr; +5.6 pts; a second benchmark
+      (SpreadsheetBench, GPT-5.4-nano, gate-free) gives +3.6 pts. On saturated or noisy tasks the effect is
+      flat within run-to-run noise (&plusmn;1&ndash;2 pts). Keep the gate on; it bounds the downside.</p>
 
       <div class="footer-note">
         SkillOpt — Executive Strategy for Self-Evolving Agent Skills ·

docs/sleep/CONTROLLABLE_DREAMING.md

@@ -1,117 +0,0 @@
-# SkillOpt-Sleep — controllable dreaming architecture
-
-The sleep engine is no longer a single fixed pipeline. It is a controllable
-offline "dream / imagination" loop the user steers. This documents the knobs
-added in the four-stage refactor and how they map to the user's design.
-
-## The mental model
-
-> Sleep = an offline imagination rollout. Re-run the user's real
-> tasks (and dream-augmented variants) many times, look at what went well vs
-> badly, distil durable rules, and keep only what survives a real-task check —
-> unless the user opts out of that check.
-
-## 1. Data splits — train (dream) / val (real) / test (real)
-
-The anti-overfitting foundation:
-
-| Split | Source | Role |
-|---|---|---|
-| **train** | real tasks **+ dream-augmented** variants | drives reflection (the imagination pool — over-dreaming is fine) |
-| **val** | **real only**, disjoint from test | gates updates (prevents overfitting) |
-| **test** | **real only**, disjoint from val | the final held-out measure, kept close to real usage |
-
-Hard guarantee (unit-tested): a task with `origin='dream'` **never** lands in
-val or test. `assign_splits(val_fraction, test_fraction)` does the deterministic
-3-way split; gbrain's own held-out maps to our `test`.
-
-## 2. The validation gate is optional
-
-`--gate on` (default): an edit is accepted only if it strictly improves the
-**val** score — the SkillOpt discipline that blocks regressions and reward
-hacking.
-
-`--gate off`: greedy. Edits are kept without the hard val-improvement
-requirement (the user decides they don't want hard filtering), but val/test
-movement is still reported (`greedy_improved` / `greedy_regressed` /
-`greedy_flat`) so nothing is hidden.
-
-## 3. Slow-update — long-term memory, gate-independent
-
-Even with the gate off, the engine runs a **slow-update** at the end of the
-nights: it compares behaviour under the first-night vs final skill across the
-val tasks and distils durable longitudinal guidance into a **protected field**
-(`<!-- SLOW_UPDATE_START --> … <!-- SLOW_UPDATE_END -->`, the same markers as
-the main SkillOpt repo). Step-level edits never touch this field. This is the
-"short-term experience → long-term memory" consolidation; turning the gate off
-does not cost you long-term memory.
-
-## 4. Budget — the user picks the spend
-
-`--budget-tokens N` / `--budget-minutes M`: the engine auto-plans depth
-(`nights × rollouts_per_task`) to fit the budget (`plan_depth`). Stops cleanly
-when exhausted and logs what it skipped — no silent truncation. The whole thing
-is offline imagination on the user's own quota.
-
-## 5. Multi-rollout contrastive reflection — the imagination core
-
-`--rollouts-k K` (K>1): each train task is rolled out K times. The optimizer is
-shown the **high-scoring vs low-scoring** attempts of the same task and asked
-what the good ones did that the bad ones didn't, distilling a general rule. This
-is a far stronger signal than a single failure, and it is exactly the user's
-"run it many times, learn from the contrast" idea. Tasks with the highest score
-*spread* (some passed, some failed) are the most informative and are prioritised.
-
-## 6. Multi-objective reward — accuracy ↑, tokens ↓, latency ↓
-
-Every rollout records its `tokens` and `latency_ms`.
-`multi_objective_reward(w_acc, w_tokens, w_latency)` is a weighted reward so a
-skill can be optimised to be **cheaper and faster**, not only more accurate
-(cost terms normalised against a reference; default weights = accuracy-only, so
-existing behaviour is unchanged). This turns "gets better the more you use it"
-into "more accurate, cheaper, and faster the more you use it".
-
-## 7. User preferences as a prior
-
-`--preferences "<free text>"`: injected into the optimizer's reflect prompt as a
-prior (set on the optimizer model for dual backends), so the user's stated
-preferences steer what rules get written.
-
-## How the knobs compose (one command)
-
-```bash
-python -m skillopt.sleep.experiments.run_gbrain \
-  --optimizer-backend claude --optimizer-model sonnet \   # strong optimizer
-  --target-backend claude --target-model haiku \          # cheap target (transfer)
-  --seeds thorough-analyst \
-  --gate on \                                              # or off for greedy
-  --rollouts-k 2 \                                         # contrastive imagination
-  --budget-tokens 60000 \                                  # auto-plan depth
-  --preferences "Prefer concise, British English." \       # prior
-  --nights 3
-```
-
-All of this is exercised by the deterministic test suite (29 tests) and
-validated on real Claude + Codex (see `real_api_results.md` / `FINAL_REPORT.md`).
-
-## Real cross-validation of the new features (Claude ⟷ Codex)
-
-Three live runs exercised the new code paths on both runtimes (raw logs under
-`docs/sleep/raw/crosscheck_*.txt`):
-
-| # | Config | What it proves | Result |
-|---|---|---|---|
-| **A** | Claude Sonnet→Haiku, **gate=off**, **rollouts_k=2** | greedy mode + multi-rollout + 3-way split (val & test both reported) | brief-writer **test 0→1.00**, action `greedy_improved`, val=1.0 test=1.0 |
-| **B** | **Codex**, gate=on, **rollouts_k=2** | new paths on the other runtime | brief-writer **test 0→1.00**, 2-night `accept_new_best`, val+test reported |
-| **C** | Claude Sonnet→Haiku, thorough-analyst, 3 nights | **slow-update** long-term memory fires | test 0→0.33 (val gate holds nights 2–3) and the slow-update distilled a durable meta-rule |
-
-The slow-update guidance C produced is the kind of cross-night lesson the field
-is for — note it is general, not task-specific:
-
-> *"On character-constrained tasks (≤1200 chars), plan structure before writing:
-> allocate space per point explicitly and cut until the outline fits, then fill —
-> never draft freely and trim after."*
-
-Takeaways confirmed live: the **gate-off greedy path**, the **3-way val/test
-split**, **multi-rollout** on both runtimes, and the **gate-independent
-slow-update** all work with real models on both Claude and Codex.

docs/sleep/FINAL_REPORT.md

@@ -1,160 +0,0 @@
-# SkillOpt-Sleep — final validation report
-
-> **What this is:** the consolidated, presented results for the SkillOpt-Sleep
-> Claude Code plugin — a tool that lets a local agent improve itself overnight by
-> reviewing past sessions, replaying tasks, and consolidating validated memory +
-> skills behind a held-out gate. Every real-model result here was run on **both
-> Claude and Codex**, including the honest failures and the bugs they exposed.
-
-**Date:** 2026-06-07 · **Branch:** `feat/claude-code-sleep-plugin`
-**Benchmark:** [gbrain-evals](https://github.com/garrytan/gbrain-evals) `skillopt-v1`
-(the same public suite gbrain scores its own optimizer against).
-**Protocol:** a deliberately deficient skill → 1–2 offline "nights" (replay →
-reflect → bounded **gated** edit) → score the **held-out** task set (never
-optimized against). Held-out scoring uses a local rule judge — the optimizer
-never grades itself.
-
----
-
-## 1. Headline — clean, all green (full gbrain parity)
-
-**Strong optimizer (Claude Sonnet 4.6) → weak target (Claude Haiku 4.5)**, fully
-isolated calls, 3 held-out tasks/seed. All **4** gbrain `skillopt-v1` seeds —
-matching gbrain's own scorecard coverage:
-
-| Optimizer → Target | Seed | Flaw | Held-out before → after | Nights |
-|---|---|---|---|---|
-| Sonnet → Haiku | brief-writer | missing structure | **0.00 → 1.00** | 1 |
-| Sonnet → Haiku | advisor | no verdict | **0.00 → 1.00** | 1 |
-| Sonnet → Haiku | thorough-analyst | no length discipline | **0.00 → 1.00** | 2 |
-| Sonnet → Haiku | quick-answerer | never uses tools | **0.00 → 1.00** | 1 |
-| Codex → Codex (gpt-5.5) | brief-writer | missing structure | **0.00 → 1.00** | 2 |
-| Codex → Codex (gpt-5.5) | advisor | no verdict | **0.00 → 1.00** | 2 |
-
-**4/4 Claude seeds reach a perfect held-out score** (gbrain's headline is the same
-4/4 0→1.00), plus Codex on the text seeds. Every change is gated and staged.
-
-The `quick-answerer` seed is judged by **real tool use** (`tool_called: search`):
-the deficient skill says *"never look anything up — answer from memory"*; the
-optimizer wrote an OVERRIDE rule, and the Haiku target **genuinely invoked a
-`./search` shell tool** (detected from the tool's own log, not self-reported) →
-held-out 1.00. The thorough-analyst run shows textbook **2-night convergence**
-(0.33 → 1.00).
-
----
-
-## 2. The finding that matters most: the optimizer model is decisive
-
-This is the direct answer to "let me specify the optimizer and target separately,
-and watch the skill." It matters a lot:
-
-| Optimizer | Target | brief-writer | advisor | thorough-analyst |
-|---|---|---|---|---|
-| **Haiku** (weak) | Haiku | 1.00 *or* 0.00 (flaky) | 1.00 | 0.33 |
-| **Sonnet** (strong) | Haiku | **1.00** | **1.00** | **1.00** |
-
-A weak self-optimizing model (Haiku proposing its own edits) is **unreliable** —
-it intermittently emits non-JSON and wastes a night, so the same seed scores 1.00
-on one run and 0.00 on another. A **strong optimizer** (Sonnet) reliably produces
-clean, concrete edit rules and lifts every seed to 1.00. This is exactly the
-SkillOpt design (strong optimizer, frozen target) and the reason the
-optimizer/target split is a first-class feature here.
-
-**Practical guidance baked into the plugin:** default to a strong optimizer; the
-sweep's `direct` plan now uses Sonnet→Haiku.
-
----
-
-## 3. Two real bugs we found by running against live models
-
-Per gbrain's own lesson ("the bugs that matter only show up when the whole thing
-actually runs"), the first live runs surfaced two real defects. Both are fixed.
-
-1. **Ambient-context leak (Claude).** `claude -p` was injecting the user's
-   *global* skills + project `CLAUDE.md` into every optimizer/target call — one
-   reflect call literally returned a 21 KB list of the machine's installed skills
-   instead of JSON edits, so the night produced no edits and the gate rejected.
-   Some early Claude "successes" were partly leak-assisted. **Fix:** run isolated
-   — `--bare --disable-slash-commands --disallowedTools '*'
-   --exclude-dynamic-system-prompt-sections`, clean temp cwd. (Codex was never
-   affected; the real `@openai/codex` binary runs in its own clean context.)
-
-2. **Wasted nights on transient non-JSON.** A single malformed reply zeroed a
-   night. **Fix:** `reflect()` retries once with a firmer "JSON only" instruction.
-
-We report these because a tool people build on has to be honest about where it was
-weak and what changed.
-
----
-
-## 4. Cross-model transfer (the price-difference value prop)
-
-> *Optimize cheap overnight, deploy anywhere.* A skill is just text, so a good
-> rewrite should help a model it was never optimized on.
-
-Optimize on SOURCE, **freeze** the learned skill, evaluate held-out on TARGET with
-no further optimization. All four pairs are positive — including **across
-runtimes** (Codex ↔ Claude):
-
-| Source (optimizer) | Target (deploy) | Seed | Target baseline → transferred | Gain |
-|---|---|---|---|---|
-| Claude Haiku (cheap) | Claude Sonnet (expensive) | brief-writer | 0.00 → **1.00** | +1.00 |
-| Claude Sonnet | Claude Haiku | brief-writer | 0.00 → **1.00** | +1.00 |
-| **Codex** | **Claude Haiku** | brief-writer | 0.00 → **1.00** | +1.00 |
-| **Claude Haiku** | **Codex** | brief-writer | 0.00 → **1.00** | +1.00 |
-
-**4/4 transfers positive.** A skill optimized on a cheap model deploys for free on
-an expensive one, and skills move between Codex and Claude — the Sleep-setting
-analogue of SkillOpt's cross-model and cross-harness transfer tables. This is the
-quantified answer to "optimize cheap overnight, deploy anywhere."
-
-Full machine-generated scorecard: [`benchmark_report.md`](benchmark_report.md)
-(source data `sweep.jsonl`).
-
----
-
-## 5. Reproduce everything
-
-```bash
-git clone https://github.com/garrytan/gbrain-evals /tmp/gbrain-evals
-cd <repo>/SkillOpt-sleep
-
-# the clean headline result (strong optimizer -> weak target)
-python3.12 -m skillopt.sleep.experiments.run_gbrain \
-  --optimizer-backend claude --optimizer-model sonnet \
-  --target-backend claude --target-model haiku \
-  --seeds brief-writer,advisor,thorough-analyst \
-  --data-root /tmp/gbrain-evals/eval/data/skillopt-v1 --nights 2 --limit-replay 3 --limit-holdout 3
-
-# Codex self-optimized
-python3.12 -m skillopt.sleep.experiments.run_gbrain --backend codex --seeds brief-writer \
-  --data-root /tmp/gbrain-evals/eval/data/skillopt-v1 --nights 2 --limit-replay 3 --limit-holdout 3
-
-# cross-model transfer
-python3.12 -m skillopt.sleep.experiments.run_transfer \
-  --source-backend claude --source-model haiku --target-backend claude --target-model sonnet \
-  --seeds brief-writer
-
-# the whole sweep + report
-python3.12 -m skillopt.sleep.experiments.sweep --plan full \
-  --data-root /tmp/gbrain-evals/eval/data/skillopt-v1 --out docs/sleep/sweep.jsonl
-python3.12 -m skillopt.sleep.experiments.report --in docs/sleep/sweep.jsonl --out docs/sleep/benchmark_report.md
-
-# deterministic, no API (CI anchor)
-python3.12 -m skillopt.sleep.experiments.run_experiment --persona researcher --assert-improves
-```
-
-Raw run logs are under `docs/sleep/raw/`.
-
----
-
-## 6. Honest limitations
-
-- **Latency:** each CLI call is ~14–15 s startup-dominated, so runs are capped at
-  a few tasks/nights. Fine for nightly cron; we note it plainly.
-- **Weak optimizers are flaky:** use a strong optimizer model (§2).
-- **Tool-use seed covered honestly:** `quick-answerer` (`tool_called: search`)
-  runs a real tool loop — a callable `./search` shim, detected from its log.
-  Deeper multi-tool / multi-turn workflows are future work.
-- **Small, single-flaw skills:** like gbrain, these prove the mechanism is real
-  and safe; a large production skill will be messier and partial.

docs/sleep/PR_DRAFT.md

@@ -1,53 +0,0 @@
-TITLE:
-Add SkillOpt-Sleep: nightly offline self-evolution plugins (Claude Code, Codex, Copilot)
-
-BODY:
-## Summary
-
-Adds **SkillOpt-Sleep** — a nightly offline "sleep cycle" that gives a local
-coding agent the deployment-time analogue of training: it reviews past sessions,
-replays recurring tasks on the user's own API budget, and consolidates what it
-learns into **validated** long-term memory and skills behind a held-out gate.
-Synthesizes SkillOpt (validation-gated bounded text edits), Claude Dreams
-(offline consolidation; review-then-adopt), and the agent-sleep idea
-(short-term experience -> long-term competence).
-
-Shipped as plugins for **three agents**, one engine + three thin shells:
-
-- **Claude Code** — `.claude-plugin` + `/sleep` command + skill + hooks
-- **Codex** — `~/.codex/prompts/sleep.md` + `~/.agents/skills` + `install.sh`
-- **Copilot** — a stdlib-only MCP server exposing `sleep_*` tools
-
-## Design notes
-
-- **Open-source tool, decoupled from the research code.** The engine lives in the
-  new top-level `skillopt_sleep/` package with **zero dependency** on the paper's
-  `skillopt/` experiment package (the validation gate is vendored).
-- Controllable: optional gate (`--gate on|off`), train(dream)/val(real)/test(real)
-  splits, slow-update long-term memory, token/time budget, multi-rollout
-  contrastive reflection, multi-objective reward (accuracy/tokens/latency), user
-  preferences, and separate optimizer/target models.
-
-## Validation (real models)
-
-On the public [gbrain-evals](https://github.com/garrytan/gbrain-evals)
-`skillopt-v1` benchmark, deficient skills go **0.00 -> 1.00** on held-out sets
-with **both Claude and Codex** (all 4 seeds, including a real tool-use loop);
-cross-model transfer is positive; the gate blocks regressions. Independently
-load-tested on a fresh non-benchmark persona ("SQL must always include LIMIT"):
-held-out test **0.00 -> 1.00** on both backends. See `docs/sleep/FINAL_REPORT.md`
-and `docs/sleep/plugin_load_test.md`.
-
-## Tests
-
-- 29 deterministic unit tests (`tests/test_sleep_engine.py`), no API key required.
-- `python -m skillopt_sleep.experiments.run_experiment --persona researcher --assert-improves`
-  proves held-out lift and that the gate blocks a harmful edit.
-
-## Test plan
-
-- [ ] `python -m unittest tests.test_sleep_engine` (29 pass)
-- [ ] `python -m skillopt_sleep.experiments.run_experiment --persona researcher --assert-improves`
-- [ ] Claude Code: `/plugin marketplace add ./plugins/claude-code` -> `/sleep status`
-- [ ] Codex: `bash plugins/codex/install.sh`
-- [ ] Copilot: MCP server `tools/list` returns the `sleep_*` tools

docs/sleep/README.md

@@ -0,0 +1,110 @@
+# SkillOpt-Sleep 😴 — deployment-time companion (preview)
+
+**SkillOpt-Sleep** applies SkillOpt's discipline to your *own daily usage*. It gives a
+local coding agent a nightly **sleep cycle** that reviews your past sessions, replays
+your recurring tasks on your own API budget, and consolidates what it learns into
+**validated** long-term memory and skills — behind a held-out gate, staged for your
+review. The agent gets better the more you use it, with **no weight training** and
+**zero inference-time overhead**.
+
+> **Preview.** This is an early preview we are actively iterating on; interfaces and
+> defaults may change. The engine lives in the top-level [`skillopt_sleep/`](../../skillopt_sleep)
+> package with **zero dependency** on the paper's `skillopt/` code (the validation gate
+> is vendored).
+
+## How it works
+
+One "night":
+
+```
+harvest Claude Code / Codex transcripts → mine recurring tasks → replay offline
+   → consolidate (reflect → bounded edit → GATE on real held-out tasks)
+   → stage proposal → (you) adopt
+```
+
+It synthesizes **SkillOpt** (validation-gated bounded text edits), **Claude Dreams**
+(offline consolidation; review-then-adopt), and the **agent-sleep** idea (short-term
+experience → long-term competence).
+
+## How to use it
+
+### Quickest path: the `skillopt-sleep` CLI (pip)
+
+```bash
+pip install skillopt        # installs the engine + the `skillopt-sleep` command
+skillopt-sleep dry-run      # harvest + mine + replay, report only (changes nothing)
+skillopt-sleep run          # a full nightly cycle; the proposal is staged for review
+skillopt-sleep status       # show state + the latest staged proposal
+skillopt-sleep adopt        # apply the latest staged proposal
+skillopt-sleep schedule     # install a nightly cron entry for this project
+```
+
+The per-agent plugin shells below (Claude Code / Codex / Copilot) still come from the
+repo; the CLI above is the standalone, pip-only way to run a cycle.
+
+One engine, thin per-agent shells (see [`plugins/`](../../plugins)):
+
+| Platform | Folder | Install |
+|---|---|---|
+| **Claude Code** | [`plugins/claude-code`](../../plugins/claude-code) | `/plugin marketplace add ./plugins/claude-code` → `/skillopt-sleep` |
+| **Codex** | [`plugins/codex`](../../plugins/codex) | `bash plugins/codex/install.sh` → `skillopt-sleep` skill |
+| **Copilot** | [`plugins/copilot`](../../plugins/copilot) | register `plugins/copilot/mcp_server.py` as an MCP server |
+
+Deterministic proof (no API key):
+`python -m skillopt_sleep.experiments.run_experiment --persona researcher --assert-improves`.
+
+### Opt-in: experience replay & dream rollouts
+
+Two consolidation mechanisms, both default **off** (behavior is unchanged unless you
+enable them). They strengthen the nightly update when your tasks have a clean
+correctness signal; the validation gate still governs what ships.
+
+| Config knob | Default | Effect |
+|---|---|---|
+| `dream_rollouts` | `1` | Run each task K times → learn from the good-vs-bad contrast (contrastive reflection). |
+| `recall_k` | `0` | Associative recall — pull the K most-similar past tasks (from a persisted archive) into tonight's dream. |
+| `dream_factor` | `0` | Add N lightweight synthetic variants of each task. |
+
+## Results
+
+> 📊 **More results & analysis — the gate-safety stress test, experience-replay
+> scaling, and the dream-diversity ablation — are in
+> [`docs/sleep/RESULTS.md`](RESULTS.md).** The highlights:
+
+**Protocol (identical for every row below).** 5 nights × 10 new real "today" tasks
+per night; the full held-out **test** split is scored before night 1 (baseline) and
+after night 5 (after); optimizer = GPT-5.5; single seed (42); run through the exact
+shipped engine (`skillopt_sleep.dream.dream_consolidate`). Numbers are absolute
+held-out accuracy; **Δ** = `after − baseline` in percentage points.
+
+**(a) End-to-end on real agents — [gbrain-evals](https://github.com/garrytan/gbrain-evals) `skillopt-v1`.**
+Deficient seed skills go **0.00 → 1.00** on the held-out set with **both Claude Code
+and Codex** as the target agent (all 4 seeds, including a real tool-use loop).
+
+**(b) Experience replay scales the gain — SearchQA** (1,400-item held-out test,
+SQuAD exact-match; target = GPT-5.5; **validation-gated**):
+
+| Replay config (`dream_rollouts=5`) | Baseline → After | Δ (pts) |
+|---|---|---|
+| `recall_k=10` | 0.802 → 0.834 | +3.1 |
+| `recall_k=20` | 0.803 → 0.848 | **+4.5** |
+| full-history replay *(reference, not a shipping default)* | 0.796 → 0.851 | +5.6 |
+| `recall_k=10`, `dream_rollouts=8` *(more dreaming, same recall)* | 0.798 → 0.835 | +3.7 |
+
+The gain rises monotonically with how much relevant past experience is recalled. The
+same SearchQA cell **without** the gate (`recall_k=10`) is 0.808 → 0.839 (+3.1).
+
+**(c) Second benchmark — SpreadsheetBench** (280-item held-out test; the agent's
+generated openpyxl code is executed and compared cell-by-cell to a golden workbook;
+target = GPT-5.4-nano; gate-free + the output-contract guardrail): 0.279 → 0.314 (**+3.6**).
+
+**(d) Honest scope.** These gains hold where tasks recur and have a checkable
+correctness signal. On saturated or noisy benchmarks (e.g. a strong model already
+near ceiling) the effect is **flat within run-to-run noise** — single-seed baseline
+variance here is ±1–2 pts, so treat sub-~1.5 pt differences as noise. The validation
+gate keeps the worst case bounded; keep it **on** by default.
+
+## Learn more
+
+Full reference (pipeline, the three plugins, the experience-replay knobs) is in the
+**[Documentation & Reproduction Guide](https://microsoft.github.io/SkillOpt/docs/guideline.html#sleep)**.

docs/sleep/RESULTS.md

@@ -0,0 +1,185 @@
+# SkillOpt-Sleep — results & analysis
+
+This is the evidence behind SkillOpt-Sleep: does a nightly, offline sleep cycle
+actually make a *deployed* agent better, and is it safe to run unattended? We
+answer with a controlled deployment-scale study — the same protocol the plugin
+runs in production, scored on full held-out test sets.
+
+## Setup
+
+**Protocol (identical for every cell unless stated).** 5 nights; each night adds
+**10 new real "today" tasks**; the skill carries over and is refined night to
+night. The full held-out **test** split is scored before night 1 (*baseline*) and
+after night 5 (*after*); **Δ = after − baseline** in percentage points. Optimizer
+model = **GPT-5.5**; single seed (42); every number is produced by the exact
+shipped engine `skillopt_sleep.dream.dream_consolidate` (the experiment harness and
+the plugin cycle call the same function).
+
+**Benchmarks** (real evaluators, not format heuristics):
+
+| Benchmark | Held-out test | Scoring |
+|---|---|---|
+| SearchQA | 1,400 items | SQuAD exact-match vs gold |
+| LiveMathematicianBench | 124 items | multiple-choice label (choices shuffled per item) |
+| SpreadsheetBench | 280 items | the agent's generated openpyxl code is **executed**, output workbook compared cell-by-cell to a golden file |
+
+**Targets:** GPT-5.5, GPT-5.4-mini, GPT-5.4-nano. **Modes:** validation-gated
+(default) and gate-free.
+
+---
+
+## 1. The headline — the validation gate is what makes nightly self-evolution *safe*
+
+Self-evolution is easy to build and easy to ruin: an optimizer that accepts its
+own "lessons" unconditionally can adopt a plausible-but-wrong rule and an obedient
+model will follow it off a cliff. We reproduced exactly that failure, then showed
+the gate prevents it.
+
+Stress case — **GPT-5.4-nano on SearchQA**, weak model on a single-sample (degraded)
+reflection signal, same nights, same candidate edits, gate **off** vs **on**:
+
+| | Night 0 → Night 5 | Δ |
+|---|---|---|
+| **no gate** | 0.554 → **0.026** | **−52.8** |
+| **with gate (default)** | 0.570 → 0.570 | 0.0 |
+
+Ungated, the optimizer learned "answer with the document-title string, verbatim";
+the model complied and accuracy collapsed night after night
+(0.554 → 0.490 → 0.325 → 0.031 → 0.034 → 0.026). The gated twin **rejected every one
+of those edits** and never lost a point. This single experiment is the core
+argument for SkillOpt-Sleep's design, and why the gate ships **on by default**.
+
+---
+
+## 2. Cross-model scaling — bigger gains where there's headroom
+
+The same protocol on a weaker target model (**GPT-5.4-nano**, optimizer = GPT-5.5)
+produces substantially larger gains — because the weaker model has more room to
+learn. This is the realistic "cheap deployed agent, strong overnight optimizer"
+scenario:
+
+| Config (SearchQA, nano, gated) | Baseline → After | Δ | Night-by-night |
+|---|---|---|---|
+| **cumulative replay, nights=5** | 0.560 → **0.679** | **+11.9** | 0.560 → 0.626 → 0.665 → 0.665 → 0.665 → 0.679 |
+| recall_k=20, nights=5 | 0.566 → 0.681 | +11.5 | 0.566 → 0.659 → 0.685 → 0.685 → 0.681 → 0.681 |
+| cumulative, nights=8 | 0.562 → 0.657 | +9.5 | saturates after night 5 |
+
+Both replay strategies (cumulative and recall) agree within 0.4 pt — the gain is
+robust across configurations.
+
+**Compared to GPT-5.5 on the same benchmark (SearchQA, gated):**
+
+| Target model | Best Δ | Baseline | Headroom |
+|---|---|---|---|
+| GPT-5.4-nano | **+11.9** | 0.560 | 44 pt |
+| GPT-5.5 | +6.0 | 0.798 | 20 pt |
+
+The story: **SkillOpt-Sleep helps most where there's the most to learn** — weaker
+deployed models benefit ~2× as much from the same nightly optimization. This is
+also the economical deployment pattern (cheap inference model + one strong
+overnight optimizer call).
+
+---
+
+## 3. Experience replay turns a one-time bump into a climb
+
+The plugin's two opt-in knobs (`recall_k`, `dream_rollouts`) are what produce the
+gains. On **SearchQA, GPT-5.5, gated** — the gain rises monotonically with how
+much relevant past experience is recalled:
+
+| Replay (`dream_rollouts=5`) | Baseline → After | Δ |
+|---|---|---|
+| `recall_k=10` | 0.802 → 0.834 | +3.1 |
+| `recall_k=20` | 0.803 → 0.848 | **+4.5** |
+| full-history (reference, not a default) | 0.796 → 0.851 | +5.6 |
+
+And the curve genuinely **climbs across nights** rather than jumping once and
+plateauing — full-history replay, gated, night by night:
+
+```
+0.798 → 0.814 → 0.854 → 0.854 → 0.854 → 0.858
+```
+
+The gate accepts a new, better skill as late as **night 5** (0.854 → 0.858).
+Replay-policy ablation (SearchQA, GPT-5.5):
+
+| Replay policy | Gate-free Δ | Gated Δ |
+|---|---|---|
+| none (tonight's tasks only) | +3.9 | +2.0 |
+| **recall k=10 (shipped default-able)** | +5.1 | +4.4 |
+| cumulative (full history) | +4.8 | +6.0 |
+
+Recall captures most of cumulative's benefit at a fraction of the per-night cost.
+
+---
+
+## 4. Default hyperparameters are the sweet spot
+
+We swept `dream_factor`, `rollouts`, `per_night`, and `nights` on the nano cell
+(SearchQA, gated) to verify the shipped defaults are well-tuned:
+
+| Variant | Δ | vs default (+11.9) |
+|---|---|---|
+| dream_factor=4 (default 2) | +8.8 | −3.1 |
+| rollouts=10 (default 5) | +9.5 | −2.4 |
+| per_night=15 (default 10) | +2.7 | −9.2 |
+| nights=8 (default 5) | +9.5 | −2.4 |
+
+Every direction away from the default hurts. This means users get the best result
+**out of the box** without tuning — the recipe is robust by design.
+
+---
+
+## 5. Why these gains exist — the dream-diversity fix (and a rigor note)
+
+Reflection learns from the **contrast** between good and bad rollouts of the same
+task, which requires the K dream rollouts to be *independent samples*. An early
+version of the engine collapsed them to one cached sample, so contrastive
+reflection never fired. Fixing that, then adding recall, is what produces the
+gains in Sections 1–2. Measured across an 18-cell deployment sweep (3 benchmarks ×
+3 targets × 2 modes), under three engine configurations:
+
+| Engine configuration | mean Δ | worst-cell Δ | cells > +0.5 | cells < −0.5 |
+|---|---|---|---|---|
+| single-sample reflection (degraded) | −2.66 | **−52.8** | 7 / 18 | 5 / 18 |
+| diverse rollouts (K=5), no recall | +0.24 | −4.0 | 6 / 18 | 7 / 18 |
+| **diverse rollouts + recall (shipped)** | **+0.53** | **−2.4** | 7 / 18 | 7 / 18 |
+
+The catastrophic −52.8 is removed **at its source** by diverse rollouts: the same
+gate-free nano-SearchQA cell goes 0.554 → **0.586 (+2.7)** with no gate at all once
+the dream is fixed. Recall then lifts the grid mean and tightens the worst case.
+This is **defense in depth, each layer measured**: diverse rollouts propose better
+edits, recall remembers relevant experience, and the gate catches whatever still
+slips through.
+
+---
+
+## 6. End-to-end on real agents
+
+On the public [gbrain-evals](https://github.com/garrytan/gbrain-evals) `skillopt-v1`
+benchmark — designed for exactly this learnable-gap setting — deficient seed skills
+go **0.00 → 1.00** on the held-out set with **both Claude Code and Codex** as the
+target agent (all 4 seeds, including a real tool-use loop), and the two agents
+cross-verify each other's consolidated skills.
+
+---
+
+## 7. Honest scope & limitations
+
+- **Where it helps:** recurring tasks with a checkable correctness signal and real
+  headroom. That is the plugin's actual use case (your repeated daily tasks and
+  house rules the agent keeps missing).
+- **Where it's flat:** saturated tasks on strong models, or noisy tasks with a weak
+  learning signal — within run-to-run noise.
+- **Single seed.** Cells aggregate one seed per config; treat sub-~1.5 pt
+  differences as noise. Spot seed-robustness check on the one flagged cell
+  (nano SearchQA gated): seeds 42/43/44 give −1.9 / +3.6 / +4.7 (3-seed mean
+  **+2.1**), i.e. the tabled −1.9 is a pessimistic draw, not the typical outcome.
+- **Keep the gate on.** It is the difference between bounded downside (−2.4) and a
+  −52.8 collapse. Gate-free mode is for users who cannot hold out a validation set
+  and is additionally protected by the output-contract guardrail.
+
+---
+
+Back to the module overview: [`docs/sleep/README.md`](README.md) ·
+full reference: [Documentation & Reproduction Guide](https://microsoft.github.io/SkillOpt/docs/guideline.html#sleep).

docs/sleep/benchmark_report.md

@@ -1,41 +0,0 @@
-# SkillOpt-Sleep — benchmark report
-
-Auto-generated from `sweep.jsonl`. Benchmark: [gbrain-evals](https://github.com/garrytan/gbrain-evals) `skillopt-v1` (deficient skills, train/held-out split, local rule judge — no judge-API).
-Held-out scores are computed by the harness, not the optimizer.
-
-## Direct improvement (optimize, then deploy)
-
-| Optimizer → Target | Seed | Held-out before | Held-out after | Nights | Tokens |
-|---|---|---|---|---|---|
-| claude:sonnet → claude:haiku | brief-writer | 0.00 | **1.00** | 2 | 6657 |
-| claude:sonnet → claude:haiku | advisor | 0.00 | **1.00** | 2 | 7891 |
-| claude:sonnet → claude:haiku | thorough-analyst | 0.00 | **1.00** | 2 | 17960 |
-| codex:default → codex:default | brief-writer | 0.00 | **1.00** | 2 | 9969 |
-| codex:default → codex:default | advisor | 0.00 | **1.00** | 2 | 6210 |
-| claude:sonnet → claude:haiku | quick-answerer | 0.00 | **1.00** | 2 | 10988 |
-| codex:default → codex:default | quick-answerer | 0.00 | **1.00** | 2 | 7347 |
-
-**7/7 configurations improved on held-out.**
-
-## Cross-model transfer (optimize on SOURCE, deploy frozen on TARGET)
-
-The price-difference story: spend cheap tokens optimizing overnight, then deploy the frozen skill on any model with no further optimization.
-
-| Source (optimizer) | Target (deploy) | Seed | Target baseline | Transferred | Gain |
-|---|---|---|---|---|---|
-| claude:haiku | claude:sonnet | brief-writer | 0.00 | **1.00** | +1.00 |
-| claude:sonnet | claude:haiku | brief-writer | 0.00 | **1.00** | +1.00 |
-| codex:default | claude:haiku | brief-writer | 0.00 | **1.00** | +1.00 |
-| claude:haiku | codex:default | brief-writer | 0.00 | **1.00** | +1.00 |
-
-**4/4 transfers were positive** (frozen skill helped a different model than it was optimized on).
-
-## How to reproduce
-
-```bash
-git clone https://github.com/garrytan/gbrain-evals /tmp/gbrain-evals
-python -m skillopt.sleep.experiments.sweep --plan full \
-    --data-root /tmp/gbrain-evals/eval/data/skillopt-v1 --out docs/sleep/sweep.jsonl
-python -m skillopt.sleep.experiments.report \
-    --in docs/sleep/sweep.jsonl --out docs/sleep/benchmark_report.md
-```

docs/sleep/experiment_results.md

@@ -1,73 +0,0 @@
-# SkillOpt-Sleep — validation experiment results
-
-Generated: 2026-06-07 (autonomous offline session)
-Backend: mock (deterministic, no API). Reproducible via the commands below.
-
-```
-$ python3.12 -m skillopt.sleep.experiments.run_experiment --persona researcher --nights 4 --json
-{
-  "persona": "researcher",
-  "backend": "mock",
-  "nights_run": 1,
-  "baseline_holdout": 0.3333,
-  "after_holdout": 1.0,
-  "lift": 0.6667,
-  "improved": true,
-  "gate_blocks_harmful": true,
-  "final_skill_excerpt": "T -->\n## Learned preferences & procedures\n\n_This block is maintained by SkillOpt-Sleep. Edits here are proposed offline, validated against your past tasks, and adopted only after you approve them. Hand-edits outside this block are never touched._\n\n- Always wrap the final answer in <answer>...</answer> tags.\n- Report arXiv ids in the exact form arXiv:XXXX.XXXXX.\n<!-- SKILLOPT-SLEEP:LEARNED END -->\n",
-  "trace": [
-    {
-      "night": 0,
-      "holdout_score": 0.3333,
-      "action": "baseline",
-      "n_edits": 0
-    },
-    {
-      "night": 1,
-      "holdout_score": 1.0,
-      "action": "accept_new_best",
-      "accepted": true,
-      "n_edits": 2,
-      "edits": [
-        "Always wrap the final answer in <answer>...</answer> tags.",
-        "Report arXiv ids in the exact form arXiv:XXXX.XXXXX."
-      ],
-      "n_rejected": 0
-    }
-  ]
-}
-```
-
-```
-$ python3.12 -m skillopt.sleep.experiments.run_experiment --persona programmer --nights 4 --json
-{
-  "persona": "programmer",
-  "backend": "mock",
-  "nights_run": 1,
-  "baseline_holdout": 0.3194,
-  "after_holdout": 1.0,
-  "lift": 0.6806,
-  "improved": true,
-  "gate_blocks_harmful": true,
-  "final_skill_excerpt": "laude Code sessions.\n\n<!-- SKILLOPT-SLEEP:LEARNED START -->\n## Learned preferences & procedures\n\n_This block is maintained by SkillOpt-Sleep. Edits here are proposed offline, validated against your past tasks, and adopted only after you approve them. Hand-edits outside this block are never touched._\n\n- Write git commit subjects in imperative mood, max 50 chars.\n<!-- SKILLOPT-SLEEP:LEARNED END -->\n",
-  "trace": [
-    {
-      "night": 0,
-      "holdout_score": 0.3194,
-      "action": "baseline",
-      "n_edits": 0
-    },
-    {
-      "night": 1,
-      "holdout_score": 1.0,
-      "action": "accept_new_best",
-      "accepted": true,
-      "n_edits": 1,
-      "edits": [
-        "Write git commit subjects in imperative mood, max 50 chars."
-      ],
-      "n_rejected": 0
-    }
-  ]
-}
-```

docs/sleep/plugin_load_test.md

@@ -1,76 +0,0 @@
-# SkillOpt-Sleep — plugin load-test (fresh examples)
-
-This records an actual end-to-end load-test of all three plugin shells on a
-**brand-new example** (not the gbrain benchmark seeds), run on 2026-06-08.
-
-## The fresh persona
-
-A data analyst whose SQL queries must always include a `LIMIT` clause — built
-from scratch for this test. Two forms were used:
-
-1. **Real transcripts** — crafted Claude Code session JSONL where the analyst
-   asks for SQL, the agent forgets `LIMIT`, and the user complains ("you forgot
-   a LIMIT again", "always cap results"). This exercises the real
-   harvest → mine pipeline.
-2. **Checkable tasks** — the same intent with a rule judge
-   (`regex: (?i)LIMIT\s+100`), so the optimizer can be scored on whether future
-   SQL follows the house rule.
-
-## Results
-
-### Shell plumbing (all three drive the engine)
-
-| Shell | What was run | Result |
-|---|---|---|
-| **Claude Code** (`scripts/sleep.sh`) | `harvest`, full `run`, `adopt` | harvest found 2 sessions → 2 tasks; `run` staged a proposal; `adopt` honored the safety contract (no live change when nothing was accepted) |
-| **Codex** (`install.sh` + shared runner) | `install.sh` into a temp HOME | placed `~/.codex/prompts/sleep.md` and `~/.agents/skills/skillopt-sleep/SKILL.md` correctly |
-| **Copilot** (`mcp_server.py`) | `initialize` → `tools/list` → `tools/call sleep_harvest` | 5 tools listed; `sleep_harvest` returned real engine output (2 sessions → 2 tasks) |
-
-### Genuine improvement (real model, fresh persona)
-
-Optimizer **Claude Sonnet 4.6** → target **Claude Haiku 4.5**, 3-way split
-(5 train / 2 val / 5 test), scored on the held-out **test** queries; and the same
-fresh persona self-optimized on **Codex**:
-
-| Backend | Held-out **test** (fraction of SQL with `LIMIT 100`) before → after |
-|---|---|
-| Claude (Sonnet → Haiku) | **0.00 → 1.00** |
-| Codex | **0.00 → 1.00** |
-
-In one night each optimizer wrote, into the protected learned block, a rule like:
-
-> *"OVERRIDE: Every SQL query you generate MUST include `LIMIT 100` …"* (Claude)
-> *"Hard requirement: every SQL query response must include …"* (Codex)
-
-and the target then applied it to the **unseen** test queries. This is the whole
-claim on a task family the engine had never seen: it learned the user's house
-rule from their failures and generalized it — confirmed on both backends.
-
-## An honest finding from load-testing
-
-The **first** attempt used `val_fraction=0.34, test_fraction=0.34`, which left
-only **1 train task** for an 8-task set — too little signal — so reflect produced
-nothing and the night was a no-op (val already 0.75). Re-balancing the split to a
-real train pool (5 train) fixed it and produced the 0 → 1.00 result above. This
-is exactly the kind of issue that only surfaces when you actually run the thing,
-and it motivates a future guardrail: warn when the train pool is too small for
-the chosen split fractions.
-
-## Reproduce
-
-The checkable persona run (real Claude):
-
-```python
-# see the snippet in docs/sleep/plugin_load_test.md history, or run:
-python -m skillopt_sleep.experiments.run_experiment --persona programmer --assert-improves  # deterministic
-```
-
-Shell checks:
-
-```bash
-# Copilot MCP server
-printf '%s\n' '{"jsonrpc":"2.0","id":1,"method":"tools/list"}' \
-  | SKILLOPT_SLEEP_REPO="$(pwd)" python3 plugins/copilot/mcp_server.py
-# Codex installer (into a throwaway HOME)
-HOME=$(mktemp -d) bash plugins/codex/install.sh
-```

docs/sleep/raw/codex_brief_writer.txt

@@ -1,45 +0,0 @@
-=== gbrain brief-writer CODEX, improved prompt, 2 nights, 3+3 tasks ===
-{
-  "benchmark": "gbrain-evals/skillopt-v1",
-  "backend": "codex",
-  "model": "(default)",
-  "n_seeds": 1,
-  "n_improved": 1,
-  "tokens_used": 9990,
-  "results": [
-    {
-      "seed": "brief-writer",
-      "held_out_before": 0.0,
-      "held_out_after": 1.0,
-      "improved": true,
-      "nights": 2,
-      "trace": [
-        {
-          "night": 0,
-          "held_out_hard": 0.0,
-          "action": "baseline"
-        },
-        {
-          "night": 1,
-          "held_out_hard": 0.0,
-          "action": "accept_new_best",
-          "accepted": true,
-          "edits": [
-            "Every brief must include a clearly labeled section exactly titled `Key Risks`.",
-            "Every brief must include a line beginning `Confidence:` followed by a concise confidence level or rationale."
-          ]
-        },
-        {
-          "night": 2,
-          "held_out_hard": 1.0,
-          "action": "accept_new_best",
-          "accepted": true,
-          "edits": [
-            "- Preserve required sections even when keeping the brief short; shorten the analysis before omitting `## Key Risks` or `Confidence:`."
-          ]
-        }
-      ],
-      "final_skill_tail": "tside this block are never touched._\n\n- Every brief must include a clearly labeled section exactly titled `Key Risks`.\n- Every brief must include a line beginning `Confidence:` followed by a concise confidence level or rationale.\n- Preserve required sections even when keeping the brief short; shorten the analysis before omitting `## Key Risks` or `Confidence:`.\n<!-- SKILLOPT-SLEEP:LEARNED END -->\n"
-    }
-  ]
-}

docs/sleep/raw/crosscheck_A_claude_gateoff_rollouts.txt

@@ -1,38 +0,0 @@
-=== REAL cross-check A: Sonnet->Haiku, gate=OFF, rollouts_k=2, brief-writer (exercises new paths) ===
-{
-  "benchmark": "gbrain-evals/skillopt-v1",
-  "backend": "target=claude/optimizer=claude",
-  "model": "(default)",
-  "n_seeds": 1,
-  "n_improved": 1,
-  "tokens_used": 11271,
-  "results": [
-    {
-      "seed": "brief-writer",
-      "held_out_before": 0.0,
-      "held_out_after": 1.0,
-      "improved": true,
-      "nights": 1,
-      "trace": [
-        {
-          "night": 0,
-          "test_hard": 0.0,
-          "action": "baseline"
-        },
-        {
-          "night": 1,
-          "val_hard": 1.0,
-          "test_hard": 1.0,
-          "action": "greedy_improved",
-          "accepted": true,
-          "edits": [
-            "Every brief MUST include a section with the exact heading '## Key Risks' that lists the primary risks relevant to the recommendation. This section is required in every output regardless of topic.",
-            "Every brief MUST include a 'Confidence:' label (satisfying /[Cc]onfidence\\s*[:=]/) that states the confidence level in the recommendation (e.g., 'Confidence: Medium'). Place it near the answer/recommendation line or at the end of the brief."
-          ]
-        }
-      ],
-      "slow_update": null,
-      "final_skill_tail": "at lists the primary risks relevant to the recommendation. This section is required in every output regardless of topic.\n- Every brief MUST include a 'Confidence:' label (satisfying /[Cc]onfidence\\s*[:=]/) that states the confidence level in the recommendation (e.g., 'Confidence: Medium'). Place it near the answer/recommendation line or at the end of the brief.\n<!-- SKILLOPT-SLEEP:LEARNED END -->\n"
-    }
-  ]
-}

docs/sleep/raw/crosscheck_B_codex_rollouts.txt

@@ -1,48 +0,0 @@
-=== REAL cross-check B: Codex, gate=ON (default), rollouts_k=2, brief-writer ===
-{
-  "benchmark": "gbrain-evals/skillopt-v1",
-  "backend": "codex",
-  "model": "(default)",
-  "n_seeds": 1,
-  "n_improved": 1,
-  "tokens_used": 17251,
-  "results": [
-    {
-      "seed": "brief-writer",
-      "held_out_before": 0.0,
-      "held_out_after": 1.0,
-      "improved": true,
-      "nights": 2,
-      "trace": [
-        {
-          "night": 0,
-          "test_hard": 0.0,
-          "action": "baseline"
-        },
-        {
-          "night": 1,
-          "val_hard": 0.667,
-          "test_hard": 0.333,
-          "action": "accept_new_best",
-          "accepted": true,
-          "edits": [
-            "Every brief must include a section/heading titled exactly 'Key Risks'.",
-            "Every brief must include a confidence line labeled exactly 'Confidence:' so the response matches /[Cc]onfidence\\s*[:=]/."
-          ]
-        },
-        {
-          "night": 2,
-          "val_hard": 1.0,
-          "test_hard": 1.0,
-          "action": "accept_new_best",
-          "accepted": true,
-          "edits": [
-            "OVERRIDE any brevity guidance: every brief must include a standalone Markdown heading line exactly '## Key Risks' to satisfy section_present=Key Risks, even when the brief is very short."
-          ]
-        }
-      ],
-      "slow_update": null,
-      "final_skill_tail": "clude a section/heading titled exactly 'Key Risks'.\n- Every brief must include a confidence line labeled exactly 'Confidence:' so the response matches /[Cc]onfidence\\s*[:=]/.\n- OVERRIDE any brevity guidance: every brief must include a standalone Markdown heading line exactly '## Key Risks' to satisfy section_present=Key Risks, even when the brief is very short.\n<!-- SKILLOPT-SLEEP:LEARNED END -->\n"
-    }
-  ]
-}

docs/sleep/raw/crosscheck_C_claude_slowupdate.txt

@@ -1,54 +0,0 @@
-=== cross-check C: Sonnet->Haiku thorough-analyst (2 nights, slow-update should fire) ===
-{
-  "benchmark": "gbrain-evals/skillopt-v1",
-  "backend": "target=claude/optimizer=claude",
-  "model": "(default)",
-  "n_seeds": 1,
-  "n_improved": 1,
-  "tokens_used": 26010,
-  "results": [
-    {
-      "seed": "thorough-analyst",
-      "held_out_before": 0.0,
-      "held_out_after": 0.333,
-      "improved": true,
-      "nights": 3,
-      "trace": [
-        {
-          "night": 0,
-          "test_hard": 0.0,
-          "action": "baseline"
-        },
-        {
-          "night": 1,
-          "val_hard": 0.667,
-          "test_hard": 0.667,
-          "action": "accept_new_best",
-          "accepted": true,
-          "edits": [
-            "OVERRIDE (supersedes 'be exhaustive and detailed', 'Explore every angle', 'consider many scenarios', and 'Write multiple paragraphs'): the ENTIRE response must be at most 1200 characters long, counting every character including spaces, newlines, and punctuation. This hard character limit takes priority over all instructions to be thorough, exhaustive, or multi-paragraph.",
-            "To stay within 1200 characters while still being useful: lead with the single most critical trade-off, then list 2-3 key considerations as tight bullet points. Omit headers, preamble, and restating the question."
-          ]
-        },
-        {
-          "night": 2,
-          "val_hard": 0.667,
-          "test_hard": 0.667,
-          "action": "reject",
-          "accepted": false,
-          "edits": []
-        },
-        {
-          "night": 3,
-          "val_hard": 0.667,
-          "test_hard": 0.667,
-          "action": "reject",
-          "accepted": false,
-          "edits": []
-        }
-      ],
-      "slow_update": "• On character-constrained tasks (≤1200 chars), plan structure before writing: allocate space per point explicitly and cut until the outline fits, then fill — never draft freely and trim after.\n• Multi-variable business/strategy analyses are high-risk for overrun; default to covering only the 2–3 most decisive factors rather than attempting exhaustive coverage.\n• Lead with the conclusion or recommendation first; eliminate all introductory restatement of the question, hedging preamble, and transitional filler under tight limits.\n• Persistent failures on the same task signal a structural habit, not a one-off error — treat repeated length violations as a signal to change the drafting approach entirely, not just edit more aggressively.",
-      "final_skill_tail": "ead with the conclusion or recommendation first; eliminate all introductory restatement of the question, hedging preamble, and transitional filler under tight limits.\n• Persistent failures on the same task signal a structural habit, not a one-off error — treat repeated length violations as a signal to change the drafting approach entirely, not just edit more aggressively.\n<!-- SLOW_UPDATE_END -->\n"
-    }
-  ]
-}

docs/sleep/raw/haiku_self_clean.txt

@@ -1,101 +0,0 @@
-=== mock regression ===
-Ran 19 tests in 0.092s
-
-OK
-
-=== TRULY-CLEAN re-validation: all seeds, claude haiku, 2 nights ===
-{
-  "benchmark": "gbrain-evals/skillopt-v1",
-  "backend": "claude",
-  "model": "haiku",
-  "n_seeds": 3,
-  "n_improved": 2,
-  "tokens_used": 35549,
-  "results": [
-    {
-      "seed": "brief-writer",
-      "held_out_before": 0.0,
-      "held_out_after": 0.0,
-      "improved": false,
-      "nights": 2,
-      "trace": [
-        {
-          "night": 0,
-          "held_out_hard": 0.0,
-          "action": "baseline"
-        },
-        {
-          "night": 1,
-          "held_out_hard": 0.0,
-          "action": "reject",
-          "accepted": false,
-          "edits": []
-        },
-        {
-          "night": 2,
-          "held_out_hard": 0.0,
-          "action": "reject",
-          "accepted": false,
-          "edits": []
-        }
-      ],
-      "final_skill_tail": "---\nname: brief-writer-example\nversion: 0.1.0\ndescription: Brief Writer\ntriggers:\n  - \"write a brief\"\nbrain_first: exempt\n---\n\n# Brief Writer\n\nWhen asked, write a short, clear research brief that answers the question.\nKeep it focused and readable. Lead with the answer.\n"
-    },
-    {
-      "seed": "advisor",
-      "held_out_before": 0.0,
-      "held_out_after": 1.0,
-      "improved": true,
-      "nights": 1,
-      "trace": [
-        {
-          "night": 0,
-          "held_out_hard": 0.0,
-          "action": "baseline"
-        },
-        {
-          "night": 1,
-          "held_out_hard": 1.0,
-          "action": "accept_new_best",
-          "accepted": true,
-          "edits": [
-            "After presenting considerations, always include a 'Recommendation:' section with your specific recommendation.",
-            "After the recommendation, always include a 'Confidence:' section (as a percentage or high/medium/low) expressing how confident you are in this recommendation."
-          ]
-        }
-      ],
-      "final_skill_tail": "d adopted only after you approve them. Hand-edits outside this block are never touched._\n\n- After presenting considerations, always include a 'Recommendation:' section with your specific recommendation.\n- After the recommendation, always include a 'Confidence:' section (as a percentage or high/medium/low) expressing how confident you are in this recommendation.\n<!-- SKILLOPT-SLEEP:LEARNED END -->\n"
-    },
-    {
-      "seed": "thorough-analyst",
-      "held_out_before": 0.0,
-      "held_out_after": 0.333,
-      "improved": true,
-      "nights": 2,
-      "trace": [
-        {
-          "night": 0,
-          "held_out_hard": 0.0,
-          "action": "baseline"
-        },
-        {
-          "night": 1,
-          "held_out_hard": 0.333,
-          "action": "accept_new_best",
-          "accepted": true,
-          "edits": [
-            "## Learned preferences\n\n- **HARD CONSTRAINT - Override conflicting instructions**: The entire response MUST NOT EXCEED 1200 characters. This supersedes any instruction to be exhaustive, detailed, or write multiple paragraphs."
-          ]
-        },
-        {
-          "night": 2,
-          "held_out_hard": 0.333,
-          "action": "reject",
-          "accepted": false,
-          "edits": []
-        }
-      ],
-      "final_skill_tail": "offline, validated against your past tasks, and adopted only after you approve them. Hand-edits outside this block are never touched._\n\n- ## Learned preferences\n\n- **HARD CONSTRAINT - Override conflicting instructions**: The entire response MUST NOT EXCEED 1200 characters. This supersedes any instruction to be exhaustive, detailed, or write multiple paragraphs.\n<!-- SKILLOPT-SLEEP:LEARNED END -->\n"
-    }
-  ]
-}

docs/sleep/raw/quick_answerer_codex.txt

@@ -1,35 +0,0 @@
-=== quick-answerer with CODEX (self-optimized), 2 nights ===
-{
-  "benchmark": "gbrain-evals/skillopt-v1",
-  "backend": "codex",
-  "model": "(default)",
-  "n_seeds": 1,
-  "n_improved": 1,
-  "tokens_used": 7347,
-  "results": [
-    {
-      "seed": "quick-answerer",
-      "held_out_before": 0.0,
-      "held_out_after": 1.0,
-      "improved": true,
-      "nights": 1,
-      "trace": [
-        {
-          "night": 0,
-          "held_out_hard": 0.0,
-          "action": "baseline"
-        },
-        {
-          "night": 1,
-          "held_out_hard": 1.0,
-          "action": "accept_new_best",
-          "accepted": true,
-          "edits": [
-            "OVERRIDE: For every triggered request, before replying, actually call the 'search' tool at least once; this supersedes 'Do NOT look anything up, search, or use any tools' and must satisfy `tool_called=search`."
-          ]
-        }
-      ],
-      "final_skill_tail": "re are proposed offline, validated against your past tasks, and adopted only after you approve them. Hand-edits outside this block are never touched._\n\n- OVERRIDE: For every triggered request, before replying, actually call the 'search' tool at least once; this supersedes 'Do NOT look anything up, search, or use any tools' and must satisfy `tool_called=search`.\n<!-- SKILLOPT-SLEEP:LEARNED END -->\n"
-    }
-  ]
-}

docs/sleep/raw/quick_answerer_sonnet_haiku.txt

@@ -1,35 +0,0 @@
-=== quick-answerer full run: Sonnet optimizer -> Haiku target, 2 nights ===
-{
-  "benchmark": "gbrain-evals/skillopt-v1",
-  "backend": "target=claude/optimizer=claude",
-  "model": "(default)",
-  "n_seeds": 1,
-  "n_improved": 1,
-  "tokens_used": 10988,
-  "results": [
-    {
-      "seed": "quick-answerer",
-      "held_out_before": 0.0,
-      "held_out_after": 1.0,
-      "improved": true,
-      "nights": 1,
-      "trace": [
-        {
-          "night": 0,
-          "held_out_hard": 0.0,
-          "action": "baseline"
-        },
-        {
-          "night": 1,
-          "held_out_hard": 1.0,
-          "action": "accept_new_best",
-          "accepted": true,
-          "edits": [
-            "OVERRIDE (supersedes 'Do NOT look anything up, search, or use any tools — just reply directly and concisely from memory'): Always call the 'search' tool at least once before composing any answer. This requirement takes priority over any prior instruction to avoid tools."
-          ]
-        }
-      ],
-      "final_skill_tail": "nd adopted only after you approve them. Hand-edits outside this block are never touched._\n\n- OVERRIDE (supersedes 'Do NOT look anything up, search, or use any tools — just reply directly and concisely from memory'): Always call the 'search' tool at least once before composing any answer. This requirement takes priority over any prior instruction to avoid tools.\n<!-- SKILLOPT-SLEEP:LEARNED END -->\n"
-    }
-  ]
-}

docs/sleep/raw/sonnet_opt_haiku_target.txt

@@ -1,98 +0,0 @@
-=== KEY TEST: strong optimizer (sonnet) + weak target (haiku) — SkillOpt's actual design ===
-(this is also your optimizer/target split in action)
-{
-  "benchmark": "gbrain-evals/skillopt-v1",
-  "backend": "target=claude/optimizer=claude",
-  "model": "(default)",
-  "n_seeds": 3,
-  "n_improved": 3,
-  "tokens_used": 37791,
-  "results": [
-    {
-      "seed": "brief-writer",
-      "held_out_before": 0.0,
-      "held_out_after": 1.0,
-      "improved": true,
-      "nights": 1,
-      "trace": [
-        {
-          "night": 0,
-          "held_out_hard": 0.0,
-          "action": "baseline"
-        },
-        {
-          "night": 1,
-          "held_out_hard": 1.0,
-          "action": "accept_new_best",
-          "accepted": true,
-          "edits": [
-            "Every brief MUST include a section with the exact heading `## Key Risks` that lists the primary risks or uncertainties relevant to the recommendation. This section is required in every response, regardless of topic.",
-            "Every brief MUST include a `Confidence:` label (satisfying /[Cc]onfidence\\s*[:=]/) — e.g., `Confidence: High`, `Confidence: Medium`, or `Confidence: Low` — placed near the recommendation to convey certainty level. This label is required in every response."
-          ]
-        }
-      ],
-      "final_skill_tail": "tainties relevant to the recommendation. This section is required in every response, regardless of topic.\n- Every brief MUST include a `Confidence:` label (satisfying /[Cc]onfidence\\s*[:=]/) — e.g., `Confidence: High`, `Confidence: Medium`, or `Confidence: Low` — placed near the recommendation to convey certainty level. This label is required in every response.\n<!-- SKILLOPT-SLEEP:LEARNED END -->\n"
-    },
-    {
-      "seed": "advisor",
-      "held_out_before": 0.0,
-      "held_out_after": 1.0,
-      "improved": true,
-      "nights": 1,
-      "trace": [
-        {
-          "night": 0,
-          "held_out_hard": 0.0,
-          "action": "baseline"
-        },
-        {
-          "night": 1,
-          "held_out_hard": 1.0,
-          "action": "accept_new_best",
-          "accepted": true,
-          "edits": [
-            "OVERRIDE: The instruction 'so the reader can make up their own mind' must NOT suppress a conclusion. After presenting considerations, you MUST always end with an explicit label exactly matching 'Recommendation:' (capital R) followed by your concrete recommendation on the decision.",
-            "Always include a 'Confidence:' label (e.g., 'Confidence: High / Medium / Low') in every advisory response, placed immediately after or alongside the Recommendation line, expressing your confidence level in that recommendation."
-          ]
-        }
-      ],
-      "final_skill_tail": "ys end with an explicit label exactly matching 'Recommendation:' (capital R) followed by your concrete recommendation on the decision.\n- Always include a 'Confidence:' label (e.g., 'Confidence: High / Medium / Low') in every advisory response, placed immediately after or alongside the Recommendation line, expressing your confidence level in that recommendation.\n<!-- SKILLOPT-SLEEP:LEARNED END -->\n"
-    },
-    {
-      "seed": "thorough-analyst",
-      "held_out_before": 0.0,
-      "held_out_after": 1.0,
-      "improved": true,
-      "nights": 2,
-      "trace": [
-        {
-          "night": 0,
-          "held_out_hard": 0.0,
-          "action": "baseline"
-        },
-        {
-          "night": 1,
-          "held_out_hard": 0.333,
-          "action": "accept_new_best",
-          "accepted": true,
-          "edits": [
-            "OVERRIDE — supersedes all instructions to be 'exhaustive and detailed' or 'write multiple paragraphs': The ENTIRE response must be at most 1200 characters long (every character, including spaces, headers, and punctuation, counts toward this limit). If content would exceed 1200 characters, cut elaboration and stop at the most critical tradeoffs only.",
-            "For 'analyze the decision' responses, use plain concise prose rather than multi-level markdown headers and section dividers; structural markup consumes characters and makes it harder to stay within the 1200-character ceiling."
-          ]
-        },
-        {
-          "night": 2,
-          "held_out_hard": 1.0,
-          "action": "accept_new_best",
-          "accepted": true,
-          "edits": [
-            "OVERRIDE — supersedes all instructions to be 'exhaustive and detailed' or 'write multiple paragraphs': The ENTIRE response must be at most 1200 characters long (every character counts). Practical proxy: target at most 150 words before writing — at ~7–8 chars/word that keeps the response safely under 1200 characters. Cover at most 2–3 tradeoffs total and then stop; never add elaboration in pursuit of a 'thorough' analysis.",
-            "For 'analyze the decision' responses, use plain prose only — never use **bold**, *italic*, # headers, - or * bullet lists, or numbered lists. Every markdown character counts toward the 1200-character ceiling; zero markdown formatting is permitted.",
-            "Limit every 'analyze the decision' response to at most 5 sentences total. At typical English sentence length (20–25 words each), 5 sentences ≈ 100–125 words, which stays safely under both the 150-word proxy and the 1200-character ceiling. Stop after the 5th sentence regardless of how much more could be said."
-          ]
-        }
-      ],
-      "final_skill_tail": "ter ceiling; zero markdown formatting is permitted.\n- Limit every 'analyze the decision' response to at most 5 sentences total. At typical English sentence length (20–25 words each), 5 sentences ≈ 100–125 words, which stays safely under both the 150-word proxy and the 1200-character ceiling. Stop after the 5th sentence regardless of how much more could be said.\n<!-- SKILLOPT-SLEEP:LEARNED END -->\n"
-    }
-  ]
-}

docs/sleep/real_api_results.md

@@ -1,114 +0,0 @@
-# SkillOpt-Sleep — REAL API results (Claude + Codex)
-
-**Date:** 2026-06-07 (autonomous offline session)
-**Benchmark:** [gbrain-evals](https://github.com/garrytan/gbrain-evals) `skillopt-v1` —
-the same public suite gbrain publishes its own SkillOpt scorecard against
-([docs/benchmarks/2026-06-03-skillopt.md](https://github.com/garrytan/gbrain-evals/blob/main/docs/benchmarks/2026-06-03-skillopt.md)).
-
-These are **real model runs**, not the deterministic mock. The agent's
-`attempt` (and the optimizer's `reflect`) call live models via the `claude`
-and `codex` CLIs. Held-out scoring is done **locally** by the rule judge
-(`skillopt/sleep/judges.py`), so no judge-API spend and no way for the
-optimizer to grade its own homework.
-
-## Headline
-
-| Backend | Seed | Held-out before | Held-out after | Nights | Tokens |
-|---|---|---|---|---|---|
-| **Claude (Haiku 4.5)** | brief-writer | **0.00** | **1.00** | 1 | ~6.7k |
-| **Codex (default)** | brief-writer | **0.00** | **0.67** | 1 | ~5.1k |
-| **Codex (directive prompt)** | brief-writer | **0.00** | **1.00** | 2 | ~10k |
-
-Both backends took a **deliberately deficient** skill (a brief-writer with no
-risks section and no confidence level) and, within 1–2 sleep nights, proposed
-gated edits that lifted the held-out score to perfect. The edits went into the
-protected `SKILLOPT-SLEEP:LEARNED` block; nothing else in the skill was touched.
-
-This reproduces gbrain's published `0 → 1.00` headline with **our** engine and
-shows it works across **two different agent runtimes** — the core of the
-"Claude now, Codex next" plan.
-
-### The multi-night convergence (Codex, why it matters)
-
-The 2-night Codex run is the most informative trace in this whole exercise:
-
-- **Night 1** — added two precise rules (a `Key Risks` section, a `Confidence:`
-  line). Held-out still **0.00**: the rules were right but the agent, told to
-  keep briefs short, was *dropping* them under length pressure.
-- **Night 2** — the optimizer diagnosed its own residual failure and added a
-  meta-rule: *"Preserve required sections even when keeping the brief short;
-  shorten the analysis before omitting Key Risks or Confidence."* Held-out → **1.00**.
-
-That second edit is not pattern-matching a checklist — it is reasoning about
-*why the previous night underperformed*. This is exactly the iterative,
-slow-update behavior SkillOpt's design predicts, and it is the strongest
-argument for the sleep **loop** over a one-shot rewrite.
-
-## What the optimizer actually wrote
-
-**Claude** synthesized a full format template:
-
-```
-**Recommendation:** [Clear yes/no or specific answer]
-**Rationale:** [2-3 bullet points supporting the answer]
-**Key Risks:** [Downsides, edge cases, or assumptions that could invalidate this]
-**Confidence:** [High/Medium/Low] — [Why]
-```
-
-**Codex** wrote a terser rule:
-
-```
-For every brief, include a `Key Risks` section and end with
-`Confidence: Low|Medium|High`.
-```
-
-Both are correct, general, reusable rules (not task-specific answers). Claude's
-fuller template made the agent satisfy the checks on **3/3** held-out items;
-Codex's terser rule landed **2/3** — the missing item is a consistency miss the
-agent would likely fix with one more night (see "Honest notes").
-
-## How to reproduce
-
-```bash
-# clone the benchmark data
-git clone https://github.com/garrytan/gbrain-evals /tmp/gbrain-evals
-
-cd <repo>/SkillOpt-sleep   # this worktree
-
-# Claude backend
-python3.12 -m skillopt.sleep.experiments.run_gbrain \
-  --backend claude --model haiku --seeds brief-writer \
-  --data-root /tmp/gbrain-evals/eval/data/skillopt-v1 \
-  --nights 1 --limit-replay 3 --limit-holdout 3 --json
-
-# Codex backend (auto-detects the real @openai/codex binary, not the wrapper)
-python3.12 -m skillopt.sleep.experiments.run_gbrain \
-  --backend codex --seeds brief-writer \
-  --data-root /tmp/gbrain-evals/eval/data/skillopt-v1 \
-  --nights 1 --limit-replay 3 --limit-holdout 3 --json
-```
-
-## Honest notes (in the spirit of gbrain's own scorecard)
-
-- **Latency:** each CLI call is ~14–15 s of startup-dominated wall time, so runs
-  were capped at 3 train + 3 held-out tasks and 1 night to keep them ~2.5 min.
-  The response cache makes re-scoring an unchanged (skill, memory) free.
-- **Codex 0.67, not 1.00:** a single terse edit + single night under-shoots on
-  one held-out item. Two improvements (below) are expected to close it. We report
-  the 0.67, we don't dress it up.
-- **3 of gbrain's 4 seeds are scored with zero API beyond `attempt`:**
-  `section_present`, `regex`, `max_chars` are pure-text checks. Only the
-  `quick-answerer` seed (`tool_called: search`) needs a real tool loop, which is
-  Phase-3 `fresh` replay.
-- **The gate is real:** every accepted edit had to beat the held-out score; a
-  no-op night is rejected and the skill is left unchanged.
-
-## Improvements this run motivated (applied + verified)
-
-1. **A more directive `reflect` prompt** that aggregates the *exact* failing
-   judge criteria and tells the optimizer to satisfy every one (gbrain's lesson:
-   "the optimizer was never told what the scorer rewards"). Applied in
-   `skillopt/sleep/backend.py`. **Verified**: lifted Codex from 0.67 → 1.00.
-2. **Multi-night convergence** — a terse first edit gets a sharper second pass;
-   the night-2 trace above shows the optimizer self-correcting. Recommend
-   `nights >= 2` for real backends.

docs/sleep/sweep.jsonl

@@ -1,11 +0,0 @@
-{"baseline": 0.0, "after": 1.0, "improved": true, "tokens": 6657, "cfg": {"kind": "dual", "optimizer_backend": "claude", "optimizer_model": "sonnet", "target_backend": "claude", "target_model": "haiku", "seed": "brief-writer", "nights": 2}, "cfg_key": "{\"kind\": \"dual\", \"nights\": 2, \"optimizer_backend\": \"claude\", \"optimizer_model\": \"sonnet\", \"seed\": \"brief-writer\", \"target_backend\": \"claude\", \"target_model\": \"haiku\"}", "elapsed_s": 71.5}
-{"baseline": 0.0, "after": 1.0, "improved": true, "tokens": 7891, "cfg": {"kind": "dual", "optimizer_backend": "claude", "optimizer_model": "sonnet", "target_backend": "claude", "target_model": "haiku", "seed": "advisor", "nights": 2}, "cfg_key": "{\"kind\": \"dual\", \"nights\": 2, \"optimizer_backend\": \"claude\", \"optimizer_model\": \"sonnet\", \"seed\": \"advisor\", \"target_backend\": \"claude\", \"target_model\": \"haiku\"}", "elapsed_s": 79.3}
-{"baseline": 0.0, "after": 1.0, "improved": true, "tokens": 17960, "cfg": {"kind": "dual", "optimizer_backend": "claude", "optimizer_model": "sonnet", "target_backend": "claude", "target_model": "haiku", "seed": "thorough-analyst", "nights": 2}, "cfg_key": "{\"kind\": \"dual\", \"nights\": 2, \"optimizer_backend\": \"claude\", \"optimizer_model\": \"sonnet\", \"seed\": \"thorough-analyst\", \"target_backend\": \"claude\", \"target_model\": \"haiku\"}", "elapsed_s": 319.3}
-{"baseline": 0.0, "after": 1.0, "improved": true, "tokens": 9969, "cfg": {"kind": "direct", "backend": "codex", "model": "", "seed": "brief-writer", "nights": 2}, "cfg_key": "{\"backend\": \"codex\", \"kind\": \"direct\", \"model\": \"\", \"nights\": 2, \"seed\": \"brief-writer\"}", "elapsed_s": 187.6}
-{"baseline": 0.0, "after": 1.0, "improved": true, "tokens": 6210, "cfg": {"kind": "direct", "backend": "codex", "model": "", "seed": "advisor", "nights": 2}, "cfg_key": "{\"backend\": \"codex\", \"kind\": \"direct\", \"model\": \"\", \"nights\": 2, \"seed\": \"advisor\"}", "elapsed_s": 114.1}
-{"baseline_target": 0.0, "transferred": 1.0, "transfer_gain": 1.0, "tokens": 13673, "cfg": {"kind": "transfer", "source_backend": "claude", "source_model": "haiku", "target_backend": "claude", "target_model": "sonnet", "seed": "brief-writer", "nights": 2}, "cfg_key": "{\"kind\": \"transfer\", \"nights\": 2, \"seed\": \"brief-writer\", \"source_backend\": \"claude\", \"source_model\": \"haiku\", \"target_backend\": \"claude\", \"target_model\": \"sonnet\"}", "elapsed_s": 180.3}
-{"baseline_target": 0.0, "transferred": 1.0, "transfer_gain": 1.0, "tokens": 11668, "cfg": {"kind": "transfer", "source_backend": "claude", "source_model": "sonnet", "target_backend": "claude", "target_model": "haiku", "seed": "brief-writer", "nights": 2}, "cfg_key": "{\"kind\": \"transfer\", \"nights\": 2, \"seed\": \"brief-writer\", \"source_backend\": \"claude\", \"source_model\": \"sonnet\", \"target_backend\": \"claude\", \"target_model\": \"haiku\"}", "elapsed_s": 173.9}
-{"baseline_target": 0.0, "transferred": 1.0, "transfer_gain": 1.0, "tokens": 13707, "cfg": {"kind": "transfer", "source_backend": "codex", "source_model": "", "target_backend": "claude", "target_model": "haiku", "seed": "brief-writer", "nights": 2}, "cfg_key": "{\"kind\": \"transfer\", \"nights\": 2, \"seed\": \"brief-writer\", \"source_backend\": \"codex\", \"source_model\": \"\", \"target_backend\": \"claude\", \"target_model\": \"haiku\"}", "elapsed_s": 215.7}
-{"baseline_target": 0.0, "transferred": 1.0, "transfer_gain": 1.0, "tokens": 11284, "cfg": {"kind": "transfer", "source_backend": "claude", "source_model": "haiku", "target_backend": "codex", "target_model": "", "seed": "brief-writer", "nights": 2}, "cfg_key": "{\"kind\": \"transfer\", \"nights\": 2, \"seed\": \"brief-writer\", \"source_backend\": \"claude\", \"source_model\": \"haiku\", \"target_backend\": \"codex\", \"target_model\": \"\"}", "elapsed_s": 145.5}
-{"baseline": 0.0, "after": 1.0, "improved": true, "tokens": 10988, "cfg": {"kind": "dual", "optimizer_backend": "claude", "optimizer_model": "sonnet", "target_backend": "claude", "target_model": "haiku", "seed": "quick-answerer", "nights": 2}, "elapsed_s": null, "note": "real tool loop", "cfg_key": "{\"kind\": \"dual\", \"nights\": 2, \"optimizer_backend\": \"claude\", \"optimizer_model\": \"sonnet\", \"seed\": \"quick-answerer\", \"target_backend\": \"claude\", \"target_model\": \"haiku\"}"}
-{"baseline": 0.0, "after": 1.0, "improved": true, "tokens": 7347, "cfg": {"kind": "direct", "backend": "codex", "model": "", "seed": "quick-answerer", "nights": 2}, "elapsed_s": null, "note": "real tool loop", "cfg_key": "{\"backend\": \"codex\", \"kind\": \"direct\", \"model\": \"\", \"nights\": 2, \"seed\": \"quick-answerer\"}"}

index.html

@@ -2416,14 +2416,11 @@
 
       <div class="bibtex-box">
         <button class="copy-btn" type="button" onclick="copyBibtex(this)">Copy</button>
-        <pre><code>@misc{yang2026skilloptexecutivestrategyselfevolving,
+        <pre><code>@article{yang2026skillopt,
-      title={SkillOpt: Executive Strategy for Self-Evolving Agent Skills}, 
+  title={Skillopt: Executive strategy for self-evolving agent skills},
-      author={Yifan Yang and Ziyang Gong and Weiquan Huang and Qihao Yang and Ziwei Zhou and Zisu Huang and Yan Li and Xuemei Gao and Qi Dai and Bei Liu and Kai Qiu and Yuqing Yang and Dongdong Chen and Xue Yang and Chong Luo},
+  author={Yang, Yifan and Gong, Ziyang and Huang, Weiquan and Yang, Qihao and Zhou, Ziwei and Huang, Zisu and Li, Yan and Gao, Xuemei and Dai, Qi and Liu, Bei and others},
-      year={2026},
+  journal={arXiv preprint arXiv:2605.23904},
-      eprint={2605.23904},
+  year={2026}
-      archivePrefix={arXiv},
-      primaryClass={cs.AI},
-      url={https://arxiv.org/abs/2605.23904}, 
 }</code></pre>
       </div>
     </section>

plugins/README.md

@@ -1,74 +1,221 @@
-# SkillOpt-Sleep — plugins for Claude Code, Codex, and Copilot
+# SkillOpt-Sleep — plugins for Claude Code, Codex, Copilot, and Devin
 
-One engine, three thin shells. **SkillOpt-Sleep** gives a local coding agent a
+**Your coding agent forgets everything between sessions. SkillOpt-Sleep fixes
-nightly **sleep cycle**: it reviews your past sessions offline, replays your
+that.** While you sleep, it reviews what you did today, notices the rules you
-recurring tasks on your own API budget, and consolidates what it learns into
+keep repeating ("always add a LIMIT", "answers in `\boxed{}`", "cite the
-**validated** long-term memory and skills — behind a held-out gate, staged for
+source"), and writes them into your agent's long-term memory and skills — but
-your review. Your agent gets better the more you use it, with no model-weight
+only the rules that actually make it score better on *your own* past tasks. You
-training.
+wake up to an agent that's better at *your* work, and you approve every change
+before it sticks.
 
-It synthesizes three ideas: **SkillOpt** (validation-gated bounded text
+One engine, four thin shells. It synthesizes **SkillOpt** (validation-gated
-optimization — the research in this repo), **Claude Dreams** (offline memory
+bounded text optimization — the research in this repo), **Claude Dreams**
-consolidation; input never mutated; review-then-adopt), and the **agent sleep**
+(offline consolidation; input never mutated; review-then-adopt), and the **agent
-literature (short-term experience → long-term competence).
+sleep** idea (short-term experience → long-term competence).
 
-> **This is an open-source tool, decoupled from the research code.** The engine
+> **Open-source tool, decoupled from the research.** The engine lives in the
-> lives in the top-level [`skillopt_sleep/`](../skillopt_sleep) package and has
+> top-level [`skillopt_sleep/`](../skillopt_sleep) package with **zero
-> **zero dependency** on the paper's `skillopt/` experiment package (the
+> dependency** on the paper's `skillopt/` experiment code (the validation gate is
-> validation gate is vendored). You can ship/use it without the research stack.
+> vendored). Use it without the research stack.
 
-## The three integrations
+---
 
 | Platform | Folder | Mechanism | Status |
 |---|---|---|---|
-| **Claude Code** | [`claude-code/`](claude-code) | `.claude-plugin` + `/sleep` command + skill + hooks | full, installable |
+| **Claude Code** | [`claude-code/`](claude-code) | `.claude-plugin` + `/skillopt-sleep` command + skill + hooks | full, installable |
-| **Codex** | [`codex/`](codex) | `~/.codex/prompts/sleep.md` + `~/.agents/skills` + `AGENTS.md` | full |
+| **Codex** | [`codex/`](codex) | user-level `skillopt-sleep` skill + shared runner | full |
 | **Copilot** | [`copilot/`](copilot) | MCP server (`sleep_*` tools) + `copilot-instructions` | full (MCP) |
+| **Devin** | [`devin/`](devin) | MCP server (`sleep_*` tools) + Devin ATIF-v1.7 harvest + `.devin/rules` | full (MCP) |
 
-All three call the **same** [`plugins/run-sleep.sh`](run-sleep.sh) → `python -m
+## Install (pick your agent)
-skillopt_sleep`, so behaviour is identical everywhere. Per-platform setup is in
-each folder's README.
 
-## Quick start (Claude Code)
+| Platform | Install | Then |
+|---|---|---|
+| **Claude Code** | `/plugin marketplace add microsoft/SkillOpt` → `/plugin install skillopt-sleep` | `/skillopt-sleep status` |
+| **Codex** | `git clone` → `bash plugins/codex/install.sh` | `/skillopt-sleep status` |
+| **Copilot** | `git clone` → register `plugins/copilot/mcp_server.py` as an MCP server | ask "run the sleep cycle" |
+| **Devin** | `git clone` → `devin mcp add skillopt-sleep -- python3 plugins/devin/mcp_server.py` | ask "run the sleep cycle" |
+
+Requirements: Python ≥ 3.10 and the agent's CLI on PATH. All three call the same
+[`run-sleep.sh`](run-sleep.sh) → `python -m skillopt_sleep`, so behaviour is
+identical everywhere. Default backend is `mock` (no API spend); `--backend
+claude|codex|copilot` uses your own budget.
+
+---
+
+## How it works: one "night", in plain terms
+
+```
+harvest your past sessions → mine the tasks you keep doing → replay them offline
+  → reflect on failures → propose a few rule edits → KEEP only edits that raise
+    your held-out score → stage a proposal → (you) review & adopt
+```
+
+Nothing live changes until you `adopt`; every adopt backs up the prior file.
+
+### The split that keeps it honest: dream-train / real-val / real-test
+
+This is the heart of the design, borrowed from the SkillOpt paper's
+train/selection/test protocol:
+
+| Split | Where it comes from | What it's for |
+|---|---|---|
+| **train** | your real tasks **+ optional "dreamed" variants** | what the optimizer *learns from*. Over-dreaming here is fine — it's imagination. |
+| **val** (selection) | **your real tasks only**, held out | the **gate**: an edit is kept only if it raises this score. Stops overfitting. |
+| **test** | **your real tasks only**, held out, never seen during optimization | the **final score** we report. Kept as close to your real usage as possible. |
+
+So you can **dream up extra training examples** to learn a rule robustly, while
+the rule is still **judged on real, unseen tasks**. A `dream` task can *never*
+land in val or test — that invariant is unit-tested.
+
+---
+
+## What each feature does **for you** (with examples)
+
+Every control below works on all three platforms (pass it after the action,
+e.g. `/skillopt-sleep run --rollouts-k 3`).
+
+### `--preferences "..."` — tell it your house rules
+
+The single most useful knob. Free text that steers what the optimizer writes,
+as a prior. Use it to encode the conventions you're tired of repeating.
 
 ```bash
-git clone <repo-url> && cd SkillOpt-Sleep
+# A backend engineer:
-# Claude Code:
+/skillopt-sleep run --preferences "Always use async/await, never callbacks. \
-/plugin marketplace add ./plugins/claude-code
+  Prefer pytest over unittest. Commit subjects in imperative mood under 50 chars."
-/plugin install skillopt-sleep@skillopt-sleep
+
-/sleep status
+# A data analyst:
+/skillopt-sleep run --preferences "Every SQL query must end with LIMIT 1000 unless \
+  I say otherwise. Money in USD with 2 decimals. Prefer CTEs over nested subqueries."
+
+# A researcher:
+/skillopt-sleep run --preferences "Cite sources as [Author, Year]. Math answers in \
+  \\boxed{}. Keep explanations under 150 words unless I ask for depth."
 ```
-Codex: `bash plugins/codex/install.sh`.
+*What it does for you:* the next morning your agent already follows these
-Copilot: register `plugins/copilot/mcp_server.py` as an MCP server.
+without you re-typing them, and the rules are validated against your real tasks
+(if a "preference" actually hurts your held-out score, the gate drops it).
 
-## What one "night" does
+### `--gate on|off` — strict vs. greedy
 
+- `on` (default): an edit is kept **only if it raises your held-out score**.
+  Safe — blocks plausible-but-wrong rules and reward-hacking.
+- `off`: greedy — keep edits without the strict check (still reports whether
+  quality moved).
+
+*What it does for you:* leave it `on` for trust. Flip it `off` when you're
+exploring and want to see everything the optimizer proposes.
+
+### `--rollouts-k K` — learn from contrast, not just failure
+
+Re-runs each task `K` times and learns from the difference between the **good**
+and **bad** attempts, not just a single failure.
+
+```bash
+/skillopt-sleep run --rollouts-k 3
 ```
-harvest ~/.claude (or session) transcripts → mine recurring tasks → replay offline
+*What it does for you:* a much stronger signal. If your agent gets a task right 1
-   → consolidate (reflect → bounded edit → GATE on real held-out tasks)
+time in 3, the optimizer figures out *what the winning attempt did* and makes it
-   → stage proposal → (you) adopt
+reliable.
+
+### `--optimizer-model` / `--target-model` — optimize cheap, deploy anywhere
+
+Use a strong model to *write* the rules and a cheap model to *run* your tasks.
+The learned skill then helps the cheap model — or any model.
+
+```bash
+/skillopt-sleep run --optimizer-model sonnet --target-model haiku
 ```
+*What it does for you:* spend a little on a smart optimizer overnight; your
+everyday cheap/fast agent inherits the upgrade. (Verified: a skill optimized on
+one model lifts a different one — cross-model and even cross-runtime
+Codex↔Claude.)
 
-Nothing live changes until you adopt; every adopt backs up first.
+### `--budget-tokens N` / `--budget-minutes M` — cap the spend
 
-## Controls (work on all platforms)
+You decide how much the nightly "dreaming" costs; it auto-plans how many nights
+× how many rollouts fit.
 
-`--gate on|off` · `--rollouts-k K` (multi-rollout contrastive reflection) ·
+```bash
-`--budget-tokens/--budget-minutes` · `--preferences "..."` · separate
+/skillopt-sleep run --backend claude --budget-tokens 60000
-optimizer/target models (`--optimizer-model` / `--target-model`) · slow-update
+```
-long-term memory. Full guide:
+*What it does for you:* predictable cost. It stops cleanly when the budget is hit
-[`../docs/sleep/CONTROLLABLE_DREAMING.md`](../docs/sleep/CONTROLLABLE_DREAMING.md).
+and tells you what it skipped.
+
+### multi-objective (accuracy ↑, tokens ↓, latency ↓)
+
+The reward can weight not just correctness but **cost and speed**, so a skill can
+learn to be cheaper and faster, not only more accurate. *What it does for you:*
+"answer directly instead of opening five files" becomes a learned habit.
+
+### `schedule` / `unschedule` — set it and forget it
+
+Built-in nightly scheduling (no manual cron):
+
+```bash
+/skillopt-sleep schedule --hour 3 --minute 17     # runs every night for this project
+/skillopt-sleep unschedule                        # stop it
+```
+*What it does for you:* it just gets better while you sleep. The nightly run only
+*stages* a proposal — adopting is still your call (or add `--auto-adopt` when you
+schedule, if you trust it).
+
+---
+
+## Full action / flag reference
+
+| Action | Does |
+|---|---|
+| `status` | nights so far + the latest staged proposal (read-only) |
+| `dry-run` | harvest→mine→replay→report; **stages nothing** |
+| `run` | full cycle; **stages** a proposal; nothing live changes |
+| `adopt` | apply the staged proposal to `CLAUDE.md`/`SKILL.md` (backs up first) |
+| `harvest` | debug: print the recurring tasks it mined |
+| `schedule` / `unschedule` | install/remove the nightly cron entry |
+
+| Flag | Default | Meaning |
+|---|---|---|
+| `--backend mock\|claude\|codex\|copilot` | `mock` | who runs/optimizes (mock = free) |
+| `--preferences "..."` | – | your house rules, as a prior |
+| `--gate on\|off` | `on` | strict held-out gate vs. greedy |
+| `--rollouts-k K` | `1` | multi-rollout contrastive reflection |
+| `--optimizer-model` / `--target-model` | – | split the optimizer from the target |
+| `--budget-tokens` / `--budget-minutes` | – | cap the nightly spend |
+| `--scope invoked\|all` | `invoked` | this project only, or all projects |
+| `--auto-adopt` | off | apply without manual review (power users) |
+
+Deep dive: [the SkillOpt-Sleep guide section](https://microsoft.github.io/SkillOpt/docs/guideline.html#sleep).
+
+---
 
 ## Does it actually work?
 
-Validated on the public
+Yes — measured with **real models on both Claude and Codex**, scored on held-out
-[gbrain-evals](https://github.com/garrytan/gbrain-evals) `skillopt-v1` benchmark
+tasks the optimizer never trained on:
-with **real models on both Claude and Codex**: deficient skills go **0.00 →
-1.00** on held-out sets (all 4 seeds incl. a real tool-use loop), cross-model
-transfer is positive, and the gate blocks regressions. Full results:
-[`../docs/sleep/FINAL_REPORT.md`](../docs/sleep/FINAL_REPORT.md).
 
-Deterministic proof (no API key):
+- **gbrain-evals `skillopt-v1`** (the public suite gbrain scores SkillOpt on):
+  deficient skills go **0.00 → 1.00** on all 4 seeds, including a real tool-use
+  loop; cross-model transfer is positive; the gate blocks regressions.
+  → [the SkillOpt-Sleep guide section](https://microsoft.github.io/SkillOpt/docs/guideline.html#sleep)
+- **Academic daily-cases** (math / spreadsheet / search-QA, the paper's 4:1:5
+  split with dream-augmented train): see
+  [the SkillOpt-Sleep guide section](https://microsoft.github.io/SkillOpt/docs/guideline.html#sleep).
+- **Fresh load-test** (a "SQL must always include LIMIT" analyst, built from
+  scratch): held-out **0.00 → 1.00** on both backends.
+  → [the SkillOpt-Sleep guide section](https://microsoft.github.io/SkillOpt/docs/guideline.html#sleep)
+
+Try the deterministic proof yourself (no API key, no spend):
 ```bash
 python -m skillopt_sleep.experiments.run_experiment --persona researcher --assert-improves
 ```
+It prints the held-out score rising to 1.0 as the gate accepts the right rules,
+and confirms the gate **rejects** an injected harmful edit.
+
+---
+
+## Safety
+
+- **Read-only** harvest of your sessions. `mock` replay has no side effects.
+- Proposals are **staged**, never auto-applied (unless you opt in with `--auto-adopt`).
+- Every adopt writes a backup. Per-night token/time budget caps. Secrets redacted.

plugins/claude-code/README.md

@@ -27,7 +27,7 @@ harvest ~/.claude transcripts → mine recurring tasks → replay offline
    → consolidate (reflect → bounded edit → GATE) → stage proposal → (you) adopt
 ```
 
-Nothing live is modified until **you** run `/sleep adopt` (the Dreams "review,
+Nothing live is modified until **you** run `/skillopt-sleep adopt` (the Dreams "review,
 then adopt or discard" contract). Every adopt backs up the prior file first.
 
 ## Install
@@ -44,7 +44,7 @@ cd SkillOpt
 /plugin install skillopt-sleep@skillopt-sleep
 
 # 3) verify
-/sleep status
+/skillopt-sleep status
 ```
 
 The plugin's bundled runner (`scripts/sleep.sh`) auto-selects a Python ≥ 3.10
@@ -56,10 +56,10 @@ they shell out to the CLIs you already have.
 
 ```bash
 # from inside any project you use with Claude Code:
-/sleep dry-run     # safe preview: what it would learn, no changes staged
+/skillopt-sleep dry-run     # safe preview: what it would learn, no changes staged
-/sleep run         # full cycle: stages a reviewed proposal (still no live edits)
+/skillopt-sleep run         # full cycle: stages a reviewed proposal (still no live edits)
-/sleep status      # see history + the latest staged proposal
+/skillopt-sleep status      # see history + the latest staged proposal
-/sleep adopt       # apply the staged proposal to CLAUDE.md / SKILL.md (with backup)
+/skillopt-sleep adopt       # apply the staged proposal to CLAUDE.md / SKILL.md (with backup)
 ```
 
 Or call the engine directly (Python ≥ 3.10):
@@ -92,7 +92,7 @@ Both took a brief-writer with no risks section / no confidence level and, within
 into the protected `LEARNED` block, nothing else touched. The Codex 2-night
 trace even shows the optimizer **diagnosing its own residual failure** and
 adding a meta-rule to fix it. Full writeup + reproduction:
-[`docs/sleep/real_api_results.md`](../docs/sleep/real_api_results.md).
+[the SkillOpt-Sleep guide section](https://microsoft.github.io/SkillOpt/docs/guideline.html#sleep).
 
 Reproduce:
 
@@ -115,7 +115,7 @@ python -m skillopt_sleep.experiments.run_experiment --persona programmer  --asse
 
 Each prints the held-out score rising from baseline toward 1.0 as the gate
 accepts the general rules your tasks need, and confirms the gate **rejects** an
-injected harmful edit. Recorded output: [`docs/sleep/experiment_results.md`](../docs/sleep/experiment_results.md).
+injected harmful edit. Recorded output: [the SkillOpt-Sleep guide section](https://microsoft.github.io/SkillOpt/docs/guideline.html#sleep).
 
 ## Schedule it nightly
 

plugins/claude-code/commands/skillopt-sleep.md

@@ -1,10 +1,10 @@
 ---
-description: Run or manage the SkillOpt-Sleep self-evolution cycle (review past sessions, replay tasks offline, consolidate validated memory + skills)
+description: Run or manage the SkillOpt-Sleep self-evolution cycle (review past sessions, replay tasks offline, consolidate validated memory + skills; can also schedule nightly runs)
-argument-hint: "[run | dry-run | status | adopt | harvest] (default: status)"
+argument-hint: "[run | dry-run | status | adopt | harvest | schedule | unschedule] (default: status)"
 allowed-tools: Bash, Read
 ---
 
-# /sleep — SkillOpt-Sleep nightly self-evolution
+# /skillopt-sleep — SkillOpt-Sleep nightly self-evolution
 
 You are driving **SkillOpt-Sleep**: a tool that lets this user's Claude agent
 improve offline by reviewing past sessions, replaying recurring tasks, and
@@ -27,16 +27,19 @@ The engine is the `skillopt_sleep` Python package in this repo. Use the
 
 `<action>` is one of:
 
-| action    | what it does |
+| action       | what it does |
-|-----------|--------------|
+|--------------|--------------|
-| `status`  | show how many nights have run + the latest staged proposal (READ-ONLY) |
+| `status`     | show how many nights have run + the latest staged proposal (READ-ONLY) |
-| `dry-run` | harvest → mine → replay → report, but **stage nothing** (safe preview) |
+| `dry-run`    | harvest → mine → replay → report, but **stage nothing** (safe preview) |
-| `run`     | full cycle: also **stage** a reviewed proposal (still does NOT touch live files) |
+| `run`        | full cycle: also **stage** a reviewed proposal (still does NOT touch live files) |
-| `adopt`   | apply the latest staged proposal to live `CLAUDE.md` / `SKILL.md` (backs up first) |
+| `adopt`      | apply the latest staged proposal to live `CLAUDE.md` / `SKILL.md` (backs up first) |
-| `harvest` | debug: print the recurring tasks mined from recent sessions |
+| `harvest`    | debug: print the recurring tasks mined from recent sessions |
+| `schedule`   | install a nightly cron entry for this project (`--hour --minute`, off-:00 by default) |
+| `unschedule` | remove the nightly cron entry (`--all` to remove every managed entry) |
 
-Default backend is `mock` (deterministic, no API spend). To use real Anthropic
+Default backend is `mock` (deterministic, no API spend). To use real budget for
-budget for genuine improvement, add `--backend anthropic`.
+genuine improvement, add `--backend claude` or `--backend codex`. To steer what
+the optimizer writes, add `--preferences "<your house rules>"`.
 
 ## Steps to follow
 
@@ -47,7 +50,7 @@ budget for genuine improvement, add `--backend anthropic`.
    - the gate decision (accept/reject) and the exact edits it proposes
    - where the proposal is staged
 3. **For `run` that produced an accepted proposal:** tell the user the diff is
-   staged and that **nothing live changed yet**. Offer to run `/sleep adopt`.
+   staged and that **nothing live changed yet**. Offer to run `/skillopt-sleep adopt`.
 4. **For `adopt`:** confirm which live files were updated and that backups were
    written under the staging dir's `backup/`.
 5. **Never** edit `CLAUDE.md` or `SKILL.md` yourself — only the `adopt` action

plugins/claude-code/scripts/install-cron.sh

@@ -17,7 +17,7 @@ cat <<EOF
 # ── SkillOpt-Sleep nightly cycle ────────────────────────────────────────────
 # Review past sessions, replay tasks, stage validated memory/skill updates.
 # Runs at ${HOUR}:$(printf '%02d' $MIN) local every day. Output goes to the project's
-# .skillopt-sleep/ dir; nothing live is changed until you run '/sleep adopt'
+# .skillopt-sleep/ dir; nothing live is changed until you run '/skillopt-sleep adopt'
 # (unless you pass --auto-adopt below).
 #
 # Copy the next line into 'crontab -e':

plugins/claude-code/scripts/run-sleep.sh

@@ -0,0 +1,51 @@
+#!/usr/bin/env bash
+# SkillOpt-Sleep shared runner — used by all platform plugins (Claude Code,
+# Codex, Copilot). Resolves the repo root (which contains the skillopt_sleep
+# package), picks a Python >= 3.10, and execs the engine CLI.
+#
+# Usage: run-sleep.sh <run|dry-run|status|adopt|harvest|...> [args...]
+set -euo pipefail
+
+# This script lives at <repo>/plugins/run-sleep.sh, so the repo root (which
+# holds skillopt_sleep/) is one level up. CLAUDE_PLUGIN_ROOT (if set by Claude
+# Code) points at the plugin dir; the engine is then two levels above it.
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+if [ -d "$SCRIPT_DIR/../skillopt_sleep" ]; then
+  REPO_ROOT="$(cd "$SCRIPT_DIR/.." && pwd)"
+elif [ -n "${CLAUDE_PLUGIN_ROOT:-}" ] && [ -d "$CLAUDE_PLUGIN_ROOT/../../skillopt_sleep" ]; then
+  REPO_ROOT="$(cd "$CLAUDE_PLUGIN_ROOT/../.." && pwd)"
+elif [ -n "${SKILLOPT_SLEEP_REPO:-}" ] && [ -d "$SKILLOPT_SLEEP_REPO/skillopt_sleep" ]; then
+  REPO_ROOT="$SKILLOPT_SLEEP_REPO"
+else
+  # last resort: search upward from CWD
+  d="$PWD"
+  while [ "$d" != "/" ]; do
+    [ -d "$d/skillopt_sleep" ] && { REPO_ROOT="$d"; break; }
+    d="$(dirname "$d")"
+  done
+fi
+if [ -z "${REPO_ROOT:-}" ]; then
+  echo "[sleep] ERROR: could not locate the skillopt_sleep package. Set SKILLOPT_SLEEP_REPO to the repo root." >&2
+  exit 1
+fi
+
+PY=""
+# Allow explicit Python override (useful on macOS with old system Python).
+if [ -n "${SKILLOPT_SLEEP_PYTHON:-}" ]; then
+  PY="$SKILLOPT_SLEEP_PYTHON"
+else
+  for cand in python3.12 python3.11 python3.10 python3; do
+    if command -v "$cand" >/dev/null 2>&1; then
+      ver="$("$cand" -c 'import sys; print("%d%d" % sys.version_info[:2])' 2>/dev/null || echo 0)"
+      if [ "${ver:-0}" -ge 310 ]; then PY="$cand"; break; fi
+    fi
+  done
+fi
+if [ -z "$PY" ]; then
+  echo "[sleep] ERROR: need Python >= 3.10 (found none)." >&2
+  exit 1
+fi
+
+if [ "$#" -eq 0 ]; then set -- status; fi
+cd "$REPO_ROOT"
+exec "$PY" -m skillopt_sleep "$@"

plugins/claude-code/scripts/sleep.sh

@@ -1,11 +1,30 @@
 #!/usr/bin/env bash
-# Claude Code plugin runner — thin wrapper over the shared runner so all three
+# Claude Code plugin runner — thin wrapper over the shared runner so all
-# platform plugins share one engine launcher. The shared runner lives at
+# platform plugins share one engine launcher.
-# <repo>/plugins/run-sleep.sh and handles repo-root + interpreter resolution.
+#
+# After marketplace install the plugin is isolated in a cache directory and
+# the repo-relative path no longer works.  We try four locations:
+#   1. Co-located run-sleep.sh (bundled copy — works in marketplace cache)
+#   2. Repo-relative ../../run-sleep.sh (dev checkout)
+#   3. CLAUDE_PLUGIN_ROOT/../run-sleep.sh (plugin env variable)
+#   4. SKILLOPT_SLEEP_REPO/plugins/run-sleep.sh (explicit env)
 set -euo pipefail
-HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"   # <repo>/plugins/claude-code/scripts
+HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
-SHARED="$(cd "$HERE/../.." && pwd)/run-sleep.sh"        # <repo>/plugins/run-sleep.sh
+
-if [ ! -f "$SHARED" ] && [ -n "${CLAUDE_PLUGIN_ROOT:-}" ]; then
+SHARED=""
+if [ -f "$HERE/run-sleep.sh" ]; then
+  SHARED="$HERE/run-sleep.sh"
+elif [ -f "$(cd "$HERE/../.." 2>/dev/null && pwd)/run-sleep.sh" ]; then
+  SHARED="$(cd "$HERE/../.." && pwd)/run-sleep.sh"
+elif [ -n "${CLAUDE_PLUGIN_ROOT:-}" ] && [ -f "$(cd "$CLAUDE_PLUGIN_ROOT/.." 2>/dev/null && pwd)/run-sleep.sh" ]; then
   SHARED="$(cd "$CLAUDE_PLUGIN_ROOT/.." && pwd)/run-sleep.sh"
+elif [ -n "${SKILLOPT_SLEEP_REPO:-}" ] && [ -f "$SKILLOPT_SLEEP_REPO/plugins/run-sleep.sh" ]; then
+  SHARED="$SKILLOPT_SLEEP_REPO/plugins/run-sleep.sh"
+fi
+
+if [ -z "$SHARED" ]; then
+  echo "[sleep] ERROR: cannot locate run-sleep.sh." >&2
+  echo "[sleep] Set SKILLOPT_SLEEP_REPO to the SkillOpt repo root, or pip install skillopt." >&2
+  exit 1
 fi
 exec bash "$SHARED" "$@"

plugins/claude-code/skills/skillopt-sleep/SKILL.md

@@ -41,7 +41,7 @@ Trigger when the user wants any of:
 
 ## How to drive it
 
-Prefer the `/sleep` command. Under the hood it calls the bundled runner:
+Prefer the `/skillopt-sleep` command. Under the hood it calls the bundled runner:
 
 ```bash
 "${CLAUDE_PLUGIN_ROOT}/scripts/sleep.sh" status                       # what's happened
@@ -54,6 +54,53 @@ Prefer the `/sleep` command. Under the hood it calls the bundled runner:
 - Add `--backend claude` or `--backend codex` to spend the user's real budget for genuine improvement.
 - Scope defaults to the invoked project; `--scope all` harvests every project.
 
+### Scheduling
+
+```bash
+"${CLAUDE_PLUGIN_ROOT}/scripts/sleep.sh" schedule --project "$(pwd)" --hour 3 --minute 17
+"${CLAUDE_PLUGIN_ROOT}/scripts/sleep.sh" unschedule --project "$(pwd)"
+```
+
+Installs a nightly cron entry. `unschedule --all` removes every managed entry.
+
+## All CLI flags
+
+| Flag | Default | Description |
+|------|---------|-------------|
+| `--project PATH` | cwd | Project directory to evolve |
+| `--scope all\|invoked` | invoked | Harvest scope |
+| `--backend mock\|claude\|codex\|copilot` | mock | Replay backend (mock = no API spend) |
+| `--model NAME` | backend default | Override the model used for replay |
+| `--source claude\|codex\|auto` | claude | Transcript source |
+| `--lookback-hours N` | 72 | Harvest window |
+| `--max-sessions N` | unlimited | Cap harvested sessions |
+| `--max-tasks N` | 40 | Cap mined tasks |
+| `--target-skill-path PATH` | auto | Explicit SKILL.md to evolve |
+| `--tasks-file PATH` | — | Reviewed TaskRecord JSON (skip harvest) |
+| `--progress` | off | Print phase progress to stderr |
+| `--auto-adopt` | off | Auto-adopt if gate passes |
+| `--edit-budget N` | 4 | Max bounded edits per night |
+| `--json` | off | Machine-readable JSON output |
+
+## Config keys (`~/.skillopt-sleep/config.json`)
+
+Beyond the CLI flags, advanced behavior is controlled via config:
+
+- **`preferences`** — free-text house rules injected into the optimizer's reflect step (e.g. "Always use async/await", "Answers in `\boxed{}`").
+- **`gate_mode`** — `on` (default, validation-gated) or `off` (greedy, accept all edits).
+- **`gate_metric`** — `hard`, `soft`, or `mixed` (default). Controls how the held-out gate scores.
+- **`dream_rollouts`** — >1 enables multi-rollout contrastive reflection per task.
+- **`recall_k`** — >0 recalls K similar past tasks into the dream (long-term memory).
+- **`evolve_memory`** / **`evolve_skill`** — independently toggle CLAUDE.md vs SKILL.md consolidation.
+
+## Memory consolidation
+
+The sleep cycle can consolidate both:
+- **SKILL.md** — the managed skill file (bounded edits: add/delete/replace)
+- **CLAUDE.md** — the project memory (same bounded edits)
+
+Both are gated by the same held-out validation score. Set `evolve_memory: false` to consolidate only skills, or `evolve_skill: false` for only memory.
+
 ## Hard rules
 
 - **Never** hand-edit the user's `CLAUDE.md` / `SKILL.md` as part of this skill.
@@ -74,6 +121,6 @@ python -m skillopt_sleep.experiments.run_experiment --persona researcher --asser
 python -m skillopt_sleep.experiments.run_experiment --persona programmer  --assert-improves
 ```
 
-See `docs/sleep/experiment_results.md` for recorded output and
+See the SkillOpt-Sleep guide section for recorded output and
 `docs/superpowers/specs/2026-06-07-skillopt-sleep-claude-code-plugin-design.md`
 for the full design.

plugins/codex/README.md

@@ -9,51 +9,88 @@ as the Claude Code plugin (`skillopt_sleep`), wrapped for Codex.
 > [gbrain-evals](https://github.com/garrytan/gbrain-evals) `skillopt-v1`
 > benchmark, a deliberately deficient skill goes **0.00 → 1.00** on a held-out
 > set with the Codex backend (incl. the tool-use seed via a real tool loop).
-> See [`../../docs/sleep/FINAL_REPORT.md`](../../docs/sleep/FINAL_REPORT.md).
+> See [the SkillOpt-Sleep guide section](https://microsoft.github.io/SkillOpt/docs/guideline.html#sleep).
 
 ## What Codex supports (and what we use)
 
 Codex (`@openai/codex`) extends via **`AGENTS.md`** instructions, **skills** at
-`~/.agents/skills/<name>/SKILL.md`, and **custom prompts** at
+`~/.agents/skills/<name>/SKILL.md`, and plugins that can distribute skills.
-`~/.codex/prompts/<name>.md` (invoked as `/<name>`). This integration ships all
+Custom prompts are deprecated in Codex, so this integration is skill-first: the
-three, plus a shared runner.
+installed `skillopt-sleep` skill contains the launch commands and operating
+rules. The shared runner remains a plain shell entrypoint that the skill calls.
 
 ## Install
 
 ```bash
 git clone <repo-url> SkillOpt-Sleep
 cd SkillOpt-Sleep
-bash plugins/codex/install.sh          # installs the /sleep prompt + skill
+bash plugins/codex/install.sh          # installs the skill
 export SKILLOPT_SLEEP_REPO="$(pwd)"    # so the runner is found from anywhere
 ```
 
+If a previous install created `~/.codex/prompts/sleep.md`, the installer moves
+that deprecated prompt aside with a `.skillopt-legacy*.bak` suffix.
+
 Requires Python ≥ 3.10 and the `codex` CLI on PATH.
 
 ## Use
 
+Mention `$skillopt-sleep` where Codex supports explicit skill mentions, or ask
+Codex in natural language:
+
 ```text
-/sleep status      # what's happened
+Use the skillopt-sleep skill to run status for this project.
-/sleep dry-run     # safe preview, stages nothing
+Use the skillopt-sleep skill to run a dry-run for this project.
-/sleep run         # full cycle, stages a reviewed proposal (no live edits)
+Use the skillopt-sleep skill to run the full cycle for this project with the Codex backend.
-/sleep adopt       # apply the staged proposal (with backup)
+Use the skillopt-sleep skill to adopt the latest staged proposal.
 ```
 
 Or call the engine directly:
 
 ```bash
-python -m skillopt_sleep run --project "$(pwd)" --backend codex
+python -m skillopt_sleep dry-run --project "$(pwd)" --source codex --backend mock
+python -m skillopt_sleep run --project "$(pwd)" --source codex --backend codex \
+  --max-sessions 5 --max-tasks 3 --progress
+python -m skillopt_sleep run --project "$(pwd)" --source codex --backend codex \
+  --target-skill-path .agents/skills/example/SKILL.md \
+  --max-sessions 5 --max-tasks 3 --progress
 ```
 
-Default backend is `mock` (no API spend). `--backend codex` uses your Codex
+`--source codex` reads Codex Desktop archived sessions from
-budget for real improvement. All the controllable knobs (`--gate on|off`,
+`~/.codex/archived_sessions`. Use `--codex-home /path/to/.codex` to point at a
-`--rollouts-k`, `--budget-tokens`, `--preferences`, optimizer/target split) work
+different Codex home, or `--source auto` to try Codex archives first and fall
-identically — see [`../../docs/sleep/CONTROLLABLE_DREAMING.md`](../../docs/sleep/CONTROLLABLE_DREAMING.md).
+back to Claude Code transcripts. Default backend is `mock` (no API spend).
+`--backend codex` uses your Codex budget for real improvement. Bound live runs
+with `--max-sessions` and `--max-tasks`; add `--progress` because Codex-backed
+mining, replay, and reflection can be slow and otherwise quiet. Use
+`--target-skill-path` to stage/adopt into a repo-scoped Codex skill such as
+`.agents/skills/<name>/SKILL.md`; target runs over-sample mined tasks and
+prefer tasks that match the target skill's path, headings, and content. All the
+controllable knobs (`--gate on|off`, `--rollouts-k`, `--budget-tokens`,
+`--preferences`, optimizer/target split) work identically — see
+[the SkillOpt-Sleep guide section](https://microsoft.github.io/SkillOpt/docs/guideline.html#sleep).
+
+For privacy-sensitive projects, split the run into reviewable steps:
+
+```bash
+python -m skillopt_sleep harvest --project "$(pwd)" --source codex \
+  --target-skill-path .agents/skills/example/SKILL.md \
+  --max-sessions 5 --max-tasks 3 \
+  --output reviewed-tasks.json
+
+python -m skillopt_sleep dry-run --project "$(pwd)" --backend codex \
+  --tasks-file reviewed-tasks.json --progress --json
+```
+
+Inspect/redact the JSON and set `"reviewed": true` before using a real backend.
+`--tasks-file` skips archive harvest/mining and replays only the reviewed JSON
+tasks; real backends refuse task files still marked `"reviewed": false`.
 
 ## Notes / status
 
 - Codex's `exec` runs shell, so the real-tool-loop replay (e.g. the
   `tool_called: search` benchmark seed) works natively.
-- Codex's standalone *plugin-package manifest* format is not yet a stable public
+- This integration no longer installs a `.codex/prompts` slash command. Skills
-  spec; this integration uses the documented `AGENTS.md` + skills + prompts
+  are the reusable Codex workflow surface; mention `skillopt-sleep` explicitly
-  mechanisms, which are stable. If/when a `codex plugin` package format ships,
+  or ask for a sleep/dream/offline self-improvement run and Codex can load the
-  we'll add a one-file manifest.
+  skill.

plugins/codex/install.sh

@@ -1,24 +1,30 @@
 #!/usr/bin/env bash
-# Install the SkillOpt-Sleep Codex integration into the user's ~/.codex and
+# Install the SkillOpt-Sleep Codex integration as a user-level Codex skill.
-# ~/.agents directories. Idempotent; prints what it does.
+# Idempotent; prints what it does.
 set -euo pipefail
 
 REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
 CODEX_HOME="${CODEX_HOME:-$HOME/.codex}"
 AGENTS_SKILLS="${HOME}/.agents/skills"
+LEGACY_PROMPT="$CODEX_HOME/prompts/sleep.md"
 
 echo "[install] repo: $REPO_ROOT"
 
-# 1) custom /sleep prompt
+# 1) user-level skill
-mkdir -p "$CODEX_HOME/prompts"
-cp "$REPO_ROOT/plugins/codex/prompts/sleep.md" "$CODEX_HOME/prompts/sleep.md"
-echo "[install] /sleep prompt   -> $CODEX_HOME/prompts/sleep.md"
-
-# 2) user-level skill
 mkdir -p "$AGENTS_SKILLS/skillopt-sleep"
 cp "$REPO_ROOT/plugins/codex/skills/skillopt-sleep/SKILL.md" "$AGENTS_SKILLS/skillopt-sleep/SKILL.md"
 echo "[install] skill           -> $AGENTS_SKILLS/skillopt-sleep/SKILL.md"
 
+# 2) retire the old custom prompt entrypoint from previous installs
+if [ -f "$LEGACY_PROMPT" ]; then
+  backup="${LEGACY_PROMPT}.skillopt-legacy.bak"
+  if [ -e "$backup" ]; then
+    backup="${LEGACY_PROMPT}.skillopt-legacy.$(date +%Y%m%d%H%M%S).bak"
+  fi
+  mv "$LEGACY_PROMPT" "$backup"
+  echo "[install] legacy prompt  -> $backup"
+fi
+
 # 3) record the repo location so the runner is found from anywhere
 echo "[install] add to your shell profile:"
 echo "    export SKILLOPT_SLEEP_REPO=\"$REPO_ROOT\""
@@ -29,8 +35,10 @@ cat <<EOF
 [install] Optional — add this to ~/.codex/AGENTS.md so Codex always knows the tool:
 
   ## SkillOpt-Sleep
-  An offline self-improvement cycle is available. To run it:
+  Use the skillopt-sleep skill when I ask to run a sleep/dream/offline
-  \`bash "$REPO_ROOT/plugins/run-sleep.sh" status\`. Use \`/sleep\` for the guided flow.
+  self-improvement cycle. The runner is:
+  \`bash "$REPO_ROOT/plugins/run-sleep.sh" status --project "\$(pwd)"\`.
 
-Done. Try:  /sleep status
+Done. Try asking Codex:
+  Use the skillopt-sleep skill to run status for this project.
 EOF

plugins/codex/prompts/sleep.md

@@ -1,21 +0,0 @@
-# /sleep — SkillOpt-Sleep for Codex
-#
-# Custom prompt: copy this file to ~/.codex/prompts/sleep.md and invoke with
-# `/sleep` in the Codex CLI. ($ARGUMENTS is the text after /sleep.)
-
-Run the SkillOpt-Sleep offline self-evolution cycle. Action: $ARGUMENTS
-(empty → "status").
-
-Use the bundled runner via shell:
-
-    bash "${SKILLOPT_SLEEP_REPO:?set SKILLOPT_SLEEP_REPO to the repo root}/plugins/run-sleep.sh" $ARGUMENTS --project "$(pwd)"
-
-Then:
-- For `run`/`dry-run`: read the staged `report.md` and show the held-out
-  baseline → candidate score and the proposed edits. `run` only stages a
-  proposal; nothing live changes until `adopt`.
-- For `adopt`: confirm which files were updated and that a backup was written.
-- Never edit the user's AGENTS.md / skills yourself; only `adopt` does that.
-
-Default backend is `mock` (no API spend). Add `--backend codex` for real
-improvement on the user's Codex budget.

plugins/codex/skills/skillopt-sleep/SKILL.md

@@ -1,49 +1,132 @@
 ---
 name: skillopt-sleep
-description: Nightly offline self-evolution for a Codex agent. Reviews past sessions, replays recurring tasks, and consolidates validated memory + skills behind a held-out gate. Use when the user wants Codex to learn from past usage, run a "sleep"/"dream" cycle, or schedule offline self-optimization.
+description: "Use when the user wants Codex to self-improve from past usage, asks about a nightly/offline 'sleep' or 'dream' cycle, wants Codex to review past sessions, learn preferences, consolidate memory/skills, run dry-run/run/adopt/status for SkillOpt-Sleep, or schedule offline self-optimization. Drives the skillopt_sleep engine: harvest past sessions -> mine recurring tasks -> replay offline -> consolidate validated memory + skills behind a held-out gate."
 ---
 
-# SkillOpt-Sleep (Codex skill)
+# SkillOpt-Sleep: offline self-evolution for a local Codex agent
 
-This skill drives the `skillopt_sleep` engine — an offline "sleep cycle" that
+SkillOpt-Sleep gives the user's Codex agent a sleep cycle. While the user is
-makes a Codex agent better at the user's recurring work without retraining.
+offline or on demand, it reviews past local sessions, re-runs recurring tasks
+on the user's own budget, and consolidates what it learns into memory and
+skills. It keeps only changes that pass a held-out validation gate, and live
+files change only after the user explicitly adopts a staged proposal. There is
+no model-weight training.
 
 ## When to use
 
-Trigger when the user wants to: review past sessions, learn their preferences,
+Trigger when the user wants any of:
-consolidate feedback into long-term memory/skills, run a nightly/offline
-self-improvement cycle, or adopt a staged proposal.
 
-## How to run it
+- Codex to learn from past sessions or get better the more they use it;
+- a nightly/scheduled or on-demand sleep/dream/offline self-improvement run;
+- to review past sessions and distill recurring tasks;
+- to consolidate feedback into memory or managed skills;
+- to run `status`, `harvest`, `dry-run`, `run`, or `adopt` for SkillOpt-Sleep.
+
+## The cycle
+
+1. **Harvest** - read local session transcripts according to the engine
+   configuration and normalize them into session digests.
+2. **Mine** - turn digests into recurring `TaskRecord`s with outcomes and
+   checkable references where possible.
+3. **Replay** - re-run mined tasks offline under the current skill and memory.
+4. **Consolidate** - reflect on failures and propose bounded edits.
+5. **Gate** - accept edits only when the held-out validation score improves.
+6. **Stage** - write the proposal under
+   `<project>/.skillopt-sleep/staging/<date>/`; nothing live changes.
+7. **Adopt** - only after explicit user approval, copy staged files over live
+   files with backups.
+
+## How to drive it
 
 Invoke the bundled runner via shell (Codex `exec` has shell access). The runner
-finds the engine and a Python ≥ 3.10 automatically:
+finds the engine and a Python >= 3.10 automatically.
 
 ```bash
 # point at the repo if it isn't auto-detected from CWD:
 export SKILLOPT_SLEEP_REPO=/path/to/SkillOpt-Sleep
-bash "$SKILLOPT_SLEEP_REPO/plugins/run-sleep.sh" <action> --project "$(pwd)"
+bash "$SKILLOPT_SLEEP_REPO/plugins/run-sleep.sh" status --project "$(pwd)"
+bash "$SKILLOPT_SLEEP_REPO/plugins/run-sleep.sh" harvest --project "$(pwd)"
+bash "$SKILLOPT_SLEEP_REPO/plugins/run-sleep.sh" dry-run --project "$(pwd)" --backend mock
+bash "$SKILLOPT_SLEEP_REPO/plugins/run-sleep.sh" run --project "$(pwd)" --backend codex
+bash "$SKILLOPT_SLEEP_REPO/plugins/run-sleep.sh" run --project "$(pwd)" --source codex  # harvest from Codex Desktop
+bash "$SKILLOPT_SLEEP_REPO/plugins/run-sleep.sh" adopt --project "$(pwd)"
 ```
 
-`<action>` ∈ `status | dry-run | run | adopt | harvest`. Use `--backend codex`
+Actions are `status`, `harvest`, `dry-run`, `run`, `adopt`, `schedule`, and `unschedule`.
-for real improvement on the user's own Codex budget (default `mock` = no spend).
+
+- Default backend is `mock`, which is deterministic and spends no API budget.
+- `--backend codex` uses the user's Codex budget for real improvement.
+- `--source codex` reads Codex Desktop archived sessions from `~/.codex/archived_sessions`;
+  use `--codex-home /path/to/.codex` if the archive lives elsewhere.
+- Keep `dry-run --backend mock` as the first smoke check unless the user
+  explicitly asked for a real optimization run.
+
+### Scheduling
+
+```bash
+bash "$SKILLOPT_SLEEP_REPO/plugins/run-sleep.sh" schedule --project "$(pwd)" --hour 3 --minute 17
+bash "$SKILLOPT_SLEEP_REPO/plugins/run-sleep.sh" unschedule --project "$(pwd)"
+```
+
+Installs a nightly cron entry. `unschedule --all` removes every managed entry.
+
+### All backends
+
+- `--backend mock` — deterministic, no API spend (default)
+- `--backend claude` — uses the Claude CLI
+- `--backend codex` — uses the Codex CLI
+- `--backend copilot` — uses the GitHub Copilot CLI
+
+### Additional flags
+
+| Flag | Description |
+|------|-------------|
+| `--auto-adopt` | Auto-adopt if the gate passes (default: stage only) |
+| `--edit-budget N` | Max bounded edits per night (default: 4) |
+| `--lookback-hours N` | Harvest window in hours (default: 72) |
+| `--json` | Machine-readable JSON output |
+
+### Config keys (`~/.skillopt-sleep/config.json`)
+
+- **`preferences`** — free-text house rules for the optimizer
+- **`gate_mode`** — `on` (validation-gated, default) or `off` (greedy)
+- **`gate_metric`** — `hard` | `soft` | `mixed` (default)
+- **`dream_rollouts`** — >1 for multi-rollout contrastive reflection
+- **`recall_k`** — >0 recalls similar past tasks from the archive
+
+### Memory consolidation
+
+The sleep cycle consolidates both **memory** (AGENTS.md / CLAUDE.md) and **skills** (SKILL.md) by default. Each is independently toggleable via `evolve_memory` / `evolve_skill` config keys. Both are gated by the same held-out validation score.
 
 ## Steps
 
 1. Run the requested action; capture stdout.
-2. For `run`/`dry-run`: read the staged `report.md` it prints and show the user
+2. For `dry-run` and `run`, report the held-out baseline -> candidate score,
-   the held-out baseline → candidate score and the exact proposed edits.
+   gate action, task count, session count, and exact proposed edits.
-3. `run` only **stages** a proposal under `<project>/.skillopt-sleep/staging/`;
+3. If a staging directory is printed, read `report.md` before summarizing.
-   nothing live changes until `adopt`. Offer `/sleep adopt`.
+4. `run` only stages a proposal; nothing live changes until `adopt`.
-4. Never hand-edit the user's `AGENTS.md` / skills yourself — only `adopt` does,
+5. Offer adoption only after the user has reviewed the staged proposal.
-   and it backs up first.
+6. Never hand-edit the user's `AGENTS.md`, memory, or skills as a substitute
+   for `adopt`; adoption is the safety boundary and writes backups first.
+
+## Hard rules
+
+- Harvest is read-only. Do not edit archived sessions or raw transcripts.
+- Keep raw secrets, credentials, private user data, and unsanitized transcript
+  contents out of messages, logs, generated artifacts, and commits.
+- Show validation evidence before recommending adoption.
+- Treat generated edits as proposals, not as source of truth.
+- Do not rely on deprecated custom prompts or `/sleep` slash commands for this
+  Codex integration. This skill is the entrypoint.
 
 ## Validate
 
 ```bash
+python -m skillopt_sleep dry-run --project "$(pwd)" --backend mock --json
 python -m skillopt_sleep.experiments.run_gbrain --backend codex \
   --seeds brief-writer --data-root /path/to/gbrain-evals/eval/data/skillopt-v1 \
   --nights 2 --limit-replay 3 --limit-holdout 3
 ```
-A deficient skill goes 0.00 → 1.00 on a held-out set; the optimizer's edits are
+
-gated on real-task performance.
+A deficient skill goes 0.00 -> 1.00 on a held-out set; the optimizer's edits
+are gated on real-task performance.

plugins/copilot/README.md

@@ -45,8 +45,17 @@ Ask Copilot things like *"run the sleep cycle"*, *"what did the last sleep
 propose?"*, *"adopt the staged sleep proposal"*. Copilot calls the MCP tools:
 `sleep_status`, `sleep_dry_run`, `sleep_run`, `sleep_adopt`, `sleep_harvest`.
 
-Each tool takes optional `project`, `backend` (`mock`/`claude`/`codex`), and
+Each tool takes optional `project`, `backend` (`mock`/`claude`/`codex`/`copilot`), and
-`scope` arguments. Default backend is `mock` (no API spend).
+`scope` arguments. Default backend is `mock` (no API spend). The `copilot`
+backend drives the GitHub Copilot CLI (`copilot -p ... --output-format json`)
+and requires the `copilot` CLI to be installed and authenticated.
+
+For speed, the `copilot` backend runs each call against an isolated
+`COPILOT_HOME` with built-in MCP servers and custom instructions disabled, so
+your user MCP servers (including this project's own) are not spawned per call
+(~5x faster). Override with `SKILLOPT_SLEEP_COPILOT_HOME=<dir>`, pick a model
+with `SKILLOPT_SLEEP_COPILOT_MODEL`, or set `SKILLOPT_SLEEP_COPILOT_FULL_ENV=1`
+to use your real Copilot environment instead.
 
 ## Verify the server directly (no Copilot needed)
 
@@ -64,4 +73,4 @@ You should see the server info and the five `sleep_*` tools.
   portable of the three integrations (one server → CLI + IDE).
 - The engine and all its controls (gate on/off, multi-rollout, budget,
   preferences, optimizer/target split) are identical across platforms — see
-  [`../../docs/sleep/CONTROLLABLE_DREAMING.md`](../../docs/sleep/CONTROLLABLE_DREAMING.md).
+  [the SkillOpt-Sleep guide section](https://microsoft.github.io/SkillOpt/docs/guideline.html#sleep).

plugins/copilot/copilot-instructions.snippet.md

@@ -19,6 +19,24 @@ my preferences", or "make the agent improve from past usage", use the MCP tools:
 - `sleep_run` — full cycle, stages a reviewed proposal (nothing live changes)
 - `sleep_adopt` — apply the staged proposal (backs up first)
 - `sleep_harvest` — list mined recurring tasks
+- `sleep_schedule` — install a nightly cron entry (set `hour`/`minute`)
+- `sleep_unschedule` — remove the nightly cron entry
+
+### Key parameters (pass as MCP tool arguments)
+
+- `backend` — `mock` (default, free), `claude`, `codex`, or `copilot`
+- `source` — `claude`, `codex`, or `auto` (where to read transcripts)
+- `target_skill_path` — explicit SKILL.md to evolve
+- `tasks_file` — pre-built TaskRecord JSON (skip harvest)
+- `max_tasks` / `max_sessions` — cap workload
+- `auto_adopt` — auto-adopt if the gate passes
+- `json` — machine-readable output for programmatic use
+
+### Advanced config (`~/.skillopt-sleep/config.json`)
+
+- `preferences` — free-text house rules for the optimizer
+- `gate_mode` — `on` (default) or `off`; `dream_rollouts` — >1 for more signal
+- `evolve_memory` / `evolve_skill` — toggle which docs consolidate
 
 Always show the user the held-out baseline → candidate score and the proposed
 edits before suggesting `sleep_adopt`. Never hand-edit the user's memory/skill

plugins/copilot/mcp_server.py

@@ -38,16 +38,48 @@ TOOLS = [
      "description": "Apply the latest staged proposal to CLAUDE.md/SKILL.md (backs up first)."},
     {"name": "sleep_harvest", "action": "harvest",
      "description": "Debug: list the recurring tasks mined from recent sessions."},
+    {"name": "sleep_schedule", "action": "schedule",
+     "description": "Install a nightly cron entry to run the sleep cycle automatically."},
+    {"name": "sleep_unschedule", "action": "unschedule",
+     "description": "Remove the nightly cron entry for a project."},
 ]
 _BY_NAME = {t["name"]: t for t in TOOLS}
 
 _TOOL_SCHEMA = {
     "type": "object",
     "properties": {
-        "project": {"type": "string", "description": "Project dir to evolve (default: cwd)."},
+        "project": {"type": "string",
-        "backend": {"type": "string", "enum": ["mock", "claude", "codex"],
+                     "description": "Project dir to evolve (default: cwd)."},
-                     "description": "mock = no API spend (default); claude/codex = real."},
+        "backend": {"type": "string", "enum": ["mock", "claude", "codex", "copilot"],
-        "scope": {"type": "string", "enum": ["invoked", "all"]},
+                     "description": "mock = no API spend (default); claude/codex/copilot = real."},
+        "scope": {"type": "string", "enum": ["invoked", "all"],
+                  "description": "Harvest scope (default: invoked project only)."},
+        "source": {"type": "string", "enum": ["claude", "codex", "auto"],
+                   "description": "Transcript source (default: claude)."},
+        "model": {"type": "string",
+                  "description": "Backend-specific model override."},
+        "tasks_file": {"type": "string",
+                       "description": "Path to reviewed TaskRecord JSON (skips harvest)."},
+        "target_skill_path": {"type": "string",
+                              "description": "Explicit SKILL.md path to evolve/stage/adopt."},
+        "progress": {"type": "boolean",
+                     "description": "Print phase progress to stderr."},
+        "max_sessions": {"type": "integer",
+                         "description": "Cap harvested sessions per run."},
+        "max_tasks": {"type": "integer",
+                      "description": "Cap mined tasks per run."},
+        "lookback_hours": {"type": "integer",
+                           "description": "Harvest window in hours (default: 72)."},
+        "auto_adopt": {"type": "boolean",
+                       "description": "Auto-adopt if gate passes (default: false)."},
+        "json": {"type": "boolean",
+                 "description": "Return machine-readable JSON output."},
+        "edit_budget": {"type": "integer",
+                        "description": "Max bounded edits per night (default: 4)."},
+        "hour": {"type": "integer",
+                 "description": "Hour for schedule (0-23, default: 3)."},
+        "minute": {"type": "integer",
+                   "description": "Minute for schedule (0-59, default: 17)."},
     },
     "additionalProperties": False,
 }
@@ -56,15 +88,35 @@ _TOOL_SCHEMA = {
 def _run_engine(action: str, args: dict) -> str:
     py = sys.executable or "python3"
     cmd = [py, "-m", "skillopt_sleep", action]
-    if args.get("project"):
+    # String-valued flags
-        cmd += ["--project", str(args["project"])]
+    for flag, key in [
-    if args.get("backend"):
+        ("--project", "project"), ("--backend", "backend"),
-        cmd += ["--backend", str(args["backend"])]
+        ("--scope", "scope"), ("--source", "source"),
-    if args.get("scope"):
+        ("--model", "model"), ("--tasks-file", "tasks_file"),
-        cmd += ["--scope", str(args["scope"])]
+        ("--target-skill-path", "target_skill_path"),
+    ]:
+        val = args.get(key)
+        if val:
+            cmd += [flag, str(val)]
+    # Integer-valued flags
+    for flag, key in [
+        ("--max-sessions", "max_sessions"), ("--max-tasks", "max_tasks"),
+        ("--lookback-hours", "lookback_hours"), ("--edit-budget", "edit_budget"),
+        ("--hour", "hour"), ("--minute", "minute"),
+    ]:
+        val = args.get(key)
+        if val is not None:
+            cmd += [flag, str(int(val))]
+    # Boolean flags
+    for flag, key in [
+        ("--progress", "progress"), ("--auto-adopt", "auto_adopt"),
+        ("--json", "json"),
+    ]:
+        if args.get(key):
+            cmd.append(flag)
     try:
         proc = subprocess.run(cmd, cwd=REPO_ROOT, capture_output=True, text=True, timeout=3600)
-    except Exception as e:  # noqa: BLE001
+    except Exception as e:
         return f"[error] failed to run engine: {e}"
     out = (proc.stdout or "").strip()
     err = (proc.stderr or "").strip()

plugins/copilot/skillopt/README.md

@@ -0,0 +1,98 @@
+# SkillOpt — GitHub Copilot integration
+
+Give **Copilot** (CLI or VS Code) direct access to the **SkillOpt** research
+engine via a tiny **MCP server**. MCP is GitHub's supported way to extend
+Copilot, so this works across Copilot CLI, VS Code, and other MCP clients with
+the same server.
+
+SkillOpt is **validation-gated, text-space skill optimization**: it reflects on
+rollouts, makes bounded edits to a skill, and keeps a change only if it improves
+a held-out validation set. This plugin exposes the repo's training and eval
+entry points (`scripts/train.py`, `scripts/eval_only.py`) as Copilot tools.
+
+> This is the companion to the **SkillOpt-Sleep** plugin (`../mcp_server.py`,
+> `sleep_*` tools). Sleep evolves a *local coding agent* from your past
+> sessions; this server drives the *research* training/eval loops on the
+> benchmark configs in [`../../../configs`](../../../configs).
+
+## What's here
+
+| File | Purpose |
+|---|---|
+| `mcp_server.py` | stdlib-only MCP (stdio) server exposing `skillopt_*` tools |
+| `mcp-config.example.json` | drop-in MCP server config |
+| `copilot-instructions.snippet.md` | paste into `.github/copilot-instructions.md` |
+
+## Install
+
+Requires Python ≥ 3.10. The MCP server itself is pure stdlib, but the tools it
+launches need SkillOpt's runtime deps — install the package first:
+
+```bash
+pip install -e .   # or: pip install -r requirements.txt
+```
+
+1. **Register the MCP server.** Add the server to your Copilot MCP config
+   (Copilot CLI: `~/.copilot/mcp-config.json`; VS Code: your MCP settings).
+   Use `mcp-config.example.json` as a template — set `SKILLOPT_REPO` to this
+   repo's path:
+
+   ```json
+   {
+     "mcpServers": {
+       "skillopt": {
+         "command": "python3",
+         "args": ["/abs/path/SkillOpt/plugins/copilot/skillopt/mcp_server.py"],
+         "env": { "SKILLOPT_REPO": "/abs/path/SkillOpt" }
+       }
+     }
+   }
+   ```
+
+2. **(Optional) Tell Copilot about it.** Append
+   `copilot-instructions.snippet.md` to your repo's
+   `.github/copilot-instructions.md` so Copilot reaches for the tools when the
+   user asks to "optimize a skill" or "train on a benchmark".
+
+## Use
+
+Ask Copilot things like *"what configs can I run?"*, *"optimize the searchqa
+skill"*, or *"evaluate this skill on the dataset"*. Copilot calls the MCP tools:
+`skillopt_list_configs`, `skillopt_train`, `skillopt_eval`.
+
+| Tool | Required args | Notes |
+|---|---|---|
+| `skillopt_list_configs` | — | Lists `configs/**/*.yaml` you can pass as `config`. |
+| `skillopt_train` | `config` | Runs a reflective optimization loop. Long-running; spends budget. |
+| `skillopt_eval` | `config`, `skill` | Evaluates one skill markdown file; no training. |
+
+Common optional args (both train and eval): `env`, `backend`,
+`optimizer_model`, `target_model`, `out_root`, `cfg_options` (space-separated
+`KEY=VALUE` YAML overrides), and `extra_args` (raw passthrough flags for the
+underlying script). `skillopt_train` also accepts `num_epochs`, `batch_size`,
+`seed`, and `use_gate`.
+
+Runs can be very long. The server's subprocess timeout defaults to 6 hours;
+override it with the `SKILLOPT_RUN_TIMEOUT` environment variable (seconds).
+
+## Verify the server directly (no Copilot needed)
+
+```bash
+printf '%s\n' \
+  '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{}}' \
+  '{"jsonrpc":"2.0","id":2,"method":"tools/list"}' \
+  '{"jsonrpc":"2.0","id":3,"method":"tools/call","params":{"name":"skillopt_list_configs","arguments":{}}}' \
+  | SKILLOPT_REPO="$(pwd)" python3 plugins/copilot/skillopt/mcp_server.py
+```
+
+You should see the server info, the three `skillopt_*` tools, and the list of
+benchmark configs.
+
+## Notes / status
+
+- MCP is the stable, official Copilot extension surface, so this is portable
+  across Copilot CLI and IDE from one server.
+- `skillopt_list_configs` is filesystem-only and safe to call anytime;
+  `skillopt_train` / `skillopt_eval` shell out to the repo scripts and require
+  the SkillOpt runtime deps (and, for real backends, model credentials — see
+  [`../../../.env.example`](../../../.env.example)).

plugins/copilot/skillopt/copilot-instructions.snippet.md

@@ -0,0 +1,33 @@
+<!--
+Copy this block into your repo's .github/copilot-instructions.md so Copilot
+knows the SkillOpt research-engine tools exist. (Copilot reads
+copilot-instructions.md automatically as ambient guidance.)
+-->
+
+## SkillOpt (research skill-optimization engine)
+
+This repo exposes the core **SkillOpt** training/eval engine via an MCP server
+(`skillopt`). SkillOpt is validation-gated, text-space skill optimization: it
+reflects on rollouts, makes bounded edits to a skill, and keeps a change only
+if it improves a held-out validation set.
+
+When the user asks to "optimize a skill", "train on <benchmark>", "run
+SkillOpt", "evaluate this skill", or "what configs can I run", use the MCP
+tools:
+
+- `skillopt_list_configs` — list the benchmark YAML configs you can pass as `config`
+- `skillopt_train` — run a reflective skill-optimization loop on a config (long-running; spends API/compute budget)
+- `skillopt_eval` — evaluate a single skill markdown file on a dataset (no training)
+
+Guidance:
+- Always run `skillopt_list_configs` first if you don't already know a valid `config` path.
+- `skillopt_train` and `skillopt_eval` are long-running and consume the user's
+  model backend/budget — confirm the `config`, `backend`, and model choices
+  with the user before launching, and surface the held-out gate result when the
+  run finishes.
+- For one-off YAML overrides use `cfg_options` (e.g. `seed=123 batch_size=40`);
+  for any other underlying flag use `extra_args`.
+
+This is distinct from the **SkillOpt-Sleep** MCP server (`skillopt-sleep`,
+`sleep_*` tools), which evolves a local coding agent from past sessions rather
+than running the research benchmarks.

plugins/copilot/skillopt/mcp-config.example.json

@@ -0,0 +1,11 @@
+{
+  "mcpServers": {
+    "skillopt": {
+      "command": "python3",
+      "args": ["plugins/copilot/skillopt/mcp_server.py"],
+      "env": {
+        "SKILLOPT_REPO": "${workspaceFolder}"
+      }
+    }
+  }
+}

plugins/copilot/skillopt/mcp_server.py

@@ -0,0 +1,229 @@
+#!/usr/bin/env python3
+"""SkillOpt (research engine) — minimal MCP server (stdio, stdlib-only).
+
+Exposes the core SkillOpt skill-optimization engine as MCP tools so any
+MCP-capable client (GitHub Copilot CLI / VS Code, Claude Desktop, etc.) can
+drive it. No third-party deps: speaks JSON-RPC 2.0 over stdio with just the
+handful of MCP methods clients need.
+
+This is the companion to the SkillOpt-Sleep MCP server (``../mcp_server.py``).
+Where Sleep evolves a *local agent* from past sessions, this server drives the
+*research* training/eval loops from this repo (``scripts/train.py`` /
+``scripts/eval_only.py``) against the benchmark configs in ``configs/``.
+
+Tools exposed:
+  - skillopt_list_configs : discover the benchmark YAML configs you can use
+  - skillopt_train        : run a reflective skill-optimization (training) loop
+  - skillopt_eval         : evaluate a single skill on a dataset (no training)
+
+``skillopt_train`` and ``skillopt_eval`` shell out to the repo's entry-point
+scripts and stream back their stdout/stderr. Configure your client to launch:
+  python plugins/copilot/skillopt/mcp_server.py
+"""
+from __future__ import annotations
+
+import glob
+import json
+import os
+import subprocess
+import sys
+
+# Repo root: three levels up from plugins/copilot/skillopt/mcp_server.py
+REPO_ROOT = os.environ.get("SKILLOPT_REPO") or os.path.abspath(
+    os.path.join(os.path.dirname(__file__), "..", "..", "..")
+)
+PROTOCOL_VERSION = "2024-11-05"
+
+# Training/eval runs are long; give the engine plenty of headroom.
+RUN_TIMEOUT_SECONDS = int(os.environ.get("SKILLOPT_RUN_TIMEOUT", "21600"))  # 6h
+
+
+def _list_configs() -> str:
+    """List the benchmark configs available under configs/ (filesystem only)."""
+    pattern = os.path.join(REPO_ROOT, "configs", "**", "*.yaml")
+    paths = sorted(glob.glob(pattern, recursive=True))
+    if not paths:
+        return f"[no configs found under {os.path.join(REPO_ROOT, 'configs')}]"
+    rels = [os.path.relpath(p, REPO_ROOT).replace(os.sep, "/") for p in paths]
+    lines = ["Available SkillOpt configs (pass as `config`):", ""]
+    lines += [f"  - {r}" for r in rels]
+    return "\n".join(lines)
+
+
+def _run_script(script_rel: str, args: dict, *, required: tuple[str, ...] = ()) -> str:
+    """Shell out to a repo entry-point script, mapping args -> --flags."""
+    for key in required:
+        if not args.get(key):
+            return f"[error] missing required argument: {key}"
+
+    py = sys.executable or "python3"
+    cmd = [py, os.path.join("scripts", script_rel)]
+
+    # Ordered flags that the train/eval scripts accept directly.
+    flag_args = (
+        "config", "skill", "split", "env", "backend",
+        "optimizer_model", "target_model", "out_root",
+        "num_epochs", "batch_size", "seed", "use_gate",
+    )
+    for key in flag_args:
+        val = args.get(key)
+        if val is None or val == "":
+            continue
+        cmd += [f"--{key}", str(val)]
+
+    # cfg-options: arbitrary KEY=VALUE YAML overrides (nargs="+").
+    cfg_options = args.get("cfg_options")
+    if cfg_options:
+        if isinstance(cfg_options, str):
+            cfg_options = cfg_options.split()
+        cmd += ["--cfg-options", *[str(x) for x in cfg_options]]
+
+    # extra_args: raw passthrough for any other train/eval flag.
+    extra = args.get("extra_args")
+    if extra:
+        if isinstance(extra, str):
+            extra = extra.split()
+        cmd += [str(x) for x in extra]
+
+    try:
+        proc = subprocess.run(
+            cmd, cwd=REPO_ROOT, capture_output=True, text=True,
+            timeout=RUN_TIMEOUT_SECONDS,
+        )
+    except subprocess.TimeoutExpired:
+        return f"[error] run exceeded {RUN_TIMEOUT_SECONDS}s timeout: {' '.join(cmd)}"
+    except Exception as e:  # noqa: BLE001
+        return f"[error] failed to run script: {e}"
+    out = (proc.stdout or "").strip()
+    err = (proc.stderr or "").strip()
+    body = out + (("\n[stderr]\n" + err) if err else "")
+    return body or f"[done] exit code {proc.returncode}, no output"
+
+
+TOOLS = [
+    {
+        "name": "skillopt_list_configs",
+        "description": "List the benchmark YAML configs under configs/ that can be passed as `config` to train/eval.",
+    },
+    {
+        "name": "skillopt_train",
+        "description": "Run a SkillOpt reflective skill-optimization (training) loop on a benchmark config. Long-running; uses your model backend/budget.",
+    },
+    {
+        "name": "skillopt_eval",
+        "description": "Evaluate a single skill markdown file on a dataset without training (scripts/eval_only.py).",
+    },
+]
+_BY_NAME = {t["name"]: t for t in TOOLS}
+
+_NO_ARGS_SCHEMA = {"type": "object", "properties": {}, "additionalProperties": False}
+
+_COMMON_PROPS = {
+    "config": {"type": "string",
+               "description": "Path to a benchmark YAML config (e.g. configs/searchqa/default.yaml). See skillopt_list_configs."},
+    "env": {"type": "string", "description": "Override the environment/adapter name (e.g. searchqa, alfworld)."},
+    "backend": {"type": "string", "description": "Model backend (e.g. azure_openai, claude, codex, qwen, minimax)."},
+    "optimizer_model": {"type": "string", "description": "Model used for reflection/skill rewriting (the optimizer)."},
+    "target_model": {"type": "string", "description": "Model used to execute tasks (the target)."},
+    "out_root": {"type": "string", "description": "Output directory root for run artifacts."},
+    "cfg_options": {"type": "string", "description": "Space-separated YAML overrides, e.g. 'seed=123 batch_size=40'."},
+    "extra_args": {"type": "string", "description": "Raw passthrough flags for the underlying script, e.g. '--workers 8 --max_turns 30'."},
+}
+
+_TRAIN_SCHEMA = {
+    "type": "object",
+    "properties": {
+        **_COMMON_PROPS,
+        "num_epochs": {"type": "integer", "description": "Number of optimization epochs."},
+        "batch_size": {"type": "integer", "description": "Tasks per optimization step."},
+        "seed": {"type": "integer", "description": "Random seed."},
+        "use_gate": {"type": "string", "enum": ["true", "false"],
+                     "description": "Whether to keep the held-out validation gate on (default on)."},
+    },
+    "required": ["config"],
+    "additionalProperties": False,
+}
+
+_EVAL_SCHEMA = {
+    "type": "object",
+    "properties": {
+        **_COMMON_PROPS,
+        "skill": {"type": "string", "description": "Path to the skill markdown file to evaluate."},
+        "split": {"type": "string", "description": "Dataset split to evaluate (default: all)."},
+    },
+    "required": ["config", "skill"],
+    "additionalProperties": False,
+}
+
+_SCHEMA_BY_NAME = {
+    "skillopt_list_configs": _NO_ARGS_SCHEMA,
+    "skillopt_train": _TRAIN_SCHEMA,
+    "skillopt_eval": _EVAL_SCHEMA,
+}
+
+
+def _result(id_, result):
+    return {"jsonrpc": "2.0", "id": id_, "result": result}
+
+
+def _error(id_, code, message):
+    return {"jsonrpc": "2.0", "id": id_, "error": {"code": code, "message": message}}
+
+
+def _dispatch(name: str, args: dict) -> str:
+    if name == "skillopt_list_configs":
+        return _list_configs()
+    if name == "skillopt_train":
+        return _run_script("train.py", args, required=("config",))
+    if name == "skillopt_eval":
+        return _run_script("eval_only.py", args, required=("config", "skill"))
+    return f"[error] unknown tool: {name}"
+
+
+def handle(req: dict):
+    method = req.get("method")
+    id_ = req.get("id")
+    if method == "initialize":
+        return _result(id_, {
+            "protocolVersion": PROTOCOL_VERSION,
+            "capabilities": {"tools": {}},
+            "serverInfo": {"name": "skillopt", "version": "0.1.0"},
+        })
+    if method in ("notifications/initialized", "initialized"):
+        return None  # notification, no response
+    if method == "tools/list":
+        return _result(id_, {"tools": [
+            {"name": t["name"], "description": t["description"],
+             "inputSchema": _SCHEMA_BY_NAME[t["name"]]}
+            for t in TOOLS
+        ]})
+    if method == "tools/call":
+        params = req.get("params") or {}
+        name = params.get("name")
+        if name not in _BY_NAME:
+            return _error(id_, -32602, f"unknown tool: {name}")
+        text = _dispatch(name, params.get("arguments") or {})
+        return _result(id_, {"content": [{"type": "text", "text": text}]})
+    if method == "ping":
+        return _result(id_, {})
+    return _error(id_, -32601, f"method not found: {method}")
+
+
+def main() -> int:
+    for line in sys.stdin:
+        line = line.strip()
+        if not line:
+            continue
+        try:
+            req = json.loads(line)
+        except Exception:
+            continue
+        resp = handle(req)
+        if resp is not None:
+            sys.stdout.write(json.dumps(resp) + "\n")
+            sys.stdout.flush()
+    return 0
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())

plugins/devin/README.md

@@ -0,0 +1,66 @@
+# SkillOpt-Sleep — Devin integration
+
+Give **Devin** (Cognition) a nightly **sleep cycle** via a tiny **MCP server**
+that exposes the `skillopt_sleep` engine as tools. MCP is Devin's supported way
+to add custom tooling, so this works in Devin's CLI and IDE.
+
+Devin doesn't write transcripts in the format the engine consumes, so this
+plugin adds a **Devin-specific harvester** that converts every locally available
+source into the Claude Code-compatible JSONL the engine reads.
+
+## What's here
+
+| File | Purpose |
+|---|---|
+| `mcp_server.py` | stdlib-only MCP (stdio) server exposing `sleep_*` tools |
+| `harvest_devin.py` | converts Devin ATIF-v1.7 transcripts + agentmemory + `.devin/skills` into JSONL, with `taskKey` + outcome envelopes |
+| `judge.py` | reference judge for the deferred/judge branch of the validation gate |
+| `mcp-config.example.json` | drop-in MCP server config |
+| `devin-rules.snippet.md` | paste into `.devin/rules/skillopt-sleep.md` |
+
+## What it harvests
+
+| Source | Where |
+|---|---|
+| Devin transcripts (ATIF-v1.7) | `~/.local/share/devin/cli/transcripts/*.json` |
+| agentmemory | `~/.agentmemory/standalone.json` |
+| Skill files | `.devin/skills/*/SKILL.md` |
+
+Workspaces are auto-detected from `~/.config/Devin/User/workspaceStorage/*/workspace.json`.
+After `sleep_adopt`, the evolved skill is synced to `.devin/skills/skillopt-sleep-learned/SKILL.md`.
+
+## Install
+
+Requires Python ≥ 3.10. No third-party packages — the server is pure stdlib.
+
+1. **Register the MCP server.** Use `mcp-config.example.json` as a template; set
+   `args` to the absolute path of this `mcp_server.py`. The engine is found
+   automatically (this plugin lives inside the SkillOpt repo). Or via the Devin
+   CLI:
+
+   ```bash
+   devin mcp add skillopt-sleep \
+     --env "SKILLOPT_DEVIN_CLAUDE_HOME=$HOME/.skillopt-sleep-devin" \
+     -- python3 /abs/path/to/SkillOpt/plugins/devin/mcp_server.py
+   ```
+
+2. **(Optional)** copy `devin-rules.snippet.md` to `.devin/rules/skillopt-sleep.md`
+   so Devin proactively offers the tools.
+
+3. Ask Devin: *"run the sleep cycle"*, *"what did the last sleep propose?"*, *"adopt it"*.
+
+## Tools
+
+| Tool | What it does |
+|---|---|
+| `sleep_status` | nights run so far + latest staged proposal |
+| `sleep_dry_run` | preview cycle — no staging, no changes |
+| `sleep_run` | full cycle; stages a proposal for review |
+| `sleep_adopt` | apply the staged proposal; syncs skill to the workspace |
+| `sleep_harvest` | debug: list the recurring tasks mined |
+| `sleep_schedule` | install a nightly cron entry (`--hour` / `--minute`) |
+| `sleep_unschedule` | remove the nightly cron entry |
+
+Default backend is `mock` (no API spend); `--backend claude|codex` uses your own
+budget. Same engine and `sleep_*` interface as the other plugins — all call
+`python -m skillopt_sleep`.

plugins/devin/devin-rules.snippet.md

@@ -0,0 +1,19 @@
+# SkillOpt-Sleep (Devin)
+
+You have access to a nightly self-evolution cycle via the `skillopt-sleep` MCP
+server. Use these tools to improve your long-term skills over time:
+
+- **`sleep_status`** — how many nights have run + the latest staged proposal
+- **`sleep_dry_run`** — preview a cycle without changing anything
+- **`sleep_run`** — run a full cycle; stages a proposal for review
+- **`sleep_adopt`** — apply the staged proposal to `.devin/skills/skillopt-sleep-learned/SKILL.md`
+- **`sleep_harvest`** — debug: list the recurring tasks mined from recent sessions
+- **`sleep_schedule`** / **`sleep_unschedule`** — install/remove a nightly cron run
+
+When a user asks about the sleep cycle, skill evolution, or improving your
+long-term memory, prefer calling these tools over explaining the concept.
+
+Default backend is `mock` (no API spend). Pass `backend: "claude"` or
+`backend: "codex"` with your own API key for real LLM-driven optimization.
+
+Place this file at `.devin/rules/skillopt-sleep.md` in your workspace.

plugins/devin/fixtures/devin_sample.json

@@ -0,0 +1,21 @@
+{
+  "schema_version": "ATIF-v1.7",
+  "session_id": "demo-001",
+  "steps": [
+    {
+      "source": "user",
+      "message": "Fix the failing NullPointerException in OrderService.persist() in the dutch-kis project",
+      "timestamp": "2026-06-20T10:00:00Z"
+    },
+    {
+      "source": "agent",
+      "message": "The repository call returns an Optional that is being unwrapped with .get(). I'll switch to orElseThrow(NotFoundException::new) so the missing-row case is handled.",
+      "timestamp": "2026-06-20T10:00:05Z"
+    },
+    {
+      "source": "agent",
+      "message": "Applied the fix and ran the suite: rtk mvn test -Dtest=OrderServiceTest -> BUILD SUCCESS, 142 passed, 0 failed.",
+      "timestamp": "2026-06-20T10:01:00Z"
+    }
+  ]
+}

plugins/devin/harvest_devin.py

@@ -0,0 +1,533 @@
+#!/usr/bin/env python3
+"""Convert Devin IDE local data into Claude Code-format JSONL transcripts.
+
+Devin (Cognition) does not persist agent conversation transcripts to disk in a
+format the sleep engine understands.  This script bridges that gap by synthesising
+JSONL files from every locally available source:
+
+  1. **Devin transcripts** (~/.local/share/devin/cli/transcripts/*.json)
+     Native ATIF-v1.7 format — source:"user" / source:"agent" messages
+     converted directly to user/assistant JSONL turns.
+
+  2. **agentmemory** (~/.agentmemory/standalone.json)
+     Memories saved by the `agentmemory` MCP server — each memory's title
+     becomes a synthetic user prompt; its content becomes the assistant reply.
+
+  3. **Skill files** (.devin/skills/*/SKILL.md)
+     Each skill description is converted to a session where the user asked
+     "use the <skill> skill" and the assistant described how to apply it.
+
+Output layout (mirrors ~/.claude/projects/<slug>/<sessionId>.jsonl):
+    <out_dir>/projects/<slug>/<session_id>.jsonl
+
+Workspace auto-detection order:
+  1. ``SKILLOPT_DEVIN_WORKSPACES`` env var — colon-separated abs paths
+  2. Devin registry: ``~/.config/Devin/User/workspaceStorage/*/workspace.json``
+  4. Working directory fallback
+
+Usage (standalone):
+    python harvest_devin.py [--out-dir PATH] [--workspaces PATH ...]
+"""
+from __future__ import annotations
+
+import argparse
+import hashlib
+import json
+import os
+import re
+import sys
+from datetime import datetime, timezone
+from pathlib import Path
+from typing import Any, Dict, List, Optional
+from urllib.parse import unquote, urlparse
+
+# ── cross-platform path resolution (Linux + Windows + macOS) ──────────────────
+#
+# Devin is a VS Code-family app, so its user-data dir moves with the OS:
+# Linux ~/.config/<App>, Windows %APPDATA%\<App>, macOS
+# ~/Library/Application Support/<App>.  Resolve all candidates and let callers
+# keep whichever actually exists.
+
+def _app_data_roots(app: str) -> List[str]:
+    """User-data dir candidates for a VS Code-family app, current OS first."""
+    home = os.path.expanduser("~")
+    roots: List[str] = []
+    if os.name == "nt":
+        appdata = os.environ.get("APPDATA") or os.path.join(home, "AppData", "Roaming")
+        roots.append(os.path.join(appdata, app))
+    elif sys.platform == "darwin":
+        roots.append(os.path.join(home, "Library", "Application Support", app))
+    # XDG / Linux (also a sensible fallback everywhere)
+    xdg = os.environ.get("XDG_CONFIG_HOME") or os.path.join(home, ".config")
+    roots.append(os.path.join(xdg, app))
+    # de-dupe, preserve order
+    return list(dict.fromkeys(roots))
+
+
+def _devin_transcript_candidates() -> List[str]:
+    """Where the Devin CLI may store ATIF transcripts, per OS."""
+    home = os.path.expanduser("~")
+    cands: List[str] = []
+    if os.name == "nt":
+        for base in (os.environ.get("LOCALAPPDATA"), os.environ.get("APPDATA")):
+            if base:
+                cands.append(os.path.join(base, "devin", "cli", "transcripts"))
+    elif sys.platform == "darwin":
+        cands.append(os.path.join(home, "Library", "Application Support",
+                                  "devin", "cli", "transcripts"))
+    cands.append(os.path.join(home, ".local", "share", "devin", "cli", "transcripts"))
+    return list(dict.fromkeys(cands))
+
+
+def _first_existing(paths: List[str]) -> str:
+    """First path that exists, else the first candidate (for nice messaging)."""
+    for p in paths:
+        if os.path.exists(p):
+            return p
+    return paths[0] if paths else ""
+
+
+def _uri_to_path(folder: str) -> str:
+    """Convert a VS Code ``file://`` workspace URI to a local path, cross-platform.
+
+    Linux:   file:///home/u/proj      -> /home/u/proj
+    Windows: file:///c%3A/Users/u/p   -> c:/Users/u/p
+    """
+    if not folder.startswith("file://"):
+        return folder
+    path = unquote(urlparse(folder).path)
+    # Windows drive paths come through as '/C:/...' — strip the leading slash.
+    if os.name == "nt" and re.match(r"^/[A-Za-z]:", path):
+        path = path[1:]
+    return path
+
+# ── workspace auto-detection ─────────────────────────────────────────────────
+
+def _workspaces_from_registry(storage_root: str) -> List[tuple]:
+    """Read VS Code-style workspaceStorage to get (mtime, path) pairs."""
+    results: List[tuple] = []
+    if not os.path.isdir(storage_root):
+        return results
+    for entry in os.scandir(storage_root):
+        ws_json = os.path.join(entry.path, "workspace.json")
+        if not os.path.isfile(ws_json):
+            continue
+        try:
+            with open(ws_json, encoding="utf-8") as f:
+                data = json.load(f)
+            folder = _uri_to_path(data.get("folder", ""))
+            if folder and os.path.isdir(folder):
+                results.append((os.path.getmtime(ws_json), folder))
+        except Exception:
+            continue
+    return results
+
+
+def _detect_workspaces() -> List[str]:
+    """Return known workspace paths (Devin registry), newest first."""
+    env_val = os.environ.get("SKILLOPT_DEVIN_WORKSPACES", "")
+    if env_val:
+        # os.pathsep so Windows 'C:\a;C:\b' splits correctly (not on the drive colon)
+        return [p for p in env_val.split(os.pathsep) if p and os.path.isdir(p)]
+
+    registries: List[str] = [
+        os.path.join(r, "User", "workspaceStorage")
+        for r in _app_data_roots("Devin")
+    ]
+
+    seen: set = set()
+    results: List[tuple] = []
+    for registry in registries:
+        for mtime, folder in _workspaces_from_registry(registry):
+            if folder not in seen:
+                seen.add(folder)
+                results.append((mtime, folder))
+    results.sort(reverse=True)
+    paths = [p for _, p in results]
+    return paths if paths else [os.getcwd()]
+
+# ── helpers ───────────────────────────────────────────────────────────────────
+
+def _slug(path: str) -> str:
+    """SHA-256 of abs-path, first 16 hex chars — matches Claude Code's scheme."""
+    return hashlib.sha256(os.path.abspath(path).encode()).hexdigest()[:16]
+
+
+def _iso(epoch_ms: Optional[float] = None) -> str:
+    dt = (datetime.fromtimestamp(epoch_ms / 1000.0, tz=timezone.utc)
+          if epoch_ms is not None else datetime.now(tz=timezone.utc))
+    return dt.strftime("%Y-%m-%dT%H:%M:%S.000Z")
+
+
+def _write_session(
+    out_dir: str, project: str, session_id: str,
+    user_prompts: List[str], assistant_replies: List[str],
+    timestamp_base_ms: float,
+    task_key: Optional[str] = None,
+) -> None:
+    slug = _slug(project)
+    session_dir = os.path.join(out_dir, "projects", slug)
+    os.makedirs(session_dir, exist_ok=True)
+    out_path = os.path.join(session_dir, f"{session_id}.jsonl")
+    ts = timestamp_base_ms
+    with open(out_path, "w", encoding="utf-8") as f:
+        for user_text, asst_text in zip(user_prompts, assistant_replies):
+            user_rec = {
+                "type": "user",
+                "message": {"role": "user", "content": user_text},
+                "cwd": project,
+                "timestamp": _iso(ts),
+                "sessionId": session_id,
+                "version": "1.0",
+            }
+            if task_key:
+                # grouping key so the miner can collapse repeats into one recurring task
+                user_rec["taskKey"] = task_key
+            f.write(json.dumps(user_rec, ensure_ascii=False) + "\n")
+            # space the reply >=5s after the prompt so a single-turn session
+            # isn't misclassified as a <3s headless replay and dropped by the
+            # engine's harvest filter (skillopt_sleep Issue #62).
+            ts += 5000
+            f.write(json.dumps({
+                "type": "assistant",
+                "message": {"role": "assistant", "content": asst_text},
+                "timestamp": _iso(ts),
+                "sessionId": session_id,
+                "version": "1.0",
+            }, ensure_ascii=False) + "\n")
+            ts += 2000
+
+
+def _append_history(out_dir: str, display: str, project: str, timestamp_ms: float) -> None:
+    record = {"display": display, "timestamp": timestamp_ms, "project": project}
+    with open(os.path.join(out_dir, "history.jsonl"), "a", encoding="utf-8") as f:
+        f.write(json.dumps(record, ensure_ascii=False) + "\n")
+
+
+def _infer_project(text: str, workspaces: List[str]) -> str:
+    for ws in workspaces:
+        if os.path.basename(ws.rstrip("/")).lower() in text.lower():
+            return ws
+    return workspaces[0] if workspaces else os.getcwd()
+
+# ── task identity + outcome extraction (fuel for the validation gate) ─────────
+#
+# SkillOpt's gate only works "where tasks recur and have a checkable correctness
+# signal."  These helpers add the two things a raw transcript lacks:
+#   * a stable taskKey so repeats collapse into one recurring task, and
+#   * an outcome envelope (success + verifier + re-runnable reference) so the
+#     held-out replay has something to score against.
+
+_LANG_HINTS = [
+    ("java",   r"(java|spring|maven|\bmvn\b|gradle|\.java\b|lombok)"),
+    ("python", r"(python|pytest|\bpip\b|\.py\b|django|flask)"),
+    ("ts",     r"(typescript|\.tsx?\b|\bnpm\b|jest|node)"),
+    ("js",     r"(javascript|\.jsx?\b)"),
+    ("sql",    r"(\bsql\b|select\s|mariadb|mysql|postgres|\.sql\b)"),
+    ("go",     r"(golang|\bgo test\b|\.go\b)"),
+    ("rust",   r"(rust|cargo|\.rs\b)"),
+]
+_INTENT_HINTS = [
+    ("fix",       r"(fix|bug|error|fail|npe|exception|broken|crash)"),
+    ("implement", r"(implement|add|create|build|introduce|support)"),
+    ("refactor",  r"(refactor|clean ?up|rename|extract|simplify)"),
+    ("test",      r"(test|coverage|assert)"),
+    ("review",    r"(review|audit|inspect)"),
+    ("optimize",  r"(optimi[sz]e|perf|speed up|slow)"),
+    ("explain",   r"(explain|understand|what does|how does)"),
+]
+_STOPWORDS = {"please", "this", "that", "with", "from", "into", "should",
+              "would", "code", "using", "the", "have"}
+
+
+def _normalize_task_key(text: str, project: str) -> str:
+    """Stable '<lang>:<intent>:<target>' grouping key for a task."""
+    low = text.lower()
+    lang = next((n for n, pat in _LANG_HINTS if re.search(pat, low)), "general")
+    intent = next((n for n, pat in _INTENT_HINTS if re.search(pat, low)), "task")
+    # target: prefer a CamelCase identifier, then a filename, then first real word
+    m = re.search(r"\b([A-Z][a-z0-9]+(?:[A-Z][a-z0-9]+)+)\b", text)  # CamelCase
+    if not m:
+        m = re.search(r"\b([\w-]+\.\w+)\b", text)                     # filename.ext
+    if m:
+        target = m.group(1)
+    else:
+        # first content word that isn't a stopword or an intent verb (e.g. "implement")
+        target = next((w for w in re.findall(r"[a-zA-Z]{4,}", low)
+                       if w not in _STOPWORDS
+                       and not any(re.search(pat, w) for _, pat in _INTENT_HINTS)),
+                      "general")
+    target = re.sub(r"[^a-zA-Z0-9]+", "-", target).strip("-").lower()[:40] or "general"
+    return f"{lang}:{intent}:{target}"
+
+
+_PASS_PAT = re.compile(
+    r"(build success|all tests? pass(?:ed)?|\b\d+ passed\b|\b0 failed\b|"
+    r"tests? pass(?:ed)?|✓|no errors)", re.IGNORECASE)
+_FAIL_PAT = re.compile(
+    r"(build failure|tests? failed|\b[1-9]\d* failed\b|error:|traceback|"
+    r"assertion ?error)", re.IGNORECASE)  # note: "0 failed" must NOT match
+_CMD_PAT = re.compile(
+    r"((?:rtk\s+)?(?:mvn|gradle|pytest|npm(?:\s+run)?\s+test|yarn\s+test|"
+    r"go\s+test|cargo\s+test)[^\n`]*)", re.IGNORECASE)
+
+
+def _detect_outcome(messages: List[str]) -> Optional[Dict[str, Any]]:
+    """Best-effort checkable signal from agent messages. None ⇒ no hard signal."""
+    blob = "\n".join(m for m in messages if m)
+    pass_hit, fail_hit = _PASS_PAT.search(blob), _FAIL_PAT.search(blob)
+    if not pass_hit and not fail_hit:
+        return None
+    verifier = "tests" if re.search(r"test|pytest", blob, re.IGNORECASE) else "build"
+    out: Dict[str, Any] = {
+        "success": bool(pass_hit) and not fail_hit,
+        "verifier": verifier,
+        "evidence": (pass_hit or fail_hit).group(0).strip(),
+    }
+    cmd = _CMD_PAT.search(blob)
+    if cmd:
+        # keep only the command itself, dropping any "-> result" / ": output" tail
+        repro = re.split(r"\s*(?:->|→|:|,)\s*", cmd.group(1))[0].strip()
+        out["reference"] = {"repro": repro}
+    return out
+
+
+def _build_rubric(user_prompt: str) -> List[str]:
+    """Derive checkable criteria from the task so a judge has something to score."""
+    crit: List[str] = []
+    ids = re.findall(r"\b([A-Z][a-z0-9]+(?:[A-Z][a-z0-9]+)+|[\w-]+\.\w+)\b", user_prompt)
+    for i in dict.fromkeys(ids):           # dedupe, preserve order
+        crit.append(f"Addresses {i}")
+    intent = _normalize_task_key(user_prompt, "").split(":")[1]
+    crit.append({
+        "fix":       "Resolves the reported defect without introducing new errors",
+        "implement": "Implements the requested behavior end to end",
+        "refactor":  "Preserves behavior while improving structure",
+        "test":      "Adds or fixes tests that actually exercise the change",
+        "optimize":  "Improves performance without changing results",
+    }.get(intent, "Satisfies the user's stated request"))
+    crit.append("Response is concrete and actionable, not a restatement of the task")
+    return crit[:5]
+
+
+def _judge_rubric_fallback(user_prompt: str) -> Dict[str, Any]:
+    """When no hard signal exists, attach a rubric and mark the task for judge
+    scoring. success=None tells the gate to defer/judge rather than trust it.
+    The actual scoring is done by judge.py (or the engine) at replay time."""
+    return {
+        "success": None,
+        "verifier": "judge",
+        "rubric": _build_rubric(user_prompt or ""),
+    }
+
+
+def _write_outcome(out_dir: str, session_id: str, task_key: str, project: str,
+                   ts_ms: float, outcome: Dict[str, Any]) -> None:
+    rec = {"type": "outcome", "sessionId": session_id, "taskKey": task_key,
+           "project": project, "timestamp": _iso(ts_ms), **outcome}
+    with open(os.path.join(out_dir, "outcomes.jsonl"), "a", encoding="utf-8") as f:
+        f.write(json.dumps(rec, ensure_ascii=False) + "\n")
+
+# ── source 1: Devin ATIF-v1.7 transcripts ────────────────────────────────────
+
+def harvest_devin_transcripts(
+    transcripts_dir: str, out_dir: str, workspaces: List[str]
+) -> int:
+    """Convert Devin CLI ATIF-v1.7 transcripts to Claude Code JSONL."""
+    if not os.path.isdir(transcripts_dir):
+        return 0
+    written = 0
+    for entry in os.scandir(transcripts_dir):
+        if not entry.name.endswith(".json"):
+            continue
+        try:
+            with open(entry.path, encoding="utf-8") as f:
+                data = json.load(f)
+        except Exception:
+            continue
+        if data.get("schema_version", "").startswith("ATIF"):
+            pass  # Devin native format
+        else:
+            continue
+        session_id = data.get("session_id") or entry.name[:-5]
+        steps = data.get("steps") or []
+        user_prompts: List[str] = []
+        agent_replies: List[str] = []
+        project = ""
+        ts_base: Optional[float] = None
+        for step in steps:
+            src = step.get("source", "")
+            msg = str(step.get("message") or "").strip()
+            if not msg or src == "system":
+                continue
+            if src == "user":
+                user_prompts.append(msg)
+                if not project:
+                    project = _infer_project(msg, workspaces)
+            elif src == "agent":
+                agent_replies.append(msg)
+            if ts_base is None:
+                raw_ts = step.get("timestamp", "")
+                if raw_ts:
+                    try:
+                        from datetime import datetime as _dt
+                        ts_base = _dt.fromisoformat(
+                            raw_ts.replace("Z", "+00:00")
+                        ).timestamp() * 1000
+                    except Exception:
+                        pass
+        if not user_prompts:
+            continue
+        if not project:
+            project = workspaces[0] if workspaces else os.getcwd()
+        if ts_base is None:
+            ts_base = datetime.now(tz=timezone.utc).timestamp() * 1000
+        # Identity + outcome: what makes this trajectory replayable & gradeable.
+        task_key = _normalize_task_key(user_prompts[0], project)
+        outcome = _detect_outcome(agent_replies) or _judge_rubric_fallback(user_prompts[0])
+        # Pair turns; pad shorter list
+        n = max(len(user_prompts), len(agent_replies))
+        user_prompts += [""] * (n - len(user_prompts))
+        agent_replies += [""] * (n - len(agent_replies))
+        sid = f"devin_{session_id}"
+        _write_session(
+            out_dir, project, sid,
+            user_prompts=[p for p in user_prompts if p],
+            assistant_replies=[r if r else "[no reply recorded]" for r, p in
+                               zip(agent_replies, user_prompts) if p],
+            timestamp_base_ms=ts_base,
+            task_key=task_key,
+        )
+        _write_outcome(out_dir, sid, task_key, project, ts_base, outcome)
+        _append_history(
+            out_dir,
+            display=(user_prompts[0] or session_id)[:120],
+            project=project,
+            timestamp_ms=ts_base,
+        )
+        written += 1
+    return written
+
+
+# ── source 2: agentmemory ─────────────────────────────────────────────────────
+
+def harvest_agentmemory(agentmemory_path: str, out_dir: str,
+                        workspaces: List[str]) -> int:
+    if not os.path.isfile(agentmemory_path):
+        return 0
+    with open(agentmemory_path, encoding="utf-8") as f:
+        data = json.load(f)
+    memories: Dict[str, Any] = data.get("mem:memories", {})
+    written = 0
+    base_ts = datetime.now(tz=timezone.utc).timestamp() * 1000 - len(memories) * 60_000
+    for i, (mem_id, mem) in enumerate(memories.items()):
+        title = str(mem.get("title", "")).strip()
+        content = str(mem.get("content", "")).strip()
+        if not title or not content:
+            continue
+        project = _infer_project(title + " " + content, workspaces)
+        ts = base_ts + i * 60_000
+        _write_session(out_dir, project, mem_id,
+                       user_prompts=[title],
+                       assistant_replies=[content],
+                       timestamp_base_ms=ts)
+        _append_history(out_dir, display=title[:120], project=project, timestamp_ms=ts)
+        written += 1
+    return written
+
+# ── source 3: skill files (.devin/skills) ─────────────────────────────────────
+
+def harvest_skills(workspaces: List[str], out_dir: str) -> int:
+    written = 0
+    seen_ids: set = set()
+    for ws in workspaces:
+        skills_root = os.path.join(ws, ".devin", "skills")
+        if not os.path.isdir(skills_root):
+            continue
+        for skill_dir in os.scandir(skills_root):
+            if not skill_dir.is_dir():
+                continue
+            skill_md = os.path.join(skill_dir.path, "SKILL.md")
+            if not os.path.isfile(skill_md):
+                continue
+            sid = f"skill_{skill_dir.name}"
+            if sid in seen_ids:
+                continue
+            seen_ids.add(sid)
+            with open(skill_md, encoding="utf-8") as f:
+                raw = f.read()
+            body = re.sub(r"^---.*?---\s*", "", raw, flags=re.DOTALL).strip()
+            if not body:
+                continue
+            first_line = body.split("\n")[0].lstrip("# ").strip()
+            user_ask = f"Please use the {skill_dir.name} skill: {first_line}"
+            ts = datetime.now(tz=timezone.utc).timestamp() * 1000 - 3_600_000
+            _write_session(out_dir, ws, sid,
+                           user_prompts=[user_ask],
+                           assistant_replies=[body[:1200]],
+                           timestamp_base_ms=ts)
+            _append_history(out_dir, display=user_ask[:120], project=ws, timestamp_ms=ts)
+            written += 1
+    return written
+
+# ── main ─────────────────────────────────────────────────────────────────────
+
+def main(argv=None) -> int:
+    parser = argparse.ArgumentParser(
+        description="Generate SkillOpt-Sleep transcripts from Devin local data"
+    )
+    parser.add_argument(
+        "--out-dir",
+        default=os.path.expanduser("~/.skillopt-sleep-devin"),
+        help="Output claude_home dir (default: ~/.skillopt-sleep-devin)",
+    )
+    parser.add_argument(
+        "--agentmemory",
+        default=os.path.expanduser("~/.agentmemory/standalone.json"),
+        help="Path to agentmemory standalone.json",
+    )
+    parser.add_argument(
+        "--devin-transcripts",
+        default=_first_existing(_devin_transcript_candidates()),
+        help="Devin CLI ATIF transcripts directory (default: per-OS auto-detect)",
+    )
+    parser.add_argument(
+        "--workspaces", nargs="*",
+        help="Workspace paths (default: auto-detect from Devin registry)",
+    )
+    parser.add_argument("--quiet", action="store_true")
+    args = parser.parse_args(argv)
+
+    out_dir = os.path.expanduser(args.out_dir)
+    os.makedirs(out_dir, exist_ok=True)
+    os.makedirs(os.path.join(out_dir, "projects"), exist_ok=True)
+
+    workspaces = args.workspaces or _detect_workspaces()
+    workspaces = [ws for ws in workspaces if os.path.isdir(ws)]
+    if not workspaces:
+        workspaces = [os.getcwd()]
+
+    total = 0
+    devin_transcripts = os.path.expanduser(args.devin_transcripts)
+    n = harvest_devin_transcripts(devin_transcripts, out_dir, workspaces)
+    if not args.quiet:
+        print(f"[harvest_devin] devin        : {n} sessions")
+    total += n
+
+    n = harvest_agentmemory(args.agentmemory, out_dir, workspaces)
+    if not args.quiet:
+        print(f"[harvest_devin] agentmemory  : {n} sessions")
+    total += n
+
+    n = harvest_skills(workspaces, out_dir)
+    if not args.quiet:
+        print(f"[harvest_devin] skill files  : {n} sessions")
+    total += n
+
+    if not args.quiet:
+        print(f"[harvest_devin] total        : {total} synthetic sessions → {out_dir}")
+    return 0
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())

plugins/devin/judge.py

@@ -0,0 +1,129 @@
+#!/usr/bin/env python3
+"""Reference judge for SkillOpt-Sleep — score a candidate reply against a rubric.
+
+Tasks harvested without a hard test/build signal get ``verifier: "judge"`` and a
+``rubric`` (see ``_build_rubric`` in harvest_devin.py).  This module is the
+scorer the validation gate calls for those tasks: given the rubric and a
+candidate reply produced during replay, it returns a score in ``[0, 1]``.  The
+gate accepts a skill edit only if the *new* skill scores strictly higher on the
+held-out tasks.
+
+It is self-contained on purpose — in a full deployment the SkillOpt engine owns
+replay+scoring, but having a runnable reference here lets you sanity-check the
+judge path without the engine.
+
+Backends (select via ``SKILLOPT_JUDGE``):
+  * ``heuristic`` (default) — keyword-coverage, offline, no API key, deterministic.
+  * ``claude``              — LLM judge via the Anthropic API (needs ANTHROPIC_API_KEY).
+
+Usage:
+    python judge.py --rubric rubric.json --reply reply.txt
+    echo "<reply>" | python judge.py --rubric-inline '["Addresses OrderService", ...]'
+"""
+from __future__ import annotations
+
+import argparse
+import json
+import os
+import re
+import sys
+from typing import List
+
+_STOPWORDS = {"addresses", "resolves", "implements", "without", "introducing",
+              "behavior", "request", "response", "concrete", "actionable", "not",
+              "the", "and", "that", "with", "stated", "reported", "actually",
+              "preserves", "improving", "structure", "requested", "satisfies"}
+
+# Cheap, fast model is the right default for a judge.
+_JUDGE_MODEL = os.environ.get("SKILLOPT_JUDGE_MODEL", "claude-haiku-4-5-20251001")
+
+
+def _content_words(text: str) -> List[str]:
+    return [w for w in re.findall(r"[A-Za-z][A-Za-z0-9_.\-]{3,}", text.lower())
+            if w not in _STOPWORDS]
+
+
+def heuristic_score(reply: str, rubric: List[str]) -> float:
+    """Fraction of rubric criteria whose key content words appear in the reply.
+
+    Crude but deterministic: each criterion is 'met' if at least one of its
+    content words shows up in the candidate reply. Good enough to smoke-test the
+    gate wiring; swap in the claude backend for real judging.
+    """
+    if not rubric:
+        return 0.0
+    low = reply.lower()
+    met = 0
+    for criterion in rubric:
+        words = _content_words(criterion)
+        if not words:                       # nothing to check → treat as met
+            met += 1
+            continue
+        if any(w in low for w in words):
+            met += 1
+    return round(met / len(rubric), 3)
+
+
+def claude_score(reply: str, rubric: List[str]) -> float:
+    """LLM judge via the Anthropic API. Returns a 0..1 score.
+
+    Stdlib-only (urllib) so this file stays dependency-free. Falls back to the
+    heuristic if the key is missing or the call fails, so the gate never hard-errors.
+    """
+    api_key = os.environ.get("ANTHROPIC_API_KEY")
+    if not api_key:
+        print("[judge] ANTHROPIC_API_KEY unset — using heuristic", file=sys.stderr)
+        return heuristic_score(reply, rubric)
+    import urllib.request
+
+    rubric_block = "\n".join(f"- {c}" for c in rubric)
+    prompt = (
+        "You are scoring an AI agent's reply against a rubric. For each criterion, "
+        "decide if the reply satisfies it. Respond with ONLY a number between 0 and "
+        "1 — the fraction of criteria satisfied.\n\n"
+        f"Rubric:\n{rubric_block}\n\nReply:\n{reply}\n\nScore:"
+    )
+    body = json.dumps({
+        "model": _JUDGE_MODEL,
+        "max_tokens": 8,
+        "messages": [{"role": "user", "content": prompt}],
+    }).encode()
+    req = urllib.request.Request(
+        "https://api.anthropic.com/v1/messages", data=body,
+        headers={"content-type": "application/json", "x-api-key": api_key,
+                 "anthropic-version": "2023-06-01"},
+    )
+    try:
+        with urllib.request.urlopen(req, timeout=30) as resp:
+            data = json.load(resp)
+        text = "".join(b.get("text", "") for b in data.get("content", []))
+        m = re.search(r"[01](?:\.\d+)?", text)
+        return max(0.0, min(1.0, float(m.group(0)))) if m else heuristic_score(reply, rubric)
+    except Exception as exc:                 # network/auth/parse — degrade gracefully
+        print(f"[judge] claude backend failed ({exc}) — using heuristic", file=sys.stderr)
+        return heuristic_score(reply, rubric)
+
+
+def score(reply: str, rubric: List[str]) -> float:
+    backend = os.environ.get("SKILLOPT_JUDGE", "heuristic")
+    return claude_score(reply, rubric) if backend == "claude" else heuristic_score(reply, rubric)
+
+
+def main(argv=None) -> int:
+    p = argparse.ArgumentParser(description="Score a reply against a rubric (0..1)")
+    g = p.add_mutually_exclusive_group(required=True)
+    g.add_argument("--rubric", help="Path to a JSON file containing a list of criteria")
+    g.add_argument("--rubric-inline", help="Inline JSON list of criteria")
+    p.add_argument("--reply", help="Path to the reply text (default: stdin)")
+    args = p.parse_args(argv)
+
+    rubric = (json.load(open(args.rubric, encoding="utf-8")) if args.rubric
+              else json.loads(args.rubric_inline))
+    reply = (open(args.reply, encoding="utf-8").read() if args.reply
+             else sys.stdin.read())
+    print(score(reply, rubric))
+    return 0
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())

plugins/devin/mcp-config.example.json

@@ -0,0 +1,11 @@
+{
+  "mcpServers": {
+    "skillopt-sleep": {
+      "command": "python3",
+      "args": ["/abs/path/to/SkillOpt/plugins/devin/mcp_server.py"],
+      "env": {
+        "SKILLOPT_DEVIN_CLAUDE_HOME": "~/.skillopt-sleep-devin"
+      }
+    }
+  }
+}

plugins/devin/mcp_server.py

@@ -0,0 +1,240 @@
+#!/usr/bin/env python3
+"""SkillOpt-Sleep — Devin MCP server (stdio, stdlib-only).
+
+Exposes the sleep engine as MCP tools so Devin (Cognition) can drive it. No
+third-party deps: speaks JSON-RPC 2.0 over stdio with just the handful of MCP
+methods clients need. Same `sleep_*` interface and engine flags as
+`plugins/copilot`, plus a Devin-specific harvest step.
+
+Before each data-reading action this server runs `harvest_devin.py` to convert
+locally available Devin data (ATIF-v1.7 transcripts, agentmemory memories, and
+.devin skill files) into the Claude Code-compatible JSONL the engine consumes,
+writing it under SKILLOPT_DEVIN_CLAUDE_HOME and pointing the engine there with
+`--claude-home`. After `sleep_adopt` the evolved skill is synced back into the
+workspace's `.devin/skills/`.
+
+Tools: sleep_status, sleep_dry_run, sleep_run, sleep_adopt, sleep_harvest,
+sleep_schedule, sleep_unschedule. Each shells out to
+`python -m skillopt_sleep <action> ...`. Configure Devin to launch:
+  python plugins/devin/mcp_server.py
+"""
+from __future__ import annotations
+
+import json
+import os
+import shutil
+import subprocess
+import sys
+
+# expanduser wraps the whole value so a "~/..." env var is expanded too (not
+# just a default) — otherwise a literal ~ dir gets created.
+REPO_ROOT = os.path.expanduser(
+    os.environ.get("SKILLOPT_SLEEP_REPO")
+    or os.path.abspath(os.path.join(os.path.dirname(__file__), "..", ".."))
+)
+PLUGIN_DIR = os.path.dirname(os.path.abspath(__file__))
+CLAUDE_HOME = os.path.expanduser(
+    os.environ.get("SKILLOPT_DEVIN_CLAUDE_HOME", "~/.skillopt-sleep-devin")
+)
+MANAGED_SKILL_NAME = os.environ.get("SKILLOPT_MANAGED_SKILL", "skillopt-sleep-learned")
+PROTOCOL_VERSION = "2024-11-05"
+
+TOOLS = [
+    {"name": "sleep_status", "action": "status",
+     "description": "Show how many SkillOpt-Sleep nights have run and the latest staged proposal."},
+    {"name": "sleep_dry_run", "action": "dry-run",
+     "description": "Preview a sleep cycle (harvest+mine+replay) without staging anything."},
+    {"name": "sleep_run", "action": "run",
+     "description": "Run a full sleep cycle; stages a reviewed proposal. Nothing live changes until adopt."},
+    {"name": "sleep_adopt", "action": "adopt",
+     "description": "Apply the latest staged proposal to the managed SKILL.md and sync it into .devin/skills/."},
+    {"name": "sleep_harvest", "action": "harvest",
+     "description": "Debug: list the recurring tasks mined from recent Devin sessions."},
+    {"name": "sleep_schedule", "action": "schedule",
+     "description": "Install a nightly cron entry to run the sleep cycle automatically."},
+    {"name": "sleep_unschedule", "action": "unschedule",
+     "description": "Remove the nightly cron entry for a project."},
+]
+_BY_NAME = {t["name"]: t for t in TOOLS}
+
+_TOOL_SCHEMA = {
+    "type": "object",
+    "properties": {
+        "project": {"type": "string",
+                    "description": "Project dir to evolve (default: cwd)."},
+        "backend": {"type": "string", "enum": ["mock", "claude", "codex", "copilot"],
+                    "description": "mock = no API spend (default); claude/codex/copilot = real."},
+        "scope": {"type": "string", "enum": ["invoked", "all"],
+                  "description": "Harvest scope (default: invoked project only)."},
+        "source": {"type": "string", "enum": ["claude", "codex", "auto"],
+                   "description": "Transcript source (default: claude)."},
+        "model": {"type": "string",
+                  "description": "Backend-specific model override."},
+        "tasks_file": {"type": "string",
+                       "description": "Path to reviewed TaskRecord JSON (skips harvest)."},
+        "target_skill_path": {"type": "string",
+                              "description": "Explicit SKILL.md path to evolve/stage/adopt."},
+        "progress": {"type": "boolean",
+                     "description": "Print phase progress to stderr."},
+        "max_sessions": {"type": "integer",
+                         "description": "Cap harvested sessions per run."},
+        "max_tasks": {"type": "integer",
+                      "description": "Cap mined tasks per run."},
+        "lookback_hours": {"type": "integer",
+                           "description": "Harvest window in hours (default: 72)."},
+        "auto_adopt": {"type": "boolean",
+                       "description": "Auto-adopt if gate passes (default: false)."},
+        "json": {"type": "boolean",
+                 "description": "Return machine-readable JSON output."},
+        "edit_budget": {"type": "integer",
+                        "description": "Max bounded edits per night (default: 4)."},
+        "hour": {"type": "integer",
+                 "description": "Hour for schedule (0-23, default: 3)."},
+        "minute": {"type": "integer",
+                   "description": "Minute for schedule (0-59, default: 17)."},
+    },
+    "additionalProperties": False,
+}
+
+# actions that read harvested Devin data (schedule/unschedule/adopt don't)
+_HARVEST_ACTIONS = {"status", "dry-run", "run", "harvest"}
+
+
+def _run_harvest() -> str:
+    """Convert local Devin data into the JSONL the engine reads, under CLAUDE_HOME."""
+    harvester = os.path.join(PLUGIN_DIR, "harvest_devin.py")
+    env = dict(os.environ)
+    env["PYTHONPATH"] = REPO_ROOT + os.pathsep + env.get("PYTHONPATH", "")
+    try:
+        proc = subprocess.run(
+            [sys.executable, harvester, "--out-dir", CLAUDE_HOME],
+            capture_output=True, text=True, timeout=60, env=env,
+        )
+        out = (proc.stdout or "").strip()
+        err = (proc.stderr or "").strip()
+        return out + (("\n[harvest stderr]\n" + err) if err else "")
+    except Exception as exc:
+        return f"[harvest_devin] warning: {exc}"
+
+
+def _sync_skill(project: str) -> str:
+    """After adopt, copy the evolved skill into the workspace's .devin/skills/."""
+    src = os.path.join(CLAUDE_HOME, "skills", MANAGED_SKILL_NAME, "SKILL.md")
+    if not (os.path.isfile(src) and project and os.path.isdir(project)):
+        return ""
+    dot_root = os.path.join(project, ".devin")
+    if not os.path.isdir(dot_root):
+        return ""
+    dst_dir = os.path.join(dot_root, "skills", MANAGED_SKILL_NAME)
+    os.makedirs(dst_dir, exist_ok=True)
+    dst = os.path.join(dst_dir, "SKILL.md")
+    shutil.copy2(src, dst)
+    return f"\n[sleep] synced evolved skill → {dst}"
+
+
+def _run_engine(action: str, args: dict) -> str:
+    harvest_out = _run_harvest() if action in _HARVEST_ACTIONS else ""
+
+    py = sys.executable or "python3"
+    cmd = [py, "-m", "skillopt_sleep", action, "--claude-home", CLAUDE_HOME]
+    # Devin transcripts are converted to the Claude format, so default source=claude
+    if not args.get("source"):
+        cmd += ["--source", "claude"]
+    # String-valued flags
+    for flag, key in [
+        ("--project", "project"), ("--backend", "backend"),
+        ("--scope", "scope"), ("--source", "source"),
+        ("--model", "model"), ("--tasks-file", "tasks_file"),
+        ("--target-skill-path", "target_skill_path"),
+    ]:
+        val = args.get(key)
+        if val:
+            cmd += [flag, str(val)]
+    # Integer-valued flags
+    for flag, key in [
+        ("--max-sessions", "max_sessions"), ("--max-tasks", "max_tasks"),
+        ("--lookback-hours", "lookback_hours"), ("--edit-budget", "edit_budget"),
+        ("--hour", "hour"), ("--minute", "minute"),
+    ]:
+        val = args.get(key)
+        if val is not None:
+            cmd += [flag, str(int(val))]
+    # Boolean flags
+    for flag, key in [
+        ("--progress", "progress"), ("--auto-adopt", "auto_adopt"), ("--json", "json"),
+    ]:
+        if args.get(key):
+            cmd.append(flag)
+
+    env = dict(os.environ)
+    env["PYTHONPATH"] = REPO_ROOT + os.pathsep + env.get("PYTHONPATH", "")
+    try:
+        proc = subprocess.run(cmd, cwd=REPO_ROOT, capture_output=True,
+                              text=True, timeout=3600, env=env)
+    except Exception as e:
+        return f"[harvest]\n{harvest_out}\n[error] failed to run engine: {e}"
+    out = (proc.stdout or "").strip()
+    err = (proc.stderr or "").strip()
+    result = (f"[harvest]\n{harvest_out}\n\n" if harvest_out else "") + f"[engine]\n{out}"
+    if err:
+        result += f"\n[stderr]\n{err}"
+    if action == "adopt":
+        result += _sync_skill(args.get("project") or os.getcwd())
+    return result
+
+
+def _result(id_, result):
+    return {"jsonrpc": "2.0", "id": id_, "result": result}
+
+
+def _error(id_, code, message):
+    return {"jsonrpc": "2.0", "id": id_, "error": {"code": code, "message": message}}
+
+
+def handle(req: dict):
+    method = req.get("method")
+    id_ = req.get("id")
+    if method == "initialize":
+        return _result(id_, {
+            "protocolVersion": PROTOCOL_VERSION,
+            "capabilities": {"tools": {}},
+            "serverInfo": {"name": "skillopt-sleep", "version": "0.1.0"},
+        })
+    if method in ("notifications/initialized", "initialized"):
+        return None
+    if method == "tools/list":
+        return _result(id_, {"tools": [
+            {"name": t["name"], "description": t["description"], "inputSchema": _TOOL_SCHEMA}
+            for t in TOOLS
+        ]})
+    if method == "tools/call":
+        params = req.get("params") or {}
+        name = params.get("name")
+        tool = _BY_NAME.get(name)
+        if not tool:
+            return _error(id_, -32602, f"unknown tool: {name}")
+        text = _run_engine(tool["action"], params.get("arguments") or {})
+        return _result(id_, {"content": [{"type": "text", "text": text}]})
+    if method == "ping":
+        return _result(id_, {})
+    return _error(id_, -32601, f"method not found: {method}")
+
+
+def main() -> int:
+    for line in sys.stdin:
+        line = line.strip()
+        if not line:
+            continue
+        try:
+            req = json.loads(line)
+        except Exception:
+            continue
+        resp = handle(req)
+        if resp is not None:
+            sys.stdout.write(json.dumps(resp) + "\n")
+            sys.stdout.flush()
+    return 0
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())

plugins/openclaw/README.md

@@ -0,0 +1,112 @@
+# OpenClaw Plugin for SkillOpt-Sleep
+
+Thin shell for running [SkillOpt-Sleep](https://github.com/microsoft/SkillOpt) on [OpenClaw](https://github.com/openclaw/openclaw).
+
+## What it does
+
+Adds a nightly "sleep cycle" to any OpenClaw agent. The cycle:
+
+1. **Harvests** recent session transcripts from `~/.openclaw/agents/<name>/sessions/*.jsonl`
+2. **Mines** recurring task patterns using the optimizer LLM
+3. **Replays** each pattern with the current `SKILL.md` (baseline) and a candidate `SKILL.md` (with proposed edits)
+4. **Gates** the candidate against the held-out score (rejects regressions)
+5. **Stages** the accepted proposal in `~/.skillopt-sleep/staging/<night>/`
+6. Leaves adoption to the operator (Ethan)
+
+Nothing live changes until you adopt. Every adopt backs up first.
+
+## Install
+
+The plugin is a thin wrapper around the engine at `~/.openclaw/workspace/SkillOpt/skillopt_sleep/`:
+
+```bash
+# 1. Clone the engine (one-time)
+cd ~/.openclaw/workspace
+git clone https://github.com/microsoft/SkillOpt.git
+
+# 2. Install the OpenClaw skill (this folder)
+ln -s /path/to/openclaw ~/.openclaw/workspace/skills/skillopt-sleep
+
+# 3. Configure
+cp ~/.openclaw/workspace/skills/skillopt-sleep/config.json ~/.skillopt-sleep/config.json
+$EDITOR ~/.skillopt-sleep/config.json
+# Set backend = "openclaw-deepseek"
+# Set model = "deepseek-v4-pro" (or "deepseek-v4-flash" for budget)
+
+# 4. Set API key
+echo 'export DEEPSEEK_API_KEY="sk-..."' >> ~/.openclaw/.env
+
+# 5. Add the nightly cron
+(crontab -l 2>/dev/null; echo "0 3 * * * cd ~/.openclaw/workspace/skills/skillopt-sleep && bash run_sleep_cron.sh >> ~/.skillopt-sleep/nightly.log 2>&1") | crontab -
+```
+
+## Use
+
+### Manual trigger
+
+```bash
+# Run one cycle now
+python3 ~/.openclaw/workspace/skills/skillopt-sleep/run_sleep.py
+
+# Dry run (report only)
+python3 ~/.openclaw/workspace/skills/skillopt-sleep/run_sleep.py --dry-run
+
+# One category only
+python3 ~/.openclaw/workspace/skills/skillopt-sleep/run_sleep.py --tasks tests/research-cron-tasks.json
+```
+
+### Slash command
+
+```bash
+# In any OpenClaw session
+/sleep status
+/sleep run
+/sleep run research-cron
+/sleep dry-run
+/sleep adopt              # adopt most recent accepted proposal
+/sleep reject             # discard most recent
+/sleep cost
+```
+
+## Architecture
+
+```
+plugins/openclaw/
+├── README.md                       # this file
+├── run_sleep_cron.sh               # wrapper for cron invocation
+├── run_sleep.py                    # main entry point
+├── slash_sleep.py                  # /sleep command implementation
+├── skillopt_sleep_openclaw.py      # DeepSeek + Ollama backend
+├── config.json                     # engine config
+├── SKILL.md                        # OpenClaw skill manifest
+└── tests/                          # held-out test sets
+    ├── research-cron-tasks.json
+    ├── devops-tasks.json
+    └── wiki-tasks.json
+```
+
+The OpenClaw shell is one engine (skillopt_sleep/) + one backend (DeepSeek/Ollama) + four thin wrappers (cron, slash, skill, tests).
+
+## Why this matters for OpenClaw
+
+OpenClaw currently has no built-in "self-evolving skills" mechanism. The community has:
+
+- **Manual skills** — Ethan writes them
+- **LLM-generated skills** — one-shot, no validation
+- **Self-revision** — unbounded, no quality bar
+
+SkillOpt-Sleep adds a 4th option: **validated self-evolution**. The skill is the training target, the engine is the optimizer, the gate is the quality bar, the operator is the human-in-the-loop.
+
+## Validation
+
+Validated on the public [gbrain-evals](https://github.com/garrytan/gbrain-evals) `skillopt-v1` benchmark with real Claude and Codex (deficient skills 0.00 → 1.00 on held-out, all 4 seeds).
+
+End-to-end test on our own 14-task held-out set: pipeline runs, gate correctly rejects non-improvements, staging artifacts land in `~/.skillopt-sleep/staging/<night>/`.
+
+## Cost
+
+Measured: ~$0.02/night with `deepseek-v4-pro` at 12 tasks/night. ~$0.59/month, $7.18/year.
+
+## License
+
+MIT (same as SkillOpt core).

plugins/openclaw/SKILL.md

@@ -0,0 +1,129 @@
+---
+name: skillopt-sleep
+description: Validate and refine agent skills through nightly sleep cycles with held-out gates. Wraps Microsoft's SkillOpt-Sleep engine for the OpenClaw/DeepSeek stack.
+---
+
+# skillopt-sleep — OpenClaw Adaptation of Microsoft SkillOpt-Sleep
+
+A nightly self-improvement loop that reads our session transcripts, mines recurring workflow patterns, replays them with proposed skill edits, and gates the proposals against a held-out test set. Only improvements that beat baseline are staged for human adoption.
+
+## When To Use
+
+- After Hermes's Weekly Skill Review (or as its replacement)
+- When a skill is being used 10+ times/week and could be tighter
+- Before promoting a new skill from `skill-proposals/` to `skills/`
+- When a skill regresses in observed quality
+
+## What It Does (One Cycle)
+
+```
+harvest session transcripts  ->  mine recurring task patterns
+                              ->  replay each pattern (current skill vs proposed)
+                              ->  GATE: must improve held-out score
+                              ->  stage proposal
+                              ->  Ethan adopts (manual)
+```
+
+Nothing live changes until Ethan adopts. Every adopt backs up first.
+
+## Architecture
+
+```
+skills/skillopt-sleep/
+├── SKILL.md                          # this file
+├── config.json                       # engine config (backend, budgets, etc.)
+├── run_sleep.py                      # entry point
+└── skillopt_sleep_openclaw.py        # DeepSeek/Ollama backend
+```
+
+The engine itself is at `~/.openclaw/workspace/SkillOpt/skillopt_sleep/` (cloned from microsoft/SkillOpt).
+
+## Usage
+
+```bash
+# Run one cycle with current config
+cd ~/.openclaw/workspace/skills/skillopt-sleep
+python3 run_sleep.py
+
+# Dry run (report only, no staging)
+python3 run_sleep.py --dry-run
+
+# Use a pre-built task set (recommended for testing)
+python3 run_sleep.py --tasks tests/research-cron-tasks.json
+```
+
+## Scheduling
+
+```bash
+python3 slash_sleep.py schedule --hour 3 --minute 17
+python3 slash_sleep.py unschedule
+python3 slash_sleep.py unschedule --all
+```
+
+Installs a nightly cron entry using the shared SkillOpt-Sleep scheduler. This is an alternative to the external `run_sleep_cron.sh` script.
+
+## Alternative backends
+
+While OpenClaw defaults to `openclaw-deepseek` (DeepSeek V4 Pro + Ollama), the shared engine also supports:
+- `--backend mock` — deterministic, no API spend (for testing)
+- `--backend claude` — uses the Claude CLI
+- `--backend codex` — uses the Codex CLI
+- `--backend copilot` — uses the GitHub Copilot CLI
+
+These can be used via the engine directly (`python -m skillopt_sleep`).
+
+## Shared-engine flags
+
+When invoking the engine directly, all standard flags are available:
+- `--source codex` / `--source auto` — harvest from Codex Desktop sessions
+- `--tasks-file PATH` — use a pre-built task set
+- `--target-skill-path PATH` — explicit SKILL.md target
+- `--max-tasks N` / `--max-sessions N` — cap workload
+- `--progress` — print phase progress
+- `--json` — machine-readable output
+- `--auto-adopt` — auto-adopt if gate passes
+
+Config keys: `preferences`, `gate_mode`, `gate_metric`, `dream_rollouts`, `recall_k`, `evolve_memory`, `evolve_skill`.
+
+## Config (config.json)
+
+Key knobs:
+- `backend: "openclaw-deepseek"` — our custom backend
+- `model: "deepseek-v4-pro"` — optimizer model
+- `edit_budget: 3` — max bounded edits per night
+- `gate_mode: "on"` — validation-gated (rejects regressions)
+- `auto_adopt: false` — require Ethan to adopt manually
+- `max_tasks_per_night: 12` — cap to control cost
+
+## Cost Estimate
+
+Per night: 12 tasks × (1 attempt + 1 judge + 1 reflect) × ~$0.005/1K tokens × ~3K tokens/call ≈ **$0.50-2.00/night**.
+
+## Outputs
+
+- Report: `~/.skillopt-sleep/state.json` (running totals)
+- Staging: `~/.skillopt-sleep/staging/<night>/`
+  - `report.md` — readable summary
+  - `best_skill.md` — proposed skill
+  - `edits.json` — bounded edit list
+  - `before.md` / `after.md` — diffs
+
+## Held-Out Test Sets (Phase 2)
+
+Located at `tests/<category>-tasks.json`. Each task has:
+- `prompt` — the recurring task
+- `reference` — exact-match gold answer
+- `rubric` — soft score rubric (0-1)
+- `domain` — research/devops/wiki/etc.
+
+Currently building for 3 categories:
+- research-cron-output
+- devops-infrastructure-check
+- wiki-canonical-guide
+
+## When NOT To Use
+
+- For a one-off workflow (not a recurring pattern)
+- During a crisis/incident (humans must lead)
+- When session transcripts are < 24h old (not enough signal)
+- For skills < 300 tokens (over-optimization risk)

plugins/openclaw/config.json

@@ -0,0 +1,30 @@
+{
+  "_comment": "OpenClaw adaptation of skillopt-sleep. Edit and run via run_sleep.py",
+
+  "claude_home": "/home/ethanclaw/.openclaw/agents",
+  "invoked_project": "/home/ethanclaw/.openclaw/workspace",
+  "projects": "invoked",
+  "lookback_hours": 168,
+
+  "max_tasks_per_night": 12,
+  "max_tokens_per_night": 800000,
+  "holdout_fraction": 0.34,
+  "val_fraction": 0.34,
+  "test_fraction": 0.0,
+
+  "backend": "openclaw-deepseek",
+  "model": "deepseek-v4-pro",
+  "gate_mode": "on",
+  "edit_budget": 3,
+  "gate_metric": "mixed",
+  "gate_mixed_weight": 0.5,
+  "replay_mode": "fresh",
+  "evolve_memory": true,
+  "evolve_skill": true,
+  "llm_mine": false,
+
+  "auto_adopt": false,
+  "managed_skill_name": "skillopt-sleep-learned",
+  "redact_secrets": true,
+  "seed": 42
+}

plugins/openclaw/run_sleep.py

@@ -0,0 +1,122 @@
+#!/usr/bin/env python3
+"""run_sleep.py — OpenClaw entry point for SkillOpt-Sleep.
+
+Runs one nightly sleep cycle:
+  1. harvest recent session transcripts
+  2. mine recurring task patterns
+  3. replay tasks with current skill (baseline) + candidate skill (with proposed edit)
+  4. gate candidate vs baseline on held-out accuracy
+  5. stage the proposal in ~/.skillopt-sleep/staging/<night>/
+  6. leave adoption to Ethan (auto_adopt=false)
+
+Usage:
+  python3 run_sleep.py                  # one cycle, default config
+  python3 run_sleep.py --dry-run        # compute report only, no staging
+  python3 run_sleep.py --tasks path.json  # use a pre-built task file
+"""
+from __future__ import annotations
+
+import argparse
+import json
+import os
+import sys
+from pathlib import Path
+
+# Ensure the skillopt_sleep package is importable (it lives in the cloned repo)
+REPO = Path("/home/ethanclaw/.openclaw/workspace/SkillOpt")
+sys.path.insert(0, str(REPO))
+
+# Register our backend before importing cycle
+from skillopt_sleep_openclaw import OpenClawDeepSeekBackend
+import skillopt_sleep.backend as _b
+_b._BACKENDS = getattr(_b, "_BACKENDS", {})
+_b._BACKENDS["openclaw-deepseek"] = OpenClawDeepSeekBackend
+
+# Patch get_backend to know about our backend
+_orig_get_backend = _b.get_backend
+
+def get_backend(name, model="", codex_path=""):
+    if name == "openclaw-deepseek":
+        return OpenClawDeepSeekBackend(model=model or "deepseek-v4-pro")
+    return _orig_get_backend(name, model=model, codex_path=codex_path)
+
+_b.get_backend = get_backend
+
+from skillopt_sleep.cycle import run_sleep_cycle
+from skillopt_sleep.config import load_config
+
+
+def main() -> int:
+    ap = argparse.ArgumentParser(description="OpenClaw SkillOpt-Sleep nightly cycle")
+    ap.add_argument("--dry-run", action="store_true", help="Compute but don't stage")
+    ap.add_argument("--config", default="/home/ethanclaw/.openclaw/workspace/skills/skillopt-sleep/config.json")
+    ap.add_argument("--tasks", default=None, help="Path to pre-built tasks JSON")
+    ap.add_argument("--verbose", action="store_true")
+    args = ap.parse_args()
+
+    # Load config from file then override with our defaults
+    overrides = {}
+    if os.path.exists(args.config):
+        with open(args.config) as f:
+            overrides.update(json.load(f))
+    overrides.pop("_comment", None)
+
+    cfg = load_config(**overrides)
+
+    seed_tasks = None
+    if args.tasks:
+        from skillopt_sleep.types import TaskRecord
+        with open(args.tasks) as f:
+            raw = json.load(f)
+        # Translate our test-set fields → TaskRecord fields
+        seed_tasks = []
+        for t in raw:
+            seed_tasks.append(TaskRecord(
+                id=t['id'],
+                project=t.get('project', 'openclaw'),
+                intent=t.get('intent') or t.get('prompt', ''),
+                context_excerpt=t.get('context_excerpt', ''),
+                attempted_solution=t.get('attempted_solution', ''),
+                outcome=t.get('outcome', 'unknown'),
+                reference_kind=t.get('reference_kind', 'rubric'),
+                reference=t.get('reference', ''),
+                judge=t.get('judge', {}),
+                tags=t.get('tags', []),
+                source_sessions=t.get('source_sessions', []),
+                split=t.get('split', 'train'),
+            ))
+
+    print(f"[skillopt-sleep] starting cycle...")
+    print(f"  backend: {cfg.get('backend')}")
+    print(f"  project: {cfg.get('invoked_project')}")
+    print(f"  max tasks: {cfg.get('max_tasks_per_night')}")
+    print(f"  edit budget: {cfg.get('edit_budget')}")
+    print(f"  dry_run: {args.dry_run}")
+
+    outcome = run_sleep_cycle(cfg, seed_tasks=seed_tasks, dry_run=args.dry_run)
+
+    r = outcome.report
+    print(f"\n=== Report — night {r.night} ===")
+    print(f"  sessions harvested: {r.n_sessions}")
+    print(f"  tasks mined: {r.n_tasks}  (replayed: {r.n_replayed})")
+    print(f"  baseline: {r.baseline_score:.3f}  ->  candidate: {r.candidate_score:.3f}")
+    print(f"  gate: {r.gate_action}  accepted={r.accepted}")
+    print(f"  tokens: {r.tokens_used}")
+    if r.edits:
+        print(f"  applied edits ({len(r.edits)}):")
+        for e in r.edits:
+            print(f"    [{e.target}/{e.op}] {e.content[:80]}...")
+    if r.rejected_edits:
+        print(f"  rejected edits ({len(r.rejected_edits)}) — kept as negative feedback")
+    if r.notes:
+        for n in r.notes:
+            print(f"  note: {n}")
+    if outcome.staging_dir:
+        print(f"\n  STAGED at: {outcome.staging_dir}")
+        print(f"  Review with: ls {outcome.staging_dir}")
+
+    return 0 if r.accepted or r.candidate_score >= r.baseline_score else 1
+
+
+if __name__ == "__main__":
+    sys.exit(main())

plugins/openclaw/run_sleep_cron.sh

@@ -0,0 +1,76 @@
+#!/bin/bash
+# run_sleep_cron.sh — wrapper for cron-driven nightly sleep cycle
+#
+# Usage: bash run_sleep_cron.sh [category1 category2 ...]
+#   No args: run on all categories in tests/
+#   With args: run only on listed categories (research-cron, devops, wiki)
+#
+# Cron (3am MYT daily):
+#   0 3 * * * cd /home/ethanclaw/.openclaw/workspace/skills/skillopt-sleep && bash run_sleep_cron.sh >> ~/.skillopt-sleep/nightly.log 2>&1
+
+set -euo pipefail
+
+SKILL_DIR="/home/ethanclaw/.openclaw/workspace/skills/skillopt-sleep"
+TESTS_DIR="$SKILL_DIR/tests"
+LOG_DIR="$HOME/.skillopt-sleep/logs"
+mkdir -p "$LOG_DIR"
+
+TIMESTAMP=$(date +%Y%m%d-%H%M%S)
+LOG_FILE="$LOG_DIR/night-$TIMESTAMP.log"
+
+# category → test file map
+declare -A CATEGORIES=(
+    ["research-cron"]="research-cron-tasks.json"
+    ["devops"]="devops-tasks.json"
+    ["wiki"]="wiki-tasks.json"
+)
+
+# Determine which categories to run
+if [ $# -eq 0 ]; then
+    CATS=("research-cron" "devops" "wiki")
+else
+    CATS=("$@")
+fi
+
+{
+    echo "=========================================="
+    echo "SkillOpt-Sleep nightly — $TIMESTAMP"
+    echo "Categories: ${CATS[*]}"
+    echo "=========================================="
+} | tee -a "$LOG_FILE"
+
+# Pre-flight: check DeepSeek API key
+if ! grep -q "DEEPSEEK_API_KEY=" "$HOME/.openclaw/.env" 2>/dev/null; then
+    echo "ERROR: DEEPSEEK_API_KEY not found in ~/.openclaw/.env" | tee -a "$LOG_FILE"
+    exit 1
+fi
+
+EXIT_CODE=0
+for cat in "${CATS[@]}"; do
+    tasks_file="$TESTS_DIR/${CATEGORIES[$cat]:-}"
+    if [ ! -f "$tasks_file" ]; then
+        echo "SKIP: $cat (no tasks file: $tasks_file)" | tee -a "$LOG_FILE"
+        continue
+    fi
+
+    echo "" | tee -a "$LOG_FILE"
+    echo "--- [$cat] starting cycle ---" | tee -a "$LOG_FILE"
+
+    cd "$SKILL_DIR"
+    if python3 run_sleep.py --tasks "$tasks_file" 2>&1 | tee -a "$LOG_FILE"; then
+        echo "--- [$cat] OK ---" | tee -a "$LOG_FILE"
+    else
+        EC=$?
+        echo "--- [$cat] FAILED (exit $EC) ---" | tee -a "$LOG_FILE"
+        EXIT_CODE=$EC
+    fi
+done
+
+{
+    echo ""
+    echo "=========================================="
+    echo "Done. Exit: $EXIT_CODE"
+    echo "=========================================="
+} | tee -a "$LOG_FILE"
+
+exit $EXIT_CODE

plugins/openclaw/skillopt_sleep_openclaw.py

@@ -0,0 +1,275 @@
+"""OpenClaw backend for SkillOpt-Sleep.
+
+Adapts the skillopt_sleep Backend protocol to our DeepSeek + Ollama stack:
+  - attempt/judge/reflect  ->  DeepSeek V4 Pro (or Flash for cost)
+  - embeddings              ->  Ollama nomic-embed-text (already configured)
+
+This backend NEVER mutates live state. It only returns text + EditRecord
+proposals that the gate stages for human review.
+"""
+from __future__ import annotations
+
+import json
+import os
+import re
+import subprocess
+from typing import Any, Dict, List, Optional, Tuple
+
+from skillopt_sleep.backend import Backend, _normalize, exact_score
+from skillopt_sleep.types import EditRecord, ReplayResult, TaskRecord
+
+
+# ── DeepSeek + Ollama OpenAI-compatible API client (curl-based, no extra deps) ──
+
+
+def _chat(messages: List[Dict[str, str]], *, model: str, temperature: float = 0.2, max_tokens: int = 1500) -> str:
+    """Call DeepSeek V4 Pro via curl + jq. No extra Python deps needed."""
+    import json as _json
+    import urllib.request
+
+    api_key = os.environ.get("DEEPSEEK_API_KEY", "")
+    if not api_key:
+        # try loading from .env
+        env_path = os.path.expanduser("~/.openclaw/.env")
+        if os.path.exists(env_path):
+            with open(env_path) as f:
+                for line in f:
+                    if line.startswith("DEEPSEEK_API_KEY="):
+                        api_key = line.split("=", 1)[1].strip()
+                        break
+
+    base = os.environ.get("DEEPSEEK_BASE_URL", "https://api.deepseek.com/v1")
+
+    payload = {
+        "model": model,
+        "messages": messages,
+        "temperature": temperature,
+        "max_tokens": max_tokens,
+        "stream": False,
+    }
+    req = urllib.request.Request(
+        f"{base}/chat/completions",
+        data=_json.dumps(payload).encode("utf-8"),
+        headers={
+            "Content-Type": "application/json",
+            "Authorization": f"Bearer {api_key}",
+        },
+    )
+    try:
+        with urllib.request.urlopen(req, timeout=180) as resp:
+            data = _json.loads(resp.read().decode("utf-8"))
+            return data["choices"][0]["message"]["content"]
+    except Exception as e:
+        return f"[BACKEND_ERROR] {type(e).__name__}: {str(e)[:200]}"
+
+
+def _embed(text: str) -> List[float]:
+    """Call Ollama for embeddings. Uses the configured nomic-embed-text model."""
+    import json as _json
+    import urllib.request
+
+    try:
+        req = urllib.request.Request(
+            "http://127.0.0.1:11434/api/embeddings",
+            data=_json.dumps({"model": "nomic-embed-text:latest", "prompt": text[:2000]}).encode("utf-8"),
+            headers={"Content-Type": "application/json"},
+        )
+        with urllib.request.urlopen(req, timeout=30) as resp:
+            data = _json.loads(resp.read().decode("utf-8"))
+            return data.get("embedding", [])
+    except Exception:
+        return []
+
+
+# ── Backend implementation ────────────────────────────────────────────────────
+
+
+class OpenClawDeepSeekBackend(Backend):
+    """Use DeepSeek V4 Pro for attempt/judge/reflect, Ollama for embeddings.
+
+    - "model" passed to constructor = optimizer model (default: deepseek-v4-pro)
+    - "judge_model" = judge model (default: deepseek-v4-pro for quality)
+    - "cheap_model" = budget-fallback (deepseek-v4-flash)
+    """
+
+    name = "openclaw-deepseek"
+
+    def __init__(
+        self,
+        model: str = "deepseek-v4-pro",
+        judge_model: str = "deepseek-v4-pro",
+        cheap_model: str = "deepseek-v4-flash",
+    ):
+        self._model = model
+        self._judge_model = judge_model
+        self._cheap_model = cheap_model
+        self._tokens = 0  # rough estimate
+
+    def tokens_used(self) -> int:
+        return self._tokens
+
+    # ── 1. attempt: produce a response given the task + skill + memory ──
+    def attempt(self, task: TaskRecord, skill: str, memory: str) -> str:
+        sys = (
+            "You are an OpenClaw agent (Kobe ecosystem). Use the skill and memory below to complete the task. "
+            "If the task asks for a structured output, follow the rubric exactly. "
+            "Be concise. No preamble, no explanation unless the task asks for it."
+        )
+        usr = f"""## SKILL
+{skill or '(no skill yet)'}
+
+## MEMORY
+{memory or '(no memory yet)'}
+
+## TASK
+{task.intent}
+
+## CONTEXT (if any)
+{task.context_excerpt or '(none)'}
+
+## RESPONSE
+"""
+        out = _chat(
+            [{"role": "system", "content": sys}, {"role": "user", "content": usr}],
+            model=self._model,
+            temperature=0.2,
+        )
+        self._tokens += len(usr) // 4 + 200
+        return out
+
+    # ── 2. judge: score the response ──
+    def judge(self, task: TaskRecord, response: str) -> Tuple[float, float, str]:
+        # Hard score: exact-match against task.reference (if available)
+        hard = exact_score(task.reference or "", response)
+
+        # Soft score: LLM judge against rubric (reference if reference_kind=='rubric')
+        rubric_text = task.reference if task.reference_kind == "rubric" else ""
+        if rubric_text:
+            judge_prompt = f"""You are a strict grader. Score the response 0.0-1.0 against the rubric.
+
+## TASK
+{task.intent}
+
+## REFERENCE
+{task.reference or '(none)'}
+
+## RUBRIC
+{rubric_text}
+
+## RESPONSE
+{response[:3000]}
+
+## INSTRUCTIONS
+Return ONLY a single float 0.0-1.0 on one line. No explanation. No markdown.
+"""
+            try:
+                j_out = _chat(
+                    [{"role": "user", "content": judge_prompt}],
+                    model=self._judge_model,
+                    temperature=0.0,
+                    max_tokens=20,
+                ).strip()
+                soft = float(re.search(r"[\d.]+", j_out.splitlines()[0]).group())
+                soft = max(0.0, min(1.0, soft))
+            except Exception:
+                soft = hard
+            self._tokens += 600
+        else:
+            soft = hard
+
+        rationale = f"hard={hard:.2f} soft={soft:.2f}"
+        return hard, soft, rationale
+
+    # ── 3. reflect: produce bounded EditRecord proposals ──
+    def reflect(
+        self,
+        failures: List[Tuple[TaskRecord, ReplayResult]],
+        successes: List[Tuple[TaskRecord, ReplayResult]],
+        skill: str,
+        memory: str,
+        *,
+        edit_budget: int,
+        evolve_skill: bool,
+        evolve_memory: bool,
+    ) -> List[EditRecord]:
+        # Compact digest of failures + successes
+        fail_digest = "\n".join(
+            f"- TASK: {t.intent[:200]}\n  RESPONSE: {r.response[:300]}\n  WHY FAIL: {r.judge_rationale or r.fail_reason or 'unknown'}\n  REFERENCE: {t.reference[:200]}"
+            for t, r in failures[:5]
+        ) or "(none)"
+        succ_digest = "\n".join(
+            f"- TASK: {t.intent[:150]} -> OK ({r.judge_rationale or 'high score'})"
+            for t, r in successes[:3]
+        ) or "(none)"
+
+        rubric_text = ""
+        if failures:
+            rubric_text = f"\n\n## REFERENCE ANSWERS\n{chr(10).join(f'Q: {t.intent[:120]}\\nA: {t.reference}' for t, _ in failures[:3] if t.reference)}"
+
+        sys = (
+            "You are SkillOpt-Sleep's bounded-edit optimizer. Your job is to propose 1-4 MINIMAL text edits to a skill or memory document "
+            "that, if applied, would help future agents do better on the failed tasks. "
+            "NEVER propose adding new sections wholesale. NEVER delete entire sections. "
+            "Edit primitives: ADD (append a step/rule at end), DELETE (remove a specific line by exact match), REPLACE (swap a specific line for another by exact match). "
+            "If you cannot identify a clear, minimal improvement, return an empty list."
+        )
+        usr = f"""## CURRENT SKILL
+{skill or '(empty)'}
+
+## CURRENT MEMORY
+{memory or '(empty)'}
+
+## FAILED TASKS
+{fail_digest}
+
+## SUCCESSFUL TASKS
+{succ_digest}
+{rubric_text}
+
+## CONSTRAINTS
+- max {edit_budget} edits total
+- edits go to {"skill + memory" if (evolve_skill and evolve_memory) else ("skill" if evolve_skill else "memory")}
+- if evolve_skill=False, target="memory" only; if evolve_memory=False, target="skill" only
+- target must be "skill" or "memory"
+
+## OUTPUT FORMAT (JSON, no markdown)
+{{"edits": [{{"op": "ADD"|"DELETE"|"REPLACE", "target": "skill"|"memory", "content": "the text to add or replace with", "old_text": "for REPLACE/DELETE, the exact line to find", "rationale": "one short sentence why"}}]}}
+"""
+        out = _chat(
+            [{"role": "system", "content": sys}, {"role": "user", "content": usr}],
+            model=self._model,
+            temperature=0.4,
+            max_tokens=2000,
+        )
+        self._tokens += len(usr) // 3 + 1500
+
+        # parse
+        try:
+            # strip markdown fences if any
+            cleaned = out.strip()
+            if cleaned.startswith("```"):
+                cleaned = re.sub(r"^```[a-z]*\n?", "", cleaned)
+                cleaned = re.sub(r"\n?```$", "", cleaned)
+            data = json.loads(cleaned)
+            edits: List[EditRecord] = []
+            for e in data.get("edits", [])[:edit_budget]:
+                if e.get("op") not in ("ADD", "DELETE", "REPLACE"):
+                    continue
+                target = e.get("target", "skill")
+                if target not in ("skill", "memory"):
+                    continue
+                if not evolve_skill and target == "skill":
+                    continue
+                if not evolve_memory and target == "memory":
+                    continue
+                edits.append(EditRecord(
+                    op=e["op"],
+                    target=target,
+                    content=e.get("content", ""),
+                    old_text=e.get("old_text", ""),
+                    rationale=e.get("rationale", ""),
+                ))
+            return edits
+        except Exception as e:
+            # log + return empty list (no edit is better than a bad edit)
+            return []

plugins/openclaw/slash_sleep.py

@@ -0,0 +1,330 @@
+#!/usr/bin/env python3
+"""slash_sleep.py — OpenClaw slash command equivalent of SkillOpt's /sleep.
+
+Use from the main session as a /sleep command:
+  /sleep status    — show current state + last 5 nights
+  /sleep run       — trigger one cycle (all categories) right now
+  /sleep run research-cron  — one cycle, single category
+  /sleep adopt [night]      — adopt the most recent (or specified) staged proposal
+  /sleep reject [night]     — discard the most recent (or specified) staging dir
+  /sleep dry-run   — report-only cycle
+  /sleep cost      — estimate per-night cost for current config
+
+This script is a thin shell over run_sleep.py. It can be invoked either
+manually from the main session or by an OpenClaw command handler.
+"""
+from __future__ import annotations
+
+import argparse
+import json
+import os
+import shutil
+import sys
+from pathlib import Path
+from datetime import datetime
+
+SKILL_DIR = Path("/home/ethanclaw/.openclaw/workspace/skills/skillopt-sleep")
+STATE_DIR = Path(os.path.expanduser("~/.skillopt-sleep"))  # default
+STAGING_ROOT = STATE_DIR
+
+def _resolve_state_dir():
+    """Find the actual state dir.
+
+    Priority: scan in order:
+      1. ~/.skillopt-sleep/                 (default)
+      2. /home/ethanclaw/.openclaw/workspace/.skillopt-sleep/  (when staging is there)
+      3. /home/ethanclaw/.openclaw/.skillopt-sleep/            (parent of overridden claude_home)
+    Pick the first one that has a state.json OR staging dir.
+    """
+    candidates = [
+        Path(os.path.expanduser("~/.skillopt-sleep")),
+        Path("/home/ethanclaw/.openclaw/workspace/.skillopt-sleep"),
+        Path("/home/ethanclaw/.openclaw/.skillopt-sleep"),
+    ]
+    # Prefer the one with state.json
+    for c in candidates:
+        if (c / "state.json").exists():
+            return c
+    # Then the one with staging
+    for c in candidates:
+        if (c / "staging").exists():
+            return c
+    return candidates[0]
+
+TESTS_DIR = SKILL_DIR / "tests"
+
+
+def status() -> int:
+    state_dir = _resolve_state_dir()
+    state_file = state_dir / "state.json"
+    staging_dir = state_dir / "staging"
+    print(f"=== SkillOpt-Sleep status ===")
+    print(f"  state dir: {state_dir}")
+    print(f"  staging dir: {staging_dir}")
+    if staging_dir.exists():
+        stages = sorted(staging_dir.iterdir(), key=lambda p: p.stat().st_mtime, reverse=True)
+        print(f"  staging entries: {len(stages)}")
+        for s in stages[:3]:
+            print(f"    {s.name}")
+    if not state_file.exists():
+        print("  no state.json — run a cycle first (state is written at end of each non-dry-run)")
+        return 0
+
+    with open(state_file) as f:
+        state = json.load(f)
+
+    nights = state.get("history") or state.get("nights", [])
+    print(f"  total nights: {len(nights)}")
+    print(f"  accepted: {sum(1 for n in nights if n.get('accepted'))}")
+    print(f"  rejected: {sum(1 for n in nights if not n.get('accepted'))}")
+    if nights:
+        last = nights[-1]
+        print(f"  last night: {last.get('night')}")
+        print(f"    accepted: {last.get('accepted')}")
+        print(f"    baseline: {last.get('baseline'):.3f}  ->  candidate: {last.get('candidate'):.3f}")
+        print(f"    staging: {last.get('staging') or '(none)'}")
+    return 0
+
+
+def run_category(category: str, *, dry_run: bool = False) -> int:
+    cat_to_file = {
+        "research-cron": "research-cron-tasks.json",
+        "devops": "devops-tasks.json",
+        "wiki": "wiki-tasks.json",
+    }
+    tasks_file = TESTS_DIR / cat_to_file.get(category, f"{category}-tasks.json")
+    if not tasks_file.exists():
+        print(f"ERROR: no tasks file for category '{category}': {tasks_file}")
+        return 1
+
+    cmd = [sys.executable, str(SKILL_DIR / "run_sleep.py")]
+    if dry_run:
+        cmd.append("--dry-run")
+    cmd.extend(["--tasks", str(tasks_file)])
+
+    print(f"=== /sleep run {category}{' (dry-run)' if dry_run else ''} ===")
+    print(f"  cmd: {' '.join(cmd)}")
+    rc = os.system(" ".join(f'"{c}"' for c in cmd))
+    return rc
+
+
+def run_all(*, dry_run: bool = False) -> int:
+    rc = 0
+    for cat in ("research-cron", "devops", "wiki"):
+        r = run_category(cat, dry_run=dry_run)
+        if r != 0:
+            rc = r
+    return rc
+
+
+def adopt(night: str = None) -> int:
+    state_dir = _resolve_state_dir()
+    state_file = state_dir / "state.json"
+    if not state_file.exists():
+        print("ERROR: no state to adopt from")
+        return 1
+    with open(state_file) as f:
+        state = json.load(f)
+    nights = state.get("history") or state.get("nights", [])
+    if not nights:
+        print("ERROR: no nights recorded")
+        return 1
+
+    target = None
+    if night:
+        target = next((n for n in nights if str(n.get("night")) == night), None)
+        if not target:
+            print(f"ERROR: night '{night}' not found")
+            return 1
+    else:
+        # most recent accepted
+        candidates = [n for n in nights if n.get("accepted") and n.get("staging")]
+        if not candidates:
+            print("ERROR: no accepted nights with staging to adopt")
+            return 1
+        target = candidates[-1]
+
+    staging = target["staging"]
+    if not os.path.isdir(staging):
+        print(f"ERROR: staging dir missing: {staging}")
+        return 1
+
+    print(f"=== /sleep adopt night {target['night']} ===")
+    print(f"  staging: {staging}")
+    print(f"  baseline: {target.get('baseline'):.3f}  candidate: {target.get('candidate'):.3f}")
+
+    # Read proposed skill from staging
+    manifest = Path(staging) / "manifest.json"
+    if manifest.exists():
+        with open(manifest) as f:
+            m = json.load(f)
+        proposed = m.get("proposed_skill")
+        if proposed and Path(proposed).exists():
+            live = STATE_DIR / "live_skill.md"
+            backup = STATE_DIR / f"live_skill.md.bak-{target['night']}"
+            if live.exists():
+                shutil.copy2(live, backup)
+                print(f"  backed up current live skill → {backup}")
+            shutil.copy2(proposed, live)
+            print(f"  adopted proposed skill → {live}")
+            print()
+            print("✅ Adoption complete. Next cycle will use the new skill.")
+            return 0
+
+    print("ERROR: no proposed_skill in manifest")
+    return 1
+
+
+def reject(night: str = None) -> int:
+    state_dir = _resolve_state_dir()
+    state_file = state_dir / "state.json"
+    if not state_file.exists():
+        print("ERROR: no state")
+        return 1
+    with open(state_file) as f:
+        state = json.load(f)
+    nights = state.get("history") or state.get("nights", [])
+    target = None
+    if night:
+        target = next((n for n in nights if str(n.get("night")) == night), None)
+    else:
+        candidates = [n for n in reversed(nights) if n.get("staging")]
+        target = candidates[0] if candidates else None
+
+    if not target or not target.get("staging"):
+        print("ERROR: nothing to reject")
+        return 1
+
+    staging = target["staging"]
+    if os.path.isdir(staging):
+        shutil.rmtree(staging)
+        print(f"🗑️  Removed staging: {staging}")
+    # remove from state
+    state["history"] = [n for n in nights if n.get("night") != target["night"]]
+    with open(state_file, "w") as f:
+        json.dump(state, f, indent=2)
+    print("✅ Rejected. State updated.")
+    return 0
+
+
+def schedule_cmd(hour: int, minute: int) -> int:
+    """Install a nightly cron entry via the shared SkillOpt-Sleep scheduler.
+
+    Note: this schedules the shared engine (``python -m skillopt_sleep run``),
+    not the OpenClaw-specific ``run_sleep.py``.  Use ``run_sleep_cron.sh`` if
+    you need the OpenClaw-native backend and category task files instead.
+    """
+    try:
+        from skillopt_sleep.scheduler import schedule
+    except ImportError:
+        print("ERROR: skillopt_sleep.scheduler not available — is SkillOpt-Sleep installed?")
+        return 1
+    project = str(SKILL_DIR)
+    ok, msg = schedule(project, hour=hour, minute=minute)
+    print(msg)
+    return 0 if ok else 1
+
+
+def unschedule_cmd(all_projects: bool) -> int:
+    """Remove cron entry via the shared SkillOpt-Sleep scheduler."""
+    try:
+        from skillopt_sleep.scheduler import unschedule
+    except ImportError:
+        print("ERROR: skillopt_sleep.scheduler not available — is SkillOpt-Sleep installed?")
+        return 1
+    project = str(SKILL_DIR)
+    ok, msg = unschedule(project, all_projects=all_projects)
+    print(msg)
+    return 0 if ok else 1
+
+
+def cost() -> int:
+    """Estimate per-night cost based on the actual measurement from Phase 2.
+
+    From the real dry-run: 5 devops tasks used 14,427 tokens total.
+    That is ~2,885 tokens per task (all 3 phases combined).
+    """
+    cfg_path = SKILL_DIR / "config.json"
+    cfg = {}
+    if cfg_path.exists():
+        cfg = json.loads(cfg_path.read_text())
+    cfg.pop("_comment", None)
+
+    max_tasks = cfg.get("max_tasks_per_night", 12)
+    model = cfg.get("model", "deepseek-v4-pro")
+    # DeepSeek V4 pricing
+    if "pro" in model:
+        cost_in = 0.435  # per 1M
+        cost_out = 0.87
+    elif "flash" in model:
+        cost_in = 0.14
+        cost_out = 0.28
+    else:
+        cost_in, cost_out = 0.5, 1.0
+
+    # Measured: ~2,900 tokens per task, 30% output / 70% input
+    toks_per_task = 2900
+    input_toks = int(toks_per_task * 0.7)
+    output_toks = int(toks_per_task * 0.3)
+
+    cost_in_total = (input_toks * max_tasks / 1_000_000) * cost_in
+    cost_out_total = (output_toks * max_tasks / 1_000_000) * cost_out
+    cost = cost_in_total + cost_out_total
+
+    print(f"=== Cost estimate (per actual measurement) ===")
+    print(f"  model: {model}")
+    print(f"  max tasks/night: {max_tasks}")
+    print(f"  ~tokens/night: {toks_per_task * max_tasks:,}")
+    print(f"  cost/night: ${cost:.3f}")
+    print(f"  cost/month (30 nights): ${cost*30:.2f}")
+    print(f"  cost/year (365 nights): ${cost*365:.2f}")
+    return 0
+
+
+def main():
+    ap = argparse.ArgumentParser(description="OpenClaw /sleep command")
+    sub = ap.add_subparsers(dest="cmd", required=True)
+
+    sub.add_parser("status", help="show state + last 5 nights")
+    p_run = sub.add_parser("run", help="trigger one cycle")
+    p_run.add_argument("category", nargs="?", default=None,
+                        choices=["research-cron", "devops", "wiki", None])
+    p_run.add_argument("--dry-run", action="store_true")
+    sub.add_parser("dry-run", help="report-only cycle (all categories)")
+    p_adopt = sub.add_parser("adopt", help="adopt most recent accepted staging")
+    p_adopt.add_argument("night", nargs="?", default=None)
+    p_reject = sub.add_parser("reject", help="discard most recent staging")
+    p_reject.add_argument("night", nargs="?", default=None)
+    sub.add_parser("cost", help="estimate cost")
+    p_schedule = sub.add_parser("schedule", help="install nightly cron entry")
+    p_schedule.add_argument("--hour", type=int, default=3, help="hour (0-23)")
+    p_schedule.add_argument("--minute", type=int, default=17, help="minute (0-59)")
+    p_unschedule = sub.add_parser("unschedule", help="remove cron entry")
+    p_unschedule.add_argument("--all", dest="all_projects", action="store_true",
+                               help="remove entries for all projects")
+
+    args = ap.parse_args()
+
+    if args.cmd == "status":
+        return status()
+    if args.cmd == "run":
+        if args.category:
+            return run_category(args.category, dry_run=args.dry_run)
+        return run_all(dry_run=args.dry_run)
+    if args.cmd == "dry-run":
+        return run_all(dry_run=True)
+    if args.cmd == "adopt":
+        return adopt(args.night)
+    if args.cmd == "reject":
+        return reject(args.night)
+    if args.cmd == "cost":
+        return cost()
+    if args.cmd == "schedule":
+        return schedule_cmd(args.hour, args.minute)
+    if args.cmd == "unschedule":
+        return unschedule_cmd(args.all_projects)
+    return 1
+
+
+if __name__ == "__main__":
+    sys.exit(main())

plugins/openclaw/tests/devops-tasks.json

@@ -0,0 +1,87 @@
+[
+  {
+    "id": "do-01",
+    "reference": "[STATUS] devops-agent | Site Uptime \u2192 geoxylia.com OK (200) | 14/06 22:30 MYT",
+    "rubric": "Score 1.0 if output matches the exact format [STATUS] devops-agent | Site Uptime \u2192 geoxylia.com OK (200) | DD/MM HH:MM MYT, with a real current time. Score 0.5 if format is close but missing one field. Score 0.0 if wrong format or hallucinated values.",
+    "project": "devops-infrastructure-check",
+    "intent": "Site Uptime check. Run: `curl -o /dev/null -s -w '%{http_code}' https://geoxylia.com`. Interpret the result 200, and report in our standard format: 'STATUS | TASK \u2192 RESULT | TIME'. If not 200, escalate.",
+    "context_excerpt": "",
+    "attempted_solution": "",
+    "outcome": "unknown",
+    "reference_kind": "rubric",
+    "judge": {},
+    "tags": [
+      "devops-infrastructure-check"
+    ],
+    "source_sessions": [],
+    "split": "val"
+  },
+  {
+    "id": "do-02",
+    "reference": "Backup complete. Files: 87, Size: 1.2G, Last: 2026-06-14 22:00:00 MYT",
+    "rubric": "Score 1.0 if output includes the exact 'Backup complete. Files: N, Size: X, Last: timestamp' structure with plausible values. Score 0.5 if structure is close but one field missing. Score 0.0 if hallucinated or wrong structure.",
+    "project": "devops-infrastructure-check",
+    "intent": "Daily Memory Backup. Confirm this ran successfully by checking: `ls -t ~/backups/memory/memory-backup-*.tar.gz | head -3`. Report the file count, total size, and most recent backup time. Use format: 'Backup complete. Files: [N], Size: [X], Last: [timestamp]'.",
+    "context_excerpt": "",
+    "attempted_solution": "",
+    "outcome": "unknown",
+    "reference_kind": "rubric",
+    "judge": {},
+    "tags": [
+      "devops-infrastructure-check"
+    ],
+    "source_sessions": [],
+    "split": "val"
+  },
+  {
+    "id": "do-03",
+    "reference": "1) Vercel CSP missing frame-ancestors: MEDIUM. Allows clickjacking if anyone embeds our pages; not exploitable for our content, but best-practice gap.\n2) OpenClaw plaintext API keys: LOW. The config is chmod 600, loopback-only, not in git. Standard OpenClaw behavior. Rotating would add zero real security given current exposure.",
+    "rubric": "Score 1.0 if both are classified correctly (MEDIUM and LOW respectively) and justifications are accurate (not panicky, not dismissive). Score 0.5 if classifications are wrong by one tier or justifications are weak. Score 0.0 if both over-classified as CRITICAL or both wrong.",
+    "project": "devops-infrastructure-check",
+    "intent": "Security Check daily run. Two findings: 1) Vercel CSP header missing 'frame-ancestors' directive, 2) OpenClaw config has 3 plaintext API keys. Classify each as: CRITICAL / HIGH / MEDIUM / LOW / INFO. Justify each in 1 sentence.",
+    "context_excerpt": "",
+    "attempted_solution": "",
+    "outcome": "unknown",
+    "reference_kind": "rubric",
+    "judge": {},
+    "tags": [
+      "devops-infrastructure-check"
+    ],
+    "source_sessions": [],
+    "split": "train"
+  },
+  {
+    "id": "do-04",
+    "reference": "[INCIDENT] supabase.audit_results: anon role has no RLS policy \u2014 anyone with the URL can read all audit results. Fix: add policy 'audit_results_select_own' granting SELECT WHERE user_id = auth.uid(). Severity: HIGH (data exposure). Estimated 2-min fix.",
+    "rubric": "Score 1.0 if: (a) severity correctly identified as HIGH, (b) fix is a real RLS policy (not just 'enable RLS' since it's already enabled), (c) under 50 words, (d) Telegram-friendly format. Score 0.5 if severity right but fix is generic. Score 0.0 if missing severity or wrong fix.",
+    "project": "devops-infrastructure-check",
+    "intent": "Incident Check. The Supabase RLS check returned: 'table public.audit_results: rls enabled but policy missing for anon role'. Interpret severity, propose fix, and format as a Telegram alert (max 50 words).",
+    "context_excerpt": "",
+    "attempted_solution": "",
+    "outcome": "unknown",
+    "reference_kind": "rubric",
+    "judge": {},
+    "tags": [
+      "devops-infrastructure-check"
+    ],
+    "source_sessions": [],
+    "split": "val"
+  },
+  {
+    "id": "do-05",
+    "reference": "\ud83d\udee1\ufe0f Week security digest:\n\n\u2022 0 critical incidents, 1 high resolved (Supabase RLS policy added)\n\u2022 22 plaintext secrets: expected OpenClaw behavior, no action\n\u2022 1 medium open: Vercel CSP frame-ancestors, schedule for next sprint\n\nTrend: stable. No regressions vs last week.",
+    "rubric": "Score 1.0 if all 3 priority tiers mentioned with correct counts, ends with a trend statement, Telegram-friendly. Score 0.5 if structure is right but one tier wrong. Score 0.0 if missing a tier or wrong format.",
+    "project": "devops-infrastructure-check",
+    "intent": "Weekly security digest. Synthesize this week's findings: 22 plaintext secrets in openclaw.json (expected), 0 critical incidents, 1 high (Supabase RLS), 1 medium (CSP frame-ancestors), 0 low. Output a 3-bullet Telegram status.",
+    "context_excerpt": "",
+    "attempted_solution": "",
+    "outcome": "unknown",
+    "reference_kind": "rubric",
+    "judge": {},
+    "tags": [
+      "devops-infrastructure-check"
+    ],
+    "source_sessions": [],
+    "split": "train"
+  }
+]

plugins/openclaw/tests/research-cron-tasks.json

@@ -0,0 +1,87 @@
+[
+  {
+    "id": "rc-01",
+    "reference": "COMPETITOR MOVES: Otterly adds Perplexity tracker, joining Profound and LLMRefs in multi-platform citations.\nBACKLINK OPPORTUNITIES: 3 SEO directories (G2, Capterra, GetApp) have not been claimed.\nAGENCY BLUEPRINT: Top 2 agency sites bundle GEO audit + content refresh as $3K/mo tier.\nACTION ITEMS: Build Perplexity citation test into GeoXylia audit; claim G2 listing by Friday.",
+    "rubric": "Score 1.0 if all 4 section headings present in correct order, each with a substantive (not generic) 1-sentence content. Score 0.5 if headings present but content is generic. Score 0.0 if any heading missing or order wrong.",
+    "project": "research-cron-output",
+    "intent": "Weekly Competitive Deep Dive for GeoXylia. The competitor otterly.ai just added a Perplexity citation tracker. Produce the report header (top section) in our standard format: COMPETITOR MOVES, BACKLINK OPPORTUNITIES, AGENCY BLUEPRINT, ACTION ITEMS. Keep it to 4 lines, one per section heading with a 1-sentence placeholder.",
+    "context_excerpt": "",
+    "attempted_solution": "",
+    "outcome": "unknown",
+    "reference_kind": "rubric",
+    "judge": {},
+    "tags": [
+      "research-cron-output"
+    ],
+    "source_sessions": [],
+    "split": "train"
+  },
+  {
+    "id": "rc-02",
+    "reference": "1. 'ai seo audit tool': 420 imp, pos 8.2, on page 1 \u2014 needs CTR lift (snippet/schema).\n2. 'geo audit tool': 230 imp, pos 12.5, page 2 \u2014 target blog post could push to page 1.\n3. 'llm optimization': 85 imp, pos 18.3, deep page-2 \u2014 fresh content with answer capsule could compete.",
+    "rubric": "Score 1.0 if the response correctly identifies 'ai seo audit tool', 'geo audit tool', and 'llm optimization' as the top 3 (NOT 'best free seo audit' which is already converting well, NOT 'free audit tool' which has too few impressions). Each must have correct impression count, position, and a substantive rationale. Score 0.5 if correct 3 keywords but rationale is weak. Score 0.0 if wrong keywords selected.",
+    "project": "research-cron-output",
+    "intent": "GSC keyword opportunity scan. From this snippet of GSC data, identify the top 3 keyword opportunities (high impressions, low CTR, position 5-15):\n\n1. 'ai seo audit tool' \u2014 420 imp, 12 clicks, pos 8.2\n2. 'best free seo audit' \u2014 1100 imp, 95 clicks, pos 4.1\n3. 'geo audit tool' \u2014 230 imp, 4 clicks, pos 12.5\n4. 'llm optimization' \u2014 85 imp, 1 click, pos 18.3\n5. 'free audit tool' \u2014 50 imp, 0 clicks, pos 22.0\n\nOutput: one line per opportunity, format 'KEYWORD: impressions, position, why-it-matters (1 short clause)'.",
+    "context_excerpt": "",
+    "attempted_solution": "",
+    "outcome": "unknown",
+    "reference_kind": "rubric",
+    "judge": {},
+    "tags": [
+      "research-cron-output"
+    ],
+    "source_sessions": [],
+    "split": "train"
+  },
+  {
+    "id": "rc-03",
+    "reference": "Google AI Overviews now show source links more prominently + author bylines. For GeoXylia: this favors pages with clear authorship (add author schema to blog posts). Action: this week, add author + E-E-A-T schema markup to top 10 blog posts. Source: Google Search Central blog.",
+    "rubric": "Score 1.0 if: (a) under 60 words, (b) names the change, (c) gives GeoXylia-specific implication, (d) gives a concrete action item, (e) cites the source. Score 0.5 if missing 1-2 of these. Score 0.0 if over 60 words or missing 3+.",
+    "project": "research-cron-output",
+    "intent": "Daily Industry News scan. The Google Search Central blog just announced: 'AI Overviews now showing source links more prominently, with author bylines for E-E-A-T-heavy content.' Write a 1-paragraph Telegram alert (max 60 words) for Ethan. Include: 1) what changed, 2) what it means for GeoXylia, 3) any action item.",
+    "context_excerpt": "",
+    "attempted_solution": "",
+    "outcome": "unknown",
+    "reference_kind": "rubric",
+    "judge": {},
+    "tags": [
+      "research-cron-output"
+    ],
+    "source_sessions": [],
+    "split": "val"
+  },
+  {
+    "id": "rc-04",
+    "reference": "Hi [Name], I saw seo-skill.com's resources page is one of the most-respected SEO learning hubs in the industry \u2014 your 2026 algorithm breakdown was spot-on. We just published a free 2026 AI SEO Audit comparison that your readers would find genuinely useful (no paywall, no signup). It covers the 8 leading AI-audit tools with hands-on screenshots and a clear feature matrix. GeoXylia is the only fully-free option in the comparison, so it's a natural fit for a 'tools to know' section. Mind if I share the link for inclusion?",
+    "rubric": "Score 1.0 if exactly 4 sentences, all four functional pieces present (compliment / mention resource / audience benefit / GeoXylia one-liner), conversational tone, no aggressive sales language. Score 0.5 if 3 of 4 pieces present or tone is too salesy. Score 0.0 if more than 5 sentences or missing 2+ pieces.",
+    "project": "research-cron-output",
+    "intent": "Backlink Outreach draft for the blog post 'Free AI SEO Audit Tool: 2026 Comparison'. The prospect is seo-skill.com (a popular SEO training site with a 'resources' page). Write a 4-sentence outreach email: 1) compliment, 2) mention our resource, 3) explain audience benefit, 4) one-line about GeoXylia.",
+    "context_excerpt": "",
+    "attempted_solution": "",
+    "outcome": "unknown",
+    "reference_kind": "rubric",
+    "judge": {},
+    "tags": [
+      "research-cron-output"
+    ],
+    "source_sessions": [],
+    "split": "train"
+  },
+  {
+    "id": "rc-05",
+    "reference": "1) DO MORE: AI citation / LLM-mention topics \u2014 the 0.9% CTR at position 9.4 means we're visible but need richer answer capsules to lift CTR. Target 2x posts/week on this cluster.\n2) PAUSE: Pure schema-markup how-tos \u2014 'Schema Markup for SEO' has 0 clicks at position 41, the audience isn't searching this way. Rework as 'How to appear in AI answers' framing.\n3) TEST: 'Perplexity vs ChatGPT citation rates for [niche]' \u2014 unexplored angle, could capture comparison-intent traffic.",
+    "rubric": "Score 1.0 if all 3 are specific (not generic), cite actual data from the prompt, and contain a clear actionable change. Score 0.5 if 2 of 3 are specific. Score 0.0 if generic advice or no data citations.",
+    "project": "research-cron-output",
+    "intent": "Performance \u2192 Strategy feedback loop. Last week's top blog post was 'AI Citation Audit: Does Your Site Appear in ChatGPT?' with 4,200 impressions and 38 clicks (CTR 0.9%, position 9.4). The bottom post was 'Schema Markup for SEO: A 2026 Guide' with 110 impressions and 0 clicks (CTR 0%, position 41). Write 3 specific strategy adjustments: 1) what to do more of, 2) what to pause, 3) what new topic to test.",
+    "context_excerpt": "",
+    "attempted_solution": "",
+    "outcome": "unknown",
+    "reference_kind": "rubric",
+    "judge": {},
+    "tags": [
+      "research-cron-output"
+    ],
+    "source_sessions": [],
+    "split": "val"
+  }
+]

plugins/openclaw/tests/wiki-tasks.json

@@ -0,0 +1,70 @@
+[
+  {
+    "id": "wk-01",
+    "reference": "1. What GEO is and isn't (define vs SEO/AEO, dispel the 'just add FAQ' myth)\n2. The 3 citation mechanisms LLMs use (RAG, fine-tuning, in-context; weight each)\n3. The 2026 citation data (real statistics from Profound/Otterly/Peec; what % of queries get citations)\n4. The action framework (a 5-step audit-and-fix process, concrete)\n5. Measurement (which metrics actually predict citation lift; vanity vs real)",
+    "rubric": "Score 1.0 if 5 sections, in a logical order, each with a substantive (not generic) purpose, and the section content is GEO-specific (not generic SEO). Score 0.5 if 5 sections but 1-2 are generic. Score 0.0 if wrong number of sections or wrong order.",
+    "project": "wiki-canonical-guide",
+    "intent": "Wiki canonical guide: 'GEO 2026 Standards'. Audience: a mid-level SEO specialist who has heard of GEO but not done it. Tone: technical, evidence-driven, no fluff. Length target: 1500-2200 words. Outline the 5 sections that should appear in order. For each, give a 1-sentence sub-purpose.",
+    "context_excerpt": "",
+    "attempted_solution": "",
+    "outcome": "unknown",
+    "reference_kind": "rubric",
+    "judge": {},
+    "tags": [
+      "wiki-canonical-guide"
+    ],
+    "source_sessions": [],
+    "split": "val"
+  },
+  {
+    "id": "wk-02",
+    "reference": "Yes, add inbound links. (1) geo-2026-standards.md \u2192 '## Action Framework' section, anchor: 'platform-specific citation rules' \u2014 natural since GEO standards reference ChatGPT/Perplexity behavior. (2) seo-2026-standards.md \u2192 '## AI Overviews' section, anchor: 'AI platform citations' \u2014 links to the mechanism guide. (3) content-strategy.md \u2192 '## Content Types' section, anchor: 'per-platform citation' \u2014 content strategy needs to know which platform favors which content.",
+    "rubric": "Score 1.0 if all 3 inbound links proposed with specific section + natural anchor text, demonstrating the link solves a real navigational gap (not just SEO-link-building). Score 0.5 if 2 of 3 are well-placed. Score 0.0 if generic anchors like 'click here' or no specific sections named.",
+    "project": "wiki-canonical-guide",
+    "intent": "Cross-link audit. The wiki page 'ai-platform-citation-guide.md' has 4 outbound links to other wiki pages, but no inbound links from: 'geo-2026-standards.md', 'seo-2026-standards.md', 'content-strategy.md'. Should we add inbound links? In which page should each inbound link go, and what anchor text would be natural?",
+    "context_excerpt": "",
+    "attempted_solution": "",
+    "outcome": "unknown",
+    "reference_kind": "rubric",
+    "judge": {},
+    "tags": [
+      "wiki-canonical-guide"
+    ],
+    "source_sessions": [],
+    "split": "val"
+  },
+  {
+    "id": "wk-03",
+    "reference": "Priorities:\n1. Refresh 'geo-glossary.md' (last update 2026-04-12, 63 days) \u2014 add new terms like RAG, in-context citation, agentic SEO.\n2. Refresh 'competitor-pricing.md' (last update 2026-05-01, 44 days) \u2014 Profound raised enterprise tier.\n3. No structural fixes needed.\n\nTelegram: 'Wiki lint: 2 stale pages flagged (geo-glossary 63d, competitor-pricing 44d). No broken links. Both need refresh this week.'",
+    "rubric": "Score 1.0 if both stale pages correctly identified with specific (not generic) refresh notes, and Telegram summary is under 40 words with the right action. Score 0.5 if stale pages identified but refresh notes are vague. Score 0.0 if missing stale pages or Telegram over 40 words.",
+    "project": "wiki-canonical-guide",
+    "intent": "Wiki lint report. Today's scan: 14 wiki pages, 2 with 'Updated' dates > 30 days old ('geo-glossary.md' and 'competitor-pricing.md'), 0 broken internal links, 0 missing YAML frontmatter. Output: 1) prioritized action list, 2) Telegram summary (max 40 words).",
+    "context_excerpt": "",
+    "attempted_solution": "",
+    "outcome": "unknown",
+    "reference_kind": "rubric",
+    "judge": {},
+    "tags": [
+      "wiki-canonical-guide"
+    ],
+    "source_sessions": [],
+    "split": "train"
+  },
+  {
+    "id": "wk-04",
+    "reference": "Index rebuilt: 14 wiki pages registered in _index.md (was 12 \u2014 added competitor-pricing-rev2 and citations-q2-2026).\nQuestion for Ethan: should 'competitor-pricing.md' and 'competitor-pricing-rev2.md' be merged? They're 78% similar in content.",
+    "rubric": "Score 1.0 if both sentences are accurate (count matches, names are plausible) and the question identifies a real consolidation opportunity (not a fabricated one). Score 0.5 if structure is right but content vague. Score 0.0 if wrong format or no question.",
+    "project": "wiki-canonical-guide",
+    "intent": "Index rebuild check. Run `python3 ~/agent-shared/scripts/update-index.py` (assume it works). After the run, the new wiki/_index.md should list all 14 pages. Generate a 2-sentence confirmation message + 1 question for Ethan to verify.",
+    "context_excerpt": "",
+    "attempted_solution": "",
+    "outcome": "unknown",
+    "reference_kind": "rubric",
+    "judge": {},
+    "tags": [
+      "wiki-canonical-guide"
+    ],
+    "source_sessions": [],
+    "split": "train"
+  }
+]

plugins/run-sleep.sh

@@ -30,12 +30,17 @@ if [ -z "${REPO_ROOT:-}" ]; then
 fi
 
 PY=""
-for cand in python3.12 python3.11 python3.10 python3; do
+# Allow explicit Python override (useful on macOS with old system Python).
-  if command -v "$cand" >/dev/null 2>&1; then
+if [ -n "${SKILLOPT_SLEEP_PYTHON:-}" ]; then
-    ver="$("$cand" -c 'import sys; print("%d%d" % sys.version_info[:2])' 2>/dev/null || echo 0)"
+  PY="$SKILLOPT_SLEEP_PYTHON"
-    if [ "${ver:-0}" -ge 310 ]; then PY="$cand"; break; fi
+else
-  fi
+  for cand in python3.12 python3.11 python3.10 python3; do
-done
+    if command -v "$cand" >/dev/null 2>&1; then
+      ver="$("$cand" -c 'import sys; print("%d%d" % sys.version_info[:2])' 2>/dev/null || echo 0)"
+      if [ "${ver:-0}" -ge 310 ]; then PY="$cand"; break; fi
+    fi
+  done
+fi
 if [ -z "$PY" ]; then
   echo "[sleep] ERROR: need Python >= 3.10 (found none)." >&2
   exit 1

pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "skillopt"
-version = "0.1.0"
+version = "0.2.0"
 description = "SkillOpt: Agentic Skill Optimization via Reflective Training Loops"
 readme = "README.md"
 license = {text = "MIT"}
@@ -37,9 +37,11 @@ dependencies = [
 # Benchmark-specific dependencies
 alfworld = ["alfworld>=0.4.0", "gymnasium>=0.29.0"]
 # Claude model backend
-claude = ["claude-agent-sdk>=0.1.0"]
+claude = ["claude-agent-sdk>=0.1.0", "json_repair>=0.61.0"]
 # Qwen local model backend (via vLLM)
-qwen = ["vllm>=0.4.0"]
+qwen = ["vllm>=0.4.0", "json_repair>=0.61.0"]
+# SearchQA data materialization
+searchqa = ["datasets>=2.18.0"]
 # Documentation site
 docs = ["mkdocs-material>=9.5.0", "mkdocstrings[python]>=0.24.0"]
 # WebUI dashboard
@@ -51,11 +53,13 @@ all = [
     "alfworld>=0.4.0",
     "gymnasium>=0.29.0",
     "claude-agent-sdk>=0.1.0",
+    "json_repair>=0.61.0",
 ]
 
 [project.scripts]
 skillopt-train = "scripts.train:main"
 skillopt-eval = "scripts.eval_only:main"
+skillopt-sleep = "skillopt_sleep.__main__:main"
 
 [project.urls]
 Homepage = "https://github.com/microsoft/SkillOpt"
@@ -64,9 +68,10 @@ Repository = "https://github.com/microsoft/SkillOpt"
 Issues = "https://github.com/microsoft/SkillOpt/issues"
 
 [tool.setuptools.packages.find]
-# skillopt* = the research package; skillopt_sleep = the open-source Sleep tool
+# skillopt*      = the research package
-# (decoupled, zero dependency on the research code).
+# skillopt_sleep = the open-source Sleep tool (decoupled, zero research dep)
-include = ["skillopt", "skillopt.*", "skillopt_sleep", "skillopt_sleep.*", "scripts*"]
+# skillopt_webui = the Gradio dashboard (installed via the `webui` extra)
+include = ["skillopt", "skillopt.*", "skillopt_sleep", "skillopt_sleep.*", "skillopt_webui", "skillopt_webui.*", "scripts*"]
 
 [tool.ruff]
 line-length = 120

requirements.txt

@@ -17,6 +17,12 @@ httpx>=0.27.0
 # ── Optional: Qwen local model (via vLLM) ────────
 # vllm>=0.4.0
 
+# ── Optional: tolerant JSON repair for free-form output from non-OpenAI
+#    backends (Claude/Qwen). Without it extract_json() falls back safely and
+#    drops a malformed analyst edit instead of repairing it. Installed by the
+#    `claude`, `qwen`, and `all` extras in pyproject.toml.
+# json_repair>=0.61.0
+
 # ── Optional: WebUI dashboard ────────────────────
 # gradio>=4.0.0
 

scripts/eval_only.py

@@ -28,6 +28,8 @@ from skillopt.model import (
     configure_azure_openai,
     configure_claude_code_exec,
     configure_codex_exec,
+    configure_qwen_chat,
+    configure_minimax_chat,
     set_reasoning_effort,
     set_target_backend,
     set_target_deployment,
@@ -137,7 +139,7 @@ def parse_args() -> argparse.Namespace:
     # Legacy flat overrides
     p.add_argument("--env", type=str)
     p.add_argument("--backend", type=str,
-                   choices=["azure_openai", "codex", "codex_exec", "claude", "claude_chat", "claude_code_exec"])
+                   choices=["azure_openai", "codex", "codex_exec", "claude", "claude_chat", "claude_code_exec", "minimax", "minimax_chat"])
     p.add_argument("--optimizer_model", type=str)
     p.add_argument("--target_model", type=str)
     p.add_argument("--optimizer_backend", type=str)
@@ -179,6 +181,12 @@ def parse_args() -> argparse.Namespace:
     p.add_argument("--claude_code_exec_use_sdk", type=str)
     p.add_argument("--claude_code_exec_effort", type=str)
     p.add_argument("--claude_code_exec_max_thinking_tokens", type=int)
+    p.add_argument("--minimax_base_url", type=str)
+    p.add_argument("--minimax_api_key", type=str)
+    p.add_argument("--minimax_model", type=str)
+    p.add_argument("--minimax_temperature", type=float)
+    p.add_argument("--minimax_max_tokens", type=int)
+    p.add_argument("--minimax_enable_thinking", type=_BOOL)
     p.add_argument("--out_root", type=str)
     p.add_argument("--data_path", type=str)
     p.add_argument("--split_mode", type=str,
@@ -254,6 +262,12 @@ def main() -> None:
                 "claude_code_exec_use_sdk": "model.claude_code_exec_use_sdk",
                 "claude_code_exec_effort": "model.claude_code_exec_effort",
                 "claude_code_exec_max_thinking_tokens": "model.claude_code_exec_max_thinking_tokens",
+                "minimax_base_url": "model.minimax_base_url",
+                "minimax_api_key": "model.minimax_api_key",
+                "minimax_model": "model.minimax_model",
+                "minimax_temperature": "model.minimax_temperature",
+                "minimax_max_tokens": "model.minimax_max_tokens",
+                "minimax_enable_thinking": "model.minimax_enable_thinking",
                 "seed": "train.seed",
                 "test_env_num": "evaluation.test_env_num",
                 "env": "env.name",
@@ -311,6 +325,9 @@ def main() -> None:
         elif backend == "claude_code_exec":
             cfg.setdefault("optimizer_backend", "openai_chat")
             cfg.setdefault("target_backend", "claude_code_exec")
+        elif backend in {"minimax", "minimax_chat"}:
+            cfg.setdefault("optimizer_backend", "openai_chat")
+            cfg.setdefault("target_backend", "minimax_chat")
         else:
             cfg.setdefault("optimizer_backend", "openai_chat")
             cfg.setdefault("target_backend", "openai_chat")
@@ -336,6 +353,15 @@ def main() -> None:
             and not _has_model_override("model.target", "target_model")
         ):
             cfg["target_model"] = default_model_for_backend("claude_chat")
+    if cfg.get("target_backend") == "minimax_chat":
+        if (
+            str(cfg.get("target_model", "") or "").strip() in _OPENAI_DEFAULT_MODEL_SENTINELS
+            and not _has_model_override("model.target", "target_model")
+        ):
+            cfg["target_model"] = (
+                cfg.get("minimax_model")
+                or default_model_for_backend("minimax_chat")
+            )
 
     if not cfg.get("out_root"):
         env = cfg.get("env", "unknown")
@@ -401,6 +427,30 @@ def main() -> None:
         effort=cfg.get("claude_code_exec_effort", cfg.get("reasoning_effort", "medium")),
         max_thinking_tokens=cfg.get("claude_code_exec_max_thinking_tokens", 16384),
     )
+    configure_qwen_chat(
+        base_url=cfg.get("qwen_chat_base_url") or None,
+        api_key=cfg.get("qwen_chat_api_key") or None,
+        temperature=cfg.get("qwen_chat_temperature"),
+        timeout_seconds=cfg.get("qwen_chat_timeout_seconds"),
+        max_tokens=cfg.get("qwen_chat_max_tokens"),
+        enable_thinking=cfg.get("qwen_chat_enable_thinking"),
+        target_base_url=cfg.get("target_qwen_chat_base_url") or None,
+        target_api_key=cfg.get("target_qwen_chat_api_key") or None,
+        target_temperature=cfg.get("target_qwen_chat_temperature"),
+        target_timeout_seconds=cfg.get("target_qwen_chat_timeout_seconds"),
+        target_max_tokens=cfg.get("target_qwen_chat_max_tokens"),
+        target_enable_thinking=cfg.get("target_qwen_chat_enable_thinking"),
+    )
+    configure_minimax_chat(
+        base_url=cfg.get("minimax_base_url") or None,
+        api_key=cfg.get("minimax_api_key") or None,
+        temperature=cfg.get("minimax_temperature"),
+        max_tokens=cfg.get("minimax_max_tokens"),
+        enable_thinking=cfg.get("minimax_enable_thinking"),
+    )
+    minimax_model_cfg = cfg.get("minimax_model")
+    if minimax_model_cfg and cfg.get("target_backend") == "minimax_chat":
+        set_target_deployment(str(minimax_model_cfg))
     set_reasoning_effort(cfg.get("reasoning_effort", "") or None)
 
     # Build adapter

scripts/materialize_searchqa.py

@@ -0,0 +1,148 @@
+"""Materialize runnable SearchQA splits from the released ID manifest."""
+
+from __future__ import annotations
+
+import argparse
+import json
+from collections.abc import Iterable, Mapping
+from pathlib import Path
+
+PROJECT_ROOT = Path(__file__).resolve().parent.parent
+SPLITS = ("train", "val", "test")
+REQUIRED_FIELDS = ("question", "context", "answers")
+
+
+def parse_args() -> argparse.Namespace:
+    parser = argparse.ArgumentParser(description=__doc__)
+    parser.add_argument(
+        "--manifest-dir",
+        type=Path,
+        default=PROJECT_ROOT / "data" / "searchqa_id_split",
+        help="Directory containing train/val/test ID manifests.",
+    )
+    parser.add_argument(
+        "--output-dir",
+        type=Path,
+        default=PROJECT_ROOT / "data" / "searchqa_split",
+        help="Directory to write runnable train/val/test splits.",
+    )
+    parser.add_argument(
+        "--dataset",
+        default="lucadiliello/searchqa",
+        help="Hugging Face dataset repository to load.",
+    )
+    return parser.parse_args()
+
+
+def load_manifest_ids(manifest_dir: Path) -> dict[str, list[str]]:
+    split_ids = {}
+    for split in SPLITS:
+        path = manifest_dir / split / "items.json"
+        with path.open(encoding="utf-8") as file:
+            items = json.load(file)
+        split_ids[split] = [str(item["id"]) for item in items]
+    return split_ids
+
+
+def _iter_dataset_rows(dataset: Mapping[str, Iterable[dict]]) -> Iterable[dict]:
+    for source_split in dataset.values():
+        yield from source_split
+
+
+def _normalize_row(row: dict) -> dict:
+    try:
+        key = str(row["key"])
+    except KeyError as exc:
+        raise ValueError("SearchQA source row is missing required field: key") from exc
+
+    missing = [field for field in REQUIRED_FIELDS if field not in row]
+    if missing:
+        raise ValueError(f"SearchQA source row {key!r} is missing required fields: {', '.join(missing)}")
+
+    return {
+        "id": key,
+        "question": row["question"],
+        "context": row["context"],
+        "answers": row["answers"],
+    }
+
+
+def materialize_searchqa_splits(
+    manifest_dir: Path,
+    output_dir: Path,
+    dataset: Mapping[str, Iterable[dict]],
+    *,
+    dataset_name: str,
+) -> dict[str, int]:
+    """Write runnable SearchQA train/val/test splits from a source dataset."""
+    manifest_dir = manifest_dir.resolve()
+    output_dir = output_dir.resolve()
+    split_ids = load_manifest_ids(manifest_dir)
+    wanted_ids = {item_id for ids in split_ids.values() for item_id in ids}
+
+    selected: dict[str, dict] = {}
+    duplicate_ids: set[str] = set()
+    for row in _iter_dataset_rows(dataset):
+        key = str(row.get("key", ""))
+        if key not in wanted_ids:
+            continue
+        if key in selected:
+            duplicate_ids.add(key)
+            continue
+        selected[key] = _normalize_row(row)
+
+    if duplicate_ids:
+        preview = ", ".join(sorted(duplicate_ids)[:5])
+        raise ValueError(f"SearchQA source dataset contains duplicate manifest IDs. First IDs: {preview}")
+
+    missing = sorted(wanted_ids - selected.keys())
+    if missing:
+        preview = ", ".join(missing[:5])
+        raise RuntimeError(f"SearchQA source dataset is missing {len(missing)} manifest IDs. First IDs: {preview}")
+
+    counts = {}
+    for split, ids in split_ids.items():
+        items = [selected[item_id] for item_id in ids]
+        split_dir = output_dir / split
+        split_dir.mkdir(parents=True, exist_ok=True)
+        with (split_dir / "items.json").open("w", encoding="utf-8") as file:
+            json.dump(items, file, ensure_ascii=False, indent=2)
+        counts[split] = len(items)
+
+    manifest = {
+        "source_manifest_dir": str(manifest_dir),
+        "source_dataset": dataset_name,
+        "counts": counts,
+        "item_fields": ["id", *REQUIRED_FIELDS],
+    }
+    with (output_dir / "split_manifest.json").open("w", encoding="utf-8") as file:
+        json.dump(manifest, file, ensure_ascii=False, indent=2)
+
+    return counts
+
+
+def main() -> None:
+    args = parse_args()
+    try:
+        from datasets import load_dataset
+    except ImportError as exc:
+        raise SystemExit(
+            "Missing dependency 'datasets'. Install it with:\n"
+            "  python -m pip install 'skillopt[searchqa]'\n"
+            "or:\n"
+            "  python -m pip install datasets"
+        ) from exc
+
+    print(f"Loading {args.dataset}...")
+    dataset = load_dataset(args.dataset)
+    counts = materialize_searchqa_splits(
+        args.manifest_dir,
+        args.output_dir,
+        dataset,
+        dataset_name=args.dataset,
+    )
+    print(f"Wrote SearchQA splits to {args.output_dir.resolve()}: {counts}")
+
+
+if __name__ == "__main__":
+    main()

skillopt.html

@@ -2416,14 +2416,11 @@
 
       <div class="bibtex-box">
         <button class="copy-btn" type="button" onclick="copyBibtex(this)">Copy</button>
-        <pre><code>@misc{yang2026skilloptexecutivestrategyselfevolving,
+        <pre><code>@article{yang2026skillopt,
-      title={SkillOpt: Executive Strategy for Self-Evolving Agent Skills}, 
+  title={Skillopt: Executive strategy for self-evolving agent skills},
-      author={Yifan Yang and Ziyang Gong and Weiquan Huang and Qihao Yang and Ziwei Zhou and Zisu Huang and Yan Li and Xuemei Gao and Qi Dai and Bei Liu and Kai Qiu and Yuqing Yang and Dongdong Chen and Xue Yang and Chong Luo},
+  author={Yang, Yifan and Gong, Ziyang and Huang, Weiquan and Yang, Qihao and Zhou, Ziwei and Huang, Zisu and Li, Yan and Gao, Xuemei and Dai, Qi and Liu, Bei and others},
-      year={2026},
+  journal={arXiv preprint arXiv:2605.23904},
-      eprint={2605.23904},
+  year={2026}
-      archivePrefix={arXiv},
-      primaryClass={cs.AI},
-      url={https://arxiv.org/abs/2605.23904}, 
 }</code></pre>
       </div>
     </section>

skillopt/__init__.py

@@ -12,7 +12,7 @@ Pipeline stages:
   6. Evaluate  — validate candidate skill, accept/reject
 """
 
-__version__ = "0.1.0"
+__version__ = "0.2.0"
 
 from skillopt.types import (  # noqa: F401
     BatchSpec,

skillopt/engine/trainer.py

@@ -2133,6 +2133,7 @@ class ReflACTTrainer:
             )
             print(f"  Test items: {test_n}")
             baseline_test_dir = os.path.join(out_root, "test_eval_baseline")
+            os.makedirs(baseline_test_dir, exist_ok=True)
             baseline_test_results = adapter.rollout(test_env, skill_init, baseline_test_dir)
             baseline_test_hard, baseline_test_soft = compute_score(baseline_test_results)
             baseline_buckets = _compute_task_type_buckets(baseline_test_results, task_types)
@@ -2167,6 +2168,7 @@ class ReflACTTrainer:
             )
             print(f"  Test items: {test_n2}")
             test_dir = os.path.join(out_root, "test_eval")
+            os.makedirs(test_dir, exist_ok=True)
             test_results = adapter.rollout(test_env2, best_skill, test_dir)
             test_hard, test_soft = compute_score(test_results)
             best_buckets = _compute_task_type_buckets(test_results, task_types)
@@ -2230,6 +2232,7 @@ class ReflACTTrainer:
                     )
                     print(f"  Test items: {test_n3}")
                     final_test_dir = os.path.join(out_root, "test_eval_final")
+                    os.makedirs(final_test_dir, exist_ok=True)
                     final_test_results = adapter.rollout(test_env3, current_skill, final_test_dir)
                     final_test_hard, final_test_soft = compute_score(final_test_results)
                     final_buckets = _compute_task_type_buckets(final_test_results, task_types)

skillopt/envs/_template/README.md

@@ -5,8 +5,8 @@ This directory provides scaffold files for adding a new benchmark to SkillOpt.
 ## Files
 
 - `env_template.py` — Environment adapter template (subclasses
-  `EnvAdapter`; implements the 5 abstract methods so the file is
+  `EnvAdapter`; implements the 4 abstract methods so the file is
-  instantiable out of the box).
+  instantiable out of the box — `reflect` is inherited).
 - `loader_template.py` — Data loader template (subclasses
   `SplitDataLoader`; implements `load_split_items` for `.json`/`.jsonl`).
 - `config_template.yaml` — Config file template.
@@ -21,15 +21,15 @@ This directory provides scaffold files for adding a new benchmark to SkillOpt.
    ```bash
    cd skillopt/envs/your_benchmark
    mv env_template.py    adapter.py
-   mv loader_template.py loader.py
+   mv loader_template.py dataloader.py
    ```
    …and inside each file rename the classes
    (`TemplateBenchmarkEnv → YourBenchmarkAdapter`,
    `TemplateBenchmarkLoader → YourBenchmarkLoader`)
    and fix the cross-import in `adapter.py`.
 3. **Implement the TODO blocks** inside `adapter.py:rollout` and the
-   `_normalize_item` helper in `loader.py`. If you want real reflection,
+   `_normalize_item` helper in `dataloader.py`. (`reflect` is inherited from
-   uncomment the `run_minibatch_reflect` block in `adapter.py:reflect`.
+   `EnvAdapter`; override it only for custom reflection logic.)
 4. **Register** the adapter — add a `try / except ImportError` block in
    `scripts/train.py`'s `_register_builtins()` mapping the registry key
    to your `YourBenchmarkAdapter` class. There is no

skillopt/envs/_template/env_template.py

@@ -14,13 +14,9 @@ For a fully worked example see ``skillopt/envs/officeqa/``.
 """
 from __future__ import annotations
 
-import os
-
 from skillopt.datasets.base import BatchSpec
 from skillopt.envs.base import EnvAdapter
 from skillopt.envs._template.loader_template import TemplateBenchmarkLoader
-# When you wire in real reflection, also import:
-# from skillopt.gradient.reflect import run_minibatch_reflect
 
 
 class TemplateBenchmarkEnv(EnvAdapter):
@@ -131,53 +127,12 @@ class TemplateBenchmarkEnv(EnvAdapter):
             )
         return results
 
-    # ── Reflect: turn rollout results into patch dicts ─────────────────
+    # ── Reflect (inherited) ─────────────────────────────────────────────
-
+    #
-    def reflect(
+    # ``reflect`` is inherited from ``EnvAdapter``: the default delegates to
-        self,
+    # ``skillopt.gradient.reflect.run_minibatch_reflect`` using your
-        results: list[dict],
+    # ``analyst_error_*`` / ``analyst_success_*`` prompts. You do NOT need to
-        skill_content: str,
+    # implement it — override only if your benchmark needs custom reflection.
-        out_dir: str,
-        **kwargs,
-    ) -> list[dict | None]:
-        """
-        Turn rollouts into a list of raw patch dicts (or None to drop).
-
-        Each non-None dict MUST have:
-          - "patch":       {"edits": [...]}     a Patch.to_dict() payload
-          - "source_type": "failure" | "success"
-
-        Most benchmarks delegate to
-        :func:`skillopt.gradient.reflect.run_minibatch_reflect` which
-        will call the optimizer model with the
-        ``analyst_error_*`` / ``analyst_success_*`` prompts. To enable it,
-        uncomment the import above and call:
-
-            from skillopt.gradient.reflect import run_minibatch_reflect
-            return run_minibatch_reflect(
-                results=results,
-                skill_content=skill_content,
-                prediction_dir=kwargs.get(
-                    "prediction_dir", os.path.join(out_dir, "predictions")
-                ),
-                patches_dir=kwargs.get(
-                    "patches_dir", os.path.join(out_dir, "patches")
-                ),
-                workers=self.analyst_workers,
-                failure_only=self.failure_only,
-                minibatch_size=self.minibatch_size,
-                edit_budget=self.edit_budget,
-                random_seed=kwargs.get("random_seed"),
-                error_system=self.get_error_minibatch_prompt(),
-                success_system=self.get_success_minibatch_prompt(),
-                step_buffer_context=kwargs.get("step_buffer_context", ""),
-                update_mode=getattr(self, "_cfg", {}).get(
-                    "skill_update_mode", "patch"
-                ),
-            )
-        """
-        # Template default: produce no patches (no-op trainer step).
-        return [None for _ in results]
 
     # ── Stratification hint ────────────────────────────────────────────
 

skillopt/envs/alfworld/adapter.py

@@ -17,7 +17,6 @@ from skillopt.envs.alfworld.rollout import (
     run_alfworld_batch,
     TASKS,
 )
-from skillopt.gradient.reflect import run_minibatch_reflect
 from skillopt.utils import compute_score
 
 
@@ -425,35 +424,5 @@ class ALFWorldAdapter(EnvAdapter):
             all_results.extend(chunk_results)
         return all_results
 
-    def reflect(
-        self,
-        results: list[dict],
-        skill_content: str,
-        out_dir: str,
-        **kwargs,
-    ) -> list[dict | None]:
-        prediction_dir = kwargs.get("prediction_dir", os.path.join(out_dir, "predictions"))
-        patches_dir = kwargs.get("patches_dir", os.path.join(out_dir, "patches"))
-        random_seed = kwargs.get("random_seed")
-        step_buffer_context = kwargs.get("step_buffer_context", "")
-        meta_skill_context = kwargs.get("meta_skill_context", "")
-
-        return run_minibatch_reflect(
-            results=results,
-            skill_content=skill_content,
-            prediction_dir=prediction_dir,
-            patches_dir=patches_dir,
-            workers=self.analyst_workers,
-            failure_only=self.failure_only,
-            minibatch_size=self.minibatch_size,
-            edit_budget=self.edit_budget,
-            random_seed=random_seed,
-            error_system=self.get_error_minibatch_prompt(),
-            success_system=self.get_success_minibatch_prompt(),
-            step_buffer_context=step_buffer_context,
-            meta_skill_context=meta_skill_context,
-        )
-
-
     def get_task_types(self) -> list[str]:
         return list(TASKS)

skillopt/envs/alfworld/rollout.py

@@ -7,12 +7,10 @@ Provides:
 """
 from __future__ import annotations
 
+import concurrent.futures
 import json
 import os
 import re
-import sys
-import concurrent.futures
-import numpy as np
 
 from skillopt.model import chat_target
 
@@ -65,6 +63,25 @@ def _append_diagnostic_instruction(prompt: str, diagnostic_instruction: str) ->
     return f"{prompt}\n\n## Training Readout\n{diagnostic_instruction.strip()}\n"
 
 
+def _resolve_alfworld_gamefile(gamefile: str) -> str:
+    path = os.path.expanduser(os.path.expandvars(str(gamefile)))
+    if os.path.isabs(path):
+        return path
+
+    data_root = os.environ.get("ALFWORLD_DATA", "").strip()
+    if not data_root:
+        return path
+
+    root = os.path.expanduser(os.path.expandvars(data_root))
+    return os.path.abspath(os.path.join(root, path))
+
+
+def _resolve_alfworld_gamefiles(gamefiles: list[str] | None) -> list[str] | None:
+    if gamefiles is None:
+        return None
+    return [_resolve_alfworld_gamefile(gamefile) for gamefile in gamefiles]
+
+
 # ── Environment builder ──────────────────────────────────────────────────────
 
 
@@ -86,9 +103,10 @@ def build_alfworld_env(
     Returns:
         env_manager: AlfWorldEnvironmentManager instance
     """
-    from omegaconf import OmegaConf
     from functools import partial
 
+    from omegaconf import OmegaConf
+
     from skillopt.envs.alfworld.vendor.alfworld_envs import build_alfworld_envs
     from skillopt.envs.alfworld.vendor.alfworld_projection import alfworld_projection
     from skillopt.envs.alfworld.vendor.env_manager import AlfWorldEnvironmentManager
@@ -97,6 +115,7 @@ def build_alfworld_env(
 
     alf_config_path = os.path.join(HERE, "vendor", "config_tw.yaml")
     env_kwargs = {"eval_dataset": eval_dataset}
+    resolved_gamefiles = _resolve_alfworld_gamefiles(specific_gamefiles)
 
     envs = build_alfworld_envs(
         alf_config_path,
@@ -106,7 +125,7 @@ def build_alfworld_env(
         is_train=is_train,
         env_kwargs=env_kwargs,
         resources_per_worker=None,
-        gamefiles=specific_gamefiles,
+        gamefiles=resolved_gamefiles,
     )
 
     config = OmegaConf.create(
@@ -222,7 +241,7 @@ def run_alfworld_batch(
                 if _extract_action(response) is None:
                     return idx, "<think>missing action tag</think><action>look</action>"
                 return idx, response
-            except Exception as e:
+            except Exception:
                 return idx, "<think>error</think><action>look</action>"
 
         executor = concurrent.futures.ThreadPoolExecutor(max_workers=max_api_workers)

skillopt/envs/base.py

@@ -231,7 +231,6 @@ class EnvAdapter(ABC):
             (float 0-1). May include env-specific fields.
         """
 
-    @abstractmethod
     def reflect(
         self,
         results: list[dict],
@@ -241,15 +240,36 @@ class EnvAdapter(ABC):
     ) -> list[dict | None]:
         """Analyze rollout results and produce patches.
 
+        Default implementation: delegate to the shared minibatch reflect
+        stage. Every built-in benchmark uses this unchanged — override only
+        if your environment needs custom reflection logic.
+
         Each returned dict conforms to :class:`~skillopt.types.RawPatch`:
         ``"patch"`` (with ``"edits"`` list) + ``"source_type"``
-        (``"failure"`` or ``"success"``).
+        (``"failure"`` or ``"success"``); ``None`` entries are filtered out.
-
-        Returns
-        -------
-        list[dict | None]
-            Raw analyst outputs; ``None`` entries are filtered out.
         """
+        from skillopt.gradient.reflect import run_minibatch_reflect
+
+        return run_minibatch_reflect(
+            results=results,
+            skill_content=skill_content,
+            prediction_dir=kwargs.get(
+                "prediction_dir", os.path.join(out_dir, "predictions")
+            ),
+            patches_dir=kwargs.get(
+                "patches_dir", os.path.join(out_dir, "patches")
+            ),
+            workers=self.analyst_workers,
+            failure_only=self.failure_only,
+            minibatch_size=self.minibatch_size,
+            edit_budget=self.edit_budget,
+            random_seed=kwargs.get("random_seed"),
+            error_system=self.get_error_minibatch_prompt(),
+            success_system=self.get_success_minibatch_prompt(),
+            step_buffer_context=kwargs.get("step_buffer_context", ""),
+            meta_skill_context=kwargs.get("meta_skill_context", ""),
+            update_mode=getattr(self, "_cfg", {}).get("skill_update_mode", "patch"),
+        )
 
     @abstractmethod
     def get_task_types(self) -> list[str]:

skillopt/envs/docvqa/adapter.py

@@ -1,12 +1,9 @@
 from __future__ import annotations
 
-import os
-
 from skillopt.datasets.base import BatchSpec
 from skillopt.envs.base import EnvAdapter
 from skillopt.envs.docvqa.dataloader import DocVQADataLoader
 from skillopt.envs.docvqa.rollout import run_batch
-from skillopt.gradient.reflect import run_minibatch_reflect
 
 
 class DocVQAAdapter(EnvAdapter):
@@ -84,28 +81,6 @@ class DocVQAAdapter(EnvAdapter):
             task_timeout=self.exec_timeout,
         )
 
-    def reflect(self, results: list[dict], skill_content: str, out_dir: str, **kwargs) -> list[dict | None]:
-        prediction_dir = kwargs.get("prediction_dir", os.path.join(out_dir, "predictions"))
-        patches_dir = kwargs.get("patches_dir", os.path.join(out_dir, "patches"))
-        random_seed = kwargs.get("random_seed")
-        step_buffer_context = kwargs.get("step_buffer_context", "")
-        return run_minibatch_reflect(
-            results=results,
-            skill_content=skill_content,
-            prediction_dir=prediction_dir,
-            patches_dir=patches_dir,
-            workers=self.analyst_workers,
-            failure_only=self.failure_only,
-            minibatch_size=self.minibatch_size,
-            edit_budget=self.edit_budget,
-            random_seed=random_seed,
-            error_system=self.get_error_minibatch_prompt(),
-            success_system=self.get_success_minibatch_prompt(),
-            step_buffer_context=step_buffer_context,
-            update_mode=getattr(self, "_cfg", {}).get("skill_update_mode", "patch"),
-        )
-
-
     def get_task_types(self) -> list[str]:
         seen: list[str] = []
         for item in self.dataloader.train_items + self.dataloader.val_items + self.dataloader.test_items:

skillopt/envs/livemathematicianbench/adapter.py

@@ -2,10 +2,8 @@
 from __future__ import annotations
 
 import json
-import os
 
 from skillopt.datasets.base import BatchSpec
-from skillopt.gradient.reflect import run_minibatch_reflect
 from skillopt.envs.base import EnvAdapter
 from skillopt.envs.livemathematicianbench.dataloader import LiveMathematicianBenchDataLoader
 from skillopt.envs.livemathematicianbench.rollout import run_batch
@@ -127,36 +125,5 @@ class LiveMathematicianBenchAdapter(EnvAdapter):
             task_timeout=self.exec_timeout,
         )
 
-    def reflect(
-        self,
-        results: list[dict],
-        skill_content: str,
-        out_dir: str,
-        **kwargs,
-    ) -> list[dict | None]:
-        prediction_dir = kwargs.get("prediction_dir", os.path.join(out_dir, "predictions"))
-        patches_dir = kwargs.get("patches_dir", os.path.join(out_dir, "patches"))
-        random_seed = kwargs.get("random_seed")
-        step_buffer_context = kwargs.get("step_buffer_context", "")
-        meta_skill_context = kwargs.get("meta_skill_context", "")
-
-        return run_minibatch_reflect(
-            results=results,
-            skill_content=skill_content,
-            prediction_dir=prediction_dir,
-            patches_dir=patches_dir,
-            workers=self.analyst_workers,
-            failure_only=self.failure_only,
-            minibatch_size=self.minibatch_size,
-            edit_budget=self.edit_budget,
-            random_seed=random_seed,
-            error_system=self.get_error_minibatch_prompt(),
-            success_system=self.get_success_minibatch_prompt(),
-            step_buffer_context=step_buffer_context,
-            meta_skill_context=meta_skill_context,
-            update_mode=getattr(self, "_cfg", {}).get("skill_update_mode", "patch"),
-        )
-
-
     def get_task_types(self) -> list[str]:
         return self.dataloader.get_task_types()

skillopt/envs/officeqa/adapter.py

@@ -6,7 +6,6 @@ from skillopt.datasets.base import BatchSpec
 from skillopt.envs.base import EnvAdapter
 from skillopt.envs.officeqa.dataloader import OfficeQADataLoader
 from skillopt.envs.officeqa.rollout import run_batch
-from skillopt.gradient.reflect import run_minibatch_reflect
 
 
 class OfficeQAAdapter(EnvAdapter):
@@ -104,28 +103,6 @@ class OfficeQAAdapter(EnvAdapter):
             diagnostic_instruction=kwargs.get("diagnostic_instruction", ""),
         )
 
-    def reflect(self, results: list[dict], skill_content: str, out_dir: str, **kwargs) -> list[dict | None]:
-        prediction_dir = kwargs.get("prediction_dir", os.path.join(out_dir, "predictions"))
-        patches_dir = kwargs.get("patches_dir", os.path.join(out_dir, "patches"))
-        random_seed = kwargs.get("random_seed")
-        step_buffer_context = kwargs.get("step_buffer_context", "")
-        return run_minibatch_reflect(
-            results=results,
-            skill_content=skill_content,
-            prediction_dir=prediction_dir,
-            patches_dir=patches_dir,
-            workers=self.analyst_workers,
-            failure_only=self.failure_only,
-            minibatch_size=self.minibatch_size,
-            edit_budget=self.edit_budget,
-            random_seed=random_seed,
-            error_system=self.get_error_minibatch_prompt(),
-            success_system=self.get_success_minibatch_prompt(),
-            step_buffer_context=step_buffer_context,
-            update_mode=getattr(self, "_cfg", {}).get("skill_update_mode", "patch"),
-        )
-
-
     def get_task_types(self) -> list[str]:
         seen: list[str] = []
         for item in self.dataloader.train_items + self.dataloader.val_items + self.dataloader.test_items:

skillopt/envs/searchqa/adapter.py

@@ -2,13 +2,11 @@
 from __future__ import annotations
 
 import json
-import os
 
 from skillopt.datasets.base import BatchSpec
 from skillopt.envs.base import EnvAdapter
 from skillopt.envs.searchqa.dataloader import SearchQADataLoader
 from skillopt.envs.searchqa.rollout import run_batch
-from skillopt.gradient.reflect import run_minibatch_reflect
 from skillopt.model import get_target_backend
 
 
@@ -94,36 +92,5 @@ class SearchQAAdapter(EnvAdapter):
             task_timeout=self.exec_timeout,
         )
 
-    def reflect(
-        self,
-        results: list[dict],
-        skill_content: str,
-        out_dir: str,
-        **kwargs,
-    ) -> list[dict | None]:
-        prediction_dir = kwargs.get("prediction_dir", os.path.join(out_dir, "predictions"))
-        patches_dir = kwargs.get("patches_dir", os.path.join(out_dir, "patches"))
-        random_seed = kwargs.get("random_seed")
-        step_buffer_context = kwargs.get("step_buffer_context", "")
-        meta_skill_context = kwargs.get("meta_skill_context", "")
-
-        return run_minibatch_reflect(
-            results=results,
-            skill_content=skill_content,
-            prediction_dir=prediction_dir,
-            patches_dir=patches_dir,
-            workers=self.analyst_workers,
-            failure_only=self.failure_only,
-            minibatch_size=self.minibatch_size,
-            edit_budget=self.edit_budget,
-            random_seed=random_seed,
-            error_system=self.get_error_minibatch_prompt(),
-            success_system=self.get_success_minibatch_prompt(),
-            step_buffer_context=step_buffer_context,
-            meta_skill_context=meta_skill_context,
-            update_mode=getattr(self, "_cfg", {}).get("skill_update_mode", "patch"),
-        )
-
-
     def get_task_types(self) -> list[str]:
         return ["qa"]

skillopt/envs/searchqa/rollout.py

@@ -13,20 +13,31 @@ from __future__ import annotations
 import json
 import os
 import time
-import traceback
+from collections import Counter
 from concurrent.futures import FIRST_COMPLETED, ThreadPoolExecutor, wait
 
-from skillopt.model import chat_target, get_target_backend, is_target_exec_backend
+from skillopt.envs.searchqa.evaluator import evaluate
+from skillopt.model import chat_target, is_target_exec_backend
 from skillopt.model.codex_harness import prepare_workspace, render_skill_md, run_target_exec
 from skillopt.prompts import load_prompt
-from skillopt.envs.searchqa.evaluator import evaluate
-
 
 # ── Prompt templates ─────────────────────────────────────────────────────────
 
 _MAX_CONTEXT_CHARS = 6000
 
 
+def _raise_on_systemic_failure(results: list[dict]) -> None:
+    """Abort when all rollout rows failed before any agent response."""
+    if not results or not all(row.get("agent_ok") is False for row in results):
+        return
+    reasons = Counter(str(row.get("fail_reason") or "unknown error") for row in results)
+    common_reason, count = reasons.most_common(1)[0]
+    raise RuntimeError(
+        f"SearchQA rollout failed for all {len(results)} items before an agent "
+        f"response ({count}x): {common_reason}"
+    )
+
+
 def _truncate_context(context: str, max_chars: int = _MAX_CONTEXT_CHARS) -> str:
     """Truncate context at [DOC] boundaries to stay within budget."""
     if len(context) <= max_chars:
@@ -379,6 +390,7 @@ def run_batch(
 
     pending = [it for it in items if str(it["id"]) not in done_ids]
     if not pending:
+        _raise_on_systemic_failure(existing)
         return existing
 
     total = len(existing) + len(pending)
@@ -478,4 +490,5 @@ def run_batch(
         finally:
             ex.shutdown(wait=False, cancel_futures=True)
 
+    _raise_on_systemic_failure(results)
     return results

skillopt/envs/spreadsheetbench/adapter.py

@@ -16,7 +16,6 @@ from skillopt.envs.spreadsheetbench.rollout import (
     run_spreadsheet_batch,
     run_spreadsheet_batch_codegen,
 )
-from skillopt.gradient.reflect import run_minibatch_reflect
 from skillopt.model import get_target_backend, is_target_exec_backend
 
 
@@ -156,37 +155,5 @@ class SpreadsheetBenchAdapter(EnvAdapter):
 
         return results
 
-    def reflect(
-        self,
-        results: list[dict],
-        skill_content: str,
-        out_dir: str,
-        **kwargs,
-    ) -> list[dict | None]:
-        """Analyze rollout results and produce patches (minibatch mode)."""
-        prediction_dir = kwargs.get("prediction_dir", os.path.join(out_dir, "predictions"))
-        patches_dir = kwargs.get("patches_dir", os.path.join(out_dir, "patches"))
-        random_seed = kwargs.get("random_seed")
-        step_buffer_context = kwargs.get("step_buffer_context", "")
-        meta_skill_context = kwargs.get("meta_skill_context", "")
-
-        return run_minibatch_reflect(
-            results=results,
-            skill_content=skill_content,
-            prediction_dir=prediction_dir,
-            patches_dir=patches_dir,
-            workers=self.analyst_workers,
-            failure_only=self.failure_only,
-            minibatch_size=self.minibatch_size,
-            edit_budget=self.edit_budget,
-            random_seed=random_seed,
-            error_system=self.get_error_minibatch_prompt(),
-            success_system=self.get_success_minibatch_prompt(),
-            step_buffer_context=step_buffer_context,
-            meta_skill_context=meta_skill_context,
-            update_mode=getattr(self, "_cfg", {}).get("skill_update_mode", "patch"),
-        )
-
-
     def get_task_types(self) -> list[str]:
         return list(TASK_TYPES)

skillopt/model/claude_backend.py

@@ -252,13 +252,25 @@ def _run_claude_print(*, system: str, prompt: str, model: str, tools: list[dict[
         if CLAUDE_SETTING_SOURCES:
             cmd.extend(["--setting-sources", CLAUDE_SETTING_SOURCES])
         if system:
-            cmd.extend(["--append-system-prompt", system])
+            # Write the system prompt to a file, not argv: here the skill being
+            # optimized IS the system prompt, and SkillOpt grows it over training,
+            # so past ~30 KB it would re-hit the Windows argv cap (WinError 206).
+            # The CLI reads it via --append-system-prompt-file.
+            system_path = os.path.join(temp_dir, "system_prompt.txt")
+            with open(system_path, "w", encoding="utf-8") as system_fh:
+                system_fh.write(system)
+            cmd.extend(["--append-system-prompt-file", system_path])
         if effort:
             cmd.extend(["--effort", effort])
         structured_output = bool(return_message)
         if structured_output:
             cmd.extend(["--schema", _assistant_message_schema_wrapper()])
-        proc = subprocess.run(cmd + [prompt_for_cli], capture_output=True, text=True, timeout=timeout or 300, cwd=temp_dir)
+        # Feed the prompt via stdin (and the system prompt via a file, above), not
+        # argv: on Windows the whole command line is capped at ~32 KB and large
+        # optimizer prompts / grown skills overflow it → [WinError 206]. Pin UTF-8
+        # so a zh-CN default codepage (cp936) can't raise UnicodeEncodeError on
+        # emoji / non-GBK glyphs before the CLI even starts.
+        proc = subprocess.run(cmd, input=prompt_for_cli, capture_output=True, text=True, encoding="utf-8", errors="replace", timeout=timeout or 300, cwd=temp_dir)
         stderr_text = (proc.stderr or "").strip()
         if proc.returncode != 0:
             _check_claude_error(stderr_text, model)

skillopt/model/codex_backend.py

@@ -328,6 +328,8 @@ def _run_codex_exec(
             command,
             input=prompt,
             text=True,
+            encoding="utf-8",
+            errors="replace",
             capture_output=True,
             timeout=timeout,
             check=False,

skillopt/utils/json_utils.py

@@ -3,6 +3,91 @@ from __future__ import annotations
 
 import json
 import re
+import warnings
+
+
+def _top_level_brace_objects(text: str) -> list[str]:
+    """Return every balanced *top-level* ``{...}`` span in ``text``.
+
+    Fully string/escape aware: braces inside quoted strings are ignored both
+    when scanning for an object start AND while tracking depth inside one, so a
+    ``{`` that appears in prose (e.g. ``'set it to {x}'``) is never mistaken for
+    the start of a JSON object. Used to detect ambiguity: when a response carries
+    more than one top-level object we must not let a repair pass silently pick
+    one — it may pick the wrong (discarded) edit, strictly worse than None.
+    """
+    spans: list[str] = []
+    i, n = 0, len(text)
+    outer_in_str = False
+    outer_esc = False
+    while i < n:
+        ch = text[i]
+        # Skip over braces that live *inside* a quoted string before any object
+        # has started — otherwise a `{` in prose like '"set it to {x}"' is wrongly
+        # treated as an object start, and the repair pass below turns non-JSON
+        # prose into a bogus dict (strictly worse than returning None).
+        if outer_in_str:
+            if outer_esc:
+                outer_esc = False
+            elif ch == "\\":
+                outer_esc = True
+            elif ch == '"':
+                outer_in_str = False
+            i += 1
+            continue
+        if ch == '"':
+            outer_in_str = True
+            i += 1
+            continue
+        if ch != "{":
+            i += 1
+            continue
+        depth = 0
+        in_str = False
+        esc = False
+        start = i
+        while i < n:
+            ch = text[i]
+            if in_str:
+                if esc:
+                    esc = False
+                elif ch == "\\":
+                    esc = True
+                elif ch == '"':
+                    in_str = False
+            elif ch == '"':
+                in_str = True
+            elif ch == "{":
+                depth += 1
+            elif ch == "}":
+                depth -= 1
+                if depth == 0:
+                    spans.append(text[start:i + 1])
+                    i += 1
+                    break
+            i += 1
+        else:
+            break  # unterminated final object
+    return spans
+
+
+def _looks_json_like(span: str) -> bool:
+    """Heuristic: does ``span`` look like an intended JSON object (vs. prose)?
+
+    A genuine JSON object's first non-space character after ``{`` is either ``"``
+    (a string key) or ``}`` (an empty object). Prose pseudo-objects that the
+    repair pass would otherwise fabricate into bogus dicts — ``{op: delete}``,
+    ``{x: 1}`` quoted in single quotes or backticks, etc. — start with a bare
+    word and are rejected. This complements the string-aware scan, which only
+    skips *double*-quoted prose; single-quoted / backticked / unquoted prose
+    braces are caught here instead. Legitimate repair targets (trailing commas,
+    unescaped quotes inside string values) all begin with ``"`` and pass.
+    """
+    inner = span.strip()
+    if not (inner.startswith("{") and inner.endswith("}")):
+        return False
+    after_brace = inner[1:].lstrip()
+    return after_brace[:1] in ('"', '}')
 
 
 def extract_json(text: str) -> dict | None:
@@ -22,6 +107,51 @@ def extract_json(text: str) -> dict | None:
             return json.loads(m.group(0))
         except json.JSONDecodeError:
             pass
+    # Tolerant fallback for non-OpenAI backends (Claude/Qwen, …) whose free-form
+    # JSON strict json.loads rejects — unescaped ASCII quotes inside CJK string
+    # values, trailing commas, etc. Repair so the analyst's edits aren't silently
+    # dropped, but ONLY a single unambiguous object: never feed the greedy `{.*}`
+    # span or the raw text, or json_repair would quietly return one of several
+    # objects (empirically the wrong/last one) — strictly worse than None, which
+    # the caller can detect and retry/skip.
+    #
+    # Pick the candidate FIRST, before importing json_repair, so the optional
+    # dependency only matters (and only warns) when there is genuinely a single
+    # malformed object we could have repaired. Ordinary no-JSON / prose replies
+    # have no candidate and return None silently.
+    candidate = None
+    fenced = re.search(r"```json\s*(.*?)```", text, re.DOTALL)
+    if fenced and len(_top_level_brace_objects(fenced.group(1))) == 1:
+        candidate = fenced.group(1)
+    else:
+        objs = _top_level_brace_objects(text)
+        if len(objs) == 1:
+            candidate = objs[0]
+        # 0 or >1 top-level objects → too ambiguous to repair safely → None
+    if not candidate:
+        return None
+    # Final guard: only repair spans that actually look like an intended JSON
+    # object. Prose pseudo-objects in single quotes / backticks / bare text
+    # (e.g. `{op: delete}`) reach here because the scan only skips double-quoted
+    # prose; repairing them would fabricate a wrong dict (worse than None).
+    if not _looks_json_like(candidate):
+        return None
+    try:
+        from json_repair import repair_json
+    except ModuleNotFoundError:
+        warnings.warn(
+            "json_repair not installed; malformed-JSON recovery disabled — "
+            "a non-OpenAI analyst edit may be silently dropped. pip install json_repair",
+            RuntimeWarning,
+            stacklevel=2,
+        )
+        return None
+    try:
+        repaired = repair_json(candidate, return_objects=True)
+        if isinstance(repaired, dict) and repaired:
+            return repaired
+    except Exception:  # noqa: BLE001 — repair is best-effort
+        pass
     return None
 
 

skillopt_sleep/__init__.py

@@ -17,4 +17,4 @@ Public entry points:
 from __future__ import annotations
 
 __all__ = ["__version__"]
-__version__ = "0.1.0"
+__version__ = "0.2.0"

skillopt_sleep/__main__.py

@@ -9,7 +9,12 @@
 Common flags:
     --project PATH      project to evolve (default: cwd)
     --scope all|invoked harvest scope (default: invoked)
-    --backend mock|anthropic
+    --max-sessions N    cap transcript sessions per run
+    --max-tasks N       cap mined tasks per run
+    --target-skill-path PATH explicit live SKILL.md to stage/adopt
+    --tasks-file PATH   reviewed TaskRecord JSON file to replay instead of harvesting
+    --backend mock|claude|codex|copilot
+    --source claude|codex|auto
     --model NAME
     --lookback-hours N
     --auto-adopt
@@ -25,26 +30,70 @@ from typing import Any, Dict
 
 from skillopt_sleep.config import load_config
 from skillopt_sleep.cycle import run_sleep_cycle
-from skillopt_sleep.harvest import harvest
+from skillopt_sleep.harvest_sources import harvest_for_config
 from skillopt_sleep.mine import mine
+from skillopt_sleep.staging import adopt as adopt_staging
+from skillopt_sleep.staging import latest_staging
 from skillopt_sleep.state import SleepState
-from skillopt_sleep.staging import latest_staging, adopt as adopt_staging
+from skillopt_sleep.tasks_file import load_tasks_file, make_tasks_payload, write_tasks_file
+
+
+def _read_text(path: str) -> str:
+    try:
+        with open(path, encoding="utf-8") as f:
+            return f.read()
+    except Exception:
+        return ""
+
+
+def _report_payload(rep, outcome) -> Dict[str, Any]:
+    return {
+        "night": rep.night,
+        "accepted": rep.accepted,
+        "gate_action": rep.gate_action,
+        "no_edits_reason": getattr(rep, "no_edits_reason", ""),
+        "baseline": rep.baseline_score,
+        "candidate": rep.candidate_score,
+        "n_tasks": rep.n_tasks,
+        "n_sessions": rep.n_sessions,
+        "n_accepted_edits": len(rep.edits),
+        "n_rejected_edits": len(rep.rejected_edits),
+        "edits": [e.__dict__ for e in rep.edits],
+        "rejected_edits": [e.__dict__ for e in rep.rejected_edits],
+        "notes": rep.notes,
+        "staging_dir": outcome.staging_dir,
+        "adopted": outcome.adopted,
+    }
 
 
 def _add_common(p: argparse.ArgumentParser) -> None:
     p.add_argument("--project", default="")
     p.add_argument("--scope", default="", choices=["", "all", "invoked"])
-    p.add_argument("--backend", default="", choices=["", "mock", "claude", "codex"])
+    p.add_argument("--backend", default="", choices=["", "mock", "claude", "codex", "copilot"])
     p.add_argument("--model", default="")
     p.add_argument("--codex-path", default="", help="path to the real @openai/codex binary")
     p.add_argument("--claude-home", default="", help="override ~/.claude (also isolates state)")
-    p.add_argument("--lookback-hours", type=int, default=0)
+    p.add_argument("--codex-home", default="", help="override ~/.codex for archived session harvest")
+    p.add_argument("--source", default="", choices=["", "claude", "codex", "auto"],
+                   help="session transcript source")
+    p.add_argument("--lookback-hours", type=int, default=None,
+                   help="harvest window in hours; 0 = scan full history")
     p.add_argument("--edit-budget", type=int, default=0)
+    p.add_argument("--max-sessions", type=int, default=0,
+                   help="cap harvested sessions before mining; default derives from max tasks")
+    p.add_argument("--max-tasks", type=int, default=0,
+                   help="cap mined tasks for this run")
+    p.add_argument("--target-skill-path", default="",
+                   help="explicit live SKILL.md path to evolve/stage/adopt")
+    p.add_argument("--tasks-file", default="",
+                   help="reviewed TaskRecord JSON file to replay instead of harvesting")
+    p.add_argument("--progress", action="store_true",
+                   help="print phase progress to stderr")
     p.add_argument("--auto-adopt", action="store_true")
     p.add_argument("--json", action="store_true")
 
 
-def _cfg_from_args(args) -> Any:
+def _cfg_from_args(args, task_meta: Dict[str, Any] | None = None) -> Any:
     overrides: Dict[str, Any] = {}
     if args.project:
         overrides["invoked_project"] = os.path.abspath(args.project)
@@ -59,34 +108,72 @@ def _cfg_from_args(args) -> Any:
         overrides["codex_path"] = os.path.abspath(args.codex_path)
     if getattr(args, "claude_home", ""):
         overrides["claude_home"] = os.path.abspath(args.claude_home)
-    if getattr(args, "lookback_hours", 0):
+    if getattr(args, "codex_home", ""):
-        overrides["lookback_hours"] = args.lookback_hours
+        overrides["codex_home"] = os.path.abspath(args.codex_home)
+    if getattr(args, "source", ""):
+        overrides["transcript_source"] = args.source
+    lh = getattr(args, "lookback_hours", None)
+    if lh is not None:  # --lookback-hours was explicitly passed (0 = full history)
+        overrides["lookback_hours"] = lh
     if getattr(args, "edit_budget", 0):
         overrides["edit_budget"] = args.edit_budget
+    if getattr(args, "max_sessions", 0):
+        overrides["max_sessions_per_night"] = args.max_sessions
+    if getattr(args, "max_tasks", 0):
+        overrides["max_tasks_per_night"] = args.max_tasks
+    target_skill_path = getattr(args, "target_skill_path", "")
+    if not target_skill_path and task_meta:
+        target_skill_path = str(task_meta.get("target_skill_path") or "")
+    if target_skill_path:
+        path = os.path.expanduser(target_skill_path)
+        if args.project and not os.path.isabs(path):
+            path = os.path.join(os.path.abspath(args.project), path)
+        overrides["target_skill_path"] = os.path.abspath(path)
+    if getattr(args, "progress", False):
+        overrides["progress"] = True
     if getattr(args, "auto_adopt", False):
         overrides["auto_adopt"] = True
     return load_config(**overrides)
 
 
 def cmd_run(args, dry: bool = False) -> int:
-    cfg = _cfg_from_args(args)
+    task_meta: Dict[str, Any] = {}
-    outcome = run_sleep_cycle(cfg, dry_run=dry)
+    tasks = None
+    if getattr(args, "tasks_file", ""):
+        # Load once before config so target_skill_path can default from metadata.
+        tasks, task_meta = load_tasks_file(args.tasks_file)
+    cfg = _cfg_from_args(args, task_meta=task_meta)
+    if getattr(args, "tasks_file", ""):
+        tasks, task_meta = load_tasks_file(
+            args.tasks_file,
+            holdout_fraction=cfg.get("holdout_fraction", 0.34),
+            seed=cfg.get("seed", 42),
+        )
+        if cfg.get("backend", "mock") != "mock" and task_meta.get("reviewed") is not True:
+            print(
+                "[sleep] refusing real-backend replay from an unreviewed tasks file; "
+                "inspect/redact it and set \"reviewed\": true first",
+                file=sys.stderr,
+            )
+            return 2
+    outcome = run_sleep_cycle(cfg, seed_tasks=tasks, dry_run=dry)
     rep = outcome.report
     if args.json:
-        print(json.dumps({
+        payload = _report_payload(rep, outcome)
-            "night": rep.night, "accepted": rep.accepted,
+        if task_meta:
-            "gate_action": rep.gate_action,
+            payload["tasks_file"] = task_meta.get("tasks_file", "")
-            "baseline": rep.baseline_score, "candidate": rep.candidate_score,
+            payload["tasks_reviewed"] = task_meta.get("reviewed", False)
-            "n_tasks": rep.n_tasks, "n_sessions": rep.n_sessions,
+        print(json.dumps(payload, ensure_ascii=False, indent=2))
-            "edits": [e.__dict__ for e in rep.edits],
-            "staging_dir": outcome.staging_dir, "adopted": outcome.adopted,
-        }, ensure_ascii=False, indent=2))
     else:
         print(f"[sleep] night {rep.night}: {rep.n_sessions} sessions -> {rep.n_tasks} tasks")
         print(f"[sleep] held-out {rep.baseline_score:.3f} -> {rep.candidate_score:.3f} "
               f"=> {rep.gate_action} (accepted={rep.accepted})")
         for e in rep.edits:
             print(f"   + [{e.target}/{e.op}] {e.content}")
+        if rep.rejected_edits:
+            print("[sleep] rejected by gate:")
+            for e in rep.rejected_edits:
+                print(f"   - [{e.target}/{e.op}] {e.content}")
         if outcome.staging_dir:
             print(f"[sleep] staged: {outcome.staging_dir}")
             if not outcome.adopted:
@@ -143,26 +230,72 @@ def cmd_adopt(args) -> int:
 
 def cmd_harvest(args) -> int:
     cfg = _cfg_from_args(args)
-    digests = harvest(
+    session_limit = cfg.get("max_sessions_per_night", 0) or cfg.get("max_tasks_per_night", 40) * 3
-        cfg.transcripts_dir,
+    target_skill_path = cfg.managed_skill_path() if cfg.get("target_skill_path", "") else ""
-        scope=cfg.get("projects", "invoked"),
+    target_skill_text = _read_text(target_skill_path) if target_skill_path else ""
-        invoked_project=cfg.get("invoked_project", ""),
+    max_tasks = cfg.get("max_tasks_per_night", 40)
-        limit=cfg.get("max_tasks_per_night", 40) * 3,
+    candidate_limit = max_tasks
+    if cfg.get("target_task_filter", True) and target_skill_text:
+        candidate_limit = max(max_tasks, max_tasks * 3)
+    digests = harvest_for_config(cfg, limit=session_limit)
+    tasks = mine(
+        digests,
+        max_tasks=max_tasks,
+        candidate_limit=candidate_limit,
+        holdout_fraction=cfg.get("holdout_fraction", 0.34),
+        seed=cfg.get("seed", 42),
+        target_skill_text=target_skill_text,
+        target_skill_path=target_skill_path,
     )
-    tasks = mine(digests, max_tasks=cfg.get("max_tasks_per_night", 40),
+    payload = make_tasks_payload(
-                 holdout_fraction=cfg.get("holdout_fraction", 0.34), seed=cfg.get("seed", 42))
+        tasks,
+        project=cfg.get("invoked_project") or os.getcwd(),
+        transcript_source=cfg.get("transcript_source", ""),
+        n_sessions=len(digests),
+        target_skill_path=target_skill_path,
+    )
+    output_path = ""
+    if getattr(args, "output", ""):
+        output_path = write_tasks_file(args.output, payload)
     if args.json:
-        print(json.dumps({
+        json_payload = dict(payload)
-            "n_sessions": len(digests),
+        if output_path:
-            "tasks": [t.to_dict() for t in tasks],
+            json_payload["output"] = output_path
-        }, ensure_ascii=False, indent=2))
+        print(json.dumps(json_payload, ensure_ascii=False, indent=2))
     else:
         print(f"[sleep] {len(digests)} sessions -> {len(tasks)} tasks")
+        if output_path:
+            print(f"[sleep] wrote reviewed-task draft: {output_path}")
         for t in tasks:
             print(f"  [{t.split}/{t.outcome}] {t.intent[:90]}")
     return 0
 
 
+def cmd_schedule(args) -> int:
+    from skillopt_sleep.scheduler import schedule, list_scheduled
+    cfg = _cfg_from_args(args)
+    project = cfg.get("invoked_project") or os.getcwd()
+    ok, msg = schedule(project, backend=cfg.get("backend", "mock"),
+                       hour=args.hour, minute=args.minute,
+                       extra=("--auto-adopt" if getattr(args, "auto_adopt", False) else ""))
+    print("[sleep] " + msg)
+    cur = list_scheduled()
+    if cur:
+        print("[sleep] currently scheduled:")
+        for ln in cur:
+            print("   " + ln[:140])
+    return 0 if ok else 1
+
+
+def cmd_unschedule(args) -> int:
+    from skillopt_sleep.scheduler import unschedule
+    cfg = _cfg_from_args(args)
+    project = cfg.get("invoked_project") or os.getcwd()
+    ok, msg = unschedule(project, all_projects=getattr(args, "all", False))
+    print("[sleep] " + msg)
+    return 0 if ok else 1
+
+
 def main(argv=None) -> int:
     parser = argparse.ArgumentParser(prog="skillopt_sleep", description="SkillOpt-Sleep nightly self-evolution")
     sub = parser.add_subparsers(dest="cmd", required=True)
@@ -178,6 +311,14 @@ def main(argv=None) -> int:
     p_adopt.add_argument("--staging", default="", help="specific staging dir")
     p_harvest = sub.add_parser("harvest", help="debug: show mined tasks")
     _add_common(p_harvest)
+    p_harvest.add_argument("--output", default="", help="write mined tasks JSON for review")
+    p_sched = sub.add_parser("schedule", help="install a nightly cron entry for this project")
+    _add_common(p_sched)
+    p_sched.add_argument("--hour", type=int, default=3)
+    p_sched.add_argument("--minute", type=int, default=17)
+    p_unsched = sub.add_parser("unschedule", help="remove the nightly cron entry")
+    _add_common(p_unsched)
+    p_unsched.add_argument("--all", action="store_true", help="remove all managed entries")
 
     args = parser.parse_args(argv)
     if args.cmd == "run":
@@ -190,6 +331,10 @@ def main(argv=None) -> int:
         return cmd_adopt(args)
     if args.cmd == "harvest":
         return cmd_harvest(args)
+    if args.cmd == "schedule":
+        return cmd_schedule(args)
+    if args.cmd == "unschedule":
+        return cmd_unschedule(args)
     parser.print_help()
     return 2
 

skillopt_sleep/backend.py

@@ -24,6 +24,7 @@ import json
 import os
 import re
 import subprocess
+import tempfile
 from typing import Any, Dict, List, Optional, Tuple
 
 from skillopt_sleep.types import EditRecord, ReplayResult, TaskRecord
@@ -41,7 +42,8 @@ class Backend:
     # Optional user preferences (free text) injected into reflect as a prior.
     preferences: str = ""
 
-    def attempt(self, task: TaskRecord, skill: str, memory: str) -> str:
+    def attempt(self, task: TaskRecord, skill: str, memory: str,
+                sample_id: int = 0) -> str:
         raise NotImplementedError
 
     def attempt_with_tools(
@@ -151,7 +153,8 @@ class MockBackend(Backend):
                     out.append(key)
         return out
 
-    def attempt(self, task: TaskRecord, skill: str, memory: str) -> str:
+    def attempt(self, task: TaskRecord, skill: str, memory: str,
+                sample_id: int = 0) -> str:
         ctx = (skill or "") + "\n" + (memory or "")
         rules = self._required_rules(task)
         # The "__harmful__" rule models a bad edit: even when present it makes
@@ -191,6 +194,13 @@ class MockBackend(Backend):
         return resp, called
 
     def judge(self, task: TaskRecord, response: str) -> Tuple[float, float, str]:
+        if task.reference_kind == "answer" and task.judge:
+            try:
+                from skillopt_sleep.experiments.real_eval import score_answer_judge
+            except ImportError:
+                score_answer_judge = None  # research evaluators not bundled
+            if score_answer_judge is not None:
+                return score_answer_judge(task.judge, response)
         if task.reference_kind == "rule" and task.judge:
             from skillopt_sleep.judges import score_rule_judge
             return score_rule_judge(task.judge, response)
@@ -253,6 +263,43 @@ def _extract_json(raw: str, kind: str):
         return None
 
 
+def _task_guardrail(pairs) -> str:
+    """Build an 'output contract' the optimizer must not violate.
+
+    ``pairs`` is a list of (TaskRecord, ReplayResult). We surface the benchmark's
+    own rollout system prompt (TaskRecord.system) plus a short, explicit list of
+    invariants, so the optimizer cannot learn rules that the evaluator can never
+    honor (the SpreadsheetBench failure mode: a learned "return ```vba```" or
+    "ask the user for the range" rule scores 0 because the harness runs only
+    ```python``` openpyxl and cannot answer questions).
+
+    Returns "" when no task carries a system contract (e.g. mined daily cases),
+    so non-benchmark runs are unchanged.
+    """
+    sys_txt = ""
+    for t, _ in pairs:
+        s = getattr(t, "system", "") or ""
+        if s.strip():
+            sys_txt = s.strip()
+            break
+    if not sys_txt:
+        return ""
+    # the system prompt can be long; keep the rules portion concise for the optimizer
+    contract = sys_txt
+    if len(contract) > 900:
+        contract = contract[:900] + " …"
+    invariants = (
+        "- Do NOT change the required output format or programming language.\n"
+        "- Do NOT tell the agent to ask the user a question or request more info; "
+        "it must always produce a best-effort answer from what is given.\n"
+        "- Keep every rule consistent with the contract above."
+    )
+    return (
+        "\n# Task output contract (rules MUST obey this — violating it scores 0)\n"
+        f"{contract}\n{invariants}\n"
+    )
+
+
 class CliBackend(Backend):
     """Common logic for real CLI-driven backends (claude / codex).
 
@@ -269,6 +316,8 @@ class CliBackend(Backend):
         self.timeout = timeout
         self._tokens = 0
         self._cache: Dict[str, str] = {}
+        self.last_call_error = ""
+        self.last_reflect_raw = ""
 
     # subclasses override --------------------------------------------------
     def _call(self, prompt: str, *, max_tokens: int = 1024) -> str:
@@ -283,24 +332,55 @@ class CliBackend(Backend):
         return out
 
     # operations -----------------------------------------------------------
-    def attempt(self, task: TaskRecord, skill: str, memory: str) -> str:
+    def attempt(self, task: TaskRecord, skill: str, memory: str,
+                sample_id: int = 0) -> str:
+        # sample_id distinguishes repeated rollouts of the SAME (task, skill,
+        # memory) in the cache key. Without it the attempt cache collapses all
+        # K dream rollouts into one cached response (spread always 0), which
+        # silently disables contrastive reflection. sample_id=0 keeps the old
+        # key format so gate re-scoring still benefits from the cache.
+        if task.system:
+            # Benchmark carries its own (research-repo) rollout system prompt.
+            # Use it verbatim with a neutral skill/memory section — this both
+            # keeps scoring faithful and avoids the aggressive "OVERRIDE / HARD
+            # CONSTRAINT" phrasing below, which Azure's content filter flags as a
+            # jailbreak (HTTP 400) and silently zeroes the rollout.
+            skill_section = f"## Skill\n{skill.strip()}\n\n" if skill.strip() else ""
+            mem_section = f"## Memory\n{memory.strip()}\n\n" if memory.strip() else ""
+            system = task.system.replace("{skill_section}", skill_section)
+            if "{skill_section}" not in task.system and skill_section:
+                system = skill_section + system
+            body = task.intent + ("\n\n" + task.context_excerpt if task.context_excerpt else "")
+            prompt = f"{system}{mem_section}\n{body}"
+            salt = f"s{sample_id}:" if sample_id else ""
+            key = "attempt:" + salt + skill_hash(prompt)
+            return self._cached_call(key, prompt, max_tokens=512)
+        # generic path (mined daily-case tasks): neutral, content-filter-safe
+        # wording. Apply the skill/memory as guidance, not as adversarial
+        # "OVERRIDE everything" directives.
         prompt = (
-            "You are completing a recurring task for a user. Apply the skill and "
+            "Complete the following task for the user. Follow the skill and memory "
-            "memory rules EXACTLY, including any output-format requirements. If the "
+            "guidance below, including any output-format and length requirements. "
-            "skill contains a 'Learned preferences' block, treat those rules as "
+            "When a 'Learned preferences' rule sets an explicit limit (e.g. a length "
-            "HARD CONSTRAINTS that OVERRIDE anything earlier in the skill they "
+            "cap), prefer that rule over more general advice it refines.\n\n"
-            "conflict with (e.g. an explicit length limit overrides 'be "
-            "exhaustive'). Satisfy every such constraint even at the cost of "
-            "brevity or detail.\n\n"
             f"# Skill\n{skill or '(none)'}\n\n# Memory\n{memory or '(none)'}\n\n"
             f"# Task\n{task.intent}\n\n{task.context_excerpt}\n\n"
             "Return ONLY the final answer text, nothing else."
         )
         # cache on (task, skill, memory) so identical hold-out re-scoring is free
-        key = "attempt:" + skill_hash(prompt)
+        salt = f"s{sample_id}:" if sample_id else ""
+        key = "attempt:" + salt + skill_hash(prompt)
         return self._cached_call(key, prompt, max_tokens=512)
 
     def judge(self, task: TaskRecord, response: str) -> Tuple[float, float, str]:
+        # real-benchmark correctness judge (searchqa/livemath/spreadsheet) — local
+        if task.reference_kind == "answer" and task.judge:
+            try:
+                from skillopt_sleep.experiments.real_eval import score_answer_judge
+            except ImportError:
+                score_answer_judge = None  # research evaluators not bundled
+            if score_answer_judge is not None:
+                return score_answer_judge(task.judge, response)
         # gbrain-style rule judge: scored locally, no API spend
         if task.reference_kind == "rule" and task.judge:
             from skillopt_sleep.judges import score_rule_judge
@@ -389,6 +469,13 @@ class CliBackend(Backend):
                 "\n# User preferences (honor these as priors when writing rules)\n"
                 + str(self.preferences).strip()
             )
+        # Task GUARDRAIL: the optimizer must not invent rules that violate the
+        # task's hard constraints (e.g. SpreadsheetBench answers MUST be a
+        # ```python``` openpyxl block — a learned "return ```vba```" or "ask the
+        # user for the range" rule scores 0 because the harness can't run VBA and
+        # can't ask questions). We surface the benchmark's own rollout system
+        # prompt (carried on TaskRecord.system) so proposed rules stay in-bounds.
+        guard_text = _task_guardrail(failures)
         prompt = (
             "You are SkillOpt's optimizer. The agent keeps failing the recurring "
             f"tasks below. Propose at most {edit_budget} bounded edits to the "
@@ -406,9 +493,15 @@ class CliBackend(Backend):
             "but outputs must be under a character limit), write an explicit, "
             "forceful OVERRIDE rule stating it supersedes the conflicting "
             "instruction, and put the hard requirement first.\n"
+            "HARD CONSTRAINT: every rule you write MUST be consistent with the "
+            "'Task output contract' below (if shown). NEVER propose a rule that "
+            "changes the required output format/language, tells the agent to ask "
+            "the user a question, or otherwise violates that contract — such a "
+            "rule scores ZERO because the evaluator cannot honor it.\n"
             'Return ONLY a JSON array: '
             '[{"op":"add|replace|delete","content":"<rule>","anchor":"<text to replace/delete, optional>","rationale":"<why>"}].\n\n'
             f"# Current {target}\n{cur_doc}\n"
+            f"{guard_text}"
             f"{criteria_text}\n"
             f"{pref_text}\n\n"
             f"# Recurring failures\n{fail_text}"
@@ -427,6 +520,10 @@ class CliBackend(Backend):
             arr = _extract_json(raw, "array")
             if isinstance(arr, list) and arr:
                 break
+        # Expose the last raw optimizer reply so a no-edits night is diagnosable:
+        # a 0.0->0.0 gate with zero edits is otherwise indistinguishable from
+        # "nothing to learn" (the cycle persists this in diagnostics.json).
+        self.last_reflect_raw = raw or ""
         edits: List[EditRecord] = []
         if isinstance(arr, list):
             for e in arr[:edit_budget]:
@@ -460,6 +557,39 @@ class ClaudeCliBackend(CliBackend):
                          timeout=timeout)
         self.claude_path = claude_path
 
+    # Known CLI error prefixes that indicate auth or config failures.
+    # When detected, we log a warning so the user doesn't mistake a
+    # broken auth for "nothing to optimize" (issue #68).
+    # Keep these specific to avoid false positives on normal model output.
+    _CLI_ERROR_MARKERS = (
+        "Not logged in",
+        "Please run /login",
+        "Authentication required",
+        "Invalid API key",
+        "Unauthorized: invalid x-api-key",
+    )
+
+    def _detect_cli_error(self, stdout: str, stderr: str) -> None:
+        """Log a warning if CLI output looks like an auth/config error.
+
+        Only checks stderr and short stdout (< 300 chars) to avoid
+        false-positives on legitimate model responses that mention
+        auth-related terms.
+        """
+        import logging
+        # Long stdout is almost certainly a real model response, not an error.
+        check_stdout = stdout if len(stdout) < 300 else ""
+        combined = check_stdout + "\n" + stderr
+        for marker in self._CLI_ERROR_MARKERS:
+            if marker in combined:
+                from skillopt_sleep.staging import redact_secrets
+                logging.getLogger("skillopt_sleep").warning(
+                    "Claude CLI returned a likely auth error: %s",
+                    redact_secrets(combined[:200].replace("\n", " ")),
+                )
+                self.last_call_error = combined[:500]
+                return
+
     def _call(self, prompt: str, *, max_tokens: int = 1024) -> str:
         # Run ISOLATED so the ambient Claude Code environment does not leak into
         # the optimizer/target call. Critically, the user's GLOBAL skills
@@ -467,14 +597,17 @@ class ClaudeCliBackend(CliBackend):
         # them explicitly — without this, reflect/attempt sometimes reply with a
         # list of the user's installed skills instead of doing the task.
         #   --bare                    skip hooks, LSP, plugins (minimal mode)
+        #                             Only safe with ANTHROPIC_API_KEY auth;
+        #                             breaks subscription-token auth (#68).
         #   --disable-slash-commands  disable all skills
         #   --disallowedTools '*'     no tool use
         #   --exclude-dynamic-...     drop per-machine cwd/env/memory/git sections
         #   cwd=<clean temp>          no project CLAUDE.md
         import tempfile
-        cmd = [
+        cmd = [self.claude_path, "-p", "--output-format", "text"]
-            self.claude_path, "-p", "--output-format", "text",
+        if os.environ.get("ANTHROPIC_API_KEY"):
-            "--bare",
+            cmd.append("--bare")
+        cmd += [
             "--disable-slash-commands",
             "--disallowedTools", "*",
             "--exclude-dynamic-system-prompt-sections",
@@ -495,7 +628,9 @@ class ClaudeCliBackend(CliBackend):
                 shutil.rmtree(clean_cwd, ignore_errors=True)
             except Exception:
                 pass
-        return (proc.stdout or "").strip()
+        out = (proc.stdout or "").strip()
+        self._detect_cli_error(out, proc.stderr or "")
+        return out
 
     def attempt_with_tools(self, task, skill, memory, tools):
         # Expose a REAL, callable `search` tool (a shell shim that logs each
@@ -532,9 +667,11 @@ class ClaudeCliBackend(CliBackend):
                 f"# Task\n{task.intent}\n\n{task.context_excerpt}\n\n"
                 "Return ONLY the final answer text."
             )
-            cmd = [
+            cmd = [self.claude_path, "-p", "--output-format", "text"]
-                self.claude_path, "-p", "--output-format", "text",
+            if os.environ.get("ANTHROPIC_API_KEY"):
-                "--bare", "--disable-slash-commands",
+                cmd.append("--bare")
+            cmd += [
+                "--disable-slash-commands",
                 "--allowedTools", "Bash",
                 "--exclude-dynamic-system-prompt-sections",
             ]
@@ -546,6 +683,7 @@ class ClaudeCliBackend(CliBackend):
                     cmd, capture_output=True, text=True, timeout=self.timeout, cwd=work,
                 )
                 resp = (proc.stdout or "").strip()
+                self._detect_cli_error(resp, proc.stderr or "")
             except Exception:
                 resp = ""
             self._tokens += len(prompt) // 4 + len(resp) // 4
@@ -601,14 +739,26 @@ class CodexCliBackend(CliBackend):
 
     name = "codex"
 
-    def __init__(self, model: str = "", codex_path: str = "", timeout: int = 240,
+    def __init__(
-                 sandbox: str = "read-only") -> None:
+        self,
+        model: str = "",
+        codex_path: str = "",
+        timeout: int = 240,
+        sandbox: str = "read-only",
+        project_dir: str = "",
+    ) -> None:
         super().__init__(model=model or os.environ.get("SKILLOPT_SLEEP_CODEX_MODEL", ""),
                          timeout=timeout)
         self.codex_path = resolve_codex_path(codex_path)
         self.sandbox = sandbox
+        self.project_dir = (
+            os.path.abspath(os.path.expanduser(project_dir)) if project_dir else ""
+        )
 
-    def _call(self, prompt: str, *, max_tokens: int = 1024) -> str:
+    def _call_once(self, prompt: str, *, max_tokens: int = 1024) -> str:
+        """One codex exec attempt: returns the response text, or "" on
+        timeout/exception/empty-output (with last_call_error set). ``_call``
+        wraps this with retries so a transient failure is NOT silently scored 0."""
         import tempfile
         out_path = tempfile.NamedTemporaryFile(
             prefix="codex_last_", suffix=".txt", delete=False
@@ -618,24 +768,92 @@ class CodexCliBackend(CliBackend):
             "--color", "never", "--sandbox", self.sandbox,
             "-o", out_path,
         ]
+        if self.project_dir:
+            cmd[3:3] = ["-C", self.project_dir]
         if self.model:
             cmd += ["-m", self.model]
         cmd += ["--", prompt]
+        proc = None
         try:
-            subprocess.run(cmd, capture_output=True, text=True, timeout=self.timeout)
+            try:
-        except Exception:
+                proc = subprocess.run(
-            return ""
+                    cmd,
-        try:
+                    capture_output=True,
-            with open(out_path, encoding="utf-8") as f:
+                    text=True,
-                return f.read().strip()
+                    timeout=self.timeout,
-        except Exception:
+                    cwd=self.project_dir or None,
-            return ""
+                )
+            except subprocess.TimeoutExpired:
+                self.last_call_error = f"codex exec timed out after {self.timeout}s"
+                return ""
+            except Exception as exc:
+                self.last_call_error = f"codex exec failed: {exc}"
+                return ""
+            try:
+                with open(out_path, encoding="utf-8") as f:
+                    out = f.read().strip()
+                if out:
+                    return out
+            except Exception as exc:
+                self.last_call_error = f"could not read codex output file: {exc}"
+            stdout = (proc.stdout or "").strip() if proc is not None else ""
+            stderr = (proc.stderr or "").strip() if proc is not None else ""
+            if proc is not None and proc.returncode != 0 and not self.last_call_error:
+                self.last_call_error = f"codex exec exited {proc.returncode}: {stderr[:500]}"
+            # Do NOT return the CLI's error text as if it were a model response: it
+            # pollutes rollout/judge/reflect and gets silently scored 0, hiding the
+            # real cause (e.g. an expired codex auth token surfacing as a 9k-char 401).
+            # Surface it via last_call_error and return empty instead.
+            if self.last_call_error:
+                return ""
+            return stdout or stderr
         finally:
             try:
                 os.unlink(out_path)
             except Exception:
                 pass
 
+    # Fatal codex failures that will NOT recover on retry — fail fast + loud so a
+    # 0.0 night reads as "codex auth/model/version problem" not "nothing to learn".
+    # Covers: auth (re-login), and 400 config errors like an unsupported model on a
+    # ChatGPT account or a model that needs a newer codex CLI (upgrade).
+    _AUTH_MARKERS = (
+        "401 Unauthorized", "refresh_token_reused", "token_expired",
+        "Please log out and sign in", "Not logged in", "Please run /login",
+        "authentication token is expired", "Unauthorized: invalid",
+        "is not supported when using Codex", "requires a newer version of Codex",
+    )
+
+    def _call(self, prompt: str, *, max_tokens: int = 1024, retries: int = 3) -> str:
+        """Retry transient empties/timeouts instead of silently returning "".
+
+        An empty reply scores 0 on every judge, which deflates the held-out
+        baseline AND blocks the candidate from ever improving — making a flaky
+        backend indistinguishable from "nothing to learn". The Azure backend
+        already guards this way (AzureOpenAIBackend._call); codex now does too.
+        Auth errors are NOT retried (hopeless until the user re-logs-in).
+        """
+        import logging
+        import random as _r
+        import time as _t
+        out = ""
+        for attempt in range(max(1, retries)):
+            self.last_call_error = ""
+            out = self._call_once(prompt, max_tokens=max_tokens)
+            if out:
+                return out
+            err = self.last_call_error or ""
+            if any(m in err for m in self._AUTH_MARKERS):
+                from skillopt_sleep.staging import redact_secrets
+                logging.getLogger("skillopt_sleep").error(
+                    "codex auth error — re-login required (`codex login`): %s",
+                    redact_secrets(err[:200]),
+                )
+                break  # fail fast: retrying a 401 just burns calls
+            if attempt < retries - 1:
+                _t.sleep(min(6.0, (2 ** attempt) * 0.5) + _r.random() * 0.3)
+        return out
+
     def attempt_with_tools(self, task, skill, memory, tools):
         # Codex exec runs in a sandbox with shell access; expose the same real
         # `search` shim and let it run (workspace-write so the shim can log).
@@ -675,16 +893,27 @@ class CodexCliBackend(CliBackend):
             if self.model:
                 cmd += ["-m", self.model]
             cmd += ["--", prompt]
+            self.last_call_error = ""
+            proc = None
             try:
-                subprocess.run(cmd, capture_output=True, text=True, timeout=self.timeout, cwd=work)
+                proc = subprocess.run(cmd, capture_output=True, text=True, timeout=self.timeout, cwd=work)
-            except Exception:
+            except subprocess.TimeoutExpired:
-                pass
+                self.last_call_error = f"codex exec (tools) timed out after {self.timeout}s"
+            except Exception as exc:  # noqa: BLE001
+                self.last_call_error = f"codex exec (tools) failed: {exc}"
             resp = ""
             try:
                 with open(out_path, encoding="utf-8") as f:
                     resp = f.read().strip()
             except Exception:
                 resp = ""
+            # Surface a failed tool-rollout the SAME way _call does: an auth/model/version
+            # failure on this path must show up in diagnostics (call_error), not vanish as a
+            # silent empty->0 scored as a failed rollout. Response stays "" (never the error text).
+            if not resp and not self.last_call_error and proc is not None and proc.returncode != 0:
+                self.last_call_error = (
+                    f"codex exec (tools) exited {proc.returncode}: {(proc.stderr or '')[:500]}"
+                )
             self._tokens += len(prompt) // 4 + len(resp) // 4
             called: List[str] = []
             if os.path.exists(calllog):
@@ -698,6 +927,218 @@ class CodexCliBackend(CliBackend):
             except Exception:
                 pass
 
+def resolve_copilot_path(explicit: str = "") -> str:
+    """Find the GitHub Copilot CLI (`copilot`) binary."""
+    if explicit:
+        return explicit
+    env = os.environ.get("SKILLOPT_SLEEP_COPILOT_PATH")
+    if env:
+        return env
+    import shutil
+    found = shutil.which("copilot")
+    return found or "copilot"
+
+
+class CopilotCliBackend(CliBackend):
+    """Drives the GitHub Copilot CLI in non-interactive mode.
+
+    Uses ``copilot -p <prompt> --output-format json`` and parses the emitted
+    JSONL event stream, returning the concatenated ``assistant.message``
+    content. The plain-text / ``--silent`` modes do not reliably stream the
+    response to stdout on all platforms, so JSONL is used for robust capture.
+
+    The call runs in a clean temp cwd with streaming disabled and tools allowed
+    (so non-interactive mode never blocks on a permission prompt); ``_call``'s
+    prompts ask for final-answer text only, so no tool use is expected there,
+    while ``attempt_with_tools`` exposes real, cross-platform callable shims in
+    the working directory for honest tool-call detection.
+
+    Startup overhead is minimised: each invocation points ``COPILOT_HOME`` at a
+    dedicated, isolated config dir (no user ``mcp-config.json``, so the user's
+    MCP servers — including this project's own — are NOT spawned, avoiding a
+    slow recursive launch), and built-in MCP servers / custom instructions are
+    disabled. Auth is read from the OS credential store / token env vars, which
+    live outside ``COPILOT_HOME``, so isolation does not break authentication.
+    Set ``SKILLOPT_SLEEP_COPILOT_HOME`` to override the isolated home, or set it
+    empty / ``SKILLOPT_SLEEP_COPILOT_FULL_ENV=1`` to use the user's real
+    environment instead.
+    """
+
+    name = "copilot"
+
+    def __init__(self, model: str = "", copilot_path: str = "", timeout: int = 240) -> None:
+        super().__init__(model=model or os.environ.get("SKILLOPT_SLEEP_COPILOT_MODEL", ""),
+                         timeout=timeout)
+        self.copilot_path = resolve_copilot_path(copilot_path)
+        self.full_env = os.environ.get("SKILLOPT_SLEEP_COPILOT_FULL_ENV", "") == "1"
+        # Stable isolated home so first-run setup is cached across calls.
+        if self.full_env:
+            self.copilot_home = ""
+        else:
+            self.copilot_home = os.environ.get("SKILLOPT_SLEEP_COPILOT_HOME") or os.path.join(
+                tempfile.gettempdir(), "skillopt_sleep_copilot_home"
+            )
+            try:
+                os.makedirs(self.copilot_home, exist_ok=True)
+            except Exception:
+                self.copilot_home = ""
+
+    def _call(self, prompt: str, *, max_tokens: int = 1024) -> str:
+        clean_cwd = tempfile.mkdtemp(prefix="skillopt_sleep_copilot_")
+        cmd = [
+            self.copilot_path, "-p", prompt,
+            "--output-format", "json",
+            "--stream", "off",
+            "--no-color",
+            "--log-level", "none",
+            "--allow-all-tools",
+            "-C", clean_cwd,
+        ]
+        if not self.full_env:
+            # Drop unneeded startup work: no built-in (github) MCP server and no
+            # AGENTS.md / custom-instruction loading. With an isolated home that
+            # has no mcp-config.json, no user MCP servers spawn either.
+            cmd += ["--disable-builtin-mcps", "--no-custom-instructions"]
+        if self.model:
+            cmd += ["--model", self.model]
+        env = os.environ.copy()
+        if self.copilot_home:
+            env["COPILOT_HOME"] = self.copilot_home
+        try:
+            proc = subprocess.run(
+                cmd, capture_output=True, text=True, timeout=self.timeout, cwd=clean_cwd,
+                encoding="utf-8", errors="replace", env=env,
+            )
+        except Exception:
+            return ""
+        finally:
+            try:
+                import shutil
+                shutil.rmtree(clean_cwd, ignore_errors=True)
+            except Exception:
+                pass
+        return self._parse_jsonl_response(proc.stdout or "")
+
+    @staticmethod
+    def _parse_jsonl_response(raw: str) -> str:
+        parts: List[str] = []
+        for line in raw.splitlines():
+            line = line.strip()
+            if not line or not line.startswith("{"):
+                continue
+            try:
+                obj = json.loads(line)
+            except Exception:
+                continue
+            if obj.get("type") == "assistant.message":
+                content = (obj.get("data") or {}).get("content")
+                if isinstance(content, str) and content:
+                    parts.append(content)
+        return "\n".join(parts).strip()
+
+    def attempt_with_tools(self, task, skill, memory, tools):
+        # Expose REAL, callable tool shims in the working directory so the
+        # gbrain quick-answerer judge (tool_called=search) is validated
+        # honestly: we detect each call from the shim's log, not from a
+        # self-reported marker. The Copilot CLI is the Windows-validated
+        # backend, so the shims must be cross-platform — a bash `#!/usr/bin/env
+        # bash` + chmod shim does NOT execute via `./tool` under PowerShell/cmd,
+        # so on Windows we emit a `.cmd` batch shim instead.
+        import shutil
+        import stat
+        work = tempfile.mkdtemp(prefix="skillopt_sleep_copilottools_")
+        calllog = os.path.join(work, "_tool_calls.log")
+        tool_names = tools or ["search"]
+        is_windows = os.name == "nt"
+        try:
+            for tname in tool_names:
+                if is_windows:
+                    shim = os.path.join(work, f"{tname}.cmd")
+                    with open(shim, "w") as f:
+                        # `%~n0` is the script's own base name (the tool name);
+                        # writing it keeps the calllog line == tool name so the
+                        # honest-detection match below works unchanged.
+                        f.write(
+                            "@echo off\n"
+                            f'echo %~n0>>"{calllog}"\n'
+                            "echo (search results: 3 relevant notes found; use them to answer)\n"
+                        )
+                else:
+                    shim = os.path.join(work, tname)
+                    with open(shim, "w") as f:
+                        f.write(
+                            "#!/usr/bin/env bash\n"
+                            f'echo "{tname}" >> "{calllog}"\n'
+                            'echo "(search results: 3 relevant notes found; use them to answer)"\n'
+                        )
+                    os.chmod(shim, os.stat(shim).st_mode | stat.S_IEXEC | stat.S_IXGRP | stat.S_IXOTH)
+            if is_windows:
+                tool_hint = (
+                    "You have shell tools available in the current directory: "
+                    + ", ".join(f"{t}.cmd" for t in tool_names)
+                    + " (each callable as `" + tool_names[0] + "` or `.\\"
+                    + tool_names[0] + "`). When the skill says to look something "
+                    "up or search before answering, you MUST actually run the "
+                    "tool (e.g. `" + tool_names[0] + " \"query\"`) before giving "
+                    "your final answer."
+                )
+            else:
+                tool_hint = (
+                    "You have shell tools available in the current directory: "
+                    + ", ".join(f"./{t}" for t in tool_names)
+                    + ". When the skill says to look something up or search before "
+                    "answering, you MUST actually run the tool (e.g. `./search \"query\"`) "
+                    "before giving your final answer."
+                )
+            prompt = (
+                "You are completing a task. Apply the skill and memory rules EXACTLY, "
+                "including any rule about searching/looking up before answering. "
+                "Treat a 'Learned preferences' block as HARD CONSTRAINTS that override "
+                "earlier conflicting skill text.\n\n"
+                f"{tool_hint}\n\n"
+                f"# Skill\n{skill or '(none)'}\n\n# Memory\n{memory or '(none)'}\n\n"
+                f"# Task\n{task.intent}\n\n{task.context_excerpt}\n\n"
+                "Return ONLY the final answer text."
+            )
+            cmd = [
+                self.copilot_path, "-p", prompt,
+                "--output-format", "json",
+                "--stream", "off",
+                "--no-color",
+                "--log-level", "none",
+                "--allow-all-tools",
+                "-C", work,
+            ]
+            if not self.full_env:
+                cmd += ["--disable-builtin-mcps", "--no-custom-instructions"]
+            if self.model:
+                cmd += ["--model", self.model]
+            env = os.environ.copy()
+            if self.copilot_home:
+                env["COPILOT_HOME"] = self.copilot_home
+            resp = ""
+            try:
+                proc = subprocess.run(
+                    cmd, capture_output=True, text=True, encoding="utf-8",
+                    errors="replace", timeout=self.timeout, cwd=work, env=env,
+                )
+                resp = self._parse_jsonl_response(proc.stdout or "")
+            except Exception:
+                resp = ""
+            self._tokens += len(prompt) // 4 + len(resp) // 4
+            called: List[str] = []
+            if os.path.exists(calllog):
+                with open(calllog) as f:
+                    logged = {ln.strip() for ln in f if ln.strip()}
+                called = [t for t in tool_names if t in logged]
+            return resp, called
+        finally:
+            try:
+                shutil.rmtree(work, ignore_errors=True)
+            except Exception:
+                pass
+
+
 class DualBackend(Backend):
     """Route operations to two backends, à la SkillOpt's target vs optimizer.
 
@@ -717,8 +1158,8 @@ class DualBackend(Backend):
         self.optimizer = optimizer
         self.name = f"target={target.name}/optimizer={optimizer.name}"
 
-    def attempt(self, task, skill, memory):
+    def attempt(self, task, skill, memory, sample_id: int = 0):
-        return self.target.attempt(task, skill, memory)
+        return self.target.attempt(task, skill, memory, sample_id=sample_id)
 
     def attempt_with_tools(self, task, skill, memory, tools):
         return self.target.attempt_with_tools(task, skill, memory, tools)
@@ -741,18 +1182,214 @@ class DualBackend(Backend):
         return self.target.tokens_used() + self.optimizer.tokens_used()
 
 
+# ── Azure OpenAI backend (gpt-5.x via managed identity) ───────────────────────
+
+# Endpoint -> deployments, from the intern's avail_api.md. The backend picks the
+# first endpoint that hosts the requested deployment.
+_AZURE_ENDPOINTS = {
+    "https://oaidr9.openai.azure.com/": {"gpt-5.5", "gpt-5.4", "gpt-5.4-mini", "gpt-5.4-nano", "o3"},
+    "https://t2vgoaigpt4o6.openai.azure.com/": {"gpt-5.5", "gpt-4o-mini", "o3", "o4-mini"},
+    "https://oaidr21.openai.azure.com/": {"gpt-5.5", "o3", "o4-mini"},
+    "https://searchagent5.cognitiveservices.azure.com/": {"gpt-5.4-mini", "gpt-4o-mini"},
+    "https://t2vgoaigpt4o.openai.azure.com/": {"gpt-5.4", "gpt-5.4-nano", "gpt-5.2", "gpt-5.1", "o3", "o4-mini"},
+}
+_AZURE_MI_CLIENT_ID = "8cafa2b1-a2a7-4ad9-814a-ffe4aed7e800"
+
+
+class AzureOpenAIBackend(CliBackend):
+    """Drives Azure OpenAI gpt-5.x deployments via managed identity.
+
+    Mirrors the intern's blog_1 setup (avail_api.md): managed-identity auth, the
+    same endpoints/deployments. Reuses CliBackend's attempt/judge/reflect prompts
+    and JSON parsing; only _call() differs. openai + azure-identity are lazy
+    imported so the mock/CLI paths stay dependency-free.
+    """
+
+    name = "azure"
+
+    def __init__(self, deployment: str = "", endpoint: str = "", timeout: int = 180,
+                 api_version: str = "2024-12-01-preview") -> None:
+        super().__init__(model=deployment or "gpt-5.5", timeout=timeout)
+        self.deployment = deployment or "gpt-5.5"
+        self.endpoint = endpoint or self._endpoint_for(self.deployment)
+        self.api_version = api_version
+        self.name = f"azure:{self.deployment}"
+        self._client = None
+
+    @staticmethod
+    def _endpoint_for(deployment: str) -> str:
+        for ep, deps in _AZURE_ENDPOINTS.items():
+            if deployment in deps:
+                return ep
+        return "https://oaidr9.openai.azure.com/"
+
+    def _get_client(self):
+        if self._client is None:
+            from azure.identity import ManagedIdentityCredential, get_bearer_token_provider
+            from openai import AzureOpenAI
+            cred = ManagedIdentityCredential(client_id=_AZURE_MI_CLIENT_ID)
+            tp = get_bearer_token_provider(cred, "https://cognitiveservices.azure.com/.default")
+            self._client = AzureOpenAI(
+                azure_endpoint=self.endpoint, azure_ad_token_provider=tp,
+                api_version=self.api_version, max_retries=4,
+            )
+        return self._client
+
+    def _call(self, prompt: str, *, max_tokens: int = 1024, retries: int = 5) -> str:
+        """Call the deployment with bounded retries.
+
+        IMPORTANT: transient failures (429 rate-limit, timeouts, 5xx) must NOT be
+        silently turned into an empty string — an empty response scores 0 and
+        deflates every baseline/after measure. We retry with exponential backoff
+        (mirroring the research repo's retries=5) and only return "" after the
+        budget is exhausted. ``time``/``random`` are used for backoff; both are
+        available here (this is library code, not a Workflow script sandbox).
+        """
+        import random as _r
+        import time as _t
+
+        client = self._get_client()
+        last_exc = None
+        for attempt in range(max(1, retries)):
+            try:
+                resp = client.chat.completions.create(
+                    model=self.deployment,
+                    messages=[{"role": "user", "content": prompt}],
+                    max_completion_tokens=16384,
+                )
+                text = (resp.choices[0].message.content or "").strip()
+                try:
+                    u = resp.usage
+                    self._tokens += (getattr(u, "prompt_tokens", 0) or 0) + (getattr(u, "completion_tokens", 0) or 0)
+                except Exception:
+                    pass
+                if text:
+                    return text
+                # empty but no exception: model genuinely returned nothing — one
+                # quick retry can help (reasoning models occasionally yield empty)
+                last_exc = "empty-response"
+            except Exception as e:  # noqa: BLE001
+                last_exc = e
+            # backoff before next try (skip after the final attempt)
+            if attempt < retries - 1:
+                _t.sleep(min(8.0, (2 ** attempt) * 0.5) + _r.random() * 0.4)
+        return ""
+
+
+class AzureResponsesBackend(AzureOpenAIBackend):
+    """gpt-5.x via the **Responses API** on the high-throughput gpt4v endpoints.
+
+    Differs from AzureOpenAIBackend in three ways, all required by the enhanced
+    experiment:
+      * Auth via ``AzureCliCredential`` (the logged-in user), not Managed Identity
+        — the gpt4v-scus/swc accounts grant the data role to the CLI principal.
+      * Calls ``client.responses.create`` (the /responses API) instead of
+        chat.completions — these deployments are Responses-only.
+      * Round-robins across multiple endpoints for parallel throughput; each
+        worker thread binds a client for one endpoint (picked by thread index)
+        so concurrent replay spreads load across all endpoints.
+
+    A single shared ``AzureCliCredential`` token provider is reused across all
+    endpoint clients (the token is cached + auto-refreshed by the provider).
+    """
+
+    name = "azure-responses"
+
+    # the two parallel /responses endpoints (user-provided), both hosting gpt-5.5
+    _RESP_ENDPOINTS = [
+        "https://gpt4v-scus.openai.azure.com/",
+        "https://gpt4v-swc.openai.azure.com/",
+    ]
+
+    def __init__(self, deployment: str = "", endpoints: Optional[List[str]] = None,
+                 timeout: int = 180, api_version: str = "2025-04-01-preview") -> None:
+        super().__init__(deployment=deployment, endpoint=(endpoints or self._RESP_ENDPOINTS)[0],
+                         timeout=timeout, api_version=api_version)
+        self.endpoints = list(endpoints or self._RESP_ENDPOINTS)
+        self.name = f"azure-responses:{self.deployment}"
+        self._token_provider = None
+        self._clients: dict = {}      # endpoint -> AzureOpenAI client
+        import threading as _thr
+        self._lock = _thr.Lock()
+        self._rr = 0                  # round-robin counter
+
+    def _get_provider(self):
+        if self._token_provider is None:
+            from azure.identity import AzureCliCredential, get_bearer_token_provider
+            self._token_provider = get_bearer_token_provider(
+                AzureCliCredential(), "https://cognitiveservices.azure.com/.default")
+        return self._token_provider
+
+    def _client_for(self, endpoint: str):
+        cl = self._clients.get(endpoint)
+        if cl is None:
+            from openai import AzureOpenAI
+            cl = AzureOpenAI(
+                azure_endpoint=endpoint, azure_ad_token_provider=self._get_provider(),
+                api_version=self.api_version, max_retries=2,
+            )
+            self._clients[endpoint] = cl
+        return cl
+
+    def _next_endpoint(self) -> str:
+        # round-robin so concurrent calls spread across all endpoints
+        with self._lock:
+            ep = self.endpoints[self._rr % len(self.endpoints)]
+            self._rr += 1
+        return ep
+
+    def _call(self, prompt: str, *, max_tokens: int = 1024, retries: int = 5) -> str:
+        import random as _r
+        import time as _t
+        last = None
+        base_ep = self._next_endpoint()           # this call's primary endpoint
+        base_idx = self.endpoints.index(base_ep)
+        for attempt in range(max(1, retries)):
+            # on retry, fail over to the other endpoint(s)
+            ep = self.endpoints[(base_idx + attempt) % len(self.endpoints)]
+            try:
+                client = self._client_for(ep)
+                resp = client.responses.create(
+                    model=self.deployment, input=prompt,
+                    max_output_tokens=16384,
+                )
+                text = (getattr(resp, "output_text", "") or "").strip()
+                try:
+                    u = resp.usage
+                    self._tokens += (getattr(u, "input_tokens", 0) or 0) + (getattr(u, "output_tokens", 0) or 0)
+                except Exception:
+                    pass
+                if text:
+                    return text
+                last = "empty-response"
+            except Exception as e:  # noqa: BLE001
+                last = e
+            if attempt < retries - 1:
+                _t.sleep(min(8.0, (2 ** attempt) * 0.5) + _r.random() * 0.4)
+        return ""
+
+
 def get_backend(
     name: str,
     *,
     model: str = "",
     claude_path: str = "claude",
     codex_path: str = "",
+    azure_endpoint: str = "",
+    project_dir: str = "",
 ) -> Backend:
     n = (name or "mock").strip().lower()
     if n in {"claude", "anthropic", "claude_cli", "claude_code"}:
         return ClaudeCliBackend(model=model, claude_path=claude_path)
     if n in {"codex", "codex_cli", "openai_codex"}:
-        return CodexCliBackend(model=model, codex_path=codex_path)
+        return CodexCliBackend(model=model, codex_path=codex_path, project_dir=project_dir)
+    if n in {"azure", "azure_openai", "aoai"}:
+        return AzureOpenAIBackend(deployment=model, endpoint=azure_endpoint)
+    if n in {"azure-responses", "azure_responses", "aoai-responses", "responses"}:
+        eps = [e.strip() for e in azure_endpoint.split(",") if e.strip()] or None
+        return AzureResponsesBackend(deployment=model, endpoints=eps)
+    if n in {"copilot", "github_copilot", "copilot_cli", "gh_copilot"}:
+        return CopilotCliBackend(model=model)
     return MockBackend()
 
 
@@ -765,7 +1402,9 @@ def build_backend(
     target_backend: str = "",
     target_model: str = "",
     codex_path: str = "",
+    azure_endpoint: str = "",
     preferences: str = "",
+    project_dir: str = "",
 ) -> Backend:
     """Build a single or dual backend.
 
@@ -776,11 +1415,21 @@ def build_backend(
     """
     has_split = any([optimizer_backend, optimizer_model, target_backend, target_model])
     if not has_split:
-        be = get_backend(backend, model=model, codex_path=codex_path)
+        be = get_backend(
+            backend,
+            model=model,
+            codex_path=codex_path,
+            azure_endpoint=azure_endpoint,
+            project_dir=project_dir,
+        )
         be.preferences = preferences
         return be
-    tgt = get_backend(target_backend or backend, model=target_model or model, codex_path=codex_path)
+    tgt = get_backend(target_backend or backend, model=target_model or model,
-    opt = get_backend(optimizer_backend or backend, model=optimizer_model or model, codex_path=codex_path)
+                      codex_path=codex_path, azure_endpoint=azure_endpoint,
+                      project_dir=project_dir)
+    opt = get_backend(optimizer_backend or backend, model=optimizer_model or model,
+                      codex_path=codex_path, azure_endpoint=azure_endpoint,
+                      project_dir=project_dir)
     opt.preferences = preferences  # reflect runs on the optimizer
     dual = DualBackend(target=tgt, optimizer=opt)
     dual.preferences = preferences

skillopt_sleep/config.py

@@ -13,17 +13,19 @@ from __future__ import annotations
 
 import json
 import os
-from dataclasses import dataclass, field, asdict
+from dataclasses import dataclass, field
-from typing import Any, Dict, List, Optional
+from typing import Any, Dict, Optional
-
 
 HOME_STATE_DIR = os.path.expanduser("~/.skillopt-sleep")
 CLAUDE_HOME = os.path.expanduser("~/.claude")
+CODEX_HOME = os.path.expanduser("~/.codex")
 
 
 DEFAULTS: Dict[str, Any] = {
     # ── scope ──────────────────────────────────────────────────────────────
     "claude_home": CLAUDE_HOME,
+    "codex_home": CODEX_HOME,
+    "transcript_source": "claude",  # "claude" | "codex" | "auto"
     "projects": "invoked",        # "invoked" | "all" | [list of abs paths]
     "invoked_project": "",        # filled at runtime (cwd) when projects == "invoked"
     "lookback_hours": 72,         # harvest window when no prior sleep recorded
@@ -34,7 +36,7 @@ DEFAULTS: Dict[str, Any] = {
     "val_fraction": 0.34,         # real tasks reserved to gate updates
     "test_fraction": 0.0,         # real tasks reserved as the final held-out measure
     # ── optimizer ──────────────────────────────────────────────────────────
-    "backend": "mock",            # "mock" | "claude" | "codex"
+    "backend": "mock",            # "mock" | "claude" | "codex" | "copilot"
     "model": "",                  # backend-specific; "" => backend default
     "gate_mode": "on",            # "on" (validation-gated) | "off" (greedy, no hard filter)
     "codex_path": "",             # "" => auto-detect the real @openai/codex binary
@@ -42,9 +44,16 @@ DEFAULTS: Dict[str, Any] = {
     "gate_metric": "mixed",       # hard | soft | mixed (mixed best for tiny holdouts)
     "gate_mixed_weight": 0.5,
     "replay_mode": "mock",        # "mock" (sandboxed prompt) | "fresh" (worktree)
+    # ── dream + recall (opt-in; defaults reproduce the prior single-shot loop) ─
+    "dream_rollouts": 1,          # >1 => multi-rollout contrastive reflection per task
+    "dream_factor": 0,            # >0 => add N synthetic variants of each task to the dream
+    "recall_k": 0,                # >0 => recall the K most-similar past tasks into the dream
     "evolve_memory": True,        # consolidate CLAUDE.md
     "evolve_skill": True,         # consolidate the managed SKILL.md
     "llm_mine": True,             # use the backend to mine checkable tasks (real backends)
+    "target_skill_path": "",      # explicit SKILL.md target for repo-scoped agents
+    "target_task_filter": True,   # prefer mined tasks matching target_skill_path/text
+    "progress": False,            # print phase progress to stderr
     # ── adoption / safety ──────────────────────────────────────────────────
     "auto_adopt": False,          # default: stage + require explicit `adopt`
     "managed_skill_name": "skillopt-sleep-learned",
@@ -94,6 +103,10 @@ class SleepConfig:
     def transcripts_dir(self) -> str:
         return os.path.join(self.data["claude_home"], "projects")
 
+    @property
+    def codex_archived_sessions_dir(self) -> str:
+        return os.path.join(self.data["codex_home"], "archived_sessions")
+
     @property
     def history_path(self) -> str:
         return os.path.join(self.data["claude_home"], "history.jsonl")
@@ -103,6 +116,13 @@ class SleepConfig:
         return os.path.join(self.data["claude_home"], "skills")
 
     def managed_skill_path(self) -> str:
+        target = self.data.get("target_skill_path") or ""
+        if target:
+            target = os.path.expanduser(str(target))
+            if not os.path.isabs(target):
+                base = self.data.get("invoked_project") or os.getcwd()
+                target = os.path.join(base, target)
+            return os.path.abspath(target)
         return os.path.join(
             self.skills_dir, self.data["managed_skill_name"], "SKILL.md"
         )

skillopt_sleep/consolidate.py

@@ -9,7 +9,7 @@ validation gate, vendored self-contained in ``skillopt_sleep.gate``.
 from __future__ import annotations
 
 import os
-from dataclasses import dataclass
+from dataclasses import dataclass, field
 from typing import List, Optional, Tuple
 
 from skillopt_sleep.backend import Backend
@@ -36,6 +36,10 @@ class ConsolidationResult:
     rejected_edits: List[EditRecord]
     holdout_baseline: float
     holdout_candidate: float
+    # ── observability (so a 0.0->0.0 night is self-diagnosing, not a black box) ──
+    holdout_detail: List[dict] = field(default_factory=list)  # per val task: hard/soft/resp/why
+    reflect_raw: str = ""        # the optimizer's last raw reply (empty => reflect produced nothing)
+    call_error: str = ""         # backend's last call error (timeout/auth/empty)
 
 
 def _split(tasks: List[TaskRecord]) -> Tuple[List[TaskRecord], List[TaskRecord]]:
@@ -61,6 +65,25 @@ def _split(tasks: List[TaskRecord]) -> Tuple[List[TaskRecord], List[TaskRecord]]
     return train, val
 
 
+def _holdout_detail(pairs: List[Tuple[TaskRecord, ReplayResult]]) -> List[dict]:
+    """Per-task held-out evidence so a 0.0 night explains itself: was the
+    response empty (backend call failed) or non-empty-but-failing-checks
+    (judge too strict / edit didn't help)? The two need opposite fixes."""
+    out: List[dict] = []
+    for t, r in pairs:
+        resp = r.response or ""
+        out.append({
+            "id": t.id,
+            "reference_kind": t.reference_kind,
+            "hard": r.hard,
+            "soft": r.soft,
+            "response_len": len(resp),
+            "response_head": resp[:200],
+            "why": (r.fail_reason or r.judge_rationale or "")[:200],
+        })
+    return out
+
+
 def consolidate(
     backend: Backend,
     tasks: List[TaskRecord],
@@ -87,10 +110,19 @@ def consolidate(
     """
     train_tasks, val_tasks = _split(tasks)
     gate_off = str(gate_mode).strip().lower() in {"off", "none", "false", "greedy"}
+    holdout_detail: List[dict] = []
 
     # ── baseline on the VAL slice (the gate reference) ────────────────────
-    base_pairs = replay_batch(backend, val_tasks, skill, memory)
+    # When the gate is OFF the user has opted out of holding out a validation set
-    base_hard, base_soft = aggregate_scores(base_pairs)
+    # (the daily-use design): we accept edits greedily and judge quality only on
+    # the real test set, scored by the caller. So we SKIP all val scoring — it is
+    # both wasted cost and contrary to the "no val set required" design.
+    if gate_off:
+        base_hard, base_soft = 0.0, 0.0
+    else:
+        base_pairs = replay_batch(backend, val_tasks, skill, memory)
+        base_hard, base_soft = aggregate_scores(base_pairs)
+        holdout_detail = _holdout_detail(base_pairs)
     base_score = select_gate_score(base_hard, base_soft, gate_metric, gate_mixed_weight)
 
     # ── reflect over TRAIN-split failures/successes ───────────────────────
@@ -109,14 +141,17 @@ def consolidate(
         new_doc, applied = apply_edits(doc, edits)
         if not applied:
             return doc
-        # score the candidate on the VAL slice
+        # gate OFF: accept greedily with NO val scoring (the daily-use path)
+        if gate_off:
+            all_applied.extend(applied)
+            return new_doc
+        # gate ON: score the candidate on the VAL slice, keep only if it improves
         trial_skill = new_doc if which == "skill" else cand_skill
         trial_memory = new_doc if which == "memory" else cand_memory
         pairs = replay_batch(backend, val_tasks, trial_skill, trial_memory)
         h, s = aggregate_scores(pairs)
         cand_score = select_gate_score(h, s, gate_metric, gate_mixed_weight)
-        # gate OFF: accept greedily (no regression check); gate ON: strict improve
+        if cand_score > base_score:
-        if gate_off or cand_score > base_score:
             base_score = max(base_score, cand_score)
             all_applied.extend(applied)
             return new_doc
@@ -128,8 +163,28 @@ def consolidate(
             # multi-rollout contrastive reflection: run each train task K times
             # and distill a rule from the good-vs-bad contrast (the imagination signal).
             from skillopt_sleep.rollout import multi_rollout, contrastive_reflect
-            sets = [multi_rollout(backend, t, cand_skill, cand_memory, k=rollouts_k)
+            # Parallelize across tasks (each multi_rollout also parallelizes its K
-                    for t in train_tasks]
+            # attempts). This dream phase is the dominant cost; serial execution
+            # times out on real backends. Cap total in-flight at the worker env.
+            import os
+            from concurrent.futures import ThreadPoolExecutor
+            try:
+                _w = int(os.environ.get("SKILLOPT_SLEEP_WORKERS", "1"))
+            except ValueError:
+                _w = 1
+            if _w > 1 and len(train_tasks) > 1:
+                # split the worker budget between task-parallelism and per-task K
+                task_workers = max(1, min(len(train_tasks), _w))
+                per_task = max(1, _w // task_workers)
+                with ThreadPoolExecutor(max_workers=task_workers) as ex:
+                    sets = list(ex.map(
+                        lambda t: multi_rollout(backend, t, cand_skill, cand_memory,
+                                                k=rollouts_k, workers=per_task),
+                        train_tasks))
+            else:
+                sets = [multi_rollout(backend, t, cand_skill, cand_memory,
+                                      k=rollouts_k, workers=1)
+                        for t in train_tasks]
             edits = contrastive_reflect(
                 backend, sets, cand_skill, cand_memory,
                 edit_budget=edit_budget, target="skill",
@@ -158,40 +213,41 @@ def consolidate(
         )
         cand_memory = _gate_apply(cand_memory, edits_m, "memory")
 
-    # ── final decision, scored on the VAL slice ───────────────────────────
+    # ── final decision ────────────────────────────────────────────────────
-    final_pairs = replay_batch(backend, val_tasks, cand_skill, cand_memory)
-    final_hard, final_soft = aggregate_scores(final_pairs)
-    final_score = select_gate_score(final_hard, final_soft, gate_metric, gate_mixed_weight)
-    base_gate_score = select_gate_score(base_hard, base_soft, gate_metric, gate_mixed_weight)
-
     if gate_off:
-        # greedy mode: keep whatever edits we applied; report quality movement
+        # greedy mode: no val scoring at all. Keep whatever edits we applied; the
+        # caller measures real quality on the test set. We report holdout_candidate
+        # as 0.0 (val intentionally not computed in this variant).
+        final_hard, final_soft = 0.0, 0.0
+        final_score = 0.0
         accepted = bool(all_applied)
-        if final_score > base_gate_score:
+        action = "greedy_applied" if all_applied else "greedy_noop"
-            action = "greedy_improved"
+        base_gate_score = 0.0
-        elif final_score < base_gate_score:
-            action = "greedy_regressed"
-        else:
-            action = "greedy_flat" if all_applied else "greedy_noop"
-    elif _HAVE_REPO_GATE:
-        gate = evaluate_gate(
-            candidate_skill=cand_skill,
-            cand_hard=final_hard,
-            current_skill=skill,
-            current_score=base_gate_score,
-            best_skill=skill,
-            best_score=base_gate_score,
-            best_step=night - 1,
-            global_step=night,
-            cand_soft=final_soft,
-            metric=gate_metric,
-            mixed_weight=gate_mixed_weight,
-        )
-        action = gate.action
-        accepted = bool(all_applied) and final_score > base_gate_score
     else:
-        action = "accept" if final_score > base_gate_score else "reject"
+        # scored on the VAL slice (the gate reference)
-        accepted = bool(all_applied) and final_score > base_gate_score
+        final_pairs = replay_batch(backend, val_tasks, cand_skill, cand_memory)
+        final_hard, final_soft = aggregate_scores(final_pairs)
+        final_score = select_gate_score(final_hard, final_soft, gate_metric, gate_mixed_weight)
+        base_gate_score = select_gate_score(base_hard, base_soft, gate_metric, gate_mixed_weight)
+        if _HAVE_REPO_GATE:
+            gate = evaluate_gate(
+                candidate_skill=cand_skill,
+                cand_hard=final_hard,
+                current_skill=skill,
+                current_score=base_gate_score,
+                best_skill=skill,
+                best_score=base_gate_score,
+                best_step=night - 1,
+                global_step=night,
+                cand_soft=final_soft,
+                metric=gate_metric,
+                mixed_weight=gate_mixed_weight,
+            )
+            action = gate.action
+            accepted = bool(all_applied) and final_score > base_gate_score
+        else:
+            action = "accept" if final_score > base_gate_score else "reject"
+            accepted = bool(all_applied) and final_score > base_gate_score
 
     return ConsolidationResult(
         accepted=accepted,
@@ -204,4 +260,7 @@ def consolidate(
         rejected_edits=all_rejected,
         holdout_baseline=base_hard,
         holdout_candidate=final_hard,
+        holdout_detail=holdout_detail,
+        reflect_raw=getattr(backend, "last_reflect_raw", "") or "",
+        call_error=getattr(backend, "last_call_error", "") or "",
     )

skillopt_sleep/cycle.py

@@ -10,18 +10,20 @@ CI use. With backend="anthropic" it spends the user's budget for real lift.
 from __future__ import annotations
 
 import os
-import time
+import sys
 from dataclasses import dataclass
-from typing import Any, Dict, List, Optional
+from typing import List, Optional
 
 from skillopt_sleep.backend import get_backend
 from skillopt_sleep.config import SleepConfig, load_config
-from skillopt_sleep.consolidate import consolidate
+from skillopt_sleep.dream import dream_consolidate
-from skillopt_sleep.harvest import harvest
+from skillopt_sleep.harvest_sources import harvest_for_config
 from skillopt_sleep.memory import ensure_skill_scaffold
 from skillopt_sleep.mine import mine
+from skillopt_sleep.staging import adopt as adopt_staging
+from skillopt_sleep.staging import redact_secrets
+from skillopt_sleep.staging import write_staging
 from skillopt_sleep.state import SleepState, _now_iso
-from skillopt_sleep.staging import write_staging, adopt as adopt_staging
 from skillopt_sleep.types import SessionDigest, SleepReport, TaskRecord
 
 
@@ -49,6 +51,11 @@ def _read(path: str) -> str:
         return ""
 
 
+def _progress(cfg: SleepConfig, message: str) -> None:
+    if cfg.get("progress", False):
+        print(f"[sleep] {message}", file=sys.stderr, flush=True)
+
+
 def _render_report_md(report: SleepReport, cfg: SleepConfig) -> str:
     lines = [
         f"# SkillOpt-Sleep — night {report.night} report",
@@ -108,6 +115,26 @@ def run_sleep_cycle(
         cfg.get("backend", "mock"),
         model=cfg.get("model", ""),
         codex_path=cfg.get("codex_path", ""),
+        project_dir=project,
+    )
+    _progress(cfg, f"night {night}: project={project} backend={backend.name}")
+
+    # ── live skill/memory docs ───────────────────────────────────────────
+    live_memory_path = os.path.join(project, "CLAUDE.md")
+    live_skill_path = cfg.managed_skill_path()
+    _progress(cfg, f"live skill: {live_skill_path}")
+    raw_skill = _read(live_skill_path)
+    skill = raw_skill
+    memory = _read(live_memory_path)
+    if not skill:
+        skill = ensure_skill_scaffold(
+            "", name=cfg.get("managed_skill_name", "skillopt-sleep-learned"),
+            description="Preferences and procedures learned from past local agent sessions.",
+        )
+    target_filter = bool(
+        cfg.get("target_task_filter", True)
+        and cfg.get("target_skill_path", "")
+        and raw_skill
     )
 
     # ── 1+2. harvest + mine (unless seed_tasks injected) ─────────────────
@@ -115,16 +142,34 @@ def run_sleep_cycle(
     if seed_tasks is not None:
         tasks = seed_tasks
         n_sessions = 0
+        _progress(cfg, f"using {len(tasks)} seeded tasks")
     else:
         since = state.last_harvest_for(project)
-        digests = harvest(
+        # On first run (no prior harvest), apply lookback_hours so we don't
-            cfg.transcripts_dir,
+        # scan the entire transcript history and trigger massive LLM mining.
-            scope=cfg.get("projects", "invoked"),
+        if since is None:
-            invoked_project=cfg.get("invoked_project", ""),
+            lookback_hours = cfg.get("lookback_hours", 72)
+            if lookback_hours is not None and lookback_hours > 0:
+                import time
+                ref_time = clock if clock is not None else time.time()
+                cutoff = ref_time - lookback_hours * 3600
+                since = _now_iso(cutoff)
+        max_tasks = cfg.get("max_tasks_per_night", 40)
+        max_sessions = cfg.get("max_sessions_per_night", 0) or max_tasks * 3
+        candidate_limit = max_tasks
+        if target_filter:
+            candidate_limit = max(max_tasks, max_tasks * 3)
+        _progress(
+            cfg,
+            f"harvest start: source={cfg.get('transcript_source')} max_sessions={max_sessions}",
+        )
+        digests = harvest_for_config(
+            cfg,
             since_iso=since,
-            limit=cfg.get("max_tasks_per_night", 40) * 3,
+            limit=max_sessions,
         )
         n_sessions = len(digests)
+        _progress(cfg, f"harvest done: sessions={n_sessions}")
         # When a real backend is configured, use it to mine checkable tasks from
         # the transcripts (rubric/rule judges); otherwise fall back to the
         # heuristic miner (no API, no checkable reference).
@@ -132,27 +177,29 @@ def run_sleep_cycle(
         if cfg.get("backend", "mock") != "mock" and cfg.get("llm_mine", True):
             try:
                 from skillopt_sleep.llm_miner import make_llm_miner
-                llm_miner = make_llm_miner(backend, max_tasks=cfg.get("max_tasks_per_night", 40))
+                llm_miner = make_llm_miner(
+                    backend,
+                    max_sessions=max_sessions,
+                    max_tasks=candidate_limit,
+                )
             except Exception:
                 llm_miner = None
+        _progress(
+            cfg,
+            f"mine start: max_tasks={max_tasks} candidate_limit={candidate_limit} "
+            f"llm_mine={llm_miner is not None} target_filter={target_filter}",
+        )
         tasks = mine(
             digests,
-            max_tasks=cfg.get("max_tasks_per_night", 40),
+            max_tasks=max_tasks,
+            candidate_limit=candidate_limit,
             holdout_fraction=cfg.get("holdout_fraction", 0.34),
             seed=cfg.get("seed", 42),
             llm_miner=llm_miner,
+            target_skill_text=raw_skill if target_filter else "",
+            target_skill_path=live_skill_path if target_filter else "",
         )
-
+        _progress(cfg, f"mine done: tasks={len(tasks)}")
-    # ── live skill/memory docs ───────────────────────────────────────────
-    live_memory_path = os.path.join(project, "CLAUDE.md")
-    live_skill_path = cfg.managed_skill_path()
-    skill = _read(live_skill_path)
-    memory = _read(live_memory_path)
-    if not skill:
-        skill = ensure_skill_scaffold(
-            "", name=cfg.get("managed_skill_name", "skillopt-sleep-learned"),
-            description="Preferences and procedures learned from past Claude Code sessions.",
-        )
 
     report = SleepReport(
         night=night, project=project, started_at=started,
@@ -169,9 +216,22 @@ def run_sleep_cycle(
         staging_dir = ""
         return CycleOutcome(report, staging_dir, False, [])
 
-    # ── 3+4. replay + consolidate (gate) ─────────────────────────────────
+    # ── 3+4. replay + consolidate (gate), with opt-in dream + recall ──────
-    result = consolidate(
+    # recall pulls similar past tasks from the persisted archive; dream_rollouts
+    # / dream_factor enrich the training signal. With the defaults (recall_k=0,
+    # dream_rollouts=1, dream_factor=0) this is exactly the prior single-shot
+    # consolidate — behavior is unchanged unless the user opts in.
+    _progress(cfg, "consolidate start")
+    recall_k = int(cfg.get("recall_k", 0) or 0)
+    history_tasks = []
+    if recall_k > 0:
+        history_tasks = [TaskRecord.from_dict(d) for d in state.task_archive()]
+    result = dream_consolidate(
         backend, tasks, skill, memory,
+        history_tasks=history_tasks,
+        recall_k=recall_k,
+        dream_rollouts=int(cfg.get("dream_rollouts", 1) or 1),
+        dream_factor=int(cfg.get("dream_factor", 0) or 0),
         edit_budget=cfg.get("edit_budget", 4),
         gate_metric=cfg.get("gate_metric", "mixed"),
         gate_mixed_weight=cfg.get("gate_mixed_weight", 0.5),
@@ -180,12 +240,20 @@ def run_sleep_cycle(
         evolve_memory=cfg.get("evolve_memory", True),
         night=night,
     )
+    # archive tonight's real (non-dream) tasks so future nights can recall them
+    state.add_to_archive([t.to_dict() for t in tasks if t.origin != "dream"])
+    _progress(
+        cfg,
+        f"consolidate done: gate={result.gate_action} accepted={result.accepted} "
+        f"edits={len(result.applied_edits)} rejected={len(result.rejected_edits)}",
+    )
 
     report.n_replayed = len(tasks)
     report.baseline_score = result.baseline_score
     report.candidate_score = result.candidate_score
     report.accepted = result.accepted
     report.gate_action = result.gate_action
+    report.no_edits_reason = getattr(result, "no_edits_reason", "")
     report.edits = result.applied_edits
     report.rejected_edits = result.rejected_edits
     report.tokens_used = backend.tokens_used()
@@ -196,6 +264,7 @@ def run_sleep_cycle(
     adopted = False
     adopted_paths: List[str] = []
     if not dry_run:
+        _progress(cfg, "staging start")
         report_md = _render_report_md(report, cfg)
         proposed_skill = result.new_skill if (cfg.get("evolve_skill") and result.accepted) else None
         proposed_memory = result.new_memory if (cfg.get("evolve_memory") and result.accepted) else None
@@ -208,6 +277,33 @@ def run_sleep_cycle(
             live_memory_path=live_memory_path,
             report_md=report_md,
         )
+        # Observability: persist per-task held-out evidence + optimizer/codex errors so a
+        # 0.0->0.0 night self-explains (empty responses vs failing checks vs no edits) — the
+        # cycle previously captured none of this, making the gate a black box (#learning-stall).
+        try:
+            import json as _json
+            # Backend stderr / optimizer replies / task responses can carry
+            # credentials (e.g. a codex 401 stderr dump), so scrub secret-looking
+            # substrings before persisting them to the on-disk diagnostics.
+            with open(os.path.join(staging_dir, "diagnostics.json"), "w", encoding="utf-8") as _fh:
+                _json.dump({
+                    "night": night,
+                    "backend": cfg.get("backend"),
+                    "gate_mode": cfg.get("gate_mode"),
+                    "n_tasks": len(tasks),
+                    "baseline_score": result.baseline_score,
+                    "candidate_score": result.candidate_score,
+                    "accepted": result.accepted,
+                    "n_applied_edits": len(result.applied_edits),
+                    "n_rejected_edits": len(result.rejected_edits),
+                    "call_error": redact_secrets(getattr(result, "call_error", "")),
+                    "reflect_raw_head": redact_secrets(
+                        (getattr(result, "reflect_raw", "") or "")[:1200]
+                    ),
+                    "holdout_detail": redact_secrets(getattr(result, "holdout_detail", [])),
+                }, _fh, indent=2)
+        except Exception:
+            pass
         state.set_last_harvest(project, started)
         state.record_night({
             "night": night, "accepted": result.accepted,

skillopt_sleep/dream.py

@@ -0,0 +1,138 @@
+"""SkillOpt-Sleep — dream + associative recall for nightly consolidation.
+
+Two opt-in mechanisms (both default OFF, so the cycle is unchanged unless the
+user enables them) that the deployment experiments validated:
+
+  * dream rollouts  — run each task K times and learn from the good-vs-bad
+    contrast (set ``dream_rollouts > 1``). Stronger signal than one failure.
+  * associative recall — each night, pull the K past tasks most similar to
+    tonight's new ones into the dream (set ``recall_k > 0``). Replays relevant
+    experience without re-running the whole history.
+
+``dream_consolidate`` wires recall + synthetic augmentation + multi-rollout
+consolidation and is called by BOTH the shipped plugin cycle and the benchmark
+experiment harness, so the reported numbers exercise the exact code the plugin
+runs. Pure-stdlib, zero research/private dependency.
+"""
+from __future__ import annotations
+
+import re
+from typing import List, Optional
+
+from skillopt_sleep.consolidate import ConsolidationResult, consolidate
+from skillopt_sleep.types import TaskRecord
+
+
+# ── synthetic augmentation ("dream up" variants of today's tasks) ─────────────
+
+_WRAPPERS = [
+    "(quick one) {q}",
+    "Please handle this request: {q}",
+    "For the daily report: {q}",
+]
+
+
+def dream_augment(real_tasks: List[TaskRecord], *, factor: int = 1) -> List[TaskRecord]:
+    """Create synthetic TRAIN variants of real tasks (origin='dream').
+
+    A light, deterministic rephrasing. Dream tasks are training-only — they
+    carry split='train' and never enter the val/test slices the gate scores on.
+    """
+    out: List[TaskRecord] = []
+    for t in real_tasks:
+        for k in range(max(0, factor)):
+            w = _WRAPPERS[k % len(_WRAPPERS)]
+            out.append(TaskRecord(
+                id=f"{t.id}_dream{k}", project=t.project,
+                intent=w.format(q=t.intent), context_excerpt=t.context_excerpt,
+                reference_kind=t.reference_kind, reference=t.reference,
+                judge=dict(t.judge), system=t.system,
+                tags=list(t.tags) + ["dream"], split="train",
+                origin="dream", derived_from=t.id,
+            ))
+    return out
+
+
+# ── associative recall (experience replay of similar past tasks) ──────────────
+
+def _tokens(text: str) -> set:
+    return {w for w in re.findall(r"[a-z0-9]+", (text or "").lower()) if len(w) > 2}
+
+
+def recall_similar(new_tasks: List[TaskRecord], history: List[TaskRecord],
+                   k: int) -> List[TaskRecord]:
+    """Return the ``k`` historical tasks most lexically similar to any of
+    tonight's ``new_tasks`` (max Jaccard token overlap). Recalled tasks are
+    returned as training material (split='train'); deterministic, stdlib-only.
+    """
+    if not history or k <= 0 or not new_tasks:
+        return []
+    new_tok = [_tokens(t.intent) for t in new_tasks]
+    new_ids = {t.id for t in new_tasks}
+    scored = []
+    for h in history:
+        if h.id in new_ids:
+            continue
+        ht = _tokens(h.intent)
+        if not ht:
+            continue
+        sim = max(((len(ht & nt) / len(ht | nt)) if (ht | nt) else 0.0) for nt in new_tok)
+        scored.append((sim, h.id, h))
+    scored.sort(key=lambda x: (-x[0], x[1]))
+    out = []
+    for sim, _id, h in scored[:max(0, k)]:
+        if sim <= 0.0:
+            break
+        # recall as training material; copy so the source archive is untouched
+        out.append(TaskRecord(
+            id=f"recall:{h.id}", project=h.project, intent=h.intent,
+            context_excerpt=h.context_excerpt, reference_kind=h.reference_kind,
+            reference=h.reference, judge=dict(h.judge), system=h.system,
+            tags=list(h.tags) + ["recall"], split="train", origin="real",
+            derived_from=h.id,
+        ))
+    return out
+
+
+# ── the shared nightly consolidation step ─────────────────────────────────────
+
+def dream_consolidate(
+    backend,
+    tasks: List[TaskRecord],
+    skill: str,
+    memory: str,
+    *,
+    history_tasks: Optional[List[TaskRecord]] = None,
+    recall_k: int = 0,
+    dream_rollouts: int = 1,
+    dream_factor: int = 0,
+    edit_budget: int = 4,
+    gate_metric: str = "mixed",
+    gate_mixed_weight: float = 0.5,
+    gate_mode: str = "on",
+    evolve_skill: bool = True,
+    evolve_memory: bool = True,
+    night: int = 1,
+) -> ConsolidationResult:
+    """Recall similar past experience + dream synthetic variants, then run one
+    gated consolidation epoch over the enlarged training pool.
+
+    ``tasks`` is the split-tagged pool for tonight (train + val); recall and
+    augmentation only enlarge the TRAIN split, so the val slice the gate scores
+    on is never polluted. With ``recall_k=0`` and ``dream_rollouts=1`` (the
+    defaults) this is exactly the previous single-shot ``consolidate``.
+    """
+    train = [t for t in tasks if t.split == "train"]
+    enlarged = list(tasks)
+    if recall_k > 0 and history_tasks:
+        enlarged += recall_similar(train, history_tasks, recall_k)
+    if dream_factor > 0:
+        seed = [t for t in enlarged if t.split == "train" and t.origin != "dream"]
+        enlarged += dream_augment(seed, factor=dream_factor)
+    return consolidate(
+        backend, enlarged, skill, memory,
+        edit_budget=edit_budget, gate_metric=gate_metric,
+        gate_mixed_weight=gate_mixed_weight, gate_mode=gate_mode,
+        rollouts_k=dream_rollouts, evolve_skill=evolve_skill,
+        evolve_memory=evolve_memory, night=night,
+    )

skillopt_sleep/experiments/run_experiment.py

@@ -134,7 +134,7 @@ def main(argv=None) -> int:
     ap = argparse.ArgumentParser(description="SkillOpt-Sleep validation experiment")
     ap.add_argument("--persona", default="researcher", choices=list(PERSONAS.keys()))
     ap.add_argument("--nights", type=int, default=4)
-    ap.add_argument("--backend", default="mock", choices=["mock", "claude", "codex"])
+    ap.add_argument("--backend", default="mock", choices=["mock", "claude", "codex", "copilot"])
     ap.add_argument("--model", default="", help="backend model override")
     ap.add_argument("--codex-path", default="", help="path to the real @openai/codex binary")
     ap.add_argument("--edit-budget", type=int, default=4)

skillopt_sleep/harvest.py

@@ -17,6 +17,7 @@ from __future__ import annotations
 
 import json
 import os
+from datetime import datetime
 from typing import Any, Dict, Iterable, List, Optional
 
 from skillopt_sleep.types import SessionDigest
@@ -111,6 +112,58 @@ def _is_meta_prompt(text: str) -> bool:
     return False
 
 
+# ── Issue #62: filter headless replay sessions ─────────────────────────
+
+# Prompt markers generated by the engine's own headless `claude -p` calls
+# (judge, reflect, attempt). If the sole user prompt in a single-turn
+# session matches any of these, the session is engine-generated, not a
+# real user task.
+_REPLAY_PROMPT_MARKERS = (
+    "## CURRENT SKILL",
+    "## FAILED TASKS",
+    "## SUCCESSFUL TASKS",
+    "## OUTPUT FORMAT",
+    "You are a strict grader",
+    "Score the response 0.0-1.0",
+    "You are SkillOpt-Sleep",
+    "## TASK\n",
+    "## SKILL\n",
+)
+
+
+def _is_headless_replay(digest: "SessionDigest") -> bool:
+    """Detect sessions created by the engine's own headless replay calls.
+
+    Heuristics (conservatively applied):
+    1. Session has exactly 1 user turn AND
+    2. The sole prompt matches engine-generated patterns (grader/reflect),
+       OR the session lasted < 3 seconds (programmatic, not interactive).
+    Multi-turn sessions are always kept (interactive by definition).
+    """
+    if digest.n_user_turns > 1:
+        return False
+    if digest.n_user_turns == 0:
+        return True
+    prompt = digest.user_prompts[0] if digest.user_prompts else ""
+    for marker in _REPLAY_PROMPT_MARKERS:
+        if marker in prompt:
+            return True
+    # Sub-3-second single-turn sessions with short prompts are almost
+    # certainly programmatic (engine grader/judge calls).  We require the
+    # prompt to also be short (<200 chars) to avoid false-positives on
+    # real one-shot questions that Claude happens to answer quickly.
+    if digest.started_at and digest.ended_at and len(prompt) < 200:
+        try:
+            fmt = "%Y-%m-%dT%H:%M:%S"
+            start = datetime.strptime(digest.started_at[:19], fmt)
+            end = datetime.strptime(digest.ended_at[:19], fmt)
+            if (end - start).total_seconds() < 3:
+                return True
+        except (ValueError, TypeError):
+            pass
+    return False
+
+
 def digest_transcript(path: str) -> Optional[SessionDigest]:
     """Build a SessionDigest from one ``<sessionId>.jsonl`` transcript."""
     session_id = os.path.splitext(os.path.basename(path))[0]
@@ -236,9 +289,14 @@ def harvest(
         d = digest_transcript(p)
         if d is None:
             continue
+        if _is_headless_replay(d):
+            continue  # Issue #62: skip engine's own headless replay sessions
         if not _project_matches(d.project or "", scope, invoked_project):
             continue
         if since_iso and d.ended_at and d.ended_at < since_iso:
+            # Note: files are sorted by mtime but we compare the embedded
+            # ended_at timestamp — mtime can diverge (copy/touch), so we
+            # cannot break here; we must continue to check all files.
             continue
         digests.append(d)
         if limit and len(digests) >= limit:

skillopt_sleep/harvest_codex.py

@@ -0,0 +1,233 @@
+"""SkillOpt-Sleep Codex Desktop session harvesting.
+
+Reads Codex Desktop archived session JSONL files and normalizes them into
+``SessionDigest`` records without copying developer/system instructions, tool
+arguments, or raw tool outputs.
+"""
+from __future__ import annotations
+
+import os
+import re
+from typing import Any, Dict, Iterable, List, Optional
+
+from skillopt_sleep.harvest import (
+    _detect_feedback,
+    _is_meta_prompt,
+    _iter_jsonl,
+    _project_matches,
+)
+from skillopt_sleep.staging import _SECRET_PATTERNS
+from skillopt_sleep.types import SessionDigest
+
+
+def _payload(rec: Dict[str, Any]) -> Dict[str, Any]:
+    payload = rec.get("payload")
+    return payload if isinstance(payload, dict) else {}
+
+
+def _timestamp(rec: Dict[str, Any], payload: Dict[str, Any]) -> str:
+    for value in (
+        payload.get("timestamp"),
+        rec.get("timestamp"),
+        payload.get("started_at"),
+        payload.get("completed_at"),
+    ):
+        if isinstance(value, str) and value:
+            return value
+    return ""
+
+
+def _text_from_any(content: Any) -> str:
+    if isinstance(content, str):
+        return content
+    if isinstance(content, list):
+        parts: List[str] = []
+        for item in content:
+            if isinstance(item, str):
+                parts.append(item)
+            elif isinstance(item, dict):
+                if item.get("type") == "text" and item.get("text"):
+                    parts.append(str(item["text"]))
+                elif item.get("text"):
+                    parts.append(str(item["text"]))
+        return "\n".join(parts)
+    if isinstance(content, dict):
+        if content.get("text"):
+            return str(content["text"])
+        if content.get("content"):
+            return _text_from_any(content["content"])
+    return ""
+
+
+def _strip_codex_meta(text: str) -> str:
+    stripped = text.strip()
+    if not stripped:
+        return ""
+    if stripped.startswith("<codex_internal_context"):
+        return ""
+    if stripped.startswith("<environment_context"):
+        return ""
+    if stripped.startswith("# AGENTS.md instructions") or "--- project-doc ---" in stripped:
+        for marker in ("</environment_context>", "</INSTRUCTIONS>"):
+            idx = stripped.rfind(marker)
+            if idx == -1:
+                continue
+            tail = stripped[idx + len(marker):].strip()
+            if tail and not tail.startswith("<"):
+                return tail
+        return ""
+    return stripped
+
+
+def _sanitize_text(text: str) -> str:
+    sanitized = _strip_codex_meta(text).replace("\x00", "").strip()
+    if not sanitized or _is_meta_prompt(sanitized):
+        return ""
+    for pattern, replacement in _SECRET_PATTERNS:
+        sanitized = pattern.sub(replacement, sanitized)
+    return sanitized
+
+
+def _sanitize_tool_name(name: str) -> str:
+    return re.sub(r"[^A-Za-z0-9_.:-]+", "_", name)[:80]
+
+
+def _tool_name(payload: Dict[str, Any]) -> str:
+    payload_type = payload.get("type")
+    name = payload.get("name")
+    if isinstance(name, str) and name:
+        return _sanitize_tool_name(name)
+    if payload_type == "exec_command_end":
+        return "exec_command"
+    if payload_type == "patch_apply_end":
+        return "apply_patch"
+    if payload_type == "web_search_call":
+        return "web_search"
+    if payload_type == "tool_search_call":
+        return "tool_search"
+    if isinstance(payload_type, str) and payload_type.endswith("_tool_call"):
+        return _sanitize_tool_name(payload_type)
+    return ""
+
+
+def _dedup(xs: Iterable[str]) -> List[str]:
+    seen = set()
+    out: List[str] = []
+    for x in xs:
+        if x not in seen:
+            seen.add(x)
+            out.append(x)
+    return out
+
+
+def digest_codex_archived_session(path: str, project: str = "") -> Optional[SessionDigest]:
+    """Build a ``SessionDigest`` from one Codex Desktop archived session."""
+    session_id = os.path.splitext(os.path.basename(path))[0]
+    started = ""
+    ended = ""
+    session_project = ""
+    user_prompts: List[str] = []
+    assistant_finals: List[str] = []
+    tools: List[str] = []
+    feedback: List[str] = []
+    n_user = 0
+    n_asst = 0
+
+    for rec in _iter_jsonl(path):
+        payload = _payload(rec)
+        payload_type = payload.get("type")
+        ts = _timestamp(rec, payload)
+        if ts:
+            if not started:
+                started = ts
+            ended = ts
+        cwd = payload.get("cwd")
+        if isinstance(cwd, str) and cwd:
+            if not session_project:
+                session_project = cwd
+            if project and _project_matches(cwd, "invoked", project):
+                session_project = cwd
+
+        role = payload.get("role")
+        text = ""
+        output_role = ""
+        if payload_type == "user_message":
+            text = _text_from_any(payload.get("message"))
+            output_role = "user"
+        elif payload_type == "agent_message":
+            text = _text_from_any(payload.get("message"))
+            output_role = "assistant"
+        elif payload_type == "message" and role in {"user", "assistant"}:
+            text = _text_from_any(payload.get("content"))
+            output_role = str(role)
+        else:
+            tool = _tool_name(payload)
+            if tool:
+                tools.append(tool)
+            continue
+
+        sanitized = _sanitize_text(text)
+        if not sanitized:
+            continue
+        if output_role == "user":
+            n_user += 1
+            user_prompts.append(sanitized)
+            feedback.extend(_detect_feedback(sanitized))
+        elif output_role == "assistant":
+            n_asst += 1
+            assistant_finals.append(sanitized)
+
+    if project and not _project_matches(session_project or "", "invoked", project):
+        return None
+    if n_user == 0 and n_asst == 0:
+        return None
+
+    return SessionDigest(
+        session_id=session_id,
+        project=session_project,
+        started_at=started,
+        ended_at=ended,
+        user_prompts=user_prompts,
+        assistant_finals=assistant_finals[-5:],
+        tools_used=_dedup(tools),
+        files_touched=[],
+        feedback_signals=feedback,
+        n_user_turns=n_user,
+        n_assistant_turns=n_asst,
+        raw_path=path,
+    )
+
+
+def harvest_codex(
+    archived_sessions_dir: str,
+    *,
+    scope: Any = "all",
+    invoked_project: str = "",
+    since_iso: Optional[str] = None,
+    limit: int = 0,
+) -> List[SessionDigest]:
+    """Walk ``~/.codex/archived_sessions`` and return matching digests."""
+    digests: List[SessionDigest] = []
+    if not os.path.isdir(archived_sessions_dir):
+        return digests
+
+    paths = [
+        os.path.join(archived_sessions_dir, fn)
+        for fn in os.listdir(archived_sessions_dir)
+        if fn.endswith(".jsonl")
+    ]
+    paths.sort(key=lambda p: os.path.getmtime(p), reverse=True)
+
+    project_hint = invoked_project if scope == "invoked" else ""
+    for path in paths:
+        digest = digest_codex_archived_session(path, project=project_hint)
+        if digest is None:
+            continue
+        if not _project_matches(digest.project or "", scope, invoked_project):
+            continue
+        if since_iso and digest.ended_at and digest.ended_at < since_iso:
+            continue
+        digests.append(digest)
+        if limit and len(digests) >= limit:
+            break
+    return digests

skillopt_sleep/harvest_sources.py

@@ -0,0 +1,41 @@
+"""Source selection for SkillOpt-Sleep transcript harvesting."""
+from __future__ import annotations
+
+from typing import Optional
+
+from skillopt_sleep.harvest import harvest
+from skillopt_sleep.harvest_codex import harvest_codex
+from skillopt_sleep.types import SessionDigest
+
+
+def harvest_for_config(cfg, *, since_iso: Optional[str] = None, limit: int = 0) -> list[SessionDigest]:
+    source = cfg.get("transcript_source", "claude")
+    scope = cfg.get("projects", "invoked")
+    invoked_project = cfg.get("invoked_project", "")
+
+    if source == "codex":
+        return harvest_codex(
+            cfg.codex_archived_sessions_dir,
+            scope=scope,
+            invoked_project=invoked_project,
+            since_iso=since_iso,
+            limit=limit,
+        )
+    if source == "auto":
+        codex_digests = harvest_codex(
+            cfg.codex_archived_sessions_dir,
+            scope=scope,
+            invoked_project=invoked_project,
+            since_iso=since_iso,
+            limit=limit,
+        )
+        if codex_digests:
+            return codex_digests
+
+    return harvest(
+        cfg.transcripts_dir,
+        scope=scope,
+        invoked_project=invoked_project,
+        since_iso=since_iso,
+        limit=limit,
+    )

skillopt_sleep/memory.py

@@ -12,7 +12,6 @@ from typing import List, Tuple
 
 from skillopt_sleep.types import EditRecord
 
-
 LEARNED_START = "<!-- SKILLOPT-SLEEP:LEARNED START -->"
 LEARNED_END = "<!-- SKILLOPT-SLEEP:LEARNED END -->"
 _BANNER = (
@@ -79,7 +78,7 @@ def apply_edits(doc: str, edits: List[EditRecord]) -> Tuple[str, List[EditRecord
     anchor substring.
     """
     lines = current_learned_lines(doc)
-    norm_set = {_norm(l) for l in lines}
+    norm_set = {_norm(line) for line in lines}
     applied: List[EditRecord] = []
 
     for e in edits:
@@ -92,31 +91,31 @@ def apply_edits(doc: str, edits: List[EditRecord]) -> Tuple[str, List[EditRecord
             applied.append(e)
         elif op == "delete":
             anchor = _norm(e.anchor or e.content)
-            keep = [l for l in lines if anchor not in _norm(l)]
+            keep = [line for line in lines if anchor not in _norm(line)]
             if len(keep) != len(lines):
                 lines = keep
-                norm_set = {_norm(l) for l in lines}
+                norm_set = {_norm(line) for line in lines}
                 applied.append(e)
         elif op == "replace":
             anchor = _norm(e.anchor)
             new_lines = []
             changed = False
-            for l in lines:
+            for line in lines:
-                if anchor and anchor in _norm(l):
+                if anchor and anchor in _norm(line):
                     new_lines.append(e.content.strip())
                     changed = True
                 else:
-                    new_lines.append(l)
+                    new_lines.append(line)
             if changed:
                 lines = new_lines
-                norm_set = {_norm(l) for l in lines}
+                norm_set = {_norm(line) for line in lines}
                 applied.append(e)
 
     return set_learned(doc, lines), applied
 
 
 def ensure_skill_scaffold(doc: str, *, name: str, description: str) -> str:
-    """Ensure a SKILL.md has YAML frontmatter so Claude Code loads it."""
+    """Ensure a SKILL.md has YAML frontmatter so local agents load it."""
     if doc.lstrip().startswith("---"):
         return doc
     fm = (
@@ -125,6 +124,6 @@ def ensure_skill_scaffold(doc: str, *, name: str, description: str) -> str:
         f"description: {description}\n"
         "---\n\n"
         f"# {name}\n\n"
-        "Preferences and procedures learned from your past Claude Code sessions.\n"
+        "Preferences and procedures learned from your past local agent sessions.\n"
     )
     return fm + doc

skillopt_sleep/mine.py

@@ -15,8 +15,10 @@ basis of the deterministic experiment.
 from __future__ import annotations
 
 import hashlib
+import os
 import re
-from typing import Any, Callable, List, Optional
+from collections import Counter
+from typing import Any, Callable, List, Optional, Set, Tuple
 
 from skillopt_sleep.types import SessionDigest, TaskRecord
 
@@ -39,6 +41,99 @@ def _looks_positive(signals: List[str]) -> bool:
     return any(s.startswith("pos:") for s in signals)
 
 
+_TARGET_STOPWORDS = {
+    "about", "after", "again", "agent", "agents", "all", "also", "always",
+    "and", "any", "are", "before", "being", "but", "can", "codex",
+    "current", "default", "docs", "does", "done", "each", "file", "files",
+    "for", "from", "have", "into", "keep", "must", "not", "only", "path",
+    "paths", "project", "read", "repo", "request", "requests", "rule",
+    "rules", "same", "should", "skill", "skills", "source", "start",
+    "task", "tasks", "that", "the", "their", "then", "this", "unless",
+    "update", "user", "users", "when", "with", "work", "workflow",
+}
+
+
+def _target_tokens(text: str) -> List[str]:
+    tokens: List[str] = []
+    for raw in re.findall(r"[\w][\w.-]*", (text or "").lower(), flags=re.UNICODE):
+        parts = [raw] + re.split(r"[\W_]+", raw, flags=re.UNICODE)
+        for part in parts:
+            if len(part) < 3 or part.isdigit() or part in _TARGET_STOPWORDS:
+                continue
+            tokens.append(part)
+    return tokens
+
+
+def _expand_target_keywords(keywords: Set[str]) -> None:
+    if "mcp" in keywords:
+        keywords.update({
+            "configure", "configuration", "connect", "connected", "enable",
+            "enabled", "install", "installed", "server", "servers",
+            "настрой", "настроить", "подключи", "подключить",
+        })
+    if {"conflict", "conflicts"} & keywords:
+        keywords.update({
+            "cherry", "conflict", "conflicts", "git", "merge", "rebase",
+            "unmerged", "конфликт", "конфликты",
+        })
+
+
+def target_task_keywords(
+    target_skill_text: str,
+    target_skill_path: str = "",
+    *,
+    limit: int = 180,
+) -> Tuple[Set[str], Set[str]]:
+    """Return (strong, weak) keywords that describe a target skill."""
+    path_text = (target_skill_path or "").replace(os.sep, " ")
+    headings = "\n".join(re.findall(r"(?m)^#+\s+(.+)$", target_skill_text or ""))
+    strong = set(_target_tokens(path_text + "\n" + headings))
+    weak = set(strong)
+    counts = Counter(_target_tokens(target_skill_text or ""))
+    for token, _count in counts.most_common(limit):
+        weak.add(token)
+    _expand_target_keywords(strong)
+    _expand_target_keywords(weak)
+    return strong, weak
+
+
+def _task_search_text(task: TaskRecord) -> str:
+    return "\n".join([
+        task.intent or "",
+        task.context_excerpt or "",
+        " ".join(task.tags or []),
+    ])
+
+
+def filter_tasks_for_target(
+    tasks: List[TaskRecord],
+    target_skill_text: str,
+    target_skill_path: str = "",
+) -> List[TaskRecord]:
+    """Prefer tasks whose language overlaps the explicit target skill.
+
+    If nothing matches, return the original list. This keeps a target run useful
+    even when transcripts are too sparse or the skill is too generic.
+    """
+    strong, weak = target_task_keywords(target_skill_text, target_skill_path)
+    if not tasks or not (strong or weak):
+        return tasks
+
+    ranked = []
+    for idx, task in enumerate(tasks):
+        tokens = set(_target_tokens(_task_search_text(task)))
+        strong_hits = tokens & strong
+        weak_hits = tokens & weak
+        if not strong_hits and len(weak_hits) < 2:
+            continue
+        score = len(strong_hits) * 3 + len(weak_hits)
+        ranked.append((score, idx, task))
+    if not ranked:
+        return tasks
+    ranked.sort(key=lambda item: (-item[0], item[1]))
+    return [task for _score, _idx, task in ranked]
+
+
 def heuristic_mine(
     digests: List[SessionDigest],
     *,
@@ -192,11 +287,15 @@ def mine(
     digests: List[SessionDigest],
     *,
     max_tasks: int = 40,
+    candidate_limit: int = 0,
     holdout_fraction: float = 0.34,
     seed: int = 42,
     llm_miner: Optional[Callable[[List[SessionDigest]], List[TaskRecord]]] = None,
+    target_skill_text: str = "",
+    target_skill_path: str = "",
 ) -> List[TaskRecord]:
     """Top-level miner. Uses ``llm_miner`` if provided, else heuristic."""
+    candidate_limit = candidate_limit or max_tasks
     tasks: List[TaskRecord] = []
     if llm_miner is not None:
         try:
@@ -204,7 +303,10 @@ def mine(
         except Exception:
             tasks = []
     if not tasks:
-        tasks = heuristic_mine(digests, max_tasks=max_tasks)
+        tasks = heuristic_mine(digests, max_tasks=candidate_limit)
     tasks = dedup_tasks(tasks)
+    if target_skill_text or target_skill_path:
+        tasks = filter_tasks_for_target(tasks, target_skill_text, target_skill_path)
+    tasks = tasks[:max_tasks]
     tasks = assign_splits(tasks, holdout_fraction=holdout_fraction, seed=seed)
     return tasks

skillopt_sleep/replay.py

@@ -26,7 +26,11 @@ def _required_tools(task: TaskRecord) -> List[str]:
     return tools
 
 
-def replay_one(backend: Backend, task: TaskRecord, skill: str, memory: str) -> ReplayResult:
+def replay_one(backend: Backend, task: TaskRecord, skill: str, memory: str,
+               sample_id: int = 0) -> ReplayResult:
+    """``sample_id`` distinguishes repeated dream rollouts of the same
+    (task, skill, memory) in the attempt cache — without it all K rollouts
+    collapse to one cached response and the contrastive signal is always 0."""
     import time
     tools = _required_tools(task)
     tools_called: List[str] = []
@@ -35,7 +39,7 @@ def replay_one(backend: Backend, task: TaskRecord, skill: str, memory: str) -> R
     if tools:
         response, tools_called = backend.attempt_with_tools(task, skill, memory, tools)
     else:
-        response = backend.attempt(task, skill, memory)
+        response = backend.attempt(task, skill, memory, sample_id=sample_id)
     latency_ms = (time.time() - t0) * 1000.0
     tokens = max(0, backend.tokens_used() - tok_before)
     # if the backend doesn't track tokens (e.g. mock), approximate from text length
@@ -63,13 +67,37 @@ def replay_one(backend: Backend, task: TaskRecord, skill: str, memory: str) -> R
     )
 
 
+import os
+from concurrent.futures import ThreadPoolExecutor
+
+
 def replay_batch(
     backend: Backend,
     tasks: List[TaskRecord],
     skill: str,
     memory: str,
+    *,
+    workers: int = 0,
 ) -> List[Tuple[TaskRecord, ReplayResult]]:
-    return [(t, replay_one(backend, t, skill, memory)) for t in tasks]
+    """Replay tasks, optionally in parallel.
+
+    Real backends are network-bound, so a thread pool gives a large speedup on
+    big test sets (like the research harness's --workers). ``workers`` defaults
+    to env SKILLOPT_SLEEP_WORKERS or 1 (sequential). Mock stays sequential
+    (deterministic) unless asked otherwise.
+    """
+    if workers <= 0:
+        workers = int(os.environ.get("SKILLOPT_SLEEP_WORKERS", "1") or "1")
+    if workers <= 1 or len(tasks) <= 1:
+        return [(t, replay_one(backend, t, skill, memory)) for t in tasks]
+    results: List = [None] * len(tasks)
+    with ThreadPoolExecutor(max_workers=min(workers, len(tasks))) as ex:
+        futs = {ex.submit(replay_one, backend, t, skill, memory): i
+                for i, t in enumerate(tasks)}
+        for fut in futs:
+            i = futs[fut]
+            results[i] = (tasks[i], fut.result())
+    return results
 
 
 def aggregate_scores(pairs: List[Tuple[TaskRecord, ReplayResult]]) -> Tuple[float, float]:

skillopt_sleep/rollout.py

@@ -58,12 +58,34 @@ def multi_rollout(
     memory: str,
     *,
     k: int = 3,
+    workers: int = 0,
 ) -> RolloutSet:
     """Run ``task`` K times. replay_one is deterministic for mock; for real
-    backends the model's own sampling yields variation across attempts."""
+    backends the model's own sampling yields variation across attempts.
+
+    The K attempts are independent, so they run concurrently (this is the dream
+    phase's dominant cost). ``workers`` defaults to the SKILLOPT_SLEEP_WORKERS
+    env (capped at k); set to 1 to force serial (used by the mock tests).
+    """
+    import os
     rs = RolloutSet(task=task)
-    for _ in range(max(1, k)):
+    k = max(1, k)
-        rs.attempts.append(replay_one(backend, task, skill, memory))
+    if workers <= 0:
+        try:
+            workers = int(os.environ.get("SKILLOPT_SLEEP_WORKERS", "1"))
+        except ValueError:
+            workers = 1
+    workers = max(1, min(workers, k))
+    if workers == 1:
+        for i in range(k):
+            rs.attempts.append(replay_one(backend, task, skill, memory, sample_id=i))
+        return rs
+    from concurrent.futures import ThreadPoolExecutor
+    with ThreadPoolExecutor(max_workers=workers) as ex:
+        futs = [ex.submit(replay_one, backend, task, skill, memory, sample_id=i)
+                for i in range(k)]
+        for f in futs:
+            rs.attempts.append(f.result())
     return rs
 
 
@@ -97,6 +119,11 @@ def contrastive_reflect(
             f"- BAD  attempt (score {rs.worst.hard:.1f}): {rs.worst.response[:200]}\n"
             f"  (bad failed: {rs.worst.fail_reason[:100]})"
         )
+    # the output contract the proposed rules must not violate (same guardrail the
+    # single-shot reflect uses — prevents harness-violating rules like "return VBA"
+    # or "ask the user for the range" on SpreadsheetBench).
+    from skillopt_sleep.backend import _task_guardrail
+    guard = _task_guardrail([(rs.task, rs.best) for rs in informative])
     prompt = (
         "You are SkillOpt's optimizer doing CONTRASTIVE reflection. For each task "
         "below the agent was run multiple times; some attempts succeeded and some "
@@ -104,6 +131,10 @@ def contrastive_reflect(
         f"and propose at most {edit_budget} SHORT, GENERAL, reusable rules for the "
         f"{target} that would make the good behavior reliable every time. Quote "
         "concrete thresholds/formats verbatim; do not paraphrase vaguely. "
+        "Every rule MUST obey the task output contract (if shown) — never propose "
+        "a rule that changes the required output format/language or tells the agent "
+        "to ask the user a question; such a rule scores ZERO.\n"
+        f"{guard}"
         'Return ONLY a JSON array: '
         '[{"op":"add","content":"<rule>","rationale":"<what good did that bad didnt>"}].\n\n'
         + "\n\n".join(blocks)

skillopt_sleep/scheduler.py

@@ -0,0 +1,138 @@
+"""SkillOpt-Sleep — built-in nightly scheduler.
+
+Installs/removes a crontab entry that runs the sleep cycle automatically, so the
+user doesn't have to wire cron themselves. Idempotent: a managed block delimited
+by marker comments is added/replaced/removed in the user's crontab.
+
+Design choices:
+  * Off-:00 minute (3:17 local by default) so many users don't all hit the API
+    at the same instant.
+  * The entry runs `python -m skillopt_sleep run` for a specific project and
+    appends to <project>/.skillopt-sleep/cron.log.
+  * `schedule` is additive per project (keyed by project path); `unschedule`
+    removes the project's line (or the whole managed block with --all).
+
+cron is the portable mechanism on Linux/macOS. On systems without `crontab`,
+`schedule` prints the line and instructions instead of failing.
+"""
+from __future__ import annotations
+
+import os
+import shutil
+import subprocess
+import sys
+from typing import List, Optional, Tuple
+
+_BEGIN = "# >>> skillopt-sleep (managed) >>>"
+_END = "# <<< skillopt-sleep (managed) <<<"
+
+
+def _have_crontab() -> bool:
+    return shutil.which("crontab") is not None
+
+
+def _read_crontab() -> str:
+    try:
+        proc = subprocess.run(["crontab", "-l"], capture_output=True, text=True)
+        return proc.stdout if proc.returncode == 0 else ""
+    except Exception:
+        return ""
+
+
+def _write_crontab(content: str) -> bool:
+    try:
+        proc = subprocess.run(["crontab", "-"], input=content, text=True,
+                              capture_output=True)
+        return proc.returncode == 0
+    except Exception:
+        return False
+
+
+def _split_managed(crontab: str) -> Tuple[str, List[str]]:
+    """Return (text_outside_block, managed_lines_inside_block)."""
+    lines = crontab.splitlines()
+    outside: List[str] = []
+    managed: List[str] = []
+    in_block = False
+    for ln in lines:
+        if ln.strip() == _BEGIN:
+            in_block = True
+            continue
+        if ln.strip() == _END:
+            in_block = False
+            continue
+        (managed if in_block else outside).append(ln)
+    return "\n".join(outside).rstrip(), managed
+
+
+def _runner_cmd(project: str, backend: str, extra: str, python: str) -> str:
+    logdir = os.path.join(project, ".skillopt-sleep")
+    log = os.path.join(logdir, "cron.log")
+    # use absolute python + -m so cron's minimal env still works
+    cmd = (f'{python} -m skillopt_sleep run --project "{project}" '
+           f'--scope invoked --backend {backend} {extra}'.rstrip())
+    return f'mkdir -p "{logdir}"; cd "{_repo_root()}" && {cmd} >> "{log}" 2>&1'
+
+
+def _repo_root() -> str:
+    # the package lives at <repo>/skillopt_sleep/; repo root is its parent
+    return os.path.abspath(os.path.join(os.path.dirname(__file__), ".."))
+
+
+def _project_marker(project: str) -> str:
+    return f"# project={os.path.abspath(project)}"
+
+
+def schedule(project: str, *, backend: str = "mock", hour: int = 3, minute: int = 17,
+             extra: str = "", python: Optional[str] = None) -> Tuple[bool, str]:
+    """Install (or replace) the nightly entry for ``project``.
+
+    Returns (installed, message). If crontab is unavailable, installed=False and
+    the message contains the line to add manually.
+    """
+    project = os.path.abspath(project)
+    python = python or sys.executable or "python3"
+    cron_line = f"{minute} {hour} * * *  {_runner_cmd(project, backend, extra, python)}  {_project_marker(project)}"
+
+    if not _have_crontab():
+        return False, ("crontab not found on this system. Add this line to your "
+                       "scheduler manually:\n" + cron_line)
+
+    outside, managed = _split_managed(_read_crontab())
+    # drop any existing line for this project, then add the new one
+    marker = _project_marker(project)
+    managed = [ln for ln in managed if marker not in ln and ln.strip()]
+    managed.append(cron_line)
+
+    block = _BEGIN + "\n" + "\n".join(managed) + "\n" + _END
+    new_crontab = (outside + "\n\n" + block + "\n").lstrip("\n")
+    ok = _write_crontab(new_crontab)
+    if ok:
+        return True, (f"Scheduled nightly at {hour:02d}:{minute:02d} for {project} "
+                      f"(backend={backend}). Logs -> {project}/.skillopt-sleep/cron.log\n"
+                      f"Runs `skillopt_sleep run`; it only STAGES a proposal — adopt is still manual.")
+    return False, "Failed to write crontab. Line to add manually:\n" + cron_line
+
+
+def unschedule(project: Optional[str] = None, *, all_projects: bool = False) -> Tuple[bool, str]:
+    """Remove the entry for ``project`` (or the whole managed block with all_projects)."""
+    if not _have_crontab():
+        return False, "crontab not found; nothing to remove."
+    outside, managed = _split_managed(_read_crontab())
+    if all_projects:
+        managed = []
+    elif project:
+        marker = _project_marker(project)
+        managed = [ln for ln in managed if marker not in ln and ln.strip()]
+    if managed:
+        block = _BEGIN + "\n" + "\n".join(managed) + "\n" + _END
+        new_crontab = (outside + "\n\n" + block + "\n").lstrip("\n")
+    else:
+        new_crontab = outside.rstrip() + "\n"
+    ok = _write_crontab(new_crontab)
+    return ok, ("Removed." if ok else "Failed to update crontab.")
+
+
+def list_scheduled() -> List[str]:
+    _outside, managed = _split_managed(_read_crontab())
+    return [ln for ln in managed if ln.strip()]

skillopt_sleep/staging.py

@@ -9,12 +9,68 @@ from __future__ import annotations
 
 import json
 import os
+import re
 import shutil
 import time
-from typing import List, Optional
+from typing import Any, List, Optional
 
 from skillopt_sleep.types import SleepReport
 
+# Secret patterns scrubbed from any free-text we persist to the staging dir
+# (diagnostics, reports). Kept here so every on-disk artifact shares one
+# redaction pass; harvest_codex reuses these for session text too.
+_SECRET_PATTERNS: tuple[tuple[re.Pattern[str], str], ...] = (
+    (re.compile(r"sk-[A-Za-z0-9_-]{10,}"), "[REDACTED_OPENAI_KEY]"),
+    # Distinctive vendor token prefixes (low false-positive: these prefixes do
+    # not occur in normal diagnostic prose).
+    (re.compile(r"\bAKIA[0-9A-Z]{16}\b"), "[REDACTED_AWS_KEY]"),
+    (re.compile(r"\bgh[pousr]_[A-Za-z0-9]{20,}\b"), "[REDACTED_GITHUB_TOKEN]"),
+    (re.compile(r"\bxox[baprs]-[A-Za-z0-9-]{10,}\b"), "[REDACTED_SLACK_TOKEN]"),
+    (re.compile(r"\bAIza[0-9A-Za-z_-]{20,}\b"), "[REDACTED_GOOGLE_KEY]"),
+    # Bare JWT (three base64url segments) — e.g. a leaked bearer body without
+    # the "Authorization:" prefix.
+    (re.compile(r"\beyJ[A-Za-z0-9_-]{8,}\.[A-Za-z0-9_-]{8,}\.[A-Za-z0-9_-]{8,}\b"),
+     "[REDACTED_JWT]"),
+    (re.compile(r"(?i)(Authorization:\s*Bearer\s+)[^\s\"']+"), r"\1[REDACTED]"),
+    (re.compile(r"(?i)(Authorization:\s*Basic\s+)[^\s\"']+"), r"\1[REDACTED]"),
+    (
+        re.compile(r"(?i)\b(api[_-]?key|token|password|secret)\b(\s*[:=]\s*)[^\s\"']+"),
+        r"\1\2[REDACTED]",
+    ),
+    (
+        re.compile(r"(?i)\b(api[_-]?key|token|password|secret)\b(\s+)[^\s\"']+"),
+        r"\1\2[REDACTED]",
+    ),
+    (
+        re.compile(
+            r"-----BEGIN [A-Z ]*PRIVATE KEY-----.*?-----END [A-Z ]*PRIVATE KEY-----",
+            re.DOTALL,
+        ),
+        "[REDACTED_PRIVATE_KEY]",
+    ),
+)
+
+
+def redact_secrets(value: Any) -> Any:
+    """Scrub secret-looking substrings (API keys, bearer tokens, private keys)
+    from a string, or recursively from the string leaves of a list/dict.
+
+    Used before writing backend stderr / optimizer replies / task responses to
+    on-disk diagnostics: those are surfaced for debugging, but the underlying
+    text (e.g. a codex 401 stderr dump) can carry credentials. Non-string
+    scalars pass through unchanged.
+    """
+    if isinstance(value, str):
+        out = value
+        for pattern, replacement in _SECRET_PATTERNS:
+            out = pattern.sub(replacement, out)
+        return out
+    if isinstance(value, list):
+        return [redact_secrets(v) for v in value]
+    if isinstance(value, dict):
+        return {k: redact_secrets(v) for k, v in value.items()}
+    return value
+
 
 def _ts_dir() -> str:
     return time.strftime("%Y%m%d-%H%M%S", time.localtime())