fix(sleep): reset ClaudeCliBackend.last_call_error at each call start

Follow-up to #126. ClaudeCliBackend recorded last_call_error on failure but never reset it, so a failure followed by a successful call still reported the stale error in persisted diagnostics — making a healthy run look failed. Both _call() and attempt_with_tools() now clear it at the start, matching CodexCliBackend. Note: the "TimeoutExpired leaks the prompt into logs" concern raised in review does NOT apply on current main — the prompt is passed via stdin (input=), not argv (fixed by #98), so TimeoutExpired's str() only contains the argv (no prompt). Verified empirically. Adds a regression test asserting a prior failure is cleared on the next success. Co-Authored-By: Claude <noreply@anthropic.com>
fix(gate): resolve issue #100 (#102 )
2026-07-12 16:33:18 +00:00 · 2026-07-13 01:27:13 +09:00 · 2026-07-13 01:27:05 +09:00 · 2026-07-13 01:25:55 +09:00 · 2026-07-13 01:24:40 +09:00 · 2026-07-13 01:23:53 +09:00
142 changed files with 18168 additions and 1113 deletions
@@ -22,6 +22,11 @@ data/*
 outputs/
 logs/
 external/
 # SkillOpt-Sleep runtime state (staging proposals, config, diagnostics, cron logs)
 .skillopt-sleep/
 # SkillOpt-Sleep handoff-backend round data (prompts/answers derived from transcripts)
 .skillopt-sleep-handoff/
 .skillopt-sleep-handoff.night*.done/
 /BabyVision/
 /MMRB/
@@ -54,3 +59,8 @@ docs/render_ablation_paper_tables.py
 docs/让*
 .gradio/
 .venv
 # Local experiment launchers — contain machine-specific endpoints/identities, never commit
 tests/run_*.sh
 tests/launch_*.py
 *.launch.log
@@ -0,0 +1,113 @@
 # Changelog
 All notable changes to SkillOpt are documented here. This project adheres to
 [Semantic Versioning](https://semver.org/) and the format is based on
 [Keep a Changelog](https://keepachangelog.com/).
 ## [Unreleased]
 ### Added
 - **Handoff backend** (`--backend handoff`) for SkillOpt-Sleep — runs the
  sleep cycle with no model subprocess or API key: the engine writes each
  pending model call to `PROMPTS.md`/`pending.json` (exit code 3) and the
  user's own agent session answers into `answers/<id>.md`; re-running the
  same command resumes statelessly from the answers (typically 3–6 rounds
  per night). Mined tasks are pinned per night so answering sessions cannot
  shift the task set. Ships a `/skillopt-sleep-handoff` Claude Code command
  that automates the loop with fresh-context subagents to protect the
  held-out gate.
 ## [0.2.0] — 2026-07-02
 The headline of this release is **SkillOpt-Sleep**: a nightly offline
 self-evolution engine that harvests a coding agent's real session
 transcripts, mines recurring tasks, replays them offline, and consolidates
 short-term experience into long-term memory and skills — all behind the same
 held-out validation gate that keeps SkillOpt training honest. It ships as a
 decoupled top-level package (`skillopt_sleep/`, zero dependency on the
 research code) and as the new `skillopt-sleep` CLI.
 ### Added
 - **SkillOpt-Sleep engine** — nightly offline self-evolution cycle
  (harvest → mine → replay → consolidate) behind a validation gate, exposed
  as the `skillopt-sleep` console script and `python -m skillopt_sleep`.
  - Multi-objective reward (accuracy / tokens / latency) with user preferences.
  - Multi-rollout contrastive reflection under a token/time budget.
  - Experience replay + controllable dream rollouts (opt-in).
  - Slow-update long-term memory field (runs even with the gate off).
  - 3-way train/val/test split with `gate_mode on|off`.
  - Verifier-discipline validation gate, with a stress-test suite
    (thanks @Tanmay9223, #87).
 - **Cross-tool backends & plugin shells** for Claude Code, Codex, Copilot,
  Devin, and OpenClaw:
  - Codex Desktop transcript harvesting, skill-first Codex integration, and a
    reviewed task-file flow (thanks @Kirchberg, #48, #49, #60).
  - GitHub Copilot backend (`CopilotCliBackend`) + research-engine MCP plugin
    (thanks @Dongbumlee, #50).
  - Devin plugin: MCP server + ATIF-v1.7 harvest (thanks @xerxes-y, #88).
  - OpenClaw shell for SkillOpt-Sleep (thanks @Elzlxx, #59).
 - **SearchQA** split materialization helper and fail-fast on systemic rollout
  failures, with a `searchqa` install extra (thanks @summerview1997,
  #63, #64, #65).
 - WebUI environment loading and backend preflight (thanks @summerview1997, #63).
 ### Changed
 - Decoupled the Sleep engine into a standalone top-level `skillopt_sleep/`
  package with zero dependency on the research code.
 - Made `EnvAdapter.reflect` a shared default so reflect kwargs are no longer
  dropped (thanks @imshunsuke, #44).
 - English-only pass across the engine, plugins, and docs.
 ### Fixed
 - Windows robustness for the Claude/Codex backends, plus a hardened JSON
  fallback path (thanks @Yif-Yang, #79).
 - Reject prose pseudo-JSON wrapped in single quotes/backticks (#82).
 - Surface Codex auth/model/version failures instead of silently scoring 0
  (thanks @dmmdea, #92).
 - Redact secrets before persisting cycle diagnostics.
 - Configure the `qwen_chat`/`minimax` backends so local LLM endpoints work
  (thanks @imrehg, #85).
 - Forward the Qwen target timeout and gate `enable_thinking` for vLLM targets
  (thanks @mvanhorn, #40).
 - Make `--bare` conditional on `ANTHROPIC_API_KEY` (#68), add a
  `SKILLOPT_SLEEP_PYTHON` override with a lookback-hours first-run fallback
  (#74), and fix ALFWorld gamefile paths relative to `ALFWORLD_DATA`.
 ### Packaging
 - Bump `skillopt`, `skillopt.__version__`, and `skillopt_sleep.__version__`
  to `0.2.0`.
 - Restore `skillopt_webui` to the built wheel (it was dropped when the
  `packages.find` include list was made explicit).
 - Add the `searchqa` extra and include `json_repair` in the `claude`, `qwen`,
  and `all` extras.
 ### Acknowledgements 🙏
 v0.2.0 landed thanks to our community contributors — thank you!
 - @Kirchberg — Codex Desktop harvesting, skill-first Codex integration,
  reviewed task-file flow (#48, #49, #60)
 - @Dongbumlee — GitHub Copilot backend + research-engine MCP plugin (#50)
 - @summerview1997 — SearchQA materialization, rollout fail-fast, WebUI
  preflight (#63, #64, #65)
 - @xerxes-y — Devin plugin: MCP server + ATIF-v1.7 harvest (#88)
 - @Elzlxx — OpenClaw shell for SkillOpt-Sleep (#59)
 - @imshunsuke — shared `EnvAdapter.reflect` default + docs fixes (#43, #44)
 - @mvanhorn — Qwen timeout forwarding + `enable_thinking` gating (#40)
 - @dmmdea — surface Codex auth/model/version failures (#92)
 - @Tanmay9223 — verifier-discipline stress test (#87)
 - @imrehg — `configure_qwen_chat` for local LLM endpoints (#85)
 - @samuelgoofus-boop — community contributions
 Special thanks to @Yif-Yang for driving the SkillOpt-Sleep engine.
 **Full changelog:** https://github.com/microsoft/SkillOpt/compare/v0.1.0...v0.2.0
 ## [0.1.0] — 2026-06-02
 Initial public release: the full training loop (rollout → reflect →
 aggregate → select → update → evaluate), multi-backend support
 (OpenAI / Azure / Claude / Qwen / MiniMax), six built-in benchmarks, and the
 WebUI dashboard.
 [0.2.0]: https://github.com/microsoft/SkillOpt/releases/tag/v0.2.0
 [0.1.0]: https://github.com/microsoft/SkillOpt/releases/tag/v0.1.0
@@ -2,7 +2,22 @@
 *Train agent skills like you train neural networks — with epochs, (mini-)batchsize, learning rates, and validation gates — but without touching model weights.*
-[![Project Page](https://img.shields.io/badge/Project%20Page-SkillOpt-8dbb3c)](https://microsoft.github.io/SkillOpt/) [![Paper](https://img.shields.io/badge/Paper-arXiv-b31b1b)](https://arxiv.org/abs/2605.23904) [![Project Video](https://img.shields.io/badge/Project%20Video-Watch%20Demo-ff0000)](https://youtu.be/JUBMDTCiM0M) [![Python 3.10+](https://img.shields.io/badge/Python-3.10%2B-blue.svg)](https://www.python.org/) [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE)
+[![Project Page](https://img.shields.io/badge/Project%20Page-SkillOpt-8dbb3c)](https://microsoft.github.io/SkillOpt/) [![Paper](https://img.shields.io/badge/Paper-arXiv-b31b1b)](https://arxiv.org/abs/2605.23904) [![Project Video](https://img.shields.io/badge/Project%20Video-Watch%20Demo-ff0000)](https://youtu.be/JUBMDTCiM0M) [![PyPI](https://img.shields.io/badge/PyPI-skillopt-green.svg)](https://pypi.org/project/skillopt/) [![Python 3.10+](https://img.shields.io/badge/Python-3.10%2B-blue.svg)](https://www.python.org/) [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE)
 <p align="center">
  <a href="https://trendshift.io/repositories/38498?utm_source=trendshift-badge&utm_medium=badge&utm_campaign=badge-trendshift-38498" target="_blank" rel="noopener noreferrer"><img src="https://trendshift.io/api/badge/trendshift/repositories/38498/daily?language=Python" alt="microsoft%2FSkillOpt | Trendshift" width="250" height="55"/></a>
  <a href="https://trendshift.io/repositories/38498?utm_source=trendshift-badge&utm_medium=badge&utm_campaign=badge-trendshift-38498" target="_blank" rel="noopener noreferrer"><img src="https://trendshift.io/api/badge/trendshift/repositories/38498/weekly?language=Python" alt="microsoft%2FSkillOpt | Trendshift" width="250" height="55"/></a>
 </p>
 > 📖 **For installation, data preparation, training/eval commands, the full configuration reference, and framework internals, see the [Documentation & Reproduction Guide](https://microsoft.github.io/SkillOpt/docs/guideline.html)** (rendered on GitHub Pages).
 ---
 ## News 🔥🔥🔥
 - **[2026-07-02]** 🚀 **SkillOpt [v0.2.0](https://github.com/microsoft/SkillOpt/releases/tag/v0.2.0) is out on [PyPI](https://pypi.org/project/skillopt/)!** Headline feature: **SkillOpt-Sleep**, a nightly offline self-evolution engine (harvest → mine → replay → consolidate, all behind a held-out validation gate) with multi-objective reward, experience replay + dream rollouts, and long-term memory — now shipped as the `skillopt-sleep` CLI. This release also adds cross-tool backends and plugin shells for **Claude, Codex, Copilot, Devin, and OpenClaw**, SearchQA split materialization, Windows robustness, and hardened JSON parsing. See the [release notes](https://github.com/microsoft/SkillOpt/releases/tag/v0.2.0) for the full changelog and contributor acknowledgements.
 - **[2026-06-15]** 😴 **SkillOpt-Sleep (preview)** — a nightly offline self-evolution companion for local coding agents (Claude Code / Codex / Copilot): review past sessions, replay recurring tasks, and consolidate validated skills behind a held-out gate. See **[`docs/sleep/README.md`](docs/sleep/README.md)** for what it is, how to use it, and results.
 - **[2026-06-03]** 🎉 **[gbrain](https://github.com/garrytan/gbrain), [gbrain-evals](https://github.com/garrytan/gbrain-evals/blob/main/docs/benchmarks/2026-06-03-skillopt.md), and [darwin-skill](https://github.com/alchaincyf/darwin-skill) have all integrated SkillOpt.**
 - **[2026-06-02]** 🎉 **SkillOpt [v0.1.0](https://github.com/microsoft/SkillOpt/releases/tag/v0.1.0) is now available on [PyPI](https://pypi.org/project/skillopt/)!** Install with `pip install skillopt`. This initial release includes the full training loop (rollout → reflect → aggregate → select → update → evaluate), multi-backend support (OpenAI / Azure / Claude / Qwen / MiniMax), six built-in benchmarks, and WebUI dashboard.
 ---
@@ -44,306 +59,6 @@ https://github.com/user-attachments/assets/eb12d3bc-371c-467f-904d-91b61f339ed7
 ---
 ## Install
 ### Requirements
 - Python 3.10+
 ```bash
 git clone https://github.com/microsoft/SkillOpt.git
 cd SkillOpt
 pip install -e .
 # For the ALFWorld benchmark (optional):
 pip install -e ".[alfworld]"
 alfworld-download
 ```
 ### Configure API Credentials
 ```bash
 cp .env.example .env
 # Edit .env with your API credentials, then:
 source .env
 ```
 #### Azure OpenAI *(recommended)*
 ```bash
 export AZURE_OPENAI_ENDPOINT="https://your-resource.openai.azure.com/"
 # Option 1: API key auth
 export AZURE_OPENAI_API_KEY="your-key"
 # Option 2: Azure CLI auth (no API key needed)
 export AZURE_OPENAI_AUTH_MODE="azure_cli"
 ```
 > **Note:** `AZURE_OPENAI_ENDPOINT` is required for all three modes (`api_key`, `azure_cli`, `openai_compatible`). Without it, all LLM calls will fail.
 #### OpenAI-compatible endpoints
 ```bash
 export AZURE_OPENAI_ENDPOINT="https://api.openai.com/v1"
 export AZURE_OPENAI_API_KEY="sk-..."
 export AZURE_OPENAI_AUTH_MODE="openai_compatible"
 ```
 This routes all calls through the plain OpenAI Python client (no Azure auth, no `api-version` header).
 > **Note:** SkillOpt reuses the `AZURE_OPENAI_*` env var names even in this mode — there is no separate `OPENAI_API_KEY` knob.
 #### Anthropic Claude
 ```bash
 export ANTHROPIC_API_KEY="sk-ant-..."
 ```
 #### Qwen *(local vLLM)*
 ```bash
 export QWEN_CHAT_BASE_URL="http://localhost:8000/v1"
 export QWEN_CHAT_MODEL="Qwen/Qwen3.5-4B"
 ```
 `qwen_chat` can also be used as the optimizer backend. When optimizer and
 target should point to different local vLLM services, use the role-specific
 settings:
 ```bash
 python scripts/train.py \
    --config configs/searchqa/default.yaml \
    --optimizer_backend qwen_chat \
    --target_backend qwen_chat \
    --optimizer_model Qwen/Qwen3.5-4B \
    --target_model Qwen/Qwen3.5-4B \
    --optimizer_qwen_chat_base_url http://localhost:8001/v1 \
    --target_qwen_chat_base_url http://localhost:8000/v1
 ```
 #### MiniMax
 ```bash
 export MINIMAX_BASE_URL="https://api.minimax.io/v1"
 export MINIMAX_API_KEY="..."
 export MINIMAX_MODEL="MiniMax-M2.7"
 ```
 ---
 ## Quick Start
 ### Training
 ```bash
 # Minimal example — train on SearchQA:
 python scripts/train.py \
    --config configs/searchqa/default.yaml \
    --split_dir /path/to/your/searchqa_split \
    --azure_openai_endpoint https://your-resource.openai.azure.com/ \
    --optimizer_model gpt-5.5 \
    --target_model gpt-5.5
 # Train on LiveMathematicianBench:
 python scripts/train.py \
    --config configs/livemathematicianbench/default.yaml \
    --split_dir /path/to/your/livemath_split \
    --azure_openai_endpoint https://your-resource.openai.azure.com/ \
    --optimizer_model gpt-5.5 \
    --target_model gpt-5.5
 # Train on ALFWorld:
 python scripts/train.py \
    --config configs/alfworld/default.yaml \
    --split_dir data/alfworld_path_split \
    --azure_openai_endpoint https://your-resource.openai.azure.com/ \
    --optimizer_model gpt-5.5 \
    --target_model gpt-5.5
 ```
 Key CLI arguments:
 | Argument | Description | Example |
 |---|---|---|
 | `--config` | Benchmark config YAML | `configs/searchqa/default.yaml` |
 | `--split_dir` | Path to data split directory | `/path/to/split` |
 | `--azure_openai_endpoint` | Azure OpenAI endpoint URL | `https://your-resource.openai.azure.com/` |
 | `--optimizer_model` | Optimizer model deployment name | `gpt-5.5` |
 | `--target_model` | Target model deployment name | `gpt-5.5` |
 | `--num_epochs` | Number of training epochs | `4` |
 | `--batch_size` | Batch size per step | `40` |
 | `--workers` | Parallel rollout workers | `8` |
 | `--out_root` | Output directory | `outputs/my_run` |
 ### Eval Only
 Evaluate a trained skill on specific data splits without training:
 ```bash
 # Evaluate the packaged GPT-5.5 SearchQA skill on the test split:
 python scripts/eval_only.py \
  --config configs/searchqa/default.yaml \
  --skill ckpt/searchqa/gpt5.5_skill.md \
  --split valid_unseen \
  --split_dir /path/to/searchqa_split \
  --azure_openai_endpoint https://your-resource.openai.azure.com/
 # Evaluate on all splits (train + val + test):
 python scripts/eval_only.py \
  --config configs/searchqa/default.yaml \
  --skill ckpt/searchqa/gpt5.5_skill.md \
  --split all \
  --split_dir /path/to/searchqa_split \
  --azure_openai_endpoint https://your-resource.openai.azure.com/
 ```
 To evaluate a skill produced by your own training run, replace `--skill` with that run's best-skill path, for example `outputs/my_run/best_skill.md`.
 | Split | Description |
 |---|---|
 | `valid_unseen` | Test set |
 | `valid_seen` | Validation set |
 | `train` | Training set |
 | `all` | All splits combined (default) |
 ### Output Structure
 Each training run writes to a structured output directory:
 ```
 outputs/<run_name>/
 ├── config.json              # Flattened runtime config
 ├── history.json             # Per-step training history
 ├── runtime_state.json       # Resume checkpoint
 ├── best_skill.md            # Best validated skill document
 ├── skills/skill_vXXXX.md   # Skill snapshot per step
 ├── steps/step_XXXX/        # Per-step artifacts (patches, evals)
 ├── slow_update/epoch_XX/   # Slow update logs
 └── meta_skill/epoch_XX/    # Meta skill logs
 ```
 Re-running the same command auto-resumes from the last completed step.
 ### Pretrained Skill Artifacts
 We provide a subset of the paper's main Table 1 GPT-5.5 optimized skills in
 [`ckpt/`](ckpt/) as reference artifacts. Use them with `scripts/eval_only.py`
 to evaluate the provided skills on a matching data split without re-running
 training. See [`ckpt/README.md`](ckpt/README.md) for the full per-benchmark
 command. This is the first artifact batch; we plan to continue uploading
 the remaining optimized skills and benchmark split manifests as they are
 cleaned and verified.
 ---
 ## Data Preparation
 ### Directory layout
 SkillOpt expects data in a **split directory** with `train/`, `val/`, `test/` subdirectories, each containing a JSON file (e.g., `items.json`):
 ```
 data/my_split/
 ├── train/items.json
 ├── val/items.json
 └── test/items.json
 ```
 Each JSON file is an array of task items. The required fields depend on the benchmark. For example, SearchQA items look like:
 ```json
 [
  {
    "id": "unique_item_id",
    "question": "Who wrote the novel ...",
    "context": "[DOC] relevant passage text ...",
    "answers": ["expected answer"]
  }
 ]
 ```
 See `skillopt/envs/<benchmark>/dataloader.py` for the exact format each benchmark expects.
 > **Note:** Most benchmark datasets are not included in this repository. Prepare your own data following the format above. The exact SearchQA split used in the paper is provided at [`data/searchqa_id_split/`](data/searchqa_id_split) (400 train / 200 val / 1400 test). We are preparing the remaining benchmark split manifests for upload.
 ### Supported Benchmarks
 | Benchmark | Type | Config |
 |---|---|---|
 | SearchQA | QA | `configs/searchqa/default.yaml` |
 | ALFWorld | Embodied agent | `configs/alfworld/default.yaml` |
 | DocVQA | Document QA | `configs/docvqa/default.yaml` |
 | LiveMathematicianBench | Math | `configs/livemathematicianbench/default.yaml` |
 | SpreadsheetBench | Code generation | `configs/spreadsheetbench/default.yaml` |
 | OfficeQA | Tool-augmented QA | `configs/officeqa/default.yaml` |
 ---
 ## Configuration
 ### Default settings and paper-reproduction knobs
 `configs/_base_/default.yaml` is the single source of truth for SkillOpt's
 runtime knobs. Out of the box, every included benchmark config inherits
 from it and keeps the paper protocol visible: 4 epochs, rollout batch 40,
 reflection minibatch 8, textual learning rate 4 with cosine decay, strict
 hard validation gating, and slow-update + meta-skill enabled. One detail to
 watch is slow-update acceptance: the current `main` default is the newer
 post-submission force-accept mode, while the paper protocol and the
 paper-aligned skills under `ckpt/` use the gated semantics described in
 paper Section 3.6.
 ### Slow-update acceptance mode
 The epoch-boundary slow / meta update can be applied two ways, controlled
 by `optimizer.slow_update_gate_with_selection`:
 ```yaml
 optimizer:
  slow_update_gate_with_selection: false   # current main default
 ```
 - **`false`** *(current `main` default)*: force-accept. The
  slow-update guidance is injected into both `current_skill` and
  `best_skill` unconditionally at the epoch boundary. This is the newer
  post-submission behavior on `main`.
 - **`true`** *(paper / ckpt-skill reproduction)*: gated, matching paper
  Section 3.6 verbatim. The slow-update candidate is evaluated on the
  selection split and accepted only if it passes the same validation gate
  as a step-level edit. Use this setting when re-running optimization to
  match the paper protocol and the provenance of the provided `ckpt/` skills.
 The trainer prints which mode is active at startup
 (`[slow update] acceptance=...`). See issue #22 for the discussion that
 led to the flag.
 ### Gate metric (`hard` / `soft` / `mixed`)
 The validation gate compares candidate vs. current skills on the selection
 split using `gate_metric`:
 - **`hard`** *(default, paper)*: exact-match accuracy, strictly greater
  than the current score is required.
 - **`soft`**: per-item soft / partial-credit score. Useful when the
  selection split is small (e.g. ≤10 items) and the reward is continuous,
  where the discrete hard gate often rejects every candidate.
 - **`mixed`**: weighted average, `(1 - w) * hard + w * soft`, with `w`
  set by `gate_mixed_weight` (default `0.5`).
 Default is `hard`. Use the optional feature config below to switch.
 ### Optional feature configs
 These are **not** default SkillOpt settings — they are optional feature configs
 contributed by users for specific scenarios. The paper-reported numbers
 were obtained with the default settings, not these.
 - **[`configs/features/soft_gate.yaml`](configs/features/soft_gate.yaml)**
  *(PR #25, contributed by [@lvbaocheng](https://github.com/lvbaocheng))* —
  switches `gate_metric` to `soft` (or `mixed`). See the comment at the
  top of the file for when to use and when not to.
 ---
 ## Extensibility & WebUI
 ### Adding a new backend
@@ -383,13 +98,10 @@ python -m skillopt_webui.app
 ## Citation
 ```bibtex
-@misc{yang2026skilloptexecutivestrategyselfevolving,
+@article{yang2026skillopt,
-      title={SkillOpt: Executive Strategy for Self-Evolving Agent Skills}, 
+  title={Skillopt: Executive strategy for self-evolving agent skills},
-      author={Yifan Yang and Ziyang Gong and Weiquan Huang and Qihao Yang and Ziwei Zhou and Zisu Huang and Yan Li and Xuemei Gao and Qi Dai and Bei Liu and Kai Qiu and Yuqing Yang and Dongdong Chen and Xue Yang and Chong Luo},
+  author={Yang, Yifan and Gong, Ziyang and Huang, Weiquan and Yang, Qihao and Zhou, Ziwei and Huang, Zisu and Li, Yan and Gao, Xuemei and Dai, Qi and Liu, Bei and others},
-      year={2026},
+  journal={arXiv preprint arXiv:2605.23904},
-      eprint={2605.23904},
+  year={2026}
      archivePrefix={arXiv},
      primaryClass={cs.AI},
      url={https://arxiv.org/abs/2605.23904}
 }
 ```
@@ -81,6 +81,9 @@ optimizer:
  slow_update_gate_with_selection: false
  longitudinal_pair_policy: mixed  # mixed / changed / unchanged
  use_meta_skill: true
  use_skill_aware_reflection: false  # EmbodiSkill: split failures into SKILL_DEFECT (edit body) vs EXECUTION_LAPSE (protected appendix)
  skill_aware_appendix_source: both  # both = success+failure emit appendix notes; failure_only = only EXECUTION_LAPSE (paper-faithful)
  skill_aware_consolidate_threshold: 0  # 0 = off; >0 = LLM-consolidate the appendix when its note count exceeds N
 evaluation:
  use_gate: true
@@ -138,6 +138,20 @@ ALFWorld:
 `searchqa_id_split/` is an ID-only manifest. Each released `id` exactly matches
 the `key` field in `lucadiliello/searchqa`.
 To materialize the runnable SearchQA split used by
 `configs/searchqa/default.yaml`, install the optional dependency and run:
 ```bash
 python -m pip install 'skillopt[searchqa]'
 python scripts/materialize_searchqa.py
 ```
 This writes full examples to:
 ```text
 data/searchqa_split
 ```
 Materialized examples must include the fields consumed by the SearchQA
 environment, including:
@@ -61,6 +61,36 @@ optimizer:
  use_meta_skill: true           # Cross-epoch strategy memory
 ```
 ### Skill-Aware Reflection (optional, off by default)
 EmbodiSkill-style failure routing: the failure analyst classifies each
 failure pattern as **SKILL_DEFECT** (the rule is wrong or missing → normal
 gated body edit) or **EXECUTION_LAPSE** (a valid rule exists but was not
 followed → a short reminder appended to a protected appendix region inside
 the skill that step-level edits can never modify).
 ```yaml
 optimizer:
  use_skill_aware_reflection: false    # Master switch (default off = baseline-identical)
  skill_aware_appendix_source: both    # both | failure_only (paper-faithful S_app)
  skill_aware_consolidate_threshold: 0 # >0: LLM-compact the appendix past N notes (experimental)
 ```
 Notes:
 - The switch is resolved process-wide from the config
  (`configure_skill_aware_reflection`), so it applies to every benchmark
  with no per-adapter wiring.
 - `failure_only` restricts appendix notes to the failure analyst, matching
  the original S_app formulation; `both` additionally lets the success
  analyst re-emphasize existing rules.
 - Appendix notes bypass the validation gate by design and accumulate with
  order-preserving dedup; lapse-only steps (no body edits) still flush
  their notes.
 - Not supported together with `skill_update_mode=rewrite_from_suggestions`
  or the full-rewrite modes: whole-document rewrites can drop the appendix
  region.
 ### Evaluation
 ```yaml
@@ -1,181 +1,371 @@
 # Add a New Benchmark
-Extend SkillOpt with your own benchmark in ~100 lines of code.
+Extend SkillOpt with your own benchmark in ~200 lines of code. We will use
 a tiny worked example, `docfaithful`, that scores a target model on
 how faithfully it answers questions grounded in a small reference doc.
-## Overview
+> **Working reference.** The easiest way to copy-cargo-cult a new env is
 > to read [`skillopt/envs/officeqa/`](https://github.com/microsoft/SkillOpt/tree/main/skillopt/envs/officeqa).
 > Everything below is the same shape, simplified.
-To add a benchmark, you need:
+## What you need to build
-1. **Data Loader** — Loads and splits your dataset
+To add a benchmark you implement four things:
 2. **Environment Adapter** — Executes tasks and returns scores
 3. **Config** — YAML configuration file
-## Step 1: Create the Benchmark Package
+1. **A `SplitDataLoader` subclass** — knows how to load train / val / test
   item dicts from disk.
 2. **A rollout helper** — runs the target model on a batch of items
   under the current skill and scores each prediction.
 3. **An `EnvAdapter` subclass** — wires the loader + rollout helper into
   SkillOpt's lifecycle (`build_*_env`, `rollout`, `reflect`,
   `get_task_types`).
 4. **A YAML config** — references your env name plus the standard
   train / optimizer / gradient knobs.
 Then one line in `scripts/train.py`'s `_register_builtins()` makes it
 discoverable.
 ---
 ## Step 1 — Create the package
 ```bash
-mkdir -p skillopt/envs/my_benchmark
+mkdir -p skillopt/envs/docfaithful
-touch skillopt/envs/my_benchmark/__init__.py
+touch skillopt/envs/docfaithful/__init__.py
 ```
-## Step 2: Implement the Data Loader
+## Step 2 — Implement the data loader
-Create `skillopt/envs/my_benchmark/loader.py`:
+`skillopt/envs/docfaithful/dataloader.py`:
 ```python
-from skillopt.data.base import DataLoader, DataItem
+from __future__ import annotations
-class MyBenchmarkDataLoader(DataLoader):
+import json
-    """Load and split your benchmark data."""
+from pathlib import Path
-    
+
-    def __init__(self, data_dir: str, **kwargs):
+from skillopt.datasets.base import SplitDataLoader
-        super().__init__(**kwargs)
+
-        self.data_dir = data_dir
+
-    
+def _normalize(raw: dict) -> dict:
-    def setup(self, cfg: dict):
+    """Make sure every item has an ``id``. Other keys are env-specific."""
-        """Initialize splits based on config."""
+    return {
-        self.split_mode = cfg.get('split_mode', 'ratio')
+        "id": str(raw["uid"]),
-        # Load your data here
+        "question": raw["question"],
-        self.items = self._load_items()
+        "ground_truth": raw["answer"],
-        self._create_splits(cfg)
+        "reference_text": raw.get("reference", ""),
-    
+        "task_type": raw.get("category", "docfaithful"),
-    def _load_items(self) -> list[DataItem]:
+    }
-        """Load raw data into DataItem objects."""
+
-        items = []
+
-        # TODO: Load your data
+class DocFaithfulDataLoader(SplitDataLoader):
-        for entry in your_data:
+    """Load DocFaithful items from JSON files inside each split dir."""
-            items.append(DataItem(
+
-                id=entry['id'],
+    def load_split_items(self, split_path: str) -> list[dict]:
-                input=entry['question'],
+        # split_path is e.g. data/docfaithful_split/train/
-                ground_truth=entry['answer'],
+        json_files = sorted(Path(split_path).glob("*.json"))
-                metadata=entry.get('metadata', {})
+        if not json_files:
-            ))
+            raise FileNotFoundError(f"No .json file found in {split_path}")
-        return items
+        with json_files[0].open(encoding="utf-8") as f:
-    
+            raw = json.load(f)
-    def get_split_items(self, split: str) -> list[DataItem]:
+        return [_normalize(item) for item in raw]
        """Return items for a given split (train/valid/test)."""
        return self.splits[split]
 ```
-## Step 3: Implement the Environment Adapter
+Only `load_split_items()` is mandatory. If you also want to support
 `split_mode="ratio"` (auto-split a single raw file into train/val/test),
 override `load_raw_items(data_path)` as well — see
 `skillopt/datasets/base.py` docstrings.
-Create `skillopt/envs/my_benchmark/env.py`:
+## Step 3 — Write the rollout helper
 `skillopt/envs/docfaithful/rollout.py`:
 ```python
-from skillopt.envs.base import EnvAdapter, TaskResult
+from __future__ import annotations
-class MyBenchmarkEnv(EnvAdapter):
+import json
-    """Execute tasks and evaluate results."""
+import os
-    
+from pathlib import Path
-    def __init__(self, cfg: dict):
+
-        super().__init__(cfg)
+from skillopt.model import chat_target
-    
+
-    async def execute(self, item: DataItem, skill: str, model) -> TaskResult:
+
-        """
+def _score(prediction: str, ground_truth: str) -> tuple[int, float]:
-        Execute a single task.
+    """Trivial exact-match scorer. Replace with F1 / ROUGE / LLM-judge."""
-        
+    p = (prediction or "").strip().lower()
-        Args:
+    g = (ground_truth or "").strip().lower()
-            item: The data item to process
+    hard = int(p == g and bool(g))
-            skill: Current skill document content
+    soft = 1.0 if hard else 0.0
-            model: The target model instance
+    return hard, soft
-            
+
-        Returns:
+
-            TaskResult with prediction, score, and trajectory
+def _rollout_one(item: dict, skill_content: str,
-        """
+                 *, max_completion_tokens: int) -> dict:
-        # Build prompt with skill document
+    system = skill_content
-        prompt = self.build_prompt(item, skill)
+    user = (
-        
+        f"Question: {item['question']}\n\n"
-        # Get model response
+        f"Reference:\n{item.get('reference_text', '')}\n\n"
-        response = await model.generate(prompt)
+        "Answer:"
-        
+    )
-        # Extract prediction
+    prediction, _usage = chat_target(
-        prediction = self.parse_response(response)
+        system=system,
-        
+        user=user,
-        # Score against ground truth
+        max_completion_tokens=max_completion_tokens,
-        score = self.evaluate(prediction, item.ground_truth)
+    )
-        
+    hard, soft = _score(prediction, item.get("ground_truth", ""))
-        return TaskResult(
+    return {
-            item_id=item.id,
+        "id": str(item["id"]),
-            prediction=prediction,
+        "hard": hard,
-            score=score,
+        "soft": soft,
-            trajectory=[
+        "predicted_answer": prediction,
-                {"role": "system", "content": skill},
+        "question": item.get("question", ""),
-                {"role": "user", "content": item.input},
+        "reference_text": item.get("reference_text", ""),
-                {"role": "assistant", "content": response}
+        "task_type": item.get("task_type", "docfaithful"),
-            ]
+    }
 def run_batch(*, items: list[dict], skill_content: str, out_root: str,
              workers: int = 4, max_completion_tokens: int = 4096) -> list[dict]:
    """Run a batch of episodes sequentially or with a thread pool."""
    os.makedirs(out_root, exist_ok=True)
    # For brevity we go sequentially — swap in concurrent.futures.ThreadPoolExecutor
    # when network / model latency dominates.
    results = [
        _rollout_one(item, skill_content,
                     max_completion_tokens=max_completion_tokens)
        for item in items
    ]
    Path(out_root, "rollouts.json").write_text(
        json.dumps(results, ensure_ascii=False, indent=2)
    )
    return results
 ```
 Two design points worth flagging:
 - **Scoring lives here, not in `EnvAdapter`.** There is no `evaluate()`
  method on the ABC. Whatever signal you put in `hard` (0/1, or a float
  in [0, 1] for smoothed reward) and `soft` (float in [0, 1]) is what
  the optimizer reads.
 - **Use `skillopt.model.chat_target`**, not raw OpenAI/Claude calls.
  That routes through whichever **chat** target backend the user
  configured (`openai_chat` / `claude_chat` / `qwen_chat` /
  `minimax_chat`) without your adapter caring. Exec-style backends
  (`codex_exec`, `claude_code_exec`) need env-specific rollout code —
  see `skillopt/envs/swebench/` for an example.
 ## Step 4 — Implement the environment adapter
 `skillopt/envs/docfaithful/adapter.py`:
 ```python
 from __future__ import annotations
 from skillopt.datasets.base import BatchSpec
 from skillopt.envs.base import EnvAdapter
 from skillopt.envs.docfaithful.dataloader import DocFaithfulDataLoader
 from skillopt.envs.docfaithful.rollout import run_batch
 class DocFaithfulAdapter(EnvAdapter):
    """SkillOpt adapter for the DocFaithful benchmark."""
    def __init__(
        self,
        split_dir: str = "",
        data_path: str = "",
        split_mode: str = "split_dir",
        split_ratio: str = "2:1:7",
        split_seed: int = 42,
        split_output_dir: str = "",
        workers: int = 4,
        analyst_workers: int = 4,
        failure_only: bool = False,
        minibatch_size: int = 8,
        edit_budget: int = 4,
        seed: int = 42,
        limit: int = 0,
        max_completion_tokens: int = 4096,
    ) -> None:
        self.workers = workers
        self.analyst_workers = analyst_workers
        self.failure_only = failure_only
        self.minibatch_size = minibatch_size
        self.edit_budget = edit_budget
        self.max_completion_tokens = int(max_completion_tokens)
        self.dataloader = DocFaithfulDataLoader(
            split_dir=split_dir,
            data_path=data_path,
            split_mode=split_mode,
            split_ratio=split_ratio,
            split_seed=split_seed,
            split_output_dir=split_output_dir,
            seed=seed,
            limit=limit,
        )
-    
+
-    def evaluate(self, prediction: str, ground_truth: str) -> float:
+    # ── Lifecycle ───────────────────────────────────────────────────────
-        """
+
-        Score a prediction against ground truth.
+    def setup(self, cfg: dict) -> None:
-        
+        super().setup(cfg)
-        Returns:
+        self.dataloader.setup(cfg)
-            Float between 0.0 and 1.0
+
-        """
+    def get_dataloader(self):
-        # TODO: Implement your scoring logic
+        return self.dataloader
-        # Examples: exact match, F1, ANLS, etc.
+
-        return float(prediction.strip() == ground_truth.strip())
+    # ── Env construction ────────────────────────────────────────────────
-    
+
-    def build_prompt(self, item, skill: str) -> str:
+    def build_env_from_batch(self, batch: BatchSpec, **kwargs):
-        """Combine skill document with task input."""
+        # For dataset-backed envs the "manager" is just the items list.
-        return f"{skill}\n\n---\n\nQuestion: {item.input}"
+        return list(batch.payload or [])
-    
+
-    def parse_response(self, response: str) -> str:
+    def build_train_env(self, batch_size: int, seed: int, **kwargs):
-        """Extract the answer from model response."""
+        batch = self.dataloader.build_train_batch(
-        return response.strip()
+            batch_size=batch_size, seed=seed, **kwargs
        )
        return self.build_env_from_batch(batch, **kwargs)
    def build_eval_env(self, env_num: int, split: str, seed: int, **kwargs):
        batch = self.dataloader.build_eval_batch(
            env_num=env_num, split=split, seed=seed, **kwargs
        )
        return self.build_env_from_batch(batch, **kwargs)
    # ── The rollout method (reflect is inherited) ───────────────────────
    def rollout(self, env_manager, skill_content: str,
                out_dir: str, **kwargs) -> list[dict]:
        items: list[dict] = env_manager
        return run_batch(
            items=items,
            skill_content=skill_content,
            out_root=out_dir,
            workers=self.workers,
            max_completion_tokens=self.max_completion_tokens,
        )
    # reflect() is inherited from EnvAdapter — it delegates to
    # run_minibatch_reflect with your analyst_error_* / analyst_success_*
    # prompts. Override it only if you need custom reflection logic.
    def get_task_types(self) -> list[str]:
        seen: list[str] = []
        for item in (
            self.dataloader.train_items
            + self.dataloader.val_items
            + self.dataloader.test_items
        ):
            tt = str(item.get("task_type") or "docfaithful")
            if tt not in seen:
                seen.append(tt)
        return seen or ["docfaithful"]
 ```
-## Step 4: Register the Benchmark
+### What the rollout actually does
-Add to `skillopt/envs/__init__.py`:
+Look back at `run_batch` from Step 3 — it sends each `item["question"]`
 to the target model with `skill_content` as the system prompt, scores
 the answer against `item["ground_truth"]`, and returns a list of dicts:
 ```python
-from .my_benchmark.env import MyBenchmarkEnv
+[
-from .my_benchmark.loader import MyBenchmarkDataLoader
+    {"id": "ex_001", "hard": 1, "soft": 0.92,
-
+     "predicted_answer": "...", "question": "...",
-BENCHMARK_REGISTRY = {
+     "reference_text": item["reference_text"]},
-    # ... existing benchmarks ...
+    {"id": "ex_002", "hard": 0, "soft": 0.13, "fail_reason": "...", ...},
-    'my_benchmark': {
+    ...
-        'env': MyBenchmarkEnv,
+]
        'loader': MyBenchmarkDataLoader,
    },
 }
 ```
-## Step 5: Create Config
+The trainer only requires `id`, `hard`, `soft`. The rest is preserved on
 `RolloutResult.extras` (see `skillopt/types.py`) and is what your
 `reflect()` consumes via `run_minibatch_reflect`.
-Create `configs/my_benchmark/default.yaml`:
+## Step 5 — Register the adapter
 Edit [`scripts/train.py`](https://github.com/microsoft/SkillOpt/blob/main/scripts/train.py)
 and add to `_register_builtins()`:
 ```python
    try:
        from skillopt.envs.docfaithful.adapter import DocFaithfulAdapter
        _ENV_REGISTRY["docfaithful"] = DocFaithfulAdapter
    except ImportError:
        pass  # docfaithful deps not installed — skip
 ```
 There is **no `BENCHMARK_REGISTRY` dict in `skillopt/envs/__init__.py`** —
 the registry lives in `scripts/train.py` and is populated lazily so that
 optional deps don't break `--help`.
 ## Step 6 — Create the YAML config
 `configs/docfaithful/default.yaml`:
 ```yaml
-_base_: ['../_base_/default.yaml']
+_base_: ../_base_/default.yaml      # NOTE: string, not list
-env:
+model:
-  name: my_benchmark
+  reasoning_effort: medium
  data_path: data/my_benchmark
  split_mode: ratio
  split_ratio: "2:1:7"
 train:
  batch_size: 16
  accumulation: 1
  num_epochs: 4
-  batch_size: 40
+
 gradient:
  minibatch_size: 8
  merge_batch_size: 8
 optimizer:
  learning_rate: 4
  lr_scheduler: cosine
  use_slow_update: true
  use_meta_skill: true
-gradient:
+env:
-  analyst_workers: 16
+  name: docfaithful
  # Optional: a seed skill document. Create this file (or any markdown
  # file) yourself before the first run, or omit the key to let SkillOpt
  # start from an empty skill.
  skill_init: skillopt/envs/docfaithful/skills/initial.md
  split_mode: split_dir
  split_dir: data/docfaithful_split
  workers: 4
  max_completion_tokens: 4096
  limit: 0
 ```
-## Step 6: Run
+> ⚠️ `_base_` is currently parsed as a **string path**, not a list. Write
 > `_base_: ../_base_/default.yaml`, not `_base_: ['../_base_/default.yaml']`.
 > See [`skillopt/config.py`](https://github.com/microsoft/SkillOpt/blob/main/skillopt/config.py)
 > if you want to add list-form inheritance.
 ## Step 7 — Run
 ```bash
-python scripts/train.py --config configs/my_benchmark/default.yaml
+# If you set skill_init above, create the seed skill first:
 #   mkdir -p skillopt/envs/docfaithful/skills
 #   echo "# DocFaithful initial skill" > skillopt/envs/docfaithful/skills/initial.md
 python scripts/train.py --config configs/docfaithful/default.yaml
 ```
 If you get `ValueError: Unknown environment 'docfaithful'. Available: [...]`,
 you forgot Step 5.
 If you get `TypeError: Can't instantiate abstract class DocFaithfulAdapter`,
 you forgot to implement one of the four abstract methods on `EnvAdapter`:
 `build_train_env`, `build_eval_env`, `rollout`, `get_task_types`.
 ## Tips
-!!! tip
+- Start with `train.batch_size: 4` and `limit: 10` while debugging.
-    - Use a small `batch_size` (10-20) for initial testing
+- The `evaluate` half lives **inside your `rollout`**, not as a separate
-    - The `evaluate()` method is critical — a noisy metric will confuse the optimizer
+  method — there is no `evaluate()` in the `EnvAdapter` ABC. Score the
  prediction in `run_batch` and put the score on each result dict's
  `hard` / `soft`.
 - Noisy scoring kills the optimizer. Spend time on `run_batch`'s scoring
  before you spend time on prompts.
 - If your benchmark needs heavy optional deps (selenium, vllm, ...),
  wrap the registration block with `try / except ImportError` (Step 5)
  so people without those deps can still `--help`.
 - Copy `skillopt/envs/_template/` as a starting skeleton — it now
  implements the real abstract methods.
@@ -1,81 +1,195 @@
 # API Reference
 This page documents the public Python API SkillOpt exposes for **extending the
 framework** with new environments / benchmarks. For ready-made adapters,
 browse [`skillopt/envs/`](https://github.com/microsoft/SkillOpt/tree/main/skillopt/envs).
 > **Source of truth.** The classes below are real Python ABCs defined in
 > `skillopt/envs/base.py`, `skillopt/datasets/base.py`, `skillopt/types.py`,
 > and `skillopt/evaluation/gate.py`. If this page ever drifts, the code
 > wins — please open an issue.
 ---
 ## Core Classes
 ### `EnvAdapter`
-Abstract base class for benchmark environments.
+`skillopt/envs/base.py` — abstract adapter that connects the SkillOpt
 trainer to an environment (benchmark, simulator, REST API, ...).
 Subclasses **must** implement the five abstract methods below.
 ```python
 from abc import ABC, abstractmethod
 from skillopt.datasets.base import BaseDataLoader, BatchSpec
 class EnvAdapter(ABC):
-    async def execute(self, item, skill, model) -> TaskResult
+
-    def evaluate(self, prediction, ground_truth) -> float
+    # ── Lifecycle hooks (have defaults; override only if needed) ────────
-    def build_prompt(self, item, skill) -> str
+
    def setup(self, cfg: dict) -> None: ...
    def get_dataloader(self) -> BaseDataLoader | None: ...
    def requires_ray(self) -> bool: ...                 # default False
    # ── Abstract methods (subclasses MUST implement) ────────────────────
    @abstractmethod
    def build_train_env(self, batch_size: int, seed: int, **kwargs):
        """Return an environment-manager object to be passed to rollout()."""
    @abstractmethod
    def build_eval_env(self, env_num: int, split: str, seed: int, **kwargs):
        """Like build_train_env() but for a fixed eval split."""
    @abstractmethod
    def rollout(self, env_manager, skill_content: str,
                out_dir: str, **kwargs) -> list[dict]:
        """Run a batch of episodes with the current skill.
        Each returned dict MUST contain:
          - "id":   str        episode/task identifier
          - "hard": int (0|1)  pass/fail (may be float 0.0-1.0 if smoothed)
          - "soft": float      partial-credit score in [0.0, 1.0]
        It MAY contain env-specific extra keys (parsed into RolloutResult.extras).
        """
    @abstractmethod
    def reflect(self, results: list[dict], skill_content: str,
                out_dir: str, **kwargs) -> list[dict | None]:
        """Turn rollout results into a list of raw patch dicts.
        Each dict (or None to drop the slot) MUST contain:
          - "patch":       {"edits": [...]}     a Patch.to_dict() payload
          - "source_type": "failure" | "success"
        """
    @abstractmethod
    def get_task_types(self) -> list[str]:
        """Distinct task-type strings used for stratified sampling."""
 ```
-### `DataLoader`
+The trainer also calls a few default-implemented helpers on every adapter:
 `build_reference_text`, `get_reference_metadata`, `attach_reference_context`,
 `select_representative_items`, and `build_env_from_batch`. Read the docstrings
 in `skillopt/envs/base.py` if you need to override any of these — most
 benchmarks don't.
-Abstract base class for data loading and splitting.
+### `BaseDataLoader` / `SplitDataLoader`
 `skillopt/datasets/base.py` — episode-planning loaders.
 ```python
-class DataLoader(ABC):
+class BaseDataLoader(ABC):
-    def setup(self, cfg: dict) -> None
+    def setup(self, cfg: dict) -> None: ...
-    def get_split_items(self, split: str) -> list[DataItem]
+    @abstractmethod
    def build_train_batch(self, batch_size: int, seed: int, **kwargs) -> BatchSpec: ...
    @abstractmethod
    def build_eval_batch(self, env_num: int, split: str, seed: int, **kwargs) -> BatchSpec: ...
 class SplitDataLoader(BaseDataLoader):
    """Concrete base for dataset-backed envs with on-disk train/val/test splits.
    Subclasses only need to implement load_split_items() (and optionally
    load_raw_items() if you also want ``split_mode='ratio'``).
    """
    def load_split_items(self, split_path: str) -> list[dict]: ...
    def load_raw_items(self, data_path: str) -> list[dict]: ...   # optional
 ```
-### `ModelBackend`
+`SplitDataLoader` handles two layout modes:
-Abstract base class for LLM backends.
+| `split_mode` | What it expects |
 |---|---|
 | `"split_dir"` | A directory with `train/`, `val/`, `test/` subdirs already split. |
 | `"ratio"` | A raw dataset path + `split_ratio: "2:1:7"` style string. |
 In either case the items returned by `load_split_items()` are plain
 `dict` objects with at minimum an `"id"` key.
 ### `BatchSpec`
 `skillopt/datasets/base.py` — a slotted dataclass describing one batch
 request the trainer hands to the adapter.
 ```python
-class ModelBackend(ABC):
+@dataclass(slots=True)
-    async def generate(self, messages, **kwargs) -> ModelResponse
+class BatchSpec:
-    async def generate_with_tools(self, messages, tools, **kwargs) -> ModelResponse
+    phase: str                 # "train" | "eval"
-```
+    split: str                 # "train" | "val" | "test" | "valid_seen" | ...
-
+    seed: int
-### `Trainer`
+    batch_size: int
-
+    payload: object | None = None     # what the loader produced (e.g. list[dict])
 Main training loop orchestrator.
 ```python
 class Trainer:
    def __init__(self, cfg: dict)
    async def train(self) -> TrainResult
    async def evaluate(self, skill: str, split: str) -> EvalResult
 ```
 ## Data Classes
 ### `DataItem`
 ```python
@dataclass
 class DataItem:
    id: str
    input: str
    ground_truth: str
    metadata: dict = field(default_factory=dict)
 ```
-### `TaskResult`
+### `Edit` / `Patch`
 `skillopt/types.py` — the I/O types Reflect / Aggregate / Update produce
 and consume.
 ```python
 EditOp = Literal["append", "insert_after", "replace", "delete"]
@dataclass
-class TaskResult:
+class Edit:
-    item_id: str
+    op: EditOp
-    prediction: str
+    content: str = ""
-    score: float
+    target: str = ""
-    trajectory: list[dict]
+    support_count: int | None = None
    source_type: Literal["failure", "success"] | None = None
    merge_level: int | None = None
    update_origin: str = ""
    update_target: str = ""
@dataclass
 class Patch:
    edits: list[Edit] = field(default_factory=list)
    reasoning: str = ""
    ranking_details: dict[str, Any] | None = None
 ```
-### `ModelResponse`
+Both types support `to_dict()` / `from_dict()` for serialization.
-```python
+### `RolloutResult`
@dataclass
 class ModelResponse:
    content: str
    usage: dict
    model: str
 ```
-For detailed source code, see the [`skillopt/`](https://github.com/microsoft/SkillOpt/tree/main/skillopt) directory.
+`skillopt/types.py` — the normalised rollout return type. The trainer
 calls `RolloutResult.from_dict(...)` on each dict returned from
 `EnvAdapter.rollout()`, so the only **hard** requirement on those dicts is
 the three keys above (`id`, `hard`, `soft`). Extra fields are preserved
 into `RolloutResult.extras`.
 ### `GateResult` / `GateAction`
 `skillopt/evaluation/gate.py` — the validation-gate decision types
 returned each epoch.
 ---
 ## Registering an environment
 Environments are not registered via decorators or a `BENCHMARK_REGISTRY`
 dict. The trainer keeps a lazy registry inside `scripts/train.py` —
 `_ENV_REGISTRY` — populated by `_register_builtins()`. To add a new env
 you append a `try / except ImportError` block there. See
 [Add a New Benchmark](../guide/new-benchmark.md) for the full step-by-step.
 ---
 ## Backends (model layer)
 The model layer lives under `skillopt.model.*`. Backends are selected
 via `model.optimizer_backend` and `model.target_backend` in the config —
 not via a base class subclass. Supported values (as of this writing):
 | Backend | Optimizer? | Target? |
 |---|---|---|
 | `openai_chat` | ✓ | ✓ |
 | `claude_chat` | ✓ | ✓ |
 | `qwen_chat` | ✓ | ✓ |
 | `minimax_chat` | ✓ | ✓ |
 | `codex_exec` | — | ✓ |
 | `claude_code_exec` | — | ✓ |
 See `skillopt/model/backend_config.py` for the live whitelist and
 [`docs/reference/config.md`](./config.md) for the per-backend
 configuration keys.
@@ -0,0 +1,110 @@
 # SkillOpt-Sleep 😴 — deployment-time companion (preview)
 **SkillOpt-Sleep** applies SkillOpt's discipline to your *own daily usage*. It gives a
 local coding agent a nightly **sleep cycle** that reviews your past sessions, replays
 your recurring tasks on your own API budget, and consolidates what it learns into
 **validated** long-term memory and skills — behind a held-out gate, staged for your
 review. The agent gets better the more you use it, with **no weight training** and
 **zero inference-time overhead**.
 > **Preview.** This is an early preview we are actively iterating on; interfaces and
 > defaults may change. The engine lives in the top-level [`skillopt_sleep/`](../../skillopt_sleep)
 > package with **zero dependency** on the paper's `skillopt/` code (the validation gate
 > is vendored).
 ## How it works
 One "night":
 ```
 harvest Claude Code / Codex transcripts → mine recurring tasks → replay offline
   → consolidate (reflect → bounded edit → GATE on real held-out tasks)
   → stage proposal → (you) adopt
 ```
 It synthesizes **SkillOpt** (validation-gated bounded text edits), **Claude Dreams**
 (offline consolidation; review-then-adopt), and the **agent-sleep** idea (short-term
 experience → long-term competence).
 ## How to use it
 ### Quickest path: the `skillopt-sleep` CLI (pip)
 ```bash
 pip install skillopt        # installs the engine + the `skillopt-sleep` command
 skillopt-sleep dry-run      # harvest + mine + replay, report only (changes nothing)
 skillopt-sleep run          # a full nightly cycle; the proposal is staged for review
 skillopt-sleep status       # show state + the latest staged proposal
 skillopt-sleep adopt        # apply the latest staged proposal
 skillopt-sleep schedule     # install a nightly cron entry for this project
 ```
 The per-agent plugin shells below (Claude Code / Codex / Copilot) still come from the
 repo; the CLI above is the standalone, pip-only way to run a cycle.
 One engine, thin per-agent shells (see [`plugins/`](../../plugins)):
 | Platform | Folder | Install |
 |---|---|---|
 | **Claude Code** | [`plugins/claude-code`](../../plugins/claude-code) | `/plugin marketplace add ./plugins/claude-code` → `/skillopt-sleep` |
 | **Codex** | [`plugins/codex`](../../plugins/codex) | `bash plugins/codex/install.sh` → `skillopt-sleep` skill |
 | **Copilot** | [`plugins/copilot`](../../plugins/copilot) | register `plugins/copilot/mcp_server.py` as an MCP server |
 Deterministic proof (no API key):
 `python -m skillopt_sleep.experiments.run_experiment --persona researcher --assert-improves`.
 ### Opt-in: experience replay & dream rollouts
 Two consolidation mechanisms, both default **off** (behavior is unchanged unless you
 enable them). They strengthen the nightly update when your tasks have a clean
 correctness signal; the validation gate still governs what ships.
 | Config knob | Default | Effect |
 |---|---|---|
 | `dream_rollouts` | `1` | Run each task K times → learn from the good-vs-bad contrast (contrastive reflection). |
 | `recall_k` | `0` | Associative recall — pull the K most-similar past tasks (from a persisted archive) into tonight's dream. |
 | `dream_factor` | `0` | Add N lightweight synthetic variants of each task. |
 ## Results
 > 📊 **More results & analysis — the gate-safety stress test, experience-replay
 > scaling, and the dream-diversity ablation — are in
 > [`docs/sleep/RESULTS.md`](RESULTS.md).** The highlights:
 **Protocol (identical for every row below).** 5 nights × 10 new real "today" tasks
 per night; the full held-out **test** split is scored before night 1 (baseline) and
 after night 5 (after); optimizer = GPT-5.5; single seed (42); run through the exact
 shipped engine (`skillopt_sleep.dream.dream_consolidate`). Numbers are absolute
 held-out accuracy; **Δ** = `after − baseline` in percentage points.
 **(a) End-to-end on real agents — [gbrain-evals](https://github.com/garrytan/gbrain-evals) `skillopt-v1`.**
 Deficient seed skills go **0.00 → 1.00** on the held-out set with **both Claude Code
 and Codex** as the target agent (all 4 seeds, including a real tool-use loop).
 **(b) Experience replay scales the gain — SearchQA** (1,400-item held-out test,
 SQuAD exact-match; target = GPT-5.5; **validation-gated**):
 | Replay config (`dream_rollouts=5`) | Baseline → After | Δ (pts) |
 |---|---|---|
 | `recall_k=10` | 0.802 → 0.834 | +3.1 |
 | `recall_k=20` | 0.803 → 0.848 | **+4.5** |
 | full-history replay *(reference, not a shipping default)* | 0.796 → 0.851 | +5.6 |
 | `recall_k=10`, `dream_rollouts=8` *(more dreaming, same recall)* | 0.798 → 0.835 | +3.7 |
 The gain rises monotonically with how much relevant past experience is recalled. The
 same SearchQA cell **without** the gate (`recall_k=10`) is 0.808 → 0.839 (+3.1).
 **(c) Second benchmark — SpreadsheetBench** (280-item held-out test; the agent's
 generated openpyxl code is executed and compared cell-by-cell to a golden workbook;
 target = GPT-5.4-nano; gate-free + the output-contract guardrail): 0.279 → 0.314 (**+3.6**).
 **(d) Honest scope.** These gains hold where tasks recur and have a checkable
 correctness signal. On saturated or noisy benchmarks (e.g. a strong model already
 near ceiling) the effect is **flat within run-to-run noise** — single-seed baseline
 variance here is ±1–2 pts, so treat sub-~1.5 pt differences as noise. The validation
 gate keeps the worst case bounded; keep it **on** by default.
 ## Learn more
 Full reference (pipeline, the three plugins, the experience-replay knobs) is in the
 **[Documentation & Reproduction Guide](https://microsoft.github.io/SkillOpt/docs/guideline.html#sleep)**.
@@ -0,0 +1,185 @@
 # SkillOpt-Sleep — results & analysis
 This is the evidence behind SkillOpt-Sleep: does a nightly, offline sleep cycle
 actually make a *deployed* agent better, and is it safe to run unattended? We
 answer with a controlled deployment-scale study — the same protocol the plugin
 runs in production, scored on full held-out test sets.
 ## Setup
 **Protocol (identical for every cell unless stated).** 5 nights; each night adds
 **10 new real "today" tasks**; the skill carries over and is refined night to
 night. The full held-out **test** split is scored before night 1 (*baseline*) and
 after night 5 (*after*); **Δ = after − baseline** in percentage points. Optimizer
 model = **GPT-5.5**; single seed (42); every number is produced by the exact
 shipped engine `skillopt_sleep.dream.dream_consolidate` (the experiment harness and
 the plugin cycle call the same function).
 **Benchmarks** (real evaluators, not format heuristics):
 | Benchmark | Held-out test | Scoring |
 |---|---|---|
 | SearchQA | 1,400 items | SQuAD exact-match vs gold |
 | LiveMathematicianBench | 124 items | multiple-choice label (choices shuffled per item) |
 | SpreadsheetBench | 280 items | the agent's generated openpyxl code is **executed**, output workbook compared cell-by-cell to a golden file |
 **Targets:** GPT-5.5, GPT-5.4-mini, GPT-5.4-nano. **Modes:** validation-gated
 (default) and gate-free.
 ---
 ## 1. The headline — the validation gate is what makes nightly self-evolution *safe*
 Self-evolution is easy to build and easy to ruin: an optimizer that accepts its
 own "lessons" unconditionally can adopt a plausible-but-wrong rule and an obedient
 model will follow it off a cliff. We reproduced exactly that failure, then showed
 the gate prevents it.
 Stress case — **GPT-5.4-nano on SearchQA**, weak model on a single-sample (degraded)
 reflection signal, same nights, same candidate edits, gate **off** vs **on**:
 | | Night 0 → Night 5 | Δ |
 |---|---|---|
 | **no gate** | 0.554 → **0.026** | **−52.8** |
 | **with gate (default)** | 0.570 → 0.570 | 0.0 |
 Ungated, the optimizer learned "answer with the document-title string, verbatim";
 the model complied and accuracy collapsed night after night
 (0.554 → 0.490 → 0.325 → 0.031 → 0.034 → 0.026). The gated twin **rejected every one
 of those edits** and never lost a point. This single experiment is the core
 argument for SkillOpt-Sleep's design, and why the gate ships **on by default**.
 ---
 ## 2. Cross-model scaling — bigger gains where there's headroom
 The same protocol on a weaker target model (**GPT-5.4-nano**, optimizer = GPT-5.5)
 produces substantially larger gains — because the weaker model has more room to
 learn. This is the realistic "cheap deployed agent, strong overnight optimizer"
 scenario:
 | Config (SearchQA, nano, gated) | Baseline → After | Δ | Night-by-night |
 |---|---|---|---|
 | **cumulative replay, nights=5** | 0.560 → **0.679** | **+11.9** | 0.560 → 0.626 → 0.665 → 0.665 → 0.665 → 0.679 |
 | recall_k=20, nights=5 | 0.566 → 0.681 | +11.5 | 0.566 → 0.659 → 0.685 → 0.685 → 0.681 → 0.681 |
 | cumulative, nights=8 | 0.562 → 0.657 | +9.5 | saturates after night 5 |
 Both replay strategies (cumulative and recall) agree within 0.4 pt — the gain is
 robust across configurations.
 **Compared to GPT-5.5 on the same benchmark (SearchQA, gated):**
 | Target model | Best Δ | Baseline | Headroom |
 |---|---|---|---|
 | GPT-5.4-nano | **+11.9** | 0.560 | 44 pt |
 | GPT-5.5 | +6.0 | 0.798 | 20 pt |
 The story: **SkillOpt-Sleep helps most where there's the most to learn** — weaker
 deployed models benefit ~2× as much from the same nightly optimization. This is
 also the economical deployment pattern (cheap inference model + one strong
 overnight optimizer call).
 ---
 ## 3. Experience replay turns a one-time bump into a climb
 The plugin's two opt-in knobs (`recall_k`, `dream_rollouts`) are what produce the
 gains. On **SearchQA, GPT-5.5, gated** — the gain rises monotonically with how
 much relevant past experience is recalled:
 | Replay (`dream_rollouts=5`) | Baseline → After | Δ |
 |---|---|---|
 | `recall_k=10` | 0.802 → 0.834 | +3.1 |
 | `recall_k=20` | 0.803 → 0.848 | **+4.5** |
 | full-history (reference, not a default) | 0.796 → 0.851 | +5.6 |
 And the curve genuinely **climbs across nights** rather than jumping once and
 plateauing — full-history replay, gated, night by night:
 ```
 0.798 → 0.814 → 0.854 → 0.854 → 0.854 → 0.858
 ```
 The gate accepts a new, better skill as late as **night 5** (0.854 → 0.858).
 Replay-policy ablation (SearchQA, GPT-5.5):
 | Replay policy | Gate-free Δ | Gated Δ |
 |---|---|---|
 | none (tonight's tasks only) | +3.9 | +2.0 |
 | **recall k=10 (shipped default-able)** | +5.1 | +4.4 |
 | cumulative (full history) | +4.8 | +6.0 |
 Recall captures most of cumulative's benefit at a fraction of the per-night cost.
 ---
 ## 4. Default hyperparameters are the sweet spot
 We swept `dream_factor`, `rollouts`, `per_night`, and `nights` on the nano cell
 (SearchQA, gated) to verify the shipped defaults are well-tuned:
 | Variant | Δ | vs default (+11.9) |
 |---|---|---|
 | dream_factor=4 (default 2) | +8.8 | −3.1 |
 | rollouts=10 (default 5) | +9.5 | −2.4 |
 | per_night=15 (default 10) | +2.7 | −9.2 |
 | nights=8 (default 5) | +9.5 | −2.4 |
 Every direction away from the default hurts. This means users get the best result
 **out of the box** without tuning — the recipe is robust by design.
 ---
 ## 5. Why these gains exist — the dream-diversity fix (and a rigor note)
 Reflection learns from the **contrast** between good and bad rollouts of the same
 task, which requires the K dream rollouts to be *independent samples*. An early
 version of the engine collapsed them to one cached sample, so contrastive
 reflection never fired. Fixing that, then adding recall, is what produces the
 gains in Sections 1–2. Measured across an 18-cell deployment sweep (3 benchmarks ×
 3 targets × 2 modes), under three engine configurations:
 | Engine configuration | mean Δ | worst-cell Δ | cells > +0.5 | cells < −0.5 |
 |---|---|---|---|---|
 | single-sample reflection (degraded) | −2.66 | **−52.8** | 7 / 18 | 5 / 18 |
 | diverse rollouts (K=5), no recall | +0.24 | −4.0 | 6 / 18 | 7 / 18 |
 | **diverse rollouts + recall (shipped)** | **+0.53** | **−2.4** | 7 / 18 | 7 / 18 |
 The catastrophic −52.8 is removed **at its source** by diverse rollouts: the same
 gate-free nano-SearchQA cell goes 0.554 → **0.586 (+2.7)** with no gate at all once
 the dream is fixed. Recall then lifts the grid mean and tightens the worst case.
 This is **defense in depth, each layer measured**: diverse rollouts propose better
 edits, recall remembers relevant experience, and the gate catches whatever still
 slips through.
 ---
 ## 6. End-to-end on real agents
 On the public [gbrain-evals](https://github.com/garrytan/gbrain-evals) `skillopt-v1`
 benchmark — designed for exactly this learnable-gap setting — deficient seed skills
 go **0.00 → 1.00** on the held-out set with **both Claude Code and Codex** as the
 target agent (all 4 seeds, including a real tool-use loop), and the two agents
 cross-verify each other's consolidated skills.
 ---
 ## 7. Honest scope & limitations
 - **Where it helps:** recurring tasks with a checkable correctness signal and real
  headroom. That is the plugin's actual use case (your repeated daily tasks and
  house rules the agent keeps missing).
 - **Where it's flat:** saturated tasks on strong models, or noisy tasks with a weak
  learning signal — within run-to-run noise.
 - **Single seed.** Cells aggregate one seed per config; treat sub-~1.5 pt
  differences as noise. Spot seed-robustness check on the one flagged cell
  (nano SearchQA gated): seeds 42/43/44 give −1.9 / +3.6 / +4.7 (3-seed mean
  **+2.1**), i.e. the tabled −1.9 is a pessimistic draw, not the typical outcome.
 - **Keep the gate on.** It is the difference between bounded downside (−2.4) and a
  −52.8 collapse. Gate-free mode is for users who cannot hold out a validation set
  and is additionally protected by the output-contract guardrail.
 ---
 Back to the module overview: [`docs/sleep/README.md`](README.md) ·
 full reference: [Documentation & Reproduction Guide](https://microsoft.github.io/SkillOpt/docs/guideline.html#sleep).
@@ -0,0 +1,237 @@
 # SkillOpt Sleep — Claude Code self-evolving plugin (design)
 **Status:** approved-for-build (autonomous offline session, 2026-06-07)
 **Author:** generated for Yifan Yang, executed autonomously while user is asleep
 **Branch:** `feat/claude-code-sleep-plugin` (worktree `my_repo/SkillOpt-sleep`)
 ---
 ## 1. One-paragraph summary
 `skillopt-sleep` is a Claude Code plugin that gives a user's local Claude
 agent a nightly **sleep cycle**. While the user is offline, it (1) **harvests**
 the day's real Claude Code session transcripts from `~/.claude`, (2) **mines**
 them into discrete *task records* with checkable outcomes, (3) **replays /
 "dreams"** those tasks offline using the user's own API budget, and (4) runs
 the **SkillOpt optimizer loop** (reflect → bounded edit → held-out gate) to
 consolidate short-term experience into long-term **memory** (`CLAUDE.md`) and
 **skills** (`SKILL.md`). Only changes that pass a validation gate are kept, and
 every change is written to a **review staging area** the user approves before it
 touches live config — mirroring Claude Dream's "input store is never modified"
 safety contract. The result: an agent that measurably gets better at *this
 user's* recurring work, every night, with zero model-weight training.
 ## 2. Why this is the right synthesis of the three ingredients
 | Ingredient | What we take from it | Where it lives in this design |
 |---|---|---|
 | **SkillOpt** (your paper/code) | Skill = trainable text state; bounded add/delete/replace edits under a textual learning rate; **held-out validation gate**; rejected-edit buffer; epoch-wise slow/meta update. | The `consolidate` stage *is* a single SkillOpt epoch, reusing `skillopt.optimizer.*` and `skillopt.evaluation.gate`. |
 | **Claude Dreams** | Async offline job: read a memory store + 1–100 session transcripts → emit a **new, separate** reorganized memory store (dedup / merge / resolve contradictions / surface insights). Input never mutated; output reviewed then adopted or discarded. | The `harvest` + `consolidate-memory` stages and the **staging/adopt** safety model are modeled directly on Dreams. |
 | **Agent Sleep paper** (2605.26099) | Agents need periodic offline consolidation: short-term experience buffer → synthetic replay/self-generated data → self-update; "sleep" turns episodes into durable competence. | The whole nightly schedule, the `replay` step, and the short-term→long-term framing. |
 The key novel claim this enables for the project (and a future paper section):
 **SkillOpt's validation-gated bounded-edit optimizer is the missing "safe
 update rule" for Dream-style memory consolidation.** Dreams reorganize memory
 but don't *prove* the reorganization helps; the Sleep paper consolidates but
 assumes weight updates. SkillOpt-Sleep consolidates **text** (memory + skills)
 and **gates each change on replayed task performance**, so nightly evolution is
 both weight-free and regression-protected.
 ## 3. Goals / non-goals
 **Goals**
 1. A working Claude Code plugin: scheduled (nightly/cron) **and** user-triggered (`/sleep`).
 2. Look back over the user's real past prompts & trajectories from local `~/.claude` records.
 3. Offline "dream training": re-run mined tasks (mock-env or fresh retry) on the user's budget.
 4. Continuous evolution of **memory** (`CLAUDE.md`) and **skills** (`SKILL.md`) via the SkillOpt gate.
 5. A reproducible experiment that answers: *does the nightly loop actually improve a held-out score?*
 6. Safety: never silently overwrite user config; stage → user approves → adopt.
 **Non-goals (now)**
 - Codex version (explicitly deferred by user; architecture keeps it pluggable).
 - Anthropic managed Dreams API integration (we *emulate* Dreams locally; managed API is a future backend).
 - Model fine-tuning / weight updates (out of scope by design — text-only).
 - Fully unattended auto-adopt by default (opt-in; default is review-gated).
 ## 4. The local data we read (verified on this machine)
 - **Prompt history:** `~/.claude/history.jsonl` — one JSON/line: `{display, pastedContents, timestamp, project}`. The cross-session list of every prompt the user typed, with project path + epoch-ms timestamp.
 - **Full transcripts:** `~/.claude/projects/<path-slug>/<sessionId>.jsonl` — one record/line. Record `type` ∈ {`user`,`assistant`,`mode`,`permission-mode`,`attachment`,`file-history-snapshot`,`last-prompt`,…}. User/assistant records carry `message` (role+content blocks), plus `cwd`, `gitBranch`, `timestamp`, `sessionId`, `version`, `userType`. ~215k transcripts present on this box.
 - **Deployment targets we may evolve:**
  - Project memory: `<project>/CLAUDE.md` (and `~/.claude/CLAUDE.md` global).
  - User skills: `~/.claude/skills/<name>/SKILL.md` (frontmatter: `name`, `description`, optional `allowed-tools`, `argument-hint`).
  - Plugin skills under `~/.claude/plugins/...`.
 Everything stays **on-disk and local**; the only network calls are the LLM
 optimizer/replay calls the user already pays for.
 ## 5. Architecture
 ### 5.1 The nightly Sleep Cycle (stages)
 ```
            ┌────────────────────────── SLEEP CYCLE (one "night") ──────────────────────────┐
            │                                                                                │
 trigger →  │  1.HARVEST     2.MINE          3.REPLAY            4.CONSOLIDATE      5.STAGE   │ → wake report
 (cron or   │  read ~/.claude scan sessions  re-run tasks        SkillOpt epoch:   write to   │
  /sleep)   │  transcripts → → task records   offline (mock or    reflect→edit→     .skillopt-│
            │  + history     w/ outcomes &    fresh retry) under  GATE on held-out  sleep/    │
            │                checkable refs   current skill/mem    replay split      staging/ │
            │                                                                          ↓      │
            │                                              6.ADOPT (opt-in / user-approved)   │
            └────────────────────────────────────────────────────────────────────────────────┘
 ```
 **1. Harvest** (`harvest.py`)
 Read `history.jsonl` + per-project transcript JSONLs for a time window
 (default: since last sleep, fallback last 24–72h). Group by project (`cwd` /
 `project`). Emit normalized `SessionDigest` objects: ordered user prompts,
 assistant final texts, tool-call summary, files touched (from
 `file-history-snapshot`), git branch, errors seen, and **user-feedback signals**
 (e.g. "still broken", "that's wrong", "perfect", re-asks of the same thing).
 **2. Mine** (`mine.py`)
 Turn digests into `TaskRecord`s — the unit the optimizer trains on. A task is a
 self-contained intent (the user's request) plus an *outcome label* and, where
 possible, a **checkable reference**:
 - *Explicit success/failure* from feedback signals ("works now" after N retries → the early attempts are failures, the fix is the success exemplar).
 - *Self-consistency check*: re-derivable answers (math, lookups) get a reference; open-ended ones get an LLM-judge rubric instead.
 - Each TaskRecord: `{id, project, intent, context_excerpt, attempted_solution, outcome ∈ {success,fail,mixed}, reference_kind ∈ {exact, rubric, none}, reference, tags}`.
 Mining is itself an LLM call (the **miner**), prompt-tunable, with a deterministic regex/heuristic fallback for offline/no-key runs.
 **3. Replay / "Dream"** (`replay.py`)
 For mined tasks, re-run the intent **offline** under the *current* skill+memory
 to get a fresh trajectory & score. Two modes:
 - `mock` (default, safe): reconstruct a sandboxed prompt from the task's captured context (no live repo mutation, no network side effects) and run the target model. Deterministic, cheap, safe to run unattended.
 - `fresh` (opt-in): actually re-attempt in a throwaway git worktree of the project. Higher fidelity, heavier, never touches the user's working tree.
 Scoring: exact-match / substring for `exact` refs; LLM-judge (0–1) for `rubric` refs; this yields the `hard`/`soft` scores SkillOpt already expects.
 **4. Consolidate** (`consolidate.py`) — *this is one SkillOpt epoch*
 Reuse the existing optimizer pieces rather than reinventing:
 - `reflect`: partition replayed tasks into failure/success minibatches → propose add/delete/replace edits to **skill** and a parallel proposer for **memory** (`CLAUDE.md`). (Memory consolidation also does Dream-style dedup/merge/contradiction-resolution over existing `CLAUDE.md` lines.)
 - `aggregate` + `rank_and_select` under an **edit budget** (textual learning rate).
 - `apply_patch_with_report` → candidate skill / candidate memory.
 - **GATE** (`skillopt.evaluation.gate.evaluate_gate`): replay a *held-out* slice of tasks with the candidate; accept only if it strictly beats current. Rejected edits go to the rejected-edit buffer (negative feedback) exactly as in the paper.
 - A **slow/meta** pass across nights (not just within one night) carries durable, cross-session lessons — the literal "short-term experience → long-term knowledge" of the Sleep paper. Per-night state persists in `~/.skillopt-sleep/state.json`.
 **5. Stage** (`staging/`)
 Write `proposed_CLAUDE.md`, `proposed_SKILL.md`, a unified diff, and a
 `sleep_report.md` (what changed, why, gate deltas, token cost) into
 `<project>/.skillopt-sleep/staging/<date>/`. **Nothing live is modified.**
 **6. Adopt**
 `/sleep adopt` (or `auto_adopt: true` in config for power users) copies staged
 files over the live `CLAUDE.md` / `SKILL.md`, after a `git`-style backup. This
 is the only stage that mutates user-facing config, and it is explicit by default
 — the Dreams "review the output, then adopt or discard" contract.
 ### 5.2 Components & boundaries (each independently testable)
 ```
 skillopt/sleep/
  __init__.py
  types.py         # SessionDigest, TaskRecord, ReplayResult, SleepConfig, SleepReport (dataclasses)
  harvest.py       # ~/.claude transcripts + history.jsonl  ->  list[SessionDigest]
  mine.py          # list[SessionDigest]  ->  list[TaskRecord]   (LLM miner + heuristic fallback)
  replay.py        # TaskRecord + skill + memory  ->  ReplayResult (hard/soft)   (mock | fresh)
  consolidate.py   # ReplayResults -> candidate skill+memory -> GATE -> accepted artifacts
  memory.py        # CLAUDE.md read/merge/dedup/diff (Dream-style) + protected-region markers
  state.py         # ~/.skillopt-sleep/state.json: last_sleep, night counter, slow/meta memory
  staging.py       # write/adopt staging dir, backups
  cli.py           # `python -m skillopt.sleep {run|status|adopt|harvest|dry-run}`
  config.py        # SleepConfig load/merge (defaults + ~/.skillopt-sleep/config.yaml)
  optimizer_backend.py  # thin: route reflect/judge to a chosen backend; mock backend for tests
 skillopt-sleep-plugin/            # the Claude Code plugin surface
  .claude-plugin/plugin.json
  commands/sleep.md               # /sleep [run|status|adopt|dry-run]
  commands/sleep-status.md
  skills/skillopt-sleep/SKILL.md  # so Claude knows how to drive the engine
  hooks/hooks.json                # optional: schedule + on-session-end harvest
  scripts/*                       # shims that call `python -m skillopt.sleep ...`
 ```
 **Reuse, don't fork:** `consolidate.py` calls into existing
 `skillopt.optimizer.clip.rank_and_select`, `skillopt.gradient.aggregate.merge_patches`,
 `skillopt.optimizer.skill.apply_patch_with_report`, and
 `skillopt.evaluation.gate.evaluate_gate`. The sleep layer is an **EnvAdapter-shaped
 shim** over the user's own life, not a new optimizer.
 ### 5.3 Data flow (one task, end to end)
 ```
 history.jsonl + <session>.jsonl
   └─harvest→ SessionDigest{prompts, finals, tools, feedback}
        └─mine→ TaskRecord{intent, attempted, outcome, reference}
             └─replay(current skill+mem)→ ReplayResult{hard, soft, trajectory}
                  └─reflect→ edits(skill), edits(memory)
                       └─rank/clip(edit_budget)→ candidate
                            └─GATE(replay held-out)→ accept? → staging/  → (adopt) live CLAUDE.md/SKILL.md
 ```
 ## 6. Scheduling & triggering
 - **Cron/scheduled:** documented `crontab` line + an optional Claude Code hook; default `0 3 * * *` (3am local; pick an off-:00 minute in practice). The engine is a plain CLI so it works under cron, systemd-timer, or the Claude Code scheduler.
 - **User-triggered:** `/sleep run` (full cycle), `/sleep dry-run` (harvest+mine+replay, no edits), `/sleep status`, `/sleep adopt`.
 - **On-session-end harvest (optional hook):** cheaply append the just-finished session to the night's buffer so the 3am run has fresh data without a full rescan.
 ## 7. Safety model (hard requirements)
 1. **Never mutate live `CLAUDE.md`/`SKILL.md` except via explicit `adopt`** (or opt-in `auto_adopt`). Default = staged + reviewed (Dreams contract).
 2. **Backups:** every adopt snapshots the prior file to `staging/<date>/backup/`.
 3. **Read-only harvest:** transcripts are read, never written.
 4. **`fresh` replay runs only in throwaway worktrees**, never the user's checkout; no `rm -rf`, no force-push, network off unless `replay.network: true`.
 5. **Budget cap:** `max_tokens_per_night` + `max_tasks_per_night`; stop early when hit, log what was skipped (no silent truncation).
 6. **Secret hygiene:** redact obvious secrets from digests before they enter prompts (reuse `_redact_*` ideas from trainer).
 7. **PII/scope:** only harvest projects on an allowlist (default: the project the plugin is invoked in) or `projects: all` opt-in.
 ## 8. Validation experiment — "does it actually improve?"
 A self-contained, **deterministic-by-default** experiment lives in
 `skillopt/sleep/experiments/` and is the acceptance test for the whole idea.
 **Setup:** a synthetic "user persona" (e.g. *researcher who keeps asking for
 arXiv-id extraction in a fixed format*, or *programmer who keeps mis-formatting
 git commit messages*). We ship 12–20 tiny tasks with **exact checkable
 references**, split into `replay` (train) and `holdout` (test).
 **Procedure:**
 1. Score the holdout with an **empty** skill+memory → `baseline`.
 2. Run `N` sleep nights (each: replay train slice → reflect → gated edit).
 3. Score holdout with the evolved skill+memory → `after`.
 4. Report `after − baseline`, accept/reject counts, edit count, tokens.
 **Two backends:**
 - `mock` (default, **no API key, fully deterministic**): a scripted optimizer that proposes the known-good rule on failure and a scripted judge. Proves the *plumbing* (harvest→mine→replay→gate→adopt) monotonically improves the score and the gate blocks regressions. This is the CI-able acceptance test.
 - `anthropic` (opt-in, uses `ANTHROPIC_API_KEY`): the real optimizer/judge, to demonstrate genuine lift on the persona tasks.
 **Success criteria:**
 - Mock: `after > baseline`, gate rejects an injected harmful edit, adopt+backup works, re-run is reproducible. (Hard gate in CI.)
 - Anthropic (when run): `after ≥ baseline` on holdout with ≥1 accepted, human-readable edit; documented in the wake-up report.
 ## 9. Personas (the user's framing) → concrete recurring-task families
 - **Programmer:** commit-message conventions, repo-specific build/test commands, "always run X before Y", framework gotchas → consolidated into project `CLAUDE.md` + a `repo-workflow` skill.
 - **Researcher:** citation/format preferences, experiment-logging habits, paper-section style, dataset-path memory → `research-prefs` skill + memory.
 - **Finance/analyst:** report formatting, recurring data-pull recipes, terminology → `report-style` skill + memory.
 The engine is domain-agnostic; the persona only changes which tasks get mined.
 ## 10. Phased delivery
 - **Phase 0 — scaffold + types + harvest** (read-only, no API). Provable on this box's real `~/.claude`.
 - **Phase 1 — mine + replay(mock) + consolidate + gate + staging**, with the **mock** optimizer backend and the deterministic experiment green. *(primary deliverable of the offline session)*
 - **Phase 2 — plugin surface** (`/sleep`, skill, hooks, plugin.json) wired to the CLI.
 - **Phase 3 — real Anthropic backend** for miner/reflect/judge + `fresh` replay in worktrees.
 - **Phase 4 — slow/meta cross-night memory**, adopt automation, multi-project, polish + docs.
 This session targets **Phase 0 + Phase 1 fully**, **Phase 2 scaffolded**, and the
 **deterministic experiment passing**, all committed (not pushed) for review.
 ## 11. Open questions for the user (answer when awake)
 1. **Adopt policy:** keep default *review-gated*, or do you want `auto_adopt` for your own machine?
 2. **Scope:** harvest only the invoked project, or all projects in `~/.claude/projects`?
 3. **Real-API demo:** want me to spend live `ANTHROPIC_API_KEY` budget on the persona demo, or keep everything mock until you say go?
 4. **Skill target:** evolve a *new* dedicated `skillopt-sleep`-managed skill, or also edit your existing hand-written skills in `~/.claude/skills`?
 5. **Paper:** should this become a section/figure in the SkillOpt arXiv (Dream+Sleep framing as "deployment-time continual skill optimization")?
 ```
@@ -2416,14 +2416,11 @@
      <div class="bibtex-box">
        <button class="copy-btn" type="button" onclick="copyBibtex(this)">Copy</button>
-        <pre><code>@misc{yang2026skilloptexecutivestrategyselfevolving,
+        <pre><code>@article{yang2026skillopt,
-      title={SkillOpt: Executive Strategy for Self-Evolving Agent Skills}, 
+  title={Skillopt: Executive strategy for self-evolving agent skills},
-      author={Yifan Yang and Ziyang Gong and Weiquan Huang and Qihao Yang and Ziwei Zhou and Zisu Huang and Yan Li and Xuemei Gao and Qi Dai and Bei Liu and Kai Qiu and Yuqing Yang and Dongdong Chen and Xue Yang and Chong Luo},
+  author={Yang, Yifan and Gong, Ziyang and Huang, Weiquan and Yang, Qihao and Zhou, Ziwei and Huang, Zisu and Li, Yan and Gao, Xuemei and Dai, Qi and Liu, Bei and others},
-      year={2026},
+  journal={arXiv preprint arXiv:2605.23904},
-      eprint={2605.23904},
+  year={2026}
      archivePrefix={arXiv},
      primaryClass={cs.AI},
      url={https://arxiv.org/abs/2605.23904}, 
 }</code></pre>
      </div>
    </section>
@@ -0,0 +1,262 @@
 # SkillOpt-Sleep — plugins for Claude Code, Codex, Copilot, and Devin
 **Your coding agent forgets everything between sessions. SkillOpt-Sleep fixes
 that.** While you sleep, it reviews what you did today, notices the rules you
 keep repeating ("always add a LIMIT", "answers in `\boxed{}`", "cite the
 source"), and writes them into your agent's long-term memory and skills — but
 only the rules that actually make it score better on *your own* past tasks. You
 wake up to an agent that's better at *your* work, and you approve every change
 before it sticks.
 One engine, four thin shells. It synthesizes **SkillOpt** (validation-gated
 bounded text optimization — the research in this repo), **Claude Dreams**
 (offline consolidation; input never mutated; review-then-adopt), and the **agent
 sleep** idea (short-term experience → long-term competence).
 > **Open-source tool, decoupled from the research.** The engine lives in the
 > top-level [`skillopt_sleep/`](../skillopt_sleep) package with **zero
 > dependency** on the paper's `skillopt/` experiment code (the validation gate is
 > vendored). Use it without the research stack.
 ---
 | Platform | Folder | Mechanism | Status |
 |---|---|---|---|
 | **Claude Code** | [`claude-code/`](claude-code) | `.claude-plugin` + `/skillopt-sleep` + `/skillopt-sleep-handoff` commands + skill + hooks | full, installable |
 | **Codex** | [`codex/`](codex) | user-level `skillopt-sleep` skill + shared runner | full |
 | **Copilot** | [`copilot/`](copilot) | MCP server (`sleep_*` tools) + `copilot-instructions` | full (MCP) |
 | **Devin** | [`devin/`](devin) | MCP server (`sleep_*` tools) + Devin ATIF-v1.7 harvest + `.devin/rules` | full (MCP) |
 ## Install (pick your agent)
 | Platform | Install | Then |
 |---|---|---|
 | **Claude Code** | `/plugin marketplace add microsoft/SkillOpt` → `/plugin install skillopt-sleep` | `/skillopt-sleep status` |
 | **Codex** | `git clone` → `bash plugins/codex/install.sh` | `/skillopt-sleep status` |
 | **Copilot** | `git clone` → register `plugins/copilot/mcp_server.py` as an MCP server | ask "run the sleep cycle" |
 | **Devin** | `git clone` → `devin mcp add skillopt-sleep -- python3 plugins/devin/mcp_server.py` | ask "run the sleep cycle" |
 Requirements: Python ≥ 3.10 and the agent's CLI on PATH. All three call the same
 [`run-sleep.sh`](run-sleep.sh) → `python -m skillopt_sleep`, so behaviour is
 identical everywhere. Default backend is `mock` (no API spend); `--backend
 claude|codex|copilot` uses your own budget.
 ---
 ## How it works: one "night", in plain terms
 ```
 harvest your past sessions → mine the tasks you keep doing → replay them offline
  → reflect on failures → propose a few rule edits → KEEP only edits that raise
    your held-out score → stage a proposal → (you) review & adopt
 ```
 Nothing live changes until you `adopt`; every adopt backs up the prior file.
 ### The split that keeps it honest: dream-train / real-val / real-test
 This is the heart of the design, borrowed from the SkillOpt paper's
 train/selection/test protocol:
 | Split | Where it comes from | What it's for |
 |---|---|---|
 | **train** | your real tasks **+ optional "dreamed" variants** | what the optimizer *learns from*. Over-dreaming here is fine — it's imagination. |
 | **val** (selection) | **your real tasks only**, held out | the **gate**: an edit is kept only if it raises this score. Stops overfitting. |
 | **test** | **your real tasks only**, held out, never seen during optimization | the **final score** we report. Kept as close to your real usage as possible. |
 So you can **dream up extra training examples** to learn a rule robustly, while
 the rule is still **judged on real, unseen tasks**. A `dream` task can *never*
 land in val or test — that invariant is unit-tested.
 ---
 ## What each feature does **for you** (with examples)
 Every control below works on all three platforms (pass it after the action,
 e.g. `/skillopt-sleep run --rollouts-k 3`).
 ### `--preferences "..."` — tell it your house rules
 The single most useful knob. Free text that steers what the optimizer writes,
 as a prior. Use it to encode the conventions you're tired of repeating.
 ```bash
 # A backend engineer:
 /skillopt-sleep run --preferences "Always use async/await, never callbacks. \
  Prefer pytest over unittest. Commit subjects in imperative mood under 50 chars."
 # A data analyst:
 /skillopt-sleep run --preferences "Every SQL query must end with LIMIT 1000 unless \
  I say otherwise. Money in USD with 2 decimals. Prefer CTEs over nested subqueries."
 # A researcher:
 /skillopt-sleep run --preferences "Cite sources as [Author, Year]. Math answers in \
  \\boxed{}. Keep explanations under 150 words unless I ask for depth."
 ```
 *What it does for you:* the next morning your agent already follows these
 without you re-typing them, and the rules are validated against your real tasks
 (if a "preference" actually hurts your held-out score, the gate drops it).
 ### `--gate on|off` — strict vs. greedy
 - `on` (default): an edit is kept **only if it raises your held-out score**.
  Safe — blocks plausible-but-wrong rules and reward-hacking.
 - `off`: greedy — keep edits without the strict check (still reports whether
  quality moved).
 *What it does for you:* leave it `on` for trust. Flip it `off` when you're
 exploring and want to see everything the optimizer proposes.
 ### `--rollouts-k K` — learn from contrast, not just failure
 Re-runs each task `K` times and learns from the difference between the **good**
 and **bad** attempts, not just a single failure.
 ```bash
 /skillopt-sleep run --rollouts-k 3
 ```
 *What it does for you:* a much stronger signal. If your agent gets a task right 1
 time in 3, the optimizer figures out *what the winning attempt did* and makes it
 reliable.
 ### `--optimizer-model` / `--target-model` — optimize cheap, deploy anywhere
 Use a strong model to *write* the rules and a cheap model to *run* your tasks.
 The learned skill then helps the cheap model — or any model.
 ```bash
 /skillopt-sleep run --optimizer-model sonnet --target-model haiku
 ```
 *What it does for you:* spend a little on a smart optimizer overnight; your
 everyday cheap/fast agent inherits the upgrade. (Verified: a skill optimized on
 one model lifts a different one — cross-model and even cross-runtime
 Codex↔Claude.)
 ### `--budget-tokens N` / `--budget-minutes M` — cap the spend
 You decide how much the nightly "dreaming" costs; it auto-plans how many nights
 × how many rollouts fit.
 ```bash
 /skillopt-sleep run --backend claude --budget-tokens 60000
 ```
 *What it does for you:* predictable cost. It stops cleanly when the budget is hit
 and tells you what it skipped.
 ### multi-objective (accuracy ↑, tokens ↓, latency ↓)
 The reward can weight not just correctness but **cost and speed**, so a skill can
 learn to be cheaper and faster, not only more accurate. *What it does for you:*
 "answer directly instead of opening five files" becomes a learned habit.
 ### `--backend handoff` — session-executed calls (no API subprocess)
 For subscription seats and environments where the engine shouldn't spawn
 `claude -p` / API calls itself. The engine still runs every deterministic
 stage (harvest → mine → replay scoring → gate → stage), but each model call
 (attempt / judge / reflect) is written to a prompt file that **your own agent
 session answers between engine runs**:
 ```bash
 python -m skillopt_sleep run --backend handoff --project "$(pwd)"
 # exit 3 => .skillopt-sleep-handoff/PROMPTS.md + pending.json were written
 #   answer each prompt (each in a FRESH context) into answers/<id>.md
 # re-run the same command => it resumes from the answers and either
 #   finishes (exit 0) or stages the next prompt batch (exit 3)
 ```
 A typical night converges in 3–6 rounds: baseline attempts → reflect →
 candidate re-scoring per accepted edit. Resume is stateless — replay is
 deterministic and answers are cached by prompt hash, so re-running skips
 everything already answered. Mined tasks are pinned to
 `.skillopt-sleep-handoff/tasks.json` on the first round, so the sessions that
 answer the prompts can't shift the task set and invalidate earlier answers.
 On a completed real run the handoff directory is archived to
 `.skillopt-sleep-handoff.night<N>.done`.
 On Claude Code, `/skillopt-sleep-handoff run` drives the whole loop for you,
 answering each prompt in an isolated fresh-context subagent.
 **Integrity rule:** answer every prompt in a fresh context (a subagent with no
 conversation history). Answering from a session that has already seen the
 mined tasks and their references contaminates the held-out gate and fakes the
 improvement score.
 *What it does for you:* the sleep cycle runs entirely on your interactive
 session's subscription budget — no API key, no headless subprocess — while the
 gate, splits, and staging discipline stay in the engine.
 Limitations: `--rollouts-k > 1` gives no contrastive spread (identical prompt
 → identical answer file), and tool-loop tasks fall back to the single-shot
 `TOOL_CALL:` marker convention.
 ### `schedule` / `unschedule` — set it and forget it
 Built-in nightly scheduling (no manual cron):
 ```bash
 /skillopt-sleep schedule --hour 3 --minute 17     # runs every night for this project
 /skillopt-sleep unschedule                        # stop it
 ```
 *What it does for you:* it just gets better while you sleep. The nightly run only
 *stages* a proposal — adopting is still your call (or add `--auto-adopt` when you
 schedule, if you trust it).
 ---
 ## Full action / flag reference
 | Action | Does |
 |---|---|
 | `status` | nights so far + the latest staged proposal (read-only) |
 | `dry-run` | harvest→mine→replay→report; **stages nothing** |
 | `run` | full cycle; **stages** a proposal; nothing live changes |
 | `adopt` | apply the staged proposal to `CLAUDE.md`/`SKILL.md` (backs up first) |
 | `harvest` | debug: print the recurring tasks it mined |
 | `schedule` / `unschedule` | install/remove the nightly cron entry |
 | Flag | Default | Meaning |
 |---|---|---|
 | `--backend mock\|claude\|codex\|copilot\|handoff` | `mock` | who runs/optimizes (mock = free; handoff = your own session answers) |
 | `--preferences "..."` | – | your house rules, as a prior |
 | `--gate on\|off` | `on` | strict held-out gate vs. greedy |
 | `--rollouts-k K` | `1` | multi-rollout contrastive reflection |
 | `--optimizer-model` / `--target-model` | – | split the optimizer from the target |
 | `--budget-tokens` / `--budget-minutes` | – | cap the nightly spend |
 | `--scope invoked\|all` | `invoked` | this project only, or all projects |
 | `--auto-adopt` | off | apply without manual review (power users) |
 Deep dive: [the SkillOpt-Sleep guide section](https://microsoft.github.io/SkillOpt/docs/guideline.html#sleep).
 ---
 ## Does it actually work?
 Yes — measured with **real models on both Claude and Codex**, scored on held-out
 tasks the optimizer never trained on:
 - **gbrain-evals `skillopt-v1`** (the public suite gbrain scores SkillOpt on):
  deficient skills go **0.00 → 1.00** on all 4 seeds, including a real tool-use
  loop; cross-model transfer is positive; the gate blocks regressions.
  → [the SkillOpt-Sleep guide section](https://microsoft.github.io/SkillOpt/docs/guideline.html#sleep)
 - **Academic daily-cases** (math / spreadsheet / search-QA, the paper's 4:1:5
  split with dream-augmented train): see
  [the SkillOpt-Sleep guide section](https://microsoft.github.io/SkillOpt/docs/guideline.html#sleep).
 - **Fresh load-test** (a "SQL must always include LIMIT" analyst, built from
  scratch): held-out **0.00 → 1.00** on both backends.
  → [the SkillOpt-Sleep guide section](https://microsoft.github.io/SkillOpt/docs/guideline.html#sleep)
 Try the deterministic proof yourself (no API key, no spend):
 ```bash
 python -m skillopt_sleep.experiments.run_experiment --persona researcher --assert-improves
 ```
 It prints the held-out score rising to 1.0 as the gate accepts the right rules,
 and confirms the gate **rejects** an injected harmful edit.
 ---
 ## Safety
 - **Read-only** harvest of your sessions. `mock` replay has no side effects.
 - Proposals are **staged**, never auto-applied (unless you opt in with `--auto-adopt`).
 - Every adopt writes a backup. Per-night token/time budget caps. Secrets redacted.
@@ -0,0 +1,26 @@
 {
  "$schema": "https://anthropic.com/claude-code/marketplace.schema.json",
  "name": "skillopt-sleep",
  "description": "SkillOpt-Sleep: give your local Claude agent a nightly sleep cycle that reviews past sessions and consolidates validated memory + skills.",
  "owner": {
    "name": "Yifan Yang",
    "email": "yifanyang@microsoft.com"
  },
  "plugins": [
    {
      "name": "skillopt-sleep",
      "description": "Nightly offline self-evolution: harvest your past Claude Code sessions, replay recurring tasks on your own API budget, and consolidate what the agent learns into validated CLAUDE.md memory and SKILL.md skills, behind a held-out gate, staged for your review. Synthesizes SkillOpt (validation-gated skill optimization), Claude Dreams (offline memory consolidation), and agent sleep/consolidation.",
      "author": {
        "name": "Yifan Yang"
      },
      "category": "productivity",
      "source": {
        "source": "git-subdir",
        "url": "https://github.com/microsoft/SkillOpt.git",
        "path": "plugins/claude-code",
        "ref": "main"
      },
      "homepage": "https://github.com/microsoft/SkillOpt"
    }
  ]
 }
@@ -0,0 +1,22 @@
 {
  "name": "skillopt-sleep",
  "description": "Give your local Claude agent a nightly 'sleep cycle': it reviews your past sessions offline, replays recurring tasks on your own API budget, and consolidates what it learns into validated memory (CLAUDE.md) and skills (SKILL.md) so it gets better the more you use it. Synthesizes SkillOpt (validation-gated skill optimization), Claude Dreams (offline memory consolidation), and agent sleep/consolidation.",
  "version": "0.1.0",
  "author": {
    "name": "Yifan Yang",
    "email": "yifanyang@microsoft.com"
  },
  "homepage": "https://github.com/microsoft/SkillOpt",
  "repository": "https://github.com/microsoft/SkillOpt",
  "license": "MIT",
  "keywords": [
    "skillopt",
    "self-improvement",
    "memory-consolidation",
    "dreams",
    "sleep",
    "skills",
    "continual-learning",
    "offline-optimization"
  ]
 }
@@ -0,0 +1,161 @@
 # SkillOpt-Sleep (Claude Code plugin)
 > Give your local Claude agent a **sleep cycle**. Every night it reviews your
 > past sessions offline, replays your recurring tasks on your own API budget,
 > and consolidates what it learns into **validated** memory (`CLAUDE.md`) and
 > skills (`SKILL.md`). Your agent gets better the more you use it — no
 > model-weight training.
 SkillOpt-Sleep is the **deployment-time** companion to
 [SkillOpt](https://github.com/microsoft/SkillOpt). SkillOpt trains a skill
 offline on a benchmark; SkillOpt-Sleep applies the same discipline to *your own
 daily usage*: bounded text edits, accepted only through a held-out validation
 gate, with rejected edits kept as negative feedback.
 It synthesizes three ideas:
 | Idea | Contribution |
 |---|---|
 | **SkillOpt** | skill/memory = trainable text; bounded add/delete/replace edits; **held-out gate** keeps only changes that help. |
 | **Claude Dreams** | offline consolidation over past sessions; input never mutated; output **reviewed then adopted**. |
 | **Agent sleep** | periodic offline replay turns short-term episodes into long-term skill. |
 ## What it does (one "night")
 ```
 harvest ~/.claude transcripts → mine recurring tasks → replay offline
   → consolidate (reflect → bounded edit → GATE) → stage proposal → (you) adopt
 ```
 Nothing live is modified until **you** run `/skillopt-sleep adopt` (the Dreams "review,
 then adopt or discard" contract). Every adopt backs up the prior file first.
 ## Install
 **Requirements:** Python ≥ 3.10, and the `claude` CLI (and/or `codex` CLI) on PATH.
 ```bash
 # 1) get the code (the plugin ships inside the SkillOpt repo)
 git clone https://github.com/microsoft/SkillOpt.git
 cd SkillOpt
 # 2) add the plugin to Claude Code as a local marketplace
 /plugin marketplace add ./skillopt-sleep-plugin
 /plugin install skillopt-sleep@skillopt-sleep
 # 3) verify
 /skillopt-sleep status
 ```
 The plugin's bundled runner (`scripts/sleep.sh`) auto-selects a Python ≥ 3.10
 interpreter and calls the `skillopt_sleep` engine in the repo. No `pip install`
 is required for the default `mock` backend or for `claude`/`codex` backends —
 they shell out to the CLIs you already have.
 ## Quick start
 ```bash
 # from inside any project you use with Claude Code:
 /skillopt-sleep dry-run     # safe preview: what it would learn, no changes staged
 /skillopt-sleep run         # full cycle: stages a reviewed proposal (still no live edits)
 /skillopt-sleep status      # see history + the latest staged proposal
 /skillopt-sleep adopt       # apply the staged proposal to CLAUDE.md / SKILL.md (with backup)
 /skillopt-sleep-handoff run # same cycle, but THIS session answers the model calls
                            # (no claude -p subprocess, no API key — subscription-friendly)
 ```
 Or call the engine directly (Python ≥ 3.10):
 ```bash
 python -m skillopt_sleep run --project "$(pwd)" --scope invoked --backend mock
 python -m skillopt_sleep run --project "$(pwd)" --backend claude   # real lift via Claude
 python -m skillopt_sleep run --project "$(pwd)" --backend codex    # real lift via Codex
 ```
 Default backend is **`mock`** — deterministic, no API spend — so you can try the
 plumbing for free. Switch to `--backend claude` or `--backend codex` for genuine
 improvement on your own budget.
 ### Handoff mode (session answers the model calls)
 `--backend handoff` runs the cycle without any model subprocess: the engine
 executes the deterministic stages and writes every model call it needs to
 `.skillopt-sleep-handoff/PROMPTS.md` + `pending.json` (exit code 3). You (or
 the `/skillopt-sleep-handoff` command, which automates the loop with isolated
 fresh-context subagents) write each raw answer to `answers/<id>.md` and re-run
 the same command; it resumes from the answers and either finishes or stages
 the next batch. Typically 3–6 rounds per night.
 ```bash
 python -m skillopt_sleep run --backend handoff --project "$(pwd)"
 # ... answer .skillopt-sleep-handoff/PROMPTS.md into answers/<id>.md ...
 python -m skillopt_sleep run --backend handoff --project "$(pwd)"   # resume
 ```
 Answer every prompt in a **fresh context** — a session that has already seen
 the mined tasks and their references would contaminate the held-out gate.
 Details: [the plugins README](../README.md#--backend-handoff--session-executed-calls-no-api-subprocess).
 ## Does it actually improve? (real models, public benchmark)
 SkillOpt-Sleep is validated against [gbrain-evals](https://github.com/garrytan/gbrain-evals)'
 public `skillopt-v1` suite — the same benchmark gbrain scores its own skill
 optimizer against. We take a deliberately **deficient** skill and run one sleep
 night; held-out scoring is done by a local rule judge (no judge-API, no way to
 grade its own homework).
 | Backend | Seed | Held-out before → after | Nights |
 |---|---|---|---|
 | **Claude (Haiku 4.5)** | brief-writer | **0.00 → 1.00** | 1 |
 | **Codex** | brief-writer | **0.00 → 1.00** | 2 |
 Both took a brief-writer with no risks section / no confidence level and, within
 1–2 nights, proposed gated edits that lifted the held-out score to perfect —
 into the protected `LEARNED` block, nothing else touched. The Codex 2-night
 trace even shows the optimizer **diagnosing its own residual failure** and
 adding a meta-rule to fix it. Full writeup + reproduction:
 [the SkillOpt-Sleep guide section](https://microsoft.github.io/SkillOpt/docs/guideline.html#sleep).
 Reproduce:
 ```bash
 git clone https://github.com/garrytan/gbrain-evals /tmp/gbrain-evals
 python -m skillopt_sleep.experiments.run_gbrain --backend claude --model haiku \
  --seeds brief-writer --data-root /tmp/gbrain-evals/eval/data/skillopt-v1 \
  --nights 1 --limit-replay 3 --limit-holdout 3
 python -m skillopt_sleep.experiments.run_gbrain --backend codex \
  --seeds brief-writer --data-root /tmp/gbrain-evals/eval/data/skillopt-v1 \
  --nights 1 --limit-replay 3 --limit-holdout 3
 ```
 ## Deterministic proof (no API, no keys)
 ```bash
 python -m skillopt_sleep.experiments.run_experiment --persona researcher --assert-improves
 python -m skillopt_sleep.experiments.run_experiment --persona programmer  --assert-improves
 ```
 Each prints the held-out score rising from baseline toward 1.0 as the gate
 accepts the general rules your tasks need, and confirms the gate **rejects** an
 injected harmful edit. Recorded output: [the SkillOpt-Sleep guide section](https://microsoft.github.io/SkillOpt/docs/guideline.html#sleep).
 ## Schedule it nightly
 ```bash
 "${CLAUDE_PLUGIN_ROOT}/scripts/install-cron.sh" "$(pwd)"   # prints a crontab line; installs nothing
 ```
 ## Safety
 - **Read-only** harvest of `~/.claude`. `mock` replay has no side effects.
 - Proposals are **staged**, never auto-applied (unless you opt in with `--auto-adopt`).
 - Every adopt writes a backup under the staging dir's `backup/`.
 - Per-night **token/task budget caps**; secrets redacted from prompts.
 - `fresh` replay (Phase 3) runs only in throwaway git worktrees.
 ## Status
 Phase 1 (engine + deterministic experiment + plugin surface) is complete.
 Phase 3 adds the real-API miner/judge and `fresh` worktree replay. See
 [`docs/superpowers/specs/2026-06-07-skillopt-sleep-claude-code-plugin-design.md`](../docs/superpowers/specs/2026-06-07-skillopt-sleep-claude-code-plugin-design.md).
@@ -0,0 +1,67 @@
 ---
 description: Run the SkillOpt-Sleep cycle with the handoff backend — no API subprocess; this session answers the engine's model calls via prompt/answer files, in isolated fresh-context subagents
 argument-hint: "[run | dry-run] [--preferences \"...\"] (default: run)"
 allowed-tools: Bash, Read, Write, Task
 ---
 # /skillopt-sleep-handoff — session-executed sleep cycle
 You are driving **SkillOpt-Sleep in handoff mode**: the Python engine runs
 every deterministic stage (harvest → mine → replay scoring → gate → stage)
 and outsources each model call (attempt / judge / reflect) to YOU via
 prompt files. No `claude -p` subprocess, no API key — the model work runs
 on this session's budget, but each prompt MUST be answered in a fresh,
 isolated context so the validation gate stays honest.
 ## Requested action: $ARGUMENTS
 (If `$ARGUMENTS` is empty, treat it as `run`.)
 ## The loop
 Repeat until the engine exits 0 (done) — at most 8 rounds:
 1. **Run the engine** via the bundled runner:
   ```bash
   "${CLAUDE_PLUGIN_ROOT}/scripts/sleep.sh" <action> --backend handoff --project "$(pwd)" --scope invoked
   ```
   - exit 0 → the night is complete; go to "Finish" below.
   - exit 3 → pending model calls; continue with step 2.
   - anything else → stop and show the user the error output.
 2. **Read the batch**: `Read` `.skillopt-sleep-handoff/pending.json` in the
   project. Each entry has `id`, `prompt`, `max_tokens`, `answer_file`.
 3. **Answer each prompt in ISOLATION** — this is the integrity rule:
   - For each entry, launch a subagent (Task tool) whose ENTIRE input is
     the `prompt` text verbatim. Add nothing: no summary of this session,
     no mention of SkillOpt, no other prompts from the batch.
   - Take the subagent's reply and `Write` the raw answer text (no
     commentary, no code fences) to the entry's `answer_file`.
   - NEVER answer from this session's own context — you have seen the
     mined tasks and their references, so inline answers would contaminate
     the held-out gate and fake the improvement score.
 4. **Re-run the same engine command** — it resumes from the answers
   directory and either finishes or stages the next batch.
 ## Finish
 - `Read` the `report.md` in the staging dir the engine printed and show
  the user: held-out baseline → candidate score, the gate decision, the
  proposed edits, and where the proposal is staged.
 - Tell the user nothing live changed; offer `/skillopt-sleep adopt`.
 - The engine archives `.skillopt-sleep-handoff/` on a completed real run;
  do not delete it yourself.
 ## Safety reminders
 - **Never** edit `CLAUDE.md` or `SKILL.md` yourself — only `adopt` does
  that, with a backup.
 - Mined tasks are pinned to `.skillopt-sleep-handoff/tasks.json` on round
  one, so sessions created while answering prompts cannot shift the task
  set. Do not edit that file.
 - If a batch looks like it contains secrets or content the user would not
  want re-processed, stop and ask before answering.
@@ -0,0 +1,66 @@
 ---
 description: Run or manage the SkillOpt-Sleep self-evolution cycle (review past sessions, replay tasks offline, consolidate validated memory + skills; can also schedule nightly runs)
 argument-hint: "[run | dry-run | status | adopt | harvest | schedule | unschedule] (default: status)"
 allowed-tools: Bash, Read
 ---
 # /skillopt-sleep — SkillOpt-Sleep nightly self-evolution
 You are driving **SkillOpt-Sleep**: a tool that lets this user's Claude agent
 improve offline by reviewing past sessions, replaying recurring tasks, and
 consolidating what it learns into **validated** memory (`CLAUDE.md`) and skills
 (`SKILL.md`). It is gated like SkillOpt: a change is kept only if it improves a
 held-out replay score, and nothing live is modified until the user adopts it.
 ## Requested action: $ARGUMENTS
 (If `$ARGUMENTS` is empty, treat it as `status`.)
 ## How to run it
 The engine is the `skillopt_sleep` Python package in this repo. Use the
 **plugin's bundled runner** so the right interpreter and repo are on the path:
 ```bash
 "${CLAUDE_PLUGIN_ROOT}/scripts/sleep.sh" <action> --project "$(pwd)" --scope invoked
 ```
 `<action>` is one of:
 | action       | what it does |
 |--------------|--------------|
 | `status`     | show how many nights have run + the latest staged proposal (READ-ONLY) |
 | `dry-run`    | harvest → mine → replay → report, but **stage nothing** (safe preview) |
 | `run`        | full cycle: also **stage** a reviewed proposal (still does NOT touch live files) |
 | `adopt`      | apply the latest staged proposal to live `CLAUDE.md` / `SKILL.md` (backs up first) |
 | `harvest`    | debug: print the recurring tasks mined from recent sessions |
 | `schedule`   | install a nightly cron entry for this project (`--hour --minute`, off-:00 by default) |
 | `unschedule` | remove the nightly cron entry (`--all` to remove every managed entry) |
 Default backend is `mock` (deterministic, no API spend). To use real budget for
 genuine improvement, add `--backend claude` or `--backend codex`. To steer what
 the optimizer writes, add `--preferences "<your house rules>"`.
 ## Steps to follow
 1. **Run the requested action** via the bundled runner above. Capture stdout.
 2. **For `run` / `dry-run`:** after it completes, `Read` the generated
   `report.md` in the staging dir it prints, and show the user:
   - held-out score: baseline → candidate (the proof it helped)
   - the gate decision (accept/reject) and the exact edits it proposes
   - where the proposal is staged
 3. **For `run` that produced an accepted proposal:** tell the user the diff is
   staged and that **nothing live changed yet**. Offer to run `/skillopt-sleep adopt`.
 4. **For `adopt`:** confirm which live files were updated and that backups were
   written under the staging dir's `backup/`.
 5. **Never** edit `CLAUDE.md` or `SKILL.md` yourself — only the `adopt` action
   does that, with a backup. Respect the review gate.
 ## Safety reminders
 - Harvest is **read-only** over `~/.claude`. Replay in `mock` mode runs no
  shell side effects.
 - The cycle stages proposals; the user is in control of adoption.
 - If the user asks to schedule this nightly, point them at
  `${CLAUDE_PLUGIN_ROOT}/scripts/install-cron.sh` (prints a crontab line; does
  not install anything without confirmation).
@@ -0,0 +1,16 @@
 {
  "hooks": {
    "SessionEnd": [
      {
        "matcher": "*",
        "hooks": [
          {
            "type": "command",
            "command": "\"${CLAUDE_PLUGIN_ROOT}/hooks/on-session-end.sh\"",
            "async": true
          }
        ]
      }
    ]
  }
 }
@@ -0,0 +1,18 @@
 #!/usr/bin/env bash
 # SkillOpt-Sleep SessionEnd hook (async, best-effort, NON-BLOCKING).
 #
 # This does NOT run the optimizer. It only appends a tiny marker so the next
 # nightly cycle knows there is fresh activity to harvest, and (optionally)
 # nudges the user once that a sleep cycle is available. It must never fail the
 # session or spend API budget.
 set -uo pipefail
 PLUGIN_ROOT="${CLAUDE_PLUGIN_ROOT:-$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)}"
 STATE_DIR="${HOME}/.skillopt-sleep"
 mkdir -p "$STATE_DIR" 2>/dev/null || exit 0
 # Record that a session just ended (cheap; used for "is there new data?").
 printf '%s\t%s\n' "$(date -u +%Y-%m-%dT%H:%M:%SZ)" "${PWD}" \
  >> "$STATE_DIR/session-end.log" 2>/dev/null || true
 exit 0
@@ -0,0 +1,29 @@
 #!/usr/bin/env bash
 # Print (does NOT install) a crontab line that runs SkillOpt-Sleep nightly.
 # The user copies the line into `crontab -e` if they want it.
 set -euo pipefail
 PLUGIN_ROOT="${CLAUDE_PLUGIN_ROOT:-$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)}"
 RUNNER="$PLUGIN_ROOT/scripts/sleep.sh"
 PROJECT="${1:-$(pwd)}"
 BACKEND="${2:-mock}"
 # 3:17am local — deliberately off the :00 mark so many users don't all hit the
 # API at once (and we leave room for jitter).
 MIN=17
 HOUR=3
 cat <<EOF
 # ── SkillOpt-Sleep nightly cycle ────────────────────────────────────────────
 # Review past sessions, replay tasks, stage validated memory/skill updates.
 # Runs at ${HOUR}:$(printf '%02d' $MIN) local every day. Output goes to the project's
 # .skillopt-sleep/ dir; nothing live is changed until you run '/skillopt-sleep adopt'
 # (unless you pass --auto-adopt below).
 #
 # Copy the next line into 'crontab -e':
 ${MIN} ${HOUR} * * *  "${RUNNER}" run --project "${PROJECT}" --scope invoked --backend ${BACKEND} >> "${PROJECT}/.skillopt-sleep/cron.log" 2>&1
 #
 # For fully-autonomous adoption (power users), append: --auto-adopt
 # To spend real API budget for genuine lift, set BACKEND=anthropic above.
 # ────────────────────────────────────────────────────────────────────────────
 EOF
@@ -0,0 +1,51 @@
 #!/usr/bin/env bash
 # SkillOpt-Sleep shared runner — used by all platform plugins (Claude Code,
 # Codex, Copilot). Resolves the repo root (which contains the skillopt_sleep
 # package), picks a Python >= 3.10, and execs the engine CLI.
 #
 # Usage: run-sleep.sh <run|dry-run|status|adopt|harvest|...> [args...]
 set -euo pipefail
 # This script lives at <repo>/plugins/run-sleep.sh, so the repo root (which
 # holds skillopt_sleep/) is one level up. CLAUDE_PLUGIN_ROOT (if set by Claude
 # Code) points at the plugin dir; the engine is then two levels above it.
 SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
 if [ -d "$SCRIPT_DIR/../skillopt_sleep" ]; then
  REPO_ROOT="$(cd "$SCRIPT_DIR/.." && pwd)"
 elif [ -n "${CLAUDE_PLUGIN_ROOT:-}" ] && [ -d "$CLAUDE_PLUGIN_ROOT/../../skillopt_sleep" ]; then
  REPO_ROOT="$(cd "$CLAUDE_PLUGIN_ROOT/../.." && pwd)"
 elif [ -n "${SKILLOPT_SLEEP_REPO:-}" ] && [ -d "$SKILLOPT_SLEEP_REPO/skillopt_sleep" ]; then
  REPO_ROOT="$SKILLOPT_SLEEP_REPO"
 else
  # last resort: search upward from CWD
  d="$PWD"
  while [ "$d" != "/" ]; do
    [ -d "$d/skillopt_sleep" ] && { REPO_ROOT="$d"; break; }
    d="$(dirname "$d")"
  done
 fi
 if [ -z "${REPO_ROOT:-}" ]; then
  echo "[sleep] ERROR: could not locate the skillopt_sleep package. Set SKILLOPT_SLEEP_REPO to the repo root." >&2
  exit 1
 fi
 PY=""
 # Allow explicit Python override (useful on macOS with old system Python).
 if [ -n "${SKILLOPT_SLEEP_PYTHON:-}" ]; then
  PY="$SKILLOPT_SLEEP_PYTHON"
 else
  for cand in python3.12 python3.11 python3.10 python3; do
    if command -v "$cand" >/dev/null 2>&1; then
      ver="$("$cand" -c 'import sys; print("%d%d" % sys.version_info[:2])' 2>/dev/null || echo 0)"
      if [ "${ver:-0}" -ge 310 ]; then PY="$cand"; break; fi
    fi
  done
 fi
 if [ -z "$PY" ]; then
  echo "[sleep] ERROR: need Python >= 3.10 (found none)." >&2
  exit 1
 fi
 if [ "$#" -eq 0 ]; then set -- status; fi
 cd "$REPO_ROOT"
 exec "$PY" -m skillopt_sleep "$@"
@@ -0,0 +1,30 @@
 #!/usr/bin/env bash
 # Claude Code plugin runner — thin wrapper over the shared runner so all
 # platform plugins share one engine launcher.
 #
 # After marketplace install the plugin is isolated in a cache directory and
 # the repo-relative path no longer works.  We try four locations:
 #   1. Co-located run-sleep.sh (bundled copy — works in marketplace cache)
 #   2. Repo-relative ../../run-sleep.sh (dev checkout)
 #   3. CLAUDE_PLUGIN_ROOT/../run-sleep.sh (plugin env variable)
 #   4. SKILLOPT_SLEEP_REPO/plugins/run-sleep.sh (explicit env)
 set -euo pipefail
 HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
 SHARED=""
 if [ -f "$HERE/run-sleep.sh" ]; then
  SHARED="$HERE/run-sleep.sh"
 elif [ -f "$(cd "$HERE/../.." 2>/dev/null && pwd)/run-sleep.sh" ]; then
  SHARED="$(cd "$HERE/../.." && pwd)/run-sleep.sh"
 elif [ -n "${CLAUDE_PLUGIN_ROOT:-}" ] && [ -f "$(cd "$CLAUDE_PLUGIN_ROOT/.." 2>/dev/null && pwd)/run-sleep.sh" ]; then
  SHARED="$(cd "$CLAUDE_PLUGIN_ROOT/.." && pwd)/run-sleep.sh"
 elif [ -n "${SKILLOPT_SLEEP_REPO:-}" ] && [ -f "$SKILLOPT_SLEEP_REPO/plugins/run-sleep.sh" ]; then
  SHARED="$SKILLOPT_SLEEP_REPO/plugins/run-sleep.sh"
 fi
 if [ -z "$SHARED" ]; then
  echo "[sleep] ERROR: cannot locate run-sleep.sh." >&2
  echo "[sleep] Set SKILLOPT_SLEEP_REPO to the SkillOpt repo root, or pip install skillopt." >&2
  exit 1
 fi
 exec bash "$SHARED" "$@"
@@ -0,0 +1,126 @@
 ---
 name: skillopt-sleep
 description: "Use when the user wants their Claude agent to self-improve from past usage, asks about a nightly/offline 'sleep' or 'dream' cycle, memory/skill consolidation, or says things like 'make my agent better the more I use it', 'review my past sessions', 'learn my preferences', 'consolidate what you learned', 'run the sleep cycle', or wants to schedule offline self-optimization. Drives the skillopt_sleep engine: harvest past sessions -> mine recurring tasks -> replay offline -> consolidate validated CLAUDE.md/SKILL.md behind a held-out gate."
 ---
 # SkillOpt-Sleep: offline self-evolution for a local Claude agent
 SkillOpt-Sleep gives the user's agent a **sleep cycle**. While the user is
 offline (e.g. nightly), it reviews their real past Claude Code sessions,
 re-runs recurring tasks on their own API budget, and consolidates what it
 learns into **memory** (`CLAUDE.md`) and **skills** (`SKILL.md`) — but only
 keeps changes that pass a held-out validation gate, and only after the user
 adopts them. The agent gets measurably better at *this* user's recurring work,
 with no model-weight training. It is the deployment-time analogue of training:
 short-term experience → long-term competence.
 It synthesizes three ideas:
 - **SkillOpt** — the skill/memory doc is trainable text; bounded add/delete/replace
  edits; accepted only through a held-out gate; rejected edits become negative feedback.
 - **Claude Dreams** — offline consolidation that reads past sessions and rebuilds
  memory (dedup/merge/resolve); the input is never mutated; output is reviewed then adopted.
 - **Agent sleep** — periodic offline replay turns episodes into durable skill.
 ## When to use this skill
 Trigger when the user wants any of:
 - "make my agent learn from how I use it" / "get better the more I use it" / "remember my preferences across sessions"
 - a nightly/scheduled or on-demand **offline self-improvement / dream / sleep** run
 - to **review past sessions/trajectories** and distill recurring tasks
 - to **consolidate** feedback into `CLAUDE.md` or a managed skill
 - to **schedule** the cycle (cron) or **adopt** a staged proposal
 ## The cycle (six stages)
 1. **Harvest** — read `~/.claude/projects/*/<session>.jsonl` + `~/.claude/history.jsonl` (READ-ONLY) → session digests.
 2. **Mine** — digests → `TaskRecord`s (recurring intents + outcome labels + checkable refs where possible).
 3. **Replay** — re-run tasks offline under the *current* skill+memory → (hard, soft) scores.
 4. **Consolidate** — reflect on failures → propose bounded edits → **gate** on a held-out slice; accept only if it strictly improves.
 5. **Stage** — write `proposed_CLAUDE.md`, `proposed_SKILL.md`, a diff, and `report.md` into `<project>/.skillopt-sleep/staging/<date>/`. **Nothing live changes.**
 6. **Adopt** — explicit (or opt-in auto): copy staged files over live ones, backing up first.
 ## How to drive it
 Prefer the `/skillopt-sleep` command. Under the hood it calls the bundled runner:
 ```bash
 "${CLAUDE_PLUGIN_ROOT}/scripts/sleep.sh" status                       # what's happened
 "${CLAUDE_PLUGIN_ROOT}/scripts/sleep.sh" dry-run --project "$(pwd)"    # safe preview
 "${CLAUDE_PLUGIN_ROOT}/scripts/sleep.sh" run --project "$(pwd)"        # full cycle, stages a proposal
 "${CLAUDE_PLUGIN_ROOT}/scripts/sleep.sh" adopt --project "$(pwd)"      # apply staged proposal (with backup)
 ```
 - Default backend is `mock` (deterministic, **no API spend**) — good for trying the plumbing.
 - Add `--backend claude` or `--backend codex` to spend the user's real budget for genuine improvement.
 - Scope defaults to the invoked project; `--scope all` harvests every project.
 ### Scheduling
 ```bash
 "${CLAUDE_PLUGIN_ROOT}/scripts/sleep.sh" schedule --project "$(pwd)" --hour 3 --minute 17
 "${CLAUDE_PLUGIN_ROOT}/scripts/sleep.sh" unschedule --project "$(pwd)"
 ```
 Installs a nightly cron entry. `unschedule --all` removes every managed entry.
 ## All CLI flags
 | Flag | Default | Description |
 |------|---------|-------------|
 | `--project PATH` | cwd | Project directory to evolve |
 | `--scope all\|invoked` | invoked | Harvest scope |
 | `--backend mock\|claude\|codex\|copilot` | mock | Replay backend (mock = no API spend) |
 | `--model NAME` | backend default | Override the model used for replay |
 | `--source claude\|codex\|auto` | claude | Transcript source |
 | `--lookback-hours N` | 72 | Harvest window |
 | `--max-sessions N` | unlimited | Cap harvested sessions |
 | `--max-tasks N` | 40 | Cap mined tasks |
 | `--target-skill-path PATH` | auto | Explicit SKILL.md to evolve |
 | `--tasks-file PATH` | — | Reviewed TaskRecord JSON (skip harvest) |
 | `--progress` | off | Print phase progress to stderr |
 | `--auto-adopt` | off | Auto-adopt if gate passes |
 | `--edit-budget N` | 4 | Max bounded edits per night |
 | `--json` | off | Machine-readable JSON output |
 ## Config keys (`~/.skillopt-sleep/config.json`)
 Beyond the CLI flags, advanced behavior is controlled via config:
 - **`preferences`** — free-text house rules injected into the optimizer's reflect step (e.g. "Always use async/await", "Answers in `\boxed{}`").
 - **`gate_mode`** — `on` (default, validation-gated) or `off` (greedy, accept all edits).
 - **`gate_metric`** — `hard`, `soft`, or `mixed` (default). Controls how the held-out gate scores.
 - **`dream_rollouts`** — >1 enables multi-rollout contrastive reflection per task.
 - **`recall_k`** — >0 recalls K similar past tasks into the dream (long-term memory).
 - **`evolve_memory`** / **`evolve_skill`** — independently toggle CLAUDE.md vs SKILL.md consolidation.
 ## Memory consolidation
 The sleep cycle can consolidate both:
 - **SKILL.md** — the managed skill file (bounded edits: add/delete/replace)
 - **CLAUDE.md** — the project memory (same bounded edits)
 Both are gated by the same held-out validation score. Set `evolve_memory: false` to consolidate only skills, or `evolve_skill: false` for only memory.
 ## Hard rules
 - **Never** hand-edit the user's `CLAUDE.md` / `SKILL.md` as part of this skill.
  Only the `adopt` action changes live files, and it backs them up first.
 - Harvest is read-only. `mock` replay has no side effects.
 - Always show the user the **held-out baseline → candidate** score and the
  exact proposed edits before suggesting adoption. Evidence before adoption.
 - If asked whether it really helps, run
  `python -m skillopt_sleep.experiments.run_experiment --persona researcher --json`
  — a deterministic demo that proves held-out lift and that the gate blocks
  harmful edits.
 ## Validate / demo
 ```bash
 # deterministic proof (no API): held-out score rises, gate blocks regressions
 python -m skillopt_sleep.experiments.run_experiment --persona researcher --assert-improves
 python -m skillopt_sleep.experiments.run_experiment --persona programmer  --assert-improves
 ```
 See the SkillOpt-Sleep guide section for recorded output and
 `docs/superpowers/specs/2026-06-07-skillopt-sleep-claude-code-plugin-design.md`
 for the full design.
@@ -0,0 +1,96 @@
 # SkillOpt-Sleep — Codex integration
 Give your **Codex** agent a nightly **sleep cycle**: it reviews past sessions
 offline, replays your recurring tasks on your own Codex budget, and consolidates
 what it learns into validated memory + skills behind a held-out gate. Same engine
 as the Claude Code plugin (`skillopt_sleep`), wrapped for Codex.
 > **Verified on Codex:** on the public
 > [gbrain-evals](https://github.com/garrytan/gbrain-evals) `skillopt-v1`
 > benchmark, a deliberately deficient skill goes **0.00 → 1.00** on a held-out
 > set with the Codex backend (incl. the tool-use seed via a real tool loop).
 > See [the SkillOpt-Sleep guide section](https://microsoft.github.io/SkillOpt/docs/guideline.html#sleep).
 ## What Codex supports (and what we use)
 Codex (`@openai/codex`) extends via **`AGENTS.md`** instructions, **skills** at
 `~/.agents/skills/<name>/SKILL.md`, and plugins that can distribute skills.
 Custom prompts are deprecated in Codex, so this integration is skill-first: the
 installed `skillopt-sleep` skill contains the launch commands and operating
 rules. The shared runner remains a plain shell entrypoint that the skill calls.
 ## Install
 ```bash
 git clone <repo-url> SkillOpt-Sleep
 cd SkillOpt-Sleep
 bash plugins/codex/install.sh          # installs the skill
 export SKILLOPT_SLEEP_REPO="$(pwd)"    # so the runner is found from anywhere
 ```
 If a previous install created `~/.codex/prompts/sleep.md`, the installer moves
 that deprecated prompt aside with a `.skillopt-legacy*.bak` suffix.
 Requires Python ≥ 3.10 and the `codex` CLI on PATH.
 ## Use
 Mention `$skillopt-sleep` where Codex supports explicit skill mentions, or ask
 Codex in natural language:
 ```text
 Use the skillopt-sleep skill to run status for this project.
 Use the skillopt-sleep skill to run a dry-run for this project.
 Use the skillopt-sleep skill to run the full cycle for this project with the Codex backend.
 Use the skillopt-sleep skill to adopt the latest staged proposal.
 ```
 Or call the engine directly:
 ```bash
 python -m skillopt_sleep dry-run --project "$(pwd)" --source codex --backend mock
 python -m skillopt_sleep run --project "$(pwd)" --source codex --backend codex \
  --max-sessions 5 --max-tasks 3 --progress
 python -m skillopt_sleep run --project "$(pwd)" --source codex --backend codex \
  --target-skill-path .agents/skills/example/SKILL.md \
  --max-sessions 5 --max-tasks 3 --progress
 ```
 `--source codex` reads Codex Desktop archived sessions from
 `~/.codex/archived_sessions`. Use `--codex-home /path/to/.codex` to point at a
 different Codex home, or `--source auto` to try Codex archives first and fall
 back to Claude Code transcripts. Default backend is `mock` (no API spend).
 `--backend codex` uses your Codex budget for real improvement. Bound live runs
 with `--max-sessions` and `--max-tasks`; add `--progress` because Codex-backed
 mining, replay, and reflection can be slow and otherwise quiet. Use
 `--target-skill-path` to stage/adopt into a repo-scoped Codex skill such as
 `.agents/skills/<name>/SKILL.md`; target runs over-sample mined tasks and
 prefer tasks that match the target skill's path, headings, and content. All the
 controllable knobs (`--gate on|off`, `--rollouts-k`, `--budget-tokens`,
 `--preferences`, optimizer/target split) work identically — see
 [the SkillOpt-Sleep guide section](https://microsoft.github.io/SkillOpt/docs/guideline.html#sleep).
 For privacy-sensitive projects, split the run into reviewable steps:
 ```bash
 python -m skillopt_sleep harvest --project "$(pwd)" --source codex \
  --target-skill-path .agents/skills/example/SKILL.md \
  --max-sessions 5 --max-tasks 3 \
  --output reviewed-tasks.json
 python -m skillopt_sleep dry-run --project "$(pwd)" --backend codex \
  --tasks-file reviewed-tasks.json --progress --json
 ```
 Inspect/redact the JSON and set `"reviewed": true` before using a real backend.
 `--tasks-file` skips archive harvest/mining and replays only the reviewed JSON
 tasks; real backends refuse task files still marked `"reviewed": false`.
 ## Notes / status
 - Codex's `exec` runs shell, so the real-tool-loop replay (e.g. the
  `tool_called: search` benchmark seed) works natively.
 - This integration no longer installs a `.codex/prompts` slash command. Skills
  are the reusable Codex workflow surface; mention `skillopt-sleep` explicitly
  or ask for a sleep/dream/offline self-improvement run and Codex can load the
  skill.
@@ -0,0 +1,44 @@
 #!/usr/bin/env bash
 # Install the SkillOpt-Sleep Codex integration as a user-level Codex skill.
 # Idempotent; prints what it does.
 set -euo pipefail
 REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
 CODEX_HOME="${CODEX_HOME:-$HOME/.codex}"
 AGENTS_SKILLS="${HOME}/.agents/skills"
 LEGACY_PROMPT="$CODEX_HOME/prompts/sleep.md"
 echo "[install] repo: $REPO_ROOT"
 # 1) user-level skill
 mkdir -p "$AGENTS_SKILLS/skillopt-sleep"
 cp "$REPO_ROOT/plugins/codex/skills/skillopt-sleep/SKILL.md" "$AGENTS_SKILLS/skillopt-sleep/SKILL.md"
 echo "[install] skill           -> $AGENTS_SKILLS/skillopt-sleep/SKILL.md"
 # 2) retire the old custom prompt entrypoint from previous installs
 if [ -f "$LEGACY_PROMPT" ]; then
  backup="${LEGACY_PROMPT}.skillopt-legacy.bak"
  if [ -e "$backup" ]; then
    backup="${LEGACY_PROMPT}.skillopt-legacy.$(date +%Y%m%d%H%M%S).bak"
  fi
  mv "$LEGACY_PROMPT" "$backup"
  echo "[install] legacy prompt  -> $backup"
 fi
 # 3) record the repo location so the runner is found from anywhere
 echo "[install] add to your shell profile:"
 echo "    export SKILLOPT_SLEEP_REPO=\"$REPO_ROOT\""
 # 4) optional: append an AGENTS.md hint (only if the user opts in)
 cat <<EOF
 [install] Optional — add this to ~/.codex/AGENTS.md so Codex always knows the tool:
  ## SkillOpt-Sleep
  Use the skillopt-sleep skill when I ask to run a sleep/dream/offline
  self-improvement cycle. The runner is:
  \`bash "$REPO_ROOT/plugins/run-sleep.sh" status --project "\$(pwd)"\`.
 Done. Try asking Codex:
  Use the skillopt-sleep skill to run status for this project.
 EOF
@@ -0,0 +1,132 @@
 ---
 name: skillopt-sleep
 description: "Use when the user wants Codex to self-improve from past usage, asks about a nightly/offline 'sleep' or 'dream' cycle, wants Codex to review past sessions, learn preferences, consolidate memory/skills, run dry-run/run/adopt/status for SkillOpt-Sleep, or schedule offline self-optimization. Drives the skillopt_sleep engine: harvest past sessions -> mine recurring tasks -> replay offline -> consolidate validated memory + skills behind a held-out gate."
 ---
 # SkillOpt-Sleep: offline self-evolution for a local Codex agent
 SkillOpt-Sleep gives the user's Codex agent a sleep cycle. While the user is
 offline or on demand, it reviews past local sessions, re-runs recurring tasks
 on the user's own budget, and consolidates what it learns into memory and
 skills. It keeps only changes that pass a held-out validation gate, and live
 files change only after the user explicitly adopts a staged proposal. There is
 no model-weight training.
 ## When to use
 Trigger when the user wants any of:
 - Codex to learn from past sessions or get better the more they use it;
 - a nightly/scheduled or on-demand sleep/dream/offline self-improvement run;
 - to review past sessions and distill recurring tasks;
 - to consolidate feedback into memory or managed skills;
 - to run `status`, `harvest`, `dry-run`, `run`, or `adopt` for SkillOpt-Sleep.
 ## The cycle
 1. **Harvest** - read local session transcripts according to the engine
   configuration and normalize them into session digests.
 2. **Mine** - turn digests into recurring `TaskRecord`s with outcomes and
   checkable references where possible.
 3. **Replay** - re-run mined tasks offline under the current skill and memory.
 4. **Consolidate** - reflect on failures and propose bounded edits.
 5. **Gate** - accept edits only when the held-out validation score improves.
 6. **Stage** - write the proposal under
   `<project>/.skillopt-sleep/staging/<date>/`; nothing live changes.
 7. **Adopt** - only after explicit user approval, copy staged files over live
   files with backups.
 ## How to drive it
 Invoke the bundled runner via shell (Codex `exec` has shell access). The runner
 finds the engine and a Python >= 3.10 automatically.
 ```bash
 # point at the repo if it isn't auto-detected from CWD:
 export SKILLOPT_SLEEP_REPO=/path/to/SkillOpt-Sleep
 bash "$SKILLOPT_SLEEP_REPO/plugins/run-sleep.sh" status --project "$(pwd)"
 bash "$SKILLOPT_SLEEP_REPO/plugins/run-sleep.sh" harvest --project "$(pwd)"
 bash "$SKILLOPT_SLEEP_REPO/plugins/run-sleep.sh" dry-run --project "$(pwd)" --backend mock
 bash "$SKILLOPT_SLEEP_REPO/plugins/run-sleep.sh" run --project "$(pwd)" --backend codex
 bash "$SKILLOPT_SLEEP_REPO/plugins/run-sleep.sh" run --project "$(pwd)" --source codex  # harvest from Codex Desktop
 bash "$SKILLOPT_SLEEP_REPO/plugins/run-sleep.sh" adopt --project "$(pwd)"
 ```
 Actions are `status`, `harvest`, `dry-run`, `run`, `adopt`, `schedule`, and `unschedule`.
 - Default backend is `mock`, which is deterministic and spends no API budget.
 - `--backend codex` uses the user's Codex budget for real improvement.
 - `--source codex` reads Codex Desktop archived sessions from `~/.codex/archived_sessions`;
  use `--codex-home /path/to/.codex` if the archive lives elsewhere.
 - Keep `dry-run --backend mock` as the first smoke check unless the user
  explicitly asked for a real optimization run.
 ### Scheduling
 ```bash
 bash "$SKILLOPT_SLEEP_REPO/plugins/run-sleep.sh" schedule --project "$(pwd)" --hour 3 --minute 17
 bash "$SKILLOPT_SLEEP_REPO/plugins/run-sleep.sh" unschedule --project "$(pwd)"
 ```
 Installs a nightly cron entry. `unschedule --all` removes every managed entry.
 ### All backends
 - `--backend mock` — deterministic, no API spend (default)
 - `--backend claude` — uses the Claude CLI
 - `--backend codex` — uses the Codex CLI
 - `--backend copilot` — uses the GitHub Copilot CLI
 ### Additional flags
 | Flag | Description |
 |------|-------------|
 | `--auto-adopt` | Auto-adopt if the gate passes (default: stage only) |
 | `--edit-budget N` | Max bounded edits per night (default: 4) |
 | `--lookback-hours N` | Harvest window in hours (default: 72) |
 | `--json` | Machine-readable JSON output |
 ### Config keys (`~/.skillopt-sleep/config.json`)
 - **`preferences`** — free-text house rules for the optimizer
 - **`gate_mode`** — `on` (validation-gated, default) or `off` (greedy)
 - **`gate_metric`** — `hard` | `soft` | `mixed` (default)
 - **`dream_rollouts`** — >1 for multi-rollout contrastive reflection
 - **`recall_k`** — >0 recalls similar past tasks from the archive
 ### Memory consolidation
 The sleep cycle consolidates both **memory** (AGENTS.md / CLAUDE.md) and **skills** (SKILL.md) by default. Each is independently toggleable via `evolve_memory` / `evolve_skill` config keys. Both are gated by the same held-out validation score.
 ## Steps
 1. Run the requested action; capture stdout.
 2. For `dry-run` and `run`, report the held-out baseline -> candidate score,
   gate action, task count, session count, and exact proposed edits.
 3. If a staging directory is printed, read `report.md` before summarizing.
 4. `run` only stages a proposal; nothing live changes until `adopt`.
 5. Offer adoption only after the user has reviewed the staged proposal.
 6. Never hand-edit the user's `AGENTS.md`, memory, or skills as a substitute
   for `adopt`; adoption is the safety boundary and writes backups first.
 ## Hard rules
 - Harvest is read-only. Do not edit archived sessions or raw transcripts.
 - Keep raw secrets, credentials, private user data, and unsanitized transcript
  contents out of messages, logs, generated artifacts, and commits.
 - Show validation evidence before recommending adoption.
 - Treat generated edits as proposals, not as source of truth.
 - Do not rely on deprecated custom prompts or `/sleep` slash commands for this
  Codex integration. This skill is the entrypoint.
 ## Validate
 ```bash
 python -m skillopt_sleep dry-run --project "$(pwd)" --backend mock --json
 python -m skillopt_sleep.experiments.run_gbrain --backend codex \
  --seeds brief-writer --data-root /path/to/gbrain-evals/eval/data/skillopt-v1 \
  --nights 2 --limit-replay 3 --limit-holdout 3
 ```
 A deficient skill goes 0.00 -> 1.00 on a held-out set; the optimizer's edits
 are gated on real-task performance.
@@ -0,0 +1,76 @@
 # SkillOpt-Sleep — GitHub Copilot integration
 Give **Copilot** (CLI or VS Code) a nightly **sleep cycle** via a tiny **MCP
 server** that exposes the `skillopt_sleep` engine as tools. MCP is GitHub's
 supported way to extend Copilot, so this works across Copilot CLI, VS Code, and
 other MCP clients with the same server.
 ## What's here
 | File | Purpose |
 |---|---|
 | `mcp_server.py` | stdlib-only MCP (stdio) server exposing `sleep_*` tools |
 | `mcp-config.example.json` | drop-in MCP server config |
 | `copilot-instructions.snippet.md` | paste into `.github/copilot-instructions.md` |
 ## Install
 Requires Python ≥ 3.10. No third-party packages — the server is pure stdlib.
 1. **Register the MCP server.** Add the server to your Copilot MCP config
   (Copilot CLI: `~/.copilot/mcp-config.json`; VS Code: your MCP settings).
   Use `mcp-config.example.json` as a template — set `SKILLOPT_SLEEP_REPO` to
   this repo's path:
   ```json
   {
     "mcpServers": {
       "skillopt-sleep": {
         "command": "python3",
         "args": ["/abs/path/SkillOpt-Sleep/plugins/copilot/mcp_server.py"],
         "env": { "SKILLOPT_SLEEP_REPO": "/abs/path/SkillOpt-Sleep" }
       }
     }
   }
   ```
 2. **(Optional) Tell Copilot about it.** Append
   `copilot-instructions.snippet.md` to your repo's
   `.github/copilot-instructions.md` so Copilot reaches for the tools when the
   user asks to "run the sleep cycle".
 ## Use
 Ask Copilot things like *"run the sleep cycle"*, *"what did the last sleep
 propose?"*, *"adopt the staged sleep proposal"*. Copilot calls the MCP tools:
 `sleep_status`, `sleep_dry_run`, `sleep_run`, `sleep_adopt`, `sleep_harvest`.
 Each tool takes optional `project`, `backend` (`mock`/`claude`/`codex`/`copilot`), and
 `scope` arguments. Default backend is `mock` (no API spend). The `copilot`
 backend drives the GitHub Copilot CLI (`copilot -p ... --output-format json`)
 and requires the `copilot` CLI to be installed and authenticated.
 For speed, the `copilot` backend runs each call against an isolated
 `COPILOT_HOME` with built-in MCP servers and custom instructions disabled, so
 your user MCP servers (including this project's own) are not spawned per call
 (~5x faster). Override with `SKILLOPT_SLEEP_COPILOT_HOME=<dir>`, pick a model
 with `SKILLOPT_SLEEP_COPILOT_MODEL`, or set `SKILLOPT_SLEEP_COPILOT_FULL_ENV=1`
 to use your real Copilot environment instead.
 ## Verify the server directly (no Copilot needed)
 ```bash
 printf '%s\n' \
  '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{}}' \
  '{"jsonrpc":"2.0","id":2,"method":"tools/list"}' \
  | SKILLOPT_SLEEP_REPO="$(pwd)" python3 plugins/copilot/mcp_server.py
 ```
 You should see the server info and the five `sleep_*` tools.
 ## Notes / status
 - MCP is the stable, official Copilot extension surface, so this is the most
  portable of the three integrations (one server → CLI + IDE).
 - The engine and all its controls (gate on/off, multi-rollout, budget,
  preferences, optimizer/target split) are identical across platforms — see
  [the SkillOpt-Sleep guide section](https://microsoft.github.io/SkillOpt/docs/guideline.html#sleep).
@@ -0,0 +1,43 @@
 <!--
 Copy this block into your repo's .github/copilot-instructions.md so Copilot
 knows the SkillOpt-Sleep tools exist. (Copilot reads copilot-instructions.md
 automatically as ambient guidance.)
 -->
 ## SkillOpt-Sleep (offline self-evolution)
 This project has SkillOpt-Sleep available via an MCP server (`skillopt-sleep`).
 It gives the agent a nightly "sleep cycle": it reviews past sessions, replays
 recurring tasks offline, and consolidates validated memory + skills behind a
 held-out gate.
 When the user asks to "run the sleep cycle", "review my past sessions", "learn
 my preferences", or "make the agent improve from past usage", use the MCP tools:
 - `sleep_status` — what's happened + the latest staged proposal
 - `sleep_dry_run` — safe preview, stages nothing
 - `sleep_run` — full cycle, stages a reviewed proposal (nothing live changes)
 - `sleep_adopt` — apply the staged proposal (backs up first)
 - `sleep_harvest` — list mined recurring tasks
 - `sleep_schedule` — install a nightly cron entry (set `hour`/`minute`)
 - `sleep_unschedule` — remove the nightly cron entry
 ### Key parameters (pass as MCP tool arguments)
 - `backend` — `mock` (default, free), `claude`, `codex`, or `copilot`
 - `source` — `claude`, `codex`, or `auto` (where to read transcripts)
 - `target_skill_path` — explicit SKILL.md to evolve
 - `tasks_file` — pre-built TaskRecord JSON (skip harvest)
 - `max_tasks` / `max_sessions` — cap workload
 - `auto_adopt` — auto-adopt if the gate passes
 - `json` — machine-readable output for programmatic use
 ### Advanced config (`~/.skillopt-sleep/config.json`)
 - `preferences` — free-text house rules for the optimizer
 - `gate_mode` — `on` (default) or `off`; `dream_rollouts` — >1 for more signal
 - `evolve_memory` / `evolve_skill` — toggle which docs consolidate
 Always show the user the held-out baseline → candidate score and the proposed
 edits before suggesting `sleep_adopt`. Never hand-edit the user's memory/skill
 files; only `sleep_adopt` does that, with a backup.
@@ -0,0 +1,11 @@
 {
  "mcpServers": {
    "skillopt-sleep": {
      "command": "python3",
      "args": ["plugins/copilot/mcp_server.py"],
      "env": {
        "SKILLOPT_SLEEP_REPO": "${workspaceFolder}"
      }
    }
  }
 }
@@ -0,0 +1,180 @@
 #!/usr/bin/env python3
 """SkillOpt-Sleep — minimal MCP server (stdio, stdlib-only).
 Exposes the sleep engine as MCP tools so any MCP-capable client (GitHub Copilot
 CLI / VS Code, Claude Desktop, etc.) can drive it. No third-party deps: speaks
 JSON-RPC 2.0 over stdio with just the handful of MCP methods clients need.
 Tools exposed:
  - sleep_status   : how many nights have run + the latest staged proposal
  - sleep_dry_run  : harvest+mine+replay, report only (no staging)
  - sleep_run      : full cycle, stages a proposal (nothing live changes)
  - sleep_adopt    : apply the latest staged proposal (with backup)
  - sleep_harvest  : debug — list mined recurring tasks
 Each tool shells out to `python -m skillopt_sleep <action> ...` and returns its
 stdout. Configure your client to launch:  python plugins/copilot/mcp_server.py
 """
 from __future__ import annotations
 import json
 import os
 import subprocess
 import sys
 REPO_ROOT = os.environ.get("SKILLOPT_SLEEP_REPO") or os.path.abspath(
    os.path.join(os.path.dirname(__file__), "..", "..")
 )
 PROTOCOL_VERSION = "2024-11-05"
 TOOLS = [
    {"name": "sleep_status", "action": "status",
     "description": "Show how many SkillOpt-Sleep nights have run and the latest staged proposal."},
    {"name": "sleep_dry_run", "action": "dry-run",
     "description": "Preview a sleep cycle (harvest+mine+replay) without staging anything."},
    {"name": "sleep_run", "action": "run",
     "description": "Run a full sleep cycle; stages a reviewed proposal. Nothing live changes until adopt."},
    {"name": "sleep_adopt", "action": "adopt",
     "description": "Apply the latest staged proposal to CLAUDE.md/SKILL.md (backs up first)."},
    {"name": "sleep_harvest", "action": "harvest",
     "description": "Debug: list the recurring tasks mined from recent sessions."},
    {"name": "sleep_schedule", "action": "schedule",
     "description": "Install a nightly cron entry to run the sleep cycle automatically."},
    {"name": "sleep_unschedule", "action": "unschedule",
     "description": "Remove the nightly cron entry for a project."},
 ]
 _BY_NAME = {t["name"]: t for t in TOOLS}
 _TOOL_SCHEMA = {
    "type": "object",
    "properties": {
        "project": {"type": "string",
                     "description": "Project dir to evolve (default: cwd)."},
        "backend": {"type": "string", "enum": ["mock", "claude", "codex", "copilot"],
                     "description": "mock = no API spend (default); claude/codex/copilot = real."},
        "scope": {"type": "string", "enum": ["invoked", "all"],
                  "description": "Harvest scope (default: invoked project only)."},
        "source": {"type": "string", "enum": ["claude", "codex", "auto"],
                   "description": "Transcript source (default: claude)."},
        "model": {"type": "string",
                  "description": "Backend-specific model override."},
        "tasks_file": {"type": "string",
                       "description": "Path to reviewed TaskRecord JSON (skips harvest)."},
        "target_skill_path": {"type": "string",
                              "description": "Explicit SKILL.md path to evolve/stage/adopt."},
        "progress": {"type": "boolean",
                     "description": "Print phase progress to stderr."},
        "max_sessions": {"type": "integer",
                         "description": "Cap harvested sessions per run."},
        "max_tasks": {"type": "integer",
                      "description": "Cap mined tasks per run."},
        "lookback_hours": {"type": "integer",
                           "description": "Harvest window in hours (default: 72)."},
        "auto_adopt": {"type": "boolean",
                       "description": "Auto-adopt if gate passes (default: false)."},
        "json": {"type": "boolean",
                 "description": "Return machine-readable JSON output."},
        "edit_budget": {"type": "integer",
                        "description": "Max bounded edits per night (default: 4)."},
        "hour": {"type": "integer",
                 "description": "Hour for schedule (0-23, default: 3)."},
        "minute": {"type": "integer",
                   "description": "Minute for schedule (0-59, default: 17)."},
    },
    "additionalProperties": False,
 }
 def _run_engine(action: str, args: dict) -> str:
    py = sys.executable or "python3"
    cmd = [py, "-m", "skillopt_sleep", action]
    # String-valued flags
    for flag, key in [
        ("--project", "project"), ("--backend", "backend"),
        ("--scope", "scope"), ("--source", "source"),
        ("--model", "model"), ("--tasks-file", "tasks_file"),
        ("--target-skill-path", "target_skill_path"),
    ]:
        val = args.get(key)
        if val:
            cmd += [flag, str(val)]
    # Integer-valued flags
    for flag, key in [
        ("--max-sessions", "max_sessions"), ("--max-tasks", "max_tasks"),
        ("--lookback-hours", "lookback_hours"), ("--edit-budget", "edit_budget"),
        ("--hour", "hour"), ("--minute", "minute"),
    ]:
        val = args.get(key)
        if val is not None:
            cmd += [flag, str(int(val))]
    # Boolean flags
    for flag, key in [
        ("--progress", "progress"), ("--auto-adopt", "auto_adopt"),
        ("--json", "json"),
    ]:
        if args.get(key):
            cmd.append(flag)
    try:
        proc = subprocess.run(cmd, cwd=REPO_ROOT, capture_output=True, text=True, timeout=3600)
    except Exception as e:
        return f"[error] failed to run engine: {e}"
    out = (proc.stdout or "").strip()
    err = (proc.stderr or "").strip()
    return out + (("\n[stderr]\n" + err) if err else "")
 def _result(id_, result):
    return {"jsonrpc": "2.0", "id": id_, "result": result}
 def _error(id_, code, message):
    return {"jsonrpc": "2.0", "id": id_, "error": {"code": code, "message": message}}
 def handle(req: dict):
    method = req.get("method")
    id_ = req.get("id")
    if method == "initialize":
        return _result(id_, {
            "protocolVersion": PROTOCOL_VERSION,
            "capabilities": {"tools": {}},
            "serverInfo": {"name": "skillopt-sleep", "version": "0.1.0"},
        })
    if method in ("notifications/initialized", "initialized"):
        return None  # notification, no response
    if method == "tools/list":
        return _result(id_, {"tools": [
            {"name": t["name"], "description": t["description"], "inputSchema": _TOOL_SCHEMA}
            for t in TOOLS
        ]})
    if method == "tools/call":
        params = req.get("params") or {}
        name = params.get("name")
        tool = _BY_NAME.get(name)
        if not tool:
            return _error(id_, -32602, f"unknown tool: {name}")
        text = _run_engine(tool["action"], params.get("arguments") or {})
        return _result(id_, {"content": [{"type": "text", "text": text}]})
    if method == "ping":
        return _result(id_, {})
    return _error(id_, -32601, f"method not found: {method}")
 def main() -> int:
    for line in sys.stdin:
        line = line.strip()
        if not line:
            continue
        try:
            req = json.loads(line)
        except Exception:
            continue
        resp = handle(req)
        if resp is not None:
            sys.stdout.write(json.dumps(resp) + "\n")
            sys.stdout.flush()
    return 0
 if __name__ == "__main__":
    raise SystemExit(main())
@@ -0,0 +1,98 @@
 # SkillOpt — GitHub Copilot integration
 Give **Copilot** (CLI or VS Code) direct access to the **SkillOpt** research
 engine via a tiny **MCP server**. MCP is GitHub's supported way to extend
 Copilot, so this works across Copilot CLI, VS Code, and other MCP clients with
 the same server.
 SkillOpt is **validation-gated, text-space skill optimization**: it reflects on
 rollouts, makes bounded edits to a skill, and keeps a change only if it improves
 a held-out validation set. This plugin exposes the repo's training and eval
 entry points (`scripts/train.py`, `scripts/eval_only.py`) as Copilot tools.
 > This is the companion to the **SkillOpt-Sleep** plugin (`../mcp_server.py`,
 > `sleep_*` tools). Sleep evolves a *local coding agent* from your past
 > sessions; this server drives the *research* training/eval loops on the
 > benchmark configs in [`../../../configs`](../../../configs).
 ## What's here
 | File | Purpose |
 |---|---|
 | `mcp_server.py` | stdlib-only MCP (stdio) server exposing `skillopt_*` tools |
 | `mcp-config.example.json` | drop-in MCP server config |
 | `copilot-instructions.snippet.md` | paste into `.github/copilot-instructions.md` |
 ## Install
 Requires Python ≥ 3.10. The MCP server itself is pure stdlib, but the tools it
 launches need SkillOpt's runtime deps — install the package first:
 ```bash
 pip install -e .   # or: pip install -r requirements.txt
 ```
 1. **Register the MCP server.** Add the server to your Copilot MCP config
   (Copilot CLI: `~/.copilot/mcp-config.json`; VS Code: your MCP settings).
   Use `mcp-config.example.json` as a template — set `SKILLOPT_REPO` to this
   repo's path:
   ```json
   {
     "mcpServers": {
       "skillopt": {
         "command": "python3",
         "args": ["/abs/path/SkillOpt/plugins/copilot/skillopt/mcp_server.py"],
         "env": { "SKILLOPT_REPO": "/abs/path/SkillOpt" }
       }
     }
   }
   ```
 2. **(Optional) Tell Copilot about it.** Append
   `copilot-instructions.snippet.md` to your repo's
   `.github/copilot-instructions.md` so Copilot reaches for the tools when the
   user asks to "optimize a skill" or "train on a benchmark".
 ## Use
 Ask Copilot things like *"what configs can I run?"*, *"optimize the searchqa
 skill"*, or *"evaluate this skill on the dataset"*. Copilot calls the MCP tools:
 `skillopt_list_configs`, `skillopt_train`, `skillopt_eval`.
 | Tool | Required args | Notes |
 |---|---|---|
 | `skillopt_list_configs` | — | Lists `configs/**/*.yaml` you can pass as `config`. |
 | `skillopt_train` | `config` | Runs a reflective optimization loop. Long-running; spends budget. |
 | `skillopt_eval` | `config`, `skill` | Evaluates one skill markdown file; no training. |
 Common optional args (both train and eval): `env`, `backend`,
 `optimizer_model`, `target_model`, `out_root`, `cfg_options` (space-separated
 `KEY=VALUE` YAML overrides), and `extra_args` (raw passthrough flags for the
 underlying script). `skillopt_train` also accepts `num_epochs`, `batch_size`,
 `seed`, and `use_gate`.
 Runs can be very long. The server's subprocess timeout defaults to 6 hours;
 override it with the `SKILLOPT_RUN_TIMEOUT` environment variable (seconds).
 ## Verify the server directly (no Copilot needed)
 ```bash
 printf '%s\n' \
  '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{}}' \
  '{"jsonrpc":"2.0","id":2,"method":"tools/list"}' \
  '{"jsonrpc":"2.0","id":3,"method":"tools/call","params":{"name":"skillopt_list_configs","arguments":{}}}' \
  | SKILLOPT_REPO="$(pwd)" python3 plugins/copilot/skillopt/mcp_server.py
 ```
 You should see the server info, the three `skillopt_*` tools, and the list of
 benchmark configs.
 ## Notes / status
 - MCP is the stable, official Copilot extension surface, so this is portable
  across Copilot CLI and IDE from one server.
 - `skillopt_list_configs` is filesystem-only and safe to call anytime;
  `skillopt_train` / `skillopt_eval` shell out to the repo scripts and require
  the SkillOpt runtime deps (and, for real backends, model credentials — see
  [`../../../.env.example`](../../../.env.example)).
@@ -0,0 +1,33 @@
 <!--
 Copy this block into your repo's .github/copilot-instructions.md so Copilot
 knows the SkillOpt research-engine tools exist. (Copilot reads
 copilot-instructions.md automatically as ambient guidance.)
 -->
 ## SkillOpt (research skill-optimization engine)
 This repo exposes the core **SkillOpt** training/eval engine via an MCP server
 (`skillopt`). SkillOpt is validation-gated, text-space skill optimization: it
 reflects on rollouts, makes bounded edits to a skill, and keeps a change only
 if it improves a held-out validation set.
 When the user asks to "optimize a skill", "train on <benchmark>", "run
 SkillOpt", "evaluate this skill", or "what configs can I run", use the MCP
 tools:
 - `skillopt_list_configs` — list the benchmark YAML configs you can pass as `config`
 - `skillopt_train` — run a reflective skill-optimization loop on a config (long-running; spends API/compute budget)
 - `skillopt_eval` — evaluate a single skill markdown file on a dataset (no training)
 Guidance:
 - Always run `skillopt_list_configs` first if you don't already know a valid `config` path.
 - `skillopt_train` and `skillopt_eval` are long-running and consume the user's
  model backend/budget — confirm the `config`, `backend`, and model choices
  with the user before launching, and surface the held-out gate result when the
  run finishes.
 - For one-off YAML overrides use `cfg_options` (e.g. `seed=123 batch_size=40`);
  for any other underlying flag use `extra_args`.
 This is distinct from the **SkillOpt-Sleep** MCP server (`skillopt-sleep`,
 `sleep_*` tools), which evolves a local coding agent from past sessions rather
 than running the research benchmarks.
@@ -0,0 +1,11 @@
 {
  "mcpServers": {
    "skillopt": {
      "command": "python3",
      "args": ["plugins/copilot/skillopt/mcp_server.py"],
      "env": {
        "SKILLOPT_REPO": "${workspaceFolder}"
      }
    }
  }
 }
@@ -0,0 +1,229 @@
 #!/usr/bin/env python3
 """SkillOpt (research engine) — minimal MCP server (stdio, stdlib-only).
 Exposes the core SkillOpt skill-optimization engine as MCP tools so any
 MCP-capable client (GitHub Copilot CLI / VS Code, Claude Desktop, etc.) can
 drive it. No third-party deps: speaks JSON-RPC 2.0 over stdio with just the
 handful of MCP methods clients need.
 This is the companion to the SkillOpt-Sleep MCP server (``../mcp_server.py``).
 Where Sleep evolves a *local agent* from past sessions, this server drives the
 *research* training/eval loops from this repo (``scripts/train.py`` /
 ``scripts/eval_only.py``) against the benchmark configs in ``configs/``.
 Tools exposed:
  - skillopt_list_configs : discover the benchmark YAML configs you can use
  - skillopt_train        : run a reflective skill-optimization (training) loop
  - skillopt_eval         : evaluate a single skill on a dataset (no training)
 ``skillopt_train`` and ``skillopt_eval`` shell out to the repo's entry-point
 scripts and stream back their stdout/stderr. Configure your client to launch:
  python plugins/copilot/skillopt/mcp_server.py
 """
 from __future__ import annotations
 import glob
 import json
 import os
 import subprocess
 import sys
 # Repo root: three levels up from plugins/copilot/skillopt/mcp_server.py
 REPO_ROOT = os.environ.get("SKILLOPT_REPO") or os.path.abspath(
    os.path.join(os.path.dirname(__file__), "..", "..", "..")
 )
 PROTOCOL_VERSION = "2024-11-05"
 # Training/eval runs are long; give the engine plenty of headroom.
 RUN_TIMEOUT_SECONDS = int(os.environ.get("SKILLOPT_RUN_TIMEOUT", "21600"))  # 6h
 def _list_configs() -> str:
    """List the benchmark configs available under configs/ (filesystem only)."""
    pattern = os.path.join(REPO_ROOT, "configs", "**", "*.yaml")
    paths = sorted(glob.glob(pattern, recursive=True))
    if not paths:
        return f"[no configs found under {os.path.join(REPO_ROOT, 'configs')}]"
    rels = [os.path.relpath(p, REPO_ROOT).replace(os.sep, "/") for p in paths]
    lines = ["Available SkillOpt configs (pass as `config`):", ""]
    lines += [f"  - {r}" for r in rels]
    return "\n".join(lines)
 def _run_script(script_rel: str, args: dict, *, required: tuple[str, ...] = ()) -> str:
    """Shell out to a repo entry-point script, mapping args -> --flags."""
    for key in required:
        if not args.get(key):
            return f"[error] missing required argument: {key}"
    py = sys.executable or "python3"
    cmd = [py, os.path.join("scripts", script_rel)]
    # Ordered flags that the train/eval scripts accept directly.
    flag_args = (
        "config", "skill", "split", "env", "backend",
        "optimizer_model", "target_model", "out_root",
        "num_epochs", "batch_size", "seed", "use_gate",
    )
    for key in flag_args:
        val = args.get(key)
        if val is None or val == "":
            continue
        cmd += [f"--{key}", str(val)]
    # cfg-options: arbitrary KEY=VALUE YAML overrides (nargs="+").
    cfg_options = args.get("cfg_options")
    if cfg_options:
        if isinstance(cfg_options, str):
            cfg_options = cfg_options.split()
        cmd += ["--cfg-options", *[str(x) for x in cfg_options]]
    # extra_args: raw passthrough for any other train/eval flag.
    extra = args.get("extra_args")
    if extra:
        if isinstance(extra, str):
            extra = extra.split()
        cmd += [str(x) for x in extra]
    try:
        proc = subprocess.run(
            cmd, cwd=REPO_ROOT, capture_output=True, text=True,
            timeout=RUN_TIMEOUT_SECONDS,
        )
    except subprocess.TimeoutExpired:
        return f"[error] run exceeded {RUN_TIMEOUT_SECONDS}s timeout: {' '.join(cmd)}"
    except Exception as e:  # noqa: BLE001
        return f"[error] failed to run script: {e}"
    out = (proc.stdout or "").strip()
    err = (proc.stderr or "").strip()
    body = out + (("\n[stderr]\n" + err) if err else "")
    return body or f"[done] exit code {proc.returncode}, no output"
 TOOLS = [
    {
        "name": "skillopt_list_configs",
        "description": "List the benchmark YAML configs under configs/ that can be passed as `config` to train/eval.",
    },
    {
        "name": "skillopt_train",
        "description": "Run a SkillOpt reflective skill-optimization (training) loop on a benchmark config. Long-running; uses your model backend/budget.",
    },
    {
        "name": "skillopt_eval",
        "description": "Evaluate a single skill markdown file on a dataset without training (scripts/eval_only.py).",
    },
 ]
 _BY_NAME = {t["name"]: t for t in TOOLS}
 _NO_ARGS_SCHEMA = {"type": "object", "properties": {}, "additionalProperties": False}
 _COMMON_PROPS = {
    "config": {"type": "string",
               "description": "Path to a benchmark YAML config (e.g. configs/searchqa/default.yaml). See skillopt_list_configs."},
    "env": {"type": "string", "description": "Override the environment/adapter name (e.g. searchqa, alfworld)."},
    "backend": {"type": "string", "description": "Model backend (e.g. azure_openai, claude, codex, qwen, minimax)."},
    "optimizer_model": {"type": "string", "description": "Model used for reflection/skill rewriting (the optimizer)."},
    "target_model": {"type": "string", "description": "Model used to execute tasks (the target)."},
    "out_root": {"type": "string", "description": "Output directory root for run artifacts."},
    "cfg_options": {"type": "string", "description": "Space-separated YAML overrides, e.g. 'seed=123 batch_size=40'."},
    "extra_args": {"type": "string", "description": "Raw passthrough flags for the underlying script, e.g. '--workers 8 --max_turns 30'."},
 }
 _TRAIN_SCHEMA = {
    "type": "object",
    "properties": {
        **_COMMON_PROPS,
        "num_epochs": {"type": "integer", "description": "Number of optimization epochs."},
        "batch_size": {"type": "integer", "description": "Tasks per optimization step."},
        "seed": {"type": "integer", "description": "Random seed."},
        "use_gate": {"type": "string", "enum": ["true", "false"],
                     "description": "Whether to keep the held-out validation gate on (default on)."},
    },
    "required": ["config"],
    "additionalProperties": False,
 }
 _EVAL_SCHEMA = {
    "type": "object",
    "properties": {
        **_COMMON_PROPS,
        "skill": {"type": "string", "description": "Path to the skill markdown file to evaluate."},
        "split": {"type": "string", "description": "Dataset split to evaluate (default: all)."},
    },
    "required": ["config", "skill"],
    "additionalProperties": False,
 }
 _SCHEMA_BY_NAME = {
    "skillopt_list_configs": _NO_ARGS_SCHEMA,
    "skillopt_train": _TRAIN_SCHEMA,
    "skillopt_eval": _EVAL_SCHEMA,
 }
 def _result(id_, result):
    return {"jsonrpc": "2.0", "id": id_, "result": result}
 def _error(id_, code, message):
    return {"jsonrpc": "2.0", "id": id_, "error": {"code": code, "message": message}}
 def _dispatch(name: str, args: dict) -> str:
    if name == "skillopt_list_configs":
        return _list_configs()
    if name == "skillopt_train":
        return _run_script("train.py", args, required=("config",))
    if name == "skillopt_eval":
        return _run_script("eval_only.py", args, required=("config", "skill"))
    return f"[error] unknown tool: {name}"
 def handle(req: dict):
    method = req.get("method")
    id_ = req.get("id")
    if method == "initialize":
        return _result(id_, {
            "protocolVersion": PROTOCOL_VERSION,
            "capabilities": {"tools": {}},
            "serverInfo": {"name": "skillopt", "version": "0.1.0"},
        })
    if method in ("notifications/initialized", "initialized"):
        return None  # notification, no response
    if method == "tools/list":
        return _result(id_, {"tools": [
            {"name": t["name"], "description": t["description"],
             "inputSchema": _SCHEMA_BY_NAME[t["name"]]}
            for t in TOOLS
        ]})
    if method == "tools/call":
        params = req.get("params") or {}
        name = params.get("name")
        if name not in _BY_NAME:
            return _error(id_, -32602, f"unknown tool: {name}")
        text = _dispatch(name, params.get("arguments") or {})
        return _result(id_, {"content": [{"type": "text", "text": text}]})
    if method == "ping":
        return _result(id_, {})
    return _error(id_, -32601, f"method not found: {method}")
 def main() -> int:
    for line in sys.stdin:
        line = line.strip()
        if not line:
            continue
        try:
            req = json.loads(line)
        except Exception:
            continue
        resp = handle(req)
        if resp is not None:
            sys.stdout.write(json.dumps(resp) + "\n")
            sys.stdout.flush()
    return 0
 if __name__ == "__main__":
    raise SystemExit(main())
@@ -0,0 +1,66 @@
 # SkillOpt-Sleep — Devin integration
 Give **Devin** (Cognition) a nightly **sleep cycle** via a tiny **MCP server**
 that exposes the `skillopt_sleep` engine as tools. MCP is Devin's supported way
 to add custom tooling, so this works in Devin's CLI and IDE.
 Devin doesn't write transcripts in the format the engine consumes, so this
 plugin adds a **Devin-specific harvester** that converts every locally available
 source into the Claude Code-compatible JSONL the engine reads.
 ## What's here
 | File | Purpose |
 |---|---|
 | `mcp_server.py` | stdlib-only MCP (stdio) server exposing `sleep_*` tools |
 | `harvest_devin.py` | converts Devin ATIF-v1.7 transcripts + agentmemory + `.devin/skills` into JSONL, with `taskKey` + outcome envelopes |
 | `judge.py` | reference judge for the deferred/judge branch of the validation gate |
 | `mcp-config.example.json` | drop-in MCP server config |
 | `devin-rules.snippet.md` | paste into `.devin/rules/skillopt-sleep.md` |
 ## What it harvests
 | Source | Where |
 |---|---|
 | Devin transcripts (ATIF-v1.7) | `~/.local/share/devin/cli/transcripts/*.json` |
 | agentmemory | `~/.agentmemory/standalone.json` |
 | Skill files | `.devin/skills/*/SKILL.md` |
 Workspaces are auto-detected from `~/.config/Devin/User/workspaceStorage/*/workspace.json`.
 After `sleep_adopt`, the evolved skill is synced to `.devin/skills/skillopt-sleep-learned/SKILL.md`.
 ## Install
 Requires Python ≥ 3.10. No third-party packages — the server is pure stdlib.
 1. **Register the MCP server.** Use `mcp-config.example.json` as a template; set
   `args` to the absolute path of this `mcp_server.py`. The engine is found
   automatically (this plugin lives inside the SkillOpt repo). Or via the Devin
   CLI:
   ```bash
   devin mcp add skillopt-sleep \
     --env "SKILLOPT_DEVIN_CLAUDE_HOME=$HOME/.skillopt-sleep-devin" \
     -- python3 /abs/path/to/SkillOpt/plugins/devin/mcp_server.py
   ```
 2. **(Optional)** copy `devin-rules.snippet.md` to `.devin/rules/skillopt-sleep.md`
   so Devin proactively offers the tools.
 3. Ask Devin: *"run the sleep cycle"*, *"what did the last sleep propose?"*, *"adopt it"*.
 ## Tools
 | Tool | What it does |
 |---|---|
 | `sleep_status` | nights run so far + latest staged proposal |
 | `sleep_dry_run` | preview cycle — no staging, no changes |
 | `sleep_run` | full cycle; stages a proposal for review |
 | `sleep_adopt` | apply the staged proposal; syncs skill to the workspace |
 | `sleep_harvest` | debug: list the recurring tasks mined |
 | `sleep_schedule` | install a nightly cron entry (`--hour` / `--minute`) |
 | `sleep_unschedule` | remove the nightly cron entry |
 Default backend is `mock` (no API spend); `--backend claude|codex` uses your own
 budget. Same engine and `sleep_*` interface as the other plugins — all call
 `python -m skillopt_sleep`.
@@ -0,0 +1,19 @@
 # SkillOpt-Sleep (Devin)
 You have access to a nightly self-evolution cycle via the `skillopt-sleep` MCP
 server. Use these tools to improve your long-term skills over time:
 - **`sleep_status`** — how many nights have run + the latest staged proposal
 - **`sleep_dry_run`** — preview a cycle without changing anything
 - **`sleep_run`** — run a full cycle; stages a proposal for review
 - **`sleep_adopt`** — apply the staged proposal to `.devin/skills/skillopt-sleep-learned/SKILL.md`
 - **`sleep_harvest`** — debug: list the recurring tasks mined from recent sessions
 - **`sleep_schedule`** / **`sleep_unschedule`** — install/remove a nightly cron run
 When a user asks about the sleep cycle, skill evolution, or improving your
 long-term memory, prefer calling these tools over explaining the concept.
 Default backend is `mock` (no API spend). Pass `backend: "claude"` or
 `backend: "codex"` with your own API key for real LLM-driven optimization.
 Place this file at `.devin/rules/skillopt-sleep.md` in your workspace.
@@ -0,0 +1,21 @@
 {
  "schema_version": "ATIF-v1.7",
  "session_id": "demo-001",
  "steps": [
    {
      "source": "user",
      "message": "Fix the failing NullPointerException in OrderService.persist() in the dutch-kis project",
      "timestamp": "2026-06-20T10:00:00Z"
    },
    {
      "source": "agent",
      "message": "The repository call returns an Optional that is being unwrapped with .get(). I'll switch to orElseThrow(NotFoundException::new) so the missing-row case is handled.",
      "timestamp": "2026-06-20T10:00:05Z"
    },
    {
      "source": "agent",
      "message": "Applied the fix and ran the suite: rtk mvn test -Dtest=OrderServiceTest -> BUILD SUCCESS, 142 passed, 0 failed.",
      "timestamp": "2026-06-20T10:01:00Z"
    }
  ]
 }
@@ -0,0 +1,533 @@
 #!/usr/bin/env python3
 """Convert Devin IDE local data into Claude Code-format JSONL transcripts.
 Devin (Cognition) does not persist agent conversation transcripts to disk in a
 format the sleep engine understands.  This script bridges that gap by synthesising
 JSONL files from every locally available source:
  1. **Devin transcripts** (~/.local/share/devin/cli/transcripts/*.json)
     Native ATIF-v1.7 format — source:"user" / source:"agent" messages
     converted directly to user/assistant JSONL turns.
  2. **agentmemory** (~/.agentmemory/standalone.json)
     Memories saved by the `agentmemory` MCP server — each memory's title
     becomes a synthetic user prompt; its content becomes the assistant reply.
  3. **Skill files** (.devin/skills/*/SKILL.md)
     Each skill description is converted to a session where the user asked
     "use the <skill> skill" and the assistant described how to apply it.
 Output layout (mirrors ~/.claude/projects/<slug>/<sessionId>.jsonl):
    <out_dir>/projects/<slug>/<session_id>.jsonl
 Workspace auto-detection order:
  1. ``SKILLOPT_DEVIN_WORKSPACES`` env var — colon-separated abs paths
  2. Devin registry: ``~/.config/Devin/User/workspaceStorage/*/workspace.json``
  4. Working directory fallback
 Usage (standalone):
    python harvest_devin.py [--out-dir PATH] [--workspaces PATH ...]
 """
 from __future__ import annotations
 import argparse
 import hashlib
 import json
 import os
 import re
 import sys
 from datetime import datetime, timezone
 from pathlib import Path
 from typing import Any, Dict, List, Optional
 from urllib.parse import unquote, urlparse
 # ── cross-platform path resolution (Linux + Windows + macOS) ──────────────────
 #
 # Devin is a VS Code-family app, so its user-data dir moves with the OS:
 # Linux ~/.config/<App>, Windows %APPDATA%\<App>, macOS
 # ~/Library/Application Support/<App>.  Resolve all candidates and let callers
 # keep whichever actually exists.
 def _app_data_roots(app: str) -> List[str]:
    """User-data dir candidates for a VS Code-family app, current OS first."""
    home = os.path.expanduser("~")
    roots: List[str] = []
    if os.name == "nt":
        appdata = os.environ.get("APPDATA") or os.path.join(home, "AppData", "Roaming")
        roots.append(os.path.join(appdata, app))
    elif sys.platform == "darwin":
        roots.append(os.path.join(home, "Library", "Application Support", app))
    # XDG / Linux (also a sensible fallback everywhere)
    xdg = os.environ.get("XDG_CONFIG_HOME") or os.path.join(home, ".config")
    roots.append(os.path.join(xdg, app))
    # de-dupe, preserve order
    return list(dict.fromkeys(roots))
 def _devin_transcript_candidates() -> List[str]:
    """Where the Devin CLI may store ATIF transcripts, per OS."""
    home = os.path.expanduser("~")
    cands: List[str] = []
    if os.name == "nt":
        for base in (os.environ.get("LOCALAPPDATA"), os.environ.get("APPDATA")):
            if base:
                cands.append(os.path.join(base, "devin", "cli", "transcripts"))
    elif sys.platform == "darwin":
        cands.append(os.path.join(home, "Library", "Application Support",
                                  "devin", "cli", "transcripts"))
    cands.append(os.path.join(home, ".local", "share", "devin", "cli", "transcripts"))
    return list(dict.fromkeys(cands))
 def _first_existing(paths: List[str]) -> str:
    """First path that exists, else the first candidate (for nice messaging)."""
    for p in paths:
        if os.path.exists(p):
            return p
    return paths[0] if paths else ""
 def _uri_to_path(folder: str) -> str:
    """Convert a VS Code ``file://`` workspace URI to a local path, cross-platform.
    Linux:   file:///home/u/proj      -> /home/u/proj
    Windows: file:///c%3A/Users/u/p   -> c:/Users/u/p
    """
    if not folder.startswith("file://"):
        return folder
    path = unquote(urlparse(folder).path)
    # Windows drive paths come through as '/C:/...' — strip the leading slash.
    if os.name == "nt" and re.match(r"^/[A-Za-z]:", path):
        path = path[1:]
    return path
 # ── workspace auto-detection ─────────────────────────────────────────────────
 def _workspaces_from_registry(storage_root: str) -> List[tuple]:
    """Read VS Code-style workspaceStorage to get (mtime, path) pairs."""
    results: List[tuple] = []
    if not os.path.isdir(storage_root):
        return results
    for entry in os.scandir(storage_root):
        ws_json = os.path.join(entry.path, "workspace.json")
        if not os.path.isfile(ws_json):
            continue
        try:
            with open(ws_json, encoding="utf-8") as f:
                data = json.load(f)
            folder = _uri_to_path(data.get("folder", ""))
            if folder and os.path.isdir(folder):
                results.append((os.path.getmtime(ws_json), folder))
        except Exception:
            continue
    return results
 def _detect_workspaces() -> List[str]:
    """Return known workspace paths (Devin registry), newest first."""
    env_val = os.environ.get("SKILLOPT_DEVIN_WORKSPACES", "")
    if env_val:
        # os.pathsep so Windows 'C:\a;C:\b' splits correctly (not on the drive colon)
        return [p for p in env_val.split(os.pathsep) if p and os.path.isdir(p)]
    registries: List[str] = [
        os.path.join(r, "User", "workspaceStorage")
        for r in _app_data_roots("Devin")
    ]
    seen: set = set()
    results: List[tuple] = []
    for registry in registries:
        for mtime, folder in _workspaces_from_registry(registry):
            if folder not in seen:
                seen.add(folder)
                results.append((mtime, folder))
    results.sort(reverse=True)
    paths = [p for _, p in results]
    return paths if paths else [os.getcwd()]
 # ── helpers ───────────────────────────────────────────────────────────────────
 def _slug(path: str) -> str:
    """SHA-256 of abs-path, first 16 hex chars — matches Claude Code's scheme."""
    return hashlib.sha256(os.path.abspath(path).encode()).hexdigest()[:16]
 def _iso(epoch_ms: Optional[float] = None) -> str:
    dt = (datetime.fromtimestamp(epoch_ms / 1000.0, tz=timezone.utc)
          if epoch_ms is not None else datetime.now(tz=timezone.utc))
    return dt.strftime("%Y-%m-%dT%H:%M:%S.000Z")
 def _write_session(
    out_dir: str, project: str, session_id: str,
    user_prompts: List[str], assistant_replies: List[str],
    timestamp_base_ms: float,
    task_key: Optional[str] = None,
 ) -> None:
    slug = _slug(project)
    session_dir = os.path.join(out_dir, "projects", slug)
    os.makedirs(session_dir, exist_ok=True)
    out_path = os.path.join(session_dir, f"{session_id}.jsonl")
    ts = timestamp_base_ms
    with open(out_path, "w", encoding="utf-8") as f:
        for user_text, asst_text in zip(user_prompts, assistant_replies):
            user_rec = {
                "type": "user",
                "message": {"role": "user", "content": user_text},
                "cwd": project,
                "timestamp": _iso(ts),
                "sessionId": session_id,
                "version": "1.0",
            }
            if task_key:
                # grouping key so the miner can collapse repeats into one recurring task
                user_rec["taskKey"] = task_key
            f.write(json.dumps(user_rec, ensure_ascii=False) + "\n")
            # space the reply >=5s after the prompt so a single-turn session
            # isn't misclassified as a <3s headless replay and dropped by the
            # engine's harvest filter (skillopt_sleep Issue #62).
            ts += 5000
            f.write(json.dumps({
                "type": "assistant",
                "message": {"role": "assistant", "content": asst_text},
                "timestamp": _iso(ts),
                "sessionId": session_id,
                "version": "1.0",
            }, ensure_ascii=False) + "\n")
            ts += 2000
 def _append_history(out_dir: str, display: str, project: str, timestamp_ms: float) -> None:
    record = {"display": display, "timestamp": timestamp_ms, "project": project}
    with open(os.path.join(out_dir, "history.jsonl"), "a", encoding="utf-8") as f:
        f.write(json.dumps(record, ensure_ascii=False) + "\n")
 def _infer_project(text: str, workspaces: List[str]) -> str:
    for ws in workspaces:
        if os.path.basename(ws.rstrip("/")).lower() in text.lower():
            return ws
    return workspaces[0] if workspaces else os.getcwd()
 # ── task identity + outcome extraction (fuel for the validation gate) ─────────
 #
 # SkillOpt's gate only works "where tasks recur and have a checkable correctness
 # signal."  These helpers add the two things a raw transcript lacks:
 #   * a stable taskKey so repeats collapse into one recurring task, and
 #   * an outcome envelope (success + verifier + re-runnable reference) so the
 #     held-out replay has something to score against.
 _LANG_HINTS = [
    ("java",   r"(java|spring|maven|\bmvn\b|gradle|\.java\b|lombok)"),
    ("python", r"(python|pytest|\bpip\b|\.py\b|django|flask)"),
    ("ts",     r"(typescript|\.tsx?\b|\bnpm\b|jest|node)"),
    ("js",     r"(javascript|\.jsx?\b)"),
    ("sql",    r"(\bsql\b|select\s|mariadb|mysql|postgres|\.sql\b)"),
    ("go",     r"(golang|\bgo test\b|\.go\b)"),
    ("rust",   r"(rust|cargo|\.rs\b)"),
 ]
 _INTENT_HINTS = [
    ("fix",       r"(fix|bug|error|fail|npe|exception|broken|crash)"),
    ("implement", r"(implement|add|create|build|introduce|support)"),
    ("refactor",  r"(refactor|clean ?up|rename|extract|simplify)"),
    ("test",      r"(test|coverage|assert)"),
    ("review",    r"(review|audit|inspect)"),
    ("optimize",  r"(optimi[sz]e|perf|speed up|slow)"),
    ("explain",   r"(explain|understand|what does|how does)"),
 ]
 _STOPWORDS = {"please", "this", "that", "with", "from", "into", "should",
              "would", "code", "using", "the", "have"}
 def _normalize_task_key(text: str, project: str) -> str:
    """Stable '<lang>:<intent>:<target>' grouping key for a task."""
    low = text.lower()
    lang = next((n for n, pat in _LANG_HINTS if re.search(pat, low)), "general")
    intent = next((n for n, pat in _INTENT_HINTS if re.search(pat, low)), "task")
    # target: prefer a CamelCase identifier, then a filename, then first real word
    m = re.search(r"\b([A-Z][a-z0-9]+(?:[A-Z][a-z0-9]+)+)\b", text)  # CamelCase
    if not m:
        m = re.search(r"\b([\w-]+\.\w+)\b", text)                     # filename.ext
    if m:
        target = m.group(1)
    else:
        # first content word that isn't a stopword or an intent verb (e.g. "implement")
        target = next((w for w in re.findall(r"[a-zA-Z]{4,}", low)
                       if w not in _STOPWORDS
                       and not any(re.search(pat, w) for _, pat in _INTENT_HINTS)),
                      "general")
    target = re.sub(r"[^a-zA-Z0-9]+", "-", target).strip("-").lower()[:40] or "general"
    return f"{lang}:{intent}:{target}"
 _PASS_PAT = re.compile(
    r"(build success|all tests? pass(?:ed)?|\b\d+ passed\b|\b0 failed\b|"
    r"tests? pass(?:ed)?|✓|no errors)", re.IGNORECASE)
 _FAIL_PAT = re.compile(
    r"(build failure|tests? failed|\b[1-9]\d* failed\b|error:|traceback|"
    r"assertion ?error)", re.IGNORECASE)  # note: "0 failed" must NOT match
 _CMD_PAT = re.compile(
    r"((?:rtk\s+)?(?:mvn|gradle|pytest|npm(?:\s+run)?\s+test|yarn\s+test|"
    r"go\s+test|cargo\s+test)[^\n`]*)", re.IGNORECASE)
 def _detect_outcome(messages: List[str]) -> Optional[Dict[str, Any]]:
    """Best-effort checkable signal from agent messages. None ⇒ no hard signal."""
    blob = "\n".join(m for m in messages if m)
    pass_hit, fail_hit = _PASS_PAT.search(blob), _FAIL_PAT.search(blob)
    if not pass_hit and not fail_hit:
        return None
    verifier = "tests" if re.search(r"test|pytest", blob, re.IGNORECASE) else "build"
    out: Dict[str, Any] = {
        "success": bool(pass_hit) and not fail_hit,
        "verifier": verifier,
        "evidence": (pass_hit or fail_hit).group(0).strip(),
    }
    cmd = _CMD_PAT.search(blob)
    if cmd:
        # keep only the command itself, dropping any "-> result" / ": output" tail
        repro = re.split(r"\s*(?:->|→|:|,)\s*", cmd.group(1))[0].strip()
        out["reference"] = {"repro": repro}
    return out
 def _build_rubric(user_prompt: str) -> List[str]:
    """Derive checkable criteria from the task so a judge has something to score."""
    crit: List[str] = []
    ids = re.findall(r"\b([A-Z][a-z0-9]+(?:[A-Z][a-z0-9]+)+|[\w-]+\.\w+)\b", user_prompt)
    for i in dict.fromkeys(ids):           # dedupe, preserve order
        crit.append(f"Addresses {i}")
    intent = _normalize_task_key(user_prompt, "").split(":")[1]
    crit.append({
        "fix":       "Resolves the reported defect without introducing new errors",
        "implement": "Implements the requested behavior end to end",
        "refactor":  "Preserves behavior while improving structure",
        "test":      "Adds or fixes tests that actually exercise the change",
        "optimize":  "Improves performance without changing results",
    }.get(intent, "Satisfies the user's stated request"))
    crit.append("Response is concrete and actionable, not a restatement of the task")
    return crit[:5]
 def _judge_rubric_fallback(user_prompt: str) -> Dict[str, Any]:
    """When no hard signal exists, attach a rubric and mark the task for judge
    scoring. success=None tells the gate to defer/judge rather than trust it.
    The actual scoring is done by judge.py (or the engine) at replay time."""
    return {
        "success": None,
        "verifier": "judge",
        "rubric": _build_rubric(user_prompt or ""),
    }
 def _write_outcome(out_dir: str, session_id: str, task_key: str, project: str,
                   ts_ms: float, outcome: Dict[str, Any]) -> None:
    rec = {"type": "outcome", "sessionId": session_id, "taskKey": task_key,
           "project": project, "timestamp": _iso(ts_ms), **outcome}
    with open(os.path.join(out_dir, "outcomes.jsonl"), "a", encoding="utf-8") as f:
        f.write(json.dumps(rec, ensure_ascii=False) + "\n")
 # ── source 1: Devin ATIF-v1.7 transcripts ────────────────────────────────────
 def harvest_devin_transcripts(
    transcripts_dir: str, out_dir: str, workspaces: List[str]
 ) -> int:
    """Convert Devin CLI ATIF-v1.7 transcripts to Claude Code JSONL."""
    if not os.path.isdir(transcripts_dir):
        return 0
    written = 0
    for entry in os.scandir(transcripts_dir):
        if not entry.name.endswith(".json"):
            continue
        try:
            with open(entry.path, encoding="utf-8") as f:
                data = json.load(f)
        except Exception:
            continue
        if data.get("schema_version", "").startswith("ATIF"):
            pass  # Devin native format
        else:
            continue
        session_id = data.get("session_id") or entry.name[:-5]
        steps = data.get("steps") or []
        user_prompts: List[str] = []
        agent_replies: List[str] = []
        project = ""
        ts_base: Optional[float] = None
        for step in steps:
            src = step.get("source", "")
            msg = str(step.get("message") or "").strip()
            if not msg or src == "system":
                continue
            if src == "user":
                user_prompts.append(msg)
                if not project:
                    project = _infer_project(msg, workspaces)
            elif src == "agent":
                agent_replies.append(msg)
            if ts_base is None:
                raw_ts = step.get("timestamp", "")
                if raw_ts:
                    try:
                        from datetime import datetime as _dt
                        ts_base = _dt.fromisoformat(
                            raw_ts.replace("Z", "+00:00")
                        ).timestamp() * 1000
                    except Exception:
                        pass
        if not user_prompts:
            continue
        if not project:
            project = workspaces[0] if workspaces else os.getcwd()
        if ts_base is None:
            ts_base = datetime.now(tz=timezone.utc).timestamp() * 1000
        # Identity + outcome: what makes this trajectory replayable & gradeable.
        task_key = _normalize_task_key(user_prompts[0], project)
        outcome = _detect_outcome(agent_replies) or _judge_rubric_fallback(user_prompts[0])
        # Pair turns; pad shorter list
        n = max(len(user_prompts), len(agent_replies))
        user_prompts += [""] * (n - len(user_prompts))
        agent_replies += [""] * (n - len(agent_replies))
        sid = f"devin_{session_id}"
        _write_session(
            out_dir, project, sid,
            user_prompts=[p for p in user_prompts if p],
            assistant_replies=[r if r else "[no reply recorded]" for r, p in
                               zip(agent_replies, user_prompts) if p],
            timestamp_base_ms=ts_base,
            task_key=task_key,
        )
        _write_outcome(out_dir, sid, task_key, project, ts_base, outcome)
        _append_history(
            out_dir,
            display=(user_prompts[0] or session_id)[:120],
            project=project,
            timestamp_ms=ts_base,
        )
        written += 1
    return written
 # ── source 2: agentmemory ─────────────────────────────────────────────────────
 def harvest_agentmemory(agentmemory_path: str, out_dir: str,
                        workspaces: List[str]) -> int:
    if not os.path.isfile(agentmemory_path):
        return 0
    with open(agentmemory_path, encoding="utf-8") as f:
        data = json.load(f)
    memories: Dict[str, Any] = data.get("mem:memories", {})
    written = 0
    base_ts = datetime.now(tz=timezone.utc).timestamp() * 1000 - len(memories) * 60_000
    for i, (mem_id, mem) in enumerate(memories.items()):
        title = str(mem.get("title", "")).strip()
        content = str(mem.get("content", "")).strip()
        if not title or not content:
            continue
        project = _infer_project(title + " " + content, workspaces)
        ts = base_ts + i * 60_000
        _write_session(out_dir, project, mem_id,
                       user_prompts=[title],
                       assistant_replies=[content],
                       timestamp_base_ms=ts)
        _append_history(out_dir, display=title[:120], project=project, timestamp_ms=ts)
        written += 1
    return written
 # ── source 3: skill files (.devin/skills) ─────────────────────────────────────
 def harvest_skills(workspaces: List[str], out_dir: str) -> int:
    written = 0
    seen_ids: set = set()
    for ws in workspaces:
        skills_root = os.path.join(ws, ".devin", "skills")
        if not os.path.isdir(skills_root):
            continue
        for skill_dir in os.scandir(skills_root):
            if not skill_dir.is_dir():
                continue
            skill_md = os.path.join(skill_dir.path, "SKILL.md")
            if not os.path.isfile(skill_md):
                continue
            sid = f"skill_{skill_dir.name}"
            if sid in seen_ids:
                continue
            seen_ids.add(sid)
            with open(skill_md, encoding="utf-8") as f:
                raw = f.read()
            body = re.sub(r"^---.*?---\s*", "", raw, flags=re.DOTALL).strip()
            if not body:
                continue
            first_line = body.split("\n")[0].lstrip("# ").strip()
            user_ask = f"Please use the {skill_dir.name} skill: {first_line}"
            ts = datetime.now(tz=timezone.utc).timestamp() * 1000 - 3_600_000
            _write_session(out_dir, ws, sid,
                           user_prompts=[user_ask],
                           assistant_replies=[body[:1200]],
                           timestamp_base_ms=ts)
            _append_history(out_dir, display=user_ask[:120], project=ws, timestamp_ms=ts)
            written += 1
    return written
 # ── main ─────────────────────────────────────────────────────────────────────
 def main(argv=None) -> int:
    parser = argparse.ArgumentParser(
        description="Generate SkillOpt-Sleep transcripts from Devin local data"
    )
    parser.add_argument(
        "--out-dir",
        default=os.path.expanduser("~/.skillopt-sleep-devin"),
        help="Output claude_home dir (default: ~/.skillopt-sleep-devin)",
    )
    parser.add_argument(
        "--agentmemory",
        default=os.path.expanduser("~/.agentmemory/standalone.json"),
        help="Path to agentmemory standalone.json",
    )
    parser.add_argument(
        "--devin-transcripts",
        default=_first_existing(_devin_transcript_candidates()),
        help="Devin CLI ATIF transcripts directory (default: per-OS auto-detect)",
    )
    parser.add_argument(
        "--workspaces", nargs="*",
        help="Workspace paths (default: auto-detect from Devin registry)",
    )
    parser.add_argument("--quiet", action="store_true")
    args = parser.parse_args(argv)
    out_dir = os.path.expanduser(args.out_dir)
    os.makedirs(out_dir, exist_ok=True)
    os.makedirs(os.path.join(out_dir, "projects"), exist_ok=True)
    workspaces = args.workspaces or _detect_workspaces()
    workspaces = [ws for ws in workspaces if os.path.isdir(ws)]
    if not workspaces:
        workspaces = [os.getcwd()]
    total = 0
    devin_transcripts = os.path.expanduser(args.devin_transcripts)
    n = harvest_devin_transcripts(devin_transcripts, out_dir, workspaces)
    if not args.quiet:
        print(f"[harvest_devin] devin        : {n} sessions")
    total += n
    n = harvest_agentmemory(args.agentmemory, out_dir, workspaces)
    if not args.quiet:
        print(f"[harvest_devin] agentmemory  : {n} sessions")
    total += n
    n = harvest_skills(workspaces, out_dir)
    if not args.quiet:
        print(f"[harvest_devin] skill files  : {n} sessions")
    total += n
    if not args.quiet:
        print(f"[harvest_devin] total        : {total} synthetic sessions → {out_dir}")
    return 0
 if __name__ == "__main__":
    raise SystemExit(main())
@@ -0,0 +1,129 @@
 #!/usr/bin/env python3
 """Reference judge for SkillOpt-Sleep — score a candidate reply against a rubric.
 Tasks harvested without a hard test/build signal get ``verifier: "judge"`` and a
 ``rubric`` (see ``_build_rubric`` in harvest_devin.py).  This module is the
 scorer the validation gate calls for those tasks: given the rubric and a
 candidate reply produced during replay, it returns a score in ``[0, 1]``.  The
 gate accepts a skill edit only if the *new* skill scores strictly higher on the
 held-out tasks.
 It is self-contained on purpose — in a full deployment the SkillOpt engine owns
 replay+scoring, but having a runnable reference here lets you sanity-check the
 judge path without the engine.
 Backends (select via ``SKILLOPT_JUDGE``):
  * ``heuristic`` (default) — keyword-coverage, offline, no API key, deterministic.
  * ``claude``              — LLM judge via the Anthropic API (needs ANTHROPIC_API_KEY).
 Usage:
    python judge.py --rubric rubric.json --reply reply.txt
    echo "<reply>" | python judge.py --rubric-inline '["Addresses OrderService", ...]'
 """
 from __future__ import annotations
 import argparse
 import json
 import os
 import re
 import sys
 from typing import List
 _STOPWORDS = {"addresses", "resolves", "implements", "without", "introducing",
              "behavior", "request", "response", "concrete", "actionable", "not",
              "the", "and", "that", "with", "stated", "reported", "actually",
              "preserves", "improving", "structure", "requested", "satisfies"}
 # Cheap, fast model is the right default for a judge.
 _JUDGE_MODEL = os.environ.get("SKILLOPT_JUDGE_MODEL", "claude-haiku-4-5-20251001")
 def _content_words(text: str) -> List[str]:
    return [w for w in re.findall(r"[A-Za-z][A-Za-z0-9_.\-]{3,}", text.lower())
            if w not in _STOPWORDS]
 def heuristic_score(reply: str, rubric: List[str]) -> float:
    """Fraction of rubric criteria whose key content words appear in the reply.
    Crude but deterministic: each criterion is 'met' if at least one of its
    content words shows up in the candidate reply. Good enough to smoke-test the
    gate wiring; swap in the claude backend for real judging.
    """
    if not rubric:
        return 0.0
    low = reply.lower()
    met = 0
    for criterion in rubric:
        words = _content_words(criterion)
        if not words:                       # nothing to check → treat as met
            met += 1
            continue
        if any(w in low for w in words):
            met += 1
    return round(met / len(rubric), 3)
 def claude_score(reply: str, rubric: List[str]) -> float:
    """LLM judge via the Anthropic API. Returns a 0..1 score.
    Stdlib-only (urllib) so this file stays dependency-free. Falls back to the
    heuristic if the key is missing or the call fails, so the gate never hard-errors.
    """
    api_key = os.environ.get("ANTHROPIC_API_KEY")
    if not api_key:
        print("[judge] ANTHROPIC_API_KEY unset — using heuristic", file=sys.stderr)
        return heuristic_score(reply, rubric)
    import urllib.request
    rubric_block = "\n".join(f"- {c}" for c in rubric)
    prompt = (
        "You are scoring an AI agent's reply against a rubric. For each criterion, "
        "decide if the reply satisfies it. Respond with ONLY a number between 0 and "
        "1 — the fraction of criteria satisfied.\n\n"
        f"Rubric:\n{rubric_block}\n\nReply:\n{reply}\n\nScore:"
    )
    body = json.dumps({
        "model": _JUDGE_MODEL,
        "max_tokens": 8,
        "messages": [{"role": "user", "content": prompt}],
    }).encode()
    req = urllib.request.Request(
        "https://api.anthropic.com/v1/messages", data=body,
        headers={"content-type": "application/json", "x-api-key": api_key,
                 "anthropic-version": "2023-06-01"},
    )
    try:
        with urllib.request.urlopen(req, timeout=30) as resp:
            data = json.load(resp)
        text = "".join(b.get("text", "") for b in data.get("content", []))
        m = re.search(r"[01](?:\.\d+)?", text)
        return max(0.0, min(1.0, float(m.group(0)))) if m else heuristic_score(reply, rubric)
    except Exception as exc:                 # network/auth/parse — degrade gracefully
        print(f"[judge] claude backend failed ({exc}) — using heuristic", file=sys.stderr)
        return heuristic_score(reply, rubric)
 def score(reply: str, rubric: List[str]) -> float:
    backend = os.environ.get("SKILLOPT_JUDGE", "heuristic")
    return claude_score(reply, rubric) if backend == "claude" else heuristic_score(reply, rubric)
 def main(argv=None) -> int:
    p = argparse.ArgumentParser(description="Score a reply against a rubric (0..1)")
    g = p.add_mutually_exclusive_group(required=True)
    g.add_argument("--rubric", help="Path to a JSON file containing a list of criteria")
    g.add_argument("--rubric-inline", help="Inline JSON list of criteria")
    p.add_argument("--reply", help="Path to the reply text (default: stdin)")
    args = p.parse_args(argv)
    rubric = (json.load(open(args.rubric, encoding="utf-8")) if args.rubric
              else json.loads(args.rubric_inline))
    reply = (open(args.reply, encoding="utf-8").read() if args.reply
             else sys.stdin.read())
    print(score(reply, rubric))
    return 0
 if __name__ == "__main__":
    raise SystemExit(main())
@@ -0,0 +1,11 @@
 {
  "mcpServers": {
    "skillopt-sleep": {
      "command": "python3",
      "args": ["/abs/path/to/SkillOpt/plugins/devin/mcp_server.py"],
      "env": {
        "SKILLOPT_DEVIN_CLAUDE_HOME": "~/.skillopt-sleep-devin"
      }
    }
  }
 }
@@ -0,0 +1,240 @@
 #!/usr/bin/env python3
 """SkillOpt-Sleep — Devin MCP server (stdio, stdlib-only).
 Exposes the sleep engine as MCP tools so Devin (Cognition) can drive it. No
 third-party deps: speaks JSON-RPC 2.0 over stdio with just the handful of MCP
 methods clients need. Same `sleep_*` interface and engine flags as
 `plugins/copilot`, plus a Devin-specific harvest step.
 Before each data-reading action this server runs `harvest_devin.py` to convert
 locally available Devin data (ATIF-v1.7 transcripts, agentmemory memories, and
 .devin skill files) into the Claude Code-compatible JSONL the engine consumes,
 writing it under SKILLOPT_DEVIN_CLAUDE_HOME and pointing the engine there with
 `--claude-home`. After `sleep_adopt` the evolved skill is synced back into the
 workspace's `.devin/skills/`.
 Tools: sleep_status, sleep_dry_run, sleep_run, sleep_adopt, sleep_harvest,
 sleep_schedule, sleep_unschedule. Each shells out to
 `python -m skillopt_sleep <action> ...`. Configure Devin to launch:
  python plugins/devin/mcp_server.py
 """
 from __future__ import annotations
 import json
 import os
 import shutil
 import subprocess
 import sys
 # expanduser wraps the whole value so a "~/..." env var is expanded too (not
 # just a default) — otherwise a literal ~ dir gets created.
 REPO_ROOT = os.path.expanduser(
    os.environ.get("SKILLOPT_SLEEP_REPO")
    or os.path.abspath(os.path.join(os.path.dirname(__file__), "..", ".."))
 )
 PLUGIN_DIR = os.path.dirname(os.path.abspath(__file__))
 CLAUDE_HOME = os.path.expanduser(
    os.environ.get("SKILLOPT_DEVIN_CLAUDE_HOME", "~/.skillopt-sleep-devin")
 )
 MANAGED_SKILL_NAME = os.environ.get("SKILLOPT_MANAGED_SKILL", "skillopt-sleep-learned")
 PROTOCOL_VERSION = "2024-11-05"
 TOOLS = [
    {"name": "sleep_status", "action": "status",
     "description": "Show how many SkillOpt-Sleep nights have run and the latest staged proposal."},
    {"name": "sleep_dry_run", "action": "dry-run",
     "description": "Preview a sleep cycle (harvest+mine+replay) without staging anything."},
    {"name": "sleep_run", "action": "run",
     "description": "Run a full sleep cycle; stages a reviewed proposal. Nothing live changes until adopt."},
    {"name": "sleep_adopt", "action": "adopt",
     "description": "Apply the latest staged proposal to the managed SKILL.md and sync it into .devin/skills/."},
    {"name": "sleep_harvest", "action": "harvest",
     "description": "Debug: list the recurring tasks mined from recent Devin sessions."},
    {"name": "sleep_schedule", "action": "schedule",
     "description": "Install a nightly cron entry to run the sleep cycle automatically."},
    {"name": "sleep_unschedule", "action": "unschedule",
     "description": "Remove the nightly cron entry for a project."},
 ]
 _BY_NAME = {t["name"]: t for t in TOOLS}
 _TOOL_SCHEMA = {
    "type": "object",
    "properties": {
        "project": {"type": "string",
                    "description": "Project dir to evolve (default: cwd)."},
        "backend": {"type": "string", "enum": ["mock", "claude", "codex", "copilot"],
                    "description": "mock = no API spend (default); claude/codex/copilot = real."},
        "scope": {"type": "string", "enum": ["invoked", "all"],
                  "description": "Harvest scope (default: invoked project only)."},
        "source": {"type": "string", "enum": ["claude", "codex", "auto"],
                   "description": "Transcript source (default: claude)."},
        "model": {"type": "string",
                  "description": "Backend-specific model override."},
        "tasks_file": {"type": "string",
                       "description": "Path to reviewed TaskRecord JSON (skips harvest)."},
        "target_skill_path": {"type": "string",
                              "description": "Explicit SKILL.md path to evolve/stage/adopt."},
        "progress": {"type": "boolean",
                     "description": "Print phase progress to stderr."},
        "max_sessions": {"type": "integer",
                         "description": "Cap harvested sessions per run."},
        "max_tasks": {"type": "integer",
                      "description": "Cap mined tasks per run."},
        "lookback_hours": {"type": "integer",
                           "description": "Harvest window in hours (default: 72)."},
        "auto_adopt": {"type": "boolean",
                       "description": "Auto-adopt if gate passes (default: false)."},
        "json": {"type": "boolean",
                 "description": "Return machine-readable JSON output."},
        "edit_budget": {"type": "integer",
                        "description": "Max bounded edits per night (default: 4)."},
        "hour": {"type": "integer",
                 "description": "Hour for schedule (0-23, default: 3)."},
        "minute": {"type": "integer",
                   "description": "Minute for schedule (0-59, default: 17)."},
    },
    "additionalProperties": False,
 }
 # actions that read harvested Devin data (schedule/unschedule/adopt don't)
 _HARVEST_ACTIONS = {"status", "dry-run", "run", "harvest"}
 def _run_harvest() -> str:
    """Convert local Devin data into the JSONL the engine reads, under CLAUDE_HOME."""
    harvester = os.path.join(PLUGIN_DIR, "harvest_devin.py")
    env = dict(os.environ)
    env["PYTHONPATH"] = REPO_ROOT + os.pathsep + env.get("PYTHONPATH", "")
    try:
        proc = subprocess.run(
            [sys.executable, harvester, "--out-dir", CLAUDE_HOME],
            capture_output=True, text=True, timeout=60, env=env,
        )
        out = (proc.stdout or "").strip()
        err = (proc.stderr or "").strip()
        return out + (("\n[harvest stderr]\n" + err) if err else "")
    except Exception as exc:
        return f"[harvest_devin] warning: {exc}"
 def _sync_skill(project: str) -> str:
    """After adopt, copy the evolved skill into the workspace's .devin/skills/."""
    src = os.path.join(CLAUDE_HOME, "skills", MANAGED_SKILL_NAME, "SKILL.md")
    if not (os.path.isfile(src) and project and os.path.isdir(project)):
        return ""
    dot_root = os.path.join(project, ".devin")
    if not os.path.isdir(dot_root):
        return ""
    dst_dir = os.path.join(dot_root, "skills", MANAGED_SKILL_NAME)
    os.makedirs(dst_dir, exist_ok=True)
    dst = os.path.join(dst_dir, "SKILL.md")
    shutil.copy2(src, dst)
    return f"\n[sleep] synced evolved skill → {dst}"
 def _run_engine(action: str, args: dict) -> str:
    harvest_out = _run_harvest() if action in _HARVEST_ACTIONS else ""
    py = sys.executable or "python3"
    cmd = [py, "-m", "skillopt_sleep", action, "--claude-home", CLAUDE_HOME]
    # Devin transcripts are converted to the Claude format, so default source=claude
    if not args.get("source"):
        cmd += ["--source", "claude"]
    # String-valued flags
    for flag, key in [
        ("--project", "project"), ("--backend", "backend"),
        ("--scope", "scope"), ("--source", "source"),
        ("--model", "model"), ("--tasks-file", "tasks_file"),
        ("--target-skill-path", "target_skill_path"),
    ]:
        val = args.get(key)
        if val:
            cmd += [flag, str(val)]
    # Integer-valued flags
    for flag, key in [
        ("--max-sessions", "max_sessions"), ("--max-tasks", "max_tasks"),
        ("--lookback-hours", "lookback_hours"), ("--edit-budget", "edit_budget"),
        ("--hour", "hour"), ("--minute", "minute"),
    ]:
        val = args.get(key)
        if val is not None:
            cmd += [flag, str(int(val))]
    # Boolean flags
    for flag, key in [
        ("--progress", "progress"), ("--auto-adopt", "auto_adopt"), ("--json", "json"),
    ]:
        if args.get(key):
            cmd.append(flag)
    env = dict(os.environ)
    env["PYTHONPATH"] = REPO_ROOT + os.pathsep + env.get("PYTHONPATH", "")
    try:
        proc = subprocess.run(cmd, cwd=REPO_ROOT, capture_output=True,
                              text=True, timeout=3600, env=env)
    except Exception as e:
        return f"[harvest]\n{harvest_out}\n[error] failed to run engine: {e}"
    out = (proc.stdout or "").strip()
    err = (proc.stderr or "").strip()
    result = (f"[harvest]\n{harvest_out}\n\n" if harvest_out else "") + f"[engine]\n{out}"
    if err:
        result += f"\n[stderr]\n{err}"
    if action == "adopt":
        result += _sync_skill(args.get("project") or os.getcwd())
    return result
 def _result(id_, result):
    return {"jsonrpc": "2.0", "id": id_, "result": result}
 def _error(id_, code, message):
    return {"jsonrpc": "2.0", "id": id_, "error": {"code": code, "message": message}}
 def handle(req: dict):
    method = req.get("method")
    id_ = req.get("id")
    if method == "initialize":
        return _result(id_, {
            "protocolVersion": PROTOCOL_VERSION,
            "capabilities": {"tools": {}},
            "serverInfo": {"name": "skillopt-sleep", "version": "0.1.0"},
        })
    if method in ("notifications/initialized", "initialized"):
        return None
    if method == "tools/list":
        return _result(id_, {"tools": [
            {"name": t["name"], "description": t["description"], "inputSchema": _TOOL_SCHEMA}
            for t in TOOLS
        ]})
    if method == "tools/call":
        params = req.get("params") or {}
        name = params.get("name")
        tool = _BY_NAME.get(name)
        if not tool:
            return _error(id_, -32602, f"unknown tool: {name}")
        text = _run_engine(tool["action"], params.get("arguments") or {})
        return _result(id_, {"content": [{"type": "text", "text": text}]})
    if method == "ping":
        return _result(id_, {})
    return _error(id_, -32601, f"method not found: {method}")
 def main() -> int:
    for line in sys.stdin:
        line = line.strip()
        if not line:
            continue
        try:
            req = json.loads(line)
        except Exception:
            continue
        resp = handle(req)
        if resp is not None:
            sys.stdout.write(json.dumps(resp) + "\n")
            sys.stdout.flush()
    return 0
 if __name__ == "__main__":
    raise SystemExit(main())
@@ -0,0 +1,112 @@
 # OpenClaw Plugin for SkillOpt-Sleep
 Thin shell for running [SkillOpt-Sleep](https://github.com/microsoft/SkillOpt) on [OpenClaw](https://github.com/openclaw/openclaw).
 ## What it does
 Adds a nightly "sleep cycle" to any OpenClaw agent. The cycle:
 1. **Harvests** recent session transcripts from `~/.openclaw/agents/<name>/sessions/*.jsonl`
 2. **Mines** recurring task patterns using the optimizer LLM
 3. **Replays** each pattern with the current `SKILL.md` (baseline) and a candidate `SKILL.md` (with proposed edits)
 4. **Gates** the candidate against the held-out score (rejects regressions)
 5. **Stages** the accepted proposal in `~/.skillopt-sleep/staging/<night>/`
 6. Leaves adoption to the operator (Ethan)
 Nothing live changes until you adopt. Every adopt backs up first.
 ## Install
 The plugin is a thin wrapper around the engine at `~/.openclaw/workspace/SkillOpt/skillopt_sleep/`:
 ```bash
 # 1. Clone the engine (one-time)
 cd ~/.openclaw/workspace
 git clone https://github.com/microsoft/SkillOpt.git
 # 2. Install the OpenClaw skill (this folder)
 ln -s /path/to/openclaw ~/.openclaw/workspace/skills/skillopt-sleep
 # 3. Configure
 cp ~/.openclaw/workspace/skills/skillopt-sleep/config.json ~/.skillopt-sleep/config.json
 $EDITOR ~/.skillopt-sleep/config.json
 # Set backend = "openclaw-deepseek"
 # Set model = "deepseek-v4-pro" (or "deepseek-v4-flash" for budget)
 # 4. Set API key
 echo 'export DEEPSEEK_API_KEY="sk-..."' >> ~/.openclaw/.env
 # 5. Add the nightly cron
 (crontab -l 2>/dev/null; echo "0 3 * * * cd ~/.openclaw/workspace/skills/skillopt-sleep && bash run_sleep_cron.sh >> ~/.skillopt-sleep/nightly.log 2>&1") | crontab -
 ```
 ## Use
 ### Manual trigger
 ```bash
 # Run one cycle now
 python3 ~/.openclaw/workspace/skills/skillopt-sleep/run_sleep.py
 # Dry run (report only)
 python3 ~/.openclaw/workspace/skills/skillopt-sleep/run_sleep.py --dry-run
 # One category only
 python3 ~/.openclaw/workspace/skills/skillopt-sleep/run_sleep.py --tasks tests/research-cron-tasks.json
 ```
 ### Slash command
 ```bash
 # In any OpenClaw session
 /sleep status
 /sleep run
 /sleep run research-cron
 /sleep dry-run
 /sleep adopt              # adopt most recent accepted proposal
 /sleep reject             # discard most recent
 /sleep cost
 ```
 ## Architecture
 ```
 plugins/openclaw/
 ├── README.md                       # this file
 ├── run_sleep_cron.sh               # wrapper for cron invocation
 ├── run_sleep.py                    # main entry point
 ├── slash_sleep.py                  # /sleep command implementation
 ├── skillopt_sleep_openclaw.py      # DeepSeek + Ollama backend
 ├── config.json                     # engine config
 ├── SKILL.md                        # OpenClaw skill manifest
 └── tests/                          # held-out test sets
    ├── research-cron-tasks.json
    ├── devops-tasks.json
    └── wiki-tasks.json
 ```
 The OpenClaw shell is one engine (skillopt_sleep/) + one backend (DeepSeek/Ollama) + four thin wrappers (cron, slash, skill, tests).
 ## Why this matters for OpenClaw
 OpenClaw currently has no built-in "self-evolving skills" mechanism. The community has:
 - **Manual skills** — Ethan writes them
 - **LLM-generated skills** — one-shot, no validation
 - **Self-revision** — unbounded, no quality bar
 SkillOpt-Sleep adds a 4th option: **validated self-evolution**. The skill is the training target, the engine is the optimizer, the gate is the quality bar, the operator is the human-in-the-loop.
 ## Validation
 Validated on the public [gbrain-evals](https://github.com/garrytan/gbrain-evals) `skillopt-v1` benchmark with real Claude and Codex (deficient skills 0.00 → 1.00 on held-out, all 4 seeds).
 End-to-end test on our own 14-task held-out set: pipeline runs, gate correctly rejects non-improvements, staging artifacts land in `~/.skillopt-sleep/staging/<night>/`.
 ## Cost
 Measured: ~$0.02/night with `deepseek-v4-pro` at 12 tasks/night. ~$0.59/month, $7.18/year.
 ## License
 MIT (same as SkillOpt core).
@@ -0,0 +1,129 @@
 ---
 name: skillopt-sleep
 description: Validate and refine agent skills through nightly sleep cycles with held-out gates. Wraps Microsoft's SkillOpt-Sleep engine for the OpenClaw/DeepSeek stack.
 ---
 # skillopt-sleep — OpenClaw Adaptation of Microsoft SkillOpt-Sleep
 A nightly self-improvement loop that reads our session transcripts, mines recurring workflow patterns, replays them with proposed skill edits, and gates the proposals against a held-out test set. Only improvements that beat baseline are staged for human adoption.
 ## When To Use
 - After Hermes's Weekly Skill Review (or as its replacement)
 - When a skill is being used 10+ times/week and could be tighter
 - Before promoting a new skill from `skill-proposals/` to `skills/`
 - When a skill regresses in observed quality
 ## What It Does (One Cycle)
 ```
 harvest session transcripts  ->  mine recurring task patterns
                              ->  replay each pattern (current skill vs proposed)
                              ->  GATE: must improve held-out score
                              ->  stage proposal
                              ->  Ethan adopts (manual)
 ```
 Nothing live changes until Ethan adopts. Every adopt backs up first.
 ## Architecture
 ```
 skills/skillopt-sleep/
 ├── SKILL.md                          # this file
 ├── config.json                       # engine config (backend, budgets, etc.)
 ├── run_sleep.py                      # entry point
 └── skillopt_sleep_openclaw.py        # DeepSeek/Ollama backend
 ```
 The engine itself is at `~/.openclaw/workspace/SkillOpt/skillopt_sleep/` (cloned from microsoft/SkillOpt).
 ## Usage
 ```bash
 # Run one cycle with current config
 cd ~/.openclaw/workspace/skills/skillopt-sleep
 python3 run_sleep.py
 # Dry run (report only, no staging)
 python3 run_sleep.py --dry-run
 # Use a pre-built task set (recommended for testing)
 python3 run_sleep.py --tasks tests/research-cron-tasks.json
 ```
 ## Scheduling
 ```bash
 python3 slash_sleep.py schedule --hour 3 --minute 17
 python3 slash_sleep.py unschedule
 python3 slash_sleep.py unschedule --all
 ```
 Installs a nightly cron entry using the shared SkillOpt-Sleep scheduler. This is an alternative to the external `run_sleep_cron.sh` script.
 ## Alternative backends
 While OpenClaw defaults to `openclaw-deepseek` (DeepSeek V4 Pro + Ollama), the shared engine also supports:
 - `--backend mock` — deterministic, no API spend (for testing)
 - `--backend claude` — uses the Claude CLI
 - `--backend codex` — uses the Codex CLI
 - `--backend copilot` — uses the GitHub Copilot CLI
 These can be used via the engine directly (`python -m skillopt_sleep`).
 ## Shared-engine flags
 When invoking the engine directly, all standard flags are available:
 - `--source codex` / `--source auto` — harvest from Codex Desktop sessions
 - `--tasks-file PATH` — use a pre-built task set
 - `--target-skill-path PATH` — explicit SKILL.md target
 - `--max-tasks N` / `--max-sessions N` — cap workload
 - `--progress` — print phase progress
 - `--json` — machine-readable output
 - `--auto-adopt` — auto-adopt if gate passes
 Config keys: `preferences`, `gate_mode`, `gate_metric`, `dream_rollouts`, `recall_k`, `evolve_memory`, `evolve_skill`.
 ## Config (config.json)
 Key knobs:
 - `backend: "openclaw-deepseek"` — our custom backend
 - `model: "deepseek-v4-pro"` — optimizer model
 - `edit_budget: 3` — max bounded edits per night
 - `gate_mode: "on"` — validation-gated (rejects regressions)
 - `auto_adopt: false` — require Ethan to adopt manually
 - `max_tasks_per_night: 12` — cap to control cost
 ## Cost Estimate
 Per night: 12 tasks × (1 attempt + 1 judge + 1 reflect) × ~$0.005/1K tokens × ~3K tokens/call ≈ **$0.50-2.00/night**.
 ## Outputs
 - Report: `~/.skillopt-sleep/state.json` (running totals)
 - Staging: `~/.skillopt-sleep/staging/<night>/`
  - `report.md` — readable summary
  - `best_skill.md` — proposed skill
  - `edits.json` — bounded edit list
  - `before.md` / `after.md` — diffs
 ## Held-Out Test Sets (Phase 2)
 Located at `tests/<category>-tasks.json`. Each task has:
 - `prompt` — the recurring task
 - `reference` — exact-match gold answer
 - `rubric` — soft score rubric (0-1)
 - `domain` — research/devops/wiki/etc.
 Currently building for 3 categories:
 - research-cron-output
 - devops-infrastructure-check
 - wiki-canonical-guide
 ## When NOT To Use
 - For a one-off workflow (not a recurring pattern)
 - During a crisis/incident (humans must lead)
 - When session transcripts are < 24h old (not enough signal)
 - For skills < 300 tokens (over-optimization risk)
@@ -0,0 +1,30 @@
 {
  "_comment": "OpenClaw adaptation of skillopt-sleep. Edit and run via run_sleep.py",
  "claude_home": "/home/ethanclaw/.openclaw/agents",
  "invoked_project": "/home/ethanclaw/.openclaw/workspace",
  "projects": "invoked",
  "lookback_hours": 168,
  "max_tasks_per_night": 12,
  "max_tokens_per_night": 800000,
  "holdout_fraction": 0.34,
  "val_fraction": 0.34,
  "test_fraction": 0.0,
  "backend": "openclaw-deepseek",
  "model": "deepseek-v4-pro",
  "gate_mode": "on",
  "edit_budget": 3,
  "gate_metric": "mixed",
  "gate_mixed_weight": 0.5,
  "replay_mode": "fresh",
  "evolve_memory": true,
  "evolve_skill": true,
  "llm_mine": false,
  "auto_adopt": false,
  "managed_skill_name": "skillopt-sleep-learned",
  "redact_secrets": true,
  "seed": 42
 }
@@ -0,0 +1,122 @@
 #!/usr/bin/env python3
 """run_sleep.py — OpenClaw entry point for SkillOpt-Sleep.
 Runs one nightly sleep cycle:
  1. harvest recent session transcripts
  2. mine recurring task patterns
  3. replay tasks with current skill (baseline) + candidate skill (with proposed edit)
  4. gate candidate vs baseline on held-out accuracy
  5. stage the proposal in ~/.skillopt-sleep/staging/<night>/
  6. leave adoption to Ethan (auto_adopt=false)
 Usage:
  python3 run_sleep.py                  # one cycle, default config
  python3 run_sleep.py --dry-run        # compute report only, no staging
  python3 run_sleep.py --tasks path.json  # use a pre-built task file
 """
 from __future__ import annotations
 import argparse
 import json
 import os
 import sys
 from pathlib import Path
 # Ensure the skillopt_sleep package is importable (it lives in the cloned repo)
 REPO = Path("/home/ethanclaw/.openclaw/workspace/SkillOpt")
 sys.path.insert(0, str(REPO))
 # Register our backend before importing cycle
 from skillopt_sleep_openclaw import OpenClawDeepSeekBackend
 import skillopt_sleep.backend as _b
 _b._BACKENDS = getattr(_b, "_BACKENDS", {})
 _b._BACKENDS["openclaw-deepseek"] = OpenClawDeepSeekBackend
 # Patch get_backend to know about our backend
 _orig_get_backend = _b.get_backend
 def get_backend(name, model="", codex_path=""):
    if name == "openclaw-deepseek":
        return OpenClawDeepSeekBackend(model=model or "deepseek-v4-pro")
    return _orig_get_backend(name, model=model, codex_path=codex_path)
 _b.get_backend = get_backend
 from skillopt_sleep.cycle import run_sleep_cycle
 from skillopt_sleep.config import load_config
 def main() -> int:
    ap = argparse.ArgumentParser(description="OpenClaw SkillOpt-Sleep nightly cycle")
    ap.add_argument("--dry-run", action="store_true", help="Compute but don't stage")
    ap.add_argument("--config", default="/home/ethanclaw/.openclaw/workspace/skills/skillopt-sleep/config.json")
    ap.add_argument("--tasks", default=None, help="Path to pre-built tasks JSON")
    ap.add_argument("--verbose", action="store_true")
    args = ap.parse_args()
    # Load config from file then override with our defaults
    overrides = {}
    if os.path.exists(args.config):
        with open(args.config) as f:
            overrides.update(json.load(f))
    overrides.pop("_comment", None)
    cfg = load_config(**overrides)
    seed_tasks = None
    if args.tasks:
        from skillopt_sleep.types import TaskRecord
        with open(args.tasks) as f:
            raw = json.load(f)
        # Translate our test-set fields → TaskRecord fields
        seed_tasks = []
        for t in raw:
            seed_tasks.append(TaskRecord(
                id=t['id'],
                project=t.get('project', 'openclaw'),
                intent=t.get('intent') or t.get('prompt', ''),
                context_excerpt=t.get('context_excerpt', ''),
                attempted_solution=t.get('attempted_solution', ''),
                outcome=t.get('outcome', 'unknown'),
                reference_kind=t.get('reference_kind', 'rubric'),
                reference=t.get('reference', ''),
                judge=t.get('judge', {}),
                tags=t.get('tags', []),
                source_sessions=t.get('source_sessions', []),
                split=t.get('split', 'train'),
            ))
    print(f"[skillopt-sleep] starting cycle...")
    print(f"  backend: {cfg.get('backend')}")
    print(f"  project: {cfg.get('invoked_project')}")
    print(f"  max tasks: {cfg.get('max_tasks_per_night')}")
    print(f"  edit budget: {cfg.get('edit_budget')}")
    print(f"  dry_run: {args.dry_run}")
    outcome = run_sleep_cycle(cfg, seed_tasks=seed_tasks, dry_run=args.dry_run)
    r = outcome.report
    print(f"\n=== Report — night {r.night} ===")
    print(f"  sessions harvested: {r.n_sessions}")
    print(f"  tasks mined: {r.n_tasks}  (replayed: {r.n_replayed})")
    print(f"  baseline: {r.baseline_score:.3f}  ->  candidate: {r.candidate_score:.3f}")
    print(f"  gate: {r.gate_action}  accepted={r.accepted}")
    print(f"  tokens: {r.tokens_used}")
    if r.edits:
        print(f"  applied edits ({len(r.edits)}):")
        for e in r.edits:
            print(f"    [{e.target}/{e.op}] {e.content[:80]}...")
    if r.rejected_edits:
        print(f"  rejected edits ({len(r.rejected_edits)}) — kept as negative feedback")
    if r.notes:
        for n in r.notes:
            print(f"  note: {n}")
    if outcome.staging_dir:
        print(f"\n  STAGED at: {outcome.staging_dir}")
        print(f"  Review with: ls {outcome.staging_dir}")
    return 0 if r.accepted or r.candidate_score >= r.baseline_score else 1
 if __name__ == "__main__":
    sys.exit(main())
@@ -0,0 +1,76 @@
 #!/bin/bash
 # run_sleep_cron.sh — wrapper for cron-driven nightly sleep cycle
 #
 # Usage: bash run_sleep_cron.sh [category1 category2 ...]
 #   No args: run on all categories in tests/
 #   With args: run only on listed categories (research-cron, devops, wiki)
 #
 # Cron (3am MYT daily):
 #   0 3 * * * cd /home/ethanclaw/.openclaw/workspace/skills/skillopt-sleep && bash run_sleep_cron.sh >> ~/.skillopt-sleep/nightly.log 2>&1
 set -euo pipefail
 SKILL_DIR="/home/ethanclaw/.openclaw/workspace/skills/skillopt-sleep"
 TESTS_DIR="$SKILL_DIR/tests"
 LOG_DIR="$HOME/.skillopt-sleep/logs"
 mkdir -p "$LOG_DIR"
 TIMESTAMP=$(date +%Y%m%d-%H%M%S)
 LOG_FILE="$LOG_DIR/night-$TIMESTAMP.log"
 # category → test file map
 declare -A CATEGORIES=(
    ["research-cron"]="research-cron-tasks.json"
    ["devops"]="devops-tasks.json"
    ["wiki"]="wiki-tasks.json"
 )
 # Determine which categories to run
 if [ $# -eq 0 ]; then
    CATS=("research-cron" "devops" "wiki")
 else
    CATS=("$@")
 fi
 {
    echo "=========================================="
    echo "SkillOpt-Sleep nightly — $TIMESTAMP"
    echo "Categories: ${CATS[*]}"
    echo "=========================================="
 } | tee -a "$LOG_FILE"
 # Pre-flight: check DeepSeek API key
 if ! grep -q "DEEPSEEK_API_KEY=" "$HOME/.openclaw/.env" 2>/dev/null; then
    echo "ERROR: DEEPSEEK_API_KEY not found in ~/.openclaw/.env" | tee -a "$LOG_FILE"
    exit 1
 fi
 EXIT_CODE=0
 for cat in "${CATS[@]}"; do
    tasks_file="$TESTS_DIR/${CATEGORIES[$cat]:-}"
    if [ ! -f "$tasks_file" ]; then
        echo "SKIP: $cat (no tasks file: $tasks_file)" | tee -a "$LOG_FILE"
        continue
    fi
    echo "" | tee -a "$LOG_FILE"
    echo "--- [$cat] starting cycle ---" | tee -a "$LOG_FILE"
    cd "$SKILL_DIR"
    if python3 run_sleep.py --tasks "$tasks_file" 2>&1 | tee -a "$LOG_FILE"; then
        echo "--- [$cat] OK ---" | tee -a "$LOG_FILE"
    else
        EC=$?
        echo "--- [$cat] FAILED (exit $EC) ---" | tee -a "$LOG_FILE"
        EXIT_CODE=$EC
    fi
 done
 {
    echo ""
    echo "=========================================="
    echo "Done. Exit: $EXIT_CODE"
    echo "=========================================="
 } | tee -a "$LOG_FILE"
 exit $EXIT_CODE
@@ -0,0 +1,275 @@
 """OpenClaw backend for SkillOpt-Sleep.
 Adapts the skillopt_sleep Backend protocol to our DeepSeek + Ollama stack:
  - attempt/judge/reflect  ->  DeepSeek V4 Pro (or Flash for cost)
  - embeddings              ->  Ollama nomic-embed-text (already configured)
 This backend NEVER mutates live state. It only returns text + EditRecord
 proposals that the gate stages for human review.
 """
 from __future__ import annotations
 import json
 import os
 import re
 import subprocess
 from typing import Any, Dict, List, Optional, Tuple
 from skillopt_sleep.backend import Backend, _normalize, exact_score
 from skillopt_sleep.types import EditRecord, ReplayResult, TaskRecord
 # ── DeepSeek + Ollama OpenAI-compatible API client (curl-based, no extra deps) ──
 def _chat(messages: List[Dict[str, str]], *, model: str, temperature: float = 0.2, max_tokens: int = 1500) -> str:
    """Call DeepSeek V4 Pro via curl + jq. No extra Python deps needed."""
    import json as _json
    import urllib.request
    api_key = os.environ.get("DEEPSEEK_API_KEY", "")
    if not api_key:
        # try loading from .env
        env_path = os.path.expanduser("~/.openclaw/.env")
        if os.path.exists(env_path):
            with open(env_path) as f:
                for line in f:
                    if line.startswith("DEEPSEEK_API_KEY="):
                        api_key = line.split("=", 1)[1].strip()
                        break
    base = os.environ.get("DEEPSEEK_BASE_URL", "https://api.deepseek.com/v1")
    payload = {
        "model": model,
        "messages": messages,
        "temperature": temperature,
        "max_tokens": max_tokens,
        "stream": False,
    }
    req = urllib.request.Request(
        f"{base}/chat/completions",
        data=_json.dumps(payload).encode("utf-8"),
        headers={
            "Content-Type": "application/json",
            "Authorization": f"Bearer {api_key}",
        },
    )
    try:
        with urllib.request.urlopen(req, timeout=180) as resp:
            data = _json.loads(resp.read().decode("utf-8"))
            return data["choices"][0]["message"]["content"]
    except Exception as e:
        return f"[BACKEND_ERROR] {type(e).__name__}: {str(e)[:200]}"
 def _embed(text: str) -> List[float]:
    """Call Ollama for embeddings. Uses the configured nomic-embed-text model."""
    import json as _json
    import urllib.request
    try:
        req = urllib.request.Request(
            "http://127.0.0.1:11434/api/embeddings",
            data=_json.dumps({"model": "nomic-embed-text:latest", "prompt": text[:2000]}).encode("utf-8"),
            headers={"Content-Type": "application/json"},
        )
        with urllib.request.urlopen(req, timeout=30) as resp:
            data = _json.loads(resp.read().decode("utf-8"))
            return data.get("embedding", [])
    except Exception:
        return []
 # ── Backend implementation ────────────────────────────────────────────────────
 class OpenClawDeepSeekBackend(Backend):
    """Use DeepSeek V4 Pro for attempt/judge/reflect, Ollama for embeddings.
    - "model" passed to constructor = optimizer model (default: deepseek-v4-pro)
    - "judge_model" = judge model (default: deepseek-v4-pro for quality)
    - "cheap_model" = budget-fallback (deepseek-v4-flash)
    """
    name = "openclaw-deepseek"
    def __init__(
        self,
        model: str = "deepseek-v4-pro",
        judge_model: str = "deepseek-v4-pro",
        cheap_model: str = "deepseek-v4-flash",
    ):
        self._model = model
        self._judge_model = judge_model
        self._cheap_model = cheap_model
        self._tokens = 0  # rough estimate
    def tokens_used(self) -> int:
        return self._tokens
    # ── 1. attempt: produce a response given the task + skill + memory ──
    def attempt(self, task: TaskRecord, skill: str, memory: str) -> str:
        sys = (
            "You are an OpenClaw agent (Kobe ecosystem). Use the skill and memory below to complete the task. "
            "If the task asks for a structured output, follow the rubric exactly. "
            "Be concise. No preamble, no explanation unless the task asks for it."
        )
        usr = f"""## SKILL
 {skill or '(no skill yet)'}
 ## MEMORY
 {memory or '(no memory yet)'}
 ## TASK
 {task.intent}
 ## CONTEXT (if any)
 {task.context_excerpt or '(none)'}
 ## RESPONSE
 """
        out = _chat(
            [{"role": "system", "content": sys}, {"role": "user", "content": usr}],
            model=self._model,
            temperature=0.2,
        )
        self._tokens += len(usr) // 4 + 200
        return out
    # ── 2. judge: score the response ──
    def judge(self, task: TaskRecord, response: str) -> Tuple[float, float, str]:
        # Hard score: exact-match against task.reference (if available)
        hard = exact_score(task.reference or "", response)
        # Soft score: LLM judge against rubric (reference if reference_kind=='rubric')
        rubric_text = task.reference if task.reference_kind == "rubric" else ""
        if rubric_text:
            judge_prompt = f"""You are a strict grader. Score the response 0.0-1.0 against the rubric.
 ## TASK
 {task.intent}
 ## REFERENCE
 {task.reference or '(none)'}
 ## RUBRIC
 {rubric_text}
 ## RESPONSE
 {response[:3000]}
 ## INSTRUCTIONS
 Return ONLY a single float 0.0-1.0 on one line. No explanation. No markdown.
 """
            try:
                j_out = _chat(
                    [{"role": "user", "content": judge_prompt}],
                    model=self._judge_model,
                    temperature=0.0,
                    max_tokens=20,
                ).strip()
                soft = float(re.search(r"[\d.]+", j_out.splitlines()[0]).group())
                soft = max(0.0, min(1.0, soft))
            except Exception:
                soft = hard
            self._tokens += 600
        else:
            soft = hard
        rationale = f"hard={hard:.2f} soft={soft:.2f}"
        return hard, soft, rationale
    # ── 3. reflect: produce bounded EditRecord proposals ──
    def reflect(
        self,
        failures: List[Tuple[TaskRecord, ReplayResult]],
        successes: List[Tuple[TaskRecord, ReplayResult]],
        skill: str,
        memory: str,
        *,
        edit_budget: int,
        evolve_skill: bool,
        evolve_memory: bool,
    ) -> List[EditRecord]:
        # Compact digest of failures + successes
        fail_digest = "\n".join(
            f"- TASK: {t.intent[:200]}\n  RESPONSE: {r.response[:300]}\n  WHY FAIL: {r.judge_rationale or r.fail_reason or 'unknown'}\n  REFERENCE: {t.reference[:200]}"
            for t, r in failures[:5]
        ) or "(none)"
        succ_digest = "\n".join(
            f"- TASK: {t.intent[:150]} -> OK ({r.judge_rationale or 'high score'})"
            for t, r in successes[:3]
        ) or "(none)"
        rubric_text = ""
        if failures:
            rubric_text = f"\n\n## REFERENCE ANSWERS\n{chr(10).join(f'Q: {t.intent[:120]}\\nA: {t.reference}' for t, _ in failures[:3] if t.reference)}"
        sys = (
            "You are SkillOpt-Sleep's bounded-edit optimizer. Your job is to propose 1-4 MINIMAL text edits to a skill or memory document "
            "that, if applied, would help future agents do better on the failed tasks. "
            "NEVER propose adding new sections wholesale. NEVER delete entire sections. "
            "Edit primitives: ADD (append a step/rule at end), DELETE (remove a specific line by exact match), REPLACE (swap a specific line for another by exact match). "
            "If you cannot identify a clear, minimal improvement, return an empty list."
        )
        usr = f"""## CURRENT SKILL
 {skill or '(empty)'}
 ## CURRENT MEMORY
 {memory or '(empty)'}
 ## FAILED TASKS
 {fail_digest}
 ## SUCCESSFUL TASKS
 {succ_digest}
 {rubric_text}
 ## CONSTRAINTS
 - max {edit_budget} edits total
 - edits go to {"skill + memory" if (evolve_skill and evolve_memory) else ("skill" if evolve_skill else "memory")}
 - if evolve_skill=False, target="memory" only; if evolve_memory=False, target="skill" only
 - target must be "skill" or "memory"
 ## OUTPUT FORMAT (JSON, no markdown)
 {{"edits": [{{"op": "ADD"|"DELETE"|"REPLACE", "target": "skill"|"memory", "content": "the text to add or replace with", "old_text": "for REPLACE/DELETE, the exact line to find", "rationale": "one short sentence why"}}]}}
 """
        out = _chat(
            [{"role": "system", "content": sys}, {"role": "user", "content": usr}],
            model=self._model,
            temperature=0.4,
            max_tokens=2000,
        )
        self._tokens += len(usr) // 3 + 1500
        # parse
        try:
            # strip markdown fences if any
            cleaned = out.strip()
            if cleaned.startswith("```"):
                cleaned = re.sub(r"^```[a-z]*\n?", "", cleaned)
                cleaned = re.sub(r"\n?```$", "", cleaned)
            data = json.loads(cleaned)
            edits: List[EditRecord] = []
            for e in data.get("edits", [])[:edit_budget]:
                if e.get("op") not in ("ADD", "DELETE", "REPLACE"):
                    continue
                target = e.get("target", "skill")
                if target not in ("skill", "memory"):
                    continue
                if not evolve_skill and target == "skill":
                    continue
                if not evolve_memory and target == "memory":
                    continue
                edits.append(EditRecord(
                    op=e["op"],
                    target=target,
                    content=e.get("content", ""),
                    old_text=e.get("old_text", ""),
                    rationale=e.get("rationale", ""),
                ))
            return edits
        except Exception as e:
            # log + return empty list (no edit is better than a bad edit)
            return []
@@ -0,0 +1,330 @@
 #!/usr/bin/env python3
 """slash_sleep.py — OpenClaw slash command equivalent of SkillOpt's /sleep.
 Use from the main session as a /sleep command:
  /sleep status    — show current state + last 5 nights
  /sleep run       — trigger one cycle (all categories) right now
  /sleep run research-cron  — one cycle, single category
  /sleep adopt [night]      — adopt the most recent (or specified) staged proposal
  /sleep reject [night]     — discard the most recent (or specified) staging dir
  /sleep dry-run   — report-only cycle
  /sleep cost      — estimate per-night cost for current config
 This script is a thin shell over run_sleep.py. It can be invoked either
 manually from the main session or by an OpenClaw command handler.
 """
 from __future__ import annotations
 import argparse
 import json
 import os
 import shutil
 import sys
 from pathlib import Path
 from datetime import datetime
 SKILL_DIR = Path("/home/ethanclaw/.openclaw/workspace/skills/skillopt-sleep")
 STATE_DIR = Path(os.path.expanduser("~/.skillopt-sleep"))  # default
 STAGING_ROOT = STATE_DIR
 def _resolve_state_dir():
    """Find the actual state dir.
    Priority: scan in order:
      1. ~/.skillopt-sleep/                 (default)
      2. /home/ethanclaw/.openclaw/workspace/.skillopt-sleep/  (when staging is there)
      3. /home/ethanclaw/.openclaw/.skillopt-sleep/            (parent of overridden claude_home)
    Pick the first one that has a state.json OR staging dir.
    """
    candidates = [
        Path(os.path.expanduser("~/.skillopt-sleep")),
        Path("/home/ethanclaw/.openclaw/workspace/.skillopt-sleep"),
        Path("/home/ethanclaw/.openclaw/.skillopt-sleep"),
    ]
    # Prefer the one with state.json
    for c in candidates:
        if (c / "state.json").exists():
            return c
    # Then the one with staging
    for c in candidates:
        if (c / "staging").exists():
            return c
    return candidates[0]
 TESTS_DIR = SKILL_DIR / "tests"
 def status() -> int:
    state_dir = _resolve_state_dir()
    state_file = state_dir / "state.json"
    staging_dir = state_dir / "staging"
    print(f"=== SkillOpt-Sleep status ===")
    print(f"  state dir: {state_dir}")
    print(f"  staging dir: {staging_dir}")
    if staging_dir.exists():
        stages = sorted(staging_dir.iterdir(), key=lambda p: p.stat().st_mtime, reverse=True)
        print(f"  staging entries: {len(stages)}")
        for s in stages[:3]:
            print(f"    {s.name}")
    if not state_file.exists():
        print("  no state.json — run a cycle first (state is written at end of each non-dry-run)")
        return 0
    with open(state_file) as f:
        state = json.load(f)
    nights = state.get("history") or state.get("nights", [])
    print(f"  total nights: {len(nights)}")
    print(f"  accepted: {sum(1 for n in nights if n.get('accepted'))}")
    print(f"  rejected: {sum(1 for n in nights if not n.get('accepted'))}")
    if nights:
        last = nights[-1]
        print(f"  last night: {last.get('night')}")
        print(f"    accepted: {last.get('accepted')}")
        print(f"    baseline: {last.get('baseline'):.3f}  ->  candidate: {last.get('candidate'):.3f}")
        print(f"    staging: {last.get('staging') or '(none)'}")
    return 0
 def run_category(category: str, *, dry_run: bool = False) -> int:
    cat_to_file = {
        "research-cron": "research-cron-tasks.json",
        "devops": "devops-tasks.json",
        "wiki": "wiki-tasks.json",
    }
    tasks_file = TESTS_DIR / cat_to_file.get(category, f"{category}-tasks.json")
    if not tasks_file.exists():
        print(f"ERROR: no tasks file for category '{category}': {tasks_file}")
        return 1
    cmd = [sys.executable, str(SKILL_DIR / "run_sleep.py")]
    if dry_run:
        cmd.append("--dry-run")
    cmd.extend(["--tasks", str(tasks_file)])
    print(f"=== /sleep run {category}{' (dry-run)' if dry_run else ''} ===")
    print(f"  cmd: {' '.join(cmd)}")
    rc = os.system(" ".join(f'"{c}"' for c in cmd))
    return rc
 def run_all(*, dry_run: bool = False) -> int:
    rc = 0
    for cat in ("research-cron", "devops", "wiki"):
        r = run_category(cat, dry_run=dry_run)
        if r != 0:
            rc = r
    return rc
 def adopt(night: str = None) -> int:
    state_dir = _resolve_state_dir()
    state_file = state_dir / "state.json"
    if not state_file.exists():
        print("ERROR: no state to adopt from")
        return 1
    with open(state_file) as f:
        state = json.load(f)
    nights = state.get("history") or state.get("nights", [])
    if not nights:
        print("ERROR: no nights recorded")
        return 1
    target = None
    if night:
        target = next((n for n in nights if str(n.get("night")) == night), None)
        if not target:
            print(f"ERROR: night '{night}' not found")
            return 1
    else:
        # most recent accepted
        candidates = [n for n in nights if n.get("accepted") and n.get("staging")]
        if not candidates:
            print("ERROR: no accepted nights with staging to adopt")
            return 1
        target = candidates[-1]
    staging = target["staging"]
    if not os.path.isdir(staging):
        print(f"ERROR: staging dir missing: {staging}")
        return 1
    print(f"=== /sleep adopt night {target['night']} ===")
    print(f"  staging: {staging}")
    print(f"  baseline: {target.get('baseline'):.3f}  candidate: {target.get('candidate'):.3f}")
    # Read proposed skill from staging
    manifest = Path(staging) / "manifest.json"
    if manifest.exists():
        with open(manifest) as f:
            m = json.load(f)
        proposed = m.get("proposed_skill")
        if proposed and Path(proposed).exists():
            live = STATE_DIR / "live_skill.md"
            backup = STATE_DIR / f"live_skill.md.bak-{target['night']}"
            if live.exists():
                shutil.copy2(live, backup)
                print(f"  backed up current live skill → {backup}")
            shutil.copy2(proposed, live)
            print(f"  adopted proposed skill → {live}")
            print()
            print("✅ Adoption complete. Next cycle will use the new skill.")
            return 0
    print("ERROR: no proposed_skill in manifest")
    return 1
 def reject(night: str = None) -> int:
    state_dir = _resolve_state_dir()
    state_file = state_dir / "state.json"
    if not state_file.exists():
        print("ERROR: no state")
        return 1
    with open(state_file) as f:
        state = json.load(f)
    nights = state.get("history") or state.get("nights", [])
    target = None
    if night:
        target = next((n for n in nights if str(n.get("night")) == night), None)
    else:
        candidates = [n for n in reversed(nights) if n.get("staging")]
        target = candidates[0] if candidates else None
    if not target or not target.get("staging"):
        print("ERROR: nothing to reject")
        return 1
    staging = target["staging"]
    if os.path.isdir(staging):
        shutil.rmtree(staging)
        print(f"🗑️  Removed staging: {staging}")
    # remove from state
    state["history"] = [n for n in nights if n.get("night") != target["night"]]
    with open(state_file, "w") as f:
        json.dump(state, f, indent=2)
    print("✅ Rejected. State updated.")
    return 0
 def schedule_cmd(hour: int, minute: int) -> int:
    """Install a nightly cron entry via the shared SkillOpt-Sleep scheduler.
    Note: this schedules the shared engine (``python -m skillopt_sleep run``),
    not the OpenClaw-specific ``run_sleep.py``.  Use ``run_sleep_cron.sh`` if
    you need the OpenClaw-native backend and category task files instead.
    """
    try:
        from skillopt_sleep.scheduler import schedule
    except ImportError:
        print("ERROR: skillopt_sleep.scheduler not available — is SkillOpt-Sleep installed?")
        return 1
    project = str(SKILL_DIR)
    ok, msg = schedule(project, hour=hour, minute=minute)
    print(msg)
    return 0 if ok else 1
 def unschedule_cmd(all_projects: bool) -> int:
    """Remove cron entry via the shared SkillOpt-Sleep scheduler."""
    try:
        from skillopt_sleep.scheduler import unschedule
    except ImportError:
        print("ERROR: skillopt_sleep.scheduler not available — is SkillOpt-Sleep installed?")
        return 1
    project = str(SKILL_DIR)
    ok, msg = unschedule(project, all_projects=all_projects)
    print(msg)
    return 0 if ok else 1
 def cost() -> int:
    """Estimate per-night cost based on the actual measurement from Phase 2.
    From the real dry-run: 5 devops tasks used 14,427 tokens total.
    That is ~2,885 tokens per task (all 3 phases combined).
    """
    cfg_path = SKILL_DIR / "config.json"
    cfg = {}
    if cfg_path.exists():
        cfg = json.loads(cfg_path.read_text())
    cfg.pop("_comment", None)
    max_tasks = cfg.get("max_tasks_per_night", 12)
    model = cfg.get("model", "deepseek-v4-pro")
    # DeepSeek V4 pricing
    if "pro" in model:
        cost_in = 0.435  # per 1M
        cost_out = 0.87
    elif "flash" in model:
        cost_in = 0.14
        cost_out = 0.28
    else:
        cost_in, cost_out = 0.5, 1.0
    # Measured: ~2,900 tokens per task, 30% output / 70% input
    toks_per_task = 2900
    input_toks = int(toks_per_task * 0.7)
    output_toks = int(toks_per_task * 0.3)
    cost_in_total = (input_toks * max_tasks / 1_000_000) * cost_in
    cost_out_total = (output_toks * max_tasks / 1_000_000) * cost_out
    cost = cost_in_total + cost_out_total
    print(f"=== Cost estimate (per actual measurement) ===")
    print(f"  model: {model}")
    print(f"  max tasks/night: {max_tasks}")
    print(f"  ~tokens/night: {toks_per_task * max_tasks:,}")
    print(f"  cost/night: ${cost:.3f}")
    print(f"  cost/month (30 nights): ${cost*30:.2f}")
    print(f"  cost/year (365 nights): ${cost*365:.2f}")
    return 0
 def main():
    ap = argparse.ArgumentParser(description="OpenClaw /sleep command")
    sub = ap.add_subparsers(dest="cmd", required=True)
    sub.add_parser("status", help="show state + last 5 nights")
    p_run = sub.add_parser("run", help="trigger one cycle")
    p_run.add_argument("category", nargs="?", default=None,
                        choices=["research-cron", "devops", "wiki", None])
    p_run.add_argument("--dry-run", action="store_true")
    sub.add_parser("dry-run", help="report-only cycle (all categories)")
    p_adopt = sub.add_parser("adopt", help="adopt most recent accepted staging")
    p_adopt.add_argument("night", nargs="?", default=None)
    p_reject = sub.add_parser("reject", help="discard most recent staging")
    p_reject.add_argument("night", nargs="?", default=None)
    sub.add_parser("cost", help="estimate cost")
    p_schedule = sub.add_parser("schedule", help="install nightly cron entry")
    p_schedule.add_argument("--hour", type=int, default=3, help="hour (0-23)")
    p_schedule.add_argument("--minute", type=int, default=17, help="minute (0-59)")
    p_unschedule = sub.add_parser("unschedule", help="remove cron entry")
    p_unschedule.add_argument("--all", dest="all_projects", action="store_true",
                               help="remove entries for all projects")
    args = ap.parse_args()
    if args.cmd == "status":
        return status()
    if args.cmd == "run":
        if args.category:
            return run_category(args.category, dry_run=args.dry_run)
        return run_all(dry_run=args.dry_run)
    if args.cmd == "dry-run":
        return run_all(dry_run=True)
    if args.cmd == "adopt":
        return adopt(args.night)
    if args.cmd == "reject":
        return reject(args.night)
    if args.cmd == "cost":
        return cost()
    if args.cmd == "schedule":
        return schedule_cmd(args.hour, args.minute)
    if args.cmd == "unschedule":
        return unschedule_cmd(args.all_projects)
    return 1
 if __name__ == "__main__":
    sys.exit(main())
@@ -0,0 +1,87 @@
 [
  {
    "id": "do-01",
    "reference": "[STATUS] devops-agent | Site Uptime \u2192 geoxylia.com OK (200) | 14/06 22:30 MYT",
    "rubric": "Score 1.0 if output matches the exact format [STATUS] devops-agent | Site Uptime \u2192 geoxylia.com OK (200) | DD/MM HH:MM MYT, with a real current time. Score 0.5 if format is close but missing one field. Score 0.0 if wrong format or hallucinated values.",
    "project": "devops-infrastructure-check",
    "intent": "Site Uptime check. Run: `curl -o /dev/null -s -w '%{http_code}' https://geoxylia.com`. Interpret the result 200, and report in our standard format: 'STATUS | TASK \u2192 RESULT | TIME'. If not 200, escalate.",
    "context_excerpt": "",
    "attempted_solution": "",
    "outcome": "unknown",
    "reference_kind": "rubric",
    "judge": {},
    "tags": [
      "devops-infrastructure-check"
    ],
    "source_sessions": [],
    "split": "val"
  },
  {
    "id": "do-02",
    "reference": "Backup complete. Files: 87, Size: 1.2G, Last: 2026-06-14 22:00:00 MYT",
    "rubric": "Score 1.0 if output includes the exact 'Backup complete. Files: N, Size: X, Last: timestamp' structure with plausible values. Score 0.5 if structure is close but one field missing. Score 0.0 if hallucinated or wrong structure.",
    "project": "devops-infrastructure-check",
    "intent": "Daily Memory Backup. Confirm this ran successfully by checking: `ls -t ~/backups/memory/memory-backup-*.tar.gz | head -3`. Report the file count, total size, and most recent backup time. Use format: 'Backup complete. Files: [N], Size: [X], Last: [timestamp]'.",
    "context_excerpt": "",
    "attempted_solution": "",
    "outcome": "unknown",
    "reference_kind": "rubric",
    "judge": {},
    "tags": [
      "devops-infrastructure-check"
    ],
    "source_sessions": [],
    "split": "val"
  },
  {
    "id": "do-03",
    "reference": "1) Vercel CSP missing frame-ancestors: MEDIUM. Allows clickjacking if anyone embeds our pages; not exploitable for our content, but best-practice gap.\n2) OpenClaw plaintext API keys: LOW. The config is chmod 600, loopback-only, not in git. Standard OpenClaw behavior. Rotating would add zero real security given current exposure.",
    "rubric": "Score 1.0 if both are classified correctly (MEDIUM and LOW respectively) and justifications are accurate (not panicky, not dismissive). Score 0.5 if classifications are wrong by one tier or justifications are weak. Score 0.0 if both over-classified as CRITICAL or both wrong.",
    "project": "devops-infrastructure-check",
    "intent": "Security Check daily run. Two findings: 1) Vercel CSP header missing 'frame-ancestors' directive, 2) OpenClaw config has 3 plaintext API keys. Classify each as: CRITICAL / HIGH / MEDIUM / LOW / INFO. Justify each in 1 sentence.",
    "context_excerpt": "",
    "attempted_solution": "",
    "outcome": "unknown",
    "reference_kind": "rubric",
    "judge": {},
    "tags": [
      "devops-infrastructure-check"
    ],
    "source_sessions": [],
    "split": "train"
  },
  {
    "id": "do-04",
    "reference": "[INCIDENT] supabase.audit_results: anon role has no RLS policy \u2014 anyone with the URL can read all audit results. Fix: add policy 'audit_results_select_own' granting SELECT WHERE user_id = auth.uid(). Severity: HIGH (data exposure). Estimated 2-min fix.",
    "rubric": "Score 1.0 if: (a) severity correctly identified as HIGH, (b) fix is a real RLS policy (not just 'enable RLS' since it's already enabled), (c) under 50 words, (d) Telegram-friendly format. Score 0.5 if severity right but fix is generic. Score 0.0 if missing severity or wrong fix.",
    "project": "devops-infrastructure-check",
    "intent": "Incident Check. The Supabase RLS check returned: 'table public.audit_results: rls enabled but policy missing for anon role'. Interpret severity, propose fix, and format as a Telegram alert (max 50 words).",
    "context_excerpt": "",
    "attempted_solution": "",
    "outcome": "unknown",
    "reference_kind": "rubric",
    "judge": {},
    "tags": [
      "devops-infrastructure-check"
    ],
    "source_sessions": [],
    "split": "val"
  },
  {
    "id": "do-05",
    "reference": "\ud83d\udee1\ufe0f Week security digest:\n\n\u2022 0 critical incidents, 1 high resolved (Supabase RLS policy added)\n\u2022 22 plaintext secrets: expected OpenClaw behavior, no action\n\u2022 1 medium open: Vercel CSP frame-ancestors, schedule for next sprint\n\nTrend: stable. No regressions vs last week.",
    "rubric": "Score 1.0 if all 3 priority tiers mentioned with correct counts, ends with a trend statement, Telegram-friendly. Score 0.5 if structure is right but one tier wrong. Score 0.0 if missing a tier or wrong format.",
    "project": "devops-infrastructure-check",
    "intent": "Weekly security digest. Synthesize this week's findings: 22 plaintext secrets in openclaw.json (expected), 0 critical incidents, 1 high (Supabase RLS), 1 medium (CSP frame-ancestors), 0 low. Output a 3-bullet Telegram status.",
    "context_excerpt": "",
    "attempted_solution": "",
    "outcome": "unknown",
    "reference_kind": "rubric",
    "judge": {},
    "tags": [
      "devops-infrastructure-check"
    ],
    "source_sessions": [],
    "split": "train"
  }
 ]
@@ -0,0 +1,87 @@
 [
  {
    "id": "rc-01",
    "reference": "COMPETITOR MOVES: Otterly adds Perplexity tracker, joining Profound and LLMRefs in multi-platform citations.\nBACKLINK OPPORTUNITIES: 3 SEO directories (G2, Capterra, GetApp) have not been claimed.\nAGENCY BLUEPRINT: Top 2 agency sites bundle GEO audit + content refresh as $3K/mo tier.\nACTION ITEMS: Build Perplexity citation test into GeoXylia audit; claim G2 listing by Friday.",
    "rubric": "Score 1.0 if all 4 section headings present in correct order, each with a substantive (not generic) 1-sentence content. Score 0.5 if headings present but content is generic. Score 0.0 if any heading missing or order wrong.",
    "project": "research-cron-output",
    "intent": "Weekly Competitive Deep Dive for GeoXylia. The competitor otterly.ai just added a Perplexity citation tracker. Produce the report header (top section) in our standard format: COMPETITOR MOVES, BACKLINK OPPORTUNITIES, AGENCY BLUEPRINT, ACTION ITEMS. Keep it to 4 lines, one per section heading with a 1-sentence placeholder.",
    "context_excerpt": "",
    "attempted_solution": "",
    "outcome": "unknown",
    "reference_kind": "rubric",
    "judge": {},
    "tags": [
      "research-cron-output"
    ],
    "source_sessions": [],
    "split": "train"
  },
  {
    "id": "rc-02",
    "reference": "1. 'ai seo audit tool': 420 imp, pos 8.2, on page 1 \u2014 needs CTR lift (snippet/schema).\n2. 'geo audit tool': 230 imp, pos 12.5, page 2 \u2014 target blog post could push to page 1.\n3. 'llm optimization': 85 imp, pos 18.3, deep page-2 \u2014 fresh content with answer capsule could compete.",
    "rubric": "Score 1.0 if the response correctly identifies 'ai seo audit tool', 'geo audit tool', and 'llm optimization' as the top 3 (NOT 'best free seo audit' which is already converting well, NOT 'free audit tool' which has too few impressions). Each must have correct impression count, position, and a substantive rationale. Score 0.5 if correct 3 keywords but rationale is weak. Score 0.0 if wrong keywords selected.",
    "project": "research-cron-output",
    "intent": "GSC keyword opportunity scan. From this snippet of GSC data, identify the top 3 keyword opportunities (high impressions, low CTR, position 5-15):\n\n1. 'ai seo audit tool' \u2014 420 imp, 12 clicks, pos 8.2\n2. 'best free seo audit' \u2014 1100 imp, 95 clicks, pos 4.1\n3. 'geo audit tool' \u2014 230 imp, 4 clicks, pos 12.5\n4. 'llm optimization' \u2014 85 imp, 1 click, pos 18.3\n5. 'free audit tool' \u2014 50 imp, 0 clicks, pos 22.0\n\nOutput: one line per opportunity, format 'KEYWORD: impressions, position, why-it-matters (1 short clause)'.",
    "context_excerpt": "",
    "attempted_solution": "",
    "outcome": "unknown",
    "reference_kind": "rubric",
    "judge": {},
    "tags": [
      "research-cron-output"
    ],
    "source_sessions": [],
    "split": "train"
  },
  {
    "id": "rc-03",
    "reference": "Google AI Overviews now show source links more prominently + author bylines. For GeoXylia: this favors pages with clear authorship (add author schema to blog posts). Action: this week, add author + E-E-A-T schema markup to top 10 blog posts. Source: Google Search Central blog.",
    "rubric": "Score 1.0 if: (a) under 60 words, (b) names the change, (c) gives GeoXylia-specific implication, (d) gives a concrete action item, (e) cites the source. Score 0.5 if missing 1-2 of these. Score 0.0 if over 60 words or missing 3+.",
    "project": "research-cron-output",
    "intent": "Daily Industry News scan. The Google Search Central blog just announced: 'AI Overviews now showing source links more prominently, with author bylines for E-E-A-T-heavy content.' Write a 1-paragraph Telegram alert (max 60 words) for Ethan. Include: 1) what changed, 2) what it means for GeoXylia, 3) any action item.",
    "context_excerpt": "",
    "attempted_solution": "",
    "outcome": "unknown",
    "reference_kind": "rubric",
    "judge": {},
    "tags": [
      "research-cron-output"
    ],
    "source_sessions": [],
    "split": "val"
  },
  {
    "id": "rc-04",
    "reference": "Hi [Name], I saw seo-skill.com's resources page is one of the most-respected SEO learning hubs in the industry \u2014 your 2026 algorithm breakdown was spot-on. We just published a free 2026 AI SEO Audit comparison that your readers would find genuinely useful (no paywall, no signup). It covers the 8 leading AI-audit tools with hands-on screenshots and a clear feature matrix. GeoXylia is the only fully-free option in the comparison, so it's a natural fit for a 'tools to know' section. Mind if I share the link for inclusion?",
    "rubric": "Score 1.0 if exactly 4 sentences, all four functional pieces present (compliment / mention resource / audience benefit / GeoXylia one-liner), conversational tone, no aggressive sales language. Score 0.5 if 3 of 4 pieces present or tone is too salesy. Score 0.0 if more than 5 sentences or missing 2+ pieces.",
    "project": "research-cron-output",
    "intent": "Backlink Outreach draft for the blog post 'Free AI SEO Audit Tool: 2026 Comparison'. The prospect is seo-skill.com (a popular SEO training site with a 'resources' page). Write a 4-sentence outreach email: 1) compliment, 2) mention our resource, 3) explain audience benefit, 4) one-line about GeoXylia.",
    "context_excerpt": "",
    "attempted_solution": "",
    "outcome": "unknown",
    "reference_kind": "rubric",
    "judge": {},
    "tags": [
      "research-cron-output"
    ],
    "source_sessions": [],
    "split": "train"
  },
  {
    "id": "rc-05",
    "reference": "1) DO MORE: AI citation / LLM-mention topics \u2014 the 0.9% CTR at position 9.4 means we're visible but need richer answer capsules to lift CTR. Target 2x posts/week on this cluster.\n2) PAUSE: Pure schema-markup how-tos \u2014 'Schema Markup for SEO' has 0 clicks at position 41, the audience isn't searching this way. Rework as 'How to appear in AI answers' framing.\n3) TEST: 'Perplexity vs ChatGPT citation rates for [niche]' \u2014 unexplored angle, could capture comparison-intent traffic.",
    "rubric": "Score 1.0 if all 3 are specific (not generic), cite actual data from the prompt, and contain a clear actionable change. Score 0.5 if 2 of 3 are specific. Score 0.0 if generic advice or no data citations.",
    "project": "research-cron-output",
    "intent": "Performance \u2192 Strategy feedback loop. Last week's top blog post was 'AI Citation Audit: Does Your Site Appear in ChatGPT?' with 4,200 impressions and 38 clicks (CTR 0.9%, position 9.4). The bottom post was 'Schema Markup for SEO: A 2026 Guide' with 110 impressions and 0 clicks (CTR 0%, position 41). Write 3 specific strategy adjustments: 1) what to do more of, 2) what to pause, 3) what new topic to test.",
    "context_excerpt": "",
    "attempted_solution": "",
    "outcome": "unknown",
    "reference_kind": "rubric",
    "judge": {},
    "tags": [
      "research-cron-output"
    ],
    "source_sessions": [],
    "split": "val"
  }
 ]
@@ -0,0 +1,70 @@
 [
  {
    "id": "wk-01",
    "reference": "1. What GEO is and isn't (define vs SEO/AEO, dispel the 'just add FAQ' myth)\n2. The 3 citation mechanisms LLMs use (RAG, fine-tuning, in-context; weight each)\n3. The 2026 citation data (real statistics from Profound/Otterly/Peec; what % of queries get citations)\n4. The action framework (a 5-step audit-and-fix process, concrete)\n5. Measurement (which metrics actually predict citation lift; vanity vs real)",
    "rubric": "Score 1.0 if 5 sections, in a logical order, each with a substantive (not generic) purpose, and the section content is GEO-specific (not generic SEO). Score 0.5 if 5 sections but 1-2 are generic. Score 0.0 if wrong number of sections or wrong order.",
    "project": "wiki-canonical-guide",
    "intent": "Wiki canonical guide: 'GEO 2026 Standards'. Audience: a mid-level SEO specialist who has heard of GEO but not done it. Tone: technical, evidence-driven, no fluff. Length target: 1500-2200 words. Outline the 5 sections that should appear in order. For each, give a 1-sentence sub-purpose.",
    "context_excerpt": "",
    "attempted_solution": "",
    "outcome": "unknown",
    "reference_kind": "rubric",
    "judge": {},
    "tags": [
      "wiki-canonical-guide"
    ],
    "source_sessions": [],
    "split": "val"
  },
  {
    "id": "wk-02",
    "reference": "Yes, add inbound links. (1) geo-2026-standards.md \u2192 '## Action Framework' section, anchor: 'platform-specific citation rules' \u2014 natural since GEO standards reference ChatGPT/Perplexity behavior. (2) seo-2026-standards.md \u2192 '## AI Overviews' section, anchor: 'AI platform citations' \u2014 links to the mechanism guide. (3) content-strategy.md \u2192 '## Content Types' section, anchor: 'per-platform citation' \u2014 content strategy needs to know which platform favors which content.",
    "rubric": "Score 1.0 if all 3 inbound links proposed with specific section + natural anchor text, demonstrating the link solves a real navigational gap (not just SEO-link-building). Score 0.5 if 2 of 3 are well-placed. Score 0.0 if generic anchors like 'click here' or no specific sections named.",
    "project": "wiki-canonical-guide",
    "intent": "Cross-link audit. The wiki page 'ai-platform-citation-guide.md' has 4 outbound links to other wiki pages, but no inbound links from: 'geo-2026-standards.md', 'seo-2026-standards.md', 'content-strategy.md'. Should we add inbound links? In which page should each inbound link go, and what anchor text would be natural?",
    "context_excerpt": "",
    "attempted_solution": "",
    "outcome": "unknown",
    "reference_kind": "rubric",
    "judge": {},
    "tags": [
      "wiki-canonical-guide"
    ],
    "source_sessions": [],
    "split": "val"
  },
  {
    "id": "wk-03",
    "reference": "Priorities:\n1. Refresh 'geo-glossary.md' (last update 2026-04-12, 63 days) \u2014 add new terms like RAG, in-context citation, agentic SEO.\n2. Refresh 'competitor-pricing.md' (last update 2026-05-01, 44 days) \u2014 Profound raised enterprise tier.\n3. No structural fixes needed.\n\nTelegram: 'Wiki lint: 2 stale pages flagged (geo-glossary 63d, competitor-pricing 44d). No broken links. Both need refresh this week.'",
    "rubric": "Score 1.0 if both stale pages correctly identified with specific (not generic) refresh notes, and Telegram summary is under 40 words with the right action. Score 0.5 if stale pages identified but refresh notes are vague. Score 0.0 if missing stale pages or Telegram over 40 words.",
    "project": "wiki-canonical-guide",
    "intent": "Wiki lint report. Today's scan: 14 wiki pages, 2 with 'Updated' dates > 30 days old ('geo-glossary.md' and 'competitor-pricing.md'), 0 broken internal links, 0 missing YAML frontmatter. Output: 1) prioritized action list, 2) Telegram summary (max 40 words).",
    "context_excerpt": "",
    "attempted_solution": "",
    "outcome": "unknown",
    "reference_kind": "rubric",
    "judge": {},
    "tags": [
      "wiki-canonical-guide"
    ],
    "source_sessions": [],
    "split": "train"
  },
  {
    "id": "wk-04",
    "reference": "Index rebuilt: 14 wiki pages registered in _index.md (was 12 \u2014 added competitor-pricing-rev2 and citations-q2-2026).\nQuestion for Ethan: should 'competitor-pricing.md' and 'competitor-pricing-rev2.md' be merged? They're 78% similar in content.",
    "rubric": "Score 1.0 if both sentences are accurate (count matches, names are plausible) and the question identifies a real consolidation opportunity (not a fabricated one). Score 0.5 if structure is right but content vague. Score 0.0 if wrong format or no question.",
    "project": "wiki-canonical-guide",
    "intent": "Index rebuild check. Run `python3 ~/agent-shared/scripts/update-index.py` (assume it works). After the run, the new wiki/_index.md should list all 14 pages. Generate a 2-sentence confirmation message + 1 question for Ethan to verify.",
    "context_excerpt": "",
    "attempted_solution": "",
    "outcome": "unknown",
    "reference_kind": "rubric",
    "judge": {},
    "tags": [
      "wiki-canonical-guide"
    ],
    "source_sessions": [],
    "split": "train"
  }
 ]
@@ -0,0 +1,51 @@
 #!/usr/bin/env bash
 # SkillOpt-Sleep shared runner — used by all platform plugins (Claude Code,
 # Codex, Copilot). Resolves the repo root (which contains the skillopt_sleep
 # package), picks a Python >= 3.10, and execs the engine CLI.
 #
 # Usage: run-sleep.sh <run|dry-run|status|adopt|harvest|...> [args...]
 set -euo pipefail
 # This script lives at <repo>/plugins/run-sleep.sh, so the repo root (which
 # holds skillopt_sleep/) is one level up. CLAUDE_PLUGIN_ROOT (if set by Claude
 # Code) points at the plugin dir; the engine is then two levels above it.
 SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
 if [ -d "$SCRIPT_DIR/../skillopt_sleep" ]; then
  REPO_ROOT="$(cd "$SCRIPT_DIR/.." && pwd)"
 elif [ -n "${CLAUDE_PLUGIN_ROOT:-}" ] && [ -d "$CLAUDE_PLUGIN_ROOT/../../skillopt_sleep" ]; then
  REPO_ROOT="$(cd "$CLAUDE_PLUGIN_ROOT/../.." && pwd)"
 elif [ -n "${SKILLOPT_SLEEP_REPO:-}" ] && [ -d "$SKILLOPT_SLEEP_REPO/skillopt_sleep" ]; then
  REPO_ROOT="$SKILLOPT_SLEEP_REPO"
 else
  # last resort: search upward from CWD
  d="$PWD"
  while [ "$d" != "/" ]; do
    [ -d "$d/skillopt_sleep" ] && { REPO_ROOT="$d"; break; }
    d="$(dirname "$d")"
  done
 fi
 if [ -z "${REPO_ROOT:-}" ]; then
  echo "[sleep] ERROR: could not locate the skillopt_sleep package. Set SKILLOPT_SLEEP_REPO to the repo root." >&2
  exit 1
 fi
 PY=""
 # Allow explicit Python override (useful on macOS with old system Python).
 if [ -n "${SKILLOPT_SLEEP_PYTHON:-}" ]; then
  PY="$SKILLOPT_SLEEP_PYTHON"
 else
  for cand in python3.12 python3.11 python3.10 python3; do
    if command -v "$cand" >/dev/null 2>&1; then
      ver="$("$cand" -c 'import sys; print("%d%d" % sys.version_info[:2])' 2>/dev/null || echo 0)"
      if [ "${ver:-0}" -ge 310 ]; then PY="$cand"; break; fi
    fi
  done
 fi
 if [ -z "$PY" ]; then
  echo "[sleep] ERROR: need Python >= 3.10 (found none)." >&2
  exit 1
 fi
 if [ "$#" -eq 0 ]; then set -- status; fi
 cd "$REPO_ROOT"
 exec "$PY" -m skillopt_sleep "$@"
@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 [project]
 name = "skillopt"
-version = "0.1.0"
+version = "0.2.0"
 description = "SkillOpt: Agentic Skill Optimization via Reflective Training Loops"
 readme = "README.md"
 license = {text = "MIT"}
@@ -37,9 +37,11 @@ dependencies = [
 # Benchmark-specific dependencies
 alfworld = ["alfworld>=0.4.0", "gymnasium>=0.29.0"]
 # Claude model backend
-claude = ["claude-agent-sdk>=0.1.0"]
+claude = ["claude-agent-sdk>=0.1.0", "json_repair>=0.61.0"]
 # Qwen local model backend (via vLLM)
-qwen = ["vllm>=0.4.0"]
+qwen = ["vllm>=0.4.0", "json_repair>=0.61.0"]
 # SearchQA data materialization
 searchqa = ["datasets>=2.18.0"]
 # Documentation site
 docs = ["mkdocs-material>=9.5.0", "mkdocstrings[python]>=0.24.0"]
 # WebUI dashboard
@@ -51,11 +53,13 @@ all = [
    "alfworld>=0.4.0",
    "gymnasium>=0.29.0",
    "claude-agent-sdk>=0.1.0",
    "json_repair>=0.61.0",
 ]
 [project.scripts]
 skillopt-train = "scripts.train:main"
 skillopt-eval = "scripts.eval_only:main"
 skillopt-sleep = "skillopt_sleep.__main__:main"
 [project.urls]
 Homepage = "https://github.com/microsoft/SkillOpt"
@@ -64,7 +68,10 @@ Repository = "https://github.com/microsoft/SkillOpt"
 Issues = "https://github.com/microsoft/SkillOpt/issues"
 [tool.setuptools.packages.find]
-include = ["skillopt*", "scripts*"]
+# skillopt*      = the research package
 # skillopt_sleep = the open-source Sleep tool (decoupled, zero research dep)
 # skillopt_webui = the Gradio dashboard (installed via the `webui` extra)
 include = ["skillopt", "skillopt.*", "skillopt_sleep", "skillopt_sleep.*", "skillopt_webui", "skillopt_webui.*", "scripts*"]
 [tool.ruff]
 line-length = 120
@@ -17,6 +17,12 @@ httpx>=0.27.0
 # ── Optional: Qwen local model (via vLLM) ────────
 # vllm>=0.4.0
 # ── Optional: tolerant JSON repair for free-form output from non-OpenAI
 #    backends (Claude/Qwen). Without it extract_json() falls back safely and
 #    drops a malformed analyst edit instead of repairing it. Installed by the
 #    `claude`, `qwen`, and `all` extras in pyproject.toml.
 # json_repair>=0.61.0
 # ── Optional: WebUI dashboard ────────────────────
 # gradio>=4.0.0
@@ -28,6 +28,8 @@ from skillopt.model import (
    configure_azure_openai,
    configure_claude_code_exec,
    configure_codex_exec,
    configure_qwen_chat,
    configure_minimax_chat,
    set_reasoning_effort,
    set_target_backend,
    set_target_deployment,
@@ -137,7 +139,7 @@ def parse_args() -> argparse.Namespace:
    # Legacy flat overrides
    p.add_argument("--env", type=str)
    p.add_argument("--backend", type=str,
-                   choices=["azure_openai", "codex", "codex_exec", "claude", "claude_chat", "claude_code_exec"])
+                   choices=["azure_openai", "codex", "codex_exec", "claude", "claude_chat", "claude_code_exec", "minimax", "minimax_chat"])
    p.add_argument("--optimizer_model", type=str)
    p.add_argument("--target_model", type=str)
    p.add_argument("--optimizer_backend", type=str)
@@ -179,6 +181,12 @@ def parse_args() -> argparse.Namespace:
    p.add_argument("--claude_code_exec_use_sdk", type=str)
    p.add_argument("--claude_code_exec_effort", type=str)
    p.add_argument("--claude_code_exec_max_thinking_tokens", type=int)
    p.add_argument("--minimax_base_url", type=str)
    p.add_argument("--minimax_api_key", type=str)
    p.add_argument("--minimax_model", type=str)
    p.add_argument("--minimax_temperature", type=float)
    p.add_argument("--minimax_max_tokens", type=int)
    p.add_argument("--minimax_enable_thinking", type=_BOOL)
    p.add_argument("--out_root", type=str)
    p.add_argument("--data_path", type=str)
    p.add_argument("--split_mode", type=str,
@@ -254,6 +262,12 @@ def main() -> None:
                "claude_code_exec_use_sdk": "model.claude_code_exec_use_sdk",
                "claude_code_exec_effort": "model.claude_code_exec_effort",
                "claude_code_exec_max_thinking_tokens": "model.claude_code_exec_max_thinking_tokens",
                "minimax_base_url": "model.minimax_base_url",
                "minimax_api_key": "model.minimax_api_key",
                "minimax_model": "model.minimax_model",
                "minimax_temperature": "model.minimax_temperature",
                "minimax_max_tokens": "model.minimax_max_tokens",
                "minimax_enable_thinking": "model.minimax_enable_thinking",
                "seed": "train.seed",
                "test_env_num": "evaluation.test_env_num",
                "env": "env.name",
@@ -311,6 +325,9 @@ def main() -> None:
        elif backend == "claude_code_exec":
            cfg.setdefault("optimizer_backend", "openai_chat")
            cfg.setdefault("target_backend", "claude_code_exec")
        elif backend in {"minimax", "minimax_chat"}:
            cfg.setdefault("optimizer_backend", "openai_chat")
            cfg.setdefault("target_backend", "minimax_chat")
        else:
            cfg.setdefault("optimizer_backend", "openai_chat")
            cfg.setdefault("target_backend", "openai_chat")
@@ -336,6 +353,15 @@ def main() -> None:
            and not _has_model_override("model.target", "target_model")
        ):
            cfg["target_model"] = default_model_for_backend("claude_chat")
    if cfg.get("target_backend") == "minimax_chat":
        if (
            str(cfg.get("target_model", "") or "").strip() in _OPENAI_DEFAULT_MODEL_SENTINELS
            and not _has_model_override("model.target", "target_model")
        ):
            cfg["target_model"] = (
                cfg.get("minimax_model")
                or default_model_for_backend("minimax_chat")
            )
    if not cfg.get("out_root"):
        env = cfg.get("env", "unknown")
@@ -401,6 +427,30 @@ def main() -> None:
        effort=cfg.get("claude_code_exec_effort", cfg.get("reasoning_effort", "medium")),
        max_thinking_tokens=cfg.get("claude_code_exec_max_thinking_tokens", 16384),
    )
    configure_qwen_chat(
        base_url=cfg.get("qwen_chat_base_url") or None,
        api_key=cfg.get("qwen_chat_api_key") or None,
        temperature=cfg.get("qwen_chat_temperature"),
        timeout_seconds=cfg.get("qwen_chat_timeout_seconds"),
        max_tokens=cfg.get("qwen_chat_max_tokens"),
        enable_thinking=cfg.get("qwen_chat_enable_thinking"),
        target_base_url=cfg.get("target_qwen_chat_base_url") or None,
        target_api_key=cfg.get("target_qwen_chat_api_key") or None,
        target_temperature=cfg.get("target_qwen_chat_temperature"),
        target_timeout_seconds=cfg.get("target_qwen_chat_timeout_seconds"),
        target_max_tokens=cfg.get("target_qwen_chat_max_tokens"),
        target_enable_thinking=cfg.get("target_qwen_chat_enable_thinking"),
    )
    configure_minimax_chat(
        base_url=cfg.get("minimax_base_url") or None,
        api_key=cfg.get("minimax_api_key") or None,
        temperature=cfg.get("minimax_temperature"),
        max_tokens=cfg.get("minimax_max_tokens"),
        enable_thinking=cfg.get("minimax_enable_thinking"),
    )
    minimax_model_cfg = cfg.get("minimax_model")
    if minimax_model_cfg and cfg.get("target_backend") == "minimax_chat":
        set_target_deployment(str(minimax_model_cfg))
    set_reasoning_effort(cfg.get("reasoning_effort", "") or None)
    # Build adapter
@@ -0,0 +1,148 @@
 """Materialize runnable SearchQA splits from the released ID manifest."""
 from __future__ import annotations
 import argparse
 import json
 from collections.abc import Iterable, Mapping
 from pathlib import Path
 PROJECT_ROOT = Path(__file__).resolve().parent.parent
 SPLITS = ("train", "val", "test")
 REQUIRED_FIELDS = ("question", "context", "answers")
 def parse_args() -> argparse.Namespace:
    parser = argparse.ArgumentParser(description=__doc__)
    parser.add_argument(
        "--manifest-dir",
        type=Path,
        default=PROJECT_ROOT / "data" / "searchqa_id_split",
        help="Directory containing train/val/test ID manifests.",
    )
    parser.add_argument(
        "--output-dir",
        type=Path,
        default=PROJECT_ROOT / "data" / "searchqa_split",
        help="Directory to write runnable train/val/test splits.",
    )
    parser.add_argument(
        "--dataset",
        default="lucadiliello/searchqa",
        help="Hugging Face dataset repository to load.",
    )
    return parser.parse_args()
 def load_manifest_ids(manifest_dir: Path) -> dict[str, list[str]]:
    split_ids = {}
    for split in SPLITS:
        path = manifest_dir / split / "items.json"
        with path.open(encoding="utf-8") as file:
            items = json.load(file)
        split_ids[split] = [str(item["id"]) for item in items]
    return split_ids
 def _iter_dataset_rows(dataset: Mapping[str, Iterable[dict]]) -> Iterable[dict]:
    for source_split in dataset.values():
        yield from source_split
 def _normalize_row(row: dict) -> dict:
    try:
        key = str(row["key"])
    except KeyError as exc:
        raise ValueError("SearchQA source row is missing required field: key") from exc
    missing = [field for field in REQUIRED_FIELDS if field not in row]
    if missing:
        raise ValueError(f"SearchQA source row {key!r} is missing required fields: {', '.join(missing)}")
    return {
        "id": key,
        "question": row["question"],
        "context": row["context"],
        "answers": row["answers"],
    }
 def materialize_searchqa_splits(
    manifest_dir: Path,
    output_dir: Path,
    dataset: Mapping[str, Iterable[dict]],
    *,
    dataset_name: str,
 ) -> dict[str, int]:
    """Write runnable SearchQA train/val/test splits from a source dataset."""
    manifest_dir = manifest_dir.resolve()
    output_dir = output_dir.resolve()
    split_ids = load_manifest_ids(manifest_dir)
    wanted_ids = {item_id for ids in split_ids.values() for item_id in ids}
    selected: dict[str, dict] = {}
    duplicate_ids: set[str] = set()
    for row in _iter_dataset_rows(dataset):
        key = str(row.get("key", ""))
        if key not in wanted_ids:
            continue
        if key in selected:
            duplicate_ids.add(key)
            continue
        selected[key] = _normalize_row(row)
    if duplicate_ids:
        preview = ", ".join(sorted(duplicate_ids)[:5])
        raise ValueError(f"SearchQA source dataset contains duplicate manifest IDs. First IDs: {preview}")
    missing = sorted(wanted_ids - selected.keys())
    if missing:
        preview = ", ".join(missing[:5])
        raise RuntimeError(f"SearchQA source dataset is missing {len(missing)} manifest IDs. First IDs: {preview}")
    counts = {}
    for split, ids in split_ids.items():
        items = [selected[item_id] for item_id in ids]
        split_dir = output_dir / split
        split_dir.mkdir(parents=True, exist_ok=True)
        with (split_dir / "items.json").open("w", encoding="utf-8") as file:
            json.dump(items, file, ensure_ascii=False, indent=2)
        counts[split] = len(items)
    manifest = {
        "source_manifest_dir": str(manifest_dir),
        "source_dataset": dataset_name,
        "counts": counts,
        "item_fields": ["id", *REQUIRED_FIELDS],
    }
    with (output_dir / "split_manifest.json").open("w", encoding="utf-8") as file:
        json.dump(manifest, file, ensure_ascii=False, indent=2)
    return counts
 def main() -> None:
    args = parse_args()
    try:
        from datasets import load_dataset
    except ImportError as exc:
        raise SystemExit(
            "Missing dependency 'datasets'. Install it with:\n"
            "  python -m pip install 'skillopt[searchqa]'\n"
            "or:\n"
            "  python -m pip install datasets"
        ) from exc
    print(f"Loading {args.dataset}...")
    dataset = load_dataset(args.dataset)
    counts = materialize_searchqa_splits(
        args.manifest_dir,
        args.output_dir,
        dataset,
        dataset_name=args.dataset,
    )
    print(f"Wrote SearchQA splits to {args.output_dir.resolve()}: {counts}")
 if __name__ == "__main__":
    main()
@@ -245,6 +245,10 @@ def parse_args() -> argparse.Namespace:
    p.add_argument("--longitudinal_pair_policy", type=str,
                   choices=["mixed", "changed", "unchanged"])
    p.add_argument("--use_meta_skill", type=_BOOL)
    p.add_argument("--use_skill_aware_reflection", type=_BOOL)
    p.add_argument("--skill_aware_appendix_source", type=str,
                   choices=["both", "failure_only"])
    p.add_argument("--skill_aware_consolidate_threshold", type=int)
    p.add_argument("--data_path", type=str)
    p.add_argument("--split_mode", type=str,
                   choices=["ratio", "split_dir"])
@@ -360,6 +364,9 @@ _LEGACY_TO_STRUCTURED: dict[str, str] = {
    "slow_update_samples": "optimizer.slow_update_samples",
    "longitudinal_pair_policy": "optimizer.longitudinal_pair_policy",
    "use_meta_skill": "optimizer.use_meta_skill",
    "use_skill_aware_reflection": "optimizer.use_skill_aware_reflection",
    "skill_aware_appendix_source": "optimizer.skill_aware_appendix_source",
    "skill_aware_consolidate_threshold": "optimizer.skill_aware_consolidate_threshold",
    "use_gate": "evaluation.use_gate",
    "sel_env_num": "evaluation.sel_env_num",
    "test_env_num": "evaluation.test_env_num",
@@ -527,6 +534,7 @@ def main() -> None:
    print(f"  minibatch_size: {cfg.get('minibatch_size')}")
    print(f"  seed:           {cfg.get('seed')}")
    print(f"  meta_skill:     {cfg.get('use_meta_skill', False)}")
    print(f"  skill_aware_reflection: {cfg.get('use_skill_aware_reflection', False)}")
    print(f"  slow_update:    {cfg.get('use_slow_update', False)}")
    print(f"  out_root:       {cfg.get('out_root')}")
    print(f"{'='*60}\n")
@@ -2416,14 +2416,11 @@
      <div class="bibtex-box">
        <button class="copy-btn" type="button" onclick="copyBibtex(this)">Copy</button>
-        <pre><code>@misc{yang2026skilloptexecutivestrategyselfevolving,
+        <pre><code>@article{yang2026skillopt,
-      title={SkillOpt: Executive Strategy for Self-Evolving Agent Skills}, 
+  title={Skillopt: Executive strategy for self-evolving agent skills},
-      author={Yifan Yang and Ziyang Gong and Weiquan Huang and Qihao Yang and Ziwei Zhou and Zisu Huang and Yan Li and Xuemei Gao and Qi Dai and Bei Liu and Kai Qiu and Yuqing Yang and Dongdong Chen and Xue Yang and Chong Luo},
+  author={Yang, Yifan and Gong, Ziyang and Huang, Weiquan and Yang, Qihao and Zhou, Ziwei and Huang, Zisu and Li, Yan and Gao, Xuemei and Dai, Qi and Liu, Bei and others},
-      year={2026},
+  journal={arXiv preprint arXiv:2605.23904},
-      eprint={2605.23904},
+  year={2026}
      archivePrefix={arXiv},
      primaryClass={cs.AI},
      url={https://arxiv.org/abs/2605.23904}, 
 }</code></pre>
      </div>
    </section>
@@ -12,7 +12,7 @@ Pipeline stages:
  6. Evaluate  — validate candidate skill, accept/reject
 """
-__version__ = "0.1.0"
+__version__ = "0.2.0"
 from skillopt.types import (  # noqa: F401
    BatchSpec,
@@ -119,9 +119,15 @@ _FLATTEN_MAP: dict[str, str] = {
    "optimizer.slow_update_gate_with_selection": "slow_update_gate_with_selection",
    "optimizer.longitudinal_pair_policy": "longitudinal_pair_policy",
    "optimizer.use_meta_skill": "use_meta_skill",
    "optimizer.use_skill_aware_reflection": "use_skill_aware_reflection",
    "optimizer.skill_aware_appendix_source": "skill_aware_appendix_source",
    "optimizer.skill_aware_consolidate_threshold": "skill_aware_consolidate_threshold",
    "evaluation.use_gate": "use_gate",
    "evaluation.gate_metric": "gate_metric",
    "evaluation.gate_mixed_weight": "gate_mixed_weight",
    "evaluation.use_semantic_density": "use_semantic_density",
    "evaluation.semantic_density_weight": "semantic_density_weight",
    "evaluation.leading_words": "leading_words",
    "evaluation.sel_env_num": "sel_env_num",
    "evaluation.test_env_num": "test_env_num",
    "evaluation.eval_test": "eval_test",
@@ -155,7 +161,7 @@ def _load_yaml(path: str, _visited: set[str] | None = None) -> dict:
        raise ValueError(f"Circular _base_ inheritance: {abs_path}")
    _visited.add(abs_path)
-    with open(abs_path) as f:
+    with open(abs_path, encoding="utf-8") as f:
        cfg = yaml.safe_load(f) or {}
    base_ref = cfg.pop("_base_", None)
@@ -189,13 +195,6 @@ def flatten_config(cfg: dict) -> dict:
    flat: dict[str, Any] = {}
    evaluation_section = cfg.get("evaluation", {})
    if isinstance(evaluation_section, dict) and evaluation_section.get("use_gate") is False:
        raise ValueError(
            "Gate validation is mandatory in this branch. Remove "
            "`evaluation.use_gate: false` from the config."
        )
    # Apply the explicit mapping
    for dotted, flat_key in _FLATTEN_MAP.items():
        section, key = dotted.split(".", 1)
@@ -24,7 +24,7 @@ from collections import defaultdict
 from skillopt.datasets.base import BatchSpec
 from skillopt.envs.base import EnvAdapter
-from skillopt.evaluation.gate import evaluate_gate, select_gate_score
+from skillopt.evaluation.gate import GateResult, evaluate_gate, select_gate_score
 from skillopt.gradient.aggregate import merge_patches
 from skillopt.optimizer.meta_skill import run_meta_skill
 from skillopt.optimizer.clip import rank_and_select
@@ -32,6 +32,17 @@ from skillopt.optimizer.lr_autonomous import decide_autonomous_learning_rate
 from skillopt.optimizer.rewrite import rewrite_skill_from_suggestions
 from skillopt.optimizer.scheduler import build_scheduler
 from skillopt.optimizer.skill import apply_patch_with_report
 from skillopt.optimizer.appendix import (
    append_to_appendix_field,
    extract_appendix_notes as extract_appendix_notes_from_skill,
    inject_empty_appendix_field,
    _strip_all_appendix_fields,
 )
 from skillopt.optimizer.skill_aware import (
    configure_skill_aware_reflection,
    consolidate_appendix_notes,
    extract_appendix_notes as extract_appendix_notes_from_result,
 )
 from skillopt.optimizer.slow_update import (
    build_comparison_pairs,
    extract_slow_update_field,
@@ -48,6 +59,7 @@ from skillopt.optimizer.update_modes import (
    short_item_summary,
 )
 from skillopt.model import (
    chat_optimizer,
    configure_azure_openai,
    configure_claude_code_exec,
    configure_codex_exec,
@@ -64,6 +76,74 @@ from skillopt.model import (
 from skillopt.utils import compute_score, skill_hash
 # ── Skill-aware reflection: appendix flush ───────────────────────────────────
 def _flush_skill_aware_appendix(
    current_skill: str,
    all_raw_patches: list,
    step_rec: dict,
    step_dir: str,
    cfg: dict,
 ) -> str:
    """Append this step's EXECUTION_LAPSE notes into the protected appendix.
    Returns the (possibly) updated skill. Must be called on BOTH the normal
    update path and the skip branches: a lapse-only step yields no body
    patches by design (analysts return ``edits: []`` carriers), so the skip
    paths would otherwise silently drop every note of the step.
    """
    step_appendix_notes: list[str] = []
    for rp in all_raw_patches:
        if isinstance(rp, dict):
            step_appendix_notes.extend(extract_appendix_notes_from_result(rp))
    if not step_appendix_notes:
        return current_skill
    before_notes = extract_appendix_notes_from_skill(current_skill)
    current_skill = append_to_appendix_field(
        current_skill, step_appendix_notes,
    )
    after_notes = extract_appendix_notes_from_skill(current_skill)
    n_added = len(after_notes) - len(before_notes)
    step_rec["n_execution_lapse_notes"] = len(step_appendix_notes)
    step_rec["n_appendix_notes_added"] = n_added
    step_rec["n_appendix_notes_total"] = len(after_notes)
    with open(os.path.join(step_dir, "appendix_notes.json"), "w") as f:
        json.dump(
            {
                "step_notes": step_appendix_notes,
                "appendix_after": after_notes,
            },
            f, indent=2, ensure_ascii=False,
        )
    print(
        f"    [skill-aware] +{n_added} appendix note(s) "
        f"(total {len(after_notes)}) from {len(step_appendix_notes)} lapse signal(s)"
    )
    # Threshold-gated LLM consolidation (paper Eq.11): when the
    # appendix grows past N notes, compact it with one optimizer
    # call (dedupe / merge / shorten). 0 disables it. Any failure
    # leaves the appendix unchanged.
    consolidate_threshold = int(
        cfg.get("skill_aware_consolidate_threshold", 0) or 0
    )
    if consolidate_threshold > 0 and len(after_notes) > consolidate_threshold:
        compacted = consolidate_appendix_notes(
            after_notes, chat_fn=chat_optimizer,
        )
        if compacted and len(compacted) < len(after_notes):
            current_skill = append_to_appendix_field(
                _strip_all_appendix_fields(current_skill), compacted,
            )
            step_rec["n_appendix_notes_consolidated"] = len(compacted)
            step_rec["n_appendix_notes_total"] = len(compacted)
            print(
                f"    [skill-aware] consolidated appendix "
                f"{len(after_notes)} -> {len(compacted)} notes"
            )
    return current_skill
 # ── Patch normalization ───────────────────────────────────────────────────────
 def _normalise_patches(
@@ -467,7 +547,7 @@ def _format_step_buffer(buffer: list[dict]) -> str:
        # Failure patterns
        for p in entry.get("failure_patterns", []):
-            ids = ", ".join(p["task_ids"][:3])
+            ids = ", ".join(p["task_ids"])
            parts.append(f'  - "{p["pattern"]}" (×{p["count"]}, tasks: {ids})')
        # Rejected edits (only present on reject)
@@ -484,7 +564,7 @@ def _format_step_buffer(buffer: list[dict]) -> str:
                    content = e.get("content", "")
                    target = e.get("target", "")
                    if target:
-                        parts.append(f'    {i}. [{op}] target="{target[:80]}" → "{content}"')
+                        parts.append(f'    {i}. [{op}] target="{target}" → "{content}"')
                    else:
                        parts.append(f'    {i}. [{op}] "{content}"')
                else:
@@ -629,26 +709,26 @@ class ReflACTTrainer:
            effort=cfg.get("claude_code_exec_effort", cfg.get("reasoning_effort", "medium")),
            max_thinking_tokens=cfg.get("claude_code_exec_max_thinking_tokens", 16384),
        )
-        configure_qwen_chat(
+        configure_qwen_chat(
-            base_url=cfg.get("qwen_chat_base_url") or None,
+            base_url=cfg.get("qwen_chat_base_url") or None,
-            api_key=cfg.get("qwen_chat_api_key") or None,
+            api_key=cfg.get("qwen_chat_api_key") or None,
-            temperature=cfg.get("qwen_chat_temperature"),
+            temperature=cfg.get("qwen_chat_temperature"),
-            timeout_seconds=cfg.get("qwen_chat_timeout_seconds"),
+            timeout_seconds=cfg.get("qwen_chat_timeout_seconds"),
-            max_tokens=cfg.get("qwen_chat_max_tokens"),
+            max_tokens=cfg.get("qwen_chat_max_tokens"),
-            enable_thinking=cfg.get("qwen_chat_enable_thinking"),
+            enable_thinking=cfg.get("qwen_chat_enable_thinking"),
-            optimizer_base_url=cfg.get("optimizer_qwen_chat_base_url") or None,
+            optimizer_base_url=cfg.get("optimizer_qwen_chat_base_url") or None,
-            optimizer_api_key=cfg.get("optimizer_qwen_chat_api_key") or None,
+            optimizer_api_key=cfg.get("optimizer_qwen_chat_api_key") or None,
-            optimizer_temperature=cfg.get("optimizer_qwen_chat_temperature"),
+            optimizer_temperature=cfg.get("optimizer_qwen_chat_temperature"),
-            optimizer_timeout_seconds=cfg.get("optimizer_qwen_chat_timeout_seconds"),
+            optimizer_timeout_seconds=cfg.get("optimizer_qwen_chat_timeout_seconds"),
-            optimizer_max_tokens=cfg.get("optimizer_qwen_chat_max_tokens"),
+            optimizer_max_tokens=cfg.get("optimizer_qwen_chat_max_tokens"),
-            optimizer_enable_thinking=cfg.get("optimizer_qwen_chat_enable_thinking"),
+            optimizer_enable_thinking=cfg.get("optimizer_qwen_chat_enable_thinking"),
-            target_base_url=cfg.get("target_qwen_chat_base_url") or None,
+            target_base_url=cfg.get("target_qwen_chat_base_url") or None,
-            target_api_key=cfg.get("target_qwen_chat_api_key") or None,
+            target_api_key=cfg.get("target_qwen_chat_api_key") or None,
-            target_temperature=cfg.get("target_qwen_chat_temperature"),
+            target_temperature=cfg.get("target_qwen_chat_temperature"),
-            target_timeout_seconds=cfg.get("target_qwen_chat_timeout_seconds"),
+            target_timeout_seconds=cfg.get("target_qwen_chat_timeout_seconds"),
-            target_max_tokens=cfg.get("target_qwen_chat_max_tokens"),
+            target_max_tokens=cfg.get("target_qwen_chat_max_tokens"),
-            target_enable_thinking=cfg.get("target_qwen_chat_enable_thinking"),
+            target_enable_thinking=cfg.get("target_qwen_chat_enable_thinking"),
-        )
+        )
        configure_minimax_chat(
            base_url=cfg.get("minimax_base_url") or None,
            api_key=cfg.get("minimax_api_key") or None,
@@ -838,6 +918,16 @@ class ReflACTTrainer:
        _save_skill(out_root, 0, skill_init)
        use_skill_aware = cfg.get("use_skill_aware_reflection", False)
        # Publish the toggle process-wide so run_minibatch_reflect resolves it
        # from config for EVERY env adapter — no per-benchmark wiring needed.
        configure_skill_aware_reflection(
            use_skill_aware,
            cfg.get("skill_aware_appendix_source", "both"),
        )
        if use_skill_aware:
            current_skill = inject_empty_appendix_field(current_skill)
        def _persist_runtime_state(last_completed_step: int) -> None:
            _save_runtime_state(
                out_root,
@@ -863,11 +953,10 @@ class ReflACTTrainer:
                sel_cache[sh] = (rec["selection_hard"], rec["selection_soft"])
        # ── Baseline evaluation on selection set ─────────────────────────
-        if cfg.get("use_gate") is False:
+        # `use_gate=False` keeps validation running (selection rollout +
-            raise ValueError(
+        # scoring are unconditional below) but force-accepts every candidate
-                "Gate validation is mandatory in this branch. Remove "
+        # instead of gating it; final skill is chosen manually afterwards.
-                "`evaluation.use_gate=false` from the config."
+        use_gate = cfg.get("use_gate", True) is not False
            )
        gate_metric = str(cfg.get("gate_metric", "hard")).strip().lower()
        if gate_metric not in {"hard", "soft", "mixed"}:
            raise ValueError(
@@ -875,6 +964,15 @@ class ReflACTTrainer:
                f"got {gate_metric!r}"
            )
        gate_mixed_weight = float(cfg.get("gate_mixed_weight", 0.5))
        use_semantic_density = bool(cfg.get("use_semantic_density", False))
        semantic_density_weight = float(cfg.get("semantic_density_weight", 0.05))
        leading_words_raw = cfg.get("leading_words", None)
        leading_words = None
        if leading_words_raw is not None:
            if isinstance(leading_words_raw, str):
                leading_words = [w.strip() for w in leading_words_raw.split(",") if w.strip()]
            else:
                leading_words = list(leading_words_raw)
        if not 0.0 <= gate_mixed_weight <= 1.0:
            raise ValueError(
                f"evaluation.gate_mixed_weight must be in [0, 1], "
@@ -887,6 +985,8 @@ class ReflACTTrainer:
                if gate_metric == "mixed"
                else ""
            )
            + ("" if use_gate
               else "  (DISABLED → validation runs, candidates force-accepted)")
        )
        slow_gate_with_selection = bool(
            cfg.get("slow_update_gate_with_selection", False)
@@ -912,6 +1012,10 @@ class ReflACTTrainer:
            baseline_hard, baseline_soft = compute_score(baseline_results)
            current_score = select_gate_score(
                baseline_hard, baseline_soft, gate_metric, gate_mixed_weight,
                skill_content=skill_init,
                use_semantic_density=use_semantic_density,
                semantic_density_weight=semantic_density_weight,
                leading_words=leading_words,
            )
            best_score = current_score
            sh = skill_hash(skill_init)
@@ -1104,6 +1208,13 @@ class ReflACTTrainer:
                # ── No patches? Skip ─────────────────────────────────────
                if not all_failure_patches and not all_success_patches:
                    # Skill-aware: a lapse-only step has no body patches but
                    # may still carry appendix notes — flush them BEFORE
                    # skipping, or they would be silently dropped.
                    if use_skill_aware:
                        current_skill = _flush_skill_aware_appendix(
                            current_skill, all_raw_patches, step_rec, step_dir, cfg,
                        )
                    step_rec["action"] = "skip_no_patches"
                    step_rec["current_score"] = current_score
                    step_rec["best_score"] = best_score
@@ -1292,6 +1403,12 @@ class ReflACTTrainer:
                    is_full_rewrite_minibatch_mode(update_mode)
                    and rewrite_result is None
                ):
                    # Skill-aware: flush appendix notes before skipping (see
                    # the skip_no_patches branch above).
                    if use_skill_aware:
                        current_skill = _flush_skill_aware_appendix(
                            current_skill, all_raw_patches, step_rec, step_dir, cfg,
                        )
                    step_rec["action"] = "skip_no_rewrite"
                    step_rec["current_score"] = current_score
                    step_rec["best_score"] = best_score
@@ -1346,10 +1463,38 @@ class ReflACTTrainer:
                    cand_soft=cand_soft,
                    metric=gate_metric,
                    mixed_weight=gate_mixed_weight,
-                )
+                    use_semantic_density=use_semantic_density,
                    semantic_density_weight=semantic_density_weight,
                    leading_words=leading_words,
                ) if use_gate else None
                cand_gate_score = select_gate_score(
                    cand_hard, cand_soft, gate_metric, gate_mixed_weight,
                    skill_content=candidate_skill,
                    use_semantic_density=use_semantic_density,
                    semantic_density_weight=semantic_density_weight,
                    leading_words=leading_words,
                )
                if not use_gate:
                    # Validation ran (scores recorded above) but the gate is
                    # disabled: force-accept the candidate as the new current
                    # skill. Best-so-far is still tracked for convenience; the
                    # final skill is selected manually from the trajectory.
                    if cand_gate_score > best_score:
                        fa_best_skill = candidate_skill
                        fa_best_score = cand_gate_score
                        fa_best_step = global_step
                    else:
                        fa_best_skill = best_skill
                        fa_best_score = best_score
                        fa_best_step = best_step
                    gate = GateResult(
                        action="force_accept",
                        current_skill=candidate_skill,
                        current_score=cand_gate_score,
                        best_skill=fa_best_skill,
                        best_score=fa_best_score,
                        best_step=fa_best_step,
                    )
                step_rec["gate_metric"] = gate_metric
                step_rec["candidate_gate_score"] = cand_gate_score
                step_rec["action"] = gate.action
@@ -1360,11 +1505,18 @@ class ReflACTTrainer:
                best_skill = gate.best_skill
                best_score = gate.best_score
                best_step = gate.best_step
-                if gate.action in {"accept", "accept_new_best"}:
+                if gate.action in {"accept", "accept_new_best", "force_accept"}:
                    current_origin = f"step_{global_step:04d}"
-                if gate.action == "accept_new_best":
+                if gate.action == "accept_new_best" or (
                    gate.action == "force_accept" and best_step == global_step
                ):
                    best_origin = current_origin
                if use_skill_aware:
                    current_skill = _flush_skill_aware_appendix(
                        current_skill, all_raw_patches, step_rec, step_dir, cfg,
                    )
                if gate_metric == "hard":
                    score_label = f"hard={cand_hard:.4f}"
                elif gate_metric == "soft":
@@ -1384,6 +1536,11 @@ class ReflACTTrainer:
                        f"    [6/6 EVALUATE] ACCEPT "
                        f"{score_label} > current={prev_current:.4f}"
                    )
                elif gate.action == "force_accept":
                    print(
                        f"    [6/6 EVALUATE] FORCE-ACCEPT (gate disabled) "
                        f"{score_label}"
                    )
                else:
                    print(
                        f"    [6/6 EVALUATE] REJECT "
@@ -1514,13 +1671,13 @@ class ReflACTTrainer:
                        elif action in {
                            "accept", "accept_new_best", "force_accept",
                        }:
-                            # Force-accept mode: re-apply to both current & best.
+                            # Force-accept mode: re-apply guidance to
                            # current_skill only. best_skill must remain a
                            # faithful snapshot of the val-best step and must
                            # NOT receive force-injected slow-update content.
                            current_skill = replace_slow_update_field(
                                current_skill, slow_saved["slow_update_content"],
                            )
                            best_skill = replace_slow_update_field(
                                best_skill, slow_saved["slow_update_content"],
                            )
                elif epoch == 1:
                    # Epoch 1: inject empty placeholder
                    os.makedirs(slow_dir, exist_ok=True)
@@ -1528,7 +1685,7 @@ class ReflACTTrainer:
                    current_origin = f"slow_update_placeholder_epoch_{epoch:02d}"
                    _save_skill(out_root, global_step, current_skill)
                    with open(os.path.join(out_root, "best_skill.md"), "w") as f:
-                        f.write(best_skill if best_score > current_score else current_skill)
+                        f.write(best_skill)
                    with open(slow_done_path, "w") as f:
                        json.dump({"action": "inject_placeholder", "epoch": epoch}, f, indent=2)
                    _persist_runtime_state(global_step)
@@ -1707,6 +1864,9 @@ class ReflACTTrainer:
                                cand_soft=slow_sel_soft,
                                metric=gate_metric,
                                mixed_weight=gate_mixed_weight,
                                use_semantic_density=use_semantic_density,
                                semantic_density_weight=semantic_density_weight,
                                leading_words=leading_words,
                            )
                            slow_result["selection_hard"] = slow_sel_hard
                            slow_result["selection_soft"] = slow_sel_soft
@@ -1749,16 +1909,15 @@ class ReflACTTrainer:
                        else:
                            # ── Force-accept mode (default) ──────────────────
                            # The epoch-level longitudinal guidance is injected
-                            # into both current_skill and best_skill
+                            # into current_skill ONLY, so training continues
-                            # unconditionally — it must not be gated by
+                            # with the accumulated slow memory. best_skill is
-                            # step-level selection scores.
+                            # left untouched: it must remain a faithful snapshot
                            # of the val-best step (which may be a pre-slow step
                            # such as S_0 carrying no slow_update field at all).
                            slow_content = slow_result["slow_update_content"]
                            current_skill = replace_slow_update_field(
                                current_skill, slow_content,
                            )
                            best_skill = replace_slow_update_field(
                                best_skill, slow_content,
                            )
                            # Update caches so downstream steps use the
                            # slow-update-injected skill for hashing.
                            slow_candidate_hash = skill_hash(current_skill)
@@ -1769,7 +1928,7 @@ class ReflACTTrainer:
                            print(
                                f"    [slow update] force-injected into "
-                                f"current & best "
+                                f"current only "
                                f"({len(slow_content)} chars), "
                                f"{slow_time}s"
                            )
@@ -1922,10 +2081,74 @@ class ReflACTTrainer:
        baseline_test_soft = None
        test_hard = None
        test_soft = None
        final_test_hard = None
        final_test_soft = None
        final_selection_hard = None
        final_selection_soft = None
        if cfg["eval_test"]:
            task_types = adapter.get_task_types()
            # ── Final skill validation (valid_seen) + best promotion ─────
            # The final (last) skill may carry an epoch-end slow_update that
            # was force-injected WITHOUT a val pass (use_gate=false or
            # slow_update_gate_with_selection=false), so it never competed for
            # best. Run one real val on the final skill; if its gate score
            # beats the incumbent best, PROMOTE it to best so that best is the
            # true val-argmax over all skills (including the final slow_update).
            # When final == best, reuse the existing val score (no rollout).
            try:
                if skill_hash(current_skill) == skill_hash(best_skill):
                    final_selection_hard, final_selection_soft = best_score, None
                    print(
                        "\n  [final skill == best skill] "
                        f"final_selection_hard={best_score:.4f} (reused)"
                    )
                else:
                    fval_env, fval_n = _build_eval_env(
                        split="valid_seen",
                        env_num=cfg["sel_env_num"],
                        seed=seed,
                    )
                    fval_dir = os.path.join(out_root, "final_selection_eval")
                    fval_results = adapter.rollout(fval_env, current_skill, fval_dir)
                    final_selection_hard, final_selection_soft = compute_score(fval_results)
                    final_gate_score = select_gate_score(
                        final_selection_hard, final_selection_soft,
                        gate_metric, gate_mixed_weight,
                        skill_content=current_skill,
                        use_semantic_density=use_semantic_density,
                        semantic_density_weight=semantic_density_weight,
                        leading_words=leading_words,
                    )
                    print(
                        f"\n  [final skill val] items={fval_n} "
                        f"final_selection_hard={final_selection_hard:.4f} "
                        f"gate={final_gate_score:.4f} "
                        f"(best={best_score:.4f})"
                    )
                    if final_gate_score > best_score:
                        # Promote: the final (slow-updated) skill is val-better
                        # than the incumbent best. Make it the new best so the
                        # subsequent BEST-skill test rollout evaluates it and
                        # best/final test scores coincide.
                        print(
                            f"  [promote] final {final_gate_score:.4f} > "
                            f"best {best_score:.4f} → final becomes new best "
                            f"(step {global_step}, origin {current_origin})"
                        )
                        best_skill = current_skill
                        best_score = final_gate_score
                        best_step = global_step
                        best_origin = current_origin
                        with open(os.path.join(out_root, "best_skill.md"), "w") as f:
                            f.write(best_skill)
                        _persist_runtime_state(global_step)
            except Exception as _e:  # noqa: BLE001
                final_selection_hard = None
                final_selection_soft = None
                print(f"\n  [final skill val FAILED: {_e!r}]")
            # Baseline: S_0 on test set (valid_unseen)
            print(f"\n{'='*60}")
            print("  BASELINE TEST — evaluate initial skill on Test set (valid_unseen)")
@@ -1937,6 +2160,7 @@ class ReflACTTrainer:
            )
            print(f"  Test items: {test_n}")
            baseline_test_dir = os.path.join(out_root, "test_eval_baseline")
            os.makedirs(baseline_test_dir, exist_ok=True)
            baseline_test_results = adapter.rollout(test_env, skill_init, baseline_test_dir)
            baseline_test_hard, baseline_test_soft = compute_score(baseline_test_results)
            baseline_buckets = _compute_task_type_buckets(baseline_test_results, task_types)
@@ -1971,6 +2195,7 @@ class ReflACTTrainer:
            )
            print(f"  Test items: {test_n2}")
            test_dir = os.path.join(out_root, "test_eval")
            os.makedirs(test_dir, exist_ok=True)
            test_results = adapter.rollout(test_env2, best_skill, test_dir)
            test_hard, test_soft = compute_score(test_results)
            best_buckets = _compute_task_type_buckets(test_results, task_types)
@@ -1994,13 +2219,88 @@ class ReflACTTrainer:
                    f, indent=2, ensure_ascii=False,
                )
            # Final skill (last skill in trajectory) on test set.
            # Distinct from best_skill: with use_gate=False every candidate is
            # force-accepted so the final skill is whatever the last step
            # produced; with use_gate=True it is the last accepted skill, which
            # may differ from the best-on-val skill. We always evaluate it so
            # every run reports baseline / best-on-val / final on test.
            # Guarded so a failure here never prevents summary.json from being
            # written (the orchestrator's post-hoc safety net fills it in).
            try:
                if skill_hash(current_skill) == skill_hash(best_skill):
                    # Final == best: reuse results, skip a redundant rollout.
                    final_test_hard, final_test_soft = test_hard, test_soft
                    final_test_dir = os.path.join(out_root, "test_eval_final")
                    os.makedirs(final_test_dir, exist_ok=True)
                    with open(os.path.join(final_test_dir, "summary.json"), "w") as f:
                        json.dump(
                            {
                                k: {
                                    "total": b["total"],
                                    "hard_acc": b["hard"] / max(b["total"], 1),
                                }
                                for k, b in best_buckets.items()
                            },
                            f, indent=2, ensure_ascii=False,
                        )
                    print(
                        "\n  [final skill == best skill] "
                        f"final_test_hard={final_test_hard:.4f} (reused)"
                    )
                else:
                    print(f"\n{'='*60}")
                    print("  FINAL SKILL TEST — evaluate last skill on Test set (valid_unseen)")
                    print(f"{'='*60}")
                    test_env3, test_n3 = _build_eval_env(
                        split="valid_unseen",
                        env_num=cfg["test_env_num"],
                        seed=seed,
                    )
                    print(f"  Test items: {test_n3}")
                    final_test_dir = os.path.join(out_root, "test_eval_final")
                    os.makedirs(final_test_dir, exist_ok=True)
                    final_test_results = adapter.rollout(test_env3, current_skill, final_test_dir)
                    final_test_hard, final_test_soft = compute_score(final_test_results)
                    final_buckets = _compute_task_type_buckets(final_test_results, task_types)
                    print("\n  === Final Skill Test Results ===")
                    for task_type in task_types + ["overall"]:
                        b = final_buckets.get(task_type, {"total": 0, "hard": 0})
                        t = max(b["total"], 1)
                        print(
                            f"    {task_type:<40s}: "
                            f"hard={b['hard']}/{b['total']}={b['hard']/t:.4f}"
                        )
                    with open(os.path.join(final_test_dir, "summary.json"), "w") as f:
                        json.dump(
                            {
                                k: {
                                    "total": b["total"],
                                    "hard_acc": b["hard"] / max(b["total"], 1),
                                }
                                for k, b in final_buckets.items()
                            },
                            f, indent=2, ensure_ascii=False,
                        )
            except Exception as _e:  # noqa: BLE001
                final_test_hard = None
                final_test_soft = None
                print(f"\n  [final skill test FAILED: {_e!r}] "
                      "— will be filled by post-hoc eval")
            # Comparison
            delta_hard = (test_hard or 0) - (baseline_test_hard or 0)
-            print(f"\n  === Improvement (best vs baseline) ===")
+            print(f"\n  === Improvement vs baseline (init S_0) ===")
            print(
-                f"    hard: {baseline_test_hard:.4f} -> {test_hard:.4f}  "
+                f"    [2] best-on-val hard: {baseline_test_hard:.4f} -> {test_hard:.4f}  "
                f"(delta={delta_hard:+.4f})"
            )
            if final_test_hard is not None:
                final_delta_hard = (final_test_hard or 0) - (baseline_test_hard or 0)
                print(
                    f"    [3] final/last  hard: {baseline_test_hard:.4f} -> {final_test_hard:.4f}  "
                    f"(delta={final_delta_hard:+.4f})"
                )
        # ── Global summary ───────────────────────────────────────────────
        total_wall = time.time() - t_loop_start
@@ -2032,6 +2332,8 @@ class ReflACTTrainer:
                skill_hash(skill_init), (None, None),
            )[0],
            "best_selection_hard": best_score,
            "final_selection_hard": final_selection_hard,
            "final_selection_soft": final_selection_soft,
            "best_step": best_step,
            "current_origin": current_origin,
            "best_origin": best_origin,
@@ -2044,11 +2346,18 @@ class ReflACTTrainer:
            "baseline_test_soft": baseline_test_soft,
            "test_hard": test_hard,
            "test_soft": test_soft,
            "final_test_hard": final_test_hard,
            "final_test_soft": final_test_soft,
            "test_delta_hard": (
                (test_hard or 0) - (baseline_test_hard or 0)
                if test_hard is not None
                else None
            ),
            "final_test_delta_hard": (
                (final_test_hard or 0) - (baseline_test_hard or 0)
                if final_test_hard is not None
                else None
            ),
            "total_wall_time_s": round(total_wall, 1),
            "token_summary": token_summary,
        }
@@ -2069,8 +2378,22 @@ class ReflACTTrainer:
                    f"    epoch {es['epoch']}: accept={es['accepts']} reject={es['rejects']} "
                    f"best={es['best_score_at_epoch_end']:.4f}"
                )
        if baseline_test_hard is not None:
            print("\n  === TEST scores (3 skills, split=valid_unseen) ===")
            print(
                f"    [1] init/baseline (S_0)          : "
                f"test_hard={baseline_test_hard:.4f}"
            )
        if test_hard is not None:
-            print(f"  test_hard={test_hard:.4f} test_soft={test_soft:.4f}")
+            print(
                f"    [2] best-on-val (step {best_step})".ljust(37)
                + f": test_hard={test_hard:.4f} test_soft={test_soft:.4f}"
            )
        if final_test_hard is not None:
            print(
                f"    [3] final/last skill             : "
                f"test_hard={final_test_hard:.4f} test_soft={final_test_soft:.4f}"
            )
        if token_summary.get("_total"):
            t = token_summary["_total"]
            print(
@@ -4,16 +4,40 @@ This directory provides scaffold files for adding a new benchmark to SkillOpt.
 ## Files
- `env_template.py` — Environment adapter template
+- `env_template.py` — Environment adapter template (subclasses
- `loader_template.py` — Data loader template
+  `EnvAdapter`; implements the 4 abstract methods so the file is
- `config_template.yaml` — Config file template
+  instantiable out of the box — `reflect` is inherited).
 - `loader_template.py` — Data loader template (subclasses
  `SplitDataLoader`; implements `load_split_items` for `.json`/`.jsonl`).
 - `config_template.yaml` — Config file template.
 ## Usage
-1. Copy this directory: `cp -r skillopt/envs/_template skillopt/envs/your_benchmark`
+1. **Copy the directory:**
-2. Rename files: remove `_template` suffix
+   ```bash
-3. Implement the `TODO` sections
+   cp -r skillopt/envs/_template skillopt/envs/your_benchmark
-4. Register in `skillopt/envs/__init__.py`
+   ```
-5. Create config at `configs/your_benchmark/default.yaml`
+2. **Rename the files** (drop the `_template` suffix):
   ```bash
   cd skillopt/envs/your_benchmark
   mv env_template.py    adapter.py
   mv loader_template.py dataloader.py
   ```
   …and inside each file rename the classes
   (`TemplateBenchmarkEnv → YourBenchmarkAdapter`,
   `TemplateBenchmarkLoader → YourBenchmarkLoader`)
   and fix the cross-import in `adapter.py`.
 3. **Implement the TODO blocks** inside `adapter.py:rollout` and the
   `_normalize_item` helper in `dataloader.py`. (`reflect` is inherited from
   `EnvAdapter`; override it only for custom reflection logic.)
 4. **Register** the adapter — add a `try / except ImportError` block in
   `scripts/train.py`'s `_register_builtins()` mapping the registry key
   to your `YourBenchmarkAdapter` class. There is no
   `BENCHMARK_REGISTRY` dict in `skillopt/envs/__init__.py`; the live
   registry is `_ENV_REGISTRY` in `scripts/train.py`.
 5. **Create the config** at `configs/your_benchmark/default.yaml`
   (start from `config_template.yaml`). `_base_` is a **string path**,
   not a list.
-See the [documentation](../../docs/guide/new-benchmark.md) for the full guide.
+See the [Add a New Benchmark guide](../../../docs/guide/new-benchmark.md)
 for the full step-by-step with a worked `docfaithful` example.
@@ -4,27 +4,36 @@
 # Copy this file to configs/<your_benchmark>/default.yaml
 # and customize the values below.
-# Inherit global defaults
+# Inherit global defaults.
-_base_: ['../_base_/default.yaml']
+# NOTE: `_base_` is a string path, not a list.
 _base_: ../_base_/default.yaml
 # ── Environment ──────────────────────────────────
 env:
-  name: your_benchmark            # Must match registry key
+  name: your_benchmark            # Must match the key registered in scripts/train.py
-  data_path: data/your_benchmark  # Path to your data
+  # Optional: a seed skill document. Create this file yourself before the
  # first run, or omit the key to start from an empty skill.
  # skill_init: skillopt/envs/your_benchmark/skills/initial.md
  data_path: data/your_benchmark  # Path to your data (for split_mode: ratio)
  split_dir: ""                   # Set this and use split_mode: split_dir for pre-split data
  split_mode: ratio               # "ratio" or "split_dir"
-  split_ratio: "2:1:7"            # train:val:test
+  split_ratio: "2:1:7"            # train:val:test (used when split_mode: ratio)
-  exec_timeout: 120               # Per-task timeout (seconds)
+  workers: 4                      # Parallel rollout workers
  max_completion_tokens: 4096     # Cap per target-model call
  limit: 0                        # 0 = no limit; small int = debug sample
 # ── Training ─────────────────────────────────────
 train:
-  num_epochs: 4                   # Number of epochs
+  num_epochs: 4
-  batch_size: 40                  # Tasks per step (batch size)
+  batch_size: 40
  accumulation: 1
  seed: 42
 # ── Gradient (Reflection) ───────────────────────
 gradient:
  analyst_workers: 16             # Parallel reflection workers
  minibatch_size: 8
  merge_batch_size: 8
 # ── Optimizer ────────────────────────────────────
 optimizer:
@@ -39,7 +48,8 @@ evaluation:
  eval_test: true                 # Run test eval after training
 # ── Model ────────────────────────────────────────
 # Override only what differs from the inherited defaults.
 model:
-  backend: azure_openai           # azure_openai | openai_chat | claude_code_exec | qwen
+  optimizer_backend: openai_chat  # openai_chat | claude_chat | qwen_chat | minimax_chat
-  optimizer: gpt-4o
+  target_backend: openai_chat     # … plus codex_exec / claude_code_exec for target only
-  target: gpt-4o
+  reasoning_effort: medium
@@ -4,89 +4,148 @@ Benchmark Environment Template
 Copy this file and implement the TODO sections to add a new benchmark.
 The EnvAdapter is responsible for:
-1. Executing tasks using the target model + current skill document
+  1. Building per-batch environment managers (train and eval splits).
-2. Evaluating predictions against ground truth
+  2. Running rollouts under the current skill document.
-3. Returning structured results for the training loop
+  3. Reflecting on those rollouts into raw patch dicts.
  4. Reporting the distinct task types in your data (for stratified
     sampling).
 For a fully worked example see ``skillopt/envs/officeqa/``.
 """
 from __future__ import annotations
 from skillopt.datasets.base import BatchSpec
 from skillopt.envs.base import EnvAdapter
 from skillopt.envs._template.loader_template import TemplateBenchmarkLoader
 class TemplateBenchmarkEnv(EnvAdapter):
    """
    Environment adapter for <Your Benchmark Name>.
-    
+
-    Rename this class and implement the abstract methods below.
+    Rename this class. Each abstract method below is required by
    :class:`skillopt.envs.base.EnvAdapter`. The template implementations
    are minimal so this file is importable and instantiable; replace the
    TODOs with real logic.
    """
-    def __init__(self, cfg: dict):
+    def __init__(
-        super().__init__(cfg)
+        self,
-        # TODO: Initialize benchmark-specific state
+        split_dir: str = "",
-        # Example: self.tools = load_tools(cfg)
+        data_path: str = "",
        split_mode: str = "split_dir",
        split_ratio: str = "2:1:7",
        split_seed: int = 42,
        split_output_dir: str = "",
        workers: int = 4,
        analyst_workers: int = 4,
        failure_only: bool = False,
        minibatch_size: int = 8,
        edit_budget: int = 4,
        seed: int = 42,
        limit: int = 0,
        max_completion_tokens: int = 4096,
    ) -> None:
        self.workers = workers
        self.analyst_workers = analyst_workers
        self.failure_only = failure_only
        self.minibatch_size = minibatch_size
        self.edit_budget = edit_budget
        self.max_completion_tokens = int(max_completion_tokens)
        self.dataloader = TemplateBenchmarkLoader(
            split_dir=split_dir,
            data_path=data_path,
            split_mode=split_mode,
            split_ratio=split_ratio,
            split_seed=split_seed,
            split_output_dir=split_output_dir,
            seed=seed,
            limit=limit,
        )
-    async def execute(self, item, skill: str, model):
+    # ── Lifecycle hooks ────────────────────────────────────────────────
    def setup(self, cfg: dict) -> None:
        super().setup(cfg)
        self.dataloader.setup(cfg)
    def get_dataloader(self):
        return self.dataloader
    # ── Batch → env manager ────────────────────────────────────────────
    def build_env_from_batch(self, batch: BatchSpec, **kwargs):
        # Dataset-backed envs typically just pass items straight through.
        return list(batch.payload or [])
    def build_train_env(self, batch_size: int, seed: int, **kwargs):
        batch = self.dataloader.build_train_batch(
            batch_size=batch_size, seed=seed, **kwargs
        )
        return self.build_env_from_batch(batch, **kwargs)
    def build_eval_env(self, env_num: int, split: str, seed: int, **kwargs):
        batch = self.dataloader.build_eval_batch(
            env_num=env_num, split=split, seed=seed, **kwargs
        )
        return self.build_env_from_batch(batch, **kwargs)
    # ── Rollout: run episodes under current skill ──────────────────────
    def rollout(
        self,
        env_manager,
        skill_content: str,
        out_dir: str,
        **kwargs,
    ) -> list[dict]:
        """
-        Execute a single task with the target model.
+        Run a batch of episodes under the current skill.
-        Args:
+        TODO: replace this loop with your real rollout. For each item:
-            item: DataItem with .id, .input, .ground_truth, .metadata
+          1. Build the prompt using `skill_content` as the system message.
-            skill: Current skill document content (Markdown string)
+          2. Call your target model.
-            model: Target model backend instance
+          3. Score the prediction.
-
+          4. Return a dict with at minimum: ``id`` (str), ``hard`` (0|1),
-        Returns:
+             ``soft`` (float in [0, 1]). Add any env-specific extras you
-            TaskResult with prediction, score, and trajectory
+             need for reflect() — they will be preserved on
             ``RolloutResult.extras``.
        """
-        # Step 1: Build the prompt combining skill + task input
+        items: list[dict] = env_manager
-        prompt = self.build_prompt(item, skill)
+        results: list[dict] = []
        for item in items:
            # ── REPLACE THIS BLOCK WITH YOUR REAL ROLLOUT ──
            results.append(
                {
                    "id": str(item.get("id", "")),
                    "hard": 0,
                    "soft": 0.0,
                    "predicted_answer": "",
                    "question": item.get("question", ""),
                    "fail_reason": "template rollout — not implemented",
                }
            )
        return results
-        # Step 2: Call the target model
+    # ── Reflect (inherited) ─────────────────────────────────────────────
-        # TODO: Customize the message format for your benchmark
+    #
-        messages = [
+    # ``reflect`` is inherited from ``EnvAdapter``: the default delegates to
-            {"role": "system", "content": skill},
+    # ``skillopt.gradient.reflect.run_minibatch_reflect`` using your
-            {"role": "user", "content": item.input},
+    # ``analyst_error_*`` / ``analyst_success_*`` prompts. You do NOT need to
-        ]
+    # implement it — override only if your benchmark needs custom reflection.
        response = await model.generate(messages)
-        # Step 3: Parse the model response into a prediction
+    # ── Stratification hint ────────────────────────────────────────────
        prediction = self.parse_response(response.content)
-        # Step 4: Score the prediction
+    def get_task_types(self) -> list[str]:
-        score = self.evaluate(prediction, item.ground_truth)
+        """Distinct task-type strings used for stratified sampling."""
-
+        seen: list[str] = []
-        # Step 5: Return structured result
+        all_items = (
-        return {
+            self.dataloader.train_items
-            "item_id": item.id,
+            + self.dataloader.val_items
-            "prediction": prediction,
+            + self.dataloader.test_items
-            "score": score,
+        )
-            "trajectory": messages + [{"role": "assistant", "content": response.content}],
+        for item in all_items:
-        }
+            tt = str(item.get("task_type") or "template")
-
+            if tt not in seen:
-    def evaluate(self, prediction: str, ground_truth: str) -> float:
+                seen.append(tt)
-        """
+        return seen or ["template"]
        Score a prediction against the ground truth.
        Returns:
            Float between 0.0 (wrong) and 1.0 (correct)
        TODO: Implement your scoring metric. Common options:
        - Exact match: float(pred.strip().lower() == gt.strip().lower())
        - F1 score: compute token overlap
        - ANLS: for document QA tasks
        - Custom: any float in [0, 1]
        """
        # Placeholder — exact match
        return float(prediction.strip().lower() == ground_truth.strip().lower())
    def build_prompt(self, item, skill: str) -> str:
        """Combine skill document with task input."""
        return f"{skill}\n\n---\n\nQuestion: {item.input}"
    def parse_response(self, response: str) -> str:
        """
        Extract the answer from the model's raw response.
        TODO: Implement extraction logic. For example:
        - Extract text after "Answer:" 
        - Parse JSON output
        - Extract from code blocks
        """
        return response.strip()
@@ -1,103 +1,87 @@
 """
 Benchmark Data Loader Template
 ================================
-Copy this file and implement the TODO sections to load your benchmark data.
+Copy this file and implement ``load_split_items`` to load your benchmark
 data. The loader is a :class:`skillopt.datasets.base.SplitDataLoader`
 subclass — the base class handles both ``split_mode="split_dir"`` (read
 an existing train/val/test layout) and ``split_mode="ratio"`` (build the
 splits from a single raw file deterministically).
-The DataLoader is responsible for:
+For a fully worked example see
-1. Loading raw data from disk
+``skillopt/envs/officeqa/dataloader.py``.
 2. Splitting into train / validation / test sets
 3. Providing DataItem objects to the training loop
 """
 from __future__ import annotations
 import json
 from pathlib import Path
 from skillopt.datasets.base import SplitDataLoader
-class TemplateBenchmarkLoader:
+
 def _normalize_item(raw: dict) -> dict:
    """
    Normalise one raw entry into the dict shape SkillOpt expects.
    The only **hard** requirement is ``"id"`` (str). Add whatever extra
    fields your :class:`TemplateBenchmarkEnv.rollout` needs.
    """
    return {
        "id": str(raw.get("uid") or raw.get("id") or ""),
        "question": str(raw.get("question") or raw.get("prompt") or ""),
        "ground_truth": str(raw.get("ground_truth") or raw.get("answer") or ""),
        "task_type": str(raw.get("category") or raw.get("task_type") or "template"),
        # ── add benchmark-specific keys here ──
    }
 class TemplateBenchmarkLoader(SplitDataLoader):
    """
    Data loader for <Your Benchmark Name>.
-    
+
-    Rename this class and implement the methods below.
+    Subclass note: you usually only need to implement
    :meth:`load_split_items`. The base class drives ``setup(cfg)``,
    materialises ratio-mode splits, exposes ``train_items``,
    ``val_items``, ``test_items``, and builds ``BatchSpec`` objects on
    demand.
    If you want to support ``split_mode="ratio"`` (auto-split a single
    file into train/val/test), also implement
    :meth:`load_raw_items(data_path)` returning the full list of items.
    """
-    def __init__(self, data_dir: str = "data/your_benchmark", **kwargs):
+    def load_split_items(self, split_path: str) -> list[dict]:
-        self.data_dir = Path(data_dir)
+        """Load all items for one split directory.
        self.items = []
        self.splits = {}
-    def setup(self, cfg: dict):
+        ``split_path`` is e.g. ``data/your_benchmark/train/``. Return a
        list of dicts, each shaped like :func:`_normalize_item`'s output.
        """
-        Initialize the loader with config.
+        path = Path(split_path)
        Called once before training starts.
        Args:
            cfg: Dict with keys like 'split_mode', 'train_ratio', 'val_ratio', etc.
        """
        # Step 1: Load raw data
        self.items = self._load_items()
-        # Step 2: Create splits
+        json_files = sorted(path.glob("*.json"))
-        split_mode = cfg.get("split_mode", "ratio")
+        if json_files:
-        if split_mode == "ratio":
+            with json_files[0].open(encoding="utf-8") as f:
-            self._split_by_ratio(
+                payload = json.load(f)
-                train_ratio=cfg.get("train_ratio", 0.7),
+            if not isinstance(payload, list):
-                val_ratio=cfg.get("val_ratio", 0.15),
+                raise ValueError(
-            )
+                    f"Expected JSON array at top level of {json_files[0]}"
-        elif split_mode == "split_dir":
+                )
-            self._load_predefined_splits(cfg.get("split_dir", self.data_dir))
+            return [_normalize_item(row) for row in payload]
-    def _load_items(self) -> list:
+        jsonl_files = sorted(path.glob("*.jsonl"))
-        """
+        if jsonl_files:
-        Load raw data into structured items.
+            items: list[dict] = []
-        
+            with jsonl_files[0].open(encoding="utf-8") as f:
-        TODO: Implement data loading. Each item should have at minimum:
+                for line in f:
-        - id: unique identifier
+                    line = line.strip()
-        - input: the task input (question, instruction, etc.)
+                    if not line:
-        - ground_truth: the expected answer
+                        continue
-        - metadata: optional dict with extra info
+                    items.append(_normalize_item(json.loads(line)))
        Example:
            items = []
            for path in self.data_dir.glob("*.json"):
                data = json.loads(path.read_text())
                for entry in data:
                    items.append({
                        "id": entry["id"],
                        "input": entry["question"],
                        "ground_truth": entry["answer"],
                        "metadata": {"source": path.name},
                    })
            return items
        """
        raise NotImplementedError("Implement _load_items() for your benchmark")
-    def _split_by_ratio(self, train_ratio: float, val_ratio: float):
+        raise FileNotFoundError(
-        """Split items by ratio."""
+            f"No .json or .jsonl file found in {split_path}"
-        import random
+        )
        random.shuffle(self.items)
        n = len(self.items)
        n_train = int(n * train_ratio)
        n_val = int(n * val_ratio)
        self.splits = {
            "train": self.items[:n_train],
            "valid": self.items[n_train:n_train + n_val],
            "test": self.items[n_train + n_val:],
        }
-    def _load_predefined_splits(self, split_dir):
+    # Optional — only needed if you intend to use ``split_mode='ratio'``.
-        """Load from pre-split directories."""
+    # def load_raw_items(self, data_path: str) -> list[dict]:
-        # TODO: Implement if your benchmark has pre-defined splits
+    #     ...
        raise NotImplementedError
    def get_split_items(self, split: str) -> list:
        """
        Return items for a given split.
        Args:
            split: One of "train", "valid", "test"
        Returns:
            List of data items for the requested split
        """
        if split not in self.splits:
            raise ValueError(f"Unknown split '{split}'. Available: {list(self.splits.keys())}")
        return self.splits[split]
@@ -17,7 +17,6 @@ from skillopt.envs.alfworld.rollout import (
    run_alfworld_batch,
    TASKS,
 )
 from skillopt.gradient.reflect import run_minibatch_reflect
 from skillopt.utils import compute_score
@@ -425,35 +424,5 @@ class ALFWorldAdapter(EnvAdapter):
            all_results.extend(chunk_results)
        return all_results
    def reflect(
        self,
        results: list[dict],
        skill_content: str,
        out_dir: str,
        **kwargs,
    ) -> list[dict | None]:
        prediction_dir = kwargs.get("prediction_dir", os.path.join(out_dir, "predictions"))
        patches_dir = kwargs.get("patches_dir", os.path.join(out_dir, "patches"))
        random_seed = kwargs.get("random_seed")
        step_buffer_context = kwargs.get("step_buffer_context", "")
        meta_skill_context = kwargs.get("meta_skill_context", "")
        return run_minibatch_reflect(
            results=results,
            skill_content=skill_content,
            prediction_dir=prediction_dir,
            patches_dir=patches_dir,
            workers=self.analyst_workers,
            failure_only=self.failure_only,
            minibatch_size=self.minibatch_size,
            edit_budget=self.edit_budget,
            random_seed=random_seed,
            error_system=self.get_error_minibatch_prompt(),
            success_system=self.get_success_minibatch_prompt(),
            step_buffer_context=step_buffer_context,
            meta_skill_context=meta_skill_context,
        )
    def get_task_types(self) -> list[str]:
        return list(TASKS)
@@ -7,12 +7,10 @@ Provides:
 """
 from __future__ import annotations
 import concurrent.futures
 import json
 import os
 import re
 import sys
 import concurrent.futures
 import numpy as np
 from skillopt.model import chat_target
@@ -65,6 +63,25 @@ def _append_diagnostic_instruction(prompt: str, diagnostic_instruction: str) ->
    return f"{prompt}\n\n## Training Readout\n{diagnostic_instruction.strip()}\n"
 def _resolve_alfworld_gamefile(gamefile: str) -> str:
    path = os.path.expanduser(os.path.expandvars(str(gamefile)))
    if os.path.isabs(path):
        return path
    data_root = os.environ.get("ALFWORLD_DATA", "").strip()
    if not data_root:
        return path
    root = os.path.expanduser(os.path.expandvars(data_root))
    return os.path.abspath(os.path.join(root, path))
 def _resolve_alfworld_gamefiles(gamefiles: list[str] | None) -> list[str] | None:
    if gamefiles is None:
        return None
    return [_resolve_alfworld_gamefile(gamefile) for gamefile in gamefiles]
 # ── Environment builder ──────────────────────────────────────────────────────
@@ -86,9 +103,10 @@ def build_alfworld_env(
    Returns:
        env_manager: AlfWorldEnvironmentManager instance
    """
    from omegaconf import OmegaConf
    from functools import partial
    from omegaconf import OmegaConf
    from skillopt.envs.alfworld.vendor.alfworld_envs import build_alfworld_envs
    from skillopt.envs.alfworld.vendor.alfworld_projection import alfworld_projection
    from skillopt.envs.alfworld.vendor.env_manager import AlfWorldEnvironmentManager
@@ -97,6 +115,7 @@ def build_alfworld_env(
    alf_config_path = os.path.join(HERE, "vendor", "config_tw.yaml")
    env_kwargs = {"eval_dataset": eval_dataset}
    resolved_gamefiles = _resolve_alfworld_gamefiles(specific_gamefiles)
    envs = build_alfworld_envs(
        alf_config_path,
@@ -106,7 +125,7 @@ def build_alfworld_env(
        is_train=is_train,
        env_kwargs=env_kwargs,
        resources_per_worker=None,
-        gamefiles=specific_gamefiles,
+        gamefiles=resolved_gamefiles,
    )
    config = OmegaConf.create(
@@ -222,7 +241,7 @@ def run_alfworld_batch(
                if _extract_action(response) is None:
                    return idx, "<think>missing action tag</think><action>look</action>"
                return idx, response
-            except Exception as e:
+            except Exception:
                return idx, "<think>error</think><action>look</action>"
        executor = concurrent.futures.ThreadPoolExecutor(max_workers=max_api_workers)
@@ -231,7 +231,6 @@ class EnvAdapter(ABC):
            (float 0-1). May include env-specific fields.
        """
    @abstractmethod
    def reflect(
        self,
        results: list[dict],
@@ -241,15 +240,36 @@ class EnvAdapter(ABC):
    ) -> list[dict | None]:
        """Analyze rollout results and produce patches.
        Default implementation: delegate to the shared minibatch reflect
        stage. Every built-in benchmark uses this unchanged — override only
        if your environment needs custom reflection logic.
        Each returned dict conforms to :class:`~skillopt.types.RawPatch`:
        ``"patch"`` (with ``"edits"`` list) + ``"source_type"``
-        (``"failure"`` or ``"success"``).
+        (``"failure"`` or ``"success"``); ``None`` entries are filtered out.
        Returns
        -------
        list[dict | None]
            Raw analyst outputs; ``None`` entries are filtered out.
        """
        from skillopt.gradient.reflect import run_minibatch_reflect
        return run_minibatch_reflect(
            results=results,
            skill_content=skill_content,
            prediction_dir=kwargs.get(
                "prediction_dir", os.path.join(out_dir, "predictions")
            ),
            patches_dir=kwargs.get(
                "patches_dir", os.path.join(out_dir, "patches")
            ),
            workers=self.analyst_workers,
            failure_only=self.failure_only,
            minibatch_size=self.minibatch_size,
            edit_budget=self.edit_budget,
            random_seed=kwargs.get("random_seed"),
            error_system=self.get_error_minibatch_prompt(),
            success_system=self.get_success_minibatch_prompt(),
            step_buffer_context=kwargs.get("step_buffer_context", ""),
            meta_skill_context=kwargs.get("meta_skill_context", ""),
            update_mode=getattr(self, "_cfg", {}).get("skill_update_mode", "patch"),
        )
    @abstractmethod
    def get_task_types(self) -> list[str]:
@@ -1,12 +1,9 @@
 from __future__ import annotations
 import os
 from skillopt.datasets.base import BatchSpec
 from skillopt.envs.base import EnvAdapter
 from skillopt.envs.docvqa.dataloader import DocVQADataLoader
 from skillopt.envs.docvqa.rollout import run_batch
 from skillopt.gradient.reflect import run_minibatch_reflect
 class DocVQAAdapter(EnvAdapter):
@@ -84,28 +81,6 @@ class DocVQAAdapter(EnvAdapter):
            task_timeout=self.exec_timeout,
        )
    def reflect(self, results: list[dict], skill_content: str, out_dir: str, **kwargs) -> list[dict | None]:
        prediction_dir = kwargs.get("prediction_dir", os.path.join(out_dir, "predictions"))
        patches_dir = kwargs.get("patches_dir", os.path.join(out_dir, "patches"))
        random_seed = kwargs.get("random_seed")
        step_buffer_context = kwargs.get("step_buffer_context", "")
        return run_minibatch_reflect(
            results=results,
            skill_content=skill_content,
            prediction_dir=prediction_dir,
            patches_dir=patches_dir,
            workers=self.analyst_workers,
            failure_only=self.failure_only,
            minibatch_size=self.minibatch_size,
            edit_budget=self.edit_budget,
            random_seed=random_seed,
            error_system=self.get_error_minibatch_prompt(),
            success_system=self.get_success_minibatch_prompt(),
            step_buffer_context=step_buffer_context,
            update_mode=getattr(self, "_cfg", {}).get("skill_update_mode", "patch"),
        )
    def get_task_types(self) -> list[str]:
        seen: list[str] = []
        for item in self.dataloader.train_items + self.dataloader.val_items + self.dataloader.test_items:
@@ -2,10 +2,8 @@
 from __future__ import annotations
 import json
 import os
 from skillopt.datasets.base import BatchSpec
 from skillopt.gradient.reflect import run_minibatch_reflect
 from skillopt.envs.base import EnvAdapter
 from skillopt.envs.livemathematicianbench.dataloader import LiveMathematicianBenchDataLoader
 from skillopt.envs.livemathematicianbench.rollout import run_batch
@@ -127,36 +125,5 @@ class LiveMathematicianBenchAdapter(EnvAdapter):
            task_timeout=self.exec_timeout,
        )
    def reflect(
        self,
        results: list[dict],
        skill_content: str,
        out_dir: str,
        **kwargs,
    ) -> list[dict | None]:
        prediction_dir = kwargs.get("prediction_dir", os.path.join(out_dir, "predictions"))
        patches_dir = kwargs.get("patches_dir", os.path.join(out_dir, "patches"))
        random_seed = kwargs.get("random_seed")
        step_buffer_context = kwargs.get("step_buffer_context", "")
        meta_skill_context = kwargs.get("meta_skill_context", "")
        return run_minibatch_reflect(
            results=results,
            skill_content=skill_content,
            prediction_dir=prediction_dir,
            patches_dir=patches_dir,
            workers=self.analyst_workers,
            failure_only=self.failure_only,
            minibatch_size=self.minibatch_size,
            edit_budget=self.edit_budget,
            random_seed=random_seed,
            error_system=self.get_error_minibatch_prompt(),
            success_system=self.get_success_minibatch_prompt(),
            step_buffer_context=step_buffer_context,
            meta_skill_context=meta_skill_context,
            update_mode=getattr(self, "_cfg", {}).get("skill_update_mode", "patch"),
        )
    def get_task_types(self) -> list[str]:
        return self.dataloader.get_task_types()
@@ -6,7 +6,6 @@ from skillopt.datasets.base import BatchSpec
 from skillopt.envs.base import EnvAdapter
 from skillopt.envs.officeqa.dataloader import OfficeQADataLoader
 from skillopt.envs.officeqa.rollout import run_batch
 from skillopt.gradient.reflect import run_minibatch_reflect
 class OfficeQAAdapter(EnvAdapter):
@@ -104,28 +103,6 @@ class OfficeQAAdapter(EnvAdapter):
            diagnostic_instruction=kwargs.get("diagnostic_instruction", ""),
        )
    def reflect(self, results: list[dict], skill_content: str, out_dir: str, **kwargs) -> list[dict | None]:
        prediction_dir = kwargs.get("prediction_dir", os.path.join(out_dir, "predictions"))
        patches_dir = kwargs.get("patches_dir", os.path.join(out_dir, "patches"))
        random_seed = kwargs.get("random_seed")
        step_buffer_context = kwargs.get("step_buffer_context", "")
        return run_minibatch_reflect(
            results=results,
            skill_content=skill_content,
            prediction_dir=prediction_dir,
            patches_dir=patches_dir,
            workers=self.analyst_workers,
            failure_only=self.failure_only,
            minibatch_size=self.minibatch_size,
            edit_budget=self.edit_budget,
            random_seed=random_seed,
            error_system=self.get_error_minibatch_prompt(),
            success_system=self.get_success_minibatch_prompt(),
            step_buffer_context=step_buffer_context,
            update_mode=getattr(self, "_cfg", {}).get("skill_update_mode", "patch"),
        )
    def get_task_types(self) -> list[str]:
        seen: list[str] = []
        for item in self.dataloader.train_items + self.dataloader.val_items + self.dataloader.test_items:
@@ -2,13 +2,11 @@
 from __future__ import annotations
 import json
 import os
 from skillopt.datasets.base import BatchSpec
 from skillopt.envs.base import EnvAdapter
 from skillopt.envs.searchqa.dataloader import SearchQADataLoader
 from skillopt.envs.searchqa.rollout import run_batch
 from skillopt.gradient.reflect import run_minibatch_reflect
 from skillopt.model import get_target_backend
@@ -94,36 +92,5 @@ class SearchQAAdapter(EnvAdapter):
            task_timeout=self.exec_timeout,
        )
    def reflect(
        self,
        results: list[dict],
        skill_content: str,
        out_dir: str,
        **kwargs,
    ) -> list[dict | None]:
        prediction_dir = kwargs.get("prediction_dir", os.path.join(out_dir, "predictions"))
        patches_dir = kwargs.get("patches_dir", os.path.join(out_dir, "patches"))
        random_seed = kwargs.get("random_seed")
        step_buffer_context = kwargs.get("step_buffer_context", "")
        meta_skill_context = kwargs.get("meta_skill_context", "")
        return run_minibatch_reflect(
            results=results,
            skill_content=skill_content,
            prediction_dir=prediction_dir,
            patches_dir=patches_dir,
            workers=self.analyst_workers,
            failure_only=self.failure_only,
            minibatch_size=self.minibatch_size,
            edit_budget=self.edit_budget,
            random_seed=random_seed,
            error_system=self.get_error_minibatch_prompt(),
            success_system=self.get_success_minibatch_prompt(),
            step_buffer_context=step_buffer_context,
            meta_skill_context=meta_skill_context,
            update_mode=getattr(self, "_cfg", {}).get("skill_update_mode", "patch"),
        )
    def get_task_types(self) -> list[str]:
        return ["qa"]
@@ -13,20 +13,31 @@ from __future__ import annotations
 import json
 import os
 import time
-import traceback
+from collections import Counter
 from concurrent.futures import FIRST_COMPLETED, ThreadPoolExecutor, wait
-from skillopt.model import chat_target, get_target_backend, is_target_exec_backend
+from skillopt.envs.searchqa.evaluator import evaluate
 from skillopt.model import chat_target, is_target_exec_backend
 from skillopt.model.codex_harness import prepare_workspace, render_skill_md, run_target_exec
 from skillopt.prompts import load_prompt
 from skillopt.envs.searchqa.evaluator import evaluate
 # ── Prompt templates ─────────────────────────────────────────────────────────
 _MAX_CONTEXT_CHARS = 6000
 def _raise_on_systemic_failure(results: list[dict]) -> None:
    """Abort when all rollout rows failed before any agent response."""
    if not results or not all(row.get("agent_ok") is False for row in results):
        return
    reasons = Counter(str(row.get("fail_reason") or "unknown error") for row in results)
    common_reason, count = reasons.most_common(1)[0]
    raise RuntimeError(
        f"SearchQA rollout failed for all {len(results)} items before an agent "
        f"response ({count}x): {common_reason}"
    )
 def _truncate_context(context: str, max_chars: int = _MAX_CONTEXT_CHARS) -> str:
    """Truncate context at [DOC] boundaries to stay within budget."""
    if len(context) <= max_chars:
@@ -379,6 +390,7 @@ def run_batch(
    pending = [it for it in items if str(it["id"]) not in done_ids]
    if not pending:
        _raise_on_systemic_failure(existing)
        return existing
    total = len(existing) + len(pending)
@@ -478,4 +490,5 @@ def run_batch(
        finally:
            ex.shutdown(wait=False, cancel_futures=True)
    _raise_on_systemic_failure(results)
    return results
@@ -16,7 +16,6 @@ from skillopt.envs.spreadsheetbench.rollout import (
    run_spreadsheet_batch,
    run_spreadsheet_batch_codegen,
 )
 from skillopt.gradient.reflect import run_minibatch_reflect
 from skillopt.model import get_target_backend, is_target_exec_backend
@@ -156,37 +155,5 @@ class SpreadsheetBenchAdapter(EnvAdapter):
        return results
    def reflect(
        self,
        results: list[dict],
        skill_content: str,
        out_dir: str,
        **kwargs,
    ) -> list[dict | None]:
        """Analyze rollout results and produce patches (minibatch mode)."""
        prediction_dir = kwargs.get("prediction_dir", os.path.join(out_dir, "predictions"))
        patches_dir = kwargs.get("patches_dir", os.path.join(out_dir, "patches"))
        random_seed = kwargs.get("random_seed")
        step_buffer_context = kwargs.get("step_buffer_context", "")
        meta_skill_context = kwargs.get("meta_skill_context", "")
        return run_minibatch_reflect(
            results=results,
            skill_content=skill_content,
            prediction_dir=prediction_dir,
            patches_dir=patches_dir,
            workers=self.analyst_workers,
            failure_only=self.failure_only,
            minibatch_size=self.minibatch_size,
            edit_budget=self.edit_budget,
            random_seed=random_seed,
            error_system=self.get_error_minibatch_prompt(),
            success_system=self.get_success_minibatch_prompt(),
            step_buffer_context=step_buffer_context,
            meta_skill_context=meta_skill_context,
            update_mode=getattr(self, "_cfg", {}).get("skill_update_mode", "patch"),
        )
    def get_task_types(self) -> list[str]:
        return list(TASK_TYPES)
@@ -54,8 +54,8 @@ def _build_eval_feedback(verify_report: str) -> str:
    output and whether each cell is correct or wrong.
    """
    import re
-    lines = ["Your code executed successfully but produced incorrect results.",
+    wrong_lines = []
-             "The following cells have wrong values:"]
+    n_correct = 0
    for raw_line in verify_report.splitlines():
        raw_line = raw_line.strip()
        if not raw_line:
@@ -68,9 +68,14 @@ def _build_eval_feedback(verify_report: str) -> str:
        if m:
            cell, got_val, mark = m.groups()
            if mark == "✗":
-                lines.append(f"  {cell}: your output = {got_val} (WRONG)")
+                wrong_lines.append(f"  {cell}: your output = {got_val} (WRONG)")
            else:
-                lines.append(f"  {cell}: correct ✓")
+                n_correct += 1
    lines = ["Your code executed successfully but produced incorrect results.",
             "The following cells have wrong values:"]
    lines.extend(wrong_lines)
    if n_correct:
        lines.append(f"  ({n_correct} other cells are correct.)")
    lines.append(
        "\nPlease analyze the spreadsheet data more carefully and fix the code. "
        "Return a complete corrected Python script inside a ```python``` block."
@@ -26,7 +26,9 @@ from concurrent.futures import (
 import openpyxl
 from skillopt.envs.spreadsheetbench.react_agent import run_react
-from skillopt.envs.spreadsheetbench.evaluator import evaluate, _generate_cell_names
+from skillopt.envs.spreadsheetbench.evaluator import (
    evaluate, _generate_cell_names, _compare_cell_value,
 )
 from skillopt.envs.spreadsheetbench.executor import run_generated_code
@@ -87,6 +89,21 @@ def _find_test_cases(task_dir: str) -> list[tuple[str, str, str]]:
 # ── Auto-verify helper ──────────────────────────────────────────────────────
 # The official SpreadsheetBench evaluator never serialises cells to text — it
 # compares in memory and returns only a pass/fail bool. The per-cell report
 # below is a repo-local training aid (fed back to the model on retry and saved
 # into the trajectory for reflection). On most tasks the answer range is a
 # handful of cells, so the full report is tiny. But a few tasks have answer
 # ranges spanning tens of thousands of cells (e.g. 80-42 =
 # 'Consolidate_ALL'!A2:L8000 ≈ 96k cells); dumping every cell explodes the
 # report to several MB, floods the model's context and bloats conversation
 # files. We therefore apply the same head+tail character truncation the rest of
 # the codebase uses for oversized trajectory text (cf. reflect.py / slow_update.py
 # `text[:half] + "...[truncated]...\n" + text[-half:]`): keep the first and last
 # `_MAX_REPORT_CHARS // 2` chars so both the leading and trailing wrong cells
 # stay visible. Small reports are unchanged.
 _MAX_REPORT_CHARS = 12000      # head+tail char budget (~6000 head + 6000 tail)
 def _auto_verify_output(
    pred_path: str,
@@ -97,7 +114,8 @@ def _auto_verify_output(
    Returns a human-readable verification report that can be appended to the
    trajectory so the error analyst can see exactly what went wrong (e.g.
-    ``cell A1: got=None, expected=420``).
+    ``cell A1: got=None, expected=420``). Oversized reports are head+tail
    truncated to `_MAX_REPORT_CHARS` chars, matching the rest of the codebase.
    """
    if not os.path.exists(pred_path):
        return "Verification: output file does not exist."
@@ -129,11 +147,28 @@ def _auto_verify_output(
                lines.append(f"  Sheet '{sheet_name}' NOT FOUND in output.")
                continue
            n_empty_correct = 0   # empty-on-both correct cells collapsed to a count
            for cn in cell_names:
                gv = ws_gold[cn].value if ws_gold else "N/A"
                pv = ws_pred[cn].value
-                match = "✓" if repr(gv) == repr(pv) else "✗"
+                # Use the official cell comparator so this report's ✓/✗ agrees
                # with the real scorer (evaluate). repr() equality would wrongly
                # flag e.g. 5 vs 5.0 or None vs "" as mismatches and mislead the
                # model into "fixing" cells that already pass scoring.
                ok_cell = ws_gold is not None and _compare_cell_value(gv, pv)
                # Collapse only cells that are correct AND empty on both sides
                # (got=None, expected=None ✓): pure noise. Every other cell —
                # including non-empty correct cells — is listed in full; the
                # final head+tail char cap keeps the report bounded.
                if ok_cell and gv in (None, "") and pv in (None, ""):
                    n_empty_correct += 1
                    continue
                match = "✓" if ok_cell else "✗"
                lines.append(f"  {sheet_name}!{cn}: got={pv!r}, expected={gv!r} {match}")
            if n_empty_correct:
                lines.append(
                    f"  (+{n_empty_correct} empty cells correct, omitted)"
                )
        # Also check if any cells in the output contain formula strings
        formula_cells = []
@@ -159,7 +194,17 @@ def _auto_verify_output(
        wb_pred.close()
        wb_gold.close()
-    return "\n".join(lines)
+    report = "\n".join(lines)
    # Head+tail truncation, matching reflect.py / slow_update.py: keep the first
    # and last half so both leading and trailing wrong cells remain visible.
    if len(report) > _MAX_REPORT_CHARS:
        half = _MAX_REPORT_CHARS // 2
        report = (
            report[:half]
            + f"\n  ...[verification report truncated, {len(report)} chars total]...\n"
            + report[-half:]
        )
    return report
 # ── Per-task worker ──────────────────────────────────────────────────────────
@@ -43,11 +43,55 @@ class GateResult:
    best_step: int
 def compute_semantic_density(
    skill_content: str,
    leading_words: list[str] | None = None,
 ) -> float:
    """Compute the semantic density of leading words in a skill document."""
    if not skill_content or not skill_content.strip():
        return 0.0
    if leading_words is None:
        leading_words = [
            "MUST", "ALWAYS", "NEVER", "ONLY", "CRITICAL", "IMPORTANT",
            "RESOLVE", "PREFER", "ENSURE", "STRICT", "VERIFY"
        ]
    # Strip metadata comments to focus purely on instruction text
    skill = skill_content
    for start, end in [
        ("<!-- SLOW_UPDATE_START -->", "<!-- SLOW_UPDATE_END -->"),
        ("<!-- APPENDIX_START -->", "<!-- APPENDIX_END -->")
    ]:
        while True:
            s_idx = skill.find(start)
            if s_idx == -1:
                break
            e_idx = skill.find(end, s_idx)
            if e_idx == -1:
                skill = skill[:s_idx] + skill[s_idx + len(start):]
                break
            skill = skill[:s_idx] + skill[e_idx + len(end):]
    import re
    words = re.findall(r'[a-zA-Z0-9]+', skill.lower())
    if not words:
        return 0.0
    leading_set = {w.lower() for w in leading_words}
    leading_count = sum(1 for w in words if w in leading_set)
    return leading_count / len(words)
 def select_gate_score(
    hard: float,
    soft: float,
    metric: GateMetric = "hard",
    mixed_weight: float = 0.5,
    *,
    skill_content: str = "",
    use_semantic_density: bool = False,
    semantic_density_weight: float = 0.05,
    leading_words: list[str] | None = None,
 ) -> float:
    """Project (hard, soft) onto a single comparison metric.
@@ -60,17 +104,32 @@ def select_gate_score(
    mixed_weight
        For ``"mixed"``: weight given to ``soft``. Must be in ``[0, 1]``.
        Ignored for ``"hard"`` / ``"soft"``.
    skill_content
        The raw skill document content.
    use_semantic_density
        Whether to adjust the score based on semantic density of leading words.
    semantic_density_weight
        Scaling weight for the semantic density bonus.
    leading_words
        Optional custom list of high-influence words to prioritize.
    """
    if metric == "hard":
-        return float(hard)
+        score = float(hard)
-    if metric == "soft":
+    elif metric == "soft":
-        return float(soft)
+        score = float(soft)
-    if metric == "mixed":
+    elif metric == "mixed":
        w = max(0.0, min(1.0, float(mixed_weight)))
-        return (1.0 - w) * float(hard) + w * float(soft)
+        score = (1.0 - w) * float(hard) + w * float(soft)
-    raise ValueError(
+    else:
-        f"unknown gate metric {metric!r}; expected 'hard', 'soft', or 'mixed'"
+        raise ValueError(
-    )
+            f"unknown gate metric {metric!r}; expected 'hard', 'soft', or 'mixed'"
        )
    if use_semantic_density:
        density = compute_semantic_density(skill_content, leading_words)
        score += float(semantic_density_weight) * density
    return score
 def evaluate_gate(
@@ -86,6 +145,9 @@ def evaluate_gate(
    cand_soft: float = 0.0,
    metric: GateMetric = "hard",
    mixed_weight: float = 0.5,
    use_semantic_density: bool = False,
    semantic_density_weight: float = 0.05,
    leading_words: list[str] | None = None,
 ) -> GateResult:
    """Pure gate decision: compare candidate score to current/best.
@@ -111,6 +173,12 @@ def evaluate_gate(
        the original gate behavior.
    mixed_weight
        Weight on ``soft`` when ``metric == "mixed"``.
    use_semantic_density
        Whether to adjust the score based on semantic density of leading words.
    semantic_density_weight
        Scaling weight for the semantic density bonus.
    leading_words
        Optional custom list of high-influence words to prioritize.
    Returns
    -------
@@ -118,7 +186,16 @@ def evaluate_gate(
        Updated state; the caller decides what to do with it (print,
        mutate trainer state, log, etc.).
    """
-    cand_score = select_gate_score(cand_hard, cand_soft, metric, mixed_weight)
+    cand_score = select_gate_score(
        cand_hard,
        cand_soft,
        metric,
        mixed_weight,
        skill_content=candidate_skill,
        use_semantic_density=use_semantic_density,
        semantic_density_weight=semantic_density_weight,
        leading_words=leading_words,
    )
    if cand_score > current_score:
        if cand_score > best_score:
@@ -46,7 +46,7 @@ def _merge_batch(
        response, _ = chat_optimizer(
            system=system_prompt,
            user=user,
-            max_completion_tokens=64000 if is_full_rewrite_minibatch_mode(update_mode) else 4096,
+            max_completion_tokens=64000 if is_full_rewrite_minibatch_mode(update_mode) else 16384,
            retries=3,
            stage="merge",
        )
@@ -231,7 +231,7 @@ def merge_patches(
        response, _ = chat_optimizer(
            system=merge_final_prompt,
            user=user,
-            max_completion_tokens=64000 if is_full_rewrite_minibatch_mode(update_mode) else 4096,
+            max_completion_tokens=64000 if is_full_rewrite_minibatch_mode(update_mode) else 16384,
            retries=3,
            stage="merge",
        )
@@ -29,6 +29,13 @@ from concurrent.futures import ThreadPoolExecutor, as_completed
 from skillopt.model import chat_optimizer
 from skillopt.optimizer.meta_skill import format_meta_skill_context
 from skillopt.optimizer.skill_aware import (
    augment_error_prompt,
    augment_success_prompt,
    extract_appendix_notes,
    get_skill_aware_appendix_source,
    is_skill_aware_enabled,
 )
 from skillopt.optimizer.update_modes import (
    get_payload_items,
    is_full_rewrite_minibatch_mode,
@@ -43,19 +50,21 @@ from skillopt.utils import extract_json
 # ── Trajectory formatting ────────────────────────────────────────────────────
 _MAX_TRAJ_CHARS = 12_000
 def _clip_text(value, limit: int | None = None) -> str:
    """Render optional trajectory fields. Truncation is disabled: the optimizer
    is given the full content so it can see exactly what the agent saw/did.
-def _clip_text(value, limit: int) -> str:
+    ``limit`` is accepted for backward compatibility but ignored.
-    """Render optional trajectory fields safely before truncation."""
+    """
    if value is None:
        return ""
-    return str(value)[:limit]
+    return str(value)
 def fmt_trajectory(
    conversation: list[dict],
-    max_chars: int = _MAX_TRAJ_CHARS,
+    max_chars: int | None = None,
 ) -> str:
    """Format a conversation list into analyst-readable text.
@@ -69,37 +78,32 @@ def fmt_trajectory(
    lines: list[str] = []
    for item in conversation:
        if not isinstance(item, dict):
-            lines.append(f"[agent] {_clip_text(item, 500)}")
+            lines.append(f"[agent] {_clip_text(item)}")
            continue
        if item.get("type") == "tool_call":
-            cmd = _clip_text(item.get("cmd"), 500)
+            cmd = _clip_text(item.get("cmd"))
-            obs = _clip_text(item.get("obs"), 800)
+            obs = _clip_text(item.get("obs"))
            lines.append(f"[action] {cmd}")
            lines.append(f"[obs]    {obs}")
        elif "action" in item and "env_feedback" in item:
            step = item.get("step", "?")
-            reasoning = _clip_text(item.get("reasoning"), 300)
+            reasoning = _clip_text(item.get("reasoning"))
-            action = _clip_text(item.get("action"), 200)
+            action = _clip_text(item.get("action"))
-            feedback = _clip_text(item.get("env_feedback"), 500)
+            feedback = _clip_text(item.get("env_feedback"))
            if reasoning:
                lines.append(f"[step {step} think] {reasoning}")
            lines.append(f"[step {step} action] {action}")
            lines.append(f"[step {step} obs]    {feedback}")
        elif item.get("role") == "system":
            # Post-execution verification / enrichment info
-            msg = _clip_text(item.get("content"), 2000)
+            msg = _clip_text(item.get("content"))
            lines.append(f"[verification] {msg}")
        else:
-            msg = _clip_text(item.get("content"), 500)
+            msg = _clip_text(item.get("content"))
            role = item.get("role", "agent")
            lines.append(f"[{role}] {msg}")
-    text = "\n".join(lines)
+    return "\n".join(lines)
    if len(text) > max_chars:
        head = text[: max_chars // 2]
        tail = text[-max_chars // 2 :]
        text = head + "\n...[middle truncated]...\n" + tail
    return text
 # ── Minibatch trajectory formatting ──────────────────────────────────────────
@@ -157,7 +161,7 @@ def fmt_minibatch_trajectories(
        if reference_text:
            header += (
                f"\n#### Hidden Reference\n"
-                f"{reference_text[:4000]}\n"
+                f"{reference_text}\n"
            )
        # ── Append target context (what the agent saw) ──────────────
@@ -170,7 +174,7 @@ def fmt_minibatch_trajectories(
        if target_prompt:
            header += (
                f"\n#### Target System Prompt\n"
-                f"{target_prompt[:3000]}\n"
+                f"{target_prompt}\n"
            )
        user_prompt = item.get("target_user_prompt", "")
@@ -182,7 +186,7 @@ def fmt_minibatch_trajectories(
        if user_prompt:
            header += (
                f"\n#### Target User Prompt\n"
-                f"{user_prompt[:3000]}\n"
+                f"{user_prompt}\n"
            )
        if os.environ.get("REFLACT_CODEX_TRACE_TO_OPTIMIZER", "0") == "1":
@@ -214,7 +218,7 @@ def fmt_minibatch_trajectories(
        if preview:
            header += (
                f"\n#### Spreadsheet Preview\n"
-                f"{preview[:3000]}\n"
+                f"{preview}\n"
            )
        parts.append(header + "\n" + traj_text)
@@ -261,6 +265,7 @@ def run_error_analyst_minibatch(
    step_buffer_context: str = "",
    meta_skill_context: str = "",
    update_mode: str = "patch",
    skill_aware_reflection: bool = False,
 ) -> dict | None:
    """Analyze a minibatch of failed trajectories in one optimizer call.
@@ -290,6 +295,11 @@ def run_error_analyst_minibatch(
    """
    mode = normalize_update_mode(update_mode)
    actual_system = _resolve_prompt(system_prompt, "analyst_error", mode)
    # Skill-aware reflection: augment the resolved prompt at runtime so both
    # env-specific and generic analyst prompts get the defect/lapse instruction.
    # When the toggle is off this is a no-op (prompt byte-identical to baseline).
    if skill_aware_reflection and not is_full_rewrite_minibatch_mode(mode):
        actual_system = augment_error_prompt(actual_system)
    trajectories_text = fmt_minibatch_trajectories(items, prediction_dir)
    if not trajectories_text.strip():
@@ -323,16 +333,31 @@ def run_error_analyst_minibatch(
    try:
        response, _ = chat_optimizer(
            system=actual_system, user=user,
-            max_completion_tokens=64000 if is_full_rewrite_minibatch_mode(mode) else 4096,
+            max_completion_tokens=64000 if is_full_rewrite_minibatch_mode(mode) else 16384,
            retries=3,
            stage="analyst",
        )
        result = extract_json(response)
-        if result and "patch" in result:
+        if not result:
            return None
        notes = extract_appendix_notes(result) if skill_aware_reflection else []
        if "patch" in result:
            result["source_type"] = "failure"
            if not is_full_rewrite_minibatch_mode(mode):
                truncate_payload(result["patch"], edit_budget, mode)
            if skill_aware_reflection:
                result["appendix_notes"] = notes
            return result
        # Skill-aware: a batch may legitimately yield ONLY execution-lapse notes
        # (no body edit). Return a no-op patch so the notes still reach the
        # trainer via all_raw_patches; empty edits are dropped from the body
        # pipeline by _normalise_patches, so body behavior is unchanged.
        if skill_aware_reflection and notes:
            return {
                "source_type": "failure",
                "patch": {"reasoning": "execution-lapse only", "edits": []},
                "appendix_notes": notes,
            }
    except Exception:  # noqa: BLE001
        traceback.print_exc()
    return None
@@ -349,6 +374,8 @@ def run_success_analyst_minibatch(
    step_buffer_context: str = "",
    meta_skill_context: str = "",
    update_mode: str = "patch",
    skill_aware_reflection: bool = False,
    emit_appendix_notes: bool = True,
 ) -> dict | None:
    """Analyze a minibatch of successful trajectories in one optimizer call.
@@ -368,6 +395,11 @@ def run_success_analyst_minibatch(
    """
    mode = normalize_update_mode(update_mode)
    actual_system = _resolve_prompt(system_prompt, "analyst_success", mode)
    # Only augment + parse appendix notes on the success side when allowed.
    # failure_only mode (paper-faithful S_app) suppresses success-side notes.
    sa_emit = skill_aware_reflection and emit_appendix_notes
    if sa_emit and not is_full_rewrite_minibatch_mode(mode):
        actual_system = augment_success_prompt(actual_system)
    trajectories_text = fmt_minibatch_trajectories(items, prediction_dir)
    if not trajectories_text.strip():
@@ -398,7 +430,7 @@ def run_success_analyst_minibatch(
    try:
        response, _ = chat_optimizer(
            system=actual_system, user=user,
-            max_completion_tokens=64000 if is_full_rewrite_minibatch_mode(mode) else 4096,
+            max_completion_tokens=64000 if is_full_rewrite_minibatch_mode(mode) else 16384,
            retries=3,
            stage="analyst",
        )
@@ -407,6 +439,8 @@ def run_success_analyst_minibatch(
            result["source_type"] = "success"
            if not is_full_rewrite_minibatch_mode(mode):
                truncate_payload(result["patch"], edit_budget, mode)
            if sa_emit:
                result["appendix_notes"] = extract_appendix_notes(result)
            return result
    except Exception:  # noqa: BLE001
        traceback.print_exc()
@@ -453,6 +487,8 @@ def run_minibatch_reflect(
    step_buffer_context: str = "",
    meta_skill_context: str = "",
    update_mode: str = "patch",
    skill_aware_reflection: bool | None = None,
    skill_aware_appendix_source: str | None = None,
 ) -> list[dict | None]:
    """Full minibatch reflect stage: group → parallel optimizer calls → patches.
@@ -487,6 +523,14 @@ def run_minibatch_reflect(
    list[dict | None]
        Patch dicts (with ``source_type`` "failure" or "success").
    """
    # Resolve the skill-aware toggle: explicit kwargs win; otherwise fall back
    # to the process-wide config switch set by the trainer, so the feature is
    # env-independent and adapters need no per-benchmark wiring.
    if skill_aware_reflection is None:
        skill_aware_reflection = is_skill_aware_enabled()
    if skill_aware_appendix_source is None:
        skill_aware_appendix_source = get_skill_aware_appendix_source()
    os.makedirs(patches_dir, exist_ok=True)
    # Separate failure / success
@@ -542,6 +586,7 @@ def run_minibatch_reflect(
            trajectory_memory_context=trajectory_memory_context,
            meta_skill_context=meta_skill_context,
            update_mode=update_mode,
            skill_aware_reflection=skill_aware_reflection,
        )
        return f"minibatch_fail_{idx:03d}", patch
@@ -554,6 +599,8 @@ def run_minibatch_reflect(
            trajectory_memory_context=trajectory_memory_context,
            meta_skill_context=meta_skill_context,
            update_mode=update_mode,
            skill_aware_reflection=skill_aware_reflection,
            emit_appendix_notes=(skill_aware_appendix_source != "failure_only"),
        )
        return f"minibatch_succ_{idx:03d}", patch
@@ -13,13 +13,13 @@ from skillopt.model.backend_config import (  # noqa: F401
    configure_codex_exec,
    get_claude_code_exec_config,
    get_codex_exec_config,
    get_target_backend,
    get_optimizer_backend,
    get_target_backend,
    is_optimizer_chat_backend,
    is_target_chat_backend,
    is_target_exec_backend,
    is_optimizer_chat_backend,
    set_target_backend,
    set_optimizer_backend,
    set_target_backend,
 )
@@ -105,6 +105,16 @@ def chat_optimizer(
            reasoning_effort=reasoning_effort,
            timeout=timeout,
        )
    if get_optimizer_backend() == "minimax_chat":
        return _minimax.chat_optimizer(
            system=system,
            user=user,
            max_completion_tokens=max_completion_tokens,
            retries=retries,
            stage=stage,
            reasoning_effort=reasoning_effort,
            timeout=timeout,
        )
    return _openai.chat_optimizer(
        system=system,
        user=user,
@@ -142,6 +152,7 @@ def chat_target(
            retries=retries,
            stage=stage,
            reasoning_effort=reasoning_effort,
            timeout=timeout,
        )
    if get_target_backend() == "minimax_chat":
        return _minimax.chat_target(
@@ -203,6 +214,18 @@ def chat_optimizer_messages(
            return_message=return_message,
            timeout=timeout,
        )
    if get_optimizer_backend() == "minimax_chat":
        return _minimax.chat_target_messages(
            messages=messages,
            max_completion_tokens=max_completion_tokens,
            retries=retries,
            stage=stage,
            reasoning_effort=reasoning_effort,
            tools=tools,
            tool_choice=tool_choice,
            return_message=return_message,
            timeout=timeout,
        )
    return _openai.chat_optimizer_messages(
        messages=messages,
        max_completion_tokens=max_completion_tokens,
@@ -249,6 +272,7 @@ def chat_target_messages(
            tools=tools,
            tool_choice=tool_choice,
            return_message=return_message,
            timeout=timeout,
        )
    if get_target_backend() == "minimax_chat":
        return _minimax.chat_target_messages(
@@ -438,18 +462,21 @@ def configure_qwen_chat(
    timeout_seconds: float | str | None = None,
    max_tokens: int | str | None = None,
    enable_thinking: bool | str | None = None,
    use_max_completion_tokens: bool | str | None = None,
    optimizer_base_url: str | None = None,
    optimizer_api_key: str | None = None,
    optimizer_temperature: float | str | None = None,
    optimizer_timeout_seconds: float | str | None = None,
    optimizer_max_tokens: int | str | None = None,
    optimizer_enable_thinking: bool | str | None = None,
    optimizer_use_max_completion_tokens: bool | str | None = None,
    target_base_url: str | None = None,
    target_api_key: str | None = None,
    target_temperature: float | str | None = None,
    target_timeout_seconds: float | str | None = None,
    target_max_tokens: int | str | None = None,
    target_enable_thinking: bool | str | None = None,
    target_use_max_completion_tokens: bool | str | None = None,
 ) -> None:
    _qwen.configure_qwen_chat(
        base_url=base_url,
@@ -458,18 +485,21 @@ def configure_qwen_chat(
        timeout_seconds=timeout_seconds,
        max_tokens=max_tokens,
        enable_thinking=enable_thinking,
        use_max_completion_tokens=use_max_completion_tokens,
        optimizer_base_url=optimizer_base_url,
        optimizer_api_key=optimizer_api_key,
        optimizer_temperature=optimizer_temperature,
        optimizer_timeout_seconds=optimizer_timeout_seconds,
        optimizer_max_tokens=optimizer_max_tokens,
        optimizer_enable_thinking=optimizer_enable_thinking,
        optimizer_use_max_completion_tokens=optimizer_use_max_completion_tokens,
        target_base_url=target_base_url,
        target_api_key=target_api_key,
        target_temperature=target_temperature,
        target_timeout_seconds=target_timeout_seconds,
        target_max_tokens=target_max_tokens,
        target_enable_thinking=target_enable_thinking,
        target_use_max_completion_tokens=target_use_max_completion_tokens,
    )
@@ -252,13 +252,25 @@ def _run_claude_print(*, system: str, prompt: str, model: str, tools: list[dict[
        if CLAUDE_SETTING_SOURCES:
            cmd.extend(["--setting-sources", CLAUDE_SETTING_SOURCES])
        if system:
-            cmd.extend(["--append-system-prompt", system])
+            # Write the system prompt to a file, not argv: here the skill being
            # optimized IS the system prompt, and SkillOpt grows it over training,
            # so past ~30 KB it would re-hit the Windows argv cap (WinError 206).
            # The CLI reads it via --append-system-prompt-file.
            system_path = os.path.join(temp_dir, "system_prompt.txt")
            with open(system_path, "w", encoding="utf-8") as system_fh:
                system_fh.write(system)
            cmd.extend(["--append-system-prompt-file", system_path])
        if effort:
            cmd.extend(["--effort", effort])
        structured_output = bool(return_message)
        if structured_output:
            cmd.extend(["--schema", _assistant_message_schema_wrapper()])
-        proc = subprocess.run(cmd + [prompt_for_cli], capture_output=True, text=True, timeout=timeout or 300, cwd=temp_dir)
+        # Feed the prompt via stdin (and the system prompt via a file, above), not
        # argv: on Windows the whole command line is capped at ~32 KB and large
        # optimizer prompts / grown skills overflow it → [WinError 206]. Pin UTF-8
        # so a zh-CN default codepage (cp936) can't raise UnicodeEncodeError on
        # emoji / non-GBK glyphs before the CLI even starts.
        proc = subprocess.run(cmd, input=prompt_for_cli, capture_output=True, text=True, encoding="utf-8", errors="replace", timeout=timeout or 300, cwd=temp_dir)
        stderr_text = (proc.stderr or "").strip()
        if proc.returncode != 0:
            _check_claude_error(stderr_text, model)
@@ -328,6 +328,8 @@ def _run_codex_exec(
            command,
            input=prompt,
            text=True,
            encoding="utf-8",
            errors="replace",
            capture_output=True,
            timeout=timeout,
            check=False,
@@ -222,7 +222,7 @@ def chat_target(
    stage: str = "target",
    reasoning_effort: str | None = None,
    timeout: float | None = None,
-) -> tuple[str, dict[str, int]]:
+) -> tuple[str, dict[int]]:
    del reasoning_effort
    messages = [{"role": "system", "content": system}, {"role": "user", "content": user}]
    return _chat_messages_impl(
@@ -234,6 +234,32 @@ def chat_target(
    )
 def chat_optimizer(
    system: str,
    user: str,
    max_completion_tokens: int = 16384,
    retries: int = 5,
    stage: str = "optimizer",
    reasoning_effort: str | None = None,
    timeout: float | None = None,
 ) -> tuple[str, dict[int]]:
    """Optimizer chat call. Backend stores the trained skill; uses the same
    MiniMax-proxied OpenAI-compat endpoint as `chat_target`. Added in the
    parallel-training fix; previously missing in skillopt 0.2.0's
    miniamax backend, which forced the dispatcher into _openai.chat_optimizer
    (Azure) and produced "[skip] no usable patches" for any user running
    optimizer+target on `minimax_chat`.
    """
    messages = [{"role": "system", "content": system}, {"role": "user", "content": user}]
    return _chat_messages_impl(
        messages,
        max_completion_tokens,
        retries,
        stage,
        timeout=timeout,
    )
 def chat_target_messages(
    messages: list[dict[str, Any]],
    max_completion_tokens: int = 16384,
@@ -1,13 +1,14 @@
 """OpenAI-compatible Qwen chat backend for optimizer and target paths."""
 from __future__ import annotations
 from dataclasses import dataclass
 import json
 import os
 import threading
 import time
 import urllib.error
 import urllib.request
 from dataclasses import dataclass
 from typing import Any
 from skillopt.model.common import (
@@ -28,6 +29,7 @@ class QwenChatConfig:
    temperature: float | None
    enable_thinking: bool
    deployment: str
    use_max_completion_tokens: bool = False
 def _parse_bool(value: Any, default: bool = False) -> bool:
@@ -56,6 +58,28 @@ def _role_env(role: str, key: str, default: str) -> str:
    return os.environ.get(role_key) or os.environ.get(generic_key) or default
 # Sentinels that mean "omit this optional parameter from the request payload".
 # Reasoning models (e.g. GPT-5.x, Claude Opus 4.8) reject an explicit
 # `temperature`, so allow it to be turned off via an empty string / none / off.
 _OMIT_SENTINELS = {"", "none", "off", "null"}
 def _resolve_temperature(role: str) -> float | None:
    """Return the temperature, or None to omit it entirely.
    Unlike ``_role_env`` an *explicitly set* empty (or ``none``/``off``) value is
    honored as "omit" instead of collapsing to the default. Precedence:
    role-specific env -> generic env -> 0.7 default.
    """
    for key in (f"{role.upper()}_QWEN_CHAT_TEMPERATURE", "QWEN_CHAT_TEMPERATURE"):
        if key in os.environ:
            raw = os.environ[key].strip()
            if raw.lower() in _OMIT_SENTINELS:
                return None
            return float(raw)
    return 0.7
 def _initial_config(role: str) -> QwenChatConfig:
    role_upper = role.upper()
    deployment_env = "OPTIMIZER_DEPLOYMENT" if role == "optimizer" else "TARGET_DEPLOYMENT"
@@ -64,8 +88,9 @@ def _initial_config(role: str) -> QwenChatConfig:
        api_key=_role_env(role, "API_KEY", ""),
        timeout_seconds=float(_role_env(role, "TIMEOUT_SECONDS", "300") or 300),
        max_tokens=_parse_int(_role_env(role, "MAX_TOKENS", "8000"), 8000),
-        temperature=_parse_optional_float(_role_env(role, "TEMPERATURE", "0.7")),
+        temperature=_resolve_temperature(role),
        enable_thinking=_parse_bool(_role_env(role, "ENABLE_THINKING", "false")),
        use_max_completion_tokens=_parse_bool(_role_env(role, "USE_MAX_COMPLETION_TOKENS", "false")),
        deployment=(
            os.environ.get(f"{role_upper}_QWEN_CHAT_MODEL")
            or os.environ.get("QWEN_CHAT_MODEL")
@@ -186,12 +211,17 @@ def _chat_messages_impl(
    timeout: float | None = None,
 ) -> tuple[Any, dict[str, int]]:
    config = OPTIMIZER_CONFIG if role == "optimizer" else TARGET_CONFIG
    token_limit = min(max_completion_tokens, config.max_tokens)
    # Reasoning models on some gateways (GPT-5.x, o-series) require
    # `max_completion_tokens` and reject the legacy `max_tokens`.
    token_key = "max_completion_tokens" if config.use_max_completion_tokens else "max_tokens"
    payload: dict[str, Any] = {
        "model": deployment or config.deployment,
        "messages": _json_safe(messages),
-        "max_tokens": min(max_completion_tokens, config.max_tokens),
+        token_key: token_limit,
    }
-    payload["chat_template_kwargs"] = {"enable_thinking": config.enable_thinking}
+    if config.enable_thinking:
        payload["chat_template_kwargs"] = {"enable_thinking": True}
    if config.temperature is not None:
        payload["temperature"] = config.temperature
    if tools:
@@ -218,7 +248,7 @@ def _chat_messages_impl(
            return text, usage_info
        except Exception as e:  # noqa: BLE001
            last_err = e
-            time.sleep(min(2 ** attempt, 30))
+            time.sleep(min(2**attempt, 30))
    raise RuntimeError(f"Qwen chat call failed after {retries} retries: {last_err}")
@@ -230,18 +260,21 @@ def configure_qwen_chat(
    timeout_seconds: float | str | None = None,
    max_tokens: int | str | None = None,
    enable_thinking: bool | str | None = None,
    use_max_completion_tokens: bool | str | None = None,
    optimizer_base_url: str | None = None,
    optimizer_api_key: str | None = None,
    optimizer_temperature: float | str | None = None,
    optimizer_timeout_seconds: float | str | None = None,
    optimizer_max_tokens: int | str | None = None,
    optimizer_enable_thinking: bool | str | None = None,
    optimizer_use_max_completion_tokens: bool | str | None = None,
    target_base_url: str | None = None,
    target_api_key: str | None = None,
    target_temperature: float | str | None = None,
    target_timeout_seconds: float | str | None = None,
    target_max_tokens: int | str | None = None,
    target_enable_thinking: bool | str | None = None,
    target_use_max_completion_tokens: bool | str | None = None,
 ) -> None:
    with _config_lock:
        if base_url is not None:
@@ -255,29 +288,24 @@ def configure_qwen_chat(
        if max_tokens is not None:
            os.environ["QWEN_CHAT_MAX_TOKENS"] = str(max_tokens)
        if enable_thinking is not None:
-            os.environ["QWEN_CHAT_ENABLE_THINKING"] = (
+            os.environ["QWEN_CHAT_ENABLE_THINKING"] = "true" if _parse_bool(enable_thinking) else "false"
-                "true" if _parse_bool(enable_thinking) else "false"
+        if use_max_completion_tokens is not None:
            os.environ["QWEN_CHAT_USE_MAX_COMPLETION_TOKENS"] = (
                "true" if _parse_bool(use_max_completion_tokens) else "false"
            )
        _update_config(
            OPTIMIZER_CONFIG,
            "optimizer",
            base_url=optimizer_base_url if optimizer_base_url is not None else base_url,
            api_key=optimizer_api_key if optimizer_api_key is not None else api_key,
-            temperature=(
+            temperature=(optimizer_temperature if optimizer_temperature is not None else temperature),
-                optimizer_temperature
+            timeout_seconds=(optimizer_timeout_seconds if optimizer_timeout_seconds is not None else timeout_seconds),
                if optimizer_temperature is not None
                else temperature
            ),
            timeout_seconds=(
                optimizer_timeout_seconds
                if optimizer_timeout_seconds is not None
                else timeout_seconds
            ),
            max_tokens=optimizer_max_tokens if optimizer_max_tokens is not None else max_tokens,
-            enable_thinking=(
+            enable_thinking=(optimizer_enable_thinking if optimizer_enable_thinking is not None else enable_thinking),
-                optimizer_enable_thinking
+            use_max_completion_tokens=(
-                if optimizer_enable_thinking is not None
+                optimizer_use_max_completion_tokens
-                else enable_thinking
+                if optimizer_use_max_completion_tokens is not None
                else use_max_completion_tokens
            ),
        )
        _update_config(
@@ -286,16 +314,13 @@ def configure_qwen_chat(
            base_url=target_base_url if target_base_url is not None else base_url,
            api_key=target_api_key if target_api_key is not None else api_key,
            temperature=target_temperature if target_temperature is not None else temperature,
-            timeout_seconds=(
+            timeout_seconds=(target_timeout_seconds if target_timeout_seconds is not None else timeout_seconds),
                target_timeout_seconds
                if target_timeout_seconds is not None
                else timeout_seconds
            ),
            max_tokens=target_max_tokens if target_max_tokens is not None else max_tokens,
-            enable_thinking=(
+            enable_thinking=(target_enable_thinking if target_enable_thinking is not None else enable_thinking),
-                target_enable_thinking
+            use_max_completion_tokens=(
-                if target_enable_thinking is not None
+                target_use_max_completion_tokens
-                else enable_thinking
+                if target_use_max_completion_tokens is not None
                else use_max_completion_tokens
            ),
        )
@@ -310,6 +335,7 @@ def _update_config(
    timeout_seconds: float | str | None = None,
    max_tokens: int | str | None = None,
    enable_thinking: bool | str | None = None,
    use_max_completion_tokens: bool | str | None = None,
 ) -> None:
    env_prefix = role.upper()
    if base_url is not None:
@@ -320,7 +346,7 @@ def _update_config(
        os.environ[f"{env_prefix}_QWEN_CHAT_API_KEY"] = config.api_key
    if temperature is not None:
        raw = str(temperature).strip()
-        config.temperature = float(raw) if raw else None
+        config.temperature = None if raw.lower() in _OMIT_SENTINELS else float(raw)
        os.environ[f"{env_prefix}_QWEN_CHAT_TEMPERATURE"] = raw
    if timeout_seconds is not None:
        config.timeout_seconds = float(timeout_seconds)
@@ -330,8 +356,11 @@ def _update_config(
        os.environ[f"{env_prefix}_QWEN_CHAT_MAX_TOKENS"] = str(max_tokens)
    if enable_thinking is not None:
        config.enable_thinking = _parse_bool(enable_thinking)
-        os.environ[f"{env_prefix}_QWEN_CHAT_ENABLE_THINKING"] = (
+        os.environ[f"{env_prefix}_QWEN_CHAT_ENABLE_THINKING"] = "true" if config.enable_thinking else "false"
-            "true" if config.enable_thinking else "false"
+    if use_max_completion_tokens is not None:
        config.use_max_completion_tokens = _parse_bool(use_max_completion_tokens)
        os.environ[f"{env_prefix}_QWEN_CHAT_USE_MAX_COMPLETION_TOKENS"] = (
            "true" if config.use_max_completion_tokens else "false"
        )
@@ -0,0 +1,156 @@
 """Skill-Aware Reflection — protected appendix field (EmbodiSkill S_app).
 EmbodiSkill (paper 2605.10332v1) splits a skill into ``S = (S_body, S_app)``:
 the body holds the main prescriptive rules; the appendix only *emphasizes*
 existing valid rules that the executor failed to follow (EXECUTION_LAPSE), and
 **never introduces new rules**.
 This module owns the appendix region of the skill document. It mirrors the
 protected-field pattern of :mod:`skillopt.optimizer.slow_update`, with two
 differences:
 1. **Append semantics** (not replace): execution-lapse reminders accumulate
   across steps within a run, so new notes are merged into the existing
   appendix rather than overwriting it.
 2. **Lightweight dedup**: near-duplicate reminders are collapsed (inspired by
   GMemory's ``_dedupe_preserve_order``) so the appendix stays compact.
 The appendix lives **inside** the skill markdown, between dedicated markers, so
 it is persisted by the normal ``_save_skill`` path and is resume-safe. Step-level
 analyst edits cannot modify it (enforced by the shared protected-region check in
 :mod:`skillopt.optimizer.skill`).
 Public API
 ----------
 - :func:`has_appendix_field`        — check if markers are present
 - :func:`inject_empty_appendix_field` — add empty placeholder (skill init)
 - :func:`extract_appendix_notes`    — read current notes as a list
 - :func:`append_to_appendix_field`  — merge new notes (dedup) into the region
 """
 from __future__ import annotations
 import re
 # ── Protected field markers ─────────────────────────────────────────────────
 APPENDIX_START = "<!-- APPENDIX_START -->"
 APPENDIX_END = "<!-- APPENDIX_END -->"
 # Heading shown inside the rendered appendix block (human-readable only).
 APPENDIX_HEADING = "## Execution Notes Appendix"
 # Each note is rendered as a markdown bullet so the target model reads it as
 # ordinary guidance.
 _NOTE_BULLET_PREFIX = "- "
 # ── Dedup helpers ───────────────────────────────────────────────────────────
 def _canonicalize(text: str) -> str:
    """Normalize a note for duplicate detection (whitespace/punct/case-insensitive)."""
    normalized = re.sub(r"\s+", " ", str(text or "").strip())
    normalized = normalized.rstrip(" .;:,_-")
    return normalized.casefold()
 def _dedupe_preserve_order(notes: list[str]) -> list[str]:
    """Drop blanks and near-duplicates, preserving first-seen order."""
    seen: set[str] = set()
    deduped: list[str] = []
    for note in notes:
        text = re.sub(r"\s+", " ", str(note).strip())
        if not text:
            continue
        key = _canonicalize(text)
        if not key or key in seen:
            continue
        seen.add(key)
        deduped.append(text)
    return deduped
 # ── Field manipulation ──────────────────────────────────────────────────────
 def has_appendix_field(skill: str) -> bool:
    return APPENDIX_START in skill and APPENDIX_END in skill
 def _render_block(notes: list[str]) -> str:
    """Render the full marker-delimited appendix block for *notes*."""
    lines = [APPENDIX_START, APPENDIX_HEADING]
    for note in notes:
        lines.append(f"{_NOTE_BULLET_PREFIX}{note}")
    lines.append(APPENDIX_END)
    return "\n".join(lines)
 def inject_empty_appendix_field(skill: str) -> str:
    """Add an empty appendix placeholder at the end of *skill* (idempotent).
    Mirrors ``inject_empty_slow_update_field``: called once at skill init so the
    protected region exists before any note is written.
    """
    if has_appendix_field(skill):
        return skill
    block = f"\n\n{APPENDIX_START}\n{APPENDIX_HEADING}\n{APPENDIX_END}\n"
    return skill.rstrip() + block
 def extract_appendix_notes(skill: str) -> list[str]:
    """Return the current appendix notes as a list of strings (no markers/heading)."""
    start = skill.find(APPENDIX_START)
    end = skill.find(APPENDIX_END)
    if start == -1 or end == -1:
        return []
    inner = skill[start + len(APPENDIX_START):end].strip()
    notes: list[str] = []
    for raw_line in inner.splitlines():
        line = raw_line.strip()
        if not line:
            continue
        if line == APPENDIX_HEADING or line.lstrip("#").strip() == APPENDIX_HEADING.lstrip("#").strip():
            continue
        if line.startswith(_NOTE_BULLET_PREFIX):
            line = line[len(_NOTE_BULLET_PREFIX):].strip()
        elif line.startswith("-") or line.startswith("*"):
            line = line[1:].strip()
        if line:
            notes.append(line)
    return notes
 def _strip_all_appendix_fields(skill: str) -> str:
    """Remove every appendix marker pair (and content between) from *skill*."""
    while True:
        start = skill.find(APPENDIX_START)
        if start == -1:
            break
        end = skill.find(APPENDIX_END, start)
        if end == -1:
            skill = skill[:start] + skill[start + len(APPENDIX_START):]
            break
        skill = skill[:end + len(APPENDIX_END)].rsplit(APPENDIX_START, 1)[0] + skill[end + len(APPENDIX_END):]
    skill = skill.replace(APPENDIX_END, "")
    while "\n\n\n" in skill:
        skill = skill.replace("\n\n\n", "\n\n")
    return skill.rstrip()
 def append_to_appendix_field(skill: str, new_notes: list[str]) -> str:
    """Merge *new_notes* into the appendix region (dedup), returning updated skill.
    - If no appendix region exists yet, one is created.
    - Existing notes are preserved; new ones are appended after dedup against the
      combined set, so order is stable and duplicates are dropped.
    - Empty / whitespace-only notes are ignored. If the merged set is empty, an
      empty placeholder region is still ensured.
    """
    incoming = _dedupe_preserve_order(list(new_notes or []))
    existing = extract_appendix_notes(skill)
    merged = _dedupe_preserve_order(existing + incoming)
    base = _strip_all_appendix_fields(skill)
    block = _render_block(merged)
    return f"{base}\n\n{block}\n"
@@ -57,7 +57,7 @@ def rank_and_select(
    # Build the edit pool description for the optimizer
    edits_desc = []
    for i, edit in enumerate(edits):
-        edits_desc.append(f"[{i}] {describe_item(edit, update_mode, max_chars=500)}")
+        edits_desc.append(f"[{i}] {describe_item(edit, update_mode)}")
    user = (
        f"## Current Skill\n{skill_content}\n\n"
@@ -74,7 +74,7 @@ def rank_and_select(
    try:
        response, _ = chat_optimizer(
            system=load_prompt(prompt_name), user=user,
-            max_completion_tokens=2048, retries=3, stage="ranking",
+            max_completion_tokens=16384, retries=3, stage="ranking",
        )
        result = extract_json(response)
        if result and "selected_indices" in result:
@@ -48,7 +48,7 @@ def decide_autonomous_learning_rate(
    items = get_payload_items(merged_patch, update_mode)
    available = len(items)
    item_lines = [
-        f"[{idx}] {describe_item(item, update_mode, max_chars=700)}"
+        f"[{idx}] {describe_item(item, update_mode)}"
        for idx, item in enumerate(items)
    ]
    user = (
@@ -76,7 +76,7 @@ def decide_autonomous_learning_rate(
        response, _ = chat_optimizer(
            system=load_prompt("lr_autonomous"),
            user=user,
-            max_completion_tokens=2048,
+            max_completion_tokens=16384,
            retries=3,
            stage="lr_autonomous",
        )
@@ -14,25 +14,62 @@ if TYPE_CHECKING:
 SLOW_UPDATE_START = "<!-- SLOW_UPDATE_START -->"
 SLOW_UPDATE_END = "<!-- SLOW_UPDATE_END -->"
 # Skill-aware reflection (EmbodiSkill S_app) appendix region. Like the slow
 # update region, it is protected: step-level analyst edits must not modify it.
 APPENDIX_START = "<!-- APPENDIX_START -->"
 APPENDIX_END = "<!-- APPENDIX_END -->"
-def _is_in_slow_update_region(skill: str, target: str) -> bool:
+# All protected (start, end) marker pairs. Step-level edits cannot target text
-    """Check if *target* text falls within the protected slow update region."""
+# inside any of these regions, and `append` / `insert_after`-fallback ops are
-    start_idx = skill.find(SLOW_UPDATE_START)
+# inserted before the earliest-occurring region so protected blocks stay at the
-    end_idx = skill.find(SLOW_UPDATE_END)
+# document tail. With only the slow-update region present, every helper reduces
-    if start_idx == -1 or end_idx == -1:
+# to the original slow-update-only behavior (byte-identical skill output).
 _PROTECTED_REGIONS: tuple[tuple[str, str], ...] = (
    (SLOW_UPDATE_START, SLOW_UPDATE_END),
    (APPENDIX_START, APPENDIX_END),
 )
 def _earliest_protected_start(skill: str) -> int:
    """Index of the earliest protected-region start marker, or -1 if none."""
    positions = [
        idx
        for idx in (skill.find(start) for start, _ in _PROTECTED_REGIONS)
        if idx != -1
    ]
    return min(positions) if positions else -1
 def _is_in_protected_region(skill: str, target: str) -> bool:
    """Check if *target* text falls within any protected region."""
    if not target:
        return False
    target_idx = skill.find(target)
    if target_idx == -1:
        return False
-    region_end = end_idx + len(SLOW_UPDATE_END)
+    for start_marker, end_marker in _PROTECTED_REGIONS:
-    return start_idx <= target_idx < region_end
+        start_idx = skill.find(start_marker)
        end_idx = skill.find(end_marker)
        if start_idx == -1 or end_idx == -1:
            continue
        region_end = end_idx + len(end_marker)
        if start_idx <= target_idx < region_end:
            return True
    return False
 def _is_in_slow_update_region(skill: str, target: str) -> bool:
    """Backward-compatible alias kept for any external callers/tests."""
    return _is_in_protected_region(skill, target)
 def _strip_slow_update_markers(text: str) -> str:
-    """Remove any SLOW_UPDATE markers from edit content to prevent duplication."""
+    """Remove any protected-region markers from edit content to prevent duplication."""
    return (
        text.replace(SLOW_UPDATE_START, "")
            .replace(SLOW_UPDATE_END, "")
            .replace(APPENDIX_START, "")
            .replace(APPENDIX_END, "")
    )
@@ -54,27 +91,27 @@ def _apply_edit_with_report(skill: str, edit: EditType | dict) -> tuple[str, dic
        "status": "unknown",
    }
-    if target and _is_in_slow_update_region(skill, target):
+    if target and _is_in_protected_region(skill, target):
-        report["status"] = "skipped_protected_slow_update_region"
+        report["status"] = "skipped_protected_region"
        return skill, report
    if op == "append":
-        su_start = skill.find(SLOW_UPDATE_START)
+        prot_start = _earliest_protected_start(skill)
-        if su_start != -1:
+        if prot_start != -1:
-            before = skill[:su_start].rstrip()
+            before = skill[:prot_start].rstrip()
-            after = skill[su_start:]
+            after = skill[prot_start:]
-            report["status"] = "applied_append_before_slow_update"
+            report["status"] = "applied_append_before_protected_region"
            return before + "\n\n" + content + "\n\n" + after, report
        report["status"] = "applied_append"
        return skill.rstrip() + "\n\n" + content + "\n", report
    if op == "insert_after":
        if not target or target not in skill:
-            su_start = skill.find(SLOW_UPDATE_START)
+            prot_start = _earliest_protected_start(skill)
-            if su_start != -1:
+            if prot_start != -1:
-                before = skill[:su_start].rstrip()
+                before = skill[:prot_start].rstrip()
-                after = skill[su_start:]
+                after = skill[prot_start:]
-                report["status"] = "applied_insert_after_fallback_before_slow_update"
+                report["status"] = "applied_insert_after_fallback_before_protected_region"
                return before + "\n\n" + content + "\n\n" + after, report
            report["status"] = "applied_insert_after_fallback_append"
            return skill.rstrip() + "\n\n" + content + "\n", report
@@ -0,0 +1,206 @@
 """Skill-Aware Reflection — analyst prompt augmentation (EmbodiSkill).
 When ``use_skill_aware_reflection`` is enabled, the failure/success analysts are
 asked to additionally classify each reflection by EmbodiSkill type and to route
 **EXECUTION_LAPSE** reflections (the skill rule is correct, the executor just
 failed to follow it) into a separate ``appendix_notes`` list instead of the body
 patch. This module owns:
 1. the instruction text appended to the resolved analyst system prompt, and
 2. extraction of ``appendix_notes`` from the analyst JSON response.
 Design notes
 ------------
 - The suffix is appended **at runtime, gated by the toggle**, so env-specific and
  generic analyst prompts are augmented uniformly and — when the toggle is off —
  remain byte-identical to baseline.
 - Discrimination follows the paper / GMemory: ``SKILL_DEFECT`` = the skill rule is
  wrong / missing / underspecified (→ body edit); ``EXECUTION_LAPSE`` = the rule
  is valid but the agent didn't follow it (→ appendix reminder, body untouched).
  **When unsure, default to EXECUTION_LAPSE** (protect the body — never delete a
  valid rule over a one-off execution slip).
 - Success reflections are labeled DISCOVERY / OPTIMIZATION for logging only; their
  edit behavior is unchanged.
 """
 from __future__ import annotations
 # ── Runtime switch (config-driven, env-independent) ─────────────────────────
 #
 # The trainer calls :func:`configure_skill_aware_reflection` once at startup
 # from the resolved config. ``run_minibatch_reflect`` then picks these values
 # up automatically, so env adapters never need to thread the toggle through —
 # the feature is controlled purely by ``optimizer.use_skill_aware_reflection``
 # regardless of benchmark. Mirrors the ``configure_azure_openai`` pattern in
 # :mod:`skillopt.model`. Explicit kwargs at a call site still take precedence
 # (backward compatible).
 _RUNTIME: dict = {"enabled": False, "appendix_source": "both"}
 def configure_skill_aware_reflection(
    enabled: bool,
    appendix_source: str = "both",
 ) -> None:
    """Set the process-wide skill-aware reflection switch from config."""
    _RUNTIME["enabled"] = bool(enabled)
    _RUNTIME["appendix_source"] = str(appendix_source or "both")
 def is_skill_aware_enabled() -> bool:
    return bool(_RUNTIME["enabled"])
 def get_skill_aware_appendix_source() -> str:
    return str(_RUNTIME["appendix_source"])
 # ── Prompt suffixes ─────────────────────────────────────────────────────────
 # Appended to the FAILURE analyst system prompt when the toggle is on.
 ERROR_SUFFIX = """
 ## Skill-Aware Reflection (EmbodiSkill)
 Before proposing body edits, classify EACH failure pattern as one of:
 - **SKILL_DEFECT**: the current skill is wrong, missing, or underspecified for
  this situation — i.e. an agent that *followed the skill* would still fail, or
  the skill gives no relevant guidance. These become normal body `edits`.
 - **EXECUTION_LAPSE**: the skill ALREADY contains a relevant, correct rule that
  would have avoided the failure, but the agent did not follow it (e.g. ignored a
  rule, malformed output, copied the feedback text verbatim, emitted a non-action
  token like "stop", or otherwise broke execution unrelated to skill content).
 Discrimination test: "Is there a rule in the current skill that, if followed,
 prevents this failure?" If yes → EXECUTION_LAPSE. If no (rule absent/wrong) →
 SKILL_DEFECT. **When genuinely unsure, choose EXECUTION_LAPSE** — do not edit or
 delete a valid rule over a one-off execution slip.
 Routing:
 - SKILL_DEFECT → put the fix in `patch.edits` (body), as usual.
 - EXECUTION_LAPSE → put a concise reminder in `appendix_notes` (a flat list of
  strings). DO NOT add a body edit for it. Each note should re-emphasize the
  existing valid rule the agent failed to follow; it must NOT introduce a new
  rule. Keep notes short, concrete, and reusable.
 Add `appendix_notes` as a TOP-LEVEL key of your JSON output (a sibling of
 `patch`), e.g. `"appendix_notes": ["Follow the existing X rule before Y."]`.
 Use `[]` when there is no execution lapse. Body edits and appendix notes are
 independent: a batch may yield only edits, only notes, both, or neither.
 """
 # Appended to the SUCCESS analyst system prompt when the toggle is on.
 SUCCESS_SUFFIX = """
 ## Skill-Aware Reflection (EmbodiSkill)
 For each proposed edit, optionally label its `reflection_type` for logging:
 - **DISCOVERY**: a useful new rule not yet in the skill (typically an `append`).
 - **OPTIMIZATION**: a better way to perform an existing rule (typically a
  `replace` of that rule).
 This labeling does not change edit behavior. You may also add a top-level
 `appendix_notes` list (flat strings) if a successful trajectory reveals an
 existing valid rule worth re-emphasizing; otherwise use `[]`.
 """
 def augment_error_prompt(system_prompt: str) -> str:
    """Append the failure-analyst skill-aware instruction."""
    return system_prompt.rstrip() + "\n" + ERROR_SUFFIX
 def augment_success_prompt(system_prompt: str) -> str:
    """Append the success-analyst skill-aware instruction."""
    return system_prompt.rstrip() + "\n" + SUCCESS_SUFFIX
 # ── Response parsing ────────────────────────────────────────────────────────
 def extract_appendix_notes(result: dict | None) -> list[str]:
    """Pull a clean list of appendix-note strings from an analyst JSON result.
    Tolerant of shape: accepts a top-level ``appendix_notes`` list, a single
    string, or items wrapped in dicts with a ``note``/``content`` field. Returns
    ``[]`` for anything missing or malformed (so a non-compliant model degrades
    gracefully to baseline body-only behavior).
    """
    if not isinstance(result, dict):
        return []
    raw = result.get("appendix_notes")
    if raw is None:
        return []
    if isinstance(raw, str):
        raw = [raw]
    if not isinstance(raw, list):
        return []
    notes: list[str] = []
    for item in raw:
        if isinstance(item, str):
            text = item.strip()
        elif isinstance(item, dict):
            text = str(item.get("note") or item.get("content") or "").strip()
        else:
            text = ""
        if text:
            notes.append(text)
    return notes
 # ── Appendix consolidation (threshold-gated, paper Eq.11 UpdateSkillAppendix) ──
 _CONSOLIDATE_SYSTEM = (
    "You compact the Execution Notes Appendix of an agent skill. Each note "
    "re-emphasizes an existing skill rule the agent failed to follow. Your job "
    "is a periodic compaction pass: remove duplicates and redundant overlap, "
    "merge near-identical reminders into one, and simplify phrasing while keeping "
    "each note concrete and operational. Do not invent new rules. Preserve the "
    "distinct actionable content. Return valid JSON only."
 )
 def consolidate_appendix_notes(
    notes: list[str],
    *,
    chat_fn,
    max_completion_tokens: int = 4096,
 ) -> list[str]:
    """LLM-consolidate appendix notes: dedupe / merge / compact.
    Mirrors GMemory ``_maybe_refactor_execution_notes`` and paper Eq.11. ``chat_fn``
    is the optimizer chat callable ``(system, user, max_completion_tokens, retries,
    stage) -> (text, meta)``. On ANY failure (parse, empty, exception) the original
    notes are returned unchanged, so consolidation can never lose the appendix.
    """
    from skillopt.utils import extract_json  # local import to avoid cycles
    clean = [str(n).strip() for n in (notes or []) if str(n).strip()]
    if len(clean) < 2:
        return clean
    numbered = "\n".join(f"{i}. {n}" for i, n in enumerate(clean, 1))
    user = (
        f"## Current Execution Notes ({len(clean)} total)\n{numbered}\n\n"
        "Compact these into a shorter list without losing distinct actionable "
        "information. Merge duplicates and near-duplicates; keep each note short, "
        "concrete, and reusable. Return valid JSON only with this schema:\n"
        '{ "appendix_notes": ["compacted note 1", "compacted note 2"] }'
    )
    try:
        response, _ = chat_fn(
            system=_CONSOLIDATE_SYSTEM,
            user=user,
            max_completion_tokens=max_completion_tokens,
            retries=2,
            stage="appendix_consolidate",
        )
        result = extract_json(response)
        compacted = extract_appendix_notes(result)
        # Guard: only accept a non-empty result that actually shrinks the set.
        if compacted and len(compacted) <= len(clean):
            return compacted
    except Exception:  # noqa: BLE001
        pass
    return clean
@@ -91,18 +91,21 @@ def replace_slow_update_field(skill: str, new_content: str) -> str:
 # ── Comparison text builder ─────────────────────────────────────────────────
-# NOTE: The character limits below (whole-trajectory cap + the per-field caps in
+# NOTE: Character-length limits on the comparison samples fed to the slow-update /
-# _read_trajectory and the comparison metadata) only trim the comparison samples
+# meta-skill optimizer have been REMOVED. Previously a whole-trajectory cap plus
-# fed to the slow-update optimizer. They exist to cut token usage and speed up the
+# per-field caps (cmd/obs/reasoning/etc.) and comparison-metadata caps
-# call; they do NOT affect what gets written into the skill. If you need richer
+# (task/answer/fail_reason) trimmed this context to save optimizer tokens and
-# context for the longitudinal comparison, feel free to raise them.
+# speed up the call. They never affected what gets written into the skill — only
-_MAX_TRAJ_CHARS = 3000
+# how much longitudinal context the optimizer sees. We now pass everything through
 # at full length: the comparison input is as long as the source data is.
-def _clip_text(value, limit: int) -> str:
+def _clip_text(value, limit: int | None = None) -> str:
    # Truncation disabled: return the full text. The `limit` argument is kept only
    # for call-site compatibility and is intentionally ignored (see NOTE above).
    if value is None:
        return ""
-    return str(value)[:limit]
+    return str(value)
 def _read_trajectory(rollout_dir: str, task_id: str) -> str:
@@ -122,35 +125,32 @@ def _read_trajectory(rollout_dir: str, task_id: str) -> str:
    for entry in conversation:
        if not isinstance(entry, dict):
            continue
-        # Per-field caps (cmd/obs/reasoning/etc.) keep each trajectory compact to
+        # Per-field truncation removed: feed each step's full cmd/obs/reasoning/
-        # save tokens / time; raise them if you want fuller step detail.
+        # action/feedback/content (see NOTE above).
        if entry.get("type") == "tool_call":
-            cmd = _clip_text(entry.get("cmd"), 500)
+            cmd = _clip_text(entry.get("cmd"))
-            obs = _clip_text(entry.get("obs"), 800)
+            obs = _clip_text(entry.get("obs"))
            lines.append(f"[action] {cmd}")
            lines.append(f"[obs]    {obs}")
        elif "action" in entry and "env_feedback" in entry:
            step = entry.get("step", "?")
-            reasoning = _clip_text(entry.get("reasoning"), 300)
+            reasoning = _clip_text(entry.get("reasoning"))
-            action = _clip_text(entry.get("action"), 200)
+            action = _clip_text(entry.get("action"))
-            feedback = _clip_text(entry.get("env_feedback"), 500)
+            feedback = _clip_text(entry.get("env_feedback"))
            if reasoning:
                lines.append(f"[step {step} think] {reasoning}")
            lines.append(f"[step {step} action] {action}")
            lines.append(f"[step {step} obs]    {feedback}")
        elif entry.get("role") == "system":
-            msg = _clip_text(entry.get("content"), 1000)
+            msg = _clip_text(entry.get("content"))
            lines.append(f"[verification] {msg}")
        else:
-            msg = _clip_text(entry.get("content"), 500)
+            msg = _clip_text(entry.get("content"))
            role = entry.get("role", "agent")
            lines.append(f"[{role}] {msg}")
-    text = "\n".join(lines)
+    # Whole-trajectory truncation removed: return the full formatted trajectory.
-    if len(text) > _MAX_TRAJ_CHARS:
+    return "\n".join(lines)
        half = _MAX_TRAJ_CHARS // 2
        text = text[:half] + "\n...[truncated]...\n" + text[-half:]
    return text
 # ── Structured comparison pairs ─────────────────────────────────────────────
@@ -228,7 +228,7 @@ def save_comparison_pairs(pairs: list[dict], out_path: str) -> None:
    for p in pairs:
        slim.append({
            "id": p["id"],
-            "task": p["task"][:300],
+            "task": p["task"],
            "category": p["category"],
            "prev": p["prev"],
            "curr": p["curr"],
@@ -276,16 +276,16 @@ def format_comparison_text(pairs: list[dict]) -> str:
            prev = e["prev"]
            curr = e["curr"]
            lines.append(
-                f"\n#### Task {e['id']}: {e['task'][:300]}\n"
+                f"\n#### Task {e['id']}: {e['task']}\n"
                f"- Prev epoch: {'PASS' if prev['hard'] else 'FAIL'} "
-                f"(soft={prev['soft']:.2f}) — answer: {str(prev['predicted_answer'])[:200]}\n"
+                f"(soft={prev['soft']:.2f}) — answer: {str(prev['predicted_answer'])}\n"
                f"- Curr epoch: {'PASS' if curr['hard'] else 'FAIL'} "
-                f"(soft={curr['soft']:.2f}) — answer: {str(curr['predicted_answer'])[:200]}"
+                f"(soft={curr['soft']:.2f}) — answer: {str(curr['predicted_answer'])}"
            )
            if curr.get("fail_reason"):
-                lines.append(f"- Curr fail reason: {curr['fail_reason'][:300]}")
+                lines.append(f"- Curr fail reason: {curr['fail_reason']}")
            if prev.get("fail_reason") and not prev["hard"]:
-                lines.append(f"- Prev fail reason: {prev['fail_reason'][:300]}")
+                lines.append(f"- Prev fail reason: {prev['fail_reason']}")
            if show_traj:
                if e.get("prev_trajectory"):
@@ -70,7 +70,7 @@ def truncate_payload(container: dict, max_items: int, mode: str | None) -> dict:
    return container
-def describe_item(item: dict, mode: str | None, *, max_chars: int = 240) -> str:
+def describe_item(item: dict, mode: str | None, *, max_chars: int | None = None) -> str:
    if not isinstance(item, dict):
        return ""
    if is_full_rewrite_minibatch_mode(mode):
@@ -84,7 +84,7 @@ def describe_item(item: dict, mode: str | None, *, max_chars: int = 240) -> str:
            parts.append(f"support={item.get('support_count')}")
        new_skill = str(item.get("new_skill", "")).strip()
        if new_skill:
-            parts.append(f"new_skill_preview={new_skill[:120]!r}")
+            parts.append(f"new_skill_preview={new_skill!r}")
        text = "  ".join(parts)
    elif is_rewrite_mode(mode):
        parts = [
@@ -109,28 +109,27 @@ def describe_item(item: dict, mode: str | None, *, max_chars: int = 240) -> str:
        if item.get("support_count") is not None:
            parts.append(f"support={item.get('support_count')}")
        text = "  ".join(parts)
-    if len(text) <= max_chars:
+    # Truncation disabled: the optimizer is given the full item description.
-        return text
+    return text
    return text[: max_chars - 3].rstrip() + "..."
-def short_item_summary(item: dict, mode: str | None, *, max_chars: int = 200) -> dict[str, Any]:
+def short_item_summary(item: dict, mode: str | None, *, max_chars: int | None = None) -> dict[str, Any]:
    if is_full_rewrite_minibatch_mode(mode):
        return {
-            "title": str(item.get("title", ""))[:max_chars],
+            "title": str(item.get("title", "")),
            "change_summary": [
-                str(x)[:max_chars] for x in item.get("change_summary", [])[:3]
+                str(x) for x in item.get("change_summary", [])
            ] if isinstance(item.get("change_summary"), list) else [],
            "source_type": item.get("source_type", ""),
        }
    if is_rewrite_mode(mode):
        return {
            "type": item.get("type", "?"),
-            "title": str(item.get("title", ""))[:max_chars],
+            "title": str(item.get("title", "")),
-            "instruction": str(item.get("instruction", ""))[:max_chars],
+            "instruction": str(item.get("instruction", "")),
        }
    return {
        "op": item.get("op", "?"),
-        "content": str(item.get("content", ""))[:max_chars],
+        "content": str(item.get("content", "")),
        "target": item.get("target", ""),
    }
@@ -3,6 +3,91 @@ from __future__ import annotations
 import json
 import re
 import warnings
 def _top_level_brace_objects(text: str) -> list[str]:
    """Return every balanced *top-level* ``{...}`` span in ``text``.
    Fully string/escape aware: braces inside quoted strings are ignored both
    when scanning for an object start AND while tracking depth inside one, so a
    ``{`` that appears in prose (e.g. ``'set it to {x}'``) is never mistaken for
    the start of a JSON object. Used to detect ambiguity: when a response carries
    more than one top-level object we must not let a repair pass silently pick
    one — it may pick the wrong (discarded) edit, strictly worse than None.
    """
    spans: list[str] = []
    i, n = 0, len(text)
    outer_in_str = False
    outer_esc = False
    while i < n:
        ch = text[i]
        # Skip over braces that live *inside* a quoted string before any object
        # has started — otherwise a `{` in prose like '"set it to {x}"' is wrongly
        # treated as an object start, and the repair pass below turns non-JSON
        # prose into a bogus dict (strictly worse than returning None).
        if outer_in_str:
            if outer_esc:
                outer_esc = False
            elif ch == "\\":
                outer_esc = True
            elif ch == '"':
                outer_in_str = False
            i += 1
            continue
        if ch == '"':
            outer_in_str = True
            i += 1
            continue
        if ch != "{":
            i += 1
            continue
        depth = 0
        in_str = False
        esc = False
        start = i
        while i < n:
            ch = text[i]
            if in_str:
                if esc:
                    esc = False
                elif ch == "\\":
                    esc = True
                elif ch == '"':
                    in_str = False
            elif ch == '"':
                in_str = True
            elif ch == "{":
                depth += 1
            elif ch == "}":
                depth -= 1
                if depth == 0:
                    spans.append(text[start:i + 1])
                    i += 1
                    break
            i += 1
        else:
            break  # unterminated final object
    return spans
 def _looks_json_like(span: str) -> bool:
    """Heuristic: does ``span`` look like an intended JSON object (vs. prose)?
    A genuine JSON object's first non-space character after ``{`` is either ``"``
    (a string key) or ``}`` (an empty object). Prose pseudo-objects that the
    repair pass would otherwise fabricate into bogus dicts — ``{op: delete}``,
    ``{x: 1}`` quoted in single quotes or backticks, etc. — start with a bare
    word and are rejected. This complements the string-aware scan, which only
    skips *double*-quoted prose; single-quoted / backticked / unquoted prose
    braces are caught here instead. Legitimate repair targets (trailing commas,
    unescaped quotes inside string values) all begin with ``"`` and pass.
    """
    inner = span.strip()
    if not (inner.startswith("{") and inner.endswith("}")):
        return False
    after_brace = inner[1:].lstrip()
    return after_brace[:1] in ('"', '}')
 def extract_json(text: str) -> dict | None:
@@ -22,6 +107,51 @@ def extract_json(text: str) -> dict | None:
            return json.loads(m.group(0))
        except json.JSONDecodeError:
            pass
    # Tolerant fallback for non-OpenAI backends (Claude/Qwen, …) whose free-form
    # JSON strict json.loads rejects — unescaped ASCII quotes inside CJK string
    # values, trailing commas, etc. Repair so the analyst's edits aren't silently
    # dropped, but ONLY a single unambiguous object: never feed the greedy `{.*}`
    # span or the raw text, or json_repair would quietly return one of several
    # objects (empirically the wrong/last one) — strictly worse than None, which
    # the caller can detect and retry/skip.
    #
    # Pick the candidate FIRST, before importing json_repair, so the optional
    # dependency only matters (and only warns) when there is genuinely a single
    # malformed object we could have repaired. Ordinary no-JSON / prose replies
    # have no candidate and return None silently.
    candidate = None
    fenced = re.search(r"```json\s*(.*?)```", text, re.DOTALL)
    if fenced and len(_top_level_brace_objects(fenced.group(1))) == 1:
        candidate = fenced.group(1)
    else:
        objs = _top_level_brace_objects(text)
        if len(objs) == 1:
            candidate = objs[0]
        # 0 or >1 top-level objects → too ambiguous to repair safely → None
    if not candidate:
        return None
    # Final guard: only repair spans that actually look like an intended JSON
    # object. Prose pseudo-objects in single quotes / backticks / bare text
    # (e.g. `{op: delete}`) reach here because the scan only skips double-quoted
    # prose; repairing them would fabricate a wrong dict (worse than None).
    if not _looks_json_like(candidate):
        return None
    try:
        from json_repair import repair_json
    except ModuleNotFoundError:
        warnings.warn(
            "json_repair not installed; malformed-JSON recovery disabled — "
            "a non-OpenAI analyst edit may be silently dropped. pip install json_repair",
            RuntimeWarning,
            stacklevel=2,
        )
        return None
    try:
        repaired = repair_json(candidate, return_objects=True)
        if isinstance(repaired, dict) and repaired:
            return repaired
    except Exception:  # noqa: BLE001 — repair is best-effort
        pass
    return None
@@ -0,0 +1,20 @@
 """SkillOpt-Sleep — nightly offline self-evolution for a local Claude agent.
 A Claude Code plugin engine that gives a user's agent a "sleep cycle":
 harvest the day's real session transcripts, mine recurring tasks, replay
 them offline, and consolidate short-term experience into long-term memory
 (CLAUDE.md) and skills (SKILL.md) behind a SkillOpt validation gate.
 Synthesizes three ideas:
  * SkillOpt  — validation-gated bounded text optimization (this repo)
  * Dreams    — offline memory consolidation, input never mutated
  * Sleep     — short-term experience -> long-term competence, offline
 Public entry points:
  * skillopt_sleep.cli      — `python -m skillopt_sleep ...`
  * skillopt_sleep.cycle.run_sleep_cycle(...)
 """
 from __future__ import annotations
 __all__ = ["__version__"]
 __version__ = "0.2.0"
@@ -0,0 +1,532 @@
 """SkillOpt-Sleep — command-line interface.
    python -m skillopt_sleep run        # full cycle: harvest->mine->replay->gate->stage
    python -m skillopt_sleep dry-run    # same but report only, no staging/adopt
    python -m skillopt_sleep status     # show state + latest staged proposal
    python -m skillopt_sleep adopt      # apply the latest staged proposal (with backup)
    python -m skillopt_sleep harvest    # just print what would be mined (debug)
 Common flags:
    --project PATH      project to evolve (default: cwd)
    --scope all|invoked harvest scope (default: invoked)
    --max-sessions N    cap transcript sessions per run
    --max-tasks N       cap mined tasks per run
    --target-skill-path PATH explicit live SKILL.md to stage/adopt
    --tasks-file PATH   reviewed TaskRecord JSON file to replay instead of harvesting
    --backend mock|claude|codex|copilot|handoff
    --source claude|codex|auto
    --model NAME
    --lookback-hours N
    --auto-adopt
    --json              machine-readable output
 """
 from __future__ import annotations
 import argparse
 import json
 import os
 import sys
 from typing import Any, Dict
 from skillopt_sleep.config import load_config
 from skillopt_sleep.cycle import run_sleep_cycle
 from skillopt_sleep.harvest_sources import harvest_for_config
 from skillopt_sleep.mine import mine
 from skillopt_sleep.staging import adopt as adopt_staging
 from skillopt_sleep.staging import latest_staging
 from skillopt_sleep.state import SleepState
 from skillopt_sleep.tasks_file import load_tasks_file, make_tasks_payload, write_tasks_file
 def _read_text(path: str) -> str:
    try:
        with open(path, encoding="utf-8") as f:
            return f.read()
    except Exception:
        return ""
 def _report_payload(rep, outcome) -> Dict[str, Any]:
    return {
        "night": rep.night,
        "accepted": rep.accepted,
        "gate_action": rep.gate_action,
        "no_edits_reason": getattr(rep, "no_edits_reason", ""),
        "baseline": rep.baseline_score,
        "candidate": rep.candidate_score,
        "n_tasks": rep.n_tasks,
        "n_sessions": rep.n_sessions,
        "n_accepted_edits": len(rep.edits),
        "n_rejected_edits": len(rep.rejected_edits),
        "edits": [e.__dict__ for e in rep.edits],
        "rejected_edits": [e.__dict__ for e in rep.rejected_edits],
        "notes": rep.notes,
        "staging_dir": outcome.staging_dir,
        "adopted": outcome.adopted,
    }
 def _add_common(p: argparse.ArgumentParser) -> None:
    p.add_argument("--project", default="")
    p.add_argument("--scope", default="", choices=["", "all", "invoked"])
    p.add_argument("--backend", default="",
                   choices=["", "mock", "claude", "codex", "copilot", "handoff"])
    p.add_argument("--model", default="")
    p.add_argument("--codex-path", default="", help="path to the real @openai/codex binary")
    p.add_argument("--claude-home", default="", help="override ~/.claude (also isolates state)")
    p.add_argument("--codex-home", default="", help="override ~/.codex for archived session harvest")
    p.add_argument("--source", default="", choices=["", "claude", "codex", "auto"],
                   help="session transcript source")
    p.add_argument("--lookback-hours", type=int, default=None,
                   help="harvest window in hours; 0 = scan full history")
    p.add_argument("--edit-budget", type=int, default=0)
    p.add_argument("--max-sessions", type=int, default=0,
                   help="cap harvested sessions before mining; default derives from max tasks")
    p.add_argument("--max-tasks", type=int, default=0,
                   help="cap mined tasks for this run")
    p.add_argument("--target-skill-path", default="",
                   help="explicit live SKILL.md path to evolve/stage/adopt")
    p.add_argument("--tasks-file", default="",
                   help="reviewed TaskRecord JSON file to replay instead of harvesting")
    p.add_argument("--progress", action="store_true",
                   help="print phase progress to stderr")
    p.add_argument("--auto-adopt", action="store_true")
    p.add_argument("--json", action="store_true")
 def _cfg_from_args(args, task_meta: Dict[str, Any] | None = None) -> Any:
    overrides: Dict[str, Any] = {}
    if args.project:
        overrides["invoked_project"] = os.path.abspath(args.project)
        overrides["projects"] = "invoked"
    if args.scope:
        overrides["projects"] = args.scope
    if args.backend:
        overrides["backend"] = args.backend
    if args.model:
        overrides["model"] = args.model
    if getattr(args, "codex_path", ""):
        overrides["codex_path"] = os.path.abspath(args.codex_path)
    if getattr(args, "claude_home", ""):
        overrides["claude_home"] = os.path.abspath(args.claude_home)
    if getattr(args, "codex_home", ""):
        overrides["codex_home"] = os.path.abspath(args.codex_home)
    if getattr(args, "source", ""):
        overrides["transcript_source"] = args.source
    lh = getattr(args, "lookback_hours", None)
    if lh is not None:  # --lookback-hours was explicitly passed (0 = full history)
        overrides["lookback_hours"] = lh
    if getattr(args, "edit_budget", 0):
        overrides["edit_budget"] = args.edit_budget
    if getattr(args, "max_sessions", 0):
        overrides["max_sessions_per_night"] = args.max_sessions
    if getattr(args, "max_tasks", 0):
        overrides["max_tasks_per_night"] = args.max_tasks
    target_skill_path = getattr(args, "target_skill_path", "")
    if not target_skill_path and task_meta:
        target_skill_path = str(task_meta.get("target_skill_path") or "")
    if target_skill_path:
        path = os.path.expanduser(target_skill_path)
        if args.project and not os.path.isabs(path):
            path = os.path.join(os.path.abspath(args.project), path)
        overrides["target_skill_path"] = os.path.abspath(path)
    if getattr(args, "progress", False):
        overrides["progress"] = True
    if getattr(args, "auto_adopt", False):
        overrides["auto_adopt"] = True
    return load_config(**overrides)
 def cmd_run(args, dry: bool = False) -> int:
    task_meta: Dict[str, Any] = {}
    tasks = None
    if getattr(args, "tasks_file", ""):
        # Load once before config so target_skill_path can default from metadata.
        tasks, task_meta = load_tasks_file(args.tasks_file)
    cfg = _cfg_from_args(args, task_meta=task_meta)
    if getattr(args, "tasks_file", ""):
        tasks, task_meta = load_tasks_file(
            args.tasks_file,
            holdout_fraction=cfg.get("holdout_fraction", 0.34),
            seed=cfg.get("seed", 42),
        )
        if cfg.get("backend", "mock") != "mock" and task_meta.get("reviewed") is not True:
            print(
                "[sleep] refusing real-backend replay from an unreviewed tasks file; "
                "inspect/redact it and set \"reviewed\": true first",
                file=sys.stderr,
            )
            return 2
    if cfg.get("backend", "mock") == "handoff":
        return _run_handoff(cfg, args, seed_tasks=tasks, task_meta=task_meta, dry=dry)
    outcome = run_sleep_cycle(cfg, seed_tasks=tasks, dry_run=dry)
    _print_run_report(outcome, args, task_meta)
    return 0
 def _print_run_report(outcome, args, task_meta: Dict[str, Any]) -> None:
    rep = outcome.report
    if args.json:
        payload = _report_payload(rep, outcome)
        if task_meta:
            payload["tasks_file"] = task_meta.get("tasks_file", "")
            payload["tasks_reviewed"] = task_meta.get("reviewed", False)
        print(json.dumps(payload, ensure_ascii=False, indent=2))
    else:
        print(f"[sleep] night {rep.night}: {rep.n_sessions} sessions -> {rep.n_tasks} tasks")
        print(f"[sleep] held-out {rep.baseline_score:.3f} -> {rep.candidate_score:.3f} "
              f"=> {rep.gate_action} (accepted={rep.accepted})")
        for e in rep.edits:
            print(f"   + [{e.target}/{e.op}] {e.content}")
        if rep.rejected_edits:
            print("[sleep] rejected by gate:")
            for e in rep.rejected_edits:
                print(f"   - [{e.target}/{e.op}] {e.content}")
        if outcome.staging_dir:
            print(f"[sleep] staged: {outcome.staging_dir}")
            if not outcome.adopted:
                print("[sleep] review it, then: python -m skillopt_sleep adopt")
        if outcome.adopted:
            print(f"[sleep] auto-adopted: {', '.join(outcome.adopted_paths)}")
 def _handoff_dir_for(cfg) -> str:
    project = cfg.get("invoked_project") or os.getcwd()
    return os.environ.get("SKILLOPT_SLEEP_HANDOFF_DIR", "") or os.path.join(
        project, ".skillopt-sleep-handoff"
    )
 def _redact_deep(obj):
    """Redact secret-looking substrings in every string of a JSON-like tree."""
    from skillopt_sleep.staging import redact_secrets
    if isinstance(obj, str):
        return redact_secrets(obj)
    if isinstance(obj, list):
        return [_redact_deep(x) for x in obj]
    if isinstance(obj, dict):
        return {k: _redact_deep(v) for k, v in obj.items()}
    return obj
 def _flush_handoff(backend, args) -> int:
    prompts_path = backend.flush_pending()
    if args.json:
        print(json.dumps({
            "handoff_pending": len(backend.pending),
            "prompts": prompts_path,
            "answers_dir": backend.answers_dir,
        }, ensure_ascii=False, indent=2))
    else:
        print(f"[sleep] handoff: {len(backend.pending)} model call(s) need answers")
        print(f"[sleep] prompts: {prompts_path}")
        print(f"[sleep] write each raw answer to {backend.answers_dir}/<id>.md, "
              "then re-run this exact command to resume")
    return 3
 def _handoff_mine_and_pin(cfg, args, backend, snapshot: str, dry: bool):
    """Harvest + mine with the same knobs as run_sleep_cycle (harvest window,
    target-skill filter, candidate-limit bump, LLM mining — routed through the
    handoff files like every other model call), then pin the result to
    ``tasks.json``. Session digests are pinned too, so the sessions created
    while answering prompts cannot change what gets mined between rounds.
    Returns ``(exit_code, tasks)``; ``tasks is None`` means exit now.
    """
    import time
    from skillopt_sleep.handoff_backend import PendingCalls
    from skillopt_sleep.state import SleepState, _now_iso
    from skillopt_sleep.types import SessionDigest
    project = cfg.get("invoked_project") or os.getcwd()
    state = SleepState.load(cfg.state_path)
    started = _now_iso()
    digests_path = os.path.join(backend.handoff_dir, "digests.json")
    digests = None
    if os.path.exists(digests_path):
        try:
            with open(digests_path, encoding="utf-8") as f:
                raw = json.load(f)
            known = set(SessionDigest.__dataclass_fields__)
            digests = [SessionDigest(**{k: v for k, v in d.items() if k in known})
                       for d in raw]
        except Exception:
            # Corrupted/truncated pin (e.g. an interrupted earlier round):
            # fall back to a fresh harvest instead of crashing the run.
            print("[sleep] handoff: digests.json unreadable — re-harvesting",
                  file=sys.stderr)
            digests = None
    if digests is None:
        since = state.last_harvest_for(project)
        lookback_hours = cfg.get("lookback_hours", 72)
        if since is None and lookback_hours and lookback_hours > 0:
            since = _now_iso(time.time() - lookback_hours * 3600)
        max_tasks = cfg.get("max_tasks_per_night", 40)
        session_limit = cfg.get("max_sessions_per_night", 0) or max_tasks * 3
        digests = harvest_for_config(cfg, since_iso=since, limit=session_limit)
        os.makedirs(backend.handoff_dir, exist_ok=True)
        with open(digests_path, "w", encoding="utf-8") as f:
            json.dump(_redact_deep([d.to_dict() for d in digests]), f,
                      ensure_ascii=False, indent=2)
    max_tasks = cfg.get("max_tasks_per_night", 40)
    session_limit = cfg.get("max_sessions_per_night", 0) or max_tasks * 3
    target_skill_path = cfg.managed_skill_path() if cfg.get("target_skill_path", "") else ""
    target_skill_text = _read_text(target_skill_path) if target_skill_path else ""
    candidate_limit = max_tasks
    if cfg.get("target_task_filter", True) and target_skill_text:
        candidate_limit = max(max_tasks, max_tasks * 3)
    llm_miner = None
    if cfg.get("llm_mine", True):
        try:
            from skillopt_sleep.llm_miner import make_llm_miner
            llm_miner = make_llm_miner(
                backend, max_sessions=session_limit, max_tasks=candidate_limit,
            )
        except Exception:
            llm_miner = None
    try:
        tasks = mine(
            digests,
            max_tasks=max_tasks,
            candidate_limit=candidate_limit,
            holdout_fraction=cfg.get("holdout_fraction", 0.34),
            seed=cfg.get("seed", 42),
            llm_miner=llm_miner,
            target_skill_text=target_skill_text,
            target_skill_path=target_skill_path,
        )
    except PendingCalls:
        tasks = []
    if backend.pending:
        # LLM mining needs answers before the task set can be pinned.
        return _flush_handoff(backend, args), None
    if not tasks:
        print("[sleep] handoff: no tasks mined — nothing to consolidate")
        if not dry:
            # Advance the harvest window like run_sleep_cycle's no-tasks
            # branch, or every later run re-scans the same stale window.
            state.set_last_harvest(project, started)
            state.save()
        return 0, None
    payload = make_tasks_payload(
        tasks,
        project=project,
        transcript_source=cfg.get("transcript_source", ""),
        n_sessions=len(digests),
        target_skill_path=target_skill_path,
    )
    # NOT marked reviewed: feeding this snapshot back through --tasks-file
    # with a real backend must still hit the human-review gate above. The
    # driver itself loads it directly, with the same trust as in-cycle mining.
    write_tasks_file(snapshot, _redact_deep(payload))
    print(f"[sleep] handoff: pinned {len(tasks)} tasks -> {snapshot}")
    return 0, tasks
 def _run_handoff(cfg, args, *, seed_tasks, task_meta: Dict[str, Any], dry: bool) -> int:
    """Drive the handoff backend: run until model calls are needed, then
    write the prompt batch and exit 3; on a fully-answered run, finish
    normally. Session digests and mined tasks are pinned under the handoff
    dir on the first rounds so wall-clock time between rounds (including
    the very sessions that answer the prompts) cannot change the task set
    and invalidate earlier answers.
    """
    from skillopt_sleep.handoff_backend import HandoffBackend, PendingCalls
    hdir = _handoff_dir_for(cfg)
    backend = HandoffBackend(model=cfg.get("model", ""), handoff_dir=hdir)
    tasks = seed_tasks
    if tasks is None:
        snapshot = os.path.join(hdir, "tasks.json")
        if os.path.exists(snapshot):
            tasks, _meta = load_tasks_file(
                snapshot,
                holdout_fraction=cfg.get("holdout_fraction", 0.34),
                seed=cfg.get("seed", 42),
            )
        else:
            rc, tasks = _handoff_mine_and_pin(cfg, args, backend, snapshot, dry)
            if tasks is None:
                return rc
    outcome = None
    try:
        outcome = run_sleep_cycle(cfg, seed_tasks=tasks, dry_run=dry, backend=backend)
    except PendingCalls:
        pass
    if backend.pending:
        return _flush_handoff(backend, args)
    _print_run_report(outcome, args, task_meta)
    # A completed real run ends the night: archive the handoff dir so the
    # next night re-harvests instead of replaying the pinned snapshot.
    if not dry and outcome.staging_dir and os.path.isdir(hdir):
        import time
        done = f"{hdir}.night{outcome.report.night}.done"
        if os.path.exists(done):
            done = f"{done}.{int(time.time())}"
        os.rename(hdir, done)
        print(f"[sleep] handoff: archived round data -> {done}")
    return 0
 def cmd_status(args) -> int:
    cfg = _cfg_from_args(args)
    state = SleepState.load(cfg.state_path)
    project = cfg.get("invoked_project") or os.getcwd()
    latest = latest_staging(project)
    info = {
        "night": state.night,
        "state_path": cfg.state_path,
        "project": project,
        "history_tail": state.data.get("history", [])[-5:],
        "latest_staging": latest,
        "slow_memory_chars": len(state.slow_memory),
    }
    if args.json:
        print(json.dumps(info, ensure_ascii=False, indent=2))
    else:
        print(f"[sleep] nights so far: {state.night}")
        print(f"[sleep] project: {project}")
        if latest:
            print(f"[sleep] latest staged proposal: {latest}")
            rp = os.path.join(latest, "report.md")
            if os.path.exists(rp):
                with open(rp) as f:
                    print("\n" + f.read())
        else:
            print("[sleep] no staged proposals yet.")
    return 0
 def cmd_adopt(args) -> int:
    cfg = _cfg_from_args(args)
    project = cfg.get("invoked_project") or os.getcwd()
    target = args.staging or latest_staging(project)
    if not target or not os.path.isdir(target):
        print("[sleep] nothing to adopt (no staging dir).")
        return 1
    updated = adopt_staging(target)
    print(f"[sleep] adopted from {target}")
    for p in updated:
        print(f"   -> {p}")
    if not updated:
        print("[sleep] (proposal contained no accepted changes)")
    return 0
 def cmd_harvest(args) -> int:
    cfg = _cfg_from_args(args)
    session_limit = cfg.get("max_sessions_per_night", 0) or cfg.get("max_tasks_per_night", 40) * 3
    target_skill_path = cfg.managed_skill_path() if cfg.get("target_skill_path", "") else ""
    target_skill_text = _read_text(target_skill_path) if target_skill_path else ""
    max_tasks = cfg.get("max_tasks_per_night", 40)
    candidate_limit = max_tasks
    if cfg.get("target_task_filter", True) and target_skill_text:
        candidate_limit = max(max_tasks, max_tasks * 3)
    digests = harvest_for_config(cfg, limit=session_limit)
    tasks = mine(
        digests,
        max_tasks=max_tasks,
        candidate_limit=candidate_limit,
        holdout_fraction=cfg.get("holdout_fraction", 0.34),
        seed=cfg.get("seed", 42),
        target_skill_text=target_skill_text,
        target_skill_path=target_skill_path,
    )
    payload = make_tasks_payload(
        tasks,
        project=cfg.get("invoked_project") or os.getcwd(),
        transcript_source=cfg.get("transcript_source", ""),
        n_sessions=len(digests),
        target_skill_path=target_skill_path,
    )
    output_path = ""
    if getattr(args, "output", ""):
        output_path = write_tasks_file(args.output, payload)
    if args.json:
        json_payload = dict(payload)
        if output_path:
            json_payload["output"] = output_path
        print(json.dumps(json_payload, ensure_ascii=False, indent=2))
    else:
        print(f"[sleep] {len(digests)} sessions -> {len(tasks)} tasks")
        if output_path:
            print(f"[sleep] wrote reviewed-task draft: {output_path}")
        for t in tasks:
            print(f"  [{t.split}/{t.outcome}] {t.intent[:90]}")
    return 0
 def cmd_schedule(args) -> int:
    from skillopt_sleep.scheduler import schedule, list_scheduled
    cfg = _cfg_from_args(args)
    project = cfg.get("invoked_project") or os.getcwd()
    ok, msg = schedule(project, backend=cfg.get("backend", "mock"),
                       hour=args.hour, minute=args.minute,
                       extra=("--auto-adopt" if getattr(args, "auto_adopt", False) else ""))
    print("[sleep] " + msg)
    cur = list_scheduled()
    if cur:
        print("[sleep] currently scheduled:")
        for ln in cur:
            print("   " + ln[:140])
    return 0 if ok else 1
 def cmd_unschedule(args) -> int:
    from skillopt_sleep.scheduler import unschedule
    cfg = _cfg_from_args(args)
    project = cfg.get("invoked_project") or os.getcwd()
    ok, msg = unschedule(project, all_projects=getattr(args, "all", False))
    print("[sleep] " + msg)
    return 0 if ok else 1
 def main(argv=None) -> int:
    parser = argparse.ArgumentParser(prog="skillopt_sleep", description="SkillOpt-Sleep nightly self-evolution")
    sub = parser.add_subparsers(dest="cmd", required=True)
    p_run = sub.add_parser("run", help="run a full sleep cycle")
    _add_common(p_run)
    p_dry = sub.add_parser("dry-run", help="harvest+mine+replay, report only")
    _add_common(p_dry)
    p_status = sub.add_parser("status", help="show state + latest proposal")
    _add_common(p_status)
    p_adopt = sub.add_parser("adopt", help="apply latest staged proposal")
    _add_common(p_adopt)
    p_adopt.add_argument("--staging", default="", help="specific staging dir")
    p_harvest = sub.add_parser("harvest", help="debug: show mined tasks")
    _add_common(p_harvest)
    p_harvest.add_argument("--output", default="", help="write mined tasks JSON for review")
    p_sched = sub.add_parser("schedule", help="install a nightly cron entry for this project")
    _add_common(p_sched)
    p_sched.add_argument("--hour", type=int, default=3)
    p_sched.add_argument("--minute", type=int, default=17)
    p_unsched = sub.add_parser("unschedule", help="remove the nightly cron entry")
    _add_common(p_unsched)
    p_unsched.add_argument("--all", action="store_true", help="remove all managed entries")
    args = parser.parse_args(argv)
    if args.cmd == "run":
        return cmd_run(args, dry=False)
    if args.cmd == "dry-run":
        return cmd_run(args, dry=True)
    if args.cmd == "status":
        return cmd_status(args)
    if args.cmd == "adopt":
        return cmd_adopt(args)
    if args.cmd == "harvest":
        return cmd_harvest(args)
    if args.cmd == "schedule":
        return cmd_schedule(args)
    if args.cmd == "unschedule":
        return cmd_unschedule(args)
    parser.print_help()
    return 2
 if __name__ == "__main__":
    sys.exit(main())
@@ -0,0 +1,75 @@
 """SkillOpt-Sleep — budget controller.
 Lets the user say how much they're willing to spend on a night's "dreaming",
 in tokens or wall-clock minutes, and the engine schedules depth (how many
 rollouts × how many nights) within that budget. Stops cleanly when exhausted
 and reports what it skipped (no silent truncation).
 """
 from __future__ import annotations
 from dataclasses import dataclass
 from typing import Optional
@dataclass
 class Budget:
    max_tokens: Optional[int] = None      # None = unlimited
    max_minutes: Optional[float] = None   # None = unlimited
    _start_time: Optional[float] = None
    _tokens_at_start: int = 0
    def start(self, clock_fn, tokens_now: int) -> None:
        self._start_time = clock_fn()
        self._tokens_at_start = tokens_now
    def tokens_spent(self, tokens_now: int) -> int:
        return max(0, tokens_now - self._tokens_at_start)
    def minutes_elapsed(self, clock_fn) -> float:
        if self._start_time is None:
            return 0.0
        return (clock_fn() - self._start_time) / 60.0
    def remaining_fraction(self, *, tokens_now: int, clock_fn) -> float:
        """Smallest remaining fraction across all active limits (1.0 = fresh)."""
        fracs = [1.0]
        if self.max_tokens:
            fracs.append(max(0.0, 1.0 - self.tokens_spent(tokens_now) / self.max_tokens))
        if self.max_minutes:
            fracs.append(max(0.0, 1.0 - self.minutes_elapsed(clock_fn) / self.max_minutes))
        return min(fracs)
    def exhausted(self, *, tokens_now: int, clock_fn) -> bool:
        if self.max_tokens and self.tokens_spent(tokens_now) >= self.max_tokens:
            return True
        if self.max_minutes and self.minutes_elapsed(clock_fn) >= self.max_minutes:
            return True
        return False
    def status(self, *, tokens_now: int, clock_fn) -> str:
        parts = []
        if self.max_tokens:
            parts.append(f"tokens {self.tokens_spent(tokens_now)}/{self.max_tokens}")
        if self.max_minutes:
            parts.append(f"minutes {self.minutes_elapsed(clock_fn):.1f}/{self.max_minutes}")
        return ", ".join(parts) or "unbounded"
 def plan_depth(budget: Budget, *, n_tasks: int,
               default_nights: int = 2, default_k: int = 1) -> tuple:
    """Heuristically choose (nights, rollouts_per_task) from a token budget.
    Rough cost model: one rollout ≈ 1 unit; a night does ~n_tasks*k rollouts
    plus reflect/gate (~2*n_tasks). We scale k and nights up with more budget.
    Returns (nights, k). With no budget set, returns the defaults.
    """
    if not budget.max_tokens:
        return default_nights, default_k
    # assume ~1.5k tokens per rollout as a planning constant
    rollouts_affordable = budget.max_tokens / 1500.0
    per_night = max(1, n_tasks) * 3  # rollouts + reflect + gate, k=1
    nights = max(1, min(4, int(rollouts_affordable // per_night)))
    # spend surplus on more rollouts-per-task (contrastive signal)
    surplus = rollouts_affordable - nights * per_night
    k = max(1, min(5, 1 + int(surplus // max(1, n_tasks))))
    return nights, k
@@ -0,0 +1,162 @@
 """SkillOpt-Sleep — configuration.
 Config is JSON-first (yaml optional) so the engine and the deterministic
 experiment run with zero external dependencies. Defaults are safe:
 review-gated adoption, single-project scope, bounded token/task budgets.
 Resolution order (later wins):
  1. built-in DEFAULTS
  2. ~/.skillopt-sleep/config.json  (or .yaml if PyYAML available)
  3. explicit overrides passed to load_config(**overrides)
 """
 from __future__ import annotations
 import json
 import os
 from dataclasses import dataclass, field
 from typing import Any, Dict, Optional
 HOME_STATE_DIR = os.path.expanduser("~/.skillopt-sleep")
 CLAUDE_HOME = os.path.expanduser("~/.claude")
 CODEX_HOME = os.path.expanduser("~/.codex")
 DEFAULTS: Dict[str, Any] = {
    # ── scope ──────────────────────────────────────────────────────────────
    "claude_home": CLAUDE_HOME,
    "codex_home": CODEX_HOME,
    "transcript_source": "claude",  # "claude" | "codex" | "auto"
    "projects": "invoked",        # "invoked" | "all" | [list of abs paths]
    "invoked_project": "",        # filled at runtime (cwd) when projects == "invoked"
    "lookback_hours": 72,         # harvest window when no prior sleep recorded
    # ── budgets ────────────────────────────────────────────────────────────
    "max_tasks_per_night": 40,
    "max_tokens_per_night": 400_000,
    "holdout_fraction": 0.34,     # legacy alias for val_fraction
    "val_fraction": 0.34,         # real tasks reserved to gate updates
    "test_fraction": 0.0,         # real tasks reserved as the final held-out measure
    # ── optimizer ──────────────────────────────────────────────────────────
    "backend": "mock",            # "mock" | "claude" | "codex" | "copilot"
    "model": "",                  # backend-specific; "" => backend default
    "gate_mode": "on",            # "on" (validation-gated) | "off" (greedy, no hard filter)
    "codex_path": "",             # "" => auto-detect the real @openai/codex binary
    "edit_budget": 4,             # textual learning rate (max edits/night)
    "gate_metric": "mixed",       # hard | soft | mixed (mixed best for tiny holdouts)
    "gate_mixed_weight": 0.5,
    "replay_mode": "mock",        # "mock" (sandboxed prompt) | "fresh" (worktree)
    # ── dream + recall (opt-in; defaults reproduce the prior single-shot loop) ─
    "dream_rollouts": 1,          # >1 => multi-rollout contrastive reflection per task
    "dream_factor": 0,            # >0 => add N synthetic variants of each task to the dream
    "recall_k": 0,                # >0 => recall the K most-similar past tasks into the dream
    "evolve_memory": True,        # consolidate CLAUDE.md
    "evolve_skill": True,         # consolidate the managed SKILL.md
    "llm_mine": True,             # use the backend to mine checkable tasks (real backends)
    "target_skill_path": "",      # explicit SKILL.md target for repo-scoped agents
    "target_task_filter": True,   # prefer mined tasks matching target_skill_path/text
    "progress": False,            # print phase progress to stderr
    # ── adoption / safety ──────────────────────────────────────────────────
    "auto_adopt": False,          # default: stage + require explicit `adopt`
    "managed_skill_name": "skillopt-sleep-learned",
    "redact_secrets": True,
    "seed": 42,
 }
@dataclass
 class SleepConfig:
    data: Dict[str, Any] = field(default_factory=lambda: dict(DEFAULTS))
    # convenient attribute access -------------------------------------------
    def __getattr__(self, name: str) -> Any:
        # only called when normal attribute lookup fails
        data = object.__getattribute__(self, "data")
        if name in data:
            return data[name]
        raise AttributeError(name)
    def get(self, key: str, default: Any = None) -> Any:
        return self.data.get(key, default)
    def to_dict(self) -> Dict[str, Any]:
        return dict(self.data)
    # paths ------------------------------------------------------------------
    @property
    def state_dir(self) -> str:
        # Allow full isolation: if the caller overrides state_dir explicitly,
        # honor it; else derive from claude_home's parent so a single
        # --claude-home flag isolates transcripts AND state together; else the
        # default ~/.skillopt-sleep.
        explicit = self.data.get("state_dir")
        if explicit:
            return explicit
        ch = self.data.get("claude_home", CLAUDE_HOME)
        if os.path.abspath(ch) != os.path.abspath(CLAUDE_HOME):
            return os.path.join(os.path.dirname(os.path.abspath(ch)), ".skillopt-sleep")
        return HOME_STATE_DIR
    @property
    def state_path(self) -> str:
        return os.path.join(self.state_dir, "state.json")
    @property
    def transcripts_dir(self) -> str:
        return os.path.join(self.data["claude_home"], "projects")
    @property
    def codex_archived_sessions_dir(self) -> str:
        return os.path.join(self.data["codex_home"], "archived_sessions")
    @property
    def history_path(self) -> str:
        return os.path.join(self.data["claude_home"], "history.jsonl")
    @property
    def skills_dir(self) -> str:
        return os.path.join(self.data["claude_home"], "skills")
    def managed_skill_path(self) -> str:
        target = self.data.get("target_skill_path") or ""
        if target:
            target = os.path.expanduser(str(target))
            if not os.path.isabs(target):
                base = self.data.get("invoked_project") or os.getcwd()
                target = os.path.join(base, target)
            return os.path.abspath(target)
        return os.path.join(
            self.skills_dir, self.data["managed_skill_name"], "SKILL.md"
        )
 def _user_config_path() -> Optional[str]:
    for name in ("config.json", "config.yaml", "config.yml"):
        p = os.path.join(HOME_STATE_DIR, name)
        if os.path.exists(p):
            return p
    return None
 def _load_file(path: str) -> Dict[str, Any]:
    if path.endswith((".yaml", ".yml")):
        try:
            import yaml  # optional
            with open(path) as f:
                return yaml.safe_load(f) or {}
        except Exception:
            return {}
    with open(path) as f:
        return json.load(f)
 def load_config(**overrides: Any) -> SleepConfig:
    data = dict(DEFAULTS)
    path = _user_config_path()
    if path:
        try:
            data.update(_load_file(path) or {})
        except Exception:
            pass
    data.update({k: v for k, v in overrides.items() if v is not None})
    if data.get("projects") == "invoked" and not data.get("invoked_project"):
        data["invoked_project"] = os.getcwd()
    return SleepConfig(data=data)
@@ -0,0 +1,266 @@
 """SkillOpt-Sleep — Stage 4: consolidate (one SkillOpt epoch).
 This is the core that makes nightly evolution *safe*: it proposes bounded
 edits from replayed failures, applies them to a candidate skill/memory, then
 **gates** the candidate on a held-out slice of the user's own tasks. Only a
 candidate that strictly improves the held-out score is accepted — the SkillOpt
 validation gate, vendored self-contained in ``skillopt_sleep.gate``.
 """
 from __future__ import annotations
 import os
 from dataclasses import dataclass, field
 from typing import List, Optional, Tuple
 from skillopt_sleep.backend import Backend
 from skillopt_sleep.memory import apply_edits
 from skillopt_sleep.replay import aggregate_scores, replay_batch
 from skillopt_sleep.types import EditRecord, ReplayResult, TaskRecord
 # Self-contained validation gate (vendored from SkillOpt; zero dependency on the
 # research package, so this open-source tool stays decoupled from the paper code).
 from skillopt_sleep.gate import evaluate_gate, select_gate_score
 _HAVE_REPO_GATE = True
@dataclass
 class ConsolidationResult:
    accepted: bool
    gate_action: str
    baseline_score: float
    candidate_score: float
    new_skill: str
    new_memory: str
    applied_edits: List[EditRecord]
    rejected_edits: List[EditRecord]
    holdout_baseline: float
    holdout_candidate: float
    # ── observability (so a 0.0->0.0 night is self-diagnosing, not a black box) ──
    holdout_detail: List[dict] = field(default_factory=list)  # per val task: hard/soft/resp/why
    reflect_raw: str = ""        # the optimizer's last raw reply (empty => reflect produced nothing)
    call_error: str = ""         # backend's last call error (timeout/auth/empty)
 def _split(tasks: List[TaskRecord]) -> Tuple[List[TaskRecord], List[TaskRecord]]:
    """Return (train_tasks, val_tasks).
    train drives reflect; val gates updates. test is held out entirely from
    consolidation and is scored by the caller. Accepts legacy split names
    (replay->train, holdout->val) for robustness.
    """
    def _norm(s: str) -> str:
        return {"replay": "train", "holdout": "val"}.get(s, s)
    train = [t for t in tasks if _norm(t.split) == "train"]
    val = [t for t in tasks if _norm(t.split) == "val"]
    # be robust if a split is empty: fall back so a night still does something,
    # but never silently use test as val.
    test = [t for t in tasks if _norm(t.split) == "test"]
    if not val:
        # prefer train as the gate reference over nothing; last resort all-but-test
        val = train or [t for t in tasks if _norm(t.split) != "test"] or tasks
    if not train:
        train = val
    return train, val
 def _holdout_detail(pairs: List[Tuple[TaskRecord, ReplayResult]]) -> List[dict]:
    """Per-task held-out evidence so a 0.0 night explains itself: was the
    response empty (backend call failed) or non-empty-but-failing-checks
    (judge too strict / edit didn't help)? The two need opposite fixes."""
    out: List[dict] = []
    for t, r in pairs:
        resp = r.response or ""
        out.append({
            "id": t.id,
            "reference_kind": t.reference_kind,
            "hard": r.hard,
            "soft": r.soft,
            "response_len": len(resp),
            "response_head": resp[:200],
            "why": (r.fail_reason or r.judge_rationale or "")[:200],
        })
    return out
 def consolidate(
    backend: Backend,
    tasks: List[TaskRecord],
    skill: str,
    memory: str,
    *,
    edit_budget: int = 4,
    gate_metric: str = "mixed",
    gate_mixed_weight: float = 0.5,
    gate_mode: str = "on",       # "on" (hard/soft per gate_metric) | "off" (greedy)
    rollouts_k: int = 1,         # >1 => multi-rollout contrastive reflection
    evolve_skill: bool = True,
    evolve_memory: bool = True,
    night: int = 1,
 ) -> ConsolidationResult:
    """Run one consolidation epoch: reflect -> bounded edit -> gate.
    train tasks drive reflect; val tasks gate the update (test is held out by the
    caller). With ``gate_mode='off'`` edits are accepted greedily (no val-improve
    requirement) — the user opts out of hard filtering — but val scores are still
    recorded so the report shows whether quality moved.
    Skill and memory are evolved in sequence (skill first if both enabled).
    """
    train_tasks, val_tasks = _split(tasks)
    gate_off = str(gate_mode).strip().lower() in {"off", "none", "false", "greedy"}
    holdout_detail: List[dict] = []
    # ── baseline on the VAL slice (the gate reference) ────────────────────
    # When the gate is OFF the user has opted out of holding out a validation set
    # (the daily-use design): we accept edits greedily and judge quality only on
    # the real test set, scored by the caller. So we SKIP all val scoring — it is
    # both wasted cost and contrary to the "no val set required" design.
    if gate_off:
        base_hard, base_soft = 0.0, 0.0
    else:
        base_pairs = replay_batch(backend, val_tasks, skill, memory)
        base_hard, base_soft = aggregate_scores(base_pairs)
        holdout_detail = _holdout_detail(base_pairs)
    base_score = select_gate_score(base_hard, base_soft, gate_metric, gate_mixed_weight)
    # ── reflect over TRAIN-split failures/successes ───────────────────────
    train_pairs = replay_batch(backend, train_tasks, skill, memory)
    failures = [(t, r) for (t, r) in train_pairs if r.hard < 1.0]
    successes = [(t, r) for (t, r) in train_pairs if r.hard >= 1.0]
    cand_skill, cand_memory = skill, memory
    all_applied: List[EditRecord] = []
    all_rejected: List[EditRecord] = []
    def _gate_apply(doc: str, edits: List[EditRecord], which: str) -> str:
        nonlocal cand_skill, cand_memory, base_score, all_applied, all_rejected
        if not edits:
            return doc
        new_doc, applied = apply_edits(doc, edits)
        if not applied:
            return doc
        # gate OFF: accept greedily with NO val scoring (the daily-use path)
        if gate_off:
            all_applied.extend(applied)
            return new_doc
        # gate ON: score the candidate on the VAL slice, keep only if it improves
        trial_skill = new_doc if which == "skill" else cand_skill
        trial_memory = new_doc if which == "memory" else cand_memory
        pairs = replay_batch(backend, val_tasks, trial_skill, trial_memory)
        h, s = aggregate_scores(pairs)
        cand_score = select_gate_score(h, s, gate_metric, gate_mixed_weight)
        if cand_score > base_score:
            base_score = max(base_score, cand_score)
            all_applied.extend(applied)
            return new_doc
        all_rejected.extend(applied)
        return doc
    if evolve_skill:
        if rollouts_k > 1:
            # multi-rollout contrastive reflection: run each train task K times
            # and distill a rule from the good-vs-bad contrast (the imagination signal).
            from skillopt_sleep.rollout import multi_rollout, contrastive_reflect
            # Parallelize across tasks (each multi_rollout also parallelizes its K
            # attempts). This dream phase is the dominant cost; serial execution
            # times out on real backends. Cap total in-flight at the worker env.
            import os
            from concurrent.futures import ThreadPoolExecutor
            try:
                _w = int(os.environ.get("SKILLOPT_SLEEP_WORKERS", "1"))
            except ValueError:
                _w = 1
            if _w > 1 and len(train_tasks) > 1:
                # split the worker budget between task-parallelism and per-task K
                task_workers = max(1, min(len(train_tasks), _w))
                per_task = max(1, _w // task_workers)
                with ThreadPoolExecutor(max_workers=task_workers) as ex:
                    sets = list(ex.map(
                        lambda t: multi_rollout(backend, t, cand_skill, cand_memory,
                                                k=rollouts_k, workers=per_task),
                        train_tasks))
            else:
                sets = [multi_rollout(backend, t, cand_skill, cand_memory,
                                      k=rollouts_k, workers=1)
                        for t in train_tasks]
            edits = contrastive_reflect(
                backend, sets, cand_skill, cand_memory,
                edit_budget=edit_budget, target="skill",
            )
            # fall back to single-shot reflect if contrast yielded nothing
            if not edits:
                edits = backend.reflect(
                    failures, successes, cand_skill, cand_memory,
                    edit_budget=edit_budget, evolve_skill=True, evolve_memory=False,
                )
        else:
            edits = backend.reflect(
                failures, successes, cand_skill, cand_memory,
                edit_budget=edit_budget, evolve_skill=True, evolve_memory=False,
            )
        cand_skill = _gate_apply(cand_skill, edits, "skill")
    if evolve_memory:
        # re-evaluate failures under the (possibly improved) skill
        train_pairs2 = replay_batch(backend, train_tasks, cand_skill, cand_memory)
        failures2 = [(t, r) for (t, r) in train_pairs2 if r.hard < 1.0]
        successes2 = [(t, r) for (t, r) in train_pairs2 if r.hard >= 1.0]
        edits_m = backend.reflect(
            failures2, successes2, cand_skill, cand_memory,
            edit_budget=edit_budget, evolve_skill=False, evolve_memory=True,
        )
        cand_memory = _gate_apply(cand_memory, edits_m, "memory")
    # ── final decision ────────────────────────────────────────────────────
    if gate_off:
        # greedy mode: no val scoring at all. Keep whatever edits we applied; the
        # caller measures real quality on the test set. We report holdout_candidate
        # as 0.0 (val intentionally not computed in this variant).
        final_hard, final_soft = 0.0, 0.0
        final_score = 0.0
        accepted = bool(all_applied)
        action = "greedy_applied" if all_applied else "greedy_noop"
        base_gate_score = 0.0
    else:
        # scored on the VAL slice (the gate reference)
        final_pairs = replay_batch(backend, val_tasks, cand_skill, cand_memory)
        final_hard, final_soft = aggregate_scores(final_pairs)
        final_score = select_gate_score(final_hard, final_soft, gate_metric, gate_mixed_weight)
        base_gate_score = select_gate_score(base_hard, base_soft, gate_metric, gate_mixed_weight)
        if _HAVE_REPO_GATE:
            gate = evaluate_gate(
                candidate_skill=cand_skill,
                cand_hard=final_hard,
                current_skill=skill,
                current_score=base_gate_score,
                best_skill=skill,
                best_score=base_gate_score,
                best_step=night - 1,
                global_step=night,
                cand_soft=final_soft,
                metric=gate_metric,
                mixed_weight=gate_mixed_weight,
            )
            action = gate.action
            accepted = bool(all_applied) and final_score > base_gate_score
        else:
            action = "accept" if final_score > base_gate_score else "reject"
            accepted = bool(all_applied) and final_score > base_gate_score
    return ConsolidationResult(
        accepted=accepted,
        gate_action=action,
        baseline_score=base_gate_score,
        candidate_score=final_score,
        new_skill=cand_skill if accepted else skill,
        new_memory=cand_memory if accepted else memory,
        applied_edits=all_applied,
        rejected_edits=all_rejected,
        holdout_baseline=base_hard,
        holdout_candidate=final_hard,
        holdout_detail=holdout_detail,
        reflect_raw=getattr(backend, "last_reflect_raw", "") or "",
        call_error=getattr(backend, "last_call_error", "") or "",
    )
--- a/Show More
+++ b/Show More