docs: sync documentation with post-v0.2 changes

2026-07-14 17:11:40 +00:00
parent efb30b4bcc
commit f31bf8c06b
44 changed files with 2285 additions and 2025 deletions
@@ -8,6 +8,7 @@ export AZURE_OPENAI_API_VERSION=2024-12-01-preview
 # Authentication: choose one method
 #   Option 1: API Key
 export AZURE_OPENAI_API_KEY=
 export AZURE_OPENAI_AUTH_MODE=api_key
 #   Option 2: Azure CLI (no API key needed, recommended on Azure VMs)
 # export AZURE_OPENAI_AUTH_MODE=azure_cli
 #   Option 3: Managed Identity
@@ -15,20 +16,45 @@ export AZURE_OPENAI_API_KEY=
 # export AZURE_OPENAI_MANAGED_IDENTITY_CLIENT_ID=your-client-id
 # ── OpenAI-compatible endpoints ──────────────────────────────────────
-# Set AUTH_MODE to openai_compatible and reuse AZURE_OPENAI_ENDPOINT / _API_KEY.
+# Path 1: generic research backend. Select openai_compatible explicitly as
-# The plain OpenAI client is used; no Azure auth, no api-version header.
+# model.optimizer_backend and/or model.target_backend.
 # export OPENAI_COMPATIBLE_BASE_URL=https://api.deepseek.com/v1
 # export OPENAI_COMPATIBLE_API_KEY=sk-...
 # export OPENAI_COMPATIBLE_MODEL=deepseek-chat
 # Per-role overrides use OPTIMIZER_OPENAI_COMPATIBLE_* and
 # TARGET_OPENAI_COMPATIBLE_* (BASE_URL, API_KEY, MODEL, TEMPERATURE,
 # MAX_TOKENS, TIMEOUT_SECONDS).
 # For scripts/train.py and scripts/eval_only.py, also set model.optimizer and
 # model.target in YAML (or via --cfg-options). Those role model values are
 # applied after backend initialization and override *_MODEL environment values.
 # Path 2: research openai_chat compatibility mode. This reuses the Azure-family
 # variables but creates a plain OpenAI client (no Azure auth or api-version).
 # export AZURE_OPENAI_ENDPOINT=https://api.openai.com/v1
 # export AZURE_OPENAI_API_KEY=sk-...
 # export AZURE_OPENAI_AUTH_MODE=openai_compatible
-# ── Anthropic / Claude (for claude_chat backend) ─────────────────────
+# Path 3: SkillOpt-Sleep. `skillopt-sleep run --backend azure_openai` uses the
 # same three AZURE_* variables from path 2. Optional Sleep-only controls:
 # export SKILLOPT_SLEEP_COMPAT_MAX_TOKENS=8192
 # export SKILLOPT_SLEEP_CHAT_EXTRA_BODY='{"provider_option": true}'
 # ── Claude Code CLI (for claude_chat backend) ─────────────────────────
 # Install and authenticate the `claude` CLI before use. For a non-default path:
 # export CLAUDE_CLI_BIN=/path/to/claude
 # ANTHROPIC_API_KEY is one authentication option understood by the CLI; SkillOpt
 # does not create a direct Anthropic API client for this backend.
 # export ANTHROPIC_API_KEY=sk-ant-...
 # ── Qwen Local Model (for qwen_chat backend) ────────────────────────
 # export QWEN_CHAT_BASE_URL=http://localhost:8000/v1
 # export QWEN_CHAT_MODEL=Qwen/Qwen3.5-4B
 # The train/eval entry points likewise override this model with
 # model.optimizer/model.target for the selected Qwen roles.
 # ── MiniMax (for minimax_chat backend) ──────────────────────────────
 # export MINIMAX_BASE_URL=https://api.minimax.io/v1
 # export MINIMAX_API_KEY=...
-# export MINIMAX_MODEL=MiniMax-M2.7
+# When MiniMax is the target, set model.minimax_model in YAML. The current
 # adapter shares one deployment across MiniMax roles; mixed-backend runs cannot
 # independently select a MiniMax optimizer model and a different target model.
@@ -15,7 +15,65 @@ All notable changes to SkillOpt are documented here. This project adheres to
  per night). Mined tasks are pinned per night so answering sessions cannot
  shift the task set. Ships a `/skillopt-sleep-handoff` Claude Code command
  that automates the loop with fresh-context subagents to protect the
-  held-out gate.
+  held-out gate (thanks @dimitarvdenev, #125).
 - **Generic OpenAI-compatible research backend** for optimizer and target
  calls, with configurable base URL, API key, model, and timeout (thanks
  @nankingjing, #115).
 - **OpenAI-compatible SkillOpt-Sleep endpoint support** for providers such as
  DeepSeek and self-hosted vLLM servers (thanks @Alphaxalchemy, #129; hardened
  in #138).
 - End-to-end wiring for the documented reflection `--preferences` option
  (thanks @AKhozya, #131).
 ### Changed
 - Claude Code's Sleep plugin can now use a `pip`/`uv`-installed
  `skillopt-sleep` when no repository checkout is present (thanks
  @ichoosetoaccept, #107).
 - Qwen reasoning-model requests now use `max_completion_tokens` and omit
  unsupported temperature parameters (thanks @chirag127, #128).
 - Configuration files are read explicitly as UTF-8 (thanks @nankingjing,
  #124).
 ### Fixed
 - Preserve fractional rollout hard scores instead of coercing them to binary
  values (thanks @zixuanguo786-ctrl, #104).
 - Reject duplicate and overlapping IDs while materializing SearchQA manifests
  (thanks @zixuanguo786-ctrl, #105).
 - Make JSON-array extraction robust to unmatched braces and keep malformed
  scans linear-time (thanks @zixuanguo786-ctrl, #103; follow-up #136).
 - Package Markdown prompt assets in wheels and tolerate Windows temporary-file
  cleanup failures (thanks @nankingjing, #135; follow-up #137).
 - Exclude sub-agent transcripts and plugin-generated sessions from Sleep task
  mining (thanks @codeL1985, #99).
 - Normalize validation-gate density against the proposed edits and handle
  zero-edit candidates safely (thanks @SparshGarg999, #102).
 - Route optimizer-role MiniMax calls through the MiniMax backend (thanks
  @jcforever1, #116).
 - Surface Claude CLI spawn failures instead of silently turning them into zero
  scores (thanks @Phoenix0531-sudo, #126).
 - Improve Claude CLI behavior on Windows, including `.cmd` resolution and
  long-prompt handling (thanks @codeL1985, #98).
 - Preserve the scheduler's established annealing contract while expanding its
  endpoint and sequence coverage (thanks @nankingjing, #123; follow-up #133).
 ### Security
 - Prevent managed-identity credentials from being sent to non-Azure or
  non-HTTPS endpoints, and isolate compatible-provider request extensions
  from native Azure mode in SkillOpt-Sleep (#138, following
  @Alphaxalchemy's #129).
 ### Tests
 - Strengthen SkillOpt-Sleep verifier-discipline assertions, including recorded
  scores and gate actions (thanks @Tanmay9223, #96).
 - Add focused coverage for the validation-gate decision core and edit-budget
  schedulers (thanks @nankingjing, #122, #123).
 ### Acknowledgements 🙏
 Thank you to the contributors behind this unreleased work:
@AKhozya, @Alphaxalchemy, @Phoenix0531-sudo, @SparshGarg999,
@Tanmay9223, @chirag127, @codeL1985, @dimitarvdenev,
@ichoosetoaccept, @jcforever1, @nankingjing, and
@zixuanguo786-ctrl.
 ## [0.2.0] — 2026-07-02
@@ -7,7 +7,7 @@ Thank you for your interest in contributing! SkillOpt welcomes contributions of
 ```bash
 git clone https://github.com/microsoft/SkillOpt.git
 cd SkillOpt
-pip install -e ".[dev]"
+python -m pip install -e ".[dev,docs]"
 ```
 ## How to Contribute
@@ -16,23 +16,28 @@ pip install -e ".[dev]"
 Open a GitHub issue with reproduction steps, expected/actual behavior, and your config file (remove API keys).
 ### 🔧 Add a Benchmark
-See the [guide](docs/guide/new-benchmark.md) and use the scaffold at `skillopt/envs/_template/`.
+See the [guide](docs/guide/new-benchmark.md) and use the scaffold at
 `skillopt/envs/_template/`. Register the adapter lazily in both
 `scripts/train.py` and `scripts/eval_only.py`, and add focused tests.
 ### 🤖 Add a Model Backend
-See the [guide](docs/guide/new-backend.md).
+First check whether the built-in `openai_compatible` backend covers the
 provider. Otherwise follow the function-based backend contract in the
 [backend guide](docs/guide/new-backend.md), including routing, configuration,
 token accounting, and no-network tests.
 ### 📝 Improve Documentation
 ```bash
-pip install -e ".[docs]"
+python -m mkdocs serve   # Preview at http://localhost:8000
 mkdocs serve   # Preview at http://localhost:8000
 ```
 ## Pull Request Process
 1. Fork the repo and create a feature branch
-2. Make changes and test with an existing benchmark
+2. Make changes and run focused tests plus `python -m pytest -q`
 3. Submit a PR with a clear description
-4. Ensure CI passes
+4. For documentation changes, run `python -m mkdocs build --strict`
 5. Ensure CI passes
 ## Code Style
 - Follow existing patterns in the codebase
@@ -9,12 +9,12 @@
  <a href="https://trendshift.io/repositories/38498?utm_source=trendshift-badge&utm_medium=badge&utm_campaign=badge-trendshift-38498" target="_blank" rel="noopener noreferrer"><img src="https://trendshift.io/api/badge/trendshift/repositories/38498/weekly?language=Python" alt="microsoft%2FSkillOpt | Trendshift" width="250" height="55"/></a>
 </p>
-> 📖 **For installation, data preparation, training/eval commands, the full configuration reference, and framework internals, see the [Documentation & Reproduction Guide](https://microsoft.github.io/SkillOpt/docs/guideline.html)** (rendered on GitHub Pages).
+> 📖 **For installation, data preparation, training/eval commands, configuration, and framework internals, start with the versioned [SkillOpt documentation](https://github.com/microsoft/SkillOpt/blob/main/docs/index.md). A concise rendered overview is available in the [Documentation & Reproduction Guide](https://microsoft.github.io/SkillOpt/docs/guideline.html). We also maintain a [Changelog](CHANGELOG.md) for released and unreleased changes.**
 ---
 ## News 🔥🔥🔥
- **[2026-07-02]** 🚀 **SkillOpt [v0.2.0](https://github.com/microsoft/SkillOpt/releases/tag/v0.2.0) is out on [PyPI](https://pypi.org/project/skillopt/)!** Headline feature: **SkillOpt-Sleep**, a nightly offline self-evolution engine (harvest → mine → replay → consolidate, all behind a held-out validation gate) with multi-objective reward, experience replay + dream rollouts, and long-term memory — now shipped as the `skillopt-sleep` CLI. This release also adds cross-tool backends and plugin shells for **Claude, Codex, Copilot, Devin, and OpenClaw**, SearchQA split materialization, Windows robustness, and hardened JSON parsing. See the [release notes](https://github.com/microsoft/SkillOpt/releases/tag/v0.2.0) for the full changelog and contributor acknowledgements.
+- **[2026-07-02]** 🚀 **SkillOpt [v0.2.0](https://github.com/microsoft/SkillOpt/releases/tag/v0.2.0) is out on [PyPI](https://pypi.org/project/skillopt/)!** Headline feature: **SkillOpt-Sleep**, a nightly offline self-evolution engine (harvest → mine → replay → consolidate behind a held-out validation gate), now shipped as the `skillopt-sleep` CLI. It also includes experimental multi-objective, replay, and dream-rollout controls; the main CLI keeps conservative defaults and does not expose every experiment-harness control as a flag. The release source adds integration shells for **Claude Code, Codex, Copilot, and Devin**, plus an **OpenClaw reference adaptation**; these plugin/MCP files live in the repository rather than the PyPI wheel. It also adds SearchQA split materialization, Windows robustness, and hardened JSON parsing. See the [release notes](https://github.com/microsoft/SkillOpt/releases/tag/v0.2.0) for full release details and contributor acknowledgements.
 - **[2026-06-15]** 😴 **SkillOpt-Sleep (preview)** — a nightly offline self-evolution companion for local coding agents (Claude Code / Codex / Copilot): review past sessions, replay recurring tasks, and consolidate validated skills behind a held-out gate. See **[`docs/sleep/README.md`](docs/sleep/README.md)** for what it is, how to use it, and results.
 - **[2026-06-03]** 🎉 **[gbrain](https://github.com/garrytan/gbrain), [gbrain-evals](https://github.com/garrytan/gbrain-evals/blob/main/docs/benchmarks/2026-06-03-skillopt.md), and [darwin-skill](https://github.com/alchaincyf/darwin-skill) have all integrated SkillOpt.**
 - **[2026-06-02]** 🎉 **SkillOpt [v0.1.0](https://github.com/microsoft/SkillOpt/releases/tag/v0.1.0) is now available on [PyPI](https://pypi.org/project/skillopt/)!** Install with `pip install skillopt`. This initial release includes the full training loop (rollout → reflect → aggregate → select → update → evaluate), multi-backend support (OpenAI / Azure / Claude / Qwen / MiniMax), six built-in benchmarks, and WebUI dashboard.
@@ -31,9 +31,9 @@ which reliably improves over its starting point under feedback.
 **SkillOpt treats the skill document as the trainable state of a frozen
 agent**, and trains it with the discipline that makes weight-space
 optimization reproducible. A separate optimizer model turns scored rollouts
-into bounded add / delete / replace edits on a single skill document; a
+into bounded add / delete / replace edits on a single skill document; in the
-candidate edit is accepted only when it strictly improves a held-out
+default paper-style path, a candidate edit is accepted only when it strictly
-validation score. A textual learning-rate budget, a rejected-edit buffer,
+improves a held-out validation score. A textual learning-rate budget, a rejected-edit buffer,
 and an epoch-wise slow / meta update make skill training stable while
 adding **zero inference-time model calls** at deployment.
@@ -64,7 +64,9 @@ https://github.com/user-attachments/assets/eb12d3bc-371c-467f-904d-91b61f339ed7
 ### Adding a new backend
 A backend = a chat / exec target (e.g. `openai_chat`, `claude_chat`,
-`qwen_chat`, `minimax_chat`, `codex_exec`, `claude_code_exec`). See
+`qwen_chat`, `minimax_chat`, `openai_compatible`, `codex_exec`,
 `claude_code_exec`). If a provider implements the OpenAI Chat Completions
 protocol, try the built-in `openai_compatible` backend before adding code. See
 [`docs/guide/new-backend.md`](docs/guide/new-backend.md) for the full
 contract; in short you add a `skillopt/model/<name>_backend.py` module,
 register it in `skillopt/model/common.py` + `backend_config.py`, and wire
@@ -73,8 +75,9 @@ and `minimax_backend.py` are good templates.
 ### Adding a new benchmark
-A benchmark = a `skillopt/envs/<name>/` package with a `dataloader.py`, a
+A benchmark = a `skillopt/envs/<name>/` package with an adapter, a data loader,
-`rollout.py`, and an `initial.md` seed skill. See
+a scored rollout helper, a YAML config, and optionally an initial seed skill.
 See
 [`docs/guide/new-benchmark.md`](docs/guide/new-benchmark.md) for the full
 contract; the simplest reference is `skillopt/envs/searchqa/`.
@@ -93,6 +96,9 @@ python -m skillopt_webui.app
 | `--host` | `0.0.0.0` | Bind address |
 | `--share` | off | Create a public Gradio share link |
 The default host listens on every network interface. Use
 `--host 127.0.0.1` for local-only access.
 ---
 ## Citation
@@ -10,9 +10,10 @@ provided skills on a given split without re-running the training loop.
 > skills as portable artifacts. If you want to *train* your own skill,
 > use `scripts/train.py` per the top-level README.
 >
-> This is the first artifact batch. We plan to continue uploading the
+> This is the first optimized-skill artifact batch. We plan to continue
-> remaining optimized skills and benchmark split manifests as they are
+> uploading remaining paper artifacts as they are cleaned and verified. All
-> cleaned and verified.
+> six lightweight ID/path split manifests are already checked in under
 > `data/`; most still require materializing their upstream benchmark payload.
 ## What's here
@@ -35,12 +36,17 @@ longitudinal guidance — that's expected, not a formatting issue.
 invoking the optimizer. Example for SearchQA against the test split:
 ```bash
 # The checked-in SearchQA split is ID-only; materialize full examples first.
 python -m pip install -e ".[searchqa]"
 python scripts/materialize_searchqa.py
 python scripts/eval_only.py \
  --config configs/searchqa/default.yaml \
  --skill ckpt/searchqa/gpt5.5_skill.md \
  --split valid_unseen \
-  --split_dir data/searchqa_id_split \
+  --split_dir data/searchqa_split \
  --azure_openai_endpoint https://your-resource.openai.azure.com/ \
  --azure_openai_auth_mode api_key \
  --target_model gpt-5.5
 ```
@@ -52,13 +58,16 @@ is the selection / validation split, `train` is the training split, and
 ## On comparing to the paper numbers
 To compare against the paper-reported cells, use the same dataset split and
-scorer. SearchQA's split is checked in at `data/searchqa_id_split/` (400
+scorer. SearchQA's ID manifest is checked in at `data/searchqa_id_split/` (400
-train / 200 selection / 1400 test). For the other benchmarks, point
+train / 200 selection / 1400 test); the materializer writes the runnable
-`--split_dir` at your own materialized split; the loader is deterministic
+payload to `data/searchqa_split/`. All six lightweight split manifests are
-from `split_seed` (default `42`) + `split_ratio` (default `2:1:7`) when
+checked in under `data/`. ALFWorld's manifest records game-file paths; the
-`split_mode: ratio` is used, so a given `data_path` + seed reproduces
+other ID manifests still require you to materialize the corresponding
-across machines. Explicit per-benchmark split manifests are being prepared
+upstream benchmark payload into the documented `split_dir`. See
-for upload — see issues #14 and #21.
+[`data/README.md`](../data/README.md) for the exact status of each benchmark.
 When using `split_mode: ratio` instead, the loader is deterministic from
 `split_seed` (default `42`) + `split_ratio` (default `2:1:7`), so a given
 `data_path` + seed reproduces across machines.
 ## Why force-accept vs. gated slow-update matters
@@ -74,6 +83,5 @@ Current `main` defaults to `false` (force-accept mode), a newer
 post-submission behavior where the slow-update guidance is written into
 `current_skill` and `best_skill` unconditionally at the epoch boundary. If
 you re-train with the current default, you may produce a *different*
-`best_skill.md` than the one checked in here. Both modes are supported;
+`best_skill.md` than the one checked in here. Both modes are supported; see
-see the top-level README's "Configuration -> Slow-update acceptance mode"
+the [configuration reference](../docs/reference/config.md).
 section.
@@ -15,6 +15,7 @@ pip install -e ".[dev]"
 ### 🐛 Bug Reports
 Open an issue with:
 - Steps to reproduce
 - Expected vs actual behavior
 - Config file used (sanitize API keys)
@@ -25,21 +26,29 @@ Open an issue with:
 See [Add a New Benchmark](guide/new-benchmark.md) for the implementation guide.
 **Checklist:**
- [ ] Data loader in `skillopt/envs/<benchmark>/loader.py`
+
- [ ] Environment adapter in `skillopt/envs/<benchmark>/env.py`
+- [ ] Data loader in `skillopt/envs/<benchmark>/dataloader.py`
 - [ ] Scored rollout implementation in `skillopt/envs/<benchmark>/rollout.py`
 - [ ] Per-item `predictions/<id>/conversation.json` artifacts for shared reflection
 - [ ] Environment adapter in `skillopt/envs/<benchmark>/adapter.py`
 - [ ] Config file in `configs/<benchmark>/default.yaml`
- [ ] Registration in `skillopt/envs/__init__.py`
+- [ ] Lazy registration in `scripts/train.py` and `scripts/eval_only.py`
- [ ] Documentation page in `docs/`
+- [ ] Focused tests and an optional seed skill referenced by `env.skill_init`
 - [ ] Documentation update
 ### 🤖 New Model Backend
 See [Add a New Model Backend](guide/new-backend.md) for the implementation guide.
 **Checklist:**
- [ ] Backend in `skillopt/model/<backend>.py`
+
- [ ] Registration in `skillopt/model/__init__.py`
+- [ ] Function-based backend module in `skillopt/model/<name>_backend.py`
- [ ] API key entry in `.env.example`
+- [ ] Alias and default model in `skillopt/model/common.py`
- [ ] Documentation update
+- [ ] Optimizer/target whitelist entries in `skillopt/model/backend_config.py`
 - [ ] Dispatch, token tracking, and setter forwarding in `skillopt/model/__init__.py`
 - [ ] YAML/CLI wiring when the backend exposes structured config fields
 - [ ] Focused routing, configuration, tool-call, and token-accounting tests
 - [ ] `.env.example` and backend/configuration reference updates
 ### 📝 Documentation
@@ -59,9 +68,9 @@ mkdocs serve  # Preview at http://localhost:8000
 ## Pull Request Process
 1. Fork the repository
-2. Create a feature branch: `git checkout -b feature/my-benchmark`
+2. Create a feature branch: `git switch -c feature/my-benchmark`
 3. Make your changes
-4. Test with an existing benchmark config
+4. Run focused tests, the full test suite, and `mkdocs build --strict` when docs change
 5. Submit a PR with a clear description
 ## License
@@ -20,15 +20,53 @@ Benchmark configs inherit from `_base_/default.yaml` and override specific value
 ## Key Parameters
-### Model
+### Model Backends
 `optimizer_backend` controls reflection and skill editing;
 `target_backend` controls task rollout. The legacy `backend` field remains for
 backward compatibility, but explicit role fields are the clearest configuration.
 ```yaml
 model:
-  backend: azure_openai          # azure_openai | openai_chat | claude_code_exec | qwen
+  backend: azure_openai          # High-level compatibility label
-  optimizer: gpt-5.5               # Optimizer model (for reflection)
+  optimizer_backend: openai_chat
-  target: gpt-5.5               # Target model (for rollout)
+  target_backend: openai_chat
  optimizer: gpt-5.5             # Optimizer deployment/model
  target: gpt-5.5                # Target deployment/model
  azure_openai_auth_mode: api_key
 ```
 | Backend | Optimizer | Target | Configuration |
 |---|:---:|:---:|---|
 | `openai_chat` | ✓ | ✓ | Azure OpenAI, or its explicit compatibility auth mode |
 | `openai_compatible` | ✓ | ✓ | Generic OpenAI Chat Completions endpoint |
 | `claude_chat` | ✓ | ✓ | Claude Code CLI (`claude -p`) |
 | `qwen_chat` | ✓ | ✓ | Qwen served through an OpenAI-compatible local endpoint |
 | `minimax_chat` | ✓ | ✓ | MiniMax API |
 | `codex_exec` | — | ✓ | Codex CLI execution harness |
 | `claude_code_exec` | — | ✓ | Claude Code CLI execution harness |
 The current MiniMax adapter has one shared deployment. Set
 `model.minimax_model` when MiniMax is the target; a mixed-backend run cannot
 independently select a MiniMax optimizer model and a different target model.
 For a generic compatible provider, select the role backends explicitly rather
 than relying on a high-level shorthand:
 ```yaml
 model:
  optimizer_backend: openai_compatible
  target_backend: openai_compatible
  optimizer: deepseek-chat
  target: deepseek-chat
 ```
 The train/eval entry points apply `model.optimizer` and `model.target` after
 backend initialization. For the selected roles, these YAML values override
 `OPENAI_COMPATIBLE_MODEL`, `QWEN_CHAT_MODEL`, and their per-role environment
 forms. The environment model variables mainly seed direct library use; always
 set the role models in a training or evaluation config.
 ### Training
 ```yaml
@@ -96,9 +134,16 @@ Notes:
 ```yaml
 evaluation:
  use_gate: true                 # Validation gating (accept/reject updates)
  gate_metric: hard              # hard | soft | mixed
  gate_mixed_weight: 0.5         # Soft-score weight when metric=mixed
  use_semantic_density: false    # Optional instruction-density bonus
  eval_test: true                # Run test evaluation after training
 ```
 The default and paper-style setting is `use_gate: true`. Setting it to `false`
 still records selection scores but force-accepts every candidate, so it changes
 the optimization semantics and should be reported explicitly.
 ### Environment (Data)
 ```yaml
@@ -117,9 +162,10 @@ Override any config value from the command line:
 ```bash
 python scripts/train.py \
  --config configs/searchqa/default.yaml \
-  optimizer.learning_rate=16 \
+  --cfg-options \
-  optimizer.lr_scheduler=linear \
+    optimizer.learning_rate=16 \
-  gradient.analyst_workers=8
+    optimizer.lr_scheduler=linear \
    gradient.analyst_workers=8
 ```
 ## Environment Variables
@@ -128,11 +174,38 @@ Model credentials are loaded from environment variables:
 | Variable | Backend | Description |
 |---|---|---|
-| `AZURE_OPENAI_ENDPOINT` | azure_openai | Azure resource endpoint |
+| `AZURE_OPENAI_ENDPOINT` | `openai_chat` | Azure resource URL, or compatibility-mode base URL |
-| `AZURE_OPENAI_API_KEY` | azure_openai | Azure API key |
+| `AZURE_OPENAI_API_VERSION` | `openai_chat` | Azure API version |
-| `OPENAI_API_KEY` | openai | OpenAI API key |
+| `AZURE_OPENAI_AUTH_MODE` | `openai_chat` | `api_key`, `azure_cli`, `managed_identity`, or `openai_compatible` |
-| `ANTHROPIC_API_KEY` | claude | Anthropic API key |
+| `AZURE_OPENAI_API_KEY` | `openai_chat` | Required when auth mode is `api_key` or `openai_compatible` |
-| `QWEN_API_BASE` | qwen | Local Qwen vLLM endpoint |
+| `OPENAI_COMPATIBLE_BASE_URL` | `openai_compatible` | Generic Chat Completions base URL |
 | `OPENAI_COMPATIBLE_API_KEY` | `openai_compatible` | Provider API key; optional for local servers |
 | `OPENAI_COMPATIBLE_MODEL` | `openai_compatible` | Shared provider model ID for direct library use; train/eval YAML role models take precedence |
 | `CLAUDE_CLI_BIN` | `claude_chat` | Optional path to the `claude` executable; defaults to `claude` |
 | `ANTHROPIC_API_KEY` | `claude_chat` | Optional authentication method understood by the Claude CLI, not a direct SkillOpt API client |
 | `QWEN_CHAT_BASE_URL` | `qwen_chat` | Local Qwen/vLLM endpoint |
 | `QWEN_CHAT_MODEL` | `qwen_chat` | Served model name for direct library use; train/eval YAML role models take precedence |
 | `MINIMAX_BASE_URL` | `minimax_chat` | MiniMax-compatible base URL |
 | `MINIMAX_API_KEY` | `minimax_chat` | MiniMax API key |
 `OPTIMIZER_` and `TARGET_` prefixes provide per-role overrides for the
 Azure, OpenAI-compatible, and Qwen variable families. See the
 [Configuration Reference](../reference/config.md) for exact names.
 `claude_chat` launches the installed Claude Code CLI with `claude -p`; install
 and authenticate that CLI before use. Setting `ANTHROPIC_API_KEY` is one way
 the CLI may authenticate, but SkillOpt does not call the Anthropic API
 directly through this backend.
 ### Three OpenAI-compatible paths
 - Research, generic provider: select `openai_compatible` and use
  `OPENAI_COMPATIBLE_*`.
 - Research, Azure-family compatibility mode: keep `openai_chat`, set
  `AZURE_OPENAI_AUTH_MODE=openai_compatible`, and use `AZURE_OPENAI_*`.
 - SkillOpt-Sleep: run with `--backend azure_openai` and use the same
  compatibility-mode `AZURE_OPENAI_*` variables. Sleep does not read the
  research backend's role-specific variables.
 ## Full Reference
@@ -14,10 +14,9 @@ SkillOpt is designed around a core insight: **optimizing natural-language prompt
 | **Gradient aggregation** | Patch aggregation | Merge similar edits |
 | **Gradient clipping** | Edit selection | Cap max edits per step |
 | **Learning rate** | `learning_rate` | Max number of edits applied per step |
-| **LR scheduler** | `lr_scheduler` | Decay schedule: cosine, linear, constant |
+| **LR scheduler** | `lr_scheduler` | Edit-budget schedule: cosine, linear, constant, or autonomous |
 | **SGD step** | Skill update | Apply selected patches to document |
 | **Validation set** | Selection split | Gate checks improvement before accepting |
 | **Early stopping** | Gate patience | Reject updates that don't improve |
 | **Training step** | Step | One rollout → reflect → update cycle |
 | **Epoch** | Epoch | Full pass with slow update + meta memory |
 | **Momentum** | Slow update | Longitudinal comparison at epoch boundary |
@@ -34,7 +33,10 @@ SkillOpt is designed around a core insight: **optimizing natural-language prompt
 1. **Familiar mental model**: ML practitioners immediately understand how to tune SkillOpt
 2. **Principled hyperparameter search**: Grid search over `learning_rate` × `lr_scheduler` works just like in DL
-3. **Proven mechanisms**: Gating ≈ validation-based selection, patience ≈ early stopping, slow update ≈ momentum — all with strong theoretical motivation
+3. **Reusable mechanisms**: Gating provides validation-based model selection, while slow update plays a momentum-like role across epochs
 The gate is a per-candidate accept/reject decision. SkillOpt does not implement
 a gate-patience counter or stop training after a run of rejected candidates.
 ## Hyperparameter Transfer Rules
@@ -4,17 +4,43 @@ This guide walks through running a complete SkillOpt training on SearchQA.
 ## 1. Choose a Benchmark
-SkillOpt includes ready-to-use configs for several benchmarks:
+SkillOpt includes ready-to-use configs for several benchmarks. End-to-end
 runtime depends on the chosen models, provider latency, worker limits, and
 dataset size, so the project does not promise fixed wall-clock estimates.
-| Benchmark | Difficulty | Typical Runtime |
+| Benchmark | Modality | Additional setup |
 |---|---|---|
-| SearchQA | ⭐ Easy | ~30 min |
+| SearchQA | Text QA | Materialize the released ID manifest |
-| DocVQA | ⭐⭐ Medium | ~2 hours |
+| DocVQA | Document/image QA | Obtain and materialize images and examples |
-| ALFWorld | ⭐⭐⭐ Hard | ~3 hours |
+| ALFWorld | Embodied agent | Install ALFWorld and download its assets |
-We'll use **SearchQA** as it's the fastest to complete.
+We'll use **SearchQA** because it is the simplest text-only walkthrough.
-## 2. Configure
+## 2. Install and Materialize SearchQA
 The repository contains a stable SearchQA ID manifest, not the full runnable
 examples. From a source checkout, install the data extra and materialize the
 split once:
 ```bash
 python -m pip install -e ".[searchqa]"
 python scripts/materialize_searchqa.py
 ```
 By default, the materializer reads `data/searchqa_id_split/` and writes the
 train/validation/test payloads expected by the config to
 `data/searchqa_split/`; both paths have command-line overrides.
 ## 3. Configure
 Configure and export one model backend as described in
 [Installation](installation.md#environment-variables). For example:
 ```bash
 cp .env.example .env
 # Edit .env, choose one authentication mode, then export it:
 set -a; source .env; set +a
 ```
 Review the config file:
@@ -42,56 +68,55 @@ evaluation:
  use_gate: true          # (validation gating)
 ```
-## 3. Train
+## 4. Train
 ```bash
-python scripts/train.py --config configs/searchqa/default.yaml
+python scripts/train.py \
  --config configs/searchqa/default.yaml \
  --out_root outputs/searchqa_first_run
 ```
-You'll see output like:
+The command prints the resolved backend/data configuration, per-step rollout
 and gate progress, and the generated output directory.
 ## 5. Monitor
 The explicit `--out_root` above creates this run directory:
 ```
-[Step 1/8] Rollout: 20 items, 4 workers...
+outputs/searchqa_first_run/
-[Step 1/8] Score: 0.65 → Reflect...
+├── config.json
-[Step 1/8] 6 edit patches generated
+├── runtime_state.json
 [Step 1/8] Selected 4 edits (lr=8, cosine → 7.7)
 [Step 1/8] Gate: val score 0.68 > 0.65 ✓ ACCEPT
 [Step 2/8] ...
 ```
 ## 4. Monitor
 Training outputs are saved to `outputs/<benchmark>/<run_id>/`:
 ```
 outputs/searchqa/2024-01-15_10-30-00/
 ├── steps/
 │   ├── step_0001/
 │   │   ├── candidate_skill.md
 │   │   ├── step_record.json
 │   │   └── trajectory_digest.json
 │   └── step_0002/
 ├── slow_update/
 │   └── epoch_02/
 ├── meta_skill/
 │   └── epoch_02/
 ├── skills/
 │   └── step_0001.md
 ├── best_skill.md
 ├── history.json
-└── config.yaml
+├── best_skill.md
 ├── skills/
 │   └── skill_vXXXX.md
 ├── steps/
 │   └── step_XXXX/
 │       ├── candidate_skill.md
 │       ├── step_record.json
 │       └── trajectory_digest.json
 ├── slow_update/
 │   └── epoch_XX/
 └── meta_skill/
    └── epoch_XX/
 ```
-## 5. Evaluate
+## 6. Evaluate
 Evaluate the best skill on the test split:
 ```bash
 python scripts/eval_only.py \
  --config configs/searchqa/default.yaml \
-  --skill outputs/searchqa/<run_id>/skills/best_skill.md
+  --skill outputs/searchqa_first_run/best_skill.md \
  --split valid_unseen
 ```
 The `--skill` path above is the training artifact. Evaluation writes
 `eval_summary.json` to its own timestamped `outputs/eval_.../` directory unless
 you pass an explicit `--out_root`; it does not overwrite the training run.
 ## WebUI
 Prefer a graphical interface? Launch the WebUI:
@@ -101,7 +126,9 @@ pip install -e ".[webui]"
 python -m skillopt_webui.app
 ```
-Then open `http://localhost:7860` in your browser to configure parameters and launch training.
+Then open `http://localhost:7860` in your browser to configure parameters and
 launch training. The default host is `0.0.0.0`; pass `--host 127.0.0.1` for a
 local-only dashboard.
 ## Next Steps
@@ -3,16 +3,44 @@
 ## Requirements
 - Python ≥ 3.10
- At least one model API key (Azure OpenAI, OpenAI, Anthropic, or local Qwen)
+- For research training/evaluation, access to at least one configured model
  backend (hosted API, local server, or an installed execution CLI)
 - The SkillOpt-Sleep `mock` backend needs no credentials
-## Quick Install
+## Choose an Install
 ### PyPI
 Use PyPI for the Python packages and installed commands:
 ```bash
 python -m pip install skillopt
 skillopt-sleep --help
 ```
 This installs `skillopt-train`, `skillopt-eval`, and `skillopt-sleep`. The wheel
 does not include the repository's benchmark configs, data materializers,
 agent-integration shells/MCP servers, or development tests; use a source
 checkout for those files.
 !!! important "PyPI versus `main`"
    These docs track the latest `main`. The current PyPI release is `0.2.0`.
    The generic research `openai_compatible` backend, SkillOpt-Sleep handoff,
    Sleep support for non-Azure OpenAI-compatible endpoints, and the Sleep
    `--preferences` flag landed after that release and require a source install
    from `main` until the next release.
 ### Source checkout
 ```bash
 git clone https://github.com/microsoft/SkillOpt.git
 cd SkillOpt
-pip install -e .
+python -m pip install -e .
 ```
 Use the source checkout for paper reproduction, built-in benchmark configs,
 and contributions.
 ## Optional Dependencies
 Install extras for specific benchmarks or backends:
@@ -20,68 +48,115 @@ Install extras for specific benchmarks or backends:
 === "ALFWorld"
    ```bash
-    pip install -e ".[alfworld]"
+    python -m pip install -e ".[alfworld]"
    ```
-=== "Claude Backend"
+=== "Claude agent SDK (optional)"
    ```bash
-    pip install -e ".[claude]"
+    python -m pip install -e ".[claude]"
    ```
    This extra does not install the `claude` executable. The research
    `claude_chat` backend launches `claude -p`, so install and authenticate the
    Claude Code CLI separately. The SDK extra is only needed when selecting an
    SDK-backed Claude Code exec path.
 === "Qwen (Local)"
    ```bash
-    pip install -e ".[qwen]"
+    python -m pip install -e ".[qwen]"
    ```
 === "SearchQA data"
    ```bash
    python -m pip install -e ".[searchqa]"
    ```
 === "WebUI"
    ```bash
-    pip install -e ".[webui]"
+    python -m pip install -e ".[webui]"
    ```
 === "Development"
    ```bash
-    pip install -e ".[dev]"
+    python -m pip install -e ".[dev]"
    ```
 === "All"
    ```bash
-    pip install -e ".[alfworld,claude,qwen,webui,dev]"
+    python -m pip install -e ".[alfworld,claude,qwen,searchqa,webui,docs,dev]"
    ```
 ## Environment Variables
-Copy the example `.env` file and fill in your credentials:
+From a source checkout, copy the template and fill in only the backend you
 will use:
 ```bash
 cp .env.example .env
 ```
-Edit `.env` with your API keys:
+SkillOpt does not automatically load `.env`; export it into the current shell
 before running commands:
-```ini
+```bash
-# Azure OpenAI (default backend)
+set -a
-AZURE_OPENAI_ENDPOINT=https://your-resource.openai.azure.com/
+source .env
-AZURE_OPENAI_API_KEY=your-key
+set +a
 # Or use OpenAI directly
 OPENAI_API_KEY=sk-...
 # Or Anthropic Claude
 ANTHROPIC_API_KEY=sk-ant-...
 ```
 For Azure OpenAI with API-key authentication, the minimum settings are:
 ```ini
 AZURE_OPENAI_ENDPOINT=https://your-resource.openai.azure.com/
 AZURE_OPENAI_API_VERSION=2024-12-01-preview
 AZURE_OPENAI_API_KEY=your-key
 AZURE_OPENAI_AUTH_MODE=api_key
 ```
 Use `AZURE_OPENAI_AUTH_MODE=azure_cli` for Azure CLI credentials, or
 `managed_identity` with an optional
 `AZURE_OPENAI_MANAGED_IDENTITY_CLIENT_ID`.
 The research `claude_chat` backend is a Claude Code CLI adapter, not a direct
 Anthropic API client. Install and authenticate `claude`, and set
 `CLAUDE_CLI_BIN` only if the executable is not available as `claude` on
 `PATH`. `ANTHROPIC_API_KEY` is one authentication option the CLI may consume.
 OpenAI-compatible servers have three distinct entry points:
 1. The research engine's generic `openai_compatible` backend uses
   `OPENAI_COMPATIBLE_BASE_URL`, `OPENAI_COMPATIBLE_API_KEY`, and
   `OPENAI_COMPATIBLE_MODEL`.
 2. The research `openai_chat` backend can use
   `AZURE_OPENAI_AUTH_MODE=openai_compatible` with
   `AZURE_OPENAI_ENDPOINT` and `AZURE_OPENAI_API_KEY`.
 3. SkillOpt-Sleep uses the same Azure-family variables as item 2 with
   `skillopt-sleep run --backend azure_openai`.
 For research train/eval commands, `model.optimizer` and `model.target` in the
 YAML config are applied after backend initialization. They override model-name
 environment variables such as `OPENAI_COMPATIBLE_MODEL` and
 `QWEN_CHAT_MODEL`; set both role models explicitly when selecting those
 backends.
 !!! tip
-    You only need credentials for the backend you plan to use. Azure OpenAI is the default.
+    You only need to configure the backend you plan to use. See
    [Configuration](configuration.md#model-backends) for exact backend names
    and role-specific overrides.
 ## Verify Installation
 ```bash
 python -c "import skillopt; print('SkillOpt ready!')"
 skillopt-train --help
 skillopt-eval --help
 skillopt-sleep --help
 ```
 ## Next Steps
@@ -35,9 +35,14 @@ Use the split names your adapter maps to SkillOpt phases:
 - `val` or `valid_seen` for selection/gating
 - `test` or `valid_unseen` for final evaluation
-## 2. Support an offline mock mode
+## 2. Support a genuinely offline mock mode
-Add a configuration flag such as `mock: true` to your adapter. In mock mode, `rollout()` should return deterministic responses without calling external model APIs.
+Add a configuration flag such as `mock: true` to your adapter. In mock mode,
 `rollout()` should return deterministic responses without calling external
 model APIs. The inherited `EnvAdapter.reflect()` does call the configured
 optimizer backend, so a no-credential smoke test must also override
 `reflect()` in mock mode to return a small, schema-valid deterministic patch
 (and delegate to `super().reflect(...)` otherwise).
 This lets you verify the SkillOpt loop with a fast command such as:
@@ -46,13 +51,14 @@ python scripts/train.py \
  --config configs/myenv/tiny_mock.yaml
 ```
-Mock mode should still write the same artifacts as a real run, for example:
+Mock mode should still exercise the trainer's normal artifact paths, including:
- `responses.json`
+- `config.json`, `runtime_state.json`, and `history.json`
- `rollout_results.json`
+- `skills/skill_vXXXX.md`
- `ranked_edits.json`
+- `steps/step_XXXX/ranked_edits.json`
- `candidate_skill.md`
+- `steps/step_XXXX/candidate_skill.md`
- `summary.json`
+- `steps/step_XXXX/step_record.json`
 - the final `summary.json`
 ## 3. Keep the smoke config tiny
@@ -127,7 +133,7 @@ For the real tiny run, verify that:
 - the run completes
 - `summary.json` is written
- `ranked_edits.json` contains the expected ranking metadata
+- the step directory's `ranked_edits.json` contains the expected ranking metadata
 - any optimizer bridge log marks the response schema as valid
 - no generated files are written outside `out_root`
@@ -1,11 +1,17 @@
 # Add a New Model Backend
-SkillOpt supports multiple LLM backends. This guide shows how to add your own.
+SkillOpt's model layer is function-based: each chat backend is a Python module
 that exposes the call, token-tracking, and deployment-setting functions used by
 `skillopt.model`. There is no backend base class or registry object to subclass.
 ## Built-in: the generic OpenAI-compatible backend
 !!! note "Version requirement"
    This backend landed after v0.2.0. Install from the latest `main` until it is
    included in the next release.
 Before writing a new backend, check whether your provider already speaks the
-OpenAI Chat Completions protocol. Most do — in which case you can use the
+OpenAI Chat Completions protocol. Most do, in which case you can use the
 built-in **`openai_compatible`** backend
 (`skillopt/model/openai_compatible_backend.py`) with no code changes.
@@ -21,15 +27,16 @@ A single `base_url` + `api_key` pair lets you point SkillOpt at, for example:
 | LiteLLM proxy | `http://localhost:4000` | any proxied model |
 | OpenRouter / Fireworks / xAI / … | provider base URL | provider model id |
-Select it as the optimizer and/or target backend:
+### Python API
 Select and configure the backend directly when embedding SkillOpt as a Python
 library:
 ```python
 import skillopt.model as model
-# Shorthand: use it for both optimizer and target.
+# Use the generic backend for both optimizer and target calls.
 model.set_backend("openai_compatible")
 # Point it at a provider (shared, or per-role with optimizer_*/target_*).
 model.configure_openai_compatible(
    base_url="https://api.deepseek.com/v1",
    api_key="sk-...",
@@ -37,147 +44,157 @@ model.configure_openai_compatible(
 )
 ```
-Or configure it entirely through environment variables (role-specific
+`configure_openai_compatible()` also accepts `optimizer_*` and `target_*`
-`OPTIMIZER_*` / `TARGET_*` variants override the shared ones):
+arguments when the two roles use different endpoints or models.
 ### Environment variables
 The shared variables below configure both roles. Role-specific
 `OPTIMIZER_OPENAI_COMPATIBLE_*` and `TARGET_OPENAI_COMPATIBLE_*` variables take
 precedence:
 ```bash
 export TARGET_BACKEND=openai_compatible
 export OPENAI_COMPATIBLE_BASE_URL="https://api.groq.com/openai/v1"
 export OPENAI_COMPATIBLE_API_KEY="gsk_..."
 export OPENAI_COMPATIBLE_MODEL="llama-3.3-70b-versatile"
 # Optional: OPENAI_COMPATIBLE_TEMPERATURE, _MAX_TOKENS, _TIMEOUT_SECONDS
 ```
-The backend uses the official `openai` SDK, records token usage through the
+For direct library use, `OPTIMIZER_BACKEND=openai_compatible` and/or
-shared tracker, supports tool/function calling via
+`TARGET_BACKEND=openai_compatible` select the role. The training and evaluation
-`chat_target_messages(..., tools=...)`, and exposes
+scripts resolve backend selection from their config, so set the split fields
-`count_tokens()` (tiktoken with a character-based fallback for non-OpenAI
+explicitly there:
 models). Only write a brand-new backend if your provider is *not*
 OpenAI-compatible.
 ## Backend Architecture
 ```
 skillopt/model/
 ├── base.py           # Abstract base class
 ├── azure_openai.py   # Azure OpenAI backend
 ├── openai_model.py   # Direct OpenAI backend
 ├── claude.py         # Anthropic Claude backend
 ├── qwen.py           # Local Qwen (vLLM) backend
 └── your_backend.py   # Your new backend
 ```
 ## Step 1: Create the Backend
 Create `skillopt/model/your_backend.py`:
 ```python
 from skillopt.model.base import ModelBackend, ModelResponse
 class YourBackend(ModelBackend):
    """Your custom model backend."""
    def __init__(self, cfg: dict):
        super().__init__(cfg)
        self.model_name = cfg.get('model_name', 'your-default-model')
        self.api_key = os.environ.get('YOUR_API_KEY', '')
        self.client = self._init_client()
    def _init_client(self):
        """Initialize API client."""
        # TODO: Set up your API client
        pass
    async def generate(
        self,
        messages: list[dict],
        temperature: float = 0.7,
        max_tokens: int = 4096,
        **kwargs
    ) -> ModelResponse:
        """
        Generate a completion.
        Args:
            messages: Chat messages [{"role": "...", "content": "..."}]
            temperature: Sampling temperature
            max_tokens: Maximum tokens in response
        Returns:
            ModelResponse with content, usage, and metadata
        """
        response = await self.client.chat(
            model=self.model_name,
            messages=messages,
            temperature=temperature,
            max_tokens=max_tokens,
        )
        return ModelResponse(
            content=response.text,
            usage={
                'prompt_tokens': response.usage.input,
                'completion_tokens': response.usage.output,
            },
            model=self.model_name,
        )
    async def generate_with_tools(
        self,
        messages: list[dict],
        tools: list[dict],
        **kwargs
    ) -> ModelResponse:
        """Generate with tool/function calling support."""
        # Optional: implement if your model supports tool use
        raise NotImplementedError("Tool use not supported")
 ```
 ## Step 2: Register the Backend
 Add to `skillopt/model/__init__.py`:
 ```python
 from .your_backend import YourBackend
 BACKEND_REGISTRY = {
    # ... existing backends ...
    'your_backend': YourBackend,
 }
 ```
 ## Step 3: Configure
 Use your backend in any config:
 ```yaml
 model:
-  backend: your_backend
+  optimizer_backend: openai_compatible
-  model_name: your-model-id
+  target_backend: openai_compatible
-  temperature: 0.7
+  optimizer: llama-3.3-70b-versatile
-  max_tokens: 4096
+  target: llama-3.3-70b-versatile
 ```
-Set credentials via environment variable:
+Equivalently, override those fields on the command line:
 ```bash
-export YOUR_API_KEY="your-key"
+python scripts/train.py --config configs/searchqa/default.yaml \
  --cfg-options \
  model.optimizer_backend=openai_compatible \
  model.target_backend=openai_compatible \
  model.optimizer=llama-3.3-70b-versatile \
  model.target=llama-3.3-70b-versatile
 ```
-## Required Interface
+Do not rely on the legacy high-level `model.backend` label to replace the two
 role-specific fields in a structured config.
-Your backend must implement these methods:
+The generic backend uses the official `openai` SDK and the Chat Completions
 API. It records token usage through the shared tracker, supports provider tool
 calling through `chat_*_messages(..., tools=...)`, and exposes `count_tokens()`
 (tiktoken when available, with a character-based fallback). Provider-specific
 Responses API features are outside this backend's contract.
-| Method | Required | Description |
+Only write a new backend when the provider is not compatible with this surface
-|---|---|---|
+or requires behavior that cannot be expressed by its configuration.
 | `generate()` | ✅ | Basic text generation |
 | `generate_with_tools()` | Optional | Tool/function calling |
 | `count_tokens()` | Optional | Token counting for context management |
-## Tips
+## Backend architecture
-!!! tip
+The active split optimizer/target dispatcher is the public
-    - Test your backend with `python -c "from skillopt.model.your_backend import YourBackend"` first
+`skillopt/model/__init__.py` module:
-    - Use `async` methods for all API calls — SkillOpt uses asyncio throughout
+
-    - Implement retry logic with exponential backoff for production use
+```text
-    - Add your API key to `.env.example` when submitting a PR
+skillopt/model/
 ├── common.py                       # aliases, default models, token/response helpers
 ├── backend_config.py               # optimizer/target whitelists and runtime selection
 ├── __init__.py                     # public API and split-role dispatch
 ├── openai_compatible_backend.py    # generic Chat Completions example
 ├── qwen_backend.py                 # raw-HTTP chat example with per-role config
 ├── minimax_backend.py              # compact raw-HTTP chat example
 ├── codex_harness.py                # target-only exec harnesses
 └── router.py                       # legacy single-backend compatibility surface
 ```
 `router.py` is not the dispatcher used by the current training loop. Update it
 only if the new backend must also be exposed through that legacy single-backend
 API.
 ## Step 1: implement the module contract
 Create a module such as `skillopt/model/your_backend.py`. Copy the signatures
 from `openai_compatible_backend.py` or `qwen_backend.py`; model calls in the
 current framework are synchronous.
 For a chat backend that supports both roles, the public module surface is:
 | Function | Purpose |
 |---|---|
 | `chat_optimizer(...)` | Optimizer system/user call; returns `(text, usage)` |
 | `chat_target(...)` | Target system/user call; returns `(text, usage)` |
 | `chat_optimizer_messages(...)` | Optimizer message-list call, including optional tools |
 | `chat_target_messages(...)` | Target message-list call, including optional tools |
 | `get_token_summary()` | Return per-stage counters plus `_total` |
 | `reset_token_tracker()` | Clear this backend's counters |
 | `set_optimizer_deployment(name)` | Change the optimizer model at runtime |
 | `set_target_deployment(name)` | Change the target model at runtime |
 | `set_reasoning_effort(effort)` | Apply or safely ignore the shared reasoning setting |
 Every call returns a usage dict with `prompt_tokens`, `completion_tokens`, and
 `total_tokens`. Use `TokenTracker` from `skillopt.model.common` and record each
 call exactly once. Message-list calls that accept tools should return the
 compatibility message objects from `common.py` when `return_message=True`.
 Provider-specific configuration helpers and `count_tokens()` are optional, but
 their state must be safe to update while calls may run concurrently. Keep
 credentials out of logs and persisted artifacts.
 Exec-style targets do not implement this chat contract. They are target-only
 and are integrated through `codex_harness.py` plus environment-specific rollout
 code.
 ## Step 2: register and route the backend
 A new backend normally requires all of the following:
 1. Add its canonical name, aliases, and default model to
   `skillopt/model/common.py`.
 2. Add the canonical name to the appropriate optimizer and/or target whitelist
   in `skillopt/model/backend_config.py`. Do not advertise a role the module
   cannot execute.
 3. Import the module in `skillopt/model/__init__.py` and add dispatch branches
   for every supported call surface.
 4. Include its counters in `get_token_summary()` / `reset_token_tracker()` and
   forward the shared deployment/reasoning setters where applicable.
 5. If it has YAML settings, add structured-to-flat mappings in
   `skillopt/config.py`, wire them through `scripts/train.py` and
   `scripts/eval_only.py`, and document their precedence over environment
   variables.
 6. Update `router.py` only when legacy single-backend compatibility is part of
   the intended feature.
 Backend selection in `scripts/train.py` must use
 `model.optimizer_backend` and `model.target_backend`. A high-level
 `model.backend` alias alone is not a substitute for this explicit split.
 ## Step 3: test the integration
 Add focused tests under `tests/` that do not call a live provider. At minimum,
 cover:
 - optimizer and target whitelist validation;
 - routing for text and message-list calls;
 - role-specific configuration precedence;
 - tool-call compatibility, if supported;
 - deployment/reasoning setters;
 - token accounting, including a single correct `_total`;
 - actionable errors for missing credentials or invalid responses.
 Then run the focused test, the full suite, and the documentation build:
 ```bash
 python -m pytest tests/test_your_backend.py -q
 python -m pytest tests/ -q
 mkdocs build --strict
 ```
 Also update `.env.example`, the configuration reference, and the backend table
 in the API reference. Add an optional dependency extra only when the backend
 requires a package that is not already a core dependency.
@@ -14,16 +14,18 @@ To add a benchmark you implement four things:
 1. **A `SplitDataLoader` subclass** — knows how to load train / val / test
   item dicts from disk.
-2. **A rollout helper** — runs the target model on a batch of items
+2. **A rollout helper** — runs the target model on a batch of items, scores
-   under the current skill and scores each prediction.
+   each prediction, and persists the per-item conversation consumed by the
   shared reflection stage.
 3. **An `EnvAdapter` subclass** — wires the loader + rollout helper into
-   SkillOpt's lifecycle (`build_*_env`, `rollout`, `reflect`,
+   SkillOpt's lifecycle (`build_*_env`, `rollout`, and `get_task_types`).
-   `get_task_types`).
+   The shared `reflect()` implementation is inherited unless the benchmark
   needs custom reflection logic.
 4. **A YAML config** — references your env name plus the standard
   train / optimizer / gradient knobs.
-Then one line in `scripts/train.py`'s `_register_builtins()` makes it
+Then lazy registration in the training and evaluation scripts makes it
-discoverable.
+discoverable without importing optional dependencies at startup.
 ---
@@ -99,8 +101,8 @@ def _score(prediction: str, ground_truth: str) -> tuple[int, float]:
    return hard, soft
-def _rollout_one(item: dict, skill_content: str,
+def _rollout_one(item: dict, skill_content: str, *, prediction_dir: Path,
-                 *, max_completion_tokens: int) -> dict:
+                 max_completion_tokens: int) -> dict:
    system = skill_content
    user = (
        f"Question: {item['question']}\n\n"
@@ -113,14 +115,33 @@ def _rollout_one(item: dict, skill_content: str,
        max_completion_tokens=max_completion_tokens,
    )
    hard, soft = _score(prediction, item.get("ground_truth", ""))
    # EnvAdapter.reflect() reads this exact trajectory path. Keep item IDs
    # unique and filesystem-safe.
    task_dir = prediction_dir / str(item["id"])
    task_dir.mkdir(parents=True, exist_ok=True)
    conversation = [
        {"role": "system", "content": system},
        {"role": "user", "content": user},
        {"role": "assistant", "content": prediction},
    ]
    (task_dir / "conversation.json").write_text(
        json.dumps(conversation, ensure_ascii=False, indent=2),
        encoding="utf-8",
    )
    return {
        "id": str(item["id"]),
        "hard": hard,
        "soft": soft,
        "predicted_answer": prediction,
        "task_description": item.get("question", ""),
        "question": item.get("question", ""),
        "reference_text": item.get("reference_text", ""),
        "task_type": item.get("task_type", "docfaithful"),
        "target_system_prompt": system,
        "target_user_prompt": user,
        "n_turns": 1,
    }
@@ -128,15 +149,18 @@ def run_batch(*, items: list[dict], skill_content: str, out_root: str,
              workers: int = 4, max_completion_tokens: int = 4096) -> list[dict]:
    """Run a batch of episodes sequentially or with a thread pool."""
    os.makedirs(out_root, exist_ok=True)
    prediction_dir = Path(out_root, "predictions")
    # For brevity we go sequentially — swap in concurrent.futures.ThreadPoolExecutor
    # when network / model latency dominates.
    results = [
        _rollout_one(item, skill_content,
                     prediction_dir=prediction_dir,
                     max_completion_tokens=max_completion_tokens)
        for item in items
    ]
    Path(out_root, "rollouts.json").write_text(
-        json.dumps(results, ensure_ascii=False, indent=2)
+        json.dumps(results, ensure_ascii=False, indent=2),
        encoding="utf-8",
    )
    return results
 ```
@@ -150,9 +174,17 @@ Two design points worth flagging:
 - **Use `skillopt.model.chat_target`**, not raw OpenAI/Claude calls.
  That routes through whichever **chat** target backend the user
  configured (`openai_chat` / `claude_chat` / `qwen_chat` /
-  `minimax_chat`) without your adapter caring. Exec-style backends
+  `minimax_chat` / `openai_compatible`) without your adapter caring.
-  (`codex_exec`, `claude_code_exec`) need env-specific rollout code —
+  Exec-style backends (`codex_exec`, `claude_code_exec`) need
-  see `skillopt/envs/swebench/` for an example.
+  environment-specific rollout code —
  see `skillopt/model/codex_harness.py` together with the rollout modules in
  `skillopt/envs/searchqa/`, `skillopt/envs/docvqa/`, or
  `skillopt/envs/officeqa/` for working examples.
 - **Persist a conversation for reflection.** The shared `EnvAdapter.reflect()`
  looks under `<rollout_dir>/predictions/<result-id>/conversation.json` and
  skips results whose trajectory is absent or empty. Returning `hard`/`soft`
  scores alone is sufficient for evaluation, but it cannot produce learning
  patches.
 ## Step 4 — Implement the environment adapter
@@ -277,9 +309,11 @@ the answer against `item["ground_truth"]`, and returns a list of dicts:
 ]
 ```
-The trainer only requires `id`, `hard`, `soft`. The rest is preserved on
+The trainer requires `id`, `hard`, and `soft` for scoring. The remaining fields
-`RolloutResult.extras` (see `skillopt/types.py`) and is what your
+are preserved on `RolloutResult.extras` (see `skillopt/types.py`). The shared
-`reflect()` consumes via `run_minibatch_reflect`.
+reflection implementation combines those fields with each persisted
 `predictions/<id>/conversation.json`; without that file the result is omitted
 from reflection.
 ## Step 5 — Register the adapter
@@ -294,9 +328,11 @@ and add to `_register_builtins()`:
        pass  # docfaithful deps not installed — skip
 ```
-There is **no `BENCHMARK_REGISTRY` dict in `skillopt/envs/__init__.py`** —
+Mirror the same lazy registration in
-the registry lives in `scripts/train.py` and is populated lazily so that
+[`scripts/eval_only.py`](https://github.com/microsoft/SkillOpt/blob/main/scripts/eval_only.py)
-optional deps don't break `--help`.
+so standalone evaluation can resolve the environment too. There is **no
 `BENCHMARK_REGISTRY` dict in `skillopt/envs/__init__.py`**; both entry points
 keep a small lazy registry so optional dependencies do not break `--help`.
 ## Step 6 — Create the YAML config
@@ -322,9 +358,7 @@ optimizer:
 env:
  name: docfaithful
-  # Optional: a seed skill document. Create this file (or any markdown
+  # Point to an existing Markdown file. Use an empty file to start blank.
  # file) yourself before the first run, or omit the key to let SkillOpt
  # start from an empty skill.
  skill_init: skillopt/envs/docfaithful/skills/initial.md
  split_mode: split_dir
  split_dir: data/docfaithful_split
@@ -341,7 +375,7 @@ env:
 ## Step 7 — Run
 ```bash
-# If you set skill_init above, create the seed skill first:
+# Create the file referenced by env.skill_init before the first run:
 #   mkdir -p skillopt/envs/docfaithful/skills
 #   echo "# DocFaithful initial skill" > skillopt/envs/docfaithful/skills/initial.md
@@ -364,8 +398,11 @@ you forgot to implement one of the four abstract methods on `EnvAdapter`:
  `hard` / `soft`.
 - Noisy scoring kills the optimizer. Spend time on `run_batch`'s scoring
  before you spend time on prompts.
 - If training repeatedly reports `skip_no_patches`, first verify that every
  rollout result has a non-empty
  `rollout/predictions/<id>/conversation.json` using the same `id` string.
 - If your benchmark needs heavy optional deps (selenium, vllm, ...),
-  wrap the registration block with `try / except ImportError` (Step 5)
+  wrap both registration blocks with `try / except ImportError` (Step 5)
  so people without those deps can still `--help`.
 - Copy `skillopt/envs/_template/` as a starting skeleton — it now
  implements the real abstract methods.
@@ -38,23 +38,45 @@ During training, the skill document is modified by **edit patches**:
 2. **Modifications**: Refining existing rules that are partially correct
 3. **Deletions**: Removing rules that consistently lead to errors
-Each edit is validated through the **gate** mechanism before being permanently accepted.
+Selected edits are applied together to produce a candidate skill. With the
 validation gate enabled, that candidate replaces the current skill only when
 its score on the selection split strictly improves.
 SkillOpt may maintain two protected, machine-managed regions:
 ```markdown
 <!-- SLOW_UPDATE_START -->
 ... epoch-level longitudinal guidance ...
 <!-- SLOW_UPDATE_END -->
 <!-- APPENDIX_START -->
 ... skill-aware execution reminders ...
 <!-- APPENDIX_END -->
 ```
 Normal edit patches cannot modify either region. Slow update owns the first;
 optional skill-aware reflection owns the second. Preserve these markers when
 copying or manually inspecting a trained skill.
 ## Initial Skill
 You can start training with:
- **Empty skill**: The system learns everything from scratch
+- **Empty skill**: Point `env.skill_init` to an empty Markdown file
 - **Seed skill**: Provide initial instructions to bootstrap training
 - **Pre-trained skill**: Transfer a skill from a related benchmark
 Configure the initial skill in your YAML:
 ```yaml
-train:
+env:
-  init_skill: "path/to/initial_skill.md"  # or omit for empty
+  skill_init: path/to/initial_skill.md
 ```
 To start from scratch, create an empty Markdown file and use its path. A missing
 path currently also starts blank, so using an explicit file avoids silently
 treating a typo as an empty skill.
 ## Skill Quality Metrics
 Track your skill's evolution through:
@@ -62,15 +84,16 @@ Track your skill's evolution through:
 - **Validation score**: Primary metric on the selection split
 - **Test score**: Final metric on held-out test data
 - **Skill length**: Total tokens in the document
- **Edit acceptance rate**: Fraction of proposed edits that pass gating
+- **Candidate acceptance rate**: Fraction of candidate skill updates that pass
  gating; multiple proposed edits can be combined into one candidate
 ## Best Practices
 !!! tip "Tips for better skills"
    1. **Start with a seed skill** (`env.skill_init`) if you have domain knowledge — it converges faster
    2. **Use cosine LR schedule** — aggressive early exploration + careful late refinement
-    3. **Enable slow update** (`use_slow_update: true`) to prevent forgetting across epochs
+    3. **Enable slow update** (`optimizer.use_slow_update: true`) to counter forgetting across epochs
-    4. **Enable meta skill** (`use_meta_skill: true`) so the optimizer accumulates strategy memory
+    4. **Enable meta skill** (`optimizer.use_meta_skill: true`) so the optimizer accumulates strategy memory
 ## Next Steps
@@ -37,12 +37,11 @@ scores = evaluate(predictions, ground_truth)
 ### 2. Reflect (Backward Pass)
-The **optimizer** model analyzes failed trajectories and produces **edit patches** — structured suggestions for improving the skill document.
+The **optimizer** model analyzes trajectory minibatches and produces **edit
-
+patches** — structured suggestions for improving the skill document. Failure
-Two modes:
+minibatches are always eligible for analysis; successful trajectories are also
-
+analyzed unless `gradient.failure_only` is enabled. Independent minibatches can
- **Shallow**: Analyze each trajectory independently
+run concurrently according to `gradient.analyst_workers`.
 - **Deep**: Cross-reference multiple failures to find systemic issues
 ```python
 # Analogy: computing gradients
@@ -74,17 +73,31 @@ Selected edits are applied to the skill document, producing a new version.
 ### 6. Gate (Validation)
-The updated skill is evaluated on a **selection split** (analogous to a validation set). The update is only accepted if performance improves.
+The updated skill is evaluated on a **selection split** (analogous to a
 validation set). With the gate enabled, the candidate is accepted only when its
 configured gate score (`hard`, `soft`, or `mixed`) is strictly higher than the
 current skill's score. With `evaluation.use_gate: false`, validation is still
 recorded but candidates are force-accepted.
 ## Epoch Boundary Mechanisms
 ### Slow Update
-At the end of each epoch (starting from epoch 2), the system performs a **longitudinal comparison**: it rolls out both the previous epoch's skill and the current skill on the same samples, categorizes items as improved/regressed/persistent_fail/stable_success, then generates high-level **guidance** that is injected into the skill document. This prevents catastrophic forgetting of earlier improvements.
+At the end of each epoch (starting from epoch 2), the system performs a
 **longitudinal comparison**: it rolls out both the previous epoch's skill and
 the current skill on the same samples, categorizes items as
 improved/regressed/persistent-fail/stable-success, then generates high-level
 **guidance** for the skill document. Depending on
 `optimizer.slow_update_gate_with_selection`, that guidance is either checked on
 the selection split or applied unconditionally. Its purpose is to counter
 cross-epoch forgetting.
 ### Meta Skill
-A **meta-skill memory** accumulates high-level strategy notes across the entire training run. At the end of each epoch, the optimizer reflects on what changed between epochs and produces a compact memory that is provided as additional context during future reflection steps.
+A **meta-skill memory** accumulates high-level strategy notes across the training
 run. Starting at the end of epoch 2, the optimizer compares the previous and
 current epoch, writes a compact memory, and provides the prior epoch's memory as
 additional context during later reflection and update stages.
 ## Next Steps
@@ -18,6 +18,19 @@ hide:
 ---
 ## Two Complementary Workflows
 | Workflow | Package / command | Use it for |
 |---|---|---|
 | **Research engine** | `skillopt`, `skillopt-train`, `skillopt-eval` | Train and evaluate skill documents on explicit benchmark splits. |
 | **SkillOpt-Sleep (preview)** | `skillopt_sleep`, `skillopt-sleep` | Review supported coding-agent sessions and stage proposed memory/skill updates for human adoption. |
 They share the idea of bounded text updates and validation, but they are
 separate entry points with different configs and safety boundaries. Start with
 the [SkillOpt-Sleep overview](sleep/README.md) before using real session data.
 ---
 ## How It Works
 <div class="pipeline-container" markdown>
@@ -106,29 +119,52 @@ SkillOpt brings the familiar deep-learning training paradigm to agentic prompt o
 | **ALFWorld** | Embodied AI | `configs/alfworld/` |
 | **OfficeQA** | Enterprise QA | `configs/officeqa/` |
 | **SearchQA** | Open-domain QA | `configs/searchqa/` |
-| **LiveMathBench** | Math reasoning | `configs/livemathematicianbench/` |
+| **LiveMathematicianBench** | Math reasoning | `configs/livemathematicianbench/` |
-| **SWEBench** | Software Engineering | `configs/swebench/` |
+| **SpreadsheetBench** | Spreadsheet editing | `configs/spreadsheetbench/` |
-| + 5 more | Various | See [docs](guide/first-experiment.md) |
+
 ---
 ## Model Backends
 Optimizer and target roles are configured separately. Chat backends include
 Azure OpenAI (`openai_chat`), the provider-neutral
 `openai_compatible` backend, the Claude Code CLI (`claude_chat`), Qwen, and
 MiniMax. Codex and Claude Code exec harnesses are target-only and require
 adapter support. Despite its name, `claude_chat` launches `claude -p`; it is
 not a direct Anthropic API client.
 If a provider implements OpenAI Chat Completions, begin with the
 [built-in compatible backend](guide/new-backend.md#built-in-the-generic-openai-compatible-backend)
 instead of adding a new integration. See [Configuration](guide/configuration.md)
 for authentication and per-role overrides.
 ---
 ## Quick Example
 ```bash
-# Install
+# Clone and install the research checkout plus the SearchQA data extra
-pip install -e .
+git clone https://github.com/microsoft/SkillOpt.git
 cd SkillOpt
 python -m pip install -e ".[searchqa]"
-# Configure credentials
+# Configure credentials (choose one auth mode in .env)
-export AZURE_OPENAI_ENDPOINT="https://your-resource.openai.azure.com/"
+cp .env.example .env
-export AZURE_OPENAI_API_KEY="your-key"
+set -a; source .env; set +a
-# Train on SearchQA
+# Materialize the runnable split from the checked-in ID manifest
-python scripts/train.py --config configs/searchqa/default.yaml
+python scripts/materialize_searchqa.py
 # Train on SearchQA into a predictable output directory
 python scripts/train.py \
  --config configs/searchqa/default.yaml \
  --out_root outputs/searchqa_quickstart
 # Evaluate best skill
 python scripts/eval_only.py \
  --config configs/searchqa/default.yaml \
-  --skill outputs/best_skill.md
+  --skill outputs/searchqa_quickstart/best_skill.md \
  --split valid_unseen
 ```
 ---
@@ -167,4 +203,13 @@ python scripts/eval_only.py \
    [:octicons-arrow-right-24: WebUI Guide](guide/first-experiment.md#webui)
 -   :material-weather-night:{ .lg .middle } **SkillOpt-Sleep**
    ---
    Test the deployment companion with the no-provider mock path, then review
    its data boundary before selecting a real backend.
    [:octicons-arrow-right-24: Sleep Overview](sleep/README.md)
 </div>
@@ -17,7 +17,9 @@ browse [`skillopt/envs/`](https://github.com/microsoft/SkillOpt/tree/main/skillo
 `skillopt/envs/base.py` — abstract adapter that connects the SkillOpt
 trainer to an environment (benchmark, simulator, REST API, ...).
-Subclasses **must** implement the five abstract methods below.
+Subclasses **must** implement the four abstract methods below. Reflection has a
 shared default implementation and only needs to be overridden for
 environment-specific behavior.
 ```python
 from abc import ABC, abstractmethod
@@ -30,6 +32,10 @@ class EnvAdapter(ABC):
    def setup(self, cfg: dict) -> None: ...
    def get_dataloader(self) -> BaseDataLoader | None: ...
    def requires_ray(self) -> bool: ...                 # default False
    def reflect(self, results: list[dict], skill_content: str,
                out_dir: str, **kwargs) -> list[dict | None]:
        """Delegate to the shared minibatch reflection pipeline."""
        ...
    # ── Abstract methods (subclasses MUST implement) ────────────────────
@@ -53,26 +59,21 @@ class EnvAdapter(ABC):
        It MAY contain env-specific extra keys (parsed into RolloutResult.extras).
        """
    @abstractmethod
    def reflect(self, results: list[dict], skill_content: str,
                out_dir: str, **kwargs) -> list[dict | None]:
        """Turn rollout results into a list of raw patch dicts.
        Each dict (or None to drop the slot) MUST contain:
          - "patch":       {"edits": [...]}     a Patch.to_dict() payload
          - "source_type": "failure" | "success"
        """
    @abstractmethod
    def get_task_types(self) -> list[str]:
        """Distinct task-type strings used for stratified sampling."""
 ```
-The trainer also calls a few default-implemented helpers on every adapter:
+The default `reflect()` delegates to `run_minibatch_reflect` and returns raw
 patch dicts with a `patch` payload plus a `failure` or `success` source type.
 It expects each rollout to persist a non-empty trajectory at
 `<rollout_dir>/predictions/<result-id>/conversation.json`; results without that
 file can be scored but are skipped during reflection.
 The trainer also calls several default-implemented helpers on every adapter:
 `build_reference_text`, `get_reference_metadata`, `attach_reference_context`,
 `select_representative_items`, and `build_env_from_batch`. Read the docstrings
 in `skillopt/envs/base.py` if you need to override any of these — most
-benchmarks don't.
+benchmarks do not.
 ### `BaseDataLoader` / `SplitDataLoader`
@@ -161,16 +162,17 @@ into `RolloutResult.extras`.
 ### `GateResult` / `GateAction`
 `skillopt/evaluation/gate.py` — the validation-gate decision types
-returned each epoch.
+returned for each candidate optimization step, and optionally for a separate
 epoch-end slow-update candidate.
 ---
 ## Registering an environment
 Environments are not registered via decorators or a `BENCHMARK_REGISTRY`
-dict. The trainer keeps a lazy registry inside `scripts/train.py` —
+dict. The training and standalone-evaluation entry points each keep a lazy
-`_ENV_REGISTRY` — populated by `_register_builtins()`. To add a new env
+`_ENV_REGISTRY`, populated by `_register_builtins()` in `scripts/train.py` and
-you append a `try / except ImportError` block there. See
+`scripts/eval_only.py`. Add the environment to both entry points. See
 [Add a New Benchmark](../guide/new-benchmark.md) for the full step-by-step.
 ---
@@ -187,6 +189,7 @@ not via a base class subclass. Supported values (as of this writing):
 | `claude_chat` | ✓ | ✓ |
 | `qwen_chat` | ✓ | ✓ |
 | `minimax_chat` | ✓ | ✓ |
 | `openai_compatible` | ✓ | ✓ |
 | `codex_exec` | — | ✓ |
 | `claude_code_exec` | — | ✓ |
@@ -1,9 +1,17 @@
 # CLI Reference
 > **Version note.** This reference tracks `main`. PyPI 0.2.0 does not yet
 > include the generic research `openai_compatible` backend, Sleep handoff,
 > Sleep support for non-Azure OpenAI-compatible endpoints, or the Sleep
 > `--preferences` flag; use a source install from `main` for those features
 > until the next release.
 ## Training
 ```bash
 python scripts/train.py --config <config.yaml> [overrides...]
 # Installed equivalent:
 skillopt-train --config <config.yaml> [overrides...]
 ```
 ### Arguments
@@ -11,13 +19,15 @@ python scripts/train.py --config <config.yaml> [overrides...]
 | Argument | Description |
 |---|---|
 | `--config` | Path to YAML config file (required) |
-| `key=value` | Override any config parameter |
+| `--cfg-options key=value [...]` | Override structured config parameters |
 ### Examples
 ```bash
 # Basic training
-python scripts/train.py --config configs/searchqa/default.yaml
+python scripts/train.py \
  --config configs/searchqa/default.yaml \
  --out_root outputs/searchqa_run
 # With overrides
 python scripts/train.py \
@@ -34,6 +44,8 @@ python scripts/train.py \
 ```bash
 python scripts/eval_only.py --config <config.yaml> --skill <skill.md>
 # Installed equivalent:
 skillopt-eval --config <config.yaml> --skill <skill.md>
 ```
 ### Arguments
@@ -42,7 +54,8 @@ python scripts/eval_only.py --config <config.yaml> --skill <skill.md>
 |---|---|
 | `--config` | Path to YAML config file (required) |
 | `--skill` | Path to skill document to evaluate (required) |
-| `--split` | Evaluation split: `test` (default), `valid`, `train` |
+| `--split` | `train`, `valid_seen`, `valid_unseen`, or `all` (default) |
 | `--cfg-options` | One or more `section.key=value` overrides |
 ### Examples
@@ -50,15 +63,64 @@ python scripts/eval_only.py --config <config.yaml> --skill <skill.md>
 # Evaluate best skill on test set
 python scripts/eval_only.py \
  --config configs/searchqa/default.yaml \
-  --skill outputs/searchqa/run_001/skills/best_skill.md
+  --skill outputs/searchqa_run/best_skill.md \
  --split valid_unseen
 # Evaluate on validation set
 python scripts/eval_only.py \
  --config configs/searchqa/default.yaml \
-  --skill outputs/searchqa/run_001/skills/best_skill.md \
+  --skill outputs/searchqa_run/best_skill.md \
-  --split valid
+  --split valid_seen
 ```
 `--skill` consumes the artifact produced by training. Unless `--out_root` is
 set for evaluation, `eval_only.py` creates a separate timestamped
 `outputs/eval_<env>_<model>_<timestamp>/` directory and writes
 `eval_summary.json` there; it does not modify the training run directory.
 For the generic OpenAI-compatible research backend, select the role backends
 explicitly:
 ```bash
 python scripts/train.py \
  --config configs/searchqa/default.yaml \
  --cfg-options \
    model.optimizer_backend=openai_compatible \
    model.target_backend=openai_compatible \
    model.optimizer=deepseek-chat \
    model.target=deepseek-chat
 ```
 ## SkillOpt-Sleep
 ```bash
 skillopt-sleep <action> [options]
 # Equivalent from a source checkout:
 python -m skillopt_sleep <action> [options]
 ```
 Actions are `run`, `dry-run`, `status`, `adopt`, `harvest`, `schedule`, and
 `unschedule`. Common options include:
 | Argument | Description |
 |---|---|
 | `--project PATH` | Project to evolve (default: current directory) |
 | `--scope invoked\|all` | Harvest this project or all projects |
 | `--source claude\|codex\|auto` | Transcript source |
 | `--backend mock\|claude\|codex\|copilot\|handoff\|azure_openai` | Replay/optimizer backend |
 | `--model NAME` | Backend-specific model override |
 | `--preferences TEXT` | House rules supplied to reflection |
 | `--lookback-hours N` | Initial transcript lookback; `0` scans all history |
 | `--max-sessions N` / `--max-tasks N` | Bound the harvested workload |
 | `--target-skill-path PATH` | Explicit skill document to stage/adopt |
 | `--tasks-file PATH` | Replay a reviewed task JSON file instead of harvesting |
 | `--edit-budget N` | Maximum bounded edits for the night |
 | `--progress` / `--json` | Progress or machine-readable output |
 | `--auto-adopt` | Apply an accepted staged proposal automatically |
 Backend-specific setup for compatible endpoints is documented in
 [OpenAI-compatible endpoints for SkillOpt-Sleep](../sleep/openai-compatible-endpoints.md).
 ## WebUI
 ```bash
@@ -68,4 +130,8 @@ python -m skillopt_webui.app [--port PORT] [--share]
 | Argument | Default | Description |
 |---|---|---|
 | `--port` | 7860 | Port number |
 | `--host` | `0.0.0.0` | Server bind address |
 | `--share` | false | Create public Gradio link |
 The default host binds every network interface. Use `--host 127.0.0.1` when
 the dashboard should be reachable only from the local machine.
@@ -1,85 +1,174 @@
 # Configuration Reference
-Complete reference for all SkillOpt configuration parameters.
+SkillOpt loads structured YAML, resolves `_base_` inheritance, and flattens
 the result for the trainer. Shipped defaults live in
 `configs/_base_/default.yaml`; benchmark configs override them.
-## Model
+## Model and Backend Selection
 Use explicit optimizer and target backends when the two roles differ or when
 selecting the generic OpenAI-compatible backend.
 | Backend | Optimizer | Target |
 |---|:---:|:---:|
 | `openai_chat` | ✓ | ✓ |
 | `openai_compatible` | ✓ | ✓ |
 | `claude_chat` | ✓ | ✓ |
 | `qwen_chat` | ✓ | ✓ |
 | `minimax_chat` | ✓ | ✓ |
 | `codex_exec` | — | ✓ |
 | `claude_code_exec` | — | ✓ |
 MiniMax currently has one shared deployment. `model.minimax_model` is applied
 when MiniMax is the target; mixed-backend runs cannot independently choose a
 MiniMax optimizer model and a different target model.
 | Parameter | Type | Default | Description |
 |---|---|---|---|
-| `model.backend` | str | `azure_openai` | Backend: `azure_openai` / `openai_chat` / `claude_code_exec` / `qwen` |
+| `model.backend` | str | `azure_openai` | Backward-compatible high-level run label |
-| `model.optimizer` | str | `gpt-5.5` | Optimizer model (for reflection & slow update) |
+| `model.optimizer` | str | `gpt-5.5` | Optimizer deployment/model |
-| `model.target` | str | `gpt-5.5` | Target model (for rollout execution) |
+| `model.target` | str | `gpt-5.5` | Target deployment/model |
-| `model.reasoning_effort` | str | `medium` | Reasoning effort level |
+| `model.optimizer_backend` | str | `openai_chat` | Optimizer client path; chat backends only |
-| `model.optimizer_backend` | str | `openai_chat` | Optimizer backend: `openai_chat` / `claude_chat` / `qwen_chat` / `minimax_chat` |
+| `model.target_backend` | str | `openai_chat` | Target client path; chat or exec backend |
-| `model.target_backend` | str | `openai_chat` | Target backend: chat backends plus execution harnesses |
+| `model.reasoning_effort` | str | `medium` | Shared reasoning effort |
-| `model.qwen_chat_base_url` | str | `http://localhost:8000/v1` | Shared Qwen/vLLM OpenAI-compatible endpoint |
+| `model.rewrite_reasoning_effort` | str | empty | Optional full-rewrite effort override |
-| `model.qwen_chat_enable_thinking` | bool | `false` | Shared Qwen thinking flag |
+| `model.rewrite_max_completion_tokens` | int | `64000` | Full-rewrite output cap |
-| `model.optimizer_qwen_chat_base_url` | str | — | Optimizer-specific Qwen/vLLM endpoint; overrides shared `qwen_chat_base_url` |
+
-| `model.target_qwen_chat_base_url` | str | — | Target-specific Qwen/vLLM endpoint; overrides shared `qwen_chat_base_url` |
+### Azure/OpenAI `openai_chat`
 | Parameter | Default | Description |
 |---|---|---|
 | `model.azure_openai_endpoint` | empty | Shared Azure resource URL or compatibility-mode base URL |
 | `model.azure_openai_api_version` | `2024-12-01-preview` | Azure API version |
 | `model.azure_openai_api_key` | empty | Key for `api_key` or compatibility auth |
 | `model.azure_openai_auth_mode` | empty | Config value; empty falls back to env, whose default is `azure_cli` |
 | `model.azure_openai_ad_scope` | Azure Cognitive Services scope | AAD token scope |
 | `model.azure_openai_managed_identity_client_id` | empty | Optional user-assigned identity client ID |
 Every shared key also has an `optimizer_azure_openai_*` and
 `target_azure_openai_*` form.
 ### Claude `claude_chat`
 `claude_chat` launches an installed, authenticated Claude Code CLI with
 `claude -p`; it does not instantiate an Anthropic API client. The executable
 defaults to `claude` and can be overridden with `CLAUDE_CLI_BIN`.
 `ANTHROPIC_API_KEY` is one authentication option understood by the CLI.
 ### Qwen, MiniMax, and Exec Backends
 | Parameter family | Description |
 |---|---|
 | `model.qwen_chat_*` | Shared `base_url`, `api_key`, `temperature`, `timeout_seconds`, `max_tokens`, and `enable_thinking` |
 | `model.optimizer_qwen_chat_*` / `model.target_qwen_chat_*` | Per-role Qwen overrides |
 | `model.minimax_*` | MiniMax `base_url`, `api_key`, shared `minimax_model`, `temperature`, `max_tokens`, and `enable_thinking`; `minimax_model` applies when MiniMax is the target |
 | `model.codex_exec_*` | Codex path, sandbox, profile, SDK mode, reasoning, network/search, and approval policy |
 | `model.claude_code_exec_*` | Claude path, profile, SDK mode, effort, and thinking-token cap |
 ## Training (`train`)
-| Parameter | Type | Default | DL Analogy | Description |
+| Parameter | Type | Default | Description |
-|---|---|---|---|---|
+|---|---|---|---|
-| `train.num_epochs` | int | 4 | Epochs | Number of training epochs |
+| `train.num_epochs` | int | `4` | Training epochs |
-| `train.batch_size` | int | 40 | Batch size | Tasks sampled per step |
+| `train.train_size` | int | `0` | `0` derives the size from the dataset split |
-| `train.accumulation` | int | 1 | Gradient accumulation | Accumulation rounds per step |
+| `train.steps_per_epoch` | int | derived | Runtime field recomputed from train size, batch size, and accumulation; configured values are overwritten |
-| `train.seed` | int | 42 | Random seed | Reproducibility seed |
+| `train.batch_size` | int | `40` | Tasks sampled per step |
 | `train.accumulation` | int | `1` | Accumulation rounds per step |
 | `train.seed` | int | `42` | Random seed |
 ## Gradient / Reflection (`gradient`)
 | Parameter | Type | Default | Description |
 |---|---|---|---|
-| `gradient.minibatch_size` | int | 8 | Reflect minibatch size |
+| `gradient.minibatch_size` | int | `8` | Reflect minibatch size |
-| `gradient.merge_batch_size` | int | 8 | Patch merge batch size |
+| `gradient.merge_batch_size` | int | `8` | Patch merge batch size |
-| `gradient.analyst_workers` | int | 16 | Parallel reflection workers |
+| `gradient.analyst_workers` | int | `16` | Parallel reflection workers |
-| `gradient.max_analyst_rounds` | int | 3 | Max rounds of analyst reflection |
+| `gradient.max_analyst_rounds` | int | `3` | Maximum analyst rounds |
-| `gradient.failure_only` | bool | `false` | Only reflect on failures |
+| `gradient.failure_only` | bool | `false` | Reflect only on failures |
 ## Optimizer (`optimizer`)
-| Parameter | Type | Default | DL Analogy | Description |
+| Parameter | Type | Default | Description |
-|---|---|---|---|---|
+|---|---|---|---|
-| `optimizer.learning_rate` | int | 4 | Learning rate | Max edit patches per step (edit budget) |
+| `optimizer.learning_rate` | int | `4` | Maximum edit patches per step |
-| `optimizer.min_learning_rate` | int | 2 | Min LR | Min edits for decay schedulers |
+| `optimizer.min_learning_rate` | int | `2` | Floor for decaying schedules |
-| `optimizer.lr_scheduler` | str | `cosine` | LR schedule | `constant` / `linear` / `cosine` / `autonomous` |
+| `optimizer.lr_scheduler` | str | `cosine` | `constant`, `linear`, `cosine`, or `autonomous` |
-| `optimizer.skill_update_mode` | str | `patch` | — | `patch` / `rewrite_from_suggestions` / `full_rewrite_minibatch` |
+| `optimizer.lr_control_mode` | str | `fixed` | `fixed`, `autonomous`, or `none` |
-| `optimizer.use_slow_update` | bool | `true` | Momentum | Epoch-boundary longitudinal comparison & guidance |
+| `optimizer.skill_update_mode` | str | `patch` | `patch`, `rewrite_from_suggestions`, or `full_rewrite_minibatch` |
-| `optimizer.slow_update_samples` | int | 20 | — | Samples for slow update evaluation |
+| `optimizer.use_slow_update` | bool | `true` | Epoch-boundary longitudinal update |
-| `optimizer.use_meta_skill` | bool | `true` | Meta-learning | Cross-epoch optimizer-side strategy memory |
+| `optimizer.slow_update_samples` | int | `20` | Longitudinal evaluation samples |
-| `optimizer.longitudinal_pair_policy` | str | `mixed` | — | `mixed` / `changed` / `unchanged` |
+| `optimizer.slow_update_gate_with_selection` | bool | `false` | Gate slow-update guidance on the selection split |
 | `optimizer.longitudinal_pair_policy` | str | `mixed` | `mixed`, `changed`, or `unchanged` |
 | `optimizer.use_meta_skill` | bool | `true` | Cross-epoch optimizer memory |
 | `optimizer.use_skill_aware_reflection` | bool | `false` | Enable skill-defect vs execution-lapse routing |
 | `optimizer.skill_aware_appendix_source` | str | `both` | `both` or `failure_only` |
 | `optimizer.skill_aware_consolidate_threshold` | int | `0` | Appendix compaction threshold; `0` disables it |
 ## Evaluation (`evaluation`)
 | Parameter | Type | Default | Description |
 |---|---|---|---|
-| `evaluation.use_gate` | bool | `true` | Enable validation gating (accept/reject updates) |
+| `evaluation.use_gate` | bool | `true` | Accept only improvements when enabled; `false` records validation but force-accepts each candidate |
-| `evaluation.eval_test` | bool | `true` | Run test evaluation after training |
+| `evaluation.gate_metric` | str | `hard` | `hard`, `soft`, or `mixed` |
 | `evaluation.gate_mixed_weight` | float | `0.5` | Soft-score weight for `mixed` |
 | `evaluation.use_semantic_density` | bool | `false` | Add the optional instruction-density bonus |
 | `evaluation.semantic_density_weight` | float | `0.05` | Density bonus weight |
 | `evaluation.leading_words` | list/str | built in | Optional custom high-influence words |
 | `evaluation.sel_env_num` | int | `0` | Selection size; `0` uses the full split |
 | `evaluation.test_env_num` | int | `0` | Test size; `0` uses the full split |
 | `evaluation.eval_test` | bool | `true` | Run final test evaluation |
 ## Environment (`env`)
 | Parameter | Type | Default | Description |
 |---|---|---|---|
-| `env.name` | str | — | Benchmark name (e.g., `searchqa`, `docvqa`) |
+| `env.name` | str | empty | Benchmark name |
-| `env.data_path` | str | — | Path to dataset |
+| `env.skill_init` | str | empty | Initial skill document |
 | `env.skill_init` | str | — | Path to initial seed skill (optional) |
 | `env.split_mode` | str | `ratio` | `ratio` or `split_dir` |
-| `env.split_ratio` | str | `2:1:7` | Train:val:test ratio |
+| `env.split_ratio` | str | benchmark/default | Train:validation:test ratio |
-| `env.exec_timeout` | int | 120 | Per-task timeout in seconds |
+| `env.split_seed` | int | `42` | Deterministic split seed |
-| `env.out_root` | str | — | Output directory |
+| `env.split_dir` | str | empty | Materialized train/val/test directory |
 | `env.data_path` | str | empty | Raw data path for ratio mode |
 | `env.split_output_dir` | str | empty | Optional materialized split output |
 | `env.exec_timeout` | int | `120` | Per-task timeout in seconds |
 | `env.out_root` | str | generated by the train/eval CLIs | Output directory |
-## Azure OpenAI Credentials
+Benchmark-specific `env` keys are passed through to the adapter.
 ## Credential Environment Variables
 ### Azure-family backend
 | Variable | Description |
 |---|---|
-| `AZURE_OPENAI_ENDPOINT` / `model.azure_openai_endpoint` | Azure resource endpoint |
+| `AZURE_OPENAI_ENDPOINT` | Shared Azure endpoint or compatibility base URL |
-| `AZURE_OPENAI_API_KEY` / `model.azure_openai_api_key` | Azure API key |
+| `AZURE_OPENAI_API_VERSION` | Azure API version |
-| `OPENAI_API_KEY` | OpenAI API key (for `openai_chat` backend) |
+| `AZURE_OPENAI_AUTH_MODE` | `api_key`, `azure_cli`, `managed_identity`, or `openai_compatible` |
-| `ANTHROPIC_API_KEY` | Anthropic API key (for `claude_code_exec` backend) |
+| `AZURE_OPENAI_API_KEY` | Key for `api_key` or `openai_compatible` mode |
-| `QWEN_CHAT_BASE_URL` | Shared local vLLM endpoint for `qwen_chat` |
+| `AZURE_OPENAI_AD_SCOPE` | Optional AAD scope |
-| `QWEN_CHAT_MODEL` | Shared served model name for `qwen_chat` |
+| `AZURE_OPENAI_MANAGED_IDENTITY_CLIENT_ID` | Optional managed-identity client ID |
-| `QWEN_CHAT_API_KEY` | Optional API key for the shared Qwen endpoint |
+
-| `OPTIMIZER_QWEN_CHAT_BASE_URL` | Optimizer-specific local vLLM endpoint |
+Use `OPTIMIZER_AZURE_OPENAI_*` and `TARGET_AZURE_OPENAI_*` for role-specific
-| `OPTIMIZER_QWEN_CHAT_MODEL` | Optimizer-specific served model name |
+overrides.
-| `TARGET_QWEN_CHAT_BASE_URL` | Target-specific local vLLM endpoint |
+
-| `TARGET_QWEN_CHAT_MODEL` | Target-specific served model name |
+### Generic OpenAI-compatible backend
 | Variable suffix | Shared / per-role forms |
 |---|---|
 | `BASE_URL` | `OPENAI_COMPATIBLE_BASE_URL`, `OPTIMIZER_OPENAI_COMPATIBLE_BASE_URL`, `TARGET_OPENAI_COMPATIBLE_BASE_URL` |
 | `API_KEY` | Corresponding shared/optimizer/target `*_API_KEY` names |
 | `MODEL` | Corresponding shared/optimizer/target `*_MODEL` names |
 | `TEMPERATURE` | Corresponding shared/optimizer/target `*_TEMPERATURE` names |
 | `MAX_TOKENS` | Corresponding shared/optimizer/target `*_MAX_TOKENS` names |
 | `TIMEOUT_SECONDS` | Corresponding shared/optimizer/target `*_TIMEOUT_SECONDS` names |
 The train/eval entry points set deployments from YAML `model.optimizer` and
 `model.target` after backend initialization. For selected OpenAI-compatible or
 Qwen roles, those values override the corresponding `*_MODEL` environment
 variables; the environment model names mainly seed direct library use.
 Other backend families use the authenticated Claude CLI (`CLAUDE_CLI_BIN`;
 optionally `ANTHROPIC_API_KEY`), `QWEN_CHAT_*`, and `MINIMAX_*`.
 SkillOpt-Sleep's compatible endpoint uses `AZURE_OPENAI_*`, not the research
 backend's `OPENAI_COMPATIBLE_*`; see
 [the Sleep endpoint guide](../sleep/openai-compatible-endpoints.md).
@@ -4,11 +4,11 @@
 local coding agent a nightly **sleep cycle** that reviews your past sessions, replays
 your recurring tasks on your own API budget, and consolidates what it learns into
 **validated** long-term memory and skills — behind a held-out gate, staged for your
-review. The agent gets better the more you use it, with **no weight training** and
+review. It requires **no weight training** and adds no separate optimization loop to
-**zero inference-time overhead**.
+normal agent requests.
 > **Preview.** This is an early preview we are actively iterating on; interfaces and
-> defaults may change. The engine lives in the top-level [`skillopt_sleep/`](../../skillopt_sleep)
+> defaults may change. The engine lives in the top-level [`skillopt_sleep/`](https://github.com/microsoft/SkillOpt/tree/main/skillopt_sleep)
 > package with **zero dependency** on the paper's `skillopt/` code (the validation gate
 > is vendored).
@@ -26,29 +26,49 @@ It synthesizes **SkillOpt** (validation-gated bounded text edits), **Claude Drea
 (offline consolidation; review-then-adopt), and the **agent-sleep** idea (short-term
 experience → long-term competence).
 > **Data boundary.** Harvesting is local and read-only. The `mock` backend makes no
 > provider calls. A real backend, however, sends truncated excerpts from harvested
 > sessions and derived tasks to the provider you select for mining, replay, judging,
 > and reflection. Outbound prompts are not currently guaranteed to be secret-free;
 > review your transcript source and provider policy before running on sensitive
 > projects. For a reviewable workflow, harvest to a task file, inspect/redact it, mark
 > it `"reviewed": true`, and then replay that file with the real backend.
 ## How to use it
 ### Quickest path: the `skillopt-sleep` CLI (pip)
 ```bash
 pip install skillopt        # installs the engine + the `skillopt-sleep` command
-skillopt-sleep dry-run      # harvest + mine + replay, report only (changes nothing)
+skillopt-sleep dry-run      # harvest + mine + replay, report only; stages nothing
 skillopt-sleep run          # a full nightly cycle; the proposal is staged for review
 skillopt-sleep status       # show state + the latest staged proposal
 skillopt-sleep adopt        # apply the latest staged proposal
 skillopt-sleep schedule     # install a nightly cron entry for this project
 ```
-The per-agent plugin shells below (Claude Code / Codex / Copilot) still come from the
+> **Version note.** This page tracks `main`. PyPI 0.2.0 provides the base
-repo; the CLI above is the standalone, pip-only way to run a cycle.
+> commands above. Sleep handoff, non-Azure OpenAI-compatible endpoints, and
 > `--preferences` landed later and require a source install from `main` until
 > the next release.
-One engine, thin per-agent shells (see [`plugins/`](../../plugins)):
+The per-agent integrations below still come from the repo; the CLI above is the
 standalone, pip-only way to run a cycle. Claude Code, Codex, Copilot, and Devin wrap
 the shared engine. OpenClaw is a separate reference adaptation and has its own setup.
 One engine, thin per-agent shells (see [`plugins/`](https://github.com/microsoft/SkillOpt/tree/main/plugins)):
 | Platform | Folder | Install |
 |---|---|---|
-| **Claude Code** | [`plugins/claude-code`](../../plugins/claude-code) | `/plugin marketplace add ./plugins/claude-code` → `/skillopt-sleep` |
+| **Claude Code** | [`plugins/claude-code`](https://github.com/microsoft/SkillOpt/tree/main/plugins/claude-code) | `/plugin marketplace add ./plugins/claude-code` → `/skillopt-sleep` |
-| **Codex** | [`plugins/codex`](../../plugins/codex) | `bash plugins/codex/install.sh` → `skillopt-sleep` skill |
+| **Codex** | [`plugins/codex`](https://github.com/microsoft/SkillOpt/tree/main/plugins/codex) | `bash plugins/codex/install.sh` → `skillopt-sleep` skill |
-| **Copilot** | [`plugins/copilot`](../../plugins/copilot) | register `plugins/copilot/mcp_server.py` as an MCP server |
+| **Copilot** | [`plugins/copilot`](https://github.com/microsoft/SkillOpt/tree/main/plugins/copilot) | register `plugins/copilot/mcp_server.py` as an MCP server |
 | **Devin** | [`plugins/devin`](https://github.com/microsoft/SkillOpt/tree/main/plugins/devin) | register `plugins/devin/mcp_server.py` as an MCP server |
 | **OpenClaw** | [`plugins/openclaw`](https://github.com/microsoft/SkillOpt/tree/main/plugins/openclaw) | adapt the reference wrapper and paths for your installation |
 To use DeepSeek, vLLM, Ollama, or another Chat Completions server, see
 **[OpenAI-compatible endpoints](openai-compatible-endpoints.md)**. That guide also
 documents the separate HTTPS-only boundary for Azure managed-identity credentials.
 Deterministic proof (no API key):
 `python -m skillopt_sleep.experiments.run_experiment --persona researcher --assert-improves`.
@@ -71,11 +91,12 @@ correctness signal; the validation gate still governs what ships.
 > scaling, and the dream-diversity ablation — are in
 > [`docs/sleep/RESULTS.md`](RESULTS.md).** The highlights:
-**Protocol (identical for every row below).** 5 nights × 10 new real "today" tasks
+**Controlled experiment recipe (not the shipping CLI defaults).** 5 nights × 10 new
-per night; the full held-out **test** split is scored before night 1 (baseline) and
+real "today" tasks per night; the full held-out **test** split is scored before night
-after night 5 (after); optimizer = GPT-5.5; single seed (42); run through the exact
+1 (baseline) and after night 5 (after); optimizer = GPT-5.5; single seed (42). The
-shipped engine (`skillopt_sleep.dream.dream_consolidate`). Numbers are absolute
+experiments use the shipped consolidation and gate components, while the nightly CLI
-held-out accuracy; **Δ** = `after − baseline` in percentage points.
+and benchmark harnesses remain separate entry points. Numbers are absolute held-out
 accuracy; **Δ** = `after − baseline` in percentage points.
 **(a) End-to-end on real agents — [gbrain-evals](https://github.com/garrytan/gbrain-evals) `skillopt-v1`.**
 Deficient seed skills go **0.00 → 1.00** on the held-out set with **both Claude Code
@@ -106,5 +127,6 @@ gate keeps the worst case bounded; keep it **on** by default.
 ## Learn more
-Full reference (pipeline, the three plugins, the experience-replay knobs) is in the
+See the [SkillOpt documentation index](../index.md), the
-**[Documentation & Reproduction Guide](https://microsoft.github.io/SkillOpt/docs/guideline.html#sleep)**.
+[CLI reference](../reference/cli.md), and the integration-specific READMEs under
 [`plugins/`](https://github.com/microsoft/SkillOpt/tree/main/plugins).
@@ -2,8 +2,9 @@
 This is the evidence behind SkillOpt-Sleep: does a nightly, offline sleep cycle
 actually make a *deployed* agent better, and is it safe to run unattended? We
-answer with a controlled deployment-scale study — the same protocol the plugin
+answer with a controlled deployment-scale study built from the same shipped
-runs in production, scored on full held-out test sets.
+consolidation and gate components. Its multi-night benchmark recipe is an
 experiment configuration, not the default configuration of the nightly CLI.
 ## Setup
@@ -11,9 +12,10 @@ runs in production, scored on full held-out test sets.
 **10 new real "today" tasks**; the skill carries over and is refined night to
 night. The full held-out **test** split is scored before night 1 (*baseline*) and
 after night 5 (*after*); **Δ = after − baseline** in percentage points. Optimizer
-model = **GPT-5.5**; single seed (42); every number is produced by the exact
+model = **GPT-5.5**; single seed (42). The measurements use the shipped replay,
-shipped engine `skillopt_sleep.dream.dream_consolidate` (the experiment harness and
+consolidation, and gate implementations. The nightly CLI and the checked-in
-the plugin cycle call the same function).
+benchmark convenience harnesses are separate entry points and do not all call one
 shared wrapper function.
 **Benchmarks** (real evaluators, not format heuristics):
@@ -106,27 +108,31 @@ Replay-policy ablation (SearchQA, GPT-5.5):
 | Replay policy | Gate-free Δ | Gated Δ |
 |---|---|---|
 | none (tonight's tasks only) | +3.9 | +2.0 |
-| **recall k=10 (shipped default-able)** | +5.1 | +4.4 |
+| **recall k=10 (opt-in experiment)** | +5.1 | +4.4 |
 | cumulative (full history) | +4.8 | +6.0 |
 Recall captures most of cumulative's benefit at a fraction of the per-night cost.
 ---
-## 4. Default hyperparameters are the sweet spot
+## 4. Sensitivity around the experiment recipe
 We swept `dream_factor`, `rollouts`, `per_night`, and `nights` on the nano cell
-(SearchQA, gated) to verify the shipped defaults are well-tuned:
+(SearchQA, gated) around the study recipe: `dream_factor=2`, `rollouts=5`,
 `per_night=10`, and `nights=5`. These are **experiment values**, not the shipping
 defaults (`dream_factor=0`, `dream_rollouts=1`, and `recall_k=0`):
-| Variant | Δ | vs default (+11.9) |
+| Variant | Δ | vs experiment baseline (+11.9) |
 |---|---|---|
-| dream_factor=4 (default 2) | +8.8 | −3.1 |
+| dream_factor=4 (baseline 2) | +8.8 | −3.1 |
-| rollouts=10 (default 5) | +9.5 | −2.4 |
+| rollouts=10 (baseline 5) | +9.5 | −2.4 |
-| per_night=15 (default 10) | +2.7 | −9.2 |
+| per_night=15 (baseline 10) | +2.7 | −9.2 |
-| nights=8 (default 5) | +9.5 | −2.4 |
+| nights=8 (baseline 5) | +9.5 | −2.4 |
-Every direction away from the default hurts. This means users get the best result
+Every tested direction away from that baseline reduced the measured gain in this
-**out of the box** without tuning — the recipe is robust by design.
+cell. The result supports that particular study recipe; it does not establish a
 universal optimum. Shipping stays conservative, and users must opt in to additional
 dream rollouts or recall after considering task quality and provider cost.
 ---
@@ -143,7 +149,7 @@ gains in Sections 1–2. Measured across an 18-cell deployment sweep (3 benchmar
 |---|---|---|---|---|
 | single-sample reflection (degraded) | −2.66 | **−52.8** | 7 / 18 | 5 / 18 |
 | diverse rollouts (K=5), no recall | +0.24 | −4.0 | 6 / 18 | 7 / 18 |
-| **diverse rollouts + recall (shipped)** | **+0.53** | **−2.4** | 7 / 18 | 7 / 18 |
+| **diverse rollouts + recall (experiment recipe)** | **+0.53** | **−2.4** | 7 / 18 | 7 / 18 |
 The catastrophic −52.8 is removed **at its source** by diverse rollouts: the same
 gate-free nano-SearchQA cell goes 0.554 → **0.586 (+2.7)** with no gate at all once
@@ -182,4 +188,4 @@ cross-verify each other's consolidated skills.
 ---
 Back to the module overview: [`docs/sleep/README.md`](README.md) ·
-full reference: [Documentation & Reproduction Guide](https://microsoft.github.io/SkillOpt/docs/guideline.html#sleep).
+documentation index: [SkillOpt documentation](../index.md).
@@ -1,11 +1,16 @@
 # OpenAI-compatible endpoints for SkillOpt-Sleep (DeepSeek, local vLLM, …)
-This document describes an enhancement to the `azure_openai` backend in
+This document describes the `azure_openai` backend in
-`skillopt_sleep/backend.py` that lets SkillOpt-Sleep drive **any
+`skillopt_sleep/backend.py`, which can drive servers that implement the expected
-OpenAI-compatible chat-completions endpoint** — for example DeepSeek's hosted
+OpenAI-compatible Chat Completions request shape — for example DeepSeek's hosted
 API or a self-hosted vLLM/Ollama server — in addition to native Azure OpenAI
-deployments. It also documents a concrete end-to-end integration: running the
+deployments. The included runner is a sanitized unattended-launch example that
-nightly sleep cycle inside the Antigravity IDE against DeepSeek.
+was originally used alongside Antigravity; it is not an Antigravity transcript
 integration.
 > **Version requirement.** This capability landed after v0.2.0. Until the next
 > release, install SkillOpt from the latest `main`; the current PyPI 0.2.0
 > package does not provide this compatible-endpoint path.
 ## What changed
@@ -32,15 +37,17 @@ is unchanged:
   every rollout `0.0` with no diagnostic.)
 4. **Managed-identity credential guard.** The managed-identity path attaches an
-   Azure AD bearer token to every request. If a custom endpoint outside
+   Azure AD bearer token to every request. It therefore accepts only an **HTTPS**
-   `*.openai.azure.com` / `*.cognitiveservices.azure.com` is configured without
+   endpoint whose hostname ends in `*.openai.azure.com` or
-   explicit compat auth, the backend now raises a clear `ValueError` instead of
+   `*.cognitiveservices.azure.com`. An HTTP endpoint — even one with an
-   sending Azure credentials to an arbitrary host.
+   Azure-looking hostname — and any host outside those suffixes are rejected
   before a credential-bearing client is created.
 5. **Provider-neutral request shape.** In compat mode the backend sends only the
   standard OpenAI-compatible contract (`model`, `messages`, `max_tokens`).
   Provider-specific request fields are **opt-in** via environment variables
-   (below) — nothing is inferred from model-name substrings.
+   (below) and are attached only in compat mode — nothing is inferred from
   model-name substrings, and the native Azure request remains unchanged.
 6. **Reliable error state.** `_call()` records the last exception in
   `self.last_call_error` (surfaced in `diagnostics.json`), clears it when a
@@ -57,16 +64,34 @@ sleep cycle):
 | Variable | Meaning |
 |---|---|
 | `AZURE_OPENAI_AUTH_MODE` | `openai_compatible` (or `compat`/`openai`) selects the plain OpenAI client. Unset/other = Azure managed identity (default). |
-| `AZURE_OPENAI_ENDPOINT` | Base URL of the server, e.g. `https://api.deepseek.com`. |
+| `AZURE_OPENAI_ENDPOINT` | Base URL of the server, e.g. `https://api.deepseek.com`. Azure managed identity requires HTTPS plus an approved Azure hostname. |
-| `AZURE_OPENAI_API_KEY` | API key sent by the compat client. |
+| `AZURE_OPENAI_API_KEY` | API key sent by the compat client to the configured base URL. |
 | `SKILLOPT_SLEEP_COMPAT_MAX_TOKENS` | Optional int (default `8192`): `max_tokens` sent in compat mode. |
-| `SKILLOPT_SLEEP_CHAT_EXTRA_BODY` | Optional JSON object passed as `extra_body` for provider-specific fields. |
+| `SKILLOPT_SLEEP_CHAT_EXTRA_BODY` | Optional JSON object passed as `extra_body` for provider-specific fields in compat mode only. It is ignored in native Azure mode. |
 ## Data and transport boundaries
 - Harvesting reads local transcripts without modifying them, and the `mock`
  backend makes no provider calls. A real backend sends **truncated transcript
  excerpts and derived task content** to the selected provider for mining,
  replay, judging, and reflection.
 - Outbound prompts are not currently guaranteed to be free of secrets. Review
  the provider's data policy and avoid a third-party endpoint for sensitive
  transcripts unless you have first inspected and redacted the task material.
  One reviewable path is `skillopt-sleep harvest --output tasks.json`, followed
  by a reviewed `--tasks-file` run.
 - Use HTTPS for every remote compatible provider. Plain HTTP is appropriate only
  for an explicitly trusted loopback development server such as
  `http://127.0.0.1:8000/v1`; the compat client sends its API key to the configured
  URL.
 - Azure managed-identity credentials have the stricter invariant described
  above: HTTPS **and** an approved Azure hostname are both mandatory.
 ## How to use it
 ```bash
 export AZURE_OPENAI_AUTH_MODE=openai_compatible
-export AZURE_OPENAI_ENDPOINT=https://api.deepseek.com   # no /v1, no trailing path
+export AZURE_OPENAI_ENDPOINT=https://api.deepseek.com   # DeepSeek base URL
 export AZURE_OPENAI_API_KEY=sk-...                       # your provider key
 # DeepSeek reasoning models: enable the thinking channel (opt-in, not inferred)
@@ -79,37 +104,49 @@ skillopt-sleep run \
  --project /path/to/your/project
 ```
-The same pattern works for any OpenAI-compatible server — point
+The same pattern works for a server that implements this Chat Completions
-`AZURE_OPENAI_ENDPOINT` at it, set a matching `--model`, and omit
+contract: point `AZURE_OPENAI_ENDPOINT` at the provider-specific base URL, set a
-`SKILLOPT_SLEEP_CHAT_EXTRA_BODY` unless your provider needs extra request
+matching `--model`, and omit `SKILLOPT_SLEEP_CHAT_EXTRA_BODY` unless the provider
-fields.
+needs extra request fields. Self-hosted vLLM and Ollama commonly use a `/v1` base
 path, for example `http://127.0.0.1:8000/v1` or
 `http://127.0.0.1:11434/v1`.
-## End-to-end integration: Antigravity + DeepSeek
+`--project` selects the project/transcript scope and the project `CLAUDE.md`; it
 does **not** by itself select an arbitrary project `SKILL.md`. Pass
 `--target-skill-path path/to/SKILL.md` when a specific skill is the optimization
 target. Without that flag, SkillOpt-Sleep uses its configured managed skill.
-The [`examples/`](examples/) directory contains a sanitized reference of how this
+## Unattended runner example (originally used with Antigravity)
-was wired into the [Antigravity](https://antigravity.google/) agent IDE so the
+
-sleep cycle runs unattended:
+The [`examples/`](https://github.com/microsoft/SkillOpt/tree/main/docs/sleep/examples) directory contains a sanitized reference for running
 the compatible backend unattended:
 - **`examples/runner.py`** — a thin launcher that loads a provider key from an
  `.env` file, exports the variables above, invokes `skillopt-sleep run` with
  the DeepSeek backend, and **exits with the child's return code** so
-  supervisors see failures as failures. It also implements a `session-end` hook
+  supervisors see failures as failures. Its `session-end` action writes a small
-  that appends task-outcome metadata to a rollout-evidence log (wired to
+  local rollout-evidence event as an example hook target.
  Antigravity's `Stop` hook) so future nights have richer sessions to mine.
 - **`examples/watchdog.py`** — a minimal supervisor loop that invokes the runner
  on a fixed interval (e.g. every 4 hours) and logs non-zero exits as failures.
  On Windows this is registered as a Scheduled Task so it survives logout; on
  Linux/macOS a `systemd` timer or cron entry serves the same role.
-### Verified result
+The current engine does **not** read `brain/rollout-evidence.jsonl`, and it does
 not harvest Antigravity transcripts. That hook output is illustrative metadata,
 not additional training evidence. A real run must use a supported Claude
 Code/Codex transcript source or a reviewed task file converted by the operator.
-On a Windows 11 host, driving the cycle against `deepseek-v4-pro` in
+### Contributor-reported validation
-`openai_compatible` mode:
+
 The contributor reported the following results from a private Windows 11 setup
 driving the cycle against `deepseek-v4-pro` in `openai_compatible` mode. They are
 useful integration evidence, but the private session set is not a reproducible
 benchmark bundled with this repository:
 - A direct backend smoke test returns a live completion (no `404`,
  `last_call_error` empty, client type `OpenAI`).
- A full nightly cycle mined tasks from real IDE sessions and the held-out
+- A full nightly cycle using the configured session source moved the held-out
-  validation gate moved from `0.250 → 1.000`, **accepting** a DeepSeek-authored
+  validation gate from `0.250 → 1.000`, **accepting** a DeepSeek-authored
  skill edit (`accept_new_best`). `diagnostics.json` for that night reports
  `"backend": "azure_openai"` with a non-empty token count and an empty
  `call_error` — i.e. a genuine optimization night, versus the prior all-`0.0`
@@ -124,13 +161,13 @@ Deterministic no-network coverage for the new behavior lives in
 endpoint/auth guard, request kwargs, retry error-state, empty-response
 diagnostics, and runner exit-code propagation).
-## A note on Gemini (optional, unverified fallback)
+## Unsupported Gemini proxy branch in the example
-`examples/runner.py` also contains a fallback branch that, when only a Gemini key
+`examples/runner.py` still contains an illustrative branch that routes the
-is present, routes the **`claude` CLI backend** through a local
+**`claude` CLI backend** through a loopback Anthropic-compatible proxy such as
-Anthropic-compatible proxy (e.g. [LiteLLM](https://github.com/BerriAI/litellm) on
+[LiteLLM](https://github.com/BerriAI/litellm). It is not a native Gemini backend,
-`http://127.0.0.1:4000`) by setting `ANTHROPIC_BASE_URL`/`ANTHROPIC_API_KEY`.
+has no validated model mapping in this example, and is not part of the supported
-There is **no native Gemini backend** in SkillOpt, and this proxy path was not
+path documented here. The sample currently enters that branch whenever no
-independently validated in this work — it is included only as a configuration
+DeepSeek key is found, so a production adaptation should remove it or replace it
-example. The verified, supported path in this document is DeepSeek via
+with an explicit opt-in, a separately configured model, and a trusted isolated
-`openai_compatible` mode. Treat the Gemini branch as illustrative, not tested.
+loopback proxy. Do not treat this branch as tested Gemini support.
@@ -1,5 +1,13 @@
 # SkillOpt Sleep — Claude Code self-evolving plugin (design)
 > **Historical design proposal.** This document records the June 2026 design
 > target and includes planned controls that are not part of the current nightly
 > CLI. It is not an installation or configuration reference. For implemented
 > behavior, flags, defaults, and data boundaries, use
 > [`docs/sleep/README.md`](../../sleep/README.md),
 > [`docs/reference/cli.md`](../../reference/cli.md), and
 > [`plugins/README.md`](https://github.com/microsoft/SkillOpt/blob/main/plugins/README.md).
 **Status:** approved-for-build (autonomous offline session, 2026-06-07)
 **Author:** generated for Yifan Yang, executed autonomously while user is asleep
 **Branch:** `feat/claude-code-sleep-plugin` (worktree `my_repo/SkillOpt-sleep`)
@@ -234,4 +242,3 @@ This session targets **Phase 0 + Phase 1 fully**, **Phase 2 scaffolded**, and th
 3. **Real-API demo:** want me to spend live `ANTHROPIC_API_KEY` budget on the persona demo, or keep everything mock until you say go?
 4. **Skill target:** evolve a *new* dedicated `skillopt-sleep`-managed skill, or also edit your existing hand-written skills in `~/.claude/skills`?
 5. **Paper:** should this become a section/figure in the SkillOpt arXiv (Dream+Sleep framing as "deployment-time continual skill optimization")?
 ```
@@ -1778,6 +1778,7 @@
      <a href="#evolution">Evolution</a>
      <a href="#transfer">Transfer</a>
      <a href="#citation">Citation</a>
      <a href="https://github.com/microsoft/SkillOpt/blob/main/docs/index.md" target="_blank" rel="noopener">Docs</a>
      <a href="https://github.com/microsoft/SkillOpt" target="_blank" rel="noopener">Code</a>
    </nav>
  </header>
@@ -1915,7 +1916,7 @@
          <h3>A skill is external state for an agent.</h3>
          <p>
            Instead of fine-tuning a model or hand-maintaining prompts, SkillOpt runs
-            the frozen agent on scored batches, asks a separate optimizer model to
+            the frozen agent on scored batches, asks an optimizer model to
            propose structured edits, and accepts a candidate only when validation
            performance improves.
          </p>
@@ -2427,7 +2428,7 @@
    <footer class="footer">
      <span>SkillOpt: Executive Strategy for Self-Evolving Agent Skills</span>
-      <span><a href="https://github.com/microsoft/SkillOpt" target="_blank" rel="noopener">Code</a> / <a href="#citation">Citation</a></span>
+      <span><a href="https://github.com/microsoft/SkillOpt/blob/main/docs/index.md" target="_blank" rel="noopener">Docs</a> / <a href="https://github.com/microsoft/SkillOpt" target="_blank" rel="noopener">Code</a> / <a href="#citation">Citation</a></span>
    </footer>
  </main>
  <script>
@@ -47,6 +47,10 @@ nav:
    - Training Loop: guide/training-loop.md
    - Skill Document: guide/skill-document.md
    - Deep Learning Analogy: guide/dl-analogy.md
  - SkillOpt-Sleep:
    - Overview: sleep/README.md
    - OpenAI-compatible Endpoints: sleep/openai-compatible-endpoints.md
    - Results: sleep/RESULTS.md
  - Extension Guides:
    - Add a New Benchmark: guide/new-benchmark.md
    - Local Environment Smoke Tests: guide/local-env-smoke.md
@@ -1,262 +1,178 @@
-# SkillOpt-Sleep — plugins for Claude Code, Codex, Copilot, and Devin
+# SkillOpt-Sleep integrations
-**Your coding agent forgets everything between sessions. SkillOpt-Sleep fixes
+**SkillOpt-Sleep** reviews recent agent sessions, mines recurring tasks, replays
-that.** While you sleep, it reviews what you did today, notices the rules you
+them, and proposes bounded updates to memory and skills. A held-out validation
-keep repeating ("always add a LIMIT", "answers in `\boxed{}`", "cite the
+gate decides whether a proposal is worth staging, and nothing live changes until
-source"), and writes them into your agent's long-term memory and skills — but
+the user explicitly adopts it.
 only the rules that actually make it score better on *your own* past tasks. You
 wake up to an agent that's better at *your* work, and you approve every change
 before it sticks.
-One engine, four thin shells. It synthesizes **SkillOpt** (validation-gated
+The shared engine lives in [`skillopt_sleep/`](../skillopt_sleep) and has no
-bounded text optimization — the research in this repo), **Claude Dreams**
+runtime dependency on the paper's `skillopt/` experiment package.
 (offline consolidation; input never mutated; review-then-adopt), and the **agent
 sleep** idea (short-term experience → long-term competence).
-> **Open-source tool, decoupled from the research.** The engine lives in the
+## Available integrations
 > top-level [`skillopt_sleep/`](../skillopt_sleep) package with **zero
 > dependency** on the paper's `skillopt/` experiment code (the validation gate is
 > vendored). Use it without the research stack.
---
+Four integrations wrap the shared `skillopt_sleep` CLI. OpenClaw is a separate
 reference adaptation with its own backend and setup assumptions.
 | Platform | Folder | Mechanism | Status |
 |---|---|---|---|
-| **Claude Code** | [`claude-code/`](claude-code) | `.claude-plugin` + `/skillopt-sleep` + `/skillopt-sleep-handoff` commands + skill + hooks | full, installable |
+| **Claude Code** | [`claude-code/`](claude-code) | marketplace plugin, commands, skill, and hooks | installable shared-engine integration |
-| **Codex** | [`codex/`](codex) | user-level `skillopt-sleep` skill + shared runner | full |
+| **Codex** | [`codex/`](codex) | user-level skill and shared runner | installable shared-engine integration |
-| **Copilot** | [`copilot/`](copilot) | MCP server (`sleep_*` tools) + `copilot-instructions` | full (MCP) |
+| **GitHub Copilot** | [`copilot/`](copilot) | MCP server exposing seven `sleep_*` tools | shared-engine MCP integration |
-| **Devin** | [`devin/`](devin) | MCP server (`sleep_*` tools) + Devin ATIF-v1.7 harvest + `.devin/rules` | full (MCP) |
+| **Devin** | [`devin/`](devin) | MCP server plus Devin transcript conversion | shared-engine MCP integration |
 | **OpenClaw** | [`openclaw/`](openclaw) | custom DeepSeek/Ollama wrapper | independent reference adaptation; review and adapt before use |
-## Install (pick your agent)
+## Install
 Clone the repository first unless an installed `skillopt-sleep` CLI is sufficient
 for your workflow.
 | Platform | Install | Then |
 |---|---|---|
-| **Claude Code** | `/plugin marketplace add microsoft/SkillOpt` → `/plugin install skillopt-sleep` | `/skillopt-sleep status` |
+| **Claude Code** | from the repository root, `/plugin marketplace add ./plugins/claude-code`, then `/plugin install skillopt-sleep@skillopt-sleep` | `/skillopt-sleep status` |
-| **Codex** | `git clone` → `bash plugins/codex/install.sh` | `/skillopt-sleep status` |
+| **Codex** | `bash plugins/codex/install.sh` | ask Codex to use the `skillopt-sleep` skill |
-| **Copilot** | `git clone` → register `plugins/copilot/mcp_server.py` as an MCP server | ask "run the sleep cycle" |
+| **Copilot** | register `plugins/copilot/mcp_server.py` using its example MCP config | ask Copilot to run `sleep_status` |
-| **Devin** | `git clone` → `devin mcp add skillopt-sleep -- python3 plugins/devin/mcp_server.py` | ask "run the sleep cycle" |
+| **Devin** | register `plugins/devin/mcp_server.py` using its example MCP config | ask Devin to run `sleep_status` |
 | **OpenClaw** | follow and adapt [`openclaw/README.md`](openclaw/README.md) | validate paths, credentials, and tasks locally |
-Requirements: Python ≥ 3.10 and the agent's CLI on PATH. All three call the same
+Python 3.10 or newer is required. Real CLI backends also require the selected
-[`run-sleep.sh`](run-sleep.sh) → `python -m skillopt_sleep`, so behaviour is
+agent CLI to be installed and authenticated.
 identical everywhere. Default backend is `mock` (no API spend); `--backend
 claude|codex|copilot` uses your own budget.
---
+The shared [`run-sleep.sh`](run-sleep.sh) supports both source checkouts and
 installed packages. If it cannot find the repository, it tries the
 `skillopt-sleep` executable on `PATH` (including `uv tool`/`pipx` installs), then
 an importable `skillopt_sleep` module. Install with `uv tool install skillopt` or
 `pip install skillopt` when using that fallback.
-## How it works: one "night", in plain terms
+> **Version note.** This integration reference tracks `main`. PyPI 0.2.0
 > supports the base Sleep CLI, while handoff, Sleep support for non-Azure
 > OpenAI-compatible endpoints, and `--preferences` require a source checkout
 > from `main` until the next release.
-```
+## One sleep cycle
-harvest your past sessions → mine the tasks you keep doing → replay them offline
+
-  → reflect on failures → propose a few rule edits → KEEP only edits that raise
+```text
-    your held-out score → stage a proposal → (you) review & adopt
+harvest supported local sessions → mine recurring tasks → replay tasks
  → reflect and propose bounded edits → validate on held-out real tasks
  → stage proposal → (you) review and adopt
 ```
-Nothing live changes until you `adopt`; every adopt backs up the prior file.
+The default backend is `mock`: it makes no provider calls and is useful for
 checking plumbing. A real backend is required for model-driven mining and genuine
 optimization.
-### The split that keeps it honest: dream-train / real-val / real-test
+## Data boundary
-This is the heart of the design, borrowed from the SkillOpt paper's
+- Harvesting is local and read-only. The `mock` backend has no model-provider
-train/selection/test protocol:
+  data path and no API spend.
 - A real backend sends truncated transcript excerpts and derived task content to
  the provider selected for mining, replay, judging, and reflection.
 - Outbound prompts are not currently guaranteed to be free of secrets. Do not
  use a third-party provider on sensitive transcripts without reviewing the data
  source and the provider's retention policy.
 - For a reviewable workflow, export tasks first, inspect and redact the JSON, set
  its top-level `"reviewed"` field to `true`, and then use the task file with a
  real backend:
-| Split | Where it comes from | What it's for |
+  ```bash
  python -m skillopt_sleep harvest --project "$(pwd)" --output reviewed-tasks.json
  python -m skillopt_sleep dry-run --project "$(pwd)" --backend codex \
    --tasks-file reviewed-tasks.json --progress
  ```
  Real backends reject task files that are still marked unreviewed.
 For the separate API-key and Azure managed-identity transport boundaries, see
 [OpenAI-compatible endpoints](../docs/sleep/openai-compatible-endpoints.md).
 ## Supported CLI surface
 Actions:
 | Action | Behavior |
 |---|---|
 | `status` | show state and the latest staged proposal |
 | `dry-run` | harvest, mine, replay, and report; stage nothing |
 | `run` | run the full cycle and stage a proposal |
 | `adopt` | apply the latest staged proposal, with backups |
 | `harvest` | inspect or export mined tasks |
 | `schedule` / `unschedule` | install or remove the managed nightly cron entry |
 Common implemented flags include:
 | Flag | Default | Purpose |
 |---|---|---|
-| **train** | your real tasks **+ optional "dreamed" variants** | what the optimizer *learns from*. Over-dreaming here is fine — it's imagination. |
+| `--backend mock\|claude\|codex\|copilot\|handoff\|azure_openai` | `mock` | select who performs model calls |
-| **val** (selection) | **your real tasks only**, held out | the **gate**: an edit is kept only if it raises this score. Stops overfitting. |
+| `--model NAME` | backend default | select a backend-specific model |
-| **test** | **your real tasks only**, held out, never seen during optimization | the **final score** we report. Kept as close to your real usage as possible. |
+| `--source claude\|codex\|auto` | `claude` | select the transcript source |
 | `--project PATH` | current directory | select the project and invoked harvest scope |
 | `--scope invoked\|all` | `invoked` | limit transcript harvesting |
 | `--target-skill-path PATH` | managed skill | select a specific `SKILL.md` to stage/adopt |
 | `--tasks-file PATH` | none | replay a reviewed task file instead of harvesting |
 | `--max-sessions N` / `--max-tasks N` | unset → `3 × tasks` / `40` tasks | bound harvested work; these are not hard token or wall-clock budgets |
 | `--edit-budget N` | `4` | cap bounded edits per cycle |
 | `--preferences "..."` | empty | add house rules to the reflection prior |
 | `--progress` | off | print phase progress to stderr |
 | `--auto-adopt` | off | adopt an accepted proposal without a separate command |
 | `--json` | off | emit machine-readable output where supported |
-So you can **dream up extra training examples** to learn a rule robustly, while
+The nightly CLI does **not** currently expose `--gate`, `--rollouts-k`,
-the rule is still **judged on real, unseen tasks**. A `dream` task can *never*
+`--optimizer-model`, `--target-model`, `--budget-tokens`, or `--budget-minutes`.
-land in val or test — that invariant is unit-tested.
+Do not pass experiment-harness flags to the main CLI.
---
+### Preferences
-## What each feature does **for you** (with examples)
+`--preferences` is the main user-facing steering knob:
 Every control below works on all three platforms (pass it after the action,
 e.g. `/skillopt-sleep run --rollouts-k 3`).
 ### `--preferences "..."` — tell it your house rules
 The single most useful knob. Free text that steers what the optimizer writes,
 as a prior. Use it to encode the conventions you're tired of repeating.
 ```bash
-# A backend engineer:
+python -m skillopt_sleep run --backend codex --project "$(pwd)" \
-/skillopt-sleep run --preferences "Always use async/await, never callbacks. \
+  --preferences "Prefer pytest. Keep commit subjects imperative and concise."
  Prefer pytest over unittest. Commit subjects in imperative mood under 50 chars."
 # A data analyst:
 /skillopt-sleep run --preferences "Every SQL query must end with LIMIT 1000 unless \
  I say otherwise. Money in USD with 2 decimals. Prefer CTEs over nested subqueries."
 # A researcher:
 /skillopt-sleep run --preferences "Cite sources as [Author, Year]. Math answers in \
  \\boxed{}. Keep explanations under 150 words unless I ask for depth."
 ```
 *What it does for you:* the next morning your agent already follows these
 without you re-typing them, and the rules are validated against your real tasks
 (if a "preference" actually hurts your held-out score, the gate drops it).
-### `--gate on|off` — strict vs. greedy
+Preferences guide reflection but remain subject to the validation gate.
- `on` (default): an edit is kept **only if it raises your held-out score**.
+### Advanced config
  Safe — blocks plausible-but-wrong rules and reward-hacking.
 - `off`: greedy — keep edits without the strict check (still reports whether
  quality moved).
-*What it does for you:* leave it `on` for trust. Flip it `off` when you're
+The JSON/YAML config under `~/.skillopt-sleep/` supports additional engine keys,
-exploring and want to see everything the optimizer proposes.
+including `gate_mode`, `gate_metric`, `dream_rollouts`, `dream_factor`, `recall_k`,
 `evolve_memory`, and `evolve_skill`. These are config keys, not aliases for the
 unsupported CLI flags listed above. Shipping defaults are conservative:
 `gate_mode="on"`, `dream_rollouts=1`, `dream_factor=0`, and `recall_k=0`.
-### `--rollouts-k K` — learn from contrast, not just failure
+### Handoff backend
-Re-runs each task `K` times and learns from the difference between the **good**
+`--backend handoff` keeps model subprocesses out of the engine. It writes pending
-and **bad** attempts, not just a single failure.
+model calls to `.skillopt-sleep-handoff/PROMPTS.md` and `pending.json`, exits with
-
+code 3, and resumes after answers are placed in `answers/<id>.md`:
 ```bash
 /skillopt-sleep run --rollouts-k 3
 ```
 *What it does for you:* a much stronger signal. If your agent gets a task right 1
 time in 3, the optimizer figures out *what the winning attempt did* and makes it
 reliable.
 ### `--optimizer-model` / `--target-model` — optimize cheap, deploy anywhere
 Use a strong model to *write* the rules and a cheap model to *run* your tasks.
 The learned skill then helps the cheap model — or any model.
 ```bash
 /skillopt-sleep run --optimizer-model sonnet --target-model haiku
 ```
 *What it does for you:* spend a little on a smart optimizer overnight; your
 everyday cheap/fast agent inherits the upgrade. (Verified: a skill optimized on
 one model lifts a different one — cross-model and even cross-runtime
 Codex↔Claude.)
 ### `--budget-tokens N` / `--budget-minutes M` — cap the spend
 You decide how much the nightly "dreaming" costs; it auto-plans how many nights
 × how many rollouts fit.
 ```bash
 /skillopt-sleep run --backend claude --budget-tokens 60000
 ```
 *What it does for you:* predictable cost. It stops cleanly when the budget is hit
 and tells you what it skipped.
 ### multi-objective (accuracy ↑, tokens ↓, latency ↓)
 The reward can weight not just correctness but **cost and speed**, so a skill can
 learn to be cheaper and faster, not only more accurate. *What it does for you:*
 "answer directly instead of opening five files" becomes a learned habit.
 ### `--backend handoff` — session-executed calls (no API subprocess)
 For subscription seats and environments where the engine shouldn't spawn
 `claude -p` / API calls itself. The engine still runs every deterministic
 stage (harvest → mine → replay scoring → gate → stage), but each model call
 (attempt / judge / reflect) is written to a prompt file that **your own agent
 session answers between engine runs**:
 ```bash
 python -m skillopt_sleep run --backend handoff --project "$(pwd)"
-# exit 3 => .skillopt-sleep-handoff/PROMPTS.md + pending.json were written
+# answer each prompt in a fresh context, then run the same command again
 #   answer each prompt (each in a FRESH context) into answers/<id>.md
 # re-run the same command => it resumes from the answers and either
 #   finishes (exit 0) or stages the next prompt batch (exit 3)
 ```
-A typical night converges in 3–6 rounds: baseline attempts → reflect →
+Answering held-out prompts from a context that has already seen their references
-candidate re-scoring per accepted edit. Resume is stateless — replay is
+contaminates the validation gate. Claude Code's `/skillopt-sleep-handoff` command
-deterministic and answers are cached by prompt hash, so re-running skips
+automates the loop with isolated fresh-context subagents.
 everything already answered. Mined tasks are pinned to
 `.skillopt-sleep-handoff/tasks.json` on the first round, so the sessions that
 answer the prompts can't shift the task set and invalidate earlier answers.
 On a completed real run the handoff directory is archived to
 `.skillopt-sleep-handoff.night<N>.done`.
-On Claude Code, `/skillopt-sleep-handoff run` drives the whole loop for you,
+## Validation
 answering each prompt in an isolated fresh-context subagent.
-**Integrity rule:** answer every prompt in a fresh context (a subagent with no
+The deterministic no-provider check exercises consolidation and the gate:
 conversation history). Answering from a session that has already seen the
 mined tasks and their references contaminates the held-out gate and fakes the
 improvement score.
 *What it does for you:* the sleep cycle runs entirely on your interactive
 session's subscription budget — no API key, no headless subprocess — while the
 gate, splits, and staging discipline stay in the engine.
 Limitations: `--rollouts-k > 1` gives no contrastive spread (identical prompt
 → identical answer file), and tool-loop tasks fall back to the single-shot
 `TOOL_CALL:` marker convention.
 ### `schedule` / `unschedule` — set it and forget it
 Built-in nightly scheduling (no manual cron):
 ```bash
-/skillopt-sleep schedule --hour 3 --minute 17     # runs every night for this project
+python -m skillopt_sleep.experiments.run_experiment \
-/skillopt-sleep unschedule                        # stop it
+  --persona researcher --assert-improves
 ```
 *What it does for you:* it just gets better while you sleep. The nightly run only
 *stages* a proposal — adopting is still your call (or add `--auto-adopt` when you
 schedule, if you trust it).
---
+Real-model benchmark results and their limitations are documented in
 [`docs/sleep/RESULTS.md`](../docs/sleep/RESULTS.md). The benchmark recipes are not
 the shipping CLI defaults.
-## Full action / flag reference
+## Safety summary
-| Action | Does |
+- Session harvesting is read-only.
-|---|---|
+- `mock` replay makes no provider calls.
-| `status` | nights so far + the latest staged proposal (read-only) |
+- `run` stages proposals; `adopt` is the normal live-change boundary.
-| `dry-run` | harvest→mine→replay→report; **stages nothing** |
+- Adoption backs up existing target files.
-| `run` | full cycle; **stages** a proposal; nothing live changes |
+- `--max-sessions` and `--max-tasks` bound work, but the main CLI does not yet
-| `adopt` | apply the staged proposal to `CLAUDE.md`/`SKILL.md` (backs up first) |
+  enforce a hard token or elapsed-time budget.
-| `harvest` | debug: print the recurring tasks it mined |
+- Treat real-backend transcript excerpts as data shared with the selected
-| `schedule` / `unschedule` | install/remove the nightly cron entry |
+  provider.
 | Flag | Default | Meaning |
 |---|---|---|
 | `--backend mock\|claude\|codex\|copilot\|handoff` | `mock` | who runs/optimizes (mock = free; handoff = your own session answers) |
 | `--preferences "..."` | – | your house rules, as a prior |
 | `--gate on\|off` | `on` | strict held-out gate vs. greedy |
 | `--rollouts-k K` | `1` | multi-rollout contrastive reflection |
 | `--optimizer-model` / `--target-model` | – | split the optimizer from the target |
 | `--budget-tokens` / `--budget-minutes` | – | cap the nightly spend |
 | `--scope invoked\|all` | `invoked` | this project only, or all projects |
 | `--auto-adopt` | off | apply without manual review (power users) |
 Deep dive: [the SkillOpt-Sleep guide section](https://microsoft.github.io/SkillOpt/docs/guideline.html#sleep).
 ---
 ## Does it actually work?
 Yes — measured with **real models on both Claude and Codex**, scored on held-out
 tasks the optimizer never trained on:
 - **gbrain-evals `skillopt-v1`** (the public suite gbrain scores SkillOpt on):
  deficient skills go **0.00 → 1.00** on all 4 seeds, including a real tool-use
  loop; cross-model transfer is positive; the gate blocks regressions.
  → [the SkillOpt-Sleep guide section](https://microsoft.github.io/SkillOpt/docs/guideline.html#sleep)
 - **Academic daily-cases** (math / spreadsheet / search-QA, the paper's 4:1:5
  split with dream-augmented train): see
  [the SkillOpt-Sleep guide section](https://microsoft.github.io/SkillOpt/docs/guideline.html#sleep).
 - **Fresh load-test** (a "SQL must always include LIMIT" analyst, built from
  scratch): held-out **0.00 → 1.00** on both backends.
  → [the SkillOpt-Sleep guide section](https://microsoft.github.io/SkillOpt/docs/guideline.html#sleep)
 Try the deterministic proof yourself (no API key, no spend):
 ```bash
 python -m skillopt_sleep.experiments.run_experiment --persona researcher --assert-improves
 ```
 It prints the held-out score rising to 1.0 as the gate accepts the right rules,
 and confirms the gate **rejects** an injected harmful edit.
 ---
 ## Safety
 - **Read-only** harvest of your sessions. `mock` replay has no side effects.
 - Proposals are **staged**, never auto-applied (unless you opt in with `--auto-adopt`).
 - Every adopt writes a backup. Per-night token/time budget caps. Secrets redacted.
@@ -10,7 +10,7 @@ SkillOpt-Sleep is the **deployment-time** companion to
 [SkillOpt](https://github.com/microsoft/SkillOpt). SkillOpt trains a skill
 offline on a benchmark; SkillOpt-Sleep applies the same discipline to *your own
 daily usage*: bounded text edits, accepted only through a held-out validation
-gate, with rejected edits kept as negative feedback.
+gate, with rejected candidates recorded in the cycle report for review.
 It synthesizes three ideas:
@@ -32,7 +32,8 @@ then adopt or discard" contract). Every adopt backs up the prior file first.
 ## Install
-**Requirements:** Python ≥ 3.10, and the `claude` CLI (and/or `codex` CLI) on PATH.
+**Requirements:** Python ≥ 3.10. A real CLI backend additionally requires its
 corresponding `claude` or `codex` executable on `PATH` and authenticated.
 ```bash
 # 1) get the code (the plugin ships inside the SkillOpt repo)
@@ -40,7 +41,7 @@ git clone https://github.com/microsoft/SkillOpt.git
 cd SkillOpt
 # 2) add the plugin to Claude Code as a local marketplace
-/plugin marketplace add ./skillopt-sleep-plugin
+/plugin marketplace add ./plugins/claude-code
 /plugin install skillopt-sleep@skillopt-sleep
 # 3) verify
@@ -48,15 +49,21 @@ cd SkillOpt
 ```
 The plugin's bundled runner (`scripts/sleep.sh`) auto-selects a Python ≥ 3.10
-interpreter and calls the `skillopt_sleep` engine in the repo. No `pip install`
+interpreter and calls the `skillopt_sleep` engine. A source checkout needs no
-is required for the default `mock` backend or for `claude`/`codex` backends —
+`pip install`. If the marketplace cache does not contain a usable source tree,
-they shell out to the CLIs you already have.
+the shared runner falls back first to a `skillopt-sleep` executable on `PATH`
 (including `uv tool`/`pipx` installs), then to an importable Python module. Use
 `uv tool install skillopt` or `pip install skillopt` for that fallback.
 > **Version note.** This page tracks `main`. PyPI 0.2.0 provides the base Sleep
 > CLI, but handoff mode and `--preferences` require a source checkout from
 > `main` until the next release.
 ## Quick start
 ```bash
 # from inside any project you use with Claude Code:
-/skillopt-sleep dry-run     # safe preview: what it would learn, no changes staged
+/skillopt-sleep dry-run     # preview what it would learn; no changes staged
 /skillopt-sleep run         # full cycle: stages a reviewed proposal (still no live edits)
 /skillopt-sleep status      # see history + the latest staged proposal
 /skillopt-sleep adopt       # apply the staged proposal to CLAUDE.md / SKILL.md (with backup)
@@ -74,8 +81,19 @@ python -m skillopt_sleep run --project "$(pwd)" --backend codex    # real lift v
 ```
 Default backend is **`mock`** — deterministic, no API spend — so you can try the
-plumbing for free. Switch to `--backend claude` or `--backend codex` for genuine
+plumbing for free. Switch to `--backend claude` or `--backend codex` for
-improvement on your own budget.
+model-driven mining and optimization on your own budget; an accepted gain is
 task- and model-dependent, not guaranteed.
 ### Data boundary for real backends
 Harvesting `~/.claude` is local and read-only, and the `mock` backend makes no
 provider calls. A real backend sends truncated transcript excerpts and derived
 tasks to the selected provider for mining, replay, judging, and reflection.
 Outbound prompts are not currently guaranteed to be secret-free. Review your
 session data and provider policy before using a real backend on a sensitive
 project; the [shared integration guide](../README.md#data-boundary) describes a
 reviewed task-file workflow.
 ### Handoff mode (session answers the model calls)
@@ -95,7 +113,7 @@ python -m skillopt_sleep run --backend handoff --project "$(pwd)"   # resume
 Answer every prompt in a **fresh context** — a session that has already seen
 the mined tasks and their references would contaminate the held-out gate.
-Details: [the plugins README](../README.md#--backend-handoff--session-executed-calls-no-api-subprocess).
+Details: [the plugins README](../README.md#handoff-backend).
 ## Does it actually improve? (real models, public benchmark)
@@ -114,8 +132,8 @@ Both took a brief-writer with no risks section / no confidence level and, within
 1–2 nights, proposed gated edits that lifted the held-out score to perfect —
 into the protected `LEARNED` block, nothing else touched. The Codex 2-night
 trace even shows the optimizer **diagnosing its own residual failure** and
-adding a meta-rule to fix it. Full writeup + reproduction:
+adding a meta-rule to fix it. See the recorded results and limitations in
-[the SkillOpt-Sleep guide section](https://microsoft.github.io/SkillOpt/docs/guideline.html#sleep).
+[`docs/sleep/RESULTS.md`](../../docs/sleep/RESULTS.md).
 Reproduce:
@@ -138,24 +156,32 @@ python -m skillopt_sleep.experiments.run_experiment --persona programmer  --asse
 Each prints the held-out score rising from baseline toward 1.0 as the gate
 accepts the general rules your tasks need, and confirms the gate **rejects** an
-injected harmful edit. Recorded output: [the SkillOpt-Sleep guide section](https://microsoft.github.io/SkillOpt/docs/guideline.html#sleep).
+injected harmful edit. Context for the measured experiments is in
 [`docs/sleep/RESULTS.md`](../../docs/sleep/RESULTS.md).
 ## Schedule it nightly
 ```bash
-"${CLAUDE_PLUGIN_ROOT}/scripts/install-cron.sh" "$(pwd)"   # prints a crontab line; installs nothing
+/skillopt-sleep schedule --hour 3 --minute 17
 /skillopt-sleep unschedule
 ```
 The built-in scheduler creates a managed cron entry and logs under the project.
 The scheduled run stages proposals unless `--auto-adopt` is explicitly selected.
 ## Safety
 - **Read-only** harvest of `~/.claude`. `mock` replay has no side effects.
 - Proposals are **staged**, never auto-applied (unless you opt in with `--auto-adopt`).
 - Every adopt writes a backup under the staging dir's `backup/`.
- Per-night **token/task budget caps**; secrets redacted from prompts.
+- `--max-sessions` and `--max-tasks` bound work, but the main CLI does not enforce
- `fresh` replay (Phase 3) runs only in throwaway git worktrees.
+  a hard token or wall-clock budget.
 - Real backends share truncated session/task content with the selected provider;
  do not assume outbound prompts have been fully redacted.
 ## Status
-Phase 1 (engine + deterministic experiment + plugin surface) is complete.
+The engine, deterministic experiment, Claude/Codex CLI backends, handoff mode,
-Phase 3 adds the real-API miner/judge and `fresh` worktree replay. See
+and staged adoption flow are implemented. Advanced experiment-harness flags are
-[`docs/superpowers/specs/2026-06-07-skillopt-sleep-claude-code-plugin-design.md`](../docs/superpowers/specs/2026-06-07-skillopt-sleep-claude-code-plugin-design.md).
+not automatically available on the nightly CLI; see the
 [shared integration reference](../README.md#supported-cli-surface).
@@ -21,10 +21,11 @@ isolated context so the validation gate stays honest.
 Repeat until the engine exits 0 (done) — at most 8 rounds:
-1. **Run the engine** via the bundled runner:
+1. **Run the engine** via the bundled runner. Split `$ARGUMENTS` into the action
   and remaining options, and preserve those options on every resumed round:
   ```bash
-   "${CLAUDE_PLUGIN_ROOT}/scripts/sleep.sh" <action> --backend handoff --project "$(pwd)" --scope invoked
+   "${CLAUDE_PLUGIN_ROOT}/scripts/sleep.sh" <action> --backend handoff --project "$(pwd)" --scope invoked <remaining options>
   ```
   - exit 0 → the night is complete; go to "Finish" below.
@@ -49,10 +50,13 @@ Repeat until the engine exits 0 (done) — at most 8 rounds:
 ## Finish
- `Read` the `report.md` in the staging dir the engine printed and show
+- For `run`, if the engine prints a staging directory, `Read` its `report.md`
-  the user: held-out baseline → candidate score, the gate decision, the
+  and show the user: held-out baseline → candidate score, the gate decision,
-  proposed edits, and where the proposal is staged.
+  the proposed edits, and where the proposal is staged. If an accepted proposal
- Tell the user nothing live changed; offer `/skillopt-sleep adopt`.
+  was staged, tell the user nothing live changed and offer
  `/skillopt-sleep adopt`.
 - For `dry-run`, no staging directory or `report.md` is created; summarize the
  final stdout instead.
 - The engine archives `.skillopt-sleep-handoff/` on a completed real run;
  do not delete it yourself.
@@ -65,3 +69,6 @@ Repeat until the engine exits 0 (done) — at most 8 rounds:
  set. Do not edit that file.
 - If a batch looks like it contains secrets or content the user would not
  want re-processed, stop and ask before answering.
 - Handoff files apply pattern-based secret redaction, but that is not a
  guarantee that prompts are free of sensitive data. Treat the pending batch as
  private user data and do not copy it into chat, logs, or commits.
@@ -1,5 +1,5 @@
 ---
-description: Run or manage the SkillOpt-Sleep self-evolution cycle (review past sessions, replay tasks offline, consolidate validated memory + skills; can also schedule nightly runs)
+description: Run or manage the SkillOpt-Sleep self-evolution cycle (review past sessions, replay tasks through a selected backend, consolidate validated memory + skills, or schedule nightly runs)
 argument-hint: "[run | dry-run | status | adopt | harvest | schedule | unschedule] (default: status)"
 allowed-tools: Bash, Read
 ---
@@ -7,10 +7,11 @@ allowed-tools: Bash, Read
 # /skillopt-sleep — SkillOpt-Sleep nightly self-evolution
 You are driving **SkillOpt-Sleep**: a tool that lets this user's Claude agent
-improve offline by reviewing past sessions, replaying recurring tasks, and
+improve from past usage by reviewing sessions, replaying recurring tasks, and
 consolidating what it learns into **validated** memory (`CLAUDE.md`) and skills
-(`SKILL.md`). It is gated like SkillOpt: a change is kept only if it improves a
+(`SKILL.md`). With the default gate enabled, a change is kept only if it improves
-held-out replay score, and nothing live is modified until the user adopts it.
+a held-out replay score. Nothing live is modified until adoption unless the
 user explicitly requests `--auto-adopt`.
 ## Requested action: $ARGUMENTS
@@ -18,11 +19,14 @@ held-out replay score, and nothing live is modified until the user adopts it.
 ## How to run it
-The engine is the `skillopt_sleep` Python package in this repo. Use the
+The engine is the `skillopt_sleep` Python package in this repo. Split
-**plugin's bundled runner** so the right interpreter and repo are on the path:
+`$ARGUMENTS` into the first action token and its remaining options, then use the
 **plugin's bundled runner** so the right interpreter and repo are on the path.
 Preserve the user's remaining options (for example `--preferences`, `--backend`,
 or `--target-skill-path`) instead of silently dropping them:
 ```bash
-"${CLAUDE_PLUGIN_ROOT}/scripts/sleep.sh" <action> --project "$(pwd)" --scope invoked
+"${CLAUDE_PLUGIN_ROOT}/scripts/sleep.sh" <action> --project "$(pwd)" --scope invoked <remaining options>
 ```
 `<action>` is one of:
@@ -30,37 +34,49 @@ The engine is the `skillopt_sleep` Python package in this repo. Use the
 | action       | what it does |
 |--------------|--------------|
 | `status`     | show how many nights have run + the latest staged proposal (READ-ONLY) |
-| `dry-run`    | harvest → mine → replay → report, but **stage nothing** (safe preview) |
+| `dry-run`    | harvest → mine → replay → report, but **stage nothing** (no-staging preview) |
-| `run`        | full cycle: also **stage** a reviewed proposal (still does NOT touch live files) |
+| `run`        | full cycle: **stage** a validation report and any accepted proposal; only explicit `--auto-adopt` may also update live files |
 | `adopt`      | apply the latest staged proposal to live `CLAUDE.md` / `SKILL.md` (backs up first) |
 | `harvest`    | debug: print the recurring tasks mined from recent sessions |
 | `schedule`   | install a nightly cron entry for this project (`--hour --minute`, off-:00 by default) |
 | `unschedule` | remove the nightly cron entry (`--all` to remove every managed entry) |
 Default backend is `mock` (deterministic, no API spend). To use real budget for
-genuine improvement, add `--backend claude` or `--backend codex`. To steer what
+model-driven optimization, add `--backend claude` or `--backend codex`. An
-the optimizer writes, add `--preferences "<your house rules>"`.
+accepted gain is evidence on this run's held-out tasks, not a guarantee of
 general improvement; results depend on the tasks, model, and checks. To steer
 what the optimizer writes, add `--preferences "<your house rules>"`.
 ## Steps to follow
-1. **Run the requested action** via the bundled runner above. Capture stdout.
+1. **Run the requested action** via the bundled runner above. Capture stdout and
-2. **For `run` / `dry-run`:** after it completes, `Read` the generated
+   stderr.
-   `report.md` in the staging dir it prints, and show the user:
+2. **For `run`:** if it prints a staging directory, `Read` its `report.md` and
-   - held-out score: baseline → candidate (the proof it helped)
+   show the user:
   - held-out score: baseline → candidate (evidence on this run's held-out tasks)
   - the gate decision (accept/reject) and the exact edits it proposes
   - where the proposal is staged
-3. **For `run` that produced an accepted proposal:** tell the user the diff is
+3. **For `dry-run`:** no staging directory or `report.md` is created. Summarize
-   staged and that **nothing live changed yet**. Offer to run `/skillopt-sleep adopt`.
+   the score, gate decision, and edits from stdout (or request `--json` when
-4. **For `adopt`:** confirm which live files were updated and that backups were
+   machine-readable output is useful).
 4. **For `run` that produced an accepted proposal:** inspect whether stdout says
   it was auto-adopted. If not, tell the user nothing live changed and offer
   `/skillopt-sleep adopt`; if it was, report the updated paths explicitly.
 5. **For `adopt`:** confirm which live files were updated and that backups were
   written under the staging dir's `backup/`.
-5. **Never** edit `CLAUDE.md` or `SKILL.md` yourself — only the `adopt` action
+6. **Never** edit `CLAUDE.md` or `SKILL.md` yourself — let the engine's explicit
-   does that, with a backup. Respect the review gate.
+   `adopt` or user-requested `--auto-adopt` path apply its manifest and backup
   behavior. Respect the review gate.
 ## Safety reminders
 - Harvest is **read-only** over `~/.claude`. Replay in `mock` mode runs no
  shell side effects.
- The cycle stages proposals; the user is in control of adoption.
+- The cycle stages proposals by default; auto-adoption requires explicit opt-in.
- If the user asks to schedule this nightly, point them at
+- A real backend sends truncated transcript excerpts and derived tasks to its
-  `${CLAUDE_PLUGIN_ROOT}/scripts/install-cron.sh` (prints a crontab line; does
+  provider for mining, replay, judging, and reflection. Pattern-based redaction
-  not install anything without confirmation).
+  is not a guarantee that outbound prompts are secret-free. For sensitive data,
  use `mock` or first run `harvest --output <file>`, review/redact the file, set
  `"reviewed": true`, and then pass it with `--tasks-file`.
 - `schedule` manages a cron entry when `crontab` is available; otherwise it
  prints a line for manual installation.
@@ -24,6 +24,7 @@ cat <<EOF
 ${MIN} ${HOUR} * * *  "${RUNNER}" run --project "${PROJECT}" --scope invoked --backend ${BACKEND} >> "${PROJECT}/.skillopt-sleep/cron.log" 2>&1
 #
 # For fully-autonomous adoption (power users), append: --auto-adopt
-# To spend real API budget for genuine lift, set BACKEND=anthropic above.
+# To use the authenticated Claude CLI for model-driven optimization, set
 # BACKEND=claude above.
 # ────────────────────────────────────────────────────────────────────────────
 EOF
@@ -1,25 +1,29 @@
 ---
 name: skillopt-sleep
-description: "Use when the user wants their Claude agent to self-improve from past usage, asks about a nightly/offline 'sleep' or 'dream' cycle, memory/skill consolidation, or says things like 'make my agent better the more I use it', 'review my past sessions', 'learn my preferences', 'consolidate what you learned', 'run the sleep cycle', or wants to schedule offline self-optimization. Drives the skillopt_sleep engine: harvest past sessions -> mine recurring tasks -> replay offline -> consolidate validated CLAUDE.md/SKILL.md behind a held-out gate."
+description: "Use when the user wants their Claude agent to self-improve from past usage, asks about a nightly/offline 'sleep' or 'dream' cycle, memory/skill consolidation, or says things like 'make my agent better the more I use it', 'review my past sessions', 'learn my preferences', 'consolidate what you learned', 'run the sleep cycle', or wants to schedule background self-optimization. Drives the skillopt_sleep engine: harvest past sessions -> mine recurring tasks -> replay through a selected backend -> consolidate validated CLAUDE.md/SKILL.md behind a held-out gate."
 ---
-# SkillOpt-Sleep: offline self-evolution for a local Claude agent
+# SkillOpt-Sleep: usage-driven self-evolution for a local Claude agent
-SkillOpt-Sleep gives the user's agent a **sleep cycle**. While the user is
+SkillOpt-Sleep gives the user's agent a **sleep cycle**. On demand or on a
-offline (e.g. nightly), it reviews their real past Claude Code sessions,
+nightly schedule, it reviews real past Claude Code sessions, re-runs recurring
-re-runs recurring tasks on their own API budget, and consolidates what it
+tasks through the selected backend, and consolidates what it
-learns into **memory** (`CLAUDE.md`) and **skills** (`SKILL.md`) — but only
+learns into **memory** (`CLAUDE.md`) and **skills** (`SKILL.md`). With the
-keeps changes that pass a held-out validation gate, and only after the user
+default validation gate enabled, it keeps only changes that improve a held-out
-adopts them. The agent gets measurably better at *this* user's recurring work,
+score. Live files change only through explicit adoption or a user-requested
 `--auto-adopt`. It aims to improve this user's recurring work, while making
 each accepted proposal measurable on the run's held-out tasks,
 with no model-weight training. It is the deployment-time analogue of training:
 short-term experience → long-term competence.
 It synthesizes three ideas:
 - **SkillOpt** — the skill/memory doc is trainable text; bounded add/delete/replace
-  edits; accepted only through a held-out gate; rejected edits become negative feedback.
+  edits; accepted only through a held-out gate; rejected edits are recorded in
- **Claude Dreams** — offline consolidation that reads past sessions and rebuilds
+  the run report for review.
-  memory (dedup/merge/resolve); the input is never mutated; output is reviewed then adopted.
+- **Claude Dreams** — consolidation that reads past sessions and proposes changes
- **Agent sleep** — periodic offline replay turns episodes into durable skill.
+  inside protected learned blocks; the input is never mutated, and output is
  reviewed before adoption.
 - **Agent sleep** — periodic background replay turns episodes into durable skill.
 ## When to use this skill
@@ -34,9 +38,14 @@ Trigger when the user wants any of:
 1. **Harvest** — read `~/.claude/projects/*/<session>.jsonl` + `~/.claude/history.jsonl` (READ-ONLY) → session digests.
 2. **Mine** — digests → `TaskRecord`s (recurring intents + outcome labels + checkable refs where possible).
-3. **Replay** — re-run tasks offline under the *current* skill+memory → (hard, soft) scores.
+3. **Replay** — re-run tasks through the selected backend under the *current*
-4. **Consolidate** — reflect on failures → propose bounded edits → **gate** on a held-out slice; accept only if it strictly improves.
+   skill+memory → (hard, soft) scores.
-5. **Stage** — write `proposed_CLAUDE.md`, `proposed_SKILL.md`, a diff, and `report.md` into `<project>/.skillopt-sleep/staging/<date>/`. **Nothing live changes.**
+4. **Consolidate** — reflect on failures → propose bounded edits → **gate** on a held-out slice; with the default gate enabled, accept only if it strictly improves.
 5. **Stage** — write the accepted `proposed_CLAUDE.md` and/or
   `proposed_SKILL.md`, plus `report.md`, `report.json`, `manifest.json`, and
   `diagnostics.json` into `<project>/.skillopt-sleep/staging/<timestamp>/`.
   **Nothing live changes.** A rejected run still has a report but no proposed
   live-file replacement.
 6. **Adopt** — explicit (or opt-in auto): copy staged files over live ones, backing up first.
 ## How to drive it
@@ -45,14 +54,20 @@ Prefer the `/skillopt-sleep` command. Under the hood it calls the bundled runner
 ```bash
 "${CLAUDE_PLUGIN_ROOT}/scripts/sleep.sh" status                       # what's happened
-"${CLAUDE_PLUGIN_ROOT}/scripts/sleep.sh" dry-run --project "$(pwd)"    # safe preview
+"${CLAUDE_PLUGIN_ROOT}/scripts/sleep.sh" dry-run --project "$(pwd)"    # no-staging preview
 "${CLAUDE_PLUGIN_ROOT}/scripts/sleep.sh" run --project "$(pwd)"        # full cycle, stages a proposal
 "${CLAUDE_PLUGIN_ROOT}/scripts/sleep.sh" adopt --project "$(pwd)"      # apply staged proposal (with backup)
 ```
 - Default backend is `mock` (deterministic, **no API spend**) — good for trying the plumbing.
- Add `--backend claude` or `--backend codex` to spend the user's real budget for genuine improvement.
+- Add `--backend claude` or `--backend codex` to spend the user's real budget
- Scope defaults to the invoked project; `--scope all` harvests every project.
+  for model-driven optimization. A held-out gain is run-specific evidence, not
  a guarantee of broader improvement; results depend on the tasks, model, and
  checks.
 - Scope defaults to the invoked project; `--scope all` harvests every Claude
  project into the current run's configured targets.
 - A real backend sends truncated transcript/task content to its provider. See
  the data-boundary rules below before using one with sensitive sessions.
 ### Scheduling
@@ -63,25 +78,30 @@ Prefer the `/skillopt-sleep` command. Under the hood it calls the bundled runner
 Installs a nightly cron entry. `unschedule --all` removes every managed entry.
-## All CLI flags
+## Common CLI flags
 | Flag | Default | Description |
 |------|---------|-------------|
 | `--project PATH` | cwd | Project directory to evolve |
 | `--scope all\|invoked` | invoked | Harvest scope |
-| `--backend mock\|claude\|codex\|copilot` | mock | Replay backend (mock = no API spend) |
+| `--backend mock\|claude\|codex\|copilot\|handoff\|azure_openai` | mock | Backend (mock = no provider calls) |
 | `--model NAME` | backend default | Override the model used for replay |
 | `--source claude\|codex\|auto` | claude | Transcript source |
 | `--lookback-hours N` | 72 | Harvest window |
-| `--max-sessions N` | unlimited | Cap harvested sessions |
+| `--max-sessions N` | derived | Cap harvested sessions; defaults to 3 × max tasks (120 with current defaults) |
 | `--max-tasks N` | 40 | Cap mined tasks |
-| `--target-skill-path PATH` | auto | Explicit SKILL.md to evolve |
+| `--target-skill-path PATH` | `~/.claude/skills/skillopt-sleep-learned/SKILL.md` | Explicit SKILL.md to evolve |
 | `--tasks-file PATH` | — | Reviewed TaskRecord JSON (skip harvest) |
 | `--progress` | off | Print phase progress to stderr |
 | `--auto-adopt` | off | Auto-adopt if gate passes |
 | `--edit-budget N` | 4 | Max bounded edits per night |
 | `--preferences TEXT` | empty | Add house rules to the optimizer's reflection prior |
 | `--json` | off | Machine-readable JSON output |
 The CLI also has source/runtime path overrides (`--claude-home`, `--codex-home`,
 and `--codex-path`) and action-specific flags. Use
 `python -m skillopt_sleep <action> --help` as the authoritative surface.
 ## Config keys (`~/.skillopt-sleep/config.json`)
 Beyond the CLI flags, advanced behavior is controlled via config:
@@ -99,28 +119,37 @@ The sleep cycle can consolidate both:
 - **SKILL.md** — the managed skill file (bounded edits: add/delete/replace)
 - **CLAUDE.md** — the project memory (same bounded edits)
-Both are gated by the same held-out validation score. Set `evolve_memory: false` to consolidate only skills, or `evolve_skill: false` for only memory.
+With the default gate enabled, both are evaluated by the same held-out score.
 Set `evolve_memory: false` to consolidate only skills, or `evolve_skill: false`
 for only memory.
 ## Hard rules
 - **Never** hand-edit the user's `CLAUDE.md` / `SKILL.md` as part of this skill.
-  Only the `adopt` action changes live files, and it backs them up first.
+  Let the engine's explicit `adopt` or user-requested `--auto-adopt` path apply
  the staging manifest and back up existing live files first.
 - Harvest is read-only. `mock` replay has no side effects.
 - Real backends send truncated transcript excerpts and derived tasks to the
  selected provider for mining, replay, judging, and reflection. The Claude
  transcript path is not guaranteed to remove every secret before those calls.
  Review provider policy and session contents first. For sensitive data, use
  `mock` or run `harvest --output <file>`, inspect/redact the JSON, set
  `"reviewed": true`, and replay it with `--tasks-file`; real backends refuse an
  unreviewed task file.
 - Always show the user the **held-out baseline → candidate** score and the
  exact proposed edits before suggesting adoption. Evidence before adoption.
- If asked whether it really helps, run
+- If asked to demonstrate the mechanism without provider calls, run
  `python -m skillopt_sleep.experiments.run_experiment --persona researcher --json`
-  — a deterministic demo that proves held-out lift and that the gate blocks
+  — a deterministic synthetic demo of held-out lift and gate rejection. It
-  harmful edits.
+  validates the mechanism, not effectiveness on the user's own tasks.
 ## Validate / demo
 ```bash
-# deterministic proof (no API): held-out score rises, gate blocks regressions
+# deterministic synthetic demo (no API): score rises and the gate blocks a regression
 python -m skillopt_sleep.experiments.run_experiment --persona researcher --assert-improves
 python -m skillopt_sleep.experiments.run_experiment --persona programmer  --assert-improves
 ```
-See the SkillOpt-Sleep guide section for recorded output and
+See the [SkillOpt-Sleep documentation](https://github.com/microsoft/SkillOpt/tree/main/docs/sleep)
-`docs/superpowers/specs/2026-06-07-skillopt-sleep-claude-code-plugin-design.md`
+for recorded results, limitations, and the supported integration surface.
 for the full design.
@@ -9,7 +9,8 @@ as the Claude Code plugin (`skillopt_sleep`), wrapped for Codex.
 > [gbrain-evals](https://github.com/garrytan/gbrain-evals) `skillopt-v1`
 > benchmark, a deliberately deficient skill goes **0.00 → 1.00** on a held-out
 > set with the Codex backend (incl. the tool-use seed via a real tool loop).
-> See [the SkillOpt-Sleep guide section](https://microsoft.github.io/SkillOpt/docs/guideline.html#sleep).
+> See the recorded results and limitations in
 > [`docs/sleep/RESULTS.md`](../../docs/sleep/RESULTS.md).
 ## What Codex supports (and what we use)
@@ -22,8 +23,8 @@ rules. The shared runner remains a plain shell entrypoint that the skill calls.
 ## Install
 ```bash
-git clone <repo-url> SkillOpt-Sleep
+git clone https://github.com/microsoft/SkillOpt.git
-cd SkillOpt-Sleep
+cd SkillOpt
 bash plugins/codex/install.sh          # installs the skill
 export SKILLOPT_SLEEP_REPO="$(pwd)"    # so the runner is found from anywhere
 ```
@@ -60,15 +61,19 @@ python -m skillopt_sleep run --project "$(pwd)" --source codex --backend codex \
 `~/.codex/archived_sessions`. Use `--codex-home /path/to/.codex` to point at a
 different Codex home, or `--source auto` to try Codex archives first and fall
 back to Claude Code transcripts. Default backend is `mock` (no API spend).
-`--backend codex` uses your Codex budget for real improvement. Bound live runs
+`--backend codex` uses your Codex budget for model-driven optimization; an
 accepted gain is task-dependent, not guaranteed. Bound live runs
 with `--max-sessions` and `--max-tasks`; add `--progress` because Codex-backed
 mining, replay, and reflection can be slow and otherwise quiet. Use
 `--target-skill-path` to stage/adopt into a repo-scoped Codex skill such as
 `.agents/skills/<name>/SKILL.md`; target runs over-sample mined tasks and
-prefer tasks that match the target skill's path, headings, and content. All the
+prefer tasks that match the target skill's path, headings, and content. The
-controllable knobs (`--gate on|off`, `--rollouts-k`, `--budget-tokens`,
+implemented main-CLI flags work the same across the shared integrations, and
-`--preferences`, optimizer/target split) work identically — see
+`--preferences "..."` is available for house rules. Advanced keys such as
-[the SkillOpt-Sleep guide section](https://microsoft.github.io/SkillOpt/docs/guideline.html#sleep).
+`gate_mode`, `dream_rollouts`, and `recall_k` belong in the Sleep config; the
 nightly CLI does not expose `--gate`, `--rollouts-k`, token/time-budget, or
 optimizer/target-split flags. See the
 [shared CLI reference](../README.md#supported-cli-surface).
 For privacy-sensitive projects, split the run into reviewable steps:
@@ -86,6 +91,11 @@ Inspect/redact the JSON and set `"reviewed": true` before using a real backend.
 `--tasks-file` skips archive harvest/mining and replays only the reviewed JSON
 tasks; real backends refuse task files still marked `"reviewed": false`.
 This review step matters even though the Codex transcript converter removes
 known secret-shaped strings: pattern-based redaction is not a guarantee. A real
 backend sends truncated transcript/task content to the selected provider, while
 `--backend mock` makes no provider calls.
 ## Notes / status
 - Codex's `exec` runs shell, so the real-tool-loop replay (e.g. the
@@ -1,16 +1,23 @@
 ---
 name: skillopt-sleep
-description: "Use when the user wants Codex to self-improve from past usage, asks about a nightly/offline 'sleep' or 'dream' cycle, wants Codex to review past sessions, learn preferences, consolidate memory/skills, run dry-run/run/adopt/status for SkillOpt-Sleep, or schedule offline self-optimization. Drives the skillopt_sleep engine: harvest past sessions -> mine recurring tasks -> replay offline -> consolidate validated memory + skills behind a held-out gate."
+description: "Use when the user wants Codex to self-improve from past usage, asks about a nightly/offline 'sleep' or 'dream' cycle, wants Codex to review past sessions, learn preferences, consolidate memory/skills, run dry-run/run/adopt/status for SkillOpt-Sleep, or schedule background self-optimization. Drives the skillopt_sleep engine: harvest past sessions -> mine recurring tasks -> replay through a selected backend -> consolidate validated memory + skills behind a held-out gate."
 ---
-# SkillOpt-Sleep: offline self-evolution for a local Codex agent
+# SkillOpt-Sleep: usage-driven self-evolution for a local Codex agent
-SkillOpt-Sleep gives the user's Codex agent a sleep cycle. While the user is
+SkillOpt-Sleep gives the user's Codex agent a sleep cycle. On demand or on a
-offline or on demand, it reviews past local sessions, re-runs recurring tasks
+nightly schedule, it reviews past local sessions, re-runs recurring tasks
-on the user's own budget, and consolidates what it learns into memory and
+through the selected backend, and proposes changes to a configured skill and to
-skills. It keeps only changes that pass a held-out validation gate, and live
+the project's `CLAUDE.md`. With the default validation gate enabled, it keeps
-files change only after the user explicitly adopts a staged proposal. There is
+only changes that improve a held-out score. Live files change only through
-no model-weight training.
+explicit adoption or a user-requested `--auto-adopt`. There is no model-weight
 training.
 The current shared engine does **not** write `AGENTS.md`. For a Codex-visible
 result, always select a Codex skill explicitly with `--target-skill-path` (for
 example `.agents/skills/<name>/SKILL.md`). If project `CLAUDE.md` is not a
 desired secondary target, set `"evolve_memory": false` in
 `~/.skillopt-sleep/config.json` before running.
 ## When to use
@@ -28,13 +35,15 @@ Trigger when the user wants any of:
   configuration and normalize them into session digests.
 2. **Mine** - turn digests into recurring `TaskRecord`s with outcomes and
   checkable references where possible.
-3. **Replay** - re-run mined tasks offline under the current skill and memory.
+3. **Replay** - re-run mined tasks through the selected backend under the
   current skill and memory.
 4. **Consolidate** - reflect on failures and propose bounded edits.
-5. **Gate** - accept edits only when the held-out validation score improves.
+5. **Gate** - with the default gate enabled, accept edits only when the held-out
   validation score improves.
 6. **Stage** - write the proposal under
   `<project>/.skillopt-sleep/staging/<date>/`; nothing live changes.
-7. **Adopt** - only after explicit user approval, copy staged files over live
+7. **Adopt** - explicitly, or through user-requested auto-adopt, copy staged
-   files with backups.
+   files over live files with backups for existing targets.
 ## How to drive it
@@ -43,32 +52,47 @@ finds the engine and a Python >= 3.10 automatically.
 ```bash
 # point at the repo if it isn't auto-detected from CWD:
-export SKILLOPT_SLEEP_REPO=/path/to/SkillOpt-Sleep
+export SKILLOPT_SLEEP_REPO=/path/to/SkillOpt
 TARGET_SKILL=.agents/skills/example/SKILL.md
 bash "$SKILLOPT_SLEEP_REPO/plugins/run-sleep.sh" status --project "$(pwd)"
-bash "$SKILLOPT_SLEEP_REPO/plugins/run-sleep.sh" harvest --project "$(pwd)"
+bash "$SKILLOPT_SLEEP_REPO/plugins/run-sleep.sh" harvest --project "$(pwd)" \
-bash "$SKILLOPT_SLEEP_REPO/plugins/run-sleep.sh" dry-run --project "$(pwd)" --backend mock
+  --source codex --target-skill-path "$TARGET_SKILL"
-bash "$SKILLOPT_SLEEP_REPO/plugins/run-sleep.sh" run --project "$(pwd)" --backend codex
+bash "$SKILLOPT_SLEEP_REPO/plugins/run-sleep.sh" dry-run --project "$(pwd)" \
-bash "$SKILLOPT_SLEEP_REPO/plugins/run-sleep.sh" run --project "$(pwd)" --source codex  # harvest from Codex Desktop
+  --source codex --target-skill-path "$TARGET_SKILL" --backend mock
 bash "$SKILLOPT_SLEEP_REPO/plugins/run-sleep.sh" run --project "$(pwd)" \
  --source codex --target-skill-path "$TARGET_SKILL" --backend codex \
  --max-sessions 5 --max-tasks 3 --progress
 bash "$SKILLOPT_SLEEP_REPO/plugins/run-sleep.sh" adopt --project "$(pwd)"
 ```
 Actions are `status`, `harvest`, `dry-run`, `run`, `adopt`, `schedule`, and `unschedule`.
 - Default backend is `mock`, which is deterministic and spends no API budget.
- `--backend codex` uses the user's Codex budget for real improvement.
+- `--backend codex` uses the user's Codex budget for model-driven optimization.
  An accepted held-out gain is run-specific evidence, not a guarantee of
  broader improvement; results depend on the tasks, model, and checks.
 - `--source codex` reads Codex Desktop archived sessions from `~/.codex/archived_sessions`;
  use `--codex-home /path/to/.codex` if the archive lives elsewhere.
 - `--target-skill-path` is required for a Codex skill target. Without it, the
  shared default is a Claude-managed skill under `~/.claude/skills/`, not an
  `.agents` skill.
 - Keep `dry-run --backend mock` as the first smoke check unless the user
  explicitly asked for a real optimization run.
 ### Scheduling
 ```bash
-bash "$SKILLOPT_SLEEP_REPO/plugins/run-sleep.sh" schedule --project "$(pwd)" --hour 3 --minute 17
+bash "$SKILLOPT_SLEEP_REPO/plugins/run-sleep.sh" schedule --project "$(pwd)" \
  --backend codex --hour 3 --minute 17
 bash "$SKILLOPT_SLEEP_REPO/plugins/run-sleep.sh" unschedule --project "$(pwd)"
 ```
-Installs a nightly cron entry. `unschedule --all` removes every managed entry.
+The scheduler persists the project, backend, time, and optional auto-adopt flag;
 it does not persist `--source` or `--target-skill-path` from this command. Before
 scheduling a Codex-targeted run, set `"transcript_source": "codex"` and an
 absolute `"target_skill_path"` in `~/.skillopt-sleep/config.json`. On systems
 without `crontab`, `schedule` prints a line for manual installation.
 `unschedule --all` removes every managed entry.
 ### All backends
@@ -76,6 +100,8 @@ Installs a nightly cron entry. `unschedule --all` removes every managed entry.
 - `--backend claude` — uses the Claude CLI
 - `--backend codex` — uses the Codex CLI
 - `--backend copilot` — uses the GitHub Copilot CLI
 - `--backend handoff` — emits prompt/answer files for an interactive session
 - `--backend azure_openai` — uses the configured Azure OpenAI endpoint
 ### Additional flags
@@ -96,7 +122,10 @@ Installs a nightly cron entry. `unschedule --all` removes every managed entry.
 ### Memory consolidation
-The sleep cycle consolidates both **memory** (AGENTS.md / CLAUDE.md) and **skills** (SKILL.md) by default. Each is independently toggleable via `evolve_memory` / `evolve_skill` config keys. Both are gated by the same held-out validation score.
+The shared sleep cycle consolidates project **memory** (`CLAUDE.md`) and the
 selected **skill** (`SKILL.md`) by default. It does not update `AGENTS.md`.
 Each target is independently toggleable through `evolve_memory` /
 `evolve_skill`, and both are gated by the same held-out validation score.
 ## Steps
@@ -104,16 +133,23 @@ The sleep cycle consolidates both **memory** (AGENTS.md / CLAUDE.md) and **skill
 2. For `dry-run` and `run`, report the held-out baseline -> candidate score,
   gate action, task count, session count, and exact proposed edits.
 3. If a staging directory is printed, read `report.md` before summarizing.
-4. `run` only stages a proposal; nothing live changes until `adopt`.
+4. `run` stages by default; if `--auto-adopt` was explicitly supplied, report
-5. Offer adoption only after the user has reviewed the staged proposal.
+   the paths it updated instead of claiming nothing changed.
-6. Never hand-edit the user's `AGENTS.md`, memory, or skills as a substitute
+5. Offer adoption only after the user has reviewed a still-staged proposal.
-   for `adopt`; adoption is the safety boundary and writes backups first.
+6. Never hand-edit the configured `CLAUDE.md` or target skill as a substitute
   for the engine's adopt path; adoption is the safety boundary and backs up
   existing targets first.
 ## Hard rules
 - Harvest is read-only. Do not edit archived sessions or raw transcripts.
- Keep raw secrets, credentials, private user data, and unsanitized transcript
+- Codex transcript harvesting removes known secret-shaped strings, developer
-  contents out of messages, logs, generated artifacts, and commits.
+  instructions, and raw tool payloads, but pattern-based redaction is not a
  guarantee. A real backend still sends truncated transcript/task content to
  its provider. Review sensitive sessions and provider policy first; prefer a
  reviewed `--tasks-file` workflow when the data boundary matters.
 - Keep raw secrets, credentials, private user data, and transcript contents out
  of messages, logs, generated artifacts, and commits.
 - Show validation evidence before recommending adoption.
 - Treat generated edits as proposals, not as source of truth.
 - Do not rely on deprecated custom prompts or `/sleep` slash commands for this
@@ -122,11 +158,16 @@ The sleep cycle consolidates both **memory** (AGENTS.md / CLAUDE.md) and **skill
 ## Validate
 ```bash
-python -m skillopt_sleep dry-run --project "$(pwd)" --backend mock --json
+python -m skillopt_sleep dry-run --project "$(pwd)" --source codex \
  --target-skill-path .agents/skills/example/SKILL.md --backend mock --json
 python -m skillopt_sleep.experiments.run_gbrain --backend codex \
  --seeds brief-writer --data-root /path/to/gbrain-evals/eval/data/skillopt-v1 \
  --nights 2 --limit-replay 3 --limit-holdout 3
 ```
-A deficient skill goes 0.00 -> 1.00 on a held-out set; the optimizer's edits
+In the recorded `brief-writer` gbrain run, the deliberately deficient fixture
-are gated on real-task performance.
+went 0.00 -> 1.00 on that run's held-out set. Treat this as reproducible
 benchmark evidence for that configuration, not a guarantee for other skills,
 tasks, or models; see the
 [recorded results](https://github.com/microsoft/SkillOpt/blob/main/docs/sleep/RESULTS.md)
 for context and limitations.
@@ -27,8 +27,8 @@ Requires Python ≥ 3.10. No third-party packages — the server is pure stdlib.
     "mcpServers": {
       "skillopt-sleep": {
         "command": "python3",
-         "args": ["/abs/path/SkillOpt-Sleep/plugins/copilot/mcp_server.py"],
+        "args": ["/abs/path/SkillOpt/plugins/copilot/mcp_server.py"],
-         "env": { "SKILLOPT_SLEEP_REPO": "/abs/path/SkillOpt-Sleep" }
+        "env": { "SKILLOPT_SLEEP_REPO": "/abs/path/SkillOpt" }
       }
     }
   }
@@ -42,14 +42,21 @@ Requires Python ≥ 3.10. No third-party packages — the server is pure stdlib.
 ## Use
 Ask Copilot things like *"run the sleep cycle"*, *"what did the last sleep
-propose?"*, *"adopt the staged sleep proposal"*. Copilot calls the MCP tools:
+propose?"*, *"adopt the staged sleep proposal"*. The server exposes seven MCP
-`sleep_status`, `sleep_dry_run`, `sleep_run`, `sleep_adopt`, `sleep_harvest`.
+tools: `sleep_status`, `sleep_dry_run`, `sleep_run`, `sleep_adopt`,
 `sleep_harvest`, `sleep_schedule`, and `sleep_unschedule`.
 Each tool takes optional `project`, `backend` (`mock`/`claude`/`codex`/`copilot`), and
 `scope` arguments. Default backend is `mock` (no API spend). The `copilot`
 backend drives the GitHub Copilot CLI (`copilot -p ... --output-format json`)
 and requires the `copilot` CLI to be installed and authenticated.
 Harvesting is local and read-only, and the default `mock` backend makes no
 provider calls. A real backend sends truncated transcript excerpts and derived
 tasks to the selected provider. Outbound prompts are not currently guaranteed
 to be secret-free; review sensitive data and provider policy first. See the
 [shared data-boundary guidance](../README.md#data-boundary).
 For speed, the `copilot` backend runs each call against an isolated
 `COPILOT_HOME` with built-in MCP servers and custom instructions disabled, so
 your user MCP servers (including this project's own) are not spawned per call
@@ -65,12 +72,13 @@ printf '%s\n' \
  '{"jsonrpc":"2.0","id":2,"method":"tools/list"}' \
  | SKILLOPT_SLEEP_REPO="$(pwd)" python3 plugins/copilot/mcp_server.py
 ```
-You should see the server info and the five `sleep_*` tools.
+You should see the server info and all seven `sleep_*` tools.
 ## Notes / status
 - MCP is the stable, official Copilot extension surface, so this is the most
-  portable of the three integrations (one server → CLI + IDE).
+  portable shared-engine integration (one server → CLI + IDE).
- The engine and all its controls (gate on/off, multi-rollout, budget,
+- The MCP schema exposes the main CLI's implemented controls, including task and
-  preferences, optimizer/target split) are identical across platforms — see
+  session caps, target-skill selection, scheduling, and staged adoption. It does
-  [the SkillOpt-Sleep guide section](https://microsoft.github.io/SkillOpt/docs/guideline.html#sleep).
+  not add experiment-only gate, rollout, token/time-budget, or optimizer/target
  split flags. See the [shared CLI reference](../README.md#supported-cli-surface).
@@ -8,26 +8,29 @@ automatically as ambient guidance.)
 This project has SkillOpt-Sleep available via an MCP server (`skillopt-sleep`).
 It gives the agent a nightly "sleep cycle": it reviews past sessions, replays
-recurring tasks offline, and consolidates validated memory + skills behind a
+recurring tasks through a selected backend, and stages validation-gated changes
-held-out gate.
+to project `CLAUDE.md` and a configured `SKILL.md`.
 When the user asks to "run the sleep cycle", "review my past sessions", "learn
 my preferences", or "make the agent improve from past usage", use the MCP tools:
 - `sleep_status` — what's happened + the latest staged proposal
- `sleep_dry_run` — safe preview, stages nothing
+- `sleep_dry_run` — no-staging preview; a real backend still makes provider calls
- `sleep_run` — full cycle, stages a reviewed proposal (nothing live changes)
+- `sleep_run` — full cycle, stages a validation-gated proposal by default;
- `sleep_adopt` — apply the staged proposal (backs up first)
+  explicit `auto_adopt` may update live files
 - `sleep_adopt` — apply the staged proposal (backs up an existing live file first)
 - `sleep_harvest` — list mined recurring tasks
 - `sleep_schedule` — install a nightly cron entry (set `hour`/`minute`)
 - `sleep_unschedule` — remove the nightly cron entry
 ### Key parameters (pass as MCP tool arguments)
- `backend` — `mock` (default, free), `claude`, `codex`, or `copilot`
+- `backend` — `mock` (default, no provider calls), `claude`, `codex`, or `copilot`
 - `source` — `claude`, `codex`, or `auto` (where to read transcripts)
- `target_skill_path` — explicit SKILL.md to evolve
+- `target_skill_path` — explicit SKILL.md to evolve; use this for a skill that
- `tasks_file` — pre-built TaskRecord JSON (skip harvest)
+  the current agent actually loads
 - `tasks_file` — reviewed TaskRecord JSON (skip harvest); real backends require
  its metadata to contain `"reviewed": true`
 - `max_tasks` / `max_sessions` — cap workload
 - `auto_adopt` — auto-adopt if the gate passes
 - `json` — machine-readable output for programmatic use
@@ -40,4 +43,14 @@ my preferences", or "make the agent improve from past usage", use the MCP tools:
 Always show the user the held-out baseline → candidate score and the proposed
 edits before suggesting `sleep_adopt`. Never hand-edit the user's memory/skill
-files; only `sleep_adopt` does that, with a backup.
+files; use `sleep_adopt` (or an explicitly requested `auto_adopt`) so the engine
 applies its staging manifest and backup behavior.
 Harvesting is local and read-only, and `backend: "mock"` makes no provider
 calls. A real backend sends truncated transcript excerpts and derived tasks to
 the selected provider; outbound prompts are not guaranteed to be secret-free.
 Review sensitive data and provider policy before selecting a real backend.
 `sleep_schedule` persists only the project, backend, time, and optional
 auto-adopt setting. Put a non-default transcript source or target skill in
 `~/.skillopt-sleep/config.json` before scheduling it.
@@ -68,9 +68,18 @@ skill"*, or *"evaluate this skill on the dataset"*. Copilot calls the MCP tools:
 Common optional args (both train and eval): `env`, `backend`,
 `optimizer_model`, `target_model`, `out_root`, `cfg_options` (space-separated
-`KEY=VALUE` YAML overrides), and `extra_args` (raw passthrough flags for the
+`section.key=value` YAML overrides), and `extra_args` (raw passthrough flags
-underlying script). `skillopt_train` also accepts `num_epochs`, `batch_size`,
+for the underlying script). `skillopt_train` also accepts `num_epochs`,
-`seed`, and `use_gate`.
+`batch_size`, `seed`, and `use_gate`. `use_gate` defaults to `true`; setting it
 to `false` still records validation scores but force-accepts every candidate,
 which changes the optimization semantics.
 The MCP schema's `backend` argument follows the underlying script's
 `--backend` choices. Role-specific and generic OpenAI-compatible backends can
 be selected through config or `extra_args` (for example,
 `--optimizer_backend openai_compatible --target_backend openai_compatible`);
 see the repository's [backend guide](../../../docs/guide/new-backend.md) for
 the required environment variables.
 Runs can be very long. The server's subprocess timeout defaults to 6 hours;
 override it with the `SKILLOPT_RUN_TIMEOUT` environment variable (seconds).
@@ -23,9 +23,10 @@ Guidance:
 - Always run `skillopt_list_configs` first if you don't already know a valid `config` path.
 - `skillopt_train` and `skillopt_eval` are long-running and consume the user's
  model backend/budget — confirm the `config`, `backend`, and model choices
-  with the user before launching, and surface the held-out gate result when the
+  with the user before launching. Surface the held-out gate result for training,
-  run finishes.
+  or the evaluation score and output directory for eval-only runs.
- For one-off YAML overrides use `cfg_options` (e.g. `seed=123 batch_size=40`);
+- For one-off YAML overrides use dotted `cfg_options` for structured configs
  (e.g. `train.seed=123 train.batch_size=40`);
  for any other underlying flag use `extra_args`.
 This is distinct from the **SkillOpt-Sleep** MCP server (`skillopt-sleep`,
@@ -54,13 +54,25 @@ Requires Python ≥ 3.10. No third-party packages — the server is pure stdlib.
 | Tool | What it does |
 |---|---|
 | `sleep_status` | nights run so far + latest staged proposal |
-| `sleep_dry_run` | preview cycle — no staging, no changes |
+| `sleep_dry_run` | preview cycle — no staging; a real backend still makes provider calls |
 | `sleep_run` | full cycle; stages a proposal for review |
 | `sleep_adopt` | apply the staged proposal; syncs skill to the workspace |
 | `sleep_harvest` | debug: list the recurring tasks mined |
 | `sleep_schedule` | install a nightly cron entry (`--hour` / `--minute`) |
 | `sleep_unschedule` | remove the nightly cron entry |
-Default backend is `mock` (no API spend); `--backend claude|codex` uses your own
+Default backend is `mock` (no API spend); the `claude`, `codex`, and `copilot`
-budget. Same engine and `sleep_*` interface as the other plugins — all call
+backends use the corresponding authenticated CLI and budget. The seven tools
-`python -m skillopt_sleep`.
+call the same `python -m skillopt_sleep` actions as the other shared-engine
 integrations.
 ## Data boundary
 The Devin harvester reads local ATIF transcripts, agentmemory, and skill files
 and converts them into the engine's session format. The `mock` backend keeps
 that workflow local. A real backend sends truncated excerpts and derived tasks
 to the selected provider for mining, replay, judging, and reflection. The
 conversion step is not a guarantee that outbound prompts contain no secrets;
 review sensitive sources and provider policy before enabling a real backend.
 See the [shared data-boundary guidance](../README.md#data-boundary) and
 [implemented CLI reference](../README.md#supported-cli-surface).
@@ -3,17 +3,36 @@
 You have access to a nightly self-evolution cycle via the `skillopt-sleep` MCP
 server. Use these tools to improve your long-term skills over time:
- **`sleep_status`** — how many nights have run + the latest staged proposal
+- **`sleep_status`** — refresh the converted local cache, then show how many
- **`sleep_dry_run`** — preview a cycle without changing anything
+  nights have run and the latest staged proposal
- **`sleep_run`** — run a full cycle; stages a proposal for review
+- **`sleep_dry_run`** — refresh the converted local cache and preview a cycle
- **`sleep_adopt`** — apply the staged proposal to `.devin/skills/skillopt-sleep-learned/SKILL.md`
+  without engine staging/adoption; a real backend still makes provider calls
 - **`sleep_run`** — run a full cycle; stages a proposal by default, while an
  explicit `auto_adopt` may also update live files
 - **`sleep_adopt`** — apply the staged proposal, then sync the managed skill to
  `.devin/skills/skillopt-sleep-learned/SKILL.md` when `project` is the Devin
  workspace and that workspace already contains a `.devin/` directory
 - **`sleep_harvest`** — debug: list the recurring tasks mined from recent sessions
- **`sleep_schedule`** / **`sleep_unschedule`** — install/remove a nightly cron run
+- **`sleep_schedule`** / **`sleep_unschedule`** — low-level shared-engine cron
  controls; the current scheduled command does not run Devin's conversion step,
  so do not use it as an unattended Devin-harvest workflow
-When a user asks about the sleep cycle, skill evolution, or improving your
+When a user asks about the sleep cycle or skill evolution, prefer calling these
-long-term memory, prefer calling these tools over explaining the concept.
+tools over explaining the concept.
-Default backend is `mock` (no API spend). Pass `backend: "claude"` or
+Always pass the absolute Devin workspace as `project`, especially for
-`backend: "codex"` with your own API key for real LLM-driven optimization.
+`sleep_adopt`. Default backend is `mock` (no provider calls). The `claude`,
 `codex`, and `copilot` backend values use the corresponding installed and
 authenticated CLI; they do not require this plugin to implement a separate
 API-key flow.
 The Devin conversion and mock workflow stay local. A real backend sends
 truncated transcript excerpts and derived tasks to the selected provider for
 mining, replay, judging, and reflection; conversion is not a guarantee that
 outbound prompts contain no secrets. Review local sources and provider policy
 before selecting a real backend.
 For a reviewed task file, pass `tasks_file`; before using it with a real backend,
 inspect/redact it and ensure its metadata contains `"reviewed": true`.
 Place this file at `.devin/rules/skillopt-sleep.md` in your workspace.
@@ -1,112 +1,115 @@
-# OpenClaw Plugin for SkillOpt-Sleep
+# OpenClaw reference adaptation for SkillOpt-Sleep
-Thin shell for running [SkillOpt-Sleep](https://github.com/microsoft/SkillOpt) on [OpenClaw](https://github.com/openclaw/openclaw).
+This directory is a contributed reference for connecting
 [SkillOpt-Sleep](https://github.com/microsoft/SkillOpt) to
 [OpenClaw](https://github.com/openclaw/openclaw) with a custom DeepSeek/Ollama
 backend.
-## What it does
+> **Reference status.** This is not one of the shared, plug-and-play
 > `skillopt_sleep` wrappers. Several scripts and the sample config contain
 > environment-specific absolute paths and assumptions from the original setup,
 > and the contributed wrapper has unresolved Python 3.10 syntax and backend
 > factory-signature gaps. The current checkout is not directly runnable; treat
 > it as porting source material, not an installation.
-Adds a nightly "sleep cycle" to any OpenClaw agent. The cycle:
+## Included components
-1. **Harvests** recent session transcripts from `~/.openclaw/agents/<name>/sessions/*.jsonl`
+| File | Purpose |
-2. **Mines** recurring task patterns using the optimizer LLM
+|---|---|
-3. **Replays** each pattern with the current `SKILL.md` (baseline) and a candidate `SKILL.md` (with proposed edits)
+| `run_sleep.py` | custom cycle entry point |
-4. **Gates** the candidate against the held-out score (rejects regressions)
+| `skillopt_sleep_openclaw.py` | DeepSeek Chat Completions backend plus local Ollama embeddings |
-5. **Stages** the accepted proposal in `~/.skillopt-sleep/staging/<night>/`
+| `run_sleep_cron.sh` | category-oriented cron wrapper |
-6. Leaves adoption to the operator (Ethan)
+| `slash_sleep.py` | experimental `/sleep` command helper |
 | `config.json` | example engine configuration |
 | `SKILL.md` | OpenClaw skill manifest |
 | `tests/*.json` | example task sets for research, DevOps, and wiki workflows |
-Nothing live changes until you adopt. Every adopt backs up first.
+The adaptation imports the shared engine but registers its own backend and
 maintains its own wrapper behavior. Changes to the shared CLI documentation do
 not automatically make every option available through these custom scripts.
-## Install
+## Intended cycle
-The plugin is a thin wrapper around the engine at `~/.openclaw/workspace/SkillOpt/skillopt_sleep/`:
+```text
-
+harvest supported session data or load a task file
-```bash
+  → replay with the current skill
-# 1. Clone the engine (one-time)
+  → propose bounded edits
-cd ~/.openclaw/workspace
+  → validate the candidate on held-out tasks
-git clone https://github.com/microsoft/SkillOpt.git
+  → stage a proposal for operator review
 # 2. Install the OpenClaw skill (this folder)
 ln -s /path/to/openclaw ~/.openclaw/workspace/skills/skillopt-sleep
 # 3. Configure
 cp ~/.openclaw/workspace/skills/skillopt-sleep/config.json ~/.skillopt-sleep/config.json
 $EDITOR ~/.skillopt-sleep/config.json
 # Set backend = "openclaw-deepseek"
 # Set model = "deepseek-v4-pro" (or "deepseek-v4-flash" for budget)
 # 4. Set API key
 echo 'export DEEPSEEK_API_KEY="sk-..."' >> ~/.openclaw/.env
 # 5. Add the nightly cron
 (crontab -l 2>/dev/null; echo "0 3 * * * cd ~/.openclaw/workspace/skills/skillopt-sleep && bash run_sleep_cron.sh >> ~/.skillopt-sleep/nightly.log 2>&1") | crontab -
 ```
-## Use
+The intended safety boundary is manual adoption: review the generated report and
 staged files before changing a live skill.
-### Manual trigger
+## Adapt before use
-```bash
+1. Clone SkillOpt into a location you control:
 # Run one cycle now
 python3 ~/.openclaw/workspace/skills/skillopt-sleep/run_sleep.py
-# Dry run (report only)
+   ```bash
-python3 ~/.openclaw/workspace/skills/skillopt-sleep/run_sleep.py --dry-run
+   git clone https://github.com/microsoft/SkillOpt.git
   cd SkillOpt/plugins/openclaw
   ```
-# One category only
+2. Inspect and replace the sample absolute paths in `run_sleep.py`,
-python3 ~/.openclaw/workspace/skills/skillopt-sleep/run_sleep.py --tasks tests/research-cron-tasks.json
+   `slash_sleep.py`, `run_sleep_cron.sh`, and `config.json`. Confirm the engine
-```
+   checkout, OpenClaw workspace, state directory, skill directory, and task-file
   paths all point to isolated test locations.
-### Slash command
+3. Review `config.json`. In particular, do not assume that values such as
   `max_tokens_per_night` or `replay_mode` are enforced by this custom wrapper
   merely because they appear in the example config.
-```bash
+4. Supply credentials through your normal secret-management mechanism. Do not
-# In any OpenClaw session
+   commit a DeepSeek key or place it in a world-readable file.
 /sleep status
 /sleep run
 /sleep run research-cron
 /sleep dry-run
 /sleep adopt              # adopt most recent accepted proposal
 /sleep reject             # discard most recent
 /sleep cost
 ```
-## Architecture
+5. Resolve every known porting gap listed in [`SKILL.md`](SKILL.md), add
   isolated tests for your adapted backend, and verify that `--help` imports
   cleanly on Python 3.10+. Only then start with a dry run and one reviewed
   task file. The target command should be shaped like:
-```
+   ```bash
-plugins/openclaw/
+   cd /path/to/SkillOpt/plugins/openclaw
-├── README.md                       # this file
+   python3 run_sleep.py --config /path/to/reviewed-config.json \
-├── run_sleep_cron.sh               # wrapper for cron invocation
+     --tasks tests/research-cron-tasks.json --dry-run
-├── run_sleep.py                    # main entry point
+   ```
 ├── slash_sleep.py                  # /sleep command implementation
 ├── skillopt_sleep_openclaw.py      # DeepSeek + Ollama backend
 ├── config.json                     # engine config
 ├── SKILL.md                        # OpenClaw skill manifest
 └── tests/                          # held-out test sets
    ├── research-cron-tasks.json
    ├── devops-tasks.json
    └── wiki-tasks.json
 ```
-The OpenClaw shell is one engine (skillopt_sleep/) + one backend (DeepSeek/Ollama) + four thin wrappers (cron, slash, skill, tests).
+6. Inspect the report, paths, network destinations, and proposed edits before
   considering a non-dry run or scheduling.
-## Why this matters for OpenClaw
+## Data boundary
-OpenClaw currently has no built-in "self-evolving skills" mechanism. The community has:
+The custom `openclaw-deepseek` backend sends task, skill, response, rubric, and
 reflection content to the configured DeepSeek endpoint. Its embedding helper can
 send truncated text to the configured local Ollama service. Do not assume these
 outbound prompts have been fully redacted; inspect transcript/task inputs and the
 provider's retention policy before using real data.
- **Manual skills** — Ethan writes them
+Use HTTPS for a remote DeepSeek-compatible endpoint. Keep any plaintext Ollama
- **LLM-generated skills** — one-shot, no validation
+endpoint on a trusted loopback interface. For a network-free engine smoke test,
- **Self-revision** — unbounded, no quality bar
+use the shared SkillOpt-Sleep CLI with `--backend mock` rather than assuming this
 custom wrapper is isolated.
-SkillOpt-Sleep adds a 4th option: **validated self-evolution**. The skill is the training target, the engine is the optimizer, the gate is the quality bar, the operator is the human-in-the-loop.
+## Scheduling
-## Validation
+`run_sleep_cron.sh` and the scheduling helpers are examples, not portable
 installers. Adapt their paths, create log directories, verify their environment,
 and run the exact command manually before adding a cron entry. Scheduled runs
 must preserve the same manual-adoption and credential boundaries as interactive
 runs.
-Validated on the public [gbrain-evals](https://github.com/garrytan/gbrain-evals) `skillopt-v1` benchmark with real Claude and Codex (deficient skills 0.00 → 1.00 on held-out, all 4 seeds).
+## Validation scope
-End-to-end test on our own 14-task held-out set: pipeline runs, gate correctly rejects non-improvements, staging artifacts land in `~/.skillopt-sleep/staging/<night>/`.
+The bundled JSON files are example held-out task sets, not a universal OpenClaw
 benchmark. Provider cost and quality depend on the selected model, task content,
 number of calls, and pricing at run time; this reference does not promise a fixed
 nightly cost. Validate the adapted workflow in an isolated workspace before
 using it on live skills.
-## Cost
+For the supported shared-engine CLI and its current flags, see the
-
+[integration reference](../README.md#supported-cli-surface). For measured
-Measured: ~$0.02/night with `deepseek-v4-pro` at 12 tasks/night. ~$0.59/month, $7.18/year.
+SkillOpt-Sleep results and limitations, see
 [`docs/sleep/RESULTS.md`](../../docs/sleep/RESULTS.md).
 ## License
-MIT (same as SkillOpt core).
+MIT, consistent with SkillOpt core.
@@ -1,129 +1,103 @@
 ---
 name: skillopt-sleep
-description: Validate and refine agent skills through nightly sleep cycles with held-out gates. Wraps Microsoft's SkillOpt-Sleep engine for the OpenClaw/DeepSeek stack.
+description: Reference-only OpenClaw adaptation of SkillOpt-Sleep. Use it to study or port the contributed DeepSeek wrapper, not as a ready-to-run installation.
 ---
-# skillopt-sleep — OpenClaw Adaptation of Microsoft SkillOpt-Sleep
+# SkillOpt-Sleep OpenClaw reference adaptation
-A nightly self-improvement loop that reads our session transcripts, mines recurring workflow patterns, replays them with proposed skill edits, and gates the proposals against a held-out test set. Only improvements that beat baseline are staged for human adoption.
+This directory is a contributed **reference**, not a supported, plug-and-play
 OpenClaw integration. It illustrates one way to connect the shared
 `skillopt_sleep` cycle to a custom DeepSeek Chat Completions backend and a set of
 environment-specific task fixtures.
-## When To Use
+Do not run or schedule the files unchanged. Several scripts and the sample
 configuration preserve assumptions from the contributor's original machine,
 and parts of the wrapper have not yet been ported to the current shared-engine
 interfaces. Start with the directory's [README.md](README.md), which is the
 authoritative status and adaptation guide.
- After Hermes's Weekly Skill Review (or as its replacement)
+## What is included
 - When a skill is being used 10+ times/week and could be tighter
 - Before promoting a new skill from `skill-proposals/` to `skills/`
 - When a skill regresses in observed quality
-## What It Does (One Cycle)
+- `skillopt_sleep_openclaw.py` — a contributed DeepSeek backend prototype. It
  also contains an Ollama embedding helper, but that helper is not wired into
  the current shared sleep cycle.
 - `run_sleep.py` — a custom cycle wrapper with environment-specific paths and a
  backend-registration shim.
 - `slash_sleep.py` — an experimental command helper written for an older
  staging-manifest shape.
 - `run_sleep_cron.sh` — a machine-specific category runner, not a portable cron
  installer.
 - `config.json` — a sample configuration, not a set of guaranteed or enforced
  runtime limits.
 - `tests/*.json` — example task fixtures from one environment, not a universal
  OpenClaw benchmark.
-```
+## Known porting gaps
 harvest session transcripts  ->  mine recurring task patterns
                              ->  replay each pattern (current skill vs proposed)
                              ->  GATE: must improve held-out score
                              ->  stage proposal
                              ->  Ethan adopts (manual)
 ```
-Nothing live changes until Ethan adopts. Every adopt backs up first.
+Before treating this as an integration, a maintainer must at least:
-## Architecture
+1. Replace every absolute workspace, repository, state, skill, log, and task
   path with explicit user configuration.
 2. Update the custom backend factory to the current `get_backend` call contract,
   including the project directory, and update its backend methods and edit
   records to the current protocol.
 3. Replace the experimental adoption logic with the current staging manifest
   and `skillopt_sleep.staging.adopt` behavior. Current staging artifacts use
   `proposed_SKILL.md` / `proposed_CLAUDE.md`, `manifest.json`, and report files;
   they do not expose the old `manifest.proposed_skill` field.
 4. Decide how real OpenClaw transcripts are converted into a supported session
   format. Pointing `claude_home` at an arbitrary agent directory does not by
   itself make its files Claude Code-compatible JSONL.
 5. Build scheduling around the adapted wrapper. The shared scheduler launches
   the shared CLI; it does not automatically preserve this custom backend or
   its category task-file flow.
 6. Add isolated end-to-end tests for dry-run, accepted/rejected gates, staging,
   adoption and backup, credential failure, and scheduled execution.
-```
+Until those gaps are resolved, use the supported shared
-skills/skillopt-sleep/
+`python -m skillopt_sleep` CLI with `--backend mock` to test SkillOpt-Sleep itself,
-├── SKILL.md                          # this file
+and treat this directory only as source material for a future OpenClaw port.
 ├── config.json                       # engine config (backend, budgets, etc.)
 ├── run_sleep.py                      # entry point
 └── skillopt_sleep_openclaw.py        # DeepSeek/Ollama backend
 ```
-The engine itself is at `~/.openclaw/workspace/SkillOpt/skillopt_sleep/` (cloned from microsoft/SkillOpt).
+## Shared-engine features are not wrapper features
-## Usage
+At this revision the supported shared CLI backends are `mock`, `claude`,
 `codex`, `copilot`, `handoff`, and `azure_openai`; the
 [plugin integration reference](../README.md#supported-cli-surface) is the
 authoritative list. The shared engine can consolidate a selected skill and
 project `CLAUDE.md` memory (controlled by `evolve_skill` and `evolve_memory`),
 and its `schedule` / `unschedule` actions manage shared-engine cron entries.
 Those capabilities do **not** make the custom OpenClaw wrapper portable: the
 shared scheduler will not invoke the prototype backend or its category
 fixtures. Use the shared documentation for those features, not this reference
 SKILL.
-```bash
+## Data and credential boundary
 # Run one cycle with current config
 cd ~/.openclaw/workspace/skills/skillopt-sleep
 python3 run_sleep.py
-# Dry run (report only, no staging)
+The prototype DeepSeek backend sends task, skill, memory, response, rubric, and
-python3 run_sleep.py --dry-run
+reflection content to its configured Chat Completions endpoint. Its source also
 contains a helper that can send text to an Ollama service if a future port wires
 that helper into the cycle. Neither path should be assumed to remove every
 secret or private detail.
-# Use a pre-built task set (recommended for testing)
+Before any port is tested with real data:
 python3 run_sleep.py --tasks tests/research-cron-tasks.json
 ```
-## Scheduling
+- use isolated, synthetic or explicitly reviewed task files;
 - replace sample business names, personal references, URLs, and machine paths;
 - load credentials through the operator's secret-management mechanism;
 - verify TLS and retention policy for every remote endpoint; and
 - inspect all staged artifacts before adoption.
-```bash
+The bundled fixtures are examples only. Their scores and any old cost estimates
-python3 slash_sleep.py schedule --hour 3 --minute 17
+do not establish effectiveness, safety, or a stable nightly price for another
-python3 slash_sleep.py unschedule
+OpenClaw deployment.
 python3 slash_sleep.py unschedule --all
 ```
-Installs a nightly cron entry using the shared SkillOpt-Sleep scheduler. This is an alternative to the external `run_sleep_cron.sh` script.
+## Further information
-## Alternative backends
+- [OpenClaw README](README.md) — current reference status and adaptation checklist
 - [plugin integration reference](../README.md) — supported shared-engine CLI
  surface and data boundary
 - [SkillOpt-Sleep documentation](../../docs/sleep/README.md) — concepts,
  results, and limitations
-While OpenClaw defaults to `openclaw-deepseek` (DeepSeek V4 Pro + Ollama), the shared engine also supports:
+Contributions that turn this reference into a portable integration should add
- `--backend mock` — deterministic, no API spend (for testing)
+tests and update all three documents together.
 - `--backend claude` — uses the Claude CLI
 - `--backend codex` — uses the Codex CLI
 - `--backend copilot` — uses the GitHub Copilot CLI
 These can be used via the engine directly (`python -m skillopt_sleep`).
 ## Shared-engine flags
 When invoking the engine directly, all standard flags are available:
 - `--source codex` / `--source auto` — harvest from Codex Desktop sessions
 - `--tasks-file PATH` — use a pre-built task set
 - `--target-skill-path PATH` — explicit SKILL.md target
 - `--max-tasks N` / `--max-sessions N` — cap workload
 - `--progress` — print phase progress
 - `--json` — machine-readable output
 - `--auto-adopt` — auto-adopt if gate passes
 Config keys: `preferences`, `gate_mode`, `gate_metric`, `dream_rollouts`, `recall_k`, `evolve_memory`, `evolve_skill`.
 ## Config (config.json)
 Key knobs:
 - `backend: "openclaw-deepseek"` — our custom backend
 - `model: "deepseek-v4-pro"` — optimizer model
 - `edit_budget: 3` — max bounded edits per night
 - `gate_mode: "on"` — validation-gated (rejects regressions)
 - `auto_adopt: false` — require Ethan to adopt manually
 - `max_tasks_per_night: 12` — cap to control cost
 ## Cost Estimate
 Per night: 12 tasks × (1 attempt + 1 judge + 1 reflect) × ~$0.005/1K tokens × ~3K tokens/call ≈ **$0.50-2.00/night**.
 ## Outputs
 - Report: `~/.skillopt-sleep/state.json` (running totals)
 - Staging: `~/.skillopt-sleep/staging/<night>/`
  - `report.md` — readable summary
  - `best_skill.md` — proposed skill
  - `edits.json` — bounded edit list
  - `before.md` / `after.md` — diffs
 ## Held-Out Test Sets (Phase 2)
 Located at `tests/<category>-tasks.json`. Each task has:
 - `prompt` — the recurring task
 - `reference` — exact-match gold answer
 - `rubric` — soft score rubric (0-1)
 - `domain` — research/devops/wiki/etc.
 Currently building for 3 categories:
 - research-cron-output
 - devops-infrastructure-check
 - wiki-canonical-guide
 ## When NOT To Use
 - For a one-off workflow (not a recurring pattern)
 - During a crisis/incident (humans must lead)
 - When session transcripts are < 24h old (not enough signal)
 - For skills < 300 tokens (over-optimization risk)
@@ -1778,6 +1778,7 @@
      <a href="#evolution">Evolution</a>
      <a href="#transfer">Transfer</a>
      <a href="#citation">Citation</a>
      <a href="https://github.com/microsoft/SkillOpt/blob/main/docs/index.md" target="_blank" rel="noopener">Docs</a>
      <a href="https://github.com/microsoft/SkillOpt" target="_blank" rel="noopener">Code</a>
    </nav>
  </header>
@@ -1915,7 +1916,7 @@
          <h3>A skill is external state for an agent.</h3>
          <p>
            Instead of fine-tuning a model or hand-maintaining prompts, SkillOpt runs
-            the frozen agent on scored batches, asks a separate optimizer model to
+            the frozen agent on scored batches, asks an optimizer model to
            propose structured edits, and accepts a candidate only when validation
            performance improves.
          </p>
@@ -2427,7 +2428,7 @@
    <footer class="footer">
      <span>SkillOpt: Executive Strategy for Self-Evolving Agent Skills</span>
-      <span><a href="https://github.com/microsoft/SkillOpt" target="_blank" rel="noopener">Code</a> / <a href="#citation">Citation</a></span>
+      <span><a href="https://github.com/microsoft/SkillOpt/blob/main/docs/index.md" target="_blank" rel="noopener">Docs</a> / <a href="https://github.com/microsoft/SkillOpt" target="_blank" rel="noopener">Code</a> / <a href="#citation">Citation</a></span>
    </footer>
  </main>
  <script>
@@ -28,13 +28,16 @@ This directory provides scaffold files for adding a new benchmark to SkillOpt.
   `TemplateBenchmarkLoader → YourBenchmarkLoader`)
   and fix the cross-import in `adapter.py`.
 3. **Implement the TODO blocks** inside `adapter.py:rollout` and the
-   `_normalize_item` helper in `dataloader.py`. (`reflect` is inherited from
+   `_normalize_item` helper in `dataloader.py`. In addition to returning
-   `EnvAdapter`; override it only for custom reflection logic.)
+   `id`/`hard`/`soft`, persist each non-empty trajectory at
-4. **Register** the adapter — add a `try / except ImportError` block in
+   `<out_dir>/predictions/<id>/conversation.json`; the inherited `reflect`
-   `scripts/train.py`'s `_register_builtins()` mapping the registry key
+   method reads those files. Override `reflect` only for custom reflection
-   to your `YourBenchmarkAdapter` class. There is no
+   logic.
-   `BENCHMARK_REGISTRY` dict in `skillopt/envs/__init__.py`; the live
+4. **Register** the adapter — add matching `try / except ImportError` blocks
-   registry is `_ENV_REGISTRY` in `scripts/train.py`.
+   to `_register_builtins()` in both `scripts/train.py` and
   `scripts/eval_only.py`, mapping the registry key to your
   `YourBenchmarkAdapter` class. There is no `BENCHMARK_REGISTRY` dict in
   `skillopt/envs/__init__.py`; each CLI keeps its own lazy `_ENV_REGISTRY`.
 5. **Create the config** at `configs/your_benchmark/default.yaml`
   (start from `config_template.yaml`). `_base_` is a **string path**,
   not a list.